2024-01-27T02:19:11.977 INFO:root:teuthology version: 0.0.1.dev251+gc856e0f 2024-01-27T02:19:11.978 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --name yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi --archive /home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533593 --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} -- /home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533593/orig.config.yaml 2024-01-27T02:19:12.025 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-27T02:19:12.104 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533593 branch: wip-yuri4-testing-2024-01-25-1331-quincy description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} email: yweinste@redhat.com first_in_suite: false job_id: '7533593' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-yuri4-testing-2024-01-25-1331-quincy ceph: cephfs: ec_profile: - disabled conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 client.0: debug client: 20 debug objectcacher: 20 debug objecter: 20 client.1: debug client: 20 debug objectcacher: 20 debug objecter: 20 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon osd backfillfull ratio: 0.6 mon osd full ratio: 0.7 mon osd nearfull ratio: 0.6 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 memstore device bytes: 200000000 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 1.0 osd mon report interval: 5 osd objectstore: memstore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL - PG_RECOVERY_FULL - PG_DEGRADED sha1: 7495007658df82356749387b98425d4a7fcef9b6 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 7495007658df82356749387b98425d4a7fcef9b6 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri4-testing-2024-01-25-1331-quincy sha1: 7495007658df82356749387b98425d4a7fcef9b6 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1285 sha1: 7495007658df82356749387b98425d4a7fcef9b6 sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: wip-yuri4-testing-2024-01-25-1331-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_7495007658df82356749387b98425d4a7fcef9b6/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 7495007658df82356749387b98425d4a7fcef9b6 targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKinMYP627ZsZXwAlG5DwvLGSjUmKPgeWgJdoetEIMrsmJ9/ttaaTGA8euU9fQXMA1AiUIRCyYETVGpzaMcdBW4= smithi188.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDaXqOH8orBjhRAqKw+EP2xxh7LgX535tX97+GEXhxwOtfxlNkcktqRwp6fCNryFWaHXhZcGtAIVYedLyqQvsOM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_full teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: c856e0f9244dd29a50697123b480d63f16a4ed81 timestamp: 2024-01-26_01:07:29 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-01-27T02:19:12.105 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_7495007658df82356749387b98425d4a7fcef9b6/qa; will attempt to use it 2024-01-27T02:19:12.105 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_7495007658df82356749387b98425d4a7fcef9b6/qa/tasks 2024-01-27T02:19:12.106 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-01-27T02:19:12.107 INFO:teuthology.task.internal:Checking packages... 2024-01-27T02:19:12.131 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '7495007658df82356749387b98425d4a7fcef9b6' 2024-01-27T02:19:12.131 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-01-27T02:19:12.131 INFO:teuthology.packaging:ref: None 2024-01-27T02:19:12.131 INFO:teuthology.packaging:tag: None 2024-01-27T02:19:12.131 INFO:teuthology.packaging:branch: wip-yuri4-testing-2024-01-25-1331-quincy 2024-01-27T02:19:12.131 INFO:teuthology.packaging:sha1: 7495007658df82356749387b98425d4a7fcef9b6 2024-01-27T02:19:12.132 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri4-testing-2024-01-25-1331-quincy 2024-01-27T02:19:12.359 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-472.g74950076 2024-01-27T02:19:12.361 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-01-27T02:19:12.367 INFO:teuthology.task.internal:no buildpackages task found 2024-01-27T02:19:12.367 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-01-27T02:19:12.391 INFO:teuthology.task.internal:Saving configuration 2024-01-27T02:19:12.407 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-01-27T02:19:12.414 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-27T02:19:12.437 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533593', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-27 02:14:06.922275', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKinMYP627ZsZXwAlG5DwvLGSjUmKPgeWgJdoetEIMrsmJ9/ttaaTGA8euU9fQXMA1AiUIRCyYETVGpzaMcdBW4='} 2024-01-27T02:19:12.460 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533593', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-27 02:14:06.920906', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDaXqOH8orBjhRAqKw+EP2xxh7LgX535tX97+GEXhxwOtfxlNkcktqRwp6fCNryFWaHXhZcGtAIVYedLyqQvsOM='} 2024-01-27T02:19:12.461 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-01-27T02:19:12.467 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-01-27T02:19:12.467 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-01-27T02:19:12.467 INFO:teuthology.run_tasks:Running task console_log... 2024-01-27T02:19:12.532 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5726846160>, signals=[15]) 2024-01-27T02:19:12.532 INFO:teuthology.run_tasks:Running task internal.connect... 2024-01-27T02:19:12.539 INFO:teuthology.task.internal:Opening connections... 2024-01-27T02:19:12.539 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:19:12.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T02:19:12.617 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:19:12.618 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T02:19:12.692 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-01-27T02:19:12.700 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-01-27T02:19:12.722 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-01-27T02:19:12.722 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-01-27T02:19:12.783 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="Red Hat Enterprise Linux" 2024-01-27T02:19:12.783 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="8.6 (Ootpa)" 2024-01-27T02:19:12.783 INFO:teuthology.orchestra.run.smithi007.stdout:ID="rhel" 2024-01-27T02:19:12.783 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="fedora" 2024-01-27T02:19:12.783 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="8.6" 2024-01-27T02:19:12.783 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el8" 2024-01-27T02:19:12.783 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-27T02:19:12.783 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-01-27T02:19:12.783 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-27T02:19:12.784 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://www.redhat.com/" 2024-01-27T02:19:12.784 INFO:teuthology.orchestra.run.smithi007.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-27T02:19:12.784 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-27T02:19:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:19:12.784 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-27T02:19:12.784 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-27T02:19:12.784 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-27T02:19:12.784 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-27T02:19:12.785 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-01-27T02:19:12.803 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2024-01-27T02:19:12.823 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2024-01-27T02:19:12.824 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2024-01-27T02:19:12.883 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="Red Hat Enterprise Linux" 2024-01-27T02:19:12.883 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="8.6 (Ootpa)" 2024-01-27T02:19:12.883 INFO:teuthology.orchestra.run.smithi188.stdout:ID="rhel" 2024-01-27T02:19:12.883 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="fedora" 2024-01-27T02:19:12.883 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="8.6" 2024-01-27T02:19:12.884 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el8" 2024-01-27T02:19:12.884 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-27T02:19:12.884 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2024-01-27T02:19:12.884 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-27T02:19:12.884 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://www.redhat.com/" 2024-01-27T02:19:12.884 INFO:teuthology.orchestra.run.smithi188.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-27T02:19:12.884 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-27T02:19:12.884 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:19:12.884 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-27T02:19:12.884 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-27T02:19:12.884 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-27T02:19:12.884 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-27T02:19:12.885 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2024-01-27T02:19:12.903 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-01-27T02:19:12.912 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-01-27T02:19:12.918 INFO:teuthology.task.internal:Checking for old test directory... 2024-01-27T02:19:12.918 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2024-01-27T02:19:12.920 DEBUG:teuthology.orchestra.run.smithi188:> test '!' -e /home/ubuntu/cephtest 2024-01-27T02:19:12.942 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-01-27T02:19:12.954 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-01-27T02:19:12.954 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2024-01-27T02:19:12.981 DEBUG:teuthology.orchestra.run.smithi188:> test -z $(ls -A /var/lib/ceph) 2024-01-27T02:19:13.010 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-01-27T02:19:13.066 INFO:teuthology.run_tasks:Running task kernel... 2024-01-27T02:19:13.081 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-01-27T02:19:13.082 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-01-27T02:19:13.082 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-01-27T02:19:13.082 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-27T02:19:13.083 DEBUG:teuthology.orchestra.run.smithi188:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-27T02:19:13.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-27T02:19:13.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-27T02:19:13.105 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-01-27T02:19:13.105 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2024-01-27T02:19:13.162 INFO:teuthology.orchestra.run.smithi188.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-01-27T02:19:13.163 INFO:teuthology.task.kernel:Running kernel on smithi188: 4.18.0-513.9.1.el8_9.x86_64 2024-01-27T02:19:13.163 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum install -y kernel 2024-01-27T02:19:13.166 INFO:teuthology.orchestra.run.smithi007.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-01-27T02:19:13.166 INFO:teuthology.task.kernel:Running kernel on smithi007: 4.18.0-513.9.1.el8_9.x86_64 2024-01-27T02:19:13.166 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum install -y kernel 2024-01-27T02:19:14.666 INFO:teuthology.orchestra.run.smithi188.stdout:Updating Subscription Management repositories. 2024-01-27T02:19:14.666 INFO:teuthology.orchestra.run.smithi188.stdout:Unable to read consumer identity 2024-01-27T02:19:14.786 INFO:teuthology.orchestra.run.smithi007.stdout:Updating Subscription Management repositories. 2024-01-27T02:19:14.786 INFO:teuthology.orchestra.run.smithi007.stdout:Unable to read consumer identity 2024-01-27T02:19:14.846 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:19:14.846 INFO:teuthology.orchestra.run.smithi188.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-27T02:19:14.847 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:19:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:19:15.068 INFO:teuthology.orchestra.run.smithi007.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-27T02:19:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:19:16.773 INFO:teuthology.orchestra.run.smithi188.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2024-01-27T02:19:16.898 INFO:teuthology.orchestra.run.smithi007.stdout:Copr repo for python3-asyncssh owned by ceph 17 kB/s | 3.5 kB 00:00 2024-01-27T02:19:17.665 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2024-01-27T02:19:17.816 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:01 2024-01-27T02:19:21.374 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 448 kB/s | 24 kB 00:00 2024-01-27T02:19:21.640 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 378 kB/s | 24 kB 00:00 2024-01-27T02:19:22.143 INFO:teuthology.orchestra.run.smithi188.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-01-27T02:19:22.143 INFO:teuthology.orchestra.run.smithi188.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-01-27T02:19:22.231 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-01-27T02:19:22.232 INFO:teuthology.orchestra.run.smithi188.stdout:Nothing to do. 2024-01-27T02:19:22.232 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-01-27T02:19:22.383 DEBUG:teuthology.orchestra.run.smithi188:> echo no | sudo yum reinstall kernel || true 2024-01-27T02:19:22.435 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-01-27T02:19:22.435 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-01-27T02:19:22.499 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-01-27T02:19:22.501 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-01-27T02:19:22.501 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-01-27T02:19:22.659 DEBUG:teuthology.orchestra.run.smithi007:> echo no | sudo yum reinstall kernel || true 2024-01-27T02:19:22.758 INFO:teuthology.orchestra.run.smithi188.stdout:Updating Subscription Management repositories. 2024-01-27T02:19:22.759 INFO:teuthology.orchestra.run.smithi188.stdout:Unable to read consumer identity 2024-01-27T02:19:22.965 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:19:22.965 INFO:teuthology.orchestra.run.smithi188.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-27T02:19:22.965 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:19:23.047 INFO:teuthology.orchestra.run.smithi007.stdout:Updating Subscription Management repositories. 2024-01-27T02:19:23.047 INFO:teuthology.orchestra.run.smithi007.stdout:Unable to read consumer identity 2024-01-27T02:19:23.275 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:00:02 ago on Sat 27 Jan 2024 02:19:21 AM UTC. 2024-01-27T02:19:23.284 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:19:23.284 INFO:teuthology.orchestra.run.smithi007.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-27T02:19:23.284 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:19:23.345 INFO:teuthology.orchestra.run.smithi188.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-01-27T02:19:23.353 INFO:teuthology.orchestra.run.smithi188.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-01-27T02:19:23.388 INFO:teuthology.orchestra.run.smithi188.stderr:Error: No packages marked for reinstall. 2024-01-27T02:19:23.436 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum reinstall -y kernel || true 2024-01-27T02:19:23.647 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:02 ago on Sat 27 Jan 2024 02:19:21 AM UTC. 2024-01-27T02:19:23.733 INFO:teuthology.orchestra.run.smithi007.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-01-27T02:19:23.733 INFO:teuthology.orchestra.run.smithi007.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-01-27T02:19:23.770 INFO:teuthology.orchestra.run.smithi188.stdout:Updating Subscription Management repositories. 2024-01-27T02:19:23.771 INFO:teuthology.orchestra.run.smithi188.stdout:Unable to read consumer identity 2024-01-27T02:19:23.797 INFO:teuthology.orchestra.run.smithi007.stderr:Error: No packages marked for reinstall. 2024-01-27T02:19:23.853 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum reinstall -y kernel || true 2024-01-27T02:19:24.017 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:19:24.018 INFO:teuthology.orchestra.run.smithi188.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-27T02:19:24.018 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:19:24.238 INFO:teuthology.orchestra.run.smithi007.stdout:Updating Subscription Management repositories. 2024-01-27T02:19:24.238 INFO:teuthology.orchestra.run.smithi007.stdout:Unable to read consumer identity 2024-01-27T02:19:24.330 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:00:03 ago on Sat 27 Jan 2024 02:19:21 AM UTC. 2024-01-27T02:19:24.393 INFO:teuthology.orchestra.run.smithi188.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-01-27T02:19:24.393 INFO:teuthology.orchestra.run.smithi188.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-01-27T02:19:24.421 INFO:teuthology.orchestra.run.smithi188.stderr:Error: No packages marked for reinstall. 2024-01-27T02:19:24.443 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:19:24.443 INFO:teuthology.orchestra.run.smithi007.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-27T02:19:24.443 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:19:24.473 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q kernel | sort -rV | head -n 1 2024-01-27T02:19:24.641 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-01-27T02:19:24.641 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-01-27T02:19:24.641 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-01-27T02:19:24.641 DEBUG:teuthology.parallel:result is None 2024-01-27T02:19:24.802 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:03 ago on Sat 27 Jan 2024 02:19:21 AM UTC. 2024-01-27T02:19:24.876 INFO:teuthology.orchestra.run.smithi007.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-01-27T02:19:24.876 INFO:teuthology.orchestra.run.smithi007.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-01-27T02:19:24.904 INFO:teuthology.orchestra.run.smithi007.stderr:Error: No packages marked for reinstall. 2024-01-27T02:19:24.964 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-01-27T02:19:25.139 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-01-27T02:19:25.140 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-01-27T02:19:25.140 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-01-27T02:19:25.140 DEBUG:teuthology.parallel:result is None 2024-01-27T02:19:25.140 INFO:teuthology.run_tasks:Running task internal.base... 2024-01-27T02:19:25.149 INFO:teuthology.task.internal:Creating test directory... 2024-01-27T02:19:25.149 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-27T02:19:25.151 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-27T02:19:25.169 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-01-27T02:19:25.176 INFO:teuthology.run_tasks:Running task internal.archive... 2024-01-27T02:19:25.182 INFO:teuthology.task.internal:Creating archive directory... 2024-01-27T02:19:25.183 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-27T02:19:25.210 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-27T02:19:25.247 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-01-27T02:19:25.255 INFO:teuthology.task.internal:Enabling coredump saving... 2024-01-27T02:19:25.255 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-27T02:19:25.274 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-27T02:19:25.308 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-27T02:19:25.322 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-27T02:19:25.331 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-27T02:19:25.343 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-27T02:19:25.344 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-01-27T02:19:25.352 INFO:teuthology.task.internal:Configuring sudo... 2024-01-27T02:19:25.352 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-27T02:19:25.375 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-27T02:19:25.411 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-01-27T02:19:25.420 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-01-27T02:19:25.420 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-27T02:19:25.452 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-27T02:19:25.471 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-27T02:19:25.561 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-27T02:19:25.629 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-27T02:19:25.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-27T02:19:25.726 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:19:25.726 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-27T02:19:25.800 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-27T02:19:25.842 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-27T02:19:25.901 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-27T02:19:25.987 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-27T02:19:26.029 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:19:26.030 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-27T02:19:26.097 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2024-01-27T02:19:26.099 DEBUG:teuthology.orchestra.run.smithi188:> sudo service rsyslog restart 2024-01-27T02:19:26.153 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-27T02:19:26.191 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-27T02:19:26.549 INFO:teuthology.run_tasks:Running task internal.timer... 2024-01-27T02:19:26.557 INFO:teuthology.task.internal:Starting timer... 2024-01-27T02:19:26.557 INFO:teuthology.run_tasks:Running task pcp... 2024-01-27T02:19:26.576 INFO:teuthology.run_tasks:Running task selinux... 2024-01-27T02:19:26.585 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-01-27T02:19:26.585 DEBUG:teuthology.orchestra.run.smithi007:> sudo service auditd rotate 2024-01-27T02:19:26.652 INFO:teuthology.orchestra.run.smithi007.stdout:Rotating logs: 2024-01-27T02:19:26.654 DEBUG:teuthology.orchestra.run.smithi188:> sudo service auditd rotate 2024-01-27T02:19:26.709 INFO:teuthology.orchestra.run.smithi188.stdout:Rotating logs: 2024-01-27T02:19:26.711 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-01-27T02:19:26.711 DEBUG:teuthology.orchestra.run.smithi007:> /usr/sbin/getenforce 2024-01-27T02:19:26.740 INFO:teuthology.orchestra.run.smithi007.stdout:Permissive 2024-01-27T02:19:26.740 DEBUG:teuthology.orchestra.run.smithi188:> /usr/sbin/getenforce 2024-01-27T02:19:26.762 INFO:teuthology.orchestra.run.smithi188.stdout:Permissive 2024-01-27T02:19:26.762 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi007.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi188.front.sepia.ceph.com': 'permissive'} 2024-01-27T02:19:26.762 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-27T02:19:26.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-27T02:19:26.813 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-27T02:19:26.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-27T02:19:26.840 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-01-27T02:19:26.840 DEBUG:teuthology.orchestra.run.smithi007:> sudo /usr/sbin/setenforce permissive 2024-01-27T02:19:26.883 DEBUG:teuthology.orchestra.run.smithi188:> sudo /usr/sbin/setenforce permissive 2024-01-27T02:19:26.910 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-01-27T02:19:26.919 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-01-27T02:19:26.937 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-01-27T02:19:26.938 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi188.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-01-27T02:28:50.707 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi188.front.sepia.ceph.com')] 2024-01-27T02:28:50.708 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-01-27T02:28:50.709 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T02:28:50.789 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-01-27T02:28:50.822 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-01-27T02:28:50.822 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-01-27T02:28:50.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T02:28:50.898 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-01-27T02:28:50.965 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-01-27T02:28:50.965 INFO:teuthology.run_tasks:Running task clock... 2024-01-27T02:28:50.976 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-01-27T02:28:50.976 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-27T02:28:50.976 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-27T02:28:50.979 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-27T02:28:50.979 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-27T02:28:51.016 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-27T02:28:51.034 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-27T02:28:51.037 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-27T02:28:51.051 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-27T02:28:51.111 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: ntpd: command not found 2024-01-27T02:28:51.126 INFO:teuthology.orchestra.run.smithi007.stdout:506 Cannot talk to daemon 2024-01-27T02:28:51.131 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: ntpd: command not found 2024-01-27T02:28:51.143 INFO:teuthology.orchestra.run.smithi188.stdout:506 Cannot talk to daemon 2024-01-27T02:28:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-27T02:28:51.158 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-27T02:28:51.161 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-27T02:28:51.172 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-27T02:28:51.249 INFO:teuthology.orchestra.run.smithi007.stderr:bash: ntpq: command not found 2024-01-27T02:28:51.250 INFO:teuthology.orchestra.run.smithi188.stderr:bash: ntpq: command not found 2024-01-27T02:28:51.254 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-27T02:28:51.254 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-01-27T02:28:51.254 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-27T02:28:51.254 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-27T02:28:51.254 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-27T02:28:51.254 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-27T02:28:51.255 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-27T02:28:51.256 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-01-27T02:28:51.256 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-27T02:28:51.256 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-27T02:28:51.256 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-27T02:28:51.256 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-27T02:28:51.256 INFO:teuthology.run_tasks:Running task install... 2024-01-27T02:28:51.265 DEBUG:teuthology.task.install:project ceph 2024-01-27T02:28:51.265 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7495007658df82356749387b98425d4a7fcef9b6'}} 2024-01-27T02:28:51.265 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '7495007658df82356749387b98425d4a7fcef9b6'} 2024-01-27T02:28:51.266 INFO:teuthology.task.install:Using flavor: default 2024-01-27T02:28:51.271 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-01-27T02:28:51.271 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-01-27T02:28:51.272 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7495007658df82356749387b98425d4a7fcef9b6', 'tag': None, 'wait_for_package': False} 2024-01-27T02:28:51.272 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7495007658df82356749387b98425d4a7fcef9b6 2024-01-27T02:28:51.274 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7495007658df82356749387b98425d4a7fcef9b6', 'tag': None, 'wait_for_package': False} 2024-01-27T02:28:51.274 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7495007658df82356749387b98425d4a7fcef9b6 2024-01-27T02:28:51.422 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-01-25-1331-quincy/7495007658df82356749387b98425d4a7fcef9b6/centos/8/flavors/default/ 2024-01-27T02:28:51.423 INFO:teuthology.task.install.rpm:Package version is 17.2.7-472.g74950076 2024-01-27T02:28:51.427 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-01-25-1331-quincy/7495007658df82356749387b98425d4a7fcef9b6/centos/8/flavors/default/ 2024-01-27T02:28:51.428 INFO:teuthology.task.install.rpm:Package version is 17.2.7-472.g74950076 2024-01-27T02:28:51.634 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-01-25-1331-quincy/7495007658df82356749387b98425d4a7fcef9b6/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-01-25-1331-quincy/7495007658df82356749387b98425d4a7fcef9b6/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-01-25-1331-quincy/7495007658df82356749387b98425d4a7fcef9b6/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-27T02:28:51.634 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:28:51.634 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-27T02:28:51.642 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-01-25-1331-quincy/7495007658df82356749387b98425d4a7fcef9b6/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-01-25-1331-quincy/7495007658df82356749387b98425d4a7fcef9b6/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri4-testing-2024-01-25-1331-quincy/7495007658df82356749387b98425d4a7fcef9b6/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-27T02:28:51.642 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:28:51.642 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-27T02:28:51.669 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2024-01-27T02:28:51.670 DEBUG:teuthology.orchestra.run.smithi188:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7495007658df82356749387b98425d4a7fcef9b6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-27T02:28:51.671 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2024-01-27T02:28:51.671 DEBUG:teuthology.orchestra.run.smithi007:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7495007658df82356749387b98425d4a7fcef9b6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-27T02:28:51.738 DEBUG:teuthology.orchestra.run.smithi188:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-27T02:28:51.748 DEBUG:teuthology.orchestra.run.smithi007:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-27T02:28:51.815 DEBUG:teuthology.orchestra.run.smithi188:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-27T02:28:51.837 DEBUG:teuthology.orchestra.run.smithi007:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-27T02:28:51.870 INFO:teuthology.orchestra.run.smithi007.stdout:check_obsoletes = 1 2024-01-27T02:28:51.872 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum clean all 2024-01-27T02:28:51.881 INFO:teuthology.orchestra.run.smithi188.stdout:check_obsoletes = 1 2024-01-27T02:28:51.883 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum clean all 2024-01-27T02:28:52.261 INFO:teuthology.orchestra.run.smithi188.stdout:Updating Subscription Management repositories. 2024-01-27T02:28:52.308 INFO:teuthology.orchestra.run.smithi007.stdout:Updating Subscription Management repositories. 2024-01-27T02:28:53.717 INFO:teuthology.orchestra.run.smithi188.stdout:57 files removed 2024-01-27T02:28:53.770 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2024-01-27T02:28:53.788 INFO:teuthology.orchestra.run.smithi007.stdout:57 files removed 2024-01-27T02:28:53.851 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2024-01-27T02:28:54.106 INFO:teuthology.orchestra.run.smithi188.stdout:Updating Subscription Management repositories. 2024-01-27T02:28:54.236 INFO:teuthology.orchestra.run.smithi007.stdout:Updating Subscription Management repositories. 2024-01-27T02:28:56.021 INFO:teuthology.orchestra.run.smithi188.stdout:ceph packages for x86_64 198 kB/s | 78 kB 00:00 2024-01-27T02:28:56.229 INFO:teuthology.orchestra.run.smithi007.stdout:ceph packages for x86_64 194 kB/s | 78 kB 00:00 2024-01-27T02:28:56.388 INFO:teuthology.orchestra.run.smithi188.stdout:ceph noarch packages 28 kB/s | 9.2 kB 00:00 2024-01-27T02:28:56.537 INFO:teuthology.orchestra.run.smithi007.stdout:ceph noarch packages 34 kB/s | 9.2 kB 00:00 2024-01-27T02:28:56.680 INFO:teuthology.orchestra.run.smithi188.stdout:ceph source packages 5.4 kB/s | 1.5 kB 00:00 2024-01-27T02:28:56.838 INFO:teuthology.orchestra.run.smithi007.stdout:ceph source packages 5.4 kB/s | 1.5 kB 00:00 2024-01-27T02:28:56.913 INFO:teuthology.orchestra.run.smithi188.stdout:Copr repo for python3-asyncssh owned by ceph 17 kB/s | 3.5 kB 00:00 2024-01-27T02:28:57.006 INFO:teuthology.orchestra.run.smithi007.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2024-01-27T02:28:58.482 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2024-01-27T02:28:58.792 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 9.2 MB/s | 16 MB 00:01 2024-01-27T02:29:02.015 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 473 kB/s | 24 kB 00:00 2024-01-27T02:29:02.814 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 520 kB/s | 24 kB 00:00 2024-01-27T02:29:03.358 INFO:teuthology.orchestra.run.smithi188.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 46 MB/s | 59 MB 00:01 2024-01-27T02:29:04.266 INFO:teuthology.orchestra.run.smithi007.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 65 MB 00:01 2024-01-27T02:29:14.700 INFO:teuthology.orchestra.run.smithi188.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 42 MB/s | 65 MB 00:01 2024-01-27T02:29:15.882 INFO:teuthology.orchestra.run.smithi007.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 46 MB/s | 59 MB 00:01 2024-01-27T02:29:24.950 INFO:teuthology.orchestra.run.smithi188.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2024-01-27T02:29:27.609 INFO:teuthology.orchestra.run.smithi007.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2024-01-27T02:29:31.972 INFO:teuthology.orchestra.run.smithi188.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-01-27T02:29:31.974 INFO:teuthology.orchestra.run.smithi188.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-01-27T02:29:31.992 INFO:teuthology.orchestra.run.smithi188.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-01-27T02:29:31.994 INFO:teuthology.orchestra.run.smithi188.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-01-27T02:29:31.995 INFO:teuthology.orchestra.run.smithi188.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-01-27T02:29:31.997 INFO:teuthology.orchestra.run.smithi188.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-01-27T02:29:31.999 INFO:teuthology.orchestra.run.smithi188.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-01-27T02:29:32.004 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-01-27T02:29:32.005 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-01-27T02:29:32.206 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-01-27T02:29:32.220 INFO:teuthology.orchestra.run.smithi188.stdout:=========================================================================================================================== 2024-01-27T02:29:32.220 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-01-27T02:29:32.220 INFO:teuthology.orchestra.run.smithi188.stdout:=========================================================================================================================== 2024-01-27T02:29:32.220 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-01-27T02:29:32.220 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-01-27T02:29:32.220 INFO:teuthology.orchestra.run.smithi188.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-01-27T02:29:32.220 INFO:teuthology.orchestra.run.smithi188.stdout: ceph x86_64 2:17.2.7-472.g74950076.el8 ceph 6.4 k 2024-01-27T02:29:32.221 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base x86_64 2:17.2.7-472.g74950076.el8 ceph 5.2 M 2024-01-27T02:29:32.221 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse x86_64 2:17.2.7-472.g74950076.el8 ceph 877 k 2024-01-27T02:29:32.221 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-472.g74950076.el8 ceph 162 k 2024-01-27T02:29:32.221 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr x86_64 2:17.2.7-472.g74950076.el8 ceph 1.6 M 2024-01-27T02:29:32.221 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm noarch 2:17.2.7-472.g74950076.el8 ceph-noarch 121 k 2024-01-27T02:29:32.221 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard noarch 2:17.2.7-472.g74950076.el8 ceph-noarch 1.7 M 2024-01-27T02:29:32.221 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-472.g74950076.el8 ceph-noarch 7.4 M 2024-01-27T02:29:32.221 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook noarch 2:17.2.7-472.g74950076.el8 ceph-noarch 47 k 2024-01-27T02:29:32.221 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw x86_64 2:17.2.7-472.g74950076.el8 ceph 17 M 2024-01-27T02:29:32.221 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test x86_64 2:17.2.7-472.g74950076.el8 ceph 52 M 2024-01-27T02:29:32.221 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume noarch 2:17.2.7-472.g74950076.el8 ceph-noarch 267 k 2024-01-27T02:29:32.221 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm noarch 2:17.2.7-472.g74950076.el8 ceph-noarch 80 k 2024-01-27T02:29:32.221 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror x86_64 2:17.2.7-472.g74950076.el8 ceph 255 k 2024-01-27T02:29:32.221 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top noarch 2:17.2.7-472.g74950076.el8 ceph-noarch 18 k 2024-01-27T02:29:32.222 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-01-27T02:29:32.222 INFO:teuthology.orchestra.run.smithi188.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-01-27T02:29:32.222 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-01-27T02:29:32.222 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel x86_64 2:17.2.7-472.g74950076.el8 ceph 25 k 2024-01-27T02:29:32.222 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2 x86_64 2:17.2.7-472.g74950076.el8 ceph 797 k 2024-01-27T02:29:32.222 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel x86_64 2:17.2.7-472.g74950076.el8 ceph 130 k 2024-01-27T02:29:32.222 INFO:teuthology.orchestra.run.smithi188.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-01-27T02:29:32.222 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-01-27T02:29:32.222 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-01-27T02:29:32.222 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs x86_64 2:17.2.7-472.g74950076.el8 ceph 190 k 2024-01-27T02:29:32.222 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados x86_64 2:17.2.7-472.g74950076.el8 ceph 370 k 2024-01-27T02:29:32.222 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd x86_64 2:17.2.7-472.g74950076.el8 ceph 359 k 2024-01-27T02:29:32.222 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw x86_64 2:17.2.7-472.g74950076.el8 ceph 106 k 2024-01-27T02:29:32.223 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse x86_64 2:17.2.7-472.g74950076.el8 ceph 90 k 2024-01-27T02:29:32.223 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror x86_64 2:17.2.7-472.g74950076.el8 ceph 3.5 M 2024-01-27T02:29:32.223 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd x86_64 2:17.2.7-472.g74950076.el8 ceph 175 k 2024-01-27T02:29:32.223 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-01-27T02:29:32.223 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrading: 2024-01-27T02:29:32.223 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-01-27T02:29:32.223 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-01-27T02:29:32.223 INFO:teuthology.orchestra.run.smithi188.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-01-27T02:29:32.223 INFO:teuthology.orchestra.run.smithi188.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-01-27T02:29:32.223 INFO:teuthology.orchestra.run.smithi188.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-01-27T02:29:32.223 INFO:teuthology.orchestra.run.smithi188.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-01-27T02:29:32.223 INFO:teuthology.orchestra.run.smithi188.stdout: librados2 x86_64 2:17.2.7-472.g74950076.el8 ceph 3.5 M 2024-01-27T02:29:32.224 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1 x86_64 2:17.2.7-472.g74950076.el8 ceph 3.6 M 2024-01-27T02:29:32.224 INFO:teuthology.orchestra.run.smithi188.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-01-27T02:29:32.224 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-01-27T02:29:32.224 INFO:teuthology.orchestra.run.smithi188.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-01-27T02:29:32.224 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-01-27T02:29:32.224 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-01-27T02:29:32.224 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-01-27T02:29:32.224 INFO:teuthology.orchestra.run.smithi188.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-01-27T02:29:32.224 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-01-27T02:29:32.224 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-01-27T02:29:32.224 INFO:teuthology.orchestra.run.smithi188.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-01-27T02:29:32.224 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common x86_64 2:17.2.7-472.g74950076.el8 ceph 23 M 2024-01-27T02:29:32.225 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards noarch 2:17.2.7-472.g74950076.el8 ceph-noarch 23 k 2024-01-27T02:29:32.225 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds x86_64 2:17.2.7-472.g74950076.el8 ceph 2.3 M 2024-01-27T02:29:32.225 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core noarch 2:17.2.7-472.g74950076.el8 ceph-noarch 239 k 2024-01-27T02:29:32.225 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon x86_64 2:17.2.7-472.g74950076.el8 ceph 3.9 M 2024-01-27T02:29:32.225 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd x86_64 2:17.2.7-472.g74950076.el8 ceph 17 M 2024-01-27T02:29:32.225 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts noarch 2:17.2.7-472.g74950076.el8 ceph-noarch 14 k 2024-01-27T02:29:32.225 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux x86_64 2:17.2.7-472.g74950076.el8 ceph 24 k 2024-01-27T02:29:32.225 INFO:teuthology.orchestra.run.smithi188.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-01-27T02:29:32.225 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-01-27T02:29:32.225 INFO:teuthology.orchestra.run.smithi188.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-01-27T02:29:32.225 INFO:teuthology.orchestra.run.smithi188.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-01-27T02:29:32.225 INFO:teuthology.orchestra.run.smithi188.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-01-27T02:29:32.225 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite x86_64 2:17.2.7-472.g74950076.el8 ceph 178 k 2024-01-27T02:29:32.225 INFO:teuthology.orchestra.run.smithi188.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-01-27T02:29:32.225 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-01-27T02:29:32.226 INFO:teuthology.orchestra.run.smithi188.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-01-27T02:29:32.226 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-01-27T02:29:32.226 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2024-01-27T02:29:32.226 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1 x86_64 2:17.2.7-472.g74950076.el8 ceph 506 k 2024-01-27T02:29:32.226 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-01-27T02:29:32.226 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2 x86_64 2:17.2.7-472.g74950076.el8 ceph 5.6 M 2024-01-27T02:29:32.226 INFO:teuthology.orchestra.run.smithi188.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-01-27T02:29:32.226 INFO:teuthology.orchestra.run.smithi188.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-01-27T02:29:32.226 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-01-27T02:29:32.226 INFO:teuthology.orchestra.run.smithi188.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-01-27T02:29:32.226 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-01-27T02:29:32.226 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-01-27T02:29:32.226 INFO:teuthology.orchestra.run.smithi188.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2024-01-27T02:29:32.226 INFO:teuthology.orchestra.run.smithi188.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-01-27T02:29:32.226 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-01-27T02:29:32.227 INFO:teuthology.orchestra.run.smithi188.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-01-27T02:29:32.227 INFO:teuthology.orchestra.run.smithi188.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-01-27T02:29:32.227 INFO:teuthology.orchestra.run.smithi188.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-01-27T02:29:32.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-01-27T02:29:32.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-01-27T02:29:32.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-01-27T02:29:32.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-01-27T02:29:32.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-01-27T02:29:32.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse x86_64 2:17.2.7-472.g74950076.el8 ceph 45 k 2024-01-27T02:29:32.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common x86_64 2:17.2.7-472.g74950076.el8 ceph 115 k 2024-01-27T02:29:32.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-01-27T02:29:32.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-01-27T02:29:32.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-01-27T02:29:32.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-01-27T02:29:32.227 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2024-01-27T02:29:32.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-01-27T02:29:32.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-01-27T02:29:32.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-01-27T02:29:32.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-01-27T02:29:32.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-01-27T02:29:32.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-01-27T02:29:32.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-01-27T02:29:32.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-01-27T02:29:32.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-01-27T02:29:32.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-01-27T02:29:32.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-01-27T02:29:32.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-01-27T02:29:32.228 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-01-27T02:29:32.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-01-27T02:29:32.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-01-27T02:29:32.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-01-27T02:29:32.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-01-27T02:29:32.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-01-27T02:29:32.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-01-27T02:29:32.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-01-27T02:29:32.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-01-27T02:29:32.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-01-27T02:29:32.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-01-27T02:29:32.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-01-27T02:29:32.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-01-27T02:29:32.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-01-27T02:29:32.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-01-27T02:29:32.229 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-01-27T02:29:32.230 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-01-27T02:29:32.230 INFO:teuthology.orchestra.run.smithi188.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-01-27T02:29:32.230 INFO:teuthology.orchestra.run.smithi188.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-01-27T02:29:32.230 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-01-27T02:29:32.230 INFO:teuthology.orchestra.run.smithi188.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-01-27T02:29:32.230 INFO:teuthology.orchestra.run.smithi188.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-01-27T02:29:32.230 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-01-27T02:29:32.230 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-01-27T02:29:32.230 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-01-27T02:29:32.230 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-01-27T02:29:32.230 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-01-27T02:29:32.230 INFO:teuthology.orchestra.run.smithi188.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-01-27T02:29:32.230 INFO:teuthology.orchestra.run.smithi188.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-01-27T02:29:32.230 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-01-27T02:29:32.231 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-01-27T02:29:32.231 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:29:32.231 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-01-27T02:29:32.231 INFO:teuthology.orchestra.run.smithi188.stdout:=========================================================================================================================== 2024-01-27T02:29:32.231 INFO:teuthology.orchestra.run.smithi188.stdout:Install 124 Packages 2024-01-27T02:29:32.231 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrade 16 Packages 2024-01-27T02:29:32.231 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:29:32.231 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 212 M 2024-01-27T02:29:32.231 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-01-27T02:29:32.383 INFO:teuthology.orchestra.run.smithi188.stdout:(1/140): ceph-17.2.7-472.g74950076.el8.x86_64.r 56 kB/s | 6.4 kB 00:00 2024-01-27T02:29:32.567 INFO:teuthology.orchestra.run.smithi188.stdout:(2/140): ceph-fuse-17.2.7-472.g74950076.el8.x86 4.7 MB/s | 877 kB 00:00 2024-01-27T02:29:32.626 INFO:teuthology.orchestra.run.smithi188.stdout:(3/140): ceph-immutable-object-cache-17.2.7-472 2.7 MB/s | 162 kB 00:00 2024-01-27T02:29:32.926 INFO:teuthology.orchestra.run.smithi188.stdout:(4/140): ceph-mds-17.2.7-472.g74950076.el8.x86_ 7.7 MB/s | 2.3 MB 00:00 2024-01-27T02:29:33.111 INFO:teuthology.orchestra.run.smithi188.stdout:(5/140): ceph-base-17.2.7-472.g74950076.el8.x86 6.2 MB/s | 5.2 MB 00:00 2024-01-27T02:29:33.151 INFO:teuthology.orchestra.run.smithi188.stdout:(6/140): ceph-mgr-17.2.7-472.g74950076.el8.x86_ 7.0 MB/s | 1.6 MB 00:00 2024-01-27T02:29:33.485 INFO:teuthology.orchestra.run.smithi188.stdout:(7/140): ceph-mon-17.2.7-472.g74950076.el8.x86_ 11 MB/s | 3.9 MB 00:00 2024-01-27T02:29:34.727 INFO:teuthology.orchestra.run.smithi188.stdout:(8/140): ceph-radosgw-17.2.7-472.g74950076.el8. 14 MB/s | 17 MB 00:01 2024-01-27T02:29:34.835 INFO:teuthology.orchestra.run.smithi188.stdout:(9/140): ceph-selinux-17.2.7-472.g74950076.el8. 225 kB/s | 24 kB 00:00 2024-01-27T02:29:35.527 INFO:teuthology.orchestra.run.smithi188.stdout:(10/140): ceph-osd-17.2.7-472.g74950076.el8.x86 7.0 MB/s | 17 MB 00:02 2024-01-27T02:29:35.561 INFO:teuthology.orchestra.run.smithi007.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-01-27T02:29:35.562 INFO:teuthology.orchestra.run.smithi007.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-01-27T02:29:35.583 INFO:teuthology.orchestra.run.smithi007.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-01-27T02:29:35.585 INFO:teuthology.orchestra.run.smithi007.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-01-27T02:29:35.586 INFO:teuthology.orchestra.run.smithi007.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-01-27T02:29:35.588 INFO:teuthology.orchestra.run.smithi007.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-01-27T02:29:35.591 INFO:teuthology.orchestra.run.smithi007.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-01-27T02:29:35.596 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-01-27T02:29:35.597 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-01-27T02:29:35.778 INFO:teuthology.orchestra.run.smithi188.stdout:(11/140): ceph-common-17.2.7-472.g74950076.el8. 6.7 MB/s | 23 MB 00:03 2024-01-27T02:29:35.803 INFO:teuthology.orchestra.run.smithi188.stdout:(12/140): cephfs-mirror-17.2.7-472.g74950076.el 927 kB/s | 255 kB 00:00 2024-01-27T02:29:35.827 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-01-27T02:29:35.843 INFO:teuthology.orchestra.run.smithi007.stdout:=========================================================================================================================== 2024-01-27T02:29:35.844 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-01-27T02:29:35.844 INFO:teuthology.orchestra.run.smithi007.stdout:=========================================================================================================================== 2024-01-27T02:29:35.844 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-01-27T02:29:35.844 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-01-27T02:29:35.844 INFO:teuthology.orchestra.run.smithi007.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-01-27T02:29:35.844 INFO:teuthology.orchestra.run.smithi007.stdout: ceph x86_64 2:17.2.7-472.g74950076.el8 ceph 6.4 k 2024-01-27T02:29:35.844 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base x86_64 2:17.2.7-472.g74950076.el8 ceph 5.2 M 2024-01-27T02:29:35.844 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse x86_64 2:17.2.7-472.g74950076.el8 ceph 877 k 2024-01-27T02:29:35.844 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-472.g74950076.el8 ceph 162 k 2024-01-27T02:29:35.844 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr x86_64 2:17.2.7-472.g74950076.el8 ceph 1.6 M 2024-01-27T02:29:35.844 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm noarch 2:17.2.7-472.g74950076.el8 ceph-noarch 121 k 2024-01-27T02:29:35.844 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard noarch 2:17.2.7-472.g74950076.el8 ceph-noarch 1.7 M 2024-01-27T02:29:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-472.g74950076.el8 ceph-noarch 7.4 M 2024-01-27T02:29:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook noarch 2:17.2.7-472.g74950076.el8 ceph-noarch 47 k 2024-01-27T02:29:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw x86_64 2:17.2.7-472.g74950076.el8 ceph 17 M 2024-01-27T02:29:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test x86_64 2:17.2.7-472.g74950076.el8 ceph 52 M 2024-01-27T02:29:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume noarch 2:17.2.7-472.g74950076.el8 ceph-noarch 267 k 2024-01-27T02:29:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm noarch 2:17.2.7-472.g74950076.el8 ceph-noarch 80 k 2024-01-27T02:29:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror x86_64 2:17.2.7-472.g74950076.el8 ceph 255 k 2024-01-27T02:29:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top noarch 2:17.2.7-472.g74950076.el8 ceph-noarch 18 k 2024-01-27T02:29:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-01-27T02:29:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-01-27T02:29:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-01-27T02:29:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel x86_64 2:17.2.7-472.g74950076.el8 ceph 25 k 2024-01-27T02:29:35.845 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 x86_64 2:17.2.7-472.g74950076.el8 ceph 797 k 2024-01-27T02:29:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel x86_64 2:17.2.7-472.g74950076.el8 ceph 130 k 2024-01-27T02:29:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-01-27T02:29:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-01-27T02:29:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-01-27T02:29:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs x86_64 2:17.2.7-472.g74950076.el8 ceph 190 k 2024-01-27T02:29:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados x86_64 2:17.2.7-472.g74950076.el8 ceph 370 k 2024-01-27T02:29:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd x86_64 2:17.2.7-472.g74950076.el8 ceph 359 k 2024-01-27T02:29:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw x86_64 2:17.2.7-472.g74950076.el8 ceph 106 k 2024-01-27T02:29:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse x86_64 2:17.2.7-472.g74950076.el8 ceph 90 k 2024-01-27T02:29:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror x86_64 2:17.2.7-472.g74950076.el8 ceph 3.5 M 2024-01-27T02:29:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd x86_64 2:17.2.7-472.g74950076.el8 ceph 175 k 2024-01-27T02:29:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-01-27T02:29:35.846 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2024-01-27T02:29:35.846 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-01-27T02:29:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-01-27T02:29:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-01-27T02:29:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-01-27T02:29:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-01-27T02:29:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-01-27T02:29:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 x86_64 2:17.2.7-472.g74950076.el8 ceph 3.5 M 2024-01-27T02:29:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1 x86_64 2:17.2.7-472.g74950076.el8 ceph 3.6 M 2024-01-27T02:29:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-01-27T02:29:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-01-27T02:29:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-01-27T02:29:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-01-27T02:29:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-01-27T02:29:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-01-27T02:29:35.847 INFO:teuthology.orchestra.run.smithi007.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-01-27T02:29:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-01-27T02:29:35.848 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-01-27T02:29:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-01-27T02:29:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common x86_64 2:17.2.7-472.g74950076.el8 ceph 23 M 2024-01-27T02:29:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards noarch 2:17.2.7-472.g74950076.el8 ceph-noarch 23 k 2024-01-27T02:29:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds x86_64 2:17.2.7-472.g74950076.el8 ceph 2.3 M 2024-01-27T02:29:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core noarch 2:17.2.7-472.g74950076.el8 ceph-noarch 239 k 2024-01-27T02:29:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon x86_64 2:17.2.7-472.g74950076.el8 ceph 3.9 M 2024-01-27T02:29:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd x86_64 2:17.2.7-472.g74950076.el8 ceph 17 M 2024-01-27T02:29:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts noarch 2:17.2.7-472.g74950076.el8 ceph-noarch 14 k 2024-01-27T02:29:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux x86_64 2:17.2.7-472.g74950076.el8 ceph 24 k 2024-01-27T02:29:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-01-27T02:29:35.848 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-01-27T02:29:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-01-27T02:29:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-01-27T02:29:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-01-27T02:29:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite x86_64 2:17.2.7-472.g74950076.el8 ceph 178 k 2024-01-27T02:29:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-01-27T02:29:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-01-27T02:29:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-01-27T02:29:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-01-27T02:29:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2024-01-27T02:29:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 x86_64 2:17.2.7-472.g74950076.el8 ceph 506 k 2024-01-27T02:29:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-01-27T02:29:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2 x86_64 2:17.2.7-472.g74950076.el8 ceph 5.6 M 2024-01-27T02:29:35.849 INFO:teuthology.orchestra.run.smithi007.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-01-27T02:29:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-01-27T02:29:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-01-27T02:29:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-01-27T02:29:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-01-27T02:29:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-01-27T02:29:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2024-01-27T02:29:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-01-27T02:29:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-01-27T02:29:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-01-27T02:29:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-01-27T02:29:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-01-27T02:29:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-01-27T02:29:35.850 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-01-27T02:29:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-01-27T02:29:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-01-27T02:29:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-01-27T02:29:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse x86_64 2:17.2.7-472.g74950076.el8 ceph 45 k 2024-01-27T02:29:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common x86_64 2:17.2.7-472.g74950076.el8 ceph 115 k 2024-01-27T02:29:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-01-27T02:29:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-01-27T02:29:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-01-27T02:29:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-01-27T02:29:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2024-01-27T02:29:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-01-27T02:29:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-01-27T02:29:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-01-27T02:29:35.851 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-01-27T02:29:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-01-27T02:29:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-01-27T02:29:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-01-27T02:29:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-01-27T02:29:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-01-27T02:29:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-01-27T02:29:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-01-27T02:29:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-01-27T02:29:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-01-27T02:29:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-01-27T02:29:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-01-27T02:29:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-01-27T02:29:35.852 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-01-27T02:29:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-01-27T02:29:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-01-27T02:29:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-01-27T02:29:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-01-27T02:29:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-01-27T02:29:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-01-27T02:29:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-01-27T02:29:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-01-27T02:29:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-01-27T02:29:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-01-27T02:29:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-01-27T02:29:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-01-27T02:29:35.853 INFO:teuthology.orchestra.run.smithi007.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-01-27T02:29:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-01-27T02:29:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-01-27T02:29:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-01-27T02:29:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-01-27T02:29:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-01-27T02:29:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-01-27T02:29:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-01-27T02:29:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-01-27T02:29:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-01-27T02:29:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-01-27T02:29:35.854 INFO:teuthology.orchestra.run.smithi007.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-01-27T02:29:35.855 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-01-27T02:29:35.855 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-01-27T02:29:35.855 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:29:35.855 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-01-27T02:29:35.855 INFO:teuthology.orchestra.run.smithi007.stdout:=========================================================================================================================== 2024-01-27T02:29:35.855 INFO:teuthology.orchestra.run.smithi007.stdout:Install 124 Packages 2024-01-27T02:29:35.855 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 16 Packages 2024-01-27T02:29:35.855 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:29:35.855 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 212 M 2024-01-27T02:29:35.856 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-01-27T02:29:35.878 INFO:teuthology.orchestra.run.smithi188.stdout:(13/140): libcephfs-devel-17.2.7-472.g74950076. 248 kB/s | 25 kB 00:00 2024-01-27T02:29:35.937 INFO:teuthology.orchestra.run.smithi188.stdout:(14/140): libcephfs2-17.2.7-472.g74950076.el8.x 5.8 MB/s | 797 kB 00:00 2024-01-27T02:29:35.987 INFO:teuthology.orchestra.run.smithi188.stdout:(15/140): libcephsqlite-17.2.7-472.g74950076.el 1.6 MB/s | 178 kB 00:00 2024-01-27T02:29:36.012 INFO:teuthology.orchestra.run.smithi188.stdout:(16/140): librados-devel-17.2.7-472.g74950076.e 1.7 MB/s | 130 kB 00:00 2024-01-27T02:29:36.194 INFO:teuthology.orchestra.run.smithi188.stdout:(17/140): libradosstriper1-17.2.7-472.g74950076 2.4 MB/s | 506 kB 00:00 2024-01-27T02:29:36.243 INFO:teuthology.orchestra.run.smithi007.stdout:(1/140): ceph-17.2.7-472.g74950076.el8.x86_64.r 27 kB/s | 6.4 kB 00:00 2024-01-27T02:29:36.296 INFO:teuthology.orchestra.run.smithi188.stdout:(18/140): python3-ceph-argparse-17.2.7-472.g749 437 kB/s | 45 kB 00:00 2024-01-27T02:29:36.396 INFO:teuthology.orchestra.run.smithi188.stdout:(19/140): python3-ceph-common-17.2.7-472.g74950 1.1 MB/s | 115 kB 00:00 2024-01-27T02:29:36.539 INFO:teuthology.orchestra.run.smithi188.stdout:(20/140): python3-cephfs-17.2.7-472.g74950076.e 1.3 MB/s | 190 kB 00:00 2024-01-27T02:29:36.664 INFO:teuthology.orchestra.run.smithi188.stdout:(21/140): python3-rados-17.2.7-472.g74950076.el 2.9 MB/s | 370 kB 00:00 2024-01-27T02:29:36.797 INFO:teuthology.orchestra.run.smithi188.stdout:(22/140): python3-rbd-17.2.7-472.g74950076.el8. 2.6 MB/s | 359 kB 00:00 2024-01-27T02:29:36.898 INFO:teuthology.orchestra.run.smithi188.stdout:(23/140): librgw2-17.2.7-472.g74950076.el8.x86_ 6.3 MB/s | 5.6 MB 00:00 2024-01-27T02:29:36.923 INFO:teuthology.orchestra.run.smithi188.stdout:(24/140): python3-rgw-17.2.7-472.g74950076.el8. 850 kB/s | 106 kB 00:00 2024-01-27T02:29:36.948 INFO:teuthology.orchestra.run.smithi188.stdout:(25/140): rbd-fuse-17.2.7-472.g74950076.el8.x86 1.8 MB/s | 90 kB 00:00 2024-01-27T02:29:36.984 INFO:teuthology.orchestra.run.smithi007.stdout:(2/140): ceph-fuse-17.2.7-472.g74950076.el8.x86 1.2 MB/s | 877 kB 00:00 2024-01-27T02:29:36.998 INFO:teuthology.orchestra.run.smithi188.stdout:(26/140): rbd-nbd-17.2.7-472.g74950076.el8.x86_ 3.4 MB/s | 175 kB 00:00 2024-01-27T02:29:37.065 INFO:teuthology.orchestra.run.smithi188.stdout:(27/140): ceph-grafana-dashboards-17.2.7-472.g7 348 kB/s | 23 kB 00:00 2024-01-27T02:29:37.110 INFO:teuthology.orchestra.run.smithi007.stdout:(3/140): ceph-immutable-object-cache-17.2.7-472 1.3 MB/s | 162 kB 00:00 2024-01-27T02:29:37.132 INFO:teuthology.orchestra.run.smithi188.stdout:(28/140): ceph-mgr-cephadm-17.2.7-472.g74950076 1.8 MB/s | 121 kB 00:00 2024-01-27T02:29:37.474 INFO:teuthology.orchestra.run.smithi188.stdout:(29/140): rbd-mirror-17.2.7-472.g74950076.el8.x 6.3 MB/s | 3.5 MB 00:00 2024-01-27T02:29:37.535 INFO:teuthology.orchestra.run.smithi188.stdout:(30/140): ceph-mgr-dashboard-17.2.7-472.g749500 4.2 MB/s | 1.7 MB 00:00 2024-01-27T02:29:37.633 INFO:teuthology.orchestra.run.smithi188.stdout:(31/140): ceph-mgr-modules-core-17.2.7-472.g749 2.4 MB/s | 239 kB 00:00 2024-01-27T02:29:37.717 INFO:teuthology.orchestra.run.smithi188.stdout:(32/140): ceph-mgr-rook-17.2.7-472.g74950076.el 568 kB/s | 47 kB 00:00 2024-01-27T02:29:37.800 INFO:teuthology.orchestra.run.smithi188.stdout:(33/140): ceph-prometheus-alerts-17.2.7-472.g74 174 kB/s | 14 kB 00:00 2024-01-27T02:29:37.901 INFO:teuthology.orchestra.run.smithi188.stdout:(34/140): ceph-volume-17.2.7-472.g74950076.el8. 2.6 MB/s | 267 kB 00:00 2024-01-27T02:29:37.993 INFO:teuthology.orchestra.run.smithi188.stdout:(35/140): cephadm-17.2.7-472.g74950076.el8.noar 875 kB/s | 80 kB 00:00 2024-01-27T02:29:38.002 INFO:teuthology.orchestra.run.smithi007.stdout:(4/140): ceph-mds-17.2.7-472.g74950076.el8.x86_ 2.6 MB/s | 2.3 MB 00:00 2024-01-27T02:29:38.068 INFO:teuthology.orchestra.run.smithi188.stdout:(36/140): cephfs-top-17.2.7-472.g74950076.el8.n 241 kB/s | 18 kB 00:00 2024-01-27T02:29:38.277 INFO:teuthology.orchestra.run.smithi007.stdout:(5/140): ceph-base-17.2.7-472.g74950076.el8.x86 2.3 MB/s | 5.2 MB 00:02 2024-01-27T02:29:38.469 INFO:teuthology.orchestra.run.smithi007.stdout:(6/140): ceph-mgr-17.2.7-472.g74950076.el8.x86_ 3.4 MB/s | 1.6 MB 00:00 2024-01-27T02:29:38.585 INFO:teuthology.orchestra.run.smithi188.stdout:(37/140): ceph-mgr-diskprediction-local-17.2.7- 6.7 MB/s | 7.4 MB 00:01 2024-01-27T02:29:38.794 INFO:teuthology.orchestra.run.smithi007.stdout:(7/140): ceph-mon-17.2.7-472.g74950076.el8.x86_ 7.6 MB/s | 3.9 MB 00:00 2024-01-27T02:29:39.086 INFO:teuthology.orchestra.run.smithi188.stdout:(38/140): ceph-test-17.2.7-472.g74950076.el8.x8 12 MB/s | 52 MB 00:04 2024-01-27T02:29:39.111 INFO:teuthology.orchestra.run.smithi188.stdout:(39/140): gperftools-libs-2.7-9.el8.x86_64.rpm 294 kB/s | 306 kB 00:01 2024-01-27T02:29:39.378 INFO:teuthology.orchestra.run.smithi188.stdout:(40/140): python3-asyncssh-2.7.0-2.el8.noarch.r 1.7 MB/s | 476 kB 00:00 2024-01-27T02:29:39.436 INFO:teuthology.orchestra.run.smithi188.stdout:(41/140): liboath-2.6.2-3.el8.x86_64.rpm 69 kB/s | 59 kB 00:00 2024-01-27T02:29:39.486 INFO:teuthology.orchestra.run.smithi188.stdout:(42/140): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 402 kB/s | 44 kB 00:00 2024-01-27T02:29:39.537 INFO:teuthology.orchestra.run.smithi188.stdout:(43/140): libunwind-1.3.1-3.el8.x86_64.rpm 168 kB/s | 75 kB 00:00 2024-01-27T02:29:39.687 INFO:teuthology.orchestra.run.smithi188.stdout:(44/140): python3-cachetools-3.1.1-4.el8.noarch 165 kB/s | 33 kB 00:00 2024-01-27T02:29:39.721 INFO:teuthology.orchestra.run.smithi188.stdout:(45/140): python3-certifi-2018.10.15-7.el8.noar 89 kB/s | 16 kB 00:00 2024-01-27T02:29:39.746 INFO:teuthology.orchestra.run.smithi188.stdout:(46/140): python3-beautifulsoup4-4.6.3-2.el8.1. 598 kB/s | 185 kB 00:00 2024-01-27T02:29:39.813 INFO:teuthology.orchestra.run.smithi188.stdout:(47/140): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2024-01-27T02:29:39.872 INFO:teuthology.orchestra.run.smithi188.stdout:(48/140): python3-google-auth-1.1.1-10.el8.noar 648 kB/s | 81 kB 00:00 2024-01-27T02:29:39.939 INFO:teuthology.orchestra.run.smithi188.stdout:(49/140): python3-jaraco-6.2-6.el8.noarch.rpm 89 kB/s | 11 kB 00:00 2024-01-27T02:29:39.981 INFO:teuthology.orchestra.run.smithi188.stdout:(50/140): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2024-01-27T02:29:40.031 INFO:teuthology.orchestra.run.smithi188.stdout:(51/140): python3-cherrypy-18.4.0-1.el8.noarch. 1.2 MB/s | 384 kB 00:00 2024-01-27T02:29:40.098 INFO:teuthology.orchestra.run.smithi188.stdout:(52/140): python3-logutils-0.3.5-11.el8.noarch. 417 kB/s | 49 kB 00:00 2024-01-27T02:29:40.139 INFO:teuthology.orchestra.run.smithi188.stdout:(53/140): python3-more-itertools-7.2.0-3.el8.no 541 kB/s | 59 kB 00:00 2024-01-27T02:29:40.206 INFO:teuthology.orchestra.run.smithi188.stdout:(54/140): python3-kubernetes-11.0.0-6.el8.noarc 4.4 MB/s | 1.2 MB 00:00 2024-01-27T02:29:40.236 INFO:teuthology.orchestra.run.smithi007.stdout:(8/140): ceph-osd-17.2.7-472.g74950076.el8.x86_ 9.5 MB/s | 17 MB 00:01 2024-01-27T02:29:40.258 INFO:teuthology.orchestra.run.smithi188.stdout:(55/140): python3-natsort-7.1.1-2.el8.noarch.rp 376 kB/s | 60 kB 00:00 2024-01-27T02:29:40.316 INFO:teuthology.orchestra.run.smithi188.stdout:(56/140): python3-pecan-1.3.2-9.el8.noarch.rpm 1.6 MB/s | 283 kB 00:00 2024-01-27T02:29:40.332 INFO:teuthology.orchestra.run.smithi188.stdout:(57/140): python3-portend-2.6-1.el8.noarch.rpm 124 kB/s | 16 kB 00:00 2024-01-27T02:29:40.374 INFO:teuthology.orchestra.run.smithi188.stdout:(58/140): python3-repoze-lru-0.7-6.el8.noarch.r 291 kB/s | 34 kB 00:00 2024-01-27T02:29:40.441 INFO:teuthology.orchestra.run.smithi188.stdout:(59/140): python3-routes-2.4.1-12.el8.noarch.rp 1.8 MB/s | 196 kB 00:00 2024-01-27T02:29:40.453 INFO:teuthology.orchestra.run.smithi007.stdout:(9/140): ceph-radosgw-17.2.7-472.g74950076.el8. 10 MB/s | 17 MB 00:01 2024-01-27T02:29:40.466 INFO:teuthology.orchestra.run.smithi188.stdout:(60/140): python3-rsa-4.9-2.el8.noarch.rpm 481 kB/s | 64 kB 00:00 2024-01-27T02:29:40.470 INFO:teuthology.orchestra.run.smithi007.stdout:(10/140): ceph-selinux-17.2.7-472.g74950076.el8 104 kB/s | 24 kB 00:00 2024-01-27T02:29:40.483 INFO:teuthology.orchestra.run.smithi188.stdout:(61/140): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2024-01-27T02:29:40.534 INFO:teuthology.orchestra.run.smithi188.stdout:(62/140): python3-singledispatch-3.4.0.3-18.el8 259 kB/s | 24 kB 00:00 2024-01-27T02:29:40.537 INFO:teuthology.orchestra.run.smithi007.stdout:(11/140): cephfs-mirror-17.2.7-472.g74950076.el 3.7 MB/s | 255 kB 00:00 2024-01-27T02:29:40.595 INFO:teuthology.orchestra.run.smithi007.stdout:(12/140): libcephfs-devel-17.2.7-472.g74950076. 426 kB/s | 25 kB 00:00 2024-01-27T02:29:40.596 INFO:teuthology.orchestra.run.smithi188.stdout:(63/140): python3-tempora-1.14.1-5.el8.noarch.r 226 kB/s | 29 kB 00:00 2024-01-27T02:29:40.618 INFO:teuthology.orchestra.run.smithi188.stdout:(64/140): python3-trustme-0.6.0-4.el8.noarch.rp 205 kB/s | 27 kB 00:00 2024-01-27T02:29:40.651 INFO:teuthology.orchestra.run.smithi188.stdout:(65/140): python3-waitress-1.4.3-1.el8.noarch.r 2.1 MB/s | 255 kB 00:00 2024-01-27T02:29:40.735 INFO:teuthology.orchestra.run.smithi188.stdout:(66/140): python3-webob-1.8.5-1.el8.1.noarch.rp 1.8 MB/s | 251 kB 00:00 2024-01-27T02:29:40.737 INFO:teuthology.orchestra.run.smithi007.stdout:(13/140): libcephfs2-17.2.7-472.g74950076.el8.x 5.5 MB/s | 797 kB 00:00 2024-01-27T02:29:40.760 INFO:teuthology.orchestra.run.smithi188.stdout:(67/140): python3-websocket-client-0.56.0-5.el8 430 kB/s | 61 kB 00:00 2024-01-27T02:29:40.786 INFO:teuthology.orchestra.run.smithi188.stdout:(68/140): python3-webtest-2.0.33-1.el8.noarch.r 638 kB/s | 85 kB 00:00 2024-01-27T02:29:40.803 INFO:teuthology.orchestra.run.smithi188.stdout:(69/140): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.1 MB/s | 70 kB 00:00 2024-01-27T02:29:40.836 INFO:teuthology.orchestra.run.smithi188.stdout:(70/140): python3-zc-lockfile-2.0-2.el8.noarch. 228 kB/s | 23 kB 00:00 2024-01-27T02:29:40.970 INFO:teuthology.orchestra.run.smithi188.stdout:(71/140): python3-jsonpointer-1.10-11.el8.noarc 120 kB/s | 20 kB 00:00 2024-01-27T02:29:41.012 INFO:teuthology.orchestra.run.smithi188.stdout:(72/140): python3-prettytable-0.7.2-14.el8.noar 251 kB/s | 44 kB 00:00 2024-01-27T02:29:41.062 INFO:teuthology.orchestra.run.smithi007.stdout:(14/140): ceph-common-17.2.7-472.g74950076.el8. 4.6 MB/s | 23 MB 00:05 2024-01-27T02:29:41.087 INFO:teuthology.orchestra.run.smithi188.stdout:(73/140): python3-pyasn1-modules-0.3.7-6.el8.no 941 kB/s | 110 kB 00:00 2024-01-27T02:29:41.088 INFO:teuthology.orchestra.run.smithi007.stdout:(15/140): libcephsqlite-17.2.7-472.g74950076.el 508 kB/s | 178 kB 00:00 2024-01-27T02:29:41.122 INFO:teuthology.orchestra.run.smithi007.stdout:(16/140): librados-devel-17.2.7-472.g74950076.e 2.1 MB/s | 130 kB 00:00 2024-01-27T02:29:41.154 INFO:teuthology.orchestra.run.smithi188.stdout:(74/140): thrift-0.13.0-2.el8.x86_64.rpm 4.4 MB/s | 1.7 MB 00:00 2024-01-27T02:29:41.163 INFO:teuthology.orchestra.run.smithi007.stdout:(17/140): libradosstriper1-17.2.7-472.g74950076 6.6 MB/s | 506 kB 00:00 2024-01-27T02:29:41.179 INFO:teuthology.orchestra.run.smithi188.stdout:(75/140): python3-jsonpatch-1.21-2.el8.noarch.r 159 kB/s | 27 kB 00:00 2024-01-27T02:29:41.205 INFO:teuthology.orchestra.run.smithi188.stdout:(76/140): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2024-01-27T02:29:41.214 INFO:teuthology.orchestra.run.smithi007.stdout:(18/140): python3-ceph-argparse-17.2.7-472.g749 895 kB/s | 45 kB 00:00 2024-01-27T02:29:41.281 INFO:teuthology.orchestra.run.smithi007.stdout:(19/140): python3-ceph-common-17.2.7-472.g74950 1.7 MB/s | 115 kB 00:00 2024-01-27T02:29:41.355 INFO:teuthology.orchestra.run.smithi188.stdout:(77/140): bison-3.0.4-10.el8.x86_64.rpm 3.8 MB/s | 688 kB 00:00 2024-01-27T02:29:41.356 INFO:teuthology.orchestra.run.smithi007.stdout:(20/140): python3-cephfs-17.2.7-472.g74950076.e 2.5 MB/s | 190 kB 00:00 2024-01-27T02:29:41.388 INFO:teuthology.orchestra.run.smithi188.stdout:(78/140): flex-2.6.1-9.el8.x86_64.rpm 1.8 MB/s | 320 kB 00:00 2024-01-27T02:29:41.414 INFO:teuthology.orchestra.run.smithi188.stdout:(79/140): python3-werkzeug-0.12.2-4.el8.noarch. 1.7 MB/s | 457 kB 00:00 2024-01-27T02:29:41.439 INFO:teuthology.orchestra.run.smithi007.stdout:(21/140): python3-rados-17.2.7-472.g74950076.el 4.3 MB/s | 370 kB 00:00 2024-01-27T02:29:41.489 INFO:teuthology.orchestra.run.smithi188.stdout:(80/140): libtool-2.4.6-25.el8.x86_64.rpm 5.2 MB/s | 709 kB 00:00 2024-01-27T02:29:41.506 INFO:teuthology.orchestra.run.smithi188.stdout:(81/140): python3-markupsafe-0.23-19.el8.x86_64 331 kB/s | 39 kB 00:00 2024-01-27T02:29:41.523 INFO:teuthology.orchestra.run.smithi007.stdout:(22/140): python3-rbd-17.2.7-472.g74950076.el8. 4.2 MB/s | 359 kB 00:00 2024-01-27T02:29:41.540 INFO:teuthology.orchestra.run.smithi188.stdout:(82/140): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.0 MB/s | 259 kB 00:00 2024-01-27T02:29:41.598 INFO:teuthology.orchestra.run.smithi007.stdout:(23/140): python3-rgw-17.2.7-472.g74950076.el8. 1.4 MB/s | 106 kB 00:00 2024-01-27T02:29:41.615 INFO:teuthology.orchestra.run.smithi188.stdout:(83/140): oniguruma-6.8.2-2.el8.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2024-01-27T02:29:41.632 INFO:teuthology.orchestra.run.smithi188.stdout:(84/140): python3-pyOpenSSL-19.0.0-1.el8.noarch 825 kB/s | 103 kB 00:00 2024-01-27T02:29:41.674 INFO:teuthology.orchestra.run.smithi007.stdout:(24/140): rbd-fuse-17.2.7-472.g74950076.el8.x86 1.2 MB/s | 90 kB 00:00 2024-01-27T02:29:41.749 INFO:teuthology.orchestra.run.smithi188.stdout:(85/140): python3-babel-2.5.1-7.el8.noarch.rpm 23 MB/s | 4.8 MB 00:00 2024-01-27T02:29:41.782 INFO:teuthology.orchestra.run.smithi188.stdout:(86/140): python3-jinja2-2.10.1-3.el8.noarch.rp 3.1 MB/s | 538 kB 00:00 2024-01-27T02:29:41.832 INFO:teuthology.orchestra.run.smithi007.stdout:(25/140): librgw2-17.2.7-472.g74950076.el8.x86_ 7.8 MB/s | 5.6 MB 00:00 2024-01-27T02:29:41.841 INFO:teuthology.orchestra.run.smithi188.stdout:(87/140): socat-1.7.4.1-1.el8.x86_64.rpm 3.4 MB/s | 323 kB 00:00 2024-01-27T02:29:41.923 INFO:teuthology.orchestra.run.smithi188.stdout:(88/140): python3-numpy-1.14.3-10.el8.x86_64.rp 13 MB/s | 3.7 MB 00:00 2024-01-27T02:29:41.950 INFO:teuthology.orchestra.run.smithi188.stdout:(89/140): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.3 MB/s | 225 kB 00:00 2024-01-27T02:29:41.984 INFO:teuthology.orchestra.run.smithi188.stdout:(90/140): librdkafka-0.11.4-3.el8.x86_64.rpm 2.4 MB/s | 354 kB 00:00 2024-01-27T02:29:41.999 INFO:teuthology.orchestra.run.smithi007.stdout:(26/140): rbd-nbd-17.2.7-472.g74950076.el8.x86_ 1.0 MB/s | 175 kB 00:00 2024-01-27T02:29:42.058 INFO:teuthology.orchestra.run.smithi007.stdout:(27/140): rbd-mirror-17.2.7-472.g74950076.el8.x 9.0 MB/s | 3.5 MB 00:00 2024-01-27T02:29:42.067 INFO:teuthology.orchestra.run.smithi188.stdout:(91/140): python3-mako-1.0.6-14.el8.noarch.rpm 1.3 MB/s | 157 kB 00:00 2024-01-27T02:29:42.083 INFO:teuthology.orchestra.run.smithi007.stdout:(28/140): ceph-grafana-dashboards-17.2.7-472.g7 278 kB/s | 23 kB 00:00 2024-01-27T02:29:42.125 INFO:teuthology.orchestra.run.smithi007.stdout:(29/140): ceph-mgr-cephadm-17.2.7-472.g74950076 1.8 MB/s | 121 kB 00:00 2024-01-27T02:29:42.143 INFO:teuthology.orchestra.run.smithi188.stdout:(92/140): openblas-threads-0.3.15-6.el8.x86_64. 24 MB/s | 5.2 MB 00:00 2024-01-27T02:29:42.276 INFO:teuthology.orchestra.run.smithi188.stdout:(93/140): openblas-0.3.15-6.el8.x86_64.rpm 17 MB/s | 5.0 MB 00:00 2024-01-27T02:29:42.301 INFO:teuthology.orchestra.run.smithi188.stdout:(94/140): jq-1.6-7.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2024-01-27T02:29:42.335 INFO:teuthology.orchestra.run.smithi188.stdout:(95/140): automake-1.16.1-8.el8.noarch.rpm 2.6 MB/s | 714 kB 00:00 2024-01-27T02:29:42.342 INFO:teuthology.orchestra.run.smithi007.stdout:(30/140): ceph-mgr-dashboard-17.2.7-472.g749500 6.6 MB/s | 1.7 MB 00:00 2024-01-27T02:29:42.394 INFO:teuthology.orchestra.run.smithi188.stdout:(96/140): python3-pytz-2017.2-11.el8.noarch.rpm 463 kB/s | 54 kB 00:00 2024-01-27T02:29:42.419 INFO:teuthology.orchestra.run.smithi188.stdout:(97/140): libverto-devel-0.3.0-5.el8.x86_64.rpm 214 kB/s | 18 kB 00:00 2024-01-27T02:29:42.442 INFO:teuthology.orchestra.run.smithi007.stdout:(31/140): ceph-mgr-modules-core-17.2.7-472.g749 2.3 MB/s | 239 kB 00:00 2024-01-27T02:29:42.528 INFO:teuthology.orchestra.run.smithi188.stdout:(98/140): python3-oauthlib-2.1.0-1.el8.noarch.r 1.4 MB/s | 155 kB 00:00 2024-01-27T02:29:42.553 INFO:teuthology.orchestra.run.smithi188.stdout:(99/140): python3-jwt-1.6.1-2.el8.noarch.rpm 268 kB/s | 43 kB 00:00 2024-01-27T02:29:42.573 INFO:teuthology.orchestra.run.smithi007.stdout:(32/140): ceph-mgr-rook-17.2.7-472.g74950076.el 363 kB/s | 47 kB 00:00 2024-01-27T02:29:42.668 INFO:teuthology.orchestra.run.smithi007.stdout:(33/140): ceph-prometheus-alerts-17.2.7-472.g74 153 kB/s | 14 kB 00:00 2024-01-27T02:29:42.754 INFO:teuthology.orchestra.run.smithi188.stdout:(100/140): python3-scipy-1.0.0-21.module+el8.9. 30 MB/s | 14 MB 00:00 2024-01-27T02:29:42.776 INFO:teuthology.orchestra.run.smithi007.stdout:(34/140): ceph-volume-17.2.7-472.g74950076.el8. 2.4 MB/s | 267 kB 00:00 2024-01-27T02:29:42.779 INFO:teuthology.orchestra.run.smithi188.stdout:(101/140): python3-requests-oauthlib-1.0.0-1.el 189 kB/s | 43 kB 00:00 2024-01-27T02:29:42.804 INFO:teuthology.orchestra.run.smithi188.stdout:(102/140): python3-pycparser-2.14-14.el8.noarch 396 kB/s | 109 kB 00:00 2024-01-27T02:29:42.869 INFO:teuthology.orchestra.run.smithi007.stdout:(35/140): ceph-mgr-diskprediction-local-17.2.7- 10 MB/s | 7.4 MB 00:00 2024-01-27T02:29:42.885 INFO:teuthology.orchestra.run.smithi007.stdout:(36/140): cephadm-17.2.7-472.g74950076.el8.noar 741 kB/s | 80 kB 00:00 2024-01-27T02:29:42.919 INFO:teuthology.orchestra.run.smithi007.stdout:(37/140): cephfs-top-17.2.7-472.g74950076.el8.n 360 kB/s | 18 kB 00:00 2024-01-27T02:29:42.946 INFO:teuthology.orchestra.run.smithi188.stdout:(103/140): libaio-devel-0.3.112-1.el8.x86_64.rp 97 kB/s | 19 kB 00:00 2024-01-27T02:29:42.971 INFO:teuthology.orchestra.run.smithi188.stdout:(104/140): libzstd-devel-1.4.4-1.el8.x86_64.rpm 228 kB/s | 44 kB 00:00 2024-01-27T02:29:42.996 INFO:teuthology.orchestra.run.smithi188.stdout:(105/140): pcre2-devel-10.32-2.el8.x86_64.rpm 3.1 MB/s | 605 kB 00:00 2024-01-27T02:29:43.088 INFO:teuthology.orchestra.run.smithi188.stdout:(106/140): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.6 MB/s | 229 kB 00:00 2024-01-27T02:29:43.114 INFO:teuthology.orchestra.run.smithi188.stdout:(107/140): libselinux-devel-2.9-5.el8.x86_64.rp 1.7 MB/s | 200 kB 00:00 2024-01-27T02:29:43.139 INFO:teuthology.orchestra.run.smithi188.stdout:(108/140): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.3 MB/s | 220 kB 00:00 2024-01-27T02:29:43.222 INFO:teuthology.orchestra.run.smithi188.stdout:(109/140): keyutils-libs-devel-1.5.10-9.el8.x86 360 kB/s | 48 kB 00:00 2024-01-27T02:29:43.273 INFO:teuthology.orchestra.run.smithi188.stdout:(110/140): zlib-devel-1.2.11-18.el8_5.x86_64.rp 432 kB/s | 58 kB 00:00 2024-01-27T02:29:43.290 INFO:teuthology.orchestra.run.smithi188.stdout:(111/140): libsepol-devel-2.9-3.el8.x86_64.rpm 495 kB/s | 87 kB 00:00 2024-01-27T02:29:43.340 INFO:teuthology.orchestra.run.smithi188.stdout:(112/140): libcom_err-devel-1.45.6-4.el8.x86_64 329 kB/s | 39 kB 00:00 2024-01-27T02:29:43.361 INFO:teuthology.orchestra.run.smithi007.stdout:(38/140): liboath-2.6.2-3.el8.x86_64.rpm 133 kB/s | 59 kB 00:00 2024-01-27T02:29:43.415 INFO:teuthology.orchestra.run.smithi188.stdout:(113/140): fuse-2.9.7-17.el8.x86_64.rpm 584 kB/s | 83 kB 00:00 2024-01-27T02:29:43.449 INFO:teuthology.orchestra.run.smithi188.stdout:(114/140): libgfortran-8.5.0-20.el8.x86_64.rpm 4.0 MB/s | 645 kB 00:00 2024-01-27T02:29:43.508 INFO:teuthology.orchestra.run.smithi188.stdout:(115/140): libquadmath-8.5.0-20.el8.x86_64.rpm 1.0 MB/s | 172 kB 00:00 2024-01-27T02:29:43.533 INFO:teuthology.orchestra.run.smithi188.stdout:(116/140): libuuid-devel-2.32.1-43.el8.x86_64.r 851 kB/s | 100 kB 00:00 2024-01-27T02:29:43.608 INFO:teuthology.orchestra.run.smithi188.stdout:(117/140): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2024-01-27T02:29:43.642 INFO:teuthology.orchestra.run.smithi188.stdout:(118/140): xfsprogs-devel-5.0.0-12.el8.x86_64.r 669 kB/s | 73 kB 00:00 2024-01-27T02:29:43.701 INFO:teuthology.orchestra.run.smithi188.stdout:(119/140): python3-cryptography-3.2.1-6.el8.x86 2.8 MB/s | 559 kB 00:00 2024-01-27T02:29:43.726 INFO:teuthology.orchestra.run.smithi188.stdout:(120/140): elfutils-libelf-devel-0.189-3.el8.x8 527 kB/s | 62 kB 00:00 2024-01-27T02:29:43.751 INFO:teuthology.orchestra.run.smithi188.stdout:(121/140): librabbitmq-0.9.0-4.el8.x86_64.rpm 431 kB/s | 47 kB 00:00 2024-01-27T02:29:43.860 INFO:teuthology.orchestra.run.smithi188.stdout:(122/140): krb5-devel-1.18.2-26.el8_9.x86_64.rp 3.5 MB/s | 562 kB 00:00 2024-01-27T02:29:43.895 INFO:teuthology.orchestra.run.smithi007.stdout:(39/140): ceph-test-17.2.7-472.g74950076.el8.x8 15 MB/s | 52 MB 00:03 2024-01-27T02:29:43.920 INFO:teuthology.orchestra.run.smithi007.stdout:(40/140): libunwind-1.3.1-3.el8.x86_64.rpm 134 kB/s | 75 kB 00:00 2024-01-27T02:29:43.945 INFO:teuthology.orchestra.run.smithi007.stdout:(41/140): gperftools-libs-2.7-9.el8.x86_64.rpm 289 kB/s | 306 kB 00:01 2024-01-27T02:29:43.952 INFO:teuthology.orchestra.run.smithi188.stdout:(123/140): openssl-devel-1.1.1k-12.el8_9.x86_64 10 MB/s | 2.3 MB 00:00 2024-01-27T02:29:43.977 INFO:teuthology.orchestra.run.smithi188.stdout:(124/140): NetworkManager-initscripts-updown-1. 639 kB/s | 144 kB 00:00 2024-01-27T02:29:44.037 INFO:teuthology.orchestra.run.smithi007.stdout:(42/140): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 373 kB/s | 44 kB 00:00 2024-01-27T02:29:44.113 INFO:teuthology.orchestra.run.smithi007.stdout:(43/140): python3-beautifulsoup4-4.6.3-2.el8.1. 1.1 MB/s | 185 kB 00:00 2024-01-27T02:29:44.127 INFO:teuthology.orchestra.run.smithi188.stdout:(125/140): elfutils-libelf-0.189-3.el8.x86_64.r 1.5 MB/s | 232 kB 00:00 2024-01-27T02:29:44.154 INFO:teuthology.orchestra.run.smithi007.stdout:(44/140): python3-cachetools-3.1.1-4.el8.noarch 281 kB/s | 33 kB 00:00 2024-01-27T02:29:44.186 INFO:teuthology.orchestra.run.smithi188.stdout:(126/140): elfutils-libs-0.189-3.el8.x86_64.rpm 5.1 MB/s | 303 kB 00:00 2024-01-27T02:29:44.213 INFO:teuthology.orchestra.run.smithi007.stdout:(45/140): python3-certifi-2018.10.15-7.el8.noar 163 kB/s | 16 kB 00:00 2024-01-27T02:29:44.286 INFO:teuthology.orchestra.run.smithi188.stdout:(127/140): librados2-17.2.7-472.g74950076.el8.x 8.1 MB/s | 3.5 MB 00:00 2024-01-27T02:29:44.322 INFO:teuthology.orchestra.run.smithi007.stdout:(46/140): python3-cheroot-8.5.2-1.el8.noarch.rp 1.0 MB/s | 173 kB 00:00 2024-01-27T02:29:44.336 INFO:teuthology.orchestra.run.smithi188.stdout:(128/140): libmount-2.32.1-43.el8.x86_64.rpm 1.5 MB/s | 237 kB 00:00 2024-01-27T02:29:44.397 INFO:teuthology.orchestra.run.smithi007.stdout:(47/140): python3-cherrypy-18.4.0-1.el8.noarch. 2.0 MB/s | 384 kB 00:00 2024-01-27T02:29:44.403 INFO:teuthology.orchestra.run.smithi188.stdout:(129/140): libsmartcols-2.32.1-43.el8.x86_64.rp 1.5 MB/s | 179 kB 00:00 2024-01-27T02:29:44.447 INFO:teuthology.orchestra.run.smithi007.stdout:(48/140): python3-google-auth-1.1.1-10.el8.noar 649 kB/s | 81 kB 00:00 2024-01-27T02:29:44.462 INFO:teuthology.orchestra.run.smithi188.stdout:(130/140): librbd1-17.2.7-472.g74950076.el8.x86 7.0 MB/s | 3.6 MB 00:00 2024-01-27T02:29:44.487 INFO:teuthology.orchestra.run.smithi188.stdout:(131/140): libuuid-2.32.1-43.el8.x86_64.rpm 658 kB/s | 99 kB 00:00 2024-01-27T02:29:44.498 INFO:teuthology.orchestra.run.smithi007.stdout:(49/140): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2024-01-27T02:29:44.556 INFO:teuthology.orchestra.run.smithi007.stdout:(50/140): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2024-01-27T02:29:44.581 INFO:teuthology.orchestra.run.smithi007.stdout:(51/140): python3-asyncssh-2.7.0-2.el8.noarch.r 693 kB/s | 476 kB 00:00 2024-01-27T02:29:44.629 INFO:teuthology.orchestra.run.smithi188.stdout:(132/140): util-linux-2.32.1-43.el8.x86_64.rpm 11 MB/s | 2.5 MB 00:00 2024-01-27T02:29:44.655 INFO:teuthology.orchestra.run.smithi188.stdout:(133/140): util-linux-user-2.32.1-43.el8.x86_64 534 kB/s | 103 kB 00:00 2024-01-27T02:29:44.673 INFO:teuthology.orchestra.run.smithi007.stdout:(52/140): python3-logutils-0.3.5-11.el8.noarch. 416 kB/s | 49 kB 00:00 2024-01-27T02:29:44.696 INFO:teuthology.orchestra.run.smithi188.stdout:(134/140): iproute-6.2.0-5.el8_9.x86_64.rpm 4.0 MB/s | 854 kB 00:00 2024-01-27T02:29:44.698 INFO:teuthology.orchestra.run.smithi007.stdout:(53/140): python3-more-itertools-7.2.0-3.el8.no 502 kB/s | 59 kB 00:00 2024-01-27T02:29:44.766 INFO:teuthology.orchestra.run.smithi007.stdout:(54/140): python3-kubernetes-11.0.0-6.el8.noarc 4.4 MB/s | 1.2 MB 00:00 2024-01-27T02:29:44.791 INFO:teuthology.orchestra.run.smithi007.stdout:(55/140): python3-natsort-7.1.1-2.el8.noarch.rp 514 kB/s | 60 kB 00:00 2024-01-27T02:29:44.833 INFO:teuthology.orchestra.run.smithi007.stdout:(56/140): python3-pecan-1.3.2-9.el8.noarch.rpm 2.1 MB/s | 283 kB 00:00 2024-01-27T02:29:44.891 INFO:teuthology.orchestra.run.smithi007.stdout:(57/140): python3-portend-2.6-1.el8.noarch.rpm 124 kB/s | 16 kB 00:00 2024-01-27T02:29:44.917 INFO:teuthology.orchestra.run.smithi007.stdout:(58/140): python3-repoze-lru-0.7-6.el8.noarch.r 268 kB/s | 34 kB 00:00 2024-01-27T02:29:44.958 INFO:teuthology.orchestra.run.smithi007.stdout:(59/140): python3-routes-2.4.1-12.el8.noarch.rp 1.5 MB/s | 196 kB 00:00 2024-01-27T02:29:45.000 INFO:teuthology.orchestra.run.smithi007.stdout:(60/140): python3-rsa-4.9-2.el8.noarch.rpm 590 kB/s | 64 kB 00:00 2024-01-27T02:29:45.005 INFO:teuthology.orchestra.run.smithi188.stdout:(135/140): libblkid-2.32.1-43.el8.x86_64.rpm 587 kB/s | 220 kB 00:00 2024-01-27T02:29:45.026 INFO:teuthology.orchestra.run.smithi007.stdout:(61/140): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2024-01-27T02:29:45.059 INFO:teuthology.orchestra.run.smithi007.stdout:(62/140): python3-singledispatch-3.4.0.3-18.el8 238 kB/s | 24 kB 00:00 2024-01-27T02:29:45.101 INFO:teuthology.orchestra.run.smithi007.stdout:(63/140): python3-tempora-1.14.1-5.el8.noarch.r 291 kB/s | 29 kB 00:00 2024-01-27T02:29:45.135 INFO:teuthology.orchestra.run.smithi007.stdout:(64/140): python3-trustme-0.6.0-4.el8.noarch.rp 253 kB/s | 27 kB 00:00 2024-01-27T02:29:45.193 INFO:teuthology.orchestra.run.smithi007.stdout:(65/140): python3-waitress-1.4.3-1.el8.noarch.r 1.9 MB/s | 255 kB 00:00 2024-01-27T02:29:45.219 INFO:teuthology.orchestra.run.smithi007.stdout:(66/140): python3-webob-1.8.5-1.el8.1.noarch.rp 2.1 MB/s | 251 kB 00:00 2024-01-27T02:29:45.252 INFO:teuthology.orchestra.run.smithi007.stdout:(67/140): python3-websocket-client-0.56.0-5.el8 522 kB/s | 61 kB 00:00 2024-01-27T02:29:45.264 INFO:teuthology.orchestra.run.smithi188.stdout:(136/140): libfdisk-2.32.1-43.el8.x86_64.rpm 417 kB/s | 254 kB 00:00 2024-01-27T02:29:45.303 INFO:teuthology.orchestra.run.smithi007.stdout:(68/140): python3-webtest-2.0.33-1.el8.noarch.r 784 kB/s | 85 kB 00:00 2024-01-27T02:29:45.306 INFO:teuthology.orchestra.run.smithi188.stdout:(137/140): openssl-1.1.1k-12.el8_9.x86_64.rpm 2.3 MB/s | 711 kB 00:00 2024-01-27T02:29:45.328 INFO:teuthology.orchestra.run.smithi007.stdout:(69/140): python3-zc-lockfile-2.0-2.el8.noarch. 211 kB/s | 23 kB 00:00 2024-01-27T02:29:45.339 INFO:teuthology.orchestra.run.smithi188.stdout:(138/140): xfsprogs-5.0.0-12.el8.x86_64.rpm 1.7 MB/s | 1.1 MB 00:00 2024-01-27T02:29:45.353 INFO:teuthology.orchestra.run.smithi007.stdout:(70/140): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2024-01-27T02:29:45.415 INFO:teuthology.orchestra.run.smithi188.stdout:(139/140): openssl-libs-1.1.1k-12.el8_9.x86_64. 9.8 MB/s | 1.5 MB 00:00 2024-01-27T02:29:45.456 INFO:teuthology.orchestra.run.smithi188.stdout:(140/140): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.2 MB/s | 333 kB 00:00 2024-01-27T02:29:45.460 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-01-27T02:29:45.460 INFO:teuthology.orchestra.run.smithi188.stdout:Total 16 MB/s | 212 MB 00:13 2024-01-27T02:29:45.470 INFO:teuthology.orchestra.run.smithi007.stdout:(71/140): libverto-devel-0.3.0-5.el8.x86_64.rpm 126 kB/s | 18 kB 00:00 2024-01-27T02:29:45.495 INFO:teuthology.orchestra.run.smithi007.stdout:(72/140): python3-jwt-1.6.1-2.el8.noarch.rpm 319 kB/s | 43 kB 00:00 2024-01-27T02:29:45.596 INFO:teuthology.orchestra.run.smithi007.stdout:(73/140): python3-oauthlib-2.1.0-1.el8.noarch.r 1.2 MB/s | 155 kB 00:00 2024-01-27T02:29:45.654 INFO:teuthology.orchestra.run.smithi007.stdout:(74/140): python3-pycparser-2.14-14.el8.noarch. 691 kB/s | 109 kB 00:00 2024-01-27T02:29:45.697 INFO:teuthology.orchestra.run.smithi007.stdout:(75/140): thrift-0.13.0-2.el8.x86_64.rpm 3.9 MB/s | 1.7 MB 00:00 2024-01-27T02:29:45.713 INFO:teuthology.orchestra.run.smithi007.stdout:(76/140): python3-requests-oauthlib-1.0.0-1.el8 363 kB/s | 43 kB 00:00 2024-01-27T02:29:45.789 INFO:teuthology.orchestra.run.smithi007.stdout:(77/140): libaio-devel-0.3.112-1.el8.x86_64.rpm 138 kB/s | 19 kB 00:00 2024-01-27T02:29:45.839 INFO:teuthology.orchestra.run.smithi007.stdout:(78/140): libzstd-devel-1.4.4-1.el8.x86_64.rpm 308 kB/s | 44 kB 00:00 2024-01-27T02:29:45.889 INFO:teuthology.orchestra.run.smithi007.stdout:(79/140): pcre2-devel-10.32-2.el8.x86_64.rpm 3.5 MB/s | 605 kB 00:00 2024-01-27T02:29:45.931 INFO:teuthology.orchestra.run.smithi007.stdout:(80/140): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.6 MB/s | 229 kB 00:00 2024-01-27T02:29:46.023 INFO:teuthology.orchestra.run.smithi007.stdout:(81/140): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.2 MB/s | 220 kB 00:00 2024-01-27T02:29:46.049 INFO:teuthology.orchestra.run.smithi007.stdout:(82/140): libselinux-devel-2.9-5.el8.x86_64.rpm 1.2 MB/s | 200 kB 00:00 2024-01-27T02:29:46.107 INFO:teuthology.orchestra.run.smithi007.stdout:(83/140): keyutils-libs-devel-1.5.10-9.el8.x86_ 274 kB/s | 48 kB 00:00 2024-01-27T02:29:46.157 INFO:teuthology.orchestra.run.smithi007.stdout:(84/140): libsepol-devel-2.9-3.el8.x86_64.rpm 650 kB/s | 87 kB 00:00 2024-01-27T02:29:46.174 INFO:teuthology.orchestra.run.smithi007.stdout:(85/140): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 459 kB/s | 58 kB 00:00 2024-01-27T02:29:46.258 INFO:teuthology.orchestra.run.smithi007.stdout:(86/140): fuse-2.9.7-17.el8.x86_64.rpm 828 kB/s | 83 kB 00:00 2024-01-27T02:29:46.275 INFO:teuthology.orchestra.run.smithi007.stdout:(87/140): libcom_err-devel-1.45.6-4.el8.x86_64. 230 kB/s | 39 kB 00:00 2024-01-27T02:29:46.358 INFO:teuthology.orchestra.run.smithi007.stdout:(88/140): libgfortran-8.5.0-20.el8.x86_64.rpm 3.4 MB/s | 645 kB 00:00 2024-01-27T02:29:46.384 INFO:teuthology.orchestra.run.smithi007.stdout:(89/140): libquadmath-8.5.0-20.el8.x86_64.rpm 1.3 MB/s | 172 kB 00:00 2024-01-27T02:29:46.396 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-01-27T02:29:46.426 INFO:teuthology.orchestra.run.smithi007.stdout:(90/140): libuuid-devel-2.32.1-43.el8.x86_64.rp 662 kB/s | 100 kB 00:00 2024-01-27T02:29:46.484 INFO:teuthology.orchestra.run.smithi007.stdout:(91/140): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.9 MB/s | 238 kB 00:00 2024-01-27T02:29:46.509 INFO:teuthology.orchestra.run.smithi007.stdout:(92/140): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 868 kB/s | 73 kB 00:00 2024-01-27T02:29:46.576 INFO:teuthology.orchestra.run.smithi007.stdout:(93/140): python3-cryptography-3.2.1-6.el8.x86_ 2.8 MB/s | 559 kB 00:00 2024-01-27T02:29:46.602 INFO:teuthology.orchestra.run.smithi007.stdout:(94/140): elfutils-libelf-devel-0.189-3.el8.x86 526 kB/s | 62 kB 00:00 2024-01-27T02:29:46.643 INFO:teuthology.orchestra.run.smithi007.stdout:(95/140): librabbitmq-0.9.0-4.el8.x86_64.rpm 351 kB/s | 47 kB 00:00 2024-01-27T02:29:46.735 INFO:teuthology.orchestra.run.smithi007.stdout:(96/140): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 3.5 MB/s | 562 kB 00:00 2024-01-27T02:29:46.761 INFO:teuthology.orchestra.run.smithi007.stdout:(97/140): NetworkManager-initscripts-updown-1.4 1.2 MB/s | 144 kB 00:00 2024-01-27T02:29:46.788 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-01-27T02:29:46.788 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-01-27T02:29:46.819 INFO:teuthology.orchestra.run.smithi007.stdout:(98/140): openssl-devel-1.1.1k-12.el8_9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-01-27T02:29:46.886 INFO:teuthology.orchestra.run.smithi007.stdout:(99/140): python3-jsonpointer-1.10-11.el8.noarc 133 kB/s | 20 kB 00:00 2024-01-27T02:29:46.911 INFO:teuthology.orchestra.run.smithi007.stdout:(100/140): python3-prettytable-0.7.2-14.el8.noa 292 kB/s | 44 kB 00:00 2024-01-27T02:29:46.928 INFO:teuthology.orchestra.run.smithi007.stdout:(101/140): python3-pyasn1-modules-0.3.7-6.el8.n 1.0 MB/s | 110 kB 00:00 2024-01-27T02:29:47.020 INFO:teuthology.orchestra.run.smithi007.stdout:(102/140): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.1 MB/s | 126 kB 00:00 2024-01-27T02:29:47.112 INFO:teuthology.orchestra.run.smithi007.stdout:(103/140): python3-jsonpatch-1.21-2.el8.noarch. 117 kB/s | 27 kB 00:00 2024-01-27T02:29:47.321 INFO:teuthology.orchestra.run.smithi007.stdout:(104/140): python3-werkzeug-0.12.2-4.el8.noarch 1.1 MB/s | 457 kB 00:00 2024-01-27T02:29:47.355 INFO:teuthology.orchestra.run.smithi007.stdout:(105/140): bison-3.0.4-10.el8.x86_64.rpm 2.0 MB/s | 688 kB 00:00 2024-01-27T02:29:47.372 INFO:teuthology.orchestra.run.smithi007.stdout:(106/140): flex-2.6.1-9.el8.x86_64.rpm 1.2 MB/s | 320 kB 00:00 2024-01-27T02:29:47.405 INFO:teuthology.orchestra.run.smithi007.stdout:(107/140): libtool-2.4.6-25.el8.x86_64.rpm 8.3 MB/s | 709 kB 00:00 2024-01-27T02:29:47.464 INFO:teuthology.orchestra.run.smithi007.stdout:(108/140): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.8 MB/s | 259 kB 00:00 2024-01-27T02:29:47.489 INFO:teuthology.orchestra.run.smithi007.stdout:(109/140): python3-markupsafe-0.23-19.el8.x86_6 290 kB/s | 39 kB 00:00 2024-01-27T02:29:47.539 INFO:teuthology.orchestra.run.smithi007.stdout:(110/140): oniguruma-6.8.2-2.el8.x86_64.rpm 1.4 MB/s | 187 kB 00:00 2024-01-27T02:29:47.573 INFO:teuthology.orchestra.run.smithi007.stdout:(111/140): python3-pyOpenSSL-19.0.0-1.el8.noarc 951 kB/s | 103 kB 00:00 2024-01-27T02:29:47.690 INFO:teuthology.orchestra.run.smithi007.stdout:(112/140): python3-babel-2.5.1-7.el8.noarch.rpm 24 MB/s | 4.8 MB 00:00 2024-01-27T02:29:47.724 INFO:teuthology.orchestra.run.smithi007.stdout:(113/140): python3-jinja2-2.10.1-3.el8.noarch.r 2.9 MB/s | 538 kB 00:00 2024-01-27T02:29:47.807 INFO:teuthology.orchestra.run.smithi007.stdout:(114/140): python3-numpy-1.14.3-10.el8.x86_64.r 16 MB/s | 3.7 MB 00:00 2024-01-27T02:29:47.833 INFO:teuthology.orchestra.run.smithi007.stdout:(115/140): socat-1.7.4.1-1.el8.x86_64.rpm 2.2 MB/s | 323 kB 00:00 2024-01-27T02:29:47.858 INFO:teuthology.orchestra.run.smithi007.stdout:(116/140): python3-numpy-f2py-1.14.3-10.el8.x86 1.6 MB/s | 225 kB 00:00 2024-01-27T02:29:47.956 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-01-27T02:29:47.965 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-01-27T02:29:47.975 INFO:teuthology.orchestra.run.smithi007.stdout:(117/140): librdkafka-0.11.4-3.el8.x86_64.rpm 2.1 MB/s | 354 kB 00:00 2024-01-27T02:29:48.000 INFO:teuthology.orchestra.run.smithi007.stdout:(118/140): python3-mako-1.0.6-14.el8.noarch.rpm 1.1 MB/s | 157 kB 00:00 2024-01-27T02:29:48.100 INFO:teuthology.orchestra.run.smithi007.stdout:(119/140): openblas-threads-0.3.15-6.el8.x86_64 19 MB/s | 5.2 MB 00:00 2024-01-27T02:29:48.167 INFO:teuthology.orchestra.run.smithi007.stdout:(120/140): automake-1.16.1-8.el8.noarch.rpm 4.2 MB/s | 714 kB 00:00 2024-01-27T02:29:48.226 INFO:teuthology.orchestra.run.smithi007.stdout:(121/140): jq-1.6-7.el8.x86_64.rpm 1.6 MB/s | 202 kB 00:00 2024-01-27T02:29:48.301 INFO:teuthology.orchestra.run.smithi007.stdout:(122/140): openblas-0.3.15-6.el8.x86_64.rpm 15 MB/s | 5.0 MB 00:00 2024-01-27T02:29:48.326 INFO:teuthology.orchestra.run.smithi007.stdout:(123/140): python3-pytz-2017.2-11.el8.noarch.rp 341 kB/s | 54 kB 00:00 2024-01-27T02:29:48.668 INFO:teuthology.orchestra.run.smithi007.stdout:(124/140): python3-scipy-1.0.0-21.module+el8.9. 31 MB/s | 14 MB 00:00 2024-01-27T02:29:48.760 INFO:teuthology.orchestra.run.smithi007.stdout:(125/140): librados2-17.2.7-472.g74950076.el8.x 7.6 MB/s | 3.5 MB 00:00 2024-01-27T02:29:48.827 INFO:teuthology.orchestra.run.smithi007.stdout:(126/140): librbd1-17.2.7-472.g74950076.el8.x86 7.2 MB/s | 3.6 MB 00:00 2024-01-27T02:29:48.852 INFO:teuthology.orchestra.run.smithi007.stdout:(127/140): elfutils-libelf-0.189-3.el8.x86_64.r 1.2 MB/s | 232 kB 00:00 2024-01-27T02:29:48.928 INFO:teuthology.orchestra.run.smithi007.stdout:(128/140): elfutils-libs-0.189-3.el8.x86_64.rpm 1.8 MB/s | 303 kB 00:00 2024-01-27T02:29:48.995 INFO:teuthology.orchestra.run.smithi007.stdout:(129/140): libmount-2.32.1-43.el8.x86_64.rpm 1.4 MB/s | 237 kB 00:00 2024-01-27T02:29:49.020 INFO:teuthology.orchestra.run.smithi007.stdout:(130/140): libsmartcols-2.32.1-43.el8.x86_64.rp 1.0 MB/s | 179 kB 00:00 2024-01-27T02:29:49.062 INFO:teuthology.orchestra.run.smithi007.stdout:(131/140): libuuid-2.32.1-43.el8.x86_64.rpm 739 kB/s | 99 kB 00:00 2024-01-27T02:29:49.179 INFO:teuthology.orchestra.run.smithi007.stdout:(132/140): util-linux-2.32.1-43.el8.x86_64.rpm 13 MB/s | 2.5 MB 00:00 2024-01-27T02:29:49.195 INFO:teuthology.orchestra.run.smithi007.stdout:(133/140): util-linux-user-2.32.1-43.el8.x86_64 585 kB/s | 103 kB 00:00 2024-01-27T02:29:49.238 INFO:teuthology.orchestra.run.smithi007.stdout:(134/140): iproute-6.2.0-5.el8_9.x86_64.rpm 4.7 MB/s | 854 kB 00:00 2024-01-27T02:29:49.330 INFO:teuthology.orchestra.run.smithi007.stdout:(135/140): libfdisk-2.32.1-43.el8.x86_64.rpm 1.9 MB/s | 254 kB 00:00 2024-01-27T02:29:49.355 INFO:teuthology.orchestra.run.smithi007.stdout:(136/140): libblkid-2.32.1-43.el8.x86_64.rpm 1.2 MB/s | 220 kB 00:00 2024-01-27T02:29:49.397 INFO:teuthology.orchestra.run.smithi007.stdout:(137/140): xfsprogs-5.0.0-12.el8.x86_64.rpm 6.7 MB/s | 1.1 MB 00:00 2024-01-27T02:29:49.439 INFO:teuthology.orchestra.run.smithi007.stdout:(138/140): openssl-1.1.1k-12.el8_9.x86_64.rpm 6.4 MB/s | 711 kB 00:00 2024-01-27T02:29:49.506 INFO:teuthology.orchestra.run.smithi007.stdout:(139/140): xfsdump-3.1.8-7.el8_9.x86_64.rpm 3.0 MB/s | 333 kB 00:00 2024-01-27T02:29:49.556 INFO:teuthology.orchestra.run.smithi007.stdout:(140/140): openssl-libs-1.1.1k-12.el8_9.x86_64. 7.3 MB/s | 1.5 MB 00:00 2024-01-27T02:29:49.558 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-01-27T02:29:49.559 INFO:teuthology.orchestra.run.smithi007.stdout:Total 15 MB/s | 212 MB 00:13 2024-01-27T02:29:50.588 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-01-27T02:29:51.034 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-01-27T02:29:51.034 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-01-27T02:29:52.314 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-01-27T02:29:52.324 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-01-27T02:29:53.556 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-01-27T02:29:53.690 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-01-27T02:29:54.004 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/156 2024-01-27T02:29:54.079 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/156 2024-01-27T02:29:54.266 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 2/156 2024-01-27T02:29:54.340 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 2/156 2024-01-27T02:29:54.541 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 3/156 2024-01-27T02:29:54.610 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 3/156 2024-01-27T02:29:54.939 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/156 2024-01-27T02:29:55.123 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : thrift-0.13.0-2.el8.x86_64 5/156 2024-01-27T02:29:55.333 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/156 2024-01-27T02:29:55.402 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/156 2024-01-27T02:29:55.920 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 7/156 2024-01-27T02:29:56.116 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/156 2024-01-27T02:29:56.529 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/156 2024-01-27T02:29:56.733 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librados2-2:17.2.7-472.g74950076.el8.x86_64 9/156 2024-01-27T02:29:57.146 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:17.2.7-472.g74950076.el8.x86_64 9/156 2024-01-27T02:29:57.349 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librbd1-2:17.2.7-472.g74950076.el8.x86_64 10/156 2024-01-27T02:29:57.479 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:17.2.7-472.g74950076.el8.x86_64 10/156 2024-01-27T02:29:57.649 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs2-2:17.2.7-472.g74950076.el8.x86_64 11/156 2024-01-27T02:29:57.747 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephfs2-2:17.2.7-472.g74950076.el8.x86_64 11/156 2024-01-27T02:29:57.851 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-01-27T02:29:58.230 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-01-27T02:29:58.418 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rados-2:17.2.7-472.g74950076.el8.x86_64 12/156 2024-01-27T02:29:58.588 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/156 2024-01-27T02:29:58.591 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librgw2-2:17.2.7-472.g74950076.el8.x86_64 13/156 2024-01-27T02:29:58.663 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/156 2024-01-27T02:29:58.781 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librgw2-2:17.2.7-472.g74950076.el8.x86_64 13/156 2024-01-27T02:29:58.828 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 2/156 2024-01-27T02:29:58.909 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 2/156 2024-01-27T02:29:59.023 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/156 2024-01-27T02:29:59.070 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 3/156 2024-01-27T02:29:59.105 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/156 2024-01-27T02:29:59.145 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 3/156 2024-01-27T02:29:59.352 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/156 2024-01-27T02:29:59.474 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/156 2024-01-27T02:29:59.568 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/156 2024-01-27T02:29:59.633 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : thrift-0.13.0-2.el8.x86_64 5/156 2024-01-27T02:29:59.739 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/156 2024-01-27T02:29:59.830 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/156 2024-01-27T02:29:59.957 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephsqlite-2:17.2.7-472.g74950076.el8.x86_64 18/156 2024-01-27T02:29:59.986 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/156 2024-01-27T02:30:00.064 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephsqlite-2:17.2.7-472.g74950076.el8.x86_64 18/156 2024-01-27T02:30:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/156 2024-01-27T02:30:00.199 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libradosstriper1-2:17.2.7-472.g74950076.el8.x86_ 19/156 2024-01-27T02:30:00.294 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/156 2024-01-27T02:30:00.299 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libradosstriper1-2:17.2.7-472.g74950076.el8.x86_ 19/156 2024-01-27T02:30:00.482 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 20/156 2024-01-27T02:30:00.659 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 20/156 2024-01-27T02:30:00.697 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/156 2024-01-27T02:30:00.840 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 21/156 2024-01-27T02:30:00.885 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librados2-2:17.2.7-472.g74950076.el8.x86_64 9/156 2024-01-27T02:30:01.314 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:17.2.7-472.g74950076.el8.x86_64 9/156 2024-01-27T02:30:01.477 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librbd1-2:17.2.7-472.g74950076.el8.x86_64 10/156 2024-01-27T02:30:01.481 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 21/156 2024-01-27T02:30:01.627 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:17.2.7-472.g74950076.el8.x86_64 10/156 2024-01-27T02:30:01.690 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 22/156 2024-01-27T02:30:01.811 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs2-2:17.2.7-472.g74950076.el8.x86_64 11/156 2024-01-27T02:30:01.918 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs2-2:17.2.7-472.g74950076.el8.x86_64 11/156 2024-01-27T02:30:02.296 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 22/156 2024-01-27T02:30:02.489 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-0.3.15-6.el8.x86_64 23/156 2024-01-27T02:30:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rados-2:17.2.7-472.g74950076.el8.x86_64 12/156 2024-01-27T02:30:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librgw2-2:17.2.7-472.g74950076.el8.x86_64 13/156 2024-01-27T02:30:02.905 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librgw2-2:17.2.7-472.g74950076.el8.x86_64 13/156 2024-01-27T02:30:03.127 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/156 2024-01-27T02:30:03.162 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 23/156 2024-01-27T02:30:03.217 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/156 2024-01-27T02:30:03.446 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 24/156 2024-01-27T02:30:03.450 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/156 2024-01-27T02:30:03.637 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 25/156 2024-01-27T02:30:03.675 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/156 2024-01-27T02:30:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/156 2024-01-27T02:30:03.826 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 26/156 2024-01-27T02:30:03.940 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephsqlite-2:17.2.7-472.g74950076.el8.x86_64 18/156 2024-01-27T02:30:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephsqlite-2:17.2.7-472.g74950076.el8.x86_64 18/156 2024-01-27T02:30:04.143 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 27/156 2024-01-27T02:30:04.177 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libradosstriper1-2:17.2.7-472.g74950076.el8.x86_ 19/156 2024-01-27T02:30:04.261 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libradosstriper1-2:17.2.7-472.g74950076.el8.x86_ 19/156 2024-01-27T02:30:04.409 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/156 2024-01-27T02:30:04.448 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 28/156 2024-01-27T02:30:04.701 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 21/156 2024-01-27T02:30:04.733 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 29/156 2024-01-27T02:30:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/156 2024-01-27T02:30:04.944 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-common-2:17.2.7-472.g74950076.el8.x 30/156 2024-01-27T02:30:05.052 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 23/156 2024-01-27T02:30:05.117 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-argparse-2:17.2.7-472.g74950076.el8 31/156 2024-01-27T02:30:05.194 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 23/156 2024-01-27T02:30:05.343 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 24/156 2024-01-27T02:30:05.389 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cephfs-2:17.2.7-472.g74950076.el8.x86_64 32/156 2024-01-27T02:30:06.039 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 24/156 2024-01-27T02:30:06.193 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 25/156 2024-01-27T02:30:06.339 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 33/156 2024-01-27T02:30:06.686 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 34/156 2024-01-27T02:30:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 25/156 2024-01-27T02:30:06.968 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 35/156 2024-01-27T02:30:07.010 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-0.3.15-6.el8.x86_64 26/156 2024-01-27T02:30:07.194 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/156 2024-01-27T02:30:07.783 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 26/156 2024-01-27T02:30:08.068 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 27/156 2024-01-27T02:30:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 28/156 2024-01-27T02:30:08.520 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 29/156 2024-01-27T02:30:08.761 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-common-2:17.2.7-472.g74950076.el8.x 30/156 2024-01-27T02:30:08.915 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-argparse-2:17.2.7-472.g74950076.el8 31/156 2024-01-27T02:30:08.943 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/156 2024-01-27T02:30:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cephfs-2:17.2.7-472.g74950076.el8.x86_64 32/156 2024-01-27T02:30:09.297 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 37/156 2024-01-27T02:30:09.363 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/156 2024-01-27T02:30:09.533 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rsa-4.9-2.el8.noarch 38/156 2024-01-27T02:30:09.757 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 39/156 2024-01-27T02:30:10.037 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rgw-2:17.2.7-472.g74950076.el8.x86_64 40/156 2024-01-27T02:30:10.242 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rbd-2:17.2.7-472.g74950076.el8.x86_64 41/156 2024-01-27T02:30:10.477 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librados-devel-2:17.2.7-472.g74950076.el8.x86_64 42/156 2024-01-27T02:30:10.704 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/156 2024-01-27T02:30:10.786 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/156 2024-01-27T02:30:10.988 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/156 2024-01-27T02:30:11.087 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/156 2024-01-27T02:30:11.302 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/156 2024-01-27T02:30:11.338 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/156 2024-01-27T02:30:11.574 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/156 2024-01-27T02:30:11.700 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 34/156 2024-01-27T02:30:11.814 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/156 2024-01-27T02:30:12.259 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/156 2024-01-27T02:30:12.646 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/156 2024-01-27T02:30:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/156 2024-01-27T02:30:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 35/156 2024-01-27T02:30:12.916 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/156 2024-01-27T02:30:13.151 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/156 2024-01-27T02:30:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 36/156 2024-01-27T02:30:13.342 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/156 2024-01-27T02:30:13.484 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 37/156 2024-01-27T02:30:13.565 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/156 2024-01-27T02:30:13.713 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rsa-4.9-2.el8.noarch 38/156 2024-01-27T02:30:13.816 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/156 2024-01-27T02:30:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 39/156 2024-01-27T02:30:14.098 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rgw-2:17.2.7-472.g74950076.el8.x86_64 40/156 2024-01-27T02:30:14.142 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/156 2024-01-27T02:30:14.315 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rbd-2:17.2.7-472.g74950076.el8.x86_64 41/156 2024-01-27T02:30:14.419 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/156 2024-01-27T02:30:14.492 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librados-devel-2:17.2.7-472.g74950076.el8.x86_64 42/156 2024-01-27T02:30:14.659 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/156 2024-01-27T02:30:14.683 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/156 2024-01-27T02:30:14.769 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/156 2024-01-27T02:30:14.842 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/156 2024-01-27T02:30:15.000 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/156 2024-01-27T02:30:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/156 2024-01-27T02:30:15.108 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/156 2024-01-27T02:30:15.294 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/156 2024-01-27T02:30:15.454 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/156 2024-01-27T02:30:15.505 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/156 2024-01-27T02:30:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/156 2024-01-27T02:30:15.756 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 60/156 2024-01-27T02:30:15.980 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/156 2024-01-27T02:30:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/156 2024-01-27T02:30:16.285 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 62/156 2024-01-27T02:30:16.567 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/156 2024-01-27T02:30:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/156 2024-01-27T02:30:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : automake-1.16.1-8.el8.noarch 49/156 2024-01-27T02:30:16.973 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 63/156 2024-01-27T02:30:17.041 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 50/156 2024-01-27T02:30:17.126 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 50/156 2024-01-27T02:30:17.343 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : jq-1.6-7.el8.x86_64 51/156 2024-01-27T02:30:17.351 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 64/156 2024-01-27T02:30:17.483 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 52/156 2024-01-27T02:30:17.611 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 65/156 2024-01-27T02:30:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/156 2024-01-27T02:30:17.803 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/156 2024-01-27T02:30:17.820 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 66/156 2024-01-27T02:30:18.005 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : fuse-2.9.7-17.el8.x86_64 55/156 2024-01-27T02:30:18.021 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/156 2024-01-27T02:30:18.244 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 56/156 2024-01-27T02:30:18.271 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/156 2024-01-27T02:30:18.416 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 57/156 2024-01-27T02:30:18.605 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 58/156 2024-01-27T02:30:18.620 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/156 2024-01-27T02:30:18.781 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 59/156 2024-01-27T02:30:18.880 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : automake-1.16.1-8.el8.noarch 70/156 2024-01-27T02:30:19.044 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 71/156 2024-01-27T02:30:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 60/156 2024-01-27T02:30:19.133 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 71/156 2024-01-27T02:30:19.337 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : jq-1.6-7.el8.x86_64 72/156 2024-01-27T02:30:19.386 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 61/156 2024-01-27T02:30:19.543 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 73/156 2024-01-27T02:30:19.631 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 62/156 2024-01-27T02:30:19.736 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 74/156 2024-01-27T02:30:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 63/156 2024-01-27T02:30:19.946 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 75/156 2024-01-27T02:30:20.002 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 64/156 2024-01-27T02:30:20.136 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 76/156 2024-01-27T02:30:20.381 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 77/156 2024-01-27T02:30:20.387 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 65/156 2024-01-27T02:30:20.667 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 78/156 2024-01-27T02:30:20.697 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 66/156 2024-01-27T02:30:20.882 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 67/156 2024-01-27T02:30:20.901 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 79/156 2024-01-27T02:30:21.056 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 80/156 2024-01-27T02:30:21.243 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 68/156 2024-01-27T02:30:21.272 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 81/156 2024-01-27T02:30:21.495 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 82/156 2024-01-27T02:30:21.501 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 69/156 2024-01-27T02:30:21.757 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 83/156 2024-01-27T02:30:21.829 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 70/156 2024-01-27T02:30:21.997 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 84/156 2024-01-27T02:30:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 71/156 2024-01-27T02:30:22.197 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 85/156 2024-01-27T02:30:22.202 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 72/156 2024-01-27T02:30:22.394 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 73/156 2024-01-27T02:30:22.399 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 86/156 2024-01-27T02:30:22.590 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 74/156 2024-01-27T02:30:22.647 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 87/156 2024-01-27T02:30:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 75/156 2024-01-27T02:30:22.878 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 88/156 2024-01-27T02:30:23.001 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 76/156 2024-01-27T02:30:23.091 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 89/156 2024-01-27T02:30:23.193 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 77/156 2024-01-27T02:30:23.376 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-portend-2.6-1.el8.noarch 90/156 2024-01-27T02:30:23.455 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 78/156 2024-01-27T02:30:23.634 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 91/156 2024-01-27T02:30:23.682 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 79/156 2024-01-27T02:30:23.808 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 92/156 2024-01-27T02:30:23.846 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 80/156 2024-01-27T02:30:24.037 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 81/156 2024-01-27T02:30:24.049 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 93/156 2024-01-27T02:30:24.246 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 82/156 2024-01-27T02:30:24.573 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 83/156 2024-01-27T02:30:24.770 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 84/156 2024-01-27T02:30:24.961 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 85/156 2024-01-27T02:30:25.140 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 94/156 2024-01-27T02:30:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 86/156 2024-01-27T02:30:25.376 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 87/156 2024-01-27T02:30:25.480 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 95/156 2024-01-27T02:30:25.581 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 88/156 2024-01-27T02:30:25.734 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 96/156 2024-01-27T02:30:25.738 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 89/156 2024-01-27T02:30:26.009 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-portend-2.6-1.el8.noarch 90/156 2024-01-27T02:30:26.043 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 97/156 2024-01-27T02:30:26.224 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 91/156 2024-01-27T02:30:26.334 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 98/156 2024-01-27T02:30:26.416 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 92/156 2024-01-27T02:30:26.542 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 99/156 2024-01-27T02:30:26.644 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 93/156 2024-01-27T02:30:26.760 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 100/156 2024-01-27T02:30:27.814 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 94/156 2024-01-27T02:30:28.169 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 95/156 2024-01-27T02:30:28.370 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 96/156 2024-01-27T02:30:28.699 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 97/156 2024-01-27T02:30:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 98/156 2024-01-27T02:30:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 99/156 2024-01-27T02:30:29.865 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 100/156 2024-01-27T02:30:31.204 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:17.2.7-472.g74950076.el8.x86_64 101/156 2024-01-27T02:30:31.434 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-common-2:17.2.7-472.g74950076.el8.x86_64 101/156 2024-01-27T02:30:31.993 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:17.2.7-472.g74950076.el8.x86_64 101/156 2024-01-27T02:30:32.296 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-base-2:17.2.7-472.g74950076.el8.x86_64 102/156 2024-01-27T02:30:32.376 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-base-2:17.2.7-472.g74950076.el8.x86_64 102/156 2024-01-27T02:30:32.558 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-selinux-2:17.2.7-472.g74950076.el8.x86_64 103/156 2024-01-27T02:30:34.109 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:17.2.7-472.g74950076.el8.x86_64 101/156 2024-01-27T02:30:34.360 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-common-2:17.2.7-472.g74950076.el8.x86_64 101/156 2024-01-27T02:30:34.982 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:17.2.7-472.g74950076.el8.x86_64 101/156 2024-01-27T02:30:35.524 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-base-2:17.2.7-472.g74950076.el8.x86_64 102/156 2024-01-27T02:30:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-base-2:17.2.7-472.g74950076.el8.x86_64 102/156 2024-01-27T02:30:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-selinux-2:17.2.7-472.g74950076.el8.x86_64 103/156 2024-01-27T02:30:52.227 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-selinux-2:17.2.7-472.g74950076.el8.x86_64 103/156 2024-01-27T02:30:52.228 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /sys 2024-01-27T02:30:52.228 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /proc 2024-01-27T02:30:52.228 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /dev 2024-01-27T02:30:52.228 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /run 2024-01-27T02:30:52.228 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /mnt 2024-01-27T02:30:52.228 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /var/tmp 2024-01-27T02:30:52.228 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /home 2024-01-27T02:30:52.228 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /tmp 2024-01-27T02:30:52.229 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /dev 2024-01-27T02:30:52.229 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:30:52.620 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-volume-2:17.2.7-472.g74950076.el8.noarch 104/156 2024-01-27T02:30:54.091 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-volume-2:17.2.7-472.g74950076.el8.noarch 104/156 2024-01-27T02:30:54.307 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-osd-2:17.2.7-472.g74950076.el8.x86_64 105/156 2024-01-27T02:30:54.529 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-osd-2:17.2.7-472.g74950076.el8.x86_64 105/156 2024-01-27T02:30:54.711 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mds-2:17.2.7-472.g74950076.el8.x86_64 106/156 2024-01-27T02:30:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mds-2:17.2.7-472.g74950076.el8.x86_64 106/156 2024-01-27T02:30:55.278 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mon-2:17.2.7-472.g74950076.el8.x86_64 107/156 2024-01-27T02:30:55.296 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mon-2:17.2.7-472.g74950076.el8.x86_64 107/156 2024-01-27T02:30:57.100 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephadm-2:17.2.7-472.g74950076.el8.noarch 108/156 2024-01-27T02:30:57.285 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephadm-2:17.2.7-472.g74950076.el8.noarch 108/156 2024-01-27T02:30:57.458 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-472.g74950076.el 109/156 2024-01-27T02:30:57.622 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-472.g74950076.e 110/156 2024-01-27T02:30:57.877 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-472.g74950076.el8.noar 111/156 2024-01-27T02:30:58.034 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-selinux-2:17.2.7-472.g74950076.el8.x86_64 103/156 2024-01-27T02:30:58.035 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /sys 2024-01-27T02:30:58.035 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /proc 2024-01-27T02:30:58.035 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /dev 2024-01-27T02:30:58.035 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /run 2024-01-27T02:30:58.035 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /mnt 2024-01-27T02:30:58.035 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /var/tmp 2024-01-27T02:30:58.035 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /home 2024-01-27T02:30:58.035 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /tmp 2024-01-27T02:30:58.035 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /dev 2024-01-27T02:30:58.035 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:30:58.096 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-472.g74950076.el8.noar 111/156 2024-01-27T02:30:58.389 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-472.g74950076.el8.no 112/156 2024-01-27T02:30:58.427 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-volume-2:17.2.7-472.g74950076.el8.noarch 104/156 2024-01-27T02:30:59.354 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-472.g74950076.el8.no 112/156 2024-01-27T02:30:59.568 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-472.g7495 113/156 2024-01-27T02:30:59.661 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-472.g7495 113/156 2024-01-27T02:31:00.045 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-472.g74950076.el8 114/156 2024-01-27T02:31:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-volume-2:17.2.7-472.g74950076.el8.noarch 104/156 2024-01-27T02:31:00.236 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-osd-2:17.2.7-472.g74950076.el8.x86_64 105/156 2024-01-27T02:31:00.281 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-2:17.2.7-472.g74950076.el8.x86_64 115/156 2024-01-27T02:31:00.318 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-2:17.2.7-472.g74950076.el8.x86_64 115/156 2024-01-27T02:31:00.483 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-osd-2:17.2.7-472.g74950076.el8.x86_64 105/156 2024-01-27T02:31:00.518 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-rook-2:17.2.7-472.g74950076.el8.noarch 116/156 2024-01-27T02:31:00.530 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-472.g74950076.el8.noarch 116/156 2024-01-27T02:31:00.685 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mds-2:17.2.7-472.g74950076.el8.x86_64 106/156 2024-01-27T02:31:01.118 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mds-2:17.2.7-472.g74950076.el8.x86_64 106/156 2024-01-27T02:31:01.268 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mon-2:17.2.7-472.g74950076.el8.x86_64 107/156 2024-01-27T02:31:01.289 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mon-2:17.2.7-472.g74950076.el8.x86_64 107/156 2024-01-27T02:31:02.135 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-2:17.2.7-472.g74950076.el8.x86_64 117/156 2024-01-27T02:31:02.339 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-radosgw-2:17.2.7-472.g74950076.el8.x86_64 118/156 2024-01-27T02:31:02.430 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-472.g74950076.el8.x86_64 118/156 2024-01-27T02:31:02.609 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-472.g749500 119/156 2024-01-27T02:31:02.652 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-472.g749500 119/156 2024-01-27T02:31:02.851 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-mirror-2:17.2.7-472.g74950076.el8.x86_64 120/156 2024-01-27T02:31:03.196 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephfs-mirror-2:17.2.7-472.g74950076.el8.x86_64 120/156 2024-01-27T02:31:03.338 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:17.2.7-472.g74950076.el8.noarch 108/156 2024-01-27T02:31:03.342 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-mirror-2:17.2.7-472.g74950076.el8.x86_64 121/156 2024-01-27T02:31:03.506 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephadm-2:17.2.7-472.g74950076.el8.noarch 108/156 2024-01-27T02:31:03.683 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-472.g74950076.el 109/156 2024-01-27T02:31:03.866 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-472.g74950076.e 110/156 2024-01-27T02:31:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-472.g74950076.el8.noar 111/156 2024-01-27T02:31:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-472.g74950076.el8.noar 111/156 2024-01-27T02:31:04.619 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-472.g74950076.el8.no 112/156 2024-01-27T02:31:05.693 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-472.g74950076.el8.no 112/156 2024-01-27T02:31:05.850 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-472.g7495 113/156 2024-01-27T02:31:05.958 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-472.g7495 113/156 2024-01-27T02:31:06.348 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-472.g74950076.el8 114/156 2024-01-27T02:31:06.511 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-2:17.2.7-472.g74950076.el8.x86_64 115/156 2024-01-27T02:31:06.555 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-2:17.2.7-472.g74950076.el8.x86_64 115/156 2024-01-27T02:31:06.692 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-rook-2:17.2.7-472.g74950076.el8.noarch 116/156 2024-01-27T02:31:06.707 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-472.g74950076.el8.noarch 116/156 2024-01-27T02:31:08.225 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: rbd-mirror-2:17.2.7-472.g74950076.el8.x86_64 121/156 2024-01-27T02:31:08.483 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-2:17.2.7-472.g74950076.el8.x86_64 117/156 2024-01-27T02:31:08.584 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-test-2:17.2.7-472.g74950076.el8.x86_64 122/156 2024-01-27T02:31:08.652 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-radosgw-2:17.2.7-472.g74950076.el8.x86_64 118/156 2024-01-27T02:31:08.754 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-472.g74950076.el8.x86_64 118/156 2024-01-27T02:31:08.766 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libtool-2.4.6-25.el8.x86_64 123/156 2024-01-27T02:31:08.900 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-472.g749500 119/156 2024-01-27T02:31:08.951 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-472.g749500 119/156 2024-01-27T02:31:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-mirror-2:17.2.7-472.g74950076.el8.x86_64 120/156 2024-01-27T02:31:09.454 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 123/156 2024-01-27T02:31:09.528 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephfs-mirror-2:17.2.7-472.g74950076.el8.x86_64 120/156 2024-01-27T02:31:09.666 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-mirror-2:17.2.7-472.g74950076.el8.x86_64 121/156 2024-01-27T02:31:09.762 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 124/156 2024-01-27T02:31:10.058 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 125/156 2024-01-27T02:31:10.263 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-fuse-2:17.2.7-472.g74950076.el8.x86_64 126/156 2024-01-27T02:31:10.493 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 127/156 2024-01-27T02:31:10.703 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 128/156 2024-01-27T02:31:10.906 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs-devel-2:17.2.7-472.g74950076.el8.x86_6 129/156 2024-01-27T02:31:11.249 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 130/156 2024-01-27T02:31:11.640 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 131/156 2024-01-27T02:31:11.887 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 132/156 2024-01-27T02:31:12.107 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-top-2:17.2.7-472.g74950076.el8.noarch 133/156 2024-01-27T02:31:12.357 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-fuse-2:17.2.7-472.g74950076.el8.x86_64 134/156 2024-01-27T02:31:12.638 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-nbd-2:17.2.7-472.g74950076.el8.x86_64 135/156 2024-01-27T02:31:12.867 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 136/156 2024-01-27T02:31:13.082 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 137/156 2024-01-27T02:31:13.092 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 137/156 2024-01-27T02:31:13.278 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 138/156 2024-01-27T02:31:13.473 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flex-2.6.1-9.el8.x86_64 139/156 2024-01-27T02:31:13.554 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 139/156 2024-01-27T02:31:13.806 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : bison-3.0.4-10.el8.x86_64 140/156 2024-01-27T02:31:13.821 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 140/156 2024-01-27T02:31:14.024 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 141/156 2024-01-27T02:31:14.209 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 142/156 2024-01-27T02:31:14.377 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 143/156 2024-01-27T02:31:14.377 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 144/156 2024-01-27T02:31:14.676 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 144/156 2024-01-27T02:31:14.677 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 145/156 2024-01-27T02:31:14.904 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 145/156 2024-01-27T02:31:14.904 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 146/156 2024-01-27T02:31:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-mirror-2:17.2.7-472.g74950076.el8.x86_64 121/156 2024-01-27T02:31:15.196 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 146/156 2024-01-27T02:31:15.433 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 147/156 2024-01-27T02:31:15.624 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 148/156 2024-01-27T02:31:15.916 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 149/156 2024-01-27T02:31:15.917 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 150/156 2024-01-27T02:31:16.201 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 150/156 2024-01-27T02:31:16.201 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 151/156 2024-01-27T02:31:16.265 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-test-2:17.2.7-472.g74950076.el8.x86_64 122/156 2024-01-27T02:31:16.467 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 151/156 2024-01-27T02:31:16.467 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 152/156 2024-01-27T02:31:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 123/156 2024-01-27T02:31:16.717 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 152/156 2024-01-27T02:31:16.717 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 153/156 2024-01-27T02:31:16.758 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 124/156 2024-01-27T02:31:16.934 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 153/156 2024-01-27T02:31:16.934 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 154/156 2024-01-27T02:31:16.993 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-fuse-2:17.2.7-472.g74950076.el8.x86_64 125/156 2024-01-27T02:31:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libtool-2.4.6-25.el8.x86_64 126/156 2024-01-27T02:31:17.208 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 154/156 2024-01-27T02:31:17.216 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 126/156 2024-01-27T02:31:17.400 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 155/156 2024-01-27T02:31:17.401 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 156/156 2024-01-27T02:31:17.421 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 127/156 2024-01-27T02:31:17.611 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 128/156 2024-01-27T02:31:17.808 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs-devel-2:17.2.7-472.g74950076.el8.x86_6 129/156 2024-01-27T02:31:18.120 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 130/156 2024-01-27T02:31:18.329 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 156/156 2024-01-27T02:31:18.330 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-2:17.2.7-472.g74950076.el8.x86_64 1/156 2024-01-27T02:31:18.330 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-base-2:17.2.7-472.g74950076.el8.x86_64 2/156 2024-01-27T02:31:18.330 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-common-2:17.2.7-472.g74950076.el8.x86_64 3/156 2024-01-27T02:31:18.330 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-fuse-2:17.2.7-472.g74950076.el8.x86_64 4/156 2024-01-27T02:31:18.330 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-472.g749500 5/156 2024-01-27T02:31:18.330 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mds-2:17.2.7-472.g74950076.el8.x86_64 6/156 2024-01-27T02:31:18.330 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-2:17.2.7-472.g74950076.el8.x86_64 7/156 2024-01-27T02:31:18.330 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mon-2:17.2.7-472.g74950076.el8.x86_64 8/156 2024-01-27T02:31:18.330 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-osd-2:17.2.7-472.g74950076.el8.x86_64 9/156 2024-01-27T02:31:18.330 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-radosgw-2:17.2.7-472.g74950076.el8.x86_64 10/156 2024-01-27T02:31:18.330 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-selinux-2:17.2.7-472.g74950076.el8.x86_64 11/156 2024-01-27T02:31:18.331 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-test-2:17.2.7-472.g74950076.el8.x86_64 12/156 2024-01-27T02:31:18.331 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-mirror-2:17.2.7-472.g74950076.el8.x86_64 13/156 2024-01-27T02:31:18.331 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs-devel-2:17.2.7-472.g74950076.el8.x86_6 14/156 2024-01-27T02:31:18.331 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs2-2:17.2.7-472.g74950076.el8.x86_64 15/156 2024-01-27T02:31:18.331 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephsqlite-2:17.2.7-472.g74950076.el8.x86_64 16/156 2024-01-27T02:31:18.331 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados-devel-2:17.2.7-472.g74950076.el8.x86_64 17/156 2024-01-27T02:31:18.331 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libradosstriper1-2:17.2.7-472.g74950076.el8.x86_ 18/156 2024-01-27T02:31:18.331 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librgw2-2:17.2.7-472.g74950076.el8.x86_64 19/156 2024-01-27T02:31:18.331 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-argparse-2:17.2.7-472.g74950076.el8 20/156 2024-01-27T02:31:18.331 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-common-2:17.2.7-472.g74950076.el8.x 21/156 2024-01-27T02:31:18.331 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cephfs-2:17.2.7-472.g74950076.el8.x86_64 22/156 2024-01-27T02:31:18.331 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rados-2:17.2.7-472.g74950076.el8.x86_64 23/156 2024-01-27T02:31:18.331 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rbd-2:17.2.7-472.g74950076.el8.x86_64 24/156 2024-01-27T02:31:18.331 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rgw-2:17.2.7-472.g74950076.el8.x86_64 25/156 2024-01-27T02:31:18.332 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-fuse-2:17.2.7-472.g74950076.el8.x86_64 26/156 2024-01-27T02:31:18.332 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-mirror-2:17.2.7-472.g74950076.el8.x86_64 27/156 2024-01-27T02:31:18.332 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-nbd-2:17.2.7-472.g74950076.el8.x86_64 28/156 2024-01-27T02:31:18.344 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-472.g74950076.e 29/156 2024-01-27T02:31:18.344 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-472.g74950076.el8.noar 30/156 2024-01-27T02:31:18.344 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-472.g74950076.el8.no 31/156 2024-01-27T02:31:18.344 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-472.g7495 32/156 2024-01-27T02:31:18.344 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-472.g74950076.el8 33/156 2024-01-27T02:31:18.344 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-rook-2:17.2.7-472.g74950076.el8.noarch 34/156 2024-01-27T02:31:18.344 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-472.g74950076.el 35/156 2024-01-27T02:31:18.344 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-volume-2:17.2.7-472.g74950076.el8.noarch 36/156 2024-01-27T02:31:18.344 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephadm-2:17.2.7-472.g74950076.el8.noarch 37/156 2024-01-27T02:31:18.344 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-top-2:17.2.7-472.g74950076.el8.noarch 38/156 2024-01-27T02:31:18.345 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/156 2024-01-27T02:31:18.345 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/156 2024-01-27T02:31:18.345 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/156 2024-01-27T02:31:18.345 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/156 2024-01-27T02:31:18.345 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/156 2024-01-27T02:31:18.345 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/156 2024-01-27T02:31:18.345 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/156 2024-01-27T02:31:18.345 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/156 2024-01-27T02:31:18.345 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/156 2024-01-27T02:31:18.345 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/156 2024-01-27T02:31:18.345 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/156 2024-01-27T02:31:18.345 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/156 2024-01-27T02:31:18.345 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/156 2024-01-27T02:31:18.345 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/156 2024-01-27T02:31:18.346 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/156 2024-01-27T02:31:18.346 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/156 2024-01-27T02:31:18.346 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/156 2024-01-27T02:31:18.346 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/156 2024-01-27T02:31:18.346 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/156 2024-01-27T02:31:18.346 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/156 2024-01-27T02:31:18.346 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/156 2024-01-27T02:31:18.346 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/156 2024-01-27T02:31:18.346 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/156 2024-01-27T02:31:18.346 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/156 2024-01-27T02:31:18.346 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/156 2024-01-27T02:31:18.346 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/156 2024-01-27T02:31:18.347 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/156 2024-01-27T02:31:18.347 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/156 2024-01-27T02:31:18.347 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/156 2024-01-27T02:31:18.347 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/156 2024-01-27T02:31:18.347 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/156 2024-01-27T02:31:18.347 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/156 2024-01-27T02:31:18.347 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/156 2024-01-27T02:31:18.347 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/156 2024-01-27T02:31:18.347 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/156 2024-01-27T02:31:18.347 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/156 2024-01-27T02:31:18.347 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/156 2024-01-27T02:31:18.347 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/156 2024-01-27T02:31:18.348 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/156 2024-01-27T02:31:18.348 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/156 2024-01-27T02:31:18.348 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/156 2024-01-27T02:31:18.348 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/156 2024-01-27T02:31:18.348 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/156 2024-01-27T02:31:18.348 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/156 2024-01-27T02:31:18.348 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 83/156 2024-01-27T02:31:18.348 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 84/156 2024-01-27T02:31:18.348 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 85/156 2024-01-27T02:31:18.348 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 86/156 2024-01-27T02:31:18.348 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/156 2024-01-27T02:31:18.348 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/156 2024-01-27T02:31:18.349 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/156 2024-01-27T02:31:18.349 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 90/156 2024-01-27T02:31:18.349 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 91/156 2024-01-27T02:31:18.349 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 92/156 2024-01-27T02:31:18.349 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 93/156 2024-01-27T02:31:18.349 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : automake-1.16.1-8.el8.noarch 94/156 2024-01-27T02:31:18.349 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : jq-1.6-7.el8.x86_64 95/156 2024-01-27T02:31:18.349 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 96/156 2024-01-27T02:31:18.349 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 97/156 2024-01-27T02:31:18.349 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 98/156 2024-01-27T02:31:18.349 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 99/156 2024-01-27T02:31:18.349 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 100/156 2024-01-27T02:31:18.349 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 101/156 2024-01-27T02:31:18.349 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 102/156 2024-01-27T02:31:18.350 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 103/156 2024-01-27T02:31:18.350 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 104/156 2024-01-27T02:31:18.350 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 105/156 2024-01-27T02:31:18.350 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 106/156 2024-01-27T02:31:18.350 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 107/156 2024-01-27T02:31:18.350 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 108/156 2024-01-27T02:31:18.350 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 109/156 2024-01-27T02:31:18.350 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 110/156 2024-01-27T02:31:18.350 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 111/156 2024-01-27T02:31:18.350 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 112/156 2024-01-27T02:31:18.350 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 113/156 2024-01-27T02:31:18.350 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 114/156 2024-01-27T02:31:18.350 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 115/156 2024-01-27T02:31:18.350 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 116/156 2024-01-27T02:31:18.351 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 117/156 2024-01-27T02:31:18.351 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 118/156 2024-01-27T02:31:18.351 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 119/156 2024-01-27T02:31:18.351 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 120/156 2024-01-27T02:31:18.351 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 121/156 2024-01-27T02:31:18.351 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 122/156 2024-01-27T02:31:18.351 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 123/156 2024-01-27T02:31:18.351 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 124/156 2024-01-27T02:31:18.351 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:17.2.7-472.g74950076.el8.x86_64 125/156 2024-01-27T02:31:18.351 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 126/156 2024-01-27T02:31:18.351 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:17.2.7-472.g74950076.el8.x86_64 127/156 2024-01-27T02:31:18.351 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 128/156 2024-01-27T02:31:18.351 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 129/156 2024-01-27T02:31:18.352 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 130/156 2024-01-27T02:31:18.352 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 131/156 2024-01-27T02:31:18.352 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 132/156 2024-01-27T02:31:18.352 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 133/156 2024-01-27T02:31:18.352 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 134/156 2024-01-27T02:31:18.352 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 135/156 2024-01-27T02:31:18.352 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 136/156 2024-01-27T02:31:18.352 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 137/156 2024-01-27T02:31:18.353 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 138/156 2024-01-27T02:31:18.353 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 139/156 2024-01-27T02:31:18.353 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 140/156 2024-01-27T02:31:18.353 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 141/156 2024-01-27T02:31:18.353 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 142/156 2024-01-27T02:31:18.353 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 143/156 2024-01-27T02:31:18.353 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 144/156 2024-01-27T02:31:18.353 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 145/156 2024-01-27T02:31:18.353 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 146/156 2024-01-27T02:31:18.353 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 147/156 2024-01-27T02:31:18.353 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 148/156 2024-01-27T02:31:18.353 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 149/156 2024-01-27T02:31:18.353 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 150/156 2024-01-27T02:31:18.353 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 151/156 2024-01-27T02:31:18.354 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 152/156 2024-01-27T02:31:18.354 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 153/156 2024-01-27T02:31:18.354 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 154/156 2024-01-27T02:31:18.354 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 155/156 2024-01-27T02:31:18.443 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 131/156 2024-01-27T02:31:18.654 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 132/156 2024-01-27T02:31:18.849 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-top-2:17.2.7-472.g74950076.el8.noarch 133/156 2024-01-27T02:31:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-fuse-2:17.2.7-472.g74950076.el8.x86_64 134/156 2024-01-27T02:31:19.345 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-nbd-2:17.2.7-472.g74950076.el8.x86_64 135/156 2024-01-27T02:31:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 136/156 2024-01-27T02:31:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flex-2.6.1-9.el8.x86_64 137/156 2024-01-27T02:31:19.878 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 137/156 2024-01-27T02:31:20.090 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : bison-3.0.4-10.el8.x86_64 138/156 2024-01-27T02:31:20.106 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 138/156 2024-01-27T02:31:20.256 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 139/156 2024-01-27T02:31:20.268 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 139/156 2024-01-27T02:31:20.417 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 140/156 2024-01-27T02:31:20.580 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 141/156 2024-01-27T02:31:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 142/156 2024-01-27T02:31:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 143/156 2024-01-27T02:31:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 144/156 2024-01-27T02:31:21.184 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 144/156 2024-01-27T02:31:21.184 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 145/156 2024-01-27T02:31:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 145/156 2024-01-27T02:31:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 146/156 2024-01-27T02:31:21.630 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 146/156 2024-01-27T02:31:21.841 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 147/156 2024-01-27T02:31:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 148/156 2024-01-27T02:31:22.241 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 149/156 2024-01-27T02:31:22.241 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 150/156 2024-01-27T02:31:22.459 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 150/156 2024-01-27T02:31:22.460 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 151/156 2024-01-27T02:31:22.717 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 151/156 2024-01-27T02:31:22.717 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 152/156 2024-01-27T02:31:23.000 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 152/156 2024-01-27T02:31:23.001 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 153/156 2024-01-27T02:31:23.233 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 153/156 2024-01-27T02:31:23.233 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 154/156 2024-01-27T02:31:23.467 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 154/156 2024-01-27T02:31:23.617 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 155/156 2024-01-27T02:31:23.617 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 156/156 2024-01-27T02:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 156/156 2024-01-27T02:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-2:17.2.7-472.g74950076.el8.x86_64 1/156 2024-01-27T02:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-base-2:17.2.7-472.g74950076.el8.x86_64 2/156 2024-01-27T02:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-common-2:17.2.7-472.g74950076.el8.x86_64 3/156 2024-01-27T02:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-fuse-2:17.2.7-472.g74950076.el8.x86_64 4/156 2024-01-27T02:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-472.g749500 5/156 2024-01-27T02:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mds-2:17.2.7-472.g74950076.el8.x86_64 6/156 2024-01-27T02:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-2:17.2.7-472.g74950076.el8.x86_64 7/156 2024-01-27T02:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mon-2:17.2.7-472.g74950076.el8.x86_64 8/156 2024-01-27T02:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-osd-2:17.2.7-472.g74950076.el8.x86_64 9/156 2024-01-27T02:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-radosgw-2:17.2.7-472.g74950076.el8.x86_64 10/156 2024-01-27T02:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-selinux-2:17.2.7-472.g74950076.el8.x86_64 11/156 2024-01-27T02:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-test-2:17.2.7-472.g74950076.el8.x86_64 12/156 2024-01-27T02:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-mirror-2:17.2.7-472.g74950076.el8.x86_64 13/156 2024-01-27T02:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs-devel-2:17.2.7-472.g74950076.el8.x86_6 14/156 2024-01-27T02:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs2-2:17.2.7-472.g74950076.el8.x86_64 15/156 2024-01-27T02:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephsqlite-2:17.2.7-472.g74950076.el8.x86_64 16/156 2024-01-27T02:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados-devel-2:17.2.7-472.g74950076.el8.x86_64 17/156 2024-01-27T02:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libradosstriper1-2:17.2.7-472.g74950076.el8.x86_ 18/156 2024-01-27T02:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librgw2-2:17.2.7-472.g74950076.el8.x86_64 19/156 2024-01-27T02:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-argparse-2:17.2.7-472.g74950076.el8 20/156 2024-01-27T02:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-common-2:17.2.7-472.g74950076.el8.x 21/156 2024-01-27T02:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cephfs-2:17.2.7-472.g74950076.el8.x86_64 22/156 2024-01-27T02:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rados-2:17.2.7-472.g74950076.el8.x86_64 23/156 2024-01-27T02:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rbd-2:17.2.7-472.g74950076.el8.x86_64 24/156 2024-01-27T02:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rgw-2:17.2.7-472.g74950076.el8.x86_64 25/156 2024-01-27T02:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-fuse-2:17.2.7-472.g74950076.el8.x86_64 26/156 2024-01-27T02:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-mirror-2:17.2.7-472.g74950076.el8.x86_64 27/156 2024-01-27T02:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-nbd-2:17.2.7-472.g74950076.el8.x86_64 28/156 2024-01-27T02:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-472.g74950076.e 29/156 2024-01-27T02:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-472.g74950076.el8.noar 30/156 2024-01-27T02:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-472.g74950076.el8.no 31/156 2024-01-27T02:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-472.g7495 32/156 2024-01-27T02:31:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-472.g74950076.el8 33/156 2024-01-27T02:31:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-rook-2:17.2.7-472.g74950076.el8.noarch 34/156 2024-01-27T02:31:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-472.g74950076.el 35/156 2024-01-27T02:31:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-volume-2:17.2.7-472.g74950076.el8.noarch 36/156 2024-01-27T02:31:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephadm-2:17.2.7-472.g74950076.el8.noarch 37/156 2024-01-27T02:31:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-top-2:17.2.7-472.g74950076.el8.noarch 38/156 2024-01-27T02:31:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/156 2024-01-27T02:31:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/156 2024-01-27T02:31:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/156 2024-01-27T02:31:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/156 2024-01-27T02:31:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/156 2024-01-27T02:31:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/156 2024-01-27T02:31:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/156 2024-01-27T02:31:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/156 2024-01-27T02:31:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/156 2024-01-27T02:31:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/156 2024-01-27T02:31:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/156 2024-01-27T02:31:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/156 2024-01-27T02:31:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/156 2024-01-27T02:31:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/156 2024-01-27T02:31:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/156 2024-01-27T02:31:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/156 2024-01-27T02:31:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/156 2024-01-27T02:31:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/156 2024-01-27T02:31:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/156 2024-01-27T02:31:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/156 2024-01-27T02:31:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/156 2024-01-27T02:31:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/156 2024-01-27T02:31:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/156 2024-01-27T02:31:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/156 2024-01-27T02:31:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/156 2024-01-27T02:31:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/156 2024-01-27T02:31:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/156 2024-01-27T02:31:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/156 2024-01-27T02:31:24.638 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/156 2024-01-27T02:31:24.638 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/156 2024-01-27T02:31:24.638 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/156 2024-01-27T02:31:24.638 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/156 2024-01-27T02:31:24.638 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/156 2024-01-27T02:31:24.638 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/156 2024-01-27T02:31:24.638 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/156 2024-01-27T02:31:24.638 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/156 2024-01-27T02:31:24.638 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/156 2024-01-27T02:31:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/156 2024-01-27T02:31:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/156 2024-01-27T02:31:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/156 2024-01-27T02:31:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/156 2024-01-27T02:31:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/156 2024-01-27T02:31:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/156 2024-01-27T02:31:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/156 2024-01-27T02:31:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/156 2024-01-27T02:31:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/156 2024-01-27T02:31:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/156 2024-01-27T02:31:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/156 2024-01-27T02:31:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/156 2024-01-27T02:31:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/156 2024-01-27T02:31:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/156 2024-01-27T02:31:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 90/156 2024-01-27T02:31:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 91/156 2024-01-27T02:31:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 92/156 2024-01-27T02:31:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 93/156 2024-01-27T02:31:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 94/156 2024-01-27T02:31:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 95/156 2024-01-27T02:31:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 96/156 2024-01-27T02:31:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 97/156 2024-01-27T02:31:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 98/156 2024-01-27T02:31:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 99/156 2024-01-27T02:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 100/156 2024-01-27T02:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 101/156 2024-01-27T02:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 102/156 2024-01-27T02:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 103/156 2024-01-27T02:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 104/156 2024-01-27T02:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : bison-3.0.4-10.el8.x86_64 105/156 2024-01-27T02:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flex-2.6.1-9.el8.x86_64 106/156 2024-01-27T02:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 107/156 2024-01-27T02:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 108/156 2024-01-27T02:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 109/156 2024-01-27T02:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 110/156 2024-01-27T02:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 111/156 2024-01-27T02:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 112/156 2024-01-27T02:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 113/156 2024-01-27T02:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 114/156 2024-01-27T02:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 115/156 2024-01-27T02:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 116/156 2024-01-27T02:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 117/156 2024-01-27T02:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 118/156 2024-01-27T02:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 119/156 2024-01-27T02:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 120/156 2024-01-27T02:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : automake-1.16.1-8.el8.noarch 121/156 2024-01-27T02:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : jq-1.6-7.el8.x86_64 122/156 2024-01-27T02:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 123/156 2024-01-27T02:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 124/156 2024-01-27T02:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:17.2.7-472.g74950076.el8.x86_64 125/156 2024-01-27T02:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 126/156 2024-01-27T02:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:17.2.7-472.g74950076.el8.x86_64 127/156 2024-01-27T02:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 128/156 2024-01-27T02:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 129/156 2024-01-27T02:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 130/156 2024-01-27T02:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 131/156 2024-01-27T02:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 132/156 2024-01-27T02:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 133/156 2024-01-27T02:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 134/156 2024-01-27T02:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 135/156 2024-01-27T02:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 136/156 2024-01-27T02:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 137/156 2024-01-27T02:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 138/156 2024-01-27T02:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 139/156 2024-01-27T02:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 140/156 2024-01-27T02:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 141/156 2024-01-27T02:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 142/156 2024-01-27T02:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 143/156 2024-01-27T02:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 144/156 2024-01-27T02:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 145/156 2024-01-27T02:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 146/156 2024-01-27T02:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 147/156 2024-01-27T02:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 148/156 2024-01-27T02:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 149/156 2024-01-27T02:31:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 150/156 2024-01-27T02:31:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 151/156 2024-01-27T02:31:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 152/156 2024-01-27T02:31:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 153/156 2024-01-27T02:31:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 154/156 2024-01-27T02:31:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 155/156 2024-01-27T02:31:29.738 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 156/156 2024-01-27T02:31:29.738 INFO:teuthology.orchestra.run.smithi188.stdout:Installed products updated. 2024-01-27T02:31:30.317 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:30.318 INFO:teuthology.orchestra.run.smithi188.stdout:Upgraded: 2024-01-27T02:31:30.318 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-01-27T02:31:30.318 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-01-27T02:31:30.318 INFO:teuthology.orchestra.run.smithi188.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-01-27T02:31:30.318 INFO:teuthology.orchestra.run.smithi188.stdout: libblkid-2.32.1-43.el8.x86_64 2024-01-27T02:31:30.318 INFO:teuthology.orchestra.run.smithi188.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-01-27T02:31:30.318 INFO:teuthology.orchestra.run.smithi188.stdout: libmount-2.32.1-43.el8.x86_64 2024-01-27T02:31:30.318 INFO:teuthology.orchestra.run.smithi188.stdout: librados2-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.318 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.318 INFO:teuthology.orchestra.run.smithi188.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-01-27T02:31:30.318 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-2.32.1-43.el8.x86_64 2024-01-27T02:31:30.318 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-01-27T02:31:30.318 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-01-27T02:31:30.318 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-2.32.1-43.el8.x86_64 2024-01-27T02:31:30.319 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-01-27T02:31:30.319 INFO:teuthology.orchestra.run.smithi188.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-01-27T02:31:30.319 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-01-27T02:31:30.319 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-01-27T02:31:30.319 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-01-27T02:31:30.319 INFO:teuthology.orchestra.run.smithi188.stdout: automake-1.16.1-8.el8.noarch 2024-01-27T02:31:30.319 INFO:teuthology.orchestra.run.smithi188.stdout: bison-3.0.4-10.el8.x86_64 2024-01-27T02:31:30.319 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.319 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.319 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.319 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.319 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards-2:17.2.7-472.g74950076.el8.noarch 2024-01-27T02:31:30.319 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.319 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.319 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.320 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm-2:17.2.7-472.g74950076.el8.noarch 2024-01-27T02:31:30.320 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard-2:17.2.7-472.g74950076.el8.noarch 2024-01-27T02:31:30.320 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local-2:17.2.7-472.g74950076.el8.noarch 2024-01-27T02:31:30.320 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core-2:17.2.7-472.g74950076.el8.noarch 2024-01-27T02:31:30.320 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook-2:17.2.7-472.g74950076.el8.noarch 2024-01-27T02:31:30.320 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.320 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.320 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts-2:17.2.7-472.g74950076.el8.noarch 2024-01-27T02:31:30.320 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.320 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.320 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.320 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume-2:17.2.7-472.g74950076.el8.noarch 2024-01-27T02:31:30.320 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm-2:17.2.7-472.g74950076.el8.noarch 2024-01-27T02:31:30.321 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.321 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top-2:17.2.7-472.g74950076.el8.noarch 2024-01-27T02:31:30.321 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-01-27T02:31:30.321 INFO:teuthology.orchestra.run.smithi188.stdout: flex-2.6.1-9.el8.x86_64 2024-01-27T02:31:30.321 INFO:teuthology.orchestra.run.smithi188.stdout: fuse-2.9.7-17.el8.x86_64 2024-01-27T02:31:30.321 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-01-27T02:31:30.321 INFO:teuthology.orchestra.run.smithi188.stdout: jq-1.6-7.el8.x86_64 2024-01-27T02:31:30.321 INFO:teuthology.orchestra.run.smithi188.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-01-27T02:31:30.321 INFO:teuthology.orchestra.run.smithi188.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-01-27T02:31:30.321 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-01-27T02:31:30.321 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.321 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.321 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.321 INFO:teuthology.orchestra.run.smithi188.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-01-27T02:31:30.322 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-01-27T02:31:30.322 INFO:teuthology.orchestra.run.smithi188.stdout: liboath-2.6.2-3.el8.x86_64 2024-01-27T02:31:30.322 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-01-27T02:31:30.322 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq-0.9.0-4.el8.x86_64 2024-01-27T02:31:30.322 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.322 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.322 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-01-27T02:31:30.322 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.322 INFO:teuthology.orchestra.run.smithi188.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-01-27T02:31:30.322 INFO:teuthology.orchestra.run.smithi188.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-01-27T02:31:30.322 INFO:teuthology.orchestra.run.smithi188.stdout: libtool-2.4.6-25.el8.x86_64 2024-01-27T02:31:30.322 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind-1.3.1-3.el8.x86_64 2024-01-27T02:31:30.322 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-01-27T02:31:30.322 INFO:teuthology.orchestra.run.smithi188.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-01-27T02:31:30.322 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-01-27T02:31:30.323 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-01-27T02:31:30.323 INFO:teuthology.orchestra.run.smithi188.stdout: oniguruma-6.8.2-2.el8.x86_64 2024-01-27T02:31:30.323 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-0.3.15-6.el8.x86_64 2024-01-27T02:31:30.323 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-01-27T02:31:30.323 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-01-27T02:31:30.323 INFO:teuthology.orchestra.run.smithi188.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-01-27T02:31:30.323 INFO:teuthology.orchestra.run.smithi188.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-01-27T02:31:30.323 INFO:teuthology.orchestra.run.smithi188.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-01-27T02:31:30.323 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-01-27T02:31:30.323 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel-2.5.1-7.el8.noarch 2024-01-27T02:31:30.323 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-01-27T02:31:30.323 INFO:teuthology.orchestra.run.smithi188.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-01-27T02:31:30.323 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-01-27T02:31:30.323 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.323 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.324 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.324 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-01-27T02:31:30.324 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-01-27T02:31:30.324 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-01-27T02:31:30.324 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-01-27T02:31:30.324 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2024-01-27T02:31:30.324 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-01-27T02:31:30.324 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-6.2-6.el8.noarch 2024-01-27T02:31:30.324 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-01-27T02:31:30.324 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-01-27T02:31:30.324 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-01-27T02:31:30.324 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-01-27T02:31:30.324 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-01-27T02:31:30.325 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-01-27T02:31:30.325 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-01-27T02:31:30.325 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako-1.0.6-14.el8.noarch 2024-01-27T02:31:30.325 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-01-27T02:31:30.325 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-01-27T02:31:30.325 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-01-27T02:31:30.325 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-01-27T02:31:30.325 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-01-27T02:31:30.325 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-01-27T02:31:30.325 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-01-27T02:31:30.325 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend-2.6-1.el8.noarch 2024-01-27T02:31:30.325 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-01-27T02:31:30.325 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-01-27T02:31:30.325 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-01-27T02:31:30.325 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-01-27T02:31:30.325 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser-2.14-14.el8.noarch 2024-01-27T02:31:30.326 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz-2017.2-11.el8.noarch 2024-01-27T02:31:30.326 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.326 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.326 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-01-27T02:31:30.326 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-01-27T02:31:30.326 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.326 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes-2.4.1-12.el8.noarch 2024-01-27T02:31:30.326 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa-4.9-2.el8.noarch 2024-01-27T02:31:30.326 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-01-27T02:31:30.326 INFO:teuthology.orchestra.run.smithi188.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-01-27T02:31:30.326 INFO:teuthology.orchestra.run.smithi188.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-01-27T02:31:30.326 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-01-27T02:31:30.326 INFO:teuthology.orchestra.run.smithi188.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-01-27T02:31:30.326 INFO:teuthology.orchestra.run.smithi188.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-01-27T02:31:30.326 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-01-27T02:31:30.326 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-01-27T02:31:30.327 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-01-27T02:31:30.327 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-01-27T02:31:30.327 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-01-27T02:31:30.327 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.327 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.327 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:30.327 INFO:teuthology.orchestra.run.smithi188.stdout: socat-1.7.4.1-1.el8.x86_64 2024-01-27T02:31:30.327 INFO:teuthology.orchestra.run.smithi188.stdout: thrift-0.13.0-2.el8.x86_64 2024-01-27T02:31:30.327 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-01-27T02:31:30.327 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-01-27T02:31:30.327 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-01-27T02:31:30.327 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:30.328 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-01-27T02:31:30.530 DEBUG:teuthology.parallel:result is None 2024-01-27T02:31:35.155 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 156/156 2024-01-27T02:31:35.155 INFO:teuthology.orchestra.run.smithi007.stdout:Installed products updated. 2024-01-27T02:31:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:35.815 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2024-01-27T02:31:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-01-27T02:31:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-01-27T02:31:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-01-27T02:31:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: libblkid-2.32.1-43.el8.x86_64 2024-01-27T02:31:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-01-27T02:31:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: libmount-2.32.1-43.el8.x86_64 2024-01-27T02:31:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: librados2-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-01-27T02:31:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-2.32.1-43.el8.x86_64 2024-01-27T02:31:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-01-27T02:31:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-01-27T02:31:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-2.32.1-43.el8.x86_64 2024-01-27T02:31:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-01-27T02:31:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-01-27T02:31:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-01-27T02:31:35.817 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-01-27T02:31:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-01-27T02:31:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: automake-1.16.1-8.el8.noarch 2024-01-27T02:31:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: bison-3.0.4-10.el8.x86_64 2024-01-27T02:31:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards-2:17.2.7-472.g74950076.el8.noarch 2024-01-27T02:31:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm-2:17.2.7-472.g74950076.el8.noarch 2024-01-27T02:31:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard-2:17.2.7-472.g74950076.el8.noarch 2024-01-27T02:31:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local-2:17.2.7-472.g74950076.el8.noarch 2024-01-27T02:31:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core-2:17.2.7-472.g74950076.el8.noarch 2024-01-27T02:31:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook-2:17.2.7-472.g74950076.el8.noarch 2024-01-27T02:31:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts-2:17.2.7-472.g74950076.el8.noarch 2024-01-27T02:31:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume-2:17.2.7-472.g74950076.el8.noarch 2024-01-27T02:31:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm-2:17.2.7-472.g74950076.el8.noarch 2024-01-27T02:31:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top-2:17.2.7-472.g74950076.el8.noarch 2024-01-27T02:31:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-01-27T02:31:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: flex-2.6.1-9.el8.x86_64 2024-01-27T02:31:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: fuse-2.9.7-17.el8.x86_64 2024-01-27T02:31:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-01-27T02:31:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: jq-1.6-7.el8.x86_64 2024-01-27T02:31:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-01-27T02:31:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-01-27T02:31:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-01-27T02:31:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-01-27T02:31:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-01-27T02:31:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: liboath-2.6.2-3.el8.x86_64 2024-01-27T02:31:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-01-27T02:31:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq-0.9.0-4.el8.x86_64 2024-01-27T02:31:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-01-27T02:31:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-01-27T02:31:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-01-27T02:31:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: libtool-2.4.6-25.el8.x86_64 2024-01-27T02:31:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind-1.3.1-3.el8.x86_64 2024-01-27T02:31:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-01-27T02:31:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-01-27T02:31:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-01-27T02:31:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-01-27T02:31:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: oniguruma-6.8.2-2.el8.x86_64 2024-01-27T02:31:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-0.3.15-6.el8.x86_64 2024-01-27T02:31:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-01-27T02:31:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-01-27T02:31:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-01-27T02:31:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-01-27T02:31:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-01-27T02:31:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-01-27T02:31:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel-2.5.1-7.el8.noarch 2024-01-27T02:31:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-01-27T02:31:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-01-27T02:31:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-01-27T02:31:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-01-27T02:31:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-01-27T02:31:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-01-27T02:31:35.823 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-01-27T02:31:35.823 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2024-01-27T02:31:35.823 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-01-27T02:31:35.823 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-6.2-6.el8.noarch 2024-01-27T02:31:35.823 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-01-27T02:31:35.823 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-01-27T02:31:35.823 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-01-27T02:31:35.823 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-01-27T02:31:35.824 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-01-27T02:31:35.824 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-01-27T02:31:35.824 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-01-27T02:31:35.824 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako-1.0.6-14.el8.noarch 2024-01-27T02:31:35.824 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-01-27T02:31:35.824 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-01-27T02:31:35.825 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-01-27T02:31:35.825 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-01-27T02:31:35.825 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-01-27T02:31:35.825 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-01-27T02:31:35.825 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-01-27T02:31:35.825 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend-2.6-1.el8.noarch 2024-01-27T02:31:35.825 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-01-27T02:31:35.826 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-01-27T02:31:35.826 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-01-27T02:31:35.826 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-01-27T02:31:35.826 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser-2.14-14.el8.noarch 2024-01-27T02:31:35.826 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz-2017.2-11.el8.noarch 2024-01-27T02:31:35.826 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.827 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.827 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-01-27T02:31:35.827 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-01-27T02:31:35.827 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.827 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes-2.4.1-12.el8.noarch 2024-01-27T02:31:35.827 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa-4.9-2.el8.noarch 2024-01-27T02:31:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-01-27T02:31:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-01-27T02:31:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-01-27T02:31:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-01-27T02:31:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-01-27T02:31:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-01-27T02:31:35.829 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-01-27T02:31:35.829 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-01-27T02:31:35.829 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-01-27T02:31:35.829 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-01-27T02:31:35.829 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-01-27T02:31:35.829 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.829 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.829 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd-2:17.2.7-472.g74950076.el8.x86_64 2024-01-27T02:31:35.829 INFO:teuthology.orchestra.run.smithi007.stdout: socat-1.7.4.1-1.el8.x86_64 2024-01-27T02:31:35.830 INFO:teuthology.orchestra.run.smithi007.stdout: thrift-0.13.0-2.el8.x86_64 2024-01-27T02:31:35.830 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-01-27T02:31:35.830 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-01-27T02:31:35.830 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-01-27T02:31:35.830 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:35.831 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-01-27T02:31:36.025 DEBUG:teuthology.parallel:result is None 2024-01-27T02:31:36.026 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7495007658df82356749387b98425d4a7fcef9b6 2024-01-27T02:31:36.172 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-27T02:31:36.202 INFO:teuthology.orchestra.run.smithi007.stdout:17.2.7-472.g74950076.el8 2024-01-27T02:31:36.203 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-472.g74950076.el8 2024-01-27T02:31:36.203 INFO:teuthology.task.install:The correct ceph version 17.2.7-472.g74950076 is installed. 2024-01-27T02:31:36.205 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7495007658df82356749387b98425d4a7fcef9b6 2024-01-27T02:31:36.350 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-27T02:31:36.378 INFO:teuthology.orchestra.run.smithi188.stdout:17.2.7-472.g74950076.el8 2024-01-27T02:31:36.379 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-472.g74950076.el8 2024-01-27T02:31:36.379 INFO:teuthology.task.install:The correct ceph version 17.2.7-472.g74950076 is installed. 2024-01-27T02:31:36.381 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-01-27T02:31:36.381 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:36.381 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-27T02:31:36.435 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:31:36.435 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-27T02:31:36.483 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-01-27T02:31:36.483 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:36.483 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2024-01-27T02:31:36.518 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-27T02:31:36.593 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:31:36.593 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/daemon-helper 2024-01-27T02:31:36.625 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-27T02:31:36.694 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-01-27T02:31:36.695 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:36.695 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-27T02:31:36.729 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-27T02:31:36.804 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:31:36.804 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-27T02:31:36.834 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-27T02:31:36.903 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-01-27T02:31:36.903 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:36.903 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2024-01-27T02:31:36.937 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-27T02:31:37.012 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:31:37.012 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/stdin-killer 2024-01-27T02:31:37.040 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-27T02:31:37.109 INFO:teuthology.run_tasks:Running task ceph... 2024-01-27T02:31:37.176 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-01-27T02:31:37.176 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /var/log/ceph 2024-01-27T02:31:37.179 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 777 /var/log/ceph 2024-01-27T02:31:37.210 INFO:tasks.ceph:Disabling ceph logrotate... 2024-01-27T02:31:37.211 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-27T02:31:37.254 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-27T02:31:37.285 INFO:tasks.ceph:Creating extra log directories... 2024-01-27T02:31:37.285 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-27T02:31:37.328 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-27T02:31:37.367 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-01-27T02:31:37.367 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-01-27T02:31:37.368 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-01-27T02:31:37.368 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:37.368 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-01-27T02:31:37.423 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-01-27T02:31:37.550 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-01-27T02:31:37.584 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:31:37.584 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-01-27T02:31:37.602 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-01-27T02:31:37.717 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-01-27T02:31:37.749 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-01-27T02:31:37.749 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-01-27T02:31:37.749 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533593', 'branch': 'wip-yuri4-testing-2024-01-25-1331-quincy', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7533593', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-yuri4-testing-2024-01-25-1331-quincy'}, 'ceph': {'cephfs': {'ec_profile': ['disabled']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': '7495007658df82356749387b98425d4a7fcef9b6'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7495007658df82356749387b98425d4a7fcef9b6'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri4-testing-2024-01-25-1331-quincy', 'sha1': '7495007658df82356749387b98425d4a7fcef9b6'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1285, 'sha1': '7495007658df82356749387b98425d4a7fcef9b6', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'fs', 'suite_branch': 'wip-yuri4-testing-2024-01-25-1331-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_7495007658df82356749387b98425d4a7fcef9b6/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '7495007658df82356749387b98425d4a7fcef9b6', 'targets': {'smithi007.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKinMYP627ZsZXwAlG5DwvLGSjUmKPgeWgJdoetEIMrsmJ9/ttaaTGA8euU9fQXMA1AiUIRCyYETVGpzaMcdBW4=', 'smithi188.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDaXqOH8orBjhRAqKw+EP2xxh7LgX535tX97+GEXhxwOtfxlNkcktqRwp6fCNryFWaHXhZcGtAIVYedLyqQvsOM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '7495007658df82356749387b98425d4a7fcef9b6'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['disabled']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': '7495007658df82356749387b98425d4a7fcef9b6', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_full']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'c856e0f9244dd29a50697123b480d63f16a4ed81', 'timestamp': '2024-01-26_01:07:29', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-01-27T02:31:37.750 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-27T02:31:37.752 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-27T02:31:37.801 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/run/ceph 2024-01-27T02:31:37.815 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/run/ceph 2024-01-27T02:31:37.868 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:37.868 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2024-01-27T02:31:37.907 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-27T02:31:37.908 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2024-01-27T02:31:37.969 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-27T02:31:37.969 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-27T02:31:37.969 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 169884 Links: 1 2024-01-27T02:31:37.970 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-27T02:31:37.970 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-01-27T02:31:37.970 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-01-27 02:31:24.177555322 +0000 2024-01-27T02:31:37.970 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-01-27 02:28:19.680809547 +0000 2024-01-27T02:31:37.970 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-01-27 02:28:19.680809547 +0000 2024-01-27T02:31:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2024-01-27T02:31:37.970 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-27T02:31:38.041 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-01-27T02:31:38.042 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-01-27T02:31:38.042 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.00011812 s, 4.3 MB/s 2024-01-27T02:31:38.043 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-27T02:31:38.107 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2024-01-27T02:31:38.169 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-27T02:31:38.169 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-27T02:31:38.170 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 177532 Links: 1 2024-01-27T02:31:38.170 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-27T02:31:38.170 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-01-27T02:31:38.170 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-01-27 02:31:24.177555322 +0000 2024-01-27T02:31:38.170 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-01-27 02:28:20.045804037 +0000 2024-01-27T02:31:38.170 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-01-27 02:28:20.045804037 +0000 2024-01-27T02:31:38.170 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2024-01-27T02:31:38.170 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-27T02:31:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-01-27T02:31:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-01-27T02:31:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000132146 s, 3.9 MB/s 2024-01-27T02:31:38.243 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-27T02:31:38.306 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2024-01-27T02:31:38.368 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-27T02:31:38.368 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-27T02:31:38.368 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 176807 Links: 1 2024-01-27T02:31:38.368 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-27T02:31:38.368 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-01-27T02:31:38.368 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-01-27 02:31:24.177555322 +0000 2024-01-27T02:31:38.368 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-01-27 02:28:20.413798481 +0000 2024-01-27T02:31:38.369 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-01-27 02:28:20.413798481 +0000 2024-01-27T02:31:38.369 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2024-01-27T02:31:38.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-27T02:31:38.441 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-01-27T02:31:38.441 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-01-27T02:31:38.441 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000110338 s, 4.6 MB/s 2024-01-27T02:31:38.443 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-27T02:31:38.506 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2024-01-27T02:31:38.569 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-27T02:31:38.569 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-27T02:31:38.569 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 176937 Links: 1 2024-01-27T02:31:38.569 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-27T02:31:38.570 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-01-27T02:31:38.570 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-01-27 02:31:24.177555322 +0000 2024-01-27T02:31:38.570 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-01-27 02:28:20.778792970 +0000 2024-01-27T02:31:38.570 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-01-27 02:28:20.778792970 +0000 2024-01-27T02:31:38.570 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2024-01-27T02:31:38.571 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-27T02:31:38.642 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-01-27T02:31:38.642 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-01-27T02:31:38.642 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000125213 s, 4.1 MB/s 2024-01-27T02:31:38.643 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-27T02:31:38.706 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-27T02:31:38.706 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:31:38.707 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/scratch_devs of=/dev/stdout 2024-01-27T02:31:38.727 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-27T02:31:38.727 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_1 2024-01-27T02:31:38.787 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-27T02:31:38.787 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-27T02:31:38.787 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 6h/6d Inode: 178251 Links: 1 2024-01-27T02:31:38.787 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-27T02:31:38.787 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-01-27T02:31:38.787 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-01-27 02:31:17.960864893 +0000 2024-01-27T02:31:38.787 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-01-27 02:27:40.586333231 +0000 2024-01-27T02:31:38.787 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-01-27 02:27:40.586333231 +0000 2024-01-27T02:31:38.787 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: - 2024-01-27T02:31:38.788 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-27T02:31:38.856 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-01-27T02:31:38.856 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-01-27T02:31:38.856 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000273507 s, 1.9 MB/s 2024-01-27T02:31:38.857 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-27T02:31:38.918 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_2 2024-01-27T02:31:38.976 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-27T02:31:38.976 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-27T02:31:38.976 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 6h/6d Inode: 174874 Links: 1 2024-01-27T02:31:38.977 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-27T02:31:38.977 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-01-27T02:31:38.977 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-01-27 02:31:17.960864893 +0000 2024-01-27T02:31:38.977 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-01-27 02:27:40.927326830 +0000 2024-01-27T02:31:38.977 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-01-27 02:27:40.927326830 +0000 2024-01-27T02:31:38.977 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: - 2024-01-27T02:31:38.977 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-27T02:31:39.043 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-01-27T02:31:39.043 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-01-27T02:31:39.044 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000245099 s, 2.1 MB/s 2024-01-27T02:31:39.048 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-27T02:31:39.106 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_3 2024-01-27T02:31:39.165 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-27T02:31:39.165 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-27T02:31:39.165 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 6h/6d Inode: 170901 Links: 1 2024-01-27T02:31:39.165 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-27T02:31:39.165 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-01-27T02:31:39.165 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-01-27 02:31:17.960864893 +0000 2024-01-27T02:31:39.165 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-01-27 02:27:41.258320617 +0000 2024-01-27T02:31:39.165 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-01-27 02:27:41.258320617 +0000 2024-01-27T02:31:39.165 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: - 2024-01-27T02:31:39.166 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-27T02:31:39.231 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-01-27T02:31:39.231 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-01-27T02:31:39.231 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000166481 s, 3.1 MB/s 2024-01-27T02:31:39.232 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-27T02:31:39.289 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_4 2024-01-27T02:31:39.346 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-27T02:31:39.347 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-27T02:31:39.347 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 6h/6d Inode: 175848 Links: 1 2024-01-27T02:31:39.347 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-27T02:31:39.347 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-01-27T02:31:39.347 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-01-27 02:31:17.960864893 +0000 2024-01-27T02:31:39.347 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-01-27 02:27:41.631313616 +0000 2024-01-27T02:31:39.347 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-01-27 02:27:41.631313616 +0000 2024-01-27T02:31:39.347 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: - 2024-01-27T02:31:39.347 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-27T02:31:39.413 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-01-27T02:31:39.413 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-01-27T02:31:39.413 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000303777 s, 1.7 MB/s 2024-01-27T02:31:39.414 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-27T02:31:39.471 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-27T02:31:39.471 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-01-27T02:31:39.471 INFO:tasks.ceph:Generating config... 2024-01-27T02:31:39.473 INFO:tasks.ceph:[client] client mount timeout = 600 2024-01-27T02:31:39.473 INFO:tasks.ceph:[client] debug client = 20 2024-01-27T02:31:39.473 INFO:tasks.ceph:[client] debug ms = 1 2024-01-27T02:31:39.473 INFO:tasks.ceph:[client] pid file = 2024-01-27T02:31:39.473 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-01-27T02:31:39.473 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-01-27T02:31:39.473 INFO:tasks.ceph:[client.0] debug client = 20 2024-01-27T02:31:39.473 INFO:tasks.ceph:[client.0] debug objectcacher = 20 2024-01-27T02:31:39.473 INFO:tasks.ceph:[client.0] debug objecter = 20 2024-01-27T02:31:39.473 INFO:tasks.ceph:[client.1] debug client = 20 2024-01-27T02:31:39.473 INFO:tasks.ceph:[client.1] debug objectcacher = 20 2024-01-27T02:31:39.473 INFO:tasks.ceph:[client.1] debug objecter = 20 2024-01-27T02:31:39.474 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-01-27T02:31:39.474 INFO:tasks.ceph:[mds] debug mds = 20 2024-01-27T02:31:39.474 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-01-27T02:31:39.474 INFO:tasks.ceph:[mds] debug ms = 1 2024-01-27T02:31:39.474 INFO:tasks.ceph:[mds] mds debug frag = True 2024-01-27T02:31:39.474 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-01-27T02:31:39.474 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-01-27T02:31:39.474 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-01-27T02:31:39.474 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-01-27T02:31:39.474 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-01-27T02:31:39.474 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-01-27T02:31:39.474 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-01-27T02:31:39.474 INFO:tasks.ceph:[mgr] debug ms = 1 2024-01-27T02:31:39.474 INFO:tasks.ceph:[mon] debug mon = 20 2024-01-27T02:31:39.474 INFO:tasks.ceph:[mon] debug ms = 1 2024-01-27T02:31:39.474 INFO:tasks.ceph:[mon] debug paxos = 20 2024-01-27T02:31:39.475 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-01-27T02:31:39.475 INFO:tasks.ceph:[mon] mon osd backfillfull ratio = 0.6 2024-01-27T02:31:39.475 INFO:tasks.ceph:[mon] mon osd full ratio = 0.7 2024-01-27T02:31:39.475 INFO:tasks.ceph:[mon] mon osd nearfull ratio = 0.6 2024-01-27T02:31:39.475 INFO:tasks.ceph:[osd] bdev async discard = True 2024-01-27T02:31:39.475 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-01-27T02:31:39.475 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-01-27T02:31:39.475 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-01-27T02:31:39.475 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-01-27T02:31:39.475 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-01-27T02:31:39.475 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-01-27T02:31:39.475 INFO:tasks.ceph:[osd] debug ms = 1 2024-01-27T02:31:39.475 INFO:tasks.ceph:[osd] debug osd = 20 2024-01-27T02:31:39.475 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-01-27T02:31:39.475 INFO:tasks.ceph:[osd] memstore device bytes = 200000000 2024-01-27T02:31:39.476 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-01-27T02:31:39.476 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-01-27T02:31:39.476 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-01-27T02:31:39.476 INFO:tasks.ceph:[osd] osd failsafe full ratio = 1.0 2024-01-27T02:31:39.476 INFO:tasks.ceph:[osd] osd mon report interval = 5 2024-01-27T02:31:39.476 INFO:tasks.ceph:[osd] osd objectstore = memstore 2024-01-27T02:31:39.476 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-01-27T02:31:39.476 INFO:tasks.ceph:Setting up mon.a... 2024-01-27T02:31:39.476 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-01-27T02:31:39.538 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.keyring 2024-01-27T02:31:39.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-01-27T02:31:39.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-27T02:31:39.622 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.7'), ('mon.b', '172.21.15.188'), ('mon.c', '[v2:172.21.15.188:3301,v1:172.21.15.188:6790]')] 2024-01-27T02:31:39.622 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.7,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-01-27T02:31:39.623 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:39.623 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-01-27T02:31:39.683 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.7 --add b 172.21.15.188 --addv c '[v2:172.21.15.188:3301,v1:172.21.15.188:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-01-27T02:31:39.776 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-01-27T02:31:39.777 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: generated fsid 17e1bcca-e647-4a58-b0ac-7d39d8508814 2024-01-27T02:31:39.777 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = octopus 2024-01-27T02:31:39.777 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 0 2024-01-27T02:31:39.777 INFO:teuthology.orchestra.run.smithi007.stdout:fsid 17e1bcca-e647-4a58-b0ac-7d39d8508814 2024-01-27T02:31:39.777 INFO:teuthology.orchestra.run.smithi007.stdout:last_changed 2024-01-27T02:31:39.777214+0000 2024-01-27T02:31:39.777 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-01-27T02:31:39.777214+0000 2024-01-27T02:31:39.777 INFO:teuthology.orchestra.run.smithi007.stdout:min_mon_release 15 (octopus) 2024-01-27T02:31:39.777 INFO:teuthology.orchestra.run.smithi007.stdout:election_strategy: 1 2024-01-27T02:31:39.777 INFO:teuthology.orchestra.run.smithi007.stdout:0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-01-27T02:31:39.778 INFO:teuthology.orchestra.run.smithi007.stdout:1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.b 2024-01-27T02:31:39.778 INFO:teuthology.orchestra.run.smithi007.stdout:2: [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] mon.c 2024-01-27T02:31:39.778 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-01-27T02:31:39.779 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-01-27T02:31:39.798 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 17e1bcca-e647-4a58-b0ac-7d39d8508814... 2024-01-27T02:31:39.799 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-27T02:31:39.842 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-27T02:31:39.888 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2024-01-27T02:31:39.889 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2024-01-27T02:31:39.889 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-27T02:31:39.889 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2024-01-27T02:31:39.889 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.889 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2024-01-27T02:31:39.889 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.889 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2024-01-27T02:31:39.889 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.889 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2024-01-27T02:31:39.889 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2024-01-27T02:31:39.889 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.889 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2024-01-27T02:31:39.889 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2024-01-27T02:31:39.889 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.890 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-27T02:31:39.890 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2024-01-27T02:31:39.890 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.890 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2024-01-27T02:31:39.890 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-27T02:31:39.890 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.890 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2024-01-27T02:31:39.890 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.890 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2024-01-27T02:31:39.890 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2024-01-27T02:31:39.890 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2024-01-27T02:31:39.890 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2024-01-27T02:31:39.890 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2024-01-27T02:31:39.890 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2024-01-27T02:31:39.890 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2024-01-27T02:31:39.891 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-27T02:31:39.891 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-27T02:31:39.891 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2024-01-27T02:31:39.891 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.891 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-27T02:31:39.891 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.891 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2024-01-27T02:31:39.891 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.891 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2024-01-27T02:31:39.891 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.891 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2024-01-27T02:31:39.891 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2024-01-27T02:31:39.891 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2024-01-27T02:31:39.892 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = "172.21.15.7,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-01-27T02:31:39.892 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on skipped message = False 2024-01-27T02:31:39.892 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = 17e1bcca-e647-4a58-b0ac-7d39d8508814 2024-01-27T02:31:39.892 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.892 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2024-01-27T02:31:39.892 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2024-01-27T02:31:39.892 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.892 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2024-01-27T02:31:39.892 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2024-01-27T02:31:39.892 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2024-01-27T02:31:39.892 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.892 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2024-01-27T02:31:39.892 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2024-01-27T02:31:39.893 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.893 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2024-01-27T02:31:39.893 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2024-01-27T02:31:39.893 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2024-01-27T02:31:39.893 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.893 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2024-01-27T02:31:39.893 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2024-01-27T02:31:39.893 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.893 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2024-01-27T02:31:39.893 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.893 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2024-01-27T02:31:39.893 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.893 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2024-01-27T02:31:39.893 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.894 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2024-01-27T02:31:39.894 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2024-01-27T02:31:39.894 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.894 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2024-01-27T02:31:39.894 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2024-01-27T02:31:39.894 INFO:teuthology.orchestra.run.smithi188.stdout: bdev async discard = True 2024-01-27T02:31:39.894 INFO:teuthology.orchestra.run.smithi188.stdout: bdev enable discard = True 2024-01-27T02:31:39.894 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore allocator = bitmap 2024-01-27T02:31:39.894 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore block size = 96636764160 2024-01-27T02:31:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore fsck on mount = True 2024-01-27T02:31:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluefs = 1/20 2024-01-27T02:31:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluestore = 1/20 2024-01-27T02:31:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-01-27T02:31:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2024-01-27T02:31:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: debug rocksdb = 4/10 2024-01-27T02:31:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: memstore device bytes = 200000000 2024-01-27T02:31:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd backfillfull_ratio = 0.85 2024-01-27T02:31:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd full ratio = 0.9 2024-01-27T02:31:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd nearfull ratio = 0.8 2024-01-27T02:31:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: osd failsafe full ratio = 1.0 2024-01-27T02:31:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: osd mon report interval = 5 2024-01-27T02:31:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: osd objectstore = memstore 2024-01-27T02:31:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-01-27T02:31:39.895 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.896 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2024-01-27T02:31:39.896 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-01-27T02:31:39.896 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2024-01-27T02:31:39.896 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-01-27T02:31:39.896 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-01-27T02:31:39.896 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-01-27T02:31:39.896 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-01-27T02:31:39.896 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2024-01-27T02:31:39.896 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.896 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2024-01-27T02:31:39.896 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-01-27T02:31:39.896 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-01-27T02:31:39.896 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2024-01-27T02:31:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-01-27T02:31:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2024-01-27T02:31:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2024-01-27T02:31:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-01-27T02:31:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2024-01-27T02:31:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2024-01-27T02:31:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-01-27T02:31:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-27T02:31:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2024-01-27T02:31:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2024-01-27T02:31:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2024-01-27T02:31:39.897 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-27T02:31:39.898 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-27T02:31:39.898 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.898 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2024-01-27T02:31:39.898 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2024-01-27T02:31:39.898 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.898 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2024-01-27T02:31:39.898 INFO:teuthology.orchestra.run.smithi188.stdout: mon op complaint time = 120 2024-01-27T02:31:39.898 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd backfillfull ratio = 0.6 2024-01-27T02:31:39.898 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd full ratio = 0.7 2024-01-27T02:31:39.898 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd nearfull ratio = 0.6 2024-01-27T02:31:39.898 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-27T02:31:39.898 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2024-01-27T02:31:39.898 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2024-01-27T02:31:39.899 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2024-01-27T02:31:39.899 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2024-01-27T02:31:39.899 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-27T02:31:39.899 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-27T02:31:39.899 INFO:teuthology.orchestra.run.smithi188.stdout: client mount timeout = 600 2024-01-27T02:31:39.899 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-01-27T02:31:39.899 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-01-27T02:31:39.899 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = "" 2024-01-27T02:31:39.899 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-01-27T02:31:39.899 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-01-27T02:31:39.899 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2024-01-27T02:31:39.899 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.b] 2024-01-27T02:31:39.899 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.c] 2024-01-27T02:31:39.899 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2024-01-27T02:31:39.900 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2024-01-27T02:31:39.900 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2024-01-27T02:31:39.900 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2024-01-27T02:31:39.900 INFO:teuthology.orchestra.run.smithi188.stdout:[client.0] 2024-01-27T02:31:39.900 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-01-27T02:31:39.900 INFO:teuthology.orchestra.run.smithi188.stdout: debug objectcacher = 20 2024-01-27T02:31:39.900 INFO:teuthology.orchestra.run.smithi188.stdout: debug objecter = 20 2024-01-27T02:31:39.900 INFO:teuthology.orchestra.run.smithi188.stdout:[client.1] 2024-01-27T02:31:39.900 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-01-27T02:31:39.900 INFO:teuthology.orchestra.run.smithi188.stdout: debug objectcacher = 20 2024-01-27T02:31:39.900 INFO:teuthology.orchestra.run.smithi188.stdout: debug objecter = 20 2024-01-27T02:31:39.900 INFO:teuthology.orchestra.run.smithi188.stdout:[mds] 2024-01-27T02:31:39.900 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds = 20 2024-01-27T02:31:39.900 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds balancer = 20 2024-01-27T02:31:39.901 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-01-27T02:31:39.901 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug frag = True 2024-01-27T02:31:39.901 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug scatterstat = True 2024-01-27T02:31:39.901 INFO:teuthology.orchestra.run.smithi188.stdout: mds op complaint time = 180 2024-01-27T02:31:39.901 INFO:teuthology.orchestra.run.smithi188.stdout: mds verify scatter = True 2024-01-27T02:31:39.901 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-01-27T02:31:39.901 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-01-27T02:31:39.901 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-01-27T02:31:39.902 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-01-27T02:31:39.902 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2024-01-27T02:31:39.902 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-27T02:31:39.902 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2024-01-27T02:31:39.902 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.902 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2024-01-27T02:31:39.902 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.902 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2024-01-27T02:31:39.902 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.902 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2024-01-27T02:31:39.902 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2024-01-27T02:31:39.902 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2024-01-27T02:31:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2024-01-27T02:31:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-27T02:31:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2024-01-27T02:31:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2024-01-27T02:31:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-27T02:31:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2024-01-27T02:31:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2024-01-27T02:31:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2024-01-27T02:31:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2024-01-27T02:31:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2024-01-27T02:31:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2024-01-27T02:31:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2024-01-27T02:31:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2024-01-27T02:31:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-27T02:31:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-27T02:31:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2024-01-27T02:31:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-27T02:31:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2024-01-27T02:31:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2024-01-27T02:31:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2024-01-27T02:31:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2024-01-27T02:31:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2024-01-27T02:31:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.7,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-01-27T02:31:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on skipped message = False 2024-01-27T02:31:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 17e1bcca-e647-4a58-b0ac-7d39d8508814 2024-01-27T02:31:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.905 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2024-01-27T02:31:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2024-01-27T02:31:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2024-01-27T02:31:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2024-01-27T02:31:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2024-01-27T02:31:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2024-01-27T02:31:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2024-01-27T02:31:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2024-01-27T02:31:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2024-01-27T02:31:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2024-01-27T02:31:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2024-01-27T02:31:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2024-01-27T02:31:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2024-01-27T02:31:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2024-01-27T02:31:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2024-01-27T02:31:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2024-01-27T02:31:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2024-01-27T02:31:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2024-01-27T02:31:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2024-01-27T02:31:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: bdev async discard = True 2024-01-27T02:31:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: bdev enable discard = True 2024-01-27T02:31:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore allocator = bitmap 2024-01-27T02:31:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2024-01-27T02:31:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2024-01-27T02:31:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 1/20 2024-01-27T02:31:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 1/20 2024-01-27T02:31:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-01-27T02:31:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2024-01-27T02:31:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 4/10 2024-01-27T02:31:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: memstore device bytes = 200000000 2024-01-27T02:31:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2024-01-27T02:31:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2024-01-27T02:31:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2024-01-27T02:31:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 1.0 2024-01-27T02:31:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: osd mon report interval = 5 2024-01-27T02:31:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = memstore 2024-01-27T02:31:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2024-01-27T02:31:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.908 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2024-01-27T02:31:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-01-27T02:31:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2024-01-27T02:31:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-01-27T02:31:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-01-27T02:31:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-01-27T02:31:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-01-27T02:31:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2024-01-27T02:31:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.909 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2024-01-27T02:31:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-01-27T02:31:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-01-27T02:31:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2024-01-27T02:31:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-01-27T02:31:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2024-01-27T02:31:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2024-01-27T02:31:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-01-27T02:31:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2024-01-27T02:31:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2024-01-27T02:31:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-01-27T02:31:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-27T02:31:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2024-01-27T02:31:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2024-01-27T02:31:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2024-01-27T02:31:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-27T02:31:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-27T02:31:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2024-01-27T02:31:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2024-01-27T02:31:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2024-01-27T02:31:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: mon op complaint time = 120 2024-01-27T02:31:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull ratio = 0.6 2024-01-27T02:31:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.7 2024-01-27T02:31:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.6 2024-01-27T02:31:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:39.911 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2024-01-27T02:31:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2024-01-27T02:31:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2024-01-27T02:31:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2024-01-27T02:31:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-27T02:31:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-27T02:31:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: client mount timeout = 600 2024-01-27T02:31:39.912 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-01-27T02:31:39.912 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-01-27T02:31:39.912 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = "" 2024-01-27T02:31:39.912 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2024-01-27T02:31:39.912 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2024-01-27T02:31:39.912 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2024-01-27T02:31:39.912 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2024-01-27T02:31:39.912 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2024-01-27T02:31:39.912 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2024-01-27T02:31:39.912 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2024-01-27T02:31:39.912 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2024-01-27T02:31:39.912 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2024-01-27T02:31:39.912 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-01-27T02:31:39.912 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-01-27T02:31:39.912 INFO:teuthology.orchestra.run.smithi007.stdout: debug objectcacher = 20 2024-01-27T02:31:39.913 INFO:teuthology.orchestra.run.smithi007.stdout: debug objecter = 20 2024-01-27T02:31:39.913 INFO:teuthology.orchestra.run.smithi007.stdout:[client.1] 2024-01-27T02:31:39.913 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-01-27T02:31:39.913 INFO:teuthology.orchestra.run.smithi007.stdout: debug objectcacher = 20 2024-01-27T02:31:39.913 INFO:teuthology.orchestra.run.smithi007.stdout: debug objecter = 20 2024-01-27T02:31:39.913 INFO:teuthology.orchestra.run.smithi007.stdout:[mds] 2024-01-27T02:31:39.913 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds = 20 2024-01-27T02:31:39.913 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds balancer = 20 2024-01-27T02:31:39.913 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-01-27T02:31:39.913 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug frag = True 2024-01-27T02:31:39.913 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug scatterstat = True 2024-01-27T02:31:39.913 INFO:teuthology.orchestra.run.smithi007.stdout: mds op complaint time = 180 2024-01-27T02:31:39.913 INFO:teuthology.orchestra.run.smithi007.stdout: mds verify scatter = True 2024-01-27T02:31:39.913 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2024-01-27T02:31:39.913 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2024-01-27T02:31:39.914 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2024-01-27T02:31:39.915 INFO:tasks.ceph:Creating admin key on mon.a... 2024-01-27T02:31:39.915 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-01-27T02:31:40.039 INFO:tasks.ceph:Copying monmap to all nodes... 2024-01-27T02:31:40.040 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:40.040 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-01-27T02:31:40.060 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:40.060 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-01-27T02:31:40.121 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:31:40.121 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:40.122 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-27T02:31:40.122 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-27T02:31:40.336 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:40.336 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-27T02:31:40.375 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:31:40.375 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:31:40.375 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-27T02:31:40.375 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-27T02:31:40.413 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:31:40.413 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-27T02:31:40.470 INFO:tasks.ceph:Setting up mon nodes... 2024-01-27T02:31:40.470 INFO:tasks.ceph:Setting up mgr nodes... 2024-01-27T02:31:40.470 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-01-27T02:31:40.534 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-01-27T02:31:40.536 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-01-27T02:31:40.602 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-01-27T02:31:40.607 INFO:tasks.ceph:Setting up mds nodes... 2024-01-27T02:31:40.608 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-01-27T02:31:40.671 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-01-27T02:31:40.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-01-27T02:31:40.706 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-01-27T02:31:40.810 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-01-27T02:31:40.812 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-01-27T02:31:40.843 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-01-27T02:31:40.894 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-01-27T02:31:40.896 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-01-27T02:31:40.923 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-01-27T02:31:41.013 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-01-27T02:31:41.015 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-01-27T02:31:41.042 INFO:tasks.ceph_client:Setting up client nodes... 2024-01-27T02:31:41.042 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-01-27T02:31:41.096 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-01-27T02:31:41.109 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-01-27T02:31:41.157 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-01-27T02:31:41.242 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-01-27T02:31:41.283 INFO:teuthology.orchestra.run.smithi188.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-01-27T02:31:41.295 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-01-27T02:31:41.375 INFO:teuthology.orchestra.run.smithi188.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-01-27T02:31:41.388 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-01-27T02:31:41.389 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-01-27T02:31:41.389 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-01-27T02:31:41.423 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-27T02:31:41.423 INFO:tasks.ceph:role: osd.0 2024-01-27T02:31:41.423 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:31:41.423 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-27T02:31:41.497 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-27T02:31:41.497 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-27T02:31:41.497 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-27T02:31:41.497 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-27T02:31:41.497 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-27T02:31:41.497 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-01-27T02:31:41.497 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-27T02:31:41.498 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-27T02:31:41.498 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-27T02:31:41.498 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-27T02:31:41.520 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-01-27T02:31:41.522 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-01-27T02:31:41.522 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-01-27T02:31:41.612 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-01-27T02:31:41.650 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-01-27T02:31:41.724 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-27T02:31:41.724 INFO:tasks.ceph:role: osd.1 2024-01-27T02:31:41.724 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:31:41.725 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-27T02:31:41.799 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-27T02:31:41.799 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-27T02:31:41.799 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-27T02:31:41.799 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-27T02:31:41.799 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-27T02:31:41.799 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-01-27T02:31:41.799 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-27T02:31:41.799 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-27T02:31:41.799 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-27T02:31:41.800 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-27T02:31:41.821 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-01-27T02:31:41.823 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-01-27T02:31:41.823 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-01-27T02:31:41.914 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-01-27T02:31:41.950 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-01-27T02:31:42.024 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-27T02:31:42.024 INFO:tasks.ceph:role: osd.2 2024-01-27T02:31:42.024 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:31:42.024 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-27T02:31:42.099 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-27T02:31:42.099 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-27T02:31:42.099 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-27T02:31:42.099 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-27T02:31:42.099 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-27T02:31:42.099 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-01-27T02:31:42.099 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-27T02:31:42.099 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-27T02:31:42.099 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-27T02:31:42.099 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-27T02:31:42.121 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-01-27T02:31:42.123 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-01-27T02:31:42.124 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-01-27T02:31:42.212 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-01-27T02:31:42.248 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-01-27T02:31:42.322 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-27T02:31:42.323 INFO:tasks.ceph:role: osd.3 2024-01-27T02:31:42.323 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:31:42.323 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-01-27T02:31:42.398 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-01-27T02:31:42.398 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-27T02:31:42.398 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-27T02:31:42.398 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-27T02:31:42.398 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-27T02:31:42.398 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-01-27T02:31:42.398 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-27T02:31:42.398 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-27T02:31:42.399 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-27T02:31:42.399 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-27T02:31:42.421 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-01-27T02:31:42.423 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-01-27T02:31:42.423 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-01-27T02:31:42.511 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-01-27T02:31:42.548 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-27T02:31:42.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:42.642+0000 7fdec5f48540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-01-27T02:31:42.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:42.642+0000 7fdec5f48540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-01-27T02:31:42.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:42.642+0000 7fdec5f48540 -1 memstore(/var/lib/ceph/osd/ceph-0) /var/lib/ceph/osd/ceph-0 2024-01-27T02:31:42.649 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-01-27T02:31:42.681 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-27T02:31:42.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:42.775+0000 7f0469d5d540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-01-27T02:31:42.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:42.775+0000 7f0469d5d540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-01-27T02:31:42.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:42.776+0000 7f0469d5d540 -1 memstore(/var/lib/ceph/osd/ceph-1) /var/lib/ceph/osd/ceph-1 2024-01-27T02:31:42.783 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-01-27T02:31:42.815 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-27T02:31:42.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:42.908+0000 7fea16a30540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-01-27T02:31:42.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:42.908+0000 7fea16a30540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-01-27T02:31:42.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:42.909+0000 7fea16a30540 -1 memstore(/var/lib/ceph/osd/ceph-2) /var/lib/ceph/osd/ceph-2 2024-01-27T02:31:42.915 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-01-27T02:31:42.947 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-27T02:31:43.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:43.042+0000 7fbe21de0540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-01-27T02:31:43.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:43.042+0000 7fbe21de0540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-01-27T02:31:43.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:43.042+0000 7fbe21de0540 -1 memstore(/var/lib/ceph/osd/ceph-3) /var/lib/ceph/osd/ceph-3 2024-01-27T02:31:43.049 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-01-27T02:31:43.082 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-01-27T02:31:43.115 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-27T02:31:43.115 INFO:tasks.ceph:role: osd.4 2024-01-27T02:31:43.115 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:31:43.115 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-27T02:31:43.192 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-27T02:31:43.192 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-27T02:31:43.192 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-27T02:31:43.192 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-27T02:31:43.192 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-27T02:31:43.193 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-01-27T02:31:43.193 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-27T02:31:43.193 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-27T02:31:43.193 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-27T02:31:43.193 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-27T02:31:43.622 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-01-27T02:31:43.625 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-01-27T02:31:43.625 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-01-27T02:31:43.802 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-01-27T02:31:43.840 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-01-27T02:31:43.909 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-27T02:31:43.909 INFO:tasks.ceph:role: osd.5 2024-01-27T02:31:43.909 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:31:43.910 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-27T02:31:43.985 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-27T02:31:43.985 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-27T02:31:43.986 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-27T02:31:43.986 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-27T02:31:43.986 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-27T02:31:43.986 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-01-27T02:31:43.986 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-27T02:31:43.986 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-27T02:31:43.986 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-27T02:31:43.986 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-27T02:31:44.400 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-01-27T02:31:44.402 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-01-27T02:31:44.402 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-01-27T02:31:44.518 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-01-27T02:31:44.550 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-01-27T02:31:44.617 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-27T02:31:44.618 INFO:tasks.ceph:role: osd.6 2024-01-27T02:31:44.618 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:31:44.618 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-27T02:31:44.693 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-27T02:31:44.693 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-27T02:31:44.693 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-27T02:31:44.693 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-27T02:31:44.693 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-27T02:31:44.693 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-01-27T02:31:44.693 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-27T02:31:44.693 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-27T02:31:44.693 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-27T02:31:44.693 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-27T02:31:45.140 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-01-27T02:31:45.142 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-01-27T02:31:45.142 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-01-27T02:31:45.279 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-01-27T02:31:45.311 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-01-27T02:31:45.379 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-27T02:31:45.380 INFO:tasks.ceph:role: osd.7 2024-01-27T02:31:45.380 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:31:45.380 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-01-27T02:31:45.455 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-01-27T02:31:45.455 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-27T02:31:45.455 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-27T02:31:45.455 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-27T02:31:45.455 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-27T02:31:45.455 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-01-27T02:31:45.455 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-27T02:31:45.455 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-27T02:31:45.455 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-27T02:31:45.455 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-27T02:31:45.901 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-01-27T02:31:45.903 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-01-27T02:31:45.903 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-01-27T02:31:46.087 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-01-27T02:31:46.119 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-27T02:31:46.204 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:46.204+0000 7fb43be19540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-01-27T02:31:46.205 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:46.205+0000 7fb43be19540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-01-27T02:31:46.205 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:46.205+0000 7fb43be19540 -1 memstore(/var/lib/ceph/osd/ceph-4) /var/lib/ceph/osd/ceph-4 2024-01-27T02:31:46.211 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-01-27T02:31:46.238 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-27T02:31:46.321 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:46.321+0000 7ff971f4a540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-01-27T02:31:46.321 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:46.321+0000 7ff971f4a540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-01-27T02:31:46.321 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:46.322+0000 7ff971f4a540 -1 memstore(/var/lib/ceph/osd/ceph-5) /var/lib/ceph/osd/ceph-5 2024-01-27T02:31:46.331 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-01-27T02:31:46.398 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-27T02:31:46.480 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:46.480+0000 7f873e018540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-01-27T02:31:46.481 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:46.481+0000 7f873e018540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-01-27T02:31:46.481 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:46.481+0000 7f873e018540 -1 memstore(/var/lib/ceph/osd/ceph-6) /var/lib/ceph/osd/ceph-6 2024-01-27T02:31:46.489 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-01-27T02:31:46.557 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-27T02:31:46.640 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:46.639+0000 7f5524f13540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-01-27T02:31:46.641 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:46.639+0000 7f5524f13540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-01-27T02:31:46.641 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:46.640+0000 7f5524f13540 -1 memstore(/var/lib/ceph/osd/ceph-7) /var/lib/ceph/osd/ceph-7 2024-01-27T02:31:46.645 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-01-27T02:31:46.712 INFO:tasks.ceph:Reading keys from all nodes... 2024-01-27T02:31:46.713 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:46.713 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-01-27T02:31:46.747 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:46.747 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-01-27T02:31:46.821 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:46.822 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-01-27T02:31:46.897 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:46.897 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-01-27T02:31:46.972 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:46.972 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-01-27T02:31:47.046 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:47.046 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-01-27T02:31:47.120 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:47.120 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-01-27T02:31:47.194 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:31:47.194 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-01-27T02:31:47.222 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:31:47.222 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-01-27T02:31:47.289 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:31:47.289 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-01-27T02:31:47.355 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:31:47.355 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-01-27T02:31:47.422 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:31:47.422 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-01-27T02:31:47.489 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:31:47.489 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-01-27T02:31:47.556 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:31:47.556 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-01-27T02:31:47.622 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:47.622 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-01-27T02:31:47.643 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:47.644 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-01-27T02:31:47.706 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:31:47.706 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-01-27T02:31:47.723 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:31:47.723 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-01-27T02:31:47.780 INFO:tasks.ceph:Adding keys to all mons... 2024-01-27T02:31:47.780 DEBUG:teuthology.orchestra.run.smithi007:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-27T02:31:47.783 DEBUG:teuthology.orchestra.run.smithi188:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-27T02:31:47.823 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.y] 2024-01-27T02:31:47.823 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAMa7Rls7TaHxAALS3ghxKtfmgQfIELiD9+Bg== 2024-01-27T02:31:47.823 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2024-01-27T02:31:47.823 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAMa7RlHdcPKBAAfhnh1I3ZSQEXWJXlXCs98Q== 2024-01-27T02:31:47.823 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2024-01-27T02:31:47.823 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAMa7RlJxRKMBAAS0j7/zZeqGglOxCK1pg0WA== 2024-01-27T02:31:47.823 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.0] 2024-01-27T02:31:47.823 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAOa7Rlh6VSJhAAttuCte1WhNEvUkv5x/IKQQ== 2024-01-27T02:31:47.823 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.1] 2024-01-27T02:31:47.823 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAOa7Rlu/RGLhAAKzaCNhzYKhsNhxkMvY++gw== 2024-01-27T02:31:47.824 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.2] 2024-01-27T02:31:47.824 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAOa7RliRMyNhAAr0tG3evmVV7ZlGxbxfWPMw== 2024-01-27T02:31:47.824 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.3] 2024-01-27T02:31:47.824 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAPa7RlRuSNAhAA3e9atlb+ffvlnLhRYbWZPw== 2024-01-27T02:31:47.824 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.x] 2024-01-27T02:31:47.824 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAMa7Rlfb7qIxAAlsJYiSolJ7D8EykGWP62bA== 2024-01-27T02:31:47.824 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2024-01-27T02:31:47.824 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAMa7Rl8eVYNRAAfP8rOqO50RpoNZ1+1j2DXQ== 2024-01-27T02:31:47.824 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2024-01-27T02:31:47.824 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQANa7RlNwfUABAALRQAoHoGhniqCvTQOqbbug== 2024-01-27T02:31:47.824 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.4] 2024-01-27T02:31:47.824 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQASa7RlfeA6DBAAXpklgbNHRO+5G+fdIERjRw== 2024-01-27T02:31:47.824 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.5] 2024-01-27T02:31:47.824 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQASa7Rl77ItExAABB7uABm63ubE/YUs2B5pGw== 2024-01-27T02:31:47.825 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.6] 2024-01-27T02:31:47.825 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQASa7RljDuwHBAAPPAlhoqS03qPpq58jmevLA== 2024-01-27T02:31:47.825 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.7] 2024-01-27T02:31:47.825 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQASa7Rl1YElJhAAqHP/QEdl1SEalGcbQwPwAg== 2024-01-27T02:31:47.825 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-01-27T02:31:47.825 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQANa7RlaEqaBRAATFiIeW8lT9BdNuN0m3x9dQ== 2024-01-27T02:31:47.825 INFO:teuthology.orchestra.run.smithi007.stdout:[client.2] 2024-01-27T02:31:47.825 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQANa7Rl5fRqCRAA/5swTGknnpP64TAh+2X41A== 2024-01-27T02:31:47.825 INFO:teuthology.orchestra.run.smithi007.stdout:[client.1] 2024-01-27T02:31:47.826 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQANa7RlNADmEBAAJXwdJ35ylP5coy1dtHoD3g== 2024-01-27T02:31:47.826 INFO:teuthology.orchestra.run.smithi007.stdout:[client.3] 2024-01-27T02:31:47.826 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQANa7Rl0SdwFhAAhCmw9ViiJbHVfS5a2r/D+w== 2024-01-27T02:31:47.844 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr.y] 2024-01-27T02:31:47.845 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAMa7Rls7TaHxAALS3ghxKtfmgQfIELiD9+Bg== 2024-01-27T02:31:47.848 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2024-01-27T02:31:47.848 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAMa7RlHdcPKBAAfhnh1I3ZSQEXWJXlXCs98Q== 2024-01-27T02:31:47.848 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2024-01-27T02:31:47.848 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAMa7RlJxRKMBAAS0j7/zZeqGglOxCK1pg0WA== 2024-01-27T02:31:47.848 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.0] 2024-01-27T02:31:47.848 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAOa7Rlh6VSJhAAttuCte1WhNEvUkv5x/IKQQ== 2024-01-27T02:31:47.848 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.1] 2024-01-27T02:31:47.848 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAOa7Rlu/RGLhAAKzaCNhzYKhsNhxkMvY++gw== 2024-01-27T02:31:47.848 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.2] 2024-01-27T02:31:47.848 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAOa7RliRMyNhAAr0tG3evmVV7ZlGxbxfWPMw== 2024-01-27T02:31:47.848 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.3] 2024-01-27T02:31:47.848 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAPa7RlRuSNAhAA3e9atlb+ffvlnLhRYbWZPw== 2024-01-27T02:31:47.848 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr.x] 2024-01-27T02:31:47.849 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAMa7Rlfb7qIxAAlsJYiSolJ7D8EykGWP62bA== 2024-01-27T02:31:47.849 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2024-01-27T02:31:47.849 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAMa7Rl8eVYNRAAfP8rOqO50RpoNZ1+1j2DXQ== 2024-01-27T02:31:47.849 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2024-01-27T02:31:47.849 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQANa7RlNwfUABAALRQAoHoGhniqCvTQOqbbug== 2024-01-27T02:31:47.849 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.4] 2024-01-27T02:31:47.849 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQASa7RlfeA6DBAAXpklgbNHRO+5G+fdIERjRw== 2024-01-27T02:31:47.849 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.5] 2024-01-27T02:31:47.849 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQASa7Rl77ItExAABB7uABm63ubE/YUs2B5pGw== 2024-01-27T02:31:47.849 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.6] 2024-01-27T02:31:47.849 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQASa7RljDuwHBAAPPAlhoqS03qPpq58jmevLA== 2024-01-27T02:31:47.849 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.7] 2024-01-27T02:31:47.849 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQASa7Rl1YElJhAAqHP/QEdl1SEalGcbQwPwAg== 2024-01-27T02:31:47.849 INFO:teuthology.orchestra.run.smithi188.stdout:[client.0] 2024-01-27T02:31:47.849 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQANa7RlaEqaBRAATFiIeW8lT9BdNuN0m3x9dQ== 2024-01-27T02:31:47.850 INFO:teuthology.orchestra.run.smithi188.stdout:[client.2] 2024-01-27T02:31:47.850 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQANa7Rl5fRqCRAA/5swTGknnpP64TAh+2X41A== 2024-01-27T02:31:47.850 INFO:teuthology.orchestra.run.smithi188.stdout:[client.1] 2024-01-27T02:31:47.850 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQANa7RlNADmEBAAJXwdJ35ylP5coy1dtHoD3g== 2024-01-27T02:31:47.850 INFO:teuthology.orchestra.run.smithi188.stdout:[client.3] 2024-01-27T02:31:47.850 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQANa7Rl0SdwFhAAhCmw9ViiJbHVfS5a2r/D+w== 2024-01-27T02:31:47.851 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-27T02:31:47.870 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-27T02:31:47.960 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-27T02:31:47.962 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-27T02:31:48.195 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-27T02:31:48.198 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-27T02:31:48.260 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-27T02:31:48.262 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-27T02:31:48.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-27T02:31:48.321 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-27T02:31:48.374 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-27T02:31:48.377 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-27T02:31:48.430 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-27T02:31:48.433 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-27T02:31:48.486 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-27T02:31:48.488 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-27T02:31:48.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-27T02:31:48.543 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-27T02:31:48.600 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-27T02:31:48.602 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-27T02:31:48.658 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-27T02:31:48.661 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-27T02:31:48.714 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-27T02:31:48.716 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-27T02:31:48.769 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-27T02:31:48.772 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-27T02:31:48.825 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-27T02:31:48.827 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-27T02:31:48.880 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-27T02:31:48.882 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-27T02:31:48.938 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-27T02:31:48.940 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-27T02:31:48.996 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-27T02:31:48.998 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-27T02:31:49.054 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-27T02:31:49.056 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-27T02:31:49.112 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-01-27T02:31:49.112 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-01-27T02:31:49.142 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-27T02:31:49.245 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-01-27T02:31:49.276 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-01-27T02:31:49.303 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-27T02:31:49.396 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-01-27T02:31:49.422 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-01-27T02:31:49.490 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-27T02:31:49.587 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-01-27T02:31:49.614 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-27T02:31:49.617 DEBUG:teuthology.orchestra.run.smithi188:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-27T02:31:49.670 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-01-27T02:31:49.671 INFO:tasks.ceph.mon.a:Restarting daemon 2024-01-27T02:31:49.671 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-01-27T02:31:49.678 INFO:tasks.ceph.mon.a:Started 2024-01-27T02:31:49.678 INFO:tasks.ceph.mon.b:Restarting daemon 2024-01-27T02:31:49.679 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-01-27T02:31:49.713 INFO:tasks.ceph.mon.b:Started 2024-01-27T02:31:49.713 INFO:tasks.ceph.mon.c:Restarting daemon 2024-01-27T02:31:49.714 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-01-27T02:31:49.716 INFO:tasks.ceph.mon.c:Started 2024-01-27T02:31:49.716 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-01-27T02:31:49.717 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-01-27T02:31:49.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-01-27T02:31:49.719 INFO:tasks.ceph.mgr.y:Started 2024-01-27T02:31:49.720 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-01-27T02:31:49.720 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-01-27T02:31:49.722 INFO:tasks.ceph.mgr.x:Started 2024-01-27T02:31:49.722 INFO:tasks.ceph:Setting crush tunables to default 2024-01-27T02:31:49.722 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd crush tunables default 2024-01-27T02:31:49.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:49.856+0000 7fbb53c19700 1 Processor -- start 2024-01-27T02:31:49.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:49.856+0000 7fbb53c19700 1 -- start start 2024-01-27T02:31:49.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:49.857+0000 7fbb53c19700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbb4c1416c0 0x7fbb4c0cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:49.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:49.857+0000 7fbb53c19700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb4c141a90 0x7fbb4c0cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:49.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:49.857+0000 7fbb53c19700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb4c15c7d0 con 0x7fbb4c0cab00 2024-01-27T02:31:49.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:49.857+0000 7fbb53c19700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb4c0cd9c0 con 0x7fbb4c141a90 2024-01-27T02:31:49.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:49.857+0000 7fbb53c19700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb4c0c9640 con 0x7fbb4c1416c0 2024-01-27T02:31:49.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:49.857+0000 7fbb511b4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb4c141a90 0x7fbb4c0cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:49.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:49.857+0000 7fbb511b4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb4c141a90 0x7fbb4c0cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49014/0 (socket says 172.21.15.7:49014) 2024-01-27T02:31:49.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:49.857+0000 7fbb511b4700 1 -- 172.21.15.7:0/3567428226 learned_addr learned my addr 172.21.15.7:0/3567428226 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:31:49.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:49.857+0000 7fbb519b5700 1 --2- 172.21.15.7:0/3567428226 >> v2:172.21.15.188:3300/0 conn(0x7fbb4c1416c0 0x7fbb4c0cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:49.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:49.857+0000 7fbb511b4700 1 -- 172.21.15.7:0/3567428226 >> v2:172.21.15.7:3300/0 conn(0x7fbb4c141a90 msgr2=0x7fbb4c0cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-01-27T02:31:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:49.857+0000 7fbb511b4700 1 -- 172.21.15.7:0/3567428226 >> v2:172.21.15.7:3300/0 conn(0x7fbb4c141a90 msgr2=0x7fbb4c0cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-27T02:31:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:49.857+0000 7fbb511b4700 1 --2- 172.21.15.7:0/3567428226 >> v2:172.21.15.7:3300/0 conn(0x7fbb4c141a90 0x7fbb4c0cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-01-27T02:31:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:49.857+0000 7fbb511b4700 1 --2- 172.21.15.7:0/3567428226 >> v2:172.21.15.7:3300/0 conn(0x7fbb4c141a90 0x7fbb4c0cbc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-27T02:31:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:49.857+0000 7fbb519b5700 1 -- 172.21.15.7:0/3567428226 >> v2:172.21.15.188:3300/0 conn(0x7fbb4c1416c0 msgr2=0x7fbb4c0cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-27T02:31:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:49.857+0000 7fbb519b5700 1 -- 172.21.15.7:0/3567428226 >> v2:172.21.15.188:3300/0 conn(0x7fbb4c1416c0 msgr2=0x7fbb4c0cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-27T02:31:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:49.857+0000 7fbb519b5700 1 --2- 172.21.15.7:0/3567428226 >> v2:172.21.15.188:3300/0 conn(0x7fbb4c1416c0 0x7fbb4c0cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-01-27T02:31:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:49.857+0000 7fbb519b5700 1 --2- 172.21.15.7:0/3567428226 >> v2:172.21.15.188:3300/0 conn(0x7fbb4c1416c0 0x7fbb4c0cb6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-27T02:31:50.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.033+0000 7fbb509b3700 1 -- 172.21.15.7:0/3567428226 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625344880 0 0) 0x7fbb4c15c7d0 con 0x7fbb4c0cab00 2024-01-27T02:31:50.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.033+0000 7fbb509b3700 1 -- 172.21.15.7:0/3567428226 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb34003570 con 0x7fbb4c0cab00 2024-01-27T02:31:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.036+0000 7fbb509b3700 1 -- 172.21.15.7:0/3567428226 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1112248033 0 0) 0x7fbb34003570 con 0x7fbb4c0cab00 2024-01-27T02:31:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.036+0000 7fbb509b3700 1 -- 172.21.15.7:0/3567428226 >> v2:172.21.15.188:3300/0 conn(0x7fbb4c1416c0 msgr2=0x7fbb4c0cb6e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-27T02:31:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.036+0000 7fbb509b3700 1 --2- 172.21.15.7:0/3567428226 >> v2:172.21.15.188:3300/0 conn(0x7fbb4c1416c0 0x7fbb4c0cb6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.036+0000 7fbb509b3700 1 -- 172.21.15.7:0/3567428226 >> v2:172.21.15.7:3300/0 conn(0x7fbb4c141a90 msgr2=0x7fbb4c0cbc20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-27T02:31:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.036+0000 7fbb509b3700 1 --2- 172.21.15.7:0/3567428226 >> v2:172.21.15.7:3300/0 conn(0x7fbb4c141a90 0x7fbb4c0cbc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.036+0000 7fbb509b3700 1 -- 172.21.15.7:0/3567428226 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4c15bbc0 con 0x7fbb4c0cab00 2024-01-27T02:31:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.036+0000 7fbb509b3700 1 -- 172.21.15.7:0/3567428226 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7fbb3c001470 con 0x7fbb4c0cab00 2024-01-27T02:31:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.036+0000 7fbb509b3700 1 -- 172.21.15.7:0/3567428226 >> v1:172.21.15.188:6789/0 conn(0x7fbb4c0cab00 legacy=0x7fbb4c0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.036+0000 7fbb509b3700 1 --2- 172.21.15.7:0/3567428226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb34003b30 0x7fbb34003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.036+0000 7fbb509b3700 1 --2- 172.21.15.7:0/3567428226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb34004810 0x7fbb34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.036+0000 7fbb509b3700 1 --2- 172.21.15.7:0/3567428226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb340052e0 0x7fbb3400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.036+0000 7fbb509b3700 1 -- 172.21.15.7:0/3567428226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c0cd9c0 con 0x7fbb340052e0 2024-01-27T02:31:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.036+0000 7fbb509b3700 1 -- 172.21.15.7:0/3567428226 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c0c9640 con 0x7fbb34004810 2024-01-27T02:31:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.036+0000 7fbb509b3700 1 -- 172.21.15.7:0/3567428226 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3400c320 con 0x7fbb34003b30 2024-01-27T02:31:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.036+0000 7fbb511b4700 1 --2- 172.21.15.7:0/3567428226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb34004810 0x7fbb34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.036+0000 7fbb521b6700 1 --2- 172.21.15.7:0/3567428226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb340052e0 0x7fbb3400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.036+0000 7fbb519b5700 1 --2- 172.21.15.7:0/3567428226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb34003b30 0x7fbb34003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:50.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.037+0000 7fbb511b4700 1 -- 172.21.15.7:0/3567428226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb34003b30 msgr2=0x7fbb34003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:31:50.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.037+0000 7fbb511b4700 1 --2- 172.21.15.7:0/3567428226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb34003b30 0x7fbb34003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:50.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.037+0000 7fbb511b4700 1 -- 172.21.15.7:0/3567428226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb340052e0 msgr2=0x7fbb3400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:31:50.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.037+0000 7fbb511b4700 1 --2- 172.21.15.7:0/3567428226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb340052e0 0x7fbb3400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:50.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.037+0000 7fbb511b4700 1 -- 172.21.15.7:0/3567428226 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb34003570 con 0x7fbb34004810 2024-01-27T02:31:50.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.037+0000 7fbb521b6700 1 --2- 172.21.15.7:0/3567428226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb340052e0 0x7fbb3400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:31:50.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.037+0000 7fbb521b6700 1 -- 172.21.15.7:0/3567428226 reap_dead start 2024-01-27T02:31:50.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.037+0000 7fbb511b4700 1 --2- 172.21.15.7:0/3567428226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb34004810 0x7fbb34004c80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbb40009060 tx=0x7fbb40008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e939c003f8175ca server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:31:50.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.038+0000 7fbb509b3700 1 -- 172.21.15.7:0/3567428226 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4000c070 con 0x7fbb34004810 2024-01-27T02:31:50.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.038+0000 7fbb509b3700 1 -- 172.21.15.7:0/3567428226 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4000e5e0 con 0x7fbb34004810 2024-01-27T02:31:50.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.038+0000 7fbb509b3700 1 -- 172.21.15.7:0/3567428226 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4000e790 con 0x7fbb34004810 2024-01-27T02:31:50.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.039+0000 7fbb53c19700 1 -- 172.21.15.7:0/3567428226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb34004810 msgr2=0x7fbb34004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.039+0000 7fbb53c19700 1 --2- 172.21.15.7:0/3567428226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb34004810 0x7fbb34004c80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbb40009060 tx=0x7fbb40008040 comp rx=0 tx=0).stop 2024-01-27T02:31:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.039+0000 7fbb53c19700 1 -- 172.21.15.7:0/3567428226 shutdown_connections 2024-01-27T02:31:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.039+0000 7fbb53c19700 1 --2- 172.21.15.7:0/3567428226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb34004810 0x7fbb34004c80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.039+0000 7fbb53c19700 1 -- 172.21.15.7:0/3567428226 >> 172.21.15.7:0/3567428226 conn(0x7fbb4c063580 msgr2=0x7fbb4c063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:31:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.039+0000 7fbb53c19700 1 -- 172.21.15.7:0/3567428226 shutdown_connections 2024-01-27T02:31:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.040+0000 7fbb53c19700 1 -- 172.21.15.7:0/3567428226 wait complete. 2024-01-27T02:31:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.041+0000 7fbb53c19700 1 Processor -- start 2024-01-27T02:31:50.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.041+0000 7fbb53c19700 1 -- start start 2024-01-27T02:31:50.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.042+0000 7fbb53c19700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb34004810 0x7fbb4c1575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:50.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.042+0000 7fbb53c19700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb4c158490 0x7fbb4c0ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:50.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.042+0000 7fbb53c19700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c0ce6b0 0x7fbb4c159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:50.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.042+0000 7fbb53c19700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c1597a0 con 0x7fbb4c0ce6b0 2024-01-27T02:31:50.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.042+0000 7fbb53c19700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c158900 con 0x7fbb34004810 2024-01-27T02:31:50.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.042+0000 7fbb53c19700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c0ceb20 con 0x7fbb4c158490 2024-01-27T02:31:50.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.042+0000 7fbb519b5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb34004810 0x7fbb4c1575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:50.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.042+0000 7fbb519b5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb34004810 0x7fbb4c1575f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:54738/0 (socket says 172.21.15.7:54738) 2024-01-27T02:31:50.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.042+0000 7fbb519b5700 1 -- 172.21.15.7:0/3385973145 learned_addr learned my addr 172.21.15.7:0/3385973145 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:31:50.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.043+0000 7fbb511b4700 1 --2- 172.21.15.7:0/3385973145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb4c158490 0x7fbb4c0ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:50.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.043+0000 7fbb521b6700 1 --2- 172.21.15.7:0/3385973145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c0ce6b0 0x7fbb4c159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:50.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.043+0000 7fbb519b5700 1 -- 172.21.15.7:0/3385973145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb4c158490 msgr2=0x7fbb4c0ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:50.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.043+0000 7fbb519b5700 1 --2- 172.21.15.7:0/3385973145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb4c158490 0x7fbb4c0ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:50.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.043+0000 7fbb519b5700 1 -- 172.21.15.7:0/3385973145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c0ce6b0 msgr2=0x7fbb4c159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:50.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.043+0000 7fbb519b5700 1 --2- 172.21.15.7:0/3385973145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c0ce6b0 0x7fbb4c159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:50.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.043+0000 7fbb519b5700 1 -- 172.21.15.7:0/3385973145 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4c158b20 con 0x7fbb34004810 2024-01-27T02:31:50.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.043+0000 7fbb511b4700 1 --2- 172.21.15.7:0/3385973145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb4c158490 0x7fbb4c0ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:31:50.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.043+0000 7fbb521b6700 1 --2- 172.21.15.7:0/3385973145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c0ce6b0 0x7fbb4c159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:31:50.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.043+0000 7fbb519b5700 1 --2- 172.21.15.7:0/3385973145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb34004810 0x7fbb4c1575f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400f560 tx=0x7fbb44009770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:31:50.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.043+0000 7fbb49ffb700 1 -- 172.21.15.7:0/3385973145 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb440146d0 con 0x7fbb34004810 2024-01-27T02:31:50.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.043+0000 7fbb49ffb700 1 -- 172.21.15.7:0/3385973145 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb44014830 con 0x7fbb34004810 2024-01-27T02:31:50.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.043+0000 7fbb53c19700 1 -- 172.21.15.7:0/3385973145 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb4c158e00 con 0x7fbb34004810 2024-01-27T02:31:50.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.044+0000 7fbb49ffb700 1 -- 172.21.15.7:0/3385973145 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb44014bd0 con 0x7fbb34004810 2024-01-27T02:31:50.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.044+0000 7fbb53c19700 1 -- 172.21.15.7:0/3385973145 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb4c1593c0 con 0x7fbb34004810 2024-01-27T02:31:50.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.044+0000 7fbb49ffb700 1 -- 172.21.15.7:0/3385973145 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fbb44003020 con 0x7fbb34004810 2024-01-27T02:31:50.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.044+0000 7fbb49ffb700 1 -- 172.21.15.7:0/3385973145 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fbb4401ddf0 con 0x7fbb34004810 2024-01-27T02:31:50.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.045+0000 7fbb53c19700 1 -- 172.21.15.7:0/3385973145 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb4c159a10 con 0x7fbb34004810 2024-01-27T02:31:50.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.048+0000 7fbb49ffb700 1 -- 172.21.15.7:0/3385973145 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fbb44018030 con 0x7fbb34004810 2024-01-27T02:31:50.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.129+0000 7fbb53c19700 1 -- 172.21.15.7:0/3385973145 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fbb4c0c9ca0 con 0x7fbb34004810 2024-01-27T02:31:50.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.132+0000 7fbb49ffb700 1 -- 172.21.15.7:0/3385973145 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fbb4401d460 con 0x7fbb34004810 2024-01-27T02:31:50.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.133+0000 7fbb53c19700 1 -- 172.21.15.7:0/3385973145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb34004810 msgr2=0x7fbb4c1575f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:50.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.133+0000 7fbb53c19700 1 --2- 172.21.15.7:0/3385973145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb34004810 0x7fbb4c1575f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400f560 tx=0x7fbb44009770 comp rx=0 tx=0).stop 2024-01-27T02:31:50.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.133+0000 7fbb53c19700 1 -- 172.21.15.7:0/3385973145 shutdown_connections 2024-01-27T02:31:50.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.133+0000 7fbb53c19700 1 --2- 172.21.15.7:0/3385973145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c0ce6b0 0x7fbb4c159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:50.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.133+0000 7fbb53c19700 1 --2- 172.21.15.7:0/3385973145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb34004810 0x7fbb4c1575f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:50.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.133+0000 7fbb53c19700 1 --2- 172.21.15.7:0/3385973145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb4c158490 0x7fbb4c0ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:50.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.133+0000 7fbb53c19700 1 -- 172.21.15.7:0/3385973145 >> 172.21.15.7:0/3385973145 conn(0x7fbb4c063580 msgr2=0x7fbb4c0caef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:31:50.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.133+0000 7fbb53c19700 1 -- 172.21.15.7:0/3385973145 shutdown_connections 2024-01-27T02:31:50.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:50.133+0000 7fbb53c19700 1 -- 172.21.15.7:0/3385973145 wait complete. 2024-01-27T02:31:50.135 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2024-01-27T02:31:50.147 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-01-27T02:31:50.147 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:50.148 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-01-27T02:31:50.178 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:50.178 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-01-27T02:31:50.248 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:50.248 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-01-27T02:31:50.318 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-27T02:31:50.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-01-27T02:31:50.387 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:31:50.388 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-01-27T02:31:50.395 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:50.395+0000 7ff197f771c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-27T02:31:50.414 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:31:50.414 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-01-27T02:31:50.439 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:50.439+0000 7fa51bf991c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-27T02:31:50.477 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:50.477+0000 7ff197f771c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-27T02:31:50.481 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:31:50.481 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-01-27T02:31:50.528 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:50.527+0000 7fa51bf991c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-27T02:31:50.547 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-27T02:31:50.548 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-01-27T02:31:50.552 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:50.552+0000 7ff197f771c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-27T02:31:50.577 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 0 2024-01-27T02:31:50.609 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:50.609+0000 7fa51bf991c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-27T02:31:50.624 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:50.624+0000 7ff197f771c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-27T02:31:50.687 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:50.686+0000 7fa51bf991c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-27T02:31:50.735 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.735+0000 7ff9a348b700 1 Processor -- start 2024-01-27T02:31:50.736 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.736+0000 7ff9a348b700 1 -- start start 2024-01-27T02:31:50.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.736+0000 7ff9a348b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff99c15a180 0x7ff99c0c3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:50.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.736+0000 7ff9a348b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff99c0ca3b0 0x7ff99c0aef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:50.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.736+0000 7ff9a348b700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff99c0b1450 con 0x7ff99c15a550 2024-01-27T02:31:50.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.736+0000 7ff9a348b700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff99c0b18e0 con 0x7ff99c0ca3b0 2024-01-27T02:31:50.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.736+0000 7ff9a348b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c0b0840 con 0x7ff99c15a180 2024-01-27T02:31:50.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.737+0000 7ff9a1227700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff99c15a180 0x7ff99c0c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:50.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.737+0000 7ff9a1a28700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff99c0ca3b0 0x7ff99c0aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:50.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.737+0000 7ff9a1a28700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff99c0ca3b0 0x7ff99c0aef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:55858/0 (socket says 172.21.15.188:55858) 2024-01-27T02:31:50.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.737+0000 7ff9a1227700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff99c15a180 0x7ff99c0c3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:51106/0 (socket says 172.21.15.188:51106) 2024-01-27T02:31:50.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.737+0000 7ff9a1a28700 1 -- 172.21.15.188:0/2520077375 learned_addr learned my addr 172.21.15.188:0/2520077375 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-27T02:31:50.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.737+0000 7ff9a1227700 1 -- 172.21.15.188:0/2520077375 >> v2:172.21.15.188:3300/0 conn(0x7ff99c0ca3b0 msgr2=0x7ff99c0aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:31:50.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.737+0000 7ff9a1227700 1 --2- 172.21.15.188:0/2520077375 >> v2:172.21.15.188:3300/0 conn(0x7ff99c0ca3b0 0x7ff99c0aef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:50.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.737+0000 7ff9a1227700 1 -- 172.21.15.188:0/2520077375 >> v1:172.21.15.188:6789/0 conn(0x7ff99c15a550 legacy=0x7ff99c0c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:50.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.737+0000 7ff9a1227700 1 -- 172.21.15.188:0/2520077375 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c0b0b20 con 0x7ff99c15a180 2024-01-27T02:31:50.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.737+0000 7ff993fff700 1 -- 172.21.15.188:0/2520077375 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3398880124 0 0) 0x7ff99c0b1450 con 0x7ff99c15a550 2024-01-27T02:31:50.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.737+0000 7ff9a1a28700 1 --2- 172.21.15.188:0/2520077375 >> v2:172.21.15.188:3300/0 conn(0x7ff99c0ca3b0 0x7ff99c0aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:31:50.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.737+0000 7ff9a1227700 1 --2- 172.21.15.188:0/2520077375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff99c15a180 0x7ff99c0c3750 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff9980098d0 tx=0x7ff998009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=215eda72af346220 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:31:50.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.737+0000 7ff993fff700 1 -- 172.21.15.188:0/2520077375 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998005c10 con 0x7ff99c15a180 2024-01-27T02:31:50.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.737+0000 7ff993fff700 1 -- 172.21.15.188:0/2520077375 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff998005d70 con 0x7ff99c15a180 2024-01-27T02:31:50.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.737+0000 7ff993fff700 1 -- 172.21.15.188:0/2520077375 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998006280 con 0x7ff99c15a180 2024-01-27T02:31:50.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.738+0000 7ff9a348b700 1 -- 172.21.15.188:0/2520077375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff99c15a180 msgr2=0x7ff99c0c3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:50.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.738+0000 7ff9a348b700 1 --2- 172.21.15.188:0/2520077375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff99c15a180 0x7ff99c0c3750 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff9980098d0 tx=0x7ff998009120 comp rx=0 tx=0).stop 2024-01-27T02:31:50.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.738+0000 7ff9a348b700 1 -- 172.21.15.188:0/2520077375 shutdown_connections 2024-01-27T02:31:50.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.738+0000 7ff9a348b700 1 --2- 172.21.15.188:0/2520077375 >> v2:172.21.15.188:3300/0 conn(0x7ff99c0ca3b0 0x7ff99c0aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:50.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.738+0000 7ff9a348b700 1 --2- 172.21.15.188:0/2520077375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff99c15a180 0x7ff99c0c3750 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:50.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.738+0000 7ff9a348b700 1 -- 172.21.15.188:0/2520077375 >> 172.21.15.188:0/2520077375 conn(0x7ff99c0679b0 msgr2=0x7ff99c067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:31:50.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.738+0000 7ff9a348b700 1 -- 172.21.15.188:0/2520077375 shutdown_connections 2024-01-27T02:31:50.741 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.738+0000 7ff9a348b700 1 -- 172.21.15.188:0/2520077375 wait complete. 2024-01-27T02:31:50.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.739+0000 7ff9a348b700 1 Processor -- start 2024-01-27T02:31:50.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.739+0000 7ff9a348b700 1 -- start start 2024-01-27T02:31:50.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.740+0000 7ff9a348b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff99c0ca3b0 0x7ff99c1583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:50.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.740+0000 7ff9a348b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff99c15a180 0x7ff99c15c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:50.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.740+0000 7ff9a348b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff99c15a550 0x7ff99c06a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:50.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.740+0000 7ff9a348b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c098860 con 0x7ff99c15a180 2024-01-27T02:31:50.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.740+0000 7ff9a348b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c097ae0 con 0x7ff99c0ca3b0 2024-01-27T02:31:50.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.740+0000 7ff9a348b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c097d80 con 0x7ff99c15a550 2024-01-27T02:31:50.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.740+0000 7ff9a1227700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff99c0ca3b0 0x7ff99c1583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:50.743 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.740+0000 7ff9a1a28700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff99c15a550 0x7ff99c06a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:50.744 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.740+0000 7ff9a1227700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff99c0ca3b0 0x7ff99c1583f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:55860/0 (socket says 172.21.15.188:55860) 2024-01-27T02:31:50.744 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.740+0000 7ff9a1a28700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff99c15a550 0x7ff99c06a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:51120/0 (socket says 172.21.15.188:51120) 2024-01-27T02:31:50.744 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.740+0000 7ff9a1227700 1 -- 172.21.15.188:0/1249040978 learned_addr learned my addr 172.21.15.188:0/1249040978 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-27T02:31:50.744 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.740+0000 7ff9a0a26700 1 --2- 172.21.15.188:0/1249040978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff99c15a180 0x7ff99c15c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:50.744 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.740+0000 7ff9a1227700 1 -- 172.21.15.188:0/1249040978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff99c15a550 msgr2=0x7ff99c06a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:50.744 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.740+0000 7ff9a1227700 1 --2- 172.21.15.188:0/1249040978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff99c15a550 0x7ff99c06a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:50.744 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.740+0000 7ff9a1227700 1 -- 172.21.15.188:0/1249040978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff99c15a180 msgr2=0x7ff99c15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:50.744 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.740+0000 7ff9a1227700 1 --2- 172.21.15.188:0/1249040978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff99c15a180 0x7ff99c15c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:50.744 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.740+0000 7ff9a1227700 1 -- 172.21.15.188:0/1249040978 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff994009ae0 con 0x7ff99c0ca3b0 2024-01-27T02:31:50.744 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.740+0000 7ff9a1a28700 1 --2- 172.21.15.188:0/1249040978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff99c15a550 0x7ff99c06a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:31:50.744 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.741+0000 7ff9a1227700 1 --2- 172.21.15.188:0/1249040978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff99c0ca3b0 0x7ff99c1583f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff998009f70 tx=0x7ff998016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:31:50.744 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.741+0000 7ff9917fa700 1 -- 172.21.15.188:0/1249040978 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99801d780 con 0x7ff99c0ca3b0 2024-01-27T02:31:50.744 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.741+0000 7ff9917fa700 1 -- 172.21.15.188:0/1249040978 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99801d8e0 con 0x7ff99c0ca3b0 2024-01-27T02:31:50.744 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.741+0000 7ff9a348b700 1 -- 172.21.15.188:0/1249040978 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9980093c0 con 0x7ff99c0ca3b0 2024-01-27T02:31:50.745 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.741+0000 7ff9917fa700 1 -- 172.21.15.188:0/1249040978 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99801dca0 con 0x7ff99c0ca3b0 2024-01-27T02:31:50.745 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.741+0000 7ff9a348b700 1 -- 172.21.15.188:0/1249040978 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99c098340 con 0x7ff99c0ca3b0 2024-01-27T02:31:50.745 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.742+0000 7ff9917fa700 1 -- 172.21.15.188:0/1249040978 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7ff99801b910 con 0x7ff99c0ca3b0 2024-01-27T02:31:50.745 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.742+0000 7ff9917fa700 1 -- 172.21.15.188:0/1249040978 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7ff998011770 con 0x7ff99c0ca3b0 2024-01-27T02:31:50.745 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.742+0000 7ff9a348b700 1 -- 172.21.15.188:0/1249040978 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff99c0b0120 con 0x7ff99c0ca3b0 2024-01-27T02:31:50.745 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.744+0000 7ff9917fa700 1 -- 172.21.15.188:0/1249040978 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7ff998033060 con 0x7ff99c0ca3b0 2024-01-27T02:31:50.802 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:50.802+0000 7ff197f771c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-27T02:31:50.827 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.827+0000 7ff9a348b700 1 -- 172.21.15.188:0/1249040978 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0} v 0) v1 -- 0x7ff99c072060 con 0x7ff99c0ca3b0 2024-01-27T02:31:50.830 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.830+0000 7ff9917fa700 1 -- 172.21.15.188:0/1249040978 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff99801bda0 con 0x7ff99c0ca3b0 2024-01-27T02:31:50.830 INFO:teuthology.orchestra.run.smithi188.stdout:0 2024-01-27T02:31:50.832 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.831+0000 7ff9a348b700 1 -- 172.21.15.188:0/1249040978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff99c0ca3b0 msgr2=0x7ff99c1583f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:50.832 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.831+0000 7ff9a348b700 1 --2- 172.21.15.188:0/1249040978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff99c0ca3b0 0x7ff99c1583f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff998009f70 tx=0x7ff998016000 comp rx=0 tx=0).stop 2024-01-27T02:31:50.832 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.831+0000 7ff9a348b700 1 -- 172.21.15.188:0/1249040978 shutdown_connections 2024-01-27T02:31:50.832 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.831+0000 7ff9a348b700 1 --2- 172.21.15.188:0/1249040978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff99c15a550 0x7ff99c06a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:50.832 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.831+0000 7ff9a348b700 1 --2- 172.21.15.188:0/1249040978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff99c0ca3b0 0x7ff99c1583f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:50.832 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.831+0000 7ff9a348b700 1 --2- 172.21.15.188:0/1249040978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff99c15a180 0x7ff99c15c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:50.832 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.831+0000 7ff9a348b700 1 -- 172.21.15.188:0/1249040978 >> 172.21.15.188:0/1249040978 conn(0x7ff99c0679b0 msgr2=0x7ff99c0af050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:31:50.832 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.831+0000 7ff9a348b700 1 -- 172.21.15.188:0/1249040978 shutdown_connections 2024-01-27T02:31:50.832 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:50.832+0000 7ff9a348b700 1 -- 172.21.15.188:0/1249040978 wait complete. 2024-01-27T02:31:50.843 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 1 2024-01-27T02:31:50.877 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:50.877+0000 7fa51bf991c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-27T02:31:50.956 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:50.956+0000 7ff197f771c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-27T02:31:51.001 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.001+0000 7fd62a4e5700 1 Processor -- start 2024-01-27T02:31:51.002 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.001+0000 7fd62a4e5700 1 -- start start 2024-01-27T02:31:51.005 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.003+0000 7fd62a4e5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd624143670 0x7fd624063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.005 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.003+0000 7fd62a4e5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd624143a40 0x7fd624063ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.005 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.003+0000 7fd62a4e5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6240cc290 con 0x7fd6240aa8b0 2024-01-27T02:31:51.005 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.003+0000 7fd62a4e5700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6240cb960 con 0x7fd624143670 2024-01-27T02:31:51.005 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.003+0000 7fd62a4e5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6240cbc40 con 0x7fd624143a40 2024-01-27T02:31:51.005 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.003+0000 7fd628a82700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd624143a40 0x7fd624063ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.005 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.003+0000 7fd628a82700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd624143a40 0x7fd624063ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:51126/0 (socket says 172.21.15.188:51126) 2024-01-27T02:31:51.005 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.003+0000 7fd628a82700 1 -- 172.21.15.188:0/2462169295 learned_addr learned my addr 172.21.15.188:0/2462169295 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-27T02:31:51.005 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.003+0000 7fd623fff700 1 --2- 172.21.15.188:0/2462169295 >> v2:172.21.15.7:3300/0 conn(0x7fd624143670 0x7fd624063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.006 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.003+0000 7fd628a82700 1 -- 172.21.15.188:0/2462169295 >> v2:172.21.15.7:3300/0 conn(0x7fd624143670 msgr2=0x7fd624063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:31:51.006 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.003+0000 7fd628a82700 1 --2- 172.21.15.188:0/2462169295 >> v2:172.21.15.7:3300/0 conn(0x7fd624143670 0x7fd624063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.006 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.003+0000 7fd628a82700 1 -- 172.21.15.188:0/2462169295 >> v1:172.21.15.7:6789/0 conn(0x7fd6240aa8b0 legacy=0x7fd6240c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.006 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.003+0000 7fd628a82700 1 -- 172.21.15.188:0/2462169295 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6240cbf20 con 0x7fd624143a40 2024-01-27T02:31:51.006 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.003+0000 7fd628a82700 1 --2- 172.21.15.188:0/2462169295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd624143a40 0x7fd624063ac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009f20 tx=0x7fd61c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12d098e5ae89f7ca server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:31:51.006 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.003+0000 7fd622ffd700 1 -- 172.21.15.188:0/2462169295 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c00e070 con 0x7fd624143a40 2024-01-27T02:31:51.006 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.004+0000 7fd622ffd700 1 -- 172.21.15.188:0/2462169295 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61c00b9d0 con 0x7fd624143a40 2024-01-27T02:31:51.006 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.004+0000 7fd622ffd700 1 -- 172.21.15.188:0/2462169295 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0092e0 con 0x7fd624143a40 2024-01-27T02:31:51.006 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.004+0000 7fd62a4e5700 1 -- 172.21.15.188:0/2462169295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd624143a40 msgr2=0x7fd624063ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.006 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.004+0000 7fd62a4e5700 1 --2- 172.21.15.188:0/2462169295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd624143a40 0x7fd624063ac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009f20 tx=0x7fd61c009840 comp rx=0 tx=0).stop 2024-01-27T02:31:51.006 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.004+0000 7fd62a4e5700 1 -- 172.21.15.188:0/2462169295 shutdown_connections 2024-01-27T02:31:51.006 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.004+0000 7fd62a4e5700 1 --2- 172.21.15.188:0/2462169295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd624143a40 0x7fd624063ac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.006 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.004+0000 7fd62a4e5700 1 --2- 172.21.15.188:0/2462169295 >> v2:172.21.15.7:3300/0 conn(0x7fd624143670 0x7fd624063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.006 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.004+0000 7fd62a4e5700 1 -- 172.21.15.188:0/2462169295 >> 172.21.15.188:0/2462169295 conn(0x7fd6240cdbf0 msgr2=0x7fd6240cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:31:51.007 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.004+0000 7fd62a4e5700 1 -- 172.21.15.188:0/2462169295 shutdown_connections 2024-01-27T02:31:51.007 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.005+0000 7fd62a4e5700 1 -- 172.21.15.188:0/2462169295 wait complete. 2024-01-27T02:31:51.009 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.006+0000 7fd62a4e5700 1 Processor -- start 2024-01-27T02:31:51.009 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.006+0000 7fd62a4e5700 1 -- start start 2024-01-27T02:31:51.009 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.006+0000 7fd62a4e5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6240aa8b0 0x7fd6240c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.009 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.006+0000 7fd62a4e5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd624143670 0x7fd6240b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.009 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.006+0000 7fd62a4e5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd624143a40 0x7fd62415fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.009 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.006+0000 7fd62a4e5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6240c4e40 con 0x7fd624143670 2024-01-27T02:31:51.009 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.006+0000 7fd62a4e5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6240c5120 con 0x7fd624143a40 2024-01-27T02:31:51.010 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.006+0000 7fd62a4e5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6240c5400 con 0x7fd6240aa8b0 2024-01-27T02:31:51.010 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.007+0000 7fd623fff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6240aa8b0 0x7fd6240c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.010 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.007+0000 7fd628a82700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd624143a40 0x7fd62415fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.010 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.007+0000 7fd623fff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6240aa8b0 0x7fd6240c5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:51138/0 (socket says 172.21.15.188:51138) 2024-01-27T02:31:51.010 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.007+0000 7fd623fff700 1 -- 172.21.15.188:0/339206984 learned_addr learned my addr 172.21.15.188:0/339206984 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-27T02:31:51.010 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.007+0000 7fd6237fe700 1 --2- 172.21.15.188:0/339206984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd624143670 0x7fd6240b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.010 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.007+0000 7fd628a82700 1 -- 172.21.15.188:0/339206984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6240aa8b0 msgr2=0x7fd6240c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.010 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.007+0000 7fd628a82700 1 --2- 172.21.15.188:0/339206984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6240aa8b0 0x7fd6240c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.010 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.007+0000 7fd628a82700 1 -- 172.21.15.188:0/339206984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd624143670 msgr2=0x7fd6240b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.010 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.007+0000 7fd628a82700 1 --2- 172.21.15.188:0/339206984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd624143670 0x7fd6240b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.010 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.007+0000 7fd628a82700 1 -- 172.21.15.188:0/339206984 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c009ae0 con 0x7fd624143a40 2024-01-27T02:31:51.010 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.007+0000 7fd628a82700 1 --2- 172.21.15.188:0/339206984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd624143a40 0x7fd62415fdd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd61c015000 tx=0x7fd61c018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:31:51.010 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.007+0000 7fd60bfff700 1 -- 172.21.15.188:0/339206984 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c007ac0 con 0x7fd624143a40 2024-01-27T02:31:51.010 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.007+0000 7fd60bfff700 1 -- 172.21.15.188:0/339206984 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61c007c20 con 0x7fd624143a40 2024-01-27T02:31:51.011 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.007+0000 7fd62a4e5700 1 -- 172.21.15.188:0/339206984 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6240c56e0 con 0x7fd624143a40 2024-01-27T02:31:51.011 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.007+0000 7fd60bfff700 1 -- 172.21.15.188:0/339206984 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c01d420 con 0x7fd624143a40 2024-01-27T02:31:51.011 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.007+0000 7fd62a4e5700 1 -- 172.21.15.188:0/339206984 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6240c7a80 con 0x7fd624143a40 2024-01-27T02:31:51.011 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.007+0000 7fd60bfff700 1 -- 172.21.15.188:0/339206984 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fd61c003040 con 0x7fd624143a40 2024-01-27T02:31:51.011 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.007+0000 7fd60bfff700 1 -- 172.21.15.188:0/339206984 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fd61c028820 con 0x7fd624143a40 2024-01-27T02:31:51.011 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.008+0000 7fd62a4e5700 1 -- 172.21.15.188:0/339206984 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6240c5540 con 0x7fd624143a40 2024-01-27T02:31:51.011 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.010+0000 7fd60bfff700 1 -- 172.21.15.188:0/339206984 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd6240c5540 con 0x7fd624143a40 2024-01-27T02:31:51.030 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:51.030+0000 7ff197f771c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-27T02:31:51.042 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:51.042+0000 7fa51bf991c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-27T02:31:51.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.094+0000 7fd62a4e5700 1 -- 172.21.15.188:0/339206984 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1} v 0) v1 -- 0x7fd6240c5260 con 0x7fd624143a40 2024-01-27T02:31:51.097 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.097+0000 7fd60bfff700 1 -- 172.21.15.188:0/339206984 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd6240c5260 con 0x7fd624143a40 2024-01-27T02:31:51.097 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-01-27T02:31:51.098 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.098+0000 7fd62a4e5700 1 -- 172.21.15.188:0/339206984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd624143a40 msgr2=0x7fd62415fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.098 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.098+0000 7fd62a4e5700 1 --2- 172.21.15.188:0/339206984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd624143a40 0x7fd62415fdd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd61c015000 tx=0x7fd61c018000 comp rx=0 tx=0).stop 2024-01-27T02:31:51.098 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.098+0000 7fd62a4e5700 1 -- 172.21.15.188:0/339206984 shutdown_connections 2024-01-27T02:31:51.098 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.098+0000 7fd62a4e5700 1 --2- 172.21.15.188:0/339206984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd624143a40 0x7fd62415fdd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.098 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.098+0000 7fd62a4e5700 1 --2- 172.21.15.188:0/339206984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6240aa8b0 0x7fd6240c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.099 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.098+0000 7fd62a4e5700 1 --2- 172.21.15.188:0/339206984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd624143670 0x7fd6240b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.099 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.098+0000 7fd62a4e5700 1 -- 172.21.15.188:0/339206984 >> 172.21.15.188:0/339206984 conn(0x7fd6240cdbf0 msgr2=0x7fd6240ab4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:31:51.099 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.098+0000 7fd62a4e5700 1 -- 172.21.15.188:0/339206984 shutdown_connections 2024-01-27T02:31:51.099 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.098+0000 7fd62a4e5700 1 -- 172.21.15.188:0/339206984 wait complete. 2024-01-27T02:31:51.109 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 2 2024-01-27T02:31:51.120 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:51.120+0000 7fa51bf991c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-27T02:31:51.171 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:51.171+0000 7ff197f771c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-27T02:31:51.228 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.228+0000 7f00197da700 1 Processor -- start 2024-01-27T02:31:51.229 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.228+0000 7f00197da700 1 -- start start 2024-01-27T02:31:51.231 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.229+0000 7f00197da700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f001415a550 0x7f00140c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.231 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.229+0000 7f00197da700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f00140ca1e0 0x7f00140aef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.231 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.229+0000 7f00197da700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00140b1450 con 0x7f001415a180 2024-01-27T02:31:51.231 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.229+0000 7f00197da700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00140b18e0 con 0x7f00140ca1e0 2024-01-27T02:31:51.231 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.229+0000 7f00197da700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00140ca660 con 0x7f001415a550 2024-01-27T02:31:51.232 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.229+0000 7f00127fc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f001415a550 0x7f00140c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.232 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.229+0000 7f00137fe700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f00140ca1e0 0x7f00140aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.232 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.229+0000 7f00137fe700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f00140ca1e0 0x7f00140aef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:55870/0 (socket says 172.21.15.188:55870) 2024-01-27T02:31:51.232 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.229+0000 7f00127fc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f001415a550 0x7f00140c9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:51154/0 (socket says 172.21.15.188:51154) 2024-01-27T02:31:51.232 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.229+0000 7f00137fe700 1 -- 172.21.15.188:0/2521783717 learned_addr learned my addr 172.21.15.188:0/2521783717 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-27T02:31:51.232 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.229+0000 7f00127fc700 1 -- 172.21.15.188:0/2521783717 >> v2:172.21.15.188:3300/0 conn(0x7f00140ca1e0 msgr2=0x7f00140aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:31:51.232 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.229+0000 7f00127fc700 1 --2- 172.21.15.188:0/2521783717 >> v2:172.21.15.188:3300/0 conn(0x7f00140ca1e0 0x7f00140aef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.232 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.229+0000 7f00127fc700 1 -- 172.21.15.188:0/2521783717 >> v1:172.21.15.7:6789/0 conn(0x7f001415a180 legacy=0x7f00140c3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.232 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.229+0000 7f00127fc700 1 -- 172.21.15.188:0/2521783717 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00140b0840 con 0x7f001415a550 2024-01-27T02:31:51.232 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.229+0000 7f00137fe700 1 --2- 172.21.15.188:0/2521783717 >> v2:172.21.15.188:3300/0 conn(0x7f00140ca1e0 0x7f00140aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:31:51.232 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.230+0000 7f00127fc700 1 --2- 172.21.15.188:0/2521783717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f001415a550 0x7f00140c9ca0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f00080098e0 tx=0x7f00080091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48e40b18c39a0ff2 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:31:51.233 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.230+0000 7f0011ffb700 1 -- 172.21.15.188:0/2521783717 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000800b640 con 0x7f001415a550 2024-01-27T02:31:51.233 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.230+0000 7f0011ffb700 1 -- 172.21.15.188:0/2521783717 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000800b7a0 con 0x7f001415a550 2024-01-27T02:31:51.233 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.230+0000 7f0011ffb700 1 -- 172.21.15.188:0/2521783717 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000800bb40 con 0x7f001415a550 2024-01-27T02:31:51.233 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.230+0000 7f00197da700 1 -- 172.21.15.188:0/2521783717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f001415a550 msgr2=0x7f00140c9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.233 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.230+0000 7f00197da700 1 --2- 172.21.15.188:0/2521783717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f001415a550 0x7f00140c9ca0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f00080098e0 tx=0x7f00080091a0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.233 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.230+0000 7f00197da700 1 -- 172.21.15.188:0/2521783717 shutdown_connections 2024-01-27T02:31:51.233 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.230+0000 7f00197da700 1 --2- 172.21.15.188:0/2521783717 >> v2:172.21.15.188:3300/0 conn(0x7f00140ca1e0 0x7f00140aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.233 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.230+0000 7f00197da700 1 --2- 172.21.15.188:0/2521783717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f001415a550 0x7f00140c9ca0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.233 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.230+0000 7f00197da700 1 -- 172.21.15.188:0/2521783717 >> 172.21.15.188:0/2521783717 conn(0x7f00140679b0 msgr2=0x7f0014067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:31:51.233 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.230+0000 7f00197da700 1 -- 172.21.15.188:0/2521783717 shutdown_connections 2024-01-27T02:31:51.233 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.230+0000 7f00197da700 1 -- 172.21.15.188:0/2521783717 wait complete. 2024-01-27T02:31:51.236 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.232+0000 7f00197da700 1 Processor -- start 2024-01-27T02:31:51.236 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.232+0000 7f00197da700 1 -- start start 2024-01-27T02:31:51.236 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.232+0000 7f00197da700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00140ca1e0 0x7f0014158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.236 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.232+0000 7f00197da700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f001415a180 0x7f001415c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.236 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.232+0000 7f00197da700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f001415a550 0x7f001406f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.236 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.232+0000 7f00197da700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001415cae0 con 0x7f001415a550 2024-01-27T02:31:51.236 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.232+0000 7f00197da700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001415cdc0 con 0x7f00140ca1e0 2024-01-27T02:31:51.237 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.232+0000 7f00197da700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f001415d0a0 con 0x7f001415a180 2024-01-27T02:31:51.237 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.233+0000 7f00127fc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f001415a180 0x7f001415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.237 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.233+0000 7f0012ffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00140ca1e0 0x7f0014158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.237 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.233+0000 7f00127fc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f001415a180 0x7f001415c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:51164/0 (socket says 172.21.15.188:51164) 2024-01-27T02:31:51.237 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.233+0000 7f00127fc700 1 -- 172.21.15.188:0/3604153905 learned_addr learned my addr 172.21.15.188:0/3604153905 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-27T02:31:51.237 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.233+0000 7f00137fe700 1 --2- 172.21.15.188:0/3604153905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f001415a550 0x7f001406f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.237 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.233+0000 7f00127fc700 1 -- 172.21.15.188:0/3604153905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00140ca1e0 msgr2=0x7f0014158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.237 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.233+0000 7f00127fc700 1 --2- 172.21.15.188:0/3604153905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00140ca1e0 0x7f0014158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.237 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.233+0000 7f00127fc700 1 -- 172.21.15.188:0/3604153905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f001415a550 msgr2=0x7f001406f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.237 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.233+0000 7f00127fc700 1 --2- 172.21.15.188:0/3604153905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f001415a550 0x7f001406f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.237 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.233+0000 7f00127fc700 1 -- 172.21.15.188:0/3604153905 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c009ae0 con 0x7f001415a180 2024-01-27T02:31:51.238 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.233+0000 7f0012ffd700 1 --2- 172.21.15.188:0/3604153905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00140ca1e0 0x7f0014158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:31:51.238 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.233+0000 7f00127fc700 1 --2- 172.21.15.188:0/3604153905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f001415a180 0x7f001415c190 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0008009f80 tx=0x7f0008017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:31:51.238 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.233+0000 7efffaffd700 1 -- 172.21.15.188:0/3604153905 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000800bb20 con 0x7f001415a180 2024-01-27T02:31:51.238 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.233+0000 7efffaffd700 1 -- 172.21.15.188:0/3604153905 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0008000c90 con 0x7f001415a180 2024-01-27T02:31:51.238 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.233+0000 7efffaffd700 1 -- 172.21.15.188:0/3604153905 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0008011650 con 0x7f001415a180 2024-01-27T02:31:51.238 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.233+0000 7f00197da700 1 -- 172.21.15.188:0/3604153905 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0008009420 con 0x7f001415a180 2024-01-27T02:31:51.238 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.233+0000 7f00197da700 1 -- 172.21.15.188:0/3604153905 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0014158b60 con 0x7f001415a180 2024-01-27T02:31:51.238 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.234+0000 7efffaffd700 1 -- 172.21.15.188:0/3604153905 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f0008011c80 con 0x7f001415a180 2024-01-27T02:31:51.238 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.234+0000 7efffaffd700 1 -- 172.21.15.188:0/3604153905 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f0008025ce0 con 0x7f001415a180 2024-01-27T02:31:51.238 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.234+0000 7f00197da700 1 -- 172.21.15.188:0/3604153905 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001415d8a0 con 0x7f001415a180 2024-01-27T02:31:51.239 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.236+0000 7efffaffd700 1 -- 172.21.15.188:0/3604153905 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f0008029070 con 0x7f001415a180 2024-01-27T02:31:51.249 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:51.249+0000 7ff197f771c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-27T02:31:51.272 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:51.272+0000 7fa51bf991c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-27T02:31:51.320 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.320+0000 7f00197da700 1 -- 172.21.15.188:0/3604153905 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2} v 0) v1 -- 0x7f001415d6b0 con 0x7f001415a180 2024-01-27T02:31:51.323 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.323+0000 7efffaffd700 1 -- 172.21.15.188:0/3604153905 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0008015020 con 0x7f001415a180 2024-01-27T02:31:51.323 INFO:teuthology.orchestra.run.smithi188.stdout:2 2024-01-27T02:31:51.325 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.324+0000 7f00197da700 1 -- 172.21.15.188:0/3604153905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f001415a180 msgr2=0x7f001415c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.325 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.324+0000 7f00197da700 1 --2- 172.21.15.188:0/3604153905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f001415a180 0x7f001415c190 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0008009f80 tx=0x7f0008017000 comp rx=0 tx=0).stop 2024-01-27T02:31:51.325 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.324+0000 7f00197da700 1 -- 172.21.15.188:0/3604153905 shutdown_connections 2024-01-27T02:31:51.325 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.324+0000 7f00197da700 1 --2- 172.21.15.188:0/3604153905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f001415a550 0x7f001406f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.325 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.324+0000 7f00197da700 1 --2- 172.21.15.188:0/3604153905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00140ca1e0 0x7f0014158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.325 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.324+0000 7f00197da700 1 --2- 172.21.15.188:0/3604153905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f001415a180 0x7f001415c190 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.325 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.324+0000 7f00197da700 1 -- 172.21.15.188:0/3604153905 >> 172.21.15.188:0/3604153905 conn(0x7f00140679b0 msgr2=0x7f00141594f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:31:51.325 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.324+0000 7f00197da700 1 -- 172.21.15.188:0/3604153905 shutdown_connections 2024-01-27T02:31:51.325 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.324+0000 7f00197da700 1 -- 172.21.15.188:0/3604153905 wait complete. 2024-01-27T02:31:51.336 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 3 2024-01-27T02:31:51.356 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:51.356+0000 7fa51bf991c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-27T02:31:51.439 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:51.439+0000 7ff197f771c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-27T02:31:51.479 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.478+0000 7f3590fd1700 1 Processor -- start 2024-01-27T02:31:51.479 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.479+0000 7f3590fd1700 1 -- start start 2024-01-27T02:31:51.483 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.479+0000 7f3590fd1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f358c157250 0x7f358c0cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.483 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.479+0000 7f3590fd1700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f358c070d00 con 0x7f358c156e80 2024-01-27T02:31:51.483 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.480+0000 7f3590fd1700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f358c0c29c0 con 0x7f358c0c2120 2024-01-27T02:31:51.483 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.480+0000 7f3590fd1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f358c0c24f0 con 0x7f358c157250 2024-01-27T02:31:51.484 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.480+0000 7f358ad9d700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f358c0c2120 0x7f358c0aab10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:47312/0 (socket says 172.21.15.188:47312) 2024-01-27T02:31:51.484 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.480+0000 7f358ad9d700 1 -- 172.21.15.188:0/1346934616 learned_addr learned my addr 172.21.15.188:0/1346934616 (peer_addr_for_me v1:172.21.15.188:0/0) 2024-01-27T02:31:51.484 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.480+0000 7f3589d9b700 1 --2- 172.21.15.188:0/1346934616 >> v2:172.21.15.7:3300/0 conn(0x7f358c157250 0x7f358c0cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.484 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.480+0000 7f358959a700 1 -- 172.21.15.188:0/1346934616 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1850019031 0 0) 0x7f358c0c29c0 con 0x7f358c0c2120 2024-01-27T02:31:51.484 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.480+0000 7f358959a700 1 -- 172.21.15.188:0/1346934616 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3574003570 con 0x7f358c0c2120 2024-01-27T02:31:51.484 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.480+0000 7f358959a700 1 -- 172.21.15.188:0/1346934616 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1635657862 0 0) 0x7f3574003570 con 0x7f358c0c2120 2024-01-27T02:31:51.484 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.480+0000 7f358959a700 1 -- 172.21.15.188:0/1346934616 >> v2:172.21.15.7:3300/0 conn(0x7f358c157250 msgr2=0x7f358c0cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:31:51.484 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.480+0000 7f358959a700 1 --2- 172.21.15.188:0/1346934616 >> v2:172.21.15.7:3300/0 conn(0x7f358c157250 0x7f358c0cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.484 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.480+0000 7f358959a700 1 -- 172.21.15.188:0/1346934616 >> v1:172.21.15.7:6789/0 conn(0x7f358c156e80 legacy=0x7f358c0cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.484 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.480+0000 7f358959a700 1 -- 172.21.15.188:0/1346934616 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c0aa050 con 0x7f358c0c2120 2024-01-27T02:31:51.484 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.480+0000 7f358959a700 1 -- 172.21.15.188:0/1346934616 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f3584001810 con 0x7f358c0c2120 2024-01-27T02:31:51.484 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.480+0000 7f358959a700 1 -- 172.21.15.188:0/1346934616 >> v1:172.21.15.188:6789/0 conn(0x7f358c0c2120 legacy=0x7f358c0aab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.485 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.480+0000 7f358959a700 1 --2- 172.21.15.188:0/1346934616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3574003a60 0x7f3574003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.485 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.480+0000 7f358959a700 1 --2- 172.21.15.188:0/1346934616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3574004810 0x7f3574004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.485 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.480+0000 7f358959a700 1 --2- 172.21.15.188:0/1346934616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35740051c0 0x7f357400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.485 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.480+0000 7f358959a700 1 -- 172.21.15.188:0/1346934616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357400c0a0 con 0x7f3574004810 2024-01-27T02:31:51.485 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.480+0000 7f358959a700 1 -- 172.21.15.188:0/1346934616 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357400c210 con 0x7f35740051c0 2024-01-27T02:31:51.485 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.480+0000 7f358959a700 1 -- 172.21.15.188:0/1346934616 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357400e5e0 con 0x7f3574003a60 2024-01-27T02:31:51.485 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.480+0000 7f358ad9d700 1 --2- 172.21.15.188:0/1346934616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35740051c0 0x7f357400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.485 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.481+0000 7f3589d9b700 1 --2- 172.21.15.188:0/1346934616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3574004810 0x7f3574004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.485 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.481+0000 7f358ad9d700 1 -- 172.21.15.188:0/1346934616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3574003a60 msgr2=0x7f3574003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:31:51.485 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.481+0000 7f358a59c700 1 --2- 172.21.15.188:0/1346934616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3574003a60 0x7f3574003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.485 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.481+0000 7f358ad9d700 1 --2- 172.21.15.188:0/1346934616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3574003a60 0x7f3574003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.485 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.481+0000 7f358ad9d700 1 -- 172.21.15.188:0/1346934616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3574004810 msgr2=0x7f3574004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:31:51.485 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.481+0000 7f358ad9d700 1 --2- 172.21.15.188:0/1346934616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3574004810 0x7f3574004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.485 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.481+0000 7f358ad9d700 1 -- 172.21.15.188:0/1346934616 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3574003570 con 0x7f35740051c0 2024-01-27T02:31:51.486 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.481+0000 7f358a59c700 1 --2- 172.21.15.188:0/1346934616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3574003a60 0x7f3574003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:31:51.486 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.481+0000 7f358ad9d700 1 -- 172.21.15.188:0/1346934616 reap_dead start 2024-01-27T02:31:51.486 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.481+0000 7f358ad9d700 1 --2- 172.21.15.188:0/1346934616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35740051c0 0x7f357400ba60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f358400a920 tx=0x7f35840021f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ece9c1351973bd78 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:31:51.486 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.481+0000 7f358959a700 1 -- 172.21.15.188:0/1346934616 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3584007150 con 0x7f35740051c0 2024-01-27T02:31:51.486 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.481+0000 7f358959a700 1 -- 172.21.15.188:0/1346934616 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35840072d0 con 0x7f35740051c0 2024-01-27T02:31:51.486 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.481+0000 7f358959a700 1 -- 172.21.15.188:0/1346934616 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3584007690 con 0x7f35740051c0 2024-01-27T02:31:51.486 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.482+0000 7f3590fd1700 1 -- 172.21.15.188:0/1346934616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35740051c0 msgr2=0x7f357400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.486 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.482+0000 7f3590fd1700 1 --2- 172.21.15.188:0/1346934616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35740051c0 0x7f357400ba60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f358400a920 tx=0x7f35840021f0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.486 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.482+0000 7f3590fd1700 1 -- 172.21.15.188:0/1346934616 shutdown_connections 2024-01-27T02:31:51.486 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.482+0000 7f3590fd1700 1 --2- 172.21.15.188:0/1346934616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35740051c0 0x7f357400ba60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.486 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.482+0000 7f3590fd1700 1 -- 172.21.15.188:0/1346934616 >> 172.21.15.188:0/1346934616 conn(0x7f358c0679b0 msgr2=0x7f358c067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:31:51.486 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.482+0000 7f3590fd1700 1 -- 172.21.15.188:0/1346934616 shutdown_connections 2024-01-27T02:31:51.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.482+0000 7f3590fd1700 1 -- 172.21.15.188:0/1346934616 wait complete. 2024-01-27T02:31:51.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.483+0000 7f3590fd1700 1 Processor -- start 2024-01-27T02:31:51.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.483+0000 7f3590fd1700 1 -- start start 2024-01-27T02:31:51.489 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.484+0000 7f3590fd1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35740051c0 0x7f358c14dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.489 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.484+0000 7f3590fd1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f358c09a350 0x7f358c09a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.489 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.484+0000 7f3590fd1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f358c06aa10 0x7f358c06ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.490 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.484+0000 7f3590fd1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c09b080 con 0x7f35740051c0 2024-01-27T02:31:51.490 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.484+0000 7f3590fd1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c06b3b0 con 0x7f358c09a350 2024-01-27T02:31:51.490 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.484+0000 7f3590fd1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358c14fa70 con 0x7f358c06aa10 2024-01-27T02:31:51.490 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.484+0000 7f3589d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f358c09a350 0x7f358c09a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.490 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.484+0000 7f358ad9d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f358c06aa10 0x7f358c06ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.490 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.484+0000 7f3589d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f358c09a350 0x7f358c09a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:55888/0 (socket says 172.21.15.188:55888) 2024-01-27T02:31:51.490 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.484+0000 7f3589d9b700 1 -- 172.21.15.188:0/2500193939 learned_addr learned my addr 172.21.15.188:0/2500193939 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-27T02:31:51.490 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.484+0000 7f358a59c700 1 --2- 172.21.15.188:0/2500193939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35740051c0 0x7f358c14dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.490 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.484+0000 7f3589d9b700 1 -- 172.21.15.188:0/2500193939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f358c06aa10 msgr2=0x7f358c06ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.490 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.484+0000 7f3589d9b700 1 --2- 172.21.15.188:0/2500193939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f358c06aa10 0x7f358c06ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.490 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.484+0000 7f3589d9b700 1 -- 172.21.15.188:0/2500193939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35740051c0 msgr2=0x7f358c14dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.490 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.484+0000 7f3589d9b700 1 --2- 172.21.15.188:0/2500193939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35740051c0 0x7f358c14dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.491 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.484+0000 7f3589d9b700 1 -- 172.21.15.188:0/2500193939 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c09ada0 con 0x7f358c09a350 2024-01-27T02:31:51.491 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.484+0000 7f358ad9d700 1 --2- 172.21.15.188:0/2500193939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f358c06aa10 0x7f358c06ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:31:51.491 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.484+0000 7f3589d9b700 1 --2- 172.21.15.188:0/2500193939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f358c09a350 0x7f358c09a7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3580011780 tx=0x7f3580008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:31:51.491 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.484+0000 7f357a7fc700 1 -- 172.21.15.188:0/2500193939 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580015670 con 0x7f358c09a350 2024-01-27T02:31:51.491 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.484+0000 7f357a7fc700 1 -- 172.21.15.188:0/2500193939 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35800157d0 con 0x7f358c09a350 2024-01-27T02:31:51.491 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.484+0000 7f357a7fc700 1 -- 172.21.15.188:0/2500193939 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580015b70 con 0x7f358c09a350 2024-01-27T02:31:51.491 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.484+0000 7f3590fd1700 1 -- 172.21.15.188:0/2500193939 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358c14f420 con 0x7f358c09a350 2024-01-27T02:31:51.491 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.485+0000 7f3590fd1700 1 -- 172.21.15.188:0/2500193939 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358c14e290 con 0x7f358c09a350 2024-01-27T02:31:51.491 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.485+0000 7f357a7fc700 1 -- 172.21.15.188:0/2500193939 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f358001e6d0 con 0x7f358c09a350 2024-01-27T02:31:51.491 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.485+0000 7f357a7fc700 1 -- 172.21.15.188:0/2500193939 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f3580013900 con 0x7f358c09a350 2024-01-27T02:31:51.491 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.485+0000 7f3590fd1700 1 -- 172.21.15.188:0/2500193939 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f358c14f280 con 0x7f358c09a350 2024-01-27T02:31:51.491 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.487+0000 7f357a7fc700 1 -- 172.21.15.188:0/2500193939 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f358c14f280 con 0x7f358c09a350 2024-01-27T02:31:51.545 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:51.545+0000 7fa51bf991c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-27T02:31:51.577 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:51.577+0000 7ff197f771c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-27T02:31:51.589 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.589+0000 7f3590fd1700 1 -- 172.21.15.188:0/2500193939 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3} v 0) v1 -- 0x7f358c14fbe0 con 0x7f358c09a350 2024-01-27T02:31:51.592 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.592+0000 7f357a7fc700 1 -- 172.21.15.188:0/2500193939 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3580023060 con 0x7f358c09a350 2024-01-27T02:31:51.592 INFO:teuthology.orchestra.run.smithi188.stdout:3 2024-01-27T02:31:51.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.593+0000 7f3590fd1700 1 -- 172.21.15.188:0/2500193939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f358c09a350 msgr2=0x7f358c09a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.593+0000 7f3590fd1700 1 --2- 172.21.15.188:0/2500193939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f358c09a350 0x7f358c09a7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3580011780 tx=0x7f3580008040 comp rx=0 tx=0).stop 2024-01-27T02:31:51.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.593+0000 7f3590fd1700 1 -- 172.21.15.188:0/2500193939 shutdown_connections 2024-01-27T02:31:51.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.593+0000 7f3590fd1700 1 --2- 172.21.15.188:0/2500193939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f358c06aa10 0x7f358c06ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.593+0000 7f3590fd1700 1 --2- 172.21.15.188:0/2500193939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35740051c0 0x7f358c14dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.593+0000 7f3590fd1700 1 --2- 172.21.15.188:0/2500193939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f358c09a350 0x7f358c09a7c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.593+0000 7f3590fd1700 1 -- 172.21.15.188:0/2500193939 >> 172.21.15.188:0/2500193939 conn(0x7f358c0679b0 msgr2=0x7f358c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:31:51.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.593+0000 7f3590fd1700 1 -- 172.21.15.188:0/2500193939 shutdown_connections 2024-01-27T02:31:51.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.593+0000 7f3590fd1700 1 -- 172.21.15.188:0/2500193939 wait complete. 2024-01-27T02:31:51.605 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 4 2024-01-27T02:31:51.687 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:51.686+0000 7fa51bf991c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-27T02:31:51.722 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.722+0000 7ffb303c4700 1 Processor -- start 2024-01-27T02:31:51.723 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.722+0000 7ffb303c4700 1 -- start start 2024-01-27T02:31:51.725 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.723+0000 7ffb303c4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb28156e80 0x7ffb280cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.725 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.723+0000 7ffb303c4700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffb28157250 0x7ffb280cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.725 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.723+0000 7ffb303c4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb28070d00 con 0x7ffb280c2120 2024-01-27T02:31:51.725 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.723+0000 7ffb303c4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb280c24f0 con 0x7ffb28156e80 2024-01-27T02:31:51.725 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.723+0000 7ffb303c4700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb280c29c0 con 0x7ffb28157250 2024-01-27T02:31:51.725 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.723+0000 7ffb2d95f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffb28157250 0x7ffb280cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.723+0000 7ffb2d95f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffb28157250 0x7ffb280cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:55904/0 (socket says 172.21.15.188:55904) 2024-01-27T02:31:51.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.723+0000 7ffb2d95f700 1 -- 172.21.15.188:0/267949374 learned_addr learned my addr 172.21.15.188:0/267949374 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-27T02:31:51.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.724+0000 7ffb2e160700 1 --2- 172.21.15.188:0/267949374 >> v2:172.21.15.7:3300/0 conn(0x7ffb28156e80 0x7ffb280cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.724+0000 7ffb2d95f700 1 -- 172.21.15.188:0/267949374 >> v2:172.21.15.7:3300/0 conn(0x7ffb28156e80 msgr2=0x7ffb280cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:31:51.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.724+0000 7ffb2d95f700 1 --2- 172.21.15.188:0/267949374 >> v2:172.21.15.7:3300/0 conn(0x7ffb28156e80 0x7ffb280cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.724+0000 7ffb2d95f700 1 -- 172.21.15.188:0/267949374 >> v1:172.21.15.7:6789/0 conn(0x7ffb280c2120 legacy=0x7ffb2815ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.724+0000 7ffb2d95f700 1 -- 172.21.15.188:0/267949374 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb280aa050 con 0x7ffb28157250 2024-01-27T02:31:51.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.724+0000 7ffb2e160700 1 --2- 172.21.15.188:0/267949374 >> v2:172.21.15.7:3300/0 conn(0x7ffb28156e80 0x7ffb280cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:31:51.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.724+0000 7ffb2d95f700 1 --2- 172.21.15.188:0/267949374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb28157250 0x7ffb280cb7c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ffb28071220 tx=0x7ffb1c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86a5b11bb5889a1b server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:31:51.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.724+0000 7ffb2d15e700 1 -- 172.21.15.188:0/267949374 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c00b860 con 0x7ffb28157250 2024-01-27T02:31:51.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.724+0000 7ffb2d15e700 1 -- 172.21.15.188:0/267949374 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1c00b9c0 con 0x7ffb28157250 2024-01-27T02:31:51.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.724+0000 7ffb2d15e700 1 -- 172.21.15.188:0/267949374 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c00bd60 con 0x7ffb28157250 2024-01-27T02:31:51.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.724+0000 7ffb303c4700 1 -- 172.21.15.188:0/267949374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb28157250 msgr2=0x7ffb280cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.725+0000 7ffb303c4700 1 --2- 172.21.15.188:0/267949374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb28157250 0x7ffb280cb7c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ffb28071220 tx=0x7ffb1c009250 comp rx=0 tx=0).stop 2024-01-27T02:31:51.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.725+0000 7ffb303c4700 1 -- 172.21.15.188:0/267949374 shutdown_connections 2024-01-27T02:31:51.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.725+0000 7ffb303c4700 1 --2- 172.21.15.188:0/267949374 >> v2:172.21.15.7:3300/0 conn(0x7ffb28156e80 0x7ffb280cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.725+0000 7ffb303c4700 1 --2- 172.21.15.188:0/267949374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb28157250 0x7ffb280cb7c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.725+0000 7ffb303c4700 1 -- 172.21.15.188:0/267949374 >> 172.21.15.188:0/267949374 conn(0x7ffb280679b0 msgr2=0x7ffb28067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:31:51.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.725+0000 7ffb303c4700 1 -- 172.21.15.188:0/267949374 shutdown_connections 2024-01-27T02:31:51.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.725+0000 7ffb303c4700 1 -- 172.21.15.188:0/267949374 wait complete. 2024-01-27T02:31:51.729 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.726+0000 7ffb303c4700 1 Processor -- start 2024-01-27T02:31:51.729 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.726+0000 7ffb303c4700 1 -- start start 2024-01-27T02:31:51.729 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.726+0000 7ffb303c4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb280c2120 0x7ffb2815c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.729 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.726+0000 7ffb303c4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb28156e80 0x7ffb2816a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.726+0000 7ffb303c4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb28157250 0x7ffb280c4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.726+0000 7ffb303c4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2806dab0 con 0x7ffb28157250 2024-01-27T02:31:51.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.726+0000 7ffb303c4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2806dd90 con 0x7ffb28156e80 2024-01-27T02:31:51.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.726+0000 7ffb303c4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2816b160 con 0x7ffb280c2120 2024-01-27T02:31:51.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.727+0000 7ffb2e160700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb280c2120 0x7ffb2815c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.727+0000 7ffb2d95f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb28156e80 0x7ffb2816a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.727+0000 7ffb2e160700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb280c2120 0x7ffb2815c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:51210/0 (socket says 172.21.15.188:51210) 2024-01-27T02:31:51.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.727+0000 7ffb2d95f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb28156e80 0x7ffb2816a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:55918/0 (socket says 172.21.15.188:55918) 2024-01-27T02:31:51.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.727+0000 7ffb2e160700 1 -- 172.21.15.188:0/585789649 learned_addr learned my addr 172.21.15.188:0/585789649 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-27T02:31:51.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.727+0000 7ffb2e961700 1 --2- 172.21.15.188:0/585789649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb28157250 0x7ffb280c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.727+0000 7ffb2d95f700 1 -- 172.21.15.188:0/585789649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb280c2120 msgr2=0x7ffb2815c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.727+0000 7ffb2d95f700 1 --2- 172.21.15.188:0/585789649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb280c2120 0x7ffb2815c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.727+0000 7ffb2d95f700 1 -- 172.21.15.188:0/585789649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb28157250 msgr2=0x7ffb280c4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.727+0000 7ffb2d95f700 1 --2- 172.21.15.188:0/585789649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb28157250 0x7ffb280c4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.727+0000 7ffb2d95f700 1 -- 172.21.15.188:0/585789649 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1c0094f0 con 0x7ffb28156e80 2024-01-27T02:31:51.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.727+0000 7ffb2e160700 1 --2- 172.21.15.188:0/585789649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb280c2120 0x7ffb2815c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:31:51.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.727+0000 7ffb2d95f700 1 --2- 172.21.15.188:0/585789649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb28156e80 0x7ffb2816a910 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ffb28068ea0 tx=0x7ffb1c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:31:51.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.727+0000 7ffb267fc700 1 -- 172.21.15.188:0/585789649 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c006bc0 con 0x7ffb28156e80 2024-01-27T02:31:51.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.727+0000 7ffb267fc700 1 -- 172.21.15.188:0/585789649 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1c01f070 con 0x7ffb28156e80 2024-01-27T02:31:51.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.727+0000 7ffb303c4700 1 -- 172.21.15.188:0/585789649 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2816b440 con 0x7ffb28156e80 2024-01-27T02:31:51.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.727+0000 7ffb267fc700 1 -- 172.21.15.188:0/585789649 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c0266f0 con 0x7ffb28156e80 2024-01-27T02:31:51.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.727+0000 7ffb303c4700 1 -- 172.21.15.188:0/585789649 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb2816ae50 con 0x7ffb28156e80 2024-01-27T02:31:51.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.727+0000 7ffb267fc700 1 -- 172.21.15.188:0/585789649 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7ffb1c01a430 con 0x7ffb28156e80 2024-01-27T02:31:51.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.727+0000 7ffb267fc700 1 -- 172.21.15.188:0/585789649 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7ffb1c021070 con 0x7ffb28156e80 2024-01-27T02:31:51.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.728+0000 7ffb303c4700 1 -- 172.21.15.188:0/585789649 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb280c2df0 con 0x7ffb28156e80 2024-01-27T02:31:51.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.730+0000 7ffb267fc700 1 -- 172.21.15.188:0/585789649 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7ffb1c015020 con 0x7ffb28156e80 2024-01-27T02:31:51.814 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.814+0000 7ffb303c4700 1 -- 172.21.15.188:0/585789649 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4} v 0) v1 -- 0x7ffb2806df00 con 0x7ffb28156e80 2024-01-27T02:31:51.817 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.817+0000 7ffb267fc700 1 -- 172.21.15.188:0/585789649 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7ffb1c01ae10 con 0x7ffb28156e80 2024-01-27T02:31:51.817 INFO:teuthology.orchestra.run.smithi188.stdout:4 2024-01-27T02:31:51.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.818+0000 7ffb303c4700 1 -- 172.21.15.188:0/585789649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb28156e80 msgr2=0x7ffb2816a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.818+0000 7ffb303c4700 1 --2- 172.21.15.188:0/585789649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb28156e80 0x7ffb2816a910 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ffb28068ea0 tx=0x7ffb1c017000 comp rx=0 tx=0).stop 2024-01-27T02:31:51.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.818+0000 7ffb303c4700 1 -- 172.21.15.188:0/585789649 shutdown_connections 2024-01-27T02:31:51.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.818+0000 7ffb303c4700 1 --2- 172.21.15.188:0/585789649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb28157250 0x7ffb280c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.818+0000 7ffb303c4700 1 --2- 172.21.15.188:0/585789649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb280c2120 0x7ffb2815c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.818+0000 7ffb303c4700 1 --2- 172.21.15.188:0/585789649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb28156e80 0x7ffb2816a910 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.818+0000 7ffb303c4700 1 -- 172.21.15.188:0/585789649 >> 172.21.15.188:0/585789649 conn(0x7ffb280679b0 msgr2=0x7ffb280c81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:31:51.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.818+0000 7ffb303c4700 1 -- 172.21.15.188:0/585789649 shutdown_connections 2024-01-27T02:31:51.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.818+0000 7ffb303c4700 1 -- 172.21.15.188:0/585789649 wait complete. 2024-01-27T02:31:51.830 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 5 2024-01-27T02:31:51.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.950+0000 7f4f37796700 1 Processor -- start 2024-01-27T02:31:51.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.950+0000 7f4f37796700 1 -- start start 2024-01-27T02:31:51.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.951+0000 7f4f37796700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f30063c90 0x7f4f30164ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.951+0000 7f4f37796700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f300cc290 con 0x7f4f300aa8b0 2024-01-27T02:31:51.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.951+0000 7f4f37796700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f300cb960 con 0x7f4f30143670 2024-01-27T02:31:51.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.951+0000 7f4f37796700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f300cbc40 con 0x7f4f30063c90 2024-01-27T02:31:51.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.951+0000 7f4f35532700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f4f30143670 0x7f4f30063580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:47320/0 (socket says 172.21.15.188:47320) 2024-01-27T02:31:51.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.951+0000 7f4f35d33700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f30063c90 0x7f4f30164ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.951+0000 7f4f35532700 1 -- 172.21.15.188:0/1507110637 learned_addr learned my addr 172.21.15.188:0/1507110637 (peer_addr_for_me v1:172.21.15.188:0/0) 2024-01-27T02:31:51.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.952+0000 7f4f27fff700 1 -- 172.21.15.188:0/1507110637 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190887166 0 0) 0x7f4f300cb960 con 0x7f4f30143670 2024-01-27T02:31:51.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.952+0000 7f4f35d33700 1 -- 172.21.15.188:0/1507110637 >> v1:172.21.15.188:6789/0 conn(0x7f4f30143670 legacy=0x7f4f30063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.952+0000 7f4f35d33700 1 -- 172.21.15.188:0/1507110637 >> v1:172.21.15.7:6789/0 conn(0x7f4f300aa8b0 legacy=0x7f4f300c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.952+0000 7f4f35d33700 1 -- 172.21.15.188:0/1507110637 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f300cbf20 con 0x7f4f30063c90 2024-01-27T02:31:51.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.952+0000 7f4f35d33700 1 --2- 172.21.15.188:0/1507110637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f30063c90 0x7f4f30164ff0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009e90 tx=0x7f4f28009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7ea1f4277afe128 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:31:51.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.952+0000 7f4f27fff700 1 -- 172.21.15.188:0/1507110637 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2800e070 con 0x7f4f30063c90 2024-01-27T02:31:51.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.952+0000 7f4f27fff700 1 -- 172.21.15.188:0/1507110637 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2800bc30 con 0x7f4f30063c90 2024-01-27T02:31:51.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.952+0000 7f4f27fff700 1 -- 172.21.15.188:0/1507110637 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f28012430 con 0x7f4f30063c90 2024-01-27T02:31:51.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.952+0000 7f4f37796700 1 -- 172.21.15.188:0/1507110637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f30063c90 msgr2=0x7f4f30164ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.952+0000 7f4f37796700 1 --2- 172.21.15.188:0/1507110637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f30063c90 0x7f4f30164ff0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009e90 tx=0x7f4f28009780 comp rx=0 tx=0).stop 2024-01-27T02:31:51.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.953+0000 7f4f37796700 1 -- 172.21.15.188:0/1507110637 shutdown_connections 2024-01-27T02:31:51.955 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.953+0000 7f4f37796700 1 --2- 172.21.15.188:0/1507110637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f30063c90 0x7f4f30164ff0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.955 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.953+0000 7f4f37796700 1 -- 172.21.15.188:0/1507110637 >> 172.21.15.188:0/1507110637 conn(0x7f4f300cdbf0 msgr2=0x7f4f300cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:31:51.955 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.953+0000 7f4f37796700 1 -- 172.21.15.188:0/1507110637 shutdown_connections 2024-01-27T02:31:51.955 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.953+0000 7f4f37796700 1 -- 172.21.15.188:0/1507110637 wait complete. 2024-01-27T02:31:51.955 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.954+0000 7f4f37796700 1 Processor -- start 2024-01-27T02:31:51.955 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.954+0000 7f4f37796700 1 -- start start 2024-01-27T02:31:51.957 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f37796700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f30063c90 0x7f4f3006be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.957 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f37796700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f300aa8b0 0x7f4f3015ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.957 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f37796700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f30143670 0x7f4f3016e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:51.957 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f37796700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3016d8f0 con 0x7f4f30063c90 2024-01-27T02:31:51.957 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f37796700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3016deb0 con 0x7f4f30143670 2024-01-27T02:31:51.957 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f37796700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f300c4040 con 0x7f4f300aa8b0 2024-01-27T02:31:51.957 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f35d33700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f30143670 0x7f4f3016e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.957 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f34d31700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f300aa8b0 0x7f4f3015ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.957 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f35d33700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f30143670 0x7f4f3016e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:55922/0 (socket says 172.21.15.188:55922) 2024-01-27T02:31:51.958 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f35d33700 1 -- 172.21.15.188:0/353710984 learned_addr learned my addr 172.21.15.188:0/353710984 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-27T02:31:51.958 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f35532700 1 --2- 172.21.15.188:0/353710984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f30063c90 0x7f4f3006be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:51.958 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f34d31700 1 -- 172.21.15.188:0/353710984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f30143670 msgr2=0x7f4f3016e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.958 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f34d31700 1 --2- 172.21.15.188:0/353710984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f30143670 0x7f4f3016e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.958 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f34d31700 1 -- 172.21.15.188:0/353710984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f30063c90 msgr2=0x7f4f3006be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:51.958 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f34d31700 1 --2- 172.21.15.188:0/353710984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f30063c90 0x7f4f3006be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:51.958 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f34d31700 1 -- 172.21.15.188:0/353710984 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f28009a20 con 0x7f4f300aa8b0 2024-01-27T02:31:51.958 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f35d33700 1 --2- 172.21.15.188:0/353710984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f30143670 0x7f4f3016e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:31:51.958 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f34d31700 1 --2- 172.21.15.188:0/353710984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f300aa8b0 0x7f4f3015ca60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4f2000c5c0 tx=0x7f4f20007ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:31:51.959 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f257fa700 1 -- 172.21.15.188:0/353710984 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f20011590 con 0x7f4f300aa8b0 2024-01-27T02:31:51.959 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f257fa700 1 -- 172.21.15.188:0/353710984 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f20011710 con 0x7f4f300aa8b0 2024-01-27T02:31:51.959 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f37796700 1 -- 172.21.15.188:0/353710984 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f3015d090 con 0x7f4f300aa8b0 2024-01-27T02:31:51.959 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f257fa700 1 -- 172.21.15.188:0/353710984 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f20011ad0 con 0x7f4f300aa8b0 2024-01-27T02:31:51.959 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.955+0000 7f4f37796700 1 -- 172.21.15.188:0/353710984 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f3016db40 con 0x7f4f300aa8b0 2024-01-27T02:31:51.959 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.956+0000 7f4f257fa700 1 -- 172.21.15.188:0/353710984 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4f2001c6c0 con 0x7f4f300aa8b0 2024-01-27T02:31:51.959 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.956+0000 7f4f257fa700 1 -- 172.21.15.188:0/353710984 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f4f200129a0 con 0x7f4f300aa8b0 2024-01-27T02:31:51.959 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.956+0000 7f4f37796700 1 -- 172.21.15.188:0/353710984 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f30072060 con 0x7f4f300aa8b0 2024-01-27T02:31:51.960 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:51.960+0000 7f4f257fa700 1 -- 172.21.15.188:0/353710984 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f4f20017070 con 0x7f4f300aa8b0 2024-01-27T02:31:51.967 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:51.967+0000 7ff197f771c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-27T02:31:52.045 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.045+0000 7f4f37796700 1 -- 172.21.15.188:0/353710984 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5} v 0) v1 -- 0x7f4f3016dff0 con 0x7f4f300aa8b0 2024-01-27T02:31:52.050 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.050+0000 7f4f257fa700 1 -- 172.21.15.188:0/353710984 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4f2001cba0 con 0x7f4f300aa8b0 2024-01-27T02:31:52.050 INFO:teuthology.orchestra.run.smithi188.stdout:5 2024-01-27T02:31:52.052 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.051+0000 7f4f37796700 1 -- 172.21.15.188:0/353710984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f300aa8b0 msgr2=0x7f4f3015ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:52.052 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.051+0000 7f4f37796700 1 --2- 172.21.15.188:0/353710984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f300aa8b0 0x7f4f3015ca60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4f2000c5c0 tx=0x7f4f20007ec0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.052 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.051+0000 7f4f37796700 1 -- 172.21.15.188:0/353710984 shutdown_connections 2024-01-27T02:31:52.052 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.051+0000 7f4f37796700 1 --2- 172.21.15.188:0/353710984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f30143670 0x7f4f3016e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.052 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.051+0000 7f4f37796700 1 --2- 172.21.15.188:0/353710984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f30063c90 0x7f4f3006be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.052 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.051+0000 7f4f37796700 1 --2- 172.21.15.188:0/353710984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f300aa8b0 0x7f4f3015ca60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.052 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.051+0000 7f4f37796700 1 -- 172.21.15.188:0/353710984 >> 172.21.15.188:0/353710984 conn(0x7f4f300cdbf0 msgr2=0x7f4f30063580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:31:52.052 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.051+0000 7f4f37796700 1 -- 172.21.15.188:0/353710984 shutdown_connections 2024-01-27T02:31:52.052 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.051+0000 7f4f37796700 1 -- 172.21.15.188:0/353710984 wait complete. 2024-01-27T02:31:52.063 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 6 2024-01-27T02:31:52.095 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:52.094+0000 7fa51bf991c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-27T02:31:52.180 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.180+0000 7fc9b577e700 1 Processor -- start 2024-01-27T02:31:52.181 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.180+0000 7fc9b577e700 1 -- start start 2024-01-27T02:31:52.183 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.181+0000 7fc9b577e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc9b0156b60 0x7fc9b00ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:52.183 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.181+0000 7fc9b577e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc9b0156f30 0x7fc9b015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:52.183 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.181+0000 7fc9b577e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9b00aeaa0 con 0x7fc9b00ad9f0 2024-01-27T02:31:52.183 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.181+0000 7fc9b577e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9b00ade90 con 0x7fc9b0156b60 2024-01-27T02:31:52.183 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.181+0000 7fc9b577e700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9b00ae170 con 0x7fc9b0156f30 2024-01-27T02:31:52.183 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.181+0000 7fc9af7fe700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc9b0156f30 0x7fc9b015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:52.183 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.181+0000 7fc9af7fe700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc9b0156f30 0x7fc9b015a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:55936/0 (socket says 172.21.15.188:55936) 2024-01-27T02:31:52.183 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.181+0000 7fc9af7fe700 1 -- 172.21.15.188:0/1419389789 learned_addr learned my addr 172.21.15.188:0/1419389789 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-27T02:31:52.183 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.181+0000 7fc9aeffd700 1 --2- 172.21.15.188:0/1419389789 >> v2:172.21.15.7:3300/0 conn(0x7fc9b0156b60 0x7fc9b00ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:52.183 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.182+0000 7fc9af7fe700 1 -- 172.21.15.188:0/1419389789 >> v2:172.21.15.7:3300/0 conn(0x7fc9b0156b60 msgr2=0x7fc9b00ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:31:52.184 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.182+0000 7fc9af7fe700 1 --2- 172.21.15.188:0/1419389789 >> v2:172.21.15.7:3300/0 conn(0x7fc9b0156b60 0x7fc9b00ad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.184 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.182+0000 7fc9af7fe700 1 -- 172.21.15.188:0/1419389789 >> v1:172.21.15.7:6789/0 conn(0x7fc9b00ad9f0 legacy=0x7fc9b00aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:52.184 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.182+0000 7fc9af7fe700 1 -- 172.21.15.188:0/1419389789 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b00ae450 con 0x7fc9b0156f30 2024-01-27T02:31:52.184 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.182+0000 7fc9af7fe700 1 --2- 172.21.15.188:0/1419389789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9b0156f30 0x7fc9b015a9b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc9b00afdd0 tx=0x7fc9a8009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc5b2070d3f4a62f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:31:52.184 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.182+0000 7fc9adffb700 1 -- 172.21.15.188:0/1419389789 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a800e070 con 0x7fc9b0156f30 2024-01-27T02:31:52.184 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.182+0000 7fc9adffb700 1 -- 172.21.15.188:0/1419389789 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a800ba20 con 0x7fc9b0156f30 2024-01-27T02:31:52.184 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.182+0000 7fc9adffb700 1 -- 172.21.15.188:0/1419389789 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a80092e0 con 0x7fc9b0156f30 2024-01-27T02:31:52.184 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.182+0000 7fc9b577e700 1 -- 172.21.15.188:0/1419389789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9b0156f30 msgr2=0x7fc9b015a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:52.184 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.182+0000 7fc9b577e700 1 --2- 172.21.15.188:0/1419389789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9b0156f30 0x7fc9b015a9b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc9b00afdd0 tx=0x7fc9a8009840 comp rx=0 tx=0).stop 2024-01-27T02:31:52.184 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.183+0000 7fc9b577e700 1 -- 172.21.15.188:0/1419389789 shutdown_connections 2024-01-27T02:31:52.184 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.183+0000 7fc9b577e700 1 --2- 172.21.15.188:0/1419389789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9b0156f30 0x7fc9b015a9b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.184 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.183+0000 7fc9b577e700 1 --2- 172.21.15.188:0/1419389789 >> v2:172.21.15.7:3300/0 conn(0x7fc9b0156b60 0x7fc9b00ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.185 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.183+0000 7fc9b577e700 1 -- 172.21.15.188:0/1419389789 >> 172.21.15.188:0/1419389789 conn(0x7fc9b0068480 msgr2=0x7fc9b00cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:31:52.185 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.183+0000 7fc9b577e700 1 -- 172.21.15.188:0/1419389789 shutdown_connections 2024-01-27T02:31:52.185 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.183+0000 7fc9b577e700 1 -- 172.21.15.188:0/1419389789 wait complete. 2024-01-27T02:31:52.187 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.184+0000 7fc9b577e700 1 Processor -- start 2024-01-27T02:31:52.187 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.184+0000 7fc9b577e700 1 -- start start 2024-01-27T02:31:52.187 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc9b577e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9b00ad9f0 0x7fc9b0146da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:52.187 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc9b577e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9b0156b60 0x7fc9b00ac610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:52.187 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc9b577e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9b0156f30 0x7fc9b00acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:52.187 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc9b577e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b014de00 con 0x7fc9b0156f30 2024-01-27T02:31:52.187 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc9b577e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b0153020 con 0x7fc9b00ad9f0 2024-01-27T02:31:52.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc9b577e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b01532a0 con 0x7fc9b0156b60 2024-01-27T02:31:52.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc9aeffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9b00ad9f0 0x7fc9b0146da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:52.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc9ae7fc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9b0156b60 0x7fc9b00ac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:52.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc9aeffd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9b00ad9f0 0x7fc9b0146da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:55944/0 (socket says 172.21.15.188:55944) 2024-01-27T02:31:52.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc9aeffd700 1 -- 172.21.15.188:0/4224532227 learned_addr learned my addr 172.21.15.188:0/4224532227 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-27T02:31:52.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc9af7fe700 1 --2- 172.21.15.188:0/4224532227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9b0156f30 0x7fc9b00acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:52.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc9ae7fc700 1 -- 172.21.15.188:0/4224532227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9b00ad9f0 msgr2=0x7fc9b0146da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:52.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc9ae7fc700 1 --2- 172.21.15.188:0/4224532227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9b00ad9f0 0x7fc9b0146da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc9ae7fc700 1 -- 172.21.15.188:0/4224532227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9b0156f30 msgr2=0x7fc9b00acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:52.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc9ae7fc700 1 --2- 172.21.15.188:0/4224532227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9b0156f30 0x7fc9b00acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc9ae7fc700 1 -- 172.21.15.188:0/4224532227 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a8009ae0 con 0x7fc9b0156b60 2024-01-27T02:31:52.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc9aeffd700 1 --2- 172.21.15.188:0/4224532227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9b00ad9f0 0x7fc9b0146da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:31:52.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc9ae7fc700 1 --2- 172.21.15.188:0/4224532227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9b0156b60 0x7fc9b00ac610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400c590 tx=0x7fc9a4007ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:31:52.189 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc996ffd700 1 -- 172.21.15.188:0/4224532227 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4011590 con 0x7fc9b0156b60 2024-01-27T02:31:52.189 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc996ffd700 1 -- 172.21.15.188:0/4224532227 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a4011710 con 0x7fc9b0156b60 2024-01-27T02:31:52.189 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc9b577e700 1 -- 172.21.15.188:0/4224532227 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b014e820 con 0x7fc9b0156b60 2024-01-27T02:31:52.189 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc996ffd700 1 -- 172.21.15.188:0/4224532227 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4011ad0 con 0x7fc9b0156b60 2024-01-27T02:31:52.189 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.185+0000 7fc9b577e700 1 -- 172.21.15.188:0/4224532227 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b01472e0 con 0x7fc9b0156b60 2024-01-27T02:31:52.189 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.186+0000 7fc996ffd700 1 -- 172.21.15.188:0/4224532227 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fc9a401c6c0 con 0x7fc9b0156b60 2024-01-27T02:31:52.189 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.186+0000 7fc996ffd700 1 -- 172.21.15.188:0/4224532227 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7fc9a4012a20 con 0x7fc9b0156b60 2024-01-27T02:31:52.189 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.187+0000 7fc9b577e700 1 -- 172.21.15.188:0/4224532227 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b01534c0 con 0x7fc9b0156b60 2024-01-27T02:31:52.189 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.189+0000 7fc996ffd700 1 -- 172.21.15.188:0/4224532227 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fc9b01534c0 con 0x7fc9b0156b60 2024-01-27T02:31:52.192 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:52.192+0000 7ff197f771c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-27T02:31:52.286 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.286+0000 7fc9b577e700 1 -- 172.21.15.188:0/4224532227 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6} v 0) v1 -- 0x7fc9b0072060 con 0x7fc9b0156b60 2024-01-27T02:31:52.294 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.294+0000 7fc996ffd700 1 -- 172.21.15.188:0/4224532227 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc9a4017070 con 0x7fc9b0156b60 2024-01-27T02:31:52.294 INFO:teuthology.orchestra.run.smithi188.stdout:6 2024-01-27T02:31:52.296 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.295+0000 7fc9b577e700 1 -- 172.21.15.188:0/4224532227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9b0156b60 msgr2=0x7fc9b00ac610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:52.296 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.295+0000 7fc9b577e700 1 --2- 172.21.15.188:0/4224532227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9b0156b60 0x7fc9b00ac610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400c590 tx=0x7fc9a4007ec0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.296 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.295+0000 7fc9b577e700 1 -- 172.21.15.188:0/4224532227 shutdown_connections 2024-01-27T02:31:52.296 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.295+0000 7fc9b577e700 1 --2- 172.21.15.188:0/4224532227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9b0156f30 0x7fc9b00acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.296 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.295+0000 7fc9b577e700 1 --2- 172.21.15.188:0/4224532227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9b00ad9f0 0x7fc9b0146da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.296 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.295+0000 7fc9b577e700 1 --2- 172.21.15.188:0/4224532227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9b0156b60 0x7fc9b00ac610 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.296 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.295+0000 7fc9b577e700 1 -- 172.21.15.188:0/4224532227 >> 172.21.15.188:0/4224532227 conn(0x7fc9b0068480 msgr2=0x7fc9b00b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:31:52.296 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.295+0000 7fc9b577e700 1 -- 172.21.15.188:0/4224532227 shutdown_connections 2024-01-27T02:31:52.296 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.295+0000 7fc9b577e700 1 -- 172.21.15.188:0/4224532227 wait complete. 2024-01-27T02:31:52.308 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 7 2024-01-27T02:31:52.333 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:52.333+0000 7fa51bf991c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-27T02:31:52.427 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.427+0000 7fe760db0700 1 Processor -- start 2024-01-27T02:31:52.427 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.427+0000 7fe760db0700 1 -- start start 2024-01-27T02:31:52.429 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.428+0000 7fe760db0700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe75c15bfe0 0x7fe75c0cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:52.430 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.428+0000 7fe760db0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe75c1583e0 0x7fe75c0aef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:52.430 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.428+0000 7fe760db0700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe75c0cd0d0 con 0x7fe75c15c9b0 2024-01-27T02:31:52.430 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.428+0000 7fe760db0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe75c0cc4c0 con 0x7fe75c1583e0 2024-01-27T02:31:52.430 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.428+0000 7fe760db0700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe75c0cc7a0 con 0x7fe75c15bfe0 2024-01-27T02:31:52.430 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.428+0000 7fe75a59c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe75c15bfe0 0x7fe75c0cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:52.430 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.428+0000 7fe75a59c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe75c15bfe0 0x7fe75c0cbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:55952/0 (socket says 172.21.15.188:55952) 2024-01-27T02:31:52.430 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.428+0000 7fe759d9b700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fe75c15c9b0 0x7fe75c157cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:47328/0 (socket says 172.21.15.188:47328) 2024-01-27T02:31:52.430 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.428+0000 7fe75a59c700 1 -- 172.21.15.188:0/61536323 learned_addr learned my addr 172.21.15.188:0/61536323 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-27T02:31:52.430 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.428+0000 7fe75ad9d700 1 --2- 172.21.15.188:0/61536323 >> v2:172.21.15.7:3300/0 conn(0x7fe75c1583e0 0x7fe75c0aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:52.430 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.428+0000 7fe75a59c700 1 -- 172.21.15.188:0/61536323 >> v2:172.21.15.7:3300/0 conn(0x7fe75c1583e0 msgr2=0x7fe75c0aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:31:52.430 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.428+0000 7fe75a59c700 1 --2- 172.21.15.188:0/61536323 >> v2:172.21.15.7:3300/0 conn(0x7fe75c1583e0 0x7fe75c0aef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.430 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.428+0000 7fe75a59c700 1 -- 172.21.15.188:0/61536323 >> v1:172.21.15.188:6789/0 conn(0x7fe75c15c9b0 legacy=0x7fe75c157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:52.430 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.428+0000 7fe75a59c700 1 -- 172.21.15.188:0/61536323 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75c0cca80 con 0x7fe75c15bfe0 2024-01-27T02:31:52.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.428+0000 7fe75959a700 1 -- 172.21.15.188:0/61536323 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 128409943 0 0) 0x7fe75c0cd0d0 con 0x7fe75c15c9b0 2024-01-27T02:31:52.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.428+0000 7fe75a59c700 1 --2- 172.21.15.188:0/61536323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe75c15bfe0 0x7fe75c0cbc90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0c43c0 tx=0x7fe74c009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef577ad77329906f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:31:52.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.428+0000 7fe75959a700 1 -- 172.21.15.188:0/61536323 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c005c10 con 0x7fe75c15bfe0 2024-01-27T02:31:52.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.429+0000 7fe75959a700 1 -- 172.21.15.188:0/61536323 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c005d70 con 0x7fe75c15bfe0 2024-01-27T02:31:52.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.429+0000 7fe75959a700 1 -- 172.21.15.188:0/61536323 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c00b880 con 0x7fe75c15bfe0 2024-01-27T02:31:52.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.429+0000 7fe760db0700 1 -- 172.21.15.188:0/61536323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe75c15bfe0 msgr2=0x7fe75c0cbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:52.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.429+0000 7fe760db0700 1 --2- 172.21.15.188:0/61536323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe75c15bfe0 0x7fe75c0cbc90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0c43c0 tx=0x7fe74c009220 comp rx=0 tx=0).stop 2024-01-27T02:31:52.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.429+0000 7fe760db0700 1 -- 172.21.15.188:0/61536323 shutdown_connections 2024-01-27T02:31:52.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.429+0000 7fe760db0700 1 --2- 172.21.15.188:0/61536323 >> v2:172.21.15.7:3300/0 conn(0x7fe75c1583e0 0x7fe75c0aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.429+0000 7fe760db0700 1 --2- 172.21.15.188:0/61536323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe75c15bfe0 0x7fe75c0cbc90 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.429+0000 7fe760db0700 1 -- 172.21.15.188:0/61536323 >> 172.21.15.188:0/61536323 conn(0x7fe75c0679b0 msgr2=0x7fe75c067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:31:52.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.429+0000 7fe760db0700 1 -- 172.21.15.188:0/61536323 shutdown_connections 2024-01-27T02:31:52.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.429+0000 7fe760db0700 1 -- 172.21.15.188:0/61536323 wait complete. 2024-01-27T02:31:52.434 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.430+0000 7fe760db0700 1 Processor -- start 2024-01-27T02:31:52.434 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.430+0000 7fe760db0700 1 -- start start 2024-01-27T02:31:52.434 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.431+0000 7fe760db0700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe75c1583e0 0x7fe75c09a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:52.434 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.431+0000 7fe760db0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe75c15bfe0 0x7fe75c0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:52.434 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.431+0000 7fe760db0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe75c15c9b0 0x7fe75c0c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:52.434 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.431+0000 7fe760db0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c098650 con 0x7fe75c15c9b0 2024-01-27T02:31:52.434 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.431+0000 7fe760db0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c098320 con 0x7fe75c15bfe0 2024-01-27T02:31:52.434 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.431+0000 7fe760db0700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c09a970 con 0x7fe75c1583e0 2024-01-27T02:31:52.434 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.431+0000 7fe759d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe75c15bfe0 0x7fe75c0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:52.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.431+0000 7fe75a59c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe75c1583e0 0x7fe75c09a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:52.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.431+0000 7fe759d9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe75c15bfe0 0x7fe75c0c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:55962/0 (socket says 172.21.15.188:55962) 2024-01-27T02:31:52.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.431+0000 7fe759d9b700 1 -- 172.21.15.188:0/835555885 learned_addr learned my addr 172.21.15.188:0/835555885 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-27T02:31:52.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.431+0000 7fe75ad9d700 1 --2- 172.21.15.188:0/835555885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe75c15c9b0 0x7fe75c0c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:52.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.431+0000 7fe759d9b700 1 -- 172.21.15.188:0/835555885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe75c1583e0 msgr2=0x7fe75c09a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:52.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.431+0000 7fe759d9b700 1 --2- 172.21.15.188:0/835555885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe75c1583e0 0x7fe75c09a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.431+0000 7fe759d9b700 1 -- 172.21.15.188:0/835555885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe75c15c9b0 msgr2=0x7fe75c0c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:52.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.431+0000 7fe759d9b700 1 --2- 172.21.15.188:0/835555885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe75c15c9b0 0x7fe75c0c5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.431+0000 7fe759d9b700 1 -- 172.21.15.188:0/835555885 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74c009480 con 0x7fe75c15bfe0 2024-01-27T02:31:52.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.432+0000 7fe759d9b700 1 --2- 172.21.15.188:0/835555885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe75c15bfe0 0x7fe75c0c8dd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe75000c5b0 tx=0x7fe750001040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:31:52.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.432+0000 7fe74a7fc700 1 -- 172.21.15.188:0/835555885 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe750005a70 con 0x7fe75c15bfe0 2024-01-27T02:31:52.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.432+0000 7fe74a7fc700 1 -- 172.21.15.188:0/835555885 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe750005bd0 con 0x7fe75c15bfe0 2024-01-27T02:31:52.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.432+0000 7fe74a7fc700 1 -- 172.21.15.188:0/835555885 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7500070d0 con 0x7fe75c15bfe0 2024-01-27T02:31:52.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.432+0000 7fe760db0700 1 -- 172.21.15.188:0/835555885 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe75c0c9610 con 0x7fe75c15bfe0 2024-01-27T02:31:52.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.432+0000 7fe760db0700 1 -- 172.21.15.188:0/835555885 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75c0c9bd0 con 0x7fe75c15bfe0 2024-01-27T02:31:52.436 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.432+0000 7fe74a7fc700 1 -- 172.21.15.188:0/835555885 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fe75001c440 con 0x7fe75c15bfe0 2024-01-27T02:31:52.436 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.432+0000 7fe74a7fc700 1 -- 172.21.15.188:0/835555885 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7fe750012a70 con 0x7fe75c15bfe0 2024-01-27T02:31:52.436 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.433+0000 7fe760db0700 1 -- 172.21.15.188:0/835555885 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75c0c9df0 con 0x7fe75c15bfe0 2024-01-27T02:31:52.436 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.435+0000 7fe74a7fc700 1 -- 172.21.15.188:0/835555885 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fe750017070 con 0x7fe75c15bfe0 2024-01-27T02:31:52.441 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:52.442+0000 7ff197f771c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-27T02:31:52.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.518+0000 7fe760db0700 1 -- 172.21.15.188:0/835555885 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7} v 0) v1 -- 0x7fe75c0987c0 con 0x7fe75c15bfe0 2024-01-27T02:31:52.520 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:52.521+0000 7ff197f771c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-27T02:31:52.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.522+0000 7fe74a7fc700 1 -- 172.21.15.188:0/835555885 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe750007700 con 0x7fe75c15bfe0 2024-01-27T02:31:52.522 INFO:teuthology.orchestra.run.smithi188.stdout:7 2024-01-27T02:31:52.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.523+0000 7fe760db0700 1 -- 172.21.15.188:0/835555885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe75c15bfe0 msgr2=0x7fe75c0c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:52.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.523+0000 7fe760db0700 1 --2- 172.21.15.188:0/835555885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe75c15bfe0 0x7fe75c0c8dd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe75000c5b0 tx=0x7fe750001040 comp rx=0 tx=0).stop 2024-01-27T02:31:52.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.523+0000 7fe760db0700 1 -- 172.21.15.188:0/835555885 shutdown_connections 2024-01-27T02:31:52.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.523+0000 7fe760db0700 1 --2- 172.21.15.188:0/835555885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe75c15c9b0 0x7fe75c0c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.523+0000 7fe760db0700 1 --2- 172.21.15.188:0/835555885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe75c1583e0 0x7fe75c09a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.523+0000 7fe760db0700 1 --2- 172.21.15.188:0/835555885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe75c15bfe0 0x7fe75c0c8dd0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.523+0000 7fe760db0700 1 -- 172.21.15.188:0/835555885 >> 172.21.15.188:0/835555885 conn(0x7fe75c0679b0 msgr2=0x7fe75c06eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:31:52.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.523+0000 7fe760db0700 1 -- 172.21.15.188:0/835555885 shutdown_connections 2024-01-27T02:31:52.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-27T02:31:52.523+0000 7fe760db0700 1 -- 172.21.15.188:0/835555885 wait complete. 2024-01-27T02:31:52.534 INFO:tasks.ceph.osd.0:Restarting daemon 2024-01-27T02:31:52.535 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-01-27T02:31:52.537 INFO:tasks.ceph.osd.0:Started 2024-01-27T02:31:52.537 INFO:tasks.ceph.osd.1:Restarting daemon 2024-01-27T02:31:52.537 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-01-27T02:31:52.539 INFO:tasks.ceph.osd.1:Started 2024-01-27T02:31:52.539 INFO:tasks.ceph.osd.2:Restarting daemon 2024-01-27T02:31:52.539 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-01-27T02:31:52.541 INFO:tasks.ceph.osd.2:Started 2024-01-27T02:31:52.541 INFO:tasks.ceph.osd.3:Restarting daemon 2024-01-27T02:31:52.542 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-01-27T02:31:52.544 INFO:tasks.ceph.osd.3:Started 2024-01-27T02:31:52.544 INFO:tasks.ceph.osd.4:Restarting daemon 2024-01-27T02:31:52.544 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-01-27T02:31:52.546 INFO:tasks.ceph.osd.4:Started 2024-01-27T02:31:52.546 INFO:tasks.ceph.osd.5:Restarting daemon 2024-01-27T02:31:52.546 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-01-27T02:31:52.548 INFO:tasks.ceph.osd.5:Started 2024-01-27T02:31:52.548 INFO:tasks.ceph.osd.6:Restarting daemon 2024-01-27T02:31:52.548 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-01-27T02:31:52.550 INFO:tasks.ceph.osd.6:Started 2024-01-27T02:31:52.550 INFO:tasks.ceph.osd.7:Restarting daemon 2024-01-27T02:31:52.550 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-01-27T02:31:52.552 INFO:tasks.ceph.osd.7:Started 2024-01-27T02:31:52.552 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:31:52.618 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:52.618+0000 7fa51bf991c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-27T02:31:52.629 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-01-27T02:31:52.629+0000 7ff9072fa540 -1 Falling back to public interface 2024-01-27T02:31:52.629 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-01-27T02:31:52.629+0000 7f2d99c59540 -1 Falling back to public interface 2024-01-27T02:31:52.629 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-01-27T02:31:52.629+0000 7ff90171a700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-01-27T02:31:52.632 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-01-27T02:31:52.632+0000 7f3eea176540 -1 Falling back to public interface 2024-01-27T02:31:52.640 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-01-27T02:31:52.637+0000 7f70c3220540 -1 Falling back to public interface 2024-01-27T02:31:52.647 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-01-27T02:31:52.647+0000 7ff9072fa540 -1 osd.4 0 log_to_monitors true 2024-01-27T02:31:52.649 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-01-27T02:31:52.649+0000 7f3eea176540 -1 osd.7 0 log_to_monitors true 2024-01-27T02:31:52.651 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-01-27T02:31:52.651+0000 7f2d99c59540 -1 osd.5 0 log_to_monitors true 2024-01-27T02:31:52.652 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-01-27T02:31:52.651+0000 7f9da5505540 -1 Falling back to public interface 2024-01-27T02:31:52.655 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-01-27T02:31:52.655+0000 7f81e74eb540 -1 Falling back to public interface 2024-01-27T02:31:52.659 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-01-27T02:31:52.659+0000 7f278857d540 -1 Falling back to public interface 2024-01-27T02:31:52.659 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-01-27T02:31:52.659+0000 7f70c3220540 -1 osd.1 0 log_to_monitors true 2024-01-27T02:31:52.667 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-01-27T02:31:52.666+0000 7f9da5505540 -1 osd.3 0 log_to_monitors true 2024-01-27T02:31:52.667 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-01-27T02:31:52.667+0000 7fcab6a4a540 -1 Falling back to public interface 2024-01-27T02:31:52.670 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-01-27T02:31:52.670+0000 7f81e74eb540 -1 osd.6 0 log_to_monitors true 2024-01-27T02:31:52.682 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-01-27T02:31:52.681+0000 7f278857d540 -1 osd.0 0 log_to_monitors true 2024-01-27T02:31:52.689 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-01-27T02:31:52.689+0000 7fcab6a4a540 -1 osd.2 0 log_to_monitors true 2024-01-27T02:31:52.735 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:52.734+0000 7fa51bf991c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-27T02:31:52.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.745+0000 7f2564925700 1 Processor -- start 2024-01-27T02:31:52.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.745+0000 7f2564925700 1 -- start start 2024-01-27T02:31:52.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.746+0000 7f2564925700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f25600ca540 0x7f25600cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:52.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.746+0000 7f2564925700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25600ce5e0 con 0x7f25600c4cc0 2024-01-27T02:31:52.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.746+0000 7f2564925700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25600cd9d0 con 0x7f256015a100 2024-01-27T02:31:52.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.746+0000 7f2564925700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25600cdcb0 con 0x7f25600ca540 2024-01-27T02:31:52.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.746+0000 7f255dd9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f25600c4cc0 0x7f2560161d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51492/0 (socket says 172.21.15.7:51492) 2024-01-27T02:31:52.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.746+0000 7f255dd9b700 1 -- 172.21.15.7:0/1836613069 learned_addr learned my addr 172.21.15.7:0/1836613069 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:31:52.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.746+0000 7f255ed9d700 1 --2- 172.21.15.7:0/1836613069 >> v2:172.21.15.188:3300/0 conn(0x7f25600ca540 0x7f25600cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.746+0000 7f255d59a700 1 -- 172.21.15.7:0/1836613069 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 379797221 0 0) 0x7f25600ce5e0 con 0x7f25600c4cc0 2024-01-27T02:31:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.746+0000 7f255d59a700 1 -- 172.21.15.7:0/1836613069 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2548003570 con 0x7f25600c4cc0 2024-01-27T02:31:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.746+0000 7f255d59a700 1 -- 172.21.15.7:0/1836613069 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3369934159 0 0) 0x7f25600cd9d0 con 0x7f256015a100 2024-01-27T02:31:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.746+0000 7f255d59a700 1 -- 172.21.15.7:0/1836613069 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25600ce5e0 con 0x7f256015a100 2024-01-27T02:31:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.746+0000 7f255ed9d700 1 -- 172.21.15.7:0/1836613069 >> v1:172.21.15.188:6789/0 conn(0x7f256015a100 legacy=0x7f25600c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.746+0000 7f255ed9d700 1 -- 172.21.15.7:0/1836613069 >> v1:172.21.15.7:6789/0 conn(0x7f25600c4cc0 legacy=0x7f2560161d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.746+0000 7f255ed9d700 1 -- 172.21.15.7:0/1836613069 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25600cdf90 con 0x7f25600ca540 2024-01-27T02:31:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.747+0000 7f255ed9d700 1 --2- 172.21.15.7:0/1836613069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25600ca540 0x7f25600cd2f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2558009e60 tx=0x7f2558009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=616da4b6c4fcac6f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:31:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.747+0000 7f255d59a700 1 -- 172.21.15.7:0/1836613069 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255800e070 con 0x7f25600ca540 2024-01-27T02:31:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.747+0000 7f255d59a700 1 -- 172.21.15.7:0/1836613069 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255800bab0 con 0x7f25600ca540 2024-01-27T02:31:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.747+0000 7f255d59a700 1 -- 172.21.15.7:0/1836613069 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255800be70 con 0x7f25600ca540 2024-01-27T02:31:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.747+0000 7f2564925700 1 -- 172.21.15.7:0/1836613069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25600ca540 msgr2=0x7f25600cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.747+0000 7f2564925700 1 --2- 172.21.15.7:0/1836613069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25600ca540 0x7f25600cd2f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2558009e60 tx=0x7f2558009780 comp rx=0 tx=0).stop 2024-01-27T02:31:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.748+0000 7f2564925700 1 -- 172.21.15.7:0/1836613069 shutdown_connections 2024-01-27T02:31:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.748+0000 7f2564925700 1 --2- 172.21.15.7:0/1836613069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25600ca540 0x7f25600cd2f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.748+0000 7f2564925700 1 -- 172.21.15.7:0/1836613069 >> 172.21.15.7:0/1836613069 conn(0x7f2560098020 msgr2=0x7f25600711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:31:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.748+0000 7f2564925700 1 -- 172.21.15.7:0/1836613069 shutdown_connections 2024-01-27T02:31:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.748+0000 7f2564925700 1 -- 172.21.15.7:0/1836613069 wait complete. 2024-01-27T02:31:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.749+0000 7f2564925700 1 Processor -- start 2024-01-27T02:31:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.749+0000 7f2564925700 1 -- start start 2024-01-27T02:31:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.750+0000 7f2564925700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25600c4cc0 0x7f256015cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.750+0000 7f2564925700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25600ca540 0x7f256016a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.750+0000 7f2564925700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256015a100 0x7f256016a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.750+0000 7f2564925700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256016ab30 con 0x7f256015a100 2024-01-27T02:31:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.750+0000 7f2564925700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25600b3ef0 con 0x7f25600ca540 2024-01-27T02:31:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.750+0000 7f2564925700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25600b4380 con 0x7f25600c4cc0 2024-01-27T02:31:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.750+0000 7f255ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256015a100 0x7f256016a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.750+0000 7f255ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256015a100 0x7f256016a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37290/0 (socket says 172.21.15.7:37290) 2024-01-27T02:31:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.750+0000 7f255ed9d700 1 -- 172.21.15.7:0/463259165 learned_addr learned my addr 172.21.15.7:0/463259165 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:31:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.750+0000 7f255dd9b700 1 --2- 172.21.15.7:0/463259165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25600ca540 0x7f256016a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.750+0000 7f255e59c700 1 --2- 172.21.15.7:0/463259165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25600c4cc0 0x7f256015cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.750+0000 7f255ed9d700 1 -- 172.21.15.7:0/463259165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25600c4cc0 msgr2=0x7f256015cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.750+0000 7f255ed9d700 1 --2- 172.21.15.7:0/463259165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25600c4cc0 0x7f256015cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.750+0000 7f255ed9d700 1 -- 172.21.15.7:0/463259165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25600ca540 msgr2=0x7f256016a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.750+0000 7f255ed9d700 1 --2- 172.21.15.7:0/463259165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25600ca540 0x7f256016a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.750+0000 7f255ed9d700 1 -- 172.21.15.7:0/463259165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2558009a20 con 0x7f256015a100 2024-01-27T02:31:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.750+0000 7f255ed9d700 1 --2- 172.21.15.7:0/463259165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256015a100 0x7f256016a5f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f256006ea80 tx=0x7f2558019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:31:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.751+0000 7f254e7fc700 1 -- 172.21.15.7:0/463259165 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2558012db0 con 0x7f256015a100 2024-01-27T02:31:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.751+0000 7f254e7fc700 1 -- 172.21.15.7:0/463259165 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2558026480 con 0x7f256015a100 2024-01-27T02:31:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.751+0000 7f254e7fc700 1 -- 172.21.15.7:0/463259165 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2558026840 con 0x7f256015a100 2024-01-27T02:31:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.751+0000 7f2564925700 1 -- 172.21.15.7:0/463259165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25600b35c0 con 0x7f256015a100 2024-01-27T02:31:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.751+0000 7f2564925700 1 -- 172.21.15.7:0/463259165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25600b3b80 con 0x7f256015a100 2024-01-27T02:31:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.751+0000 7f254e7fc700 1 -- 172.21.15.7:0/463259165 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f2558026bb0 con 0x7f256015a100 2024-01-27T02:31:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.751+0000 7f254e7fc700 1 -- 172.21.15.7:0/463259165 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f255801b400 con 0x7f256015a100 2024-01-27T02:31:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.752+0000 7f253ffff700 1 -- 172.21.15.7:0/463259165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25380052b0 con 0x7f256015a100 2024-01-27T02:31:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.754+0000 7f254e7fc700 1 -- 172.21.15.7:0/463259165 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f25580456d0 con 0x7f256015a100 2024-01-27T02:31:52.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.828+0000 7f253ffff700 1 -- 172.21.15.7:0/463259165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2538005830 con 0x7f256015a100 2024-01-27T02:31:52.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.829+0000 7f254e7fc700 1 -- 172.21.15.7:0/463259165 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f2558034030 con 0x7f256015a100 2024-01-27T02:31:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:31:52.830 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":10,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:31:52.519857+0000","last_up_change":"0.000000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:31:52.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.830+0000 7f253ffff700 1 -- 172.21.15.7:0/463259165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256015a100 msgr2=0x7f256016a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:52.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.830+0000 7f253ffff700 1 --2- 172.21.15.7:0/463259165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256015a100 0x7f256016a5f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f256006ea80 tx=0x7f2558019000 comp rx=0 tx=0).stop 2024-01-27T02:31:52.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.830+0000 7f253ffff700 1 -- 172.21.15.7:0/463259165 shutdown_connections 2024-01-27T02:31:52.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.830+0000 7f253ffff700 1 --2- 172.21.15.7:0/463259165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256015a100 0x7f256016a5f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.830+0000 7f253ffff700 1 --2- 172.21.15.7:0/463259165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25600c4cc0 0x7f256015cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.830+0000 7f253ffff700 1 --2- 172.21.15.7:0/463259165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25600ca540 0x7f256016a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:52.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.830+0000 7f253ffff700 1 -- 172.21.15.7:0/463259165 >> 172.21.15.7:0/463259165 conn(0x7f2560098020 msgr2=0x7f2560154ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:31:52.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.830+0000 7f253ffff700 1 -- 172.21.15.7:0/463259165 shutdown_connections 2024-01-27T02:31:52.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:31:52.830+0000 7f253ffff700 1 -- 172.21.15.7:0/463259165 wait complete. 2024-01-27T02:31:52.845 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-01-27T02:31:52.845 INFO:tasks.ceph:Waiting for OSDs to come up 2024-01-27T02:31:53.235 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:53.234+0000 7ff197f771c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-27T02:31:53.314 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:53.314+0000 7ff197f771c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-27T02:31:53.443 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:53.443+0000 7ff197f771c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-27T02:31:53.518 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:53.517+0000 7fa51bf991c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-27T02:31:53.608 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:53.607+0000 7fa51bf991c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-27T02:31:53.650 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-27T02:31:53.759 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:53.759+0000 7fa51bf991c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-27T02:31:53.770 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.770+0000 7fd443a5f700 1 Processor -- start 2024-01-27T02:31:53.771 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.770+0000 7fd443a5f700 1 -- start start 2024-01-27T02:31:53.773 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.771+0000 7fd443a5f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd43c0d1050 0x7fd43c0cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:53.773 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.771+0000 7fd443a5f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd43c0d1420 0x7fd43c0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:53.773 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.771+0000 7fd443a5f700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd43c0cdd60 con 0x7fd43c159ba0 2024-01-27T02:31:53.773 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.771+0000 7fd443a5f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43c0ce850 con 0x7fd43c0d1050 2024-01-27T02:31:53.773 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.771+0000 7fd443a5f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c15a6a0 con 0x7fd43c0d1420 2024-01-27T02:31:53.773 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.771+0000 7fd4417fb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd43c0d1050 0x7fd43c0cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:53.773 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.771+0000 7fd4417fb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd43c0d1050 0x7fd43c0cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37302/0 (socket says 172.21.15.7:37302) 2024-01-27T02:31:53.773 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.771+0000 7fd4417fb700 1 -- 172.21.15.7:0/168880917 learned_addr learned my addr 172.21.15.7:0/168880917 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:31:53.773 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.771+0000 7fd441ffc700 1 --1- 172.21.15.7:0/168880917 >> v1:172.21.15.188:6789/0 conn(0x7fd43c159ba0 0x7fd43c159f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:34276/0 (socket says 172.21.15.7:34276) 2024-01-27T02:31:53.773 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.771+0000 7fd440ffa700 1 --2- 172.21.15.7:0/168880917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd43c0d1420 0x7fd43c0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:53.773 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.772+0000 7fd4417fb700 1 -- 172.21.15.7:0/168880917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd43c0d1420 msgr2=0x7fd43c0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:31:53.773 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.772+0000 7fd4417fb700 1 --2- 172.21.15.7:0/168880917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd43c0d1420 0x7fd43c0cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:53.774 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.772+0000 7fd4417fb700 1 -- 172.21.15.7:0/168880917 >> v1:172.21.15.188:6789/0 conn(0x7fd43c159ba0 legacy=0x7fd43c159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:53.774 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.772+0000 7fd43bfff700 1 -- 172.21.15.7:0/168880917 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247609383 0 0) 0x7fd43c0cdd60 con 0x7fd43c159ba0 2024-01-27T02:31:53.774 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.772+0000 7fd4417fb700 1 -- 172.21.15.7:0/168880917 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43c0cdfb0 con 0x7fd43c0d1050 2024-01-27T02:31:53.774 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.772+0000 7fd4417fb700 1 --2- 172.21.15.7:0/168880917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43c0d1050 0x7fd43c0cc280 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd434009990 tx=0x7fd434009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cce8d0565f67a9 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:31:53.774 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.772+0000 7fd43bfff700 1 -- 172.21.15.7:0/168880917 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434009e80 con 0x7fd43c0d1050 2024-01-27T02:31:53.774 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.772+0000 7fd43bfff700 1 -- 172.21.15.7:0/168880917 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd434005c10 con 0x7fd43c0d1050 2024-01-27T02:31:53.774 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.772+0000 7fd43bfff700 1 -- 172.21.15.7:0/168880917 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43400b650 con 0x7fd43c0d1050 2024-01-27T02:31:53.775 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.773+0000 7fd443a5f700 1 -- 172.21.15.7:0/168880917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43c0d1050 msgr2=0x7fd43c0cc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:53.775 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.773+0000 7fd443a5f700 1 --2- 172.21.15.7:0/168880917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43c0d1050 0x7fd43c0cc280 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd434009990 tx=0x7fd434009220 comp rx=0 tx=0).stop 2024-01-27T02:31:53.775 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.773+0000 7fd443a5f700 1 -- 172.21.15.7:0/168880917 shutdown_connections 2024-01-27T02:31:53.775 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.773+0000 7fd443a5f700 1 --2- 172.21.15.7:0/168880917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43c0d1050 0x7fd43c0cc280 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:53.775 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.773+0000 7fd443a5f700 1 --2- 172.21.15.7:0/168880917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd43c0d1420 0x7fd43c0cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:53.775 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.773+0000 7fd443a5f700 1 -- 172.21.15.7:0/168880917 >> 172.21.15.7:0/168880917 conn(0x7fd43c0679b0 msgr2=0x7fd43c067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:31:53.776 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.773+0000 7fd443a5f700 1 -- 172.21.15.7:0/168880917 shutdown_connections 2024-01-27T02:31:53.776 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.773+0000 7fd443a5f700 1 -- 172.21.15.7:0/168880917 wait complete. 2024-01-27T02:31:53.776 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.774+0000 7fd443a5f700 1 Processor -- start 2024-01-27T02:31:53.776 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.774+0000 7fd443a5f700 1 -- start start 2024-01-27T02:31:53.778 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.775+0000 7fd443a5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43c0d1050 0x7fd43c0c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:53.778 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.775+0000 7fd443a5f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd43c0d1420 0x7fd43c147910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:53.779 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.775+0000 7fd443a5f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd43c159ba0 0x7fd43c0ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:31:53.779 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.775+0000 7fd443a5f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c0c9780 con 0x7fd43c0d1050 2024-01-27T02:31:53.779 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.775+0000 7fd443a5f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c0c8b70 con 0x7fd43c159ba0 2024-01-27T02:31:53.779 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.775+0000 7fd443a5f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c0c8e50 con 0x7fd43c0d1420 2024-01-27T02:31:53.779 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.775+0000 7fd4417fb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43c0d1050 0x7fd43c0c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:53.779 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.775+0000 7fd4417fb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43c0d1050 0x7fd43c0c9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37304/0 (socket says 172.21.15.7:37304) 2024-01-27T02:31:53.779 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.775+0000 7fd4417fb700 1 -- 172.21.15.7:0/314229356 learned_addr learned my addr 172.21.15.7:0/314229356 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:31:53.779 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.775+0000 7fd440ffa700 1 --2- 172.21.15.7:0/314229356 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd43c0d1420 0x7fd43c147910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:53.779 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.775+0000 7fd441ffc700 1 --2- 172.21.15.7:0/314229356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd43c159ba0 0x7fd43c0ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:31:53.779 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.776+0000 7fd4417fb700 1 -- 172.21.15.7:0/314229356 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd43c0d1420 msgr2=0x7fd43c147910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:53.779 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.776+0000 7fd4417fb700 1 --2- 172.21.15.7:0/314229356 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd43c0d1420 0x7fd43c147910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:53.779 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.776+0000 7fd4417fb700 1 -- 172.21.15.7:0/314229356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd43c159ba0 msgr2=0x7fd43c0ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:53.779 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.776+0000 7fd4417fb700 1 --2- 172.21.15.7:0/314229356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd43c159ba0 0x7fd43c0ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:53.779 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.776+0000 7fd4417fb700 1 -- 172.21.15.7:0/314229356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd434009480 con 0x7fd43c0d1050 2024-01-27T02:31:53.780 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.776+0000 7fd440ffa700 1 --2- 172.21.15.7:0/314229356 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd43c0d1420 0x7fd43c147910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:31:53.780 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.776+0000 7fd4417fb700 1 --2- 172.21.15.7:0/314229356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43c0d1050 0x7fd43c0c9c10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd43c089dc0 tx=0x7fd434016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:31:53.780 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.776+0000 7fd4397fa700 1 -- 172.21.15.7:0/314229356 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43401db60 con 0x7fd43c0d1050 2024-01-27T02:31:53.780 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.776+0000 7fd4397fa700 1 -- 172.21.15.7:0/314229356 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43401dce0 con 0x7fd43c0d1050 2024-01-27T02:31:53.780 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.776+0000 7fd443a5f700 1 -- 172.21.15.7:0/314229356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd43c0c9130 con 0x7fd43c0d1050 2024-01-27T02:31:53.780 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.776+0000 7fd4397fa700 1 -- 172.21.15.7:0/314229356 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43401b6d0 con 0x7fd43c0d1050 2024-01-27T02:31:53.780 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.776+0000 7fd443a5f700 1 -- 172.21.15.7:0/314229356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd43c147e50 con 0x7fd43c0d1050 2024-01-27T02:31:53.780 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.777+0000 7fd4397fa700 1 -- 172.21.15.7:0/314229356 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fd43401ba40 con 0x7fd43c0d1050 2024-01-27T02:31:53.780 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.777+0000 7fd4397fa700 1 -- 172.21.15.7:0/314229356 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 1879+0+0 (secure 0 0 0) 0x7fd434026c60 con 0x7fd43c0d1050 2024-01-27T02:31:53.780 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.777+0000 7fd443a5f700 1 -- 172.21.15.7:0/314229356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd43c0ca150 con 0x7fd43c0d1050 2024-01-27T02:31:53.781 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.779+0000 7fd4397fa700 1 -- 172.21.15.7:0/314229356 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd43402b020 con 0x7fd43c0d1050 2024-01-27T02:31:53.854 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.854+0000 7fd443a5f700 1 -- 172.21.15.7:0/314229356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd43c0c9a60 con 0x7fd43c0d1050 2024-01-27T02:31:53.855 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.854+0000 7fd4397fa700 1 -- 172.21.15.7:0/314229356 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fd434026490 con 0x7fd43c0d1050 2024-01-27T02:31:53.855 INFO:teuthology.misc.health.smithi007.stdout: 2024-01-27T02:31:53.855 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":11,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:31:53.520803+0000","last_up_change":"0.000000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:31:53.856 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.855+0000 7fd443a5f700 1 -- 172.21.15.7:0/314229356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43c0d1050 msgr2=0x7fd43c0c9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:31:53.856 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.855+0000 7fd443a5f700 1 --2- 172.21.15.7:0/314229356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43c0d1050 0x7fd43c0c9c10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd43c089dc0 tx=0x7fd434016000 comp rx=0 tx=0).stop 2024-01-27T02:31:53.856 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.855+0000 7fd443a5f700 1 -- 172.21.15.7:0/314229356 shutdown_connections 2024-01-27T02:31:53.856 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.855+0000 7fd443a5f700 1 --2- 172.21.15.7:0/314229356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd43c159ba0 0x7fd43c0ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:53.856 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.855+0000 7fd443a5f700 1 --2- 172.21.15.7:0/314229356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43c0d1050 0x7fd43c0c9c10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:53.856 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.855+0000 7fd443a5f700 1 --2- 172.21.15.7:0/314229356 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd43c0d1420 0x7fd43c147910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:31:53.856 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.855+0000 7fd443a5f700 1 -- 172.21.15.7:0/314229356 >> 172.21.15.7:0/314229356 conn(0x7fd43c0679b0 msgr2=0x7fd43c158f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:31:53.857 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.855+0000 7fd443a5f700 1 -- 172.21.15.7:0/314229356 shutdown_connections 2024-01-27T02:31:53.857 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:31:53.855+0000 7fd443a5f700 1 -- 172.21.15.7:0/314229356 wait complete. 2024-01-27T02:31:53.867 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-01-27T02:31:53.990 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:53.990+0000 7ff197f771c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-27T02:31:54.146 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:54.146+0000 7ff197f771c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-27T02:31:54.307 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:54.307+0000 7ff197f771c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-27T02:31:54.392 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:54.392+0000 7fa51bf991c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-27T02:31:54.541 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-01-27T02:31:54.541+0000 7f3ee1590700 -1 osd.7 0 waiting for initial osdmap 2024-01-27T02:31:54.542 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-01-27T02:31:54.542+0000 7f9d9d120700 -1 osd.3 0 waiting for initial osdmap 2024-01-27T02:31:54.542 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-01-27T02:31:54.542+0000 7ff8fe714700 -1 osd.4 0 waiting for initial osdmap 2024-01-27T02:31:54.543 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-01-27T02:31:54.543+0000 7f3edb389700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-27T02:31:54.543 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-01-27T02:31:54.543+0000 7ff8f850d700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-27T02:31:54.544 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-01-27T02:31:54.543+0000 7f277f997700 -1 osd.0 0 waiting for initial osdmap 2024-01-27T02:31:54.544 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-01-27T02:31:54.544+0000 7f9d96718700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-27T02:31:54.545 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-01-27T02:31:54.545+0000 7f2779790700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-27T02:31:54.548 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-01-27T02:31:54.548+0000 7f2d91073700 -1 osd.5 0 waiting for initial osdmap 2024-01-27T02:31:54.549 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-01-27T02:31:54.549+0000 7f81de905700 -1 osd.6 0 waiting for initial osdmap 2024-01-27T02:31:54.550 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-01-27T02:31:54.550+0000 7f2d8ae6c700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-27T02:31:54.550 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-01-27T02:31:54.551+0000 7f81d86fe700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-27T02:31:54.551 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-01-27T02:31:54.551+0000 7f70ba63a700 -1 osd.1 0 waiting for initial osdmap 2024-01-27T02:31:54.552 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-01-27T02:31:54.552+0000 7fcaae665700 -1 osd.2 0 waiting for initial osdmap 2024-01-27T02:31:54.554 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-01-27T02:31:54.553+0000 7f70b4433700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-27T02:31:54.554 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-01-27T02:31:54.554+0000 7fcaa7c5d700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-27T02:31:54.568 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:54.567+0000 7fa51bf991c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-27T02:31:54.751 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:54.750+0000 7fa51bf991c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-27T02:31:55.103 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:55.103+0000 7ff197f771c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-27T02:31:55.317 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:55.317+0000 7ff197f771c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-27T02:31:55.387 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:55.387+0000 7ff197f771c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-27T02:31:55.547 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:55.547+0000 7ff197f771c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-27T02:31:55.655 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:55.655+0000 7fa51bf991c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-27T02:31:55.898 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:55.898+0000 7fa51bf991c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-27T02:31:55.979 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:55.979+0000 7fa51bf991c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-27T02:31:56.160 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-27T02:31:56.159+0000 7fa51bf991c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-27T02:31:56.585 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:56.585+0000 7ff156a2e700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-01-27T02:31:56.585 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-27T02:31:56.585+0000 7ff156a2e700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-01-27T02:32:00.675 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-27T02:32:00.804 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.804+0000 7f1046f51700 1 Processor -- start 2024-01-27T02:32:00.804 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.804+0000 7f1046f51700 1 -- start start 2024-01-27T02:32:00.806 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.804+0000 7f1046f51700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1040156ab0 0x7f1040158e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:00.806 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.804+0000 7f1046f51700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104015a4c0 con 0x7f104015b000 2024-01-27T02:32:00.806 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.804+0000 7f1046f51700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10401598b0 con 0x7f10401593b0 2024-01-27T02:32:00.806 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.804+0000 7f1046f51700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1040159b90 con 0x7f1040156ab0 2024-01-27T02:32:00.806 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.805+0000 7f1044ced700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1040156ab0 0x7f1040158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:00.806 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.805+0000 7f10454ee700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f104015b000 0x7f1040156e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51502/0 (socket says 172.21.15.7:51502) 2024-01-27T02:32:00.806 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.805+0000 7f10454ee700 1 -- 172.21.15.7:0/965065229 learned_addr learned my addr 172.21.15.7:0/965065229 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:00.806 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.805+0000 7f1044ced700 1 -- 172.21.15.7:0/965065229 >> v1:172.21.15.188:6789/0 conn(0x7f10401593b0 legacy=0x7f104015a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:00.806 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.805+0000 7f1044ced700 1 -- 172.21.15.7:0/965065229 >> v1:172.21.15.7:6789/0 conn(0x7f104015b000 legacy=0x7f1040156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:00.806 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.805+0000 7f1044ced700 1 -- 172.21.15.7:0/965065229 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1040159e70 con 0x7f1040156ab0 2024-01-27T02:32:00.806 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.805+0000 7f1044ced700 1 --2- 172.21.15.7:0/965065229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1040156ab0 0x7f1040158e70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f10340098e0 tx=0x7f10340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df000b127bb12014 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:00.806 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.805+0000 7f103f7fe700 1 -- 172.21.15.7:0/965065229 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034009e80 con 0x7f1040156ab0 2024-01-27T02:32:00.807 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.805+0000 7f103f7fe700 1 -- 172.21.15.7:0/965065229 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103400b860 con 0x7f1040156ab0 2024-01-27T02:32:00.807 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.805+0000 7f103f7fe700 1 -- 172.21.15.7:0/965065229 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103400bbe0 con 0x7f1040156ab0 2024-01-27T02:32:00.807 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.806+0000 7f1046f51700 1 -- 172.21.15.7:0/965065229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1040156ab0 msgr2=0x7f1040158e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:00.808 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.806+0000 7f1046f51700 1 --2- 172.21.15.7:0/965065229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1040156ab0 0x7f1040158e70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f10340098e0 tx=0x7f10340091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:00.808 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.806+0000 7f1046f51700 1 -- 172.21.15.7:0/965065229 shutdown_connections 2024-01-27T02:32:00.808 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.806+0000 7f1046f51700 1 --2- 172.21.15.7:0/965065229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1040156ab0 0x7f1040158e70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:00.808 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.806+0000 7f1046f51700 1 -- 172.21.15.7:0/965065229 >> 172.21.15.7:0/965065229 conn(0x7f1040068480 msgr2=0x7f10400cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:00.808 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.806+0000 7f1046f51700 1 -- 172.21.15.7:0/965065229 shutdown_connections 2024-01-27T02:32:00.808 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.806+0000 7f1046f51700 1 -- 172.21.15.7:0/965065229 wait complete. 2024-01-27T02:32:00.808 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.807+0000 7f1046f51700 1 Processor -- start 2024-01-27T02:32:00.808 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.807+0000 7f1046f51700 1 -- start start 2024-01-27T02:32:00.810 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.808+0000 7f1046f51700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1040156ab0 0x7f10401534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:00.810 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.808+0000 7f1046f51700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10401593b0 0x7f10400abe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:00.810 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.808+0000 7f1046f51700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104015b000 0x7f1040063580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:00.810 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.808+0000 7f1046f51700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10400a8bd0 con 0x7f1040156ab0 2024-01-27T02:32:00.810 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.808+0000 7f1046f51700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10400a7fc0 con 0x7f104015b000 2024-01-27T02:32:00.810 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.808+0000 7f1046f51700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10400a82a0 con 0x7f10401593b0 2024-01-27T02:32:00.810 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.808+0000 7f1044ced700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1040156ab0 0x7f10401534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:00.811 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.808+0000 7f1044ced700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1040156ab0 0x7f10401534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37328/0 (socket says 172.21.15.7:37328) 2024-01-27T02:32:00.811 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.808+0000 7f1044ced700 1 -- 172.21.15.7:0/3263620916 learned_addr learned my addr 172.21.15.7:0/3263620916 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:00.811 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.809+0000 7f103ffff700 1 --2- 172.21.15.7:0/3263620916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10401593b0 0x7f10400abe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:00.811 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.809+0000 7f10454ee700 1 --2- 172.21.15.7:0/3263620916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104015b000 0x7f1040063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:00.811 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.809+0000 7f1044ced700 1 -- 172.21.15.7:0/3263620916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10401593b0 msgr2=0x7f10400abe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:00.811 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.809+0000 7f1044ced700 1 --2- 172.21.15.7:0/3263620916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10401593b0 0x7f10400abe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:00.811 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.809+0000 7f1044ced700 1 -- 172.21.15.7:0/3263620916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104015b000 msgr2=0x7f1040063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:00.811 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.809+0000 7f1044ced700 1 --2- 172.21.15.7:0/3263620916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104015b000 0x7f1040063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:00.811 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.809+0000 7f1044ced700 1 -- 172.21.15.7:0/3263620916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1034009420 con 0x7f1040156ab0 2024-01-27T02:32:00.811 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.809+0000 7f1044ced700 1 --2- 172.21.15.7:0/3263620916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1040156ab0 0x7f10401534f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1040069c30 tx=0x7f1034017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:00.811 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.809+0000 7f103cff9700 1 -- 172.21.15.7:0/3263620916 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103400bb20 con 0x7f1040156ab0 2024-01-27T02:32:00.811 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.809+0000 7f103cff9700 1 -- 172.21.15.7:0/3263620916 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1034025420 con 0x7f1040156ab0 2024-01-27T02:32:00.811 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.809+0000 7f1046f51700 1 -- 172.21.15.7:0/3263620916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10400a8580 con 0x7f1040156ab0 2024-01-27T02:32:00.811 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.809+0000 7f103cff9700 1 -- 172.21.15.7:0/3263620916 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10340257c0 con 0x7f1040156ab0 2024-01-27T02:32:00.811 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.809+0000 7f1046f51700 1 -- 172.21.15.7:0/3263620916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10400ac370 con 0x7f1040156ab0 2024-01-27T02:32:00.812 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.810+0000 7f1046f51700 1 -- 172.21.15.7:0/3263620916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10400a8eb0 con 0x7f1040156ab0 2024-01-27T02:32:00.813 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.810+0000 7f103cff9700 1 -- 172.21.15.7:0/3263620916 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 84686+0+0 (secure 0 0 0) 0x7f1034023070 con 0x7f1040156ab0 2024-01-27T02:32:00.813 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.811+0000 7f103cff9700 1 --2- 172.21.15.7:0/3263620916 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1024066ea0 0x7f1024069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:00.813 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.811+0000 7f103cff9700 1 -- 172.21.15.7:0/3263620916 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f10340857d0 con 0x7f1040156ab0 2024-01-27T02:32:00.813 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.811+0000 7f103ffff700 1 --2- 172.21.15.7:0/3263620916 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1024066ea0 0x7f1024069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:00.813 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.812+0000 7f103ffff700 1 --2- 172.21.15.7:0/3263620916 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1024066ea0 0x7f1024069350 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f10400a93c0 tx=0x7f102c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:00.816 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:00.816+0000 7f103cff9700 1 -- 172.21.15.7:0/3263620916 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1034052e70 con 0x7f1040156ab0 2024-01-27T02:32:01.019 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:01.018+0000 7f1046f51700 1 -- 172.21.15.7:0/3263620916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10400ac560 con 0x7f1040156ab0 2024-01-27T02:32:01.020 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:01.019+0000 7f103cff9700 1 -- 172.21.15.7:0/3263620916 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12079 (secure 0 0 0) 0x7f1034056490 con 0x7f1040156ab0 2024-01-27T02:32:01.020 INFO:teuthology.misc.health.smithi007.stdout: 2024-01-27T02:32:01.020 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":17,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:32:00.579459+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:32:01.023 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:01.021+0000 7f1046f51700 1 -- 172.21.15.7:0/3263620916 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1024066ea0 msgr2=0x7f1024069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:01.023 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:01.021+0000 7f1046f51700 1 --2- 172.21.15.7:0/3263620916 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1024066ea0 0x7f1024069350 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f10400a93c0 tx=0x7f102c009040 comp rx=0 tx=0).stop 2024-01-27T02:32:01.023 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:01.021+0000 7f1046f51700 1 -- 172.21.15.7:0/3263620916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1040156ab0 msgr2=0x7f10401534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:01.023 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:01.021+0000 7f1046f51700 1 --2- 172.21.15.7:0/3263620916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1040156ab0 0x7f10401534f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1040069c30 tx=0x7f1034017000 comp rx=0 tx=0).stop 2024-01-27T02:32:01.023 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:01.021+0000 7f1046f51700 1 -- 172.21.15.7:0/3263620916 shutdown_connections 2024-01-27T02:32:01.023 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:01.021+0000 7f1046f51700 1 --2- 172.21.15.7:0/3263620916 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1024066ea0 0x7f1024069350 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.023 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:01.021+0000 7f1046f51700 1 --2- 172.21.15.7:0/3263620916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104015b000 0x7f1040063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.023 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:01.021+0000 7f1046f51700 1 --2- 172.21.15.7:0/3263620916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1040156ab0 0x7f10401534f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.023 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:01.021+0000 7f1046f51700 1 --2- 172.21.15.7:0/3263620916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10401593b0 0x7f10400abe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.023 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:01.021+0000 7f1046f51700 1 -- 172.21.15.7:0/3263620916 >> 172.21.15.7:0/3263620916 conn(0x7f1040068480 msgr2=0x7f1040156e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:01.023 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:01.021+0000 7f1046f51700 1 -- 172.21.15.7:0/3263620916 shutdown_connections 2024-01-27T02:32:01.023 INFO:teuthology.misc.health.smithi007.stderr:2024-01-27T02:32:01.021+0000 7f1046f51700 1 -- 172.21.15.7:0/3263620916 wait complete. 2024-01-27T02:32:01.035 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-01-27T02:32:01.035 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-01-27T02:32:01.035 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-27T02:32:01.035 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-27T02:32:01.037 INFO:tasks.ceph.mds.a:Started 2024-01-27T02:32:01.038 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-27T02:32:01.038 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-27T02:32:01.039 INFO:tasks.ceph.mds.c:Started 2024-01-27T02:32:01.040 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-27T02:32:01.040 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-27T02:32:01.042 INFO:tasks.ceph.mds.b:Started 2024-01-27T02:32:01.042 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-27T02:32:01.042 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-27T02:32:01.044 INFO:tasks.ceph.mds.d:Started 2024-01-27T02:32:01.044 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-01-27T02:32:01.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:32:01.125 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-27T02:32:01.263 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-27T02:32:01.267 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-01-27T02:32:01.267 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-01-27T02:32:01.268 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-01-27T02:32:01.129+0000 7f47b4486700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-01-27T02:32:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.182+0000 7f08b3e13700 1 Processor -- start 2024-01-27T02:32:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.182+0000 7f08b3e13700 1 -- start start 2024-01-27T02:32:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.183+0000 7f08b3e13700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f08ac0c4cc0 0x7f08ac0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.183+0000 7f08b3e13700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f08ac0ca540 0x7f08ac163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.183+0000 7f08b3e13700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ac0ce5e0 con 0x7f08ac15a100 2024-01-27T02:32:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.183+0000 7f08b3e13700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ac0cd9d0 con 0x7f08ac0ca540 2024-01-27T02:32:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.183+0000 7f08b3e13700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ac0cdcb0 con 0x7f08ac0c4cc0 2024-01-27T02:32:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.183+0000 7f08b23b0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f08ac0ca540 0x7f08ac163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.183+0000 7f08b23b0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f08ac0ca540 0x7f08ac163d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37356/0 (socket says 172.21.15.7:37356) 2024-01-27T02:32:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.183+0000 7f08b23b0700 1 -- 172.21.15.7:0/2349193827 learned_addr learned my addr 172.21.15.7:0/2349193827 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.183+0000 7f08b13ae700 1 --2- 172.21.15.7:0/2349193827 >> v2:172.21.15.188:3300/0 conn(0x7f08ac0c4cc0 0x7f08ac0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.184+0000 7f08b23b0700 1 -- 172.21.15.7:0/2349193827 >> v2:172.21.15.188:3300/0 conn(0x7f08ac0c4cc0 msgr2=0x7f08ac0cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.184+0000 7f08b23b0700 1 --2- 172.21.15.7:0/2349193827 >> v2:172.21.15.188:3300/0 conn(0x7f08ac0c4cc0 0x7f08ac0cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.184+0000 7f08b23b0700 1 -- 172.21.15.7:0/2349193827 >> v1:172.21.15.188:6789/0 conn(0x7f08ac15a100 legacy=0x7f08ac0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.184+0000 7f08b23b0700 1 -- 172.21.15.7:0/2349193827 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac0cdf90 con 0x7f08ac0ca540 2024-01-27T02:32:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.184+0000 7f08b23b0700 1 --2- 172.21.15.7:0/2349193827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ac0ca540 0x7f08ac163d90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f08ac06e170 tx=0x7f08a4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12df85c9d6135f67 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.184+0000 7f08b0bad700 1 -- 172.21.15.7:0/2349193827 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a400e070 con 0x7f08ac0ca540 2024-01-27T02:32:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.184+0000 7f08b0bad700 1 -- 172.21.15.7:0/2349193827 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a400ba20 con 0x7f08ac0ca540 2024-01-27T02:32:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.184+0000 7f08b0bad700 1 -- 172.21.15.7:0/2349193827 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a40092e0 con 0x7f08ac0ca540 2024-01-27T02:32:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.184+0000 7f08b3e13700 1 -- 172.21.15.7:0/2349193827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ac0ca540 msgr2=0x7f08ac163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.184+0000 7f08b3e13700 1 --2- 172.21.15.7:0/2349193827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ac0ca540 0x7f08ac163d90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f08ac06e170 tx=0x7f08a4009840 comp rx=0 tx=0).stop 2024-01-27T02:32:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.185+0000 7f08b3e13700 1 -- 172.21.15.7:0/2349193827 shutdown_connections 2024-01-27T02:32:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.185+0000 7f08b3e13700 1 --2- 172.21.15.7:0/2349193827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ac0ca540 0x7f08ac163d90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.185+0000 7f08b3e13700 1 --2- 172.21.15.7:0/2349193827 >> v2:172.21.15.188:3300/0 conn(0x7f08ac0c4cc0 0x7f08ac0cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.185+0000 7f08b3e13700 1 -- 172.21.15.7:0/2349193827 >> 172.21.15.7:0/2349193827 conn(0x7f08ac098020 msgr2=0x7f08ac0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.185+0000 7f08b3e13700 1 -- 172.21.15.7:0/2349193827 shutdown_connections 2024-01-27T02:32:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.185+0000 7f08b3e13700 1 -- 172.21.15.7:0/2349193827 wait complete. 2024-01-27T02:32:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.186+0000 7f08b3e13700 1 Processor -- start 2024-01-27T02:32:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.186+0000 7f08b3e13700 1 -- start start 2024-01-27T02:32:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.187+0000 7f08b3e13700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08ac0c4cc0 0x7f08ac14abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.187+0000 7f08b3e13700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08ac0ca540 0x7f08ac15a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.187+0000 7f08b3e13700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ac15a100 0x7f08ac16d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.187+0000 7f08b3e13700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac0b3500 con 0x7f08ac15a100 2024-01-27T02:32:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.187+0000 7f08b3e13700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac0b3ac0 con 0x7f08ac0c4cc0 2024-01-27T02:32:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.187+0000 7f08b3e13700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac16dfd0 con 0x7f08ac0ca540 2024-01-27T02:32:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.187+0000 7f08b23b0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ac15a100 0x7f08ac16d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.187+0000 7f08b23b0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ac15a100 0x7f08ac16d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37368/0 (socket says 172.21.15.7:37368) 2024-01-27T02:32:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.187+0000 7f08b23b0700 1 -- 172.21.15.7:0/80976248 learned_addr learned my addr 172.21.15.7:0/80976248 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.187+0000 7f08b1baf700 1 --2- 172.21.15.7:0/80976248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08ac0c4cc0 0x7f08ac14abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.187+0000 7f08b13ae700 1 --2- 172.21.15.7:0/80976248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08ac0ca540 0x7f08ac15a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.187+0000 7f08b23b0700 1 -- 172.21.15.7:0/80976248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08ac0ca540 msgr2=0x7f08ac15a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.187+0000 7f08b23b0700 1 --2- 172.21.15.7:0/80976248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08ac0ca540 0x7f08ac15a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.187+0000 7f08b23b0700 1 -- 172.21.15.7:0/80976248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08ac0c4cc0 msgr2=0x7f08ac14abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:01.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.187+0000 7f08b23b0700 1 --2- 172.21.15.7:0/80976248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08ac0c4cc0 0x7f08ac14abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.187+0000 7f08b23b0700 1 -- 172.21.15.7:0/80976248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a4009ae0 con 0x7f08ac15a100 2024-01-27T02:32:01.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.187+0000 7f08b23b0700 1 --2- 172.21.15.7:0/80976248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ac15a100 0x7f08ac16d550 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f08ac06ea80 tx=0x7f08a4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:01.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.187+0000 7f08a1ffb700 1 -- 172.21.15.7:0/80976248 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a4008a80 con 0x7f08ac15a100 2024-01-27T02:32:01.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.187+0000 7f08a1ffb700 1 -- 172.21.15.7:0/80976248 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a4008be0 con 0x7f08ac15a100 2024-01-27T02:32:01.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.188+0000 7f08b3e13700 1 -- 172.21.15.7:0/80976248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ac16e2b0 con 0x7f08ac15a100 2024-01-27T02:32:01.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.188+0000 7f08a1ffb700 1 -- 172.21.15.7:0/80976248 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a4026460 con 0x7f08ac15a100 2024-01-27T02:32:01.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.188+0000 7f08b3e13700 1 -- 172.21.15.7:0/80976248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ac14b120 con 0x7f08ac15a100 2024-01-27T02:32:01.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.189+0000 7f08b3e13700 1 -- 172.21.15.7:0/80976248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ac04c710 con 0x7f08ac15a100 2024-01-27T02:32:01.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.189+0000 7f08a1ffb700 1 -- 172.21.15.7:0/80976248 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 84686+0+0 (secure 0 0 0) 0x7f08a4003040 con 0x7f08ac15a100 2024-01-27T02:32:01.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.190+0000 7f08a1ffb700 1 --2- 172.21.15.7:0/80976248 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0890066ea0 0x7f0890069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:01.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.190+0000 7f08a1ffb700 1 -- 172.21.15.7:0/80976248 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f08a4085ec0 con 0x7f08ac15a100 2024-01-27T02:32:01.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.190+0000 7f08b1baf700 1 --2- 172.21.15.7:0/80976248 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0890066ea0 0x7f0890069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:01.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.190+0000 7f08b1baf700 1 --2- 172.21.15.7:0/80976248 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0890066ea0 0x7f0890069350 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f08a80039e0 tx=0x7f08a8007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:01.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.194+0000 7f08a1ffb700 1 -- 172.21.15.7:0/80976248 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08a4053560 con 0x7f08ac15a100 2024-01-27T02:32:01.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.400+0000 7f08b3e13700 1 -- 172.21.15.7:0/80976248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08ac0b3c00 con 0x7f08ac15a100 2024-01-27T02:32:01.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.400+0000 7f08a1ffb700 1 -- 172.21.15.7:0/80976248 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12079 (secure 0 0 0) 0x7f08a4056b80 con 0x7f08ac15a100 2024-01-27T02:32:01.402 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:01.402 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:32:00.579459+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:32:01.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.403+0000 7f08b3e13700 1 -- 172.21.15.7:0/80976248 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0890066ea0 msgr2=0x7f0890069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:01.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.403+0000 7f08b3e13700 1 --2- 172.21.15.7:0/80976248 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0890066ea0 0x7f0890069350 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f08a80039e0 tx=0x7f08a8007ec0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.403+0000 7f08b3e13700 1 -- 172.21.15.7:0/80976248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ac15a100 msgr2=0x7f08ac16d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:01.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.403+0000 7f08b3e13700 1 --2- 172.21.15.7:0/80976248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ac15a100 0x7f08ac16d550 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f08ac06ea80 tx=0x7f08a4018000 comp rx=0 tx=0).stop 2024-01-27T02:32:01.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.403+0000 7f08b3e13700 1 -- 172.21.15.7:0/80976248 shutdown_connections 2024-01-27T02:32:01.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.403+0000 7f08b3e13700 1 --2- 172.21.15.7:0/80976248 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0890066ea0 0x7f0890069350 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.403+0000 7f08b3e13700 1 --2- 172.21.15.7:0/80976248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ac15a100 0x7f08ac16d550 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.403+0000 7f08b3e13700 1 --2- 172.21.15.7:0/80976248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08ac0c4cc0 0x7f08ac14abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.403+0000 7f08b3e13700 1 --2- 172.21.15.7:0/80976248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08ac0ca540 0x7f08ac15a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.403+0000 7f08b3e13700 1 -- 172.21.15.7:0/80976248 >> 172.21.15.7:0/80976248 conn(0x7f08ac098020 msgr2=0x7f08ac0c9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:01.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.403+0000 7f08b3e13700 1 -- 172.21.15.7:0/80976248 shutdown_connections 2024-01-27T02:32:01.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.403+0000 7f08b3e13700 1 -- 172.21.15.7:0/80976248 wait complete. 2024-01-27T02:32:01.417 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-27T02:32:01.418 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:32:01.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.551+0000 7f202d745700 1 Processor -- start 2024-01-27T02:32:01.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.551+0000 7f202d745700 1 -- start start 2024-01-27T02:32:01.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.552+0000 7f202d745700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f202815d460 0x7f20280d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:01.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.552+0000 7f202d745700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20280cf280 0x7f202815e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:01.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.552+0000 7f202d745700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20280b38d0 con 0x7f202815d830 2024-01-27T02:32:01.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.552+0000 7f202d745700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20280d0390 con 0x7f202815d460 2024-01-27T02:32:01.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.552+0000 7f202d745700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20280cf780 con 0x7f20280cf280 2024-01-27T02:32:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.552+0000 7f2026ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f202815d460 0x7f20280d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.552+0000 7f2026ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f202815d460 0x7f20280d0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40860/0 (socket says 172.21.15.7:40860) 2024-01-27T02:32:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.552+0000 7f20277fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f202815d830 0x7f20280cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35540/0 (socket says 172.21.15.7:35540) 2024-01-27T02:32:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.552+0000 7f2026ffd700 1 -- 172.21.15.7:0/2300805332 learned_addr learned my addr 172.21.15.7:0/2300805332 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.553+0000 7f20267fc700 1 --2- 172.21.15.7:0/2300805332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20280cf280 0x7f202815e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.553+0000 7f2026ffd700 1 -- 172.21.15.7:0/2300805332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20280cf280 msgr2=0x7f202815e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.553+0000 7f2026ffd700 1 --2- 172.21.15.7:0/2300805332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20280cf280 0x7f202815e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.553+0000 7f2026ffd700 1 -- 172.21.15.7:0/2300805332 >> v1:172.21.15.7:6789/0 conn(0x7f202815d830 legacy=0x7f20280cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.553+0000 7f2026ffd700 1 -- 172.21.15.7:0/2300805332 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20280cfa60 con 0x7f202815d460 2024-01-27T02:32:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.553+0000 7f2025ffb700 1 -- 172.21.15.7:0/2300805332 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2163852467 0 0) 0x7f2020001490 con 0x7f202815d830 2024-01-27T02:32:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.553+0000 7f2026ffd700 1 --2- 172.21.15.7:0/2300805332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f202815d460 0x7f20280d0820 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f20180099b0 tx=0x7f2018009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a677a82f5adfae server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.553+0000 7f2025ffb700 1 -- 172.21.15.7:0/2300805332 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201800b860 con 0x7f202815d460 2024-01-27T02:32:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.553+0000 7f2025ffb700 1 -- 172.21.15.7:0/2300805332 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201800b9c0 con 0x7f202815d460 2024-01-27T02:32:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.553+0000 7f2025ffb700 1 -- 172.21.15.7:0/2300805332 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201800bd60 con 0x7f202815d460 2024-01-27T02:32:01.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.554+0000 7f202d745700 1 -- 172.21.15.7:0/2300805332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f202815d460 msgr2=0x7f20280d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:01.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.554+0000 7f202d745700 1 --2- 172.21.15.7:0/2300805332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f202815d460 0x7f20280d0820 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f20180099b0 tx=0x7f2018009250 comp rx=0 tx=0).stop 2024-01-27T02:32:01.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.554+0000 7f202d745700 1 -- 172.21.15.7:0/2300805332 shutdown_connections 2024-01-27T02:32:01.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.554+0000 7f202d745700 1 --2- 172.21.15.7:0/2300805332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f202815d460 0x7f20280d0820 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.554+0000 7f202d745700 1 --2- 172.21.15.7:0/2300805332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20280cf280 0x7f202815e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.554+0000 7f202d745700 1 -- 172.21.15.7:0/2300805332 >> 172.21.15.7:0/2300805332 conn(0x7f2028063580 msgr2=0x7f2028063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:01.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.554+0000 7f202d745700 1 -- 172.21.15.7:0/2300805332 shutdown_connections 2024-01-27T02:32:01.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.554+0000 7f202d745700 1 -- 172.21.15.7:0/2300805332 wait complete. 2024-01-27T02:32:01.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.555+0000 7f202d745700 1 Processor -- start 2024-01-27T02:32:01.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.555+0000 7f202d745700 1 -- start start 2024-01-27T02:32:01.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.556+0000 7f202d745700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20280cf280 0x7f20280ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:01.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.556+0000 7f202d745700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f202815d460 0x7f20280b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.556+0000 7f202d745700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f202815d830 0x7f20281591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.556+0000 7f202d745700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20280c8170 con 0x7f202815d830 2024-01-27T02:32:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.556+0000 7f202d745700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20280c8450 con 0x7f202815d460 2024-01-27T02:32:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.556+0000 7f202d745700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20280c8730 con 0x7f20280cf280 2024-01-27T02:32:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.556+0000 7f20277fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f202815d830 0x7f20281591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.556+0000 7f20277fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f202815d830 0x7f20281591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40870/0 (socket says 172.21.15.7:40870) 2024-01-27T02:32:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.556+0000 7f20277fe700 1 -- 172.21.15.7:0/3613570208 learned_addr learned my addr 172.21.15.7:0/3613570208 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.556+0000 7f2026ffd700 1 --2- 172.21.15.7:0/3613570208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20280cf280 0x7f20280ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.556+0000 7f20267fc700 1 --2- 172.21.15.7:0/3613570208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f202815d460 0x7f20280b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.556+0000 7f20277fe700 1 -- 172.21.15.7:0/3613570208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20280cf280 msgr2=0x7f20280ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.556+0000 7f20277fe700 1 --2- 172.21.15.7:0/3613570208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20280cf280 0x7f20280ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.556+0000 7f20277fe700 1 -- 172.21.15.7:0/3613570208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f202815d460 msgr2=0x7f20280b6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.556+0000 7f20277fe700 1 --2- 172.21.15.7:0/3613570208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f202815d460 0x7f20280b6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.556+0000 7f20277fe700 1 -- 172.21.15.7:0/3613570208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20180094f0 con 0x7f202815d830 2024-01-27T02:32:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.556+0000 7f2026ffd700 1 --2- 172.21.15.7:0/3613570208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20280cf280 0x7f20280ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.556+0000 7f20267fc700 1 --2- 172.21.15.7:0/3613570208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f202815d460 0x7f20280b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.557+0000 7f20277fe700 1 --2- 172.21.15.7:0/3613570208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f202815d830 0x7f20281591d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f202000ccc0 tx=0x7f2020001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.557+0000 7f200effd700 1 -- 172.21.15.7:0/3613570208 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202000ce00 con 0x7f202815d830 2024-01-27T02:32:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.557+0000 7f200effd700 1 -- 172.21.15.7:0/3613570208 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2020005df0 con 0x7f202815d830 2024-01-27T02:32:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.557+0000 7f202d745700 1 -- 172.21.15.7:0/3613570208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20280c8a10 con 0x7f202815d830 2024-01-27T02:32:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.557+0000 7f200effd700 1 -- 172.21.15.7:0/3613570208 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202000abb0 con 0x7f202815d830 2024-01-27T02:32:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.557+0000 7f202d745700 1 -- 172.21.15.7:0/3613570208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202816b3f0 con 0x7f202815d830 2024-01-27T02:32:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.558+0000 7f202d745700 1 -- 172.21.15.7:0/3613570208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2028072060 con 0x7f202815d830 2024-01-27T02:32:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.558+0000 7f200effd700 1 -- 172.21.15.7:0/3613570208 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 84686+0+0 (secure 0 0 0) 0x7f20200070f0 con 0x7f202815d830 2024-01-27T02:32:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.559+0000 7f200effd700 1 --2- 172.21.15.7:0/3613570208 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2014066ea0 0x7f2014069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.559+0000 7f200effd700 1 -- 172.21.15.7:0/3613570208 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f20200840a0 con 0x7f202815d830 2024-01-27T02:32:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.559+0000 7f2026ffd700 1 --2- 172.21.15.7:0/3613570208 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2014066ea0 0x7f2014069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.559+0000 7f2026ffd700 1 --2- 172.21.15.7:0/3613570208 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2014066ea0 0x7f2014069350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f20180099b0 tx=0x7f2018017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:01.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.563+0000 7f200effd700 1 -- 172.21.15.7:0/3613570208 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2020051880 con 0x7f202815d830 2024-01-27T02:32:01.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.766+0000 7f202d745700 1 -- 172.21.15.7:0/3613570208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f20280b38d0 con 0x7f202815d830 2024-01-27T02:32:01.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.767+0000 7f200effd700 1 -- 172.21.15.7:0/3613570208 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f20280b38d0 con 0x7f202815d830 2024-01-27T02:32:01.769 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:32:01.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.769+0000 7f202d745700 1 -- 172.21.15.7:0/3613570208 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2014066ea0 msgr2=0x7f2014069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:01.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.769+0000 7f202d745700 1 --2- 172.21.15.7:0/3613570208 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2014066ea0 0x7f2014069350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f20180099b0 tx=0x7f2018017000 comp rx=0 tx=0).stop 2024-01-27T02:32:01.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.769+0000 7f202d745700 1 -- 172.21.15.7:0/3613570208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f202815d830 msgr2=0x7f20281591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:01.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.769+0000 7f202d745700 1 --2- 172.21.15.7:0/3613570208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f202815d830 0x7f20281591d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f202000ccc0 tx=0x7f2020001a70 comp rx=0 tx=0).stop 2024-01-27T02:32:01.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.769+0000 7f202d745700 1 -- 172.21.15.7:0/3613570208 shutdown_connections 2024-01-27T02:32:01.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.769+0000 7f202d745700 1 --2- 172.21.15.7:0/3613570208 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2014066ea0 0x7f2014069350 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.769+0000 7f202d745700 1 --2- 172.21.15.7:0/3613570208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f202815d830 0x7f20281591d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.769+0000 7f202d745700 1 --2- 172.21.15.7:0/3613570208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20280cf280 0x7f20280ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.769+0000 7f202d745700 1 --2- 172.21.15.7:0/3613570208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f202815d460 0x7f20280b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.769+0000 7f202d745700 1 -- 172.21.15.7:0/3613570208 >> 172.21.15.7:0/3613570208 conn(0x7f2028063580 msgr2=0x7f20280cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:01.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.769+0000 7f202d745700 1 -- 172.21.15.7:0/3613570208 shutdown_connections 2024-01-27T02:32:01.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.769+0000 7f202d745700 1 -- 172.21.15.7:0/3613570208 wait complete. 2024-01-27T02:32:01.784 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:32:01.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.913+0000 7f59b67e0700 1 Processor -- start 2024-01-27T02:32:01.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.913+0000 7f59b67e0700 1 -- start start 2024-01-27T02:32:01.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.914+0000 7f59b67e0700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f59b00c4cc0 0x7f59b00cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:01.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.914+0000 7f59b67e0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f59b00ca540 0x7f59b0163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:01.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.914+0000 7f59b67e0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b00ce5e0 con 0x7f59b015a100 2024-01-27T02:32:01.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.914+0000 7f59b67e0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b00cd9d0 con 0x7f59b00ca540 2024-01-27T02:32:01.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.914+0000 7f59b67e0700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b00cdcb0 con 0x7f59b00c4cc0 2024-01-27T02:32:01.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.914+0000 7f59b4d7d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f59b00ca540 0x7f59b0163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:01.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.914+0000 7f59affff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f59b015a100 0x7f59b00c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35554/0 (socket says 172.21.15.7:35554) 2024-01-27T02:32:01.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.914+0000 7f59affff700 1 -- 172.21.15.7:0/64273693 learned_addr learned my addr 172.21.15.7:0/64273693 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:01.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.914+0000 7f59af7fe700 1 --2- 172.21.15.7:0/64273693 >> v2:172.21.15.188:3300/0 conn(0x7f59b00c4cc0 0x7f59b00cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:01.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.915+0000 7f59b4d7d700 1 -- 172.21.15.7:0/64273693 >> v2:172.21.15.188:3300/0 conn(0x7f59b00c4cc0 msgr2=0x7f59b00cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:01.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.915+0000 7f59b4d7d700 1 --2- 172.21.15.7:0/64273693 >> v2:172.21.15.188:3300/0 conn(0x7f59b00c4cc0 0x7f59b00cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.915+0000 7f59b4d7d700 1 -- 172.21.15.7:0/64273693 >> v1:172.21.15.7:6789/0 conn(0x7f59b015a100 legacy=0x7f59b00c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:01.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.915+0000 7f59b4d7d700 1 -- 172.21.15.7:0/64273693 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b00cdf90 con 0x7f59b00ca540 2024-01-27T02:32:01.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.915+0000 7f59b4d7d700 1 --2- 172.21.15.7:0/64273693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b00ca540 0x7f59b0163d90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f59b006e170 tx=0x7f59a8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110f01ed1c5328a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:01.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.915+0000 7f59aeffd700 1 -- 172.21.15.7:0/64273693 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a800e070 con 0x7f59b00ca540 2024-01-27T02:32:01.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.915+0000 7f59aeffd700 1 -- 172.21.15.7:0/64273693 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a800bc30 con 0x7f59b00ca540 2024-01-27T02:32:01.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.915+0000 7f59aeffd700 1 -- 172.21.15.7:0/64273693 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a8012430 con 0x7f59b00ca540 2024-01-27T02:32:01.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.916+0000 7f59b67e0700 1 -- 172.21.15.7:0/64273693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b00ca540 msgr2=0x7f59b0163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:01.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.916+0000 7f59b67e0700 1 --2- 172.21.15.7:0/64273693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b00ca540 0x7f59b0163d90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f59b006e170 tx=0x7f59a8009780 comp rx=0 tx=0).stop 2024-01-27T02:32:01.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.916+0000 7f59b67e0700 1 -- 172.21.15.7:0/64273693 shutdown_connections 2024-01-27T02:32:01.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.916+0000 7f59b67e0700 1 --2- 172.21.15.7:0/64273693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b00ca540 0x7f59b0163d90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.916+0000 7f59b67e0700 1 --2- 172.21.15.7:0/64273693 >> v2:172.21.15.188:3300/0 conn(0x7f59b00c4cc0 0x7f59b00cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.916+0000 7f59b67e0700 1 -- 172.21.15.7:0/64273693 >> 172.21.15.7:0/64273693 conn(0x7f59b0098020 msgr2=0x7f59b00711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:01.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.916+0000 7f59b67e0700 1 -- 172.21.15.7:0/64273693 shutdown_connections 2024-01-27T02:32:01.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.916+0000 7f59b67e0700 1 -- 172.21.15.7:0/64273693 wait complete. 2024-01-27T02:32:01.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.917+0000 7f59b67e0700 1 Processor -- start 2024-01-27T02:32:01.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.917+0000 7f59b67e0700 1 -- start start 2024-01-27T02:32:01.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.918+0000 7f59b67e0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b00c4cc0 0x7f59b00b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:01.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.918+0000 7f59b67e0700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59b00ca540 0x7f59b00b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:01.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.918+0000 7f59b67e0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59b015a100 0x7f59b0151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:01.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.918+0000 7f59b67e0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b00b2040 con 0x7f59b00c4cc0 2024-01-27T02:32:01.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.918+0000 7f59b67e0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b016a0a0 con 0x7f59b015a100 2024-01-27T02:32:01.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.918+0000 7f59b67e0700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b0152220 con 0x7f59b00ca540 2024-01-27T02:32:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.918+0000 7f59affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b00c4cc0 0x7f59b00b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.918+0000 7f59affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b00c4cc0 0x7f59b00b3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40876/0 (socket says 172.21.15.7:40876) 2024-01-27T02:32:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.918+0000 7f59affff700 1 -- 172.21.15.7:0/3714938498 learned_addr learned my addr 172.21.15.7:0/3714938498 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.918+0000 7f59af7fe700 1 --2- 172.21.15.7:0/3714938498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59b00ca540 0x7f59b00b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.918+0000 7f59b4d7d700 1 --2- 172.21.15.7:0/3714938498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59b015a100 0x7f59b0151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.918+0000 7f59affff700 1 -- 172.21.15.7:0/3714938498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59b00ca540 msgr2=0x7f59b00b1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.918+0000 7f59affff700 1 --2- 172.21.15.7:0/3714938498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59b00ca540 0x7f59b00b1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.918+0000 7f59affff700 1 -- 172.21.15.7:0/3714938498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59b015a100 msgr2=0x7f59b0151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.918+0000 7f59affff700 1 --2- 172.21.15.7:0/3714938498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59b015a100 0x7f59b0151ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.918+0000 7f59affff700 1 -- 172.21.15.7:0/3714938498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a8009a20 con 0x7f59b00c4cc0 2024-01-27T02:32:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.918+0000 7f59af7fe700 1 --2- 172.21.15.7:0/3714938498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59b00ca540 0x7f59b00b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:32:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.918+0000 7f59b4d7d700 1 --2- 172.21.15.7:0/3714938498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59b015a100 0x7f59b0151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:32:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.919+0000 7f59affff700 1 --2- 172.21.15.7:0/3714938498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b00c4cc0 0x7f59b00b3c60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f59a000c590 tx=0x7f59a0001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:01.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.919+0000 7f5997fff700 1 -- 172.21.15.7:0/3714938498 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a0005b30 con 0x7f59b00c4cc0 2024-01-27T02:32:01.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.919+0000 7f5997fff700 1 -- 172.21.15.7:0/3714938498 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a0005cb0 con 0x7f59b00c4cc0 2024-01-27T02:32:01.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.919+0000 7f5997fff700 1 -- 172.21.15.7:0/3714938498 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a0007320 con 0x7f59b00c4cc0 2024-01-27T02:32:01.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.919+0000 7f59b67e0700 1 -- 172.21.15.7:0/3714938498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b00b2270 con 0x7f59b00c4cc0 2024-01-27T02:32:01.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.919+0000 7f59b67e0700 1 -- 172.21.15.7:0/3714938498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b00b2830 con 0x7f59b00c4cc0 2024-01-27T02:32:01.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.920+0000 7f59b67e0700 1 -- 172.21.15.7:0/3714938498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b004c710 con 0x7f59b00c4cc0 2024-01-27T02:32:01.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.920+0000 7f5997fff700 1 -- 172.21.15.7:0/3714938498 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 84686+0+0 (secure 0 0 0) 0x7f59a001b3b0 con 0x7f59b00c4cc0 2024-01-27T02:32:01.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.921+0000 7f5997fff700 1 --2- 172.21.15.7:0/3714938498 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f599c066ea0 0x7f599c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:01.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.921+0000 7f5997fff700 1 -- 172.21.15.7:0/3714938498 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f59a0083600 con 0x7f59b00c4cc0 2024-01-27T02:32:01.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.921+0000 7f59af7fe700 1 --2- 172.21.15.7:0/3714938498 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f599c066ea0 0x7f599c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:01.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.922+0000 7f59af7fe700 1 --2- 172.21.15.7:0/3714938498 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f599c066ea0 0x7f599c069350 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f59b015bc80 tx=0x7f59a4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:01.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:01.925+0000 7f5997fff700 1 -- 172.21.15.7:0/3714938498 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59a0050ca0 con 0x7f59b00c4cc0 2024-01-27T02:32:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.178+0000 7f59b67e0700 1 -- 172.21.15.7:0/3714938498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59b00ce270 con 0x7f59b00c4cc0 2024-01-27T02:32:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.178+0000 7f5997fff700 1 -- 172.21.15.7:0/3714938498 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3524 (secure 0 0 0) 0x7f59a0017de0 con 0x7f59b00c4cc0 2024-01-27T02:32:02.180 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:02.180 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3594367777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3594367777},{"type":"v1","addr":"172.21.15.7:6834","nonce":3594367777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1079101213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1079101213},{"type":"v1","addr":"172.21.15.188:6837","nonce":1079101213}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3241424244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3241424244},{"type":"v1","addr":"172.21.15.7:6835","nonce":3241424244}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2571237935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2571237935},{"type":"v1","addr":"172.21.15.188:6836","nonce":2571237935}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-01-27T02:32:02.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.180+0000 7f59b67e0700 1 -- 172.21.15.7:0/3714938498 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f599c066ea0 msgr2=0x7f599c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:02.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.180+0000 7f59b67e0700 1 --2- 172.21.15.7:0/3714938498 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f599c066ea0 0x7f599c069350 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f59b015bc80 tx=0x7f59a4008040 comp rx=0 tx=0).stop 2024-01-27T02:32:02.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.180+0000 7f59b67e0700 1 -- 172.21.15.7:0/3714938498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b00c4cc0 msgr2=0x7f59b00b3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:02.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.180+0000 7f59b67e0700 1 --2- 172.21.15.7:0/3714938498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b00c4cc0 0x7f59b00b3c60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f59a000c590 tx=0x7f59a0001020 comp rx=0 tx=0).stop 2024-01-27T02:32:02.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.180+0000 7f59b67e0700 1 -- 172.21.15.7:0/3714938498 shutdown_connections 2024-01-27T02:32:02.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.180+0000 7f59b67e0700 1 --2- 172.21.15.7:0/3714938498 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f599c066ea0 0x7f599c069350 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.180+0000 7f59b67e0700 1 --2- 172.21.15.7:0/3714938498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59b015a100 0x7f59b0151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.180+0000 7f59b67e0700 1 --2- 172.21.15.7:0/3714938498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b00c4cc0 0x7f59b00b3c60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.180+0000 7f59b67e0700 1 --2- 172.21.15.7:0/3714938498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59b00ca540 0x7f59b00b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.180+0000 7f59b67e0700 1 -- 172.21.15.7:0/3714938498 >> 172.21.15.7:0/3714938498 conn(0x7f59b0098020 msgr2=0x7f59b0154c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.180+0000 7f59b67e0700 1 -- 172.21.15.7:0/3714938498 shutdown_connections 2024-01-27T02:32:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.180+0000 7f59b67e0700 1 -- 172.21.15.7:0/3714938498 wait complete. 2024-01-27T02:32:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 3 2024-01-27T02:32:02.195 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:32:02.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.328+0000 7f9007a65700 1 Processor -- start 2024-01-27T02:32:02.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.328+0000 7f9007a65700 1 -- start start 2024-01-27T02:32:02.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.329+0000 7f9007a65700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f90000c9bb0 0x7f9000157f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:02.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.329+0000 7f9007a65700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f90000c9f80 0x7f900006b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:02.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.329+0000 7f9007a65700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f900006e330 0x7f900006e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:02.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.329+0000 7f9007a65700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f900006cd10 con 0x7f90000c9bb0 2024-01-27T02:32:02.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.329+0000 7f9007a65700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f900006ecd0 con 0x7f90000c9f80 2024-01-27T02:32:02.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.329+0000 7f9007a65700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900006c880 con 0x7f900006e330 2024-01-27T02:32:02.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.329+0000 7f9005801700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f90000c9bb0 0x7f9000157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:02.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.329+0000 7f9005801700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f90000c9bb0 0x7f9000157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40890/0 (socket says 172.21.15.7:40890) 2024-01-27T02:32:02.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.329+0000 7f9005801700 1 -- 172.21.15.7:0/1093387902 learned_addr learned my addr 172.21.15.7:0/1093387902 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:02.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.329+0000 7f9005000700 1 --2- 172.21.15.7:0/1093387902 >> v2:172.21.15.188:3300/0 conn(0x7f90000c9f80 0x7f900006b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.329+0000 7f9006002700 1 --2- 172.21.15.7:0/1093387902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f900006e330 0x7f900006e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.329+0000 7f9005801700 1 -- 172.21.15.7:0/1093387902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f900006e330 msgr2=0x7f900006e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.329+0000 7f9005801700 1 --2- 172.21.15.7:0/1093387902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f900006e330 0x7f900006e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.329+0000 7f9005801700 1 -- 172.21.15.7:0/1093387902 >> v2:172.21.15.188:3300/0 conn(0x7f90000c9f80 msgr2=0x7f900006b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.329+0000 7f9005801700 1 --2- 172.21.15.7:0/1093387902 >> v2:172.21.15.188:3300/0 conn(0x7f90000c9f80 0x7f900006b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.329+0000 7f9005801700 1 -- 172.21.15.7:0/1093387902 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900006bc70 con 0x7f90000c9bb0 2024-01-27T02:32:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.330+0000 7f9005801700 1 --2- 172.21.15.7:0/1093387902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90000c9bb0 0x7f9000157f70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f90000cdb90 tx=0x7f8ffc00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7691d51766346129 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.330+0000 7f8ff7fff700 1 -- 172.21.15.7:0/1093387902 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc00db40 con 0x7f90000c9bb0 2024-01-27T02:32:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.330+0000 7f8ff7fff700 1 -- 172.21.15.7:0/1093387902 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ffc00dca0 con 0x7f90000c9bb0 2024-01-27T02:32:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.330+0000 7f8ff7fff700 1 -- 172.21.15.7:0/1093387902 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc0196b0 con 0x7f90000c9bb0 2024-01-27T02:32:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.330+0000 7f9007a65700 1 -- 172.21.15.7:0/1093387902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90000c9bb0 msgr2=0x7f9000157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.330+0000 7f9007a65700 1 --2- 172.21.15.7:0/1093387902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90000c9bb0 0x7f9000157f70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f90000cdb90 tx=0x7f8ffc00b4f0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.330+0000 7f9007a65700 1 -- 172.21.15.7:0/1093387902 shutdown_connections 2024-01-27T02:32:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.330+0000 7f9007a65700 1 --2- 172.21.15.7:0/1093387902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f900006e330 0x7f900006e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.330+0000 7f9007a65700 1 --2- 172.21.15.7:0/1093387902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90000c9bb0 0x7f9000157f70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.330+0000 7f9007a65700 1 --2- 172.21.15.7:0/1093387902 >> v2:172.21.15.188:3300/0 conn(0x7f90000c9f80 0x7f900006b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.330+0000 7f9007a65700 1 -- 172.21.15.7:0/1093387902 >> 172.21.15.7:0/1093387902 conn(0x7f90000cdc30 msgr2=0x7f90000ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:02.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.331+0000 7f9007a65700 1 -- 172.21.15.7:0/1093387902 shutdown_connections 2024-01-27T02:32:02.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.331+0000 7f9007a65700 1 -- 172.21.15.7:0/1093387902 wait complete. 2024-01-27T02:32:02.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.332+0000 7f9007a65700 1 Processor -- start 2024-01-27T02:32:02.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.332+0000 7f9007a65700 1 -- start start 2024-01-27T02:32:02.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.333+0000 7f9007a65700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f900006e330 0x7f90001530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:02.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.333+0000 7f9007a65700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90000c9bb0 0x7f90000afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:02.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.333+0000 7f9007a65700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90000c9f80 0x7f9000155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:02.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.333+0000 7f9007a65700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90000b0b50 con 0x7f900006e330 2024-01-27T02:32:02.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.333+0000 7f9007a65700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90000b0e30 con 0x7f90000c9bb0 2024-01-27T02:32:02.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.333+0000 7f9007a65700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90000b1110 con 0x7f90000c9f80 2024-01-27T02:32:02.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.333+0000 7f9005801700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f900006e330 0x7f90001530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:02.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.333+0000 7f9005801700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f900006e330 0x7f90001530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40906/0 (socket says 172.21.15.7:40906) 2024-01-27T02:32:02.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.333+0000 7f9005801700 1 -- 172.21.15.7:0/3804963485 learned_addr learned my addr 172.21.15.7:0/3804963485 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:02.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.333+0000 7f9006002700 1 --2- 172.21.15.7:0/3804963485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90000c9f80 0x7f9000155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:02.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.333+0000 7f9005000700 1 --2- 172.21.15.7:0/3804963485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90000c9bb0 0x7f90000afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:02.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.333+0000 7f9005801700 1 -- 172.21.15.7:0/3804963485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90000c9f80 msgr2=0x7f9000155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:02.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.333+0000 7f9005801700 1 --2- 172.21.15.7:0/3804963485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90000c9f80 0x7f9000155b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.333+0000 7f9005801700 1 -- 172.21.15.7:0/3804963485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90000c9bb0 msgr2=0x7f90000afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:02.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.333+0000 7f9005801700 1 --2- 172.21.15.7:0/3804963485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90000c9bb0 0x7f90000afe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.333+0000 7f9005801700 1 -- 172.21.15.7:0/3804963485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc00b790 con 0x7f900006e330 2024-01-27T02:32:02.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.333+0000 7f9005801700 1 --2- 172.21.15.7:0/3804963485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f900006e330 0x7f90001530c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00b530 tx=0x7f8ffc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:02.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.333+0000 7f8ff57fa700 1 -- 172.21.15.7:0/3804963485 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc010680 con 0x7f900006e330 2024-01-27T02:32:02.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.334+0000 7f8ff57fa700 1 -- 172.21.15.7:0/3804963485 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ffc0107e0 con 0x7f900006e330 2024-01-27T02:32:02.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.334+0000 7f9007a65700 1 -- 172.21.15.7:0/3804963485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90000b13f0 con 0x7f900006e330 2024-01-27T02:32:02.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.334+0000 7f8ff57fa700 1 -- 172.21.15.7:0/3804963485 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc010ba0 con 0x7f900006e330 2024-01-27T02:32:02.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.334+0000 7f9007a65700 1 -- 172.21.15.7:0/3804963485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9000153600 con 0x7f900006e330 2024-01-27T02:32:02.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.335+0000 7f9007a65700 1 -- 172.21.15.7:0/3804963485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90000b1910 con 0x7f900006e330 2024-01-27T02:32:02.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.335+0000 7f8ff57fa700 1 -- 172.21.15.7:0/3804963485 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 84686+0+0 (secure 0 0 0) 0x7f8ffc02c440 con 0x7f900006e330 2024-01-27T02:32:02.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.336+0000 7f8ff57fa700 1 --2- 172.21.15.7:0/3804963485 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8fe4066ea0 0x7f8fe4069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:02.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.336+0000 7f8ff57fa700 1 -- 172.21.15.7:0/3804963485 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8ffc014070 con 0x7f900006e330 2024-01-27T02:32:02.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.336+0000 7f9005000700 1 --2- 172.21.15.7:0/3804963485 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8fe4066ea0 0x7f8fe4069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:02.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.336+0000 7f9005000700 1 --2- 172.21.15.7:0/3804963485 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8fe4066ea0 0x7f8fe4069350 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00093f0 tx=0x7f8ff0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:02.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.340+0000 7f8ff57fa700 1 -- 172.21.15.7:0/3804963485 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ffc0533a0 con 0x7f900006e330 2024-01-27T02:32:02.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.543+0000 7f9007a65700 1 -- 172.21.15.7:0/3804963485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9000139f50 con 0x7f900006e330 2024-01-27T02:32:02.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.544+0000 7f8ff57fa700 1 -- 172.21.15.7:0/3804963485 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12079 (secure 0 0 0) 0x7f8ffc0569c0 con 0x7f900006e330 2024-01-27T02:32:02.546 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:02.546 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:32:00.579459+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:32:02.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.546+0000 7f9007a65700 1 -- 172.21.15.7:0/3804963485 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8fe4066ea0 msgr2=0x7f8fe4069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:02.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.546+0000 7f9007a65700 1 --2- 172.21.15.7:0/3804963485 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8fe4066ea0 0x7f8fe4069350 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00093f0 tx=0x7f8ff0008040 comp rx=0 tx=0).stop 2024-01-27T02:32:02.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.546+0000 7f9007a65700 1 -- 172.21.15.7:0/3804963485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f900006e330 msgr2=0x7f90001530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:02.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.546+0000 7f9007a65700 1 --2- 172.21.15.7:0/3804963485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f900006e330 0x7f90001530c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00b530 tx=0x7f8ffc01b000 comp rx=0 tx=0).stop 2024-01-27T02:32:02.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.546+0000 7f9007a65700 1 -- 172.21.15.7:0/3804963485 shutdown_connections 2024-01-27T02:32:02.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.546+0000 7f9007a65700 1 --2- 172.21.15.7:0/3804963485 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8fe4066ea0 0x7f8fe4069350 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.546+0000 7f9007a65700 1 --2- 172.21.15.7:0/3804963485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90000c9f80 0x7f9000155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.546+0000 7f9007a65700 1 --2- 172.21.15.7:0/3804963485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f900006e330 0x7f90001530c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.546+0000 7f9007a65700 1 --2- 172.21.15.7:0/3804963485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90000c9bb0 0x7f90000afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.546+0000 7f9007a65700 1 -- 172.21.15.7:0/3804963485 >> 172.21.15.7:0/3804963485 conn(0x7f90000cdc30 msgr2=0x7f9000153ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:02.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.546+0000 7f9007a65700 1 -- 172.21.15.7:0/3804963485 shutdown_connections 2024-01-27T02:32:02.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.546+0000 7f9007a65700 1 -- 172.21.15.7:0/3804963485 wait complete. 2024-01-27T02:32:02.561 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-27T02:32:02.561 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:32:02.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.696+0000 7f85a8738700 1 Processor -- start 2024-01-27T02:32:02.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.696+0000 7f85a8738700 1 -- start start 2024-01-27T02:32:02.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.697+0000 7f85a8738700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85a0143670 0x7f85a0063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:02.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.697+0000 7f85a8738700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f85a0143a40 0x7f85a0063ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:02.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.697+0000 7f85a8738700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a00cc290 con 0x7f85a00aa8b0 2024-01-27T02:32:02.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.697+0000 7f85a8738700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a00cb960 con 0x7f85a0143a40 2024-01-27T02:32:02.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.697+0000 7f85a8738700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a00cbc40 con 0x7f85a0143670 2024-01-27T02:32:02.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.697+0000 7f85a64d4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85a0143670 0x7f85a0063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:02.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.697+0000 7f85a6cd5700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f85a0143a40 0x7f85a0063ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:02.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.697+0000 7f85a5cd3700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f85a00aa8b0 0x7f85a00c8230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:33992/0 (socket says 172.21.15.7:33992) 2024-01-27T02:32:02.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.697+0000 7f85a5cd3700 1 -- 172.21.15.7:0/800943970 learned_addr learned my addr 172.21.15.7:0/800943970 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:02.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.697+0000 7f85a54d2700 1 -- 172.21.15.7:0/800943970 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1147791331 0 0) 0x7f85a00cc290 con 0x7f85a00aa8b0 2024-01-27T02:32:02.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.698+0000 7f85a54d2700 1 -- 172.21.15.7:0/800943970 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f858c003570 con 0x7f85a00aa8b0 2024-01-27T02:32:02.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.698+0000 7f85a6cd5700 1 -- 172.21.15.7:0/800943970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85a0143670 msgr2=0x7f85a0063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:02.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.698+0000 7f85a6cd5700 1 --2- 172.21.15.7:0/800943970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85a0143670 0x7f85a0063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.698+0000 7f85a6cd5700 1 -- 172.21.15.7:0/800943970 >> v1:172.21.15.188:6789/0 conn(0x7f85a00aa8b0 legacy=0x7f85a00c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:02.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.698+0000 7f85a6cd5700 1 -- 172.21.15.7:0/800943970 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a00cbf20 con 0x7f85a0143a40 2024-01-27T02:32:02.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.698+0000 7f85a64d4700 1 --2- 172.21.15.7:0/800943970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85a0143670 0x7f85a0063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:32:02.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.698+0000 7f85a6cd5700 1 --2- 172.21.15.7:0/800943970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85a0143a40 0x7f85a0063ac0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8598009e60 tx=0x7f8598009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f7a06afcab2c349 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:02.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.698+0000 7f85a54d2700 1 -- 172.21.15.7:0/800943970 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859800e070 con 0x7f85a0143a40 2024-01-27T02:32:02.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.698+0000 7f85a54d2700 1 -- 172.21.15.7:0/800943970 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859800bab0 con 0x7f85a0143a40 2024-01-27T02:32:02.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.698+0000 7f85a54d2700 1 -- 172.21.15.7:0/800943970 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859800be70 con 0x7f85a0143a40 2024-01-27T02:32:02.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.699+0000 7f85a8738700 1 -- 172.21.15.7:0/800943970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85a0143a40 msgr2=0x7f85a0063ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:02.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.699+0000 7f85a8738700 1 --2- 172.21.15.7:0/800943970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85a0143a40 0x7f85a0063ac0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8598009e60 tx=0x7f8598009780 comp rx=0 tx=0).stop 2024-01-27T02:32:02.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.699+0000 7f85a8738700 1 -- 172.21.15.7:0/800943970 shutdown_connections 2024-01-27T02:32:02.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.699+0000 7f85a8738700 1 --2- 172.21.15.7:0/800943970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85a0143a40 0x7f85a0063ac0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.699+0000 7f85a8738700 1 --2- 172.21.15.7:0/800943970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85a0143670 0x7f85a0063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.699+0000 7f85a8738700 1 -- 172.21.15.7:0/800943970 >> 172.21.15.7:0/800943970 conn(0x7f85a00cdbf0 msgr2=0x7f85a00cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:02.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.699+0000 7f85a8738700 1 -- 172.21.15.7:0/800943970 shutdown_connections 2024-01-27T02:32:02.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.699+0000 7f85a8738700 1 -- 172.21.15.7:0/800943970 wait complete. 2024-01-27T02:32:02.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.700+0000 7f85a8738700 1 Processor -- start 2024-01-27T02:32:02.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.700+0000 7f85a8738700 1 -- start start 2024-01-27T02:32:02.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.701+0000 7f85a8738700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a00aa8b0 0x7f85a00c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:02.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.701+0000 7f85a8738700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85a0143670 0x7f85a0141a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:02.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.701+0000 7f85a8738700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85a0143a40 0x7f85a00c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:02.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.701+0000 7f85a8738700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a00c55d0 con 0x7f85a00aa8b0 2024-01-27T02:32:02.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.701+0000 7f85a8738700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a00c73d0 con 0x7f85a0143670 2024-01-27T02:32:02.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.701+0000 7f85a8738700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a00c4ca0 con 0x7f85a0143a40 2024-01-27T02:32:02.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.701+0000 7f85a64d4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a00aa8b0 0x7f85a00c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:02.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.701+0000 7f85a64d4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a00aa8b0 0x7f85a00c5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40910/0 (socket says 172.21.15.7:40910) 2024-01-27T02:32:02.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.701+0000 7f85a64d4700 1 -- 172.21.15.7:0/2529605167 learned_addr learned my addr 172.21.15.7:0/2529605167 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:02.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.701+0000 7f85a6cd5700 1 --2- 172.21.15.7:0/2529605167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85a0143a40 0x7f85a00c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:02.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.701+0000 7f85a5cd3700 1 --2- 172.21.15.7:0/2529605167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85a0143670 0x7f85a0141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:02.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.701+0000 7f85a64d4700 1 -- 172.21.15.7:0/2529605167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85a0143a40 msgr2=0x7f85a00c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:02.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.701+0000 7f85a64d4700 1 --2- 172.21.15.7:0/2529605167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85a0143a40 0x7f85a00c6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.701+0000 7f85a64d4700 1 -- 172.21.15.7:0/2529605167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85a0143670 msgr2=0x7f85a0141a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:02.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.701+0000 7f85a64d4700 1 --2- 172.21.15.7:0/2529605167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85a0143670 0x7f85a0141a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.701+0000 7f85a64d4700 1 -- 172.21.15.7:0/2529605167 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859c009420 con 0x7f85a00aa8b0 2024-01-27T02:32:02.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.702+0000 7f85a64d4700 1 --2- 172.21.15.7:0/2529605167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a00aa8b0 0x7f85a00c5a60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f859c00ec80 tx=0x7f859c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:02.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.702+0000 7f85967fc700 1 -- 172.21.15.7:0/2529605167 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c00c9d0 con 0x7f85a00aa8b0 2024-01-27T02:32:02.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.702+0000 7f85967fc700 1 -- 172.21.15.7:0/2529605167 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859c00cb30 con 0x7f85a00aa8b0 2024-01-27T02:32:02.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.702+0000 7f85a8738700 1 -- 172.21.15.7:0/2529605167 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8598009a20 con 0x7f85a00aa8b0 2024-01-27T02:32:02.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.702+0000 7f85967fc700 1 -- 172.21.15.7:0/2529605167 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c00ceb0 con 0x7f85a00aa8b0 2024-01-27T02:32:02.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.702+0000 7f85a8738700 1 -- 172.21.15.7:0/2529605167 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a00c5260 con 0x7f85a00aa8b0 2024-01-27T02:32:02.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.703+0000 7f85a8738700 1 -- 172.21.15.7:0/2529605167 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a00c5fa0 con 0x7f85a00aa8b0 2024-01-27T02:32:02.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.703+0000 7f85967fc700 1 -- 172.21.15.7:0/2529605167 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f859c013680 con 0x7f85a00aa8b0 2024-01-27T02:32:02.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.704+0000 7f85967fc700 1 --2- 172.21.15.7:0/2529605167 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8584066f30 0x7f85840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:02.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.704+0000 7f85967fc700 1 -- 172.21.15.7:0/2529605167 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f859c084630 con 0x7f85a00aa8b0 2024-01-27T02:32:02.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.704+0000 7f85a5cd3700 1 --2- 172.21.15.7:0/2529605167 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8584066f30 0x7f85840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:02.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.704+0000 7f85a5cd3700 1 --2- 172.21.15.7:0/2529605167 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8584066f30 0x7f85840693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f85a00acdf0 tx=0x7f8590009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:02.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.708+0000 7f85967fc700 1 -- 172.21.15.7:0/2529605167 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f859c051bf0 con 0x7f85a00aa8b0 2024-01-27T02:32:02.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.912+0000 7f85a8738700 1 -- 172.21.15.7:0/2529605167 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f85a00c58b0 con 0x7f85a00aa8b0 2024-01-27T02:32:02.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.912+0000 7f85967fc700 1 -- 172.21.15.7:0/2529605167 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f859c055210 con 0x7f85a00aa8b0 2024-01-27T02:32:02.914 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:32:02.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.914+0000 7f85a8738700 1 -- 172.21.15.7:0/2529605167 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8584066f30 msgr2=0x7f85840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:02.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.914+0000 7f85a8738700 1 --2- 172.21.15.7:0/2529605167 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8584066f30 0x7f85840693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f85a00acdf0 tx=0x7f8590009040 comp rx=0 tx=0).stop 2024-01-27T02:32:02.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.914+0000 7f85a8738700 1 -- 172.21.15.7:0/2529605167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a00aa8b0 msgr2=0x7f85a00c5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:02.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.914+0000 7f85a8738700 1 --2- 172.21.15.7:0/2529605167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a00aa8b0 0x7f85a00c5a60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f859c00ec80 tx=0x7f859c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.914+0000 7f85a8738700 1 -- 172.21.15.7:0/2529605167 shutdown_connections 2024-01-27T02:32:02.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.914+0000 7f85a8738700 1 --2- 172.21.15.7:0/2529605167 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8584066f30 0x7f85840693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.914+0000 7f85a8738700 1 --2- 172.21.15.7:0/2529605167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85a0143a40 0x7f85a00c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.914+0000 7f85a8738700 1 --2- 172.21.15.7:0/2529605167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a00aa8b0 0x7f85a00c5a60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.914+0000 7f85a8738700 1 --2- 172.21.15.7:0/2529605167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85a0143670 0x7f85a0141a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:02.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.914+0000 7f85a8738700 1 -- 172.21.15.7:0/2529605167 >> 172.21.15.7:0/2529605167 conn(0x7f85a00cdbf0 msgr2=0x7f85a0142360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:02.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.914+0000 7f85a8738700 1 -- 172.21.15.7:0/2529605167 shutdown_connections 2024-01-27T02:32:02.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:02.915+0000 7f85a8738700 1 -- 172.21.15.7:0/2529605167 wait complete. 2024-01-27T02:32:02.930 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-27T02:32:03.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.059+0000 7f7013d2e700 1 Processor -- start 2024-01-27T02:32:03.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.060+0000 7f7013d2e700 1 -- start start 2024-01-27T02:32:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.061+0000 7f7013d2e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f700c15a100 0x7f700c0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.061+0000 7f7013d2e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f700c0cb5b0 0x7f700c0c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.061+0000 7f7013d2e700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700c0c4f50 con 0x7f700c0c4af0 2024-01-27T02:32:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.061+0000 7f7013d2e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f700c0ce5e0 con 0x7f700c15a100 2024-01-27T02:32:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.061+0000 7f7013d2e700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f700c0cd9d0 con 0x7f700c0cb5b0 2024-01-27T02:32:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.061+0000 7f7011aca700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f700c15a100 0x7f700c0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.061+0000 7f7011aca700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f700c15a100 0x7f700c0cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40922/0 (socket says 172.21.15.7:40922) 2024-01-27T02:32:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.061+0000 7f7011aca700 1 -- 172.21.15.7:0/3178398826 learned_addr learned my addr 172.21.15.7:0/3178398826 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.061+0000 7f70112c9700 1 --2- 172.21.15.7:0/3178398826 >> v2:172.21.15.188:3300/0 conn(0x7f700c0cb5b0 0x7f700c0c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:03.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.061+0000 7f7011aca700 1 -- 172.21.15.7:0/3178398826 >> v2:172.21.15.188:3300/0 conn(0x7f700c0cb5b0 msgr2=0x7f700c0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:03.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.061+0000 7f7011aca700 1 --2- 172.21.15.7:0/3178398826 >> v2:172.21.15.188:3300/0 conn(0x7f700c0cb5b0 0x7f700c0c45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:03.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.061+0000 7f7011aca700 1 -- 172.21.15.7:0/3178398826 >> v1:172.21.15.188:6789/0 conn(0x7f700c0c4af0 legacy=0x7f700c0c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:03.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.061+0000 7f7011aca700 1 -- 172.21.15.7:0/3178398826 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c0cdcb0 con 0x7f700c15a100 2024-01-27T02:32:03.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.061+0000 7f7011aca700 1 --2- 172.21.15.7:0/3178398826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f700c15a100 0x7f700c0cd2f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f700c06e170 tx=0x7f7008009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b298bf73ca2277eb server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:03.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.061+0000 7f7010ac8700 1 -- 172.21.15.7:0/3178398826 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700800b820 con 0x7f700c15a100 2024-01-27T02:32:03.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.061+0000 7f7010ac8700 1 -- 172.21.15.7:0/3178398826 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700800b980 con 0x7f700c15a100 2024-01-27T02:32:03.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.061+0000 7f7010ac8700 1 -- 172.21.15.7:0/3178398826 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700800bd20 con 0x7f700c15a100 2024-01-27T02:32:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.062+0000 7f7013d2e700 1 -- 172.21.15.7:0/3178398826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f700c15a100 msgr2=0x7f700c0cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.062+0000 7f7013d2e700 1 --2- 172.21.15.7:0/3178398826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f700c15a100 0x7f700c0cd2f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f700c06e170 tx=0x7f7008009250 comp rx=0 tx=0).stop 2024-01-27T02:32:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.062+0000 7f7013d2e700 1 -- 172.21.15.7:0/3178398826 shutdown_connections 2024-01-27T02:32:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.062+0000 7f7013d2e700 1 --2- 172.21.15.7:0/3178398826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f700c15a100 0x7f700c0cd2f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.062+0000 7f7013d2e700 1 --2- 172.21.15.7:0/3178398826 >> v2:172.21.15.188:3300/0 conn(0x7f700c0cb5b0 0x7f700c0c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.062+0000 7f7013d2e700 1 -- 172.21.15.7:0/3178398826 >> 172.21.15.7:0/3178398826 conn(0x7f700c098020 msgr2=0x7f700c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.062+0000 7f7013d2e700 1 -- 172.21.15.7:0/3178398826 shutdown_connections 2024-01-27T02:32:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.062+0000 7f7013d2e700 1 -- 172.21.15.7:0/3178398826 wait complete. 2024-01-27T02:32:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.063+0000 7f7013d2e700 1 Processor -- start 2024-01-27T02:32:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.063+0000 7f7013d2e700 1 -- start start 2024-01-27T02:32:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.064+0000 7f7013d2e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f700c0c4af0 0x7f700c0b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.064+0000 7f7013d2e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f700c0cb5b0 0x7f700c149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.064+0000 7f7013d2e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f700c15a100 0x7f700c067fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.064+0000 7f7013d2e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0b4950 con 0x7f700c0cb5b0 2024-01-27T02:32:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.064+0000 7f7013d2e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0b3f80 con 0x7f700c15a100 2024-01-27T02:32:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.064+0000 7f7013d2e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c14a250 con 0x7f700c0c4af0 2024-01-27T02:32:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.064+0000 7f70112c9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f700c0cb5b0 0x7f700c149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.064+0000 7f70112c9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f700c0cb5b0 0x7f700c149d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40934/0 (socket says 172.21.15.7:40934) 2024-01-27T02:32:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.064+0000 7f70112c9700 1 -- 172.21.15.7:0/1540238386 learned_addr learned my addr 172.21.15.7:0/1540238386 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.065+0000 7f7011aca700 1 --2- 172.21.15.7:0/1540238386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f700c0c4af0 0x7f700c0b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.065+0000 7f70122cb700 1 --2- 172.21.15.7:0/1540238386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f700c15a100 0x7f700c067fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.065+0000 7f70112c9700 1 -- 172.21.15.7:0/1540238386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f700c0c4af0 msgr2=0x7f700c0b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.065+0000 7f70112c9700 1 --2- 172.21.15.7:0/1540238386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f700c0c4af0 0x7f700c0b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.065+0000 7f70112c9700 1 -- 172.21.15.7:0/1540238386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f700c15a100 msgr2=0x7f700c067fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.065+0000 7f70112c9700 1 --2- 172.21.15.7:0/1540238386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f700c15a100 0x7f700c067fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.065+0000 7f70112c9700 1 -- 172.21.15.7:0/1540238386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70080094f0 con 0x7f700c0cb5b0 2024-01-27T02:32:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.065+0000 7f70112c9700 1 --2- 172.21.15.7:0/1540238386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f700c0cb5b0 0x7f700c149d10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00d890 tx=0x7f6ffc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.065+0000 7f7001ffb700 1 -- 172.21.15.7:0/1540238386 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc00b600 con 0x7f700c0cb5b0 2024-01-27T02:32:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.065+0000 7f7001ffb700 1 -- 172.21.15.7:0/1540238386 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ffc00b760 con 0x7f700c0cb5b0 2024-01-27T02:32:03.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.065+0000 7f7013d2e700 1 -- 172.21.15.7:0/1540238386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c0b3370 con 0x7f700c0cb5b0 2024-01-27T02:32:03.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.065+0000 7f7001ffb700 1 -- 172.21.15.7:0/1540238386 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc00bae0 con 0x7f700c0cb5b0 2024-01-27T02:32:03.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.065+0000 7f7013d2e700 1 -- 172.21.15.7:0/1540238386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c0b3930 con 0x7f700c0cb5b0 2024-01-27T02:32:03.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.066+0000 7f7013d2e700 1 -- 172.21.15.7:0/1540238386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f700c0b40f0 con 0x7f700c0cb5b0 2024-01-27T02:32:03.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.066+0000 7f7001ffb700 1 -- 172.21.15.7:0/1540238386 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6ffc01e460 con 0x7f700c0cb5b0 2024-01-27T02:32:03.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.067+0000 7f7001ffb700 1 --2- 172.21.15.7:0/1540238386 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6ff0066f30 0x7f6ff00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:03.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.067+0000 7f7001ffb700 1 -- 172.21.15.7:0/1540238386 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6ffc084870 con 0x7f700c0cb5b0 2024-01-27T02:32:03.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.067+0000 7f7011aca700 1 --2- 172.21.15.7:0/1540238386 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6ff0066f30 0x7f6ff00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:03.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.068+0000 7f7011aca700 1 --2- 172.21.15.7:0/1540238386 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6ff0066f30 0x7f6ff00693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f700c06e170 tx=0x7f7008016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:03.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.071+0000 7f7001ffb700 1 -- 172.21.15.7:0/1540238386 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ffc051e30 con 0x7f700c0cb5b0 2024-01-27T02:32:03.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.275+0000 7f7013d2e700 1 -- 172.21.15.7:0/1540238386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f700c04c710 con 0x7f700c0cb5b0 2024-01-27T02:32:03.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.275+0000 7f7001ffb700 1 -- 172.21.15.7:0/1540238386 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6ffc055450 con 0x7f700c0cb5b0 2024-01-27T02:32:03.277 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:03.278 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-27T02:32:03.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:03.278 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-27T02:32:03.278 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-27T02:32:03.278 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:03.278 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-27T02:32:03.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.277+0000 7f7013d2e700 1 -- 172.21.15.7:0/1540238386 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6ff0066f30 msgr2=0x7f6ff00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:03.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.277+0000 7f7013d2e700 1 --2- 172.21.15.7:0/1540238386 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6ff0066f30 0x7f6ff00693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f700c06e170 tx=0x7f7008016000 comp rx=0 tx=0).stop 2024-01-27T02:32:03.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.277+0000 7f7013d2e700 1 -- 172.21.15.7:0/1540238386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f700c0cb5b0 msgr2=0x7f700c149d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:03.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.277+0000 7f7013d2e700 1 --2- 172.21.15.7:0/1540238386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f700c0cb5b0 0x7f700c149d10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00d890 tx=0x7f6ffc008040 comp rx=0 tx=0).stop 2024-01-27T02:32:03.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.277+0000 7f7013d2e700 1 -- 172.21.15.7:0/1540238386 shutdown_connections 2024-01-27T02:32:03.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.277+0000 7f7013d2e700 1 --2- 172.21.15.7:0/1540238386 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6ff0066f30 0x7f6ff00693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:03.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.277+0000 7f7013d2e700 1 --2- 172.21.15.7:0/1540238386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f700c15a100 0x7f700c067fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:03.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.277+0000 7f7013d2e700 1 --2- 172.21.15.7:0/1540238386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f700c0c4af0 0x7f700c0b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:03.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.277+0000 7f7013d2e700 1 --2- 172.21.15.7:0/1540238386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f700c0cb5b0 0x7f700c149d10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:03.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.277+0000 7f7013d2e700 1 -- 172.21.15.7:0/1540238386 >> 172.21.15.7:0/1540238386 conn(0x7f700c098020 msgr2=0x7f700c155ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:03.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.277+0000 7f7013d2e700 1 -- 172.21.15.7:0/1540238386 shutdown_connections 2024-01-27T02:32:03.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.277+0000 7f7013d2e700 1 -- 172.21.15.7:0/1540238386 wait complete. 2024-01-27T02:32:03.292 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-27T02:32:03.292 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-27T02:32:03.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.425+0000 7f739f315700 1 Processor -- start 2024-01-27T02:32:03.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.425+0000 7f739f315700 1 -- start start 2024-01-27T02:32:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.426+0000 7f739f315700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f73980aff60 0x7f73980cc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.426+0000 7f739f315700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73980b0330 0x7f73980b0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.426+0000 7f739f315700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f739806b950 con 0x7f73980c9400 2024-01-27T02:32:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.426+0000 7f739f315700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73980c9be0 con 0x7f73980aff60 2024-01-27T02:32:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.426+0000 7f739f315700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73980c9ec0 con 0x7f73980b0330 2024-01-27T02:32:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.427+0000 7f739d8b2700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f73980c9400 0x7f739806e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35568/0 (socket says 172.21.15.7:35568) 2024-01-27T02:32:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.427+0000 7f739d8b2700 1 -- 172.21.15.7:0/3116830240 learned_addr learned my addr 172.21.15.7:0/3116830240 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.427+0000 7f739c8b0700 1 --2- 172.21.15.7:0/3116830240 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73980b0330 0x7f73980b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.427+0000 7f739d0b1700 1 --2- 172.21.15.7:0/3116830240 >> v2:172.21.15.188:3300/0 conn(0x7f73980aff60 0x7f73980cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.427+0000 7f738bfff700 1 -- 172.21.15.7:0/3116830240 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3041028829 0 0) 0x7f739806b950 con 0x7f73980c9400 2024-01-27T02:32:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.427+0000 7f738bfff700 1 -- 172.21.15.7:0/3116830240 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7384003570 con 0x7f73980c9400 2024-01-27T02:32:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.427+0000 7f739c8b0700 1 -- 172.21.15.7:0/3116830240 >> v2:172.21.15.188:3300/0 conn(0x7f73980aff60 msgr2=0x7f73980cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.427+0000 7f739c8b0700 1 --2- 172.21.15.7:0/3116830240 >> v2:172.21.15.188:3300/0 conn(0x7f73980aff60 0x7f73980cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.427+0000 7f739c8b0700 1 -- 172.21.15.7:0/3116830240 >> v1:172.21.15.7:6789/0 conn(0x7f73980c9400 legacy=0x7f739806e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.427+0000 7f739c8b0700 1 -- 172.21.15.7:0/3116830240 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739806b5e0 con 0x7f73980b0330 2024-01-27T02:32:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.427+0000 7f739c8b0700 1 --2- 172.21.15.7:0/3116830240 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73980b0330 0x7f73980b0960 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f73940098e0 tx=0x7f73940091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae37795d3f266bc6 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.428+0000 7f738bfff700 1 -- 172.21.15.7:0/3116830240 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7394011630 con 0x7f73980b0330 2024-01-27T02:32:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.428+0000 7f738bfff700 1 -- 172.21.15.7:0/3116830240 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7394011790 con 0x7f73980b0330 2024-01-27T02:32:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.428+0000 7f738bfff700 1 -- 172.21.15.7:0/3116830240 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7394011b30 con 0x7f73980b0330 2024-01-27T02:32:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.428+0000 7f739f315700 1 -- 172.21.15.7:0/3116830240 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73980b0330 msgr2=0x7f73980b0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.428+0000 7f739f315700 1 --2- 172.21.15.7:0/3116830240 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73980b0330 0x7f73980b0960 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f73940098e0 tx=0x7f73940091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.428+0000 7f739f315700 1 -- 172.21.15.7:0/3116830240 shutdown_connections 2024-01-27T02:32:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.428+0000 7f739f315700 1 --2- 172.21.15.7:0/3116830240 >> v2:172.21.15.188:3300/0 conn(0x7f73980aff60 0x7f73980cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.428+0000 7f739f315700 1 --2- 172.21.15.7:0/3116830240 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73980b0330 0x7f73980b0960 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.428+0000 7f739f315700 1 -- 172.21.15.7:0/3116830240 >> 172.21.15.7:0/3116830240 conn(0x7f73980ca610 msgr2=0x7f73980caa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.428+0000 7f739f315700 1 -- 172.21.15.7:0/3116830240 shutdown_connections 2024-01-27T02:32:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.428+0000 7f739f315700 1 -- 172.21.15.7:0/3116830240 wait complete. 2024-01-27T02:32:03.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.429+0000 7f739f315700 1 Processor -- start 2024-01-27T02:32:03.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.430+0000 7f739f315700 1 -- start start 2024-01-27T02:32:03.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.430+0000 7f739f315700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73980aff60 0x7f73980b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.431+0000 7f739f315700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73980b0330 0x7f7398063c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.431+0000 7f739f315700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73980c9400 0x7f7398155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.431+0000 7f739f315700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73980b1250 con 0x7f73980c9400 2024-01-27T02:32:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.431+0000 7f739f315700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73981595c0 con 0x7f73980aff60 2024-01-27T02:32:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.431+0000 7f739f315700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7398159860 con 0x7f73980b0330 2024-01-27T02:32:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.431+0000 7f739d8b2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73980c9400 0x7f7398155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.431+0000 7f739d8b2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73980c9400 0x7f7398155a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40950/0 (socket says 172.21.15.7:40950) 2024-01-27T02:32:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.431+0000 7f739c8b0700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73980b0330 0x7f7398063c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.431+0000 7f739d0b1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73980aff60 0x7f73980b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.431+0000 7f739d8b2700 1 -- 172.21.15.7:0/1823657422 learned_addr learned my addr 172.21.15.7:0/1823657422 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.431+0000 7f739d8b2700 1 -- 172.21.15.7:0/1823657422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73980b0330 msgr2=0x7f7398063c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.431+0000 7f739d8b2700 1 --2- 172.21.15.7:0/1823657422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73980b0330 0x7f7398063c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.431+0000 7f739d8b2700 1 -- 172.21.15.7:0/1823657422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73980aff60 msgr2=0x7f73980b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.431+0000 7f739d8b2700 1 --2- 172.21.15.7:0/1823657422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73980aff60 0x7f73980b0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:03.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.431+0000 7f739d8b2700 1 -- 172.21.15.7:0/1823657422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7394009420 con 0x7f73980c9400 2024-01-27T02:32:03.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.431+0000 7f739d8b2700 1 --2- 172.21.15.7:0/1823657422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73980c9400 0x7f7398155a20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7390001460 tx=0x7f7390001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:03.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.431+0000 7f73897fa700 1 -- 172.21.15.7:0/1823657422 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739000a870 con 0x7f73980c9400 2024-01-27T02:32:03.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.431+0000 7f73897fa700 1 -- 172.21.15.7:0/1823657422 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739000a9d0 con 0x7f73980c9400 2024-01-27T02:32:03.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.431+0000 7f73897fa700 1 -- 172.21.15.7:0/1823657422 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739000ad50 con 0x7f73980c9400 2024-01-27T02:32:03.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.431+0000 7f739f315700 1 -- 172.21.15.7:0/1823657422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7398159b40 con 0x7f73980c9400 2024-01-27T02:32:03.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.431+0000 7f739f315700 1 -- 172.21.15.7:0/1823657422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f739815a100 con 0x7f73980c9400 2024-01-27T02:32:03.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.432+0000 7f739f315700 1 -- 172.21.15.7:0/1823657422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f739804c710 con 0x7f73980c9400 2024-01-27T02:32:03.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.433+0000 7f73897fa700 1 -- 172.21.15.7:0/1823657422 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7390007300 con 0x7f73980c9400 2024-01-27T02:32:03.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.433+0000 7f73897fa700 1 --2- 172.21.15.7:0/1823657422 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f737c066f30 0x7f737c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:03.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.434+0000 7f73897fa700 1 -- 172.21.15.7:0/1823657422 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7390084480 con 0x7f73980c9400 2024-01-27T02:32:03.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.434+0000 7f739d0b1700 1 --2- 172.21.15.7:0/1823657422 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f737c066f30 0x7f737c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:03.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.434+0000 7f739d0b1700 1 --2- 172.21.15.7:0/1823657422 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f737c066f30 0x7f737c0693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f73980c9190 tx=0x7f738c0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:03.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.438+0000 7f73897fa700 1 -- 172.21.15.7:0/1823657422 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7390051ac0 con 0x7f73980c9400 2024-01-27T02:32:03.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:03.643+0000 7f739f315700 1 -- 172.21.15.7:0/1823657422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f73980c9820 con 0x7f73980c9400 2024-01-27T02:32:04.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.607+0000 7f73897fa700 1 -- 172.21.15.7:0/1823657422 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f73900550e0 con 0x7f73980c9400 2024-01-27T02:32:04.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.611+0000 7f739f315700 1 -- 172.21.15.7:0/1823657422 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f737c066f30 msgr2=0x7f737c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:04.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.611+0000 7f739f315700 1 --2- 172.21.15.7:0/1823657422 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f737c066f30 0x7f737c0693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f73980c9190 tx=0x7f738c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:04.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.611+0000 7f739f315700 1 -- 172.21.15.7:0/1823657422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73980c9400 msgr2=0x7f7398155a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:04.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.611+0000 7f739f315700 1 --2- 172.21.15.7:0/1823657422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73980c9400 0x7f7398155a20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7390001460 tx=0x7f7390001a40 comp rx=0 tx=0).stop 2024-01-27T02:32:04.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.612+0000 7f739f315700 1 -- 172.21.15.7:0/1823657422 shutdown_connections 2024-01-27T02:32:04.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.612+0000 7f739f315700 1 --2- 172.21.15.7:0/1823657422 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f737c066f30 0x7f737c0693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:04.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.612+0000 7f739f315700 1 --2- 172.21.15.7:0/1823657422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73980c9400 0x7f7398155a20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:04.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.612+0000 7f739f315700 1 --2- 172.21.15.7:0/1823657422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73980aff60 0x7f73980b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:04.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.612+0000 7f739f315700 1 --2- 172.21.15.7:0/1823657422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73980b0330 0x7f7398063c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:04.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.612+0000 7f739f315700 1 -- 172.21.15.7:0/1823657422 >> 172.21.15.7:0/1823657422 conn(0x7f73980ca610 msgr2=0x7f739814bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:04.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.612+0000 7f739f315700 1 -- 172.21.15.7:0/1823657422 shutdown_connections 2024-01-27T02:32:04.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.612+0000 7f739f315700 1 -- 172.21.15.7:0/1823657422 wait complete. 2024-01-27T02:32:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-27T02:32:04.631 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-27T02:32:04.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.771+0000 7f6333451700 1 Processor -- start 2024-01-27T02:32:04.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.771+0000 7f6333451700 1 -- start start 2024-01-27T02:32:04.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.772+0000 7f6333451700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f632c156f30 0x7f632c0aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:04.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.772+0000 7f6333451700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f632c0af470 0x7f632c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:04.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.772+0000 7f6333451700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f632c0aeaa0 con 0x7f632c156b60 2024-01-27T02:32:04.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.772+0000 7f6333451700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f632c0adbc0 con 0x7f632c0af470 2024-01-27T02:32:04.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.772+0000 7f6333451700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f632c0ade90 con 0x7f632c156f30 2024-01-27T02:32:04.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.772+0000 7f63319ee700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f632c0af470 0x7f632c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:04.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.772+0000 7f63319ee700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f632c0af470 0x7f632c15a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40956/0 (socket says 172.21.15.7:40956) 2024-01-27T02:32:04.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.772+0000 7f63319ee700 1 -- 172.21.15.7:0/554055153 learned_addr learned my addr 172.21.15.7:0/554055153 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:04.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.772+0000 7f63309ec700 1 --2- 172.21.15.7:0/554055153 >> v2:172.21.15.188:3300/0 conn(0x7f632c156f30 0x7f632c0aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:04.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.773+0000 7f63319ee700 1 -- 172.21.15.7:0/554055153 >> v2:172.21.15.188:3300/0 conn(0x7f632c156f30 msgr2=0x7f632c0aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:04.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.773+0000 7f63319ee700 1 --2- 172.21.15.7:0/554055153 >> v2:172.21.15.188:3300/0 conn(0x7f632c156f30 0x7f632c0aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:04.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.773+0000 7f63319ee700 1 -- 172.21.15.7:0/554055153 >> v1:172.21.15.188:6789/0 conn(0x7f632c156b60 legacy=0x7f632c0ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:04.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.773+0000 7f63319ee700 1 -- 172.21.15.7:0/554055153 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c0ae170 con 0x7f632c0af470 2024-01-27T02:32:04.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.773+0000 7f63319ee700 1 --2- 172.21.15.7:0/554055153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c0af470 0x7f632c15a9b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6324009f20 tx=0x7f6324009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91f01bb20dce1e7 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:04.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.773+0000 7f6323fff700 1 -- 172.21.15.7:0/554055153 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632400e070 con 0x7f632c0af470 2024-01-27T02:32:04.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.773+0000 7f6323fff700 1 -- 172.21.15.7:0/554055153 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632400b990 con 0x7f632c0af470 2024-01-27T02:32:04.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.773+0000 7f6323fff700 1 -- 172.21.15.7:0/554055153 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63240092e0 con 0x7f632c0af470 2024-01-27T02:32:04.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.774+0000 7f6333451700 1 -- 172.21.15.7:0/554055153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c0af470 msgr2=0x7f632c15a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:04.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.774+0000 7f6333451700 1 --2- 172.21.15.7:0/554055153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c0af470 0x7f632c15a9b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6324009f20 tx=0x7f6324009840 comp rx=0 tx=0).stop 2024-01-27T02:32:04.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.774+0000 7f6333451700 1 -- 172.21.15.7:0/554055153 shutdown_connections 2024-01-27T02:32:04.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.774+0000 7f6333451700 1 --2- 172.21.15.7:0/554055153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c0af470 0x7f632c15a9b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:04.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.774+0000 7f6333451700 1 --2- 172.21.15.7:0/554055153 >> v2:172.21.15.188:3300/0 conn(0x7f632c156f30 0x7f632c0aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:04.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.774+0000 7f6333451700 1 -- 172.21.15.7:0/554055153 >> 172.21.15.7:0/554055153 conn(0x7f632c068480 msgr2=0x7f632c0cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:04.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.774+0000 7f6333451700 1 -- 172.21.15.7:0/554055153 shutdown_connections 2024-01-27T02:32:04.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.774+0000 7f6333451700 1 -- 172.21.15.7:0/554055153 wait complete. 2024-01-27T02:32:04.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.775+0000 7f6333451700 1 Processor -- start 2024-01-27T02:32:04.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.775+0000 7f6333451700 1 -- start start 2024-01-27T02:32:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.776+0000 7f6333451700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f632c0af470 0x7f632c1415c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.776+0000 7f6333451700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f632c156b60 0x7f632c0a8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.776+0000 7f6333451700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c156f30 0x7f632c0a9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.776+0000 7f6333451700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c15c580 con 0x7f632c156f30 2024-01-27T02:32:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.776+0000 7f6333451700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c1546d0 con 0x7f632c156b60 2024-01-27T02:32:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.776+0000 7f6333451700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f632c153da0 con 0x7f632c0af470 2024-01-27T02:32:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.776+0000 7f63319ee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c156f30 0x7f632c0a9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.776+0000 7f63319ee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c156f30 0x7f632c0a9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40964/0 (socket says 172.21.15.7:40964) 2024-01-27T02:32:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.776+0000 7f63319ee700 1 -- 172.21.15.7:0/3427245462 learned_addr learned my addr 172.21.15.7:0/3427245462 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.776+0000 7f63311ed700 1 --2- 172.21.15.7:0/3427245462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f632c0af470 0x7f632c1415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.776+0000 7f63309ec700 1 --2- 172.21.15.7:0/3427245462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f632c156b60 0x7f632c0a8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.776+0000 7f63319ee700 1 -- 172.21.15.7:0/3427245462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f632c0af470 msgr2=0x7f632c1415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.776+0000 7f63319ee700 1 --2- 172.21.15.7:0/3427245462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f632c0af470 0x7f632c1415c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:04.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.776+0000 7f63319ee700 1 -- 172.21.15.7:0/3427245462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f632c156b60 msgr2=0x7f632c0a8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:04.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.777+0000 7f63319ee700 1 --2- 172.21.15.7:0/3427245462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f632c156b60 0x7f632c0a8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:04.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.777+0000 7f63319ee700 1 -- 172.21.15.7:0/3427245462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6324009ae0 con 0x7f632c156f30 2024-01-27T02:32:04.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.777+0000 7f63319ee700 1 --2- 172.21.15.7:0/3427245462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c156f30 0x7f632c0a9180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f632c0700c0 tx=0x7f6324018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:04.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.777+0000 7f63217fa700 1 -- 172.21.15.7:0/3427245462 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324020070 con 0x7f632c156f30 2024-01-27T02:32:04.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.777+0000 7f63217fa700 1 -- 172.21.15.7:0/3427245462 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63240032f0 con 0x7f632c156f30 2024-01-27T02:32:04.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.777+0000 7f6333451700 1 -- 172.21.15.7:0/3427245462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f632c154080 con 0x7f632c156f30 2024-01-27T02:32:04.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.777+0000 7f63217fa700 1 -- 172.21.15.7:0/3427245462 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632401bda0 con 0x7f632c156f30 2024-01-27T02:32:04.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.777+0000 7f6333451700 1 -- 172.21.15.7:0/3427245462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f632c141b00 con 0x7f632c156f30 2024-01-27T02:32:04.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.778+0000 7f6333451700 1 -- 172.21.15.7:0/3427245462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f632c072060 con 0x7f632c156f30 2024-01-27T02:32:04.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.778+0000 7f63217fa700 1 -- 172.21.15.7:0/3427245462 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6324003040 con 0x7f632c156f30 2024-01-27T02:32:04.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.779+0000 7f63217fa700 1 --2- 172.21.15.7:0/3427245462 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6310066f30 0x7f63100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:04.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.779+0000 7f63217fa700 1 -- 172.21.15.7:0/3427245462 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f63240873b0 con 0x7f632c156f30 2024-01-27T02:32:04.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.779+0000 7f63311ed700 1 --2- 172.21.15.7:0/3427245462 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6310066f30 0x7f63100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:04.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.780+0000 7f63311ed700 1 --2- 172.21.15.7:0/3427245462 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6310066f30 0x7f63100693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f632c155f80 tx=0x7f631c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:04.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.784+0000 7f63217fa700 1 -- 172.21.15.7:0/3427245462 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6324054800 con 0x7f632c156f30 2024-01-27T02:32:04.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:04.986+0000 7f6333451700 1 -- 172.21.15.7:0/3427245462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f632c154810 con 0x7f632c156f30 2024-01-27T02:32:05.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.612+0000 7f63217fa700 1 -- 172.21.15.7:0/3427245462 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6324057e20 con 0x7f632c156f30 2024-01-27T02:32:05.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.622+0000 7f6333451700 1 -- 172.21.15.7:0/3427245462 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6310066f30 msgr2=0x7f63100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:05.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.622+0000 7f6333451700 1 --2- 172.21.15.7:0/3427245462 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6310066f30 0x7f63100693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f632c155f80 tx=0x7f631c009040 comp rx=0 tx=0).stop 2024-01-27T02:32:05.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.622+0000 7f6333451700 1 -- 172.21.15.7:0/3427245462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c156f30 msgr2=0x7f632c0a9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:05.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.622+0000 7f6333451700 1 --2- 172.21.15.7:0/3427245462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c156f30 0x7f632c0a9180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f632c0700c0 tx=0x7f6324018000 comp rx=0 tx=0).stop 2024-01-27T02:32:05.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.623+0000 7f6333451700 1 -- 172.21.15.7:0/3427245462 shutdown_connections 2024-01-27T02:32:05.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.623+0000 7f6333451700 1 --2- 172.21.15.7:0/3427245462 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6310066f30 0x7f63100693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:05.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.623+0000 7f6333451700 1 --2- 172.21.15.7:0/3427245462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c156f30 0x7f632c0a9180 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:05.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.623+0000 7f6333451700 1 --2- 172.21.15.7:0/3427245462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f632c0af470 0x7f632c1415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:05.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.623+0000 7f6333451700 1 --2- 172.21.15.7:0/3427245462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f632c156b60 0x7f632c0a8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:05.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.623+0000 7f6333451700 1 -- 172.21.15.7:0/3427245462 >> 172.21.15.7:0/3427245462 conn(0x7f632c068480 msgr2=0x7f632c0b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:05.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.623+0000 7f6333451700 1 -- 172.21.15.7:0/3427245462 shutdown_connections 2024-01-27T02:32:05.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.623+0000 7f6333451700 1 -- 172.21.15.7:0/3427245462 wait complete. 2024-01-27T02:32:05.626 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-27T02:32:05.641 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-27T02:32:05.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.777+0000 7fc9c7320700 1 Processor -- start 2024-01-27T02:32:05.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.777+0000 7fc9c7320700 1 -- start start 2024-01-27T02:32:05.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.778+0000 7fc9c7320700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9c015e3d0 0x7fc9c00a8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:05.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.778+0000 7fc9c7320700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc9c015e7a0 0x7fc9c0063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:05.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.778+0000 7fc9c7320700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c00a80c0 con 0x7fc9c0063ac0 2024-01-27T02:32:05.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.778+0000 7fc9c7320700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9c00ce6b0 con 0x7fc9c015e7a0 2024-01-27T02:32:05.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.778+0000 7fc9c7320700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c0063f40 con 0x7fc9c015e3d0 2024-01-27T02:32:05.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.778+0000 7fc9c48bb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc9c015e7a0 0x7fc9c0063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:05.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.778+0000 7fc9c48bb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc9c015e7a0 0x7fc9c0063580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40976/0 (socket says 172.21.15.7:40976) 2024-01-27T02:32:05.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.778+0000 7fc9c48bb700 1 -- 172.21.15.7:0/4237340418 learned_addr learned my addr 172.21.15.7:0/4237340418 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:05.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.778+0000 7fc9c58bd700 1 --1- 172.21.15.7:0/4237340418 >> v1:172.21.15.7:6789/0 conn(0x7fc9c0063ac0 0x7fc9c00afa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35580/0 (socket says 172.21.15.7:35580) 2024-01-27T02:32:05.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.779+0000 7fc9c50bc700 1 --2- 172.21.15.7:0/4237340418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9c015e3d0 0x7fc9c00a8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:05.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.779+0000 7fc9c48bb700 1 -- 172.21.15.7:0/4237340418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9c015e3d0 msgr2=0x7fc9c00a8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:05.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.779+0000 7fc9c48bb700 1 --2- 172.21.15.7:0/4237340418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9c015e3d0 0x7fc9c00a8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:05.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.779+0000 7fc9c48bb700 1 -- 172.21.15.7:0/4237340418 >> v1:172.21.15.7:6789/0 conn(0x7fc9c0063ac0 legacy=0x7fc9c00afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:05.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.779+0000 7fc9c48bb700 1 -- 172.21.15.7:0/4237340418 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c00a7790 con 0x7fc9c015e7a0 2024-01-27T02:32:05.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.779+0000 7fc9c50bc700 1 --2- 172.21.15.7:0/4237340418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9c015e3d0 0x7fc9c00a8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:05.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.779+0000 7fc9c48bb700 1 --2- 172.21.15.7:0/4237340418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c015e7a0 0x7fc9c0063580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc9b00099b0 tx=0x7fc9b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b7995c1c9033b2 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:05.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.779+0000 7fc9b7fff700 1 -- 172.21.15.7:0/4237340418 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b000b860 con 0x7fc9c015e7a0 2024-01-27T02:32:05.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.779+0000 7fc9b7fff700 1 -- 172.21.15.7:0/4237340418 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b000b9c0 con 0x7fc9c015e7a0 2024-01-27T02:32:05.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.779+0000 7fc9b7fff700 1 -- 172.21.15.7:0/4237340418 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b000bd60 con 0x7fc9c015e7a0 2024-01-27T02:32:05.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.780+0000 7fc9c7320700 1 -- 172.21.15.7:0/4237340418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c015e7a0 msgr2=0x7fc9c0063580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:05.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.780+0000 7fc9c7320700 1 --2- 172.21.15.7:0/4237340418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c015e7a0 0x7fc9c0063580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc9b00099b0 tx=0x7fc9b0009250 comp rx=0 tx=0).stop 2024-01-27T02:32:05.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.780+0000 7fc9c7320700 1 -- 172.21.15.7:0/4237340418 shutdown_connections 2024-01-27T02:32:05.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.780+0000 7fc9c7320700 1 --2- 172.21.15.7:0/4237340418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9c015e3d0 0x7fc9c00a8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:05.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.780+0000 7fc9c7320700 1 --2- 172.21.15.7:0/4237340418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c015e7a0 0x7fc9c0063580 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:05.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.780+0000 7fc9c7320700 1 -- 172.21.15.7:0/4237340418 >> 172.21.15.7:0/4237340418 conn(0x7fc9c0068480 msgr2=0x7fc9c00cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:05.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.780+0000 7fc9c7320700 1 -- 172.21.15.7:0/4237340418 shutdown_connections 2024-01-27T02:32:05.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.780+0000 7fc9c7320700 1 -- 172.21.15.7:0/4237340418 wait complete. 2024-01-27T02:32:05.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.781+0000 7fc9c7320700 1 Processor -- start 2024-01-27T02:32:05.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.782+0000 7fc9c7320700 1 -- start start 2024-01-27T02:32:05.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.783+0000 7fc9c7320700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c0063ac0 0x7fc9c015f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:05.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.783+0000 7fc9c7320700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9c015e3d0 0x7fc9c00b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:05.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.783+0000 7fc9c7320700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9c015e7a0 0x7fc9c01609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:05.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.783+0000 7fc9c7320700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c00b3990 con 0x7fc9c0063ac0 2024-01-27T02:32:05.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.783+0000 7fc9c7320700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c00b1110 con 0x7fc9c015e7a0 2024-01-27T02:32:05.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.783+0000 7fc9c7320700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c01530b0 con 0x7fc9c015e3d0 2024-01-27T02:32:05.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.783+0000 7fc9c50bc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c0063ac0 0x7fc9c015f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:05.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.783+0000 7fc9c50bc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c0063ac0 0x7fc9c015f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40988/0 (socket says 172.21.15.7:40988) 2024-01-27T02:32:05.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.783+0000 7fc9c50bc700 1 -- 172.21.15.7:0/1081665003 learned_addr learned my addr 172.21.15.7:0/1081665003 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:05.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.783+0000 7fc9c48bb700 1 --2- 172.21.15.7:0/1081665003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9c015e3d0 0x7fc9c00b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:05.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.783+0000 7fc9c58bd700 1 --2- 172.21.15.7:0/1081665003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9c015e7a0 0x7fc9c01609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:05.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.783+0000 7fc9c50bc700 1 -- 172.21.15.7:0/1081665003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9c015e3d0 msgr2=0x7fc9c00b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:05.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.783+0000 7fc9c50bc700 1 --2- 172.21.15.7:0/1081665003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9c015e3d0 0x7fc9c00b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:05.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.783+0000 7fc9c50bc700 1 -- 172.21.15.7:0/1081665003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9c015e7a0 msgr2=0x7fc9c01609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:05.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.783+0000 7fc9c50bc700 1 --2- 172.21.15.7:0/1081665003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9c015e7a0 0x7fc9c01609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:05.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.783+0000 7fc9c50bc700 1 -- 172.21.15.7:0/1081665003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b00094f0 con 0x7fc9c0063ac0 2024-01-27T02:32:05.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.783+0000 7fc9c58bd700 1 --2- 172.21.15.7:0/1081665003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9c015e7a0 0x7fc9c01609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:32:05.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.783+0000 7fc9c50bc700 1 --2- 172.21.15.7:0/1081665003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c0063ac0 0x7fc9c015f160 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00e8d0 tx=0x7fc9bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:05.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.783+0000 7fc9b57fa700 1 -- 172.21.15.7:0/1081665003 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc00ee70 con 0x7fc9c0063ac0 2024-01-27T02:32:05.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.784+0000 7fc9b57fa700 1 -- 172.21.15.7:0/1081665003 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9bc00c5e0 con 0x7fc9c0063ac0 2024-01-27T02:32:05.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.784+0000 7fc9c7320700 1 -- 172.21.15.7:0/1081665003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c0152dd0 con 0x7fc9c0063ac0 2024-01-27T02:32:05.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.784+0000 7fc9b57fa700 1 -- 172.21.15.7:0/1081665003 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc00c980 con 0x7fc9c0063ac0 2024-01-27T02:32:05.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.784+0000 7fc9c7320700 1 -- 172.21.15.7:0/1081665003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c00b0da0 con 0x7fc9c0063ac0 2024-01-27T02:32:05.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.785+0000 7fc9c7320700 1 -- 172.21.15.7:0/1081665003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c0072060 con 0x7fc9c0063ac0 2024-01-27T02:32:05.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.785+0000 7fc9b57fa700 1 -- 172.21.15.7:0/1081665003 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc9bc013410 con 0x7fc9c0063ac0 2024-01-27T02:32:05.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.786+0000 7fc9b57fa700 1 --2- 172.21.15.7:0/1081665003 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc9a4066f30 0x7fc9a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:05.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.786+0000 7fc9b57fa700 1 -- 172.21.15.7:0/1081665003 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fc9bc0845c0 con 0x7fc9c0063ac0 2024-01-27T02:32:05.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.786+0000 7fc9c48bb700 1 --2- 172.21.15.7:0/1081665003 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc9a4066f30 0x7fc9a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:05.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.786+0000 7fc9c48bb700 1 --2- 172.21.15.7:0/1081665003 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc9a4066f30 0x7fc9a40693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc9b00099b0 tx=0x7fc9b0017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:05.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:05.791+0000 7fc9b57fa700 1 -- 172.21.15.7:0/1081665003 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc9bc051890 con 0x7fc9c0063ac0 2024-01-27T02:32:06.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.043+0000 7fc9c7320700 1 -- 172.21.15.7:0/1081665003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc9c015b490 con 0x7fc9c0063ac0 2024-01-27T02:32:06.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.795+0000 7fc9b57fa700 1 -- 172.21.15.7:0/1081665003 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc9bc054eb0 con 0x7fc9c0063ac0 2024-01-27T02:32:06.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.798+0000 7fc9c7320700 1 -- 172.21.15.7:0/1081665003 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc9a4066f30 msgr2=0x7fc9a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.798+0000 7fc9c7320700 1 --2- 172.21.15.7:0/1081665003 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc9a4066f30 0x7fc9a40693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc9b00099b0 tx=0x7fc9b0017000 comp rx=0 tx=0).stop 2024-01-27T02:32:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.798+0000 7fc9c7320700 1 -- 172.21.15.7:0/1081665003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c0063ac0 msgr2=0x7fc9c015f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.798+0000 7fc9c7320700 1 --2- 172.21.15.7:0/1081665003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c0063ac0 0x7fc9c015f160 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00e8d0 tx=0x7fc9bc008040 comp rx=0 tx=0).stop 2024-01-27T02:32:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.799+0000 7fc9c7320700 1 -- 172.21.15.7:0/1081665003 shutdown_connections 2024-01-27T02:32:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.799+0000 7fc9c7320700 1 --2- 172.21.15.7:0/1081665003 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc9a4066f30 0x7fc9a40693e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.799+0000 7fc9c7320700 1 --2- 172.21.15.7:0/1081665003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9c015e7a0 0x7fc9c01609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.799+0000 7fc9c7320700 1 --2- 172.21.15.7:0/1081665003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c0063ac0 0x7fc9c015f160 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.799+0000 7fc9c7320700 1 --2- 172.21.15.7:0/1081665003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9c015e3d0 0x7fc9c00b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.799+0000 7fc9c7320700 1 -- 172.21.15.7:0/1081665003 >> 172.21.15.7:0/1081665003 conn(0x7fc9c0068480 msgr2=0x7fc9c0154030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.799+0000 7fc9c7320700 1 -- 172.21.15.7:0/1081665003 shutdown_connections 2024-01-27T02:32:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.800+0000 7fc9c7320700 1 -- 172.21.15.7:0/1081665003 wait complete. 2024-01-27T02:32:06.804 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 2 and data pool 3 2024-01-27T02:32:06.817 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:32:06.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.951+0000 7fa3c1a65700 1 Processor -- start 2024-01-27T02:32:06.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.951+0000 7fa3c1a65700 1 -- start start 2024-01-27T02:32:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.952+0000 7fa3c1a65700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3bc0a7fe0 0x7fa3bc06f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.952+0000 7fa3c1a65700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3bc15a880 con 0x7fa3bc0a7c10 2024-01-27T02:32:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.952+0000 7fa3c1a65700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3bc0af950 con 0x7fa3bc15dfb0 2024-01-27T02:32:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.952+0000 7fa3c1a65700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc0afde0 con 0x7fa3bc0a7fe0 2024-01-27T02:32:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.952+0000 7fa3bb7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa3bc0a7c10 0x7fa3bc0ae340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35584/0 (socket says 172.21.15.7:35584) 2024-01-27T02:32:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.952+0000 7fa3bb7fe700 1 -- 172.21.15.7:0/683321483 learned_addr learned my addr 172.21.15.7:0/683321483 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.952+0000 7fa3ba7fc700 1 -- 172.21.15.7:0/683321483 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973066527 0 0) 0x7fa3bc15a880 con 0x7fa3bc0a7c10 2024-01-27T02:32:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.952+0000 7fa3baffd700 1 --2- 172.21.15.7:0/683321483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3bc0a7fe0 0x7fa3bc06f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.952+0000 7fa3ba7fc700 1 -- 172.21.15.7:0/683321483 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3a4003570 con 0x7fa3bc0a7c10 2024-01-27T02:32:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.953+0000 7fa3ba7fc700 1 -- 172.21.15.7:0/683321483 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2073447103 0 0) 0x7fa3a4003570 con 0x7fa3bc0a7c10 2024-01-27T02:32:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.953+0000 7fa3ba7fc700 1 -- 172.21.15.7:0/683321483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3bc0a7fe0 msgr2=0x7fa3bc06f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.953+0000 7fa3ba7fc700 1 --2- 172.21.15.7:0/683321483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3bc0a7fe0 0x7fa3bc06f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.953+0000 7fa3ba7fc700 1 -- 172.21.15.7:0/683321483 >> v1:172.21.15.188:6789/0 conn(0x7fa3bc15dfb0 legacy=0x7fa3bc0b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.953+0000 7fa3ba7fc700 1 -- 172.21.15.7:0/683321483 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3bc0aed40 con 0x7fa3bc0a7c10 2024-01-27T02:32:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.953+0000 7fa3ba7fc700 1 -- 172.21.15.7:0/683321483 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7fa3ac001470 con 0x7fa3bc0a7c10 2024-01-27T02:32:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.953+0000 7fa3ba7fc700 1 -- 172.21.15.7:0/683321483 >> v1:172.21.15.7:6789/0 conn(0x7fa3bc0a7c10 legacy=0x7fa3bc0ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.953+0000 7fa3ba7fc700 1 --2- 172.21.15.7:0/683321483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3a4003a90 0x7fa3a4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.953+0000 7fa3ba7fc700 1 --2- 172.21.15.7:0/683321483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a4004810 0x7fa3a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.953+0000 7fa3ba7fc700 1 --2- 172.21.15.7:0/683321483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3bc0a7fe0 0x7fa3a40051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.953+0000 7fa3ba7fc700 1 -- 172.21.15.7:0/683321483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a400bb30 con 0x7fa3a4004810 2024-01-27T02:32:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.953+0000 7fa3ba7fc700 1 -- 172.21.15.7:0/683321483 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a400bca0 con 0x7fa3a4003a90 2024-01-27T02:32:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.953+0000 7fa3ba7fc700 1 -- 172.21.15.7:0/683321483 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a400be60 con 0x7fa3bc0a7fe0 2024-01-27T02:32:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.953+0000 7fa3baffd700 1 --2- 172.21.15.7:0/683321483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a4004810 0x7fa3a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.953+0000 7fa3bb7fe700 1 --2- 172.21.15.7:0/683321483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3a4003a90 0x7fa3a4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.953+0000 7fa3bbfff700 1 --2- 172.21.15.7:0/683321483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3bc0a7fe0 0x7fa3a40051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.953+0000 7fa3baffd700 1 -- 172.21.15.7:0/683321483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3bc0a7fe0 msgr2=0x7fa3a40051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.953+0000 7fa3baffd700 1 --2- 172.21.15.7:0/683321483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3bc0a7fe0 0x7fa3a40051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.953+0000 7fa3baffd700 1 -- 172.21.15.7:0/683321483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3a4003a90 msgr2=0x7fa3a4003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.953+0000 7fa3baffd700 1 --2- 172.21.15.7:0/683321483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3a4003a90 0x7fa3a4003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.953+0000 7fa3baffd700 1 -- 172.21.15.7:0/683321483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a4003570 con 0x7fa3a4004810 2024-01-27T02:32:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.953+0000 7fa3baffd700 1 --2- 172.21.15.7:0/683321483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a4004810 0x7fa3a4004c80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0a83d0 tx=0x7fa3b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641e08d2cf33f349 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.954+0000 7fa3ba7fc700 1 -- 172.21.15.7:0/683321483 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b000e710 con 0x7fa3a4004810 2024-01-27T02:32:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.954+0000 7fa3ba7fc700 1 -- 172.21.15.7:0/683321483 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b000e870 con 0x7fa3a4004810 2024-01-27T02:32:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.954+0000 7fa3ba7fc700 1 -- 172.21.15.7:0/683321483 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b000ec30 con 0x7fa3a4004810 2024-01-27T02:32:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.954+0000 7fa3c1a65700 1 -- 172.21.15.7:0/683321483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a4004810 msgr2=0x7fa3a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.954+0000 7fa3c1a65700 1 --2- 172.21.15.7:0/683321483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a4004810 0x7fa3a4004c80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0a83d0 tx=0x7fa3b00091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.954+0000 7fa3bbfff700 1 -- 172.21.15.7:0/683321483 reap_dead start 2024-01-27T02:32:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.954+0000 7fa3c1a65700 1 -- 172.21.15.7:0/683321483 shutdown_connections 2024-01-27T02:32:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.954+0000 7fa3c1a65700 1 -- 172.21.15.7:0/683321483 >> 172.21.15.7:0/683321483 conn(0x7fa3bc0679b0 msgr2=0x7fa3bc067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.954+0000 7fa3c1a65700 1 -- 172.21.15.7:0/683321483 shutdown_connections 2024-01-27T02:32:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.954+0000 7fa3c1a65700 1 -- 172.21.15.7:0/683321483 wait complete. 2024-01-27T02:32:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.955+0000 7fa3c1a65700 1 Processor -- start 2024-01-27T02:32:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.956+0000 7fa3c1a65700 1 -- start start 2024-01-27T02:32:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.956+0000 7fa3c1a65700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3a4004810 0x7fa3bc146c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.956+0000 7fa3c1a65700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3bc0c2160 0x7fa3bc0c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.956+0000 7fa3c1a65700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3bc15c6c0 0x7fa3bc15cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.956+0000 7fa3c1a65700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc098500 con 0x7fa3bc0c2160 2024-01-27T02:32:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.956+0000 7fa3c1a65700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc0c2b10 con 0x7fa3a4004810 2024-01-27T02:32:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.956+0000 7fa3c1a65700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc15d080 con 0x7fa3bc15c6c0 2024-01-27T02:32:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.956+0000 7fa3baffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3bc0c2160 0x7fa3bc0c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.956+0000 7fa3baffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3bc0c2160 0x7fa3bc0c25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41016/0 (socket says 172.21.15.7:41016) 2024-01-27T02:32:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.956+0000 7fa3baffd700 1 -- 172.21.15.7:0/2272439180 learned_addr learned my addr 172.21.15.7:0/2272439180 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.957+0000 7fa3bb7fe700 1 --2- 172.21.15.7:0/2272439180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3a4004810 0x7fa3bc146c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.957+0000 7fa3bbfff700 1 --2- 172.21.15.7:0/2272439180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3bc15c6c0 0x7fa3bc15cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.957+0000 7fa3baffd700 1 -- 172.21.15.7:0/2272439180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3bc15c6c0 msgr2=0x7fa3bc15cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.957+0000 7fa3baffd700 1 --2- 172.21.15.7:0/2272439180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3bc15c6c0 0x7fa3bc15cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.957+0000 7fa3baffd700 1 -- 172.21.15.7:0/2272439180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3a4004810 msgr2=0x7fa3bc146c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.957+0000 7fa3baffd700 1 --2- 172.21.15.7:0/2272439180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3a4004810 0x7fa3bc146c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.957+0000 7fa3baffd700 1 -- 172.21.15.7:0/2272439180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3bc098710 con 0x7fa3bc0c2160 2024-01-27T02:32:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.957+0000 7fa3baffd700 1 --2- 172.21.15.7:0/2272439180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3bc0c2160 0x7fa3bc0c25d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000e4b0 tx=0x7fa3b001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.957+0000 7fa3a37fe700 1 -- 172.21.15.7:0/2272439180 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0022070 con 0x7fa3bc0c2160 2024-01-27T02:32:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.957+0000 7fa3a37fe700 1 -- 172.21.15.7:0/2272439180 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b000e9e0 con 0x7fa3bc0c2160 2024-01-27T02:32:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.957+0000 7fa3a37fe700 1 -- 172.21.15.7:0/2272439180 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0027470 con 0x7fa3bc0c2160 2024-01-27T02:32:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.957+0000 7fa3c1a65700 1 -- 172.21.15.7:0/2272439180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3bc0989f0 con 0x7fa3bc0c2160 2024-01-27T02:32:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.957+0000 7fa3c1a65700 1 -- 172.21.15.7:0/2272439180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3bc098fb0 con 0x7fa3bc0c2160 2024-01-27T02:32:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.958+0000 7fa3c1a65700 1 -- 172.21.15.7:0/2272439180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3bc04c710 con 0x7fa3bc0c2160 2024-01-27T02:32:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.959+0000 7fa3a37fe700 1 -- 172.21.15.7:0/2272439180 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa3b00278a0 con 0x7fa3bc0c2160 2024-01-27T02:32:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.959+0000 7fa3a37fe700 1 --2- 172.21.15.7:0/2272439180 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa3a8066f30 0x7fa3a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.959+0000 7fa3a37fe700 1 -- 172.21.15.7:0/2272439180 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa3b00860c0 con 0x7fa3bc0c2160 2024-01-27T02:32:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.959+0000 7fa3bb7fe700 1 --2- 172.21.15.7:0/2272439180 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa3a8066f30 0x7fa3a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.960+0000 7fa3bb7fe700 1 --2- 172.21.15.7:0/2272439180 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa3a8066f30 0x7fa3a80693e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac007f30 tx=0x7fa3ac001e50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:06.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:06.964+0000 7fa3a37fe700 1 -- 172.21.15.7:0/2272439180 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3b0053400 con 0x7fa3bc0c2160 2024-01-27T02:32:07.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.170+0000 7fa3c1a65700 1 -- 172.21.15.7:0/2272439180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3bc0af020 con 0x7fa3bc0c2160 2024-01-27T02:32:07.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.171+0000 7fa3a37fe700 1 -- 172.21.15.7:0/2272439180 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15315 (secure 0 0 0) 0x7fa3b0056a20 con 0x7fa3bc0c2160 2024-01-27T02:32:07.172 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:07.172 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:32:06.045353+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:32:03.675611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-27T02:32:05.018885+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:32:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.173+0000 7fa3c1a65700 1 -- 172.21.15.7:0/2272439180 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa3a8066f30 msgr2=0x7fa3a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.173+0000 7fa3c1a65700 1 --2- 172.21.15.7:0/2272439180 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa3a8066f30 0x7fa3a80693e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac007f30 tx=0x7fa3ac001e50 comp rx=0 tx=0).stop 2024-01-27T02:32:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.173+0000 7fa3c1a65700 1 -- 172.21.15.7:0/2272439180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3bc0c2160 msgr2=0x7fa3bc0c25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.173+0000 7fa3c1a65700 1 --2- 172.21.15.7:0/2272439180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3bc0c2160 0x7fa3bc0c25d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000e4b0 tx=0x7fa3b001a000 comp rx=0 tx=0).stop 2024-01-27T02:32:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.173+0000 7fa3c1a65700 1 -- 172.21.15.7:0/2272439180 shutdown_connections 2024-01-27T02:32:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.173+0000 7fa3c1a65700 1 --2- 172.21.15.7:0/2272439180 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa3a8066f30 0x7fa3a80693e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.173+0000 7fa3c1a65700 1 --2- 172.21.15.7:0/2272439180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3bc15c6c0 0x7fa3bc15cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.173+0000 7fa3c1a65700 1 --2- 172.21.15.7:0/2272439180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3a4004810 0x7fa3bc146c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.173+0000 7fa3c1a65700 1 --2- 172.21.15.7:0/2272439180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3bc0c2160 0x7fa3bc0c25d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.173+0000 7fa3c1a65700 1 -- 172.21.15.7:0/2272439180 >> 172.21.15.7:0/2272439180 conn(0x7fa3bc0679b0 msgr2=0x7fa3bc158250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.173+0000 7fa3c1a65700 1 -- 172.21.15.7:0/2272439180 shutdown_connections 2024-01-27T02:32:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.173+0000 7fa3c1a65700 1 -- 172.21.15.7:0/2272439180 wait complete. 2024-01-27T02:32:07.188 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:32:07.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.318+0000 7f732911c700 1 Processor -- start 2024-01-27T02:32:07.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.318+0000 7f732911c700 1 -- start start 2024-01-27T02:32:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.319+0000 7f732911c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7324147d60 0x7f73240c3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.319+0000 7f732911c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f73240c8ce0 0x7f7324063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.319+0000 7f732911c700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73240c3780 con 0x7f7324148130 2024-01-27T02:32:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.319+0000 7f732911c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73240c2b70 con 0x7f73240c8ce0 2024-01-27T02:32:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.319+0000 7f732911c700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73240c2e50 con 0x7f7324147d60 2024-01-27T02:32:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.319+0000 7f732359e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f73240c8ce0 0x7f7324063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.319+0000 7f732359e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f73240c8ce0 0x7f7324063580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41022/0 (socket says 172.21.15.7:41022) 2024-01-27T02:32:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.319+0000 7f732359e700 1 -- 172.21.15.7:0/3973119309 learned_addr learned my addr 172.21.15.7:0/3973119309 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.319+0000 7f7322d9d700 1 --2- 172.21.15.7:0/3973119309 >> v2:172.21.15.188:3300/0 conn(0x7f7324147d60 0x7f73240c3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.319+0000 7f732359e700 1 -- 172.21.15.7:0/3973119309 >> v2:172.21.15.188:3300/0 conn(0x7f7324147d60 msgr2=0x7f73240c3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.319+0000 7f732359e700 1 --2- 172.21.15.7:0/3973119309 >> v2:172.21.15.188:3300/0 conn(0x7f7324147d60 0x7f73240c3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.319+0000 7f732359e700 1 -- 172.21.15.7:0/3973119309 >> v1:172.21.15.188:6789/0 conn(0x7f7324148130 legacy=0x7f73240c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.319+0000 7f732359e700 1 -- 172.21.15.7:0/3973119309 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73240c3130 con 0x7f73240c8ce0 2024-01-27T02:32:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.319+0000 7f7322d9d700 1 --2- 172.21.15.7:0/3973119309 >> v2:172.21.15.188:3300/0 conn(0x7f7324147d60 0x7f73240c3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.319+0000 7f732359e700 1 --2- 172.21.15.7:0/3973119309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73240c8ce0 0x7f7324063580 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f73240b06f0 tx=0x7f731c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647c0ebc076825a9 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.320+0000 7f7321d9b700 1 -- 172.21.15.7:0/3973119309 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c00e070 con 0x7f73240c8ce0 2024-01-27T02:32:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.320+0000 7f7321d9b700 1 -- 172.21.15.7:0/3973119309 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731c00ba50 con 0x7f73240c8ce0 2024-01-27T02:32:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.320+0000 7f7321d9b700 1 -- 172.21.15.7:0/3973119309 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c0092e0 con 0x7f73240c8ce0 2024-01-27T02:32:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.320+0000 7f732911c700 1 -- 172.21.15.7:0/3973119309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73240c8ce0 msgr2=0x7f7324063580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.320+0000 7f732911c700 1 --2- 172.21.15.7:0/3973119309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73240c8ce0 0x7f7324063580 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f73240b06f0 tx=0x7f731c009840 comp rx=0 tx=0).stop 2024-01-27T02:32:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.320+0000 7f732911c700 1 -- 172.21.15.7:0/3973119309 shutdown_connections 2024-01-27T02:32:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.320+0000 7f732911c700 1 --2- 172.21.15.7:0/3973119309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73240c8ce0 0x7f7324063580 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.320+0000 7f732911c700 1 --2- 172.21.15.7:0/3973119309 >> v2:172.21.15.188:3300/0 conn(0x7f7324147d60 0x7f73240c3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:07.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.320+0000 7f732911c700 1 -- 172.21.15.7:0/3973119309 >> 172.21.15.7:0/3973119309 conn(0x7f73240cdc30 msgr2=0x7f73240ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:07.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.320+0000 7f732911c700 1 -- 172.21.15.7:0/3973119309 shutdown_connections 2024-01-27T02:32:07.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.320+0000 7f732911c700 1 -- 172.21.15.7:0/3973119309 wait complete. 2024-01-27T02:32:07.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.322+0000 7f732911c700 1 Processor -- start 2024-01-27T02:32:07.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.322+0000 7f732911c700 1 -- start start 2024-01-27T02:32:07.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.322+0000 7f732911c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73240c8ce0 0x7f73240c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:07.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.322+0000 7f732911c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7324147d60 0x7f732415de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:07.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.322+0000 7f732911c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7324148130 0x7f73240c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:07.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.323+0000 7f732911c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73240c5530 con 0x7f7324148130 2024-01-27T02:32:07.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.323+0000 7f732911c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73240c5810 con 0x7f73240c8ce0 2024-01-27T02:32:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.323+0000 7f732911c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73240c5af0 con 0x7f7324147d60 2024-01-27T02:32:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.323+0000 7f732359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7324148130 0x7f73240c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.323+0000 7f732359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7324148130 0x7f73240c65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41024/0 (socket says 172.21.15.7:41024) 2024-01-27T02:32:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.323+0000 7f732359e700 1 -- 172.21.15.7:0/3283478689 learned_addr learned my addr 172.21.15.7:0/3283478689 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.323+0000 7f7322d9d700 1 --2- 172.21.15.7:0/3283478689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73240c8ce0 0x7f73240c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.323+0000 7f732259c700 1 --2- 172.21.15.7:0/3283478689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7324147d60 0x7f732415de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.323+0000 7f732359e700 1 -- 172.21.15.7:0/3283478689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7324147d60 msgr2=0x7f732415de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.323+0000 7f732359e700 1 --2- 172.21.15.7:0/3283478689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7324147d60 0x7f732415de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.323+0000 7f732359e700 1 -- 172.21.15.7:0/3283478689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73240c8ce0 msgr2=0x7f73240c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.323+0000 7f732359e700 1 --2- 172.21.15.7:0/3283478689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73240c8ce0 0x7f73240c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.323+0000 7f732359e700 1 -- 172.21.15.7:0/3283478689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731c009ae0 con 0x7f7324148130 2024-01-27T02:32:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.323+0000 7f732359e700 1 --2- 172.21.15.7:0/3283478689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7324148130 0x7f73240c65d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f731c009880 tx=0x7f731c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.323+0000 7f730affd700 1 -- 172.21.15.7:0/3283478689 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c00be60 con 0x7f7324148130 2024-01-27T02:32:07.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.323+0000 7f730affd700 1 -- 172.21.15.7:0/3283478689 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731c008d20 con 0x7f7324148130 2024-01-27T02:32:07.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.323+0000 7f732911c700 1 -- 172.21.15.7:0/3283478689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73240c5dd0 con 0x7f7324148130 2024-01-27T02:32:07.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.323+0000 7f730affd700 1 -- 172.21.15.7:0/3283478689 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c0266b0 con 0x7f7324148130 2024-01-27T02:32:07.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.324+0000 7f732911c700 1 -- 172.21.15.7:0/3283478689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73240c5220 con 0x7f7324148130 2024-01-27T02:32:07.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.324+0000 7f732911c700 1 -- 172.21.15.7:0/3283478689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f732404c710 con 0x7f7324148130 2024-01-27T02:32:07.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.326+0000 7f730affd700 1 -- 172.21.15.7:0/3283478689 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f731c003040 con 0x7f7324148130 2024-01-27T02:32:07.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.326+0000 7f730affd700 1 --2- 172.21.15.7:0/3283478689 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7310066f30 0x7f73100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:07.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.326+0000 7f730affd700 1 -- 172.21.15.7:0/3283478689 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f731c086860 con 0x7f7324148130 2024-01-27T02:32:07.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.326+0000 7f7322d9d700 1 --2- 172.21.15.7:0/3283478689 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7310066f30 0x7f73100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:07.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.327+0000 7f7322d9d700 1 --2- 172.21.15.7:0/3283478689 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7310066f30 0x7f73100693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7314006f00 tx=0x7f7314008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.330+0000 7f730affd700 1 -- 172.21.15.7:0/3283478689 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f731c053b50 con 0x7f7324148130 2024-01-27T02:32:07.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.534+0000 7f732911c700 1 -- 172.21.15.7:0/3283478689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f732406f750 con 0x7f7324148130 2024-01-27T02:32:07.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.535+0000 7f730affd700 1 -- 172.21.15.7:0/3283478689 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15315 (secure 0 0 0) 0x7f731c057170 con 0x7f7324148130 2024-01-27T02:32:07.537 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:07.537 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:32:06.045353+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:32:03.675611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-27T02:32:05.018885+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:32:07.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.537+0000 7f732911c700 1 -- 172.21.15.7:0/3283478689 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7310066f30 msgr2=0x7f73100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:07.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.537+0000 7f732911c700 1 --2- 172.21.15.7:0/3283478689 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7310066f30 0x7f73100693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7314006f00 tx=0x7f7314008040 comp rx=0 tx=0).stop 2024-01-27T02:32:07.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.537+0000 7f732911c700 1 -- 172.21.15.7:0/3283478689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7324148130 msgr2=0x7f73240c65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:07.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.537+0000 7f732911c700 1 --2- 172.21.15.7:0/3283478689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7324148130 0x7f73240c65d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f731c009880 tx=0x7f731c018000 comp rx=0 tx=0).stop 2024-01-27T02:32:07.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.537+0000 7f732911c700 1 -- 172.21.15.7:0/3283478689 shutdown_connections 2024-01-27T02:32:07.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.537+0000 7f732911c700 1 --2- 172.21.15.7:0/3283478689 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7310066f30 0x7f73100693e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:07.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.537+0000 7f732911c700 1 --2- 172.21.15.7:0/3283478689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7324148130 0x7f73240c65d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:07.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.537+0000 7f732911c700 1 --2- 172.21.15.7:0/3283478689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73240c8ce0 0x7f73240c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:07.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.537+0000 7f732911c700 1 --2- 172.21.15.7:0/3283478689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7324147d60 0x7f732415de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:07.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.537+0000 7f732911c700 1 -- 172.21.15.7:0/3283478689 >> 172.21.15.7:0/3283478689 conn(0x7f73240cdc30 msgr2=0x7f73240b3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:07.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.537+0000 7f732911c700 1 -- 172.21.15.7:0/3283478689 shutdown_connections 2024-01-27T02:32:07.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.537+0000 7f732911c700 1 -- 172.21.15.7:0/3283478689 wait complete. 2024-01-27T02:32:07.552 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-27T02:32:07.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.687+0000 7eff01965700 1 Processor -- start 2024-01-27T02:32:07.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.687+0000 7eff01965700 1 -- start start 2024-01-27T02:32:07.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.687+0000 7eff01965700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efefc0cc400 0x7efefc0c8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:07.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.687+0000 7eff01965700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efefc15b5f0 con 0x7efefc154600 2024-01-27T02:32:07.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.687+0000 7eff01965700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efefc15aa20 con 0x7efefc0c9be0 2024-01-27T02:32:07.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.687+0000 7eff01965700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efefc06dc80 con 0x7efefc0cc400 2024-01-27T02:32:07.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.688+0000 7efefaffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efefc154600 0x7efefc0c94d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35586/0 (socket says 172.21.15.7:35586) 2024-01-27T02:32:07.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.688+0000 7efefaffd700 1 -- 172.21.15.7:0/3888731473 learned_addr learned my addr 172.21.15.7:0/3888731473 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:07.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.688+0000 7efefb7fe700 1 --2- 172.21.15.7:0/3888731473 >> v2:172.21.15.188:3300/0 conn(0x7efefc0cc400 0x7efefc0c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:07.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.688+0000 7efef9ffb700 1 -- 172.21.15.7:0/3888731473 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1835060994 0 0) 0x7efefc15b5f0 con 0x7efefc154600 2024-01-27T02:32:07.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.688+0000 7efef9ffb700 1 -- 172.21.15.7:0/3888731473 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efee8003570 con 0x7efefc154600 2024-01-27T02:32:07.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.688+0000 7efef9ffb700 1 -- 172.21.15.7:0/3888731473 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3255304700 0 0) 0x7efefc15aa20 con 0x7efefc0c9be0 2024-01-27T02:32:07.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.688+0000 7efef9ffb700 1 -- 172.21.15.7:0/3888731473 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efefc15b5f0 con 0x7efefc0c9be0 2024-01-27T02:32:07.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.688+0000 7efefb7fe700 1 -- 172.21.15.7:0/3888731473 >> v1:172.21.15.188:6789/0 conn(0x7efefc0c9be0 legacy=0x7efefc0c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:07.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.688+0000 7efefb7fe700 1 -- 172.21.15.7:0/3888731473 >> v1:172.21.15.7:6789/0 conn(0x7efefc154600 legacy=0x7efefc0c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:07.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.688+0000 7efef9ffb700 1 -- 172.21.15.7:0/3888731473 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1121093867 0 0) 0x7efee8003570 con 0x7efefc154600 2024-01-27T02:32:07.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.688+0000 7efefb7fe700 1 -- 172.21.15.7:0/3888731473 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efefc15acc0 con 0x7efefc0cc400 2024-01-27T02:32:07.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.688+0000 7efefb7fe700 1 --2- 172.21.15.7:0/3888731473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efefc0cc400 0x7efefc0c8640 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efef4009e60 tx=0x7efef4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=accd0103daca1599 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.689+0000 7efef9ffb700 1 -- 172.21.15.7:0/3888731473 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef400e070 con 0x7efefc0cc400 2024-01-27T02:32:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.689+0000 7efef9ffb700 1 -- 172.21.15.7:0/3888731473 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef400b7a0 con 0x7efefc0cc400 2024-01-27T02:32:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.689+0000 7efef9ffb700 1 -- 172.21.15.7:0/3888731473 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec001470 con 0x7efefc0cc400 2024-01-27T02:32:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.689+0000 7eff01965700 1 -- 172.21.15.7:0/3888731473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efefc0cc400 msgr2=0x7efefc0c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.689+0000 7eff01965700 1 --2- 172.21.15.7:0/3888731473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efefc0cc400 0x7efefc0c8640 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efef4009e60 tx=0x7efef4009780 comp rx=0 tx=0).stop 2024-01-27T02:32:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.689+0000 7eff01965700 1 -- 172.21.15.7:0/3888731473 shutdown_connections 2024-01-27T02:32:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.689+0000 7eff01965700 1 --2- 172.21.15.7:0/3888731473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efefc0cc400 0x7efefc0c8640 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.689+0000 7eff01965700 1 -- 172.21.15.7:0/3888731473 >> 172.21.15.7:0/3888731473 conn(0x7efefc063580 msgr2=0x7efefc063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.689+0000 7eff01965700 1 -- 172.21.15.7:0/3888731473 shutdown_connections 2024-01-27T02:32:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.689+0000 7eff01965700 1 -- 172.21.15.7:0/3888731473 wait complete. 2024-01-27T02:32:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.691+0000 7eff01965700 1 Processor -- start 2024-01-27T02:32:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.691+0000 7eff01965700 1 -- start start 2024-01-27T02:32:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.691+0000 7eff01965700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efefc0c9be0 0x7efefc0aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.691+0000 7eff01965700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efefc0cc400 0x7efefc14a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.691+0000 7eff01965700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efefc154600 0x7efefc0ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.691+0000 7eff01965700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc1597c0 con 0x7efefc154600 2024-01-27T02:32:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.691+0000 7eff01965700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc152f30 con 0x7efefc0cc400 2024-01-27T02:32:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.691+0000 7eff01965700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efefc158ed0 con 0x7efefc0c9be0 2024-01-27T02:32:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.692+0000 7efefb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efefc154600 0x7efefc0ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.692+0000 7efefb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efefc154600 0x7efefc0ace00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41026/0 (socket says 172.21.15.7:41026) 2024-01-27T02:32:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.692+0000 7efefb7fe700 1 -- 172.21.15.7:0/2260224874 learned_addr learned my addr 172.21.15.7:0/2260224874 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.692+0000 7efefaffd700 1 --2- 172.21.15.7:0/2260224874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efefc0c9be0 0x7efefc0aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.692+0000 7efefa7fc700 1 --2- 172.21.15.7:0/2260224874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efefc0cc400 0x7efefc14a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.692+0000 7efefb7fe700 1 -- 172.21.15.7:0/2260224874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efefc0c9be0 msgr2=0x7efefc0aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.692+0000 7efefb7fe700 1 --2- 172.21.15.7:0/2260224874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efefc0c9be0 0x7efefc0aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.692+0000 7efefb7fe700 1 -- 172.21.15.7:0/2260224874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efefc0cc400 msgr2=0x7efefc14a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.692+0000 7efefb7fe700 1 --2- 172.21.15.7:0/2260224874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efefc0cc400 0x7efefc14a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.692+0000 7efefb7fe700 1 -- 172.21.15.7:0/2260224874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef4009a20 con 0x7efefc154600 2024-01-27T02:32:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.692+0000 7efefaffd700 1 --2- 172.21.15.7:0/2260224874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efefc0c9be0 0x7efefc0aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.692+0000 7efefb7fe700 1 --2- 172.21.15.7:0/2260224874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efefc154600 0x7efefc0ace00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efef4003000 tx=0x7efef4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.692+0000 7efef2ffd700 1 -- 172.21.15.7:0/2260224874 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef4012cd0 con 0x7efefc154600 2024-01-27T02:32:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.692+0000 7efef2ffd700 1 -- 172.21.15.7:0/2260224874 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef4012e30 con 0x7efefc154600 2024-01-27T02:32:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.692+0000 7eff01965700 1 -- 172.21.15.7:0/2260224874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efefc159170 con 0x7efefc154600 2024-01-27T02:32:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.692+0000 7efef2ffd700 1 -- 172.21.15.7:0/2260224874 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef401e660 con 0x7efefc154600 2024-01-27T02:32:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.692+0000 7eff01965700 1 -- 172.21.15.7:0/2260224874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efefc0ad500 con 0x7efefc154600 2024-01-27T02:32:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.693+0000 7eff01965700 1 -- 172.21.15.7:0/2260224874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efefc072060 con 0x7efefc154600 2024-01-27T02:32:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.694+0000 7efef2ffd700 1 -- 172.21.15.7:0/2260224874 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7efef401e9d0 con 0x7efefc154600 2024-01-27T02:32:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.694+0000 7efef2ffd700 1 --2- 172.21.15.7:0/2260224874 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7efedc066f30 0x7efedc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.694+0000 7efef2ffd700 1 -- 172.21.15.7:0/2260224874 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efef4086830 con 0x7efefc154600 2024-01-27T02:32:07.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.694+0000 7efefaffd700 1 --2- 172.21.15.7:0/2260224874 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7efedc066f30 0x7efedc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:07.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.695+0000 7efefaffd700 1 --2- 172.21.15.7:0/2260224874 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7efedc066f30 0x7efedc0693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efefc06f900 tx=0x7efeec000bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:07.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.699+0000 7efef2ffd700 1 -- 172.21.15.7:0/2260224874 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efef4053a00 con 0x7efefc154600 2024-01-27T02:32:07.751 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:32:07.753 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:32:07.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:07.962+0000 7eff01965700 1 -- 172.21.15.7:0/2260224874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7efefc15afa0 con 0x7efefc154600 2024-01-27T02:32:08.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.806+0000 7efef2ffd700 1 -- 172.21.15.7:0/2260224874 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7efef4057020 con 0x7efefc154600 2024-01-27T02:32:08.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.808+0000 7eff01965700 1 -- 172.21.15.7:0/2260224874 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7efedc066f30 msgr2=0x7efedc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:08.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.808+0000 7eff01965700 1 --2- 172.21.15.7:0/2260224874 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7efedc066f30 0x7efedc0693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efefc06f900 tx=0x7efeec000bc0 comp rx=0 tx=0).stop 2024-01-27T02:32:08.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.808+0000 7eff01965700 1 -- 172.21.15.7:0/2260224874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efefc154600 msgr2=0x7efefc0ace00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:08.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.808+0000 7eff01965700 1 --2- 172.21.15.7:0/2260224874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efefc154600 0x7efefc0ace00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efef4003000 tx=0x7efef4017000 comp rx=0 tx=0).stop 2024-01-27T02:32:08.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.809+0000 7eff01965700 1 -- 172.21.15.7:0/2260224874 shutdown_connections 2024-01-27T02:32:08.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.809+0000 7eff01965700 1 --2- 172.21.15.7:0/2260224874 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7efedc066f30 0x7efedc0693e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:08.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.809+0000 7eff01965700 1 --2- 172.21.15.7:0/2260224874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efefc154600 0x7efefc0ace00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:08.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.809+0000 7eff01965700 1 --2- 172.21.15.7:0/2260224874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efefc0c9be0 0x7efefc0aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:08.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.809+0000 7eff01965700 1 --2- 172.21.15.7:0/2260224874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efefc0cc400 0x7efefc14a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:08.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.809+0000 7eff01965700 1 -- 172.21.15.7:0/2260224874 >> 172.21.15.7:0/2260224874 conn(0x7efefc063580 msgr2=0x7efefc063950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:08.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.809+0000 7eff01965700 1 -- 172.21.15.7:0/2260224874 shutdown_connections 2024-01-27T02:32:08.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.809+0000 7eff01965700 1 -- 172.21.15.7:0/2260224874 wait complete. 2024-01-27T02:32:08.826 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-27T02:32:08.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.964+0000 7fbc2e5cb700 1 Processor -- start 2024-01-27T02:32:08.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.964+0000 7fbc2e5cb700 1 -- start start 2024-01-27T02:32:08.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.965+0000 7fbc2e5cb700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbc28156b60 0x7fbc280ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:08.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.965+0000 7fbc2e5cb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc28156f30 0x7fbc2815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:08.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.965+0000 7fbc2e5cb700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc280aeaa0 con 0x7fbc280ad9f0 2024-01-27T02:32:08.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.965+0000 7fbc2e5cb700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc280ade90 con 0x7fbc28156b60 2024-01-27T02:32:08.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.965+0000 7fbc2e5cb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc280ae170 con 0x7fbc28156f30 2024-01-27T02:32:08.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.965+0000 7fbc277fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbc280ad9f0 0x7fbc280aef30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35596/0 (socket says 172.21.15.7:35596) 2024-01-27T02:32:08.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.966+0000 7fbc277fe700 1 -- 172.21.15.7:0/1654926415 learned_addr learned my addr 172.21.15.7:0/1654926415 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:08.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.966+0000 7fbc27fff700 1 --2- 172.21.15.7:0/1654926415 >> v2:172.21.15.188:3300/0 conn(0x7fbc28156b60 0x7fbc280ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:08.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.966+0000 7fbc2cb68700 1 --2- 172.21.15.7:0/1654926415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc28156f30 0x7fbc2815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:08.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.966+0000 7fbc26ffd700 1 -- 172.21.15.7:0/1654926415 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3313419226 0 0) 0x7fbc280aeaa0 con 0x7fbc280ad9f0 2024-01-27T02:32:08.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.966+0000 7fbc26ffd700 1 -- 172.21.15.7:0/1654926415 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc10003570 con 0x7fbc280ad9f0 2024-01-27T02:32:08.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.966+0000 7fbc26ffd700 1 -- 172.21.15.7:0/1654926415 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 522758213 0 0) 0x7fbc10003570 con 0x7fbc280ad9f0 2024-01-27T02:32:08.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.966+0000 7fbc27fff700 1 -- 172.21.15.7:0/1654926415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc28156f30 msgr2=0x7fbc2815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:08.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.966+0000 7fbc27fff700 1 --2- 172.21.15.7:0/1654926415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc28156f30 0x7fbc2815a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:08.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.966+0000 7fbc27fff700 1 -- 172.21.15.7:0/1654926415 >> v1:172.21.15.7:6789/0 conn(0x7fbc280ad9f0 legacy=0x7fbc280aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:08.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.966+0000 7fbc27fff700 1 -- 172.21.15.7:0/1654926415 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc280ae450 con 0x7fbc28156b60 2024-01-27T02:32:08.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.966+0000 7fbc2cb68700 1 --2- 172.21.15.7:0/1654926415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc28156f30 0x7fbc2815a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:32:08.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.966+0000 7fbc27fff700 1 --2- 172.21.15.7:0/1654926415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc28156b60 0x7fbc280ad4b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbc180098e0 tx=0x7fbc180091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=723a05ecb5cf7946 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:08.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.966+0000 7fbc26ffd700 1 -- 172.21.15.7:0/1654926415 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c001470 con 0x7fbc28156b60 2024-01-27T02:32:08.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.967+0000 7fbc26ffd700 1 -- 172.21.15.7:0/1654926415 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc18011420 con 0x7fbc28156b60 2024-01-27T02:32:08.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.967+0000 7fbc26ffd700 1 -- 172.21.15.7:0/1654926415 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc180117c0 con 0x7fbc28156b60 2024-01-27T02:32:08.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.967+0000 7fbc2e5cb700 1 -- 172.21.15.7:0/1654926415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc28156b60 msgr2=0x7fbc280ad4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:08.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.967+0000 7fbc2e5cb700 1 --2- 172.21.15.7:0/1654926415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc28156b60 0x7fbc280ad4b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbc180098e0 tx=0x7fbc180091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:08.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.967+0000 7fbc2e5cb700 1 -- 172.21.15.7:0/1654926415 shutdown_connections 2024-01-27T02:32:08.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.967+0000 7fbc2e5cb700 1 --2- 172.21.15.7:0/1654926415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc28156f30 0x7fbc2815a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:08.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.967+0000 7fbc2e5cb700 1 --2- 172.21.15.7:0/1654926415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc28156b60 0x7fbc280ad4b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:08.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.967+0000 7fbc2e5cb700 1 -- 172.21.15.7:0/1654926415 >> 172.21.15.7:0/1654926415 conn(0x7fbc28068480 msgr2=0x7fbc280cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:08.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.967+0000 7fbc2e5cb700 1 -- 172.21.15.7:0/1654926415 shutdown_connections 2024-01-27T02:32:08.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.967+0000 7fbc2e5cb700 1 -- 172.21.15.7:0/1654926415 wait complete. 2024-01-27T02:32:08.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.968+0000 7fbc2e5cb700 1 Processor -- start 2024-01-27T02:32:08.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.969+0000 7fbc2e5cb700 1 -- start start 2024-01-27T02:32:08.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.969+0000 7fbc2e5cb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc280ad9f0 0x7fbc2814a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:08.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.969+0000 7fbc2e5cb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc28156b60 0x7fbc280c7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:08.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.969+0000 7fbc2e5cb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc28156f30 0x7fbc2806b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:08.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.969+0000 7fbc2e5cb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc280b1b30 con 0x7fbc28156b60 2024-01-27T02:32:08.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.969+0000 7fbc2e5cb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc280b0f60 con 0x7fbc280ad9f0 2024-01-27T02:32:08.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.969+0000 7fbc2e5cb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc280b1200 con 0x7fbc28156f30 2024-01-27T02:32:08.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.969+0000 7fbc277fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc28156b60 0x7fbc280c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:08.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.969+0000 7fbc277fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc28156b60 0x7fbc280c7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41040/0 (socket says 172.21.15.7:41040) 2024-01-27T02:32:08.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.969+0000 7fbc277fe700 1 -- 172.21.15.7:0/1082193515 learned_addr learned my addr 172.21.15.7:0/1082193515 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:08.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.970+0000 7fbc2cb68700 1 --2- 172.21.15.7:0/1082193515 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc28156f30 0x7fbc2806b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:08.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.970+0000 7fbc27fff700 1 --2- 172.21.15.7:0/1082193515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc280ad9f0 0x7fbc2814a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:08.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.970+0000 7fbc277fe700 1 -- 172.21.15.7:0/1082193515 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc28156f30 msgr2=0x7fbc2806b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:08.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.970+0000 7fbc277fe700 1 --2- 172.21.15.7:0/1082193515 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc28156f30 0x7fbc2806b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:08.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.970+0000 7fbc277fe700 1 -- 172.21.15.7:0/1082193515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc280ad9f0 msgr2=0x7fbc2814a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:08.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.970+0000 7fbc277fe700 1 --2- 172.21.15.7:0/1082193515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc280ad9f0 0x7fbc2814a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:08.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.970+0000 7fbc277fe700 1 -- 172.21.15.7:0/1082193515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc20009a20 con 0x7fbc28156b60 2024-01-27T02:32:08.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.970+0000 7fbc277fe700 1 --2- 172.21.15.7:0/1082193515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc28156b60 0x7fbc280c7730 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00cb90 tx=0x7fbc1c001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:08.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.970+0000 7fbc0ffff700 1 -- 172.21.15.7:0/1082193515 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c007360 con 0x7fbc28156b60 2024-01-27T02:32:08.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.970+0000 7fbc0ffff700 1 -- 172.21.15.7:0/1082193515 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1c0074e0 con 0x7fbc28156b60 2024-01-27T02:32:08.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.970+0000 7fbc2e5cb700 1 -- 172.21.15.7:0/1082193515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc18009420 con 0x7fbc28156b60 2024-01-27T02:32:08.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.970+0000 7fbc0ffff700 1 -- 172.21.15.7:0/1082193515 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c004830 con 0x7fbc28156b60 2024-01-27T02:32:08.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.970+0000 7fbc2e5cb700 1 -- 172.21.15.7:0/1082193515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc280b17c0 con 0x7fbc28156b60 2024-01-27T02:32:08.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.971+0000 7fbc2e5cb700 1 -- 172.21.15.7:0/1082193515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc280b1ca0 con 0x7fbc28156b60 2024-01-27T02:32:08.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.972+0000 7fbc0ffff700 1 -- 172.21.15.7:0/1082193515 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbc1c004bc0 con 0x7fbc28156b60 2024-01-27T02:32:08.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.972+0000 7fbc0ffff700 1 --2- 172.21.15.7:0/1082193515 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbc14066f30 0x7fbc140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:08.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.972+0000 7fbc0ffff700 1 -- 172.21.15.7:0/1082193515 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbc1c083d10 con 0x7fbc28156b60 2024-01-27T02:32:08.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.972+0000 7fbc27fff700 1 --2- 172.21.15.7:0/1082193515 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbc14066f30 0x7fbc140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:08.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.973+0000 7fbc27fff700 1 --2- 172.21.15.7:0/1082193515 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbc14066f30 0x7fbc140693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbc180093f0 tx=0x7fbc18016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:08.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:08.977+0000 7fbc0ffff700 1 -- 172.21.15.7:0/1082193515 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc1c0510c0 con 0x7fbc28156b60 2024-01-27T02:32:09.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.228+0000 7fbc2e5cb700 1 -- 172.21.15.7:0/1082193515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbc28072060 con 0x7fbc28156b60 2024-01-27T02:32:09.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.813+0000 7fbc0ffff700 1 -- 172.21.15.7:0/1082193515 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbc1c018020 con 0x7fbc28156b60 2024-01-27T02:32:09.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.816+0000 7fbc2e5cb700 1 -- 172.21.15.7:0/1082193515 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbc14066f30 msgr2=0x7fbc140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:09.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.816+0000 7fbc2e5cb700 1 --2- 172.21.15.7:0/1082193515 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbc14066f30 0x7fbc140693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbc180093f0 tx=0x7fbc18016000 comp rx=0 tx=0).stop 2024-01-27T02:32:09.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.816+0000 7fbc2e5cb700 1 -- 172.21.15.7:0/1082193515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc28156b60 msgr2=0x7fbc280c7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:09.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.816+0000 7fbc2e5cb700 1 --2- 172.21.15.7:0/1082193515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc28156b60 0x7fbc280c7730 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00cb90 tx=0x7fbc1c001b50 comp rx=0 tx=0).stop 2024-01-27T02:32:09.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.816+0000 7fbc2e5cb700 1 -- 172.21.15.7:0/1082193515 shutdown_connections 2024-01-27T02:32:09.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.816+0000 7fbc2e5cb700 1 --2- 172.21.15.7:0/1082193515 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbc14066f30 0x7fbc140693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:09.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.816+0000 7fbc2e5cb700 1 --2- 172.21.15.7:0/1082193515 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc28156f30 0x7fbc2806b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:09.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.816+0000 7fbc2e5cb700 1 --2- 172.21.15.7:0/1082193515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc280ad9f0 0x7fbc2814a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:09.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.816+0000 7fbc2e5cb700 1 --2- 172.21.15.7:0/1082193515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc28156b60 0x7fbc280c7730 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:09.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.816+0000 7fbc2e5cb700 1 -- 172.21.15.7:0/1082193515 >> 172.21.15.7:0/1082193515 conn(0x7fbc28068480 msgr2=0x7fbc28153a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:09.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.816+0000 7fbc2e5cb700 1 -- 172.21.15.7:0/1082193515 shutdown_connections 2024-01-27T02:32:09.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.816+0000 7fbc2e5cb700 1 -- 172.21.15.7:0/1082193515 wait complete. 2024-01-27T02:32:09.832 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:32:09.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.966+0000 7fb6c06b8700 1 Processor -- start 2024-01-27T02:32:09.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.966+0000 7fb6c06b8700 1 -- start start 2024-01-27T02:32:09.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.967+0000 7fb6c06b8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6b806fa70 0x7fb6b8165280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:09.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.967+0000 7fb6c06b8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b815a880 con 0x7fb6b80a7c10 2024-01-27T02:32:09.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.967+0000 7fb6c06b8700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b80af950 con 0x7fb6b80a7fe0 2024-01-27T02:32:09.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.967+0000 7fb6c06b8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b80afde0 con 0x7fb6b806fa70 2024-01-27T02:32:09.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.967+0000 7fb6be454700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb6b80a7c10 0x7fb6b80ae340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35602/0 (socket says 172.21.15.7:35602) 2024-01-27T02:32:09.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.967+0000 7fb6be454700 1 -- 172.21.15.7:0/6740322 learned_addr learned my addr 172.21.15.7:0/6740322 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:09.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.967+0000 7fb6bec55700 1 --2- 172.21.15.7:0/6740322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6b806fa70 0x7fb6b8165280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:09.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.967+0000 7fb6bd452700 1 -- 172.21.15.7:0/6740322 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2914427806 0 0) 0x7fb6b815a880 con 0x7fb6b80a7c10 2024-01-27T02:32:09.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.968+0000 7fb6bd452700 1 -- 172.21.15.7:0/6740322 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6a4003570 con 0x7fb6b80a7c10 2024-01-27T02:32:09.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.968+0000 7fb6bd452700 1 -- 172.21.15.7:0/6740322 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2503327491 0 0) 0x7fb6b80af950 con 0x7fb6b80a7fe0 2024-01-27T02:32:09.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.968+0000 7fb6bd452700 1 -- 172.21.15.7:0/6740322 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6b815a880 con 0x7fb6b80a7fe0 2024-01-27T02:32:09.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.968+0000 7fb6bec55700 1 -- 172.21.15.7:0/6740322 >> v1:172.21.15.188:6789/0 conn(0x7fb6b80a7fe0 legacy=0x7fb6b806f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:09.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.968+0000 7fb6bec55700 1 -- 172.21.15.7:0/6740322 >> v1:172.21.15.7:6789/0 conn(0x7fb6b80a7c10 legacy=0x7fb6b80ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:09.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.968+0000 7fb6bd452700 1 -- 172.21.15.7:0/6740322 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2759463013 0 0) 0x7fb6a4003570 con 0x7fb6b80a7c10 2024-01-27T02:32:09.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.968+0000 7fb6bec55700 1 -- 172.21.15.7:0/6740322 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b80aed40 con 0x7fb6b806fa70 2024-01-27T02:32:09.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.968+0000 7fb6bec55700 1 --2- 172.21.15.7:0/6740322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6b806fa70 0x7fb6b8165280 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009e90 tx=0x7fb6b0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54fb6eb33a553828 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:09.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.968+0000 7fb6bd452700 1 -- 172.21.15.7:0/6740322 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b000e070 con 0x7fb6b806fa70 2024-01-27T02:32:09.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.968+0000 7fb6bd452700 1 -- 172.21.15.7:0/6740322 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b000b870 con 0x7fb6b806fa70 2024-01-27T02:32:09.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.968+0000 7fb6bd452700 1 -- 172.21.15.7:0/6740322 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4001470 con 0x7fb6b806fa70 2024-01-27T02:32:09.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.969+0000 7fb6c06b8700 1 -- 172.21.15.7:0/6740322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6b806fa70 msgr2=0x7fb6b8165280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:09.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.969+0000 7fb6c06b8700 1 --2- 172.21.15.7:0/6740322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6b806fa70 0x7fb6b8165280 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009e90 tx=0x7fb6b0009780 comp rx=0 tx=0).stop 2024-01-27T02:32:09.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.969+0000 7fb6c06b8700 1 -- 172.21.15.7:0/6740322 shutdown_connections 2024-01-27T02:32:09.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.969+0000 7fb6c06b8700 1 --2- 172.21.15.7:0/6740322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6b806fa70 0x7fb6b8165280 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:09.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.969+0000 7fb6c06b8700 1 -- 172.21.15.7:0/6740322 >> 172.21.15.7:0/6740322 conn(0x7fb6b80679b0 msgr2=0x7fb6b8067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:09.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.969+0000 7fb6c06b8700 1 -- 172.21.15.7:0/6740322 shutdown_connections 2024-01-27T02:32:09.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.969+0000 7fb6c06b8700 1 -- 172.21.15.7:0/6740322 wait complete. 2024-01-27T02:32:09.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.970+0000 7fb6c06b8700 1 Processor -- start 2024-01-27T02:32:09.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.970+0000 7fb6c06b8700 1 -- start start 2024-01-27T02:32:09.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.971+0000 7fb6c06b8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6b806fa70 0x7fb6b8098d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:09.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.971+0000 7fb6c06b8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6b80a7c10 0x7fb6b806aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:09.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.971+0000 7fb6c06b8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6b80a7fe0 0x7fb6b814fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:09.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.971+0000 7fb6c06b8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b806b430 con 0x7fb6b80a7fe0 2024-01-27T02:32:09.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.971+0000 7fb6c06b8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b8099fd0 con 0x7fb6b806fa70 2024-01-27T02:32:09.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.971+0000 7fb6c06b8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b809a460 con 0x7fb6b80a7c10 2024-01-27T02:32:09.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.971+0000 7fb6bec55700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6b80a7fe0 0x7fb6b814fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:09.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.971+0000 7fb6bec55700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6b80a7fe0 0x7fb6b814fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41042/0 (socket says 172.21.15.7:41042) 2024-01-27T02:32:09.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.971+0000 7fb6bec55700 1 -- 172.21.15.7:0/3051365858 learned_addr learned my addr 172.21.15.7:0/3051365858 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:09.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.971+0000 7fb6bdc53700 1 --2- 172.21.15.7:0/3051365858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6b80a7c10 0x7fb6b806aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:09.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.971+0000 7fb6be454700 1 --2- 172.21.15.7:0/3051365858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6b806fa70 0x7fb6b8098d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:09.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.971+0000 7fb6bec55700 1 -- 172.21.15.7:0/3051365858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6b80a7c10 msgr2=0x7fb6b806aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:09.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.971+0000 7fb6bec55700 1 --2- 172.21.15.7:0/3051365858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6b80a7c10 0x7fb6b806aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:09.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.971+0000 7fb6bec55700 1 -- 172.21.15.7:0/3051365858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6b806fa70 msgr2=0x7fb6b8098d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:09.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.971+0000 7fb6bec55700 1 --2- 172.21.15.7:0/3051365858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6b806fa70 0x7fb6b8098d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:09.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.971+0000 7fb6bec55700 1 -- 172.21.15.7:0/3051365858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b0009a20 con 0x7fb6b80a7fe0 2024-01-27T02:32:09.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.971+0000 7fb6be454700 1 --2- 172.21.15.7:0/3051365858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6b806fa70 0x7fb6b8098d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:32:09.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.972+0000 7fb6bec55700 1 --2- 172.21.15.7:0/3051365858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6b80a7fe0 0x7fb6b814fb00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0003000 tx=0x7fb6b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:09.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.972+0000 7fb6ae7fc700 1 -- 172.21.15.7:0/3051365858 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0012d40 con 0x7fb6b80a7fe0 2024-01-27T02:32:09.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.972+0000 7fb6ae7fc700 1 -- 172.21.15.7:0/3051365858 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b001e450 con 0x7fb6b80a7fe0 2024-01-27T02:32:09.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.972+0000 7fb6ae7fc700 1 -- 172.21.15.7:0/3051365858 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0012ea0 con 0x7fb6b80a7fe0 2024-01-27T02:32:09.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.972+0000 7fb6c06b8700 1 -- 172.21.15.7:0/3051365858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b80993c0 con 0x7fb6b80a7fe0 2024-01-27T02:32:09.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.972+0000 7fb6c06b8700 1 -- 172.21.15.7:0/3051365858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b8099980 con 0x7fb6b80a7fe0 2024-01-27T02:32:09.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.973+0000 7fb6c06b8700 1 -- 172.21.15.7:0/3051365858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b809a160 con 0x7fb6b80a7fe0 2024-01-27T02:32:09.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.974+0000 7fb6ae7fc700 1 -- 172.21.15.7:0/3051365858 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb6b001eae0 con 0x7fb6b80a7fe0 2024-01-27T02:32:09.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.975+0000 7fb6ae7fc700 1 --2- 172.21.15.7:0/3051365858 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb69c066f30 0x7fb69c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:09.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.975+0000 7fb6ae7fc700 1 -- 172.21.15.7:0/3051365858 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb6b0086980 con 0x7fb6b80a7fe0 2024-01-27T02:32:09.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.975+0000 7fb6be454700 1 --2- 172.21.15.7:0/3051365858 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb69c066f30 0x7fb69c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:09.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.976+0000 7fb6be454700 1 --2- 172.21.15.7:0/3051365858 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb69c066f30 0x7fb69c0693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40015a0 tx=0x7fb6b4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:09.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:09.979+0000 7fb6ae7fc700 1 -- 172.21.15.7:0/3051365858 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6b0053b50 con 0x7fb6b80a7fe0 2024-01-27T02:32:10.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.228+0000 7fb6c06b8700 1 -- 172.21.15.7:0/3051365858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6b804c710 con 0x7fb6b80a7fe0 2024-01-27T02:32:10.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.228+0000 7fb6ae7fc700 1 -- 172.21.15.7:0/3051365858 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7fb6b0057170 con 0x7fb6b80a7fe0 2024-01-27T02:32:10.230 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:10.230 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3594367777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3594367777},{"type":"v1","addr":"172.21.15.7:6834","nonce":3594367777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1079101213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1079101213},{"type":"v1","addr":"172.21.15.188:6837","nonce":1079101213}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3241424244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3241424244},{"type":"v1","addr":"172.21.15.7:6835","nonce":3241424244}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:32:06.045693+0000","modified":"2024-01-27T02:32:09.806398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6836/2571237935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2571237935},{"type":"v1","addr":"172.21.15.188:6836","nonce":2571237935}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-27T02:32:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.230+0000 7fb6c06b8700 1 -- 172.21.15.7:0/3051365858 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb69c066f30 msgr2=0x7fb69c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.230+0000 7fb6c06b8700 1 --2- 172.21.15.7:0/3051365858 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb69c066f30 0x7fb69c0693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40015a0 tx=0x7fb6b4009040 comp rx=0 tx=0).stop 2024-01-27T02:32:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.230+0000 7fb6c06b8700 1 -- 172.21.15.7:0/3051365858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6b80a7fe0 msgr2=0x7fb6b814fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.230+0000 7fb6c06b8700 1 --2- 172.21.15.7:0/3051365858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6b80a7fe0 0x7fb6b814fb00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0003000 tx=0x7fb6b0017000 comp rx=0 tx=0).stop 2024-01-27T02:32:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.230+0000 7fb6c06b8700 1 -- 172.21.15.7:0/3051365858 shutdown_connections 2024-01-27T02:32:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.230+0000 7fb6c06b8700 1 --2- 172.21.15.7:0/3051365858 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb69c066f30 0x7fb69c0693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.230+0000 7fb6c06b8700 1 --2- 172.21.15.7:0/3051365858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6b80a7fe0 0x7fb6b814fb00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.230+0000 7fb6c06b8700 1 --2- 172.21.15.7:0/3051365858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6b806fa70 0x7fb6b8098d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.230+0000 7fb6c06b8700 1 --2- 172.21.15.7:0/3051365858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6b80a7c10 0x7fb6b806aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.230+0000 7fb6c06b8700 1 -- 172.21.15.7:0/3051365858 >> 172.21.15.7:0/3051365858 conn(0x7fb6b80679b0 msgr2=0x7fb6b8151a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:10.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.230+0000 7fb6c06b8700 1 -- 172.21.15.7:0/3051365858 shutdown_connections 2024-01-27T02:32:10.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.230+0000 7fb6c06b8700 1 -- 172.21.15.7:0/3051365858 wait complete. 2024-01-27T02:32:10.233 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-01-27T02:32:10.246 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:32:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.376+0000 7f3623afc700 1 Processor -- start 2024-01-27T02:32:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.376+0000 7f3623afc700 1 -- start start 2024-01-27T02:32:10.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.377+0000 7f3623afc700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f361c157ca0 0x7f361c0c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:10.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.377+0000 7f3623afc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f361c0cdbf0 0x7f361c0ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:10.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.377+0000 7f3623afc700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f361c0ca020 con 0x7f361c0c3ae0 2024-01-27T02:32:10.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.377+0000 7f3623afc700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f361c0c4f40 con 0x7f361c157ca0 2024-01-27T02:32:10.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.377+0000 7f3623afc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c0c5210 con 0x7f361c0cdbf0 2024-01-27T02:32:10.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.377+0000 7f3622099700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f361c0cdbf0 0x7f361c0ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:10.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.377+0000 7f3621898700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f361c157ca0 0x7f361c0c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:10.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.377+0000 7f3621097700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f361c0c3ae0 0x7f361c0cd4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:34056/0 (socket says 172.21.15.7:34056) 2024-01-27T02:32:10.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.377+0000 7f3622099700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f361c0cdbf0 0x7f361c0ca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:34342/0 (socket says 172.21.15.7:34342) 2024-01-27T02:32:10.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.377+0000 7f3621097700 1 -- 172.21.15.7:0/2457204214 learned_addr learned my addr 172.21.15.7:0/2457204214 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:10.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.378+0000 7f3621898700 1 -- 172.21.15.7:0/2457204214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f361c0cdbf0 msgr2=0x7f361c0ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.378+0000 7f3621898700 1 --2- 172.21.15.7:0/2457204214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f361c0cdbf0 0x7f361c0ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.378+0000 7f3621898700 1 -- 172.21.15.7:0/2457204214 >> v1:172.21.15.188:6789/0 conn(0x7f361c0c3ae0 legacy=0x7f361c0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.378+0000 7f3621898700 1 -- 172.21.15.7:0/2457204214 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c0c54f0 con 0x7f361c157ca0 2024-01-27T02:32:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.378+0000 7f3620896700 1 -- 172.21.15.7:0/2457204214 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 452014436 0 0) 0x7f361c0ca020 con 0x7f361c0c3ae0 2024-01-27T02:32:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.378+0000 7f3622099700 1 --2- 172.21.15.7:0/2457204214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f361c0cdbf0 0x7f361c0ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:32:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.378+0000 7f3621898700 1 --2- 172.21.15.7:0/2457204214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f361c157ca0 0x7f361c0c4a00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f36180098e0 tx=0x7f36180091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=308ff737b51f91a7 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.378+0000 7f3620896700 1 -- 172.21.15.7:0/2457204214 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361800b640 con 0x7f361c157ca0 2024-01-27T02:32:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.378+0000 7f3620896700 1 -- 172.21.15.7:0/2457204214 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361800b7a0 con 0x7f361c157ca0 2024-01-27T02:32:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.378+0000 7f3620896700 1 -- 172.21.15.7:0/2457204214 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361800bb40 con 0x7f361c157ca0 2024-01-27T02:32:10.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.379+0000 7f3623afc700 1 -- 172.21.15.7:0/2457204214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f361c157ca0 msgr2=0x7f361c0c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:10.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.379+0000 7f3623afc700 1 --2- 172.21.15.7:0/2457204214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f361c157ca0 0x7f361c0c4a00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f36180098e0 tx=0x7f36180091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.379+0000 7f3623afc700 1 -- 172.21.15.7:0/2457204214 shutdown_connections 2024-01-27T02:32:10.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.379+0000 7f3623afc700 1 --2- 172.21.15.7:0/2457204214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f361c0cdbf0 0x7f361c0ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.379+0000 7f3623afc700 1 --2- 172.21.15.7:0/2457204214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f361c157ca0 0x7f361c0c4a00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.379+0000 7f3623afc700 1 -- 172.21.15.7:0/2457204214 >> 172.21.15.7:0/2457204214 conn(0x7f361c098020 msgr2=0x7f361c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:10.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.379+0000 7f3623afc700 1 -- 172.21.15.7:0/2457204214 shutdown_connections 2024-01-27T02:32:10.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.379+0000 7f3623afc700 1 -- 172.21.15.7:0/2457204214 wait complete. 2024-01-27T02:32:10.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.380+0000 7f3623afc700 1 Processor -- start 2024-01-27T02:32:10.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.380+0000 7f3623afc700 1 -- start start 2024-01-27T02:32:10.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.381+0000 7f3623afc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f361c0c3ae0 0x7f361c0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:10.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.381+0000 7f3623afc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f361c0cdbf0 0x7f361c142700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:10.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.381+0000 7f3623afc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f361c157ca0 0x7f361c15ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:10.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.381+0000 7f3623afc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c15da40 con 0x7f361c0c3ae0 2024-01-27T02:32:10.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.381+0000 7f3623afc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c0b48f0 con 0x7f361c157ca0 2024-01-27T02:32:10.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.381+0000 7f3623afc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c15d710 con 0x7f361c0cdbf0 2024-01-27T02:32:10.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.381+0000 7f3621898700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f361c0c3ae0 0x7f361c0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:10.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.381+0000 7f3621898700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f361c0c3ae0 0x7f361c0b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41050/0 (socket says 172.21.15.7:41050) 2024-01-27T02:32:10.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.381+0000 7f3621898700 1 -- 172.21.15.7:0/1403847083 learned_addr learned my addr 172.21.15.7:0/1403847083 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:10.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.381+0000 7f3621097700 1 --2- 172.21.15.7:0/1403847083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f361c0cdbf0 0x7f361c142700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:10.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.381+0000 7f3622099700 1 --2- 172.21.15.7:0/1403847083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f361c157ca0 0x7f361c15ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:10.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.382+0000 7f3621898700 1 -- 172.21.15.7:0/1403847083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f361c0cdbf0 msgr2=0x7f361c142700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:10.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.382+0000 7f3621898700 1 --2- 172.21.15.7:0/1403847083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f361c0cdbf0 0x7f361c142700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.382+0000 7f3621898700 1 -- 172.21.15.7:0/1403847083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f361c157ca0 msgr2=0x7f361c15ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:10.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.382+0000 7f3621898700 1 --2- 172.21.15.7:0/1403847083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f361c157ca0 0x7f361c15ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.382+0000 7f3621898700 1 -- 172.21.15.7:0/1403847083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3614009a20 con 0x7f361c0c3ae0 2024-01-27T02:32:10.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.382+0000 7f3621898700 1 --2- 172.21.15.7:0/1403847083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f361c0c3ae0 0x7f361c0b43b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f36180093f0 tx=0x7f3618017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:10.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.382+0000 7f360dffb700 1 -- 172.21.15.7:0/1403847083 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361800bb20 con 0x7f361c0c3ae0 2024-01-27T02:32:10.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.382+0000 7f360dffb700 1 -- 172.21.15.7:0/1403847083 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3618011420 con 0x7f361c0c3ae0 2024-01-27T02:32:10.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.382+0000 7f3623afc700 1 -- 172.21.15.7:0/1403847083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3618009420 con 0x7f361c0c3ae0 2024-01-27T02:32:10.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.382+0000 7f360dffb700 1 -- 172.21.15.7:0/1403847083 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618011820 con 0x7f361c0c3ae0 2024-01-27T02:32:10.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.382+0000 7f3623afc700 1 -- 172.21.15.7:0/1403847083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f361c0b2bc0 con 0x7f361c0c3ae0 2024-01-27T02:32:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.383+0000 7f3623afc700 1 -- 172.21.15.7:0/1403847083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f361c15d580 con 0x7f361c0c3ae0 2024-01-27T02:32:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.384+0000 7f360dffb700 1 -- 172.21.15.7:0/1403847083 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3618011c50 con 0x7f361c0c3ae0 2024-01-27T02:32:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.384+0000 7f360dffb700 1 --2- 172.21.15.7:0/1403847083 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3600066f30 0x7f36000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.384+0000 7f360dffb700 1 -- 172.21.15.7:0/1403847083 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3618085cc0 con 0x7f361c0c3ae0 2024-01-27T02:32:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.384+0000 7f3621097700 1 --2- 172.21.15.7:0/1403847083 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3600066f30 0x7f36000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.385+0000 7f3621097700 1 --2- 172.21.15.7:0/1403847083 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3600066f30 0x7f36000693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f361c0b2760 tx=0x7f3610007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:10.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.388+0000 7f360dffb700 1 -- 172.21.15.7:0/1403847083 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3618052ff0 con 0x7f361c0c3ae0 2024-01-27T02:32:10.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.592+0000 7f3623afc700 1 -- 172.21.15.7:0/1403847083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f361c04c710 con 0x7f361c0c3ae0 2024-01-27T02:32:10.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.593+0000 7f360dffb700 1 -- 172.21.15.7:0/1403847083 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f3618056610 con 0x7f361c0c3ae0 2024-01-27T02:32:10.595 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:10.595 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:32:07.627822+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:32:03.675611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-27T02:32:05.018885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:32:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.595+0000 7f3623afc700 1 -- 172.21.15.7:0/1403847083 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3600066f30 msgr2=0x7f36000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.595+0000 7f3623afc700 1 --2- 172.21.15.7:0/1403847083 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3600066f30 0x7f36000693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f361c0b2760 tx=0x7f3610007d30 comp rx=0 tx=0).stop 2024-01-27T02:32:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.595+0000 7f3623afc700 1 -- 172.21.15.7:0/1403847083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f361c0c3ae0 msgr2=0x7f361c0b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.595+0000 7f3623afc700 1 --2- 172.21.15.7:0/1403847083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f361c0c3ae0 0x7f361c0b43b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f36180093f0 tx=0x7f3618017000 comp rx=0 tx=0).stop 2024-01-27T02:32:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.595+0000 7f3623afc700 1 -- 172.21.15.7:0/1403847083 shutdown_connections 2024-01-27T02:32:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.595+0000 7f3623afc700 1 --2- 172.21.15.7:0/1403847083 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3600066f30 0x7f36000693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.595+0000 7f3623afc700 1 --2- 172.21.15.7:0/1403847083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f361c157ca0 0x7f361c15ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.595+0000 7f3623afc700 1 --2- 172.21.15.7:0/1403847083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f361c0c3ae0 0x7f361c0b43b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.595+0000 7f3623afc700 1 --2- 172.21.15.7:0/1403847083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f361c0cdbf0 0x7f361c142700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.595+0000 7f3623afc700 1 -- 172.21.15.7:0/1403847083 >> 172.21.15.7:0/1403847083 conn(0x7f361c098020 msgr2=0x7f361c06cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.596+0000 7f3623afc700 1 -- 172.21.15.7:0/1403847083 shutdown_connections 2024-01-27T02:32:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.596+0000 7f3623afc700 1 -- 172.21.15.7:0/1403847083 wait complete. 2024-01-27T02:32:10.610 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-27T02:32:10.611 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:32:10.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.741+0000 7f822a496700 1 Processor -- start 2024-01-27T02:32:10.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.741+0000 7f822a496700 1 -- start start 2024-01-27T02:32:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.742+0000 7f822a496700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f822415a100 0x7f82240cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.742+0000 7f822a496700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f82240c4cc0 0x7f8224163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.742+0000 7f822a496700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82240ce5e0 con 0x7f82240cb5b0 2024-01-27T02:32:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.742+0000 7f822a496700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82240cd9d0 con 0x7f82240c4cc0 2024-01-27T02:32:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.742+0000 7f822a496700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82240cdcb0 con 0x7f822415a100 2024-01-27T02:32:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.742+0000 7f8228a33700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f82240c4cc0 0x7f8224163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.742+0000 7f8228a33700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f82240c4cc0 0x7f8224163d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41060/0 (socket says 172.21.15.7:41060) 2024-01-27T02:32:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.742+0000 7f8228a33700 1 -- 172.21.15.7:0/3328809345 learned_addr learned my addr 172.21.15.7:0/3328809345 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.742+0000 7f8223fff700 1 --2- 172.21.15.7:0/3328809345 >> v2:172.21.15.188:3300/0 conn(0x7f822415a100 0x7f82240cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.742+0000 7f8228a33700 1 -- 172.21.15.7:0/3328809345 >> v2:172.21.15.188:3300/0 conn(0x7f822415a100 msgr2=0x7f82240cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.742+0000 7f8228a33700 1 --2- 172.21.15.7:0/3328809345 >> v2:172.21.15.188:3300/0 conn(0x7f822415a100 0x7f82240cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.742+0000 7f8228a33700 1 -- 172.21.15.7:0/3328809345 >> v1:172.21.15.188:6789/0 conn(0x7f82240cb5b0 legacy=0x7f82240c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.742+0000 7f8228a33700 1 -- 172.21.15.7:0/3328809345 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82240cdf90 con 0x7f82240c4cc0 2024-01-27T02:32:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.742+0000 7f8222ffd700 1 -- 172.21.15.7:0/3328809345 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3117529777 0 0) 0x7f82240ce5e0 con 0x7f82240cb5b0 2024-01-27T02:32:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.742+0000 7f8228a33700 1 --2- 172.21.15.7:0/3328809345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82240c4cc0 0x7f8224163d90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f822406d710 tx=0x7f821c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895fc2713630601e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.743+0000 7f8222ffd700 1 -- 172.21.15.7:0/3328809345 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c00e070 con 0x7f82240c4cc0 2024-01-27T02:32:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.743+0000 7f8222ffd700 1 -- 172.21.15.7:0/3328809345 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821c00ba20 con 0x7f82240c4cc0 2024-01-27T02:32:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.743+0000 7f8222ffd700 1 -- 172.21.15.7:0/3328809345 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c0092e0 con 0x7f82240c4cc0 2024-01-27T02:32:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.743+0000 7f822a496700 1 -- 172.21.15.7:0/3328809345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82240c4cc0 msgr2=0x7f8224163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.743+0000 7f822a496700 1 --2- 172.21.15.7:0/3328809345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82240c4cc0 0x7f8224163d90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f822406d710 tx=0x7f821c009840 comp rx=0 tx=0).stop 2024-01-27T02:32:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.743+0000 7f822a496700 1 -- 172.21.15.7:0/3328809345 shutdown_connections 2024-01-27T02:32:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.743+0000 7f822a496700 1 --2- 172.21.15.7:0/3328809345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82240c4cc0 0x7f8224163d90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.743+0000 7f822a496700 1 --2- 172.21.15.7:0/3328809345 >> v2:172.21.15.188:3300/0 conn(0x7f822415a100 0x7f82240cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.743+0000 7f822a496700 1 -- 172.21.15.7:0/3328809345 >> 172.21.15.7:0/3328809345 conn(0x7f8224098020 msgr2=0x7f82240711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:10.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.743+0000 7f822a496700 1 -- 172.21.15.7:0/3328809345 shutdown_connections 2024-01-27T02:32:10.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.743+0000 7f822a496700 1 -- 172.21.15.7:0/3328809345 wait complete. 2024-01-27T02:32:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.744+0000 7f822a496700 1 Processor -- start 2024-01-27T02:32:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.745+0000 7f822a496700 1 -- start start 2024-01-27T02:32:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.745+0000 7f822a496700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82240c4cc0 0x7f822416a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.745+0000 7f822a496700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82240cb5b0 0x7f822415c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.745+0000 7f822a496700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822415a100 0x7f8224149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.745+0000 7f822a496700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82240c7cf0 con 0x7f82240c4cc0 2024-01-27T02:32:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.745+0000 7f822a496700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822416a6b0 con 0x7f822415a100 2024-01-27T02:32:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.745+0000 7f822a496700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82240c5df0 con 0x7f82240cb5b0 2024-01-27T02:32:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.746+0000 7f8223fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82240c4cc0 0x7f822416a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.746+0000 7f8223fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82240c4cc0 0x7f822416a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41070/0 (socket says 172.21.15.7:41070) 2024-01-27T02:32:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.746+0000 7f8223fff700 1 -- 172.21.15.7:0/736380693 learned_addr learned my addr 172.21.15.7:0/736380693 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.746+0000 7f82237fe700 1 --2- 172.21.15.7:0/736380693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82240cb5b0 0x7f822415c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.746+0000 7f8228a33700 1 --2- 172.21.15.7:0/736380693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822415a100 0x7f8224149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.746+0000 7f8223fff700 1 -- 172.21.15.7:0/736380693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82240cb5b0 msgr2=0x7f822415c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.746+0000 7f8223fff700 1 --2- 172.21.15.7:0/736380693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82240cb5b0 0x7f822415c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.746+0000 7f8223fff700 1 -- 172.21.15.7:0/736380693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822415a100 msgr2=0x7f8224149aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.746+0000 7f8223fff700 1 --2- 172.21.15.7:0/736380693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822415a100 0x7f8224149aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.746+0000 7f8223fff700 1 -- 172.21.15.7:0/736380693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c009ae0 con 0x7f82240c4cc0 2024-01-27T02:32:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.746+0000 7f82237fe700 1 --2- 172.21.15.7:0/736380693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82240cb5b0 0x7f822415c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.746+0000 7f8228a33700 1 --2- 172.21.15.7:0/736380693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822415a100 0x7f8224149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.746+0000 7f8223fff700 1 --2- 172.21.15.7:0/736380693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82240c4cc0 0x7f822416a170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f821400e930 tx=0x7f82140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.746+0000 7f820bfff700 1 -- 172.21.15.7:0/736380693 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821400eed0 con 0x7f82240c4cc0 2024-01-27T02:32:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.746+0000 7f820bfff700 1 -- 172.21.15.7:0/736380693 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821400c850 con 0x7f82240c4cc0 2024-01-27T02:32:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.746+0000 7f822a496700 1 -- 172.21.15.7:0/736380693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82240c6c50 con 0x7f82240c4cc0 2024-01-27T02:32:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.746+0000 7f820bfff700 1 -- 172.21.15.7:0/736380693 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821400cbf0 con 0x7f82240c4cc0 2024-01-27T02:32:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.746+0000 7f822a496700 1 -- 172.21.15.7:0/736380693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82240c6600 con 0x7f82240c4cc0 2024-01-27T02:32:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.747+0000 7f822a496700 1 -- 172.21.15.7:0/736380693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f822404c710 con 0x7f82240c4cc0 2024-01-27T02:32:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.748+0000 7f820bfff700 1 -- 172.21.15.7:0/736380693 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8214013470 con 0x7f82240c4cc0 2024-01-27T02:32:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.748+0000 7f820bfff700 1 --2- 172.21.15.7:0/736380693 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8210066f30 0x7f82100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:10.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.748+0000 7f820bfff700 1 -- 172.21.15.7:0/736380693 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8214084760 con 0x7f82240c4cc0 2024-01-27T02:32:10.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.748+0000 7f82237fe700 1 --2- 172.21.15.7:0/736380693 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8210066f30 0x7f82100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:10.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.749+0000 7f82237fe700 1 --2- 172.21.15.7:0/736380693 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8210066f30 0x7f82100693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f82240b41c0 tx=0x7f8218009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:10.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.752+0000 7f820bfff700 1 -- 172.21.15.7:0/736380693 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82140519d0 con 0x7f82240c4cc0 2024-01-27T02:32:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.953+0000 7f822a496700 1 -- 172.21.15.7:0/736380693 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f82240ce270 con 0x7f8210066f30 2024-01-27T02:32:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.961+0000 7f820bfff700 1 -- 172.21.15.7:0/736380693 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+257304 (secure 0 0 0) 0x7f82240ce270 con 0x7f8210066f30 2024-01-27T02:32:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.964+0000 7f822a496700 1 -- 172.21.15.7:0/736380693 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8210066f30 msgr2=0x7f82100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.964+0000 7f822a496700 1 --2- 172.21.15.7:0/736380693 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8210066f30 0x7f82100693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f82240b41c0 tx=0x7f8218009040 comp rx=0 tx=0).stop 2024-01-27T02:32:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.964+0000 7f822a496700 1 -- 172.21.15.7:0/736380693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82240c4cc0 msgr2=0x7f822416a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.964+0000 7f822a496700 1 --2- 172.21.15.7:0/736380693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82240c4cc0 0x7f822416a170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f821400e930 tx=0x7f82140091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.964+0000 7f822a496700 1 -- 172.21.15.7:0/736380693 shutdown_connections 2024-01-27T02:32:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.964+0000 7f822a496700 1 --2- 172.21.15.7:0/736380693 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8210066f30 0x7f82100693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.964+0000 7f822a496700 1 --2- 172.21.15.7:0/736380693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822415a100 0x7f8224149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.964+0000 7f822a496700 1 --2- 172.21.15.7:0/736380693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82240c4cc0 0x7f822416a170 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.964+0000 7f822a496700 1 --2- 172.21.15.7:0/736380693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82240cb5b0 0x7f822415c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.964+0000 7f822a496700 1 -- 172.21.15.7:0/736380693 >> 172.21.15.7:0/736380693 conn(0x7f8224098020 msgr2=0x7f82240c9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.964+0000 7f822a496700 1 -- 172.21.15.7:0/736380693 shutdown_connections 2024-01-27T02:32:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:10.964+0000 7f822a496700 1 -- 172.21.15.7:0/736380693 wait complete. 2024-01-27T02:32:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:32:10.980 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-01-27T02:32:09.572481+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1632,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560867,"statfs":{"total":1600000000,"available":1598328711,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.964513"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616502+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:44:13.569068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617845+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617854+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614017+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.619412+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:32:24.198014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.618664+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:45:37.035302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616761+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:17:03.351766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.622881+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615495+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614084+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615294+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612594+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:08.590979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.621489+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615194+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:08.590979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617367+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616115+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:06.985231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615481+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612774+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617263+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:36:11.295485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.624071+0000","last_change":"2024-01-27T02:32:05.624071+0000","last_active":"2024-01-27T02:32:05.624071+0000","last_peered":"2024-01-27T02:32:05.624071+0000","last_clean":"2024-01-27T02:32:05.624071+0000","last_became_active":"2024-01-27T02:32:05.623818+0000","last_became_peered":"2024-01-27T02:32:05.623818+0000","last_unstale":"2024-01-27T02:32:05.624071+0000","last_undegraded":"2024-01-27T02:32:05.624071+0000","last_fullsized":"2024-01-27T02:32:05.624071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:34:05.180050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.622620+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.624866+0000","last_change":"2024-01-27T02:32:05.624866+0000","last_active":"2024-01-27T02:32:05.624866+0000","last_peered":"2024-01-27T02:32:05.624866+0000","last_clean":"2024-01-27T02:32:05.624866+0000","last_became_active":"2024-01-27T02:32:05.624368+0000","last_became_peered":"2024-01-27T02:32:05.624368+0000","last_unstale":"2024-01-27T02:32:05.624866+0000","last_undegraded":"2024-01-27T02:32:05.624866+0000","last_fullsized":"2024-01-27T02:32:05.624866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617401+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.623255+0000","last_change":"2024-01-27T02:32:05.623255+0000","last_active":"2024-01-27T02:32:05.623255+0000","last_peered":"2024-01-27T02:32:05.623255+0000","last_clean":"2024-01-27T02:32:05.623255+0000","last_became_active":"2024-01-27T02:32:05.622821+0000","last_became_peered":"2024-01-27T02:32:05.622821+0000","last_unstale":"2024-01-27T02:32:05.623255+0000","last_undegraded":"2024-01-27T02:32:05.623255+0000","last_fullsized":"2024-01-27T02:32:05.623255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:33.333681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615102+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.626014+0000","last_change":"2024-01-27T02:32:05.626014+0000","last_active":"2024-01-27T02:32:05.626014+0000","last_peered":"2024-01-27T02:32:05.626014+0000","last_clean":"2024-01-27T02:32:05.626014+0000","last_became_active":"2024-01-27T02:32:05.623176+0000","last_became_peered":"2024-01-27T02:32:05.623176+0000","last_unstale":"2024-01-27T02:32:05.626014+0000","last_undegraded":"2024-01-27T02:32:05.626014+0000","last_fullsized":"2024-01-27T02:32:05.626014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:37:59.246901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616282+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.622086+0000","last_change":"2024-01-27T02:32:05.622086+0000","last_active":"2024-01-27T02:32:05.622086+0000","last_peered":"2024-01-27T02:32:05.622086+0000","last_clean":"2024-01-27T02:32:05.622086+0000","last_became_active":"2024-01-27T02:32:05.621465+0000","last_became_peered":"2024-01-27T02:32:05.621465+0000","last_unstale":"2024-01-27T02:32:05.622086+0000","last_undegraded":"2024-01-27T02:32:05.622086+0000","last_fullsized":"2024-01-27T02:32:05.622086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:04:28.890844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612345+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:42.580188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.617530+0000","last_change":"2024-01-27T02:32:05.617530+0000","last_active":"2024-01-27T02:32:05.617530+0000","last_peered":"2024-01-27T02:32:05.617530+0000","last_clean":"2024-01-27T02:32:05.617530+0000","last_became_active":"2024-01-27T02:32:05.617290+0000","last_became_peered":"2024-01-27T02:32:05.617290+0000","last_unstale":"2024-01-27T02:32:05.617530+0000","last_undegraded":"2024-01-27T02:32:05.617530+0000","last_fullsized":"2024-01-27T02:32:05.617530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:41.575605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614574+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:53.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.623693+0000","last_change":"2024-01-27T02:32:05.623693+0000","last_active":"2024-01-27T02:32:05.623693+0000","last_peered":"2024-01-27T02:32:05.623693+0000","last_clean":"2024-01-27T02:32:05.623693+0000","last_became_active":"2024-01-27T02:32:05.623440+0000","last_became_peered":"2024-01-27T02:32:05.623440+0000","last_unstale":"2024-01-27T02:32:05.623693+0000","last_undegraded":"2024-01-27T02:32:05.623693+0000","last_fullsized":"2024-01-27T02:32:05.623693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:44:52.929407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.619517+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.619476+0000","last_change":"2024-01-27T02:32:05.619476+0000","last_active":"2024-01-27T02:32:05.619476+0000","last_peered":"2024-01-27T02:32:05.619476+0000","last_clean":"2024-01-27T02:32:05.619476+0000","last_became_active":"2024-01-27T02:32:05.619194+0000","last_became_peered":"2024-01-27T02:32:05.619194+0000","last_unstale":"2024-01-27T02:32:05.619476+0000","last_undegraded":"2024-01-27T02:32:05.619476+0000","last_fullsized":"2024-01-27T02:32:05.619476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615596+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.623827+0000","last_change":"2024-01-27T02:32:05.623827+0000","last_active":"2024-01-27T02:32:05.623827+0000","last_peered":"2024-01-27T02:32:05.623827+0000","last_clean":"2024-01-27T02:32:05.623827+0000","last_became_active":"2024-01-27T02:32:05.623602+0000","last_became_peered":"2024-01-27T02:32:05.623602+0000","last_unstale":"2024-01-27T02:32:05.623827+0000","last_undegraded":"2024-01-27T02:32:05.623827+0000","last_fullsized":"2024-01-27T02:32:05.623827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613756+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.621283+0000","last_change":"2024-01-27T02:32:05.621283+0000","last_active":"2024-01-27T02:32:05.621283+0000","last_peered":"2024-01-27T02:32:05.621283+0000","last_clean":"2024-01-27T02:32:05.621283+0000","last_became_active":"2024-01-27T02:32:05.621034+0000","last_became_peered":"2024-01-27T02:32:05.621034+0000","last_unstale":"2024-01-27T02:32:05.621283+0000","last_undegraded":"2024-01-27T02:32:05.621283+0000","last_fullsized":"2024-01-27T02:32:05.621283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:07.586396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614840+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:42.580188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.620524+0000","last_change":"2024-01-27T02:32:05.620524+0000","last_active":"2024-01-27T02:32:05.620524+0000","last_peered":"2024-01-27T02:32:05.620524+0000","last_clean":"2024-01-27T02:32:05.620524+0000","last_became_active":"2024-01-27T02:32:05.620169+0000","last_became_peered":"2024-01-27T02:32:05.620169+0000","last_unstale":"2024-01-27T02:32:05.620524+0000","last_undegraded":"2024-01-27T02:32:05.620524+0000","last_fullsized":"2024-01-27T02:32:05.620524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612315+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:53.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.625720+0000","last_change":"2024-01-27T02:32:05.625720+0000","last_active":"2024-01-27T02:32:05.625720+0000","last_peered":"2024-01-27T02:32:05.625720+0000","last_clean":"2024-01-27T02:32:05.625720+0000","last_became_active":"2024-01-27T02:32:05.625394+0000","last_became_peered":"2024-01-27T02:32:05.625394+0000","last_unstale":"2024-01-27T02:32:05.625720+0000","last_undegraded":"2024-01-27T02:32:05.625720+0000","last_fullsized":"2024-01-27T02:32:05.625720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.621410+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:16:23.225540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.619745+0000","last_change":"2024-01-27T02:32:05.619745+0000","last_active":"2024-01-27T02:32:05.619745+0000","last_peered":"2024-01-27T02:32:05.619745+0000","last_clean":"2024-01-27T02:32:05.619745+0000","last_became_active":"2024-01-27T02:32:05.619371+0000","last_became_peered":"2024-01-27T02:32:05.619371+0000","last_unstale":"2024-01-27T02:32:05.619745+0000","last_undegraded":"2024-01-27T02:32:05.619745+0000","last_fullsized":"2024-01-27T02:32:05.619745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:12:20.892542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614205+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:04:29.895427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.626895+0000","last_change":"2024-01-27T02:32:05.626895+0000","last_active":"2024-01-27T02:32:05.626895+0000","last_peered":"2024-01-27T02:32:05.626895+0000","last_clean":"2024-01-27T02:32:05.626895+0000","last_became_active":"2024-01-27T02:32:05.626682+0000","last_became_peered":"2024-01-27T02:32:05.626682+0000","last_unstale":"2024-01-27T02:32:05.626895+0000","last_undegraded":"2024-01-27T02:32:05.626895+0000","last_fullsized":"2024-01-27T02:32:05.626895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:41.575605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.618952+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:11:54.230178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.619939+0000","last_change":"2024-01-27T02:32:05.619939+0000","last_active":"2024-01-27T02:32:05.619939+0000","last_peered":"2024-01-27T02:32:05.619939+0000","last_clean":"2024-01-27T02:32:05.619939+0000","last_became_active":"2024-01-27T02:32:05.619489+0000","last_became_peered":"2024-01-27T02:32:05.619489+0000","last_unstale":"2024-01-27T02:32:05.619939+0000","last_undegraded":"2024-01-27T02:32:05.619939+0000","last_fullsized":"2024-01-27T02:32:05.619939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:07.586396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615600+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:58:54.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.621851+0000","last_change":"2024-01-27T02:32:05.621851+0000","last_active":"2024-01-27T02:32:05.621851+0000","last_peered":"2024-01-27T02:32:05.621851+0000","last_clean":"2024-01-27T02:32:05.621851+0000","last_became_active":"2024-01-27T02:32:05.621487+0000","last_became_peered":"2024-01-27T02:32:05.621487+0000","last_unstale":"2024-01-27T02:32:05.621851+0000","last_undegraded":"2024-01-27T02:32:05.621851+0000","last_fullsized":"2024-01-27T02:32:05.621851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:07.586396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616380+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:06.985231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.623147+0000","last_change":"2024-01-27T02:32:05.623147+0000","last_active":"2024-01-27T02:32:05.623147+0000","last_peered":"2024-01-27T02:32:05.623147+0000","last_clean":"2024-01-27T02:32:05.623147+0000","last_became_active":"2024-01-27T02:32:05.622650+0000","last_became_peered":"2024-01-27T02:32:05.622650+0000","last_unstale":"2024-01-27T02:32:05.623147+0000","last_undegraded":"2024-01-27T02:32:05.623147+0000","last_fullsized":"2024-01-27T02:32:05.623147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:30:53.128829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615959+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.621921+0000","last_change":"2024-01-27T02:32:05.621921+0000","last_active":"2024-01-27T02:32:05.621921+0000","last_peered":"2024-01-27T02:32:05.621921+0000","last_clean":"2024-01-27T02:32:05.621921+0000","last_became_active":"2024-01-27T02:32:05.620450+0000","last_became_peered":"2024-01-27T02:32:05.620450+0000","last_unstale":"2024-01-27T02:32:05.621921+0000","last_undegraded":"2024-01-27T02:32:05.621921+0000","last_fullsized":"2024-01-27T02:32:05.621921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:33.333681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613665+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.622783+0000","last_change":"2024-01-27T02:32:05.622783+0000","last_active":"2024-01-27T02:32:05.622783+0000","last_peered":"2024-01-27T02:32:05.622783+0000","last_clean":"2024-01-27T02:32:05.622783+0000","last_became_active":"2024-01-27T02:32:05.622557+0000","last_became_peered":"2024-01-27T02:32:05.622557+0000","last_unstale":"2024-01-27T02:32:05.622783+0000","last_undegraded":"2024-01-27T02:32:05.622783+0000","last_fullsized":"2024-01-27T02:32:05.622783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612935+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:04:29.895427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615166+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.613447+0000","last_change":"2024-01-27T02:32:05.613447+0000","last_active":"2024-01-27T02:32:05.613447+0000","last_peered":"2024-01-27T02:32:05.613447+0000","last_clean":"2024-01-27T02:32:05.613447+0000","last_became_active":"2024-01-27T02:32:05.613203+0000","last_became_peered":"2024-01-27T02:32:05.613203+0000","last_unstale":"2024-01-27T02:32:05.613447+0000","last_undegraded":"2024-01-27T02:32:05.613447+0000","last_fullsized":"2024-01-27T02:32:05.613447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:41.575605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616703+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:47.819053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.622673+0000","last_change":"2024-01-27T02:32:05.622673+0000","last_active":"2024-01-27T02:32:05.622673+0000","last_peered":"2024-01-27T02:32:05.622673+0000","last_clean":"2024-01-27T02:32:05.622673+0000","last_became_active":"2024-01-27T02:32:05.622195+0000","last_became_peered":"2024-01-27T02:32:05.622195+0000","last_unstale":"2024-01-27T02:32:05.622673+0000","last_undegraded":"2024-01-27T02:32:05.622673+0000","last_fullsized":"2024-01-27T02:32:05.622673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":161,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-01-27T02:32:04.612144+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:32:04.612144+0000","last_peered":"2024-01-27T02:32:04.612144+0000","last_clean":"2024-01-27T02:32:04.612144+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:32:04.612144+0000","last_undegraded":"2024-01-27T02:32:04.612144+0000","last_fullsized":"2024-01-27T02:32:04.612144+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616221+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.626552+0000","last_change":"2024-01-27T02:32:05.626552+0000","last_active":"2024-01-27T02:32:05.626552+0000","last_peered":"2024-01-27T02:32:05.626552+0000","last_clean":"2024-01-27T02:32:05.626552+0000","last_became_active":"2024-01-27T02:32:05.626348+0000","last_became_peered":"2024-01-27T02:32:05.626348+0000","last_unstale":"2024-01-27T02:32:05.626552+0000","last_undegraded":"2024-01-27T02:32:05.626552+0000","last_fullsized":"2024-01-27T02:32:05.626552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:18.034484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.625885+0000","last_change":"2024-01-27T02:32:05.625885+0000","last_active":"2024-01-27T02:32:05.625885+0000","last_peered":"2024-01-27T02:32:05.625885+0000","last_clean":"2024-01-27T02:32:05.625885+0000","last_became_active":"2024-01-27T02:32:05.622692+0000","last_became_peered":"2024-01-27T02:32:05.622692+0000","last_unstale":"2024-01-27T02:32:05.625885+0000","last_undegraded":"2024-01-27T02:32:05.625885+0000","last_fullsized":"2024-01-27T02:32:05.625885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:06.920232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616950+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:58:54.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.621014+0000","last_change":"2024-01-27T02:32:05.621014+0000","last_active":"2024-01-27T02:32:05.621014+0000","last_peered":"2024-01-27T02:32:05.621014+0000","last_clean":"2024-01-27T02:32:05.621014+0000","last_became_active":"2024-01-27T02:32:05.620775+0000","last_became_peered":"2024-01-27T02:32:05.620775+0000","last_unstale":"2024-01-27T02:32:05.621014+0000","last_undegraded":"2024-01-27T02:32:05.621014+0000","last_fullsized":"2024-01-27T02:32:05.621014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617909+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.624870+0000","last_change":"2024-01-27T02:32:05.624870+0000","last_active":"2024-01-27T02:32:05.624870+0000","last_peered":"2024-01-27T02:32:05.624870+0000","last_clean":"2024-01-27T02:32:05.624870+0000","last_became_active":"2024-01-27T02:32:05.624644+0000","last_became_peered":"2024-01-27T02:32:05.624644+0000","last_unstale":"2024-01-27T02:32:05.624870+0000","last_undegraded":"2024-01-27T02:32:05.624870+0000","last_fullsized":"2024-01-27T02:32:05.624870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616980+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.621287+0000","last_change":"2024-01-27T02:32:05.621287+0000","last_active":"2024-01-27T02:32:05.621287+0000","last_peered":"2024-01-27T02:32:05.621287+0000","last_clean":"2024-01-27T02:32:05.621287+0000","last_became_active":"2024-01-27T02:32:05.620002+0000","last_became_peered":"2024-01-27T02:32:05.620002+0000","last_unstale":"2024-01-27T02:32:05.621287+0000","last_undegraded":"2024-01-27T02:32:05.621287+0000","last_fullsized":"2024-01-27T02:32:05.621287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:44.489465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612241+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:44:53.933990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.623449+0000","last_change":"2024-01-27T02:32:05.623449+0000","last_active":"2024-01-27T02:32:05.623449+0000","last_peered":"2024-01-27T02:32:05.623449+0000","last_clean":"2024-01-27T02:32:05.623449+0000","last_became_active":"2024-01-27T02:32:05.623193+0000","last_became_peered":"2024-01-27T02:32:05.623193+0000","last_unstale":"2024-01-27T02:32:05.623449+0000","last_undegraded":"2024-01-27T02:32:05.623449+0000","last_fullsized":"2024-01-27T02:32:05.623449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:44.489465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616309+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:08.590979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.625193+0000","last_change":"2024-01-27T02:32:05.625193+0000","last_active":"2024-01-27T02:32:05.625193+0000","last_peered":"2024-01-27T02:32:05.625193+0000","last_clean":"2024-01-27T02:32:05.625193+0000","last_became_active":"2024-01-27T02:32:05.624951+0000","last_became_peered":"2024-01-27T02:32:05.624951+0000","last_unstale":"2024-01-27T02:32:05.625193+0000","last_undegraded":"2024-01-27T02:32:05.625193+0000","last_fullsized":"2024-01-27T02:32:05.625193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:04:28.890844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616904+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:31.534822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.625673+0000","last_change":"2024-01-27T02:32:05.625673+0000","last_active":"2024-01-27T02:32:05.625673+0000","last_peered":"2024-01-27T02:32:05.625673+0000","last_clean":"2024-01-27T02:32:05.625673+0000","last_became_active":"2024-01-27T02:32:05.623551+0000","last_became_peered":"2024-01-27T02:32:05.623551+0000","last_unstale":"2024-01-27T02:32:05.625673+0000","last_undegraded":"2024-01-27T02:32:05.625673+0000","last_fullsized":"2024-01-27T02:32:05.625673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614474+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.620877+0000","last_change":"2024-01-27T02:32:05.620877+0000","last_active":"2024-01-27T02:32:05.620877+0000","last_peered":"2024-01-27T02:32:05.620877+0000","last_clean":"2024-01-27T02:32:05.620877+0000","last_became_active":"2024-01-27T02:32:05.620600+0000","last_became_peered":"2024-01-27T02:32:05.620600+0000","last_unstale":"2024-01-27T02:32:05.620877+0000","last_undegraded":"2024-01-27T02:32:05.620877+0000","last_fullsized":"2024-01-27T02:32:05.620877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615243+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.619486+0000","last_change":"2024-01-27T02:32:05.619486+0000","last_active":"2024-01-27T02:32:05.619486+0000","last_peered":"2024-01-27T02:32:05.619486+0000","last_clean":"2024-01-27T02:32:05.619486+0000","last_became_active":"2024-01-27T02:32:05.619147+0000","last_became_peered":"2024-01-27T02:32:05.619147+0000","last_unstale":"2024-01-27T02:32:05.619486+0000","last_undegraded":"2024-01-27T02:32:05.619486+0000","last_fullsized":"2024-01-27T02:32:05.619486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613357+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.613146+0000","last_change":"2024-01-27T02:32:05.613146+0000","last_active":"2024-01-27T02:32:05.613146+0000","last_peered":"2024-01-27T02:32:05.613146+0000","last_clean":"2024-01-27T02:32:05.613146+0000","last_became_active":"2024-01-27T02:32:05.612852+0000","last_became_peered":"2024-01-27T02:32:05.612852+0000","last_unstale":"2024-01-27T02:32:05.613146+0000","last_undegraded":"2024-01-27T02:32:05.613146+0000","last_fullsized":"2024-01-27T02:32:05.613146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617656+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:45:37.035302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616731+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612111+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616985+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:44:13.569068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615702+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:31.534822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617415+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:45.494048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613738+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614627+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.620974+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613231+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617923+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614822+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:58:54.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615379+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:47.819053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613678+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":631,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194681,"statfs":{"total":200000000,"available":199353793,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":637,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194675,"statfs":{"total":200000000,"available":199347530,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]}]}],"pool_statfs":[]}} 2024-01-27T02:32:10.987 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:32:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.153+0000 7f03daa6f700 1 Processor -- start 2024-01-27T02:32:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.153+0000 7f03daa6f700 1 -- start start 2024-01-27T02:32:11.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.154+0000 7f03daa6f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03d415a100 0x7f03d40cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.154+0000 7f03daa6f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d40ce5e0 con 0x7f03d40c4cc0 2024-01-27T02:32:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.154+0000 7f03daa6f700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d40cd9d0 con 0x7f03d40cb5b0 2024-01-27T02:32:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.154+0000 7f03daa6f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40cdcb0 con 0x7f03d415a100 2024-01-27T02:32:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.154+0000 7f03d900c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f03d40c4cc0 0x7f03d4163d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35612/0 (socket says 172.21.15.7:35612) 2024-01-27T02:32:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.154+0000 7f03d900c700 1 -- 172.21.15.7:0/3711117757 learned_addr learned my addr 172.21.15.7:0/3711117757 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.154+0000 7f03d880b700 1 --2- 172.21.15.7:0/3711117757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03d415a100 0x7f03d40cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.155+0000 7f03d37fe700 1 -- 172.21.15.7:0/3711117757 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251953520 0 0) 0x7f03d40ce5e0 con 0x7f03d40c4cc0 2024-01-27T02:32:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.155+0000 7f03d37fe700 1 -- 172.21.15.7:0/3711117757 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03c4003570 con 0x7f03d40c4cc0 2024-01-27T02:32:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.155+0000 7f03d37fe700 1 -- 172.21.15.7:0/3711117757 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140399886 0 0) 0x7f03d40cd9d0 con 0x7f03d40cb5b0 2024-01-27T02:32:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.155+0000 7f03d37fe700 1 -- 172.21.15.7:0/3711117757 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03d40ce5e0 con 0x7f03d40cb5b0 2024-01-27T02:32:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.155+0000 7f03d880b700 1 -- 172.21.15.7:0/3711117757 >> v1:172.21.15.188:6789/0 conn(0x7f03d40cb5b0 legacy=0x7f03d40c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.155+0000 7f03d880b700 1 -- 172.21.15.7:0/3711117757 >> v1:172.21.15.7:6789/0 conn(0x7f03d40c4cc0 legacy=0x7f03d4163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.155+0000 7f03d880b700 1 -- 172.21.15.7:0/3711117757 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d40cdf90 con 0x7f03d415a100 2024-01-27T02:32:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.155+0000 7f03d880b700 1 --2- 172.21.15.7:0/3711117757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03d415a100 0x7f03d40cd2f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009910 tx=0x7f03c80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=784068e3b84d868 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:11.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.155+0000 7f03d37fe700 1 -- 172.21.15.7:0/3711117757 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8011630 con 0x7f03d415a100 2024-01-27T02:32:11.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.155+0000 7f03d37fe700 1 -- 172.21.15.7:0/3711117757 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c8011790 con 0x7f03d415a100 2024-01-27T02:32:11.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.155+0000 7f03d37fe700 1 -- 172.21.15.7:0/3711117757 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8011b30 con 0x7f03d415a100 2024-01-27T02:32:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.156+0000 7f03daa6f700 1 -- 172.21.15.7:0/3711117757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03d415a100 msgr2=0x7f03d40cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.156+0000 7f03daa6f700 1 --2- 172.21.15.7:0/3711117757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03d415a100 0x7f03d40cd2f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009910 tx=0x7f03c80091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.156+0000 7f03daa6f700 1 -- 172.21.15.7:0/3711117757 shutdown_connections 2024-01-27T02:32:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.156+0000 7f03daa6f700 1 --2- 172.21.15.7:0/3711117757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03d415a100 0x7f03d40cd2f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.156+0000 7f03daa6f700 1 -- 172.21.15.7:0/3711117757 >> 172.21.15.7:0/3711117757 conn(0x7f03d4098020 msgr2=0x7f03d40711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.156+0000 7f03daa6f700 1 -- 172.21.15.7:0/3711117757 shutdown_connections 2024-01-27T02:32:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.156+0000 7f03daa6f700 1 -- 172.21.15.7:0/3711117757 wait complete. 2024-01-27T02:32:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.157+0000 7f03daa6f700 1 Processor -- start 2024-01-27T02:32:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.157+0000 7f03daa6f700 1 -- start start 2024-01-27T02:32:11.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.158+0000 7f03daa6f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03d40c4cc0 0x7f03d415c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:11.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.158+0000 7f03daa6f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d40cb5b0 0x7f03d40b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:11.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.158+0000 7f03daa6f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03d415a100 0x7f03d4067ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:11.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.158+0000 7f03daa6f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40b1b00 con 0x7f03d40cb5b0 2024-01-27T02:32:11.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.158+0000 7f03daa6f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40b1de0 con 0x7f03d40c4cc0 2024-01-27T02:32:11.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.158+0000 7f03daa6f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40b20c0 con 0x7f03d415a100 2024-01-27T02:32:11.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.158+0000 7f03d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d40cb5b0 0x7f03d40b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:11.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.158+0000 7f03d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d40cb5b0 0x7f03d40b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41074/0 (socket says 172.21.15.7:41074) 2024-01-27T02:32:11.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.158+0000 7f03d3fff700 1 -- 172.21.15.7:0/3133764343 learned_addr learned my addr 172.21.15.7:0/3133764343 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:11.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.158+0000 7f03d900c700 1 --2- 172.21.15.7:0/3133764343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03d415a100 0x7f03d4067ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:11.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.158+0000 7f03d880b700 1 --2- 172.21.15.7:0/3133764343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03d40c4cc0 0x7f03d415c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:11.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.158+0000 7f03d3fff700 1 -- 172.21.15.7:0/3133764343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03d415a100 msgr2=0x7f03d4067ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:11.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.158+0000 7f03d3fff700 1 --2- 172.21.15.7:0/3133764343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03d415a100 0x7f03d4067ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:11.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.158+0000 7f03d3fff700 1 -- 172.21.15.7:0/3133764343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03d40c4cc0 msgr2=0x7f03d415c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.158+0000 7f03d3fff700 1 --2- 172.21.15.7:0/3133764343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03d40c4cc0 0x7f03d415c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.159+0000 7f03d3fff700 1 -- 172.21.15.7:0/3133764343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c8009420 con 0x7f03d40cb5b0 2024-01-27T02:32:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.159+0000 7f03d3fff700 1 --2- 172.21.15.7:0/3133764343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d40cb5b0 0x7f03d40b41c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f03c000c790 tx=0x7f03c0007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.159+0000 7f03d0ff9700 1 -- 172.21.15.7:0/3133764343 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c0010510 con 0x7f03d40cb5b0 2024-01-27T02:32:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.159+0000 7f03d0ff9700 1 -- 172.21.15.7:0/3133764343 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c0010670 con 0x7f03d40cb5b0 2024-01-27T02:32:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.159+0000 7f03daa6f700 1 -- 172.21.15.7:0/3133764343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d40b23a0 con 0x7f03d40cb5b0 2024-01-27T02:32:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.159+0000 7f03d0ff9700 1 -- 172.21.15.7:0/3133764343 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c0010a10 con 0x7f03d40cb5b0 2024-01-27T02:32:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.159+0000 7f03daa6f700 1 -- 172.21.15.7:0/3133764343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d40b4700 con 0x7f03d40cb5b0 2024-01-27T02:32:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.160+0000 7f03daa6f700 1 -- 172.21.15.7:0/3133764343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d40b28c0 con 0x7f03d40cb5b0 2024-01-27T02:32:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.160+0000 7f03d0ff9700 1 -- 172.21.15.7:0/3133764343 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f03c001b3b0 con 0x7f03d40cb5b0 2024-01-27T02:32:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.161+0000 7f03d0ff9700 1 --2- 172.21.15.7:0/3133764343 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f03b8066f30 0x7f03b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.161+0000 7f03d0ff9700 1 -- 172.21.15.7:0/3133764343 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f03c0083dd0 con 0x7f03d40cb5b0 2024-01-27T02:32:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.161+0000 7f03d880b700 1 --2- 172.21.15.7:0/3133764343 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f03b8066f30 0x7f03b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.162+0000 7f03d880b700 1 --2- 172.21.15.7:0/3133764343 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f03b8066f30 0x7f03b80693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009860 tx=0x7f03c8017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.165+0000 7f03d0ff9700 1 -- 172.21.15.7:0/3133764343 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f03d40b28c0 con 0x7f03d40cb5b0 2024-01-27T02:32:11.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.365+0000 7f03daa6f700 1 -- 172.21.15.7:0/3133764343 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f03d40b1f50 con 0x7f03b8066f30 2024-01-27T02:32:11.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.373+0000 7f03d0ff9700 1 -- 172.21.15.7:0/3133764343 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+257304 (secure 0 0 0) 0x7f03d40b1f50 con 0x7f03b8066f30 2024-01-27T02:32:11.376 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.376+0000 7f03daa6f700 1 -- 172.21.15.7:0/3133764343 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f03b8066f30 msgr2=0x7f03b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.376+0000 7f03daa6f700 1 --2- 172.21.15.7:0/3133764343 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f03b8066f30 0x7f03b80693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009860 tx=0x7f03c8017000 comp rx=0 tx=0).stop 2024-01-27T02:32:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.376+0000 7f03daa6f700 1 -- 172.21.15.7:0/3133764343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d40cb5b0 msgr2=0x7f03d40b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.376+0000 7f03daa6f700 1 --2- 172.21.15.7:0/3133764343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d40cb5b0 0x7f03d40b41c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f03c000c790 tx=0x7f03c0007e10 comp rx=0 tx=0).stop 2024-01-27T02:32:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.376+0000 7f03daa6f700 1 -- 172.21.15.7:0/3133764343 shutdown_connections 2024-01-27T02:32:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.376+0000 7f03daa6f700 1 --2- 172.21.15.7:0/3133764343 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f03b8066f30 0x7f03b80693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.376+0000 7f03daa6f700 1 --2- 172.21.15.7:0/3133764343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03d415a100 0x7f03d4067ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.376+0000 7f03daa6f700 1 --2- 172.21.15.7:0/3133764343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03d40c4cc0 0x7f03d415c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.376+0000 7f03daa6f700 1 --2- 172.21.15.7:0/3133764343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d40cb5b0 0x7f03d40b41c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.376+0000 7f03daa6f700 1 -- 172.21.15.7:0/3133764343 >> 172.21.15.7:0/3133764343 conn(0x7f03d4098020 msgr2=0x7f03d414eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.376+0000 7f03daa6f700 1 -- 172.21.15.7:0/3133764343 shutdown_connections 2024-01-27T02:32:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.376+0000 7f03daa6f700 1 -- 172.21.15.7:0/3133764343 wait complete. 2024-01-27T02:32:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:32:11.391 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-01-27T02:32:09.572481+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1632,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560867,"statfs":{"total":1600000000,"available":1598328711,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.964513"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616502+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:44:13.569068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617845+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617854+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614017+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.619412+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:32:24.198014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.618664+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:45:37.035302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616761+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:17:03.351766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.622881+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615495+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614084+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615294+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612594+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:08.590979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.621489+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615194+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:08.590979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617367+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616115+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:06.985231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615481+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612774+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617263+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:36:11.295485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.624071+0000","last_change":"2024-01-27T02:32:05.624071+0000","last_active":"2024-01-27T02:32:05.624071+0000","last_peered":"2024-01-27T02:32:05.624071+0000","last_clean":"2024-01-27T02:32:05.624071+0000","last_became_active":"2024-01-27T02:32:05.623818+0000","last_became_peered":"2024-01-27T02:32:05.623818+0000","last_unstale":"2024-01-27T02:32:05.624071+0000","last_undegraded":"2024-01-27T02:32:05.624071+0000","last_fullsized":"2024-01-27T02:32:05.624071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:34:05.180050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.622620+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.624866+0000","last_change":"2024-01-27T02:32:05.624866+0000","last_active":"2024-01-27T02:32:05.624866+0000","last_peered":"2024-01-27T02:32:05.624866+0000","last_clean":"2024-01-27T02:32:05.624866+0000","last_became_active":"2024-01-27T02:32:05.624368+0000","last_became_peered":"2024-01-27T02:32:05.624368+0000","last_unstale":"2024-01-27T02:32:05.624866+0000","last_undegraded":"2024-01-27T02:32:05.624866+0000","last_fullsized":"2024-01-27T02:32:05.624866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617401+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.623255+0000","last_change":"2024-01-27T02:32:05.623255+0000","last_active":"2024-01-27T02:32:05.623255+0000","last_peered":"2024-01-27T02:32:05.623255+0000","last_clean":"2024-01-27T02:32:05.623255+0000","last_became_active":"2024-01-27T02:32:05.622821+0000","last_became_peered":"2024-01-27T02:32:05.622821+0000","last_unstale":"2024-01-27T02:32:05.623255+0000","last_undegraded":"2024-01-27T02:32:05.623255+0000","last_fullsized":"2024-01-27T02:32:05.623255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:33.333681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615102+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.626014+0000","last_change":"2024-01-27T02:32:05.626014+0000","last_active":"2024-01-27T02:32:05.626014+0000","last_peered":"2024-01-27T02:32:05.626014+0000","last_clean":"2024-01-27T02:32:05.626014+0000","last_became_active":"2024-01-27T02:32:05.623176+0000","last_became_peered":"2024-01-27T02:32:05.623176+0000","last_unstale":"2024-01-27T02:32:05.626014+0000","last_undegraded":"2024-01-27T02:32:05.626014+0000","last_fullsized":"2024-01-27T02:32:05.626014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:37:59.246901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616282+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.622086+0000","last_change":"2024-01-27T02:32:05.622086+0000","last_active":"2024-01-27T02:32:05.622086+0000","last_peered":"2024-01-27T02:32:05.622086+0000","last_clean":"2024-01-27T02:32:05.622086+0000","last_became_active":"2024-01-27T02:32:05.621465+0000","last_became_peered":"2024-01-27T02:32:05.621465+0000","last_unstale":"2024-01-27T02:32:05.622086+0000","last_undegraded":"2024-01-27T02:32:05.622086+0000","last_fullsized":"2024-01-27T02:32:05.622086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:04:28.890844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612345+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:42.580188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.617530+0000","last_change":"2024-01-27T02:32:05.617530+0000","last_active":"2024-01-27T02:32:05.617530+0000","last_peered":"2024-01-27T02:32:05.617530+0000","last_clean":"2024-01-27T02:32:05.617530+0000","last_became_active":"2024-01-27T02:32:05.617290+0000","last_became_peered":"2024-01-27T02:32:05.617290+0000","last_unstale":"2024-01-27T02:32:05.617530+0000","last_undegraded":"2024-01-27T02:32:05.617530+0000","last_fullsized":"2024-01-27T02:32:05.617530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:41.575605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614574+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:53.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.623693+0000","last_change":"2024-01-27T02:32:05.623693+0000","last_active":"2024-01-27T02:32:05.623693+0000","last_peered":"2024-01-27T02:32:05.623693+0000","last_clean":"2024-01-27T02:32:05.623693+0000","last_became_active":"2024-01-27T02:32:05.623440+0000","last_became_peered":"2024-01-27T02:32:05.623440+0000","last_unstale":"2024-01-27T02:32:05.623693+0000","last_undegraded":"2024-01-27T02:32:05.623693+0000","last_fullsized":"2024-01-27T02:32:05.623693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:44:52.929407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.619517+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.619476+0000","last_change":"2024-01-27T02:32:05.619476+0000","last_active":"2024-01-27T02:32:05.619476+0000","last_peered":"2024-01-27T02:32:05.619476+0000","last_clean":"2024-01-27T02:32:05.619476+0000","last_became_active":"2024-01-27T02:32:05.619194+0000","last_became_peered":"2024-01-27T02:32:05.619194+0000","last_unstale":"2024-01-27T02:32:05.619476+0000","last_undegraded":"2024-01-27T02:32:05.619476+0000","last_fullsized":"2024-01-27T02:32:05.619476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615596+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.623827+0000","last_change":"2024-01-27T02:32:05.623827+0000","last_active":"2024-01-27T02:32:05.623827+0000","last_peered":"2024-01-27T02:32:05.623827+0000","last_clean":"2024-01-27T02:32:05.623827+0000","last_became_active":"2024-01-27T02:32:05.623602+0000","last_became_peered":"2024-01-27T02:32:05.623602+0000","last_unstale":"2024-01-27T02:32:05.623827+0000","last_undegraded":"2024-01-27T02:32:05.623827+0000","last_fullsized":"2024-01-27T02:32:05.623827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613756+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.621283+0000","last_change":"2024-01-27T02:32:05.621283+0000","last_active":"2024-01-27T02:32:05.621283+0000","last_peered":"2024-01-27T02:32:05.621283+0000","last_clean":"2024-01-27T02:32:05.621283+0000","last_became_active":"2024-01-27T02:32:05.621034+0000","last_became_peered":"2024-01-27T02:32:05.621034+0000","last_unstale":"2024-01-27T02:32:05.621283+0000","last_undegraded":"2024-01-27T02:32:05.621283+0000","last_fullsized":"2024-01-27T02:32:05.621283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:07.586396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614840+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:42.580188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.620524+0000","last_change":"2024-01-27T02:32:05.620524+0000","last_active":"2024-01-27T02:32:05.620524+0000","last_peered":"2024-01-27T02:32:05.620524+0000","last_clean":"2024-01-27T02:32:05.620524+0000","last_became_active":"2024-01-27T02:32:05.620169+0000","last_became_peered":"2024-01-27T02:32:05.620169+0000","last_unstale":"2024-01-27T02:32:05.620524+0000","last_undegraded":"2024-01-27T02:32:05.620524+0000","last_fullsized":"2024-01-27T02:32:05.620524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612315+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:53.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.625720+0000","last_change":"2024-01-27T02:32:05.625720+0000","last_active":"2024-01-27T02:32:05.625720+0000","last_peered":"2024-01-27T02:32:05.625720+0000","last_clean":"2024-01-27T02:32:05.625720+0000","last_became_active":"2024-01-27T02:32:05.625394+0000","last_became_peered":"2024-01-27T02:32:05.625394+0000","last_unstale":"2024-01-27T02:32:05.625720+0000","last_undegraded":"2024-01-27T02:32:05.625720+0000","last_fullsized":"2024-01-27T02:32:05.625720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.621410+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:16:23.225540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.619745+0000","last_change":"2024-01-27T02:32:05.619745+0000","last_active":"2024-01-27T02:32:05.619745+0000","last_peered":"2024-01-27T02:32:05.619745+0000","last_clean":"2024-01-27T02:32:05.619745+0000","last_became_active":"2024-01-27T02:32:05.619371+0000","last_became_peered":"2024-01-27T02:32:05.619371+0000","last_unstale":"2024-01-27T02:32:05.619745+0000","last_undegraded":"2024-01-27T02:32:05.619745+0000","last_fullsized":"2024-01-27T02:32:05.619745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:12:20.892542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614205+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:04:29.895427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.626895+0000","last_change":"2024-01-27T02:32:05.626895+0000","last_active":"2024-01-27T02:32:05.626895+0000","last_peered":"2024-01-27T02:32:05.626895+0000","last_clean":"2024-01-27T02:32:05.626895+0000","last_became_active":"2024-01-27T02:32:05.626682+0000","last_became_peered":"2024-01-27T02:32:05.626682+0000","last_unstale":"2024-01-27T02:32:05.626895+0000","last_undegraded":"2024-01-27T02:32:05.626895+0000","last_fullsized":"2024-01-27T02:32:05.626895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:41.575605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.618952+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:11:54.230178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.619939+0000","last_change":"2024-01-27T02:32:05.619939+0000","last_active":"2024-01-27T02:32:05.619939+0000","last_peered":"2024-01-27T02:32:05.619939+0000","last_clean":"2024-01-27T02:32:05.619939+0000","last_became_active":"2024-01-27T02:32:05.619489+0000","last_became_peered":"2024-01-27T02:32:05.619489+0000","last_unstale":"2024-01-27T02:32:05.619939+0000","last_undegraded":"2024-01-27T02:32:05.619939+0000","last_fullsized":"2024-01-27T02:32:05.619939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:07.586396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615600+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:58:54.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.621851+0000","last_change":"2024-01-27T02:32:05.621851+0000","last_active":"2024-01-27T02:32:05.621851+0000","last_peered":"2024-01-27T02:32:05.621851+0000","last_clean":"2024-01-27T02:32:05.621851+0000","last_became_active":"2024-01-27T02:32:05.621487+0000","last_became_peered":"2024-01-27T02:32:05.621487+0000","last_unstale":"2024-01-27T02:32:05.621851+0000","last_undegraded":"2024-01-27T02:32:05.621851+0000","last_fullsized":"2024-01-27T02:32:05.621851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:07.586396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616380+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:06.985231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.623147+0000","last_change":"2024-01-27T02:32:05.623147+0000","last_active":"2024-01-27T02:32:05.623147+0000","last_peered":"2024-01-27T02:32:05.623147+0000","last_clean":"2024-01-27T02:32:05.623147+0000","last_became_active":"2024-01-27T02:32:05.622650+0000","last_became_peered":"2024-01-27T02:32:05.622650+0000","last_unstale":"2024-01-27T02:32:05.623147+0000","last_undegraded":"2024-01-27T02:32:05.623147+0000","last_fullsized":"2024-01-27T02:32:05.623147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:30:53.128829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615959+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.621921+0000","last_change":"2024-01-27T02:32:05.621921+0000","last_active":"2024-01-27T02:32:05.621921+0000","last_peered":"2024-01-27T02:32:05.621921+0000","last_clean":"2024-01-27T02:32:05.621921+0000","last_became_active":"2024-01-27T02:32:05.620450+0000","last_became_peered":"2024-01-27T02:32:05.620450+0000","last_unstale":"2024-01-27T02:32:05.621921+0000","last_undegraded":"2024-01-27T02:32:05.621921+0000","last_fullsized":"2024-01-27T02:32:05.621921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:33.333681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613665+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.622783+0000","last_change":"2024-01-27T02:32:05.622783+0000","last_active":"2024-01-27T02:32:05.622783+0000","last_peered":"2024-01-27T02:32:05.622783+0000","last_clean":"2024-01-27T02:32:05.622783+0000","last_became_active":"2024-01-27T02:32:05.622557+0000","last_became_peered":"2024-01-27T02:32:05.622557+0000","last_unstale":"2024-01-27T02:32:05.622783+0000","last_undegraded":"2024-01-27T02:32:05.622783+0000","last_fullsized":"2024-01-27T02:32:05.622783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612935+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:04:29.895427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615166+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.613447+0000","last_change":"2024-01-27T02:32:05.613447+0000","last_active":"2024-01-27T02:32:05.613447+0000","last_peered":"2024-01-27T02:32:05.613447+0000","last_clean":"2024-01-27T02:32:05.613447+0000","last_became_active":"2024-01-27T02:32:05.613203+0000","last_became_peered":"2024-01-27T02:32:05.613203+0000","last_unstale":"2024-01-27T02:32:05.613447+0000","last_undegraded":"2024-01-27T02:32:05.613447+0000","last_fullsized":"2024-01-27T02:32:05.613447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:41.575605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616703+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:47.819053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.622673+0000","last_change":"2024-01-27T02:32:05.622673+0000","last_active":"2024-01-27T02:32:05.622673+0000","last_peered":"2024-01-27T02:32:05.622673+0000","last_clean":"2024-01-27T02:32:05.622673+0000","last_became_active":"2024-01-27T02:32:05.622195+0000","last_became_peered":"2024-01-27T02:32:05.622195+0000","last_unstale":"2024-01-27T02:32:05.622673+0000","last_undegraded":"2024-01-27T02:32:05.622673+0000","last_fullsized":"2024-01-27T02:32:05.622673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":161,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-01-27T02:32:04.612144+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:32:04.612144+0000","last_peered":"2024-01-27T02:32:04.612144+0000","last_clean":"2024-01-27T02:32:04.612144+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:32:04.612144+0000","last_undegraded":"2024-01-27T02:32:04.612144+0000","last_fullsized":"2024-01-27T02:32:04.612144+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616221+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.626552+0000","last_change":"2024-01-27T02:32:05.626552+0000","last_active":"2024-01-27T02:32:05.626552+0000","last_peered":"2024-01-27T02:32:05.626552+0000","last_clean":"2024-01-27T02:32:05.626552+0000","last_became_active":"2024-01-27T02:32:05.626348+0000","last_became_peered":"2024-01-27T02:32:05.626348+0000","last_unstale":"2024-01-27T02:32:05.626552+0000","last_undegraded":"2024-01-27T02:32:05.626552+0000","last_fullsized":"2024-01-27T02:32:05.626552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:18.034484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.625885+0000","last_change":"2024-01-27T02:32:05.625885+0000","last_active":"2024-01-27T02:32:05.625885+0000","last_peered":"2024-01-27T02:32:05.625885+0000","last_clean":"2024-01-27T02:32:05.625885+0000","last_became_active":"2024-01-27T02:32:05.622692+0000","last_became_peered":"2024-01-27T02:32:05.622692+0000","last_unstale":"2024-01-27T02:32:05.625885+0000","last_undegraded":"2024-01-27T02:32:05.625885+0000","last_fullsized":"2024-01-27T02:32:05.625885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:06.920232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616950+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:58:54.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.621014+0000","last_change":"2024-01-27T02:32:05.621014+0000","last_active":"2024-01-27T02:32:05.621014+0000","last_peered":"2024-01-27T02:32:05.621014+0000","last_clean":"2024-01-27T02:32:05.621014+0000","last_became_active":"2024-01-27T02:32:05.620775+0000","last_became_peered":"2024-01-27T02:32:05.620775+0000","last_unstale":"2024-01-27T02:32:05.621014+0000","last_undegraded":"2024-01-27T02:32:05.621014+0000","last_fullsized":"2024-01-27T02:32:05.621014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617909+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.624870+0000","last_change":"2024-01-27T02:32:05.624870+0000","last_active":"2024-01-27T02:32:05.624870+0000","last_peered":"2024-01-27T02:32:05.624870+0000","last_clean":"2024-01-27T02:32:05.624870+0000","last_became_active":"2024-01-27T02:32:05.624644+0000","last_became_peered":"2024-01-27T02:32:05.624644+0000","last_unstale":"2024-01-27T02:32:05.624870+0000","last_undegraded":"2024-01-27T02:32:05.624870+0000","last_fullsized":"2024-01-27T02:32:05.624870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616980+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.621287+0000","last_change":"2024-01-27T02:32:05.621287+0000","last_active":"2024-01-27T02:32:05.621287+0000","last_peered":"2024-01-27T02:32:05.621287+0000","last_clean":"2024-01-27T02:32:05.621287+0000","last_became_active":"2024-01-27T02:32:05.620002+0000","last_became_peered":"2024-01-27T02:32:05.620002+0000","last_unstale":"2024-01-27T02:32:05.621287+0000","last_undegraded":"2024-01-27T02:32:05.621287+0000","last_fullsized":"2024-01-27T02:32:05.621287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:44.489465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612241+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:44:53.933990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.623449+0000","last_change":"2024-01-27T02:32:05.623449+0000","last_active":"2024-01-27T02:32:05.623449+0000","last_peered":"2024-01-27T02:32:05.623449+0000","last_clean":"2024-01-27T02:32:05.623449+0000","last_became_active":"2024-01-27T02:32:05.623193+0000","last_became_peered":"2024-01-27T02:32:05.623193+0000","last_unstale":"2024-01-27T02:32:05.623449+0000","last_undegraded":"2024-01-27T02:32:05.623449+0000","last_fullsized":"2024-01-27T02:32:05.623449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:44.489465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616309+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:08.590979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.625193+0000","last_change":"2024-01-27T02:32:05.625193+0000","last_active":"2024-01-27T02:32:05.625193+0000","last_peered":"2024-01-27T02:32:05.625193+0000","last_clean":"2024-01-27T02:32:05.625193+0000","last_became_active":"2024-01-27T02:32:05.624951+0000","last_became_peered":"2024-01-27T02:32:05.624951+0000","last_unstale":"2024-01-27T02:32:05.625193+0000","last_undegraded":"2024-01-27T02:32:05.625193+0000","last_fullsized":"2024-01-27T02:32:05.625193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:04:28.890844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616904+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:31.534822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.625673+0000","last_change":"2024-01-27T02:32:05.625673+0000","last_active":"2024-01-27T02:32:05.625673+0000","last_peered":"2024-01-27T02:32:05.625673+0000","last_clean":"2024-01-27T02:32:05.625673+0000","last_became_active":"2024-01-27T02:32:05.623551+0000","last_became_peered":"2024-01-27T02:32:05.623551+0000","last_unstale":"2024-01-27T02:32:05.625673+0000","last_undegraded":"2024-01-27T02:32:05.625673+0000","last_fullsized":"2024-01-27T02:32:05.625673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614474+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.620877+0000","last_change":"2024-01-27T02:32:05.620877+0000","last_active":"2024-01-27T02:32:05.620877+0000","last_peered":"2024-01-27T02:32:05.620877+0000","last_clean":"2024-01-27T02:32:05.620877+0000","last_became_active":"2024-01-27T02:32:05.620600+0000","last_became_peered":"2024-01-27T02:32:05.620600+0000","last_unstale":"2024-01-27T02:32:05.620877+0000","last_undegraded":"2024-01-27T02:32:05.620877+0000","last_fullsized":"2024-01-27T02:32:05.620877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615243+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.619486+0000","last_change":"2024-01-27T02:32:05.619486+0000","last_active":"2024-01-27T02:32:05.619486+0000","last_peered":"2024-01-27T02:32:05.619486+0000","last_clean":"2024-01-27T02:32:05.619486+0000","last_became_active":"2024-01-27T02:32:05.619147+0000","last_became_peered":"2024-01-27T02:32:05.619147+0000","last_unstale":"2024-01-27T02:32:05.619486+0000","last_undegraded":"2024-01-27T02:32:05.619486+0000","last_fullsized":"2024-01-27T02:32:05.619486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613357+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.613146+0000","last_change":"2024-01-27T02:32:05.613146+0000","last_active":"2024-01-27T02:32:05.613146+0000","last_peered":"2024-01-27T02:32:05.613146+0000","last_clean":"2024-01-27T02:32:05.613146+0000","last_became_active":"2024-01-27T02:32:05.612852+0000","last_became_peered":"2024-01-27T02:32:05.612852+0000","last_unstale":"2024-01-27T02:32:05.613146+0000","last_undegraded":"2024-01-27T02:32:05.613146+0000","last_fullsized":"2024-01-27T02:32:05.613146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617656+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:45:37.035302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616731+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612111+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616985+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:44:13.569068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615702+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:31.534822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617415+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:45.494048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613738+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614627+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.620974+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613231+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617923+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614822+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:58:54.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615379+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:47.819053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613678+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":631,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194681,"statfs":{"total":200000000,"available":199353793,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":637,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194675,"statfs":{"total":200000000,"available":199347530,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]}]}],"pool_statfs":[]}} 2024-01-27T02:32:11.395 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:32:11.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.565+0000 7fe2a1084700 1 Processor -- start 2024-01-27T02:32:11.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.565+0000 7fe2a1084700 1 -- start start 2024-01-27T02:32:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.566+0000 7fe2a1084700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe29c0cdbf0 0x7fe29c162c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.566+0000 7fe2a1084700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe29c0ca020 con 0x7fe29c157ca0 2024-01-27T02:32:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.566+0000 7fe2a1084700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe29c0c5210 con 0x7fe29c0c3ae0 2024-01-27T02:32:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.566+0000 7fe2a1084700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe29c0c54f0 con 0x7fe29c0cdbf0 2024-01-27T02:32:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.566+0000 7fe29b59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe29c0cdbf0 0x7fe29c162c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.566+0000 7fe29ad9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe29c157ca0 0x7fe29c0c4a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51312/0 (socket says 172.21.15.7:51312) 2024-01-27T02:32:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.567+0000 7fe29ad9d700 1 -- 172.21.15.7:0/1242897375 learned_addr learned my addr 172.21.15.7:0/1242897375 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.567+0000 7fe29b59e700 1 -- 172.21.15.7:0/1242897375 >> v1:172.21.15.188:6789/0 conn(0x7fe29c0c3ae0 legacy=0x7fe29c0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.567+0000 7fe29b59e700 1 -- 172.21.15.7:0/1242897375 >> v1:172.21.15.7:6789/0 conn(0x7fe29c157ca0 legacy=0x7fe29c0c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.567+0000 7fe299d9b700 1 -- 172.21.15.7:0/1242897375 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132822226 0 0) 0x7fe29c0ca020 con 0x7fe29c157ca0 2024-01-27T02:32:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.567+0000 7fe29b59e700 1 -- 172.21.15.7:0/1242897375 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c0c57d0 con 0x7fe29c0cdbf0 2024-01-27T02:32:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.567+0000 7fe29b59e700 1 --2- 172.21.15.7:0/1242897375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe29c0cdbf0 0x7fe29c162c00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe294009e60 tx=0x7fe294009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f66ab3c7d55b99 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:11.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.567+0000 7fe299d9b700 1 -- 172.21.15.7:0/1242897375 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29400e070 con 0x7fe29c0cdbf0 2024-01-27T02:32:11.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.567+0000 7fe299d9b700 1 -- 172.21.15.7:0/1242897375 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29400bc30 con 0x7fe29c0cdbf0 2024-01-27T02:32:11.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.567+0000 7fe299d9b700 1 -- 172.21.15.7:0/1242897375 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294012430 con 0x7fe29c0cdbf0 2024-01-27T02:32:11.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.568+0000 7fe2a1084700 1 -- 172.21.15.7:0/1242897375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe29c0cdbf0 msgr2=0x7fe29c162c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:11.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.568+0000 7fe2a1084700 1 --2- 172.21.15.7:0/1242897375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe29c0cdbf0 0x7fe29c162c00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe294009e60 tx=0x7fe294009780 comp rx=0 tx=0).stop 2024-01-27T02:32:11.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.568+0000 7fe2a1084700 1 -- 172.21.15.7:0/1242897375 shutdown_connections 2024-01-27T02:32:11.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.568+0000 7fe2a1084700 1 --2- 172.21.15.7:0/1242897375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe29c0cdbf0 0x7fe29c162c00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:11.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.568+0000 7fe2a1084700 1 -- 172.21.15.7:0/1242897375 >> 172.21.15.7:0/1242897375 conn(0x7fe29c098020 msgr2=0x7fe29c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:11.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.568+0000 7fe2a1084700 1 -- 172.21.15.7:0/1242897375 shutdown_connections 2024-01-27T02:32:11.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.568+0000 7fe2a1084700 1 -- 172.21.15.7:0/1242897375 wait complete. 2024-01-27T02:32:11.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.569+0000 7fe2a1084700 1 Processor -- start 2024-01-27T02:32:11.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.569+0000 7fe2a1084700 1 -- start start 2024-01-27T02:32:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.570+0000 7fe2a1084700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe29c0c3ae0 0x7fe29c156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.570+0000 7fe2a1084700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe29c0cdbf0 0x7fe29c153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.570+0000 7fe2a1084700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe29c157ca0 0x7fe29c152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.570+0000 7fe2a1084700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c157020 con 0x7fe29c0cdbf0 2024-01-27T02:32:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.570+0000 7fe2a1084700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c156650 con 0x7fe29c157ca0 2024-01-27T02:32:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.570+0000 7fe2a1084700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c06ca90 con 0x7fe29c0c3ae0 2024-01-27T02:32:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.570+0000 7fe29a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe29c0cdbf0 0x7fe29c153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.570+0000 7fe29a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe29c0cdbf0 0x7fe29c153c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53540/0 (socket says 172.21.15.7:53540) 2024-01-27T02:32:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.570+0000 7fe29a59c700 1 -- 172.21.15.7:0/2631569563 learned_addr learned my addr 172.21.15.7:0/2631569563 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.570+0000 7fe29ad9d700 1 --2- 172.21.15.7:0/2631569563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe29c0c3ae0 0x7fe29c156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.570+0000 7fe29b59e700 1 --2- 172.21.15.7:0/2631569563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe29c157ca0 0x7fe29c152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.570+0000 7fe29a59c700 1 -- 172.21.15.7:0/2631569563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe29c0c3ae0 msgr2=0x7fe29c156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.570+0000 7fe29a59c700 1 --2- 172.21.15.7:0/2631569563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe29c0c3ae0 0x7fe29c156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.570+0000 7fe29a59c700 1 -- 172.21.15.7:0/2631569563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe29c157ca0 msgr2=0x7fe29c152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.570+0000 7fe29a59c700 1 --2- 172.21.15.7:0/2631569563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe29c157ca0 0x7fe29c152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.570+0000 7fe29a59c700 1 -- 172.21.15.7:0/2631569563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe294009a20 con 0x7fe29c0cdbf0 2024-01-27T02:32:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.571+0000 7fe29a59c700 1 --2- 172.21.15.7:0/2631569563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe29c0cdbf0 0x7fe29c153c90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe29000c4e0 tx=0x7fe29000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.571+0000 7fe282ffd700 1 -- 172.21.15.7:0/2631569563 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe290011590 con 0x7fe29c0cdbf0 2024-01-27T02:32:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.571+0000 7fe282ffd700 1 -- 172.21.15.7:0/2631569563 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe290011710 con 0x7fe29c0cdbf0 2024-01-27T02:32:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.571+0000 7fe2a1084700 1 -- 172.21.15.7:0/2631569563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29c1541d0 con 0x7fe29c0cdbf0 2024-01-27T02:32:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.571+0000 7fe282ffd700 1 -- 172.21.15.7:0/2631569563 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe290011ad0 con 0x7fe29c0cdbf0 2024-01-27T02:32:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.571+0000 7fe2a1084700 1 -- 172.21.15.7:0/2631569563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29c06d010 con 0x7fe29c0cdbf0 2024-01-27T02:32:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.572+0000 7fe2a1084700 1 -- 172.21.15.7:0/2631569563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29c156800 con 0x7fe29c0cdbf0 2024-01-27T02:32:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.572+0000 7fe282ffd700 1 -- 172.21.15.7:0/2631569563 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe29001c440 con 0x7fe29c0cdbf0 2024-01-27T02:32:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.573+0000 7fe282ffd700 1 --2- 172.21.15.7:0/2631569563 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe288066f30 0x7fe2880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.573+0000 7fe282ffd700 1 -- 172.21.15.7:0/2631569563 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe2900839f0 con 0x7fe29c0cdbf0 2024-01-27T02:32:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.573+0000 7fe29ad9d700 1 --2- 172.21.15.7:0/2631569563 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe288066f30 0x7fe2880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.574+0000 7fe29ad9d700 1 --2- 172.21.15.7:0/2631569563 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe288066f30 0x7fe2880693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0c8380 tx=0x7fe28c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.577+0000 7fe282ffd700 1 -- 172.21.15.7:0/2631569563 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe29c156800 con 0x7fe29c0cdbf0 2024-01-27T02:32:11.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.778+0000 7fe2a1084700 1 -- 172.21.15.7:0/2631569563 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe29c1567c0 con 0x7fe288066f30 2024-01-27T02:32:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.786+0000 7fe282ffd700 1 -- 172.21.15.7:0/2631569563 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+257304 (secure 0 0 0) 0x7fe29c1567c0 con 0x7fe288066f30 2024-01-27T02:32:11.790 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.789+0000 7fe2a1084700 1 -- 172.21.15.7:0/2631569563 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe288066f30 msgr2=0x7fe2880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.789+0000 7fe2a1084700 1 --2- 172.21.15.7:0/2631569563 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe288066f30 0x7fe2880693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0c8380 tx=0x7fe28c009040 comp rx=0 tx=0).stop 2024-01-27T02:32:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.789+0000 7fe2a1084700 1 -- 172.21.15.7:0/2631569563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe29c0cdbf0 msgr2=0x7fe29c153c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.789+0000 7fe2a1084700 1 --2- 172.21.15.7:0/2631569563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe29c0cdbf0 0x7fe29c153c90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe29000c4e0 tx=0x7fe29000c8a0 comp rx=0 tx=0).stop 2024-01-27T02:32:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.789+0000 7fe2a1084700 1 -- 172.21.15.7:0/2631569563 shutdown_connections 2024-01-27T02:32:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.789+0000 7fe2a1084700 1 --2- 172.21.15.7:0/2631569563 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe288066f30 0x7fe2880693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.789+0000 7fe2a1084700 1 --2- 172.21.15.7:0/2631569563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe29c157ca0 0x7fe29c152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.789+0000 7fe2a1084700 1 --2- 172.21.15.7:0/2631569563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe29c0c3ae0 0x7fe29c156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.789+0000 7fe2a1084700 1 --2- 172.21.15.7:0/2631569563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe29c0cdbf0 0x7fe29c153c90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.789+0000 7fe2a1084700 1 -- 172.21.15.7:0/2631569563 >> 172.21.15.7:0/2631569563 conn(0x7fe29c098020 msgr2=0x7fe29c0ca4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.789+0000 7fe2a1084700 1 -- 172.21.15.7:0/2631569563 shutdown_connections 2024-01-27T02:32:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.789+0000 7fe2a1084700 1 -- 172.21.15.7:0/2631569563 wait complete. 2024-01-27T02:32:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:32:11.803 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-01-27T02:32:11.573004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1632,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560867,"statfs":{"total":1600000000,"available":1598328711,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001678"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616502+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:44:13.569068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617845+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617854+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614017+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.619412+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:32:24.198014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.618664+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:45:37.035302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616761+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:17:03.351766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.622881+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615495+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614084+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615294+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612594+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:08.590979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.621489+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615194+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:08.590979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617367+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616115+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:06.985231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615481+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612774+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617263+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:36:11.295485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.624071+0000","last_change":"2024-01-27T02:32:05.624071+0000","last_active":"2024-01-27T02:32:05.624071+0000","last_peered":"2024-01-27T02:32:05.624071+0000","last_clean":"2024-01-27T02:32:05.624071+0000","last_became_active":"2024-01-27T02:32:05.623818+0000","last_became_peered":"2024-01-27T02:32:05.623818+0000","last_unstale":"2024-01-27T02:32:05.624071+0000","last_undegraded":"2024-01-27T02:32:05.624071+0000","last_fullsized":"2024-01-27T02:32:05.624071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:34:05.180050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.622620+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.624866+0000","last_change":"2024-01-27T02:32:05.624866+0000","last_active":"2024-01-27T02:32:05.624866+0000","last_peered":"2024-01-27T02:32:05.624866+0000","last_clean":"2024-01-27T02:32:05.624866+0000","last_became_active":"2024-01-27T02:32:05.624368+0000","last_became_peered":"2024-01-27T02:32:05.624368+0000","last_unstale":"2024-01-27T02:32:05.624866+0000","last_undegraded":"2024-01-27T02:32:05.624866+0000","last_fullsized":"2024-01-27T02:32:05.624866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617401+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.623255+0000","last_change":"2024-01-27T02:32:05.623255+0000","last_active":"2024-01-27T02:32:05.623255+0000","last_peered":"2024-01-27T02:32:05.623255+0000","last_clean":"2024-01-27T02:32:05.623255+0000","last_became_active":"2024-01-27T02:32:05.622821+0000","last_became_peered":"2024-01-27T02:32:05.622821+0000","last_unstale":"2024-01-27T02:32:05.623255+0000","last_undegraded":"2024-01-27T02:32:05.623255+0000","last_fullsized":"2024-01-27T02:32:05.623255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:33.333681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615102+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.626014+0000","last_change":"2024-01-27T02:32:05.626014+0000","last_active":"2024-01-27T02:32:05.626014+0000","last_peered":"2024-01-27T02:32:05.626014+0000","last_clean":"2024-01-27T02:32:05.626014+0000","last_became_active":"2024-01-27T02:32:05.623176+0000","last_became_peered":"2024-01-27T02:32:05.623176+0000","last_unstale":"2024-01-27T02:32:05.626014+0000","last_undegraded":"2024-01-27T02:32:05.626014+0000","last_fullsized":"2024-01-27T02:32:05.626014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:37:59.246901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616282+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.622086+0000","last_change":"2024-01-27T02:32:05.622086+0000","last_active":"2024-01-27T02:32:05.622086+0000","last_peered":"2024-01-27T02:32:05.622086+0000","last_clean":"2024-01-27T02:32:05.622086+0000","last_became_active":"2024-01-27T02:32:05.621465+0000","last_became_peered":"2024-01-27T02:32:05.621465+0000","last_unstale":"2024-01-27T02:32:05.622086+0000","last_undegraded":"2024-01-27T02:32:05.622086+0000","last_fullsized":"2024-01-27T02:32:05.622086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:04:28.890844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612345+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:42.580188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.617530+0000","last_change":"2024-01-27T02:32:05.617530+0000","last_active":"2024-01-27T02:32:05.617530+0000","last_peered":"2024-01-27T02:32:05.617530+0000","last_clean":"2024-01-27T02:32:05.617530+0000","last_became_active":"2024-01-27T02:32:05.617290+0000","last_became_peered":"2024-01-27T02:32:05.617290+0000","last_unstale":"2024-01-27T02:32:05.617530+0000","last_undegraded":"2024-01-27T02:32:05.617530+0000","last_fullsized":"2024-01-27T02:32:05.617530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:41.575605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614574+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:53.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.623693+0000","last_change":"2024-01-27T02:32:05.623693+0000","last_active":"2024-01-27T02:32:05.623693+0000","last_peered":"2024-01-27T02:32:05.623693+0000","last_clean":"2024-01-27T02:32:05.623693+0000","last_became_active":"2024-01-27T02:32:05.623440+0000","last_became_peered":"2024-01-27T02:32:05.623440+0000","last_unstale":"2024-01-27T02:32:05.623693+0000","last_undegraded":"2024-01-27T02:32:05.623693+0000","last_fullsized":"2024-01-27T02:32:05.623693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:44:52.929407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.619517+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.619476+0000","last_change":"2024-01-27T02:32:05.619476+0000","last_active":"2024-01-27T02:32:05.619476+0000","last_peered":"2024-01-27T02:32:05.619476+0000","last_clean":"2024-01-27T02:32:05.619476+0000","last_became_active":"2024-01-27T02:32:05.619194+0000","last_became_peered":"2024-01-27T02:32:05.619194+0000","last_unstale":"2024-01-27T02:32:05.619476+0000","last_undegraded":"2024-01-27T02:32:05.619476+0000","last_fullsized":"2024-01-27T02:32:05.619476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615596+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.623827+0000","last_change":"2024-01-27T02:32:05.623827+0000","last_active":"2024-01-27T02:32:05.623827+0000","last_peered":"2024-01-27T02:32:05.623827+0000","last_clean":"2024-01-27T02:32:05.623827+0000","last_became_active":"2024-01-27T02:32:05.623602+0000","last_became_peered":"2024-01-27T02:32:05.623602+0000","last_unstale":"2024-01-27T02:32:05.623827+0000","last_undegraded":"2024-01-27T02:32:05.623827+0000","last_fullsized":"2024-01-27T02:32:05.623827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613756+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.621283+0000","last_change":"2024-01-27T02:32:05.621283+0000","last_active":"2024-01-27T02:32:05.621283+0000","last_peered":"2024-01-27T02:32:05.621283+0000","last_clean":"2024-01-27T02:32:05.621283+0000","last_became_active":"2024-01-27T02:32:05.621034+0000","last_became_peered":"2024-01-27T02:32:05.621034+0000","last_unstale":"2024-01-27T02:32:05.621283+0000","last_undegraded":"2024-01-27T02:32:05.621283+0000","last_fullsized":"2024-01-27T02:32:05.621283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:07.586396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614840+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:42.580188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.620524+0000","last_change":"2024-01-27T02:32:05.620524+0000","last_active":"2024-01-27T02:32:05.620524+0000","last_peered":"2024-01-27T02:32:05.620524+0000","last_clean":"2024-01-27T02:32:05.620524+0000","last_became_active":"2024-01-27T02:32:05.620169+0000","last_became_peered":"2024-01-27T02:32:05.620169+0000","last_unstale":"2024-01-27T02:32:05.620524+0000","last_undegraded":"2024-01-27T02:32:05.620524+0000","last_fullsized":"2024-01-27T02:32:05.620524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612315+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:53.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.625720+0000","last_change":"2024-01-27T02:32:05.625720+0000","last_active":"2024-01-27T02:32:05.625720+0000","last_peered":"2024-01-27T02:32:05.625720+0000","last_clean":"2024-01-27T02:32:05.625720+0000","last_became_active":"2024-01-27T02:32:05.625394+0000","last_became_peered":"2024-01-27T02:32:05.625394+0000","last_unstale":"2024-01-27T02:32:05.625720+0000","last_undegraded":"2024-01-27T02:32:05.625720+0000","last_fullsized":"2024-01-27T02:32:05.625720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.621410+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:16:23.225540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.619745+0000","last_change":"2024-01-27T02:32:05.619745+0000","last_active":"2024-01-27T02:32:05.619745+0000","last_peered":"2024-01-27T02:32:05.619745+0000","last_clean":"2024-01-27T02:32:05.619745+0000","last_became_active":"2024-01-27T02:32:05.619371+0000","last_became_peered":"2024-01-27T02:32:05.619371+0000","last_unstale":"2024-01-27T02:32:05.619745+0000","last_undegraded":"2024-01-27T02:32:05.619745+0000","last_fullsized":"2024-01-27T02:32:05.619745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:12:20.892542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614205+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:04:29.895427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.626895+0000","last_change":"2024-01-27T02:32:05.626895+0000","last_active":"2024-01-27T02:32:05.626895+0000","last_peered":"2024-01-27T02:32:05.626895+0000","last_clean":"2024-01-27T02:32:05.626895+0000","last_became_active":"2024-01-27T02:32:05.626682+0000","last_became_peered":"2024-01-27T02:32:05.626682+0000","last_unstale":"2024-01-27T02:32:05.626895+0000","last_undegraded":"2024-01-27T02:32:05.626895+0000","last_fullsized":"2024-01-27T02:32:05.626895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:41.575605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.618952+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:11:54.230178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.619939+0000","last_change":"2024-01-27T02:32:05.619939+0000","last_active":"2024-01-27T02:32:05.619939+0000","last_peered":"2024-01-27T02:32:05.619939+0000","last_clean":"2024-01-27T02:32:05.619939+0000","last_became_active":"2024-01-27T02:32:05.619489+0000","last_became_peered":"2024-01-27T02:32:05.619489+0000","last_unstale":"2024-01-27T02:32:05.619939+0000","last_undegraded":"2024-01-27T02:32:05.619939+0000","last_fullsized":"2024-01-27T02:32:05.619939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:07.586396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615600+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:58:54.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.621851+0000","last_change":"2024-01-27T02:32:05.621851+0000","last_active":"2024-01-27T02:32:05.621851+0000","last_peered":"2024-01-27T02:32:05.621851+0000","last_clean":"2024-01-27T02:32:05.621851+0000","last_became_active":"2024-01-27T02:32:05.621487+0000","last_became_peered":"2024-01-27T02:32:05.621487+0000","last_unstale":"2024-01-27T02:32:05.621851+0000","last_undegraded":"2024-01-27T02:32:05.621851+0000","last_fullsized":"2024-01-27T02:32:05.621851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:07.586396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616380+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:06.985231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.623147+0000","last_change":"2024-01-27T02:32:05.623147+0000","last_active":"2024-01-27T02:32:05.623147+0000","last_peered":"2024-01-27T02:32:05.623147+0000","last_clean":"2024-01-27T02:32:05.623147+0000","last_became_active":"2024-01-27T02:32:05.622650+0000","last_became_peered":"2024-01-27T02:32:05.622650+0000","last_unstale":"2024-01-27T02:32:05.623147+0000","last_undegraded":"2024-01-27T02:32:05.623147+0000","last_fullsized":"2024-01-27T02:32:05.623147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:30:53.128829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615959+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.621921+0000","last_change":"2024-01-27T02:32:05.621921+0000","last_active":"2024-01-27T02:32:05.621921+0000","last_peered":"2024-01-27T02:32:05.621921+0000","last_clean":"2024-01-27T02:32:05.621921+0000","last_became_active":"2024-01-27T02:32:05.620450+0000","last_became_peered":"2024-01-27T02:32:05.620450+0000","last_unstale":"2024-01-27T02:32:05.621921+0000","last_undegraded":"2024-01-27T02:32:05.621921+0000","last_fullsized":"2024-01-27T02:32:05.621921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:33.333681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613665+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.622783+0000","last_change":"2024-01-27T02:32:05.622783+0000","last_active":"2024-01-27T02:32:05.622783+0000","last_peered":"2024-01-27T02:32:05.622783+0000","last_clean":"2024-01-27T02:32:05.622783+0000","last_became_active":"2024-01-27T02:32:05.622557+0000","last_became_peered":"2024-01-27T02:32:05.622557+0000","last_unstale":"2024-01-27T02:32:05.622783+0000","last_undegraded":"2024-01-27T02:32:05.622783+0000","last_fullsized":"2024-01-27T02:32:05.622783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612935+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:04:29.895427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615166+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.613447+0000","last_change":"2024-01-27T02:32:05.613447+0000","last_active":"2024-01-27T02:32:05.613447+0000","last_peered":"2024-01-27T02:32:05.613447+0000","last_clean":"2024-01-27T02:32:05.613447+0000","last_became_active":"2024-01-27T02:32:05.613203+0000","last_became_peered":"2024-01-27T02:32:05.613203+0000","last_unstale":"2024-01-27T02:32:05.613447+0000","last_undegraded":"2024-01-27T02:32:05.613447+0000","last_fullsized":"2024-01-27T02:32:05.613447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:41.575605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616703+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:47.819053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.622673+0000","last_change":"2024-01-27T02:32:05.622673+0000","last_active":"2024-01-27T02:32:05.622673+0000","last_peered":"2024-01-27T02:32:05.622673+0000","last_clean":"2024-01-27T02:32:05.622673+0000","last_became_active":"2024-01-27T02:32:05.622195+0000","last_became_peered":"2024-01-27T02:32:05.622195+0000","last_unstale":"2024-01-27T02:32:05.622673+0000","last_undegraded":"2024-01-27T02:32:05.622673+0000","last_fullsized":"2024-01-27T02:32:05.622673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":161,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-01-27T02:32:04.612144+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:32:04.612144+0000","last_peered":"2024-01-27T02:32:04.612144+0000","last_clean":"2024-01-27T02:32:04.612144+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:32:04.612144+0000","last_undegraded":"2024-01-27T02:32:04.612144+0000","last_fullsized":"2024-01-27T02:32:04.612144+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616221+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.626552+0000","last_change":"2024-01-27T02:32:05.626552+0000","last_active":"2024-01-27T02:32:05.626552+0000","last_peered":"2024-01-27T02:32:05.626552+0000","last_clean":"2024-01-27T02:32:05.626552+0000","last_became_active":"2024-01-27T02:32:05.626348+0000","last_became_peered":"2024-01-27T02:32:05.626348+0000","last_unstale":"2024-01-27T02:32:05.626552+0000","last_undegraded":"2024-01-27T02:32:05.626552+0000","last_fullsized":"2024-01-27T02:32:05.626552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:18.034484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.625885+0000","last_change":"2024-01-27T02:32:05.625885+0000","last_active":"2024-01-27T02:32:05.625885+0000","last_peered":"2024-01-27T02:32:05.625885+0000","last_clean":"2024-01-27T02:32:05.625885+0000","last_became_active":"2024-01-27T02:32:05.622692+0000","last_became_peered":"2024-01-27T02:32:05.622692+0000","last_unstale":"2024-01-27T02:32:05.625885+0000","last_undegraded":"2024-01-27T02:32:05.625885+0000","last_fullsized":"2024-01-27T02:32:05.625885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:06.920232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616950+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:58:54.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.621014+0000","last_change":"2024-01-27T02:32:05.621014+0000","last_active":"2024-01-27T02:32:05.621014+0000","last_peered":"2024-01-27T02:32:05.621014+0000","last_clean":"2024-01-27T02:32:05.621014+0000","last_became_active":"2024-01-27T02:32:05.620775+0000","last_became_peered":"2024-01-27T02:32:05.620775+0000","last_unstale":"2024-01-27T02:32:05.621014+0000","last_undegraded":"2024-01-27T02:32:05.621014+0000","last_fullsized":"2024-01-27T02:32:05.621014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617909+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.624870+0000","last_change":"2024-01-27T02:32:05.624870+0000","last_active":"2024-01-27T02:32:05.624870+0000","last_peered":"2024-01-27T02:32:05.624870+0000","last_clean":"2024-01-27T02:32:05.624870+0000","last_became_active":"2024-01-27T02:32:05.624644+0000","last_became_peered":"2024-01-27T02:32:05.624644+0000","last_unstale":"2024-01-27T02:32:05.624870+0000","last_undegraded":"2024-01-27T02:32:05.624870+0000","last_fullsized":"2024-01-27T02:32:05.624870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616980+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.621287+0000","last_change":"2024-01-27T02:32:05.621287+0000","last_active":"2024-01-27T02:32:05.621287+0000","last_peered":"2024-01-27T02:32:05.621287+0000","last_clean":"2024-01-27T02:32:05.621287+0000","last_became_active":"2024-01-27T02:32:05.620002+0000","last_became_peered":"2024-01-27T02:32:05.620002+0000","last_unstale":"2024-01-27T02:32:05.621287+0000","last_undegraded":"2024-01-27T02:32:05.621287+0000","last_fullsized":"2024-01-27T02:32:05.621287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:44.489465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612241+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:44:53.933990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.623449+0000","last_change":"2024-01-27T02:32:05.623449+0000","last_active":"2024-01-27T02:32:05.623449+0000","last_peered":"2024-01-27T02:32:05.623449+0000","last_clean":"2024-01-27T02:32:05.623449+0000","last_became_active":"2024-01-27T02:32:05.623193+0000","last_became_peered":"2024-01-27T02:32:05.623193+0000","last_unstale":"2024-01-27T02:32:05.623449+0000","last_undegraded":"2024-01-27T02:32:05.623449+0000","last_fullsized":"2024-01-27T02:32:05.623449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:44.489465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616309+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:08.590979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.625193+0000","last_change":"2024-01-27T02:32:05.625193+0000","last_active":"2024-01-27T02:32:05.625193+0000","last_peered":"2024-01-27T02:32:05.625193+0000","last_clean":"2024-01-27T02:32:05.625193+0000","last_became_active":"2024-01-27T02:32:05.624951+0000","last_became_peered":"2024-01-27T02:32:05.624951+0000","last_unstale":"2024-01-27T02:32:05.625193+0000","last_undegraded":"2024-01-27T02:32:05.625193+0000","last_fullsized":"2024-01-27T02:32:05.625193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:04:28.890844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616904+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:31.534822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.625673+0000","last_change":"2024-01-27T02:32:05.625673+0000","last_active":"2024-01-27T02:32:05.625673+0000","last_peered":"2024-01-27T02:32:05.625673+0000","last_clean":"2024-01-27T02:32:05.625673+0000","last_became_active":"2024-01-27T02:32:05.623551+0000","last_became_peered":"2024-01-27T02:32:05.623551+0000","last_unstale":"2024-01-27T02:32:05.625673+0000","last_undegraded":"2024-01-27T02:32:05.625673+0000","last_fullsized":"2024-01-27T02:32:05.625673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614474+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.620877+0000","last_change":"2024-01-27T02:32:05.620877+0000","last_active":"2024-01-27T02:32:05.620877+0000","last_peered":"2024-01-27T02:32:05.620877+0000","last_clean":"2024-01-27T02:32:05.620877+0000","last_became_active":"2024-01-27T02:32:05.620600+0000","last_became_peered":"2024-01-27T02:32:05.620600+0000","last_unstale":"2024-01-27T02:32:05.620877+0000","last_undegraded":"2024-01-27T02:32:05.620877+0000","last_fullsized":"2024-01-27T02:32:05.620877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615243+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.619486+0000","last_change":"2024-01-27T02:32:05.619486+0000","last_active":"2024-01-27T02:32:05.619486+0000","last_peered":"2024-01-27T02:32:05.619486+0000","last_clean":"2024-01-27T02:32:05.619486+0000","last_became_active":"2024-01-27T02:32:05.619147+0000","last_became_peered":"2024-01-27T02:32:05.619147+0000","last_unstale":"2024-01-27T02:32:05.619486+0000","last_undegraded":"2024-01-27T02:32:05.619486+0000","last_fullsized":"2024-01-27T02:32:05.619486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613357+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.613146+0000","last_change":"2024-01-27T02:32:05.613146+0000","last_active":"2024-01-27T02:32:05.613146+0000","last_peered":"2024-01-27T02:32:05.613146+0000","last_clean":"2024-01-27T02:32:05.613146+0000","last_became_active":"2024-01-27T02:32:05.612852+0000","last_became_peered":"2024-01-27T02:32:05.612852+0000","last_unstale":"2024-01-27T02:32:05.613146+0000","last_undegraded":"2024-01-27T02:32:05.613146+0000","last_fullsized":"2024-01-27T02:32:05.613146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617656+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:45:37.035302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616731+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612111+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616985+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:44:13.569068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615702+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:31.534822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617415+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:45.494048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613738+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614627+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.620974+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613231+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617923+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614822+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:58:54.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615379+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:47.819053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613678+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":631,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194681,"statfs":{"total":200000000,"available":199353793,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":637,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194675,"statfs":{"total":200000000,"available":199347530,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]}]}],"pool_statfs":[]}} 2024-01-27T02:32:11.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3f is not active+clean 2024-01-27T02:32:11.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.616502+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T05:44:13.569068+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:32:11.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2024-01-27T02:32:11.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.617845+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:32:11.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3d is not active+clean 2024-01-27T02:32:11.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.617854+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:32:11.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3c is not active+clean 2024-01-27T02:32:11.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.614017+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:32:11.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3b is not active+clean 2024-01-27T02:32:11.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.619412+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T09:32:24.198014+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:32:11.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3a is not active+clean 2024-01-27T02:32:11.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.618664+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T05:45:37.035302+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:32:11.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.39 is not active+clean 2024-01-27T02:32:11.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.616761+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T06:17:03.351766+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:32:11.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2024-01-27T02:32:11.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.622881+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:32:11.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.37 is not active+clean 2024-01-27T02:32:11.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.615495+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:32:11.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.36 is not active+clean 2024-01-27T02:32:11.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.614084+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:32:11.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.35 is not active+clean 2024-01-27T02:32:11.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.615294+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:32:11.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.34 is not active+clean 2024-01-27T02:32:11.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.612594+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:59:08.590979+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:32:11.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.33 is not active+clean 2024-01-27T02:32:11.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.621489+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:32:11.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.32 is not active+clean 2024-01-27T02:32:11.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.615194+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:59:08.590979+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:32:11.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2024-01-27T02:32:11.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.617367+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:32:11.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.30 is not active+clean 2024-01-27T02:32:11.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.616115+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T06:48:06.985231+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:32:11.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2f is not active+clean 2024-01-27T02:32:11.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.615481+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:32:11.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2e is not active+clean 2024-01-27T02:32:11.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.612774+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:32:11.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2d is not active+clean 2024-01-27T02:32:11.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.617263+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T03:36:11.295485+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:32:11.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2024-01-27T02:32:11.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.622620+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:32:11.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2024-01-27T02:32:11.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.617401+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:32:11.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2024-01-27T02:32:11.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.615102+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:32:11.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2024-01-27T02:32:11.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.616282+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:32:11.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2024-01-27T02:32:11.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.612345+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:51:42.580188+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:32:11.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2024-01-27T02:32:11.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.614574+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T06:57:53.532825+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:32:11.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2024-01-27T02:32:11.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.619517+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:32:11.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2024-01-27T02:32:11.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.615596+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:32:11.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2024-01-27T02:32:11.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.613756+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:32:11.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2024-01-27T02:32:11.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.614840+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:51:42.580188+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:32:11.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2024-01-27T02:32:11.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.612315+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T06:57:53.532825+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:32:11.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2024-01-27T02:32:11.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.621410+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T12:16:23.225540+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:32:11.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2024-01-27T02:32:11.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.614205+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T09:04:29.895427+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:32:11.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2024-01-27T02:32:11.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.618952+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T12:11:54.230178+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:32:11.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2024-01-27T02:32:11.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.615600+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:58:54.122949+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:32:11.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2024-01-27T02:32:11.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.616380+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T06:48:06.985231+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:32:11.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2024-01-27T02:32:11.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.615959+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:32:11.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2024-01-27T02:32:11.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.613665+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:32:11.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2024-01-27T02:32:11.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.612935+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T09:04:29.895427+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:32:11.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2024-01-27T02:32:11.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.615166+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:32:11.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2024-01-27T02:32:11.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.616703+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:43:47.819053+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:32:11.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2024-01-27T02:32:11.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.616221+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:32:11.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2024-01-27T02:32:11.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.616950+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:58:54.122949+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:32:11.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2024-01-27T02:32:11.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.617909+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:32:11.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2024-01-27T02:32:11.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.616980+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:32:11.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2024-01-27T02:32:11.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.612241+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T03:44:53.933990+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:32:11.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2024-01-27T02:32:11.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.616309+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:59:08.590979+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:32:11.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2024-01-27T02:32:11.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.616904+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:46:31.534822+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:32:11.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2024-01-27T02:32:11.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.614474+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:32:11.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2024-01-27T02:32:11.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.615243+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:32:11.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2024-01-27T02:32:11.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.613357+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:32:11.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2024-01-27T02:32:11.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.617656+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T05:45:37.035302+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:32:11.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.20 is not active+clean 2024-01-27T02:32:11.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.616731+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:32:11.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.21 is not active+clean 2024-01-27T02:32:11.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.612111+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:32:11.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.22 is not active+clean 2024-01-27T02:32:11.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.616985+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T05:44:13.569068+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:32:11.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.23 is not active+clean 2024-01-27T02:32:11.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.615702+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:46:31.534822+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:32:11.818 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.24 is not active+clean 2024-01-27T02:32:11.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.617415+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T08:49:45.494048+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:32:11.818 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.25 is not active+clean 2024-01-27T02:32:11.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.613738+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:32:11.818 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.26 is not active+clean 2024-01-27T02:32:11.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.614627+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:32:11.818 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2024-01-27T02:32:11.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.620974+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:32:11.818 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2024-01-27T02:32:11.819 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.613231+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:32:11.819 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2024-01-27T02:32:11.819 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.617923+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:32:11.819 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2a is not active+clean 2024-01-27T02:32:11.819 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.614822+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:58:54.122949+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:32:11.819 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2b is not active+clean 2024-01-27T02:32:11.819 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.615379+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:43:47.819053+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:32:11.819 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2c is not active+clean 2024-01-27T02:32:11.819 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:32:05.607209+0000', 'last_change': '2024-01-27T02:32:05.613678+0000', 'last_active': '2024-01-27T02:32:05.607209+0000', 'last_peered': '2024-01-27T02:32:05.607209+0000', 'last_clean': '2024-01-27T02:32:05.607209+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:32:05.607209+0000', 'last_undegraded': '2024-01-27T02:32:05.607209+0000', 'last_fullsized': '2024-01-27T02:32:05.607209+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'last_clean_scrub_stamp': '2024-01-27T02:32:05.607209+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:32:11.820 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-01-27T02:32:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.976+0000 7f100b0fc700 1 Processor -- start 2024-01-27T02:32:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.976+0000 7f100b0fc700 1 -- start start 2024-01-27T02:32:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.976+0000 7f100b0fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1004157f80 0x7f100406e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:11.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.976+0000 7f100b0fc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f100406e7e0 0x7f10040c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:11.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.976+0000 7f100b0fc700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1004151840 con 0x7f1004139f50 2024-01-27T02:32:11.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.977+0000 7f100b0fc700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f100406ec60 con 0x7f1004157f80 2024-01-27T02:32:11.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.977+0000 7f100b0fc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10040c85a0 con 0x7f100406e7e0 2024-01-27T02:32:11.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.977+0000 7f1003fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1004157f80 0x7f100406e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:11.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.977+0000 7f1008e98700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1004139f50 0x7f100406bd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51328/0 (socket says 172.21.15.7:51328) 2024-01-27T02:32:11.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.977+0000 7f1008e98700 1 -- 172.21.15.7:0/4097613494 learned_addr learned my addr 172.21.15.7:0/4097613494 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:11.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.977+0000 7f1009699700 1 --2- 172.21.15.7:0/4097613494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f100406e7e0 0x7f10040c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:11.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.977+0000 7f10037fe700 1 -- 172.21.15.7:0/4097613494 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1433775194 0 0) 0x7f0ff4001290 con 0x7f1004139f50 2024-01-27T02:32:11.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.977+0000 7f10037fe700 1 -- 172.21.15.7:0/4097613494 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fec003570 con 0x7f1004139f50 2024-01-27T02:32:11.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.977+0000 7f1003fff700 1 -- 172.21.15.7:0/4097613494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f100406e7e0 msgr2=0x7f10040c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:11.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.977+0000 7f1003fff700 1 --2- 172.21.15.7:0/4097613494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f100406e7e0 0x7f10040c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.977+0000 7f1003fff700 1 -- 172.21.15.7:0/4097613494 >> v1:172.21.15.7:6789/0 conn(0x7f1004139f50 legacy=0x7f100406bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.977+0000 7f1003fff700 1 -- 172.21.15.7:0/4097613494 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100406c580 con 0x7f1004157f80 2024-01-27T02:32:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.977+0000 7f1003fff700 1 --2- 172.21.15.7:0/4097613494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1004157f80 0x7f100406e2a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0ff80098e0 tx=0x7f0ff80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d734c9de38a55b7 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.977+0000 7f10037fe700 1 -- 172.21.15.7:0/4097613494 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8011630 con 0x7f1004157f80 2024-01-27T02:32:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.977+0000 7f10037fe700 1 -- 172.21.15.7:0/4097613494 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff8011790 con 0x7f1004157f80 2024-01-27T02:32:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.977+0000 7f10037fe700 1 -- 172.21.15.7:0/4097613494 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8011b30 con 0x7f1004157f80 2024-01-27T02:32:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.978+0000 7f100b0fc700 1 -- 172.21.15.7:0/4097613494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1004157f80 msgr2=0x7f100406e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.978+0000 7f100b0fc700 1 --2- 172.21.15.7:0/4097613494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1004157f80 0x7f100406e2a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0ff80098e0 tx=0x7f0ff80091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.978+0000 7f100b0fc700 1 -- 172.21.15.7:0/4097613494 shutdown_connections 2024-01-27T02:32:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.978+0000 7f100b0fc700 1 --2- 172.21.15.7:0/4097613494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f100406e7e0 0x7f10040c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.978+0000 7f100b0fc700 1 --2- 172.21.15.7:0/4097613494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1004157f80 0x7f100406e2a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.978+0000 7f100b0fc700 1 -- 172.21.15.7:0/4097613494 >> 172.21.15.7:0/4097613494 conn(0x7f10040cdbf0 msgr2=0x7f10040cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.978+0000 7f100b0fc700 1 -- 172.21.15.7:0/4097613494 shutdown_connections 2024-01-27T02:32:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.978+0000 7f100b0fc700 1 -- 172.21.15.7:0/4097613494 wait complete. 2024-01-27T02:32:11.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.979+0000 7f100b0fc700 1 Processor -- start 2024-01-27T02:32:11.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.980+0000 7f100b0fc700 1 -- start start 2024-01-27T02:32:11.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.980+0000 7f100b0fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f100406e7e0 0x7f10041564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:11.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.980+0000 7f100b0fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1004139f50 0x7f100414c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:11.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.980+0000 7f100b0fc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1004157f80 0x7f10040c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:11.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.980+0000 7f100b0fc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1004138e20 con 0x7f1004139f50 2024-01-27T02:32:11.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.980+0000 7f100b0fc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10040c5560 con 0x7f100406e7e0 2024-01-27T02:32:11.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.980+0000 7f100b0fc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10040c5840 con 0x7f1004157f80 2024-01-27T02:32:11.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.980+0000 7f1003fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1004139f50 0x7f100414c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:11.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.980+0000 7f1003fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1004139f50 0x7f100414c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53560/0 (socket says 172.21.15.7:53560) 2024-01-27T02:32:11.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.980+0000 7f1003fff700 1 -- 172.21.15.7:0/209959235 learned_addr learned my addr 172.21.15.7:0/209959235 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:11.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.980+0000 7f1009699700 1 --2- 172.21.15.7:0/209959235 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1004157f80 0x7f10040c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:11.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.981+0000 7f1008e98700 1 --2- 172.21.15.7:0/209959235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f100406e7e0 0x7f10041564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:11.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.981+0000 7f1003fff700 1 -- 172.21.15.7:0/209959235 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1004157f80 msgr2=0x7f10040c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:11.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.981+0000 7f1003fff700 1 --2- 172.21.15.7:0/209959235 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1004157f80 0x7f10040c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:11.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.981+0000 7f1003fff700 1 -- 172.21.15.7:0/209959235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f100406e7e0 msgr2=0x7f10041564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:11.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.981+0000 7f1003fff700 1 --2- 172.21.15.7:0/209959235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f100406e7e0 0x7f10041564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:11.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.981+0000 7f1003fff700 1 -- 172.21.15.7:0/209959235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff8009420 con 0x7f1004139f50 2024-01-27T02:32:11.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.981+0000 7f1003fff700 1 --2- 172.21.15.7:0/209959235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1004139f50 0x7f100414c400 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1004069d90 tx=0x7f0ff8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:11.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.981+0000 7f1000ff9700 1 -- 172.21.15.7:0/209959235 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8000d10 con 0x7f1004139f50 2024-01-27T02:32:11.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.981+0000 7f1000ff9700 1 -- 172.21.15.7:0/209959235 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff8011900 con 0x7f1004139f50 2024-01-27T02:32:11.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.981+0000 7f100b0fc700 1 -- 172.21.15.7:0/209959235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f100414bc00 con 0x7f1004139f50 2024-01-27T02:32:11.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.981+0000 7f1000ff9700 1 -- 172.21.15.7:0/209959235 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8025420 con 0x7f1004139f50 2024-01-27T02:32:11.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.981+0000 7f100b0fc700 1 -- 172.21.15.7:0/209959235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f100414c940 con 0x7f1004139f50 2024-01-27T02:32:11.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.982+0000 7f100b0fc700 1 -- 172.21.15.7:0/209959235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f100404c710 con 0x7f1004139f50 2024-01-27T02:32:11.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.983+0000 7f1000ff9700 1 -- 172.21.15.7:0/209959235 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0ff8023070 con 0x7f1004139f50 2024-01-27T02:32:11.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.983+0000 7f1000ff9700 1 --2- 172.21.15.7:0/209959235 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0ff0066f30 0x7f0ff00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:11.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.983+0000 7f1000ff9700 1 -- 172.21.15.7:0/209959235 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0ff8085880 con 0x7f1004139f50 2024-01-27T02:32:11.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.983+0000 7f1008e98700 1 --2- 172.21.15.7:0/209959235 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0ff0066f30 0x7f0ff00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:11.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.984+0000 7f1008e98700 1 --2- 172.21.15.7:0/209959235 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0ff0066f30 0x7f0ff00693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40013f0 tx=0x7f0ff4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:11.988+0000 7f1000ff9700 1 -- 172.21.15.7:0/209959235 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ff8052ba0 con 0x7f1004139f50 2024-01-27T02:32:12.155 INFO:tasks.ceph.mds.d.smithi188.stderr:2024-01-27T02:32:12.154+0000 7ff55429a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-27T02:32:12.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.278+0000 7f100b0fc700 1 -- 172.21.15.7:0/209959235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f100406c860 con 0x7f1004139f50 2024-01-27T02:32:12.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.279+0000 7f1000ff9700 1 -- 172.21.15.7:0/209959235 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1027 (secure 0 0 0) 0x7f0ff802a420 con 0x7f1004139f50 2024-01-27T02:32:12.280 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:12.280 INFO:teuthology.orchestra.run.smithi007.stdout:{"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":22,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":21,"num_osds":8,"num_up_osds":8,"osd_up_since":1706322715,"num_in_osds":8,"osd_in_since":1706322712,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":1671289,"bytes_avail":1598328711,"bytes_total":1600000000,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4273}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-01-27T02:31:50.017785+0000","services":{}},"progress_events":{}} 2024-01-27T02:32:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.281+0000 7f100b0fc700 1 -- 172.21.15.7:0/209959235 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0ff0066f30 msgr2=0x7f0ff00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.281+0000 7f100b0fc700 1 --2- 172.21.15.7:0/209959235 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0ff0066f30 0x7f0ff00693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40013f0 tx=0x7f0ff4009040 comp rx=0 tx=0).stop 2024-01-27T02:32:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.281+0000 7f100b0fc700 1 -- 172.21.15.7:0/209959235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1004139f50 msgr2=0x7f100414c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.281+0000 7f100b0fc700 1 --2- 172.21.15.7:0/209959235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1004139f50 0x7f100414c400 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1004069d90 tx=0x7f0ff8017000 comp rx=0 tx=0).stop 2024-01-27T02:32:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.281+0000 7f100b0fc700 1 -- 172.21.15.7:0/209959235 shutdown_connections 2024-01-27T02:32:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.281+0000 7f100b0fc700 1 --2- 172.21.15.7:0/209959235 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0ff0066f30 0x7f0ff00693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.281+0000 7f100b0fc700 1 --2- 172.21.15.7:0/209959235 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1004157f80 0x7f10040c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:12.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.281+0000 7f100b0fc700 1 --2- 172.21.15.7:0/209959235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f100406e7e0 0x7f10041564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:12.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.281+0000 7f100b0fc700 1 --2- 172.21.15.7:0/209959235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1004139f50 0x7f100414c400 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:12.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.281+0000 7f100b0fc700 1 -- 172.21.15.7:0/209959235 >> 172.21.15.7:0/209959235 conn(0x7f10040cdbf0 msgr2=0x7f1004158fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:12.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.281+0000 7f100b0fc700 1 -- 172.21.15.7:0/209959235 shutdown_connections 2024-01-27T02:32:12.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.281+0000 7f100b0fc700 1 -- 172.21.15.7:0/209959235 wait complete. 2024-01-27T02:32:12.296 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-01-27T02:32:12.296 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:32:12.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.427+0000 7f731ca40700 1 Processor -- start 2024-01-27T02:32:12.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.427+0000 7f731ca40700 1 -- start start 2024-01-27T02:32:12.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.428+0000 7f731ca40700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7318157ca0 0x7f73180c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:12.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.428+0000 7f731ca40700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f73180c3ae0 0x7f73180cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.428+0000 7f731ca40700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73180ca020 con 0x7f7318158070 2024-01-27T02:32:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.428+0000 7f731ca40700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73180c4f40 con 0x7f7318157ca0 2024-01-27T02:32:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.428+0000 7f731ca40700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73180c5210 con 0x7f73180c3ae0 2024-01-27T02:32:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.428+0000 7f731659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7318157ca0 0x7f73180c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.428+0000 7f731659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7318157ca0 0x7f73180c4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53574/0 (socket says 172.21.15.7:53574) 2024-01-27T02:32:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.428+0000 7f731659c700 1 -- 172.21.15.7:0/2070838991 learned_addr learned my addr 172.21.15.7:0/2070838991 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.428+0000 7f7315d9b700 1 --2- 172.21.15.7:0/2070838991 >> v2:172.21.15.188:3300/0 conn(0x7f73180c3ae0 0x7f73180cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.428+0000 7f731659c700 1 -- 172.21.15.7:0/2070838991 >> v2:172.21.15.188:3300/0 conn(0x7f73180c3ae0 msgr2=0x7f73180cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.428+0000 7f731659c700 1 --2- 172.21.15.7:0/2070838991 >> v2:172.21.15.188:3300/0 conn(0x7f73180c3ae0 0x7f73180cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.428+0000 7f731659c700 1 -- 172.21.15.7:0/2070838991 >> v1:172.21.15.188:6789/0 conn(0x7f7318158070 legacy=0x7f731815bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.428+0000 7f731659c700 1 -- 172.21.15.7:0/2070838991 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73180c54f0 con 0x7f7318157ca0 2024-01-27T02:32:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.429+0000 7f731659c700 1 --2- 172.21.15.7:0/2070838991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7318157ca0 0x7f73180c4a00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f731806e170 tx=0x7f7300009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e26f06344574f0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.429+0000 7f731559a700 1 -- 172.21.15.7:0/2070838991 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7300009e70 con 0x7f7318157ca0 2024-01-27T02:32:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.429+0000 7f731559a700 1 -- 172.21.15.7:0/2070838991 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730000b860 con 0x7f7318157ca0 2024-01-27T02:32:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.429+0000 7f731559a700 1 -- 172.21.15.7:0/2070838991 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730000bc00 con 0x7f7318157ca0 2024-01-27T02:32:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.429+0000 7f731ca40700 1 -- 172.21.15.7:0/2070838991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7318157ca0 msgr2=0x7f73180c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.429+0000 7f731ca40700 1 --2- 172.21.15.7:0/2070838991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7318157ca0 0x7f73180c4a00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f731806e170 tx=0x7f7300009250 comp rx=0 tx=0).stop 2024-01-27T02:32:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.429+0000 7f731ca40700 1 -- 172.21.15.7:0/2070838991 shutdown_connections 2024-01-27T02:32:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.429+0000 7f731ca40700 1 --2- 172.21.15.7:0/2070838991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7318157ca0 0x7f73180c4a00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.429+0000 7f731ca40700 1 --2- 172.21.15.7:0/2070838991 >> v2:172.21.15.188:3300/0 conn(0x7f73180c3ae0 0x7f73180cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.429+0000 7f731ca40700 1 -- 172.21.15.7:0/2070838991 >> 172.21.15.7:0/2070838991 conn(0x7f7318098020 msgr2=0x7f73180711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.430+0000 7f731ca40700 1 -- 172.21.15.7:0/2070838991 shutdown_connections 2024-01-27T02:32:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.430+0000 7f731ca40700 1 -- 172.21.15.7:0/2070838991 wait complete. 2024-01-27T02:32:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.431+0000 7f731ca40700 1 Processor -- start 2024-01-27T02:32:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.431+0000 7f731ca40700 1 -- start start 2024-01-27T02:32:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f731ca40700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73180c3ae0 0x7f7318150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f731ca40700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7318157ca0 0x7f7318156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f731ca40700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7318158070 0x7f73181570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f731ca40700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7318156120 con 0x7f73180c3ae0 2024-01-27T02:32:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f731ca40700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73180c6020 con 0x7f7318158070 2024-01-27T02:32:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f731ca40700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7318155510 con 0x7f7318157ca0 2024-01-27T02:32:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f731659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73180c3ae0 0x7f7318150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f731659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73180c3ae0 0x7f7318150060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53582/0 (socket says 172.21.15.7:53582) 2024-01-27T02:32:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f731659c700 1 -- 172.21.15.7:0/1592953013 learned_addr learned my addr 172.21.15.7:0/1592953013 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f7315d9b700 1 --2- 172.21.15.7:0/1592953013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7318157ca0 0x7f7318156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f7316d9d700 1 --2- 172.21.15.7:0/1592953013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7318158070 0x7f73181570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f731659c700 1 -- 172.21.15.7:0/1592953013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7318157ca0 msgr2=0x7f7318156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f731659c700 1 --2- 172.21.15.7:0/1592953013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7318157ca0 0x7f7318156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f731659c700 1 -- 172.21.15.7:0/1592953013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7318158070 msgr2=0x7f73181570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f731659c700 1 --2- 172.21.15.7:0/1592953013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7318158070 0x7f73181570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f731659c700 1 -- 172.21.15.7:0/1592953013 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73000094f0 con 0x7f73180c3ae0 2024-01-27T02:32:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f7315d9b700 1 --2- 172.21.15.7:0/1592953013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7318157ca0 0x7f7318156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f7316d9d700 1 --2- 172.21.15.7:0/1592953013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7318158070 0x7f73181570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f731659c700 1 --2- 172.21.15.7:0/1592953013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73180c3ae0 0x7f7318150060 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f731806ea80 tx=0x7f7300017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f730e7fc700 1 -- 172.21.15.7:0/1592953013 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7300011420 con 0x7f73180c3ae0 2024-01-27T02:32:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f730e7fc700 1 -- 172.21.15.7:0/1592953013 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7300011580 con 0x7f73180c3ae0 2024-01-27T02:32:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f731ca40700 1 -- 172.21.15.7:0/1592953013 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73181557f0 con 0x7f73180c3ae0 2024-01-27T02:32:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f730e7fc700 1 -- 172.21.15.7:0/1592953013 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7300011940 con 0x7f73180c3ae0 2024-01-27T02:32:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.432+0000 7f731ca40700 1 -- 172.21.15.7:0/1592953013 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7318155db0 con 0x7f73180c3ae0 2024-01-27T02:32:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.433+0000 7f731ca40700 1 -- 172.21.15.7:0/1592953013 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7318155f40 con 0x7f73180c3ae0 2024-01-27T02:32:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.434+0000 7f730e7fc700 1 -- 172.21.15.7:0/1592953013 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f730002c3b0 con 0x7f73180c3ae0 2024-01-27T02:32:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.434+0000 7f730e7fc700 1 --2- 172.21.15.7:0/1592953013 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f72f8066f30 0x7f72f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.434+0000 7f730e7fc700 1 -- 172.21.15.7:0/1592953013 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7300085be0 con 0x7f73180c3ae0 2024-01-27T02:32:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.434+0000 7f7315d9b700 1 --2- 172.21.15.7:0/1592953013 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f72f8066f30 0x7f72f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.435+0000 7f7315d9b700 1 --2- 172.21.15.7:0/1592953013 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f72f8066f30 0x7f72f80693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7308009d80 tx=0x7f7308008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:12.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.439+0000 7f730e7fc700 1 -- 172.21.15.7:0/1592953013 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7300052e50 con 0x7f73180c3ae0 2024-01-27T02:32:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.640+0000 7f731ca40700 1 -- 172.21.15.7:0/1592953013 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7318156260 con 0x7f72f8066f30 2024-01-27T02:32:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.648+0000 7f730e7fc700 1 -- 172.21.15.7:0/1592953013 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+257304 (secure 0 0 0) 0x7f7308009aa0 con 0x7f72f8066f30 2024-01-27T02:32:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.651+0000 7f731ca40700 1 -- 172.21.15.7:0/1592953013 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f72f8066f30 msgr2=0x7f72f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.651+0000 7f731ca40700 1 --2- 172.21.15.7:0/1592953013 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f72f8066f30 0x7f72f80693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7308009d80 tx=0x7f7308008040 comp rx=0 tx=0).stop 2024-01-27T02:32:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.651+0000 7f731ca40700 1 -- 172.21.15.7:0/1592953013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73180c3ae0 msgr2=0x7f7318150060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.651+0000 7f731ca40700 1 --2- 172.21.15.7:0/1592953013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73180c3ae0 0x7f7318150060 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f731806ea80 tx=0x7f7300017000 comp rx=0 tx=0).stop 2024-01-27T02:32:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.651+0000 7f731ca40700 1 -- 172.21.15.7:0/1592953013 shutdown_connections 2024-01-27T02:32:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.651+0000 7f731ca40700 1 --2- 172.21.15.7:0/1592953013 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f72f8066f30 0x7f72f80693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.651+0000 7f731ca40700 1 --2- 172.21.15.7:0/1592953013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7318158070 0x7f73181570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.651+0000 7f731ca40700 1 --2- 172.21.15.7:0/1592953013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73180c3ae0 0x7f7318150060 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.651+0000 7f731ca40700 1 --2- 172.21.15.7:0/1592953013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7318157ca0 0x7f7318156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.651+0000 7f731ca40700 1 -- 172.21.15.7:0/1592953013 >> 172.21.15.7:0/1592953013 conn(0x7f7318098020 msgr2=0x7f731806bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.651+0000 7f731ca40700 1 -- 172.21.15.7:0/1592953013 shutdown_connections 2024-01-27T02:32:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:12.651+0000 7f731ca40700 1 -- 172.21.15.7:0/1592953013 wait complete. 2024-01-27T02:32:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:32:12.666 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-01-27T02:32:11.573004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1632,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560867,"statfs":{"total":1600000000,"available":1598328711,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001678"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616502+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:44:13.569068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617845+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617854+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614017+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.619412+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:32:24.198014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.618664+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:45:37.035302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616761+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:17:03.351766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.622881+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615495+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614084+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615294+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612594+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:08.590979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.621489+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615194+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:08.590979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617367+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616115+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:06.985231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615481+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612774+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617263+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:36:11.295485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.624071+0000","last_change":"2024-01-27T02:32:05.624071+0000","last_active":"2024-01-27T02:32:05.624071+0000","last_peered":"2024-01-27T02:32:05.624071+0000","last_clean":"2024-01-27T02:32:05.624071+0000","last_became_active":"2024-01-27T02:32:05.623818+0000","last_became_peered":"2024-01-27T02:32:05.623818+0000","last_unstale":"2024-01-27T02:32:05.624071+0000","last_undegraded":"2024-01-27T02:32:05.624071+0000","last_fullsized":"2024-01-27T02:32:05.624071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:34:05.180050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.622620+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.624866+0000","last_change":"2024-01-27T02:32:05.624866+0000","last_active":"2024-01-27T02:32:05.624866+0000","last_peered":"2024-01-27T02:32:05.624866+0000","last_clean":"2024-01-27T02:32:05.624866+0000","last_became_active":"2024-01-27T02:32:05.624368+0000","last_became_peered":"2024-01-27T02:32:05.624368+0000","last_unstale":"2024-01-27T02:32:05.624866+0000","last_undegraded":"2024-01-27T02:32:05.624866+0000","last_fullsized":"2024-01-27T02:32:05.624866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617401+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.623255+0000","last_change":"2024-01-27T02:32:05.623255+0000","last_active":"2024-01-27T02:32:05.623255+0000","last_peered":"2024-01-27T02:32:05.623255+0000","last_clean":"2024-01-27T02:32:05.623255+0000","last_became_active":"2024-01-27T02:32:05.622821+0000","last_became_peered":"2024-01-27T02:32:05.622821+0000","last_unstale":"2024-01-27T02:32:05.623255+0000","last_undegraded":"2024-01-27T02:32:05.623255+0000","last_fullsized":"2024-01-27T02:32:05.623255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:33.333681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615102+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.626014+0000","last_change":"2024-01-27T02:32:05.626014+0000","last_active":"2024-01-27T02:32:05.626014+0000","last_peered":"2024-01-27T02:32:05.626014+0000","last_clean":"2024-01-27T02:32:05.626014+0000","last_became_active":"2024-01-27T02:32:05.623176+0000","last_became_peered":"2024-01-27T02:32:05.623176+0000","last_unstale":"2024-01-27T02:32:05.626014+0000","last_undegraded":"2024-01-27T02:32:05.626014+0000","last_fullsized":"2024-01-27T02:32:05.626014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:37:59.246901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616282+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.622086+0000","last_change":"2024-01-27T02:32:05.622086+0000","last_active":"2024-01-27T02:32:05.622086+0000","last_peered":"2024-01-27T02:32:05.622086+0000","last_clean":"2024-01-27T02:32:05.622086+0000","last_became_active":"2024-01-27T02:32:05.621465+0000","last_became_peered":"2024-01-27T02:32:05.621465+0000","last_unstale":"2024-01-27T02:32:05.622086+0000","last_undegraded":"2024-01-27T02:32:05.622086+0000","last_fullsized":"2024-01-27T02:32:05.622086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:04:28.890844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612345+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:42.580188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.617530+0000","last_change":"2024-01-27T02:32:05.617530+0000","last_active":"2024-01-27T02:32:05.617530+0000","last_peered":"2024-01-27T02:32:05.617530+0000","last_clean":"2024-01-27T02:32:05.617530+0000","last_became_active":"2024-01-27T02:32:05.617290+0000","last_became_peered":"2024-01-27T02:32:05.617290+0000","last_unstale":"2024-01-27T02:32:05.617530+0000","last_undegraded":"2024-01-27T02:32:05.617530+0000","last_fullsized":"2024-01-27T02:32:05.617530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:41.575605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614574+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:53.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.623693+0000","last_change":"2024-01-27T02:32:05.623693+0000","last_active":"2024-01-27T02:32:05.623693+0000","last_peered":"2024-01-27T02:32:05.623693+0000","last_clean":"2024-01-27T02:32:05.623693+0000","last_became_active":"2024-01-27T02:32:05.623440+0000","last_became_peered":"2024-01-27T02:32:05.623440+0000","last_unstale":"2024-01-27T02:32:05.623693+0000","last_undegraded":"2024-01-27T02:32:05.623693+0000","last_fullsized":"2024-01-27T02:32:05.623693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:44:52.929407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.619517+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.619476+0000","last_change":"2024-01-27T02:32:05.619476+0000","last_active":"2024-01-27T02:32:05.619476+0000","last_peered":"2024-01-27T02:32:05.619476+0000","last_clean":"2024-01-27T02:32:05.619476+0000","last_became_active":"2024-01-27T02:32:05.619194+0000","last_became_peered":"2024-01-27T02:32:05.619194+0000","last_unstale":"2024-01-27T02:32:05.619476+0000","last_undegraded":"2024-01-27T02:32:05.619476+0000","last_fullsized":"2024-01-27T02:32:05.619476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615596+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.623827+0000","last_change":"2024-01-27T02:32:05.623827+0000","last_active":"2024-01-27T02:32:05.623827+0000","last_peered":"2024-01-27T02:32:05.623827+0000","last_clean":"2024-01-27T02:32:05.623827+0000","last_became_active":"2024-01-27T02:32:05.623602+0000","last_became_peered":"2024-01-27T02:32:05.623602+0000","last_unstale":"2024-01-27T02:32:05.623827+0000","last_undegraded":"2024-01-27T02:32:05.623827+0000","last_fullsized":"2024-01-27T02:32:05.623827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613756+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.621283+0000","last_change":"2024-01-27T02:32:05.621283+0000","last_active":"2024-01-27T02:32:05.621283+0000","last_peered":"2024-01-27T02:32:05.621283+0000","last_clean":"2024-01-27T02:32:05.621283+0000","last_became_active":"2024-01-27T02:32:05.621034+0000","last_became_peered":"2024-01-27T02:32:05.621034+0000","last_unstale":"2024-01-27T02:32:05.621283+0000","last_undegraded":"2024-01-27T02:32:05.621283+0000","last_fullsized":"2024-01-27T02:32:05.621283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:07.586396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614840+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:42.580188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.620524+0000","last_change":"2024-01-27T02:32:05.620524+0000","last_active":"2024-01-27T02:32:05.620524+0000","last_peered":"2024-01-27T02:32:05.620524+0000","last_clean":"2024-01-27T02:32:05.620524+0000","last_became_active":"2024-01-27T02:32:05.620169+0000","last_became_peered":"2024-01-27T02:32:05.620169+0000","last_unstale":"2024-01-27T02:32:05.620524+0000","last_undegraded":"2024-01-27T02:32:05.620524+0000","last_fullsized":"2024-01-27T02:32:05.620524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612315+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:53.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.625720+0000","last_change":"2024-01-27T02:32:05.625720+0000","last_active":"2024-01-27T02:32:05.625720+0000","last_peered":"2024-01-27T02:32:05.625720+0000","last_clean":"2024-01-27T02:32:05.625720+0000","last_became_active":"2024-01-27T02:32:05.625394+0000","last_became_peered":"2024-01-27T02:32:05.625394+0000","last_unstale":"2024-01-27T02:32:05.625720+0000","last_undegraded":"2024-01-27T02:32:05.625720+0000","last_fullsized":"2024-01-27T02:32:05.625720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.621410+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:16:23.225540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.619745+0000","last_change":"2024-01-27T02:32:05.619745+0000","last_active":"2024-01-27T02:32:05.619745+0000","last_peered":"2024-01-27T02:32:05.619745+0000","last_clean":"2024-01-27T02:32:05.619745+0000","last_became_active":"2024-01-27T02:32:05.619371+0000","last_became_peered":"2024-01-27T02:32:05.619371+0000","last_unstale":"2024-01-27T02:32:05.619745+0000","last_undegraded":"2024-01-27T02:32:05.619745+0000","last_fullsized":"2024-01-27T02:32:05.619745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:12:20.892542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614205+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:04:29.895427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.626895+0000","last_change":"2024-01-27T02:32:05.626895+0000","last_active":"2024-01-27T02:32:05.626895+0000","last_peered":"2024-01-27T02:32:05.626895+0000","last_clean":"2024-01-27T02:32:05.626895+0000","last_became_active":"2024-01-27T02:32:05.626682+0000","last_became_peered":"2024-01-27T02:32:05.626682+0000","last_unstale":"2024-01-27T02:32:05.626895+0000","last_undegraded":"2024-01-27T02:32:05.626895+0000","last_fullsized":"2024-01-27T02:32:05.626895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:41.575605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.618952+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:11:54.230178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.619939+0000","last_change":"2024-01-27T02:32:05.619939+0000","last_active":"2024-01-27T02:32:05.619939+0000","last_peered":"2024-01-27T02:32:05.619939+0000","last_clean":"2024-01-27T02:32:05.619939+0000","last_became_active":"2024-01-27T02:32:05.619489+0000","last_became_peered":"2024-01-27T02:32:05.619489+0000","last_unstale":"2024-01-27T02:32:05.619939+0000","last_undegraded":"2024-01-27T02:32:05.619939+0000","last_fullsized":"2024-01-27T02:32:05.619939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:07.586396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615600+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:58:54.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.621851+0000","last_change":"2024-01-27T02:32:05.621851+0000","last_active":"2024-01-27T02:32:05.621851+0000","last_peered":"2024-01-27T02:32:05.621851+0000","last_clean":"2024-01-27T02:32:05.621851+0000","last_became_active":"2024-01-27T02:32:05.621487+0000","last_became_peered":"2024-01-27T02:32:05.621487+0000","last_unstale":"2024-01-27T02:32:05.621851+0000","last_undegraded":"2024-01-27T02:32:05.621851+0000","last_fullsized":"2024-01-27T02:32:05.621851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:07.586396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616380+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:06.985231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.623147+0000","last_change":"2024-01-27T02:32:05.623147+0000","last_active":"2024-01-27T02:32:05.623147+0000","last_peered":"2024-01-27T02:32:05.623147+0000","last_clean":"2024-01-27T02:32:05.623147+0000","last_became_active":"2024-01-27T02:32:05.622650+0000","last_became_peered":"2024-01-27T02:32:05.622650+0000","last_unstale":"2024-01-27T02:32:05.623147+0000","last_undegraded":"2024-01-27T02:32:05.623147+0000","last_fullsized":"2024-01-27T02:32:05.623147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:30:53.128829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615959+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.621921+0000","last_change":"2024-01-27T02:32:05.621921+0000","last_active":"2024-01-27T02:32:05.621921+0000","last_peered":"2024-01-27T02:32:05.621921+0000","last_clean":"2024-01-27T02:32:05.621921+0000","last_became_active":"2024-01-27T02:32:05.620450+0000","last_became_peered":"2024-01-27T02:32:05.620450+0000","last_unstale":"2024-01-27T02:32:05.621921+0000","last_undegraded":"2024-01-27T02:32:05.621921+0000","last_fullsized":"2024-01-27T02:32:05.621921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:33.333681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613665+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.622783+0000","last_change":"2024-01-27T02:32:05.622783+0000","last_active":"2024-01-27T02:32:05.622783+0000","last_peered":"2024-01-27T02:32:05.622783+0000","last_clean":"2024-01-27T02:32:05.622783+0000","last_became_active":"2024-01-27T02:32:05.622557+0000","last_became_peered":"2024-01-27T02:32:05.622557+0000","last_unstale":"2024-01-27T02:32:05.622783+0000","last_undegraded":"2024-01-27T02:32:05.622783+0000","last_fullsized":"2024-01-27T02:32:05.622783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612935+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:04:29.895427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615166+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.613447+0000","last_change":"2024-01-27T02:32:05.613447+0000","last_active":"2024-01-27T02:32:05.613447+0000","last_peered":"2024-01-27T02:32:05.613447+0000","last_clean":"2024-01-27T02:32:05.613447+0000","last_became_active":"2024-01-27T02:32:05.613203+0000","last_became_peered":"2024-01-27T02:32:05.613203+0000","last_unstale":"2024-01-27T02:32:05.613447+0000","last_undegraded":"2024-01-27T02:32:05.613447+0000","last_fullsized":"2024-01-27T02:32:05.613447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:41.575605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616703+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:47.819053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.622673+0000","last_change":"2024-01-27T02:32:05.622673+0000","last_active":"2024-01-27T02:32:05.622673+0000","last_peered":"2024-01-27T02:32:05.622673+0000","last_clean":"2024-01-27T02:32:05.622673+0000","last_became_active":"2024-01-27T02:32:05.622195+0000","last_became_peered":"2024-01-27T02:32:05.622195+0000","last_unstale":"2024-01-27T02:32:05.622673+0000","last_undegraded":"2024-01-27T02:32:05.622673+0000","last_fullsized":"2024-01-27T02:32:05.622673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":161,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-01-27T02:32:04.612144+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:32:04.612144+0000","last_peered":"2024-01-27T02:32:04.612144+0000","last_clean":"2024-01-27T02:32:04.612144+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:32:04.612144+0000","last_undegraded":"2024-01-27T02:32:04.612144+0000","last_fullsized":"2024-01-27T02:32:04.612144+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616221+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.626552+0000","last_change":"2024-01-27T02:32:05.626552+0000","last_active":"2024-01-27T02:32:05.626552+0000","last_peered":"2024-01-27T02:32:05.626552+0000","last_clean":"2024-01-27T02:32:05.626552+0000","last_became_active":"2024-01-27T02:32:05.626348+0000","last_became_peered":"2024-01-27T02:32:05.626348+0000","last_unstale":"2024-01-27T02:32:05.626552+0000","last_undegraded":"2024-01-27T02:32:05.626552+0000","last_fullsized":"2024-01-27T02:32:05.626552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:18.034484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.625885+0000","last_change":"2024-01-27T02:32:05.625885+0000","last_active":"2024-01-27T02:32:05.625885+0000","last_peered":"2024-01-27T02:32:05.625885+0000","last_clean":"2024-01-27T02:32:05.625885+0000","last_became_active":"2024-01-27T02:32:05.622692+0000","last_became_peered":"2024-01-27T02:32:05.622692+0000","last_unstale":"2024-01-27T02:32:05.625885+0000","last_undegraded":"2024-01-27T02:32:05.625885+0000","last_fullsized":"2024-01-27T02:32:05.625885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:06.920232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616950+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:58:54.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.621014+0000","last_change":"2024-01-27T02:32:05.621014+0000","last_active":"2024-01-27T02:32:05.621014+0000","last_peered":"2024-01-27T02:32:05.621014+0000","last_clean":"2024-01-27T02:32:05.621014+0000","last_became_active":"2024-01-27T02:32:05.620775+0000","last_became_peered":"2024-01-27T02:32:05.620775+0000","last_unstale":"2024-01-27T02:32:05.621014+0000","last_undegraded":"2024-01-27T02:32:05.621014+0000","last_fullsized":"2024-01-27T02:32:05.621014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617909+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.624870+0000","last_change":"2024-01-27T02:32:05.624870+0000","last_active":"2024-01-27T02:32:05.624870+0000","last_peered":"2024-01-27T02:32:05.624870+0000","last_clean":"2024-01-27T02:32:05.624870+0000","last_became_active":"2024-01-27T02:32:05.624644+0000","last_became_peered":"2024-01-27T02:32:05.624644+0000","last_unstale":"2024-01-27T02:32:05.624870+0000","last_undegraded":"2024-01-27T02:32:05.624870+0000","last_fullsized":"2024-01-27T02:32:05.624870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616980+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.621287+0000","last_change":"2024-01-27T02:32:05.621287+0000","last_active":"2024-01-27T02:32:05.621287+0000","last_peered":"2024-01-27T02:32:05.621287+0000","last_clean":"2024-01-27T02:32:05.621287+0000","last_became_active":"2024-01-27T02:32:05.620002+0000","last_became_peered":"2024-01-27T02:32:05.620002+0000","last_unstale":"2024-01-27T02:32:05.621287+0000","last_undegraded":"2024-01-27T02:32:05.621287+0000","last_fullsized":"2024-01-27T02:32:05.621287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:44.489465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612241+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:44:53.933990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.623449+0000","last_change":"2024-01-27T02:32:05.623449+0000","last_active":"2024-01-27T02:32:05.623449+0000","last_peered":"2024-01-27T02:32:05.623449+0000","last_clean":"2024-01-27T02:32:05.623449+0000","last_became_active":"2024-01-27T02:32:05.623193+0000","last_became_peered":"2024-01-27T02:32:05.623193+0000","last_unstale":"2024-01-27T02:32:05.623449+0000","last_undegraded":"2024-01-27T02:32:05.623449+0000","last_fullsized":"2024-01-27T02:32:05.623449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:44.489465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616309+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:08.590979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.625193+0000","last_change":"2024-01-27T02:32:05.625193+0000","last_active":"2024-01-27T02:32:05.625193+0000","last_peered":"2024-01-27T02:32:05.625193+0000","last_clean":"2024-01-27T02:32:05.625193+0000","last_became_active":"2024-01-27T02:32:05.624951+0000","last_became_peered":"2024-01-27T02:32:05.624951+0000","last_unstale":"2024-01-27T02:32:05.625193+0000","last_undegraded":"2024-01-27T02:32:05.625193+0000","last_fullsized":"2024-01-27T02:32:05.625193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:04:28.890844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616904+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:31.534822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.625673+0000","last_change":"2024-01-27T02:32:05.625673+0000","last_active":"2024-01-27T02:32:05.625673+0000","last_peered":"2024-01-27T02:32:05.625673+0000","last_clean":"2024-01-27T02:32:05.625673+0000","last_became_active":"2024-01-27T02:32:05.623551+0000","last_became_peered":"2024-01-27T02:32:05.623551+0000","last_unstale":"2024-01-27T02:32:05.625673+0000","last_undegraded":"2024-01-27T02:32:05.625673+0000","last_fullsized":"2024-01-27T02:32:05.625673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614474+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.620877+0000","last_change":"2024-01-27T02:32:05.620877+0000","last_active":"2024-01-27T02:32:05.620877+0000","last_peered":"2024-01-27T02:32:05.620877+0000","last_clean":"2024-01-27T02:32:05.620877+0000","last_became_active":"2024-01-27T02:32:05.620600+0000","last_became_peered":"2024-01-27T02:32:05.620600+0000","last_unstale":"2024-01-27T02:32:05.620877+0000","last_undegraded":"2024-01-27T02:32:05.620877+0000","last_fullsized":"2024-01-27T02:32:05.620877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615243+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.619486+0000","last_change":"2024-01-27T02:32:05.619486+0000","last_active":"2024-01-27T02:32:05.619486+0000","last_peered":"2024-01-27T02:32:05.619486+0000","last_clean":"2024-01-27T02:32:05.619486+0000","last_became_active":"2024-01-27T02:32:05.619147+0000","last_became_peered":"2024-01-27T02:32:05.619147+0000","last_unstale":"2024-01-27T02:32:05.619486+0000","last_undegraded":"2024-01-27T02:32:05.619486+0000","last_fullsized":"2024-01-27T02:32:05.619486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613357+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":8,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-27T02:32:05.613146+0000","last_change":"2024-01-27T02:32:05.613146+0000","last_active":"2024-01-27T02:32:05.613146+0000","last_peered":"2024-01-27T02:32:05.613146+0000","last_clean":"2024-01-27T02:32:05.613146+0000","last_became_active":"2024-01-27T02:32:05.612852+0000","last_became_peered":"2024-01-27T02:32:05.612852+0000","last_unstale":"2024-01-27T02:32:05.613146+0000","last_undegraded":"2024-01-27T02:32:05.613146+0000","last_fullsized":"2024-01-27T02:32:05.613146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617656+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:45:37.035302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616731+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.612111+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.616985+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:44:13.569068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615702+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:31.534822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617415+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:45.494048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613738+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614627+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.620974+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613231+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.617923+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.614822+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:58:54.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.615379+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:47.819053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:32:05.607209+0000","last_change":"2024-01-27T02:32:05.613678+0000","last_active":"2024-01-27T02:32:05.607209+0000","last_peered":"2024-01-27T02:32:05.607209+0000","last_clean":"2024-01-27T02:32:05.607209+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:32:05.607209+0000","last_undegraded":"2024-01-27T02:32:05.607209+0000","last_fullsized":"2024-01-27T02:32:05.607209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":631,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194681,"statfs":{"total":200000000,"available":199353793,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":637,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194675,"statfs":{"total":200000000,"available":199347530,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":60,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195251,"statfs":{"total":200000000,"available":199937898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]}]}],"pool_statfs":[]}} 2024-01-27T02:32:15.671 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:32:15.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.815+0000 7fb349a1c700 1 Processor -- start 2024-01-27T02:32:15.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.815+0000 7fb349a1c700 1 -- start start 2024-01-27T02:32:15.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.815+0000 7fb349a1c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb3440cae30 0x7fb344165060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:15.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.815+0000 7fb349a1c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3440cb250 con 0x7fb3440cd9e0 2024-01-27T02:32:15.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.816+0000 7fb349a1c700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3440cc980 con 0x7fb34415a450 2024-01-27T02:32:15.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.816+0000 7fb349a1c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3440ccc20 con 0x7fb3440cae30 2024-01-27T02:32:15.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.816+0000 7fb3437fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb3440cae30 0x7fb344165060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:15.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.816+0000 7fb3437fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb3440cae30 0x7fb344165060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53588/0 (socket says 172.21.15.7:53588) 2024-01-27T02:32:15.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.816+0000 7fb3437fe700 1 -- 172.21.15.7:0/2968093420 learned_addr learned my addr 172.21.15.7:0/2968093420 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:15.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.816+0000 7fb3427fc700 1 --1- 172.21.15.7:0/2968093420 >> v1:172.21.15.7:6789/0 conn(0x7fb3440cd9e0 0x7fb3440cde60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51344/0 (socket says 172.21.15.7:51344) 2024-01-27T02:32:15.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.816+0000 7fb3437fe700 1 -- 172.21.15.7:0/2968093420 >> v1:172.21.15.188:6789/0 conn(0x7fb34415a450 legacy=0x7fb3440b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:15.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.816+0000 7fb3437fe700 1 -- 172.21.15.7:0/2968093420 >> v1:172.21.15.7:6789/0 conn(0x7fb3440cd9e0 legacy=0x7fb3440cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:15.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.816+0000 7fb341ffb700 1 -- 172.21.15.7:0/2968093420 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946388365 0 0) 0x7fb338001110 con 0x7fb3440cd9e0 2024-01-27T02:32:15.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.816+0000 7fb3437fe700 1 -- 172.21.15.7:0/2968093420 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3440ccf00 con 0x7fb3440cae30 2024-01-27T02:32:15.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.816+0000 7fb3437fe700 1 --2- 172.21.15.7:0/2968093420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3440cae30 0x7fb344165060 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009f20 tx=0x7fb33c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc41f9276da7ea0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:15.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.816+0000 7fb341ffb700 1 -- 172.21.15.7:0/2968093420 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c00e070 con 0x7fb3440cae30 2024-01-27T02:32:15.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.816+0000 7fb341ffb700 1 -- 172.21.15.7:0/2968093420 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33c00ba80 con 0x7fb3440cae30 2024-01-27T02:32:15.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.816+0000 7fb341ffb700 1 -- 172.21.15.7:0/2968093420 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c0092e0 con 0x7fb3440cae30 2024-01-27T02:32:15.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.817+0000 7fb349a1c700 1 -- 172.21.15.7:0/2968093420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3440cae30 msgr2=0x7fb344165060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:15.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.817+0000 7fb349a1c700 1 --2- 172.21.15.7:0/2968093420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3440cae30 0x7fb344165060 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009f20 tx=0x7fb33c009840 comp rx=0 tx=0).stop 2024-01-27T02:32:15.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.817+0000 7fb349a1c700 1 -- 172.21.15.7:0/2968093420 shutdown_connections 2024-01-27T02:32:15.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.817+0000 7fb349a1c700 1 --2- 172.21.15.7:0/2968093420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3440cae30 0x7fb344165060 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:15.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.817+0000 7fb349a1c700 1 -- 172.21.15.7:0/2968093420 >> 172.21.15.7:0/2968093420 conn(0x7fb3440711c0 msgr2=0x7fb3440ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:15.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.817+0000 7fb349a1c700 1 -- 172.21.15.7:0/2968093420 shutdown_connections 2024-01-27T02:32:15.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.817+0000 7fb349a1c700 1 -- 172.21.15.7:0/2968093420 wait complete. 2024-01-27T02:32:15.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.819+0000 7fb349a1c700 1 Processor -- start 2024-01-27T02:32:15.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.819+0000 7fb349a1c700 1 -- start start 2024-01-27T02:32:15.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.820+0000 7fb349a1c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3440cae30 0x7fb344157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:15.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.820+0000 7fb349a1c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3440cd9e0 0x7fb34414b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:15.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.820+0000 7fb349a1c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb34415a450 0x7fb34416a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:15.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.820+0000 7fb349a1c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440c7c40 con 0x7fb3440cd9e0 2024-01-27T02:32:15.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.820+0000 7fb349a1c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34415cef0 con 0x7fb3440cae30 2024-01-27T02:32:15.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.820+0000 7fb349a1c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440c7310 con 0x7fb34415a450 2024-01-27T02:32:15.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.820+0000 7fb3427fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3440cd9e0 0x7fb34414b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:15.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.820+0000 7fb3427fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3440cd9e0 0x7fb34414b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53598/0 (socket says 172.21.15.7:53598) 2024-01-27T02:32:15.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.820+0000 7fb3427fc700 1 -- 172.21.15.7:0/3414452288 learned_addr learned my addr 172.21.15.7:0/3414452288 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:15.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.820+0000 7fb3437fe700 1 --2- 172.21.15.7:0/3414452288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb34415a450 0x7fb34416a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:15.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.820+0000 7fb342ffd700 1 --2- 172.21.15.7:0/3414452288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3440cae30 0x7fb344157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:15.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.820+0000 7fb3427fc700 1 -- 172.21.15.7:0/3414452288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb34415a450 msgr2=0x7fb34416a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:15.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.820+0000 7fb3427fc700 1 --2- 172.21.15.7:0/3414452288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb34415a450 0x7fb34416a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:15.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.820+0000 7fb3427fc700 1 -- 172.21.15.7:0/3414452288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3440cae30 msgr2=0x7fb344157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:15.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.820+0000 7fb3427fc700 1 --2- 172.21.15.7:0/3414452288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3440cae30 0x7fb344157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:15.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.820+0000 7fb3427fc700 1 -- 172.21.15.7:0/3414452288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c009ae0 con 0x7fb3440cd9e0 2024-01-27T02:32:15.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.821+0000 7fb3427fc700 1 --2- 172.21.15.7:0/3414452288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3440cd9e0 0x7fb34414b280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb33800c810 tx=0x7fb338007c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:15.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.821+0000 7fb32affd700 1 -- 172.21.15.7:0/3414452288 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338007300 con 0x7fb3440cd9e0 2024-01-27T02:32:15.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.821+0000 7fb32affd700 1 -- 172.21.15.7:0/3414452288 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3380074a0 con 0x7fb3440cd9e0 2024-01-27T02:32:15.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.821+0000 7fb349a1c700 1 -- 172.21.15.7:0/3414452288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3440c75f0 con 0x7fb3440cd9e0 2024-01-27T02:32:15.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.821+0000 7fb32affd700 1 -- 172.21.15.7:0/3414452288 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338004830 con 0x7fb3440cd9e0 2024-01-27T02:32:15.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.821+0000 7fb349a1c700 1 -- 172.21.15.7:0/3414452288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34415cb30 con 0x7fb3440cd9e0 2024-01-27T02:32:15.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.822+0000 7fb349a1c700 1 -- 172.21.15.7:0/3414452288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3440c7db0 con 0x7fb3440cd9e0 2024-01-27T02:32:15.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.822+0000 7fb32affd700 1 -- 172.21.15.7:0/3414452288 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb338004ba0 con 0x7fb3440cd9e0 2024-01-27T02:32:15.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.823+0000 7fb32affd700 1 --2- 172.21.15.7:0/3414452288 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb330066f30 0x7fb3300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:15.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.823+0000 7fb32affd700 1 -- 172.21.15.7:0/3414452288 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb338083780 con 0x7fb3440cd9e0 2024-01-27T02:32:15.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.823+0000 7fb342ffd700 1 --2- 172.21.15.7:0/3414452288 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb330066f30 0x7fb3300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:15.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.824+0000 7fb342ffd700 1 --2- 172.21.15.7:0/3414452288 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb330066f30 0x7fb3300693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb3340037f0 tx=0x7fb334009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:15.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:15.827+0000 7fb32affd700 1 -- 172.21.15.7:0/3414452288 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb33804c5a0 con 0x7fb3440cd9e0 2024-01-27T02:32:16.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.025+0000 7fb349a1c700 1 -- 172.21.15.7:0/3414452288 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb3440cb250 con 0x7fb330066f30 2024-01-27T02:32:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.033+0000 7fb32affd700 1 -- 172.21.15.7:0/3414452288 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260133 (secure 0 0 0) 0x7fb3440cb250 con 0x7fb330066f30 2024-01-27T02:32:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:16.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.036+0000 7fb349a1c700 1 -- 172.21.15.7:0/3414452288 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb330066f30 msgr2=0x7fb3300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:16.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.036+0000 7fb349a1c700 1 --2- 172.21.15.7:0/3414452288 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb330066f30 0x7fb3300693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb3340037f0 tx=0x7fb334009040 comp rx=0 tx=0).stop 2024-01-27T02:32:16.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.036+0000 7fb349a1c700 1 -- 172.21.15.7:0/3414452288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3440cd9e0 msgr2=0x7fb34414b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:16.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.036+0000 7fb349a1c700 1 --2- 172.21.15.7:0/3414452288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3440cd9e0 0x7fb34414b280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb33800c810 tx=0x7fb338007c10 comp rx=0 tx=0).stop 2024-01-27T02:32:16.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.036+0000 7fb349a1c700 1 -- 172.21.15.7:0/3414452288 shutdown_connections 2024-01-27T02:32:16.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.036+0000 7fb349a1c700 1 --2- 172.21.15.7:0/3414452288 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb330066f30 0x7fb3300693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.036+0000 7fb349a1c700 1 --2- 172.21.15.7:0/3414452288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb34415a450 0x7fb34416a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.036+0000 7fb349a1c700 1 --2- 172.21.15.7:0/3414452288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3440cae30 0x7fb344157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.036+0000 7fb349a1c700 1 --2- 172.21.15.7:0/3414452288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3440cd9e0 0x7fb34414b280 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.036+0000 7fb349a1c700 1 -- 172.21.15.7:0/3414452288 >> 172.21.15.7:0/3414452288 conn(0x7fb3440711c0 msgr2=0x7fb3440cb600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:16.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.036+0000 7fb349a1c700 1 -- 172.21.15.7:0/3414452288 shutdown_connections 2024-01-27T02:32:16.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.036+0000 7fb349a1c700 1 -- 172.21.15.7:0/3414452288 wait complete. 2024-01-27T02:32:16.044 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:32:16.051 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-01-27T02:32:15.573938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1744,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560755,"statfs":{"total":1600000000,"available":1598213428,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.945197"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.645906+0000","last_change":"2024-01-27T02:32:06.645807+0000","last_active":"2024-01-27T02:32:06.645906+0000","last_peered":"2024-01-27T02:32:06.645906+0000","last_clean":"2024-01-27T02:32:06.645906+0000","last_became_active":"2024-01-27T02:32:06.645346+0000","last_became_peered":"2024-01-27T02:32:06.645346+0000","last_unstale":"2024-01-27T02:32:06.645906+0000","last_undegraded":"2024-01-27T02:32:06.645906+0000","last_fullsized":"2024-01-27T02:32:06.645906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:44:24.628811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.639731+0000","last_change":"2024-01-27T02:32:06.639629+0000","last_active":"2024-01-27T02:32:06.639731+0000","last_peered":"2024-01-27T02:32:06.639731+0000","last_clean":"2024-01-27T02:32:06.639731+0000","last_became_active":"2024-01-27T02:32:06.638325+0000","last_became_peered":"2024-01-27T02:32:06.638325+0000","last_unstale":"2024-01-27T02:32:06.639731+0000","last_undegraded":"2024-01-27T02:32:06.639731+0000","last_fullsized":"2024-01-27T02:32:06.639731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:06.985231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.636966+0000","last_change":"2024-01-27T02:32:06.636541+0000","last_active":"2024-01-27T02:32:06.636966+0000","last_peered":"2024-01-27T02:32:06.636966+0000","last_clean":"2024-01-27T02:32:06.636966+0000","last_became_active":"2024-01-27T02:32:06.636201+0000","last_became_peered":"2024-01-27T02:32:06.636201+0000","last_unstale":"2024-01-27T02:32:06.636966+0000","last_undegraded":"2024-01-27T02:32:06.636966+0000","last_fullsized":"2024-01-27T02:32:06.636966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:44:13.569068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.649506+0000","last_change":"2024-01-27T02:32:06.649197+0000","last_active":"2024-01-27T02:32:06.649506+0000","last_peered":"2024-01-27T02:32:06.649506+0000","last_clean":"2024-01-27T02:32:06.649506+0000","last_became_active":"2024-01-27T02:32:06.648503+0000","last_became_peered":"2024-01-27T02:32:06.648503+0000","last_unstale":"2024-01-27T02:32:06.649506+0000","last_undegraded":"2024-01-27T02:32:06.649506+0000","last_fullsized":"2024-01-27T02:32:06.649506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:44:01.051017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.645086+0000","last_change":"2024-01-27T02:32:06.643865+0000","last_active":"2024-01-27T02:32:06.645086+0000","last_peered":"2024-01-27T02:32:06.645086+0000","last_clean":"2024-01-27T02:32:06.645086+0000","last_became_active":"2024-01-27T02:32:06.641009+0000","last_became_peered":"2024-01-27T02:32:06.641009+0000","last_unstale":"2024-01-27T02:32:06.645086+0000","last_undegraded":"2024-01-27T02:32:06.645086+0000","last_fullsized":"2024-01-27T02:32:06.645086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:48:03.053143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.647987+0000","last_change":"2024-01-27T02:32:06.647906+0000","last_active":"2024-01-27T02:32:06.647987+0000","last_peered":"2024-01-27T02:32:06.647987+0000","last_clean":"2024-01-27T02:32:06.647987+0000","last_became_active":"2024-01-27T02:32:06.647577+0000","last_became_peered":"2024-01-27T02:32:06.647577+0000","last_unstale":"2024-01-27T02:32:06.647987+0000","last_undegraded":"2024-01-27T02:32:06.647987+0000","last_fullsized":"2024-01-27T02:32:06.647987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:16:23.225540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.645142+0000","last_change":"2024-01-27T02:32:06.644654+0000","last_active":"2024-01-27T02:32:06.645142+0000","last_peered":"2024-01-27T02:32:06.645142+0000","last_clean":"2024-01-27T02:32:06.645142+0000","last_became_active":"2024-01-27T02:32:06.642941+0000","last_became_peered":"2024-01-27T02:32:06.642941+0000","last_unstale":"2024-01-27T02:32:06.645142+0000","last_undegraded":"2024-01-27T02:32:06.645142+0000","last_fullsized":"2024-01-27T02:32:06.645142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:43:05.599796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642600+0000","last_change":"2024-01-27T02:32:06.642517+0000","last_active":"2024-01-27T02:32:06.642600+0000","last_peered":"2024-01-27T02:32:06.642600+0000","last_clean":"2024-01-27T02:32:06.642600+0000","last_became_active":"2024-01-27T02:32:06.641240+0000","last_became_peered":"2024-01-27T02:32:06.641240+0000","last_unstale":"2024-01-27T02:32:06.642600+0000","last_undegraded":"2024-01-27T02:32:06.642600+0000","last_fullsized":"2024-01-27T02:32:06.642600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:11:54.230178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.650062+0000","last_change":"2024-01-27T02:32:06.649943+0000","last_active":"2024-01-27T02:32:06.650062+0000","last_peered":"2024-01-27T02:32:06.650062+0000","last_clean":"2024-01-27T02:32:06.650062+0000","last_became_active":"2024-01-27T02:32:06.649685+0000","last_became_peered":"2024-01-27T02:32:06.649685+0000","last_unstale":"2024-01-27T02:32:06.650062+0000","last_undegraded":"2024-01-27T02:32:06.650062+0000","last_fullsized":"2024-01-27T02:32:06.650062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:31:12.724216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.646882+0000","last_change":"2024-01-27T02:32:06.646820+0000","last_active":"2024-01-27T02:32:06.646882+0000","last_peered":"2024-01-27T02:32:06.646882+0000","last_clean":"2024-01-27T02:32:06.646882+0000","last_became_active":"2024-01-27T02:32:06.646586+0000","last_became_peered":"2024-01-27T02:32:06.646586+0000","last_unstale":"2024-01-27T02:32:06.646882+0000","last_undegraded":"2024-01-27T02:32:06.646882+0000","last_fullsized":"2024-01-27T02:32:06.646882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:06.985231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.649997+0000","last_change":"2024-01-27T02:32:06.649925+0000","last_active":"2024-01-27T02:32:06.649997+0000","last_peered":"2024-01-27T02:32:06.649997+0000","last_clean":"2024-01-27T02:32:06.649997+0000","last_became_active":"2024-01-27T02:32:06.649685+0000","last_became_peered":"2024-01-27T02:32:06.649685+0000","last_unstale":"2024-01-27T02:32:06.649997+0000","last_undegraded":"2024-01-27T02:32:06.649997+0000","last_fullsized":"2024-01-27T02:32:06.649997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:36:11.295485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642706+0000","last_change":"2024-01-27T02:32:06.642576+0000","last_active":"2024-01-27T02:32:06.642706+0000","last_peered":"2024-01-27T02:32:06.642706+0000","last_clean":"2024-01-27T02:32:06.642706+0000","last_became_active":"2024-01-27T02:32:06.641583+0000","last_became_peered":"2024-01-27T02:32:06.641583+0000","last_unstale":"2024-01-27T02:32:06.642706+0000","last_undegraded":"2024-01-27T02:32:06.642706+0000","last_fullsized":"2024-01-27T02:32:06.642706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:45.494048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642822+0000","last_change":"2024-01-27T02:32:06.642731+0000","last_active":"2024-01-27T02:32:06.642822+0000","last_peered":"2024-01-27T02:32:06.642822+0000","last_clean":"2024-01-27T02:32:06.642822+0000","last_became_active":"2024-01-27T02:32:06.641619+0000","last_became_peered":"2024-01-27T02:32:06.641619+0000","last_unstale":"2024-01-27T02:32:06.642822+0000","last_undegraded":"2024-01-27T02:32:06.642822+0000","last_fullsized":"2024-01-27T02:32:06.642822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:44:24.628811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.635576+0000","last_change":"2024-01-27T02:32:06.635401+0000","last_active":"2024-01-27T02:32:06.635576+0000","last_peered":"2024-01-27T02:32:06.635576+0000","last_clean":"2024-01-27T02:32:06.635576+0000","last_became_active":"2024-01-27T02:32:06.635136+0000","last_became_peered":"2024-01-27T02:32:06.635136+0000","last_unstale":"2024-01-27T02:32:06.635576+0000","last_undegraded":"2024-01-27T02:32:06.635576+0000","last_fullsized":"2024-01-27T02:32:06.635576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:17:03.351766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.639565+0000","last_change":"2024-01-27T02:32:06.639126+0000","last_active":"2024-01-27T02:32:06.639565+0000","last_peered":"2024-01-27T02:32:06.639565+0000","last_clean":"2024-01-27T02:32:06.639565+0000","last_became_active":"2024-01-27T02:32:06.638696+0000","last_became_peered":"2024-01-27T02:32:06.638696+0000","last_unstale":"2024-01-27T02:32:06.639565+0000","last_undegraded":"2024-01-27T02:32:06.639565+0000","last_fullsized":"2024-01-27T02:32:06.639565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:17:03.351766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.647702+0000","last_change":"2024-01-27T02:32:06.646066+0000","last_active":"2024-01-27T02:32:06.647702+0000","last_peered":"2024-01-27T02:32:06.647702+0000","last_clean":"2024-01-27T02:32:06.647702+0000","last_became_active":"2024-01-27T02:32:06.644886+0000","last_became_peered":"2024-01-27T02:32:06.644886+0000","last_unstale":"2024-01-27T02:32:06.647702+0000","last_undegraded":"2024-01-27T02:32:06.647702+0000","last_fullsized":"2024-01-27T02:32:06.647702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:11:54.230178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.649539+0000","last_change":"2024-01-27T02:32:06.649461+0000","last_active":"2024-01-27T02:32:06.649539+0000","last_peered":"2024-01-27T02:32:06.649539+0000","last_clean":"2024-01-27T02:32:06.649539+0000","last_became_active":"2024-01-27T02:32:06.648614+0000","last_became_peered":"2024-01-27T02:32:06.648614+0000","last_unstale":"2024-01-27T02:32:06.649539+0000","last_undegraded":"2024-01-27T02:32:06.649539+0000","last_fullsized":"2024-01-27T02:32:06.649539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:13:38.187522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.638138+0000","last_change":"2024-01-27T02:32:06.638050+0000","last_active":"2024-01-27T02:32:06.638138+0000","last_peered":"2024-01-27T02:32:06.638138+0000","last_clean":"2024-01-27T02:32:06.638138+0000","last_became_active":"2024-01-27T02:32:06.637390+0000","last_became_peered":"2024-01-27T02:32:06.637390+0000","last_unstale":"2024-01-27T02:32:06.638138+0000","last_undegraded":"2024-01-27T02:32:06.638138+0000","last_fullsized":"2024-01-27T02:32:06.638138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642381+0000","last_change":"2024-01-27T02:32:06.642245+0000","last_active":"2024-01-27T02:32:06.642381+0000","last_peered":"2024-01-27T02:32:06.642381+0000","last_clean":"2024-01-27T02:32:06.642381+0000","last_became_active":"2024-01-27T02:32:06.641079+0000","last_became_peered":"2024-01-27T02:32:06.641079+0000","last_unstale":"2024-01-27T02:32:06.642381+0000","last_undegraded":"2024-01-27T02:32:06.642381+0000","last_fullsized":"2024-01-27T02:32:06.642381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:38.023976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.802275+0000","last_change":"2024-01-27T02:32:05.624071+0000","last_active":"2024-01-27T02:32:06.802275+0000","last_peered":"2024-01-27T02:32:06.802275+0000","last_clean":"2024-01-27T02:32:06.802275+0000","last_became_active":"2024-01-27T02:32:05.623818+0000","last_became_peered":"2024-01-27T02:32:05.623818+0000","last_unstale":"2024-01-27T02:32:06.802275+0000","last_undegraded":"2024-01-27T02:32:06.802275+0000","last_fullsized":"2024-01-27T02:32:06.802275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:34:05.180050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.641588+0000","last_change":"2024-01-27T02:32:06.641507+0000","last_active":"2024-01-27T02:32:06.641588+0000","last_peered":"2024-01-27T02:32:06.641588+0000","last_clean":"2024-01-27T02:32:06.641588+0000","last_became_active":"2024-01-27T02:32:06.640928+0000","last_became_peered":"2024-01-27T02:32:06.640928+0000","last_unstale":"2024-01-27T02:32:06.641588+0000","last_undegraded":"2024-01-27T02:32:06.641588+0000","last_fullsized":"2024-01-27T02:32:06.641588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:32:24.198014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.803197+0000","last_change":"2024-01-27T02:32:05.624866+0000","last_active":"2024-01-27T02:32:06.803197+0000","last_peered":"2024-01-27T02:32:06.803197+0000","last_clean":"2024-01-27T02:32:06.803197+0000","last_became_active":"2024-01-27T02:32:05.624368+0000","last_became_peered":"2024-01-27T02:32:05.624368+0000","last_unstale":"2024-01-27T02:32:06.803197+0000","last_undegraded":"2024-01-27T02:32:06.803197+0000","last_fullsized":"2024-01-27T02:32:06.803197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642222+0000","last_change":"2024-01-27T02:32:06.642157+0000","last_active":"2024-01-27T02:32:06.642222+0000","last_peered":"2024-01-27T02:32:06.642222+0000","last_clean":"2024-01-27T02:32:06.642222+0000","last_became_active":"2024-01-27T02:32:06.641894+0000","last_became_peered":"2024-01-27T02:32:06.641894+0000","last_unstale":"2024-01-27T02:32:06.642222+0000","last_undegraded":"2024-01-27T02:32:06.642222+0000","last_fullsized":"2024-01-27T02:32:06.642222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.802467+0000","last_change":"2024-01-27T02:32:05.623255+0000","last_active":"2024-01-27T02:32:06.802467+0000","last_peered":"2024-01-27T02:32:06.802467+0000","last_clean":"2024-01-27T02:32:06.802467+0000","last_became_active":"2024-01-27T02:32:05.622821+0000","last_became_peered":"2024-01-27T02:32:05.622821+0000","last_unstale":"2024-01-27T02:32:06.802467+0000","last_undegraded":"2024-01-27T02:32:06.802467+0000","last_fullsized":"2024-01-27T02:32:06.802467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:33.333681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.648821+0000","last_change":"2024-01-27T02:32:06.648760+0000","last_active":"2024-01-27T02:32:06.648821+0000","last_peered":"2024-01-27T02:32:06.648821+0000","last_clean":"2024-01-27T02:32:06.648821+0000","last_became_active":"2024-01-27T02:32:06.648279+0000","last_became_peered":"2024-01-27T02:32:06.648279+0000","last_unstale":"2024-01-27T02:32:06.648821+0000","last_undegraded":"2024-01-27T02:32:06.648821+0000","last_fullsized":"2024-01-27T02:32:06.648821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:32:24.198014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.802483+0000","last_change":"2024-01-27T02:32:05.626014+0000","last_active":"2024-01-27T02:32:06.802483+0000","last_peered":"2024-01-27T02:32:06.802483+0000","last_clean":"2024-01-27T02:32:06.802483+0000","last_became_active":"2024-01-27T02:32:05.623176+0000","last_became_peered":"2024-01-27T02:32:05.623176+0000","last_unstale":"2024-01-27T02:32:06.802483+0000","last_undegraded":"2024-01-27T02:32:06.802483+0000","last_fullsized":"2024-01-27T02:32:06.802483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:37:59.246901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.641268+0000","last_change":"2024-01-27T02:32:06.637696+0000","last_active":"2024-01-27T02:32:06.641268+0000","last_peered":"2024-01-27T02:32:06.641268+0000","last_clean":"2024-01-27T02:32:06.641268+0000","last_became_active":"2024-01-27T02:32:06.637457+0000","last_became_peered":"2024-01-27T02:32:06.637457+0000","last_unstale":"2024-01-27T02:32:06.641268+0000","last_undegraded":"2024-01-27T02:32:06.641268+0000","last_fullsized":"2024-01-27T02:32:06.641268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:17:03.351766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.803103+0000","last_change":"2024-01-27T02:32:05.622086+0000","last_active":"2024-01-27T02:32:06.803103+0000","last_peered":"2024-01-27T02:32:06.803103+0000","last_clean":"2024-01-27T02:32:06.803103+0000","last_became_active":"2024-01-27T02:32:05.621465+0000","last_became_peered":"2024-01-27T02:32:05.621465+0000","last_unstale":"2024-01-27T02:32:06.803103+0000","last_undegraded":"2024-01-27T02:32:06.803103+0000","last_fullsized":"2024-01-27T02:32:06.803103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:04:28.890844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.644927+0000","last_change":"2024-01-27T02:32:06.644850+0000","last_active":"2024-01-27T02:32:06.644927+0000","last_peered":"2024-01-27T02:32:06.644927+0000","last_clean":"2024-01-27T02:32:06.644927+0000","last_became_active":"2024-01-27T02:32:06.644442+0000","last_became_peered":"2024-01-27T02:32:06.644442+0000","last_unstale":"2024-01-27T02:32:06.644927+0000","last_undegraded":"2024-01-27T02:32:06.644927+0000","last_fullsized":"2024-01-27T02:32:06.644927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.801277+0000","last_change":"2024-01-27T02:32:05.617530+0000","last_active":"2024-01-27T02:32:06.801277+0000","last_peered":"2024-01-27T02:32:06.801277+0000","last_clean":"2024-01-27T02:32:06.801277+0000","last_became_active":"2024-01-27T02:32:05.617290+0000","last_became_peered":"2024-01-27T02:32:05.617290+0000","last_unstale":"2024-01-27T02:32:06.801277+0000","last_undegraded":"2024-01-27T02:32:06.801277+0000","last_fullsized":"2024-01-27T02:32:06.801277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:41.575605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642446+0000","last_change":"2024-01-27T02:32:06.642339+0000","last_active":"2024-01-27T02:32:06.642446+0000","last_peered":"2024-01-27T02:32:06.642446+0000","last_clean":"2024-01-27T02:32:06.642446+0000","last_became_active":"2024-01-27T02:32:06.642027+0000","last_became_peered":"2024-01-27T02:32:06.642027+0000","last_unstale":"2024-01-27T02:32:06.642446+0000","last_undegraded":"2024-01-27T02:32:06.642446+0000","last_fullsized":"2024-01-27T02:32:06.642446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:45.494048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.628837+0000","last_change":"2024-01-27T02:32:05.623693+0000","last_active":"2024-01-27T02:32:06.628837+0000","last_peered":"2024-01-27T02:32:06.628837+0000","last_clean":"2024-01-27T02:32:06.628837+0000","last_became_active":"2024-01-27T02:32:05.623440+0000","last_became_peered":"2024-01-27T02:32:05.623440+0000","last_unstale":"2024-01-27T02:32:06.628837+0000","last_undegraded":"2024-01-27T02:32:06.628837+0000","last_fullsized":"2024-01-27T02:32:06.628837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:44:52.929407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.637714+0000","last_change":"2024-01-27T02:32:06.635720+0000","last_active":"2024-01-27T02:32:06.637714+0000","last_peered":"2024-01-27T02:32:06.637714+0000","last_clean":"2024-01-27T02:32:06.637714+0000","last_became_active":"2024-01-27T02:32:06.635457+0000","last_became_peered":"2024-01-27T02:32:06.635457+0000","last_unstale":"2024-01-27T02:32:06.637714+0000","last_undegraded":"2024-01-27T02:32:06.637714+0000","last_fullsized":"2024-01-27T02:32:06.637714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:36:11.295485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.633418+0000","last_change":"2024-01-27T02:32:05.619476+0000","last_active":"2024-01-27T02:32:06.633418+0000","last_peered":"2024-01-27T02:32:06.633418+0000","last_clean":"2024-01-27T02:32:06.633418+0000","last_became_active":"2024-01-27T02:32:05.619194+0000","last_became_peered":"2024-01-27T02:32:05.619194+0000","last_unstale":"2024-01-27T02:32:06.633418+0000","last_undegraded":"2024-01-27T02:32:06.633418+0000","last_fullsized":"2024-01-27T02:32:06.633418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.634597+0000","last_change":"2024-01-27T02:32:06.634504+0000","last_active":"2024-01-27T02:32:06.634597+0000","last_peered":"2024-01-27T02:32:06.634597+0000","last_clean":"2024-01-27T02:32:06.634597+0000","last_became_active":"2024-01-27T02:32:06.634220+0000","last_became_peered":"2024-01-27T02:32:06.634220+0000","last_unstale":"2024-01-27T02:32:06.634597+0000","last_undegraded":"2024-01-27T02:32:06.634597+0000","last_fullsized":"2024-01-27T02:32:06.634597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:45:37.035302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.635103+0000","last_change":"2024-01-27T02:32:05.623827+0000","last_active":"2024-01-27T02:32:06.635103+0000","last_peered":"2024-01-27T02:32:06.635103+0000","last_clean":"2024-01-27T02:32:06.635103+0000","last_became_active":"2024-01-27T02:32:05.623602+0000","last_became_peered":"2024-01-27T02:32:05.623602+0000","last_unstale":"2024-01-27T02:32:06.635103+0000","last_undegraded":"2024-01-27T02:32:06.635103+0000","last_fullsized":"2024-01-27T02:32:06.635103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.649471+0000","last_change":"2024-01-27T02:32:06.649029+0000","last_active":"2024-01-27T02:32:06.649471+0000","last_peered":"2024-01-27T02:32:06.649471+0000","last_clean":"2024-01-27T02:32:06.649471+0000","last_became_active":"2024-01-27T02:32:06.647963+0000","last_became_peered":"2024-01-27T02:32:06.647963+0000","last_unstale":"2024-01-27T02:32:06.649471+0000","last_undegraded":"2024-01-27T02:32:06.649471+0000","last_fullsized":"2024-01-27T02:32:06.649471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:43:05.599796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.635934+0000","last_change":"2024-01-27T02:32:05.621283+0000","last_active":"2024-01-27T02:32:06.635934+0000","last_peered":"2024-01-27T02:32:06.635934+0000","last_clean":"2024-01-27T02:32:06.635934+0000","last_became_active":"2024-01-27T02:32:05.621034+0000","last_became_peered":"2024-01-27T02:32:05.621034+0000","last_unstale":"2024-01-27T02:32:06.635934+0000","last_undegraded":"2024-01-27T02:32:06.635934+0000","last_fullsized":"2024-01-27T02:32:06.635934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:07.586396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.649134+0000","last_change":"2024-01-27T02:32:06.649052+0000","last_active":"2024-01-27T02:32:06.649134+0000","last_peered":"2024-01-27T02:32:06.649134+0000","last_clean":"2024-01-27T02:32:06.649134+0000","last_became_active":"2024-01-27T02:32:06.648837+0000","last_became_peered":"2024-01-27T02:32:06.648837+0000","last_unstale":"2024-01-27T02:32:06.649134+0000","last_undegraded":"2024-01-27T02:32:06.649134+0000","last_fullsized":"2024-01-27T02:32:06.649134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.802664+0000","last_change":"2024-01-27T02:32:05.620524+0000","last_active":"2024-01-27T02:32:06.802664+0000","last_peered":"2024-01-27T02:32:06.802664+0000","last_clean":"2024-01-27T02:32:06.802664+0000","last_became_active":"2024-01-27T02:32:05.620169+0000","last_became_peered":"2024-01-27T02:32:05.620169+0000","last_unstale":"2024-01-27T02:32:06.802664+0000","last_undegraded":"2024-01-27T02:32:06.802664+0000","last_fullsized":"2024-01-27T02:32:06.802664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.641576+0000","last_change":"2024-01-27T02:32:06.641250+0000","last_active":"2024-01-27T02:32:06.641576+0000","last_peered":"2024-01-27T02:32:06.641576+0000","last_clean":"2024-01-27T02:32:06.641576+0000","last_became_active":"2024-01-27T02:32:06.640947+0000","last_became_peered":"2024-01-27T02:32:06.640947+0000","last_unstale":"2024-01-27T02:32:06.641576+0000","last_undegraded":"2024-01-27T02:32:06.641576+0000","last_fullsized":"2024-01-27T02:32:06.641576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.804224+0000","last_change":"2024-01-27T02:32:05.625720+0000","last_active":"2024-01-27T02:32:06.804224+0000","last_peered":"2024-01-27T02:32:06.804224+0000","last_clean":"2024-01-27T02:32:06.804224+0000","last_became_active":"2024-01-27T02:32:05.625394+0000","last_became_peered":"2024-01-27T02:32:05.625394+0000","last_unstale":"2024-01-27T02:32:06.804224+0000","last_undegraded":"2024-01-27T02:32:06.804224+0000","last_fullsized":"2024-01-27T02:32:06.804224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.643130+0000","last_change":"2024-01-27T02:32:06.643058+0000","last_active":"2024-01-27T02:32:06.643130+0000","last_peered":"2024-01-27T02:32:06.643130+0000","last_clean":"2024-01-27T02:32:06.643130+0000","last_became_active":"2024-01-27T02:32:06.642818+0000","last_became_peered":"2024-01-27T02:32:06.642818+0000","last_unstale":"2024-01-27T02:32:06.643130+0000","last_undegraded":"2024-01-27T02:32:06.643130+0000","last_fullsized":"2024-01-27T02:32:06.643130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:15:06.036893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.801374+0000","last_change":"2024-01-27T02:32:05.619745+0000","last_active":"2024-01-27T02:32:06.801374+0000","last_peered":"2024-01-27T02:32:06.801374+0000","last_clean":"2024-01-27T02:32:06.801374+0000","last_became_active":"2024-01-27T02:32:05.619371+0000","last_became_peered":"2024-01-27T02:32:05.619371+0000","last_unstale":"2024-01-27T02:32:06.801374+0000","last_undegraded":"2024-01-27T02:32:06.801374+0000","last_fullsized":"2024-01-27T02:32:06.801374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:12:20.892542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.646863+0000","last_change":"2024-01-27T02:32:06.646797+0000","last_active":"2024-01-27T02:32:06.646863+0000","last_peered":"2024-01-27T02:32:06.646863+0000","last_clean":"2024-01-27T02:32:06.646863+0000","last_became_active":"2024-01-27T02:32:06.646253+0000","last_became_peered":"2024-01-27T02:32:06.646253+0000","last_unstale":"2024-01-27T02:32:06.646863+0000","last_undegraded":"2024-01-27T02:32:06.646863+0000","last_fullsized":"2024-01-27T02:32:06.646863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:58:54.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.643949+0000","last_change":"2024-01-27T02:32:05.626895+0000","last_active":"2024-01-27T02:32:06.643949+0000","last_peered":"2024-01-27T02:32:06.643949+0000","last_clean":"2024-01-27T02:32:06.643949+0000","last_became_active":"2024-01-27T02:32:05.626682+0000","last_became_peered":"2024-01-27T02:32:05.626682+0000","last_unstale":"2024-01-27T02:32:06.643949+0000","last_undegraded":"2024-01-27T02:32:06.643949+0000","last_fullsized":"2024-01-27T02:32:06.643949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:41.575605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.645112+0000","last_change":"2024-01-27T02:32:06.644466+0000","last_active":"2024-01-27T02:32:06.645112+0000","last_peered":"2024-01-27T02:32:06.645112+0000","last_clean":"2024-01-27T02:32:06.645112+0000","last_became_active":"2024-01-27T02:32:06.643544+0000","last_became_peered":"2024-01-27T02:32:06.643544+0000","last_unstale":"2024-01-27T02:32:06.645112+0000","last_undegraded":"2024-01-27T02:32:06.645112+0000","last_fullsized":"2024-01-27T02:32:06.645112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:22:27.441031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.635250+0000","last_change":"2024-01-27T02:32:05.619939+0000","last_active":"2024-01-27T02:32:06.635250+0000","last_peered":"2024-01-27T02:32:06.635250+0000","last_clean":"2024-01-27T02:32:06.635250+0000","last_became_active":"2024-01-27T02:32:05.619489+0000","last_became_peered":"2024-01-27T02:32:05.619489+0000","last_unstale":"2024-01-27T02:32:06.635250+0000","last_undegraded":"2024-01-27T02:32:06.635250+0000","last_fullsized":"2024-01-27T02:32:06.635250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:07.586396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.638890+0000","last_change":"2024-01-27T02:32:06.638821+0000","last_active":"2024-01-27T02:32:06.638890+0000","last_peered":"2024-01-27T02:32:06.638890+0000","last_clean":"2024-01-27T02:32:06.638890+0000","last_became_active":"2024-01-27T02:32:06.638486+0000","last_became_peered":"2024-01-27T02:32:06.638486+0000","last_unstale":"2024-01-27T02:32:06.638890+0000","last_undegraded":"2024-01-27T02:32:06.638890+0000","last_fullsized":"2024-01-27T02:32:06.638890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:45:37.035302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.806345+0000","last_change":"2024-01-27T02:32:05.621851+0000","last_active":"2024-01-27T02:32:06.806345+0000","last_peered":"2024-01-27T02:32:06.806345+0000","last_clean":"2024-01-27T02:32:06.806345+0000","last_became_active":"2024-01-27T02:32:05.621487+0000","last_became_peered":"2024-01-27T02:32:05.621487+0000","last_unstale":"2024-01-27T02:32:06.806345+0000","last_undegraded":"2024-01-27T02:32:06.806345+0000","last_fullsized":"2024-01-27T02:32:06.806345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:07.586396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.645157+0000","last_change":"2024-01-27T02:32:06.644806+0000","last_active":"2024-01-27T02:32:06.645157+0000","last_peered":"2024-01-27T02:32:06.645157+0000","last_clean":"2024-01-27T02:32:06.645157+0000","last_became_active":"2024-01-27T02:32:06.643567+0000","last_became_peered":"2024-01-27T02:32:06.643567+0000","last_unstale":"2024-01-27T02:32:06.645157+0000","last_undegraded":"2024-01-27T02:32:06.645157+0000","last_fullsized":"2024-01-27T02:32:06.645157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:44:01.051017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.801050+0000","last_change":"2024-01-27T02:32:05.623147+0000","last_active":"2024-01-27T02:32:06.801050+0000","last_peered":"2024-01-27T02:32:06.801050+0000","last_clean":"2024-01-27T02:32:06.801050+0000","last_became_active":"2024-01-27T02:32:05.622650+0000","last_became_peered":"2024-01-27T02:32:05.622650+0000","last_unstale":"2024-01-27T02:32:06.801050+0000","last_undegraded":"2024-01-27T02:32:06.801050+0000","last_fullsized":"2024-01-27T02:32:06.801050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:30:53.128829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.645128+0000","last_change":"2024-01-27T02:32:06.644626+0000","last_active":"2024-01-27T02:32:06.645128+0000","last_peered":"2024-01-27T02:32:06.645128+0000","last_clean":"2024-01-27T02:32:06.645128+0000","last_became_active":"2024-01-27T02:32:06.644149+0000","last_became_peered":"2024-01-27T02:32:06.644149+0000","last_unstale":"2024-01-27T02:32:06.645128+0000","last_undegraded":"2024-01-27T02:32:06.645128+0000","last_fullsized":"2024-01-27T02:32:06.645128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:13:38.187522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.632259+0000","last_change":"2024-01-27T02:32:05.621921+0000","last_active":"2024-01-27T02:32:06.632259+0000","last_peered":"2024-01-27T02:32:06.632259+0000","last_clean":"2024-01-27T02:32:06.632259+0000","last_became_active":"2024-01-27T02:32:05.620450+0000","last_became_peered":"2024-01-27T02:32:05.620450+0000","last_unstale":"2024-01-27T02:32:06.632259+0000","last_undegraded":"2024-01-27T02:32:06.632259+0000","last_fullsized":"2024-01-27T02:32:06.632259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:33.333681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.649453+0000","last_change":"2024-01-27T02:32:06.648997+0000","last_active":"2024-01-27T02:32:06.649453+0000","last_peered":"2024-01-27T02:32:06.649453+0000","last_clean":"2024-01-27T02:32:06.649453+0000","last_became_active":"2024-01-27T02:32:06.648180+0000","last_became_peered":"2024-01-27T02:32:06.648180+0000","last_unstale":"2024-01-27T02:32:06.649453+0000","last_undegraded":"2024-01-27T02:32:06.649453+0000","last_fullsized":"2024-01-27T02:32:06.649453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:15:38.764945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.636045+0000","last_change":"2024-01-27T02:32:05.622783+0000","last_active":"2024-01-27T02:32:06.636045+0000","last_peered":"2024-01-27T02:32:06.636045+0000","last_clean":"2024-01-27T02:32:06.636045+0000","last_became_active":"2024-01-27T02:32:05.622557+0000","last_became_peered":"2024-01-27T02:32:05.622557+0000","last_unstale":"2024-01-27T02:32:06.636045+0000","last_undegraded":"2024-01-27T02:32:06.636045+0000","last_fullsized":"2024-01-27T02:32:06.636045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.638811+0000","last_change":"2024-01-27T02:32:06.638626+0000","last_active":"2024-01-27T02:32:06.638811+0000","last_peered":"2024-01-27T02:32:06.638811+0000","last_clean":"2024-01-27T02:32:06.638811+0000","last_became_active":"2024-01-27T02:32:06.638382+0000","last_became_peered":"2024-01-27T02:32:06.638382+0000","last_unstale":"2024-01-27T02:32:06.638811+0000","last_undegraded":"2024-01-27T02:32:06.638811+0000","last_fullsized":"2024-01-27T02:32:06.638811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:31.534822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.635736+0000","last_change":"2024-01-27T02:32:06.635617+0000","last_active":"2024-01-27T02:32:06.635736+0000","last_peered":"2024-01-27T02:32:06.635736+0000","last_clean":"2024-01-27T02:32:06.635736+0000","last_became_active":"2024-01-27T02:32:06.635367+0000","last_became_peered":"2024-01-27T02:32:06.635367+0000","last_unstale":"2024-01-27T02:32:06.635736+0000","last_undegraded":"2024-01-27T02:32:06.635736+0000","last_fullsized":"2024-01-27T02:32:06.635736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:36:11.295485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.630282+0000","last_change":"2024-01-27T02:32:05.613447+0000","last_active":"2024-01-27T02:32:06.630282+0000","last_peered":"2024-01-27T02:32:06.630282+0000","last_clean":"2024-01-27T02:32:06.630282+0000","last_became_active":"2024-01-27T02:32:05.613203+0000","last_became_peered":"2024-01-27T02:32:05.613203+0000","last_unstale":"2024-01-27T02:32:06.630282+0000","last_undegraded":"2024-01-27T02:32:06.630282+0000","last_fullsized":"2024-01-27T02:32:06.630282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:41.575605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.648445+0000","last_change":"2024-01-27T02:32:06.648364+0000","last_active":"2024-01-27T02:32:06.648445+0000","last_peered":"2024-01-27T02:32:06.648445+0000","last_clean":"2024-01-27T02:32:06.648445+0000","last_became_active":"2024-01-27T02:32:06.648115+0000","last_became_peered":"2024-01-27T02:32:06.648115+0000","last_unstale":"2024-01-27T02:32:06.648445+0000","last_undegraded":"2024-01-27T02:32:06.648445+0000","last_fullsized":"2024-01-27T02:32:06.648445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:43:05.599796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.644051+0000","last_change":"2024-01-27T02:32:05.622673+0000","last_active":"2024-01-27T02:32:06.644051+0000","last_peered":"2024-01-27T02:32:06.644051+0000","last_clean":"2024-01-27T02:32:06.644051+0000","last_became_active":"2024-01-27T02:32:05.622195+0000","last_became_peered":"2024-01-27T02:32:05.622195+0000","last_unstale":"2024-01-27T02:32:06.644051+0000","last_undegraded":"2024-01-27T02:32:06.644051+0000","last_fullsized":"2024-01-27T02:32:06.644051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.637753+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:32:06.637753+0000","last_peered":"2024-01-27T02:32:06.637753+0000","last_clean":"2024-01-27T02:32:06.637753+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:32:06.637753+0000","last_undegraded":"2024-01-27T02:32:06.637753+0000","last_fullsized":"2024-01-27T02:32:06.637753+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.646119+0000","last_change":"2024-01-27T02:32:06.646043+0000","last_active":"2024-01-27T02:32:06.646119+0000","last_peered":"2024-01-27T02:32:06.646119+0000","last_clean":"2024-01-27T02:32:06.646119+0000","last_became_active":"2024-01-27T02:32:06.645836+0000","last_became_peered":"2024-01-27T02:32:06.645836+0000","last_unstale":"2024-01-27T02:32:06.646119+0000","last_undegraded":"2024-01-27T02:32:06.646119+0000","last_fullsized":"2024-01-27T02:32:06.646119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.636869+0000","last_change":"2024-01-27T02:32:05.626552+0000","last_active":"2024-01-27T02:32:06.636869+0000","last_peered":"2024-01-27T02:32:06.636869+0000","last_clean":"2024-01-27T02:32:06.636869+0000","last_became_active":"2024-01-27T02:32:05.626348+0000","last_became_peered":"2024-01-27T02:32:05.626348+0000","last_unstale":"2024-01-27T02:32:06.636869+0000","last_undegraded":"2024-01-27T02:32:06.636869+0000","last_fullsized":"2024-01-27T02:32:06.636869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:18.034484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.636979+0000","last_change":"2024-01-27T02:32:05.625885+0000","last_active":"2024-01-27T02:32:06.636979+0000","last_peered":"2024-01-27T02:32:06.636979+0000","last_clean":"2024-01-27T02:32:06.636979+0000","last_became_active":"2024-01-27T02:32:05.622692+0000","last_became_peered":"2024-01-27T02:32:05.622692+0000","last_unstale":"2024-01-27T02:32:06.636979+0000","last_undegraded":"2024-01-27T02:32:06.636979+0000","last_fullsized":"2024-01-27T02:32:06.636979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:06.920232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.639545+0000","last_change":"2024-01-27T02:32:06.638962+0000","last_active":"2024-01-27T02:32:06.639545+0000","last_peered":"2024-01-27T02:32:06.639545+0000","last_clean":"2024-01-27T02:32:06.639545+0000","last_became_active":"2024-01-27T02:32:06.638238+0000","last_became_peered":"2024-01-27T02:32:06.638238+0000","last_unstale":"2024-01-27T02:32:06.639545+0000","last_undegraded":"2024-01-27T02:32:06.639545+0000","last_fullsized":"2024-01-27T02:32:06.639545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:44:13.569068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-27T02:32:07.864093+0000","last_change":"2024-01-27T02:32:05.621014+0000","last_active":"2024-01-27T02:32:07.864093+0000","last_peered":"2024-01-27T02:32:07.864093+0000","last_clean":"2024-01-27T02:32:07.864093+0000","last_became_active":"2024-01-27T02:32:05.620775+0000","last_became_peered":"2024-01-27T02:32:05.620775+0000","last_unstale":"2024-01-27T02:32:07.864093+0000","last_undegraded":"2024-01-27T02:32:07.864093+0000","last_fullsized":"2024-01-27T02:32:07.864093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642354+0000","last_change":"2024-01-27T02:32:06.642266+0000","last_active":"2024-01-27T02:32:06.642354+0000","last_peered":"2024-01-27T02:32:06.642354+0000","last_clean":"2024-01-27T02:32:06.642354+0000","last_became_active":"2024-01-27T02:32:06.641998+0000","last_became_peered":"2024-01-27T02:32:06.641998+0000","last_unstale":"2024-01-27T02:32:06.642354+0000","last_undegraded":"2024-01-27T02:32:06.642354+0000","last_fullsized":"2024-01-27T02:32:06.642354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:58:54.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.802073+0000","last_change":"2024-01-27T02:32:05.624870+0000","last_active":"2024-01-27T02:32:06.802073+0000","last_peered":"2024-01-27T02:32:06.802073+0000","last_clean":"2024-01-27T02:32:06.802073+0000","last_became_active":"2024-01-27T02:32:05.624644+0000","last_became_peered":"2024-01-27T02:32:05.624644+0000","last_unstale":"2024-01-27T02:32:06.802073+0000","last_undegraded":"2024-01-27T02:32:06.802073+0000","last_fullsized":"2024-01-27T02:32:06.802073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.637689+0000","last_change":"2024-01-27T02:32:06.637609+0000","last_active":"2024-01-27T02:32:06.637689+0000","last_peered":"2024-01-27T02:32:06.637689+0000","last_clean":"2024-01-27T02:32:06.637689+0000","last_became_active":"2024-01-27T02:32:06.637282+0000","last_became_peered":"2024-01-27T02:32:06.637282+0000","last_unstale":"2024-01-27T02:32:06.637689+0000","last_undegraded":"2024-01-27T02:32:06.637689+0000","last_fullsized":"2024-01-27T02:32:06.637689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:06.985231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.637206+0000","last_change":"2024-01-27T02:32:05.621287+0000","last_active":"2024-01-27T02:32:06.637206+0000","last_peered":"2024-01-27T02:32:06.637206+0000","last_clean":"2024-01-27T02:32:06.637206+0000","last_became_active":"2024-01-27T02:32:05.620002+0000","last_became_peered":"2024-01-27T02:32:05.620002+0000","last_unstale":"2024-01-27T02:32:06.637206+0000","last_undegraded":"2024-01-27T02:32:06.637206+0000","last_fullsized":"2024-01-27T02:32:06.637206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:44.489465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.637891+0000","last_change":"2024-01-27T02:32:06.637830+0000","last_active":"2024-01-27T02:32:06.637891+0000","last_peered":"2024-01-27T02:32:06.637891+0000","last_clean":"2024-01-27T02:32:06.637891+0000","last_became_active":"2024-01-27T02:32:06.637387+0000","last_became_peered":"2024-01-27T02:32:06.637387+0000","last_unstale":"2024-01-27T02:32:06.637891+0000","last_undegraded":"2024-01-27T02:32:06.637891+0000","last_fullsized":"2024-01-27T02:32:06.637891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:47.819053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.632380+0000","last_change":"2024-01-27T02:32:05.623449+0000","last_active":"2024-01-27T02:32:06.632380+0000","last_peered":"2024-01-27T02:32:06.632380+0000","last_clean":"2024-01-27T02:32:06.632380+0000","last_became_active":"2024-01-27T02:32:05.623193+0000","last_became_peered":"2024-01-27T02:32:05.623193+0000","last_unstale":"2024-01-27T02:32:06.632380+0000","last_undegraded":"2024-01-27T02:32:06.632380+0000","last_fullsized":"2024-01-27T02:32:06.632380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:44.489465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.632750+0000","last_change":"2024-01-27T02:32:06.632652+0000","last_active":"2024-01-27T02:32:06.632750+0000","last_peered":"2024-01-27T02:32:06.632750+0000","last_clean":"2024-01-27T02:32:06.632750+0000","last_became_active":"2024-01-27T02:32:06.632366+0000","last_became_peered":"2024-01-27T02:32:06.632366+0000","last_unstale":"2024-01-27T02:32:06.632750+0000","last_undegraded":"2024-01-27T02:32:06.632750+0000","last_fullsized":"2024-01-27T02:32:06.632750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:06.985231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.631717+0000","last_change":"2024-01-27T02:32:05.625193+0000","last_active":"2024-01-27T02:32:06.631717+0000","last_peered":"2024-01-27T02:32:06.631717+0000","last_clean":"2024-01-27T02:32:06.631717+0000","last_became_active":"2024-01-27T02:32:05.624951+0000","last_became_peered":"2024-01-27T02:32:05.624951+0000","last_unstale":"2024-01-27T02:32:06.631717+0000","last_undegraded":"2024-01-27T02:32:06.631717+0000","last_fullsized":"2024-01-27T02:32:06.631717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:04:28.890844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642383+0000","last_change":"2024-01-27T02:32:06.642267+0000","last_active":"2024-01-27T02:32:06.642383+0000","last_peered":"2024-01-27T02:32:06.642383+0000","last_clean":"2024-01-27T02:32:06.642383+0000","last_became_active":"2024-01-27T02:32:06.640004+0000","last_became_peered":"2024-01-27T02:32:06.640004+0000","last_unstale":"2024-01-27T02:32:06.642383+0000","last_undegraded":"2024-01-27T02:32:06.642383+0000","last_fullsized":"2024-01-27T02:32:06.642383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:45:37.035302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.801718+0000","last_change":"2024-01-27T02:32:05.625673+0000","last_active":"2024-01-27T02:32:06.801718+0000","last_peered":"2024-01-27T02:32:06.801718+0000","last_clean":"2024-01-27T02:32:06.801718+0000","last_became_active":"2024-01-27T02:32:05.623551+0000","last_became_peered":"2024-01-27T02:32:05.623551+0000","last_unstale":"2024-01-27T02:32:06.801718+0000","last_undegraded":"2024-01-27T02:32:06.801718+0000","last_fullsized":"2024-01-27T02:32:06.801718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.639544+0000","last_change":"2024-01-27T02:32:06.639489+0000","last_active":"2024-01-27T02:32:06.639544+0000","last_peered":"2024-01-27T02:32:06.639544+0000","last_clean":"2024-01-27T02:32:06.639544+0000","last_became_active":"2024-01-27T02:32:06.639246+0000","last_became_peered":"2024-01-27T02:32:06.639246+0000","last_unstale":"2024-01-27T02:32:06.639544+0000","last_undegraded":"2024-01-27T02:32:06.639544+0000","last_fullsized":"2024-01-27T02:32:06.639544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.629629+0000","last_change":"2024-01-27T02:32:05.620877+0000","last_active":"2024-01-27T02:32:06.629629+0000","last_peered":"2024-01-27T02:32:06.629629+0000","last_clean":"2024-01-27T02:32:06.629629+0000","last_became_active":"2024-01-27T02:32:05.620600+0000","last_became_peered":"2024-01-27T02:32:05.620600+0000","last_unstale":"2024-01-27T02:32:06.629629+0000","last_undegraded":"2024-01-27T02:32:06.629629+0000","last_fullsized":"2024-01-27T02:32:06.629629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.650085+0000","last_change":"2024-01-27T02:32:06.649956+0000","last_active":"2024-01-27T02:32:06.650085+0000","last_peered":"2024-01-27T02:32:06.650085+0000","last_clean":"2024-01-27T02:32:06.650085+0000","last_became_active":"2024-01-27T02:32:06.649759+0000","last_became_peered":"2024-01-27T02:32:06.649759+0000","last_unstale":"2024-01-27T02:32:06.650085+0000","last_undegraded":"2024-01-27T02:32:06.650085+0000","last_fullsized":"2024-01-27T02:32:06.650085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:56:39.515436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.801885+0000","last_change":"2024-01-27T02:32:05.619486+0000","last_active":"2024-01-27T02:32:06.801885+0000","last_peered":"2024-01-27T02:32:06.801885+0000","last_clean":"2024-01-27T02:32:06.801885+0000","last_became_active":"2024-01-27T02:32:05.619147+0000","last_became_peered":"2024-01-27T02:32:05.619147+0000","last_unstale":"2024-01-27T02:32:06.801885+0000","last_undegraded":"2024-01-27T02:32:06.801885+0000","last_fullsized":"2024-01-27T02:32:06.801885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.650011+0000","last_change":"2024-01-27T02:32:06.649606+0000","last_active":"2024-01-27T02:32:06.650011+0000","last_peered":"2024-01-27T02:32:06.650011+0000","last_clean":"2024-01-27T02:32:06.650011+0000","last_became_active":"2024-01-27T02:32:06.648805+0000","last_became_peered":"2024-01-27T02:32:06.648805+0000","last_unstale":"2024-01-27T02:32:06.650011+0000","last_undegraded":"2024-01-27T02:32:06.650011+0000","last_fullsized":"2024-01-27T02:32:06.650011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:31:12.724216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.804019+0000","last_change":"2024-01-27T02:32:05.613146+0000","last_active":"2024-01-27T02:32:06.804019+0000","last_peered":"2024-01-27T02:32:06.804019+0000","last_clean":"2024-01-27T02:32:06.804019+0000","last_became_active":"2024-01-27T02:32:05.612852+0000","last_became_peered":"2024-01-27T02:32:05.612852+0000","last_unstale":"2024-01-27T02:32:06.804019+0000","last_undegraded":"2024-01-27T02:32:06.804019+0000","last_fullsized":"2024-01-27T02:32:06.804019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.641582+0000","last_change":"2024-01-27T02:32:06.641444+0000","last_active":"2024-01-27T02:32:06.641582+0000","last_peered":"2024-01-27T02:32:06.641582+0000","last_clean":"2024-01-27T02:32:06.641582+0000","last_became_active":"2024-01-27T02:32:06.640698+0000","last_became_peered":"2024-01-27T02:32:06.640698+0000","last_unstale":"2024-01-27T02:32:06.641582+0000","last_undegraded":"2024-01-27T02:32:06.641582+0000","last_fullsized":"2024-01-27T02:32:06.641582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:15:38.764945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.639806+0000","last_change":"2024-01-27T02:32:06.639727+0000","last_active":"2024-01-27T02:32:06.639806+0000","last_peered":"2024-01-27T02:32:06.639806+0000","last_clean":"2024-01-27T02:32:06.639806+0000","last_became_active":"2024-01-27T02:32:06.639457+0000","last_became_peered":"2024-01-27T02:32:06.639457+0000","last_unstale":"2024-01-27T02:32:06.639806+0000","last_undegraded":"2024-01-27T02:32:06.639806+0000","last_fullsized":"2024-01-27T02:32:06.639806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:17:03.351766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.646128+0000","last_change":"2024-01-27T02:32:06.646051+0000","last_active":"2024-01-27T02:32:06.646128+0000","last_peered":"2024-01-27T02:32:06.646128+0000","last_clean":"2024-01-27T02:32:06.646128+0000","last_became_active":"2024-01-27T02:32:06.645857+0000","last_became_peered":"2024-01-27T02:32:06.645857+0000","last_unstale":"2024-01-27T02:32:06.646128+0000","last_undegraded":"2024-01-27T02:32:06.646128+0000","last_fullsized":"2024-01-27T02:32:06.646128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.648193+0000","last_change":"2024-01-27T02:32:06.648063+0000","last_active":"2024-01-27T02:32:06.648193+0000","last_peered":"2024-01-27T02:32:06.648193+0000","last_clean":"2024-01-27T02:32:06.648193+0000","last_became_active":"2024-01-27T02:32:06.647524+0000","last_became_peered":"2024-01-27T02:32:06.647524+0000","last_unstale":"2024-01-27T02:32:06.648193+0000","last_undegraded":"2024-01-27T02:32:06.648193+0000","last_fullsized":"2024-01-27T02:32:06.648193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:56:39.515436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.639513+0000","last_change":"2024-01-27T02:32:06.639430+0000","last_active":"2024-01-27T02:32:06.639513+0000","last_peered":"2024-01-27T02:32:06.639513+0000","last_clean":"2024-01-27T02:32:06.639513+0000","last_became_active":"2024-01-27T02:32:06.639149+0000","last_became_peered":"2024-01-27T02:32:06.639149+0000","last_unstale":"2024-01-27T02:32:06.639513+0000","last_undegraded":"2024-01-27T02:32:06.639513+0000","last_fullsized":"2024-01-27T02:32:06.639513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:36:11.295485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.641457+0000","last_change":"2024-01-27T02:32:06.640757+0000","last_active":"2024-01-27T02:32:06.641457+0000","last_peered":"2024-01-27T02:32:06.641457+0000","last_clean":"2024-01-27T02:32:06.641457+0000","last_became_active":"2024-01-27T02:32:06.637783+0000","last_became_peered":"2024-01-27T02:32:06.637783+0000","last_unstale":"2024-01-27T02:32:06.641457+0000","last_undegraded":"2024-01-27T02:32:06.641457+0000","last_fullsized":"2024-01-27T02:32:06.641457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:44:13.569068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.633658+0000","last_change":"2024-01-27T02:32:06.633366+0000","last_active":"2024-01-27T02:32:06.633658+0000","last_peered":"2024-01-27T02:32:06.633658+0000","last_clean":"2024-01-27T02:32:06.633658+0000","last_became_active":"2024-01-27T02:32:06.633117+0000","last_became_peered":"2024-01-27T02:32:06.633117+0000","last_unstale":"2024-01-27T02:32:06.633658+0000","last_undegraded":"2024-01-27T02:32:06.633658+0000","last_fullsized":"2024-01-27T02:32:06.633658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.645347+0000","last_change":"2024-01-27T02:32:06.645277+0000","last_active":"2024-01-27T02:32:06.645347+0000","last_peered":"2024-01-27T02:32:06.645347+0000","last_clean":"2024-01-27T02:32:06.645347+0000","last_became_active":"2024-01-27T02:32:06.645042+0000","last_became_peered":"2024-01-27T02:32:06.645042+0000","last_unstale":"2024-01-27T02:32:06.645347+0000","last_undegraded":"2024-01-27T02:32:06.645347+0000","last_fullsized":"2024-01-27T02:32:06.645347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:44:13.569068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.637735+0000","last_change":"2024-01-27T02:32:06.636359+0000","last_active":"2024-01-27T02:32:06.637735+0000","last_peered":"2024-01-27T02:32:06.637735+0000","last_clean":"2024-01-27T02:32:06.637735+0000","last_became_active":"2024-01-27T02:32:06.636084+0000","last_became_peered":"2024-01-27T02:32:06.636084+0000","last_unstale":"2024-01-27T02:32:06.637735+0000","last_undegraded":"2024-01-27T02:32:06.637735+0000","last_fullsized":"2024-01-27T02:32:06.637735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.648901+0000","last_change":"2024-01-27T02:32:06.648767+0000","last_active":"2024-01-27T02:32:06.648901+0000","last_peered":"2024-01-27T02:32:06.648901+0000","last_clean":"2024-01-27T02:32:06.648901+0000","last_became_active":"2024-01-27T02:32:06.647799+0000","last_became_peered":"2024-01-27T02:32:06.647799+0000","last_unstale":"2024-01-27T02:32:06.648901+0000","last_undegraded":"2024-01-27T02:32:06.648901+0000","last_fullsized":"2024-01-27T02:32:06.648901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:36:11.295485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.641482+0000","last_change":"2024-01-27T02:32:06.640928+0000","last_active":"2024-01-27T02:32:06.641482+0000","last_peered":"2024-01-27T02:32:06.641482+0000","last_clean":"2024-01-27T02:32:06.641482+0000","last_became_active":"2024-01-27T02:32:06.638110+0000","last_became_peered":"2024-01-27T02:32:06.638110+0000","last_unstale":"2024-01-27T02:32:06.641482+0000","last_undegraded":"2024-01-27T02:32:06.641482+0000","last_fullsized":"2024-01-27T02:32:06.641482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:17:03.351766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.649522+0000","last_change":"2024-01-27T02:32:06.649302+0000","last_active":"2024-01-27T02:32:06.649522+0000","last_peered":"2024-01-27T02:32:06.649522+0000","last_clean":"2024-01-27T02:32:06.649522+0000","last_became_active":"2024-01-27T02:32:06.648315+0000","last_became_peered":"2024-01-27T02:32:06.648315+0000","last_unstale":"2024-01-27T02:32:06.649522+0000","last_undegraded":"2024-01-27T02:32:06.649522+0000","last_fullsized":"2024-01-27T02:32:06.649522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:44:24.628811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.646984+0000","last_change":"2024-01-27T02:32:06.646935+0000","last_active":"2024-01-27T02:32:06.646984+0000","last_peered":"2024-01-27T02:32:06.646984+0000","last_clean":"2024-01-27T02:32:06.646984+0000","last_became_active":"2024-01-27T02:32:06.646591+0000","last_became_peered":"2024-01-27T02:32:06.646591+0000","last_unstale":"2024-01-27T02:32:06.646984+0000","last_undegraded":"2024-01-27T02:32:06.646984+0000","last_fullsized":"2024-01-27T02:32:06.646984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.636417+0000","last_change":"2024-01-27T02:32:06.636337+0000","last_active":"2024-01-27T02:32:06.636417+0000","last_peered":"2024-01-27T02:32:06.636417+0000","last_clean":"2024-01-27T02:32:06.636417+0000","last_became_active":"2024-01-27T02:32:06.636086+0000","last_became_peered":"2024-01-27T02:32:06.636086+0000","last_unstale":"2024-01-27T02:32:06.636417+0000","last_undegraded":"2024-01-27T02:32:06.636417+0000","last_fullsized":"2024-01-27T02:32:06.636417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194662,"statfs":{"total":200000000,"available":199333925,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":74,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195237,"statfs":{"total":200000000,"available":199923311,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194661,"statfs":{"total":200000000,"available":199333869,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195239,"statfs":{"total":200000000,"available":199924847,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":74,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195237,"statfs":{"total":200000000,"available":199923311,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924721,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195239,"statfs":{"total":200000000,"available":199924757,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]}]}],"pool_statfs":[]}} 2024-01-27T02:32:16.055 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-27T02:32:16.055 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-01-27T02:32:16.055 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-01-27T02:32:16.055 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-01-27T02:32:16.056 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-01-27T02:32:16.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.224+0000 7f6af2b0e700 1 Processor -- start 2024-01-27T02:32:16.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.224+0000 7f6af2b0e700 1 -- start start 2024-01-27T02:32:16.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.225+0000 7f6af2b0e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6aec0c9f80 0x7f6aec157f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:16.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.225+0000 7f6af2b0e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aec06c880 con 0x7f6aec0c9bb0 2024-01-27T02:32:16.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.225+0000 7f6af2b0e700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aec06bc70 con 0x7f6aec06e330 2024-01-27T02:32:16.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.225+0000 7f6af2b0e700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aec06cd10 con 0x7f6aec0c9f80 2024-01-27T02:32:16.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.225+0000 7f6af08aa700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6aec0c9bb0 0x7f6aec06b400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51348/0 (socket says 172.21.15.7:51348) 2024-01-27T02:32:16.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.225+0000 7f6af08aa700 1 -- 172.21.15.7:0/2128904701 learned_addr learned my addr 172.21.15.7:0/2128904701 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:16.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.225+0000 7f6aebfff700 1 --2- 172.21.15.7:0/2128904701 >> v2:172.21.15.188:3300/0 conn(0x7f6aec0c9f80 0x7f6aec157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:16.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.225+0000 7f6aeb7fe700 1 -- 172.21.15.7:0/2128904701 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607251274 0 0) 0x7f6aec06c880 con 0x7f6aec0c9bb0 2024-01-27T02:32:16.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.225+0000 7f6aeb7fe700 1 -- 172.21.15.7:0/2128904701 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6adc003570 con 0x7f6aec0c9bb0 2024-01-27T02:32:16.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.225+0000 7f6aeb7fe700 1 -- 172.21.15.7:0/2128904701 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3265183309 0 0) 0x7f6aec06bc70 con 0x7f6aec06e330 2024-01-27T02:32:16.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.225+0000 7f6aeb7fe700 1 -- 172.21.15.7:0/2128904701 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6aec06c880 con 0x7f6aec06e330 2024-01-27T02:32:16.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.225+0000 7f6aeb7fe700 1 -- 172.21.15.7:0/2128904701 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3557847497 0 0) 0x7f6adc003570 con 0x7f6aec0c9bb0 2024-01-27T02:32:16.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.225+0000 7f6aebfff700 1 -- 172.21.15.7:0/2128904701 >> v1:172.21.15.188:6789/0 conn(0x7f6aec06e330 legacy=0x7f6aec06e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:16.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.225+0000 7f6aebfff700 1 -- 172.21.15.7:0/2128904701 >> v1:172.21.15.7:6789/0 conn(0x7f6aec0c9bb0 legacy=0x7f6aec06b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:16.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.225+0000 7f6aebfff700 1 -- 172.21.15.7:0/2128904701 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec06bf50 con 0x7f6aec0c9f80 2024-01-27T02:32:16.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.226+0000 7f6aebfff700 1 --2- 172.21.15.7:0/2128904701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6aec0c9f80 0x7f6aec157f70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80098e0 tx=0x7f6ad80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=244243b063cb495d server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:16.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.226+0000 7f6aeb7fe700 1 -- 172.21.15.7:0/2128904701 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0001470 con 0x7f6aec0c9f80 2024-01-27T02:32:16.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.226+0000 7f6aeb7fe700 1 -- 172.21.15.7:0/2128904701 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad8011420 con 0x7f6aec0c9f80 2024-01-27T02:32:16.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.226+0000 7f6aeb7fe700 1 -- 172.21.15.7:0/2128904701 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad80117c0 con 0x7f6aec0c9f80 2024-01-27T02:32:16.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.226+0000 7f6af2b0e700 1 -- 172.21.15.7:0/2128904701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6aec0c9f80 msgr2=0x7f6aec157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:16.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.226+0000 7f6af2b0e700 1 --2- 172.21.15.7:0/2128904701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6aec0c9f80 0x7f6aec157f70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80098e0 tx=0x7f6ad80091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.227+0000 7f6af2b0e700 1 -- 172.21.15.7:0/2128904701 shutdown_connections 2024-01-27T02:32:16.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.227+0000 7f6af2b0e700 1 --2- 172.21.15.7:0/2128904701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6aec0c9f80 0x7f6aec157f70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.227+0000 7f6af2b0e700 1 -- 172.21.15.7:0/2128904701 >> 172.21.15.7:0/2128904701 conn(0x7f6aec0cdc30 msgr2=0x7f6aec0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:16.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.227+0000 7f6af2b0e700 1 -- 172.21.15.7:0/2128904701 shutdown_connections 2024-01-27T02:32:16.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.227+0000 7f6af2b0e700 1 -- 172.21.15.7:0/2128904701 wait complete. 2024-01-27T02:32:16.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.228+0000 7f6af2b0e700 1 Processor -- start 2024-01-27T02:32:16.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.228+0000 7f6af2b0e700 1 -- start start 2024-01-27T02:32:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.229+0000 7f6af2b0e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6aec06e330 0x7f6aec0b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.229+0000 7f6af2b0e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6aec0c9bb0 0x7f6aec138940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.229+0000 7f6af2b0e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aec0c9f80 0x7f6aec15cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.229+0000 7f6af2b0e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec0c44d0 con 0x7f6aec0c9f80 2024-01-27T02:32:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.229+0000 7f6af2b0e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec138e80 con 0x7f6aec06e330 2024-01-27T02:32:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.229+0000 7f6af2b0e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec0b13c0 con 0x7f6aec0c9bb0 2024-01-27T02:32:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.229+0000 7f6af10ab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aec0c9f80 0x7f6aec15cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.229+0000 7f6af10ab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aec0c9f80 0x7f6aec15cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53602/0 (socket says 172.21.15.7:53602) 2024-01-27T02:32:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.229+0000 7f6af10ab700 1 -- 172.21.15.7:0/2025402148 learned_addr learned my addr 172.21.15.7:0/2025402148 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.229+0000 7f6aebfff700 1 --2- 172.21.15.7:0/2025402148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6aec0c9bb0 0x7f6aec138940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.229+0000 7f6af08aa700 1 --2- 172.21.15.7:0/2025402148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6aec06e330 0x7f6aec0b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.229+0000 7f6af10ab700 1 -- 172.21.15.7:0/2025402148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6aec0c9bb0 msgr2=0x7f6aec138940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.229+0000 7f6af10ab700 1 --2- 172.21.15.7:0/2025402148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6aec0c9bb0 0x7f6aec138940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.229+0000 7f6af10ab700 1 -- 172.21.15.7:0/2025402148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6aec06e330 msgr2=0x7f6aec0b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.229+0000 7f6af10ab700 1 --2- 172.21.15.7:0/2025402148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6aec06e330 0x7f6aec0b34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.229+0000 7f6af10ab700 1 -- 172.21.15.7:0/2025402148 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad8009420 con 0x7f6aec0c9f80 2024-01-27T02:32:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.229+0000 7f6af10ab700 1 --2- 172.21.15.7:0/2025402148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aec0c9f80 0x7f6aec15cbd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400c910 tx=0x7f6ae4007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.229+0000 7f6ae8ff9700 1 -- 172.21.15.7:0/2025402148 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4012630 con 0x7f6aec0c9f80 2024-01-27T02:32:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.230+0000 7f6ae8ff9700 1 -- 172.21.15.7:0/2025402148 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae4012790 con 0x7f6aec0c9f80 2024-01-27T02:32:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.230+0000 7f6ae8ff9700 1 -- 172.21.15.7:0/2025402148 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4012b50 con 0x7f6aec0c9f80 2024-01-27T02:32:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.230+0000 7f6af2b0e700 1 -- 172.21.15.7:0/2025402148 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aec15d110 con 0x7f6aec0c9f80 2024-01-27T02:32:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.230+0000 7f6af2b0e700 1 -- 172.21.15.7:0/2025402148 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aec15d3a0 con 0x7f6aec0c9f80 2024-01-27T02:32:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.231+0000 7f6af2b0e700 1 -- 172.21.15.7:0/2025402148 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aec04c710 con 0x7f6aec0c9f80 2024-01-27T02:32:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.231+0000 7f6ae8ff9700 1 -- 172.21.15.7:0/2025402148 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6ae401a3b0 con 0x7f6aec0c9f80 2024-01-27T02:32:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.231+0000 7f6ae8ff9700 1 --2- 172.21.15.7:0/2025402148 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6ad0066f30 0x7f6ad00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.232+0000 7f6ae8ff9700 1 -- 172.21.15.7:0/2025402148 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6ae4084530 con 0x7f6aec0c9f80 2024-01-27T02:32:16.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.232+0000 7f6af08aa700 1 --2- 172.21.15.7:0/2025402148 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6ad0066f30 0x7f6ad00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:16.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.232+0000 7f6af08aa700 1 --2- 172.21.15.7:0/2025402148 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6ad0066f30 0x7f6ad00693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0001ed0 tx=0x7f6ae00019d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:16.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.236+0000 7f6ae8ff9700 1 -- 172.21.15.7:0/2025402148 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ae40517a0 con 0x7f6aec0c9f80 2024-01-27T02:32:16.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.499+0000 7f6af2b0e700 1 -- 172.21.15.7:0/2025402148 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f6aec0b1500 con 0x7f6aec0c9f80 2024-01-27T02:32:16.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.502+0000 7f6ae8ff9700 1 -- 172.21.15.7:0/2025402148 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163706 (secure 0 0 0) 0x7f6ae4054dc0 con 0x7f6aec0c9f80 2024-01-27T02:32:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:16.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.505+0000 7f6af2b0e700 1 -- 172.21.15.7:0/2025402148 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6ad0066f30 msgr2=0x7f6ad00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:16.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.505+0000 7f6af2b0e700 1 --2- 172.21.15.7:0/2025402148 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6ad0066f30 0x7f6ad00693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0001ed0 tx=0x7f6ae00019d0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.505+0000 7f6af2b0e700 1 -- 172.21.15.7:0/2025402148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aec0c9f80 msgr2=0x7f6aec15cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:16.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.505+0000 7f6af2b0e700 1 --2- 172.21.15.7:0/2025402148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aec0c9f80 0x7f6aec15cbd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400c910 tx=0x7f6ae4007f90 comp rx=0 tx=0).stop 2024-01-27T02:32:16.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.505+0000 7f6af2b0e700 1 -- 172.21.15.7:0/2025402148 shutdown_connections 2024-01-27T02:32:16.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.505+0000 7f6af2b0e700 1 --2- 172.21.15.7:0/2025402148 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6ad0066f30 0x7f6ad00693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.505+0000 7f6af2b0e700 1 --2- 172.21.15.7:0/2025402148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aec0c9f80 0x7f6aec15cbd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.505+0000 7f6af2b0e700 1 --2- 172.21.15.7:0/2025402148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6aec06e330 0x7f6aec0b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.505+0000 7f6af2b0e700 1 --2- 172.21.15.7:0/2025402148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6aec0c9bb0 0x7f6aec138940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.505+0000 7f6af2b0e700 1 -- 172.21.15.7:0/2025402148 >> 172.21.15.7:0/2025402148 conn(0x7f6aec0cdc30 msgr2=0x7f6aec0c8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:16.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.505+0000 7f6af2b0e700 1 -- 172.21.15.7:0/2025402148 shutdown_connections 2024-01-27T02:32:16.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.505+0000 7f6af2b0e700 1 -- 172.21.15.7:0/2025402148 wait complete. 2024-01-27T02:32:16.520 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":5,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":38712},{"type":"v1","addr":"172.21.15.188:6833","nonce":38712}]},"active_addr":"172.21.15.188:6833/38712","active_change":"2024-01-27T02:31:55.561040+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.188:0","nonce":222227063}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.188:0","nonce":608547207}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.188:0","nonce":2623201448}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.188:0","nonce":2620353874}]}]} 2024-01-27T02:32:16.523 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-01-27T02:32:16.523 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-01-27T02:32:16.523 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:32:16.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.653+0000 7f62436d1700 1 Processor -- start 2024-01-27T02:32:16.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.653+0000 7f62436d1700 1 -- start start 2024-01-27T02:32:16.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.654+0000 7f62436d1700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f623c0b1c00 0x7f623c06e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:16.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.654+0000 7f62436d1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f623c15b5f0 0x7f623c1566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.654+0000 7f62436d1700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f623c157610 con 0x7f623c0b1fd0 2024-01-27T02:32:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.654+0000 7f62436d1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f623c157310 con 0x7f623c15b5f0 2024-01-27T02:32:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.654+0000 7f62436d1700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f623c06ee00 con 0x7f623c0b1c00 2024-01-27T02:32:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.654+0000 7f6240c6c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f623c15b5f0 0x7f623c1566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.654+0000 7f6240c6c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f623c15b5f0 0x7f623c1566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53614/0 (socket says 172.21.15.7:53614) 2024-01-27T02:32:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.654+0000 7f6240c6c700 1 -- 172.21.15.7:0/2791371801 learned_addr learned my addr 172.21.15.7:0/2791371801 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.655+0000 7f624146d700 1 --2- 172.21.15.7:0/2791371801 >> v2:172.21.15.188:3300/0 conn(0x7f623c0b1c00 0x7f623c06e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.655+0000 7f6240c6c700 1 -- 172.21.15.7:0/2791371801 >> v2:172.21.15.188:3300/0 conn(0x7f623c0b1c00 msgr2=0x7f623c06e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.655+0000 7f6240c6c700 1 --2- 172.21.15.7:0/2791371801 >> v2:172.21.15.188:3300/0 conn(0x7f623c0b1c00 0x7f623c06e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.655+0000 7f6240c6c700 1 -- 172.21.15.7:0/2791371801 >> v1:172.21.15.188:6789/0 conn(0x7f623c0b1fd0 legacy=0x7f623c156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.655+0000 7f6240c6c700 1 -- 172.21.15.7:0/2791371801 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c06f0e0 con 0x7f623c15b5f0 2024-01-27T02:32:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.655+0000 7f6240c6c700 1 --2- 172.21.15.7:0/2791371801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c15b5f0 0x7f623c1566c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f623c06e130 tx=0x7f622c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e0bcf23fa70ba4 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.655+0000 7f6233fff700 1 -- 172.21.15.7:0/2791371801 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c00dba0 con 0x7f623c15b5f0 2024-01-27T02:32:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.655+0000 7f6233fff700 1 -- 172.21.15.7:0/2791371801 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622c00dd00 con 0x7f623c15b5f0 2024-01-27T02:32:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.655+0000 7f6233fff700 1 -- 172.21.15.7:0/2791371801 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c0196d0 con 0x7f623c15b5f0 2024-01-27T02:32:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.656+0000 7f62436d1700 1 -- 172.21.15.7:0/2791371801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c15b5f0 msgr2=0x7f623c1566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.656+0000 7f62436d1700 1 --2- 172.21.15.7:0/2791371801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c15b5f0 0x7f623c1566c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f623c06e130 tx=0x7f622c00b4f0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.656+0000 7f62436d1700 1 -- 172.21.15.7:0/2791371801 shutdown_connections 2024-01-27T02:32:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.656+0000 7f62436d1700 1 --2- 172.21.15.7:0/2791371801 >> v2:172.21.15.188:3300/0 conn(0x7f623c0b1c00 0x7f623c06e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.656+0000 7f62436d1700 1 --2- 172.21.15.7:0/2791371801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c15b5f0 0x7f623c1566c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.656+0000 7f62436d1700 1 -- 172.21.15.7:0/2791371801 >> 172.21.15.7:0/2791371801 conn(0x7f623c068430 msgr2=0x7f623c06be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.656+0000 7f62436d1700 1 -- 172.21.15.7:0/2791371801 shutdown_connections 2024-01-27T02:32:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.656+0000 7f62436d1700 1 -- 172.21.15.7:0/2791371801 wait complete. 2024-01-27T02:32:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.657+0000 7f62436d1700 1 Processor -- start 2024-01-27T02:32:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.657+0000 7f62436d1700 1 -- start start 2024-01-27T02:32:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.658+0000 7f62436d1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c0b1c00 0x7f623c1546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.658+0000 7f62436d1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f623c0b1fd0 0x7f623c16d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.658+0000 7f62436d1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f623c15b5f0 0x7f623c0acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.658+0000 7f62436d1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c0ad080 con 0x7f623c0b1c00 2024-01-27T02:32:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.658+0000 7f62436d1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c0ade80 con 0x7f623c15b5f0 2024-01-27T02:32:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.658+0000 7f62436d1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623c0ad270 con 0x7f623c0b1fd0 2024-01-27T02:32:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.658+0000 7f624146d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c0b1c00 0x7f623c1546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.658+0000 7f624146d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c0b1c00 0x7f623c1546f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53624/0 (socket says 172.21.15.7:53624) 2024-01-27T02:32:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.658+0000 7f624146d700 1 -- 172.21.15.7:0/319033967 learned_addr learned my addr 172.21.15.7:0/319033967 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.658+0000 7f6240c6c700 1 --2- 172.21.15.7:0/319033967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f623c0b1fd0 0x7f623c16d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.658+0000 7f6241c6e700 1 --2- 172.21.15.7:0/319033967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f623c15b5f0 0x7f623c0acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.659+0000 7f624146d700 1 -- 172.21.15.7:0/319033967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f623c0b1fd0 msgr2=0x7f623c16d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.659+0000 7f624146d700 1 --2- 172.21.15.7:0/319033967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f623c0b1fd0 0x7f623c16d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.659+0000 7f624146d700 1 -- 172.21.15.7:0/319033967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f623c15b5f0 msgr2=0x7f623c0acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.659+0000 7f624146d700 1 --2- 172.21.15.7:0/319033967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f623c15b5f0 0x7f623c0acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.659+0000 7f624146d700 1 -- 172.21.15.7:0/319033967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c00b790 con 0x7f623c0b1c00 2024-01-27T02:32:16.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.659+0000 7f6240c6c700 1 --2- 172.21.15.7:0/319033967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f623c0b1fd0 0x7f623c16d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:16.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.659+0000 7f624146d700 1 --2- 172.21.15.7:0/319033967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c0b1c00 0x7f623c1546f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f623800ec80 tx=0x7f62380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:16.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.659+0000 7f62317fa700 1 -- 172.21.15.7:0/319033967 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623800c850 con 0x7f623c0b1c00 2024-01-27T02:32:16.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.659+0000 7f62317fa700 1 -- 172.21.15.7:0/319033967 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623800c9b0 con 0x7f623c0b1c00 2024-01-27T02:32:16.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.659+0000 7f62436d1700 1 -- 172.21.15.7:0/319033967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623c0ad550 con 0x7f623c0b1c00 2024-01-27T02:32:16.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.659+0000 7f62317fa700 1 -- 172.21.15.7:0/319033967 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623800cd50 con 0x7f623c0b1c00 2024-01-27T02:32:16.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.659+0000 7f62436d1700 1 -- 172.21.15.7:0/319033967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623c0adb10 con 0x7f623c0b1c00 2024-01-27T02:32:16.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.660+0000 7f62436d1700 1 -- 172.21.15.7:0/319033967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623c04c710 con 0x7f623c0b1c00 2024-01-27T02:32:16.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.660+0000 7f62317fa700 1 -- 172.21.15.7:0/319033967 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6238013680 con 0x7f623c0b1c00 2024-01-27T02:32:16.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.661+0000 7f62317fa700 1 --2- 172.21.15.7:0/319033967 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6220066f30 0x7f62200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:16.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.661+0000 7f62317fa700 1 -- 172.21.15.7:0/319033967 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6238084890 con 0x7f623c0b1c00 2024-01-27T02:32:16.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.661+0000 7f6240c6c700 1 --2- 172.21.15.7:0/319033967 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6220066f30 0x7f62200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:16.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.662+0000 7f6240c6c700 1 --2- 172.21.15.7:0/319033967 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6220066f30 0x7f62200693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f623c06e130 tx=0x7f622c01b000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.665+0000 7f62317fa700 1 -- 172.21.15.7:0/319033967 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6238051a60 con 0x7f623c0b1c00 2024-01-27T02:32:16.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.870+0000 7f62436d1700 1 -- 172.21.15.7:0/319033967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f623c06f3c0 con 0x7f623c0b1c00 2024-01-27T02:32:16.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.871+0000 7f62317fa700 1 -- 172.21.15.7:0/319033967 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f6238055080 con 0x7f623c0b1c00 2024-01-27T02:32:16.873 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:16.873 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:32:07.627822+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:32:03.675611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-27T02:32:05.018885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:32:16.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.873+0000 7f62436d1700 1 -- 172.21.15.7:0/319033967 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6220066f30 msgr2=0x7f62200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:16.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.873+0000 7f62436d1700 1 --2- 172.21.15.7:0/319033967 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6220066f30 0x7f62200693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f623c06e130 tx=0x7f622c01b000 comp rx=0 tx=0).stop 2024-01-27T02:32:16.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.873+0000 7f62436d1700 1 -- 172.21.15.7:0/319033967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c0b1c00 msgr2=0x7f623c1546f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:16.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.873+0000 7f62436d1700 1 --2- 172.21.15.7:0/319033967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c0b1c00 0x7f623c1546f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f623800ec80 tx=0x7f62380091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.873+0000 7f62436d1700 1 -- 172.21.15.7:0/319033967 shutdown_connections 2024-01-27T02:32:16.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.873+0000 7f62436d1700 1 --2- 172.21.15.7:0/319033967 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6220066f30 0x7f62200693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.873+0000 7f62436d1700 1 --2- 172.21.15.7:0/319033967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f623c15b5f0 0x7f623c0acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.873+0000 7f62436d1700 1 --2- 172.21.15.7:0/319033967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623c0b1c00 0x7f623c1546f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.873+0000 7f62436d1700 1 --2- 172.21.15.7:0/319033967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f623c0b1fd0 0x7f623c16d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:16.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.873+0000 7f62436d1700 1 -- 172.21.15.7:0/319033967 >> 172.21.15.7:0/319033967 conn(0x7f623c068430 msgr2=0x7f623c157bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:16.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.873+0000 7f62436d1700 1 -- 172.21.15.7:0/319033967 shutdown_connections 2024-01-27T02:32:16.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:16.873+0000 7f62436d1700 1 -- 172.21.15.7:0/319033967 wait complete. 2024-01-27T02:32:16.888 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-01-27T02:32:16.888 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:32:17.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.018+0000 7f912faff700 1 Processor -- start 2024-01-27T02:32:17.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.018+0000 7f912faff700 1 -- start start 2024-01-27T02:32:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.019+0000 7f912faff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f91280a7c10 0x7f91280ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.019+0000 7f912faff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91280ae880 0x7f912806f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.019+0000 7f912faff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f912815a880 con 0x7f91280a7fe0 2024-01-27T02:32:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.019+0000 7f912faff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91280afde0 con 0x7f91280a7c10 2024-01-27T02:32:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.019+0000 7f912faff700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91280af950 con 0x7f91280ae880 2024-01-27T02:32:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.019+0000 7f912d89b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f91280a7c10 0x7f91280ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.019+0000 7f912e09c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f91280a7fe0 0x7f912806dae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51360/0 (socket says 172.21.15.7:51360) 2024-01-27T02:32:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.019+0000 7f912e09c700 1 -- 172.21.15.7:0/2467855782 learned_addr learned my addr 172.21.15.7:0/2467855782 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.019+0000 7f912d09a700 1 --2- 172.21.15.7:0/2467855782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91280ae880 0x7f912806f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.019+0000 7f912d89b700 1 -- 172.21.15.7:0/2467855782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91280ae880 msgr2=0x7f912806f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.019+0000 7f912d89b700 1 --2- 172.21.15.7:0/2467855782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91280ae880 0x7f912806f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.019+0000 7f912d89b700 1 -- 172.21.15.7:0/2467855782 >> v1:172.21.15.7:6789/0 conn(0x7f91280a7fe0 legacy=0x7f912806dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.019+0000 7f912d89b700 1 -- 172.21.15.7:0/2467855782 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91280aed40 con 0x7f91280a7c10 2024-01-27T02:32:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.019+0000 7f912c899700 1 -- 172.21.15.7:0/2467855782 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4234049219 0 0) 0x7f912815a880 con 0x7f91280a7fe0 2024-01-27T02:32:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.020+0000 7f912d89b700 1 --2- 172.21.15.7:0/2467855782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91280a7c10 0x7f91280ae340 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f91240098e0 tx=0x7f91240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6796d99222f24cb4 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.020+0000 7f912c899700 1 -- 172.21.15.7:0/2467855782 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912400b640 con 0x7f91280a7c10 2024-01-27T02:32:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.020+0000 7f912c899700 1 -- 172.21.15.7:0/2467855782 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912400b7a0 con 0x7f91280a7c10 2024-01-27T02:32:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.020+0000 7f912c899700 1 -- 172.21.15.7:0/2467855782 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912400bb40 con 0x7f91280a7c10 2024-01-27T02:32:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.020+0000 7f912faff700 1 -- 172.21.15.7:0/2467855782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91280a7c10 msgr2=0x7f91280ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.020+0000 7f912faff700 1 --2- 172.21.15.7:0/2467855782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91280a7c10 0x7f91280ae340 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f91240098e0 tx=0x7f91240091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.020+0000 7f912faff700 1 -- 172.21.15.7:0/2467855782 shutdown_connections 2024-01-27T02:32:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.020+0000 7f912faff700 1 --2- 172.21.15.7:0/2467855782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91280a7c10 0x7f91280ae340 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.020+0000 7f912faff700 1 --2- 172.21.15.7:0/2467855782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91280ae880 0x7f912806f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.020+0000 7f912faff700 1 -- 172.21.15.7:0/2467855782 >> 172.21.15.7:0/2467855782 conn(0x7f91280679b0 msgr2=0x7f9128067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.020+0000 7f912faff700 1 -- 172.21.15.7:0/2467855782 shutdown_connections 2024-01-27T02:32:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.020+0000 7f912faff700 1 -- 172.21.15.7:0/2467855782 wait complete. 2024-01-27T02:32:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.022+0000 7f912faff700 1 Processor -- start 2024-01-27T02:32:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.022+0000 7f912faff700 1 -- start start 2024-01-27T02:32:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.022+0000 7f912faff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91280a7c10 0x7f9128150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.022+0000 7f912faff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91280a7fe0 0x7f912815c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.022+0000 7f912faff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91280ae880 0x7f9128159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.022+0000 7f912faff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9128151310 con 0x7f91280a7fe0 2024-01-27T02:32:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.022+0000 7f912faff700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912815cff0 con 0x7f91280a7c10 2024-01-27T02:32:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.022+0000 7f912faff700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912815d2d0 con 0x7f91280ae880 2024-01-27T02:32:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.023+0000 7f912d09a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91280a7fe0 0x7f912815c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.023+0000 7f912d09a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91280a7fe0 0x7f912815c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53630/0 (socket says 172.21.15.7:53630) 2024-01-27T02:32:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.023+0000 7f912d09a700 1 -- 172.21.15.7:0/3673294770 learned_addr learned my addr 172.21.15.7:0/3673294770 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.023+0000 7f912e09c700 1 --2- 172.21.15.7:0/3673294770 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91280ae880 0x7f9128159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.023+0000 7f912d89b700 1 --2- 172.21.15.7:0/3673294770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91280a7c10 0x7f9128150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.023+0000 7f912d09a700 1 -- 172.21.15.7:0/3673294770 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91280ae880 msgr2=0x7f9128159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.023+0000 7f912d09a700 1 --2- 172.21.15.7:0/3673294770 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91280ae880 0x7f9128159510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.023+0000 7f912d09a700 1 -- 172.21.15.7:0/3673294770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91280a7c10 msgr2=0x7f9128150dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.023+0000 7f912d09a700 1 --2- 172.21.15.7:0/3673294770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91280a7c10 0x7f9128150dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.023+0000 7f912d09a700 1 -- 172.21.15.7:0/3673294770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9124009420 con 0x7f91280a7fe0 2024-01-27T02:32:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.023+0000 7f912e09c700 1 --2- 172.21.15.7:0/3673294770 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91280ae880 0x7f9128159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.023+0000 7f912d09a700 1 --2- 172.21.15.7:0/3673294770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91280a7fe0 0x7f912815c7a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f911800e900 tx=0x7f91180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.023+0000 7f911dffb700 1 -- 172.21.15.7:0/3673294770 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911800eea0 con 0x7f91280a7fe0 2024-01-27T02:32:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.023+0000 7f911dffb700 1 -- 172.21.15.7:0/3673294770 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911800c850 con 0x7f91280a7fe0 2024-01-27T02:32:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.023+0000 7f912faff700 1 -- 172.21.15.7:0/3673294770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912815d5b0 con 0x7f91280a7fe0 2024-01-27T02:32:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.023+0000 7f911dffb700 1 -- 172.21.15.7:0/3673294770 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911800cc10 con 0x7f91280a7fe0 2024-01-27T02:32:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.024+0000 7f912faff700 1 -- 172.21.15.7:0/3673294770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912815cce0 con 0x7f91280a7fe0 2024-01-27T02:32:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.025+0000 7f912faff700 1 -- 172.21.15.7:0/3673294770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f912804c710 con 0x7f91280a7fe0 2024-01-27T02:32:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.025+0000 7f911dffb700 1 -- 172.21.15.7:0/3673294770 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9118013470 con 0x7f91280a7fe0 2024-01-27T02:32:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.026+0000 7f911dffb700 1 --2- 172.21.15.7:0/3673294770 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f910c066f30 0x7f910c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.026+0000 7f911dffb700 1 -- 172.21.15.7:0/3673294770 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f91180846d0 con 0x7f91280a7fe0 2024-01-27T02:32:17.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.026+0000 7f912d89b700 1 --2- 172.21.15.7:0/3673294770 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f910c066f30 0x7f910c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.026+0000 7f912d89b700 1 --2- 172.21.15.7:0/3673294770 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f910c066f30 0x7f910c0693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f91240093f0 tx=0x7f9124017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.030+0000 7f911dffb700 1 -- 172.21.15.7:0/3673294770 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9118051940 con 0x7f91280a7fe0 2024-01-27T02:32:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.235+0000 7f912faff700 1 -- 172.21.15.7:0/3673294770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91280af020 con 0x7f91280a7fe0 2024-01-27T02:32:17.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.236+0000 7f911dffb700 1 -- 172.21.15.7:0/3673294770 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f9118054f60 con 0x7f91280a7fe0 2024-01-27T02:32:17.237 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:32:07.627822+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:32:03.675611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-27T02:32:05.018885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:32:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.238+0000 7f912faff700 1 -- 172.21.15.7:0/3673294770 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f910c066f30 msgr2=0x7f910c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.238+0000 7f912faff700 1 --2- 172.21.15.7:0/3673294770 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f910c066f30 0x7f910c0693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f91240093f0 tx=0x7f9124017000 comp rx=0 tx=0).stop 2024-01-27T02:32:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.238+0000 7f912faff700 1 -- 172.21.15.7:0/3673294770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91280a7fe0 msgr2=0x7f912815c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.238+0000 7f912faff700 1 --2- 172.21.15.7:0/3673294770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91280a7fe0 0x7f912815c7a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f911800e900 tx=0x7f91180091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.239+0000 7f912faff700 1 -- 172.21.15.7:0/3673294770 shutdown_connections 2024-01-27T02:32:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.239+0000 7f912faff700 1 --2- 172.21.15.7:0/3673294770 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f910c066f30 0x7f910c0693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.239+0000 7f912faff700 1 --2- 172.21.15.7:0/3673294770 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91280ae880 0x7f9128159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.239+0000 7f912faff700 1 --2- 172.21.15.7:0/3673294770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91280a7c10 0x7f9128150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.239+0000 7f912faff700 1 --2- 172.21.15.7:0/3673294770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91280a7fe0 0x7f912815c7a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.239+0000 7f912faff700 1 -- 172.21.15.7:0/3673294770 >> 172.21.15.7:0/3673294770 conn(0x7f91280679b0 msgr2=0x7f91280b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.239+0000 7f912faff700 1 -- 172.21.15.7:0/3673294770 shutdown_connections 2024-01-27T02:32:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.239+0000 7f912faff700 1 -- 172.21.15.7:0/3673294770 wait complete. 2024-01-27T02:32:17.254 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-01-27T02:32:17.254 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-01-27T02:32:17.255 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-01-27T02:32:17.255 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-01-27T02:32:17.256 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-01-27T02:32:17.256 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-01-27T02:32:17.256 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-01-27T02:32:17.257 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-01-27T02:32:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.484+0000 7ffae40cd700 1 Processor -- start 2024-01-27T02:32:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.484+0000 7ffae40cd700 1 -- start start 2024-01-27T02:32:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.485+0000 7ffae40cd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffadc06b680 0x7ffadc0ab980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.485+0000 7ffae40cd700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffadc0af8b0 0x7ffadc069500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.485+0000 7ffae40cd700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffadc06bb00 con 0x7ffadc0cc5c0 2024-01-27T02:32:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.485+0000 7ffae40cd700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffadc0abec0 con 0x7ffadc06b680 2024-01-27T02:32:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.485+0000 7ffae40cd700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffadc0c79f0 con 0x7ffadc0af8b0 2024-01-27T02:32:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.486+0000 7ffae1668700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffadc06b680 0x7ffadc0ab980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.486+0000 7ffae1668700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffadc06b680 0x7ffadc0ab980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53640/0 (socket says 172.21.15.7:53640) 2024-01-27T02:32:17.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.486+0000 7ffae1668700 1 -- 172.21.15.7:0/2582721078 learned_addr learned my addr 172.21.15.7:0/2582721078 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.486+0000 7ffae1668700 1 -- 172.21.15.7:0/2582721078 >> v2:172.21.15.188:3300/0 conn(0x7ffadc0af8b0 msgr2=0x7ffadc069500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.486+0000 7ffae1668700 1 --2- 172.21.15.7:0/2582721078 >> v2:172.21.15.188:3300/0 conn(0x7ffadc0af8b0 0x7ffadc069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.486+0000 7ffae1668700 1 -- 172.21.15.7:0/2582721078 >> v1:172.21.15.7:6789/0 conn(0x7ffadc0cc5c0 legacy=0x7ffadc06e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.486+0000 7ffae1668700 1 -- 172.21.15.7:0/2582721078 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffadc06ecd0 con 0x7ffadc06b680 2024-01-27T02:32:17.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.487+0000 7ffae1668700 1 --2- 172.21.15.7:0/2582721078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffadc06b680 0x7ffadc0ab980 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0099b0 tx=0x7ffacc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686c988f4176f9b1 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.490+0000 7ffae0e67700 1 -- 172.21.15.7:0/2582721078 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc00b7e0 con 0x7ffadc06b680 2024-01-27T02:32:17.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.490+0000 7ffae0e67700 1 -- 172.21.15.7:0/2582721078 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffacc00b940 con 0x7ffadc06b680 2024-01-27T02:32:17.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.490+0000 7ffae0e67700 1 -- 172.21.15.7:0/2582721078 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc00bce0 con 0x7ffadc06b680 2024-01-27T02:32:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.491+0000 7ffae40cd700 1 -- 172.21.15.7:0/2582721078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffadc06b680 msgr2=0x7ffadc0ab980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.491+0000 7ffae40cd700 1 --2- 172.21.15.7:0/2582721078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffadc06b680 0x7ffadc0ab980 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0099b0 tx=0x7ffacc009250 comp rx=0 tx=0).stop 2024-01-27T02:32:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.491+0000 7ffae40cd700 1 -- 172.21.15.7:0/2582721078 shutdown_connections 2024-01-27T02:32:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.491+0000 7ffae40cd700 1 --2- 172.21.15.7:0/2582721078 >> v2:172.21.15.188:3300/0 conn(0x7ffadc0af8b0 0x7ffadc069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.491+0000 7ffae40cd700 1 --2- 172.21.15.7:0/2582721078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffadc06b680 0x7ffadc0ab980 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.492+0000 7ffae40cd700 1 -- 172.21.15.7:0/2582721078 >> 172.21.15.7:0/2582721078 conn(0x7ffadc064090 msgr2=0x7ffadc0cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.492+0000 7ffae40cd700 1 -- 172.21.15.7:0/2582721078 shutdown_connections 2024-01-27T02:32:17.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.492+0000 7ffae40cd700 1 -- 172.21.15.7:0/2582721078 wait complete. 2024-01-27T02:32:17.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.492+0000 7f11376d5700 1 Processor -- start 2024-01-27T02:32:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7ffae40cd700 1 Processor -- start 2024-01-27T02:32:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7ffae40cd700 1 -- start start 2024-01-27T02:32:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.492+0000 7f11376d5700 1 -- start start 2024-01-27T02:32:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7f11376d5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f113006d360 0x7f11300ae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7f11376d5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11300c9aa0 0x7f11300aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7f11376d5700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11300af2e0 con 0x7f11300c96d0 2024-01-27T02:32:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7f11376d5700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1130162150 con 0x7f113006d360 2024-01-27T02:32:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7f11376d5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1130162360 con 0x7f11300c9aa0 2024-01-27T02:32:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7f1134c70700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f113006d360 0x7f11300ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7f1134c70700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f113006d360 0x7f11300ae860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53662/0 (socket says 172.21.15.7:53662) 2024-01-27T02:32:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7f1134c70700 1 -- 172.21.15.7:0/2201141337 learned_addr learned my addr 172.21.15.7:0/2201141337 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.492+0000 7f33a7e23700 1 Processor -- start 2024-01-27T02:32:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.493+0000 7f33a7e23700 1 -- start start 2024-01-27T02:32:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7f33a7e23700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f33a006b680 0x7f33a00af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7f33a7e23700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33a0069500 0x7f33a0069980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7f33a7e23700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a006bb00 con 0x7f33a00ab980 2024-01-27T02:32:17.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7f33a7e23700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a0069ec0 con 0x7f33a006b680 2024-01-27T02:32:17.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7f33a7e23700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a00c79f0 con 0x7f33a0069500 2024-01-27T02:32:17.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7f33a5bbf700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f33a00ab980 0x7f33a006e4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51374/0 (socket says 172.21.15.7:51374) 2024-01-27T02:32:17.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7f33a5bbf700 1 -- 172.21.15.7:0/2523839450 learned_addr learned my addr 172.21.15.7:0/2523839450 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:17.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7f33a63c0700 1 --2- 172.21.15.7:0/2523839450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33a0069500 0x7f33a0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7f1135c72700 1 --2- 172.21.15.7:0/2201141337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11300c9aa0 0x7f11300aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7f1134c70700 1 -- 172.21.15.7:0/2201141337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11300c9aa0 msgr2=0x7f11300aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7f1134c70700 1 --2- 172.21.15.7:0/2201141337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11300c9aa0 0x7f11300aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7f1134c70700 1 -- 172.21.15.7:0/2201141337 >> v1:172.21.15.188:6789/0 conn(0x7f11300c96d0 legacy=0x7f11300cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7f1134c70700 1 -- 172.21.15.7:0/2201141337 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1130162640 con 0x7f113006d360 2024-01-27T02:32:17.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.492+0000 7f1347048700 1 Processor -- start 2024-01-27T02:32:17.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.492+0000 7f1347048700 1 -- start start 2024-01-27T02:32:17.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.493+0000 7f1347048700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1340069130 0x7f1340069570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.493+0000 7f1347048700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f134006e9c0 con 0x7f134015e7b0 2024-01-27T02:32:17.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.493+0000 7f1347048700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1340069ab0 con 0x7f134006b710 2024-01-27T02:32:17.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7f1347048700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f134006bb80 con 0x7f1340069130 2024-01-27T02:32:17.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7f13455e5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1340069130 0x7f1340069570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7f13455e5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1340069130 0x7f1340069570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53656/0 (socket says 172.21.15.7:53656) 2024-01-27T02:32:17.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7f1134c70700 1 --2- 172.21.15.7:0/2201141337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f113006d360 0x7f11300ae860 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f112c0099b0 tx=0x7f112c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96619cd4eb7aba5e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.492+0000 7fa814bae700 1 Processor -- start 2024-01-27T02:32:17.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.492+0000 7fa814bae700 1 -- start start 2024-01-27T02:32:17.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7fa814bae700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa81015e7b0 0x7fa81006b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7fa814bae700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa810069130 0x7fa810069570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7fa814bae700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa81006e9c0 con 0x7fa81006d5d0 2024-01-27T02:32:17.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7fa814bae700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa810069ab0 con 0x7fa81015e7b0 2024-01-27T02:32:17.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7fa814bae700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa81006ddb0 con 0x7fa810069130 2024-01-27T02:32:17.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7fa80ed9d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa810069130 0x7fa810069570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7fa80ed9d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa810069130 0x7fa810069570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:54072/0 (socket says 172.21.15.7:54072) 2024-01-27T02:32:17.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7fa80ed9d700 1 -- 172.21.15.7:0/4017179770 learned_addr learned my addr 172.21.15.7:0/4017179770 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7fa80ed9d700 1 -- 172.21.15.7:0/4017179770 >> v2:172.21.15.7:3300/0 conn(0x7fa81015e7b0 msgr2=0x7fa81006b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7fa80ed9d700 1 --2- 172.21.15.7:0/4017179770 >> v2:172.21.15.7:3300/0 conn(0x7fa81015e7b0 0x7fa81006b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7fa80ed9d700 1 -- 172.21.15.7:0/4017179770 >> v1:172.21.15.188:6789/0 conn(0x7fa81006d5d0 legacy=0x7fa81006ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7fa80ed9d700 1 -- 172.21.15.7:0/4017179770 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81006e090 con 0x7fa810069130 2024-01-27T02:32:17.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7fa80ed9d700 1 --2- 172.21.15.7:0/4017179770 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa810069130 0x7fa810069570 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa8000099b0 tx=0x7fa800009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff3afa74f53ceef server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.494+0000 7f13455e5700 1 -- 172.21.15.7:0/3951026023 learned_addr learned my addr 172.21.15.7:0/3951026023 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7f13455e5700 1 -- 172.21.15.7:0/3951026023 >> v1:172.21.15.188:6789/0 conn(0x7f134006b710 legacy=0x7f134006ee50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-27T02:32:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7f13455e5700 1 -- 172.21.15.7:0/3951026023 >> v1:172.21.15.7:6789/0 conn(0x7f134015e7b0 legacy=0x7f134006d5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-27T02:32:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7f13455e5700 1 -- 172.21.15.7:0/3951026023 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134006ddb0 con 0x7f1340069130 2024-01-27T02:32:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7f13455e5700 1 --2- 172.21.15.7:0/3951026023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1340069130 0x7f1340069570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1338009f20 tx=0x7f1338009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95472c26122c99a6 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7ffae40cd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffadc06b680 0x7ffadc157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7ffae40cd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffadc0af8b0 0x7ffadc0c0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7ffae40cd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffadc0cc5c0 0x7ffadc0691e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7ffae40cd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc0aee60 con 0x7ffadc0cc5c0 2024-01-27T02:32:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7ffae40cd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc0ac150 con 0x7ffadc0af8b0 2024-01-27T02:32:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.495+0000 7ffae40cd700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc0aeb80 con 0x7ffadc06b680 2024-01-27T02:32:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.496+0000 7ffae1668700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffadc0af8b0 0x7ffadc0c0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.496+0000 7ffae1668700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffadc0af8b0 0x7ffadc0c0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:56360/0 (socket says 172.21.15.7:56360) 2024-01-27T02:32:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.496+0000 7ffae1668700 1 -- 172.21.15.7:0/829107171 learned_addr learned my addr 172.21.15.7:0/829107171 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.496+0000 7ffae266a700 1 --2- 172.21.15.7:0/829107171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffadc0cc5c0 0x7ffadc0691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.496+0000 7ffae1e69700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffadc06b680 0x7ffadc157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.496+0000 7f33a53be700 1 --2- 172.21.15.7:0/2523839450 >> v2:172.21.15.7:3300/0 conn(0x7f33a006b680 0x7f33a00af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.496+0000 7f33a63c0700 1 -- 172.21.15.7:0/2523839450 >> v2:172.21.15.7:3300/0 conn(0x7f33a006b680 msgr2=0x7f33a00af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.496+0000 7f33a63c0700 1 --2- 172.21.15.7:0/2523839450 >> v2:172.21.15.7:3300/0 conn(0x7f33a006b680 0x7f33a00af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.496+0000 7f33a63c0700 1 -- 172.21.15.7:0/2523839450 >> v1:172.21.15.7:6789/0 conn(0x7f33a00ab980 legacy=0x7f33a006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.496+0000 7f33a63c0700 1 -- 172.21.15.7:0/2523839450 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a006ecd0 con 0x7f33a0069500 2024-01-27T02:32:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.496+0000 7ffae266a700 1 -- 172.21.15.7:0/829107171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffadc06b680 msgr2=0x7ffadc157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.496+0000 7ffae266a700 1 --2- 172.21.15.7:0/829107171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffadc06b680 0x7ffadc157dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.496+0000 7ffae266a700 1 -- 172.21.15.7:0/829107171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffadc0af8b0 msgr2=0x7ffadc0c0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.496+0000 7ffae266a700 1 --2- 172.21.15.7:0/829107171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffadc0af8b0 0x7ffadc0c0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.496+0000 7ffae266a700 1 -- 172.21.15.7:0/829107171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffacc0094f0 con 0x7ffadc0cc5c0 2024-01-27T02:32:17.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.496+0000 7f33a63c0700 1 --2- 172.21.15.7:0/2523839450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33a0069500 0x7f33a0069980 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3398009c00 tx=0x7f3398009ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f691654b2ce0b961 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.497+0000 7ffae266a700 1 --2- 172.21.15.7:0/829107171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffadc0cc5c0 0x7ffadc0691e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ffad400bad0 tx=0x7ffad4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.497+0000 7ffad1ffb700 1 -- 172.21.15.7:0/829107171 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad400c8c0 con 0x7ffadc0cc5c0 2024-01-27T02:32:17.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.496+0000 7f1127fff700 1 -- 172.21.15.7:0/2201141337 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c00b760 con 0x7f113006d360 2024-01-27T02:32:17.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.496+0000 7f1127fff700 1 -- 172.21.15.7:0/2201141337 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112c00b8c0 con 0x7f113006d360 2024-01-27T02:32:17.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.496+0000 7f1127fff700 1 -- 172.21.15.7:0/2201141337 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c00bc60 con 0x7f113006d360 2024-01-27T02:32:17.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.497+0000 7f133f7fe700 1 -- 172.21.15.7:0/3951026023 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133800e070 con 0x7f1340069130 2024-01-27T02:32:17.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.497+0000 7f133f7fe700 1 -- 172.21.15.7:0/3951026023 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133800ba00 con 0x7f1340069130 2024-01-27T02:32:17.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.497+0000 7f133f7fe700 1 -- 172.21.15.7:0/3951026023 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13380092e0 con 0x7f1340069130 2024-01-27T02:32:17.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.497+0000 7ffad1ffb700 1 -- 172.21.15.7:0/829107171 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad400ca40 con 0x7ffadc0cc5c0 2024-01-27T02:32:17.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.497+0000 7ffad1ffb700 1 -- 172.21.15.7:0/829107171 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad400ce00 con 0x7ffadc0cc5c0 2024-01-27T02:32:17.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.497+0000 7ffae40cd700 1 -- 172.21.15.7:0/829107171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffadc0ac3f0 con 0x7ffadc0cc5c0 2024-01-27T02:32:17.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.497+0000 7f11376d5700 1 -- 172.21.15.7:0/2201141337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f113006d360 msgr2=0x7f11300ae860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.497+0000 7f11376d5700 1 --2- 172.21.15.7:0/2201141337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f113006d360 0x7f11300ae860 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f112c0099b0 tx=0x7f112c009250 comp rx=0 tx=0).stop 2024-01-27T02:32:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.498+0000 7f1347048700 1 -- 172.21.15.7:0/3951026023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1340069130 msgr2=0x7f1340069570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.498+0000 7f1347048700 1 --2- 172.21.15.7:0/3951026023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1340069130 0x7f1340069570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1338009f20 tx=0x7f1338009840 comp rx=0 tx=0).stop 2024-01-27T02:32:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.497+0000 7ffae40cd700 1 -- 172.21.15.7:0/829107171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffadc158300 con 0x7ffadc0cc5c0 2024-01-27T02:32:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.498+0000 7f33a4bbd700 1 -- 172.21.15.7:0/2523839450 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339800dac0 con 0x7f33a0069500 2024-01-27T02:32:17.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.498+0000 7f33a4bbd700 1 -- 172.21.15.7:0/2523839450 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339800dc20 con 0x7f33a0069500 2024-01-27T02:32:17.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.498+0000 7f33a4bbd700 1 -- 172.21.15.7:0/2523839450 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398010430 con 0x7f33a0069500 2024-01-27T02:32:17.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.498+0000 7ffae40cd700 1 -- 172.21.15.7:0/829107171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffadc04c710 con 0x7ffadc0cc5c0 2024-01-27T02:32:17.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.498+0000 7f11376d5700 1 -- 172.21.15.7:0/2201141337 shutdown_connections 2024-01-27T02:32:17.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.499+0000 7f11376d5700 1 --2- 172.21.15.7:0/2201141337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11300c9aa0 0x7f11300aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.499+0000 7f11376d5700 1 --2- 172.21.15.7:0/2201141337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f113006d360 0x7f11300ae860 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.499+0000 7f11376d5700 1 -- 172.21.15.7:0/2201141337 >> 172.21.15.7:0/2201141337 conn(0x7f1130063580 msgr2=0x7f1130063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.499+0000 7f33a7e23700 1 -- 172.21.15.7:0/2523839450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33a0069500 msgr2=0x7f33a0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.499+0000 7f33a7e23700 1 --2- 172.21.15.7:0/2523839450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33a0069500 0x7f33a0069980 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3398009c00 tx=0x7f3398009ce0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.499+0000 7fa80d59a700 1 -- 172.21.15.7:0/4017179770 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80000b820 con 0x7fa810069130 2024-01-27T02:32:17.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.499+0000 7fa80d59a700 1 -- 172.21.15.7:0/4017179770 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80000b980 con 0x7fa810069130 2024-01-27T02:32:17.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.499+0000 7fa80d59a700 1 -- 172.21.15.7:0/4017179770 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80000bd00 con 0x7fa810069130 2024-01-27T02:32:17.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.500+0000 7f11376d5700 1 -- 172.21.15.7:0/2201141337 shutdown_connections 2024-01-27T02:32:17.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.500+0000 7f11376d5700 1 -- 172.21.15.7:0/2201141337 wait complete. 2024-01-27T02:32:17.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.500+0000 7f2b5968e700 1 Processor -- start 2024-01-27T02:32:17.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.500+0000 7f2b5968e700 1 -- start start 2024-01-27T02:32:17.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.500+0000 7ffad1ffb700 1 -- 172.21.15.7:0/829107171 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ffad4014620 con 0x7ffadc0cc5c0 2024-01-27T02:32:17.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.500+0000 7fa814bae700 1 -- 172.21.15.7:0/4017179770 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa810069130 msgr2=0x7fa810069570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.500+0000 7fa814bae700 1 --2- 172.21.15.7:0/4017179770 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa810069130 0x7fa810069570 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa8000099b0 tx=0x7fa800009250 comp rx=0 tx=0).stop 2024-01-27T02:32:17.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.500+0000 7f33a7e23700 1 -- 172.21.15.7:0/2523839450 shutdown_connections 2024-01-27T02:32:17.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.500+0000 7f33a7e23700 1 --2- 172.21.15.7:0/2523839450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33a0069500 0x7f33a0069980 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.500+0000 7f33a7e23700 1 --2- 172.21.15.7:0/2523839450 >> v2:172.21.15.7:3300/0 conn(0x7f33a006b680 0x7f33a00af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7f33a7e23700 1 -- 172.21.15.7:0/2523839450 >> 172.21.15.7:0/2523839450 conn(0x7f33a0064090 msgr2=0x7f33a00cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7ffad1ffb700 1 --2- 172.21.15.7:0/829107171 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffac0066f30 0x7ffac00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7fa814bae700 1 -- 172.21.15.7:0/4017179770 shutdown_connections 2024-01-27T02:32:17.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7fa814bae700 1 --2- 172.21.15.7:0/4017179770 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa810069130 0x7fa810069570 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7fa814bae700 1 --2- 172.21.15.7:0/4017179770 >> v2:172.21.15.7:3300/0 conn(0x7fa81015e7b0 0x7fa81006b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7fa814bae700 1 -- 172.21.15.7:0/4017179770 >> 172.21.15.7:0/4017179770 conn(0x7fa810064090 msgr2=0x7fa8100cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7ffad1ffb700 1 -- 172.21.15.7:0/829107171 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffad4085170 con 0x7ffadc0cc5c0 2024-01-27T02:32:17.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7f1347048700 1 -- 172.21.15.7:0/3951026023 shutdown_connections 2024-01-27T02:32:17.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7f1347048700 1 --2- 172.21.15.7:0/3951026023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1340069130 0x7f1340069570 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7f1347048700 1 -- 172.21.15.7:0/3951026023 >> 172.21.15.7:0/3951026023 conn(0x7f1340064090 msgr2=0x7f13400cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7fa814bae700 1 -- 172.21.15.7:0/4017179770 shutdown_connections 2024-01-27T02:32:17.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7f1347048700 1 -- 172.21.15.7:0/3951026023 shutdown_connections 2024-01-27T02:32:17.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7f11376d5700 1 Processor -- start 2024-01-27T02:32:17.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7f33a7e23700 1 -- 172.21.15.7:0/2523839450 shutdown_connections 2024-01-27T02:32:17.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7f2b5968e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2b5406e4b0 0x7f2b540b0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7f2b5968e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b5406e880 0x7f2b540cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.502+0000 7f2b5968e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2b540cbd20 0x7f2b5406c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.502+0000 7f2b5968e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b540b0ea0 con 0x7f2b540cbd20 2024-01-27T02:32:17.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.502+0000 7f2b5968e700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b540cc550 con 0x7f2b5406e4b0 2024-01-27T02:32:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.502+0000 7f2b5968e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5406b550 con 0x7f2b5406e880 2024-01-27T02:32:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.502+0000 7f2b52ffd700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2b5406e4b0 0x7f2b540b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.502+0000 7f2b52ffd700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2b5406e4b0 0x7f2b540b0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:56362/0 (socket says 172.21.15.7:56362) 2024-01-27T02:32:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.502+0000 7f2b52ffd700 1 -- 172.21.15.7:0/263467093 learned_addr learned my addr 172.21.15.7:0/263467093 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.502+0000 7f2b52ffd700 1 -- 172.21.15.7:0/263467093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b5406e880 msgr2=0x7f2b540cb7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-27T02:32:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.502+0000 7f2b52ffd700 1 --2- 172.21.15.7:0/263467093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b5406e880 0x7f2b540cb7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.502+0000 7f2b52ffd700 1 -- 172.21.15.7:0/263467093 >> v2:172.21.15.7:3300/0 conn(0x7f2b540cbd20 msgr2=0x7f2b5406c030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-27T02:32:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.502+0000 7f2b52ffd700 1 --2- 172.21.15.7:0/263467093 >> v2:172.21.15.7:3300/0 conn(0x7f2b540cbd20 0x7f2b5406c030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.502+0000 7f2b52ffd700 1 -- 172.21.15.7:0/263467093 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5406b830 con 0x7f2b5406e4b0 2024-01-27T02:32:17.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.502+0000 7f2b52ffd700 1 --2- 172.21.15.7:0/263467093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b5406e4b0 0x7f2b540b0960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2b5406dad0 tx=0x7f2b44009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ec90ea1c89c88a7 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7f1347048700 1 -- 172.21.15.7:0/3951026023 wait complete. 2024-01-27T02:32:17.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.502+0000 7f1347048700 1 Processor -- start 2024-01-27T02:32:17.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7f33a7e23700 1 -- 172.21.15.7:0/2523839450 wait complete. 2024-01-27T02:32:17.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7fa814bae700 1 -- 172.21.15.7:0/4017179770 wait complete. 2024-01-27T02:32:17.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.503+0000 7fa814bae700 1 Processor -- start 2024-01-27T02:32:17.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7ffad1ffb700 1 --2- 172.21.15.7:0/829107171 >> [v2:172.21.15.7:6808/39310,v1:172.21.15.7:6809/39310] conn(0x7ffac006f140 0x7ffac0071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7ffad1ffb700 1 -- 172.21.15.7:0/829107171 --> [v2:172.21.15.7:6808/39310,v1:172.21.15.7:6809/39310] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffac0071c00 con 0x7ffac006f140 2024-01-27T02:32:17.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7ffad1ffb700 1 -- 172.21.15.7:0/829107171 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffad4085460 con 0x7ffadc0cc5c0 2024-01-27T02:32:17.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7ffae1e69700 1 --2- 172.21.15.7:0/829107171 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffac0066f30 0x7ffac00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.503+0000 7f33a7e23700 1 Processor -- start 2024-01-27T02:32:17.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.501+0000 7f11376d5700 1 -- start start 2024-01-27T02:32:17.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.503+0000 7f11376d5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f113006d360 0x7f113014c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.503+0000 7f11376d5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11300c96d0 0x7f11300bda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.503+0000 7f11376d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11300c9aa0 0x7f11300bdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.503+0000 7f11376d5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1130152270 con 0x7f11300c9aa0 2024-01-27T02:32:17.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.503+0000 7f11376d5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1130151f90 con 0x7f113006d360 2024-01-27T02:32:17.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.503+0000 7f11376d5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11300b0520 con 0x7f11300c96d0 2024-01-27T02:32:17.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.503+0000 7f33a7e23700 1 -- start start 2024-01-27T02:32:17.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.503+0000 7f1135471700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f113006d360 0x7f113014c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.503+0000 7f1135471700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f113006d360 0x7f113014c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:56366/0 (socket says 172.21.15.7:56366) 2024-01-27T02:32:17.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.503+0000 7f1135471700 1 -- 172.21.15.7:0/1732184761 learned_addr learned my addr 172.21.15.7:0/1732184761 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.503+0000 7f1347048700 1 -- start start 2024-01-27T02:32:17.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f1347048700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1340069130 0x7f1340146ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f1347048700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f134006b710 0x7f1340147420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f1347048700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f134015e7b0 0x7f13401481b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f1347048700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1340149bd0 con 0x7f134006b710 2024-01-27T02:32:17.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f1347048700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1340148f50 con 0x7f134015e7b0 2024-01-27T02:32:17.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f1347048700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1340149230 con 0x7f1340069130 2024-01-27T02:32:17.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f1135c72700 1 --2- 172.21.15.7:0/1732184761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11300c9aa0 0x7f11300bdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f1134c70700 1 --2- 172.21.15.7:0/1732184761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11300c96d0 0x7f11300bda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f133ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f134006b710 0x7f1340147420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f133ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f134006b710 0x7f1340147420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53686/0 (socket says 172.21.15.7:53686) 2024-01-27T02:32:17.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f133ffff700 1 -- 172.21.15.7:0/4198678155 learned_addr learned my addr 172.21.15.7:0/4198678155 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f13455e5700 1 --2- 172.21.15.7:0/4198678155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f134015e7b0 0x7f13401481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.503+0000 7fa814bae700 1 -- start start 2024-01-27T02:32:17.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7fa814bae700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa810069130 0x7fa810148130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7fa814bae700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa81006d5d0 0x7fa810149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7fa814bae700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa81015e7b0 0x7fa81014a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7fa814bae700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8101475c0 con 0x7fa81015e7b0 2024-01-27T02:32:17.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7fa814bae700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa810149670 con 0x7fa810069130 2024-01-27T02:32:17.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7fa814bae700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa810148a60 con 0x7fa81006d5d0 2024-01-27T02:32:17.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f33a7e23700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33a0069500 0x7f33a00ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f33a7e23700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33a006b680 0x7f33a00c0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f33a7e23700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33a00ab980 0x7f33a00aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f33a7e23700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a00ae9c0 con 0x7f33a0069500 2024-01-27T02:32:17.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f33a7e23700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a00addb0 con 0x7f33a006b680 2024-01-27T02:32:17.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f33a7e23700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a00ae090 con 0x7f33a00ab980 2024-01-27T02:32:17.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f1135c72700 1 -- 172.21.15.7:0/1732184761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11300c96d0 msgr2=0x7f11300bda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f1135c72700 1 --2- 172.21.15.7:0/1732184761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11300c96d0 0x7f11300bda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f1135c72700 1 -- 172.21.15.7:0/1732184761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f113006d360 msgr2=0x7f113014c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f1135c72700 1 --2- 172.21.15.7:0/1732184761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f113006d360 0x7f113014c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.504+0000 7f1135c72700 1 -- 172.21.15.7:0/1732184761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112c0094f0 con 0x7f11300c9aa0 2024-01-27T02:32:17.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7fa80dd9b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa81006d5d0 0x7fa810149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7fa80e59c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa810069130 0x7fa810148130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7fa80dd9b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa81006d5d0 0x7fa810149b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:54092/0 (socket says 172.21.15.7:54092) 2024-01-27T02:32:17.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7fa80dd9b700 1 -- 172.21.15.7:0/722701962 learned_addr learned my addr 172.21.15.7:0/722701962 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7f13455e5700 1 -- 172.21.15.7:0/4198678155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1340069130 msgr2=0x7f1340146ee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-27T02:32:17.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7f13455e5700 1 --2- 172.21.15.7:0/4198678155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1340069130 0x7f1340146ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7f13455e5700 1 -- 172.21.15.7:0/4198678155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f134006b710 msgr2=0x7f1340147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7f13455e5700 1 --2- 172.21.15.7:0/4198678155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f134006b710 0x7f1340147420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7f13455e5700 1 -- 172.21.15.7:0/4198678155 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1338009ae0 con 0x7f134015e7b0 2024-01-27T02:32:17.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7ffae1e69700 1 --2- 172.21.15.7:0/829107171 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffac0066f30 0x7ffac00693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ffad8006fd0 tx=0x7ffad8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7f2b51ffb700 1 -- 172.21.15.7:0/263467093 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4400b860 con 0x7f2b5406e4b0 2024-01-27T02:32:17.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7f2b51ffb700 1 -- 172.21.15.7:0/263467093 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4400b9c0 con 0x7f2b5406e4b0 2024-01-27T02:32:17.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7f2b51ffb700 1 -- 172.21.15.7:0/263467093 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4400bd60 con 0x7f2b5406e4b0 2024-01-27T02:32:17.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7ffae1668700 1 --2- 172.21.15.7:0/829107171 >> [v2:172.21.15.7:6808/39310,v1:172.21.15.7:6809/39310] conn(0x7ffac006f140 0x7ffac0071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7f1135c72700 1 --2- 172.21.15.7:0/1732184761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11300c9aa0 0x7f11300bdf60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f112000c650 tx=0x7f1120008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7fb2a055c700 1 Processor -- start 2024-01-27T02:32:17.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7f13455e5700 1 --2- 172.21.15.7:0/4198678155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f134015e7b0 0x7f13401481b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1338014000 tx=0x7f1338017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7f33a53be700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33a006b680 0x7f33a00c0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7f33a63c0700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33a00ab980 0x7f33a00aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7f33a63c0700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33a00ab980 0x7f33a00aee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:54090/0 (socket says 172.21.15.7:54090) 2024-01-27T02:32:17.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7f33a63c0700 1 -- 172.21.15.7:0/3117514081 learned_addr learned my addr 172.21.15.7:0/3117514081 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7fa80dd9b700 1 -- 172.21.15.7:0/722701962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa810069130 msgr2=0x7fa810148130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7fa80dd9b700 1 --2- 172.21.15.7:0/722701962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa810069130 0x7fa810148130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7fa80dd9b700 1 -- 172.21.15.7:0/722701962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa81015e7b0 msgr2=0x7fa81014a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7fa80dd9b700 1 --2- 172.21.15.7:0/722701962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa81015e7b0 0x7fa81014a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7fa80dd9b700 1 -- 172.21.15.7:0/722701962 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8000094f0 con 0x7fa81006d5d0 2024-01-27T02:32:17.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7fa80dd9b700 1 --2- 172.21.15.7:0/722701962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa81006d5d0 0x7fa810149b00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa80400c5d0 tx=0x7fa804007f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.506+0000 7fa7fe7fc700 1 -- 172.21.15.7:0/722701962 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa804011590 con 0x7fa81006d5d0 2024-01-27T02:32:17.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.506+0000 7fa7fe7fc700 1 -- 172.21.15.7:0/722701962 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa804011710 con 0x7fa81006d5d0 2024-01-27T02:32:17.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.506+0000 7fa7fe7fc700 1 -- 172.21.15.7:0/722701962 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa804011ad0 con 0x7fa81006d5d0 2024-01-27T02:32:17.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.506+0000 7fa814bae700 1 -- 172.21.15.7:0/722701962 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa810148d40 con 0x7fa81006d5d0 2024-01-27T02:32:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.506+0000 7fa814bae700 1 -- 172.21.15.7:0/722701962 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa810149300 con 0x7fa81006d5d0 2024-01-27T02:32:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.508+0000 7fa814bae700 1 -- 172.21.15.7:0/722701962 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa7f4000fd0 con 0x7fa81006d5d0 2024-01-27T02:32:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7f133cff9700 1 -- 172.21.15.7:0/4198678155 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13380092e0 con 0x7f134015e7b0 2024-01-27T02:32:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.506+0000 7f133cff9700 1 -- 172.21.15.7:0/4198678155 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1338012df0 con 0x7f134015e7b0 2024-01-27T02:32:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.506+0000 7f133cff9700 1 -- 172.21.15.7:0/4198678155 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133801e660 con 0x7f134015e7b0 2024-01-27T02:32:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7f11257fa700 1 -- 172.21.15.7:0/1732184761 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1120011600 con 0x7f11300c9aa0 2024-01-27T02:32:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7f11257fa700 1 -- 172.21.15.7:0/1732184761 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1120011760 con 0x7f11300c9aa0 2024-01-27T02:32:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7f11257fa700 1 -- 172.21.15.7:0/1732184761 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1120011b00 con 0x7f11300c9aa0 2024-01-27T02:32:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.506+0000 7f11376d5700 1 -- 172.21.15.7:0/1732184761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11300b0800 con 0x7f11300c9aa0 2024-01-27T02:32:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.506+0000 7f11376d5700 1 -- 172.21.15.7:0/1732184761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1130151260 con 0x7f11300c9aa0 2024-01-27T02:32:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7f33a5bbf700 1 --2- 172.21.15.7:0/3117514081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33a0069500 0x7f33a00ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.506+0000 7f33a5bbf700 1 -- 172.21.15.7:0/3117514081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33a00ab980 msgr2=0x7f33a00aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.506+0000 7f33a5bbf700 1 --2- 172.21.15.7:0/3117514081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33a00ab980 0x7f33a00aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.506+0000 7f33a5bbf700 1 -- 172.21.15.7:0/3117514081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33a006b680 msgr2=0x7f33a00c0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.506+0000 7f33a5bbf700 1 --2- 172.21.15.7:0/3117514081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33a006b680 0x7f33a00c0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.506+0000 7f33a5bbf700 1 -- 172.21.15.7:0/3117514081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3398009790 con 0x7f33a0069500 2024-01-27T02:32:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.506+0000 7f33a5bbf700 1 --2- 172.21.15.7:0/3117514081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33a0069500 0x7f33a00ad7b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f339c00c4a0 tx=0x7f339c007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.506+0000 7f3395ffb700 1 -- 172.21.15.7:0/3117514081 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c00ce50 con 0x7f33a0069500 2024-01-27T02:32:17.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.506+0000 7f3395ffb700 1 -- 172.21.15.7:0/3117514081 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339c005b30 con 0x7f33a0069500 2024-01-27T02:32:17.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.506+0000 7f3395ffb700 1 -- 172.21.15.7:0/3117514081 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c0070d0 con 0x7f33a0069500 2024-01-27T02:32:17.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.506+0000 7f2b5968e700 1 -- 172.21.15.7:0/263467093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b5406e4b0 msgr2=0x7f2b540b0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.506+0000 7f2b5968e700 1 --2- 172.21.15.7:0/263467093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b5406e4b0 0x7f2b540b0960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2b5406dad0 tx=0x7f2b44009250 comp rx=0 tx=0).stop 2024-01-27T02:32:17.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.507+0000 7f2b5968e700 1 -- 172.21.15.7:0/263467093 shutdown_connections 2024-01-27T02:32:17.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.507+0000 7f2b5968e700 1 --2- 172.21.15.7:0/263467093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b5406e4b0 0x7f2b540b0960 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.507+0000 7f2b5968e700 1 --2- 172.21.15.7:0/263467093 >> v2:172.21.15.7:3300/0 conn(0x7f2b540cbd20 0x7f2b5406c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.507+0000 7f2b5968e700 1 --2- 172.21.15.7:0/263467093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b5406e880 0x7f2b540cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.507+0000 7f2b5968e700 1 -- 172.21.15.7:0/263467093 >> 172.21.15.7:0/263467093 conn(0x7f2b540ca610 msgr2=0x7f2b540caa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.507+0000 7f2b5968e700 1 -- 172.21.15.7:0/263467093 shutdown_connections 2024-01-27T02:32:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.507+0000 7f2b5968e700 1 -- 172.21.15.7:0/263467093 wait complete. 2024-01-27T02:32:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.508+0000 7f11376d5700 1 -- 172.21.15.7:0/1732184761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f110c000fd0 con 0x7f11300c9aa0 2024-01-27T02:32:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.505+0000 7fb2a055c700 1 -- start start 2024-01-27T02:32:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.507+0000 7fb2a055c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2980c96d0 0x7fb2980cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.507+0000 7fb2a055c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb298162360 con 0x7fb2980c9aa0 2024-01-27T02:32:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.507+0000 7fb2a055c700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb298162640 con 0x7fb29806d360 2024-01-27T02:32:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.507+0000 7fb2a055c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb298162150 con 0x7fb2980c96d0 2024-01-27T02:32:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.507+0000 7fb29daf7700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fb29806d360 0x7fb2980ae860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:45398/0 (socket says 172.21.15.7:45398) 2024-01-27T02:32:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.507+0000 7fb29daf7700 1 -- 172.21.15.7:0/1770321874 learned_addr learned my addr 172.21.15.7:0/1770321874 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.507+0000 7fb29e2f8700 1 --2- 172.21.15.7:0/1770321874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2980c96d0 0x7fb2980cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.507+0000 7fb29d2f6700 1 -- 172.21.15.7:0/1770321874 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3639613624 0 0) 0x7fb298162640 con 0x7fb29806d360 2024-01-27T02:32:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.507+0000 7fb29d2f6700 1 -- 172.21.15.7:0/1770321874 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb284003570 con 0x7fb29806d360 2024-01-27T02:32:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.508+0000 7fb29e2f8700 1 -- 172.21.15.7:0/1770321874 >> v1:172.21.15.188:6789/0 conn(0x7fb29806d360 legacy=0x7fb2980ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.508+0000 7fb29e2f8700 1 -- 172.21.15.7:0/1770321874 >> v1:172.21.15.7:6789/0 conn(0x7fb2980c9aa0 legacy=0x7fb2980ca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.508+0000 7fb29d2f6700 1 -- 172.21.15.7:0/1770321874 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3262400146 0 0) 0x7fb284003570 con 0x7fb29806d360 2024-01-27T02:32:17.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.508+0000 7fb29e2f8700 1 -- 172.21.15.7:0/1770321874 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb298162920 con 0x7fb2980c96d0 2024-01-27T02:32:17.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.508+0000 7ffae1668700 1 --2- 172.21.15.7:0/829107171 >> [v2:172.21.15.7:6808/39310,v1:172.21.15.7:6809/39310] conn(0x7ffac006f140 0x7ffac0071550 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.508+0000 7fb29e2f8700 1 --2- 172.21.15.7:0/1770321874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2980c96d0 0x7fb2980cc080 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009890 tx=0x7fb28c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=523092669cf5de7 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.508+0000 7fb29d2f6700 1 -- 172.21.15.7:0/1770321874 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294001470 con 0x7fb2980c96d0 2024-01-27T02:32:17.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.508+0000 7fb29d2f6700 1 -- 172.21.15.7:0/1770321874 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28c011420 con 0x7fb2980c96d0 2024-01-27T02:32:17.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.508+0000 7fb29d2f6700 1 -- 172.21.15.7:0/1770321874 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c0117c0 con 0x7fb2980c96d0 2024-01-27T02:32:17.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.509+0000 7f1347048700 1 -- 172.21.15.7:0/4198678155 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1340149510 con 0x7f134015e7b0 2024-01-27T02:32:17.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.509+0000 7f1347048700 1 -- 172.21.15.7:0/4198678155 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13401435a0 con 0x7f134015e7b0 2024-01-27T02:32:17.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.509+0000 7ffad1ffb700 1 -- 172.21.15.7:0/829107171 <== osd.3 v2:172.21.15.7:6808/39310 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7ffac0071c00 con 0x7ffac006f140 2024-01-27T02:32:17.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.509+0000 7f33a7e23700 1 -- 172.21.15.7:0/3117514081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a00ae370 con 0x7f33a0069500 2024-01-27T02:32:17.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.509+0000 7f33a7e23700 1 -- 172.21.15.7:0/3117514081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a00c1020 con 0x7f33a0069500 2024-01-27T02:32:17.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.509+0000 7fa7fe7fc700 1 -- 172.21.15.7:0/722701962 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa80401c460 con 0x7fa81006d5d0 2024-01-27T02:32:17.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.510+0000 7fa7fe7fc700 1 --2- 172.21.15.7:0/722701962 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa7f0067080 0x7fa7f0069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.510+0000 7fa7fe7fc700 1 -- 172.21.15.7:0/722701962 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa804084d10 con 0x7fa81006d5d0 2024-01-27T02:32:17.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.510+0000 7fa80e59c700 1 --2- 172.21.15.7:0/722701962 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa7f0067080 0x7fa7f0069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.510+0000 7fa7fe7fc700 1 --2- 172.21.15.7:0/722701962 >> [v2:172.21.15.7:6816/39319,v1:172.21.15.7:6817/39319] conn(0x7fa7f006f290 0x7fa7f00716a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.510+0000 7fa7fe7fc700 1 -- 172.21.15.7:0/722701962 --> [v2:172.21.15.7:6816/39319,v1:172.21.15.7:6817/39319] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa7f0071d50 con 0x7fa7f006f290 2024-01-27T02:32:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.510+0000 7fa7fe7fc700 1 -- 172.21.15.7:0/722701962 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa804085040 con 0x7fa81006d5d0 2024-01-27T02:32:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.511+0000 7fa80e59c700 1 --2- 172.21.15.7:0/722701962 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa7f0067080 0x7fa7f0069530 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa808007580 tx=0x7fa808009780 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.511+0000 7fa80ed9d700 1 --2- 172.21.15.7:0/722701962 >> [v2:172.21.15.7:6816/39319,v1:172.21.15.7:6817/39319] conn(0x7fa7f006f290 0x7fa7f00716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.511+0000 7fa80ed9d700 1 --2- 172.21.15.7:0/722701962 >> [v2:172.21.15.7:6816/39319,v1:172.21.15.7:6817/39319] conn(0x7fa7f006f290 0x7fa7f00716a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.509+0000 7fb2a055c700 1 -- 172.21.15.7:0/1770321874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2980c96d0 msgr2=0x7fb2980cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.509+0000 7fb2a055c700 1 --2- 172.21.15.7:0/1770321874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2980c96d0 0x7fb2980cc080 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009890 tx=0x7fb28c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.510+0000 7fb2a055c700 1 -- 172.21.15.7:0/1770321874 shutdown_connections 2024-01-27T02:32:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.510+0000 7fb2a055c700 1 --2- 172.21.15.7:0/1770321874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2980c96d0 0x7fb2980cc080 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009890 tx=0x7fb28c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.510+0000 7fb2a055c700 1 -- 172.21.15.7:0/1770321874 >> 172.21.15.7:0/1770321874 conn(0x7fb298063580 msgr2=0x7fb298063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.510+0000 7fb2a055c700 1 -- 172.21.15.7:0/1770321874 shutdown_connections 2024-01-27T02:32:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.511+0000 7fb2a055c700 1 -- 172.21.15.7:0/1770321874 wait complete. 2024-01-27T02:32:17.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.513+0000 7fb2a055c700 1 Processor -- start 2024-01-27T02:32:17.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.513+0000 7fb2a055c700 1 -- start start 2024-01-27T02:32:17.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.511+0000 7f1347048700 1 -- 172.21.15.7:0/4198678155 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f134004c710 con 0x7f134015e7b0 2024-01-27T02:32:17.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.511+0000 7f133cff9700 1 -- 172.21.15.7:0/4198678155 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f133801ea90 con 0x7f134015e7b0 2024-01-27T02:32:17.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.511+0000 7f133cff9700 1 --2- 172.21.15.7:0/4198678155 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f132c066f30 0x7f132c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.512+0000 7f133cff9700 1 -- 172.21.15.7:0/4198678155 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1338086270 con 0x7f134015e7b0 2024-01-27T02:32:17.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.512+0000 7f133cff9700 1 --2- 172.21.15.7:0/4198678155 >> [v2:172.21.15.7:6824/39325,v1:172.21.15.7:6825/39325] conn(0x7f132c06f0c0 0x7f132c0714d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.512+0000 7f133cff9700 1 -- 172.21.15.7:0/4198678155 --> [v2:172.21.15.7:6824/39325,v1:172.21.15.7:6825/39325] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f132c071b80 con 0x7f132c06f0c0 2024-01-27T02:32:17.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.512+0000 7f1344de4700 1 --2- 172.21.15.7:0/4198678155 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f132c066f30 0x7f132c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.512+0000 7f133cff9700 1 -- 172.21.15.7:0/4198678155 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1338053440 con 0x7f134015e7b0 2024-01-27T02:32:17.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.510+0000 7f11257fa700 1 -- 172.21.15.7:0/1732184761 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f11200223e0 con 0x7f11300c9aa0 2024-01-27T02:32:17.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.511+0000 7f11257fa700 1 --2- 172.21.15.7:0/1732184761 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1114067050 0x7f1114069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.511+0000 7f11257fa700 1 -- 172.21.15.7:0/1732184761 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f11200845f0 con 0x7f11300c9aa0 2024-01-27T02:32:17.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.511+0000 7f11257fa700 1 --2- 172.21.15.7:0/1732184761 >> [v2:172.21.15.188:6801/39374,v1:172.21.15.188:6803/39374] conn(0x7f111406f260 0x7f1114071670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.511+0000 7f11257fa700 1 -- 172.21.15.7:0/1732184761 --> [v2:172.21.15.188:6801/39374,v1:172.21.15.188:6803/39374] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1114071d20 con 0x7f111406f260 2024-01-27T02:32:17.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.511+0000 7f11257fa700 1 -- 172.21.15.7:0/1732184761 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1120084920 con 0x7f11300c9aa0 2024-01-27T02:32:17.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.513+0000 7f1134c70700 1 --2- 172.21.15.7:0/1732184761 >> [v2:172.21.15.188:6801/39374,v1:172.21.15.188:6803/39374] conn(0x7f111406f260 0x7f1114071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.513+0000 7f1135471700 1 --2- 172.21.15.7:0/1732184761 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1114067050 0x7f1114069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.510+0000 7f33a7e23700 1 -- 172.21.15.7:0/3117514081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f33a004c710 con 0x7f33a0069500 2024-01-27T02:32:17.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.509+0000 7f2b5968e700 1 Processor -- start 2024-01-27T02:32:17.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.511+0000 7f2b5968e700 1 -- start start 2024-01-27T02:32:17.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.512+0000 7f2b5968e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b5406e4b0 0x7f2b5416e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.512+0000 7f2b5968e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b5406e880 0x7f2b540af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.512+0000 7f2b5968e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b540cbd20 0x7f2b541562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.512+0000 7f2b5968e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5416edb0 con 0x7f2b540cbd20 2024-01-27T02:32:17.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.512+0000 7f2b5968e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b54158450 con 0x7f2b5406e4b0 2024-01-27T02:32:17.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.512+0000 7f2b5968e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b540afc00 con 0x7f2b5406e880 2024-01-27T02:32:17.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.512+0000 7f2b527fc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b5406e880 0x7f2b540af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.512+0000 7f2b527fc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b5406e880 0x7f2b540af6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:54112/0 (socket says 172.21.15.7:54112) 2024-01-27T02:32:17.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.512+0000 7f2b527fc700 1 -- 172.21.15.7:0/1967153375 learned_addr learned my addr 172.21.15.7:0/1967153375 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.513+0000 7f2b52ffd700 1 --2- 172.21.15.7:0/1967153375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b5406e4b0 0x7f2b5416e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.513+0000 7f2b537fe700 1 --2- 172.21.15.7:0/1967153375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b540cbd20 0x7f2b541562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.513+0000 7f2b527fc700 1 -- 172.21.15.7:0/1967153375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b5406e4b0 msgr2=0x7f2b5416e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.513+0000 7f2b527fc700 1 --2- 172.21.15.7:0/1967153375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b5406e4b0 0x7f2b5416e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.513+0000 7f2b527fc700 1 -- 172.21.15.7:0/1967153375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b540cbd20 msgr2=0x7f2b541562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.513+0000 7f2b527fc700 1 --2- 172.21.15.7:0/1967153375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b540cbd20 0x7f2b541562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.513+0000 7f2b527fc700 1 -- 172.21.15.7:0/1967153375 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b440094f0 con 0x7f2b5406e880 2024-01-27T02:32:17.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.512+0000 7f3395ffb700 1 -- 172.21.15.7:0/3117514081 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f339c0074c0 con 0x7f33a0069500 2024-01-27T02:32:17.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.513+0000 7f3395ffb700 1 --2- 172.21.15.7:0/3117514081 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3384067080 0x7f3384069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.513+0000 7f3395ffb700 1 -- 172.21.15.7:0/3117514081 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f339c083f50 con 0x7f33a0069500 2024-01-27T02:32:17.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.513+0000 7f3395ffb700 1 --2- 172.21.15.7:0/3117514081 >> [v2:172.21.15.188:6800/39373,v1:172.21.15.188:6802/39373] conn(0x7f338406f210 0x7f3384071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.513+0000 7f3395ffb700 1 -- 172.21.15.7:0/3117514081 --> [v2:172.21.15.188:6800/39373,v1:172.21.15.188:6802/39373] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3384071cd0 con 0x7f338406f210 2024-01-27T02:32:17.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.513+0000 7f2b527fc700 1 --2- 172.21.15.7:0/1967153375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b5406e880 0x7f2b540af6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800b630 tx=0x7f2b480091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.513+0000 7f1134c70700 1 --2- 172.21.15.7:0/1732184761 >> [v2:172.21.15.188:6801/39374,v1:172.21.15.188:6803/39374] conn(0x7f111406f260 0x7f1114071670 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.513+0000 7f133ffff700 1 --2- 172.21.15.7:0/4198678155 >> [v2:172.21.15.7:6824/39325,v1:172.21.15.7:6825/39325] conn(0x7f132c06f0c0 0x7f132c0714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.514+0000 7f1135471700 1 --2- 172.21.15.7:0/1732184761 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1114067050 0x7f1114069500 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1128007fd0 tx=0x7f11280018a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.513+0000 7f33a53be700 1 --2- 172.21.15.7:0/3117514081 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3384067080 0x7f3384069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.514+0000 7f33a63c0700 1 --2- 172.21.15.7:0/3117514081 >> [v2:172.21.15.188:6800/39373,v1:172.21.15.188:6802/39373] conn(0x7f338406f210 0x7f3384071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.513+0000 7f2b3affd700 1 -- 172.21.15.7:0/1967153375 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4800f510 con 0x7f2b5406e880 2024-01-27T02:32:17.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.513+0000 7f2b3affd700 1 -- 172.21.15.7:0/1967153375 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4800f690 con 0x7f2b5406e880 2024-01-27T02:32:17.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.513+0000 7f2b3affd700 1 -- 172.21.15.7:0/1967153375 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4800fa50 con 0x7f2b5406e880 2024-01-27T02:32:17.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.514+0000 7f2b5968e700 1 -- 172.21.15.7:0/1967153375 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b54157560 con 0x7f2b5406e880 2024-01-27T02:32:17.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.514+0000 7f11257fa700 1 -- 172.21.15.7:0/1732184761 <== osd.4 v2:172.21.15.188:6801/39374 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f1114071d20 con 0x7f111406f260 2024-01-27T02:32:17.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.514+0000 7f33a63c0700 1 --2- 172.21.15.7:0/3117514081 >> [v2:172.21.15.188:6800/39373,v1:172.21.15.188:6802/39373] conn(0x7f338406f210 0x7f3384071620 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.514+0000 7f133ffff700 1 --2- 172.21.15.7:0/4198678155 >> [v2:172.21.15.7:6824/39325,v1:172.21.15.7:6825/39325] conn(0x7f132c06f0c0 0x7f132c0714d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.514+0000 7f1344de4700 1 --2- 172.21.15.7:0/4198678155 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f132c066f30 0x7f132c0693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1334009740 tx=0x7f1334006c60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.514+0000 7f2b5968e700 1 -- 172.21.15.7:0/1967153375 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b54157b20 con 0x7f2b5406e880 2024-01-27T02:32:17.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.515+0000 7fa7fe7fc700 1 -- 172.21.15.7:0/722701962 <== osd.0 v2:172.21.15.7:6816/39319 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7fa7f0071d50 con 0x7fa7f006f290 2024-01-27T02:32:17.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.514+0000 7fb2a055c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb29806d360 0x7fb298166740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.514+0000 7fb2a055c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2980c9aa0 0x7fb29814d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.514+0000 7fb2a055c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb298166c80 0x7fb29814df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.514+0000 7fb2a055c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2980c2010 con 0x7fb298166c80 2024-01-27T02:32:17.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.514+0000 7fb2a055c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb298167190 con 0x7fb29806d360 2024-01-27T02:32:17.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.514+0000 7fb2a055c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb298167470 con 0x7fb2980c9aa0 2024-01-27T02:32:17.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.515+0000 7fb29daf7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2980c9aa0 0x7fb29814d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.515+0000 7fb29daf7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2980c9aa0 0x7fb29814d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:54124/0 (socket says 172.21.15.7:54124) 2024-01-27T02:32:17.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.515+0000 7fb29daf7700 1 -- 172.21.15.7:0/3706275572 learned_addr learned my addr 172.21.15.7:0/3706275572 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.515+0000 7fb29eaf9700 1 --2- 172.21.15.7:0/3706275572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb298166c80 0x7fb29814df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.515+0000 7fb29eaf9700 1 -- 172.21.15.7:0/3706275572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2980c9aa0 msgr2=0x7fb29814d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.515+0000 7fb29eaf9700 1 --2- 172.21.15.7:0/3706275572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2980c9aa0 0x7fb29814d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.515+0000 7fb29eaf9700 1 -- 172.21.15.7:0/3706275572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb29806d360 msgr2=0x7fb298166740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.515+0000 7fb29eaf9700 1 --2- 172.21.15.7:0/3706275572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb29806d360 0x7fb298166740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.515+0000 7fb29eaf9700 1 -- 172.21.15.7:0/3706275572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28c009420 con 0x7fb298166c80 2024-01-27T02:32:17.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.515+0000 7f133cff9700 1 -- 172.21.15.7:0/4198678155 <== osd.2 v2:172.21.15.7:6824/39325 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f132c071b80 con 0x7f132c06f0c0 2024-01-27T02:32:17.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.515+0000 7f33a53be700 1 --2- 172.21.15.7:0/3117514081 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3384067080 0x7f3384069530 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f33a006a2e0 tx=0x7f3390008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.515+0000 7f724f42c700 1 Processor -- start 2024-01-27T02:32:17.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.515+0000 7f724f42c700 1 -- start start 2024-01-27T02:32:17.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.516+0000 7f724f42c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f724815e7b0 0x7f724806b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.516+0000 7f724f42c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f724806d5d0 0x7f724806ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.516+0000 7f724f42c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f724806e9c0 con 0x7f7248069130 2024-01-27T02:32:17.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.516+0000 7f724f42c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f724806ddb0 con 0x7f724806d5d0 2024-01-27T02:32:17.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.516+0000 7f724f42c700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f724806e090 con 0x7f724815e7b0 2024-01-27T02:32:17.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.516+0000 7f724d1c8700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f724815e7b0 0x7f724806b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.516+0000 7f724d1c8700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f724815e7b0 0x7f724806b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:56430/0 (socket says 172.21.15.7:56430) 2024-01-27T02:32:17.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.516+0000 7f724d1c8700 1 -- 172.21.15.7:0/658184594 learned_addr learned my addr 172.21.15.7:0/658184594 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.517+0000 7f724d1c8700 1 -- 172.21.15.7:0/658184594 >> v2:172.21.15.7:3300/0 conn(0x7f724806d5d0 msgr2=0x7f724806ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.517+0000 7f724d1c8700 1 --2- 172.21.15.7:0/658184594 >> v2:172.21.15.7:3300/0 conn(0x7f724806d5d0 0x7f724806ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.517+0000 7f724d1c8700 1 -- 172.21.15.7:0/658184594 >> v1:172.21.15.7:6789/0 conn(0x7f7248069130 legacy=0x7f72480af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.517+0000 7f724d1c8700 1 -- 172.21.15.7:0/658184594 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724806e370 con 0x7f724815e7b0 2024-01-27T02:32:17.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.516+0000 7f3395ffb700 1 -- 172.21.15.7:0/3117514081 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f339c0511a0 con 0x7f33a0069500 2024-01-27T02:32:17.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.516+0000 7f3395ffb700 1 -- 172.21.15.7:0/3117514081 <== osd.5 v2:172.21.15.188:6800/39373 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f3384071cd0 con 0x7f338406f210 2024-01-27T02:32:17.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.519+0000 7f2b5968e700 1 -- 172.21.15.7:0/1967153375 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2b5404c710 con 0x7f2b5406e880 2024-01-27T02:32:17.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.520+0000 7f2b3affd700 1 -- 172.21.15.7:0/1967153375 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2b4801a400 con 0x7f2b5406e880 2024-01-27T02:32:17.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.520+0000 7f2b3affd700 1 --2- 172.21.15.7:0/1967153375 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2b40066f30 0x7f2b400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.520+0000 7f2b3affd700 1 -- 172.21.15.7:0/1967153375 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2b4808ab20 con 0x7f2b5406e880 2024-01-27T02:32:17.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.520+0000 7f2b3affd700 1 --2- 172.21.15.7:0/1967153375 >> [v2:172.21.15.188:6816/39375,v1:172.21.15.188:6817/39375] conn(0x7f2b4006f140 0x7f2b40071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.520+0000 7f2b3affd700 1 -- 172.21.15.7:0/1967153375 --> [v2:172.21.15.188:6816/39375,v1:172.21.15.188:6817/39375] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b40071c00 con 0x7f2b4006f140 2024-01-27T02:32:17.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.520+0000 7f2b3affd700 1 -- 172.21.15.7:0/1967153375 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b4808ae20 con 0x7f2b5406e880 2024-01-27T02:32:17.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.521+0000 7fb29eaf9700 1 --2- 172.21.15.7:0/3706275572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb298166c80 0x7fb29814df10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb29000bb00 tx=0x7fb290009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.521+0000 7f2b52ffd700 1 --2- 172.21.15.7:0/1967153375 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2b40066f30 0x7f2b400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.521+0000 7fb28a7fc700 1 -- 172.21.15.7:0/3706275572 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29000c840 con 0x7fb298166c80 2024-01-27T02:32:17.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.521+0000 7fb28a7fc700 1 -- 172.21.15.7:0/3706275572 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29000c9c0 con 0x7fb298166c80 2024-01-27T02:32:17.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.521+0000 7fb28a7fc700 1 -- 172.21.15.7:0/3706275572 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29000cd80 con 0x7fb298166c80 2024-01-27T02:32:17.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.522+0000 7f2b52ffd700 1 --2- 172.21.15.7:0/1967153375 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2b40066f30 0x7f2b400693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2b5406dad0 tx=0x7f2b44017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.522+0000 7fb2a055c700 1 -- 172.21.15.7:0/3706275572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb298167750 con 0x7fb298166c80 2024-01-27T02:32:17.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.522+0000 7fb2a055c700 1 -- 172.21.15.7:0/3706275572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2980c3be0 con 0x7fb298166c80 2024-01-27T02:32:17.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.525+0000 7fb28a7fc700 1 -- 172.21.15.7:0/3706275572 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb290014620 con 0x7fb298166c80 2024-01-27T02:32:17.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.525+0000 7fb2a055c700 1 -- 172.21.15.7:0/3706275572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb29804c710 con 0x7fb298166c80 2024-01-27T02:32:17.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.526+0000 7fb28a7fc700 1 --2- 172.21.15.7:0/3706275572 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb27c067080 0x7fb27c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.526+0000 7fb28a7fc700 1 -- 172.21.15.7:0/3706275572 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb2900843f0 con 0x7fb298166c80 2024-01-27T02:32:17.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.526+0000 7fb28a7fc700 1 --2- 172.21.15.7:0/3706275572 >> [v2:172.21.15.7:6800/39309,v1:172.21.15.7:6801/39309] conn(0x7fb27c06f210 0x7fb27c071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.526+0000 7fb28a7fc700 1 -- 172.21.15.7:0/3706275572 --> [v2:172.21.15.7:6800/39309,v1:172.21.15.7:6801/39309] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb27c071cd0 con 0x7fb27c06f210 2024-01-27T02:32:17.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.526+0000 7fb29e2f8700 1 --2- 172.21.15.7:0/3706275572 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb27c067080 0x7fb27c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.526+0000 7fb29daf7700 1 --2- 172.21.15.7:0/3706275572 >> [v2:172.21.15.7:6800/39309,v1:172.21.15.7:6801/39309] conn(0x7fb27c06f210 0x7fb27c071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.527+0000 7fb29daf7700 1 --2- 172.21.15.7:0/3706275572 >> [v2:172.21.15.7:6800/39309,v1:172.21.15.7:6801/39309] conn(0x7fb27c06f210 0x7fb27c071620 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.527+0000 7fb29e2f8700 1 --2- 172.21.15.7:0/3706275572 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb27c067080 0x7fb27c069530 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb29806c2c0 tx=0x7fb28c016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.529+0000 7fb28a7fc700 1 -- 172.21.15.7:0/3706275572 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2900098c0 con 0x7fb298166c80 2024-01-27T02:32:17.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.529+0000 7fb28a7fc700 1 -- 172.21.15.7:0/3706275572 <== osd.1 v2:172.21.15.7:6800/39309 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7fb27c071cd0 con 0x7fb27c06f210 2024-01-27T02:32:17.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.522+0000 7f724d1c8700 1 --2- 172.21.15.7:0/658184594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f724815e7b0 0x7f724806b710 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f72480cd660 tx=0x7f7240009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1be9496471fe02d5 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.522+0000 7f723ffff700 1 -- 172.21.15.7:0/658184594 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724000e070 con 0x7f724815e7b0 2024-01-27T02:32:17.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.522+0000 7f723ffff700 1 -- 172.21.15.7:0/658184594 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724000ba10 con 0x7f724815e7b0 2024-01-27T02:32:17.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.522+0000 7f723ffff700 1 -- 172.21.15.7:0/658184594 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72400092e0 con 0x7f724815e7b0 2024-01-27T02:32:17.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.524+0000 7f724f42c700 1 -- 172.21.15.7:0/658184594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f724815e7b0 msgr2=0x7f724806b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.524+0000 7f724f42c700 1 --2- 172.21.15.7:0/658184594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f724815e7b0 0x7f724806b710 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f72480cd660 tx=0x7f7240009840 comp rx=0 tx=0).stop 2024-01-27T02:32:17.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.524+0000 7f724f42c700 1 -- 172.21.15.7:0/658184594 shutdown_connections 2024-01-27T02:32:17.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.524+0000 7f724f42c700 1 --2- 172.21.15.7:0/658184594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f724815e7b0 0x7f724806b710 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.524+0000 7f724f42c700 1 --2- 172.21.15.7:0/658184594 >> v2:172.21.15.7:3300/0 conn(0x7f724806d5d0 0x7f724806ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.524+0000 7f724f42c700 1 -- 172.21.15.7:0/658184594 >> 172.21.15.7:0/658184594 conn(0x7f7248064090 msgr2=0x7f72480cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.525+0000 7f724f42c700 1 -- 172.21.15.7:0/658184594 shutdown_connections 2024-01-27T02:32:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.525+0000 7f724f42c700 1 -- 172.21.15.7:0/658184594 wait complete. 2024-01-27T02:32:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.527+0000 7f724f42c700 1 Processor -- start 2024-01-27T02:32:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.528+0000 7f724f42c700 1 -- start start 2024-01-27T02:32:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.529+0000 7f724f42c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7248069130 0x7f724815cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.529+0000 7f724f42c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f724806d5d0 0x7f724815d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.529+0000 7f724f42c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f724815e7b0 0x7f7248087700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.529+0000 7f724f42c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72480c1b60 con 0x7f724815e7b0 2024-01-27T02:32:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.529+0000 7f724f42c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72480c1230 con 0x7f724806d5d0 2024-01-27T02:32:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.529+0000 7f724f42c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72480c1510 con 0x7f7248069130 2024-01-27T02:32:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.529+0000 7f724d9c9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f724815e7b0 0x7f7248087700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.529+0000 7f724d9c9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f724815e7b0 0x7f7248087700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53748/0 (socket says 172.21.15.7:53748) 2024-01-27T02:32:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.529+0000 7f724d9c9700 1 -- 172.21.15.7:0/1507461566 learned_addr learned my addr 172.21.15.7:0/1507461566 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.531+0000 7f2b537fe700 1 --2- 172.21.15.7:0/1967153375 >> [v2:172.21.15.188:6816/39375,v1:172.21.15.188:6817/39375] conn(0x7f2b4006f140 0x7f2b40071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.530+0000 7f724d9c9700 1 -- 172.21.15.7:0/1507461566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7248069130 msgr2=0x7f724815cb20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-27T02:32:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.530+0000 7f724d9c9700 1 --2- 172.21.15.7:0/1507461566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7248069130 0x7f724815cb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.530+0000 7f724d9c9700 1 -- 172.21.15.7:0/1507461566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f724806d5d0 msgr2=0x7f724815d060 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-27T02:32:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.530+0000 7f724d9c9700 1 --2- 172.21.15.7:0/1507461566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f724806d5d0 0x7f724815d060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.530+0000 7f724d9c9700 1 -- 172.21.15.7:0/1507461566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7240009ae0 con 0x7f724815e7b0 2024-01-27T02:32:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.530+0000 7f724d9c9700 1 --2- 172.21.15.7:0/1507461566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f724815e7b0 0x7f7248087700 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f723800b5e0 tx=0x7f7238009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.531+0000 7f2b537fe700 1 --2- 172.21.15.7:0/1967153375 >> [v2:172.21.15.188:6816/39375,v1:172.21.15.188:6817/39375] conn(0x7f2b4006f140 0x7f2b40071550 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.532+0000 7f2b3affd700 1 -- 172.21.15.7:0/1967153375 <== osd.7 v2:172.21.15.188:6816/39375 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f2b40071c00 con 0x7f2b4006f140 2024-01-27T02:32:17.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.535+0000 7ffae40cd700 1 -- 172.21.15.7:0/829107171 --> [v2:172.21.15.7:6808/39310,v1:172.21.15.7:6809/39310] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffadc0c70c0 con 0x7ffac006f140 2024-01-27T02:32:17.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.538+0000 7f11376d5700 1 -- 172.21.15.7:0/1732184761 --> [v2:172.21.15.188:6801/39374,v1:172.21.15.188:6803/39374] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f110c002d30 con 0x7f111406f260 2024-01-27T02:32:17.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.538+0000 7ffad1ffb700 1 -- 172.21.15.7:0/829107171 <== osd.3 v2:172.21.15.7:6808/39310 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffadc0c70c0 con 0x7ffac006f140 2024-01-27T02:32:17.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.539+0000 7f11257fa700 1 -- 172.21.15.7:0/1732184761 <== osd.4 v2:172.21.15.188:6801/39374 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f110c002d30 con 0x7f111406f260 2024-01-27T02:32:17.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.539+0000 7f723d7fa700 1 -- 172.21.15.7:0/1507461566 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723800f510 con 0x7f724815e7b0 2024-01-27T02:32:17.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.539+0000 7f723d7fa700 1 -- 172.21.15.7:0/1507461566 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723800f690 con 0x7f724815e7b0 2024-01-27T02:32:17.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.539+0000 7f723d7fa700 1 -- 172.21.15.7:0/1507461566 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723800fa50 con 0x7f724815e7b0 2024-01-27T02:32:17.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.539+0000 7f724f42c700 1 -- 172.21.15.7:0/1507461566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72480c17f0 con 0x7f724815e7b0 2024-01-27T02:32:17.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.539+0000 7f724f42c700 1 -- 172.21.15.7:0/1507461566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7248085220 con 0x7f724815e7b0 2024-01-27T02:32:17.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.541+0000 7fa814bae700 1 -- 172.21.15.7:0/722701962 --> [v2:172.21.15.7:6816/39319,v1:172.21.15.7:6817/39319] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa7f4002d30 con 0x7fa7f006f290 2024-01-27T02:32:17.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.542+0000 7ffae40cd700 1 -- 172.21.15.7:0/829107171 >> [v2:172.21.15.7:6808/39310,v1:172.21.15.7:6809/39310] conn(0x7ffac006f140 msgr2=0x7ffac0071550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.542+0000 7ffae40cd700 1 --2- 172.21.15.7:0/829107171 >> [v2:172.21.15.7:6808/39310,v1:172.21.15.7:6809/39310] conn(0x7ffac006f140 0x7ffac0071550 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.542+0000 7f723d7fa700 1 -- 172.21.15.7:0/1507461566 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f723801a400 con 0x7f724815e7b0 2024-01-27T02:32:17.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.541+0000 7f1347048700 1 -- 172.21.15.7:0/4198678155 --> [v2:172.21.15.7:6824/39325,v1:172.21.15.7:6825/39325] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f13401486f0 con 0x7f132c06f0c0 2024-01-27T02:32:17.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.542+0000 7ffae40cd700 1 -- 172.21.15.7:0/829107171 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffac0066f30 msgr2=0x7ffac00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.542+0000 7ffae40cd700 1 --2- 172.21.15.7:0/829107171 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffac0066f30 0x7ffac00693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ffad8006fd0 tx=0x7ffad8008040 comp rx=0 tx=0).stop 2024-01-27T02:32:17.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.542+0000 7ffae40cd700 1 -- 172.21.15.7:0/829107171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffadc0cc5c0 msgr2=0x7ffadc0691e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.542+0000 7ffae40cd700 1 --2- 172.21.15.7:0/829107171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffadc0cc5c0 0x7ffadc0691e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ffad400bad0 tx=0x7ffad4009780 comp rx=0 tx=0).stop 2024-01-27T02:32:17.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.542+0000 7ffae266a700 1 -- 172.21.15.7:0/829107171 reap_dead start 2024-01-27T02:32:17.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.542+0000 7ffae40cd700 1 -- 172.21.15.7:0/829107171 shutdown_connections 2024-01-27T02:32:17.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.542+0000 7ffae40cd700 1 -- 172.21.15.7:0/829107171 >> 172.21.15.7:0/829107171 conn(0x7ffadc064090 msgr2=0x7ffadc0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.542+0000 7f723d7fa700 1 --2- 172.21.15.7:0/1507461566 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f722c067000 0x7f722c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.542+0000 7f723d7fa700 1 -- 172.21.15.7:0/1507461566 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7238083f70 con 0x7f724815e7b0 2024-01-27T02:32:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.542+0000 7f133cff9700 1 -- 172.21.15.7:0/4198678155 <== osd.2 v2:172.21.15.7:6824/39325 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f13401486f0 con 0x7f132c06f0c0 2024-01-27T02:32:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.542+0000 7f724d1c8700 1 --2- 172.21.15.7:0/1507461566 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f722c067000 0x7f722c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.543+0000 7f724f42c700 1 --2- 172.21.15.7:0/1507461566 >> [v2:172.21.15.188:6824/39412,v1:172.21.15.188:6825/39412] conn(0x7f72480afbb0 0x7f7248088150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.543+0000 7f724f42c700 1 -- 172.21.15.7:0/1507461566 --> [v2:172.21.15.188:6824/39412,v1:172.21.15.188:6825/39412] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f72480008d0 con 0x7f72480afbb0 2024-01-27T02:32:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.543+0000 7f1347048700 1 -- 172.21.15.7:0/4198678155 >> [v2:172.21.15.7:6824/39325,v1:172.21.15.7:6825/39325] conn(0x7f132c06f0c0 msgr2=0x7f132c0714d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.543+0000 7f1347048700 1 --2- 172.21.15.7:0/4198678155 >> [v2:172.21.15.7:6824/39325,v1:172.21.15.7:6825/39325] conn(0x7f132c06f0c0 0x7f132c0714d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.543+0000 7f1347048700 1 -- 172.21.15.7:0/4198678155 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f132c066f30 msgr2=0x7f132c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.543+0000 7f1347048700 1 --2- 172.21.15.7:0/4198678155 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f132c066f30 0x7f132c0693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1334009740 tx=0x7f1334006c60 comp rx=0 tx=0).stop 2024-01-27T02:32:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.543+0000 7f724c9c7700 1 --2- 172.21.15.7:0/1507461566 >> [v2:172.21.15.188:6824/39412,v1:172.21.15.188:6825/39412] conn(0x7f72480afbb0 0x7f7248088150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.543+0000 7f1347048700 1 -- 172.21.15.7:0/4198678155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f134015e7b0 msgr2=0x7f13401481b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.543+0000 7f1347048700 1 --2- 172.21.15.7:0/4198678155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f134015e7b0 0x7f13401481b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1338014000 tx=0x7f1338017000 comp rx=0 tx=0).stop 2024-01-27T02:32:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.543+0000 7f13455e5700 1 -- 172.21.15.7:0/4198678155 reap_dead start 2024-01-27T02:32:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.544+0000 7fa7fe7fc700 1 -- 172.21.15.7:0/722701962 <== osd.0 v2:172.21.15.7:6816/39319 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa7f4002d30 con 0x7fa7f006f290 2024-01-27T02:32:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.543+0000 7f1347048700 1 -- 172.21.15.7:0/4198678155 shutdown_connections 2024-01-27T02:32:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.543+0000 7f1347048700 1 -- 172.21.15.7:0/4198678155 >> 172.21.15.7:0/4198678155 conn(0x7f1340064090 msgr2=0x7f13400b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.544+0000 7f1347048700 1 -- 172.21.15.7:0/4198678155 shutdown_connections 2024-01-27T02:32:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.543+0000 7f11376d5700 1 -- 172.21.15.7:0/1732184761 >> [v2:172.21.15.188:6801/39374,v1:172.21.15.188:6803/39374] conn(0x7f111406f260 msgr2=0x7f1114071670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.543+0000 7f11376d5700 1 --2- 172.21.15.7:0/1732184761 >> [v2:172.21.15.188:6801/39374,v1:172.21.15.188:6803/39374] conn(0x7f111406f260 0x7f1114071670 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.544+0000 7f11376d5700 1 -- 172.21.15.7:0/1732184761 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1114067050 msgr2=0x7f1114069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.544+0000 7f11376d5700 1 --2- 172.21.15.7:0/1732184761 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1114067050 0x7f1114069500 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1128007fd0 tx=0x7f11280018a0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.544+0000 7f11376d5700 1 -- 172.21.15.7:0/1732184761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11300c9aa0 msgr2=0x7f11300bdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.544+0000 7f11376d5700 1 --2- 172.21.15.7:0/1732184761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11300c9aa0 0x7f11300bdf60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f112000c650 tx=0x7f1120008040 comp rx=0 tx=0).stop 2024-01-27T02:32:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.544+0000 7f1135c72700 1 -- 172.21.15.7:0/1732184761 reap_dead start 2024-01-27T02:32:17.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.544+0000 7f11376d5700 1 -- 172.21.15.7:0/1732184761 shutdown_connections 2024-01-27T02:32:17.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.544+0000 7f11376d5700 1 -- 172.21.15.7:0/1732184761 >> 172.21.15.7:0/1732184761 conn(0x7f1130063580 msgr2=0x7f11300ca930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.543+0000 7f724d1c8700 1 --2- 172.21.15.7:0/1507461566 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f722c067000 0x7f722c0694b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7240009880 tx=0x7f7240017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.543+0000 7f724c9c7700 1 --2- 172.21.15.7:0/1507461566 >> [v2:172.21.15.188:6824/39412,v1:172.21.15.188:6825/39412] conn(0x7f72480afbb0 0x7f7248088150 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.544+0000 7f11376d5700 1 -- 172.21.15.7:0/1732184761 shutdown_connections 2024-01-27T02:32:17.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.544+0000 7f11376d5700 1 -- 172.21.15.7:0/1732184761 wait complete. 2024-01-27T02:32:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.544+0000 7f723d7fa700 1 -- 172.21.15.7:0/1507461566 <== osd.6 v2:172.21.15.188:6824/39412 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f72480008d0 con 0x7f72480afbb0 2024-01-27T02:32:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.544+0000 7f1347048700 1 -- 172.21.15.7:0/4198678155 wait complete. 2024-01-27T02:32:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.547+0000 7ffae40cd700 1 -- 172.21.15.7:0/829107171 shutdown_connections 2024-01-27T02:32:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.547+0000 7ffae40cd700 1 -- 172.21.15.7:0/829107171 wait complete. 2024-01-27T02:32:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.548+0000 7fb2a055c700 1 -- 172.21.15.7:0/3706275572 --> [v2:172.21.15.7:6800/39309,v1:172.21.15.7:6801/39309] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb298161c70 con 0x7fb27c06f210 2024-01-27T02:32:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.550+0000 7f33a7e23700 1 -- 172.21.15.7:0/3117514081 --> [v2:172.21.15.188:6800/39373,v1:172.21.15.188:6802/39373] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f33a00c70c0 con 0x7f338406f210 2024-01-27T02:32:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.551+0000 7f3395ffb700 1 -- 172.21.15.7:0/3117514081 <== osd.5 v2:172.21.15.188:6800/39373 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f33a00c70c0 con 0x7f338406f210 2024-01-27T02:32:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.552+0000 7f33a7e23700 1 -- 172.21.15.7:0/3117514081 >> [v2:172.21.15.188:6800/39373,v1:172.21.15.188:6802/39373] conn(0x7f338406f210 msgr2=0x7f3384071620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.552+0000 7f33a7e23700 1 --2- 172.21.15.7:0/3117514081 >> [v2:172.21.15.188:6800/39373,v1:172.21.15.188:6802/39373] conn(0x7f338406f210 0x7f3384071620 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.552+0000 7f33a7e23700 1 -- 172.21.15.7:0/3117514081 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3384067080 msgr2=0x7f3384069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.552+0000 7f33a7e23700 1 --2- 172.21.15.7:0/3117514081 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3384067080 0x7f3384069530 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f33a006a2e0 tx=0x7f3390008040 comp rx=0 tx=0).stop 2024-01-27T02:32:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.552+0000 7f33a7e23700 1 -- 172.21.15.7:0/3117514081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33a0069500 msgr2=0x7f33a00ad7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.552+0000 7f33a7e23700 1 --2- 172.21.15.7:0/3117514081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33a0069500 0x7f33a00ad7b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f339c00c4a0 tx=0x7f339c007f10 comp rx=0 tx=0).stop 2024-01-27T02:32:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.552+0000 7f33a63c0700 1 -- 172.21.15.7:0/3117514081 reap_dead start 2024-01-27T02:32:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.553+0000 7f33a7e23700 1 -- 172.21.15.7:0/3117514081 shutdown_connections 2024-01-27T02:32:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.553+0000 7f33a7e23700 1 -- 172.21.15.7:0/3117514081 >> 172.21.15.7:0/3117514081 conn(0x7f33a0064090 msgr2=0x7f33a00cb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.554+0000 7fb28a7fc700 1 -- 172.21.15.7:0/3706275572 <== osd.1 v2:172.21.15.7:6800/39309 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb298161c70 con 0x7fb27c06f210 2024-01-27T02:32:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.555+0000 7f33a7e23700 1 -- 172.21.15.7:0/3117514081 shutdown_connections 2024-01-27T02:32:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.558+0000 7fb2a055c700 1 -- 172.21.15.7:0/3706275572 >> [v2:172.21.15.7:6800/39309,v1:172.21.15.7:6801/39309] conn(0x7fb27c06f210 msgr2=0x7fb27c071620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.558+0000 7fb2a055c700 1 --2- 172.21.15.7:0/3706275572 >> [v2:172.21.15.7:6800/39309,v1:172.21.15.7:6801/39309] conn(0x7fb27c06f210 0x7fb27c071620 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.558+0000 7fb2a055c700 1 -- 172.21.15.7:0/3706275572 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb27c067080 msgr2=0x7fb27c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.558+0000 7fb2a055c700 1 --2- 172.21.15.7:0/3706275572 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb27c067080 0x7fb27c069530 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb29806c2c0 tx=0x7fb28c016000 comp rx=0 tx=0).stop 2024-01-27T02:32:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.558+0000 7fb2a055c700 1 -- 172.21.15.7:0/3706275572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb298166c80 msgr2=0x7fb29814df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.558+0000 7fb2a055c700 1 --2- 172.21.15.7:0/3706275572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb298166c80 0x7fb29814df10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb29000bb00 tx=0x7fb290009780 comp rx=0 tx=0).stop 2024-01-27T02:32:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.561+0000 7fa814bae700 1 -- 172.21.15.7:0/722701962 >> [v2:172.21.15.7:6816/39319,v1:172.21.15.7:6817/39319] conn(0x7fa7f006f290 msgr2=0x7fa7f00716a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.561+0000 7fa814bae700 1 --2- 172.21.15.7:0/722701962 >> [v2:172.21.15.7:6816/39319,v1:172.21.15.7:6817/39319] conn(0x7fa7f006f290 0x7fa7f00716a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.561+0000 7fb29eaf9700 1 -- 172.21.15.7:0/3706275572 reap_dead start 2024-01-27T02:32:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.561+0000 7f2b5968e700 1 -- 172.21.15.7:0/1967153375 --> [v2:172.21.15.188:6816/39375,v1:172.21.15.188:6817/39375] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2b540692d0 con 0x7f2b4006f140 2024-01-27T02:32:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.561+0000 7fa814bae700 1 -- 172.21.15.7:0/722701962 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa7f0067080 msgr2=0x7fa7f0069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.561+0000 7fa814bae700 1 --2- 172.21.15.7:0/722701962 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa7f0067080 0x7fa7f0069530 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa808007580 tx=0x7fa808009780 comp rx=0 tx=0).stop 2024-01-27T02:32:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.561+0000 7fa814bae700 1 -- 172.21.15.7:0/722701962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa81006d5d0 msgr2=0x7fa810149b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.561+0000 7fa814bae700 1 --2- 172.21.15.7:0/722701962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa81006d5d0 0x7fa810149b00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa80400c5d0 tx=0x7fa804007f60 comp rx=0 tx=0).stop 2024-01-27T02:32:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.561+0000 7fb2a055c700 1 -- 172.21.15.7:0/3706275572 shutdown_connections 2024-01-27T02:32:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.561+0000 7fb2a055c700 1 -- 172.21.15.7:0/3706275572 >> 172.21.15.7:0/3706275572 conn(0x7fb298063580 msgr2=0x7fb29806a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.561+0000 7fa80ed9d700 1 -- 172.21.15.7:0/722701962 reap_dead start 2024-01-27T02:32:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.561+0000 7fb2a055c700 1 -- 172.21.15.7:0/3706275572 shutdown_connections 2024-01-27T02:32:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.562+0000 7f2b3affd700 1 -- 172.21.15.7:0/1967153375 <== osd.7 v2:172.21.15.188:6816/39375 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2b540692d0 con 0x7f2b4006f140 2024-01-27T02:32:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.561+0000 7fb2a055c700 1 -- 172.21.15.7:0/3706275572 wait complete. 2024-01-27T02:32:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.562+0000 7fa814bae700 1 -- 172.21.15.7:0/722701962 shutdown_connections 2024-01-27T02:32:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.562+0000 7fa814bae700 1 -- 172.21.15.7:0/722701962 >> 172.21.15.7:0/722701962 conn(0x7fa810064090 msgr2=0x7fa8100b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.562+0000 7fa814bae700 1 -- 172.21.15.7:0/722701962 shutdown_connections 2024-01-27T02:32:17.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.562+0000 7fa814bae700 1 -- 172.21.15.7:0/722701962 wait complete. 2024-01-27T02:32:17.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.562+0000 7f2b5968e700 1 -- 172.21.15.7:0/1967153375 >> [v2:172.21.15.188:6816/39375,v1:172.21.15.188:6817/39375] conn(0x7f2b4006f140 msgr2=0x7f2b40071550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.562+0000 7f2b5968e700 1 --2- 172.21.15.7:0/1967153375 >> [v2:172.21.15.188:6816/39375,v1:172.21.15.188:6817/39375] conn(0x7f2b4006f140 0x7f2b40071550 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.562+0000 7f2b5968e700 1 -- 172.21.15.7:0/1967153375 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2b40066f30 msgr2=0x7f2b400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.562+0000 7f2b5968e700 1 --2- 172.21.15.7:0/1967153375 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2b40066f30 0x7f2b400693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2b5406dad0 tx=0x7f2b44017000 comp rx=0 tx=0).stop 2024-01-27T02:32:17.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.562+0000 7f2b5968e700 1 -- 172.21.15.7:0/1967153375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b5406e880 msgr2=0x7f2b540af6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.562+0000 7f2b5968e700 1 --2- 172.21.15.7:0/1967153375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b5406e880 0x7f2b540af6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800b630 tx=0x7f2b480091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.562+0000 7f2b537fe700 1 -- 172.21.15.7:0/1967153375 reap_dead start 2024-01-27T02:32:17.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.563+0000 7f2b5968e700 1 -- 172.21.15.7:0/1967153375 shutdown_connections 2024-01-27T02:32:17.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.563+0000 7f2b5968e700 1 -- 172.21.15.7:0/1967153375 >> 172.21.15.7:0/1967153375 conn(0x7f2b540ca610 msgr2=0x7f2b540b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.579 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-27T02:32:17.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.566+0000 7f33a7e23700 1 -- 172.21.15.7:0/3117514081 wait complete. 2024-01-27T02:32:17.580 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-27T02:32:17.580 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-27T02:32:17.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.567+0000 7f2b5968e700 1 -- 172.21.15.7:0/1967153375 shutdown_connections 2024-01-27T02:32:17.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.567+0000 7f2b5968e700 1 -- 172.21.15.7:0/1967153375 wait complete. 2024-01-27T02:32:17.581 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-27T02:32:17.582 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-27T02:32:17.582 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-27T02:32:17.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.571+0000 7f724f42c700 1 -- 172.21.15.7:0/1507461566 --> [v2:172.21.15.188:6824/39412,v1:172.21.15.188:6825/39412] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f724805e3d0 con 0x7f72480afbb0 2024-01-27T02:32:17.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.571+0000 7f723d7fa700 1 -- 172.21.15.7:0/1507461566 <== osd.6 v2:172.21.15.188:6824/39412 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f724805e3d0 con 0x7f72480afbb0 2024-01-27T02:32:17.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.571+0000 7f724f42c700 1 -- 172.21.15.7:0/1507461566 >> [v2:172.21.15.188:6824/39412,v1:172.21.15.188:6825/39412] conn(0x7f72480afbb0 msgr2=0x7f7248088150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.571+0000 7f724f42c700 1 --2- 172.21.15.7:0/1507461566 >> [v2:172.21.15.188:6824/39412,v1:172.21.15.188:6825/39412] conn(0x7f72480afbb0 0x7f7248088150 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.572+0000 7f724f42c700 1 -- 172.21.15.7:0/1507461566 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f722c067000 msgr2=0x7f722c0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.572+0000 7f724f42c700 1 --2- 172.21.15.7:0/1507461566 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f722c067000 0x7f722c0694b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7240009880 tx=0x7f7240017000 comp rx=0 tx=0).stop 2024-01-27T02:32:17.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.572+0000 7f724f42c700 1 -- 172.21.15.7:0/1507461566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f724815e7b0 msgr2=0x7f7248087700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.572+0000 7f724f42c700 1 --2- 172.21.15.7:0/1507461566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f724815e7b0 0x7f7248087700 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f723800b5e0 tx=0x7f7238009120 comp rx=0 tx=0).stop 2024-01-27T02:32:17.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.572+0000 7f724d9c9700 1 -- 172.21.15.7:0/1507461566 reap_dead start 2024-01-27T02:32:17.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.572+0000 7f724f42c700 1 -- 172.21.15.7:0/1507461566 shutdown_connections 2024-01-27T02:32:17.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.572+0000 7f724f42c700 1 -- 172.21.15.7:0/1507461566 >> 172.21.15.7:0/1507461566 conn(0x7f7248064090 msgr2=0x7f724814e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.573+0000 7f724f42c700 1 -- 172.21.15.7:0/1507461566 shutdown_connections 2024-01-27T02:32:17.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.573+0000 7f724f42c700 1 -- 172.21.15.7:0/1507461566 wait complete. 2024-01-27T02:32:17.588 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-27T02:32:17.588 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-01-27T02:32:17.589 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-27T02:32:17.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-27T02:32:17.590 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-27T02:32:17.591 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-27T02:32:17.593 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-27T02:32:17.594 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-27T02:32:17.595 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-27T02:32:17.596 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-01-27T02:32:17.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.806+0000 7f85e75f6700 1 Processor -- start 2024-01-27T02:32:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.806+0000 7f95ee395700 1 Processor -- start 2024-01-27T02:32:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.806+0000 7f95ee395700 1 -- start start 2024-01-27T02:32:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.808+0000 7f95ee395700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f95e80ad390 0x7f95e80a7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.806+0000 7f85e75f6700 1 -- start start 2024-01-27T02:32:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.808+0000 7f85e75f6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f85e00c96d0 0x7f85e00cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.808+0000 7f85e75f6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e0162360 con 0x7f85e00c9aa0 2024-01-27T02:32:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.808+0000 7f85e75f6700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e0162640 con 0x7f85e006d360 2024-01-27T02:32:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.808+0000 7f85e75f6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e0162150 con 0x7f85e00c96d0 2024-01-27T02:32:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.808+0000 7f95ee395700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e80afd30 con 0x7f95e8156a70 2024-01-27T02:32:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.808+0000 7f95ee395700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e806df80 con 0x7f95e81566a0 2024-01-27T02:32:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.808+0000 7f95ee395700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e80af770 con 0x7f95e80ad390 2024-01-27T02:32:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.808+0000 7f95e7fff700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f95e81566a0 0x7f95e80aee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:45414/0 (socket says 172.21.15.7:45414) 2024-01-27T02:32:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.808+0000 7f95e7fff700 1 -- 172.21.15.7:0/511804573 learned_addr learned my addr 172.21.15.7:0/511804573 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.808+0000 7f85e4b91700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f85e006d360 0x7f85e00ae860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:45428/0 (socket says 172.21.15.7:45428) 2024-01-27T02:32:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.808+0000 7f85e4b91700 1 -- 172.21.15.7:0/1239106719 learned_addr learned my addr 172.21.15.7:0/1239106719 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7f95ec932700 1 --2- 172.21.15.7:0/511804573 >> v2:172.21.15.188:3300/0 conn(0x7f95e80ad390 0x7f95e80a7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7f85d7fff700 1 -- 172.21.15.7:0/1239106719 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2859766248 0 0) 0x7f85e0162640 con 0x7f85e006d360 2024-01-27T02:32:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7f85d7fff700 1 -- 172.21.15.7:0/1239106719 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85cc003570 con 0x7f85e006d360 2024-01-27T02:32:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7fcd7ff38700 1 Processor -- start 2024-01-27T02:32:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7f95e6ffd700 1 -- 172.21.15.7:0/511804573 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1126441771 0 0) 0x7f95e806df80 con 0x7f95e81566a0 2024-01-27T02:32:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7f95e6ffd700 1 -- 172.21.15.7:0/511804573 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95d0003570 con 0x7f95e81566a0 2024-01-27T02:32:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.810+0000 7fcd7ff38700 1 -- start start 2024-01-27T02:32:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.811+0000 7fcd7ff38700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd780c9aa0 0x7fcd780aef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.811+0000 7fcd7ff38700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd78162360 con 0x7fcd7806d360 2024-01-27T02:32:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.811+0000 7fcd7ff38700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd78162640 con 0x7fcd780c96d0 2024-01-27T02:32:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.811+0000 7fcd7ff38700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd78162150 con 0x7fcd780c9aa0 2024-01-27T02:32:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.811+0000 7fcd7d4d3700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcd7806d360 0x7fcd780ae860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51430/0 (socket says 172.21.15.7:51430) 2024-01-27T02:32:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.811+0000 7fcd7d4d3700 1 -- 172.21.15.7:0/2864623850 learned_addr learned my addr 172.21.15.7:0/2864623850 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7f85d7fff700 1 -- 172.21.15.7:0/1239106719 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 160843007 0 0) 0x7f85cc003570 con 0x7f85e006d360 2024-01-27T02:32:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7f85d7fff700 1 -- 172.21.15.7:0/1239106719 >> v2:172.21.15.7:3300/0 conn(0x7f85e00c96d0 msgr2=0x7f85e00cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7f85d7fff700 1 --2- 172.21.15.7:0/1239106719 >> v2:172.21.15.7:3300/0 conn(0x7f85e00c96d0 0x7f85e00cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7f85d7fff700 1 -- 172.21.15.7:0/1239106719 >> v1:172.21.15.7:6789/0 conn(0x7f85e00c9aa0 legacy=0x7f85e00ca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7f85d7fff700 1 -- 172.21.15.7:0/1239106719 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e0162920 con 0x7f85e006d360 2024-01-27T02:32:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7f85d7fff700 1 -- 172.21.15.7:0/1239106719 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f85dc001470 con 0x7f85e006d360 2024-01-27T02:32:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7f85d7fff700 1 -- 172.21.15.7:0/1239106719 >> v1:172.21.15.188:6789/0 conn(0x7f85e006d360 legacy=0x7f85e00ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7f85d7fff700 1 --2- 172.21.15.7:0/1239106719 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85cc003a60 0x7f85cc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7f85d7fff700 1 --2- 172.21.15.7:0/1239106719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85cc004810 0x7f85cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7f85d7fff700 1 --2- 172.21.15.7:0/1239106719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85cc0053c0 0x7f85cc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7f85d7fff700 1 -- 172.21.15.7:0/1239106719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e0162150 con 0x7f85cc004810 2024-01-27T02:32:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7f85d7fff700 1 -- 172.21.15.7:0/1239106719 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc00c250 con 0x7f85cc0053c0 2024-01-27T02:32:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.810+0000 7f85d7fff700 1 -- 172.21.15.7:0/1239106719 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc00c410 con 0x7f85cc003a60 2024-01-27T02:32:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.810+0000 7f85e4b91700 1 --2- 172.21.15.7:0/1239106719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85cc004810 0x7f85cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.811+0000 7f85e5b93700 1 --2- 172.21.15.7:0/1239106719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85cc0053c0 0x7f85cc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.811+0000 7f85e5392700 1 --2- 172.21.15.7:0/1239106719 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85cc003a60 0x7f85cc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.811+0000 7f85e5b93700 1 -- 172.21.15.7:0/1239106719 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85cc003a60 msgr2=0x7f85cc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.811+0000 7f85e5b93700 1 --2- 172.21.15.7:0/1239106719 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85cc003a60 0x7f85cc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.811+0000 7f85e5b93700 1 -- 172.21.15.7:0/1239106719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85cc004810 msgr2=0x7f85cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.811+0000 7f85e5b93700 1 --2- 172.21.15.7:0/1239106719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85cc004810 0x7f85cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.811+0000 7f85e5b93700 1 -- 172.21.15.7:0/1239106719 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e0162920 con 0x7f85cc0053c0 2024-01-27T02:32:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.811+0000 7f85e5b93700 1 -- 172.21.15.7:0/1239106719 reap_dead start 2024-01-27T02:32:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.812+0000 7f85e5b93700 1 --2- 172.21.15.7:0/1239106719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85cc0053c0 0x7f85cc00ba60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f85d8001fc0 tx=0x7f85d8001640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2481df4882cd4d34 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7f95e6ffd700 1 -- 172.21.15.7:0/511804573 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4001754815 0 0) 0x7f95d0003570 con 0x7f95e81566a0 2024-01-27T02:32:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7f95e6ffd700 1 -- 172.21.15.7:0/511804573 >> v2:172.21.15.188:3300/0 conn(0x7f95e80ad390 msgr2=0x7f95e80a7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7f95e6ffd700 1 --2- 172.21.15.7:0/511804573 >> v2:172.21.15.188:3300/0 conn(0x7f95e80ad390 0x7f95e80a7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7f95e6ffd700 1 -- 172.21.15.7:0/511804573 >> v1:172.21.15.7:6789/0 conn(0x7f95e8156a70 legacy=0x7f95e80acc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.809+0000 7f95e6ffd700 1 -- 172.21.15.7:0/511804573 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e80afa50 con 0x7f95e81566a0 2024-01-27T02:32:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.810+0000 7f95e6ffd700 1 -- 172.21.15.7:0/511804573 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f95e00017f0 con 0x7f95e81566a0 2024-01-27T02:32:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.810+0000 7f95e6ffd700 1 -- 172.21.15.7:0/511804573 >> v1:172.21.15.188:6789/0 conn(0x7f95e81566a0 legacy=0x7f95e80aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.810+0000 7f95e6ffd700 1 --2- 172.21.15.7:0/511804573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d0003a60 0x7f95d0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.810+0000 7f95e6ffd700 1 --2- 172.21.15.7:0/511804573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95d0004810 0x7f95d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.810+0000 7f95e6ffd700 1 --2- 172.21.15.7:0/511804573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95d00051c0 0x7f95d000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.810+0000 7f95e6ffd700 1 -- 172.21.15.7:0/511804573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e80af770 con 0x7f95d0003a60 2024-01-27T02:32:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.810+0000 7f95e6ffd700 1 -- 172.21.15.7:0/511804573 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d0014990 con 0x7f95d00051c0 2024-01-27T02:32:17.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.810+0000 7f95e6ffd700 1 -- 172.21.15.7:0/511804573 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d0014b50 con 0x7f95d0004810 2024-01-27T02:32:17.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.810+0000 7f95e7fff700 1 --2- 172.21.15.7:0/511804573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d0003a60 0x7f95d0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.810+0000 7f95e7fff700 1 -- 172.21.15.7:0/511804573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95d0004810 msgr2=0x7f95d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.811+0000 7f95e7fff700 1 --2- 172.21.15.7:0/511804573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95d0004810 0x7f95d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.811+0000 7f95e7fff700 1 -- 172.21.15.7:0/511804573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95d00051c0 msgr2=0x7f95d000ba60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-27T02:32:17.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.811+0000 7f95e7fff700 1 --2- 172.21.15.7:0/511804573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95d00051c0 0x7f95d000ba60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.811+0000 7f95e7fff700 1 -- 172.21.15.7:0/511804573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e80afa50 con 0x7f95d0003a60 2024-01-27T02:32:17.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.812+0000 7fcd7e4d5700 1 --2- 172.21.15.7:0/2864623850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd780c9aa0 0x7fcd780aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.812+0000 7f95e7fff700 1 --2- 172.21.15.7:0/511804573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d0003a60 0x7f95d0003e70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f95e000a990 tx=0x7f95e00021c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2709658517dd55f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.813+0000 7f85d7fff700 1 -- 172.21.15.7:0/1239106719 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d8001950 con 0x7f85cc0053c0 2024-01-27T02:32:17.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.813+0000 7f85d7fff700 1 -- 172.21.15.7:0/1239106719 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d800c8b0 con 0x7f85cc0053c0 2024-01-27T02:32:17.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.813+0000 7f85d7fff700 1 -- 172.21.15.7:0/1239106719 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d800cc30 con 0x7f85cc0053c0 2024-01-27T02:32:17.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.813+0000 7fedcf55d700 1 Processor -- start 2024-01-27T02:32:17.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.814+0000 7fcd7ccd2700 1 -- 172.21.15.7:0/2864623850 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018216979 0 0) 0x7fcd78162360 con 0x7fcd7806d360 2024-01-27T02:32:17.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.814+0000 7fcd7ccd2700 1 -- 172.21.15.7:0/2864623850 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd64003570 con 0x7fcd7806d360 2024-01-27T02:32:17.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.814+0000 7fcd7ccd2700 1 -- 172.21.15.7:0/2864623850 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4023037517 0 0) 0x7fcd78162640 con 0x7fcd780c96d0 2024-01-27T02:32:17.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.814+0000 7fcd7ccd2700 1 -- 172.21.15.7:0/2864623850 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd78162360 con 0x7fcd780c96d0 2024-01-27T02:32:17.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.813+0000 7f95e6ffd700 1 -- 172.21.15.7:0/511804573 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e00070d0 con 0x7f95d0003a60 2024-01-27T02:32:17.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.813+0000 7f95e6ffd700 1 -- 172.21.15.7:0/511804573 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e0007230 con 0x7f95d0003a60 2024-01-27T02:32:17.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.813+0000 7f95e6ffd700 1 -- 172.21.15.7:0/511804573 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e0007610 con 0x7f95d0003a60 2024-01-27T02:32:17.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.814+0000 7f95ee395700 1 -- 172.21.15.7:0/511804573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d0003a60 msgr2=0x7f95d0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.814+0000 7f95ee395700 1 --2- 172.21.15.7:0/511804573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d0003a60 0x7f95d0003e70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f95e000a990 tx=0x7f95e00021c0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.814+0000 7f95ec932700 1 -- 172.21.15.7:0/511804573 reap_dead start 2024-01-27T02:32:17.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.814+0000 7fcd7e4d5700 1 -- 172.21.15.7:0/2864623850 >> v1:172.21.15.188:6789/0 conn(0x7fcd780c96d0 legacy=0x7fcd780cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.814+0000 7fcd7e4d5700 1 -- 172.21.15.7:0/2864623850 >> v1:172.21.15.7:6789/0 conn(0x7fcd7806d360 legacy=0x7fcd780ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.814+0000 7fcd7e4d5700 1 -- 172.21.15.7:0/2864623850 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd78162920 con 0x7fcd780c9aa0 2024-01-27T02:32:17.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.814+0000 7fedcf55d700 1 -- start start 2024-01-27T02:32:17.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.815+0000 7fcd7e4d5700 1 --2- 172.21.15.7:0/2864623850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd780c9aa0 0x7fcd780aef70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009910 tx=0x7fcd680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6236f85684f9b97e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.815+0000 7fedcf55d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedc81404e0 0x7fedc80cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.815+0000 7fedcf55d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fedc806e9f0 0x7fedc80cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.815+0000 7fedcf55d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc80cce30 con 0x7fedc80b09f0 2024-01-27T02:32:17.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.815+0000 7fedcf55d700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc806b3b0 con 0x7fedc806e9f0 2024-01-27T02:32:17.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.815+0000 7fedcf55d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc80cc3e0 con 0x7fedc81404e0 2024-01-27T02:32:17.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.815+0000 7f85e75f6700 1 -- 172.21.15.7:0/1239106719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85cc0053c0 msgr2=0x7f85cc00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.815+0000 7f85e75f6700 1 --2- 172.21.15.7:0/1239106719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85cc0053c0 0x7f85cc00ba60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f85d8001fc0 tx=0x7f85d8001640 comp rx=0 tx=0).stop 2024-01-27T02:32:17.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.815+0000 7fcd7ccd2700 1 -- 172.21.15.7:0/2864623850 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd70001470 con 0x7fcd780c9aa0 2024-01-27T02:32:17.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.815+0000 7fcd7ccd2700 1 -- 172.21.15.7:0/2864623850 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd68011420 con 0x7fcd780c9aa0 2024-01-27T02:32:17.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.815+0000 7fcd7ccd2700 1 -- 172.21.15.7:0/2864623850 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd680117c0 con 0x7fcd780c9aa0 2024-01-27T02:32:17.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.815+0000 7fedccaf8700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fedc80b09f0 0x7fedc806e2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51444/0 (socket says 172.21.15.7:51444) 2024-01-27T02:32:17.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.815+0000 7fedccaf8700 1 -- 172.21.15.7:0/2335135985 learned_addr learned my addr 172.21.15.7:0/2335135985 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:17.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.816+0000 7fedbffff700 1 -- 172.21.15.7:0/2335135985 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158519793 0 0) 0x7fedc80cce30 con 0x7fedc80b09f0 2024-01-27T02:32:17.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.816+0000 7fedbffff700 1 -- 172.21.15.7:0/2335135985 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedb4003570 con 0x7fedc80b09f0 2024-01-27T02:32:17.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.816+0000 7fcd7ff38700 1 -- 172.21.15.7:0/2864623850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd780c9aa0 msgr2=0x7fcd780aef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.816+0000 7fcd7ff38700 1 --2- 172.21.15.7:0/2864623850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd780c9aa0 0x7fcd780aef70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009910 tx=0x7fcd680091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.816+0000 7fedbffff700 1 -- 172.21.15.7:0/2335135985 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4059700138 0 0) 0x7fedb4003570 con 0x7fedc80b09f0 2024-01-27T02:32:17.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.816+0000 7fedbffff700 1 -- 172.21.15.7:0/2335135985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedc81404e0 msgr2=0x7fedc80cb800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-27T02:32:17.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.816+0000 7fedbffff700 1 --2- 172.21.15.7:0/2335135985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedc81404e0 0x7fedc80cb800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.816+0000 7fedbffff700 1 -- 172.21.15.7:0/2335135985 >> v2:172.21.15.188:3300/0 conn(0x7fedc806e9f0 msgr2=0x7fedc80cbd40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-27T02:32:17.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.816+0000 7fedbffff700 1 --2- 172.21.15.7:0/2335135985 >> v2:172.21.15.188:3300/0 conn(0x7fedc806e9f0 0x7fedc80cbd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.816+0000 7fedbffff700 1 -- 172.21.15.7:0/2335135985 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc80cc6c0 con 0x7fedc80b09f0 2024-01-27T02:32:17.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.816+0000 7fedbffff700 1 -- 172.21.15.7:0/2335135985 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7fedb8001470 con 0x7fedc80b09f0 2024-01-27T02:32:17.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.816+0000 7fedbffff700 1 -- 172.21.15.7:0/2335135985 >> v1:172.21.15.7:6789/0 conn(0x7fedc80b09f0 legacy=0x7fedc806e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.816+0000 7fedbffff700 1 --2- 172.21.15.7:0/2335135985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fedb4003ae0 0x7fedb4003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.816+0000 7fedbffff700 1 --2- 172.21.15.7:0/2335135985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedb4004810 0x7fedb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.816+0000 7fedbffff700 1 --2- 172.21.15.7:0/2335135985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedb4005310 0x7fedb400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.816+0000 7fedbffff700 1 -- 172.21.15.7:0/2335135985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc806b3b0 con 0x7fedb4005310 2024-01-27T02:32:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.816+0000 7fedbffff700 1 -- 172.21.15.7:0/2335135985 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc80cc3e0 con 0x7fedb4003ae0 2024-01-27T02:32:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.816+0000 7fedbffff700 1 -- 172.21.15.7:0/2335135985 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb400c280 con 0x7fedb4004810 2024-01-27T02:32:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.815+0000 7f95ee395700 1 -- 172.21.15.7:0/511804573 shutdown_connections 2024-01-27T02:32:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.815+0000 7f95ee395700 1 --2- 172.21.15.7:0/511804573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d0003a60 0x7f95d0003e70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.815+0000 7f95ee395700 1 -- 172.21.15.7:0/511804573 >> 172.21.15.7:0/511804573 conn(0x7f95e80679b0 msgr2=0x7f95e8067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.815+0000 7f95ee395700 1 -- 172.21.15.7:0/511804573 shutdown_connections 2024-01-27T02:32:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.815+0000 7f95ee395700 1 -- 172.21.15.7:0/511804573 wait complete. 2024-01-27T02:32:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.812+0000 7f7fcdc93700 1 Processor -- start 2024-01-27T02:32:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.812+0000 7f7fcdc93700 1 -- start start 2024-01-27T02:32:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.813+0000 7f7fcdc93700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fc8069380 0x7f7fc80c7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.813+0000 7f7fcdc93700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7fc80c8750 0x7f7fc80883f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.813+0000 7f7fcdc93700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7fc806eb10 0x7f7fc8088930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.813+0000 7f7fcdc93700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc816a840 con 0x7f7fc80c8750 2024-01-27T02:32:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.813+0000 7f7fcdc93700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc8088e70 con 0x7f7fc806eb10 2024-01-27T02:32:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.813+0000 7f7fcdc93700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc806e880 con 0x7f7fc8069380 2024-01-27T02:32:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.814+0000 7f7fc7fff700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7fc806eb10 0x7f7fc8088930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.814+0000 7f7fc7fff700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7fc806eb10 0x7f7fc8088930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:56472/0 (socket says 172.21.15.7:56472) 2024-01-27T02:32:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.814+0000 7f7fc7fff700 1 -- 172.21.15.7:0/2029859489 learned_addr learned my addr 172.21.15.7:0/2029859489 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.815+0000 7f7fc6ffd700 1 --2- 172.21.15.7:0/2029859489 >> v2:172.21.15.7:3300/0 conn(0x7f7fc80c8750 0x7f7fc80883f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.816+0000 7f7fc77fe700 1 --2- 172.21.15.7:0/2029859489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fc8069380 0x7f7fc80c7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.816+0000 7f85e75f6700 1 -- 172.21.15.7:0/1239106719 shutdown_connections 2024-01-27T02:32:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.816+0000 7f85e75f6700 1 --2- 172.21.15.7:0/1239106719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85cc0053c0 0x7f85cc00ba60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.816+0000 7f85e75f6700 1 -- 172.21.15.7:0/1239106719 >> 172.21.15.7:0/1239106719 conn(0x7f85e0063580 msgr2=0x7f85e0063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.817+0000 7fcd7ff38700 1 -- 172.21.15.7:0/2864623850 shutdown_connections 2024-01-27T02:32:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.817+0000 7fcd7ff38700 1 --2- 172.21.15.7:0/2864623850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd780c9aa0 0x7fcd780aef70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.817+0000 7fcd7ff38700 1 -- 172.21.15.7:0/2864623850 >> 172.21.15.7:0/2864623850 conn(0x7fcd78063580 msgr2=0x7fcd78063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.817+0000 7fedcd2f9700 1 --2- 172.21.15.7:0/2335135985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fedb4003ae0 0x7fedb4003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.817+0000 7f7fc7fff700 1 -- 172.21.15.7:0/2029859489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fc8069380 msgr2=0x7f7fc80c7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.817+0000 7f7fc7fff700 1 --2- 172.21.15.7:0/2029859489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fc8069380 0x7f7fc80c7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.817+0000 7f7fc7fff700 1 -- 172.21.15.7:0/2029859489 >> v2:172.21.15.7:3300/0 conn(0x7f7fc80c8750 msgr2=0x7f7fc80883f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.817+0000 7f7fc7fff700 1 --2- 172.21.15.7:0/2029859489 >> v2:172.21.15.7:3300/0 conn(0x7f7fc80c8750 0x7f7fc80883f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.817+0000 7f7fc7fff700 1 -- 172.21.15.7:0/2029859489 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc80c76b0 con 0x7f7fc806eb10 2024-01-27T02:32:17.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.817+0000 7f7fc7fff700 1 --2- 172.21.15.7:0/2029859489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fc806eb10 0x7f7fc8088930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7fc806a780 tx=0x7f7fc0007820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=813d7a9fd189f14 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.817+0000 7fedccaf8700 1 --2- 172.21.15.7:0/2335135985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedb4004810 0x7fedb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.817+0000 7f95ee395700 1 Processor -- start 2024-01-27T02:32:17.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.817+0000 7fcd7ff38700 1 -- 172.21.15.7:0/2864623850 shutdown_connections 2024-01-27T02:32:17.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.817+0000 7f7fc67fc700 1 -- 172.21.15.7:0/2029859489 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc000c040 con 0x7f7fc806eb10 2024-01-27T02:32:17.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.817+0000 7f85e75f6700 1 -- 172.21.15.7:0/1239106719 shutdown_connections 2024-01-27T02:32:17.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.818+0000 7fedcdafa700 1 --2- 172.21.15.7:0/2335135985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedb4005310 0x7fedb400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.818+0000 7fedcd2f9700 1 -- 172.21.15.7:0/2335135985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedb4004810 msgr2=0x7fedb4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.818+0000 7fedcd2f9700 1 --2- 172.21.15.7:0/2335135985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedb4004810 0x7fedb4004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.818+0000 7fedcd2f9700 1 -- 172.21.15.7:0/2335135985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedb4005310 msgr2=0x7fedb400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.818+0000 7fedcd2f9700 1 --2- 172.21.15.7:0/2335135985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedb4005310 0x7fedb400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.818+0000 7fedcd2f9700 1 -- 172.21.15.7:0/2335135985 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc80cc6c0 con 0x7fedb4003ae0 2024-01-27T02:32:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.818+0000 7fedcdafa700 1 --2- 172.21.15.7:0/2335135985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedb4005310 0x7fedb400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:32:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.818+0000 7fedcd2f9700 1 --2- 172.21.15.7:0/2335135985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fedb4003ae0 0x7fedb4003ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fedc806dba0 tx=0x7fedc4006ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b11aa889d60267e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.818+0000 7f85e75f6700 1 -- 172.21.15.7:0/1239106719 wait complete. 2024-01-27T02:32:17.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.817+0000 7f7fc67fc700 1 -- 172.21.15.7:0/2029859489 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc0008c70 con 0x7f7fc806eb10 2024-01-27T02:32:17.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.818+0000 7f7fc67fc700 1 -- 172.21.15.7:0/2029859489 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0012430 con 0x7f7fc806eb10 2024-01-27T02:32:17.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.819+0000 7f7fcdc93700 1 -- 172.21.15.7:0/2029859489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fc806eb10 msgr2=0x7f7fc8088930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.819+0000 7f7fcdc93700 1 --2- 172.21.15.7:0/2029859489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fc806eb10 0x7f7fc8088930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7fc806a780 tx=0x7f7fc0007820 comp rx=0 tx=0).stop 2024-01-27T02:32:17.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.819+0000 7f7fcdc93700 1 -- 172.21.15.7:0/2029859489 shutdown_connections 2024-01-27T02:32:17.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.819+0000 7f7fcdc93700 1 --2- 172.21.15.7:0/2029859489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fc806eb10 0x7f7fc8088930 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.819+0000 7f7fcdc93700 1 --2- 172.21.15.7:0/2029859489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fc8069380 0x7f7fc80c7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.819+0000 7f7fcdc93700 1 --2- 172.21.15.7:0/2029859489 >> v2:172.21.15.7:3300/0 conn(0x7f7fc80c8750 0x7f7fc80883f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.819+0000 7f7fcdc93700 1 -- 172.21.15.7:0/2029859489 >> 172.21.15.7:0/2029859489 conn(0x7f7fc8064090 msgr2=0x7f7fc80cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.819+0000 7f7fcdc93700 1 -- 172.21.15.7:0/2029859489 shutdown_connections 2024-01-27T02:32:17.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.818+0000 7fedbffff700 1 -- 172.21.15.7:0/2335135985 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc4011400 con 0x7fedb4003ae0 2024-01-27T02:32:17.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.818+0000 7fedbffff700 1 -- 172.21.15.7:0/2335135985 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc4011560 con 0x7fedb4003ae0 2024-01-27T02:32:17.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.818+0000 7fedbffff700 1 -- 172.21.15.7:0/2335135985 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc4011920 con 0x7fedb4003ae0 2024-01-27T02:32:17.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.819+0000 7fedcf55d700 1 -- 172.21.15.7:0/2335135985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fedb4003ae0 msgr2=0x7fedb4003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.819+0000 7fedcf55d700 1 --2- 172.21.15.7:0/2335135985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fedb4003ae0 0x7fedb4003ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fedc806dba0 tx=0x7fedc4006ef0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.819+0000 7fedcdafa700 1 -- 172.21.15.7:0/2335135985 reap_dead start 2024-01-27T02:32:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.818+0000 7f95ee395700 1 -- start start 2024-01-27T02:32:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.819+0000 7f95ee395700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d0003a60 0x7f95e80cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.819+0000 7f95ee395700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e80abc70 0x7f95e8152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.819+0000 7f95ee395700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e80ca970 0x7f95e80cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.819+0000 7f95ee395700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e80a8770 con 0x7f95d0003a60 2024-01-27T02:32:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.819+0000 7f95ee395700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e80cb310 con 0x7f95e80abc70 2024-01-27T02:32:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.819+0000 7f95ee395700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e8166500 con 0x7f95e80ca970 2024-01-27T02:32:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.819+0000 7fedcf55d700 1 -- 172.21.15.7:0/2335135985 shutdown_connections 2024-01-27T02:32:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.819+0000 7fedcf55d700 1 -- 172.21.15.7:0/2335135985 >> 172.21.15.7:0/2335135985 conn(0x7fedc80cdc30 msgr2=0x7fedc80ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.820+0000 7f95ec932700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e80ca970 0x7f95e80cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.820+0000 7f95e77fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e80abc70 0x7f95e8152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.820+0000 7f95ec932700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e80ca970 0x7f95e80cadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:54192/0 (socket says 172.21.15.7:54192) 2024-01-27T02:32:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.820+0000 7f95ec932700 1 -- 172.21.15.7:0/3279102483 learned_addr learned my addr 172.21.15.7:0/3279102483 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.818+0000 7fcd7ff38700 1 -- 172.21.15.7:0/2864623850 wait complete. 2024-01-27T02:32:17.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.820+0000 7f85e75f6700 1 Processor -- start 2024-01-27T02:32:17.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.820+0000 7f85e75f6700 1 -- start start 2024-01-27T02:32:17.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.820+0000 7f95ec932700 1 -- 172.21.15.7:0/3279102483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e80abc70 msgr2=0x7f95e8152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.820+0000 7f95ec932700 1 --2- 172.21.15.7:0/3279102483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e80abc70 0x7f95e8152c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.820+0000 7f95ec932700 1 -- 172.21.15.7:0/3279102483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d0003a60 msgr2=0x7f95e80cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.820+0000 7f95ec932700 1 --2- 172.21.15.7:0/3279102483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d0003a60 0x7f95e80cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.820+0000 7f95ec932700 1 -- 172.21.15.7:0/3279102483 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e80cd0b0 con 0x7f95e80ca970 2024-01-27T02:32:17.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.820+0000 7fcd7ff38700 1 Processor -- start 2024-01-27T02:32:17.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.819+0000 7f7fcdc93700 1 -- 172.21.15.7:0/2029859489 wait complete. 2024-01-27T02:32:17.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.821+0000 7fedcf55d700 1 -- 172.21.15.7:0/2335135985 shutdown_connections 2024-01-27T02:32:17.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.821+0000 7f7fcdc93700 1 Processor -- start 2024-01-27T02:32:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.821+0000 7f95ec932700 1 --2- 172.21.15.7:0/3279102483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e80ca970 0x7f95e80cadd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f95e806def0 tx=0x7f95d8008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.821+0000 7f95cffff700 1 -- 172.21.15.7:0/3279102483 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d80116a0 con 0x7f95e80ca970 2024-01-27T02:32:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.821+0000 7f95cffff700 1 -- 172.21.15.7:0/3279102483 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d8011800 con 0x7f95e80ca970 2024-01-27T02:32:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.821+0000 7f95cffff700 1 -- 172.21.15.7:0/3279102483 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d80119b0 con 0x7f95e80ca970 2024-01-27T02:32:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.821+0000 7fcd7ff38700 1 -- start start 2024-01-27T02:32:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7fcd7ff38700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd7806d360 0x7fcd780c3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7fcd7ff38700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd780c96d0 0x7fcd780c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7fcd7ff38700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd780c9aa0 0x7fcd7806a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7fcd7ff38700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd780c6d10 con 0x7fcd780c96d0 2024-01-27T02:32:17.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7fcd7ff38700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7814ba90 con 0x7fcd7806d360 2024-01-27T02:32:17.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7fcd7ff38700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7815d120 con 0x7fcd780c9aa0 2024-01-27T02:32:17.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7fcd7e4d5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd780c9aa0 0x7fcd7806a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7fcd7e4d5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd780c9aa0 0x7fcd7806a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:54196/0 (socket says 172.21.15.7:54196) 2024-01-27T02:32:17.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7fcd7e4d5700 1 -- 172.21.15.7:0/3368010339 learned_addr learned my addr 172.21.15.7:0/3368010339 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.823+0000 7fcd7dcd4700 1 --2- 172.21.15.7:0/3368010339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd7806d360 0x7fcd780c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.821+0000 7f7fcdc93700 1 -- start start 2024-01-27T02:32:17.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7f7fcdc93700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc8069380 0x7f7fc8162e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7f7fcdc93700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fc806eb10 0x7f7fc80ac1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7f7fcdc93700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fc80c8750 0x7f7fc80ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7f7fcdc93700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc80ab7c0 con 0x7f7fc8069380 2024-01-27T02:32:17.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7f7fcdc93700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc80ac710 con 0x7f7fc80c8750 2024-01-27T02:32:17.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7f7fcdc93700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc80cae00 con 0x7f7fc806eb10 2024-01-27T02:32:17.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7f7fc6ffd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fc806eb10 0x7f7fc80ac1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7f85e75f6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85cc0053c0 0x7f85e0176e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7f85e75f6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85e015b880 0x7f85e00c4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7f85e75f6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85e015dbb0 0x7f85e00c1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7f85e75f6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e015ec40 con 0x7f85e015dbb0 2024-01-27T02:32:17.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7f85e75f6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e015c080 con 0x7f85e015b880 2024-01-27T02:32:17.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7f85e75f6700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e00c5a80 con 0x7f85cc0053c0 2024-01-27T02:32:17.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7f85e4b91700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85e015b880 0x7f85e00c4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7f85e4b91700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85e015b880 0x7f85e00c4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:56506/0 (socket says 172.21.15.7:56506) 2024-01-27T02:32:17.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7f85e4b91700 1 -- 172.21.15.7:0/713752609 learned_addr learned my addr 172.21.15.7:0/713752609 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.822+0000 7f85e5392700 1 --2- 172.21.15.7:0/713752609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85cc0053c0 0x7f85e0176e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.823+0000 7f85e4b91700 1 -- 172.21.15.7:0/713752609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85cc0053c0 msgr2=0x7f85e0176e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.823+0000 7f85e4b91700 1 --2- 172.21.15.7:0/713752609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85cc0053c0 0x7f85e0176e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.823+0000 7f85e4b91700 1 -- 172.21.15.7:0/713752609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85e015dbb0 msgr2=0x7f85e00c1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.823+0000 7f85e4b91700 1 --2- 172.21.15.7:0/713752609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85e015dbb0 0x7f85e00c1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.823+0000 7f85e4b91700 1 -- 172.21.15.7:0/713752609 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e00c3a90 con 0x7f85e015b880 2024-01-27T02:32:17.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.823+0000 7f85e4b91700 1 --2- 172.21.15.7:0/713752609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85e015b880 0x7f85e00c4200 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f85dc007390 tx=0x7f85dc001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.821+0000 7fedcf55d700 1 -- 172.21.15.7:0/2335135985 wait complete. 2024-01-27T02:32:17.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.823+0000 7fedcf55d700 1 Processor -- start 2024-01-27T02:32:17.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.823+0000 7f7fc6ffd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fc806eb10 0x7f7fc80ac1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:54216/0 (socket says 172.21.15.7:54216) 2024-01-27T02:32:17.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.823+0000 7f7fc6ffd700 1 -- 172.21.15.7:0/3474662902 learned_addr learned my addr 172.21.15.7:0/3474662902 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.823+0000 7f7fc7fff700 1 --2- 172.21.15.7:0/3474662902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fc80c8750 0x7f7fc80ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.823+0000 7f7fc77fe700 1 --2- 172.21.15.7:0/3474662902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc8069380 0x7f7fc8162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.823+0000 7fcd7d4d3700 1 --2- 172.21.15.7:0/3368010339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd780c96d0 0x7fcd780c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.824+0000 7fcd7e4d5700 1 -- 172.21.15.7:0/3368010339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd7806d360 msgr2=0x7fcd780c3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.824+0000 7fcd7e4d5700 1 --2- 172.21.15.7:0/3368010339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd7806d360 0x7fcd780c3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.824+0000 7fcd7e4d5700 1 -- 172.21.15.7:0/3368010339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd780c96d0 msgr2=0x7fcd780c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.824+0000 7fcd7e4d5700 1 --2- 172.21.15.7:0/3368010339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd780c96d0 0x7fcd780c5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.824+0000 7fcd7e4d5700 1 -- 172.21.15.7:0/3368010339 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd68009420 con 0x7fcd780c9aa0 2024-01-27T02:32:17.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.825+0000 7fcd7e4d5700 1 --2- 172.21.15.7:0/3368010339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd780c9aa0 0x7fcd7806a140 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009860 tx=0x7fcd68017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.825+0000 7fcd6dffb700 1 -- 172.21.15.7:0/3368010339 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68010e00 con 0x7fcd780c9aa0 2024-01-27T02:32:17.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.825+0000 7fcd6dffb700 1 -- 172.21.15.7:0/3368010339 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6801d430 con 0x7fcd780c9aa0 2024-01-27T02:32:17.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.825+0000 7fcd6dffb700 1 -- 172.21.15.7:0/3368010339 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6801d7f0 con 0x7fcd780c9aa0 2024-01-27T02:32:17.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.825+0000 7fcd7ff38700 1 -- 172.21.15.7:0/3368010339 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd780c6a30 con 0x7fcd780c9aa0 2024-01-27T02:32:17.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.826+0000 7fcd7ff38700 1 -- 172.21.15.7:0/3368010339 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd780c0980 con 0x7fcd780c9aa0 2024-01-27T02:32:17.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.823+0000 7f85d57fa700 1 -- 172.21.15.7:0/713752609 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc00f9d0 con 0x7f85e015b880 2024-01-27T02:32:17.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.823+0000 7f85d57fa700 1 -- 172.21.15.7:0/713752609 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85dc00fb50 con 0x7f85e015b880 2024-01-27T02:32:17.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.823+0000 7f85e75f6700 1 -- 172.21.15.7:0/713752609 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e00c3d70 con 0x7f85e015b880 2024-01-27T02:32:17.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.823+0000 7f85e75f6700 1 -- 172.21.15.7:0/713752609 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e01778c0 con 0x7f85e015b880 2024-01-27T02:32:17.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.826+0000 7f85e75f6700 1 -- 172.21.15.7:0/713752609 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e00c3f00 con 0x7f85e015b880 2024-01-27T02:32:17.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.827+0000 7f85d57fa700 1 -- 172.21.15.7:0/713752609 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc0133b0 con 0x7f85e015b880 2024-01-27T02:32:17.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.827+0000 7f85d57fa700 1 -- 172.21.15.7:0/713752609 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f85dc013720 con 0x7f85e015b880 2024-01-27T02:32:17.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.828+0000 7f85d57fa700 1 --2- 172.21.15.7:0/713752609 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f85c4067130 0x7f85c40695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.828+0000 7f85d57fa700 1 -- 172.21.15.7:0/713752609 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f85dc084470 con 0x7f85e015b880 2024-01-27T02:32:17.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.823+0000 7fedcf55d700 1 -- start start 2024-01-27T02:32:17.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.825+0000 7fedcf55d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fedb4003ae0 0x7fedc8133210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.825+0000 7fedcf55d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedc8133770 0x7fedc8176820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.825+0000 7fedcf55d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedc81366e0 0x7fedc8136b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.825+0000 7fedcf55d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc8176d60 con 0x7fedc8133770 2024-01-27T02:32:17.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.825+0000 7fedcf55d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc80c4ce0 con 0x7fedb4003ae0 2024-01-27T02:32:17.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.825+0000 7fedcf55d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc81370a0 con 0x7fedc81366e0 2024-01-27T02:32:17.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.828+0000 7fedccaf8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedc8133770 0x7fedc8176820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.828+0000 7fedccaf8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedc8133770 0x7fedc8176820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53828/0 (socket says 172.21.15.7:53828) 2024-01-27T02:32:17.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.828+0000 7fedccaf8700 1 -- 172.21.15.7:0/1630129187 learned_addr learned my addr 172.21.15.7:0/1630129187 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.828+0000 7fedccaf8700 1 -- 172.21.15.7:0/1630129187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedc81366e0 msgr2=0x7fedc8136b60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-27T02:32:17.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.828+0000 7fedccaf8700 1 --2- 172.21.15.7:0/1630129187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedc81366e0 0x7fedc8136b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.828+0000 7fedccaf8700 1 -- 172.21.15.7:0/1630129187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fedb4003ae0 msgr2=0x7fedc8133210 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-27T02:32:17.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.828+0000 7fedccaf8700 1 --2- 172.21.15.7:0/1630129187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fedb4003ae0 0x7fedc8133210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.828+0000 7fedccaf8700 1 -- 172.21.15.7:0/1630129187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc80cb390 con 0x7fedc8133770 2024-01-27T02:32:17.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.821+0000 7f95ee395700 1 -- 172.21.15.7:0/3279102483 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e80cd390 con 0x7f95e80ca970 2024-01-27T02:32:17.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.821+0000 7f95ee395700 1 -- 172.21.15.7:0/3279102483 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e80a8400 con 0x7f95e80ca970 2024-01-27T02:32:17.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.827+0000 7f95cffff700 1 -- 172.21.15.7:0/3279102483 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f95d8012460 con 0x7f95e80ca970 2024-01-27T02:32:17.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.827+0000 7f95cffff700 1 --2- 172.21.15.7:0/3279102483 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f95d4067080 0x7f95d4069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.827+0000 7f95cffff700 1 -- 172.21.15.7:0/3279102483 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f95d8016020 con 0x7f95e80ca970 2024-01-27T02:32:17.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.828+0000 7f95e7fff700 1 --2- 172.21.15.7:0/3279102483 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f95d4067080 0x7f95d4069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.823+0000 7f7e0a231700 1 Processor -- start 2024-01-27T02:32:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.824+0000 7f7e0a231700 1 -- start start 2024-01-27T02:32:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.825+0000 7f7e0a231700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7dfc0de400 0x7f7dfc006250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.825+0000 7f7e0a231700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dfc006790 0x7f7dfc034c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.825+0000 7f7e0a231700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dfc0df6d0 con 0x7f7dfc0de030 2024-01-27T02:32:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.825+0000 7f7e0a231700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dfc006c10 con 0x7f7dfc0de400 2024-01-27T02:32:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.825+0000 7f7e0a231700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc04fb40 con 0x7f7dfc006790 2024-01-27T02:32:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.824+0000 7f7fc77fe700 1 -- 172.21.15.7:0/3474662902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fc806eb10 msgr2=0x7f7fc80ac1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.824+0000 7f7fc77fe700 1 --2- 172.21.15.7:0/3474662902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fc806eb10 0x7f7fc80ac1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.824+0000 7f7fc77fe700 1 -- 172.21.15.7:0/3474662902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fc80c8750 msgr2=0x7f7fc80ca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.824+0000 7f7fc77fe700 1 --2- 172.21.15.7:0/3474662902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fc80c8750 0x7f7fc80ca5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.824+0000 7f7fc77fe700 1 -- 172.21.15.7:0/3474662902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc0007ac0 con 0x7f7fc8069380 2024-01-27T02:32:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.825+0000 7f7fc77fe700 1 --2- 172.21.15.7:0/3474662902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc8069380 0x7f7fc8162e00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00db20 tx=0x7f7fbc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.825+0000 7f7faf7fe700 1 -- 172.21.15.7:0/3474662902 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc00a840 con 0x7f7fc8069380 2024-01-27T02:32:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.825+0000 7f7faf7fe700 1 -- 172.21.15.7:0/3474662902 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fbc00a9a0 con 0x7f7fc8069380 2024-01-27T02:32:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.825+0000 7f7faf7fe700 1 -- 172.21.15.7:0/3474662902 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc00ad40 con 0x7f7fc8069380 2024-01-27T02:32:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.825+0000 7f7fcdc93700 1 -- 172.21.15.7:0/3474662902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc80cb0e0 con 0x7f7fc8069380 2024-01-27T02:32:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.825+0000 7f7fcdc93700 1 -- 172.21.15.7:0/3474662902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc80abcb0 con 0x7f7fc8069380 2024-01-27T02:32:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.827+0000 7f7fcdc93700 1 -- 172.21.15.7:0/3474662902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc80abed0 con 0x7f7fc8069380 2024-01-27T02:32:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.828+0000 7f85e5392700 1 --2- 172.21.15.7:0/713752609 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f85c4067130 0x7f85c40695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.828+0000 7f95ee395700 1 -- 172.21.15.7:0/3279102483 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c80052b0 con 0x7f95e80ca970 2024-01-27T02:32:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.829+0000 7f7e09a30700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dfc006790 0x7f7dfc034c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.829+0000 7f7e09a30700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dfc006790 0x7f7dfc034c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:54220/0 (socket says 172.21.15.7:54220) 2024-01-27T02:32:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.829+0000 7f7e09a30700 1 -- 172.21.15.7:0/3743981422 learned_addr learned my addr 172.21.15.7:0/3743981422 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.829+0000 7fedccaf8700 1 --2- 172.21.15.7:0/1630129187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedc8133770 0x7fedc8176820 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fedb800dc00 tx=0x7fedb8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.829+0000 7f95e7fff700 1 --2- 172.21.15.7:0/3279102483 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f95d4067080 0x7f95d4069530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f95e000b010 tx=0x7f95e0016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.829+0000 7f7e09a30700 1 -- 172.21.15.7:0/3743981422 >> v2:172.21.15.7:3300/0 conn(0x7f7dfc0de400 msgr2=0x7f7dfc006250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.829+0000 7f7e09a30700 1 --2- 172.21.15.7:0/3743981422 >> v2:172.21.15.7:3300/0 conn(0x7f7dfc0de400 0x7f7dfc006250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.829+0000 7f7e09a30700 1 -- 172.21.15.7:0/3743981422 >> v1:172.21.15.7:6789/0 conn(0x7f7dfc0de030 legacy=0x7f7dfc050900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.829+0000 7f7e09a30700 1 -- 172.21.15.7:0/3743981422 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dfc04fe20 con 0x7f7dfc006790 2024-01-27T02:32:17.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.829+0000 7fedbd7fa700 1 -- 172.21.15.7:0/1630129187 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8007360 con 0x7fedc8133770 2024-01-27T02:32:17.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.829+0000 7fedbd7fa700 1 -- 172.21.15.7:0/1630129187 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb80074e0 con 0x7fedc8133770 2024-01-27T02:32:17.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.829+0000 7fedbd7fa700 1 -- 172.21.15.7:0/1630129187 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8004830 con 0x7fedc8133770 2024-01-27T02:32:17.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.829+0000 7fcd6dffb700 1 -- 172.21.15.7:0/3368010339 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcd6801dbe0 con 0x7fcd780c9aa0 2024-01-27T02:32:17.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.830+0000 7fcd6dffb700 1 --2- 172.21.15.7:0/3368010339 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcd5c067050 0x7fcd5c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.830+0000 7fcd6dffb700 1 -- 172.21.15.7:0/3368010339 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcd68086450 con 0x7fcd780c9aa0 2024-01-27T02:32:17.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.830+0000 7fcd7dcd4700 1 --2- 172.21.15.7:0/3368010339 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcd5c067050 0x7fcd5c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.831+0000 7f85e5392700 1 --2- 172.21.15.7:0/713752609 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f85c4067130 0x7f85c40695e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f85e004cf90 tx=0x7f85d0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.830+0000 7fedcf55d700 1 -- 172.21.15.7:0/1630129187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc80c5b00 con 0x7fedc8133770 2024-01-27T02:32:17.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.830+0000 7fedcf55d700 1 -- 172.21.15.7:0/1630129187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc80c51d0 con 0x7fedc8133770 2024-01-27T02:32:17.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.830+0000 7f7e09a30700 1 --2- 172.21.15.7:0/3743981422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dfc006790 0x7f7dfc034c50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7e04077b60 tx=0x7f7e04067820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d77f65cc7bee8026 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.831+0000 7fcd7dcd4700 1 --2- 172.21.15.7:0/3368010339 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcd5c067050 0x7fcd5c069500 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcd74007fd0 tx=0x7fcd74009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.831+0000 7fedcf55d700 1 -- 172.21.15.7:0/1630129187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedac0052e0 con 0x7fedc8133770 2024-01-27T02:32:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.831+0000 7fcd7ff38700 1 -- 172.21.15.7:0/3368010339 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd580052b0 con 0x7fcd780c9aa0 2024-01-27T02:32:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.832+0000 7f85d57fa700 1 -- 172.21.15.7:0/713752609 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f85dc051740 con 0x7f85e015b880 2024-01-27T02:32:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.834+0000 7f7dfbfff700 1 -- 172.21.15.7:0/3743981422 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e040679b0 con 0x7f7dfc006790 2024-01-27T02:32:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.834+0000 7f95cffff700 1 -- 172.21.15.7:0/3279102483 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95d8051580 con 0x7f95e80ca970 2024-01-27T02:32:17.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.838+0000 7f7dfbfff700 1 -- 172.21.15.7:0/3743981422 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e04067b30 con 0x7f7dfc006790 2024-01-27T02:32:17.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.838+0000 7f7dfbfff700 1 -- 172.21.15.7:0/3743981422 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04065700 con 0x7f7dfc006790 2024-01-27T02:32:17.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.839+0000 7f7e0a231700 1 -- 172.21.15.7:0/3743981422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dfc006790 msgr2=0x7f7dfc034c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.839+0000 7f7e0a231700 1 --2- 172.21.15.7:0/3743981422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dfc006790 0x7f7dfc034c50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7e04077b60 tx=0x7f7e04067820 comp rx=0 tx=0).stop 2024-01-27T02:32:17.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.839+0000 7fcd6dffb700 1 -- 172.21.15.7:0/3368010339 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd68053750 con 0x7fcd780c9aa0 2024-01-27T02:32:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.839+0000 7f7e0a231700 1 -- 172.21.15.7:0/3743981422 shutdown_connections 2024-01-27T02:32:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.839+0000 7f7e0a231700 1 --2- 172.21.15.7:0/3743981422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dfc006790 0x7f7dfc034c50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.839+0000 7f7e0a231700 1 --2- 172.21.15.7:0/3743981422 >> v2:172.21.15.7:3300/0 conn(0x7f7dfc0de400 0x7f7dfc006250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.839+0000 7f7e0a231700 1 -- 172.21.15.7:0/3743981422 >> 172.21.15.7:0/3743981422 conn(0x7f7dfc055ae0 msgr2=0x7f7dfc051ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.842+0000 7fedbd7fa700 1 -- 172.21.15.7:0/1630129187 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fedb8004c20 con 0x7fedc8133770 2024-01-27T02:32:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.842+0000 7fedbd7fa700 1 --2- 172.21.15.7:0/1630129187 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fedb4068bf0 0x7fedb406b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.842+0000 7fedbd7fa700 1 -- 172.21.15.7:0/1630129187 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fedb80849c0 con 0x7fedc8133770 2024-01-27T02:32:17.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.843+0000 7fedcd2f9700 1 --2- 172.21.15.7:0/1630129187 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fedb4068bf0 0x7fedb406b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.841+0000 7f7faf7fe700 1 -- 172.21.15.7:0/3474662902 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7fbc019690 con 0x7f7fc8069380 2024-01-27T02:32:17.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.841+0000 7f7faf7fe700 1 --2- 172.21.15.7:0/3474662902 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7fb4067080 0x7f7fb4069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.841+0000 7f7faf7fe700 1 -- 172.21.15.7:0/3474662902 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7fbc014070 con 0x7f7fc8069380 2024-01-27T02:32:17.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.844+0000 7fedcd2f9700 1 --2- 172.21.15.7:0/1630129187 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fedb4068bf0 0x7fedb406b0a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fedc806dba0 tx=0x7fedc4010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.844+0000 7f7e0a231700 1 -- 172.21.15.7:0/3743981422 shutdown_connections 2024-01-27T02:32:17.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.844+0000 7f7fc6ffd700 1 --2- 172.21.15.7:0/3474662902 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7fb4067080 0x7f7fb4069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.844+0000 7f7fc6ffd700 1 --2- 172.21.15.7:0/3474662902 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7fb4067080 0x7f7fb4069530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7fc80c3d70 tx=0x7f7fb8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.845+0000 7f7faf7fe700 1 -- 172.21.15.7:0/3474662902 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7fbc051fc0 con 0x7f7fc8069380 2024-01-27T02:32:17.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.844+0000 7f7e0a231700 1 -- 172.21.15.7:0/3743981422 wait complete. 2024-01-27T02:32:17.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.846+0000 7f7e0a231700 1 Processor -- start 2024-01-27T02:32:17.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.846+0000 7f7e0a231700 1 -- start start 2024-01-27T02:32:17.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.848+0000 7f7e0a231700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dfc006790 0x7f7dfc04a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.848+0000 7f7e0a231700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dfc0de030 0x7f7dfc0d60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.848+0000 7f7e0a231700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dfc0de400 0x7f7dfc04e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.848+0000 7f7e0a231700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc0d6600 con 0x7f7dfc0de030 2024-01-27T02:32:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.848+0000 7f7e0a231700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc048250 con 0x7f7dfc006790 2024-01-27T02:32:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.848+0000 7f7e0a231700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc0f6830 con 0x7f7dfc0de400 2024-01-27T02:32:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.848+0000 7f7e0922f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dfc006790 0x7f7dfc04a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.848+0000 7f7e0922f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dfc006790 0x7f7dfc04a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:56536/0 (socket says 172.21.15.7:56536) 2024-01-27T02:32:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.848+0000 7f7e0922f700 1 -- 172.21.15.7:0/180504932 learned_addr learned my addr 172.21.15.7:0/180504932 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.849+0000 7f7e0922f700 1 -- 172.21.15.7:0/180504932 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dfc0de400 msgr2=0x7f7dfc04e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.849+0000 7f7e0922f700 1 --2- 172.21.15.7:0/180504932 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dfc0de400 0x7f7dfc04e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.849+0000 7f7e0922f700 1 -- 172.21.15.7:0/180504932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dfc0de030 msgr2=0x7f7dfc0d60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.849+0000 7f7e0922f700 1 --2- 172.21.15.7:0/180504932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dfc0de030 0x7f7dfc0d60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.849+0000 7f7e0922f700 1 -- 172.21.15.7:0/180504932 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dfc04ae10 con 0x7f7dfc006790 2024-01-27T02:32:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.849+0000 7f7e0922f700 1 --2- 172.21.15.7:0/180504932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dfc006790 0x7f7dfc04a710 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000b600 tx=0x7f7e000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.850+0000 7f7df97fa700 1 -- 172.21.15.7:0/180504932 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0000f510 con 0x7f7dfc006790 2024-01-27T02:32:17.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.850+0000 7f7df97fa700 1 -- 172.21.15.7:0/180504932 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e0000f690 con 0x7f7dfc006790 2024-01-27T02:32:17.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.850+0000 7f7df97fa700 1 -- 172.21.15.7:0/180504932 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0000fa50 con 0x7f7dfc006790 2024-01-27T02:32:17.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.851+0000 7f7e0a231700 1 -- 172.21.15.7:0/180504932 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dfc04b0f0 con 0x7f7dfc006790 2024-01-27T02:32:17.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.851+0000 7f7e0a231700 1 -- 172.21.15.7:0/180504932 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dfc038100 con 0x7f7dfc006790 2024-01-27T02:32:17.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.856+0000 7fedbd7fa700 1 -- 172.21.15.7:0/1630129187 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fedb8051c00 con 0x7fedc8133770 2024-01-27T02:32:17.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.852+0000 7f7e0a231700 1 -- 172.21.15.7:0/180504932 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dfc005710 con 0x7f7dfc006790 2024-01-27T02:32:17.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.854+0000 7f7df97fa700 1 -- 172.21.15.7:0/180504932 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7e0001a400 con 0x7f7dfc006790 2024-01-27T02:32:17.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.854+0000 7f7df97fa700 1 --2- 172.21.15.7:0/180504932 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7de8067080 0x7f7de8069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.854+0000 7f7df97fa700 1 -- 172.21.15.7:0/180504932 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7e00084000 con 0x7f7dfc006790 2024-01-27T02:32:17.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.856+0000 7f7e08a2e700 1 --2- 172.21.15.7:0/180504932 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7de8067080 0x7f7de8069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.859+0000 7f7df97fa700 1 -- 172.21.15.7:0/180504932 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e000512d0 con 0x7f7dfc006790 2024-01-27T02:32:17.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.861+0000 7f7e08a2e700 1 --2- 172.21.15.7:0/180504932 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7de8067080 0x7f7de8069530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00ae50 tx=0x7f7df4006d10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.886+0000 7fceea2e3700 1 Processor -- start 2024-01-27T02:32:17.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.886+0000 7fceea2e3700 1 -- start start 2024-01-27T02:32:17.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.888+0000 7fceea2e3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcee40c8750 0x7fcee40c7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.888+0000 7fceea2e3700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcee4088b00 0x7fcee40af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.888+0000 7fceea2e3700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee406e880 con 0x7fcee4069380 2024-01-27T02:32:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.888+0000 7fceea2e3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee416a840 con 0x7fcee40c8750 2024-01-27T02:32:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.888+0000 7fceea2e3700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee40c76b0 con 0x7fcee4088b00 2024-01-27T02:32:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.889+0000 7fcee37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcee40c8750 0x7fcee40c7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.889+0000 7fcee37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcee40c8750 0x7fcee40c7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53870/0 (socket says 172.21.15.7:53870) 2024-01-27T02:32:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.889+0000 7fcee37fe700 1 -- 172.21.15.7:0/4267009491 learned_addr learned my addr 172.21.15.7:0/4267009491 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.889+0000 7fcee8880700 1 --2- 172.21.15.7:0/4267009491 >> v2:172.21.15.188:3300/0 conn(0x7fcee4088b00 0x7fcee40af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.889+0000 7fcee37fe700 1 -- 172.21.15.7:0/4267009491 >> v2:172.21.15.188:3300/0 conn(0x7fcee4088b00 msgr2=0x7fcee40af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.889+0000 7fcee37fe700 1 --2- 172.21.15.7:0/4267009491 >> v2:172.21.15.188:3300/0 conn(0x7fcee4088b00 0x7fcee40af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.889+0000 7fcee37fe700 1 -- 172.21.15.7:0/4267009491 >> v1:172.21.15.188:6789/0 conn(0x7fcee4069380 legacy=0x7fcee40883f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.889+0000 7fcee37fe700 1 -- 172.21.15.7:0/4267009491 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee40c7990 con 0x7fcee40c8750 2024-01-27T02:32:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.889+0000 7fcee37fe700 1 --2- 172.21.15.7:0/4267009491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcee40c8750 0x7fcee40c7020 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcee406a4a0 tx=0x7fced8008250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c009f8309e252a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.892+0000 7fcee2ffd700 1 -- 172.21.15.7:0/4267009491 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced800e900 con 0x7fcee40c8750 2024-01-27T02:32:17.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.892+0000 7fcee2ffd700 1 -- 172.21.15.7:0/4267009491 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced8010040 con 0x7fcee40c8750 2024-01-27T02:32:17.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.892+0000 7fcee2ffd700 1 -- 172.21.15.7:0/4267009491 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced800ec90 con 0x7fcee40c8750 2024-01-27T02:32:17.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.893+0000 7fceea2e3700 1 -- 172.21.15.7:0/4267009491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcee40c8750 msgr2=0x7fcee40c7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.893+0000 7fceea2e3700 1 --2- 172.21.15.7:0/4267009491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcee40c8750 0x7fcee40c7020 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcee406a4a0 tx=0x7fced8008250 comp rx=0 tx=0).stop 2024-01-27T02:32:17.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.893+0000 7fceea2e3700 1 -- 172.21.15.7:0/4267009491 shutdown_connections 2024-01-27T02:32:17.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.893+0000 7fceea2e3700 1 --2- 172.21.15.7:0/4267009491 >> v2:172.21.15.188:3300/0 conn(0x7fcee4088b00 0x7fcee40af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.893+0000 7fceea2e3700 1 --2- 172.21.15.7:0/4267009491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcee40c8750 0x7fcee40c7020 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.893+0000 7fceea2e3700 1 -- 172.21.15.7:0/4267009491 >> 172.21.15.7:0/4267009491 conn(0x7fcee4064090 msgr2=0x7fcee40cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.893+0000 7fceea2e3700 1 -- 172.21.15.7:0/4267009491 shutdown_connections 2024-01-27T02:32:17.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.893+0000 7fceea2e3700 1 -- 172.21.15.7:0/4267009491 wait complete. 2024-01-27T02:32:17.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.895+0000 7fceea2e3700 1 Processor -- start 2024-01-27T02:32:17.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.896+0000 7fceea2e3700 1 -- start start 2024-01-27T02:32:17.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.897+0000 7fceea2e3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcee4069380 0x7fcee417f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.897+0000 7fceea2e3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcee4088b00 0x7fcee40a9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.897+0000 7fceea2e3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcee40c8750 0x7fcee40aa170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.897+0000 7fceea2e3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee40c31e0 con 0x7fcee4088b00 2024-01-27T02:32:17.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.897+0000 7fceea2e3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee40aa6b0 con 0x7fcee4069380 2024-01-27T02:32:17.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.897+0000 7fceea2e3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee40aa960 con 0x7fcee40c8750 2024-01-27T02:32:17.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.897+0000 7fcee37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcee4088b00 0x7fcee40a9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.897+0000 7fcee37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcee4088b00 0x7fcee40a9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53872/0 (socket says 172.21.15.7:53872) 2024-01-27T02:32:17.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.897+0000 7fcee37fe700 1 -- 172.21.15.7:0/454143739 learned_addr learned my addr 172.21.15.7:0/454143739 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.898+0000 7fcee37fe700 1 -- 172.21.15.7:0/454143739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcee40c8750 msgr2=0x7fcee40aa170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-27T02:32:17.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.898+0000 7fcee37fe700 1 --2- 172.21.15.7:0/454143739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcee40c8750 0x7fcee40aa170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.898+0000 7fcee37fe700 1 -- 172.21.15.7:0/454143739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcee4069380 msgr2=0x7fcee417f3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-27T02:32:17.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.898+0000 7fcee37fe700 1 --2- 172.21.15.7:0/454143739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcee4069380 0x7fcee417f3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.898+0000 7fcee37fe700 1 -- 172.21.15.7:0/454143739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced80084f0 con 0x7fcee4088b00 2024-01-27T02:32:17.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.899+0000 7fcee37fe700 1 --2- 172.21.15.7:0/454143739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcee4088b00 0x7fcee40a9c30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fced8008290 tx=0x7fced801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.899+0000 7fcecbfff700 1 -- 172.21.15.7:0/454143739 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced8019390 con 0x7fcee4088b00 2024-01-27T02:32:17.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.899+0000 7fcecbfff700 1 -- 172.21.15.7:0/454143739 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced80194f0 con 0x7fcee4088b00 2024-01-27T02:32:17.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.899+0000 7fcecbfff700 1 -- 172.21.15.7:0/454143739 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced8016660 con 0x7fcee4088b00 2024-01-27T02:32:17.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.900+0000 7fceea2e3700 1 -- 172.21.15.7:0/454143739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee40aac40 con 0x7fcee4088b00 2024-01-27T02:32:17.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.900+0000 7fceea2e3700 1 -- 172.21.15.7:0/454143739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee40ab200 con 0x7fcee4088b00 2024-01-27T02:32:17.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.896+0000 7fe770c73700 1 Processor -- start 2024-01-27T02:32:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.896+0000 7fe770c73700 1 -- start start 2024-01-27T02:32:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.898+0000 7fe770c73700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe76c06b680 0x7fe76c06e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.898+0000 7fe770c73700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe76c0af8b0 0x7fe76c069500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.898+0000 7fe770c73700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe76c0c79f0 con 0x7fe76c0cc5c0 2024-01-27T02:32:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.898+0000 7fe770c73700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe76c06bae0 con 0x7fe76c06b680 2024-01-27T02:32:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.898+0000 7fe770c73700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe76c06ea00 con 0x7fe76c0af8b0 2024-01-27T02:32:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.899+0000 7fe769d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe76c06b680 0x7fe76c06e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.899+0000 7fe769d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe76c06b680 0x7fe76c06e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53878/0 (socket says 172.21.15.7:53878) 2024-01-27T02:32:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.899+0000 7fe769d9b700 1 -- 172.21.15.7:0/803157182 learned_addr learned my addr 172.21.15.7:0/803157182 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.899+0000 7fe769d9b700 1 -- 172.21.15.7:0/803157182 >> v2:172.21.15.188:3300/0 conn(0x7fe76c0af8b0 msgr2=0x7fe76c069500 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-27T02:32:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.899+0000 7fe769d9b700 1 --2- 172.21.15.7:0/803157182 >> v2:172.21.15.188:3300/0 conn(0x7fe76c0af8b0 0x7fe76c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.899+0000 7fe769d9b700 1 -- 172.21.15.7:0/803157182 >> v1:172.21.15.188:6789/0 conn(0x7fe76c0cc5c0 legacy=0x7fe76c0ab8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-27T02:32:17.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.899+0000 7fe769d9b700 1 -- 172.21.15.7:0/803157182 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76c06ecd0 con 0x7fe76c06b680 2024-01-27T02:32:17.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.900+0000 7fe769d9b700 1 --2- 172.21.15.7:0/803157182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c06b680 0x7fe76c06e4c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0099b0 tx=0x7fe75c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9137b5f2fc3851be server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.903+0000 7fcecbfff700 1 -- 172.21.15.7:0/454143739 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fced80169d0 con 0x7fcee4088b00 2024-01-27T02:32:17.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.903+0000 7fcecbfff700 1 --2- 172.21.15.7:0/454143739 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fced0067080 0x7fced0069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.903+0000 7fcecbfff700 1 -- 172.21.15.7:0/454143739 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fced8085540 con 0x7fcee4088b00 2024-01-27T02:32:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.902+0000 7fe76959a700 1 -- 172.21.15.7:0/803157182 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00b870 con 0x7fe76c06b680 2024-01-27T02:32:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.902+0000 7fe76959a700 1 -- 172.21.15.7:0/803157182 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c00b9d0 con 0x7fe76c06b680 2024-01-27T02:32:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.902+0000 7fe76959a700 1 -- 172.21.15.7:0/803157182 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00bd70 con 0x7fe76c06b680 2024-01-27T02:32:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.904+0000 7fe770c73700 1 -- 172.21.15.7:0/803157182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c06b680 msgr2=0x7fe76c06e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.904+0000 7fe770c73700 1 --2- 172.21.15.7:0/803157182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c06b680 0x7fe76c06e4c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0099b0 tx=0x7fe75c009250 comp rx=0 tx=0).stop 2024-01-27T02:32:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.904+0000 7fe770c73700 1 -- 172.21.15.7:0/803157182 shutdown_connections 2024-01-27T02:32:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.904+0000 7fe770c73700 1 --2- 172.21.15.7:0/803157182 >> v2:172.21.15.188:3300/0 conn(0x7fe76c0af8b0 0x7fe76c069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.904+0000 7fe770c73700 1 --2- 172.21.15.7:0/803157182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c06b680 0x7fe76c06e4c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.904+0000 7fe770c73700 1 -- 172.21.15.7:0/803157182 >> 172.21.15.7:0/803157182 conn(0x7fe76c064090 msgr2=0x7fe76c0cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.904+0000 7fe770c73700 1 -- 172.21.15.7:0/803157182 shutdown_connections 2024-01-27T02:32:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.904+0000 7fe770c73700 1 -- 172.21.15.7:0/803157182 wait complete. 2024-01-27T02:32:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.905+0000 7fceea2e3700 1 -- 172.21.15.7:0/454143739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee404c710 con 0x7fcee4088b00 2024-01-27T02:32:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.906+0000 7fcee3fff700 1 --2- 172.21.15.7:0/454143739 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fced0067080 0x7fced0069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.906+0000 7fe770c73700 1 Processor -- start 2024-01-27T02:32:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.906+0000 7fe770c73700 1 -- start start 2024-01-27T02:32:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.908+0000 7fe770c73700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe76c06b680 0x7fe76c0ad260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.908+0000 7fe770c73700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe76c0af8b0 0x7fe76c0c4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.908+0000 7fe770c73700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c0cc5c0 0x7fe76c0c4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.908+0000 7fe770c73700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c0acdd0 con 0x7fe76c0cc5c0 2024-01-27T02:32:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.908+0000 7fe770c73700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c16ec00 con 0x7fe76c0af8b0 2024-01-27T02:32:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.908+0000 7fe770c73700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76c0ac1c0 con 0x7fe76c06b680 2024-01-27T02:32:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.908+0000 7fe76ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c0cc5c0 0x7fe76c0c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.908+0000 7fe76ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c0cc5c0 0x7fe76c0c4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53894/0 (socket says 172.21.15.7:53894) 2024-01-27T02:32:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.908+0000 7fe76ad9d700 1 -- 172.21.15.7:0/978169685 learned_addr learned my addr 172.21.15.7:0/978169685 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.909+0000 7fe76ad9d700 1 -- 172.21.15.7:0/978169685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe76c06b680 msgr2=0x7fe76c0ad260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-27T02:32:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.909+0000 7fe76ad9d700 1 --2- 172.21.15.7:0/978169685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe76c06b680 0x7fe76c0ad260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.909+0000 7fe76ad9d700 1 -- 172.21.15.7:0/978169685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe76c0af8b0 msgr2=0x7fe76c0c4440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-27T02:32:17.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.909+0000 7fe76ad9d700 1 --2- 172.21.15.7:0/978169685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe76c0af8b0 0x7fe76c0c4440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:17.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.909+0000 7fe76ad9d700 1 -- 172.21.15.7:0/978169685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75c0094f0 con 0x7fe76c0cc5c0 2024-01-27T02:32:17.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.909+0000 7fe76ad9d700 1 --2- 172.21.15.7:0/978169685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c0cc5c0 0x7fe76c0c4980 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe76400bbb0 tx=0x7fe764009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.911+0000 7fcee3fff700 1 --2- 172.21.15.7:0/454143739 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fced0067080 0x7fced0069530 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00ad10 tx=0x7fcedc009370 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.913+0000 7fcecbfff700 1 -- 172.21.15.7:0/454143739 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fced80527c0 con 0x7fcee4088b00 2024-01-27T02:32:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.913+0000 7fe75a7fc700 1 -- 172.21.15.7:0/978169685 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7640092e0 con 0x7fe76c0cc5c0 2024-01-27T02:32:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.913+0000 7fe75a7fc700 1 -- 172.21.15.7:0/978169685 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76400c7a0 con 0x7fe76c0cc5c0 2024-01-27T02:32:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.913+0000 7fe75a7fc700 1 -- 172.21.15.7:0/978169685 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76400cb40 con 0x7fe76c0cc5c0 2024-01-27T02:32:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.913+0000 7fe770c73700 1 -- 172.21.15.7:0/978169685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe76c0ac4a0 con 0x7fe76c0cc5c0 2024-01-27T02:32:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.913+0000 7fe770c73700 1 -- 172.21.15.7:0/978169685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe76c0aca60 con 0x7fe76c0cc5c0 2024-01-27T02:32:17.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.917+0000 7fe75a7fc700 1 -- 172.21.15.7:0/978169685 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe764012070 con 0x7fe76c0cc5c0 2024-01-27T02:32:17.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.918+0000 7fe75a7fc700 1 --2- 172.21.15.7:0/978169685 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe74c067080 0x7fe74c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:17.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.918+0000 7fe75a7fc700 1 -- 172.21.15.7:0/978169685 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe764084290 con 0x7fe76c0cc5c0 2024-01-27T02:32:17.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.918+0000 7fe770c73700 1 -- 172.21.15.7:0/978169685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7500052b0 con 0x7fe76c0cc5c0 2024-01-27T02:32:17.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.919+0000 7fe76a59c700 1 --2- 172.21.15.7:0/978169685 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe74c067080 0x7fe74c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:17.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.928+0000 7fe75a7fc700 1 -- 172.21.15.7:0/978169685 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7640514e0 con 0x7fe76c0cc5c0 2024-01-27T02:32:17.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:17.954+0000 7fe76a59c700 1 --2- 172.21.15.7:0/978169685 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe74c067080 0x7fe74c069530 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe7600096a0 tx=0x7fe760006bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:18.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.264+0000 7fcd7ff38700 1 -- 172.21.15.7:0/3368010339 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcd58005560 con 0x7fcd780c9aa0 2024-01-27T02:32:18.266 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-27T02:32:18.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.264+0000 7f7fcdc93700 1 -- 172.21.15.7:0/3474662902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7fc80cb270 con 0x7f7fc8069380 2024-01-27T02:32:18.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.264+0000 7f7faf7fe700 1 -- 172.21.15.7:0/3474662902 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7fbc0555e0 con 0x7f7fc8069380 2024-01-27T02:32:18.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.265+0000 7f85e75f6700 1 -- 172.21.15.7:0/713752609 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f85e004c710 con 0x7f85e015b880 2024-01-27T02:32:18.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.265+0000 7fcd6dffb700 1 -- 172.21.15.7:0/3368010339 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcd68006cb0 con 0x7fcd780c9aa0 2024-01-27T02:32:18.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.266+0000 7f85d57fa700 1 -- 172.21.15.7:0/713752609 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f85dc054d60 con 0x7f85e015b880 2024-01-27T02:32:18.269 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-27T02:32:18.269 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-27T02:32:18.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.269+0000 7f85e75f6700 1 -- 172.21.15.7:0/713752609 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f85c4067130 msgr2=0x7f85c40695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.269+0000 7f85e75f6700 1 --2- 172.21.15.7:0/713752609 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f85c4067130 0x7f85c40695e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f85e004cf90 tx=0x7f85d0008040 comp rx=0 tx=0).stop 2024-01-27T02:32:18.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.269+0000 7f85e75f6700 1 -- 172.21.15.7:0/713752609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85e015b880 msgr2=0x7f85e00c4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.269+0000 7f85e75f6700 1 --2- 172.21.15.7:0/713752609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85e015b880 0x7f85e00c4200 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f85dc007390 tx=0x7f85dc001e50 comp rx=0 tx=0).stop 2024-01-27T02:32:18.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.269+0000 7f85e75f6700 1 -- 172.21.15.7:0/713752609 shutdown_connections 2024-01-27T02:32:18.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.269+0000 7f85e75f6700 1 --2- 172.21.15.7:0/713752609 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f85c4067130 0x7f85c40695e0 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f85e004cf90 tx=0x7f85d0008040 comp rx=0 tx=0).stop 2024-01-27T02:32:18.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.269+0000 7f85e75f6700 1 --2- 172.21.15.7:0/713752609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85e015dbb0 0x7f85e00c1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.269+0000 7f85e75f6700 1 --2- 172.21.15.7:0/713752609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85cc0053c0 0x7f85e0176e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.269+0000 7f85e75f6700 1 --2- 172.21.15.7:0/713752609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85e015b880 0x7f85e00c4200 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.269+0000 7f85e75f6700 1 -- 172.21.15.7:0/713752609 >> 172.21.15.7:0/713752609 conn(0x7f85e0063580 msgr2=0x7f85e00caba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:18.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.269+0000 7f95ee395700 1 -- 172.21.15.7:0/3279102483 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f95c8005560 con 0x7f95e80ca970 2024-01-27T02:32:18.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.270+0000 7f85e75f6700 1 -- 172.21.15.7:0/713752609 shutdown_connections 2024-01-27T02:32:18.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.270+0000 7fcd7ff38700 1 -- 172.21.15.7:0/3368010339 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcd5c067050 msgr2=0x7fcd5c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.270+0000 7fcd7ff38700 1 --2- 172.21.15.7:0/3368010339 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcd5c067050 0x7fcd5c069500 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcd74007fd0 tx=0x7fcd74009040 comp rx=0 tx=0).stop 2024-01-27T02:32:18.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.270+0000 7fcd7ff38700 1 -- 172.21.15.7:0/3368010339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd780c9aa0 msgr2=0x7fcd7806a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.270+0000 7fcd7ff38700 1 --2- 172.21.15.7:0/3368010339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd780c9aa0 0x7fcd7806a140 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009860 tx=0x7fcd68017000 comp rx=0 tx=0).stop 2024-01-27T02:32:18.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.270+0000 7f7fcdc93700 1 -- 172.21.15.7:0/3474662902 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7fb4067080 msgr2=0x7f7fb4069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.270+0000 7f7fcdc93700 1 --2- 172.21.15.7:0/3474662902 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7fb4067080 0x7f7fb4069530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7fc80c3d70 tx=0x7f7fb8008040 comp rx=0 tx=0).stop 2024-01-27T02:32:18.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.270+0000 7f7fcdc93700 1 -- 172.21.15.7:0/3474662902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc8069380 msgr2=0x7f7fc8162e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.270+0000 7f7fcdc93700 1 --2- 172.21.15.7:0/3474662902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc8069380 0x7f7fc8162e00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00db20 tx=0x7f7fbc008040 comp rx=0 tx=0).stop 2024-01-27T02:32:18.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.270+0000 7fcd7ff38700 1 -- 172.21.15.7:0/3368010339 shutdown_connections 2024-01-27T02:32:18.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.270+0000 7fcd7ff38700 1 --2- 172.21.15.7:0/3368010339 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcd5c067050 0x7fcd5c069500 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.270+0000 7fcd7ff38700 1 --2- 172.21.15.7:0/3368010339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd780c9aa0 0x7fcd7806a140 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.270+0000 7fcd7ff38700 1 --2- 172.21.15.7:0/3368010339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd7806d360 0x7fcd780c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.270+0000 7f85e75f6700 1 -- 172.21.15.7:0/713752609 wait complete. 2024-01-27T02:32:18.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.270+0000 7fcd7ff38700 1 --2- 172.21.15.7:0/3368010339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd780c96d0 0x7fcd780c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.270+0000 7fcd7ff38700 1 -- 172.21.15.7:0/3368010339 >> 172.21.15.7:0/3368010339 conn(0x7fcd78063580 msgr2=0x7fcd780c7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:18.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.270+0000 7fcd7ff38700 1 -- 172.21.15.7:0/3368010339 shutdown_connections 2024-01-27T02:32:18.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.270+0000 7fcd7ff38700 1 -- 172.21.15.7:0/3368010339 wait complete. 2024-01-27T02:32:18.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.271+0000 7f95cffff700 1 -- 172.21.15.7:0/3279102483 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f95d800c030 con 0x7f95e80ca970 2024-01-27T02:32:18.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.271+0000 7f7fcdc93700 1 -- 172.21.15.7:0/3474662902 shutdown_connections 2024-01-27T02:32:18.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.271+0000 7f7fcdc93700 1 --2- 172.21.15.7:0/3474662902 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7fb4067080 0x7f7fb4069530 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.271+0000 7f7fcdc93700 1 --2- 172.21.15.7:0/3474662902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fc80c8750 0x7f7fc80ca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.271+0000 7f7fcdc93700 1 --2- 172.21.15.7:0/3474662902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc8069380 0x7f7fc8162e00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.271+0000 7f7fcdc93700 1 --2- 172.21.15.7:0/3474662902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fc806eb10 0x7f7fc80ac1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.271+0000 7f7fcdc93700 1 -- 172.21.15.7:0/3474662902 >> 172.21.15.7:0/3474662902 conn(0x7f7fc8064090 msgr2=0x7f7fc80cbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:18.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.271+0000 7f7fcdc93700 1 -- 172.21.15.7:0/3474662902 shutdown_connections 2024-01-27T02:32:18.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.271+0000 7f7fcdc93700 1 -- 172.21.15.7:0/3474662902 wait complete. 2024-01-27T02:32:18.282 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-27T02:32:18.286 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-27T02:32:18.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.281+0000 7fedcf55d700 1 -- 172.21.15.7:0/1630129187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fedac005860 con 0x7fedc8133770 2024-01-27T02:32:18.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.282+0000 7fedbd7fa700 1 -- 172.21.15.7:0/1630129187 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fedb8055220 con 0x7fedc8133770 2024-01-27T02:32:18.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.284+0000 7f7e0a231700 1 -- 172.21.15.7:0/180504932 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f7dfc050100 con 0x7f7dfc006790 2024-01-27T02:32:18.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.285+0000 7f95ee395700 1 -- 172.21.15.7:0/3279102483 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f95d4067080 msgr2=0x7f95d4069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.285+0000 7f95ee395700 1 --2- 172.21.15.7:0/3279102483 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f95d4067080 0x7f95d4069530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f95e000b010 tx=0x7f95e0016000 comp rx=0 tx=0).stop 2024-01-27T02:32:18.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.285+0000 7f95ee395700 1 -- 172.21.15.7:0/3279102483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e80ca970 msgr2=0x7f95e80cadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.285+0000 7f95ee395700 1 --2- 172.21.15.7:0/3279102483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e80ca970 0x7f95e80cadd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f95e806def0 tx=0x7f95d8008040 comp rx=0 tx=0).stop 2024-01-27T02:32:18.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.285+0000 7f95ee395700 1 -- 172.21.15.7:0/3279102483 shutdown_connections 2024-01-27T02:32:18.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.285+0000 7f95ee395700 1 --2- 172.21.15.7:0/3279102483 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f95d4067080 0x7f95d4069530 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.285+0000 7f95ee395700 1 --2- 172.21.15.7:0/3279102483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e80ca970 0x7f95e80cadd0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.285+0000 7f95ee395700 1 --2- 172.21.15.7:0/3279102483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d0003a60 0x7f95e80cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.285+0000 7f95ee395700 1 --2- 172.21.15.7:0/3279102483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e80abc70 0x7f95e8152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.285+0000 7f95ee395700 1 -- 172.21.15.7:0/3279102483 >> 172.21.15.7:0/3279102483 conn(0x7f95e80679b0 msgr2=0x7f95e8157060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:18.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.285+0000 7fedcf55d700 1 -- 172.21.15.7:0/1630129187 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fedb4068bf0 msgr2=0x7fedb406b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.285+0000 7fedcf55d700 1 --2- 172.21.15.7:0/1630129187 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fedb4068bf0 0x7fedb406b0a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fedc806dba0 tx=0x7fedc4010000 comp rx=0 tx=0).stop 2024-01-27T02:32:18.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.285+0000 7fedcf55d700 1 -- 172.21.15.7:0/1630129187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedc8133770 msgr2=0x7fedc8176820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.285+0000 7fedcf55d700 1 --2- 172.21.15.7:0/1630129187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedc8133770 0x7fedc8176820 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fedb800dc00 tx=0x7fedb8009040 comp rx=0 tx=0).stop 2024-01-27T02:32:18.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.285+0000 7f95ee395700 1 -- 172.21.15.7:0/3279102483 shutdown_connections 2024-01-27T02:32:18.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.285+0000 7f95ee395700 1 -- 172.21.15.7:0/3279102483 wait complete. 2024-01-27T02:32:18.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.286+0000 7fedcf55d700 1 -- 172.21.15.7:0/1630129187 shutdown_connections 2024-01-27T02:32:18.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.286+0000 7fedcf55d700 1 --2- 172.21.15.7:0/1630129187 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fedb4068bf0 0x7fedb406b0a0 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fedc806dba0 tx=0x7fedc4010000 comp rx=0 tx=0).stop 2024-01-27T02:32:18.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.286+0000 7fedcf55d700 1 --2- 172.21.15.7:0/1630129187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedc81366e0 0x7fedc8136b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.286+0000 7fedcf55d700 1 --2- 172.21.15.7:0/1630129187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fedb4003ae0 0x7fedc8133210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.286+0000 7fedcf55d700 1 --2- 172.21.15.7:0/1630129187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedc8133770 0x7fedc8176820 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.286+0000 7fedcf55d700 1 -- 172.21.15.7:0/1630129187 >> 172.21.15.7:0/1630129187 conn(0x7fedc80cdc30 msgr2=0x7fedc80b2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:18.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.289+0000 7f7df97fa700 1 -- 172.21.15.7:0/180504932 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7e000548f0 con 0x7f7dfc006790 2024-01-27T02:32:18.294 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-27T02:32:18.294 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-01-27T02:32:18.295 DEBUG:teuthology.parallel:result is None 2024-01-27T02:32:18.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.294+0000 7f7e0a231700 1 -- 172.21.15.7:0/180504932 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7de8067080 msgr2=0x7f7de8069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.294+0000 7f7e0a231700 1 --2- 172.21.15.7:0/180504932 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7de8067080 0x7f7de8069530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00ae50 tx=0x7f7df4006d10 comp rx=0 tx=0).stop 2024-01-27T02:32:18.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.294+0000 7f7e0a231700 1 -- 172.21.15.7:0/180504932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dfc006790 msgr2=0x7f7dfc04a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.294+0000 7f7e0a231700 1 --2- 172.21.15.7:0/180504932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dfc006790 0x7f7dfc04a710 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000b600 tx=0x7f7e000091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.294+0000 7fedcf55d700 1 -- 172.21.15.7:0/1630129187 shutdown_connections 2024-01-27T02:32:18.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.294+0000 7f7e0a231700 1 -- 172.21.15.7:0/180504932 shutdown_connections 2024-01-27T02:32:18.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.294+0000 7f7e0a231700 1 --2- 172.21.15.7:0/180504932 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7de8067080 0x7f7de8069530 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.294+0000 7f7e0a231700 1 --2- 172.21.15.7:0/180504932 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dfc0de400 0x7f7dfc04e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.294+0000 7f7e0a231700 1 --2- 172.21.15.7:0/180504932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dfc006790 0x7f7dfc04a710 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000b600 tx=0x7f7e000091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.294+0000 7f7e0a231700 1 --2- 172.21.15.7:0/180504932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dfc0de030 0x7f7dfc0d60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.294+0000 7f7e0a231700 1 -- 172.21.15.7:0/180504932 >> 172.21.15.7:0/180504932 conn(0x7f7dfc055ae0 msgr2=0x7f7dfc0e0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:18.298 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-01-27T02:32:18.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.296+0000 7fedcf55d700 1 -- 172.21.15.7:0/1630129187 wait complete. 2024-01-27T02:32:18.298 DEBUG:teuthology.parallel:result is None 2024-01-27T02:32:18.300 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-01-27T02:32:18.300 DEBUG:teuthology.parallel:result is None 2024-01-27T02:32:18.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.301+0000 7f7e0a231700 1 -- 172.21.15.7:0/180504932 shutdown_connections 2024-01-27T02:32:18.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.301+0000 7f7e0a231700 1 -- 172.21.15.7:0/180504932 wait complete. 2024-01-27T02:32:18.305 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-01-27T02:32:18.305 DEBUG:teuthology.parallel:result is None 2024-01-27T02:32:18.318 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-01-27T02:32:18.319 DEBUG:teuthology.parallel:result is None 2024-01-27T02:32:18.319 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-01-27T02:32:18.319 DEBUG:teuthology.parallel:result is None 2024-01-27T02:32:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.343+0000 7fceea2e3700 1 -- 172.21.15.7:0/454143739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcee40c7c70 con 0x7fcee4088b00 2024-01-27T02:32:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.344+0000 7fcecbfff700 1 -- 172.21.15.7:0/454143739 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fced8026090 con 0x7fcee4088b00 2024-01-27T02:32:18.345 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-27T02:32:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.346+0000 7fceea2e3700 1 -- 172.21.15.7:0/454143739 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fced0067080 msgr2=0x7fced0069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.346+0000 7fceea2e3700 1 --2- 172.21.15.7:0/454143739 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fced0067080 0x7fced0069530 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00ad10 tx=0x7fcedc009370 comp rx=0 tx=0).stop 2024-01-27T02:32:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.346+0000 7fceea2e3700 1 -- 172.21.15.7:0/454143739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcee4088b00 msgr2=0x7fcee40a9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.346+0000 7fceea2e3700 1 --2- 172.21.15.7:0/454143739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcee4088b00 0x7fcee40a9c30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fced8008290 tx=0x7fced801a000 comp rx=0 tx=0).stop 2024-01-27T02:32:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.346+0000 7fceea2e3700 1 -- 172.21.15.7:0/454143739 shutdown_connections 2024-01-27T02:32:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.346+0000 7fceea2e3700 1 --2- 172.21.15.7:0/454143739 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fced0067080 0x7fced0069530 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.346+0000 7fceea2e3700 1 --2- 172.21.15.7:0/454143739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcee40c8750 0x7fcee40aa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.346+0000 7fceea2e3700 1 --2- 172.21.15.7:0/454143739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcee4069380 0x7fcee417f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.346+0000 7fceea2e3700 1 --2- 172.21.15.7:0/454143739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcee4088b00 0x7fcee40a9c30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.346+0000 7fceea2e3700 1 -- 172.21.15.7:0/454143739 >> 172.21.15.7:0/454143739 conn(0x7fcee4064090 msgr2=0x7fcee4169d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.346+0000 7fceea2e3700 1 -- 172.21.15.7:0/454143739 shutdown_connections 2024-01-27T02:32:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.346+0000 7fceea2e3700 1 -- 172.21.15.7:0/454143739 wait complete. 2024-01-27T02:32:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.350+0000 7fe770c73700 1 -- 172.21.15.7:0/978169685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe750005560 con 0x7fe76c0cc5c0 2024-01-27T02:32:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.350+0000 7fe75a7fc700 1 -- 172.21.15.7:0/978169685 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe764054b00 con 0x7fe76c0cc5c0 2024-01-27T02:32:18.352 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-27T02:32:18.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.352+0000 7fe770c73700 1 -- 172.21.15.7:0/978169685 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe74c067080 msgr2=0x7fe74c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.352+0000 7fe770c73700 1 --2- 172.21.15.7:0/978169685 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe74c067080 0x7fe74c069530 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe7600096a0 tx=0x7fe760006bc0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.353+0000 7fe770c73700 1 -- 172.21.15.7:0/978169685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c0cc5c0 msgr2=0x7fe76c0c4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.353+0000 7fe770c73700 1 --2- 172.21.15.7:0/978169685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c0cc5c0 0x7fe76c0c4980 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe76400bbb0 tx=0x7fe764009840 comp rx=0 tx=0).stop 2024-01-27T02:32:18.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.353+0000 7fe770c73700 1 -- 172.21.15.7:0/978169685 shutdown_connections 2024-01-27T02:32:18.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.353+0000 7fe770c73700 1 --2- 172.21.15.7:0/978169685 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe74c067080 0x7fe74c069530 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.353+0000 7fe770c73700 1 --2- 172.21.15.7:0/978169685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe76c0cc5c0 0x7fe76c0c4980 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.353+0000 7fe770c73700 1 --2- 172.21.15.7:0/978169685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe76c06b680 0x7fe76c0ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.353+0000 7fe770c73700 1 --2- 172.21.15.7:0/978169685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe76c0af8b0 0x7fe76c0c4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.353+0000 7fe770c73700 1 -- 172.21.15.7:0/978169685 >> 172.21.15.7:0/978169685 conn(0x7fe76c064090 msgr2=0x7fe76c0c7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:18.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.353+0000 7fe770c73700 1 -- 172.21.15.7:0/978169685 shutdown_connections 2024-01-27T02:32:18.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.353+0000 7fe770c73700 1 -- 172.21.15.7:0/978169685 wait complete. 2024-01-27T02:32:18.365 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-01-27T02:32:18.365 DEBUG:teuthology.parallel:result is None 2024-01-27T02:32:18.370 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-01-27T02:32:18.370 DEBUG:teuthology.parallel:result is None 2024-01-27T02:32:18.370 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-27T02:32:18.370 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:32:18.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.543+0000 7f2ee1ccf700 1 Processor -- start 2024-01-27T02:32:18.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.543+0000 7f2ee1ccf700 1 -- start start 2024-01-27T02:32:18.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.544+0000 7f2ee1ccf700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2edc0c3ae0 0x7f2edc0cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:18.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.544+0000 7f2ee1ccf700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2edc0ca020 con 0x7f2edc157ca0 2024-01-27T02:32:18.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.544+0000 7f2ee1ccf700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2edc0c5210 con 0x7f2edc158070 2024-01-27T02:32:18.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.544+0000 7f2ee1ccf700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc0c54f0 con 0x7f2edc0c3ae0 2024-01-27T02:32:18.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.544+0000 7f2edb7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2edc157ca0 0x7f2edc0c4a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51454/0 (socket says 172.21.15.7:51454) 2024-01-27T02:32:18.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.544+0000 7f2edb7fe700 1 -- 172.21.15.7:0/266456476 learned_addr learned my addr 172.21.15.7:0/266456476 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:18.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.545+0000 7f2edaffd700 1 --2- 172.21.15.7:0/266456476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2edc0c3ae0 0x7f2edc0cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:18.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.545+0000 7f2eda7fc700 1 -- 172.21.15.7:0/266456476 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076189483 0 0) 0x7f2edc0ca020 con 0x7f2edc157ca0 2024-01-27T02:32:18.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.545+0000 7f2eda7fc700 1 -- 172.21.15.7:0/266456476 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ec8003570 con 0x7f2edc157ca0 2024-01-27T02:32:18.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.545+0000 7f2eda7fc700 1 -- 172.21.15.7:0/266456476 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 908337497 0 0) 0x7f2edc0c5210 con 0x7f2edc158070 2024-01-27T02:32:18.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.545+0000 7f2eda7fc700 1 -- 172.21.15.7:0/266456476 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2edc0ca020 con 0x7f2edc158070 2024-01-27T02:32:18.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.545+0000 7f2edaffd700 1 -- 172.21.15.7:0/266456476 >> v1:172.21.15.188:6789/0 conn(0x7f2edc158070 legacy=0x7f2edc0ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.545+0000 7f2edaffd700 1 -- 172.21.15.7:0/266456476 >> v1:172.21.15.7:6789/0 conn(0x7f2edc157ca0 legacy=0x7f2edc0c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.545+0000 7f2edaffd700 1 -- 172.21.15.7:0/266456476 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2edc0c57d0 con 0x7f2edc0c3ae0 2024-01-27T02:32:18.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.545+0000 7f2edaffd700 1 --2- 172.21.15.7:0/266456476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2edc0c3ae0 0x7f2edc0cd4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009910 tx=0x7f2ec40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec2f748b0e5e9595 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:18.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.545+0000 7f2eda7fc700 1 -- 172.21.15.7:0/266456476 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4011630 con 0x7f2edc0c3ae0 2024-01-27T02:32:18.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.545+0000 7f2eda7fc700 1 -- 172.21.15.7:0/266456476 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec4011790 con 0x7f2edc0c3ae0 2024-01-27T02:32:18.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.545+0000 7f2eda7fc700 1 -- 172.21.15.7:0/266456476 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4011b30 con 0x7f2edc0c3ae0 2024-01-27T02:32:18.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.546+0000 7f2ee1ccf700 1 -- 172.21.15.7:0/266456476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2edc0c3ae0 msgr2=0x7f2edc0cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.546+0000 7f2ee1ccf700 1 --2- 172.21.15.7:0/266456476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2edc0c3ae0 0x7f2edc0cd4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009910 tx=0x7f2ec40091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.546+0000 7f2ee1ccf700 1 -- 172.21.15.7:0/266456476 shutdown_connections 2024-01-27T02:32:18.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.546+0000 7f2ee1ccf700 1 --2- 172.21.15.7:0/266456476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2edc0c3ae0 0x7f2edc0cd4e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.546+0000 7f2ee1ccf700 1 -- 172.21.15.7:0/266456476 >> 172.21.15.7:0/266456476 conn(0x7f2edc098020 msgr2=0x7f2edc0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:18.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.546+0000 7f2ee1ccf700 1 -- 172.21.15.7:0/266456476 shutdown_connections 2024-01-27T02:32:18.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.546+0000 7f2ee1ccf700 1 -- 172.21.15.7:0/266456476 wait complete. 2024-01-27T02:32:18.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.548+0000 7f2ee1ccf700 1 Processor -- start 2024-01-27T02:32:18.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.548+0000 7f2ee1ccf700 1 -- start start 2024-01-27T02:32:18.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.549+0000 7f2ee1ccf700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2edc0c3ae0 0x7f2edc1525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:18.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.549+0000 7f2ee1ccf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2edc157ca0 0x7f2edc155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:18.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.549+0000 7f2ee1ccf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2edc158070 0x7f2edc0b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:18.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.549+0000 7f2ee1ccf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc0c6dc0 con 0x7f2edc157ca0 2024-01-27T02:32:18.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.549+0000 7f2ee1ccf700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc0c61b0 con 0x7f2edc158070 2024-01-27T02:32:18.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.549+0000 7f2ee1ccf700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc0c6490 con 0x7f2edc0c3ae0 2024-01-27T02:32:18.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.549+0000 7f2edaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2edc157ca0 0x7f2edc155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:18.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.549+0000 7f2edaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2edc157ca0 0x7f2edc155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53898/0 (socket says 172.21.15.7:53898) 2024-01-27T02:32:18.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.549+0000 7f2edaffd700 1 -- 172.21.15.7:0/4005703842 learned_addr learned my addr 172.21.15.7:0/4005703842 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:18.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.549+0000 7f2edb7fe700 1 --2- 172.21.15.7:0/4005703842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2edc0c3ae0 0x7f2edc1525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:18.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.549+0000 7f2edbfff700 1 --2- 172.21.15.7:0/4005703842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2edc158070 0x7f2edc0b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:18.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.549+0000 7f2edaffd700 1 -- 172.21.15.7:0/4005703842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2edc0c3ae0 msgr2=0x7f2edc1525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.549+0000 7f2edaffd700 1 --2- 172.21.15.7:0/4005703842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2edc0c3ae0 0x7f2edc1525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.549+0000 7f2edaffd700 1 -- 172.21.15.7:0/4005703842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2edc158070 msgr2=0x7f2edc0b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.549+0000 7f2edaffd700 1 --2- 172.21.15.7:0/4005703842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2edc158070 0x7f2edc0b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.549+0000 7f2edaffd700 1 -- 172.21.15.7:0/4005703842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec4009420 con 0x7f2edc157ca0 2024-01-27T02:32:18.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.549+0000 7f2edb7fe700 1 --2- 172.21.15.7:0/4005703842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2edc0c3ae0 0x7f2edc1525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:32:18.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.550+0000 7f2edaffd700 1 --2- 172.21.15.7:0/4005703842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2edc157ca0 0x7f2edc155a70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009f00 tx=0x7f2ec4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:18.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.550+0000 7f2ed37fe700 1 -- 172.21.15.7:0/4005703842 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec401f070 con 0x7f2edc157ca0 2024-01-27T02:32:18.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.550+0000 7f2ed37fe700 1 -- 172.21.15.7:0/4005703842 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec4000bc0 con 0x7f2edc157ca0 2024-01-27T02:32:18.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.550+0000 7f2ee1ccf700 1 -- 172.21.15.7:0/4005703842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2edc0c6770 con 0x7f2edc157ca0 2024-01-27T02:32:18.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.550+0000 7f2ed37fe700 1 -- 172.21.15.7:0/4005703842 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4025420 con 0x7f2edc157ca0 2024-01-27T02:32:18.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.550+0000 7f2ee1ccf700 1 -- 172.21.15.7:0/4005703842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2edc152b20 con 0x7f2edc157ca0 2024-01-27T02:32:18.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.551+0000 7f2ee1ccf700 1 -- 172.21.15.7:0/4005703842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2edc06f650 con 0x7f2edc157ca0 2024-01-27T02:32:18.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.551+0000 7f2ed37fe700 1 -- 172.21.15.7:0/4005703842 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2ec4023070 con 0x7f2edc157ca0 2024-01-27T02:32:18.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.552+0000 7f2ed37fe700 1 --2- 172.21.15.7:0/4005703842 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2ebc066f30 0x7f2ebc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:18.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.552+0000 7f2ed37fe700 1 -- 172.21.15.7:0/4005703842 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2ec40858f0 con 0x7f2edc157ca0 2024-01-27T02:32:18.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.552+0000 7f2edb7fe700 1 --2- 172.21.15.7:0/4005703842 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2ebc066f30 0x7f2ebc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:18.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.552+0000 7f2edb7fe700 1 --2- 172.21.15.7:0/4005703842 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2ebc066f30 0x7f2ebc0693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc007fd0 tx=0x7f2ecc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:18.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.556+0000 7f2ed37fe700 1 -- 172.21.15.7:0/4005703842 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ec4052b60 con 0x7f2edc157ca0 2024-01-27T02:32:18.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.758+0000 7f2ee1ccf700 1 -- 172.21.15.7:0/4005703842 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2edc04c710 con 0x7f2ebc066f30 2024-01-27T02:32:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.766+0000 7f2ed37fe700 1 -- 172.21.15.7:0/4005703842 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260205 (secure 0 0 0) 0x7f2edc04c710 con 0x7f2ebc066f30 2024-01-27T02:32:18.770 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:18.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.769+0000 7f2ee1ccf700 1 -- 172.21.15.7:0/4005703842 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2ebc066f30 msgr2=0x7f2ebc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.769+0000 7f2ee1ccf700 1 --2- 172.21.15.7:0/4005703842 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2ebc066f30 0x7f2ebc0693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc007fd0 tx=0x7f2ecc009040 comp rx=0 tx=0).stop 2024-01-27T02:32:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.769+0000 7f2ee1ccf700 1 -- 172.21.15.7:0/4005703842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2edc157ca0 msgr2=0x7f2edc155a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.769+0000 7f2ee1ccf700 1 --2- 172.21.15.7:0/4005703842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2edc157ca0 0x7f2edc155a70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009f00 tx=0x7f2ec4017000 comp rx=0 tx=0).stop 2024-01-27T02:32:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.769+0000 7f2ee1ccf700 1 -- 172.21.15.7:0/4005703842 shutdown_connections 2024-01-27T02:32:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.769+0000 7f2ee1ccf700 1 --2- 172.21.15.7:0/4005703842 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2ebc066f30 0x7f2ebc0693e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.769+0000 7f2ee1ccf700 1 --2- 172.21.15.7:0/4005703842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2edc158070 0x7f2edc0b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.769+0000 7f2ee1ccf700 1 --2- 172.21.15.7:0/4005703842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2edc0c3ae0 0x7f2edc1525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.769+0000 7f2ee1ccf700 1 --2- 172.21.15.7:0/4005703842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2edc157ca0 0x7f2edc155a70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.769+0000 7f2ee1ccf700 1 -- 172.21.15.7:0/4005703842 >> 172.21.15.7:0/4005703842 conn(0x7f2edc098020 msgr2=0x7f2edc06be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.769+0000 7f2ee1ccf700 1 -- 172.21.15.7:0/4005703842 shutdown_connections 2024-01-27T02:32:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.769+0000 7f2ee1ccf700 1 -- 172.21.15.7:0/4005703842 wait complete. 2024-01-27T02:32:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:32:18.784 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-01-27T02:32:17.574441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1744,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560755,"statfs":{"total":1600000000,"available":1598213428,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002508"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.645906+0000","last_change":"2024-01-27T02:32:06.645807+0000","last_active":"2024-01-27T02:32:06.645906+0000","last_peered":"2024-01-27T02:32:06.645906+0000","last_clean":"2024-01-27T02:32:06.645906+0000","last_became_active":"2024-01-27T02:32:06.645346+0000","last_became_peered":"2024-01-27T02:32:06.645346+0000","last_unstale":"2024-01-27T02:32:06.645906+0000","last_undegraded":"2024-01-27T02:32:06.645906+0000","last_fullsized":"2024-01-27T02:32:06.645906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:44:24.628811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.639731+0000","last_change":"2024-01-27T02:32:06.639629+0000","last_active":"2024-01-27T02:32:06.639731+0000","last_peered":"2024-01-27T02:32:06.639731+0000","last_clean":"2024-01-27T02:32:06.639731+0000","last_became_active":"2024-01-27T02:32:06.638325+0000","last_became_peered":"2024-01-27T02:32:06.638325+0000","last_unstale":"2024-01-27T02:32:06.639731+0000","last_undegraded":"2024-01-27T02:32:06.639731+0000","last_fullsized":"2024-01-27T02:32:06.639731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:06.985231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.636966+0000","last_change":"2024-01-27T02:32:06.636541+0000","last_active":"2024-01-27T02:32:06.636966+0000","last_peered":"2024-01-27T02:32:06.636966+0000","last_clean":"2024-01-27T02:32:06.636966+0000","last_became_active":"2024-01-27T02:32:06.636201+0000","last_became_peered":"2024-01-27T02:32:06.636201+0000","last_unstale":"2024-01-27T02:32:06.636966+0000","last_undegraded":"2024-01-27T02:32:06.636966+0000","last_fullsized":"2024-01-27T02:32:06.636966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:44:13.569068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.649506+0000","last_change":"2024-01-27T02:32:06.649197+0000","last_active":"2024-01-27T02:32:06.649506+0000","last_peered":"2024-01-27T02:32:06.649506+0000","last_clean":"2024-01-27T02:32:06.649506+0000","last_became_active":"2024-01-27T02:32:06.648503+0000","last_became_peered":"2024-01-27T02:32:06.648503+0000","last_unstale":"2024-01-27T02:32:06.649506+0000","last_undegraded":"2024-01-27T02:32:06.649506+0000","last_fullsized":"2024-01-27T02:32:06.649506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:44:01.051017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.645086+0000","last_change":"2024-01-27T02:32:06.643865+0000","last_active":"2024-01-27T02:32:06.645086+0000","last_peered":"2024-01-27T02:32:06.645086+0000","last_clean":"2024-01-27T02:32:06.645086+0000","last_became_active":"2024-01-27T02:32:06.641009+0000","last_became_peered":"2024-01-27T02:32:06.641009+0000","last_unstale":"2024-01-27T02:32:06.645086+0000","last_undegraded":"2024-01-27T02:32:06.645086+0000","last_fullsized":"2024-01-27T02:32:06.645086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:48:03.053143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.647987+0000","last_change":"2024-01-27T02:32:06.647906+0000","last_active":"2024-01-27T02:32:06.647987+0000","last_peered":"2024-01-27T02:32:06.647987+0000","last_clean":"2024-01-27T02:32:06.647987+0000","last_became_active":"2024-01-27T02:32:06.647577+0000","last_became_peered":"2024-01-27T02:32:06.647577+0000","last_unstale":"2024-01-27T02:32:06.647987+0000","last_undegraded":"2024-01-27T02:32:06.647987+0000","last_fullsized":"2024-01-27T02:32:06.647987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:16:23.225540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.645142+0000","last_change":"2024-01-27T02:32:06.644654+0000","last_active":"2024-01-27T02:32:06.645142+0000","last_peered":"2024-01-27T02:32:06.645142+0000","last_clean":"2024-01-27T02:32:06.645142+0000","last_became_active":"2024-01-27T02:32:06.642941+0000","last_became_peered":"2024-01-27T02:32:06.642941+0000","last_unstale":"2024-01-27T02:32:06.645142+0000","last_undegraded":"2024-01-27T02:32:06.645142+0000","last_fullsized":"2024-01-27T02:32:06.645142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:43:05.599796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642600+0000","last_change":"2024-01-27T02:32:06.642517+0000","last_active":"2024-01-27T02:32:06.642600+0000","last_peered":"2024-01-27T02:32:06.642600+0000","last_clean":"2024-01-27T02:32:06.642600+0000","last_became_active":"2024-01-27T02:32:06.641240+0000","last_became_peered":"2024-01-27T02:32:06.641240+0000","last_unstale":"2024-01-27T02:32:06.642600+0000","last_undegraded":"2024-01-27T02:32:06.642600+0000","last_fullsized":"2024-01-27T02:32:06.642600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:11:54.230178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.650062+0000","last_change":"2024-01-27T02:32:06.649943+0000","last_active":"2024-01-27T02:32:06.650062+0000","last_peered":"2024-01-27T02:32:06.650062+0000","last_clean":"2024-01-27T02:32:06.650062+0000","last_became_active":"2024-01-27T02:32:06.649685+0000","last_became_peered":"2024-01-27T02:32:06.649685+0000","last_unstale":"2024-01-27T02:32:06.650062+0000","last_undegraded":"2024-01-27T02:32:06.650062+0000","last_fullsized":"2024-01-27T02:32:06.650062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:31:12.724216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.646882+0000","last_change":"2024-01-27T02:32:06.646820+0000","last_active":"2024-01-27T02:32:06.646882+0000","last_peered":"2024-01-27T02:32:06.646882+0000","last_clean":"2024-01-27T02:32:06.646882+0000","last_became_active":"2024-01-27T02:32:06.646586+0000","last_became_peered":"2024-01-27T02:32:06.646586+0000","last_unstale":"2024-01-27T02:32:06.646882+0000","last_undegraded":"2024-01-27T02:32:06.646882+0000","last_fullsized":"2024-01-27T02:32:06.646882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:06.985231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.649997+0000","last_change":"2024-01-27T02:32:06.649925+0000","last_active":"2024-01-27T02:32:06.649997+0000","last_peered":"2024-01-27T02:32:06.649997+0000","last_clean":"2024-01-27T02:32:06.649997+0000","last_became_active":"2024-01-27T02:32:06.649685+0000","last_became_peered":"2024-01-27T02:32:06.649685+0000","last_unstale":"2024-01-27T02:32:06.649997+0000","last_undegraded":"2024-01-27T02:32:06.649997+0000","last_fullsized":"2024-01-27T02:32:06.649997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:36:11.295485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642706+0000","last_change":"2024-01-27T02:32:06.642576+0000","last_active":"2024-01-27T02:32:06.642706+0000","last_peered":"2024-01-27T02:32:06.642706+0000","last_clean":"2024-01-27T02:32:06.642706+0000","last_became_active":"2024-01-27T02:32:06.641583+0000","last_became_peered":"2024-01-27T02:32:06.641583+0000","last_unstale":"2024-01-27T02:32:06.642706+0000","last_undegraded":"2024-01-27T02:32:06.642706+0000","last_fullsized":"2024-01-27T02:32:06.642706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:45.494048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642822+0000","last_change":"2024-01-27T02:32:06.642731+0000","last_active":"2024-01-27T02:32:06.642822+0000","last_peered":"2024-01-27T02:32:06.642822+0000","last_clean":"2024-01-27T02:32:06.642822+0000","last_became_active":"2024-01-27T02:32:06.641619+0000","last_became_peered":"2024-01-27T02:32:06.641619+0000","last_unstale":"2024-01-27T02:32:06.642822+0000","last_undegraded":"2024-01-27T02:32:06.642822+0000","last_fullsized":"2024-01-27T02:32:06.642822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:44:24.628811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.635576+0000","last_change":"2024-01-27T02:32:06.635401+0000","last_active":"2024-01-27T02:32:06.635576+0000","last_peered":"2024-01-27T02:32:06.635576+0000","last_clean":"2024-01-27T02:32:06.635576+0000","last_became_active":"2024-01-27T02:32:06.635136+0000","last_became_peered":"2024-01-27T02:32:06.635136+0000","last_unstale":"2024-01-27T02:32:06.635576+0000","last_undegraded":"2024-01-27T02:32:06.635576+0000","last_fullsized":"2024-01-27T02:32:06.635576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:17:03.351766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.639565+0000","last_change":"2024-01-27T02:32:06.639126+0000","last_active":"2024-01-27T02:32:06.639565+0000","last_peered":"2024-01-27T02:32:06.639565+0000","last_clean":"2024-01-27T02:32:06.639565+0000","last_became_active":"2024-01-27T02:32:06.638696+0000","last_became_peered":"2024-01-27T02:32:06.638696+0000","last_unstale":"2024-01-27T02:32:06.639565+0000","last_undegraded":"2024-01-27T02:32:06.639565+0000","last_fullsized":"2024-01-27T02:32:06.639565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:17:03.351766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.647702+0000","last_change":"2024-01-27T02:32:06.646066+0000","last_active":"2024-01-27T02:32:06.647702+0000","last_peered":"2024-01-27T02:32:06.647702+0000","last_clean":"2024-01-27T02:32:06.647702+0000","last_became_active":"2024-01-27T02:32:06.644886+0000","last_became_peered":"2024-01-27T02:32:06.644886+0000","last_unstale":"2024-01-27T02:32:06.647702+0000","last_undegraded":"2024-01-27T02:32:06.647702+0000","last_fullsized":"2024-01-27T02:32:06.647702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:11:54.230178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.649539+0000","last_change":"2024-01-27T02:32:06.649461+0000","last_active":"2024-01-27T02:32:06.649539+0000","last_peered":"2024-01-27T02:32:06.649539+0000","last_clean":"2024-01-27T02:32:06.649539+0000","last_became_active":"2024-01-27T02:32:06.648614+0000","last_became_peered":"2024-01-27T02:32:06.648614+0000","last_unstale":"2024-01-27T02:32:06.649539+0000","last_undegraded":"2024-01-27T02:32:06.649539+0000","last_fullsized":"2024-01-27T02:32:06.649539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:13:38.187522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.638138+0000","last_change":"2024-01-27T02:32:06.638050+0000","last_active":"2024-01-27T02:32:06.638138+0000","last_peered":"2024-01-27T02:32:06.638138+0000","last_clean":"2024-01-27T02:32:06.638138+0000","last_became_active":"2024-01-27T02:32:06.637390+0000","last_became_peered":"2024-01-27T02:32:06.637390+0000","last_unstale":"2024-01-27T02:32:06.638138+0000","last_undegraded":"2024-01-27T02:32:06.638138+0000","last_fullsized":"2024-01-27T02:32:06.638138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642381+0000","last_change":"2024-01-27T02:32:06.642245+0000","last_active":"2024-01-27T02:32:06.642381+0000","last_peered":"2024-01-27T02:32:06.642381+0000","last_clean":"2024-01-27T02:32:06.642381+0000","last_became_active":"2024-01-27T02:32:06.641079+0000","last_became_peered":"2024-01-27T02:32:06.641079+0000","last_unstale":"2024-01-27T02:32:06.642381+0000","last_undegraded":"2024-01-27T02:32:06.642381+0000","last_fullsized":"2024-01-27T02:32:06.642381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:38.023976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.802275+0000","last_change":"2024-01-27T02:32:05.624071+0000","last_active":"2024-01-27T02:32:06.802275+0000","last_peered":"2024-01-27T02:32:06.802275+0000","last_clean":"2024-01-27T02:32:06.802275+0000","last_became_active":"2024-01-27T02:32:05.623818+0000","last_became_peered":"2024-01-27T02:32:05.623818+0000","last_unstale":"2024-01-27T02:32:06.802275+0000","last_undegraded":"2024-01-27T02:32:06.802275+0000","last_fullsized":"2024-01-27T02:32:06.802275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:34:05.180050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.641588+0000","last_change":"2024-01-27T02:32:06.641507+0000","last_active":"2024-01-27T02:32:06.641588+0000","last_peered":"2024-01-27T02:32:06.641588+0000","last_clean":"2024-01-27T02:32:06.641588+0000","last_became_active":"2024-01-27T02:32:06.640928+0000","last_became_peered":"2024-01-27T02:32:06.640928+0000","last_unstale":"2024-01-27T02:32:06.641588+0000","last_undegraded":"2024-01-27T02:32:06.641588+0000","last_fullsized":"2024-01-27T02:32:06.641588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:32:24.198014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.803197+0000","last_change":"2024-01-27T02:32:05.624866+0000","last_active":"2024-01-27T02:32:06.803197+0000","last_peered":"2024-01-27T02:32:06.803197+0000","last_clean":"2024-01-27T02:32:06.803197+0000","last_became_active":"2024-01-27T02:32:05.624368+0000","last_became_peered":"2024-01-27T02:32:05.624368+0000","last_unstale":"2024-01-27T02:32:06.803197+0000","last_undegraded":"2024-01-27T02:32:06.803197+0000","last_fullsized":"2024-01-27T02:32:06.803197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642222+0000","last_change":"2024-01-27T02:32:06.642157+0000","last_active":"2024-01-27T02:32:06.642222+0000","last_peered":"2024-01-27T02:32:06.642222+0000","last_clean":"2024-01-27T02:32:06.642222+0000","last_became_active":"2024-01-27T02:32:06.641894+0000","last_became_peered":"2024-01-27T02:32:06.641894+0000","last_unstale":"2024-01-27T02:32:06.642222+0000","last_undegraded":"2024-01-27T02:32:06.642222+0000","last_fullsized":"2024-01-27T02:32:06.642222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.802467+0000","last_change":"2024-01-27T02:32:05.623255+0000","last_active":"2024-01-27T02:32:06.802467+0000","last_peered":"2024-01-27T02:32:06.802467+0000","last_clean":"2024-01-27T02:32:06.802467+0000","last_became_active":"2024-01-27T02:32:05.622821+0000","last_became_peered":"2024-01-27T02:32:05.622821+0000","last_unstale":"2024-01-27T02:32:06.802467+0000","last_undegraded":"2024-01-27T02:32:06.802467+0000","last_fullsized":"2024-01-27T02:32:06.802467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:33.333681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.648821+0000","last_change":"2024-01-27T02:32:06.648760+0000","last_active":"2024-01-27T02:32:06.648821+0000","last_peered":"2024-01-27T02:32:06.648821+0000","last_clean":"2024-01-27T02:32:06.648821+0000","last_became_active":"2024-01-27T02:32:06.648279+0000","last_became_peered":"2024-01-27T02:32:06.648279+0000","last_unstale":"2024-01-27T02:32:06.648821+0000","last_undegraded":"2024-01-27T02:32:06.648821+0000","last_fullsized":"2024-01-27T02:32:06.648821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:32:24.198014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.802483+0000","last_change":"2024-01-27T02:32:05.626014+0000","last_active":"2024-01-27T02:32:06.802483+0000","last_peered":"2024-01-27T02:32:06.802483+0000","last_clean":"2024-01-27T02:32:06.802483+0000","last_became_active":"2024-01-27T02:32:05.623176+0000","last_became_peered":"2024-01-27T02:32:05.623176+0000","last_unstale":"2024-01-27T02:32:06.802483+0000","last_undegraded":"2024-01-27T02:32:06.802483+0000","last_fullsized":"2024-01-27T02:32:06.802483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:37:59.246901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.641268+0000","last_change":"2024-01-27T02:32:06.637696+0000","last_active":"2024-01-27T02:32:06.641268+0000","last_peered":"2024-01-27T02:32:06.641268+0000","last_clean":"2024-01-27T02:32:06.641268+0000","last_became_active":"2024-01-27T02:32:06.637457+0000","last_became_peered":"2024-01-27T02:32:06.637457+0000","last_unstale":"2024-01-27T02:32:06.641268+0000","last_undegraded":"2024-01-27T02:32:06.641268+0000","last_fullsized":"2024-01-27T02:32:06.641268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:17:03.351766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.803103+0000","last_change":"2024-01-27T02:32:05.622086+0000","last_active":"2024-01-27T02:32:06.803103+0000","last_peered":"2024-01-27T02:32:06.803103+0000","last_clean":"2024-01-27T02:32:06.803103+0000","last_became_active":"2024-01-27T02:32:05.621465+0000","last_became_peered":"2024-01-27T02:32:05.621465+0000","last_unstale":"2024-01-27T02:32:06.803103+0000","last_undegraded":"2024-01-27T02:32:06.803103+0000","last_fullsized":"2024-01-27T02:32:06.803103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:04:28.890844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.644927+0000","last_change":"2024-01-27T02:32:06.644850+0000","last_active":"2024-01-27T02:32:06.644927+0000","last_peered":"2024-01-27T02:32:06.644927+0000","last_clean":"2024-01-27T02:32:06.644927+0000","last_became_active":"2024-01-27T02:32:06.644442+0000","last_became_peered":"2024-01-27T02:32:06.644442+0000","last_unstale":"2024-01-27T02:32:06.644927+0000","last_undegraded":"2024-01-27T02:32:06.644927+0000","last_fullsized":"2024-01-27T02:32:06.644927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.801277+0000","last_change":"2024-01-27T02:32:05.617530+0000","last_active":"2024-01-27T02:32:06.801277+0000","last_peered":"2024-01-27T02:32:06.801277+0000","last_clean":"2024-01-27T02:32:06.801277+0000","last_became_active":"2024-01-27T02:32:05.617290+0000","last_became_peered":"2024-01-27T02:32:05.617290+0000","last_unstale":"2024-01-27T02:32:06.801277+0000","last_undegraded":"2024-01-27T02:32:06.801277+0000","last_fullsized":"2024-01-27T02:32:06.801277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:41.575605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642446+0000","last_change":"2024-01-27T02:32:06.642339+0000","last_active":"2024-01-27T02:32:06.642446+0000","last_peered":"2024-01-27T02:32:06.642446+0000","last_clean":"2024-01-27T02:32:06.642446+0000","last_became_active":"2024-01-27T02:32:06.642027+0000","last_became_peered":"2024-01-27T02:32:06.642027+0000","last_unstale":"2024-01-27T02:32:06.642446+0000","last_undegraded":"2024-01-27T02:32:06.642446+0000","last_fullsized":"2024-01-27T02:32:06.642446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:45.494048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.628837+0000","last_change":"2024-01-27T02:32:05.623693+0000","last_active":"2024-01-27T02:32:06.628837+0000","last_peered":"2024-01-27T02:32:06.628837+0000","last_clean":"2024-01-27T02:32:06.628837+0000","last_became_active":"2024-01-27T02:32:05.623440+0000","last_became_peered":"2024-01-27T02:32:05.623440+0000","last_unstale":"2024-01-27T02:32:06.628837+0000","last_undegraded":"2024-01-27T02:32:06.628837+0000","last_fullsized":"2024-01-27T02:32:06.628837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:44:52.929407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.637714+0000","last_change":"2024-01-27T02:32:06.635720+0000","last_active":"2024-01-27T02:32:06.637714+0000","last_peered":"2024-01-27T02:32:06.637714+0000","last_clean":"2024-01-27T02:32:06.637714+0000","last_became_active":"2024-01-27T02:32:06.635457+0000","last_became_peered":"2024-01-27T02:32:06.635457+0000","last_unstale":"2024-01-27T02:32:06.637714+0000","last_undegraded":"2024-01-27T02:32:06.637714+0000","last_fullsized":"2024-01-27T02:32:06.637714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:36:11.295485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.633418+0000","last_change":"2024-01-27T02:32:05.619476+0000","last_active":"2024-01-27T02:32:06.633418+0000","last_peered":"2024-01-27T02:32:06.633418+0000","last_clean":"2024-01-27T02:32:06.633418+0000","last_became_active":"2024-01-27T02:32:05.619194+0000","last_became_peered":"2024-01-27T02:32:05.619194+0000","last_unstale":"2024-01-27T02:32:06.633418+0000","last_undegraded":"2024-01-27T02:32:06.633418+0000","last_fullsized":"2024-01-27T02:32:06.633418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.634597+0000","last_change":"2024-01-27T02:32:06.634504+0000","last_active":"2024-01-27T02:32:06.634597+0000","last_peered":"2024-01-27T02:32:06.634597+0000","last_clean":"2024-01-27T02:32:06.634597+0000","last_became_active":"2024-01-27T02:32:06.634220+0000","last_became_peered":"2024-01-27T02:32:06.634220+0000","last_unstale":"2024-01-27T02:32:06.634597+0000","last_undegraded":"2024-01-27T02:32:06.634597+0000","last_fullsized":"2024-01-27T02:32:06.634597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:45:37.035302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.635103+0000","last_change":"2024-01-27T02:32:05.623827+0000","last_active":"2024-01-27T02:32:06.635103+0000","last_peered":"2024-01-27T02:32:06.635103+0000","last_clean":"2024-01-27T02:32:06.635103+0000","last_became_active":"2024-01-27T02:32:05.623602+0000","last_became_peered":"2024-01-27T02:32:05.623602+0000","last_unstale":"2024-01-27T02:32:06.635103+0000","last_undegraded":"2024-01-27T02:32:06.635103+0000","last_fullsized":"2024-01-27T02:32:06.635103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.649471+0000","last_change":"2024-01-27T02:32:06.649029+0000","last_active":"2024-01-27T02:32:06.649471+0000","last_peered":"2024-01-27T02:32:06.649471+0000","last_clean":"2024-01-27T02:32:06.649471+0000","last_became_active":"2024-01-27T02:32:06.647963+0000","last_became_peered":"2024-01-27T02:32:06.647963+0000","last_unstale":"2024-01-27T02:32:06.649471+0000","last_undegraded":"2024-01-27T02:32:06.649471+0000","last_fullsized":"2024-01-27T02:32:06.649471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:43:05.599796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.635934+0000","last_change":"2024-01-27T02:32:05.621283+0000","last_active":"2024-01-27T02:32:06.635934+0000","last_peered":"2024-01-27T02:32:06.635934+0000","last_clean":"2024-01-27T02:32:06.635934+0000","last_became_active":"2024-01-27T02:32:05.621034+0000","last_became_peered":"2024-01-27T02:32:05.621034+0000","last_unstale":"2024-01-27T02:32:06.635934+0000","last_undegraded":"2024-01-27T02:32:06.635934+0000","last_fullsized":"2024-01-27T02:32:06.635934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:07.586396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.649134+0000","last_change":"2024-01-27T02:32:06.649052+0000","last_active":"2024-01-27T02:32:06.649134+0000","last_peered":"2024-01-27T02:32:06.649134+0000","last_clean":"2024-01-27T02:32:06.649134+0000","last_became_active":"2024-01-27T02:32:06.648837+0000","last_became_peered":"2024-01-27T02:32:06.648837+0000","last_unstale":"2024-01-27T02:32:06.649134+0000","last_undegraded":"2024-01-27T02:32:06.649134+0000","last_fullsized":"2024-01-27T02:32:06.649134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.802664+0000","last_change":"2024-01-27T02:32:05.620524+0000","last_active":"2024-01-27T02:32:06.802664+0000","last_peered":"2024-01-27T02:32:06.802664+0000","last_clean":"2024-01-27T02:32:06.802664+0000","last_became_active":"2024-01-27T02:32:05.620169+0000","last_became_peered":"2024-01-27T02:32:05.620169+0000","last_unstale":"2024-01-27T02:32:06.802664+0000","last_undegraded":"2024-01-27T02:32:06.802664+0000","last_fullsized":"2024-01-27T02:32:06.802664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.641576+0000","last_change":"2024-01-27T02:32:06.641250+0000","last_active":"2024-01-27T02:32:06.641576+0000","last_peered":"2024-01-27T02:32:06.641576+0000","last_clean":"2024-01-27T02:32:06.641576+0000","last_became_active":"2024-01-27T02:32:06.640947+0000","last_became_peered":"2024-01-27T02:32:06.640947+0000","last_unstale":"2024-01-27T02:32:06.641576+0000","last_undegraded":"2024-01-27T02:32:06.641576+0000","last_fullsized":"2024-01-27T02:32:06.641576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.804224+0000","last_change":"2024-01-27T02:32:05.625720+0000","last_active":"2024-01-27T02:32:06.804224+0000","last_peered":"2024-01-27T02:32:06.804224+0000","last_clean":"2024-01-27T02:32:06.804224+0000","last_became_active":"2024-01-27T02:32:05.625394+0000","last_became_peered":"2024-01-27T02:32:05.625394+0000","last_unstale":"2024-01-27T02:32:06.804224+0000","last_undegraded":"2024-01-27T02:32:06.804224+0000","last_fullsized":"2024-01-27T02:32:06.804224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.643130+0000","last_change":"2024-01-27T02:32:06.643058+0000","last_active":"2024-01-27T02:32:06.643130+0000","last_peered":"2024-01-27T02:32:06.643130+0000","last_clean":"2024-01-27T02:32:06.643130+0000","last_became_active":"2024-01-27T02:32:06.642818+0000","last_became_peered":"2024-01-27T02:32:06.642818+0000","last_unstale":"2024-01-27T02:32:06.643130+0000","last_undegraded":"2024-01-27T02:32:06.643130+0000","last_fullsized":"2024-01-27T02:32:06.643130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:15:06.036893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.801374+0000","last_change":"2024-01-27T02:32:05.619745+0000","last_active":"2024-01-27T02:32:06.801374+0000","last_peered":"2024-01-27T02:32:06.801374+0000","last_clean":"2024-01-27T02:32:06.801374+0000","last_became_active":"2024-01-27T02:32:05.619371+0000","last_became_peered":"2024-01-27T02:32:05.619371+0000","last_unstale":"2024-01-27T02:32:06.801374+0000","last_undegraded":"2024-01-27T02:32:06.801374+0000","last_fullsized":"2024-01-27T02:32:06.801374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:12:20.892542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.646863+0000","last_change":"2024-01-27T02:32:06.646797+0000","last_active":"2024-01-27T02:32:06.646863+0000","last_peered":"2024-01-27T02:32:06.646863+0000","last_clean":"2024-01-27T02:32:06.646863+0000","last_became_active":"2024-01-27T02:32:06.646253+0000","last_became_peered":"2024-01-27T02:32:06.646253+0000","last_unstale":"2024-01-27T02:32:06.646863+0000","last_undegraded":"2024-01-27T02:32:06.646863+0000","last_fullsized":"2024-01-27T02:32:06.646863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:58:54.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.643949+0000","last_change":"2024-01-27T02:32:05.626895+0000","last_active":"2024-01-27T02:32:06.643949+0000","last_peered":"2024-01-27T02:32:06.643949+0000","last_clean":"2024-01-27T02:32:06.643949+0000","last_became_active":"2024-01-27T02:32:05.626682+0000","last_became_peered":"2024-01-27T02:32:05.626682+0000","last_unstale":"2024-01-27T02:32:06.643949+0000","last_undegraded":"2024-01-27T02:32:06.643949+0000","last_fullsized":"2024-01-27T02:32:06.643949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:41.575605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.645112+0000","last_change":"2024-01-27T02:32:06.644466+0000","last_active":"2024-01-27T02:32:06.645112+0000","last_peered":"2024-01-27T02:32:06.645112+0000","last_clean":"2024-01-27T02:32:06.645112+0000","last_became_active":"2024-01-27T02:32:06.643544+0000","last_became_peered":"2024-01-27T02:32:06.643544+0000","last_unstale":"2024-01-27T02:32:06.645112+0000","last_undegraded":"2024-01-27T02:32:06.645112+0000","last_fullsized":"2024-01-27T02:32:06.645112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:22:27.441031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.635250+0000","last_change":"2024-01-27T02:32:05.619939+0000","last_active":"2024-01-27T02:32:06.635250+0000","last_peered":"2024-01-27T02:32:06.635250+0000","last_clean":"2024-01-27T02:32:06.635250+0000","last_became_active":"2024-01-27T02:32:05.619489+0000","last_became_peered":"2024-01-27T02:32:05.619489+0000","last_unstale":"2024-01-27T02:32:06.635250+0000","last_undegraded":"2024-01-27T02:32:06.635250+0000","last_fullsized":"2024-01-27T02:32:06.635250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:07.586396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.638890+0000","last_change":"2024-01-27T02:32:06.638821+0000","last_active":"2024-01-27T02:32:06.638890+0000","last_peered":"2024-01-27T02:32:06.638890+0000","last_clean":"2024-01-27T02:32:06.638890+0000","last_became_active":"2024-01-27T02:32:06.638486+0000","last_became_peered":"2024-01-27T02:32:06.638486+0000","last_unstale":"2024-01-27T02:32:06.638890+0000","last_undegraded":"2024-01-27T02:32:06.638890+0000","last_fullsized":"2024-01-27T02:32:06.638890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:45:37.035302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.806345+0000","last_change":"2024-01-27T02:32:05.621851+0000","last_active":"2024-01-27T02:32:06.806345+0000","last_peered":"2024-01-27T02:32:06.806345+0000","last_clean":"2024-01-27T02:32:06.806345+0000","last_became_active":"2024-01-27T02:32:05.621487+0000","last_became_peered":"2024-01-27T02:32:05.621487+0000","last_unstale":"2024-01-27T02:32:06.806345+0000","last_undegraded":"2024-01-27T02:32:06.806345+0000","last_fullsized":"2024-01-27T02:32:06.806345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:07.586396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.645157+0000","last_change":"2024-01-27T02:32:06.644806+0000","last_active":"2024-01-27T02:32:06.645157+0000","last_peered":"2024-01-27T02:32:06.645157+0000","last_clean":"2024-01-27T02:32:06.645157+0000","last_became_active":"2024-01-27T02:32:06.643567+0000","last_became_peered":"2024-01-27T02:32:06.643567+0000","last_unstale":"2024-01-27T02:32:06.645157+0000","last_undegraded":"2024-01-27T02:32:06.645157+0000","last_fullsized":"2024-01-27T02:32:06.645157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:44:01.051017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.801050+0000","last_change":"2024-01-27T02:32:05.623147+0000","last_active":"2024-01-27T02:32:06.801050+0000","last_peered":"2024-01-27T02:32:06.801050+0000","last_clean":"2024-01-27T02:32:06.801050+0000","last_became_active":"2024-01-27T02:32:05.622650+0000","last_became_peered":"2024-01-27T02:32:05.622650+0000","last_unstale":"2024-01-27T02:32:06.801050+0000","last_undegraded":"2024-01-27T02:32:06.801050+0000","last_fullsized":"2024-01-27T02:32:06.801050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:30:53.128829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.645128+0000","last_change":"2024-01-27T02:32:06.644626+0000","last_active":"2024-01-27T02:32:06.645128+0000","last_peered":"2024-01-27T02:32:06.645128+0000","last_clean":"2024-01-27T02:32:06.645128+0000","last_became_active":"2024-01-27T02:32:06.644149+0000","last_became_peered":"2024-01-27T02:32:06.644149+0000","last_unstale":"2024-01-27T02:32:06.645128+0000","last_undegraded":"2024-01-27T02:32:06.645128+0000","last_fullsized":"2024-01-27T02:32:06.645128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:13:38.187522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.632259+0000","last_change":"2024-01-27T02:32:05.621921+0000","last_active":"2024-01-27T02:32:06.632259+0000","last_peered":"2024-01-27T02:32:06.632259+0000","last_clean":"2024-01-27T02:32:06.632259+0000","last_became_active":"2024-01-27T02:32:05.620450+0000","last_became_peered":"2024-01-27T02:32:05.620450+0000","last_unstale":"2024-01-27T02:32:06.632259+0000","last_undegraded":"2024-01-27T02:32:06.632259+0000","last_fullsized":"2024-01-27T02:32:06.632259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:33.333681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.649453+0000","last_change":"2024-01-27T02:32:06.648997+0000","last_active":"2024-01-27T02:32:06.649453+0000","last_peered":"2024-01-27T02:32:06.649453+0000","last_clean":"2024-01-27T02:32:06.649453+0000","last_became_active":"2024-01-27T02:32:06.648180+0000","last_became_peered":"2024-01-27T02:32:06.648180+0000","last_unstale":"2024-01-27T02:32:06.649453+0000","last_undegraded":"2024-01-27T02:32:06.649453+0000","last_fullsized":"2024-01-27T02:32:06.649453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:15:38.764945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.636045+0000","last_change":"2024-01-27T02:32:05.622783+0000","last_active":"2024-01-27T02:32:06.636045+0000","last_peered":"2024-01-27T02:32:06.636045+0000","last_clean":"2024-01-27T02:32:06.636045+0000","last_became_active":"2024-01-27T02:32:05.622557+0000","last_became_peered":"2024-01-27T02:32:05.622557+0000","last_unstale":"2024-01-27T02:32:06.636045+0000","last_undegraded":"2024-01-27T02:32:06.636045+0000","last_fullsized":"2024-01-27T02:32:06.636045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.638811+0000","last_change":"2024-01-27T02:32:06.638626+0000","last_active":"2024-01-27T02:32:06.638811+0000","last_peered":"2024-01-27T02:32:06.638811+0000","last_clean":"2024-01-27T02:32:06.638811+0000","last_became_active":"2024-01-27T02:32:06.638382+0000","last_became_peered":"2024-01-27T02:32:06.638382+0000","last_unstale":"2024-01-27T02:32:06.638811+0000","last_undegraded":"2024-01-27T02:32:06.638811+0000","last_fullsized":"2024-01-27T02:32:06.638811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:31.534822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.635736+0000","last_change":"2024-01-27T02:32:06.635617+0000","last_active":"2024-01-27T02:32:06.635736+0000","last_peered":"2024-01-27T02:32:06.635736+0000","last_clean":"2024-01-27T02:32:06.635736+0000","last_became_active":"2024-01-27T02:32:06.635367+0000","last_became_peered":"2024-01-27T02:32:06.635367+0000","last_unstale":"2024-01-27T02:32:06.635736+0000","last_undegraded":"2024-01-27T02:32:06.635736+0000","last_fullsized":"2024-01-27T02:32:06.635736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:36:11.295485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.630282+0000","last_change":"2024-01-27T02:32:05.613447+0000","last_active":"2024-01-27T02:32:06.630282+0000","last_peered":"2024-01-27T02:32:06.630282+0000","last_clean":"2024-01-27T02:32:06.630282+0000","last_became_active":"2024-01-27T02:32:05.613203+0000","last_became_peered":"2024-01-27T02:32:05.613203+0000","last_unstale":"2024-01-27T02:32:06.630282+0000","last_undegraded":"2024-01-27T02:32:06.630282+0000","last_fullsized":"2024-01-27T02:32:06.630282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:41.575605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.648445+0000","last_change":"2024-01-27T02:32:06.648364+0000","last_active":"2024-01-27T02:32:06.648445+0000","last_peered":"2024-01-27T02:32:06.648445+0000","last_clean":"2024-01-27T02:32:06.648445+0000","last_became_active":"2024-01-27T02:32:06.648115+0000","last_became_peered":"2024-01-27T02:32:06.648115+0000","last_unstale":"2024-01-27T02:32:06.648445+0000","last_undegraded":"2024-01-27T02:32:06.648445+0000","last_fullsized":"2024-01-27T02:32:06.648445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:43:05.599796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.644051+0000","last_change":"2024-01-27T02:32:05.622673+0000","last_active":"2024-01-27T02:32:06.644051+0000","last_peered":"2024-01-27T02:32:06.644051+0000","last_clean":"2024-01-27T02:32:06.644051+0000","last_became_active":"2024-01-27T02:32:05.622195+0000","last_became_peered":"2024-01-27T02:32:05.622195+0000","last_unstale":"2024-01-27T02:32:06.644051+0000","last_undegraded":"2024-01-27T02:32:06.644051+0000","last_fullsized":"2024-01-27T02:32:06.644051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.637753+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:32:06.637753+0000","last_peered":"2024-01-27T02:32:06.637753+0000","last_clean":"2024-01-27T02:32:06.637753+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:32:06.637753+0000","last_undegraded":"2024-01-27T02:32:06.637753+0000","last_fullsized":"2024-01-27T02:32:06.637753+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.646119+0000","last_change":"2024-01-27T02:32:06.646043+0000","last_active":"2024-01-27T02:32:06.646119+0000","last_peered":"2024-01-27T02:32:06.646119+0000","last_clean":"2024-01-27T02:32:06.646119+0000","last_became_active":"2024-01-27T02:32:06.645836+0000","last_became_peered":"2024-01-27T02:32:06.645836+0000","last_unstale":"2024-01-27T02:32:06.646119+0000","last_undegraded":"2024-01-27T02:32:06.646119+0000","last_fullsized":"2024-01-27T02:32:06.646119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.636869+0000","last_change":"2024-01-27T02:32:05.626552+0000","last_active":"2024-01-27T02:32:06.636869+0000","last_peered":"2024-01-27T02:32:06.636869+0000","last_clean":"2024-01-27T02:32:06.636869+0000","last_became_active":"2024-01-27T02:32:05.626348+0000","last_became_peered":"2024-01-27T02:32:05.626348+0000","last_unstale":"2024-01-27T02:32:06.636869+0000","last_undegraded":"2024-01-27T02:32:06.636869+0000","last_fullsized":"2024-01-27T02:32:06.636869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:18.034484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.636979+0000","last_change":"2024-01-27T02:32:05.625885+0000","last_active":"2024-01-27T02:32:06.636979+0000","last_peered":"2024-01-27T02:32:06.636979+0000","last_clean":"2024-01-27T02:32:06.636979+0000","last_became_active":"2024-01-27T02:32:05.622692+0000","last_became_peered":"2024-01-27T02:32:05.622692+0000","last_unstale":"2024-01-27T02:32:06.636979+0000","last_undegraded":"2024-01-27T02:32:06.636979+0000","last_fullsized":"2024-01-27T02:32:06.636979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:06.920232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.639545+0000","last_change":"2024-01-27T02:32:06.638962+0000","last_active":"2024-01-27T02:32:06.639545+0000","last_peered":"2024-01-27T02:32:06.639545+0000","last_clean":"2024-01-27T02:32:06.639545+0000","last_became_active":"2024-01-27T02:32:06.638238+0000","last_became_peered":"2024-01-27T02:32:06.638238+0000","last_unstale":"2024-01-27T02:32:06.639545+0000","last_undegraded":"2024-01-27T02:32:06.639545+0000","last_fullsized":"2024-01-27T02:32:06.639545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:44:13.569068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-27T02:32:07.864093+0000","last_change":"2024-01-27T02:32:05.621014+0000","last_active":"2024-01-27T02:32:07.864093+0000","last_peered":"2024-01-27T02:32:07.864093+0000","last_clean":"2024-01-27T02:32:07.864093+0000","last_became_active":"2024-01-27T02:32:05.620775+0000","last_became_peered":"2024-01-27T02:32:05.620775+0000","last_unstale":"2024-01-27T02:32:07.864093+0000","last_undegraded":"2024-01-27T02:32:07.864093+0000","last_fullsized":"2024-01-27T02:32:07.864093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642354+0000","last_change":"2024-01-27T02:32:06.642266+0000","last_active":"2024-01-27T02:32:06.642354+0000","last_peered":"2024-01-27T02:32:06.642354+0000","last_clean":"2024-01-27T02:32:06.642354+0000","last_became_active":"2024-01-27T02:32:06.641998+0000","last_became_peered":"2024-01-27T02:32:06.641998+0000","last_unstale":"2024-01-27T02:32:06.642354+0000","last_undegraded":"2024-01-27T02:32:06.642354+0000","last_fullsized":"2024-01-27T02:32:06.642354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:58:54.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.802073+0000","last_change":"2024-01-27T02:32:05.624870+0000","last_active":"2024-01-27T02:32:06.802073+0000","last_peered":"2024-01-27T02:32:06.802073+0000","last_clean":"2024-01-27T02:32:06.802073+0000","last_became_active":"2024-01-27T02:32:05.624644+0000","last_became_peered":"2024-01-27T02:32:05.624644+0000","last_unstale":"2024-01-27T02:32:06.802073+0000","last_undegraded":"2024-01-27T02:32:06.802073+0000","last_fullsized":"2024-01-27T02:32:06.802073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.637689+0000","last_change":"2024-01-27T02:32:06.637609+0000","last_active":"2024-01-27T02:32:06.637689+0000","last_peered":"2024-01-27T02:32:06.637689+0000","last_clean":"2024-01-27T02:32:06.637689+0000","last_became_active":"2024-01-27T02:32:06.637282+0000","last_became_peered":"2024-01-27T02:32:06.637282+0000","last_unstale":"2024-01-27T02:32:06.637689+0000","last_undegraded":"2024-01-27T02:32:06.637689+0000","last_fullsized":"2024-01-27T02:32:06.637689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:06.985231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.637206+0000","last_change":"2024-01-27T02:32:05.621287+0000","last_active":"2024-01-27T02:32:06.637206+0000","last_peered":"2024-01-27T02:32:06.637206+0000","last_clean":"2024-01-27T02:32:06.637206+0000","last_became_active":"2024-01-27T02:32:05.620002+0000","last_became_peered":"2024-01-27T02:32:05.620002+0000","last_unstale":"2024-01-27T02:32:06.637206+0000","last_undegraded":"2024-01-27T02:32:06.637206+0000","last_fullsized":"2024-01-27T02:32:06.637206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:44.489465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.637891+0000","last_change":"2024-01-27T02:32:06.637830+0000","last_active":"2024-01-27T02:32:06.637891+0000","last_peered":"2024-01-27T02:32:06.637891+0000","last_clean":"2024-01-27T02:32:06.637891+0000","last_became_active":"2024-01-27T02:32:06.637387+0000","last_became_peered":"2024-01-27T02:32:06.637387+0000","last_unstale":"2024-01-27T02:32:06.637891+0000","last_undegraded":"2024-01-27T02:32:06.637891+0000","last_fullsized":"2024-01-27T02:32:06.637891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:47.819053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.632380+0000","last_change":"2024-01-27T02:32:05.623449+0000","last_active":"2024-01-27T02:32:06.632380+0000","last_peered":"2024-01-27T02:32:06.632380+0000","last_clean":"2024-01-27T02:32:06.632380+0000","last_became_active":"2024-01-27T02:32:05.623193+0000","last_became_peered":"2024-01-27T02:32:05.623193+0000","last_unstale":"2024-01-27T02:32:06.632380+0000","last_undegraded":"2024-01-27T02:32:06.632380+0000","last_fullsized":"2024-01-27T02:32:06.632380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:44.489465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.632750+0000","last_change":"2024-01-27T02:32:06.632652+0000","last_active":"2024-01-27T02:32:06.632750+0000","last_peered":"2024-01-27T02:32:06.632750+0000","last_clean":"2024-01-27T02:32:06.632750+0000","last_became_active":"2024-01-27T02:32:06.632366+0000","last_became_peered":"2024-01-27T02:32:06.632366+0000","last_unstale":"2024-01-27T02:32:06.632750+0000","last_undegraded":"2024-01-27T02:32:06.632750+0000","last_fullsized":"2024-01-27T02:32:06.632750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:06.985231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.631717+0000","last_change":"2024-01-27T02:32:05.625193+0000","last_active":"2024-01-27T02:32:06.631717+0000","last_peered":"2024-01-27T02:32:06.631717+0000","last_clean":"2024-01-27T02:32:06.631717+0000","last_became_active":"2024-01-27T02:32:05.624951+0000","last_became_peered":"2024-01-27T02:32:05.624951+0000","last_unstale":"2024-01-27T02:32:06.631717+0000","last_undegraded":"2024-01-27T02:32:06.631717+0000","last_fullsized":"2024-01-27T02:32:06.631717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:04:28.890844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642383+0000","last_change":"2024-01-27T02:32:06.642267+0000","last_active":"2024-01-27T02:32:06.642383+0000","last_peered":"2024-01-27T02:32:06.642383+0000","last_clean":"2024-01-27T02:32:06.642383+0000","last_became_active":"2024-01-27T02:32:06.640004+0000","last_became_peered":"2024-01-27T02:32:06.640004+0000","last_unstale":"2024-01-27T02:32:06.642383+0000","last_undegraded":"2024-01-27T02:32:06.642383+0000","last_fullsized":"2024-01-27T02:32:06.642383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:45:37.035302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.801718+0000","last_change":"2024-01-27T02:32:05.625673+0000","last_active":"2024-01-27T02:32:06.801718+0000","last_peered":"2024-01-27T02:32:06.801718+0000","last_clean":"2024-01-27T02:32:06.801718+0000","last_became_active":"2024-01-27T02:32:05.623551+0000","last_became_peered":"2024-01-27T02:32:05.623551+0000","last_unstale":"2024-01-27T02:32:06.801718+0000","last_undegraded":"2024-01-27T02:32:06.801718+0000","last_fullsized":"2024-01-27T02:32:06.801718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.639544+0000","last_change":"2024-01-27T02:32:06.639489+0000","last_active":"2024-01-27T02:32:06.639544+0000","last_peered":"2024-01-27T02:32:06.639544+0000","last_clean":"2024-01-27T02:32:06.639544+0000","last_became_active":"2024-01-27T02:32:06.639246+0000","last_became_peered":"2024-01-27T02:32:06.639246+0000","last_unstale":"2024-01-27T02:32:06.639544+0000","last_undegraded":"2024-01-27T02:32:06.639544+0000","last_fullsized":"2024-01-27T02:32:06.639544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.629629+0000","last_change":"2024-01-27T02:32:05.620877+0000","last_active":"2024-01-27T02:32:06.629629+0000","last_peered":"2024-01-27T02:32:06.629629+0000","last_clean":"2024-01-27T02:32:06.629629+0000","last_became_active":"2024-01-27T02:32:05.620600+0000","last_became_peered":"2024-01-27T02:32:05.620600+0000","last_unstale":"2024-01-27T02:32:06.629629+0000","last_undegraded":"2024-01-27T02:32:06.629629+0000","last_fullsized":"2024-01-27T02:32:06.629629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.650085+0000","last_change":"2024-01-27T02:32:06.649956+0000","last_active":"2024-01-27T02:32:06.650085+0000","last_peered":"2024-01-27T02:32:06.650085+0000","last_clean":"2024-01-27T02:32:06.650085+0000","last_became_active":"2024-01-27T02:32:06.649759+0000","last_became_peered":"2024-01-27T02:32:06.649759+0000","last_unstale":"2024-01-27T02:32:06.650085+0000","last_undegraded":"2024-01-27T02:32:06.650085+0000","last_fullsized":"2024-01-27T02:32:06.650085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:56:39.515436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.801885+0000","last_change":"2024-01-27T02:32:05.619486+0000","last_active":"2024-01-27T02:32:06.801885+0000","last_peered":"2024-01-27T02:32:06.801885+0000","last_clean":"2024-01-27T02:32:06.801885+0000","last_became_active":"2024-01-27T02:32:05.619147+0000","last_became_peered":"2024-01-27T02:32:05.619147+0000","last_unstale":"2024-01-27T02:32:06.801885+0000","last_undegraded":"2024-01-27T02:32:06.801885+0000","last_fullsized":"2024-01-27T02:32:06.801885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.650011+0000","last_change":"2024-01-27T02:32:06.649606+0000","last_active":"2024-01-27T02:32:06.650011+0000","last_peered":"2024-01-27T02:32:06.650011+0000","last_clean":"2024-01-27T02:32:06.650011+0000","last_became_active":"2024-01-27T02:32:06.648805+0000","last_became_peered":"2024-01-27T02:32:06.648805+0000","last_unstale":"2024-01-27T02:32:06.650011+0000","last_undegraded":"2024-01-27T02:32:06.650011+0000","last_fullsized":"2024-01-27T02:32:06.650011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:31:12.724216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.804019+0000","last_change":"2024-01-27T02:32:05.613146+0000","last_active":"2024-01-27T02:32:06.804019+0000","last_peered":"2024-01-27T02:32:06.804019+0000","last_clean":"2024-01-27T02:32:06.804019+0000","last_became_active":"2024-01-27T02:32:05.612852+0000","last_became_peered":"2024-01-27T02:32:05.612852+0000","last_unstale":"2024-01-27T02:32:06.804019+0000","last_undegraded":"2024-01-27T02:32:06.804019+0000","last_fullsized":"2024-01-27T02:32:06.804019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.641582+0000","last_change":"2024-01-27T02:32:06.641444+0000","last_active":"2024-01-27T02:32:06.641582+0000","last_peered":"2024-01-27T02:32:06.641582+0000","last_clean":"2024-01-27T02:32:06.641582+0000","last_became_active":"2024-01-27T02:32:06.640698+0000","last_became_peered":"2024-01-27T02:32:06.640698+0000","last_unstale":"2024-01-27T02:32:06.641582+0000","last_undegraded":"2024-01-27T02:32:06.641582+0000","last_fullsized":"2024-01-27T02:32:06.641582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:15:38.764945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.639806+0000","last_change":"2024-01-27T02:32:06.639727+0000","last_active":"2024-01-27T02:32:06.639806+0000","last_peered":"2024-01-27T02:32:06.639806+0000","last_clean":"2024-01-27T02:32:06.639806+0000","last_became_active":"2024-01-27T02:32:06.639457+0000","last_became_peered":"2024-01-27T02:32:06.639457+0000","last_unstale":"2024-01-27T02:32:06.639806+0000","last_undegraded":"2024-01-27T02:32:06.639806+0000","last_fullsized":"2024-01-27T02:32:06.639806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:17:03.351766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.646128+0000","last_change":"2024-01-27T02:32:06.646051+0000","last_active":"2024-01-27T02:32:06.646128+0000","last_peered":"2024-01-27T02:32:06.646128+0000","last_clean":"2024-01-27T02:32:06.646128+0000","last_became_active":"2024-01-27T02:32:06.645857+0000","last_became_peered":"2024-01-27T02:32:06.645857+0000","last_unstale":"2024-01-27T02:32:06.646128+0000","last_undegraded":"2024-01-27T02:32:06.646128+0000","last_fullsized":"2024-01-27T02:32:06.646128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.648193+0000","last_change":"2024-01-27T02:32:06.648063+0000","last_active":"2024-01-27T02:32:06.648193+0000","last_peered":"2024-01-27T02:32:06.648193+0000","last_clean":"2024-01-27T02:32:06.648193+0000","last_became_active":"2024-01-27T02:32:06.647524+0000","last_became_peered":"2024-01-27T02:32:06.647524+0000","last_unstale":"2024-01-27T02:32:06.648193+0000","last_undegraded":"2024-01-27T02:32:06.648193+0000","last_fullsized":"2024-01-27T02:32:06.648193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:56:39.515436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.639513+0000","last_change":"2024-01-27T02:32:06.639430+0000","last_active":"2024-01-27T02:32:06.639513+0000","last_peered":"2024-01-27T02:32:06.639513+0000","last_clean":"2024-01-27T02:32:06.639513+0000","last_became_active":"2024-01-27T02:32:06.639149+0000","last_became_peered":"2024-01-27T02:32:06.639149+0000","last_unstale":"2024-01-27T02:32:06.639513+0000","last_undegraded":"2024-01-27T02:32:06.639513+0000","last_fullsized":"2024-01-27T02:32:06.639513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:36:11.295485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.641457+0000","last_change":"2024-01-27T02:32:06.640757+0000","last_active":"2024-01-27T02:32:06.641457+0000","last_peered":"2024-01-27T02:32:06.641457+0000","last_clean":"2024-01-27T02:32:06.641457+0000","last_became_active":"2024-01-27T02:32:06.637783+0000","last_became_peered":"2024-01-27T02:32:06.637783+0000","last_unstale":"2024-01-27T02:32:06.641457+0000","last_undegraded":"2024-01-27T02:32:06.641457+0000","last_fullsized":"2024-01-27T02:32:06.641457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:44:13.569068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.633658+0000","last_change":"2024-01-27T02:32:06.633366+0000","last_active":"2024-01-27T02:32:06.633658+0000","last_peered":"2024-01-27T02:32:06.633658+0000","last_clean":"2024-01-27T02:32:06.633658+0000","last_became_active":"2024-01-27T02:32:06.633117+0000","last_became_peered":"2024-01-27T02:32:06.633117+0000","last_unstale":"2024-01-27T02:32:06.633658+0000","last_undegraded":"2024-01-27T02:32:06.633658+0000","last_fullsized":"2024-01-27T02:32:06.633658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.645347+0000","last_change":"2024-01-27T02:32:06.645277+0000","last_active":"2024-01-27T02:32:06.645347+0000","last_peered":"2024-01-27T02:32:06.645347+0000","last_clean":"2024-01-27T02:32:06.645347+0000","last_became_active":"2024-01-27T02:32:06.645042+0000","last_became_peered":"2024-01-27T02:32:06.645042+0000","last_unstale":"2024-01-27T02:32:06.645347+0000","last_undegraded":"2024-01-27T02:32:06.645347+0000","last_fullsized":"2024-01-27T02:32:06.645347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:44:13.569068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.637735+0000","last_change":"2024-01-27T02:32:06.636359+0000","last_active":"2024-01-27T02:32:06.637735+0000","last_peered":"2024-01-27T02:32:06.637735+0000","last_clean":"2024-01-27T02:32:06.637735+0000","last_became_active":"2024-01-27T02:32:06.636084+0000","last_became_peered":"2024-01-27T02:32:06.636084+0000","last_unstale":"2024-01-27T02:32:06.637735+0000","last_undegraded":"2024-01-27T02:32:06.637735+0000","last_fullsized":"2024-01-27T02:32:06.637735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.648901+0000","last_change":"2024-01-27T02:32:06.648767+0000","last_active":"2024-01-27T02:32:06.648901+0000","last_peered":"2024-01-27T02:32:06.648901+0000","last_clean":"2024-01-27T02:32:06.648901+0000","last_became_active":"2024-01-27T02:32:06.647799+0000","last_became_peered":"2024-01-27T02:32:06.647799+0000","last_unstale":"2024-01-27T02:32:06.648901+0000","last_undegraded":"2024-01-27T02:32:06.648901+0000","last_fullsized":"2024-01-27T02:32:06.648901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:36:11.295485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.641482+0000","last_change":"2024-01-27T02:32:06.640928+0000","last_active":"2024-01-27T02:32:06.641482+0000","last_peered":"2024-01-27T02:32:06.641482+0000","last_clean":"2024-01-27T02:32:06.641482+0000","last_became_active":"2024-01-27T02:32:06.638110+0000","last_became_peered":"2024-01-27T02:32:06.638110+0000","last_unstale":"2024-01-27T02:32:06.641482+0000","last_undegraded":"2024-01-27T02:32:06.641482+0000","last_fullsized":"2024-01-27T02:32:06.641482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:17:03.351766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.649522+0000","last_change":"2024-01-27T02:32:06.649302+0000","last_active":"2024-01-27T02:32:06.649522+0000","last_peered":"2024-01-27T02:32:06.649522+0000","last_clean":"2024-01-27T02:32:06.649522+0000","last_became_active":"2024-01-27T02:32:06.648315+0000","last_became_peered":"2024-01-27T02:32:06.648315+0000","last_unstale":"2024-01-27T02:32:06.649522+0000","last_undegraded":"2024-01-27T02:32:06.649522+0000","last_fullsized":"2024-01-27T02:32:06.649522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:44:24.628811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.646984+0000","last_change":"2024-01-27T02:32:06.646935+0000","last_active":"2024-01-27T02:32:06.646984+0000","last_peered":"2024-01-27T02:32:06.646984+0000","last_clean":"2024-01-27T02:32:06.646984+0000","last_became_active":"2024-01-27T02:32:06.646591+0000","last_became_peered":"2024-01-27T02:32:06.646591+0000","last_unstale":"2024-01-27T02:32:06.646984+0000","last_undegraded":"2024-01-27T02:32:06.646984+0000","last_fullsized":"2024-01-27T02:32:06.646984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.636417+0000","last_change":"2024-01-27T02:32:06.636337+0000","last_active":"2024-01-27T02:32:06.636417+0000","last_peered":"2024-01-27T02:32:06.636417+0000","last_clean":"2024-01-27T02:32:06.636417+0000","last_became_active":"2024-01-27T02:32:06.636086+0000","last_became_peered":"2024-01-27T02:32:06.636086+0000","last_unstale":"2024-01-27T02:32:06.636417+0000","last_undegraded":"2024-01-27T02:32:06.636417+0000","last_fullsized":"2024-01-27T02:32:06.636417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194662,"statfs":{"total":200000000,"available":199333925,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":74,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195237,"statfs":{"total":200000000,"available":199923311,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194661,"statfs":{"total":200000000,"available":199333869,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195239,"statfs":{"total":200000000,"available":199924847,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":74,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195237,"statfs":{"total":200000000,"available":199923311,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924721,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195239,"statfs":{"total":200000000,"available":199924757,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]}]}],"pool_statfs":[]}} 2024-01-27T02:32:18.788 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:32:18.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.958+0000 7f7cc7f63700 1 Processor -- start 2024-01-27T02:32:18.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.958+0000 7f7cc7f63700 1 -- start start 2024-01-27T02:32:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.959+0000 7f7cc7f63700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7cc015e320 0x7f7cc0063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.959+0000 7f7cc7f63700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cc00b3810 0x7f7cc00ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.959+0000 7f7cc7f63700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cc00b3380 con 0x7f7cc015e6f0 2024-01-27T02:32:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.959+0000 7f7cc7f63700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cc00cea70 con 0x7f7cc015e320 2024-01-27T02:32:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.959+0000 7f7cc7f63700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc00b2a50 con 0x7f7cc00b3810 2024-01-27T02:32:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.959+0000 7f7cc54fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7cc015e6f0 0x7f7cc00cdd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51456/0 (socket says 172.21.15.7:51456) 2024-01-27T02:32:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.959+0000 7f7cc54fe700 1 -- 172.21.15.7:0/3763002421 learned_addr learned my addr 172.21.15.7:0/3763002421 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.960+0000 7f7cc6500700 1 --2- 172.21.15.7:0/3763002421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cc00b3810 0x7f7cc00ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.960+0000 7f7cc5cff700 1 --2- 172.21.15.7:0/3763002421 >> v2:172.21.15.188:3300/0 conn(0x7f7cc015e320 0x7f7cc0063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.960+0000 7f7cc4cfd700 1 -- 172.21.15.7:0/3763002421 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2853304877 0 0) 0x7f7cc00b3380 con 0x7f7cc015e6f0 2024-01-27T02:32:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.960+0000 7f7cc4cfd700 1 -- 172.21.15.7:0/3763002421 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cac003570 con 0x7f7cc015e6f0 2024-01-27T02:32:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.960+0000 7f7cc4cfd700 1 -- 172.21.15.7:0/3763002421 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 546667605 0 0) 0x7f7cac003570 con 0x7f7cc015e6f0 2024-01-27T02:32:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.960+0000 7f7cc4cfd700 1 -- 172.21.15.7:0/3763002421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cc00b3810 msgr2=0x7f7cc00ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.960+0000 7f7cc4cfd700 1 --2- 172.21.15.7:0/3763002421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cc00b3810 0x7f7cc00ce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.960+0000 7f7cc4cfd700 1 -- 172.21.15.7:0/3763002421 >> v2:172.21.15.188:3300/0 conn(0x7f7cc015e320 msgr2=0x7f7cc0063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.960+0000 7f7cc4cfd700 1 --2- 172.21.15.7:0/3763002421 >> v2:172.21.15.188:3300/0 conn(0x7f7cc015e320 0x7f7cc0063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.960+0000 7f7cc4cfd700 1 -- 172.21.15.7:0/3763002421 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc00b2d30 con 0x7f7cc015e6f0 2024-01-27T02:32:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.960+0000 7f7cc4cfd700 1 -- 172.21.15.7:0/3763002421 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f7cbc001470 con 0x7f7cc015e6f0 2024-01-27T02:32:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.960+0000 7f7cc4cfd700 1 -- 172.21.15.7:0/3763002421 >> v1:172.21.15.7:6789/0 conn(0x7f7cc015e6f0 legacy=0x7f7cc00cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.960+0000 7f7cc4cfd700 1 --2- 172.21.15.7:0/3763002421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cac003ae0 0x7f7cac003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.960+0000 7f7cc4cfd700 1 --2- 172.21.15.7:0/3763002421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cac004810 0x7f7cac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.960+0000 7f7cc4cfd700 1 --2- 172.21.15.7:0/3763002421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cc00b3810 0x7f7cac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.960+0000 7f7cc4cfd700 1 -- 172.21.15.7:0/3763002421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc00cea70 con 0x7f7cac004810 2024-01-27T02:32:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.960+0000 7f7cc4cfd700 1 -- 172.21.15.7:0/3763002421 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc00b2a50 con 0x7f7cac003ae0 2024-01-27T02:32:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.960+0000 7f7cc4cfd700 1 -- 172.21.15.7:0/3763002421 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac00bb10 con 0x7f7cc00b3810 2024-01-27T02:32:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.960+0000 7f7cc54fe700 1 --2- 172.21.15.7:0/3763002421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cac004810 0x7f7cac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.960+0000 7f7cc5cff700 1 --2- 172.21.15.7:0/3763002421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cac003ae0 0x7f7cac003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.960+0000 7f7cc6500700 1 --2- 172.21.15.7:0/3763002421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cc00b3810 0x7f7cac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.961+0000 7f7cc54fe700 1 -- 172.21.15.7:0/3763002421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cc00b3810 msgr2=0x7f7cac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.961+0000 7f7cc54fe700 1 --2- 172.21.15.7:0/3763002421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cc00b3810 0x7f7cac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.961+0000 7f7cc54fe700 1 -- 172.21.15.7:0/3763002421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cac003ae0 msgr2=0x7f7cac003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.961+0000 7f7cc54fe700 1 --2- 172.21.15.7:0/3763002421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cac003ae0 0x7f7cac003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.961+0000 7f7cc54fe700 1 -- 172.21.15.7:0/3763002421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac003570 con 0x7f7cac004810 2024-01-27T02:32:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.961+0000 7f7cc54fe700 1 --2- 172.21.15.7:0/3763002421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cac004810 0x7f7cac004c80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00c2a0 tx=0x7f7cc00b32f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72547021c1cb8d59 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.961+0000 7f7cc4cfd700 1 -- 172.21.15.7:0/3763002421 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc00cbd0 con 0x7f7cac004810 2024-01-27T02:32:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.961+0000 7f7cc4cfd700 1 -- 172.21.15.7:0/3763002421 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cbc00cd50 con 0x7f7cac004810 2024-01-27T02:32:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.961+0000 7f7cc4cfd700 1 -- 172.21.15.7:0/3763002421 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc005b70 con 0x7f7cac004810 2024-01-27T02:32:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.961+0000 7f7cc7f63700 1 -- 172.21.15.7:0/3763002421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cac004810 msgr2=0x7f7cac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.961+0000 7f7cc7f63700 1 --2- 172.21.15.7:0/3763002421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cac004810 0x7f7cac004c80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00c2a0 tx=0x7f7cc00b32f0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.961+0000 7f7cc6500700 1 -- 172.21.15.7:0/3763002421 reap_dead start 2024-01-27T02:32:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.962+0000 7f7cc7f63700 1 -- 172.21.15.7:0/3763002421 shutdown_connections 2024-01-27T02:32:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.962+0000 7f7cc7f63700 1 -- 172.21.15.7:0/3763002421 >> 172.21.15.7:0/3763002421 conn(0x7f7cc0068430 msgr2=0x7f7cc006be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.962+0000 7f7cc7f63700 1 -- 172.21.15.7:0/3763002421 shutdown_connections 2024-01-27T02:32:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.962+0000 7f7cc7f63700 1 -- 172.21.15.7:0/3763002421 wait complete. 2024-01-27T02:32:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.963+0000 7f7cc7f63700 1 Processor -- start 2024-01-27T02:32:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.963+0000 7f7cc7f63700 1 -- start start 2024-01-27T02:32:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cc7f63700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cac004810 0x7f7cc00c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cc7f63700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cc01501d0 0x7f7cc00c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cc7f63700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cc00c7da0 0x7f7cc00c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cc7f63700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc00c6640 con 0x7f7cac004810 2024-01-27T02:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cc7f63700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc00c8240 con 0x7f7cc00c7da0 2024-01-27T02:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cc7f63700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc00c5a30 con 0x7f7cc01501d0 2024-01-27T02:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cc5cff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cac004810 0x7f7cc00c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cc5cff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cac004810 0x7f7cc00c4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53912/0 (socket says 172.21.15.7:53912) 2024-01-27T02:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cc5cff700 1 -- 172.21.15.7:0/2619857030 learned_addr learned my addr 172.21.15.7:0/2619857030 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cc54fe700 1 --2- 172.21.15.7:0/2619857030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cc01501d0 0x7f7cc00c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cc6500700 1 --2- 172.21.15.7:0/2619857030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cc00c7da0 0x7f7cc00c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cc5cff700 1 -- 172.21.15.7:0/2619857030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cc01501d0 msgr2=0x7f7cc00c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cc5cff700 1 --2- 172.21.15.7:0/2619857030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cc01501d0 0x7f7cc00c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cc5cff700 1 -- 172.21.15.7:0/2619857030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cc00c7da0 msgr2=0x7f7cc00c29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cc5cff700 1 --2- 172.21.15.7:0/2619857030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cc00c7da0 0x7f7cc00c29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cc5cff700 1 -- 172.21.15.7:0/2619857030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc00c5d10 con 0x7f7cac004810 2024-01-27T02:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cc54fe700 1 --2- 172.21.15.7:0/2619857030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cc01501d0 0x7f7cc00c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cc6500700 1 --2- 172.21.15.7:0/2619857030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cc00c7da0 0x7f7cc00c29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cc5cff700 1 --2- 172.21.15.7:0/2619857030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cac004810 0x7f7cc00c4f00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4011d70 tx=0x7f7cb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cb1ffb700 1 -- 172.21.15.7:0/2619857030 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4009e20 con 0x7f7cac004810 2024-01-27T02:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cb1ffb700 1 -- 172.21.15.7:0/2619857030 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb400fc70 con 0x7f7cac004810 2024-01-27T02:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cb1ffb700 1 -- 172.21.15.7:0/2619857030 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4014440 con 0x7f7cac004810 2024-01-27T02:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.964+0000 7f7cc7f63700 1 -- 172.21.15.7:0/2619857030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc00c5ff0 con 0x7f7cac004810 2024-01-27T02:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.965+0000 7f7cc7f63700 1 -- 172.21.15.7:0/2619857030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc00c5440 con 0x7f7cac004810 2024-01-27T02:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.966+0000 7f7cc7f63700 1 -- 172.21.15.7:0/2619857030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc00c6490 con 0x7f7cac004810 2024-01-27T02:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.966+0000 7f7cb1ffb700 1 -- 172.21.15.7:0/2619857030 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7cb4014870 con 0x7f7cac004810 2024-01-27T02:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.966+0000 7f7cb1ffb700 1 --2- 172.21.15.7:0/2619857030 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ca4066f30 0x7f7ca40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.967+0000 7f7cb1ffb700 1 -- 172.21.15.7:0/2619857030 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7cb4018030 con 0x7f7cac004810 2024-01-27T02:32:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.967+0000 7f7cc54fe700 1 --2- 172.21.15.7:0/2619857030 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ca4066f30 0x7f7ca40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.967+0000 7f7cc54fe700 1 --2- 172.21.15.7:0/2619857030 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ca4066f30 0x7f7ca40693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7cc00cd390 tx=0x7f7cbc015000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:18.971+0000 7f7cb1ffb700 1 -- 172.21.15.7:0/2619857030 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7cb4052280 con 0x7f7cac004810 2024-01-27T02:32:19.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.169+0000 7f7cc7f63700 1 -- 172.21.15.7:0/2619857030 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7cc00ca440 con 0x7f7ca4066f30 2024-01-27T02:32:19.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.177+0000 7f7cb1ffb700 1 -- 172.21.15.7:0/2619857030 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260205 (secure 0 0 0) 0x7f7cc00ca440 con 0x7f7ca4066f30 2024-01-27T02:32:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:19.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.180+0000 7f7cc7f63700 1 -- 172.21.15.7:0/2619857030 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ca4066f30 msgr2=0x7f7ca40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.180+0000 7f7cc7f63700 1 --2- 172.21.15.7:0/2619857030 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ca4066f30 0x7f7ca40693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7cc00cd390 tx=0x7f7cbc015000 comp rx=0 tx=0).stop 2024-01-27T02:32:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.180+0000 7f7cc7f63700 1 -- 172.21.15.7:0/2619857030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cac004810 msgr2=0x7f7cc00c4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.180+0000 7f7cc7f63700 1 --2- 172.21.15.7:0/2619857030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cac004810 0x7f7cc00c4f00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4011d70 tx=0x7f7cb40091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.180+0000 7f7cc7f63700 1 -- 172.21.15.7:0/2619857030 shutdown_connections 2024-01-27T02:32:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.180+0000 7f7cc7f63700 1 --2- 172.21.15.7:0/2619857030 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ca4066f30 0x7f7ca40693e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.180+0000 7f7cc7f63700 1 --2- 172.21.15.7:0/2619857030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cc00c7da0 0x7f7cc00c29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.180+0000 7f7cc7f63700 1 --2- 172.21.15.7:0/2619857030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cac004810 0x7f7cc00c4f00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.180+0000 7f7cc7f63700 1 --2- 172.21.15.7:0/2619857030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cc01501d0 0x7f7cc00c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.180+0000 7f7cc7f63700 1 -- 172.21.15.7:0/2619857030 >> 172.21.15.7:0/2619857030 conn(0x7f7cc0068430 msgr2=0x7f7cc015efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.180+0000 7f7cc7f63700 1 -- 172.21.15.7:0/2619857030 shutdown_connections 2024-01-27T02:32:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.180+0000 7f7cc7f63700 1 -- 172.21.15.7:0/2619857030 wait complete. 2024-01-27T02:32:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:32:19.195 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-01-27T02:32:17.574441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1744,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560755,"statfs":{"total":1600000000,"available":1598213428,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002508"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.645906+0000","last_change":"2024-01-27T02:32:06.645807+0000","last_active":"2024-01-27T02:32:06.645906+0000","last_peered":"2024-01-27T02:32:06.645906+0000","last_clean":"2024-01-27T02:32:06.645906+0000","last_became_active":"2024-01-27T02:32:06.645346+0000","last_became_peered":"2024-01-27T02:32:06.645346+0000","last_unstale":"2024-01-27T02:32:06.645906+0000","last_undegraded":"2024-01-27T02:32:06.645906+0000","last_fullsized":"2024-01-27T02:32:06.645906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:44:24.628811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.639731+0000","last_change":"2024-01-27T02:32:06.639629+0000","last_active":"2024-01-27T02:32:06.639731+0000","last_peered":"2024-01-27T02:32:06.639731+0000","last_clean":"2024-01-27T02:32:06.639731+0000","last_became_active":"2024-01-27T02:32:06.638325+0000","last_became_peered":"2024-01-27T02:32:06.638325+0000","last_unstale":"2024-01-27T02:32:06.639731+0000","last_undegraded":"2024-01-27T02:32:06.639731+0000","last_fullsized":"2024-01-27T02:32:06.639731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:06.985231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.636966+0000","last_change":"2024-01-27T02:32:06.636541+0000","last_active":"2024-01-27T02:32:06.636966+0000","last_peered":"2024-01-27T02:32:06.636966+0000","last_clean":"2024-01-27T02:32:06.636966+0000","last_became_active":"2024-01-27T02:32:06.636201+0000","last_became_peered":"2024-01-27T02:32:06.636201+0000","last_unstale":"2024-01-27T02:32:06.636966+0000","last_undegraded":"2024-01-27T02:32:06.636966+0000","last_fullsized":"2024-01-27T02:32:06.636966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:44:13.569068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.649506+0000","last_change":"2024-01-27T02:32:06.649197+0000","last_active":"2024-01-27T02:32:06.649506+0000","last_peered":"2024-01-27T02:32:06.649506+0000","last_clean":"2024-01-27T02:32:06.649506+0000","last_became_active":"2024-01-27T02:32:06.648503+0000","last_became_peered":"2024-01-27T02:32:06.648503+0000","last_unstale":"2024-01-27T02:32:06.649506+0000","last_undegraded":"2024-01-27T02:32:06.649506+0000","last_fullsized":"2024-01-27T02:32:06.649506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:44:01.051017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.645086+0000","last_change":"2024-01-27T02:32:06.643865+0000","last_active":"2024-01-27T02:32:06.645086+0000","last_peered":"2024-01-27T02:32:06.645086+0000","last_clean":"2024-01-27T02:32:06.645086+0000","last_became_active":"2024-01-27T02:32:06.641009+0000","last_became_peered":"2024-01-27T02:32:06.641009+0000","last_unstale":"2024-01-27T02:32:06.645086+0000","last_undegraded":"2024-01-27T02:32:06.645086+0000","last_fullsized":"2024-01-27T02:32:06.645086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:48:03.053143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.647987+0000","last_change":"2024-01-27T02:32:06.647906+0000","last_active":"2024-01-27T02:32:06.647987+0000","last_peered":"2024-01-27T02:32:06.647987+0000","last_clean":"2024-01-27T02:32:06.647987+0000","last_became_active":"2024-01-27T02:32:06.647577+0000","last_became_peered":"2024-01-27T02:32:06.647577+0000","last_unstale":"2024-01-27T02:32:06.647987+0000","last_undegraded":"2024-01-27T02:32:06.647987+0000","last_fullsized":"2024-01-27T02:32:06.647987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:16:23.225540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.645142+0000","last_change":"2024-01-27T02:32:06.644654+0000","last_active":"2024-01-27T02:32:06.645142+0000","last_peered":"2024-01-27T02:32:06.645142+0000","last_clean":"2024-01-27T02:32:06.645142+0000","last_became_active":"2024-01-27T02:32:06.642941+0000","last_became_peered":"2024-01-27T02:32:06.642941+0000","last_unstale":"2024-01-27T02:32:06.645142+0000","last_undegraded":"2024-01-27T02:32:06.645142+0000","last_fullsized":"2024-01-27T02:32:06.645142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:43:05.599796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642600+0000","last_change":"2024-01-27T02:32:06.642517+0000","last_active":"2024-01-27T02:32:06.642600+0000","last_peered":"2024-01-27T02:32:06.642600+0000","last_clean":"2024-01-27T02:32:06.642600+0000","last_became_active":"2024-01-27T02:32:06.641240+0000","last_became_peered":"2024-01-27T02:32:06.641240+0000","last_unstale":"2024-01-27T02:32:06.642600+0000","last_undegraded":"2024-01-27T02:32:06.642600+0000","last_fullsized":"2024-01-27T02:32:06.642600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:11:54.230178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.650062+0000","last_change":"2024-01-27T02:32:06.649943+0000","last_active":"2024-01-27T02:32:06.650062+0000","last_peered":"2024-01-27T02:32:06.650062+0000","last_clean":"2024-01-27T02:32:06.650062+0000","last_became_active":"2024-01-27T02:32:06.649685+0000","last_became_peered":"2024-01-27T02:32:06.649685+0000","last_unstale":"2024-01-27T02:32:06.650062+0000","last_undegraded":"2024-01-27T02:32:06.650062+0000","last_fullsized":"2024-01-27T02:32:06.650062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:31:12.724216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.646882+0000","last_change":"2024-01-27T02:32:06.646820+0000","last_active":"2024-01-27T02:32:06.646882+0000","last_peered":"2024-01-27T02:32:06.646882+0000","last_clean":"2024-01-27T02:32:06.646882+0000","last_became_active":"2024-01-27T02:32:06.646586+0000","last_became_peered":"2024-01-27T02:32:06.646586+0000","last_unstale":"2024-01-27T02:32:06.646882+0000","last_undegraded":"2024-01-27T02:32:06.646882+0000","last_fullsized":"2024-01-27T02:32:06.646882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:06.985231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.649997+0000","last_change":"2024-01-27T02:32:06.649925+0000","last_active":"2024-01-27T02:32:06.649997+0000","last_peered":"2024-01-27T02:32:06.649997+0000","last_clean":"2024-01-27T02:32:06.649997+0000","last_became_active":"2024-01-27T02:32:06.649685+0000","last_became_peered":"2024-01-27T02:32:06.649685+0000","last_unstale":"2024-01-27T02:32:06.649997+0000","last_undegraded":"2024-01-27T02:32:06.649997+0000","last_fullsized":"2024-01-27T02:32:06.649997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:36:11.295485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642706+0000","last_change":"2024-01-27T02:32:06.642576+0000","last_active":"2024-01-27T02:32:06.642706+0000","last_peered":"2024-01-27T02:32:06.642706+0000","last_clean":"2024-01-27T02:32:06.642706+0000","last_became_active":"2024-01-27T02:32:06.641583+0000","last_became_peered":"2024-01-27T02:32:06.641583+0000","last_unstale":"2024-01-27T02:32:06.642706+0000","last_undegraded":"2024-01-27T02:32:06.642706+0000","last_fullsized":"2024-01-27T02:32:06.642706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:45.494048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642822+0000","last_change":"2024-01-27T02:32:06.642731+0000","last_active":"2024-01-27T02:32:06.642822+0000","last_peered":"2024-01-27T02:32:06.642822+0000","last_clean":"2024-01-27T02:32:06.642822+0000","last_became_active":"2024-01-27T02:32:06.641619+0000","last_became_peered":"2024-01-27T02:32:06.641619+0000","last_unstale":"2024-01-27T02:32:06.642822+0000","last_undegraded":"2024-01-27T02:32:06.642822+0000","last_fullsized":"2024-01-27T02:32:06.642822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:44:24.628811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.635576+0000","last_change":"2024-01-27T02:32:06.635401+0000","last_active":"2024-01-27T02:32:06.635576+0000","last_peered":"2024-01-27T02:32:06.635576+0000","last_clean":"2024-01-27T02:32:06.635576+0000","last_became_active":"2024-01-27T02:32:06.635136+0000","last_became_peered":"2024-01-27T02:32:06.635136+0000","last_unstale":"2024-01-27T02:32:06.635576+0000","last_undegraded":"2024-01-27T02:32:06.635576+0000","last_fullsized":"2024-01-27T02:32:06.635576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:17:03.351766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.639565+0000","last_change":"2024-01-27T02:32:06.639126+0000","last_active":"2024-01-27T02:32:06.639565+0000","last_peered":"2024-01-27T02:32:06.639565+0000","last_clean":"2024-01-27T02:32:06.639565+0000","last_became_active":"2024-01-27T02:32:06.638696+0000","last_became_peered":"2024-01-27T02:32:06.638696+0000","last_unstale":"2024-01-27T02:32:06.639565+0000","last_undegraded":"2024-01-27T02:32:06.639565+0000","last_fullsized":"2024-01-27T02:32:06.639565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:17:03.351766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.647702+0000","last_change":"2024-01-27T02:32:06.646066+0000","last_active":"2024-01-27T02:32:06.647702+0000","last_peered":"2024-01-27T02:32:06.647702+0000","last_clean":"2024-01-27T02:32:06.647702+0000","last_became_active":"2024-01-27T02:32:06.644886+0000","last_became_peered":"2024-01-27T02:32:06.644886+0000","last_unstale":"2024-01-27T02:32:06.647702+0000","last_undegraded":"2024-01-27T02:32:06.647702+0000","last_fullsized":"2024-01-27T02:32:06.647702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:11:54.230178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.649539+0000","last_change":"2024-01-27T02:32:06.649461+0000","last_active":"2024-01-27T02:32:06.649539+0000","last_peered":"2024-01-27T02:32:06.649539+0000","last_clean":"2024-01-27T02:32:06.649539+0000","last_became_active":"2024-01-27T02:32:06.648614+0000","last_became_peered":"2024-01-27T02:32:06.648614+0000","last_unstale":"2024-01-27T02:32:06.649539+0000","last_undegraded":"2024-01-27T02:32:06.649539+0000","last_fullsized":"2024-01-27T02:32:06.649539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:13:38.187522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.638138+0000","last_change":"2024-01-27T02:32:06.638050+0000","last_active":"2024-01-27T02:32:06.638138+0000","last_peered":"2024-01-27T02:32:06.638138+0000","last_clean":"2024-01-27T02:32:06.638138+0000","last_became_active":"2024-01-27T02:32:06.637390+0000","last_became_peered":"2024-01-27T02:32:06.637390+0000","last_unstale":"2024-01-27T02:32:06.638138+0000","last_undegraded":"2024-01-27T02:32:06.638138+0000","last_fullsized":"2024-01-27T02:32:06.638138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:07.924815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642381+0000","last_change":"2024-01-27T02:32:06.642245+0000","last_active":"2024-01-27T02:32:06.642381+0000","last_peered":"2024-01-27T02:32:06.642381+0000","last_clean":"2024-01-27T02:32:06.642381+0000","last_became_active":"2024-01-27T02:32:06.641079+0000","last_became_peered":"2024-01-27T02:32:06.641079+0000","last_unstale":"2024-01-27T02:32:06.642381+0000","last_undegraded":"2024-01-27T02:32:06.642381+0000","last_fullsized":"2024-01-27T02:32:06.642381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:38.023976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.802275+0000","last_change":"2024-01-27T02:32:05.624071+0000","last_active":"2024-01-27T02:32:06.802275+0000","last_peered":"2024-01-27T02:32:06.802275+0000","last_clean":"2024-01-27T02:32:06.802275+0000","last_became_active":"2024-01-27T02:32:05.623818+0000","last_became_peered":"2024-01-27T02:32:05.623818+0000","last_unstale":"2024-01-27T02:32:06.802275+0000","last_undegraded":"2024-01-27T02:32:06.802275+0000","last_fullsized":"2024-01-27T02:32:06.802275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:34:05.180050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.641588+0000","last_change":"2024-01-27T02:32:06.641507+0000","last_active":"2024-01-27T02:32:06.641588+0000","last_peered":"2024-01-27T02:32:06.641588+0000","last_clean":"2024-01-27T02:32:06.641588+0000","last_became_active":"2024-01-27T02:32:06.640928+0000","last_became_peered":"2024-01-27T02:32:06.640928+0000","last_unstale":"2024-01-27T02:32:06.641588+0000","last_undegraded":"2024-01-27T02:32:06.641588+0000","last_fullsized":"2024-01-27T02:32:06.641588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:32:24.198014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.803197+0000","last_change":"2024-01-27T02:32:05.624866+0000","last_active":"2024-01-27T02:32:06.803197+0000","last_peered":"2024-01-27T02:32:06.803197+0000","last_clean":"2024-01-27T02:32:06.803197+0000","last_became_active":"2024-01-27T02:32:05.624368+0000","last_became_peered":"2024-01-27T02:32:05.624368+0000","last_unstale":"2024-01-27T02:32:06.803197+0000","last_undegraded":"2024-01-27T02:32:06.803197+0000","last_fullsized":"2024-01-27T02:32:06.803197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642222+0000","last_change":"2024-01-27T02:32:06.642157+0000","last_active":"2024-01-27T02:32:06.642222+0000","last_peered":"2024-01-27T02:32:06.642222+0000","last_clean":"2024-01-27T02:32:06.642222+0000","last_became_active":"2024-01-27T02:32:06.641894+0000","last_became_peered":"2024-01-27T02:32:06.641894+0000","last_unstale":"2024-01-27T02:32:06.642222+0000","last_undegraded":"2024-01-27T02:32:06.642222+0000","last_fullsized":"2024-01-27T02:32:06.642222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.802467+0000","last_change":"2024-01-27T02:32:05.623255+0000","last_active":"2024-01-27T02:32:06.802467+0000","last_peered":"2024-01-27T02:32:06.802467+0000","last_clean":"2024-01-27T02:32:06.802467+0000","last_became_active":"2024-01-27T02:32:05.622821+0000","last_became_peered":"2024-01-27T02:32:05.622821+0000","last_unstale":"2024-01-27T02:32:06.802467+0000","last_undegraded":"2024-01-27T02:32:06.802467+0000","last_fullsized":"2024-01-27T02:32:06.802467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:33.333681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.648821+0000","last_change":"2024-01-27T02:32:06.648760+0000","last_active":"2024-01-27T02:32:06.648821+0000","last_peered":"2024-01-27T02:32:06.648821+0000","last_clean":"2024-01-27T02:32:06.648821+0000","last_became_active":"2024-01-27T02:32:06.648279+0000","last_became_peered":"2024-01-27T02:32:06.648279+0000","last_unstale":"2024-01-27T02:32:06.648821+0000","last_undegraded":"2024-01-27T02:32:06.648821+0000","last_fullsized":"2024-01-27T02:32:06.648821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:32:24.198014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.802483+0000","last_change":"2024-01-27T02:32:05.626014+0000","last_active":"2024-01-27T02:32:06.802483+0000","last_peered":"2024-01-27T02:32:06.802483+0000","last_clean":"2024-01-27T02:32:06.802483+0000","last_became_active":"2024-01-27T02:32:05.623176+0000","last_became_peered":"2024-01-27T02:32:05.623176+0000","last_unstale":"2024-01-27T02:32:06.802483+0000","last_undegraded":"2024-01-27T02:32:06.802483+0000","last_fullsized":"2024-01-27T02:32:06.802483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:37:59.246901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.641268+0000","last_change":"2024-01-27T02:32:06.637696+0000","last_active":"2024-01-27T02:32:06.641268+0000","last_peered":"2024-01-27T02:32:06.641268+0000","last_clean":"2024-01-27T02:32:06.641268+0000","last_became_active":"2024-01-27T02:32:06.637457+0000","last_became_peered":"2024-01-27T02:32:06.637457+0000","last_unstale":"2024-01-27T02:32:06.641268+0000","last_undegraded":"2024-01-27T02:32:06.641268+0000","last_fullsized":"2024-01-27T02:32:06.641268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:17:03.351766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.803103+0000","last_change":"2024-01-27T02:32:05.622086+0000","last_active":"2024-01-27T02:32:06.803103+0000","last_peered":"2024-01-27T02:32:06.803103+0000","last_clean":"2024-01-27T02:32:06.803103+0000","last_became_active":"2024-01-27T02:32:05.621465+0000","last_became_peered":"2024-01-27T02:32:05.621465+0000","last_unstale":"2024-01-27T02:32:06.803103+0000","last_undegraded":"2024-01-27T02:32:06.803103+0000","last_fullsized":"2024-01-27T02:32:06.803103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:04:28.890844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.644927+0000","last_change":"2024-01-27T02:32:06.644850+0000","last_active":"2024-01-27T02:32:06.644927+0000","last_peered":"2024-01-27T02:32:06.644927+0000","last_clean":"2024-01-27T02:32:06.644927+0000","last_became_active":"2024-01-27T02:32:06.644442+0000","last_became_peered":"2024-01-27T02:32:06.644442+0000","last_unstale":"2024-01-27T02:32:06.644927+0000","last_undegraded":"2024-01-27T02:32:06.644927+0000","last_fullsized":"2024-01-27T02:32:06.644927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:38:00.251484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.801277+0000","last_change":"2024-01-27T02:32:05.617530+0000","last_active":"2024-01-27T02:32:06.801277+0000","last_peered":"2024-01-27T02:32:06.801277+0000","last_clean":"2024-01-27T02:32:06.801277+0000","last_became_active":"2024-01-27T02:32:05.617290+0000","last_became_peered":"2024-01-27T02:32:05.617290+0000","last_unstale":"2024-01-27T02:32:06.801277+0000","last_undegraded":"2024-01-27T02:32:06.801277+0000","last_fullsized":"2024-01-27T02:32:06.801277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:41.575605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642446+0000","last_change":"2024-01-27T02:32:06.642339+0000","last_active":"2024-01-27T02:32:06.642446+0000","last_peered":"2024-01-27T02:32:06.642446+0000","last_clean":"2024-01-27T02:32:06.642446+0000","last_became_active":"2024-01-27T02:32:06.642027+0000","last_became_peered":"2024-01-27T02:32:06.642027+0000","last_unstale":"2024-01-27T02:32:06.642446+0000","last_undegraded":"2024-01-27T02:32:06.642446+0000","last_fullsized":"2024-01-27T02:32:06.642446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:45.494048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.628837+0000","last_change":"2024-01-27T02:32:05.623693+0000","last_active":"2024-01-27T02:32:06.628837+0000","last_peered":"2024-01-27T02:32:06.628837+0000","last_clean":"2024-01-27T02:32:06.628837+0000","last_became_active":"2024-01-27T02:32:05.623440+0000","last_became_peered":"2024-01-27T02:32:05.623440+0000","last_unstale":"2024-01-27T02:32:06.628837+0000","last_undegraded":"2024-01-27T02:32:06.628837+0000","last_fullsized":"2024-01-27T02:32:06.628837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:44:52.929407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.637714+0000","last_change":"2024-01-27T02:32:06.635720+0000","last_active":"2024-01-27T02:32:06.637714+0000","last_peered":"2024-01-27T02:32:06.637714+0000","last_clean":"2024-01-27T02:32:06.637714+0000","last_became_active":"2024-01-27T02:32:06.635457+0000","last_became_peered":"2024-01-27T02:32:06.635457+0000","last_unstale":"2024-01-27T02:32:06.637714+0000","last_undegraded":"2024-01-27T02:32:06.637714+0000","last_fullsized":"2024-01-27T02:32:06.637714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:36:11.295485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.633418+0000","last_change":"2024-01-27T02:32:05.619476+0000","last_active":"2024-01-27T02:32:06.633418+0000","last_peered":"2024-01-27T02:32:06.633418+0000","last_clean":"2024-01-27T02:32:06.633418+0000","last_became_active":"2024-01-27T02:32:05.619194+0000","last_became_peered":"2024-01-27T02:32:05.619194+0000","last_unstale":"2024-01-27T02:32:06.633418+0000","last_undegraded":"2024-01-27T02:32:06.633418+0000","last_fullsized":"2024-01-27T02:32:06.633418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.634597+0000","last_change":"2024-01-27T02:32:06.634504+0000","last_active":"2024-01-27T02:32:06.634597+0000","last_peered":"2024-01-27T02:32:06.634597+0000","last_clean":"2024-01-27T02:32:06.634597+0000","last_became_active":"2024-01-27T02:32:06.634220+0000","last_became_peered":"2024-01-27T02:32:06.634220+0000","last_unstale":"2024-01-27T02:32:06.634597+0000","last_undegraded":"2024-01-27T02:32:06.634597+0000","last_fullsized":"2024-01-27T02:32:06.634597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:45:37.035302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.635103+0000","last_change":"2024-01-27T02:32:05.623827+0000","last_active":"2024-01-27T02:32:06.635103+0000","last_peered":"2024-01-27T02:32:06.635103+0000","last_clean":"2024-01-27T02:32:06.635103+0000","last_became_active":"2024-01-27T02:32:05.623602+0000","last_became_peered":"2024-01-27T02:32:05.623602+0000","last_unstale":"2024-01-27T02:32:06.635103+0000","last_undegraded":"2024-01-27T02:32:06.635103+0000","last_fullsized":"2024-01-27T02:32:06.635103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.649471+0000","last_change":"2024-01-27T02:32:06.649029+0000","last_active":"2024-01-27T02:32:06.649471+0000","last_peered":"2024-01-27T02:32:06.649471+0000","last_clean":"2024-01-27T02:32:06.649471+0000","last_became_active":"2024-01-27T02:32:06.647963+0000","last_became_peered":"2024-01-27T02:32:06.647963+0000","last_unstale":"2024-01-27T02:32:06.649471+0000","last_undegraded":"2024-01-27T02:32:06.649471+0000","last_fullsized":"2024-01-27T02:32:06.649471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:43:05.599796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.635934+0000","last_change":"2024-01-27T02:32:05.621283+0000","last_active":"2024-01-27T02:32:06.635934+0000","last_peered":"2024-01-27T02:32:06.635934+0000","last_clean":"2024-01-27T02:32:06.635934+0000","last_became_active":"2024-01-27T02:32:05.621034+0000","last_became_peered":"2024-01-27T02:32:05.621034+0000","last_unstale":"2024-01-27T02:32:06.635934+0000","last_undegraded":"2024-01-27T02:32:06.635934+0000","last_fullsized":"2024-01-27T02:32:06.635934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:07.586396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.649134+0000","last_change":"2024-01-27T02:32:06.649052+0000","last_active":"2024-01-27T02:32:06.649134+0000","last_peered":"2024-01-27T02:32:06.649134+0000","last_clean":"2024-01-27T02:32:06.649134+0000","last_became_active":"2024-01-27T02:32:06.648837+0000","last_became_peered":"2024-01-27T02:32:06.648837+0000","last_unstale":"2024-01-27T02:32:06.649134+0000","last_undegraded":"2024-01-27T02:32:06.649134+0000","last_fullsized":"2024-01-27T02:32:06.649134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.802664+0000","last_change":"2024-01-27T02:32:05.620524+0000","last_active":"2024-01-27T02:32:06.802664+0000","last_peered":"2024-01-27T02:32:06.802664+0000","last_clean":"2024-01-27T02:32:06.802664+0000","last_became_active":"2024-01-27T02:32:05.620169+0000","last_became_peered":"2024-01-27T02:32:05.620169+0000","last_unstale":"2024-01-27T02:32:06.802664+0000","last_undegraded":"2024-01-27T02:32:06.802664+0000","last_fullsized":"2024-01-27T02:32:06.802664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.641576+0000","last_change":"2024-01-27T02:32:06.641250+0000","last_active":"2024-01-27T02:32:06.641576+0000","last_peered":"2024-01-27T02:32:06.641576+0000","last_clean":"2024-01-27T02:32:06.641576+0000","last_became_active":"2024-01-27T02:32:06.640947+0000","last_became_peered":"2024-01-27T02:32:06.640947+0000","last_unstale":"2024-01-27T02:32:06.641576+0000","last_undegraded":"2024-01-27T02:32:06.641576+0000","last_fullsized":"2024-01-27T02:32:06.641576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:34.338264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.804224+0000","last_change":"2024-01-27T02:32:05.625720+0000","last_active":"2024-01-27T02:32:06.804224+0000","last_peered":"2024-01-27T02:32:06.804224+0000","last_clean":"2024-01-27T02:32:06.804224+0000","last_became_active":"2024-01-27T02:32:05.625394+0000","last_became_peered":"2024-01-27T02:32:05.625394+0000","last_unstale":"2024-01-27T02:32:06.804224+0000","last_undegraded":"2024-01-27T02:32:06.804224+0000","last_fullsized":"2024-01-27T02:32:06.804224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.643130+0000","last_change":"2024-01-27T02:32:06.643058+0000","last_active":"2024-01-27T02:32:06.643130+0000","last_peered":"2024-01-27T02:32:06.643130+0000","last_clean":"2024-01-27T02:32:06.643130+0000","last_became_active":"2024-01-27T02:32:06.642818+0000","last_became_peered":"2024-01-27T02:32:06.642818+0000","last_unstale":"2024-01-27T02:32:06.643130+0000","last_undegraded":"2024-01-27T02:32:06.643130+0000","last_fullsized":"2024-01-27T02:32:06.643130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:15:06.036893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.801374+0000","last_change":"2024-01-27T02:32:05.619745+0000","last_active":"2024-01-27T02:32:06.801374+0000","last_peered":"2024-01-27T02:32:06.801374+0000","last_clean":"2024-01-27T02:32:06.801374+0000","last_became_active":"2024-01-27T02:32:05.619371+0000","last_became_peered":"2024-01-27T02:32:05.619371+0000","last_unstale":"2024-01-27T02:32:06.801374+0000","last_undegraded":"2024-01-27T02:32:06.801374+0000","last_fullsized":"2024-01-27T02:32:06.801374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:12:20.892542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.646863+0000","last_change":"2024-01-27T02:32:06.646797+0000","last_active":"2024-01-27T02:32:06.646863+0000","last_peered":"2024-01-27T02:32:06.646863+0000","last_clean":"2024-01-27T02:32:06.646863+0000","last_became_active":"2024-01-27T02:32:06.646253+0000","last_became_peered":"2024-01-27T02:32:06.646253+0000","last_unstale":"2024-01-27T02:32:06.646863+0000","last_undegraded":"2024-01-27T02:32:06.646863+0000","last_fullsized":"2024-01-27T02:32:06.646863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:58:54.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.643949+0000","last_change":"2024-01-27T02:32:05.626895+0000","last_active":"2024-01-27T02:32:06.643949+0000","last_peered":"2024-01-27T02:32:06.643949+0000","last_clean":"2024-01-27T02:32:06.643949+0000","last_became_active":"2024-01-27T02:32:05.626682+0000","last_became_peered":"2024-01-27T02:32:05.626682+0000","last_unstale":"2024-01-27T02:32:06.643949+0000","last_undegraded":"2024-01-27T02:32:06.643949+0000","last_fullsized":"2024-01-27T02:32:06.643949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:41.575605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.645112+0000","last_change":"2024-01-27T02:32:06.644466+0000","last_active":"2024-01-27T02:32:06.645112+0000","last_peered":"2024-01-27T02:32:06.645112+0000","last_clean":"2024-01-27T02:32:06.645112+0000","last_became_active":"2024-01-27T02:32:06.643544+0000","last_became_peered":"2024-01-27T02:32:06.643544+0000","last_unstale":"2024-01-27T02:32:06.645112+0000","last_undegraded":"2024-01-27T02:32:06.645112+0000","last_fullsized":"2024-01-27T02:32:06.645112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:22:27.441031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.635250+0000","last_change":"2024-01-27T02:32:05.619939+0000","last_active":"2024-01-27T02:32:06.635250+0000","last_peered":"2024-01-27T02:32:06.635250+0000","last_clean":"2024-01-27T02:32:06.635250+0000","last_became_active":"2024-01-27T02:32:05.619489+0000","last_became_peered":"2024-01-27T02:32:05.619489+0000","last_unstale":"2024-01-27T02:32:06.635250+0000","last_undegraded":"2024-01-27T02:32:06.635250+0000","last_fullsized":"2024-01-27T02:32:06.635250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:07.586396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.638890+0000","last_change":"2024-01-27T02:32:06.638821+0000","last_active":"2024-01-27T02:32:06.638890+0000","last_peered":"2024-01-27T02:32:06.638890+0000","last_clean":"2024-01-27T02:32:06.638890+0000","last_became_active":"2024-01-27T02:32:06.638486+0000","last_became_peered":"2024-01-27T02:32:06.638486+0000","last_unstale":"2024-01-27T02:32:06.638890+0000","last_undegraded":"2024-01-27T02:32:06.638890+0000","last_fullsized":"2024-01-27T02:32:06.638890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:45:37.035302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.806345+0000","last_change":"2024-01-27T02:32:05.621851+0000","last_active":"2024-01-27T02:32:06.806345+0000","last_peered":"2024-01-27T02:32:06.806345+0000","last_clean":"2024-01-27T02:32:06.806345+0000","last_became_active":"2024-01-27T02:32:05.621487+0000","last_became_peered":"2024-01-27T02:32:05.621487+0000","last_unstale":"2024-01-27T02:32:06.806345+0000","last_undegraded":"2024-01-27T02:32:06.806345+0000","last_fullsized":"2024-01-27T02:32:06.806345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:59:07.586396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.645157+0000","last_change":"2024-01-27T02:32:06.644806+0000","last_active":"2024-01-27T02:32:06.645157+0000","last_peered":"2024-01-27T02:32:06.645157+0000","last_clean":"2024-01-27T02:32:06.645157+0000","last_became_active":"2024-01-27T02:32:06.643567+0000","last_became_peered":"2024-01-27T02:32:06.643567+0000","last_unstale":"2024-01-27T02:32:06.645157+0000","last_undegraded":"2024-01-27T02:32:06.645157+0000","last_fullsized":"2024-01-27T02:32:06.645157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:44:01.051017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.801050+0000","last_change":"2024-01-27T02:32:05.623147+0000","last_active":"2024-01-27T02:32:06.801050+0000","last_peered":"2024-01-27T02:32:06.801050+0000","last_clean":"2024-01-27T02:32:06.801050+0000","last_became_active":"2024-01-27T02:32:05.622650+0000","last_became_peered":"2024-01-27T02:32:05.622650+0000","last_unstale":"2024-01-27T02:32:06.801050+0000","last_undegraded":"2024-01-27T02:32:06.801050+0000","last_fullsized":"2024-01-27T02:32:06.801050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:30:53.128829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.645128+0000","last_change":"2024-01-27T02:32:06.644626+0000","last_active":"2024-01-27T02:32:06.645128+0000","last_peered":"2024-01-27T02:32:06.645128+0000","last_clean":"2024-01-27T02:32:06.645128+0000","last_became_active":"2024-01-27T02:32:06.644149+0000","last_became_peered":"2024-01-27T02:32:06.644149+0000","last_unstale":"2024-01-27T02:32:06.645128+0000","last_undegraded":"2024-01-27T02:32:06.645128+0000","last_fullsized":"2024-01-27T02:32:06.645128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:13:38.187522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.632259+0000","last_change":"2024-01-27T02:32:05.621921+0000","last_active":"2024-01-27T02:32:06.632259+0000","last_peered":"2024-01-27T02:32:06.632259+0000","last_clean":"2024-01-27T02:32:06.632259+0000","last_became_active":"2024-01-27T02:32:05.620450+0000","last_became_peered":"2024-01-27T02:32:05.620450+0000","last_unstale":"2024-01-27T02:32:06.632259+0000","last_undegraded":"2024-01-27T02:32:06.632259+0000","last_fullsized":"2024-01-27T02:32:06.632259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:28:33.333681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.649453+0000","last_change":"2024-01-27T02:32:06.648997+0000","last_active":"2024-01-27T02:32:06.649453+0000","last_peered":"2024-01-27T02:32:06.649453+0000","last_clean":"2024-01-27T02:32:06.649453+0000","last_became_active":"2024-01-27T02:32:06.648180+0000","last_became_peered":"2024-01-27T02:32:06.648180+0000","last_unstale":"2024-01-27T02:32:06.649453+0000","last_undegraded":"2024-01-27T02:32:06.649453+0000","last_fullsized":"2024-01-27T02:32:06.649453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:15:38.764945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.636045+0000","last_change":"2024-01-27T02:32:05.622783+0000","last_active":"2024-01-27T02:32:06.636045+0000","last_peered":"2024-01-27T02:32:06.636045+0000","last_clean":"2024-01-27T02:32:06.636045+0000","last_became_active":"2024-01-27T02:32:05.622557+0000","last_became_peered":"2024-01-27T02:32:05.622557+0000","last_unstale":"2024-01-27T02:32:06.636045+0000","last_undegraded":"2024-01-27T02:32:06.636045+0000","last_fullsized":"2024-01-27T02:32:06.636045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.638811+0000","last_change":"2024-01-27T02:32:06.638626+0000","last_active":"2024-01-27T02:32:06.638811+0000","last_peered":"2024-01-27T02:32:06.638811+0000","last_clean":"2024-01-27T02:32:06.638811+0000","last_became_active":"2024-01-27T02:32:06.638382+0000","last_became_peered":"2024-01-27T02:32:06.638382+0000","last_unstale":"2024-01-27T02:32:06.638811+0000","last_undegraded":"2024-01-27T02:32:06.638811+0000","last_fullsized":"2024-01-27T02:32:06.638811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:31.534822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.635736+0000","last_change":"2024-01-27T02:32:06.635617+0000","last_active":"2024-01-27T02:32:06.635736+0000","last_peered":"2024-01-27T02:32:06.635736+0000","last_clean":"2024-01-27T02:32:06.635736+0000","last_became_active":"2024-01-27T02:32:06.635367+0000","last_became_peered":"2024-01-27T02:32:06.635367+0000","last_unstale":"2024-01-27T02:32:06.635736+0000","last_undegraded":"2024-01-27T02:32:06.635736+0000","last_fullsized":"2024-01-27T02:32:06.635736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:36:11.295485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.630282+0000","last_change":"2024-01-27T02:32:05.613447+0000","last_active":"2024-01-27T02:32:06.630282+0000","last_peered":"2024-01-27T02:32:06.630282+0000","last_clean":"2024-01-27T02:32:06.630282+0000","last_became_active":"2024-01-27T02:32:05.613203+0000","last_became_peered":"2024-01-27T02:32:05.613203+0000","last_unstale":"2024-01-27T02:32:06.630282+0000","last_undegraded":"2024-01-27T02:32:06.630282+0000","last_fullsized":"2024-01-27T02:32:06.630282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:51:41.575605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.648445+0000","last_change":"2024-01-27T02:32:06.648364+0000","last_active":"2024-01-27T02:32:06.648445+0000","last_peered":"2024-01-27T02:32:06.648445+0000","last_clean":"2024-01-27T02:32:06.648445+0000","last_became_active":"2024-01-27T02:32:06.648115+0000","last_became_peered":"2024-01-27T02:32:06.648115+0000","last_unstale":"2024-01-27T02:32:06.648445+0000","last_undegraded":"2024-01-27T02:32:06.648445+0000","last_fullsized":"2024-01-27T02:32:06.648445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:43:05.599796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.644051+0000","last_change":"2024-01-27T02:32:05.622673+0000","last_active":"2024-01-27T02:32:06.644051+0000","last_peered":"2024-01-27T02:32:06.644051+0000","last_clean":"2024-01-27T02:32:06.644051+0000","last_became_active":"2024-01-27T02:32:05.622195+0000","last_became_peered":"2024-01-27T02:32:05.622195+0000","last_unstale":"2024-01-27T02:32:06.644051+0000","last_undegraded":"2024-01-27T02:32:06.644051+0000","last_fullsized":"2024-01-27T02:32:06.644051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.637753+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:32:06.637753+0000","last_peered":"2024-01-27T02:32:06.637753+0000","last_clean":"2024-01-27T02:32:06.637753+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:32:06.637753+0000","last_undegraded":"2024-01-27T02:32:06.637753+0000","last_fullsized":"2024-01-27T02:32:06.637753+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.646119+0000","last_change":"2024-01-27T02:32:06.646043+0000","last_active":"2024-01-27T02:32:06.646119+0000","last_peered":"2024-01-27T02:32:06.646119+0000","last_clean":"2024-01-27T02:32:06.646119+0000","last_became_active":"2024-01-27T02:32:06.645836+0000","last_became_peered":"2024-01-27T02:32:06.645836+0000","last_unstale":"2024-01-27T02:32:06.646119+0000","last_undegraded":"2024-01-27T02:32:06.646119+0000","last_fullsized":"2024-01-27T02:32:06.646119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:05.115586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.636869+0000","last_change":"2024-01-27T02:32:05.626552+0000","last_active":"2024-01-27T02:32:06.636869+0000","last_peered":"2024-01-27T02:32:06.636869+0000","last_clean":"2024-01-27T02:32:06.636869+0000","last_became_active":"2024-01-27T02:32:05.626348+0000","last_became_peered":"2024-01-27T02:32:05.626348+0000","last_unstale":"2024-01-27T02:32:06.636869+0000","last_undegraded":"2024-01-27T02:32:06.636869+0000","last_fullsized":"2024-01-27T02:32:06.636869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:18.034484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.636979+0000","last_change":"2024-01-27T02:32:05.625885+0000","last_active":"2024-01-27T02:32:06.636979+0000","last_peered":"2024-01-27T02:32:06.636979+0000","last_clean":"2024-01-27T02:32:06.636979+0000","last_became_active":"2024-01-27T02:32:05.622692+0000","last_became_peered":"2024-01-27T02:32:05.622692+0000","last_unstale":"2024-01-27T02:32:06.636979+0000","last_undegraded":"2024-01-27T02:32:06.636979+0000","last_fullsized":"2024-01-27T02:32:06.636979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:06.920232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.639545+0000","last_change":"2024-01-27T02:32:06.638962+0000","last_active":"2024-01-27T02:32:06.639545+0000","last_peered":"2024-01-27T02:32:06.639545+0000","last_clean":"2024-01-27T02:32:06.639545+0000","last_became_active":"2024-01-27T02:32:06.638238+0000","last_became_peered":"2024-01-27T02:32:06.638238+0000","last_unstale":"2024-01-27T02:32:06.639545+0000","last_undegraded":"2024-01-27T02:32:06.639545+0000","last_fullsized":"2024-01-27T02:32:06.639545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:44:13.569068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-27T02:32:07.864093+0000","last_change":"2024-01-27T02:32:05.621014+0000","last_active":"2024-01-27T02:32:07.864093+0000","last_peered":"2024-01-27T02:32:07.864093+0000","last_clean":"2024-01-27T02:32:07.864093+0000","last_became_active":"2024-01-27T02:32:05.620775+0000","last_became_peered":"2024-01-27T02:32:05.620775+0000","last_unstale":"2024-01-27T02:32:07.864093+0000","last_undegraded":"2024-01-27T02:32:07.864093+0000","last_fullsized":"2024-01-27T02:32:07.864093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642354+0000","last_change":"2024-01-27T02:32:06.642266+0000","last_active":"2024-01-27T02:32:06.642354+0000","last_peered":"2024-01-27T02:32:06.642354+0000","last_clean":"2024-01-27T02:32:06.642354+0000","last_became_active":"2024-01-27T02:32:06.641998+0000","last_became_peered":"2024-01-27T02:32:06.641998+0000","last_unstale":"2024-01-27T02:32:06.642354+0000","last_undegraded":"2024-01-27T02:32:06.642354+0000","last_fullsized":"2024-01-27T02:32:06.642354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:58:54.122949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.802073+0000","last_change":"2024-01-27T02:32:05.624870+0000","last_active":"2024-01-27T02:32:06.802073+0000","last_peered":"2024-01-27T02:32:06.802073+0000","last_clean":"2024-01-27T02:32:06.802073+0000","last_became_active":"2024-01-27T02:32:05.624644+0000","last_became_peered":"2024-01-27T02:32:05.624644+0000","last_unstale":"2024-01-27T02:32:06.802073+0000","last_undegraded":"2024-01-27T02:32:06.802073+0000","last_fullsized":"2024-01-27T02:32:06.802073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.637689+0000","last_change":"2024-01-27T02:32:06.637609+0000","last_active":"2024-01-27T02:32:06.637689+0000","last_peered":"2024-01-27T02:32:06.637689+0000","last_clean":"2024-01-27T02:32:06.637689+0000","last_became_active":"2024-01-27T02:32:06.637282+0000","last_became_peered":"2024-01-27T02:32:06.637282+0000","last_unstale":"2024-01-27T02:32:06.637689+0000","last_undegraded":"2024-01-27T02:32:06.637689+0000","last_fullsized":"2024-01-27T02:32:06.637689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:06.985231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.637206+0000","last_change":"2024-01-27T02:32:05.621287+0000","last_active":"2024-01-27T02:32:06.637206+0000","last_peered":"2024-01-27T02:32:06.637206+0000","last_clean":"2024-01-27T02:32:06.637206+0000","last_became_active":"2024-01-27T02:32:05.620002+0000","last_became_peered":"2024-01-27T02:32:05.620002+0000","last_unstale":"2024-01-27T02:32:06.637206+0000","last_undegraded":"2024-01-27T02:32:06.637206+0000","last_fullsized":"2024-01-27T02:32:06.637206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:44.489465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.637891+0000","last_change":"2024-01-27T02:32:06.637830+0000","last_active":"2024-01-27T02:32:06.637891+0000","last_peered":"2024-01-27T02:32:06.637891+0000","last_clean":"2024-01-27T02:32:06.637891+0000","last_became_active":"2024-01-27T02:32:06.637387+0000","last_became_peered":"2024-01-27T02:32:06.637387+0000","last_unstale":"2024-01-27T02:32:06.637891+0000","last_undegraded":"2024-01-27T02:32:06.637891+0000","last_fullsized":"2024-01-27T02:32:06.637891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:47.819053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.632380+0000","last_change":"2024-01-27T02:32:05.623449+0000","last_active":"2024-01-27T02:32:06.632380+0000","last_peered":"2024-01-27T02:32:06.632380+0000","last_clean":"2024-01-27T02:32:06.632380+0000","last_became_active":"2024-01-27T02:32:05.623193+0000","last_became_peered":"2024-01-27T02:32:05.623193+0000","last_unstale":"2024-01-27T02:32:06.632380+0000","last_undegraded":"2024-01-27T02:32:06.632380+0000","last_fullsized":"2024-01-27T02:32:06.632380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:49:44.489465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.632750+0000","last_change":"2024-01-27T02:32:06.632652+0000","last_active":"2024-01-27T02:32:06.632750+0000","last_peered":"2024-01-27T02:32:06.632750+0000","last_clean":"2024-01-27T02:32:06.632750+0000","last_became_active":"2024-01-27T02:32:06.632366+0000","last_became_peered":"2024-01-27T02:32:06.632366+0000","last_unstale":"2024-01-27T02:32:06.632750+0000","last_undegraded":"2024-01-27T02:32:06.632750+0000","last_fullsized":"2024-01-27T02:32:06.632750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:06.985231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.631717+0000","last_change":"2024-01-27T02:32:05.625193+0000","last_active":"2024-01-27T02:32:06.631717+0000","last_peered":"2024-01-27T02:32:06.631717+0000","last_clean":"2024-01-27T02:32:06.631717+0000","last_became_active":"2024-01-27T02:32:05.624951+0000","last_became_peered":"2024-01-27T02:32:05.624951+0000","last_unstale":"2024-01-27T02:32:06.631717+0000","last_undegraded":"2024-01-27T02:32:06.631717+0000","last_fullsized":"2024-01-27T02:32:06.631717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:04:28.890844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.642383+0000","last_change":"2024-01-27T02:32:06.642267+0000","last_active":"2024-01-27T02:32:06.642383+0000","last_peered":"2024-01-27T02:32:06.642383+0000","last_clean":"2024-01-27T02:32:06.642383+0000","last_became_active":"2024-01-27T02:32:06.640004+0000","last_became_peered":"2024-01-27T02:32:06.640004+0000","last_unstale":"2024-01-27T02:32:06.642383+0000","last_undegraded":"2024-01-27T02:32:06.642383+0000","last_fullsized":"2024-01-27T02:32:06.642383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:45:37.035302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.801718+0000","last_change":"2024-01-27T02:32:05.625673+0000","last_active":"2024-01-27T02:32:06.801718+0000","last_peered":"2024-01-27T02:32:06.801718+0000","last_clean":"2024-01-27T02:32:06.801718+0000","last_became_active":"2024-01-27T02:32:05.623551+0000","last_became_peered":"2024-01-27T02:32:05.623551+0000","last_unstale":"2024-01-27T02:32:06.801718+0000","last_undegraded":"2024-01-27T02:32:06.801718+0000","last_fullsized":"2024-01-27T02:32:06.801718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.639544+0000","last_change":"2024-01-27T02:32:06.639489+0000","last_active":"2024-01-27T02:32:06.639544+0000","last_peered":"2024-01-27T02:32:06.639544+0000","last_clean":"2024-01-27T02:32:06.639544+0000","last_became_active":"2024-01-27T02:32:06.639246+0000","last_became_peered":"2024-01-27T02:32:06.639246+0000","last_unstale":"2024-01-27T02:32:06.639544+0000","last_undegraded":"2024-01-27T02:32:06.639544+0000","last_fullsized":"2024-01-27T02:32:06.639544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.629629+0000","last_change":"2024-01-27T02:32:05.620877+0000","last_active":"2024-01-27T02:32:06.629629+0000","last_peered":"2024-01-27T02:32:06.629629+0000","last_clean":"2024-01-27T02:32:06.629629+0000","last_became_active":"2024-01-27T02:32:05.620600+0000","last_became_peered":"2024-01-27T02:32:05.620600+0000","last_unstale":"2024-01-27T02:32:06.629629+0000","last_undegraded":"2024-01-27T02:32:06.629629+0000","last_fullsized":"2024-01-27T02:32:06.629629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:43:46.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.650085+0000","last_change":"2024-01-27T02:32:06.649956+0000","last_active":"2024-01-27T02:32:06.650085+0000","last_peered":"2024-01-27T02:32:06.650085+0000","last_clean":"2024-01-27T02:32:06.650085+0000","last_became_active":"2024-01-27T02:32:06.649759+0000","last_became_peered":"2024-01-27T02:32:06.649759+0000","last_unstale":"2024-01-27T02:32:06.650085+0000","last_undegraded":"2024-01-27T02:32:06.650085+0000","last_fullsized":"2024-01-27T02:32:06.650085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:56:39.515436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.801885+0000","last_change":"2024-01-27T02:32:05.619486+0000","last_active":"2024-01-27T02:32:06.801885+0000","last_peered":"2024-01-27T02:32:06.801885+0000","last_clean":"2024-01-27T02:32:06.801885+0000","last_became_active":"2024-01-27T02:32:05.619147+0000","last_became_peered":"2024-01-27T02:32:05.619147+0000","last_unstale":"2024-01-27T02:32:06.801885+0000","last_undegraded":"2024-01-27T02:32:06.801885+0000","last_fullsized":"2024-01-27T02:32:06.801885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:52.528242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.650011+0000","last_change":"2024-01-27T02:32:06.649606+0000","last_active":"2024-01-27T02:32:06.650011+0000","last_peered":"2024-01-27T02:32:06.650011+0000","last_clean":"2024-01-27T02:32:06.650011+0000","last_became_active":"2024-01-27T02:32:06.648805+0000","last_became_peered":"2024-01-27T02:32:06.648805+0000","last_unstale":"2024-01-27T02:32:06.650011+0000","last_undegraded":"2024-01-27T02:32:06.650011+0000","last_fullsized":"2024-01-27T02:32:06.650011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:31:12.724216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.804019+0000","last_change":"2024-01-27T02:32:05.613146+0000","last_active":"2024-01-27T02:32:06.804019+0000","last_peered":"2024-01-27T02:32:06.804019+0000","last_clean":"2024-01-27T02:32:06.804019+0000","last_became_active":"2024-01-27T02:32:05.612852+0000","last_became_peered":"2024-01-27T02:32:05.612852+0000","last_unstale":"2024-01-27T02:32:06.804019+0000","last_undegraded":"2024-01-27T02:32:06.804019+0000","last_fullsized":"2024-01-27T02:32:06.804019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:04.602626+0000","last_clean_scrub_stamp":"2024-01-27T02:32:04.602626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:46:30.530239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.641582+0000","last_change":"2024-01-27T02:32:06.641444+0000","last_active":"2024-01-27T02:32:06.641582+0000","last_peered":"2024-01-27T02:32:06.641582+0000","last_clean":"2024-01-27T02:32:06.641582+0000","last_became_active":"2024-01-27T02:32:06.640698+0000","last_became_peered":"2024-01-27T02:32:06.640698+0000","last_unstale":"2024-01-27T02:32:06.641582+0000","last_undegraded":"2024-01-27T02:32:06.641582+0000","last_fullsized":"2024-01-27T02:32:06.641582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:15:38.764945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.639806+0000","last_change":"2024-01-27T02:32:06.639727+0000","last_active":"2024-01-27T02:32:06.639806+0000","last_peered":"2024-01-27T02:32:06.639806+0000","last_clean":"2024-01-27T02:32:06.639806+0000","last_became_active":"2024-01-27T02:32:06.639457+0000","last_became_peered":"2024-01-27T02:32:06.639457+0000","last_unstale":"2024-01-27T02:32:06.639806+0000","last_undegraded":"2024-01-27T02:32:06.639806+0000","last_fullsized":"2024-01-27T02:32:06.639806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:17:03.351766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.646128+0000","last_change":"2024-01-27T02:32:06.646051+0000","last_active":"2024-01-27T02:32:06.646128+0000","last_peered":"2024-01-27T02:32:06.646128+0000","last_clean":"2024-01-27T02:32:06.646128+0000","last_became_active":"2024-01-27T02:32:06.645857+0000","last_became_peered":"2024-01-27T02:32:06.645857+0000","last_unstale":"2024-01-27T02:32:06.646128+0000","last_undegraded":"2024-01-27T02:32:06.646128+0000","last_fullsized":"2024-01-27T02:32:06.646128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:19.039067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.648193+0000","last_change":"2024-01-27T02:32:06.648063+0000","last_active":"2024-01-27T02:32:06.648193+0000","last_peered":"2024-01-27T02:32:06.648193+0000","last_clean":"2024-01-27T02:32:06.648193+0000","last_became_active":"2024-01-27T02:32:06.647524+0000","last_became_peered":"2024-01-27T02:32:06.647524+0000","last_unstale":"2024-01-27T02:32:06.648193+0000","last_undegraded":"2024-01-27T02:32:06.648193+0000","last_fullsized":"2024-01-27T02:32:06.648193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:56:39.515436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.639513+0000","last_change":"2024-01-27T02:32:06.639430+0000","last_active":"2024-01-27T02:32:06.639513+0000","last_peered":"2024-01-27T02:32:06.639513+0000","last_clean":"2024-01-27T02:32:06.639513+0000","last_became_active":"2024-01-27T02:32:06.639149+0000","last_became_peered":"2024-01-27T02:32:06.639149+0000","last_unstale":"2024-01-27T02:32:06.639513+0000","last_undegraded":"2024-01-27T02:32:06.639513+0000","last_fullsized":"2024-01-27T02:32:06.639513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:36:11.295485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.641457+0000","last_change":"2024-01-27T02:32:06.640757+0000","last_active":"2024-01-27T02:32:06.641457+0000","last_peered":"2024-01-27T02:32:06.641457+0000","last_clean":"2024-01-27T02:32:06.641457+0000","last_became_active":"2024-01-27T02:32:06.637783+0000","last_became_peered":"2024-01-27T02:32:06.637783+0000","last_unstale":"2024-01-27T02:32:06.641457+0000","last_undegraded":"2024-01-27T02:32:06.641457+0000","last_fullsized":"2024-01-27T02:32:06.641457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:44:13.569068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.633658+0000","last_change":"2024-01-27T02:32:06.633366+0000","last_active":"2024-01-27T02:32:06.633658+0000","last_peered":"2024-01-27T02:32:06.633658+0000","last_clean":"2024-01-27T02:32:06.633658+0000","last_became_active":"2024-01-27T02:32:06.633117+0000","last_became_peered":"2024-01-27T02:32:06.633117+0000","last_unstale":"2024-01-27T02:32:06.633658+0000","last_undegraded":"2024-01-27T02:32:06.633658+0000","last_fullsized":"2024-01-27T02:32:06.633658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.645347+0000","last_change":"2024-01-27T02:32:06.645277+0000","last_active":"2024-01-27T02:32:06.645347+0000","last_peered":"2024-01-27T02:32:06.645347+0000","last_clean":"2024-01-27T02:32:06.645347+0000","last_became_active":"2024-01-27T02:32:06.645042+0000","last_became_peered":"2024-01-27T02:32:06.645042+0000","last_unstale":"2024-01-27T02:32:06.645347+0000","last_undegraded":"2024-01-27T02:32:06.645347+0000","last_fullsized":"2024-01-27T02:32:06.645347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:44:13.569068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.637735+0000","last_change":"2024-01-27T02:32:06.636359+0000","last_active":"2024-01-27T02:32:06.637735+0000","last_peered":"2024-01-27T02:32:06.637735+0000","last_clean":"2024-01-27T02:32:06.637735+0000","last_became_active":"2024-01-27T02:32:06.636084+0000","last_became_peered":"2024-01-27T02:32:06.636084+0000","last_unstale":"2024-01-27T02:32:06.637735+0000","last_undegraded":"2024-01-27T02:32:06.637735+0000","last_fullsized":"2024-01-27T02:32:06.637735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:06:21.885744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.648901+0000","last_change":"2024-01-27T02:32:06.648767+0000","last_active":"2024-01-27T02:32:06.648901+0000","last_peered":"2024-01-27T02:32:06.648901+0000","last_clean":"2024-01-27T02:32:06.648901+0000","last_became_active":"2024-01-27T02:32:06.647799+0000","last_became_peered":"2024-01-27T02:32:06.647799+0000","last_unstale":"2024-01-27T02:32:06.648901+0000","last_undegraded":"2024-01-27T02:32:06.648901+0000","last_fullsized":"2024-01-27T02:32:06.648901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:36:11.295485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.641482+0000","last_change":"2024-01-27T02:32:06.640928+0000","last_active":"2024-01-27T02:32:06.641482+0000","last_peered":"2024-01-27T02:32:06.641482+0000","last_clean":"2024-01-27T02:32:06.641482+0000","last_became_active":"2024-01-27T02:32:06.638110+0000","last_became_peered":"2024-01-27T02:32:06.638110+0000","last_unstale":"2024-01-27T02:32:06.641482+0000","last_undegraded":"2024-01-27T02:32:06.641482+0000","last_fullsized":"2024-01-27T02:32:06.641482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:17:03.351766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.649522+0000","last_change":"2024-01-27T02:32:06.649302+0000","last_active":"2024-01-27T02:32:06.649522+0000","last_peered":"2024-01-27T02:32:06.649522+0000","last_clean":"2024-01-27T02:32:06.649522+0000","last_became_active":"2024-01-27T02:32:06.648315+0000","last_became_peered":"2024-01-27T02:32:06.648315+0000","last_unstale":"2024-01-27T02:32:06.649522+0000","last_undegraded":"2024-01-27T02:32:06.649522+0000","last_fullsized":"2024-01-27T02:32:06.649522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:44:24.628811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.646984+0000","last_change":"2024-01-27T02:32:06.646935+0000","last_active":"2024-01-27T02:32:06.646984+0000","last_peered":"2024-01-27T02:32:06.646984+0000","last_clean":"2024-01-27T02:32:06.646984+0000","last_became_active":"2024-01-27T02:32:06.646591+0000","last_became_peered":"2024-01-27T02:32:06.646591+0000","last_unstale":"2024-01-27T02:32:06.646984+0000","last_undegraded":"2024-01-27T02:32:06.646984+0000","last_fullsized":"2024-01-27T02:32:06.646984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:18.114549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-27T02:32:06.636417+0000","last_change":"2024-01-27T02:32:06.636337+0000","last_active":"2024-01-27T02:32:06.636417+0000","last_peered":"2024-01-27T02:32:06.636417+0000","last_clean":"2024-01-27T02:32:06.636417+0000","last_became_active":"2024-01-27T02:32:06.636086+0000","last_became_peered":"2024-01-27T02:32:06.636086+0000","last_unstale":"2024-01-27T02:32:06.636417+0000","last_undegraded":"2024-01-27T02:32:06.636417+0000","last_fullsized":"2024-01-27T02:32:06.636417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:05.607209+0000","last_clean_scrub_stamp":"2024-01-27T02:32:05.607209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:08:47.607068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194662,"statfs":{"total":200000000,"available":199333925,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":74,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195237,"statfs":{"total":200000000,"available":199923311,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194661,"statfs":{"total":200000000,"available":199333869,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195239,"statfs":{"total":200000000,"available":199924847,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":74,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195237,"statfs":{"total":200000000,"available":199923311,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924721,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195239,"statfs":{"total":200000000,"available":199924757,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]}]}],"pool_statfs":[]}} 2024-01-27T02:32:19.199 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-01-27T02:32:19.199 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-01-27T02:32:19.199 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-01-27T02:32:19.199 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-01-27T02:32:19.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.367+0000 7f241321b700 1 Processor -- start 2024-01-27T02:32:19.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.367+0000 7f241321b700 1 -- start start 2024-01-27T02:32:19.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.368+0000 7f241321b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f240c15d750 0x7f240c0b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:19.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.368+0000 7f241321b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f240c069200 0x7f240c0c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:19.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.368+0000 7f241321b700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f240c068660 con 0x7f240c0cf7c0 2024-01-27T02:32:19.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.368+0000 7f241321b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f240c0ce9f0 con 0x7f240c069200 2024-01-27T02:32:19.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.368+0000 7f241321b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f240c067a50 con 0x7f240c15d750 2024-01-27T02:32:19.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.368+0000 7f24117b8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f240c069200 0x7f240c0c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:19.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.368+0000 7f24117b8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f240c069200 0x7f240c0c9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53928/0 (socket says 172.21.15.7:53928) 2024-01-27T02:32:19.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.368+0000 7f24117b8700 1 -- 172.21.15.7:0/1969789010 learned_addr learned my addr 172.21.15.7:0/1969789010 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:19.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.368+0000 7f2410fb7700 1 --2- 172.21.15.7:0/1969789010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f240c15d750 0x7f240c0b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:19.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.368+0000 7f24117b8700 1 -- 172.21.15.7:0/1969789010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f240c15d750 msgr2=0x7f240c0b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:19.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.368+0000 7f24117b8700 1 --2- 172.21.15.7:0/1969789010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f240c15d750 0x7f240c0b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:19.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.368+0000 7f24117b8700 1 -- 172.21.15.7:0/1969789010 >> v1:172.21.15.188:6789/0 conn(0x7f240c0cf7c0 legacy=0x7f240c068af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:19.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.368+0000 7f24117b8700 1 -- 172.21.15.7:0/1969789010 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240c067d30 con 0x7f240c069200 2024-01-27T02:32:19.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.368+0000 7f24117b8700 1 --2- 172.21.15.7:0/1969789010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240c069200 0x7f240c0c9d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2404009f20 tx=0x7f2404009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ef80c508205d05 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:19.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.369+0000 7f240b7fe700 1 -- 172.21.15.7:0/1969789010 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240400e070 con 0x7f240c069200 2024-01-27T02:32:19.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.369+0000 7f240b7fe700 1 -- 172.21.15.7:0/1969789010 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240400b9d0 con 0x7f240c069200 2024-01-27T02:32:19.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.369+0000 7f240b7fe700 1 -- 172.21.15.7:0/1969789010 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24040092e0 con 0x7f240c069200 2024-01-27T02:32:19.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.369+0000 7f241321b700 1 -- 172.21.15.7:0/1969789010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240c069200 msgr2=0x7f240c0c9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:19.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.369+0000 7f241321b700 1 --2- 172.21.15.7:0/1969789010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240c069200 0x7f240c0c9d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2404009f20 tx=0x7f2404009840 comp rx=0 tx=0).stop 2024-01-27T02:32:19.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.369+0000 7f241321b700 1 -- 172.21.15.7:0/1969789010 shutdown_connections 2024-01-27T02:32:19.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.369+0000 7f241321b700 1 --2- 172.21.15.7:0/1969789010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240c069200 0x7f240c0c9d90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:19.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.369+0000 7f241321b700 1 --2- 172.21.15.7:0/1969789010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f240c15d750 0x7f240c0b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:19.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.369+0000 7f241321b700 1 -- 172.21.15.7:0/1969789010 >> 172.21.15.7:0/1969789010 conn(0x7f240c0711c0 msgr2=0x7f240c0ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:19.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.369+0000 7f241321b700 1 -- 172.21.15.7:0/1969789010 shutdown_connections 2024-01-27T02:32:19.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.369+0000 7f241321b700 1 -- 172.21.15.7:0/1969789010 wait complete. 2024-01-27T02:32:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.370+0000 7f241321b700 1 Processor -- start 2024-01-27T02:32:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.371+0000 7f241321b700 1 -- start start 2024-01-27T02:32:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.371+0000 7f241321b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f240c069200 0x7f240c0c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.371+0000 7f241321b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240c15d750 0x7f240c14c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.371+0000 7f241321b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f240c0b4630 0x7f240c0c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.371+0000 7f241321b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c0b2740 con 0x7f240c15d750 2024-01-27T02:32:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.371+0000 7f241321b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c0b2130 con 0x7f240c0b4630 2024-01-27T02:32:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.371+0000 7f241321b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f240c0c3880 con 0x7f240c069200 2024-01-27T02:32:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.372+0000 7f240bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240c15d750 0x7f240c14c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.372+0000 7f2410fb7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f240c069200 0x7f240c0c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.372+0000 7f2410fb7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f240c069200 0x7f240c0c69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:54312/0 (socket says 172.21.15.7:54312) 2024-01-27T02:32:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.372+0000 7f2410fb7700 1 -- 172.21.15.7:0/2150130005 learned_addr learned my addr 172.21.15.7:0/2150130005 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.372+0000 7f24117b8700 1 --2- 172.21.15.7:0/2150130005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f240c0b4630 0x7f240c0c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.372+0000 7f240bfff700 1 -- 172.21.15.7:0/2150130005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f240c069200 msgr2=0x7f240c0c69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.372+0000 7f240bfff700 1 --2- 172.21.15.7:0/2150130005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f240c069200 0x7f240c0c69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.372+0000 7f240bfff700 1 -- 172.21.15.7:0/2150130005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f240c0b4630 msgr2=0x7f240c0c2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.372+0000 7f240bfff700 1 --2- 172.21.15.7:0/2150130005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f240c0b4630 0x7f240c0c2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.372+0000 7f240bfff700 1 -- 172.21.15.7:0/2150130005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2404009ae0 con 0x7f240c15d750 2024-01-27T02:32:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.372+0000 7f240bfff700 1 --2- 172.21.15.7:0/2150130005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240c15d750 0x7f240c14c1e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f240000c590 tx=0x7f2400007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.372+0000 7f2408ff9700 1 -- 172.21.15.7:0/2150130005 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2400011590 con 0x7f240c15d750 2024-01-27T02:32:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.372+0000 7f2408ff9700 1 -- 172.21.15.7:0/2150130005 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2400011710 con 0x7f240c15d750 2024-01-27T02:32:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.372+0000 7f241321b700 1 -- 172.21.15.7:0/2150130005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f240c0b23d0 con 0x7f240c15d750 2024-01-27T02:32:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.372+0000 7f2408ff9700 1 -- 172.21.15.7:0/2150130005 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2400011ad0 con 0x7f240c15d750 2024-01-27T02:32:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.373+0000 7f241321b700 1 -- 172.21.15.7:0/2150130005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240c14c720 con 0x7f240c15d750 2024-01-27T02:32:19.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.373+0000 7f241321b700 1 -- 172.21.15.7:0/2150130005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f240c04c710 con 0x7f240c15d750 2024-01-27T02:32:19.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.374+0000 7f2408ff9700 1 -- 172.21.15.7:0/2150130005 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f240001b3b0 con 0x7f240c15d750 2024-01-27T02:32:19.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.374+0000 7f2408ff9700 1 --2- 172.21.15.7:0/2150130005 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f23f8066f30 0x7f23f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:19.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.374+0000 7f2408ff9700 1 -- 172.21.15.7:0/2150130005 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f24000838e0 con 0x7f240c15d750 2024-01-27T02:32:19.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.375+0000 7f2410fb7700 1 --2- 172.21.15.7:0/2150130005 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f23f8066f30 0x7f23f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:19.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.375+0000 7f2410fb7700 1 --2- 172.21.15.7:0/2150130005 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f23f8066f30 0x7f23f80693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f240c0c67c0 tx=0x7f23fc009120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:19.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.379+0000 7f2408ff9700 1 -- 172.21.15.7:0/2150130005 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2400050b50 con 0x7f240c15d750 2024-01-27T02:32:19.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.629+0000 7f241321b700 1 -- 172.21.15.7:0/2150130005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f240c068010 con 0x7f240c15d750 2024-01-27T02:32:19.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.630+0000 7f2408ff9700 1 -- 172.21.15.7:0/2150130005 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f2400017020 con 0x7f240c15d750 2024-01-27T02:32:19.632 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:19.632 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-01-27T02:32:19.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.632+0000 7f241321b700 1 -- 172.21.15.7:0/2150130005 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f23f8066f30 msgr2=0x7f23f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:19.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.632+0000 7f241321b700 1 --2- 172.21.15.7:0/2150130005 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f23f8066f30 0x7f23f80693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f240c0c67c0 tx=0x7f23fc009120 comp rx=0 tx=0).stop 2024-01-27T02:32:19.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.632+0000 7f241321b700 1 -- 172.21.15.7:0/2150130005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240c15d750 msgr2=0x7f240c14c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:19.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.632+0000 7f241321b700 1 --2- 172.21.15.7:0/2150130005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240c15d750 0x7f240c14c1e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f240000c590 tx=0x7f2400007ec0 comp rx=0 tx=0).stop 2024-01-27T02:32:19.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.632+0000 7f241321b700 1 -- 172.21.15.7:0/2150130005 shutdown_connections 2024-01-27T02:32:19.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.632+0000 7f241321b700 1 --2- 172.21.15.7:0/2150130005 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f23f8066f30 0x7f23f80693e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:19.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.632+0000 7f241321b700 1 --2- 172.21.15.7:0/2150130005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f240c0b4630 0x7f240c0c2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.632+0000 7f241321b700 1 --2- 172.21.15.7:0/2150130005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f240c069200 0x7f240c0c69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.632+0000 7f241321b700 1 --2- 172.21.15.7:0/2150130005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240c15d750 0x7f240c14c1e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.632+0000 7f241321b700 1 -- 172.21.15.7:0/2150130005 >> 172.21.15.7:0/2150130005 conn(0x7f240c0711c0 msgr2=0x7f240c15c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.632+0000 7f241321b700 1 -- 172.21.15.7:0/2150130005 shutdown_connections 2024-01-27T02:32:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.632+0000 7f241321b700 1 -- 172.21.15.7:0/2150130005 wait complete. 2024-01-27T02:32:19.647 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-01-27T02:32:19.648 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:32:19.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.779+0000 7fa634239700 1 Processor -- start 2024-01-27T02:32:19.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.779+0000 7fa634239700 1 -- start start 2024-01-27T02:32:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.780+0000 7fa634239700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa62c1566a0 0x7fa62c0a7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.780+0000 7fa634239700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa62c156a70 0x7fa62c0aee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.780+0000 7fa634239700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62c0afd30 con 0x7fa62c0709e0 2024-01-27T02:32:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.780+0000 7fa634239700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa62c06df80 con 0x7fa62c1566a0 2024-01-27T02:32:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.780+0000 7fa634239700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa62c0af770 con 0x7fa62c156a70 2024-01-27T02:32:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.780+0000 7fa631fd5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa62c1566a0 0x7fa62c0a7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.780+0000 7fa6327d6700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa62c0709e0 0x7fa62c0acc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51468/0 (socket says 172.21.15.7:51468) 2024-01-27T02:32:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.780+0000 7fa6327d6700 1 -- 172.21.15.7:0/2406942057 learned_addr learned my addr 172.21.15.7:0/2406942057 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.780+0000 7fa631fd5700 1 --2- 172.21.15.7:0/2406942057 >> v2:172.21.15.7:3300/0 conn(0x7fa62c1566a0 0x7fa62c0a7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53938/0 (socket says 172.21.15.7:53938) 2024-01-27T02:32:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.780+0000 7fa6317d4700 1 --2- 172.21.15.7:0/2406942057 >> v2:172.21.15.188:3300/0 conn(0x7fa62c156a70 0x7fa62c0aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.780+0000 7fa631fd5700 1 -- 172.21.15.7:0/2406942057 >> v2:172.21.15.188:3300/0 conn(0x7fa62c156a70 msgr2=0x7fa62c0aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.780+0000 7fa631fd5700 1 --2- 172.21.15.7:0/2406942057 >> v2:172.21.15.188:3300/0 conn(0x7fa62c156a70 0x7fa62c0aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.780+0000 7fa631fd5700 1 -- 172.21.15.7:0/2406942057 >> v1:172.21.15.7:6789/0 conn(0x7fa62c0709e0 legacy=0x7fa62c0acc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.780+0000 7fa631fd5700 1 -- 172.21.15.7:0/2406942057 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62c0afa50 con 0x7fa62c1566a0 2024-01-27T02:32:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.780+0000 7fa630fd3700 1 -- 172.21.15.7:0/2406942057 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3379715791 0 0) 0x7fa62c0afd30 con 0x7fa62c0709e0 2024-01-27T02:32:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.781+0000 7fa631fd5700 1 --2- 172.21.15.7:0/2406942057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa62c1566a0 0x7fa62c0a7e20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa62c06d4f0 tx=0x7fa6200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3d3c59bd98a36f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.781+0000 7fa630fd3700 1 -- 172.21.15.7:0/2406942057 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62000b6b0 con 0x7fa62c1566a0 2024-01-27T02:32:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.781+0000 7fa630fd3700 1 -- 172.21.15.7:0/2406942057 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62000b810 con 0x7fa62c1566a0 2024-01-27T02:32:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.781+0000 7fa630fd3700 1 -- 172.21.15.7:0/2406942057 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62000bbb0 con 0x7fa62c1566a0 2024-01-27T02:32:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.781+0000 7fa634239700 1 -- 172.21.15.7:0/2406942057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa62c1566a0 msgr2=0x7fa62c0a7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.781+0000 7fa634239700 1 --2- 172.21.15.7:0/2406942057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa62c1566a0 0x7fa62c0a7e20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa62c06d4f0 tx=0x7fa6200091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.781+0000 7fa634239700 1 -- 172.21.15.7:0/2406942057 shutdown_connections 2024-01-27T02:32:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.781+0000 7fa634239700 1 --2- 172.21.15.7:0/2406942057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa62c1566a0 0x7fa62c0a7e20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.781+0000 7fa634239700 1 --2- 172.21.15.7:0/2406942057 >> v2:172.21.15.188:3300/0 conn(0x7fa62c156a70 0x7fa62c0aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:19.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.781+0000 7fa634239700 1 -- 172.21.15.7:0/2406942057 >> 172.21.15.7:0/2406942057 conn(0x7fa62c0679b0 msgr2=0x7fa62c067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:19.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.782+0000 7fa634239700 1 -- 172.21.15.7:0/2406942057 shutdown_connections 2024-01-27T02:32:19.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.782+0000 7fa634239700 1 -- 172.21.15.7:0/2406942057 wait complete. 2024-01-27T02:32:19.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.783+0000 7fa634239700 1 Processor -- start 2024-01-27T02:32:19.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.783+0000 7fa634239700 1 -- start start 2024-01-27T02:32:19.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.784+0000 7fa634239700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa62c0709e0 0x7fa62c160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:19.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.784+0000 7fa634239700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa62c1566a0 0x7fa62c06eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:19.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.784+0000 7fa634239700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa62c156a70 0x7fa62c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:19.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.784+0000 7fa634239700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c159540 con 0x7fa62c0709e0 2024-01-27T02:32:19.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.784+0000 7fa634239700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c0ab5b0 con 0x7fa62c1566a0 2024-01-27T02:32:19.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.784+0000 7fa634239700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c158c10 con 0x7fa62c156a70 2024-01-27T02:32:19.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.784+0000 7fa631fd5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa62c0709e0 0x7fa62c160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:19.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.784+0000 7fa631fd5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa62c0709e0 0x7fa62c160b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53952/0 (socket says 172.21.15.7:53952) 2024-01-27T02:32:19.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.784+0000 7fa631fd5700 1 -- 172.21.15.7:0/4237093443 learned_addr learned my addr 172.21.15.7:0/4237093443 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:19.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.784+0000 7fa6327d6700 1 --2- 172.21.15.7:0/4237093443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa62c156a70 0x7fa62c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:19.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.784+0000 7fa6317d4700 1 --2- 172.21.15.7:0/4237093443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa62c1566a0 0x7fa62c06eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:19.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.784+0000 7fa631fd5700 1 -- 172.21.15.7:0/4237093443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa62c156a70 msgr2=0x7fa62c06f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:19.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.784+0000 7fa631fd5700 1 --2- 172.21.15.7:0/4237093443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa62c156a70 0x7fa62c06f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:19.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.784+0000 7fa631fd5700 1 -- 172.21.15.7:0/4237093443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa62c1566a0 msgr2=0x7fa62c06eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:19.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.784+0000 7fa631fd5700 1 --2- 172.21.15.7:0/4237093443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa62c1566a0 0x7fa62c06eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:19.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.784+0000 7fa631fd5700 1 -- 172.21.15.7:0/4237093443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa620009420 con 0x7fa62c0709e0 2024-01-27T02:32:19.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.784+0000 7fa631fd5700 1 --2- 172.21.15.7:0/4237093443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa62c0709e0 0x7fa62c160b40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa62c068eb0 tx=0x7fa620017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:19.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.784+0000 7fa61dffb700 1 -- 172.21.15.7:0/4237093443 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa620011420 con 0x7fa62c0709e0 2024-01-27T02:32:19.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.784+0000 7fa61dffb700 1 -- 172.21.15.7:0/4237093443 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa620011580 con 0x7fa62c0709e0 2024-01-27T02:32:19.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.784+0000 7fa61dffb700 1 -- 172.21.15.7:0/4237093443 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa620011940 con 0x7fa62c0709e0 2024-01-27T02:32:19.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.784+0000 7fa634239700 1 -- 172.21.15.7:0/4237093443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa62c158ef0 con 0x7fa62c0709e0 2024-01-27T02:32:19.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.785+0000 7fa634239700 1 -- 172.21.15.7:0/4237093443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa62c161080 con 0x7fa62c0709e0 2024-01-27T02:32:19.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.786+0000 7fa634239700 1 -- 172.21.15.7:0/4237093443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa62c04c710 con 0x7fa62c0709e0 2024-01-27T02:32:19.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.786+0000 7fa61dffb700 1 -- 172.21.15.7:0/4237093443 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa620011d30 con 0x7fa62c0709e0 2024-01-27T02:32:19.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.786+0000 7fa61dffb700 1 --2- 172.21.15.7:0/4237093443 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa610066f30 0x7fa6100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:19.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.787+0000 7fa61dffb700 1 -- 172.21.15.7:0/4237093443 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa620085b10 con 0x7fa62c0709e0 2024-01-27T02:32:19.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.787+0000 7fa6317d4700 1 --2- 172.21.15.7:0/4237093443 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa610066f30 0x7fa6100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:19.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.787+0000 7fa6317d4700 1 --2- 172.21.15.7:0/4237093443 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa610066f30 0x7fa6100693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa62c09ae20 tx=0x7fa628008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:19.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.791+0000 7fa61dffb700 1 -- 172.21.15.7:0/4237093443 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa620052ce0 con 0x7fa62c0709e0 2024-01-27T02:32:20.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.998+0000 7fa634239700 1 -- 172.21.15.7:0/4237093443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa62c06dc10 con 0x7fa62c0709e0 2024-01-27T02:32:20.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:19.999+0000 7fa61dffb700 1 -- 172.21.15.7:0/4237093443 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15302 (secure 0 0 0) 0x7fa620015020 con 0x7fa62c0709e0 2024-01-27T02:32:20.001 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:20.001 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:32:07.627822+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:32:03.675611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-27T02:32:05.018885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:32:20.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.002+0000 7fa634239700 1 -- 172.21.15.7:0/4237093443 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa610066f30 msgr2=0x7fa6100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.002+0000 7fa634239700 1 --2- 172.21.15.7:0/4237093443 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa610066f30 0x7fa6100693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa62c09ae20 tx=0x7fa628008040 comp rx=0 tx=0).stop 2024-01-27T02:32:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.002+0000 7fa634239700 1 -- 172.21.15.7:0/4237093443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa62c0709e0 msgr2=0x7fa62c160b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.002+0000 7fa634239700 1 --2- 172.21.15.7:0/4237093443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa62c0709e0 0x7fa62c160b40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa62c068eb0 tx=0x7fa620017000 comp rx=0 tx=0).stop 2024-01-27T02:32:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.002+0000 7fa634239700 1 -- 172.21.15.7:0/4237093443 shutdown_connections 2024-01-27T02:32:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.002+0000 7fa634239700 1 --2- 172.21.15.7:0/4237093443 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa610066f30 0x7fa6100693e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.002+0000 7fa634239700 1 --2- 172.21.15.7:0/4237093443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa62c156a70 0x7fa62c06f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.002+0000 7fa634239700 1 --2- 172.21.15.7:0/4237093443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa62c0709e0 0x7fa62c160b40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.002+0000 7fa634239700 1 --2- 172.21.15.7:0/4237093443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa62c1566a0 0x7fa62c06eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.002+0000 7fa634239700 1 -- 172.21.15.7:0/4237093443 >> 172.21.15.7:0/4237093443 conn(0x7fa62c0679b0 msgr2=0x7fa62c06e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.002+0000 7fa634239700 1 -- 172.21.15.7:0/4237093443 shutdown_connections 2024-01-27T02:32:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.002+0000 7fa634239700 1 -- 172.21.15.7:0/4237093443 wait complete. 2024-01-27T02:32:20.017 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-27T02:32:03.675611+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-27T02:32:05.018885+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-27T02:32:20.017 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:32:20.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.151+0000 7ff3ccf17700 1 Processor -- start 2024-01-27T02:32:20.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.151+0000 7ff3ccf17700 1 -- start start 2024-01-27T02:32:20.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.152+0000 7ff3ccf17700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3c81416c0 0x7ff3c80cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:20.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.152+0000 7ff3ccf17700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff3c8141a90 0x7ff3c80cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:20.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.152+0000 7ff3ccf17700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c815c7d0 con 0x7ff3c80cab00 2024-01-27T02:32:20.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.152+0000 7ff3ccf17700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c80cd9c0 con 0x7ff3c8141a90 2024-01-27T02:32:20.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.152+0000 7ff3ccf17700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c80c9640 con 0x7ff3c81416c0 2024-01-27T02:32:20.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.152+0000 7ff3c659c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3c81416c0 0x7ff3c80cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:20.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.152+0000 7ff3bdd9b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff3c8141a90 0x7ff3c80cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:20.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.152+0000 7ff3c6d9d700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7ff3c80cab00 0x7ff3c80caf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:45472/0 (socket says 172.21.15.7:45472) 2024-01-27T02:32:20.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.152+0000 7ff3c6d9d700 1 -- 172.21.15.7:0/4029809764 learned_addr learned my addr 172.21.15.7:0/4029809764 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:20.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.153+0000 7ff3c5d9b700 1 -- 172.21.15.7:0/4029809764 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583600661 0 0) 0x7ff3c815c7d0 con 0x7ff3c80cab00 2024-01-27T02:32:20.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.153+0000 7ff3c5d9b700 1 -- 172.21.15.7:0/4029809764 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3b4003570 con 0x7ff3c80cab00 2024-01-27T02:32:20.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.153+0000 7ff3bdd9b700 1 -- 172.21.15.7:0/4029809764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3c81416c0 msgr2=0x7ff3c80cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:20.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.153+0000 7ff3bdd9b700 1 --2- 172.21.15.7:0/4029809764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3c81416c0 0x7ff3c80cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.153+0000 7ff3bdd9b700 1 -- 172.21.15.7:0/4029809764 >> v1:172.21.15.188:6789/0 conn(0x7ff3c80cab00 legacy=0x7ff3c80caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:20.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.153+0000 7ff3bdd9b700 1 -- 172.21.15.7:0/4029809764 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c815bbc0 con 0x7ff3c8141a90 2024-01-27T02:32:20.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.153+0000 7ff3c659c700 1 --2- 172.21.15.7:0/4029809764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3c81416c0 0x7ff3c80cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:32:20.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.153+0000 7ff3bdd9b700 1 --2- 172.21.15.7:0/4029809764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3c8141a90 0x7ff3c80cbc20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00098e0 tx=0x7ff3b00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8335579a394172a5 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:20.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.153+0000 7ff3c5d9b700 1 -- 172.21.15.7:0/4029809764 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0011630 con 0x7ff3c8141a90 2024-01-27T02:32:20.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.153+0000 7ff3c5d9b700 1 -- 172.21.15.7:0/4029809764 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b0011790 con 0x7ff3c8141a90 2024-01-27T02:32:20.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.153+0000 7ff3c5d9b700 1 -- 172.21.15.7:0/4029809764 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0011b30 con 0x7ff3c8141a90 2024-01-27T02:32:20.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.154+0000 7ff3ccf17700 1 -- 172.21.15.7:0/4029809764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3c8141a90 msgr2=0x7ff3c80cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:20.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.154+0000 7ff3ccf17700 1 --2- 172.21.15.7:0/4029809764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3c8141a90 0x7ff3c80cbc20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00098e0 tx=0x7ff3b00091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.154+0000 7ff3ccf17700 1 -- 172.21.15.7:0/4029809764 shutdown_connections 2024-01-27T02:32:20.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.154+0000 7ff3ccf17700 1 --2- 172.21.15.7:0/4029809764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3c81416c0 0x7ff3c80cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.154+0000 7ff3ccf17700 1 --2- 172.21.15.7:0/4029809764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3c8141a90 0x7ff3c80cbc20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.154+0000 7ff3ccf17700 1 -- 172.21.15.7:0/4029809764 >> 172.21.15.7:0/4029809764 conn(0x7ff3c8063580 msgr2=0x7ff3c8063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:20.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.154+0000 7ff3ccf17700 1 -- 172.21.15.7:0/4029809764 shutdown_connections 2024-01-27T02:32:20.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.154+0000 7ff3ccf17700 1 -- 172.21.15.7:0/4029809764 wait complete. 2024-01-27T02:32:20.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.155+0000 7ff3ccf17700 1 Processor -- start 2024-01-27T02:32:20.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.155+0000 7ff3ccf17700 1 -- start start 2024-01-27T02:32:20.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.156+0000 7ff3ccf17700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3c80cab00 0x7ff3c80c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:20.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.156+0000 7ff3ccf17700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3c81416c0 0x7ff3c8068af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:20.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.156+0000 7ff3ccf17700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3c8141a90 0x7ff3c813cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:20.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.156+0000 7ff3ccf17700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c80686d0 con 0x7ff3c8141a90 2024-01-27T02:32:20.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.156+0000 7ff3ccf17700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c80c8bd0 con 0x7ff3c80cab00 2024-01-27T02:32:20.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.156+0000 7ff3ccf17700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c8067a50 con 0x7ff3c81416c0 2024-01-27T02:32:20.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.156+0000 7ff3c6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3c8141a90 0x7ff3c813cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:20.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.156+0000 7ff3c6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3c8141a90 0x7ff3c813cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53954/0 (socket says 172.21.15.7:53954) 2024-01-27T02:32:20.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.156+0000 7ff3c6d9d700 1 -- 172.21.15.7:0/1193695096 learned_addr learned my addr 172.21.15.7:0/1193695096 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:20.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.156+0000 7ff3bdd9b700 1 --2- 172.21.15.7:0/1193695096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3c81416c0 0x7ff3c8068af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:20.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.156+0000 7ff3c659c700 1 --2- 172.21.15.7:0/1193695096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3c80cab00 0x7ff3c80c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:20.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.157+0000 7ff3c6d9d700 1 -- 172.21.15.7:0/1193695096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3c81416c0 msgr2=0x7ff3c8068af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:20.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.157+0000 7ff3c6d9d700 1 --2- 172.21.15.7:0/1193695096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3c81416c0 0x7ff3c8068af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.157+0000 7ff3c6d9d700 1 -- 172.21.15.7:0/1193695096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3c80cab00 msgr2=0x7ff3c80c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:20.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.157+0000 7ff3c6d9d700 1 --2- 172.21.15.7:0/1193695096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3c80cab00 0x7ff3c80c8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.157+0000 7ff3c6d9d700 1 -- 172.21.15.7:0/1193695096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b80093c0 con 0x7ff3c8141a90 2024-01-27T02:32:20.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.157+0000 7ff3c6d9d700 1 --2- 172.21.15.7:0/1193695096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3c8141a90 0x7ff3c813cb40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0001460 tx=0x7ff3c0001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:20.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.157+0000 7ff3beffd700 1 -- 172.21.15.7:0/1193695096 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c000a870 con 0x7ff3c8141a90 2024-01-27T02:32:20.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.157+0000 7ff3beffd700 1 -- 172.21.15.7:0/1193695096 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c000a9d0 con 0x7ff3c8141a90 2024-01-27T02:32:20.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.157+0000 7ff3beffd700 1 -- 172.21.15.7:0/1193695096 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c000ad50 con 0x7ff3c8141a90 2024-01-27T02:32:20.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.157+0000 7ff3ccf17700 1 -- 172.21.15.7:0/1193695096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b0009420 con 0x7ff3c8141a90 2024-01-27T02:32:20.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.157+0000 7ff3ccf17700 1 -- 172.21.15.7:0/1193695096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c8068010 con 0x7ff3c8141a90 2024-01-27T02:32:20.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.158+0000 7ff3ccf17700 1 -- 172.21.15.7:0/1193695096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c8068810 con 0x7ff3c8141a90 2024-01-27T02:32:20.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.158+0000 7ff3beffd700 1 -- 172.21.15.7:0/1193695096 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff3c0007300 con 0x7ff3c8141a90 2024-01-27T02:32:20.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.159+0000 7ff3beffd700 1 --2- 172.21.15.7:0/1193695096 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff3a8066f30 0x7ff3a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:20.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.159+0000 7ff3beffd700 1 -- 172.21.15.7:0/1193695096 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff3c00847d0 con 0x7ff3c8141a90 2024-01-27T02:32:20.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.159+0000 7ff3c659c700 1 --2- 172.21.15.7:0/1193695096 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff3a8066f30 0x7ff3a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:20.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.159+0000 7ff3c659c700 1 --2- 172.21.15.7:0/1193695096 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff3a8066f30 0x7ff3a80693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009f30 tx=0x7ff3b8009120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:20.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.163+0000 7ff3beffd700 1 -- 172.21.15.7:0/1193695096 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3c8068810 con 0x7ff3c8141a90 2024-01-27T02:32:20.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.365+0000 7ff3ccf17700 1 -- 172.21.15.7:0/1193695096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3c8068810 con 0x7ff3c8141a90 2024-01-27T02:32:20.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.366+0000 7ff3beffd700 1 -- 172.21.15.7:0/1193695096 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff3c8068810 con 0x7ff3c8141a90 2024-01-27T02:32:20.367 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:32:20.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.368+0000 7ff3ccf17700 1 -- 172.21.15.7:0/1193695096 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff3a8066f30 msgr2=0x7ff3a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:20.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.368+0000 7ff3ccf17700 1 --2- 172.21.15.7:0/1193695096 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff3a8066f30 0x7ff3a80693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009f30 tx=0x7ff3b8009120 comp rx=0 tx=0).stop 2024-01-27T02:32:20.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.368+0000 7ff3ccf17700 1 -- 172.21.15.7:0/1193695096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3c8141a90 msgr2=0x7ff3c813cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:20.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.368+0000 7ff3ccf17700 1 --2- 172.21.15.7:0/1193695096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3c8141a90 0x7ff3c813cb40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0001460 tx=0x7ff3c0001a40 comp rx=0 tx=0).stop 2024-01-27T02:32:20.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.368+0000 7ff3ccf17700 1 -- 172.21.15.7:0/1193695096 shutdown_connections 2024-01-27T02:32:20.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.368+0000 7ff3ccf17700 1 --2- 172.21.15.7:0/1193695096 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff3a8066f30 0x7ff3a80693e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.368+0000 7ff3ccf17700 1 --2- 172.21.15.7:0/1193695096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3c8141a90 0x7ff3c813cb40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.368+0000 7ff3ccf17700 1 --2- 172.21.15.7:0/1193695096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3c80cab00 0x7ff3c80c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.368+0000 7ff3ccf17700 1 --2- 172.21.15.7:0/1193695096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3c81416c0 0x7ff3c8068af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.368+0000 7ff3ccf17700 1 -- 172.21.15.7:0/1193695096 >> 172.21.15.7:0/1193695096 conn(0x7ff3c8063580 msgr2=0x7ff3c80cc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:20.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.368+0000 7ff3ccf17700 1 -- 172.21.15.7:0/1193695096 shutdown_connections 2024-01-27T02:32:20.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.368+0000 7ff3ccf17700 1 -- 172.21.15.7:0/1193695096 wait complete. 2024-01-27T02:32:20.383 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-27T02:32:20.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.516+0000 7fa0c64a2700 1 Processor -- start 2024-01-27T02:32:20.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.516+0000 7fa0c64a2700 1 -- start start 2024-01-27T02:32:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.517+0000 7fa0c64a2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c01416c0 0x7fa0c00cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.517+0000 7fa0c64a2700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa0c00cb210 0x7fa0c00cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.517+0000 7fa0c64a2700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c015c7d0 con 0x7fa0c0141a90 2024-01-27T02:32:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.517+0000 7fa0c64a2700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c00cd9c0 con 0x7fa0c00cb210 2024-01-27T02:32:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.517+0000 7fa0c64a2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c00c9640 con 0x7fa0c01416c0 2024-01-27T02:32:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.517+0000 7fa0c4a3f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa0c00cb210 0x7fa0c00cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.517+0000 7fa0bffff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c01416c0 0x7fa0c00cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.517+0000 7fa0bf7fe700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fa0c0141a90 0x7fa0c00cab00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:45482/0 (socket says 172.21.15.7:45482) 2024-01-27T02:32:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.517+0000 7fa0bf7fe700 1 -- 172.21.15.7:0/363914599 learned_addr learned my addr 172.21.15.7:0/363914599 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.518+0000 7fa0c4a3f700 1 -- 172.21.15.7:0/363914599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c01416c0 msgr2=0x7fa0c00cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.518+0000 7fa0c4a3f700 1 --2- 172.21.15.7:0/363914599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c01416c0 0x7fa0c00cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.518+0000 7fa0c4a3f700 1 -- 172.21.15.7:0/363914599 >> v1:172.21.15.188:6789/0 conn(0x7fa0c0141a90 legacy=0x7fa0c00cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.518+0000 7fa0c4a3f700 1 -- 172.21.15.7:0/363914599 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c015bbc0 con 0x7fa0c00cb210 2024-01-27T02:32:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.518+0000 7fa0beffd700 1 -- 172.21.15.7:0/363914599 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 212863022 0 0) 0x7fa0c015c7d0 con 0x7fa0c0141a90 2024-01-27T02:32:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.518+0000 7fa0c4a3f700 1 --2- 172.21.15.7:0/363914599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c00cb210 0x7fa0c00cbc20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009e60 tx=0x7fa0b8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d0b8e122f0ca36c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.518+0000 7fa0beffd700 1 -- 172.21.15.7:0/363914599 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b800e070 con 0x7fa0c00cb210 2024-01-27T02:32:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.518+0000 7fa0beffd700 1 -- 172.21.15.7:0/363914599 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b800bb60 con 0x7fa0c00cb210 2024-01-27T02:32:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.518+0000 7fa0beffd700 1 -- 172.21.15.7:0/363914599 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8012430 con 0x7fa0c00cb210 2024-01-27T02:32:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.519+0000 7fa0c64a2700 1 -- 172.21.15.7:0/363914599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c00cb210 msgr2=0x7fa0c00cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.519+0000 7fa0c64a2700 1 --2- 172.21.15.7:0/363914599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c00cb210 0x7fa0c00cbc20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009e60 tx=0x7fa0b8009780 comp rx=0 tx=0).stop 2024-01-27T02:32:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.519+0000 7fa0c64a2700 1 -- 172.21.15.7:0/363914599 shutdown_connections 2024-01-27T02:32:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.519+0000 7fa0c64a2700 1 --2- 172.21.15.7:0/363914599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c00cb210 0x7fa0c00cbc20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.519+0000 7fa0c64a2700 1 --2- 172.21.15.7:0/363914599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c01416c0 0x7fa0c00cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.519+0000 7fa0c64a2700 1 -- 172.21.15.7:0/363914599 >> 172.21.15.7:0/363914599 conn(0x7fa0c0063580 msgr2=0x7fa0c0063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.519+0000 7fa0c64a2700 1 -- 172.21.15.7:0/363914599 shutdown_connections 2024-01-27T02:32:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.519+0000 7fa0c64a2700 1 -- 172.21.15.7:0/363914599 wait complete. 2024-01-27T02:32:20.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.520+0000 7fa0c64a2700 1 Processor -- start 2024-01-27T02:32:20.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.520+0000 7fa0c64a2700 1 -- start start 2024-01-27T02:32:20.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.521+0000 7fa0c64a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c00cb210 0x7fa0c00c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:20.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.521+0000 7fa0c64a2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c01416c0 0x7fa0c0068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:20.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.521+0000 7fa0c64a2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c0141a90 0x7fa0c00c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:20.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.521+0000 7fa0c64a2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c0158cd0 con 0x7fa0c00cb210 2024-01-27T02:32:20.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.521+0000 7fa0c64a2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c0158fb0 con 0x7fa0c01416c0 2024-01-27T02:32:20.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.521+0000 7fa0c64a2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c0068bd0 con 0x7fa0c0141a90 2024-01-27T02:32:20.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.521+0000 7fa0bffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c00cb210 0x7fa0c00c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:20.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.521+0000 7fa0bffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c00cb210 0x7fa0c00c61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53970/0 (socket says 172.21.15.7:53970) 2024-01-27T02:32:20.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.521+0000 7fa0bffff700 1 -- 172.21.15.7:0/2160373122 learned_addr learned my addr 172.21.15.7:0/2160373122 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:20.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.521+0000 7fa0bf7fe700 1 --2- 172.21.15.7:0/2160373122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c01416c0 0x7fa0c0068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:20.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.521+0000 7fa0c4a3f700 1 --2- 172.21.15.7:0/2160373122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c0141a90 0x7fa0c00c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:20.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.521+0000 7fa0bffff700 1 -- 172.21.15.7:0/2160373122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c0141a90 msgr2=0x7fa0c00c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:20.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.521+0000 7fa0bffff700 1 --2- 172.21.15.7:0/2160373122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c0141a90 0x7fa0c00c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.521+0000 7fa0bffff700 1 -- 172.21.15.7:0/2160373122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c01416c0 msgr2=0x7fa0c0068370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:20.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.521+0000 7fa0bffff700 1 --2- 172.21.15.7:0/2160373122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c01416c0 0x7fa0c0068370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.522+0000 7fa0bffff700 1 -- 172.21.15.7:0/2160373122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b8009a20 con 0x7fa0c00cb210 2024-01-27T02:32:20.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.522+0000 7fa0bf7fe700 1 --2- 172.21.15.7:0/2160373122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c01416c0 0x7fa0c0068370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:20.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.522+0000 7fa0bffff700 1 --2- 172.21.15.7:0/2160373122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c00cb210 0x7fa0c00c61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000ecb0 tx=0x7fa0b0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:20.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.522+0000 7fa0a7fff700 1 -- 172.21.15.7:0/2160373122 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b000c9d0 con 0x7fa0c00cb210 2024-01-27T02:32:20.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.522+0000 7fa0a7fff700 1 -- 172.21.15.7:0/2160373122 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b000cb30 con 0x7fa0c00cb210 2024-01-27T02:32:20.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.522+0000 7fa0c64a2700 1 -- 172.21.15.7:0/2160373122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c0068eb0 con 0x7fa0c00cb210 2024-01-27T02:32:20.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.522+0000 7fa0a7fff700 1 -- 172.21.15.7:0/2160373122 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b000ceb0 con 0x7fa0c00cb210 2024-01-27T02:32:20.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.522+0000 7fa0c64a2700 1 -- 172.21.15.7:0/2160373122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c00688b0 con 0x7fa0c00cb210 2024-01-27T02:32:20.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.523+0000 7fa0c64a2700 1 -- 172.21.15.7:0/2160373122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c0072060 con 0x7fa0c00cb210 2024-01-27T02:32:20.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.523+0000 7fa0a7fff700 1 -- 172.21.15.7:0/2160373122 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa0b0013680 con 0x7fa0c00cb210 2024-01-27T02:32:20.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.524+0000 7fa0a7fff700 1 --2- 172.21.15.7:0/2160373122 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa0ac066f30 0x7fa0ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:20.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.524+0000 7fa0a7fff700 1 -- 172.21.15.7:0/2160373122 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa0b0084b90 con 0x7fa0c00cb210 2024-01-27T02:32:20.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.524+0000 7fa0bf7fe700 1 --2- 172.21.15.7:0/2160373122 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa0ac066f30 0x7fa0ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:20.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.525+0000 7fa0bf7fe700 1 --2- 172.21.15.7:0/2160373122 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa0ac066f30 0x7fa0ac0693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00c92f0 tx=0x7fa0b4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:20.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.528+0000 7fa0a7fff700 1 -- 172.21.15.7:0/2160373122 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0b0051e00 con 0x7fa0c00cb210 2024-01-27T02:32:20.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.731+0000 7fa0c64a2700 1 -- 172.21.15.7:0/2160373122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa0c0159290 con 0x7fa0c00cb210 2024-01-27T02:32:20.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.732+0000 7fa0a7fff700 1 -- 172.21.15.7:0/2160373122 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa0b00170a0 con 0x7fa0c00cb210 2024-01-27T02:32:20.733 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-27T02:32:20.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.734+0000 7fa0c64a2700 1 -- 172.21.15.7:0/2160373122 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa0ac066f30 msgr2=0x7fa0ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:20.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.734+0000 7fa0c64a2700 1 --2- 172.21.15.7:0/2160373122 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa0ac066f30 0x7fa0ac0693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00c92f0 tx=0x7fa0b4009040 comp rx=0 tx=0).stop 2024-01-27T02:32:20.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.734+0000 7fa0c64a2700 1 -- 172.21.15.7:0/2160373122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c00cb210 msgr2=0x7fa0c00c61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:20.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.734+0000 7fa0c64a2700 1 --2- 172.21.15.7:0/2160373122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c00cb210 0x7fa0c00c61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000ecb0 tx=0x7fa0b0009120 comp rx=0 tx=0).stop 2024-01-27T02:32:20.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.734+0000 7fa0c64a2700 1 -- 172.21.15.7:0/2160373122 shutdown_connections 2024-01-27T02:32:20.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.734+0000 7fa0c64a2700 1 --2- 172.21.15.7:0/2160373122 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa0ac066f30 0x7fa0ac0693e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.734+0000 7fa0c64a2700 1 --2- 172.21.15.7:0/2160373122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c0141a90 0x7fa0c00c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.734+0000 7fa0c64a2700 1 --2- 172.21.15.7:0/2160373122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c00cb210 0x7fa0c00c61e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.734+0000 7fa0c64a2700 1 --2- 172.21.15.7:0/2160373122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c01416c0 0x7fa0c0068370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.734+0000 7fa0c64a2700 1 -- 172.21.15.7:0/2160373122 >> 172.21.15.7:0/2160373122 conn(0x7fa0c0063580 msgr2=0x7fa0c00c4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:20.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.734+0000 7fa0c64a2700 1 -- 172.21.15.7:0/2160373122 shutdown_connections 2024-01-27T02:32:20.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.734+0000 7fa0c64a2700 1 -- 172.21.15.7:0/2160373122 wait complete. 2024-01-27T02:32:20.749 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-27T02:32:20.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.882+0000 7f21232fc700 1 Processor -- start 2024-01-27T02:32:20.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.882+0000 7f21232fc700 1 -- start start 2024-01-27T02:32:20.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.883+0000 7f21232fc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f211c154920 0x7f211c0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:20.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.883+0000 7f21232fc700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f211c0ad7c0 0x7f211c063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:20.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.883+0000 7f21232fc700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f211c0adec0 con 0x7f211c154550 2024-01-27T02:32:20.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.883+0000 7f21232fc700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f211c0ae1a0 con 0x7f211c0ad7c0 2024-01-27T02:32:20.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.883+0000 7f21232fc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211c15a850 con 0x7f211c154920 2024-01-27T02:32:20.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.883+0000 7f2120897700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f211c154920 0x7f211c0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:20.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.883+0000 7f2121899700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f211c0ad7c0 0x7f211c063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:20.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.884+0000 7f2120897700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f211c154920 0x7f211c0ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:54378/0 (socket says 172.21.15.7:54378) 2024-01-27T02:32:20.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.884+0000 7f2120897700 1 -- 172.21.15.7:0/4249940860 learned_addr learned my addr 172.21.15.7:0/4249940860 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:20.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.884+0000 7f2121098700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f211c154550 0x7f211c15b050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:45484/0 (socket says 172.21.15.7:45484) 2024-01-27T02:32:20.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.884+0000 7f2120897700 1 -- 172.21.15.7:0/4249940860 >> v2:172.21.15.188:3300/0 conn(0x7f211c0ad7c0 msgr2=0x7f211c063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:20.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.884+0000 7f2120897700 1 --2- 172.21.15.7:0/4249940860 >> v2:172.21.15.188:3300/0 conn(0x7f211c0ad7c0 0x7f211c063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.884+0000 7f2120897700 1 -- 172.21.15.7:0/4249940860 >> v1:172.21.15.188:6789/0 conn(0x7f211c154550 legacy=0x7f211c15b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:20.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.884+0000 7f2120897700 1 -- 172.21.15.7:0/4249940860 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211c15ab30 con 0x7f211c154920 2024-01-27T02:32:20.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.884+0000 7f2120897700 1 --2- 172.21.15.7:0/4249940860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f211c154920 0x7f211c0ce6b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f21100099b0 tx=0x7f2110009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb3fb53a5a2ab362 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:20.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.884+0000 7f210ffff700 1 -- 172.21.15.7:0/4249940860 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110009ea0 con 0x7f211c154920 2024-01-27T02:32:20.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.884+0000 7f210ffff700 1 -- 172.21.15.7:0/4249940860 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211000b860 con 0x7f211c154920 2024-01-27T02:32:20.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.884+0000 7f210ffff700 1 -- 172.21.15.7:0/4249940860 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211000bc00 con 0x7f211c154920 2024-01-27T02:32:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.885+0000 7f21232fc700 1 -- 172.21.15.7:0/4249940860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f211c154920 msgr2=0x7f211c0ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.885+0000 7f21232fc700 1 --2- 172.21.15.7:0/4249940860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f211c154920 0x7f211c0ce6b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f21100099b0 tx=0x7f2110009250 comp rx=0 tx=0).stop 2024-01-27T02:32:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.885+0000 7f21232fc700 1 -- 172.21.15.7:0/4249940860 shutdown_connections 2024-01-27T02:32:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.885+0000 7f21232fc700 1 --2- 172.21.15.7:0/4249940860 >> v2:172.21.15.188:3300/0 conn(0x7f211c0ad7c0 0x7f211c063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.885+0000 7f21232fc700 1 --2- 172.21.15.7:0/4249940860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f211c154920 0x7f211c0ce6b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.885+0000 7f21232fc700 1 -- 172.21.15.7:0/4249940860 >> 172.21.15.7:0/4249940860 conn(0x7f211c068480 msgr2=0x7f211c0cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.885+0000 7f21232fc700 1 -- 172.21.15.7:0/4249940860 shutdown_connections 2024-01-27T02:32:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.885+0000 7f21232fc700 1 -- 172.21.15.7:0/4249940860 wait complete. 2024-01-27T02:32:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.886+0000 7f21232fc700 1 Processor -- start 2024-01-27T02:32:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.886+0000 7f21232fc700 1 -- start start 2024-01-27T02:32:20.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.887+0000 7f21232fc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f211c0ad7c0 0x7f211c0cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.887+0000 7f21232fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f211c154550 0x7f211c1556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.887+0000 7f21232fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f211c154920 0x7f211c1516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.887+0000 7f21232fc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c151220 con 0x7f211c154550 2024-01-27T02:32:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.887+0000 7f21232fc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c150330 con 0x7f211c154920 2024-01-27T02:32:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.887+0000 7f21232fc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211c150610 con 0x7f211c0ad7c0 2024-01-27T02:32:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.887+0000 7f2120897700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f211c154550 0x7f211c1556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.887+0000 7f2120897700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f211c154550 0x7f211c1556d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53982/0 (socket says 172.21.15.7:53982) 2024-01-27T02:32:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.887+0000 7f2120897700 1 -- 172.21.15.7:0/3260906798 learned_addr learned my addr 172.21.15.7:0/3260906798 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.887+0000 7f2121098700 1 --2- 172.21.15.7:0/3260906798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f211c0ad7c0 0x7f211c0cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.887+0000 7f2121899700 1 --2- 172.21.15.7:0/3260906798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f211c154920 0x7f211c1516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.888+0000 7f2120897700 1 -- 172.21.15.7:0/3260906798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f211c0ad7c0 msgr2=0x7f211c0cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.888+0000 7f2120897700 1 --2- 172.21.15.7:0/3260906798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f211c0ad7c0 0x7f211c0cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.888+0000 7f2120897700 1 -- 172.21.15.7:0/3260906798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f211c154920 msgr2=0x7f211c1516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.888+0000 7f2120897700 1 --2- 172.21.15.7:0/3260906798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f211c154920 0x7f211c1516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:20.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.888+0000 7f2120897700 1 -- 172.21.15.7:0/3260906798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21100094f0 con 0x7f211c154550 2024-01-27T02:32:20.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.888+0000 7f2121098700 1 --2- 172.21.15.7:0/3260906798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f211c0ad7c0 0x7f211c0cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:32:20.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.888+0000 7f2120897700 1 --2- 172.21.15.7:0/3260906798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f211c154550 0x7f211c1556d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f211c15a990 tx=0x7f2110017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:20.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.890+0000 7f210d7fa700 1 -- 172.21.15.7:0/3260906798 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110000d10 con 0x7f211c154550 2024-01-27T02:32:20.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.890+0000 7f210d7fa700 1 -- 172.21.15.7:0/3260906798 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211000bbe0 con 0x7f211c154550 2024-01-27T02:32:20.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.890+0000 7f21232fc700 1 -- 172.21.15.7:0/3260906798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211c1508f0 con 0x7f211c154550 2024-01-27T02:32:20.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.890+0000 7f210d7fa700 1 -- 172.21.15.7:0/3260906798 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110011670 con 0x7f211c154550 2024-01-27T02:32:20.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.890+0000 7f21232fc700 1 -- 172.21.15.7:0/3260906798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211c150eb0 con 0x7f211c154550 2024-01-27T02:32:20.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.891+0000 7f21232fc700 1 -- 172.21.15.7:0/3260906798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f211c072060 con 0x7f211c154550 2024-01-27T02:32:20.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.891+0000 7f210d7fa700 1 -- 172.21.15.7:0/3260906798 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2110011aa0 con 0x7f211c154550 2024-01-27T02:32:20.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.892+0000 7f210d7fa700 1 --2- 172.21.15.7:0/3260906798 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2100066f30 0x7f21000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:20.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.892+0000 7f210d7fa700 1 -- 172.21.15.7:0/3260906798 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2110085d40 con 0x7f211c154550 2024-01-27T02:32:20.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.892+0000 7f2121098700 1 --2- 172.21.15.7:0/3260906798 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2100066f30 0x7f21000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:20.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.893+0000 7f2121098700 1 --2- 172.21.15.7:0/3260906798 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2100066f30 0x7f21000693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2118001710 tx=0x7f2118007eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:20.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:20.896+0000 7f210d7fa700 1 -- 172.21.15.7:0/3260906798 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2110052fb0 con 0x7f211c154550 2024-01-27T02:32:21.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.103+0000 7f21232fc700 1 -- 172.21.15.7:0/3260906798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f211c06f780 con 0x7f211c154550 2024-01-27T02:32:21.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.104+0000 7f210d7fa700 1 -- 172.21.15.7:0/3260906798 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f21100150a0 con 0x7f211c154550 2024-01-27T02:32:21.105 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-27T02:32:21.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.106+0000 7f21232fc700 1 -- 172.21.15.7:0/3260906798 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2100066f30 msgr2=0x7f21000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:21.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.106+0000 7f21232fc700 1 --2- 172.21.15.7:0/3260906798 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2100066f30 0x7f21000693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2118001710 tx=0x7f2118007eb0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.106+0000 7f21232fc700 1 -- 172.21.15.7:0/3260906798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f211c154550 msgr2=0x7f211c1556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:21.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.106+0000 7f21232fc700 1 --2- 172.21.15.7:0/3260906798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f211c154550 0x7f211c1556d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f211c15a990 tx=0x7f2110017000 comp rx=0 tx=0).stop 2024-01-27T02:32:21.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.106+0000 7f21232fc700 1 -- 172.21.15.7:0/3260906798 shutdown_connections 2024-01-27T02:32:21.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.106+0000 7f21232fc700 1 --2- 172.21.15.7:0/3260906798 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2100066f30 0x7f21000693e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.106+0000 7f21232fc700 1 --2- 172.21.15.7:0/3260906798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f211c154920 0x7f211c1516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.106+0000 7f21232fc700 1 --2- 172.21.15.7:0/3260906798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f211c0ad7c0 0x7f211c0cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.106+0000 7f21232fc700 1 --2- 172.21.15.7:0/3260906798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f211c154550 0x7f211c1556d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.106+0000 7f21232fc700 1 -- 172.21.15.7:0/3260906798 >> 172.21.15.7:0/3260906798 conn(0x7f211c068480 msgr2=0x7f211c0b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:21.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.106+0000 7f21232fc700 1 -- 172.21.15.7:0/3260906798 shutdown_connections 2024-01-27T02:32:21.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.106+0000 7f21232fc700 1 -- 172.21.15.7:0/3260906798 wait complete. 2024-01-27T02:32:21.121 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:32:21.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.250+0000 7f73d3fe8700 1 Processor -- start 2024-01-27T02:32:21.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.251+0000 7f73d3fe8700 1 -- start start 2024-01-27T02:32:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.251+0000 7f73d3fe8700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f73cc06a9f0 0x7f73cc15c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.251+0000 7f73d3fe8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f73cc06adc0 0x7f73cc158520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.252+0000 7f73d3fe8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73cc06c880 con 0x7f73cc158a60 2024-01-27T02:32:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.252+0000 7f73d3fe8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73cc158ee0 con 0x7f73cc06adc0 2024-01-27T02:32:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.252+0000 7f73d3fe8700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73cc15c210 con 0x7f73cc06a9f0 2024-01-27T02:32:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.252+0000 7f73d1583700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f73cc06adc0 0x7f73cc158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.252+0000 7f73d2585700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f73cc158a60 0x7f73cc063580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51478/0 (socket says 172.21.15.7:51478) 2024-01-27T02:32:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.252+0000 7f73d2585700 1 -- 172.21.15.7:0/2376581971 learned_addr learned my addr 172.21.15.7:0/2376581971 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.252+0000 7f73d1d84700 1 --2- 172.21.15.7:0/2376581971 >> v2:172.21.15.188:3300/0 conn(0x7f73cc06a9f0 0x7f73cc15c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.252+0000 7f73d1583700 1 -- 172.21.15.7:0/2376581971 >> v2:172.21.15.188:3300/0 conn(0x7f73cc06a9f0 msgr2=0x7f73cc15c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.252+0000 7f73d1583700 1 --2- 172.21.15.7:0/2376581971 >> v2:172.21.15.188:3300/0 conn(0x7f73cc06a9f0 0x7f73cc15c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.252+0000 7f73d1583700 1 -- 172.21.15.7:0/2376581971 >> v1:172.21.15.7:6789/0 conn(0x7f73cc158a60 legacy=0x7f73cc063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:21.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.252+0000 7f73d1583700 1 -- 172.21.15.7:0/2376581971 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73cc1593e0 con 0x7f73cc06adc0 2024-01-27T02:32:21.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.252+0000 7f73d0d82700 1 -- 172.21.15.7:0/2376581971 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2417803682 0 0) 0x7f73cc06c880 con 0x7f73cc158a60 2024-01-27T02:32:21.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.252+0000 7f73d1583700 1 --2- 172.21.15.7:0/2376581971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73cc06adc0 0x7f73cc158520 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0cb700 tx=0x7f73c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3630c29f9db09b7d server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:21.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.252+0000 7f73d0d82700 1 -- 172.21.15.7:0/2376581971 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c8011630 con 0x7f73cc06adc0 2024-01-27T02:32:21.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.252+0000 7f73d0d82700 1 -- 172.21.15.7:0/2376581971 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c8011790 con 0x7f73cc06adc0 2024-01-27T02:32:21.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.252+0000 7f73d0d82700 1 -- 172.21.15.7:0/2376581971 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c8011b30 con 0x7f73cc06adc0 2024-01-27T02:32:21.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.253+0000 7f73d3fe8700 1 -- 172.21.15.7:0/2376581971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73cc06adc0 msgr2=0x7f73cc158520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:21.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.253+0000 7f73d3fe8700 1 --2- 172.21.15.7:0/2376581971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73cc06adc0 0x7f73cc158520 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0cb700 tx=0x7f73c80091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.253+0000 7f73d3fe8700 1 -- 172.21.15.7:0/2376581971 shutdown_connections 2024-01-27T02:32:21.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.253+0000 7f73d3fe8700 1 --2- 172.21.15.7:0/2376581971 >> v2:172.21.15.188:3300/0 conn(0x7f73cc06a9f0 0x7f73cc15c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.253+0000 7f73d3fe8700 1 --2- 172.21.15.7:0/2376581971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73cc06adc0 0x7f73cc158520 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.253+0000 7f73d3fe8700 1 -- 172.21.15.7:0/2376581971 >> 172.21.15.7:0/2376581971 conn(0x7f73cc0cdc30 msgr2=0x7f73cc0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:21.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.253+0000 7f73d3fe8700 1 -- 172.21.15.7:0/2376581971 shutdown_connections 2024-01-27T02:32:21.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.253+0000 7f73d3fe8700 1 -- 172.21.15.7:0/2376581971 wait complete. 2024-01-27T02:32:21.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.254+0000 7f73d3fe8700 1 Processor -- start 2024-01-27T02:32:21.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.254+0000 7f73d3fe8700 1 -- start start 2024-01-27T02:32:21.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.255+0000 7f73d3fe8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73cc06a9f0 0x7f73cc14cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:21.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.255+0000 7f73d3fe8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73cc06adc0 0x7f73cc0c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:21.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.255+0000 7f73d3fe8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73cc158a60 0x7f73cc0c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:21.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.255+0000 7f73d3fe8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc0b1030 con 0x7f73cc06adc0 2024-01-27T02:32:21.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.255+0000 7f73d3fe8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc0b1280 con 0x7f73cc158a60 2024-01-27T02:32:21.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.255+0000 7f73d3fe8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc0b1560 con 0x7f73cc06a9f0 2024-01-27T02:32:21.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.255+0000 7f73d1583700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73cc06adc0 0x7f73cc0c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:21.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.255+0000 7f73d1583700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73cc06adc0 0x7f73cc0c6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53996/0 (socket says 172.21.15.7:53996) 2024-01-27T02:32:21.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.255+0000 7f73d1583700 1 -- 172.21.15.7:0/1259031616 learned_addr learned my addr 172.21.15.7:0/1259031616 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:21.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.255+0000 7f73d2585700 1 --2- 172.21.15.7:0/1259031616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73cc158a60 0x7f73cc0c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:21.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.255+0000 7f73d1d84700 1 --2- 172.21.15.7:0/1259031616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73cc06a9f0 0x7f73cc14cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:21.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.256+0000 7f73d1583700 1 -- 172.21.15.7:0/1259031616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73cc06a9f0 msgr2=0x7f73cc14cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:21.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.256+0000 7f73d1583700 1 --2- 172.21.15.7:0/1259031616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73cc06a9f0 0x7f73cc14cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.256+0000 7f73d1583700 1 -- 172.21.15.7:0/1259031616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73cc158a60 msgr2=0x7f73cc0c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:21.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.256+0000 7f73d1583700 1 --2- 172.21.15.7:0/1259031616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73cc158a60 0x7f73cc0c7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.256+0000 7f73d1583700 1 -- 172.21.15.7:0/1259031616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c8009420 con 0x7f73cc06adc0 2024-01-27T02:32:21.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.256+0000 7f73d2585700 1 --2- 172.21.15.7:0/1259031616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73cc158a60 0x7f73cc0c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:21.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.256+0000 7f73d1583700 1 --2- 172.21.15.7:0/1259031616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73cc06adc0 0x7f73cc0c6d20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f73c8013000 tx=0x7f73c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:21.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.256+0000 7f73bdffb700 1 -- 172.21.15.7:0/1259031616 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c8011b10 con 0x7f73cc06adc0 2024-01-27T02:32:21.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.256+0000 7f73bdffb700 1 -- 172.21.15.7:0/1259031616 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c80253f0 con 0x7f73cc06adc0 2024-01-27T02:32:21.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.256+0000 7f73d3fe8700 1 -- 172.21.15.7:0/1259031616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73cc0b1840 con 0x7f73cc06adc0 2024-01-27T02:32:21.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.256+0000 7f73bdffb700 1 -- 172.21.15.7:0/1259031616 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c8025790 con 0x7f73cc06adc0 2024-01-27T02:32:21.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.256+0000 7f73d3fe8700 1 -- 172.21.15.7:0/1259031616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73cc0b1e00 con 0x7f73cc06adc0 2024-01-27T02:32:21.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.257+0000 7f73d3fe8700 1 -- 172.21.15.7:0/1259031616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73cc04c710 con 0x7f73cc06adc0 2024-01-27T02:32:21.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.258+0000 7f73bdffb700 1 -- 172.21.15.7:0/1259031616 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f73c8025b80 con 0x7f73cc06adc0 2024-01-27T02:32:21.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.258+0000 7f73bdffb700 1 --2- 172.21.15.7:0/1259031616 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f73b0066f30 0x7f73b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:21.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.258+0000 7f73bdffb700 1 -- 172.21.15.7:0/1259031616 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f73c8085eb0 con 0x7f73cc06adc0 2024-01-27T02:32:21.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.258+0000 7f73d1d84700 1 --2- 172.21.15.7:0/1259031616 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f73b0066f30 0x7f73b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:21.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.259+0000 7f73d1d84700 1 --2- 172.21.15.7:0/1259031616 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f73b0066f30 0x7f73b00693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009d40 tx=0x7f73c0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:21.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.262+0000 7f73bdffb700 1 -- 172.21.15.7:0/1259031616 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73c8053120 con 0x7f73cc06adc0 2024-01-27T02:32:21.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.511+0000 7f73d3fe8700 1 -- 172.21.15.7:0/1259031616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73cc0c7db0 con 0x7f73cc06adc0 2024-01-27T02:32:21.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.512+0000 7f73bdffb700 1 -- 172.21.15.7:0/1259031616 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f73c8056740 con 0x7f73cc06adc0 2024-01-27T02:32:21.513 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:21.513 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3594367777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3594367777},{"type":"v1","addr":"172.21.15.7:6834","nonce":3594367777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1079101213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1079101213},{"type":"v1","addr":"172.21.15.188:6837","nonce":1079101213}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3241424244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3241424244},{"type":"v1","addr":"172.21.15.7:6835","nonce":3241424244}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:32:06.045693+0000","modified":"2024-01-27T02:32:09.806398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6836/2571237935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2571237935},{"type":"v1","addr":"172.21.15.188:6836","nonce":2571237935}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-27T02:32:21.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.514+0000 7f73d3fe8700 1 -- 172.21.15.7:0/1259031616 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f73b0066f30 msgr2=0x7f73b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:21.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.514+0000 7f73d3fe8700 1 --2- 172.21.15.7:0/1259031616 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f73b0066f30 0x7f73b00693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009d40 tx=0x7f73c0008040 comp rx=0 tx=0).stop 2024-01-27T02:32:21.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.514+0000 7f73d3fe8700 1 -- 172.21.15.7:0/1259031616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73cc06adc0 msgr2=0x7f73cc0c6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:21.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.514+0000 7f73d3fe8700 1 --2- 172.21.15.7:0/1259031616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73cc06adc0 0x7f73cc0c6d20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f73c8013000 tx=0x7f73c8017000 comp rx=0 tx=0).stop 2024-01-27T02:32:21.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.514+0000 7f73d3fe8700 1 -- 172.21.15.7:0/1259031616 shutdown_connections 2024-01-27T02:32:21.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.514+0000 7f73d3fe8700 1 --2- 172.21.15.7:0/1259031616 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f73b0066f30 0x7f73b00693e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.514+0000 7f73d3fe8700 1 --2- 172.21.15.7:0/1259031616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73cc158a60 0x7f73cc0c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.514+0000 7f73d3fe8700 1 --2- 172.21.15.7:0/1259031616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73cc06a9f0 0x7f73cc14cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.514+0000 7f73d3fe8700 1 --2- 172.21.15.7:0/1259031616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73cc06adc0 0x7f73cc0c6d20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.514+0000 7f73d3fe8700 1 -- 172.21.15.7:0/1259031616 >> 172.21.15.7:0/1259031616 conn(0x7f73cc0cdc30 msgr2=0x7f73cc06e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:21.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.514+0000 7f73d3fe8700 1 -- 172.21.15.7:0/1259031616 shutdown_connections 2024-01-27T02:32:21.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.514+0000 7f73d3fe8700 1 -- 172.21.15.7:0/1259031616 wait complete. 2024-01-27T02:32:21.517 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-01-27T02:32:21.529 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:32:21.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.661+0000 7f51e6735700 1 Processor -- start 2024-01-27T02:32:21.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.661+0000 7f51e6735700 1 -- start start 2024-01-27T02:32:21.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.662+0000 7f51e6735700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e00c9f80 0x7f51e0157f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:21.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.662+0000 7f51e6735700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f51e006e330 0x7f51e006e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:21.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.662+0000 7f51e6735700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e006c880 con 0x7f51e00c9bb0 2024-01-27T02:32:21.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.662+0000 7f51e6735700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e006cd10 con 0x7f51e006e330 2024-01-27T02:32:21.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.662+0000 7f51e6735700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e006ecf0 con 0x7f51e00c9f80 2024-01-27T02:32:21.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.662+0000 7f51df7fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e00c9f80 0x7f51e0157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:21.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.662+0000 7f51e4cd2700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f51e006e330 0x7f51e006e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:21.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.662+0000 7f51dffff700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f51e00c9bb0 0x7f51e006b400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:46286/0 (socket says 172.21.15.7:46286) 2024-01-27T02:32:21.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.662+0000 7f51dffff700 1 -- 172.21.15.7:0/2503527757 learned_addr learned my addr 172.21.15.7:0/2503527757 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:21.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.662+0000 7f51deffd700 1 -- 172.21.15.7:0/2503527757 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2377454790 0 0) 0x7f51e006c880 con 0x7f51e00c9bb0 2024-01-27T02:32:21.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.662+0000 7f51deffd700 1 -- 172.21.15.7:0/2503527757 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51c8003570 con 0x7f51e00c9bb0 2024-01-27T02:32:21.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.662+0000 7f51df7fe700 1 -- 172.21.15.7:0/2503527757 >> v2:172.21.15.188:3300/0 conn(0x7f51e006e330 msgr2=0x7f51e006e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:21.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.662+0000 7f51df7fe700 1 --2- 172.21.15.7:0/2503527757 >> v2:172.21.15.188:3300/0 conn(0x7f51e006e330 0x7f51e006e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.662+0000 7f51df7fe700 1 -- 172.21.15.7:0/2503527757 >> v1:172.21.15.188:6789/0 conn(0x7f51e00c9bb0 legacy=0x7f51e006b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:21.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.662+0000 7f51df7fe700 1 -- 172.21.15.7:0/2503527757 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e006bc70 con 0x7f51e00c9f80 2024-01-27T02:32:21.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.662+0000 7f51e4cd2700 1 --2- 172.21.15.7:0/2503527757 >> v2:172.21.15.188:3300/0 conn(0x7f51e006e330 0x7f51e006e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:32:21.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.663+0000 7f51df7fe700 1 --2- 172.21.15.7:0/2503527757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e00c9f80 0x7f51e0157f70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f51d400bb90 tx=0x7f51d400b430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fd49613bc40628e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:21.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.663+0000 7f51deffd700 1 -- 172.21.15.7:0/2503527757 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d400da70 con 0x7f51e00c9f80 2024-01-27T02:32:21.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.663+0000 7f51deffd700 1 -- 172.21.15.7:0/2503527757 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d400dbd0 con 0x7f51e00c9f80 2024-01-27T02:32:21.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.663+0000 7f51deffd700 1 -- 172.21.15.7:0/2503527757 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4019480 con 0x7f51e00c9f80 2024-01-27T02:32:21.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.663+0000 7f51e6735700 1 -- 172.21.15.7:0/2503527757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e00c9f80 msgr2=0x7f51e0157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:21.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.663+0000 7f51e6735700 1 --2- 172.21.15.7:0/2503527757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e00c9f80 0x7f51e0157f70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f51d400bb90 tx=0x7f51d400b430 comp rx=0 tx=0).stop 2024-01-27T02:32:21.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.663+0000 7f51e6735700 1 -- 172.21.15.7:0/2503527757 shutdown_connections 2024-01-27T02:32:21.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.663+0000 7f51e6735700 1 --2- 172.21.15.7:0/2503527757 >> v2:172.21.15.188:3300/0 conn(0x7f51e006e330 0x7f51e006e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.663+0000 7f51e6735700 1 --2- 172.21.15.7:0/2503527757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e00c9f80 0x7f51e0157f70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.663+0000 7f51e6735700 1 -- 172.21.15.7:0/2503527757 >> 172.21.15.7:0/2503527757 conn(0x7f51e00cdc30 msgr2=0x7f51e00ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:21.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.663+0000 7f51e6735700 1 -- 172.21.15.7:0/2503527757 shutdown_connections 2024-01-27T02:32:21.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.663+0000 7f51e6735700 1 -- 172.21.15.7:0/2503527757 wait complete. 2024-01-27T02:32:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.664+0000 7f51e6735700 1 Processor -- start 2024-01-27T02:32:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.665+0000 7f51e6735700 1 -- start start 2024-01-27T02:32:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.665+0000 7f51e6735700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e006e330 0x7f51e0155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.665+0000 7f51e6735700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e00c9bb0 0x7f51e00b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.665+0000 7f51e6735700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51e00c9f80 0x7f51e00b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.665+0000 7f51e6735700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00c3220 con 0x7f51e00c9bb0 2024-01-27T02:32:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.665+0000 7f51e6735700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00b1510 con 0x7f51e00c9f80 2024-01-27T02:32:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.665+0000 7f51e6735700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00b17f0 con 0x7f51e006e330 2024-01-27T02:32:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.666+0000 7f51df7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e00c9bb0 0x7f51e00b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.666+0000 7f51df7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e00c9bb0 0x7f51e00b1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56184/0 (socket says 172.21.15.7:56184) 2024-01-27T02:32:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.666+0000 7f51df7fe700 1 -- 172.21.15.7:0/2192870470 learned_addr learned my addr 172.21.15.7:0/2192870470 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:21.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.666+0000 7f51dffff700 1 --2- 172.21.15.7:0/2192870470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e006e330 0x7f51e0155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:21.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.666+0000 7f51e4cd2700 1 --2- 172.21.15.7:0/2192870470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51e00c9f80 0x7f51e00b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:21.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.666+0000 7f51df7fe700 1 -- 172.21.15.7:0/2192870470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e006e330 msgr2=0x7f51e0155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:21.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.666+0000 7f51df7fe700 1 --2- 172.21.15.7:0/2192870470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e006e330 0x7f51e0155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.666+0000 7f51df7fe700 1 -- 172.21.15.7:0/2192870470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51e00c9f80 msgr2=0x7f51e00b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:21.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.666+0000 7f51df7fe700 1 --2- 172.21.15.7:0/2192870470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51e00c9f80 0x7f51e00b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.666+0000 7f51df7fe700 1 -- 172.21.15.7:0/2192870470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d8009a20 con 0x7f51e00c9bb0 2024-01-27T02:32:21.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.666+0000 7f51e4cd2700 1 --2- 172.21.15.7:0/2192870470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51e00c9f80 0x7f51e00b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:21.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.666+0000 7f51df7fe700 1 --2- 172.21.15.7:0/2192870470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e00c9bb0 0x7f51e00b1f60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f51d400b2c0 tx=0x7f51d401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:21.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.666+0000 7f51c7fff700 1 -- 172.21.15.7:0/2192870470 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4010750 con 0x7f51e00c9bb0 2024-01-27T02:32:21.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.666+0000 7f51c7fff700 1 -- 172.21.15.7:0/2192870470 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d401e050 con 0x7f51e00c9bb0 2024-01-27T02:32:21.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.666+0000 7f51e6735700 1 -- 172.21.15.7:0/2192870470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d400b6d0 con 0x7f51e00c9bb0 2024-01-27T02:32:21.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.666+0000 7f51c7fff700 1 -- 172.21.15.7:0/2192870470 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4010b00 con 0x7f51e00c9bb0 2024-01-27T02:32:21.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.667+0000 7f51e6735700 1 -- 172.21.15.7:0/2192870470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e00c2eb0 con 0x7f51e00c9bb0 2024-01-27T02:32:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.667+0000 7f51e6735700 1 -- 172.21.15.7:0/2192870470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e00c3390 con 0x7f51e00c9bb0 2024-01-27T02:32:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.668+0000 7f51c7fff700 1 -- 172.21.15.7:0/2192870470 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f51d4024470 con 0x7f51e00c9bb0 2024-01-27T02:32:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.668+0000 7f51c7fff700 1 --2- 172.21.15.7:0/2192870470 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f51cc066f30 0x7f51cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.668+0000 7f51c7fff700 1 -- 172.21.15.7:0/2192870470 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f51d4014070 con 0x7f51e00c9bb0 2024-01-27T02:32:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.668+0000 7f51dffff700 1 --2- 172.21.15.7:0/2192870470 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f51cc066f30 0x7f51cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.669+0000 7f51dffff700 1 --2- 172.21.15.7:0/2192870470 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f51cc066f30 0x7f51cc0693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f51d0007fd0 tx=0x7f51d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:21.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.673+0000 7f51c7fff700 1 -- 172.21.15.7:0/2192870470 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f51d408f080 con 0x7f51e00c9bb0 2024-01-27T02:32:21.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.878+0000 7f51e6735700 1 -- 172.21.15.7:0/2192870470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51e004c710 con 0x7f51e00c9bb0 2024-01-27T02:32:21.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.879+0000 7f51c7fff700 1 -- 172.21.15.7:0/2192870470 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f51d4056030 con 0x7f51e00c9bb0 2024-01-27T02:32:21.880 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:21.880 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:32:07.627822+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:32:03.675611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-27T02:32:05.018885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:32:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.881+0000 7f51e6735700 1 -- 172.21.15.7:0/2192870470 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f51cc066f30 msgr2=0x7f51cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.881+0000 7f51e6735700 1 --2- 172.21.15.7:0/2192870470 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f51cc066f30 0x7f51cc0693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f51d0007fd0 tx=0x7f51d0009040 comp rx=0 tx=0).stop 2024-01-27T02:32:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.881+0000 7f51e6735700 1 -- 172.21.15.7:0/2192870470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e00c9bb0 msgr2=0x7f51e00b1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.881+0000 7f51e6735700 1 --2- 172.21.15.7:0/2192870470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e00c9bb0 0x7f51e00b1f60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f51d400b2c0 tx=0x7f51d401b000 comp rx=0 tx=0).stop 2024-01-27T02:32:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.881+0000 7f51e6735700 1 -- 172.21.15.7:0/2192870470 shutdown_connections 2024-01-27T02:32:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.881+0000 7f51e6735700 1 --2- 172.21.15.7:0/2192870470 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f51cc066f30 0x7f51cc0693e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.881+0000 7f51e6735700 1 --2- 172.21.15.7:0/2192870470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51e00c9f80 0x7f51e00b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.881+0000 7f51e6735700 1 --2- 172.21.15.7:0/2192870470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e006e330 0x7f51e0155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.881+0000 7f51e6735700 1 --2- 172.21.15.7:0/2192870470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e00c9bb0 0x7f51e00b1f60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.881+0000 7f51e6735700 1 -- 172.21.15.7:0/2192870470 >> 172.21.15.7:0/2192870470 conn(0x7f51e00cdc30 msgr2=0x7f51e00cc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:21.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.881+0000 7f51e6735700 1 -- 172.21.15.7:0/2192870470 shutdown_connections 2024-01-27T02:32:21.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:21.881+0000 7f51e6735700 1 -- 172.21.15.7:0/2192870470 wait complete. 2024-01-27T02:32:21.897 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-27T02:32:03.675611+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-27T02:32:05.018885+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-27T02:32:21.897 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:32:22.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.031+0000 7f1eca7ff700 1 Processor -- start 2024-01-27T02:32:22.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.031+0000 7f1eca7ff700 1 -- start start 2024-01-27T02:32:22.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.032+0000 7f1eca7ff700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1ec40cba60 0x7f1ec40ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:22.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.032+0000 7f1eca7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec40679b0 con 0x7f1ec406b4e0 2024-01-27T02:32:22.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.032+0000 7f1eca7ff700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec4067c90 con 0x7f1ec40cbe30 2024-01-27T02:32:22.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.032+0000 7f1eca7ff700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec40cca50 con 0x7f1ec40cba60 2024-01-27T02:32:22.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.032+0000 7f1ec8d9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1ec406b4e0 0x7f1ec406b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43586/0 (socket says 172.21.15.7:43586) 2024-01-27T02:32:22.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.032+0000 7f1ec8d9c700 1 -- 172.21.15.7:0/3822120974 learned_addr learned my addr 172.21.15.7:0/3822120974 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:22.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.032+0000 7f1ec3fff700 1 --2- 172.21.15.7:0/3822120974 >> v2:172.21.15.188:3300/0 conn(0x7f1ec40cba60 0x7f1ec40ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:22.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.033+0000 7f1ec2ffd700 1 -- 172.21.15.7:0/3822120974 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1211466376 0 0) 0x7f1ec40679b0 con 0x7f1ec406b4e0 2024-01-27T02:32:22.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.033+0000 7f1ec2ffd700 1 -- 172.21.15.7:0/3822120974 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eb0003570 con 0x7f1ec406b4e0 2024-01-27T02:32:22.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.033+0000 7f1ec2ffd700 1 -- 172.21.15.7:0/3822120974 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1159339663 0 0) 0x7f1ec4067c90 con 0x7f1ec40cbe30 2024-01-27T02:32:22.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.033+0000 7f1ec2ffd700 1 -- 172.21.15.7:0/3822120974 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ec40679b0 con 0x7f1ec40cbe30 2024-01-27T02:32:22.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.033+0000 7f1ec3fff700 1 -- 172.21.15.7:0/3822120974 >> v1:172.21.15.188:6789/0 conn(0x7f1ec40cbe30 legacy=0x7f1ec415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:22.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.033+0000 7f1ec3fff700 1 -- 172.21.15.7:0/3822120974 >> v1:172.21.15.7:6789/0 conn(0x7f1ec406b4e0 legacy=0x7f1ec406b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:22.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.033+0000 7f1ec3fff700 1 -- 172.21.15.7:0/3822120974 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec40ccd30 con 0x7f1ec40cba60 2024-01-27T02:32:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.033+0000 7f1ec3fff700 1 --2- 172.21.15.7:0/3822120974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ec40cba60 0x7f1ec40ce410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40098d0 tx=0x7f1eb4009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80ec8adccf714906 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.033+0000 7f1ec2ffd700 1 -- 172.21.15.7:0/3822120974 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4005c10 con 0x7f1ec40cba60 2024-01-27T02:32:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.033+0000 7f1ec2ffd700 1 -- 172.21.15.7:0/3822120974 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb4005d70 con 0x7f1ec40cba60 2024-01-27T02:32:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.033+0000 7f1ec2ffd700 1 -- 172.21.15.7:0/3822120974 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4006280 con 0x7f1ec40cba60 2024-01-27T02:32:22.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.034+0000 7f1eca7ff700 1 -- 172.21.15.7:0/3822120974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ec40cba60 msgr2=0x7f1ec40ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:22.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.034+0000 7f1eca7ff700 1 --2- 172.21.15.7:0/3822120974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ec40cba60 0x7f1ec40ce410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40098d0 tx=0x7f1eb4009120 comp rx=0 tx=0).stop 2024-01-27T02:32:22.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.034+0000 7f1eca7ff700 1 -- 172.21.15.7:0/3822120974 shutdown_connections 2024-01-27T02:32:22.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.034+0000 7f1eca7ff700 1 --2- 172.21.15.7:0/3822120974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ec40cba60 0x7f1ec40ce410 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.034+0000 7f1eca7ff700 1 -- 172.21.15.7:0/3822120974 >> 172.21.15.7:0/3822120974 conn(0x7f1ec4063580 msgr2=0x7f1ec4063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:22.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.034+0000 7f1eca7ff700 1 -- 172.21.15.7:0/3822120974 shutdown_connections 2024-01-27T02:32:22.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.034+0000 7f1eca7ff700 1 -- 172.21.15.7:0/3822120974 wait complete. 2024-01-27T02:32:22.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.035+0000 7f1eca7ff700 1 Processor -- start 2024-01-27T02:32:22.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.035+0000 7f1eca7ff700 1 -- start start 2024-01-27T02:32:22.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.036+0000 7f1eca7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ec406b4e0 0x7f1ec4158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:22.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.036+0000 7f1eca7ff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ec40cba60 0x7f1ec40aca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:22.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.036+0000 7f1eca7ff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ec40cbe30 0x7f1ec40acf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:22.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.036+0000 7f1eca7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec40b11f0 con 0x7f1ec406b4e0 2024-01-27T02:32:22.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.036+0000 7f1eca7ff700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec41585d0 con 0x7f1ec40cbe30 2024-01-27T02:32:22.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.036+0000 7f1eca7ff700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec40b05e0 con 0x7f1ec40cba60 2024-01-27T02:32:22.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.036+0000 7f1ec3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ec406b4e0 0x7f1ec4158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:22.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.036+0000 7f1ec3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ec406b4e0 0x7f1ec4158090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56186/0 (socket says 172.21.15.7:56186) 2024-01-27T02:32:22.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.036+0000 7f1ec3fff700 1 -- 172.21.15.7:0/3733243477 learned_addr learned my addr 172.21.15.7:0/3733243477 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:22.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.036+0000 7f1ec8d9c700 1 --2- 172.21.15.7:0/3733243477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ec40cbe30 0x7f1ec40acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:22.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.036+0000 7f1ec37fe700 1 --2- 172.21.15.7:0/3733243477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ec40cba60 0x7f1ec40aca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:22.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.036+0000 7f1ec3fff700 1 -- 172.21.15.7:0/3733243477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ec40cba60 msgr2=0x7f1ec40aca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:22.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.037+0000 7f1ec3fff700 1 --2- 172.21.15.7:0/3733243477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ec40cba60 0x7f1ec40aca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.037+0000 7f1ec3fff700 1 -- 172.21.15.7:0/3733243477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ec40cbe30 msgr2=0x7f1ec40acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:22.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.037+0000 7f1ec3fff700 1 --2- 172.21.15.7:0/3733243477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ec40cbe30 0x7f1ec40acf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.037+0000 7f1ec3fff700 1 -- 172.21.15.7:0/3733243477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb40093c0 con 0x7f1ec406b4e0 2024-01-27T02:32:22.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.037+0000 7f1ec37fe700 1 --2- 172.21.15.7:0/3733243477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ec40cba60 0x7f1ec40aca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:22.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.037+0000 7f1ec3fff700 1 --2- 172.21.15.7:0/3733243477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ec406b4e0 0x7f1ec4158090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4005d40 tx=0x7f1eb4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:22.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.037+0000 7f1ebbfff700 1 -- 172.21.15.7:0/3733243477 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb401d7b0 con 0x7f1ec406b4e0 2024-01-27T02:32:22.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.037+0000 7f1ebbfff700 1 -- 172.21.15.7:0/3733243477 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb401d930 con 0x7f1ec406b4e0 2024-01-27T02:32:22.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.037+0000 7f1eca7ff700 1 -- 172.21.15.7:0/3733243477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec40b08c0 con 0x7f1ec406b4e0 2024-01-27T02:32:22.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.037+0000 7f1ebbfff700 1 -- 172.21.15.7:0/3733243477 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb401dcf0 con 0x7f1ec406b4e0 2024-01-27T02:32:22.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.037+0000 7f1eca7ff700 1 -- 172.21.15.7:0/3733243477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec40b0e80 con 0x7f1ec406b4e0 2024-01-27T02:32:22.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.038+0000 7f1eca7ff700 1 -- 172.21.15.7:0/3733243477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec4072060 con 0x7f1ec406b4e0 2024-01-27T02:32:22.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.038+0000 7f1ebbfff700 1 -- 172.21.15.7:0/3733243477 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1eb401b690 con 0x7f1ec406b4e0 2024-01-27T02:32:22.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.039+0000 7f1ebbfff700 1 --2- 172.21.15.7:0/3733243477 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1ea4066f30 0x7f1ea40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:22.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.039+0000 7f1ebbfff700 1 -- 172.21.15.7:0/3733243477 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1eb40861d0 con 0x7f1ec406b4e0 2024-01-27T02:32:22.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.039+0000 7f1ec37fe700 1 --2- 172.21.15.7:0/3733243477 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1ea4066f30 0x7f1ea40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:22.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.040+0000 7f1ec37fe700 1 --2- 172.21.15.7:0/3733243477 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1ea4066f30 0x7f1ea40693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40cc260 tx=0x7f1eac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:22.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.043+0000 7f1ebbfff700 1 -- 172.21.15.7:0/3733243477 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1eb4053440 con 0x7f1ec406b4e0 2024-01-27T02:32:22.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.245+0000 7f1eca7ff700 1 -- 172.21.15.7:0/3733243477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1ec406d5c0 con 0x7f1ec406b4e0 2024-01-27T02:32:22.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.246+0000 7f1ebbfff700 1 -- 172.21.15.7:0/3733243477 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1eb402a0a0 con 0x7f1ec406b4e0 2024-01-27T02:32:22.247 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:32:22.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.248+0000 7f1eca7ff700 1 -- 172.21.15.7:0/3733243477 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1ea4066f30 msgr2=0x7f1ea40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.248+0000 7f1eca7ff700 1 --2- 172.21.15.7:0/3733243477 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1ea4066f30 0x7f1ea40693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40cc260 tx=0x7f1eac009040 comp rx=0 tx=0).stop 2024-01-27T02:32:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.248+0000 7f1eca7ff700 1 -- 172.21.15.7:0/3733243477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ec406b4e0 msgr2=0x7f1ec4158090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.248+0000 7f1eca7ff700 1 --2- 172.21.15.7:0/3733243477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ec406b4e0 0x7f1ec4158090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4005d40 tx=0x7f1eb4016000 comp rx=0 tx=0).stop 2024-01-27T02:32:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.248+0000 7f1eca7ff700 1 -- 172.21.15.7:0/3733243477 shutdown_connections 2024-01-27T02:32:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.248+0000 7f1eca7ff700 1 --2- 172.21.15.7:0/3733243477 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1ea4066f30 0x7f1ea40693e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.248+0000 7f1eca7ff700 1 --2- 172.21.15.7:0/3733243477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ec40cbe30 0x7f1ec40acf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.248+0000 7f1eca7ff700 1 --2- 172.21.15.7:0/3733243477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ec406b4e0 0x7f1ec4158090 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.248+0000 7f1eca7ff700 1 --2- 172.21.15.7:0/3733243477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ec40cba60 0x7f1ec40aca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.248+0000 7f1eca7ff700 1 -- 172.21.15.7:0/3733243477 >> 172.21.15.7:0/3733243477 conn(0x7f1ec4063580 msgr2=0x7f1ec4065700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.248+0000 7f1eca7ff700 1 -- 172.21.15.7:0/3733243477 shutdown_connections 2024-01-27T02:32:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.248+0000 7f1eca7ff700 1 -- 172.21.15.7:0/3733243477 wait complete. 2024-01-27T02:32:22.263 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-27T02:32:22.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.396+0000 7f28ded90700 1 Processor -- start 2024-01-27T02:32:22.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.396+0000 7f28ded90700 1 -- start start 2024-01-27T02:32:22.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.397+0000 7f28ded90700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f28d8154550 0x7f28d80ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:22.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.397+0000 7f28ded90700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f28d8154920 0x7f28d80ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:22.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.397+0000 7f28ded90700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28d815b050 0x7f28d8063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:22.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.397+0000 7f28ded90700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d80adec0 con 0x7f28d8154920 2024-01-27T02:32:22.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.397+0000 7f28ded90700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d80ae1a0 con 0x7f28d8154550 2024-01-27T02:32:22.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.397+0000 7f28ded90700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d815a850 con 0x7f28d815b050 2024-01-27T02:32:22.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.397+0000 7f28d7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f28d8154920 0x7f28d80ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:22.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.397+0000 7f28d7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f28d8154920 0x7f28d80ad7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56190/0 (socket says 172.21.15.7:56190) 2024-01-27T02:32:22.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.397+0000 7f28d7fff700 1 -- 172.21.15.7:0/425103120 learned_addr learned my addr 172.21.15.7:0/425103120 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:22.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.398+0000 7f28dd32d700 1 --2- 172.21.15.7:0/425103120 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28d815b050 0x7f28d8063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:22.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.398+0000 7f28dcb2c700 1 --2- 172.21.15.7:0/425103120 >> v2:172.21.15.188:3300/0 conn(0x7f28d8154550 0x7f28d80ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:22.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.398+0000 7f28d7fff700 1 -- 172.21.15.7:0/425103120 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28d815b050 msgr2=0x7f28d8063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:22.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.398+0000 7f28d7fff700 1 --2- 172.21.15.7:0/425103120 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28d815b050 0x7f28d8063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.398+0000 7f28d7fff700 1 -- 172.21.15.7:0/425103120 >> v2:172.21.15.188:3300/0 conn(0x7f28d8154550 msgr2=0x7f28d80ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:22.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.398+0000 7f28d7fff700 1 --2- 172.21.15.7:0/425103120 >> v2:172.21.15.188:3300/0 conn(0x7f28d8154550 0x7f28d80ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.398+0000 7f28d7fff700 1 -- 172.21.15.7:0/425103120 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d815ab30 con 0x7f28d8154920 2024-01-27T02:32:22.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.398+0000 7f28d7fff700 1 --2- 172.21.15.7:0/425103120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d8154920 0x7f28d80ad7c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f28d8068f50 tx=0x7f28c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1b2f57a9183385 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.398+0000 7f28d77fe700 1 -- 172.21.15.7:0/425103120 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c400b640 con 0x7f28d8154920 2024-01-27T02:32:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.398+0000 7f28d77fe700 1 -- 172.21.15.7:0/425103120 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c400b7a0 con 0x7f28d8154920 2024-01-27T02:32:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.398+0000 7f28d77fe700 1 -- 172.21.15.7:0/425103120 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c400bb60 con 0x7f28d8154920 2024-01-27T02:32:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.399+0000 7f28ded90700 1 -- 172.21.15.7:0/425103120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d8154920 msgr2=0x7f28d80ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.399+0000 7f28ded90700 1 --2- 172.21.15.7:0/425103120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d8154920 0x7f28d80ad7c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f28d8068f50 tx=0x7f28c4009250 comp rx=0 tx=0).stop 2024-01-27T02:32:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.399+0000 7f28ded90700 1 -- 172.21.15.7:0/425103120 shutdown_connections 2024-01-27T02:32:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.399+0000 7f28ded90700 1 --2- 172.21.15.7:0/425103120 >> v2:172.21.15.188:3300/0 conn(0x7f28d8154550 0x7f28d80ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.399+0000 7f28ded90700 1 --2- 172.21.15.7:0/425103120 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28d815b050 0x7f28d8063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.399+0000 7f28ded90700 1 --2- 172.21.15.7:0/425103120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d8154920 0x7f28d80ad7c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.399+0000 7f28ded90700 1 -- 172.21.15.7:0/425103120 >> 172.21.15.7:0/425103120 conn(0x7f28d8068480 msgr2=0x7f28d80cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.399+0000 7f28ded90700 1 -- 172.21.15.7:0/425103120 shutdown_connections 2024-01-27T02:32:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.399+0000 7f28ded90700 1 -- 172.21.15.7:0/425103120 wait complete. 2024-01-27T02:32:22.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.400+0000 7f28ded90700 1 Processor -- start 2024-01-27T02:32:22.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.400+0000 7f28ded90700 1 -- start start 2024-01-27T02:32:22.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.401+0000 7f28ded90700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d8154550 0x7f28d81575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:22.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.401+0000 7f28ded90700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28d8154920 0x7f28d816a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:22.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.401+0000 7f28ded90700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28d815b050 0x7f28d815c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:22.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.401+0000 7f28ded90700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d8155ce0 con 0x7f28d8154550 2024-01-27T02:32:22.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.401+0000 7f28ded90700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d8155fc0 con 0x7f28d8154920 2024-01-27T02:32:22.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.401+0000 7f28ded90700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d816a890 con 0x7f28d815b050 2024-01-27T02:32:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.401+0000 7f28dcb2c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d8154550 0x7f28d81575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.401+0000 7f28dcb2c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d8154550 0x7f28d81575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56206/0 (socket says 172.21.15.7:56206) 2024-01-27T02:32:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.401+0000 7f28dcb2c700 1 -- 172.21.15.7:0/3001453400 learned_addr learned my addr 172.21.15.7:0/3001453400 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.401+0000 7f28dd32d700 1 --2- 172.21.15.7:0/3001453400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28d815b050 0x7f28d815c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.401+0000 7f28d7fff700 1 --2- 172.21.15.7:0/3001453400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28d8154920 0x7f28d816a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.402+0000 7f28dcb2c700 1 -- 172.21.15.7:0/3001453400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28d815b050 msgr2=0x7f28d815c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.402+0000 7f28dcb2c700 1 --2- 172.21.15.7:0/3001453400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28d815b050 0x7f28d815c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.402+0000 7f28dcb2c700 1 -- 172.21.15.7:0/3001453400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28d8154920 msgr2=0x7f28d816a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.402+0000 7f28dcb2c700 1 --2- 172.21.15.7:0/3001453400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28d8154920 0x7f28d816a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.402+0000 7f28dcb2c700 1 -- 172.21.15.7:0/3001453400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c40094f0 con 0x7f28d8154550 2024-01-27T02:32:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.402+0000 7f28d7fff700 1 --2- 172.21.15.7:0/3001453400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28d8154920 0x7f28d816a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:32:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.402+0000 7f28dd32d700 1 --2- 172.21.15.7:0/3001453400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28d815b050 0x7f28d815c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:32:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.402+0000 7f28dcb2c700 1 --2- 172.21.15.7:0/3001453400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d8154550 0x7f28d81575b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f28cc00ec80 tx=0x7f28cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.402+0000 7f28d4ff9700 1 -- 172.21.15.7:0/3001453400 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc00c9d0 con 0x7f28d8154550 2024-01-27T02:32:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.402+0000 7f28d4ff9700 1 -- 172.21.15.7:0/3001453400 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28cc00cb30 con 0x7f28d8154550 2024-01-27T02:32:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.402+0000 7f28ded90700 1 -- 172.21.15.7:0/3001453400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d816ab30 con 0x7f28d8154550 2024-01-27T02:32:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.402+0000 7f28d4ff9700 1 -- 172.21.15.7:0/3001453400 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc013470 con 0x7f28d8154550 2024-01-27T02:32:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.402+0000 7f28ded90700 1 -- 172.21.15.7:0/3001453400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d816b0f0 con 0x7f28d8154550 2024-01-27T02:32:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.403+0000 7f28ded90700 1 -- 172.21.15.7:0/3001453400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d8072060 con 0x7f28d8154550 2024-01-27T02:32:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.403+0000 7f28d4ff9700 1 -- 172.21.15.7:0/3001453400 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f28cc013820 con 0x7f28d8154550 2024-01-27T02:32:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.404+0000 7f28d4ff9700 1 --2- 172.21.15.7:0/3001453400 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f28bc066f30 0x7f28bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.404+0000 7f28d4ff9700 1 -- 172.21.15.7:0/3001453400 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f28cc084960 con 0x7f28d8154550 2024-01-27T02:32:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.404+0000 7f28d7fff700 1 --2- 172.21.15.7:0/3001453400 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f28bc066f30 0x7f28bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:22.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.405+0000 7f28d7fff700 1 --2- 172.21.15.7:0/3001453400 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f28bc066f30 0x7f28bc0693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f28d806e330 tx=0x7f28c4017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:22.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.408+0000 7f28d4ff9700 1 -- 172.21.15.7:0/3001453400 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28cc051b30 con 0x7f28d8154550 2024-01-27T02:32:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.610+0000 7f28ded90700 1 -- 172.21.15.7:0/3001453400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f28d806dcd0 con 0x7f28d8154550 2024-01-27T02:32:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.610+0000 7f28d4ff9700 1 -- 172.21.15.7:0/3001453400 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f28cc055150 con 0x7f28d8154550 2024-01-27T02:32:22.612 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-27T02:32:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.612+0000 7f28ded90700 1 -- 172.21.15.7:0/3001453400 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f28bc066f30 msgr2=0x7f28bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.612+0000 7f28ded90700 1 --2- 172.21.15.7:0/3001453400 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f28bc066f30 0x7f28bc0693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f28d806e330 tx=0x7f28c4017000 comp rx=0 tx=0).stop 2024-01-27T02:32:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.613+0000 7f28ded90700 1 -- 172.21.15.7:0/3001453400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d8154550 msgr2=0x7f28d81575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.613+0000 7f28ded90700 1 --2- 172.21.15.7:0/3001453400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d8154550 0x7f28d81575b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f28cc00ec80 tx=0x7f28cc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.613+0000 7f28ded90700 1 -- 172.21.15.7:0/3001453400 shutdown_connections 2024-01-27T02:32:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.613+0000 7f28ded90700 1 --2- 172.21.15.7:0/3001453400 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f28bc066f30 0x7f28bc0693e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.613+0000 7f28ded90700 1 --2- 172.21.15.7:0/3001453400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28d815b050 0x7f28d815c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.613+0000 7f28ded90700 1 --2- 172.21.15.7:0/3001453400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d8154550 0x7f28d81575b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.613+0000 7f28ded90700 1 --2- 172.21.15.7:0/3001453400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28d8154920 0x7f28d816a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.613+0000 7f28ded90700 1 -- 172.21.15.7:0/3001453400 >> 172.21.15.7:0/3001453400 conn(0x7f28d8068480 msgr2=0x7f28d80b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.613+0000 7f28ded90700 1 -- 172.21.15.7:0/3001453400 shutdown_connections 2024-01-27T02:32:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.613+0000 7f28ded90700 1 -- 172.21.15.7:0/3001453400 wait complete. 2024-01-27T02:32:22.627 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-27T02:32:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.762+0000 7f9cc002b700 1 Processor -- start 2024-01-27T02:32:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.762+0000 7f9cc002b700 1 -- start start 2024-01-27T02:32:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.763+0000 7f9cc002b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9cb80af6b0 0x7f9cb80b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.763+0000 7f9cc002b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cb80b14f0 0x7f9cb80c34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.763+0000 7f9cc002b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cb80cb690 con 0x7f9cb80afa80 2024-01-27T02:32:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.763+0000 7f9cc002b700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cb806e1e0 con 0x7f9cb80af6b0 2024-01-27T02:32:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.763+0000 7f9cc002b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb80b1a90 con 0x7f9cb80b14f0 2024-01-27T02:32:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.763+0000 7f9cbd5c6700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9cb80afa80 0x7f9cb806dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43602/0 (socket says 172.21.15.7:43602) 2024-01-27T02:32:22.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.763+0000 7f9cbd5c6700 1 -- 172.21.15.7:0/1515093442 learned_addr learned my addr 172.21.15.7:0/1515093442 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:22.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.763+0000 7f9cbe5c8700 1 --2- 172.21.15.7:0/1515093442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cb80b14f0 0x7f9cb80c34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:22.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.763+0000 7f9cbddc7700 1 --2- 172.21.15.7:0/1515093442 >> v2:172.21.15.188:3300/0 conn(0x7f9cb80af6b0 0x7f9cb80b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:22.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.763+0000 7f9cbcdc5700 1 -- 172.21.15.7:0/1515093442 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2282500172 0 0) 0x7f9cb80cb690 con 0x7f9cb80afa80 2024-01-27T02:32:22.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.763+0000 7f9cbcdc5700 1 -- 172.21.15.7:0/1515093442 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ca4003570 con 0x7f9cb80afa80 2024-01-27T02:32:22.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.764+0000 7f9cbe5c8700 1 -- 172.21.15.7:0/1515093442 >> v2:172.21.15.188:3300/0 conn(0x7f9cb80af6b0 msgr2=0x7f9cb80b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:22.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.764+0000 7f9cbe5c8700 1 --2- 172.21.15.7:0/1515093442 >> v2:172.21.15.188:3300/0 conn(0x7f9cb80af6b0 0x7f9cb80b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.764+0000 7f9cbe5c8700 1 -- 172.21.15.7:0/1515093442 >> v1:172.21.15.7:6789/0 conn(0x7f9cb80afa80 legacy=0x7f9cb806dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:22.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.764+0000 7f9cbe5c8700 1 -- 172.21.15.7:0/1515093442 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb80b1d70 con 0x7f9cb80b14f0 2024-01-27T02:32:22.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.764+0000 7f9cbe5c8700 1 --2- 172.21.15.7:0/1515093442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cb80b14f0 0x7f9cb80c34d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009e60 tx=0x7f9cb0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=435c2ce9a3a1c3b0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:22.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.764+0000 7f9cbcdc5700 1 -- 172.21.15.7:0/1515093442 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb000e070 con 0x7f9cb80b14f0 2024-01-27T02:32:22.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.764+0000 7f9cbcdc5700 1 -- 172.21.15.7:0/1515093442 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb000bab0 con 0x7f9cb80b14f0 2024-01-27T02:32:22.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.764+0000 7f9cbcdc5700 1 -- 172.21.15.7:0/1515093442 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb000be70 con 0x7f9cb80b14f0 2024-01-27T02:32:22.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.765+0000 7f9cc002b700 1 -- 172.21.15.7:0/1515093442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cb80b14f0 msgr2=0x7f9cb80c34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:22.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.765+0000 7f9cc002b700 1 --2- 172.21.15.7:0/1515093442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cb80b14f0 0x7f9cb80c34d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009e60 tx=0x7f9cb0009780 comp rx=0 tx=0).stop 2024-01-27T02:32:22.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.765+0000 7f9cc002b700 1 -- 172.21.15.7:0/1515093442 shutdown_connections 2024-01-27T02:32:22.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.765+0000 7f9cc002b700 1 --2- 172.21.15.7:0/1515093442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cb80b14f0 0x7f9cb80c34d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.765+0000 7f9cc002b700 1 --2- 172.21.15.7:0/1515093442 >> v2:172.21.15.188:3300/0 conn(0x7f9cb80af6b0 0x7f9cb80b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.765+0000 7f9cc002b700 1 -- 172.21.15.7:0/1515093442 >> 172.21.15.7:0/1515093442 conn(0x7f9cb80679b0 msgr2=0x7f9cb8067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:22.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.765+0000 7f9cc002b700 1 -- 172.21.15.7:0/1515093442 shutdown_connections 2024-01-27T02:32:22.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.765+0000 7f9cc002b700 1 -- 172.21.15.7:0/1515093442 wait complete. 2024-01-27T02:32:22.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.766+0000 7f9cc002b700 1 Processor -- start 2024-01-27T02:32:22.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.766+0000 7f9cc002b700 1 -- start start 2024-01-27T02:32:22.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.767+0000 7f9cc002b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cb80af6b0 0x7f9cb8150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:22.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.767+0000 7f9cc002b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cb80afa80 0x7f9cb806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:22.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.767+0000 7f9cc002b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb80b14f0 0x7f9cb80c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:22.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.767+0000 7f9cc002b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb8151320 con 0x7f9cb80b14f0 2024-01-27T02:32:22.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.767+0000 7f9cc002b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb806ad80 con 0x7f9cb80af6b0 2024-01-27T02:32:22.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.767+0000 7f9cc002b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb80c81d0 con 0x7f9cb80afa80 2024-01-27T02:32:22.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.767+0000 7f9cbe5c8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb80b14f0 0x7f9cb80c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:22.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.767+0000 7f9cbe5c8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb80b14f0 0x7f9cb80c7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56220/0 (socket says 172.21.15.7:56220) 2024-01-27T02:32:22.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.767+0000 7f9cbe5c8700 1 -- 172.21.15.7:0/4003462661 learned_addr learned my addr 172.21.15.7:0/4003462661 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:22.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.767+0000 7f9cbd5c6700 1 --2- 172.21.15.7:0/4003462661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cb80afa80 0x7f9cb806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:22.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.767+0000 7f9cbddc7700 1 --2- 172.21.15.7:0/4003462661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cb80af6b0 0x7f9cb8150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:22.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.767+0000 7f9cbe5c8700 1 -- 172.21.15.7:0/4003462661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cb80afa80 msgr2=0x7f9cb806a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:22.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.767+0000 7f9cbe5c8700 1 --2- 172.21.15.7:0/4003462661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cb80afa80 0x7f9cb806a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.767+0000 7f9cbe5c8700 1 -- 172.21.15.7:0/4003462661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cb80af6b0 msgr2=0x7f9cb8150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:22.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.767+0000 7f9cbe5c8700 1 --2- 172.21.15.7:0/4003462661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cb80af6b0 0x7f9cb8150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.767+0000 7f9cbe5c8700 1 -- 172.21.15.7:0/4003462661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb0009a20 con 0x7f9cb80b14f0 2024-01-27T02:32:22.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.767+0000 7f9cbe5c8700 1 --2- 172.21.15.7:0/4003462661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb80b14f0 0x7f9cb80c7c90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80b1bd0 tx=0x7f9cb0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:22.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.768+0000 7f9cadffb700 1 -- 172.21.15.7:0/4003462661 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0012e10 con 0x7f9cb80b14f0 2024-01-27T02:32:22.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.768+0000 7f9cadffb700 1 -- 172.21.15.7:0/4003462661 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb0027470 con 0x7f9cb80b14f0 2024-01-27T02:32:22.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.768+0000 7f9cadffb700 1 -- 172.21.15.7:0/4003462661 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0027830 con 0x7f9cb80b14f0 2024-01-27T02:32:22.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.768+0000 7f9cc002b700 1 -- 172.21.15.7:0/4003462661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb8143e20 con 0x7f9cb80b14f0 2024-01-27T02:32:22.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.768+0000 7f9cc002b700 1 -- 172.21.15.7:0/4003462661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb80ca1a0 con 0x7f9cb80b14f0 2024-01-27T02:32:22.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.769+0000 7f9cc002b700 1 -- 172.21.15.7:0/4003462661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cb8072060 con 0x7f9cb80b14f0 2024-01-27T02:32:22.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.769+0000 7f9cadffb700 1 -- 172.21.15.7:0/4003462661 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9cb0024070 con 0x7f9cb80b14f0 2024-01-27T02:32:22.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.770+0000 7f9cadffb700 1 --2- 172.21.15.7:0/4003462661 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9c9c066f30 0x7f9c9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:22.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.770+0000 7f9cadffb700 1 -- 172.21.15.7:0/4003462661 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9cb0086fd0 con 0x7f9cb80b14f0 2024-01-27T02:32:22.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.770+0000 7f9cbddc7700 1 --2- 172.21.15.7:0/4003462661 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9c9c066f30 0x7f9c9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:22.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.770+0000 7f9cbddc7700 1 --2- 172.21.15.7:0/4003462661 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9c9c066f30 0x7f9c9c0693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9cb81555b0 tx=0x7f9cb400b780 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:22.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.774+0000 7f9cadffb700 1 -- 172.21.15.7:0/4003462661 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9cb00541a0 con 0x7f9cb80b14f0 2024-01-27T02:32:22.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.977+0000 7f9cc002b700 1 -- 172.21.15.7:0/4003462661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9cb806a620 con 0x7f9cb80b14f0 2024-01-27T02:32:22.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.977+0000 7f9cadffb700 1 -- 172.21.15.7:0/4003462661 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9cb002c0a0 con 0x7f9cb80b14f0 2024-01-27T02:32:22.979 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-27T02:32:22.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.980+0000 7f9cc002b700 1 -- 172.21.15.7:0/4003462661 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9c9c066f30 msgr2=0x7f9c9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:22.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.980+0000 7f9cc002b700 1 --2- 172.21.15.7:0/4003462661 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9c9c066f30 0x7f9c9c0693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9cb81555b0 tx=0x7f9cb400b780 comp rx=0 tx=0).stop 2024-01-27T02:32:22.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.980+0000 7f9cc002b700 1 -- 172.21.15.7:0/4003462661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb80b14f0 msgr2=0x7f9cb80c7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:22.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.980+0000 7f9cc002b700 1 --2- 172.21.15.7:0/4003462661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb80b14f0 0x7f9cb80c7c90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80b1bd0 tx=0x7f9cb0019000 comp rx=0 tx=0).stop 2024-01-27T02:32:22.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.980+0000 7f9cc002b700 1 -- 172.21.15.7:0/4003462661 shutdown_connections 2024-01-27T02:32:22.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.980+0000 7f9cc002b700 1 --2- 172.21.15.7:0/4003462661 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9c9c066f30 0x7f9c9c0693e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.980+0000 7f9cc002b700 1 --2- 172.21.15.7:0/4003462661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb80b14f0 0x7f9cb80c7c90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.980+0000 7f9cc002b700 1 --2- 172.21.15.7:0/4003462661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cb80af6b0 0x7f9cb8150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.980+0000 7f9cc002b700 1 --2- 172.21.15.7:0/4003462661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cb80afa80 0x7f9cb806a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:22.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.980+0000 7f9cc002b700 1 -- 172.21.15.7:0/4003462661 >> 172.21.15.7:0/4003462661 conn(0x7f9cb80679b0 msgr2=0x7f9cb8067d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:22.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.980+0000 7f9cc002b700 1 -- 172.21.15.7:0/4003462661 shutdown_connections 2024-01-27T02:32:22.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:22.980+0000 7f9cc002b700 1 -- 172.21.15.7:0/4003462661 wait complete. 2024-01-27T02:32:22.995 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:32:23.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.124+0000 7f3b16fc4700 1 Processor -- start 2024-01-27T02:32:23.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.124+0000 7f3b16fc4700 1 -- start start 2024-01-27T02:32:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.126+0000 7f3b16fc4700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3b1015be10 0x7f3b100b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.126+0000 7f3b16fc4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3b1015c1e0 0x7f3b100c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.126+0000 7f3b16fc4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b100b1470 con 0x7f3b100b0180 2024-01-27T02:32:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.126+0000 7f3b16fc4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b100b0600 con 0x7f3b1015c1e0 2024-01-27T02:32:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.126+0000 7f3b16fc4700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b100b0860 con 0x7f3b1015be10 2024-01-27T02:32:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.126+0000 7f3b15561700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3b1015c1e0 0x7f3b100c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.126+0000 7f3b0ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3b100b0180 0x7f3b100aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43606/0 (socket says 172.21.15.7:43606) 2024-01-27T02:32:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.126+0000 7f3b0ffff700 1 -- 172.21.15.7:0/708519995 learned_addr learned my addr 172.21.15.7:0/708519995 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.126+0000 7f3b14d60700 1 --2- 172.21.15.7:0/708519995 >> v2:172.21.15.188:3300/0 conn(0x7f3b1015be10 0x7f3b100b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.126+0000 7f3b15561700 1 -- 172.21.15.7:0/708519995 >> v2:172.21.15.188:3300/0 conn(0x7f3b1015be10 msgr2=0x7f3b100b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.126+0000 7f3b15561700 1 --2- 172.21.15.7:0/708519995 >> v2:172.21.15.188:3300/0 conn(0x7f3b1015be10 0x7f3b100b1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.126+0000 7f3b15561700 1 -- 172.21.15.7:0/708519995 >> v1:172.21.15.7:6789/0 conn(0x7f3b100b0180 legacy=0x7f3b100aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:23.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.126+0000 7f3b15561700 1 -- 172.21.15.7:0/708519995 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b100b0b40 con 0x7f3b1015c1e0 2024-01-27T02:32:23.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.126+0000 7f3b0f7fe700 1 -- 172.21.15.7:0/708519995 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2499120442 0 0) 0x7f3b100b1470 con 0x7f3b100b0180 2024-01-27T02:32:23.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.126+0000 7f3b14d60700 1 --2- 172.21.15.7:0/708519995 >> v2:172.21.15.188:3300/0 conn(0x7f3b1015be10 0x7f3b100b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:23.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.126+0000 7f3b15561700 1 --2- 172.21.15.7:0/708519995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b1015c1e0 0x7f3b100c9500 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3b100693e0 tx=0x7f3b08009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87132af45a879ca0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:23.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.126+0000 7f3b0f7fe700 1 -- 172.21.15.7:0/708519995 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0800e070 con 0x7f3b1015c1e0 2024-01-27T02:32:23.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.127+0000 7f3b0f7fe700 1 -- 172.21.15.7:0/708519995 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0800bc30 con 0x7f3b1015c1e0 2024-01-27T02:32:23.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.127+0000 7f3b0f7fe700 1 -- 172.21.15.7:0/708519995 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b08012430 con 0x7f3b1015c1e0 2024-01-27T02:32:23.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.127+0000 7f3b16fc4700 1 -- 172.21.15.7:0/708519995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b1015c1e0 msgr2=0x7f3b100c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:23.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.127+0000 7f3b16fc4700 1 --2- 172.21.15.7:0/708519995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b1015c1e0 0x7f3b100c9500 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3b100693e0 tx=0x7f3b08009780 comp rx=0 tx=0).stop 2024-01-27T02:32:23.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.127+0000 7f3b16fc4700 1 -- 172.21.15.7:0/708519995 shutdown_connections 2024-01-27T02:32:23.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.127+0000 7f3b16fc4700 1 --2- 172.21.15.7:0/708519995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b1015c1e0 0x7f3b100c9500 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.127+0000 7f3b16fc4700 1 --2- 172.21.15.7:0/708519995 >> v2:172.21.15.188:3300/0 conn(0x7f3b1015be10 0x7f3b100b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.127+0000 7f3b16fc4700 1 -- 172.21.15.7:0/708519995 >> 172.21.15.7:0/708519995 conn(0x7f3b100679b0 msgr2=0x7f3b10067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:23.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.127+0000 7f3b16fc4700 1 -- 172.21.15.7:0/708519995 shutdown_connections 2024-01-27T02:32:23.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.127+0000 7f3b16fc4700 1 -- 172.21.15.7:0/708519995 wait complete. 2024-01-27T02:32:23.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.128+0000 7f3b16fc4700 1 Processor -- start 2024-01-27T02:32:23.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.129+0000 7f3b16fc4700 1 -- start start 2024-01-27T02:32:23.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.129+0000 7f3b16fc4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b100b0180 0x7f3b10153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:23.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.129+0000 7f3b16fc4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b1015be10 0x7f3b1016d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:23.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.129+0000 7f3b16fc4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b1015c1e0 0x7f3b1016dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:23.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.129+0000 7f3b16fc4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1016e630 con 0x7f3b1015be10 2024-01-27T02:32:23.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.129+0000 7f3b16fc4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1009a310 con 0x7f3b1015c1e0 2024-01-27T02:32:23.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.129+0000 7f3b16fc4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b10099420 con 0x7f3b100b0180 2024-01-27T02:32:23.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.130+0000 7f3b0ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b1015be10 0x7f3b1016d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:23.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.130+0000 7f3b0ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b1015be10 0x7f3b1016d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56242/0 (socket says 172.21.15.7:56242) 2024-01-27T02:32:23.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.130+0000 7f3b0ffff700 1 -- 172.21.15.7:0/1796976455 learned_addr learned my addr 172.21.15.7:0/1796976455 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:23.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.130+0000 7f3b14d60700 1 --2- 172.21.15.7:0/1796976455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b100b0180 0x7f3b10153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:23.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.130+0000 7f3b15561700 1 --2- 172.21.15.7:0/1796976455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b1015c1e0 0x7f3b1016dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:23.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.130+0000 7f3b0ffff700 1 -- 172.21.15.7:0/1796976455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b100b0180 msgr2=0x7f3b10153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:23.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.130+0000 7f3b0ffff700 1 --2- 172.21.15.7:0/1796976455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b100b0180 0x7f3b10153ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.130+0000 7f3b0ffff700 1 -- 172.21.15.7:0/1796976455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b1015c1e0 msgr2=0x7f3b1016dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:23.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.130+0000 7f3b0ffff700 1 --2- 172.21.15.7:0/1796976455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b1015c1e0 0x7f3b1016dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.130+0000 7f3b0ffff700 1 -- 172.21.15.7:0/1796976455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b08009a20 con 0x7f3b1015be10 2024-01-27T02:32:23.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.130+0000 7f3b0ffff700 1 --2- 172.21.15.7:0/1796976455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b1015be10 0x7f3b1016d7c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3af800c810 tx=0x7f3af8007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:23.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.130+0000 7f3b0cff9700 1 -- 172.21.15.7:0/1796976455 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8005c00 con 0x7f3b1015be10 2024-01-27T02:32:23.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.130+0000 7f3b0cff9700 1 -- 172.21.15.7:0/1796976455 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af8005d60 con 0x7f3b1015be10 2024-01-27T02:32:23.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.130+0000 7f3b0cff9700 1 -- 172.21.15.7:0/1796976455 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8007320 con 0x7f3b1015be10 2024-01-27T02:32:23.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.130+0000 7f3b16fc4700 1 -- 172.21.15.7:0/1796976455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b10099700 con 0x7f3b1015be10 2024-01-27T02:32:23.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.130+0000 7f3b16fc4700 1 -- 172.21.15.7:0/1796976455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b10099cc0 con 0x7f3b1015be10 2024-01-27T02:32:23.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.131+0000 7f3b16fc4700 1 -- 172.21.15.7:0/1796976455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1004c710 con 0x7f3b1015be10 2024-01-27T02:32:23.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.132+0000 7f3b0cff9700 1 -- 172.21.15.7:0/1796976455 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3af801b400 con 0x7f3b1015be10 2024-01-27T02:32:23.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.132+0000 7f3b0cff9700 1 --2- 172.21.15.7:0/1796976455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3af0066f30 0x7f3af00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:23.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.132+0000 7f3b0cff9700 1 -- 172.21.15.7:0/1796976455 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3af80840f0 con 0x7f3b1015be10 2024-01-27T02:32:23.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.133+0000 7f3b14d60700 1 --2- 172.21.15.7:0/1796976455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3af0066f30 0x7f3af00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:23.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.133+0000 7f3b14d60700 1 --2- 172.21.15.7:0/1796976455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3af0066f30 0x7f3af00693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3b00006f00 tx=0x7f3b00008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:23.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.137+0000 7f3b0cff9700 1 -- 172.21.15.7:0/1796976455 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3af8051340 con 0x7f3b1015be10 2024-01-27T02:32:23.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.388+0000 7f3b16fc4700 1 -- 172.21.15.7:0/1796976455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b100b0e20 con 0x7f3b1015be10 2024-01-27T02:32:23.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.388+0000 7f3b0cff9700 1 -- 172.21.15.7:0/1796976455 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f3af8054960 con 0x7f3b1015be10 2024-01-27T02:32:23.390 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:23.390 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3594367777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3594367777},{"type":"v1","addr":"172.21.15.7:6834","nonce":3594367777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1079101213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1079101213},{"type":"v1","addr":"172.21.15.188:6837","nonce":1079101213}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3241424244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3241424244},{"type":"v1","addr":"172.21.15.7:6835","nonce":3241424244}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:32:06.045693+0000","modified":"2024-01-27T02:32:09.806398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6836/2571237935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2571237935},{"type":"v1","addr":"172.21.15.188:6836","nonce":2571237935}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-27T02:32:23.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.390+0000 7f3b16fc4700 1 -- 172.21.15.7:0/1796976455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3af0066f30 msgr2=0x7f3af00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:23.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.390+0000 7f3b16fc4700 1 --2- 172.21.15.7:0/1796976455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3af0066f30 0x7f3af00693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3b00006f00 tx=0x7f3b00008040 comp rx=0 tx=0).stop 2024-01-27T02:32:23.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.390+0000 7f3b16fc4700 1 -- 172.21.15.7:0/1796976455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b1015be10 msgr2=0x7f3b1016d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:23.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.390+0000 7f3b16fc4700 1 --2- 172.21.15.7:0/1796976455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b1015be10 0x7f3b1016d7c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3af800c810 tx=0x7f3af8007db0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.390+0000 7f3b16fc4700 1 -- 172.21.15.7:0/1796976455 shutdown_connections 2024-01-27T02:32:23.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.390+0000 7f3b16fc4700 1 --2- 172.21.15.7:0/1796976455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3af0066f30 0x7f3af00693e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.390+0000 7f3b16fc4700 1 --2- 172.21.15.7:0/1796976455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b1015c1e0 0x7f3b1016dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.390+0000 7f3b16fc4700 1 --2- 172.21.15.7:0/1796976455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b100b0180 0x7f3b10153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.390+0000 7f3b16fc4700 1 --2- 172.21.15.7:0/1796976455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b1015be10 0x7f3b1016d7c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.391+0000 7f3b16fc4700 1 -- 172.21.15.7:0/1796976455 >> 172.21.15.7:0/1796976455 conn(0x7f3b100679b0 msgr2=0x7f3b10069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:23.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.391+0000 7f3b16fc4700 1 -- 172.21.15.7:0/1796976455 shutdown_connections 2024-01-27T02:32:23.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.391+0000 7f3b16fc4700 1 -- 172.21.15.7:0/1796976455 wait complete. 2024-01-27T02:32:23.393 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-01-27T02:32:23.405 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:32:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.535+0000 7f06a72ec700 1 Processor -- start 2024-01-27T02:32:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.535+0000 7f06a72ec700 1 -- start start 2024-01-27T02:32:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.536+0000 7f06a72ec700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f06a015a100 0x7f06a00cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.536+0000 7f06a72ec700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a00cb5b0 0x7f06a00c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.536+0000 7f06a72ec700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a00ce5e0 con 0x7f06a00c4af0 2024-01-27T02:32:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.536+0000 7f06a72ec700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a00c4f70 con 0x7f06a015a100 2024-01-27T02:32:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.536+0000 7f06a72ec700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a00cd9d0 con 0x7f06a00cb5b0 2024-01-27T02:32:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.537+0000 7f06a4887700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a00cb5b0 0x7f06a00c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.537+0000 7f06a5088700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f06a015a100 0x7f06a00cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.537+0000 7f06a5889700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f06a00c4af0 0x7f06a00c9530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:46304/0 (socket says 172.21.15.7:46304) 2024-01-27T02:32:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.537+0000 7f06a5889700 1 -- 172.21.15.7:0/688414509 learned_addr learned my addr 172.21.15.7:0/688414509 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.537+0000 7f0693fff700 1 -- 172.21.15.7:0/688414509 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487584539 0 0) 0x7f06a00ce5e0 con 0x7f06a00c4af0 2024-01-27T02:32:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.537+0000 7f0693fff700 1 -- 172.21.15.7:0/688414509 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f068c003570 con 0x7f06a00c4af0 2024-01-27T02:32:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.537+0000 7f06a5088700 1 -- 172.21.15.7:0/688414509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a00cb5b0 msgr2=0x7f06a00c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.537+0000 7f06a5088700 1 --2- 172.21.15.7:0/688414509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a00cb5b0 0x7f06a00c45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.537+0000 7f06a5088700 1 -- 172.21.15.7:0/688414509 >> v1:172.21.15.188:6789/0 conn(0x7f06a00c4af0 legacy=0x7f06a00c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.537+0000 7f06a5088700 1 -- 172.21.15.7:0/688414509 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a00cdcb0 con 0x7f06a015a100 2024-01-27T02:32:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.537+0000 7f06a4887700 1 --2- 172.21.15.7:0/688414509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a00cb5b0 0x7f06a00c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:32:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.537+0000 7f06a5088700 1 --2- 172.21.15.7:0/688414509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a015a100 0x7f06a00cd2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f069c0098e0 tx=0x7f069c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48d965d9cdf70eb1 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.537+0000 7f0693fff700 1 -- 172.21.15.7:0/688414509 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c011630 con 0x7f06a015a100 2024-01-27T02:32:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.537+0000 7f0693fff700 1 -- 172.21.15.7:0/688414509 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069c011790 con 0x7f06a015a100 2024-01-27T02:32:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.537+0000 7f0693fff700 1 -- 172.21.15.7:0/688414509 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c011b30 con 0x7f06a015a100 2024-01-27T02:32:23.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.538+0000 7f06a72ec700 1 -- 172.21.15.7:0/688414509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a015a100 msgr2=0x7f06a00cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:23.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.538+0000 7f06a72ec700 1 --2- 172.21.15.7:0/688414509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a015a100 0x7f06a00cd2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f069c0098e0 tx=0x7f069c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.538+0000 7f06a72ec700 1 -- 172.21.15.7:0/688414509 shutdown_connections 2024-01-27T02:32:23.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.538+0000 7f06a72ec700 1 --2- 172.21.15.7:0/688414509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a015a100 0x7f06a00cd2f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.538+0000 7f06a72ec700 1 --2- 172.21.15.7:0/688414509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a00cb5b0 0x7f06a00c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.538+0000 7f06a72ec700 1 -- 172.21.15.7:0/688414509 >> 172.21.15.7:0/688414509 conn(0x7f06a0098020 msgr2=0x7f06a00711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.538+0000 7f06a72ec700 1 -- 172.21.15.7:0/688414509 shutdown_connections 2024-01-27T02:32:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.538+0000 7f06a72ec700 1 -- 172.21.15.7:0/688414509 wait complete. 2024-01-27T02:32:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.539+0000 7f06a72ec700 1 Processor -- start 2024-01-27T02:32:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.540+0000 7f06a72ec700 1 -- start start 2024-01-27T02:32:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.540+0000 7f06a72ec700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a00c4af0 0x7f06a0156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.540+0000 7f06a72ec700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a00cb5b0 0x7f06a00c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.540+0000 7f06a72ec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06a015a100 0x7f06a006c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.540+0000 7f06a72ec700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a00b2bf0 con 0x7f06a015a100 2024-01-27T02:32:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.540+0000 7f06a72ec700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a00c64e0 con 0x7f06a00c4af0 2024-01-27T02:32:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.540+0000 7f06a72ec700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a0157300 con 0x7f06a00cb5b0 2024-01-27T02:32:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.541+0000 7f06a5889700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06a015a100 0x7f06a006c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.541+0000 7f06a5889700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06a015a100 0x7f06a006c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56248/0 (socket says 172.21.15.7:56248) 2024-01-27T02:32:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.541+0000 7f06a5889700 1 -- 172.21.15.7:0/260007192 learned_addr learned my addr 172.21.15.7:0/260007192 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.541+0000 7f06a4887700 1 --2- 172.21.15.7:0/260007192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a00cb5b0 0x7f06a00c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.541+0000 7f06a5088700 1 --2- 172.21.15.7:0/260007192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a00c4af0 0x7f06a0156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.541+0000 7f06a5889700 1 -- 172.21.15.7:0/260007192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a00cb5b0 msgr2=0x7f06a00c6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.541+0000 7f06a5889700 1 --2- 172.21.15.7:0/260007192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a00cb5b0 0x7f06a00c6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.541+0000 7f06a5889700 1 -- 172.21.15.7:0/260007192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a00c4af0 msgr2=0x7f06a0156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.541+0000 7f06a5889700 1 --2- 172.21.15.7:0/260007192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a00c4af0 0x7f06a0156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.541+0000 7f06a5889700 1 -- 172.21.15.7:0/260007192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0694009420 con 0x7f06a015a100 2024-01-27T02:32:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.541+0000 7f06a5889700 1 --2- 172.21.15.7:0/260007192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06a015a100 0x7f06a006c6e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f069800cc40 tx=0x7f0698001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.541+0000 7f06917fa700 1 -- 172.21.15.7:0/260007192 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069800a830 con 0x7f06a015a100 2024-01-27T02:32:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.541+0000 7f06917fa700 1 -- 172.21.15.7:0/260007192 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069800a990 con 0x7f06a015a100 2024-01-27T02:32:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.541+0000 7f06a72ec700 1 -- 172.21.15.7:0/260007192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069c009420 con 0x7f06a015a100 2024-01-27T02:32:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.541+0000 7f06917fa700 1 -- 172.21.15.7:0/260007192 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069800ad10 con 0x7f06a015a100 2024-01-27T02:32:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.541+0000 7f06a72ec700 1 -- 172.21.15.7:0/260007192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a00c5e90 con 0x7f06a015a100 2024-01-27T02:32:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.542+0000 7f06a72ec700 1 -- 172.21.15.7:0/260007192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a00c6690 con 0x7f06a015a100 2024-01-27T02:32:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.543+0000 7f06917fa700 1 -- 172.21.15.7:0/260007192 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0698007300 con 0x7f06a015a100 2024-01-27T02:32:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.543+0000 7f06917fa700 1 --2- 172.21.15.7:0/260007192 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0684066f30 0x7f06840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.543+0000 7f06917fa700 1 -- 172.21.15.7:0/260007192 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0698084830 con 0x7f06a015a100 2024-01-27T02:32:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.544+0000 7f06a5088700 1 --2- 172.21.15.7:0/260007192 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0684066f30 0x7f06840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.544+0000 7f06a5088700 1 --2- 172.21.15.7:0/260007192 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0684066f30 0x7f06840693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f069c0091e0 tx=0x7f069c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.548+0000 7f06917fa700 1 -- 172.21.15.7:0/260007192 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f06a00c6690 con 0x7f06a015a100 2024-01-27T02:32:23.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.752+0000 7f06a72ec700 1 -- 172.21.15.7:0/260007192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06a00c6690 con 0x7f06a015a100 2024-01-27T02:32:23.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.753+0000 7f06917fa700 1 -- 172.21.15.7:0/260007192 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f06a00c6690 con 0x7f06a015a100 2024-01-27T02:32:23.755 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:23.755 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:32:07.627822+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:32:03.675611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-27T02:32:05.018885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:32:23.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.755+0000 7f06a72ec700 1 -- 172.21.15.7:0/260007192 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0684066f30 msgr2=0x7f06840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:23.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.756+0000 7f06a72ec700 1 --2- 172.21.15.7:0/260007192 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0684066f30 0x7f06840693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f069c0091e0 tx=0x7f069c017000 comp rx=0 tx=0).stop 2024-01-27T02:32:23.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.756+0000 7f06a72ec700 1 -- 172.21.15.7:0/260007192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06a015a100 msgr2=0x7f06a006c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:23.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.756+0000 7f06a72ec700 1 --2- 172.21.15.7:0/260007192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06a015a100 0x7f06a006c6e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f069800cc40 tx=0x7f0698001a40 comp rx=0 tx=0).stop 2024-01-27T02:32:23.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.756+0000 7f06a72ec700 1 -- 172.21.15.7:0/260007192 shutdown_connections 2024-01-27T02:32:23.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.756+0000 7f06a72ec700 1 --2- 172.21.15.7:0/260007192 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0684066f30 0x7f06840693e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.756+0000 7f06a72ec700 1 --2- 172.21.15.7:0/260007192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06a015a100 0x7f06a006c6e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.756+0000 7f06a72ec700 1 --2- 172.21.15.7:0/260007192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a00c4af0 0x7f06a0156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.756+0000 7f06a72ec700 1 --2- 172.21.15.7:0/260007192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a00cb5b0 0x7f06a00c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.756+0000 7f06a72ec700 1 -- 172.21.15.7:0/260007192 >> 172.21.15.7:0/260007192 conn(0x7f06a0098020 msgr2=0x7f06a0158b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:23.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.756+0000 7f06a72ec700 1 -- 172.21.15.7:0/260007192 shutdown_connections 2024-01-27T02:32:23.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.756+0000 7f06a72ec700 1 -- 172.21.15.7:0/260007192 wait complete. 2024-01-27T02:32:23.771 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:32:23.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.901+0000 7fa65c49d700 1 Processor -- start 2024-01-27T02:32:23.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.901+0000 7fa65c49d700 1 -- start start 2024-01-27T02:32:23.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.902+0000 7fa65c49d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6540ad390 0x7fa6540a7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:23.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.902+0000 7fa65c49d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6540afd30 con 0x7fa6541566a0 2024-01-27T02:32:23.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.902+0000 7fa65c49d700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65406df80 con 0x7fa654156a70 2024-01-27T02:32:23.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.902+0000 7fa65c49d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6540af770 con 0x7fa6540ad390 2024-01-27T02:32:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.902+0000 7fa65a239700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa6541566a0 0x7fa6540aee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43610/0 (socket says 172.21.15.7:43610) 2024-01-27T02:32:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.902+0000 7fa65a239700 1 -- 172.21.15.7:0/1395586458 learned_addr learned my addr 172.21.15.7:0/1395586458 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.902+0000 7fa65aa3a700 1 --2- 172.21.15.7:0/1395586458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6540ad390 0x7fa6540a7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.902+0000 7fa659237700 1 -- 172.21.15.7:0/1395586458 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 238259106 0 0) 0x7fa6540afd30 con 0x7fa6541566a0 2024-01-27T02:32:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.903+0000 7fa659237700 1 -- 172.21.15.7:0/1395586458 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa63c003570 con 0x7fa6541566a0 2024-01-27T02:32:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.903+0000 7fa659237700 1 -- 172.21.15.7:0/1395586458 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936716704 0 0) 0x7fa65406df80 con 0x7fa654156a70 2024-01-27T02:32:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.903+0000 7fa659237700 1 -- 172.21.15.7:0/1395586458 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6540afd30 con 0x7fa654156a70 2024-01-27T02:32:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.903+0000 7fa65aa3a700 1 -- 172.21.15.7:0/1395586458 >> v1:172.21.15.188:6789/0 conn(0x7fa654156a70 legacy=0x7fa6540acc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.903+0000 7fa65aa3a700 1 -- 172.21.15.7:0/1395586458 >> v1:172.21.15.7:6789/0 conn(0x7fa6541566a0 legacy=0x7fa6540aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.903+0000 7fa65aa3a700 1 -- 172.21.15.7:0/1395586458 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6540afa50 con 0x7fa6540ad390 2024-01-27T02:32:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.903+0000 7fa65aa3a700 1 --2- 172.21.15.7:0/1395586458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6540ad390 0x7fa6540a7e20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa64c009910 tx=0x7fa64c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6637c9241810068 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.903+0000 7fa659237700 1 -- 172.21.15.7:0/1395586458 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c011630 con 0x7fa6540ad390 2024-01-27T02:32:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.903+0000 7fa659237700 1 -- 172.21.15.7:0/1395586458 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64c011790 con 0x7fa6540ad390 2024-01-27T02:32:23.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.903+0000 7fa659237700 1 -- 172.21.15.7:0/1395586458 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c011b30 con 0x7fa6540ad390 2024-01-27T02:32:23.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.904+0000 7fa65c49d700 1 -- 172.21.15.7:0/1395586458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6540ad390 msgr2=0x7fa6540a7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:23.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.904+0000 7fa65c49d700 1 --2- 172.21.15.7:0/1395586458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6540ad390 0x7fa6540a7e20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa64c009910 tx=0x7fa64c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.904+0000 7fa65c49d700 1 -- 172.21.15.7:0/1395586458 shutdown_connections 2024-01-27T02:32:23.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.904+0000 7fa65c49d700 1 --2- 172.21.15.7:0/1395586458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6540ad390 0x7fa6540a7e20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.904+0000 7fa65c49d700 1 -- 172.21.15.7:0/1395586458 >> 172.21.15.7:0/1395586458 conn(0x7fa6540679b0 msgr2=0x7fa654067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:23.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.904+0000 7fa65c49d700 1 -- 172.21.15.7:0/1395586458 shutdown_connections 2024-01-27T02:32:23.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.904+0000 7fa65c49d700 1 -- 172.21.15.7:0/1395586458 wait complete. 2024-01-27T02:32:23.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.905+0000 7fa65c49d700 1 Processor -- start 2024-01-27T02:32:23.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.905+0000 7fa65c49d700 1 -- start start 2024-01-27T02:32:23.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.906+0000 7fa65c49d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6540ad390 0x7fa654097a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:23.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.906+0000 7fa65c49d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6541566a0 0x7fa654169870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:23.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.906+0000 7fa65c49d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa654156a70 0x7fa6541582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:23.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.906+0000 7fa65c49d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa654148510 con 0x7fa6541566a0 2024-01-27T02:32:23.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.906+0000 7fa65c49d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa654157220 con 0x7fa654156a70 2024-01-27T02:32:23.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.906+0000 7fa65c49d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa654157500 con 0x7fa6540ad390 2024-01-27T02:32:23.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.906+0000 7fa659a38700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6541566a0 0x7fa654169870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:23.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.906+0000 7fa659a38700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6541566a0 0x7fa654169870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56252/0 (socket says 172.21.15.7:56252) 2024-01-27T02:32:23.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.906+0000 7fa659a38700 1 -- 172.21.15.7:0/1011082453 learned_addr learned my addr 172.21.15.7:0/1011082453 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:23.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.906+0000 7fa65a239700 1 --2- 172.21.15.7:0/1011082453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6540ad390 0x7fa654097a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:23.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.906+0000 7fa65aa3a700 1 --2- 172.21.15.7:0/1011082453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa654156a70 0x7fa6541582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:23.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.906+0000 7fa659a38700 1 -- 172.21.15.7:0/1011082453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6540ad390 msgr2=0x7fa654097a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:23.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.906+0000 7fa659a38700 1 --2- 172.21.15.7:0/1011082453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6540ad390 0x7fa654097a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.906+0000 7fa659a38700 1 -- 172.21.15.7:0/1011082453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa654156a70 msgr2=0x7fa6541582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:23.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.906+0000 7fa659a38700 1 --2- 172.21.15.7:0/1011082453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa654156a70 0x7fa6541582c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:23.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.906+0000 7fa659a38700 1 -- 172.21.15.7:0/1011082453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c009420 con 0x7fa6541566a0 2024-01-27T02:32:23.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.907+0000 7fa659a38700 1 --2- 172.21.15.7:0/1011082453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6541566a0 0x7fa654169870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa64800c790 tx=0x7fa648007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:23.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.907+0000 7fa6527fc700 1 -- 172.21.15.7:0/1011082453 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa648010510 con 0x7fa6541566a0 2024-01-27T02:32:23.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.907+0000 7fa6527fc700 1 -- 172.21.15.7:0/1011082453 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa648010670 con 0x7fa6541566a0 2024-01-27T02:32:23.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.907+0000 7fa6527fc700 1 -- 172.21.15.7:0/1011082453 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa648010a10 con 0x7fa6541566a0 2024-01-27T02:32:23.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.907+0000 7fa65c49d700 1 -- 172.21.15.7:0/1011082453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6541577e0 con 0x7fa6541566a0 2024-01-27T02:32:23.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.907+0000 7fa65c49d700 1 -- 172.21.15.7:0/1011082453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa654157da0 con 0x7fa6541566a0 2024-01-27T02:32:23.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.908+0000 7fa65c49d700 1 -- 172.21.15.7:0/1011082453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa65404c710 con 0x7fa6541566a0 2024-01-27T02:32:23.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.909+0000 7fa6527fc700 1 -- 172.21.15.7:0/1011082453 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa64801b3b0 con 0x7fa6541566a0 2024-01-27T02:32:23.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.909+0000 7fa6527fc700 1 --2- 172.21.15.7:0/1011082453 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa640066f30 0x7fa6400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:23.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.909+0000 7fa6527fc700 1 -- 172.21.15.7:0/1011082453 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa648084280 con 0x7fa6541566a0 2024-01-27T02:32:23.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.909+0000 7fa65a239700 1 --2- 172.21.15.7:0/1011082453 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa640066f30 0x7fa6400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:23.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.910+0000 7fa65a239700 1 --2- 172.21.15.7:0/1011082453 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa640066f30 0x7fa6400693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa644005c50 tx=0x7fa644009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:23.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:23.913+0000 7fa6527fc700 1 -- 172.21.15.7:0/1011082453 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6480514d0 con 0x7fa6541566a0 2024-01-27T02:32:24.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.161+0000 7fa65c49d700 1 -- 172.21.15.7:0/1011082453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa654069410 con 0x7fa6541566a0 2024-01-27T02:32:24.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.162+0000 7fa6527fc700 1 -- 172.21.15.7:0/1011082453 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7fa648017020 con 0x7fa6541566a0 2024-01-27T02:32:24.163 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:24.163 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3594367777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3594367777},{"type":"v1","addr":"172.21.15.7:6834","nonce":3594367777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1079101213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1079101213},{"type":"v1","addr":"172.21.15.188:6837","nonce":1079101213}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3241424244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3241424244},{"type":"v1","addr":"172.21.15.7:6835","nonce":3241424244}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:32:06.045693+0000","modified":"2024-01-27T02:32:09.806398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6836/2571237935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2571237935},{"type":"v1","addr":"172.21.15.188:6836","nonce":2571237935}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-27T02:32:24.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.164+0000 7fa65c49d700 1 -- 172.21.15.7:0/1011082453 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa640066f30 msgr2=0x7fa6400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:24.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.164+0000 7fa65c49d700 1 --2- 172.21.15.7:0/1011082453 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa640066f30 0x7fa6400693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa644005c50 tx=0x7fa644009040 comp rx=0 tx=0).stop 2024-01-27T02:32:24.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.164+0000 7fa65c49d700 1 -- 172.21.15.7:0/1011082453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6541566a0 msgr2=0x7fa654169870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:24.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.164+0000 7fa65c49d700 1 --2- 172.21.15.7:0/1011082453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6541566a0 0x7fa654169870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa64800c790 tx=0x7fa648007e10 comp rx=0 tx=0).stop 2024-01-27T02:32:24.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.164+0000 7fa65c49d700 1 -- 172.21.15.7:0/1011082453 shutdown_connections 2024-01-27T02:32:24.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.164+0000 7fa65c49d700 1 --2- 172.21.15.7:0/1011082453 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa640066f30 0x7fa6400693e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.164+0000 7fa65c49d700 1 --2- 172.21.15.7:0/1011082453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa654156a70 0x7fa6541582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.164+0000 7fa65c49d700 1 --2- 172.21.15.7:0/1011082453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6540ad390 0x7fa654097a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.164+0000 7fa65c49d700 1 --2- 172.21.15.7:0/1011082453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6541566a0 0x7fa654169870 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.164+0000 7fa65c49d700 1 -- 172.21.15.7:0/1011082453 >> 172.21.15.7:0/1011082453 conn(0x7fa6540679b0 msgr2=0x7fa6540b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:24.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.164+0000 7fa65c49d700 1 -- 172.21.15.7:0/1011082453 shutdown_connections 2024-01-27T02:32:24.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.164+0000 7fa65c49d700 1 -- 172.21.15.7:0/1011082453 wait complete. 2024-01-27T02:32:24.167 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-01-27T02:32:24.582 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-27T02:32:06.045693+0000', 'modified': '2024-01-27T02:32:09.806398+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4273}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4273': {'gid': 4273, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6836/2571237935', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 2571237935}, {'type': 'v1', 'addr': '172.21.15.188:6836', 'nonce': 2571237935}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-27T02:32:24.582 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-27T02:32:24.583 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-01-27T02:32:24.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.722+0000 7f2387a11700 1 Processor -- start 2024-01-27T02:32:24.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.723+0000 7f2387a11700 1 -- start start 2024-01-27T02:32:24.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.723+0000 7f2387a11700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23800679b0 0x7f23800cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:24.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.723+0000 7f2387a11700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f23800ce2c0 0x7f23800ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:24.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.723+0000 7f2387a11700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23800c9e50 con 0x7f238015d590 2024-01-27T02:32:24.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.723+0000 7f2387a11700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23800ce780 con 0x7f23800ce2c0 2024-01-27T02:32:24.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.723+0000 7f2387a11700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23800c9520 con 0x7f23800679b0 2024-01-27T02:32:24.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.724+0000 7f2385fae700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f23800ce2c0 0x7f23800ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:24.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.724+0000 7f23857ad700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f238015d590 0x7f23800c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43622/0 (socket says 172.21.15.7:43622) 2024-01-27T02:32:24.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.724+0000 7f23857ad700 1 -- 172.21.15.7:0/3241266041 learned_addr learned my addr 172.21.15.7:0/3241266041 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:24.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.724+0000 7f2385fae700 1 --2- 172.21.15.7:0/3241266041 >> v2:172.21.15.7:3300/0 conn(0x7f23800ce2c0 0x7f23800ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56268/0 (socket says 172.21.15.7:56268) 2024-01-27T02:32:24.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.724+0000 7f2384fac700 1 --2- 172.21.15.7:0/3241266041 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23800679b0 0x7f23800cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:24.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.724+0000 7f2385fae700 1 -- 172.21.15.7:0/3241266041 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23800679b0 msgr2=0x7f23800cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:24.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.724+0000 7f2385fae700 1 --2- 172.21.15.7:0/3241266041 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23800679b0 0x7f23800cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.724+0000 7f2385fae700 1 -- 172.21.15.7:0/3241266041 >> v1:172.21.15.7:6789/0 conn(0x7f238015d590 legacy=0x7f23800c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:24.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.724+0000 7f2385fae700 1 -- 172.21.15.7:0/3241266041 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23800c9800 con 0x7f23800ce2c0 2024-01-27T02:32:24.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.724+0000 7f2373fff700 1 -- 172.21.15.7:0/3241266041 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093525525 0 0) 0x7f23800c9e50 con 0x7f238015d590 2024-01-27T02:32:24.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.724+0000 7f2384fac700 1 --2- 172.21.15.7:0/3241266041 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23800679b0 0x7f23800cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:24.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.724+0000 7f2385fae700 1 --2- 172.21.15.7:0/3241266041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23800ce2c0 0x7f23800ca2e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2378009e60 tx=0x7f2378009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92b06296a9ac5ce server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:24.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.724+0000 7f2373fff700 1 -- 172.21.15.7:0/3241266041 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237800e070 con 0x7f23800ce2c0 2024-01-27T02:32:24.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.724+0000 7f2373fff700 1 -- 172.21.15.7:0/3241266041 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237800bc30 con 0x7f23800ce2c0 2024-01-27T02:32:24.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.724+0000 7f2373fff700 1 -- 172.21.15.7:0/3241266041 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378012430 con 0x7f23800ce2c0 2024-01-27T02:32:24.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.725+0000 7f2387a11700 1 -- 172.21.15.7:0/3241266041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23800ce2c0 msgr2=0x7f23800ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:24.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.725+0000 7f2387a11700 1 --2- 172.21.15.7:0/3241266041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23800ce2c0 0x7f23800ca2e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2378009e60 tx=0x7f2378009780 comp rx=0 tx=0).stop 2024-01-27T02:32:24.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.725+0000 7f2387a11700 1 -- 172.21.15.7:0/3241266041 shutdown_connections 2024-01-27T02:32:24.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.725+0000 7f2387a11700 1 --2- 172.21.15.7:0/3241266041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23800ce2c0 0x7f23800ca2e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.725+0000 7f2387a11700 1 --2- 172.21.15.7:0/3241266041 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23800679b0 0x7f23800cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.725+0000 7f2387a11700 1 -- 172.21.15.7:0/3241266041 >> 172.21.15.7:0/3241266041 conn(0x7f2380098020 msgr2=0x7f23800711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:24.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.725+0000 7f2387a11700 1 -- 172.21.15.7:0/3241266041 shutdown_connections 2024-01-27T02:32:24.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.725+0000 7f2387a11700 1 -- 172.21.15.7:0/3241266041 wait complete. 2024-01-27T02:32:24.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.726+0000 7f2387a11700 1 Processor -- start 2024-01-27T02:32:24.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.726+0000 7f2387a11700 1 -- start start 2024-01-27T02:32:24.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.727+0000 7f2387a11700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23800679b0 0x7f23800b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:24.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.727+0000 7f2387a11700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23800ce2c0 0x7f2380157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:24.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.727+0000 7f2387a11700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f238015d590 0x7f23800c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:24.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.727+0000 7f2387a11700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23801608f0 con 0x7f23800ce2c0 2024-01-27T02:32:24.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.727+0000 7f2387a11700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23800b3f10 con 0x7f23800679b0 2024-01-27T02:32:24.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.727+0000 7f2387a11700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23800b4960 con 0x7f238015d590 2024-01-27T02:32:24.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.727+0000 7f2384fac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23800ce2c0 0x7f2380157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:24.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.727+0000 7f2384fac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23800ce2c0 0x7f2380157b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56280/0 (socket says 172.21.15.7:56280) 2024-01-27T02:32:24.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.727+0000 7f2384fac700 1 -- 172.21.15.7:0/3547810829 learned_addr learned my addr 172.21.15.7:0/3547810829 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:24.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.727+0000 7f2385fae700 1 --2- 172.21.15.7:0/3547810829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f238015d590 0x7f23800c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:24.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.727+0000 7f23857ad700 1 --2- 172.21.15.7:0/3547810829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23800679b0 0x7f23800b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:24.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.728+0000 7f2384fac700 1 -- 172.21.15.7:0/3547810829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f238015d590 msgr2=0x7f23800c6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:24.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.728+0000 7f2384fac700 1 --2- 172.21.15.7:0/3547810829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f238015d590 0x7f23800c6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.728+0000 7f2384fac700 1 -- 172.21.15.7:0/3547810829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23800679b0 msgr2=0x7f23800b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:24.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.728+0000 7f2384fac700 1 --2- 172.21.15.7:0/3547810829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23800679b0 0x7f23800b4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.728+0000 7f2384fac700 1 -- 172.21.15.7:0/3547810829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2378009a20 con 0x7f23800ce2c0 2024-01-27T02:32:24.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.728+0000 7f23857ad700 1 --2- 172.21.15.7:0/3547810829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23800679b0 0x7f23800b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:24.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.728+0000 7f2384fac700 1 --2- 172.21.15.7:0/3547810829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23800ce2c0 0x7f2380157b90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f237400e900 tx=0x7f2374008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:24.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.728+0000 7f23717fa700 1 -- 172.21.15.7:0/3547810829 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2374013410 con 0x7f23800ce2c0 2024-01-27T02:32:24.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.728+0000 7f23717fa700 1 -- 172.21.15.7:0/3547810829 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2374013570 con 0x7f23800ce2c0 2024-01-27T02:32:24.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.728+0000 7f23717fa700 1 -- 172.21.15.7:0/3547810829 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2374013910 con 0x7f23800ce2c0 2024-01-27T02:32:24.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.728+0000 7f2387a11700 1 -- 172.21.15.7:0/3547810829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f238015def0 con 0x7f23800ce2c0 2024-01-27T02:32:24.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.728+0000 7f2387a11700 1 -- 172.21.15.7:0/3547810829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f238016d6b0 con 0x7f23800ce2c0 2024-01-27T02:32:24.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.730+0000 7f23717fa700 1 -- 172.21.15.7:0/3547810829 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2374013cc0 con 0x7f23800ce2c0 2024-01-27T02:32:24.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.730+0000 7f23717fa700 1 --2- 172.21.15.7:0/3547810829 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2364066f30 0x7f23640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:24.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.730+0000 7f23717fa700 1 -- 172.21.15.7:0/3547810829 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2374084290 con 0x7f23800ce2c0 2024-01-27T02:32:24.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.730+0000 7f23857ad700 1 --2- 172.21.15.7:0/3547810829 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2364066f30 0x7f23640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:24.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.731+0000 7f23857ad700 1 --2- 172.21.15.7:0/3547810829 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2364066f30 0x7f23640693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f237c001890 tx=0x7f237c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:24.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.732+0000 7f2390d14b80 1 Processor -- start 2024-01-27T02:32:24.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.732+0000 7f2390d14b80 1 -- start start 2024-01-27T02:32:24.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.733+0000 7f2390d14b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e16ebab010 0x55e16ebab3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:24.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.733+0000 7f2390d14b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebf3360 0x55e16ebd5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:24.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.733+0000 7f2390d14b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebaa5d0 0x55e16ebd5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:24.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.733+0000 7f2390d14b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16eaae9f0 con 0x55e16ebf3360 2024-01-27T02:32:24.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.733+0000 7f2390d14b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16eb00130 con 0x55e16ebab010 2024-01-27T02:32:24.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.733+0000 7f2390d14b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e16eafd930 con 0x55e16ebaa5d0 2024-01-27T02:32:24.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.733+0000 7f2384fac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebf3360 0x55e16ebd5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:24.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.733+0000 7f2384fac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebf3360 0x55e16ebd5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56290/0 (socket says 172.21.15.7:56290) 2024-01-27T02:32:24.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.733+0000 7f2384fac700 1 -- 172.21.15.7:0/877297372 learned_addr learned my addr 172.21.15.7:0/877297372 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:24.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.733+0000 7f2385fae700 1 --2- 172.21.15.7:0/877297372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebaa5d0 0x55e16ebd5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:24.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.733+0000 7f23857ad700 1 --2- 172.21.15.7:0/877297372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e16ebab010 0x55e16ebab3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:24.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.733+0000 7f2384fac700 1 -- 172.21.15.7:0/877297372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebaa5d0 msgr2=0x55e16ebd5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:24.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.733+0000 7f2384fac700 1 --2- 172.21.15.7:0/877297372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebaa5d0 0x55e16ebd5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.733+0000 7f2384fac700 1 -- 172.21.15.7:0/877297372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e16ebab010 msgr2=0x55e16ebab3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:24.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.733+0000 7f2384fac700 1 --2- 172.21.15.7:0/877297372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e16ebab010 0x55e16ebab3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.733+0000 7f2384fac700 1 -- 172.21.15.7:0/877297372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e16e938b10 con 0x55e16ebf3360 2024-01-27T02:32:24.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.733+0000 7f23857ad700 1 --2- 172.21.15.7:0/877297372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e16ebab010 0x55e16ebab3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:24.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.734+0000 7f2384fac700 1 --2- 172.21.15.7:0/877297372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebf3360 0x55e16ebd5840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f23740827e0 tx=0x7f2374082710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37a6032009d3056 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:24.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.734+0000 7f236a7fc700 1 -- 172.21.15.7:0/877297372 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23740840c0 con 0x55e16ebf3360 2024-01-27T02:32:24.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.734+0000 7f236a7fc700 1 -- 172.21.15.7:0/877297372 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23740551c0 con 0x55e16ebf3360 2024-01-27T02:32:24.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.734+0000 7f236a7fc700 1 -- 172.21.15.7:0/877297372 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2374058410 con 0x55e16ebf3360 2024-01-27T02:32:24.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.734+0000 7f2390d14b80 1 -- 172.21.15.7:0/877297372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebf3360 msgr2=0x55e16ebd5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:24.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.734+0000 7f2390d14b80 1 --2- 172.21.15.7:0/877297372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebf3360 0x55e16ebd5840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f23740827e0 tx=0x7f2374082710 comp rx=0 tx=0).stop 2024-01-27T02:32:24.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.734+0000 7f2390d14b80 1 -- 172.21.15.7:0/877297372 shutdown_connections 2024-01-27T02:32:24.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.734+0000 7f2390d14b80 1 --2- 172.21.15.7:0/877297372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebaa5d0 0x55e16ebd5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.734+0000 7f2390d14b80 1 --2- 172.21.15.7:0/877297372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e16ebab010 0x55e16ebab3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.734+0000 7f2390d14b80 1 --2- 172.21.15.7:0/877297372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebf3360 0x55e16ebd5840 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.734+0000 7f2390d14b80 1 -- 172.21.15.7:0/877297372 >> 172.21.15.7:0/877297372 conn(0x55e16ebaaa70 msgr2=0x55e16ebf3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:24.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.735+0000 7f2390d14b80 1 -- 172.21.15.7:0/877297372 shutdown_connections 2024-01-27T02:32:24.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.735+0000 7f2390d14b80 1 -- 172.21.15.7:0/877297372 wait complete. 2024-01-27T02:32:24.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.735+0000 7f2390d14b80 1 -- start start 2024-01-27T02:32:24.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.735+0000 7f2390d14b80 1 Processor -- start 2024-01-27T02:32:24.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.736+0000 7f2390d14b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e16ebaa5d0 0x55e16ebf2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:24.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.736+0000 7f2390d14b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebab010 0x55e16ebd7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.736+0000 7f2390d14b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebf3360 0x55e16ebd7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.736+0000 7f2390d14b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16eaf9720 con 0x55e16ebab010 2024-01-27T02:32:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.736+0000 7f2390d14b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16eaa51b0 con 0x55e16ebaa5d0 2024-01-27T02:32:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.736+0000 7f2390d14b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e16eaa4e50 con 0x55e16ebf3360 2024-01-27T02:32:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.736+0000 7f2390d14b80 10 client.0 _ll_register_callbacks cb 0x55e16e9077b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-27T02:32:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.736+0000 7f2384fac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebab010 0x55e16ebd7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.736+0000 7f23857ad700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e16ebaa5d0 0x55e16ebf2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.736+0000 7f2384fac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebab010 0x55e16ebd7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56302/0 (socket says 172.21.15.7:56302) 2024-01-27T02:32:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.736+0000 7f2384fac700 1 -- 172.21.15.7:0/1402069127 learned_addr learned my addr 172.21.15.7:0/1402069127 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.736+0000 7f2385fae700 1 --2- 172.21.15.7:0/1402069127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebf3360 0x55e16ebd7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.737+0000 7f2384fac700 1 -- 172.21.15.7:0/1402069127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebf3360 msgr2=0x55e16ebd7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.737+0000 7f2384fac700 1 --2- 172.21.15.7:0/1402069127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebf3360 0x55e16ebd7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.737+0000 7f2384fac700 1 -- 172.21.15.7:0/1402069127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e16ebaa5d0 msgr2=0x55e16ebf2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:24.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.737+0000 7f2384fac700 1 --2- 172.21.15.7:0/1402069127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e16ebaa5d0 0x55e16ebf2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.737+0000 7f2384fac700 1 -- 172.21.15.7:0/1402069127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f238016d6b0 con 0x55e16ebab010 2024-01-27T02:32:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.737+0000 7f2385fae700 1 --2- 172.21.15.7:0/1402069127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebf3360 0x55e16ebd7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.737+0000 7f2384fac700 1 --2- 172.21.15.7:0/1402069127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebab010 0x55e16ebd7510 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2374055820 tx=0x7f237406d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6104fdbf9977189 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.737+0000 7f235f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-27T02:32:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.737+0000 7f235f7fe700 1 -- 172.21.15.7:0/1402069127 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23740840c0 con 0x55e16ebab010 2024-01-27T02:32:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.737+0000 7f235f7fe700 1 -- 172.21.15.7:0/1402069127 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f237405c160 con 0x55e16ebab010 2024-01-27T02:32:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.737+0000 7f2390d14b80 1 -- 172.21.15.7:0/1402069127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e16eb9e190 con 0x55e16ebab010 2024-01-27T02:32:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.737+0000 7f235f7fe700 1 -- 172.21.15.7:0/1402069127 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2374058410 con 0x55e16ebab010 2024-01-27T02:32:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.737+0000 7f2390d14b80 10 client.4590 fetch_fsmap learned FSMap version 8 2024-01-27T02:32:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.737+0000 7f2390d14b80 1 -- 172.21.15.7:0/1402069127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55e16eac7290 con 0x55e16ebab010 2024-01-27T02:32:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.737+0000 7f235f7fe700 1 -- 172.21.15.7:0/1402069127 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2374075070 con 0x55e16ebab010 2024-01-27T02:32:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.737+0000 7f235f7fe700 1 -- 172.21.15.7:0/1402069127 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f237405bcd0 con 0x55e16ebab010 2024-01-27T02:32:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.737+0000 7f2390d14b80 10 client.4590 fetch_fsmap finished waiting for FSMap version 8 2024-01-27T02:32:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.737+0000 7f2390d14b80 10 client.4590 resolve_mds: resolved name 'd' to daemon mds.d 2024-01-27T02:32:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.737+0000 7f2390d14b80 20 client.4590 populate_metadata read hostname 'smithi007' 2024-01-27T02:32:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.737+0000 7f2390d14b80 1 --2- 172.21.15.7:0/1402069127 >> [v2:172.21.15.188:6834/2571237935,v1:172.21.15.188:6836/2571237935] conn(0x55e16ebeb100 0x55e16ebeb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.737+0000 7f2390d14b80 4 client.4590 mds_command: new command op to 4273 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-27T02:32:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.737+0000 7f2390d14b80 1 -- 172.21.15.7:0/1402069127 --> [v2:172.21.15.188:6834/2571237935,v1:172.21.15.188:6836/2571237935] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e16ea9bee0 con 0x55e16ebeb100 2024-01-27T02:32:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.738+0000 7f23857ad700 1 --2- 172.21.15.7:0/1402069127 >> [v2:172.21.15.188:6834/2571237935,v1:172.21.15.188:6836/2571237935] conn(0x55e16ebeb100 0x55e16ebeb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.738+0000 7f23857ad700 1 --2- 172.21.15.7:0/1402069127 >> [v2:172.21.15.188:6834/2571237935,v1:172.21.15.188:6836/2571237935] conn(0x55e16ebeb100 0x55e16ebeb4d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39f1c69478822329 server_cookie=541ead1b83747955 in_seq=0 out_seq=0 2024-01-27T02:32:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.738+0000 7f235f7fe700 10 client.4590 ms_handle_connect on v2:172.21.15.188:6834/2571237935 2024-01-27T02:32:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.740+0000 7f235f7fe700 1 -- 172.21.15.7:0/1402069127 <== mds.0 v2:172.21.15.188:6834/2571237935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17146 (crc 0 0 0) 0x55e16ea9bee0 con 0x55e16ebeb100 2024-01-27T02:32:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.740+0000 7f235f7fe700 10 client.4590 handle_command_reply: tid=0 2024-01-27T02:32:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.740+0000 7f2390d14b80 1 client.4590 shutdown 2024-01-27T02:32:24.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.740+0000 7f2390d14b80 1 -- 172.21.15.7:0/1402069127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebab010 msgr2=0x55e16ebd7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:24.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.740+0000 7f2390d14b80 1 --2- 172.21.15.7:0/1402069127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebab010 0x55e16ebd7510 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2374055820 tx=0x7f237406d000 comp rx=0 tx=0).stop 2024-01-27T02:32:24.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.740+0000 7f2390d14b80 1 -- 172.21.15.7:0/1402069127 shutdown_connections 2024-01-27T02:32:24.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.740+0000 7f2390d14b80 1 --2- 172.21.15.7:0/1402069127 >> [v2:172.21.15.188:6834/2571237935,v1:172.21.15.188:6836/2571237935] conn(0x55e16ebeb100 0x55e16ebeb4d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.740+0000 7f2390d14b80 1 --2- 172.21.15.7:0/1402069127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebf3360 0x55e16ebd7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.740+0000 7f2390d14b80 1 --2- 172.21.15.7:0/1402069127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e16ebaa5d0 0x55e16ebf2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.740+0000 7f235f7fe700 0 client.4590 ms_handle_reset on v2:172.21.15.188:6834/2571237935 2024-01-27T02:32:24.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.740+0000 7f2390d14b80 1 --2- 172.21.15.7:0/1402069127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebab010 0x55e16ebd7510 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.740+0000 7f2390d14b80 1 -- 172.21.15.7:0/1402069127 >> 172.21.15.7:0/1402069127 conn(0x55e16ebaaa70 msgr2=0x55e16eba67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:24.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.740+0000 7f2390d14b80 1 -- 172.21.15.7:0/1402069127 shutdown_connections 2024-01-27T02:32:24.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.740+0000 7f2390d14b80 1 -- 172.21.15.7:0/1402069127 wait complete. 2024-01-27T02:32:24.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.740+0000 7f2390d14b80 20 client.4590 trim_cache size 0 max 16384 2024-01-27T02:32:24.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.757+0000 7f2390d14b80 1 Processor -- start 2024-01-27T02:32:24.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.757+0000 7f2390d14b80 1 -- start start 2024-01-27T02:32:24.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.758+0000 7f2390d14b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebaa5d0 0x55e16ebf3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:24.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.758+0000 7f2390d14b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebab010 0x55e16ebd73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:24.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.758+0000 7f2390d14b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e16ebf3360 0x55e16ebd7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:24.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.758+0000 7f2390d14b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16e9f1e40 con 0x55e16ebab010 2024-01-27T02:32:24.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.758+0000 7f2390d14b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16ebed430 con 0x55e16ebf3360 2024-01-27T02:32:24.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.758+0000 7f2390d14b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e16ebd8b70 con 0x55e16ebaa5d0 2024-01-27T02:32:24.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.758+0000 7f2384fac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebab010 0x55e16ebd73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:24.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.758+0000 7f2384fac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebab010 0x55e16ebd73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56318/0 (socket says 172.21.15.7:56318) 2024-01-27T02:32:24.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.758+0000 7f2384fac700 1 -- 172.21.15.7:0/1387480736 learned_addr learned my addr 172.21.15.7:0/1387480736 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:24.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.758+0000 7f2385fae700 1 --2- 172.21.15.7:0/1387480736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e16ebf3360 0x55e16ebd7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:24.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.758+0000 7f23857ad700 1 --2- 172.21.15.7:0/1387480736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebaa5d0 0x55e16ebf3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:24.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.758+0000 7f2384fac700 1 -- 172.21.15.7:0/1387480736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebaa5d0 msgr2=0x55e16ebf3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:24.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.758+0000 7f2384fac700 1 --2- 172.21.15.7:0/1387480736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebaa5d0 0x55e16ebf3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.758+0000 7f2384fac700 1 -- 172.21.15.7:0/1387480736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e16ebf3360 msgr2=0x55e16ebd7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:24.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.758+0000 7f2384fac700 1 --2- 172.21.15.7:0/1387480736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e16ebf3360 0x55e16ebd7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.758+0000 7f2384fac700 1 -- 172.21.15.7:0/1387480736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e16e990c80 con 0x55e16ebab010 2024-01-27T02:32:24.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.758+0000 7f2385fae700 1 --2- 172.21.15.7:0/1387480736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e16ebf3360 0x55e16ebd7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:32:24.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.758+0000 7f23857ad700 1 --2- 172.21.15.7:0/1387480736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebaa5d0 0x55e16ebf3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:32:24.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.758+0000 7f2384fac700 1 --2- 172.21.15.7:0/1387480736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebab010 0x55e16ebd73d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2374054ab0 tx=0x7f237407f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f6dd4fcc9c51b8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:24.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.760+0000 7f236affd700 1 -- 172.21.15.7:0/1387480736 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23740840c0 con 0x55e16ebab010 2024-01-27T02:32:24.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.760+0000 7f236affd700 1 -- 172.21.15.7:0/1387480736 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2374075070 con 0x55e16ebab010 2024-01-27T02:32:24.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.760+0000 7f236affd700 1 -- 172.21.15.7:0/1387480736 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237405c160 con 0x55e16ebab010 2024-01-27T02:32:24.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.760+0000 7f2390d14b80 1 -- 172.21.15.7:0/1387480736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebab010 msgr2=0x55e16ebd73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:24.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.760+0000 7f2390d14b80 1 --2- 172.21.15.7:0/1387480736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebab010 0x55e16ebd73d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2374054ab0 tx=0x7f237407f040 comp rx=0 tx=0).stop 2024-01-27T02:32:24.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.760+0000 7f2390d14b80 1 -- 172.21.15.7:0/1387480736 shutdown_connections 2024-01-27T02:32:24.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.760+0000 7f2390d14b80 1 --2- 172.21.15.7:0/1387480736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e16ebf3360 0x55e16ebd7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.760+0000 7f2390d14b80 1 --2- 172.21.15.7:0/1387480736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebaa5d0 0x55e16ebf3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.760+0000 7f2390d14b80 1 --2- 172.21.15.7:0/1387480736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebab010 0x55e16ebd73d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.760+0000 7f2390d14b80 1 -- 172.21.15.7:0/1387480736 >> 172.21.15.7:0/1387480736 conn(0x55e16ebaaa70 msgr2=0x55e16eba4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:24.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.760+0000 7f2390d14b80 1 -- 172.21.15.7:0/1387480736 shutdown_connections 2024-01-27T02:32:24.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.760+0000 7f2390d14b80 1 -- 172.21.15.7:0/1387480736 wait complete. 2024-01-27T02:32:24.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.760+0000 7f2390d14b80 1 -- start start 2024-01-27T02:32:24.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.760+0000 7f2390d14b80 1 Processor -- start 2024-01-27T02:32:24.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f2390d14b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e16ebaa5d0 0x55e16ec53ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:24.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f2390d14b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebab010 0x55e16ebeb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:24.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f2390d14b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebf3360 0x55e16ebebb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:24.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f2390d14b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16ec55b20 con 0x55e16ebf3360 2024-01-27T02:32:24.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f2390d14b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16ec55f80 con 0x55e16ebaa5d0 2024-01-27T02:32:24.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f2390d14b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e16ebf3f30 con 0x55e16ebab010 2024-01-27T02:32:24.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f2390d14b80 10 client.0 _ll_register_callbacks cb 0x55e16e9077b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-27T02:32:24.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f2385fae700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebf3360 0x55e16ebebb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:24.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f2385fae700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebf3360 0x55e16ebebb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56322/0 (socket says 172.21.15.7:56322) 2024-01-27T02:32:24.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f2385fae700 1 -- 172.21.15.7:0/543363390 learned_addr learned my addr 172.21.15.7:0/543363390 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:24.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f2384fac700 1 --2- 172.21.15.7:0/543363390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebab010 0x55e16ebeb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:24.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f23857ad700 1 --2- 172.21.15.7:0/543363390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e16ebaa5d0 0x55e16ec53ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:24.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f2385fae700 1 -- 172.21.15.7:0/543363390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebab010 msgr2=0x55e16ebeb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:24.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f2385fae700 1 --2- 172.21.15.7:0/543363390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebab010 0x55e16ebeb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f2385fae700 1 -- 172.21.15.7:0/543363390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e16ebaa5d0 msgr2=0x55e16ec53ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:24.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f2385fae700 1 --2- 172.21.15.7:0/543363390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e16ebaa5d0 0x55e16ec53ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f2385fae700 1 -- 172.21.15.7:0/543363390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e16eac7290 con 0x55e16ebf3360 2024-01-27T02:32:24.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f2384fac700 1 --2- 172.21.15.7:0/543363390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebab010 0x55e16ebeb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:24.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f2385fae700 1 --2- 172.21.15.7:0/543363390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebf3360 0x55e16ebebb10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f238006ea80 tx=0x7f2378019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407cdb578856768a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:24.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f235f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-27T02:32:24.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f235f7fe700 1 -- 172.21.15.7:0/543363390 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237801e6b0 con 0x55e16ebf3360 2024-01-27T02:32:24.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f235f7fe700 1 -- 172.21.15.7:0/543363390 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f237801e810 con 0x55e16ebf3360 2024-01-27T02:32:24.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.762+0000 7f2390d14b80 1 -- 172.21.15.7:0/543363390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23740551c0 con 0x55e16ebf3360 2024-01-27T02:32:24.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.763+0000 7f235f7fe700 1 -- 172.21.15.7:0/543363390 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f237801ebe0 con 0x55e16ebf3360 2024-01-27T02:32:24.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.763+0000 7f2390d14b80 10 client.4596 fetch_fsmap learned FSMap version 8 2024-01-27T02:32:24.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.763+0000 7f2390d14b80 1 -- 172.21.15.7:0/543363390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55e16e936fe0 con 0x55e16ebf3360 2024-01-27T02:32:24.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.763+0000 7f235f7fe700 1 -- 172.21.15.7:0/543363390 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23740551c0 con 0x55e16ebf3360 2024-01-27T02:32:24.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.763+0000 7f235f7fe700 1 -- 172.21.15.7:0/543363390 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f2378029430 con 0x55e16ebf3360 2024-01-27T02:32:24.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.763+0000 7f2390d14b80 10 client.4596 fetch_fsmap finished waiting for FSMap version 8 2024-01-27T02:32:24.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.763+0000 7f2390d14b80 10 client.4596 resolve_mds: resolved name 'd' to daemon mds.d 2024-01-27T02:32:24.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.763+0000 7f2390d14b80 20 client.4596 populate_metadata read hostname 'smithi007' 2024-01-27T02:32:24.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.763+0000 7f2390d14b80 1 --2- 172.21.15.7:0/543363390 >> [v2:172.21.15.188:6834/2571237935,v1:172.21.15.188:6836/2571237935] conn(0x55e16ec0be20 0x55e16ec0c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:24.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.763+0000 7f2390d14b80 4 client.4596 mds_command: new command op to 4273 tid=0[{"prefix": "status"}] 2024-01-27T02:32:24.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.763+0000 7f2390d14b80 1 -- 172.21.15.7:0/543363390 --> [v2:172.21.15.188:6834/2571237935,v1:172.21.15.188:6836/2571237935] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e16ea9bc00 con 0x55e16ec0be20 2024-01-27T02:32:24.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.763+0000 7f23857ad700 1 --2- 172.21.15.7:0/543363390 >> [v2:172.21.15.188:6834/2571237935,v1:172.21.15.188:6836/2571237935] conn(0x55e16ec0be20 0x55e16ec0c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:24.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.764+0000 7f235f7fe700 10 client.4596 ms_handle_connect on v2:172.21.15.188:6834/2571237935 2024-01-27T02:32:24.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.764+0000 7f23857ad700 1 --2- 172.21.15.7:0/543363390 >> [v2:172.21.15.188:6834/2571237935,v1:172.21.15.188:6836/2571237935] conn(0x55e16ec0be20 0x55e16ec0c1f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e0173700a7e60dd server_cookie=d471050d3e44c8ce in_seq=0 out_seq=0 2024-01-27T02:32:24.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.764+0000 7f235f7fe700 1 -- 172.21.15.7:0/543363390 <== mds.0 v2:172.21.15.188:6834/2571237935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55e16ea9bc00 con 0x55e16ec0be20 2024-01-27T02:32:24.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.764+0000 7f235f7fe700 10 client.4596 handle_command_reply: tid=0 2024-01-27T02:32:24.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.764+0000 7f2390d14b80 1 client.4596 shutdown 2024-01-27T02:32:24.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.764+0000 7f2390d14b80 1 -- 172.21.15.7:0/543363390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebf3360 msgr2=0x55e16ebebb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:24.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.764+0000 7f2390d14b80 1 --2- 172.21.15.7:0/543363390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebf3360 0x55e16ebebb10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f238006ea80 tx=0x7f2378019000 comp rx=0 tx=0).stop 2024-01-27T02:32:24.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.764+0000 7f2390d14b80 1 -- 172.21.15.7:0/543363390 shutdown_connections 2024-01-27T02:32:24.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.764+0000 7f2390d14b80 1 --2- 172.21.15.7:0/543363390 >> [v2:172.21.15.188:6834/2571237935,v1:172.21.15.188:6836/2571237935] conn(0x55e16ec0be20 0x55e16ec0c1f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.764+0000 7f2390d14b80 1 --2- 172.21.15.7:0/543363390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e16ebf3360 0x55e16ebebb10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.764+0000 7f2390d14b80 1 --2- 172.21.15.7:0/543363390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e16ebaa5d0 0x55e16ec53ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.764+0000 7f235f7fe700 0 client.4596 ms_handle_reset on v2:172.21.15.188:6834/2571237935 2024-01-27T02:32:24.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.764+0000 7f2390d14b80 1 --2- 172.21.15.7:0/543363390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e16ebab010 0x55e16ebeb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.764+0000 7f2390d14b80 1 -- 172.21.15.7:0/543363390 >> 172.21.15.7:0/543363390 conn(0x55e16ebaaa70 msgr2=0x55e16ebd72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:24.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.764+0000 7f2390d14b80 1 -- 172.21.15.7:0/543363390 shutdown_connections 2024-01-27T02:32:24.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.764+0000 7f2390d14b80 1 -- 172.21.15.7:0/543363390 wait complete. 2024-01-27T02:32:24.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.764+0000 7f2390d14b80 20 client.4596 trim_cache size 0 max 16384 2024-01-27T02:32:24.771 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:32:24.771 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:32:24.771 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-27T02:32:24.771 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4273, 2024-01-27T02:32:24.772 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-27T02:32:24.772 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-27T02:32:24.772 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-27T02:32:24.772 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 17.970456856999999, 2024-01-27T02:32:24.772 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 8, 2024-01-27T02:32:24.772 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 21, 2024-01-27T02:32:24.772 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 21, 2024-01-27T02:32:24.772 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 23.634787758000002 2024-01-27T02:32:24.772 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:32:24.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.765+0000 7f2387a11700 1 -- 172.21.15.7:0/3547810829 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2364066f30 msgr2=0x7f23640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:24.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.765+0000 7f2387a11700 1 --2- 172.21.15.7:0/3547810829 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2364066f30 0x7f23640693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f237c001890 tx=0x7f237c009040 comp rx=0 tx=0).stop 2024-01-27T02:32:24.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.765+0000 7f2387a11700 1 -- 172.21.15.7:0/3547810829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23800ce2c0 msgr2=0x7f2380157b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:24.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.765+0000 7f2387a11700 1 --2- 172.21.15.7:0/3547810829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23800ce2c0 0x7f2380157b90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f237400e900 tx=0x7f2374008040 comp rx=0 tx=0).stop 2024-01-27T02:32:24.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.765+0000 7f2387a11700 1 -- 172.21.15.7:0/3547810829 shutdown_connections 2024-01-27T02:32:24.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.765+0000 7f2387a11700 1 --2- 172.21.15.7:0/3547810829 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2364066f30 0x7f23640693e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.765+0000 7f2387a11700 1 --2- 172.21.15.7:0/3547810829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f238015d590 0x7f23800c6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.765+0000 7f2387a11700 1 --2- 172.21.15.7:0/3547810829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23800679b0 0x7f23800b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.765+0000 7f2387a11700 1 --2- 172.21.15.7:0/3547810829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23800ce2c0 0x7f2380157b90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:24.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.765+0000 7f2387a11700 1 -- 172.21.15.7:0/3547810829 >> 172.21.15.7:0/3547810829 conn(0x7f2380098020 msgr2=0x7f238006bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:24.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.765+0000 7f2387a11700 1 -- 172.21.15.7:0/3547810829 shutdown_connections 2024-01-27T02:32:24.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:24.765+0000 7f2387a11700 1 -- 172.21.15.7:0/3547810829 wait complete. 2024-01-27T02:32:24.780 INFO:teuthology.run_tasks:Running task kclient... 2024-01-27T02:32:24.800 INFO:tasks.kclient:Mounting kernel clients... 2024-01-27T02:32:24.801 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-01-27T02:32:24.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:24.801 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-27T02:32:24.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:24.821 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-01-27T02:32:24.896 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2024-01-27T02:32:24.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-27T02:32:24.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:24.900 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-27T02:32:24.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:24.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-01-27T02:32:25.040 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2024-01-27T02:32:25.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-27T02:32:25.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:25.041 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-01-27T02:32:25.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:25.063 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2024-01-27T02:32:25.139 INFO:teuthology.orchestra.run.smithi188.stderr:Cannot find device "ceph-brx" 2024-01-27T02:32:25.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-27T02:32:25.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:25.140 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-01-27T02:32:25.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:25.201 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2024-01-27T02:32:25.271 INFO:teuthology.orchestra.run.smithi188.stderr:Cannot find device "ceph-brx" 2024-01-27T02:32:25.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-27T02:32:25.273 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-01-27T02:32:25.273 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:32:25.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.413+0000 7febe977f700 1 Processor -- start 2024-01-27T02:32:25.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.413+0000 7febe977f700 1 -- start start 2024-01-27T02:32:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.414+0000 7febe977f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7febe4156c00 0x7febe4156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.414+0000 7febe977f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7febe406e040 0x7febe40c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.414+0000 7febe977f700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe406dbb0 con 0x7febe406e4b0 2024-01-27T02:32:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.414+0000 7febe977f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe40c8bc0 con 0x7febe406e040 2024-01-27T02:32:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.414+0000 7febe977f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe40c88e0 con 0x7febe4156c00 2024-01-27T02:32:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.414+0000 7febe27fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7febe406e040 0x7febe40c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.414+0000 7febe27fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7febe406e040 0x7febe40c92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56324/0 (socket says 172.21.15.7:56324) 2024-01-27T02:32:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.414+0000 7febe27fc700 1 -- 172.21.15.7:0/1856224631 learned_addr learned my addr 172.21.15.7:0/1856224631 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.415+0000 7febe2ffd700 1 --2- 172.21.15.7:0/1856224631 >> v2:172.21.15.188:3300/0 conn(0x7febe4156c00 0x7febe4156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.415+0000 7febe27fc700 1 -- 172.21.15.7:0/1856224631 >> v2:172.21.15.188:3300/0 conn(0x7febe4156c00 msgr2=0x7febe4156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.415+0000 7febe27fc700 1 --2- 172.21.15.7:0/1856224631 >> v2:172.21.15.188:3300/0 conn(0x7febe4156c00 0x7febe4156fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.415+0000 7febe27fc700 1 -- 172.21.15.7:0/1856224631 >> v1:172.21.15.188:6789/0 conn(0x7febe406e4b0 legacy=0x7febe40c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.415+0000 7febe27fc700 1 -- 172.21.15.7:0/1856224631 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe406d560 con 0x7febe406e040 2024-01-27T02:32:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.415+0000 7febe27fc700 1 --2- 172.21.15.7:0/1856224631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe406e040 0x7febe40c92f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7febe40c3a40 tx=0x7febd8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c22576cf4abd415 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.415+0000 7febe1ffb700 1 -- 172.21.15.7:0/1856224631 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd800b860 con 0x7febe406e040 2024-01-27T02:32:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.415+0000 7febe1ffb700 1 -- 172.21.15.7:0/1856224631 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd800b9c0 con 0x7febe406e040 2024-01-27T02:32:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.415+0000 7febe1ffb700 1 -- 172.21.15.7:0/1856224631 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd800bd60 con 0x7febe406e040 2024-01-27T02:32:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.416+0000 7febe977f700 1 -- 172.21.15.7:0/1856224631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe406e040 msgr2=0x7febe40c92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.416+0000 7febe977f700 1 --2- 172.21.15.7:0/1856224631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe406e040 0x7febe40c92f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7febe40c3a40 tx=0x7febd8009250 comp rx=0 tx=0).stop 2024-01-27T02:32:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.416+0000 7febe977f700 1 -- 172.21.15.7:0/1856224631 shutdown_connections 2024-01-27T02:32:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.416+0000 7febe977f700 1 --2- 172.21.15.7:0/1856224631 >> v2:172.21.15.188:3300/0 conn(0x7febe4156c00 0x7febe4156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.416+0000 7febe977f700 1 --2- 172.21.15.7:0/1856224631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe406e040 0x7febe40c92f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.416+0000 7febe977f700 1 -- 172.21.15.7:0/1856224631 >> 172.21.15.7:0/1856224631 conn(0x7febe4063580 msgr2=0x7febe4063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.416+0000 7febe977f700 1 -- 172.21.15.7:0/1856224631 shutdown_connections 2024-01-27T02:32:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.416+0000 7febe977f700 1 -- 172.21.15.7:0/1856224631 wait complete. 2024-01-27T02:32:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.417+0000 7febe977f700 1 Processor -- start 2024-01-27T02:32:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.417+0000 7febe977f700 1 -- start start 2024-01-27T02:32:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.418+0000 7febe977f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febe406e040 0x7febe40b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.418+0000 7febe977f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febe406e4b0 0x7febe414db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.418+0000 7febe977f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe4156c00 0x7febe414e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.418+0000 7febe977f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe40c2d60 con 0x7febe4156c00 2024-01-27T02:32:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.418+0000 7febe977f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe40c3040 con 0x7febe406e4b0 2024-01-27T02:32:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.418+0000 7febe977f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe40c3320 con 0x7febe406e040 2024-01-27T02:32:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.418+0000 7febe37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe4156c00 0x7febe414e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.418+0000 7febe37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe4156c00 0x7febe414e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56336/0 (socket says 172.21.15.7:56336) 2024-01-27T02:32:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.418+0000 7febe37fe700 1 -- 172.21.15.7:0/1754619648 learned_addr learned my addr 172.21.15.7:0/1754619648 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.418+0000 7febe2ffd700 1 --2- 172.21.15.7:0/1754619648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febe406e040 0x7febe40b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.418+0000 7febe27fc700 1 --2- 172.21.15.7:0/1754619648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febe406e4b0 0x7febe414db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.418+0000 7febe37fe700 1 -- 172.21.15.7:0/1754619648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febe406e040 msgr2=0x7febe40b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.418+0000 7febe37fe700 1 --2- 172.21.15.7:0/1754619648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febe406e040 0x7febe40b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.418+0000 7febe37fe700 1 -- 172.21.15.7:0/1754619648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febe406e4b0 msgr2=0x7febe414db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.418+0000 7febe37fe700 1 --2- 172.21.15.7:0/1754619648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febe406e4b0 0x7febe414db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.418+0000 7febe37fe700 1 -- 172.21.15.7:0/1754619648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd80094f0 con 0x7febe4156c00 2024-01-27T02:32:25.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.419+0000 7febe37fe700 1 --2- 172.21.15.7:0/1754619648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe4156c00 0x7febe414e090 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7febdc00c8e0 tx=0x7febdc001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:25.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.419+0000 7febcaffd700 1 -- 172.21.15.7:0/1754619648 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc00ce80 con 0x7febe4156c00 2024-01-27T02:32:25.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.419+0000 7febcaffd700 1 -- 172.21.15.7:0/1754619648 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febdc00a7e0 con 0x7febe4156c00 2024-01-27T02:32:25.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.419+0000 7febcaffd700 1 -- 172.21.15.7:0/1754619648 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc00ab80 con 0x7febe4156c00 2024-01-27T02:32:25.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.419+0000 7febe977f700 1 -- 172.21.15.7:0/1754619648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe414d070 con 0x7febe4156c00 2024-01-27T02:32:25.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.419+0000 7febe977f700 1 -- 172.21.15.7:0/1754619648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe414d630 con 0x7febe4156c00 2024-01-27T02:32:25.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.420+0000 7febe977f700 1 -- 172.21.15.7:0/1754619648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe414d820 con 0x7febe4156c00 2024-01-27T02:32:25.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.420+0000 7febcaffd700 1 -- 172.21.15.7:0/1754619648 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7febdc012420 con 0x7febe4156c00 2024-01-27T02:32:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.421+0000 7febcaffd700 1 --2- 172.21.15.7:0/1754619648 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7febd0066f30 0x7febd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.421+0000 7febcaffd700 1 -- 172.21.15.7:0/1754619648 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7febdc084820 con 0x7febe4156c00 2024-01-27T02:32:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.421+0000 7febe2ffd700 1 --2- 172.21.15.7:0/1754619648 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7febd0066f30 0x7febd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.422+0000 7febe2ffd700 1 --2- 172.21.15.7:0/1754619648 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7febd0066f30 0x7febd00693e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7febd4006fd0 tx=0x7febd4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.426+0000 7febcaffd700 1 -- 172.21.15.7:0/1754619648 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7febdc051a90 con 0x7febe4156c00 2024-01-27T02:32:25.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.675+0000 7febe977f700 1 -- 172.21.15.7:0/1754619648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7febe404c710 con 0x7febe4156c00 2024-01-27T02:32:25.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.675+0000 7febcaffd700 1 -- 172.21.15.7:0/1754619648 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7febdc0550b0 con 0x7febe4156c00 2024-01-27T02:32:25.676 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:32:25.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.678+0000 7febe977f700 1 -- 172.21.15.7:0/1754619648 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7febd0066f30 msgr2=0x7febd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:25.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.678+0000 7febe977f700 1 --2- 172.21.15.7:0/1754619648 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7febd0066f30 0x7febd00693e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7febd4006fd0 tx=0x7febd4008040 comp rx=0 tx=0).stop 2024-01-27T02:32:25.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.678+0000 7febe977f700 1 -- 172.21.15.7:0/1754619648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe4156c00 msgr2=0x7febe414e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:25.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.678+0000 7febe977f700 1 --2- 172.21.15.7:0/1754619648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe4156c00 0x7febe414e090 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7febdc00c8e0 tx=0x7febdc001710 comp rx=0 tx=0).stop 2024-01-27T02:32:25.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.678+0000 7febe977f700 1 -- 172.21.15.7:0/1754619648 shutdown_connections 2024-01-27T02:32:25.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.678+0000 7febe977f700 1 --2- 172.21.15.7:0/1754619648 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7febd0066f30 0x7febd00693e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:25.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.678+0000 7febe977f700 1 --2- 172.21.15.7:0/1754619648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe4156c00 0x7febe414e090 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:25.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.678+0000 7febe977f700 1 --2- 172.21.15.7:0/1754619648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febe406e040 0x7febe40b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:25.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.678+0000 7febe977f700 1 --2- 172.21.15.7:0/1754619648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febe406e4b0 0x7febe414db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:25.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.678+0000 7febe977f700 1 -- 172.21.15.7:0/1754619648 >> 172.21.15.7:0/1754619648 conn(0x7febe4063580 msgr2=0x7febe40cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:25.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.678+0000 7febe977f700 1 -- 172.21.15.7:0/1754619648 shutdown_connections 2024-01-27T02:32:25.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:25.678+0000 7febe977f700 1 -- 172.21.15.7:0/1754619648 wait complete. 2024-01-27T02:32:25.692 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-27T02:32:25.692 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:32:25.692 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-27T02:32:25.692 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-27T02:32:25.692 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-27T02:32:25.692 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-27T02:32:25.693 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-27T02:32:25.693 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:32:25.693 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:32:25.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:25.693 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-27T02:32:25.712 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:32:25.712 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:32:25.712 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:32:25.713 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:25.713 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-27T02:32:25.713 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:25.713 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:32:25.713 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-27T02:32:25.713 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-27T02:32:25.713 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:32:25.713 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-27T02:32:25.713 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-27T02:32:25.713 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:32:25.713 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-27T02:32:25.713 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-27T02:32:25.713 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-27T02:32:25.713 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42365sec preferred_lft 42365sec 2024-01-27T02:32:25.714 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-27T02:32:25.714 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:25.714 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:32:25.714 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-27T02:32:25.714 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-27T02:32:25.714 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-27T02:32:25.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:25.714 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-27T02:32:25.715 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-27T02:32:25.715 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2024-01-27T02:32:25.715 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2024-01-27T02:32:25.715 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2024-01-27T02:32:25.715 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-27T02:32:25.715 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-27T02:32:25.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:25.894 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:32:25.926 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-27T02:32:25.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:25.928 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-27T02:32:26.002 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-27T02:32:26.003 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:32:26.003 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-27T02:32:26.003 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-27T02:32:26.003 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:32:26.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:26.003 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-27T02:32:26.004 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-27T02:32:26.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-27T02:32:26.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-27T02:32:26.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-27T02:32:26.004 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-27T02:32:26.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:26.239 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-27T02:32:26.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:26.260 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-01-27T02:32:26.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:26.322 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-27T02:32:26.322 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-27T02:32:26.322 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-27T02:32:26.322 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-01-27T02:32:26.323 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-27T02:32:26.434 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-01-27T02:32:26.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:26.435 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-27T02:32:26.435 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-27T02:32:26.435 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-01-27T02:32:26.435 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-27T02:32:26.435 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-01-27T02:32:26.435 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-01-27T02:32:26.435 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-01-27T02:32:26.435 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-27T02:32:26.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:26.659 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-27T02:32:26.659 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-27T02:32:26.660 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-27T02:32:26.660 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2024-01-27T02:32:26.660 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-27T02:32:26.771 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:32:26.771 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-27T02:32:26.790 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:32:26.791 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:32:26.791 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-27T02:32:26.851 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:32:26.851 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:32:26.851 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:32:26.947 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:32:26.947 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:32:26.947 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:32:27.025 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:32:27.025 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:32:27.025 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:32:27.025 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:32:27.026 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:32:27.043 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:32:27.043 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:32:27.043 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:32:27.043 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:32:27.043 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:32:27.043 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:32:27.043 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:32:27.043 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:32:27.046 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:32:27.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:27.046 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-27T02:32:27.077 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-01-27T02:32:27.077 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:32:27.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.253+0000 7f37ca8d7700 1 Processor -- start 2024-01-27T02:32:27.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.253+0000 7f37ca8d7700 1 -- start start 2024-01-27T02:32:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.254+0000 7f37ca8d7700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f37c40d10a0 0x7f37c40b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.254+0000 7f37ca8d7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37c4159b40 0x7f37c4159f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.254+0000 7f37ca8d7700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c40b4650 con 0x7f37c40d1470 2024-01-27T02:32:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.254+0000 7f37ca8d7700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c40b34d0 con 0x7f37c40d10a0 2024-01-27T02:32:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.254+0000 7f37ca8d7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c40b3d60 con 0x7f37c4159b40 2024-01-27T02:32:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.254+0000 7f37c8e74700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37c4159b40 0x7f37c4159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.254+0000 7f37c3fff700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f37c40d10a0 0x7f37c40b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.254+0000 7f37c37fe700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f37c40d1470 0x7f37c40b2dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:46342/0 (socket says 172.21.15.7:46342) 2024-01-27T02:32:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.254+0000 7f37c8e74700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37c4159b40 0x7f37c4159f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:48338/0 (socket says 172.21.15.7:48338) 2024-01-27T02:32:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.254+0000 7f37c37fe700 1 -- 172.21.15.7:0/1438346325 learned_addr learned my addr 172.21.15.7:0/1438346325 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.254+0000 7f37c8e74700 1 -- 172.21.15.7:0/1438346325 >> v2:172.21.15.188:3300/0 conn(0x7f37c40d10a0 msgr2=0x7f37c40b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.254+0000 7f37c2ffd700 1 -- 172.21.15.7:0/1438346325 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355736692 0 0) 0x7f37c40b4650 con 0x7f37c40d1470 2024-01-27T02:32:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.254+0000 7f37c8e74700 1 --2- 172.21.15.7:0/1438346325 >> v2:172.21.15.188:3300/0 conn(0x7f37c40d10a0 0x7f37c40b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.254+0000 7f37c8e74700 1 -- 172.21.15.7:0/1438346325 >> v1:172.21.15.188:6789/0 conn(0x7f37c40d1470 legacy=0x7f37c40b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.254+0000 7f37c8e74700 1 -- 172.21.15.7:0/1438346325 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c40b4000 con 0x7f37c4159b40 2024-01-27T02:32:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.255+0000 7f37c8e74700 1 --2- 172.21.15.7:0/1438346325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37c4159b40 0x7f37c4159f80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009e60 tx=0x7f37bc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2263f05b1515fba server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.255+0000 7f37c2ffd700 1 -- 172.21.15.7:0/1438346325 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc00e070 con 0x7f37c4159b40 2024-01-27T02:32:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.255+0000 7f37c2ffd700 1 -- 172.21.15.7:0/1438346325 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37bc00bb60 con 0x7f37c4159b40 2024-01-27T02:32:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.255+0000 7f37c2ffd700 1 -- 172.21.15.7:0/1438346325 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc012430 con 0x7f37c4159b40 2024-01-27T02:32:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.255+0000 7f37ca8d7700 1 -- 172.21.15.7:0/1438346325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37c4159b40 msgr2=0x7f37c4159f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.255+0000 7f37ca8d7700 1 --2- 172.21.15.7:0/1438346325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37c4159b40 0x7f37c4159f80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009e60 tx=0x7f37bc009780 comp rx=0 tx=0).stop 2024-01-27T02:32:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.255+0000 7f37ca8d7700 1 -- 172.21.15.7:0/1438346325 shutdown_connections 2024-01-27T02:32:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.255+0000 7f37ca8d7700 1 --2- 172.21.15.7:0/1438346325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37c4159b40 0x7f37c4159f80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:27.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.255+0000 7f37ca8d7700 1 --2- 172.21.15.7:0/1438346325 >> v2:172.21.15.188:3300/0 conn(0x7f37c40d10a0 0x7f37c40b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:27.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.255+0000 7f37ca8d7700 1 -- 172.21.15.7:0/1438346325 >> 172.21.15.7:0/1438346325 conn(0x7f37c40679b0 msgr2=0x7f37c4067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:27.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.255+0000 7f37ca8d7700 1 -- 172.21.15.7:0/1438346325 shutdown_connections 2024-01-27T02:32:27.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.255+0000 7f37ca8d7700 1 -- 172.21.15.7:0/1438346325 wait complete. 2024-01-27T02:32:27.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.257+0000 7f37ca8d7700 1 Processor -- start 2024-01-27T02:32:27.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.257+0000 7f37ca8d7700 1 -- start start 2024-01-27T02:32:27.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.258+0000 7f37ca8d7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37c40d10a0 0x7f37c4156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:27.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.258+0000 7f37ca8d7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c40d1470 0x7f37c41471c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:27.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.258+0000 7f37ca8d7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37c4159b40 0x7f37c40c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:27.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.258+0000 7f37ca8d7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c40c5ac0 con 0x7f37c40d1470 2024-01-27T02:32:27.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.258+0000 7f37ca8d7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c40c4eb0 con 0x7f37c4159b40 2024-01-27T02:32:27.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.258+0000 7f37ca8d7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c40c5190 con 0x7f37c40d10a0 2024-01-27T02:32:27.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.258+0000 7f37c37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c40d1470 0x7f37c41471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:27.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.258+0000 7f37c37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c40d1470 0x7f37c41471c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56342/0 (socket says 172.21.15.7:56342) 2024-01-27T02:32:27.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.258+0000 7f37c37fe700 1 -- 172.21.15.7:0/38672817 learned_addr learned my addr 172.21.15.7:0/38672817 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:27.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.258+0000 7f37c3fff700 1 --2- 172.21.15.7:0/38672817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37c40d10a0 0x7f37c4156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:27.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.258+0000 7f37c8e74700 1 --2- 172.21.15.7:0/38672817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37c4159b40 0x7f37c40c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:27.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.258+0000 7f37c37fe700 1 -- 172.21.15.7:0/38672817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37c40d10a0 msgr2=0x7f37c4156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:27.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.258+0000 7f37c37fe700 1 --2- 172.21.15.7:0/38672817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37c40d10a0 0x7f37c4156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:27.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.258+0000 7f37c37fe700 1 -- 172.21.15.7:0/38672817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37c4159b40 msgr2=0x7f37c40c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:27.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.258+0000 7f37c37fe700 1 --2- 172.21.15.7:0/38672817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37c4159b40 0x7f37c40c9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:27.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.258+0000 7f37c37fe700 1 -- 172.21.15.7:0/38672817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37bc009a20 con 0x7f37c40d1470 2024-01-27T02:32:27.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.258+0000 7f37c8e74700 1 --2- 172.21.15.7:0/38672817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37c4159b40 0x7f37c40c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:27.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.258+0000 7f37c37fe700 1 --2- 172.21.15.7:0/38672817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c40d1470 0x7f37c41471c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f37b800c7e0 tx=0x7f37b8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:27.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.259+0000 7f37abfff700 1 -- 172.21.15.7:0/38672817 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b8005b30 con 0x7f37c40d1470 2024-01-27T02:32:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.259+0000 7f37abfff700 1 -- 172.21.15.7:0/38672817 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b8005c90 con 0x7f37c40d1470 2024-01-27T02:32:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.259+0000 7f37abfff700 1 -- 172.21.15.7:0/38672817 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b8007300 con 0x7f37c40d1470 2024-01-27T02:32:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.259+0000 7f37ca8d7700 1 -- 172.21.15.7:0/38672817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c40c5470 con 0x7f37c40d1470 2024-01-27T02:32:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.259+0000 7f37ca8d7700 1 -- 172.21.15.7:0/38672817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c41565a0 con 0x7f37c40d1470 2024-01-27T02:32:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.260+0000 7f37ca8d7700 1 -- 172.21.15.7:0/38672817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c404c710 con 0x7f37c40d1470 2024-01-27T02:32:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.260+0000 7f37abfff700 1 -- 172.21.15.7:0/38672817 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f37b80076b0 con 0x7f37c40d1470 2024-01-27T02:32:27.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.261+0000 7f37abfff700 1 --2- 172.21.15.7:0/38672817 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f37b0066f30 0x7f37b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:27.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.261+0000 7f37abfff700 1 -- 172.21.15.7:0/38672817 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f37b8083a90 con 0x7f37c40d1470 2024-01-27T02:32:27.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.261+0000 7f37c3fff700 1 --2- 172.21.15.7:0/38672817 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f37b0066f30 0x7f37b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:27.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.261+0000 7f37c3fff700 1 --2- 172.21.15.7:0/38672817 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f37b0066f30 0x7f37b00693e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009db0 tx=0x7f37b40091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.265+0000 7f37abfff700 1 -- 172.21.15.7:0/38672817 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37b8050d00 con 0x7f37c40d1470 2024-01-27T02:32:27.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.518+0000 7f37ca8d7700 1 -- 172.21.15.7:0/38672817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f37c40b42e0 con 0x7f37c40d1470 2024-01-27T02:32:27.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.518+0000 7f37abfff700 1 -- 172.21.15.7:0/38672817 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f37b8017070 con 0x7f37c40d1470 2024-01-27T02:32:27.519 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:32:27.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.520+0000 7f37ca8d7700 1 -- 172.21.15.7:0/38672817 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f37b0066f30 msgr2=0x7f37b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:27.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.520+0000 7f37ca8d7700 1 --2- 172.21.15.7:0/38672817 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f37b0066f30 0x7f37b00693e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009db0 tx=0x7f37b40091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:27.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.520+0000 7f37ca8d7700 1 -- 172.21.15.7:0/38672817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c40d1470 msgr2=0x7f37c41471c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:27.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.520+0000 7f37ca8d7700 1 --2- 172.21.15.7:0/38672817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c40d1470 0x7f37c41471c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f37b800c7e0 tx=0x7f37b8007df0 comp rx=0 tx=0).stop 2024-01-27T02:32:27.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.520+0000 7f37ca8d7700 1 -- 172.21.15.7:0/38672817 shutdown_connections 2024-01-27T02:32:27.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.520+0000 7f37ca8d7700 1 --2- 172.21.15.7:0/38672817 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f37b0066f30 0x7f37b00693e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:27.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.520+0000 7f37ca8d7700 1 --2- 172.21.15.7:0/38672817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37c4159b40 0x7f37c40c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:27.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.520+0000 7f37ca8d7700 1 --2- 172.21.15.7:0/38672817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37c40d10a0 0x7f37c4156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:27.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.520+0000 7f37ca8d7700 1 --2- 172.21.15.7:0/38672817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c40d1470 0x7f37c41471c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:27.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.520+0000 7f37ca8d7700 1 -- 172.21.15.7:0/38672817 >> 172.21.15.7:0/38672817 conn(0x7f37c40679b0 msgr2=0x7f37c415dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:27.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.520+0000 7f37ca8d7700 1 -- 172.21.15.7:0/38672817 shutdown_connections 2024-01-27T02:32:27.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:27.520+0000 7f37ca8d7700 1 -- 172.21.15.7:0/38672817 wait complete. 2024-01-27T02:32:27.534 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-27T02:32:27.534 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:32:27.534 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-27T02:32:27.534 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-01-27T02:32:27.534 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-01-27T02:32:27.534 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-27T02:32:27.534 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-27T02:32:27.534 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:32:27.535 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-01-27T02:32:27.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:27.535 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-27T02:32:27.554 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:32:27.554 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:32:27.554 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:32:27.554 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:27.554 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-27T02:32:27.554 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:27.555 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:32:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-27T02:32:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-27T02:32:27.555 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:32:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-27T02:32:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-27T02:32:27.555 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:32:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-27T02:32:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-27T02:32:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-27T02:32:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42364sec preferred_lft 42364sec 2024-01-27T02:32:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-27T02:32:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:27.555 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:32:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-27T02:32:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-27T02:32:27.556 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:32:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 4a:57:d9:3c:c5:a0 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:32:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-27T02:32:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::2477:9eff:fe39:433c/64 scope link tentative 2024-01-27T02:32:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:27.556 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:32:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 4a:57:d9:3c:c5:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-27T02:32:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::4857:d9ff:fe3c:c5a0/64 scope link tentative 2024-01-27T02:32:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:27.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:27.560 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:32:27.625 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-27T02:32:27.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:27.626 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-27T02:32:27.692 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-27T02:32:27.692 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:32:27.692 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-27T02:32:27.692 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-27T02:32:27.692 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:32:27.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:27.693 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-27T02:32:27.693 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-27T02:32:27.693 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-27T02:32:27.693 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-27T02:32:27.694 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-27T02:32:27.694 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-27T02:32:27.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:27.818 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-27T02:32:27.838 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-27T02:32:27.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:27.839 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-01-27T02:32:27.900 INFO:teuthology.orchestra.run.smithi007.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-01-27T02:32:27.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:27.901 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-27T02:32:27.901 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-27T02:32:27.901 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-27T02:32:27.901 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-01-27T02:32:27.901 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-27T02:32:28.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:28.012 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-01-27T02:32:28.058 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:32:28.059 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:32:28.059 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:32:28.059 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:28.059 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-27T02:32:28.059 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:28.059 INFO:teuthology.orchestra.run.smithi007.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:32:28.059 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether be:2f:ac:8d:be:7f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-27T02:32:28.059 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-27T02:32:28.059 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:28.059 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::bc2f:acff:fe8d:be7f/64 scope link tentative 2024-01-27T02:32:28.059 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:28.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:28.073 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-01-27T02:32:28.164 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:32:28.164 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:32:28.164 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:32:28.164 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:28.164 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-27T02:32:28.165 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:28.165 INFO:teuthology.orchestra.run.smithi007.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:32:28.165 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether be:2f:ac:8d:be:7f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-27T02:32:28.165 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-27T02:32:28.165 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:28.165 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::bc2f:acff:fe8d:be7f/64 scope link tentative 2024-01-27T02:32:28.165 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:28.175 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-01-27T02:32:28.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:28.175 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-27T02:32:28.175 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-27T02:32:28.176 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-01-27T02:32:28.176 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-01-27T02:32:28.176 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-01-27T02:32:28.176 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-01-27T02:32:28.176 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-01-27T02:32:28.176 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-27T02:32:28.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:28.412 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-27T02:32:28.412 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-27T02:32:28.412 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.1 up 2024-01-27T02:32:28.412 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.1 master ceph-brx 2024-01-27T02:32:28.413 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-27T02:32:28.514 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:32:28.514 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-01-27T02:32:28.534 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-01-27T02:32:28.534 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:32:28.535 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-01-27T02:32:28.596 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:32:28.596 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:32:28.596 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-01-27T02:32:28.691 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-01-27T02:32:28.691 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-01-27T02:32:28.691 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:32:28.701 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-01-27T02:32:28.701 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-01-27T02:32:28.701 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:32:28.701 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:32:28.701 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:32:28.719 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-01-27T02:32:28.720 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-01-27T02:32:28.720 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:32:28.720 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:32:28.720 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:32:28.720 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:32:28.720 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:32:28.720 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:32:28.722 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:32:28.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:28.723 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-01-27T02:32:28.754 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-01-27T02:32:28.754 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:32:28.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.934+0000 7f215144e700 1 Processor -- start 2024-01-27T02:32:28.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.934+0000 7f215144e700 1 -- start start 2024-01-27T02:32:28.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.935+0000 7f215144e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f214c159f40 0x7f214c0c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:28.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.935+0000 7f215144e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f214c0c30c0 con 0x7f214c0c3a90 2024-01-27T02:32:28.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.935+0000 7f215144e700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f214c15e960 con 0x7f214c0aafc0 2024-01-27T02:32:28.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.935+0000 7f215144e700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f214c0c4550 con 0x7f214c159f40 2024-01-27T02:32:28.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.935+0000 7f214a7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f214c0c3a90 0x7f214c0aa8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43632/0 (socket says 172.21.15.7:43632) 2024-01-27T02:32:28.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.935+0000 7f214a7fc700 1 -- 172.21.15.7:0/1620183754 learned_addr learned my addr 172.21.15.7:0/1620183754 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:28.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.936+0000 7f214affd700 1 --2- 172.21.15.7:0/1620183754 >> v2:172.21.15.188:3300/0 conn(0x7f214c159f40 0x7f214c0c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:28.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.936+0000 7f2149ffb700 1 -- 172.21.15.7:0/1620183754 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992202895 0 0) 0x7f214c0c30c0 con 0x7f214c0c3a90 2024-01-27T02:32:28.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.936+0000 7f2149ffb700 1 -- 172.21.15.7:0/1620183754 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2134003570 con 0x7f214c0c3a90 2024-01-27T02:32:28.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.936+0000 7f214affd700 1 -- 172.21.15.7:0/1620183754 >> v1:172.21.15.188:6789/0 conn(0x7f214c0aafc0 legacy=0x7f214c063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:28.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.936+0000 7f214affd700 1 -- 172.21.15.7:0/1620183754 >> v1:172.21.15.7:6789/0 conn(0x7f214c0c3a90 legacy=0x7f214c0aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:28.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.936+0000 7f214affd700 1 -- 172.21.15.7:0/1620183754 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214c15ec40 con 0x7f214c159f40 2024-01-27T02:32:28.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.936+0000 7f214affd700 1 --2- 172.21.15.7:0/1620183754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f214c159f40 0x7f214c0c3550 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f213c0098e0 tx=0x7f213c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4b6959cab591b4b server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:28.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.936+0000 7f2149ffb700 1 -- 172.21.15.7:0/1620183754 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c00b640 con 0x7f214c159f40 2024-01-27T02:32:28.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.936+0000 7f2149ffb700 1 -- 172.21.15.7:0/1620183754 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c00b7a0 con 0x7f214c159f40 2024-01-27T02:32:28.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.936+0000 7f2149ffb700 1 -- 172.21.15.7:0/1620183754 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c00bb40 con 0x7f214c159f40 2024-01-27T02:32:28.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.937+0000 7f215144e700 1 -- 172.21.15.7:0/1620183754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f214c159f40 msgr2=0x7f214c0c3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:28.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.937+0000 7f215144e700 1 --2- 172.21.15.7:0/1620183754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f214c159f40 0x7f214c0c3550 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f213c0098e0 tx=0x7f213c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:28.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.937+0000 7f215144e700 1 -- 172.21.15.7:0/1620183754 shutdown_connections 2024-01-27T02:32:28.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.937+0000 7f215144e700 1 --2- 172.21.15.7:0/1620183754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f214c159f40 0x7f214c0c3550 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:28.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.937+0000 7f215144e700 1 -- 172.21.15.7:0/1620183754 >> 172.21.15.7:0/1620183754 conn(0x7f214c0cdc30 msgr2=0x7f214c0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:28.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.937+0000 7f215144e700 1 -- 172.21.15.7:0/1620183754 shutdown_connections 2024-01-27T02:32:28.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.937+0000 7f215144e700 1 -- 172.21.15.7:0/1620183754 wait complete. 2024-01-27T02:32:28.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.938+0000 7f215144e700 1 Processor -- start 2024-01-27T02:32:28.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.939+0000 7f215144e700 1 -- start start 2024-01-27T02:32:28.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.939+0000 7f215144e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f214c0aafc0 0x7f214c150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:28.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.939+0000 7f215144e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f214c0c3a90 0x7f214c15a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:28.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.939+0000 7f215144e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f214c159f40 0x7f214c159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:28.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.939+0000 7f215144e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214c0b1920 con 0x7f214c0aafc0 2024-01-27T02:32:28.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.939+0000 7f215144e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214c15b240 con 0x7f214c159f40 2024-01-27T02:32:28.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.939+0000 7f215144e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f214c15ac80 con 0x7f214c0c3a90 2024-01-27T02:32:28.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.939+0000 7f214affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f214c0aafc0 0x7f214c150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:28.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.939+0000 7f214affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f214c0aafc0 0x7f214c150ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56344/0 (socket says 172.21.15.7:56344) 2024-01-27T02:32:28.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.939+0000 7f214affd700 1 -- 172.21.15.7:0/2554458630 learned_addr learned my addr 172.21.15.7:0/2554458630 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:28.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.940+0000 7f214a7fc700 1 --2- 172.21.15.7:0/2554458630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f214c0c3a90 0x7f214c15a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:28.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.940+0000 7f214b7fe700 1 --2- 172.21.15.7:0/2554458630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f214c159f40 0x7f214c159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:28.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.940+0000 7f214affd700 1 -- 172.21.15.7:0/2554458630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f214c0c3a90 msgr2=0x7f214c15a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:28.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.940+0000 7f214affd700 1 --2- 172.21.15.7:0/2554458630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f214c0c3a90 0x7f214c15a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:28.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.940+0000 7f214affd700 1 -- 172.21.15.7:0/2554458630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f214c159f40 msgr2=0x7f214c159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:28.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.940+0000 7f214affd700 1 --2- 172.21.15.7:0/2554458630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f214c159f40 0x7f214c159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:28.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.940+0000 7f214affd700 1 -- 172.21.15.7:0/2554458630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213c009420 con 0x7f214c0aafc0 2024-01-27T02:32:28.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.940+0000 7f214affd700 1 --2- 172.21.15.7:0/2554458630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f214c0aafc0 0x7f214c150ab0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f213c0093f0 tx=0x7f213c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:28.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.940+0000 7f2132ffd700 1 -- 172.21.15.7:0/2554458630 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c000d10 con 0x7f214c0aafc0 2024-01-27T02:32:28.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.940+0000 7f2132ffd700 1 -- 172.21.15.7:0/2554458630 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c00bb20 con 0x7f214c0aafc0 2024-01-27T02:32:28.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.940+0000 7f2132ffd700 1 -- 172.21.15.7:0/2554458630 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c011670 con 0x7f214c0aafc0 2024-01-27T02:32:28.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.940+0000 7f215144e700 1 -- 172.21.15.7:0/2554458630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f214c15af60 con 0x7f214c0aafc0 2024-01-27T02:32:28.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.940+0000 7f215144e700 1 -- 172.21.15.7:0/2554458630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f214c150ff0 con 0x7f214c0aafc0 2024-01-27T02:32:28.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.941+0000 7f215144e700 1 -- 172.21.15.7:0/2554458630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f214c04c710 con 0x7f214c0aafc0 2024-01-27T02:32:28.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.942+0000 7f2132ffd700 1 -- 172.21.15.7:0/2554458630 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f213c011aa0 con 0x7f214c0aafc0 2024-01-27T02:32:28.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.942+0000 7f2132ffd700 1 --2- 172.21.15.7:0/2554458630 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2138066f30 0x7f21380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:28.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.942+0000 7f2132ffd700 1 -- 172.21.15.7:0/2554458630 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f213c085ef0 con 0x7f214c0aafc0 2024-01-27T02:32:28.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.942+0000 7f214a7fc700 1 --2- 172.21.15.7:0/2554458630 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2138066f30 0x7f21380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:28.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.943+0000 7f214a7fc700 1 --2- 172.21.15.7:0/2554458630 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2138066f30 0x7f21380693e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f214c0b32f0 tx=0x7f2140009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:28.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:28.947+0000 7f2132ffd700 1 -- 172.21.15.7:0/2554458630 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f213c0530c0 con 0x7f214c0aafc0 2024-01-27T02:32:29.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:29.198+0000 7f215144e700 1 -- 172.21.15.7:0/2554458630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f214c06f770 con 0x7f214c0aafc0 2024-01-27T02:32:29.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:29.199+0000 7f2132ffd700 1 -- 172.21.15.7:0/2554458630 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f213c0566e0 con 0x7f214c0aafc0 2024-01-27T02:32:29.200 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:32:29.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:29.201+0000 7f215144e700 1 -- 172.21.15.7:0/2554458630 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2138066f30 msgr2=0x7f21380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:29.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:29.201+0000 7f215144e700 1 --2- 172.21.15.7:0/2554458630 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2138066f30 0x7f21380693e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f214c0b32f0 tx=0x7f2140009040 comp rx=0 tx=0).stop 2024-01-27T02:32:29.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:29.201+0000 7f215144e700 1 -- 172.21.15.7:0/2554458630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f214c0aafc0 msgr2=0x7f214c150ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:29.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:29.201+0000 7f215144e700 1 --2- 172.21.15.7:0/2554458630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f214c0aafc0 0x7f214c150ab0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f213c0093f0 tx=0x7f213c017000 comp rx=0 tx=0).stop 2024-01-27T02:32:29.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:29.201+0000 7f215144e700 1 -- 172.21.15.7:0/2554458630 shutdown_connections 2024-01-27T02:32:29.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:29.201+0000 7f215144e700 1 --2- 172.21.15.7:0/2554458630 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2138066f30 0x7f21380693e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:29.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:29.201+0000 7f215144e700 1 --2- 172.21.15.7:0/2554458630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f214c159f40 0x7f214c159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:29.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:29.201+0000 7f215144e700 1 --2- 172.21.15.7:0/2554458630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f214c0aafc0 0x7f214c150ab0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:29.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:29.201+0000 7f215144e700 1 --2- 172.21.15.7:0/2554458630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f214c0c3a90 0x7f214c15a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:29.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:29.201+0000 7f215144e700 1 -- 172.21.15.7:0/2554458630 >> 172.21.15.7:0/2554458630 conn(0x7f214c0cdc30 msgr2=0x7f214c0aa8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:29.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:29.201+0000 7f215144e700 1 -- 172.21.15.7:0/2554458630 shutdown_connections 2024-01-27T02:32:29.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:29.201+0000 7f215144e700 1 -- 172.21.15.7:0/2554458630 wait complete. 2024-01-27T02:32:29.214 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-27T02:32:29.215 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:32:29.215 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-01-27T02:32:29.215 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-27T02:32:29.215 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-27T02:32:29.215 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-27T02:32:29.215 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-27T02:32:29.215 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:32:29.215 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:32:29.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:29.215 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-01-27T02:32:29.236 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:32:29.236 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:32:29.237 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:32:29.237 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:29.237 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-01-27T02:32:29.237 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:29.237 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:32:29.237 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:32:29.237 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-01-27T02:32:29.237 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:32:29.237 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:32:29.237 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-01-27T02:32:29.237 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:32:29.237 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:32:29.237 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:32:29.238 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:32:29.238 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-27T02:32:29.238 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42369sec preferred_lft 42369sec 2024-01-27T02:32:29.238 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b1:d6f7:baf0:bc17/64 scope link noprefixroute 2024-01-27T02:32:29.238 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:29.238 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-27T02:32:29.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:29.238 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-27T02:32:29.238 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-27T02:32:29.239 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add name ceph-brx type bridge 2024-01-27T02:32:29.239 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip addr flush dev ceph-brx 2024-01-27T02:32:29.239 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set ceph-brx up 2024-01-27T02:32:29.239 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-27T02:32:29.239 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-27T02:32:29.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:29.413 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:32:29.441 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-01-27T02:32:29.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:29.443 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-01-27T02:32:29.515 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-01-27T02:32:29.515 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:32:29.515 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-27T02:32:29.515 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-27T02:32:29.515 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:32:29.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:29.516 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-27T02:32:29.516 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-27T02:32:29.516 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-27T02:32:29.516 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-27T02:32:29.516 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-27T02:32:29.516 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-27T02:32:29.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:29.757 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-01-27T02:32:29.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:29.775 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list-id 2024-01-27T02:32:29.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:29.833 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-27T02:32:29.833 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-27T02:32:29.833 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-27T02:32:29.833 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-01-27T02:32:29.833 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-27T02:32:29.929 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-01-27T02:32:29.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:29.929 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-27T02:32:29.929 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-27T02:32:29.930 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-01-27T02:32:29.930 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-27T02:32:29.930 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-01-27T02:32:29.930 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-01-27T02:32:29.930 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-01-27T02:32:29.930 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-27T02:32:30.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:30.139 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-27T02:32:30.139 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-27T02:32:30.140 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2024-01-27T02:32:30.140 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set dev brx.0 master ceph-brx 2024-01-27T02:32:30.140 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-27T02:32:30.237 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:32:30.238 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-27T02:32:30.254 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:32:30.254 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:32:30.254 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-27T02:32:30.311 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:32:30.311 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:32:30.312 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:32:30.394 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:32:30.395 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:32:30.395 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:32:30.479 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:32:30.479 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:32:30.479 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:32:30.480 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:32:30.480 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:32:30.497 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:32:30.497 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:32:30.497 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:32:30.497 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:32:30.497 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:32:30.497 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:32:30.497 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:32:30.497 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:32:30.499 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:32:30.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:30.499 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-27T02:32:30.566 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-01-27T02:32:30.566 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:32:30.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.709+0000 7eff5a83a700 1 Processor -- start 2024-01-27T02:32:30.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.709+0000 7eff5a83a700 1 -- start start 2024-01-27T02:32:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.710+0000 7eff5a83a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7eff540a7c10 0x7eff540ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.710+0000 7eff5a83a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff540a7fe0 0x7eff540b0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.710+0000 7eff5a83a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff5415a880 con 0x7eff540ae880 2024-01-27T02:32:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.710+0000 7eff5a83a700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff540afde0 con 0x7eff540a7c10 2024-01-27T02:32:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.710+0000 7eff5a83a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff540af950 con 0x7eff540a7fe0 2024-01-27T02:32:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.710+0000 7eff537fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7eff540ae880 0x7eff5406f360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43644/0 (socket says 172.21.15.7:43644) 2024-01-27T02:32:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.711+0000 7eff537fe700 1 -- 172.21.15.7:0/188600795 learned_addr learned my addr 172.21.15.7:0/188600795 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.711+0000 7eff58dd7700 1 --2- 172.21.15.7:0/188600795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff540a7fe0 0x7eff540b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.711+0000 7eff53fff700 1 --2- 172.21.15.7:0/188600795 >> v2:172.21.15.188:3300/0 conn(0x7eff540a7c10 0x7eff540ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.711+0000 7eff52ffd700 1 -- 172.21.15.7:0/188600795 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799132815 0 0) 0x7eff5415a880 con 0x7eff540ae880 2024-01-27T02:32:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.711+0000 7eff52ffd700 1 -- 172.21.15.7:0/188600795 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff3c003570 con 0x7eff540ae880 2024-01-27T02:32:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.711+0000 7eff58dd7700 1 -- 172.21.15.7:0/188600795 >> v2:172.21.15.188:3300/0 conn(0x7eff540a7c10 msgr2=0x7eff540ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.711+0000 7eff58dd7700 1 --2- 172.21.15.7:0/188600795 >> v2:172.21.15.188:3300/0 conn(0x7eff540a7c10 0x7eff540ae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.711+0000 7eff58dd7700 1 -- 172.21.15.7:0/188600795 >> v1:172.21.15.7:6789/0 conn(0x7eff540ae880 legacy=0x7eff5406f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.711+0000 7eff52ffd700 1 -- 172.21.15.7:0/188600795 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1189011640 0 0) 0x7eff3c003570 con 0x7eff540ae880 2024-01-27T02:32:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.711+0000 7eff58dd7700 1 -- 172.21.15.7:0/188600795 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff540aed40 con 0x7eff540a7fe0 2024-01-27T02:32:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.711+0000 7eff53fff700 1 --2- 172.21.15.7:0/188600795 >> v2:172.21.15.188:3300/0 conn(0x7eff540a7c10 0x7eff540ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:32:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.711+0000 7eff58dd7700 1 --2- 172.21.15.7:0/188600795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff540a7fe0 0x7eff540b0180 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009e60 tx=0x7eff4c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=521bae73e647245a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.712+0000 7eff52ffd700 1 -- 172.21.15.7:0/188600795 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4c00e070 con 0x7eff540a7fe0 2024-01-27T02:32:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.712+0000 7eff52ffd700 1 -- 172.21.15.7:0/188600795 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4c00b7a0 con 0x7eff540a7fe0 2024-01-27T02:32:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.712+0000 7eff52ffd700 1 -- 172.21.15.7:0/188600795 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff48001470 con 0x7eff540a7fe0 2024-01-27T02:32:30.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.712+0000 7eff5a83a700 1 -- 172.21.15.7:0/188600795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff540a7fe0 msgr2=0x7eff540b0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.712+0000 7eff5a83a700 1 --2- 172.21.15.7:0/188600795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff540a7fe0 0x7eff540b0180 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009e60 tx=0x7eff4c009780 comp rx=0 tx=0).stop 2024-01-27T02:32:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.712+0000 7eff5a83a700 1 -- 172.21.15.7:0/188600795 shutdown_connections 2024-01-27T02:32:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.712+0000 7eff5a83a700 1 --2- 172.21.15.7:0/188600795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff540a7fe0 0x7eff540b0180 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.712+0000 7eff5a83a700 1 --2- 172.21.15.7:0/188600795 >> v2:172.21.15.188:3300/0 conn(0x7eff540a7c10 0x7eff540ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.712+0000 7eff5a83a700 1 -- 172.21.15.7:0/188600795 >> 172.21.15.7:0/188600795 conn(0x7eff540679b0 msgr2=0x7eff54067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.713+0000 7eff5a83a700 1 -- 172.21.15.7:0/188600795 shutdown_connections 2024-01-27T02:32:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.713+0000 7eff5a83a700 1 -- 172.21.15.7:0/188600795 wait complete. 2024-01-27T02:32:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.714+0000 7eff5a83a700 1 Processor -- start 2024-01-27T02:32:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.714+0000 7eff5a83a700 1 -- start start 2024-01-27T02:32:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.715+0000 7eff5a83a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff540a7c10 0x7eff54152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.715+0000 7eff5a83a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff540a7fe0 0x7eff5409a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.715+0000 7eff5a83a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff540ae880 0x7eff54097a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.715+0000 7eff5a83a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff541509d0 con 0x7eff540a7fe0 2024-01-27T02:32:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.715+0000 7eff5a83a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5416ca80 con 0x7eff540ae880 2024-01-27T02:32:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.715+0000 7eff5a83a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5416be70 con 0x7eff540a7c10 2024-01-27T02:32:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.715+0000 7eff537fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff540a7fe0 0x7eff5409a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.715+0000 7eff537fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff540a7fe0 0x7eff5409a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56356/0 (socket says 172.21.15.7:56356) 2024-01-27T02:32:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.715+0000 7eff537fe700 1 -- 172.21.15.7:0/2015590530 learned_addr learned my addr 172.21.15.7:0/2015590530 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.715+0000 7eff58dd7700 1 --2- 172.21.15.7:0/2015590530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff540ae880 0x7eff54097a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.715+0000 7eff53fff700 1 --2- 172.21.15.7:0/2015590530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff540a7c10 0x7eff54152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.715+0000 7eff537fe700 1 -- 172.21.15.7:0/2015590530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff540a7c10 msgr2=0x7eff54152b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.715+0000 7eff537fe700 1 --2- 172.21.15.7:0/2015590530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff540a7c10 0x7eff54152b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.715+0000 7eff537fe700 1 -- 172.21.15.7:0/2015590530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff540ae880 msgr2=0x7eff54097a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.715+0000 7eff537fe700 1 --2- 172.21.15.7:0/2015590530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff540ae880 0x7eff54097a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.715+0000 7eff537fe700 1 -- 172.21.15.7:0/2015590530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff44009420 con 0x7eff540a7fe0 2024-01-27T02:32:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.715+0000 7eff537fe700 1 --2- 172.21.15.7:0/2015590530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff540a7fe0 0x7eff5409a770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7eff4800c8e0 tx=0x7eff48001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.716+0000 7eff3bfff700 1 -- 172.21.15.7:0/2015590530 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff48005b30 con 0x7eff540a7fe0 2024-01-27T02:32:30.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.716+0000 7eff3bfff700 1 -- 172.21.15.7:0/2015590530 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff48005c90 con 0x7eff540a7fe0 2024-01-27T02:32:30.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.716+0000 7eff5a83a700 1 -- 172.21.15.7:0/2015590530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff4c009a20 con 0x7eff540a7fe0 2024-01-27T02:32:30.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.716+0000 7eff3bfff700 1 -- 172.21.15.7:0/2015590530 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff48007340 con 0x7eff540a7fe0 2024-01-27T02:32:30.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.716+0000 7eff5a83a700 1 -- 172.21.15.7:0/2015590530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff5416c430 con 0x7eff540a7fe0 2024-01-27T02:32:30.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.717+0000 7eff5a83a700 1 -- 172.21.15.7:0/2015590530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff5404c710 con 0x7eff540a7fe0 2024-01-27T02:32:30.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.717+0000 7eff3bfff700 1 -- 172.21.15.7:0/2015590530 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7eff4801b400 con 0x7eff540a7fe0 2024-01-27T02:32:30.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.717+0000 7eff3bfff700 1 --2- 172.21.15.7:0/2015590530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7eff40066f30 0x7eff400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:30.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.718+0000 7eff3bfff700 1 -- 172.21.15.7:0/2015590530 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7eff480841c0 con 0x7eff540a7fe0 2024-01-27T02:32:30.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.718+0000 7eff53fff700 1 --2- 172.21.15.7:0/2015590530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7eff40066f30 0x7eff400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:30.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.718+0000 7eff53fff700 1 --2- 172.21.15.7:0/2015590530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7eff40066f30 0x7eff400693e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7eff44009f40 tx=0x7eff440091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:30.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.722+0000 7eff3bfff700 1 -- 172.21.15.7:0/2015590530 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff48051390 con 0x7eff540a7fe0 2024-01-27T02:32:30.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.973+0000 7eff5a83a700 1 -- 172.21.15.7:0/2015590530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7eff5416cbc0 con 0x7eff540a7fe0 2024-01-27T02:32:30.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.974+0000 7eff3bfff700 1 -- 172.21.15.7:0/2015590530 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7eff480549b0 con 0x7eff540a7fe0 2024-01-27T02:32:30.974 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:32:30.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.976+0000 7eff5a83a700 1 -- 172.21.15.7:0/2015590530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7eff40066f30 msgr2=0x7eff400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:30.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.976+0000 7eff5a83a700 1 --2- 172.21.15.7:0/2015590530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7eff40066f30 0x7eff400693e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7eff44009f40 tx=0x7eff440091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:30.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.976+0000 7eff5a83a700 1 -- 172.21.15.7:0/2015590530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff540a7fe0 msgr2=0x7eff5409a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.976+0000 7eff5a83a700 1 --2- 172.21.15.7:0/2015590530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff540a7fe0 0x7eff5409a770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7eff4800c8e0 tx=0x7eff48001970 comp rx=0 tx=0).stop 2024-01-27T02:32:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.976+0000 7eff5a83a700 1 -- 172.21.15.7:0/2015590530 shutdown_connections 2024-01-27T02:32:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.976+0000 7eff5a83a700 1 --2- 172.21.15.7:0/2015590530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7eff40066f30 0x7eff400693e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.976+0000 7eff5a83a700 1 --2- 172.21.15.7:0/2015590530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff540ae880 0x7eff54097a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.976+0000 7eff5a83a700 1 --2- 172.21.15.7:0/2015590530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff540a7c10 0x7eff54152b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.976+0000 7eff5a83a700 1 --2- 172.21.15.7:0/2015590530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff540a7fe0 0x7eff5409a770 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.976+0000 7eff5a83a700 1 -- 172.21.15.7:0/2015590530 >> 172.21.15.7:0/2015590530 conn(0x7eff540679b0 msgr2=0x7eff5414f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.976+0000 7eff5a83a700 1 -- 172.21.15.7:0/2015590530 shutdown_connections 2024-01-27T02:32:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:30.976+0000 7eff5a83a700 1 -- 172.21.15.7:0/2015590530 wait complete. 2024-01-27T02:32:30.990 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-27T02:32:30.990 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:32:30.990 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-01-27T02:32:30.990 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-01-27T02:32:30.990 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-01-27T02:32:30.990 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-27T02:32:30.990 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-27T02:32:30.990 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:32:30.990 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-01-27T02:32:30.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:30.991 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-01-27T02:32:31.010 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:32:31.010 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:32:31.010 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:32:31.010 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:31.010 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-01-27T02:32:31.010 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:31.010 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:32:31.010 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:32:31.011 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-01-27T02:32:31.011 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:32:31.011 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:32:31.011 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-01-27T02:32:31.011 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:32:31.011 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:32:31.011 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:32:31.011 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:32:31.011 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-27T02:32:31.011 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42367sec preferred_lft 42367sec 2024-01-27T02:32:31.011 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b1:d6f7:baf0:bc17/64 scope link noprefixroute 2024-01-27T02:32:31.011 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:31.011 INFO:teuthology.orchestra.run.smithi188.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:32:31.011 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b0:92:71:90:9d brd ff:ff:ff:ff:ff:ff 2024-01-27T02:32:31.012 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-27T02:32:31.012 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:31.012 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::10f8:32ff:feed:a752/64 scope link tentative 2024-01-27T02:32:31.012 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:31.012 INFO:teuthology.orchestra.run.smithi188.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:32:31.012 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b0:92:71:90:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-27T02:32:31.012 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b8b0:92ff:fe71:909d/64 scope link tentative 2024-01-27T02:32:31.012 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:31.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:31.012 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:32:31.078 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-01-27T02:32:31.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:31.079 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-01-27T02:32:31.140 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-01-27T02:32:31.140 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:32:31.140 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-27T02:32:31.140 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-27T02:32:31.140 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:32:31.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:31.141 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-27T02:32:31.141 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-27T02:32:31.141 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-27T02:32:31.141 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-27T02:32:31.141 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-27T02:32:31.141 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-27T02:32:31.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:31.249 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-01-27T02:32:31.266 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-27T02:32:31.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:31.267 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list-id 2024-01-27T02:32:31.323 INFO:teuthology.orchestra.run.smithi188.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-01-27T02:32:31.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:31.323 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-27T02:32:31.323 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-27T02:32:31.323 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-27T02:32:31.323 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-01-27T02:32:31.323 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-27T02:32:31.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:31.417 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-01-27T02:32:31.452 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:32:31.452 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:32:31.452 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:32:31.452 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:31.452 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-01-27T02:32:31.453 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:31.453 INFO:teuthology.orchestra.run.smithi188.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:32:31.453 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b8:f9:81:98:45 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-27T02:32:31.453 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-27T02:32:31.453 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:31.453 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b8b8:f9ff:fe81:9845/64 scope link tentative 2024-01-27T02:32:31.453 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:31.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:31.468 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-01-27T02:32:31.548 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:32:31.548 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:32:31.548 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:32:31.548 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:31.548 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-01-27T02:32:31.548 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:31.548 INFO:teuthology.orchestra.run.smithi188.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:32:31.548 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b8:f9:81:98:45 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-27T02:32:31.548 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-27T02:32:31.549 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:31.549 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b8b8:f9ff:fe81:9845/64 scope link tentative 2024-01-27T02:32:31.549 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:32:31.561 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-01-27T02:32:31.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:31.561 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-27T02:32:31.561 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-27T02:32:31.562 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-01-27T02:32:31.562 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-01-27T02:32:31.562 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-01-27T02:32:31.562 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-01-27T02:32:31.562 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-01-27T02:32:31.562 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-27T02:32:31.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:31.790 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-27T02:32:31.790 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-27T02:32:31.790 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.1 up 2024-01-27T02:32:31.790 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set dev brx.1 master ceph-brx 2024-01-27T02:32:31.790 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-27T02:32:31.875 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:32:31.875 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-01-27T02:32:31.891 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-01-27T02:32:31.892 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:32:31.892 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-01-27T02:32:31.949 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:32:31.949 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:32:31.949 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-01-27T02:32:32.031 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-01-27T02:32:32.031 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-01-27T02:32:32.031 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:32:32.039 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-01-27T02:32:32.040 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-01-27T02:32:32.040 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:32:32.040 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:32:32.040 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:32:32.057 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-01-27T02:32:32.057 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-01-27T02:32:32.058 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:32:32.058 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:32:32.058 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:32:32.058 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:32:32.058 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:32:32.058 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:32:32.060 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:32:32.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:32.063 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-01-27T02:32:32.127 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-01-27T02:32:32.161 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:32:32.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.302+0000 7f70db227700 1 Processor -- start 2024-01-27T02:32:32.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.302+0000 7f70db227700 1 -- start start 2024-01-27T02:32:32.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.303+0000 7f70db227700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f70d40c9bb0 0x7f70d4157f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:32.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.303+0000 7f70db227700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f70d40c9f80 0x7f70d406b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:32.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.303+0000 7f70db227700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d406c880 con 0x7f70d406e330 2024-01-27T02:32:32.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.303+0000 7f70db227700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d406cd10 con 0x7f70d40c9f80 2024-01-27T02:32:32.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.303+0000 7f70db227700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d406bc70 con 0x7f70d40c9bb0 2024-01-27T02:32:32.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.303+0000 7f70d3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f70d40c9f80 0x7f70d406b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:32.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.303+0000 7f70d97c4700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f70d406e330 0x7f70d406e750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45492/0 (socket says 172.21.15.7:45492) 2024-01-27T02:32:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.303+0000 7f70d97c4700 1 -- 172.21.15.7:0/1038610012 learned_addr learned my addr 172.21.15.7:0/1038610012 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.303+0000 7f70d8fc3700 1 --2- 172.21.15.7:0/1038610012 >> v2:172.21.15.188:3300/0 conn(0x7f70d40c9bb0 0x7f70d4157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.303+0000 7f70d3fff700 1 -- 172.21.15.7:0/1038610012 >> v2:172.21.15.188:3300/0 conn(0x7f70d40c9bb0 msgr2=0x7f70d4157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.303+0000 7f70d3fff700 1 --2- 172.21.15.7:0/1038610012 >> v2:172.21.15.188:3300/0 conn(0x7f70d40c9bb0 0x7f70d4157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.303+0000 7f70d3fff700 1 -- 172.21.15.7:0/1038610012 >> v1:172.21.15.7:6789/0 conn(0x7f70d406e330 legacy=0x7f70d406e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.303+0000 7f70d3fff700 1 -- 172.21.15.7:0/1038610012 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d406bf50 con 0x7f70d40c9f80 2024-01-27T02:32:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.303+0000 7f70d37fe700 1 -- 172.21.15.7:0/1038610012 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2661669264 0 0) 0x7f70d406c880 con 0x7f70d406e330 2024-01-27T02:32:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.304+0000 7f70d3fff700 1 --2- 172.21.15.7:0/1038610012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70d40c9f80 0x7f70d406b400 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f70d406e020 tx=0x7f70c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c62746396052fad server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.304+0000 7f70d37fe700 1 -- 172.21.15.7:0/1038610012 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c000b6b0 con 0x7f70d40c9f80 2024-01-27T02:32:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.304+0000 7f70d37fe700 1 -- 172.21.15.7:0/1038610012 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c000b810 con 0x7f70d40c9f80 2024-01-27T02:32:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.304+0000 7f70d37fe700 1 -- 172.21.15.7:0/1038610012 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c000bbb0 con 0x7f70d40c9f80 2024-01-27T02:32:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.304+0000 7f70db227700 1 -- 172.21.15.7:0/1038610012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70d40c9f80 msgr2=0x7f70d406b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.304+0000 7f70db227700 1 --2- 172.21.15.7:0/1038610012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70d40c9f80 0x7f70d406b400 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f70d406e020 tx=0x7f70c00091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:32.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.304+0000 7f70db227700 1 -- 172.21.15.7:0/1038610012 shutdown_connections 2024-01-27T02:32:32.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.304+0000 7f70db227700 1 --2- 172.21.15.7:0/1038610012 >> v2:172.21.15.188:3300/0 conn(0x7f70d40c9bb0 0x7f70d4157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:32.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.304+0000 7f70db227700 1 --2- 172.21.15.7:0/1038610012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70d40c9f80 0x7f70d406b400 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:32.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.304+0000 7f70db227700 1 -- 172.21.15.7:0/1038610012 >> 172.21.15.7:0/1038610012 conn(0x7f70d40cdc30 msgr2=0x7f70d40ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:32.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.304+0000 7f70db227700 1 -- 172.21.15.7:0/1038610012 shutdown_connections 2024-01-27T02:32:32.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.305+0000 7f70db227700 1 -- 172.21.15.7:0/1038610012 wait complete. 2024-01-27T02:32:32.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.306+0000 7f70db227700 1 Processor -- start 2024-01-27T02:32:32.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.306+0000 7f70db227700 1 -- start start 2024-01-27T02:32:32.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.307+0000 7f70db227700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70d406e330 0x7f70d40b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:32.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.307+0000 7f70db227700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70d40c9bb0 0x7f70d415b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:32.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.307+0000 7f70db227700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70d40c9f80 0x7f70d4154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:32.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.307+0000 7f70db227700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d415c900 con 0x7f70d40c9bb0 2024-01-27T02:32:32.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.307+0000 7f70d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70d40c9bb0 0x7f70d415b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:32.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.307+0000 7f70db227700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d415cba0 con 0x7f70d40c9f80 2024-01-27T02:32:32.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.307+0000 7f70db227700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d415ce80 con 0x7f70d406e330 2024-01-27T02:32:32.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.307+0000 7f70d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70d40c9bb0 0x7f70d415b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58002/0 (socket says 172.21.15.7:58002) 2024-01-27T02:32:32.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.307+0000 7f70d3fff700 1 -- 172.21.15.7:0/385755558 learned_addr learned my addr 172.21.15.7:0/385755558 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:32.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.307+0000 7f70d97c4700 1 --2- 172.21.15.7:0/385755558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70d40c9f80 0x7f70d4154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:32.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.307+0000 7f70d8fc3700 1 --2- 172.21.15.7:0/385755558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70d406e330 0x7f70d40b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:32.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.307+0000 7f70d3fff700 1 -- 172.21.15.7:0/385755558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70d406e330 msgr2=0x7f70d40b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:32.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.307+0000 7f70d3fff700 1 --2- 172.21.15.7:0/385755558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70d406e330 0x7f70d40b1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:32.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.307+0000 7f70d3fff700 1 -- 172.21.15.7:0/385755558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70d40c9f80 msgr2=0x7f70d4154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:32.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.307+0000 7f70d3fff700 1 --2- 172.21.15.7:0/385755558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70d40c9f80 0x7f70d4154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:32.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.307+0000 7f70d3fff700 1 -- 172.21.15.7:0/385755558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c0009420 con 0x7f70d40c9bb0 2024-01-27T02:32:32.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.307+0000 7f70d3fff700 1 --2- 172.21.15.7:0/385755558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70d40c9bb0 0x7f70d415b680 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f70c0013000 tx=0x7f70c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:32.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.308+0000 7f70d0ff9700 1 -- 172.21.15.7:0/385755558 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c000bb90 con 0x7f70d40c9bb0 2024-01-27T02:32:32.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.308+0000 7f70d0ff9700 1 -- 172.21.15.7:0/385755558 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c0011420 con 0x7f70d40c9bb0 2024-01-27T02:32:32.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.308+0000 7f70d0ff9700 1 -- 172.21.15.7:0/385755558 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c00117c0 con 0x7f70d40c9bb0 2024-01-27T02:32:32.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.308+0000 7f70db227700 1 -- 172.21.15.7:0/385755558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d415d160 con 0x7f70d40c9bb0 2024-01-27T02:32:32.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.308+0000 7f70db227700 1 -- 172.21.15.7:0/385755558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d4154d40 con 0x7f70d40c9bb0 2024-01-27T02:32:32.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.309+0000 7f70db227700 1 -- 172.21.15.7:0/385755558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d40c5cb0 con 0x7f70d40c9bb0 2024-01-27T02:32:32.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.309+0000 7f70d0ff9700 1 -- 172.21.15.7:0/385755558 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f70c0011bb0 con 0x7f70d40c9bb0 2024-01-27T02:32:32.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.310+0000 7f70d0ff9700 1 --2- 172.21.15.7:0/385755558 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f70b8066f30 0x7f70b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:32.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.310+0000 7f70d0ff9700 1 -- 172.21.15.7:0/385755558 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f70c0085dd0 con 0x7f70d40c9bb0 2024-01-27T02:32:32.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.310+0000 7f70d8fc3700 1 --2- 172.21.15.7:0/385755558 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f70b8066f30 0x7f70b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:32.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.310+0000 7f70d8fc3700 1 --2- 172.21.15.7:0/385755558 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f70b8066f30 0x7f70b80693e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f70c8006fd0 tx=0x7f70c8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:32.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.314+0000 7f70d0ff9700 1 -- 172.21.15.7:0/385755558 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70c00530c0 con 0x7f70d40c9bb0 2024-01-27T02:32:32.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.521+0000 7f70db227700 1 -- 172.21.15.7:0/385755558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70d404c710 con 0x7f70d40c9bb0 2024-01-27T02:32:32.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.522+0000 7f70d0ff9700 1 -- 172.21.15.7:0/385755558 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f70c0011e60 con 0x7f70d40c9bb0 2024-01-27T02:32:32.522 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:32.522 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:32:07.627822+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:32:03.675611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-27T02:32:05.018885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:32:32.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.524+0000 7f70db227700 1 -- 172.21.15.7:0/385755558 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f70b8066f30 msgr2=0x7f70b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:32.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.524+0000 7f70db227700 1 --2- 172.21.15.7:0/385755558 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f70b8066f30 0x7f70b80693e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f70c8006fd0 tx=0x7f70c8008040 comp rx=0 tx=0).stop 2024-01-27T02:32:32.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.524+0000 7f70db227700 1 -- 172.21.15.7:0/385755558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70d40c9bb0 msgr2=0x7f70d415b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:32.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.524+0000 7f70db227700 1 --2- 172.21.15.7:0/385755558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70d40c9bb0 0x7f70d415b680 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f70c0013000 tx=0x7f70c0017000 comp rx=0 tx=0).stop 2024-01-27T02:32:32.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.524+0000 7f70db227700 1 -- 172.21.15.7:0/385755558 shutdown_connections 2024-01-27T02:32:32.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.524+0000 7f70db227700 1 --2- 172.21.15.7:0/385755558 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f70b8066f30 0x7f70b80693e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:32.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.524+0000 7f70db227700 1 --2- 172.21.15.7:0/385755558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70d40c9f80 0x7f70d4154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:32.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.524+0000 7f70db227700 1 --2- 172.21.15.7:0/385755558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70d406e330 0x7f70d40b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:32.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.524+0000 7f70db227700 1 --2- 172.21.15.7:0/385755558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70d40c9bb0 0x7f70d415b680 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:32.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.524+0000 7f70db227700 1 -- 172.21.15.7:0/385755558 >> 172.21.15.7:0/385755558 conn(0x7f70d40cdc30 msgr2=0x7f70d4158540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:32.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.524+0000 7f70db227700 1 -- 172.21.15.7:0/385755558 shutdown_connections 2024-01-27T02:32:32.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.524+0000 7f70db227700 1 -- 172.21.15.7:0/385755558 wait complete. 2024-01-27T02:32:32.538 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-27T02:32:03.675611+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-27T02:32:05.018885+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-27T02:32:32.538 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:32:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.671+0000 7fc58bbc8700 1 Processor -- start 2024-01-27T02:32:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.671+0000 7fc58bbc8700 1 -- start start 2024-01-27T02:32:32.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.672+0000 7fc58bbc8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5840cbd20 0x7fc5840b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:32.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.672+0000 7fc58bbc8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc5841412d0 0x7fc584141730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:32.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.672+0000 7fc58bbc8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5840c9660 con 0x7fc5840cb950 2024-01-27T02:32:32.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.672+0000 7fc58bbc8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc58415c980 con 0x7fc5841412d0 2024-01-27T02:32:32.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.672+0000 7fc58bbc8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc584141c70 con 0x7fc5840cbd20 2024-01-27T02:32:32.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.672+0000 7fc58a165700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc5841412d0 0x7fc584141730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:32.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.672+0000 7fc589964700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc5840cb950 0x7fc5840ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45508/0 (socket says 172.21.15.7:45508) 2024-01-27T02:32:32.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.672+0000 7fc589964700 1 -- 172.21.15.7:0/2863920971 learned_addr learned my addr 172.21.15.7:0/2863920971 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:32.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.673+0000 7fc589163700 1 --2- 172.21.15.7:0/2863920971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5840cbd20 0x7fc5840b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:32.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.673+0000 7fc58a165700 1 -- 172.21.15.7:0/2863920971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5840cbd20 msgr2=0x7fc5840b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:32.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.673+0000 7fc58a165700 1 --2- 172.21.15.7:0/2863920971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5840cbd20 0x7fc5840b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:32.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.673+0000 7fc58a165700 1 -- 172.21.15.7:0/2863920971 >> v1:172.21.15.7:6789/0 conn(0x7fc5840cb950 legacy=0x7fc5840ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:32.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.673+0000 7fc58a165700 1 -- 172.21.15.7:0/2863920971 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58415be00 con 0x7fc5841412d0 2024-01-27T02:32:32.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.673+0000 7fc58a165700 1 --2- 172.21.15.7:0/2863920971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5841412d0 0x7fc584141730 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009e60 tx=0x7fc57c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b85f5f0819386e7 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:32.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.673+0000 7fc588962700 1 -- 172.21.15.7:0/2863920971 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c00e070 con 0x7fc5841412d0 2024-01-27T02:32:32.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.673+0000 7fc588962700 1 -- 172.21.15.7:0/2863920971 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57c00bc30 con 0x7fc5841412d0 2024-01-27T02:32:32.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.673+0000 7fc588962700 1 -- 172.21.15.7:0/2863920971 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c012430 con 0x7fc5841412d0 2024-01-27T02:32:32.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.674+0000 7fc58bbc8700 1 -- 172.21.15.7:0/2863920971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5841412d0 msgr2=0x7fc584141730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:32.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.674+0000 7fc58bbc8700 1 --2- 172.21.15.7:0/2863920971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5841412d0 0x7fc584141730 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009e60 tx=0x7fc57c009780 comp rx=0 tx=0).stop 2024-01-27T02:32:32.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.674+0000 7fc58bbc8700 1 -- 172.21.15.7:0/2863920971 shutdown_connections 2024-01-27T02:32:32.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.674+0000 7fc58bbc8700 1 --2- 172.21.15.7:0/2863920971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5841412d0 0x7fc584141730 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:32.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.674+0000 7fc58bbc8700 1 --2- 172.21.15.7:0/2863920971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5840cbd20 0x7fc5840b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:32.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.674+0000 7fc58bbc8700 1 -- 172.21.15.7:0/2863920971 >> 172.21.15.7:0/2863920971 conn(0x7fc584063580 msgr2=0x7fc584063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:32.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.674+0000 7fc58bbc8700 1 -- 172.21.15.7:0/2863920971 shutdown_connections 2024-01-27T02:32:32.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.674+0000 7fc58bbc8700 1 -- 172.21.15.7:0/2863920971 wait complete. 2024-01-27T02:32:32.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.675+0000 7fc58bbc8700 1 Processor -- start 2024-01-27T02:32:32.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.675+0000 7fc58bbc8700 1 -- start start 2024-01-27T02:32:32.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.676+0000 7fc58bbc8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5840cb950 0x7fc58416af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:32.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.676+0000 7fc58bbc8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5840cbd20 0x7fc58416b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:32.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.676+0000 7fc58bbc8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5841412d0 0x7fc58416be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:32.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.676+0000 7fc58bbc8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58416aae0 con 0x7fc5840cbd20 2024-01-27T02:32:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.676+0000 7fc58bbc8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5840cf6c0 con 0x7fc5841412d0 2024-01-27T02:32:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.676+0000 7fc58bbc8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc584169ed0 con 0x7fc5840cb950 2024-01-27T02:32:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.676+0000 7fc589163700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5840cbd20 0x7fc58416b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.676+0000 7fc589163700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5840cbd20 0x7fc58416b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58020/0 (socket says 172.21.15.7:58020) 2024-01-27T02:32:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.676+0000 7fc589163700 1 -- 172.21.15.7:0/977304563 learned_addr learned my addr 172.21.15.7:0/977304563 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.676+0000 7fc589964700 1 --2- 172.21.15.7:0/977304563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5840cb950 0x7fc58416af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.676+0000 7fc58a165700 1 --2- 172.21.15.7:0/977304563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5841412d0 0x7fc58416be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.676+0000 7fc589163700 1 -- 172.21.15.7:0/977304563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5840cb950 msgr2=0x7fc58416af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.676+0000 7fc589163700 1 --2- 172.21.15.7:0/977304563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5840cb950 0x7fc58416af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.676+0000 7fc589163700 1 -- 172.21.15.7:0/977304563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5841412d0 msgr2=0x7fc58416be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.676+0000 7fc589163700 1 --2- 172.21.15.7:0/977304563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5841412d0 0x7fc58416be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.676+0000 7fc589163700 1 -- 172.21.15.7:0/977304563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57c009a20 con 0x7fc5840cbd20 2024-01-27T02:32:32.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.677+0000 7fc589163700 1 --2- 172.21.15.7:0/977304563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5840cbd20 0x7fc58416b900 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc57800d890 tx=0x7fc578008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:32.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.677+0000 7fc575ffb700 1 -- 172.21.15.7:0/977304563 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5780125f0 con 0x7fc5840cbd20 2024-01-27T02:32:32.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.677+0000 7fc575ffb700 1 -- 172.21.15.7:0/977304563 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc578012750 con 0x7fc5840cbd20 2024-01-27T02:32:32.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.677+0000 7fc58bbc8700 1 -- 172.21.15.7:0/977304563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc58416a1b0 con 0x7fc5840cbd20 2024-01-27T02:32:32.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.677+0000 7fc575ffb700 1 -- 172.21.15.7:0/977304563 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc578012af0 con 0x7fc5840cbd20 2024-01-27T02:32:32.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.677+0000 7fc58bbc8700 1 -- 172.21.15.7:0/977304563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58416a770 con 0x7fc5840cbd20 2024-01-27T02:32:32.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.678+0000 7fc58bbc8700 1 -- 172.21.15.7:0/977304563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc58416ac50 con 0x7fc5840cbd20 2024-01-27T02:32:32.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.678+0000 7fc575ffb700 1 -- 172.21.15.7:0/977304563 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc57801e460 con 0x7fc5840cbd20 2024-01-27T02:32:32.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.679+0000 7fc575ffb700 1 --2- 172.21.15.7:0/977304563 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc568066f30 0x7fc5680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:32.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.679+0000 7fc575ffb700 1 -- 172.21.15.7:0/977304563 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc578084ad0 con 0x7fc5840cbd20 2024-01-27T02:32:32.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.679+0000 7fc589964700 1 --2- 172.21.15.7:0/977304563 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc568066f30 0x7fc5680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:32.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.680+0000 7fc589964700 1 --2- 172.21.15.7:0/977304563 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc568066f30 0x7fc5680693e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc580001950 tx=0x7fc580009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:32.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.683+0000 7fc575ffb700 1 -- 172.21.15.7:0/977304563 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc578051d40 con 0x7fc5840cbd20 2024-01-27T02:32:32.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.887+0000 7fc58bbc8700 1 -- 172.21.15.7:0/977304563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc584072060 con 0x7fc5840cbd20 2024-01-27T02:32:32.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.887+0000 7fc575ffb700 1 -- 172.21.15.7:0/977304563 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc578055360 con 0x7fc5840cbd20 2024-01-27T02:32:32.888 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:32:32.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.889+0000 7fc58bbc8700 1 -- 172.21.15.7:0/977304563 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc568066f30 msgr2=0x7fc5680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:32.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.889+0000 7fc58bbc8700 1 --2- 172.21.15.7:0/977304563 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc568066f30 0x7fc5680693e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc580001950 tx=0x7fc580009040 comp rx=0 tx=0).stop 2024-01-27T02:32:32.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.890+0000 7fc58bbc8700 1 -- 172.21.15.7:0/977304563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5840cbd20 msgr2=0x7fc58416b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:32.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.890+0000 7fc58bbc8700 1 --2- 172.21.15.7:0/977304563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5840cbd20 0x7fc58416b900 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc57800d890 tx=0x7fc578008040 comp rx=0 tx=0).stop 2024-01-27T02:32:32.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.890+0000 7fc58bbc8700 1 -- 172.21.15.7:0/977304563 shutdown_connections 2024-01-27T02:32:32.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.890+0000 7fc58bbc8700 1 --2- 172.21.15.7:0/977304563 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc568066f30 0x7fc5680693e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:32.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.890+0000 7fc58bbc8700 1 --2- 172.21.15.7:0/977304563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5841412d0 0x7fc58416be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:32.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.890+0000 7fc58bbc8700 1 --2- 172.21.15.7:0/977304563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5840cb950 0x7fc58416af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:32.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.890+0000 7fc58bbc8700 1 --2- 172.21.15.7:0/977304563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5840cbd20 0x7fc58416b900 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:32.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.890+0000 7fc58bbc8700 1 -- 172.21.15.7:0/977304563 >> 172.21.15.7:0/977304563 conn(0x7fc584063580 msgr2=0x7fc5840c3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:32.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.890+0000 7fc58bbc8700 1 -- 172.21.15.7:0/977304563 shutdown_connections 2024-01-27T02:32:32.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:32.890+0000 7fc58bbc8700 1 -- 172.21.15.7:0/977304563 wait complete. 2024-01-27T02:32:32.903 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-27T02:32:33.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.038+0000 7f106932c700 1 Processor -- start 2024-01-27T02:32:33.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.038+0000 7f106932c700 1 -- start start 2024-01-27T02:32:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.038+0000 7f106932c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f10640cbe30 0x7f10640ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.038+0000 7f106932c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f106406b4e0 0x7f106406b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.038+0000 7f106932c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10640679b0 con 0x7f10640cba60 2024-01-27T02:32:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.039+0000 7f106932c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1064067c90 con 0x7f10640cbe30 2024-01-27T02:32:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.039+0000 7f106932c700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f106406be20 con 0x7f106406b4e0 2024-01-27T02:32:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.039+0000 7f10627fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f10640cbe30 0x7f10640ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.039+0000 7f1062ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f10640cba60 0x7f106415b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45514/0 (socket says 172.21.15.7:45514) 2024-01-27T02:32:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.039+0000 7f1062ffd700 1 -- 172.21.15.7:0/3504642208 learned_addr learned my addr 172.21.15.7:0/3504642208 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.039+0000 7f10637fe700 1 --2- 172.21.15.7:0/3504642208 >> v2:172.21.15.188:3300/0 conn(0x7f106406b4e0 0x7f106406b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:33.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.039+0000 7f1061ffb700 1 -- 172.21.15.7:0/3504642208 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 954626478 0 0) 0x7f10640679b0 con 0x7f10640cba60 2024-01-27T02:32:33.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.039+0000 7f10627fc700 1 -- 172.21.15.7:0/3504642208 >> v2:172.21.15.188:3300/0 conn(0x7f106406b4e0 msgr2=0x7f106406b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:33.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.039+0000 7f10627fc700 1 --2- 172.21.15.7:0/3504642208 >> v2:172.21.15.188:3300/0 conn(0x7f106406b4e0 0x7f106406b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.039+0000 7f10627fc700 1 -- 172.21.15.7:0/3504642208 >> v1:172.21.15.7:6789/0 conn(0x7f10640cba60 legacy=0x7f106415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:33.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.039+0000 7f10627fc700 1 -- 172.21.15.7:0/3504642208 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10640cca50 con 0x7f10640cbe30 2024-01-27T02:32:33.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.039+0000 7f10627fc700 1 --2- 172.21.15.7:0/3504642208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10640cbe30 0x7f10640ce410 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f10640b1800 tx=0x7f104c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db997c41050d30a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:33.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.039+0000 7f1061ffb700 1 -- 172.21.15.7:0/3504642208 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c011630 con 0x7f10640cbe30 2024-01-27T02:32:33.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.039+0000 7f1061ffb700 1 -- 172.21.15.7:0/3504642208 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104c011790 con 0x7f10640cbe30 2024-01-27T02:32:33.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.039+0000 7f1061ffb700 1 -- 172.21.15.7:0/3504642208 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c011b30 con 0x7f10640cbe30 2024-01-27T02:32:33.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.040+0000 7f106932c700 1 -- 172.21.15.7:0/3504642208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10640cbe30 msgr2=0x7f10640ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:33.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.040+0000 7f106932c700 1 --2- 172.21.15.7:0/3504642208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10640cbe30 0x7f10640ce410 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f10640b1800 tx=0x7f104c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.040+0000 7f106932c700 1 -- 172.21.15.7:0/3504642208 shutdown_connections 2024-01-27T02:32:33.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.040+0000 7f106932c700 1 --2- 172.21.15.7:0/3504642208 >> v2:172.21.15.188:3300/0 conn(0x7f106406b4e0 0x7f106406b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.040+0000 7f106932c700 1 --2- 172.21.15.7:0/3504642208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10640cbe30 0x7f10640ce410 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.040+0000 7f106932c700 1 -- 172.21.15.7:0/3504642208 >> 172.21.15.7:0/3504642208 conn(0x7f1064063580 msgr2=0x7f1064063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:33.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.040+0000 7f106932c700 1 -- 172.21.15.7:0/3504642208 shutdown_connections 2024-01-27T02:32:33.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.040+0000 7f106932c700 1 -- 172.21.15.7:0/3504642208 wait complete. 2024-01-27T02:32:33.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.041+0000 7f106932c700 1 Processor -- start 2024-01-27T02:32:33.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.042+0000 7f106932c700 1 -- start start 2024-01-27T02:32:33.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.042+0000 7f106932c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f106406b4e0 0x7f10640b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:33.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.042+0000 7f106932c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10640cba60 0x7f10640ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:33.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.042+0000 7f106932c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10640cbe30 0x7f10640ab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:33.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.042+0000 7f106932c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10640ac250 con 0x7f106406b4e0 2024-01-27T02:32:33.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.042+0000 7f106932c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10640b2d40 con 0x7f10640cba60 2024-01-27T02:32:33.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.042+0000 7f106932c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10640b3020 con 0x7f10640cbe30 2024-01-27T02:32:33.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.042+0000 7f1062ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f106406b4e0 0x7f10640b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:33.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.042+0000 7f1062ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f106406b4e0 0x7f10640b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58048/0 (socket says 172.21.15.7:58048) 2024-01-27T02:32:33.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.042+0000 7f1062ffd700 1 -- 172.21.15.7:0/1523231804 learned_addr learned my addr 172.21.15.7:0/1523231804 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:33.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.042+0000 7f10627fc700 1 --2- 172.21.15.7:0/1523231804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10640cba60 0x7f10640ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:33.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.043+0000 7f10637fe700 1 --2- 172.21.15.7:0/1523231804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10640cbe30 0x7f10640ab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:33.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.043+0000 7f1062ffd700 1 -- 172.21.15.7:0/1523231804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10640cbe30 msgr2=0x7f10640ab2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:33.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.043+0000 7f1062ffd700 1 --2- 172.21.15.7:0/1523231804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10640cbe30 0x7f10640ab2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.043+0000 7f1062ffd700 1 -- 172.21.15.7:0/1523231804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10640cba60 msgr2=0x7f10640ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:33.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.043+0000 7f1062ffd700 1 --2- 172.21.15.7:0/1523231804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10640cba60 0x7f10640ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.043+0000 7f1062ffd700 1 -- 172.21.15.7:0/1523231804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104c009420 con 0x7f106406b4e0 2024-01-27T02:32:33.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.043+0000 7f1062ffd700 1 --2- 172.21.15.7:0/1523231804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f106406b4e0 0x7f10640b2760 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f105400c7e0 tx=0x7f1054007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:33.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.043+0000 7f105affd700 1 -- 172.21.15.7:0/1523231804 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10540072e0 con 0x7f106406b4e0 2024-01-27T02:32:33.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.043+0000 7f105affd700 1 -- 172.21.15.7:0/1523231804 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1054007460 con 0x7f106406b4e0 2024-01-27T02:32:33.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.043+0000 7f106932c700 1 -- 172.21.15.7:0/1523231804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10640b3300 con 0x7f106406b4e0 2024-01-27T02:32:33.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.043+0000 7f105affd700 1 -- 172.21.15.7:0/1523231804 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10540057f0 con 0x7f106406b4e0 2024-01-27T02:32:33.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.043+0000 7f106932c700 1 -- 172.21.15.7:0/1523231804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10640ab820 con 0x7f106406b4e0 2024-01-27T02:32:33.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.044+0000 7f106932c700 1 -- 172.21.15.7:0/1523231804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10640ac750 con 0x7f106406b4e0 2024-01-27T02:32:33.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.045+0000 7f105affd700 1 -- 172.21.15.7:0/1523231804 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1054005b60 con 0x7f106406b4e0 2024-01-27T02:32:33.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.045+0000 7f105affd700 1 --2- 172.21.15.7:0/1523231804 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1044066f30 0x7f10440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:33.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.045+0000 7f105affd700 1 -- 172.21.15.7:0/1523231804 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1054001e40 con 0x7f106406b4e0 2024-01-27T02:32:33.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.046+0000 7f10627fc700 1 --2- 172.21.15.7:0/1523231804 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1044066f30 0x7f10440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:33.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.046+0000 7f10627fc700 1 --2- 172.21.15.7:0/1523231804 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1044066f30 0x7f10440693e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f10640cc260 tx=0x7f104c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:33.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.050+0000 7f105affd700 1 -- 172.21.15.7:0/1523231804 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f105404b5a0 con 0x7f106406b4e0 2024-01-27T02:32:33.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.252+0000 7f106932c700 1 -- 172.21.15.7:0/1523231804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f10640b34f0 con 0x7f106406b4e0 2024-01-27T02:32:33.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.253+0000 7f105affd700 1 -- 172.21.15.7:0/1523231804 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f10640ac750 con 0x7f106406b4e0 2024-01-27T02:32:33.253 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-27T02:32:33.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.255+0000 7f106932c700 1 -- 172.21.15.7:0/1523231804 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1044066f30 msgr2=0x7f10440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:33.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.255+0000 7f106932c700 1 --2- 172.21.15.7:0/1523231804 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1044066f30 0x7f10440693e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f10640cc260 tx=0x7f104c017000 comp rx=0 tx=0).stop 2024-01-27T02:32:33.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.255+0000 7f106932c700 1 -- 172.21.15.7:0/1523231804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f106406b4e0 msgr2=0x7f10640b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:33.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.255+0000 7f106932c700 1 --2- 172.21.15.7:0/1523231804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f106406b4e0 0x7f10640b2760 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f105400c7e0 tx=0x7f1054007c80 comp rx=0 tx=0).stop 2024-01-27T02:32:33.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.255+0000 7f106932c700 1 -- 172.21.15.7:0/1523231804 shutdown_connections 2024-01-27T02:32:33.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.255+0000 7f106932c700 1 --2- 172.21.15.7:0/1523231804 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1044066f30 0x7f10440693e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.255+0000 7f106932c700 1 --2- 172.21.15.7:0/1523231804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10640cbe30 0x7f10640ab2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.255+0000 7f106932c700 1 --2- 172.21.15.7:0/1523231804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f106406b4e0 0x7f10640b2760 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.255+0000 7f106932c700 1 --2- 172.21.15.7:0/1523231804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10640cba60 0x7f10640ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.255+0000 7f106932c700 1 -- 172.21.15.7:0/1523231804 >> 172.21.15.7:0/1523231804 conn(0x7f1064063580 msgr2=0x7f1064068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:33.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.255+0000 7f106932c700 1 -- 172.21.15.7:0/1523231804 shutdown_connections 2024-01-27T02:32:33.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.255+0000 7f106932c700 1 -- 172.21.15.7:0/1523231804 wait complete. 2024-01-27T02:32:33.269 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-27T02:32:33.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.402+0000 7f15cf5b8700 1 Processor -- start 2024-01-27T02:32:33.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.402+0000 7f15cf5b8700 1 -- start start 2024-01-27T02:32:33.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.403+0000 7f15cf5b8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f15c80cb950 0x7f15c80b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:33.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.403+0000 7f15cf5b8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15c80c9660 con 0x7f15c81412d0 2024-01-27T02:32:33.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.403+0000 7f15cf5b8700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15c815be00 con 0x7f15c80cbd20 2024-01-27T02:32:33.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.403+0000 7f15cf5b8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15c815c980 con 0x7f15c80cb950 2024-01-27T02:32:33.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.403+0000 7f15cd354700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f15c80cb950 0x7f15c80b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:33.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.403+0000 7f15cdb55700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f15c81412d0 0x7f15c8141730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45524/0 (socket says 172.21.15.7:45524) 2024-01-27T02:32:33.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.404+0000 7f15cdb55700 1 -- 172.21.15.7:0/1033906399 learned_addr learned my addr 172.21.15.7:0/1033906399 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:33.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.404+0000 7f15bbfff700 1 -- 172.21.15.7:0/1033906399 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 811918810 0 0) 0x7f15c815be00 con 0x7f15c80cbd20 2024-01-27T02:32:33.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.404+0000 7f15cd354700 1 -- 172.21.15.7:0/1033906399 >> v1:172.21.15.188:6789/0 conn(0x7f15c80cbd20 legacy=0x7f15c80ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:33.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.404+0000 7f15cd354700 1 -- 172.21.15.7:0/1033906399 >> v1:172.21.15.7:6789/0 conn(0x7f15c81412d0 legacy=0x7f15c8141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:33.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.404+0000 7f15cd354700 1 -- 172.21.15.7:0/1033906399 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c815c0e0 con 0x7f15c80cb950 2024-01-27T02:32:33.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.404+0000 7f15cd354700 1 --2- 172.21.15.7:0/1033906399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15c80cb950 0x7f15c80b1ac0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f15bc0098e0 tx=0x7f15bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91dc0871c1ef281d server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:33.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.404+0000 7f15bbfff700 1 -- 172.21.15.7:0/1033906399 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc009e80 con 0x7f15c80cb950 2024-01-27T02:32:33.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.404+0000 7f15bbfff700 1 -- 172.21.15.7:0/1033906399 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15bc00b760 con 0x7f15c80cb950 2024-01-27T02:32:33.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.404+0000 7f15bbfff700 1 -- 172.21.15.7:0/1033906399 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc00bae0 con 0x7f15c80cb950 2024-01-27T02:32:33.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.405+0000 7f15cf5b8700 1 -- 172.21.15.7:0/1033906399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15c80cb950 msgr2=0x7f15c80b1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:33.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.405+0000 7f15cf5b8700 1 --2- 172.21.15.7:0/1033906399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15c80cb950 0x7f15c80b1ac0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f15bc0098e0 tx=0x7f15bc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.405+0000 7f15cf5b8700 1 -- 172.21.15.7:0/1033906399 shutdown_connections 2024-01-27T02:32:33.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.405+0000 7f15cf5b8700 1 --2- 172.21.15.7:0/1033906399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15c80cb950 0x7f15c80b1ac0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.405+0000 7f15cf5b8700 1 -- 172.21.15.7:0/1033906399 >> 172.21.15.7:0/1033906399 conn(0x7f15c8063580 msgr2=0x7f15c8063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:33.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.405+0000 7f15cf5b8700 1 -- 172.21.15.7:0/1033906399 shutdown_connections 2024-01-27T02:32:33.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.405+0000 7f15cf5b8700 1 -- 172.21.15.7:0/1033906399 wait complete. 2024-01-27T02:32:33.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.407+0000 7f15cf5b8700 1 Processor -- start 2024-01-27T02:32:33.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.407+0000 7f15cf5b8700 1 -- start start 2024-01-27T02:32:33.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.408+0000 7f15cf5b8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15c80cb950 0x7f15c80b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:33.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.408+0000 7f15cf5b8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15c80cbd20 0x7f15c80c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:33.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.408+0000 7f15cf5b8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15c81412d0 0x7f15c80c4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:33.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.408+0000 7f15cf5b8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c80c7a60 con 0x7f15c80cb950 2024-01-27T02:32:33.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.408+0000 7f15cf5b8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c80c6e50 con 0x7f15c81412d0 2024-01-27T02:32:33.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.408+0000 7f15cf5b8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c80c7130 con 0x7f15c80cbd20 2024-01-27T02:32:33.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.408+0000 7f15cd354700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15c80cb950 0x7f15c80b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:33.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.408+0000 7f15cd354700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15c80cb950 0x7f15c80b0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58068/0 (socket says 172.21.15.7:58068) 2024-01-27T02:32:33.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.408+0000 7f15cd354700 1 -- 172.21.15.7:0/4204504486 learned_addr learned my addr 172.21.15.7:0/4204504486 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:33.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.408+0000 7f15cdb55700 1 --2- 172.21.15.7:0/4204504486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15c81412d0 0x7f15c80c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:33.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.408+0000 7f15ccb53700 1 --2- 172.21.15.7:0/4204504486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15c80cbd20 0x7f15c80c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:33.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.408+0000 7f15cd354700 1 -- 172.21.15.7:0/4204504486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15c80cbd20 msgr2=0x7f15c80c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:33.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.408+0000 7f15cd354700 1 --2- 172.21.15.7:0/4204504486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15c80cbd20 0x7f15c80c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.408+0000 7f15cd354700 1 -- 172.21.15.7:0/4204504486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15c81412d0 msgr2=0x7f15c80c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.408+0000 7f15cd354700 1 --2- 172.21.15.7:0/4204504486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15c81412d0 0x7f15c80c4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.408+0000 7f15cd354700 1 -- 172.21.15.7:0/4204504486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc009420 con 0x7f15c80cb950 2024-01-27T02:32:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.408+0000 7f15cdb55700 1 --2- 172.21.15.7:0/4204504486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15c81412d0 0x7f15c80c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.409+0000 7f15cd354700 1 --2- 172.21.15.7:0/4204504486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15c80cb950 0x7f15c80b0d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f15bc0030c0 tx=0x7f15bc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.409+0000 7f15b97fa700 1 -- 172.21.15.7:0/4204504486 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc00bae0 con 0x7f15c80cb950 2024-01-27T02:32:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.409+0000 7f15b97fa700 1 -- 172.21.15.7:0/4204504486 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15bc01c460 con 0x7f15c80cb950 2024-01-27T02:32:33.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.409+0000 7f15b97fa700 1 -- 172.21.15.7:0/4204504486 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc01c820 con 0x7f15c80cb950 2024-01-27T02:32:33.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.409+0000 7f15cf5b8700 1 -- 172.21.15.7:0/4204504486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c80c7410 con 0x7f15c80cb950 2024-01-27T02:32:33.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.409+0000 7f15cf5b8700 1 -- 172.21.15.7:0/4204504486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c80b1280 con 0x7f15c80cb950 2024-01-27T02:32:33.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.410+0000 7f15cf5b8700 1 -- 172.21.15.7:0/4204504486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15c80c7d40 con 0x7f15c80cb950 2024-01-27T02:32:33.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.410+0000 7f15b97fa700 1 -- 172.21.15.7:0/4204504486 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f15bc01cbd0 con 0x7f15c80cb950 2024-01-27T02:32:33.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.411+0000 7f15b97fa700 1 --2- 172.21.15.7:0/4204504486 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f15ac066f30 0x7f15ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:33.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.411+0000 7f15b97fa700 1 -- 172.21.15.7:0/4204504486 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f15bc086980 con 0x7f15c80cb950 2024-01-27T02:32:33.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.411+0000 7f15ccb53700 1 --2- 172.21.15.7:0/4204504486 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f15ac066f30 0x7f15ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:33.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.411+0000 7f15ccb53700 1 --2- 172.21.15.7:0/4204504486 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f15ac066f30 0x7f15ac0693e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f15c812d170 tx=0x7f15c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:33.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.415+0000 7f15b97fa700 1 -- 172.21.15.7:0/4204504486 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f15bc053bf0 con 0x7f15c80cb950 2024-01-27T02:32:33.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.616+0000 7f15cf5b8700 1 -- 172.21.15.7:0/4204504486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f15c80c7d40 con 0x7f15c80cb950 2024-01-27T02:32:33.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.616+0000 7f15b97fa700 1 -- 172.21.15.7:0/4204504486 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f15bc057210 con 0x7f15c80cb950 2024-01-27T02:32:33.617 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-27T02:32:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.618+0000 7f15cf5b8700 1 -- 172.21.15.7:0/4204504486 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f15ac066f30 msgr2=0x7f15ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.618+0000 7f15cf5b8700 1 --2- 172.21.15.7:0/4204504486 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f15ac066f30 0x7f15ac0693e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f15c812d170 tx=0x7f15c4009040 comp rx=0 tx=0).stop 2024-01-27T02:32:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.618+0000 7f15cf5b8700 1 -- 172.21.15.7:0/4204504486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15c80cb950 msgr2=0x7f15c80b0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.618+0000 7f15cf5b8700 1 --2- 172.21.15.7:0/4204504486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15c80cb950 0x7f15c80b0d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f15bc0030c0 tx=0x7f15bc016000 comp rx=0 tx=0).stop 2024-01-27T02:32:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.619+0000 7f15cf5b8700 1 -- 172.21.15.7:0/4204504486 shutdown_connections 2024-01-27T02:32:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.619+0000 7f15cf5b8700 1 --2- 172.21.15.7:0/4204504486 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f15ac066f30 0x7f15ac0693e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.619+0000 7f15cf5b8700 1 --2- 172.21.15.7:0/4204504486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15c81412d0 0x7f15c80c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.619+0000 7f15cf5b8700 1 --2- 172.21.15.7:0/4204504486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15c80cb950 0x7f15c80b0d40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.619+0000 7f15cf5b8700 1 --2- 172.21.15.7:0/4204504486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15c80cbd20 0x7f15c80c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.619+0000 7f15cf5b8700 1 -- 172.21.15.7:0/4204504486 >> 172.21.15.7:0/4204504486 conn(0x7f15c8063580 msgr2=0x7f15c80cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.619+0000 7f15cf5b8700 1 -- 172.21.15.7:0/4204504486 shutdown_connections 2024-01-27T02:32:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.619+0000 7f15cf5b8700 1 -- 172.21.15.7:0/4204504486 wait complete. 2024-01-27T02:32:33.633 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:32:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.764+0000 7f1a64373700 1 Processor -- start 2024-01-27T02:32:33.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.764+0000 7f1a64373700 1 -- start start 2024-01-27T02:32:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.765+0000 7f1a64373700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1a5c157ca0 0x7f1a5c0c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.765+0000 7f1a64373700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a5c0cdbf0 0x7f1a5c0ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.765+0000 7f1a64373700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a5c0ca020 con 0x7f1a5c0c3ae0 2024-01-27T02:32:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.765+0000 7f1a64373700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a5c0c4f40 con 0x7f1a5c157ca0 2024-01-27T02:32:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.765+0000 7f1a64373700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0c5210 con 0x7f1a5c0cdbf0 2024-01-27T02:32:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.766+0000 7f1a6210f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1a5c157ca0 0x7f1a5c0c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.766+0000 7f1a62910700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a5c0cdbf0 0x7f1a5c0ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.766+0000 7f1a6190e700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f1a5c0c3ae0 0x7f1a5c0cd4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:53812/0 (socket says 172.21.15.7:53812) 2024-01-27T02:32:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.766+0000 7f1a6190e700 1 -- 172.21.15.7:0/3819191420 learned_addr learned my addr 172.21.15.7:0/3819191420 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.766+0000 7f1a62910700 1 -- 172.21.15.7:0/3819191420 >> v2:172.21.15.188:3300/0 conn(0x7f1a5c157ca0 msgr2=0x7f1a5c0c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:33.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.766+0000 7f1a62910700 1 --2- 172.21.15.7:0/3819191420 >> v2:172.21.15.188:3300/0 conn(0x7f1a5c157ca0 0x7f1a5c0c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.766+0000 7f1a62910700 1 -- 172.21.15.7:0/3819191420 >> v1:172.21.15.188:6789/0 conn(0x7f1a5c0c3ae0 legacy=0x7f1a5c0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:33.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.766+0000 7f1a62910700 1 -- 172.21.15.7:0/3819191420 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c0c54f0 con 0x7f1a5c0cdbf0 2024-01-27T02:32:33.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.766+0000 7f1a6110d700 1 -- 172.21.15.7:0/3819191420 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 933584255 0 0) 0x7f1a5c0ca020 con 0x7f1a5c0c3ae0 2024-01-27T02:32:33.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.766+0000 7f1a6210f700 1 --2- 172.21.15.7:0/3819191420 >> v2:172.21.15.188:3300/0 conn(0x7f1a5c157ca0 0x7f1a5c0c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:32:33.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.766+0000 7f1a62910700 1 --2- 172.21.15.7:0/3819191420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a5c0cdbf0 0x7f1a5c0ca4b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009e60 tx=0x7f1a54009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3df0ee0f317781d1 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:33.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.766+0000 7f1a6110d700 1 -- 172.21.15.7:0/3819191420 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5400e070 con 0x7f1a5c0cdbf0 2024-01-27T02:32:33.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.767+0000 7f1a6110d700 1 -- 172.21.15.7:0/3819191420 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5400bb60 con 0x7f1a5c0cdbf0 2024-01-27T02:32:33.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.767+0000 7f1a6110d700 1 -- 172.21.15.7:0/3819191420 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a54012430 con 0x7f1a5c0cdbf0 2024-01-27T02:32:33.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.767+0000 7f1a64373700 1 -- 172.21.15.7:0/3819191420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a5c0cdbf0 msgr2=0x7f1a5c0ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:33.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.767+0000 7f1a64373700 1 --2- 172.21.15.7:0/3819191420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a5c0cdbf0 0x7f1a5c0ca4b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009e60 tx=0x7f1a54009780 comp rx=0 tx=0).stop 2024-01-27T02:32:33.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.767+0000 7f1a64373700 1 -- 172.21.15.7:0/3819191420 shutdown_connections 2024-01-27T02:32:33.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.767+0000 7f1a64373700 1 --2- 172.21.15.7:0/3819191420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a5c0cdbf0 0x7f1a5c0ca4b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.767+0000 7f1a64373700 1 --2- 172.21.15.7:0/3819191420 >> v2:172.21.15.188:3300/0 conn(0x7f1a5c157ca0 0x7f1a5c0c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.767+0000 7f1a64373700 1 -- 172.21.15.7:0/3819191420 >> 172.21.15.7:0/3819191420 conn(0x7f1a5c098020 msgr2=0x7f1a5c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:33.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.767+0000 7f1a64373700 1 -- 172.21.15.7:0/3819191420 shutdown_connections 2024-01-27T02:32:33.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.767+0000 7f1a64373700 1 -- 172.21.15.7:0/3819191420 wait complete. 2024-01-27T02:32:33.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.769+0000 7f1a64373700 1 Processor -- start 2024-01-27T02:32:33.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.769+0000 7f1a64373700 1 -- start start 2024-01-27T02:32:33.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.770+0000 7f1a64373700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c0c3ae0 0x7f1a5c0c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:33.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.770+0000 7f1a64373700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a5c0cdbf0 0x7f1a5c150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:33.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.770+0000 7f1a64373700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a5c157ca0 0x7f1a5c167fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:33.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.770+0000 7f1a64373700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0c8c90 con 0x7f1a5c0c3ae0 2024-01-27T02:32:33.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.770+0000 7f1a64373700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0c6d70 con 0x7f1a5c0cdbf0 2024-01-27T02:32:33.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.770+0000 7f1a64373700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0c7050 con 0x7f1a5c157ca0 2024-01-27T02:32:33.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.770+0000 7f1a6210f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c0c3ae0 0x7f1a5c0c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:33.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.770+0000 7f1a6210f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c0c3ae0 0x7f1a5c0c8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58070/0 (socket says 172.21.15.7:58070) 2024-01-27T02:32:33.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.770+0000 7f1a6210f700 1 -- 172.21.15.7:0/2689270214 learned_addr learned my addr 172.21.15.7:0/2689270214 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:33.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.770+0000 7f1a6190e700 1 --2- 172.21.15.7:0/2689270214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a5c0cdbf0 0x7f1a5c150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:33.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.770+0000 7f1a62910700 1 --2- 172.21.15.7:0/2689270214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a5c157ca0 0x7f1a5c167fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:33.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.770+0000 7f1a6210f700 1 -- 172.21.15.7:0/2689270214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a5c157ca0 msgr2=0x7f1a5c167fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:33.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.770+0000 7f1a6210f700 1 --2- 172.21.15.7:0/2689270214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a5c157ca0 0x7f1a5c167fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.770+0000 7f1a6210f700 1 -- 172.21.15.7:0/2689270214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a5c0cdbf0 msgr2=0x7f1a5c150400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:33.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.770+0000 7f1a6210f700 1 --2- 172.21.15.7:0/2689270214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a5c0cdbf0 0x7f1a5c150400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.770+0000 7f1a6210f700 1 -- 172.21.15.7:0/2689270214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a58009420 con 0x7f1a5c0c3ae0 2024-01-27T02:32:33.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.771+0000 7f1a6210f700 1 --2- 172.21.15.7:0/2689270214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c0c3ae0 0x7f1a5c0c8750 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800ec80 tx=0x7f1a580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:33.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.771+0000 7f1a527fc700 1 -- 172.21.15.7:0/2689270214 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5800c9d0 con 0x7f1a5c0c3ae0 2024-01-27T02:32:33.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.771+0000 7f1a527fc700 1 -- 172.21.15.7:0/2689270214 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5800cb30 con 0x7f1a5c0c3ae0 2024-01-27T02:32:33.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.771+0000 7f1a64373700 1 -- 172.21.15.7:0/2689270214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a54009a20 con 0x7f1a5c0c3ae0 2024-01-27T02:32:33.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.771+0000 7f1a527fc700 1 -- 172.21.15.7:0/2689270214 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5800ceb0 con 0x7f1a5c0c3ae0 2024-01-27T02:32:33.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.771+0000 7f1a64373700 1 -- 172.21.15.7:0/2689270214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5c0c7610 con 0x7f1a5c0c3ae0 2024-01-27T02:32:33.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.772+0000 7f1a64373700 1 -- 172.21.15.7:0/2689270214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a5c04c710 con 0x7f1a5c0c3ae0 2024-01-27T02:32:33.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.772+0000 7f1a527fc700 1 -- 172.21.15.7:0/2689270214 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1a58013680 con 0x7f1a5c0c3ae0 2024-01-27T02:32:33.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.773+0000 7f1a527fc700 1 --2- 172.21.15.7:0/2689270214 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a40066f30 0x7f1a400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:33.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.773+0000 7f1a527fc700 1 -- 172.21.15.7:0/2689270214 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1a580848d0 con 0x7f1a5c0c3ae0 2024-01-27T02:32:33.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.773+0000 7f1a6190e700 1 --2- 172.21.15.7:0/2689270214 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a40066f30 0x7f1a400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:33.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.774+0000 7f1a6190e700 1 --2- 172.21.15.7:0/2689270214 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a40066f30 0x7f1a400693e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c098f20 tx=0x7f1a4c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:33.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.777+0000 7f1a527fc700 1 -- 172.21.15.7:0/2689270214 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a58051b40 con 0x7f1a5c0c3ae0 2024-01-27T02:32:33.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.988+0000 7f1a64373700 1 -- 172.21.15.7:0/2689270214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a5c0c57d0 con 0x7f1a5c0c3ae0 2024-01-27T02:32:33.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.989+0000 7f1a527fc700 1 -- 172.21.15.7:0/2689270214 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f1a58055160 con 0x7f1a5c0c3ae0 2024-01-27T02:32:33.989 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:33.990 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:32:07.627822+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:32:03.675611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-27T02:32:05.018885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:32:33.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.991+0000 7f1a64373700 1 -- 172.21.15.7:0/2689270214 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a40066f30 msgr2=0x7f1a400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:33.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.991+0000 7f1a64373700 1 --2- 172.21.15.7:0/2689270214 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a40066f30 0x7f1a400693e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c098f20 tx=0x7f1a4c009040 comp rx=0 tx=0).stop 2024-01-27T02:32:33.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.991+0000 7f1a64373700 1 -- 172.21.15.7:0/2689270214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c0c3ae0 msgr2=0x7f1a5c0c8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:33.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.991+0000 7f1a64373700 1 --2- 172.21.15.7:0/2689270214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c0c3ae0 0x7f1a5c0c8750 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800ec80 tx=0x7f1a580091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.991+0000 7f1a64373700 1 -- 172.21.15.7:0/2689270214 shutdown_connections 2024-01-27T02:32:33.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.991+0000 7f1a64373700 1 --2- 172.21.15.7:0/2689270214 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a40066f30 0x7f1a400693e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.991+0000 7f1a64373700 1 --2- 172.21.15.7:0/2689270214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a5c157ca0 0x7f1a5c167fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.991+0000 7f1a64373700 1 --2- 172.21.15.7:0/2689270214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c0c3ae0 0x7f1a5c0c8750 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.991+0000 7f1a64373700 1 --2- 172.21.15.7:0/2689270214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a5c0cdbf0 0x7f1a5c150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:33.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.991+0000 7f1a64373700 1 -- 172.21.15.7:0/2689270214 >> 172.21.15.7:0/2689270214 conn(0x7f1a5c098020 msgr2=0x7f1a5c0ca4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:33.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.992+0000 7f1a64373700 1 -- 172.21.15.7:0/2689270214 shutdown_connections 2024-01-27T02:32:33.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:33.992+0000 7f1a64373700 1 -- 172.21.15.7:0/2689270214 wait complete. 2024-01-27T02:32:34.006 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-27T02:32:03.675611+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-27T02:32:05.018885+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-27T02:32:34.006 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:32:34.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.140+0000 7f320def7700 1 Processor -- start 2024-01-27T02:32:34.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.141+0000 7f320def7700 1 -- start start 2024-01-27T02:32:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.141+0000 7f320def7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3208154550 0x7f32080ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.141+0000 7f320def7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32080ad7c0 0x7f3208063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.141+0000 7f320def7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32080adec0 con 0x7f3208154920 2024-01-27T02:32:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.141+0000 7f320def7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32080ae1a0 con 0x7f3208154550 2024-01-27T02:32:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.141+0000 7f320def7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320815a850 con 0x7f32080ad7c0 2024-01-27T02:32:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.142+0000 7f32077fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3208154550 0x7f32080ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.142+0000 7f3206ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3208154920 0x7f320815b050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45536/0 (socket says 172.21.15.7:45536) 2024-01-27T02:32:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.142+0000 7f3206ffd700 1 -- 172.21.15.7:0/3414540044 learned_addr learned my addr 172.21.15.7:0/3414540044 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.142+0000 7f3207fff700 1 --2- 172.21.15.7:0/3414540044 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32080ad7c0 0x7f3208063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.142+0000 7f32067fc700 1 -- 172.21.15.7:0/3414540044 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1515804862 0 0) 0x7f32080adec0 con 0x7f3208154920 2024-01-27T02:32:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.142+0000 7f32067fc700 1 -- 172.21.15.7:0/3414540044 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31f0003570 con 0x7f3208154920 2024-01-27T02:32:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.142+0000 7f32077fe700 1 -- 172.21.15.7:0/3414540044 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32080ad7c0 msgr2=0x7f3208063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.142+0000 7f32077fe700 1 --2- 172.21.15.7:0/3414540044 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32080ad7c0 0x7f3208063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.142+0000 7f32077fe700 1 -- 172.21.15.7:0/3414540044 >> v1:172.21.15.7:6789/0 conn(0x7f3208154920 legacy=0x7f320815b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.142+0000 7f32077fe700 1 -- 172.21.15.7:0/3414540044 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320815ab30 con 0x7f3208154550 2024-01-27T02:32:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.142+0000 7f32077fe700 1 --2- 172.21.15.7:0/3414540044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3208154550 0x7f32080ce6b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f31f80098e0 tx=0x7f31f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4a8b8fe6f97589 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.142+0000 7f32067fc700 1 -- 172.21.15.7:0/3414540044 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f8011630 con 0x7f3208154550 2024-01-27T02:32:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.142+0000 7f32067fc700 1 -- 172.21.15.7:0/3414540044 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f8011790 con 0x7f3208154550 2024-01-27T02:32:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.142+0000 7f32067fc700 1 -- 172.21.15.7:0/3414540044 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f8011b30 con 0x7f3208154550 2024-01-27T02:32:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.143+0000 7f320def7700 1 -- 172.21.15.7:0/3414540044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3208154550 msgr2=0x7f32080ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.143+0000 7f320def7700 1 --2- 172.21.15.7:0/3414540044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3208154550 0x7f32080ce6b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f31f80098e0 tx=0x7f31f80091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.143+0000 7f320def7700 1 -- 172.21.15.7:0/3414540044 shutdown_connections 2024-01-27T02:32:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.143+0000 7f320def7700 1 --2- 172.21.15.7:0/3414540044 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32080ad7c0 0x7f3208063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.143+0000 7f320def7700 1 --2- 172.21.15.7:0/3414540044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3208154550 0x7f32080ce6b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.143+0000 7f320def7700 1 -- 172.21.15.7:0/3414540044 >> 172.21.15.7:0/3414540044 conn(0x7f3208068480 msgr2=0x7f32080cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.143+0000 7f320def7700 1 -- 172.21.15.7:0/3414540044 shutdown_connections 2024-01-27T02:32:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.143+0000 7f320def7700 1 -- 172.21.15.7:0/3414540044 wait complete. 2024-01-27T02:32:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.144+0000 7f320def7700 1 Processor -- start 2024-01-27T02:32:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.144+0000 7f320def7700 1 -- start start 2024-01-27T02:32:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.145+0000 7f320def7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32080ad7c0 0x7f320816a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.145+0000 7f320def7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3208154550 0x7f320814a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.145+0000 7f320def7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3208154920 0x7f32080c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.145+0000 7f320def7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3208155ce0 con 0x7f32080ad7c0 2024-01-27T02:32:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.145+0000 7f320def7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3208156580 con 0x7f3208154920 2024-01-27T02:32:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.145+0000 7f320def7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32080c4760 con 0x7f3208154550 2024-01-27T02:32:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.145+0000 7f32077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32080ad7c0 0x7f320816a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.145+0000 7f32077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32080ad7c0 0x7f320816a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58084/0 (socket says 172.21.15.7:58084) 2024-01-27T02:32:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.145+0000 7f32077fe700 1 -- 172.21.15.7:0/450053010 learned_addr learned my addr 172.21.15.7:0/450053010 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.146+0000 7f3207fff700 1 --2- 172.21.15.7:0/450053010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3208154920 0x7f32080c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.146+0000 7f3206ffd700 1 --2- 172.21.15.7:0/450053010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3208154550 0x7f320814a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.146+0000 7f32077fe700 1 -- 172.21.15.7:0/450053010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3208154550 msgr2=0x7f320814a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.146+0000 7f32077fe700 1 --2- 172.21.15.7:0/450053010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3208154550 0x7f320814a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.146+0000 7f32077fe700 1 -- 172.21.15.7:0/450053010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3208154920 msgr2=0x7f32080c3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.146+0000 7f32077fe700 1 --2- 172.21.15.7:0/450053010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3208154920 0x7f32080c3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.146+0000 7f32077fe700 1 -- 172.21.15.7:0/450053010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f8009420 con 0x7f32080ad7c0 2024-01-27T02:32:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.146+0000 7f32077fe700 1 --2- 172.21.15.7:0/450053010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32080ad7c0 0x7f320816a630 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f32080700c0 tx=0x7f31f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.146+0000 7f31ef7fe700 1 -- 172.21.15.7:0/450053010 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f8011b10 con 0x7f32080ad7c0 2024-01-27T02:32:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.146+0000 7f31ef7fe700 1 -- 172.21.15.7:0/450053010 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f80253f0 con 0x7f32080ad7c0 2024-01-27T02:32:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.146+0000 7f320def7700 1 -- 172.21.15.7:0/450053010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3208155f30 con 0x7f32080ad7c0 2024-01-27T02:32:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.146+0000 7f31ef7fe700 1 -- 172.21.15.7:0/450053010 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f8025790 con 0x7f32080ad7c0 2024-01-27T02:32:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.146+0000 7f320def7700 1 -- 172.21.15.7:0/450053010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320816ab70 con 0x7f32080ad7c0 2024-01-27T02:32:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.147+0000 7f320def7700 1 -- 172.21.15.7:0/450053010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3208072060 con 0x7f32080ad7c0 2024-01-27T02:32:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.148+0000 7f31ef7fe700 1 -- 172.21.15.7:0/450053010 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f31f8025b80 con 0x7f32080ad7c0 2024-01-27T02:32:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.148+0000 7f31ef7fe700 1 --2- 172.21.15.7:0/450053010 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f31f4066f30 0x7f31f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.148+0000 7f31ef7fe700 1 -- 172.21.15.7:0/450053010 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f31f8085db0 con 0x7f32080ad7c0 2024-01-27T02:32:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.148+0000 7f3206ffd700 1 --2- 172.21.15.7:0/450053010 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f31f4066f30 0x7f31f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.149+0000 7f3206ffd700 1 --2- 172.21.15.7:0/450053010 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f31f4066f30 0x7f31f40693e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3208157b70 tx=0x7f31fc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.152+0000 7f31ef7fe700 1 -- 172.21.15.7:0/450053010 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31f8053020 con 0x7f32080ad7c0 2024-01-27T02:32:34.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.354+0000 7f320def7700 1 -- 172.21.15.7:0/450053010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f32080a84c0 con 0x7f32080ad7c0 2024-01-27T02:32:34.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.355+0000 7f31ef7fe700 1 -- 172.21.15.7:0/450053010 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f31f8056640 con 0x7f32080ad7c0 2024-01-27T02:32:34.355 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:32:34.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.357+0000 7f320def7700 1 -- 172.21.15.7:0/450053010 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f31f4066f30 msgr2=0x7f31f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:34.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.357+0000 7f320def7700 1 --2- 172.21.15.7:0/450053010 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f31f4066f30 0x7f31f40693e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3208157b70 tx=0x7f31fc009040 comp rx=0 tx=0).stop 2024-01-27T02:32:34.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.357+0000 7f320def7700 1 -- 172.21.15.7:0/450053010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32080ad7c0 msgr2=0x7f320816a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:34.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.357+0000 7f320def7700 1 --2- 172.21.15.7:0/450053010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32080ad7c0 0x7f320816a630 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f32080700c0 tx=0x7f31f8017000 comp rx=0 tx=0).stop 2024-01-27T02:32:34.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.357+0000 7f320def7700 1 -- 172.21.15.7:0/450053010 shutdown_connections 2024-01-27T02:32:34.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.357+0000 7f320def7700 1 --2- 172.21.15.7:0/450053010 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f31f4066f30 0x7f31f40693e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.357+0000 7f320def7700 1 --2- 172.21.15.7:0/450053010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3208154920 0x7f32080c3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.357+0000 7f320def7700 1 --2- 172.21.15.7:0/450053010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32080ad7c0 0x7f320816a630 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.357+0000 7f320def7700 1 --2- 172.21.15.7:0/450053010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3208154550 0x7f320814a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.357+0000 7f320def7700 1 -- 172.21.15.7:0/450053010 >> 172.21.15.7:0/450053010 conn(0x7f3208068480 msgr2=0x7f320806ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:34.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.357+0000 7f320def7700 1 -- 172.21.15.7:0/450053010 shutdown_connections 2024-01-27T02:32:34.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.357+0000 7f320def7700 1 -- 172.21.15.7:0/450053010 wait complete. 2024-01-27T02:32:34.371 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-27T02:32:34.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.505+0000 7fce8b751700 1 Processor -- start 2024-01-27T02:32:34.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.505+0000 7fce8b751700 1 -- start start 2024-01-27T02:32:34.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.506+0000 7fce8b751700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fce840ce510 0x7fce84158050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:34.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.506+0000 7fce8b751700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce840b3510 con 0x7fce84063580 2024-01-27T02:32:34.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.506+0000 7fce8b751700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8406f8b0 con 0x7fce84157c80 2024-01-27T02:32:34.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.506+0000 7fce8b751700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce840b2900 con 0x7fce840ce510 2024-01-27T02:32:34.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.506+0000 7fce88cec700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fce840ce510 0x7fce84158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:34.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.506+0000 7fce89cee700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fce84063580 0x7fce840ad4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45548/0 (socket says 172.21.15.7:45548) 2024-01-27T02:32:34.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.506+0000 7fce89cee700 1 -- 172.21.15.7:0/1626042879 learned_addr learned my addr 172.21.15.7:0/1626042879 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:34.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.507+0000 7fce88cec700 1 -- 172.21.15.7:0/1626042879 >> v1:172.21.15.188:6789/0 conn(0x7fce84157c80 legacy=0x7fce840b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:34.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.507+0000 7fce88cec700 1 -- 172.21.15.7:0/1626042879 >> v1:172.21.15.7:6789/0 conn(0x7fce84063580 legacy=0x7fce840ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:34.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.507+0000 7fce88cec700 1 -- 172.21.15.7:0/1626042879 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce840b2be0 con 0x7fce840ce510 2024-01-27T02:32:34.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.507+0000 7fce7bfff700 1 -- 172.21.15.7:0/1626042879 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368845272 0 0) 0x7fce840b3510 con 0x7fce84063580 2024-01-27T02:32:34.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.507+0000 7fce88cec700 1 --2- 172.21.15.7:0/1626042879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce840ce510 0x7fce84158050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fce740098e0 tx=0x7fce740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939f4668ad1436ac server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:34.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.507+0000 7fce7bfff700 1 -- 172.21.15.7:0/1626042879 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce74009e80 con 0x7fce840ce510 2024-01-27T02:32:34.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.507+0000 7fce7bfff700 1 -- 172.21.15.7:0/1626042879 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7400b860 con 0x7fce840ce510 2024-01-27T02:32:34.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.507+0000 7fce7bfff700 1 -- 172.21.15.7:0/1626042879 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7400bbe0 con 0x7fce840ce510 2024-01-27T02:32:34.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.508+0000 7fce8b751700 1 -- 172.21.15.7:0/1626042879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce840ce510 msgr2=0x7fce84158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:34.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.508+0000 7fce8b751700 1 --2- 172.21.15.7:0/1626042879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce840ce510 0x7fce84158050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fce740098e0 tx=0x7fce740091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.508+0000 7fce8b751700 1 -- 172.21.15.7:0/1626042879 shutdown_connections 2024-01-27T02:32:34.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.508+0000 7fce8b751700 1 --2- 172.21.15.7:0/1626042879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce840ce510 0x7fce84158050 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.508+0000 7fce8b751700 1 -- 172.21.15.7:0/1626042879 >> 172.21.15.7:0/1626042879 conn(0x7fce84068480 msgr2=0x7fce840cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:34.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.508+0000 7fce8b751700 1 -- 172.21.15.7:0/1626042879 shutdown_connections 2024-01-27T02:32:34.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.508+0000 7fce8b751700 1 -- 172.21.15.7:0/1626042879 wait complete. 2024-01-27T02:32:34.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.509+0000 7fce8b751700 1 Processor -- start 2024-01-27T02:32:34.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.509+0000 7fce8b751700 1 -- start start 2024-01-27T02:32:34.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.510+0000 7fce8b751700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce84063580 0x7fce84149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:34.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.510+0000 7fce8b751700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce840ce510 0x7fce841599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:34.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.510+0000 7fce8b751700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce84157c80 0x7fce84155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:34.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.510+0000 7fce8b751700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8415a9b0 con 0x7fce84157c80 2024-01-27T02:32:34.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.510+0000 7fce8b751700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce840c2e30 con 0x7fce84063580 2024-01-27T02:32:34.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.510+0000 7fce8b751700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce84149c90 con 0x7fce840ce510 2024-01-27T02:32:34.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.510+0000 7fce89cee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce84157c80 0x7fce84155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:34.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.510+0000 7fce89cee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce84157c80 0x7fce84155c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58098/0 (socket says 172.21.15.7:58098) 2024-01-27T02:32:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.510+0000 7fce89cee700 1 -- 172.21.15.7:0/4200698135 learned_addr learned my addr 172.21.15.7:0/4200698135 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.510+0000 7fce894ed700 1 --2- 172.21.15.7:0/4200698135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce84063580 0x7fce84149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.510+0000 7fce88cec700 1 --2- 172.21.15.7:0/4200698135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce840ce510 0x7fce841599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.510+0000 7fce89cee700 1 -- 172.21.15.7:0/4200698135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce840ce510 msgr2=0x7fce841599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.510+0000 7fce89cee700 1 --2- 172.21.15.7:0/4200698135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce840ce510 0x7fce841599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.510+0000 7fce89cee700 1 -- 172.21.15.7:0/4200698135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce84063580 msgr2=0x7fce84149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.510+0000 7fce89cee700 1 --2- 172.21.15.7:0/4200698135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce84063580 0x7fce84149750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.511+0000 7fce89cee700 1 -- 172.21.15.7:0/4200698135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce74009420 con 0x7fce84157c80 2024-01-27T02:32:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.511+0000 7fce88cec700 1 --2- 172.21.15.7:0/4200698135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce840ce510 0x7fce841599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:32:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.511+0000 7fce89cee700 1 --2- 172.21.15.7:0/4200698135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce84157c80 0x7fce84155c20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00cc90 tx=0x7fce7c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:34.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.511+0000 7fce797fa700 1 -- 172.21.15.7:0/4200698135 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c00cea0 con 0x7fce84157c80 2024-01-27T02:32:34.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.511+0000 7fce797fa700 1 -- 172.21.15.7:0/4200698135 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7c00ab70 con 0x7fce84157c80 2024-01-27T02:32:34.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.511+0000 7fce8b751700 1 -- 172.21.15.7:0/4200698135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce8415a3f0 con 0x7fce84157c80 2024-01-27T02:32:34.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.511+0000 7fce797fa700 1 -- 172.21.15.7:0/4200698135 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c0070f0 con 0x7fce84157c80 2024-01-27T02:32:34.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.511+0000 7fce8b751700 1 -- 172.21.15.7:0/4200698135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce840c2ac0 con 0x7fce84157c80 2024-01-27T02:32:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.512+0000 7fce8b751700 1 -- 172.21.15.7:0/4200698135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce84072060 con 0x7fce84157c80 2024-01-27T02:32:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.512+0000 7fce797fa700 1 -- 172.21.15.7:0/4200698135 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fce7c0074e0 con 0x7fce84157c80 2024-01-27T02:32:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.513+0000 7fce797fa700 1 --2- 172.21.15.7:0/4200698135 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fce68066f30 0x7fce680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.513+0000 7fce797fa700 1 -- 172.21.15.7:0/4200698135 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fce7c084920 con 0x7fce84157c80 2024-01-27T02:32:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.513+0000 7fce894ed700 1 --2- 172.21.15.7:0/4200698135 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fce68066f30 0x7fce680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.514+0000 7fce894ed700 1 --2- 172.21.15.7:0/4200698135 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fce68066f30 0x7fce680693e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fce840ce4e0 tx=0x7fce80009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:34.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.518+0000 7fce797fa700 1 -- 172.21.15.7:0/4200698135 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce7c051c10 con 0x7fce84157c80 2024-01-27T02:32:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.721+0000 7fce8b751700 1 -- 172.21.15.7:0/4200698135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fce840b2ec0 con 0x7fce84157c80 2024-01-27T02:32:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.721+0000 7fce797fa700 1 -- 172.21.15.7:0/4200698135 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fce7c055230 con 0x7fce84157c80 2024-01-27T02:32:34.724 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-27T02:32:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.723+0000 7fce8b751700 1 -- 172.21.15.7:0/4200698135 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fce68066f30 msgr2=0x7fce680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.723+0000 7fce8b751700 1 --2- 172.21.15.7:0/4200698135 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fce68066f30 0x7fce680693e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fce840ce4e0 tx=0x7fce80009040 comp rx=0 tx=0).stop 2024-01-27T02:32:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.723+0000 7fce8b751700 1 -- 172.21.15.7:0/4200698135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce84157c80 msgr2=0x7fce84155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.723+0000 7fce8b751700 1 --2- 172.21.15.7:0/4200698135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce84157c80 0x7fce84155c20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00cc90 tx=0x7fce7c001a80 comp rx=0 tx=0).stop 2024-01-27T02:32:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.723+0000 7fce8b751700 1 -- 172.21.15.7:0/4200698135 shutdown_connections 2024-01-27T02:32:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.723+0000 7fce8b751700 1 --2- 172.21.15.7:0/4200698135 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fce68066f30 0x7fce680693e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.723+0000 7fce8b751700 1 --2- 172.21.15.7:0/4200698135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce84157c80 0x7fce84155c20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.723+0000 7fce8b751700 1 --2- 172.21.15.7:0/4200698135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce84063580 0x7fce84149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.723+0000 7fce8b751700 1 --2- 172.21.15.7:0/4200698135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce840ce510 0x7fce841599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.723+0000 7fce8b751700 1 -- 172.21.15.7:0/4200698135 >> 172.21.15.7:0/4200698135 conn(0x7fce84068480 msgr2=0x7fce84153620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:34.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.723+0000 7fce8b751700 1 -- 172.21.15.7:0/4200698135 shutdown_connections 2024-01-27T02:32:34.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.723+0000 7fce8b751700 1 -- 172.21.15.7:0/4200698135 wait complete. 2024-01-27T02:32:34.737 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-27T02:32:34.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.871+0000 7f5ab8b8c700 1 Processor -- start 2024-01-27T02:32:34.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.871+0000 7f5ab8b8c700 1 -- start start 2024-01-27T02:32:34.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.872+0000 7f5ab8b8c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ab41416c0 0x7f5ab40cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:34.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.873+0000 7f5ab8b8c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5ab4141a90 0x7f5ab40cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:34.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.873+0000 7f5ab8b8c700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ab415c7d0 con 0x7f5ab40cab00 2024-01-27T02:32:34.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.873+0000 7f5ab8b8c700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ab40cd9c0 con 0x7f5ab4141a90 2024-01-27T02:32:34.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.873+0000 7f5ab8b8c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab40c9640 con 0x7f5ab41416c0 2024-01-27T02:32:34.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.873+0000 7f5ab1d9b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5ab4141a90 0x7f5ab40cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:34.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.873+0000 7f5ab2d9d700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f5ab40cab00 0x7f5ab40caf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:53842/0 (socket says 172.21.15.7:53842) 2024-01-27T02:32:34.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.873+0000 7f5ab259c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ab41416c0 0x7f5ab40cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:34.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.873+0000 7f5ab2d9d700 1 -- 172.21.15.7:0/182312780 learned_addr learned my addr 172.21.15.7:0/182312780 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:34.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.873+0000 7f5ab159a700 1 -- 172.21.15.7:0/182312780 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3564549281 0 0) 0x7f5ab415c7d0 con 0x7f5ab40cab00 2024-01-27T02:32:34.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.873+0000 7f5ab159a700 1 -- 172.21.15.7:0/182312780 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a9c003570 con 0x7f5ab40cab00 2024-01-27T02:32:34.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.873+0000 7f5ab1d9b700 1 -- 172.21.15.7:0/182312780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ab41416c0 msgr2=0x7f5ab40cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:34.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.873+0000 7f5ab1d9b700 1 --2- 172.21.15.7:0/182312780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ab41416c0 0x7f5ab40cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.873+0000 7f5ab1d9b700 1 -- 172.21.15.7:0/182312780 >> v1:172.21.15.188:6789/0 conn(0x7f5ab40cab00 legacy=0x7f5ab40caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:34.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.873+0000 7f5ab1d9b700 1 -- 172.21.15.7:0/182312780 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab415bbc0 con 0x7f5ab4141a90 2024-01-27T02:32:34.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.873+0000 7f5ab259c700 1 --2- 172.21.15.7:0/182312780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ab41416c0 0x7f5ab40cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:32:34.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.874+0000 7f5ab1d9b700 1 --2- 172.21.15.7:0/182312780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ab4141a90 0x7f5ab40cbc20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5aa80098e0 tx=0x7f5aa80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cbd08b42a1a57d0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:34.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.874+0000 7f5ab159a700 1 -- 172.21.15.7:0/182312780 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8011630 con 0x7f5ab4141a90 2024-01-27T02:32:34.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.874+0000 7f5ab159a700 1 -- 172.21.15.7:0/182312780 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa8011790 con 0x7f5ab4141a90 2024-01-27T02:32:34.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.874+0000 7f5ab159a700 1 -- 172.21.15.7:0/182312780 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8011b30 con 0x7f5ab4141a90 2024-01-27T02:32:34.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.874+0000 7f5ab8b8c700 1 -- 172.21.15.7:0/182312780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ab4141a90 msgr2=0x7f5ab40cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:34.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.874+0000 7f5ab8b8c700 1 --2- 172.21.15.7:0/182312780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ab4141a90 0x7f5ab40cbc20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5aa80098e0 tx=0x7f5aa80091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.874+0000 7f5ab8b8c700 1 -- 172.21.15.7:0/182312780 shutdown_connections 2024-01-27T02:32:34.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.874+0000 7f5ab8b8c700 1 --2- 172.21.15.7:0/182312780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ab41416c0 0x7f5ab40cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.874+0000 7f5ab8b8c700 1 --2- 172.21.15.7:0/182312780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ab4141a90 0x7f5ab40cbc20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.874+0000 7f5ab8b8c700 1 -- 172.21.15.7:0/182312780 >> 172.21.15.7:0/182312780 conn(0x7f5ab4063580 msgr2=0x7f5ab4063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:34.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.874+0000 7f5ab8b8c700 1 -- 172.21.15.7:0/182312780 shutdown_connections 2024-01-27T02:32:34.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.875+0000 7f5ab8b8c700 1 -- 172.21.15.7:0/182312780 wait complete. 2024-01-27T02:32:34.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.876+0000 7f5ab8b8c700 1 Processor -- start 2024-01-27T02:32:34.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.876+0000 7f5ab8b8c700 1 -- start start 2024-01-27T02:32:34.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.876+0000 7f5ab8b8c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ab40cab00 0x7f5ab40c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:34.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.876+0000 7f5ab8b8c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ab41416c0 0x7f5ab4068af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:34.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.876+0000 7f5ab8b8c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab4141a90 0x7f5ab413cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:34.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.876+0000 7f5ab8b8c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab40686d0 con 0x7f5ab4141a90 2024-01-27T02:32:34.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.876+0000 7f5ab8b8c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab40c8bd0 con 0x7f5ab41416c0 2024-01-27T02:32:34.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.876+0000 7f5ab8b8c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab4067a50 con 0x7f5ab40cab00 2024-01-27T02:32:34.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.877+0000 7f5ab2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab4141a90 0x7f5ab413cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:34.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.877+0000 7f5ab2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab4141a90 0x7f5ab413cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58106/0 (socket says 172.21.15.7:58106) 2024-01-27T02:32:34.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.877+0000 7f5ab2d9d700 1 -- 172.21.15.7:0/2037809842 learned_addr learned my addr 172.21.15.7:0/2037809842 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:34.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.877+0000 7f5ab1d9b700 1 --2- 172.21.15.7:0/2037809842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ab41416c0 0x7f5ab4068af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:34.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.877+0000 7f5ab259c700 1 --2- 172.21.15.7:0/2037809842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ab40cab00 0x7f5ab40c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:34.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.877+0000 7f5ab2d9d700 1 -- 172.21.15.7:0/2037809842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ab40cab00 msgr2=0x7f5ab40c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:34.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.877+0000 7f5ab2d9d700 1 --2- 172.21.15.7:0/2037809842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ab40cab00 0x7f5ab40c8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.877+0000 7f5ab2d9d700 1 -- 172.21.15.7:0/2037809842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ab41416c0 msgr2=0x7f5ab4068af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:34.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.877+0000 7f5ab2d9d700 1 --2- 172.21.15.7:0/2037809842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ab41416c0 0x7f5ab4068af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:34.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.877+0000 7f5ab2d9d700 1 -- 172.21.15.7:0/2037809842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa40093c0 con 0x7f5ab4141a90 2024-01-27T02:32:34.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.877+0000 7f5ab1d9b700 1 --2- 172.21.15.7:0/2037809842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ab41416c0 0x7f5ab4068af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:32:34.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.877+0000 7f5ab2d9d700 1 --2- 172.21.15.7:0/2037809842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab4141a90 0x7f5ab413cb40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5aac001460 tx=0x7f5aac001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:34.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.877+0000 7f5aa27fc700 1 -- 172.21.15.7:0/2037809842 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac00a870 con 0x7f5ab4141a90 2024-01-27T02:32:34.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.878+0000 7f5aa27fc700 1 -- 172.21.15.7:0/2037809842 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aac00a9d0 con 0x7f5ab4141a90 2024-01-27T02:32:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.878+0000 7f5aa27fc700 1 -- 172.21.15.7:0/2037809842 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac00ad50 con 0x7f5ab4141a90 2024-01-27T02:32:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.878+0000 7f5ab8b8c700 1 -- 172.21.15.7:0/2037809842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa8009420 con 0x7f5ab4141a90 2024-01-27T02:32:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.878+0000 7f5ab8b8c700 1 -- 172.21.15.7:0/2037809842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab4068010 con 0x7f5ab4141a90 2024-01-27T02:32:34.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.879+0000 7f5ab8b8c700 1 -- 172.21.15.7:0/2037809842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab4068810 con 0x7f5ab4141a90 2024-01-27T02:32:34.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.879+0000 7f5aa27fc700 1 -- 172.21.15.7:0/2037809842 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5aac007300 con 0x7f5ab4141a90 2024-01-27T02:32:34.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.879+0000 7f5aa27fc700 1 --2- 172.21.15.7:0/2037809842 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5a94066f30 0x7f5a940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:34.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.879+0000 7f5aa27fc700 1 -- 172.21.15.7:0/2037809842 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5aac0847d0 con 0x7f5ab4141a90 2024-01-27T02:32:34.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.880+0000 7f5ab259c700 1 --2- 172.21.15.7:0/2037809842 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5a94066f30 0x7f5a940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:34.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.880+0000 7f5ab259c700 1 --2- 172.21.15.7:0/2037809842 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5a94066f30 0x7f5a940693e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009f30 tx=0x7f5aa4009120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:34.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:34.884+0000 7f5aa27fc700 1 -- 172.21.15.7:0/2037809842 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ab4068810 con 0x7f5ab4141a90 2024-01-27T02:32:35.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.087+0000 7f5ab8b8c700 1 -- 172.21.15.7:0/2037809842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5ab4068810 con 0x7f5ab4141a90 2024-01-27T02:32:35.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.088+0000 7f5aa27fc700 1 -- 172.21.15.7:0/2037809842 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5ab4068810 con 0x7f5ab4141a90 2024-01-27T02:32:35.089 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-27T02:32:35.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.090+0000 7f5ab8b8c700 1 -- 172.21.15.7:0/2037809842 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5a94066f30 msgr2=0x7f5a940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:35.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.090+0000 7f5ab8b8c700 1 --2- 172.21.15.7:0/2037809842 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5a94066f30 0x7f5a940693e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009f30 tx=0x7f5aa4009120 comp rx=0 tx=0).stop 2024-01-27T02:32:35.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.090+0000 7f5ab8b8c700 1 -- 172.21.15.7:0/2037809842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab4141a90 msgr2=0x7f5ab413cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:35.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.090+0000 7f5ab8b8c700 1 --2- 172.21.15.7:0/2037809842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab4141a90 0x7f5ab413cb40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5aac001460 tx=0x7f5aac001a40 comp rx=0 tx=0).stop 2024-01-27T02:32:35.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.090+0000 7f5ab8b8c700 1 -- 172.21.15.7:0/2037809842 shutdown_connections 2024-01-27T02:32:35.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.090+0000 7f5ab8b8c700 1 --2- 172.21.15.7:0/2037809842 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5a94066f30 0x7f5a940693e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.090+0000 7f5ab8b8c700 1 --2- 172.21.15.7:0/2037809842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab4141a90 0x7f5ab413cb40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.090+0000 7f5ab8b8c700 1 --2- 172.21.15.7:0/2037809842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ab40cab00 0x7f5ab40c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.090+0000 7f5ab8b8c700 1 --2- 172.21.15.7:0/2037809842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ab41416c0 0x7f5ab4068af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.090+0000 7f5ab8b8c700 1 -- 172.21.15.7:0/2037809842 >> 172.21.15.7:0/2037809842 conn(0x7f5ab4063580 msgr2=0x7f5ab40cc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:35.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.090+0000 7f5ab8b8c700 1 -- 172.21.15.7:0/2037809842 shutdown_connections 2024-01-27T02:32:35.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.090+0000 7f5ab8b8c700 1 -- 172.21.15.7:0/2037809842 wait complete. 2024-01-27T02:32:35.105 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:32:35.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.236+0000 7f8c3bbd8700 1 Processor -- start 2024-01-27T02:32:35.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.236+0000 7f8c3bbd8700 1 -- start start 2024-01-27T02:32:35.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.237+0000 7f8c3bbd8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8c340a7c10 0x7f8c340ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:35.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.237+0000 7f8c3bbd8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c340ae880 0x7f8c3406f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:35.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.237+0000 7f8c3bbd8700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c3415a880 con 0x7f8c340a7fe0 2024-01-27T02:32:35.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.237+0000 7f8c3bbd8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c340afde0 con 0x7f8c340a7c10 2024-01-27T02:32:35.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.237+0000 7f8c3bbd8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c340af950 con 0x7f8c340ae880 2024-01-27T02:32:35.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.237+0000 7f8c39974700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8c340a7c10 0x7f8c340ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:35.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.237+0000 7f8c39974700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8c340a7c10 0x7f8c340ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58116/0 (socket says 172.21.15.7:58116) 2024-01-27T02:32:35.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.237+0000 7f8c39974700 1 -- 172.21.15.7:0/3025730751 learned_addr learned my addr 172.21.15.7:0/3025730751 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:35.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.237+0000 7f8c39173700 1 --2- 172.21.15.7:0/3025730751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c340ae880 0x7f8c3406f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:35.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.238+0000 7f8c39974700 1 -- 172.21.15.7:0/3025730751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c340ae880 msgr2=0x7f8c3406f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:35.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.238+0000 7f8c39974700 1 --2- 172.21.15.7:0/3025730751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c340ae880 0x7f8c3406f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.238+0000 7f8c39974700 1 -- 172.21.15.7:0/3025730751 >> v1:172.21.15.188:6789/0 conn(0x7f8c340a7fe0 legacy=0x7f8c3406dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:35.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.238+0000 7f8c39974700 1 -- 172.21.15.7:0/3025730751 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c340aed40 con 0x7f8c340a7c10 2024-01-27T02:32:35.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.238+0000 7f8c39974700 1 --2- 172.21.15.7:0/3025730751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c340a7c10 0x7f8c340ae340 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8c300099b0 tx=0x7f8c30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502164083fdb6ef server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:35.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.238+0000 7f8c38972700 1 -- 172.21.15.7:0/3025730751 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30009ea0 con 0x7f8c340a7c10 2024-01-27T02:32:35.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.238+0000 7f8c38972700 1 -- 172.21.15.7:0/3025730751 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3000b860 con 0x7f8c340a7c10 2024-01-27T02:32:35.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.238+0000 7f8c38972700 1 -- 172.21.15.7:0/3025730751 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3000bc00 con 0x7f8c340a7c10 2024-01-27T02:32:35.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.239+0000 7f8c3bbd8700 1 -- 172.21.15.7:0/3025730751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c340a7c10 msgr2=0x7f8c340ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:35.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.239+0000 7f8c3bbd8700 1 --2- 172.21.15.7:0/3025730751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c340a7c10 0x7f8c340ae340 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8c300099b0 tx=0x7f8c30009250 comp rx=0 tx=0).stop 2024-01-27T02:32:35.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.239+0000 7f8c3bbd8700 1 -- 172.21.15.7:0/3025730751 shutdown_connections 2024-01-27T02:32:35.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.239+0000 7f8c3bbd8700 1 --2- 172.21.15.7:0/3025730751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c340a7c10 0x7f8c340ae340 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.239+0000 7f8c3bbd8700 1 --2- 172.21.15.7:0/3025730751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c340ae880 0x7f8c3406f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.239+0000 7f8c3bbd8700 1 -- 172.21.15.7:0/3025730751 >> 172.21.15.7:0/3025730751 conn(0x7f8c340679b0 msgr2=0x7f8c34067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:35.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.239+0000 7f8c3bbd8700 1 -- 172.21.15.7:0/3025730751 shutdown_connections 2024-01-27T02:32:35.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.239+0000 7f8c3bbd8700 1 -- 172.21.15.7:0/3025730751 wait complete. 2024-01-27T02:32:35.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.240+0000 7f8c3bbd8700 1 Processor -- start 2024-01-27T02:32:35.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.240+0000 7f8c3bbd8700 1 -- start start 2024-01-27T02:32:35.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.241+0000 7f8c3bbd8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c340a7c10 0x7f8c34098930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:35.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.241+0000 7f8c3bbd8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c340a7fe0 0x7f8c34154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:35.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.241+0000 7f8c3bbd8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c340ae880 0x7f8c3415ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:35.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.241+0000 7f8c3bbd8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c341546e0 con 0x7f8c340ae880 2024-01-27T02:32:35.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.241+0000 7f8c3bbd8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c34153db0 con 0x7f8c340a7c10 2024-01-27T02:32:35.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.241+0000 7f8c3bbd8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c34154090 con 0x7f8c340a7fe0 2024-01-27T02:32:35.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.241+0000 7f8c3a175700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c340ae880 0x7f8c3415ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:35.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.241+0000 7f8c3a175700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c340ae880 0x7f8c3415ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58126/0 (socket says 172.21.15.7:58126) 2024-01-27T02:32:35.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.241+0000 7f8c3a175700 1 -- 172.21.15.7:0/2364766429 learned_addr learned my addr 172.21.15.7:0/2364766429 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:35.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.241+0000 7f8c39173700 1 --2- 172.21.15.7:0/2364766429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c340a7fe0 0x7f8c34154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:35.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.241+0000 7f8c39974700 1 --2- 172.21.15.7:0/2364766429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c340a7c10 0x7f8c34098930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:35.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.241+0000 7f8c3a175700 1 -- 172.21.15.7:0/2364766429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c340a7fe0 msgr2=0x7f8c34154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:35.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.241+0000 7f8c3a175700 1 --2- 172.21.15.7:0/2364766429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c340a7fe0 0x7f8c34154b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.241+0000 7f8c3a175700 1 -- 172.21.15.7:0/2364766429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c340a7c10 msgr2=0x7f8c34098930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:35.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.241+0000 7f8c3a175700 1 --2- 172.21.15.7:0/2364766429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c340a7c10 0x7f8c34098930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.241+0000 7f8c3a175700 1 -- 172.21.15.7:0/2364766429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c300094f0 con 0x7f8c340ae880 2024-01-27T02:32:35.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.242+0000 7f8c3a175700 1 --2- 172.21.15.7:0/2364766429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c340ae880 0x7f8c3415ba00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00c910 tx=0x7f8c2c0016b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:35.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.242+0000 7f8c29ffb700 1 -- 172.21.15.7:0/2364766429 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c012630 con 0x7f8c340ae880 2024-01-27T02:32:35.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.242+0000 7f8c29ffb700 1 -- 172.21.15.7:0/2364766429 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2c0127b0 con 0x7f8c340ae880 2024-01-27T02:32:35.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.242+0000 7f8c3bbd8700 1 -- 172.21.15.7:0/2364766429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c34154370 con 0x7f8c340ae880 2024-01-27T02:32:35.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.242+0000 7f8c29ffb700 1 -- 172.21.15.7:0/2364766429 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c012b70 con 0x7f8c340ae880 2024-01-27T02:32:35.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.242+0000 7f8c3bbd8700 1 -- 172.21.15.7:0/2364766429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c341539f0 con 0x7f8c340ae880 2024-01-27T02:32:35.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.243+0000 7f8c3bbd8700 1 -- 172.21.15.7:0/2364766429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c3404c710 con 0x7f8c340ae880 2024-01-27T02:32:35.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.243+0000 7f8c29ffb700 1 -- 172.21.15.7:0/2364766429 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8c2c01a3a0 con 0x7f8c340ae880 2024-01-27T02:32:35.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.244+0000 7f8c29ffb700 1 --2- 172.21.15.7:0/2364766429 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8c18066f30 0x7f8c180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:35.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.244+0000 7f8c29ffb700 1 -- 172.21.15.7:0/2364766429 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8c2c084600 con 0x7f8c340ae880 2024-01-27T02:32:35.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.244+0000 7f8c39974700 1 --2- 172.21.15.7:0/2364766429 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8c18066f30 0x7f8c180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:35.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.245+0000 7f8c39974700 1 --2- 172.21.15.7:0/2364766429 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8c18066f30 0x7f8c180693e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8c300099b0 tx=0x7f8c30017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:35.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.249+0000 7f8c29ffb700 1 -- 172.21.15.7:0/2364766429 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c2c051870 con 0x7f8c340ae880 2024-01-27T02:32:35.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.454+0000 7f8c3bbd8700 1 -- 172.21.15.7:0/2364766429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c340af020 con 0x7f8c340ae880 2024-01-27T02:32:35.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.454+0000 7f8c29ffb700 1 -- 172.21.15.7:0/2364766429 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f8c2c054e90 con 0x7f8c340ae880 2024-01-27T02:32:35.455 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:35.455 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:32:07.627822+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:32:03.675611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-27T02:32:05.018885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:32:35.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.457+0000 7f8c3bbd8700 1 -- 172.21.15.7:0/2364766429 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8c18066f30 msgr2=0x7f8c180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:35.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.457+0000 7f8c3bbd8700 1 --2- 172.21.15.7:0/2364766429 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8c18066f30 0x7f8c180693e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8c300099b0 tx=0x7f8c30017000 comp rx=0 tx=0).stop 2024-01-27T02:32:35.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.457+0000 7f8c3bbd8700 1 -- 172.21.15.7:0/2364766429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c340ae880 msgr2=0x7f8c3415ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:35.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.457+0000 7f8c3bbd8700 1 --2- 172.21.15.7:0/2364766429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c340ae880 0x7f8c3415ba00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00c910 tx=0x7f8c2c0016b0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.457+0000 7f8c3bbd8700 1 -- 172.21.15.7:0/2364766429 shutdown_connections 2024-01-27T02:32:35.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.457+0000 7f8c3bbd8700 1 --2- 172.21.15.7:0/2364766429 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8c18066f30 0x7f8c180693e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.457+0000 7f8c3bbd8700 1 --2- 172.21.15.7:0/2364766429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c340ae880 0x7f8c3415ba00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.457+0000 7f8c3bbd8700 1 --2- 172.21.15.7:0/2364766429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c340a7c10 0x7f8c34098930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.457+0000 7f8c3bbd8700 1 --2- 172.21.15.7:0/2364766429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c340a7fe0 0x7f8c34154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.457+0000 7f8c3bbd8700 1 -- 172.21.15.7:0/2364766429 >> 172.21.15.7:0/2364766429 conn(0x7f8c340679b0 msgr2=0x7f8c34159ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:35.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.457+0000 7f8c3bbd8700 1 -- 172.21.15.7:0/2364766429 shutdown_connections 2024-01-27T02:32:35.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.457+0000 7f8c3bbd8700 1 -- 172.21.15.7:0/2364766429 wait complete. 2024-01-27T02:32:35.471 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-27T02:32:03.675611+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-27T02:32:05.018885+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-27T02:32:35.471 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:32:35.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.609+0000 7f67a9c58700 1 Processor -- start 2024-01-27T02:32:35.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.609+0000 7f67a9c58700 1 -- start start 2024-01-27T02:32:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.609+0000 7f67a9c58700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f67a415d460 0x7f67a40d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.609+0000 7f67a9c58700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67a40cf280 0x7f67a415e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.609+0000 7f67a9c58700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a40b38d0 con 0x7f67a415d830 2024-01-27T02:32:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.609+0000 7f67a9c58700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a40d0390 con 0x7f67a415d460 2024-01-27T02:32:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.609+0000 7f67a9c58700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a40cf780 con 0x7f67a40cf280 2024-01-27T02:32:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.610+0000 7f67a37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f67a415d460 0x7f67a40d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.610+0000 7f67a3fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f67a415d830 0x7f67a40cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45554/0 (socket says 172.21.15.7:45554) 2024-01-27T02:32:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.610+0000 7f67a3fff700 1 -- 172.21.15.7:0/213146966 learned_addr learned my addr 172.21.15.7:0/213146966 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.610+0000 7f67a2ffd700 1 --2- 172.21.15.7:0/213146966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67a40cf280 0x7f67a415e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.610+0000 7f67a37fe700 1 -- 172.21.15.7:0/213146966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67a40cf280 msgr2=0x7f67a415e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.610+0000 7f67a37fe700 1 --2- 172.21.15.7:0/213146966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67a40cf280 0x7f67a415e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.610+0000 7f67a37fe700 1 -- 172.21.15.7:0/213146966 >> v1:172.21.15.7:6789/0 conn(0x7f67a415d830 legacy=0x7f67a40cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.610+0000 7f67a37fe700 1 -- 172.21.15.7:0/213146966 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a40cfa60 con 0x7f67a415d460 2024-01-27T02:32:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.610+0000 7f67a27fc700 1 -- 172.21.15.7:0/213146966 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1639816366 0 0) 0x7f679c001490 con 0x7f67a415d830 2024-01-27T02:32:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.610+0000 7f67a37fe700 1 --2- 172.21.15.7:0/213146966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a415d460 0x7f67a40d0820 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f67940098e0 tx=0x7f67940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5425d7e8306dbf9f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.610+0000 7f67a27fc700 1 -- 172.21.15.7:0/213146966 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6794009e80 con 0x7f67a415d460 2024-01-27T02:32:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.610+0000 7f67a27fc700 1 -- 172.21.15.7:0/213146966 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679400b860 con 0x7f67a415d460 2024-01-27T02:32:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.610+0000 7f67a27fc700 1 -- 172.21.15.7:0/213146966 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679400bbe0 con 0x7f67a415d460 2024-01-27T02:32:35.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.611+0000 7f67a9c58700 1 -- 172.21.15.7:0/213146966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a415d460 msgr2=0x7f67a40d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:35.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.611+0000 7f67a9c58700 1 --2- 172.21.15.7:0/213146966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a415d460 0x7f67a40d0820 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f67940098e0 tx=0x7f67940091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.611+0000 7f67a9c58700 1 -- 172.21.15.7:0/213146966 shutdown_connections 2024-01-27T02:32:35.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.611+0000 7f67a9c58700 1 --2- 172.21.15.7:0/213146966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a415d460 0x7f67a40d0820 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.611+0000 7f67a9c58700 1 --2- 172.21.15.7:0/213146966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67a40cf280 0x7f67a415e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.611+0000 7f67a9c58700 1 -- 172.21.15.7:0/213146966 >> 172.21.15.7:0/213146966 conn(0x7f67a4063580 msgr2=0x7f67a4063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:35.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.611+0000 7f67a9c58700 1 -- 172.21.15.7:0/213146966 shutdown_connections 2024-01-27T02:32:35.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.611+0000 7f67a9c58700 1 -- 172.21.15.7:0/213146966 wait complete. 2024-01-27T02:32:35.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.613+0000 7f67a9c58700 1 Processor -- start 2024-01-27T02:32:35.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.613+0000 7f67a9c58700 1 -- start start 2024-01-27T02:32:35.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.614+0000 7f67a9c58700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67a40cf280 0x7f67a40ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:35.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.614+0000 7f67a9c58700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67a415d460 0x7f67a40b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:35.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.614+0000 7f67a9c58700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a415d830 0x7f67a41591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:35.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.614+0000 7f67a9c58700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a40c8170 con 0x7f67a415d830 2024-01-27T02:32:35.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.614+0000 7f67a9c58700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a40c8450 con 0x7f67a415d460 2024-01-27T02:32:35.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.614+0000 7f67a9c58700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a40c8730 con 0x7f67a40cf280 2024-01-27T02:32:35.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.614+0000 7f67a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a415d830 0x7f67a41591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:35.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.614+0000 7f67a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a415d830 0x7f67a41591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58142/0 (socket says 172.21.15.7:58142) 2024-01-27T02:32:35.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.614+0000 7f67a3fff700 1 -- 172.21.15.7:0/207134698 learned_addr learned my addr 172.21.15.7:0/207134698 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:35.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.614+0000 7f67a37fe700 1 --2- 172.21.15.7:0/207134698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67a40cf280 0x7f67a40ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:35.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.614+0000 7f67a2ffd700 1 --2- 172.21.15.7:0/207134698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67a415d460 0x7f67a40b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:35.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.615+0000 7f67a3fff700 1 -- 172.21.15.7:0/207134698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67a40cf280 msgr2=0x7f67a40ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:35.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.615+0000 7f67a3fff700 1 --2- 172.21.15.7:0/207134698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67a40cf280 0x7f67a40ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.615+0000 7f67a3fff700 1 -- 172.21.15.7:0/207134698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67a415d460 msgr2=0x7f67a40b6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:35.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.615+0000 7f67a3fff700 1 --2- 172.21.15.7:0/207134698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67a415d460 0x7f67a40b6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.615+0000 7f67a3fff700 1 -- 172.21.15.7:0/207134698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6794009420 con 0x7f67a415d830 2024-01-27T02:32:35.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.615+0000 7f67a37fe700 1 --2- 172.21.15.7:0/207134698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67a40cf280 0x7f67a40ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:35.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.615+0000 7f67a3fff700 1 --2- 172.21.15.7:0/207134698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a415d830 0x7f67a41591d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f679c00cbf0 tx=0x7f679c0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:35.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.615+0000 7f678b7fe700 1 -- 172.21.15.7:0/207134698 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c00a910 con 0x7f67a415d830 2024-01-27T02:32:35.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.615+0000 7f678b7fe700 1 -- 172.21.15.7:0/207134698 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679c00aa70 con 0x7f67a415d830 2024-01-27T02:32:35.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.615+0000 7f678b7fe700 1 -- 172.21.15.7:0/207134698 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c00ae30 con 0x7f67a415d830 2024-01-27T02:32:35.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.615+0000 7f67a9c58700 1 -- 172.21.15.7:0/207134698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a40c8a10 con 0x7f67a415d830 2024-01-27T02:32:35.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.615+0000 7f67a9c58700 1 -- 172.21.15.7:0/207134698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a416b3f0 con 0x7f67a415d830 2024-01-27T02:32:35.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.616+0000 7f67a9c58700 1 -- 172.21.15.7:0/207134698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a4072060 con 0x7f67a415d830 2024-01-27T02:32:35.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.617+0000 7f678b7fe700 1 -- 172.21.15.7:0/207134698 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f679c007300 con 0x7f67a415d830 2024-01-27T02:32:35.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.617+0000 7f678b7fe700 1 --2- 172.21.15.7:0/207134698 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6790066f30 0x7f67900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:35.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.617+0000 7f678b7fe700 1 -- 172.21.15.7:0/207134698 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f679c0846e0 con 0x7f67a415d830 2024-01-27T02:32:35.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.617+0000 7f67a37fe700 1 --2- 172.21.15.7:0/207134698 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6790066f30 0x7f67900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:35.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.618+0000 7f67a37fe700 1 --2- 172.21.15.7:0/207134698 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6790066f30 0x7f67900693e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f67940091e0 tx=0x7f6794017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:35.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.621+0000 7f678b7fe700 1 -- 172.21.15.7:0/207134698 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f679c0519d0 con 0x7f67a415d830 2024-01-27T02:32:35.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.826+0000 7f67a9c58700 1 -- 172.21.15.7:0/207134698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f67a40b38d0 con 0x7f67a415d830 2024-01-27T02:32:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.827+0000 7f678b7fe700 1 -- 172.21.15.7:0/207134698 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f67a40b38d0 con 0x7f67a415d830 2024-01-27T02:32:35.827 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:32:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.829+0000 7f67a9c58700 1 -- 172.21.15.7:0/207134698 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6790066f30 msgr2=0x7f67900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.829+0000 7f67a9c58700 1 --2- 172.21.15.7:0/207134698 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6790066f30 0x7f67900693e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f67940091e0 tx=0x7f6794017000 comp rx=0 tx=0).stop 2024-01-27T02:32:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.829+0000 7f67a9c58700 1 -- 172.21.15.7:0/207134698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a415d830 msgr2=0x7f67a41591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.829+0000 7f67a9c58700 1 --2- 172.21.15.7:0/207134698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a415d830 0x7f67a41591d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f679c00cbf0 tx=0x7f679c0019b0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.829+0000 7f67a9c58700 1 -- 172.21.15.7:0/207134698 shutdown_connections 2024-01-27T02:32:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.829+0000 7f67a9c58700 1 --2- 172.21.15.7:0/207134698 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6790066f30 0x7f67900693e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.829+0000 7f67a9c58700 1 --2- 172.21.15.7:0/207134698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a415d830 0x7f67a41591d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.829+0000 7f67a9c58700 1 --2- 172.21.15.7:0/207134698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67a40cf280 0x7f67a40ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.829+0000 7f67a9c58700 1 --2- 172.21.15.7:0/207134698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67a415d460 0x7f67a40b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.829+0000 7f67a9c58700 1 -- 172.21.15.7:0/207134698 >> 172.21.15.7:0/207134698 conn(0x7f67a4063580 msgr2=0x7f67a40cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.829+0000 7f67a9c58700 1 -- 172.21.15.7:0/207134698 shutdown_connections 2024-01-27T02:32:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.829+0000 7f67a9c58700 1 -- 172.21.15.7:0/207134698 wait complete. 2024-01-27T02:32:35.843 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-27T02:32:35.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.978+0000 7f85c1d01700 1 Processor -- start 2024-01-27T02:32:35.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.978+0000 7f85c1d01700 1 -- start start 2024-01-27T02:32:35.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.979+0000 7f85c1d01700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f85bc0cba60 0x7f85bc0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:35.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.979+0000 7f85c1d01700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85bc06b4e0 0x7f85bc06b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:35.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.979+0000 7f85c1d01700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85bc0679b0 con 0x7f85bc0cbe30 2024-01-27T02:32:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.979+0000 7f85c1d01700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85bc067c90 con 0x7f85bc0cba60 2024-01-27T02:32:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.979+0000 7f85c1d01700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc06be60 con 0x7f85bc06b4e0 2024-01-27T02:32:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.979+0000 7f85baffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f85bc0cbe30 0x7f85bc15b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45564/0 (socket says 172.21.15.7:45564) 2024-01-27T02:32:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.979+0000 7f85baffd700 1 -- 172.21.15.7:0/880611072 learned_addr learned my addr 172.21.15.7:0/880611072 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.979+0000 7f85bb7fe700 1 --2- 172.21.15.7:0/880611072 >> v2:172.21.15.188:3300/0 conn(0x7f85bc0cba60 0x7f85bc0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.979+0000 7f85bbfff700 1 --2- 172.21.15.7:0/880611072 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85bc06b4e0 0x7f85bc06b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.979+0000 7f85ba7fc700 1 -- 172.21.15.7:0/880611072 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3922816703 0 0) 0x7f85bc0679b0 con 0x7f85bc0cbe30 2024-01-27T02:32:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.979+0000 7f85ba7fc700 1 -- 172.21.15.7:0/880611072 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85a8003570 con 0x7f85bc0cbe30 2024-01-27T02:32:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.980+0000 7f85bb7fe700 1 -- 172.21.15.7:0/880611072 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85bc06b4e0 msgr2=0x7f85bc06b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.980+0000 7f85bb7fe700 1 --2- 172.21.15.7:0/880611072 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85bc06b4e0 0x7f85bc06b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.980+0000 7f85bb7fe700 1 -- 172.21.15.7:0/880611072 >> v1:172.21.15.7:6789/0 conn(0x7f85bc0cbe30 legacy=0x7f85bc15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.980+0000 7f85bb7fe700 1 -- 172.21.15.7:0/880611072 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc0cca50 con 0x7f85bc0cba60 2024-01-27T02:32:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.980+0000 7f85bbfff700 1 --2- 172.21.15.7:0/880611072 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85bc06b4e0 0x7f85bc06b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:32:35.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.980+0000 7f85bb7fe700 1 --2- 172.21.15.7:0/880611072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85bc0cba60 0x7f85bc0ce410 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0098d0 tx=0x7f85ac009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24626351dd4412ad server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:35.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.980+0000 7f85ba7fc700 1 -- 172.21.15.7:0/880611072 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac005c10 con 0x7f85bc0cba60 2024-01-27T02:32:35.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.980+0000 7f85ba7fc700 1 -- 172.21.15.7:0/880611072 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ac005d70 con 0x7f85bc0cba60 2024-01-27T02:32:35.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.980+0000 7f85ba7fc700 1 -- 172.21.15.7:0/880611072 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac006280 con 0x7f85bc0cba60 2024-01-27T02:32:35.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.981+0000 7f85c1d01700 1 -- 172.21.15.7:0/880611072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85bc0cba60 msgr2=0x7f85bc0ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:35.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.981+0000 7f85c1d01700 1 --2- 172.21.15.7:0/880611072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85bc0cba60 0x7f85bc0ce410 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0098d0 tx=0x7f85ac009120 comp rx=0 tx=0).stop 2024-01-27T02:32:35.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.981+0000 7f85c1d01700 1 -- 172.21.15.7:0/880611072 shutdown_connections 2024-01-27T02:32:35.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.981+0000 7f85c1d01700 1 --2- 172.21.15.7:0/880611072 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85bc06b4e0 0x7f85bc06b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.981+0000 7f85c1d01700 1 --2- 172.21.15.7:0/880611072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85bc0cba60 0x7f85bc0ce410 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.981+0000 7f85c1d01700 1 -- 172.21.15.7:0/880611072 >> 172.21.15.7:0/880611072 conn(0x7f85bc063580 msgr2=0x7f85bc063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:35.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.981+0000 7f85c1d01700 1 -- 172.21.15.7:0/880611072 shutdown_connections 2024-01-27T02:32:35.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.981+0000 7f85c1d01700 1 -- 172.21.15.7:0/880611072 wait complete. 2024-01-27T02:32:35.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.982+0000 7f85c1d01700 1 Processor -- start 2024-01-27T02:32:35.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.982+0000 7f85c1d01700 1 -- start start 2024-01-27T02:32:35.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.983+0000 7f85c1d01700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85bc06b4e0 0x7f85bc0abf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:35.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.983+0000 7f85c1d01700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85bc0cba60 0x7f85bc0ac470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:35.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.983+0000 7f85c1d01700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85bc0cbe30 0x7f85bc16a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:35.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.983+0000 7f85c1d01700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0ab5f0 con 0x7f85bc0cba60 2024-01-27T02:32:35.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.983+0000 7f85c1d01700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0aa9e0 con 0x7f85bc06b4e0 2024-01-27T02:32:35.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.983+0000 7f85c1d01700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0aacc0 con 0x7f85bc0cbe30 2024-01-27T02:32:35.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.983+0000 7f85baffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85bc0cba60 0x7f85bc0ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:35.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.983+0000 7f85baffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85bc0cba60 0x7f85bc0ac470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58144/0 (socket says 172.21.15.7:58144) 2024-01-27T02:32:35.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.983+0000 7f85baffd700 1 -- 172.21.15.7:0/972796724 learned_addr learned my addr 172.21.15.7:0/972796724 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:35.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.983+0000 7f85bbfff700 1 --2- 172.21.15.7:0/972796724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85bc0cbe30 0x7f85bc16a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:35.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.983+0000 7f85bb7fe700 1 --2- 172.21.15.7:0/972796724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85bc06b4e0 0x7f85bc0abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:35.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.984+0000 7f85baffd700 1 -- 172.21.15.7:0/972796724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85bc0cbe30 msgr2=0x7f85bc16a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:35.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.984+0000 7f85baffd700 1 --2- 172.21.15.7:0/972796724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85bc0cbe30 0x7f85bc16a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.984+0000 7f85baffd700 1 -- 172.21.15.7:0/972796724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85bc06b4e0 msgr2=0x7f85bc0abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:35.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.984+0000 7f85baffd700 1 --2- 172.21.15.7:0/972796724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85bc06b4e0 0x7f85bc0abf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:35.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.984+0000 7f85baffd700 1 -- 172.21.15.7:0/972796724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b4009a20 con 0x7f85bc0cba60 2024-01-27T02:32:35.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.984+0000 7f85baffd700 1 --2- 172.21.15.7:0/972796724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85bc0cba60 0x7f85bc0ac470 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f85a400c560 tx=0x7f85a4007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:35.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.984+0000 7f85b37fe700 1 -- 172.21.15.7:0/972796724 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4005c00 con 0x7f85bc0cba60 2024-01-27T02:32:35.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.984+0000 7f85b37fe700 1 -- 172.21.15.7:0/972796724 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a4005d60 con 0x7f85bc0cba60 2024-01-27T02:32:35.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.984+0000 7f85b37fe700 1 -- 172.21.15.7:0/972796724 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4007300 con 0x7f85bc0cba60 2024-01-27T02:32:35.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.984+0000 7f85c1d01700 1 -- 172.21.15.7:0/972796724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ac0093c0 con 0x7f85bc0cba60 2024-01-27T02:32:35.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.984+0000 7f85c1d01700 1 -- 172.21.15.7:0/972796724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85bc0ab280 con 0x7f85bc0cba60 2024-01-27T02:32:35.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.985+0000 7f85c1d01700 1 -- 172.21.15.7:0/972796724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85bc0ab960 con 0x7f85bc0cba60 2024-01-27T02:32:35.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.985+0000 7f85b37fe700 1 -- 172.21.15.7:0/972796724 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f85a40076b0 con 0x7f85bc0cba60 2024-01-27T02:32:35.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.986+0000 7f85b37fe700 1 --2- 172.21.15.7:0/972796724 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f859c066f30 0x7f859c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:35.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.986+0000 7f85b37fe700 1 -- 172.21.15.7:0/972796724 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f85a4083910 con 0x7f85bc0cba60 2024-01-27T02:32:35.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.986+0000 7f85bb7fe700 1 --2- 172.21.15.7:0/972796724 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f859c066f30 0x7f859c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:35.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.987+0000 7f85bb7fe700 1 --2- 172.21.15.7:0/972796724 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f859c066f30 0x7f859c0693e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0ad880 tx=0x7f85ac016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:35.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:35.990+0000 7f85b37fe700 1 -- 172.21.15.7:0/972796724 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f85a4050c40 con 0x7f85bc0cba60 2024-01-27T02:32:36.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.192+0000 7f85c1d01700 1 -- 172.21.15.7:0/972796724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f85bc072060 con 0x7f85bc0cba60 2024-01-27T02:32:36.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.193+0000 7f85b37fe700 1 -- 172.21.15.7:0/972796724 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f85a4017020 con 0x7f85bc0cba60 2024-01-27T02:32:36.193 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-27T02:32:36.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.195+0000 7f85c1d01700 1 -- 172.21.15.7:0/972796724 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f859c066f30 msgr2=0x7f859c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:36.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.195+0000 7f85c1d01700 1 --2- 172.21.15.7:0/972796724 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f859c066f30 0x7f859c0693e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0ad880 tx=0x7f85ac016000 comp rx=0 tx=0).stop 2024-01-27T02:32:36.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.195+0000 7f85c1d01700 1 -- 172.21.15.7:0/972796724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85bc0cba60 msgr2=0x7f85bc0ac470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:36.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.195+0000 7f85c1d01700 1 --2- 172.21.15.7:0/972796724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85bc0cba60 0x7f85bc0ac470 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f85a400c560 tx=0x7f85a4007dd0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.195+0000 7f85c1d01700 1 -- 172.21.15.7:0/972796724 shutdown_connections 2024-01-27T02:32:36.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.195+0000 7f85c1d01700 1 --2- 172.21.15.7:0/972796724 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f859c066f30 0x7f859c0693e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.195+0000 7f85c1d01700 1 --2- 172.21.15.7:0/972796724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85bc0cbe30 0x7f85bc16a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.195+0000 7f85c1d01700 1 --2- 172.21.15.7:0/972796724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85bc06b4e0 0x7f85bc0abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.195+0000 7f85c1d01700 1 --2- 172.21.15.7:0/972796724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85bc0cba60 0x7f85bc0ac470 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.195+0000 7f85c1d01700 1 -- 172.21.15.7:0/972796724 >> 172.21.15.7:0/972796724 conn(0x7f85bc063580 msgr2=0x7f85bc156740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:36.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.195+0000 7f85c1d01700 1 -- 172.21.15.7:0/972796724 shutdown_connections 2024-01-27T02:32:36.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.195+0000 7f85c1d01700 1 -- 172.21.15.7:0/972796724 wait complete. 2024-01-27T02:32:36.209 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-27T02:32:36.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.343+0000 7fe89c9bf700 1 Processor -- start 2024-01-27T02:32:36.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.343+0000 7fe89c9bf700 1 -- start start 2024-01-27T02:32:36.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.344+0000 7fe89c9bf700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8980cf280 0x7fe8980d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:36.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.344+0000 7fe89c9bf700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe89815d830 0x7fe8980cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:36.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.344+0000 7fe89c9bf700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8980b38d0 con 0x7fe89815d460 2024-01-27T02:32:36.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.344+0000 7fe89c9bf700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8980d0390 con 0x7fe89815d830 2024-01-27T02:32:36.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.344+0000 7fe89c9bf700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8980cf780 con 0x7fe8980cf280 2024-01-27T02:32:36.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.344+0000 7fe89659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe89815d460 0x7fe89815e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45576/0 (socket says 172.21.15.7:45576) 2024-01-27T02:32:36.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.344+0000 7fe89659c700 1 -- 172.21.15.7:0/2227553134 learned_addr learned my addr 172.21.15.7:0/2227553134 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:36.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.344+0000 7fe896d9d700 1 --2- 172.21.15.7:0/2227553134 >> v2:172.21.15.188:3300/0 conn(0x7fe89815d830 0x7fe8980cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:36.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.344+0000 7fe88dd9b700 1 --2- 172.21.15.7:0/2227553134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8980cf280 0x7fe8980d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:36.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.344+0000 7fe895d9b700 1 -- 172.21.15.7:0/2227553134 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715739059 0 0) 0x7fe888001110 con 0x7fe89815d460 2024-01-27T02:32:36.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.344+0000 7fe895d9b700 1 -- 172.21.15.7:0/2227553134 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe884003570 con 0x7fe89815d460 2024-01-27T02:32:36.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.345+0000 7fe896d9d700 1 -- 172.21.15.7:0/2227553134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8980cf280 msgr2=0x7fe8980d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:36.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.345+0000 7fe896d9d700 1 --2- 172.21.15.7:0/2227553134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8980cf280 0x7fe8980d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.345+0000 7fe896d9d700 1 -- 172.21.15.7:0/2227553134 >> v1:172.21.15.7:6789/0 conn(0x7fe89815d460 legacy=0x7fe89815e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:36.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.345+0000 7fe896d9d700 1 -- 172.21.15.7:0/2227553134 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8980cfa60 con 0x7fe89815d830 2024-01-27T02:32:36.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.345+0000 7fe896d9d700 1 --2- 172.21.15.7:0/2227553134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe89815d830 0x7fe8980cd700 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe890009e60 tx=0x7fe890009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c43604a05c0019f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:36.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.345+0000 7fe895d9b700 1 -- 172.21.15.7:0/2227553134 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89000e070 con 0x7fe89815d830 2024-01-27T02:32:36.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.345+0000 7fe895d9b700 1 -- 172.21.15.7:0/2227553134 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89000bab0 con 0x7fe89815d830 2024-01-27T02:32:36.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.345+0000 7fe895d9b700 1 -- 172.21.15.7:0/2227553134 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89000be70 con 0x7fe89815d830 2024-01-27T02:32:36.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.346+0000 7fe89c9bf700 1 -- 172.21.15.7:0/2227553134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe89815d830 msgr2=0x7fe8980cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:36.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.346+0000 7fe89c9bf700 1 --2- 172.21.15.7:0/2227553134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe89815d830 0x7fe8980cd700 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe890009e60 tx=0x7fe890009780 comp rx=0 tx=0).stop 2024-01-27T02:32:36.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.346+0000 7fe89c9bf700 1 -- 172.21.15.7:0/2227553134 shutdown_connections 2024-01-27T02:32:36.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.346+0000 7fe89c9bf700 1 --2- 172.21.15.7:0/2227553134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe89815d830 0x7fe8980cd700 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.346+0000 7fe89c9bf700 1 --2- 172.21.15.7:0/2227553134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8980cf280 0x7fe8980d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.346+0000 7fe89c9bf700 1 -- 172.21.15.7:0/2227553134 >> 172.21.15.7:0/2227553134 conn(0x7fe898063580 msgr2=0x7fe898063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:36.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.346+0000 7fe89c9bf700 1 -- 172.21.15.7:0/2227553134 shutdown_connections 2024-01-27T02:32:36.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.346+0000 7fe89c9bf700 1 -- 172.21.15.7:0/2227553134 wait complete. 2024-01-27T02:32:36.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.347+0000 7fe89c9bf700 1 Processor -- start 2024-01-27T02:32:36.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.347+0000 7fe89c9bf700 1 -- start start 2024-01-27T02:32:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.348+0000 7fe89c9bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8980cf280 0x7fe8980c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.348+0000 7fe89c9bf700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe89815d460 0x7fe89814b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.348+0000 7fe89c9bf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe89815d830 0x7fe8980b6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.348+0000 7fe89c9bf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89816a400 con 0x7fe8980cf280 2024-01-27T02:32:36.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.348+0000 7fe89c9bf700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89816a640 con 0x7fe89815d830 2024-01-27T02:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.348+0000 7fe89c9bf700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89816a920 con 0x7fe89815d460 2024-01-27T02:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.348+0000 7fe89659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8980cf280 0x7fe8980c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.348+0000 7fe89659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8980cf280 0x7fe8980c8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58148/0 (socket says 172.21.15.7:58148) 2024-01-27T02:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.348+0000 7fe89659c700 1 -- 172.21.15.7:0/4271332301 learned_addr learned my addr 172.21.15.7:0/4271332301 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.348+0000 7fe896d9d700 1 --2- 172.21.15.7:0/4271332301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe89815d830 0x7fe8980b6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.348+0000 7fe88dd9b700 1 --2- 172.21.15.7:0/4271332301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe89815d460 0x7fe89814b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.348+0000 7fe89659c700 1 -- 172.21.15.7:0/4271332301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe89815d460 msgr2=0x7fe89814b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.348+0000 7fe89659c700 1 --2- 172.21.15.7:0/4271332301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe89815d460 0x7fe89814b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.348+0000 7fe89659c700 1 -- 172.21.15.7:0/4271332301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe89815d830 msgr2=0x7fe8980b6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.348+0000 7fe89659c700 1 --2- 172.21.15.7:0/4271332301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe89815d830 0x7fe8980b6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.348+0000 7fe89659c700 1 -- 172.21.15.7:0/4271332301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe890009a20 con 0x7fe8980cf280 2024-01-27T02:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.348+0000 7fe89659c700 1 --2- 172.21.15.7:0/4271332301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8980cf280 0x7fe8980c8cb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe88800c7c0 tx=0x7fe888007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.349+0000 7fe88effd700 1 -- 172.21.15.7:0/4271332301 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888005b30 con 0x7fe8980cf280 2024-01-27T02:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.349+0000 7fe88effd700 1 -- 172.21.15.7:0/4271332301 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe888005c90 con 0x7fe8980cf280 2024-01-27T02:32:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.349+0000 7fe88effd700 1 -- 172.21.15.7:0/4271332301 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888007340 con 0x7fe8980cf280 2024-01-27T02:32:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.349+0000 7fe89c9bf700 1 -- 172.21.15.7:0/4271332301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89816ac00 con 0x7fe8980cf280 2024-01-27T02:32:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.349+0000 7fe89c9bf700 1 -- 172.21.15.7:0/4271332301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89814bea0 con 0x7fe8980cf280 2024-01-27T02:32:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.350+0000 7fe89c9bf700 1 -- 172.21.15.7:0/4271332301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe89816da10 con 0x7fe8980cf280 2024-01-27T02:32:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.350+0000 7fe88effd700 1 -- 172.21.15.7:0/4271332301 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe8880076f0 con 0x7fe8980cf280 2024-01-27T02:32:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.350+0000 7fe88effd700 1 --2- 172.21.15.7:0/4271332301 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe878066f30 0x7fe8780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:36.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.351+0000 7fe88effd700 1 -- 172.21.15.7:0/4271332301 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe888083e80 con 0x7fe8980cf280 2024-01-27T02:32:36.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.351+0000 7fe88dd9b700 1 --2- 172.21.15.7:0/4271332301 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe878066f30 0x7fe8780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:36.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.351+0000 7fe88dd9b700 1 --2- 172.21.15.7:0/4271332301 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe878066f30 0x7fe8780693e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe8800093f0 tx=0x7fe880008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:36.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.355+0000 7fe88effd700 1 -- 172.21.15.7:0/4271332301 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe888051170 con 0x7fe8980cf280 2024-01-27T02:32:36.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.560+0000 7fe89c9bf700 1 -- 172.21.15.7:0/4271332301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe8980b38d0 con 0x7fe8980cf280 2024-01-27T02:32:36.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.561+0000 7fe88effd700 1 -- 172.21.15.7:0/4271332301 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe8980b38d0 con 0x7fe8980cf280 2024-01-27T02:32:36.566 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-27T02:32:36.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.563+0000 7fe89c9bf700 1 -- 172.21.15.7:0/4271332301 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe878066f30 msgr2=0x7fe8780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:36.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.563+0000 7fe89c9bf700 1 --2- 172.21.15.7:0/4271332301 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe878066f30 0x7fe8780693e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe8800093f0 tx=0x7fe880008040 comp rx=0 tx=0).stop 2024-01-27T02:32:36.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.563+0000 7fe89c9bf700 1 -- 172.21.15.7:0/4271332301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8980cf280 msgr2=0x7fe8980c8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:36.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.563+0000 7fe89c9bf700 1 --2- 172.21.15.7:0/4271332301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8980cf280 0x7fe8980c8cb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe88800c7c0 tx=0x7fe888007c70 comp rx=0 tx=0).stop 2024-01-27T02:32:36.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.563+0000 7fe89c9bf700 1 -- 172.21.15.7:0/4271332301 shutdown_connections 2024-01-27T02:32:36.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.563+0000 7fe89c9bf700 1 --2- 172.21.15.7:0/4271332301 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe878066f30 0x7fe8780693e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.563+0000 7fe89c9bf700 1 --2- 172.21.15.7:0/4271332301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe89815d830 0x7fe8980b6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.563+0000 7fe89c9bf700 1 --2- 172.21.15.7:0/4271332301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8980cf280 0x7fe8980c8cb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.563+0000 7fe89c9bf700 1 --2- 172.21.15.7:0/4271332301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe89815d460 0x7fe89814b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.563+0000 7fe89c9bf700 1 -- 172.21.15.7:0/4271332301 >> 172.21.15.7:0/4271332301 conn(0x7fe898063580 msgr2=0x7fe898063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:36.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.563+0000 7fe89c9bf700 1 -- 172.21.15.7:0/4271332301 shutdown_connections 2024-01-27T02:32:36.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.563+0000 7fe89c9bf700 1 -- 172.21.15.7:0/4271332301 wait complete. 2024-01-27T02:32:36.577 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:32:36.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.710+0000 7f6f8ad00700 1 Processor -- start 2024-01-27T02:32:36.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.710+0000 7f6f8ad00700 1 -- start start 2024-01-27T02:32:36.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.710+0000 7f6f8ad00700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6f84063c90 0x7f6f84158590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:36.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.710+0000 7f6f8ad00700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6f8406adc0 0x7f6f84158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:36.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.710+0000 7f6f8ad00700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f84159570 con 0x7f6f8406a9f0 2024-01-27T02:32:36.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.710+0000 7f6f8ad00700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f84159010 con 0x7f6f84063c90 2024-01-27T02:32:36.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.710+0000 7f6f8ad00700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f84159290 con 0x7f6f8406adc0 2024-01-27T02:32:36.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.711+0000 7f6f83fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6f84063c90 0x7f6f84158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:36.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.711+0000 7f6f88a9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6f8406a9f0 0x7f6f84063580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45590/0 (socket says 172.21.15.7:45590) 2024-01-27T02:32:36.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.711+0000 7f6f88a9c700 1 -- 172.21.15.7:0/56245888 learned_addr learned my addr 172.21.15.7:0/56245888 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.711+0000 7f6f8929d700 1 --2- 172.21.15.7:0/56245888 >> v2:172.21.15.188:3300/0 conn(0x7f6f8406adc0 0x7f6f84158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.711+0000 7f6f83fff700 1 -- 172.21.15.7:0/56245888 >> v2:172.21.15.188:3300/0 conn(0x7f6f8406adc0 msgr2=0x7f6f84158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.711+0000 7f6f83fff700 1 --2- 172.21.15.7:0/56245888 >> v2:172.21.15.188:3300/0 conn(0x7f6f8406adc0 0x7f6f84158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.711+0000 7f6f83fff700 1 -- 172.21.15.7:0/56245888 >> v1:172.21.15.7:6789/0 conn(0x7f6f8406a9f0 legacy=0x7f6f84063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.711+0000 7f6f83fff700 1 -- 172.21.15.7:0/56245888 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8415bd00 con 0x7f6f84063c90 2024-01-27T02:32:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.711+0000 7f6f837fe700 1 -- 172.21.15.7:0/56245888 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623426736 0 0) 0x7f6f78001110 con 0x7f6f8406a9f0 2024-01-27T02:32:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.711+0000 7f6f83fff700 1 --2- 172.21.15.7:0/56245888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f84063c90 0x7f6f84158590 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6f840b0770 tx=0x7f6f700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b63c8be9aae261 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.711+0000 7f6f837fe700 1 -- 172.21.15.7:0/56245888 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70011630 con 0x7f6f84063c90 2024-01-27T02:32:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.711+0000 7f6f837fe700 1 -- 172.21.15.7:0/56245888 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f70011790 con 0x7f6f84063c90 2024-01-27T02:32:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.711+0000 7f6f837fe700 1 -- 172.21.15.7:0/56245888 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70011b30 con 0x7f6f84063c90 2024-01-27T02:32:36.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.712+0000 7f6f8ad00700 1 -- 172.21.15.7:0/56245888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f84063c90 msgr2=0x7f6f84158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:36.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.712+0000 7f6f8ad00700 1 --2- 172.21.15.7:0/56245888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f84063c90 0x7f6f84158590 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6f840b0770 tx=0x7f6f700091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.712+0000 7f6f8ad00700 1 -- 172.21.15.7:0/56245888 shutdown_connections 2024-01-27T02:32:36.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.712+0000 7f6f8ad00700 1 --2- 172.21.15.7:0/56245888 >> v2:172.21.15.188:3300/0 conn(0x7f6f8406adc0 0x7f6f84158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.712+0000 7f6f8ad00700 1 --2- 172.21.15.7:0/56245888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f84063c90 0x7f6f84158590 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.712+0000 7f6f8ad00700 1 -- 172.21.15.7:0/56245888 >> 172.21.15.7:0/56245888 conn(0x7f6f840cdc30 msgr2=0x7f6f840ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.712+0000 7f6f8ad00700 1 -- 172.21.15.7:0/56245888 shutdown_connections 2024-01-27T02:32:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.712+0000 7f6f8ad00700 1 -- 172.21.15.7:0/56245888 wait complete. 2024-01-27T02:32:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.713+0000 7f6f8ad00700 1 Processor -- start 2024-01-27T02:32:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.714+0000 7f6f8ad00700 1 -- start start 2024-01-27T02:32:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.714+0000 7f6f8ad00700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f84063c90 0x7f6f84138440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.714+0000 7f6f8ad00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f8406a9f0 0x7f6f840b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.714+0000 7f6f8ad00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f8406adc0 0x7f6f841502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.714+0000 7f6f8ad00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f84168000 con 0x7f6f8406adc0 2024-01-27T02:32:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.714+0000 7f6f8ad00700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f841681f0 con 0x7f6f8406a9f0 2024-01-27T02:32:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.714+0000 7f6f8ad00700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f841684d0 con 0x7f6f84063c90 2024-01-27T02:32:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.715+0000 7f6f8929d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f8406adc0 0x7f6f841502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.715+0000 7f6f8929d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f8406adc0 0x7f6f841502c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58176/0 (socket says 172.21.15.7:58176) 2024-01-27T02:32:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.715+0000 7f6f8929d700 1 -- 172.21.15.7:0/1698130911 learned_addr learned my addr 172.21.15.7:0/1698130911 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.715+0000 7f6f88a9c700 1 --2- 172.21.15.7:0/1698130911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f84063c90 0x7f6f84138440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.715+0000 7f6f83fff700 1 --2- 172.21.15.7:0/1698130911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f8406a9f0 0x7f6f840b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.715+0000 7f6f8929d700 1 -- 172.21.15.7:0/1698130911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f84063c90 msgr2=0x7f6f84138440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.715+0000 7f6f8929d700 1 --2- 172.21.15.7:0/1698130911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f84063c90 0x7f6f84138440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.715+0000 7f6f8929d700 1 -- 172.21.15.7:0/1698130911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f8406a9f0 msgr2=0x7f6f840b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.715+0000 7f6f8929d700 1 --2- 172.21.15.7:0/1698130911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f8406a9f0 0x7f6f840b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.715+0000 7f6f8929d700 1 -- 172.21.15.7:0/1698130911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f70009420 con 0x7f6f8406adc0 2024-01-27T02:32:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.715+0000 7f6f8929d700 1 --2- 172.21.15.7:0/1698130911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f8406adc0 0x7f6f841502c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009e90 tx=0x7f6f7c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.715+0000 7f6f80ff9700 1 -- 172.21.15.7:0/1698130911 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c00cb70 con 0x7f6f8406adc0 2024-01-27T02:32:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.715+0000 7f6f80ff9700 1 -- 172.21.15.7:0/1698130911 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7c00f040 con 0x7f6f8406adc0 2024-01-27T02:32:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.715+0000 7f6f8ad00700 1 -- 172.21.15.7:0/1698130911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f841687b0 con 0x7f6f8406adc0 2024-01-27T02:32:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.716+0000 7f6f80ff9700 1 -- 172.21.15.7:0/1698130911 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c014480 con 0x7f6f8406adc0 2024-01-27T02:32:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.716+0000 7f6f8ad00700 1 -- 172.21.15.7:0/1698130911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f84168d70 con 0x7f6f8406adc0 2024-01-27T02:32:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.717+0000 7f6f8ad00700 1 -- 172.21.15.7:0/1698130911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f84168f90 con 0x7f6f8406adc0 2024-01-27T02:32:36.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.717+0000 7f6f80ff9700 1 -- 172.21.15.7:0/1698130911 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6f7c014830 con 0x7f6f8406adc0 2024-01-27T02:32:36.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.717+0000 7f6f80ff9700 1 --2- 172.21.15.7:0/1698130911 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6f68066f60 0x7f6f68069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:36.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.717+0000 7f6f80ff9700 1 -- 172.21.15.7:0/1698130911 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6f7c085240 con 0x7f6f8406adc0 2024-01-27T02:32:36.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.718+0000 7f6f88a9c700 1 --2- 172.21.15.7:0/1698130911 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6f68066f60 0x7f6f68069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:36.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.718+0000 7f6f88a9c700 1 --2- 172.21.15.7:0/1698130911 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6f68066f60 0x7f6f68069410 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6f8413a4a0 tx=0x7f6f78007b90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:36.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.722+0000 7f6f80ff9700 1 -- 172.21.15.7:0/1698130911 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f7c052410 con 0x7f6f8406adc0 2024-01-27T02:32:36.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.927+0000 7f6f8ad00700 1 -- 172.21.15.7:0/1698130911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f8404c710 con 0x7f6f8406adc0 2024-01-27T02:32:36.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.928+0000 7f6f80ff9700 1 -- 172.21.15.7:0/1698130911 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f6f7c055a30 con 0x7f6f8406adc0 2024-01-27T02:32:36.928 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:36.929 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:32:07.627822+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:32:03.675611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-27T02:32:05.018885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:32:36.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.930+0000 7f6f8ad00700 1 -- 172.21.15.7:0/1698130911 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6f68066f60 msgr2=0x7f6f68069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:36.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.930+0000 7f6f8ad00700 1 --2- 172.21.15.7:0/1698130911 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6f68066f60 0x7f6f68069410 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6f8413a4a0 tx=0x7f6f78007b90 comp rx=0 tx=0).stop 2024-01-27T02:32:36.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.930+0000 7f6f8ad00700 1 -- 172.21.15.7:0/1698130911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f8406adc0 msgr2=0x7f6f841502c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:36.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.930+0000 7f6f8ad00700 1 --2- 172.21.15.7:0/1698130911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f8406adc0 0x7f6f841502c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009e90 tx=0x7f6f7c00b040 comp rx=0 tx=0).stop 2024-01-27T02:32:36.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.930+0000 7f6f8ad00700 1 -- 172.21.15.7:0/1698130911 shutdown_connections 2024-01-27T02:32:36.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.930+0000 7f6f8ad00700 1 --2- 172.21.15.7:0/1698130911 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6f68066f60 0x7f6f68069410 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.930+0000 7f6f8ad00700 1 --2- 172.21.15.7:0/1698130911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f8406adc0 0x7f6f841502c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.930+0000 7f6f8ad00700 1 --2- 172.21.15.7:0/1698130911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f84063c90 0x7f6f84138440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.930+0000 7f6f8ad00700 1 --2- 172.21.15.7:0/1698130911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f8406a9f0 0x7f6f840b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:36.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.930+0000 7f6f8ad00700 1 -- 172.21.15.7:0/1698130911 >> 172.21.15.7:0/1698130911 conn(0x7f6f840cdc30 msgr2=0x7f6f8413a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:36.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.930+0000 7f6f8ad00700 1 -- 172.21.15.7:0/1698130911 shutdown_connections 2024-01-27T02:32:36.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:36.930+0000 7f6f8ad00700 1 -- 172.21.15.7:0/1698130911 wait complete. 2024-01-27T02:32:36.945 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-27T02:32:03.675611+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-27T02:32:05.018885+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-27T02:32:36.946 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:32:37.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.080+0000 7f9c46e1a700 1 Processor -- start 2024-01-27T02:32:37.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.080+0000 7f9c46e1a700 1 -- start start 2024-01-27T02:32:37.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.081+0000 7f9c46e1a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c4015bfe0 0x7f9c400cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:37.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.081+0000 7f9c46e1a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c401583e0 0x7f9c400aef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:37.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.081+0000 7f9c46e1a700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c400cd0d0 con 0x7f9c4015c9b0 2024-01-27T02:32:37.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.081+0000 7f9c46e1a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c400cc4c0 con 0x7f9c401583e0 2024-01-27T02:32:37.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.081+0000 7f9c46e1a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c400cc7a0 con 0x7f9c4015bfe0 2024-01-27T02:32:37.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.081+0000 7f9c453b7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c401583e0 0x7f9c400aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:37.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.081+0000 7f9c453b7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c401583e0 0x7f9c400aef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58184/0 (socket says 172.21.15.7:58184) 2024-01-27T02:32:37.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.081+0000 7f9c453b7700 1 -- 172.21.15.7:0/3031272259 learned_addr learned my addr 172.21.15.7:0/3031272259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:37.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.081+0000 7f9c44bb6700 1 --2- 172.21.15.7:0/3031272259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c4015bfe0 0x7f9c400cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:37.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.082+0000 7f9c453b7700 1 -- 172.21.15.7:0/3031272259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c4015bfe0 msgr2=0x7f9c400cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:37.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.082+0000 7f9c453b7700 1 --2- 172.21.15.7:0/3031272259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c4015bfe0 0x7f9c400cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:37.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.082+0000 7f9c453b7700 1 -- 172.21.15.7:0/3031272259 >> v1:172.21.15.188:6789/0 conn(0x7f9c4015c9b0 legacy=0x7f9c40157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:37.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.082+0000 7f9c453b7700 1 -- 172.21.15.7:0/3031272259 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c400cca80 con 0x7f9c401583e0 2024-01-27T02:32:37.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.082+0000 7f9c3f7fe700 1 -- 172.21.15.7:0/3031272259 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3925474252 0 0) 0x7f9c400cd0d0 con 0x7f9c4015c9b0 2024-01-27T02:32:37.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.082+0000 7f9c453b7700 1 --2- 172.21.15.7:0/3031272259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c401583e0 0x7f9c400aef20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009f20 tx=0x7f9c38009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf51f5d8de8280e0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:37.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.082+0000 7f9c3f7fe700 1 -- 172.21.15.7:0/3031272259 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3800e070 con 0x7f9c401583e0 2024-01-27T02:32:37.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.082+0000 7f9c3f7fe700 1 -- 172.21.15.7:0/3031272259 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3800b9d0 con 0x7f9c401583e0 2024-01-27T02:32:37.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.082+0000 7f9c3f7fe700 1 -- 172.21.15.7:0/3031272259 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c380092e0 con 0x7f9c401583e0 2024-01-27T02:32:37.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.083+0000 7f9c46e1a700 1 -- 172.21.15.7:0/3031272259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c401583e0 msgr2=0x7f9c400aef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:37.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.083+0000 7f9c46e1a700 1 --2- 172.21.15.7:0/3031272259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c401583e0 0x7f9c400aef20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009f20 tx=0x7f9c38009840 comp rx=0 tx=0).stop 2024-01-27T02:32:37.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.083+0000 7f9c46e1a700 1 -- 172.21.15.7:0/3031272259 shutdown_connections 2024-01-27T02:32:37.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.083+0000 7f9c46e1a700 1 --2- 172.21.15.7:0/3031272259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c401583e0 0x7f9c400aef20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:37.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.083+0000 7f9c46e1a700 1 --2- 172.21.15.7:0/3031272259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c4015bfe0 0x7f9c400cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:37.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.083+0000 7f9c46e1a700 1 -- 172.21.15.7:0/3031272259 >> 172.21.15.7:0/3031272259 conn(0x7f9c400679b0 msgr2=0x7f9c40067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:37.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.083+0000 7f9c46e1a700 1 -- 172.21.15.7:0/3031272259 shutdown_connections 2024-01-27T02:32:37.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.083+0000 7f9c46e1a700 1 -- 172.21.15.7:0/3031272259 wait complete. 2024-01-27T02:32:37.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.084+0000 7f9c46e1a700 1 Processor -- start 2024-01-27T02:32:37.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.084+0000 7f9c46e1a700 1 -- start start 2024-01-27T02:32:37.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.085+0000 7f9c46e1a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c401583e0 0x7f9c40169fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:37.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.085+0000 7f9c46e1a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c4015bfe0 0x7f9c4016dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:37.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.085+0000 7f9c46e1a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c4015c9b0 0x7f9c400c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.085+0000 7f9c46e1a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4016e4f0 con 0x7f9c401583e0 2024-01-27T02:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.085+0000 7f9c46e1a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4016af70 con 0x7f9c4015bfe0 2024-01-27T02:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.085+0000 7f9c46e1a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4016a6d0 con 0x7f9c4015c9b0 2024-01-27T02:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.085+0000 7f9c44bb6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c401583e0 0x7f9c40169fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.085+0000 7f9c44bb6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c401583e0 0x7f9c40169fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58198/0 (socket says 172.21.15.7:58198) 2024-01-27T02:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.085+0000 7f9c44bb6700 1 -- 172.21.15.7:0/2027016550 learned_addr learned my addr 172.21.15.7:0/2027016550 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.085+0000 7f9c3ffff700 1 --2- 172.21.15.7:0/2027016550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c4015bfe0 0x7f9c4016dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.085+0000 7f9c453b7700 1 --2- 172.21.15.7:0/2027016550 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c4015c9b0 0x7f9c400c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.085+0000 7f9c44bb6700 1 -- 172.21.15.7:0/2027016550 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c4015c9b0 msgr2=0x7f9c400c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.085+0000 7f9c44bb6700 1 --2- 172.21.15.7:0/2027016550 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c4015c9b0 0x7f9c400c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.085+0000 7f9c44bb6700 1 -- 172.21.15.7:0/2027016550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c4015bfe0 msgr2=0x7f9c4016dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.085+0000 7f9c44bb6700 1 --2- 172.21.15.7:0/2027016550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c4015bfe0 0x7f9c4016dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.085+0000 7f9c44bb6700 1 -- 172.21.15.7:0/2027016550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c38009ae0 con 0x7f9c401583e0 2024-01-27T02:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.085+0000 7f9c3ffff700 1 --2- 172.21.15.7:0/2027016550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c4015bfe0 0x7f9c4016dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:32:37.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.086+0000 7f9c44bb6700 1 --2- 172.21.15.7:0/2027016550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c401583e0 0x7f9c40169fd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000ea00 tx=0x7f9c30009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:37.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.086+0000 7f9c3cff9700 1 -- 172.21.15.7:0/2027016550 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000c850 con 0x7f9c401583e0 2024-01-27T02:32:37.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.086+0000 7f9c3cff9700 1 -- 172.21.15.7:0/2027016550 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3000c9b0 con 0x7f9c401583e0 2024-01-27T02:32:37.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.086+0000 7f9c46e1a700 1 -- 172.21.15.7:0/2027016550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c4016a9b0 con 0x7f9c401583e0 2024-01-27T02:32:37.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.086+0000 7f9c3cff9700 1 -- 172.21.15.7:0/2027016550 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000cd30 con 0x7f9c401583e0 2024-01-27T02:32:37.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.086+0000 7f9c46e1a700 1 -- 172.21.15.7:0/2027016550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c400c8b10 con 0x7f9c401583e0 2024-01-27T02:32:37.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.087+0000 7f9c46e1a700 1 -- 172.21.15.7:0/2027016550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c400c8970 con 0x7f9c401583e0 2024-01-27T02:32:37.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.088+0000 7f9c3cff9700 1 -- 172.21.15.7:0/2027016550 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9c30013680 con 0x7f9c401583e0 2024-01-27T02:32:37.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.088+0000 7f9c3cff9700 1 --2- 172.21.15.7:0/2027016550 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9c2c066f30 0x7f9c2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:37.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.088+0000 7f9c3cff9700 1 -- 172.21.15.7:0/2027016550 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9c30084bb0 con 0x7f9c401583e0 2024-01-27T02:32:37.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.088+0000 7f9c3ffff700 1 --2- 172.21.15.7:0/2027016550 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9c2c066f30 0x7f9c2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:37.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.089+0000 7f9c3ffff700 1 --2- 172.21.15.7:0/2027016550 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9c2c066f30 0x7f9c2c0693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9c340011e0 tx=0x7f9c34009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:37.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.092+0000 7f9c3cff9700 1 -- 172.21.15.7:0/2027016550 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c400c8970 con 0x7f9c401583e0 2024-01-27T02:32:37.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.293+0000 7f9c46e1a700 1 -- 172.21.15.7:0/2027016550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9c400c8970 con 0x7f9c401583e0 2024-01-27T02:32:37.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.294+0000 7f9c3cff9700 1 -- 172.21.15.7:0/2027016550 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9c400c8970 con 0x7f9c401583e0 2024-01-27T02:32:37.295 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:32:37.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.296+0000 7f9c46e1a700 1 -- 172.21.15.7:0/2027016550 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9c2c066f30 msgr2=0x7f9c2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:37.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.296+0000 7f9c46e1a700 1 --2- 172.21.15.7:0/2027016550 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9c2c066f30 0x7f9c2c0693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9c340011e0 tx=0x7f9c34009040 comp rx=0 tx=0).stop 2024-01-27T02:32:37.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.296+0000 7f9c46e1a700 1 -- 172.21.15.7:0/2027016550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c401583e0 msgr2=0x7f9c40169fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:37.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.296+0000 7f9c46e1a700 1 --2- 172.21.15.7:0/2027016550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c401583e0 0x7f9c40169fd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000ea00 tx=0x7f9c30009120 comp rx=0 tx=0).stop 2024-01-27T02:32:37.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.296+0000 7f9c46e1a700 1 -- 172.21.15.7:0/2027016550 shutdown_connections 2024-01-27T02:32:37.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.296+0000 7f9c46e1a700 1 --2- 172.21.15.7:0/2027016550 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9c2c066f30 0x7f9c2c0693e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:37.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.296+0000 7f9c46e1a700 1 --2- 172.21.15.7:0/2027016550 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c4015c9b0 0x7f9c400c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:37.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.296+0000 7f9c46e1a700 1 --2- 172.21.15.7:0/2027016550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c401583e0 0x7f9c40169fd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:37.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.296+0000 7f9c46e1a700 1 --2- 172.21.15.7:0/2027016550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c4015bfe0 0x7f9c4016dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:37.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.296+0000 7f9c46e1a700 1 -- 172.21.15.7:0/2027016550 >> 172.21.15.7:0/2027016550 conn(0x7f9c400679b0 msgr2=0x7f9c401566f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:37.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.296+0000 7f9c46e1a700 1 -- 172.21.15.7:0/2027016550 shutdown_connections 2024-01-27T02:32:37.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.296+0000 7f9c46e1a700 1 -- 172.21.15.7:0/2027016550 wait complete. 2024-01-27T02:32:37.311 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-27T02:32:37.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.445+0000 7f5b1b77b700 1 Processor -- start 2024-01-27T02:32:37.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.445+0000 7f5b1b77b700 1 -- start start 2024-01-27T02:32:37.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.446+0000 7f5b1b77b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b140cbaf0 0x7f5b1415f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:37.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.446+0000 7f5b1b77b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b1415fdf0 con 0x7f5b1406bbf0 2024-01-27T02:32:37.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.446+0000 7f5b1b77b700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b140ccc50 con 0x7f5b140cbec0 2024-01-27T02:32:37.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.446+0000 7f5b1b77b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1415fc60 con 0x7f5b140cbaf0 2024-01-27T02:32:37.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.446+0000 7f5b19d18700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5b1406bbf0 0x7f5b140679b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45602/0 (socket says 172.21.15.7:45602) 2024-01-27T02:32:37.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.446+0000 7f5b19d18700 1 -- 172.21.15.7:0/1111044561 learned_addr learned my addr 172.21.15.7:0/1111044561 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:37.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.447+0000 7f5b19517700 1 --2- 172.21.15.7:0/1111044561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b140cbaf0 0x7f5b1415f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:37.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.447+0000 7f5b07fff700 1 -- 172.21.15.7:0/1111044561 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1671381458 0 0) 0x7f5b1415fdf0 con 0x7f5b1406bbf0 2024-01-27T02:32:37.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.447+0000 7f5b07fff700 1 -- 172.21.15.7:0/1111044561 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b00003570 con 0x7f5b1406bbf0 2024-01-27T02:32:37.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.447+0000 7f5b07fff700 1 -- 172.21.15.7:0/1111044561 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983202086 0 0) 0x7f5b140ccc50 con 0x7f5b140cbec0 2024-01-27T02:32:37.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.447+0000 7f5b07fff700 1 -- 172.21.15.7:0/1111044561 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b1415fdf0 con 0x7f5b140cbec0 2024-01-27T02:32:37.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.447+0000 7f5b19517700 1 -- 172.21.15.7:0/1111044561 >> v1:172.21.15.188:6789/0 conn(0x7f5b140cbec0 legacy=0x7f5b1406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:37.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.447+0000 7f5b19517700 1 -- 172.21.15.7:0/1111044561 >> v1:172.21.15.7:6789/0 conn(0x7f5b1406bbf0 legacy=0x7f5b140679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:37.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.447+0000 7f5b19517700 1 -- 172.21.15.7:0/1111044561 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b140cc950 con 0x7f5b140cbaf0 2024-01-27T02:32:37.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.447+0000 7f5b19517700 1 --2- 172.21.15.7:0/1111044561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b140cbaf0 0x7f5b1415f650 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009900 tx=0x7f5b08009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cecc28ef3ff986b2 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:37.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.447+0000 7f5b07fff700 1 -- 172.21.15.7:0/1111044561 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b08005c10 con 0x7f5b140cbaf0 2024-01-27T02:32:37.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.447+0000 7f5b07fff700 1 -- 172.21.15.7:0/1111044561 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b08005d70 con 0x7f5b140cbaf0 2024-01-27T02:32:37.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.447+0000 7f5b07fff700 1 -- 172.21.15.7:0/1111044561 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b08006260 con 0x7f5b140cbaf0 2024-01-27T02:32:37.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.448+0000 7f5b1b77b700 1 -- 172.21.15.7:0/1111044561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b140cbaf0 msgr2=0x7f5b1415f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:37.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.448+0000 7f5b1b77b700 1 --2- 172.21.15.7:0/1111044561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b140cbaf0 0x7f5b1415f650 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009900 tx=0x7f5b08009120 comp rx=0 tx=0).stop 2024-01-27T02:32:37.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.448+0000 7f5b1b77b700 1 -- 172.21.15.7:0/1111044561 shutdown_connections 2024-01-27T02:32:37.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.448+0000 7f5b1b77b700 1 --2- 172.21.15.7:0/1111044561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b140cbaf0 0x7f5b1415f650 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:37.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.448+0000 7f5b1b77b700 1 -- 172.21.15.7:0/1111044561 >> 172.21.15.7:0/1111044561 conn(0x7f5b14063580 msgr2=0x7f5b14063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:37.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.448+0000 7f5b1b77b700 1 -- 172.21.15.7:0/1111044561 shutdown_connections 2024-01-27T02:32:37.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.448+0000 7f5b1b77b700 1 -- 172.21.15.7:0/1111044561 wait complete. 2024-01-27T02:32:37.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.450+0000 7f5b1b77b700 1 Processor -- start 2024-01-27T02:32:37.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.450+0000 7f5b1b77b700 1 -- start start 2024-01-27T02:32:37.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.451+0000 7f5b1b77b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b1406bbf0 0x7f5b14153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:37.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.451+0000 7f5b1b77b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b140cbaf0 0x7f5b140c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:37.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.451+0000 7f5b1b77b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b140cbec0 0x7f5b140ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:37.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.451+0000 7f5b1b77b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b14154150 con 0x7f5b140cbec0 2024-01-27T02:32:37.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.451+0000 7f5b1b77b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140b2b80 con 0x7f5b140cbaf0 2024-01-27T02:32:37.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.451+0000 7f5b1b77b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140b2e60 con 0x7f5b1406bbf0 2024-01-27T02:32:37.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.451+0000 7f5b19d18700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b140cbec0 0x7f5b140ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:37.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.451+0000 7f5b19d18700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b140cbec0 0x7f5b140ca470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58202/0 (socket says 172.21.15.7:58202) 2024-01-27T02:32:37.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.451+0000 7f5b19d18700 1 -- 172.21.15.7:0/4243929350 learned_addr learned my addr 172.21.15.7:0/4243929350 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:37.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.451+0000 7f5b18d16700 1 --2- 172.21.15.7:0/4243929350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b140cbaf0 0x7f5b140c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:37.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.451+0000 7f5b19517700 1 --2- 172.21.15.7:0/4243929350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b1406bbf0 0x7f5b14153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:37.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.451+0000 7f5b19d18700 1 -- 172.21.15.7:0/4243929350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b1406bbf0 msgr2=0x7f5b14153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:37.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.451+0000 7f5b19d18700 1 --2- 172.21.15.7:0/4243929350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b1406bbf0 0x7f5b14153c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:37.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.451+0000 7f5b19d18700 1 -- 172.21.15.7:0/4243929350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b140cbaf0 msgr2=0x7f5b140c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:37.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.451+0000 7f5b19d18700 1 --2- 172.21.15.7:0/4243929350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b140cbaf0 0x7f5b140c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:37.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.451+0000 7f5b19d18700 1 -- 172.21.15.7:0/4243929350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b080093c0 con 0x7f5b140cbec0 2024-01-27T02:32:37.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.451+0000 7f5b19d18700 1 --2- 172.21.15.7:0/4243929350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b140cbec0 0x7f5b140ca470 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00cc60 tx=0x7f5b0c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:37.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.452+0000 7f5b057fa700 1 -- 172.21.15.7:0/4243929350 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c00ce70 con 0x7f5b140cbec0 2024-01-27T02:32:37.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.452+0000 7f5b057fa700 1 -- 172.21.15.7:0/4243929350 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c00ab70 con 0x7f5b140cbec0 2024-01-27T02:32:37.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.452+0000 7f5b1b77b700 1 -- 172.21.15.7:0/4243929350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b140b3140 con 0x7f5b140cbec0 2024-01-27T02:32:37.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.452+0000 7f5b057fa700 1 -- 172.21.15.7:0/4243929350 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0070f0 con 0x7f5b140cbec0 2024-01-27T02:32:37.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.452+0000 7f5b1b77b700 1 -- 172.21.15.7:0/4243929350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b140b3700 con 0x7f5b140cbec0 2024-01-27T02:32:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.453+0000 7f5b1b77b700 1 -- 172.21.15.7:0/4243929350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b14072060 con 0x7f5b140cbec0 2024-01-27T02:32:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.453+0000 7f5b057fa700 1 -- 172.21.15.7:0/4243929350 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5b0c0074a0 con 0x7f5b140cbec0 2024-01-27T02:32:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.453+0000 7f5b057fa700 1 --2- 172.21.15.7:0/4243929350 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5af8066f30 0x7f5af80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.454+0000 7f5b057fa700 1 -- 172.21.15.7:0/4243929350 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5b0c0849d0 con 0x7f5b140cbec0 2024-01-27T02:32:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.454+0000 7f5b19517700 1 --2- 172.21.15.7:0/4243929350 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5af8066f30 0x7f5af80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.454+0000 7f5b19517700 1 --2- 172.21.15.7:0/4243929350 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5af8066f30 0x7f5af80693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009160 tx=0x7f5b08016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:37.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.458+0000 7f5b057fa700 1 -- 172.21.15.7:0/4243929350 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b0c055750 con 0x7f5b140cbec0 2024-01-27T02:32:37.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.661+0000 7f5b1b77b700 1 -- 172.21.15.7:0/4243929350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5b140b38f0 con 0x7f5b140cbec0 2024-01-27T02:32:37.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.662+0000 7f5b057fa700 1 -- 172.21.15.7:0/4243929350 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5b0c0552e0 con 0x7f5b140cbec0 2024-01-27T02:32:37.662 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-27T02:32:37.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.664+0000 7f5b1b77b700 1 -- 172.21.15.7:0/4243929350 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5af8066f30 msgr2=0x7f5af80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:37.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.664+0000 7f5b1b77b700 1 --2- 172.21.15.7:0/4243929350 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5af8066f30 0x7f5af80693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009160 tx=0x7f5b08016000 comp rx=0 tx=0).stop 2024-01-27T02:32:37.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.664+0000 7f5b1b77b700 1 -- 172.21.15.7:0/4243929350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b140cbec0 msgr2=0x7f5b140ca470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:37.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.664+0000 7f5b1b77b700 1 --2- 172.21.15.7:0/4243929350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b140cbec0 0x7f5b140ca470 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00cc60 tx=0x7f5b0c001ae0 comp rx=0 tx=0).stop 2024-01-27T02:32:37.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.664+0000 7f5b1b77b700 1 -- 172.21.15.7:0/4243929350 shutdown_connections 2024-01-27T02:32:37.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.664+0000 7f5b1b77b700 1 --2- 172.21.15.7:0/4243929350 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5af8066f30 0x7f5af80693e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:37.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.664+0000 7f5b1b77b700 1 --2- 172.21.15.7:0/4243929350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b140cbec0 0x7f5b140ca470 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:37.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.664+0000 7f5b1b77b700 1 --2- 172.21.15.7:0/4243929350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b1406bbf0 0x7f5b14153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:37.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.664+0000 7f5b1b77b700 1 --2- 172.21.15.7:0/4243929350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b140cbaf0 0x7f5b140c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:37.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.664+0000 7f5b1b77b700 1 -- 172.21.15.7:0/4243929350 >> 172.21.15.7:0/4243929350 conn(0x7f5b14063580 msgr2=0x7f5b140694a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:37.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.664+0000 7f5b1b77b700 1 -- 172.21.15.7:0/4243929350 shutdown_connections 2024-01-27T02:32:37.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.664+0000 7f5b1b77b700 1 -- 172.21.15.7:0/4243929350 wait complete. 2024-01-27T02:32:37.678 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-27T02:32:37.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.811+0000 7f18509e4700 1 Processor -- start 2024-01-27T02:32:37.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.811+0000 7f18509e4700 1 -- start start 2024-01-27T02:32:37.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.812+0000 7f18509e4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f184c1419e0 0x7f184c0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:37.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.812+0000 7f18509e4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f184c0c9660 con 0x7f184c0cb950 2024-01-27T02:32:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.812+0000 7f18509e4700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f184c15be00 con 0x7f184c0cbd20 2024-01-27T02:32:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.812+0000 7f18509e4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f184c15c980 con 0x7f184c1419e0 2024-01-27T02:32:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.812+0000 7f184a59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f184c0cb950 0x7f184c0ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45612/0 (socket says 172.21.15.7:45612) 2024-01-27T02:32:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.812+0000 7f184ad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f184c1419e0 0x7f184c0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.812+0000 7f184a59c700 1 -- 172.21.15.7:0/1677726538 learned_addr learned my addr 172.21.15.7:0/1677726538 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.813+0000 7f184959a700 1 -- 172.21.15.7:0/1677726538 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 694758376 0 0) 0x7f184c0c9660 con 0x7f184c0cb950 2024-01-27T02:32:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.813+0000 7f184959a700 1 -- 172.21.15.7:0/1677726538 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1838003570 con 0x7f184c0cb950 2024-01-27T02:32:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.813+0000 7f184ad9d700 1 -- 172.21.15.7:0/1677726538 >> v1:172.21.15.188:6789/0 conn(0x7f184c0cbd20 legacy=0x7f184c1412d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.813+0000 7f184ad9d700 1 -- 172.21.15.7:0/1677726538 >> v1:172.21.15.7:6789/0 conn(0x7f184c0cb950 legacy=0x7f184c0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.813+0000 7f184ad9d700 1 -- 172.21.15.7:0/1677726538 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184c15c0e0 con 0x7f184c1419e0 2024-01-27T02:32:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.813+0000 7f184ad9d700 1 --2- 172.21.15.7:0/1677726538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f184c1419e0 0x7f184c0b1ac0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1844009e60 tx=0x7f1844009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abf50cec1a7b656 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.813+0000 7f184959a700 1 -- 172.21.15.7:0/1677726538 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184400e070 con 0x7f184c1419e0 2024-01-27T02:32:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.813+0000 7f184959a700 1 -- 172.21.15.7:0/1677726538 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184400bb60 con 0x7f184c1419e0 2024-01-27T02:32:37.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.813+0000 7f184959a700 1 -- 172.21.15.7:0/1677726538 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1844012430 con 0x7f184c1419e0 2024-01-27T02:32:37.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.814+0000 7f18509e4700 1 -- 172.21.15.7:0/1677726538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f184c1419e0 msgr2=0x7f184c0b1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:37.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.814+0000 7f18509e4700 1 --2- 172.21.15.7:0/1677726538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f184c1419e0 0x7f184c0b1ac0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1844009e60 tx=0x7f1844009780 comp rx=0 tx=0).stop 2024-01-27T02:32:37.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.814+0000 7f18509e4700 1 -- 172.21.15.7:0/1677726538 shutdown_connections 2024-01-27T02:32:37.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.814+0000 7f18509e4700 1 --2- 172.21.15.7:0/1677726538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f184c1419e0 0x7f184c0b1ac0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:37.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.814+0000 7f18509e4700 1 -- 172.21.15.7:0/1677726538 >> 172.21.15.7:0/1677726538 conn(0x7f184c063580 msgr2=0x7f184c063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:37.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.814+0000 7f18509e4700 1 -- 172.21.15.7:0/1677726538 shutdown_connections 2024-01-27T02:32:37.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.814+0000 7f18509e4700 1 -- 172.21.15.7:0/1677726538 wait complete. 2024-01-27T02:32:37.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.815+0000 7f18509e4700 1 Processor -- start 2024-01-27T02:32:37.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.815+0000 7f18509e4700 1 -- start start 2024-01-27T02:32:37.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.816+0000 7f18509e4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f184c0cb950 0x7f184c140610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:37.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.816+0000 7f18509e4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f184c0cbd20 0x7f184c0c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:37.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.816+0000 7f18509e4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f184c1419e0 0x7f184c0b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:37.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.816+0000 7f18509e4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184c15b690 con 0x7f184c0cbd20 2024-01-27T02:32:37.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.816+0000 7f18509e4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184c0c64a0 con 0x7f184c1419e0 2024-01-27T02:32:37.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.816+0000 7f18509e4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184c16e840 con 0x7f184c0cb950 2024-01-27T02:32:37.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.816+0000 7f1849d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f184c0cbd20 0x7f184c0c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.816+0000 7f1849d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f184c0cbd20 0x7f184c0c6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58228/0 (socket says 172.21.15.7:58228) 2024-01-27T02:32:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.816+0000 7f1849d9b700 1 -- 172.21.15.7:0/1170469512 learned_addr learned my addr 172.21.15.7:0/1170469512 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.816+0000 7f184ad9d700 1 --2- 172.21.15.7:0/1170469512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f184c1419e0 0x7f184c0b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.816+0000 7f184a59c700 1 --2- 172.21.15.7:0/1170469512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f184c0cb950 0x7f184c140610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.816+0000 7f1849d9b700 1 -- 172.21.15.7:0/1170469512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f184c0cb950 msgr2=0x7f184c140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.816+0000 7f1849d9b700 1 --2- 172.21.15.7:0/1170469512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f184c0cb950 0x7f184c140610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.816+0000 7f1849d9b700 1 -- 172.21.15.7:0/1170469512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f184c1419e0 msgr2=0x7f184c0b09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.816+0000 7f1849d9b700 1 --2- 172.21.15.7:0/1170469512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f184c1419e0 0x7f184c0b09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.816+0000 7f1849d9b700 1 -- 172.21.15.7:0/1170469512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1844009a20 con 0x7f184c0cbd20 2024-01-27T02:32:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.817+0000 7f1849d9b700 1 --2- 172.21.15.7:0/1170469512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f184c0cbd20 0x7f184c0c6930 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f183c00c590 tx=0x7f183c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.817+0000 7f18427fc700 1 -- 172.21.15.7:0/1170469512 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c011590 con 0x7f184c0cbd20 2024-01-27T02:32:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.817+0000 7f18427fc700 1 -- 172.21.15.7:0/1170469512 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183c011710 con 0x7f184c0cbd20 2024-01-27T02:32:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.817+0000 7f18509e4700 1 -- 172.21.15.7:0/1170469512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f184c16e560 con 0x7f184c0cbd20 2024-01-27T02:32:37.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.817+0000 7f18427fc700 1 -- 172.21.15.7:0/1170469512 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c011ad0 con 0x7f184c0cbd20 2024-01-27T02:32:37.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.817+0000 7f18509e4700 1 -- 172.21.15.7:0/1170469512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f184c0c6130 con 0x7f184c0cbd20 2024-01-27T02:32:37.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.818+0000 7f18509e4700 1 -- 172.21.15.7:0/1170469512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f184c072060 con 0x7f184c0cbd20 2024-01-27T02:32:37.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.818+0000 7f18427fc700 1 -- 172.21.15.7:0/1170469512 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f183c01b3b0 con 0x7f184c0cbd20 2024-01-27T02:32:37.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.819+0000 7f18427fc700 1 --2- 172.21.15.7:0/1170469512 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f182c066f30 0x7f182c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:37.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.819+0000 7f18427fc700 1 -- 172.21.15.7:0/1170469512 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f183c083a70 con 0x7f184c0cbd20 2024-01-27T02:32:37.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.819+0000 7f184a59c700 1 --2- 172.21.15.7:0/1170469512 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f182c066f30 0x7f182c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:37.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.819+0000 7f184a59c700 1 --2- 172.21.15.7:0/1170469512 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f182c066f30 0x7f182c0693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f18340058d0 tx=0x7f1834009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:37.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:37.823+0000 7f18427fc700 1 -- 172.21.15.7:0/1170469512 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f183c050ce0 con 0x7f184c0cbd20 2024-01-27T02:32:37.832 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:32:37.835 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:32:38.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.030+0000 7f18509e4700 1 -- 172.21.15.7:0/1170469512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f184c0c65e0 con 0x7f184c0cbd20 2024-01-27T02:32:38.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.030+0000 7f18427fc700 1 -- 172.21.15.7:0/1170469512 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f183c017070 con 0x7f184c0cbd20 2024-01-27T02:32:38.031 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-27T02:32:38.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.033+0000 7f18509e4700 1 -- 172.21.15.7:0/1170469512 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f182c066f30 msgr2=0x7f182c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:38.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.033+0000 7f18509e4700 1 --2- 172.21.15.7:0/1170469512 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f182c066f30 0x7f182c0693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f18340058d0 tx=0x7f1834009040 comp rx=0 tx=0).stop 2024-01-27T02:32:38.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.033+0000 7f18509e4700 1 -- 172.21.15.7:0/1170469512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f184c0cbd20 msgr2=0x7f184c0c6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:38.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.033+0000 7f18509e4700 1 --2- 172.21.15.7:0/1170469512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f184c0cbd20 0x7f184c0c6930 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f183c00c590 tx=0x7f183c00c8a0 comp rx=0 tx=0).stop 2024-01-27T02:32:38.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.033+0000 7f18509e4700 1 -- 172.21.15.7:0/1170469512 shutdown_connections 2024-01-27T02:32:38.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.033+0000 7f18509e4700 1 --2- 172.21.15.7:0/1170469512 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f182c066f30 0x7f182c0693e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:38.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.033+0000 7f18509e4700 1 --2- 172.21.15.7:0/1170469512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f184c1419e0 0x7f184c0b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:38.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.033+0000 7f18509e4700 1 --2- 172.21.15.7:0/1170469512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f184c0cb950 0x7f184c140610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:38.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.033+0000 7f18509e4700 1 --2- 172.21.15.7:0/1170469512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f184c0cbd20 0x7f184c0c6930 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:38.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.033+0000 7f18509e4700 1 -- 172.21.15.7:0/1170469512 >> 172.21.15.7:0/1170469512 conn(0x7f184c063580 msgr2=0x7f184c0cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:38.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.033+0000 7f18509e4700 1 -- 172.21.15.7:0/1170469512 shutdown_connections 2024-01-27T02:32:38.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.033+0000 7f18509e4700 1 -- 172.21.15.7:0/1170469512 wait complete. 2024-01-27T02:32:38.068 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestClusterFull) That when an OSD epoch barrier is set on an MDS, subsequently 2024-01-27T02:32:38.068 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestClusterFull) 2024-01-27T02:32:38.069 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2024-01-27T02:32:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.201+0000 7f3694439700 1 Processor -- start 2024-01-27T02:32:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.202+0000 7f3694439700 1 -- start start 2024-01-27T02:32:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.203+0000 7f3694439700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f368c15a100 0x7f368c0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.203+0000 7f3694439700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f368c0cb5b0 0x7f368c0c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.203+0000 7f3694439700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f368c0c4f50 con 0x7f368c0c4af0 2024-01-27T02:32:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.203+0000 7f3694439700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368c0ce5e0 con 0x7f368c15a100 2024-01-27T02:32:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.203+0000 7f3694439700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368c0cd9d0 con 0x7f368c0cb5b0 2024-01-27T02:32:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.203+0000 7f36921d5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f368c15a100 0x7f368c0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.203+0000 7f36921d5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f368c15a100 0x7f368c0cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58238/0 (socket says 172.21.15.7:58238) 2024-01-27T02:32:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.203+0000 7f36921d5700 1 -- 172.21.15.7:0/3797818048 learned_addr learned my addr 172.21.15.7:0/3797818048 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.203+0000 7f36919d4700 1 --2- 172.21.15.7:0/3797818048 >> v2:172.21.15.188:3300/0 conn(0x7f368c0cb5b0 0x7f368c0c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.203+0000 7f36921d5700 1 -- 172.21.15.7:0/3797818048 >> v2:172.21.15.188:3300/0 conn(0x7f368c0cb5b0 msgr2=0x7f368c0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.203+0000 7f36921d5700 1 --2- 172.21.15.7:0/3797818048 >> v2:172.21.15.188:3300/0 conn(0x7f368c0cb5b0 0x7f368c0c45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.203+0000 7f36921d5700 1 -- 172.21.15.7:0/3797818048 >> v1:172.21.15.188:6789/0 conn(0x7f368c0c4af0 legacy=0x7f368c0c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.203+0000 7f36921d5700 1 -- 172.21.15.7:0/3797818048 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368c0cdcb0 con 0x7f368c15a100 2024-01-27T02:32:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.203+0000 7f36921d5700 1 --2- 172.21.15.7:0/3797818048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368c15a100 0x7f368c0cd2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f368c06e170 tx=0x7f3688009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb208f93c10c048 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.204+0000 7f36911d3700 1 -- 172.21.15.7:0/3797818048 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368800b860 con 0x7f368c15a100 2024-01-27T02:32:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.204+0000 7f36911d3700 1 -- 172.21.15.7:0/3797818048 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368800b9c0 con 0x7f368c15a100 2024-01-27T02:32:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.204+0000 7f36911d3700 1 -- 172.21.15.7:0/3797818048 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368800bd60 con 0x7f368c15a100 2024-01-27T02:32:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.204+0000 7f3694439700 1 -- 172.21.15.7:0/3797818048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368c15a100 msgr2=0x7f368c0cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.204+0000 7f3694439700 1 --2- 172.21.15.7:0/3797818048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368c15a100 0x7f368c0cd2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f368c06e170 tx=0x7f3688009250 comp rx=0 tx=0).stop 2024-01-27T02:32:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.204+0000 7f3694439700 1 -- 172.21.15.7:0/3797818048 shutdown_connections 2024-01-27T02:32:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.204+0000 7f3694439700 1 --2- 172.21.15.7:0/3797818048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368c15a100 0x7f368c0cd2f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.204+0000 7f3694439700 1 --2- 172.21.15.7:0/3797818048 >> v2:172.21.15.188:3300/0 conn(0x7f368c0cb5b0 0x7f368c0c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.204+0000 7f3694439700 1 -- 172.21.15.7:0/3797818048 >> 172.21.15.7:0/3797818048 conn(0x7f368c098020 msgr2=0x7f368c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.204+0000 7f3694439700 1 -- 172.21.15.7:0/3797818048 shutdown_connections 2024-01-27T02:32:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.204+0000 7f3694439700 1 -- 172.21.15.7:0/3797818048 wait complete. 2024-01-27T02:32:38.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.206+0000 7f3694439700 1 Processor -- start 2024-01-27T02:32:38.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.206+0000 7f3694439700 1 -- start start 2024-01-27T02:32:38.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.206+0000 7f3694439700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f368c0c4af0 0x7f368c160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:38.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.206+0000 7f3694439700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f368c0cb5b0 0x7f368c16d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:38.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.206+0000 7f3694439700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368c15a100 0x7f368c16d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:38.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.206+0000 7f3694439700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c15e6b0 con 0x7f368c15a100 2024-01-27T02:32:38.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.206+0000 7f3694439700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c15f4b0 con 0x7f368c0cb5b0 2024-01-27T02:32:38.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.206+0000 7f3694439700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c15e8a0 con 0x7f368c0c4af0 2024-01-27T02:32:38.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.206+0000 7f36929d6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368c15a100 0x7f368c16d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:38.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.207+0000 7f36929d6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368c15a100 0x7f368c16d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58244/0 (socket says 172.21.15.7:58244) 2024-01-27T02:32:38.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.207+0000 7f36929d6700 1 -- 172.21.15.7:0/1138888331 learned_addr learned my addr 172.21.15.7:0/1138888331 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:38.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.207+0000 7f36919d4700 1 --2- 172.21.15.7:0/1138888331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f368c0cb5b0 0x7f368c16d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:38.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.207+0000 7f36921d5700 1 --2- 172.21.15.7:0/1138888331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f368c0c4af0 0x7f368c160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:38.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.207+0000 7f36929d6700 1 -- 172.21.15.7:0/1138888331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f368c0c4af0 msgr2=0x7f368c160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:38.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.207+0000 7f36929d6700 1 --2- 172.21.15.7:0/1138888331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f368c0c4af0 0x7f368c160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:38.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.207+0000 7f36929d6700 1 -- 172.21.15.7:0/1138888331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f368c0cb5b0 msgr2=0x7f368c16d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:38.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.207+0000 7f36929d6700 1 --2- 172.21.15.7:0/1138888331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f368c0cb5b0 0x7f368c16d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:38.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.207+0000 7f36929d6700 1 -- 172.21.15.7:0/1138888331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36880094f0 con 0x7f368c15a100 2024-01-27T02:32:38.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.207+0000 7f36929d6700 1 --2- 172.21.15.7:0/1138888331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368c15a100 0x7f368c16d9a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f368400c8e0 tx=0x7f3684001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:38.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.207+0000 7f36827fc700 1 -- 172.21.15.7:0/1138888331 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3684012630 con 0x7f368c15a100 2024-01-27T02:32:38.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.207+0000 7f36827fc700 1 -- 172.21.15.7:0/1138888331 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3684012790 con 0x7f368c15a100 2024-01-27T02:32:38.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.207+0000 7f3694439700 1 -- 172.21.15.7:0/1138888331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368c15eb80 con 0x7f368c15a100 2024-01-27T02:32:38.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.207+0000 7f36827fc700 1 -- 172.21.15.7:0/1138888331 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3684012b50 con 0x7f368c15a100 2024-01-27T02:32:38.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.207+0000 7f3694439700 1 -- 172.21.15.7:0/1138888331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368c15f140 con 0x7f368c15a100 2024-01-27T02:32:38.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.208+0000 7f3694439700 1 -- 172.21.15.7:0/1138888331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368c0c7510 con 0x7f368c15a100 2024-01-27T02:32:38.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.209+0000 7f36827fc700 1 -- 172.21.15.7:0/1138888331 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f368401a3a0 con 0x7f368c15a100 2024-01-27T02:32:38.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.209+0000 7f36827fc700 1 --2- 172.21.15.7:0/1138888331 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3670066f30 0x7f36700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:38.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.209+0000 7f36827fc700 1 -- 172.21.15.7:0/1138888331 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3684084540 con 0x7f368c15a100 2024-01-27T02:32:38.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.210+0000 7f36921d5700 1 --2- 172.21.15.7:0/1138888331 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3670066f30 0x7f36700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:38.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.210+0000 7f36921d5700 1 --2- 172.21.15.7:0/1138888331 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3670066f30 0x7f36700693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f368c06e170 tx=0x7f3688017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:38.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.214+0000 7f36827fc700 1 -- 172.21.15.7:0/1138888331 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36840551a0 con 0x7f368c15a100 2024-01-27T02:32:38.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.499+0000 7f3694439700 1 -- 172.21.15.7:0/1138888331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) v1 -- 0x7f368c0c4f50 con 0x7f368c15a100 2024-01-27T02:32:38.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.947+0000 7f36827fc700 1 -- 172.21.15.7:0/1138888331 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v49) v1 ==== 133+0+0 (secure 0 0 0) 0x7f368c0c4f50 con 0x7f368c15a100 2024-01-27T02:32:38.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.950+0000 7f3694439700 1 -- 172.21.15.7:0/1138888331 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3670066f30 msgr2=0x7f36700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:38.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.950+0000 7f3694439700 1 --2- 172.21.15.7:0/1138888331 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3670066f30 0x7f36700693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f368c06e170 tx=0x7f3688017000 comp rx=0 tx=0).stop 2024-01-27T02:32:38.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.950+0000 7f3694439700 1 -- 172.21.15.7:0/1138888331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368c15a100 msgr2=0x7f368c16d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:38.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.950+0000 7f3694439700 1 --2- 172.21.15.7:0/1138888331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368c15a100 0x7f368c16d9a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f368400c8e0 tx=0x7f3684001710 comp rx=0 tx=0).stop 2024-01-27T02:32:38.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.950+0000 7f3694439700 1 -- 172.21.15.7:0/1138888331 shutdown_connections 2024-01-27T02:32:38.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.950+0000 7f3694439700 1 --2- 172.21.15.7:0/1138888331 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3670066f30 0x7f36700693e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:38.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.950+0000 7f3694439700 1 --2- 172.21.15.7:0/1138888331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368c15a100 0x7f368c16d9a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:38.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.950+0000 7f3694439700 1 --2- 172.21.15.7:0/1138888331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f368c0c4af0 0x7f368c160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:38.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.950+0000 7f3694439700 1 --2- 172.21.15.7:0/1138888331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f368c0cb5b0 0x7f368c16d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:38.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.950+0000 7f3694439700 1 -- 172.21.15.7:0/1138888331 >> 172.21.15.7:0/1138888331 conn(0x7f368c098020 msgr2=0x7f368c0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:38.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.950+0000 7f3694439700 1 -- 172.21.15.7:0/1138888331 shutdown_connections 2024-01-27T02:32:38.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:38.950+0000 7f3694439700 1 -- 172.21.15.7:0/1138888331 wait complete. 2024-01-27T02:32:38.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-01-27T02:32:39.117 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_objectstore":"memstore"} 2024-01-27T02:32:39.118 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-01-27T02:32:39.118 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-27T02:32:39.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.252+0000 7fd647a01700 1 Processor -- start 2024-01-27T02:32:39.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.252+0000 7fd647a01700 1 -- start start 2024-01-27T02:32:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.253+0000 7fd647a01700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd6400cee40 0x7fd64015b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.253+0000 7fd647a01700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6400ca420 0x7fd640157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.253+0000 7fd647a01700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd64015c3e0 con 0x7fd64015be80 2024-01-27T02:32:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.253+0000 7fd647a01700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd64015c6c0 con 0x7fd6400cee40 2024-01-27T02:32:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.253+0000 7fd647a01700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6400afc40 con 0x7fd6400ca420 2024-01-27T02:32:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.254+0000 7fd64579d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd6400cee40 0x7fd64015b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.254+0000 7fd645f9e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6400ca420 0x7fd640157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.254+0000 7fd64579d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd6400cee40 0x7fd64015b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:36320/0 (socket says 172.21.15.7:36320) 2024-01-27T02:32:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.254+0000 7fd644f9c700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fd64015be80 0x7fd640157790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:53902/0 (socket says 172.21.15.7:53902) 2024-01-27T02:32:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.254+0000 7fd645f9e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6400ca420 0x7fd640157ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:58760/0 (socket says 172.21.15.7:58760) 2024-01-27T02:32:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.254+0000 7fd644f9c700 1 -- 172.21.15.7:0/1866631384 learned_addr learned my addr 172.21.15.7:0/1866631384 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.254+0000 7fd645f9e700 1 -- 172.21.15.7:0/1866631384 >> v2:172.21.15.188:3300/0 conn(0x7fd6400cee40 msgr2=0x7fd64015b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.254+0000 7fd645f9e700 1 --2- 172.21.15.7:0/1866631384 >> v2:172.21.15.188:3300/0 conn(0x7fd6400cee40 0x7fd64015b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:39.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.254+0000 7fd645f9e700 1 -- 172.21.15.7:0/1866631384 >> v1:172.21.15.188:6789/0 conn(0x7fd64015be80 legacy=0x7fd640157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:39.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.254+0000 7fd645f9e700 1 -- 172.21.15.7:0/1866631384 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6400aff20 con 0x7fd6400ca420 2024-01-27T02:32:39.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.254+0000 7fd64579d700 1 --2- 172.21.15.7:0/1866631384 >> v2:172.21.15.188:3300/0 conn(0x7fd6400cee40 0x7fd64015b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:32:39.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.254+0000 7fd645f9e700 1 --2- 172.21.15.7:0/1866631384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6400ca420 0x7fd640157ea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd638009e60 tx=0x7fd638009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe8dd3a6285138e0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:39.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.254+0000 7fd633fff700 1 -- 172.21.15.7:0/1866631384 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63800e070 con 0x7fd6400ca420 2024-01-27T02:32:39.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.255+0000 7fd633fff700 1 -- 172.21.15.7:0/1866631384 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63800bb60 con 0x7fd6400ca420 2024-01-27T02:32:39.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.255+0000 7fd633fff700 1 -- 172.21.15.7:0/1866631384 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638012430 con 0x7fd6400ca420 2024-01-27T02:32:39.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.255+0000 7fd647a01700 1 -- 172.21.15.7:0/1866631384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6400ca420 msgr2=0x7fd640157ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:39.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.255+0000 7fd647a01700 1 --2- 172.21.15.7:0/1866631384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6400ca420 0x7fd640157ea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd638009e60 tx=0x7fd638009780 comp rx=0 tx=0).stop 2024-01-27T02:32:39.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.255+0000 7fd647a01700 1 -- 172.21.15.7:0/1866631384 shutdown_connections 2024-01-27T02:32:39.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.255+0000 7fd647a01700 1 --2- 172.21.15.7:0/1866631384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6400ca420 0x7fd640157ea0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:39.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.255+0000 7fd647a01700 1 --2- 172.21.15.7:0/1866631384 >> v2:172.21.15.188:3300/0 conn(0x7fd6400cee40 0x7fd64015b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:39.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.255+0000 7fd647a01700 1 -- 172.21.15.7:0/1866631384 >> 172.21.15.7:0/1866631384 conn(0x7fd64006f2c0 msgr2=0x7fd64006f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:39.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.255+0000 7fd647a01700 1 -- 172.21.15.7:0/1866631384 shutdown_connections 2024-01-27T02:32:39.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.255+0000 7fd647a01700 1 -- 172.21.15.7:0/1866631384 wait complete. 2024-01-27T02:32:39.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.256+0000 7fd647a01700 1 Processor -- start 2024-01-27T02:32:39.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.256+0000 7fd647a01700 1 -- start start 2024-01-27T02:32:39.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.257+0000 7fd647a01700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6400ca420 0x7fd640068af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:39.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.257+0000 7fd647a01700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6400cee40 0x7fd6400b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:39.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.257+0000 7fd647a01700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd64015be80 0x7fd6400b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:39.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.257+0000 7fd647a01700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd640067a50 con 0x7fd64015be80 2024-01-27T02:32:39.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.257+0000 7fd647a01700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd640067d30 con 0x7fd6400cee40 2024-01-27T02:32:39.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.257+0000 7fd647a01700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd640068010 con 0x7fd6400ca420 2024-01-27T02:32:39.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.258+0000 7fd645f9e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd64015be80 0x7fd6400b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:39.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.258+0000 7fd645f9e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd64015be80 0x7fd6400b45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58260/0 (socket says 172.21.15.7:58260) 2024-01-27T02:32:39.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.258+0000 7fd645f9e700 1 -- 172.21.15.7:0/3898692454 learned_addr learned my addr 172.21.15.7:0/3898692454 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:39.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.258+0000 7fd644f9c700 1 --2- 172.21.15.7:0/3898692454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6400cee40 0x7fd6400b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:39.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.258+0000 7fd64579d700 1 --2- 172.21.15.7:0/3898692454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6400ca420 0x7fd640068af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:39.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.258+0000 7fd644f9c700 1 -- 172.21.15.7:0/3898692454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6400ca420 msgr2=0x7fd640068af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:39.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.258+0000 7fd644f9c700 1 --2- 172.21.15.7:0/3898692454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6400ca420 0x7fd640068af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:39.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.258+0000 7fd644f9c700 1 -- 172.21.15.7:0/3898692454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd64015be80 msgr2=0x7fd6400b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:39.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.258+0000 7fd644f9c700 1 --2- 172.21.15.7:0/3898692454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd64015be80 0x7fd6400b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:39.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.258+0000 7fd644f9c700 1 -- 172.21.15.7:0/3898692454 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c009420 con 0x7fd6400cee40 2024-01-27T02:32:39.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.258+0000 7fd645f9e700 1 --2- 172.21.15.7:0/3898692454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd64015be80 0x7fd6400b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:32:39.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.258+0000 7fd644f9c700 1 --2- 172.21.15.7:0/3898692454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6400cee40 0x7fd6400b4090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd63400c4a0 tx=0x7fd634007f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:39.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.258+0000 7fd6317fa700 1 -- 172.21.15.7:0/3898692454 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63400ce50 con 0x7fd6400cee40 2024-01-27T02:32:39.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.258+0000 7fd6317fa700 1 -- 172.21.15.7:0/3898692454 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd634005b30 con 0x7fd6400cee40 2024-01-27T02:32:39.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.259+0000 7fd647a01700 1 -- 172.21.15.7:0/3898692454 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd638009a20 con 0x7fd6400cee40 2024-01-27T02:32:39.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.259+0000 7fd6317fa700 1 -- 172.21.15.7:0/3898692454 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6340070d0 con 0x7fd6400cee40 2024-01-27T02:32:39.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.259+0000 7fd647a01700 1 -- 172.21.15.7:0/3898692454 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6400685d0 con 0x7fd6400cee40 2024-01-27T02:32:39.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.260+0000 7fd647a01700 1 -- 172.21.15.7:0/3898692454 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd64006b320 con 0x7fd6400cee40 2024-01-27T02:32:39.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.260+0000 7fd6317fa700 1 -- 172.21.15.7:0/3898692454 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd6340074c0 con 0x7fd6400cee40 2024-01-27T02:32:39.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.260+0000 7fd6317fa700 1 --2- 172.21.15.7:0/3898692454 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd624066f30 0x7fd6240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:39.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.260+0000 7fd6317fa700 1 -- 172.21.15.7:0/3898692454 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd634083840 con 0x7fd6400cee40 2024-01-27T02:32:39.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.261+0000 7fd64579d700 1 --2- 172.21.15.7:0/3898692454 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd624066f30 0x7fd6240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:39.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.261+0000 7fd64579d700 1 --2- 172.21.15.7:0/3898692454 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd624066f30 0x7fd6240693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009f40 tx=0x7fd63c0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:39.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.265+0000 7fd6317fa700 1 -- 172.21.15.7:0/3898692454 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd634050ab0 con 0x7fd6400cee40 2024-01-27T02:32:39.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.469+0000 7fd647a01700 1 -- 172.21.15.7:0/3898692454 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd6400687f0 con 0x7fd6400cee40 2024-01-27T02:32:39.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.474+0000 7fd6317fa700 1 -- 172.21.15.7:0/3898692454 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd6340170c0 con 0x7fd6400cee40 2024-01-27T02:32:39.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.477+0000 7fd647a01700 1 -- 172.21.15.7:0/3898692454 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd624066f30 msgr2=0x7fd6240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:39.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.477+0000 7fd647a01700 1 --2- 172.21.15.7:0/3898692454 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd624066f30 0x7fd6240693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009f40 tx=0x7fd63c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:39.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.477+0000 7fd647a01700 1 -- 172.21.15.7:0/3898692454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6400cee40 msgr2=0x7fd6400b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:39.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.477+0000 7fd647a01700 1 --2- 172.21.15.7:0/3898692454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6400cee40 0x7fd6400b4090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd63400c4a0 tx=0x7fd634007f10 comp rx=0 tx=0).stop 2024-01-27T02:32:39.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.477+0000 7fd647a01700 1 -- 172.21.15.7:0/3898692454 shutdown_connections 2024-01-27T02:32:39.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.477+0000 7fd647a01700 1 --2- 172.21.15.7:0/3898692454 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd624066f30 0x7fd6240693e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:39.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.477+0000 7fd647a01700 1 --2- 172.21.15.7:0/3898692454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd64015be80 0x7fd6400b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:39.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.477+0000 7fd647a01700 1 --2- 172.21.15.7:0/3898692454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6400ca420 0x7fd640068af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:39.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.477+0000 7fd647a01700 1 --2- 172.21.15.7:0/3898692454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6400cee40 0x7fd6400b4090 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:39.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.477+0000 7fd647a01700 1 -- 172.21.15.7:0/3898692454 >> 172.21.15.7:0/3898692454 conn(0x7fd64006f2c0 msgr2=0x7fd6400cbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:39.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.477+0000 7fd647a01700 1 -- 172.21.15.7:0/3898692454 shutdown_connections 2024-01-27T02:32:39.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:39.477+0000 7fd647a01700 1 -- 172.21.15.7:0/3898692454 wait complete. 2024-01-27T02:32:39.491 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-27T02:32:39.491 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:32:39.534 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:32:39.580 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-27T02:32:39.580 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-27T02:32:39.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:32:39.580 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-01-27T02:32:39.620 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:32:39.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:39.620 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-27T02:32:39.700 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-27T02:32:39.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:32:39.701 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-01-27T02:32:39.748 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:32:39.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:39.748 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-27T02:32:39.788 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-01-27T02:32:39.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:32:39.788 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-01-27T02:32:39.838 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:32:39.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:39.839 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-27T02:32:39.860 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-01-27T02:32:39.861 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:32:39.861 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-01-27T02:32:39.897 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:32:39.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:32:39.898 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-27T02:32:39.956 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-27T02:32:39.956 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:32:40.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.096+0000 7fb6db77f700 1 Processor -- start 2024-01-27T02:32:40.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.096+0000 7fb6db77f700 1 -- start start 2024-01-27T02:32:40.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.097+0000 7fb6db77f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d415a100 0x7fb6d40cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.097+0000 7fb6db77f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb6d40c4cc0 0x7fb6d4163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.097+0000 7fb6db77f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d40ce5e0 con 0x7fb6d40cb5b0 2024-01-27T02:32:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.097+0000 7fb6db77f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d40cd9d0 con 0x7fb6d40c4cc0 2024-01-27T02:32:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.097+0000 7fb6db77f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d40cdcb0 con 0x7fb6d415a100 2024-01-27T02:32:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.097+0000 7fb6d8d1a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb6d40cb5b0 0x7fb6d40c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45616/0 (socket says 172.21.15.7:45616) 2024-01-27T02:32:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.097+0000 7fb6d8d1a700 1 -- 172.21.15.7:0/3167986764 learned_addr learned my addr 172.21.15.7:0/3167986764 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.097+0000 7fb6d951b700 1 --2- 172.21.15.7:0/3167986764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d415a100 0x7fb6d40cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.097+0000 7fb6d9d1c700 1 --2- 172.21.15.7:0/3167986764 >> v2:172.21.15.188:3300/0 conn(0x7fb6d40c4cc0 0x7fb6d4163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.098+0000 7fb6cbfff700 1 -- 172.21.15.7:0/3167986764 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3286175550 0 0) 0x7fb6d40ce5e0 con 0x7fb6d40cb5b0 2024-01-27T02:32:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.098+0000 7fb6cbfff700 1 -- 172.21.15.7:0/3167986764 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6c0003570 con 0x7fb6d40cb5b0 2024-01-27T02:32:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.098+0000 7fb6d951b700 1 -- 172.21.15.7:0/3167986764 >> v2:172.21.15.188:3300/0 conn(0x7fb6d40c4cc0 msgr2=0x7fb6d4163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.098+0000 7fb6d951b700 1 --2- 172.21.15.7:0/3167986764 >> v2:172.21.15.188:3300/0 conn(0x7fb6d40c4cc0 0x7fb6d4163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.098+0000 7fb6d951b700 1 -- 172.21.15.7:0/3167986764 >> v1:172.21.15.7:6789/0 conn(0x7fb6d40cb5b0 legacy=0x7fb6d40c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:40.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.098+0000 7fb6d951b700 1 -- 172.21.15.7:0/3167986764 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d40cdf90 con 0x7fb6d415a100 2024-01-27T02:32:40.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.098+0000 7fb6d9d1c700 1 --2- 172.21.15.7:0/3167986764 >> v2:172.21.15.188:3300/0 conn(0x7fb6d40c4cc0 0x7fb6d4163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:32:40.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.098+0000 7fb6d951b700 1 --2- 172.21.15.7:0/3167986764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d415a100 0x7fb6d40cd2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00098e0 tx=0x7fb6d00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccb1db622eaff2c8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:40.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.098+0000 7fb6cbfff700 1 -- 172.21.15.7:0/3167986764 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0011630 con 0x7fb6d415a100 2024-01-27T02:32:40.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.098+0000 7fb6cbfff700 1 -- 172.21.15.7:0/3167986764 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d0011790 con 0x7fb6d415a100 2024-01-27T02:32:40.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.098+0000 7fb6cbfff700 1 -- 172.21.15.7:0/3167986764 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0011b30 con 0x7fb6d415a100 2024-01-27T02:32:40.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.099+0000 7fb6db77f700 1 -- 172.21.15.7:0/3167986764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d415a100 msgr2=0x7fb6d40cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:40.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.099+0000 7fb6db77f700 1 --2- 172.21.15.7:0/3167986764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d415a100 0x7fb6d40cd2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00098e0 tx=0x7fb6d00091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:40.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.099+0000 7fb6db77f700 1 -- 172.21.15.7:0/3167986764 shutdown_connections 2024-01-27T02:32:40.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.099+0000 7fb6db77f700 1 --2- 172.21.15.7:0/3167986764 >> v2:172.21.15.188:3300/0 conn(0x7fb6d40c4cc0 0x7fb6d4163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:40.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.099+0000 7fb6db77f700 1 --2- 172.21.15.7:0/3167986764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d415a100 0x7fb6d40cd2f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:40.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.099+0000 7fb6db77f700 1 -- 172.21.15.7:0/3167986764 >> 172.21.15.7:0/3167986764 conn(0x7fb6d4098020 msgr2=0x7fb6d40711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:40.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.099+0000 7fb6db77f700 1 -- 172.21.15.7:0/3167986764 shutdown_connections 2024-01-27T02:32:40.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.099+0000 7fb6db77f700 1 -- 172.21.15.7:0/3167986764 wait complete. 2024-01-27T02:32:40.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.100+0000 7fb6db77f700 1 Processor -- start 2024-01-27T02:32:40.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.100+0000 7fb6db77f700 1 -- start start 2024-01-27T02:32:40.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.101+0000 7fb6db77f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d40c4cc0 0x7fb6d415eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:40.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.101+0000 7fb6db77f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d40cb5b0 0x7fb6d4067bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:40.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.101+0000 7fb6db77f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6d415a100 0x7fb6d4068130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:40.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.101+0000 7fb6db77f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d416b480 con 0x7fb6d40c4cc0 2024-01-27T02:32:40.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.101+0000 7fb6db77f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d40b45f0 con 0x7fb6d415a100 2024-01-27T02:32:40.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.101+0000 7fb6db77f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d416b1a0 con 0x7fb6d40cb5b0 2024-01-27T02:32:40.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.101+0000 7fb6d951b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d40c4cc0 0x7fb6d415eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:40.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.101+0000 7fb6d951b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d40c4cc0 0x7fb6d415eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58272/0 (socket says 172.21.15.7:58272) 2024-01-27T02:32:40.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.101+0000 7fb6d951b700 1 -- 172.21.15.7:0/499469129 learned_addr learned my addr 172.21.15.7:0/499469129 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:40.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.102+0000 7fb6d8d1a700 1 --2- 172.21.15.7:0/499469129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d40cb5b0 0x7fb6d4067bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:40.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.102+0000 7fb6d9d1c700 1 --2- 172.21.15.7:0/499469129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6d415a100 0x7fb6d4068130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:40.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.102+0000 7fb6d951b700 1 -- 172.21.15.7:0/499469129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d40cb5b0 msgr2=0x7fb6d4067bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:40.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.102+0000 7fb6d951b700 1 --2- 172.21.15.7:0/499469129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d40cb5b0 0x7fb6d4067bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:40.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.102+0000 7fb6d951b700 1 -- 172.21.15.7:0/499469129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6d415a100 msgr2=0x7fb6d4068130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:40.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.102+0000 7fb6d951b700 1 --2- 172.21.15.7:0/499469129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6d415a100 0x7fb6d4068130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:40.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.102+0000 7fb6d951b700 1 -- 172.21.15.7:0/499469129 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6cc009a20 con 0x7fb6d40c4cc0 2024-01-27T02:32:40.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.102+0000 7fb6d951b700 1 --2- 172.21.15.7:0/499469129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d40c4cc0 0x7fb6d415eed0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb6d406f8c0 tx=0x7fb6d0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:40.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.102+0000 7fb6c97fa700 1 -- 172.21.15.7:0/499469129 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d001f070 con 0x7fb6d40c4cc0 2024-01-27T02:32:40.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.102+0000 7fb6c97fa700 1 -- 172.21.15.7:0/499469129 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d0000bc0 con 0x7fb6d40c4cc0 2024-01-27T02:32:40.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.102+0000 7fb6db77f700 1 -- 172.21.15.7:0/499469129 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d0009420 con 0x7fb6d40c4cc0 2024-01-27T02:32:40.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.102+0000 7fb6c97fa700 1 -- 172.21.15.7:0/499469129 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0011b30 con 0x7fb6d40c4cc0 2024-01-27T02:32:40.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.102+0000 7fb6db77f700 1 -- 172.21.15.7:0/499469129 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d416aeb0 con 0x7fb6d40c4cc0 2024-01-27T02:32:40.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.103+0000 7fb6db77f700 1 -- 172.21.15.7:0/499469129 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d404c710 con 0x7fb6d40c4cc0 2024-01-27T02:32:40.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.104+0000 7fb6c97fa700 1 -- 172.21.15.7:0/499469129 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb6d0023070 con 0x7fb6d40c4cc0 2024-01-27T02:32:40.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.104+0000 7fb6c97fa700 1 --2- 172.21.15.7:0/499469129 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb6b8066f30 0x7fb6b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:40.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.104+0000 7fb6c97fa700 1 -- 172.21.15.7:0/499469129 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb6d00857e0 con 0x7fb6d40c4cc0 2024-01-27T02:32:40.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.104+0000 7fb6d8d1a700 1 --2- 172.21.15.7:0/499469129 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb6b8066f30 0x7fb6b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:40.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.105+0000 7fb6d8d1a700 1 --2- 172.21.15.7:0/499469129 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb6b8066f30 0x7fb6b80693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4157910 tx=0x7fb6c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:40.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.109+0000 7fb6c97fa700 1 -- 172.21.15.7:0/499469129 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6d0052a50 con 0x7fb6d40c4cc0 2024-01-27T02:32:40.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.358+0000 7fb6db77f700 1 -- 172.21.15.7:0/499469129 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6d40ce270 con 0x7fb6d40c4cc0 2024-01-27T02:32:40.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.359+0000 7fb6c97fa700 1 -- 172.21.15.7:0/499469129 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7fb6d002a020 con 0x7fb6d40c4cc0 2024-01-27T02:32:40.360 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:40.360 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3594367777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3594367777},{"type":"v1","addr":"172.21.15.7:6834","nonce":3594367777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1079101213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1079101213},{"type":"v1","addr":"172.21.15.188:6837","nonce":1079101213}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3241424244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3241424244},{"type":"v1","addr":"172.21.15.7:6835","nonce":3241424244}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:32:06.045693+0000","modified":"2024-01-27T02:32:09.806398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6836/2571237935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2571237935},{"type":"v1","addr":"172.21.15.188:6836","nonce":2571237935}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-27T02:32:40.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.361+0000 7fb6db77f700 1 -- 172.21.15.7:0/499469129 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb6b8066f30 msgr2=0x7fb6b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:40.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.361+0000 7fb6db77f700 1 --2- 172.21.15.7:0/499469129 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb6b8066f30 0x7fb6b80693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4157910 tx=0x7fb6c4009040 comp rx=0 tx=0).stop 2024-01-27T02:32:40.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.361+0000 7fb6db77f700 1 -- 172.21.15.7:0/499469129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d40c4cc0 msgr2=0x7fb6d415eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:40.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.361+0000 7fb6db77f700 1 --2- 172.21.15.7:0/499469129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d40c4cc0 0x7fb6d415eed0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb6d406f8c0 tx=0x7fb6d0017000 comp rx=0 tx=0).stop 2024-01-27T02:32:40.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.361+0000 7fb6db77f700 1 -- 172.21.15.7:0/499469129 shutdown_connections 2024-01-27T02:32:40.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.362+0000 7fb6db77f700 1 --2- 172.21.15.7:0/499469129 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb6b8066f30 0x7fb6b80693e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:40.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.362+0000 7fb6db77f700 1 --2- 172.21.15.7:0/499469129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6d415a100 0x7fb6d4068130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:40.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.362+0000 7fb6db77f700 1 --2- 172.21.15.7:0/499469129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d40c4cc0 0x7fb6d415eed0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:40.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.362+0000 7fb6db77f700 1 --2- 172.21.15.7:0/499469129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d40cb5b0 0x7fb6d4067bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:40.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.362+0000 7fb6db77f700 1 -- 172.21.15.7:0/499469129 >> 172.21.15.7:0/499469129 conn(0x7fb6d4098020 msgr2=0x7fb6d41590d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:40.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.362+0000 7fb6db77f700 1 -- 172.21.15.7:0/499469129 shutdown_connections 2024-01-27T02:32:40.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.362+0000 7fb6db77f700 1 -- 172.21.15.7:0/499469129 wait complete. 2024-01-27T02:32:40.363 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-01-27T02:32:40.376 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:32:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.507+0000 7f366a6ba700 1 Processor -- start 2024-01-27T02:32:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.507+0000 7f366a6ba700 1 -- start start 2024-01-27T02:32:40.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.508+0000 7f366a6ba700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36640b1aa0 0x7f36640af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:40.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.508+0000 7f366a6ba700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f366406f870 con 0x7f36640a7bd0 2024-01-27T02:32:40.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.508+0000 7f366a6ba700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f366406ec60 con 0x7f36640a7fa0 2024-01-27T02:32:40.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.508+0000 7f366a6ba700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f366406fe40 con 0x7f36640b1aa0 2024-01-27T02:32:40.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.508+0000 7f3663fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f36640a7bd0 0x7f36640b1390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45620/0 (socket says 172.21.15.7:45620) 2024-01-27T02:32:40.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.508+0000 7f3663fff700 1 -- 172.21.15.7:0/1938931446 learned_addr learned my addr 172.21.15.7:0/1938931446 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:40.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.509+0000 7f36637fe700 1 --2- 172.21.15.7:0/1938931446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36640b1aa0 0x7f36640af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:40.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.509+0000 7f3662ffd700 1 -- 172.21.15.7:0/1938931446 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3505973642 0 0) 0x7f366406f870 con 0x7f36640a7bd0 2024-01-27T02:32:40.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.509+0000 7f3662ffd700 1 -- 172.21.15.7:0/1938931446 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f364c003570 con 0x7f36640a7bd0 2024-01-27T02:32:40.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.509+0000 7f36637fe700 1 -- 172.21.15.7:0/1938931446 >> v1:172.21.15.188:6789/0 conn(0x7f36640a7fa0 legacy=0x7f3664097bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:40.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.509+0000 7f36637fe700 1 -- 172.21.15.7:0/1938931446 >> v1:172.21.15.7:6789/0 conn(0x7f36640a7bd0 legacy=0x7f36640b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:40.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.509+0000 7f3662ffd700 1 -- 172.21.15.7:0/1938931446 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3251813392 0 0) 0x7f366406ec60 con 0x7f36640a7fa0 2024-01-27T02:32:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.509+0000 7f36637fe700 1 -- 172.21.15.7:0/1938931446 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366406ef40 con 0x7f36640b1aa0 2024-01-27T02:32:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.509+0000 7f36637fe700 1 --2- 172.21.15.7:0/1938931446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36640b1aa0 0x7f36640af6d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3658009910 tx=0x7f36580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2aed2427c2382c9 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.509+0000 7f3662ffd700 1 -- 172.21.15.7:0/1938931446 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365800b680 con 0x7f36640b1aa0 2024-01-27T02:32:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.509+0000 7f3662ffd700 1 -- 172.21.15.7:0/1938931446 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365800b7e0 con 0x7f36640b1aa0 2024-01-27T02:32:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.509+0000 7f3662ffd700 1 -- 172.21.15.7:0/1938931446 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365800bba0 con 0x7f36640b1aa0 2024-01-27T02:32:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.510+0000 7f366a6ba700 1 -- 172.21.15.7:0/1938931446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36640b1aa0 msgr2=0x7f36640af6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.510+0000 7f366a6ba700 1 --2- 172.21.15.7:0/1938931446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36640b1aa0 0x7f36640af6d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3658009910 tx=0x7f36580091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.510+0000 7f366a6ba700 1 -- 172.21.15.7:0/1938931446 shutdown_connections 2024-01-27T02:32:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.510+0000 7f366a6ba700 1 --2- 172.21.15.7:0/1938931446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36640b1aa0 0x7f36640af6d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.510+0000 7f366a6ba700 1 -- 172.21.15.7:0/1938931446 >> 172.21.15.7:0/1938931446 conn(0x7f36640679b0 msgr2=0x7f3664067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.510+0000 7f366a6ba700 1 -- 172.21.15.7:0/1938931446 shutdown_connections 2024-01-27T02:32:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.510+0000 7f366a6ba700 1 -- 172.21.15.7:0/1938931446 wait complete. 2024-01-27T02:32:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.511+0000 7f366a6ba700 1 Processor -- start 2024-01-27T02:32:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.512+0000 7f366a6ba700 1 -- start start 2024-01-27T02:32:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.512+0000 7f366a6ba700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36640a7bd0 0x7f36640a8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.512+0000 7f366a6ba700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36640a7fa0 0x7f36640a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.512+0000 7f366a6ba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36640b1aa0 0x7f3664098f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.512+0000 7f366a6ba700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36640aa1f0 con 0x7f36640b1aa0 2024-01-27T02:32:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.512+0000 7f366a6ba700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36640a9820 con 0x7f36640a7fa0 2024-01-27T02:32:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.512+0000 7f366a6ba700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3664099490 con 0x7f36640a7bd0 2024-01-27T02:32:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.512+0000 7f3668c57700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36640b1aa0 0x7f3664098f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.512+0000 7f3668c57700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36640b1aa0 0x7f3664098f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58286/0 (socket says 172.21.15.7:58286) 2024-01-27T02:32:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.512+0000 7f3668c57700 1 -- 172.21.15.7:0/982905476 learned_addr learned my addr 172.21.15.7:0/982905476 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.512+0000 7f36637fe700 1 --2- 172.21.15.7:0/982905476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36640a7fa0 0x7f36640a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.512+0000 7f3663fff700 1 --2- 172.21.15.7:0/982905476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36640a7bd0 0x7f36640a8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.513+0000 7f3668c57700 1 -- 172.21.15.7:0/982905476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36640a7bd0 msgr2=0x7f36640a8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.513+0000 7f3668c57700 1 --2- 172.21.15.7:0/982905476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36640a7bd0 0x7f36640a8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.513+0000 7f3668c57700 1 -- 172.21.15.7:0/982905476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36640a7fa0 msgr2=0x7f36640a9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.513+0000 7f3668c57700 1 --2- 172.21.15.7:0/982905476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36640a7fa0 0x7f36640a9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.513+0000 7f3668c57700 1 -- 172.21.15.7:0/982905476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3658009420 con 0x7f36640b1aa0 2024-01-27T02:32:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.513+0000 7f3668c57700 1 --2- 172.21.15.7:0/982905476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36640b1aa0 0x7f3664098f50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f365c001560 tx=0x7f365c001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.513+0000 7f364bfff700 1 -- 172.21.15.7:0/982905476 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c004160 con 0x7f36640b1aa0 2024-01-27T02:32:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.513+0000 7f364bfff700 1 -- 172.21.15.7:0/982905476 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365c00a7e0 con 0x7f36640b1aa0 2024-01-27T02:32:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.513+0000 7f366a6ba700 1 -- 172.21.15.7:0/982905476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36640a8c10 con 0x7f36640b1aa0 2024-01-27T02:32:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.513+0000 7f364bfff700 1 -- 172.21.15.7:0/982905476 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c00aba0 con 0x7f36640b1aa0 2024-01-27T02:32:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.513+0000 7f366a6ba700 1 -- 172.21.15.7:0/982905476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36640a91d0 con 0x7f36640b1aa0 2024-01-27T02:32:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.514+0000 7f366a6ba700 1 -- 172.21.15.7:0/982905476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36640a9990 con 0x7f36640b1aa0 2024-01-27T02:32:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.515+0000 7f364bfff700 1 -- 172.21.15.7:0/982905476 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f365c012420 con 0x7f36640b1aa0 2024-01-27T02:32:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.515+0000 7f364bfff700 1 --2- 172.21.15.7:0/982905476 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3650066f30 0x7f36500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:40.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.515+0000 7f364bfff700 1 -- 172.21.15.7:0/982905476 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f365c0842d0 con 0x7f36640b1aa0 2024-01-27T02:32:40.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.515+0000 7f3663fff700 1 --2- 172.21.15.7:0/982905476 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3650066f30 0x7f36500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:40.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.516+0000 7f3663fff700 1 --2- 172.21.15.7:0/982905476 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3650066f30 0x7f36500693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3654007fd0 tx=0x7f3654009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:40.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.520+0000 7f364bfff700 1 -- 172.21.15.7:0/982905476 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f365c051540 con 0x7f36640b1aa0 2024-01-27T02:32:40.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.722+0000 7f366a6ba700 1 -- 172.21.15.7:0/982905476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f366404c710 con 0x7f36640b1aa0 2024-01-27T02:32:40.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.723+0000 7f364bfff700 1 -- 172.21.15.7:0/982905476 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f365c054b60 con 0x7f36640b1aa0 2024-01-27T02:32:40.723 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:40.723 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:32:07.627822+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:32:03.675611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-27T02:32:05.018885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:32:40.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.725+0000 7f366a6ba700 1 -- 172.21.15.7:0/982905476 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3650066f30 msgr2=0x7f36500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:40.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.725+0000 7f366a6ba700 1 --2- 172.21.15.7:0/982905476 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3650066f30 0x7f36500693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3654007fd0 tx=0x7f3654009040 comp rx=0 tx=0).stop 2024-01-27T02:32:40.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.725+0000 7f366a6ba700 1 -- 172.21.15.7:0/982905476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36640b1aa0 msgr2=0x7f3664098f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:40.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.725+0000 7f366a6ba700 1 --2- 172.21.15.7:0/982905476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36640b1aa0 0x7f3664098f50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f365c001560 tx=0x7f365c001770 comp rx=0 tx=0).stop 2024-01-27T02:32:40.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.725+0000 7f366a6ba700 1 -- 172.21.15.7:0/982905476 shutdown_connections 2024-01-27T02:32:40.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.725+0000 7f366a6ba700 1 --2- 172.21.15.7:0/982905476 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3650066f30 0x7f36500693e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:40.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.725+0000 7f366a6ba700 1 --2- 172.21.15.7:0/982905476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36640b1aa0 0x7f3664098f50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:40.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.725+0000 7f366a6ba700 1 --2- 172.21.15.7:0/982905476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36640a7bd0 0x7f36640a8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:40.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.725+0000 7f366a6ba700 1 --2- 172.21.15.7:0/982905476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36640a7fa0 0x7f36640a9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:40.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.725+0000 7f366a6ba700 1 -- 172.21.15.7:0/982905476 >> 172.21.15.7:0/982905476 conn(0x7f36640679b0 msgr2=0x7f36640b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:40.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.725+0000 7f366a6ba700 1 -- 172.21.15.7:0/982905476 shutdown_connections 2024-01-27T02:32:40.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.725+0000 7f366a6ba700 1 -- 172.21.15.7:0/982905476 wait complete. 2024-01-27T02:32:40.739 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-27T02:32:03.675611+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-27T02:32:05.018885+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-27T02:32:40.739 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:32:40.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.874+0000 7f0ff67b7700 1 Processor -- start 2024-01-27T02:32:40.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.874+0000 7f0ff67b7700 1 -- start start 2024-01-27T02:32:40.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.875+0000 7f0ff67b7700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0ff00cbe30 0x7f0ff00ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:40.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.875+0000 7f0ff67b7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff00679b0 con 0x7f0ff006b4e0 2024-01-27T02:32:40.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.875+0000 7f0ff67b7700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff0067c90 con 0x7f0ff00cba60 2024-01-27T02:32:40.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.875+0000 7f0ff67b7700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff00cca50 con 0x7f0ff00cbe30 2024-01-27T02:32:40.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.875+0000 7f0ff4d54700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0ff006b4e0 0x7f0ff006b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45630/0 (socket says 172.21.15.7:45630) 2024-01-27T02:32:40.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.875+0000 7f0ff4d54700 1 -- 172.21.15.7:0/2442533742 learned_addr learned my addr 172.21.15.7:0/2442533742 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:40.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.875+0000 7f0fef7fe700 1 --2- 172.21.15.7:0/2442533742 >> v2:172.21.15.188:3300/0 conn(0x7f0ff00cbe30 0x7f0ff00ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:40.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.875+0000 7f0feeffd700 1 -- 172.21.15.7:0/2442533742 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720358757 0 0) 0x7f0ff00679b0 con 0x7f0ff006b4e0 2024-01-27T02:32:40.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.875+0000 7f0feeffd700 1 -- 172.21.15.7:0/2442533742 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fd8003570 con 0x7f0ff006b4e0 2024-01-27T02:32:40.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.875+0000 7f0fef7fe700 1 -- 172.21.15.7:0/2442533742 >> v1:172.21.15.188:6789/0 conn(0x7f0ff00cba60 legacy=0x7f0ff015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:40.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.875+0000 7f0feeffd700 1 -- 172.21.15.7:0/2442533742 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 719650215 0 0) 0x7f0ff0067c90 con 0x7f0ff00cba60 2024-01-27T02:32:40.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.875+0000 7f0fef7fe700 1 -- 172.21.15.7:0/2442533742 >> v1:172.21.15.7:6789/0 conn(0x7f0ff006b4e0 legacy=0x7f0ff006b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:40.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.875+0000 7f0fef7fe700 1 -- 172.21.15.7:0/2442533742 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff00ccd30 con 0x7f0ff00cbe30 2024-01-27T02:32:40.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.876+0000 7f0fef7fe700 1 --2- 172.21.15.7:0/2442533742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ff00cbe30 0x7f0ff00ce410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40098e0 tx=0x7f0fe40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3d0a07a3c70df24 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:40.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.876+0000 7f0feeffd700 1 -- 172.21.15.7:0/2442533742 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe400b640 con 0x7f0ff00cbe30 2024-01-27T02:32:40.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.876+0000 7f0feeffd700 1 -- 172.21.15.7:0/2442533742 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe400b7a0 con 0x7f0ff00cbe30 2024-01-27T02:32:40.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.876+0000 7f0feeffd700 1 -- 172.21.15.7:0/2442533742 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe400bb40 con 0x7f0ff00cbe30 2024-01-27T02:32:40.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.876+0000 7f0ff67b7700 1 -- 172.21.15.7:0/2442533742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ff00cbe30 msgr2=0x7f0ff00ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:40.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.876+0000 7f0ff67b7700 1 --2- 172.21.15.7:0/2442533742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ff00cbe30 0x7f0ff00ce410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40098e0 tx=0x7f0fe40091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:40.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.876+0000 7f0ff67b7700 1 -- 172.21.15.7:0/2442533742 shutdown_connections 2024-01-27T02:32:40.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.876+0000 7f0ff67b7700 1 --2- 172.21.15.7:0/2442533742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ff00cbe30 0x7f0ff00ce410 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:40.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.876+0000 7f0ff67b7700 1 -- 172.21.15.7:0/2442533742 >> 172.21.15.7:0/2442533742 conn(0x7f0ff0063580 msgr2=0x7f0ff0063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:40.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.876+0000 7f0ff67b7700 1 -- 172.21.15.7:0/2442533742 shutdown_connections 2024-01-27T02:32:40.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.877+0000 7f0ff67b7700 1 -- 172.21.15.7:0/2442533742 wait complete. 2024-01-27T02:32:40.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.878+0000 7f0ff67b7700 1 Processor -- start 2024-01-27T02:32:40.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.878+0000 7f0ff67b7700 1 -- start start 2024-01-27T02:32:40.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.879+0000 7f0ff67b7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ff006b4e0 0x7f0ff00b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:40.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.879+0000 7f0ff67b7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ff00cba60 0x7f0ff015edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:40.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.879+0000 7f0ff67b7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ff00cbe30 0x7f0ff00b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:40.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.879+0000 7f0ff67b7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff00b08d0 con 0x7f0ff00cbe30 2024-01-27T02:32:40.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.879+0000 7f0ff67b7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff00b48b0 con 0x7f0ff00cba60 2024-01-27T02:32:40.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.879+0000 7f0ff67b7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff00abd00 con 0x7f0ff006b4e0 2024-01-27T02:32:40.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.879+0000 7f0ff4d54700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ff00cbe30 0x7f0ff00b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:40.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.879+0000 7f0ff4d54700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ff00cbe30 0x7f0ff00b3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58296/0 (socket says 172.21.15.7:58296) 2024-01-27T02:32:40.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.879+0000 7f0ff4d54700 1 -- 172.21.15.7:0/1780981628 learned_addr learned my addr 172.21.15.7:0/1780981628 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:40.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.879+0000 7f0fef7fe700 1 --2- 172.21.15.7:0/1780981628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ff00cba60 0x7f0ff015edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:40.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.879+0000 7f0feffff700 1 --2- 172.21.15.7:0/1780981628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ff006b4e0 0x7f0ff00b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:40.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.879+0000 7f0ff4d54700 1 -- 172.21.15.7:0/1780981628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ff006b4e0 msgr2=0x7f0ff00b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:40.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.879+0000 7f0ff4d54700 1 --2- 172.21.15.7:0/1780981628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ff006b4e0 0x7f0ff00b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:40.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.879+0000 7f0ff4d54700 1 -- 172.21.15.7:0/1780981628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ff00cba60 msgr2=0x7f0ff015edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:40.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.879+0000 7f0ff4d54700 1 --2- 172.21.15.7:0/1780981628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ff00cba60 0x7f0ff015edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:40.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.879+0000 7f0ff4d54700 1 -- 172.21.15.7:0/1780981628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe4009420 con 0x7f0ff00cbe30 2024-01-27T02:32:40.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.879+0000 7f0ff4d54700 1 --2- 172.21.15.7:0/1780981628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ff00cbe30 0x7f0ff00b3e60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800cc40 tx=0x7f0fe8001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:40.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.879+0000 7f0fd7fff700 1 -- 172.21.15.7:0/1780981628 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe800a8b0 con 0x7f0ff00cbe30 2024-01-27T02:32:40.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.879+0000 7f0fd7fff700 1 -- 172.21.15.7:0/1780981628 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe800aa10 con 0x7f0ff00cbe30 2024-01-27T02:32:40.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.879+0000 7f0ff67b7700 1 -- 172.21.15.7:0/1780981628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff00ac040 con 0x7f0ff00cbe30 2024-01-27T02:32:40.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.880+0000 7f0fd7fff700 1 -- 172.21.15.7:0/1780981628 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe800ad90 con 0x7f0ff00cbe30 2024-01-27T02:32:40.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.880+0000 7f0ff67b7700 1 -- 172.21.15.7:0/1780981628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff00b1fc0 con 0x7f0ff00cbe30 2024-01-27T02:32:40.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.881+0000 7f0ff67b7700 1 -- 172.21.15.7:0/1780981628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff00ac220 con 0x7f0ff00cbe30 2024-01-27T02:32:40.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.881+0000 7f0fd7fff700 1 -- 172.21.15.7:0/1780981628 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0fe8007300 con 0x7f0ff00cbe30 2024-01-27T02:32:40.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.882+0000 7f0fd7fff700 1 --2- 172.21.15.7:0/1780981628 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0fdc066f30 0x7f0fdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:40.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.882+0000 7f0fd7fff700 1 -- 172.21.15.7:0/1780981628 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0fe8084870 con 0x7f0ff00cbe30 2024-01-27T02:32:40.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.882+0000 7f0feffff700 1 --2- 172.21.15.7:0/1780981628 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0fdc066f30 0x7f0fdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:40.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.883+0000 7f0feffff700 1 --2- 172.21.15.7:0/1780981628 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0fdc066f30 0x7f0fdc0693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0007fd0 tx=0x7f0fe0007d10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:40.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:40.886+0000 7f0fd7fff700 1 -- 172.21.15.7:0/1780981628 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0fe8051b60 con 0x7f0ff00cbe30 2024-01-27T02:32:41.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.089+0000 7f0ff67b7700 1 -- 172.21.15.7:0/1780981628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0ff0072060 con 0x7f0ff00cbe30 2024-01-27T02:32:41.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.089+0000 7f0fd7fff700 1 -- 172.21.15.7:0/1780981628 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0fe8055180 con 0x7f0ff00cbe30 2024-01-27T02:32:41.090 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:32:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.091+0000 7f0ff67b7700 1 -- 172.21.15.7:0/1780981628 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0fdc066f30 msgr2=0x7f0fdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.091+0000 7f0ff67b7700 1 --2- 172.21.15.7:0/1780981628 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0fdc066f30 0x7f0fdc0693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0007fd0 tx=0x7f0fe0007d10 comp rx=0 tx=0).stop 2024-01-27T02:32:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.092+0000 7f0ff67b7700 1 -- 172.21.15.7:0/1780981628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ff00cbe30 msgr2=0x7f0ff00b3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.092+0000 7f0ff67b7700 1 --2- 172.21.15.7:0/1780981628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ff00cbe30 0x7f0ff00b3e60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800cc40 tx=0x7f0fe8001aa0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.092+0000 7f0ff67b7700 1 -- 172.21.15.7:0/1780981628 shutdown_connections 2024-01-27T02:32:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.092+0000 7f0ff67b7700 1 --2- 172.21.15.7:0/1780981628 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0fdc066f30 0x7f0fdc0693e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.092+0000 7f0ff67b7700 1 --2- 172.21.15.7:0/1780981628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ff00cbe30 0x7f0ff00b3e60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.092+0000 7f0ff67b7700 1 --2- 172.21.15.7:0/1780981628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ff006b4e0 0x7f0ff00b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.092+0000 7f0ff67b7700 1 --2- 172.21.15.7:0/1780981628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ff00cba60 0x7f0ff015edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.092+0000 7f0ff67b7700 1 -- 172.21.15.7:0/1780981628 >> 172.21.15.7:0/1780981628 conn(0x7f0ff0063580 msgr2=0x7f0ff0065700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.092+0000 7f0ff67b7700 1 -- 172.21.15.7:0/1780981628 shutdown_connections 2024-01-27T02:32:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.092+0000 7f0ff67b7700 1 -- 172.21.15.7:0/1780981628 wait complete. 2024-01-27T02:32:41.106 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-27T02:32:41.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.240+0000 7fde9cd77700 1 Processor -- start 2024-01-27T02:32:41.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.240+0000 7fde9cd77700 1 -- start start 2024-01-27T02:32:41.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.241+0000 7fde9cd77700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde980cc8d0 0x7fde980693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:41.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.241+0000 7fde9cd77700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde98069930 con 0x7fde980680c0 2024-01-27T02:32:41.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.241+0000 7fde9cd77700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde9815e820 con 0x7fde980b0f70 2024-01-27T02:32:41.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.241+0000 7fde9cd77700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde98068840 con 0x7fde980cc8d0 2024-01-27T02:32:41.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.241+0000 7fde96d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde980cc8d0 0x7fde980693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:41.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.241+0000 7fde8dd9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fde980680c0 0x7fde980c9340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45640/0 (socket says 172.21.15.7:45640) 2024-01-27T02:32:41.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.241+0000 7fde8dd9b700 1 -- 172.21.15.7:0/20450771 learned_addr learned my addr 172.21.15.7:0/20450771 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:41.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.241+0000 7fde96d9d700 1 --2- 172.21.15.7:0/20450771 >> v2:172.21.15.7:3300/0 conn(0x7fde980cc8d0 0x7fde980693f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58298/0 (socket says 172.21.15.7:58298) 2024-01-27T02:32:41.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.241+0000 7fde96d9d700 1 -- 172.21.15.7:0/20450771 >> v1:172.21.15.188:6789/0 conn(0x7fde980b0f70 legacy=0x7fde980679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:41.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.241+0000 7fde96d9d700 1 -- 172.21.15.7:0/20450771 >> v1:172.21.15.7:6789/0 conn(0x7fde980680c0 legacy=0x7fde980c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:41.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.241+0000 7fde96d9d700 1 -- 172.21.15.7:0/20450771 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9815dc10 con 0x7fde980cc8d0 2024-01-27T02:32:41.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.241+0000 7fde95d9b700 1 -- 172.21.15.7:0/20450771 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1845840122 0 0) 0x7fde98069930 con 0x7fde980680c0 2024-01-27T02:32:41.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.242+0000 7fde96d9d700 1 --2- 172.21.15.7:0/20450771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde980cc8d0 0x7fde980693f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fde90009e60 tx=0x7fde90009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955e3c5866211fd7 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:41.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.242+0000 7fde95d9b700 1 -- 172.21.15.7:0/20450771 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9000e070 con 0x7fde980cc8d0 2024-01-27T02:32:41.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.242+0000 7fde95d9b700 1 -- 172.21.15.7:0/20450771 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9000bc30 con 0x7fde980cc8d0 2024-01-27T02:32:41.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.242+0000 7fde95d9b700 1 -- 172.21.15.7:0/20450771 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde90012430 con 0x7fde980cc8d0 2024-01-27T02:32:41.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.242+0000 7fde9cd77700 1 -- 172.21.15.7:0/20450771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde980cc8d0 msgr2=0x7fde980693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:41.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.242+0000 7fde9cd77700 1 --2- 172.21.15.7:0/20450771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde980cc8d0 0x7fde980693f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fde90009e60 tx=0x7fde90009780 comp rx=0 tx=0).stop 2024-01-27T02:32:41.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.242+0000 7fde9cd77700 1 -- 172.21.15.7:0/20450771 shutdown_connections 2024-01-27T02:32:41.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.242+0000 7fde9cd77700 1 --2- 172.21.15.7:0/20450771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde980cc8d0 0x7fde980693f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.242+0000 7fde9cd77700 1 -- 172.21.15.7:0/20450771 >> 172.21.15.7:0/20450771 conn(0x7fde98063580 msgr2=0x7fde98063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:41.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.242+0000 7fde9cd77700 1 -- 172.21.15.7:0/20450771 shutdown_connections 2024-01-27T02:32:41.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.242+0000 7fde9cd77700 1 -- 172.21.15.7:0/20450771 wait complete. 2024-01-27T02:32:41.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.244+0000 7fde9cd77700 1 Processor -- start 2024-01-27T02:32:41.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.244+0000 7fde9cd77700 1 -- start start 2024-01-27T02:32:41.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.245+0000 7fde9cd77700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde980680c0 0x7fde980aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:41.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.245+0000 7fde9cd77700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde980b0f70 0x7fde980acad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:41.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.245+0000 7fde9cd77700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde980cc8d0 0x7fde980ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:41.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.245+0000 7fde9cd77700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde980abdc0 con 0x7fde980680c0 2024-01-27T02:32:41.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.245+0000 7fde9cd77700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde980ab1b0 con 0x7fde980b0f70 2024-01-27T02:32:41.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.245+0000 7fde9cd77700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde980ab490 con 0x7fde980cc8d0 2024-01-27T02:32:41.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.245+0000 7fde9659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde980680c0 0x7fde980aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:41.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.245+0000 7fde9659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde980680c0 0x7fde980aa8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58302/0 (socket says 172.21.15.7:58302) 2024-01-27T02:32:41.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.245+0000 7fde9659c700 1 -- 172.21.15.7:0/4030740726 learned_addr learned my addr 172.21.15.7:0/4030740726 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:41.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.245+0000 7fde96d9d700 1 --2- 172.21.15.7:0/4030740726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde980cc8d0 0x7fde980ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:41.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.245+0000 7fde8dd9b700 1 --2- 172.21.15.7:0/4030740726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde980b0f70 0x7fde980acad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:41.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.245+0000 7fde9659c700 1 -- 172.21.15.7:0/4030740726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde980cc8d0 msgr2=0x7fde980ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:41.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.245+0000 7fde9659c700 1 --2- 172.21.15.7:0/4030740726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde980cc8d0 0x7fde980ad010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.245+0000 7fde9659c700 1 -- 172.21.15.7:0/4030740726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde980b0f70 msgr2=0x7fde980acad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:41.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.245+0000 7fde9659c700 1 --2- 172.21.15.7:0/4030740726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde980b0f70 0x7fde980acad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.245+0000 7fde9659c700 1 -- 172.21.15.7:0/4030740726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde90009a20 con 0x7fde980680c0 2024-01-27T02:32:41.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.245+0000 7fde8dd9b700 1 --2- 172.21.15.7:0/4030740726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde980b0f70 0x7fde980acad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:41.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.245+0000 7fde9659c700 1 --2- 172.21.15.7:0/4030740726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde980680c0 0x7fde980aa8b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fde8800c590 tx=0x7fde8800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:41.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.245+0000 7fde8effd700 1 -- 172.21.15.7:0/4030740726 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde88011590 con 0x7fde980680c0 2024-01-27T02:32:41.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.245+0000 7fde8effd700 1 -- 172.21.15.7:0/4030740726 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde880116f0 con 0x7fde980680c0 2024-01-27T02:32:41.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.245+0000 7fde9cd77700 1 -- 172.21.15.7:0/4030740726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde980ab770 con 0x7fde980680c0 2024-01-27T02:32:41.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.245+0000 7fde8effd700 1 -- 172.21.15.7:0/4030740726 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde88011ab0 con 0x7fde980680c0 2024-01-27T02:32:41.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.246+0000 7fde9cd77700 1 -- 172.21.15.7:0/4030740726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde98150150 con 0x7fde980680c0 2024-01-27T02:32:41.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.247+0000 7fde9cd77700 1 -- 172.21.15.7:0/4030740726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde98072060 con 0x7fde980680c0 2024-01-27T02:32:41.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.247+0000 7fde8effd700 1 -- 172.21.15.7:0/4030740726 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fde8801b3b0 con 0x7fde980680c0 2024-01-27T02:32:41.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.247+0000 7fde8effd700 1 --2- 172.21.15.7:0/4030740726 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fde78066f30 0x7fde780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:41.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.248+0000 7fde8effd700 1 -- 172.21.15.7:0/4030740726 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fde88083ac0 con 0x7fde980680c0 2024-01-27T02:32:41.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.248+0000 7fde8dd9b700 1 --2- 172.21.15.7:0/4030740726 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fde78066f30 0x7fde780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:41.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.248+0000 7fde8dd9b700 1 --2- 172.21.15.7:0/4030740726 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fde78066f30 0x7fde780693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fde980b0ac0 tx=0x7fde80009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:41.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.252+0000 7fde8effd700 1 -- 172.21.15.7:0/4030740726 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde88050d30 con 0x7fde980680c0 2024-01-27T02:32:41.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.456+0000 7fde9cd77700 1 -- 172.21.15.7:0/4030740726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde9815def0 con 0x7fde980680c0 2024-01-27T02:32:41.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.456+0000 7fde8effd700 1 -- 172.21.15.7:0/4030740726 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fde88017020 con 0x7fde980680c0 2024-01-27T02:32:41.457 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-27T02:32:41.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.458+0000 7fde9cd77700 1 -- 172.21.15.7:0/4030740726 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fde78066f30 msgr2=0x7fde780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:41.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.458+0000 7fde9cd77700 1 --2- 172.21.15.7:0/4030740726 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fde78066f30 0x7fde780693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fde980b0ac0 tx=0x7fde80009040 comp rx=0 tx=0).stop 2024-01-27T02:32:41.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.458+0000 7fde9cd77700 1 -- 172.21.15.7:0/4030740726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde980680c0 msgr2=0x7fde980aa8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:41.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.458+0000 7fde9cd77700 1 --2- 172.21.15.7:0/4030740726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde980680c0 0x7fde980aa8b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fde8800c590 tx=0x7fde8800c950 comp rx=0 tx=0).stop 2024-01-27T02:32:41.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.458+0000 7fde9cd77700 1 -- 172.21.15.7:0/4030740726 shutdown_connections 2024-01-27T02:32:41.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.458+0000 7fde9cd77700 1 --2- 172.21.15.7:0/4030740726 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fde78066f30 0x7fde780693e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.458+0000 7fde9cd77700 1 --2- 172.21.15.7:0/4030740726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde980cc8d0 0x7fde980ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.458+0000 7fde9cd77700 1 --2- 172.21.15.7:0/4030740726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde980680c0 0x7fde980aa8b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.458+0000 7fde9cd77700 1 --2- 172.21.15.7:0/4030740726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde980b0f70 0x7fde980acad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.458+0000 7fde9cd77700 1 -- 172.21.15.7:0/4030740726 >> 172.21.15.7:0/4030740726 conn(0x7fde98063580 msgr2=0x7fde980b2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:41.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.458+0000 7fde9cd77700 1 -- 172.21.15.7:0/4030740726 shutdown_connections 2024-01-27T02:32:41.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.458+0000 7fde9cd77700 1 -- 172.21.15.7:0/4030740726 wait complete. 2024-01-27T02:32:41.472 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-27T02:32:41.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.607+0000 7f51a23f2700 1 Processor -- start 2024-01-27T02:32:41.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.607+0000 7f51a23f2700 1 -- start start 2024-01-27T02:32:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.608+0000 7f51a23f2700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f519c154920 0x7f519c0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.608+0000 7f51a23f2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f519c0ad7c0 0x7f519c063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.608+0000 7f51a23f2700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f519c0adec0 con 0x7f519c154550 2024-01-27T02:32:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.608+0000 7f51a23f2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519c0ae1a0 con 0x7f519c0ad7c0 2024-01-27T02:32:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.608+0000 7f51a23f2700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519c15a850 con 0x7f519c154920 2024-01-27T02:32:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.608+0000 7f51a098f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f519c0ad7c0 0x7f519c063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:41.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.608+0000 7f51a098f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f519c0ad7c0 0x7f519c063580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50228/0 (socket says 172.21.15.7:50228) 2024-01-27T02:32:41.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.608+0000 7f51a098f700 1 -- 172.21.15.7:0/194009656 learned_addr learned my addr 172.21.15.7:0/194009656 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:41.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.608+0000 7f519b7fe700 1 --2- 172.21.15.7:0/194009656 >> v2:172.21.15.188:3300/0 conn(0x7f519c154920 0x7f519c0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:41.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.608+0000 7f51a098f700 1 -- 172.21.15.7:0/194009656 >> v2:172.21.15.188:3300/0 conn(0x7f519c154920 msgr2=0x7f519c0ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:41.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.608+0000 7f51a098f700 1 --2- 172.21.15.7:0/194009656 >> v2:172.21.15.188:3300/0 conn(0x7f519c154920 0x7f519c0ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.608+0000 7f51a098f700 1 -- 172.21.15.7:0/194009656 >> v1:172.21.15.188:6789/0 conn(0x7f519c154550 legacy=0x7f519c15b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:41.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.608+0000 7f51a098f700 1 -- 172.21.15.7:0/194009656 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519c15ab30 con 0x7f519c0ad7c0 2024-01-27T02:32:41.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.608+0000 7f51a098f700 1 --2- 172.21.15.7:0/194009656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519c0ad7c0 0x7f519c063580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f519c0b32f0 tx=0x7f5194009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311d3510a4997df2 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:41.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.609+0000 7f519affd700 1 -- 172.21.15.7:0/194009656 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519400e070 con 0x7f519c0ad7c0 2024-01-27T02:32:41.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.609+0000 7f519affd700 1 -- 172.21.15.7:0/194009656 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519400ba20 con 0x7f519c0ad7c0 2024-01-27T02:32:41.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.609+0000 7f519affd700 1 -- 172.21.15.7:0/194009656 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51940092e0 con 0x7f519c0ad7c0 2024-01-27T02:32:41.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.609+0000 7f51a23f2700 1 -- 172.21.15.7:0/194009656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519c0ad7c0 msgr2=0x7f519c063580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:41.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.609+0000 7f51a23f2700 1 --2- 172.21.15.7:0/194009656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519c0ad7c0 0x7f519c063580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f519c0b32f0 tx=0x7f5194009840 comp rx=0 tx=0).stop 2024-01-27T02:32:41.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.609+0000 7f51a23f2700 1 -- 172.21.15.7:0/194009656 shutdown_connections 2024-01-27T02:32:41.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.609+0000 7f51a23f2700 1 --2- 172.21.15.7:0/194009656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519c0ad7c0 0x7f519c063580 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.609+0000 7f51a23f2700 1 --2- 172.21.15.7:0/194009656 >> v2:172.21.15.188:3300/0 conn(0x7f519c154920 0x7f519c0ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.609+0000 7f51a23f2700 1 -- 172.21.15.7:0/194009656 >> 172.21.15.7:0/194009656 conn(0x7f519c068480 msgr2=0x7f519c0cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:41.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.609+0000 7f51a23f2700 1 -- 172.21.15.7:0/194009656 shutdown_connections 2024-01-27T02:32:41.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.609+0000 7f51a23f2700 1 -- 172.21.15.7:0/194009656 wait complete. 2024-01-27T02:32:41.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.610+0000 7f51a23f2700 1 Processor -- start 2024-01-27T02:32:41.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.611+0000 7f51a23f2700 1 -- start start 2024-01-27T02:32:41.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.611+0000 7f51a23f2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519c0ad7c0 0x7f519c156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:41.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.611+0000 7f51a23f2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f519c154550 0x7f519c154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:41.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.611+0000 7f51a23f2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f519c154920 0x7f519c155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:41.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.611+0000 7f51a23f2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c1565f0 con 0x7f519c0ad7c0 2024-01-27T02:32:41.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.611+0000 7f51a23f2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c155770 con 0x7f519c154550 2024-01-27T02:32:41.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.611+0000 7f51a23f2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c155970 con 0x7f519c154920 2024-01-27T02:32:41.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.612+0000 7f519bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519c0ad7c0 0x7f519c156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:41.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.612+0000 7f519bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519c0ad7c0 0x7f519c156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50238/0 (socket says 172.21.15.7:50238) 2024-01-27T02:32:41.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.612+0000 7f519bfff700 1 -- 172.21.15.7:0/2428230429 learned_addr learned my addr 172.21.15.7:0/2428230429 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:41.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.612+0000 7f51a098f700 1 --2- 172.21.15.7:0/2428230429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f519c154920 0x7f519c155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:41.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.612+0000 7f519b7fe700 1 --2- 172.21.15.7:0/2428230429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f519c154550 0x7f519c154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:41.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.612+0000 7f519bfff700 1 -- 172.21.15.7:0/2428230429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f519c154920 msgr2=0x7f519c155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:41.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.612+0000 7f519bfff700 1 --2- 172.21.15.7:0/2428230429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f519c154920 0x7f519c155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.612+0000 7f519bfff700 1 -- 172.21.15.7:0/2428230429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f519c154550 msgr2=0x7f519c154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:41.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.612+0000 7f519bfff700 1 --2- 172.21.15.7:0/2428230429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f519c154550 0x7f519c154cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.612+0000 7f519bfff700 1 -- 172.21.15.7:0/2428230429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5194009ae0 con 0x7f519c0ad7c0 2024-01-27T02:32:41.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.612+0000 7f519b7fe700 1 --2- 172.21.15.7:0/2428230429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f519c154550 0x7f519c154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:41.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.612+0000 7f519bfff700 1 --2- 172.21.15.7:0/2428230429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519c0ad7c0 0x7f519c156b90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f518c00c590 tx=0x7f518c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:41.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.612+0000 7f5183fff700 1 -- 172.21.15.7:0/2428230429 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c011590 con 0x7f519c0ad7c0 2024-01-27T02:32:41.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.612+0000 7f5183fff700 1 -- 172.21.15.7:0/2428230429 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c011710 con 0x7f519c0ad7c0 2024-01-27T02:32:41.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.612+0000 7f51a23f2700 1 -- 172.21.15.7:0/2428230429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f519c155c50 con 0x7f519c0ad7c0 2024-01-27T02:32:41.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.612+0000 7f5183fff700 1 -- 172.21.15.7:0/2428230429 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c011ad0 con 0x7f519c0ad7c0 2024-01-27T02:32:41.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.612+0000 7f51a23f2700 1 -- 172.21.15.7:0/2428230429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519c156210 con 0x7f519c0ad7c0 2024-01-27T02:32:41.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.614+0000 7f51a23f2700 1 -- 172.21.15.7:0/2428230429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f519c072060 con 0x7f519c0ad7c0 2024-01-27T02:32:41.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.614+0000 7f5183fff700 1 -- 172.21.15.7:0/2428230429 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f518c01b3b0 con 0x7f519c0ad7c0 2024-01-27T02:32:41.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.614+0000 7f5183fff700 1 --2- 172.21.15.7:0/2428230429 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5188066f30 0x7f51880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:41.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.615+0000 7f5183fff700 1 -- 172.21.15.7:0/2428230429 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f518c083b10 con 0x7f519c0ad7c0 2024-01-27T02:32:41.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.615+0000 7f519b7fe700 1 --2- 172.21.15.7:0/2428230429 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5188066f30 0x7f51880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:41.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.615+0000 7f519b7fe700 1 --2- 172.21.15.7:0/2428230429 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5188066f30 0x7f51880693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f519c0c2a40 tx=0x7f5190008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:41.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.619+0000 7f5183fff700 1 -- 172.21.15.7:0/2428230429 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f518c050ce0 con 0x7f519c0ad7c0 2024-01-27T02:32:41.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.822+0000 7f51a23f2700 1 -- 172.21.15.7:0/2428230429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f519c0b2ac0 con 0x7f519c0ad7c0 2024-01-27T02:32:41.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.822+0000 7f5183fff700 1 -- 172.21.15.7:0/2428230429 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f518c017070 con 0x7f519c0ad7c0 2024-01-27T02:32:41.823 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-27T02:32:41.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.824+0000 7f51a23f2700 1 -- 172.21.15.7:0/2428230429 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5188066f30 msgr2=0x7f51880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:41.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.824+0000 7f51a23f2700 1 --2- 172.21.15.7:0/2428230429 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5188066f30 0x7f51880693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f519c0c2a40 tx=0x7f5190008040 comp rx=0 tx=0).stop 2024-01-27T02:32:41.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.824+0000 7f51a23f2700 1 -- 172.21.15.7:0/2428230429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519c0ad7c0 msgr2=0x7f519c156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:41.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.824+0000 7f51a23f2700 1 --2- 172.21.15.7:0/2428230429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519c0ad7c0 0x7f519c156b90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f518c00c590 tx=0x7f518c00c8a0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.824+0000 7f51a23f2700 1 -- 172.21.15.7:0/2428230429 shutdown_connections 2024-01-27T02:32:41.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.824+0000 7f51a23f2700 1 --2- 172.21.15.7:0/2428230429 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5188066f30 0x7f51880693e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.824+0000 7f51a23f2700 1 --2- 172.21.15.7:0/2428230429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f519c154920 0x7f519c155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.824+0000 7f51a23f2700 1 --2- 172.21.15.7:0/2428230429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519c0ad7c0 0x7f519c156b90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.824+0000 7f51a23f2700 1 --2- 172.21.15.7:0/2428230429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f519c154550 0x7f519c154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.824+0000 7f51a23f2700 1 -- 172.21.15.7:0/2428230429 >> 172.21.15.7:0/2428230429 conn(0x7f519c068480 msgr2=0x7f519c063580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:41.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.824+0000 7f51a23f2700 1 -- 172.21.15.7:0/2428230429 shutdown_connections 2024-01-27T02:32:41.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.824+0000 7f51a23f2700 1 -- 172.21.15.7:0/2428230429 wait complete. 2024-01-27T02:32:41.838 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:32:41.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.970+0000 7ff2ddb53700 1 Processor -- start 2024-01-27T02:32:41.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.970+0000 7ff2ddb53700 1 -- start start 2024-01-27T02:32:41.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.971+0000 7ff2ddb53700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff2d80c3ae0 0x7ff2d80cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:41.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.971+0000 7ff2ddb53700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff2d8158070 0x7ff2d80cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:41.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.971+0000 7ff2ddb53700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d80ca020 con 0x7ff2d8157ca0 2024-01-27T02:32:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.971+0000 7ff2ddb53700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d80c5210 con 0x7ff2d80c3ae0 2024-01-27T02:32:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.971+0000 7ff2ddb53700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d80c54f0 con 0x7ff2d8158070 2024-01-27T02:32:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.971+0000 7ff2d6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff2d80c3ae0 0x7ff2d80cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.972+0000 7ff2d77fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff2d8157ca0 0x7ff2d80c4a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42856/0 (socket says 172.21.15.7:42856) 2024-01-27T02:32:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.972+0000 7ff2d6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff2d80c3ae0 0x7ff2d80cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50240/0 (socket says 172.21.15.7:50240) 2024-01-27T02:32:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.972+0000 7ff2d77fe700 1 -- 172.21.15.7:0/3059076599 learned_addr learned my addr 172.21.15.7:0/3059076599 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.972+0000 7ff2d7fff700 1 --2- 172.21.15.7:0/3059076599 >> v2:172.21.15.188:3300/0 conn(0x7ff2d8158070 0x7ff2d80cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.972+0000 7ff2d6ffd700 1 -- 172.21.15.7:0/3059076599 >> v2:172.21.15.188:3300/0 conn(0x7ff2d8158070 msgr2=0x7ff2d80cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.972+0000 7ff2d6ffd700 1 --2- 172.21.15.7:0/3059076599 >> v2:172.21.15.188:3300/0 conn(0x7ff2d8158070 0x7ff2d80cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.972+0000 7ff2d6ffd700 1 -- 172.21.15.7:0/3059076599 >> v1:172.21.15.7:6789/0 conn(0x7ff2d8157ca0 legacy=0x7ff2d80c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.972+0000 7ff2d6ffd700 1 -- 172.21.15.7:0/3059076599 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d80c57d0 con 0x7ff2d80c3ae0 2024-01-27T02:32:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.972+0000 7ff2d67fc700 1 -- 172.21.15.7:0/3059076599 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1989735581 0 0) 0x7ff2d80ca020 con 0x7ff2d8157ca0 2024-01-27T02:32:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.972+0000 7ff2d6ffd700 1 --2- 172.21.15.7:0/3059076599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2d80c3ae0 0x7ff2d80cd4e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff2d806f890 tx=0x7ff2cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80569dcf6bf7ad5 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:41.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.972+0000 7ff2d67fc700 1 -- 172.21.15.7:0/3059076599 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc011630 con 0x7ff2d80c3ae0 2024-01-27T02:32:41.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.972+0000 7ff2d67fc700 1 -- 172.21.15.7:0/3059076599 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2cc011790 con 0x7ff2d80c3ae0 2024-01-27T02:32:41.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.972+0000 7ff2d67fc700 1 -- 172.21.15.7:0/3059076599 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc011b30 con 0x7ff2d80c3ae0 2024-01-27T02:32:41.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.973+0000 7ff2ddb53700 1 -- 172.21.15.7:0/3059076599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2d80c3ae0 msgr2=0x7ff2d80cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:41.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.973+0000 7ff2ddb53700 1 --2- 172.21.15.7:0/3059076599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2d80c3ae0 0x7ff2d80cd4e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff2d806f890 tx=0x7ff2cc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.973+0000 7ff2ddb53700 1 -- 172.21.15.7:0/3059076599 shutdown_connections 2024-01-27T02:32:41.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.973+0000 7ff2ddb53700 1 --2- 172.21.15.7:0/3059076599 >> v2:172.21.15.188:3300/0 conn(0x7ff2d8158070 0x7ff2d80cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.973+0000 7ff2ddb53700 1 --2- 172.21.15.7:0/3059076599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2d80c3ae0 0x7ff2d80cd4e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.973+0000 7ff2ddb53700 1 -- 172.21.15.7:0/3059076599 >> 172.21.15.7:0/3059076599 conn(0x7ff2d8098020 msgr2=0x7ff2d80711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:41.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.973+0000 7ff2ddb53700 1 -- 172.21.15.7:0/3059076599 shutdown_connections 2024-01-27T02:32:41.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.973+0000 7ff2ddb53700 1 -- 172.21.15.7:0/3059076599 wait complete. 2024-01-27T02:32:41.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.974+0000 7ff2ddb53700 1 Processor -- start 2024-01-27T02:32:41.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.974+0000 7ff2ddb53700 1 -- start start 2024-01-27T02:32:41.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.975+0000 7ff2ddb53700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d80c3ae0 0x7ff2d8167470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.975+0000 7ff2ddb53700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2d8157ca0 0x7ff2d80c7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.975+0000 7ff2ddb53700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2d8158070 0x7ff2d80c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.975+0000 7ff2ddb53700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d81540d0 con 0x7ff2d8157ca0 2024-01-27T02:32:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.975+0000 7ff2ddb53700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d81543b0 con 0x7ff2d8158070 2024-01-27T02:32:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.975+0000 7ff2ddb53700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d8154690 con 0x7ff2d80c3ae0 2024-01-27T02:32:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.975+0000 7ff2d6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2d8157ca0 0x7ff2d80c7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.975+0000 7ff2d6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2d8157ca0 0x7ff2d80c7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50244/0 (socket says 172.21.15.7:50244) 2024-01-27T02:32:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.975+0000 7ff2d6ffd700 1 -- 172.21.15.7:0/1922047076 learned_addr learned my addr 172.21.15.7:0/1922047076 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.975+0000 7ff2d7fff700 1 --2- 172.21.15.7:0/1922047076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2d8158070 0x7ff2d80c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.975+0000 7ff2d77fe700 1 --2- 172.21.15.7:0/1922047076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d80c3ae0 0x7ff2d8167470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.976+0000 7ff2d6ffd700 1 -- 172.21.15.7:0/1922047076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d80c3ae0 msgr2=0x7ff2d8167470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.976+0000 7ff2d6ffd700 1 --2- 172.21.15.7:0/1922047076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d80c3ae0 0x7ff2d8167470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.976+0000 7ff2d6ffd700 1 -- 172.21.15.7:0/1922047076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2d8158070 msgr2=0x7ff2d80c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.976+0000 7ff2d6ffd700 1 --2- 172.21.15.7:0/1922047076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2d8158070 0x7ff2d80c7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.976+0000 7ff2d6ffd700 1 -- 172.21.15.7:0/1922047076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2cc009420 con 0x7ff2d8157ca0 2024-01-27T02:32:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.976+0000 7ff2d7fff700 1 --2- 172.21.15.7:0/1922047076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2d8158070 0x7ff2d80c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.976+0000 7ff2d6ffd700 1 --2- 172.21.15.7:0/1922047076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2d8157ca0 0x7ff2d80c7400 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff2d806ea80 tx=0x7ff2cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.976+0000 7ff2bf7fe700 1 -- 172.21.15.7:0/1922047076 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc011b10 con 0x7ff2d8157ca0 2024-01-27T02:32:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.976+0000 7ff2bf7fe700 1 -- 172.21.15.7:0/1922047076 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2cc0253f0 con 0x7ff2d8157ca0 2024-01-27T02:32:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.976+0000 7ff2ddb53700 1 -- 172.21.15.7:0/1922047076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d8154970 con 0x7ff2d8157ca0 2024-01-27T02:32:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.976+0000 7ff2bf7fe700 1 -- 172.21.15.7:0/1922047076 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc025790 con 0x7ff2d8157ca0 2024-01-27T02:32:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.976+0000 7ff2ddb53700 1 -- 172.21.15.7:0/1922047076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d8154f30 con 0x7ff2d8157ca0 2024-01-27T02:32:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.977+0000 7ff2ddb53700 1 -- 172.21.15.7:0/1922047076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d804c710 con 0x7ff2d8157ca0 2024-01-27T02:32:41.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.978+0000 7ff2bf7fe700 1 -- 172.21.15.7:0/1922047076 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff2cc025b80 con 0x7ff2d8157ca0 2024-01-27T02:32:41.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.978+0000 7ff2bf7fe700 1 --2- 172.21.15.7:0/1922047076 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff2c4066f30 0x7ff2c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:41.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.978+0000 7ff2bf7fe700 1 -- 172.21.15.7:0/1922047076 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff2cc085e40 con 0x7ff2d8157ca0 2024-01-27T02:32:41.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.978+0000 7ff2d77fe700 1 --2- 172.21.15.7:0/1922047076 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff2c4066f30 0x7ff2c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:41.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.979+0000 7ff2d77fe700 1 --2- 172.21.15.7:0/1922047076 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff2c4066f30 0x7ff2c40693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80011e0 tx=0x7ff2c8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:41.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:41.982+0000 7ff2bf7fe700 1 -- 172.21.15.7:0/1922047076 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2cc0530b0 con 0x7ff2d8157ca0 2024-01-27T02:32:42.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.238+0000 7ff2ddb53700 1 -- 172.21.15.7:0/1922047076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2d80c9bb0 con 0x7ff2d8157ca0 2024-01-27T02:32:42.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.239+0000 7ff2bf7fe700 1 -- 172.21.15.7:0/1922047076 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7ff2cc0566d0 con 0x7ff2d8157ca0 2024-01-27T02:32:42.239 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:42.239 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3594367777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3594367777},{"type":"v1","addr":"172.21.15.7:6834","nonce":3594367777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1079101213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1079101213},{"type":"v1","addr":"172.21.15.188:6837","nonce":1079101213}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3241424244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3241424244},{"type":"v1","addr":"172.21.15.7:6835","nonce":3241424244}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:32:06.045693+0000","modified":"2024-01-27T02:32:09.806398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6836/2571237935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2571237935},{"type":"v1","addr":"172.21.15.188:6836","nonce":2571237935}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-27T02:32:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.241+0000 7ff2ddb53700 1 -- 172.21.15.7:0/1922047076 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff2c4066f30 msgr2=0x7ff2c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.241+0000 7ff2ddb53700 1 --2- 172.21.15.7:0/1922047076 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff2c4066f30 0x7ff2c40693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80011e0 tx=0x7ff2c8009040 comp rx=0 tx=0).stop 2024-01-27T02:32:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.241+0000 7ff2ddb53700 1 -- 172.21.15.7:0/1922047076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2d8157ca0 msgr2=0x7ff2d80c7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.241+0000 7ff2ddb53700 1 --2- 172.21.15.7:0/1922047076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2d8157ca0 0x7ff2d80c7400 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff2d806ea80 tx=0x7ff2cc017000 comp rx=0 tx=0).stop 2024-01-27T02:32:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.241+0000 7ff2ddb53700 1 -- 172.21.15.7:0/1922047076 shutdown_connections 2024-01-27T02:32:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.241+0000 7ff2ddb53700 1 --2- 172.21.15.7:0/1922047076 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff2c4066f30 0x7ff2c40693e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.241+0000 7ff2ddb53700 1 --2- 172.21.15.7:0/1922047076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2d8158070 0x7ff2d80c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.241+0000 7ff2ddb53700 1 --2- 172.21.15.7:0/1922047076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d80c3ae0 0x7ff2d8167470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.241+0000 7ff2ddb53700 1 --2- 172.21.15.7:0/1922047076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2d8157ca0 0x7ff2d80c7400 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.241+0000 7ff2ddb53700 1 -- 172.21.15.7:0/1922047076 >> 172.21.15.7:0/1922047076 conn(0x7ff2d8098020 msgr2=0x7ff2d806c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.241+0000 7ff2ddb53700 1 -- 172.21.15.7:0/1922047076 shutdown_connections 2024-01-27T02:32:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.241+0000 7ff2ddb53700 1 -- 172.21.15.7:0/1922047076 wait complete. 2024-01-27T02:32:42.242 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-01-27T02:32:42.255 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:32:42.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.386+0000 7fc316efd700 1 Processor -- start 2024-01-27T02:32:42.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.386+0000 7fc316efd700 1 -- start start 2024-01-27T02:32:42.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.387+0000 7fc316efd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc3100b1c00 0x7fc31006e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:42.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.387+0000 7fc316efd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc31015b5f0 0x7fc3101566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:42.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.388+0000 7fc316efd700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc3100b1fd0 0x7fc310156c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:42.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.388+0000 7fc316efd700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc310157610 con 0x7fc3100b1c00 2024-01-27T02:32:42.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.388+0000 7fc316efd700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc31006ee00 con 0x7fc3100b1fd0 2024-01-27T02:32:42.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.388+0000 7fc316efd700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc31006f0e0 con 0x7fc31015b5f0 2024-01-27T02:32:42.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.388+0000 7fc314c99700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc3100b1c00 0x7fc31006e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:42.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.388+0000 7fc314c99700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc3100b1c00 0x7fc31006e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50246/0 (socket says 172.21.15.7:50246) 2024-01-27T02:32:42.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.388+0000 7fc314c99700 1 -- 172.21.15.7:0/2492255287 learned_addr learned my addr 172.21.15.7:0/2492255287 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:42.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.388+0000 7fc30ffff700 1 --2- 172.21.15.7:0/2492255287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc31015b5f0 0x7fc3101566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:42.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.388+0000 7fc31549a700 1 --2- 172.21.15.7:0/2492255287 >> v2:172.21.15.188:3300/0 conn(0x7fc3100b1fd0 0x7fc310156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:42.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.388+0000 7fc314c99700 1 -- 172.21.15.7:0/2492255287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc31015b5f0 msgr2=0x7fc3101566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:42.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.388+0000 7fc314c99700 1 --2- 172.21.15.7:0/2492255287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc31015b5f0 0x7fc3101566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:42.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.388+0000 7fc314c99700 1 -- 172.21.15.7:0/2492255287 >> v2:172.21.15.188:3300/0 conn(0x7fc3100b1fd0 msgr2=0x7fc310156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:42.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.388+0000 7fc314c99700 1 --2- 172.21.15.7:0/2492255287 >> v2:172.21.15.188:3300/0 conn(0x7fc3100b1fd0 0x7fc310156c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:42.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.388+0000 7fc314c99700 1 -- 172.21.15.7:0/2492255287 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31006f3c0 con 0x7fc3100b1c00 2024-01-27T02:32:42.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.388+0000 7fc314c99700 1 --2- 172.21.15.7:0/2492255287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3100b1c00 0x7fc31006e540 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc31006c6a0 tx=0x7fc300009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63dbcc3257417ac server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:42.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.388+0000 7fc30f7fe700 1 -- 172.21.15.7:0/2492255287 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30000b860 con 0x7fc3100b1c00 2024-01-27T02:32:42.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.388+0000 7fc30f7fe700 1 -- 172.21.15.7:0/2492255287 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30000b9c0 con 0x7fc3100b1c00 2024-01-27T02:32:42.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.388+0000 7fc30f7fe700 1 -- 172.21.15.7:0/2492255287 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30000bd60 con 0x7fc3100b1c00 2024-01-27T02:32:42.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.389+0000 7fc316efd700 1 -- 172.21.15.7:0/2492255287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3100b1c00 msgr2=0x7fc31006e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:42.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.389+0000 7fc316efd700 1 --2- 172.21.15.7:0/2492255287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3100b1c00 0x7fc31006e540 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc31006c6a0 tx=0x7fc300009250 comp rx=0 tx=0).stop 2024-01-27T02:32:42.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.389+0000 7fc316efd700 1 -- 172.21.15.7:0/2492255287 shutdown_connections 2024-01-27T02:32:42.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.389+0000 7fc316efd700 1 --2- 172.21.15.7:0/2492255287 >> v2:172.21.15.188:3300/0 conn(0x7fc3100b1fd0 0x7fc310156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:42.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.389+0000 7fc316efd700 1 --2- 172.21.15.7:0/2492255287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3100b1c00 0x7fc31006e540 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:42.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.389+0000 7fc316efd700 1 --2- 172.21.15.7:0/2492255287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc31015b5f0 0x7fc3101566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:42.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.389+0000 7fc316efd700 1 -- 172.21.15.7:0/2492255287 >> 172.21.15.7:0/2492255287 conn(0x7fc310068430 msgr2=0x7fc31006be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:42.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.389+0000 7fc316efd700 1 -- 172.21.15.7:0/2492255287 shutdown_connections 2024-01-27T02:32:42.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.389+0000 7fc316efd700 1 -- 172.21.15.7:0/2492255287 wait complete. 2024-01-27T02:32:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.390+0000 7fc316efd700 1 Processor -- start 2024-01-27T02:32:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.391+0000 7fc316efd700 1 -- start start 2024-01-27T02:32:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.391+0000 7fc316efd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3100b1c00 0x7fc310168f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.391+0000 7fc316efd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3100b1fd0 0x7fc310140700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.391+0000 7fc316efd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc31015b5f0 0x7fc310147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.391+0000 7fc316efd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31014e910 con 0x7fc3100b1c00 2024-01-27T02:32:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.391+0000 7fc316efd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc310169440 con 0x7fc31015b5f0 2024-01-27T02:32:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.391+0000 7fc316efd700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3101411d0 con 0x7fc3100b1fd0 2024-01-27T02:32:42.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.391+0000 7fc314c99700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3100b1c00 0x7fc310168f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:42.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.391+0000 7fc314c99700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3100b1c00 0x7fc310168f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50256/0 (socket says 172.21.15.7:50256) 2024-01-27T02:32:42.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.392+0000 7fc314c99700 1 -- 172.21.15.7:0/3371241783 learned_addr learned my addr 172.21.15.7:0/3371241783 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:42.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.392+0000 7fc31549a700 1 --2- 172.21.15.7:0/3371241783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc31015b5f0 0x7fc310147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:42.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.392+0000 7fc30ffff700 1 --2- 172.21.15.7:0/3371241783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3100b1fd0 0x7fc310140700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:42.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.392+0000 7fc314c99700 1 -- 172.21.15.7:0/3371241783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3100b1fd0 msgr2=0x7fc310140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:42.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.392+0000 7fc314c99700 1 --2- 172.21.15.7:0/3371241783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3100b1fd0 0x7fc310140700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:42.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.392+0000 7fc314c99700 1 -- 172.21.15.7:0/3371241783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc31015b5f0 msgr2=0x7fc310147f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:42.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.392+0000 7fc314c99700 1 --2- 172.21.15.7:0/3371241783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc31015b5f0 0x7fc310147f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:42.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.392+0000 7fc314c99700 1 -- 172.21.15.7:0/3371241783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3000094f0 con 0x7fc3100b1c00 2024-01-27T02:32:42.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.392+0000 7fc30ffff700 1 --2- 172.21.15.7:0/3371241783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3100b1fd0 0x7fc310140700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:42.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.392+0000 7fc314c99700 1 --2- 172.21.15.7:0/3371241783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3100b1c00 0x7fc310168f00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc31006dfe0 tx=0x7fc300017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:42.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.392+0000 7fc30cff9700 1 -- 172.21.15.7:0/3371241783 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc300011e70 con 0x7fc3100b1c00 2024-01-27T02:32:42.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.392+0000 7fc30cff9700 1 -- 172.21.15.7:0/3371241783 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc300024440 con 0x7fc3100b1c00 2024-01-27T02:32:42.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.392+0000 7fc316efd700 1 -- 172.21.15.7:0/3371241783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3101414b0 con 0x7fc3100b1c00 2024-01-27T02:32:42.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.392+0000 7fc30cff9700 1 -- 172.21.15.7:0/3371241783 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc300024800 con 0x7fc3100b1c00 2024-01-27T02:32:42.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.392+0000 7fc316efd700 1 -- 172.21.15.7:0/3371241783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc310141a70 con 0x7fc3100b1c00 2024-01-27T02:32:42.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.393+0000 7fc316efd700 1 -- 172.21.15.7:0/3371241783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc31004c710 con 0x7fc3100b1c00 2024-01-27T02:32:42.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.394+0000 7fc30cff9700 1 -- 172.21.15.7:0/3371241783 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc30001a420 con 0x7fc3100b1c00 2024-01-27T02:32:42.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.394+0000 7fc30cff9700 1 --2- 172.21.15.7:0/3371241783 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc2f0066f30 0x7fc2f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:42.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.394+0000 7fc30cff9700 1 -- 172.21.15.7:0/3371241783 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc3000863f0 con 0x7fc3100b1c00 2024-01-27T02:32:42.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.394+0000 7fc30ffff700 1 --2- 172.21.15.7:0/3371241783 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc2f0066f30 0x7fc2f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:42.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.395+0000 7fc30ffff700 1 --2- 172.21.15.7:0/3371241783 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc2f0066f30 0x7fc2f00693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8006fd0 tx=0x7fc2f8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:42.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.399+0000 7fc30cff9700 1 -- 172.21.15.7:0/3371241783 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc300053660 con 0x7fc3100b1c00 2024-01-27T02:32:42.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.601+0000 7fc316efd700 1 -- 172.21.15.7:0/3371241783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc31006ea80 con 0x7fc3100b1c00 2024-01-27T02:32:42.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.602+0000 7fc30cff9700 1 -- 172.21.15.7:0/3371241783 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15302 (secure 0 0 0) 0x7fc300056c80 con 0x7fc3100b1c00 2024-01-27T02:32:42.603 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:42.603 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:32:07.627822+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:32:03.675611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-27T02:32:05.018885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:32:42.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.604+0000 7fc316efd700 1 -- 172.21.15.7:0/3371241783 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc2f0066f30 msgr2=0x7fc2f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:42.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.604+0000 7fc316efd700 1 --2- 172.21.15.7:0/3371241783 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc2f0066f30 0x7fc2f00693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8006fd0 tx=0x7fc2f8008040 comp rx=0 tx=0).stop 2024-01-27T02:32:42.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.604+0000 7fc316efd700 1 -- 172.21.15.7:0/3371241783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3100b1c00 msgr2=0x7fc310168f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:42.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.604+0000 7fc316efd700 1 --2- 172.21.15.7:0/3371241783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3100b1c00 0x7fc310168f00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc31006dfe0 tx=0x7fc300017000 comp rx=0 tx=0).stop 2024-01-27T02:32:42.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.604+0000 7fc316efd700 1 -- 172.21.15.7:0/3371241783 shutdown_connections 2024-01-27T02:32:42.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.604+0000 7fc316efd700 1 --2- 172.21.15.7:0/3371241783 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc2f0066f30 0x7fc2f00693e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:42.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.604+0000 7fc316efd700 1 --2- 172.21.15.7:0/3371241783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc31015b5f0 0x7fc310147f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:42.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.604+0000 7fc316efd700 1 --2- 172.21.15.7:0/3371241783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3100b1c00 0x7fc310168f00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:42.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.604+0000 7fc316efd700 1 --2- 172.21.15.7:0/3371241783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3100b1fd0 0x7fc310140700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:42.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.604+0000 7fc316efd700 1 -- 172.21.15.7:0/3371241783 >> 172.21.15.7:0/3371241783 conn(0x7fc310068430 msgr2=0x7fc310159460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:42.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.605+0000 7fc316efd700 1 -- 172.21.15.7:0/3371241783 shutdown_connections 2024-01-27T02:32:42.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.605+0000 7fc316efd700 1 -- 172.21.15.7:0/3371241783 wait complete. 2024-01-27T02:32:42.618 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-27T02:32:42.619 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:32:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.750+0000 7f997824f700 1 Processor -- start 2024-01-27T02:32:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.750+0000 7f997824f700 1 -- start start 2024-01-27T02:32:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.751+0000 7f997824f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99700b1af0 0x7f99700b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.751+0000 7f997824f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f99700c5060 0x7f99700c5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.751+0000 7f997824f700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99700c4bd0 con 0x7f99700ca140 2024-01-27T02:32:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.751+0000 7f997824f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99700c59c0 con 0x7f99700c5060 2024-01-27T02:32:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.751+0000 7f997824f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99700cac20 con 0x7f99700b1af0 2024-01-27T02:32:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.751+0000 7f99757ea700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99700b1af0 0x7f99700b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.751+0000 7f99767ec700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f99700c5060 0x7f99700c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.751+0000 7f99757ea700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99700b1af0 0x7f99700b1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:33724/0 (socket says 172.21.15.7:33724) 2024-01-27T02:32:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.751+0000 7f99757ea700 1 -- 172.21.15.7:0/452904900 learned_addr learned my addr 172.21.15.7:0/452904900 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.751+0000 7f9975feb700 1 --1- 172.21.15.7:0/452904900 >> v1:172.21.15.188:6789/0 conn(0x7f99700ca140 0x7f99700ca510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:38678/0 (socket says 172.21.15.7:38678) 2024-01-27T02:32:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.752+0000 7f99757ea700 1 -- 172.21.15.7:0/452904900 >> v2:172.21.15.188:3300/0 conn(0x7f99700c5060 msgr2=0x7f99700c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.752+0000 7f99757ea700 1 --2- 172.21.15.7:0/452904900 >> v2:172.21.15.188:3300/0 conn(0x7f99700c5060 0x7f99700c5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.752+0000 7f99757ea700 1 -- 172.21.15.7:0/452904900 >> v1:172.21.15.188:6789/0 conn(0x7f99700ca140 legacy=0x7f99700ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.752+0000 7f9974fe9700 1 -- 172.21.15.7:0/452904900 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2517389040 0 0) 0x7f99700c4bd0 con 0x7f99700ca140 2024-01-27T02:32:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.752+0000 7f99757ea700 1 -- 172.21.15.7:0/452904900 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99700caea0 con 0x7f99700b1af0 2024-01-27T02:32:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.752+0000 7f99757ea700 1 --2- 172.21.15.7:0/452904900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99700b1af0 0x7f99700b1f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f99600099b0 tx=0x7f9960009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=423c5154830f0a77 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.752+0000 7f9974fe9700 1 -- 172.21.15.7:0/452904900 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9960009ea0 con 0x7f99700b1af0 2024-01-27T02:32:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.752+0000 7f9974fe9700 1 -- 172.21.15.7:0/452904900 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996000b860 con 0x7f99700b1af0 2024-01-27T02:32:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.752+0000 7f9974fe9700 1 -- 172.21.15.7:0/452904900 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996000bc00 con 0x7f99700b1af0 2024-01-27T02:32:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.753+0000 7f997824f700 1 -- 172.21.15.7:0/452904900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99700b1af0 msgr2=0x7f99700b1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.753+0000 7f997824f700 1 --2- 172.21.15.7:0/452904900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99700b1af0 0x7f99700b1f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f99600099b0 tx=0x7f9960009250 comp rx=0 tx=0).stop 2024-01-27T02:32:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.753+0000 7f997824f700 1 -- 172.21.15.7:0/452904900 shutdown_connections 2024-01-27T02:32:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.753+0000 7f997824f700 1 --2- 172.21.15.7:0/452904900 >> v2:172.21.15.188:3300/0 conn(0x7f99700c5060 0x7f99700c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.753+0000 7f997824f700 1 --2- 172.21.15.7:0/452904900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99700b1af0 0x7f99700b1f20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.753+0000 7f997824f700 1 -- 172.21.15.7:0/452904900 >> 172.21.15.7:0/452904900 conn(0x7f99700711c0 msgr2=0x7f99700ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.753+0000 7f997824f700 1 -- 172.21.15.7:0/452904900 shutdown_connections 2024-01-27T02:32:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.753+0000 7f997824f700 1 -- 172.21.15.7:0/452904900 wait complete. 2024-01-27T02:32:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.754+0000 7f997824f700 1 Processor -- start 2024-01-27T02:32:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.754+0000 7f997824f700 1 -- start start 2024-01-27T02:32:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.755+0000 7f997824f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99700b1af0 0x7f99700c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.755+0000 7f997824f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99700c5060 0x7f99700afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.755+0000 7f997824f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99700ca140 0x7f99700b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.755+0000 7f997824f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997014dff0 con 0x7f99700c5060 2024-01-27T02:32:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.755+0000 7f997824f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9970067d80 con 0x7f99700b1af0 2024-01-27T02:32:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.755+0000 7f997824f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99700b0290 con 0x7f99700ca140 2024-01-27T02:32:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.755+0000 7f99757ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99700c5060 0x7f99700afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.755+0000 7f99757ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99700c5060 0x7f99700afbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50262/0 (socket says 172.21.15.7:50262) 2024-01-27T02:32:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.755+0000 7f99757ea700 1 -- 172.21.15.7:0/2908932768 learned_addr learned my addr 172.21.15.7:0/2908932768 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.755+0000 7f99767ec700 1 --2- 172.21.15.7:0/2908932768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99700ca140 0x7f99700b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.755+0000 7f9975feb700 1 --2- 172.21.15.7:0/2908932768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99700b1af0 0x7f99700c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.755+0000 7f99757ea700 1 -- 172.21.15.7:0/2908932768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99700ca140 msgr2=0x7f99700b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.755+0000 7f99757ea700 1 --2- 172.21.15.7:0/2908932768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99700ca140 0x7f99700b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.755+0000 7f99757ea700 1 -- 172.21.15.7:0/2908932768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99700b1af0 msgr2=0x7f99700c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.755+0000 7f99757ea700 1 --2- 172.21.15.7:0/2908932768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99700b1af0 0x7f99700c6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.755+0000 7f99757ea700 1 -- 172.21.15.7:0/2908932768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99600094f0 con 0x7f99700c5060 2024-01-27T02:32:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.755+0000 7f9975feb700 1 --2- 172.21.15.7:0/2908932768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99700b1af0 0x7f99700c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.756+0000 7f99757ea700 1 --2- 172.21.15.7:0/2908932768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99700c5060 0x7f99700afbb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f99700c5b00 tx=0x7f9960017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.756+0000 7f9965ffb700 1 -- 172.21.15.7:0/2908932768 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996000bbe0 con 0x7f99700c5060 2024-01-27T02:32:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.756+0000 7f9965ffb700 1 -- 172.21.15.7:0/2908932768 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9960011420 con 0x7f99700c5060 2024-01-27T02:32:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.756+0000 7f997824f700 1 -- 172.21.15.7:0/2908932768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99700b0570 con 0x7f99700c5060 2024-01-27T02:32:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.756+0000 7f9965ffb700 1 -- 172.21.15.7:0/2908932768 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99600117e0 con 0x7f99700c5060 2024-01-27T02:32:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.756+0000 7f997824f700 1 -- 172.21.15.7:0/2908932768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99700b0b30 con 0x7f99700c5060 2024-01-27T02:32:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.757+0000 7f997824f700 1 -- 172.21.15.7:0/2908932768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9970067f10 con 0x7f99700c5060 2024-01-27T02:32:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.757+0000 7f9965ffb700 1 -- 172.21.15.7:0/2908932768 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9960011bd0 con 0x7f99700c5060 2024-01-27T02:32:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.758+0000 7f9965ffb700 1 --2- 172.21.15.7:0/2908932768 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9954066f30 0x7f99540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.758+0000 7f9965ffb700 1 -- 172.21.15.7:0/2908932768 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9960085e30 con 0x7f99700c5060 2024-01-27T02:32:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.758+0000 7f9975feb700 1 --2- 172.21.15.7:0/2908932768 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9954066f30 0x7f99540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.759+0000 7f9975feb700 1 --2- 172.21.15.7:0/2908932768 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9954066f30 0x7f99540693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f996c007250 tx=0x7f996c007e80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:42.762+0000 7f9965ffb700 1 -- 172.21.15.7:0/2908932768 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9960053000 con 0x7f99700c5060 2024-01-27T02:32:43.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.014+0000 7f997824f700 1 -- 172.21.15.7:0/2908932768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f997004c710 con 0x7f99700c5060 2024-01-27T02:32:43.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.015+0000 7f9965ffb700 1 -- 172.21.15.7:0/2908932768 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f9960015090 con 0x7f99700c5060 2024-01-27T02:32:43.016 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:43.016 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3594367777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3594367777},{"type":"v1","addr":"172.21.15.7:6834","nonce":3594367777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1079101213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1079101213},{"type":"v1","addr":"172.21.15.188:6837","nonce":1079101213}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3241424244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3241424244},{"type":"v1","addr":"172.21.15.7:6835","nonce":3241424244}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:32:06.045693+0000","modified":"2024-01-27T02:32:09.806398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6836/2571237935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2571237935},{"type":"v1","addr":"172.21.15.188:6836","nonce":2571237935}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-27T02:32:43.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.017+0000 7f997824f700 1 -- 172.21.15.7:0/2908932768 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9954066f30 msgr2=0x7f99540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:43.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.017+0000 7f997824f700 1 --2- 172.21.15.7:0/2908932768 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9954066f30 0x7f99540693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f996c007250 tx=0x7f996c007e80 comp rx=0 tx=0).stop 2024-01-27T02:32:43.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.017+0000 7f997824f700 1 -- 172.21.15.7:0/2908932768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99700c5060 msgr2=0x7f99700afbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:43.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.017+0000 7f997824f700 1 --2- 172.21.15.7:0/2908932768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99700c5060 0x7f99700afbb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f99700c5b00 tx=0x7f9960017000 comp rx=0 tx=0).stop 2024-01-27T02:32:43.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.018+0000 7f997824f700 1 -- 172.21.15.7:0/2908932768 shutdown_connections 2024-01-27T02:32:43.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.018+0000 7f997824f700 1 --2- 172.21.15.7:0/2908932768 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9954066f30 0x7f99540693e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.018+0000 7f997824f700 1 --2- 172.21.15.7:0/2908932768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99700ca140 0x7f99700b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.018+0000 7f997824f700 1 --2- 172.21.15.7:0/2908932768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99700b1af0 0x7f99700c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.018+0000 7f997824f700 1 --2- 172.21.15.7:0/2908932768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99700c5060 0x7f99700afbb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.018+0000 7f997824f700 1 -- 172.21.15.7:0/2908932768 >> 172.21.15.7:0/2908932768 conn(0x7f99700711c0 msgr2=0x7f997006b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:43.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.018+0000 7f997824f700 1 -- 172.21.15.7:0/2908932768 shutdown_connections 2024-01-27T02:32:43.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.018+0000 7f997824f700 1 -- 172.21.15.7:0/2908932768 wait complete. 2024-01-27T02:32:43.019 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-01-27T02:32:43.032 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:32:43.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.163+0000 7f9f60996700 1 Processor -- start 2024-01-27T02:32:43.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.163+0000 7f9f60996700 1 -- start start 2024-01-27T02:32:43.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.164+0000 7f9f60996700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9f5c063c90 0x7f9f5c164cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:43.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.164+0000 7f9f60996700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5c0b0e30 con 0x7f9f5c0ac640 2024-01-27T02:32:43.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.164+0000 7f9f60996700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5c0acab0 con 0x7f9f5c159fb0 2024-01-27T02:32:43.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.164+0000 7f9f60996700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f5c0ad2e0 con 0x7f9f5c063c90 2024-01-27T02:32:43.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.164+0000 7f9f5ad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9f5c063c90 0x7f9f5c164cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:43.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.164+0000 7f9f59d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9f5c0ac640 0x7f9f5c063580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42870/0 (socket says 172.21.15.7:42870) 2024-01-27T02:32:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.164+0000 7f9f59d9b700 1 -- 172.21.15.7:0/1403028247 learned_addr learned my addr 172.21.15.7:0/1403028247 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.164+0000 7f9f5ad9d700 1 -- 172.21.15.7:0/1403028247 >> v1:172.21.15.188:6789/0 conn(0x7f9f5c159fb0 legacy=0x7f9f5c06b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.164+0000 7f9f5ad9d700 1 -- 172.21.15.7:0/1403028247 >> v1:172.21.15.7:6789/0 conn(0x7f9f5c0ac640 legacy=0x7f9f5c063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.164+0000 7f9f5ad9d700 1 -- 172.21.15.7:0/1403028247 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5c0acd20 con 0x7f9f5c063c90 2024-01-27T02:32:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.165+0000 7f9f5ad9d700 1 --2- 172.21.15.7:0/1403028247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f5c063c90 0x7f9f5c164cb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9f54009e60 tx=0x7f9f54009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d30feb9c2b75ee server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.165+0000 7f9f5959a700 1 -- 172.21.15.7:0/1403028247 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5400e070 con 0x7f9f5c063c90 2024-01-27T02:32:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.165+0000 7f9f5959a700 1 -- 172.21.15.7:0/1403028247 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5400bc30 con 0x7f9f5c063c90 2024-01-27T02:32:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.165+0000 7f9f5959a700 1 -- 172.21.15.7:0/1403028247 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54012430 con 0x7f9f5c063c90 2024-01-27T02:32:43.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.165+0000 7f9f60996700 1 -- 172.21.15.7:0/1403028247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f5c063c90 msgr2=0x7f9f5c164cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:43.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.165+0000 7f9f60996700 1 --2- 172.21.15.7:0/1403028247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f5c063c90 0x7f9f5c164cb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9f54009e60 tx=0x7f9f54009780 comp rx=0 tx=0).stop 2024-01-27T02:32:43.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.165+0000 7f9f60996700 1 -- 172.21.15.7:0/1403028247 shutdown_connections 2024-01-27T02:32:43.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.165+0000 7f9f60996700 1 --2- 172.21.15.7:0/1403028247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f5c063c90 0x7f9f5c164cb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.166+0000 7f9f60996700 1 -- 172.21.15.7:0/1403028247 >> 172.21.15.7:0/1403028247 conn(0x7f9f5c068430 msgr2=0x7f9f5c06be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:43.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.166+0000 7f9f60996700 1 -- 172.21.15.7:0/1403028247 shutdown_connections 2024-01-27T02:32:43.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.166+0000 7f9f60996700 1 -- 172.21.15.7:0/1403028247 wait complete. 2024-01-27T02:32:43.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.167+0000 7f9f60996700 1 Processor -- start 2024-01-27T02:32:43.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.167+0000 7f9f60996700 1 -- start start 2024-01-27T02:32:43.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.168+0000 7f9f60996700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f5c063c90 0x7f9f5c0afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:43.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.168+0000 7f9f60996700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f5c0ac640 0x7f9f5c0ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:43.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.168+0000 7f9f60996700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f5c159fb0 0x7f9f5c0aea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.168+0000 7f9f60996700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c15a380 con 0x7f9f5c0ac640 2024-01-27T02:32:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.168+0000 7f9f60996700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c15a5d0 con 0x7f9f5c063c90 2024-01-27T02:32:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.168+0000 7f9f60996700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c15a8b0 con 0x7f9f5c159fb0 2024-01-27T02:32:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.168+0000 7f9f59d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f5c0ac640 0x7f9f5c0ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.168+0000 7f9f59d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f5c0ac640 0x7f9f5c0ae4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50288/0 (socket says 172.21.15.7:50288) 2024-01-27T02:32:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.168+0000 7f9f59d9b700 1 -- 172.21.15.7:0/1814926590 learned_addr learned my addr 172.21.15.7:0/1814926590 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.168+0000 7f9f5a59c700 1 --2- 172.21.15.7:0/1814926590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f5c063c90 0x7f9f5c0afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.168+0000 7f9f5ad9d700 1 --2- 172.21.15.7:0/1814926590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f5c159fb0 0x7f9f5c0aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.168+0000 7f9f59d9b700 1 -- 172.21.15.7:0/1814926590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f5c159fb0 msgr2=0x7f9f5c0aea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.168+0000 7f9f59d9b700 1 --2- 172.21.15.7:0/1814926590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f5c159fb0 0x7f9f5c0aea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.168+0000 7f9f59d9b700 1 -- 172.21.15.7:0/1814926590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f5c063c90 msgr2=0x7f9f5c0afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.168+0000 7f9f59d9b700 1 --2- 172.21.15.7:0/1814926590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f5c063c90 0x7f9f5c0afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.168+0000 7f9f59d9b700 1 -- 172.21.15.7:0/1814926590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f54009a20 con 0x7f9f5c0ac640 2024-01-27T02:32:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.168+0000 7f9f5a59c700 1 --2- 172.21.15.7:0/1814926590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f5c063c90 0x7f9f5c0afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:43.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.168+0000 7f9f59d9b700 1 --2- 172.21.15.7:0/1814926590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f5c0ac640 0x7f9f5c0ae4f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00c7c0 tx=0x7f9f4c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:43.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.168+0000 7f9f4a7fc700 1 -- 172.21.15.7:0/1814926590 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c005b30 con 0x7f9f5c0ac640 2024-01-27T02:32:43.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.168+0000 7f9f4a7fc700 1 -- 172.21.15.7:0/1814926590 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4c005c90 con 0x7f9f5c0ac640 2024-01-27T02:32:43.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.168+0000 7f9f60996700 1 -- 172.21.15.7:0/1814926590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5c15ab90 con 0x7f9f5c0ac640 2024-01-27T02:32:43.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.168+0000 7f9f4a7fc700 1 -- 172.21.15.7:0/1814926590 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c004a80 con 0x7f9f5c0ac640 2024-01-27T02:32:43.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.169+0000 7f9f60996700 1 -- 172.21.15.7:0/1814926590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5c15b150 con 0x7f9f5c0ac640 2024-01-27T02:32:43.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.169+0000 7f9f60996700 1 -- 172.21.15.7:0/1814926590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f5c06ca30 con 0x7f9f5c0ac640 2024-01-27T02:32:43.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.170+0000 7f9f4a7fc700 1 -- 172.21.15.7:0/1814926590 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9f4c0070d0 con 0x7f9f5c0ac640 2024-01-27T02:32:43.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.170+0000 7f9f4a7fc700 1 --2- 172.21.15.7:0/1814926590 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9f3c066f30 0x7f9f3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:43.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.171+0000 7f9f4a7fc700 1 -- 172.21.15.7:0/1814926590 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9f4c083900 con 0x7f9f5c0ac640 2024-01-27T02:32:43.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.171+0000 7f9f5a59c700 1 --2- 172.21.15.7:0/1814926590 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9f3c066f30 0x7f9f3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:43.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.171+0000 7f9f5a59c700 1 --2- 172.21.15.7:0/1814926590 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9f3c066f30 0x7f9f3c0693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9f50000c00 tx=0x7f9f50009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:43.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.175+0000 7f9f4a7fc700 1 -- 172.21.15.7:0/1814926590 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f4c04b5a0 con 0x7f9f5c0ac640 2024-01-27T02:32:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.427+0000 7f9f60996700 1 -- 172.21.15.7:0/1814926590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f5c15b370 con 0x7f9f5c0ac640 2024-01-27T02:32:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.427+0000 7f9f4a7fc700 1 -- 172.21.15.7:0/1814926590 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f9f4c050b70 con 0x7f9f5c0ac640 2024-01-27T02:32:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:43.428 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3594367777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3594367777},{"type":"v1","addr":"172.21.15.7:6834","nonce":3594367777}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1079101213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1079101213},{"type":"v1","addr":"172.21.15.188:6837","nonce":1079101213}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3241424244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3241424244},{"type":"v1","addr":"172.21.15.7:6835","nonce":3241424244}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:32:06.045693+0000","modified":"2024-01-27T02:32:09.806398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4273":{"gid":4273,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6836/2571237935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2571237935},{"type":"v1","addr":"172.21.15.188:6836","nonce":2571237935}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-27T02:32:43.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.429+0000 7f9f60996700 1 -- 172.21.15.7:0/1814926590 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9f3c066f30 msgr2=0x7f9f3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:43.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.429+0000 7f9f60996700 1 --2- 172.21.15.7:0/1814926590 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9f3c066f30 0x7f9f3c0693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9f50000c00 tx=0x7f9f50009040 comp rx=0 tx=0).stop 2024-01-27T02:32:43.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.429+0000 7f9f60996700 1 -- 172.21.15.7:0/1814926590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f5c0ac640 msgr2=0x7f9f5c0ae4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:43.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.429+0000 7f9f60996700 1 --2- 172.21.15.7:0/1814926590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f5c0ac640 0x7f9f5c0ae4f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00c7c0 tx=0x7f9f4c007f70 comp rx=0 tx=0).stop 2024-01-27T02:32:43.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.429+0000 7f9f60996700 1 -- 172.21.15.7:0/1814926590 shutdown_connections 2024-01-27T02:32:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.429+0000 7f9f60996700 1 --2- 172.21.15.7:0/1814926590 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9f3c066f30 0x7f9f3c0693e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.429+0000 7f9f60996700 1 --2- 172.21.15.7:0/1814926590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f5c159fb0 0x7f9f5c0aea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.429+0000 7f9f60996700 1 --2- 172.21.15.7:0/1814926590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f5c063c90 0x7f9f5c0afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.429+0000 7f9f60996700 1 --2- 172.21.15.7:0/1814926590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f5c0ac640 0x7f9f5c0ae4f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.429+0000 7f9f60996700 1 -- 172.21.15.7:0/1814926590 >> 172.21.15.7:0/1814926590 conn(0x7f9f5c068430 msgr2=0x7f9f5c156a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.430+0000 7f9f60996700 1 -- 172.21.15.7:0/1814926590 shutdown_connections 2024-01-27T02:32:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.430+0000 7f9f60996700 1 -- 172.21.15.7:0/1814926590 wait complete. 2024-01-27T02:32:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-01-27T02:32:43.443 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:32:43.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.575+0000 7fbd0400b700 1 Processor -- start 2024-01-27T02:32:43.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.575+0000 7fbd0400b700 1 -- start start 2024-01-27T02:32:43.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.576+0000 7fbd0400b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbcfc157ca0 0x7fbcfc0c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:43.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.576+0000 7fbd0400b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbcfc0c3ae0 0x7fbcfc0cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:43.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.576+0000 7fbd0400b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcfc0ca020 con 0x7fbcfc158070 2024-01-27T02:32:43.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.576+0000 7fbd0400b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcfc0c4f40 con 0x7fbcfc157ca0 2024-01-27T02:32:43.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.576+0000 7fbd0400b700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcfc0c5210 con 0x7fbcfc0c3ae0 2024-01-27T02:32:43.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.576+0000 7fbd01da7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbcfc157ca0 0x7fbcfc0c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:43.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.576+0000 7fbd01da7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbcfc157ca0 0x7fbcfc0c4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50302/0 (socket says 172.21.15.7:50302) 2024-01-27T02:32:43.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.576+0000 7fbd025a8700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbcfc158070 0x7fbcfc15bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42880/0 (socket says 172.21.15.7:42880) 2024-01-27T02:32:43.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.576+0000 7fbd01da7700 1 -- 172.21.15.7:0/2722510790 learned_addr learned my addr 172.21.15.7:0/2722510790 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:43.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.577+0000 7fbd015a6700 1 --2- 172.21.15.7:0/2722510790 >> v2:172.21.15.188:3300/0 conn(0x7fbcfc0c3ae0 0x7fbcfc0cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:43.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.577+0000 7fbd01da7700 1 -- 172.21.15.7:0/2722510790 >> v2:172.21.15.188:3300/0 conn(0x7fbcfc0c3ae0 msgr2=0x7fbcfc0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:43.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.577+0000 7fbd01da7700 1 --2- 172.21.15.7:0/2722510790 >> v2:172.21.15.188:3300/0 conn(0x7fbcfc0c3ae0 0x7fbcfc0cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.577+0000 7fbd01da7700 1 -- 172.21.15.7:0/2722510790 >> v1:172.21.15.7:6789/0 conn(0x7fbcfc158070 legacy=0x7fbcfc15bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:43.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.577+0000 7fbd01da7700 1 -- 172.21.15.7:0/2722510790 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcfc0c54f0 con 0x7fbcfc157ca0 2024-01-27T02:32:43.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.577+0000 7fbd00da5700 1 -- 172.21.15.7:0/2722510790 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1252585747 0 0) 0x7fbcfc0ca020 con 0x7fbcfc158070 2024-01-27T02:32:43.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.577+0000 7fbd01da7700 1 --2- 172.21.15.7:0/2722510790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcfc157ca0 0x7fbcfc0c4a00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc06e170 tx=0x7fbcf8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4924a798e65d0eda server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:43.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.577+0000 7fbd00da5700 1 -- 172.21.15.7:0/2722510790 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8009e70 con 0x7fbcfc157ca0 2024-01-27T02:32:43.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.577+0000 7fbd00da5700 1 -- 172.21.15.7:0/2722510790 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf800b860 con 0x7fbcfc157ca0 2024-01-27T02:32:43.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.577+0000 7fbd00da5700 1 -- 172.21.15.7:0/2722510790 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf800bc00 con 0x7fbcfc157ca0 2024-01-27T02:32:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.578+0000 7fbd0400b700 1 -- 172.21.15.7:0/2722510790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcfc157ca0 msgr2=0x7fbcfc0c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.578+0000 7fbd0400b700 1 --2- 172.21.15.7:0/2722510790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcfc157ca0 0x7fbcfc0c4a00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc06e170 tx=0x7fbcf8009250 comp rx=0 tx=0).stop 2024-01-27T02:32:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.578+0000 7fbd0400b700 1 -- 172.21.15.7:0/2722510790 shutdown_connections 2024-01-27T02:32:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.578+0000 7fbd0400b700 1 --2- 172.21.15.7:0/2722510790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcfc157ca0 0x7fbcfc0c4a00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.578+0000 7fbd0400b700 1 --2- 172.21.15.7:0/2722510790 >> v2:172.21.15.188:3300/0 conn(0x7fbcfc0c3ae0 0x7fbcfc0cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.578+0000 7fbd0400b700 1 -- 172.21.15.7:0/2722510790 >> 172.21.15.7:0/2722510790 conn(0x7fbcfc098020 msgr2=0x7fbcfc0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.578+0000 7fbd0400b700 1 -- 172.21.15.7:0/2722510790 shutdown_connections 2024-01-27T02:32:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.578+0000 7fbd0400b700 1 -- 172.21.15.7:0/2722510790 wait complete. 2024-01-27T02:32:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.579+0000 7fbd0400b700 1 Processor -- start 2024-01-27T02:32:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.579+0000 7fbd0400b700 1 -- start start 2024-01-27T02:32:43.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.580+0000 7fbd0400b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcfc0c3ae0 0x7fbcfc14dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:43.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.580+0000 7fbd0400b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcfc157ca0 0x7fbcfc068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:43.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.580+0000 7fbd0400b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcfc158070 0x7fbcfc068f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:43.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.580+0000 7fbd0400b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc14e140 con 0x7fbcfc157ca0 2024-01-27T02:32:43.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.580+0000 7fbd0400b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc0b3dd0 con 0x7fbcfc0c3ae0 2024-01-27T02:32:43.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.580+0000 7fbd0400b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc0b40b0 con 0x7fbcfc158070 2024-01-27T02:32:43.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.580+0000 7fbd015a6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcfc157ca0 0x7fbcfc068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:43.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.580+0000 7fbd015a6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcfc157ca0 0x7fbcfc068a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50314/0 (socket says 172.21.15.7:50314) 2024-01-27T02:32:43.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.580+0000 7fbd015a6700 1 -- 172.21.15.7:0/790633030 learned_addr learned my addr 172.21.15.7:0/790633030 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:43.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.580+0000 7fbd01da7700 1 --2- 172.21.15.7:0/790633030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcfc0c3ae0 0x7fbcfc14dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:43.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.580+0000 7fbd025a8700 1 --2- 172.21.15.7:0/790633030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcfc158070 0x7fbcfc068f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:43.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.581+0000 7fbd015a6700 1 -- 172.21.15.7:0/790633030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcfc158070 msgr2=0x7fbcfc068f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:43.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.581+0000 7fbd015a6700 1 --2- 172.21.15.7:0/790633030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcfc158070 0x7fbcfc068f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.581+0000 7fbd015a6700 1 -- 172.21.15.7:0/790633030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcfc0c3ae0 msgr2=0x7fbcfc14dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:43.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.581+0000 7fbd015a6700 1 --2- 172.21.15.7:0/790633030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcfc0c3ae0 0x7fbcfc14dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.581+0000 7fbd015a6700 1 -- 172.21.15.7:0/790633030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf80094f0 con 0x7fbcfc157ca0 2024-01-27T02:32:43.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.581+0000 7fbd025a8700 1 --2- 172.21.15.7:0/790633030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcfc158070 0x7fbcfc068f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:32:43.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.581+0000 7fbd015a6700 1 --2- 172.21.15.7:0/790633030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcfc157ca0 0x7fbcfc068a30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00ec80 tx=0x7fbcec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:43.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.581+0000 7fbcf1ffb700 1 -- 172.21.15.7:0/790633030 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec00c9d0 con 0x7fbcfc157ca0 2024-01-27T02:32:43.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.581+0000 7fbcf1ffb700 1 -- 172.21.15.7:0/790633030 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcec00cb30 con 0x7fbcfc157ca0 2024-01-27T02:32:43.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.581+0000 7fbcf1ffb700 1 -- 172.21.15.7:0/790633030 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec013470 con 0x7fbcfc157ca0 2024-01-27T02:32:43.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.581+0000 7fbd0400b700 1 -- 172.21.15.7:0/790633030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcfc0b4390 con 0x7fbcfc157ca0 2024-01-27T02:32:43.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.581+0000 7fbd0400b700 1 -- 172.21.15.7:0/790633030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcfc0b3050 con 0x7fbcfc157ca0 2024-01-27T02:32:43.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.582+0000 7fbd0400b700 1 -- 172.21.15.7:0/790633030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcfc0afc50 con 0x7fbcfc157ca0 2024-01-27T02:32:43.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.583+0000 7fbcf1ffb700 1 -- 172.21.15.7:0/790633030 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbcec013820 con 0x7fbcfc157ca0 2024-01-27T02:32:43.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.583+0000 7fbcf1ffb700 1 --2- 172.21.15.7:0/790633030 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbce0066f30 0x7fbce00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:43.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.583+0000 7fbcf1ffb700 1 -- 172.21.15.7:0/790633030 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbcec0848d0 con 0x7fbcfc157ca0 2024-01-27T02:32:43.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.584+0000 7fbd01da7700 1 --2- 172.21.15.7:0/790633030 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbce0066f30 0x7fbce00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:43.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.584+0000 7fbd01da7700 1 --2- 172.21.15.7:0/790633030 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbce0066f30 0x7fbce00693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc06e170 tx=0x7fbcf8017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:43.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.588+0000 7fbcf1ffb700 1 -- 172.21.15.7:0/790633030 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbcfc0afc50 con 0x7fbcfc157ca0 2024-01-27T02:32:43.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.792+0000 7fbd0400b700 1 -- 172.21.15.7:0/790633030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcfc0afc50 con 0x7fbcfc157ca0 2024-01-27T02:32:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.793+0000 7fbcf1ffb700 1 -- 172.21.15.7:0/790633030 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15302 (secure 0 0 0) 0x7fbcfc0afc50 con 0x7fbcfc157ca0 2024-01-27T02:32:43.794 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:43.794 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:32:07.627822+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:32:03.675611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-27T02:32:05.018885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:32:43.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.795+0000 7fbd0400b700 1 -- 172.21.15.7:0/790633030 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbce0066f30 msgr2=0x7fbce00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:43.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.795+0000 7fbd0400b700 1 --2- 172.21.15.7:0/790633030 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbce0066f30 0x7fbce00693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc06e170 tx=0x7fbcf8017000 comp rx=0 tx=0).stop 2024-01-27T02:32:43.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.795+0000 7fbd0400b700 1 -- 172.21.15.7:0/790633030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcfc157ca0 msgr2=0x7fbcfc068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:43.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.795+0000 7fbd0400b700 1 --2- 172.21.15.7:0/790633030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcfc157ca0 0x7fbcfc068a30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00ec80 tx=0x7fbcec0091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.795+0000 7fbd0400b700 1 -- 172.21.15.7:0/790633030 shutdown_connections 2024-01-27T02:32:43.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.795+0000 7fbd0400b700 1 --2- 172.21.15.7:0/790633030 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbce0066f30 0x7fbce00693e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.795+0000 7fbd0400b700 1 --2- 172.21.15.7:0/790633030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcfc158070 0x7fbcfc068f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.795+0000 7fbd0400b700 1 --2- 172.21.15.7:0/790633030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcfc0c3ae0 0x7fbcfc14dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.795+0000 7fbd0400b700 1 --2- 172.21.15.7:0/790633030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcfc157ca0 0x7fbcfc068a30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.795+0000 7fbd0400b700 1 -- 172.21.15.7:0/790633030 >> 172.21.15.7:0/790633030 conn(0x7fbcfc098020 msgr2=0x7fbcfc0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:43.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.796+0000 7fbd0400b700 1 -- 172.21.15.7:0/790633030 shutdown_connections 2024-01-27T02:32:43.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.796+0000 7fbd0400b700 1 -- 172.21.15.7:0/790633030 wait complete. 2024-01-27T02:32:43.810 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-27T02:32:43.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.942+0000 7f3845297700 1 Processor -- start 2024-01-27T02:32:43.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.942+0000 7f3845297700 1 -- start start 2024-01-27T02:32:43.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.943+0000 7f3845297700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f38400b2f80 0x7f384006a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:43.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.943+0000 7f3845297700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38400b3350 0x7f38400ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:43.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.943+0000 7f3845297700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f38400ce810 0x7f38401553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.943+0000 7f3845297700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f384006a900 con 0x7f38400b2f80 2024-01-27T02:32:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.943+0000 7f3845297700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3840159390 con 0x7f38400ce810 2024-01-27T02:32:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.943+0000 7f3845297700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3840158780 con 0x7f38400b3350 2024-01-27T02:32:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.943+0000 7f383effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f38400b2f80 0x7f384006a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.944+0000 7f383effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f38400b2f80 0x7f384006a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50320/0 (socket says 172.21.15.7:50320) 2024-01-27T02:32:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.944+0000 7f383effd700 1 -- 172.21.15.7:0/3450802982 learned_addr learned my addr 172.21.15.7:0/3450802982 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.944+0000 7f383e7fc700 1 --2- 172.21.15.7:0/3450802982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38400b3350 0x7f38400ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.944+0000 7f383f7fe700 1 --2- 172.21.15.7:0/3450802982 >> v2:172.21.15.188:3300/0 conn(0x7f38400ce810 0x7f38401553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.944+0000 7f383effd700 1 -- 172.21.15.7:0/3450802982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38400b3350 msgr2=0x7f38400ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.944+0000 7f383effd700 1 --2- 172.21.15.7:0/3450802982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38400b3350 0x7f38400ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.944+0000 7f383effd700 1 -- 172.21.15.7:0/3450802982 >> v2:172.21.15.188:3300/0 conn(0x7f38400ce810 msgr2=0x7f38401553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.944+0000 7f383effd700 1 --2- 172.21.15.7:0/3450802982 >> v2:172.21.15.188:3300/0 conn(0x7f38400ce810 0x7f38401553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.944+0000 7f383effd700 1 -- 172.21.15.7:0/3450802982 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3840158a60 con 0x7f38400b2f80 2024-01-27T02:32:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.944+0000 7f383effd700 1 --2- 172.21.15.7:0/3450802982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38400b2f80 0x7f384006a3c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f38280099b0 tx=0x7f3828009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0fc4448dff1deb server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.944+0000 7f383dffb700 1 -- 172.21.15.7:0/3450802982 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382800b640 con 0x7f38400b2f80 2024-01-27T02:32:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.944+0000 7f383dffb700 1 -- 172.21.15.7:0/3450802982 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382800b7a0 con 0x7f38400b2f80 2024-01-27T02:32:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.944+0000 7f383dffb700 1 -- 172.21.15.7:0/3450802982 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382800bb40 con 0x7f38400b2f80 2024-01-27T02:32:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.945+0000 7f3845297700 1 -- 172.21.15.7:0/3450802982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38400b2f80 msgr2=0x7f384006a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.945+0000 7f3845297700 1 --2- 172.21.15.7:0/3450802982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38400b2f80 0x7f384006a3c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f38280099b0 tx=0x7f3828009250 comp rx=0 tx=0).stop 2024-01-27T02:32:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.945+0000 7f3845297700 1 -- 172.21.15.7:0/3450802982 shutdown_connections 2024-01-27T02:32:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.945+0000 7f3845297700 1 --2- 172.21.15.7:0/3450802982 >> v2:172.21.15.188:3300/0 conn(0x7f38400ce810 0x7f38401553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.945+0000 7f3845297700 1 --2- 172.21.15.7:0/3450802982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38400b2f80 0x7f384006a3c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.945+0000 7f3845297700 1 --2- 172.21.15.7:0/3450802982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38400b3350 0x7f38400ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.945+0000 7f3845297700 1 -- 172.21.15.7:0/3450802982 >> 172.21.15.7:0/3450802982 conn(0x7f3840068430 msgr2=0x7f384006be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.945+0000 7f3845297700 1 -- 172.21.15.7:0/3450802982 shutdown_connections 2024-01-27T02:32:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.945+0000 7f3845297700 1 -- 172.21.15.7:0/3450802982 wait complete. 2024-01-27T02:32:43.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.946+0000 7f3845297700 1 Processor -- start 2024-01-27T02:32:43.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.946+0000 7f3845297700 1 -- start start 2024-01-27T02:32:43.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.947+0000 7f3845297700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38400b2f80 0x7f38400b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:43.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.947+0000 7f3845297700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38400b3350 0x7f384014d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:43.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.947+0000 7f3845297700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38400ce810 0x7f38400c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:43.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.947+0000 7f3845297700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38400b1a10 con 0x7f38400b3350 2024-01-27T02:32:43.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.947+0000 7f3845297700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384015c1d0 con 0x7f38400ce810 2024-01-27T02:32:43.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.947+0000 7f3845297700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38400b1ca0 con 0x7f38400b2f80 2024-01-27T02:32:43.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.947+0000 7f383e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38400b3350 0x7f384014d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:43.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.947+0000 7f383e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38400b3350 0x7f384014d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50332/0 (socket says 172.21.15.7:50332) 2024-01-27T02:32:43.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.947+0000 7f383e7fc700 1 -- 172.21.15.7:0/3675657956 learned_addr learned my addr 172.21.15.7:0/3675657956 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:43.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.948+0000 7f383effd700 1 --2- 172.21.15.7:0/3675657956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38400b2f80 0x7f38400b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:43.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.948+0000 7f383f7fe700 1 --2- 172.21.15.7:0/3675657956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38400ce810 0x7f38400c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:43.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.948+0000 7f383e7fc700 1 -- 172.21.15.7:0/3675657956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38400b2f80 msgr2=0x7f38400b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.948+0000 7f383e7fc700 1 --2- 172.21.15.7:0/3675657956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38400b2f80 0x7f38400b14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.948+0000 7f383e7fc700 1 -- 172.21.15.7:0/3675657956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38400ce810 msgr2=0x7f38400c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.948+0000 7f383e7fc700 1 --2- 172.21.15.7:0/3675657956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38400ce810 0x7f38400c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.948+0000 7f383e7fc700 1 -- 172.21.15.7:0/3675657956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38280094f0 con 0x7f38400b3350 2024-01-27T02:32:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.948+0000 7f383e7fc700 1 --2- 172.21.15.7:0/3675657956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38400b3350 0x7f384014d6d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f383000ec80 tx=0x7f38300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.948+0000 7f3836ffd700 1 -- 172.21.15.7:0/3675657956 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383000c8e0 con 0x7f38400b3350 2024-01-27T02:32:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.948+0000 7f3836ffd700 1 -- 172.21.15.7:0/3675657956 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383000ca60 con 0x7f38400b3350 2024-01-27T02:32:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.948+0000 7f3845297700 1 -- 172.21.15.7:0/3675657956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38400b1f80 con 0x7f38400b3350 2024-01-27T02:32:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.948+0000 7f3836ffd700 1 -- 172.21.15.7:0/3675657956 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383000ce20 con 0x7f38400b3350 2024-01-27T02:32:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.948+0000 7f3845297700 1 -- 172.21.15.7:0/3675657956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384015c470 con 0x7f38400b3350 2024-01-27T02:32:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.949+0000 7f3845297700 1 -- 172.21.15.7:0/3675657956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384015c690 con 0x7f38400b3350 2024-01-27T02:32:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.950+0000 7f3836ffd700 1 -- 172.21.15.7:0/3675657956 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3830013680 con 0x7f38400b3350 2024-01-27T02:32:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.950+0000 7f3836ffd700 1 --2- 172.21.15.7:0/3675657956 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3820066f30 0x7f38200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.950+0000 7f3836ffd700 1 -- 172.21.15.7:0/3675657956 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3830084810 con 0x7f38400b3350 2024-01-27T02:32:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.950+0000 7f383effd700 1 --2- 172.21.15.7:0/3675657956 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3820066f30 0x7f38200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.951+0000 7f383effd700 1 --2- 172.21.15.7:0/3675657956 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3820066f30 0x7f38200693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f38280099b0 tx=0x7f3828017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:43.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:43.955+0000 7f3836ffd700 1 -- 172.21.15.7:0/3675657956 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3830051a80 con 0x7f38400b3350 2024-01-27T02:32:44.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.210+0000 7f3845297700 1 -- 172.21.15.7:0/3675657956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f384004c710 con 0x7f38400b3350 2024-01-27T02:32:44.211 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-27T02:32:44.211+0000 7fb5c503a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-27T02:32:44.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.214+0000 7f3836ffd700 1 -- 172.21.15.7:0/3675657956 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f38300550a0 con 0x7f38400b3350 2024-01-27T02:32:44.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.217+0000 7f3845297700 1 -- 172.21.15.7:0/3675657956 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3820066f30 msgr2=0x7f38200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:44.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.217+0000 7f3845297700 1 --2- 172.21.15.7:0/3675657956 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3820066f30 0x7f38200693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f38280099b0 tx=0x7f3828017000 comp rx=0 tx=0).stop 2024-01-27T02:32:44.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.217+0000 7f3845297700 1 -- 172.21.15.7:0/3675657956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38400b3350 msgr2=0x7f384014d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:44.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.217+0000 7f3845297700 1 --2- 172.21.15.7:0/3675657956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38400b3350 0x7f384014d6d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f383000ec80 tx=0x7f38300091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:44.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.217+0000 7f3845297700 1 -- 172.21.15.7:0/3675657956 shutdown_connections 2024-01-27T02:32:44.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.217+0000 7f3845297700 1 --2- 172.21.15.7:0/3675657956 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3820066f30 0x7f38200693e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:44.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.217+0000 7f3845297700 1 --2- 172.21.15.7:0/3675657956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38400ce810 0x7f38400c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:44.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.217+0000 7f3845297700 1 --2- 172.21.15.7:0/3675657956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38400b2f80 0x7f38400b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:44.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.217+0000 7f3845297700 1 --2- 172.21.15.7:0/3675657956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38400b3350 0x7f384014d6d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:44.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.217+0000 7f3845297700 1 -- 172.21.15.7:0/3675657956 >> 172.21.15.7:0/3675657956 conn(0x7f3840068430 msgr2=0x7f38401540b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:44.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.217+0000 7f3845297700 1 -- 172.21.15.7:0/3675657956 shutdown_connections 2024-01-27T02:32:44.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.217+0000 7f3845297700 1 -- 172.21.15.7:0/3675657956 wait complete. 2024-01-27T02:32:44.220 INFO:tasks.ceph.mds.d.smithi188.stderr: -1807> 2024-01-27T02:32:12.154+0000 7ff55429a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-27T02:32:44.220 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-27T02:32:44.232 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-27T02:32:44.241 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-01-27T02:32:44.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.365+0000 7fc08ec9e700 1 Processor -- start 2024-01-27T02:32:44.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.366+0000 7fc08ec9e700 1 -- start start 2024-01-27T02:32:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.366+0000 7fc08ec9e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0880cba60 0x7fc0880ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.366+0000 7fc08ec9e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc0880cbe30 0x7fc08815b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.366+0000 7fc08ec9e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc08806b4e0 0x7fc08806b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.366+0000 7fc08ec9e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc08806bea0 con 0x7fc08806b4e0 2024-01-27T02:32:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.366+0000 7fc08ec9e700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0880679b0 con 0x7fc0880cbe30 2024-01-27T02:32:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.366+0000 7fc08ec9e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc088067c90 con 0x7fc0880cba60 2024-01-27T02:32:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.367+0000 7fc08d23b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc08806b4e0 0x7fc08806b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.367+0000 7fc08d23b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc08806b4e0 0x7fc08806b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50348/0 (socket says 172.21.15.7:50348) 2024-01-27T02:32:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.367+0000 7fc08d23b700 1 -- 172.21.15.7:0/3891813898 learned_addr learned my addr 172.21.15.7:0/3891813898 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.367+0000 7fc08ca3a700 1 --2- 172.21.15.7:0/3891813898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0880cba60 0x7fc0880ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.367+0000 7fc087fff700 1 --2- 172.21.15.7:0/3891813898 >> v2:172.21.15.188:3300/0 conn(0x7fc0880cbe30 0x7fc08815b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.367+0000 7fc08d23b700 1 -- 172.21.15.7:0/3891813898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0880cba60 msgr2=0x7fc0880ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.367+0000 7fc08d23b700 1 --2- 172.21.15.7:0/3891813898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0880cba60 0x7fc0880ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.367+0000 7fc08d23b700 1 -- 172.21.15.7:0/3891813898 >> v2:172.21.15.188:3300/0 conn(0x7fc0880cbe30 msgr2=0x7fc08815b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.367+0000 7fc08d23b700 1 --2- 172.21.15.7:0/3891813898 >> v2:172.21.15.188:3300/0 conn(0x7fc0880cbe30 0x7fc08815b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.367+0000 7fc08d23b700 1 -- 172.21.15.7:0/3891813898 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08815b7b0 con 0x7fc08806b4e0 2024-01-27T02:32:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.367+0000 7fc087fff700 1 --2- 172.21.15.7:0/3891813898 >> v2:172.21.15.188:3300/0 conn(0x7fc0880cbe30 0x7fc08815b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.367+0000 7fc08d23b700 1 --2- 172.21.15.7:0/3891813898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc08806b4e0 0x7fc08806b960 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc0880cfdb0 tx=0x7fc080009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf9fe80cd830772 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.367+0000 7fc0877fe700 1 -- 172.21.15.7:0/3891813898 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08000e070 con 0x7fc08806b4e0 2024-01-27T02:32:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.367+0000 7fc0877fe700 1 -- 172.21.15.7:0/3891813898 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08000ba50 con 0x7fc08806b4e0 2024-01-27T02:32:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.367+0000 7fc0877fe700 1 -- 172.21.15.7:0/3891813898 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0800092e0 con 0x7fc08806b4e0 2024-01-27T02:32:44.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.368+0000 7fc08ec9e700 1 -- 172.21.15.7:0/3891813898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc08806b4e0 msgr2=0x7fc08806b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:44.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.368+0000 7fc08ec9e700 1 --2- 172.21.15.7:0/3891813898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc08806b4e0 0x7fc08806b960 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc0880cfdb0 tx=0x7fc080009840 comp rx=0 tx=0).stop 2024-01-27T02:32:44.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.368+0000 7fc08ec9e700 1 -- 172.21.15.7:0/3891813898 shutdown_connections 2024-01-27T02:32:44.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.368+0000 7fc08ec9e700 1 --2- 172.21.15.7:0/3891813898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc08806b4e0 0x7fc08806b960 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:44.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.368+0000 7fc08ec9e700 1 --2- 172.21.15.7:0/3891813898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0880cba60 0x7fc0880ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:44.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.368+0000 7fc08ec9e700 1 --2- 172.21.15.7:0/3891813898 >> v2:172.21.15.188:3300/0 conn(0x7fc0880cbe30 0x7fc08815b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:44.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.368+0000 7fc08ec9e700 1 -- 172.21.15.7:0/3891813898 >> 172.21.15.7:0/3891813898 conn(0x7fc088063580 msgr2=0x7fc088063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:44.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.368+0000 7fc08ec9e700 1 -- 172.21.15.7:0/3891813898 shutdown_connections 2024-01-27T02:32:44.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.368+0000 7fc08ec9e700 1 -- 172.21.15.7:0/3891813898 wait complete. 2024-01-27T02:32:44.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.369+0000 7fc08ec9e700 1 Processor -- start 2024-01-27T02:32:44.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.370+0000 7fc08ec9e700 1 -- start start 2024-01-27T02:32:44.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.370+0000 7fc08ec9e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc08806b4e0 0x7fc08814b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:44.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.370+0000 7fc08ec9e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0880cba60 0x7fc08815c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:44.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.370+0000 7fc08ec9e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0880cbe30 0x7fc08815cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:44.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.370+0000 7fc08ec9e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08815df50 con 0x7fc0880cbe30 2024-01-27T02:32:44.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.370+0000 7fc08ec9e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08815e230 con 0x7fc0880cba60 2024-01-27T02:32:44.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.370+0000 7fc08ec9e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08815e510 con 0x7fc08806b4e0 2024-01-27T02:32:44.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.370+0000 7fc08d23b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0880cbe30 0x7fc08815cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:44.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.370+0000 7fc08d23b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0880cbe30 0x7fc08815cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50364/0 (socket says 172.21.15.7:50364) 2024-01-27T02:32:44.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.370+0000 7fc08d23b700 1 -- 172.21.15.7:0/3927053487 learned_addr learned my addr 172.21.15.7:0/3927053487 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:44.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.370+0000 7fc08ca3a700 1 --2- 172.21.15.7:0/3927053487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc08806b4e0 0x7fc08814b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:44.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.370+0000 7fc087fff700 1 --2- 172.21.15.7:0/3927053487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0880cba60 0x7fc08815c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:44.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.371+0000 7fc08d23b700 1 -- 172.21.15.7:0/3927053487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc08806b4e0 msgr2=0x7fc08814b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:44.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.371+0000 7fc08d23b700 1 --2- 172.21.15.7:0/3927053487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc08806b4e0 0x7fc08814b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:44.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.371+0000 7fc08d23b700 1 -- 172.21.15.7:0/3927053487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0880cba60 msgr2=0x7fc08815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:44.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.371+0000 7fc08d23b700 1 --2- 172.21.15.7:0/3927053487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0880cba60 0x7fc08815c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:44.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.371+0000 7fc08d23b700 1 -- 172.21.15.7:0/3927053487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc080009ae0 con 0x7fc0880cbe30 2024-01-27T02:32:44.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.371+0000 7fc08d23b700 1 --2- 172.21.15.7:0/3927053487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0880cbe30 0x7fc08815cee0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc08806e580 tx=0x7fc080018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:44.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.371+0000 7fc084ff9700 1 -- 172.21.15.7:0/3927053487 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0800032f0 con 0x7fc0880cbe30 2024-01-27T02:32:44.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.371+0000 7fc084ff9700 1 -- 172.21.15.7:0/3927053487 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc080008ce0 con 0x7fc0880cbe30 2024-01-27T02:32:44.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.371+0000 7fc08ec9e700 1 -- 172.21.15.7:0/3927053487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08815e7f0 con 0x7fc0880cbe30 2024-01-27T02:32:44.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.371+0000 7fc084ff9700 1 -- 172.21.15.7:0/3927053487 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0800256b0 con 0x7fc0880cbe30 2024-01-27T02:32:44.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.371+0000 7fc08ec9e700 1 -- 172.21.15.7:0/3927053487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08814bda0 con 0x7fc0880cbe30 2024-01-27T02:32:44.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.372+0000 7fc08ec9e700 1 -- 172.21.15.7:0/3927053487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc08815ed10 con 0x7fc0880cbe30 2024-01-27T02:32:44.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.373+0000 7fc084ff9700 1 -- 172.21.15.7:0/3927053487 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc080003040 con 0x7fc0880cbe30 2024-01-27T02:32:44.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.373+0000 7fc084ff9700 1 --2- 172.21.15.7:0/3927053487 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc06c066f30 0x7fc06c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:44.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.373+0000 7fc084ff9700 1 -- 172.21.15.7:0/3927053487 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fc080086100 con 0x7fc0880cbe30 2024-01-27T02:32:44.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.373+0000 7fc08ca3a700 1 --2- 172.21.15.7:0/3927053487 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc06c066f30 0x7fc06c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:44.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.374+0000 7fc08ca3a700 1 --2- 172.21.15.7:0/3927053487 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc06c066f30 0x7fc06c0693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc07c006aa0 tx=0x7fc07c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:44.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.378+0000 7fc084ff9700 1 -- 172.21.15.7:0/3927053487 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc080053320 con 0x7fc0880cbe30 2024-01-27T02:32:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:44.633+0000 7fc08ec9e700 1 -- 172.21.15.7:0/3927053487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc08815eb20 con 0x7fc0880cbe30 2024-01-27T02:32:45.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.215+0000 7fc084ff9700 1 -- 172.21.15.7:0/3927053487 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc080056940 con 0x7fc0880cbe30 2024-01-27T02:32:45.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.219+0000 7fc08ec9e700 1 -- 172.21.15.7:0/3927053487 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc06c066f30 msgr2=0x7fc06c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.219+0000 7fc08ec9e700 1 --2- 172.21.15.7:0/3927053487 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc06c066f30 0x7fc06c0693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc07c006aa0 tx=0x7fc07c008040 comp rx=0 tx=0).stop 2024-01-27T02:32:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.219+0000 7fc08ec9e700 1 -- 172.21.15.7:0/3927053487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0880cbe30 msgr2=0x7fc08815cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.219+0000 7fc08ec9e700 1 --2- 172.21.15.7:0/3927053487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0880cbe30 0x7fc08815cee0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc08806e580 tx=0x7fc080018000 comp rx=0 tx=0).stop 2024-01-27T02:32:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.219+0000 7fc08ec9e700 1 -- 172.21.15.7:0/3927053487 shutdown_connections 2024-01-27T02:32:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.219+0000 7fc08ec9e700 1 --2- 172.21.15.7:0/3927053487 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc06c066f30 0x7fc06c0693e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.219+0000 7fc08ec9e700 1 --2- 172.21.15.7:0/3927053487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0880cbe30 0x7fc08815cee0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.219+0000 7fc08ec9e700 1 --2- 172.21.15.7:0/3927053487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc08806b4e0 0x7fc08814b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.219+0000 7fc08ec9e700 1 --2- 172.21.15.7:0/3927053487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0880cba60 0x7fc08815c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.219+0000 7fc08ec9e700 1 -- 172.21.15.7:0/3927053487 >> 172.21.15.7:0/3927053487 conn(0x7fc088063580 msgr2=0x7fc0881565b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.219+0000 7fc08ec9e700 1 -- 172.21.15.7:0/3927053487 shutdown_connections 2024-01-27T02:32:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.219+0000 7fc08ec9e700 1 -- 172.21.15.7:0/3927053487 wait complete. 2024-01-27T02:32:45.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-27T02:32:45.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.374+0000 7f180b307700 1 Processor -- start 2024-01-27T02:32:45.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.375+0000 7f180b307700 1 -- start start 2024-01-27T02:32:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.375+0000 7f180b307700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f18040a7fe0 0x7f180406f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.375+0000 7f180b307700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18040afcf0 con 0x7f180415dfd0 2024-01-27T02:32:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.375+0000 7f180b307700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f180415a880 con 0x7f18040a7c10 2024-01-27T02:32:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.375+0000 7f180b307700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18040aea50 con 0x7f18040a7fe0 2024-01-27T02:32:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.376+0000 7f18098a4700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f180415dfd0 0x7f18040b0180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42896/0 (socket says 172.21.15.7:42896) 2024-01-27T02:32:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.376+0000 7f18098a4700 1 -- 172.21.15.7:0/2234053027 learned_addr learned my addr 172.21.15.7:0/2234053027 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.376+0000 7f18088a2700 1 --2- 172.21.15.7:0/2234053027 >> v2:172.21.15.188:3300/0 conn(0x7f18040a7fe0 0x7f180406f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.376+0000 7f17f7fff700 1 -- 172.21.15.7:0/2234053027 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3961624076 0 0) 0x7f18040afcf0 con 0x7f180415dfd0 2024-01-27T02:32:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.376+0000 7f17f7fff700 1 -- 172.21.15.7:0/2234053027 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17f0003570 con 0x7f180415dfd0 2024-01-27T02:32:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.376+0000 7f17f7fff700 1 -- 172.21.15.7:0/2234053027 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2315888452 0 0) 0x7f180415a880 con 0x7f18040a7c10 2024-01-27T02:32:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.376+0000 7f17f7fff700 1 -- 172.21.15.7:0/2234053027 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18040afcf0 con 0x7f18040a7c10 2024-01-27T02:32:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.376+0000 7f17f7fff700 1 -- 172.21.15.7:0/2234053027 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3697694325 0 0) 0x7f17f0003570 con 0x7f180415dfd0 2024-01-27T02:32:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.376+0000 7f17f7fff700 1 -- 172.21.15.7:0/2234053027 >> v2:172.21.15.188:3300/0 conn(0x7f18040a7fe0 msgr2=0x7f180406f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.376+0000 7f17f7fff700 1 --2- 172.21.15.7:0/2234053027 >> v2:172.21.15.188:3300/0 conn(0x7f18040a7fe0 0x7f180406f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.376+0000 7f17f7fff700 1 -- 172.21.15.7:0/2234053027 >> v1:172.21.15.188:6789/0 conn(0x7f18040a7c10 legacy=0x7f18040ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.376+0000 7f17f7fff700 1 -- 172.21.15.7:0/2234053027 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18040aec50 con 0x7f180415dfd0 2024-01-27T02:32:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.376+0000 7f17f7fff700 1 -- 172.21.15.7:0/2234053027 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f17fc0017f0 con 0x7f180415dfd0 2024-01-27T02:32:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.376+0000 7f17f7fff700 1 -- 172.21.15.7:0/2234053027 >> v1:172.21.15.7:6789/0 conn(0x7f180415dfd0 legacy=0x7f18040b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.376+0000 7f17f7fff700 1 --2- 172.21.15.7:0/2234053027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17f0003bf0 0x7f17f0003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.376+0000 7f17f7fff700 1 --2- 172.21.15.7:0/2234053027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f0004810 0x7f17f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.376+0000 7f17f7fff700 1 --2- 172.21.15.7:0/2234053027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f00051c0 0x7f17f000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.376+0000 7f17f7fff700 1 -- 172.21.15.7:0/2234053027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18040aea50 con 0x7f17f00051c0 2024-01-27T02:32:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.376+0000 7f17f7fff700 1 -- 172.21.15.7:0/2234053027 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f000c0b0 con 0x7f17f0003bf0 2024-01-27T02:32:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.376+0000 7f17f7fff700 1 -- 172.21.15.7:0/2234053027 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f000c270 con 0x7f17f0004810 2024-01-27T02:32:45.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.377+0000 7f18098a4700 1 --2- 172.21.15.7:0/2234053027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f00051c0 0x7f17f000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:45.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.377+0000 7f18090a3700 1 --2- 172.21.15.7:0/2234053027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17f0003bf0 0x7f17f0003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:45.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.377+0000 7f18088a2700 1 --2- 172.21.15.7:0/2234053027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f0004810 0x7f17f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:45.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.377+0000 7f18098a4700 1 -- 172.21.15.7:0/2234053027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f0004810 msgr2=0x7f17f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:45.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.377+0000 7f18098a4700 1 --2- 172.21.15.7:0/2234053027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f0004810 0x7f17f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:45.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.377+0000 7f18098a4700 1 -- 172.21.15.7:0/2234053027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17f0003bf0 msgr2=0x7f17f0003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:45.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.377+0000 7f18098a4700 1 --2- 172.21.15.7:0/2234053027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17f0003bf0 0x7f17f0003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:45.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.377+0000 7f18098a4700 1 -- 172.21.15.7:0/2234053027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f0003570 con 0x7f17f00051c0 2024-01-27T02:32:45.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.377+0000 7f18098a4700 1 -- 172.21.15.7:0/2234053027 reap_dead start 2024-01-27T02:32:45.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.377+0000 7f18098a4700 1 --2- 172.21.15.7:0/2234053027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f00051c0 0x7f17f000ba60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00a8d0 tx=0x7f17fc002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da56123e283ab204 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:45.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.377+0000 7f17f7fff700 1 -- 172.21.15.7:0/2234053027 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc0070d0 con 0x7f17f00051c0 2024-01-27T02:32:45.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.377+0000 7f17f7fff700 1 -- 172.21.15.7:0/2234053027 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc007270 con 0x7f17f00051c0 2024-01-27T02:32:45.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.377+0000 7f17f7fff700 1 -- 172.21.15.7:0/2234053027 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc007630 con 0x7f17f00051c0 2024-01-27T02:32:45.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.378+0000 7f180b307700 1 -- 172.21.15.7:0/2234053027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f00051c0 msgr2=0x7f17f000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.378+0000 7f180b307700 1 --2- 172.21.15.7:0/2234053027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f00051c0 0x7f17f000ba60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00a8d0 tx=0x7f17fc002190 comp rx=0 tx=0).stop 2024-01-27T02:32:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.378+0000 7f180b307700 1 -- 172.21.15.7:0/2234053027 shutdown_connections 2024-01-27T02:32:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.378+0000 7f180b307700 1 --2- 172.21.15.7:0/2234053027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f00051c0 0x7f17f000ba60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.378+0000 7f180b307700 1 -- 172.21.15.7:0/2234053027 >> 172.21.15.7:0/2234053027 conn(0x7f18040679b0 msgr2=0x7f1804067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.378+0000 7f180b307700 1 -- 172.21.15.7:0/2234053027 shutdown_connections 2024-01-27T02:32:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.378+0000 7f180b307700 1 -- 172.21.15.7:0/2234053027 wait complete. 2024-01-27T02:32:45.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.379+0000 7f180b307700 1 Processor -- start 2024-01-27T02:32:45.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.379+0000 7f180b307700 1 -- start start 2024-01-27T02:32:45.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.380+0000 7f180b307700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17f00051c0 0x7f18040a6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:45.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.380+0000 7f180b307700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f180415bb20 0x7f180415bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.380+0000 7f180b307700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1804097a40 0x7f1804097ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.380+0000 7f180b307700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180409b460 con 0x7f1804097a40 2024-01-27T02:32:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.380+0000 7f180b307700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18040992f0 con 0x7f17f00051c0 2024-01-27T02:32:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.380+0000 7f180b307700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180415c4d0 con 0x7f180415bb20 2024-01-27T02:32:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.380+0000 7f18098a4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1804097a40 0x7f1804097ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.380+0000 7f18098a4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1804097a40 0x7f1804097ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50372/0 (socket says 172.21.15.7:50372) 2024-01-27T02:32:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.380+0000 7f18098a4700 1 -- 172.21.15.7:0/4247480017 learned_addr learned my addr 172.21.15.7:0/4247480017 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.380+0000 7f18090a3700 1 --2- 172.21.15.7:0/4247480017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17f00051c0 0x7f18040a6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.380+0000 7f18088a2700 1 --2- 172.21.15.7:0/4247480017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f180415bb20 0x7f180415bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.381+0000 7f18098a4700 1 -- 172.21.15.7:0/4247480017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f180415bb20 msgr2=0x7f180415bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.381+0000 7f18098a4700 1 --2- 172.21.15.7:0/4247480017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f180415bb20 0x7f180415bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.381+0000 7f18098a4700 1 -- 172.21.15.7:0/4247480017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17f00051c0 msgr2=0x7f18040a6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.381+0000 7f18098a4700 1 --2- 172.21.15.7:0/4247480017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17f00051c0 0x7f18040a6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.381+0000 7f18098a4700 1 -- 172.21.15.7:0/4247480017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1804098420 con 0x7f1804097a40 2024-01-27T02:32:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.381+0000 7f18098a4700 1 --2- 172.21.15.7:0/4247480017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1804097a40 0x7f1804097ee0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f180406d670 tx=0x7f17fc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.381+0000 7f17f57fa700 1 -- 172.21.15.7:0/4247480017 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc004dc0 con 0x7f1804097a40 2024-01-27T02:32:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.381+0000 7f17f57fa700 1 -- 172.21.15.7:0/4247480017 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc01d420 con 0x7f1804097a40 2024-01-27T02:32:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.381+0000 7f180b307700 1 -- 172.21.15.7:0/4247480017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18040986e0 con 0x7f1804097a40 2024-01-27T02:32:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.381+0000 7f17f57fa700 1 -- 172.21.15.7:0/4247480017 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc01d7e0 con 0x7f1804097a40 2024-01-27T02:32:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.381+0000 7f180b307700 1 -- 172.21.15.7:0/4247480017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1804098ca0 con 0x7f1804097a40 2024-01-27T02:32:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.382+0000 7f180b307700 1 -- 172.21.15.7:0/4247480017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180404c710 con 0x7f1804097a40 2024-01-27T02:32:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.383+0000 7f17f57fa700 1 -- 172.21.15.7:0/4247480017 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f17fc01db90 con 0x7f1804097a40 2024-01-27T02:32:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.383+0000 7f17f57fa700 1 --2- 172.21.15.7:0/4247480017 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f17e8066f30 0x7f17e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.383+0000 7f17f57fa700 1 -- 172.21.15.7:0/4247480017 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f17fc085e50 con 0x7f1804097a40 2024-01-27T02:32:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.383+0000 7f18090a3700 1 --2- 172.21.15.7:0/4247480017 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f17e8066f30 0x7f17e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.384+0000 7f18090a3700 1 --2- 172.21.15.7:0/4247480017 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f17e8066f30 0x7f17e80693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1800000ef0 tx=0x7f1800007d80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:45.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.388+0000 7f17f57fa700 1 -- 172.21.15.7:0/4247480017 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17fc053070 con 0x7f1804097a40 2024-01-27T02:32:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:45.593+0000 7f180b307700 1 -- 172.21.15.7:0/4247480017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f18040aef30 con 0x7f1804097a40 2024-01-27T02:32:46.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.219+0000 7f17f57fa700 1 -- 172.21.15.7:0/4247480017 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) v1 ==== 181+0+0 (secure 0 0 0) 0x7f17fc056690 con 0x7f1804097a40 2024-01-27T02:32:46.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.222+0000 7f180b307700 1 -- 172.21.15.7:0/4247480017 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f17e8066f30 msgr2=0x7f17e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:46.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.222+0000 7f180b307700 1 --2- 172.21.15.7:0/4247480017 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f17e8066f30 0x7f17e80693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1800000ef0 tx=0x7f1800007d80 comp rx=0 tx=0).stop 2024-01-27T02:32:46.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.222+0000 7f180b307700 1 -- 172.21.15.7:0/4247480017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1804097a40 msgr2=0x7f1804097ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:46.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.222+0000 7f180b307700 1 --2- 172.21.15.7:0/4247480017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1804097a40 0x7f1804097ee0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f180406d670 tx=0x7f17fc016000 comp rx=0 tx=0).stop 2024-01-27T02:32:46.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.222+0000 7f180b307700 1 -- 172.21.15.7:0/4247480017 shutdown_connections 2024-01-27T02:32:46.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.222+0000 7f180b307700 1 --2- 172.21.15.7:0/4247480017 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f17e8066f30 0x7f17e80693e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:46.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.222+0000 7f180b307700 1 --2- 172.21.15.7:0/4247480017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1804097a40 0x7f1804097ee0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:46.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.222+0000 7f180b307700 1 --2- 172.21.15.7:0/4247480017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17f00051c0 0x7f18040a6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:46.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.222+0000 7f180b307700 1 --2- 172.21.15.7:0/4247480017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f180415bb20 0x7f180415bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:46.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.222+0000 7f180b307700 1 -- 172.21.15.7:0/4247480017 >> 172.21.15.7:0/4247480017 conn(0x7f18040679b0 msgr2=0x7f18041578c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:46.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.223+0000 7f180b307700 1 -- 172.21.15.7:0/4247480017 shutdown_connections 2024-01-27T02:32:46.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.223+0000 7f180b307700 1 -- 172.21.15.7:0/4247480017 wait complete. 2024-01-27T02:32:46.225 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-27T02:32:46.238 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-27T02:32:46.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.382+0000 7f425dcd9700 1 Processor -- start 2024-01-27T02:32:46.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.382+0000 7f425dcd9700 1 -- start start 2024-01-27T02:32:46.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.383+0000 7f425dcd9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f425815a450 0x7f42580b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:46.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.383+0000 7f425dcd9700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f42580cbf40 0x7f425815b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:46.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.383+0000 7f425dcd9700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42580cb6e0 con 0x7f42580cd8e0 2024-01-27T02:32:46.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.383+0000 7f425dcd9700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f425815b980 con 0x7f42580cbf40 2024-01-27T02:32:46.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.383+0000 7f425dcd9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42580cc390 con 0x7f425815a450 2024-01-27T02:32:46.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.383+0000 7f4256ffd700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f42580cbf40 0x7f425815b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:46.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.383+0000 7f42577fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f425815a450 0x7f42580b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:46.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.383+0000 7f4257fff700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f42580cd8e0 0x7f42580ceae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:38716/0 (socket says 172.21.15.7:38716) 2024-01-27T02:32:46.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.383+0000 7f4257fff700 1 -- 172.21.15.7:0/2548494599 learned_addr learned my addr 172.21.15.7:0/2548494599 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:46.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.384+0000 7f42577fe700 1 -- 172.21.15.7:0/2548494599 >> v2:172.21.15.188:3300/0 conn(0x7f42580cbf40 msgr2=0x7f425815b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.384+0000 7f42577fe700 1 --2- 172.21.15.7:0/2548494599 >> v2:172.21.15.188:3300/0 conn(0x7f42580cbf40 0x7f425815b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.384+0000 7f42577fe700 1 -- 172.21.15.7:0/2548494599 >> v1:172.21.15.188:6789/0 conn(0x7f42580cd8e0 legacy=0x7f42580ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.384+0000 7f42577fe700 1 -- 172.21.15.7:0/2548494599 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425815afb0 con 0x7f425815a450 2024-01-27T02:32:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.384+0000 7f42567fc700 1 -- 172.21.15.7:0/2548494599 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 924178757 0 0) 0x7f4250001490 con 0x7f42580cd8e0 2024-01-27T02:32:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.384+0000 7f4256ffd700 1 --2- 172.21.15.7:0/2548494599 >> v2:172.21.15.188:3300/0 conn(0x7f42580cbf40 0x7f425815b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:32:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.384+0000 7f42577fe700 1 --2- 172.21.15.7:0/2548494599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f425815a450 0x7f42580b04b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f42480098e0 tx=0x7f42480091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f05d2e30a995f435 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.384+0000 7f42567fc700 1 -- 172.21.15.7:0/2548494599 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424800b640 con 0x7f425815a450 2024-01-27T02:32:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.384+0000 7f42567fc700 1 -- 172.21.15.7:0/2548494599 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424800b7a0 con 0x7f425815a450 2024-01-27T02:32:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.384+0000 7f42567fc700 1 -- 172.21.15.7:0/2548494599 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424800bb40 con 0x7f425815a450 2024-01-27T02:32:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.385+0000 7f425dcd9700 1 -- 172.21.15.7:0/2548494599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f425815a450 msgr2=0x7f42580b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.385+0000 7f425dcd9700 1 --2- 172.21.15.7:0/2548494599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f425815a450 0x7f42580b04b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f42480098e0 tx=0x7f42480091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.385+0000 7f425dcd9700 1 -- 172.21.15.7:0/2548494599 shutdown_connections 2024-01-27T02:32:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.385+0000 7f425dcd9700 1 --2- 172.21.15.7:0/2548494599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f425815a450 0x7f42580b04b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.385+0000 7f425dcd9700 1 --2- 172.21.15.7:0/2548494599 >> v2:172.21.15.188:3300/0 conn(0x7f42580cbf40 0x7f425815b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.385+0000 7f425dcd9700 1 -- 172.21.15.7:0/2548494599 >> 172.21.15.7:0/2548494599 conn(0x7f4258068440 msgr2=0x7f4258063580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:46.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.385+0000 7f425dcd9700 1 -- 172.21.15.7:0/2548494599 shutdown_connections 2024-01-27T02:32:46.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.385+0000 7f425dcd9700 1 -- 172.21.15.7:0/2548494599 wait complete. 2024-01-27T02:32:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.386+0000 7f425dcd9700 1 Processor -- start 2024-01-27T02:32:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.386+0000 7f425dcd9700 1 -- start start 2024-01-27T02:32:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.387+0000 7f425dcd9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42580cbf40 0x7f4258156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.387+0000 7f425dcd9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42580cd8e0 0x7f42580b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.387+0000 7f425dcd9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f425815a450 0x7f4258158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.387+0000 7f425dcd9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4258156a90 con 0x7f42580cbf40 2024-01-27T02:32:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.387+0000 7f425dcd9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4258156570 con 0x7f42580cd8e0 2024-01-27T02:32:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.387+0000 7f425dcd9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42580b6460 con 0x7f425815a450 2024-01-27T02:32:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.387+0000 7f42577fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42580cbf40 0x7f4258156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.387+0000 7f42577fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42580cbf40 0x7f4258156030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50374/0 (socket says 172.21.15.7:50374) 2024-01-27T02:32:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.387+0000 7f42577fe700 1 -- 172.21.15.7:0/1218215336 learned_addr learned my addr 172.21.15.7:0/1218215336 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.387+0000 7f4257fff700 1 --2- 172.21.15.7:0/1218215336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f425815a450 0x7f4258158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.387+0000 7f4256ffd700 1 --2- 172.21.15.7:0/1218215336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42580cd8e0 0x7f42580b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.387+0000 7f42577fe700 1 -- 172.21.15.7:0/1218215336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f425815a450 msgr2=0x7f4258158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.387+0000 7f42577fe700 1 --2- 172.21.15.7:0/1218215336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f425815a450 0x7f4258158d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.387+0000 7f42577fe700 1 -- 172.21.15.7:0/1218215336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42580cd8e0 msgr2=0x7f42580b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.387+0000 7f42577fe700 1 --2- 172.21.15.7:0/1218215336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42580cd8e0 0x7f42580b49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.387+0000 7f42577fe700 1 -- 172.21.15.7:0/1218215336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f424c009420 con 0x7f42580cbf40 2024-01-27T02:32:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.388+0000 7f42577fe700 1 --2- 172.21.15.7:0/1218215336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42580cbf40 0x7f4258156030 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4248009f80 tx=0x7f4248017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.388+0000 7f423f7fe700 1 -- 172.21.15.7:0/1218215336 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424800b910 con 0x7f42580cbf40 2024-01-27T02:32:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.388+0000 7f423f7fe700 1 -- 172.21.15.7:0/1218215336 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424800ba70 con 0x7f42580cbf40 2024-01-27T02:32:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.388+0000 7f425dcd9700 1 -- 172.21.15.7:0/1218215336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4248009420 con 0x7f42580cbf40 2024-01-27T02:32:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.388+0000 7f423f7fe700 1 -- 172.21.15.7:0/1218215336 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248011670 con 0x7f42580cbf40 2024-01-27T02:32:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.388+0000 7f425dcd9700 1 -- 172.21.15.7:0/1218215336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4258157790 con 0x7f42580cbf40 2024-01-27T02:32:46.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.389+0000 7f425dcd9700 1 -- 172.21.15.7:0/1218215336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f425806dfd0 con 0x7f42580cbf40 2024-01-27T02:32:46.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.389+0000 7f423f7fe700 1 -- 172.21.15.7:0/1218215336 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4248011a60 con 0x7f42580cbf40 2024-01-27T02:32:46.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.390+0000 7f423f7fe700 1 --2- 172.21.15.7:0/1218215336 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4244066f30 0x7f42440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:46.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.390+0000 7f423f7fe700 1 -- 172.21.15.7:0/1218215336 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f4248085af0 con 0x7f42580cbf40 2024-01-27T02:32:46.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.390+0000 7f4256ffd700 1 --2- 172.21.15.7:0/1218215336 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4244066f30 0x7f42440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:46.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.391+0000 7f4256ffd700 1 --2- 172.21.15.7:0/1218215336 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4244066f30 0x7f42440693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4258156e70 tx=0x7f424c0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.394+0000 7f423f7fe700 1 -- 172.21.15.7:0/1218215336 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4248052ec0 con 0x7f42580cbf40 2024-01-27T02:32:46.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:46.600+0000 7f425dcd9700 1 -- 172.21.15.7:0/1218215336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f425804c710 con 0x7f42580cbf40 2024-01-27T02:32:47.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.221+0000 7f423f7fe700 1 -- 172.21.15.7:0/1218215336 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f42480150d0 con 0x7f42580cbf40 2024-01-27T02:32:47.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.225+0000 7f425dcd9700 1 -- 172.21.15.7:0/1218215336 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4244066f30 msgr2=0x7f42440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:47.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.225+0000 7f425dcd9700 1 --2- 172.21.15.7:0/1218215336 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4244066f30 0x7f42440693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4258156e70 tx=0x7f424c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:47.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.225+0000 7f425dcd9700 1 -- 172.21.15.7:0/1218215336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42580cbf40 msgr2=0x7f4258156030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:47.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.225+0000 7f425dcd9700 1 --2- 172.21.15.7:0/1218215336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42580cbf40 0x7f4258156030 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4248009f80 tx=0x7f4248017000 comp rx=0 tx=0).stop 2024-01-27T02:32:47.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.225+0000 7f425dcd9700 1 -- 172.21.15.7:0/1218215336 shutdown_connections 2024-01-27T02:32:47.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.225+0000 7f425dcd9700 1 --2- 172.21.15.7:0/1218215336 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4244066f30 0x7f42440693e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:47.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.225+0000 7f425dcd9700 1 --2- 172.21.15.7:0/1218215336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f425815a450 0x7f4258158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:47.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.225+0000 7f425dcd9700 1 --2- 172.21.15.7:0/1218215336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42580cbf40 0x7f4258156030 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:47.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.225+0000 7f425dcd9700 1 --2- 172.21.15.7:0/1218215336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42580cd8e0 0x7f42580b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:47.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.225+0000 7f425dcd9700 1 -- 172.21.15.7:0/1218215336 >> 172.21.15.7:0/1218215336 conn(0x7f4258068440 msgr2=0x7f42580c7640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:47.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.226+0000 7f425dcd9700 1 -- 172.21.15.7:0/1218215336 shutdown_connections 2024-01-27T02:32:47.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.226+0000 7f425dcd9700 1 -- 172.21.15.7:0/1218215336 wait complete. 2024-01-27T02:32:47.227 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-27T02:32:47.242 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-27T02:32:47.242 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-27T02:32:47.242 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-27T02:32:47.242 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:32:47.242 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-27T02:32:47.242 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-27T02:32:47.243 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-27T02:32:47.243 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:32:47.243 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-27T02:32:47.243 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-27T02:32:47.243 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-27T02:32:47.243 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:32:47.243 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-27T02:32:47.244 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-27T02:32:47.244 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-27T02:32:47.244 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:32:47.252 INFO:tasks.ceph.mds.a:Stopped 2024-01-27T02:32:47.252 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-27T02:32:47.253 INFO:tasks.ceph.mds.c:Stopped 2024-01-27T02:32:47.253 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-27T02:32:47.254 INFO:tasks.ceph.mds.b:Stopped 2024-01-27T02:32:47.255 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-27T02:32:47.255 INFO:tasks.ceph.mds.d:Stopped 2024-01-27T02:32:47.256 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-27T02:32:47.295 INFO:tasks.ceph.mds.a:Started 2024-01-27T02:32:47.295 DEBUG:teuthology.parallel:result is None 2024-01-27T02:32:47.296 INFO:tasks.ceph.mds.c:Started 2024-01-27T02:32:47.297 DEBUG:teuthology.parallel:result is None 2024-01-27T02:32:47.298 INFO:tasks.ceph.mds.b:Started 2024-01-27T02:32:47.298 DEBUG:teuthology.parallel:result is None 2024-01-27T02:32:47.298 INFO:tasks.ceph.mds.d:Started 2024-01-27T02:32:47.298 DEBUG:teuthology.parallel:result is None 2024-01-27T02:32:47.299 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-27T02:32:47.380 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-01-27T02:32:47.380 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-01-27T02:32:47.386 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-27T02:32:47.386 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-27T02:32:47.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.440+0000 7f473564c700 1 Processor -- start 2024-01-27T02:32:47.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.440+0000 7f473564c700 1 -- start start 2024-01-27T02:32:47.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.441+0000 7f473564c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4730156c00 0x7f4730156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:47.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.441+0000 7f473564c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f473006dbb0 con 0x7f473006e040 2024-01-27T02:32:47.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.441+0000 7f473564c700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47300c88e0 con 0x7f47300c9a00 2024-01-27T02:32:47.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.441+0000 7f473564c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47300c8bc0 con 0x7f4730156c00 2024-01-27T02:32:47.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.441+0000 7f472e7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f473006e040 0x7f47300c92f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42922/0 (socket says 172.21.15.7:42922) 2024-01-27T02:32:47.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.441+0000 7f472e7fc700 1 -- 172.21.15.7:0/1220709353 learned_addr learned my addr 172.21.15.7:0/1220709353 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:47.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.441+0000 7f472effd700 1 --2- 172.21.15.7:0/1220709353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4730156c00 0x7f4730156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:47.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.442+0000 7f472dffb700 1 -- 172.21.15.7:0/1220709353 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1462861526 0 0) 0x7f473006dbb0 con 0x7f473006e040 2024-01-27T02:32:47.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.442+0000 7f472dffb700 1 -- 172.21.15.7:0/1220709353 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f471c003570 con 0x7f473006e040 2024-01-27T02:32:47.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.442+0000 7f472dffb700 1 -- 172.21.15.7:0/1220709353 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3032331094 0 0) 0x7f47300c88e0 con 0x7f47300c9a00 2024-01-27T02:32:47.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.442+0000 7f472dffb700 1 -- 172.21.15.7:0/1220709353 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f473006dbb0 con 0x7f47300c9a00 2024-01-27T02:32:47.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.442+0000 7f472effd700 1 -- 172.21.15.7:0/1220709353 >> v1:172.21.15.188:6789/0 conn(0x7f47300c9a00 legacy=0x7f47300c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:47.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.442+0000 7f472effd700 1 -- 172.21.15.7:0/1220709353 >> v1:172.21.15.7:6789/0 conn(0x7f473006e040 legacy=0x7f47300c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:47.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.442+0000 7f472effd700 1 -- 172.21.15.7:0/1220709353 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473006d560 con 0x7f4730156c00 2024-01-27T02:32:47.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.442+0000 7f472effd700 1 --2- 172.21.15.7:0/1220709353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4730156c00 0x7f4730156fd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4720009910 tx=0x7f47200091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=699c471aff911361 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:47.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.442+0000 7f472dffb700 1 -- 172.21.15.7:0/1220709353 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720011630 con 0x7f4730156c00 2024-01-27T02:32:47.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.442+0000 7f472dffb700 1 -- 172.21.15.7:0/1220709353 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4720011790 con 0x7f4730156c00 2024-01-27T02:32:47.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.442+0000 7f472dffb700 1 -- 172.21.15.7:0/1220709353 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720011b30 con 0x7f4730156c00 2024-01-27T02:32:47.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.443+0000 7f473564c700 1 -- 172.21.15.7:0/1220709353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4730156c00 msgr2=0x7f4730156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:47.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.443+0000 7f473564c700 1 --2- 172.21.15.7:0/1220709353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4730156c00 0x7f4730156fd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4720009910 tx=0x7f47200091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:47.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.443+0000 7f473564c700 1 -- 172.21.15.7:0/1220709353 shutdown_connections 2024-01-27T02:32:47.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.443+0000 7f473564c700 1 --2- 172.21.15.7:0/1220709353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4730156c00 0x7f4730156fd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:47.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.443+0000 7f473564c700 1 -- 172.21.15.7:0/1220709353 >> 172.21.15.7:0/1220709353 conn(0x7f4730063580 msgr2=0x7f4730063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:47.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.443+0000 7f473564c700 1 -- 172.21.15.7:0/1220709353 shutdown_connections 2024-01-27T02:32:47.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.443+0000 7f473564c700 1 -- 172.21.15.7:0/1220709353 wait complete. 2024-01-27T02:32:47.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.444+0000 7f473564c700 1 Processor -- start 2024-01-27T02:32:47.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.444+0000 7f473564c700 1 -- start start 2024-01-27T02:32:47.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.445+0000 7f473564c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f473006e040 0x7f47300b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:47.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.445+0000 7f473564c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47300c9a00 0x7f47300ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:47.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.445+0000 7f473564c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4730156c00 0x7f473014aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:47.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.445+0000 7f473564c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4730159e90 con 0x7f4730156c00 2024-01-27T02:32:47.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.445+0000 7f473564c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47300afae0 con 0x7f473006e040 2024-01-27T02:32:47.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.445+0000 7f473564c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47301595f0 con 0x7f47300c9a00 2024-01-27T02:32:47.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.445+0000 7f472f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4730156c00 0x7f473014aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:47.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.445+0000 7f472f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4730156c00 0x7f473014aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50392/0 (socket says 172.21.15.7:50392) 2024-01-27T02:32:47.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.445+0000 7f472f7fe700 1 -- 172.21.15.7:0/3582855295 learned_addr learned my addr 172.21.15.7:0/3582855295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:47.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.445+0000 7f472effd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f473006e040 0x7f47300b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:47.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.445+0000 7f472e7fc700 1 --2- 172.21.15.7:0/3582855295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47300c9a00 0x7f47300ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:47.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.446+0000 7f472f7fe700 1 -- 172.21.15.7:0/3582855295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47300c9a00 msgr2=0x7f47300ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:47.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.446+0000 7f472f7fe700 1 --2- 172.21.15.7:0/3582855295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47300c9a00 0x7f47300ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:47.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.446+0000 7f472f7fe700 1 -- 172.21.15.7:0/3582855295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f473006e040 msgr2=0x7f47300b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:47.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.446+0000 7f472f7fe700 1 --2- 172.21.15.7:0/3582855295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f473006e040 0x7f47300b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:47.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.446+0000 7f472f7fe700 1 -- 172.21.15.7:0/3582855295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4720009420 con 0x7f4730156c00 2024-01-27T02:32:47.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.446+0000 7f472f7fe700 1 --2- 172.21.15.7:0/3582855295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4730156c00 0x7f473014aaf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f472800c910 tx=0x7f47280026f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:47.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.446+0000 7f4726ffd700 1 -- 172.21.15.7:0/3582855295 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4728012630 con 0x7f4730156c00 2024-01-27T02:32:47.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.446+0000 7f4726ffd700 1 -- 172.21.15.7:0/3582855295 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4728012790 con 0x7f4730156c00 2024-01-27T02:32:47.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.446+0000 7f4726ffd700 1 -- 172.21.15.7:0/3582855295 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4728012b50 con 0x7f4730156c00 2024-01-27T02:32:47.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.446+0000 7f473564c700 1 -- 172.21.15.7:0/3582855295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47301598d0 con 0x7f4730156c00 2024-01-27T02:32:47.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.446+0000 7f473564c700 1 -- 172.21.15.7:0/3582855295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47300b2580 con 0x7f4730156c00 2024-01-27T02:32:47.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.447+0000 7f473564c700 1 -- 172.21.15.7:0/3582855295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47300af8b0 con 0x7f4730156c00 2024-01-27T02:32:47.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.448+0000 7f4726ffd700 1 -- 172.21.15.7:0/3582855295 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f472801a3b0 con 0x7f4730156c00 2024-01-27T02:32:47.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.448+0000 7f4726ffd700 1 --2- 172.21.15.7:0/3582855295 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4710066f30 0x7f47100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:47.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.448+0000 7f4726ffd700 1 -- 172.21.15.7:0/3582855295 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4728084260 con 0x7f4730156c00 2024-01-27T02:32:47.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.448+0000 7f472effd700 1 --2- 172.21.15.7:0/3582855295 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4710066f30 0x7f47100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:47.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.449+0000 7f472effd700 1 --2- 172.21.15.7:0/3582855295 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4710066f30 0x7f47100693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f473016d850 tx=0x7f4720017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:47.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.453+0000 7f4726ffd700 1 -- 172.21.15.7:0/3582855295 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f47300af8b0 con 0x7f4730156c00 2024-01-27T02:32:47.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:47.675+0000 7f473564c700 1 -- 172.21.15.7:0/3582855295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f47300af8b0 con 0x7f4730156c00 2024-01-27T02:32:48.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.231+0000 7f4726ffd700 1 -- 172.21.15.7:0/3582855295 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) v1 ==== 97+0+0 (secure 0 0 0) 0x7f47300af8b0 con 0x7f4730156c00 2024-01-27T02:32:48.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.233+0000 7f473564c700 1 -- 172.21.15.7:0/3582855295 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4710066f30 msgr2=0x7f47100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:48.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.233+0000 7f473564c700 1 --2- 172.21.15.7:0/3582855295 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4710066f30 0x7f47100693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f473016d850 tx=0x7f4720017000 comp rx=0 tx=0).stop 2024-01-27T02:32:48.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.233+0000 7f473564c700 1 -- 172.21.15.7:0/3582855295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4730156c00 msgr2=0x7f473014aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:48.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.233+0000 7f473564c700 1 --2- 172.21.15.7:0/3582855295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4730156c00 0x7f473014aaf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f472800c910 tx=0x7f47280026f0 comp rx=0 tx=0).stop 2024-01-27T02:32:48.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.233+0000 7f473564c700 1 -- 172.21.15.7:0/3582855295 shutdown_connections 2024-01-27T02:32:48.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.233+0000 7f473564c700 1 --2- 172.21.15.7:0/3582855295 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4710066f30 0x7f47100693e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:48.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.233+0000 7f473564c700 1 --2- 172.21.15.7:0/3582855295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4730156c00 0x7f473014aaf0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:48.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.233+0000 7f473564c700 1 --2- 172.21.15.7:0/3582855295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f473006e040 0x7f47300b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:48.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.233+0000 7f473564c700 1 --2- 172.21.15.7:0/3582855295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47300c9a00 0x7f47300ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:48.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.233+0000 7f473564c700 1 -- 172.21.15.7:0/3582855295 >> 172.21.15.7:0/3582855295 conn(0x7f4730063580 msgr2=0x7f47300aed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:48.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.234+0000 7f473564c700 1 -- 172.21.15.7:0/3582855295 shutdown_connections 2024-01-27T02:32:48.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.234+0000 7f473564c700 1 -- 172.21.15.7:0/3582855295 wait complete. 2024-01-27T02:32:48.236 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-01-27T02:32:48.253 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-27T02:32:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.389+0000 7f6294eb3700 1 Processor -- start 2024-01-27T02:32:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.389+0000 7f6294eb3700 1 -- start start 2024-01-27T02:32:48.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.390+0000 7f6294eb3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f62900b1900 0x7f62900b0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:48.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.390+0000 7f6294eb3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62900b1470 con 0x7f629015c1e0 2024-01-27T02:32:48.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.390+0000 7f6294eb3700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62900b1d50 con 0x7f629015be10 2024-01-27T02:32:48.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.390+0000 7f6294eb3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62900b0860 con 0x7f62900b1900 2024-01-27T02:32:48.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.390+0000 7f628dd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f62900b1900 0x7f62900b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:48.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.390+0000 7f628ed9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f629015c1e0 0x7f62900c9500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42930/0 (socket says 172.21.15.7:42930) 2024-01-27T02:32:48.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.390+0000 7f628ed9d700 1 -- 172.21.15.7:0/1659192739 learned_addr learned my addr 172.21.15.7:0/1659192739 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:48.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.391+0000 7f628dd9b700 1 -- 172.21.15.7:0/1659192739 >> v1:172.21.15.188:6789/0 conn(0x7f629015be10 legacy=0x7f62900aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:48.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.391+0000 7f628dd9b700 1 -- 172.21.15.7:0/1659192739 >> v1:172.21.15.7:6789/0 conn(0x7f629015c1e0 legacy=0x7f62900c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:48.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.391+0000 7f628dd9b700 1 -- 172.21.15.7:0/1659192739 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62900b0b40 con 0x7f62900b1900 2024-01-27T02:32:48.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.391+0000 7f628d59a700 1 -- 172.21.15.7:0/1659192739 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510930228 0 0) 0x7f62900b1470 con 0x7f629015c1e0 2024-01-27T02:32:48.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.391+0000 7f628dd9b700 1 --2- 172.21.15.7:0/1659192739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62900b1900 0x7f62900b0180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f62780098e0 tx=0x7f62780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6582a13bb045be11 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:48.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.391+0000 7f628d59a700 1 -- 172.21.15.7:0/1659192739 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6278009e80 con 0x7f62900b1900 2024-01-27T02:32:48.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.391+0000 7f628d59a700 1 -- 172.21.15.7:0/1659192739 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627800b860 con 0x7f62900b1900 2024-01-27T02:32:48.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.391+0000 7f628d59a700 1 -- 172.21.15.7:0/1659192739 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627800bbe0 con 0x7f62900b1900 2024-01-27T02:32:48.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.391+0000 7f6294eb3700 1 -- 172.21.15.7:0/1659192739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62900b1900 msgr2=0x7f62900b0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:48.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.391+0000 7f6294eb3700 1 --2- 172.21.15.7:0/1659192739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62900b1900 0x7f62900b0180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f62780098e0 tx=0x7f62780091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:48.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.392+0000 7f6294eb3700 1 -- 172.21.15.7:0/1659192739 shutdown_connections 2024-01-27T02:32:48.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.392+0000 7f6294eb3700 1 --2- 172.21.15.7:0/1659192739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62900b1900 0x7f62900b0180 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:48.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.392+0000 7f6294eb3700 1 -- 172.21.15.7:0/1659192739 >> 172.21.15.7:0/1659192739 conn(0x7f62900679b0 msgr2=0x7f6290067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:48.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.392+0000 7f6294eb3700 1 -- 172.21.15.7:0/1659192739 shutdown_connections 2024-01-27T02:32:48.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.392+0000 7f6294eb3700 1 -- 172.21.15.7:0/1659192739 wait complete. 2024-01-27T02:32:48.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.393+0000 7f6294eb3700 1 Processor -- start 2024-01-27T02:32:48.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.393+0000 7f6294eb3700 1 -- start start 2024-01-27T02:32:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.394+0000 7f6294eb3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62900b1900 0x7f62901598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.394+0000 7f6294eb3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f629015be10 0x7f62901574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.394+0000 7f6294eb3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f629015c1e0 0x7f62901459b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.394+0000 7f6294eb3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629015aa30 con 0x7f629015c1e0 2024-01-27T02:32:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.394+0000 7f6294eb3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6290159eb0 con 0x7f629015be10 2024-01-27T02:32:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.394+0000 7f6294eb3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629015a190 con 0x7f62900b1900 2024-01-27T02:32:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.395+0000 7f628ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f629015c1e0 0x7f62901459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.395+0000 7f628ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f629015c1e0 0x7f62901459b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50396/0 (socket says 172.21.15.7:50396) 2024-01-27T02:32:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.395+0000 7f628ed9d700 1 -- 172.21.15.7:0/650268218 learned_addr learned my addr 172.21.15.7:0/650268218 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.395+0000 7f628dd9b700 1 --2- 172.21.15.7:0/650268218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f629015be10 0x7f62901574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.395+0000 7f628e59c700 1 --2- 172.21.15.7:0/650268218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62900b1900 0x7f62901598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.395+0000 7f628ed9d700 1 -- 172.21.15.7:0/650268218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62900b1900 msgr2=0x7f62901598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.395+0000 7f628ed9d700 1 --2- 172.21.15.7:0/650268218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62900b1900 0x7f62901598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.395+0000 7f628ed9d700 1 -- 172.21.15.7:0/650268218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f629015be10 msgr2=0x7f62901574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.395+0000 7f628ed9d700 1 --2- 172.21.15.7:0/650268218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f629015be10 0x7f62901574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.395+0000 7f628ed9d700 1 -- 172.21.15.7:0/650268218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6278009420 con 0x7f629015c1e0 2024-01-27T02:32:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.395+0000 7f628ed9d700 1 --2- 172.21.15.7:0/650268218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f629015c1e0 0x7f62901459b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f628800c530 tx=0x7f6288007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.395+0000 7f62867fc700 1 -- 172.21.15.7:0/650268218 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288005b30 con 0x7f629015c1e0 2024-01-27T02:32:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.395+0000 7f62867fc700 1 -- 172.21.15.7:0/650268218 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6288005c90 con 0x7f629015c1e0 2024-01-27T02:32:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.395+0000 7f6294eb3700 1 -- 172.21.15.7:0/650268218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629015a470 con 0x7f629015c1e0 2024-01-27T02:32:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.395+0000 7f62867fc700 1 -- 172.21.15.7:0/650268218 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288007320 con 0x7f629015c1e0 2024-01-27T02:32:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.395+0000 7f6294eb3700 1 -- 172.21.15.7:0/650268218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629015b730 con 0x7f629015c1e0 2024-01-27T02:32:48.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.396+0000 7f6294eb3700 1 -- 172.21.15.7:0/650268218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f629004c710 con 0x7f629015c1e0 2024-01-27T02:32:48.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.397+0000 7f62867fc700 1 -- 172.21.15.7:0/650268218 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f628801b3b0 con 0x7f629015c1e0 2024-01-27T02:32:48.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.397+0000 7f62867fc700 1 --2- 172.21.15.7:0/650268218 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6270066f30 0x7f62700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:48.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.397+0000 7f62867fc700 1 -- 172.21.15.7:0/650268218 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6288083700 con 0x7f629015c1e0 2024-01-27T02:32:48.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.397+0000 7f628e59c700 1 --2- 172.21.15.7:0/650268218 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6270066f30 0x7f62700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:48.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.398+0000 7f628e59c700 1 --2- 172.21.15.7:0/650268218 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6270066f30 0x7f62700693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6280007fd0 tx=0x7f6280009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.402+0000 7f62867fc700 1 -- 172.21.15.7:0/650268218 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6288050cc0 con 0x7f629015c1e0 2024-01-27T02:32:48.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.686+0000 7f6294eb3700 1 -- 172.21.15.7:0/650268218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f62900b0e20 con 0x7f629015c1e0 2024-01-27T02:32:48.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.687+0000 7f62867fc700 1 -- 172.21.15.7:0/650268218 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f62880542e0 con 0x7f629015c1e0 2024-01-27T02:32:48.687 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:48.687 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:32:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-01-27T02:32:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-01-27T02:32:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlHdcPKBAAfhnh1I3ZSQEXWJXlXCs98Q==", 2024-01-27T02:32:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:32:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:32:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:32:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:32:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-01-27T02:32:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rl8eVYNRAAfP8rOqO50RpoNZ1+1j2DXQ==", 2024-01-27T02:32:48.688 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:32:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:32:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:32:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:32:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-01-27T02:32:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlJxRKMBAAS0j7/zZeqGglOxCK1pg0WA==", 2024-01-27T02:32:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:32:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:32:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:32:48.689 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:32:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-01-27T02:32:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlNwfUABAALRQAoHoGhniqCvTQOqbbug==", 2024-01-27T02:32:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:32:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:32:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:32:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:32:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-01-27T02:32:48.690 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7Rlh6VSJhAAttuCte1WhNEvUkv5x/IKQQ==", 2024-01-27T02:32:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:32:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:32:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:32:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-01-27T02:32:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7Rlu/RGLhAAKzaCNhzYKhsNhxkMvY++gw==", 2024-01-27T02:32:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:32:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:32:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:32:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-01-27T02:32:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7RliRMyNhAAr0tG3evmVV7ZlGxbxfWPMw==", 2024-01-27T02:32:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:32:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:32:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:32:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-01-27T02:32:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAPa7RlRuSNAhAA3e9atlb+ffvlnLhRYbWZPw==", 2024-01-27T02:32:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:32:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:32:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:32:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-01-27T02:32:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7RlfeA6DBAAXpklgbNHRO+5G+fdIERjRw==", 2024-01-27T02:32:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:32:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:32:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:32:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-01-27T02:32:48.693 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7Rl77ItExAABB7uABm63ubE/YUs2B5pGw==", 2024-01-27T02:32:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:32:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:32:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:32:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-01-27T02:32:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7RljDuwHBAAPPAlhoqS03qPpq58jmevLA==", 2024-01-27T02:32:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:32:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:32:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:32:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-01-27T02:32:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7Rl1YElJhAAqHP/QEdl1SEalGcbQwPwAg==", 2024-01-27T02:32:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:32:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:32:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:32:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-01-27T02:32:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlaEqaBRAATFiIeW8lT9BdNuN0m3x9dQ==", 2024-01-27T02:32:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:32:48.695 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:32:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:32:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:32:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-01-27T02:32:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlNADmEBAAJXwdJ35ylP5coy1dtHoD3g==", 2024-01-27T02:32:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:32:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:32:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:32:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:32:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-01-27T02:32:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7Rl5fRqCRAA/5swTGknnpP64TAh+2X41A==", 2024-01-27T02:32:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:32:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:32:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:32:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:32:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-01-27T02:32:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7Rl0SdwFhAAhCmw9ViiJbHVfS5a2r/D+w==", 2024-01-27T02:32:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:32:48.697 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:32:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:32:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:32:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-01-27T02:32:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlsNgPARAA0Qm0oJgA8GO1oUT3ndT8Tg==", 2024-01-27T02:32:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:32:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:32:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:32:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:32:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-mds", 2024-01-27T02:32:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAWa7RlpssBARAACaVw8RPE0rMQdpVvxWi4zQ==", 2024-01-27T02:32:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-mds" 2024-01-27T02:32:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-mgr", 2024-01-27T02:32:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAWa7RlItgBARAAkypAqMk8N+sxjd6XVBxXVw==", 2024-01-27T02:32:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-mgr" 2024-01-27T02:32:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.699 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-osd", 2024-01-27T02:32:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAWa7RlwOIBARAAHZUps6wJHqbW3BkzeJccHw==", 2024-01-27T02:32:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-osd" 2024-01-27T02:32:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rbd", 2024-01-27T02:32:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAWa7RljesBARAAXWLyq9KRWp75d4PG9nOmFw==", 2024-01-27T02:32:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rbd" 2024-01-27T02:32:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.700 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-01-27T02:32:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAWa7RlZfQBARAAOeCAZULCXM2aAxnEZg9ZbA==", 2024-01-27T02:32:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-01-27T02:32:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rgw", 2024-01-27T02:32:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAWa7RlNP0BARAAggzlKHWjFy/jJfvQoZ5dpw==", 2024-01-27T02:32:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rgw" 2024-01-27T02:32:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-01-27T02:32:48.701 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rlfb7qIxAAlsJYiSolJ7D8EykGWP62bA==", 2024-01-27T02:32:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:32:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-27T02:32:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:32:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:32:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-01-27T02:32:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rls7TaHxAALS3ghxKtfmgQfIELiD9+Bg==", 2024-01-27T02:32:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:32:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:32:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-27T02:32:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:32:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.702 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:48.703 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:32:48.703 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:32:48.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.689+0000 7f6294eb3700 1 -- 172.21.15.7:0/650268218 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6270066f30 msgr2=0x7f62700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:48.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.689+0000 7f6294eb3700 1 --2- 172.21.15.7:0/650268218 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6270066f30 0x7f62700693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6280007fd0 tx=0x7f6280009040 comp rx=0 tx=0).stop 2024-01-27T02:32:48.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.689+0000 7f6294eb3700 1 -- 172.21.15.7:0/650268218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f629015c1e0 msgr2=0x7f62901459b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:48.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.689+0000 7f6294eb3700 1 --2- 172.21.15.7:0/650268218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f629015c1e0 0x7f62901459b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f628800c530 tx=0x7f6288007f90 comp rx=0 tx=0).stop 2024-01-27T02:32:48.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.689+0000 7f6294eb3700 1 -- 172.21.15.7:0/650268218 shutdown_connections 2024-01-27T02:32:48.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.689+0000 7f6294eb3700 1 --2- 172.21.15.7:0/650268218 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6270066f30 0x7f62700693e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:48.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.689+0000 7f6294eb3700 1 --2- 172.21.15.7:0/650268218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f629015c1e0 0x7f62901459b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:48.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.689+0000 7f6294eb3700 1 --2- 172.21.15.7:0/650268218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62900b1900 0x7f62901598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:48.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.689+0000 7f6294eb3700 1 --2- 172.21.15.7:0/650268218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f629015be10 0x7f62901574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:48.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.689+0000 7f6294eb3700 1 -- 172.21.15.7:0/650268218 >> 172.21.15.7:0/650268218 conn(0x7f62900679b0 msgr2=0x7f629006ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:48.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.689+0000 7f6294eb3700 1 -- 172.21.15.7:0/650268218 shutdown_connections 2024-01-27T02:32:48.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.689+0000 7f6294eb3700 1 -- 172.21.15.7:0/650268218 wait complete. 2024-01-27T02:32:48.705 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-01-27T02:32:48.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.838+0000 7f94b96eb700 1 Processor -- start 2024-01-27T02:32:48.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.838+0000 7f94b96eb700 1 -- start start 2024-01-27T02:32:48.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.839+0000 7f94b96eb700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f94b4143860 0x7f94b40d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:48.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.839+0000 7f94b96eb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f94b415e510 0x7f94b40d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:48.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.839+0000 7f94b96eb700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b40b3d90 con 0x7f94b4143c30 2024-01-27T02:32:48.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.839+0000 7f94b96eb700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b40d0ea0 con 0x7f94b415e510 2024-01-27T02:32:48.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.839+0000 7f94b96eb700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b40d1180 con 0x7f94b4143860 2024-01-27T02:32:48.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.839+0000 7f94b37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f94b415e510 0x7f94b40d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:48.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.839+0000 7f94b37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f94b415e510 0x7f94b40d0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50412/0 (socket says 172.21.15.7:50412) 2024-01-27T02:32:48.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.839+0000 7f94b37fe700 1 -- 172.21.15.7:0/3369038600 learned_addr learned my addr 172.21.15.7:0/3369038600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:48.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.839+0000 7f94b2ffd700 1 --2- 172.21.15.7:0/3369038600 >> v2:172.21.15.188:3300/0 conn(0x7f94b4143860 0x7f94b40d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:48.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.840+0000 7f94b37fe700 1 -- 172.21.15.7:0/3369038600 >> v2:172.21.15.188:3300/0 conn(0x7f94b4143860 msgr2=0x7f94b40d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:48.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.840+0000 7f94b37fe700 1 --2- 172.21.15.7:0/3369038600 >> v2:172.21.15.188:3300/0 conn(0x7f94b4143860 0x7f94b40d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:48.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.840+0000 7f94b37fe700 1 -- 172.21.15.7:0/3369038600 >> v1:172.21.15.188:6789/0 conn(0x7f94b4143c30 legacy=0x7f94b415de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:48.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.840+0000 7f94b37fe700 1 -- 172.21.15.7:0/3369038600 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b40d1460 con 0x7f94b415e510 2024-01-27T02:32:48.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.840+0000 7f94b37fe700 1 --2- 172.21.15.7:0/3369038600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94b415e510 0x7f94b40d0820 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f94b40b3a80 tx=0x7f94ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0146f07fad5a76 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:48.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.840+0000 7f94b1ffb700 1 -- 172.21.15.7:0/3369038600 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac00e070 con 0x7f94b415e510 2024-01-27T02:32:48.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.840+0000 7f94b1ffb700 1 -- 172.21.15.7:0/3369038600 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ac00ba10 con 0x7f94b415e510 2024-01-27T02:32:48.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.840+0000 7f94b1ffb700 1 -- 172.21.15.7:0/3369038600 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac0092e0 con 0x7f94b415e510 2024-01-27T02:32:48.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.841+0000 7f94b96eb700 1 -- 172.21.15.7:0/3369038600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94b415e510 msgr2=0x7f94b40d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:48.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.841+0000 7f94b96eb700 1 --2- 172.21.15.7:0/3369038600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94b415e510 0x7f94b40d0820 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f94b40b3a80 tx=0x7f94ac009840 comp rx=0 tx=0).stop 2024-01-27T02:32:48.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.841+0000 7f94b96eb700 1 -- 172.21.15.7:0/3369038600 shutdown_connections 2024-01-27T02:32:48.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.841+0000 7f94b96eb700 1 --2- 172.21.15.7:0/3369038600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94b415e510 0x7f94b40d0820 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:48.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.841+0000 7f94b96eb700 1 --2- 172.21.15.7:0/3369038600 >> v2:172.21.15.188:3300/0 conn(0x7f94b4143860 0x7f94b40d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:48.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.841+0000 7f94b96eb700 1 -- 172.21.15.7:0/3369038600 >> 172.21.15.7:0/3369038600 conn(0x7f94b4068440 msgr2=0x7f94b40709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:48.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.841+0000 7f94b96eb700 1 -- 172.21.15.7:0/3369038600 shutdown_connections 2024-01-27T02:32:48.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.841+0000 7f94b96eb700 1 -- 172.21.15.7:0/3369038600 wait complete. 2024-01-27T02:32:48.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.842+0000 7f94b96eb700 1 Processor -- start 2024-01-27T02:32:48.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.842+0000 7f94b96eb700 1 -- start start 2024-01-27T02:32:48.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.843+0000 7f94b96eb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94b4143860 0x7f94b413d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:48.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.843+0000 7f94b96eb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94b4143c30 0x7f94b413ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:48.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.843+0000 7f94b96eb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94b415e510 0x7f94b416d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:48.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.843+0000 7f94b96eb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b416aa20 con 0x7f94b4143860 2024-01-27T02:32:48.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.843+0000 7f94b96eb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b413e9a0 con 0x7f94b4143c30 2024-01-27T02:32:48.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.843+0000 7f94b96eb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b416de60 con 0x7f94b415e510 2024-01-27T02:32:48.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.843+0000 7f94b2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94b4143860 0x7f94b413d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:48.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.843+0000 7f94b2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94b4143860 0x7f94b413d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50418/0 (socket says 172.21.15.7:50418) 2024-01-27T02:32:48.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.843+0000 7f94b2ffd700 1 -- 172.21.15.7:0/1756818544 learned_addr learned my addr 172.21.15.7:0/1756818544 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:48.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.843+0000 7f94b37fe700 1 --2- 172.21.15.7:0/1756818544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94b415e510 0x7f94b416d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:48.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.843+0000 7f94b27fc700 1 --2- 172.21.15.7:0/1756818544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94b4143c30 0x7f94b413ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:48.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.843+0000 7f94b2ffd700 1 -- 172.21.15.7:0/1756818544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94b415e510 msgr2=0x7f94b416d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:48.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.843+0000 7f94b2ffd700 1 --2- 172.21.15.7:0/1756818544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94b415e510 0x7f94b416d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:48.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.843+0000 7f94b2ffd700 1 -- 172.21.15.7:0/1756818544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94b4143c30 msgr2=0x7f94b413ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:48.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.843+0000 7f94b2ffd700 1 --2- 172.21.15.7:0/1756818544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94b4143c30 0x7f94b413ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:48.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.843+0000 7f94b2ffd700 1 -- 172.21.15.7:0/1756818544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac009ae0 con 0x7f94b4143860 2024-01-27T02:32:48.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.843+0000 7f94b27fc700 1 --2- 172.21.15.7:0/1756818544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94b4143c30 0x7f94b413ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:32:48.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.844+0000 7f94b2ffd700 1 --2- 172.21.15.7:0/1756818544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94b4143860 0x7f94b413d720 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f94a400d890 tx=0x7f94a4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:48.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.844+0000 7f94aaffd700 1 -- 172.21.15.7:0/1756818544 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a400b600 con 0x7f94b4143860 2024-01-27T02:32:48.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.844+0000 7f94aaffd700 1 -- 172.21.15.7:0/1756818544 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a400b760 con 0x7f94b4143860 2024-01-27T02:32:48.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.844+0000 7f94aaffd700 1 -- 172.21.15.7:0/1756818544 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a400bae0 con 0x7f94b4143860 2024-01-27T02:32:48.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.844+0000 7f94b96eb700 1 -- 172.21.15.7:0/1756818544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b413dd90 con 0x7f94b4143860 2024-01-27T02:32:48.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.844+0000 7f94b96eb700 1 -- 172.21.15.7:0/1756818544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b413e350 con 0x7f94b4143860 2024-01-27T02:32:48.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.845+0000 7f94b96eb700 1 -- 172.21.15.7:0/1756818544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b404c710 con 0x7f94b4143860 2024-01-27T02:32:48.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.845+0000 7f94aaffd700 1 -- 172.21.15.7:0/1756818544 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f94a401e460 con 0x7f94b4143860 2024-01-27T02:32:48.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.846+0000 7f94aaffd700 1 --2- 172.21.15.7:0/1756818544 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9494066f30 0x7f94940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:48.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.846+0000 7f94aaffd700 1 -- 172.21.15.7:0/1756818544 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f94a4084890 con 0x7f94b4143860 2024-01-27T02:32:48.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.846+0000 7f94b27fc700 1 --2- 172.21.15.7:0/1756818544 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9494066f30 0x7f94940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:48.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.847+0000 7f94b27fc700 1 --2- 172.21.15.7:0/1756818544 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9494066f30 0x7f94940693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f94b415d510 tx=0x7f949c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:48.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:48.850+0000 7f94aaffd700 1 -- 172.21.15.7:0/1756818544 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94a4051e50 con 0x7f94b4143860 2024-01-27T02:32:49.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.140+0000 7f94b96eb700 1 -- 172.21.15.7:0/1756818544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f94b40d1740 con 0x7f94b4143860 2024-01-27T02:32:49.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.144+0000 7f94aaffd700 1 -- 172.21.15.7:0/1756818544 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f94a4055470 con 0x7f94b4143860 2024-01-27T02:32:49.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.146+0000 7f94b96eb700 1 -- 172.21.15.7:0/1756818544 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9494066f30 msgr2=0x7f94940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:49.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.146+0000 7f94b96eb700 1 --2- 172.21.15.7:0/1756818544 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9494066f30 0x7f94940693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f94b415d510 tx=0x7f949c009040 comp rx=0 tx=0).stop 2024-01-27T02:32:49.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.146+0000 7f94b96eb700 1 -- 172.21.15.7:0/1756818544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94b4143860 msgr2=0x7f94b413d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:49.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.146+0000 7f94b96eb700 1 --2- 172.21.15.7:0/1756818544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94b4143860 0x7f94b413d720 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f94a400d890 tx=0x7f94a4008040 comp rx=0 tx=0).stop 2024-01-27T02:32:49.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.146+0000 7f94b96eb700 1 -- 172.21.15.7:0/1756818544 shutdown_connections 2024-01-27T02:32:49.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.146+0000 7f94b96eb700 1 --2- 172.21.15.7:0/1756818544 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9494066f30 0x7f94940693e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:49.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.146+0000 7f94b96eb700 1 --2- 172.21.15.7:0/1756818544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94b415e510 0x7f94b416d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:49.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.146+0000 7f94b96eb700 1 --2- 172.21.15.7:0/1756818544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94b4143860 0x7f94b413d720 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:49.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.146+0000 7f94b96eb700 1 --2- 172.21.15.7:0/1756818544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94b4143c30 0x7f94b413ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.146+0000 7f94b96eb700 1 -- 172.21.15.7:0/1756818544 >> 172.21.15.7:0/1756818544 conn(0x7f94b4068440 msgr2=0x7f94b40b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.146+0000 7f94b96eb700 1 -- 172.21.15.7:0/1756818544 shutdown_connections 2024-01-27T02:32:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.146+0000 7f94b96eb700 1 -- 172.21.15.7:0/1756818544 wait complete. 2024-01-27T02:32:49.160 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-01-27T02:32:49.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.295+0000 7f31f5cd2700 1 Processor -- start 2024-01-27T02:32:49.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.295+0000 7f31f5cd2700 1 -- start start 2024-01-27T02:32:49.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.296+0000 7f31f5cd2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f31f00c4cc0 0x7f31f00cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:49.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.296+0000 7f31f5cd2700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f31f00ca540 0x7f31f0163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:49.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.296+0000 7f31f5cd2700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f00ce5e0 con 0x7f31f015a100 2024-01-27T02:32:49.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.296+0000 7f31f5cd2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f00cd9d0 con 0x7f31f00c4cc0 2024-01-27T02:32:49.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.296+0000 7f31f5cd2700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f00cdcb0 con 0x7f31f00ca540 2024-01-27T02:32:49.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.296+0000 7f31eeffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f31f00c4cc0 0x7f31f00cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:49.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.296+0000 7f31eeffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f31f00c4cc0 0x7f31f00cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50428/0 (socket says 172.21.15.7:50428) 2024-01-27T02:32:49.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.296+0000 7f31eeffd700 1 -- 172.21.15.7:0/3300500124 learned_addr learned my addr 172.21.15.7:0/3300500124 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:49.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.296+0000 7f31effff700 1 --2- 172.21.15.7:0/3300500124 >> v2:172.21.15.188:3300/0 conn(0x7f31f00ca540 0x7f31f0163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:49.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.296+0000 7f31eeffd700 1 -- 172.21.15.7:0/3300500124 >> v2:172.21.15.188:3300/0 conn(0x7f31f00ca540 msgr2=0x7f31f0163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:49.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.296+0000 7f31eeffd700 1 --2- 172.21.15.7:0/3300500124 >> v2:172.21.15.188:3300/0 conn(0x7f31f00ca540 0x7f31f0163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:49.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.296+0000 7f31eeffd700 1 -- 172.21.15.7:0/3300500124 >> v1:172.21.15.188:6789/0 conn(0x7f31f015a100 legacy=0x7f31f00c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:49.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.296+0000 7f31eeffd700 1 -- 172.21.15.7:0/3300500124 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f00cdf90 con 0x7f31f00c4cc0 2024-01-27T02:32:49.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.297+0000 7f31eeffd700 1 --2- 172.21.15.7:0/3300500124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f00c4cc0 0x7f31f00cd2f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f31f0069b70 tx=0x7f31e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee202325476447e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:49.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.297+0000 7f31ee7fc700 1 -- 172.21.15.7:0/3300500124 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e4009e70 con 0x7f31f00c4cc0 2024-01-27T02:32:49.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.297+0000 7f31ee7fc700 1 -- 172.21.15.7:0/3300500124 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e400b860 con 0x7f31f00c4cc0 2024-01-27T02:32:49.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.297+0000 7f31ee7fc700 1 -- 172.21.15.7:0/3300500124 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e400bc00 con 0x7f31f00c4cc0 2024-01-27T02:32:49.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.297+0000 7f31f5cd2700 1 -- 172.21.15.7:0/3300500124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f00c4cc0 msgr2=0x7f31f00cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:49.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.297+0000 7f31f5cd2700 1 --2- 172.21.15.7:0/3300500124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f00c4cc0 0x7f31f00cd2f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f31f0069b70 tx=0x7f31e4009250 comp rx=0 tx=0).stop 2024-01-27T02:32:49.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.297+0000 7f31f5cd2700 1 -- 172.21.15.7:0/3300500124 shutdown_connections 2024-01-27T02:32:49.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.297+0000 7f31f5cd2700 1 --2- 172.21.15.7:0/3300500124 >> v2:172.21.15.188:3300/0 conn(0x7f31f00ca540 0x7f31f0163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:49.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.297+0000 7f31f5cd2700 1 --2- 172.21.15.7:0/3300500124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f00c4cc0 0x7f31f00cd2f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:49.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.297+0000 7f31f5cd2700 1 -- 172.21.15.7:0/3300500124 >> 172.21.15.7:0/3300500124 conn(0x7f31f0098020 msgr2=0x7f31f00711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:49.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.297+0000 7f31f5cd2700 1 -- 172.21.15.7:0/3300500124 shutdown_connections 2024-01-27T02:32:49.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.297+0000 7f31f5cd2700 1 -- 172.21.15.7:0/3300500124 wait complete. 2024-01-27T02:32:49.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.299+0000 7f31f5cd2700 1 Processor -- start 2024-01-27T02:32:49.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.299+0000 7f31f5cd2700 1 -- start start 2024-01-27T02:32:49.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.300+0000 7f31f5cd2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f00c4cc0 0x7f31f00cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:49.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.300+0000 7f31f5cd2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31f00ca540 0x7f31f00cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:49.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.300+0000 7f31f5cd2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31f015a100 0x7f31f014d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:49.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.300+0000 7f31f5cd2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f0153f80 con 0x7f31f00c4cc0 2024-01-27T02:32:49.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.300+0000 7f31f5cd2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f0153650 con 0x7f31f00ca540 2024-01-27T02:32:49.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.300+0000 7f31f5cd2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f0153930 con 0x7f31f015a100 2024-01-27T02:32:49.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.300+0000 7f31ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f00c4cc0 0x7f31f00cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:49.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.300+0000 7f31ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f00c4cc0 0x7f31f00cb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50436/0 (socket says 172.21.15.7:50436) 2024-01-27T02:32:49.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.300+0000 7f31ef7fe700 1 -- 172.21.15.7:0/2430850762 learned_addr learned my addr 172.21.15.7:0/2430850762 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:49.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.300+0000 7f31eeffd700 1 --2- 172.21.15.7:0/2430850762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31f00ca540 0x7f31f00cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:49.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.300+0000 7f31effff700 1 --2- 172.21.15.7:0/2430850762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31f015a100 0x7f31f014d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:49.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.300+0000 7f31ef7fe700 1 -- 172.21.15.7:0/2430850762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31f015a100 msgr2=0x7f31f014d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:49.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.300+0000 7f31ef7fe700 1 --2- 172.21.15.7:0/2430850762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31f015a100 0x7f31f014d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:49.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.300+0000 7f31ef7fe700 1 -- 172.21.15.7:0/2430850762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31f00ca540 msgr2=0x7f31f00cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:49.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.300+0000 7f31ef7fe700 1 --2- 172.21.15.7:0/2430850762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31f00ca540 0x7f31f00cc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:49.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.300+0000 7f31ef7fe700 1 -- 172.21.15.7:0/2430850762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e40094f0 con 0x7f31f00c4cc0 2024-01-27T02:32:49.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.300+0000 7f31ef7fe700 1 --2- 172.21.15.7:0/2430850762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f00c4cc0 0x7f31f00cb8a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f31e000c4e0 tx=0x7f31e000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:49.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.300+0000 7f31d77fe700 1 -- 172.21.15.7:0/2430850762 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e0011590 con 0x7f31f00c4cc0 2024-01-27T02:32:49.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.300+0000 7f31d77fe700 1 -- 172.21.15.7:0/2430850762 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e0011710 con 0x7f31f00c4cc0 2024-01-27T02:32:49.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.301+0000 7f31f5cd2700 1 -- 172.21.15.7:0/2430850762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f0153c10 con 0x7f31f00c4cc0 2024-01-27T02:32:49.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.301+0000 7f31d77fe700 1 -- 172.21.15.7:0/2430850762 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e0011ad0 con 0x7f31f00c4cc0 2024-01-27T02:32:49.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.301+0000 7f31f5cd2700 1 -- 172.21.15.7:0/2430850762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f00cc730 con 0x7f31f00c4cc0 2024-01-27T02:32:49.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.302+0000 7f31f5cd2700 1 -- 172.21.15.7:0/2430850762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f0154260 con 0x7f31f00c4cc0 2024-01-27T02:32:49.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.303+0000 7f31d77fe700 1 -- 172.21.15.7:0/2430850762 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f31e001b3b0 con 0x7f31f00c4cc0 2024-01-27T02:32:49.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.303+0000 7f31d77fe700 1 --2- 172.21.15.7:0/2430850762 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f31dc066f30 0x7f31dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:49.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.303+0000 7f31d77fe700 1 -- 172.21.15.7:0/2430850762 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f31e00839b0 con 0x7f31f00c4cc0 2024-01-27T02:32:49.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.303+0000 7f31eeffd700 1 --2- 172.21.15.7:0/2430850762 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f31dc066f30 0x7f31dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:49.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.304+0000 7f31eeffd700 1 --2- 172.21.15.7:0/2430850762 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f31dc066f30 0x7f31dc0693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f31f0069b70 tx=0x7f31e4017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:49.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.307+0000 7f31d77fe700 1 -- 172.21.15.7:0/2430850762 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31e0050ff0 con 0x7f31f00c4cc0 2024-01-27T02:32:49.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.594+0000 7f31f5cd2700 1 -- 172.21.15.7:0/2430850762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f31f004c710 con 0x7f31f00c4cc0 2024-01-27T02:32:49.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.598+0000 7f31d77fe700 1 -- 172.21.15.7:0/2430850762 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f31e0017020 con 0x7f31f00c4cc0 2024-01-27T02:32:49.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.601+0000 7f31f5cd2700 1 -- 172.21.15.7:0/2430850762 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f31dc066f30 msgr2=0x7f31dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:49.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.601+0000 7f31f5cd2700 1 --2- 172.21.15.7:0/2430850762 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f31dc066f30 0x7f31dc0693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f31f0069b70 tx=0x7f31e4017000 comp rx=0 tx=0).stop 2024-01-27T02:32:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.601+0000 7f31f5cd2700 1 -- 172.21.15.7:0/2430850762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f00c4cc0 msgr2=0x7f31f00cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.601+0000 7f31f5cd2700 1 --2- 172.21.15.7:0/2430850762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f00c4cc0 0x7f31f00cb8a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f31e000c4e0 tx=0x7f31e000c7f0 comp rx=0 tx=0).stop 2024-01-27T02:32:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.601+0000 7f31f5cd2700 1 -- 172.21.15.7:0/2430850762 shutdown_connections 2024-01-27T02:32:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.601+0000 7f31f5cd2700 1 --2- 172.21.15.7:0/2430850762 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f31dc066f30 0x7f31dc0693e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.601+0000 7f31f5cd2700 1 --2- 172.21.15.7:0/2430850762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31f015a100 0x7f31f014d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.601+0000 7f31f5cd2700 1 --2- 172.21.15.7:0/2430850762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f00c4cc0 0x7f31f00cb8a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.601+0000 7f31f5cd2700 1 --2- 172.21.15.7:0/2430850762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31f00ca540 0x7f31f00cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.601+0000 7f31f5cd2700 1 -- 172.21.15.7:0/2430850762 >> 172.21.15.7:0/2430850762 conn(0x7f31f0098020 msgr2=0x7f31f014ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.601+0000 7f31f5cd2700 1 -- 172.21.15.7:0/2430850762 shutdown_connections 2024-01-27T02:32:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.601+0000 7f31f5cd2700 1 -- 172.21.15.7:0/2430850762 wait complete. 2024-01-27T02:32:49.615 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-01-27T02:32:49.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.746+0000 7fded351d700 1 Processor -- start 2024-01-27T02:32:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.746+0000 7fded351d700 1 -- start start 2024-01-27T02:32:49.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.747+0000 7fded351d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdecc06d360 0x7fdecc0ae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.747+0000 7fded351d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdecc0c9aa0 0x7fdecc0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.747+0000 7fded351d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdecc0af2e0 con 0x7fdecc0c96d0 2024-01-27T02:32:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.747+0000 7fded351d700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdecc162150 con 0x7fdecc0c9aa0 2024-01-27T02:32:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.747+0000 7fded351d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc162360 con 0x7fdecc06d360 2024-01-27T02:32:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.747+0000 7fded12b9700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdecc0c96d0 0x7fdecc0cc080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42932/0 (socket says 172.21.15.7:42932) 2024-01-27T02:32:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.747+0000 7fded12b9700 1 -- 172.21.15.7:0/95846008 learned_addr learned my addr 172.21.15.7:0/95846008 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.747+0000 7fded1aba700 1 --2- 172.21.15.7:0/95846008 >> v2:172.21.15.188:3300/0 conn(0x7fdecc0c9aa0 0x7fdecc0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.747+0000 7fded0ab8700 1 --2- 172.21.15.7:0/95846008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdecc06d360 0x7fdecc0ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.747+0000 7fdec3fff700 1 -- 172.21.15.7:0/95846008 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 175720240 0 0) 0x7fdecc0af2e0 con 0x7fdecc0c96d0 2024-01-27T02:32:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.747+0000 7fdec3fff700 1 -- 172.21.15.7:0/95846008 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdeb8003570 con 0x7fdecc0c96d0 2024-01-27T02:32:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.748+0000 7fdec3fff700 1 -- 172.21.15.7:0/95846008 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 988073275 0 0) 0x7fdeb8003570 con 0x7fdecc0c96d0 2024-01-27T02:32:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.748+0000 7fded0ab8700 1 -- 172.21.15.7:0/95846008 >> v2:172.21.15.188:3300/0 conn(0x7fdecc0c9aa0 msgr2=0x7fdecc0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.748+0000 7fded0ab8700 1 --2- 172.21.15.7:0/95846008 >> v2:172.21.15.188:3300/0 conn(0x7fdecc0c9aa0 0x7fdecc0aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.748+0000 7fded0ab8700 1 -- 172.21.15.7:0/95846008 >> v1:172.21.15.7:6789/0 conn(0x7fdecc0c96d0 legacy=0x7fdecc0cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.748+0000 7fded0ab8700 1 -- 172.21.15.7:0/95846008 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdecc162640 con 0x7fdecc06d360 2024-01-27T02:32:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.748+0000 7fded1aba700 1 --2- 172.21.15.7:0/95846008 >> v2:172.21.15.188:3300/0 conn(0x7fdecc0c9aa0 0x7fdecc0aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:32:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.748+0000 7fded0ab8700 1 --2- 172.21.15.7:0/95846008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdecc06d360 0x7fdecc0ae860 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0098e0 tx=0x7fdebc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a6249b5cc22c39 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.748+0000 7fdec3fff700 1 -- 172.21.15.7:0/95846008 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec8001470 con 0x7fdecc06d360 2024-01-27T02:32:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.748+0000 7fdec3fff700 1 -- 172.21.15.7:0/95846008 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdebc011420 con 0x7fdecc06d360 2024-01-27T02:32:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.748+0000 7fdec3fff700 1 -- 172.21.15.7:0/95846008 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc0117c0 con 0x7fdecc06d360 2024-01-27T02:32:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.749+0000 7fded351d700 1 -- 172.21.15.7:0/95846008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdecc06d360 msgr2=0x7fdecc0ae860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.749+0000 7fded351d700 1 --2- 172.21.15.7:0/95846008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdecc06d360 0x7fdecc0ae860 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0098e0 tx=0x7fdebc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.749+0000 7fded351d700 1 -- 172.21.15.7:0/95846008 shutdown_connections 2024-01-27T02:32:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.749+0000 7fded351d700 1 --2- 172.21.15.7:0/95846008 >> v2:172.21.15.188:3300/0 conn(0x7fdecc0c9aa0 0x7fdecc0aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.749+0000 7fded351d700 1 --2- 172.21.15.7:0/95846008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdecc06d360 0x7fdecc0ae860 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.749+0000 7fded351d700 1 -- 172.21.15.7:0/95846008 >> 172.21.15.7:0/95846008 conn(0x7fdecc063580 msgr2=0x7fdecc063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.749+0000 7fded351d700 1 -- 172.21.15.7:0/95846008 shutdown_connections 2024-01-27T02:32:49.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.749+0000 7fded351d700 1 -- 172.21.15.7:0/95846008 wait complete. 2024-01-27T02:32:49.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.750+0000 7fded351d700 1 Processor -- start 2024-01-27T02:32:49.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.750+0000 7fded351d700 1 -- start start 2024-01-27T02:32:49.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.751+0000 7fded351d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdecc06d360 0x7fdecc15d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:49.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.751+0000 7fded351d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdecc0c96d0 0x7fdecc1609d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:49.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.751+0000 7fded351d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdecc0c9aa0 0x7fdecc0cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:49.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.751+0000 7fded351d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc161c50 con 0x7fdecc0c9aa0 2024-01-27T02:32:49.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.751+0000 7fded351d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc0c3cc0 con 0x7fdecc0c96d0 2024-01-27T02:32:49.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.751+0000 7fded351d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc15d5d0 con 0x7fdecc06d360 2024-01-27T02:32:49.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.751+0000 7fded1aba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdecc0c9aa0 0x7fdecc0cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:49.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.751+0000 7fded1aba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdecc0c9aa0 0x7fdecc0cac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50452/0 (socket says 172.21.15.7:50452) 2024-01-27T02:32:49.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.751+0000 7fded1aba700 1 -- 172.21.15.7:0/477419554 learned_addr learned my addr 172.21.15.7:0/477419554 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:49.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.752+0000 7fded12b9700 1 --2- 172.21.15.7:0/477419554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdecc06d360 0x7fdecc15d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:49.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.752+0000 7fded0ab8700 1 --2- 172.21.15.7:0/477419554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdecc0c96d0 0x7fdecc1609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:49.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.752+0000 7fded1aba700 1 -- 172.21.15.7:0/477419554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdecc06d360 msgr2=0x7fdecc15d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:49.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.752+0000 7fded1aba700 1 --2- 172.21.15.7:0/477419554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdecc06d360 0x7fdecc15d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:49.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.752+0000 7fded1aba700 1 -- 172.21.15.7:0/477419554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdecc0c96d0 msgr2=0x7fdecc1609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:49.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.752+0000 7fded1aba700 1 --2- 172.21.15.7:0/477419554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdecc0c96d0 0x7fdecc1609d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:49.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.752+0000 7fded1aba700 1 -- 172.21.15.7:0/477419554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec4009a20 con 0x7fdecc0c9aa0 2024-01-27T02:32:49.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.752+0000 7fded0ab8700 1 --2- 172.21.15.7:0/477419554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdecc0c96d0 0x7fdecc1609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:32:49.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.752+0000 7fded1aba700 1 --2- 172.21.15.7:0/477419554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdecc0c9aa0 0x7fdecc0cac60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009f20 tx=0x7fdec400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:49.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.752+0000 7fdec17fa700 1 -- 172.21.15.7:0/477419554 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec400cc80 con 0x7fdecc0c9aa0 2024-01-27T02:32:49.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.752+0000 7fdec17fa700 1 -- 172.21.15.7:0/477419554 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec400f040 con 0x7fdecc0c9aa0 2024-01-27T02:32:49.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.752+0000 7fdec17fa700 1 -- 172.21.15.7:0/477419554 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec4014480 con 0x7fdecc0c9aa0 2024-01-27T02:32:49.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.752+0000 7fded351d700 1 -- 172.21.15.7:0/477419554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdebc009420 con 0x7fdecc0c9aa0 2024-01-27T02:32:49.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.752+0000 7fded351d700 1 -- 172.21.15.7:0/477419554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdecc161690 con 0x7fdecc0c9aa0 2024-01-27T02:32:49.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.753+0000 7fded351d700 1 -- 172.21.15.7:0/477419554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdecc0c3a90 con 0x7fdecc0c9aa0 2024-01-27T02:32:49.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.753+0000 7fdec17fa700 1 -- 172.21.15.7:0/477419554 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fdec4003020 con 0x7fdecc0c9aa0 2024-01-27T02:32:49.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.754+0000 7fdec17fa700 1 --2- 172.21.15.7:0/477419554 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdeb0066f30 0x7fdeb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:49.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.754+0000 7fdec17fa700 1 -- 172.21.15.7:0/477419554 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fdec4084be0 con 0x7fdecc0c9aa0 2024-01-27T02:32:49.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.754+0000 7fded12b9700 1 --2- 172.21.15.7:0/477419554 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdeb0066f30 0x7fdeb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:49.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.755+0000 7fded12b9700 1 --2- 172.21.15.7:0/477419554 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdeb0066f30 0x7fdeb00693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdec80011e0 tx=0x7fdec8001490 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:49.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:49.758+0000 7fdec17fa700 1 -- 172.21.15.7:0/477419554 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdecc0c3a90 con 0x7fdecc0c9aa0 2024-01-27T02:32:50.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.046+0000 7fded351d700 1 -- 172.21.15.7:0/477419554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fdecc0c3a90 con 0x7fdecc0c9aa0 2024-01-27T02:32:50.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.049+0000 7fdec17fa700 1 -- 172.21.15.7:0/477419554 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7fdecc0c3a90 con 0x7fdecc0c9aa0 2024-01-27T02:32:50.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.052+0000 7fded351d700 1 -- 172.21.15.7:0/477419554 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdeb0066f30 msgr2=0x7fdeb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:50.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.052+0000 7fded351d700 1 --2- 172.21.15.7:0/477419554 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdeb0066f30 0x7fdeb00693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdec80011e0 tx=0x7fdec8001490 comp rx=0 tx=0).stop 2024-01-27T02:32:50.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.052+0000 7fded351d700 1 -- 172.21.15.7:0/477419554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdecc0c9aa0 msgr2=0x7fdecc0cac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:50.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.052+0000 7fded351d700 1 --2- 172.21.15.7:0/477419554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdecc0c9aa0 0x7fdecc0cac60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009f20 tx=0x7fdec400b040 comp rx=0 tx=0).stop 2024-01-27T02:32:50.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.052+0000 7fded351d700 1 -- 172.21.15.7:0/477419554 shutdown_connections 2024-01-27T02:32:50.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.052+0000 7fded351d700 1 --2- 172.21.15.7:0/477419554 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdeb0066f30 0x7fdeb00693e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.052+0000 7fded351d700 1 --2- 172.21.15.7:0/477419554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdecc0c9aa0 0x7fdecc0cac60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.052+0000 7fded351d700 1 --2- 172.21.15.7:0/477419554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdecc06d360 0x7fdecc15d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.052+0000 7fded351d700 1 --2- 172.21.15.7:0/477419554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdecc0c96d0 0x7fdecc1609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.052+0000 7fded351d700 1 -- 172.21.15.7:0/477419554 >> 172.21.15.7:0/477419554 conn(0x7fdecc063580 msgr2=0x7fdecc0c7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:50.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.052+0000 7fded351d700 1 -- 172.21.15.7:0/477419554 shutdown_connections 2024-01-27T02:32:50.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.052+0000 7fded351d700 1 -- 172.21.15.7:0/477419554 wait complete. 2024-01-27T02:32:50.066 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-01-27T02:32:50.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.197+0000 7f263fd9a700 1 Processor -- start 2024-01-27T02:32:50.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.198+0000 7f263fd9a700 1 -- start start 2024-01-27T02:32:50.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.198+0000 7f263fd9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f26380b1900 0x7f26380b0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:50.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.198+0000 7f263fd9a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f263815c1e0 0x7f26380c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:50.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.199+0000 7f263fd9a700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26380b1470 con 0x7f263815be10 2024-01-27T02:32:50.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.199+0000 7f263fd9a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26380b1d50 con 0x7f26380b1900 2024-01-27T02:32:50.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.199+0000 7f263fd9a700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26380b0860 con 0x7f263815c1e0 2024-01-27T02:32:50.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.199+0000 7f263d335700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f26380b1900 0x7f26380b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:50.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.199+0000 7f263d335700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f26380b1900 0x7f26380b0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50454/0 (socket says 172.21.15.7:50454) 2024-01-27T02:32:50.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.199+0000 7f263d335700 1 -- 172.21.15.7:0/524825634 learned_addr learned my addr 172.21.15.7:0/524825634 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:50.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.199+0000 7f263e337700 1 --2- 172.21.15.7:0/524825634 >> v2:172.21.15.188:3300/0 conn(0x7f263815c1e0 0x7f26380c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:50.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.199+0000 7f263d335700 1 -- 172.21.15.7:0/524825634 >> v2:172.21.15.188:3300/0 conn(0x7f263815c1e0 msgr2=0x7f26380c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:50.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.199+0000 7f263d335700 1 --2- 172.21.15.7:0/524825634 >> v2:172.21.15.188:3300/0 conn(0x7f263815c1e0 0x7f26380c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.199+0000 7f263d335700 1 -- 172.21.15.7:0/524825634 >> v1:172.21.15.188:6789/0 conn(0x7f263815be10 legacy=0x7f26380aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:50.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.199+0000 7f263d335700 1 -- 172.21.15.7:0/524825634 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26380b0b40 con 0x7f26380b1900 2024-01-27T02:32:50.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.199+0000 7f263e337700 1 --2- 172.21.15.7:0/524825634 >> v2:172.21.15.188:3300/0 conn(0x7f263815c1e0 0x7f26380c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:50.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.199+0000 7f263d335700 1 --2- 172.21.15.7:0/524825634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26380b1900 0x7f26380b0180 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f26380add40 tx=0x7f2628009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114c3735004867b7 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:50.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.199+0000 7f263cb34700 1 -- 172.21.15.7:0/524825634 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262800b860 con 0x7f26380b1900 2024-01-27T02:32:50.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.199+0000 7f263cb34700 1 -- 172.21.15.7:0/524825634 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262800b9c0 con 0x7f26380b1900 2024-01-27T02:32:50.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.199+0000 7f263cb34700 1 -- 172.21.15.7:0/524825634 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262800bd60 con 0x7f26380b1900 2024-01-27T02:32:50.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.200+0000 7f263fd9a700 1 -- 172.21.15.7:0/524825634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26380b1900 msgr2=0x7f26380b0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:50.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.200+0000 7f263fd9a700 1 --2- 172.21.15.7:0/524825634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26380b1900 0x7f26380b0180 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f26380add40 tx=0x7f2628009250 comp rx=0 tx=0).stop 2024-01-27T02:32:50.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.200+0000 7f263fd9a700 1 -- 172.21.15.7:0/524825634 shutdown_connections 2024-01-27T02:32:50.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.200+0000 7f263fd9a700 1 --2- 172.21.15.7:0/524825634 >> v2:172.21.15.188:3300/0 conn(0x7f263815c1e0 0x7f26380c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.200+0000 7f263fd9a700 1 --2- 172.21.15.7:0/524825634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26380b1900 0x7f26380b0180 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.200+0000 7f263fd9a700 1 -- 172.21.15.7:0/524825634 >> 172.21.15.7:0/524825634 conn(0x7f26380679b0 msgr2=0x7f2638067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:50.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.200+0000 7f263fd9a700 1 -- 172.21.15.7:0/524825634 shutdown_connections 2024-01-27T02:32:50.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.200+0000 7f263fd9a700 1 -- 172.21.15.7:0/524825634 wait complete. 2024-01-27T02:32:50.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.201+0000 7f263fd9a700 1 Processor -- start 2024-01-27T02:32:50.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.201+0000 7f263fd9a700 1 -- start start 2024-01-27T02:32:50.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.202+0000 7f263fd9a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26380b1900 0x7f26380ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:50.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.202+0000 7f263fd9a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f263815be10 0x7f263815ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:50.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.202+0000 7f263fd9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f263815c1e0 0x7f263815d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:50.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.202+0000 7f263fd9a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2638169070 con 0x7f263815c1e0 2024-01-27T02:32:50.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.202+0000 7f263fd9a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2638168740 con 0x7f263815be10 2024-01-27T02:32:50.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.202+0000 7f263fd9a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2638168a20 con 0x7f26380b1900 2024-01-27T02:32:50.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.202+0000 7f263e337700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f263815c1e0 0x7f263815d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:50.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.202+0000 7f263e337700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f263815c1e0 0x7f263815d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50460/0 (socket says 172.21.15.7:50460) 2024-01-27T02:32:50.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.202+0000 7f263e337700 1 -- 172.21.15.7:0/3911851620 learned_addr learned my addr 172.21.15.7:0/3911851620 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:50.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.202+0000 7f263d335700 1 --2- 172.21.15.7:0/3911851620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f263815be10 0x7f263815ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:50.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.202+0000 7f263db36700 1 --2- 172.21.15.7:0/3911851620 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26380b1900 0x7f26380ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:50.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.203+0000 7f263e337700 1 -- 172.21.15.7:0/3911851620 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26380b1900 msgr2=0x7f26380ab9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:50.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.203+0000 7f263e337700 1 --2- 172.21.15.7:0/3911851620 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26380b1900 0x7f26380ab9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.203+0000 7f263e337700 1 -- 172.21.15.7:0/3911851620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f263815be10 msgr2=0x7f263815ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.203+0000 7f263e337700 1 --2- 172.21.15.7:0/3911851620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f263815be10 0x7f263815ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.203+0000 7f263e337700 1 -- 172.21.15.7:0/3911851620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26280094f0 con 0x7f263815c1e0 2024-01-27T02:32:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.203+0000 7f263e337700 1 --2- 172.21.15.7:0/3911851620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f263815c1e0 0x7f263815d3a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2630009fc0 tx=0x7f26300099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.203+0000 7f262dffb700 1 -- 172.21.15.7:0/3911851620 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263000dbe0 con 0x7f263815c1e0 2024-01-27T02:32:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.203+0000 7f262dffb700 1 -- 172.21.15.7:0/3911851620 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263000f040 con 0x7f263815c1e0 2024-01-27T02:32:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.203+0000 7f263fd9a700 1 -- 172.21.15.7:0/3911851620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2638168d00 con 0x7f263815c1e0 2024-01-27T02:32:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.203+0000 7f262dffb700 1 -- 172.21.15.7:0/3911851620 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2630015440 con 0x7f263815c1e0 2024-01-27T02:32:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.203+0000 7f263fd9a700 1 -- 172.21.15.7:0/3911851620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26381683c0 con 0x7f263815c1e0 2024-01-27T02:32:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.204+0000 7f263fd9a700 1 -- 172.21.15.7:0/3911851620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2638169400 con 0x7f263815c1e0 2024-01-27T02:32:50.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.205+0000 7f262dffb700 1 -- 172.21.15.7:0/3911851620 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f263001e3b0 con 0x7f263815c1e0 2024-01-27T02:32:50.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.205+0000 7f262dffb700 1 --2- 172.21.15.7:0/3911851620 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f261c066f30 0x7f261c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:50.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.205+0000 7f262dffb700 1 -- 172.21.15.7:0/3911851620 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f26300845c0 con 0x7f263815c1e0 2024-01-27T02:32:50.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.205+0000 7f263db36700 1 --2- 172.21.15.7:0/3911851620 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f261c066f30 0x7f261c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:50.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.206+0000 7f263db36700 1 --2- 172.21.15.7:0/3911851620 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f261c066f30 0x7f261c0693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2638099860 tx=0x7f2634009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:50.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.210+0000 7f262dffb700 1 -- 172.21.15.7:0/3911851620 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2638169400 con 0x7f263815c1e0 2024-01-27T02:32:50.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.494+0000 7f263fd9a700 1 -- 172.21.15.7:0/3911851620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f2638169400 con 0x7f263815c1e0 2024-01-27T02:32:50.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.498+0000 7f262dffb700 1 -- 172.21.15.7:0/3911851620 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2638169400 con 0x7f263815c1e0 2024-01-27T02:32:50.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.500+0000 7f263fd9a700 1 -- 172.21.15.7:0/3911851620 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f261c066f30 msgr2=0x7f261c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:50.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.500+0000 7f263fd9a700 1 --2- 172.21.15.7:0/3911851620 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f261c066f30 0x7f261c0693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2638099860 tx=0x7f2634009040 comp rx=0 tx=0).stop 2024-01-27T02:32:50.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.500+0000 7f263fd9a700 1 -- 172.21.15.7:0/3911851620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f263815c1e0 msgr2=0x7f263815d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:50.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.500+0000 7f263fd9a700 1 --2- 172.21.15.7:0/3911851620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f263815c1e0 0x7f263815d3a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2630009fc0 tx=0x7f26300099b0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.500+0000 7f263fd9a700 1 -- 172.21.15.7:0/3911851620 shutdown_connections 2024-01-27T02:32:50.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.500+0000 7f263fd9a700 1 --2- 172.21.15.7:0/3911851620 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f261c066f30 0x7f261c0693e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.500+0000 7f263fd9a700 1 --2- 172.21.15.7:0/3911851620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f263815c1e0 0x7f263815d3a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.500+0000 7f263fd9a700 1 --2- 172.21.15.7:0/3911851620 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26380b1900 0x7f26380ab9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.500+0000 7f263fd9a700 1 --2- 172.21.15.7:0/3911851620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f263815be10 0x7f263815ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.500+0000 7f263fd9a700 1 -- 172.21.15.7:0/3911851620 >> 172.21.15.7:0/3911851620 conn(0x7f26380679b0 msgr2=0x7f26380ae7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:50.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.500+0000 7f263fd9a700 1 -- 172.21.15.7:0/3911851620 shutdown_connections 2024-01-27T02:32:50.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.500+0000 7f263fd9a700 1 -- 172.21.15.7:0/3911851620 wait complete. 2024-01-27T02:32:50.514 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-01-27T02:32:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.646+0000 7fb735c13700 1 Processor -- start 2024-01-27T02:32:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.646+0000 7fb735c13700 1 -- start start 2024-01-27T02:32:50.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.647+0000 7fb735c13700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb73015be10 0x7fb7300b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.647+0000 7fb735c13700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb73015c1e0 0x7fb7300c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.647+0000 7fb735c13700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7300b1470 con 0x7fb7300b0180 2024-01-27T02:32:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.647+0000 7fb735c13700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7300b0600 con 0x7fb73015be10 2024-01-27T02:32:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.647+0000 7fb735c13700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300b0860 con 0x7fb73015c1e0 2024-01-27T02:32:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.648+0000 7fb72f7fe700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb73015be10 0x7fb7300b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.648+0000 7fb72ffff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb73015c1e0 0x7fb7300c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.648+0000 7fb72ffff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb73015c1e0 0x7fb7300c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:33948/0 (socket says 172.21.15.7:33948) 2024-01-27T02:32:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.648+0000 7fb72f7fe700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb73015be10 0x7fb7300b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:52924/0 (socket says 172.21.15.7:52924) 2024-01-27T02:32:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.648+0000 7fb72effd700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fb7300b0180 0x7fb7300aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:38788/0 (socket says 172.21.15.7:38788) 2024-01-27T02:32:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.648+0000 7fb72f7fe700 1 -- 172.21.15.7:0/1197991756 learned_addr learned my addr 172.21.15.7:0/1197991756 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.648+0000 7fb72ffff700 1 -- 172.21.15.7:0/1197991756 >> v2:172.21.15.188:3300/0 conn(0x7fb73015be10 msgr2=0x7fb7300b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.648+0000 7fb72ffff700 1 --2- 172.21.15.7:0/1197991756 >> v2:172.21.15.188:3300/0 conn(0x7fb73015be10 0x7fb7300b1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.648+0000 7fb72ffff700 1 -- 172.21.15.7:0/1197991756 >> v1:172.21.15.188:6789/0 conn(0x7fb7300b0180 legacy=0x7fb7300aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:50.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.648+0000 7fb72ffff700 1 -- 172.21.15.7:0/1197991756 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7300b0b40 con 0x7fb73015c1e0 2024-01-27T02:32:50.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.648+0000 7fb72f7fe700 1 --2- 172.21.15.7:0/1197991756 >> v2:172.21.15.188:3300/0 conn(0x7fb73015be10 0x7fb7300b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:32:50.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.648+0000 7fb72ffff700 1 --2- 172.21.15.7:0/1197991756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb73015c1e0 0x7fb7300c9500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb728009e60 tx=0x7fb728009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eb486b3811d8e01 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:50.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.648+0000 7fb72e7fc700 1 -- 172.21.15.7:0/1197991756 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72800e070 con 0x7fb73015c1e0 2024-01-27T02:32:50.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.649+0000 7fb72e7fc700 1 -- 172.21.15.7:0/1197991756 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72800bb60 con 0x7fb73015c1e0 2024-01-27T02:32:50.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.649+0000 7fb72e7fc700 1 -- 172.21.15.7:0/1197991756 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb728012430 con 0x7fb73015c1e0 2024-01-27T02:32:50.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.649+0000 7fb735c13700 1 -- 172.21.15.7:0/1197991756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb73015c1e0 msgr2=0x7fb7300c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:50.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.649+0000 7fb735c13700 1 --2- 172.21.15.7:0/1197991756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb73015c1e0 0x7fb7300c9500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb728009e60 tx=0x7fb728009780 comp rx=0 tx=0).stop 2024-01-27T02:32:50.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.649+0000 7fb735c13700 1 -- 172.21.15.7:0/1197991756 shutdown_connections 2024-01-27T02:32:50.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.649+0000 7fb735c13700 1 --2- 172.21.15.7:0/1197991756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb73015c1e0 0x7fb7300c9500 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.649+0000 7fb735c13700 1 --2- 172.21.15.7:0/1197991756 >> v2:172.21.15.188:3300/0 conn(0x7fb73015be10 0x7fb7300b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.649+0000 7fb735c13700 1 -- 172.21.15.7:0/1197991756 >> 172.21.15.7:0/1197991756 conn(0x7fb7300679b0 msgr2=0x7fb730067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:50.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.649+0000 7fb735c13700 1 -- 172.21.15.7:0/1197991756 shutdown_connections 2024-01-27T02:32:50.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.649+0000 7fb735c13700 1 -- 172.21.15.7:0/1197991756 wait complete. 2024-01-27T02:32:50.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.651+0000 7fb735c13700 1 Processor -- start 2024-01-27T02:32:50.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.651+0000 7fb735c13700 1 -- start start 2024-01-27T02:32:50.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.652+0000 7fb735c13700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7300b0180 0x7fb730099160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:50.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.652+0000 7fb735c13700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb73015be10 0x7fb73014bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:50.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.652+0000 7fb735c13700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb73015c1e0 0x7fb7300cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:50.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.652+0000 7fb735c13700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300cb910 con 0x7fb73015c1e0 2024-01-27T02:32:50.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.652+0000 7fb735c13700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73014c000 con 0x7fb73015be10 2024-01-27T02:32:50.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.652+0000 7fb735c13700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300997f0 con 0x7fb7300b0180 2024-01-27T02:32:50.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.652+0000 7fb72ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb73015c1e0 0x7fb7300cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:50.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.652+0000 7fb72ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb73015c1e0 0x7fb7300cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50476/0 (socket says 172.21.15.7:50476) 2024-01-27T02:32:50.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.652+0000 7fb72ffff700 1 -- 172.21.15.7:0/2627060583 learned_addr learned my addr 172.21.15.7:0/2627060583 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:50.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.652+0000 7fb72effd700 1 --2- 172.21.15.7:0/2627060583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb73015be10 0x7fb73014bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:50.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.652+0000 7fb72f7fe700 1 --2- 172.21.15.7:0/2627060583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7300b0180 0x7fb730099160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:50.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.653+0000 7fb72ffff700 1 -- 172.21.15.7:0/2627060583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7300b0180 msgr2=0x7fb730099160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:50.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.653+0000 7fb72ffff700 1 --2- 172.21.15.7:0/2627060583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7300b0180 0x7fb730099160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.653+0000 7fb72ffff700 1 -- 172.21.15.7:0/2627060583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb73015be10 msgr2=0x7fb73014bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:50.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.653+0000 7fb72ffff700 1 --2- 172.21.15.7:0/2627060583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb73015be10 0x7fb73014bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.653+0000 7fb72ffff700 1 -- 172.21.15.7:0/2627060583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7200094f0 con 0x7fb73015c1e0 2024-01-27T02:32:50.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.653+0000 7fb72ffff700 1 --2- 172.21.15.7:0/2627060583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb73015c1e0 0x7fb7300cce70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb7300b09a0 tx=0x7fb728019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:50.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.653+0000 7fb7177fe700 1 -- 172.21.15.7:0/2627060583 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb728027470 con 0x7fb73015c1e0 2024-01-27T02:32:50.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.653+0000 7fb7177fe700 1 -- 172.21.15.7:0/2627060583 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7280275f0 con 0x7fb73015c1e0 2024-01-27T02:32:50.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.653+0000 7fb735c13700 1 -- 172.21.15.7:0/2627060583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb728009a20 con 0x7fb73015c1e0 2024-01-27T02:32:50.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.653+0000 7fb7177fe700 1 -- 172.21.15.7:0/2627060583 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7280279b0 con 0x7fb73015c1e0 2024-01-27T02:32:50.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.653+0000 7fb735c13700 1 -- 172.21.15.7:0/2627060583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb730099db0 con 0x7fb73015c1e0 2024-01-27T02:32:50.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.654+0000 7fb735c13700 1 -- 172.21.15.7:0/2627060583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73004c710 con 0x7fb73015c1e0 2024-01-27T02:32:50.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.654+0000 7fb7177fe700 1 -- 172.21.15.7:0/2627060583 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb728024070 con 0x7fb73015c1e0 2024-01-27T02:32:50.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.655+0000 7fb7177fe700 1 --2- 172.21.15.7:0/2627060583 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb71c066f30 0x7fb71c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:50.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.655+0000 7fb7177fe700 1 -- 172.21.15.7:0/2627060583 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb728086b80 con 0x7fb73015c1e0 2024-01-27T02:32:50.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.656+0000 7fb72f7fe700 1 --2- 172.21.15.7:0/2627060583 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb71c066f30 0x7fb71c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:50.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.656+0000 7fb72f7fe700 1 --2- 172.21.15.7:0/2627060583 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb71c066f30 0x7fb71c0693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb720006d20 tx=0x7fb720009250 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.660+0000 7fb7177fe700 1 -- 172.21.15.7:0/2627060583 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb728054140 con 0x7fb73015c1e0 2024-01-27T02:32:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.945+0000 7fb735c13700 1 -- 172.21.15.7:0/2627060583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fb7300b0e20 con 0x7fb73015c1e0 2024-01-27T02:32:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.949+0000 7fb7177fe700 1 -- 172.21.15.7:0/2627060583 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb728057760 con 0x7fb73015c1e0 2024-01-27T02:32:50.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.952+0000 7fb735c13700 1 -- 172.21.15.7:0/2627060583 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb71c066f30 msgr2=0x7fb71c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:50.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.952+0000 7fb735c13700 1 --2- 172.21.15.7:0/2627060583 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb71c066f30 0x7fb71c0693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb720006d20 tx=0x7fb720009250 comp rx=0 tx=0).stop 2024-01-27T02:32:50.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.952+0000 7fb735c13700 1 -- 172.21.15.7:0/2627060583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb73015c1e0 msgr2=0x7fb7300cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:50.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.952+0000 7fb735c13700 1 --2- 172.21.15.7:0/2627060583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb73015c1e0 0x7fb7300cce70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb7300b09a0 tx=0x7fb728019000 comp rx=0 tx=0).stop 2024-01-27T02:32:50.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.952+0000 7fb735c13700 1 -- 172.21.15.7:0/2627060583 shutdown_connections 2024-01-27T02:32:50.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.952+0000 7fb735c13700 1 --2- 172.21.15.7:0/2627060583 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb71c066f30 0x7fb71c0693e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.952+0000 7fb735c13700 1 --2- 172.21.15.7:0/2627060583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb73015c1e0 0x7fb7300cce70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.952+0000 7fb735c13700 1 --2- 172.21.15.7:0/2627060583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7300b0180 0x7fb730099160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.952+0000 7fb735c13700 1 --2- 172.21.15.7:0/2627060583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb73015be10 0x7fb73014bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:50.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.952+0000 7fb735c13700 1 -- 172.21.15.7:0/2627060583 >> 172.21.15.7:0/2627060583 conn(0x7fb7300679b0 msgr2=0x7fb7301576c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:50.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.952+0000 7fb735c13700 1 -- 172.21.15.7:0/2627060583 shutdown_connections 2024-01-27T02:32:50.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:50.952+0000 7fb735c13700 1 -- 172.21.15.7:0/2627060583 wait complete. 2024-01-27T02:32:50.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-01-27T02:32:51.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.098+0000 7fd3b6c66700 1 Processor -- start 2024-01-27T02:32:51.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.098+0000 7fd3b6c66700 1 -- start start 2024-01-27T02:32:51.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.099+0000 7fd3b6c66700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3b00c8ab0 0x7fd3b015dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:51.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.099+0000 7fd3b6c66700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd3b00c8e80 0x7fd3b00cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:51.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.099+0000 7fd3b6c66700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3b015a9b0 con 0x7fd3b00cbbc0 2024-01-27T02:32:51.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.099+0000 7fd3b6c66700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3b00cda10 con 0x7fd3b00c8e80 2024-01-27T02:32:51.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.099+0000 7fd3b6c66700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b015ecb0 con 0x7fd3b00c8ab0 2024-01-27T02:32:51.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.099+0000 7fd3b5203700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd3b00cbbc0 0x7fd3b015ae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42948/0 (socket says 172.21.15.7:42948) 2024-01-27T02:32:51.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.099+0000 7fd3b5203700 1 -- 172.21.15.7:0/4053646310 learned_addr learned my addr 172.21.15.7:0/4053646310 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:51.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.099+0000 7fd3affff700 1 --2- 172.21.15.7:0/4053646310 >> v2:172.21.15.188:3300/0 conn(0x7fd3b00c8e80 0x7fd3b00cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:51.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.099+0000 7fd3b4a02700 1 --2- 172.21.15.7:0/4053646310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3b00c8ab0 0x7fd3b015dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:51.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.099+0000 7fd3af7fe700 1 -- 172.21.15.7:0/4053646310 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2109751429 0 0) 0x7fd3b015a9b0 con 0x7fd3b00cbbc0 2024-01-27T02:32:51.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.099+0000 7fd3af7fe700 1 -- 172.21.15.7:0/4053646310 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3a0003570 con 0x7fd3b00cbbc0 2024-01-27T02:32:51.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.099+0000 7fd3affff700 1 -- 172.21.15.7:0/4053646310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3b00c8ab0 msgr2=0x7fd3b015dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:51.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.100+0000 7fd3affff700 1 --2- 172.21.15.7:0/4053646310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3b00c8ab0 0x7fd3b015dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.100+0000 7fd3affff700 1 -- 172.21.15.7:0/4053646310 >> v1:172.21.15.7:6789/0 conn(0x7fd3b00cbbc0 legacy=0x7fd3b015ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:51.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.100+0000 7fd3affff700 1 -- 172.21.15.7:0/4053646310 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b015e410 con 0x7fd3b00c8e80 2024-01-27T02:32:51.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.100+0000 7fd3b4a02700 1 --2- 172.21.15.7:0/4053646310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3b00c8ab0 0x7fd3b015dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:32:51.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.100+0000 7fd3affff700 1 --2- 172.21.15.7:0/4053646310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3b00c8e80 0x7fd3b00cd4d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0098e0 tx=0x7fd39c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be337ea854b5a67f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:51.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.100+0000 7fd3af7fe700 1 -- 172.21.15.7:0/4053646310 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c011630 con 0x7fd3b00c8e80 2024-01-27T02:32:51.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.100+0000 7fd3af7fe700 1 -- 172.21.15.7:0/4053646310 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39c011790 con 0x7fd3b00c8e80 2024-01-27T02:32:51.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.100+0000 7fd3af7fe700 1 -- 172.21.15.7:0/4053646310 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c011b30 con 0x7fd3b00c8e80 2024-01-27T02:32:51.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.101+0000 7fd3b6c66700 1 -- 172.21.15.7:0/4053646310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3b00c8e80 msgr2=0x7fd3b00cd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:51.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.101+0000 7fd3b6c66700 1 --2- 172.21.15.7:0/4053646310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3b00c8e80 0x7fd3b00cd4d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0098e0 tx=0x7fd39c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.101+0000 7fd3b6c66700 1 -- 172.21.15.7:0/4053646310 shutdown_connections 2024-01-27T02:32:51.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.101+0000 7fd3b6c66700 1 --2- 172.21.15.7:0/4053646310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3b00c8ab0 0x7fd3b015dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.101+0000 7fd3b6c66700 1 --2- 172.21.15.7:0/4053646310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3b00c8e80 0x7fd3b00cd4d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.101+0000 7fd3b6c66700 1 -- 172.21.15.7:0/4053646310 >> 172.21.15.7:0/4053646310 conn(0x7fd3b00cdc30 msgr2=0x7fd3b00ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:51.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.101+0000 7fd3b6c66700 1 -- 172.21.15.7:0/4053646310 shutdown_connections 2024-01-27T02:32:51.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.101+0000 7fd3b6c66700 1 -- 172.21.15.7:0/4053646310 wait complete. 2024-01-27T02:32:51.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.102+0000 7fd3b6c66700 1 Processor -- start 2024-01-27T02:32:51.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.102+0000 7fd3b6c66700 1 -- start start 2024-01-27T02:32:51.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.103+0000 7fd3b6c66700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3b00c8ab0 0x7fd3b006c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:51.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.103+0000 7fd3b6c66700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3b00c8e80 0x7fd3b0156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:51.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.103+0000 7fd3b6c66700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3b00cbbc0 0x7fd3b006b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:51.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.103+0000 7fd3b6c66700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b006c3a0 con 0x7fd3b00c8e80 2024-01-27T02:32:51.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.103+0000 7fd3b6c66700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b0159520 con 0x7fd3b00cbbc0 2024-01-27T02:32:51.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.103+0000 7fd3b6c66700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b0159800 con 0x7fd3b00c8ab0 2024-01-27T02:32:51.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.103+0000 7fd3affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3b00c8e80 0x7fd3b0156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:51.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.103+0000 7fd3affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3b00c8e80 0x7fd3b0156290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50484/0 (socket says 172.21.15.7:50484) 2024-01-27T02:32:51.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.103+0000 7fd3affff700 1 -- 172.21.15.7:0/3861557238 learned_addr learned my addr 172.21.15.7:0/3861557238 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:51.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.103+0000 7fd3b4a02700 1 --2- 172.21.15.7:0/3861557238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3b00c8ab0 0x7fd3b006c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:51.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.103+0000 7fd3b5203700 1 --2- 172.21.15.7:0/3861557238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3b00cbbc0 0x7fd3b006b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:51.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.104+0000 7fd3affff700 1 -- 172.21.15.7:0/3861557238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3b00c8ab0 msgr2=0x7fd3b006c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:51.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.104+0000 7fd3affff700 1 --2- 172.21.15.7:0/3861557238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3b00c8ab0 0x7fd3b006c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.104+0000 7fd3affff700 1 -- 172.21.15.7:0/3861557238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3b00cbbc0 msgr2=0x7fd3b006b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:51.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.104+0000 7fd3affff700 1 --2- 172.21.15.7:0/3861557238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3b00cbbc0 0x7fd3b006b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.104+0000 7fd3affff700 1 -- 172.21.15.7:0/3861557238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a4009420 con 0x7fd3b00c8e80 2024-01-27T02:32:51.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.104+0000 7fd3affff700 1 --2- 172.21.15.7:0/3861557238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3b00c8e80 0x7fd3b0156290 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd3b006dfa0 tx=0x7fd39c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:51.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.104+0000 7fd3acff9700 1 -- 172.21.15.7:0/3861557238 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c011b10 con 0x7fd3b00c8e80 2024-01-27T02:32:51.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.104+0000 7fd3acff9700 1 -- 172.21.15.7:0/3861557238 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39c0253f0 con 0x7fd3b00c8e80 2024-01-27T02:32:51.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.104+0000 7fd3b6c66700 1 -- 172.21.15.7:0/3861557238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39c009420 con 0x7fd3b00c8e80 2024-01-27T02:32:51.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.104+0000 7fd3acff9700 1 -- 172.21.15.7:0/3861557238 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c0257b0 con 0x7fd3b00c8e80 2024-01-27T02:32:51.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.104+0000 7fd3b6c66700 1 -- 172.21.15.7:0/3861557238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b0159dc0 con 0x7fd3b00c8e80 2024-01-27T02:32:51.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.105+0000 7fd3b6c66700 1 -- 172.21.15.7:0/3861557238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b0159fb0 con 0x7fd3b00c8e80 2024-01-27T02:32:51.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.105+0000 7fd3acff9700 1 -- 172.21.15.7:0/3861557238 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd39c025ba0 con 0x7fd3b00c8e80 2024-01-27T02:32:51.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.106+0000 7fd3acff9700 1 --2- 172.21.15.7:0/3861557238 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd394066f30 0x7fd3940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:51.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.106+0000 7fd3acff9700 1 -- 172.21.15.7:0/3861557238 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd39c085b20 con 0x7fd3b00c8e80 2024-01-27T02:32:51.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.106+0000 7fd3b4a02700 1 --2- 172.21.15.7:0/3861557238 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd394066f30 0x7fd3940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:51.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.107+0000 7fd3b4a02700 1 --2- 172.21.15.7:0/3861557238 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd394066f30 0x7fd3940693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009d60 tx=0x7fd3a4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:51.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.110+0000 7fd3acff9700 1 -- 172.21.15.7:0/3861557238 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd39c0530e0 con 0x7fd3b00c8e80 2024-01-27T02:32:51.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.397+0000 7fd3b6c66700 1 -- 172.21.15.7:0/3861557238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fd3b004c710 con 0x7fd3b00c8e80 2024-01-27T02:32:51.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.401+0000 7fd3acff9700 1 -- 172.21.15.7:0/3861557238 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd39c056700 con 0x7fd3b00c8e80 2024-01-27T02:32:51.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.403+0000 7fd3b6c66700 1 -- 172.21.15.7:0/3861557238 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd394066f30 msgr2=0x7fd3940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:51.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.403+0000 7fd3b6c66700 1 --2- 172.21.15.7:0/3861557238 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd394066f30 0x7fd3940693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009d60 tx=0x7fd3a4008040 comp rx=0 tx=0).stop 2024-01-27T02:32:51.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.403+0000 7fd3b6c66700 1 -- 172.21.15.7:0/3861557238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3b00c8e80 msgr2=0x7fd3b0156290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:51.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.403+0000 7fd3b6c66700 1 --2- 172.21.15.7:0/3861557238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3b00c8e80 0x7fd3b0156290 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd3b006dfa0 tx=0x7fd39c017000 comp rx=0 tx=0).stop 2024-01-27T02:32:51.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.403+0000 7fd3b6c66700 1 -- 172.21.15.7:0/3861557238 shutdown_connections 2024-01-27T02:32:51.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.403+0000 7fd3b6c66700 1 --2- 172.21.15.7:0/3861557238 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd394066f30 0x7fd3940693e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.403+0000 7fd3b6c66700 1 --2- 172.21.15.7:0/3861557238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3b00cbbc0 0x7fd3b006b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.403+0000 7fd3b6c66700 1 --2- 172.21.15.7:0/3861557238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3b00c8ab0 0x7fd3b006c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.403+0000 7fd3b6c66700 1 --2- 172.21.15.7:0/3861557238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3b00c8e80 0x7fd3b0156290 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.403+0000 7fd3b6c66700 1 -- 172.21.15.7:0/3861557238 >> 172.21.15.7:0/3861557238 conn(0x7fd3b00cdc30 msgr2=0x7fd3b00c3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:51.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.403+0000 7fd3b6c66700 1 -- 172.21.15.7:0/3861557238 shutdown_connections 2024-01-27T02:32:51.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.403+0000 7fd3b6c66700 1 -- 172.21.15.7:0/3861557238 wait complete. 2024-01-27T02:32:51.417 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:32:51.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.549+0000 7f18ba1d9700 1 Processor -- start 2024-01-27T02:32:51.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.549+0000 7f18ba1d9700 1 -- start start 2024-01-27T02:32:51.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.550+0000 7f18ba1d9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18b4157ca0 0x7f18b40c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:51.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.550+0000 7f18ba1d9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18b40cdbf0 0x7f18b40ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:51.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.550+0000 7f18ba1d9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b40ca020 con 0x7f18b40c3ae0 2024-01-27T02:32:51.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.550+0000 7f18ba1d9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b40c4f40 con 0x7f18b4157ca0 2024-01-27T02:32:51.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.550+0000 7f18ba1d9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b40c5210 con 0x7f18b40cdbf0 2024-01-27T02:32:51.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.550+0000 7f18b37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18b4157ca0 0x7f18b40c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:51.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.550+0000 7f18b2ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f18b40c3ae0 0x7f18b40cd4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49142/0 (socket says 172.21.15.7:49142) 2024-01-27T02:32:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.550+0000 7f18b2ffd700 1 -- 172.21.15.7:0/3031136609 learned_addr learned my addr 172.21.15.7:0/3031136609 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.550+0000 7f18b3fff700 1 --2- 172.21.15.7:0/3031136609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18b40cdbf0 0x7f18b40ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.550+0000 7f18b37fe700 1 -- 172.21.15.7:0/3031136609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18b40cdbf0 msgr2=0x7f18b40ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.550+0000 7f18b37fe700 1 --2- 172.21.15.7:0/3031136609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18b40cdbf0 0x7f18b40ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.550+0000 7f18b37fe700 1 -- 172.21.15.7:0/3031136609 >> v1:172.21.15.7:6789/0 conn(0x7f18b40c3ae0 legacy=0x7f18b40cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.550+0000 7f18b37fe700 1 -- 172.21.15.7:0/3031136609 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b40c54f0 con 0x7f18b4157ca0 2024-01-27T02:32:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.550+0000 7f18b27fc700 1 -- 172.21.15.7:0/3031136609 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3642140720 0 0) 0x7f18b40ca020 con 0x7f18b40c3ae0 2024-01-27T02:32:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.550+0000 7f18b37fe700 1 --2- 172.21.15.7:0/3031136609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b4157ca0 0x7f18b40c4a00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009e60 tx=0x7f18a4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3427fc33099db88a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.551+0000 7f18b27fc700 1 -- 172.21.15.7:0/3031136609 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a400e070 con 0x7f18b4157ca0 2024-01-27T02:32:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.551+0000 7f18b27fc700 1 -- 172.21.15.7:0/3031136609 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a400bc30 con 0x7f18b4157ca0 2024-01-27T02:32:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.551+0000 7f18b27fc700 1 -- 172.21.15.7:0/3031136609 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4012430 con 0x7f18b4157ca0 2024-01-27T02:32:51.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.551+0000 7f18ba1d9700 1 -- 172.21.15.7:0/3031136609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b4157ca0 msgr2=0x7f18b40c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:51.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.551+0000 7f18ba1d9700 1 --2- 172.21.15.7:0/3031136609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b4157ca0 0x7f18b40c4a00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009e60 tx=0x7f18a4009780 comp rx=0 tx=0).stop 2024-01-27T02:32:51.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.551+0000 7f18ba1d9700 1 -- 172.21.15.7:0/3031136609 shutdown_connections 2024-01-27T02:32:51.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.551+0000 7f18ba1d9700 1 --2- 172.21.15.7:0/3031136609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18b40cdbf0 0x7f18b40ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.551+0000 7f18ba1d9700 1 --2- 172.21.15.7:0/3031136609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b4157ca0 0x7f18b40c4a00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.551+0000 7f18ba1d9700 1 -- 172.21.15.7:0/3031136609 >> 172.21.15.7:0/3031136609 conn(0x7f18b4098020 msgr2=0x7f18b40711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:51.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.551+0000 7f18ba1d9700 1 -- 172.21.15.7:0/3031136609 shutdown_connections 2024-01-27T02:32:51.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.551+0000 7f18ba1d9700 1 -- 172.21.15.7:0/3031136609 wait complete. 2024-01-27T02:32:51.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.553+0000 7f18ba1d9700 1 Processor -- start 2024-01-27T02:32:51.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.553+0000 7f18ba1d9700 1 -- start start 2024-01-27T02:32:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.554+0000 7f18ba1d9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b40c3ae0 0x7f18b40c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.554+0000 7f18ba1d9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18b40cdbf0 0x7f18b4145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.554+0000 7f18ba1d9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18b4157ca0 0x7f18b40b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.554+0000 7f18ba1d9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b40679b0 con 0x7f18b40c3ae0 2024-01-27T02:32:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.554+0000 7f18ba1d9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b414ec30 con 0x7f18b40cdbf0 2024-01-27T02:32:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.554+0000 7f18ba1d9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b40684f0 con 0x7f18b4157ca0 2024-01-27T02:32:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.554+0000 7f18b37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b40c3ae0 0x7f18b40c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.554+0000 7f18b37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b40c3ae0 0x7f18b40c5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38530/0 (socket says 172.21.15.7:38530) 2024-01-27T02:32:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.554+0000 7f18b37fe700 1 -- 172.21.15.7:0/3852133595 learned_addr learned my addr 172.21.15.7:0/3852133595 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.554+0000 7f18b2ffd700 1 --2- 172.21.15.7:0/3852133595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18b40cdbf0 0x7f18b4145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.554+0000 7f18b3fff700 1 --2- 172.21.15.7:0/3852133595 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18b4157ca0 0x7f18b40b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.554+0000 7f18b37fe700 1 -- 172.21.15.7:0/3852133595 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18b4157ca0 msgr2=0x7f18b40b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.554+0000 7f18b37fe700 1 --2- 172.21.15.7:0/3852133595 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18b4157ca0 0x7f18b40b2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.554+0000 7f18b37fe700 1 -- 172.21.15.7:0/3852133595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18b40cdbf0 msgr2=0x7f18b4145750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.554+0000 7f18b37fe700 1 --2- 172.21.15.7:0/3852133595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18b40cdbf0 0x7f18b4145750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.554+0000 7f18b37fe700 1 -- 172.21.15.7:0/3852133595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a4009a20 con 0x7f18b40c3ae0 2024-01-27T02:32:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.554+0000 7f18b37fe700 1 --2- 172.21.15.7:0/3852133595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b40c3ae0 0x7f18b40c5df0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f18b406ea80 tx=0x7f18a4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.554+0000 7f189b7fe700 1 -- 172.21.15.7:0/3852133595 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4027470 con 0x7f18b40c3ae0 2024-01-27T02:32:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.554+0000 7f189b7fe700 1 -- 172.21.15.7:0/3852133595 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a40275f0 con 0x7f18b40c3ae0 2024-01-27T02:32:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.554+0000 7f18ba1d9700 1 -- 172.21.15.7:0/3852133595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b4067bc0 con 0x7f18b40c3ae0 2024-01-27T02:32:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.554+0000 7f189b7fe700 1 -- 172.21.15.7:0/3852133595 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a40279b0 con 0x7f18b40c3ae0 2024-01-27T02:32:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.554+0000 7f18ba1d9700 1 -- 172.21.15.7:0/3852133595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b4068180 con 0x7f18b40c3ae0 2024-01-27T02:32:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.555+0000 7f18ba1d9700 1 -- 172.21.15.7:0/3852133595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b404c710 con 0x7f18b40c3ae0 2024-01-27T02:32:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.556+0000 7f189b7fe700 1 -- 172.21.15.7:0/3852133595 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f18a4024070 con 0x7f18b40c3ae0 2024-01-27T02:32:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.556+0000 7f189b7fe700 1 --2- 172.21.15.7:0/3852133595 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f18a0066f30 0x7f18a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.556+0000 7f189b7fe700 1 -- 172.21.15.7:0/3852133595 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f18a4086c10 con 0x7f18b40c3ae0 2024-01-27T02:32:51.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.556+0000 7f18b2ffd700 1 --2- 172.21.15.7:0/3852133595 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f18a0066f30 0x7f18a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:51.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.557+0000 7f18b2ffd700 1 --2- 172.21.15.7:0/3852133595 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f18a0066f30 0x7f18a00693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f18a80011e0 tx=0x7f18a8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.561+0000 7f189b7fe700 1 -- 172.21.15.7:0/3852133595 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18a40541d0 con 0x7f18b40c3ae0 2024-01-27T02:32:51.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.765+0000 7f18ba1d9700 1 -- 172.21.15.7:0/3852133595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18b40c57d0 con 0x7f18b40c3ae0 2024-01-27T02:32:51.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.766+0000 7f189b7fe700 1 -- 172.21.15.7:0/3852133595 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+12086 (secure 0 0 0) 0x7f18a40577f0 con 0x7f18b40c3ae0 2024-01-27T02:32:51.767 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:51.767 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:32:48.243040+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:32:51.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.768+0000 7f18ba1d9700 1 -- 172.21.15.7:0/3852133595 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f18a0066f30 msgr2=0x7f18a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:51.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.768+0000 7f18ba1d9700 1 --2- 172.21.15.7:0/3852133595 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f18a0066f30 0x7f18a00693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f18a80011e0 tx=0x7f18a8009040 comp rx=0 tx=0).stop 2024-01-27T02:32:51.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.768+0000 7f18ba1d9700 1 -- 172.21.15.7:0/3852133595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b40c3ae0 msgr2=0x7f18b40c5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:51.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.768+0000 7f18ba1d9700 1 --2- 172.21.15.7:0/3852133595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b40c3ae0 0x7f18b40c5df0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f18b406ea80 tx=0x7f18a4019000 comp rx=0 tx=0).stop 2024-01-27T02:32:51.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.768+0000 7f18ba1d9700 1 -- 172.21.15.7:0/3852133595 shutdown_connections 2024-01-27T02:32:51.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.768+0000 7f18ba1d9700 1 --2- 172.21.15.7:0/3852133595 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f18a0066f30 0x7f18a00693e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.768+0000 7f18ba1d9700 1 --2- 172.21.15.7:0/3852133595 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18b4157ca0 0x7f18b40b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.768+0000 7f18ba1d9700 1 --2- 172.21.15.7:0/3852133595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18b40c3ae0 0x7f18b40c5df0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.768+0000 7f18ba1d9700 1 --2- 172.21.15.7:0/3852133595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18b40cdbf0 0x7f18b4145750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.768+0000 7f18ba1d9700 1 -- 172.21.15.7:0/3852133595 >> 172.21.15.7:0/3852133595 conn(0x7f18b4098020 msgr2=0x7f18b41535d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:51.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.768+0000 7f18ba1d9700 1 -- 172.21.15.7:0/3852133595 shutdown_connections 2024-01-27T02:32:51.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.768+0000 7f18ba1d9700 1 -- 172.21.15.7:0/3852133595 wait complete. 2024-01-27T02:32:51.782 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-27T02:32:51.782 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:32:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.916+0000 7f5e253ce700 1 Processor -- start 2024-01-27T02:32:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.916+0000 7f5e253ce700 1 -- start start 2024-01-27T02:32:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.917+0000 7f5e253ce700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5e200cba60 0x7f5e200ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.917+0000 7f5e253ce700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e2006b4e0 0x7f5e2006b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.917+0000 7f5e253ce700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e200679b0 con 0x7f5e200cbe30 2024-01-27T02:32:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.917+0000 7f5e253ce700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e20067c90 con 0x7f5e200cba60 2024-01-27T02:32:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.917+0000 7f5e253ce700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2006be60 con 0x7f5e2006b4e0 2024-01-27T02:32:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.917+0000 7f5e1effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5e200cba60 0x7f5e200ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.918+0000 7f5e1effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5e200cba60 0x7f5e200ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38538/0 (socket says 172.21.15.7:38538) 2024-01-27T02:32:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.918+0000 7f5e1effd700 1 -- 172.21.15.7:0/741423284 learned_addr learned my addr 172.21.15.7:0/741423284 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.918+0000 7f5e1e7fc700 1 --1- 172.21.15.7:0/741423284 >> v1:172.21.15.188:6789/0 conn(0x7f5e200cbe30 0x7f5e2015b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:34240/0 (socket says 172.21.15.7:34240) 2024-01-27T02:32:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.918+0000 7f5e1f7fe700 1 --2- 172.21.15.7:0/741423284 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e2006b4e0 0x7f5e2006b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.918+0000 7f5e1effd700 1 -- 172.21.15.7:0/741423284 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e2006b4e0 msgr2=0x7f5e2006b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.918+0000 7f5e1effd700 1 --2- 172.21.15.7:0/741423284 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e2006b4e0 0x7f5e2006b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.918+0000 7f5e1effd700 1 -- 172.21.15.7:0/741423284 >> v1:172.21.15.188:6789/0 conn(0x7f5e200cbe30 legacy=0x7f5e2015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.918+0000 7f5e1effd700 1 -- 172.21.15.7:0/741423284 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e200cca50 con 0x7f5e200cba60 2024-01-27T02:32:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.918+0000 7f5e1effd700 1 --2- 172.21.15.7:0/741423284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e200cba60 0x7f5e200ce410 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009990 tx=0x7f5e10009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04f5071ad1aab0f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.918+0000 7f5e1dffb700 1 -- 172.21.15.7:0/741423284 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10009e80 con 0x7f5e200cba60 2024-01-27T02:32:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.918+0000 7f5e1dffb700 1 -- 172.21.15.7:0/741423284 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e10005c10 con 0x7f5e200cba60 2024-01-27T02:32:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.918+0000 7f5e1dffb700 1 -- 172.21.15.7:0/741423284 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1000b650 con 0x7f5e200cba60 2024-01-27T02:32:51.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.919+0000 7f5e253ce700 1 -- 172.21.15.7:0/741423284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e200cba60 msgr2=0x7f5e200ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:51.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.919+0000 7f5e253ce700 1 --2- 172.21.15.7:0/741423284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e200cba60 0x7f5e200ce410 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009990 tx=0x7f5e10009220 comp rx=0 tx=0).stop 2024-01-27T02:32:51.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.919+0000 7f5e253ce700 1 -- 172.21.15.7:0/741423284 shutdown_connections 2024-01-27T02:32:51.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.919+0000 7f5e253ce700 1 --2- 172.21.15.7:0/741423284 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e2006b4e0 0x7f5e2006b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.919+0000 7f5e253ce700 1 --2- 172.21.15.7:0/741423284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e200cba60 0x7f5e200ce410 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.919+0000 7f5e253ce700 1 -- 172.21.15.7:0/741423284 >> 172.21.15.7:0/741423284 conn(0x7f5e20063580 msgr2=0x7f5e20063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:51.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.919+0000 7f5e253ce700 1 -- 172.21.15.7:0/741423284 shutdown_connections 2024-01-27T02:32:51.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.919+0000 7f5e253ce700 1 -- 172.21.15.7:0/741423284 wait complete. 2024-01-27T02:32:51.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.920+0000 7f5e253ce700 1 Processor -- start 2024-01-27T02:32:51.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.920+0000 7f5e253ce700 1 -- start start 2024-01-27T02:32:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.921+0000 7f5e253ce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e2006b4e0 0x7f5e20155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.921+0000 7f5e253ce700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e200cba60 0x7f5e200b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.921+0000 7f5e253ce700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e200cbe30 0x7f5e2015cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.921+0000 7f5e253ce700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e200b1ff0 con 0x7f5e2006b4e0 2024-01-27T02:32:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.921+0000 7f5e253ce700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e200b22d0 con 0x7f5e200cba60 2024-01-27T02:32:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.921+0000 7f5e253ce700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e200b25b0 con 0x7f5e200cbe30 2024-01-27T02:32:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.921+0000 7f5e1effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e2006b4e0 0x7f5e20155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.921+0000 7f5e1effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e2006b4e0 0x7f5e20155e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38544/0 (socket says 172.21.15.7:38544) 2024-01-27T02:32:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.921+0000 7f5e1effd700 1 -- 172.21.15.7:0/3937768628 learned_addr learned my addr 172.21.15.7:0/3937768628 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.921+0000 7f5e1f7fe700 1 --2- 172.21.15.7:0/3937768628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e200cbe30 0x7f5e2015cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.921+0000 7f5e1e7fc700 1 --2- 172.21.15.7:0/3937768628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e200cba60 0x7f5e200b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.922+0000 7f5e1effd700 1 -- 172.21.15.7:0/3937768628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e200cbe30 msgr2=0x7f5e2015cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.922+0000 7f5e1effd700 1 --2- 172.21.15.7:0/3937768628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e200cbe30 0x7f5e2015cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.922+0000 7f5e1effd700 1 -- 172.21.15.7:0/3937768628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e200cba60 msgr2=0x7f5e200b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.922+0000 7f5e1effd700 1 --2- 172.21.15.7:0/3937768628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e200cba60 0x7f5e200b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.922+0000 7f5e1effd700 1 -- 172.21.15.7:0/3937768628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e10009480 con 0x7f5e2006b4e0 2024-01-27T02:32:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.922+0000 7f5e1effd700 1 --2- 172.21.15.7:0/3937768628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e2006b4e0 0x7f5e20155e40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000b3c0 tx=0x7f5e10016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.922+0000 7f5e16ffd700 1 -- 172.21.15.7:0/3937768628 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10011980 con 0x7f5e2006b4e0 2024-01-27T02:32:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.922+0000 7f5e16ffd700 1 -- 172.21.15.7:0/3937768628 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e10020070 con 0x7f5e2006b4e0 2024-01-27T02:32:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.922+0000 7f5e253ce700 1 -- 172.21.15.7:0/3937768628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e200b2890 con 0x7f5e2006b4e0 2024-01-27T02:32:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.922+0000 7f5e16ffd700 1 -- 172.21.15.7:0/3937768628 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10011d10 con 0x7f5e2006b4e0 2024-01-27T02:32:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.922+0000 7f5e253ce700 1 -- 172.21.15.7:0/3937768628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e20156380 con 0x7f5e2006b4e0 2024-01-27T02:32:51.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.923+0000 7f5e253ce700 1 -- 172.21.15.7:0/3937768628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e200b2db0 con 0x7f5e2006b4e0 2024-01-27T02:32:51.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.924+0000 7f5e16ffd700 1 -- 172.21.15.7:0/3937768628 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5e10022030 con 0x7f5e2006b4e0 2024-01-27T02:32:51.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.924+0000 7f5e16ffd700 1 --2- 172.21.15.7:0/3937768628 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5e00066f30 0x7f5e000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:51.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.924+0000 7f5e16ffd700 1 -- 172.21.15.7:0/3937768628 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5e10085cc0 con 0x7f5e2006b4e0 2024-01-27T02:32:51.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.924+0000 7f5e1e7fc700 1 --2- 172.21.15.7:0/3937768628 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5e00066f30 0x7f5e000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:51.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.925+0000 7f5e1e7fc700 1 --2- 172.21.15.7:0/3937768628 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5e00066f30 0x7f5e000693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5e2006cf40 tx=0x7f5e08007eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:51.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:51.928+0000 7f5e16ffd700 1 -- 172.21.15.7:0/3937768628 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e200b2db0 con 0x7f5e2006b4e0 2024-01-27T02:32:52.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.131+0000 7f5e253ce700 1 -- 172.21.15.7:0/3937768628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5e200b2db0 con 0x7f5e2006b4e0 2024-01-27T02:32:52.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.132+0000 7f5e16ffd700 1 -- 172.21.15.7:0/3937768628 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5e200b2db0 con 0x7f5e2006b4e0 2024-01-27T02:32:52.132 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:32:52.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.134+0000 7f5e253ce700 1 -- 172.21.15.7:0/3937768628 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5e00066f30 msgr2=0x7f5e000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:52.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.134+0000 7f5e253ce700 1 --2- 172.21.15.7:0/3937768628 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5e00066f30 0x7f5e000693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5e2006cf40 tx=0x7f5e08007eb0 comp rx=0 tx=0).stop 2024-01-27T02:32:52.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.134+0000 7f5e253ce700 1 -- 172.21.15.7:0/3937768628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e2006b4e0 msgr2=0x7f5e20155e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:52.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.134+0000 7f5e253ce700 1 --2- 172.21.15.7:0/3937768628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e2006b4e0 0x7f5e20155e40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000b3c0 tx=0x7f5e10016000 comp rx=0 tx=0).stop 2024-01-27T02:32:52.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.134+0000 7f5e253ce700 1 -- 172.21.15.7:0/3937768628 shutdown_connections 2024-01-27T02:32:52.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.134+0000 7f5e253ce700 1 --2- 172.21.15.7:0/3937768628 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5e00066f30 0x7f5e000693e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:52.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.134+0000 7f5e253ce700 1 --2- 172.21.15.7:0/3937768628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e200cbe30 0x7f5e2015cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:52.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.134+0000 7f5e253ce700 1 --2- 172.21.15.7:0/3937768628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e2006b4e0 0x7f5e20155e40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:52.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.134+0000 7f5e253ce700 1 --2- 172.21.15.7:0/3937768628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e200cba60 0x7f5e200b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:52.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.134+0000 7f5e253ce700 1 -- 172.21.15.7:0/3937768628 >> 172.21.15.7:0/3937768628 conn(0x7f5e20063580 msgr2=0x7f5e20069490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:52.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.134+0000 7f5e253ce700 1 -- 172.21.15.7:0/3937768628 shutdown_connections 2024-01-27T02:32:52.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.134+0000 7f5e253ce700 1 -- 172.21.15.7:0/3937768628 wait complete. 2024-01-27T02:32:52.148 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-27T02:32:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.279+0000 7f9ee2337700 1 Processor -- start 2024-01-27T02:32:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.279+0000 7f9ee2337700 1 -- start start 2024-01-27T02:32:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.280+0000 7f9ee2337700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9edc0c4cc0 0x7f9edc0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.280+0000 7f9ee2337700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9edc0ca540 0x7f9edc163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.280+0000 7f9ee2337700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9edc0ce5e0 con 0x7f9edc15a100 2024-01-27T02:32:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.280+0000 7f9ee2337700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9edc0cd9d0 con 0x7f9edc0c4cc0 2024-01-27T02:32:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.280+0000 7f9ee2337700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9edc0cdcb0 con 0x7f9edc0ca540 2024-01-27T02:32:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.280+0000 7f9edb7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9edc0c4cc0 0x7f9edc0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.280+0000 7f9edbfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9edc15a100 0x7f9edc0c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49154/0 (socket says 172.21.15.7:49154) 2024-01-27T02:32:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.280+0000 7f9edbfff700 1 -- 172.21.15.7:0/4027262773 learned_addr learned my addr 172.21.15.7:0/4027262773 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.280+0000 7f9ee08d4700 1 --2- 172.21.15.7:0/4027262773 >> v2:172.21.15.188:3300/0 conn(0x7f9edc0ca540 0x7f9edc163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.280+0000 7f9edb7fe700 1 -- 172.21.15.7:0/4027262773 >> v2:172.21.15.188:3300/0 conn(0x7f9edc0ca540 msgr2=0x7f9edc163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.281+0000 7f9edb7fe700 1 --2- 172.21.15.7:0/4027262773 >> v2:172.21.15.188:3300/0 conn(0x7f9edc0ca540 0x7f9edc163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.281+0000 7f9edb7fe700 1 -- 172.21.15.7:0/4027262773 >> v1:172.21.15.7:6789/0 conn(0x7f9edc15a100 legacy=0x7f9edc0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.281+0000 7f9edaffd700 1 -- 172.21.15.7:0/4027262773 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686487922 0 0) 0x7f9edc0ce5e0 con 0x7f9edc15a100 2024-01-27T02:32:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.281+0000 7f9edb7fe700 1 -- 172.21.15.7:0/4027262773 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9edc0cdf90 con 0x7f9edc0c4cc0 2024-01-27T02:32:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.281+0000 7f9edb7fe700 1 --2- 172.21.15.7:0/4027262773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9edc0c4cc0 0x7f9edc0cd2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9edc069b70 tx=0x7f9ed00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7600bb42fc0b2ba7 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.281+0000 7f9edaffd700 1 -- 172.21.15.7:0/4027262773 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed000b6b0 con 0x7f9edc0c4cc0 2024-01-27T02:32:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.281+0000 7f9edaffd700 1 -- 172.21.15.7:0/4027262773 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed000b810 con 0x7f9edc0c4cc0 2024-01-27T02:32:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.281+0000 7f9edaffd700 1 -- 172.21.15.7:0/4027262773 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed000bbb0 con 0x7f9edc0c4cc0 2024-01-27T02:32:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.281+0000 7f9ee2337700 1 -- 172.21.15.7:0/4027262773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9edc0c4cc0 msgr2=0x7f9edc0cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.281+0000 7f9ee2337700 1 --2- 172.21.15.7:0/4027262773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9edc0c4cc0 0x7f9edc0cd2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9edc069b70 tx=0x7f9ed00091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.282+0000 7f9ee2337700 1 -- 172.21.15.7:0/4027262773 shutdown_connections 2024-01-27T02:32:52.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.282+0000 7f9ee2337700 1 --2- 172.21.15.7:0/4027262773 >> v2:172.21.15.188:3300/0 conn(0x7f9edc0ca540 0x7f9edc163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:52.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.282+0000 7f9ee2337700 1 --2- 172.21.15.7:0/4027262773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9edc0c4cc0 0x7f9edc0cd2f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:52.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.282+0000 7f9ee2337700 1 -- 172.21.15.7:0/4027262773 >> 172.21.15.7:0/4027262773 conn(0x7f9edc098020 msgr2=0x7f9edc0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:52.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.282+0000 7f9ee2337700 1 -- 172.21.15.7:0/4027262773 shutdown_connections 2024-01-27T02:32:52.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.282+0000 7f9ee2337700 1 -- 172.21.15.7:0/4027262773 wait complete. 2024-01-27T02:32:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.283+0000 7f9ee2337700 1 Processor -- start 2024-01-27T02:32:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.283+0000 7f9ee2337700 1 -- start start 2024-01-27T02:32:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.284+0000 7f9ee2337700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9edc0c4cc0 0x7f9edc15b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.284+0000 7f9ee2337700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9edc0ca540 0x7f9edc15bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.284+0000 7f9ee2337700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9edc15a100 0x7f9edc067c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.284+0000 7f9ee2337700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc0b24c0 con 0x7f9edc0c4cc0 2024-01-27T02:32:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.284+0000 7f9ee2337700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc15d000 con 0x7f9edc0ca540 2024-01-27T02:32:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.284+0000 7f9ee2337700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc15c3f0 con 0x7f9edc15a100 2024-01-27T02:32:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.284+0000 7f9edbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9edc0c4cc0 0x7f9edc15b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.284+0000 7f9edbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9edc0c4cc0 0x7f9edc15b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38556/0 (socket says 172.21.15.7:38556) 2024-01-27T02:32:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.284+0000 7f9edbfff700 1 -- 172.21.15.7:0/1338088544 learned_addr learned my addr 172.21.15.7:0/1338088544 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.284+0000 7f9edb7fe700 1 --2- 172.21.15.7:0/1338088544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9edc0ca540 0x7f9edc15bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.284+0000 7f9ee08d4700 1 --2- 172.21.15.7:0/1338088544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9edc15a100 0x7f9edc067c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.284+0000 7f9edbfff700 1 -- 172.21.15.7:0/1338088544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9edc15a100 msgr2=0x7f9edc067c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.284+0000 7f9edbfff700 1 --2- 172.21.15.7:0/1338088544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9edc15a100 0x7f9edc067c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.284+0000 7f9edbfff700 1 -- 172.21.15.7:0/1338088544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9edc0ca540 msgr2=0x7f9edc15bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.284+0000 7f9edbfff700 1 --2- 172.21.15.7:0/1338088544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9edc0ca540 0x7f9edc15bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.284+0000 7f9edbfff700 1 -- 172.21.15.7:0/1338088544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed0009420 con 0x7f9edc0c4cc0 2024-01-27T02:32:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.284+0000 7f9ee08d4700 1 --2- 172.21.15.7:0/1338088544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9edc15a100 0x7f9edc067c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:32:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.284+0000 7f9edbfff700 1 --2- 172.21.15.7:0/1338088544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9edc0c4cc0 0x7f9edc15b810 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00c5b0 tx=0x7f9ecc001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.285+0000 7f9ec3fff700 1 -- 172.21.15.7:0/1338088544 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc005b30 con 0x7f9edc0c4cc0 2024-01-27T02:32:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.285+0000 7f9ec3fff700 1 -- 172.21.15.7:0/1338088544 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ecc005c90 con 0x7f9edc0c4cc0 2024-01-27T02:32:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.285+0000 7f9ee2337700 1 -- 172.21.15.7:0/1338088544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9edc15c6d0 con 0x7f9edc0c4cc0 2024-01-27T02:32:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.285+0000 7f9ec3fff700 1 -- 172.21.15.7:0/1338088544 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc007320 con 0x7f9edc0c4cc0 2024-01-27T02:32:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.285+0000 7f9ee2337700 1 -- 172.21.15.7:0/1338088544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9edc15cc90 con 0x7f9edc0c4cc0 2024-01-27T02:32:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.286+0000 7f9ee2337700 1 -- 172.21.15.7:0/1338088544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9edc15d2e0 con 0x7f9edc0c4cc0 2024-01-27T02:32:52.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.286+0000 7f9ec3fff700 1 -- 172.21.15.7:0/1338088544 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9ecc01b400 con 0x7f9edc0c4cc0 2024-01-27T02:32:52.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.287+0000 7f9ec3fff700 1 --2- 172.21.15.7:0/1338088544 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9ec8066f30 0x7f9ec80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:52.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.287+0000 7f9ec3fff700 1 -- 172.21.15.7:0/1338088544 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9ecc083d60 con 0x7f9edc0c4cc0 2024-01-27T02:32:52.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.287+0000 7f9edb7fe700 1 --2- 172.21.15.7:0/1338088544 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9ec8066f30 0x7f9ec80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:52.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.287+0000 7f9edb7fe700 1 --2- 172.21.15.7:0/1338088544 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9ec8066f30 0x7f9ec80693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00093f0 tx=0x7f9ed0017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.291+0000 7f9ec3fff700 1 -- 172.21.15.7:0/1338088544 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ecc0513a0 con 0x7f9edc0c4cc0 2024-01-27T02:32:52.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.499+0000 7f9ee2337700 1 -- 172.21.15.7:0/1338088544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9edc04c710 con 0x7f9edc0c4cc0 2024-01-27T02:32:52.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.499+0000 7f9ec3fff700 1 -- 172.21.15.7:0/1338088544 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9ecc0549c0 con 0x7f9edc0c4cc0 2024-01-27T02:32:52.500 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:52.500 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-27T02:32:52.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:32:52.500 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-27T02:32:52.500 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-27T02:32:52.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:32:52.500 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-27T02:32:52.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.501+0000 7f9ee2337700 1 -- 172.21.15.7:0/1338088544 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9ec8066f30 msgr2=0x7f9ec80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:52.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.501+0000 7f9ee2337700 1 --2- 172.21.15.7:0/1338088544 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9ec8066f30 0x7f9ec80693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00093f0 tx=0x7f9ed0017000 comp rx=0 tx=0).stop 2024-01-27T02:32:52.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.501+0000 7f9ee2337700 1 -- 172.21.15.7:0/1338088544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9edc0c4cc0 msgr2=0x7f9edc15b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:52.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.501+0000 7f9ee2337700 1 --2- 172.21.15.7:0/1338088544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9edc0c4cc0 0x7f9edc15b810 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00c5b0 tx=0x7f9ecc001070 comp rx=0 tx=0).stop 2024-01-27T02:32:52.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.501+0000 7f9ee2337700 1 -- 172.21.15.7:0/1338088544 shutdown_connections 2024-01-27T02:32:52.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.501+0000 7f9ee2337700 1 --2- 172.21.15.7:0/1338088544 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9ec8066f30 0x7f9ec80693e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:52.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.501+0000 7f9ee2337700 1 --2- 172.21.15.7:0/1338088544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9edc15a100 0x7f9edc067c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:52.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.501+0000 7f9ee2337700 1 --2- 172.21.15.7:0/1338088544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9edc0c4cc0 0x7f9edc15b810 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:52.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.501+0000 7f9ee2337700 1 --2- 172.21.15.7:0/1338088544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9edc0ca540 0x7f9edc15bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:52.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.501+0000 7f9ee2337700 1 -- 172.21.15.7:0/1338088544 >> 172.21.15.7:0/1338088544 conn(0x7f9edc098020 msgr2=0x7f9edc06c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:52.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.501+0000 7f9ee2337700 1 -- 172.21.15.7:0/1338088544 shutdown_connections 2024-01-27T02:32:52.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.501+0000 7f9ee2337700 1 -- 172.21.15.7:0/1338088544 wait complete. 2024-01-27T02:32:52.516 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-27T02:32:52.516 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-27T02:32:52.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.651+0000 7ff19a8f2700 1 Processor -- start 2024-01-27T02:32:52.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.651+0000 7ff19a8f2700 1 -- start start 2024-01-27T02:32:52.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.652+0000 7ff19a8f2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff19415a020 0x7ff19415e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:52.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.652+0000 7ff19a8f2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff1940680c0 0x7ff1940c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:52.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.652+0000 7ff19a8f2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1940ce6a0 con 0x7ff19406bb30 2024-01-27T02:32:52.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.652+0000 7ff19a8f2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff19415ec30 con 0x7ff1940680c0 2024-01-27T02:32:52.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.652+0000 7ff19a8f2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1940ceb30 con 0x7ff19415a020 2024-01-27T02:32:52.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.652+0000 7ff18bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff19406bb30 0x7ff1940679b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49166/0 (socket says 172.21.15.7:49166) 2024-01-27T02:32:52.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.652+0000 7ff198e8f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff1940680c0 0x7ff1940c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:52.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.652+0000 7ff18bfff700 1 -- 172.21.15.7:0/3185797034 learned_addr learned my addr 172.21.15.7:0/3185797034 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:52.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.652+0000 7ff193fff700 1 --2- 172.21.15.7:0/3185797034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff19415a020 0x7ff19415e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:52.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.653+0000 7ff198e8f700 1 -- 172.21.15.7:0/3185797034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff19415a020 msgr2=0x7ff19415e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:52.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.653+0000 7ff198e8f700 1 --2- 172.21.15.7:0/3185797034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff19415a020 0x7ff19415e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:52.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.653+0000 7ff198e8f700 1 -- 172.21.15.7:0/3185797034 >> v1:172.21.15.7:6789/0 conn(0x7ff19406bb30 legacy=0x7ff1940679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:52.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.653+0000 7ff198e8f700 1 -- 172.21.15.7:0/3185797034 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff194069570 con 0x7ff1940680c0 2024-01-27T02:32:52.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.653+0000 7ff198e8f700 1 --2- 172.21.15.7:0/3185797034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1940680c0 0x7ff1940c5dd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009e60 tx=0x7ff18c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c77107227def2d server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:52.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.653+0000 7ff1937fe700 1 -- 172.21.15.7:0/3185797034 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c00e070 con 0x7ff1940680c0 2024-01-27T02:32:52.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.653+0000 7ff1937fe700 1 -- 172.21.15.7:0/3185797034 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18c00bc30 con 0x7ff1940680c0 2024-01-27T02:32:52.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.653+0000 7ff1937fe700 1 -- 172.21.15.7:0/3185797034 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c012430 con 0x7ff1940680c0 2024-01-27T02:32:52.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.654+0000 7ff19a8f2700 1 -- 172.21.15.7:0/3185797034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1940680c0 msgr2=0x7ff1940c5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:52.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.654+0000 7ff19a8f2700 1 --2- 172.21.15.7:0/3185797034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1940680c0 0x7ff1940c5dd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009e60 tx=0x7ff18c009780 comp rx=0 tx=0).stop 2024-01-27T02:32:52.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.654+0000 7ff19a8f2700 1 -- 172.21.15.7:0/3185797034 shutdown_connections 2024-01-27T02:32:52.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.654+0000 7ff19a8f2700 1 --2- 172.21.15.7:0/3185797034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1940680c0 0x7ff1940c5dd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:52.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.654+0000 7ff19a8f2700 1 --2- 172.21.15.7:0/3185797034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff19415a020 0x7ff19415e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:52.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.654+0000 7ff19a8f2700 1 -- 172.21.15.7:0/3185797034 >> 172.21.15.7:0/3185797034 conn(0x7ff1940714b0 msgr2=0x7ff19406b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:52.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.654+0000 7ff19a8f2700 1 -- 172.21.15.7:0/3185797034 shutdown_connections 2024-01-27T02:32:52.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.654+0000 7ff19a8f2700 1 -- 172.21.15.7:0/3185797034 wait complete. 2024-01-27T02:32:52.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.655+0000 7ff19a8f2700 1 Processor -- start 2024-01-27T02:32:52.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.655+0000 7ff19a8f2700 1 -- start start 2024-01-27T02:32:52.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.656+0000 7ff19a8f2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1940680c0 0x7ff19416e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:52.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.656+0000 7ff19a8f2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff19406bb30 0x7ff19415db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:52.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.656+0000 7ff19a8f2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff19415a020 0x7ff194156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:52.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.656+0000 7ff19a8f2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19415d6a0 con 0x7ff19415a020 2024-01-27T02:32:52.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.656+0000 7ff19a8f2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19416e7e0 con 0x7ff1940680c0 2024-01-27T02:32:52.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.656+0000 7ff19a8f2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19415ca90 con 0x7ff19406bb30 2024-01-27T02:32:52.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.656+0000 7ff198e8f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff19415a020 0x7ff194156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:52.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.656+0000 7ff198e8f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff19415a020 0x7ff194156470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38576/0 (socket says 172.21.15.7:38576) 2024-01-27T02:32:52.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.656+0000 7ff198e8f700 1 -- 172.21.15.7:0/3843822412 learned_addr learned my addr 172.21.15.7:0/3843822412 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:52.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.656+0000 7ff193fff700 1 --2- 172.21.15.7:0/3843822412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1940680c0 0x7ff19416e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.656+0000 7ff18bfff700 1 --2- 172.21.15.7:0/3843822412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff19406bb30 0x7ff19415db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.656+0000 7ff198e8f700 1 -- 172.21.15.7:0/3843822412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff19406bb30 msgr2=0x7ff19415db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.656+0000 7ff198e8f700 1 --2- 172.21.15.7:0/3843822412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff19406bb30 0x7ff19415db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.656+0000 7ff198e8f700 1 -- 172.21.15.7:0/3843822412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1940680c0 msgr2=0x7ff19416e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.656+0000 7ff198e8f700 1 --2- 172.21.15.7:0/3843822412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1940680c0 0x7ff19416e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.656+0000 7ff198e8f700 1 -- 172.21.15.7:0/3843822412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18c009a20 con 0x7ff19415a020 2024-01-27T02:32:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.656+0000 7ff198e8f700 1 --2- 172.21.15.7:0/3843822412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff19415a020 0x7ff194156470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff18c013000 tx=0x7ff18c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.656+0000 7ff190ff9700 1 -- 172.21.15.7:0/3843822412 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c012e80 con 0x7ff19415a020 2024-01-27T02:32:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.656+0000 7ff190ff9700 1 -- 172.21.15.7:0/3843822412 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18c027470 con 0x7ff19415a020 2024-01-27T02:32:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.657+0000 7ff190ff9700 1 -- 172.21.15.7:0/3843822412 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c027810 con 0x7ff19415a020 2024-01-27T02:32:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.657+0000 7ff19a8f2700 1 -- 172.21.15.7:0/3843822412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19415cd70 con 0x7ff19415a020 2024-01-27T02:32:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.657+0000 7ff19a8f2700 1 -- 172.21.15.7:0/3843822412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19415d330 con 0x7ff19415a020 2024-01-27T02:32:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.657+0000 7ff19a8f2700 1 -- 172.21.15.7:0/3843822412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19404c710 con 0x7ff19415a020 2024-01-27T02:32:52.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.658+0000 7ff190ff9700 1 -- 172.21.15.7:0/3843822412 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff18c024070 con 0x7ff19415a020 2024-01-27T02:32:52.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.658+0000 7ff190ff9700 1 --2- 172.21.15.7:0/3843822412 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff174066f30 0x7ff1740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:52.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.659+0000 7ff190ff9700 1 -- 172.21.15.7:0/3843822412 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff18c086870 con 0x7ff19415a020 2024-01-27T02:32:52.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.659+0000 7ff193fff700 1 --2- 172.21.15.7:0/3843822412 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff174066f30 0x7ff1740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:52.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.659+0000 7ff193fff700 1 --2- 172.21.15.7:0/3843822412 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff174066f30 0x7ff1740693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff184006fd0 tx=0x7ff184008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:52.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.663+0000 7ff190ff9700 1 -- 172.21.15.7:0/3843822412 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff18c053e30 con 0x7ff19415a020 2024-01-27T02:32:52.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:52.865+0000 7ff19a8f2700 1 -- 172.21.15.7:0/3843822412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff194069850 con 0x7ff19415a020 2024-01-27T02:32:53.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.410+0000 7ff190ff9700 1 -- 172.21.15.7:0/3843822412 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff18c02c0b0 con 0x7ff19415a020 2024-01-27T02:32:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.415+0000 7ff19a8f2700 1 -- 172.21.15.7:0/3843822412 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff174066f30 msgr2=0x7ff1740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.415+0000 7ff19a8f2700 1 --2- 172.21.15.7:0/3843822412 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff174066f30 0x7ff1740693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff184006fd0 tx=0x7ff184008040 comp rx=0 tx=0).stop 2024-01-27T02:32:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.415+0000 7ff19a8f2700 1 -- 172.21.15.7:0/3843822412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff19415a020 msgr2=0x7ff194156470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.415+0000 7ff19a8f2700 1 --2- 172.21.15.7:0/3843822412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff19415a020 0x7ff194156470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff18c013000 tx=0x7ff18c019000 comp rx=0 tx=0).stop 2024-01-27T02:32:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.415+0000 7ff19a8f2700 1 -- 172.21.15.7:0/3843822412 shutdown_connections 2024-01-27T02:32:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.415+0000 7ff19a8f2700 1 --2- 172.21.15.7:0/3843822412 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff174066f30 0x7ff1740693e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.415+0000 7ff19a8f2700 1 --2- 172.21.15.7:0/3843822412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff19415a020 0x7ff194156470 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.415+0000 7ff19a8f2700 1 --2- 172.21.15.7:0/3843822412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1940680c0 0x7ff19416e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.415+0000 7ff19a8f2700 1 --2- 172.21.15.7:0/3843822412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff19406bb30 0x7ff19415db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.415+0000 7ff19a8f2700 1 -- 172.21.15.7:0/3843822412 >> 172.21.15.7:0/3843822412 conn(0x7ff1940714b0 msgr2=0x7ff1941573d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:53.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.416+0000 7ff19a8f2700 1 -- 172.21.15.7:0/3843822412 shutdown_connections 2024-01-27T02:32:53.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.416+0000 7ff19a8f2700 1 -- 172.21.15.7:0/3843822412 wait complete. 2024-01-27T02:32:53.417 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-27T02:32:53.431 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-27T02:32:53.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.574+0000 7f8fe7785700 1 Processor -- start 2024-01-27T02:32:53.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.574+0000 7f8fe7785700 1 -- start start 2024-01-27T02:32:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.575+0000 7f8fe7785700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fe0141520 0x7f8fe00b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.575+0000 7f8fe7785700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8fe01418f0 0x7f8fe006e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.575+0000 7f8fe7785700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fe006f330 con 0x7f8fe006e860 2024-01-27T02:32:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.575+0000 7f8fe7785700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fe00c88e0 con 0x7f8fe01418f0 2024-01-27T02:32:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.575+0000 7f8fe7785700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe006ed70 con 0x7f8fe0141520 2024-01-27T02:32:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.575+0000 7f8fe4d20700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8fe01418f0 0x7f8fe006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.575+0000 7f8fe5d22700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8fe006e860 0x7f8fe0063580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49174/0 (socket says 172.21.15.7:49174) 2024-01-27T02:32:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.575+0000 7f8fe5d22700 1 -- 172.21.15.7:0/668208892 learned_addr learned my addr 172.21.15.7:0/668208892 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.575+0000 7f8fe5521700 1 --2- 172.21.15.7:0/668208892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fe0141520 0x7f8fe00b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.575+0000 7f8fd7fff700 1 -- 172.21.15.7:0/668208892 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2343313239 0 0) 0x7f8fe006f330 con 0x7f8fe006e860 2024-01-27T02:32:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.575+0000 7f8fd7fff700 1 -- 172.21.15.7:0/668208892 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fcc003570 con 0x7f8fe006e860 2024-01-27T02:32:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.576+0000 7f8fe4d20700 1 -- 172.21.15.7:0/668208892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fe0141520 msgr2=0x7f8fe00b09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.576+0000 7f8fe4d20700 1 --2- 172.21.15.7:0/668208892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fe0141520 0x7f8fe00b09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.576+0000 7f8fe4d20700 1 -- 172.21.15.7:0/668208892 >> v1:172.21.15.7:6789/0 conn(0x7f8fe006e860 legacy=0x7f8fe0063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.576+0000 7f8fe4d20700 1 -- 172.21.15.7:0/668208892 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe006f050 con 0x7f8fe01418f0 2024-01-27T02:32:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.576+0000 7f8fe4d20700 1 --2- 172.21.15.7:0/668208892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe01418f0 0x7f8fe006e320 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00098e0 tx=0x7f8fd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2488dfc01a109646 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.576+0000 7f8fd7fff700 1 -- 172.21.15.7:0/668208892 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0011630 con 0x7f8fe01418f0 2024-01-27T02:32:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.576+0000 7f8fd7fff700 1 -- 172.21.15.7:0/668208892 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd0011790 con 0x7f8fe01418f0 2024-01-27T02:32:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.576+0000 7f8fd7fff700 1 -- 172.21.15.7:0/668208892 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0011b30 con 0x7f8fe01418f0 2024-01-27T02:32:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.577+0000 7f8fe7785700 1 -- 172.21.15.7:0/668208892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe01418f0 msgr2=0x7f8fe006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.577+0000 7f8fe7785700 1 --2- 172.21.15.7:0/668208892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe01418f0 0x7f8fe006e320 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00098e0 tx=0x7f8fd00091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.577+0000 7f8fe7785700 1 -- 172.21.15.7:0/668208892 shutdown_connections 2024-01-27T02:32:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.577+0000 7f8fe7785700 1 --2- 172.21.15.7:0/668208892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fe0141520 0x7f8fe00b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.577+0000 7f8fe7785700 1 --2- 172.21.15.7:0/668208892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe01418f0 0x7f8fe006e320 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.577+0000 7f8fe7785700 1 -- 172.21.15.7:0/668208892 >> 172.21.15.7:0/668208892 conn(0x7f8fe00cdbf0 msgr2=0x7f8fe00cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.577+0000 7f8fe7785700 1 -- 172.21.15.7:0/668208892 shutdown_connections 2024-01-27T02:32:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.577+0000 7f8fe7785700 1 -- 172.21.15.7:0/668208892 wait complete. 2024-01-27T02:32:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.578+0000 7f8fe7785700 1 Processor -- start 2024-01-27T02:32:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.578+0000 7f8fe7785700 1 -- start start 2024-01-27T02:32:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.579+0000 7f8fe7785700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fe006e860 0x7f8fe013e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.579+0000 7f8fe7785700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe0141520 0x7f8fe00af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.579+0000 7f8fe7785700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fe01418f0 0x7f8fe00afef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.579+0000 7f8fe7785700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe00c78a0 con 0x7f8fe0141520 2024-01-27T02:32:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.579+0000 7f8fe7785700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe00c4dd0 con 0x7f8fe006e860 2024-01-27T02:32:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.579+0000 7f8fe7785700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe015b140 con 0x7f8fe01418f0 2024-01-27T02:32:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.579+0000 7f8fe4d20700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe0141520 0x7f8fe00af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.579+0000 7f8fe4d20700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe0141520 0x7f8fe00af9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38598/0 (socket says 172.21.15.7:38598) 2024-01-27T02:32:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.579+0000 7f8fe4d20700 1 -- 172.21.15.7:0/2910311833 learned_addr learned my addr 172.21.15.7:0/2910311833 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.579+0000 7f8fe5d22700 1 --2- 172.21.15.7:0/2910311833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fe01418f0 0x7f8fe00afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.579+0000 7f8fe5521700 1 --2- 172.21.15.7:0/2910311833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fe006e860 0x7f8fe013e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.579+0000 7f8fe4d20700 1 -- 172.21.15.7:0/2910311833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fe01418f0 msgr2=0x7f8fe00afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.579+0000 7f8fe4d20700 1 --2- 172.21.15.7:0/2910311833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fe01418f0 0x7f8fe00afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.579+0000 7f8fe4d20700 1 -- 172.21.15.7:0/2910311833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fe006e860 msgr2=0x7f8fe013e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.579+0000 7f8fe4d20700 1 --2- 172.21.15.7:0/2910311833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fe006e860 0x7f8fe013e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.579+0000 7f8fe4d20700 1 -- 172.21.15.7:0/2910311833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd0009420 con 0x7f8fe0141520 2024-01-27T02:32:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.579+0000 7f8fe5521700 1 --2- 172.21.15.7:0/2910311833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fe006e860 0x7f8fe013e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:32:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.580+0000 7f8fe4d20700 1 --2- 172.21.15.7:0/2910311833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe0141520 0x7f8fe00af9b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0068c20 tx=0x7f8fd0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.580+0000 7f8fd57fa700 1 -- 172.21.15.7:0/2910311833 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0011900 con 0x7f8fe0141520 2024-01-27T02:32:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.580+0000 7f8fd57fa700 1 -- 172.21.15.7:0/2910311833 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd0011a80 con 0x7f8fe0141520 2024-01-27T02:32:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.580+0000 7f8fe7785700 1 -- 172.21.15.7:0/2910311833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe015b420 con 0x7f8fe0141520 2024-01-27T02:32:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.580+0000 7f8fd57fa700 1 -- 172.21.15.7:0/2910311833 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0025670 con 0x7f8fe0141520 2024-01-27T02:32:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.580+0000 7f8fe7785700 1 -- 172.21.15.7:0/2910311833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe015b9e0 con 0x7f8fe0141520 2024-01-27T02:32:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.581+0000 7f8fe7785700 1 -- 172.21.15.7:0/2910311833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fe00c4f60 con 0x7f8fe0141520 2024-01-27T02:32:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.582+0000 7f8fd57fa700 1 -- 172.21.15.7:0/2910311833 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8fd0023070 con 0x7f8fe0141520 2024-01-27T02:32:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.582+0000 7f8fd57fa700 1 --2- 172.21.15.7:0/2910311833 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8fc4066f30 0x7f8fc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.582+0000 7f8fd57fa700 1 -- 172.21.15.7:0/2910311833 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f8fd00857b0 con 0x7f8fe0141520 2024-01-27T02:32:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.582+0000 7f8fe5521700 1 --2- 172.21.15.7:0/2910311833 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8fc4066f30 0x7f8fc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:53.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.583+0000 7f8fe5521700 1 --2- 172.21.15.7:0/2910311833 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8fc4066f30 0x7f8fc40693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc006f00 tx=0x7f8fdc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:53.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.587+0000 7f8fd57fa700 1 -- 172.21.15.7:0/2910311833 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8fd0052c00 con 0x7f8fe0141520 2024-01-27T02:32:53.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:53.791+0000 7f8fe7785700 1 -- 172.21.15.7:0/2910311833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8fe0072060 con 0x7f8fe0141520 2024-01-27T02:32:54.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.417+0000 7f8fd57fa700 1 -- 172.21.15.7:0/2910311833 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8fd00150d0 con 0x7f8fe0141520 2024-01-27T02:32:54.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.425+0000 7f8fe7785700 1 -- 172.21.15.7:0/2910311833 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8fc4066f30 msgr2=0x7f8fc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:54.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.425+0000 7f8fe7785700 1 --2- 172.21.15.7:0/2910311833 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8fc4066f30 0x7f8fc40693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc006f00 tx=0x7f8fdc008040 comp rx=0 tx=0).stop 2024-01-27T02:32:54.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.425+0000 7f8fe7785700 1 -- 172.21.15.7:0/2910311833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe0141520 msgr2=0x7f8fe00af9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:54.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.425+0000 7f8fe7785700 1 --2- 172.21.15.7:0/2910311833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe0141520 0x7f8fe00af9b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0068c20 tx=0x7f8fd0017000 comp rx=0 tx=0).stop 2024-01-27T02:32:54.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.426+0000 7f8fe7785700 1 -- 172.21.15.7:0/2910311833 shutdown_connections 2024-01-27T02:32:54.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.426+0000 7f8fe7785700 1 --2- 172.21.15.7:0/2910311833 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8fc4066f30 0x7f8fc40693e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:54.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.426+0000 7f8fe7785700 1 --2- 172.21.15.7:0/2910311833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fe01418f0 0x7f8fe00afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:54.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.426+0000 7f8fe7785700 1 --2- 172.21.15.7:0/2910311833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fe006e860 0x7f8fe013e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:54.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.426+0000 7f8fe7785700 1 --2- 172.21.15.7:0/2910311833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe0141520 0x7f8fe00af9b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:54.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.426+0000 7f8fe7785700 1 -- 172.21.15.7:0/2910311833 >> 172.21.15.7:0/2910311833 conn(0x7f8fe00cdbf0 msgr2=0x7f8fe0159c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:54.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.427+0000 7f8fe7785700 1 -- 172.21.15.7:0/2910311833 shutdown_connections 2024-01-27T02:32:54.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.427+0000 7f8fe7785700 1 -- 172.21.15.7:0/2910311833 wait complete. 2024-01-27T02:32:54.435 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-27T02:32:54.446 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-27T02:32:54.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.583+0000 7f0977db6700 1 Processor -- start 2024-01-27T02:32:54.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.583+0000 7f0977db6700 1 -- start start 2024-01-27T02:32:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.584+0000 7f0977db6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f097015d460 0x7f09700d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.584+0000 7f0977db6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09700b38d0 con 0x7f097015d830 2024-01-27T02:32:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.584+0000 7f0977db6700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09700d0390 con 0x7f09700cf280 2024-01-27T02:32:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.584+0000 7f0977db6700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09700cf780 con 0x7f097015d460 2024-01-27T02:32:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.584+0000 7f0976353700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f097015d830 0x7f09700cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49182/0 (socket says 172.21.15.7:49182) 2024-01-27T02:32:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.584+0000 7f0976353700 1 -- 172.21.15.7:0/1269630613 learned_addr learned my addr 172.21.15.7:0/1269630613 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.584+0000 7f0975b52700 1 --2- 172.21.15.7:0/1269630613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f097015d460 0x7f09700d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.585+0000 7f0974b50700 1 -- 172.21.15.7:0/1269630613 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 196900681 0 0) 0x7f0968001490 con 0x7f097015d830 2024-01-27T02:32:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.585+0000 7f0974b50700 1 -- 172.21.15.7:0/1269630613 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f095c003570 con 0x7f097015d830 2024-01-27T02:32:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.585+0000 7f0974b50700 1 -- 172.21.15.7:0/1269630613 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 118563245 0 0) 0x7f09700d0390 con 0x7f09700cf280 2024-01-27T02:32:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.585+0000 7f0974b50700 1 -- 172.21.15.7:0/1269630613 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0968001490 con 0x7f09700cf280 2024-01-27T02:32:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.585+0000 7f0975b52700 1 -- 172.21.15.7:0/1269630613 >> v1:172.21.15.188:6789/0 conn(0x7f09700cf280 legacy=0x7f097015e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:54.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.585+0000 7f0974b50700 1 -- 172.21.15.7:0/1269630613 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2358905258 0 0) 0x7f095c003570 con 0x7f097015d830 2024-01-27T02:32:54.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.585+0000 7f0975b52700 1 -- 172.21.15.7:0/1269630613 >> v1:172.21.15.7:6789/0 conn(0x7f097015d830 legacy=0x7f09700cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:54.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.585+0000 7f0975b52700 1 -- 172.21.15.7:0/1269630613 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09700cfa60 con 0x7f097015d460 2024-01-27T02:32:54.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.585+0000 7f0975b52700 1 --2- 172.21.15.7:0/1269630613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f097015d460 0x7f09700d0820 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f096c009910 tx=0x7f096c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e8c7f3bb9b9e2ed server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:54.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.585+0000 7f0974b50700 1 -- 172.21.15.7:0/1269630613 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0968001980 con 0x7f097015d460 2024-01-27T02:32:54.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.585+0000 7f0974b50700 1 -- 172.21.15.7:0/1269630613 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096c011420 con 0x7f097015d460 2024-01-27T02:32:54.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.585+0000 7f0974b50700 1 -- 172.21.15.7:0/1269630613 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c0117c0 con 0x7f097015d460 2024-01-27T02:32:54.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.586+0000 7f0977db6700 1 -- 172.21.15.7:0/1269630613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f097015d460 msgr2=0x7f09700d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:54.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.586+0000 7f0977db6700 1 --2- 172.21.15.7:0/1269630613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f097015d460 0x7f09700d0820 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f096c009910 tx=0x7f096c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:54.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.586+0000 7f0977db6700 1 -- 172.21.15.7:0/1269630613 shutdown_connections 2024-01-27T02:32:54.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.586+0000 7f0977db6700 1 --2- 172.21.15.7:0/1269630613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f097015d460 0x7f09700d0820 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:54.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.586+0000 7f0977db6700 1 -- 172.21.15.7:0/1269630613 >> 172.21.15.7:0/1269630613 conn(0x7f0970063580 msgr2=0x7f0970063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:54.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.586+0000 7f0977db6700 1 -- 172.21.15.7:0/1269630613 shutdown_connections 2024-01-27T02:32:54.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.586+0000 7f0977db6700 1 -- 172.21.15.7:0/1269630613 wait complete. 2024-01-27T02:32:54.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.588+0000 7f0977db6700 1 Processor -- start 2024-01-27T02:32:54.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.588+0000 7f0977db6700 1 -- start start 2024-01-27T02:32:54.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.589+0000 7f0977db6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09700cf280 0x7f09700b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.589+0000 7f0977db6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f097015d460 0x7f09701447c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.589+0000 7f0977db6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f097015d830 0x7f097015a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.589+0000 7f0977db6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09700b3ff0 con 0x7f097015d830 2024-01-27T02:32:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.589+0000 7f0977db6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09700b42d0 con 0x7f097015d460 2024-01-27T02:32:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.589+0000 7f0977db6700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09700b45b0 con 0x7f09700cf280 2024-01-27T02:32:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.589+0000 7f0976353700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f097015d830 0x7f097015a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.589+0000 7f0976353700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f097015d830 0x7f097015a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38606/0 (socket says 172.21.15.7:38606) 2024-01-27T02:32:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.589+0000 7f0976353700 1 -- 172.21.15.7:0/4065899291 learned_addr learned my addr 172.21.15.7:0/4065899291 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.589+0000 7f0975b52700 1 --2- 172.21.15.7:0/4065899291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09700cf280 0x7f09700b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.589+0000 7f0975351700 1 --2- 172.21.15.7:0/4065899291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f097015d460 0x7f09701447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.589+0000 7f0976353700 1 -- 172.21.15.7:0/4065899291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09700cf280 msgr2=0x7f09700b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.589+0000 7f0976353700 1 --2- 172.21.15.7:0/4065899291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09700cf280 0x7f09700b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.589+0000 7f0976353700 1 -- 172.21.15.7:0/4065899291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f097015d460 msgr2=0x7f09701447c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.589+0000 7f0976353700 1 --2- 172.21.15.7:0/4065899291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f097015d460 0x7f09701447c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.589+0000 7f0976353700 1 -- 172.21.15.7:0/4065899291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c009420 con 0x7f097015d830 2024-01-27T02:32:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.590+0000 7f0976353700 1 --2- 172.21.15.7:0/4065899291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f097015d830 0x7f097015a110 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f096800cd60 tx=0x7f0968007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.590+0000 7f0965ffb700 1 -- 172.21.15.7:0/4065899291 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0968004dc0 con 0x7f097015d830 2024-01-27T02:32:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.590+0000 7f0965ffb700 1 -- 172.21.15.7:0/4065899291 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096800a8e0 con 0x7f097015d830 2024-01-27T02:32:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.590+0000 7f0965ffb700 1 -- 172.21.15.7:0/4065899291 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096800aca0 con 0x7f097015d830 2024-01-27T02:32:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.590+0000 7f0977db6700 1 -- 172.21.15.7:0/4065899291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09700b4890 con 0x7f097015d830 2024-01-27T02:32:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.590+0000 7f0977db6700 1 -- 172.21.15.7:0/4065899291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09700b3ca0 con 0x7f097015d830 2024-01-27T02:32:54.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.591+0000 7f0977db6700 1 -- 172.21.15.7:0/4065899291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09700b4db0 con 0x7f097015d830 2024-01-27T02:32:54.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.591+0000 7f0965ffb700 1 -- 172.21.15.7:0/4065899291 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0968007360 con 0x7f097015d830 2024-01-27T02:32:54.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.592+0000 7f0965ffb700 1 --2- 172.21.15.7:0/4065899291 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0954066f30 0x7f09540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:54.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.592+0000 7f0965ffb700 1 -- 172.21.15.7:0/4065899291 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f0968083ba0 con 0x7f097015d830 2024-01-27T02:32:54.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.592+0000 7f0975b52700 1 --2- 172.21.15.7:0/4065899291 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0954066f30 0x7f09540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:54.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.593+0000 7f0975b52700 1 --2- 172.21.15.7:0/4065899291 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0954066f30 0x7f09540693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f096c0030c0 tx=0x7f096c016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:54.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.597+0000 7f0965ffb700 1 -- 172.21.15.7:0/4065899291 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0968050ef0 con 0x7f097015d830 2024-01-27T02:32:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:54.848+0000 7f0977db6700 1 -- 172.21.15.7:0/4065899291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f09700b38a0 con 0x7f097015d830 2024-01-27T02:32:55.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.804+0000 7f0965ffb700 1 -- 172.21.15.7:0/4065899291 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f09680180c0 con 0x7f097015d830 2024-01-27T02:32:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.809+0000 7f0977db6700 1 -- 172.21.15.7:0/4065899291 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0954066f30 msgr2=0x7f09540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.809+0000 7f0977db6700 1 --2- 172.21.15.7:0/4065899291 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0954066f30 0x7f09540693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f096c0030c0 tx=0x7f096c016000 comp rx=0 tx=0).stop 2024-01-27T02:32:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.809+0000 7f0977db6700 1 -- 172.21.15.7:0/4065899291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f097015d830 msgr2=0x7f097015a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:55.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.809+0000 7f0977db6700 1 --2- 172.21.15.7:0/4065899291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f097015d830 0x7f097015a110 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f096800cd60 tx=0x7f0968007d20 comp rx=0 tx=0).stop 2024-01-27T02:32:55.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.809+0000 7f0977db6700 1 -- 172.21.15.7:0/4065899291 shutdown_connections 2024-01-27T02:32:55.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.809+0000 7f0977db6700 1 --2- 172.21.15.7:0/4065899291 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0954066f30 0x7f09540693e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:55.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.809+0000 7f0977db6700 1 --2- 172.21.15.7:0/4065899291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f097015d830 0x7f097015a110 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:55.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.809+0000 7f0977db6700 1 --2- 172.21.15.7:0/4065899291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09700cf280 0x7f09700b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:55.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.809+0000 7f0977db6700 1 --2- 172.21.15.7:0/4065899291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f097015d460 0x7f09701447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:55.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.809+0000 7f0977db6700 1 -- 172.21.15.7:0/4065899291 >> 172.21.15.7:0/4065899291 conn(0x7f0970063580 msgr2=0x7f0970157d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:55.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.813+0000 7f0977db6700 1 -- 172.21.15.7:0/4065899291 shutdown_connections 2024-01-27T02:32:55.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.813+0000 7f0977db6700 1 -- 172.21.15.7:0/4065899291 wait complete. 2024-01-27T02:32:55.816 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 4 and data pool 5 2024-01-27T02:32:55.831 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:32:55.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.968+0000 7fbaf51bb700 1 Processor -- start 2024-01-27T02:32:55.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.968+0000 7fbaf51bb700 1 -- start start 2024-01-27T02:32:55.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.969+0000 7fbaf51bb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbaf00c3ae0 0x7fbaf00cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:55.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.969+0000 7fbaf51bb700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaf00ca020 con 0x7fbaf0157ca0 2024-01-27T02:32:55.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.969+0000 7fbaf51bb700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaf00c5210 con 0x7fbaf0158070 2024-01-27T02:32:55.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.969+0000 7fbaf51bb700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaf00c54f0 con 0x7fbaf00c3ae0 2024-01-27T02:32:55.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.969+0000 7fbaee59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbaf00c3ae0 0x7fbaf00cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:55.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.969+0000 7fbaee59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbaf00c3ae0 0x7fbaf00cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38622/0 (socket says 172.21.15.7:38622) 2024-01-27T02:32:55.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.969+0000 7fbaeed9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbaf0157ca0 0x7fbaf00c4a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49188/0 (socket says 172.21.15.7:49188) 2024-01-27T02:32:55.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.969+0000 7fbaee59c700 1 -- 172.21.15.7:0/3112140021 learned_addr learned my addr 172.21.15.7:0/3112140021 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:55.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.969+0000 7fbaee59c700 1 -- 172.21.15.7:0/3112140021 >> v1:172.21.15.188:6789/0 conn(0x7fbaf0158070 legacy=0x7fbaf00ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:55.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.969+0000 7fbaee59c700 1 -- 172.21.15.7:0/3112140021 >> v1:172.21.15.7:6789/0 conn(0x7fbaf0157ca0 legacy=0x7fbaf00c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:55.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.969+0000 7fbaee59c700 1 -- 172.21.15.7:0/3112140021 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf00c57d0 con 0x7fbaf00c3ae0 2024-01-27T02:32:55.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.969+0000 7fbaedd9b700 1 -- 172.21.15.7:0/3112140021 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248146643 0 0) 0x7fbaf00ca020 con 0x7fbaf0157ca0 2024-01-27T02:32:55.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.970+0000 7fbaee59c700 1 --2- 172.21.15.7:0/3112140021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf00c3ae0 0x7fbaf00cd4e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbae40099b0 tx=0x7fbae4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c074ac5a55ef35ea server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:55.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.970+0000 7fbaedd9b700 1 -- 172.21.15.7:0/3112140021 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae400b860 con 0x7fbaf00c3ae0 2024-01-27T02:32:55.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.970+0000 7fbaedd9b700 1 -- 172.21.15.7:0/3112140021 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae400b9c0 con 0x7fbaf00c3ae0 2024-01-27T02:32:55.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.970+0000 7fbaedd9b700 1 -- 172.21.15.7:0/3112140021 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae400bd60 con 0x7fbaf00c3ae0 2024-01-27T02:32:55.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.970+0000 7fbaf51bb700 1 -- 172.21.15.7:0/3112140021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf00c3ae0 msgr2=0x7fbaf00cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:55.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.970+0000 7fbaf51bb700 1 --2- 172.21.15.7:0/3112140021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf00c3ae0 0x7fbaf00cd4e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbae40099b0 tx=0x7fbae4009250 comp rx=0 tx=0).stop 2024-01-27T02:32:55.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.970+0000 7fbaf51bb700 1 -- 172.21.15.7:0/3112140021 shutdown_connections 2024-01-27T02:32:55.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.970+0000 7fbaf51bb700 1 --2- 172.21.15.7:0/3112140021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf00c3ae0 0x7fbaf00cd4e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:55.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.970+0000 7fbaf51bb700 1 -- 172.21.15.7:0/3112140021 >> 172.21.15.7:0/3112140021 conn(0x7fbaf0098020 msgr2=0x7fbaf00711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:55.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.970+0000 7fbaf51bb700 1 -- 172.21.15.7:0/3112140021 shutdown_connections 2024-01-27T02:32:55.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.970+0000 7fbaf51bb700 1 -- 172.21.15.7:0/3112140021 wait complete. 2024-01-27T02:32:55.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.972+0000 7fbaf51bb700 1 Processor -- start 2024-01-27T02:32:55.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.972+0000 7fbaf51bb700 1 -- start start 2024-01-27T02:32:55.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.973+0000 7fbaf51bb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbaf00c3ae0 0x7fbaf00b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:55.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.973+0000 7fbaf51bb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbaf0157ca0 0x7fbaf014cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:55.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.973+0000 7fbaf51bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf0158070 0x7fbaf00c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.973+0000 7fbaf51bb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf00c74f0 con 0x7fbaf0158070 2024-01-27T02:32:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.973+0000 7fbaf51bb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf00c7a90 con 0x7fbaf0157ca0 2024-01-27T02:32:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.973+0000 7fbaf51bb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf00b28e0 con 0x7fbaf00c3ae0 2024-01-27T02:32:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.974+0000 7fbaef59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf0158070 0x7fbaf00c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.974+0000 7fbaef59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf0158070 0x7fbaf00c7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38638/0 (socket says 172.21.15.7:38638) 2024-01-27T02:32:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.974+0000 7fbaef59e700 1 -- 172.21.15.7:0/36392349 learned_addr learned my addr 172.21.15.7:0/36392349 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.974+0000 7fbaeed9d700 1 --2- 172.21.15.7:0/36392349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbaf00c3ae0 0x7fbaf00b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.974+0000 7fbaee59c700 1 --2- 172.21.15.7:0/36392349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbaf0157ca0 0x7fbaf014cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.974+0000 7fbaef59e700 1 -- 172.21.15.7:0/36392349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbaf00c3ae0 msgr2=0x7fbaf00b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.974+0000 7fbaef59e700 1 --2- 172.21.15.7:0/36392349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbaf00c3ae0 0x7fbaf00b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.974+0000 7fbaef59e700 1 -- 172.21.15.7:0/36392349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbaf0157ca0 msgr2=0x7fbaf014cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.974+0000 7fbaef59e700 1 --2- 172.21.15.7:0/36392349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbaf0157ca0 0x7fbaf014cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.974+0000 7fbaef59e700 1 -- 172.21.15.7:0/36392349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae40094f0 con 0x7fbaf0158070 2024-01-27T02:32:55.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.974+0000 7fbaef59e700 1 --2- 172.21.15.7:0/36392349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf0158070 0x7fbaf00c7f20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbae800c870 tx=0x7fbae8001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:55.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.974+0000 7fbad6ffd700 1 -- 172.21.15.7:0/36392349 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae8012630 con 0x7fbaf0158070 2024-01-27T02:32:55.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.974+0000 7fbad6ffd700 1 -- 172.21.15.7:0/36392349 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae80127b0 con 0x7fbaf0158070 2024-01-27T02:32:55.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.974+0000 7fbaf51bb700 1 -- 172.21.15.7:0/36392349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf0150850 con 0x7fbaf0158070 2024-01-27T02:32:55.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.974+0000 7fbad6ffd700 1 -- 172.21.15.7:0/36392349 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae8012b70 con 0x7fbaf0158070 2024-01-27T02:32:55.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.974+0000 7fbaf51bb700 1 -- 172.21.15.7:0/36392349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf0150290 con 0x7fbaf0158070 2024-01-27T02:32:55.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.975+0000 7fbaf51bb700 1 -- 172.21.15.7:0/36392349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaf00c7c40 con 0x7fbaf0158070 2024-01-27T02:32:55.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.976+0000 7fbad6ffd700 1 -- 172.21.15.7:0/36392349 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbae8003030 con 0x7fbaf0158070 2024-01-27T02:32:55.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.976+0000 7fbad6ffd700 1 --2- 172.21.15.7:0/36392349 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbadc066f30 0x7fbadc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:55.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.976+0000 7fbad6ffd700 1 -- 172.21.15.7:0/36392349 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbae8084380 con 0x7fbaf0158070 2024-01-27T02:32:55.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.977+0000 7fbaeed9d700 1 --2- 172.21.15.7:0/36392349 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbadc066f30 0x7fbadc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:55.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.977+0000 7fbaeed9d700 1 --2- 172.21.15.7:0/36392349 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbadc066f30 0x7fbadc0693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbae0007fd0 tx=0x7fbae0007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:55.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:55.981+0000 7fbad6ffd700 1 -- 172.21.15.7:0/36392349 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbaf00c7c40 con 0x7fbaf0158070 2024-01-27T02:32:56.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.185+0000 7fbaf51bb700 1 -- 172.21.15.7:0/36392349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbaf00c7c40 con 0x7fbaf0158070 2024-01-27T02:32:56.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.186+0000 7fbad6ffd700 1 -- 172.21.15.7:0/36392349 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+15315 (secure 0 0 0) 0x7fbaf00c7c40 con 0x7fbaf0158070 2024-01-27T02:32:56.186 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:56.186 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":31,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:32:54.849477+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:32:52.897548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-27T02:32:53.822754+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:32:56.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.188+0000 7fbaf51bb700 1 -- 172.21.15.7:0/36392349 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbadc066f30 msgr2=0x7fbadc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:56.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.188+0000 7fbaf51bb700 1 --2- 172.21.15.7:0/36392349 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbadc066f30 0x7fbadc0693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbae0007fd0 tx=0x7fbae0007d30 comp rx=0 tx=0).stop 2024-01-27T02:32:56.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.188+0000 7fbaf51bb700 1 -- 172.21.15.7:0/36392349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf0158070 msgr2=0x7fbaf00c7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:56.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.188+0000 7fbaf51bb700 1 --2- 172.21.15.7:0/36392349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf0158070 0x7fbaf00c7f20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbae800c870 tx=0x7fbae8001670 comp rx=0 tx=0).stop 2024-01-27T02:32:56.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.188+0000 7fbaf51bb700 1 -- 172.21.15.7:0/36392349 shutdown_connections 2024-01-27T02:32:56.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.188+0000 7fbaf51bb700 1 --2- 172.21.15.7:0/36392349 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbadc066f30 0x7fbadc0693e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:56.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.188+0000 7fbaf51bb700 1 --2- 172.21.15.7:0/36392349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf0158070 0x7fbaf00c7f20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:56.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.188+0000 7fbaf51bb700 1 --2- 172.21.15.7:0/36392349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbaf00c3ae0 0x7fbaf00b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:56.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.188+0000 7fbaf51bb700 1 --2- 172.21.15.7:0/36392349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbaf0157ca0 0x7fbaf014cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:56.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.188+0000 7fbaf51bb700 1 -- 172.21.15.7:0/36392349 >> 172.21.15.7:0/36392349 conn(0x7fbaf0098020 msgr2=0x7fbaf006c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:56.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.188+0000 7fbaf51bb700 1 -- 172.21.15.7:0/36392349 shutdown_connections 2024-01-27T02:32:56.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.188+0000 7fbaf51bb700 1 -- 172.21.15.7:0/36392349 wait complete. 2024-01-27T02:32:56.202 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:32:56.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.332+0000 7f8d2b52d700 1 Processor -- start 2024-01-27T02:32:56.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.333+0000 7f8d2b52d700 1 -- start start 2024-01-27T02:32:56.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.334+0000 7f8d2b52d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8d240c9bb0 0x7f8d24157f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:56.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.334+0000 7f8d2b52d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8d240c9f80 0x7f8d2406b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:56.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.334+0000 7f8d2b52d700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2406c880 con 0x7f8d2406e330 2024-01-27T02:32:56.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.334+0000 7f8d2b52d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2406cd10 con 0x7f8d240c9f80 2024-01-27T02:32:56.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.334+0000 7f8d2b52d700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2406bc70 con 0x7f8d240c9bb0 2024-01-27T02:32:56.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.334+0000 7f8d28ac8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8d240c9f80 0x7f8d2406b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:56.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.334+0000 7f8d28ac8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8d240c9f80 0x7f8d2406b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38652/0 (socket says 172.21.15.7:38652) 2024-01-27T02:32:56.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.334+0000 7f8d28ac8700 1 -- 172.21.15.7:0/1404576151 learned_addr learned my addr 172.21.15.7:0/1404576151 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:56.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.334+0000 7f8d292c9700 1 --2- 172.21.15.7:0/1404576151 >> v2:172.21.15.188:3300/0 conn(0x7f8d240c9bb0 0x7f8d24157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:56.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.334+0000 7f8d28ac8700 1 -- 172.21.15.7:0/1404576151 >> v2:172.21.15.188:3300/0 conn(0x7f8d240c9bb0 msgr2=0x7f8d24157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:56.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.334+0000 7f8d28ac8700 1 --2- 172.21.15.7:0/1404576151 >> v2:172.21.15.188:3300/0 conn(0x7f8d240c9bb0 0x7f8d24157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:56.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.334+0000 7f8d28ac8700 1 -- 172.21.15.7:0/1404576151 >> v1:172.21.15.188:6789/0 conn(0x7f8d2406e330 legacy=0x7f8d2406e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:56.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.334+0000 7f8d28ac8700 1 -- 172.21.15.7:0/1404576151 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2406bf50 con 0x7f8d240c9f80 2024-01-27T02:32:56.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.334+0000 7f8d28ac8700 1 --2- 172.21.15.7:0/1404576151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d240c9f80 0x7f8d2406b400 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8d2406e020 tx=0x7f8d14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22222dfd5b6a6871 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:56.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.335+0000 7f8d1bfff700 1 -- 172.21.15.7:0/1404576151 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1400b860 con 0x7f8d240c9f80 2024-01-27T02:32:56.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.335+0000 7f8d1bfff700 1 -- 172.21.15.7:0/1404576151 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1400b9c0 con 0x7f8d240c9f80 2024-01-27T02:32:56.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.335+0000 7f8d1bfff700 1 -- 172.21.15.7:0/1404576151 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1400bd60 con 0x7f8d240c9f80 2024-01-27T02:32:56.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.335+0000 7f8d2b52d700 1 -- 172.21.15.7:0/1404576151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d240c9f80 msgr2=0x7f8d2406b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:56.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.335+0000 7f8d2b52d700 1 --2- 172.21.15.7:0/1404576151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d240c9f80 0x7f8d2406b400 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8d2406e020 tx=0x7f8d14009250 comp rx=0 tx=0).stop 2024-01-27T02:32:56.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.335+0000 7f8d2b52d700 1 -- 172.21.15.7:0/1404576151 shutdown_connections 2024-01-27T02:32:56.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.335+0000 7f8d2b52d700 1 --2- 172.21.15.7:0/1404576151 >> v2:172.21.15.188:3300/0 conn(0x7f8d240c9bb0 0x7f8d24157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:56.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.335+0000 7f8d2b52d700 1 --2- 172.21.15.7:0/1404576151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d240c9f80 0x7f8d2406b400 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:56.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.335+0000 7f8d2b52d700 1 -- 172.21.15.7:0/1404576151 >> 172.21.15.7:0/1404576151 conn(0x7f8d240cdc30 msgr2=0x7f8d240ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:56.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.335+0000 7f8d2b52d700 1 -- 172.21.15.7:0/1404576151 shutdown_connections 2024-01-27T02:32:56.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.335+0000 7f8d2b52d700 1 -- 172.21.15.7:0/1404576151 wait complete. 2024-01-27T02:32:56.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.336+0000 7f8d2b52d700 1 Processor -- start 2024-01-27T02:32:56.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.337+0000 7f8d2b52d700 1 -- start start 2024-01-27T02:32:56.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.337+0000 7f8d2b52d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d2406e330 0x7f8d240c2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:56.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.337+0000 7f8d2b52d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d240c9bb0 0x7f8d24139190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:56.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.337+0000 7f8d2b52d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d240c9f80 0x7f8d241396d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:56.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.337+0000 7f8d2b52d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2415c230 con 0x7f8d240c9f80 2024-01-27T02:32:56.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.337+0000 7f8d2b52d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d240b13a0 con 0x7f8d2406e330 2024-01-27T02:32:56.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.337+0000 7f8d2b52d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d240b0a70 con 0x7f8d240c9bb0 2024-01-27T02:32:56.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.338+0000 7f8d29aca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d240c9f80 0x7f8d241396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:56.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.338+0000 7f8d29aca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d240c9f80 0x7f8d241396d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38656/0 (socket says 172.21.15.7:38656) 2024-01-27T02:32:56.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.338+0000 7f8d29aca700 1 -- 172.21.15.7:0/646275441 learned_addr learned my addr 172.21.15.7:0/646275441 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:56.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.338+0000 7f8d28ac8700 1 --2- 172.21.15.7:0/646275441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d240c9bb0 0x7f8d24139190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:56.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.338+0000 7f8d292c9700 1 --2- 172.21.15.7:0/646275441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d2406e330 0x7f8d240c2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:56.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.338+0000 7f8d29aca700 1 -- 172.21.15.7:0/646275441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d240c9bb0 msgr2=0x7f8d24139190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:56.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.338+0000 7f8d29aca700 1 --2- 172.21.15.7:0/646275441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d240c9bb0 0x7f8d24139190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:56.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.338+0000 7f8d29aca700 1 -- 172.21.15.7:0/646275441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d2406e330 msgr2=0x7f8d240c2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:56.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.338+0000 7f8d29aca700 1 --2- 172.21.15.7:0/646275441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d2406e330 0x7f8d240c2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:56.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.338+0000 7f8d29aca700 1 -- 172.21.15.7:0/646275441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d140094f0 con 0x7f8d240c9f80 2024-01-27T02:32:56.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.338+0000 7f8d29aca700 1 --2- 172.21.15.7:0/646275441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d240c9f80 0x7f8d241396d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00c840 tx=0x7f8d1c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:56.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.338+0000 7f8d197fa700 1 -- 172.21.15.7:0/646275441 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c012630 con 0x7f8d240c9f80 2024-01-27T02:32:56.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.338+0000 7f8d197fa700 1 -- 172.21.15.7:0/646275441 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c0127b0 con 0x7f8d240c9f80 2024-01-27T02:32:56.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.338+0000 7f8d2b52d700 1 -- 172.21.15.7:0/646275441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d240b0d50 con 0x7f8d240c9f80 2024-01-27T02:32:56.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.338+0000 7f8d197fa700 1 -- 172.21.15.7:0/646275441 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c012b70 con 0x7f8d240c9f80 2024-01-27T02:32:56.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.338+0000 7f8d2b52d700 1 -- 172.21.15.7:0/646275441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d240b1830 con 0x7f8d240c9f80 2024-01-27T02:32:56.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.340+0000 7f8d197fa700 1 -- 172.21.15.7:0/646275441 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8d1c01a3a0 con 0x7f8d240c9f80 2024-01-27T02:32:56.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.340+0000 7f8d2b52d700 1 -- 172.21.15.7:0/646275441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d240b1510 con 0x7f8d240c9f80 2024-01-27T02:32:56.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.340+0000 7f8d197fa700 1 --2- 172.21.15.7:0/646275441 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8d08066f30 0x7f8d080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:56.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.340+0000 7f8d197fa700 1 -- 172.21.15.7:0/646275441 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8d1c084690 con 0x7f8d240c9f80 2024-01-27T02:32:56.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.341+0000 7f8d292c9700 1 --2- 172.21.15.7:0/646275441 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8d08066f30 0x7f8d080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:56.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.341+0000 7f8d292c9700 1 --2- 172.21.15.7:0/646275441 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8d08066f30 0x7f8d080693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8d20006fd0 tx=0x7f8d20008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:56.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.345+0000 7f8d197fa700 1 -- 172.21.15.7:0/646275441 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d1c051900 con 0x7f8d240c9f80 2024-01-27T02:32:56.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.549+0000 7f8d2b52d700 1 -- 172.21.15.7:0/646275441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d2404c710 con 0x7f8d240c9f80 2024-01-27T02:32:56.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.550+0000 7f8d197fa700 1 -- 172.21.15.7:0/646275441 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f8d1c054f20 con 0x7f8d240c9f80 2024-01-27T02:32:56.551 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:56.551 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":32,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:32:56.417162+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:32:52.897548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-27T02:32:53.822754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:32:56.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.553+0000 7f8d2b52d700 1 -- 172.21.15.7:0/646275441 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8d08066f30 msgr2=0x7f8d080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:56.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.553+0000 7f8d2b52d700 1 --2- 172.21.15.7:0/646275441 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8d08066f30 0x7f8d080693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8d20006fd0 tx=0x7f8d20008040 comp rx=0 tx=0).stop 2024-01-27T02:32:56.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.553+0000 7f8d2b52d700 1 -- 172.21.15.7:0/646275441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d240c9f80 msgr2=0x7f8d241396d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:56.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.553+0000 7f8d2b52d700 1 --2- 172.21.15.7:0/646275441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d240c9f80 0x7f8d241396d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00c840 tx=0x7f8d1c001670 comp rx=0 tx=0).stop 2024-01-27T02:32:56.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.553+0000 7f8d2b52d700 1 -- 172.21.15.7:0/646275441 shutdown_connections 2024-01-27T02:32:56.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.553+0000 7f8d2b52d700 1 --2- 172.21.15.7:0/646275441 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8d08066f30 0x7f8d080693e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:56.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.553+0000 7f8d2b52d700 1 --2- 172.21.15.7:0/646275441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d240c9f80 0x7f8d241396d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:56.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.553+0000 7f8d2b52d700 1 --2- 172.21.15.7:0/646275441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d2406e330 0x7f8d240c2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:56.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.553+0000 7f8d2b52d700 1 --2- 172.21.15.7:0/646275441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d240c9bb0 0x7f8d24139190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:56.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.553+0000 7f8d2b52d700 1 -- 172.21.15.7:0/646275441 >> 172.21.15.7:0/646275441 conn(0x7f8d240cdc30 msgr2=0x7f8d240cd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:56.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.553+0000 7f8d2b52d700 1 -- 172.21.15.7:0/646275441 shutdown_connections 2024-01-27T02:32:56.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.553+0000 7f8d2b52d700 1 -- 172.21.15.7:0/646275441 wait complete. 2024-01-27T02:32:56.567 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-27T02:32:56.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.702+0000 7f0184124700 1 Processor -- start 2024-01-27T02:32:56.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.702+0000 7f0184124700 1 -- start start 2024-01-27T02:32:56.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.703+0000 7f0184124700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f017c0c1fd0 0x7f017c068fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:56.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.703+0000 7f0184124700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f017c0cf2c0 0x7f017c0b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:56.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.703+0000 7f0184124700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f017c0cc130 con 0x7f017c069500 2024-01-27T02:32:56.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.703+0000 7f0184124700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f017c0c9550 con 0x7f017c0cf2c0 2024-01-27T02:32:56.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.703+0000 7f0184124700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f017c0cb600 con 0x7f017c0c1fd0 2024-01-27T02:32:56.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.703+0000 7f01826c1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f017c0cf2c0 0x7f017c0b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:56.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.703+0000 7f01816bf700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f017c069500 0x7f017c0cebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49202/0 (socket says 172.21.15.7:49202) 2024-01-27T02:32:56.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.703+0000 7f01816bf700 1 -- 172.21.15.7:0/2068047268 learned_addr learned my addr 172.21.15.7:0/2068047268 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:56.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.703+0000 7f0181ec0700 1 --2- 172.21.15.7:0/2068047268 >> v2:172.21.15.188:3300/0 conn(0x7f017c0c1fd0 0x7f017c068fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:56.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.703+0000 7f01826c1700 1 -- 172.21.15.7:0/2068047268 >> v2:172.21.15.188:3300/0 conn(0x7f017c0c1fd0 msgr2=0x7f017c068fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:56.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.703+0000 7f01826c1700 1 --2- 172.21.15.7:0/2068047268 >> v2:172.21.15.188:3300/0 conn(0x7f017c0c1fd0 0x7f017c068fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:56.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.703+0000 7f01826c1700 1 -- 172.21.15.7:0/2068047268 >> v1:172.21.15.7:6789/0 conn(0x7f017c069500 legacy=0x7f017c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:56.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.703+0000 7f0180ebe700 1 -- 172.21.15.7:0/2068047268 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 612801499 0 0) 0x7f017c0cc130 con 0x7f017c069500 2024-01-27T02:32:56.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.704+0000 7f01826c1700 1 -- 172.21.15.7:0/2068047268 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f017c0cb800 con 0x7f017c0cf2c0 2024-01-27T02:32:56.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.704+0000 7f01826c1700 1 --2- 172.21.15.7:0/2068047268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f017c0cf2c0 0x7f017c0b3d80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f017c06c700 tx=0x7f0174009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c93ac4b940e7b8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:56.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.704+0000 7f0180ebe700 1 -- 172.21.15.7:0/2068047268 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017400e070 con 0x7f017c0cf2c0 2024-01-27T02:32:56.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.704+0000 7f0180ebe700 1 -- 172.21.15.7:0/2068047268 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017400bb60 con 0x7f017c0cf2c0 2024-01-27T02:32:56.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.704+0000 7f0180ebe700 1 -- 172.21.15.7:0/2068047268 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0174012430 con 0x7f017c0cf2c0 2024-01-27T02:32:56.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.704+0000 7f0184124700 1 -- 172.21.15.7:0/2068047268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f017c0cf2c0 msgr2=0x7f017c0b3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:56.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.704+0000 7f0184124700 1 --2- 172.21.15.7:0/2068047268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f017c0cf2c0 0x7f017c0b3d80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f017c06c700 tx=0x7f0174009780 comp rx=0 tx=0).stop 2024-01-27T02:32:56.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.704+0000 7f0184124700 1 -- 172.21.15.7:0/2068047268 shutdown_connections 2024-01-27T02:32:56.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.704+0000 7f0184124700 1 --2- 172.21.15.7:0/2068047268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f017c0cf2c0 0x7f017c0b3d80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:56.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.705+0000 7f0184124700 1 --2- 172.21.15.7:0/2068047268 >> v2:172.21.15.188:3300/0 conn(0x7f017c0c1fd0 0x7f017c068fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:56.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.705+0000 7f0184124700 1 -- 172.21.15.7:0/2068047268 >> 172.21.15.7:0/2068047268 conn(0x7f017c06f2c0 msgr2=0x7f017c06f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:56.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.705+0000 7f0184124700 1 -- 172.21.15.7:0/2068047268 shutdown_connections 2024-01-27T02:32:56.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.705+0000 7f0184124700 1 -- 172.21.15.7:0/2068047268 wait complete. 2024-01-27T02:32:56.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.706+0000 7f0184124700 1 Processor -- start 2024-01-27T02:32:56.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.706+0000 7f0184124700 1 -- start start 2024-01-27T02:32:56.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.707+0000 7f0184124700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f017c069500 0x7f017c16d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:56.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.707+0000 7f0184124700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f017c0c1fd0 0x7f017c0c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:56.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.707+0000 7f0184124700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f017c0cf2c0 0x7f017c0c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:56.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.707+0000 7f0184124700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c16ee60 con 0x7f017c0cf2c0 2024-01-27T02:32:56.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.707+0000 7f0184124700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c16e9d0 con 0x7f017c0c1fd0 2024-01-27T02:32:56.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.707+0000 7f0184124700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f017c16e0a0 con 0x7f017c069500 2024-01-27T02:32:56.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.707+0000 7f01826c1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f017c0cf2c0 0x7f017c0c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:56.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.707+0000 7f01826c1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f017c0cf2c0 0x7f017c0c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38668/0 (socket says 172.21.15.7:38668) 2024-01-27T02:32:56.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.707+0000 7f01826c1700 1 -- 172.21.15.7:0/3737668635 learned_addr learned my addr 172.21.15.7:0/3737668635 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:56.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.707+0000 7f01816bf700 1 --2- 172.21.15.7:0/3737668635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f017c0c1fd0 0x7f017c0c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:56.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.707+0000 7f0181ec0700 1 --2- 172.21.15.7:0/3737668635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f017c069500 0x7f017c16d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:56.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.707+0000 7f01826c1700 1 -- 172.21.15.7:0/3737668635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f017c069500 msgr2=0x7f017c16d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:56.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.707+0000 7f01826c1700 1 --2- 172.21.15.7:0/3737668635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f017c069500 0x7f017c16d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:56.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.707+0000 7f01826c1700 1 -- 172.21.15.7:0/3737668635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f017c0c1fd0 msgr2=0x7f017c0c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:56.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.707+0000 7f01826c1700 1 --2- 172.21.15.7:0/3737668635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f017c0c1fd0 0x7f017c0c65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:56.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.707+0000 7f01826c1700 1 -- 172.21.15.7:0/3737668635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0174009a20 con 0x7f017c0cf2c0 2024-01-27T02:32:56.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.707+0000 7f0181ec0700 1 --2- 172.21.15.7:0/3737668635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f017c069500 0x7f017c16d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:32:56.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.707+0000 7f01826c1700 1 --2- 172.21.15.7:0/3737668635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f017c0cf2c0 0x7f017c0c6b10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f017c0719d0 tx=0x7f0174019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:56.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.707+0000 7f0171ffb700 1 -- 172.21.15.7:0/3737668635 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0174012e60 con 0x7f017c0cf2c0 2024-01-27T02:32:56.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.707+0000 7f0171ffb700 1 -- 172.21.15.7:0/3737668635 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017401d630 con 0x7f017c0cf2c0 2024-01-27T02:32:56.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.708+0000 7f0184124700 1 -- 172.21.15.7:0/3737668635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f017c16e380 con 0x7f017c0cf2c0 2024-01-27T02:32:56.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.708+0000 7f0171ffb700 1 -- 172.21.15.7:0/3737668635 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017401d9f0 con 0x7f017c0cf2c0 2024-01-27T02:32:56.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.708+0000 7f0184124700 1 -- 172.21.15.7:0/3737668635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f017c0c5f20 con 0x7f017c0cf2c0 2024-01-27T02:32:56.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.709+0000 7f0184124700 1 -- 172.21.15.7:0/3737668635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f017c0b4490 con 0x7f017c0cf2c0 2024-01-27T02:32:56.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.709+0000 7f0171ffb700 1 -- 172.21.15.7:0/3737668635 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0174024070 con 0x7f017c0cf2c0 2024-01-27T02:32:56.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.710+0000 7f0171ffb700 1 --2- 172.21.15.7:0/3737668635 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0160066f30 0x7f01600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:56.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.710+0000 7f0171ffb700 1 -- 172.21.15.7:0/3737668635 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0174086d20 con 0x7f017c0cf2c0 2024-01-27T02:32:56.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.710+0000 7f0181ec0700 1 --2- 172.21.15.7:0/3737668635 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0160066f30 0x7f01600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:56.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.710+0000 7f0181ec0700 1 --2- 172.21.15.7:0/3737668635 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0160066f30 0x7f01600693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0178006f00 tx=0x7f0178008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:56.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.714+0000 7f0171ffb700 1 -- 172.21.15.7:0/3737668635 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0174053f90 con 0x7f017c0cf2c0 2024-01-27T02:32:56.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:56.967+0000 7f0184124700 1 -- 172.21.15.7:0/3737668635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f017c072060 con 0x7f017c0cf2c0 2024-01-27T02:32:57.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.810+0000 7f0171ffb700 1 -- 172.21.15.7:0/3737668635 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0174009880 con 0x7f017c0cf2c0 2024-01-27T02:32:57.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.812+0000 7f0184124700 1 -- 172.21.15.7:0/3737668635 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0160066f30 msgr2=0x7f01600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:57.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.812+0000 7f0184124700 1 --2- 172.21.15.7:0/3737668635 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0160066f30 0x7f01600693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0178006f00 tx=0x7f0178008040 comp rx=0 tx=0).stop 2024-01-27T02:32:57.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.812+0000 7f0184124700 1 -- 172.21.15.7:0/3737668635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f017c0cf2c0 msgr2=0x7f017c0c6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:57.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.812+0000 7f0184124700 1 --2- 172.21.15.7:0/3737668635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f017c0cf2c0 0x7f017c0c6b10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f017c0719d0 tx=0x7f0174019000 comp rx=0 tx=0).stop 2024-01-27T02:32:57.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.813+0000 7f0184124700 1 -- 172.21.15.7:0/3737668635 shutdown_connections 2024-01-27T02:32:57.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.813+0000 7f0184124700 1 --2- 172.21.15.7:0/3737668635 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0160066f30 0x7f01600693e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:57.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.813+0000 7f0184124700 1 --2- 172.21.15.7:0/3737668635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f017c0cf2c0 0x7f017c0c6b10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:57.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.813+0000 7f0184124700 1 --2- 172.21.15.7:0/3737668635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f017c069500 0x7f017c16d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:57.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.813+0000 7f0184124700 1 --2- 172.21.15.7:0/3737668635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f017c0c1fd0 0x7f017c0c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:57.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.813+0000 7f0184124700 1 -- 172.21.15.7:0/3737668635 >> 172.21.15.7:0/3737668635 conn(0x7f017c06f2c0 msgr2=0x7f017c0cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:57.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.813+0000 7f0184124700 1 -- 172.21.15.7:0/3737668635 shutdown_connections 2024-01-27T02:32:57.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.813+0000 7f0184124700 1 -- 172.21.15.7:0/3737668635 wait complete. 2024-01-27T02:32:57.828 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-27T02:32:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.967+0000 7f4a7a919700 1 Processor -- start 2024-01-27T02:32:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.967+0000 7f4a7a919700 1 -- start start 2024-01-27T02:32:57.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.968+0000 7f4a7a919700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4a74063c90 0x7f4a740ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:57.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.968+0000 7f4a7a919700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a740adec0 con 0x7f4a74154920 2024-01-27T02:32:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.968+0000 7f4a7a919700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a740ae1a0 con 0x7f4a74154550 2024-01-27T02:32:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.968+0000 7f4a7a919700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a7415a850 con 0x7f4a74063c90 2024-01-27T02:32:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.968+0000 7f4a737fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4a74154920 0x7f4a74063580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49212/0 (socket says 172.21.15.7:49212) 2024-01-27T02:32:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.968+0000 7f4a737fe700 1 -- 172.21.15.7:0/3415301609 learned_addr learned my addr 172.21.15.7:0/3415301609 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.969+0000 7f4a78eb6700 1 --2- 172.21.15.7:0/3415301609 >> v2:172.21.15.188:3300/0 conn(0x7f4a74063c90 0x7f4a740ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.969+0000 7f4a72ffd700 1 -- 172.21.15.7:0/3415301609 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3877109515 0 0) 0x7f4a740adec0 con 0x7f4a74154920 2024-01-27T02:32:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.969+0000 7f4a72ffd700 1 -- 172.21.15.7:0/3415301609 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a5c003570 con 0x7f4a74154920 2024-01-27T02:32:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.969+0000 7f4a72ffd700 1 -- 172.21.15.7:0/3415301609 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2335345814 0 0) 0x7f4a740ae1a0 con 0x7f4a74154550 2024-01-27T02:32:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.969+0000 7f4a72ffd700 1 -- 172.21.15.7:0/3415301609 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a740adec0 con 0x7f4a74154550 2024-01-27T02:32:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.969+0000 7f4a78eb6700 1 -- 172.21.15.7:0/3415301609 >> v1:172.21.15.188:6789/0 conn(0x7f4a74154550 legacy=0x7f4a7415b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.969+0000 7f4a78eb6700 1 -- 172.21.15.7:0/3415301609 >> v1:172.21.15.7:6789/0 conn(0x7f4a74154920 legacy=0x7f4a74063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.969+0000 7f4a72ffd700 1 -- 172.21.15.7:0/3415301609 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2722146353 0 0) 0x7f4a5c003570 con 0x7f4a74154920 2024-01-27T02:32:57.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.969+0000 7f4a78eb6700 1 -- 172.21.15.7:0/3415301609 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a7415ab30 con 0x7f4a74063c90 2024-01-27T02:32:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.969+0000 7f4a78eb6700 1 --2- 172.21.15.7:0/3415301609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a74063c90 0x7f4a740ce6b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009e60 tx=0x7f4a6c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8b945c574487c9 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.969+0000 7f4a72ffd700 1 -- 172.21.15.7:0/3415301609 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c00e070 con 0x7f4a74063c90 2024-01-27T02:32:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.969+0000 7f4a72ffd700 1 -- 172.21.15.7:0/3415301609 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a6c00b7a0 con 0x7f4a74063c90 2024-01-27T02:32:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.969+0000 7f4a72ffd700 1 -- 172.21.15.7:0/3415301609 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68001470 con 0x7f4a74063c90 2024-01-27T02:32:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.970+0000 7f4a7a919700 1 -- 172.21.15.7:0/3415301609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a74063c90 msgr2=0x7f4a740ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.970+0000 7f4a7a919700 1 --2- 172.21.15.7:0/3415301609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a74063c90 0x7f4a740ce6b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009e60 tx=0x7f4a6c009780 comp rx=0 tx=0).stop 2024-01-27T02:32:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.970+0000 7f4a7a919700 1 -- 172.21.15.7:0/3415301609 shutdown_connections 2024-01-27T02:32:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.970+0000 7f4a7a919700 1 --2- 172.21.15.7:0/3415301609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a74063c90 0x7f4a740ce6b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.970+0000 7f4a7a919700 1 -- 172.21.15.7:0/3415301609 >> 172.21.15.7:0/3415301609 conn(0x7f4a74068480 msgr2=0x7f4a740cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.970+0000 7f4a7a919700 1 -- 172.21.15.7:0/3415301609 shutdown_connections 2024-01-27T02:32:57.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.970+0000 7f4a7a919700 1 -- 172.21.15.7:0/3415301609 wait complete. 2024-01-27T02:32:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.971+0000 7f4a7a919700 1 Processor -- start 2024-01-27T02:32:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.972+0000 7f4a7a919700 1 -- start start 2024-01-27T02:32:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.972+0000 7f4a7a919700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a74063c90 0x7f4a74145720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.972+0000 7f4a7a919700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a74154550 0x7f4a7415dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.972+0000 7f4a7a919700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a74154920 0x7f4a740a7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.972+0000 7f4a7a919700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a74145c60 con 0x7f4a74154550 2024-01-27T02:32:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.972+0000 7f4a7a919700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7415f140 con 0x7f4a74063c90 2024-01-27T02:32:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.972+0000 7f4a7a919700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7415f5d0 con 0x7f4a74154920 2024-01-27T02:32:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.972+0000 7f4a737fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a74154550 0x7f4a7415dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.972+0000 7f4a737fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a74154550 0x7f4a7415dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38670/0 (socket says 172.21.15.7:38670) 2024-01-27T02:32:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.972+0000 7f4a737fe700 1 -- 172.21.15.7:0/635283496 learned_addr learned my addr 172.21.15.7:0/635283496 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.973+0000 7f4a78eb6700 1 --2- 172.21.15.7:0/635283496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a74154920 0x7f4a740a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.973+0000 7f4a73fff700 1 --2- 172.21.15.7:0/635283496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a74063c90 0x7f4a74145720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.973+0000 7f4a737fe700 1 -- 172.21.15.7:0/635283496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a74154920 msgr2=0x7f4a740a7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.973+0000 7f4a737fe700 1 --2- 172.21.15.7:0/635283496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a74154920 0x7f4a740a7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.973+0000 7f4a737fe700 1 -- 172.21.15.7:0/635283496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a74063c90 msgr2=0x7f4a74145720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.973+0000 7f4a737fe700 1 --2- 172.21.15.7:0/635283496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a74063c90 0x7f4a74145720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.973+0000 7f4a737fe700 1 -- 172.21.15.7:0/635283496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6c009a20 con 0x7f4a74154550 2024-01-27T02:32:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.973+0000 7f4a73fff700 1 --2- 172.21.15.7:0/635283496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a74063c90 0x7f4a74145720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.973+0000 7f4a78eb6700 1 --2- 172.21.15.7:0/635283496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a74154920 0x7f4a740a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.973+0000 7f4a737fe700 1 --2- 172.21.15.7:0/635283496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a74154550 0x7f4a7415dac0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800c930 tx=0x7f4a68001560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.973+0000 7f4a5bfff700 1 -- 172.21.15.7:0/635283496 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6800ce70 con 0x7f4a74154550 2024-01-27T02:32:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.973+0000 7f4a5bfff700 1 -- 172.21.15.7:0/635283496 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a68005af0 con 0x7f4a74154550 2024-01-27T02:32:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.973+0000 7f4a7a919700 1 -- 172.21.15.7:0/635283496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a7415e530 con 0x7f4a74154550 2024-01-27T02:32:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.973+0000 7f4a5bfff700 1 -- 172.21.15.7:0/635283496 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a680070f0 con 0x7f4a74154550 2024-01-27T02:32:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.973+0000 7f4a7a919700 1 -- 172.21.15.7:0/635283496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a7415eaf0 con 0x7f4a74154550 2024-01-27T02:32:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.974+0000 7f4a7a919700 1 -- 172.21.15.7:0/635283496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a7415f2b0 con 0x7f4a74154550 2024-01-27T02:32:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.975+0000 7f4a5bfff700 1 -- 172.21.15.7:0/635283496 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4a680074e0 con 0x7f4a74154550 2024-01-27T02:32:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.975+0000 7f4a5bfff700 1 --2- 172.21.15.7:0/635283496 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a60066f30 0x7f4a600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.975+0000 7f4a5bfff700 1 -- 172.21.15.7:0/635283496 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4a68084280 con 0x7f4a74154550 2024-01-27T02:32:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.975+0000 7f4a73fff700 1 --2- 172.21.15.7:0/635283496 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a60066f30 0x7f4a600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.976+0000 7f4a73fff700 1 --2- 172.21.15.7:0/635283496 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a60066f30 0x7f4a600693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4a64003990 tx=0x7f4a64009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:57.980+0000 7f4a5bfff700 1 -- 172.21.15.7:0/635283496 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a68051570 con 0x7f4a74154550 2024-01-27T02:32:58.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.232+0000 7f4a7a919700 1 -- 172.21.15.7:0/635283496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4a74072060 con 0x7f4a74154550 2024-01-27T02:32:58.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.814+0000 7f4a5bfff700 1 -- 172.21.15.7:0/635283496 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4a68054b90 con 0x7f4a74154550 2024-01-27T02:32:58.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.817+0000 7f4a7a919700 1 -- 172.21.15.7:0/635283496 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a60066f30 msgr2=0x7f4a600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.817+0000 7f4a7a919700 1 --2- 172.21.15.7:0/635283496 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a60066f30 0x7f4a600693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4a64003990 tx=0x7f4a64009040 comp rx=0 tx=0).stop 2024-01-27T02:32:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.817+0000 7f4a7a919700 1 -- 172.21.15.7:0/635283496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a74154550 msgr2=0x7f4a7415dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.817+0000 7f4a7a919700 1 --2- 172.21.15.7:0/635283496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a74154550 0x7f4a7415dac0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800c930 tx=0x7f4a68001560 comp rx=0 tx=0).stop 2024-01-27T02:32:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.817+0000 7f4a7a919700 1 -- 172.21.15.7:0/635283496 shutdown_connections 2024-01-27T02:32:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.817+0000 7f4a7a919700 1 --2- 172.21.15.7:0/635283496 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a60066f30 0x7f4a600693e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.817+0000 7f4a7a919700 1 --2- 172.21.15.7:0/635283496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a74154920 0x7f4a740a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.817+0000 7f4a7a919700 1 --2- 172.21.15.7:0/635283496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a74063c90 0x7f4a74145720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.817+0000 7f4a7a919700 1 --2- 172.21.15.7:0/635283496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a74154550 0x7f4a7415dac0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.817+0000 7f4a7a919700 1 -- 172.21.15.7:0/635283496 >> 172.21.15.7:0/635283496 conn(0x7f4a74068480 msgr2=0x7f4a74063580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.817+0000 7f4a7a919700 1 -- 172.21.15.7:0/635283496 shutdown_connections 2024-01-27T02:32:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.817+0000 7f4a7a919700 1 -- 172.21.15.7:0/635283496 wait complete. 2024-01-27T02:32:58.833 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:32:58.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.967+0000 7f7d60360700 1 Processor -- start 2024-01-27T02:32:58.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.967+0000 7f7d60360700 1 -- start start 2024-01-27T02:32:58.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.968+0000 7f7d60360700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7d58063c90 0x7f7d580c8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:58.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.968+0000 7f7d60360700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d580cb7a0 con 0x7f7d581438f0 2024-01-27T02:32:58.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.968+0000 7f7d60360700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d580cb9f0 con 0x7f7d58143520 2024-01-27T02:32:58.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.968+0000 7f7d60360700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d580cbcd0 con 0x7f7d58063c90 2024-01-27T02:32:58.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.968+0000 7f7d5e8fd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7d581438f0 0x7f7d580c8750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49220/0 (socket says 172.21.15.7:49220) 2024-01-27T02:32:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.968+0000 7f7d5e8fd700 1 -- 172.21.15.7:0/1388708006 learned_addr learned my addr 172.21.15.7:0/1388708006 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.968+0000 7f7d5d8fb700 1 --2- 172.21.15.7:0/1388708006 >> v2:172.21.15.188:3300/0 conn(0x7f7d58063c90 0x7f7d580c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.968+0000 7f7d5d0fa700 1 -- 172.21.15.7:0/1388708006 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508562134 0 0) 0x7f7d580cb7a0 con 0x7f7d581438f0 2024-01-27T02:32:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.968+0000 7f7d5d0fa700 1 -- 172.21.15.7:0/1388708006 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d44003570 con 0x7f7d581438f0 2024-01-27T02:32:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5d0fa700 1 -- 172.21.15.7:0/1388708006 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4095675522 0 0) 0x7f7d580cb9f0 con 0x7f7d58143520 2024-01-27T02:32:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5d0fa700 1 -- 172.21.15.7:0/1388708006 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d580cb7a0 con 0x7f7d58143520 2024-01-27T02:32:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5d0fa700 1 -- 172.21.15.7:0/1388708006 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3376868615 0 0) 0x7f7d44003570 con 0x7f7d581438f0 2024-01-27T02:32:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5d0fa700 1 -- 172.21.15.7:0/1388708006 >> v2:172.21.15.188:3300/0 conn(0x7f7d58063c90 msgr2=0x7f7d580c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5d0fa700 1 --2- 172.21.15.7:0/1388708006 >> v2:172.21.15.188:3300/0 conn(0x7f7d58063c90 0x7f7d580c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5d0fa700 1 -- 172.21.15.7:0/1388708006 >> v1:172.21.15.188:6789/0 conn(0x7f7d58143520 legacy=0x7f7d58063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5d0fa700 1 -- 172.21.15.7:0/1388708006 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d580cbfb0 con 0x7f7d581438f0 2024-01-27T02:32:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5d0fa700 1 -- 172.21.15.7:0/1388708006 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f7d500017f0 con 0x7f7d581438f0 2024-01-27T02:32:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5d8fb700 1 --2- 172.21.15.7:0/1388708006 >> v2:172.21.15.188:3300/0 conn(0x7f7d58063c90 0x7f7d580c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:32:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5d0fa700 1 -- 172.21.15.7:0/1388708006 >> v1:172.21.15.7:6789/0 conn(0x7f7d581438f0 legacy=0x7f7d580c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5d0fa700 1 --2- 172.21.15.7:0/1388708006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d44003bf0 0x7f7d44003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5d0fa700 1 --2- 172.21.15.7:0/1388708006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d44004810 0x7f7d44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5d0fa700 1 --2- 172.21.15.7:0/1388708006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d44005230 0x7f7d4400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5d0fa700 1 -- 172.21.15.7:0/1388708006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d580cbcd0 con 0x7f7d44005230 2024-01-27T02:32:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5d0fa700 1 -- 172.21.15.7:0/1388708006 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4400c200 con 0x7f7d44004810 2024-01-27T02:32:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5d0fa700 1 -- 172.21.15.7:0/1388708006 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4400c3c0 con 0x7f7d44003bf0 2024-01-27T02:32:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5e8fd700 1 --2- 172.21.15.7:0/1388708006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d44005230 0x7f7d4400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5d8fb700 1 --2- 172.21.15.7:0/1388708006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d44004810 0x7f7d44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5e0fc700 1 --2- 172.21.15.7:0/1388708006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d44003bf0 0x7f7d44003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5e8fd700 1 -- 172.21.15.7:0/1388708006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d44003bf0 msgr2=0x7f7d44003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5e8fd700 1 --2- 172.21.15.7:0/1388708006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d44003bf0 0x7f7d44003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5e8fd700 1 -- 172.21.15.7:0/1388708006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d44004810 msgr2=0x7f7d44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5e8fd700 1 --2- 172.21.15.7:0/1388708006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d44004810 0x7f7d44004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5e8fd700 1 -- 172.21.15.7:0/1388708006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d44003570 con 0x7f7d44005230 2024-01-27T02:32:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.969+0000 7f7d5e8fd700 1 -- 172.21.15.7:0/1388708006 reap_dead start 2024-01-27T02:32:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.970+0000 7f7d5e8fd700 1 --2- 172.21.15.7:0/1388708006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d44005230 0x7f7d4400ba60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000a8c0 tx=0x7f7d50002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7290cb03391ee8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.970+0000 7f7d5d0fa700 1 -- 172.21.15.7:0/1388708006 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50012400 con 0x7f7d44005230 2024-01-27T02:32:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.970+0000 7f7d5d0fa700 1 -- 172.21.15.7:0/1388708006 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d50012580 con 0x7f7d44005230 2024-01-27T02:32:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.970+0000 7f7d5d0fa700 1 -- 172.21.15.7:0/1388708006 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50012940 con 0x7f7d44005230 2024-01-27T02:32:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.970+0000 7f7d60360700 1 -- 172.21.15.7:0/1388708006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d44005230 msgr2=0x7f7d4400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.970+0000 7f7d60360700 1 --2- 172.21.15.7:0/1388708006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d44005230 0x7f7d4400ba60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000a8c0 tx=0x7f7d50002190 comp rx=0 tx=0).stop 2024-01-27T02:32:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.970+0000 7f7d60360700 1 -- 172.21.15.7:0/1388708006 shutdown_connections 2024-01-27T02:32:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.970+0000 7f7d60360700 1 --2- 172.21.15.7:0/1388708006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d44005230 0x7f7d4400ba60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.970+0000 7f7d60360700 1 -- 172.21.15.7:0/1388708006 >> 172.21.15.7:0/1388708006 conn(0x7f7d580cdbd0 msgr2=0x7f7d580cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.970+0000 7f7d60360700 1 -- 172.21.15.7:0/1388708006 shutdown_connections 2024-01-27T02:32:58.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.970+0000 7f7d60360700 1 -- 172.21.15.7:0/1388708006 wait complete. 2024-01-27T02:32:58.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.972+0000 7f7d60360700 1 Processor -- start 2024-01-27T02:32:58.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.972+0000 7f7d60360700 1 -- start start 2024-01-27T02:32:58.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.973+0000 7f7d60360700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d44005230 0x7f7d580ad440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:58.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.973+0000 7f7d60360700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d580aee00 0x7f7d5815cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:58.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.973+0000 7f7d60360700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d580b2b00 0x7f7d580b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:58.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.973+0000 7f7d60360700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d580b34e0 con 0x7f7d580b2b00 2024-01-27T02:32:58.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.973+0000 7f7d60360700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5815a800 con 0x7f7d580aee00 2024-01-27T02:32:58.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.973+0000 7f7d60360700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d580ad980 con 0x7f7d44005230 2024-01-27T02:32:58.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.973+0000 7f7d5e8fd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d580b2b00 0x7f7d580b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:58.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.973+0000 7f7d5e8fd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d580b2b00 0x7f7d580b2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38682/0 (socket says 172.21.15.7:38682) 2024-01-27T02:32:58.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.973+0000 7f7d5e8fd700 1 -- 172.21.15.7:0/4239932238 learned_addr learned my addr 172.21.15.7:0/4239932238 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:58.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.973+0000 7f7d5d8fb700 1 --2- 172.21.15.7:0/4239932238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d580aee00 0x7f7d5815cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:58.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.973+0000 7f7d5e0fc700 1 --2- 172.21.15.7:0/4239932238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d44005230 0x7f7d580ad440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:58.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.974+0000 7f7d5e8fd700 1 -- 172.21.15.7:0/4239932238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d44005230 msgr2=0x7f7d580ad440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:58.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.974+0000 7f7d5e8fd700 1 --2- 172.21.15.7:0/4239932238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d44005230 0x7f7d580ad440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.974+0000 7f7d5e8fd700 1 -- 172.21.15.7:0/4239932238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d580aee00 msgr2=0x7f7d5815cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.974+0000 7f7d5e8fd700 1 --2- 172.21.15.7:0/4239932238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d580aee00 0x7f7d5815cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.974+0000 7f7d5e8fd700 1 -- 172.21.15.7:0/4239932238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d580afb80 con 0x7f7d580b2b00 2024-01-27T02:32:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.974+0000 7f7d5d8fb700 1 --2- 172.21.15.7:0/4239932238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d580aee00 0x7f7d5815cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.974+0000 7f7d5e8fd700 1 --2- 172.21.15.7:0/4239932238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d580b2b00 0x7f7d580b2fa0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7d50000f80 tx=0x7f7d50016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.974+0000 7f7d4e7fc700 1 -- 172.21.15.7:0/4239932238 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50022070 con 0x7f7d580b2b00 2024-01-27T02:32:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.974+0000 7f7d4e7fc700 1 -- 172.21.15.7:0/4239932238 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d50001eb0 con 0x7f7d580b2b00 2024-01-27T02:32:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.974+0000 7f7d60360700 1 -- 172.21.15.7:0/4239932238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d580aba10 con 0x7f7d580b2b00 2024-01-27T02:32:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.974+0000 7f7d4e7fc700 1 -- 172.21.15.7:0/4239932238 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d500013e0 con 0x7f7d580b2b00 2024-01-27T02:32:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.974+0000 7f7d60360700 1 -- 172.21.15.7:0/4239932238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d580af5c0 con 0x7f7d580b2b00 2024-01-27T02:32:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.975+0000 7f7d60360700 1 -- 172.21.15.7:0/4239932238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d5815a990 con 0x7f7d580b2b00 2024-01-27T02:32:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.975+0000 7f7d4e7fc700 1 -- 172.21.15.7:0/4239932238 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7d50004bb0 con 0x7f7d580b2b00 2024-01-27T02:32:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.976+0000 7f7d4e7fc700 1 --2- 172.21.15.7:0/4239932238 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7d3c066f30 0x7f7d3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.976+0000 7f7d4e7fc700 1 -- 172.21.15.7:0/4239932238 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7d50086460 con 0x7f7d580b2b00 2024-01-27T02:32:58.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.976+0000 7f7d5e0fc700 1 --2- 172.21.15.7:0/4239932238 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7d3c066f30 0x7f7d3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:58.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.977+0000 7f7d5e0fc700 1 --2- 172.21.15.7:0/4239932238 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7d3c066f30 0x7f7d3c0693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7d54001130 tx=0x7f7d54009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:58.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:58.980+0000 7f7d4e7fc700 1 -- 172.21.15.7:0/4239932238 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d500536d0 con 0x7f7d580b2b00 2024-01-27T02:32:59.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.229+0000 7f7d60360700 1 -- 172.21.15.7:0/4239932238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d5815a990 con 0x7f7d580b2b00 2024-01-27T02:32:59.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.229+0000 7f7d4e7fc700 1 -- 172.21.15.7:0/4239932238 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f7d50012900 con 0x7f7d580b2b00 2024-01-27T02:32:59.231 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:59.231 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4864,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/892460188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":892460188},{"type":"v1","addr":"172.21.15.188:6837","nonce":892460188}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4867,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1657319349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1657319349},{"type":"v1","addr":"172.21.15.188:6836","nonce":1657319349}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4879,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2190571534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2190571534},{"type":"v1","addr":"172.21.15.7:6834","nonce":2190571534}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:32:54.849803+0000","modified":"2024-01-27T02:32:58.809118+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4882},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4882":{"gid":4882,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6835/1734594727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1734594727},{"type":"v1","addr":"172.21.15.7:6835","nonce":1734594727}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-01-27T02:32:59.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.232+0000 7f7d60360700 1 -- 172.21.15.7:0/4239932238 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7d3c066f30 msgr2=0x7f7d3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:59.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.232+0000 7f7d60360700 1 --2- 172.21.15.7:0/4239932238 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7d3c066f30 0x7f7d3c0693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7d54001130 tx=0x7f7d54009040 comp rx=0 tx=0).stop 2024-01-27T02:32:59.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.232+0000 7f7d60360700 1 -- 172.21.15.7:0/4239932238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d580b2b00 msgr2=0x7f7d580b2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:59.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.232+0000 7f7d60360700 1 --2- 172.21.15.7:0/4239932238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d580b2b00 0x7f7d580b2fa0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7d50000f80 tx=0x7f7d50016000 comp rx=0 tx=0).stop 2024-01-27T02:32:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.232+0000 7f7d60360700 1 -- 172.21.15.7:0/4239932238 shutdown_connections 2024-01-27T02:32:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.232+0000 7f7d60360700 1 --2- 172.21.15.7:0/4239932238 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7d3c066f30 0x7f7d3c0693e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.232+0000 7f7d60360700 1 --2- 172.21.15.7:0/4239932238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d580b2b00 0x7f7d580b2fa0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.232+0000 7f7d60360700 1 --2- 172.21.15.7:0/4239932238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d44005230 0x7f7d580ad440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.232+0000 7f7d60360700 1 --2- 172.21.15.7:0/4239932238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d580aee00 0x7f7d5815cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.232+0000 7f7d60360700 1 -- 172.21.15.7:0/4239932238 >> 172.21.15.7:0/4239932238 conn(0x7f7d580cdbd0 msgr2=0x7f7d580cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.232+0000 7f7d60360700 1 -- 172.21.15.7:0/4239932238 shutdown_connections 2024-01-27T02:32:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.232+0000 7f7d60360700 1 -- 172.21.15.7:0/4239932238 wait complete. 2024-01-27T02:32:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2024-01-27T02:32:59.240 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-01-27T02:32:59.237+0000 7f7052291700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-27T02:32:59.246 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:32:59.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.377+0000 7f708cb86700 1 Processor -- start 2024-01-27T02:32:59.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.377+0000 7f708cb86700 1 -- start start 2024-01-27T02:32:59.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.378+0000 7f708cb86700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7088159f60 0x7f70880c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:59.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.378+0000 7f708cb86700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f70880ca150 0x7f7088063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:59.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.378+0000 7f708cb86700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f708815ead0 con 0x7f708815a330 2024-01-27T02:32:59.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.378+0000 7f708cb86700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f708815df00 con 0x7f7088159f60 2024-01-27T02:32:59.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.378+0000 7f708cb86700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f708815e1a0 con 0x7f70880ca150 2024-01-27T02:32:59.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.378+0000 7f708659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7088159f60 0x7f70880c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:59.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.378+0000 7f7086d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f708815a330 0x7f708806a400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49236/0 (socket says 172.21.15.7:49236) 2024-01-27T02:32:59.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.378+0000 7f7086d9d700 1 -- 172.21.15.7:0/3770718135 learned_addr learned my addr 172.21.15.7:0/3770718135 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:59.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.378+0000 7f7085d9b700 1 --2- 172.21.15.7:0/3770718135 >> v2:172.21.15.188:3300/0 conn(0x7f70880ca150 0x7f7088063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:59.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.379+0000 7f708659c700 1 -- 172.21.15.7:0/3770718135 >> v2:172.21.15.188:3300/0 conn(0x7f70880ca150 msgr2=0x7f7088063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:59.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.379+0000 7f708659c700 1 --2- 172.21.15.7:0/3770718135 >> v2:172.21.15.188:3300/0 conn(0x7f70880ca150 0x7f7088063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.379+0000 7f708659c700 1 -- 172.21.15.7:0/3770718135 >> v1:172.21.15.7:6789/0 conn(0x7f708815a330 legacy=0x7f708806a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:59.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.379+0000 7f708659c700 1 -- 172.21.15.7:0/3770718135 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708815e480 con 0x7f7088159f60 2024-01-27T02:32:59.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.379+0000 7f708659c700 1 --2- 172.21.15.7:0/3770718135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7088159f60 0x7f70880c9c10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f70880c4b60 tx=0x7f70780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e82cd0ac0cf82d server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:59.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.379+0000 7f708559a700 1 -- 172.21.15.7:0/3770718135 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707800b6b0 con 0x7f7088159f60 2024-01-27T02:32:59.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.379+0000 7f708559a700 1 -- 172.21.15.7:0/3770718135 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707800b810 con 0x7f7088159f60 2024-01-27T02:32:59.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.379+0000 7f708559a700 1 -- 172.21.15.7:0/3770718135 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707800bbb0 con 0x7f7088159f60 2024-01-27T02:32:59.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.380+0000 7f708cb86700 1 -- 172.21.15.7:0/3770718135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7088159f60 msgr2=0x7f70880c9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:59.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.380+0000 7f708cb86700 1 --2- 172.21.15.7:0/3770718135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7088159f60 0x7f70880c9c10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f70880c4b60 tx=0x7f70780091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.380+0000 7f708cb86700 1 -- 172.21.15.7:0/3770718135 shutdown_connections 2024-01-27T02:32:59.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.380+0000 7f708cb86700 1 --2- 172.21.15.7:0/3770718135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7088159f60 0x7f70880c9c10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.380+0000 7f708cb86700 1 --2- 172.21.15.7:0/3770718135 >> v2:172.21.15.188:3300/0 conn(0x7f70880ca150 0x7f7088063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.380+0000 7f708cb86700 1 -- 172.21.15.7:0/3770718135 >> 172.21.15.7:0/3770718135 conn(0x7f7088068480 msgr2=0x7f70880cdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.380+0000 7f708cb86700 1 -- 172.21.15.7:0/3770718135 shutdown_connections 2024-01-27T02:32:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.380+0000 7f708cb86700 1 -- 172.21.15.7:0/3770718135 wait complete. 2024-01-27T02:32:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.381+0000 7f708cb86700 1 Processor -- start 2024-01-27T02:32:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.381+0000 7f708cb86700 1 -- start start 2024-01-27T02:32:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.382+0000 7f708cb86700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70880ca150 0x7f708816a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.382+0000 7f708cb86700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7088159f60 0x7f70880cc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.382+0000 7f708cb86700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f708815a330 0x7f7088156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.382+0000 7f708cb86700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708816b9f0 con 0x7f70880ca150 2024-01-27T02:32:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.382+0000 7f708cb86700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70881581c0 con 0x7f708815a330 2024-01-27T02:32:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.382+0000 7f708cb86700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70880c6450 con 0x7f7088159f60 2024-01-27T02:32:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.382+0000 7f708659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70880ca150 0x7f708816a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.382+0000 7f708659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70880ca150 0x7f708816a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38700/0 (socket says 172.21.15.7:38700) 2024-01-27T02:32:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.382+0000 7f708659c700 1 -- 172.21.15.7:0/4277081457 learned_addr learned my addr 172.21.15.7:0/4277081457 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.382+0000 7f7085d9b700 1 --2- 172.21.15.7:0/4277081457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7088159f60 0x7f70880cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.382+0000 7f7086d9d700 1 --2- 172.21.15.7:0/4277081457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f708815a330 0x7f7088156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.382+0000 7f708659c700 1 -- 172.21.15.7:0/4277081457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7088159f60 msgr2=0x7f70880cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.382+0000 7f708659c700 1 --2- 172.21.15.7:0/4277081457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7088159f60 0x7f70880cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.382+0000 7f708659c700 1 -- 172.21.15.7:0/4277081457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f708815a330 msgr2=0x7f7088156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.382+0000 7f708659c700 1 --2- 172.21.15.7:0/4277081457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f708815a330 0x7f7088156c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.382+0000 7f708659c700 1 -- 172.21.15.7:0/4277081457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7078009420 con 0x7f70880ca150 2024-01-27T02:32:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.382+0000 7f7086d9d700 1 --2- 172.21.15.7:0/4277081457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f708815a330 0x7f7088156c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:32:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.382+0000 7f708659c700 1 --2- 172.21.15.7:0/4277081457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70880ca150 0x7f708816a490 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f708806cea0 tx=0x7f7078017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.382+0000 7f707e7fc700 1 -- 172.21.15.7:0/4277081457 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7078011420 con 0x7f70880ca150 2024-01-27T02:32:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.383+0000 7f707e7fc700 1 -- 172.21.15.7:0/4277081457 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7078011580 con 0x7f70880ca150 2024-01-27T02:32:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.383+0000 7f708cb86700 1 -- 172.21.15.7:0/4277081457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70880cc710 con 0x7f70880ca150 2024-01-27T02:32:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.383+0000 7f707e7fc700 1 -- 172.21.15.7:0/4277081457 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7078011940 con 0x7f70880ca150 2024-01-27T02:32:59.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.383+0000 7f708cb86700 1 -- 172.21.15.7:0/4277081457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70880cccd0 con 0x7f70880ca150 2024-01-27T02:32:59.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.384+0000 7f708cb86700 1 -- 172.21.15.7:0/4277081457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70880c6220 con 0x7f70880ca150 2024-01-27T02:32:59.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.384+0000 7f707e7fc700 1 -- 172.21.15.7:0/4277081457 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7078011d30 con 0x7f70880ca150 2024-01-27T02:32:59.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.385+0000 7f707e7fc700 1 --2- 172.21.15.7:0/4277081457 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7068066f30 0x7f70680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:59.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.385+0000 7f707e7fc700 1 -- 172.21.15.7:0/4277081457 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7078085a60 con 0x7f70880ca150 2024-01-27T02:32:59.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.385+0000 7f7085d9b700 1 --2- 172.21.15.7:0/4277081457 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7068066f30 0x7f70680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:59.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.385+0000 7f7085d9b700 1 --2- 172.21.15.7:0/4277081457 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7068066f30 0x7f70680693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f70880acb70 tx=0x7f7070008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:59.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.389+0000 7f707e7fc700 1 -- 172.21.15.7:0/4277081457 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70880c6220 con 0x7f70880ca150 2024-01-27T02:32:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.592+0000 7f708cb86700 1 -- 172.21.15.7:0/4277081457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70880c6730 con 0x7f70880ca150 2024-01-27T02:32:59.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.593+0000 7f707e7fc700 1 -- 172.21.15.7:0/4277081457 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f7078015090 con 0x7f70880ca150 2024-01-27T02:32:59.593 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:59.593 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":32,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:32:56.417162+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:32:52.897548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-27T02:32:53.822754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:32:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.595+0000 7f708cb86700 1 -- 172.21.15.7:0/4277081457 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7068066f30 msgr2=0x7f70680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.595+0000 7f708cb86700 1 --2- 172.21.15.7:0/4277081457 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7068066f30 0x7f70680693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f70880acb70 tx=0x7f7070008040 comp rx=0 tx=0).stop 2024-01-27T02:32:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.595+0000 7f708cb86700 1 -- 172.21.15.7:0/4277081457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70880ca150 msgr2=0x7f708816a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.595+0000 7f708cb86700 1 --2- 172.21.15.7:0/4277081457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70880ca150 0x7f708816a490 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f708806cea0 tx=0x7f7078017000 comp rx=0 tx=0).stop 2024-01-27T02:32:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.595+0000 7f708cb86700 1 -- 172.21.15.7:0/4277081457 shutdown_connections 2024-01-27T02:32:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.595+0000 7f708cb86700 1 --2- 172.21.15.7:0/4277081457 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7068066f30 0x7f70680693e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.595+0000 7f708cb86700 1 --2- 172.21.15.7:0/4277081457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f708815a330 0x7f7088156c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.595+0000 7f708cb86700 1 --2- 172.21.15.7:0/4277081457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70880ca150 0x7f708816a490 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.595+0000 7f708cb86700 1 --2- 172.21.15.7:0/4277081457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7088159f60 0x7f70880cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.595+0000 7f708cb86700 1 -- 172.21.15.7:0/4277081457 >> 172.21.15.7:0/4277081457 conn(0x7f7088068480 msgr2=0x7f7088158f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.595+0000 7f708cb86700 1 -- 172.21.15.7:0/4277081457 shutdown_connections 2024-01-27T02:32:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.595+0000 7f708cb86700 1 -- 172.21.15.7:0/4277081457 wait complete. 2024-01-27T02:32:59.609 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-27T02:32:59.610 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:32:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.741+0000 7fd82d0ac700 1 Processor -- start 2024-01-27T02:32:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.741+0000 7fd82d0ac700 1 -- start start 2024-01-27T02:32:59.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.742+0000 7fd82d0ac700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd8280a7fe0 0x7fd82806f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:59.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.742+0000 7fd82d0ac700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd82815dfb0 0x7fd82806f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:59.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.742+0000 7fd82d0ac700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd82815a880 con 0x7fd8280a7c10 2024-01-27T02:32:59.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.742+0000 7fd82d0ac700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8280afde0 con 0x7fd8280a7fe0 2024-01-27T02:32:59.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.742+0000 7fd82d0ac700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8280af950 con 0x7fd82815dfb0 2024-01-27T02:32:59.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.742+0000 7fd826d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd8280a7c10 0x7fd8280ae340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49244/0 (socket says 172.21.15.7:49244) 2024-01-27T02:32:59.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.742+0000 7fd826d9d700 1 -- 172.21.15.7:0/2579809486 learned_addr learned my addr 172.21.15.7:0/2579809486 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:32:59.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.742+0000 7fd82659c700 1 --2- 172.21.15.7:0/2579809486 >> v2:172.21.15.188:3300/0 conn(0x7fd8280a7fe0 0x7fd82806f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:59.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.742+0000 7fd82759e700 1 --2- 172.21.15.7:0/2579809486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd82815dfb0 0x7fd82806f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:59.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.742+0000 7fd825d9b700 1 -- 172.21.15.7:0/2579809486 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2838508288 0 0) 0x7fd82815a880 con 0x7fd8280a7c10 2024-01-27T02:32:59.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.742+0000 7fd825d9b700 1 -- 172.21.15.7:0/2579809486 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd810003570 con 0x7fd8280a7c10 2024-01-27T02:32:59.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.742+0000 7fd82659c700 1 -- 172.21.15.7:0/2579809486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd82815dfb0 msgr2=0x7fd82806f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:32:59.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.742+0000 7fd82659c700 1 --2- 172.21.15.7:0/2579809486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd82815dfb0 0x7fd82806f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.742+0000 7fd82659c700 1 -- 172.21.15.7:0/2579809486 >> v1:172.21.15.7:6789/0 conn(0x7fd8280a7c10 legacy=0x7fd8280ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:59.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.742+0000 7fd82659c700 1 -- 172.21.15.7:0/2579809486 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8280aed40 con 0x7fd8280a7fe0 2024-01-27T02:32:59.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.743+0000 7fd82659c700 1 --2- 172.21.15.7:0/2579809486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8280a7fe0 0x7fd82806f360 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd81c0098e0 tx=0x7fd81c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=221e1b3a44a3330f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:59.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.743+0000 7fd825d9b700 1 -- 172.21.15.7:0/2579809486 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c011630 con 0x7fd8280a7fe0 2024-01-27T02:32:59.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.743+0000 7fd825d9b700 1 -- 172.21.15.7:0/2579809486 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81c011790 con 0x7fd8280a7fe0 2024-01-27T02:32:59.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.743+0000 7fd825d9b700 1 -- 172.21.15.7:0/2579809486 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c011b30 con 0x7fd8280a7fe0 2024-01-27T02:32:59.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.743+0000 7fd82d0ac700 1 -- 172.21.15.7:0/2579809486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8280a7fe0 msgr2=0x7fd82806f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:59.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.743+0000 7fd82d0ac700 1 --2- 172.21.15.7:0/2579809486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8280a7fe0 0x7fd82806f360 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd81c0098e0 tx=0x7fd81c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.743+0000 7fd82d0ac700 1 -- 172.21.15.7:0/2579809486 shutdown_connections 2024-01-27T02:32:59.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.743+0000 7fd82d0ac700 1 --2- 172.21.15.7:0/2579809486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd82815dfb0 0x7fd82806f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.743+0000 7fd82d0ac700 1 --2- 172.21.15.7:0/2579809486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8280a7fe0 0x7fd82806f360 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.743+0000 7fd82d0ac700 1 -- 172.21.15.7:0/2579809486 >> 172.21.15.7:0/2579809486 conn(0x7fd8280679b0 msgr2=0x7fd828067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:59.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.743+0000 7fd82d0ac700 1 -- 172.21.15.7:0/2579809486 shutdown_connections 2024-01-27T02:32:59.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.743+0000 7fd82d0ac700 1 -- 172.21.15.7:0/2579809486 wait complete. 2024-01-27T02:32:59.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.745+0000 7fd82d0ac700 1 Processor -- start 2024-01-27T02:32:59.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.745+0000 7fd82d0ac700 1 -- start start 2024-01-27T02:32:59.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd82d0ac700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8280a7c10 0x7fd828144350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:59.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd82d0ac700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8280a7fe0 0x7fd82815b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:59.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd82d0ac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd82815dfb0 0x7fd828144a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:59.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd82d0ac700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82815cc70 con 0x7fd82815dfb0 2024-01-27T02:32:59.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd82d0ac700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82816a920 con 0x7fd8280a7c10 2024-01-27T02:32:59.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd82d0ac700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd82815c060 con 0x7fd8280a7fe0 2024-01-27T02:32:59.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd82759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd82815dfb0 0x7fd828144a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:59.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd82759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd82815dfb0 0x7fd828144a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38706/0 (socket says 172.21.15.7:38706) 2024-01-27T02:32:59.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd82759e700 1 -- 172.21.15.7:0/2919766910 learned_addr learned my addr 172.21.15.7:0/2919766910 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:32:59.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd82659c700 1 --2- 172.21.15.7:0/2919766910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8280a7fe0 0x7fd82815b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:59.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd826d9d700 1 --2- 172.21.15.7:0/2919766910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8280a7c10 0x7fd828144350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:59.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd82759e700 1 -- 172.21.15.7:0/2919766910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8280a7fe0 msgr2=0x7fd82815b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:59.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd82759e700 1 --2- 172.21.15.7:0/2919766910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8280a7fe0 0x7fd82815b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd82759e700 1 -- 172.21.15.7:0/2919766910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8280a7c10 msgr2=0x7fd828144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:59.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd82759e700 1 --2- 172.21.15.7:0/2919766910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8280a7c10 0x7fd828144350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd82759e700 1 -- 172.21.15.7:0/2919766910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c009420 con 0x7fd82815dfb0 2024-01-27T02:32:59.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd82659c700 1 --2- 172.21.15.7:0/2919766910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8280a7fe0 0x7fd82815b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:59.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd826d9d700 1 --2- 172.21.15.7:0/2919766910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8280a7c10 0x7fd828144350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:32:59.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd82759e700 1 --2- 172.21.15.7:0/2919766910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd82815dfb0 0x7fd828144a40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd820009d70 tx=0x7fd82000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:59.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd80effd700 1 -- 172.21.15.7:0/2919766910 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82000cb50 con 0x7fd82815dfb0 2024-01-27T02:32:59.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd80effd700 1 -- 172.21.15.7:0/2919766910 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82000f040 con 0x7fd82815dfb0 2024-01-27T02:32:59.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd82d0ac700 1 -- 172.21.15.7:0/2919766910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd82815c340 con 0x7fd82815dfb0 2024-01-27T02:32:59.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd80effd700 1 -- 172.21.15.7:0/2919766910 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820014480 con 0x7fd82815dfb0 2024-01-27T02:32:59.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.746+0000 7fd82d0ac700 1 -- 172.21.15.7:0/2919766910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82815c900 con 0x7fd82815dfb0 2024-01-27T02:32:59.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.747+0000 7fd82d0ac700 1 -- 172.21.15.7:0/2919766910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8280ad370 con 0x7fd82815dfb0 2024-01-27T02:32:59.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.748+0000 7fd80effd700 1 -- 172.21.15.7:0/2919766910 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd820014830 con 0x7fd82815dfb0 2024-01-27T02:32:59.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.748+0000 7fd80effd700 1 --2- 172.21.15.7:0/2919766910 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd814066f30 0x7fd8140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:32:59.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.748+0000 7fd80effd700 1 -- 172.21.15.7:0/2919766910 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd8200851c0 con 0x7fd82815dfb0 2024-01-27T02:32:59.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.749+0000 7fd826d9d700 1 --2- 172.21.15.7:0/2919766910 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd814066f30 0x7fd8140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:32:59.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.749+0000 7fd826d9d700 1 --2- 172.21.15.7:0/2919766910 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd814066f30 0x7fd8140693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd8180058d0 tx=0x7fd818009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:32:59.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.753+0000 7fd80effd700 1 -- 172.21.15.7:0/2919766910 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd820052430 con 0x7fd82815dfb0 2024-01-27T02:32:59.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.952+0000 7fd82d0ac700 1 -- 172.21.15.7:0/2919766910 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd82804c710 con 0x7fd814066f30 2024-01-27T02:32:59.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.960+0000 7fd80effd700 1 -- 172.21.15.7:0/2919766910 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260109 (secure 0 0 0) 0x7fd82804c710 con 0x7fd814066f30 2024-01-27T02:32:59.964 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:32:59.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.963+0000 7fd82d0ac700 1 -- 172.21.15.7:0/2919766910 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd814066f30 msgr2=0x7fd8140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.964+0000 7fd82d0ac700 1 --2- 172.21.15.7:0/2919766910 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd814066f30 0x7fd8140693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd8180058d0 tx=0x7fd818009040 comp rx=0 tx=0).stop 2024-01-27T02:32:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.964+0000 7fd82d0ac700 1 -- 172.21.15.7:0/2919766910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd82815dfb0 msgr2=0x7fd828144a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:32:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.964+0000 7fd82d0ac700 1 --2- 172.21.15.7:0/2919766910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd82815dfb0 0x7fd828144a40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd820009d70 tx=0x7fd82000b040 comp rx=0 tx=0).stop 2024-01-27T02:32:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.964+0000 7fd82d0ac700 1 -- 172.21.15.7:0/2919766910 shutdown_connections 2024-01-27T02:32:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.964+0000 7fd82d0ac700 1 --2- 172.21.15.7:0/2919766910 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd814066f30 0x7fd8140693e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.964+0000 7fd82d0ac700 1 --2- 172.21.15.7:0/2919766910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd82815dfb0 0x7fd828144a40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.964+0000 7fd82d0ac700 1 --2- 172.21.15.7:0/2919766910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8280a7c10 0x7fd828144350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.964+0000 7fd82d0ac700 1 --2- 172.21.15.7:0/2919766910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8280a7fe0 0x7fd82815b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:32:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.964+0000 7fd82d0ac700 1 -- 172.21.15.7:0/2919766910 >> 172.21.15.7:0/2919766910 conn(0x7fd8280679b0 msgr2=0x7fd82806dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:32:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.964+0000 7fd82d0ac700 1 -- 172.21.15.7:0/2919766910 shutdown_connections 2024-01-27T02:32:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:32:59.964+0000 7fd82d0ac700 1 -- 172.21.15.7:0/2919766910 wait complete. 2024-01-27T02:32:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:32:59.978 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2024-01-27T02:32:59.585219+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2192,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560307,"statfs":{"total":1600000000,"available":1597754592,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001475"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.432620+0000","last_change":"2024-01-27T02:32:55.432559+0000","last_active":"2024-01-27T02:32:55.432620+0000","last_peered":"2024-01-27T02:32:55.432620+0000","last_clean":"2024-01-27T02:32:55.432620+0000","last_became_active":"2024-01-27T02:32:55.432290+0000","last_became_peered":"2024-01-27T02:32:55.432290+0000","last_unstale":"2024-01-27T02:32:55.432620+0000","last_undegraded":"2024-01-27T02:32:55.432620+0000","last_fullsized":"2024-01-27T02:32:55.432620+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:28:22.292009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.429694+0000","last_change":"2024-01-27T02:32:55.429005+0000","last_active":"2024-01-27T02:32:55.429694+0000","last_peered":"2024-01-27T02:32:55.429694+0000","last_clean":"2024-01-27T02:32:55.429694+0000","last_became_active":"2024-01-27T02:32:55.428741+0000","last_became_peered":"2024-01-27T02:32:55.428741+0000","last_unstale":"2024-01-27T02:32:55.429694+0000","last_undegraded":"2024-01-27T02:32:55.429694+0000","last_fullsized":"2024-01-27T02:32:55.429694+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:20:24.718650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.428777+0000","last_change":"2024-01-27T02:32:55.428777+0000","last_active":"2024-01-27T02:32:55.428777+0000","last_peered":"2024-01-27T02:32:55.428777+0000","last_clean":"2024-01-27T02:32:55.428777+0000","last_became_active":"2024-01-27T02:32:55.428551+0000","last_became_peered":"2024-01-27T02:32:55.428551+0000","last_unstale":"2024-01-27T02:32:55.428777+0000","last_undegraded":"2024-01-27T02:32:55.428777+0000","last_fullsized":"2024-01-27T02:32:55.428777+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:18:22.974631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.427680+0000","last_change":"2024-01-27T02:32:55.427680+0000","last_active":"2024-01-27T02:32:55.427680+0000","last_peered":"2024-01-27T02:32:55.427680+0000","last_clean":"2024-01-27T02:32:55.427680+0000","last_became_active":"2024-01-27T02:32:55.427171+0000","last_became_peered":"2024-01-27T02:32:55.427171+0000","last_unstale":"2024-01-27T02:32:55.427680+0000","last_undegraded":"2024-01-27T02:32:55.427680+0000","last_fullsized":"2024-01-27T02:32:55.427680+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:41.306732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.424421+0000","last_change":"2024-01-27T02:32:55.424421+0000","last_active":"2024-01-27T02:32:55.424421+0000","last_peered":"2024-01-27T02:32:55.424421+0000","last_clean":"2024-01-27T02:32:55.424421+0000","last_became_active":"2024-01-27T02:32:55.424127+0000","last_became_peered":"2024-01-27T02:32:55.424127+0000","last_unstale":"2024-01-27T02:32:55.424421+0000","last_undegraded":"2024-01-27T02:32:55.424421+0000","last_fullsized":"2024-01-27T02:32:55.424421+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:36:24.317025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.427590+0000","last_change":"2024-01-27T02:32:55.427405+0000","last_active":"2024-01-27T02:32:55.427590+0000","last_peered":"2024-01-27T02:32:55.427590+0000","last_clean":"2024-01-27T02:32:55.427590+0000","last_became_active":"2024-01-27T02:32:55.427005+0000","last_became_peered":"2024-01-27T02:32:55.427005+0000","last_unstale":"2024-01-27T02:32:55.427590+0000","last_undegraded":"2024-01-27T02:32:55.427590+0000","last_fullsized":"2024-01-27T02:32:55.427590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:55:24.921329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.434431+0000","last_change":"2024-01-27T02:32:55.434431+0000","last_active":"2024-01-27T02:32:55.434431+0000","last_peered":"2024-01-27T02:32:55.434431+0000","last_clean":"2024-01-27T02:32:55.434431+0000","last_became_active":"2024-01-27T02:32:55.434017+0000","last_became_peered":"2024-01-27T02:32:55.434017+0000","last_unstale":"2024-01-27T02:32:55.434431+0000","last_undegraded":"2024-01-27T02:32:55.434431+0000","last_fullsized":"2024-01-27T02:32:55.434431+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:20:24.718650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.430034+0000","last_change":"2024-01-27T02:32:55.430034+0000","last_active":"2024-01-27T02:32:55.430034+0000","last_peered":"2024-01-27T02:32:55.430034+0000","last_clean":"2024-01-27T02:32:55.430034+0000","last_became_active":"2024-01-27T02:32:55.428610+0000","last_became_peered":"2024-01-27T02:32:55.428610+0000","last_unstale":"2024-01-27T02:32:55.430034+0000","last_undegraded":"2024-01-27T02:32:55.430034+0000","last_fullsized":"2024-01-27T02:32:55.430034+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:19:26.991510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.425916+0000","last_change":"2024-01-27T02:32:55.425841+0000","last_active":"2024-01-27T02:32:55.425916+0000","last_peered":"2024-01-27T02:32:55.425916+0000","last_clean":"2024-01-27T02:32:55.425916+0000","last_became_active":"2024-01-27T02:32:55.425595+0000","last_became_peered":"2024-01-27T02:32:55.425595+0000","last_unstale":"2024-01-27T02:32:55.425916+0000","last_undegraded":"2024-01-27T02:32:55.425916+0000","last_fullsized":"2024-01-27T02:32:55.425916+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:31:14.098748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.423222+0000","last_change":"2024-01-27T02:32:55.423222+0000","last_active":"2024-01-27T02:32:55.423222+0000","last_peered":"2024-01-27T02:32:55.423222+0000","last_clean":"2024-01-27T02:32:55.423222+0000","last_became_active":"2024-01-27T02:32:55.422972+0000","last_became_peered":"2024-01-27T02:32:55.422972+0000","last_unstale":"2024-01-27T02:32:55.423222+0000","last_undegraded":"2024-01-27T02:32:55.423222+0000","last_fullsized":"2024-01-27T02:32:55.423222+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:23:47.824914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.436132+0000","last_change":"2024-01-27T02:32:55.436132+0000","last_active":"2024-01-27T02:32:55.436132+0000","last_peered":"2024-01-27T02:32:55.436132+0000","last_clean":"2024-01-27T02:32:55.436132+0000","last_became_active":"2024-01-27T02:32:55.435868+0000","last_became_peered":"2024-01-27T02:32:55.435868+0000","last_unstale":"2024-01-27T02:32:55.436132+0000","last_undegraded":"2024-01-27T02:32:55.436132+0000","last_fullsized":"2024-01-27T02:32:55.436132+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:46:09.976295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.433280+0000","last_change":"2024-01-27T02:32:55.433280+0000","last_active":"2024-01-27T02:32:55.433280+0000","last_peered":"2024-01-27T02:32:55.433280+0000","last_clean":"2024-01-27T02:32:55.433280+0000","last_became_active":"2024-01-27T02:32:55.433031+0000","last_became_peered":"2024-01-27T02:32:55.433031+0000","last_unstale":"2024-01-27T02:32:55.433280+0000","last_undegraded":"2024-01-27T02:32:55.433280+0000","last_fullsized":"2024-01-27T02:32:55.433280+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:07:27.939664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.423167+0000","last_change":"2024-01-27T02:32:55.423167+0000","last_active":"2024-01-27T02:32:55.423167+0000","last_peered":"2024-01-27T02:32:55.423167+0000","last_clean":"2024-01-27T02:32:55.423167+0000","last_became_active":"2024-01-27T02:32:55.422900+0000","last_became_peered":"2024-01-27T02:32:55.422900+0000","last_unstale":"2024-01-27T02:32:55.423167+0000","last_undegraded":"2024-01-27T02:32:55.423167+0000","last_fullsized":"2024-01-27T02:32:55.423167+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:40:12.903562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.425220+0000","last_change":"2024-01-27T02:32:55.424027+0000","last_active":"2024-01-27T02:32:55.425220+0000","last_peered":"2024-01-27T02:32:55.425220+0000","last_clean":"2024-01-27T02:32:55.425220+0000","last_became_active":"2024-01-27T02:32:55.423082+0000","last_became_peered":"2024-01-27T02:32:55.423082+0000","last_unstale":"2024-01-27T02:32:55.425220+0000","last_undegraded":"2024-01-27T02:32:55.425220+0000","last_fullsized":"2024-01-27T02:32:55.425220+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:18:22.974631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.434120+0000","last_change":"2024-01-27T02:32:55.434120+0000","last_active":"2024-01-27T02:32:55.434120+0000","last_peered":"2024-01-27T02:32:55.434120+0000","last_clean":"2024-01-27T02:32:55.434120+0000","last_became_active":"2024-01-27T02:32:55.433842+0000","last_became_peered":"2024-01-27T02:32:55.433842+0000","last_unstale":"2024-01-27T02:32:55.434120+0000","last_undegraded":"2024-01-27T02:32:55.434120+0000","last_fullsized":"2024-01-27T02:32:55.434120+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:41.306732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.431903+0000","last_change":"2024-01-27T02:32:55.431903+0000","last_active":"2024-01-27T02:32:55.431903+0000","last_peered":"2024-01-27T02:32:55.431903+0000","last_clean":"2024-01-27T02:32:55.431903+0000","last_became_active":"2024-01-27T02:32:55.431559+0000","last_became_peered":"2024-01-27T02:32:55.431559+0000","last_unstale":"2024-01-27T02:32:55.431903+0000","last_undegraded":"2024-01-27T02:32:55.431903+0000","last_fullsized":"2024-01-27T02:32:55.431903+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:31:20.996303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.434269+0000","last_change":"2024-01-27T02:32:55.434269+0000","last_active":"2024-01-27T02:32:55.434269+0000","last_peered":"2024-01-27T02:32:55.434269+0000","last_clean":"2024-01-27T02:32:55.434269+0000","last_became_active":"2024-01-27T02:32:55.433715+0000","last_became_peered":"2024-01-27T02:32:55.433715+0000","last_unstale":"2024-01-27T02:32:55.434269+0000","last_undegraded":"2024-01-27T02:32:55.434269+0000","last_fullsized":"2024-01-27T02:32:55.434269+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:08:14.832481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.427507+0000","last_change":"2024-01-27T02:32:55.425793+0000","last_active":"2024-01-27T02:32:55.427507+0000","last_peered":"2024-01-27T02:32:55.427507+0000","last_clean":"2024-01-27T02:32:55.427507+0000","last_became_active":"2024-01-27T02:32:55.424878+0000","last_became_peered":"2024-01-27T02:32:55.424878+0000","last_unstale":"2024-01-27T02:32:55.427507+0000","last_undegraded":"2024-01-27T02:32:55.427507+0000","last_fullsized":"2024-01-27T02:32:55.427507+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:23:00.299763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.425248+0000","last_change":"2024-01-27T02:32:55.424197+0000","last_active":"2024-01-27T02:32:55.425248+0000","last_peered":"2024-01-27T02:32:55.425248+0000","last_clean":"2024-01-27T02:32:55.425248+0000","last_became_active":"2024-01-27T02:32:55.423416+0000","last_became_peered":"2024-01-27T02:32:55.423416+0000","last_unstale":"2024-01-27T02:32:55.425248+0000","last_undegraded":"2024-01-27T02:32:55.425248+0000","last_fullsized":"2024-01-27T02:32:55.425248+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:31:20.996303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.425846+0000","last_change":"2024-01-27T02:32:55.425846+0000","last_active":"2024-01-27T02:32:55.425846+0000","last_peered":"2024-01-27T02:32:55.425846+0000","last_clean":"2024-01-27T02:32:55.425846+0000","last_became_active":"2024-01-27T02:32:55.425529+0000","last_became_peered":"2024-01-27T02:32:55.425529+0000","last_unstale":"2024-01-27T02:32:55.425846+0000","last_undegraded":"2024-01-27T02:32:55.425846+0000","last_fullsized":"2024-01-27T02:32:55.425846+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:17:56.186917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.432777+0000","last_change":"2024-01-27T02:32:55.432711+0000","last_active":"2024-01-27T02:32:55.432777+0000","last_peered":"2024-01-27T02:32:55.432777+0000","last_clean":"2024-01-27T02:32:55.432777+0000","last_became_active":"2024-01-27T02:32:55.432486+0000","last_became_peered":"2024-01-27T02:32:55.432486+0000","last_unstale":"2024-01-27T02:32:55.432777+0000","last_undegraded":"2024-01-27T02:32:55.432777+0000","last_fullsized":"2024-01-27T02:32:55.432777+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:08:23.991383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.429788+0000","last_change":"2024-01-27T02:32:55.429330+0000","last_active":"2024-01-27T02:32:55.429788+0000","last_peered":"2024-01-27T02:32:55.429788+0000","last_clean":"2024-01-27T02:32:55.429788+0000","last_became_active":"2024-01-27T02:32:55.429067+0000","last_became_peered":"2024-01-27T02:32:55.429067+0000","last_unstale":"2024-01-27T02:32:55.429788+0000","last_undegraded":"2024-01-27T02:32:55.429788+0000","last_fullsized":"2024-01-27T02:32:55.429788+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:08:02.904513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.433117+0000","last_change":"2024-01-27T02:32:55.433117+0000","last_active":"2024-01-27T02:32:55.433117+0000","last_peered":"2024-01-27T02:32:55.433117+0000","last_clean":"2024-01-27T02:32:55.433117+0000","last_became_active":"2024-01-27T02:32:55.432532+0000","last_became_peered":"2024-01-27T02:32:55.432532+0000","last_unstale":"2024-01-27T02:32:55.433117+0000","last_undegraded":"2024-01-27T02:32:55.433117+0000","last_fullsized":"2024-01-27T02:32:55.433117+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:08:23.991383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.427809+0000","last_change":"2024-01-27T02:32:55.427809+0000","last_active":"2024-01-27T02:32:55.427809+0000","last_peered":"2024-01-27T02:32:55.427809+0000","last_clean":"2024-01-27T02:32:55.427809+0000","last_became_active":"2024-01-27T02:32:55.427419+0000","last_became_peered":"2024-01-27T02:32:55.427419+0000","last_unstale":"2024-01-27T02:32:55.427809+0000","last_undegraded":"2024-01-27T02:32:55.427809+0000","last_fullsized":"2024-01-27T02:32:55.427809+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:42:38.742239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.428755+0000","last_change":"2024-01-27T02:32:55.428755+0000","last_active":"2024-01-27T02:32:55.428755+0000","last_peered":"2024-01-27T02:32:55.428755+0000","last_clean":"2024-01-27T02:32:55.428755+0000","last_became_active":"2024-01-27T02:32:55.428463+0000","last_became_peered":"2024-01-27T02:32:55.428463+0000","last_unstale":"2024-01-27T02:32:55.428755+0000","last_undegraded":"2024-01-27T02:32:55.428755+0000","last_fullsized":"2024-01-27T02:32:55.428755+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:26:44.336012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.427571+0000","last_change":"2024-01-27T02:32:55.427241+0000","last_active":"2024-01-27T02:32:55.427571+0000","last_peered":"2024-01-27T02:32:55.427571+0000","last_clean":"2024-01-27T02:32:55.427571+0000","last_became_active":"2024-01-27T02:32:55.426483+0000","last_became_peered":"2024-01-27T02:32:55.426483+0000","last_unstale":"2024-01-27T02:32:55.427571+0000","last_undegraded":"2024-01-27T02:32:55.427571+0000","last_fullsized":"2024-01-27T02:32:55.427571+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:33:30.530805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.428928+0000","last_change":"2024-01-27T02:32:55.428928+0000","last_active":"2024-01-27T02:32:55.428928+0000","last_peered":"2024-01-27T02:32:55.428928+0000","last_clean":"2024-01-27T02:32:55.428928+0000","last_became_active":"2024-01-27T02:32:55.428452+0000","last_became_peered":"2024-01-27T02:32:55.428452+0000","last_unstale":"2024-01-27T02:32:55.428928+0000","last_undegraded":"2024-01-27T02:32:55.428928+0000","last_fullsized":"2024-01-27T02:32:55.428928+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:42:38.742239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.431320+0000","last_change":"2024-01-27T02:32:55.431320+0000","last_active":"2024-01-27T02:32:55.431320+0000","last_peered":"2024-01-27T02:32:55.431320+0000","last_clean":"2024-01-27T02:32:55.431320+0000","last_became_active":"2024-01-27T02:32:55.431083+0000","last_became_peered":"2024-01-27T02:32:55.431083+0000","last_unstale":"2024-01-27T02:32:55.431320+0000","last_undegraded":"2024-01-27T02:32:55.431320+0000","last_fullsized":"2024-01-27T02:32:55.431320+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:55:24.921329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.430856+0000","last_change":"2024-01-27T02:32:55.430562+0000","last_active":"2024-01-27T02:32:55.430856+0000","last_peered":"2024-01-27T02:32:55.430856+0000","last_clean":"2024-01-27T02:32:55.430856+0000","last_became_active":"2024-01-27T02:32:55.430293+0000","last_became_peered":"2024-01-27T02:32:55.430293+0000","last_unstale":"2024-01-27T02:32:55.430856+0000","last_undegraded":"2024-01-27T02:32:55.430856+0000","last_fullsized":"2024-01-27T02:32:55.430856+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:17:11.996957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.434818+0000","last_change":"2024-01-27T02:32:55.434818+0000","last_active":"2024-01-27T02:32:55.434818+0000","last_peered":"2024-01-27T02:32:55.434818+0000","last_clean":"2024-01-27T02:32:55.434818+0000","last_became_active":"2024-01-27T02:32:55.434416+0000","last_became_peered":"2024-01-27T02:32:55.434416+0000","last_unstale":"2024-01-27T02:32:55.434818+0000","last_undegraded":"2024-01-27T02:32:55.434818+0000","last_fullsized":"2024-01-27T02:32:55.434818+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:07:27.939664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.431513+0000","last_change":"2024-01-27T02:32:55.431399+0000","last_active":"2024-01-27T02:32:55.431513+0000","last_peered":"2024-01-27T02:32:55.431513+0000","last_clean":"2024-01-27T02:32:55.431513+0000","last_became_active":"2024-01-27T02:32:55.431159+0000","last_became_peered":"2024-01-27T02:32:55.431159+0000","last_unstale":"2024-01-27T02:32:55.431513+0000","last_undegraded":"2024-01-27T02:32:55.431513+0000","last_fullsized":"2024-01-27T02:32:55.431513+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:45:18.498656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.432090+0000","last_change":"2024-01-27T02:32:55.432090+0000","last_active":"2024-01-27T02:32:55.432090+0000","last_peered":"2024-01-27T02:32:55.432090+0000","last_clean":"2024-01-27T02:32:55.432090+0000","last_became_active":"2024-01-27T02:32:55.431868+0000","last_became_peered":"2024-01-27T02:32:55.431868+0000","last_unstale":"2024-01-27T02:32:55.432090+0000","last_undegraded":"2024-01-27T02:32:55.432090+0000","last_fullsized":"2024-01-27T02:32:55.432090+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:11:03.308052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.432329+0000","last_change":"2024-01-27T02:32:55.432329+0000","last_active":"2024-01-27T02:32:55.432329+0000","last_peered":"2024-01-27T02:32:55.432329+0000","last_clean":"2024-01-27T02:32:55.432329+0000","last_became_active":"2024-01-27T02:32:55.432054+0000","last_became_peered":"2024-01-27T02:32:55.432054+0000","last_unstale":"2024-01-27T02:32:55.432329+0000","last_undegraded":"2024-01-27T02:32:55.432329+0000","last_fullsized":"2024-01-27T02:32:55.432329+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:17:11.996957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.813275+0000","last_change":"2024-01-27T02:32:54.430272+0000","last_active":"2024-01-27T02:32:55.813275+0000","last_peered":"2024-01-27T02:32:55.813275+0000","last_clean":"2024-01-27T02:32:55.813275+0000","last_became_active":"2024-01-27T02:32:54.430013+0000","last_became_peered":"2024-01-27T02:32:54.430013+0000","last_unstale":"2024-01-27T02:32:55.813275+0000","last_undegraded":"2024-01-27T02:32:55.813275+0000","last_fullsized":"2024-01-27T02:32:55.813275+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:45:51.423427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.433531+0000","last_change":"2024-01-27T02:32:55.433446+0000","last_active":"2024-01-27T02:32:55.433531+0000","last_peered":"2024-01-27T02:32:55.433531+0000","last_clean":"2024-01-27T02:32:55.433531+0000","last_became_active":"2024-01-27T02:32:55.433064+0000","last_became_peered":"2024-01-27T02:32:55.433064+0000","last_unstale":"2024-01-27T02:32:55.433531+0000","last_undegraded":"2024-01-27T02:32:55.433531+0000","last_fullsized":"2024-01-27T02:32:55.433531+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:00:44.207288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.426954+0000","last_change":"2024-01-27T02:32:54.433274+0000","last_active":"2024-01-27T02:32:55.426954+0000","last_peered":"2024-01-27T02:32:55.426954+0000","last_clean":"2024-01-27T02:32:55.426954+0000","last_became_active":"2024-01-27T02:32:54.432995+0000","last_became_peered":"2024-01-27T02:32:54.432995+0000","last_unstale":"2024-01-27T02:32:55.426954+0000","last_undegraded":"2024-01-27T02:32:55.426954+0000","last_fullsized":"2024-01-27T02:32:55.426954+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:56:58.873502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.431392+0000","last_change":"2024-01-27T02:32:55.430889+0000","last_active":"2024-01-27T02:32:55.431392+0000","last_peered":"2024-01-27T02:32:55.431392+0000","last_clean":"2024-01-27T02:32:55.431392+0000","last_became_active":"2024-01-27T02:32:55.430647+0000","last_became_peered":"2024-01-27T02:32:55.430647+0000","last_unstale":"2024-01-27T02:32:55.431392+0000","last_undegraded":"2024-01-27T02:32:55.431392+0000","last_fullsized":"2024-01-27T02:32:55.431392+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:00:44.207288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.814523+0000","last_change":"2024-01-27T02:32:54.429820+0000","last_active":"2024-01-27T02:32:55.814523+0000","last_peered":"2024-01-27T02:32:55.814523+0000","last_clean":"2024-01-27T02:32:55.814523+0000","last_became_active":"2024-01-27T02:32:54.429548+0000","last_became_peered":"2024-01-27T02:32:54.429548+0000","last_unstale":"2024-01-27T02:32:55.814523+0000","last_undegraded":"2024-01-27T02:32:55.814523+0000","last_fullsized":"2024-01-27T02:32:55.814523+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:45:17.490073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.427121+0000","last_change":"2024-01-27T02:32:55.427121+0000","last_active":"2024-01-27T02:32:55.427121+0000","last_peered":"2024-01-27T02:32:55.427121+0000","last_clean":"2024-01-27T02:32:55.427121+0000","last_became_active":"2024-01-27T02:32:55.426845+0000","last_became_peered":"2024-01-27T02:32:55.426845+0000","last_unstale":"2024-01-27T02:32:55.427121+0000","last_undegraded":"2024-01-27T02:32:55.427121+0000","last_fullsized":"2024-01-27T02:32:55.427121+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:31:14.098748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.812735+0000","last_change":"2024-01-27T02:32:54.428490+0000","last_active":"2024-01-27T02:32:55.812735+0000","last_peered":"2024-01-27T02:32:55.812735+0000","last_clean":"2024-01-27T02:32:55.812735+0000","last_became_active":"2024-01-27T02:32:54.428289+0000","last_became_peered":"2024-01-27T02:32:54.428289+0000","last_unstale":"2024-01-27T02:32:55.812735+0000","last_undegraded":"2024-01-27T02:32:55.812735+0000","last_fullsized":"2024-01-27T02:32:55.812735+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:54:01.417398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.430351+0000","last_change":"2024-01-27T02:32:55.430351+0000","last_active":"2024-01-27T02:32:55.430351+0000","last_peered":"2024-01-27T02:32:55.430351+0000","last_clean":"2024-01-27T02:32:55.430351+0000","last_became_active":"2024-01-27T02:32:55.429263+0000","last_became_peered":"2024-01-27T02:32:55.429263+0000","last_unstale":"2024-01-27T02:32:55.430351+0000","last_undegraded":"2024-01-27T02:32:55.430351+0000","last_fullsized":"2024-01-27T02:32:55.430351+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:18:22.974631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.813999+0000","last_change":"2024-01-27T02:32:54.427710+0000","last_active":"2024-01-27T02:32:55.813999+0000","last_peered":"2024-01-27T02:32:55.813999+0000","last_clean":"2024-01-27T02:32:55.813999+0000","last_became_active":"2024-01-27T02:32:54.427171+0000","last_became_peered":"2024-01-27T02:32:54.427171+0000","last_unstale":"2024-01-27T02:32:55.813999+0000","last_undegraded":"2024-01-27T02:32:55.813999+0000","last_fullsized":"2024-01-27T02:32:55.813999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:45:03.287924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.432822+0000","last_change":"2024-01-27T02:32:55.432822+0000","last_active":"2024-01-27T02:32:55.432822+0000","last_peered":"2024-01-27T02:32:55.432822+0000","last_clean":"2024-01-27T02:32:55.432822+0000","last_became_active":"2024-01-27T02:32:55.432445+0000","last_became_peered":"2024-01-27T02:32:55.432445+0000","last_unstale":"2024-01-27T02:32:55.432822+0000","last_undegraded":"2024-01-27T02:32:55.432822+0000","last_fullsized":"2024-01-27T02:32:55.432822+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:11:03.308052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.814024+0000","last_change":"2024-01-27T02:32:54.426583+0000","last_active":"2024-01-27T02:32:55.814024+0000","last_peered":"2024-01-27T02:32:55.814024+0000","last_clean":"2024-01-27T02:32:55.814024+0000","last_became_active":"2024-01-27T02:32:54.426339+0000","last_became_peered":"2024-01-27T02:32:54.426339+0000","last_unstale":"2024-01-27T02:32:55.814024+0000","last_undegraded":"2024-01-27T02:32:55.814024+0000","last_fullsized":"2024-01-27T02:32:55.814024+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:40:13.068334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.434849+0000","last_change":"2024-01-27T02:32:55.433990+0000","last_active":"2024-01-27T02:32:55.434849+0000","last_peered":"2024-01-27T02:32:55.434849+0000","last_clean":"2024-01-27T02:32:55.434849+0000","last_became_active":"2024-01-27T02:32:55.433762+0000","last_became_peered":"2024-01-27T02:32:55.433762+0000","last_unstale":"2024-01-27T02:32:55.434849+0000","last_undegraded":"2024-01-27T02:32:55.434849+0000","last_fullsized":"2024-01-27T02:32:55.434849+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:26:44.336012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.419619+0000","last_change":"2024-01-27T02:32:54.425055+0000","last_active":"2024-01-27T02:32:55.419619+0000","last_peered":"2024-01-27T02:32:55.419619+0000","last_clean":"2024-01-27T02:32:55.419619+0000","last_became_active":"2024-01-27T02:32:54.421312+0000","last_became_peered":"2024-01-27T02:32:54.421312+0000","last_unstale":"2024-01-27T02:32:55.419619+0000","last_undegraded":"2024-01-27T02:32:55.419619+0000","last_fullsized":"2024-01-27T02:32:55.419619+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:08:22.982800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.425056+0000","last_change":"2024-01-27T02:32:55.425056+0000","last_active":"2024-01-27T02:32:55.425056+0000","last_peered":"2024-01-27T02:32:55.425056+0000","last_clean":"2024-01-27T02:32:55.425056+0000","last_became_active":"2024-01-27T02:32:55.424571+0000","last_became_peered":"2024-01-27T02:32:55.424571+0000","last_unstale":"2024-01-27T02:32:55.425056+0000","last_undegraded":"2024-01-27T02:32:55.425056+0000","last_fullsized":"2024-01-27T02:32:55.425056+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:27:49.416295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.425295+0000","last_change":"2024-01-27T02:32:54.428730+0000","last_active":"2024-01-27T02:32:55.425295+0000","last_peered":"2024-01-27T02:32:55.425295+0000","last_clean":"2024-01-27T02:32:55.425295+0000","last_became_active":"2024-01-27T02:32:54.428159+0000","last_became_peered":"2024-01-27T02:32:54.428159+0000","last_unstale":"2024-01-27T02:32:55.425295+0000","last_undegraded":"2024-01-27T02:32:55.425295+0000","last_fullsized":"2024-01-27T02:32:55.425295+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:45:51.423427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.432670+0000","last_change":"2024-01-27T02:32:55.432670+0000","last_active":"2024-01-27T02:32:55.432670+0000","last_peered":"2024-01-27T02:32:55.432670+0000","last_clean":"2024-01-27T02:32:55.432670+0000","last_became_active":"2024-01-27T02:32:55.432127+0000","last_became_peered":"2024-01-27T02:32:55.432127+0000","last_unstale":"2024-01-27T02:32:55.432670+0000","last_undegraded":"2024-01-27T02:32:55.432670+0000","last_fullsized":"2024-01-27T02:32:55.432670+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:27:49.416295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.418043+0000","last_change":"2024-01-27T02:32:54.432300+0000","last_active":"2024-01-27T02:32:55.418043+0000","last_peered":"2024-01-27T02:32:55.418043+0000","last_clean":"2024-01-27T02:32:55.418043+0000","last_became_active":"2024-01-27T02:32:54.431991+0000","last_became_peered":"2024-01-27T02:32:54.431991+0000","last_unstale":"2024-01-27T02:32:55.418043+0000","last_undegraded":"2024-01-27T02:32:55.418043+0000","last_fullsized":"2024-01-27T02:32:55.418043+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:37:59.891632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.427548+0000","last_change":"2024-01-27T02:32:55.426731+0000","last_active":"2024-01-27T02:32:55.427548+0000","last_peered":"2024-01-27T02:32:55.427548+0000","last_clean":"2024-01-27T02:32:55.427548+0000","last_became_active":"2024-01-27T02:32:55.425599+0000","last_became_peered":"2024-01-27T02:32:55.425599+0000","last_unstale":"2024-01-27T02:32:55.427548+0000","last_undegraded":"2024-01-27T02:32:55.427548+0000","last_fullsized":"2024-01-27T02:32:55.427548+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:19:30.393076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-01-27T02:32:54.427908+0000","last_change":"2024-01-27T02:32:54.427283+0000","last_active":"2024-01-27T02:32:54.427908+0000","last_peered":"2024-01-27T02:32:54.427908+0000","last_clean":"2024-01-27T02:32:54.427908+0000","last_became_active":"2024-01-27T02:32:54.427007+0000","last_became_peered":"2024-01-27T02:32:54.427007+0000","last_unstale":"2024-01-27T02:32:54.427908+0000","last_undegraded":"2024-01-27T02:32:54.427908+0000","last_fullsized":"2024-01-27T02:32:54.427908+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:35:02.936237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.429717+0000","last_change":"2024-01-27T02:32:55.429108+0000","last_active":"2024-01-27T02:32:55.429717+0000","last_peered":"2024-01-27T02:32:55.429717+0000","last_clean":"2024-01-27T02:32:55.429717+0000","last_became_active":"2024-01-27T02:32:55.428793+0000","last_became_peered":"2024-01-27T02:32:55.428793+0000","last_unstale":"2024-01-27T02:32:55.429717+0000","last_undegraded":"2024-01-27T02:32:55.429717+0000","last_fullsized":"2024-01-27T02:32:55.429717+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:04:28.768090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.812811+0000","last_change":"2024-01-27T02:32:54.432123+0000","last_active":"2024-01-27T02:32:55.812811+0000","last_peered":"2024-01-27T02:32:55.812811+0000","last_clean":"2024-01-27T02:32:55.812811+0000","last_became_active":"2024-01-27T02:32:54.431900+0000","last_became_peered":"2024-01-27T02:32:54.431900+0000","last_unstale":"2024-01-27T02:32:55.812811+0000","last_undegraded":"2024-01-27T02:32:55.812811+0000","last_fullsized":"2024-01-27T02:32:55.812811+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:45:17.490073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.429720+0000","last_change":"2024-01-27T02:32:55.429720+0000","last_active":"2024-01-27T02:32:55.429720+0000","last_peered":"2024-01-27T02:32:55.429720+0000","last_clean":"2024-01-27T02:32:55.429720+0000","last_became_active":"2024-01-27T02:32:55.429498+0000","last_became_peered":"2024-01-27T02:32:55.429498+0000","last_unstale":"2024-01-27T02:32:55.429720+0000","last_undegraded":"2024-01-27T02:32:55.429720+0000","last_fullsized":"2024-01-27T02:32:55.429720+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:20:24.718650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.816355+0000","last_change":"2024-01-27T02:32:54.431720+0000","last_active":"2024-01-27T02:32:55.816355+0000","last_peered":"2024-01-27T02:32:55.816355+0000","last_clean":"2024-01-27T02:32:55.816355+0000","last_became_active":"2024-01-27T02:32:54.430893+0000","last_became_peered":"2024-01-27T02:32:54.430893+0000","last_unstale":"2024-01-27T02:32:55.816355+0000","last_undegraded":"2024-01-27T02:32:55.816355+0000","last_fullsized":"2024-01-27T02:32:55.816355+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:28:21.283426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.424597+0000","last_change":"2024-01-27T02:32:55.424597+0000","last_active":"2024-01-27T02:32:55.424597+0000","last_peered":"2024-01-27T02:32:55.424597+0000","last_clean":"2024-01-27T02:32:55.424597+0000","last_became_active":"2024-01-27T02:32:55.424298+0000","last_became_peered":"2024-01-27T02:32:55.424298+0000","last_unstale":"2024-01-27T02:32:55.424597+0000","last_undegraded":"2024-01-27T02:32:55.424597+0000","last_fullsized":"2024-01-27T02:32:55.424597+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:36:33.291109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.813518+0000","last_change":"2024-01-27T02:32:54.428574+0000","last_active":"2024-01-27T02:32:55.813518+0000","last_peered":"2024-01-27T02:32:55.813518+0000","last_clean":"2024-01-27T02:32:55.813518+0000","last_became_active":"2024-01-27T02:32:54.427872+0000","last_became_peered":"2024-01-27T02:32:54.427872+0000","last_unstale":"2024-01-27T02:32:55.813518+0000","last_undegraded":"2024-01-27T02:32:55.813518+0000","last_fullsized":"2024-01-27T02:32:55.813518+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:31:55.674636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.431174+0000","last_change":"2024-01-27T02:32:55.430523+0000","last_active":"2024-01-27T02:32:55.431174+0000","last_peered":"2024-01-27T02:32:55.431174+0000","last_clean":"2024-01-27T02:32:55.431174+0000","last_became_active":"2024-01-27T02:32:55.430273+0000","last_became_peered":"2024-01-27T02:32:55.430273+0000","last_unstale":"2024-01-27T02:32:55.431174+0000","last_undegraded":"2024-01-27T02:32:55.431174+0000","last_fullsized":"2024-01-27T02:32:55.431174+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:46:09.976295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-01-27T02:32:54.431632+0000","last_change":"2024-01-27T02:32:54.431394+0000","last_active":"2024-01-27T02:32:54.431632+0000","last_peered":"2024-01-27T02:32:54.431632+0000","last_clean":"2024-01-27T02:32:54.431632+0000","last_became_active":"2024-01-27T02:32:54.431120+0000","last_became_peered":"2024-01-27T02:32:54.431120+0000","last_unstale":"2024-01-27T02:32:54.431632+0000","last_undegraded":"2024-01-27T02:32:54.431632+0000","last_fullsized":"2024-01-27T02:32:54.431632+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:36:23.308442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.430204+0000","last_change":"2024-01-27T02:32:55.430204+0000","last_active":"2024-01-27T02:32:55.430204+0000","last_peered":"2024-01-27T02:32:55.430204+0000","last_clean":"2024-01-27T02:32:55.430204+0000","last_became_active":"2024-01-27T02:32:55.428926+0000","last_became_peered":"2024-01-27T02:32:55.428926+0000","last_unstale":"2024-01-27T02:32:55.430204+0000","last_undegraded":"2024-01-27T02:32:55.430204+0000","last_fullsized":"2024-01-27T02:32:55.430204+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:55:24.921329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.418005+0000","last_change":"2024-01-27T02:32:54.431435+0000","last_active":"2024-01-27T02:32:55.418005+0000","last_peered":"2024-01-27T02:32:55.418005+0000","last_clean":"2024-01-27T02:32:55.418005+0000","last_became_active":"2024-01-27T02:32:54.431177+0000","last_became_peered":"2024-01-27T02:32:54.431177+0000","last_unstale":"2024-01-27T02:32:55.418005+0000","last_undegraded":"2024-01-27T02:32:55.418005+0000","last_fullsized":"2024-01-27T02:32:55.418005+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:17:10.988374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.433263+0000","last_change":"2024-01-27T02:32:55.433183+0000","last_active":"2024-01-27T02:32:55.433263+0000","last_peered":"2024-01-27T02:32:55.433263+0000","last_clean":"2024-01-27T02:32:55.433263+0000","last_became_active":"2024-01-27T02:32:55.432953+0000","last_became_peered":"2024-01-27T02:32:55.432953+0000","last_unstale":"2024-01-27T02:32:55.433263+0000","last_undegraded":"2024-01-27T02:32:55.433263+0000","last_fullsized":"2024-01-27T02:32:55.433263+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:17:11.996957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.817669+0000","last_change":"2024-01-27T02:32:54.429801+0000","last_active":"2024-01-27T02:32:55.817669+0000","last_peered":"2024-01-27T02:32:55.817669+0000","last_clean":"2024-01-27T02:32:55.817669+0000","last_became_active":"2024-01-27T02:32:54.429523+0000","last_became_peered":"2024-01-27T02:32:54.429523+0000","last_unstale":"2024-01-27T02:32:55.817669+0000","last_undegraded":"2024-01-27T02:32:55.817669+0000","last_fullsized":"2024-01-27T02:32:55.817669+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:06:09.577224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.426830+0000","last_change":"2024-01-27T02:32:55.426171+0000","last_active":"2024-01-27T02:32:55.426830+0000","last_peered":"2024-01-27T02:32:55.426830+0000","last_clean":"2024-01-27T02:32:55.426830+0000","last_became_active":"2024-01-27T02:32:55.425915+0000","last_became_peered":"2024-01-27T02:32:55.425915+0000","last_unstale":"2024-01-27T02:32:55.426830+0000","last_undegraded":"2024-01-27T02:32:55.426830+0000","last_fullsized":"2024-01-27T02:32:55.426830+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:36:33.291109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.421723+0000","last_change":"2024-01-27T02:32:54.433830+0000","last_active":"2024-01-27T02:32:55.421723+0000","last_peered":"2024-01-27T02:32:55.421723+0000","last_clean":"2024-01-27T02:32:55.421723+0000","last_became_active":"2024-01-27T02:32:54.433524+0000","last_became_peered":"2024-01-27T02:32:54.433524+0000","last_unstale":"2024-01-27T02:32:55.421723+0000","last_undegraded":"2024-01-27T02:32:55.421723+0000","last_fullsized":"2024-01-27T02:32:55.421723+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:45:51.423427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.427654+0000","last_change":"2024-01-27T02:32:55.427654+0000","last_active":"2024-01-27T02:32:55.427654+0000","last_peered":"2024-01-27T02:32:55.427654+0000","last_clean":"2024-01-27T02:32:55.427654+0000","last_became_active":"2024-01-27T02:32:55.426903+0000","last_became_peered":"2024-01-27T02:32:55.426903+0000","last_unstale":"2024-01-27T02:32:55.427654+0000","last_undegraded":"2024-01-27T02:32:55.427654+0000","last_fullsized":"2024-01-27T02:32:55.427654+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:46:09.976295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.813143+0000","last_change":"2024-01-27T02:32:54.428404+0000","last_active":"2024-01-27T02:32:55.813143+0000","last_peered":"2024-01-27T02:32:55.813143+0000","last_clean":"2024-01-27T02:32:55.813143+0000","last_became_active":"2024-01-27T02:32:54.427562+0000","last_became_peered":"2024-01-27T02:32:54.427562+0000","last_unstale":"2024-01-27T02:32:55.813143+0000","last_undegraded":"2024-01-27T02:32:55.813143+0000","last_fullsized":"2024-01-27T02:32:55.813143+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:42:08.346789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.428750+0000","last_change":"2024-01-27T02:32:55.428750+0000","last_active":"2024-01-27T02:32:55.428750+0000","last_peered":"2024-01-27T02:32:55.428750+0000","last_clean":"2024-01-27T02:32:55.428750+0000","last_became_active":"2024-01-27T02:32:55.427769+0000","last_became_peered":"2024-01-27T02:32:55.427769+0000","last_unstale":"2024-01-27T02:32:55.428750+0000","last_undegraded":"2024-01-27T02:32:55.428750+0000","last_fullsized":"2024-01-27T02:32:55.428750+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:46:59.789872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.813387+0000","last_change":"2024-01-27T02:32:54.425812+0000","last_active":"2024-01-27T02:32:55.813387+0000","last_peered":"2024-01-27T02:32:55.813387+0000","last_clean":"2024-01-27T02:32:55.813387+0000","last_became_active":"2024-01-27T02:32:54.425573+0000","last_became_peered":"2024-01-27T02:32:54.425573+0000","last_unstale":"2024-01-27T02:32:55.813387+0000","last_undegraded":"2024-01-27T02:32:55.813387+0000","last_fullsized":"2024-01-27T02:32:55.813387+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:53:21.551567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.425996+0000","last_change":"2024-01-27T02:32:55.425996+0000","last_active":"2024-01-27T02:32:55.425996+0000","last_peered":"2024-01-27T02:32:55.425996+0000","last_clean":"2024-01-27T02:32:55.425996+0000","last_became_active":"2024-01-27T02:32:55.425741+0000","last_became_peered":"2024-01-27T02:32:55.425741+0000","last_unstale":"2024-01-27T02:32:55.425996+0000","last_undegraded":"2024-01-27T02:32:55.425996+0000","last_fullsized":"2024-01-27T02:32:55.425996+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:15:51.134403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.426117+0000","last_change":"2024-01-27T02:32:54.427847+0000","last_active":"2024-01-27T02:32:55.426117+0000","last_peered":"2024-01-27T02:32:55.426117+0000","last_clean":"2024-01-27T02:32:55.426117+0000","last_became_active":"2024-01-27T02:32:54.427480+0000","last_became_peered":"2024-01-27T02:32:54.427480+0000","last_unstale":"2024-01-27T02:32:55.426117+0000","last_undegraded":"2024-01-27T02:32:55.426117+0000","last_fullsized":"2024-01-27T02:32:55.426117+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:58:29.492017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.426752+0000","last_change":"2024-01-27T02:32:55.426752+0000","last_active":"2024-01-27T02:32:55.426752+0000","last_peered":"2024-01-27T02:32:55.426752+0000","last_clean":"2024-01-27T02:32:55.426752+0000","last_became_active":"2024-01-27T02:32:55.426479+0000","last_became_peered":"2024-01-27T02:32:55.426479+0000","last_unstale":"2024-01-27T02:32:55.426752+0000","last_undegraded":"2024-01-27T02:32:55.426752+0000","last_fullsized":"2024-01-27T02:32:55.426752+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:56:59.882085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.817450+0000","last_change":"2024-01-27T02:32:54.428946+0000","last_active":"2024-01-27T02:32:55.817450+0000","last_peered":"2024-01-27T02:32:55.817450+0000","last_clean":"2024-01-27T02:32:55.817450+0000","last_became_active":"2024-01-27T02:32:54.428552+0000","last_became_peered":"2024-01-27T02:32:54.428552+0000","last_unstale":"2024-01-27T02:32:55.817450+0000","last_undegraded":"2024-01-27T02:32:55.817450+0000","last_fullsized":"2024-01-27T02:32:55.817450+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:15:50.125821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.429001+0000","last_change":"2024-01-27T02:32:55.429001+0000","last_active":"2024-01-27T02:32:55.429001+0000","last_peered":"2024-01-27T02:32:55.429001+0000","last_clean":"2024-01-27T02:32:55.429001+0000","last_became_active":"2024-01-27T02:32:55.428745+0000","last_became_peered":"2024-01-27T02:32:55.428745+0000","last_unstale":"2024-01-27T02:32:55.429001+0000","last_undegraded":"2024-01-27T02:32:55.429001+0000","last_fullsized":"2024-01-27T02:32:55.429001+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:42:09.355371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.426210+0000","last_change":"2024-01-27T02:32:54.430722+0000","last_active":"2024-01-27T02:32:55.426210+0000","last_peered":"2024-01-27T02:32:55.426210+0000","last_clean":"2024-01-27T02:32:55.426210+0000","last_became_active":"2024-01-27T02:32:54.430517+0000","last_became_peered":"2024-01-27T02:32:54.430517+0000","last_unstale":"2024-01-27T02:32:55.426210+0000","last_undegraded":"2024-01-27T02:32:55.426210+0000","last_fullsized":"2024-01-27T02:32:55.426210+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:31:55.674636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.430911+0000","last_change":"2024-01-27T02:32:55.430911+0000","last_active":"2024-01-27T02:32:55.430911+0000","last_peered":"2024-01-27T02:32:55.430911+0000","last_clean":"2024-01-27T02:32:55.430911+0000","last_became_active":"2024-01-27T02:32:55.430643+0000","last_became_peered":"2024-01-27T02:32:55.430643+0000","last_unstale":"2024-01-27T02:32:55.430911+0000","last_undegraded":"2024-01-27T02:32:55.430911+0000","last_fullsized":"2024-01-27T02:32:55.430911+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:40:14.076916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.425618+0000","last_change":"2024-01-27T02:32:54.432055+0000","last_active":"2024-01-27T02:32:55.425618+0000","last_peered":"2024-01-27T02:32:55.425618+0000","last_clean":"2024-01-27T02:32:55.425618+0000","last_became_active":"2024-01-27T02:32:54.431830+0000","last_became_peered":"2024-01-27T02:32:54.431830+0000","last_unstale":"2024-01-27T02:32:55.425618+0000","last_undegraded":"2024-01-27T02:32:55.425618+0000","last_fullsized":"2024-01-27T02:32:55.425618+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:16:07.293532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.434662+0000","last_change":"2024-01-27T02:32:55.434662+0000","last_active":"2024-01-27T02:32:55.434662+0000","last_peered":"2024-01-27T02:32:55.434662+0000","last_clean":"2024-01-27T02:32:55.434662+0000","last_became_active":"2024-01-27T02:32:55.434080+0000","last_became_peered":"2024-01-27T02:32:55.434080+0000","last_unstale":"2024-01-27T02:32:55.434662+0000","last_undegraded":"2024-01-27T02:32:55.434662+0000","last_fullsized":"2024-01-27T02:32:55.434662+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:27:49.416295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-01-27T02:32:54.431128+0000","last_change":"2024-01-27T02:32:54.431077+0000","last_active":"2024-01-27T02:32:54.431128+0000","last_peered":"2024-01-27T02:32:54.431128+0000","last_clean":"2024-01-27T02:32:54.431128+0000","last_became_active":"2024-01-27T02:32:54.430833+0000","last_became_peered":"2024-01-27T02:32:54.430833+0000","last_unstale":"2024-01-27T02:32:54.431128+0000","last_undegraded":"2024-01-27T02:32:54.431128+0000","last_fullsized":"2024-01-27T02:32:54.431128+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:06:09.577224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.428454+0000","last_change":"2024-01-27T02:32:55.428454+0000","last_active":"2024-01-27T02:32:55.428454+0000","last_peered":"2024-01-27T02:32:55.428454+0000","last_clean":"2024-01-27T02:32:55.428454+0000","last_became_active":"2024-01-27T02:32:55.428154+0000","last_became_peered":"2024-01-27T02:32:55.428154+0000","last_unstale":"2024-01-27T02:32:55.428454+0000","last_undegraded":"2024-01-27T02:32:55.428454+0000","last_fullsized":"2024-01-27T02:32:55.428454+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:35:03.944819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.812631+0000","last_change":"2024-01-27T02:32:54.431112+0000","last_active":"2024-01-27T02:32:55.812631+0000","last_peered":"2024-01-27T02:32:55.812631+0000","last_clean":"2024-01-27T02:32:55.812631+0000","last_became_active":"2024-01-27T02:32:54.430334+0000","last_became_peered":"2024-01-27T02:32:54.430334+0000","last_unstale":"2024-01-27T02:32:55.812631+0000","last_undegraded":"2024-01-27T02:32:55.812631+0000","last_fullsized":"2024-01-27T02:32:55.812631+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:35:02.936237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.434288+0000","last_change":"2024-01-27T02:32:55.434288+0000","last_active":"2024-01-27T02:32:55.434288+0000","last_peered":"2024-01-27T02:32:55.434288+0000","last_clean":"2024-01-27T02:32:55.434288+0000","last_became_active":"2024-01-27T02:32:55.433886+0000","last_became_peered":"2024-01-27T02:32:55.433886+0000","last_unstale":"2024-01-27T02:32:55.434288+0000","last_undegraded":"2024-01-27T02:32:55.434288+0000","last_fullsized":"2024-01-27T02:32:55.434288+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:08:14.832481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-01-27T02:32:54.429057+0000","last_change":"2024-01-27T02:32:54.428797+0000","last_active":"2024-01-27T02:32:54.429057+0000","last_peered":"2024-01-27T02:32:54.429057+0000","last_clean":"2024-01-27T02:32:54.429057+0000","last_became_active":"2024-01-27T02:32:54.428226+0000","last_became_peered":"2024-01-27T02:32:54.428226+0000","last_unstale":"2024-01-27T02:32:54.429057+0000","last_undegraded":"2024-01-27T02:32:54.429057+0000","last_fullsized":"2024-01-27T02:32:54.429057+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:36:23.308442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.434509+0000","last_change":"2024-01-27T02:32:55.434509+0000","last_active":"2024-01-27T02:32:55.434509+0000","last_peered":"2024-01-27T02:32:55.434509+0000","last_clean":"2024-01-27T02:32:55.434509+0000","last_became_active":"2024-01-27T02:32:55.434237+0000","last_became_peered":"2024-01-27T02:32:55.434237+0000","last_unstale":"2024-01-27T02:32:55.434509+0000","last_undegraded":"2024-01-27T02:32:55.434509+0000","last_fullsized":"2024-01-27T02:32:55.434509+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:08:02.904513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-01-27T02:32:54.430699+0000","last_change":"2024-01-27T02:32:54.430115+0000","last_active":"2024-01-27T02:32:54.430699+0000","last_peered":"2024-01-27T02:32:54.430699+0000","last_clean":"2024-01-27T02:32:54.430699+0000","last_became_active":"2024-01-27T02:32:54.429096+0000","last_became_peered":"2024-01-27T02:32:54.429096+0000","last_unstale":"2024-01-27T02:32:54.430699+0000","last_undegraded":"2024-01-27T02:32:54.430699+0000","last_fullsized":"2024-01-27T02:32:54.430699+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:56:58.873502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.433245+0000","last_change":"2024-01-27T02:32:55.432722+0000","last_active":"2024-01-27T02:32:55.433245+0000","last_peered":"2024-01-27T02:32:55.433245+0000","last_clean":"2024-01-27T02:32:55.433245+0000","last_became_active":"2024-01-27T02:32:55.432439+0000","last_became_peered":"2024-01-27T02:32:55.432439+0000","last_unstale":"2024-01-27T02:32:55.433245+0000","last_undegraded":"2024-01-27T02:32:55.433245+0000","last_fullsized":"2024-01-27T02:32:55.433245+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:38:00.900215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-01-27T02:32:54.427929+0000","last_change":"2024-01-27T02:32:54.427576+0000","last_active":"2024-01-27T02:32:54.427929+0000","last_peered":"2024-01-27T02:32:54.427929+0000","last_clean":"2024-01-27T02:32:54.427929+0000","last_became_active":"2024-01-27T02:32:54.427340+0000","last_became_peered":"2024-01-27T02:32:54.427340+0000","last_unstale":"2024-01-27T02:32:54.427929+0000","last_undegraded":"2024-01-27T02:32:54.427929+0000","last_fullsized":"2024-01-27T02:32:54.427929+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:17:55.178334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.433493+0000","last_change":"2024-01-27T02:32:55.433493+0000","last_active":"2024-01-27T02:32:55.433493+0000","last_peered":"2024-01-27T02:32:55.433493+0000","last_clean":"2024-01-27T02:32:55.433493+0000","last_became_active":"2024-01-27T02:32:55.433250+0000","last_became_peered":"2024-01-27T02:32:55.433250+0000","last_unstale":"2024-01-27T02:32:55.433493+0000","last_undegraded":"2024-01-27T02:32:55.433493+0000","last_fullsized":"2024-01-27T02:32:55.433493+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:00:44.207288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-01-27T02:32:54.430792+0000","last_change":"2024-01-27T02:32:54.430585+0000","last_active":"2024-01-27T02:32:54.430792+0000","last_peered":"2024-01-27T02:32:54.430792+0000","last_clean":"2024-01-27T02:32:54.430792+0000","last_became_active":"2024-01-27T02:32:54.429985+0000","last_became_peered":"2024-01-27T02:32:54.429985+0000","last_unstale":"2024-01-27T02:32:54.430792+0000","last_undegraded":"2024-01-27T02:32:54.430792+0000","last_fullsized":"2024-01-27T02:32:54.430792+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:17:55.178334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.425268+0000","last_change":"2024-01-27T02:32:55.424345+0000","last_active":"2024-01-27T02:32:55.425268+0000","last_peered":"2024-01-27T02:32:55.425268+0000","last_clean":"2024-01-27T02:32:55.425268+0000","last_became_active":"2024-01-27T02:32:55.423765+0000","last_became_peered":"2024-01-27T02:32:55.423765+0000","last_unstale":"2024-01-27T02:32:55.425268+0000","last_undegraded":"2024-01-27T02:32:55.425268+0000","last_fullsized":"2024-01-27T02:32:55.425268+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:19:26.991510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.427333+0000","last_change":"2024-01-27T02:32:54.430012+0000","last_active":"2024-01-27T02:32:55.427333+0000","last_peered":"2024-01-27T02:32:55.427333+0000","last_clean":"2024-01-27T02:32:55.427333+0000","last_became_active":"2024-01-27T02:32:54.429786+0000","last_became_peered":"2024-01-27T02:32:54.429786+0000","last_unstale":"2024-01-27T02:32:55.427333+0000","last_undegraded":"2024-01-27T02:32:55.427333+0000","last_fullsized":"2024-01-27T02:32:55.427333+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:11:35.438367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.432303+0000","last_change":"2024-01-27T02:32:55.432236+0000","last_active":"2024-01-27T02:32:55.432303+0000","last_peered":"2024-01-27T02:32:55.432303+0000","last_clean":"2024-01-27T02:32:55.432303+0000","last_became_active":"2024-01-27T02:32:55.431653+0000","last_became_peered":"2024-01-27T02:32:55.431653+0000","last_unstale":"2024-01-27T02:32:55.432303+0000","last_undegraded":"2024-01-27T02:32:55.432303+0000","last_fullsized":"2024-01-27T02:32:55.432303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:38:00.900215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.420148+0000","last_change":"2024-01-27T02:32:54.432509+0000","last_active":"2024-01-27T02:32:55.420148+0000","last_peered":"2024-01-27T02:32:55.420148+0000","last_clean":"2024-01-27T02:32:55.420148+0000","last_became_active":"2024-01-27T02:32:54.432268+0000","last_became_peered":"2024-01-27T02:32:54.432268+0000","last_unstale":"2024-01-27T02:32:55.420148+0000","last_undegraded":"2024-01-27T02:32:55.420148+0000","last_fullsized":"2024-01-27T02:32:55.420148+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:00:43.198705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.430320+0000","last_change":"2024-01-27T02:32:55.430320+0000","last_active":"2024-01-27T02:32:55.430320+0000","last_peered":"2024-01-27T02:32:55.430320+0000","last_clean":"2024-01-27T02:32:55.430320+0000","last_became_active":"2024-01-27T02:32:55.428673+0000","last_became_peered":"2024-01-27T02:32:55.428673+0000","last_unstale":"2024-01-27T02:32:55.430320+0000","last_undegraded":"2024-01-27T02:32:55.430320+0000","last_fullsized":"2024-01-27T02:32:55.430320+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:31:14.098748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.426312+0000","last_change":"2024-01-27T02:32:54.426805+0000","last_active":"2024-01-27T02:32:55.426312+0000","last_peered":"2024-01-27T02:32:55.426312+0000","last_clean":"2024-01-27T02:32:55.426312+0000","last_became_active":"2024-01-27T02:32:54.426589+0000","last_became_peered":"2024-01-27T02:32:54.426589+0000","last_unstale":"2024-01-27T02:32:55.426312+0000","last_undegraded":"2024-01-27T02:32:55.426312+0000","last_fullsized":"2024-01-27T02:32:55.426312+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:56:58.873502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-01-27T02:32:54.430613+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:32:54.430613+0000","last_peered":"2024-01-27T02:32:54.430613+0000","last_clean":"2024-01-27T02:32:54.430613+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:32:54.430613+0000","last_undegraded":"2024-01-27T02:32:54.430613+0000","last_fullsized":"2024-01-27T02:32:54.430613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":705,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194607,"statfs":{"total":200000000,"available":199277617,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195183,"statfs":{"total":200000000,"available":199867487,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":128,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195183,"statfs":{"total":200000000,"available":199868075,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":705,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194607,"statfs":{"total":200000000,"available":199277707,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]}]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":128,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195183,"statfs":{"total":200000000,"available":199868075,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199866517,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":135,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195177,"statfs":{"total":200000000,"available":199861683,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195183,"statfs":{"total":200000000,"available":199867431,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]}]}],"pool_statfs":[]}} 2024-01-27T02:32:59.982 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:33:00.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.151+0000 7f73150c3700 1 Processor -- start 2024-01-27T02:33:00.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.152+0000 7f73150c3700 1 -- start start 2024-01-27T02:33:00.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.152+0000 7f73150c3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f73100a7c10 0x7f73100ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:00.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.152+0000 7f73150c3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73100a7fe0 0x7f73100b0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:00.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.152+0000 7f73150c3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f731015a880 con 0x7f73100ae880 2024-01-27T02:33:00.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.152+0000 7f73150c3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73100afde0 con 0x7f73100a7c10 2024-01-27T02:33:00.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.152+0000 7f73150c3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73100af950 con 0x7f73100a7fe0 2024-01-27T02:33:00.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.153+0000 7f730ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f73100a7c10 0x7f73100ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:00.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.153+0000 7f730e59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f73100ae880 0x7f731006f360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49246/0 (socket says 172.21.15.7:49246) 2024-01-27T02:33:00.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.153+0000 7f730ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f73100a7c10 0x7f73100ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38720/0 (socket says 172.21.15.7:38720) 2024-01-27T02:33:00.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.153+0000 7f730e59c700 1 -- 172.21.15.7:0/2893739532 learned_addr learned my addr 172.21.15.7:0/2893739532 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:00.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.153+0000 7f730f59e700 1 --2- 172.21.15.7:0/2893739532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73100a7fe0 0x7f73100b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:00.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.153+0000 7f730ed9d700 1 -- 172.21.15.7:0/2893739532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73100a7fe0 msgr2=0x7f73100b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:00.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.153+0000 7f730ed9d700 1 --2- 172.21.15.7:0/2893739532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73100a7fe0 0x7f73100b0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:00.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.153+0000 7f730ed9d700 1 -- 172.21.15.7:0/2893739532 >> v1:172.21.15.7:6789/0 conn(0x7f73100ae880 legacy=0x7f731006f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:00.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.153+0000 7f730ed9d700 1 -- 172.21.15.7:0/2893739532 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73100aed40 con 0x7f73100a7c10 2024-01-27T02:33:00.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.153+0000 7f730dd9b700 1 -- 172.21.15.7:0/2893739532 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2569585666 0 0) 0x7f731015a880 con 0x7f73100ae880 2024-01-27T02:33:00.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.153+0000 7f730f59e700 1 --2- 172.21.15.7:0/2893739532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73100a7fe0 0x7f73100b0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:00.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.153+0000 7f730ed9d700 1 --2- 172.21.15.7:0/2893739532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73100a7c10 0x7f73100ae340 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f73000098e0 tx=0x7f73000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a39875a44eeabb server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:00.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.153+0000 7f730dd9b700 1 -- 172.21.15.7:0/2893739532 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7300009e80 con 0x7f73100a7c10 2024-01-27T02:33:00.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.153+0000 7f730dd9b700 1 -- 172.21.15.7:0/2893739532 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730000b860 con 0x7f73100a7c10 2024-01-27T02:33:00.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.153+0000 7f730dd9b700 1 -- 172.21.15.7:0/2893739532 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730000bbe0 con 0x7f73100a7c10 2024-01-27T02:33:00.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.154+0000 7f73150c3700 1 -- 172.21.15.7:0/2893739532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73100a7c10 msgr2=0x7f73100ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:00.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.154+0000 7f73150c3700 1 --2- 172.21.15.7:0/2893739532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73100a7c10 0x7f73100ae340 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f73000098e0 tx=0x7f73000091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:00.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.154+0000 7f73150c3700 1 -- 172.21.15.7:0/2893739532 shutdown_connections 2024-01-27T02:33:00.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.154+0000 7f73150c3700 1 --2- 172.21.15.7:0/2893739532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73100a7fe0 0x7f73100b0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:00.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.154+0000 7f73150c3700 1 --2- 172.21.15.7:0/2893739532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73100a7c10 0x7f73100ae340 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:00.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.154+0000 7f73150c3700 1 -- 172.21.15.7:0/2893739532 >> 172.21.15.7:0/2893739532 conn(0x7f73100679b0 msgr2=0x7f7310067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:00.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.154+0000 7f73150c3700 1 -- 172.21.15.7:0/2893739532 shutdown_connections 2024-01-27T02:33:00.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.154+0000 7f73150c3700 1 -- 172.21.15.7:0/2893739532 wait complete. 2024-01-27T02:33:00.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.155+0000 7f73150c3700 1 Processor -- start 2024-01-27T02:33:00.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.155+0000 7f73150c3700 1 -- start start 2024-01-27T02:33:00.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.156+0000 7f73150c3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73100a7c10 0x7f731014ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:00.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.156+0000 7f73150c3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73100a7fe0 0x7f7310151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:00.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.156+0000 7f73150c3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73100ae880 0x7f7310098400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:00.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.156+0000 7f73150c3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731015bf30 con 0x7f73100a7fe0 2024-01-27T02:33:00.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.156+0000 7f73150c3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731014f1f0 con 0x7f73100a7c10 2024-01-27T02:33:00.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.156+0000 7f73150c3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7310151a80 con 0x7f73100ae880 2024-01-27T02:33:00.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.156+0000 7f730e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73100a7fe0 0x7f7310151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:00.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.156+0000 7f730e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73100a7fe0 0x7f7310151540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38722/0 (socket says 172.21.15.7:38722) 2024-01-27T02:33:00.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.156+0000 7f730e59c700 1 -- 172.21.15.7:0/1963130814 learned_addr learned my addr 172.21.15.7:0/1963130814 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:00.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.157+0000 7f730f59e700 1 --2- 172.21.15.7:0/1963130814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73100ae880 0x7f7310098400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:00.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.157+0000 7f730ed9d700 1 --2- 172.21.15.7:0/1963130814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73100a7c10 0x7f731014ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:00.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.157+0000 7f730e59c700 1 -- 172.21.15.7:0/1963130814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73100ae880 msgr2=0x7f7310098400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:00.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.157+0000 7f730e59c700 1 --2- 172.21.15.7:0/1963130814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73100ae880 0x7f7310098400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:00.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.157+0000 7f730e59c700 1 -- 172.21.15.7:0/1963130814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73100a7c10 msgr2=0x7f731014ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:00.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.157+0000 7f730e59c700 1 --2- 172.21.15.7:0/1963130814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73100a7c10 0x7f731014ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:00.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.157+0000 7f730e59c700 1 -- 172.21.15.7:0/1963130814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7300009420 con 0x7f73100a7fe0 2024-01-27T02:33:00.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.157+0000 7f730e59c700 1 --2- 172.21.15.7:0/1963130814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73100a7fe0 0x7f7310151540 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f730400c610 tx=0x7f7304000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:00.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.157+0000 7f72f6ffd700 1 -- 172.21.15.7:0/1963130814 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7304005b30 con 0x7f73100a7fe0 2024-01-27T02:33:00.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.157+0000 7f72f6ffd700 1 -- 172.21.15.7:0/1963130814 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7304005c90 con 0x7f73100a7fe0 2024-01-27T02:33:00.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.157+0000 7f73150c3700 1 -- 172.21.15.7:0/1963130814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f731015b620 con 0x7f73100a7fe0 2024-01-27T02:33:00.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.157+0000 7f72f6ffd700 1 -- 172.21.15.7:0/1963130814 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7304007300 con 0x7f73100a7fe0 2024-01-27T02:33:00.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.157+0000 7f73150c3700 1 -- 172.21.15.7:0/1963130814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f731015b8e0 con 0x7f73100a7fe0 2024-01-27T02:33:00.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.158+0000 7f73150c3700 1 -- 172.21.15.7:0/1963130814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f731004c710 con 0x7f73100a7fe0 2024-01-27T02:33:00.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.159+0000 7f72f6ffd700 1 -- 172.21.15.7:0/1963130814 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f730401b3b0 con 0x7f73100a7fe0 2024-01-27T02:33:00.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.159+0000 7f72f6ffd700 1 --2- 172.21.15.7:0/1963130814 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f72fc066f30 0x7f72fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:00.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.159+0000 7f72f6ffd700 1 -- 172.21.15.7:0/1963130814 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7304083a70 con 0x7f73100a7fe0 2024-01-27T02:33:00.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.159+0000 7f730ed9d700 1 --2- 172.21.15.7:0/1963130814 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f72fc066f30 0x7f72fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:00.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.160+0000 7f730ed9d700 1 --2- 172.21.15.7:0/1963130814 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f72fc066f30 0x7f72fc0693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f73000091e0 tx=0x7f7300017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:00.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.163+0000 7f72f6ffd700 1 -- 172.21.15.7:0/1963130814 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7304050ce0 con 0x7f73100a7fe0 2024-01-27T02:33:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.362+0000 7f73150c3700 1 -- 172.21.15.7:0/1963130814 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f73100af020 con 0x7f72fc066f30 2024-01-27T02:33:00.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.370+0000 7f72f6ffd700 1 -- 172.21.15.7:0/1963130814 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260109 (secure 0 0 0) 0x7f73100af020 con 0x7f72fc066f30 2024-01-27T02:33:00.374 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:00.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.373+0000 7f73150c3700 1 -- 172.21.15.7:0/1963130814 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f72fc066f30 msgr2=0x7f72fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:00.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.373+0000 7f73150c3700 1 --2- 172.21.15.7:0/1963130814 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f72fc066f30 0x7f72fc0693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f73000091e0 tx=0x7f7300017000 comp rx=0 tx=0).stop 2024-01-27T02:33:00.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.373+0000 7f73150c3700 1 -- 172.21.15.7:0/1963130814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73100a7fe0 msgr2=0x7f7310151540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:00.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.373+0000 7f73150c3700 1 --2- 172.21.15.7:0/1963130814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73100a7fe0 0x7f7310151540 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f730400c610 tx=0x7f7304000fe0 comp rx=0 tx=0).stop 2024-01-27T02:33:00.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.374+0000 7f73150c3700 1 -- 172.21.15.7:0/1963130814 shutdown_connections 2024-01-27T02:33:00.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.374+0000 7f73150c3700 1 --2- 172.21.15.7:0/1963130814 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f72fc066f30 0x7f72fc0693e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:00.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.374+0000 7f73150c3700 1 --2- 172.21.15.7:0/1963130814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73100ae880 0x7f7310098400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:00.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.374+0000 7f73150c3700 1 --2- 172.21.15.7:0/1963130814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73100a7c10 0x7f731014ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:00.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.374+0000 7f73150c3700 1 --2- 172.21.15.7:0/1963130814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73100a7fe0 0x7f7310151540 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:00.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.374+0000 7f73150c3700 1 -- 172.21.15.7:0/1963130814 >> 172.21.15.7:0/1963130814 conn(0x7f73100679b0 msgr2=0x7f731015aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:00.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.374+0000 7f73150c3700 1 -- 172.21.15.7:0/1963130814 shutdown_connections 2024-01-27T02:33:00.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.374+0000 7f73150c3700 1 -- 172.21.15.7:0/1963130814 wait complete. 2024-01-27T02:33:00.377 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:33:00.387 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2024-01-27T02:32:59.585219+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2192,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560307,"statfs":{"total":1600000000,"available":1597754592,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001475"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.432620+0000","last_change":"2024-01-27T02:32:55.432559+0000","last_active":"2024-01-27T02:32:55.432620+0000","last_peered":"2024-01-27T02:32:55.432620+0000","last_clean":"2024-01-27T02:32:55.432620+0000","last_became_active":"2024-01-27T02:32:55.432290+0000","last_became_peered":"2024-01-27T02:32:55.432290+0000","last_unstale":"2024-01-27T02:32:55.432620+0000","last_undegraded":"2024-01-27T02:32:55.432620+0000","last_fullsized":"2024-01-27T02:32:55.432620+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:28:22.292009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.429694+0000","last_change":"2024-01-27T02:32:55.429005+0000","last_active":"2024-01-27T02:32:55.429694+0000","last_peered":"2024-01-27T02:32:55.429694+0000","last_clean":"2024-01-27T02:32:55.429694+0000","last_became_active":"2024-01-27T02:32:55.428741+0000","last_became_peered":"2024-01-27T02:32:55.428741+0000","last_unstale":"2024-01-27T02:32:55.429694+0000","last_undegraded":"2024-01-27T02:32:55.429694+0000","last_fullsized":"2024-01-27T02:32:55.429694+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:20:24.718650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.428777+0000","last_change":"2024-01-27T02:32:55.428777+0000","last_active":"2024-01-27T02:32:55.428777+0000","last_peered":"2024-01-27T02:32:55.428777+0000","last_clean":"2024-01-27T02:32:55.428777+0000","last_became_active":"2024-01-27T02:32:55.428551+0000","last_became_peered":"2024-01-27T02:32:55.428551+0000","last_unstale":"2024-01-27T02:32:55.428777+0000","last_undegraded":"2024-01-27T02:32:55.428777+0000","last_fullsized":"2024-01-27T02:32:55.428777+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:18:22.974631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.427680+0000","last_change":"2024-01-27T02:32:55.427680+0000","last_active":"2024-01-27T02:32:55.427680+0000","last_peered":"2024-01-27T02:32:55.427680+0000","last_clean":"2024-01-27T02:32:55.427680+0000","last_became_active":"2024-01-27T02:32:55.427171+0000","last_became_peered":"2024-01-27T02:32:55.427171+0000","last_unstale":"2024-01-27T02:32:55.427680+0000","last_undegraded":"2024-01-27T02:32:55.427680+0000","last_fullsized":"2024-01-27T02:32:55.427680+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:41.306732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.424421+0000","last_change":"2024-01-27T02:32:55.424421+0000","last_active":"2024-01-27T02:32:55.424421+0000","last_peered":"2024-01-27T02:32:55.424421+0000","last_clean":"2024-01-27T02:32:55.424421+0000","last_became_active":"2024-01-27T02:32:55.424127+0000","last_became_peered":"2024-01-27T02:32:55.424127+0000","last_unstale":"2024-01-27T02:32:55.424421+0000","last_undegraded":"2024-01-27T02:32:55.424421+0000","last_fullsized":"2024-01-27T02:32:55.424421+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:36:24.317025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.427590+0000","last_change":"2024-01-27T02:32:55.427405+0000","last_active":"2024-01-27T02:32:55.427590+0000","last_peered":"2024-01-27T02:32:55.427590+0000","last_clean":"2024-01-27T02:32:55.427590+0000","last_became_active":"2024-01-27T02:32:55.427005+0000","last_became_peered":"2024-01-27T02:32:55.427005+0000","last_unstale":"2024-01-27T02:32:55.427590+0000","last_undegraded":"2024-01-27T02:32:55.427590+0000","last_fullsized":"2024-01-27T02:32:55.427590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:55:24.921329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.434431+0000","last_change":"2024-01-27T02:32:55.434431+0000","last_active":"2024-01-27T02:32:55.434431+0000","last_peered":"2024-01-27T02:32:55.434431+0000","last_clean":"2024-01-27T02:32:55.434431+0000","last_became_active":"2024-01-27T02:32:55.434017+0000","last_became_peered":"2024-01-27T02:32:55.434017+0000","last_unstale":"2024-01-27T02:32:55.434431+0000","last_undegraded":"2024-01-27T02:32:55.434431+0000","last_fullsized":"2024-01-27T02:32:55.434431+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:20:24.718650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.430034+0000","last_change":"2024-01-27T02:32:55.430034+0000","last_active":"2024-01-27T02:32:55.430034+0000","last_peered":"2024-01-27T02:32:55.430034+0000","last_clean":"2024-01-27T02:32:55.430034+0000","last_became_active":"2024-01-27T02:32:55.428610+0000","last_became_peered":"2024-01-27T02:32:55.428610+0000","last_unstale":"2024-01-27T02:32:55.430034+0000","last_undegraded":"2024-01-27T02:32:55.430034+0000","last_fullsized":"2024-01-27T02:32:55.430034+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:19:26.991510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.425916+0000","last_change":"2024-01-27T02:32:55.425841+0000","last_active":"2024-01-27T02:32:55.425916+0000","last_peered":"2024-01-27T02:32:55.425916+0000","last_clean":"2024-01-27T02:32:55.425916+0000","last_became_active":"2024-01-27T02:32:55.425595+0000","last_became_peered":"2024-01-27T02:32:55.425595+0000","last_unstale":"2024-01-27T02:32:55.425916+0000","last_undegraded":"2024-01-27T02:32:55.425916+0000","last_fullsized":"2024-01-27T02:32:55.425916+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:31:14.098748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.423222+0000","last_change":"2024-01-27T02:32:55.423222+0000","last_active":"2024-01-27T02:32:55.423222+0000","last_peered":"2024-01-27T02:32:55.423222+0000","last_clean":"2024-01-27T02:32:55.423222+0000","last_became_active":"2024-01-27T02:32:55.422972+0000","last_became_peered":"2024-01-27T02:32:55.422972+0000","last_unstale":"2024-01-27T02:32:55.423222+0000","last_undegraded":"2024-01-27T02:32:55.423222+0000","last_fullsized":"2024-01-27T02:32:55.423222+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:23:47.824914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.436132+0000","last_change":"2024-01-27T02:32:55.436132+0000","last_active":"2024-01-27T02:32:55.436132+0000","last_peered":"2024-01-27T02:32:55.436132+0000","last_clean":"2024-01-27T02:32:55.436132+0000","last_became_active":"2024-01-27T02:32:55.435868+0000","last_became_peered":"2024-01-27T02:32:55.435868+0000","last_unstale":"2024-01-27T02:32:55.436132+0000","last_undegraded":"2024-01-27T02:32:55.436132+0000","last_fullsized":"2024-01-27T02:32:55.436132+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:46:09.976295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.433280+0000","last_change":"2024-01-27T02:32:55.433280+0000","last_active":"2024-01-27T02:32:55.433280+0000","last_peered":"2024-01-27T02:32:55.433280+0000","last_clean":"2024-01-27T02:32:55.433280+0000","last_became_active":"2024-01-27T02:32:55.433031+0000","last_became_peered":"2024-01-27T02:32:55.433031+0000","last_unstale":"2024-01-27T02:32:55.433280+0000","last_undegraded":"2024-01-27T02:32:55.433280+0000","last_fullsized":"2024-01-27T02:32:55.433280+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:07:27.939664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.423167+0000","last_change":"2024-01-27T02:32:55.423167+0000","last_active":"2024-01-27T02:32:55.423167+0000","last_peered":"2024-01-27T02:32:55.423167+0000","last_clean":"2024-01-27T02:32:55.423167+0000","last_became_active":"2024-01-27T02:32:55.422900+0000","last_became_peered":"2024-01-27T02:32:55.422900+0000","last_unstale":"2024-01-27T02:32:55.423167+0000","last_undegraded":"2024-01-27T02:32:55.423167+0000","last_fullsized":"2024-01-27T02:32:55.423167+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:40:12.903562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.425220+0000","last_change":"2024-01-27T02:32:55.424027+0000","last_active":"2024-01-27T02:32:55.425220+0000","last_peered":"2024-01-27T02:32:55.425220+0000","last_clean":"2024-01-27T02:32:55.425220+0000","last_became_active":"2024-01-27T02:32:55.423082+0000","last_became_peered":"2024-01-27T02:32:55.423082+0000","last_unstale":"2024-01-27T02:32:55.425220+0000","last_undegraded":"2024-01-27T02:32:55.425220+0000","last_fullsized":"2024-01-27T02:32:55.425220+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:18:22.974631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.434120+0000","last_change":"2024-01-27T02:32:55.434120+0000","last_active":"2024-01-27T02:32:55.434120+0000","last_peered":"2024-01-27T02:32:55.434120+0000","last_clean":"2024-01-27T02:32:55.434120+0000","last_became_active":"2024-01-27T02:32:55.433842+0000","last_became_peered":"2024-01-27T02:32:55.433842+0000","last_unstale":"2024-01-27T02:32:55.434120+0000","last_undegraded":"2024-01-27T02:32:55.434120+0000","last_fullsized":"2024-01-27T02:32:55.434120+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:57:41.306732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.431903+0000","last_change":"2024-01-27T02:32:55.431903+0000","last_active":"2024-01-27T02:32:55.431903+0000","last_peered":"2024-01-27T02:32:55.431903+0000","last_clean":"2024-01-27T02:32:55.431903+0000","last_became_active":"2024-01-27T02:32:55.431559+0000","last_became_peered":"2024-01-27T02:32:55.431559+0000","last_unstale":"2024-01-27T02:32:55.431903+0000","last_undegraded":"2024-01-27T02:32:55.431903+0000","last_fullsized":"2024-01-27T02:32:55.431903+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:31:20.996303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.434269+0000","last_change":"2024-01-27T02:32:55.434269+0000","last_active":"2024-01-27T02:32:55.434269+0000","last_peered":"2024-01-27T02:32:55.434269+0000","last_clean":"2024-01-27T02:32:55.434269+0000","last_became_active":"2024-01-27T02:32:55.433715+0000","last_became_peered":"2024-01-27T02:32:55.433715+0000","last_unstale":"2024-01-27T02:32:55.434269+0000","last_undegraded":"2024-01-27T02:32:55.434269+0000","last_fullsized":"2024-01-27T02:32:55.434269+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:08:14.832481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.427507+0000","last_change":"2024-01-27T02:32:55.425793+0000","last_active":"2024-01-27T02:32:55.427507+0000","last_peered":"2024-01-27T02:32:55.427507+0000","last_clean":"2024-01-27T02:32:55.427507+0000","last_became_active":"2024-01-27T02:32:55.424878+0000","last_became_peered":"2024-01-27T02:32:55.424878+0000","last_unstale":"2024-01-27T02:32:55.427507+0000","last_undegraded":"2024-01-27T02:32:55.427507+0000","last_fullsized":"2024-01-27T02:32:55.427507+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:23:00.299763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.425248+0000","last_change":"2024-01-27T02:32:55.424197+0000","last_active":"2024-01-27T02:32:55.425248+0000","last_peered":"2024-01-27T02:32:55.425248+0000","last_clean":"2024-01-27T02:32:55.425248+0000","last_became_active":"2024-01-27T02:32:55.423416+0000","last_became_peered":"2024-01-27T02:32:55.423416+0000","last_unstale":"2024-01-27T02:32:55.425248+0000","last_undegraded":"2024-01-27T02:32:55.425248+0000","last_fullsized":"2024-01-27T02:32:55.425248+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:31:20.996303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.425846+0000","last_change":"2024-01-27T02:32:55.425846+0000","last_active":"2024-01-27T02:32:55.425846+0000","last_peered":"2024-01-27T02:32:55.425846+0000","last_clean":"2024-01-27T02:32:55.425846+0000","last_became_active":"2024-01-27T02:32:55.425529+0000","last_became_peered":"2024-01-27T02:32:55.425529+0000","last_unstale":"2024-01-27T02:32:55.425846+0000","last_undegraded":"2024-01-27T02:32:55.425846+0000","last_fullsized":"2024-01-27T02:32:55.425846+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:17:56.186917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.432777+0000","last_change":"2024-01-27T02:32:55.432711+0000","last_active":"2024-01-27T02:32:55.432777+0000","last_peered":"2024-01-27T02:32:55.432777+0000","last_clean":"2024-01-27T02:32:55.432777+0000","last_became_active":"2024-01-27T02:32:55.432486+0000","last_became_peered":"2024-01-27T02:32:55.432486+0000","last_unstale":"2024-01-27T02:32:55.432777+0000","last_undegraded":"2024-01-27T02:32:55.432777+0000","last_fullsized":"2024-01-27T02:32:55.432777+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:08:23.991383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.429788+0000","last_change":"2024-01-27T02:32:55.429330+0000","last_active":"2024-01-27T02:32:55.429788+0000","last_peered":"2024-01-27T02:32:55.429788+0000","last_clean":"2024-01-27T02:32:55.429788+0000","last_became_active":"2024-01-27T02:32:55.429067+0000","last_became_peered":"2024-01-27T02:32:55.429067+0000","last_unstale":"2024-01-27T02:32:55.429788+0000","last_undegraded":"2024-01-27T02:32:55.429788+0000","last_fullsized":"2024-01-27T02:32:55.429788+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:08:02.904513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.433117+0000","last_change":"2024-01-27T02:32:55.433117+0000","last_active":"2024-01-27T02:32:55.433117+0000","last_peered":"2024-01-27T02:32:55.433117+0000","last_clean":"2024-01-27T02:32:55.433117+0000","last_became_active":"2024-01-27T02:32:55.432532+0000","last_became_peered":"2024-01-27T02:32:55.432532+0000","last_unstale":"2024-01-27T02:32:55.433117+0000","last_undegraded":"2024-01-27T02:32:55.433117+0000","last_fullsized":"2024-01-27T02:32:55.433117+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:08:23.991383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.427809+0000","last_change":"2024-01-27T02:32:55.427809+0000","last_active":"2024-01-27T02:32:55.427809+0000","last_peered":"2024-01-27T02:32:55.427809+0000","last_clean":"2024-01-27T02:32:55.427809+0000","last_became_active":"2024-01-27T02:32:55.427419+0000","last_became_peered":"2024-01-27T02:32:55.427419+0000","last_unstale":"2024-01-27T02:32:55.427809+0000","last_undegraded":"2024-01-27T02:32:55.427809+0000","last_fullsized":"2024-01-27T02:32:55.427809+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:42:38.742239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.428755+0000","last_change":"2024-01-27T02:32:55.428755+0000","last_active":"2024-01-27T02:32:55.428755+0000","last_peered":"2024-01-27T02:32:55.428755+0000","last_clean":"2024-01-27T02:32:55.428755+0000","last_became_active":"2024-01-27T02:32:55.428463+0000","last_became_peered":"2024-01-27T02:32:55.428463+0000","last_unstale":"2024-01-27T02:32:55.428755+0000","last_undegraded":"2024-01-27T02:32:55.428755+0000","last_fullsized":"2024-01-27T02:32:55.428755+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:26:44.336012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.427571+0000","last_change":"2024-01-27T02:32:55.427241+0000","last_active":"2024-01-27T02:32:55.427571+0000","last_peered":"2024-01-27T02:32:55.427571+0000","last_clean":"2024-01-27T02:32:55.427571+0000","last_became_active":"2024-01-27T02:32:55.426483+0000","last_became_peered":"2024-01-27T02:32:55.426483+0000","last_unstale":"2024-01-27T02:32:55.427571+0000","last_undegraded":"2024-01-27T02:32:55.427571+0000","last_fullsized":"2024-01-27T02:32:55.427571+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:33:30.530805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.428928+0000","last_change":"2024-01-27T02:32:55.428928+0000","last_active":"2024-01-27T02:32:55.428928+0000","last_peered":"2024-01-27T02:32:55.428928+0000","last_clean":"2024-01-27T02:32:55.428928+0000","last_became_active":"2024-01-27T02:32:55.428452+0000","last_became_peered":"2024-01-27T02:32:55.428452+0000","last_unstale":"2024-01-27T02:32:55.428928+0000","last_undegraded":"2024-01-27T02:32:55.428928+0000","last_fullsized":"2024-01-27T02:32:55.428928+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:42:38.742239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.431320+0000","last_change":"2024-01-27T02:32:55.431320+0000","last_active":"2024-01-27T02:32:55.431320+0000","last_peered":"2024-01-27T02:32:55.431320+0000","last_clean":"2024-01-27T02:32:55.431320+0000","last_became_active":"2024-01-27T02:32:55.431083+0000","last_became_peered":"2024-01-27T02:32:55.431083+0000","last_unstale":"2024-01-27T02:32:55.431320+0000","last_undegraded":"2024-01-27T02:32:55.431320+0000","last_fullsized":"2024-01-27T02:32:55.431320+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:55:24.921329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.430856+0000","last_change":"2024-01-27T02:32:55.430562+0000","last_active":"2024-01-27T02:32:55.430856+0000","last_peered":"2024-01-27T02:32:55.430856+0000","last_clean":"2024-01-27T02:32:55.430856+0000","last_became_active":"2024-01-27T02:32:55.430293+0000","last_became_peered":"2024-01-27T02:32:55.430293+0000","last_unstale":"2024-01-27T02:32:55.430856+0000","last_undegraded":"2024-01-27T02:32:55.430856+0000","last_fullsized":"2024-01-27T02:32:55.430856+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:17:11.996957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.434818+0000","last_change":"2024-01-27T02:32:55.434818+0000","last_active":"2024-01-27T02:32:55.434818+0000","last_peered":"2024-01-27T02:32:55.434818+0000","last_clean":"2024-01-27T02:32:55.434818+0000","last_became_active":"2024-01-27T02:32:55.434416+0000","last_became_peered":"2024-01-27T02:32:55.434416+0000","last_unstale":"2024-01-27T02:32:55.434818+0000","last_undegraded":"2024-01-27T02:32:55.434818+0000","last_fullsized":"2024-01-27T02:32:55.434818+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:07:27.939664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.431513+0000","last_change":"2024-01-27T02:32:55.431399+0000","last_active":"2024-01-27T02:32:55.431513+0000","last_peered":"2024-01-27T02:32:55.431513+0000","last_clean":"2024-01-27T02:32:55.431513+0000","last_became_active":"2024-01-27T02:32:55.431159+0000","last_became_peered":"2024-01-27T02:32:55.431159+0000","last_unstale":"2024-01-27T02:32:55.431513+0000","last_undegraded":"2024-01-27T02:32:55.431513+0000","last_fullsized":"2024-01-27T02:32:55.431513+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:45:18.498656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.432090+0000","last_change":"2024-01-27T02:32:55.432090+0000","last_active":"2024-01-27T02:32:55.432090+0000","last_peered":"2024-01-27T02:32:55.432090+0000","last_clean":"2024-01-27T02:32:55.432090+0000","last_became_active":"2024-01-27T02:32:55.431868+0000","last_became_peered":"2024-01-27T02:32:55.431868+0000","last_unstale":"2024-01-27T02:32:55.432090+0000","last_undegraded":"2024-01-27T02:32:55.432090+0000","last_fullsized":"2024-01-27T02:32:55.432090+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:11:03.308052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.432329+0000","last_change":"2024-01-27T02:32:55.432329+0000","last_active":"2024-01-27T02:32:55.432329+0000","last_peered":"2024-01-27T02:32:55.432329+0000","last_clean":"2024-01-27T02:32:55.432329+0000","last_became_active":"2024-01-27T02:32:55.432054+0000","last_became_peered":"2024-01-27T02:32:55.432054+0000","last_unstale":"2024-01-27T02:32:55.432329+0000","last_undegraded":"2024-01-27T02:32:55.432329+0000","last_fullsized":"2024-01-27T02:32:55.432329+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:17:11.996957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.813275+0000","last_change":"2024-01-27T02:32:54.430272+0000","last_active":"2024-01-27T02:32:55.813275+0000","last_peered":"2024-01-27T02:32:55.813275+0000","last_clean":"2024-01-27T02:32:55.813275+0000","last_became_active":"2024-01-27T02:32:54.430013+0000","last_became_peered":"2024-01-27T02:32:54.430013+0000","last_unstale":"2024-01-27T02:32:55.813275+0000","last_undegraded":"2024-01-27T02:32:55.813275+0000","last_fullsized":"2024-01-27T02:32:55.813275+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:45:51.423427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.433531+0000","last_change":"2024-01-27T02:32:55.433446+0000","last_active":"2024-01-27T02:32:55.433531+0000","last_peered":"2024-01-27T02:32:55.433531+0000","last_clean":"2024-01-27T02:32:55.433531+0000","last_became_active":"2024-01-27T02:32:55.433064+0000","last_became_peered":"2024-01-27T02:32:55.433064+0000","last_unstale":"2024-01-27T02:32:55.433531+0000","last_undegraded":"2024-01-27T02:32:55.433531+0000","last_fullsized":"2024-01-27T02:32:55.433531+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:00:44.207288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.426954+0000","last_change":"2024-01-27T02:32:54.433274+0000","last_active":"2024-01-27T02:32:55.426954+0000","last_peered":"2024-01-27T02:32:55.426954+0000","last_clean":"2024-01-27T02:32:55.426954+0000","last_became_active":"2024-01-27T02:32:54.432995+0000","last_became_peered":"2024-01-27T02:32:54.432995+0000","last_unstale":"2024-01-27T02:32:55.426954+0000","last_undegraded":"2024-01-27T02:32:55.426954+0000","last_fullsized":"2024-01-27T02:32:55.426954+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:56:58.873502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.431392+0000","last_change":"2024-01-27T02:32:55.430889+0000","last_active":"2024-01-27T02:32:55.431392+0000","last_peered":"2024-01-27T02:32:55.431392+0000","last_clean":"2024-01-27T02:32:55.431392+0000","last_became_active":"2024-01-27T02:32:55.430647+0000","last_became_peered":"2024-01-27T02:32:55.430647+0000","last_unstale":"2024-01-27T02:32:55.431392+0000","last_undegraded":"2024-01-27T02:32:55.431392+0000","last_fullsized":"2024-01-27T02:32:55.431392+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:00:44.207288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.814523+0000","last_change":"2024-01-27T02:32:54.429820+0000","last_active":"2024-01-27T02:32:55.814523+0000","last_peered":"2024-01-27T02:32:55.814523+0000","last_clean":"2024-01-27T02:32:55.814523+0000","last_became_active":"2024-01-27T02:32:54.429548+0000","last_became_peered":"2024-01-27T02:32:54.429548+0000","last_unstale":"2024-01-27T02:32:55.814523+0000","last_undegraded":"2024-01-27T02:32:55.814523+0000","last_fullsized":"2024-01-27T02:32:55.814523+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:45:17.490073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.427121+0000","last_change":"2024-01-27T02:32:55.427121+0000","last_active":"2024-01-27T02:32:55.427121+0000","last_peered":"2024-01-27T02:32:55.427121+0000","last_clean":"2024-01-27T02:32:55.427121+0000","last_became_active":"2024-01-27T02:32:55.426845+0000","last_became_peered":"2024-01-27T02:32:55.426845+0000","last_unstale":"2024-01-27T02:32:55.427121+0000","last_undegraded":"2024-01-27T02:32:55.427121+0000","last_fullsized":"2024-01-27T02:32:55.427121+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:31:14.098748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.812735+0000","last_change":"2024-01-27T02:32:54.428490+0000","last_active":"2024-01-27T02:32:55.812735+0000","last_peered":"2024-01-27T02:32:55.812735+0000","last_clean":"2024-01-27T02:32:55.812735+0000","last_became_active":"2024-01-27T02:32:54.428289+0000","last_became_peered":"2024-01-27T02:32:54.428289+0000","last_unstale":"2024-01-27T02:32:55.812735+0000","last_undegraded":"2024-01-27T02:32:55.812735+0000","last_fullsized":"2024-01-27T02:32:55.812735+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:54:01.417398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.430351+0000","last_change":"2024-01-27T02:32:55.430351+0000","last_active":"2024-01-27T02:32:55.430351+0000","last_peered":"2024-01-27T02:32:55.430351+0000","last_clean":"2024-01-27T02:32:55.430351+0000","last_became_active":"2024-01-27T02:32:55.429263+0000","last_became_peered":"2024-01-27T02:32:55.429263+0000","last_unstale":"2024-01-27T02:32:55.430351+0000","last_undegraded":"2024-01-27T02:32:55.430351+0000","last_fullsized":"2024-01-27T02:32:55.430351+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:18:22.974631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.813999+0000","last_change":"2024-01-27T02:32:54.427710+0000","last_active":"2024-01-27T02:32:55.813999+0000","last_peered":"2024-01-27T02:32:55.813999+0000","last_clean":"2024-01-27T02:32:55.813999+0000","last_became_active":"2024-01-27T02:32:54.427171+0000","last_became_peered":"2024-01-27T02:32:54.427171+0000","last_unstale":"2024-01-27T02:32:55.813999+0000","last_undegraded":"2024-01-27T02:32:55.813999+0000","last_fullsized":"2024-01-27T02:32:55.813999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:45:03.287924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.432822+0000","last_change":"2024-01-27T02:32:55.432822+0000","last_active":"2024-01-27T02:32:55.432822+0000","last_peered":"2024-01-27T02:32:55.432822+0000","last_clean":"2024-01-27T02:32:55.432822+0000","last_became_active":"2024-01-27T02:32:55.432445+0000","last_became_peered":"2024-01-27T02:32:55.432445+0000","last_unstale":"2024-01-27T02:32:55.432822+0000","last_undegraded":"2024-01-27T02:32:55.432822+0000","last_fullsized":"2024-01-27T02:32:55.432822+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:11:03.308052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.814024+0000","last_change":"2024-01-27T02:32:54.426583+0000","last_active":"2024-01-27T02:32:55.814024+0000","last_peered":"2024-01-27T02:32:55.814024+0000","last_clean":"2024-01-27T02:32:55.814024+0000","last_became_active":"2024-01-27T02:32:54.426339+0000","last_became_peered":"2024-01-27T02:32:54.426339+0000","last_unstale":"2024-01-27T02:32:55.814024+0000","last_undegraded":"2024-01-27T02:32:55.814024+0000","last_fullsized":"2024-01-27T02:32:55.814024+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:40:13.068334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.434849+0000","last_change":"2024-01-27T02:32:55.433990+0000","last_active":"2024-01-27T02:32:55.434849+0000","last_peered":"2024-01-27T02:32:55.434849+0000","last_clean":"2024-01-27T02:32:55.434849+0000","last_became_active":"2024-01-27T02:32:55.433762+0000","last_became_peered":"2024-01-27T02:32:55.433762+0000","last_unstale":"2024-01-27T02:32:55.434849+0000","last_undegraded":"2024-01-27T02:32:55.434849+0000","last_fullsized":"2024-01-27T02:32:55.434849+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:26:44.336012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.419619+0000","last_change":"2024-01-27T02:32:54.425055+0000","last_active":"2024-01-27T02:32:55.419619+0000","last_peered":"2024-01-27T02:32:55.419619+0000","last_clean":"2024-01-27T02:32:55.419619+0000","last_became_active":"2024-01-27T02:32:54.421312+0000","last_became_peered":"2024-01-27T02:32:54.421312+0000","last_unstale":"2024-01-27T02:32:55.419619+0000","last_undegraded":"2024-01-27T02:32:55.419619+0000","last_fullsized":"2024-01-27T02:32:55.419619+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:08:22.982800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.425056+0000","last_change":"2024-01-27T02:32:55.425056+0000","last_active":"2024-01-27T02:32:55.425056+0000","last_peered":"2024-01-27T02:32:55.425056+0000","last_clean":"2024-01-27T02:32:55.425056+0000","last_became_active":"2024-01-27T02:32:55.424571+0000","last_became_peered":"2024-01-27T02:32:55.424571+0000","last_unstale":"2024-01-27T02:32:55.425056+0000","last_undegraded":"2024-01-27T02:32:55.425056+0000","last_fullsized":"2024-01-27T02:32:55.425056+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:27:49.416295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.425295+0000","last_change":"2024-01-27T02:32:54.428730+0000","last_active":"2024-01-27T02:32:55.425295+0000","last_peered":"2024-01-27T02:32:55.425295+0000","last_clean":"2024-01-27T02:32:55.425295+0000","last_became_active":"2024-01-27T02:32:54.428159+0000","last_became_peered":"2024-01-27T02:32:54.428159+0000","last_unstale":"2024-01-27T02:32:55.425295+0000","last_undegraded":"2024-01-27T02:32:55.425295+0000","last_fullsized":"2024-01-27T02:32:55.425295+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:45:51.423427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.432670+0000","last_change":"2024-01-27T02:32:55.432670+0000","last_active":"2024-01-27T02:32:55.432670+0000","last_peered":"2024-01-27T02:32:55.432670+0000","last_clean":"2024-01-27T02:32:55.432670+0000","last_became_active":"2024-01-27T02:32:55.432127+0000","last_became_peered":"2024-01-27T02:32:55.432127+0000","last_unstale":"2024-01-27T02:32:55.432670+0000","last_undegraded":"2024-01-27T02:32:55.432670+0000","last_fullsized":"2024-01-27T02:32:55.432670+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:27:49.416295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.418043+0000","last_change":"2024-01-27T02:32:54.432300+0000","last_active":"2024-01-27T02:32:55.418043+0000","last_peered":"2024-01-27T02:32:55.418043+0000","last_clean":"2024-01-27T02:32:55.418043+0000","last_became_active":"2024-01-27T02:32:54.431991+0000","last_became_peered":"2024-01-27T02:32:54.431991+0000","last_unstale":"2024-01-27T02:32:55.418043+0000","last_undegraded":"2024-01-27T02:32:55.418043+0000","last_fullsized":"2024-01-27T02:32:55.418043+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:37:59.891632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.427548+0000","last_change":"2024-01-27T02:32:55.426731+0000","last_active":"2024-01-27T02:32:55.427548+0000","last_peered":"2024-01-27T02:32:55.427548+0000","last_clean":"2024-01-27T02:32:55.427548+0000","last_became_active":"2024-01-27T02:32:55.425599+0000","last_became_peered":"2024-01-27T02:32:55.425599+0000","last_unstale":"2024-01-27T02:32:55.427548+0000","last_undegraded":"2024-01-27T02:32:55.427548+0000","last_fullsized":"2024-01-27T02:32:55.427548+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:19:30.393076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-01-27T02:32:54.427908+0000","last_change":"2024-01-27T02:32:54.427283+0000","last_active":"2024-01-27T02:32:54.427908+0000","last_peered":"2024-01-27T02:32:54.427908+0000","last_clean":"2024-01-27T02:32:54.427908+0000","last_became_active":"2024-01-27T02:32:54.427007+0000","last_became_peered":"2024-01-27T02:32:54.427007+0000","last_unstale":"2024-01-27T02:32:54.427908+0000","last_undegraded":"2024-01-27T02:32:54.427908+0000","last_fullsized":"2024-01-27T02:32:54.427908+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:35:02.936237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.429717+0000","last_change":"2024-01-27T02:32:55.429108+0000","last_active":"2024-01-27T02:32:55.429717+0000","last_peered":"2024-01-27T02:32:55.429717+0000","last_clean":"2024-01-27T02:32:55.429717+0000","last_became_active":"2024-01-27T02:32:55.428793+0000","last_became_peered":"2024-01-27T02:32:55.428793+0000","last_unstale":"2024-01-27T02:32:55.429717+0000","last_undegraded":"2024-01-27T02:32:55.429717+0000","last_fullsized":"2024-01-27T02:32:55.429717+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:04:28.768090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.812811+0000","last_change":"2024-01-27T02:32:54.432123+0000","last_active":"2024-01-27T02:32:55.812811+0000","last_peered":"2024-01-27T02:32:55.812811+0000","last_clean":"2024-01-27T02:32:55.812811+0000","last_became_active":"2024-01-27T02:32:54.431900+0000","last_became_peered":"2024-01-27T02:32:54.431900+0000","last_unstale":"2024-01-27T02:32:55.812811+0000","last_undegraded":"2024-01-27T02:32:55.812811+0000","last_fullsized":"2024-01-27T02:32:55.812811+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:45:17.490073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.429720+0000","last_change":"2024-01-27T02:32:55.429720+0000","last_active":"2024-01-27T02:32:55.429720+0000","last_peered":"2024-01-27T02:32:55.429720+0000","last_clean":"2024-01-27T02:32:55.429720+0000","last_became_active":"2024-01-27T02:32:55.429498+0000","last_became_peered":"2024-01-27T02:32:55.429498+0000","last_unstale":"2024-01-27T02:32:55.429720+0000","last_undegraded":"2024-01-27T02:32:55.429720+0000","last_fullsized":"2024-01-27T02:32:55.429720+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:20:24.718650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.816355+0000","last_change":"2024-01-27T02:32:54.431720+0000","last_active":"2024-01-27T02:32:55.816355+0000","last_peered":"2024-01-27T02:32:55.816355+0000","last_clean":"2024-01-27T02:32:55.816355+0000","last_became_active":"2024-01-27T02:32:54.430893+0000","last_became_peered":"2024-01-27T02:32:54.430893+0000","last_unstale":"2024-01-27T02:32:55.816355+0000","last_undegraded":"2024-01-27T02:32:55.816355+0000","last_fullsized":"2024-01-27T02:32:55.816355+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:28:21.283426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.424597+0000","last_change":"2024-01-27T02:32:55.424597+0000","last_active":"2024-01-27T02:32:55.424597+0000","last_peered":"2024-01-27T02:32:55.424597+0000","last_clean":"2024-01-27T02:32:55.424597+0000","last_became_active":"2024-01-27T02:32:55.424298+0000","last_became_peered":"2024-01-27T02:32:55.424298+0000","last_unstale":"2024-01-27T02:32:55.424597+0000","last_undegraded":"2024-01-27T02:32:55.424597+0000","last_fullsized":"2024-01-27T02:32:55.424597+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:36:33.291109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.813518+0000","last_change":"2024-01-27T02:32:54.428574+0000","last_active":"2024-01-27T02:32:55.813518+0000","last_peered":"2024-01-27T02:32:55.813518+0000","last_clean":"2024-01-27T02:32:55.813518+0000","last_became_active":"2024-01-27T02:32:54.427872+0000","last_became_peered":"2024-01-27T02:32:54.427872+0000","last_unstale":"2024-01-27T02:32:55.813518+0000","last_undegraded":"2024-01-27T02:32:55.813518+0000","last_fullsized":"2024-01-27T02:32:55.813518+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:31:55.674636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.431174+0000","last_change":"2024-01-27T02:32:55.430523+0000","last_active":"2024-01-27T02:32:55.431174+0000","last_peered":"2024-01-27T02:32:55.431174+0000","last_clean":"2024-01-27T02:32:55.431174+0000","last_became_active":"2024-01-27T02:32:55.430273+0000","last_became_peered":"2024-01-27T02:32:55.430273+0000","last_unstale":"2024-01-27T02:32:55.431174+0000","last_undegraded":"2024-01-27T02:32:55.431174+0000","last_fullsized":"2024-01-27T02:32:55.431174+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:46:09.976295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-01-27T02:32:54.431632+0000","last_change":"2024-01-27T02:32:54.431394+0000","last_active":"2024-01-27T02:32:54.431632+0000","last_peered":"2024-01-27T02:32:54.431632+0000","last_clean":"2024-01-27T02:32:54.431632+0000","last_became_active":"2024-01-27T02:32:54.431120+0000","last_became_peered":"2024-01-27T02:32:54.431120+0000","last_unstale":"2024-01-27T02:32:54.431632+0000","last_undegraded":"2024-01-27T02:32:54.431632+0000","last_fullsized":"2024-01-27T02:32:54.431632+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:36:23.308442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.430204+0000","last_change":"2024-01-27T02:32:55.430204+0000","last_active":"2024-01-27T02:32:55.430204+0000","last_peered":"2024-01-27T02:32:55.430204+0000","last_clean":"2024-01-27T02:32:55.430204+0000","last_became_active":"2024-01-27T02:32:55.428926+0000","last_became_peered":"2024-01-27T02:32:55.428926+0000","last_unstale":"2024-01-27T02:32:55.430204+0000","last_undegraded":"2024-01-27T02:32:55.430204+0000","last_fullsized":"2024-01-27T02:32:55.430204+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:55:24.921329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.418005+0000","last_change":"2024-01-27T02:32:54.431435+0000","last_active":"2024-01-27T02:32:55.418005+0000","last_peered":"2024-01-27T02:32:55.418005+0000","last_clean":"2024-01-27T02:32:55.418005+0000","last_became_active":"2024-01-27T02:32:54.431177+0000","last_became_peered":"2024-01-27T02:32:54.431177+0000","last_unstale":"2024-01-27T02:32:55.418005+0000","last_undegraded":"2024-01-27T02:32:55.418005+0000","last_fullsized":"2024-01-27T02:32:55.418005+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:17:10.988374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.433263+0000","last_change":"2024-01-27T02:32:55.433183+0000","last_active":"2024-01-27T02:32:55.433263+0000","last_peered":"2024-01-27T02:32:55.433263+0000","last_clean":"2024-01-27T02:32:55.433263+0000","last_became_active":"2024-01-27T02:32:55.432953+0000","last_became_peered":"2024-01-27T02:32:55.432953+0000","last_unstale":"2024-01-27T02:32:55.433263+0000","last_undegraded":"2024-01-27T02:32:55.433263+0000","last_fullsized":"2024-01-27T02:32:55.433263+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:17:11.996957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.817669+0000","last_change":"2024-01-27T02:32:54.429801+0000","last_active":"2024-01-27T02:32:55.817669+0000","last_peered":"2024-01-27T02:32:55.817669+0000","last_clean":"2024-01-27T02:32:55.817669+0000","last_became_active":"2024-01-27T02:32:54.429523+0000","last_became_peered":"2024-01-27T02:32:54.429523+0000","last_unstale":"2024-01-27T02:32:55.817669+0000","last_undegraded":"2024-01-27T02:32:55.817669+0000","last_fullsized":"2024-01-27T02:32:55.817669+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:06:09.577224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.426830+0000","last_change":"2024-01-27T02:32:55.426171+0000","last_active":"2024-01-27T02:32:55.426830+0000","last_peered":"2024-01-27T02:32:55.426830+0000","last_clean":"2024-01-27T02:32:55.426830+0000","last_became_active":"2024-01-27T02:32:55.425915+0000","last_became_peered":"2024-01-27T02:32:55.425915+0000","last_unstale":"2024-01-27T02:32:55.426830+0000","last_undegraded":"2024-01-27T02:32:55.426830+0000","last_fullsized":"2024-01-27T02:32:55.426830+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:36:33.291109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.421723+0000","last_change":"2024-01-27T02:32:54.433830+0000","last_active":"2024-01-27T02:32:55.421723+0000","last_peered":"2024-01-27T02:32:55.421723+0000","last_clean":"2024-01-27T02:32:55.421723+0000","last_became_active":"2024-01-27T02:32:54.433524+0000","last_became_peered":"2024-01-27T02:32:54.433524+0000","last_unstale":"2024-01-27T02:32:55.421723+0000","last_undegraded":"2024-01-27T02:32:55.421723+0000","last_fullsized":"2024-01-27T02:32:55.421723+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:45:51.423427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.427654+0000","last_change":"2024-01-27T02:32:55.427654+0000","last_active":"2024-01-27T02:32:55.427654+0000","last_peered":"2024-01-27T02:32:55.427654+0000","last_clean":"2024-01-27T02:32:55.427654+0000","last_became_active":"2024-01-27T02:32:55.426903+0000","last_became_peered":"2024-01-27T02:32:55.426903+0000","last_unstale":"2024-01-27T02:32:55.427654+0000","last_undegraded":"2024-01-27T02:32:55.427654+0000","last_fullsized":"2024-01-27T02:32:55.427654+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:46:09.976295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.813143+0000","last_change":"2024-01-27T02:32:54.428404+0000","last_active":"2024-01-27T02:32:55.813143+0000","last_peered":"2024-01-27T02:32:55.813143+0000","last_clean":"2024-01-27T02:32:55.813143+0000","last_became_active":"2024-01-27T02:32:54.427562+0000","last_became_peered":"2024-01-27T02:32:54.427562+0000","last_unstale":"2024-01-27T02:32:55.813143+0000","last_undegraded":"2024-01-27T02:32:55.813143+0000","last_fullsized":"2024-01-27T02:32:55.813143+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:42:08.346789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.428750+0000","last_change":"2024-01-27T02:32:55.428750+0000","last_active":"2024-01-27T02:32:55.428750+0000","last_peered":"2024-01-27T02:32:55.428750+0000","last_clean":"2024-01-27T02:32:55.428750+0000","last_became_active":"2024-01-27T02:32:55.427769+0000","last_became_peered":"2024-01-27T02:32:55.427769+0000","last_unstale":"2024-01-27T02:32:55.428750+0000","last_undegraded":"2024-01-27T02:32:55.428750+0000","last_fullsized":"2024-01-27T02:32:55.428750+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:46:59.789872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.813387+0000","last_change":"2024-01-27T02:32:54.425812+0000","last_active":"2024-01-27T02:32:55.813387+0000","last_peered":"2024-01-27T02:32:55.813387+0000","last_clean":"2024-01-27T02:32:55.813387+0000","last_became_active":"2024-01-27T02:32:54.425573+0000","last_became_peered":"2024-01-27T02:32:54.425573+0000","last_unstale":"2024-01-27T02:32:55.813387+0000","last_undegraded":"2024-01-27T02:32:55.813387+0000","last_fullsized":"2024-01-27T02:32:55.813387+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:53:21.551567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.425996+0000","last_change":"2024-01-27T02:32:55.425996+0000","last_active":"2024-01-27T02:32:55.425996+0000","last_peered":"2024-01-27T02:32:55.425996+0000","last_clean":"2024-01-27T02:32:55.425996+0000","last_became_active":"2024-01-27T02:32:55.425741+0000","last_became_peered":"2024-01-27T02:32:55.425741+0000","last_unstale":"2024-01-27T02:32:55.425996+0000","last_undegraded":"2024-01-27T02:32:55.425996+0000","last_fullsized":"2024-01-27T02:32:55.425996+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:15:51.134403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.426117+0000","last_change":"2024-01-27T02:32:54.427847+0000","last_active":"2024-01-27T02:32:55.426117+0000","last_peered":"2024-01-27T02:32:55.426117+0000","last_clean":"2024-01-27T02:32:55.426117+0000","last_became_active":"2024-01-27T02:32:54.427480+0000","last_became_peered":"2024-01-27T02:32:54.427480+0000","last_unstale":"2024-01-27T02:32:55.426117+0000","last_undegraded":"2024-01-27T02:32:55.426117+0000","last_fullsized":"2024-01-27T02:32:55.426117+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:58:29.492017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.426752+0000","last_change":"2024-01-27T02:32:55.426752+0000","last_active":"2024-01-27T02:32:55.426752+0000","last_peered":"2024-01-27T02:32:55.426752+0000","last_clean":"2024-01-27T02:32:55.426752+0000","last_became_active":"2024-01-27T02:32:55.426479+0000","last_became_peered":"2024-01-27T02:32:55.426479+0000","last_unstale":"2024-01-27T02:32:55.426752+0000","last_undegraded":"2024-01-27T02:32:55.426752+0000","last_fullsized":"2024-01-27T02:32:55.426752+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:56:59.882085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.817450+0000","last_change":"2024-01-27T02:32:54.428946+0000","last_active":"2024-01-27T02:32:55.817450+0000","last_peered":"2024-01-27T02:32:55.817450+0000","last_clean":"2024-01-27T02:32:55.817450+0000","last_became_active":"2024-01-27T02:32:54.428552+0000","last_became_peered":"2024-01-27T02:32:54.428552+0000","last_unstale":"2024-01-27T02:32:55.817450+0000","last_undegraded":"2024-01-27T02:32:55.817450+0000","last_fullsized":"2024-01-27T02:32:55.817450+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:15:50.125821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.429001+0000","last_change":"2024-01-27T02:32:55.429001+0000","last_active":"2024-01-27T02:32:55.429001+0000","last_peered":"2024-01-27T02:32:55.429001+0000","last_clean":"2024-01-27T02:32:55.429001+0000","last_became_active":"2024-01-27T02:32:55.428745+0000","last_became_peered":"2024-01-27T02:32:55.428745+0000","last_unstale":"2024-01-27T02:32:55.429001+0000","last_undegraded":"2024-01-27T02:32:55.429001+0000","last_fullsized":"2024-01-27T02:32:55.429001+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:42:09.355371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.426210+0000","last_change":"2024-01-27T02:32:54.430722+0000","last_active":"2024-01-27T02:32:55.426210+0000","last_peered":"2024-01-27T02:32:55.426210+0000","last_clean":"2024-01-27T02:32:55.426210+0000","last_became_active":"2024-01-27T02:32:54.430517+0000","last_became_peered":"2024-01-27T02:32:54.430517+0000","last_unstale":"2024-01-27T02:32:55.426210+0000","last_undegraded":"2024-01-27T02:32:55.426210+0000","last_fullsized":"2024-01-27T02:32:55.426210+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:31:55.674636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.430911+0000","last_change":"2024-01-27T02:32:55.430911+0000","last_active":"2024-01-27T02:32:55.430911+0000","last_peered":"2024-01-27T02:32:55.430911+0000","last_clean":"2024-01-27T02:32:55.430911+0000","last_became_active":"2024-01-27T02:32:55.430643+0000","last_became_peered":"2024-01-27T02:32:55.430643+0000","last_unstale":"2024-01-27T02:32:55.430911+0000","last_undegraded":"2024-01-27T02:32:55.430911+0000","last_fullsized":"2024-01-27T02:32:55.430911+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:40:14.076916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.425618+0000","last_change":"2024-01-27T02:32:54.432055+0000","last_active":"2024-01-27T02:32:55.425618+0000","last_peered":"2024-01-27T02:32:55.425618+0000","last_clean":"2024-01-27T02:32:55.425618+0000","last_became_active":"2024-01-27T02:32:54.431830+0000","last_became_peered":"2024-01-27T02:32:54.431830+0000","last_unstale":"2024-01-27T02:32:55.425618+0000","last_undegraded":"2024-01-27T02:32:55.425618+0000","last_fullsized":"2024-01-27T02:32:55.425618+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:16:07.293532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.434662+0000","last_change":"2024-01-27T02:32:55.434662+0000","last_active":"2024-01-27T02:32:55.434662+0000","last_peered":"2024-01-27T02:32:55.434662+0000","last_clean":"2024-01-27T02:32:55.434662+0000","last_became_active":"2024-01-27T02:32:55.434080+0000","last_became_peered":"2024-01-27T02:32:55.434080+0000","last_unstale":"2024-01-27T02:32:55.434662+0000","last_undegraded":"2024-01-27T02:32:55.434662+0000","last_fullsized":"2024-01-27T02:32:55.434662+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:27:49.416295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-01-27T02:32:54.431128+0000","last_change":"2024-01-27T02:32:54.431077+0000","last_active":"2024-01-27T02:32:54.431128+0000","last_peered":"2024-01-27T02:32:54.431128+0000","last_clean":"2024-01-27T02:32:54.431128+0000","last_became_active":"2024-01-27T02:32:54.430833+0000","last_became_peered":"2024-01-27T02:32:54.430833+0000","last_unstale":"2024-01-27T02:32:54.431128+0000","last_undegraded":"2024-01-27T02:32:54.431128+0000","last_fullsized":"2024-01-27T02:32:54.431128+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:06:09.577224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.428454+0000","last_change":"2024-01-27T02:32:55.428454+0000","last_active":"2024-01-27T02:32:55.428454+0000","last_peered":"2024-01-27T02:32:55.428454+0000","last_clean":"2024-01-27T02:32:55.428454+0000","last_became_active":"2024-01-27T02:32:55.428154+0000","last_became_peered":"2024-01-27T02:32:55.428154+0000","last_unstale":"2024-01-27T02:32:55.428454+0000","last_undegraded":"2024-01-27T02:32:55.428454+0000","last_fullsized":"2024-01-27T02:32:55.428454+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:35:03.944819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.812631+0000","last_change":"2024-01-27T02:32:54.431112+0000","last_active":"2024-01-27T02:32:55.812631+0000","last_peered":"2024-01-27T02:32:55.812631+0000","last_clean":"2024-01-27T02:32:55.812631+0000","last_became_active":"2024-01-27T02:32:54.430334+0000","last_became_peered":"2024-01-27T02:32:54.430334+0000","last_unstale":"2024-01-27T02:32:55.812631+0000","last_undegraded":"2024-01-27T02:32:55.812631+0000","last_fullsized":"2024-01-27T02:32:55.812631+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:35:02.936237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.434288+0000","last_change":"2024-01-27T02:32:55.434288+0000","last_active":"2024-01-27T02:32:55.434288+0000","last_peered":"2024-01-27T02:32:55.434288+0000","last_clean":"2024-01-27T02:32:55.434288+0000","last_became_active":"2024-01-27T02:32:55.433886+0000","last_became_peered":"2024-01-27T02:32:55.433886+0000","last_unstale":"2024-01-27T02:32:55.434288+0000","last_undegraded":"2024-01-27T02:32:55.434288+0000","last_fullsized":"2024-01-27T02:32:55.434288+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:08:14.832481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-01-27T02:32:54.429057+0000","last_change":"2024-01-27T02:32:54.428797+0000","last_active":"2024-01-27T02:32:54.429057+0000","last_peered":"2024-01-27T02:32:54.429057+0000","last_clean":"2024-01-27T02:32:54.429057+0000","last_became_active":"2024-01-27T02:32:54.428226+0000","last_became_peered":"2024-01-27T02:32:54.428226+0000","last_unstale":"2024-01-27T02:32:54.429057+0000","last_undegraded":"2024-01-27T02:32:54.429057+0000","last_fullsized":"2024-01-27T02:32:54.429057+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:36:23.308442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.434509+0000","last_change":"2024-01-27T02:32:55.434509+0000","last_active":"2024-01-27T02:32:55.434509+0000","last_peered":"2024-01-27T02:32:55.434509+0000","last_clean":"2024-01-27T02:32:55.434509+0000","last_became_active":"2024-01-27T02:32:55.434237+0000","last_became_peered":"2024-01-27T02:32:55.434237+0000","last_unstale":"2024-01-27T02:32:55.434509+0000","last_undegraded":"2024-01-27T02:32:55.434509+0000","last_fullsized":"2024-01-27T02:32:55.434509+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:08:02.904513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-01-27T02:32:54.430699+0000","last_change":"2024-01-27T02:32:54.430115+0000","last_active":"2024-01-27T02:32:54.430699+0000","last_peered":"2024-01-27T02:32:54.430699+0000","last_clean":"2024-01-27T02:32:54.430699+0000","last_became_active":"2024-01-27T02:32:54.429096+0000","last_became_peered":"2024-01-27T02:32:54.429096+0000","last_unstale":"2024-01-27T02:32:54.430699+0000","last_undegraded":"2024-01-27T02:32:54.430699+0000","last_fullsized":"2024-01-27T02:32:54.430699+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:56:58.873502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.433245+0000","last_change":"2024-01-27T02:32:55.432722+0000","last_active":"2024-01-27T02:32:55.433245+0000","last_peered":"2024-01-27T02:32:55.433245+0000","last_clean":"2024-01-27T02:32:55.433245+0000","last_became_active":"2024-01-27T02:32:55.432439+0000","last_became_peered":"2024-01-27T02:32:55.432439+0000","last_unstale":"2024-01-27T02:32:55.433245+0000","last_undegraded":"2024-01-27T02:32:55.433245+0000","last_fullsized":"2024-01-27T02:32:55.433245+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:38:00.900215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-01-27T02:32:54.427929+0000","last_change":"2024-01-27T02:32:54.427576+0000","last_active":"2024-01-27T02:32:54.427929+0000","last_peered":"2024-01-27T02:32:54.427929+0000","last_clean":"2024-01-27T02:32:54.427929+0000","last_became_active":"2024-01-27T02:32:54.427340+0000","last_became_peered":"2024-01-27T02:32:54.427340+0000","last_unstale":"2024-01-27T02:32:54.427929+0000","last_undegraded":"2024-01-27T02:32:54.427929+0000","last_fullsized":"2024-01-27T02:32:54.427929+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:17:55.178334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.433493+0000","last_change":"2024-01-27T02:32:55.433493+0000","last_active":"2024-01-27T02:32:55.433493+0000","last_peered":"2024-01-27T02:32:55.433493+0000","last_clean":"2024-01-27T02:32:55.433493+0000","last_became_active":"2024-01-27T02:32:55.433250+0000","last_became_peered":"2024-01-27T02:32:55.433250+0000","last_unstale":"2024-01-27T02:32:55.433493+0000","last_undegraded":"2024-01-27T02:32:55.433493+0000","last_fullsized":"2024-01-27T02:32:55.433493+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:00:44.207288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-01-27T02:32:54.430792+0000","last_change":"2024-01-27T02:32:54.430585+0000","last_active":"2024-01-27T02:32:54.430792+0000","last_peered":"2024-01-27T02:32:54.430792+0000","last_clean":"2024-01-27T02:32:54.430792+0000","last_became_active":"2024-01-27T02:32:54.429985+0000","last_became_peered":"2024-01-27T02:32:54.429985+0000","last_unstale":"2024-01-27T02:32:54.430792+0000","last_undegraded":"2024-01-27T02:32:54.430792+0000","last_fullsized":"2024-01-27T02:32:54.430792+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:17:55.178334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.425268+0000","last_change":"2024-01-27T02:32:55.424345+0000","last_active":"2024-01-27T02:32:55.425268+0000","last_peered":"2024-01-27T02:32:55.425268+0000","last_clean":"2024-01-27T02:32:55.425268+0000","last_became_active":"2024-01-27T02:32:55.423765+0000","last_became_peered":"2024-01-27T02:32:55.423765+0000","last_unstale":"2024-01-27T02:32:55.425268+0000","last_undegraded":"2024-01-27T02:32:55.425268+0000","last_fullsized":"2024-01-27T02:32:55.425268+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:19:26.991510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.427333+0000","last_change":"2024-01-27T02:32:54.430012+0000","last_active":"2024-01-27T02:32:55.427333+0000","last_peered":"2024-01-27T02:32:55.427333+0000","last_clean":"2024-01-27T02:32:55.427333+0000","last_became_active":"2024-01-27T02:32:54.429786+0000","last_became_peered":"2024-01-27T02:32:54.429786+0000","last_unstale":"2024-01-27T02:32:55.427333+0000","last_undegraded":"2024-01-27T02:32:55.427333+0000","last_fullsized":"2024-01-27T02:32:55.427333+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:11:35.438367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.432303+0000","last_change":"2024-01-27T02:32:55.432236+0000","last_active":"2024-01-27T02:32:55.432303+0000","last_peered":"2024-01-27T02:32:55.432303+0000","last_clean":"2024-01-27T02:32:55.432303+0000","last_became_active":"2024-01-27T02:32:55.431653+0000","last_became_peered":"2024-01-27T02:32:55.431653+0000","last_unstale":"2024-01-27T02:32:55.432303+0000","last_undegraded":"2024-01-27T02:32:55.432303+0000","last_fullsized":"2024-01-27T02:32:55.432303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:38:00.900215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.420148+0000","last_change":"2024-01-27T02:32:54.432509+0000","last_active":"2024-01-27T02:32:55.420148+0000","last_peered":"2024-01-27T02:32:55.420148+0000","last_clean":"2024-01-27T02:32:55.420148+0000","last_became_active":"2024-01-27T02:32:54.432268+0000","last_became_peered":"2024-01-27T02:32:54.432268+0000","last_unstale":"2024-01-27T02:32:55.420148+0000","last_undegraded":"2024-01-27T02:32:55.420148+0000","last_fullsized":"2024-01-27T02:32:55.420148+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:00:43.198705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.430320+0000","last_change":"2024-01-27T02:32:55.430320+0000","last_active":"2024-01-27T02:32:55.430320+0000","last_peered":"2024-01-27T02:32:55.430320+0000","last_clean":"2024-01-27T02:32:55.430320+0000","last_became_active":"2024-01-27T02:32:55.428673+0000","last_became_peered":"2024-01-27T02:32:55.428673+0000","last_unstale":"2024-01-27T02:32:55.430320+0000","last_undegraded":"2024-01-27T02:32:55.430320+0000","last_fullsized":"2024-01-27T02:32:55.430320+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:54.410338+0000","last_clean_scrub_stamp":"2024-01-27T02:32:54.410338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:31:14.098748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-27T02:32:55.426312+0000","last_change":"2024-01-27T02:32:54.426805+0000","last_active":"2024-01-27T02:32:55.426312+0000","last_peered":"2024-01-27T02:32:55.426312+0000","last_clean":"2024-01-27T02:32:55.426312+0000","last_became_active":"2024-01-27T02:32:54.426589+0000","last_became_peered":"2024-01-27T02:32:54.426589+0000","last_unstale":"2024-01-27T02:32:55.426312+0000","last_undegraded":"2024-01-27T02:32:55.426312+0000","last_fullsized":"2024-01-27T02:32:55.426312+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:32:53.401755+0000","last_clean_scrub_stamp":"2024-01-27T02:32:53.401755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:56:58.873502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-01-27T02:32:54.430613+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:32:54.430613+0000","last_peered":"2024-01-27T02:32:54.430613+0000","last_clean":"2024-01-27T02:32:54.430613+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:32:54.430613+0000","last_undegraded":"2024-01-27T02:32:54.430613+0000","last_fullsized":"2024-01-27T02:32:54.430613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":705,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194607,"statfs":{"total":200000000,"available":199277617,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195183,"statfs":{"total":200000000,"available":199867487,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":128,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195183,"statfs":{"total":200000000,"available":199868075,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":705,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194607,"statfs":{"total":200000000,"available":199277707,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]}]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":128,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195183,"statfs":{"total":200000000,"available":199868075,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199866517,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":135,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195177,"statfs":{"total":200000000,"available":199861683,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195183,"statfs":{"total":200000000,"available":199867431,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]}]}],"pool_statfs":[]}} 2024-01-27T02:33:00.391 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-27T02:33:00.391 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-27T02:33:00.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.568+0000 7ff4aa292700 1 Processor -- start 2024-01-27T02:33:00.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.568+0000 7ff4aa292700 1 -- start start 2024-01-27T02:33:00.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.570+0000 7ff4aa292700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff4a4156b60 0x7ff4a40ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:00.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.570+0000 7ff4aa292700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a40aeaa0 con 0x7ff4a40ad9f0 2024-01-27T02:33:00.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.570+0000 7ff4aa292700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a40ade90 con 0x7ff4a4156f30 2024-01-27T02:33:00.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.570+0000 7ff4aa292700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a40ae170 con 0x7ff4a4156b60 2024-01-27T02:33:00.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.570+0000 7ff4a37fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff4a40ad9f0 0x7ff4a40aef30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49248/0 (socket says 172.21.15.7:49248) 2024-01-27T02:33:00.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.570+0000 7ff4a37fe700 1 -- 172.21.15.7:0/760505614 learned_addr learned my addr 172.21.15.7:0/760505614 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:00.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.570+0000 7ff4a3fff700 1 --2- 172.21.15.7:0/760505614 >> v2:172.21.15.188:3300/0 conn(0x7ff4a4156b60 0x7ff4a40ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:00.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.570+0000 7ff4a2ffd700 1 -- 172.21.15.7:0/760505614 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2151994846 0 0) 0x7ff4a40aeaa0 con 0x7ff4a40ad9f0 2024-01-27T02:33:00.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.570+0000 7ff4a2ffd700 1 -- 172.21.15.7:0/760505614 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff48c003570 con 0x7ff4a40ad9f0 2024-01-27T02:33:00.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.570+0000 7ff4a2ffd700 1 -- 172.21.15.7:0/760505614 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4088756033 0 0) 0x7ff48c003570 con 0x7ff4a40ad9f0 2024-01-27T02:33:00.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.570+0000 7ff4a2ffd700 1 -- 172.21.15.7:0/760505614 >> v2:172.21.15.188:3300/0 conn(0x7ff4a4156b60 msgr2=0x7ff4a40ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:00.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.570+0000 7ff4a2ffd700 1 --2- 172.21.15.7:0/760505614 >> v2:172.21.15.188:3300/0 conn(0x7ff4a4156b60 0x7ff4a40ad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:00.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.570+0000 7ff4a2ffd700 1 -- 172.21.15.7:0/760505614 >> v1:172.21.15.188:6789/0 conn(0x7ff4a4156f30 legacy=0x7ff4a415a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:00.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.570+0000 7ff4a2ffd700 1 -- 172.21.15.7:0/760505614 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a40ae450 con 0x7ff4a40ad9f0 2024-01-27T02:33:00.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.570+0000 7ff4a2ffd700 1 -- 172.21.15.7:0/760505614 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7ff498001470 con 0x7ff4a40ad9f0 2024-01-27T02:33:00.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.570+0000 7ff4a2ffd700 1 -- 172.21.15.7:0/760505614 >> v1:172.21.15.7:6789/0 conn(0x7ff4a40ad9f0 legacy=0x7ff4a40aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:00.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.571+0000 7ff4a2ffd700 1 --2- 172.21.15.7:0/760505614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff48c003a60 0x7ff48c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:00.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.571+0000 7ff4a2ffd700 1 --2- 172.21.15.7:0/760505614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff48c004810 0x7ff48c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:00.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.571+0000 7ff4a2ffd700 1 --2- 172.21.15.7:0/760505614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff48c0051c0 0x7ff48c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:00.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.571+0000 7ff4a2ffd700 1 -- 172.21.15.7:0/760505614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a40ae170 con 0x7ff48c003a60 2024-01-27T02:33:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.571+0000 7ff4a2ffd700 1 -- 172.21.15.7:0/760505614 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c00c050 con 0x7ff48c004810 2024-01-27T02:33:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.571+0000 7ff4a2ffd700 1 -- 172.21.15.7:0/760505614 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c00c210 con 0x7ff48c0051c0 2024-01-27T02:33:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.571+0000 7ff4a3fff700 1 --2- 172.21.15.7:0/760505614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff48c003a60 0x7ff48c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.571+0000 7ff4a882f700 1 --2- 172.21.15.7:0/760505614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff48c0051c0 0x7ff48c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.571+0000 7ff4a37fe700 1 --2- 172.21.15.7:0/760505614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff48c004810 0x7ff48c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.571+0000 7ff4a3fff700 1 -- 172.21.15.7:0/760505614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff48c0051c0 msgr2=0x7ff48c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.571+0000 7ff4a3fff700 1 --2- 172.21.15.7:0/760505614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff48c0051c0 0x7ff48c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.571+0000 7ff4a3fff700 1 -- 172.21.15.7:0/760505614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff48c004810 msgr2=0x7ff48c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.571+0000 7ff4a3fff700 1 --2- 172.21.15.7:0/760505614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff48c004810 0x7ff48c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.571+0000 7ff4a3fff700 1 -- 172.21.15.7:0/760505614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c003570 con 0x7ff48c003a60 2024-01-27T02:33:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.571+0000 7ff4a882f700 1 --2- 172.21.15.7:0/760505614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff48c0051c0 0x7ff48c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.571+0000 7ff4a37fe700 1 --2- 172.21.15.7:0/760505614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff48c004810 0x7ff48c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.571+0000 7ff4a882f700 1 -- 172.21.15.7:0/760505614 reap_dead start 2024-01-27T02:33:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.571+0000 7ff4a3fff700 1 --2- 172.21.15.7:0/760505614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff48c003a60 0x7ff48c003e30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff494009c00 tx=0x7ff4940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57cd7d8188a0ec98 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.571+0000 7ff4a2ffd700 1 -- 172.21.15.7:0/760505614 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49400e710 con 0x7ff48c003a60 2024-01-27T02:33:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.571+0000 7ff4a2ffd700 1 -- 172.21.15.7:0/760505614 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49400e870 con 0x7ff48c003a60 2024-01-27T02:33:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.571+0000 7ff4a2ffd700 1 -- 172.21.15.7:0/760505614 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49400ec30 con 0x7ff48c003a60 2024-01-27T02:33:00.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.572+0000 7ff4aa292700 1 -- 172.21.15.7:0/760505614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff48c003a60 msgr2=0x7ff48c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:00.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.572+0000 7ff4aa292700 1 --2- 172.21.15.7:0/760505614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff48c003a60 0x7ff48c003e30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff494009c00 tx=0x7ff4940091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:00.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.572+0000 7ff4aa292700 1 -- 172.21.15.7:0/760505614 shutdown_connections 2024-01-27T02:33:00.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.572+0000 7ff4aa292700 1 --2- 172.21.15.7:0/760505614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff48c003a60 0x7ff48c003e30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:00.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.572+0000 7ff4aa292700 1 -- 172.21.15.7:0/760505614 >> 172.21.15.7:0/760505614 conn(0x7ff4a4068480 msgr2=0x7ff4a40cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:00.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.572+0000 7ff4aa292700 1 -- 172.21.15.7:0/760505614 shutdown_connections 2024-01-27T02:33:00.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.572+0000 7ff4aa292700 1 -- 172.21.15.7:0/760505614 wait complete. 2024-01-27T02:33:00.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.573+0000 7ff4aa292700 1 Processor -- start 2024-01-27T02:33:00.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.573+0000 7ff4aa292700 1 -- start start 2024-01-27T02:33:00.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.574+0000 7ff4aa292700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff48c003a60 0x7ff4a4149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:00.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.574+0000 7ff4aa292700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4a414bff0 0x7ff4a4149c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:00.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.574+0000 7ff4aa292700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a414aaa0 0x7ff4a414af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.574+0000 7ff4aa292700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a4147270 con 0x7ff4a414aaa0 2024-01-27T02:33:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.574+0000 7ff4aa292700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a4137de0 con 0x7ff48c003a60 2024-01-27T02:33:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.574+0000 7ff4aa292700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a414c560 con 0x7ff4a414bff0 2024-01-27T02:33:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.574+0000 7ff4a882f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a414aaa0 0x7ff4a414af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.574+0000 7ff4a882f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a414aaa0 0x7ff4a414af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38736/0 (socket says 172.21.15.7:38736) 2024-01-27T02:33:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.574+0000 7ff4a882f700 1 -- 172.21.15.7:0/2021959699 learned_addr learned my addr 172.21.15.7:0/2021959699 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.575+0000 7ff4a37fe700 1 --2- 172.21.15.7:0/2021959699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4a414bff0 0x7ff4a4149c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.575+0000 7ff4a3fff700 1 --2- 172.21.15.7:0/2021959699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff48c003a60 0x7ff4a4149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.575+0000 7ff4a882f700 1 -- 172.21.15.7:0/2021959699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4a414bff0 msgr2=0x7ff4a4149c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.575+0000 7ff4a882f700 1 --2- 172.21.15.7:0/2021959699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4a414bff0 0x7ff4a4149c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.575+0000 7ff4a882f700 1 -- 172.21.15.7:0/2021959699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff48c003a60 msgr2=0x7ff4a4149740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.575+0000 7ff4a882f700 1 --2- 172.21.15.7:0/2021959699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff48c003a60 0x7ff4a4149740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.575+0000 7ff4a882f700 1 -- 172.21.15.7:0/2021959699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a414c840 con 0x7ff4a414aaa0 2024-01-27T02:33:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.575+0000 7ff4a3fff700 1 --2- 172.21.15.7:0/2021959699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff48c003a60 0x7ff4a4149740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.575+0000 7ff4a882f700 1 --2- 172.21.15.7:0/2021959699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a414aaa0 0x7ff4a414af10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff4a406dab0 tx=0x7ff49c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.575+0000 7ff48bfff700 1 -- 172.21.15.7:0/2021959699 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c012590 con 0x7ff4a414aaa0 2024-01-27T02:33:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.575+0000 7ff48bfff700 1 -- 172.21.15.7:0/2021959699 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49c012710 con 0x7ff4a414aaa0 2024-01-27T02:33:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.575+0000 7ff4aa292700 1 -- 172.21.15.7:0/2021959699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a414cb20 con 0x7ff4a414aaa0 2024-01-27T02:33:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.575+0000 7ff48bfff700 1 -- 172.21.15.7:0/2021959699 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c012ad0 con 0x7ff4a414aaa0 2024-01-27T02:33:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.575+0000 7ff4aa292700 1 -- 172.21.15.7:0/2021959699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a414b450 con 0x7ff4a414aaa0 2024-01-27T02:33:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.576+0000 7ff4aa292700 1 -- 172.21.15.7:0/2021959699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a4072060 con 0x7ff4a414aaa0 2024-01-27T02:33:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.577+0000 7ff48bfff700 1 -- 172.21.15.7:0/2021959699 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff49c003030 con 0x7ff4a414aaa0 2024-01-27T02:33:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.577+0000 7ff48bfff700 1 --2- 172.21.15.7:0/2021959699 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff490066f30 0x7ff4900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.577+0000 7ff48bfff700 1 -- 172.21.15.7:0/2021959699 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff49c084440 con 0x7ff4a414aaa0 2024-01-27T02:33:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.577+0000 7ff4a3fff700 1 --2- 172.21.15.7:0/2021959699 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff490066f30 0x7ff4900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.578+0000 7ff4a3fff700 1 --2- 172.21.15.7:0/2021959699 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff490066f30 0x7ff4900693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff4940091e0 tx=0x7ff494019000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.582+0000 7ff48bfff700 1 -- 172.21.15.7:0/2021959699 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff49c055140 con 0x7ff4a414aaa0 2024-01-27T02:33:00.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.868+0000 7ff4aa292700 1 -- 172.21.15.7:0/2021959699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff4a41473b0 con 0x7ff4a414aaa0 2024-01-27T02:33:00.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.871+0000 7ff48bfff700 1 -- 172.21.15.7:0/2021959699 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7ff49c054cd0 con 0x7ff4a414aaa0 2024-01-27T02:33:00.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.873+0000 7ff4aa292700 1 -- 172.21.15.7:0/2021959699 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff490066f30 msgr2=0x7ff4900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:00.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.873+0000 7ff4aa292700 1 --2- 172.21.15.7:0/2021959699 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff490066f30 0x7ff4900693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff4940091e0 tx=0x7ff494019000 comp rx=0 tx=0).stop 2024-01-27T02:33:00.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.873+0000 7ff4aa292700 1 -- 172.21.15.7:0/2021959699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a414aaa0 msgr2=0x7ff4a414af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:00.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.873+0000 7ff4aa292700 1 --2- 172.21.15.7:0/2021959699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a414aaa0 0x7ff4a414af10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff4a406dab0 tx=0x7ff49c009040 comp rx=0 tx=0).stop 2024-01-27T02:33:00.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.873+0000 7ff4aa292700 1 -- 172.21.15.7:0/2021959699 shutdown_connections 2024-01-27T02:33:00.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.873+0000 7ff4aa292700 1 --2- 172.21.15.7:0/2021959699 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff490066f30 0x7ff4900693e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:00.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.873+0000 7ff4aa292700 1 --2- 172.21.15.7:0/2021959699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a414aaa0 0x7ff4a414af10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:00.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.873+0000 7ff4aa292700 1 --2- 172.21.15.7:0/2021959699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff48c003a60 0x7ff4a4149740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:00.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.873+0000 7ff4aa292700 1 --2- 172.21.15.7:0/2021959699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4a414bff0 0x7ff4a4149c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:00.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.873+0000 7ff4aa292700 1 -- 172.21.15.7:0/2021959699 >> 172.21.15.7:0/2021959699 conn(0x7ff4a4068480 msgr2=0x7ff4a40b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:00.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.873+0000 7ff4aa292700 1 -- 172.21.15.7:0/2021959699 shutdown_connections 2024-01-27T02:33:00.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:00.874+0000 7ff4aa292700 1 -- 172.21.15.7:0/2021959699 wait complete. 2024-01-27T02:33:00.877 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-01-27T02:33:00.887 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:33:01.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.018+0000 7f8ed3304700 1 Processor -- start 2024-01-27T02:33:01.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.019+0000 7f8ed3304700 1 -- start start 2024-01-27T02:33:01.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.020+0000 7f8ed3304700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8ecc159f40 0x7f8ecc0c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.020+0000 7f8ed3304700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ecc0c30c0 con 0x7f8ecc0aafc0 2024-01-27T02:33:01.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.020+0000 7f8ed3304700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ecc15e960 con 0x7f8ecc0c3a90 2024-01-27T02:33:01.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.020+0000 7f8ed3304700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ecc0c4550 con 0x7f8ecc159f40 2024-01-27T02:33:01.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.020+0000 7f8ed10a0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8ecc159f40 0x7f8ecc0c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.020+0000 7f8ed18a1700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8ecc0aafc0 0x7f8ecc063580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49250/0 (socket says 172.21.15.7:49250) 2024-01-27T02:33:01.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.020+0000 7f8ed18a1700 1 -- 172.21.15.7:0/348751849 learned_addr learned my addr 172.21.15.7:0/348751849 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:01.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.020+0000 7f8ed10a0700 1 -- 172.21.15.7:0/348751849 >> v1:172.21.15.188:6789/0 conn(0x7f8ecc0c3a90 legacy=0x7f8ecc0aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:01.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.020+0000 7f8ed10a0700 1 -- 172.21.15.7:0/348751849 >> v1:172.21.15.7:6789/0 conn(0x7f8ecc0aafc0 legacy=0x7f8ecc063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:01.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.020+0000 7f8ed10a0700 1 -- 172.21.15.7:0/348751849 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ecc15ec40 con 0x7f8ecc159f40 2024-01-27T02:33:01.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.020+0000 7f8ed10a0700 1 --2- 172.21.15.7:0/348751849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ecc159f40 0x7f8ecc0c3550 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80098e0 tx=0x7f8ec80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8db809962451764 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:01.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.020+0000 7f8ec3fff700 1 -- 172.21.15.7:0/348751849 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8009e80 con 0x7f8ecc159f40 2024-01-27T02:33:01.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.021+0000 7f8ec3fff700 1 -- 172.21.15.7:0/348751849 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec800b860 con 0x7f8ecc159f40 2024-01-27T02:33:01.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.021+0000 7f8ec3fff700 1 -- 172.21.15.7:0/348751849 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec800bbe0 con 0x7f8ecc159f40 2024-01-27T02:33:01.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.021+0000 7f8ed3304700 1 -- 172.21.15.7:0/348751849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ecc159f40 msgr2=0x7f8ecc0c3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:01.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.021+0000 7f8ed3304700 1 --2- 172.21.15.7:0/348751849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ecc159f40 0x7f8ecc0c3550 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80098e0 tx=0x7f8ec80091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.021+0000 7f8ed3304700 1 -- 172.21.15.7:0/348751849 shutdown_connections 2024-01-27T02:33:01.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.021+0000 7f8ed3304700 1 --2- 172.21.15.7:0/348751849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ecc159f40 0x7f8ecc0c3550 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.021+0000 7f8ed3304700 1 -- 172.21.15.7:0/348751849 >> 172.21.15.7:0/348751849 conn(0x7f8ecc0cdc30 msgr2=0x7f8ecc0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:01.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.021+0000 7f8ed3304700 1 -- 172.21.15.7:0/348751849 shutdown_connections 2024-01-27T02:33:01.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.021+0000 7f8ed3304700 1 -- 172.21.15.7:0/348751849 wait complete. 2024-01-27T02:33:01.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.022+0000 7f8ed3304700 1 Processor -- start 2024-01-27T02:33:01.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.022+0000 7f8ed3304700 1 -- start start 2024-01-27T02:33:01.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.023+0000 7f8ed3304700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ecc0aafc0 0x7f8ecc0b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.023+0000 7f8ed3304700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ecc0c3a90 0x7f8ecc15ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.023+0000 7f8ed3304700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ecc159f40 0x7f8ecc0c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.023+0000 7f8ed3304700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc15d3c0 con 0x7f8ecc0aafc0 2024-01-27T02:33:01.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.023+0000 7f8ed3304700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc0c79f0 con 0x7f8ecc0c3a90 2024-01-27T02:33:01.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.023+0000 7f8ed3304700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc0b20b0 con 0x7f8ecc159f40 2024-01-27T02:33:01.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.023+0000 7f8ed10a0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ecc0aafc0 0x7f8ecc0b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.024+0000 7f8ed10a0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ecc0aafc0 0x7f8ecc0b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38758/0 (socket says 172.21.15.7:38758) 2024-01-27T02:33:01.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.024+0000 7f8ed10a0700 1 -- 172.21.15.7:0/3117570671 learned_addr learned my addr 172.21.15.7:0/3117570671 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:01.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.024+0000 7f8ed18a1700 1 --2- 172.21.15.7:0/3117570671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ecc159f40 0x7f8ecc0c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.024+0000 7f8ed089f700 1 --2- 172.21.15.7:0/3117570671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ecc0c3a90 0x7f8ecc15ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.024+0000 7f8ed10a0700 1 -- 172.21.15.7:0/3117570671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ecc159f40 msgr2=0x7f8ecc0c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:01.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.024+0000 7f8ed10a0700 1 --2- 172.21.15.7:0/3117570671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ecc159f40 0x7f8ecc0c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.024+0000 7f8ed10a0700 1 -- 172.21.15.7:0/3117570671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ecc0c3a90 msgr2=0x7f8ecc15ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:01.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.024+0000 7f8ed10a0700 1 --2- 172.21.15.7:0/3117570671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ecc0c3a90 0x7f8ecc15ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.024+0000 7f8ed10a0700 1 -- 172.21.15.7:0/3117570671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec8009420 con 0x7f8ecc0aafc0 2024-01-27T02:33:01.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.024+0000 7f8ed089f700 1 --2- 172.21.15.7:0/3117570671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ecc0c3a90 0x7f8ecc15ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.024+0000 7f8ed10a0700 1 --2- 172.21.15.7:0/3117570671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ecc0aafc0 0x7f8ecc0b1a20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc06dcc0 tx=0x7f8ec8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.024+0000 7f8ec17fa700 1 -- 172.21.15.7:0/3117570671 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec800baa0 con 0x7f8ecc0aafc0 2024-01-27T02:33:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.024+0000 7f8ec17fa700 1 -- 172.21.15.7:0/3117570671 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec8025420 con 0x7f8ecc0aafc0 2024-01-27T02:33:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.024+0000 7f8ed3304700 1 -- 172.21.15.7:0/3117570671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ecc16a8c0 con 0x7f8ecc0aafc0 2024-01-27T02:33:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.024+0000 7f8ec17fa700 1 -- 172.21.15.7:0/3117570671 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80257e0 con 0x7f8ecc0aafc0 2024-01-27T02:33:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.024+0000 7f8ed3304700 1 -- 172.21.15.7:0/3117570671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ecc16ae80 con 0x7f8ecc0aafc0 2024-01-27T02:33:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.025+0000 7f8ed3304700 1 -- 172.21.15.7:0/3117570671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ecc04c710 con 0x7f8ecc0aafc0 2024-01-27T02:33:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.026+0000 7f8ec17fa700 1 -- 172.21.15.7:0/3117570671 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8ec8023070 con 0x7f8ecc0aafc0 2024-01-27T02:33:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.026+0000 7f8ec17fa700 1 --2- 172.21.15.7:0/3117570671 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8eb0066f30 0x7f8eb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.026+0000 7f8ec17fa700 1 -- 172.21.15.7:0/3117570671 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8ec8085c50 con 0x7f8ecc0aafc0 2024-01-27T02:33:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.027+0000 7f8ed089f700 1 --2- 172.21.15.7:0/3117570671 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8eb0066f30 0x7f8eb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.027+0000 7f8ed089f700 1 --2- 172.21.15.7:0/3117570671 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8eb0066f30 0x7f8eb00693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc007fd0 tx=0x7f8ebc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:01.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.031+0000 7f8ec17fa700 1 -- 172.21.15.7:0/3117570671 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ec808b020 con 0x7f8ecc0aafc0 2024-01-27T02:33:01.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.280+0000 7f8ed3304700 1 -- 172.21.15.7:0/3117570671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ecc06f770 con 0x7f8ecc0aafc0 2024-01-27T02:33:01.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.281+0000 7f8ec17fa700 1 -- 172.21.15.7:0/3117570671 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f8ec8052ec0 con 0x7f8ecc0aafc0 2024-01-27T02:33:01.281 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:01.282 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4864,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/892460188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":892460188},{"type":"v1","addr":"172.21.15.188:6837","nonce":892460188}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4867,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1657319349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1657319349},{"type":"v1","addr":"172.21.15.188:6836","nonce":1657319349}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4879,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2190571534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2190571534},{"type":"v1","addr":"172.21.15.7:6834","nonce":2190571534}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:32:54.849803+0000","modified":"2024-01-27T02:32:58.809118+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4882},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4882":{"gid":4882,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6835/1734594727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1734594727},{"type":"v1","addr":"172.21.15.7:6835","nonce":1734594727}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-01-27T02:33:01.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.283+0000 7f8ed3304700 1 -- 172.21.15.7:0/3117570671 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8eb0066f30 msgr2=0x7f8eb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:01.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.283+0000 7f8ed3304700 1 --2- 172.21.15.7:0/3117570671 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8eb0066f30 0x7f8eb00693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc007fd0 tx=0x7f8ebc009040 comp rx=0 tx=0).stop 2024-01-27T02:33:01.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.283+0000 7f8ed3304700 1 -- 172.21.15.7:0/3117570671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ecc0aafc0 msgr2=0x7f8ecc0b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:01.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.283+0000 7f8ed3304700 1 --2- 172.21.15.7:0/3117570671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ecc0aafc0 0x7f8ecc0b1a20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc06dcc0 tx=0x7f8ec8017000 comp rx=0 tx=0).stop 2024-01-27T02:33:01.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.283+0000 7f8ed3304700 1 -- 172.21.15.7:0/3117570671 shutdown_connections 2024-01-27T02:33:01.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.283+0000 7f8ed3304700 1 --2- 172.21.15.7:0/3117570671 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8eb0066f30 0x7f8eb00693e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.283+0000 7f8ed3304700 1 --2- 172.21.15.7:0/3117570671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ecc159f40 0x7f8ecc0c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.283+0000 7f8ed3304700 1 --2- 172.21.15.7:0/3117570671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ecc0aafc0 0x7f8ecc0b1a20 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.283+0000 7f8ed3304700 1 --2- 172.21.15.7:0/3117570671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ecc0c3a90 0x7f8ecc15ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.283+0000 7f8ed3304700 1 -- 172.21.15.7:0/3117570671 >> 172.21.15.7:0/3117570671 conn(0x7f8ecc0cdc30 msgr2=0x7f8ecc0aa8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:01.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.283+0000 7f8ed3304700 1 -- 172.21.15.7:0/3117570671 shutdown_connections 2024-01-27T02:33:01.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.283+0000 7f8ed3304700 1 -- 172.21.15.7:0/3117570671 wait complete. 2024-01-27T02:33:01.285 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2024-01-27T02:33:01.699 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-27T02:32:54.849803+0000', 'modified': '2024-01-27T02:32:58.809118+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4882}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4882': {'gid': 4882, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.7:6835/1734594727', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6833', 'nonce': 1734594727}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 1734594727}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-27T02:33:01.700 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-27T02:33:01.700 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-27T02:33:01.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.841+0000 7f2bb370f700 1 Processor -- start 2024-01-27T02:33:01.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.842+0000 7f2bb370f700 1 -- start start 2024-01-27T02:33:01.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.842+0000 7f2bb370f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bac15a100 0x7f2bac0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.843+0000 7f2bb370f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2bac0c4cc0 0x7f2bac163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.843+0000 7f2bb370f700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bac0ce5e0 con 0x7f2bac0cb5b0 2024-01-27T02:33:01.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.843+0000 7f2bb370f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bac0cd9d0 con 0x7f2bac0c4cc0 2024-01-27T02:33:01.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.843+0000 7f2bb370f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac0cdcb0 con 0x7f2bac15a100 2024-01-27T02:33:01.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.843+0000 7f2bb14ab700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bac15a100 0x7f2bac0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.843+0000 7f2bb1cac700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2bac0c4cc0 0x7f2bac163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.843+0000 7f2bb0caa700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f2bac0cb5b0 0x7f2bac0c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:60808/0 (socket says 172.21.15.7:60808) 2024-01-27T02:33:01.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.843+0000 7f2bb0caa700 1 -- 172.21.15.7:0/3682720421 learned_addr learned my addr 172.21.15.7:0/3682720421 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:01.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.843+0000 7f2bb1cac700 1 --2- 172.21.15.7:0/3682720421 >> v2:172.21.15.188:3300/0 conn(0x7f2bac0c4cc0 0x7f2bac163d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:48570/0 (socket says 172.21.15.7:48570) 2024-01-27T02:33:01.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.843+0000 7f2ba3fff700 1 -- 172.21.15.7:0/3682720421 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1630693758 0 0) 0x7f2bac0ce5e0 con 0x7f2bac0cb5b0 2024-01-27T02:33:01.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.843+0000 7f2ba3fff700 1 -- 172.21.15.7:0/3682720421 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b98003570 con 0x7f2bac0cb5b0 2024-01-27T02:33:01.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.843+0000 7f2bb1cac700 1 -- 172.21.15.7:0/3682720421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bac15a100 msgr2=0x7f2bac0cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:01.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.843+0000 7f2bb1cac700 1 --2- 172.21.15.7:0/3682720421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bac15a100 0x7f2bac0cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.843+0000 7f2bb1cac700 1 -- 172.21.15.7:0/3682720421 >> v1:172.21.15.188:6789/0 conn(0x7f2bac0cb5b0 legacy=0x7f2bac0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:01.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.843+0000 7f2bb1cac700 1 -- 172.21.15.7:0/3682720421 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bac0cdf90 con 0x7f2bac0c4cc0 2024-01-27T02:33:01.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.843+0000 7f2bb14ab700 1 --2- 172.21.15.7:0/3682720421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bac15a100 0x7f2bac0cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:33:01.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.844+0000 7f2bb1cac700 1 --2- 172.21.15.7:0/3682720421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bac0c4cc0 0x7f2bac163d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009e60 tx=0x7f2ba4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bad864b0be1f331 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:01.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.844+0000 7f2ba3fff700 1 -- 172.21.15.7:0/3682720421 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba400e070 con 0x7f2bac0c4cc0 2024-01-27T02:33:01.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.844+0000 7f2ba3fff700 1 -- 172.21.15.7:0/3682720421 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba400bab0 con 0x7f2bac0c4cc0 2024-01-27T02:33:01.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.844+0000 7f2ba3fff700 1 -- 172.21.15.7:0/3682720421 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba400be70 con 0x7f2bac0c4cc0 2024-01-27T02:33:01.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.844+0000 7f2bb370f700 1 -- 172.21.15.7:0/3682720421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bac0c4cc0 msgr2=0x7f2bac163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:01.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.844+0000 7f2bb370f700 1 --2- 172.21.15.7:0/3682720421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bac0c4cc0 0x7f2bac163d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009e60 tx=0x7f2ba4009780 comp rx=0 tx=0).stop 2024-01-27T02:33:01.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.844+0000 7f2bb370f700 1 -- 172.21.15.7:0/3682720421 shutdown_connections 2024-01-27T02:33:01.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.844+0000 7f2bb370f700 1 --2- 172.21.15.7:0/3682720421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bac0c4cc0 0x7f2bac163d90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.844+0000 7f2bb370f700 1 --2- 172.21.15.7:0/3682720421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bac15a100 0x7f2bac0cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.844+0000 7f2bb370f700 1 -- 172.21.15.7:0/3682720421 >> 172.21.15.7:0/3682720421 conn(0x7f2bac098020 msgr2=0x7f2bac0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:01.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.844+0000 7f2bb370f700 1 -- 172.21.15.7:0/3682720421 shutdown_connections 2024-01-27T02:33:01.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.845+0000 7f2bb370f700 1 -- 172.21.15.7:0/3682720421 wait complete. 2024-01-27T02:33:01.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.846+0000 7f2bb370f700 1 Processor -- start 2024-01-27T02:33:01.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.846+0000 7f2bb370f700 1 -- start start 2024-01-27T02:33:01.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.847+0000 7f2bb370f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bac0c4cc0 0x7f2bac14dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.847+0000 7f2bb370f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bac0cb5b0 0x7f2bac154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.847+0000 7f2bb370f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bac15a100 0x7f2bac14faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.847+0000 7f2bb370f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac1558f0 con 0x7f2bac0cb5b0 2024-01-27T02:33:01.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.847+0000 7f2bb370f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac154700 con 0x7f2bac0c4cc0 2024-01-27T02:33:01.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.847+0000 7f2bb370f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac153810 con 0x7f2bac15a100 2024-01-27T02:33:01.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.847+0000 7f2bb0caa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bac0cb5b0 0x7f2bac154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.847+0000 7f2bb0caa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bac0cb5b0 0x7f2bac154b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40972/0 (socket says 172.21.15.7:40972) 2024-01-27T02:33:01.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.847+0000 7f2bb0caa700 1 -- 172.21.15.7:0/1996845363 learned_addr learned my addr 172.21.15.7:0/1996845363 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:01.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.847+0000 7f2bb1cac700 1 --2- 172.21.15.7:0/1996845363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bac15a100 0x7f2bac14faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.847+0000 7f2bb14ab700 1 --2- 172.21.15.7:0/1996845363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bac0c4cc0 0x7f2bac14dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.847+0000 7f2bb0caa700 1 -- 172.21.15.7:0/1996845363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bac15a100 msgr2=0x7f2bac14faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:01.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.847+0000 7f2bb0caa700 1 --2- 172.21.15.7:0/1996845363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bac15a100 0x7f2bac14faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.847+0000 7f2bb0caa700 1 -- 172.21.15.7:0/1996845363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bac0c4cc0 msgr2=0x7f2bac14dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:01.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.847+0000 7f2bb0caa700 1 --2- 172.21.15.7:0/1996845363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bac0c4cc0 0x7f2bac14dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.847+0000 7f2bb0caa700 1 -- 172.21.15.7:0/1996845363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba8009420 con 0x7f2bac0cb5b0 2024-01-27T02:33:01.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.847+0000 7f2bb0caa700 1 --2- 172.21.15.7:0/1996845363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bac0cb5b0 0x7f2bac154b90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00c7c0 tx=0x7f2b9c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:01.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.848+0000 7f2ba17fa700 1 -- 172.21.15.7:0/1996845363 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c005cd0 con 0x7f2bac0cb5b0 2024-01-27T02:33:01.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.848+0000 7f2ba17fa700 1 -- 172.21.15.7:0/1996845363 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9c005e30 con 0x7f2bac0cb5b0 2024-01-27T02:33:01.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.848+0000 7f2ba17fa700 1 -- 172.21.15.7:0/1996845363 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c007320 con 0x7f2bac0cb5b0 2024-01-27T02:33:01.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.848+0000 7f2bb370f700 1 -- 172.21.15.7:0/1996845363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba4009a20 con 0x7f2bac0cb5b0 2024-01-27T02:33:01.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.848+0000 7f2bb370f700 1 -- 172.21.15.7:0/1996845363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bac153ba0 con 0x7f2bac0cb5b0 2024-01-27T02:33:01.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.849+0000 7f2ba17fa700 1 -- 172.21.15.7:0/1996845363 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2b9c01b3b0 con 0x7f2bac0cb5b0 2024-01-27T02:33:01.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.850+0000 7f2ba17fa700 1 --2- 172.21.15.7:0/1996845363 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2b90066f30 0x7f2b900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.850+0000 7f2ba17fa700 1 -- 172.21.15.7:0/1996845363 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2b9c083ba0 con 0x7f2bac0cb5b0 2024-01-27T02:33:01.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.850+0000 7f2bb14ab700 1 --2- 172.21.15.7:0/1996845363 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2b90066f30 0x7f2b900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.850+0000 7f2bb14ab700 1 --2- 172.21.15.7:0/1996845363 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2b90066f30 0x7f2b900693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009ea0 tx=0x7f2ba80091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:01.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.852+0000 7f2bbca12b80 1 Processor -- start 2024-01-27T02:33:01.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.852+0000 7f2bbca12b80 1 -- start start 2024-01-27T02:33:01.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.853+0000 7f2bbca12b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a531584010 0x55a5315843e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.853+0000 7f2bbca12b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5315cc360 0x55a5315ae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.853+0000 7f2bbca12b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5315835d0 0x55a5315aed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.853+0000 7f2bbca12b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5314879f0 con 0x55a5315835d0 2024-01-27T02:33:01.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.853+0000 7f2bbca12b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5314d9130 con 0x55a531584010 2024-01-27T02:33:01.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.853+0000 7f2bbca12b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5314d6930 con 0x55a5315cc360 2024-01-27T02:33:01.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.853+0000 7f2bb1cac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5315835d0 0x55a5315aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.853+0000 7f2bb1cac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5315835d0 0x55a5315aed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40990/0 (socket says 172.21.15.7:40990) 2024-01-27T02:33:01.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.853+0000 7f2bb1cac700 1 -- 172.21.15.7:0/2686542974 learned_addr learned my addr 172.21.15.7:0/2686542974 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:01.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.853+0000 7f2bb14ab700 1 --2- 172.21.15.7:0/2686542974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a531584010 0x55a5315843e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.853+0000 7f2bb0caa700 1 --2- 172.21.15.7:0/2686542974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5315cc360 0x55a5315ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.853+0000 7f2bb1cac700 1 -- 172.21.15.7:0/2686542974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5315cc360 msgr2=0x55a5315ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:01.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.853+0000 7f2bb1cac700 1 --2- 172.21.15.7:0/2686542974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5315cc360 0x55a5315ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.853+0000 7f2bb1cac700 1 -- 172.21.15.7:0/2686542974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a531584010 msgr2=0x55a5315843e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:01.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.853+0000 7f2bb1cac700 1 --2- 172.21.15.7:0/2686542974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a531584010 0x55a5315843e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.853+0000 7f2bb1cac700 1 -- 172.21.15.7:0/2686542974 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a531311b10 con 0x55a5315835d0 2024-01-27T02:33:01.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.853+0000 7f2bb0caa700 1 --2- 172.21.15.7:0/2686542974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5315cc360 0x55a5315ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:01.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.853+0000 7f2bb1cac700 1 --2- 172.21.15.7:0/2686542974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5315835d0 0x55a5315aed80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400b610 tx=0x7f2ba4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c01190688479ce server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:01.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.853+0000 7f2b967fc700 1 -- 172.21.15.7:0/2686542974 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba400bda0 con 0x55a5315835d0 2024-01-27T02:33:01.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.853+0000 7f2b967fc700 1 -- 172.21.15.7:0/2686542974 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba4012640 con 0x55a5315835d0 2024-01-27T02:33:01.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.853+0000 7f2b967fc700 1 -- 172.21.15.7:0/2686542974 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba40129e0 con 0x55a5315835d0 2024-01-27T02:33:01.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.854+0000 7f2bbca12b80 1 -- 172.21.15.7:0/2686542974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5315835d0 msgr2=0x55a5315aed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:01.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.854+0000 7f2bbca12b80 1 --2- 172.21.15.7:0/2686542974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5315835d0 0x55a5315aed80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400b610 tx=0x7f2ba4019000 comp rx=0 tx=0).stop 2024-01-27T02:33:01.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.854+0000 7f2bbca12b80 1 -- 172.21.15.7:0/2686542974 shutdown_connections 2024-01-27T02:33:01.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.854+0000 7f2bbca12b80 1 --2- 172.21.15.7:0/2686542974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5315835d0 0x55a5315aed80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.854+0000 7f2bbca12b80 1 --2- 172.21.15.7:0/2686542974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a531584010 0x55a5315843e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.854+0000 7f2bbca12b80 1 --2- 172.21.15.7:0/2686542974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5315cc360 0x55a5315ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.854+0000 7f2bbca12b80 1 -- 172.21.15.7:0/2686542974 >> 172.21.15.7:0/2686542974 conn(0x55a531583a70 msgr2=0x55a5315cc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:01.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.854+0000 7f2bbca12b80 1 -- 172.21.15.7:0/2686542974 shutdown_connections 2024-01-27T02:33:01.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.854+0000 7f2bbca12b80 1 -- 172.21.15.7:0/2686542974 wait complete. 2024-01-27T02:33:01.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.855+0000 7f2bbca12b80 1 -- start start 2024-01-27T02:33:01.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.855+0000 7f2bbca12b80 1 Processor -- start 2024-01-27T02:33:01.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.856+0000 7f2bbca12b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5315835d0 0x55a53159b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.856+0000 7f2bbca12b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a531584010 0x55a5315c3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.856+0000 7f2bbca12b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5315cc360 0x55a53157d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.856+0000 7f2bbca12b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5314d2720 con 0x55a5315835d0 2024-01-27T02:33:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.856+0000 7f2bbca12b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a53147e1b0 con 0x55a5315cc360 2024-01-27T02:33:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.856+0000 7f2bbca12b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a53147de50 con 0x55a531584010 2024-01-27T02:33:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.856+0000 7f2bbca12b80 10 client.0 _ll_register_callbacks cb 0x55a5312e07b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-27T02:33:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.856+0000 7f2bb14ab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5315835d0 0x55a53159b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.856+0000 7f2bb14ab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5315835d0 0x55a53159b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40998/0 (socket says 172.21.15.7:40998) 2024-01-27T02:33:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.856+0000 7f2bb14ab700 1 -- 172.21.15.7:0/4137000215 learned_addr learned my addr 172.21.15.7:0/4137000215 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.856+0000 7f2bb0caa700 1 --2- 172.21.15.7:0/4137000215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a531584010 0x55a5315c3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.856+0000 7f2bb14ab700 1 -- 172.21.15.7:0/4137000215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a531584010 msgr2=0x55a5315c3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.856+0000 7f2bb1cac700 1 --2- 172.21.15.7:0/4137000215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5315cc360 0x55a53157d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.856+0000 7f2bb14ab700 1 --2- 172.21.15.7:0/4137000215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a531584010 0x55a5315c3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.856+0000 7f2bb14ab700 1 -- 172.21.15.7:0/4137000215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5315cc360 msgr2=0x55a53157d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:01.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.856+0000 7f2bb14ab700 1 --2- 172.21.15.7:0/4137000215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5315cc360 0x55a53157d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.856+0000 7f2bb14ab700 1 -- 172.21.15.7:0/4137000215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bac0cdf90 con 0x55a5315835d0 2024-01-27T02:33:01.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.856+0000 7f2bb0caa700 1 --2- 172.21.15.7:0/4137000215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a531584010 0x55a5315c3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:33:01.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.857+0000 7f2bb14ab700 1 --2- 172.21.15.7:0/4137000215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5315835d0 0x55a53159b520 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8014c90 tx=0x7f2ba8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1af148c94232d46 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:01.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.857+0000 7f2b8b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-27T02:33:01.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.857+0000 7f2b8b7fe700 1 -- 172.21.15.7:0/4137000215 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8013bc0 con 0x55a5315835d0 2024-01-27T02:33:01.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.857+0000 7f2b8b7fe700 1 -- 172.21.15.7:0/4137000215 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2ba8013d20 con 0x55a5315835d0 2024-01-27T02:33:01.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.857+0000 7f2bbca12b80 1 -- 172.21.15.7:0/4137000215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a531577190 con 0x55a5315835d0 2024-01-27T02:33:01.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.857+0000 7f2b8b7fe700 1 -- 172.21.15.7:0/4137000215 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ba8023680 con 0x55a5315835d0 2024-01-27T02:33:01.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.857+0000 7f2bbca12b80 10 client.5043 fetch_fsmap learned FSMap version 17 2024-01-27T02:33:01.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.857+0000 7f2bbca12b80 1 -- 172.21.15.7:0/4137000215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55a5314a0290 con 0x55a5315835d0 2024-01-27T02:33:01.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.857+0000 7f2b8b7fe700 1 -- 172.21.15.7:0/4137000215 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a531577190 con 0x55a5315835d0 2024-01-27T02:33:01.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.857+0000 7f2b8b7fe700 1 -- 172.21.15.7:0/4137000215 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f2ba80237e0 con 0x55a5315835d0 2024-01-27T02:33:01.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.857+0000 7f2bbca12b80 10 client.5043 fetch_fsmap finished waiting for FSMap version 17 2024-01-27T02:33:01.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.857+0000 7f2bbca12b80 10 client.5043 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-27T02:33:01.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.857+0000 7f2bbca12b80 20 client.5043 populate_metadata read hostname 'smithi007' 2024-01-27T02:33:01.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.857+0000 7f2bbca12b80 1 --2- 172.21.15.7:0/4137000215 >> [v2:172.21.15.7:6833/1734594727,v1:172.21.15.7:6835/1734594727] conn(0x55a53157d780 0x55a5315d9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.857+0000 7f2bbca12b80 4 client.5043 mds_command: new command op to 4882 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-27T02:33:01.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.857+0000 7f2bbca12b80 1 -- 172.21.15.7:0/4137000215 --> [v2:172.21.15.7:6833/1734594727,v1:172.21.15.7:6835/1734594727] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a531474ee0 con 0x55a53157d780 2024-01-27T02:33:01.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.858+0000 7f2bb0caa700 1 --2- 172.21.15.7:0/4137000215 >> [v2:172.21.15.7:6833/1734594727,v1:172.21.15.7:6835/1734594727] conn(0x55a53157d780 0x55a5315d9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.858+0000 7f2b8b7fe700 10 client.5043 ms_handle_connect on v2:172.21.15.7:6833/1734594727 2024-01-27T02:33:01.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.858+0000 7f2bb0caa700 1 --2- 172.21.15.7:0/4137000215 >> [v2:172.21.15.7:6833/1734594727,v1:172.21.15.7:6835/1734594727] conn(0x55a53157d780 0x55a5315d9ae0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8a8c00ddb11cab7 server_cookie=2dfd049ac47f16a1 in_seq=0 out_seq=0 2024-01-27T02:33:01.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.858+0000 7f2b8b7fe700 1 -- 172.21.15.7:0/4137000215 <== mds.0 v2:172.21.15.7:6833/1734594727 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17146 (crc 0 0 0) 0x55a531474ee0 con 0x55a53157d780 2024-01-27T02:33:01.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.858+0000 7f2b8b7fe700 10 client.5043 handle_command_reply: tid=0 2024-01-27T02:33:01.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.858+0000 7f2bbca12b80 1 client.5043 shutdown 2024-01-27T02:33:01.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.859+0000 7f2bbca12b80 1 -- 172.21.15.7:0/4137000215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5315835d0 msgr2=0x55a53159b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:01.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.859+0000 7f2bbca12b80 1 --2- 172.21.15.7:0/4137000215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5315835d0 0x55a53159b520 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8014c90 tx=0x7f2ba8016000 comp rx=0 tx=0).stop 2024-01-27T02:33:01.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.859+0000 7f2bbca12b80 1 -- 172.21.15.7:0/4137000215 shutdown_connections 2024-01-27T02:33:01.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.859+0000 7f2bbca12b80 1 --2- 172.21.15.7:0/4137000215 >> [v2:172.21.15.7:6833/1734594727,v1:172.21.15.7:6835/1734594727] conn(0x55a53157d780 0x55a5315d9ae0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.859+0000 7f2bbca12b80 1 --2- 172.21.15.7:0/4137000215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5315cc360 0x55a53157d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.859+0000 7f2bbca12b80 1 --2- 172.21.15.7:0/4137000215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5315835d0 0x55a53159b520 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.859+0000 7f2b8b7fe700 0 client.5043 ms_handle_reset on v2:172.21.15.7:6833/1734594727 2024-01-27T02:33:01.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.859+0000 7f2bbca12b80 1 --2- 172.21.15.7:0/4137000215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a531584010 0x55a5315c3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.859+0000 7f2bbca12b80 1 -- 172.21.15.7:0/4137000215 >> 172.21.15.7:0/4137000215 conn(0x55a531583a70 msgr2=0x55a5315ab770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:01.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.859+0000 7f2bbca12b80 1 -- 172.21.15.7:0/4137000215 shutdown_connections 2024-01-27T02:33:01.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.859+0000 7f2bbca12b80 1 -- 172.21.15.7:0/4137000215 wait complete. 2024-01-27T02:33:01.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.859+0000 7f2bbca12b80 20 client.5043 trim_cache size 0 max 16384 2024-01-27T02:33:01.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.877+0000 7f2bbca12b80 1 Processor -- start 2024-01-27T02:33:01.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.877+0000 7f2bbca12b80 1 -- start start 2024-01-27T02:33:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.878+0000 7f2bbca12b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5315835d0 0x55a5315cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.878+0000 7f2bbca12b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a531584010 0x55a5315cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.878+0000 7f2bbca12b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5315cc360 0x55a5315d7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.878+0000 7f2bbca12b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5313cae40 con 0x55a5315cc360 2024-01-27T02:33:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.878+0000 7f2bbca12b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a53157ce50 con 0x55a531584010 2024-01-27T02:33:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.878+0000 7f2bbca12b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5315c76d0 con 0x55a5315835d0 2024-01-27T02:33:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.879+0000 7f2bb1cac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5315cc360 0x55a5315d7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.879+0000 7f2bb1cac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5315cc360 0x55a5315d7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41014/0 (socket says 172.21.15.7:41014) 2024-01-27T02:33:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.879+0000 7f2bb1cac700 1 -- 172.21.15.7:0/662998789 learned_addr learned my addr 172.21.15.7:0/662998789 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.879+0000 7f2bb14ab700 1 --2- 172.21.15.7:0/662998789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5315835d0 0x55a5315cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.879+0000 7f2bb0caa700 1 --2- 172.21.15.7:0/662998789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a531584010 0x55a5315cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.879+0000 7f2bb1cac700 1 -- 172.21.15.7:0/662998789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5315835d0 msgr2=0x55a5315cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.879+0000 7f2bb1cac700 1 --2- 172.21.15.7:0/662998789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5315835d0 0x55a5315cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.879+0000 7f2bb1cac700 1 -- 172.21.15.7:0/662998789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a531584010 msgr2=0x55a5315cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:01.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.879+0000 7f2bb1cac700 1 --2- 172.21.15.7:0/662998789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a531584010 0x55a5315cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.879+0000 7f2bb1cac700 1 -- 172.21.15.7:0/662998789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a531369c80 con 0x55a5315cc360 2024-01-27T02:33:01.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.879+0000 7f2bb1cac700 1 --2- 172.21.15.7:0/662998789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5315cc360 0x55a5315d7ee0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40097c0 tx=0x7f2ba4021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986b7d1f901bb17e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:01.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.879+0000 7f2b96ffd700 1 -- 172.21.15.7:0/662998789 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba402d070 con 0x55a5315cc360 2024-01-27T02:33:01.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.879+0000 7f2b96ffd700 1 -- 172.21.15.7:0/662998789 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba4009880 con 0x55a5315cc360 2024-01-27T02:33:01.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.879+0000 7f2b96ffd700 1 -- 172.21.15.7:0/662998789 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba40127b0 con 0x55a5315cc360 2024-01-27T02:33:01.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.880+0000 7f2bbca12b80 1 -- 172.21.15.7:0/662998789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5315cc360 msgr2=0x55a5315d7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:01.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.880+0000 7f2bbca12b80 1 --2- 172.21.15.7:0/662998789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5315cc360 0x55a5315d7ee0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40097c0 tx=0x7f2ba4021040 comp rx=0 tx=0).stop 2024-01-27T02:33:01.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.880+0000 7f2bbca12b80 1 -- 172.21.15.7:0/662998789 shutdown_connections 2024-01-27T02:33:01.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.880+0000 7f2bbca12b80 1 --2- 172.21.15.7:0/662998789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5315cc360 0x55a5315d7ee0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.880+0000 7f2bbca12b80 1 --2- 172.21.15.7:0/662998789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5315835d0 0x55a5315cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.880+0000 7f2bbca12b80 1 --2- 172.21.15.7:0/662998789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a531584010 0x55a5315cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.880+0000 7f2bbca12b80 1 -- 172.21.15.7:0/662998789 >> 172.21.15.7:0/662998789 conn(0x55a531583a70 msgr2=0x55a5315c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:01.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.880+0000 7f2bbca12b80 1 -- 172.21.15.7:0/662998789 shutdown_connections 2024-01-27T02:33:01.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.880+0000 7f2bbca12b80 1 -- 172.21.15.7:0/662998789 wait complete. 2024-01-27T02:33:01.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.881+0000 7f2bbca12b80 1 -- start start 2024-01-27T02:33:01.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.881+0000 7f2bbca12b80 1 Processor -- start 2024-01-27T02:33:01.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.882+0000 7f2bbca12b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5315835d0 0x55a5315d96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.882+0000 7f2bbca12b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a531584010 0x55a5315cba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.882+0000 7f2bbca12b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5315cc360 0x55a53162d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.882+0000 7f2bbca12b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a53162d190 con 0x55a531584010 2024-01-27T02:33:01.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.882+0000 7f2bbca12b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a531594120 con 0x55a5315835d0 2024-01-27T02:33:01.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.882+0000 7f2bbca12b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a531594400 con 0x55a5315cc360 2024-01-27T02:33:01.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.882+0000 7f2bbca12b80 10 client.0 _ll_register_callbacks cb 0x55a5312e07b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-27T02:33:01.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.882+0000 7f2bb0caa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a531584010 0x55a5315cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.882+0000 7f2bb0caa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a531584010 0x55a5315cba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41018/0 (socket says 172.21.15.7:41018) 2024-01-27T02:33:01.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.882+0000 7f2bb0caa700 1 -- 172.21.15.7:0/618528117 learned_addr learned my addr 172.21.15.7:0/618528117 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:01.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.882+0000 7f2bb14ab700 1 --2- 172.21.15.7:0/618528117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5315835d0 0x55a5315d96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.882+0000 7f2bb1cac700 1 --2- 172.21.15.7:0/618528117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5315cc360 0x55a53162d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.882+0000 7f2bb0caa700 1 -- 172.21.15.7:0/618528117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5315cc360 msgr2=0x55a53162d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:01.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.882+0000 7f2bb0caa700 1 --2- 172.21.15.7:0/618528117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5315cc360 0x55a53162d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.882+0000 7f2bb0caa700 1 -- 172.21.15.7:0/618528117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5315835d0 msgr2=0x55a5315d96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:01.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.882+0000 7f2bb0caa700 1 --2- 172.21.15.7:0/618528117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5315835d0 0x55a5315d96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.882+0000 7f2bb0caa700 1 -- 172.21.15.7:0/618528117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a531311b10 con 0x55a531584010 2024-01-27T02:33:01.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.883+0000 7f2bb0caa700 1 --2- 172.21.15.7:0/618528117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a531584010 0x55a5315cba50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00cdb0 tx=0x7f2b9c082020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f4277548c6edd3 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:01.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.883+0000 7f2b8b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-27T02:33:01.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.883+0000 7f2b8b7fe700 1 -- 172.21.15.7:0/618528117 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c057d20 con 0x55a531584010 2024-01-27T02:33:01.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.883+0000 7f2b8b7fe700 1 -- 172.21.15.7:0/618528117 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2b9c0076d0 con 0x55a531584010 2024-01-27T02:33:01.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.883+0000 7f2bbca12b80 1 -- 172.21.15.7:0/618528117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ba4012640 con 0x55a531584010 2024-01-27T02:33:01.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.883+0000 7f2b8b7fe700 1 -- 172.21.15.7:0/618528117 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b9c05b5e0 con 0x55a531584010 2024-01-27T02:33:01.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.883+0000 7f2bbca12b80 10 client.5049 fetch_fsmap learned FSMap version 17 2024-01-27T02:33:01.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.883+0000 7f2bbca12b80 1 -- 172.21.15.7:0/618528117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55a53130ffe0 con 0x55a531584010 2024-01-27T02:33:01.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.883+0000 7f2b8b7fe700 1 -- 172.21.15.7:0/618528117 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba4012640 con 0x55a531584010 2024-01-27T02:33:01.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.883+0000 7f2b8b7fe700 1 -- 172.21.15.7:0/618528117 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f2b9c070430 con 0x55a531584010 2024-01-27T02:33:01.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.883+0000 7f2bbca12b80 10 client.5049 fetch_fsmap finished waiting for FSMap version 17 2024-01-27T02:33:01.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.883+0000 7f2bbca12b80 10 client.5049 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-27T02:33:01.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.883+0000 7f2bbca12b80 20 client.5049 populate_metadata read hostname 'smithi007' 2024-01-27T02:33:01.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.883+0000 7f2bbca12b80 1 --2- 172.21.15.7:0/618528117 >> [v2:172.21.15.7:6833/1734594727,v1:172.21.15.7:6835/1734594727] conn(0x55a531581f80 0x55a531582350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:01.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.883+0000 7f2bbca12b80 4 client.5049 mds_command: new command op to 4882 tid=0[{"prefix": "status"}] 2024-01-27T02:33:01.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.883+0000 7f2bbca12b80 1 -- 172.21.15.7:0/618528117 --> [v2:172.21.15.7:6833/1734594727,v1:172.21.15.7:6835/1734594727] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a531474c00 con 0x55a531581f80 2024-01-27T02:33:01.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.883+0000 7f2bb14ab700 1 --2- 172.21.15.7:0/618528117 >> [v2:172.21.15.7:6833/1734594727,v1:172.21.15.7:6835/1734594727] conn(0x55a531581f80 0x55a531582350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:01.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.884+0000 7f2b8b7fe700 10 client.5049 ms_handle_connect on v2:172.21.15.7:6833/1734594727 2024-01-27T02:33:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.884+0000 7f2bb14ab700 1 --2- 172.21.15.7:0/618528117 >> [v2:172.21.15.7:6833/1734594727,v1:172.21.15.7:6835/1734594727] conn(0x55a531581f80 0x55a531582350 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e133a040841807e server_cookie=243d7f9e7215a7ca in_seq=0 out_seq=0 2024-01-27T02:33:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.884+0000 7f2b8b7fe700 1 -- 172.21.15.7:0/618528117 <== mds.0 v2:172.21.15.7:6833/1734594727 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55a531474c00 con 0x55a531581f80 2024-01-27T02:33:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.884+0000 7f2b8b7fe700 10 client.5049 handle_command_reply: tid=0 2024-01-27T02:33:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.884+0000 7f2bbca12b80 1 client.5049 shutdown 2024-01-27T02:33:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.884+0000 7f2bbca12b80 1 -- 172.21.15.7:0/618528117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a531584010 msgr2=0x55a5315cba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.884+0000 7f2bbca12b80 1 --2- 172.21.15.7:0/618528117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a531584010 0x55a5315cba50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00cdb0 tx=0x7f2b9c082020 comp rx=0 tx=0).stop 2024-01-27T02:33:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.884+0000 7f2bbca12b80 1 -- 172.21.15.7:0/618528117 shutdown_connections 2024-01-27T02:33:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.884+0000 7f2bbca12b80 1 --2- 172.21.15.7:0/618528117 >> [v2:172.21.15.7:6833/1734594727,v1:172.21.15.7:6835/1734594727] conn(0x55a531581f80 0x55a531582350 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.884+0000 7f2bbca12b80 1 --2- 172.21.15.7:0/618528117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5315cc360 0x55a53162d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.884+0000 7f2bbca12b80 1 --2- 172.21.15.7:0/618528117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5315835d0 0x55a5315d96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.884+0000 7f2b8b7fe700 0 client.5049 ms_handle_reset on v2:172.21.15.7:6833/1734594727 2024-01-27T02:33:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.884+0000 7f2bbca12b80 1 --2- 172.21.15.7:0/618528117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a531584010 0x55a5315cba50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.884+0000 7f2bbca12b80 1 -- 172.21.15.7:0/618528117 >> 172.21.15.7:0/618528117 conn(0x55a531583a70 msgr2=0x55a5315c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.884+0000 7f2bbca12b80 1 -- 172.21.15.7:0/618528117 shutdown_connections 2024-01-27T02:33:01.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.885+0000 7f2bbca12b80 1 -- 172.21.15.7:0/618528117 wait complete. 2024-01-27T02:33:01.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.885+0000 7f2bbca12b80 20 client.5049 trim_cache size 0 max 16384 2024-01-27T02:33:01.891 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:33:01.891 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:33:01.891 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-27T02:33:01.891 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4882, 2024-01-27T02:33:01.891 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-27T02:33:01.891 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-27T02:33:01.891 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-27T02:33:01.891 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.0795181229999997, 2024-01-27T02:33:01.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 17, 2024-01-27T02:33:01.892 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 32, 2024-01-27T02:33:01.892 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 32, 2024-01-27T02:33:01.892 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 14.495910618 2024-01-27T02:33:01.892 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:33:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.885+0000 7f2bb370f700 1 -- 172.21.15.7:0/1996845363 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2b90066f30 msgr2=0x7f2b900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.885+0000 7f2bb370f700 1 --2- 172.21.15.7:0/1996845363 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2b90066f30 0x7f2b900693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009ea0 tx=0x7f2ba80091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.885+0000 7f2bb370f700 1 -- 172.21.15.7:0/1996845363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bac0cb5b0 msgr2=0x7f2bac154b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.885+0000 7f2bb370f700 1 --2- 172.21.15.7:0/1996845363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bac0cb5b0 0x7f2bac154b90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00c7c0 tx=0x7f2b9c007e00 comp rx=0 tx=0).stop 2024-01-27T02:33:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.885+0000 7f2bb370f700 1 -- 172.21.15.7:0/1996845363 shutdown_connections 2024-01-27T02:33:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.885+0000 7f2bb370f700 1 --2- 172.21.15.7:0/1996845363 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2b90066f30 0x7f2b900693e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.885+0000 7f2bb370f700 1 --2- 172.21.15.7:0/1996845363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bac15a100 0x7f2bac14faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.885+0000 7f2bb370f700 1 --2- 172.21.15.7:0/1996845363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bac0c4cc0 0x7f2bac14dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.885+0000 7f2bb370f700 1 --2- 172.21.15.7:0/1996845363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bac0cb5b0 0x7f2bac154b90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.885+0000 7f2bb370f700 1 -- 172.21.15.7:0/1996845363 >> 172.21.15.7:0/1996845363 conn(0x7f2bac098020 msgr2=0x7f2bac15fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.885+0000 7f2bb370f700 1 -- 172.21.15.7:0/1996845363 shutdown_connections 2024-01-27T02:33:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:01.885+0000 7f2bb370f700 1 -- 172.21.15.7:0/1996845363 wait complete. 2024-01-27T02:33:01.899 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:33:02.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.030+0000 7f06a99aa700 1 Processor -- start 2024-01-27T02:33:02.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.030+0000 7f06a99aa700 1 -- start start 2024-01-27T02:33:02.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.031+0000 7f06a99aa700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f06a415d3a0 0x7f06a40cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:02.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.031+0000 7f06a99aa700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f06a406ea60 0x7f06a406eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:02.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.031+0000 7f06a99aa700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a406e5d0 con 0x7f06a415d770 2024-01-27T02:33:02.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.031+0000 7f06a99aa700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a40cbe10 con 0x7f06a415d3a0 2024-01-27T02:33:02.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.031+0000 7f06a99aa700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a40ce860 con 0x7f06a406ea60 2024-01-27T02:33:02.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.031+0000 7f06a2ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f06a415d3a0 0x7f06a40cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:02.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.031+0000 7f06a27fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f06a415d770 0x7f06a40c4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34080/0 (socket says 172.21.15.7:34080) 2024-01-27T02:33:02.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.031+0000 7f06a27fc700 1 -- 172.21.15.7:0/1944678886 learned_addr learned my addr 172.21.15.7:0/1944678886 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:02.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.031+0000 7f06a37fe700 1 --2- 172.21.15.7:0/1944678886 >> v2:172.21.15.188:3300/0 conn(0x7f06a406ea60 0x7f06a406eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:02.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.031+0000 7f06a2ffd700 1 -- 172.21.15.7:0/1944678886 >> v2:172.21.15.188:3300/0 conn(0x7f06a406ea60 msgr2=0x7f06a406eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.031+0000 7f06a2ffd700 1 --2- 172.21.15.7:0/1944678886 >> v2:172.21.15.188:3300/0 conn(0x7f06a406ea60 0x7f06a406eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.031+0000 7f06a2ffd700 1 -- 172.21.15.7:0/1944678886 >> v1:172.21.15.7:6789/0 conn(0x7f06a415d770 legacy=0x7f06a40c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.031+0000 7f06a2ffd700 1 -- 172.21.15.7:0/1944678886 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a40cd8f0 con 0x7f06a415d3a0 2024-01-27T02:33:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.031+0000 7f06a1ffb700 1 -- 172.21.15.7:0/1944678886 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4160268627 0 0) 0x7f06a406e5d0 con 0x7f06a415d770 2024-01-27T02:33:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.032+0000 7f06a2ffd700 1 --2- 172.21.15.7:0/1944678886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06a415d3a0 0x7f06a40cd310 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f06a40ad330 tx=0x7f06940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96144c9f45162a18 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.032+0000 7f06a1ffb700 1 -- 172.21.15.7:0/1944678886 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694011630 con 0x7f06a415d3a0 2024-01-27T02:33:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.032+0000 7f06a1ffb700 1 -- 172.21.15.7:0/1944678886 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0694011790 con 0x7f06a415d3a0 2024-01-27T02:33:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.032+0000 7f06a1ffb700 1 -- 172.21.15.7:0/1944678886 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694011b30 con 0x7f06a415d3a0 2024-01-27T02:33:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.032+0000 7f06a99aa700 1 -- 172.21.15.7:0/1944678886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06a415d3a0 msgr2=0x7f06a40cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.032+0000 7f06a99aa700 1 --2- 172.21.15.7:0/1944678886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06a415d3a0 0x7f06a40cd310 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f06a40ad330 tx=0x7f06940091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.032+0000 7f06a99aa700 1 -- 172.21.15.7:0/1944678886 shutdown_connections 2024-01-27T02:33:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.032+0000 7f06a99aa700 1 --2- 172.21.15.7:0/1944678886 >> v2:172.21.15.188:3300/0 conn(0x7f06a406ea60 0x7f06a406eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.032+0000 7f06a99aa700 1 --2- 172.21.15.7:0/1944678886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06a415d3a0 0x7f06a40cd310 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.032+0000 7f06a99aa700 1 -- 172.21.15.7:0/1944678886 >> 172.21.15.7:0/1944678886 conn(0x7f06a4068480 msgr2=0x7f06a40cdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.033+0000 7f06a99aa700 1 -- 172.21.15.7:0/1944678886 shutdown_connections 2024-01-27T02:33:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.033+0000 7f06a99aa700 1 -- 172.21.15.7:0/1944678886 wait complete. 2024-01-27T02:33:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.034+0000 7f06a99aa700 1 Processor -- start 2024-01-27T02:33:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.034+0000 7f06a99aa700 1 -- start start 2024-01-27T02:33:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.035+0000 7f06a99aa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06a406ea60 0x7f06a41583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.035+0000 7f06a99aa700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a415d3a0 0x7f06a40aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.035+0000 7f06a99aa700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a415d770 0x7f06a4159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.035+0000 7f06a99aa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a40b0cf0 con 0x7f06a406ea60 2024-01-27T02:33:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.035+0000 7f06a99aa700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a415e1b0 con 0x7f06a415d770 2024-01-27T02:33:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.035+0000 7f06a99aa700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a415e490 con 0x7f06a415d3a0 2024-01-27T02:33:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.035+0000 7f06a2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06a406ea60 0x7f06a41583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.035+0000 7f06a2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06a406ea60 0x7f06a41583d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41036/0 (socket says 172.21.15.7:41036) 2024-01-27T02:33:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.035+0000 7f06a2ffd700 1 -- 172.21.15.7:0/1906597097 learned_addr learned my addr 172.21.15.7:0/1906597097 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.035+0000 7f06a27fc700 1 --2- 172.21.15.7:0/1906597097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a415d3a0 0x7f06a40aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.035+0000 7f06a37fe700 1 --2- 172.21.15.7:0/1906597097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a415d770 0x7f06a4159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.035+0000 7f06a2ffd700 1 -- 172.21.15.7:0/1906597097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a415d3a0 msgr2=0x7f06a40aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.035+0000 7f06a2ffd700 1 --2- 172.21.15.7:0/1906597097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a415d3a0 0x7f06a40aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.035+0000 7f06a2ffd700 1 -- 172.21.15.7:0/1906597097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a415d770 msgr2=0x7f06a4159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.035+0000 7f06a2ffd700 1 --2- 172.21.15.7:0/1906597097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a415d770 0x7f06a4159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.035+0000 7f06a2ffd700 1 -- 172.21.15.7:0/1906597097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0694009420 con 0x7f06a406ea60 2024-01-27T02:33:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.035+0000 7f06a37fe700 1 --2- 172.21.15.7:0/1906597097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a415d770 0x7f06a4159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.035+0000 7f06a27fc700 1 --2- 172.21.15.7:0/1906597097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a415d3a0 0x7f06a40aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.035+0000 7f06a2ffd700 1 --2- 172.21.15.7:0/1906597097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06a406ea60 0x7f06a41583d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f06a4069ca0 tx=0x7f0694017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.036+0000 7f069affd700 1 -- 172.21.15.7:0/1906597097 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694011b10 con 0x7f06a406ea60 2024-01-27T02:33:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.036+0000 7f069affd700 1 -- 172.21.15.7:0/1906597097 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06940253f0 con 0x7f06a406ea60 2024-01-27T02:33:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.036+0000 7f06a99aa700 1 -- 172.21.15.7:0/1906597097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a415e770 con 0x7f06a406ea60 2024-01-27T02:33:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.036+0000 7f069affd700 1 -- 172.21.15.7:0/1906597097 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694025790 con 0x7f06a406ea60 2024-01-27T02:33:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.036+0000 7f06a99aa700 1 -- 172.21.15.7:0/1906597097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a40af430 con 0x7f06a406ea60 2024-01-27T02:33:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.037+0000 7f06a99aa700 1 -- 172.21.15.7:0/1906597097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a40b0ac0 con 0x7f06a406ea60 2024-01-27T02:33:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.037+0000 7f069affd700 1 -- 172.21.15.7:0/1906597097 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0694025b80 con 0x7f06a406ea60 2024-01-27T02:33:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.037+0000 7f069affd700 1 --2- 172.21.15.7:0/1906597097 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0684066f30 0x7f06840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.038+0000 7f069affd700 1 -- 172.21.15.7:0/1906597097 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0694085ec0 con 0x7f06a406ea60 2024-01-27T02:33:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.038+0000 7f06a27fc700 1 --2- 172.21.15.7:0/1906597097 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0684066f30 0x7f06840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.038+0000 7f06a27fc700 1 --2- 172.21.15.7:0/1906597097 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0684066f30 0x7f06840693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f06a415a8c0 tx=0x7f068c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.042+0000 7f069affd700 1 -- 172.21.15.7:0/1906597097 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f06a40b0ac0 con 0x7f06a406ea60 2024-01-27T02:33:02.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.291+0000 7f06a99aa700 1 -- 172.21.15.7:0/1906597097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f06a40b0ac0 con 0x7f06a406ea60 2024-01-27T02:33:02.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.291+0000 7f069affd700 1 -- 172.21.15.7:0/1906597097 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f06a40b0ac0 con 0x7f06a406ea60 2024-01-27T02:33:02.292 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:33:02.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.294+0000 7f06a99aa700 1 -- 172.21.15.7:0/1906597097 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0684066f30 msgr2=0x7f06840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:02.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.294+0000 7f06a99aa700 1 --2- 172.21.15.7:0/1906597097 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0684066f30 0x7f06840693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f06a415a8c0 tx=0x7f068c009040 comp rx=0 tx=0).stop 2024-01-27T02:33:02.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.294+0000 7f06a99aa700 1 -- 172.21.15.7:0/1906597097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06a406ea60 msgr2=0x7f06a41583d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:02.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.294+0000 7f06a99aa700 1 --2- 172.21.15.7:0/1906597097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06a406ea60 0x7f06a41583d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f06a4069ca0 tx=0x7f0694017000 comp rx=0 tx=0).stop 2024-01-27T02:33:02.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.294+0000 7f06a99aa700 1 -- 172.21.15.7:0/1906597097 shutdown_connections 2024-01-27T02:33:02.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.294+0000 7f06a99aa700 1 --2- 172.21.15.7:0/1906597097 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0684066f30 0x7f06840693e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:02.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.294+0000 7f06a99aa700 1 --2- 172.21.15.7:0/1906597097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a415d770 0x7f06a4159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:02.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.294+0000 7f06a99aa700 1 --2- 172.21.15.7:0/1906597097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06a406ea60 0x7f06a41583d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:02.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.294+0000 7f06a99aa700 1 --2- 172.21.15.7:0/1906597097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a415d3a0 0x7f06a40aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:02.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.294+0000 7f06a99aa700 1 -- 172.21.15.7:0/1906597097 >> 172.21.15.7:0/1906597097 conn(0x7f06a4068480 msgr2=0x7f06a415b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:02.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.294+0000 7f06a99aa700 1 -- 172.21.15.7:0/1906597097 shutdown_connections 2024-01-27T02:33:02.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:02.294+0000 7f06a99aa700 1 -- 172.21.15.7:0/1906597097 wait complete. 2024-01-27T02:33:02.308 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-27T02:33:02.308 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:33:02.308 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-27T02:33:02.308 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-27T02:33:02.308 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-27T02:33:02.308 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-27T02:33:02.308 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-27T02:33:02.308 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:33:02.308 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:33:02.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:02.309 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-27T02:33:02.328 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:33:02.328 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:33:02.328 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:33:02.329 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:02.329 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-27T02:33:02.329 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:02.329 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:33:02.329 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-27T02:33:02.329 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-27T02:33:02.329 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:33:02.329 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-27T02:33:02.329 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-27T02:33:02.329 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:33:02.329 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-27T02:33:02.329 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-27T02:33:02.330 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-27T02:33:02.330 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42329sec preferred_lft 42329sec 2024-01-27T02:33:02.330 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-27T02:33:02.330 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:02.330 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:33:02.330 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-27T02:33:02.330 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-27T02:33:02.330 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:33:02.330 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 4a:57:d9:3c:c5:a0 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:33:02.330 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-27T02:33:02.330 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:02.330 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::2477:9eff:fe39:433c/64 scope link 2024-01-27T02:33:02.330 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:02.331 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:33:02.331 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 4a:57:d9:3c:c5:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-27T02:33:02.331 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::4857:d9ff:fe3c:c5a0/64 scope link 2024-01-27T02:33:02.331 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:02.331 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:33:02.331 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ca:6e:b8:df:53:bf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-27T02:33:02.331 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c86e:b8ff:fedf:53bf/64 scope link 2024-01-27T02:33:02.331 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:02.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:02.331 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:33:02.400 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-27T02:33:02.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:02.402 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-27T02:33:02.466 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-27T02:33:02.466 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:33:02.466 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-27T02:33:02.466 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-27T02:33:02.466 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:33:02.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:02.466 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-27T02:33:02.466 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-27T02:33:02.467 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-27T02:33:02.467 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-27T02:33:02.467 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-27T02:33:02.467 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-27T02:33:02.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:02.591 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-27T02:33:02.611 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-27T02:33:02.611 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-27T02:33:02.612 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:33:02.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:02.612 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-27T02:33:02.686 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:33:02.686 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-27T02:33:02.748 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:33:02.749 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:33:02.749 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-27T02:33:02.810 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:33:02.810 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:33:02.810 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:33:02.905 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:33:02.905 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:33:02.905 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:33:02.915 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:33:02.915 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:33:02.915 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:33:02.915 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:33:02.915 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:33:02.933 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:33:02.933 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:33:02.934 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:33:02.934 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:33:02.934 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:33:02.934 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:33:02.934 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:33:02.934 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:33:02.936 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:33:02.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:02.937 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-27T02:33:02.969 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:33:03.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.148+0000 7f04c5106700 1 Processor -- start 2024-01-27T02:33:03.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.149+0000 7f04c5106700 1 -- start start 2024-01-27T02:33:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.149+0000 7f04c5106700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f04c0063c90 0x7f04c0158590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.149+0000 7f04c5106700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c0159570 con 0x7f04c006adc0 2024-01-27T02:33:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.149+0000 7f04c5106700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c0159290 con 0x7f04c006a9f0 2024-01-27T02:33:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.149+0000 7f04c5106700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c015bd00 con 0x7f04c0063c90 2024-01-27T02:33:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.150+0000 7f04be59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f04c0063c90 0x7f04c0158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.150+0000 7f04bf59e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f04c006adc0 0x7f04c0158ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34094/0 (socket says 172.21.15.7:34094) 2024-01-27T02:33:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.150+0000 7f04bf59e700 1 -- 172.21.15.7:0/1447670915 learned_addr learned my addr 172.21.15.7:0/1447670915 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.150+0000 7f04be59c700 1 -- 172.21.15.7:0/1447670915 >> v1:172.21.15.188:6789/0 conn(0x7f04c006a9f0 legacy=0x7f04c0063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.150+0000 7f04be59c700 1 -- 172.21.15.7:0/1447670915 >> v1:172.21.15.7:6789/0 conn(0x7f04c006adc0 legacy=0x7f04c0158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.150+0000 7f04be59c700 1 -- 172.21.15.7:0/1447670915 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c015bfe0 con 0x7f04c0063c90 2024-01-27T02:33:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.150+0000 7f04be59c700 1 --2- 172.21.15.7:0/1447670915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c0063c90 0x7f04c0158590 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f04b00098e0 tx=0x7f04b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742e11cba86e06bd server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.150+0000 7f04bdd9b700 1 -- 172.21.15.7:0/1447670915 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0009e80 con 0x7f04c0063c90 2024-01-27T02:33:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.150+0000 7f04bdd9b700 1 -- 172.21.15.7:0/1447670915 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b000b860 con 0x7f04c0063c90 2024-01-27T02:33:03.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.150+0000 7f04bdd9b700 1 -- 172.21.15.7:0/1447670915 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b000bbe0 con 0x7f04c0063c90 2024-01-27T02:33:03.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.151+0000 7f04c5106700 1 -- 172.21.15.7:0/1447670915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c0063c90 msgr2=0x7f04c0158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:03.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.151+0000 7f04c5106700 1 --2- 172.21.15.7:0/1447670915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c0063c90 0x7f04c0158590 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f04b00098e0 tx=0x7f04b00091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:03.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.151+0000 7f04c5106700 1 -- 172.21.15.7:0/1447670915 shutdown_connections 2024-01-27T02:33:03.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.151+0000 7f04c5106700 1 --2- 172.21.15.7:0/1447670915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c0063c90 0x7f04c0158590 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:03.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.151+0000 7f04c5106700 1 -- 172.21.15.7:0/1447670915 >> 172.21.15.7:0/1447670915 conn(0x7f04c00cdc30 msgr2=0x7f04c00ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:03.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.151+0000 7f04c5106700 1 -- 172.21.15.7:0/1447670915 shutdown_connections 2024-01-27T02:33:03.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.151+0000 7f04c5106700 1 -- 172.21.15.7:0/1447670915 wait complete. 2024-01-27T02:33:03.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.152+0000 7f04c5106700 1 Processor -- start 2024-01-27T02:33:03.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.152+0000 7f04c5106700 1 -- start start 2024-01-27T02:33:03.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.153+0000 7f04c5106700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04c0063c90 0x7f04c0169590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:03.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.153+0000 7f04c5106700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c006a9f0 0x7f04c0168be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:03.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.153+0000 7f04c5106700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04c006adc0 0x7f04c013ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:03.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.153+0000 7f04c5106700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c0169ad0 con 0x7f04c006a9f0 2024-01-27T02:33:03.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.153+0000 7f04c5106700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c00c5950 con 0x7f04c006adc0 2024-01-27T02:33:03.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.153+0000 7f04c5106700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c00c4d80 con 0x7f04c0063c90 2024-01-27T02:33:03.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.153+0000 7f04be59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c006a9f0 0x7f04c0168be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:03.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.153+0000 7f04be59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c006a9f0 0x7f04c0168be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41054/0 (socket says 172.21.15.7:41054) 2024-01-27T02:33:03.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.153+0000 7f04be59c700 1 -- 172.21.15.7:0/1920145867 learned_addr learned my addr 172.21.15.7:0/1920145867 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:03.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.153+0000 7f04bf59e700 1 --2- 172.21.15.7:0/1920145867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04c006adc0 0x7f04c013ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:03.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.154+0000 7f04bed9d700 1 --2- 172.21.15.7:0/1920145867 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04c0063c90 0x7f04c0169590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:03.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.154+0000 7f04be59c700 1 -- 172.21.15.7:0/1920145867 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04c0063c90 msgr2=0x7f04c0169590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:03.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.154+0000 7f04be59c700 1 --2- 172.21.15.7:0/1920145867 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04c0063c90 0x7f04c0169590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:03.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.154+0000 7f04be59c700 1 -- 172.21.15.7:0/1920145867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04c006adc0 msgr2=0x7f04c013ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:03.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.154+0000 7f04be59c700 1 --2- 172.21.15.7:0/1920145867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04c006adc0 0x7f04c013ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:03.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.154+0000 7f04be59c700 1 -- 172.21.15.7:0/1920145867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b0009420 con 0x7f04c006a9f0 2024-01-27T02:33:03.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.154+0000 7f04bed9d700 1 --2- 172.21.15.7:0/1920145867 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04c0063c90 0x7f04c0169590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:03.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.154+0000 7f04be59c700 1 --2- 172.21.15.7:0/1920145867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c006a9f0 0x7f04c0168be0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f04b0013000 tx=0x7f04b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:03.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.154+0000 7f04b6ffd700 1 -- 172.21.15.7:0/1920145867 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b000baa0 con 0x7f04c006a9f0 2024-01-27T02:33:03.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.154+0000 7f04b6ffd700 1 -- 172.21.15.7:0/1920145867 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b0025420 con 0x7f04c006a9f0 2024-01-27T02:33:03.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.154+0000 7f04b6ffd700 1 -- 172.21.15.7:0/1920145867 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b00257c0 con 0x7f04c006a9f0 2024-01-27T02:33:03.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.154+0000 7f04c5106700 1 -- 172.21.15.7:0/1920145867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c00c5020 con 0x7f04c006a9f0 2024-01-27T02:33:03.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.154+0000 7f04c5106700 1 -- 172.21.15.7:0/1920145867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c00c55e0 con 0x7f04c006a9f0 2024-01-27T02:33:03.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.155+0000 7f04c5106700 1 -- 172.21.15.7:0/1920145867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c00c5ac0 con 0x7f04c006a9f0 2024-01-27T02:33:03.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.156+0000 7f04b6ffd700 1 -- 172.21.15.7:0/1920145867 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f04b0023070 con 0x7f04c006a9f0 2024-01-27T02:33:03.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.156+0000 7f04b6ffd700 1 --2- 172.21.15.7:0/1920145867 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f04a0066f30 0x7f04a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:03.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.156+0000 7f04b6ffd700 1 -- 172.21.15.7:0/1920145867 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f04b0085be0 con 0x7f04c006a9f0 2024-01-27T02:33:03.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.156+0000 7f04bed9d700 1 --2- 172.21.15.7:0/1920145867 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f04a0066f30 0x7f04a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:03.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.157+0000 7f04bed9d700 1 --2- 172.21.15.7:0/1920145867 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f04a0066f30 0x7f04a00693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f04a8007fd0 tx=0x7f04a8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:03.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.160+0000 7f04b6ffd700 1 -- 172.21.15.7:0/1920145867 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04b0052e50 con 0x7f04c006a9f0 2024-01-27T02:33:03.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.412+0000 7f04c5106700 1 -- 172.21.15.7:0/1920145867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f04c004c710 con 0x7f04c006a9f0 2024-01-27T02:33:03.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.412+0000 7f04b6ffd700 1 -- 172.21.15.7:0/1920145867 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f04b0015070 con 0x7f04c006a9f0 2024-01-27T02:33:03.413 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:33:03.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.415+0000 7f04c5106700 1 -- 172.21.15.7:0/1920145867 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f04a0066f30 msgr2=0x7f04a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:03.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.415+0000 7f04c5106700 1 --2- 172.21.15.7:0/1920145867 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f04a0066f30 0x7f04a00693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f04a8007fd0 tx=0x7f04a8009040 comp rx=0 tx=0).stop 2024-01-27T02:33:03.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.415+0000 7f04c5106700 1 -- 172.21.15.7:0/1920145867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c006a9f0 msgr2=0x7f04c0168be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:03.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.415+0000 7f04c5106700 1 --2- 172.21.15.7:0/1920145867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c006a9f0 0x7f04c0168be0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f04b0013000 tx=0x7f04b0017000 comp rx=0 tx=0).stop 2024-01-27T02:33:03.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.415+0000 7f04c5106700 1 -- 172.21.15.7:0/1920145867 shutdown_connections 2024-01-27T02:33:03.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.415+0000 7f04c5106700 1 --2- 172.21.15.7:0/1920145867 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f04a0066f30 0x7f04a00693e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:03.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.415+0000 7f04c5106700 1 --2- 172.21.15.7:0/1920145867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04c006adc0 0x7f04c013ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:03.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.415+0000 7f04c5106700 1 --2- 172.21.15.7:0/1920145867 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04c0063c90 0x7f04c0169590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:03.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.415+0000 7f04c5106700 1 --2- 172.21.15.7:0/1920145867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c006a9f0 0x7f04c0168be0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:03.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.415+0000 7f04c5106700 1 -- 172.21.15.7:0/1920145867 >> 172.21.15.7:0/1920145867 conn(0x7f04c00cdc30 msgr2=0x7f04c00c8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:03.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.415+0000 7f04c5106700 1 -- 172.21.15.7:0/1920145867 shutdown_connections 2024-01-27T02:33:03.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:03.415+0000 7f04c5106700 1 -- 172.21.15.7:0/1920145867 wait complete. 2024-01-27T02:33:03.430 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-27T02:33:03.430 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:33:03.430 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-01-27T02:33:03.430 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-27T02:33:03.430 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-27T02:33:03.430 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-27T02:33:03.430 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-27T02:33:03.430 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:33:03.430 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:33:03.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:03.431 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-01-27T02:33:03.451 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:33:03.452 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:33:03.452 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:33:03.452 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:03.452 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-01-27T02:33:03.452 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:03.452 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:33:03.452 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:33:03.452 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-01-27T02:33:03.452 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:33:03.452 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:33:03.452 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-01-27T02:33:03.452 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:33:03.452 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:33:03.453 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:33:03.453 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:33:03.453 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-27T02:33:03.453 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42335sec preferred_lft 42335sec 2024-01-27T02:33:03.453 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b1:d6f7:baf0:bc17/64 scope link noprefixroute 2024-01-27T02:33:03.453 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:03.453 INFO:teuthology.orchestra.run.smithi188.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:33:03.453 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b0:92:71:90:9d brd ff:ff:ff:ff:ff:ff 2024-01-27T02:33:03.453 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-27T02:33:03.453 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:03.453 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::10f8:32ff:feed:a752/64 scope link 2024-01-27T02:33:03.453 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:03.453 INFO:teuthology.orchestra.run.smithi188.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:33:03.453 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b0:92:71:90:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-27T02:33:03.453 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b8b0:92ff:fe71:909d/64 scope link 2024-01-27T02:33:03.453 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:03.454 INFO:teuthology.orchestra.run.smithi188.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:33:03.454 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ee:78:03:20:3a:a9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-27T02:33:03.454 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec78:3ff:fe20:3aa9/64 scope link 2024-01-27T02:33:03.454 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:03.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:03.454 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:33:03.523 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-01-27T02:33:03.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:03.525 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-01-27T02:33:03.587 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-01-27T02:33:03.587 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:33:03.587 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-27T02:33:03.587 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-27T02:33:03.587 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:33:03.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:03.588 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-27T02:33:03.588 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-27T02:33:03.588 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-27T02:33:03.588 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-27T02:33:03.588 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-27T02:33:03.588 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-27T02:33:03.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:03.697 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-01-27T02:33:03.715 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-27T02:33:03.715 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-27T02:33:03.715 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:33:03.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:03.715 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2024-01-27T02:33:03.782 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:33:03.783 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-27T02:33:03.840 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:33:03.840 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:33:03.840 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-27T02:33:03.896 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:33:03.897 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:33:03.897 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:33:03.979 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:33:03.980 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:33:03.980 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:33:03.988 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:33:03.988 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:33:03.988 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:33:03.988 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:33:03.988 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:33:04.006 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:33:04.006 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:33:04.006 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:33:04.006 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:33:04.006 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:33:04.006 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:33:04.006 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:33:04.006 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:33:04.008 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:33:04.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:04.009 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-27T02:33:04.076 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:33:04.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.215+0000 7f3bae180700 1 Processor -- start 2024-01-27T02:33:04.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.215+0000 7f3bae180700 1 -- start start 2024-01-27T02:33:04.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.216+0000 7f3bae180700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3ba80b1af0 0x7f3ba80b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:04.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.216+0000 7f3bae180700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba80c4bd0 con 0x7f3ba80c5060 2024-01-27T02:33:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.216+0000 7f3bae180700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba80cac20 con 0x7f3ba80ca140 2024-01-27T02:33:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.216+0000 7f3bae180700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba80ce230 con 0x7f3ba80b1af0 2024-01-27T02:33:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.216+0000 7f3ba7fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3ba80c5060 0x7f3ba80ccb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34100/0 (socket says 172.21.15.7:34100) 2024-01-27T02:33:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.216+0000 7f3ba7fff700 1 -- 172.21.15.7:0/4013660374 learned_addr learned my addr 172.21.15.7:0/4013660374 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.216+0000 7f3ba6ffd700 1 --2- 172.21.15.7:0/4013660374 >> v2:172.21.15.188:3300/0 conn(0x7f3ba80b1af0 0x7f3ba80b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.216+0000 7f3ba67fc700 1 -- 172.21.15.7:0/4013660374 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1742561797 0 0) 0x7f3ba80c4bd0 con 0x7f3ba80c5060 2024-01-27T02:33:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.216+0000 7f3ba67fc700 1 -- 172.21.15.7:0/4013660374 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b90003570 con 0x7f3ba80c5060 2024-01-27T02:33:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.216+0000 7f3ba67fc700 1 -- 172.21.15.7:0/4013660374 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2951408521 0 0) 0x7f3ba80cac20 con 0x7f3ba80ca140 2024-01-27T02:33:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.217+0000 7f3ba67fc700 1 -- 172.21.15.7:0/4013660374 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ba80c4bd0 con 0x7f3ba80ca140 2024-01-27T02:33:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.217+0000 7f3ba6ffd700 1 -- 172.21.15.7:0/4013660374 >> v1:172.21.15.188:6789/0 conn(0x7f3ba80ca140 legacy=0x7f3ba80ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.217+0000 7f3ba6ffd700 1 -- 172.21.15.7:0/4013660374 >> v1:172.21.15.7:6789/0 conn(0x7f3ba80c5060 legacy=0x7f3ba80ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.217+0000 7f3ba6ffd700 1 -- 172.21.15.7:0/4013660374 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba80caea0 con 0x7f3ba80b1af0 2024-01-27T02:33:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.217+0000 7f3ba6ffd700 1 --2- 172.21.15.7:0/4013660374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ba80b1af0 0x7f3ba80b1f50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0098e0 tx=0x7f3b9c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fa7651f0f2424cb server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.217+0000 7f3ba67fc700 1 -- 172.21.15.7:0/4013660374 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c011630 con 0x7f3ba80b1af0 2024-01-27T02:33:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.217+0000 7f3ba67fc700 1 -- 172.21.15.7:0/4013660374 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9c011790 con 0x7f3ba80b1af0 2024-01-27T02:33:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.217+0000 7f3ba67fc700 1 -- 172.21.15.7:0/4013660374 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c011b30 con 0x7f3ba80b1af0 2024-01-27T02:33:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.218+0000 7f3bae180700 1 -- 172.21.15.7:0/4013660374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ba80b1af0 msgr2=0x7f3ba80b1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.218+0000 7f3bae180700 1 --2- 172.21.15.7:0/4013660374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ba80b1af0 0x7f3ba80b1f50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0098e0 tx=0x7f3b9c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.218+0000 7f3bae180700 1 -- 172.21.15.7:0/4013660374 shutdown_connections 2024-01-27T02:33:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.218+0000 7f3bae180700 1 --2- 172.21.15.7:0/4013660374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ba80b1af0 0x7f3ba80b1f50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.218+0000 7f3bae180700 1 -- 172.21.15.7:0/4013660374 >> 172.21.15.7:0/4013660374 conn(0x7f3ba80711c0 msgr2=0x7f3ba80ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.218+0000 7f3bae180700 1 -- 172.21.15.7:0/4013660374 shutdown_connections 2024-01-27T02:33:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.218+0000 7f3bae180700 1 -- 172.21.15.7:0/4013660374 wait complete. 2024-01-27T02:33:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.219+0000 7f3bae180700 1 Processor -- start 2024-01-27T02:33:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.219+0000 7f3bae180700 1 -- start start 2024-01-27T02:33:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.220+0000 7f3bae180700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ba80b1af0 0x7f3ba815c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.220+0000 7f3bae180700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba80c5060 0x7f3ba806c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.220+0000 7f3bae180700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ba80ca140 0x7f3ba806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.220+0000 7f3bae180700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba81430e0 con 0x7f3ba80c5060 2024-01-27T02:33:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.220+0000 7f3bae180700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba815daa0 con 0x7f3ba80b1af0 2024-01-27T02:33:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.220+0000 7f3bae180700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba815cc20 con 0x7f3ba80ca140 2024-01-27T02:33:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.220+0000 7f3ba6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba80c5060 0x7f3ba806c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.220+0000 7f3ba6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba80c5060 0x7f3ba806c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41062/0 (socket says 172.21.15.7:41062) 2024-01-27T02:33:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.220+0000 7f3ba6ffd700 1 -- 172.21.15.7:0/1645077752 learned_addr learned my addr 172.21.15.7:0/1645077752 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.220+0000 7f3ba77fe700 1 --2- 172.21.15.7:0/1645077752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ba80b1af0 0x7f3ba815c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.220+0000 7f3ba7fff700 1 --2- 172.21.15.7:0/1645077752 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ba80ca140 0x7f3ba806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.220+0000 7f3ba6ffd700 1 -- 172.21.15.7:0/1645077752 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ba80ca140 msgr2=0x7f3ba806cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.220+0000 7f3ba6ffd700 1 --2- 172.21.15.7:0/1645077752 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ba80ca140 0x7f3ba806cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.220+0000 7f3ba6ffd700 1 -- 172.21.15.7:0/1645077752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ba80b1af0 msgr2=0x7f3ba815c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.220+0000 7f3ba6ffd700 1 --2- 172.21.15.7:0/1645077752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ba80b1af0 0x7f3ba815c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.220+0000 7f3ba6ffd700 1 -- 172.21.15.7:0/1645077752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9c009420 con 0x7f3ba80c5060 2024-01-27T02:33:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.221+0000 7f3ba6ffd700 1 --2- 172.21.15.7:0/1645077752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba80c5060 0x7f3ba806c6b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c011f90 tx=0x7f3b9c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.221+0000 7f3b8f7fe700 1 -- 172.21.15.7:0/1645077752 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c01f070 con 0x7f3ba80c5060 2024-01-27T02:33:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.221+0000 7f3b8f7fe700 1 -- 172.21.15.7:0/1645077752 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9c011900 con 0x7f3ba80c5060 2024-01-27T02:33:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.221+0000 7f3bae180700 1 -- 172.21.15.7:0/1645077752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba815ce90 con 0x7f3ba80c5060 2024-01-27T02:33:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.221+0000 7f3b8f7fe700 1 -- 172.21.15.7:0/1645077752 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c023400 con 0x7f3ba80c5060 2024-01-27T02:33:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.221+0000 7f3bae180700 1 -- 172.21.15.7:0/1645077752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba815d450 con 0x7f3ba80c5060 2024-01-27T02:33:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.222+0000 7f3bae180700 1 -- 172.21.15.7:0/1645077752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba815dc30 con 0x7f3ba80c5060 2024-01-27T02:33:04.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.222+0000 7f3b8f7fe700 1 -- 172.21.15.7:0/1645077752 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3b9c023810 con 0x7f3ba80c5060 2024-01-27T02:33:04.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.223+0000 7f3b8f7fe700 1 --2- 172.21.15.7:0/1645077752 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3b94066f30 0x7f3b940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:04.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.223+0000 7f3b8f7fe700 1 -- 172.21.15.7:0/1645077752 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3b9c085680 con 0x7f3ba80c5060 2024-01-27T02:33:04.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.223+0000 7f3ba77fe700 1 --2- 172.21.15.7:0/1645077752 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3b94066f30 0x7f3b940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:04.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.223+0000 7f3ba77fe700 1 --2- 172.21.15.7:0/1645077752 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3b94066f30 0x7f3b940693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3b98007fd0 tx=0x7f3b98007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.227+0000 7f3b8f7fe700 1 -- 172.21.15.7:0/1645077752 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b9c0528f0 con 0x7f3ba80c5060 2024-01-27T02:33:04.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.475+0000 7f3bae180700 1 -- 172.21.15.7:0/1645077752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ba804c770 con 0x7f3ba80c5060 2024-01-27T02:33:04.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.476+0000 7f3b8f7fe700 1 -- 172.21.15.7:0/1645077752 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f3b9c055f10 con 0x7f3ba80c5060 2024-01-27T02:33:04.477 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:04.477 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4864,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/892460188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":892460188},{"type":"v1","addr":"172.21.15.188:6837","nonce":892460188}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4867,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1657319349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1657319349},{"type":"v1","addr":"172.21.15.188:6836","nonce":1657319349}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4879,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2190571534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2190571534},{"type":"v1","addr":"172.21.15.7:6834","nonce":2190571534}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:32:54.849803+0000","modified":"2024-01-27T02:32:58.809118+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4882},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4882":{"gid":4882,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6835/1734594727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1734594727},{"type":"v1","addr":"172.21.15.7:6835","nonce":1734594727}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-01-27T02:33:04.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.478+0000 7f3bae180700 1 -- 172.21.15.7:0/1645077752 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3b94066f30 msgr2=0x7f3b940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.478+0000 7f3bae180700 1 --2- 172.21.15.7:0/1645077752 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3b94066f30 0x7f3b940693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3b98007fd0 tx=0x7f3b98007d30 comp rx=0 tx=0).stop 2024-01-27T02:33:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.478+0000 7f3bae180700 1 -- 172.21.15.7:0/1645077752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba80c5060 msgr2=0x7f3ba806c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.478+0000 7f3bae180700 1 --2- 172.21.15.7:0/1645077752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba80c5060 0x7f3ba806c6b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c011f90 tx=0x7f3b9c017000 comp rx=0 tx=0).stop 2024-01-27T02:33:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.478+0000 7f3bae180700 1 -- 172.21.15.7:0/1645077752 shutdown_connections 2024-01-27T02:33:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.478+0000 7f3bae180700 1 --2- 172.21.15.7:0/1645077752 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3b94066f30 0x7f3b940693e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.478+0000 7f3bae180700 1 --2- 172.21.15.7:0/1645077752 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ba80ca140 0x7f3ba806cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.478+0000 7f3bae180700 1 --2- 172.21.15.7:0/1645077752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ba80b1af0 0x7f3ba815c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.478+0000 7f3bae180700 1 --2- 172.21.15.7:0/1645077752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba80c5060 0x7f3ba806c6b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.478+0000 7f3bae180700 1 -- 172.21.15.7:0/1645077752 >> 172.21.15.7:0/1645077752 conn(0x7f3ba80711c0 msgr2=0x7f3ba815abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.478+0000 7f3bae180700 1 -- 172.21.15.7:0/1645077752 shutdown_connections 2024-01-27T02:33:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.479+0000 7f3bae180700 1 -- 172.21.15.7:0/1645077752 wait complete. 2024-01-27T02:33:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2024-01-27T02:33:04.493 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-01-27T02:33:04.646 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","whoami":0,"id":4882,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.8293728510000005,"mdsmap_epoch":17,"osdmap_epoch":32,"osdmap_epoch_barrier":32,"uptime":17.245767625999999} 2024-01-27T02:33:04.646 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", "fs_name": "cephfs", "id": 4882, "mdsmap_epoch": 17, "osdmap_epoch": 32, "osdmap_epoch_barrier": 32, "rank_uptime": 8.829372851, "state": "up:active", "uptime": 17.245767626, "want_state": "up:active", "whoami": 0 } 2024-01-27T02:33:04.646 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2024-01-27T02:33:04.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.777+0000 7f92c2834700 1 Processor -- start 2024-01-27T02:33:04.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.777+0000 7f92c2834700 1 -- start start 2024-01-27T02:33:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.777+0000 7f92c2834700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92bc0b1aa0 0x7f92bc0b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.777+0000 7f92c2834700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92bc0c4b80 con 0x7f92bc0ca0f0 2024-01-27T02:33:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.777+0000 7f92c2834700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92bc0cabd0 con 0x7f92bc0c5010 2024-01-27T02:33:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.778+0000 7f92c2834700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc0ce1e0 con 0x7f92bc0b1aa0 2024-01-27T02:33:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.778+0000 7f92bbfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f92bc0ca0f0 0x7f92bc0ca4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34114/0 (socket says 172.21.15.7:34114) 2024-01-27T02:33:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.778+0000 7f92bbfff700 1 -- 172.21.15.7:0/1852599707 learned_addr learned my addr 172.21.15.7:0/1852599707 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.778+0000 7f92bb7fe700 1 --2- 172.21.15.7:0/1852599707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92bc0b1aa0 0x7f92bc0b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.778+0000 7f92baffd700 1 -- 172.21.15.7:0/1852599707 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 463834489 0 0) 0x7f92bc0c4b80 con 0x7f92bc0ca0f0 2024-01-27T02:33:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.778+0000 7f92baffd700 1 -- 172.21.15.7:0/1852599707 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92a4003570 con 0x7f92bc0ca0f0 2024-01-27T02:33:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.778+0000 7f92baffd700 1 -- 172.21.15.7:0/1852599707 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3572241261 0 0) 0x7f92bc0cabd0 con 0x7f92bc0c5010 2024-01-27T02:33:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.778+0000 7f92baffd700 1 -- 172.21.15.7:0/1852599707 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92bc0c4b80 con 0x7f92bc0c5010 2024-01-27T02:33:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.778+0000 7f92baffd700 1 -- 172.21.15.7:0/1852599707 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 464856212 0 0) 0x7f92a4003570 con 0x7f92bc0ca0f0 2024-01-27T02:33:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.778+0000 7f92bb7fe700 1 -- 172.21.15.7:0/1852599707 >> v1:172.21.15.188:6789/0 conn(0x7f92bc0c5010 legacy=0x7f92bc0ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.778+0000 7f92bb7fe700 1 -- 172.21.15.7:0/1852599707 >> v1:172.21.15.7:6789/0 conn(0x7f92bc0ca0f0 legacy=0x7f92bc0ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.778+0000 7f92bb7fe700 1 -- 172.21.15.7:0/1852599707 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc0cae50 con 0x7f92bc0b1aa0 2024-01-27T02:33:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.779+0000 7f92bb7fe700 1 --2- 172.21.15.7:0/1852599707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92bc0b1aa0 0x7f92bc0b1ed0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009910 tx=0x7f92b00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52b806a39498f135 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.779+0000 7f92baffd700 1 -- 172.21.15.7:0/1852599707 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac001470 con 0x7f92bc0b1aa0 2024-01-27T02:33:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.779+0000 7f92baffd700 1 -- 172.21.15.7:0/1852599707 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b0011420 con 0x7f92bc0b1aa0 2024-01-27T02:33:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.779+0000 7f92baffd700 1 -- 172.21.15.7:0/1852599707 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b00117c0 con 0x7f92bc0b1aa0 2024-01-27T02:33:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.779+0000 7f92c2834700 1 -- 172.21.15.7:0/1852599707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92bc0b1aa0 msgr2=0x7f92bc0b1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.779+0000 7f92c2834700 1 --2- 172.21.15.7:0/1852599707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92bc0b1aa0 0x7f92bc0b1ed0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009910 tx=0x7f92b00091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.779+0000 7f92c2834700 1 -- 172.21.15.7:0/1852599707 shutdown_connections 2024-01-27T02:33:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.779+0000 7f92c2834700 1 --2- 172.21.15.7:0/1852599707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92bc0b1aa0 0x7f92bc0b1ed0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.779+0000 7f92c2834700 1 -- 172.21.15.7:0/1852599707 >> 172.21.15.7:0/1852599707 conn(0x7f92bc0711c0 msgr2=0x7f92bc0cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.779+0000 7f92c2834700 1 -- 172.21.15.7:0/1852599707 shutdown_connections 2024-01-27T02:33:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.779+0000 7f92c2834700 1 -- 172.21.15.7:0/1852599707 wait complete. 2024-01-27T02:33:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.780+0000 7f92c2834700 1 Processor -- start 2024-01-27T02:33:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.781+0000 7f92c2834700 1 -- start start 2024-01-27T02:33:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.781+0000 7f92c2834700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92bc0b1aa0 0x7f92bc157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.781+0000 7f92c2834700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92bc0c5010 0x7f92bc0c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.781+0000 7f92c2834700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92bc0ca0f0 0x7f92bc15ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.781+0000 7f92c2834700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc069790 con 0x7f92bc0c5010 2024-01-27T02:33:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.781+0000 7f92c2834700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc16e820 con 0x7f92bc0b1aa0 2024-01-27T02:33:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.781+0000 7f92c2834700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc157c70 con 0x7f92bc0ca0f0 2024-01-27T02:33:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.782+0000 7f92bb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92bc0c5010 0x7f92bc0c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:04.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.782+0000 7f92bb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92bc0c5010 0x7f92bc0c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41068/0 (socket says 172.21.15.7:41068) 2024-01-27T02:33:04.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.782+0000 7f92bb7fe700 1 -- 172.21.15.7:0/4251290411 learned_addr learned my addr 172.21.15.7:0/4251290411 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:04.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.782+0000 7f92bbfff700 1 --2- 172.21.15.7:0/4251290411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92bc0b1aa0 0x7f92bc157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:04.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.782+0000 7f92c0dd1700 1 --2- 172.21.15.7:0/4251290411 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92bc0ca0f0 0x7f92bc15ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:04.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.782+0000 7f92bb7fe700 1 -- 172.21.15.7:0/4251290411 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92bc0ca0f0 msgr2=0x7f92bc15ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:04.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.782+0000 7f92bb7fe700 1 --2- 172.21.15.7:0/4251290411 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92bc0ca0f0 0x7f92bc15ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:04.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.782+0000 7f92bb7fe700 1 -- 172.21.15.7:0/4251290411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92bc0b1aa0 msgr2=0x7f92bc157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:04.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.782+0000 7f92bb7fe700 1 --2- 172.21.15.7:0/4251290411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92bc0b1aa0 0x7f92bc157730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:04.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.782+0000 7f92bb7fe700 1 -- 172.21.15.7:0/4251290411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b0009420 con 0x7f92bc0c5010 2024-01-27T02:33:04.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.782+0000 7f92bb7fe700 1 --2- 172.21.15.7:0/4251290411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92bc0c5010 0x7f92bc0c85d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f92b00091e0 tx=0x7f92b0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:04.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.782+0000 7f92a3fff700 1 -- 172.21.15.7:0/4251290411 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b001cb30 con 0x7f92bc0c5010 2024-01-27T02:33:04.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.782+0000 7f92a3fff700 1 -- 172.21.15.7:0/4251290411 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b001cc90 con 0x7f92bc0c5010 2024-01-27T02:33:04.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.782+0000 7f92c2834700 1 -- 172.21.15.7:0/4251290411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92bc172300 con 0x7f92bc0c5010 2024-01-27T02:33:04.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.782+0000 7f92a3fff700 1 -- 172.21.15.7:0/4251290411 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b001a6a0 con 0x7f92bc0c5010 2024-01-27T02:33:04.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.783+0000 7f92c2834700 1 -- 172.21.15.7:0/4251290411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92bc1728c0 con 0x7f92bc0c5010 2024-01-27T02:33:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.784+0000 7f92c2834700 1 -- 172.21.15.7:0/4251290411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92bc16e9e0 con 0x7f92bc0c5010 2024-01-27T02:33:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.784+0000 7f92a3fff700 1 -- 172.21.15.7:0/4251290411 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f92b001aa90 con 0x7f92bc0c5010 2024-01-27T02:33:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.784+0000 7f92a3fff700 1 --2- 172.21.15.7:0/4251290411 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f92a8066f30 0x7f92a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.784+0000 7f92a3fff700 1 -- 172.21.15.7:0/4251290411 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f92b00862f0 con 0x7f92bc0c5010 2024-01-27T02:33:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.785+0000 7f92bbfff700 1 --2- 172.21.15.7:0/4251290411 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f92a8066f30 0x7f92a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.785+0000 7f92bbfff700 1 --2- 172.21.15.7:0/4251290411 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f92a8066f30 0x7f92a80693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0015a0 tx=0x7f92ac000bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:04.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:04.789+0000 7f92a3fff700 1 -- 172.21.15.7:0/4251290411 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f92bc16e9e0 con 0x7f92bc0c5010 2024-01-27T02:33:05.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.073+0000 7f92c2834700 1 -- 172.21.15.7:0/4251290411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f92bc16e9e0 con 0x7f92bc0c5010 2024-01-27T02:33:05.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.074+0000 7f92a3fff700 1 -- 172.21.15.7:0/4251290411 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1620 (secure 0 0 0) 0x7f92bc16e9e0 con 0x7f92bc0c5010 2024-01-27T02:33:05.074 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:05.075 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:33:05.075 INFO:teuthology.orchestra.run.smithi007.stdout: "stats": { 2024-01-27T02:33:05.075 INFO:teuthology.orchestra.run.smithi007.stdout: "total_bytes": 1600000000, 2024-01-27T02:33:05.075 INFO:teuthology.orchestra.run.smithi007.stdout: "total_avail_bytes": 1597708948, 2024-01-27T02:33:05.075 INFO:teuthology.orchestra.run.smithi007.stdout: "total_used_bytes": 2291052, 2024-01-27T02:33:05.075 INFO:teuthology.orchestra.run.smithi007.stdout: "total_used_raw_bytes": 2291052, 2024-01-27T02:33:05.075 INFO:teuthology.orchestra.run.smithi007.stdout: "total_used_raw_ratio": 0.0014319075271487236, 2024-01-27T02:33:05.075 INFO:teuthology.orchestra.run.smithi007.stdout: "num_osds": 8, 2024-01-27T02:33:05.075 INFO:teuthology.orchestra.run.smithi007.stdout: "num_per_pool_osds": 0, 2024-01-27T02:33:05.075 INFO:teuthology.orchestra.run.smithi007.stdout: "num_per_pool_omap_osds": 0 2024-01-27T02:33:05.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:05.075 INFO:teuthology.orchestra.run.smithi007.stdout: "stats_by_class": { 2024-01-27T02:33:05.076 INFO:teuthology.orchestra.run.smithi007.stdout: "hdd": { 2024-01-27T02:33:05.076 INFO:teuthology.orchestra.run.smithi007.stdout: "total_bytes": 1600000000, 2024-01-27T02:33:05.076 INFO:teuthology.orchestra.run.smithi007.stdout: "total_avail_bytes": 1597708948, 2024-01-27T02:33:05.076 INFO:teuthology.orchestra.run.smithi007.stdout: "total_used_bytes": 2291052, 2024-01-27T02:33:05.076 INFO:teuthology.orchestra.run.smithi007.stdout: "total_used_raw_bytes": 2291052, 2024-01-27T02:33:05.076 INFO:teuthology.orchestra.run.smithi007.stdout: "total_used_raw_ratio": 0.0014319075271487236 2024-01-27T02:33:05.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:05.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:05.076 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:33:05.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:05.077 INFO:teuthology.orchestra.run.smithi007.stdout: "name": ".mgr", 2024-01-27T02:33:05.077 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1, 2024-01-27T02:33:05.077 INFO:teuthology.orchestra.run.smithi007.stdout: "stats": { 2024-01-27T02:33:05.077 INFO:teuthology.orchestra.run.smithi007.stdout: "stored": 590368, 2024-01-27T02:33:05.077 INFO:teuthology.orchestra.run.smithi007.stdout: "objects": 2, 2024-01-27T02:33:05.077 INFO:teuthology.orchestra.run.smithi007.stdout: "kb_used": 577, 2024-01-27T02:33:05.077 INFO:teuthology.orchestra.run.smithi007.stdout: "bytes_used": 590368, 2024-01-27T02:33:05.077 INFO:teuthology.orchestra.run.smithi007.stdout: "percent_used": 0.00052959454478695989, 2024-01-27T02:33:05.077 INFO:teuthology.orchestra.run.smithi007.stdout: "max_avail": 557082176 2024-01-27T02:33:05.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:05.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:05.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:05.078 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephfs_metadata", 2024-01-27T02:33:05.078 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4, 2024-01-27T02:33:05.078 INFO:teuthology.orchestra.run.smithi007.stdout: "stats": { 2024-01-27T02:33:05.078 INFO:teuthology.orchestra.run.smithi007.stdout: "stored": 2306, 2024-01-27T02:33:05.078 INFO:teuthology.orchestra.run.smithi007.stdout: "objects": 22, 2024-01-27T02:33:05.078 INFO:teuthology.orchestra.run.smithi007.stdout: "kb_used": 3, 2024-01-27T02:33:05.078 INFO:teuthology.orchestra.run.smithi007.stdout: "bytes_used": 2306, 2024-01-27T02:33:05.078 INFO:teuthology.orchestra.run.smithi007.stdout: "percent_used": 2.0697082163678715e-06, 2024-01-27T02:33:05.078 INFO:teuthology.orchestra.run.smithi007.stdout: "max_avail": 557082176 2024-01-27T02:33:05.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:05.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:05.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:05.078 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephfs_data", 2024-01-27T02:33:05.078 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5, 2024-01-27T02:33:05.078 INFO:teuthology.orchestra.run.smithi007.stdout: "stats": { 2024-01-27T02:33:05.079 INFO:teuthology.orchestra.run.smithi007.stdout: "stored": 0, 2024-01-27T02:33:05.079 INFO:teuthology.orchestra.run.smithi007.stdout: "objects": 0, 2024-01-27T02:33:05.079 INFO:teuthology.orchestra.run.smithi007.stdout: "kb_used": 0, 2024-01-27T02:33:05.079 INFO:teuthology.orchestra.run.smithi007.stdout: "bytes_used": 0, 2024-01-27T02:33:05.079 INFO:teuthology.orchestra.run.smithi007.stdout: "percent_used": 0, 2024-01-27T02:33:05.079 INFO:teuthology.orchestra.run.smithi007.stdout: "max_avail": 557082176 2024-01-27T02:33:05.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:05.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:05.079 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:05.079 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:33:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.076+0000 7f92c2834700 1 -- 172.21.15.7:0/4251290411 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f92a8066f30 msgr2=0x7f92a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.076+0000 7f92c2834700 1 --2- 172.21.15.7:0/4251290411 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f92a8066f30 0x7f92a80693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0015a0 tx=0x7f92ac000bc0 comp rx=0 tx=0).stop 2024-01-27T02:33:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.076+0000 7f92c2834700 1 -- 172.21.15.7:0/4251290411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92bc0c5010 msgr2=0x7f92bc0c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.076+0000 7f92c2834700 1 --2- 172.21.15.7:0/4251290411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92bc0c5010 0x7f92bc0c85d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f92b00091e0 tx=0x7f92b0016000 comp rx=0 tx=0).stop 2024-01-27T02:33:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.076+0000 7f92c2834700 1 -- 172.21.15.7:0/4251290411 shutdown_connections 2024-01-27T02:33:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.076+0000 7f92c2834700 1 --2- 172.21.15.7:0/4251290411 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f92a8066f30 0x7f92a80693e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.076+0000 7f92c2834700 1 --2- 172.21.15.7:0/4251290411 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92bc0ca0f0 0x7f92bc15ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:05.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.076+0000 7f92c2834700 1 --2- 172.21.15.7:0/4251290411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92bc0b1aa0 0x7f92bc157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:05.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.076+0000 7f92c2834700 1 --2- 172.21.15.7:0/4251290411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92bc0c5010 0x7f92bc0c85d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:05.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.076+0000 7f92c2834700 1 -- 172.21.15.7:0/4251290411 >> 172.21.15.7:0/4251290411 conn(0x7f92bc0711c0 msgr2=0x7f92bc0cebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:05.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.076+0000 7f92c2834700 1 -- 172.21.15.7:0/4251290411 shutdown_connections 2024-01-27T02:33:05.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.076+0000 7f92c2834700 1 -- 172.21.15.7:0/4251290411 wait complete. 2024-01-27T02:33:05.090 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-01-27T02:33:05.090 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-01-27T02:33:05.090 DEBUG:teuthology.orchestra.run.smithi007:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-01-27T02:33:05.090 DEBUG:teuthology.orchestra.run.smithi007:> os.fchmod(fd, 0O666) 2024-01-27T02:33:05.090 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(fd) 2024-01-27T02:33:05.090 DEBUG:teuthology.orchestra.run.smithi007:> os.close(fd) 2024-01-27T02:33:05.090 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-01-27T02:33:05.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:33:05.353 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-27T02:33:05.437 INFO:teuthology.orchestra.run.smithi007.stdout:client5038 2024-01-27T02:33:05.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:05.437 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/17e1bcca-e647-4a58-b0ac-7d39d8508814.client5038/osdmap') 2024-01-27T02:33:05.531 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 32 barrier 32 flags 0x588000 2024-01-27T02:33:05.532 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:33:05.532 INFO:teuthology.orchestra.run.smithi007.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:33:05.532 INFO:teuthology.orchestra.run.smithi007.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:33:05.532 INFO:teuthology.orchestra.run.smithi007.stdout:osd0 (1)172.21.15.7:6817 100% (exists, up) 100% -1 2024-01-27T02:33:05.532 INFO:teuthology.orchestra.run.smithi007.stdout:osd1 (1)172.21.15.7:6801 100% (exists, up) 100% -1 2024-01-27T02:33:05.532 INFO:teuthology.orchestra.run.smithi007.stdout:osd2 (1)172.21.15.7:6825 100% (exists, up) 100% -1 2024-01-27T02:33:05.532 INFO:teuthology.orchestra.run.smithi007.stdout:osd3 (1)172.21.15.7:6809 100% (exists, up) 100% -1 2024-01-27T02:33:05.532 INFO:teuthology.orchestra.run.smithi007.stdout:osd4 (1)172.21.15.188:6803 100% (exists, up) 100% -1 2024-01-27T02:33:05.532 INFO:teuthology.orchestra.run.smithi007.stdout:osd5 (1)172.21.15.188:6802 100% (exists, up) 100% -1 2024-01-27T02:33:05.532 INFO:teuthology.orchestra.run.smithi007.stdout:osd6 (1)172.21.15.188:6825 100% (exists, up) 100% -1 2024-01-27T02:33:05.532 INFO:teuthology.orchestra.run.smithi007.stdout:osd7 (1)172.21.15.188:6817 100% (exists, up) 100% -1 2024-01-27T02:33:05.533 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records in 2024-01-27T02:33:05.534 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records out 2024-01-27T02:33:05.534 INFO:teuthology.orchestra.run.smithi007.stderr:791 bytes copied, 4.652e-05 s, 17.0 MB/s 2024-01-27T02:33:05.534 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2024-01-27T02:33:05.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.672+0000 7f8d22466700 1 Processor -- start 2024-01-27T02:33:05.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.672+0000 7f8d22466700 1 -- start start 2024-01-27T02:33:05.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.672+0000 7f8d22466700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d1c0c8ab0 0x7f8d1c15dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:05.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.672+0000 7f8d22466700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d1c15a9b0 con 0x7f8d1c0cbbc0 2024-01-27T02:33:05.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.673+0000 7f8d22466700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d1c15ecb0 con 0x7f8d1c0c8e80 2024-01-27T02:33:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.673+0000 7f8d22466700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c15e410 con 0x7f8d1c0c8ab0 2024-01-27T02:33:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.673+0000 7f8d20a03700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8d1c0cbbc0 0x7f8d1c15ae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34120/0 (socket says 172.21.15.7:34120) 2024-01-27T02:33:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.673+0000 7f8d20a03700 1 -- 172.21.15.7:0/4099425489 learned_addr learned my addr 172.21.15.7:0/4099425489 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.673+0000 7f8d1bfff700 1 --2- 172.21.15.7:0/4099425489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d1c0c8ab0 0x7f8d1c15dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.673+0000 7f8d1affd700 1 -- 172.21.15.7:0/4099425489 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 542353802 0 0) 0x7f8d1c15a9b0 con 0x7f8d1c0cbbc0 2024-01-27T02:33:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.673+0000 7f8d1affd700 1 -- 172.21.15.7:0/4099425489 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d04003570 con 0x7f8d1c0cbbc0 2024-01-27T02:33:05.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.673+0000 7f8d1affd700 1 -- 172.21.15.7:0/4099425489 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964742027 0 0) 0x7f8d1c15ecb0 con 0x7f8d1c0c8e80 2024-01-27T02:33:05.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.673+0000 7f8d1affd700 1 -- 172.21.15.7:0/4099425489 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d1c15a9b0 con 0x7f8d1c0c8e80 2024-01-27T02:33:05.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.673+0000 7f8d1bfff700 1 -- 172.21.15.7:0/4099425489 >> v1:172.21.15.188:6789/0 conn(0x7f8d1c0c8e80 legacy=0x7f8d1c0cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:05.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.673+0000 7f8d1bfff700 1 -- 172.21.15.7:0/4099425489 >> v1:172.21.15.7:6789/0 conn(0x7f8d1c0cbbc0 legacy=0x7f8d1c15ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:05.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.673+0000 7f8d1bfff700 1 -- 172.21.15.7:0/4099425489 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c15e6f0 con 0x7f8d1c0c8ab0 2024-01-27T02:33:05.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.674+0000 7f8d1bfff700 1 --2- 172.21.15.7:0/4099425489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d1c0c8ab0 0x7f8d1c15dd80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009910 tx=0x7f8d0c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cb481896a789668 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:05.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.674+0000 7f8d1affd700 1 -- 172.21.15.7:0/4099425489 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c011630 con 0x7f8d1c0c8ab0 2024-01-27T02:33:05.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.674+0000 7f8d1affd700 1 -- 172.21.15.7:0/4099425489 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0c011790 con 0x7f8d1c0c8ab0 2024-01-27T02:33:05.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.674+0000 7f8d1affd700 1 -- 172.21.15.7:0/4099425489 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c011b30 con 0x7f8d1c0c8ab0 2024-01-27T02:33:05.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.674+0000 7f8d22466700 1 -- 172.21.15.7:0/4099425489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d1c0c8ab0 msgr2=0x7f8d1c15dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:05.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.674+0000 7f8d22466700 1 --2- 172.21.15.7:0/4099425489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d1c0c8ab0 0x7f8d1c15dd80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009910 tx=0x7f8d0c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:05.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.674+0000 7f8d22466700 1 -- 172.21.15.7:0/4099425489 shutdown_connections 2024-01-27T02:33:05.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.674+0000 7f8d22466700 1 --2- 172.21.15.7:0/4099425489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d1c0c8ab0 0x7f8d1c15dd80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:05.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.674+0000 7f8d22466700 1 -- 172.21.15.7:0/4099425489 >> 172.21.15.7:0/4099425489 conn(0x7f8d1c0cdc30 msgr2=0x7f8d1c0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:05.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.675+0000 7f8d22466700 1 -- 172.21.15.7:0/4099425489 shutdown_connections 2024-01-27T02:33:05.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.675+0000 7f8d22466700 1 -- 172.21.15.7:0/4099425489 wait complete. 2024-01-27T02:33:05.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.676+0000 7f8d22466700 1 Processor -- start 2024-01-27T02:33:05.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.676+0000 7f8d22466700 1 -- start start 2024-01-27T02:33:05.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.677+0000 7f8d22466700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d1c0c8ab0 0x7f8d1c06c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.677+0000 7f8d22466700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d1c0c8e80 0x7f8d1c155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.677+0000 7f8d22466700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d1c0cbbc0 0x7f8d1c158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.677+0000 7f8d22466700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c06b610 con 0x7f8d1c0cbbc0 2024-01-27T02:33:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.677+0000 7f8d22466700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c06b8f0 con 0x7f8d1c0c8e80 2024-01-27T02:33:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.677+0000 7f8d22466700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c06bbd0 con 0x7f8d1c0c8ab0 2024-01-27T02:33:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.677+0000 7f8d20a03700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d1c0cbbc0 0x7f8d1c158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.677+0000 7f8d20a03700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d1c0cbbc0 0x7f8d1c158d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41078/0 (socket says 172.21.15.7:41078) 2024-01-27T02:33:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.677+0000 7f8d20a03700 1 -- 172.21.15.7:0/2498727527 learned_addr learned my addr 172.21.15.7:0/2498727527 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.677+0000 7f8d1b7fe700 1 --2- 172.21.15.7:0/2498727527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d1c0c8e80 0x7f8d1c155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.677+0000 7f8d1bfff700 1 --2- 172.21.15.7:0/2498727527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d1c0c8ab0 0x7f8d1c06c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.677+0000 7f8d20a03700 1 -- 172.21.15.7:0/2498727527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d1c0c8ab0 msgr2=0x7f8d1c06c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.677+0000 7f8d20a03700 1 --2- 172.21.15.7:0/2498727527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d1c0c8ab0 0x7f8d1c06c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.677+0000 7f8d20a03700 1 -- 172.21.15.7:0/2498727527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d1c0c8e80 msgr2=0x7f8d1c155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:05.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.677+0000 7f8d20a03700 1 --2- 172.21.15.7:0/2498727527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d1c0c8e80 0x7f8d1c155bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:05.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.677+0000 7f8d20a03700 1 -- 172.21.15.7:0/2498727527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c009420 con 0x7f8d1c0cbbc0 2024-01-27T02:33:05.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.677+0000 7f8d1b7fe700 1 --2- 172.21.15.7:0/2498727527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d1c0c8e80 0x7f8d1c155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:05.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.677+0000 7f8d20a03700 1 --2- 172.21.15.7:0/2498727527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d1c0cbbc0 0x7f8d1c158d70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400cc20 tx=0x7f8d14001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:05.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.678+0000 7f8d03fff700 1 -- 172.21.15.7:0/2498727527 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1400a960 con 0x7f8d1c0cbbc0 2024-01-27T02:33:05.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.678+0000 7f8d03fff700 1 -- 172.21.15.7:0/2498727527 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1400aac0 con 0x7f8d1c0cbbc0 2024-01-27T02:33:05.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.678+0000 7f8d22466700 1 -- 172.21.15.7:0/2498727527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d1c0ad530 con 0x7f8d1c0cbbc0 2024-01-27T02:33:05.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.678+0000 7f8d03fff700 1 -- 172.21.15.7:0/2498727527 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1400ae80 con 0x7f8d1c0cbbc0 2024-01-27T02:33:05.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.678+0000 7f8d22466700 1 -- 172.21.15.7:0/2498727527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1c06cb20 con 0x7f8d1c0cbbc0 2024-01-27T02:33:05.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.679+0000 7f8d22466700 1 -- 172.21.15.7:0/2498727527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d1c06bdf0 con 0x7f8d1c0cbbc0 2024-01-27T02:33:05.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.679+0000 7f8d03fff700 1 -- 172.21.15.7:0/2498727527 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8d14007300 con 0x7f8d1c0cbbc0 2024-01-27T02:33:05.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.680+0000 7f8d03fff700 1 --2- 172.21.15.7:0/2498727527 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8d08066f30 0x7f8d080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:05.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.680+0000 7f8d03fff700 1 -- 172.21.15.7:0/2498727527 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8d14084900 con 0x7f8d1c0cbbc0 2024-01-27T02:33:05.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.680+0000 7f8d1bfff700 1 --2- 172.21.15.7:0/2498727527 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8d08066f30 0x7f8d080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:05.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.681+0000 7f8d1bfff700 1 --2- 172.21.15.7:0/2498727527 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8d08066f30 0x7f8d080693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0c97d0 tx=0x7f8d0c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:05.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.685+0000 7f8d03fff700 1 -- 172.21.15.7:0/2498727527 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d14055680 con 0x7f8d1c0cbbc0 2024-01-27T02:33:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:05.900+0000 7f8d22466700 1 -- 172.21.15.7:0/2498727527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) v1 -- 0x7f8d1c04c710 con 0x7f8d1c0cbbc0 2024-01-27T02:33:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:06.889+0000 7f8d03fff700 1 -- 172.21.15.7:0/2498727527 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v33) v1 ==== 93+0+0 (secure 0 0 0) 0x7f8d14055210 con 0x7f8d1c0cbbc0 2024-01-27T02:33:06.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:06.892+0000 7f8d22466700 1 -- 172.21.15.7:0/2498727527 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8d08066f30 msgr2=0x7f8d080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:06.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:06.892+0000 7f8d22466700 1 --2- 172.21.15.7:0/2498727527 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8d08066f30 0x7f8d080693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0c97d0 tx=0x7f8d0c017000 comp rx=0 tx=0).stop 2024-01-27T02:33:06.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:06.892+0000 7f8d22466700 1 -- 172.21.15.7:0/2498727527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d1c0cbbc0 msgr2=0x7f8d1c158d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:06.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:06.892+0000 7f8d22466700 1 --2- 172.21.15.7:0/2498727527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d1c0cbbc0 0x7f8d1c158d70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400cc20 tx=0x7f8d14001a40 comp rx=0 tx=0).stop 2024-01-27T02:33:06.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:06.893+0000 7f8d22466700 1 -- 172.21.15.7:0/2498727527 shutdown_connections 2024-01-27T02:33:06.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:06.893+0000 7f8d22466700 1 --2- 172.21.15.7:0/2498727527 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8d08066f30 0x7f8d080693e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:06.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:06.893+0000 7f8d22466700 1 --2- 172.21.15.7:0/2498727527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d1c0cbbc0 0x7f8d1c158d70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:06.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:06.893+0000 7f8d22466700 1 --2- 172.21.15.7:0/2498727527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d1c0c8ab0 0x7f8d1c06c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:06.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:06.893+0000 7f8d22466700 1 --2- 172.21.15.7:0/2498727527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d1c0c8e80 0x7f8d1c155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:06.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:06.893+0000 7f8d22466700 1 -- 172.21.15.7:0/2498727527 >> 172.21.15.7:0/2498727527 conn(0x7f8d1c0cdc30 msgr2=0x7f8d1c0c3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:06.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:06.893+0000 7f8d22466700 1 -- 172.21.15.7:0/2498727527 shutdown_connections 2024-01-27T02:33:06.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:06.893+0000 7f8d22466700 1 -- 172.21.15.7:0/2498727527 wait complete. 2024-01-27T02:33:06.896 INFO:teuthology.orchestra.run.smithi007.stderr:pauserd,pausewr is set 2024-01-27T02:33:06.909 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2024-01-27T02:33:07.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.045+0000 7f1f68d07700 1 Processor -- start 2024-01-27T02:33:07.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.045+0000 7f1f68d07700 1 -- start start 2024-01-27T02:33:07.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.046+0000 7f1f68d07700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1f640cdbf0 0x7f1f64162c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:07.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.046+0000 7f1f68d07700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f640ca020 con 0x7f1f64157ca0 2024-01-27T02:33:07.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.046+0000 7f1f68d07700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f640c5210 con 0x7f1f640c3ae0 2024-01-27T02:33:07.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.046+0000 7f1f68d07700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f640c54f0 con 0x7f1f640cdbf0 2024-01-27T02:33:07.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.046+0000 7f1f6259c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1f64157ca0 0x7f1f640c4a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34136/0 (socket says 172.21.15.7:34136) 2024-01-27T02:33:07.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.046+0000 7f1f6259c700 1 -- 172.21.15.7:0/4071656926 learned_addr learned my addr 172.21.15.7:0/4071656926 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:07.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.047+0000 7f1f62d9d700 1 --2- 172.21.15.7:0/4071656926 >> v2:172.21.15.188:3300/0 conn(0x7f1f640cdbf0 0x7f1f64162c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:07.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.047+0000 7f1f6159a700 1 -- 172.21.15.7:0/4071656926 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055007444 0 0) 0x7f1f640ca020 con 0x7f1f64157ca0 2024-01-27T02:33:07.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.047+0000 7f1f6159a700 1 -- 172.21.15.7:0/4071656926 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f4c003570 con 0x7f1f64157ca0 2024-01-27T02:33:07.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.047+0000 7f1f6159a700 1 -- 172.21.15.7:0/4071656926 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3575941584 0 0) 0x7f1f640c5210 con 0x7f1f640c3ae0 2024-01-27T02:33:07.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.047+0000 7f1f6159a700 1 -- 172.21.15.7:0/4071656926 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f640ca020 con 0x7f1f640c3ae0 2024-01-27T02:33:07.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.047+0000 7f1f62d9d700 1 -- 172.21.15.7:0/4071656926 >> v1:172.21.15.188:6789/0 conn(0x7f1f640c3ae0 legacy=0x7f1f640cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:07.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.047+0000 7f1f62d9d700 1 -- 172.21.15.7:0/4071656926 >> v1:172.21.15.7:6789/0 conn(0x7f1f64157ca0 legacy=0x7f1f640c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:07.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.047+0000 7f1f62d9d700 1 -- 172.21.15.7:0/4071656926 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f640c57d0 con 0x7f1f640cdbf0 2024-01-27T02:33:07.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.047+0000 7f1f62d9d700 1 --2- 172.21.15.7:0/4071656926 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f640cdbf0 0x7f1f64162c00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009e60 tx=0x7f1f5c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e3312b57d58023f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:07.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.047+0000 7f1f6159a700 1 -- 172.21.15.7:0/4071656926 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c00e070 con 0x7f1f640cdbf0 2024-01-27T02:33:07.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.047+0000 7f1f6159a700 1 -- 172.21.15.7:0/4071656926 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c00bab0 con 0x7f1f640cdbf0 2024-01-27T02:33:07.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.048+0000 7f1f6159a700 1 -- 172.21.15.7:0/4071656926 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c00be70 con 0x7f1f640cdbf0 2024-01-27T02:33:07.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.048+0000 7f1f68d07700 1 -- 172.21.15.7:0/4071656926 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f640cdbf0 msgr2=0x7f1f64162c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:07.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.048+0000 7f1f68d07700 1 --2- 172.21.15.7:0/4071656926 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f640cdbf0 0x7f1f64162c00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009e60 tx=0x7f1f5c009780 comp rx=0 tx=0).stop 2024-01-27T02:33:07.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.048+0000 7f1f68d07700 1 -- 172.21.15.7:0/4071656926 shutdown_connections 2024-01-27T02:33:07.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.048+0000 7f1f68d07700 1 --2- 172.21.15.7:0/4071656926 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f640cdbf0 0x7f1f64162c00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:07.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.048+0000 7f1f68d07700 1 -- 172.21.15.7:0/4071656926 >> 172.21.15.7:0/4071656926 conn(0x7f1f64098020 msgr2=0x7f1f640711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:07.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.048+0000 7f1f68d07700 1 -- 172.21.15.7:0/4071656926 shutdown_connections 2024-01-27T02:33:07.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.048+0000 7f1f68d07700 1 -- 172.21.15.7:0/4071656926 wait complete. 2024-01-27T02:33:07.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.050+0000 7f1f68d07700 1 Processor -- start 2024-01-27T02:33:07.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.050+0000 7f1f68d07700 1 -- start start 2024-01-27T02:33:07.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.051+0000 7f1f68d07700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f640c3ae0 0x7f1f640b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:07.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.051+0000 7f1f68d07700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f640cdbf0 0x7f1f6416a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:07.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.051+0000 7f1f68d07700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f64157ca0 0x7f1f641508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:07.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.051+0000 7f1f68d07700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640b3660 con 0x7f1f640c3ae0 2024-01-27T02:33:07.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.051+0000 7f1f68d07700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640b44a0 con 0x7f1f64157ca0 2024-01-27T02:33:07.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.051+0000 7f1f68d07700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640b3890 con 0x7f1f640cdbf0 2024-01-27T02:33:07.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.051+0000 7f1f6259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f640c3ae0 0x7f1f640b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:07.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.051+0000 7f1f6259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f640c3ae0 0x7f1f640b1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41090/0 (socket says 172.21.15.7:41090) 2024-01-27T02:33:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.051+0000 7f1f6259c700 1 -- 172.21.15.7:0/3039649876 learned_addr learned my addr 172.21.15.7:0/3039649876 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.051+0000 7f1f62d9d700 1 --2- 172.21.15.7:0/3039649876 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f64157ca0 0x7f1f641508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.051+0000 7f1f61d9b700 1 --2- 172.21.15.7:0/3039649876 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f640cdbf0 0x7f1f6416a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.051+0000 7f1f6259c700 1 -- 172.21.15.7:0/3039649876 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f640cdbf0 msgr2=0x7f1f6416a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.051+0000 7f1f6259c700 1 --2- 172.21.15.7:0/3039649876 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f640cdbf0 0x7f1f6416a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.051+0000 7f1f6259c700 1 -- 172.21.15.7:0/3039649876 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f64157ca0 msgr2=0x7f1f641508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.051+0000 7f1f6259c700 1 --2- 172.21.15.7:0/3039649876 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f64157ca0 0x7f1f641508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.051+0000 7f1f6259c700 1 -- 172.21.15.7:0/3039649876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c009a20 con 0x7f1f640c3ae0 2024-01-27T02:33:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.051+0000 7f1f62d9d700 1 --2- 172.21.15.7:0/3039649876 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f64157ca0 0x7f1f641508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:33:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.051+0000 7f1f61d9b700 1 --2- 172.21.15.7:0/3039649876 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f640cdbf0 0x7f1f6416a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:33:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.052+0000 7f1f6259c700 1 --2- 172.21.15.7:0/3039649876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f640c3ae0 0x7f1f640b1b00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400c790 tx=0x7f1f54007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.052+0000 7f1f527fc700 1 -- 172.21.15.7:0/3039649876 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54005b30 con 0x7f1f640c3ae0 2024-01-27T02:33:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.052+0000 7f1f527fc700 1 -- 172.21.15.7:0/3039649876 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f54005c90 con 0x7f1f640c3ae0 2024-01-27T02:33:07.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.052+0000 7f1f68d07700 1 -- 172.21.15.7:0/3039649876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f640b3b70 con 0x7f1f640c3ae0 2024-01-27T02:33:07.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.052+0000 7f1f527fc700 1 -- 172.21.15.7:0/3039649876 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54007300 con 0x7f1f640c3ae0 2024-01-27T02:33:07.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.052+0000 7f1f68d07700 1 -- 172.21.15.7:0/3039649876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f640b4130 con 0x7f1f640c3ae0 2024-01-27T02:33:07.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.053+0000 7f1f68d07700 1 -- 172.21.15.7:0/3039649876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f640b4780 con 0x7f1f640c3ae0 2024-01-27T02:33:07.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.053+0000 7f1f527fc700 1 -- 172.21.15.7:0/3039649876 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1f5401b400 con 0x7f1f640c3ae0 2024-01-27T02:33:07.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.054+0000 7f1f527fc700 1 --2- 172.21.15.7:0/3039649876 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1f44066f30 0x7f1f440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:07.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.054+0000 7f1f527fc700 1 -- 172.21.15.7:0/3039649876 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1f540840a0 con 0x7f1f640c3ae0 2024-01-27T02:33:07.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.054+0000 7f1f527fc700 1 -- 172.21.15.7:0/3039649876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=34+}) v3 -- 0x7f1f4406ea50 con 0x7f1f640c3ae0 2024-01-27T02:33:07.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.054+0000 7f1f61d9b700 1 --2- 172.21.15.7:0/3039649876 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1f44066f30 0x7f1f440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:07.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.054+0000 7f1f61d9b700 1 --2- 172.21.15.7:0/3039649876 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1f44066f30 0x7f1f440693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1f58007fd0 tx=0x7f1f58009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:07.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.058+0000 7f1f527fc700 1 -- 172.21.15.7:0/3039649876 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f54051390 con 0x7f1f640c3ae0 2024-01-27T02:33:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.274+0000 7f1f68d07700 1 -- 172.21.15.7:0/3039649876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) v1 -- 0x7f1f6404c710 con 0x7f1f640c3ae0 2024-01-27T02:33:07.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.890+0000 7f1f527fc700 1 -- 172.21.15.7:0/3039649876 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(34..34 src has 1..34) v4 ==== 280+0+0 (secure 0 0 0) 0x7f1f54055750 con 0x7f1f640c3ae0 2024-01-27T02:33:07.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.890+0000 7f1f527fc700 1 -- 172.21.15.7:0/3039649876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=35}) v3 -- 0x7f1f4406f330 con 0x7f1f640c3ae0 2024-01-27T02:33:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.893+0000 7f1f527fc700 1 -- 172.21.15.7:0/3039649876 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v34) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1f540549b0 con 0x7f1f640c3ae0 2024-01-27T02:33:07.897 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:33:07.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.896+0000 7f1f68d07700 1 -- 172.21.15.7:0/3039649876 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1f44066f30 msgr2=0x7f1f440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:07.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.896+0000 7f1f68d07700 1 --2- 172.21.15.7:0/3039649876 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1f44066f30 0x7f1f440693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1f58007fd0 tx=0x7f1f58009040 comp rx=0 tx=0).stop 2024-01-27T02:33:07.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.896+0000 7f1f68d07700 1 -- 172.21.15.7:0/3039649876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f640c3ae0 msgr2=0x7f1f640b1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:07.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.896+0000 7f1f68d07700 1 --2- 172.21.15.7:0/3039649876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f640c3ae0 0x7f1f640b1b00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400c790 tx=0x7f1f54007ea0 comp rx=0 tx=0).stop 2024-01-27T02:33:07.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.896+0000 7f1f68d07700 1 -- 172.21.15.7:0/3039649876 shutdown_connections 2024-01-27T02:33:07.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.896+0000 7f1f68d07700 1 --2- 172.21.15.7:0/3039649876 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1f44066f30 0x7f1f440693e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:07.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.896+0000 7f1f68d07700 1 --2- 172.21.15.7:0/3039649876 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f64157ca0 0x7f1f641508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:07.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.896+0000 7f1f68d07700 1 --2- 172.21.15.7:0/3039649876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f640c3ae0 0x7f1f640b1b00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:07.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.896+0000 7f1f68d07700 1 --2- 172.21.15.7:0/3039649876 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f640cdbf0 0x7f1f6416a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:07.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.896+0000 7f1f68d07700 1 -- 172.21.15.7:0/3039649876 >> 172.21.15.7:0/3039649876 conn(0x7f1f64098020 msgr2=0x7f1f640cc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:07.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.896+0000 7f1f68d07700 1 -- 172.21.15.7:0/3039649876 shutdown_connections 2024-01-27T02:33:07.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:07.896+0000 7f1f68d07700 1 -- 172.21.15.7:0/3039649876 wait complete. 2024-01-27T02:33:07.899 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:33:07.900 INFO:teuthology.orchestra.run.smithi007.stderr:pauserd,pausewr is unset 2024-01-27T02:33:07.914 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:33:08.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.091+0000 7f9a93db4700 1 Processor -- start 2024-01-27T02:33:08.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.092+0000 7f9a93db4700 1 -- start start 2024-01-27T02:33:08.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.093+0000 7f9a93db4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9a8c157ca0 0x7f9a8c0c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:08.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.093+0000 7f9a93db4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8c0c3ae0 0x7f9a8c0cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:08.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.093+0000 7f9a93db4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a8c0ca020 con 0x7f9a8c158070 2024-01-27T02:33:08.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.093+0000 7f9a93db4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a8c0c4f40 con 0x7f9a8c157ca0 2024-01-27T02:33:08.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.093+0000 7f9a93db4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c0c5210 con 0x7f9a8c0c3ae0 2024-01-27T02:33:08.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.093+0000 7f9a91b50700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9a8c157ca0 0x7f9a8c0c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:08.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.093+0000 7f9a92351700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9a8c158070 0x7f9a8c15bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34138/0 (socket says 172.21.15.7:34138) 2024-01-27T02:33:08.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.093+0000 7f9a92351700 1 -- 172.21.15.7:0/3142231173 learned_addr learned my addr 172.21.15.7:0/3142231173 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:08.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.093+0000 7f9a9134f700 1 --2- 172.21.15.7:0/3142231173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8c0c3ae0 0x7f9a8c0cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:08.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.093+0000 7f9a91b50700 1 -- 172.21.15.7:0/3142231173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8c0c3ae0 msgr2=0x7f9a8c0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:08.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.093+0000 7f9a91b50700 1 --2- 172.21.15.7:0/3142231173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8c0c3ae0 0x7f9a8c0cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:08.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.093+0000 7f9a91b50700 1 -- 172.21.15.7:0/3142231173 >> v1:172.21.15.7:6789/0 conn(0x7f9a8c158070 legacy=0x7f9a8c15bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:08.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.093+0000 7f9a91b50700 1 -- 172.21.15.7:0/3142231173 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8c0c54f0 con 0x7f9a8c157ca0 2024-01-27T02:33:08.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.093+0000 7f9a90b4e700 1 -- 172.21.15.7:0/3142231173 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1878221663 0 0) 0x7f9a8c0ca020 con 0x7f9a8c158070 2024-01-27T02:33:08.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.093+0000 7f9a91b50700 1 --2- 172.21.15.7:0/3142231173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c157ca0 0x7f9a8c0c4a00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9a880098e0 tx=0x7f9a880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4f7809796953c7 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:08.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.094+0000 7f9a90b4e700 1 -- 172.21.15.7:0/3142231173 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88009e80 con 0x7f9a8c157ca0 2024-01-27T02:33:08.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.094+0000 7f9a90b4e700 1 -- 172.21.15.7:0/3142231173 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8800b860 con 0x7f9a8c157ca0 2024-01-27T02:33:08.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.094+0000 7f9a90b4e700 1 -- 172.21.15.7:0/3142231173 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8800bbe0 con 0x7f9a8c157ca0 2024-01-27T02:33:08.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.094+0000 7f9a93db4700 1 -- 172.21.15.7:0/3142231173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c157ca0 msgr2=0x7f9a8c0c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:08.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.094+0000 7f9a93db4700 1 --2- 172.21.15.7:0/3142231173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c157ca0 0x7f9a8c0c4a00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9a880098e0 tx=0x7f9a880091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:08.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.094+0000 7f9a93db4700 1 -- 172.21.15.7:0/3142231173 shutdown_connections 2024-01-27T02:33:08.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.094+0000 7f9a93db4700 1 --2- 172.21.15.7:0/3142231173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c157ca0 0x7f9a8c0c4a00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:08.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.094+0000 7f9a93db4700 1 --2- 172.21.15.7:0/3142231173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8c0c3ae0 0x7f9a8c0cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:08.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.094+0000 7f9a93db4700 1 -- 172.21.15.7:0/3142231173 >> 172.21.15.7:0/3142231173 conn(0x7f9a8c098020 msgr2=0x7f9a8c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:08.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.094+0000 7f9a93db4700 1 -- 172.21.15.7:0/3142231173 shutdown_connections 2024-01-27T02:33:08.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.094+0000 7f9a93db4700 1 -- 172.21.15.7:0/3142231173 wait complete. 2024-01-27T02:33:08.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.095+0000 7f9a93db4700 1 Processor -- start 2024-01-27T02:33:08.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.096+0000 7f9a93db4700 1 -- start start 2024-01-27T02:33:08.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.096+0000 7f9a93db4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c0c3ae0 0x7f9a8c0c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:08.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.096+0000 7f9a93db4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8c157ca0 0x7f9a8c14ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:08.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.096+0000 7f9a93db4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a8c158070 0x7f9a8c0b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:08.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.096+0000 7f9a93db4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c0b1b90 con 0x7f9a8c0c3ae0 2024-01-27T02:33:08.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.096+0000 7f9a93db4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c0c6640 con 0x7f9a8c158070 2024-01-27T02:33:08.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.096+0000 7f9a93db4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c0c6920 con 0x7f9a8c157ca0 2024-01-27T02:33:08.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.096+0000 7f9a91b50700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c0c3ae0 0x7f9a8c0c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:08.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.096+0000 7f9a91b50700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c0c3ae0 0x7f9a8c0c5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41098/0 (socket says 172.21.15.7:41098) 2024-01-27T02:33:08.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.096+0000 7f9a91b50700 1 -- 172.21.15.7:0/3332599050 learned_addr learned my addr 172.21.15.7:0/3332599050 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:08.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.097+0000 7f9a9134f700 1 --2- 172.21.15.7:0/3332599050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8c157ca0 0x7f9a8c14ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:08.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.097+0000 7f9a92351700 1 --2- 172.21.15.7:0/3332599050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a8c158070 0x7f9a8c0b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:08.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.097+0000 7f9a91b50700 1 -- 172.21.15.7:0/3332599050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8c157ca0 msgr2=0x7f9a8c14ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:08.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.097+0000 7f9a91b50700 1 --2- 172.21.15.7:0/3332599050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8c157ca0 0x7f9a8c14ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:08.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.097+0000 7f9a91b50700 1 -- 172.21.15.7:0/3332599050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a8c158070 msgr2=0x7f9a8c0b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:08.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.097+0000 7f9a91b50700 1 --2- 172.21.15.7:0/3332599050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a8c158070 0x7f9a8c0b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:08.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.097+0000 7f9a91b50700 1 -- 172.21.15.7:0/3332599050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a88009420 con 0x7f9a8c0c3ae0 2024-01-27T02:33:08.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.097+0000 7f9a92351700 1 --2- 172.21.15.7:0/3332599050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a8c158070 0x7f9a8c0b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:08.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.097+0000 7f9a91b50700 1 --2- 172.21.15.7:0/3332599050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c0c3ae0 0x7f9a8c0c5df0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c06ea80 tx=0x7f9a88017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:08.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.097+0000 7f9a81ffb700 1 -- 172.21.15.7:0/3332599050 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8800bbe0 con 0x7f9a8c0c3ae0 2024-01-27T02:33:08.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.097+0000 7f9a81ffb700 1 -- 172.21.15.7:0/3332599050 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a88011420 con 0x7f9a8c0c3ae0 2024-01-27T02:33:08.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.097+0000 7f9a93db4700 1 -- 172.21.15.7:0/3332599050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a8c0c6c00 con 0x7f9a8c0c3ae0 2024-01-27T02:33:08.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.097+0000 7f9a81ffb700 1 -- 172.21.15.7:0/3332599050 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88011800 con 0x7f9a8c0c3ae0 2024-01-27T02:33:08.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.097+0000 7f9a93db4700 1 -- 172.21.15.7:0/3332599050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8c0c6330 con 0x7f9a8c0c3ae0 2024-01-27T02:33:08.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.099+0000 7f9a93db4700 1 -- 172.21.15.7:0/3332599050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a8c04c710 con 0x7f9a8c0c3ae0 2024-01-27T02:33:08.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.099+0000 7f9a81ffb700 1 -- 172.21.15.7:0/3332599050 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9a88011bf0 con 0x7f9a8c0c3ae0 2024-01-27T02:33:08.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.099+0000 7f9a81ffb700 1 --2- 172.21.15.7:0/3332599050 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9a70066f30 0x7f9a700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:08.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.099+0000 7f9a81ffb700 1 -- 172.21.15.7:0/3332599050 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9a88085f10 con 0x7f9a8c0c3ae0 2024-01-27T02:33:08.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.100+0000 7f9a9134f700 1 --2- 172.21.15.7:0/3332599050 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9a70066f30 0x7f9a700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:08.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.100+0000 7f9a9134f700 1 --2- 172.21.15.7:0/3332599050 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9a70066f30 0x7f9a700693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0b18a0 tx=0x7f9a7c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.104+0000 7f9a81ffb700 1 -- 172.21.15.7:0/3332599050 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a88053180 con 0x7f9a8c0c3ae0 2024-01-27T02:33:08.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.307+0000 7f9a93db4700 1 -- 172.21.15.7:0/3332599050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a8c0c57d0 con 0x7f9a8c0c3ae0 2024-01-27T02:33:08.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.308+0000 7f9a81ffb700 1 -- 172.21.15.7:0/3332599050 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f9a880567a0 con 0x7f9a8c0c3ae0 2024-01-27T02:33:08.310 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:08.310 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:33:07.888617+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:32:52.897548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-27T02:32:53.822754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:33:08.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.310+0000 7f9a93db4700 1 -- 172.21.15.7:0/3332599050 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9a70066f30 msgr2=0x7f9a700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:08.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.310+0000 7f9a93db4700 1 --2- 172.21.15.7:0/3332599050 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9a70066f30 0x7f9a700693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0b18a0 tx=0x7f9a7c008040 comp rx=0 tx=0).stop 2024-01-27T02:33:08.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.310+0000 7f9a93db4700 1 -- 172.21.15.7:0/3332599050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c0c3ae0 msgr2=0x7f9a8c0c5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:08.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.310+0000 7f9a93db4700 1 --2- 172.21.15.7:0/3332599050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c0c3ae0 0x7f9a8c0c5df0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c06ea80 tx=0x7f9a88017000 comp rx=0 tx=0).stop 2024-01-27T02:33:08.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.310+0000 7f9a93db4700 1 -- 172.21.15.7:0/3332599050 shutdown_connections 2024-01-27T02:33:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.310+0000 7f9a93db4700 1 --2- 172.21.15.7:0/3332599050 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9a70066f30 0x7f9a700693e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.310+0000 7f9a93db4700 1 --2- 172.21.15.7:0/3332599050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a8c158070 0x7f9a8c0b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.310+0000 7f9a93db4700 1 --2- 172.21.15.7:0/3332599050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c0c3ae0 0x7f9a8c0c5df0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.310+0000 7f9a93db4700 1 --2- 172.21.15.7:0/3332599050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8c157ca0 0x7f9a8c14ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.310+0000 7f9a93db4700 1 -- 172.21.15.7:0/3332599050 >> 172.21.15.7:0/3332599050 conn(0x7f9a8c098020 msgr2=0x7f9a8c0ca9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.310+0000 7f9a93db4700 1 -- 172.21.15.7:0/3332599050 shutdown_connections 2024-01-27T02:33:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.310+0000 7f9a93db4700 1 -- 172.21.15.7:0/3332599050 wait complete. 2024-01-27T02:33:08.326 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-01-27T02:33:08.326 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-01-27T02:33:08.326 DEBUG:teuthology.orchestra.run.smithi007:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-01-27T02:33:08.326 DEBUG:teuthology.orchestra.run.smithi007:> os.fchmod(fd, 0O666) 2024-01-27T02:33:08.326 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(fd) 2024-01-27T02:33:08.326 DEBUG:teuthology.orchestra.run.smithi007:> os.close(fd) 2024-01-27T02:33:08.326 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-01-27T02:33:08.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:33:08.589 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-27T02:33:08.672 INFO:teuthology.orchestra.run.smithi007.stdout:client5038 2024-01-27T02:33:08.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:08.673 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/17e1bcca-e647-4a58-b0ac-7d39d8508814.client5038/osdmap') 2024-01-27T02:33:08.767 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 32 barrier 32 flags 0x588000 2024-01-27T02:33:08.767 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:33:08.767 INFO:teuthology.orchestra.run.smithi007.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:33:08.768 INFO:teuthology.orchestra.run.smithi007.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:33:08.768 INFO:teuthology.orchestra.run.smithi007.stdout:osd0 (1)172.21.15.7:6817 100% (exists, up) 100% -1 2024-01-27T02:33:08.768 INFO:teuthology.orchestra.run.smithi007.stdout:osd1 (1)172.21.15.7:6801 100% (exists, up) 100% -1 2024-01-27T02:33:08.768 INFO:teuthology.orchestra.run.smithi007.stdout:osd2 (1)172.21.15.7:6825 100% (exists, up) 100% -1 2024-01-27T02:33:08.768 INFO:teuthology.orchestra.run.smithi007.stdout:osd3 (1)172.21.15.7:6809 100% (exists, up) 100% -1 2024-01-27T02:33:08.768 INFO:teuthology.orchestra.run.smithi007.stdout:osd4 (1)172.21.15.188:6803 100% (exists, up) 100% -1 2024-01-27T02:33:08.768 INFO:teuthology.orchestra.run.smithi007.stdout:osd5 (1)172.21.15.188:6802 100% (exists, up) 100% -1 2024-01-27T02:33:08.768 INFO:teuthology.orchestra.run.smithi007.stdout:osd6 (1)172.21.15.188:6825 100% (exists, up) 100% -1 2024-01-27T02:33:08.768 INFO:teuthology.orchestra.run.smithi007.stdout:osd7 (1)172.21.15.188:6817 100% (exists, up) 100% -1 2024-01-27T02:33:08.768 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records in 2024-01-27T02:33:08.768 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records out 2024-01-27T02:33:08.769 INFO:teuthology.orchestra.run.smithi007.stderr:791 bytes copied, 4.7855e-05 s, 16.5 MB/s 2024-01-27T02:33:08.770 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:33:08.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.907+0000 7fc2aab8b700 1 Processor -- start 2024-01-27T02:33:08.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.908+0000 7fc2aab8b700 1 -- start start 2024-01-27T02:33:08.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.908+0000 7fc2aab8b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc2a40a7fe0 0x7fc2a406f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:08.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.908+0000 7fc2aab8b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a415a880 con 0x7fc2a415dfb0 2024-01-27T02:33:08.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.908+0000 7fc2aab8b700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a40af950 con 0x7fc2a40a7c10 2024-01-27T02:33:08.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.908+0000 7fc2aab8b700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a40afde0 con 0x7fc2a40a7fe0 2024-01-27T02:33:08.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.909+0000 7fc2a9128700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc2a415dfb0 0x7fc2a40b0180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34146/0 (socket says 172.21.15.7:34146) 2024-01-27T02:33:08.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.909+0000 7fc2a9128700 1 -- 172.21.15.7:0/2270906676 learned_addr learned my addr 172.21.15.7:0/2270906676 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:08.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.909+0000 7fc2a3fff700 1 --2- 172.21.15.7:0/2270906676 >> v2:172.21.15.188:3300/0 conn(0x7fc2a40a7fe0 0x7fc2a406f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:08.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.909+0000 7fc2a37fe700 1 -- 172.21.15.7:0/2270906676 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663041499 0 0) 0x7fc2a415a880 con 0x7fc2a415dfb0 2024-01-27T02:33:08.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.909+0000 7fc2a37fe700 1 -- 172.21.15.7:0/2270906676 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc290003570 con 0x7fc2a415dfb0 2024-01-27T02:33:08.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.909+0000 7fc2a37fe700 1 -- 172.21.15.7:0/2270906676 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3835060690 0 0) 0x7fc2a40af950 con 0x7fc2a40a7c10 2024-01-27T02:33:08.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.909+0000 7fc2a37fe700 1 -- 172.21.15.7:0/2270906676 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2a415a880 con 0x7fc2a40a7c10 2024-01-27T02:33:08.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.909+0000 7fc2a3fff700 1 -- 172.21.15.7:0/2270906676 >> v1:172.21.15.188:6789/0 conn(0x7fc2a40a7c10 legacy=0x7fc2a40ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:08.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.909+0000 7fc2a37fe700 1 -- 172.21.15.7:0/2270906676 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 23785139 0 0) 0x7fc290003570 con 0x7fc2a415dfb0 2024-01-27T02:33:08.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.909+0000 7fc2a3fff700 1 -- 172.21.15.7:0/2270906676 >> v1:172.21.15.7:6789/0 conn(0x7fc2a415dfb0 legacy=0x7fc2a40b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:08.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.909+0000 7fc2a3fff700 1 -- 172.21.15.7:0/2270906676 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a40aed40 con 0x7fc2a40a7fe0 2024-01-27T02:33:08.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.909+0000 7fc2a3fff700 1 --2- 172.21.15.7:0/2270906676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2a40a7fe0 0x7fc2a406f360 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0098e0 tx=0x7fc28c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2686a7100a95bdb server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:08.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.910+0000 7fc2a37fe700 1 -- 172.21.15.7:0/2270906676 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0017f0 con 0x7fc2a40a7fe0 2024-01-27T02:33:08.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.910+0000 7fc2a37fe700 1 -- 172.21.15.7:0/2270906676 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c011420 con 0x7fc2a40a7fe0 2024-01-27T02:33:08.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.910+0000 7fc2a37fe700 1 -- 172.21.15.7:0/2270906676 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c0117c0 con 0x7fc2a40a7fe0 2024-01-27T02:33:08.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.910+0000 7fc2aab8b700 1 -- 172.21.15.7:0/2270906676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2a40a7fe0 msgr2=0x7fc2a406f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:08.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.910+0000 7fc2aab8b700 1 --2- 172.21.15.7:0/2270906676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2a40a7fe0 0x7fc2a406f360 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0098e0 tx=0x7fc28c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:08.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.910+0000 7fc2aab8b700 1 -- 172.21.15.7:0/2270906676 shutdown_connections 2024-01-27T02:33:08.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.910+0000 7fc2aab8b700 1 --2- 172.21.15.7:0/2270906676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2a40a7fe0 0x7fc2a406f360 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:08.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.910+0000 7fc2aab8b700 1 -- 172.21.15.7:0/2270906676 >> 172.21.15.7:0/2270906676 conn(0x7fc2a40679b0 msgr2=0x7fc2a4067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:08.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.910+0000 7fc2aab8b700 1 -- 172.21.15.7:0/2270906676 shutdown_connections 2024-01-27T02:33:08.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.910+0000 7fc2aab8b700 1 -- 172.21.15.7:0/2270906676 wait complete. 2024-01-27T02:33:08.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.912+0000 7fc2aab8b700 1 Processor -- start 2024-01-27T02:33:08.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.912+0000 7fc2aab8b700 1 -- start start 2024-01-27T02:33:08.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.913+0000 7fc2aab8b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2a40a7c10 0x7fc2a414ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:08.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.913+0000 7fc2aab8b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a40a7fe0 0x7fc2a4151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:08.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.913+0000 7fc2aab8b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2a415dfb0 0x7fc2a41506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:08.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.913+0000 7fc2aab8b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a4154a10 con 0x7fc2a40a7fe0 2024-01-27T02:33:08.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.913+0000 7fc2aab8b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a4154c80 con 0x7fc2a415dfb0 2024-01-27T02:33:08.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.913+0000 7fc2aab8b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a4154f60 con 0x7fc2a40a7c10 2024-01-27T02:33:08.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.913+0000 7fc2a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a40a7fe0 0x7fc2a4151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:08.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.913+0000 7fc2a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a40a7fe0 0x7fc2a4151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41114/0 (socket says 172.21.15.7:41114) 2024-01-27T02:33:08.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.913+0000 7fc2a3fff700 1 -- 172.21.15.7:0/2228612810 learned_addr learned my addr 172.21.15.7:0/2228612810 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:08.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.913+0000 7fc2a9128700 1 --2- 172.21.15.7:0/2228612810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2a415dfb0 0x7fc2a41506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:08.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.913+0000 7fc2a8927700 1 --2- 172.21.15.7:0/2228612810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2a40a7c10 0x7fc2a414ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:08.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.913+0000 7fc2a3fff700 1 -- 172.21.15.7:0/2228612810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2a40a7c10 msgr2=0x7fc2a414ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:08.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.913+0000 7fc2a3fff700 1 --2- 172.21.15.7:0/2228612810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2a40a7c10 0x7fc2a414ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:08.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.913+0000 7fc2a3fff700 1 -- 172.21.15.7:0/2228612810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2a415dfb0 msgr2=0x7fc2a41506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:08.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.913+0000 7fc2a3fff700 1 --2- 172.21.15.7:0/2228612810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2a415dfb0 0x7fc2a41506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:08.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.913+0000 7fc2a3fff700 1 -- 172.21.15.7:0/2228612810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28c009420 con 0x7fc2a40a7fe0 2024-01-27T02:33:08.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.913+0000 7fc2a3fff700 1 --2- 172.21.15.7:0/2228612810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a40a7fe0 0x7fc2a4151150 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc28c002000 tx=0x7fc28c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:08.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.913+0000 7fc2a0ff9700 1 -- 172.21.15.7:0/2228612810 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c01cb30 con 0x7fc2a40a7fe0 2024-01-27T02:33:08.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.914+0000 7fc2a0ff9700 1 -- 172.21.15.7:0/2228612810 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c01cc90 con 0x7fc2a40a7fe0 2024-01-27T02:33:08.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.914+0000 7fc2aab8b700 1 -- 172.21.15.7:0/2228612810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a4155240 con 0x7fc2a40a7fe0 2024-01-27T02:33:08.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.914+0000 7fc2a0ff9700 1 -- 172.21.15.7:0/2228612810 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c01a6a0 con 0x7fc2a40a7fe0 2024-01-27T02:33:08.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.914+0000 7fc2aab8b700 1 -- 172.21.15.7:0/2228612810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a414f470 con 0x7fc2a40a7fe0 2024-01-27T02:33:08.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.915+0000 7fc2aab8b700 1 -- 172.21.15.7:0/2228612810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a4155740 con 0x7fc2a40a7fe0 2024-01-27T02:33:08.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.915+0000 7fc2a0ff9700 1 -- 172.21.15.7:0/2228612810 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc28c01aa50 con 0x7fc2a40a7fe0 2024-01-27T02:33:08.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.916+0000 7fc2a0ff9700 1 --2- 172.21.15.7:0/2228612810 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc284066f30 0x7fc2840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:08.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.916+0000 7fc2a0ff9700 1 -- 172.21.15.7:0/2228612810 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc28c085cf0 con 0x7fc2a40a7fe0 2024-01-27T02:33:08.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.916+0000 7fc2a8927700 1 --2- 172.21.15.7:0/2228612810 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc284066f30 0x7fc2840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:08.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.916+0000 7fc2a8927700 1 --2- 172.21.15.7:0/2228612810 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc284066f30 0x7fc2840693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc294007fd0 tx=0x7fc294007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:08.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:08.920+0000 7fc2a0ff9700 1 -- 172.21.15.7:0/2228612810 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc28c08a070 con 0x7fc2a40a7fe0 2024-01-27T02:33:09.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:09.168+0000 7fc2aab8b700 1 -- 172.21.15.7:0/2228612810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2a404c710 con 0x7fc2a40a7fe0 2024-01-27T02:33:09.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:09.169+0000 7fc2a0ff9700 1 -- 172.21.15.7:0/2228612810 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4594 (secure 0 0 0) 0x7fc28c052f60 con 0x7fc2a40a7fe0 2024-01-27T02:33:09.170 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:09.170 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4864,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/892460188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":892460188},{"type":"v1","addr":"172.21.15.188:6837","nonce":892460188}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4867,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1657319349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1657319349},{"type":"v1","addr":"172.21.15.188:6836","nonce":1657319349}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4879,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2190571534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2190571534},{"type":"v1","addr":"172.21.15.7:6834","nonce":2190571534}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:32:54.849803+0000","modified":"2024-01-27T02:32:58.809118+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4882},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4882":{"gid":4882,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6835/1734594727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1734594727},{"type":"v1","addr":"172.21.15.7:6835","nonce":1734594727}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-01-27T02:33:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:09.171+0000 7fc2aab8b700 1 -- 172.21.15.7:0/2228612810 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc284066f30 msgr2=0x7fc2840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:09.171+0000 7fc2aab8b700 1 --2- 172.21.15.7:0/2228612810 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc284066f30 0x7fc2840693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc294007fd0 tx=0x7fc294007d50 comp rx=0 tx=0).stop 2024-01-27T02:33:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:09.171+0000 7fc2aab8b700 1 -- 172.21.15.7:0/2228612810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a40a7fe0 msgr2=0x7fc2a4151150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:09.171+0000 7fc2aab8b700 1 --2- 172.21.15.7:0/2228612810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a40a7fe0 0x7fc2a4151150 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc28c002000 tx=0x7fc28c016000 comp rx=0 tx=0).stop 2024-01-27T02:33:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:09.171+0000 7fc2aab8b700 1 -- 172.21.15.7:0/2228612810 shutdown_connections 2024-01-27T02:33:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:09.171+0000 7fc2aab8b700 1 --2- 172.21.15.7:0/2228612810 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc284066f30 0x7fc2840693e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:09.171+0000 7fc2aab8b700 1 --2- 172.21.15.7:0/2228612810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2a415dfb0 0x7fc2a41506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:09.171+0000 7fc2aab8b700 1 --2- 172.21.15.7:0/2228612810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2a40a7c10 0x7fc2a414ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:09.171+0000 7fc2aab8b700 1 --2- 172.21.15.7:0/2228612810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a40a7fe0 0x7fc2a4151150 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:09.171+0000 7fc2aab8b700 1 -- 172.21.15.7:0/2228612810 >> 172.21.15.7:0/2228612810 conn(0x7fc2a40679b0 msgr2=0x7fc2a40b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:09.171+0000 7fc2aab8b700 1 -- 172.21.15.7:0/2228612810 shutdown_connections 2024-01-27T02:33:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:09.171+0000 7fc2aab8b700 1 -- 172.21.15.7:0/2228612810 wait complete. 2024-01-27T02:33:09.174 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2024-01-27T02:33:09.186 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json osdmap barrier 34 2024-01-27T02:33:09.336 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2024-01-27T02:33:09.337 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-01-27T02:33:09.337 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-01-27T02:33:09.337 DEBUG:teuthology.orchestra.run.smithi007:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2024-01-27T02:33:09.337 DEBUG:teuthology.orchestra.run.smithi007:> os.fchmod(fd, 0O666) 2024-01-27T02:33:09.337 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(fd) 2024-01-27T02:33:09.337 DEBUG:teuthology.orchestra.run.smithi007:> os.close(fd) 2024-01-27T02:33:09.337 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-01-27T02:33:09.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:33:09.600 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-27T02:33:09.685 INFO:teuthology.orchestra.run.smithi007.stdout:client5038 2024-01-27T02:33:09.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:09.686 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/17e1bcca-e647-4a58-b0ac-7d39d8508814.client5038/osdmap') 2024-01-27T02:33:09.779 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 34 barrier 34 flags 0x588000 2024-01-27T02:33:09.780 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:33:09.780 INFO:teuthology.orchestra.run.smithi007.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:33:09.780 INFO:teuthology.orchestra.run.smithi007.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:33:09.780 INFO:teuthology.orchestra.run.smithi007.stdout:osd0 (1)172.21.15.7:6817 100% (exists, up) 100% -1 2024-01-27T02:33:09.780 INFO:teuthology.orchestra.run.smithi007.stdout:osd1 (1)172.21.15.7:6801 100% (exists, up) 100% -1 2024-01-27T02:33:09.780 INFO:teuthology.orchestra.run.smithi007.stdout:osd2 (1)172.21.15.7:6825 100% (exists, up) 100% -1 2024-01-27T02:33:09.780 INFO:teuthology.orchestra.run.smithi007.stdout:osd3 (1)172.21.15.7:6809 100% (exists, up) 100% -1 2024-01-27T02:33:09.780 INFO:teuthology.orchestra.run.smithi007.stdout:osd4 (1)172.21.15.188:6803 100% (exists, up) 100% -1 2024-01-27T02:33:09.780 INFO:teuthology.orchestra.run.smithi007.stdout:osd5 (1)172.21.15.188:6802 100% (exists, up) 100% -1 2024-01-27T02:33:09.781 INFO:teuthology.orchestra.run.smithi007.stdout:osd6 (1)172.21.15.188:6825 100% (exists, up) 100% -1 2024-01-27T02:33:09.781 INFO:teuthology.orchestra.run.smithi007.stdout:osd7 (1)172.21.15.188:6817 100% (exists, up) 100% -1 2024-01-27T02:33:09.781 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records in 2024-01-27T02:33:09.781 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records out 2024-01-27T02:33:09.782 INFO:teuthology.orchestra.run.smithi007.stderr:791 bytes copied, 4.6911e-05 s, 16.9 MB/s 2024-01-27T02:33:09.782 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:33:09.782 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-27T02:33:09.824 INFO:teuthology.orchestra.run.smithi007.stdout:client5038 2024-01-27T02:33:09.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:09.824 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/17e1bcca-e647-4a58-b0ac-7d39d8508814.client5038/osdmap') 2024-01-27T02:33:09.918 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 34 barrier 34 flags 0x588000 2024-01-27T02:33:09.918 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:33:09.918 INFO:teuthology.orchestra.run.smithi007.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:33:09.918 INFO:teuthology.orchestra.run.smithi007.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:33:09.918 INFO:teuthology.orchestra.run.smithi007.stdout:osd0 (1)172.21.15.7:6817 100% (exists, up) 100% -1 2024-01-27T02:33:09.918 INFO:teuthology.orchestra.run.smithi007.stdout:osd1 (1)172.21.15.7:6801 100% (exists, up) 100% -1 2024-01-27T02:33:09.918 INFO:teuthology.orchestra.run.smithi007.stdout:osd2 (1)172.21.15.7:6825 100% (exists, up) 100% -1 2024-01-27T02:33:09.918 INFO:teuthology.orchestra.run.smithi007.stdout:osd3 (1)172.21.15.7:6809 100% (exists, up) 100% -1 2024-01-27T02:33:09.918 INFO:teuthology.orchestra.run.smithi007.stdout:osd4 (1)172.21.15.188:6803 100% (exists, up) 100% -1 2024-01-27T02:33:09.918 INFO:teuthology.orchestra.run.smithi007.stdout:osd5 (1)172.21.15.188:6802 100% (exists, up) 100% -1 2024-01-27T02:33:09.918 INFO:teuthology.orchestra.run.smithi007.stdout:osd6 (1)172.21.15.188:6825 100% (exists, up) 100% -1 2024-01-27T02:33:09.919 INFO:teuthology.orchestra.run.smithi007.stdout:osd7 (1)172.21.15.188:6817 100% (exists, up) 100% -1 2024-01-27T02:33:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records in 2024-01-27T02:33:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records out 2024-01-27T02:33:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:791 bytes copied, 4.3555e-05 s, 18.2 MB/s 2024-01-27T02:33:09.920 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-01-27T02:33:09.920 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-27T02:33:09.920 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:33:09.973 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:33:10.022 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-27T02:33:10.022 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-27T02:33:10.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:33:10.022 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-27T02:33:10.094 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:33:10.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:10.094 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-27T02:33:10.116 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-27T02:33:10.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:33:10.117 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-01-27T02:33:10.166 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:33:10.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:10.166 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-27T02:33:10.184 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:33:10.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.322+0000 7f115236d700 1 Processor -- start 2024-01-27T02:33:10.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.322+0000 7f115236d700 1 -- start start 2024-01-27T02:33:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.323+0000 7f115236d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f114c156fd0 0x7f114c0c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.323+0000 7f115236d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f114c0c9830 0x7f114c0c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.323+0000 7f115236d700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f114c06dbb0 con 0x7f114c156c00 2024-01-27T02:33:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.323+0000 7f115236d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f114c0c8bc0 con 0x7f114c156fd0 2024-01-27T02:33:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.323+0000 7f115236d700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f114c0c9c50 con 0x7f114c0c9830 2024-01-27T02:33:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.323+0000 7f114b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f114c156fd0 0x7f114c0c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.323+0000 7f114b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f114c156fd0 0x7f114c0c92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41130/0 (socket says 172.21.15.7:41130) 2024-01-27T02:33:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.323+0000 7f114b7fe700 1 -- 172.21.15.7:0/3475148155 learned_addr learned my addr 172.21.15.7:0/3475148155 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.323+0000 7f115090a700 1 --2- 172.21.15.7:0/3475148155 >> v2:172.21.15.188:3300/0 conn(0x7f114c0c9830 0x7f114c0c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.323+0000 7f114b7fe700 1 -- 172.21.15.7:0/3475148155 >> v2:172.21.15.188:3300/0 conn(0x7f114c0c9830 msgr2=0x7f114c0c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.323+0000 7f114b7fe700 1 --2- 172.21.15.7:0/3475148155 >> v2:172.21.15.188:3300/0 conn(0x7f114c0c9830 0x7f114c0c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.323+0000 7f114b7fe700 1 -- 172.21.15.7:0/3475148155 >> v1:172.21.15.188:6789/0 conn(0x7f114c156c00 legacy=0x7f114c06e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:10.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.323+0000 7f114b7fe700 1 -- 172.21.15.7:0/3475148155 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114c06e750 con 0x7f114c156fd0 2024-01-27T02:33:10.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.324+0000 7f114b7fe700 1 --2- 172.21.15.7:0/3475148155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c156fd0 0x7f114c0c92f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f114c0ca170 tx=0x7f1140009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6456f89c825da1e8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:10.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.324+0000 7f114affd700 1 -- 172.21.15.7:0/3475148155 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114000b860 con 0x7f114c156fd0 2024-01-27T02:33:10.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.324+0000 7f114affd700 1 -- 172.21.15.7:0/3475148155 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f114000b9c0 con 0x7f114c156fd0 2024-01-27T02:33:10.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.324+0000 7f114affd700 1 -- 172.21.15.7:0/3475148155 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114000bd60 con 0x7f114c156fd0 2024-01-27T02:33:10.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.324+0000 7f115236d700 1 -- 172.21.15.7:0/3475148155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c156fd0 msgr2=0x7f114c0c92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:10.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.324+0000 7f115236d700 1 --2- 172.21.15.7:0/3475148155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c156fd0 0x7f114c0c92f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f114c0ca170 tx=0x7f1140009250 comp rx=0 tx=0).stop 2024-01-27T02:33:10.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.324+0000 7f115236d700 1 -- 172.21.15.7:0/3475148155 shutdown_connections 2024-01-27T02:33:10.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.324+0000 7f115236d700 1 --2- 172.21.15.7:0/3475148155 >> v2:172.21.15.188:3300/0 conn(0x7f114c0c9830 0x7f114c0c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:10.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.324+0000 7f115236d700 1 --2- 172.21.15.7:0/3475148155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c156fd0 0x7f114c0c92f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:10.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.324+0000 7f115236d700 1 -- 172.21.15.7:0/3475148155 >> 172.21.15.7:0/3475148155 conn(0x7f114c063580 msgr2=0x7f114c063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:10.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.324+0000 7f115236d700 1 -- 172.21.15.7:0/3475148155 shutdown_connections 2024-01-27T02:33:10.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.324+0000 7f115236d700 1 -- 172.21.15.7:0/3475148155 wait complete. 2024-01-27T02:33:10.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.326+0000 7f115236d700 1 Processor -- start 2024-01-27T02:33:10.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.326+0000 7f115236d700 1 -- start start 2024-01-27T02:33:10.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.327+0000 7f115236d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f114c0c9830 0x7f114c0b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:10.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.327+0000 7f115236d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c156c00 0x7f114c14cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:10.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.327+0000 7f115236d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f114c156fd0 0x7f114c151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:10.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.327+0000 7f115236d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c158000 con 0x7f114c156c00 2024-01-27T02:33:10.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.327+0000 7f115236d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c158190 con 0x7f114c0c9830 2024-01-27T02:33:10.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.327+0000 7f115236d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114c0b1570 con 0x7f114c156fd0 2024-01-27T02:33:10.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.327+0000 7f114b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c156c00 0x7f114c14cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:10.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.327+0000 7f114b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c156c00 0x7f114c14cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41134/0 (socket says 172.21.15.7:41134) 2024-01-27T02:33:10.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.327+0000 7f114b7fe700 1 -- 172.21.15.7:0/96664717 learned_addr learned my addr 172.21.15.7:0/96664717 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:10.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.327+0000 7f114bfff700 1 --2- 172.21.15.7:0/96664717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f114c0c9830 0x7f114c0b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:10.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.327+0000 7f115090a700 1 --2- 172.21.15.7:0/96664717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f114c156fd0 0x7f114c151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:10.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.327+0000 7f114b7fe700 1 -- 172.21.15.7:0/96664717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f114c156fd0 msgr2=0x7f114c151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:10.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.327+0000 7f114b7fe700 1 --2- 172.21.15.7:0/96664717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f114c156fd0 0x7f114c151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:10.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.327+0000 7f114b7fe700 1 -- 172.21.15.7:0/96664717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f114c0c9830 msgr2=0x7f114c0b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:10.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.327+0000 7f114b7fe700 1 --2- 172.21.15.7:0/96664717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f114c0c9830 0x7f114c0b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:10.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.327+0000 7f114b7fe700 1 -- 172.21.15.7:0/96664717 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11400094f0 con 0x7f114c156c00 2024-01-27T02:33:10.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.327+0000 7f114b7fe700 1 --2- 172.21.15.7:0/96664717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c156c00 0x7f114c14cd60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f114c0cc9a0 tx=0x7f1140017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:10.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.327+0000 7f1133fff700 1 -- 172.21.15.7:0/96664717 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1140024440 con 0x7f114c156c00 2024-01-27T02:33:10.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.328+0000 7f1133fff700 1 -- 172.21.15.7:0/96664717 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11400245a0 con 0x7f114c156c00 2024-01-27T02:33:10.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.328+0000 7f115236d700 1 -- 172.21.15.7:0/96664717 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f114c0b1850 con 0x7f114c156c00 2024-01-27T02:33:10.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.328+0000 7f1133fff700 1 -- 172.21.15.7:0/96664717 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1140024960 con 0x7f114c156c00 2024-01-27T02:33:10.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.328+0000 7f115236d700 1 -- 172.21.15.7:0/96664717 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114c0b1e10 con 0x7f114c156c00 2024-01-27T02:33:10.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.328+0000 7f115236d700 1 -- 172.21.15.7:0/96664717 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f114c158300 con 0x7f114c156c00 2024-01-27T02:33:10.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.329+0000 7f1133fff700 1 -- 172.21.15.7:0/96664717 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f114001a420 con 0x7f114c156c00 2024-01-27T02:33:10.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.330+0000 7f1133fff700 1 --2- 172.21.15.7:0/96664717 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1138066f30 0x7f11380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:10.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.330+0000 7f1133fff700 1 -- 172.21.15.7:0/96664717 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f11400863a0 con 0x7f114c156c00 2024-01-27T02:33:10.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.330+0000 7f114bfff700 1 --2- 172.21.15.7:0/96664717 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1138066f30 0x7f11380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:10.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.330+0000 7f114bfff700 1 --2- 172.21.15.7:0/96664717 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1138066f30 0x7f11380693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f113c003a80 tx=0x7f113c007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:10.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.334+0000 7f1133fff700 1 -- 172.21.15.7:0/96664717 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1140053570 con 0x7f114c156c00 2024-01-27T02:33:10.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.588+0000 7f115236d700 1 -- 172.21.15.7:0/96664717 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f114c04c710 con 0x7f114c156c00 2024-01-27T02:33:10.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.588+0000 7f1133fff700 1 -- 172.21.15.7:0/96664717 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f1140056b90 con 0x7f114c156c00 2024-01-27T02:33:10.590 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:10.590 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4864,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/892460188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":892460188},{"type":"v1","addr":"172.21.15.188:6837","nonce":892460188}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4867,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1657319349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1657319349},{"type":"v1","addr":"172.21.15.188:6836","nonce":1657319349}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4879,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2190571534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2190571534},{"type":"v1","addr":"172.21.15.7:6834","nonce":2190571534}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:32:54.849803+0000","modified":"2024-01-27T02:32:58.809118+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4882},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4882":{"gid":4882,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6835/1734594727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1734594727},{"type":"v1","addr":"172.21.15.7:6835","nonce":1734594727}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-01-27T02:33:10.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.591+0000 7f115236d700 1 -- 172.21.15.7:0/96664717 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1138066f30 msgr2=0x7f11380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:10.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.591+0000 7f115236d700 1 --2- 172.21.15.7:0/96664717 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1138066f30 0x7f11380693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f113c003a80 tx=0x7f113c007ec0 comp rx=0 tx=0).stop 2024-01-27T02:33:10.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.591+0000 7f115236d700 1 -- 172.21.15.7:0/96664717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c156c00 msgr2=0x7f114c14cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:10.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.591+0000 7f115236d700 1 --2- 172.21.15.7:0/96664717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c156c00 0x7f114c14cd60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f114c0cc9a0 tx=0x7f1140017000 comp rx=0 tx=0).stop 2024-01-27T02:33:10.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.591+0000 7f115236d700 1 -- 172.21.15.7:0/96664717 shutdown_connections 2024-01-27T02:33:10.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.591+0000 7f115236d700 1 --2- 172.21.15.7:0/96664717 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1138066f30 0x7f11380693e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:10.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.591+0000 7f115236d700 1 --2- 172.21.15.7:0/96664717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f114c156fd0 0x7f114c151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:10.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.591+0000 7f115236d700 1 --2- 172.21.15.7:0/96664717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f114c0c9830 0x7f114c0b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:10.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.591+0000 7f115236d700 1 --2- 172.21.15.7:0/96664717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114c156c00 0x7f114c14cd60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:10.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.591+0000 7f115236d700 1 -- 172.21.15.7:0/96664717 >> 172.21.15.7:0/96664717 conn(0x7f114c063580 msgr2=0x7f114c0cbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:10.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.591+0000 7f115236d700 1 -- 172.21.15.7:0/96664717 shutdown_connections 2024-01-27T02:33:10.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.591+0000 7f115236d700 1 -- 172.21.15.7:0/96664717 wait complete. 2024-01-27T02:33:10.593 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2024-01-27T02:33:10.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:33:10.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.735+0000 7f760496e700 1 Processor -- start 2024-01-27T02:33:10.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.735+0000 7f760496e700 1 -- start start 2024-01-27T02:33:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.736+0000 7f760496e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7600157ca0 0x7f76000c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.736+0000 7f760496e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f76000c3ae0 0x7f76000cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.737+0000 7f760496e700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76000ca020 con 0x7f7600158070 2024-01-27T02:33:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.737+0000 7f760496e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76000c4f40 con 0x7f7600157ca0 2024-01-27T02:33:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.737+0000 7f760496e700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76000c5210 con 0x7f76000c3ae0 2024-01-27T02:33:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.737+0000 7f75fe59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7600157ca0 0x7f76000c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.737+0000 7f75fe59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7600157ca0 0x7f76000c4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41138/0 (socket says 172.21.15.7:41138) 2024-01-27T02:33:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.737+0000 7f75fe59c700 1 -- 172.21.15.7:0/2825108611 learned_addr learned my addr 172.21.15.7:0/2825108611 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.737+0000 7f75fdd9b700 1 --2- 172.21.15.7:0/2825108611 >> v2:172.21.15.188:3300/0 conn(0x7f76000c3ae0 0x7f76000cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.737+0000 7f75fe59c700 1 -- 172.21.15.7:0/2825108611 >> v2:172.21.15.188:3300/0 conn(0x7f76000c3ae0 msgr2=0x7f76000cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.737+0000 7f75fe59c700 1 --2- 172.21.15.7:0/2825108611 >> v2:172.21.15.188:3300/0 conn(0x7f76000c3ae0 0x7f76000cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.737+0000 7f75fe59c700 1 -- 172.21.15.7:0/2825108611 >> v1:172.21.15.188:6789/0 conn(0x7f7600158070 legacy=0x7f760015bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.737+0000 7f75fe59c700 1 -- 172.21.15.7:0/2825108611 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76000c54f0 con 0x7f7600157ca0 2024-01-27T02:33:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.737+0000 7f75fe59c700 1 --2- 172.21.15.7:0/2825108611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7600157ca0 0x7f76000c4a00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f760006e170 tx=0x7f75f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59d9e3f278dc949 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.737+0000 7f75fd59a700 1 -- 172.21.15.7:0/2825108611 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0009e70 con 0x7f7600157ca0 2024-01-27T02:33:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.737+0000 7f75fd59a700 1 -- 172.21.15.7:0/2825108611 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f000b860 con 0x7f7600157ca0 2024-01-27T02:33:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.737+0000 7f75fd59a700 1 -- 172.21.15.7:0/2825108611 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f000bc00 con 0x7f7600157ca0 2024-01-27T02:33:10.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.738+0000 7f760496e700 1 -- 172.21.15.7:0/2825108611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7600157ca0 msgr2=0x7f76000c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:10.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.738+0000 7f760496e700 1 --2- 172.21.15.7:0/2825108611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7600157ca0 0x7f76000c4a00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f760006e170 tx=0x7f75f0009250 comp rx=0 tx=0).stop 2024-01-27T02:33:10.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.738+0000 7f760496e700 1 -- 172.21.15.7:0/2825108611 shutdown_connections 2024-01-27T02:33:10.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.738+0000 7f760496e700 1 --2- 172.21.15.7:0/2825108611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7600157ca0 0x7f76000c4a00 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:10.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.738+0000 7f760496e700 1 --2- 172.21.15.7:0/2825108611 >> v2:172.21.15.188:3300/0 conn(0x7f76000c3ae0 0x7f76000cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:10.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.738+0000 7f760496e700 1 -- 172.21.15.7:0/2825108611 >> 172.21.15.7:0/2825108611 conn(0x7f7600098020 msgr2=0x7f76000711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:10.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.738+0000 7f760496e700 1 -- 172.21.15.7:0/2825108611 shutdown_connections 2024-01-27T02:33:10.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.738+0000 7f760496e700 1 -- 172.21.15.7:0/2825108611 wait complete. 2024-01-27T02:33:10.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.739+0000 7f760496e700 1 Processor -- start 2024-01-27T02:33:10.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.740+0000 7f760496e700 1 -- start start 2024-01-27T02:33:10.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.740+0000 7f760496e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76000c3ae0 0x7f760014b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:10.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.740+0000 7f760496e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7600157ca0 0x7f7600150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:10.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.740+0000 7f760496e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7600158070 0x7f76000b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:10.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.740+0000 7f760496e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760014bdd0 con 0x7f76000c3ae0 2024-01-27T02:33:10.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.740+0000 7f760496e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76000b26f0 con 0x7f7600158070 2024-01-27T02:33:10.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.740+0000 7f760496e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76000b1b00 con 0x7f7600157ca0 2024-01-27T02:33:10.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.740+0000 7f75fe59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76000c3ae0 0x7f760014b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:10.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.740+0000 7f75fe59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76000c3ae0 0x7f760014b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41152/0 (socket says 172.21.15.7:41152) 2024-01-27T02:33:10.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.740+0000 7f75fe59c700 1 -- 172.21.15.7:0/2064548987 learned_addr learned my addr 172.21.15.7:0/2064548987 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:10.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.741+0000 7f75fdd9b700 1 --2- 172.21.15.7:0/2064548987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7600157ca0 0x7f7600150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:10.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.741+0000 7f75fed9d700 1 --2- 172.21.15.7:0/2064548987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7600158070 0x7f76000b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:10.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.741+0000 7f75fe59c700 1 -- 172.21.15.7:0/2064548987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7600157ca0 msgr2=0x7f7600150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:10.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.741+0000 7f75fe59c700 1 --2- 172.21.15.7:0/2064548987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7600157ca0 0x7f7600150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:10.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.741+0000 7f75fe59c700 1 -- 172.21.15.7:0/2064548987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7600158070 msgr2=0x7f76000b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.741+0000 7f75fe59c700 1 --2- 172.21.15.7:0/2064548987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7600158070 0x7f76000b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.741+0000 7f75fe59c700 1 -- 172.21.15.7:0/2064548987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f00094f0 con 0x7f76000c3ae0 2024-01-27T02:33:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.741+0000 7f75fe59c700 1 --2- 172.21.15.7:0/2064548987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76000c3ae0 0x7f760014b890 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f75f0006fd0 tx=0x7f75f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.741+0000 7f75ee7fc700 1 -- 172.21.15.7:0/2064548987 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f000bbe0 con 0x7f76000c3ae0 2024-01-27T02:33:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.741+0000 7f75ee7fc700 1 -- 172.21.15.7:0/2064548987 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f0011420 con 0x7f76000c3ae0 2024-01-27T02:33:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.741+0000 7f760496e700 1 -- 172.21.15.7:0/2064548987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76000b1dc0 con 0x7f76000c3ae0 2024-01-27T02:33:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.741+0000 7f75ee7fc700 1 -- 172.21.15.7:0/2064548987 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f00117e0 con 0x7f76000c3ae0 2024-01-27T02:33:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.741+0000 7f760496e700 1 -- 172.21.15.7:0/2064548987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76000b2380 con 0x7f76000c3ae0 2024-01-27T02:33:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.742+0000 7f760496e700 1 -- 172.21.15.7:0/2064548987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f760004c710 con 0x7f76000c3ae0 2024-01-27T02:33:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.743+0000 7f75ee7fc700 1 -- 172.21.15.7:0/2064548987 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f75f0011bd0 con 0x7f76000c3ae0 2024-01-27T02:33:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.743+0000 7f75ee7fc700 1 --2- 172.21.15.7:0/2064548987 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f75e0066f30 0x7f75e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.743+0000 7f75ee7fc700 1 -- 172.21.15.7:0/2064548987 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f75f0085d60 con 0x7f76000c3ae0 2024-01-27T02:33:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.743+0000 7f75fdd9b700 1 --2- 172.21.15.7:0/2064548987 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f75e0066f30 0x7f75e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.744+0000 7f75fdd9b700 1 --2- 172.21.15.7:0/2064548987 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f75e0066f30 0x7f75e00693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7600152720 tx=0x7f75f40091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.748+0000 7f75ee7fc700 1 -- 172.21.15.7:0/2064548987 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75f0052fd0 con 0x7f76000c3ae0 2024-01-27T02:33:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.953+0000 7f760496e700 1 -- 172.21.15.7:0/2064548987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76000c57d0 con 0x7f76000c3ae0 2024-01-27T02:33:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.954+0000 7f75ee7fc700 1 -- 172.21.15.7:0/2064548987 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f75f00565f0 con 0x7f76000c3ae0 2024-01-27T02:33:10.956 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:10.956 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:33:07.888617+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:32:52.897548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-27T02:32:53.822754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:33:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.957+0000 7f760496e700 1 -- 172.21.15.7:0/2064548987 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f75e0066f30 msgr2=0x7f75e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.957+0000 7f760496e700 1 --2- 172.21.15.7:0/2064548987 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f75e0066f30 0x7f75e00693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7600152720 tx=0x7f75f40091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.957+0000 7f760496e700 1 -- 172.21.15.7:0/2064548987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76000c3ae0 msgr2=0x7f760014b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.957+0000 7f760496e700 1 --2- 172.21.15.7:0/2064548987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76000c3ae0 0x7f760014b890 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f75f0006fd0 tx=0x7f75f0017000 comp rx=0 tx=0).stop 2024-01-27T02:33:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.957+0000 7f760496e700 1 -- 172.21.15.7:0/2064548987 shutdown_connections 2024-01-27T02:33:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.957+0000 7f760496e700 1 --2- 172.21.15.7:0/2064548987 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f75e0066f30 0x7f75e00693e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.957+0000 7f760496e700 1 --2- 172.21.15.7:0/2064548987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7600158070 0x7f76000b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.957+0000 7f760496e700 1 --2- 172.21.15.7:0/2064548987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76000c3ae0 0x7f760014b890 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.957+0000 7f760496e700 1 --2- 172.21.15.7:0/2064548987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7600157ca0 0x7f7600150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.957+0000 7f760496e700 1 -- 172.21.15.7:0/2064548987 >> 172.21.15.7:0/2064548987 conn(0x7f7600098020 msgr2=0x7f76000cb310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.957+0000 7f760496e700 1 -- 172.21.15.7:0/2064548987 shutdown_connections 2024-01-27T02:33:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:10.957+0000 7f760496e700 1 -- 172.21.15.7:0/2064548987 wait complete. 2024-01-27T02:33:10.971 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-27T02:32:52.897548+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-01-27T02:32:53.822754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-27T02:33:10.972 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:33:11.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.105+0000 7f8244181700 1 Processor -- start 2024-01-27T02:33:11.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.106+0000 7f8244181700 1 -- start start 2024-01-27T02:33:11.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.106+0000 7f8244181700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f823c0cbaf0 0x7f823c15f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:11.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.106+0000 7f8244181700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f823c0cbec0 0x7f823c069520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:11.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.106+0000 7f8244181700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f823c06b4e0 0x7f823c06b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:11.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.106+0000 7f8244181700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823c069a60 con 0x7f823c0cbaf0 2024-01-27T02:33:11.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.106+0000 7f8244181700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823c15fc60 con 0x7f823c06b4e0 2024-01-27T02:33:11.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.106+0000 7f8244181700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823c15fdf0 con 0x7f823c0cbec0 2024-01-27T02:33:11.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.107+0000 7f8241f1d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f823c0cbaf0 0x7f823c15f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:11.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.107+0000 7f8241f1d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f823c0cbaf0 0x7f823c15f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41154/0 (socket says 172.21.15.7:41154) 2024-01-27T02:33:11.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.107+0000 7f8241f1d700 1 -- 172.21.15.7:0/1649227825 learned_addr learned my addr 172.21.15.7:0/1649227825 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:11.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.107+0000 7f824171c700 1 --2- 172.21.15.7:0/1649227825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f823c0cbec0 0x7f823c069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:11.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.107+0000 7f824271e700 1 --2- 172.21.15.7:0/1649227825 >> v2:172.21.15.188:3300/0 conn(0x7f823c06b4e0 0x7f823c06b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:11.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.107+0000 7f8241f1d700 1 -- 172.21.15.7:0/1649227825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f823c0cbec0 msgr2=0x7f823c069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.107+0000 7f8241f1d700 1 --2- 172.21.15.7:0/1649227825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f823c0cbec0 0x7f823c069520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.107+0000 7f8241f1d700 1 -- 172.21.15.7:0/1649227825 >> v2:172.21.15.188:3300/0 conn(0x7f823c06b4e0 msgr2=0x7f823c06b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.107+0000 7f8241f1d700 1 --2- 172.21.15.7:0/1649227825 >> v2:172.21.15.188:3300/0 conn(0x7f823c06b4e0 0x7f823c06b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.107+0000 7f8241f1d700 1 -- 172.21.15.7:0/1649227825 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823c0ccc50 con 0x7f823c0cbaf0 2024-01-27T02:33:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.107+0000 7f8241f1d700 1 --2- 172.21.15.7:0/1649227825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c0cbaf0 0x7f823c15f650 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f823c0c5d10 tx=0x7f8230009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28ac9bac15c1773 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.107+0000 7f8240f1b700 1 -- 172.21.15.7:0/1649227825 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8230005c10 con 0x7f823c0cbaf0 2024-01-27T02:33:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.107+0000 7f8240f1b700 1 -- 172.21.15.7:0/1649227825 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8230005d70 con 0x7f823c0cbaf0 2024-01-27T02:33:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.108+0000 7f8240f1b700 1 -- 172.21.15.7:0/1649227825 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8230006260 con 0x7f823c0cbaf0 2024-01-27T02:33:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.108+0000 7f8244181700 1 -- 172.21.15.7:0/1649227825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c0cbaf0 msgr2=0x7f823c15f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.108+0000 7f8244181700 1 --2- 172.21.15.7:0/1649227825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c0cbaf0 0x7f823c15f650 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f823c0c5d10 tx=0x7f8230009220 comp rx=0 tx=0).stop 2024-01-27T02:33:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.108+0000 7f8244181700 1 -- 172.21.15.7:0/1649227825 shutdown_connections 2024-01-27T02:33:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.108+0000 7f8244181700 1 --2- 172.21.15.7:0/1649227825 >> v2:172.21.15.188:3300/0 conn(0x7f823c06b4e0 0x7f823c06b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.108+0000 7f8244181700 1 --2- 172.21.15.7:0/1649227825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c0cbaf0 0x7f823c15f650 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.108+0000 7f8244181700 1 --2- 172.21.15.7:0/1649227825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f823c0cbec0 0x7f823c069520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.108+0000 7f8244181700 1 -- 172.21.15.7:0/1649227825 >> 172.21.15.7:0/1649227825 conn(0x7f823c063580 msgr2=0x7f823c063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.108+0000 7f8244181700 1 -- 172.21.15.7:0/1649227825 shutdown_connections 2024-01-27T02:33:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.108+0000 7f8244181700 1 -- 172.21.15.7:0/1649227825 wait complete. 2024-01-27T02:33:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.109+0000 7f8244181700 1 Processor -- start 2024-01-27T02:33:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.110+0000 7f8244181700 1 -- start start 2024-01-27T02:33:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.110+0000 7f8244181700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f823c06b4e0 0x7f823c0c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.110+0000 7f8244181700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c0cbaf0 0x7f823c152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.110+0000 7f8244181700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f823c0cbec0 0x7f823c156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.110+0000 7f8244181700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c0b3040 con 0x7f823c0cbaf0 2024-01-27T02:33:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.110+0000 7f8244181700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c0c90b0 con 0x7f823c06b4e0 2024-01-27T02:33:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.110+0000 7f8244181700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823c1530c0 con 0x7f823c0cbec0 2024-01-27T02:33:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.110+0000 7f824171c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c0cbaf0 0x7f823c152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.110+0000 7f824171c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c0cbaf0 0x7f823c152b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41164/0 (socket says 172.21.15.7:41164) 2024-01-27T02:33:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.110+0000 7f824171c700 1 -- 172.21.15.7:0/986038186 learned_addr learned my addr 172.21.15.7:0/986038186 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.110+0000 7f8241f1d700 1 --2- 172.21.15.7:0/986038186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f823c06b4e0 0x7f823c0c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.110+0000 7f824271e700 1 --2- 172.21.15.7:0/986038186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f823c0cbec0 0x7f823c156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.111+0000 7f824171c700 1 -- 172.21.15.7:0/986038186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f823c0cbec0 msgr2=0x7f823c156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.111+0000 7f824171c700 1 --2- 172.21.15.7:0/986038186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f823c0cbec0 0x7f823c156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.111+0000 7f824171c700 1 -- 172.21.15.7:0/986038186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f823c06b4e0 msgr2=0x7f823c0c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.111+0000 7f824171c700 1 --2- 172.21.15.7:0/986038186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f823c06b4e0 0x7f823c0c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.111+0000 7f824171c700 1 -- 172.21.15.7:0/986038186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8230009480 con 0x7f823c0cbaf0 2024-01-27T02:33:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.111+0000 7f824171c700 1 --2- 172.21.15.7:0/986038186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c0cbaf0 0x7f823c152b80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f823800e900 tx=0x7f82380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.111+0000 7f822dffb700 1 -- 172.21.15.7:0/986038186 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238013680 con 0x7f823c0cbaf0 2024-01-27T02:33:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.111+0000 7f822dffb700 1 -- 172.21.15.7:0/986038186 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82380137e0 con 0x7f823c0cbaf0 2024-01-27T02:33:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.111+0000 7f822dffb700 1 -- 172.21.15.7:0/986038186 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238013ba0 con 0x7f823c0cbaf0 2024-01-27T02:33:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.111+0000 7f8244181700 1 -- 172.21.15.7:0/986038186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823c153550 con 0x7f823c0cbaf0 2024-01-27T02:33:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.112+0000 7f8244181700 1 -- 172.21.15.7:0/986038186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823c0b22d0 con 0x7f823c0cbaf0 2024-01-27T02:33:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.112+0000 7f8244181700 1 -- 172.21.15.7:0/986038186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f823c0b31f0 con 0x7f823c0cbaf0 2024-01-27T02:33:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.113+0000 7f822dffb700 1 -- 172.21.15.7:0/986038186 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f82380123d0 con 0x7f823c0cbaf0 2024-01-27T02:33:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.113+0000 7f822dffb700 1 --2- 172.21.15.7:0/986038186 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8220066f30 0x7f82200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.113+0000 7f822dffb700 1 -- 172.21.15.7:0/986038186 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f82380845d0 con 0x7f823c0cbaf0 2024-01-27T02:33:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.114+0000 7f8241f1d700 1 --2- 172.21.15.7:0/986038186 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8220066f30 0x7f82200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.114+0000 7f8241f1d700 1 --2- 172.21.15.7:0/986038186 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8220066f30 0x7f82200693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8230007fd0 tx=0x7f8230016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.118+0000 7f822dffb700 1 -- 172.21.15.7:0/986038186 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f823c0b31f0 con 0x7f823c0cbaf0 2024-01-27T02:33:11.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.318+0000 7f8244181700 1 -- 172.21.15.7:0/986038186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f823c0b31f0 con 0x7f823c0cbaf0 2024-01-27T02:33:11.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.319+0000 7f822dffb700 1 -- 172.21.15.7:0/986038186 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f823c0b31f0 con 0x7f823c0cbaf0 2024-01-27T02:33:11.321 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:33:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.321+0000 7f8244181700 1 -- 172.21.15.7:0/986038186 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8220066f30 msgr2=0x7f82200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.321+0000 7f8244181700 1 --2- 172.21.15.7:0/986038186 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8220066f30 0x7f82200693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8230007fd0 tx=0x7f8230016000 comp rx=0 tx=0).stop 2024-01-27T02:33:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.321+0000 7f8244181700 1 -- 172.21.15.7:0/986038186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c0cbaf0 msgr2=0x7f823c152b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.321+0000 7f8244181700 1 --2- 172.21.15.7:0/986038186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c0cbaf0 0x7f823c152b80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f823800e900 tx=0x7f82380091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.321+0000 7f8244181700 1 -- 172.21.15.7:0/986038186 shutdown_connections 2024-01-27T02:33:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.321+0000 7f8244181700 1 --2- 172.21.15.7:0/986038186 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8220066f30 0x7f82200693e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.321+0000 7f8244181700 1 --2- 172.21.15.7:0/986038186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f823c0cbec0 0x7f823c156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.321+0000 7f8244181700 1 --2- 172.21.15.7:0/986038186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f823c06b4e0 0x7f823c0c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.321+0000 7f8244181700 1 --2- 172.21.15.7:0/986038186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c0cbaf0 0x7f823c152b80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.321+0000 7f8244181700 1 -- 172.21.15.7:0/986038186 >> 172.21.15.7:0/986038186 conn(0x7f823c063580 msgr2=0x7f823c15eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:11.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.321+0000 7f8244181700 1 -- 172.21.15.7:0/986038186 shutdown_connections 2024-01-27T02:33:11.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.321+0000 7f8244181700 1 -- 172.21.15.7:0/986038186 wait complete. 2024-01-27T02:33:11.336 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-27T02:33:11.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.471+0000 7ff517250700 1 Processor -- start 2024-01-27T02:33:11.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.471+0000 7ff517250700 1 -- start start 2024-01-27T02:33:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.472+0000 7ff517250700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff5100cba60 0x7ff5100ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.472+0000 7ff517250700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff5100cbe30 0x7ff51015b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.472+0000 7ff517250700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5100679b0 con 0x7ff51006b4e0 2024-01-27T02:33:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.472+0000 7ff517250700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff510067c90 con 0x7ff5100cba60 2024-01-27T02:33:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.472+0000 7ff517250700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff51015b7b0 con 0x7ff5100cbe30 2024-01-27T02:33:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.472+0000 7ff514fec700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff5100cba60 0x7ff5100ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.472+0000 7ff5157ed700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff51006b4e0 0x7ff51006b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53614/0 (socket says 172.21.15.7:53614) 2024-01-27T02:33:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.472+0000 7ff5157ed700 1 -- 172.21.15.7:0/624647308 learned_addr learned my addr 172.21.15.7:0/624647308 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.472+0000 7ff50ffff700 1 --2- 172.21.15.7:0/624647308 >> v2:172.21.15.188:3300/0 conn(0x7ff5100cbe30 0x7ff51015b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.473+0000 7ff50f7fe700 1 -- 172.21.15.7:0/624647308 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2657964853 0 0) 0x7ff5100679b0 con 0x7ff51006b4e0 2024-01-27T02:33:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.473+0000 7ff50f7fe700 1 -- 172.21.15.7:0/624647308 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff500003570 con 0x7ff51006b4e0 2024-01-27T02:33:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.473+0000 7ff514fec700 1 -- 172.21.15.7:0/624647308 >> v2:172.21.15.188:3300/0 conn(0x7ff5100cbe30 msgr2=0x7ff51015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.473+0000 7ff514fec700 1 --2- 172.21.15.7:0/624647308 >> v2:172.21.15.188:3300/0 conn(0x7ff5100cbe30 0x7ff51015b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.473+0000 7ff514fec700 1 -- 172.21.15.7:0/624647308 >> v1:172.21.15.7:6789/0 conn(0x7ff51006b4e0 legacy=0x7ff51006b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.473+0000 7ff514fec700 1 -- 172.21.15.7:0/624647308 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5100cca50 con 0x7ff5100cba60 2024-01-27T02:33:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.473+0000 7ff514fec700 1 --2- 172.21.15.7:0/624647308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5100cba60 0x7ff5100ce410 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff51006d590 tx=0x7ff504009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dad561c672d327c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.473+0000 7ff50f7fe700 1 -- 172.21.15.7:0/624647308 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504005c10 con 0x7ff5100cba60 2024-01-27T02:33:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.473+0000 7ff50f7fe700 1 -- 172.21.15.7:0/624647308 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff504005d70 con 0x7ff5100cba60 2024-01-27T02:33:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.473+0000 7ff50f7fe700 1 -- 172.21.15.7:0/624647308 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504006260 con 0x7ff5100cba60 2024-01-27T02:33:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.474+0000 7ff517250700 1 -- 172.21.15.7:0/624647308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5100cba60 msgr2=0x7ff5100ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.474+0000 7ff517250700 1 --2- 172.21.15.7:0/624647308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5100cba60 0x7ff5100ce410 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff51006d590 tx=0x7ff504009120 comp rx=0 tx=0).stop 2024-01-27T02:33:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.474+0000 7ff517250700 1 -- 172.21.15.7:0/624647308 shutdown_connections 2024-01-27T02:33:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.474+0000 7ff517250700 1 --2- 172.21.15.7:0/624647308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5100cba60 0x7ff5100ce410 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.474+0000 7ff517250700 1 --2- 172.21.15.7:0/624647308 >> v2:172.21.15.188:3300/0 conn(0x7ff5100cbe30 0x7ff51015b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.474+0000 7ff517250700 1 -- 172.21.15.7:0/624647308 >> 172.21.15.7:0/624647308 conn(0x7ff510063580 msgr2=0x7ff510063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.474+0000 7ff517250700 1 -- 172.21.15.7:0/624647308 shutdown_connections 2024-01-27T02:33:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.474+0000 7ff517250700 1 -- 172.21.15.7:0/624647308 wait complete. 2024-01-27T02:33:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.475+0000 7ff517250700 1 Processor -- start 2024-01-27T02:33:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.475+0000 7ff517250700 1 -- start start 2024-01-27T02:33:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.476+0000 7ff517250700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff51006b4e0 0x7ff5100b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.476+0000 7ff517250700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5100cba60 0x7ff510156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.476+0000 7ff517250700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5100cbe30 0x7ff5100c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.476+0000 7ff517250700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100aa8b0 con 0x7ff5100cbe30 2024-01-27T02:33:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.476+0000 7ff517250700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5101553f0 con 0x7ff51006b4e0 2024-01-27T02:33:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.476+0000 7ff517250700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5101556d0 con 0x7ff5100cba60 2024-01-27T02:33:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.477+0000 7ff5157ed700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5100cbe30 0x7ff5100c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.477+0000 7ff5157ed700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5100cbe30 0x7ff5100c3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60530/0 (socket says 172.21.15.7:60530) 2024-01-27T02:33:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.477+0000 7ff5157ed700 1 -- 172.21.15.7:0/1425918050 learned_addr learned my addr 172.21.15.7:0/1425918050 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.477+0000 7ff514fec700 1 --2- 172.21.15.7:0/1425918050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff51006b4e0 0x7ff5100b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.477+0000 7ff50ffff700 1 --2- 172.21.15.7:0/1425918050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5100cba60 0x7ff510156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.477+0000 7ff5157ed700 1 -- 172.21.15.7:0/1425918050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5100cba60 msgr2=0x7ff510156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.477+0000 7ff5157ed700 1 --2- 172.21.15.7:0/1425918050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5100cba60 0x7ff510156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.477+0000 7ff5157ed700 1 -- 172.21.15.7:0/1425918050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff51006b4e0 msgr2=0x7ff5100b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.477+0000 7ff5157ed700 1 --2- 172.21.15.7:0/1425918050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff51006b4e0 0x7ff5100b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.477+0000 7ff5157ed700 1 -- 172.21.15.7:0/1425918050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5040093c0 con 0x7ff5100cbe30 2024-01-27T02:33:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.477+0000 7ff514fec700 1 --2- 172.21.15.7:0/1425918050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff51006b4e0 0x7ff5100b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.477+0000 7ff5157ed700 1 --2- 172.21.15.7:0/1425918050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5100cbe30 0x7ff5100c3a80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff50800cc40 tx=0x7ff508001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.477+0000 7ff50cff9700 1 -- 172.21.15.7:0/1425918050 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50800a9c0 con 0x7ff5100cbe30 2024-01-27T02:33:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.477+0000 7ff50cff9700 1 -- 172.21.15.7:0/1425918050 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50800ab20 con 0x7ff5100cbe30 2024-01-27T02:33:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.477+0000 7ff50cff9700 1 -- 172.21.15.7:0/1425918050 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50800aea0 con 0x7ff5100cbe30 2024-01-27T02:33:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.477+0000 7ff517250700 1 -- 172.21.15.7:0/1425918050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5101559b0 con 0x7ff5100cbe30 2024-01-27T02:33:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.477+0000 7ff517250700 1 -- 172.21.15.7:0/1425918050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5100b4950 con 0x7ff5100cbe30 2024-01-27T02:33:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.479+0000 7ff517250700 1 -- 172.21.15.7:0/1425918050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff510072060 con 0x7ff5100cbe30 2024-01-27T02:33:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.479+0000 7ff50cff9700 1 -- 172.21.15.7:0/1425918050 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff508007300 con 0x7ff5100cbe30 2024-01-27T02:33:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.479+0000 7ff50cff9700 1 --2- 172.21.15.7:0/1425918050 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff4f4066f30 0x7ff4f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.480+0000 7ff50cff9700 1 -- 172.21.15.7:0/1425918050 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff508084a50 con 0x7ff5100cbe30 2024-01-27T02:33:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.480+0000 7ff514fec700 1 --2- 172.21.15.7:0/1425918050 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff4f4066f30 0x7ff4f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.480+0000 7ff514fec700 1 --2- 172.21.15.7:0/1425918050 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff4f4066f30 0x7ff4f40693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff504009160 tx=0x7ff504016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:11.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.484+0000 7ff50cff9700 1 -- 172.21.15.7:0/1425918050 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff508051ca0 con 0x7ff5100cbe30 2024-01-27T02:33:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.686+0000 7ff517250700 1 -- 172.21.15.7:0/1425918050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff51015afd0 con 0x7ff5100cbe30 2024-01-27T02:33:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.686+0000 7ff50cff9700 1 -- 172.21.15.7:0/1425918050 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff5080552c0 con 0x7ff5100cbe30 2024-01-27T02:33:11.687 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-27T02:33:11.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.688+0000 7ff517250700 1 -- 172.21.15.7:0/1425918050 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff4f4066f30 msgr2=0x7ff4f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:11.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.688+0000 7ff517250700 1 --2- 172.21.15.7:0/1425918050 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff4f4066f30 0x7ff4f40693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff504009160 tx=0x7ff504016000 comp rx=0 tx=0).stop 2024-01-27T02:33:11.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.688+0000 7ff517250700 1 -- 172.21.15.7:0/1425918050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5100cbe30 msgr2=0x7ff5100c3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:11.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.688+0000 7ff517250700 1 --2- 172.21.15.7:0/1425918050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5100cbe30 0x7ff5100c3a80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff50800cc40 tx=0x7ff508001aa0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.688+0000 7ff517250700 1 -- 172.21.15.7:0/1425918050 shutdown_connections 2024-01-27T02:33:11.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.688+0000 7ff517250700 1 --2- 172.21.15.7:0/1425918050 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff4f4066f30 0x7ff4f40693e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.688+0000 7ff517250700 1 --2- 172.21.15.7:0/1425918050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5100cbe30 0x7ff5100c3a80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.688+0000 7ff517250700 1 --2- 172.21.15.7:0/1425918050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff51006b4e0 0x7ff5100b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.688+0000 7ff517250700 1 --2- 172.21.15.7:0/1425918050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5100cba60 0x7ff510156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.688+0000 7ff517250700 1 -- 172.21.15.7:0/1425918050 >> 172.21.15.7:0/1425918050 conn(0x7ff510063580 msgr2=0x7ff510069460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:11.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.688+0000 7ff517250700 1 -- 172.21.15.7:0/1425918050 shutdown_connections 2024-01-27T02:33:11.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.688+0000 7ff517250700 1 -- 172.21.15.7:0/1425918050 wait complete. 2024-01-27T02:33:11.703 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-27T02:33:11.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.836+0000 7fdb9edd8700 1 Processor -- start 2024-01-27T02:33:11.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.836+0000 7fdb9edd8700 1 -- start start 2024-01-27T02:33:11.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.837+0000 7fdb9edd8700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdb980cb950 0x7fdb980b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:11.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.837+0000 7fdb9edd8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb980cbd20 0x7fdb980ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:11.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.837+0000 7fdb9edd8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb980c9660 con 0x7fdb981412d0 2024-01-27T02:33:11.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.837+0000 7fdb9edd8700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb9815c980 con 0x7fdb980cb950 2024-01-27T02:33:11.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.837+0000 7fdb9edd8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9815be00 con 0x7fdb980cbd20 2024-01-27T02:33:11.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.837+0000 7fdb9d375700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdb981412d0 0x7fdb98141730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53622/0 (socket says 172.21.15.7:53622) 2024-01-27T02:33:11.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.837+0000 7fdb9d375700 1 -- 172.21.15.7:0/4049949925 learned_addr learned my addr 172.21.15.7:0/4049949925 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:11.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.837+0000 7fdb97fff700 1 --2- 172.21.15.7:0/4049949925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb980cbd20 0x7fdb980ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:11.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.837+0000 7fdb9cb74700 1 --2- 172.21.15.7:0/4049949925 >> v2:172.21.15.188:3300/0 conn(0x7fdb980cb950 0x7fdb980b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:11.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.837+0000 7fdb977fe700 1 -- 172.21.15.7:0/4049949925 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165686200 0 0) 0x7fdb980c9660 con 0x7fdb981412d0 2024-01-27T02:33:11.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.838+0000 7fdb977fe700 1 -- 172.21.15.7:0/4049949925 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb88003570 con 0x7fdb981412d0 2024-01-27T02:33:11.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.838+0000 7fdb97fff700 1 -- 172.21.15.7:0/4049949925 >> v2:172.21.15.188:3300/0 conn(0x7fdb980cb950 msgr2=0x7fdb980b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:11.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.838+0000 7fdb977fe700 1 -- 172.21.15.7:0/4049949925 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3794501672 0 0) 0x7fdb88003570 con 0x7fdb981412d0 2024-01-27T02:33:11.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.838+0000 7fdb97fff700 1 --2- 172.21.15.7:0/4049949925 >> v2:172.21.15.188:3300/0 conn(0x7fdb980cb950 0x7fdb980b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.838+0000 7fdb97fff700 1 -- 172.21.15.7:0/4049949925 >> v1:172.21.15.7:6789/0 conn(0x7fdb981412d0 legacy=0x7fdb98141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:11.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.838+0000 7fdb97fff700 1 -- 172.21.15.7:0/4049949925 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9815c0e0 con 0x7fdb980cbd20 2024-01-27T02:33:11.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.838+0000 7fdb9cb74700 1 --2- 172.21.15.7:0/4049949925 >> v2:172.21.15.188:3300/0 conn(0x7fdb980cb950 0x7fdb980b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:33:11.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.838+0000 7fdb97fff700 1 --2- 172.21.15.7:0/4049949925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb980cbd20 0x7fdb980ce4a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdb840098e0 tx=0x7fdb840091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8f9dbfa53ce976f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:11.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.838+0000 7fdb977fe700 1 -- 172.21.15.7:0/4049949925 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb900017f0 con 0x7fdb980cbd20 2024-01-27T02:33:11.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.838+0000 7fdb977fe700 1 -- 172.21.15.7:0/4049949925 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb84011420 con 0x7fdb980cbd20 2024-01-27T02:33:11.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.838+0000 7fdb977fe700 1 -- 172.21.15.7:0/4049949925 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb840117c0 con 0x7fdb980cbd20 2024-01-27T02:33:11.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.839+0000 7fdb9edd8700 1 -- 172.21.15.7:0/4049949925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb980cbd20 msgr2=0x7fdb980ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:11.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.839+0000 7fdb9edd8700 1 --2- 172.21.15.7:0/4049949925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb980cbd20 0x7fdb980ce4a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdb840098e0 tx=0x7fdb840091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.839+0000 7fdb9edd8700 1 -- 172.21.15.7:0/4049949925 shutdown_connections 2024-01-27T02:33:11.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.839+0000 7fdb9edd8700 1 --2- 172.21.15.7:0/4049949925 >> v2:172.21.15.188:3300/0 conn(0x7fdb980cb950 0x7fdb980b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.839+0000 7fdb9edd8700 1 --2- 172.21.15.7:0/4049949925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb980cbd20 0x7fdb980ce4a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.839+0000 7fdb9edd8700 1 -- 172.21.15.7:0/4049949925 >> 172.21.15.7:0/4049949925 conn(0x7fdb98063580 msgr2=0x7fdb98063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:11.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.839+0000 7fdb9edd8700 1 -- 172.21.15.7:0/4049949925 shutdown_connections 2024-01-27T02:33:11.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.839+0000 7fdb9edd8700 1 -- 172.21.15.7:0/4049949925 wait complete. 2024-01-27T02:33:11.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.840+0000 7fdb9edd8700 1 Processor -- start 2024-01-27T02:33:11.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.840+0000 7fdb9edd8700 1 -- start start 2024-01-27T02:33:11.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.841+0000 7fdb9edd8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb980cb950 0x7fdb9813ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:11.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.841+0000 7fdb9edd8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb980cbd20 0x7fdb9815b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:11.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.841+0000 7fdb9edd8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb981412d0 0x7fdb980a6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:11.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.841+0000 7fdb9edd8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9815b110 con 0x7fdb980cbd20 2024-01-27T02:33:11.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.841+0000 7fdb9edd8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9815aad0 con 0x7fdb981412d0 2024-01-27T02:33:11.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.841+0000 7fdb9edd8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9813f130 con 0x7fdb980cb950 2024-01-27T02:33:11.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.841+0000 7fdb97fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb980cbd20 0x7fdb9815b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:11.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.841+0000 7fdb97fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb980cbd20 0x7fdb9815b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60544/0 (socket says 172.21.15.7:60544) 2024-01-27T02:33:11.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.841+0000 7fdb97fff700 1 -- 172.21.15.7:0/147753287 learned_addr learned my addr 172.21.15.7:0/147753287 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:11.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.841+0000 7fdb9cb74700 1 --2- 172.21.15.7:0/147753287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb980cb950 0x7fdb9813ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:11.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.841+0000 7fdb9d375700 1 --2- 172.21.15.7:0/147753287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb981412d0 0x7fdb980a6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:11.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.842+0000 7fdb97fff700 1 -- 172.21.15.7:0/147753287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb980cb950 msgr2=0x7fdb9813ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:11.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.842+0000 7fdb97fff700 1 --2- 172.21.15.7:0/147753287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb980cb950 0x7fdb9813ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.842+0000 7fdb97fff700 1 -- 172.21.15.7:0/147753287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb981412d0 msgr2=0x7fdb980a6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:11.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.842+0000 7fdb97fff700 1 --2- 172.21.15.7:0/147753287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb981412d0 0x7fdb980a6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:11.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.842+0000 7fdb97fff700 1 -- 172.21.15.7:0/147753287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8c009420 con 0x7fdb980cbd20 2024-01-27T02:33:11.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.842+0000 7fdb97fff700 1 --2- 172.21.15.7:0/147753287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb980cbd20 0x7fdb9815b5a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdb840030c0 tx=0x7fdb84016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:11.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.842+0000 7fdb94ff9700 1 -- 172.21.15.7:0/147753287 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8401cc90 con 0x7fdb980cbd20 2024-01-27T02:33:11.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.842+0000 7fdb94ff9700 1 -- 172.21.15.7:0/147753287 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8401cdf0 con 0x7fdb980cbd20 2024-01-27T02:33:11.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.842+0000 7fdb9edd8700 1 -- 172.21.15.7:0/147753287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb84009420 con 0x7fdb980cbd20 2024-01-27T02:33:11.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.842+0000 7fdb94ff9700 1 -- 172.21.15.7:0/147753287 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84025680 con 0x7fdb980cbd20 2024-01-27T02:33:11.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.842+0000 7fdb9edd8700 1 -- 172.21.15.7:0/147753287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb98159db0 con 0x7fdb980cbd20 2024-01-27T02:33:11.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.843+0000 7fdb9edd8700 1 -- 172.21.15.7:0/147753287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb9815b280 con 0x7fdb980cbd20 2024-01-27T02:33:11.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.843+0000 7fdb94ff9700 1 -- 172.21.15.7:0/147753287 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fdb840193f0 con 0x7fdb980cbd20 2024-01-27T02:33:11.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.844+0000 7fdb94ff9700 1 --2- 172.21.15.7:0/147753287 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdb7c066f30 0x7fdb7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:11.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.844+0000 7fdb94ff9700 1 -- 172.21.15.7:0/147753287 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdb84085830 con 0x7fdb980cbd20 2024-01-27T02:33:11.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.844+0000 7fdb9cb74700 1 --2- 172.21.15.7:0/147753287 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdb7c066f30 0x7fdb7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:11.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.845+0000 7fdb9cb74700 1 --2- 172.21.15.7:0/147753287 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdb7c066f30 0x7fdb7c0693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0093f0 tx=0x7fdb8c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:11.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:11.848+0000 7fdb94ff9700 1 -- 172.21.15.7:0/147753287 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb84052b20 con 0x7fdb980cbd20 2024-01-27T02:33:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.051+0000 7fdb9edd8700 1 -- 172.21.15.7:0/147753287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdb98072060 con 0x7fdb980cbd20 2024-01-27T02:33:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.052+0000 7fdb94ff9700 1 -- 172.21.15.7:0/147753287 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdb84085020 con 0x7fdb980cbd20 2024-01-27T02:33:12.053 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-27T02:33:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.054+0000 7fdb9edd8700 1 -- 172.21.15.7:0/147753287 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdb7c066f30 msgr2=0x7fdb7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.054+0000 7fdb9edd8700 1 --2- 172.21.15.7:0/147753287 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdb7c066f30 0x7fdb7c0693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0093f0 tx=0x7fdb8c008040 comp rx=0 tx=0).stop 2024-01-27T02:33:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.054+0000 7fdb9edd8700 1 -- 172.21.15.7:0/147753287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb980cbd20 msgr2=0x7fdb9815b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.054+0000 7fdb9edd8700 1 --2- 172.21.15.7:0/147753287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb980cbd20 0x7fdb9815b5a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdb840030c0 tx=0x7fdb84016000 comp rx=0 tx=0).stop 2024-01-27T02:33:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.054+0000 7fdb9edd8700 1 -- 172.21.15.7:0/147753287 shutdown_connections 2024-01-27T02:33:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.054+0000 7fdb9edd8700 1 --2- 172.21.15.7:0/147753287 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdb7c066f30 0x7fdb7c0693e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.054+0000 7fdb9edd8700 1 --2- 172.21.15.7:0/147753287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb981412d0 0x7fdb980a6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.054+0000 7fdb9edd8700 1 --2- 172.21.15.7:0/147753287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb980cb950 0x7fdb9813ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.054+0000 7fdb9edd8700 1 --2- 172.21.15.7:0/147753287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb980cbd20 0x7fdb9815b5a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.054+0000 7fdb9edd8700 1 -- 172.21.15.7:0/147753287 >> 172.21.15.7:0/147753287 conn(0x7fdb98063580 msgr2=0x7fdb98063950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.054+0000 7fdb9edd8700 1 -- 172.21.15.7:0/147753287 shutdown_connections 2024-01-27T02:33:12.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.054+0000 7fdb9edd8700 1 -- 172.21.15.7:0/147753287 wait complete. 2024-01-27T02:33:12.069 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:33:12.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.199+0000 7f3188c3c700 1 Processor -- start 2024-01-27T02:33:12.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.199+0000 7f3188c3c700 1 -- start start 2024-01-27T02:33:12.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.200+0000 7f3188c3c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3184157ca0 0x7f31840c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:12.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.200+0000 7f3188c3c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31840ca020 con 0x7f31840cdbf0 2024-01-27T02:33:12.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.200+0000 7f3188c3c700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31840c4f40 con 0x7f31840c3ae0 2024-01-27T02:33:12.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.200+0000 7f3188c3c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31840c5210 con 0x7f3184157ca0 2024-01-27T02:33:12.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.200+0000 7f3182d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f31840cdbf0 0x7f31840ca4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53636/0 (socket says 172.21.15.7:53636) 2024-01-27T02:33:12.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.200+0000 7f3182d9d700 1 -- 172.21.15.7:0/3492418764 learned_addr learned my addr 172.21.15.7:0/3492418764 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:12.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.200+0000 7f318259c700 1 --2- 172.21.15.7:0/3492418764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3184157ca0 0x7f31840c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:12.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.200+0000 7f318159a700 1 -- 172.21.15.7:0/3492418764 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 295050761 0 0) 0x7f31840ca020 con 0x7f31840cdbf0 2024-01-27T02:33:12.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f318159a700 1 -- 172.21.15.7:0/3492418764 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3170003570 con 0x7f31840cdbf0 2024-01-27T02:33:12.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f318159a700 1 -- 172.21.15.7:0/3492418764 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2500134398 0 0) 0x7f31840c4f40 con 0x7f31840c3ae0 2024-01-27T02:33:12.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f318159a700 1 -- 172.21.15.7:0/3492418764 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31840ca020 con 0x7f31840c3ae0 2024-01-27T02:33:12.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f318159a700 1 -- 172.21.15.7:0/3492418764 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2495444485 0 0) 0x7f3170003570 con 0x7f31840cdbf0 2024-01-27T02:33:12.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f318159a700 1 -- 172.21.15.7:0/3492418764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3184157ca0 msgr2=0x7f31840c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:12.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f318159a700 1 --2- 172.21.15.7:0/3492418764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3184157ca0 0x7f31840c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f318159a700 1 -- 172.21.15.7:0/3492418764 >> v1:172.21.15.188:6789/0 conn(0x7f31840c3ae0 legacy=0x7f31840cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:12.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f318159a700 1 -- 172.21.15.7:0/3492418764 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31840c54f0 con 0x7f31840cdbf0 2024-01-27T02:33:12.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f318159a700 1 -- 172.21.15.7:0/3492418764 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f317c0017f0 con 0x7f31840cdbf0 2024-01-27T02:33:12.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f318159a700 1 -- 172.21.15.7:0/3492418764 >> v1:172.21.15.7:6789/0 conn(0x7f31840cdbf0 legacy=0x7f31840ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:12.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f318159a700 1 --2- 172.21.15.7:0/3492418764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3170003c20 0x7f3170004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:12.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f318159a700 1 --2- 172.21.15.7:0/3492418764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3184157ca0 0x7f3170004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:12.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f318159a700 1 --2- 172.21.15.7:0/3492418764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3170004d50 0x7f31700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:12.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f318159a700 1 -- 172.21.15.7:0/3492418764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31840c5210 con 0x7f3170004d50 2024-01-27T02:33:12.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f318159a700 1 -- 172.21.15.7:0/3492418764 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317000bbf0 con 0x7f3170003c20 2024-01-27T02:33:12.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f318159a700 1 -- 172.21.15.7:0/3492418764 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317000bd60 con 0x7f3184157ca0 2024-01-27T02:33:12.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f3182d9d700 1 --2- 172.21.15.7:0/3492418764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3170004d50 0x7f31700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:12.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f318259c700 1 --2- 172.21.15.7:0/3492418764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3170003c20 0x7f3170004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:12.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f3181d9b700 1 --2- 172.21.15.7:0/3492418764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3184157ca0 0x7f3170004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:12.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f3182d9d700 1 -- 172.21.15.7:0/3492418764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3184157ca0 msgr2=0x7f3170004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:12.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f3182d9d700 1 --2- 172.21.15.7:0/3492418764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3184157ca0 0x7f3170004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f3182d9d700 1 -- 172.21.15.7:0/3492418764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3170003c20 msgr2=0x7f3170004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:12.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f3182d9d700 1 --2- 172.21.15.7:0/3492418764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3170003c20 0x7f3170004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.201+0000 7f3182d9d700 1 -- 172.21.15.7:0/3492418764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3170003570 con 0x7f3170004d50 2024-01-27T02:33:12.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.202+0000 7f3182d9d700 1 --2- 172.21.15.7:0/3492418764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3170004d50 0x7f31700051c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f318406f700 tx=0x7f317c0020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72d327993e78cb4 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:12.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.202+0000 7f318159a700 1 -- 172.21.15.7:0/3492418764 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c0070d0 con 0x7f3170004d50 2024-01-27T02:33:12.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.202+0000 7f318159a700 1 -- 172.21.15.7:0/3492418764 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317c007250 con 0x7f3170004d50 2024-01-27T02:33:12.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.202+0000 7f318159a700 1 -- 172.21.15.7:0/3492418764 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c007610 con 0x7f3170004d50 2024-01-27T02:33:12.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.202+0000 7f3188c3c700 1 -- 172.21.15.7:0/3492418764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3170004d50 msgr2=0x7f31700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:12.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.202+0000 7f3188c3c700 1 --2- 172.21.15.7:0/3492418764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3170004d50 0x7f31700051c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f318406f700 tx=0x7f317c0020c0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.202+0000 7f3182d9d700 1 -- 172.21.15.7:0/3492418764 reap_dead start 2024-01-27T02:33:12.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.202+0000 7f3188c3c700 1 -- 172.21.15.7:0/3492418764 shutdown_connections 2024-01-27T02:33:12.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.202+0000 7f3188c3c700 1 -- 172.21.15.7:0/3492418764 >> 172.21.15.7:0/3492418764 conn(0x7f3184098020 msgr2=0x7f31840711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:12.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.202+0000 7f3188c3c700 1 -- 172.21.15.7:0/3492418764 shutdown_connections 2024-01-27T02:33:12.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.202+0000 7f3188c3c700 1 -- 172.21.15.7:0/3492418764 wait complete. 2024-01-27T02:33:12.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.204+0000 7f3188c3c700 1 Processor -- start 2024-01-27T02:33:12.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.204+0000 7f3188c3c700 1 -- start start 2024-01-27T02:33:12.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.204+0000 7f3188c3c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3170004d50 0x7f318414e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:12.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.204+0000 7f3188c3c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31840b2f80 0x7f31840c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:12.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.204+0000 7f3188c3c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31840b33f0 0x7f31840c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:12.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.204+0000 7f3188c3c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31840b1ee0 con 0x7f3170004d50 2024-01-27T02:33:12.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.204+0000 7f3188c3c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31840b21c0 con 0x7f31840b2f80 2024-01-27T02:33:12.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.204+0000 7f3188c3c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31840b24a0 con 0x7f31840b33f0 2024-01-27T02:33:12.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.205+0000 7f318259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3170004d50 0x7f318414e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:12.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.205+0000 7f318259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3170004d50 0x7f318414e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60572/0 (socket says 172.21.15.7:60572) 2024-01-27T02:33:12.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.205+0000 7f318259c700 1 -- 172.21.15.7:0/788893450 learned_addr learned my addr 172.21.15.7:0/788893450 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:12.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.205+0000 7f3181d9b700 1 --2- 172.21.15.7:0/788893450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31840b2f80 0x7f31840c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:12.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.205+0000 7f3182d9d700 1 --2- 172.21.15.7:0/788893450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31840b33f0 0x7f31840c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:12.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.205+0000 7f318259c700 1 -- 172.21.15.7:0/788893450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31840b33f0 msgr2=0x7f31840c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:12.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.205+0000 7f318259c700 1 --2- 172.21.15.7:0/788893450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31840b33f0 0x7f31840c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.205+0000 7f318259c700 1 -- 172.21.15.7:0/788893450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31840b2f80 msgr2=0x7f31840c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:12.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.205+0000 7f318259c700 1 --2- 172.21.15.7:0/788893450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31840b2f80 0x7f31840c5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.205+0000 7f318259c700 1 -- 172.21.15.7:0/788893450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31840b2780 con 0x7f3170004d50 2024-01-27T02:33:12.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.205+0000 7f3182d9d700 1 --2- 172.21.15.7:0/788893450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31840b33f0 0x7f31840c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:12.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.205+0000 7f318259c700 1 --2- 172.21.15.7:0/788893450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3170004d50 0x7f318414e430 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f316c006ac0 tx=0x7f316c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:12.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.205+0000 7f317a7fc700 1 -- 172.21.15.7:0/788893450 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c00f9d0 con 0x7f3170004d50 2024-01-27T02:33:12.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.205+0000 7f317a7fc700 1 -- 172.21.15.7:0/788893450 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316c00fb30 con 0x7f3170004d50 2024-01-27T02:33:12.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.205+0000 7f3188c3c700 1 -- 172.21.15.7:0/788893450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31840b2a60 con 0x7f3170004d50 2024-01-27T02:33:12.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.205+0000 7f317a7fc700 1 -- 172.21.15.7:0/788893450 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c00feb0 con 0x7f3170004d50 2024-01-27T02:33:12.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.205+0000 7f3188c3c700 1 -- 172.21.15.7:0/788893450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3184168330 con 0x7f3170004d50 2024-01-27T02:33:12.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.206+0000 7f3188c3c700 1 -- 172.21.15.7:0/788893450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f318404c710 con 0x7f3170004d50 2024-01-27T02:33:12.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.207+0000 7f317a7fc700 1 -- 172.21.15.7:0/788893450 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f316c014650 con 0x7f3170004d50 2024-01-27T02:33:12.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.207+0000 7f317a7fc700 1 --2- 172.21.15.7:0/788893450 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3164066f30 0x7f31640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:12.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.207+0000 7f317a7fc700 1 -- 172.21.15.7:0/788893450 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f316c018030 con 0x7f3170004d50 2024-01-27T02:33:12.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.207+0000 7f3181d9b700 1 --2- 172.21.15.7:0/788893450 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3164066f30 0x7f31640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:12.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.208+0000 7f3181d9b700 1 --2- 172.21.15.7:0/788893450 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3164066f30 0x7f31640693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3174003cb0 tx=0x7f3174007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:12.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.211+0000 7f317a7fc700 1 -- 172.21.15.7:0/788893450 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f316c052280 con 0x7f3170004d50 2024-01-27T02:33:12.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.463+0000 7f3188c3c700 1 -- 172.21.15.7:0/788893450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31840c57d0 con 0x7f3170004d50 2024-01-27T02:33:12.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.464+0000 7f317a7fc700 1 -- 172.21.15.7:0/788893450 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f316c022090 con 0x7f3170004d50 2024-01-27T02:33:12.465 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:12.465 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4864,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/892460188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":892460188},{"type":"v1","addr":"172.21.15.188:6837","nonce":892460188}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4867,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1657319349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1657319349},{"type":"v1","addr":"172.21.15.188:6836","nonce":1657319349}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4879,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2190571534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2190571534},{"type":"v1","addr":"172.21.15.7:6834","nonce":2190571534}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:32:54.849803+0000","modified":"2024-01-27T02:32:58.809118+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4882},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4882":{"gid":4882,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6835/1734594727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1734594727},{"type":"v1","addr":"172.21.15.7:6835","nonce":1734594727}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-01-27T02:33:12.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.466+0000 7f3188c3c700 1 -- 172.21.15.7:0/788893450 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3164066f30 msgr2=0x7f31640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:12.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.466+0000 7f3188c3c700 1 --2- 172.21.15.7:0/788893450 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3164066f30 0x7f31640693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3174003cb0 tx=0x7f3174007d30 comp rx=0 tx=0).stop 2024-01-27T02:33:12.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.466+0000 7f3188c3c700 1 -- 172.21.15.7:0/788893450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3170004d50 msgr2=0x7f318414e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.466+0000 7f3188c3c700 1 --2- 172.21.15.7:0/788893450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3170004d50 0x7f318414e430 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f316c006ac0 tx=0x7f316c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.466+0000 7f3188c3c700 1 -- 172.21.15.7:0/788893450 shutdown_connections 2024-01-27T02:33:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.466+0000 7f3188c3c700 1 --2- 172.21.15.7:0/788893450 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3164066f30 0x7f31640693e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.466+0000 7f3188c3c700 1 --2- 172.21.15.7:0/788893450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31840b33f0 0x7f31840c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.466+0000 7f3188c3c700 1 --2- 172.21.15.7:0/788893450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3170004d50 0x7f318414e430 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.466+0000 7f3188c3c700 1 --2- 172.21.15.7:0/788893450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31840b2f80 0x7f31840c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.466+0000 7f3188c3c700 1 -- 172.21.15.7:0/788893450 >> 172.21.15.7:0/788893450 conn(0x7f3184098020 msgr2=0x7f3184151830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.466+0000 7f3188c3c700 1 -- 172.21.15.7:0/788893450 shutdown_connections 2024-01-27T02:33:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.466+0000 7f3188c3c700 1 -- 172.21.15.7:0/788893450 wait complete. 2024-01-27T02:33:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2024-01-27T02:33:12.481 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:33:12.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.610+0000 7fe29ba90700 1 Processor -- start 2024-01-27T02:33:12.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.610+0000 7fe29ba90700 1 -- start start 2024-01-27T02:33:12.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.611+0000 7fe29ba90700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe2940c4cc0 0x7fe2940ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:12.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.611+0000 7fe29ba90700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe2940cd2f0 0x7fe2940679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:12.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.611+0000 7fe29ba90700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe294067ef0 con 0x7fe29415a230 2024-01-27T02:33:12.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.611+0000 7fe29ba90700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2940cd710 con 0x7fe2940cd2f0 2024-01-27T02:33:12.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.611+0000 7fe29ba90700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2940c5320 con 0x7fe2940c4cc0 2024-01-27T02:33:12.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.611+0000 7fe29a02d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe2940cd2f0 0x7fe2940679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:12.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.611+0000 7fe29982c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe29415a230 0x7fe2940c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53652/0 (socket says 172.21.15.7:53652) 2024-01-27T02:33:12.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.611+0000 7fe29982c700 1 -- 172.21.15.7:0/733851432 learned_addr learned my addr 172.21.15.7:0/733851432 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:12.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.611+0000 7fe29902b700 1 --2- 172.21.15.7:0/733851432 >> v2:172.21.15.188:3300/0 conn(0x7fe2940c4cc0 0x7fe2940ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:12.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.612+0000 7fe29a02d700 1 -- 172.21.15.7:0/733851432 >> v2:172.21.15.188:3300/0 conn(0x7fe2940c4cc0 msgr2=0x7fe2940ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:12.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.612+0000 7fe29a02d700 1 --2- 172.21.15.7:0/733851432 >> v2:172.21.15.188:3300/0 conn(0x7fe2940c4cc0 0x7fe2940ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.612+0000 7fe29a02d700 1 -- 172.21.15.7:0/733851432 >> v1:172.21.15.7:6789/0 conn(0x7fe29415a230 legacy=0x7fe2940c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:12.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.612+0000 7fe29882a700 1 -- 172.21.15.7:0/733851432 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423761854 0 0) 0x7fe294067ef0 con 0x7fe29415a230 2024-01-27T02:33:12.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.612+0000 7fe29a02d700 1 -- 172.21.15.7:0/733851432 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2940c9520 con 0x7fe2940cd2f0 2024-01-27T02:33:12.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.612+0000 7fe29a02d700 1 --2- 172.21.15.7:0/733851432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2940cd2f0 0x7fe2940679b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe29406f270 tx=0x7fe28c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b5a40d6f76b87c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:12.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.612+0000 7fe29882a700 1 -- 172.21.15.7:0/733851432 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c00e070 con 0x7fe2940cd2f0 2024-01-27T02:33:12.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.612+0000 7fe29882a700 1 -- 172.21.15.7:0/733851432 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28c00bb60 con 0x7fe2940cd2f0 2024-01-27T02:33:12.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.612+0000 7fe29882a700 1 -- 172.21.15.7:0/733851432 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c012430 con 0x7fe2940cd2f0 2024-01-27T02:33:12.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.613+0000 7fe29ba90700 1 -- 172.21.15.7:0/733851432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2940cd2f0 msgr2=0x7fe2940679b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:12.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.613+0000 7fe29ba90700 1 --2- 172.21.15.7:0/733851432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2940cd2f0 0x7fe2940679b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe29406f270 tx=0x7fe28c009780 comp rx=0 tx=0).stop 2024-01-27T02:33:12.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.613+0000 7fe29ba90700 1 -- 172.21.15.7:0/733851432 shutdown_connections 2024-01-27T02:33:12.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.613+0000 7fe29ba90700 1 --2- 172.21.15.7:0/733851432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2940cd2f0 0x7fe2940679b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.613+0000 7fe29ba90700 1 --2- 172.21.15.7:0/733851432 >> v2:172.21.15.188:3300/0 conn(0x7fe2940c4cc0 0x7fe2940ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.613+0000 7fe29ba90700 1 -- 172.21.15.7:0/733851432 >> 172.21.15.7:0/733851432 conn(0x7fe294098020 msgr2=0x7fe2940711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:12.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.613+0000 7fe29ba90700 1 -- 172.21.15.7:0/733851432 shutdown_connections 2024-01-27T02:33:12.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.613+0000 7fe29ba90700 1 -- 172.21.15.7:0/733851432 wait complete. 2024-01-27T02:33:12.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.614+0000 7fe29ba90700 1 Processor -- start 2024-01-27T02:33:12.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.614+0000 7fe29ba90700 1 -- start start 2024-01-27T02:33:12.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.615+0000 7fe29ba90700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2940c4cc0 0x7fe2940b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:12.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.615+0000 7fe29ba90700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2940cd2f0 0x7fe2941458e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:12.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.615+0000 7fe29ba90700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe29415a230 0x7fe29415dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:12.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.615+0000 7fe29ba90700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2940b2020 con 0x7fe2940cd2f0 2024-01-27T02:33:12.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.615+0000 7fe29ba90700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29415e7b0 con 0x7fe29415a230 2024-01-27T02:33:12.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.615+0000 7fe29ba90700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29415ea90 con 0x7fe2940c4cc0 2024-01-27T02:33:12.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.615+0000 7fe29902b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2940cd2f0 0x7fe2941458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:12.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.615+0000 7fe29902b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2940cd2f0 0x7fe2941458e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60592/0 (socket says 172.21.15.7:60592) 2024-01-27T02:33:12.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.615+0000 7fe29902b700 1 -- 172.21.15.7:0/4082368431 learned_addr learned my addr 172.21.15.7:0/4082368431 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:12.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.615+0000 7fe29982c700 1 --2- 172.21.15.7:0/4082368431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2940c4cc0 0x7fe2940b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:12.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.615+0000 7fe29a02d700 1 --2- 172.21.15.7:0/4082368431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe29415a230 0x7fe29415dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:12.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.615+0000 7fe29902b700 1 -- 172.21.15.7:0/4082368431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2940c4cc0 msgr2=0x7fe2940b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:12.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.615+0000 7fe29902b700 1 --2- 172.21.15.7:0/4082368431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2940c4cc0 0x7fe2940b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.615+0000 7fe29902b700 1 -- 172.21.15.7:0/4082368431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe29415a230 msgr2=0x7fe29415dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:12.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.615+0000 7fe29902b700 1 --2- 172.21.15.7:0/4082368431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe29415a230 0x7fe29415dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.615+0000 7fe29902b700 1 -- 172.21.15.7:0/4082368431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c009a20 con 0x7fe2940cd2f0 2024-01-27T02:33:12.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.616+0000 7fe29902b700 1 --2- 172.21.15.7:0/4082368431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2940cd2f0 0x7fe2941458e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe28400ea70 tx=0x7fe2840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:12.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.616+0000 7fe289ffb700 1 -- 172.21.15.7:0/4082368431 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28400eeb0 con 0x7fe2940cd2f0 2024-01-27T02:33:12.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.616+0000 7fe289ffb700 1 -- 172.21.15.7:0/4082368431 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28400c850 con 0x7fe2940cd2f0 2024-01-27T02:33:12.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.616+0000 7fe289ffb700 1 -- 172.21.15.7:0/4082368431 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28400cc30 con 0x7fe2940cd2f0 2024-01-27T02:33:12.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.616+0000 7fe29ba90700 1 -- 172.21.15.7:0/4082368431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29415ed70 con 0x7fe2940cd2f0 2024-01-27T02:33:12.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.616+0000 7fe29ba90700 1 -- 172.21.15.7:0/4082368431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29415f330 con 0x7fe2940cd2f0 2024-01-27T02:33:12.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.617+0000 7fe29ba90700 1 -- 172.21.15.7:0/4082368431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29404c710 con 0x7fe2940cd2f0 2024-01-27T02:33:12.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.617+0000 7fe289ffb700 1 -- 172.21.15.7:0/4082368431 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe284013680 con 0x7fe2940cd2f0 2024-01-27T02:33:12.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.618+0000 7fe289ffb700 1 --2- 172.21.15.7:0/4082368431 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe278066f30 0x7fe2780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:12.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.618+0000 7fe289ffb700 1 -- 172.21.15.7:0/4082368431 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe284084800 con 0x7fe2940cd2f0 2024-01-27T02:33:12.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.618+0000 7fe29982c700 1 --2- 172.21.15.7:0/4082368431 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe278066f30 0x7fe2780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:12.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.619+0000 7fe29982c700 1 --2- 172.21.15.7:0/4082368431 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe278066f30 0x7fe2780693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe2900029c0 tx=0x7fe290009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:12.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.623+0000 7fe289ffb700 1 -- 172.21.15.7:0/4082368431 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe284051a70 con 0x7fe2940cd2f0 2024-01-27T02:33:12.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.824+0000 7fe29ba90700 1 -- 172.21.15.7:0/4082368431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2940c9800 con 0x7fe2940cd2f0 2024-01-27T02:33:12.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.825+0000 7fe289ffb700 1 -- 172.21.15.7:0/4082368431 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+15302 (secure 0 0 0) 0x7fe284055090 con 0x7fe2940cd2f0 2024-01-27T02:33:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:12.869 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:33:07.888617+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:32:52.897548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-27T02:32:53.822754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:33:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.827+0000 7fe29ba90700 1 -- 172.21.15.7:0/4082368431 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe278066f30 msgr2=0x7fe2780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.827+0000 7fe29ba90700 1 --2- 172.21.15.7:0/4082368431 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe278066f30 0x7fe2780693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe2900029c0 tx=0x7fe290009040 comp rx=0 tx=0).stop 2024-01-27T02:33:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.827+0000 7fe29ba90700 1 -- 172.21.15.7:0/4082368431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2940cd2f0 msgr2=0x7fe2941458e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.827+0000 7fe29ba90700 1 --2- 172.21.15.7:0/4082368431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2940cd2f0 0x7fe2941458e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe28400ea70 tx=0x7fe2840091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.827+0000 7fe29ba90700 1 -- 172.21.15.7:0/4082368431 shutdown_connections 2024-01-27T02:33:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.827+0000 7fe29ba90700 1 --2- 172.21.15.7:0/4082368431 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe278066f30 0x7fe2780693e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.827+0000 7fe29ba90700 1 --2- 172.21.15.7:0/4082368431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe29415a230 0x7fe29415dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.827+0000 7fe29ba90700 1 --2- 172.21.15.7:0/4082368431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2940c4cc0 0x7fe2940b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.827+0000 7fe29ba90700 1 --2- 172.21.15.7:0/4082368431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2940cd2f0 0x7fe2941458e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.827+0000 7fe29ba90700 1 -- 172.21.15.7:0/4082368431 >> 172.21.15.7:0/4082368431 conn(0x7fe294098020 msgr2=0x7fe2940ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.827+0000 7fe29ba90700 1 -- 172.21.15.7:0/4082368431 shutdown_connections 2024-01-27T02:33:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:12.827+0000 7fe29ba90700 1 -- 172.21.15.7:0/4082368431 wait complete. 2024-01-27T02:33:12.872 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-27T02:33:12.872 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:33:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.004+0000 7f9340a60700 1 Processor -- start 2024-01-27T02:33:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.004+0000 7f9340a60700 1 -- start start 2024-01-27T02:33:13.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.005+0000 7f9340a60700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f933c0c8ab0 0x7f933c15dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:13.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.005+0000 7f9340a60700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f933c0cbbc0 0x7f933c0cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:13.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.005+0000 7f9340a60700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f933c15a9b0 con 0x7f933c0c8e80 2024-01-27T02:33:13.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.005+0000 7f9340a60700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f933c15ecb0 con 0x7f933c0c8ab0 2024-01-27T02:33:13.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.005+0000 7f9340a60700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f933c15e410 con 0x7f933c0cbbc0 2024-01-27T02:33:13.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.005+0000 7f933a59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f933c0c8ab0 0x7f933c15dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:13.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.005+0000 7f933a59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f933c0c8ab0 0x7f933c15dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60602/0 (socket says 172.21.15.7:60602) 2024-01-27T02:33:13.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.005+0000 7f933a59c700 1 -- 172.21.15.7:0/2486237853 learned_addr learned my addr 172.21.15.7:0/2486237853 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:13.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.005+0000 7f933ad9d700 1 --2- 172.21.15.7:0/2486237853 >> v2:172.21.15.188:3300/0 conn(0x7f933c0cbbc0 0x7f933c0cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:13.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.005+0000 7f933a59c700 1 -- 172.21.15.7:0/2486237853 >> v2:172.21.15.188:3300/0 conn(0x7f933c0cbbc0 msgr2=0x7f933c0cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:13.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.005+0000 7f933a59c700 1 --2- 172.21.15.7:0/2486237853 >> v2:172.21.15.188:3300/0 conn(0x7f933c0cbbc0 0x7f933c0cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.006+0000 7f933a59c700 1 -- 172.21.15.7:0/2486237853 >> v1:172.21.15.188:6789/0 conn(0x7f933c0c8e80 legacy=0x7f933c0cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:13.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.006+0000 7f933a59c700 1 -- 172.21.15.7:0/2486237853 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933c15e6f0 con 0x7f933c0c8ab0 2024-01-27T02:33:13.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.006+0000 7f933a59c700 1 --2- 172.21.15.7:0/2486237853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933c0c8ab0 0x7f933c15dd80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f933c15a920 tx=0x7f9330009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6e653829b6413c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:13.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.006+0000 7f933959a700 1 -- 172.21.15.7:0/2486237853 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330009e70 con 0x7f933c0c8ab0 2024-01-27T02:33:13.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.006+0000 7f933959a700 1 -- 172.21.15.7:0/2486237853 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933000b860 con 0x7f933c0c8ab0 2024-01-27T02:33:13.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.006+0000 7f933959a700 1 -- 172.21.15.7:0/2486237853 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933000bc00 con 0x7f933c0c8ab0 2024-01-27T02:33:13.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.006+0000 7f9340a60700 1 -- 172.21.15.7:0/2486237853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933c0c8ab0 msgr2=0x7f933c15dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:13.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.006+0000 7f9340a60700 1 --2- 172.21.15.7:0/2486237853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933c0c8ab0 0x7f933c15dd80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f933c15a920 tx=0x7f9330009250 comp rx=0 tx=0).stop 2024-01-27T02:33:13.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.006+0000 7f9340a60700 1 -- 172.21.15.7:0/2486237853 shutdown_connections 2024-01-27T02:33:13.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.006+0000 7f9340a60700 1 --2- 172.21.15.7:0/2486237853 >> v2:172.21.15.188:3300/0 conn(0x7f933c0cbbc0 0x7f933c0cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.006+0000 7f9340a60700 1 --2- 172.21.15.7:0/2486237853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933c0c8ab0 0x7f933c15dd80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.007+0000 7f9340a60700 1 -- 172.21.15.7:0/2486237853 >> 172.21.15.7:0/2486237853 conn(0x7f933c0cdc30 msgr2=0x7f933c0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:13.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.007+0000 7f9340a60700 1 -- 172.21.15.7:0/2486237853 shutdown_connections 2024-01-27T02:33:13.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.007+0000 7f9340a60700 1 -- 172.21.15.7:0/2486237853 wait complete. 2024-01-27T02:33:13.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.008+0000 7f9340a60700 1 Processor -- start 2024-01-27T02:33:13.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.008+0000 7f9340a60700 1 -- start start 2024-01-27T02:33:13.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.008+0000 7f9340a60700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f933c0c8ab0 0x7f933c16b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:13.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.008+0000 7f9340a60700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f933c0c8e80 0x7f933c16a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.008+0000 7f9340a60700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933c0cbbc0 0x7f933c1696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.008+0000 7f9340a60700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c16adf0 con 0x7f933c0cbbc0 2024-01-27T02:33:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.008+0000 7f9340a60700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0ad530 con 0x7f933c0c8e80 2024-01-27T02:33:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.008+0000 7f9340a60700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0ad810 con 0x7f933c0c8ab0 2024-01-27T02:33:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.009+0000 7f933ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933c0cbbc0 0x7f933c1696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.009+0000 7f933ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933c0cbbc0 0x7f933c1696b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60610/0 (socket says 172.21.15.7:60610) 2024-01-27T02:33:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.009+0000 7f933ad9d700 1 -- 172.21.15.7:0/2987317576 learned_addr learned my addr 172.21.15.7:0/2987317576 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.009+0000 7f9339d9b700 1 --2- 172.21.15.7:0/2987317576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f933c0c8e80 0x7f933c16a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.009+0000 7f933a59c700 1 --2- 172.21.15.7:0/2987317576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f933c0c8ab0 0x7f933c16b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.009+0000 7f933ad9d700 1 -- 172.21.15.7:0/2987317576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f933c0c8ab0 msgr2=0x7f933c16b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.009+0000 7f933ad9d700 1 --2- 172.21.15.7:0/2987317576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f933c0c8ab0 0x7f933c16b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.009+0000 7f933ad9d700 1 -- 172.21.15.7:0/2987317576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f933c0c8e80 msgr2=0x7f933c16a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.009+0000 7f933ad9d700 1 --2- 172.21.15.7:0/2987317576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f933c0c8e80 0x7f933c16a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.009+0000 7f933ad9d700 1 -- 172.21.15.7:0/2987317576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93300094f0 con 0x7f933c0cbbc0 2024-01-27T02:33:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.009+0000 7f933ad9d700 1 --2- 172.21.15.7:0/2987317576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933c0cbbc0 0x7f933c1696b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9334009e90 tx=0x7f933400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.009+0000 7f932a7fc700 1 -- 172.21.15.7:0/2987317576 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933400cb70 con 0x7f933c0cbbc0 2024-01-27T02:33:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.009+0000 7f932a7fc700 1 -- 172.21.15.7:0/2987317576 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933400f040 con 0x7f933c0cbbc0 2024-01-27T02:33:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.009+0000 7f9340a60700 1 -- 172.21.15.7:0/2987317576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933c0adaf0 con 0x7f933c0cbbc0 2024-01-27T02:33:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.009+0000 7f932a7fc700 1 -- 172.21.15.7:0/2987317576 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9334014480 con 0x7f933c0cbbc0 2024-01-27T02:33:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.009+0000 7f9340a60700 1 -- 172.21.15.7:0/2987317576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933c0ae0b0 con 0x7f933c0cbbc0 2024-01-27T02:33:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.010+0000 7f9340a60700 1 -- 172.21.15.7:0/2987317576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f933c04c710 con 0x7f933c0cbbc0 2024-01-27T02:33:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.011+0000 7f932a7fc700 1 -- 172.21.15.7:0/2987317576 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9334014830 con 0x7f933c0cbbc0 2024-01-27T02:33:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.012+0000 7f932a7fc700 1 --2- 172.21.15.7:0/2987317576 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f931c066f30 0x7f931c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.012+0000 7f932a7fc700 1 -- 172.21.15.7:0/2987317576 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9334084f00 con 0x7f933c0cbbc0 2024-01-27T02:33:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.012+0000 7f933a59c700 1 --2- 172.21.15.7:0/2987317576 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f931c066f30 0x7f931c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.013+0000 7f933a59c700 1 --2- 172.21.15.7:0/2987317576 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f931c066f30 0x7f931c0693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f933c15a920 tx=0x7f9330017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.016+0000 7f932a7fc700 1 -- 172.21.15.7:0/2987317576 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9334052170 con 0x7f933c0cbbc0 2024-01-27T02:33:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.268+0000 7f9340a60700 1 -- 172.21.15.7:0/2987317576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f933c0cb050 con 0x7f933c0cbbc0 2024-01-27T02:33:13.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.268+0000 7f932a7fc700 1 -- 172.21.15.7:0/2987317576 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f9334055790 con 0x7f933c0cbbc0 2024-01-27T02:33:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:13.270 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4864,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/892460188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":892460188},{"type":"v1","addr":"172.21.15.188:6837","nonce":892460188}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4867,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1657319349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1657319349},{"type":"v1","addr":"172.21.15.188:6836","nonce":1657319349}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4879,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2190571534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2190571534},{"type":"v1","addr":"172.21.15.7:6834","nonce":2190571534}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:32:54.849803+0000","modified":"2024-01-27T02:32:58.809118+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4882},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4882":{"gid":4882,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6835/1734594727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1734594727},{"type":"v1","addr":"172.21.15.7:6835","nonce":1734594727}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-01-27T02:33:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.271+0000 7f9340a60700 1 -- 172.21.15.7:0/2987317576 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f931c066f30 msgr2=0x7f931c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.271+0000 7f9340a60700 1 --2- 172.21.15.7:0/2987317576 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f931c066f30 0x7f931c0693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f933c15a920 tx=0x7f9330017000 comp rx=0 tx=0).stop 2024-01-27T02:33:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.271+0000 7f9340a60700 1 -- 172.21.15.7:0/2987317576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933c0cbbc0 msgr2=0x7f933c1696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.271+0000 7f9340a60700 1 --2- 172.21.15.7:0/2987317576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933c0cbbc0 0x7f933c1696b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9334009e90 tx=0x7f933400b040 comp rx=0 tx=0).stop 2024-01-27T02:33:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.271+0000 7f9340a60700 1 -- 172.21.15.7:0/2987317576 shutdown_connections 2024-01-27T02:33:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.271+0000 7f9340a60700 1 --2- 172.21.15.7:0/2987317576 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f931c066f30 0x7f931c0693e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.271+0000 7f9340a60700 1 --2- 172.21.15.7:0/2987317576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933c0cbbc0 0x7f933c1696b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.271+0000 7f9340a60700 1 --2- 172.21.15.7:0/2987317576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f933c0c8ab0 0x7f933c16b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.271+0000 7f9340a60700 1 --2- 172.21.15.7:0/2987317576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f933c0c8e80 0x7f933c16a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.271+0000 7f9340a60700 1 -- 172.21.15.7:0/2987317576 >> 172.21.15.7:0/2987317576 conn(0x7f933c0cdc30 msgr2=0x7f933c0ca5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.271+0000 7f9340a60700 1 -- 172.21.15.7:0/2987317576 shutdown_connections 2024-01-27T02:33:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.271+0000 7f9340a60700 1 -- 172.21.15.7:0/2987317576 wait complete. 2024-01-27T02:33:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2024-01-27T02:33:13.286 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:33:13.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.415+0000 7f22cfeeb700 1 Processor -- start 2024-01-27T02:33:13.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.415+0000 7f22cfeeb700 1 -- start start 2024-01-27T02:33:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.416+0000 7f22cfeeb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c80d10d0 0x7f22c80b2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.416+0000 7f22cfeeb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22c80b2d40 0x7f22c815e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.416+0000 7f22cfeeb700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22c80b41d0 con 0x7f22c80d14a0 2024-01-27T02:33:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.416+0000 7f22cfeeb700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22c80b3900 con 0x7f22c80b2d40 2024-01-27T02:33:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.416+0000 7f22cfeeb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c80b3340 con 0x7f22c80d10d0 2024-01-27T02:33:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.416+0000 7f22ce488700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22c80b2d40 0x7f22c815e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.416+0000 7f22ce488700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22c80b2d40 0x7f22c815e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60620/0 (socket says 172.21.15.7:60620) 2024-01-27T02:33:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.416+0000 7f22ce488700 1 -- 172.21.15.7:0/217012412 learned_addr learned my addr 172.21.15.7:0/217012412 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.417+0000 7f22cdc87700 1 --2- 172.21.15.7:0/217012412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c80d10d0 0x7f22c80b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.417+0000 7f22ce488700 1 -- 172.21.15.7:0/217012412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c80d10d0 msgr2=0x7f22c80b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.417+0000 7f22ce488700 1 --2- 172.21.15.7:0/217012412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c80d10d0 0x7f22c80b2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.417+0000 7f22ce488700 1 -- 172.21.15.7:0/217012412 >> v1:172.21.15.188:6789/0 conn(0x7f22c80d14a0 legacy=0x7f22c815de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.417+0000 7f22ce488700 1 -- 172.21.15.7:0/217012412 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c80b3620 con 0x7f22c80b2d40 2024-01-27T02:33:13.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.417+0000 7f22ce488700 1 --2- 172.21.15.7:0/217012412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c80b2d40 0x7f22c815e560 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f22c0009f20 tx=0x7f22c0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18298006b882f176 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:13.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.417+0000 7f22ccc85700 1 -- 172.21.15.7:0/217012412 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c000e070 con 0x7f22c80b2d40 2024-01-27T02:33:13.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.417+0000 7f22ccc85700 1 -- 172.21.15.7:0/217012412 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c000ba80 con 0x7f22c80b2d40 2024-01-27T02:33:13.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.417+0000 7f22ccc85700 1 -- 172.21.15.7:0/217012412 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c00092e0 con 0x7f22c80b2d40 2024-01-27T02:33:13.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.418+0000 7f22cfeeb700 1 -- 172.21.15.7:0/217012412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c80b2d40 msgr2=0x7f22c815e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:13.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.418+0000 7f22cfeeb700 1 --2- 172.21.15.7:0/217012412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c80b2d40 0x7f22c815e560 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f22c0009f20 tx=0x7f22c0009840 comp rx=0 tx=0).stop 2024-01-27T02:33:13.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.418+0000 7f22cfeeb700 1 -- 172.21.15.7:0/217012412 shutdown_connections 2024-01-27T02:33:13.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.418+0000 7f22cfeeb700 1 --2- 172.21.15.7:0/217012412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c80b2d40 0x7f22c815e560 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.418+0000 7f22cfeeb700 1 --2- 172.21.15.7:0/217012412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c80d10d0 0x7f22c80b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.418+0000 7f22cfeeb700 1 -- 172.21.15.7:0/217012412 >> 172.21.15.7:0/217012412 conn(0x7f22c80679b0 msgr2=0x7f22c8067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:13.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.418+0000 7f22cfeeb700 1 -- 172.21.15.7:0/217012412 shutdown_connections 2024-01-27T02:33:13.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.418+0000 7f22cfeeb700 1 -- 172.21.15.7:0/217012412 wait complete. 2024-01-27T02:33:13.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.419+0000 7f22cfeeb700 1 Processor -- start 2024-01-27T02:33:13.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.419+0000 7f22cfeeb700 1 -- start start 2024-01-27T02:33:13.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.420+0000 7f22cfeeb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c80b2d40 0x7f22c81520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:13.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.420+0000 7f22cfeeb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c80d10d0 0x7f22c80c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:13.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.420+0000 7f22cfeeb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c80d14a0 0x7f22c80c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:13.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.420+0000 7f22cfeeb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c8170780 con 0x7f22c80d10d0 2024-01-27T02:33:13.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.420+0000 7f22cfeeb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c816fb70 con 0x7f22c80d14a0 2024-01-27T02:33:13.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.420+0000 7f22cfeeb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c816fe50 con 0x7f22c80b2d40 2024-01-27T02:33:13.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.420+0000 7f22cd486700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c80d10d0 0x7f22c80c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:13.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.420+0000 7f22cd486700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c80d10d0 0x7f22c80c8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60634/0 (socket says 172.21.15.7:60634) 2024-01-27T02:33:13.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.420+0000 7f22cd486700 1 -- 172.21.15.7:0/4075320755 learned_addr learned my addr 172.21.15.7:0/4075320755 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:13.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.420+0000 7f22ce488700 1 --2- 172.21.15.7:0/4075320755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c80d14a0 0x7f22c80c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:13.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.420+0000 7f22cdc87700 1 --2- 172.21.15.7:0/4075320755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c80b2d40 0x7f22c81520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:13.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.420+0000 7f22cd486700 1 -- 172.21.15.7:0/4075320755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c80b2d40 msgr2=0x7f22c81520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:13.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.420+0000 7f22cd486700 1 --2- 172.21.15.7:0/4075320755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c80b2d40 0x7f22c81520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.420+0000 7f22cd486700 1 -- 172.21.15.7:0/4075320755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c80d14a0 msgr2=0x7f22c80c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:13.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.420+0000 7f22cd486700 1 --2- 172.21.15.7:0/4075320755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c80d14a0 0x7f22c80c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.420+0000 7f22cd486700 1 -- 172.21.15.7:0/4075320755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c0009ae0 con 0x7f22c80d10d0 2024-01-27T02:33:13.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.420+0000 7f22cdc87700 1 --2- 172.21.15.7:0/4075320755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c80b2d40 0x7f22c81520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:33:13.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.421+0000 7f22cd486700 1 --2- 172.21.15.7:0/4075320755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c80d10d0 0x7f22c80c8910 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00c590 tx=0x7f22bc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:13.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.421+0000 7f22b9ffb700 1 -- 172.21.15.7:0/4075320755 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc011590 con 0x7f22c80d10d0 2024-01-27T02:33:13.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.421+0000 7f22b9ffb700 1 -- 172.21.15.7:0/4075320755 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22bc0116f0 con 0x7f22c80d10d0 2024-01-27T02:33:13.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.421+0000 7f22cfeeb700 1 -- 172.21.15.7:0/4075320755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c8170130 con 0x7f22c80d10d0 2024-01-27T02:33:13.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.421+0000 7f22b9ffb700 1 -- 172.21.15.7:0/4075320755 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc011ab0 con 0x7f22c80d10d0 2024-01-27T02:33:13.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.421+0000 7f22cfeeb700 1 -- 172.21.15.7:0/4075320755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c816f840 con 0x7f22c80d10d0 2024-01-27T02:33:13.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.422+0000 7f22cfeeb700 1 -- 172.21.15.7:0/4075320755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c8170930 con 0x7f22c80d10d0 2024-01-27T02:33:13.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.423+0000 7f22b9ffb700 1 -- 172.21.15.7:0/4075320755 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f22bc01b3b0 con 0x7f22c80d10d0 2024-01-27T02:33:13.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.423+0000 7f22b9ffb700 1 --2- 172.21.15.7:0/4075320755 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f22ac066f30 0x7f22ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:13.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.423+0000 7f22b9ffb700 1 -- 172.21.15.7:0/4075320755 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f22bc083a90 con 0x7f22c80d10d0 2024-01-27T02:33:13.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.423+0000 7f22cdc87700 1 --2- 172.21.15.7:0/4075320755 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f22ac066f30 0x7f22ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:13.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.424+0000 7f22cdc87700 1 --2- 172.21.15.7:0/4075320755 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f22ac066f30 0x7f22ac0693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f22c4006fd0 tx=0x7f22c4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:13.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.428+0000 7f22b9ffb700 1 -- 172.21.15.7:0/4075320755 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22c8170930 con 0x7f22c80d10d0 2024-01-27T02:33:13.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.678+0000 7f22cfeeb700 1 -- 172.21.15.7:0/4075320755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22c804c710 con 0x7f22c80d10d0 2024-01-27T02:33:13.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.678+0000 7f22b9ffb700 1 -- 172.21.15.7:0/4075320755 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f22bc017020 con 0x7f22c80d10d0 2024-01-27T02:33:13.681 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:13.681 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4864,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/892460188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":892460188},{"type":"v1","addr":"172.21.15.188:6837","nonce":892460188}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4867,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1657319349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1657319349},{"type":"v1","addr":"172.21.15.188:6836","nonce":1657319349}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4879,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2190571534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2190571534},{"type":"v1","addr":"172.21.15.7:6834","nonce":2190571534}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:32:54.849803+0000","modified":"2024-01-27T02:32:58.809118+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4882},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4882":{"gid":4882,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6835/1734594727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1734594727},{"type":"v1","addr":"172.21.15.7:6835","nonce":1734594727}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-01-27T02:33:13.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.680+0000 7f22cfeeb700 1 -- 172.21.15.7:0/4075320755 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f22ac066f30 msgr2=0x7f22ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:13.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.680+0000 7f22cfeeb700 1 --2- 172.21.15.7:0/4075320755 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f22ac066f30 0x7f22ac0693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f22c4006fd0 tx=0x7f22c4008040 comp rx=0 tx=0).stop 2024-01-27T02:33:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.681+0000 7f22cfeeb700 1 -- 172.21.15.7:0/4075320755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c80d10d0 msgr2=0x7f22c80c8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.681+0000 7f22cfeeb700 1 --2- 172.21.15.7:0/4075320755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c80d10d0 0x7f22c80c8910 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00c590 tx=0x7f22bc00c950 comp rx=0 tx=0).stop 2024-01-27T02:33:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.681+0000 7f22cfeeb700 1 -- 172.21.15.7:0/4075320755 shutdown_connections 2024-01-27T02:33:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.681+0000 7f22cfeeb700 1 --2- 172.21.15.7:0/4075320755 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f22ac066f30 0x7f22ac0693e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.681+0000 7f22cfeeb700 1 --2- 172.21.15.7:0/4075320755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c80d14a0 0x7f22c80c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.681+0000 7f22cfeeb700 1 --2- 172.21.15.7:0/4075320755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c80b2d40 0x7f22c81520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.681+0000 7f22cfeeb700 1 --2- 172.21.15.7:0/4075320755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22c80d10d0 0x7f22c80c8910 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.681+0000 7f22cfeeb700 1 -- 172.21.15.7:0/4075320755 >> 172.21.15.7:0/4075320755 conn(0x7f22c80679b0 msgr2=0x7f22c80cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.681+0000 7f22cfeeb700 1 -- 172.21.15.7:0/4075320755 shutdown_connections 2024-01-27T02:33:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.681+0000 7f22cfeeb700 1 -- 172.21.15.7:0/4075320755 wait complete. 2024-01-27T02:33:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2024-01-27T02:33:13.696 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:33:13.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.826+0000 7f0a01e43700 1 Processor -- start 2024-01-27T02:33:13.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.826+0000 7f0a01e43700 1 -- start start 2024-01-27T02:33:13.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.827+0000 7f0a01e43700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f09fc157ca0 0x7f09fc0c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:13.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.827+0000 7f0a01e43700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f09fc0c3ae0 0x7f09fc0cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:13.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.827+0000 7f0a01e43700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09fc158070 0x7f09fc0cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:13.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.827+0000 7f0a01e43700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09fc0ca020 con 0x7f09fc0c3ae0 2024-01-27T02:33:13.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.827+0000 7f0a01e43700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09fc0c4f40 con 0x7f09fc157ca0 2024-01-27T02:33:13.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.827+0000 7f0a01e43700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc0c5210 con 0x7f09fc158070 2024-01-27T02:33:13.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.827+0000 7f09faffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f09fc0c3ae0 0x7f09fc0cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:13.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.827+0000 7f09faffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f09fc0c3ae0 0x7f09fc0cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60646/0 (socket says 172.21.15.7:60646) 2024-01-27T02:33:13.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.827+0000 7f09faffd700 1 -- 172.21.15.7:0/1950699236 learned_addr learned my addr 172.21.15.7:0/1950699236 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:13.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.827+0000 7f09fb7fe700 1 --2- 172.21.15.7:0/1950699236 >> v2:172.21.15.188:3300/0 conn(0x7f09fc157ca0 0x7f09fc0c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:13.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.827+0000 7f09fbfff700 1 --2- 172.21.15.7:0/1950699236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09fc158070 0x7f09fc0cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:13.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.827+0000 7f09faffd700 1 -- 172.21.15.7:0/1950699236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09fc158070 msgr2=0x7f09fc0cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:13.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.827+0000 7f09faffd700 1 --2- 172.21.15.7:0/1950699236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09fc158070 0x7f09fc0cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.827+0000 7f09faffd700 1 -- 172.21.15.7:0/1950699236 >> v2:172.21.15.188:3300/0 conn(0x7f09fc157ca0 msgr2=0x7f09fc0c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:13.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.827+0000 7f09faffd700 1 --2- 172.21.15.7:0/1950699236 >> v2:172.21.15.188:3300/0 conn(0x7f09fc157ca0 0x7f09fc0c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.827+0000 7f09faffd700 1 -- 172.21.15.7:0/1950699236 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc0c54f0 con 0x7f09fc0c3ae0 2024-01-27T02:33:13.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.827+0000 7f09fbfff700 1 --2- 172.21.15.7:0/1950699236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09fc158070 0x7f09fc0cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:33:13.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.827+0000 7f09fb7fe700 1 --2- 172.21.15.7:0/1950699236 >> v2:172.21.15.188:3300/0 conn(0x7f09fc157ca0 0x7f09fc0c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:33:13.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.828+0000 7f09faffd700 1 --2- 172.21.15.7:0/1950699236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc0c3ae0 0x7f09fc0cd4e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f09fc06d710 tx=0x7f09f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0cf5463342b9c0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:13.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.828+0000 7f09fa7fc700 1 -- 172.21.15.7:0/1950699236 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f000b860 con 0x7f09fc0c3ae0 2024-01-27T02:33:13.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.828+0000 7f09fa7fc700 1 -- 172.21.15.7:0/1950699236 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f000b9c0 con 0x7f09fc0c3ae0 2024-01-27T02:33:13.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.828+0000 7f09fa7fc700 1 -- 172.21.15.7:0/1950699236 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f000bd60 con 0x7f09fc0c3ae0 2024-01-27T02:33:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.828+0000 7f0a01e43700 1 -- 172.21.15.7:0/1950699236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc0c3ae0 msgr2=0x7f09fc0cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.828+0000 7f0a01e43700 1 --2- 172.21.15.7:0/1950699236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc0c3ae0 0x7f09fc0cd4e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f09fc06d710 tx=0x7f09f0009250 comp rx=0 tx=0).stop 2024-01-27T02:33:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.829+0000 7f0a01e43700 1 -- 172.21.15.7:0/1950699236 shutdown_connections 2024-01-27T02:33:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.829+0000 7f0a01e43700 1 --2- 172.21.15.7:0/1950699236 >> v2:172.21.15.188:3300/0 conn(0x7f09fc157ca0 0x7f09fc0c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.829+0000 7f0a01e43700 1 --2- 172.21.15.7:0/1950699236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09fc158070 0x7f09fc0cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.829+0000 7f0a01e43700 1 --2- 172.21.15.7:0/1950699236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc0c3ae0 0x7f09fc0cd4e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.829+0000 7f0a01e43700 1 -- 172.21.15.7:0/1950699236 >> 172.21.15.7:0/1950699236 conn(0x7f09fc098020 msgr2=0x7f09fc0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.829+0000 7f0a01e43700 1 -- 172.21.15.7:0/1950699236 shutdown_connections 2024-01-27T02:33:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.829+0000 7f0a01e43700 1 -- 172.21.15.7:0/1950699236 wait complete. 2024-01-27T02:33:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.830+0000 7f0a01e43700 1 Processor -- start 2024-01-27T02:33:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.830+0000 7f0a01e43700 1 -- start start 2024-01-27T02:33:13.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.831+0000 7f0a01e43700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09fc0c3ae0 0x7f09fc0c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:13.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.831+0000 7f0a01e43700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09fc157ca0 0x7f09fc14e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:13.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.831+0000 7f0a01e43700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc158070 0x7f09fc0b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:13.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.831+0000 7f0a01e43700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc0c79f0 con 0x7f09fc158070 2024-01-27T02:33:13.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.831+0000 7f0a01e43700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc14eeb0 con 0x7f09fc157ca0 2024-01-27T02:33:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.831+0000 7f0a01e43700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc0b3dd0 con 0x7f09fc0c3ae0 2024-01-27T02:33:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.831+0000 7f09fbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc158070 0x7f09fc0b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.831+0000 7f09fbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc158070 0x7f09fc0b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60648/0 (socket says 172.21.15.7:60648) 2024-01-27T02:33:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.831+0000 7f09fbfff700 1 -- 172.21.15.7:0/3079809185 learned_addr learned my addr 172.21.15.7:0/3079809185 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.831+0000 7f09faffd700 1 --2- 172.21.15.7:0/3079809185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09fc157ca0 0x7f09fc14e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.831+0000 7f09fb7fe700 1 --2- 172.21.15.7:0/3079809185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09fc0c3ae0 0x7f09fc0c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.831+0000 7f09fbfff700 1 -- 172.21.15.7:0/3079809185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09fc0c3ae0 msgr2=0x7f09fc0c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.831+0000 7f09fbfff700 1 --2- 172.21.15.7:0/3079809185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09fc0c3ae0 0x7f09fc0c67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.831+0000 7f09fbfff700 1 -- 172.21.15.7:0/3079809185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09fc157ca0 msgr2=0x7f09fc14e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.831+0000 7f09fbfff700 1 --2- 172.21.15.7:0/3079809185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09fc157ca0 0x7f09fc14e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.831+0000 7f09fbfff700 1 -- 172.21.15.7:0/3079809185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f00094f0 con 0x7f09fc158070 2024-01-27T02:33:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.831+0000 7f09fb7fe700 1 --2- 172.21.15.7:0/3079809185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09fc0c3ae0 0x7f09fc0c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:33:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.831+0000 7f09fbfff700 1 --2- 172.21.15.7:0/3079809185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc158070 0x7f09fc0b3890 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f09f400cf20 tx=0x7f09f4007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.832+0000 7f09e37fe700 1 -- 172.21.15.7:0/3079809185 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f400adf0 con 0x7f09fc158070 2024-01-27T02:33:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.832+0000 7f09e37fe700 1 -- 172.21.15.7:0/3079809185 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f4016420 con 0x7f09fc158070 2024-01-27T02:33:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.832+0000 7f0a01e43700 1 -- 172.21.15.7:0/3079809185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09fc0c6de0 con 0x7f09fc158070 2024-01-27T02:33:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.832+0000 7f09e37fe700 1 -- 172.21.15.7:0/3079809185 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f40167e0 con 0x7f09fc158070 2024-01-27T02:33:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.832+0000 7f0a01e43700 1 -- 172.21.15.7:0/3079809185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09fc0c73a0 con 0x7f09fc158070 2024-01-27T02:33:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.833+0000 7f0a01e43700 1 -- 172.21.15.7:0/3079809185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09fc165f90 con 0x7f09fc158070 2024-01-27T02:33:13.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.833+0000 7f09e37fe700 1 -- 172.21.15.7:0/3079809185 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f09f4016bd0 con 0x7f09fc158070 2024-01-27T02:33:13.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.834+0000 7f09e37fe700 1 --2- 172.21.15.7:0/3079809185 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f09e8066f30 0x7f09e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:13.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.834+0000 7f09e37fe700 1 -- 172.21.15.7:0/3079809185 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f09f4012070 con 0x7f09fc158070 2024-01-27T02:33:13.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.834+0000 7f09fb7fe700 1 --2- 172.21.15.7:0/3079809185 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f09e8066f30 0x7f09e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:13.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.834+0000 7f09fb7fe700 1 --2- 172.21.15.7:0/3079809185 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f09e8066f30 0x7f09e80693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f09ec008a80 tx=0x7f09ec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:13.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:13.838+0000 7f09e37fe700 1 -- 172.21.15.7:0/3079809185 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09f404e5a0 con 0x7f09fc158070 2024-01-27T02:33:14.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.045+0000 7f0a01e43700 1 -- 172.21.15.7:0/3079809185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09fc04c710 con 0x7f09fc158070 2024-01-27T02:33:14.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.046+0000 7f09e37fe700 1 -- 172.21.15.7:0/3079809185 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f09f4055c40 con 0x7f09fc158070 2024-01-27T02:33:14.047 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:14.047 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:33:07.888617+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:32:52.897548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-27T02:32:53.822754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:33:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.048+0000 7f0a01e43700 1 -- 172.21.15.7:0/3079809185 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f09e8066f30 msgr2=0x7f09e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.048+0000 7f0a01e43700 1 --2- 172.21.15.7:0/3079809185 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f09e8066f30 0x7f09e80693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f09ec008a80 tx=0x7f09ec008040 comp rx=0 tx=0).stop 2024-01-27T02:33:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.048+0000 7f0a01e43700 1 -- 172.21.15.7:0/3079809185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc158070 msgr2=0x7f09fc0b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.048+0000 7f0a01e43700 1 --2- 172.21.15.7:0/3079809185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc158070 0x7f09fc0b3890 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f09f400cf20 tx=0x7f09f4007860 comp rx=0 tx=0).stop 2024-01-27T02:33:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.048+0000 7f0a01e43700 1 -- 172.21.15.7:0/3079809185 shutdown_connections 2024-01-27T02:33:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.048+0000 7f0a01e43700 1 --2- 172.21.15.7:0/3079809185 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f09e8066f30 0x7f09e80693e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.048+0000 7f0a01e43700 1 --2- 172.21.15.7:0/3079809185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc158070 0x7f09fc0b3890 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.048+0000 7f0a01e43700 1 --2- 172.21.15.7:0/3079809185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09fc0c3ae0 0x7f09fc0c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.048+0000 7f0a01e43700 1 --2- 172.21.15.7:0/3079809185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09fc157ca0 0x7f09fc14e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.048+0000 7f0a01e43700 1 -- 172.21.15.7:0/3079809185 >> 172.21.15.7:0/3079809185 conn(0x7f09fc098020 msgr2=0x7f09fc0cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.048+0000 7f0a01e43700 1 -- 172.21.15.7:0/3079809185 shutdown_connections 2024-01-27T02:33:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.048+0000 7f0a01e43700 1 -- 172.21.15.7:0/3079809185 wait complete. 2024-01-27T02:33:14.063 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-27T02:33:14.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.194+0000 7f889e96f700 1 Processor -- start 2024-01-27T02:33:14.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.194+0000 7f889e96f700 1 -- start start 2024-01-27T02:33:14.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.195+0000 7f889e96f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f88980ca140 0x7f88980ca510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:14.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.195+0000 7f889e96f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f88980b1f50 0x7f88980ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:14.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.195+0000 7f889e96f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88980b2350 con 0x7f88980b1af0 2024-01-27T02:33:14.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.195+0000 7f889e96f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88980c4bd0 con 0x7f88980b1f50 2024-01-27T02:33:14.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.195+0000 7f889e96f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88980caea0 con 0x7f88980ca140 2024-01-27T02:33:14.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.195+0000 7f889cf0c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f88980b1f50 0x7f88980ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:14.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.195+0000 7f889cf0c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f88980b1f50 0x7f88980ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60662/0 (socket says 172.21.15.7:60662) 2024-01-27T02:33:14.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.195+0000 7f88977fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f88980b1af0 0x7f88980c5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53660/0 (socket says 172.21.15.7:53660) 2024-01-27T02:33:14.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.195+0000 7f889cf0c700 1 -- 172.21.15.7:0/4130481243 learned_addr learned my addr 172.21.15.7:0/4130481243 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:14.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.195+0000 7f8897fff700 1 --2- 172.21.15.7:0/4130481243 >> v2:172.21.15.188:3300/0 conn(0x7f88980ca140 0x7f88980ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:14.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.196+0000 7f889cf0c700 1 -- 172.21.15.7:0/4130481243 >> v2:172.21.15.188:3300/0 conn(0x7f88980ca140 msgr2=0x7f88980ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:14.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.196+0000 7f889cf0c700 1 --2- 172.21.15.7:0/4130481243 >> v2:172.21.15.188:3300/0 conn(0x7f88980ca140 0x7f88980ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:14.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.196+0000 7f889cf0c700 1 -- 172.21.15.7:0/4130481243 >> v1:172.21.15.7:6789/0 conn(0x7f88980b1af0 legacy=0x7f88980c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:14.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.196+0000 7f889cf0c700 1 -- 172.21.15.7:0/4130481243 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88980cb180 con 0x7f88980b1f50 2024-01-27T02:33:14.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.196+0000 7f8896ffd700 1 -- 172.21.15.7:0/4130481243 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1876679436 0 0) 0x7f888c001110 con 0x7f88980b1af0 2024-01-27T02:33:14.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.196+0000 7f8897fff700 1 --2- 172.21.15.7:0/4130481243 >> v2:172.21.15.188:3300/0 conn(0x7f88980ca140 0x7f88980ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:14.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.196+0000 7f889cf0c700 1 --2- 172.21.15.7:0/4130481243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88980b1f50 0x7f88980ccb10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f88980b4e20 tx=0x7f8890009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5904a6731f4ba8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:14.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.196+0000 7f8896ffd700 1 -- 172.21.15.7:0/4130481243 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889000e070 con 0x7f88980b1f50 2024-01-27T02:33:14.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.196+0000 7f8896ffd700 1 -- 172.21.15.7:0/4130481243 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889000ba50 con 0x7f88980b1f50 2024-01-27T02:33:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.196+0000 7f8896ffd700 1 -- 172.21.15.7:0/4130481243 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88900092e0 con 0x7f88980b1f50 2024-01-27T02:33:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.196+0000 7f889e96f700 1 -- 172.21.15.7:0/4130481243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88980b1f50 msgr2=0x7f88980ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.196+0000 7f889e96f700 1 --2- 172.21.15.7:0/4130481243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88980b1f50 0x7f88980ccb10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f88980b4e20 tx=0x7f8890009840 comp rx=0 tx=0).stop 2024-01-27T02:33:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.197+0000 7f889e96f700 1 -- 172.21.15.7:0/4130481243 shutdown_connections 2024-01-27T02:33:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.197+0000 7f889e96f700 1 --2- 172.21.15.7:0/4130481243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88980b1f50 0x7f88980ccb10 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.197+0000 7f889e96f700 1 --2- 172.21.15.7:0/4130481243 >> v2:172.21.15.188:3300/0 conn(0x7f88980ca140 0x7f88980ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.197+0000 7f889e96f700 1 -- 172.21.15.7:0/4130481243 >> 172.21.15.7:0/4130481243 conn(0x7f88980711c0 msgr2=0x7f88980ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.197+0000 7f889e96f700 1 -- 172.21.15.7:0/4130481243 shutdown_connections 2024-01-27T02:33:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.197+0000 7f889e96f700 1 -- 172.21.15.7:0/4130481243 wait complete. 2024-01-27T02:33:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.198+0000 7f889e96f700 1 Processor -- start 2024-01-27T02:33:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.198+0000 7f889e96f700 1 -- start start 2024-01-27T02:33:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.199+0000 7f889e96f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88980b1af0 0x7f88980cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.199+0000 7f889e96f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88980b1f50 0x7f889806c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.199+0000 7f889e96f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88980ca140 0x7f889806cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.199+0000 7f889e96f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889815e8d0 con 0x7f88980b1f50 2024-01-27T02:33:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.199+0000 7f889e96f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8898067ef0 con 0x7f88980ca140 2024-01-27T02:33:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.199+0000 7f889e96f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88980681b0 con 0x7f88980b1af0 2024-01-27T02:33:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.199+0000 7f88977fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88980b1f50 0x7f889806c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.199+0000 7f88977fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88980b1f50 0x7f889806c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60664/0 (socket says 172.21.15.7:60664) 2024-01-27T02:33:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.199+0000 7f88977fe700 1 -- 172.21.15.7:0/366662146 learned_addr learned my addr 172.21.15.7:0/366662146 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.199+0000 7f8897fff700 1 --2- 172.21.15.7:0/366662146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88980b1af0 0x7f88980cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.199+0000 7f889cf0c700 1 --2- 172.21.15.7:0/366662146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88980ca140 0x7f889806cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.199+0000 7f88977fe700 1 -- 172.21.15.7:0/366662146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88980b1af0 msgr2=0x7f88980cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.199+0000 7f88977fe700 1 --2- 172.21.15.7:0/366662146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88980b1af0 0x7f88980cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.199+0000 7f88977fe700 1 -- 172.21.15.7:0/366662146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88980ca140 msgr2=0x7f889806cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.199+0000 7f88977fe700 1 --2- 172.21.15.7:0/366662146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88980ca140 0x7f889806cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.199+0000 7f88977fe700 1 -- 172.21.15.7:0/366662146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8890009ae0 con 0x7f88980b1f50 2024-01-27T02:33:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.200+0000 7f88977fe700 1 --2- 172.21.15.7:0/366662146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88980b1f50 0x7f889806c8b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f888c00c810 tx=0x7f888c007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.200+0000 7f887ffff700 1 -- 172.21.15.7:0/366662146 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c007300 con 0x7f88980b1f50 2024-01-27T02:33:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.200+0000 7f887ffff700 1 -- 172.21.15.7:0/366662146 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888c0074a0 con 0x7f88980b1f50 2024-01-27T02:33:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.200+0000 7f889e96f700 1 -- 172.21.15.7:0/366662146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8898068490 con 0x7f88980b1f50 2024-01-27T02:33:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.200+0000 7f887ffff700 1 -- 172.21.15.7:0/366662146 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c004830 con 0x7f88980b1f50 2024-01-27T02:33:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.200+0000 7f889e96f700 1 -- 172.21.15.7:0/366662146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8898068a50 con 0x7f88980b1f50 2024-01-27T02:33:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.201+0000 7f889e96f700 1 -- 172.21.15.7:0/366662146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8898068c40 con 0x7f88980b1f50 2024-01-27T02:33:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.201+0000 7f887ffff700 1 -- 172.21.15.7:0/366662146 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f888c004ba0 con 0x7f88980b1f50 2024-01-27T02:33:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.202+0000 7f887ffff700 1 --2- 172.21.15.7:0/366662146 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8884066f30 0x7f88840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.202+0000 7f887ffff700 1 -- 172.21.15.7:0/366662146 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f888c083510 con 0x7f88980b1f50 2024-01-27T02:33:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.202+0000 7f8897fff700 1 --2- 172.21.15.7:0/366662146 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8884066f30 0x7f88840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.203+0000 7f8897fff700 1 --2- 172.21.15.7:0/366662146 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8884066f30 0x7f88840693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8888006ea0 tx=0x7f8888008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.206+0000 7f887ffff700 1 -- 172.21.15.7:0/366662146 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f888c050780 con 0x7f88980b1f50 2024-01-27T02:33:14.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.453+0000 7f889e96f700 1 -- 172.21.15.7:0/366662146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f88980b2350 con 0x7f88980b1f50 2024-01-27T02:33:14.455 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-27T02:33:14.454+0000 7fb5c503a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-27T02:33:14.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.459+0000 7f887ffff700 1 -- 172.21.15.7:0/366662146 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f88980b2350 con 0x7f88980b1f50 2024-01-27T02:33:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.462+0000 7f889e96f700 1 -- 172.21.15.7:0/366662146 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8884066f30 msgr2=0x7f88840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.462+0000 7f889e96f700 1 --2- 172.21.15.7:0/366662146 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8884066f30 0x7f88840693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8888006ea0 tx=0x7f8888008040 comp rx=0 tx=0).stop 2024-01-27T02:33:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.462+0000 7f889e96f700 1 -- 172.21.15.7:0/366662146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88980b1f50 msgr2=0x7f889806c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.462+0000 7f889e96f700 1 --2- 172.21.15.7:0/366662146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88980b1f50 0x7f889806c8b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f888c00c810 tx=0x7f888c007c50 comp rx=0 tx=0).stop 2024-01-27T02:33:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.462+0000 7f889e96f700 1 -- 172.21.15.7:0/366662146 shutdown_connections 2024-01-27T02:33:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.462+0000 7f889e96f700 1 --2- 172.21.15.7:0/366662146 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8884066f30 0x7f88840693e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.462+0000 7f889e96f700 1 --2- 172.21.15.7:0/366662146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88980ca140 0x7f889806cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.462+0000 7f889e96f700 1 --2- 172.21.15.7:0/366662146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88980b1af0 0x7f88980cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.462+0000 7f889e96f700 1 --2- 172.21.15.7:0/366662146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88980b1f50 0x7f889806c8b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.462+0000 7f889e96f700 1 -- 172.21.15.7:0/366662146 >> 172.21.15.7:0/366662146 conn(0x7f88980711c0 msgr2=0x7f8898156ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.462+0000 7f889e96f700 1 -- 172.21.15.7:0/366662146 shutdown_connections 2024-01-27T02:33:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.462+0000 7f889e96f700 1 -- 172.21.15.7:0/366662146 wait complete. 2024-01-27T02:33:14.465 INFO:tasks.ceph.mds.c.smithi007.stderr: -3442> 2024-01-27T02:32:59.237+0000 7f7052291700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-27T02:33:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-27T02:33:14.477 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-27T02:33:14.490 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-27T02:33:14.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.610+0000 7f091932d700 1 Processor -- start 2024-01-27T02:33:14.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.611+0000 7f091932d700 1 -- start start 2024-01-27T02:33:14.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.612+0000 7f091932d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f091415a180 0x7f09140c3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:14.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.612+0000 7f091932d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09140ca3b0 0x7f09140aef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:14.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.612+0000 7f091932d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09140b1450 con 0x7f091415a550 2024-01-27T02:33:14.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.612+0000 7f091932d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09140b18e0 con 0x7f091415a180 2024-01-27T02:33:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.612+0000 7f091932d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09140b0840 con 0x7f09140ca3b0 2024-01-27T02:33:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.612+0000 7f0912ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f091415a180 0x7f09140c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.612+0000 7f090bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f091415a550 0x7f09140c9ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53684/0 (socket says 172.21.15.7:53684) 2024-01-27T02:33:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.612+0000 7f090bfff700 1 -- 172.21.15.7:0/2016654648 learned_addr learned my addr 172.21.15.7:0/2016654648 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.612+0000 7f09137fe700 1 --2- 172.21.15.7:0/2016654648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09140ca3b0 0x7f09140aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.612+0000 7f0912ffd700 1 -- 172.21.15.7:0/2016654648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09140ca3b0 msgr2=0x7f09140aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.612+0000 7f0912ffd700 1 --2- 172.21.15.7:0/2016654648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09140ca3b0 0x7f09140aef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.612+0000 7f0912ffd700 1 -- 172.21.15.7:0/2016654648 >> v1:172.21.15.7:6789/0 conn(0x7f091415a550 legacy=0x7f09140c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.612+0000 7f0912ffd700 1 -- 172.21.15.7:0/2016654648 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09140b0b20 con 0x7f091415a180 2024-01-27T02:33:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.612+0000 7f09127fc700 1 -- 172.21.15.7:0/2016654648 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2094275225 0 0) 0x7f09140b1450 con 0x7f091415a550 2024-01-27T02:33:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.613+0000 7f0912ffd700 1 --2- 172.21.15.7:0/2016654648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f091415a180 0x7f09140c3750 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f09040098d0 tx=0x7f0904009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830b695d38395dd8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.613+0000 7f09127fc700 1 -- 172.21.15.7:0/2016654648 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0904005c10 con 0x7f091415a180 2024-01-27T02:33:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.613+0000 7f09127fc700 1 -- 172.21.15.7:0/2016654648 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0904005d70 con 0x7f091415a180 2024-01-27T02:33:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.613+0000 7f09127fc700 1 -- 172.21.15.7:0/2016654648 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0904006280 con 0x7f091415a180 2024-01-27T02:33:14.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.613+0000 7f091932d700 1 -- 172.21.15.7:0/2016654648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f091415a180 msgr2=0x7f09140c3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:14.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.613+0000 7f091932d700 1 --2- 172.21.15.7:0/2016654648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f091415a180 0x7f09140c3750 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f09040098d0 tx=0x7f0904009120 comp rx=0 tx=0).stop 2024-01-27T02:33:14.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.613+0000 7f091932d700 1 -- 172.21.15.7:0/2016654648 shutdown_connections 2024-01-27T02:33:14.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.613+0000 7f091932d700 1 --2- 172.21.15.7:0/2016654648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09140ca3b0 0x7f09140aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:14.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.613+0000 7f091932d700 1 --2- 172.21.15.7:0/2016654648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f091415a180 0x7f09140c3750 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.613+0000 7f091932d700 1 -- 172.21.15.7:0/2016654648 >> 172.21.15.7:0/2016654648 conn(0x7f09140679b0 msgr2=0x7f0914067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.614+0000 7f091932d700 1 -- 172.21.15.7:0/2016654648 shutdown_connections 2024-01-27T02:33:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.614+0000 7f091932d700 1 -- 172.21.15.7:0/2016654648 wait complete. 2024-01-27T02:33:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.615+0000 7f091932d700 1 Processor -- start 2024-01-27T02:33:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.615+0000 7f091932d700 1 -- start start 2024-01-27T02:33:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.615+0000 7f091932d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09140ca3b0 0x7f091415b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.615+0000 7f091932d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f091415a180 0x7f09140aa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.616+0000 7f091932d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f091415a550 0x7f091416acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.616+0000 7f091932d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091415c350 con 0x7f091415a550 2024-01-27T02:33:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.616+0000 7f091932d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091415b0d0 con 0x7f09140ca3b0 2024-01-27T02:33:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.616+0000 7f091932d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091415aac0 con 0x7f091415a180 2024-01-27T02:33:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.616+0000 7f09137fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f091415a550 0x7f091416acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.616+0000 7f09137fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f091415a550 0x7f091416acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60686/0 (socket says 172.21.15.7:60686) 2024-01-27T02:33:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.616+0000 7f09137fe700 1 -- 172.21.15.7:0/1126396820 learned_addr learned my addr 172.21.15.7:0/1126396820 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.616+0000 7f090bfff700 1 --2- 172.21.15.7:0/1126396820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f091415a180 0x7f09140aa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.616+0000 7f0912ffd700 1 --2- 172.21.15.7:0/1126396820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09140ca3b0 0x7f091415b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.616+0000 7f09137fe700 1 -- 172.21.15.7:0/1126396820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f091415a180 msgr2=0x7f09140aa3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.616+0000 7f09137fe700 1 --2- 172.21.15.7:0/1126396820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f091415a180 0x7f09140aa3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.616+0000 7f09137fe700 1 -- 172.21.15.7:0/1126396820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09140ca3b0 msgr2=0x7f091415b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.616+0000 7f09137fe700 1 --2- 172.21.15.7:0/1126396820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09140ca3b0 0x7f091415b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.616+0000 7f09137fe700 1 -- 172.21.15.7:0/1126396820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09040093c0 con 0x7f091415a550 2024-01-27T02:33:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.616+0000 7f0912ffd700 1 --2- 172.21.15.7:0/1126396820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09140ca3b0 0x7f091415b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.616+0000 7f09137fe700 1 --2- 172.21.15.7:0/1126396820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f091415a550 0x7f091416acb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f090c009d40 tx=0x7f090c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.616+0000 7f090affd700 1 -- 172.21.15.7:0/1126396820 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c009ea0 con 0x7f091415a550 2024-01-27T02:33:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.616+0000 7f090affd700 1 -- 172.21.15.7:0/1126396820 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090c00f040 con 0x7f091415a550 2024-01-27T02:33:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.616+0000 7f091932d700 1 -- 172.21.15.7:0/1126396820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091416b890 con 0x7f091415a550 2024-01-27T02:33:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.616+0000 7f090affd700 1 -- 172.21.15.7:0/1126396820 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c00cda0 con 0x7f091415a550 2024-01-27T02:33:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.617+0000 7f091932d700 1 -- 172.21.15.7:0/1126396820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091416bb70 con 0x7f091415a550 2024-01-27T02:33:14.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.617+0000 7f091932d700 1 -- 172.21.15.7:0/1126396820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091415b280 con 0x7f091415a550 2024-01-27T02:33:14.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.618+0000 7f090affd700 1 -- 172.21.15.7:0/1126396820 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f090c014690 con 0x7f091415a550 2024-01-27T02:33:14.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.618+0000 7f090affd700 1 --2- 172.21.15.7:0/1126396820 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f08f4066f30 0x7f08f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:14.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.619+0000 7f090affd700 1 -- 172.21.15.7:0/1126396820 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f090c085100 con 0x7f091415a550 2024-01-27T02:33:14.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.619+0000 7f0912ffd700 1 --2- 172.21.15.7:0/1126396820 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f08f4066f30 0x7f08f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:14.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.619+0000 7f0912ffd700 1 --2- 172.21.15.7:0/1126396820 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f08f4066f30 0x7f08f40693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f091406f4a0 tx=0x7f0904016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:14.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.623+0000 7f090affd700 1 -- 172.21.15.7:0/1126396820 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f091415b280 con 0x7f091415a550 2024-01-27T02:33:14.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:14.872+0000 7f091932d700 1 -- 172.21.15.7:0/1126396820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f091415b240 con 0x7f091415a550 2024-01-27T02:33:15.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.459+0000 7f090affd700 1 -- 172.21.15.7:0/1126396820 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f090c052280 con 0x7f091415a550 2024-01-27T02:33:15.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.462+0000 7f091932d700 1 -- 172.21.15.7:0/1126396820 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f08f4066f30 msgr2=0x7f08f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:15.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.462+0000 7f091932d700 1 --2- 172.21.15.7:0/1126396820 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f08f4066f30 0x7f08f40693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f091406f4a0 tx=0x7f0904016000 comp rx=0 tx=0).stop 2024-01-27T02:33:15.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.462+0000 7f091932d700 1 -- 172.21.15.7:0/1126396820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f091415a550 msgr2=0x7f091416acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:15.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.462+0000 7f091932d700 1 --2- 172.21.15.7:0/1126396820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f091415a550 0x7f091416acb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f090c009d40 tx=0x7f090c00b040 comp rx=0 tx=0).stop 2024-01-27T02:33:15.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.462+0000 7f091932d700 1 -- 172.21.15.7:0/1126396820 shutdown_connections 2024-01-27T02:33:15.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.462+0000 7f091932d700 1 --2- 172.21.15.7:0/1126396820 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f08f4066f30 0x7f08f40693e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:15.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.462+0000 7f091932d700 1 --2- 172.21.15.7:0/1126396820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f091415a550 0x7f091416acb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:15.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.462+0000 7f091932d700 1 --2- 172.21.15.7:0/1126396820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09140ca3b0 0x7f091415b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:15.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.462+0000 7f091932d700 1 --2- 172.21.15.7:0/1126396820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f091415a180 0x7f09140aa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:15.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.462+0000 7f091932d700 1 -- 172.21.15.7:0/1126396820 >> 172.21.15.7:0/1126396820 conn(0x7f09140679b0 msgr2=0x7f09140cc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:15.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.462+0000 7f091932d700 1 -- 172.21.15.7:0/1126396820 shutdown_connections 2024-01-27T02:33:15.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.462+0000 7f091932d700 1 -- 172.21.15.7:0/1126396820 wait complete. 2024-01-27T02:33:15.479 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-27T02:33:15.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.662+0000 7f2c43b9c700 1 Processor -- start 2024-01-27T02:33:15.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.662+0000 7f2c43b9c700 1 -- start start 2024-01-27T02:33:15.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.663+0000 7f2c43b9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2c3c0b1b70 0x7f2c3c06e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:15.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.663+0000 7f2c43b9c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c3c15a960 0x7f2c3c1567b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:15.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.663+0000 7f2c43b9c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2c3c0b1f40 0x7f2c3c156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:15.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.663+0000 7f2c43b9c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c3c157230 con 0x7f2c3c0b1b70 2024-01-27T02:33:15.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.663+0000 7f2c43b9c700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c3c06ea80 con 0x7f2c3c0b1f40 2024-01-27T02:33:15.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.663+0000 7f2c43b9c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c06ed20 con 0x7f2c3c15a960 2024-01-27T02:33:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.663+0000 7f2c41938700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2c3c0b1b70 0x7f2c3c06e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.663+0000 7f2c41938700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2c3c0b1b70 0x7f2c3c06e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60694/0 (socket says 172.21.15.7:60694) 2024-01-27T02:33:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.663+0000 7f2c41938700 1 -- 172.21.15.7:0/3645410122 learned_addr learned my addr 172.21.15.7:0/3645410122 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.663+0000 7f2c42139700 1 --2- 172.21.15.7:0/3645410122 >> v2:172.21.15.188:3300/0 conn(0x7f2c3c0b1f40 0x7f2c3c156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.663+0000 7f2c41137700 1 --2- 172.21.15.7:0/3645410122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c3c15a960 0x7f2c3c1567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.663+0000 7f2c41938700 1 -- 172.21.15.7:0/3645410122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c3c15a960 msgr2=0x7f2c3c1567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.663+0000 7f2c41938700 1 --2- 172.21.15.7:0/3645410122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c3c15a960 0x7f2c3c1567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.663+0000 7f2c41938700 1 -- 172.21.15.7:0/3645410122 >> v2:172.21.15.188:3300/0 conn(0x7f2c3c0b1f40 msgr2=0x7f2c3c156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.663+0000 7f2c41938700 1 --2- 172.21.15.7:0/3645410122 >> v2:172.21.15.188:3300/0 conn(0x7f2c3c0b1f40 0x7f2c3c156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.663+0000 7f2c41938700 1 -- 172.21.15.7:0/3645410122 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c06f000 con 0x7f2c3c0b1b70 2024-01-27T02:33:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.664+0000 7f2c41938700 1 --2- 172.21.15.7:0/3645410122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c0b1b70 0x7f2c3c06e540 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c06c6a0 tx=0x7f2c38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6285a1b2dce7a6 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.664+0000 7f2c40936700 1 -- 172.21.15.7:0/3645410122 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3800b860 con 0x7f2c3c0b1b70 2024-01-27T02:33:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.664+0000 7f2c40936700 1 -- 172.21.15.7:0/3645410122 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3800b9c0 con 0x7f2c3c0b1b70 2024-01-27T02:33:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.664+0000 7f2c40936700 1 -- 172.21.15.7:0/3645410122 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3800bd60 con 0x7f2c3c0b1b70 2024-01-27T02:33:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.664+0000 7f2c43b9c700 1 -- 172.21.15.7:0/3645410122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c0b1b70 msgr2=0x7f2c3c06e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.664+0000 7f2c43b9c700 1 --2- 172.21.15.7:0/3645410122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c0b1b70 0x7f2c3c06e540 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c06c6a0 tx=0x7f2c38009250 comp rx=0 tx=0).stop 2024-01-27T02:33:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.664+0000 7f2c43b9c700 1 -- 172.21.15.7:0/3645410122 shutdown_connections 2024-01-27T02:33:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.664+0000 7f2c43b9c700 1 --2- 172.21.15.7:0/3645410122 >> v2:172.21.15.188:3300/0 conn(0x7f2c3c0b1f40 0x7f2c3c156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.664+0000 7f2c43b9c700 1 --2- 172.21.15.7:0/3645410122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c0b1b70 0x7f2c3c06e540 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.664+0000 7f2c43b9c700 1 --2- 172.21.15.7:0/3645410122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c3c15a960 0x7f2c3c1567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.664+0000 7f2c43b9c700 1 -- 172.21.15.7:0/3645410122 >> 172.21.15.7:0/3645410122 conn(0x7f2c3c068480 msgr2=0x7f2c3c06be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.665+0000 7f2c43b9c700 1 -- 172.21.15.7:0/3645410122 shutdown_connections 2024-01-27T02:33:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.665+0000 7f2c43b9c700 1 -- 172.21.15.7:0/3645410122 wait complete. 2024-01-27T02:33:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.666+0000 7f2c43b9c700 1 Processor -- start 2024-01-27T02:33:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.666+0000 7f2c43b9c700 1 -- start start 2024-01-27T02:33:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.666+0000 7f2c43b9c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c3c0b1b70 0x7f2c3c14ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.666+0000 7f2c43b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c0b1f40 0x7f2c3c14d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.666+0000 7f2c43b9c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c3c15a960 0x7f2c3c1675a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.667+0000 7f2c43b9c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c16b8b0 con 0x7f2c3c0b1f40 2024-01-27T02:33:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.667+0000 7f2c43b9c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c16bb90 con 0x7f2c3c0b1b70 2024-01-27T02:33:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.667+0000 7f2c43b9c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c16be70 con 0x7f2c3c15a960 2024-01-27T02:33:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.667+0000 7f2c41137700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c0b1f40 0x7f2c3c14d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.667+0000 7f2c41137700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c0b1f40 0x7f2c3c14d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60708/0 (socket says 172.21.15.7:60708) 2024-01-27T02:33:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.667+0000 7f2c41137700 1 -- 172.21.15.7:0/977506302 learned_addr learned my addr 172.21.15.7:0/977506302 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.667+0000 7f2c41938700 1 --2- 172.21.15.7:0/977506302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c3c0b1b70 0x7f2c3c14ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.667+0000 7f2c42139700 1 --2- 172.21.15.7:0/977506302 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c3c15a960 0x7f2c3c1675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.667+0000 7f2c41137700 1 -- 172.21.15.7:0/977506302 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c3c15a960 msgr2=0x7f2c3c1675a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.667+0000 7f2c41137700 1 --2- 172.21.15.7:0/977506302 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c3c15a960 0x7f2c3c1675a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.667+0000 7f2c41137700 1 -- 172.21.15.7:0/977506302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c3c0b1b70 msgr2=0x7f2c3c14ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.667+0000 7f2c41137700 1 --2- 172.21.15.7:0/977506302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c3c0b1b70 0x7f2c3c14ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.667+0000 7f2c41137700 1 -- 172.21.15.7:0/977506302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c380094f0 con 0x7f2c3c0b1f40 2024-01-27T02:33:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.667+0000 7f2c42139700 1 --2- 172.21.15.7:0/977506302 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c3c15a960 0x7f2c3c1675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.667+0000 7f2c41137700 1 --2- 172.21.15.7:0/977506302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c0b1f40 0x7f2c3c14d000 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00d890 tx=0x7f2c2c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.667+0000 7f2c31ffb700 1 -- 172.21.15.7:0/977506302 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c0125f0 con 0x7f2c3c0b1f40 2024-01-27T02:33:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.667+0000 7f2c31ffb700 1 -- 172.21.15.7:0/977506302 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2c012750 con 0x7f2c3c0b1f40 2024-01-27T02:33:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.667+0000 7f2c31ffb700 1 -- 172.21.15.7:0/977506302 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c012ad0 con 0x7f2c3c0b1f40 2024-01-27T02:33:15.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.667+0000 7f2c43b9c700 1 -- 172.21.15.7:0/977506302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c3c16c150 con 0x7f2c3c0b1f40 2024-01-27T02:33:15.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.668+0000 7f2c43b9c700 1 -- 172.21.15.7:0/977506302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c3c16b460 con 0x7f2c3c0b1f40 2024-01-27T02:33:15.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.669+0000 7f2c43b9c700 1 -- 172.21.15.7:0/977506302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c3c16b680 con 0x7f2c3c0b1f40 2024-01-27T02:33:15.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.669+0000 7f2c31ffb700 1 -- 172.21.15.7:0/977506302 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2c2c01e460 con 0x7f2c3c0b1f40 2024-01-27T02:33:15.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.669+0000 7f2c31ffb700 1 --2- 172.21.15.7:0/977506302 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2c20066f30 0x7f2c200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:15.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.669+0000 7f2c31ffb700 1 -- 172.21.15.7:0/977506302 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f2c2c084c40 con 0x7f2c3c0b1f40 2024-01-27T02:33:15.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.670+0000 7f2c41938700 1 --2- 172.21.15.7:0/977506302 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2c20066f30 0x7f2c200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:15.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.670+0000 7f2c41938700 1 --2- 172.21.15.7:0/977506302 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2c20066f30 0x7f2c200693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c06c6a0 tx=0x7f2c38017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:15.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.674+0000 7f2c31ffb700 1 -- 172.21.15.7:0/977506302 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c2c051dc0 con 0x7f2c3c0b1f40 2024-01-27T02:33:15.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:15.878+0000 7f2c43b9c700 1 -- 172.21.15.7:0/977506302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2c3c04c710 con 0x7f2c3c0b1f40 2024-01-27T02:33:16.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.465+0000 7f2c31ffb700 1 -- 172.21.15.7:0/977506302 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2c2c0553e0 con 0x7f2c3c0b1f40 2024-01-27T02:33:16.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.469+0000 7f2c43b9c700 1 -- 172.21.15.7:0/977506302 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2c20066f30 msgr2=0x7f2c200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.469+0000 7f2c43b9c700 1 --2- 172.21.15.7:0/977506302 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2c20066f30 0x7f2c200693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c06c6a0 tx=0x7f2c38017000 comp rx=0 tx=0).stop 2024-01-27T02:33:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.469+0000 7f2c43b9c700 1 -- 172.21.15.7:0/977506302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c0b1f40 msgr2=0x7f2c3c14d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.469+0000 7f2c43b9c700 1 --2- 172.21.15.7:0/977506302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c0b1f40 0x7f2c3c14d000 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00d890 tx=0x7f2c2c008040 comp rx=0 tx=0).stop 2024-01-27T02:33:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.469+0000 7f2c43b9c700 1 -- 172.21.15.7:0/977506302 shutdown_connections 2024-01-27T02:33:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.469+0000 7f2c43b9c700 1 --2- 172.21.15.7:0/977506302 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2c20066f30 0x7f2c200693e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.469+0000 7f2c43b9c700 1 --2- 172.21.15.7:0/977506302 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c3c15a960 0x7f2c3c1675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.469+0000 7f2c43b9c700 1 --2- 172.21.15.7:0/977506302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c3c0b1b70 0x7f2c3c14ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.469+0000 7f2c43b9c700 1 --2- 172.21.15.7:0/977506302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c0b1f40 0x7f2c3c14d000 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.469+0000 7f2c43b9c700 1 -- 172.21.15.7:0/977506302 >> 172.21.15.7:0/977506302 conn(0x7f2c3c068480 msgr2=0x7f2c3c0a64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.469+0000 7f2c43b9c700 1 -- 172.21.15.7:0/977506302 shutdown_connections 2024-01-27T02:33:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.470+0000 7f2c43b9c700 1 -- 172.21.15.7:0/977506302 wait complete. 2024-01-27T02:33:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-27T02:33:16.487 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-27T02:33:16.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.626+0000 7f3fb7c6d700 1 Processor -- start 2024-01-27T02:33:16.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.626+0000 7f3fb7c6d700 1 -- start start 2024-01-27T02:33:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.627+0000 7f3fb7c6d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3fb0147d50 0x7f3fb00c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.627+0000 7f3fb7c6d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb0148120 0x7f3fb00c8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.627+0000 7f3fb7c6d700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fb00c3970 con 0x7f3fb00c89c0 2024-01-27T02:33:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.627+0000 7f3fb7c6d700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb01630e0 con 0x7f3fb0147d50 2024-01-27T02:33:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.627+0000 7f3fb7c6d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb00c8e60 con 0x7f3fb0148120 2024-01-27T02:33:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.627+0000 7f3fb5208700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb0148120 0x7f3fb00c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.627+0000 7f3fb5a09700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3fb0147d50 0x7f3fb00c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.627+0000 7f3fb620a700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f3fb00c89c0 0x7f3fb0063580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:53242/0 (socket says 172.21.15.7:53242) 2024-01-27T02:33:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.627+0000 7f3fb620a700 1 -- 172.21.15.7:0/711873402 learned_addr learned my addr 172.21.15.7:0/711873402 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.627+0000 7f3fb5a09700 1 -- 172.21.15.7:0/711873402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb0148120 msgr2=0x7f3fb00c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:16.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.627+0000 7f3fb5a09700 1 --2- 172.21.15.7:0/711873402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb0148120 0x7f3fb00c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:16.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.627+0000 7f3fb5a09700 1 -- 172.21.15.7:0/711873402 >> v1:172.21.15.188:6789/0 conn(0x7f3fb00c89c0 legacy=0x7f3fb0063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:16.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.627+0000 7f3fb5a09700 1 -- 172.21.15.7:0/711873402 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb00c3040 con 0x7f3fb0147d50 2024-01-27T02:33:16.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.628+0000 7f3fb5208700 1 --2- 172.21.15.7:0/711873402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb0148120 0x7f3fb00c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:33:16.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.628+0000 7f3fb5a09700 1 --2- 172.21.15.7:0/711873402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb0147d50 0x7f3fb00c3e00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3fac0098e0 tx=0x7f3fac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0490ba35614c9c7 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:16.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.628+0000 7f3fb4a07700 1 -- 172.21.15.7:0/711873402 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac00b640 con 0x7f3fb0147d50 2024-01-27T02:33:16.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.628+0000 7f3fb4a07700 1 -- 172.21.15.7:0/711873402 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fac00b7a0 con 0x7f3fb0147d50 2024-01-27T02:33:16.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.628+0000 7f3fb4a07700 1 -- 172.21.15.7:0/711873402 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac00bb40 con 0x7f3fb0147d50 2024-01-27T02:33:16.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.628+0000 7f3fb7c6d700 1 -- 172.21.15.7:0/711873402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb0147d50 msgr2=0x7f3fb00c3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:16.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.628+0000 7f3fb7c6d700 1 --2- 172.21.15.7:0/711873402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb0147d50 0x7f3fb00c3e00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3fac0098e0 tx=0x7f3fac0091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:16.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.629+0000 7f3fb7c6d700 1 -- 172.21.15.7:0/711873402 shutdown_connections 2024-01-27T02:33:16.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.629+0000 7f3fb7c6d700 1 --2- 172.21.15.7:0/711873402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb0147d50 0x7f3fb00c3e00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:16.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.629+0000 7f3fb7c6d700 1 --2- 172.21.15.7:0/711873402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb0148120 0x7f3fb00c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:16.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.629+0000 7f3fb7c6d700 1 -- 172.21.15.7:0/711873402 >> 172.21.15.7:0/711873402 conn(0x7f3fb00cdc30 msgr2=0x7f3fb00ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:16.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.629+0000 7f3fb7c6d700 1 -- 172.21.15.7:0/711873402 shutdown_connections 2024-01-27T02:33:16.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.629+0000 7f3fb7c6d700 1 -- 172.21.15.7:0/711873402 wait complete. 2024-01-27T02:33:16.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.630+0000 7f3fb7c6d700 1 Processor -- start 2024-01-27T02:33:16.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.630+0000 7f3fb7c6d700 1 -- start start 2024-01-27T02:33:16.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.631+0000 7f3fb7c6d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb00c89c0 0x7f3fb01449c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:16.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.631+0000 7f3fb7c6d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb0147d50 0x7f3fb01717c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:16.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.631+0000 7f3fb7c6d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fb0148120 0x7f3fb015dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:16.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.631+0000 7f3fb7c6d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb0144f00 con 0x7f3fb0148120 2024-01-27T02:33:16.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.631+0000 7f3fb7c6d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb0171d00 con 0x7f3fb0147d50 2024-01-27T02:33:16.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.631+0000 7f3fb7c6d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb016e180 con 0x7f3fb00c89c0 2024-01-27T02:33:16.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.631+0000 7f3fb620a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fb0148120 0x7f3fb015dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:16.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.631+0000 7f3fb620a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fb0148120 0x7f3fb015dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60714/0 (socket says 172.21.15.7:60714) 2024-01-27T02:33:16.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.631+0000 7f3fb620a700 1 -- 172.21.15.7:0/1555274645 learned_addr learned my addr 172.21.15.7:0/1555274645 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.631+0000 7f3fb5208700 1 --2- 172.21.15.7:0/1555274645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb0147d50 0x7f3fb01717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.631+0000 7f3fb5a09700 1 --2- 172.21.15.7:0/1555274645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb00c89c0 0x7f3fb01449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.631+0000 7f3fb620a700 1 -- 172.21.15.7:0/1555274645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb00c89c0 msgr2=0x7f3fb01449c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.631+0000 7f3fb620a700 1 --2- 172.21.15.7:0/1555274645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb00c89c0 0x7f3fb01449c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.631+0000 7f3fb620a700 1 -- 172.21.15.7:0/1555274645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb0147d50 msgr2=0x7f3fb01717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.631+0000 7f3fb620a700 1 --2- 172.21.15.7:0/1555274645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb0147d50 0x7f3fb01717c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.631+0000 7f3fb620a700 1 -- 172.21.15.7:0/1555274645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa0009420 con 0x7f3fb0148120 2024-01-27T02:33:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.631+0000 7f3fb620a700 1 --2- 172.21.15.7:0/1555274645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fb0148120 0x7f3fb015dac0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800cbc0 tx=0x7f3fa80019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.632+0000 7f3fa5ffb700 1 -- 172.21.15.7:0/1555274645 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa800a7e0 con 0x7f3fb0148120 2024-01-27T02:33:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.632+0000 7f3fa5ffb700 1 -- 172.21.15.7:0/1555274645 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa800a940 con 0x7f3fb0148120 2024-01-27T02:33:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.632+0000 7f3fb7c6d700 1 -- 172.21.15.7:0/1555274645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fac009420 con 0x7f3fb0148120 2024-01-27T02:33:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.632+0000 7f3fa5ffb700 1 -- 172.21.15.7:0/1555274645 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa800acc0 con 0x7f3fb0148120 2024-01-27T02:33:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.632+0000 7f3fb7c6d700 1 -- 172.21.15.7:0/1555274645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb016e460 con 0x7f3fb0148120 2024-01-27T02:33:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.633+0000 7f3fb7c6d700 1 -- 172.21.15.7:0/1555274645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb016e5f0 con 0x7f3fb0148120 2024-01-27T02:33:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.633+0000 7f3fa5ffb700 1 -- 172.21.15.7:0/1555274645 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3fa8007300 con 0x7f3fb0148120 2024-01-27T02:33:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.634+0000 7f3fa5ffb700 1 --2- 172.21.15.7:0/1555274645 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3f94066f30 0x7f3f940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.634+0000 7f3fa5ffb700 1 -- 172.21.15.7:0/1555274645 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f3fa80846a0 con 0x7f3fb0148120 2024-01-27T02:33:16.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.634+0000 7f3fb5a09700 1 --2- 172.21.15.7:0/1555274645 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3f94066f30 0x7f3f940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:16.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.634+0000 7f3fb5a09700 1 --2- 172.21.15.7:0/1555274645 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3f94066f30 0x7f3f940693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3fac0093f0 tx=0x7f3fac017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.638+0000 7f3fa5ffb700 1 -- 172.21.15.7:0/1555274645 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3fa80519e0 con 0x7f3fb0148120 2024-01-27T02:33:16.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:16.843+0000 7f3fb7c6d700 1 -- 172.21.15.7:0/1555274645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3fb004c710 con 0x7f3fb0148120 2024-01-27T02:33:17.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.465+0000 7f3fa5ffb700 1 -- 172.21.15.7:0/1555274645 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3fa8055000 con 0x7f3fb0148120 2024-01-27T02:33:17.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.469+0000 7f3fb7c6d700 1 -- 172.21.15.7:0/1555274645 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3f94066f30 msgr2=0x7f3f940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:17.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.469+0000 7f3fb7c6d700 1 --2- 172.21.15.7:0/1555274645 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3f94066f30 0x7f3f940693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3fac0093f0 tx=0x7f3fac017000 comp rx=0 tx=0).stop 2024-01-27T02:33:17.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.469+0000 7f3fb7c6d700 1 -- 172.21.15.7:0/1555274645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fb0148120 msgr2=0x7f3fb015dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:17.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.470+0000 7f3fb7c6d700 1 --2- 172.21.15.7:0/1555274645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fb0148120 0x7f3fb015dac0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800cbc0 tx=0x7f3fa80019c0 comp rx=0 tx=0).stop 2024-01-27T02:33:17.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.470+0000 7f3fb7c6d700 1 -- 172.21.15.7:0/1555274645 shutdown_connections 2024-01-27T02:33:17.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.470+0000 7f3fb7c6d700 1 --2- 172.21.15.7:0/1555274645 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3f94066f30 0x7f3f940693e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:17.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.470+0000 7f3fb7c6d700 1 --2- 172.21.15.7:0/1555274645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fb0148120 0x7f3fb015dac0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:17.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.470+0000 7f3fb7c6d700 1 --2- 172.21.15.7:0/1555274645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb00c89c0 0x7f3fb01449c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:17.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.470+0000 7f3fb7c6d700 1 --2- 172.21.15.7:0/1555274645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb0147d50 0x7f3fb01717c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:17.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.470+0000 7f3fb7c6d700 1 -- 172.21.15.7:0/1555274645 >> 172.21.15.7:0/1555274645 conn(0x7f3fb00cdc30 msgr2=0x7f3fb006e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:17.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.470+0000 7f3fb7c6d700 1 -- 172.21.15.7:0/1555274645 shutdown_connections 2024-01-27T02:33:17.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.470+0000 7f3fb7c6d700 1 -- 172.21.15.7:0/1555274645 wait complete. 2024-01-27T02:33:17.472 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-27T02:33:17.487 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-27T02:33:17.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.623+0000 7f5818424700 1 Processor -- start 2024-01-27T02:33:17.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.623+0000 7f5818424700 1 -- start start 2024-01-27T02:33:17.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.624+0000 7f5818424700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58100cbe30 0x7f58100ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.624+0000 7f5818424700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f581006b4e0 0x7f581006b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.624+0000 7f5818424700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581006be80 con 0x7f58100cba60 2024-01-27T02:33:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.624+0000 7f5818424700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58100679b0 con 0x7f581006b4e0 2024-01-27T02:33:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.624+0000 7f5818424700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5810067c90 con 0x7f58100cbe30 2024-01-27T02:33:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.625+0000 7f58169c1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f581006b4e0 0x7f581006b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.625+0000 7f58169c1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f581006b4e0 0x7f581006b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60726/0 (socket says 172.21.15.7:60726) 2024-01-27T02:33:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.625+0000 7f58169c1700 1 -- 172.21.15.7:0/2054582148 learned_addr learned my addr 172.21.15.7:0/2054582148 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.625+0000 7f58159bf700 1 --2- 172.21.15.7:0/2054582148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58100cbe30 0x7f58100ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.625+0000 7f58169c1700 1 -- 172.21.15.7:0/2054582148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58100cbe30 msgr2=0x7f58100ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.625+0000 7f58169c1700 1 --2- 172.21.15.7:0/2054582148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58100cbe30 0x7f58100ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.625+0000 7f58169c1700 1 -- 172.21.15.7:0/2054582148 >> v1:172.21.15.188:6789/0 conn(0x7f58100cba60 legacy=0x7f581015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.625+0000 7f58169c1700 1 -- 172.21.15.7:0/2054582148 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58100cca50 con 0x7f581006b4e0 2024-01-27T02:33:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.625+0000 7f58169c1700 1 --2- 172.21.15.7:0/2054582148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f581006b4e0 0x7f581006b940 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5808009f20 tx=0x7f5808009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ea01f78ca7e4b9 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:17.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.625+0000 7f58151be700 1 -- 172.21.15.7:0/2054582148 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580800e070 con 0x7f581006b4e0 2024-01-27T02:33:17.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.625+0000 7f58151be700 1 -- 172.21.15.7:0/2054582148 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580800ba80 con 0x7f581006b4e0 2024-01-27T02:33:17.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.625+0000 7f58151be700 1 -- 172.21.15.7:0/2054582148 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58080092e0 con 0x7f581006b4e0 2024-01-27T02:33:17.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.626+0000 7f5818424700 1 -- 172.21.15.7:0/2054582148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f581006b4e0 msgr2=0x7f581006b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:17.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.626+0000 7f5818424700 1 --2- 172.21.15.7:0/2054582148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f581006b4e0 0x7f581006b940 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5808009f20 tx=0x7f5808009840 comp rx=0 tx=0).stop 2024-01-27T02:33:17.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.626+0000 7f5818424700 1 -- 172.21.15.7:0/2054582148 shutdown_connections 2024-01-27T02:33:17.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.626+0000 7f5818424700 1 --2- 172.21.15.7:0/2054582148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f581006b4e0 0x7f581006b940 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:17.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.626+0000 7f5818424700 1 --2- 172.21.15.7:0/2054582148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58100cbe30 0x7f58100ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:17.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.626+0000 7f5818424700 1 -- 172.21.15.7:0/2054582148 >> 172.21.15.7:0/2054582148 conn(0x7f5810063580 msgr2=0x7f5810063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:17.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.626+0000 7f5818424700 1 -- 172.21.15.7:0/2054582148 shutdown_connections 2024-01-27T02:33:17.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.626+0000 7f5818424700 1 -- 172.21.15.7:0/2054582148 wait complete. 2024-01-27T02:33:17.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.627+0000 7f5818424700 1 Processor -- start 2024-01-27T02:33:17.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.627+0000 7f5818424700 1 -- start start 2024-01-27T02:33:17.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.628+0000 7f5818424700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581006b4e0 0x7f58101547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.628+0000 7f5818424700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58100cba60 0x7f58100ac4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.628+0000 7f5818424700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58100cbe30 0x7f58100ac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.628+0000 7f5818424700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58100b4280 con 0x7f58100cba60 2024-01-27T02:33:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.628+0000 7f5818424700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581016aad0 con 0x7f58100cbe30 2024-01-27T02:33:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.628+0000 7f5818424700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581016adb0 con 0x7f581006b4e0 2024-01-27T02:33:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.628+0000 7f58159bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58100cba60 0x7f58100ac4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.628+0000 7f58159bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58100cba60 0x7f58100ac4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60738/0 (socket says 172.21.15.7:60738) 2024-01-27T02:33:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.628+0000 7f58159bf700 1 -- 172.21.15.7:0/1345325907 learned_addr learned my addr 172.21.15.7:0/1345325907 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.629+0000 7f58161c0700 1 --2- 172.21.15.7:0/1345325907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581006b4e0 0x7f58101547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.629+0000 7f58169c1700 1 --2- 172.21.15.7:0/1345325907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58100cbe30 0x7f58100ac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.629+0000 7f58159bf700 1 -- 172.21.15.7:0/1345325907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581006b4e0 msgr2=0x7f58101547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:17.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.629+0000 7f58159bf700 1 --2- 172.21.15.7:0/1345325907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581006b4e0 0x7f58101547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.629+0000 7f58159bf700 1 -- 172.21.15.7:0/1345325907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58100cbe30 msgr2=0x7f58100ac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.629+0000 7f58159bf700 1 --2- 172.21.15.7:0/1345325907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58100cbe30 0x7f58100ac9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.629+0000 7f58159bf700 1 -- 172.21.15.7:0/1345325907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5808009ae0 con 0x7f58100cba60 2024-01-27T02:33:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.629+0000 7f58159bf700 1 --2- 172.21.15.7:0/1345325907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58100cba60 0x7f58100ac4b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f580c00d860 tx=0x7f580c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.629+0000 7f58027fc700 1 -- 172.21.15.7:0/1345325907 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c0125f0 con 0x7f58100cba60 2024-01-27T02:33:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.629+0000 7f58027fc700 1 -- 172.21.15.7:0/1345325907 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c012750 con 0x7f58100cba60 2024-01-27T02:33:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.629+0000 7f5818424700 1 -- 172.21.15.7:0/1345325907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581016b090 con 0x7f58100cba60 2024-01-27T02:33:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.629+0000 7f58027fc700 1 -- 172.21.15.7:0/1345325907 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c012af0 con 0x7f58100cba60 2024-01-27T02:33:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.629+0000 7f5818424700 1 -- 172.21.15.7:0/1345325907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58100ad090 con 0x7f58100cba60 2024-01-27T02:33:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.630+0000 7f5818424700 1 -- 172.21.15.7:0/1345325907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581016b530 con 0x7f58100cba60 2024-01-27T02:33:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.631+0000 7f58027fc700 1 -- 172.21.15.7:0/1345325907 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f580c013480 con 0x7f58100cba60 2024-01-27T02:33:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.631+0000 7f58027fc700 1 --2- 172.21.15.7:0/1345325907 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f57f4066f30 0x7f57f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.631+0000 7f58027fc700 1 -- 172.21.15.7:0/1345325907 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f580c0845b0 con 0x7f58100cba60 2024-01-27T02:33:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.631+0000 7f58161c0700 1 --2- 172.21.15.7:0/1345325907 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f57f4066f30 0x7f57f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.632+0000 7f58161c0700 1 --2- 172.21.15.7:0/1345325907 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f57f4066f30 0x7f57f40693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f581015f3c0 tx=0x7f5804009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:17.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.635+0000 7f58027fc700 1 -- 172.21.15.7:0/1345325907 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f580c0555b0 con 0x7f58100cba60 2024-01-27T02:33:17.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.840+0000 7f5818424700 1 -- 172.21.15.7:0/1345325907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5810072060 con 0x7f58100cba60 2024-01-27T02:33:17.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.845+0000 7f58027fc700 1 -- 172.21.15.7:0/1345325907 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7f580c055140 con 0x7f58100cba60 2024-01-27T02:33:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.847+0000 7f5818424700 1 -- 172.21.15.7:0/1345325907 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f57f4066f30 msgr2=0x7f57f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.847+0000 7f5818424700 1 --2- 172.21.15.7:0/1345325907 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f57f4066f30 0x7f57f40693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f581015f3c0 tx=0x7f5804009040 comp rx=0 tx=0).stop 2024-01-27T02:33:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.847+0000 7f5818424700 1 -- 172.21.15.7:0/1345325907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58100cba60 msgr2=0x7f58100ac4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.847+0000 7f5818424700 1 --2- 172.21.15.7:0/1345325907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58100cba60 0x7f58100ac4b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f580c00d860 tx=0x7f580c008040 comp rx=0 tx=0).stop 2024-01-27T02:33:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.847+0000 7f5818424700 1 -- 172.21.15.7:0/1345325907 shutdown_connections 2024-01-27T02:33:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.847+0000 7f5818424700 1 --2- 172.21.15.7:0/1345325907 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f57f4066f30 0x7f57f40693e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.847+0000 7f5818424700 1 --2- 172.21.15.7:0/1345325907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58100cbe30 0x7f58100ac9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.847+0000 7f5818424700 1 --2- 172.21.15.7:0/1345325907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581006b4e0 0x7f58101547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.847+0000 7f5818424700 1 --2- 172.21.15.7:0/1345325907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58100cba60 0x7f58100ac4b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:17.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.847+0000 7f5818424700 1 -- 172.21.15.7:0/1345325907 >> 172.21.15.7:0/1345325907 conn(0x7f5810063580 msgr2=0x7f5810068ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:17.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.847+0000 7f5818424700 1 -- 172.21.15.7:0/1345325907 shutdown_connections 2024-01-27T02:33:17.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.847+0000 7f5818424700 1 -- 172.21.15.7:0/1345325907 wait complete. 2024-01-27T02:33:17.862 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2024-01-27T02:33:17.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.992+0000 7fa6488fc700 1 Processor -- start 2024-01-27T02:33:17.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.992+0000 7fa6488fc700 1 -- start start 2024-01-27T02:33:17.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.993+0000 7fa6488fc700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa644157c00 0x7fa6440cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.993+0000 7fa6488fc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6440b3040 0x7fa64415bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.993+0000 7fa6488fc700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64415ba30 con 0x7fa6440cfd10 2024-01-27T02:33:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.993+0000 7fa6488fc700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6440d0170 con 0x7fa644157c00 2024-01-27T02:33:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.993+0000 7fa6488fc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440d0390 con 0x7fa6440b3040 2024-01-27T02:33:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.993+0000 7fa642d9d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6440b3040 0x7fa64415bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.993+0000 7fa64259c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa644157c00 0x7fa6440cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.994+0000 7fa64259c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa644157c00 0x7fa6440cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:57094/0 (socket says 172.21.15.7:57094) 2024-01-27T02:33:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.994+0000 7fa64259c700 1 -- 172.21.15.7:0/3710869895 learned_addr learned my addr 172.21.15.7:0/3710869895 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.994+0000 7fa641d9b700 1 --1- 172.21.15.7:0/3710869895 >> v1:172.21.15.188:6789/0 conn(0x7fa6440cfd10 0x7fa6440b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:53252/0 (socket says 172.21.15.7:53252) 2024-01-27T02:33:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.994+0000 7fa64259c700 1 -- 172.21.15.7:0/3710869895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6440b3040 msgr2=0x7fa64415bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.994+0000 7fa64259c700 1 --2- 172.21.15.7:0/3710869895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6440b3040 0x7fa64415bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.994+0000 7fa64259c700 1 -- 172.21.15.7:0/3710869895 >> v1:172.21.15.188:6789/0 conn(0x7fa6440cfd10 legacy=0x7fa6440b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:17.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.994+0000 7fa64259c700 1 -- 172.21.15.7:0/3710869895 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6440d0670 con 0x7fa644157c00 2024-01-27T02:33:17.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.994+0000 7fa642d9d700 1 --2- 172.21.15.7:0/3710869895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6440b3040 0x7fa64415bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:33:17.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.994+0000 7fa64259c700 1 --2- 172.21.15.7:0/3710869895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa644157c00 0x7fa6440cf7d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa6340099b0 tx=0x7fa634009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5670db7cbd23884 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:17.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.994+0000 7fa64159a700 1 -- 172.21.15.7:0/3710869895 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa634009ea0 con 0x7fa644157c00 2024-01-27T02:33:17.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.994+0000 7fa64159a700 1 -- 172.21.15.7:0/3710869895 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63400b860 con 0x7fa644157c00 2024-01-27T02:33:17.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.994+0000 7fa64159a700 1 -- 172.21.15.7:0/3710869895 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63400bc00 con 0x7fa644157c00 2024-01-27T02:33:17.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.995+0000 7fa6488fc700 1 -- 172.21.15.7:0/3710869895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa644157c00 msgr2=0x7fa6440cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:17.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.995+0000 7fa6488fc700 1 --2- 172.21.15.7:0/3710869895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa644157c00 0x7fa6440cf7d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa6340099b0 tx=0x7fa634009250 comp rx=0 tx=0).stop 2024-01-27T02:33:17.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.995+0000 7fa6488fc700 1 -- 172.21.15.7:0/3710869895 shutdown_connections 2024-01-27T02:33:17.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.995+0000 7fa6488fc700 1 --2- 172.21.15.7:0/3710869895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6440b3040 0x7fa64415bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:17.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.995+0000 7fa6488fc700 1 --2- 172.21.15.7:0/3710869895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa644157c00 0x7fa6440cf7d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:17.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.995+0000 7fa6488fc700 1 -- 172.21.15.7:0/3710869895 >> 172.21.15.7:0/3710869895 conn(0x7fa644068450 msgr2=0x7fa6440709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:17.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.995+0000 7fa6488fc700 1 -- 172.21.15.7:0/3710869895 shutdown_connections 2024-01-27T02:33:17.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.995+0000 7fa6488fc700 1 -- 172.21.15.7:0/3710869895 wait complete. 2024-01-27T02:33:17.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.996+0000 7fa6488fc700 1 Processor -- start 2024-01-27T02:33:17.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.996+0000 7fa6488fc700 1 -- start start 2024-01-27T02:33:18.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.997+0000 7fa6488fc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6440b3040 0x7fa6440b6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:18.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.997+0000 7fa6488fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6440cfd10 0x7fa64415fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:18.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.997+0000 7fa6488fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa644157c00 0x7fa644160290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:18.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.997+0000 7fa6488fc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6441520a0 con 0x7fa6440cfd10 2024-01-27T02:33:18.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.997+0000 7fa6488fc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa644152230 con 0x7fa644157c00 2024-01-27T02:33:18.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.997+0000 7fa6488fc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64415f320 con 0x7fa6440b3040 2024-01-27T02:33:18.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.997+0000 7fa641d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6440cfd10 0x7fa64415fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:18.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.997+0000 7fa641d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6440cfd10 0x7fa64415fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60744/0 (socket says 172.21.15.7:60744) 2024-01-27T02:33:18.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.997+0000 7fa641d9b700 1 -- 172.21.15.7:0/1062353411 learned_addr learned my addr 172.21.15.7:0/1062353411 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:18.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.997+0000 7fa64259c700 1 --2- 172.21.15.7:0/1062353411 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6440b3040 0x7fa6440b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:18.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.997+0000 7fa642d9d700 1 --2- 172.21.15.7:0/1062353411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa644157c00 0x7fa644160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:18.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.998+0000 7fa641d9b700 1 -- 172.21.15.7:0/1062353411 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6440b3040 msgr2=0x7fa6440b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:18.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.998+0000 7fa641d9b700 1 --2- 172.21.15.7:0/1062353411 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6440b3040 0x7fa6440b6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:18.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.998+0000 7fa641d9b700 1 -- 172.21.15.7:0/1062353411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa644157c00 msgr2=0x7fa644160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:18.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.998+0000 7fa641d9b700 1 --2- 172.21.15.7:0/1062353411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa644157c00 0x7fa644160290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:18.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.998+0000 7fa641d9b700 1 -- 172.21.15.7:0/1062353411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c009a20 con 0x7fa6440cfd10 2024-01-27T02:33:18.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.998+0000 7fa641d9b700 1 --2- 172.21.15.7:0/1062353411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6440cfd10 0x7fa64415fd50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa63800c7c0 tx=0x7fa638007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:18.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.998+0000 7fa6327fc700 1 -- 172.21.15.7:0/1062353411 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638005a70 con 0x7fa6440cfd10 2024-01-27T02:33:18.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.998+0000 7fa6327fc700 1 -- 172.21.15.7:0/1062353411 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa638005bd0 con 0x7fa6440cfd10 2024-01-27T02:33:18.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.998+0000 7fa6488fc700 1 -- 172.21.15.7:0/1062353411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6340094f0 con 0x7fa6440cfd10 2024-01-27T02:33:18.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.998+0000 7fa6327fc700 1 -- 172.21.15.7:0/1062353411 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6380070d0 con 0x7fa6440cfd10 2024-01-27T02:33:18.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.998+0000 7fa6488fc700 1 -- 172.21.15.7:0/1062353411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6440b4f60 con 0x7fa6440cfd10 2024-01-27T02:33:18.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:17.999+0000 7fa6488fc700 1 -- 172.21.15.7:0/1062353411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa64404c710 con 0x7fa6440cfd10 2024-01-27T02:33:18.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.000+0000 7fa6327fc700 1 -- 172.21.15.7:0/1062353411 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa638007480 con 0x7fa6440cfd10 2024-01-27T02:33:18.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.000+0000 7fa6327fc700 1 --2- 172.21.15.7:0/1062353411 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa624066f30 0x7fa6240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:18.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.000+0000 7fa6327fc700 1 -- 172.21.15.7:0/1062353411 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa638083630 con 0x7fa6440cfd10 2024-01-27T02:33:18.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.000+0000 7fa64259c700 1 --2- 172.21.15.7:0/1062353411 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa624066f30 0x7fa6240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:18.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.001+0000 7fa64259c700 1 --2- 172.21.15.7:0/1062353411 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa624066f30 0x7fa6240693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa6340099b0 tx=0x7fa634017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:18.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.005+0000 7fa6327fc700 1 -- 172.21.15.7:0/1062353411 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa638050ba0 con 0x7fa6440cfd10 2024-01-27T02:33:18.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.294+0000 7fa6488fc700 1 -- 172.21.15.7:0/1062353411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) v1 -- 0x7fa6440b6cb0 con 0x7fa6440cfd10 2024-01-27T02:33:18.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.468+0000 7fa6327fc700 1 -- 172.21.15.7:0/1062353411 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v87) v1 ==== 130+0+0 (secure 0 0 0) 0x7fa6380170c0 con 0x7fa6440cfd10 2024-01-27T02:33:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.471+0000 7fa6488fc700 1 -- 172.21.15.7:0/1062353411 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa624066f30 msgr2=0x7fa6240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.471+0000 7fa6488fc700 1 --2- 172.21.15.7:0/1062353411 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa624066f30 0x7fa6240693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa6340099b0 tx=0x7fa634017000 comp rx=0 tx=0).stop 2024-01-27T02:33:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.471+0000 7fa6488fc700 1 -- 172.21.15.7:0/1062353411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6440cfd10 msgr2=0x7fa64415fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.471+0000 7fa6488fc700 1 --2- 172.21.15.7:0/1062353411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6440cfd10 0x7fa64415fd50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa63800c7c0 tx=0x7fa638007f70 comp rx=0 tx=0).stop 2024-01-27T02:33:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.472+0000 7fa6488fc700 1 -- 172.21.15.7:0/1062353411 shutdown_connections 2024-01-27T02:33:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.472+0000 7fa6488fc700 1 --2- 172.21.15.7:0/1062353411 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa624066f30 0x7fa6240693e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.472+0000 7fa6488fc700 1 --2- 172.21.15.7:0/1062353411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa644157c00 0x7fa644160290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.472+0000 7fa6488fc700 1 --2- 172.21.15.7:0/1062353411 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6440b3040 0x7fa6440b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.472+0000 7fa6488fc700 1 --2- 172.21.15.7:0/1062353411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6440cfd10 0x7fa64415fd50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.472+0000 7fa6488fc700 1 -- 172.21.15.7:0/1062353411 >> 172.21.15.7:0/1062353411 conn(0x7fa644068450 msgr2=0x7fa6441593d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.472+0000 7fa6488fc700 1 -- 172.21.15.7:0/1062353411 shutdown_connections 2024-01-27T02:33:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.473+0000 7fa6488fc700 1 -- 172.21.15.7:0/1062353411 wait complete. 2024-01-27T02:33:18.489 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2024-01-27T02:33:18.489 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestClusterFull) 2024-01-27T02:33:18.490 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2024-01-27T02:33:18.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.624+0000 7fd1d6001700 1 Processor -- start 2024-01-27T02:33:18.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.624+0000 7fd1d6001700 1 -- start start 2024-01-27T02:33:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.625+0000 7fd1d6001700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1d00a7fe0 0x7fd1d006f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.625+0000 7fd1d6001700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd1d015dfb0 0x7fd1d006f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.625+0000 7fd1d6001700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1d015a880 con 0x7fd1d00a7c10 2024-01-27T02:33:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.625+0000 7fd1d6001700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1d00afde0 con 0x7fd1d015dfb0 2024-01-27T02:33:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.625+0000 7fd1d6001700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d00af950 con 0x7fd1d00a7fe0 2024-01-27T02:33:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.625+0000 7fd1cffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd1d015dfb0 0x7fd1d006f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.625+0000 7fd1cf7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd1d00a7c10 0x7fd1d00ae340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53686/0 (socket says 172.21.15.7:53686) 2024-01-27T02:33:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.625+0000 7fd1cffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd1d015dfb0 0x7fd1d006f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60756/0 (socket says 172.21.15.7:60756) 2024-01-27T02:33:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.625+0000 7fd1cf7fe700 1 -- 172.21.15.7:0/2953236756 learned_addr learned my addr 172.21.15.7:0/2953236756 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.625+0000 7fd1ceffd700 1 --2- 172.21.15.7:0/2953236756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1d00a7fe0 0x7fd1d006f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.625+0000 7fd1cffff700 1 -- 172.21.15.7:0/2953236756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1d00a7fe0 msgr2=0x7fd1d006f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.625+0000 7fd1cffff700 1 --2- 172.21.15.7:0/2953236756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1d00a7fe0 0x7fd1d006f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.625+0000 7fd1cffff700 1 -- 172.21.15.7:0/2953236756 >> v1:172.21.15.7:6789/0 conn(0x7fd1d00a7c10 legacy=0x7fd1d00ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.625+0000 7fd1cffff700 1 -- 172.21.15.7:0/2953236756 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d00aed40 con 0x7fd1d015dfb0 2024-01-27T02:33:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.625+0000 7fd1cffff700 1 --2- 172.21.15.7:0/2953236756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d015dfb0 0x7fd1d006f8a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009e60 tx=0x7fd1c8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70500d2b6de2cc86 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.626+0000 7fd1ce7fc700 1 -- 172.21.15.7:0/2953236756 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c800e070 con 0x7fd1d015dfb0 2024-01-27T02:33:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.626+0000 7fd1ce7fc700 1 -- 172.21.15.7:0/2953236756 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c800bc30 con 0x7fd1d015dfb0 2024-01-27T02:33:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.626+0000 7fd1ce7fc700 1 -- 172.21.15.7:0/2953236756 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8012430 con 0x7fd1d015dfb0 2024-01-27T02:33:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.626+0000 7fd1d6001700 1 -- 172.21.15.7:0/2953236756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d015dfb0 msgr2=0x7fd1d006f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.626+0000 7fd1d6001700 1 --2- 172.21.15.7:0/2953236756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d015dfb0 0x7fd1d006f8a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009e60 tx=0x7fd1c8009780 comp rx=0 tx=0).stop 2024-01-27T02:33:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.626+0000 7fd1d6001700 1 -- 172.21.15.7:0/2953236756 shutdown_connections 2024-01-27T02:33:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.626+0000 7fd1d6001700 1 --2- 172.21.15.7:0/2953236756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d015dfb0 0x7fd1d006f8a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.626+0000 7fd1d6001700 1 --2- 172.21.15.7:0/2953236756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1d00a7fe0 0x7fd1d006f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.626+0000 7fd1d6001700 1 -- 172.21.15.7:0/2953236756 >> 172.21.15.7:0/2953236756 conn(0x7fd1d00679b0 msgr2=0x7fd1d0067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.626+0000 7fd1d6001700 1 -- 172.21.15.7:0/2953236756 shutdown_connections 2024-01-27T02:33:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.626+0000 7fd1d6001700 1 -- 172.21.15.7:0/2953236756 wait complete. 2024-01-27T02:33:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.628+0000 7fd1d6001700 1 Processor -- start 2024-01-27T02:33:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.628+0000 7fd1d6001700 1 -- start start 2024-01-27T02:33:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.629+0000 7fd1d6001700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1d00a7c10 0x7fd1d0143a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.629+0000 7fd1d6001700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d00a7fe0 0x7fd1d015ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.629+0000 7fd1d6001700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d015dfb0 0x7fd1d015cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.629+0000 7fd1d6001700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d016b560 con 0x7fd1d00a7fe0 2024-01-27T02:33:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.629+0000 7fd1d6001700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d016bdb0 con 0x7fd1d015dfb0 2024-01-27T02:33:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.629+0000 7fd1d6001700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d0099860 con 0x7fd1d00a7c10 2024-01-27T02:33:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.629+0000 7fd1ceffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d00a7fe0 0x7fd1d015ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.629+0000 7fd1ceffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d00a7fe0 0x7fd1d015ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60762/0 (socket says 172.21.15.7:60762) 2024-01-27T02:33:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.629+0000 7fd1ceffd700 1 -- 172.21.15.7:0/432749950 learned_addr learned my addr 172.21.15.7:0/432749950 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.629+0000 7fd1cf7fe700 1 --2- 172.21.15.7:0/432749950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1d00a7c10 0x7fd1d0143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.630+0000 7fd1cffff700 1 --2- 172.21.15.7:0/432749950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d015dfb0 0x7fd1d015cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.630+0000 7fd1ceffd700 1 -- 172.21.15.7:0/432749950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1d00a7c10 msgr2=0x7fd1d0143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.630+0000 7fd1ceffd700 1 --2- 172.21.15.7:0/432749950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1d00a7c10 0x7fd1d0143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.630+0000 7fd1ceffd700 1 -- 172.21.15.7:0/432749950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d015dfb0 msgr2=0x7fd1d015cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.630+0000 7fd1ceffd700 1 --2- 172.21.15.7:0/432749950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d015dfb0 0x7fd1d015cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.630+0000 7fd1ceffd700 1 -- 172.21.15.7:0/432749950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c8009a20 con 0x7fd1d00a7fe0 2024-01-27T02:33:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.630+0000 7fd1ceffd700 1 --2- 172.21.15.7:0/432749950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d00a7fe0 0x7fd1d015ca20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400d860 tx=0x7fd1c4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.630+0000 7fd1b77fe700 1 -- 172.21.15.7:0/432749950 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c40125f0 con 0x7fd1d00a7fe0 2024-01-27T02:33:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.630+0000 7fd1b77fe700 1 -- 172.21.15.7:0/432749950 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c4012750 con 0x7fd1d00a7fe0 2024-01-27T02:33:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.630+0000 7fd1d6001700 1 -- 172.21.15.7:0/432749950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d016b760 con 0x7fd1d00a7fe0 2024-01-27T02:33:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.630+0000 7fd1b77fe700 1 -- 172.21.15.7:0/432749950 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4012af0 con 0x7fd1d00a7fe0 2024-01-27T02:33:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.630+0000 7fd1d6001700 1 -- 172.21.15.7:0/432749950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d0143f80 con 0x7fd1d00a7fe0 2024-01-27T02:33:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.631+0000 7fd1d6001700 1 -- 172.21.15.7:0/432749950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d016bf40 con 0x7fd1d00a7fe0 2024-01-27T02:33:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.632+0000 7fd1b77fe700 1 -- 172.21.15.7:0/432749950 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd1c4013480 con 0x7fd1d00a7fe0 2024-01-27T02:33:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.632+0000 7fd1b77fe700 1 --2- 172.21.15.7:0/432749950 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd1bc066f30 0x7fd1bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.632+0000 7fd1b77fe700 1 -- 172.21.15.7:0/432749950 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fd1c4084590 con 0x7fd1d00a7fe0 2024-01-27T02:33:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.632+0000 7fd1cf7fe700 1 --2- 172.21.15.7:0/432749950 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd1bc066f30 0x7fd1bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.633+0000 7fd1cf7fe700 1 --2- 172.21.15.7:0/432749950 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd1bc066f30 0x7fd1bc0693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0007130 tx=0x7fd1c0007eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.637+0000 7fd1b77fe700 1 -- 172.21.15.7:0/432749950 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd1c4055590 con 0x7fd1d00a7fe0 2024-01-27T02:33:18.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:18.923+0000 7fd1d6001700 1 -- 172.21.15.7:0/432749950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) v1 -- 0x7fd1d004c710 con 0x7fd1d00a7fe0 2024-01-27T02:33:19.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.471+0000 7fd1b77fe700 1 -- 172.21.15.7:0/432749950 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v88) v1 ==== 145+0+0 (secure 0 0 0) 0x7fd1c4055120 con 0x7fd1d00a7fe0 2024-01-27T02:33:19.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.474+0000 7fd1d6001700 1 -- 172.21.15.7:0/432749950 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd1bc066f30 msgr2=0x7fd1bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:19.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.474+0000 7fd1d6001700 1 --2- 172.21.15.7:0/432749950 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd1bc066f30 0x7fd1bc0693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0007130 tx=0x7fd1c0007eb0 comp rx=0 tx=0).stop 2024-01-27T02:33:19.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.474+0000 7fd1d6001700 1 -- 172.21.15.7:0/432749950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d00a7fe0 msgr2=0x7fd1d015ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:19.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.474+0000 7fd1d6001700 1 --2- 172.21.15.7:0/432749950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d00a7fe0 0x7fd1d015ca20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400d860 tx=0x7fd1c4008040 comp rx=0 tx=0).stop 2024-01-27T02:33:19.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.474+0000 7fd1d6001700 1 -- 172.21.15.7:0/432749950 shutdown_connections 2024-01-27T02:33:19.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.474+0000 7fd1d6001700 1 --2- 172.21.15.7:0/432749950 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd1bc066f30 0x7fd1bc0693e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:19.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.474+0000 7fd1d6001700 1 --2- 172.21.15.7:0/432749950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d015dfb0 0x7fd1d015cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:19.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.474+0000 7fd1d6001700 1 --2- 172.21.15.7:0/432749950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1d00a7c10 0x7fd1d0143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:19.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.474+0000 7fd1d6001700 1 --2- 172.21.15.7:0/432749950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d00a7fe0 0x7fd1d015ca20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:19.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.474+0000 7fd1d6001700 1 -- 172.21.15.7:0/432749950 >> 172.21.15.7:0/432749950 conn(0x7fd1d00679b0 msgr2=0x7fd1d01589b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:19.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.474+0000 7fd1d6001700 1 -- 172.21.15.7:0/432749950 shutdown_connections 2024-01-27T02:33:19.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.474+0000 7fd1d6001700 1 -- 172.21.15.7:0/432749950 wait complete. 2024-01-27T02:33:19.492 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-01-27T02:33:19.643 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_objectstore":"memstore"} 2024-01-27T02:33:19.644 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-01-27T02:33:19.644 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-27T02:33:19.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.776+0000 7fae3e8af700 1 Processor -- start 2024-01-27T02:33:19.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.777+0000 7fae3e8af700 1 -- start start 2024-01-27T02:33:19.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.777+0000 7fae3e8af700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fae380680c0 0x7fae380693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:19.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.777+0000 7fae3e8af700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae38069930 con 0x7fae380b0f70 2024-01-27T02:33:19.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.777+0000 7fae3e8af700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3815e820 con 0x7fae380cc8d0 2024-01-27T02:33:19.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.777+0000 7fae3e8af700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae38068840 con 0x7fae380680c0 2024-01-27T02:33:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.778+0000 7fae37fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fae380b0f70 0x7fae380679b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53698/0 (socket says 172.21.15.7:53698) 2024-01-27T02:33:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.778+0000 7fae37fff700 1 -- 172.21.15.7:0/817466106 learned_addr learned my addr 172.21.15.7:0/817466106 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.778+0000 7fae377fe700 1 --2- 172.21.15.7:0/817466106 >> v2:172.21.15.188:3300/0 conn(0x7fae380680c0 0x7fae380693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.778+0000 7fae36ffd700 1 -- 172.21.15.7:0/817466106 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632254401 0 0) 0x7fae38069930 con 0x7fae380b0f70 2024-01-27T02:33:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.778+0000 7fae36ffd700 1 -- 172.21.15.7:0/817466106 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae20003570 con 0x7fae380b0f70 2024-01-27T02:33:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.778+0000 7fae36ffd700 1 -- 172.21.15.7:0/817466106 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 227256261 0 0) 0x7fae3815e820 con 0x7fae380cc8d0 2024-01-27T02:33:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.778+0000 7fae36ffd700 1 -- 172.21.15.7:0/817466106 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae38069930 con 0x7fae380cc8d0 2024-01-27T02:33:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.778+0000 7fae36ffd700 1 -- 172.21.15.7:0/817466106 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2967721016 0 0) 0x7fae20003570 con 0x7fae380b0f70 2024-01-27T02:33:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.778+0000 7fae36ffd700 1 -- 172.21.15.7:0/817466106 >> v2:172.21.15.188:3300/0 conn(0x7fae380680c0 msgr2=0x7fae380693f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.778+0000 7fae36ffd700 1 --2- 172.21.15.7:0/817466106 >> v2:172.21.15.188:3300/0 conn(0x7fae380680c0 0x7fae380693f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.778+0000 7fae36ffd700 1 -- 172.21.15.7:0/817466106 >> v1:172.21.15.188:6789/0 conn(0x7fae380cc8d0 legacy=0x7fae380c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.778+0000 7fae36ffd700 1 -- 172.21.15.7:0/817466106 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3815dc10 con 0x7fae380b0f70 2024-01-27T02:33:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.778+0000 7fae36ffd700 1 -- 172.21.15.7:0/817466106 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7fae28001470 con 0x7fae380b0f70 2024-01-27T02:33:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.778+0000 7fae36ffd700 1 -- 172.21.15.7:0/817466106 >> v1:172.21.15.7:6789/0 conn(0x7fae380b0f70 legacy=0x7fae380679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.778+0000 7fae36ffd700 1 --2- 172.21.15.7:0/817466106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae20003c40 0x7fae20004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.778+0000 7fae36ffd700 1 --2- 172.21.15.7:0/817466106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae20004810 0x7fae20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.778+0000 7fae36ffd700 1 --2- 172.21.15.7:0/817466106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae20005210 0x7fae2000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.778+0000 7fae36ffd700 1 -- 172.21.15.7:0/817466106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae38068840 con 0x7fae20004810 2024-01-27T02:33:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.778+0000 7fae36ffd700 1 -- 172.21.15.7:0/817466106 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2000c130 con 0x7fae20003c40 2024-01-27T02:33:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.778+0000 7fae36ffd700 1 -- 172.21.15.7:0/817466106 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2000c2c0 con 0x7fae20005210 2024-01-27T02:33:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.778+0000 7fae377fe700 1 --2- 172.21.15.7:0/817466106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae20004810 0x7fae20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.779+0000 7fae37fff700 1 --2- 172.21.15.7:0/817466106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae20003c40 0x7fae20004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.779+0000 7fae3ce4c700 1 --2- 172.21.15.7:0/817466106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae20005210 0x7fae2000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.779+0000 7fae377fe700 1 -- 172.21.15.7:0/817466106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae20005210 msgr2=0x7fae2000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.779+0000 7fae377fe700 1 --2- 172.21.15.7:0/817466106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae20005210 0x7fae2000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:19.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.779+0000 7fae377fe700 1 -- 172.21.15.7:0/817466106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae20003c40 msgr2=0x7fae20004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:19.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.779+0000 7fae377fe700 1 --2- 172.21.15.7:0/817466106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae20003c40 0x7fae20004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:19.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.779+0000 7fae377fe700 1 -- 172.21.15.7:0/817466106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae20003570 con 0x7fae20004810 2024-01-27T02:33:19.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.779+0000 7fae3ce4c700 1 -- 172.21.15.7:0/817466106 reap_dead start 2024-01-27T02:33:19.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.779+0000 7fae377fe700 1 --2- 172.21.15.7:0/817466106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae20004810 0x7fae20004c80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009c00 tx=0x7fae2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15312b7849ab6bf server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:19.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.779+0000 7fae36ffd700 1 -- 172.21.15.7:0/817466106 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c00e710 con 0x7fae20004810 2024-01-27T02:33:19.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.779+0000 7fae36ffd700 1 -- 172.21.15.7:0/817466106 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2c00e870 con 0x7fae20004810 2024-01-27T02:33:19.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.779+0000 7fae36ffd700 1 -- 172.21.15.7:0/817466106 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c00ec30 con 0x7fae20004810 2024-01-27T02:33:19.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.780+0000 7fae3e8af700 1 -- 172.21.15.7:0/817466106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae20004810 msgr2=0x7fae20004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:19.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.780+0000 7fae3e8af700 1 --2- 172.21.15.7:0/817466106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae20004810 0x7fae20004c80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009c00 tx=0x7fae2c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:19.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.780+0000 7fae3e8af700 1 -- 172.21.15.7:0/817466106 shutdown_connections 2024-01-27T02:33:19.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.780+0000 7fae3e8af700 1 --2- 172.21.15.7:0/817466106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae20004810 0x7fae20004c80 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:19.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.780+0000 7fae3e8af700 1 -- 172.21.15.7:0/817466106 >> 172.21.15.7:0/817466106 conn(0x7fae38063580 msgr2=0x7fae38063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:19.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.780+0000 7fae3e8af700 1 -- 172.21.15.7:0/817466106 shutdown_connections 2024-01-27T02:33:19.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.780+0000 7fae3e8af700 1 -- 172.21.15.7:0/817466106 wait complete. 2024-01-27T02:33:19.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.781+0000 7fae3e8af700 1 Processor -- start 2024-01-27T02:33:19.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.781+0000 7fae3e8af700 1 -- start start 2024-01-27T02:33:19.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.782+0000 7fae3e8af700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae20004810 0x7fae381496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:19.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.782+0000 7fae3e8af700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae380c9c10 0x7fae380ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:19.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.782+0000 7fae3e8af700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae3815b450 0x7fae3815b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:19.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.782+0000 7fae3e8af700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae380ca5c0 con 0x7fae380c9c10 2024-01-27T02:33:19.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.782+0000 7fae3e8af700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3815cd70 con 0x7fae20004810 2024-01-27T02:33:19.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.782+0000 7fae3e8af700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3815c160 con 0x7fae3815b450 2024-01-27T02:33:19.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.782+0000 7fae377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae380c9c10 0x7fae380ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:19.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.782+0000 7fae377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae380c9c10 0x7fae380ca080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60770/0 (socket says 172.21.15.7:60770) 2024-01-27T02:33:19.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.782+0000 7fae377fe700 1 -- 172.21.15.7:0/2428871024 learned_addr learned my addr 172.21.15.7:0/2428871024 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:19.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.782+0000 7fae37fff700 1 --2- 172.21.15.7:0/2428871024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae20004810 0x7fae381496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:19.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.782+0000 7fae3ce4c700 1 --2- 172.21.15.7:0/2428871024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae3815b450 0x7fae3815b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:19.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.783+0000 7fae377fe700 1 -- 172.21.15.7:0/2428871024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae3815b450 msgr2=0x7fae3815b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:19.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.783+0000 7fae377fe700 1 --2- 172.21.15.7:0/2428871024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae3815b450 0x7fae3815b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:19.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.783+0000 7fae377fe700 1 -- 172.21.15.7:0/2428871024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae20004810 msgr2=0x7fae381496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:19.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.783+0000 7fae377fe700 1 --2- 172.21.15.7:0/2428871024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae20004810 0x7fae381496a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:19.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.783+0000 7fae377fe700 1 -- 172.21.15.7:0/2428871024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3815c440 con 0x7fae380c9c10 2024-01-27T02:33:19.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.783+0000 7fae377fe700 1 --2- 172.21.15.7:0/2428871024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae380c9c10 0x7fae380ca080 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00e4b0 tx=0x7fae2c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:19.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.783+0000 7fae1ffff700 1 -- 172.21.15.7:0/2428871024 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c022070 con 0x7fae380c9c10 2024-01-27T02:33:19.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.783+0000 7fae1ffff700 1 -- 172.21.15.7:0/2428871024 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2c00e9e0 con 0x7fae380c9c10 2024-01-27T02:33:19.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.783+0000 7fae1ffff700 1 -- 172.21.15.7:0/2428871024 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c01c6a0 con 0x7fae380c9c10 2024-01-27T02:33:19.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.783+0000 7fae3e8af700 1 -- 172.21.15.7:0/2428871024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3815c720 con 0x7fae380c9c10 2024-01-27T02:33:19.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.783+0000 7fae3e8af700 1 -- 172.21.15.7:0/2428871024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3815be00 con 0x7fae380c9c10 2024-01-27T02:33:19.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.784+0000 7fae3e8af700 1 -- 172.21.15.7:0/2428871024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3815cee0 con 0x7fae380c9c10 2024-01-27T02:33:19.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.784+0000 7fae1ffff700 1 -- 172.21.15.7:0/2428871024 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fae2c01cad0 con 0x7fae380c9c10 2024-01-27T02:33:19.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.785+0000 7fae1ffff700 1 --2- 172.21.15.7:0/2428871024 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fae24066f30 0x7fae240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:19.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.785+0000 7fae1ffff700 1 -- 172.21.15.7:0/2428871024 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fae2c085970 con 0x7fae380c9c10 2024-01-27T02:33:19.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.785+0000 7fae37fff700 1 --2- 172.21.15.7:0/2428871024 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fae24066f30 0x7fae240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:19.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.786+0000 7fae37fff700 1 --2- 172.21.15.7:0/2428871024 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fae24066f30 0x7fae240693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fae28001200 tx=0x7fae28001d40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:19.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.789+0000 7fae1ffff700 1 -- 172.21.15.7:0/2428871024 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae2c052f20 con 0x7fae380c9c10 2024-01-27T02:33:19.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.994+0000 7fae3e8af700 1 -- 172.21.15.7:0/2428871024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fae38072060 con 0x7fae380c9c10 2024-01-27T02:33:19.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:19.998+0000 7fae1ffff700 1 -- 172.21.15.7:0/2428871024 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v4)=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7fae2c0170c0 con 0x7fae380c9c10 2024-01-27T02:33:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.000+0000 7fae3e8af700 1 -- 172.21.15.7:0/2428871024 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fae24066f30 msgr2=0x7fae240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.000+0000 7fae3e8af700 1 --2- 172.21.15.7:0/2428871024 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fae24066f30 0x7fae240693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fae28001200 tx=0x7fae28001d40 comp rx=0 tx=0).stop 2024-01-27T02:33:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.000+0000 7fae3e8af700 1 -- 172.21.15.7:0/2428871024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae380c9c10 msgr2=0x7fae380ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.000+0000 7fae3e8af700 1 --2- 172.21.15.7:0/2428871024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae380c9c10 0x7fae380ca080 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00e4b0 tx=0x7fae2c01a000 comp rx=0 tx=0).stop 2024-01-27T02:33:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.000+0000 7fae3e8af700 1 -- 172.21.15.7:0/2428871024 shutdown_connections 2024-01-27T02:33:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.000+0000 7fae3e8af700 1 --2- 172.21.15.7:0/2428871024 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fae24066f30 0x7fae240693e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.000+0000 7fae3e8af700 1 --2- 172.21.15.7:0/2428871024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae3815b450 0x7fae3815b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.000+0000 7fae3e8af700 1 --2- 172.21.15.7:0/2428871024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae20004810 0x7fae381496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.000+0000 7fae3e8af700 1 --2- 172.21.15.7:0/2428871024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae380c9c10 0x7fae380ca080 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.000+0000 7fae3e8af700 1 -- 172.21.15.7:0/2428871024 >> 172.21.15.7:0/2428871024 conn(0x7fae38063580 msgr2=0x7fae38157310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.000+0000 7fae3e8af700 1 -- 172.21.15.7:0/2428871024 shutdown_connections 2024-01-27T02:33:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.000+0000 7fae3e8af700 1 -- 172.21.15.7:0/2428871024 wait complete. 2024-01-27T02:33:20.016 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-27T02:33:20.016 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:33:20.065 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:33:20.123 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-27T02:33:20.123 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-27T02:33:20.123 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:33:20.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.260+0000 7fb00dff5700 1 Processor -- start 2024-01-27T02:33:20.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.261+0000 7fb00dff5700 1 -- start start 2024-01-27T02:33:20.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.261+0000 7fb00dff5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0080b1f50 0x7fb0080cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:20.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.261+0000 7fb00dff5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0080c4bd0 con 0x7fb0080b1af0 2024-01-27T02:33:20.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.261+0000 7fb00dff5700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0080cac20 con 0x7fb0080ca140 2024-01-27T02:33:20.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.261+0000 7fb00dff5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080b2370 con 0x7fb0080b1f50 2024-01-27T02:33:20.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.262+0000 7fb006ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb0080b1af0 0x7fb0080c5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53704/0 (socket says 172.21.15.7:53704) 2024-01-27T02:33:20.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.262+0000 7fb006ffd700 1 -- 172.21.15.7:0/796654015 learned_addr learned my addr 172.21.15.7:0/796654015 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:20.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.262+0000 7fb007fff700 1 --2- 172.21.15.7:0/796654015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0080b1f50 0x7fb0080cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:20.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.262+0000 7fb0067fc700 1 -- 172.21.15.7:0/796654015 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4143412789 0 0) 0x7fb0080c4bd0 con 0x7fb0080b1af0 2024-01-27T02:33:20.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.262+0000 7fb0067fc700 1 -- 172.21.15.7:0/796654015 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faff0003570 con 0x7fb0080b1af0 2024-01-27T02:33:20.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.262+0000 7fb0067fc700 1 -- 172.21.15.7:0/796654015 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2002051245 0 0) 0x7fb0080cac20 con 0x7fb0080ca140 2024-01-27T02:33:20.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.262+0000 7fb0067fc700 1 -- 172.21.15.7:0/796654015 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0080c4bd0 con 0x7fb0080ca140 2024-01-27T02:33:20.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.262+0000 7fb007fff700 1 -- 172.21.15.7:0/796654015 >> v1:172.21.15.188:6789/0 conn(0x7fb0080ca140 legacy=0x7fb0080ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:20.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.262+0000 7fb007fff700 1 -- 172.21.15.7:0/796654015 >> v1:172.21.15.7:6789/0 conn(0x7fb0080b1af0 legacy=0x7fb0080c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:20.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.262+0000 7fb007fff700 1 -- 172.21.15.7:0/796654015 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0080caea0 con 0x7fb0080b1f50 2024-01-27T02:33:20.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.262+0000 7fb007fff700 1 --2- 172.21.15.7:0/796654015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0080b1f50 0x7fb0080cdb20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb000009e90 tx=0x7fb000009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e82fc6819073321 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:20.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.263+0000 7fb0067fc700 1 -- 172.21.15.7:0/796654015 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00000e070 con 0x7fb0080b1f50 2024-01-27T02:33:20.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.263+0000 7fb0067fc700 1 -- 172.21.15.7:0/796654015 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00000bb80 con 0x7fb0080b1f50 2024-01-27T02:33:20.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.263+0000 7fb0067fc700 1 -- 172.21.15.7:0/796654015 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb000012430 con 0x7fb0080b1f50 2024-01-27T02:33:20.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.263+0000 7fb00dff5700 1 -- 172.21.15.7:0/796654015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0080b1f50 msgr2=0x7fb0080cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:20.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.263+0000 7fb00dff5700 1 --2- 172.21.15.7:0/796654015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0080b1f50 0x7fb0080cdb20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb000009e90 tx=0x7fb000009780 comp rx=0 tx=0).stop 2024-01-27T02:33:20.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.263+0000 7fb00dff5700 1 -- 172.21.15.7:0/796654015 shutdown_connections 2024-01-27T02:33:20.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.263+0000 7fb00dff5700 1 --2- 172.21.15.7:0/796654015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0080b1f50 0x7fb0080cdb20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:20.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.263+0000 7fb00dff5700 1 -- 172.21.15.7:0/796654015 >> 172.21.15.7:0/796654015 conn(0x7fb0080711c0 msgr2=0x7fb0080ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:20.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.263+0000 7fb00dff5700 1 -- 172.21.15.7:0/796654015 shutdown_connections 2024-01-27T02:33:20.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.263+0000 7fb00dff5700 1 -- 172.21.15.7:0/796654015 wait complete. 2024-01-27T02:33:20.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.265+0000 7fb00dff5700 1 Processor -- start 2024-01-27T02:33:20.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.265+0000 7fb00dff5700 1 -- start start 2024-01-27T02:33:20.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.266+0000 7fb00dff5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0080b1af0 0x7fb0081476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:20.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.266+0000 7fb00dff5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0080b1f50 0x7fb0080cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:20.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.266+0000 7fb00dff5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0080ca140 0x7fb00815e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:20.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.266+0000 7fb00dff5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080ce2c0 con 0x7fb0080b1af0 2024-01-27T02:33:20.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.266+0000 7fb00dff5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080cd6b0 con 0x7fb0080ca140 2024-01-27T02:33:20.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.266+0000 7fb00dff5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080cd990 con 0x7fb0080b1f50 2024-01-27T02:33:20.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.266+0000 7fb0077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0080b1af0 0x7fb0081476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:20.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.266+0000 7fb0077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0080b1af0 0x7fb0081476d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60780/0 (socket says 172.21.15.7:60780) 2024-01-27T02:33:20.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.266+0000 7fb0077fe700 1 -- 172.21.15.7:0/2991025070 learned_addr learned my addr 172.21.15.7:0/2991025070 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:20.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.266+0000 7fb007fff700 1 --2- 172.21.15.7:0/2991025070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0080ca140 0x7fb00815e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:20.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.266+0000 7fb006ffd700 1 --2- 172.21.15.7:0/2991025070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0080b1f50 0x7fb0080cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:20.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.266+0000 7fb0077fe700 1 -- 172.21.15.7:0/2991025070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0080b1f50 msgr2=0x7fb0080cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:20.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.266+0000 7fb0077fe700 1 --2- 172.21.15.7:0/2991025070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0080b1f50 0x7fb0080cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:20.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.266+0000 7fb0077fe700 1 -- 172.21.15.7:0/2991025070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0080ca140 msgr2=0x7fb00815e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:20.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.266+0000 7fb0077fe700 1 --2- 172.21.15.7:0/2991025070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0080ca140 0x7fb00815e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:20.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.266+0000 7fb0077fe700 1 -- 172.21.15.7:0/2991025070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb000009a20 con 0x7fb0080b1af0 2024-01-27T02:33:20.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.266+0000 7fb0077fe700 1 --2- 172.21.15.7:0/2991025070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0080b1af0 0x7fb0081476d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7faff800c560 tx=0x7faff8007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:20.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.266+0000 7fafef7fe700 1 -- 172.21.15.7:0/2991025070 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8010510 con 0x7fb0080b1af0 2024-01-27T02:33:20.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.267+0000 7fafef7fe700 1 -- 172.21.15.7:0/2991025070 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff8010670 con 0x7fb0080b1af0 2024-01-27T02:33:20.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.267+0000 7fb00dff5700 1 -- 172.21.15.7:0/2991025070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0080cdc70 con 0x7fb0080b1af0 2024-01-27T02:33:20.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.267+0000 7fafef7fe700 1 -- 172.21.15.7:0/2991025070 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8010a30 con 0x7fb0080b1af0 2024-01-27T02:33:20.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.267+0000 7fb00dff5700 1 -- 172.21.15.7:0/2991025070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb008147c10 con 0x7fb0080b1af0 2024-01-27T02:33:20.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.267+0000 7fb00dff5700 1 -- 172.21.15.7:0/2991025070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00804c710 con 0x7fb0080b1af0 2024-01-27T02:33:20.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.268+0000 7fafef7fe700 1 -- 172.21.15.7:0/2991025070 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7faff801b400 con 0x7fb0080b1af0 2024-01-27T02:33:20.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.268+0000 7fafef7fe700 1 --2- 172.21.15.7:0/2991025070 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faff4066f30 0x7faff40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:20.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.269+0000 7fafef7fe700 1 -- 172.21.15.7:0/2991025070 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4939+0+0 (secure 0 0 0) 0x7faff8083ed0 con 0x7fb0080b1af0 2024-01-27T02:33:20.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.269+0000 7fb006ffd700 1 --2- 172.21.15.7:0/2991025070 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faff4066f30 0x7faff40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:20.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.269+0000 7fb006ffd700 1 --2- 172.21.15.7:0/2991025070 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faff4066f30 0x7faff40693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb00815edf0 tx=0x7faffc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:20.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.273+0000 7fafef7fe700 1 -- 172.21.15.7:0/2991025070 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faff8051440 con 0x7fb0080b1af0 2024-01-27T02:33:20.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.525+0000 7fb00dff5700 1 -- 172.21.15.7:0/2991025070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0080cb180 con 0x7fb0080b1af0 2024-01-27T02:33:20.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.525+0000 7fafef7fe700 1 -- 172.21.15.7:0/2991025070 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3526 (secure 0 0 0) 0x7faff8054a60 con 0x7fb0080b1af0 2024-01-27T02:33:20.527 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:20.527 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4864,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/892460188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":892460188},{"type":"v1","addr":"172.21.15.188:6837","nonce":892460188}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4867,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1657319349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1657319349},{"type":"v1","addr":"172.21.15.188:6836","nonce":1657319349}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4879,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2190571534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2190571534},{"type":"v1","addr":"172.21.15.7:6834","nonce":2190571534}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2038868395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2038868395},{"type":"v1","addr":"172.21.15.7:6835","nonce":2038868395}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2024-01-27T02:33:20.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.528+0000 7fb00dff5700 1 -- 172.21.15.7:0/2991025070 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faff4066f30 msgr2=0x7faff40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:20.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.528+0000 7fb00dff5700 1 --2- 172.21.15.7:0/2991025070 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faff4066f30 0x7faff40693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb00815edf0 tx=0x7faffc009040 comp rx=0 tx=0).stop 2024-01-27T02:33:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.528+0000 7fb00dff5700 1 -- 172.21.15.7:0/2991025070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0080b1af0 msgr2=0x7fb0081476d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.528+0000 7fb00dff5700 1 --2- 172.21.15.7:0/2991025070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0080b1af0 0x7fb0081476d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7faff800c560 tx=0x7faff8007dc0 comp rx=0 tx=0).stop 2024-01-27T02:33:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.528+0000 7fb00dff5700 1 -- 172.21.15.7:0/2991025070 shutdown_connections 2024-01-27T02:33:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.528+0000 7fb00dff5700 1 --2- 172.21.15.7:0/2991025070 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faff4066f30 0x7faff40693e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.528+0000 7fb00dff5700 1 --2- 172.21.15.7:0/2991025070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0080ca140 0x7fb00815e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.528+0000 7fb00dff5700 1 --2- 172.21.15.7:0/2991025070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0080b1af0 0x7fb0081476d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.528+0000 7fb00dff5700 1 --2- 172.21.15.7:0/2991025070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0080b1f50 0x7fb0080cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.528+0000 7fb00dff5700 1 -- 172.21.15.7:0/2991025070 >> 172.21.15.7:0/2991025070 conn(0x7fb0080711c0 msgr2=0x7fb0080b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.528+0000 7fb00dff5700 1 -- 172.21.15.7:0/2991025070 shutdown_connections 2024-01-27T02:33:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.528+0000 7fb00dff5700 1 -- 172.21.15.7:0/2991025070 wait complete. 2024-01-27T02:33:20.530 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 19 2024-01-27T02:33:20.543 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-27T02:33:20.543 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-27T02:33:20.543 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-27T02:33:20.543 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:33:20.544 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-27T02:33:20.544 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-27T02:33:20.544 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-27T02:33:20.544 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:33:20.544 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-27T02:33:20.544 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-27T02:33:20.544 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-27T02:33:20.544 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:33:20.545 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-27T02:33:20.545 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-27T02:33:20.545 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-27T02:33:20.545 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:33:20.552 INFO:tasks.ceph.mds.a:Stopped 2024-01-27T02:33:20.553 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-27T02:33:20.553 INFO:tasks.ceph.mds.c:Stopped 2024-01-27T02:33:20.554 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-27T02:33:20.555 INFO:tasks.ceph.mds.b:Stopped 2024-01-27T02:33:20.555 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-27T02:33:20.556 INFO:tasks.ceph.mds.d:Stopped 2024-01-27T02:33:20.556 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-27T02:33:20.595 INFO:tasks.ceph.mds.a:Started 2024-01-27T02:33:20.595 DEBUG:teuthology.parallel:result is None 2024-01-27T02:33:20.595 INFO:tasks.ceph.mds.c:Started 2024-01-27T02:33:20.596 DEBUG:teuthology.parallel:result is None 2024-01-27T02:33:20.598 INFO:tasks.ceph.mds.b:Started 2024-01-27T02:33:20.598 DEBUG:teuthology.parallel:result is None 2024-01-27T02:33:20.599 INFO:tasks.ceph.mds.d:Started 2024-01-27T02:33:20.599 DEBUG:teuthology.parallel:result is None 2024-01-27T02:33:20.599 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-27T02:33:20.684 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-01-27T02:33:20.684 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-01-27T02:33:20.685 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-27T02:33:20.685 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-27T02:33:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.738+0000 7f1a73cc9700 1 Processor -- start 2024-01-27T02:33:20.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.738+0000 7f1a73cc9700 1 -- start start 2024-01-27T02:33:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.739+0000 7f1a73cc9700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1a6c0ce310 0x7f1a6c0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.739+0000 7f1a73cc9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6c0ce730 con 0x7f1a6c15a230 2024-01-27T02:33:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.739+0000 7f1a73cc9700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6c0c5320 con 0x7f1a6c0c4cc0 2024-01-27T02:33:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.739+0000 7f1a73cc9700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6c0c9520 con 0x7f1a6c0ce310 2024-01-27T02:33:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.739+0000 7f1a71a65700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1a6c15a230 0x7f1a6c0c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53714/0 (socket says 172.21.15.7:53714) 2024-01-27T02:33:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.740+0000 7f1a71a65700 1 -- 172.21.15.7:0/2193167098 learned_addr learned my addr 172.21.15.7:0/2193167098 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.740+0000 7f1a72266700 1 --2- 172.21.15.7:0/2193167098 >> v2:172.21.15.188:3300/0 conn(0x7f1a6c0ce310 0x7f1a6c0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.740+0000 7f1a70a63700 1 -- 172.21.15.7:0/2193167098 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724863124 0 0) 0x7f1a6c0ce730 con 0x7f1a6c15a230 2024-01-27T02:33:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.740+0000 7f1a70a63700 1 -- 172.21.15.7:0/2193167098 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a58003570 con 0x7f1a6c15a230 2024-01-27T02:33:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.740+0000 7f1a70a63700 1 -- 172.21.15.7:0/2193167098 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597108186 0 0) 0x7f1a6c0c5320 con 0x7f1a6c0c4cc0 2024-01-27T02:33:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.740+0000 7f1a70a63700 1 -- 172.21.15.7:0/2193167098 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a6c0ce730 con 0x7f1a6c0c4cc0 2024-01-27T02:33:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.740+0000 7f1a72266700 1 -- 172.21.15.7:0/2193167098 >> v1:172.21.15.188:6789/0 conn(0x7f1a6c0c4cc0 legacy=0x7f1a6c0ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.740+0000 7f1a72266700 1 -- 172.21.15.7:0/2193167098 >> v1:172.21.15.7:6789/0 conn(0x7f1a6c15a230 legacy=0x7f1a6c0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:20.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.740+0000 7f1a72266700 1 -- 172.21.15.7:0/2193167098 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6c0c9800 con 0x7f1a6c0ce310 2024-01-27T02:33:20.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.740+0000 7f1a72266700 1 --2- 172.21.15.7:0/2193167098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a6c0ce310 0x7f1a6c0cd2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009e60 tx=0x7f1a64009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8cc71c8e9557970 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:20.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.740+0000 7f1a70a63700 1 -- 172.21.15.7:0/2193167098 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6400e070 con 0x7f1a6c0ce310 2024-01-27T02:33:20.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.741+0000 7f1a70a63700 1 -- 172.21.15.7:0/2193167098 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6400bab0 con 0x7f1a6c0ce310 2024-01-27T02:33:20.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.741+0000 7f1a70a63700 1 -- 172.21.15.7:0/2193167098 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6400be70 con 0x7f1a6c0ce310 2024-01-27T02:33:20.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.741+0000 7f1a73cc9700 1 -- 172.21.15.7:0/2193167098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a6c0ce310 msgr2=0x7f1a6c0cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:20.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.741+0000 7f1a73cc9700 1 --2- 172.21.15.7:0/2193167098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a6c0ce310 0x7f1a6c0cd2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009e60 tx=0x7f1a64009780 comp rx=0 tx=0).stop 2024-01-27T02:33:20.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.741+0000 7f1a73cc9700 1 -- 172.21.15.7:0/2193167098 shutdown_connections 2024-01-27T02:33:20.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.741+0000 7f1a73cc9700 1 --2- 172.21.15.7:0/2193167098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a6c0ce310 0x7f1a6c0cd2f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.741+0000 7f1a73cc9700 1 -- 172.21.15.7:0/2193167098 >> 172.21.15.7:0/2193167098 conn(0x7f1a6c098020 msgr2=0x7f1a6c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.741+0000 7f1a73cc9700 1 -- 172.21.15.7:0/2193167098 shutdown_connections 2024-01-27T02:33:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.741+0000 7f1a73cc9700 1 -- 172.21.15.7:0/2193167098 wait complete. 2024-01-27T02:33:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.743+0000 7f1a73cc9700 1 Processor -- start 2024-01-27T02:33:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.743+0000 7f1a73cc9700 1 -- start start 2024-01-27T02:33:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.743+0000 7f1a73cc9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a6c0c4cc0 0x7f1a6c0cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.743+0000 7f1a73cc9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a6c0ce310 0x7f1a6c0b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.743+0000 7f1a73cc9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a6c15a230 0x7f1a6c16b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.743+0000 7f1a73cc9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0cbdb0 con 0x7f1a6c0c4cc0 2024-01-27T02:33:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.743+0000 7f1a73cc9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0cb3e0 con 0x7f1a6c0ce310 2024-01-27T02:33:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.743+0000 7f1a73cc9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0b1d80 con 0x7f1a6c15a230 2024-01-27T02:33:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.743+0000 7f1a71a65700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a6c0c4cc0 0x7f1a6c0cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.743+0000 7f1a71a65700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a6c0c4cc0 0x7f1a6c0cb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60804/0 (socket says 172.21.15.7:60804) 2024-01-27T02:33:20.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.744+0000 7f1a71a65700 1 -- 172.21.15.7:0/3103156925 learned_addr learned my addr 172.21.15.7:0/3103156925 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:20.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.744+0000 7f1a71264700 1 --2- 172.21.15.7:0/3103156925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a6c0ce310 0x7f1a6c0b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:20.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.744+0000 7f1a72266700 1 --2- 172.21.15.7:0/3103156925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a6c15a230 0x7f1a6c16b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:20.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.744+0000 7f1a71a65700 1 -- 172.21.15.7:0/3103156925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a6c15a230 msgr2=0x7f1a6c16b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:20.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.744+0000 7f1a71a65700 1 --2- 172.21.15.7:0/3103156925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a6c15a230 0x7f1a6c16b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:20.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.744+0000 7f1a71a65700 1 -- 172.21.15.7:0/3103156925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a6c0ce310 msgr2=0x7f1a6c0b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:20.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.744+0000 7f1a71a65700 1 --2- 172.21.15.7:0/3103156925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a6c0ce310 0x7f1a6c0b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:20.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.744+0000 7f1a71a65700 1 -- 172.21.15.7:0/3103156925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a64009a20 con 0x7f1a6c0c4cc0 2024-01-27T02:33:20.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.744+0000 7f1a71264700 1 --2- 172.21.15.7:0/3103156925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a6c0ce310 0x7f1a6c0b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:20.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.744+0000 7f1a71a65700 1 --2- 172.21.15.7:0/3103156925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a6c0c4cc0 0x7f1a6c0cb870 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800c7c0 tx=0x7f1a68007ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:20.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.744+0000 7f1a61ffb700 1 -- 172.21.15.7:0/3103156925 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a680072e0 con 0x7f1a6c0c4cc0 2024-01-27T02:33:20.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.744+0000 7f1a61ffb700 1 -- 172.21.15.7:0/3103156925 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a68007440 con 0x7f1a6c0c4cc0 2024-01-27T02:33:20.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.744+0000 7f1a73cc9700 1 -- 172.21.15.7:0/3103156925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6c0ca7d0 con 0x7f1a6c0c4cc0 2024-01-27T02:33:20.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.744+0000 7f1a61ffb700 1 -- 172.21.15.7:0/3103156925 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a680057f0 con 0x7f1a6c0c4cc0 2024-01-27T02:33:20.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.744+0000 7f1a73cc9700 1 -- 172.21.15.7:0/3103156925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6c0cad90 con 0x7f1a6c0c4cc0 2024-01-27T02:33:20.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.745+0000 7f1a73cc9700 1 -- 172.21.15.7:0/3103156925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6c04c710 con 0x7f1a6c0c4cc0 2024-01-27T02:33:20.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.746+0000 7f1a61ffb700 1 -- 172.21.15.7:0/3103156925 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1a68005ba0 con 0x7f1a6c0c4cc0 2024-01-27T02:33:20.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.746+0000 7f1a61ffb700 1 --2- 172.21.15.7:0/3103156925 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a50066f30 0x7f1a500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:20.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.746+0000 7f1a61ffb700 1 -- 172.21.15.7:0/3103156925 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1a68083840 con 0x7f1a6c0c4cc0 2024-01-27T02:33:20.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.747+0000 7f1a71264700 1 --2- 172.21.15.7:0/3103156925 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a50066f30 0x7f1a500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:20.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.747+0000 7f1a71264700 1 --2- 172.21.15.7:0/3103156925 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a50066f30 0x7f1a500693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0b35b0 tx=0x7f1a5c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:20.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.751+0000 7f1a61ffb700 1 -- 172.21.15.7:0/3103156925 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a68050d20 con 0x7f1a6c0c4cc0 2024-01-27T02:33:20.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:20.970+0000 7f1a73cc9700 1 -- 172.21.15.7:0/3103156925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1a6c0cb6c0 con 0x7f1a6c0c4cc0 2024-01-27T02:33:21.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.493+0000 7f1a61ffb700 1 -- 172.21.15.7:0/3103156925 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v39) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1a68017020 con 0x7f1a6c0c4cc0 2024-01-27T02:33:21.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.496+0000 7f1a73cc9700 1 -- 172.21.15.7:0/3103156925 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a50066f30 msgr2=0x7f1a500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:21.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.496+0000 7f1a73cc9700 1 --2- 172.21.15.7:0/3103156925 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a50066f30 0x7f1a500693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0b35b0 tx=0x7f1a5c009040 comp rx=0 tx=0).stop 2024-01-27T02:33:21.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.496+0000 7f1a73cc9700 1 -- 172.21.15.7:0/3103156925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a6c0c4cc0 msgr2=0x7f1a6c0cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:21.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.496+0000 7f1a73cc9700 1 --2- 172.21.15.7:0/3103156925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a6c0c4cc0 0x7f1a6c0cb870 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800c7c0 tx=0x7f1a68007ce0 comp rx=0 tx=0).stop 2024-01-27T02:33:21.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.496+0000 7f1a73cc9700 1 -- 172.21.15.7:0/3103156925 shutdown_connections 2024-01-27T02:33:21.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.496+0000 7f1a73cc9700 1 --2- 172.21.15.7:0/3103156925 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a50066f30 0x7f1a500693e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:21.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.496+0000 7f1a73cc9700 1 --2- 172.21.15.7:0/3103156925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a6c15a230 0x7f1a6c16b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:21.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.496+0000 7f1a73cc9700 1 --2- 172.21.15.7:0/3103156925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a6c0c4cc0 0x7f1a6c0cb870 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:21.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.496+0000 7f1a73cc9700 1 --2- 172.21.15.7:0/3103156925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a6c0ce310 0x7f1a6c0b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:21.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.496+0000 7f1a73cc9700 1 -- 172.21.15.7:0/3103156925 >> 172.21.15.7:0/3103156925 conn(0x7f1a6c098020 msgr2=0x7f1a6c158da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:21.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.496+0000 7f1a73cc9700 1 -- 172.21.15.7:0/3103156925 shutdown_connections 2024-01-27T02:33:21.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.496+0000 7f1a73cc9700 1 -- 172.21.15.7:0/3103156925 wait complete. 2024-01-27T02:33:21.499 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-01-27T02:33:21.514 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-27T02:33:21.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.649+0000 7f1ac9f2a700 1 Processor -- start 2024-01-27T02:33:21.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.649+0000 7f1ac9f2a700 1 -- start start 2024-01-27T02:33:21.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.650+0000 7f1ac9f2a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1ac40637d0 0x7f1ac406ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:21.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.650+0000 7f1ac9f2a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1ac40c41e0 0x7f1ac415a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:21.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.650+0000 7f1ac9f2a700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac415b6c0 con 0x7f1ac415ac80 2024-01-27T02:33:21.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.650+0000 7f1ac9f2a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac415b100 con 0x7f1ac40637d0 2024-01-27T02:33:21.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.650+0000 7f1ac9f2a700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac415b3e0 con 0x7f1ac40c41e0 2024-01-27T02:33:21.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.650+0000 7f1ac37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1ac40637d0 0x7f1ac406ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:21.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.650+0000 7f1ac37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1ac40637d0 0x7f1ac406ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37808/0 (socket says 172.21.15.7:37808) 2024-01-27T02:33:21.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.650+0000 7f1ac37fe700 1 -- 172.21.15.7:0/38380719 learned_addr learned my addr 172.21.15.7:0/38380719 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:21.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.650+0000 7f1ac2ffd700 1 --2- 172.21.15.7:0/38380719 >> v2:172.21.15.188:3300/0 conn(0x7f1ac40c41e0 0x7f1ac415a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:21.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.650+0000 7f1ac37fe700 1 -- 172.21.15.7:0/38380719 >> v2:172.21.15.188:3300/0 conn(0x7f1ac40c41e0 msgr2=0x7f1ac415a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:21.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.650+0000 7f1ac37fe700 1 --2- 172.21.15.7:0/38380719 >> v2:172.21.15.188:3300/0 conn(0x7f1ac40c41e0 0x7f1ac415a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:21.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.650+0000 7f1ac37fe700 1 -- 172.21.15.7:0/38380719 >> v1:172.21.15.188:6789/0 conn(0x7f1ac415ac80 legacy=0x7f1ac40c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:21.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.650+0000 7f1ac37fe700 1 -- 172.21.15.7:0/38380719 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac406e3d0 con 0x7f1ac40637d0 2024-01-27T02:33:21.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.650+0000 7f1ac2ffd700 1 --2- 172.21.15.7:0/38380719 >> v2:172.21.15.188:3300/0 conn(0x7f1ac40c41e0 0x7f1ac415a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:21.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.651+0000 7f1ac37fe700 1 --2- 172.21.15.7:0/38380719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ac40637d0 0x7f1ac406ebd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40cb6b0 tx=0x7f1ab4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76658e3f31275e8c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:21.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.651+0000 7f1ac27fc700 1 -- 172.21.15.7:0/38380719 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab400b860 con 0x7f1ac40637d0 2024-01-27T02:33:21.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.651+0000 7f1ac27fc700 1 -- 172.21.15.7:0/38380719 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab400b9c0 con 0x7f1ac40637d0 2024-01-27T02:33:21.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.651+0000 7f1ac27fc700 1 -- 172.21.15.7:0/38380719 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab400bd60 con 0x7f1ac40637d0 2024-01-27T02:33:21.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.651+0000 7f1ac9f2a700 1 -- 172.21.15.7:0/38380719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ac40637d0 msgr2=0x7f1ac406ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:21.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.651+0000 7f1ac9f2a700 1 --2- 172.21.15.7:0/38380719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ac40637d0 0x7f1ac406ebd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40cb6b0 tx=0x7f1ab4009250 comp rx=0 tx=0).stop 2024-01-27T02:33:21.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.651+0000 7f1ac9f2a700 1 -- 172.21.15.7:0/38380719 shutdown_connections 2024-01-27T02:33:21.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.651+0000 7f1ac9f2a700 1 --2- 172.21.15.7:0/38380719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ac40637d0 0x7f1ac406ebd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:21.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.651+0000 7f1ac9f2a700 1 --2- 172.21.15.7:0/38380719 >> v2:172.21.15.188:3300/0 conn(0x7f1ac40c41e0 0x7f1ac415a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:21.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.651+0000 7f1ac9f2a700 1 -- 172.21.15.7:0/38380719 >> 172.21.15.7:0/38380719 conn(0x7f1ac40cdc30 msgr2=0x7f1ac40ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:21.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.652+0000 7f1ac9f2a700 1 -- 172.21.15.7:0/38380719 shutdown_connections 2024-01-27T02:33:21.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.652+0000 7f1ac9f2a700 1 -- 172.21.15.7:0/38380719 wait complete. 2024-01-27T02:33:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.653+0000 7f1ac9f2a700 1 Processor -- start 2024-01-27T02:33:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.653+0000 7f1ac9f2a700 1 -- start start 2024-01-27T02:33:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.654+0000 7f1ac9f2a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac40637d0 0x7f1ac41587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.654+0000 7f1ac9f2a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ac40c41e0 0x7f1ac40b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.654+0000 7f1ac9f2a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ac40b1e00 0x7f1ac40ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.654+0000 7f1ac9f2a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac416ea10 con 0x7f1ac40b1e00 2024-01-27T02:33:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.654+0000 7f1ac9f2a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac40b0b00 con 0x7f1ac40c41e0 2024-01-27T02:33:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.654+0000 7f1ac9f2a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac40b0de0 con 0x7f1ac40637d0 2024-01-27T02:33:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.655+0000 7f1ac3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ac40b1e00 0x7f1ac40ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.655+0000 7f1ac3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ac40b1e00 0x7f1ac40ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37810/0 (socket says 172.21.15.7:37810) 2024-01-27T02:33:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.655+0000 7f1ac3fff700 1 -- 172.21.15.7:0/1697514351 learned_addr learned my addr 172.21.15.7:0/1697514351 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.655+0000 7f1ac2ffd700 1 --2- 172.21.15.7:0/1697514351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ac40c41e0 0x7f1ac40b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.655+0000 7f1ac37fe700 1 --2- 172.21.15.7:0/1697514351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac40637d0 0x7f1ac41587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.655+0000 7f1ac3fff700 1 -- 172.21.15.7:0/1697514351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac40637d0 msgr2=0x7f1ac41587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.655+0000 7f1ac3fff700 1 --2- 172.21.15.7:0/1697514351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac40637d0 0x7f1ac41587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.655+0000 7f1ac3fff700 1 -- 172.21.15.7:0/1697514351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ac40c41e0 msgr2=0x7f1ac40b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:21.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.655+0000 7f1ac3fff700 1 --2- 172.21.15.7:0/1697514351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ac40c41e0 0x7f1ac40b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:21.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.655+0000 7f1ac3fff700 1 -- 172.21.15.7:0/1697514351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab40094f0 con 0x7f1ac40b1e00 2024-01-27T02:33:21.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.655+0000 7f1ac2ffd700 1 --2- 172.21.15.7:0/1697514351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ac40c41e0 0x7f1ac40b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:21.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.655+0000 7f1ac37fe700 1 --2- 172.21.15.7:0/1697514351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac40637d0 0x7f1ac41587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:21.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.655+0000 7f1ac3fff700 1 --2- 172.21.15.7:0/1697514351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ac40b1e00 0x7f1ac40ccde0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00c840 tx=0x7f1abc001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:21.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.655+0000 7f1abb7fe700 1 -- 172.21.15.7:0/1697514351 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc012630 con 0x7f1ac40b1e00 2024-01-27T02:33:21.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.655+0000 7f1abb7fe700 1 -- 172.21.15.7:0/1697514351 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1abc0127b0 con 0x7f1ac40b1e00 2024-01-27T02:33:21.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.655+0000 7f1ac9f2a700 1 -- 172.21.15.7:0/1697514351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac40b10c0 con 0x7f1ac40b1e00 2024-01-27T02:33:21.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.655+0000 7f1abb7fe700 1 -- 172.21.15.7:0/1697514351 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc012b70 con 0x7f1ac40b1e00 2024-01-27T02:33:21.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.655+0000 7f1ac9f2a700 1 -- 172.21.15.7:0/1697514351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac40cd320 con 0x7f1ac40b1e00 2024-01-27T02:33:21.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.657+0000 7f1ac9f2a700 1 -- 172.21.15.7:0/1697514351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac40b15e0 con 0x7f1ac40b1e00 2024-01-27T02:33:21.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.657+0000 7f1abb7fe700 1 -- 172.21.15.7:0/1697514351 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1abc003030 con 0x7f1ac40b1e00 2024-01-27T02:33:21.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.657+0000 7f1abb7fe700 1 --2- 172.21.15.7:0/1697514351 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1aa4066f30 0x7f1aa40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:21.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.657+0000 7f1abb7fe700 1 -- 172.21.15.7:0/1697514351 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1abc0841d0 con 0x7f1ac40b1e00 2024-01-27T02:33:21.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.658+0000 7f1ac37fe700 1 --2- 172.21.15.7:0/1697514351 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1aa4066f30 0x7f1aa40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:21.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.658+0000 7f1ac37fe700 1 --2- 172.21.15.7:0/1697514351 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1aa4066f30 0x7f1aa40693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40cb6b0 tx=0x7f1ab4017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:21.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.662+0000 7f1abb7fe700 1 -- 172.21.15.7:0/1697514351 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ac40b15e0 con 0x7f1ac40b1e00 2024-01-27T02:33:21.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.946+0000 7f1ac9f2a700 1 -- 172.21.15.7:0/1697514351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1ac40b15e0 con 0x7f1ac40b1e00 2024-01-27T02:33:21.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.947+0000 7f1abb7fe700 1 -- 172.21.15.7:0/1697514351 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f1ac40b15e0 con 0x7f1ac40b1e00 2024-01-27T02:33:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:21.949 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:33:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-01-27T02:33:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-01-27T02:33:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlHdcPKBAAfhnh1I3ZSQEXWJXlXCs98Q==", 2024-01-27T02:33:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:33:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:33:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:33:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:33:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:33:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-01-27T02:33:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rl8eVYNRAAfP8rOqO50RpoNZ1+1j2DXQ==", 2024-01-27T02:33:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:33:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:33:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:33:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:33:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:33:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-01-27T02:33:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlJxRKMBAAS0j7/zZeqGglOxCK1pg0WA==", 2024-01-27T02:33:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:33:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:33:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:33:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:33:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:33:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-01-27T02:33:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlNwfUABAALRQAoHoGhniqCvTQOqbbug==", 2024-01-27T02:33:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:33:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:33:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:33:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:33:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:33:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-01-27T02:33:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7Rlh6VSJhAAttuCte1WhNEvUkv5x/IKQQ==", 2024-01-27T02:33:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:33:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:33:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:33:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:33:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-01-27T02:33:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7Rlu/RGLhAAKzaCNhzYKhsNhxkMvY++gw==", 2024-01-27T02:33:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:33:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:33:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:33:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:33:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-01-27T02:33:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7RliRMyNhAAr0tG3evmVV7ZlGxbxfWPMw==", 2024-01-27T02:33:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:33:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:33:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:33:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:33:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-01-27T02:33:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAPa7RlRuSNAhAA3e9atlb+ffvlnLhRYbWZPw==", 2024-01-27T02:33:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:33:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:33:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:33:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:33:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-01-27T02:33:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7RlfeA6DBAAXpklgbNHRO+5G+fdIERjRw==", 2024-01-27T02:33:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:33:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:33:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:33:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:33:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-01-27T02:33:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7Rl77ItExAABB7uABm63ubE/YUs2B5pGw==", 2024-01-27T02:33:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:33:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:33:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:33:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:33:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-01-27T02:33:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7RljDuwHBAAPPAlhoqS03qPpq58jmevLA==", 2024-01-27T02:33:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:33:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:33:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:33:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:33:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-01-27T02:33:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7Rl1YElJhAAqHP/QEdl1SEalGcbQwPwAg==", 2024-01-27T02:33:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:33:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:33:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:33:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:33:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-01-27T02:33:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlaEqaBRAATFiIeW8lT9BdNuN0m3x9dQ==", 2024-01-27T02:33:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:33:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:33:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-01-27T02:33:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-27T02:33:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-01-27T02:33:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlNADmEBAAJXwdJ35ylP5coy1dtHoD3g==", 2024-01-27T02:33:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:33:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:33:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:33:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:33:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:33:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-01-27T02:33:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7Rl5fRqCRAA/5swTGknnpP64TAh+2X41A==", 2024-01-27T02:33:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:33:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:33:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:33:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:33:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:33:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-01-27T02:33:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7Rl0SdwFhAAhCmw9ViiJbHVfS5a2r/D+w==", 2024-01-27T02:33:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:33:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:33:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:33:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:33:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:33:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-01-27T02:33:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlsNgPARAA0Qm0oJgA8GO1oUT3ndT8Tg==", 2024-01-27T02:33:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:33:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:33:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:33:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:33:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:33:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-01-27T02:33:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rlfb7qIxAAlsJYiSolJ7D8EykGWP62bA==", 2024-01-27T02:33:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:33:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:33:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-27T02:33:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:33:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-01-27T02:33:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rls7TaHxAALS3ghxKtfmgQfIELiD9+Bg==", 2024-01-27T02:33:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:33:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:33:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-27T02:33:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:33:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:21.961 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:33:21.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.949+0000 7f1ac9f2a700 1 -- 172.21.15.7:0/1697514351 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1aa4066f30 msgr2=0x7f1aa40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:21.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.949+0000 7f1ac9f2a700 1 --2- 172.21.15.7:0/1697514351 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1aa4066f30 0x7f1aa40693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40cb6b0 tx=0x7f1ab4017000 comp rx=0 tx=0).stop 2024-01-27T02:33:21.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.949+0000 7f1ac9f2a700 1 -- 172.21.15.7:0/1697514351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ac40b1e00 msgr2=0x7f1ac40ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:21.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.949+0000 7f1ac9f2a700 1 --2- 172.21.15.7:0/1697514351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ac40b1e00 0x7f1ac40ccde0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00c840 tx=0x7f1abc001670 comp rx=0 tx=0).stop 2024-01-27T02:33:21.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.949+0000 7f1ac9f2a700 1 -- 172.21.15.7:0/1697514351 shutdown_connections 2024-01-27T02:33:21.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.949+0000 7f1ac9f2a700 1 --2- 172.21.15.7:0/1697514351 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1aa4066f30 0x7f1aa40693e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:21.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.949+0000 7f1ac9f2a700 1 --2- 172.21.15.7:0/1697514351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ac40b1e00 0x7f1ac40ccde0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:21.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.949+0000 7f1ac9f2a700 1 --2- 172.21.15.7:0/1697514351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac40637d0 0x7f1ac41587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:21.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.949+0000 7f1ac9f2a700 1 --2- 172.21.15.7:0/1697514351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ac40c41e0 0x7f1ac40b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:21.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.949+0000 7f1ac9f2a700 1 -- 172.21.15.7:0/1697514351 >> 172.21.15.7:0/1697514351 conn(0x7f1ac40cdc30 msgr2=0x7f1ac415a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:21.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.949+0000 7f1ac9f2a700 1 -- 172.21.15.7:0/1697514351 shutdown_connections 2024-01-27T02:33:21.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:21.949+0000 7f1ac9f2a700 1 -- 172.21.15.7:0/1697514351 wait complete. 2024-01-27T02:33:21.965 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:33:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.095+0000 7f6ce5851700 1 Processor -- start 2024-01-27T02:33:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.095+0000 7f6ce5851700 1 -- start start 2024-01-27T02:33:22.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.095+0000 7f6ce5851700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6ce0143860 0x7f6ce00d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.095+0000 7f6ce5851700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce00b3d90 con 0x7f6ce0143c30 2024-01-27T02:33:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.095+0000 7f6ce5851700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce00d0ea0 con 0x7f6ce015e510 2024-01-27T02:33:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.095+0000 7f6ce5851700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ce00d1180 con 0x7f6ce0143860 2024-01-27T02:33:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.096+0000 7f6cde7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6ce0143c30 0x7f6ce015de00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44674/0 (socket says 172.21.15.7:44674) 2024-01-27T02:33:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.096+0000 7f6cde7fc700 1 -- 172.21.15.7:0/510911204 learned_addr learned my addr 172.21.15.7:0/510911204 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:22.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.096+0000 7f6cdeffd700 1 --2- 172.21.15.7:0/510911204 >> v2:172.21.15.188:3300/0 conn(0x7f6ce0143860 0x7f6ce00d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:22.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.096+0000 7f6cddffb700 1 -- 172.21.15.7:0/510911204 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2915095270 0 0) 0x7f6ce00b3d90 con 0x7f6ce0143c30 2024-01-27T02:33:22.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.096+0000 7f6cddffb700 1 -- 172.21.15.7:0/510911204 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cc8003570 con 0x7f6ce0143c30 2024-01-27T02:33:22.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.096+0000 7f6cddffb700 1 -- 172.21.15.7:0/510911204 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3844610101 0 0) 0x7f6ce00d0ea0 con 0x7f6ce015e510 2024-01-27T02:33:22.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.096+0000 7f6cddffb700 1 -- 172.21.15.7:0/510911204 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ce00b3d90 con 0x7f6ce015e510 2024-01-27T02:33:22.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.096+0000 7f6cddffb700 1 -- 172.21.15.7:0/510911204 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2575120706 0 0) 0x7f6cc8003570 con 0x7f6ce0143c30 2024-01-27T02:33:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.096+0000 7f6cddffb700 1 -- 172.21.15.7:0/510911204 >> v2:172.21.15.188:3300/0 conn(0x7f6ce0143860 msgr2=0x7f6ce00d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.096+0000 7f6cddffb700 1 --2- 172.21.15.7:0/510911204 >> v2:172.21.15.188:3300/0 conn(0x7f6ce0143860 0x7f6ce00d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.096+0000 7f6cddffb700 1 -- 172.21.15.7:0/510911204 >> v1:172.21.15.188:6789/0 conn(0x7f6ce015e510 legacy=0x7f6ce00b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.096+0000 7f6cddffb700 1 -- 172.21.15.7:0/510911204 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce00d1460 con 0x7f6ce0143c30 2024-01-27T02:33:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.096+0000 7f6cddffb700 1 -- 172.21.15.7:0/510911204 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f6cd4001470 con 0x7f6ce0143c30 2024-01-27T02:33:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.096+0000 7f6cddffb700 1 -- 172.21.15.7:0/510911204 >> v1:172.21.15.7:6789/0 conn(0x7f6ce0143c30 legacy=0x7f6ce015de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.096+0000 7f6cddffb700 1 --2- 172.21.15.7:0/510911204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc8003c40 0x7f6cc8004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.096+0000 7f6cddffb700 1 --2- 172.21.15.7:0/510911204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cc8004810 0x7f6cc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.096+0000 7f6cddffb700 1 --2- 172.21.15.7:0/510911204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cc8005210 0x7f6cc800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.096+0000 7f6cddffb700 1 -- 172.21.15.7:0/510911204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce00d1180 con 0x7f6cc8003c40 2024-01-27T02:33:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.096+0000 7f6cddffb700 1 -- 172.21.15.7:0/510911204 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc800c130 con 0x7f6cc8005210 2024-01-27T02:33:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.096+0000 7f6cddffb700 1 -- 172.21.15.7:0/510911204 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc800c2c0 con 0x7f6cc8004810 2024-01-27T02:33:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.097+0000 7f6cdeffd700 1 --2- 172.21.15.7:0/510911204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc8003c40 0x7f6cc8004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.097+0000 7f6cde7fc700 1 --2- 172.21.15.7:0/510911204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cc8004810 0x7f6cc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.097+0000 7f6cdf7fe700 1 --2- 172.21.15.7:0/510911204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cc8005210 0x7f6cc800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.097+0000 7f6cdeffd700 1 -- 172.21.15.7:0/510911204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cc8004810 msgr2=0x7f6cc8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.097+0000 7f6cdeffd700 1 --2- 172.21.15.7:0/510911204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cc8004810 0x7f6cc8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.097+0000 7f6cdeffd700 1 -- 172.21.15.7:0/510911204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cc8005210 msgr2=0x7f6cc800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.097+0000 7f6cdeffd700 1 --2- 172.21.15.7:0/510911204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cc8005210 0x7f6cc800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.097+0000 7f6cdeffd700 1 -- 172.21.15.7:0/510911204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc8003570 con 0x7f6cc8003c40 2024-01-27T02:33:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.097+0000 7f6cdf7fe700 1 -- 172.21.15.7:0/510911204 reap_dead start 2024-01-27T02:33:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.097+0000 7f6cde7fc700 1 --2- 172.21.15.7:0/510911204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cc8004810 0x7f6cc8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.097+0000 7f6cdeffd700 1 --2- 172.21.15.7:0/510911204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc8003c40 0x7f6cc8004030 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0009c30 tx=0x7f6cd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a1edc4717e1872 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.097+0000 7f6cddffb700 1 -- 172.21.15.7:0/510911204 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd000e710 con 0x7f6cc8003c40 2024-01-27T02:33:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.097+0000 7f6cddffb700 1 -- 172.21.15.7:0/510911204 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd000e870 con 0x7f6cc8003c40 2024-01-27T02:33:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.097+0000 7f6cddffb700 1 -- 172.21.15.7:0/510911204 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd000ec30 con 0x7f6cc8003c40 2024-01-27T02:33:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.098+0000 7f6ce5851700 1 -- 172.21.15.7:0/510911204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc8003c40 msgr2=0x7f6cc8004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.098+0000 7f6ce5851700 1 --2- 172.21.15.7:0/510911204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc8003c40 0x7f6cc8004030 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0009c30 tx=0x7f6cd00091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.098+0000 7f6ce5851700 1 -- 172.21.15.7:0/510911204 shutdown_connections 2024-01-27T02:33:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.098+0000 7f6ce5851700 1 --2- 172.21.15.7:0/510911204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc8003c40 0x7f6cc8004030 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.098+0000 7f6ce5851700 1 -- 172.21.15.7:0/510911204 >> 172.21.15.7:0/510911204 conn(0x7f6ce0068440 msgr2=0x7f6ce00709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.098+0000 7f6ce5851700 1 -- 172.21.15.7:0/510911204 shutdown_connections 2024-01-27T02:33:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.098+0000 7f6ce5851700 1 -- 172.21.15.7:0/510911204 wait complete. 2024-01-27T02:33:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.099+0000 7f6ce5851700 1 Processor -- start 2024-01-27T02:33:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.099+0000 7f6ce5851700 1 -- start start 2024-01-27T02:33:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.100+0000 7f6ce5851700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc8003c40 0x7f6ce0159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.100+0000 7f6ce5851700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ce00c8640 0x7f6ce00cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.100+0000 7f6ce5851700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ce00cb200 0x7f6ce00ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.100+0000 7f6ce5851700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce00cb680 con 0x7f6cc8003c40 2024-01-27T02:33:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.100+0000 7f6ce5851700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce00c5ce0 con 0x7f6ce00cb200 2024-01-27T02:33:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.100+0000 7f6ce5851700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce00c6580 con 0x7f6ce00c8640 2024-01-27T02:33:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.100+0000 7f6cdeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc8003c40 0x7f6ce0159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.100+0000 7f6cdeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc8003c40 0x7f6ce0159af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37826/0 (socket says 172.21.15.7:37826) 2024-01-27T02:33:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.100+0000 7f6cdeffd700 1 -- 172.21.15.7:0/763256778 learned_addr learned my addr 172.21.15.7:0/763256778 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.100+0000 7f6cde7fc700 1 --2- 172.21.15.7:0/763256778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ce00c8640 0x7f6ce00cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.100+0000 7f6cdf7fe700 1 --2- 172.21.15.7:0/763256778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ce00cb200 0x7f6ce00ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.101+0000 7f6cdeffd700 1 -- 172.21.15.7:0/763256778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ce00c8640 msgr2=0x7f6ce00cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.101+0000 7f6cdeffd700 1 --2- 172.21.15.7:0/763256778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ce00c8640 0x7f6ce00cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.101+0000 7f6cdeffd700 1 -- 172.21.15.7:0/763256778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ce00cb200 msgr2=0x7f6ce00ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.101+0000 7f6cdeffd700 1 --2- 172.21.15.7:0/763256778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ce00cb200 0x7f6ce00ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.101+0000 7f6cdeffd700 1 -- 172.21.15.7:0/763256778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce00cdd40 con 0x7f6cc8003c40 2024-01-27T02:33:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.101+0000 7f6cdf7fe700 1 --2- 172.21.15.7:0/763256778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ce00cb200 0x7f6ce00ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.101+0000 7f6cdeffd700 1 --2- 172.21.15.7:0/763256778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc8003c40 0x7f6ce0159af0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0013fa0 tx=0x7f6cd001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.101+0000 7f6cc6ffd700 1 -- 172.21.15.7:0/763256778 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0022070 con 0x7f6cc8003c40 2024-01-27T02:33:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.101+0000 7f6cc6ffd700 1 -- 172.21.15.7:0/763256778 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd000ebf0 con 0x7f6cc8003c40 2024-01-27T02:33:22.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.101+0000 7f6ce5851700 1 -- 172.21.15.7:0/763256778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce00c5f30 con 0x7f6cc8003c40 2024-01-27T02:33:22.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.101+0000 7f6cc6ffd700 1 -- 172.21.15.7:0/763256778 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd001c6a0 con 0x7f6cc8003c40 2024-01-27T02:33:22.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.101+0000 7f6ce5851700 1 -- 172.21.15.7:0/763256778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce015a030 con 0x7f6cc8003c40 2024-01-27T02:33:22.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.102+0000 7f6ce5851700 1 -- 172.21.15.7:0/763256778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce006e1c0 con 0x7f6cc8003c40 2024-01-27T02:33:22.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.102+0000 7f6cc6ffd700 1 -- 172.21.15.7:0/763256778 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6cd001cad0 con 0x7f6cc8003c40 2024-01-27T02:33:22.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.103+0000 7f6cc6ffd700 1 --2- 172.21.15.7:0/763256778 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6ccc066f30 0x7f6ccc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:22.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.103+0000 7f6cc6ffd700 1 -- 172.21.15.7:0/763256778 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6cd0085a70 con 0x7f6cc8003c40 2024-01-27T02:33:22.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.103+0000 7f6cde7fc700 1 --2- 172.21.15.7:0/763256778 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6ccc066f30 0x7f6ccc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:22.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.104+0000 7f6cde7fc700 1 --2- 172.21.15.7:0/763256778 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6ccc066f30 0x7f6ccc0693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6ce016bf50 tx=0x7f6cd4007040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:22.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.107+0000 7f6cc6ffd700 1 -- 172.21.15.7:0/763256778 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6cd0053030 con 0x7f6cc8003c40 2024-01-27T02:33:22.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.308+0000 7f6ce5851700 1 -- 172.21.15.7:0/763256778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ce00c6860 con 0x7f6cc8003c40 2024-01-27T02:33:22.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.309+0000 7f6cc6ffd700 1 -- 172.21.15.7:0/763256778 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+12086 (secure 0 0 0) 0x7f6cd0017090 con 0x7f6cc8003c40 2024-01-27T02:33:22.310 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:22.311 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":41,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:33:21.504325+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:33:22.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.311+0000 7f6ce5851700 1 -- 172.21.15.7:0/763256778 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6ccc066f30 msgr2=0x7f6ccc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:22.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.311+0000 7f6ce5851700 1 --2- 172.21.15.7:0/763256778 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6ccc066f30 0x7f6ccc0693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6ce016bf50 tx=0x7f6cd4007040 comp rx=0 tx=0).stop 2024-01-27T02:33:22.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.311+0000 7f6ce5851700 1 -- 172.21.15.7:0/763256778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc8003c40 msgr2=0x7f6ce0159af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:22.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.311+0000 7f6ce5851700 1 --2- 172.21.15.7:0/763256778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc8003c40 0x7f6ce0159af0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0013fa0 tx=0x7f6cd001a000 comp rx=0 tx=0).stop 2024-01-27T02:33:22.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.311+0000 7f6ce5851700 1 -- 172.21.15.7:0/763256778 shutdown_connections 2024-01-27T02:33:22.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.311+0000 7f6ce5851700 1 --2- 172.21.15.7:0/763256778 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6ccc066f30 0x7f6ccc0693e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.311+0000 7f6ce5851700 1 --2- 172.21.15.7:0/763256778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ce00cb200 0x7f6ce00ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.311+0000 7f6ce5851700 1 --2- 172.21.15.7:0/763256778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc8003c40 0x7f6ce0159af0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.311+0000 7f6ce5851700 1 --2- 172.21.15.7:0/763256778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ce00c8640 0x7f6ce00cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.311+0000 7f6ce5851700 1 -- 172.21.15.7:0/763256778 >> 172.21.15.7:0/763256778 conn(0x7f6ce0068440 msgr2=0x7f6ce00b5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:22.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.311+0000 7f6ce5851700 1 -- 172.21.15.7:0/763256778 shutdown_connections 2024-01-27T02:33:22.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.311+0000 7f6ce5851700 1 -- 172.21.15.7:0/763256778 wait complete. 2024-01-27T02:33:22.327 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-27T02:33:22.327 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:33:22.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.461+0000 7f546cec4700 1 Processor -- start 2024-01-27T02:33:22.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.461+0000 7f546cec4700 1 -- start start 2024-01-27T02:33:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.462+0000 7f546cec4700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f54680cba60 0x7f54680ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.462+0000 7f546cec4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f546806b4e0 0x7f546806b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.462+0000 7f546cec4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54680679b0 con 0x7f54680cbe30 2024-01-27T02:33:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.462+0000 7f546cec4700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5468067c90 con 0x7f54680cba60 2024-01-27T02:33:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.462+0000 7f546cec4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546806be60 con 0x7f546806b4e0 2024-01-27T02:33:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.462+0000 7f5465d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f54680cbe30 0x7f546815b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44690/0 (socket says 172.21.15.7:44690) 2024-01-27T02:33:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.463+0000 7f5465d9b700 1 -- 172.21.15.7:0/845479886 learned_addr learned my addr 172.21.15.7:0/845479886 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.463+0000 7f546659c700 1 --2- 172.21.15.7:0/845479886 >> v2:172.21.15.188:3300/0 conn(0x7f54680cba60 0x7f54680ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.463+0000 7f5466d9d700 1 --2- 172.21.15.7:0/845479886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f546806b4e0 0x7f546806b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.463+0000 7f546559a700 1 -- 172.21.15.7:0/845479886 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 46060163 0 0) 0x7f54680679b0 con 0x7f54680cbe30 2024-01-27T02:33:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.463+0000 7f546559a700 1 -- 172.21.15.7:0/845479886 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5450003570 con 0x7f54680cbe30 2024-01-27T02:33:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.463+0000 7f5466d9d700 1 -- 172.21.15.7:0/845479886 >> v2:172.21.15.188:3300/0 conn(0x7f54680cba60 msgr2=0x7f54680ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.463+0000 7f5466d9d700 1 --2- 172.21.15.7:0/845479886 >> v2:172.21.15.188:3300/0 conn(0x7f54680cba60 0x7f54680ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.463+0000 7f5466d9d700 1 -- 172.21.15.7:0/845479886 >> v1:172.21.15.7:6789/0 conn(0x7f54680cbe30 legacy=0x7f546815b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.463+0000 7f546559a700 1 -- 172.21.15.7:0/845479886 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2065582633 0 0) 0x7f5450003570 con 0x7f54680cbe30 2024-01-27T02:33:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.463+0000 7f5466d9d700 1 -- 172.21.15.7:0/845479886 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54680cca50 con 0x7f546806b4e0 2024-01-27T02:33:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.463+0000 7f5466d9d700 1 --2- 172.21.15.7:0/845479886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f546806b4e0 0x7f546806b920 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5460009e60 tx=0x7f5460009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d6f00170f23d634 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.464+0000 7f546559a700 1 -- 172.21.15.7:0/845479886 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546000e070 con 0x7f546806b4e0 2024-01-27T02:33:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.464+0000 7f546559a700 1 -- 172.21.15.7:0/845479886 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546000b7a0 con 0x7f546806b4e0 2024-01-27T02:33:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.464+0000 7f546559a700 1 -- 172.21.15.7:0/845479886 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c001470 con 0x7f546806b4e0 2024-01-27T02:33:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.464+0000 7f546cec4700 1 -- 172.21.15.7:0/845479886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f546806b4e0 msgr2=0x7f546806b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.464+0000 7f546cec4700 1 --2- 172.21.15.7:0/845479886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f546806b4e0 0x7f546806b920 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5460009e60 tx=0x7f5460009780 comp rx=0 tx=0).stop 2024-01-27T02:33:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.464+0000 7f546cec4700 1 -- 172.21.15.7:0/845479886 shutdown_connections 2024-01-27T02:33:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.464+0000 7f546cec4700 1 --2- 172.21.15.7:0/845479886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f546806b4e0 0x7f546806b920 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.464+0000 7f546cec4700 1 --2- 172.21.15.7:0/845479886 >> v2:172.21.15.188:3300/0 conn(0x7f54680cba60 0x7f54680ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.464+0000 7f546cec4700 1 -- 172.21.15.7:0/845479886 >> 172.21.15.7:0/845479886 conn(0x7f5468063580 msgr2=0x7f5468063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.464+0000 7f546cec4700 1 -- 172.21.15.7:0/845479886 shutdown_connections 2024-01-27T02:33:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.464+0000 7f546cec4700 1 -- 172.21.15.7:0/845479886 wait complete. 2024-01-27T02:33:22.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.465+0000 7f546cec4700 1 Processor -- start 2024-01-27T02:33:22.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.466+0000 7f546cec4700 1 -- start start 2024-01-27T02:33:22.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.466+0000 7f546cec4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f546806b4e0 0x7f54681430c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:22.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.466+0000 7f546cec4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54680cba60 0x7f546815ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:22.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.466+0000 7f546cec4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54680cbe30 0x7f546815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:22.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.466+0000 7f546cec4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54680ad7a0 con 0x7f54680cbe30 2024-01-27T02:33:22.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.466+0000 7f546cec4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54680aaa50 con 0x7f546806b4e0 2024-01-27T02:33:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.466+0000 7f546cec4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54680aad30 con 0x7f54680cba60 2024-01-27T02:33:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.466+0000 7f5466d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54680cbe30 0x7f546815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.467+0000 7f5466d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54680cbe30 0x7f546815e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37836/0 (socket says 172.21.15.7:37836) 2024-01-27T02:33:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.467+0000 7f5466d9d700 1 -- 172.21.15.7:0/818648057 learned_addr learned my addr 172.21.15.7:0/818648057 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.467+0000 7f546659c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f546806b4e0 0x7f54681430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.467+0000 7f5465d9b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54680cba60 0x7f546815ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.467+0000 7f5466d9d700 1 -- 172.21.15.7:0/818648057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54680cba60 msgr2=0x7f546815ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.467+0000 7f5466d9d700 1 --2- 172.21.15.7:0/818648057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54680cba60 0x7f546815ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.467+0000 7f5466d9d700 1 -- 172.21.15.7:0/818648057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f546806b4e0 msgr2=0x7f54681430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.467+0000 7f5466d9d700 1 --2- 172.21.15.7:0/818648057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f546806b4e0 0x7f54681430c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.467+0000 7f5466d9d700 1 -- 172.21.15.7:0/818648057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5460009a20 con 0x7f54680cbe30 2024-01-27T02:33:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.467+0000 7f5466d9d700 1 --2- 172.21.15.7:0/818648057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54680cbe30 0x7f546815e2f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f546000b610 tx=0x7f5460018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.467+0000 7f54567fc700 1 -- 172.21.15.7:0/818648057 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460012b20 con 0x7f54680cbe30 2024-01-27T02:33:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.467+0000 7f54567fc700 1 -- 172.21.15.7:0/818648057 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5460012c80 con 0x7f54680cbe30 2024-01-27T02:33:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.467+0000 7f54567fc700 1 -- 172.21.15.7:0/818648057 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460027470 con 0x7f54680cbe30 2024-01-27T02:33:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.467+0000 7f546cec4700 1 -- 172.21.15.7:0/818648057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54680ab010 con 0x7f54680cbe30 2024-01-27T02:33:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.467+0000 7f546cec4700 1 -- 172.21.15.7:0/818648057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54680ae4b0 con 0x7f54680cbe30 2024-01-27T02:33:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.468+0000 7f546cec4700 1 -- 172.21.15.7:0/818648057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5468072060 con 0x7f54680cbe30 2024-01-27T02:33:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.469+0000 7f54567fc700 1 -- 172.21.15.7:0/818648057 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5460024070 con 0x7f54680cbe30 2024-01-27T02:33:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.469+0000 7f54567fc700 1 --2- 172.21.15.7:0/818648057 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5448066f30 0x7f54480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.469+0000 7f54567fc700 1 -- 172.21.15.7:0/818648057 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5460086670 con 0x7f54680cbe30 2024-01-27T02:33:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.470+0000 7f546659c700 1 --2- 172.21.15.7:0/818648057 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5448066f30 0x7f54480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:22.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.470+0000 7f546659c700 1 --2- 172.21.15.7:0/818648057 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5448066f30 0x7f54480693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5458009da0 tx=0x7f5458009120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:22.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.474+0000 7f54567fc700 1 -- 172.21.15.7:0/818648057 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5460053b90 con 0x7f54680cbe30 2024-01-27T02:33:22.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.679+0000 7f546cec4700 1 -- 172.21.15.7:0/818648057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f546806d5c0 con 0x7f54680cbe30 2024-01-27T02:33:22.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.679+0000 7f54567fc700 1 -- 172.21.15.7:0/818648057 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f546002c020 con 0x7f54680cbe30 2024-01-27T02:33:22.681 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:33:22.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.682+0000 7f546cec4700 1 -- 172.21.15.7:0/818648057 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5448066f30 msgr2=0x7f54480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.682+0000 7f546cec4700 1 --2- 172.21.15.7:0/818648057 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5448066f30 0x7f54480693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5458009da0 tx=0x7f5458009120 comp rx=0 tx=0).stop 2024-01-27T02:33:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.682+0000 7f546cec4700 1 -- 172.21.15.7:0/818648057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54680cbe30 msgr2=0x7f546815e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.682+0000 7f546cec4700 1 --2- 172.21.15.7:0/818648057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54680cbe30 0x7f546815e2f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f546000b610 tx=0x7f5460018000 comp rx=0 tx=0).stop 2024-01-27T02:33:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.682+0000 7f546cec4700 1 -- 172.21.15.7:0/818648057 shutdown_connections 2024-01-27T02:33:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.682+0000 7f546cec4700 1 --2- 172.21.15.7:0/818648057 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5448066f30 0x7f54480693e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.682+0000 7f546cec4700 1 --2- 172.21.15.7:0/818648057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54680cbe30 0x7f546815e2f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.682+0000 7f546cec4700 1 --2- 172.21.15.7:0/818648057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f546806b4e0 0x7f54681430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.682+0000 7f546cec4700 1 --2- 172.21.15.7:0/818648057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54680cba60 0x7f546815ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.682+0000 7f546cec4700 1 -- 172.21.15.7:0/818648057 >> 172.21.15.7:0/818648057 conn(0x7f5468063580 msgr2=0x7f54681579f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.682+0000 7f546cec4700 1 -- 172.21.15.7:0/818648057 shutdown_connections 2024-01-27T02:33:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.682+0000 7f546cec4700 1 -- 172.21.15.7:0/818648057 wait complete. 2024-01-27T02:33:22.697 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-27T02:33:22.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.827+0000 7f72b24d0700 1 Processor -- start 2024-01-27T02:33:22.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.827+0000 7f72b24d0700 1 -- start start 2024-01-27T02:33:22.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.828+0000 7f72b24d0700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f72ac159f40 0x7f72ac0c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:22.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.828+0000 7f72b24d0700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72ac0aafc0 0x7f72ac063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:22.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.828+0000 7f72b24d0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72ac0c30c0 con 0x7f72ac0c3a90 2024-01-27T02:33:22.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.828+0000 7f72b24d0700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72ac0c4550 con 0x7f72ac159f40 2024-01-27T02:33:22.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.828+0000 7f72b24d0700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac15e960 con 0x7f72ac0aafc0 2024-01-27T02:33:22.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.828+0000 7f72ab7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f72ac0c3a90 0x7f72ac0aa8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44698/0 (socket says 172.21.15.7:44698) 2024-01-27T02:33:22.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.828+0000 7f72ab7fe700 1 -- 172.21.15.7:0/3088021636 learned_addr learned my addr 172.21.15.7:0/3088021636 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:22.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.828+0000 7f72abfff700 1 --2- 172.21.15.7:0/3088021636 >> v2:172.21.15.188:3300/0 conn(0x7f72ac159f40 0x7f72ac0c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:22.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.828+0000 7f72b0a6d700 1 --2- 172.21.15.7:0/3088021636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72ac0aafc0 0x7f72ac063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:22.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.828+0000 7f72aaffd700 1 -- 172.21.15.7:0/3088021636 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 293361733 0 0) 0x7f72ac0c30c0 con 0x7f72ac0c3a90 2024-01-27T02:33:22.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.828+0000 7f72aaffd700 1 -- 172.21.15.7:0/3088021636 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7294003570 con 0x7f72ac0c3a90 2024-01-27T02:33:22.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.828+0000 7f72b0a6d700 1 -- 172.21.15.7:0/3088021636 >> v2:172.21.15.188:3300/0 conn(0x7f72ac159f40 msgr2=0x7f72ac0c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:22.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.828+0000 7f72b0a6d700 1 --2- 172.21.15.7:0/3088021636 >> v2:172.21.15.188:3300/0 conn(0x7f72ac159f40 0x7f72ac0c3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.828+0000 7f72b0a6d700 1 -- 172.21.15.7:0/3088021636 >> v1:172.21.15.7:6789/0 conn(0x7f72ac0c3a90 legacy=0x7f72ac0aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:22.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.828+0000 7f72b0a6d700 1 -- 172.21.15.7:0/3088021636 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ac15ec40 con 0x7f72ac0aafc0 2024-01-27T02:33:22.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.828+0000 7f72abfff700 1 --2- 172.21.15.7:0/3088021636 >> v2:172.21.15.188:3300/0 conn(0x7f72ac159f40 0x7f72ac0c3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:33:22.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.829+0000 7f72b0a6d700 1 --2- 172.21.15.7:0/3088021636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72ac0aafc0 0x7f72ac063580 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009e60 tx=0x7f72a4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0a846c40cde793c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:22.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.829+0000 7f72aaffd700 1 -- 172.21.15.7:0/3088021636 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a400e070 con 0x7f72ac0aafc0 2024-01-27T02:33:22.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.829+0000 7f72aaffd700 1 -- 172.21.15.7:0/3088021636 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a400bab0 con 0x7f72ac0aafc0 2024-01-27T02:33:22.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.829+0000 7f72aaffd700 1 -- 172.21.15.7:0/3088021636 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a400be70 con 0x7f72ac0aafc0 2024-01-27T02:33:22.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.829+0000 7f72b24d0700 1 -- 172.21.15.7:0/3088021636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72ac0aafc0 msgr2=0x7f72ac063580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:22.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.829+0000 7f72b24d0700 1 --2- 172.21.15.7:0/3088021636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72ac0aafc0 0x7f72ac063580 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009e60 tx=0x7f72a4009780 comp rx=0 tx=0).stop 2024-01-27T02:33:22.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.829+0000 7f72b24d0700 1 -- 172.21.15.7:0/3088021636 shutdown_connections 2024-01-27T02:33:22.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.829+0000 7f72b24d0700 1 --2- 172.21.15.7:0/3088021636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72ac0aafc0 0x7f72ac063580 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.829+0000 7f72b24d0700 1 --2- 172.21.15.7:0/3088021636 >> v2:172.21.15.188:3300/0 conn(0x7f72ac159f40 0x7f72ac0c3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.829+0000 7f72b24d0700 1 -- 172.21.15.7:0/3088021636 >> 172.21.15.7:0/3088021636 conn(0x7f72ac0cdc30 msgr2=0x7f72ac0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:22.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.829+0000 7f72b24d0700 1 -- 172.21.15.7:0/3088021636 shutdown_connections 2024-01-27T02:33:22.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.830+0000 7f72b24d0700 1 -- 172.21.15.7:0/3088021636 wait complete. 2024-01-27T02:33:22.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.831+0000 7f72b24d0700 1 Processor -- start 2024-01-27T02:33:22.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.831+0000 7f72b24d0700 1 -- start start 2024-01-27T02:33:22.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.832+0000 7f72b24d0700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72ac0aafc0 0x7f72ac158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:22.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.832+0000 7f72b24d0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72ac0c3a90 0x7f72ac0c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:22.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.832+0000 7f72b24d0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72ac159f40 0x7f72ac0c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:22.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.832+0000 7f72b24d0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac169f40 con 0x7f72ac0c3a90 2024-01-27T02:33:22.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.832+0000 7f72b24d0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac16a220 con 0x7f72ac159f40 2024-01-27T02:33:22.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.832+0000 7f72b24d0700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac16a500 con 0x7f72ac0aafc0 2024-01-27T02:33:22.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.832+0000 7f72ab7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72ac0c3a90 0x7f72ac0c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:22.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.832+0000 7f72ab7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72ac0c3a90 0x7f72ac0c7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37848/0 (socket says 172.21.15.7:37848) 2024-01-27T02:33:22.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.832+0000 7f72ab7fe700 1 -- 172.21.15.7:0/365439530 learned_addr learned my addr 172.21.15.7:0/365439530 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:22.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.832+0000 7f72abfff700 1 --2- 172.21.15.7:0/365439530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72ac0aafc0 0x7f72ac158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:22.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.832+0000 7f72b0a6d700 1 --2- 172.21.15.7:0/365439530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72ac159f40 0x7f72ac0c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:22.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.832+0000 7f72ab7fe700 1 -- 172.21.15.7:0/365439530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72ac0aafc0 msgr2=0x7f72ac158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:22.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.832+0000 7f72ab7fe700 1 --2- 172.21.15.7:0/365439530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72ac0aafc0 0x7f72ac158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.832+0000 7f72ab7fe700 1 -- 172.21.15.7:0/365439530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72ac159f40 msgr2=0x7f72ac0c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:22.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.832+0000 7f72ab7fe700 1 --2- 172.21.15.7:0/365439530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72ac159f40 0x7f72ac0c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:22.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.832+0000 7f72ab7fe700 1 -- 172.21.15.7:0/365439530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729c009420 con 0x7f72ac0c3a90 2024-01-27T02:33:22.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.832+0000 7f72ab7fe700 1 --2- 172.21.15.7:0/365439530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72ac0c3a90 0x7f72ac0c7560 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f72a000c7c0 tx=0x7f72a0007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:22.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.833+0000 7f7293fff700 1 -- 172.21.15.7:0/365439530 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a0005c00 con 0x7f72ac0c3a90 2024-01-27T02:33:22.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.833+0000 7f7293fff700 1 -- 172.21.15.7:0/365439530 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a0005d60 con 0x7f72ac0c3a90 2024-01-27T02:33:22.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.833+0000 7f72b24d0700 1 -- 172.21.15.7:0/365439530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a4009a20 con 0x7f72ac0c3a90 2024-01-27T02:33:22.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.833+0000 7f7293fff700 1 -- 172.21.15.7:0/365439530 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a0007320 con 0x7f72ac0c3a90 2024-01-27T02:33:22.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.833+0000 7f72b24d0700 1 -- 172.21.15.7:0/365439530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ac16aac0 con 0x7f72ac0c3a90 2024-01-27T02:33:22.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.834+0000 7f72b24d0700 1 -- 172.21.15.7:0/365439530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72ac16ace0 con 0x7f72ac0c3a90 2024-01-27T02:33:22.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.834+0000 7f7293fff700 1 -- 172.21.15.7:0/365439530 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f72a0007690 con 0x7f72ac0c3a90 2024-01-27T02:33:22.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.834+0000 7f7293fff700 1 --2- 172.21.15.7:0/365439530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7298066f30 0x7f72980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:22.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.835+0000 7f7293fff700 1 -- 172.21.15.7:0/365439530 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f72a0083c80 con 0x7f72ac0c3a90 2024-01-27T02:33:22.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.835+0000 7f72abfff700 1 --2- 172.21.15.7:0/365439530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7298066f30 0x7f72980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:22.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.835+0000 7f72abfff700 1 --2- 172.21.15.7:0/365439530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7298066f30 0x7f72980693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f729c0093f0 tx=0x7f729c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:22.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:22.839+0000 7f7293fff700 1 -- 172.21.15.7:0/365439530 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72a00512c0 con 0x7f72ac0c3a90 2024-01-27T02:33:23.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.046+0000 7f72b24d0700 1 -- 172.21.15.7:0/365439530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f72ac04c710 con 0x7f72ac0c3a90 2024-01-27T02:33:23.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.047+0000 7f7293fff700 1 -- 172.21.15.7:0/365439530 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) v1 ==== 84+0+65 (secure 0 0 0) 0x7f72a00548e0 con 0x7f72ac0c3a90 2024-01-27T02:33:23.048 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:23.048 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-27T02:33:23.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:23.049 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-27T02:33:23.049 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-27T02:33:23.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:23.049 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-27T02:33:23.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.049+0000 7f72b24d0700 1 -- 172.21.15.7:0/365439530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7298066f30 msgr2=0x7f72980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:23.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.049+0000 7f72b24d0700 1 --2- 172.21.15.7:0/365439530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7298066f30 0x7f72980693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f729c0093f0 tx=0x7f729c008040 comp rx=0 tx=0).stop 2024-01-27T02:33:23.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.049+0000 7f72b24d0700 1 -- 172.21.15.7:0/365439530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72ac0c3a90 msgr2=0x7f72ac0c7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:23.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.049+0000 7f72b24d0700 1 --2- 172.21.15.7:0/365439530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72ac0c3a90 0x7f72ac0c7560 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f72a000c7c0 tx=0x7f72a0007dd0 comp rx=0 tx=0).stop 2024-01-27T02:33:23.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.049+0000 7f72b24d0700 1 -- 172.21.15.7:0/365439530 shutdown_connections 2024-01-27T02:33:23.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.049+0000 7f72b24d0700 1 --2- 172.21.15.7:0/365439530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7298066f30 0x7f72980693e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:23.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.049+0000 7f72b24d0700 1 --2- 172.21.15.7:0/365439530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72ac159f40 0x7f72ac0c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:23.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.049+0000 7f72b24d0700 1 --2- 172.21.15.7:0/365439530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72ac0aafc0 0x7f72ac158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:23.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.049+0000 7f72b24d0700 1 --2- 172.21.15.7:0/365439530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72ac0c3a90 0x7f72ac0c7560 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:23.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.049+0000 7f72b24d0700 1 -- 172.21.15.7:0/365439530 >> 172.21.15.7:0/365439530 conn(0x7f72ac0cdc30 msgr2=0x7f72ac0aa8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:23.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.049+0000 7f72b24d0700 1 -- 172.21.15.7:0/365439530 shutdown_connections 2024-01-27T02:33:23.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.049+0000 7f72b24d0700 1 -- 172.21.15.7:0/365439530 wait complete. 2024-01-27T02:33:23.064 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-27T02:33:23.064 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-27T02:33:23.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.198+0000 7f6ff2e77700 1 Processor -- start 2024-01-27T02:33:23.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.198+0000 7f6ff2e77700 1 -- start start 2024-01-27T02:33:23.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.199+0000 7f6ff2e77700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fec0680c0 0x7f6fec164fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:23.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.199+0000 7f6ff2e77700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fec0ce6a0 con 0x7f6fec15a020 2024-01-27T02:33:23.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.199+0000 7f6ff2e77700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fec0ceb30 con 0x7f6fec15e600 2024-01-27T02:33:23.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.199+0000 7f6ff2e77700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec15ec30 con 0x7f6fec0680c0 2024-01-27T02:33:23.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.199+0000 7f6ff0c13700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6fec15a020 0x7f6fec06bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44704/0 (socket says 172.21.15.7:44704) 2024-01-27T02:33:23.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.199+0000 7f6ff0c13700 1 -- 172.21.15.7:0/3205079644 learned_addr learned my addr 172.21.15.7:0/3205079644 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:23.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6ff1414700 1 --2- 172.21.15.7:0/3205079644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fec0680c0 0x7f6fec164fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:23.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6feb7fe700 1 -- 172.21.15.7:0/3205079644 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3529957761 0 0) 0x7f6fec0ce6a0 con 0x7f6fec15a020 2024-01-27T02:33:23.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6feb7fe700 1 -- 172.21.15.7:0/3205079644 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fdc003570 con 0x7f6fec15a020 2024-01-27T02:33:23.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6feb7fe700 1 -- 172.21.15.7:0/3205079644 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2221666283 0 0) 0x7f6fec0ceb30 con 0x7f6fec15e600 2024-01-27T02:33:23.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6feb7fe700 1 -- 172.21.15.7:0/3205079644 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fec0ce6a0 con 0x7f6fec15e600 2024-01-27T02:33:23.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6feb7fe700 1 -- 172.21.15.7:0/3205079644 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1586482878 0 0) 0x7f6fdc003570 con 0x7f6fec15a020 2024-01-27T02:33:23.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6feb7fe700 1 -- 172.21.15.7:0/3205079644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fec0680c0 msgr2=0x7f6fec164fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:23.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6feb7fe700 1 --2- 172.21.15.7:0/3205079644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fec0680c0 0x7f6fec164fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6feb7fe700 1 -- 172.21.15.7:0/3205079644 >> v1:172.21.15.188:6789/0 conn(0x7f6fec15e600 legacy=0x7f6fec0679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6feb7fe700 1 -- 172.21.15.7:0/3205079644 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec069570 con 0x7f6fec15a020 2024-01-27T02:33:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6feb7fe700 1 -- 172.21.15.7:0/3205079644 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f6fe0001470 con 0x7f6fec15a020 2024-01-27T02:33:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6feb7fe700 1 -- 172.21.15.7:0/3205079644 >> v1:172.21.15.7:6789/0 conn(0x7f6fec15a020 legacy=0x7f6fec06bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6feb7fe700 1 --2- 172.21.15.7:0/3205079644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fdc003c20 0x7f6fdc004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6feb7fe700 1 --2- 172.21.15.7:0/3205079644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fec0680c0 0x7f6fdc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6feb7fe700 1 --2- 172.21.15.7:0/3205079644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fdc004d50 0x7f6fdc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6feb7fe700 1 -- 172.21.15.7:0/3205079644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec15ec30 con 0x7f6fdc004d50 2024-01-27T02:33:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6feb7fe700 1 -- 172.21.15.7:0/3205079644 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc00bb40 con 0x7f6fdc003c20 2024-01-27T02:33:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6feb7fe700 1 -- 172.21.15.7:0/3205079644 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc00bd00 con 0x7f6fec0680c0 2024-01-27T02:33:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6ff1414700 1 --2- 172.21.15.7:0/3205079644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fdc004d50 0x7f6fdc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6ff0c13700 1 --2- 172.21.15.7:0/3205079644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fdc003c20 0x7f6fdc004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:23.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6febfff700 1 --2- 172.21.15.7:0/3205079644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fec0680c0 0x7f6fdc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:23.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6ff1414700 1 -- 172.21.15.7:0/3205079644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fec0680c0 msgr2=0x7f6fdc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:23.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6ff1414700 1 --2- 172.21.15.7:0/3205079644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fec0680c0 0x7f6fdc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:23.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6ff1414700 1 -- 172.21.15.7:0/3205079644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fdc003c20 msgr2=0x7f6fdc004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:23.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6ff1414700 1 --2- 172.21.15.7:0/3205079644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fdc003c20 0x7f6fdc004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:23.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6ff1414700 1 -- 172.21.15.7:0/3205079644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fdc003570 con 0x7f6fdc004d50 2024-01-27T02:33:23.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.200+0000 7f6febfff700 1 --2- 172.21.15.7:0/3205079644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fec0680c0 0x7f6fdc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:23.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.201+0000 7f6ff1414700 1 --2- 172.21.15.7:0/3205079644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fdc004d50 0x7f6fdc0051c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0684c0 tx=0x7f6fe400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71f557e77d13b18 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:23.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.201+0000 7f6feb7fe700 1 -- 172.21.15.7:0/3205079644 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe400ea90 con 0x7f6fdc004d50 2024-01-27T02:33:23.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.201+0000 7f6feb7fe700 1 -- 172.21.15.7:0/3205079644 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe400f040 con 0x7f6fdc004d50 2024-01-27T02:33:23.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.201+0000 7f6feb7fe700 1 -- 172.21.15.7:0/3205079644 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe400ee40 con 0x7f6fdc004d50 2024-01-27T02:33:23.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.201+0000 7f6ff2e77700 1 -- 172.21.15.7:0/3205079644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fdc004d50 msgr2=0x7f6fdc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:23.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.201+0000 7f6ff2e77700 1 --2- 172.21.15.7:0/3205079644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fdc004d50 0x7f6fdc0051c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0684c0 tx=0x7f6fe400b040 comp rx=0 tx=0).stop 2024-01-27T02:33:23.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.201+0000 7f6ff1414700 1 -- 172.21.15.7:0/3205079644 reap_dead start 2024-01-27T02:33:23.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.201+0000 7f6ff2e77700 1 -- 172.21.15.7:0/3205079644 shutdown_connections 2024-01-27T02:33:23.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.201+0000 7f6ff2e77700 1 -- 172.21.15.7:0/3205079644 >> 172.21.15.7:0/3205079644 conn(0x7f6fec0714b0 msgr2=0x7f6fec06b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:23.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.202+0000 7f6ff2e77700 1 -- 172.21.15.7:0/3205079644 shutdown_connections 2024-01-27T02:33:23.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.202+0000 7f6ff2e77700 1 -- 172.21.15.7:0/3205079644 wait complete. 2024-01-27T02:33:23.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.203+0000 7f6ff2e77700 1 Processor -- start 2024-01-27T02:33:23.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.203+0000 7f6ff2e77700 1 -- start start 2024-01-27T02:33:23.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.204+0000 7f6ff2e77700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fec14dd90 0x7f6fec150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:23.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.204+0000 7f6ff2e77700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fec151910 0x7f6fec0c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:23.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.204+0000 7f6ff2e77700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec0cb4d0 0x7f6fec0c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:23.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.204+0000 7f6ff2e77700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec14e160 con 0x7f6fec0cb4d0 2024-01-27T02:33:23.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.204+0000 7f6ff2e77700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec151480 con 0x7f6fec151910 2024-01-27T02:33:23.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.204+0000 7f6ff2e77700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec1505a0 con 0x7f6fec14dd90 2024-01-27T02:33:23.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.204+0000 7f6ff1414700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec0cb4d0 0x7f6fec0c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:23.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.204+0000 7f6ff1414700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec0cb4d0 0x7f6fec0c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37864/0 (socket says 172.21.15.7:37864) 2024-01-27T02:33:23.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.204+0000 7f6ff1414700 1 -- 172.21.15.7:0/4290883930 learned_addr learned my addr 172.21.15.7:0/4290883930 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:23.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.204+0000 7f6ff0c13700 1 --2- 172.21.15.7:0/4290883930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fec14dd90 0x7f6fec150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:23.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.204+0000 7f6febfff700 1 --2- 172.21.15.7:0/4290883930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fec151910 0x7f6fec0c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:23.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.204+0000 7f6ff1414700 1 -- 172.21.15.7:0/4290883930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fec14dd90 msgr2=0x7f6fec150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:23.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.204+0000 7f6ff1414700 1 --2- 172.21.15.7:0/4290883930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fec14dd90 0x7f6fec150060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:23.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.204+0000 7f6ff1414700 1 -- 172.21.15.7:0/4290883930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fec151910 msgr2=0x7f6fec0c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:23.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.204+0000 7f6ff1414700 1 --2- 172.21.15.7:0/4290883930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fec151910 0x7f6fec0c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:23.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.204+0000 7f6ff1414700 1 -- 172.21.15.7:0/4290883930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec150870 con 0x7f6fec0cb4d0 2024-01-27T02:33:23.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.204+0000 7f6ff1414700 1 --2- 172.21.15.7:0/4290883930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec0cb4d0 0x7f6fec0c86b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400afd0 tx=0x7f6fe401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.205+0000 7f6fe8ff9700 1 -- 172.21.15.7:0/4290883930 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe400ee90 con 0x7f6fec0cb4d0 2024-01-27T02:33:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.205+0000 7f6fe8ff9700 1 -- 172.21.15.7:0/4290883930 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe401d630 con 0x7f6fec0cb4d0 2024-01-27T02:33:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.205+0000 7f6ff2e77700 1 -- 172.21.15.7:0/4290883930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fec150b50 con 0x7f6fec0cb4d0 2024-01-27T02:33:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.205+0000 7f6fe8ff9700 1 -- 172.21.15.7:0/4290883930 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe401da30 con 0x7f6fec0cb4d0 2024-01-27T02:33:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.205+0000 7f6ff2e77700 1 -- 172.21.15.7:0/4290883930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fec151110 con 0x7f6fec0cb4d0 2024-01-27T02:33:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.206+0000 7f6ff2e77700 1 -- 172.21.15.7:0/4290883930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fec04c710 con 0x7f6fec0cb4d0 2024-01-27T02:33:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.206+0000 7f6fe8ff9700 1 -- 172.21.15.7:0/4290883930 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6fe4003040 con 0x7f6fec0cb4d0 2024-01-27T02:33:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.207+0000 7f6fe8ff9700 1 --2- 172.21.15.7:0/4290883930 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6fd0066f30 0x7f6fd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.207+0000 7f6fe8ff9700 1 -- 172.21.15.7:0/4290883930 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6fe4085e10 con 0x7f6fec0cb4d0 2024-01-27T02:33:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.207+0000 7f6ff0c13700 1 --2- 172.21.15.7:0/4290883930 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6fd0066f30 0x7f6fd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.207+0000 7f6ff0c13700 1 --2- 172.21.15.7:0/4290883930 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6fd0066f30 0x7f6fd00693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0000e50 tx=0x7f6fe0001e10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:23.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.211+0000 7f6fe8ff9700 1 -- 172.21.15.7:0/4290883930 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6fe4089030 con 0x7f6fec0cb4d0 2024-01-27T02:33:23.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.412+0000 7f6ff2e77700 1 -- 172.21.15.7:0/4290883930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6fec06c270 con 0x7f6fec0cb4d0 2024-01-27T02:33:23.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.511+0000 7f6fe8ff9700 1 -- 172.21.15.7:0/4290883930 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6fe4053330 con 0x7f6fec0cb4d0 2024-01-27T02:33:23.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.514+0000 7f6ff2e77700 1 -- 172.21.15.7:0/4290883930 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6fd0066f30 msgr2=0x7f6fd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:23.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.514+0000 7f6ff2e77700 1 --2- 172.21.15.7:0/4290883930 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6fd0066f30 0x7f6fd00693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0000e50 tx=0x7f6fe0001e10 comp rx=0 tx=0).stop 2024-01-27T02:33:23.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.514+0000 7f6ff2e77700 1 -- 172.21.15.7:0/4290883930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec0cb4d0 msgr2=0x7f6fec0c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:23.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.514+0000 7f6ff2e77700 1 --2- 172.21.15.7:0/4290883930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec0cb4d0 0x7f6fec0c86b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400afd0 tx=0x7f6fe401a000 comp rx=0 tx=0).stop 2024-01-27T02:33:23.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.514+0000 7f6ff2e77700 1 -- 172.21.15.7:0/4290883930 shutdown_connections 2024-01-27T02:33:23.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.514+0000 7f6ff2e77700 1 --2- 172.21.15.7:0/4290883930 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6fd0066f30 0x7f6fd00693e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:23.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.514+0000 7f6ff2e77700 1 --2- 172.21.15.7:0/4290883930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec0cb4d0 0x7f6fec0c86b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:23.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.514+0000 7f6ff2e77700 1 --2- 172.21.15.7:0/4290883930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fec14dd90 0x7f6fec150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:23.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.514+0000 7f6ff2e77700 1 --2- 172.21.15.7:0/4290883930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fec151910 0x7f6fec0c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:23.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.514+0000 7f6ff2e77700 1 -- 172.21.15.7:0/4290883930 >> 172.21.15.7:0/4290883930 conn(0x7f6fec0714b0 msgr2=0x7f6fec06ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:23.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.515+0000 7f6ff2e77700 1 -- 172.21.15.7:0/4290883930 shutdown_connections 2024-01-27T02:33:23.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.515+0000 7f6ff2e77700 1 -- 172.21.15.7:0/4290883930 wait complete. 2024-01-27T02:33:23.518 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-27T02:33:23.531 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-27T02:33:23.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.670+0000 7fe8ba930700 1 Processor -- start 2024-01-27T02:33:23.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.670+0000 7fe8ba930700 1 -- start start 2024-01-27T02:33:23.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.671+0000 7fe8ba930700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8b41416c0 0x7fe8b40cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:23.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.671+0000 7fe8ba930700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe8b4141a90 0x7fe8b40cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:23.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.671+0000 7fe8ba930700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b415c7d0 con 0x7fe8b40cab00 2024-01-27T02:33:23.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.671+0000 7fe8ba930700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b40cd9c0 con 0x7fe8b4141a90 2024-01-27T02:33:23.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.671+0000 7fe8ba930700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b40c9640 con 0x7fe8b41416c0 2024-01-27T02:33:23.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.671+0000 7fe8b3fff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8b41416c0 0x7fe8b40cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:23.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.671+0000 7fe8b37fe700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe8b4141a90 0x7fe8b40cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:23.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.671+0000 7fe8b8ecd700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fe8b40cab00 0x7fe8b40caf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:50270/0 (socket says 172.21.15.7:50270) 2024-01-27T02:33:23.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.671+0000 7fe8b8ecd700 1 -- 172.21.15.7:0/3312103917 learned_addr learned my addr 172.21.15.7:0/3312103917 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:23.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.672+0000 7fe8b3fff700 1 -- 172.21.15.7:0/3312103917 >> v2:172.21.15.188:3300/0 conn(0x7fe8b4141a90 msgr2=0x7fe8b40cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:23.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.672+0000 7fe8b3fff700 1 --2- 172.21.15.7:0/3312103917 >> v2:172.21.15.188:3300/0 conn(0x7fe8b4141a90 0x7fe8b40cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:23.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.672+0000 7fe8b3fff700 1 -- 172.21.15.7:0/3312103917 >> v1:172.21.15.188:6789/0 conn(0x7fe8b40cab00 legacy=0x7fe8b40caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:23.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.672+0000 7fe8b3fff700 1 -- 172.21.15.7:0/3312103917 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b415bbc0 con 0x7fe8b41416c0 2024-01-27T02:33:23.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.672+0000 7fe8b3fff700 1 --2- 172.21.15.7:0/3312103917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8b41416c0 0x7fe8b40cb6e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40098d0 tx=0x7fe8a4009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f862b90fd1407f8b server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:23.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.672+0000 7fe8b2ffd700 1 -- 172.21.15.7:0/3312103917 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4005c10 con 0x7fe8b41416c0 2024-01-27T02:33:23.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.672+0000 7fe8b2ffd700 1 -- 172.21.15.7:0/3312103917 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a4005d70 con 0x7fe8b41416c0 2024-01-27T02:33:23.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.672+0000 7fe8b2ffd700 1 -- 172.21.15.7:0/3312103917 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4006280 con 0x7fe8b41416c0 2024-01-27T02:33:23.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.673+0000 7fe8ba930700 1 -- 172.21.15.7:0/3312103917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8b41416c0 msgr2=0x7fe8b40cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:23.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.673+0000 7fe8ba930700 1 --2- 172.21.15.7:0/3312103917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8b41416c0 0x7fe8b40cb6e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40098d0 tx=0x7fe8a4009120 comp rx=0 tx=0).stop 2024-01-27T02:33:23.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.673+0000 7fe8ba930700 1 -- 172.21.15.7:0/3312103917 shutdown_connections 2024-01-27T02:33:23.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.673+0000 7fe8ba930700 1 --2- 172.21.15.7:0/3312103917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8b41416c0 0x7fe8b40cb6e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:23.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.673+0000 7fe8ba930700 1 --2- 172.21.15.7:0/3312103917 >> v2:172.21.15.188:3300/0 conn(0x7fe8b4141a90 0x7fe8b40cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:23.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.673+0000 7fe8ba930700 1 -- 172.21.15.7:0/3312103917 >> 172.21.15.7:0/3312103917 conn(0x7fe8b4063580 msgr2=0x7fe8b4063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:23.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.673+0000 7fe8ba930700 1 -- 172.21.15.7:0/3312103917 shutdown_connections 2024-01-27T02:33:23.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.673+0000 7fe8ba930700 1 -- 172.21.15.7:0/3312103917 wait complete. 2024-01-27T02:33:23.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.674+0000 7fe8ba930700 1 Processor -- start 2024-01-27T02:33:23.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.674+0000 7fe8ba930700 1 -- start start 2024-01-27T02:33:23.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.675+0000 7fe8ba930700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b40cab00 0x7fe8b4157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:23.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.675+0000 7fe8ba930700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8b41416c0 0x7fe8b40caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:23.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.675+0000 7fe8ba930700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8b4141a90 0x7fe8b40cb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:23.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.675+0000 7fe8ba930700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b4158470 con 0x7fe8b40cab00 2024-01-27T02:33:23.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.675+0000 7fe8ba930700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b4158750 con 0x7fe8b41416c0 2024-01-27T02:33:23.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.675+0000 7fe8ba930700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b4158a30 con 0x7fe8b4141a90 2024-01-27T02:33:23.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.675+0000 7fe8b3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b40cab00 0x7fe8b4157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:23.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.675+0000 7fe8b3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b40cab00 0x7fe8b4157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37878/0 (socket says 172.21.15.7:37878) 2024-01-27T02:33:23.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.675+0000 7fe8b3fff700 1 -- 172.21.15.7:0/2177842104 learned_addr learned my addr 172.21.15.7:0/2177842104 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:23.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.675+0000 7fe8b37fe700 1 --2- 172.21.15.7:0/2177842104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8b41416c0 0x7fe8b40caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:23.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.675+0000 7fe8b8ecd700 1 --2- 172.21.15.7:0/2177842104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8b4141a90 0x7fe8b40cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:23.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.675+0000 7fe8b3fff700 1 -- 172.21.15.7:0/2177842104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8b4141a90 msgr2=0x7fe8b40cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:23.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.675+0000 7fe8b3fff700 1 --2- 172.21.15.7:0/2177842104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8b4141a90 0x7fe8b40cb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:23.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.675+0000 7fe8b3fff700 1 -- 172.21.15.7:0/2177842104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8b41416c0 msgr2=0x7fe8b40caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:23.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.675+0000 7fe8b3fff700 1 --2- 172.21.15.7:0/2177842104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8b41416c0 0x7fe8b40caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:23.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.675+0000 7fe8b3fff700 1 -- 172.21.15.7:0/2177842104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a40093c0 con 0x7fe8b40cab00 2024-01-27T02:33:23.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.676+0000 7fe8b3fff700 1 --2- 172.21.15.7:0/2177842104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b40cab00 0x7fe8b4157990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe8b40c9780 tx=0x7fe8a4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:23.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.676+0000 7fe8abfff700 1 -- 172.21.15.7:0/2177842104 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a401a630 con 0x7fe8b40cab00 2024-01-27T02:33:23.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.676+0000 7fe8abfff700 1 -- 172.21.15.7:0/2177842104 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a401a7b0 con 0x7fe8b40cab00 2024-01-27T02:33:23.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.676+0000 7fe8ba930700 1 -- 172.21.15.7:0/2177842104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b4158d10 con 0x7fe8b40cab00 2024-01-27T02:33:23.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.676+0000 7fe8abfff700 1 -- 172.21.15.7:0/2177842104 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a401ab70 con 0x7fe8b40cab00 2024-01-27T02:33:23.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.676+0000 7fe8ba930700 1 -- 172.21.15.7:0/2177842104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b40c7890 con 0x7fe8b40cab00 2024-01-27T02:33:23.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.677+0000 7fe8ba930700 1 -- 172.21.15.7:0/2177842104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b4072060 con 0x7fe8b40cab00 2024-01-27T02:33:23.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.678+0000 7fe8abfff700 1 -- 172.21.15.7:0/2177842104 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe8a4011420 con 0x7fe8b40cab00 2024-01-27T02:33:23.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.678+0000 7fe8abfff700 1 --2- 172.21.15.7:0/2177842104 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe894066f30 0x7fe8940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:23.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.678+0000 7fe8abfff700 1 -- 172.21.15.7:0/2177842104 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fe8a4085880 con 0x7fe8b40cab00 2024-01-27T02:33:23.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.679+0000 7fe8b37fe700 1 --2- 172.21.15.7:0/2177842104 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe894066f30 0x7fe8940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:23.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.679+0000 7fe8b37fe700 1 --2- 172.21.15.7:0/2177842104 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe894066f30 0x7fe8940693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe8b40c85a0 tx=0x7fe89c0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:23.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.683+0000 7fe8abfff700 1 -- 172.21.15.7:0/2177842104 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8a4052cd0 con 0x7fe8b40cab00 2024-01-27T02:33:23.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:23.889+0000 7fe8ba930700 1 -- 172.21.15.7:0/2177842104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe8b415bea0 con 0x7fe8b40cab00 2024-01-27T02:33:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.519+0000 7fe8abfff700 1 -- 172.21.15.7:0/2177842104 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe8a402b0d0 con 0x7fe8b40cab00 2024-01-27T02:33:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.528+0000 7fe8ba930700 1 -- 172.21.15.7:0/2177842104 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe894066f30 msgr2=0x7fe8940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.528+0000 7fe8ba930700 1 --2- 172.21.15.7:0/2177842104 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe894066f30 0x7fe8940693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe8b40c85a0 tx=0x7fe89c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.528+0000 7fe8ba930700 1 -- 172.21.15.7:0/2177842104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b40cab00 msgr2=0x7fe8b4157990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.528+0000 7fe8ba930700 1 --2- 172.21.15.7:0/2177842104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b40cab00 0x7fe8b4157990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe8b40c9780 tx=0x7fe8a4016000 comp rx=0 tx=0).stop 2024-01-27T02:33:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.529+0000 7fe8ba930700 1 -- 172.21.15.7:0/2177842104 shutdown_connections 2024-01-27T02:33:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.529+0000 7fe8ba930700 1 --2- 172.21.15.7:0/2177842104 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe894066f30 0x7fe8940693e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.529+0000 7fe8ba930700 1 --2- 172.21.15.7:0/2177842104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8b4141a90 0x7fe8b40cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.529+0000 7fe8ba930700 1 --2- 172.21.15.7:0/2177842104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b40cab00 0x7fe8b4157990 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.529+0000 7fe8ba930700 1 --2- 172.21.15.7:0/2177842104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8b41416c0 0x7fe8b40caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.529+0000 7fe8ba930700 1 -- 172.21.15.7:0/2177842104 >> 172.21.15.7:0/2177842104 conn(0x7fe8b4063580 msgr2=0x7fe8b40c4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.530+0000 7fe8ba930700 1 -- 172.21.15.7:0/2177842104 shutdown_connections 2024-01-27T02:33:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.530+0000 7fe8ba930700 1 -- 172.21.15.7:0/2177842104 wait complete. 2024-01-27T02:33:24.533 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-27T02:33:24.548 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-27T02:33:24.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.685+0000 7fe1c306e700 1 Processor -- start 2024-01-27T02:33:24.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.685+0000 7fe1c306e700 1 -- start start 2024-01-27T02:33:24.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.685+0000 7fe1c306e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe1bc141a90 0x7fe1bc0cab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:24.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.686+0000 7fe1c306e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1bc15c7d0 con 0x7fe1bc1416c0 2024-01-27T02:33:24.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.686+0000 7fe1c306e700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1bc0c9640 con 0x7fe1bc0cbdf0 2024-01-27T02:33:24.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.686+0000 7fe1c306e700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1bc15bbc0 con 0x7fe1bc141a90 2024-01-27T02:33:24.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.686+0000 7fe1c0e0a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe1bc1416c0 0x7fe1bc0cb6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44710/0 (socket says 172.21.15.7:44710) 2024-01-27T02:33:24.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.686+0000 7fe1c0e0a700 1 -- 172.21.15.7:0/803473163 learned_addr learned my addr 172.21.15.7:0/803473163 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:24.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.686+0000 7fe1bbfff700 1 --2- 172.21.15.7:0/803473163 >> v2:172.21.15.188:3300/0 conn(0x7fe1bc141a90 0x7fe1bc0cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:24.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.686+0000 7fe1bb7fe700 1 -- 172.21.15.7:0/803473163 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093010420 0 0) 0x7fe1bc15c7d0 con 0x7fe1bc1416c0 2024-01-27T02:33:24.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.686+0000 7fe1bb7fe700 1 -- 172.21.15.7:0/803473163 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1ac003570 con 0x7fe1bc1416c0 2024-01-27T02:33:24.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.686+0000 7fe1bb7fe700 1 -- 172.21.15.7:0/803473163 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1580596227 0 0) 0x7fe1ac003570 con 0x7fe1bc1416c0 2024-01-27T02:33:24.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.686+0000 7fe1bb7fe700 1 -- 172.21.15.7:0/803473163 >> v2:172.21.15.188:3300/0 conn(0x7fe1bc141a90 msgr2=0x7fe1bc0cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:24.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.686+0000 7fe1bb7fe700 1 --2- 172.21.15.7:0/803473163 >> v2:172.21.15.188:3300/0 conn(0x7fe1bc141a90 0x7fe1bc0cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:24.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.686+0000 7fe1bb7fe700 1 -- 172.21.15.7:0/803473163 >> v1:172.21.15.188:6789/0 conn(0x7fe1bc0cbdf0 legacy=0x7fe1bc0cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:24.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.686+0000 7fe1bb7fe700 1 -- 172.21.15.7:0/803473163 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc15bea0 con 0x7fe1bc1416c0 2024-01-27T02:33:24.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.686+0000 7fe1bb7fe700 1 -- 172.21.15.7:0/803473163 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7fe1b00015f0 con 0x7fe1bc1416c0 2024-01-27T02:33:24.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.686+0000 7fe1bb7fe700 1 -- 172.21.15.7:0/803473163 >> v1:172.21.15.7:6789/0 conn(0x7fe1bc1416c0 legacy=0x7fe1bc0cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:24.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.686+0000 7fe1bb7fe700 1 --2- 172.21.15.7:0/803473163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1ac003a60 0x7fe1ac003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.687+0000 7fe1bb7fe700 1 --2- 172.21.15.7:0/803473163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1ac004810 0x7fe1ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.687+0000 7fe1bb7fe700 1 --2- 172.21.15.7:0/803473163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1ac0051c0 0x7fe1ac00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.687+0000 7fe1bb7fe700 1 -- 172.21.15.7:0/803473163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc15bbc0 con 0x7fe1ac003a60 2024-01-27T02:33:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.687+0000 7fe1bb7fe700 1 -- 172.21.15.7:0/803473163 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac00c160 con 0x7fe1ac0051c0 2024-01-27T02:33:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.687+0000 7fe1bb7fe700 1 -- 172.21.15.7:0/803473163 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac00c320 con 0x7fe1ac004810 2024-01-27T02:33:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.687+0000 7fe1bbfff700 1 --2- 172.21.15.7:0/803473163 >> v2:172.21.15.188:3300/0 conn(0x7fe1bc141a90 0x7fe1bc0cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:33:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.687+0000 7fe1c0e0a700 1 --2- 172.21.15.7:0/803473163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1ac003a60 0x7fe1ac003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.687+0000 7fe1c160b700 1 --2- 172.21.15.7:0/803473163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1ac0051c0 0x7fe1ac00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.687+0000 7fe1bbfff700 1 --2- 172.21.15.7:0/803473163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1ac004810 0x7fe1ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.687+0000 7fe1c0e0a700 1 -- 172.21.15.7:0/803473163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1ac004810 msgr2=0x7fe1ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.687+0000 7fe1c0e0a700 1 --2- 172.21.15.7:0/803473163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1ac004810 0x7fe1ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.687+0000 7fe1c0e0a700 1 -- 172.21.15.7:0/803473163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1ac0051c0 msgr2=0x7fe1ac00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.687+0000 7fe1c0e0a700 1 --2- 172.21.15.7:0/803473163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1ac0051c0 0x7fe1ac00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.687+0000 7fe1c0e0a700 1 -- 172.21.15.7:0/803473163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ac003570 con 0x7fe1ac003a60 2024-01-27T02:33:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.687+0000 7fe1c160b700 1 -- 172.21.15.7:0/803473163 reap_dead start 2024-01-27T02:33:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.687+0000 7fe1bbfff700 1 --2- 172.21.15.7:0/803473163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1ac004810 0x7fe1ac004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.687+0000 7fe1c0e0a700 1 --2- 172.21.15.7:0/803473163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1ac003a60 0x7fe1ac003e70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000c3c0 tx=0x7fe1b0007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c7e807ed85d357 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.687+0000 7fe1bb7fe700 1 -- 172.21.15.7:0/803473163 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b000cc90 con 0x7fe1ac003a60 2024-01-27T02:33:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.687+0000 7fe1bb7fe700 1 -- 172.21.15.7:0/803473163 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b000cdf0 con 0x7fe1ac003a60 2024-01-27T02:33:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.687+0000 7fe1bb7fe700 1 -- 172.21.15.7:0/803473163 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0007320 con 0x7fe1ac003a60 2024-01-27T02:33:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.688+0000 7fe1c306e700 1 -- 172.21.15.7:0/803473163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1ac003a60 msgr2=0x7fe1ac003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.688+0000 7fe1c306e700 1 --2- 172.21.15.7:0/803473163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1ac003a60 0x7fe1ac003e70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000c3c0 tx=0x7fe1b0007920 comp rx=0 tx=0).stop 2024-01-27T02:33:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.688+0000 7fe1c306e700 1 -- 172.21.15.7:0/803473163 shutdown_connections 2024-01-27T02:33:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.688+0000 7fe1c306e700 1 --2- 172.21.15.7:0/803473163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1ac003a60 0x7fe1ac003e70 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.688+0000 7fe1c306e700 1 -- 172.21.15.7:0/803473163 >> 172.21.15.7:0/803473163 conn(0x7fe1bc063580 msgr2=0x7fe1bc063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.688+0000 7fe1c306e700 1 -- 172.21.15.7:0/803473163 shutdown_connections 2024-01-27T02:33:24.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.688+0000 7fe1c306e700 1 -- 172.21.15.7:0/803473163 wait complete. 2024-01-27T02:33:24.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.689+0000 7fe1c306e700 1 Processor -- start 2024-01-27T02:33:24.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.689+0000 7fe1c306e700 1 -- start start 2024-01-27T02:33:24.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.690+0000 7fe1c306e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1ac003a60 0x7fe1bc139b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:24.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.690+0000 7fe1c306e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc14c7b0 0x7fe1bc13c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:24.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.690+0000 7fe1c306e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1bc0c6bf0 0x7fe1bc0c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:24.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.690+0000 7fe1c306e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc157d20 con 0x7fe1bc14c7b0 2024-01-27T02:33:24.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.690+0000 7fe1c306e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc0c6760 con 0x7fe1ac003a60 2024-01-27T02:33:24.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.690+0000 7fe1c306e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc14cc20 con 0x7fe1bc0c6bf0 2024-01-27T02:33:24.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.690+0000 7fe1bbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc14c7b0 0x7fe1bc13c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:24.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.690+0000 7fe1bbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc14c7b0 0x7fe1bc13c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37884/0 (socket says 172.21.15.7:37884) 2024-01-27T02:33:24.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.690+0000 7fe1bbfff700 1 -- 172.21.15.7:0/817779528 learned_addr learned my addr 172.21.15.7:0/817779528 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:24.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.690+0000 7fe1c0e0a700 1 --2- 172.21.15.7:0/817779528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1ac003a60 0x7fe1bc139b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:24.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.690+0000 7fe1c160b700 1 --2- 172.21.15.7:0/817779528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1bc0c6bf0 0x7fe1bc0c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:24.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.691+0000 7fe1bbfff700 1 -- 172.21.15.7:0/817779528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1bc0c6bf0 msgr2=0x7fe1bc0c7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:24.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.691+0000 7fe1bbfff700 1 --2- 172.21.15.7:0/817779528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1bc0c6bf0 0x7fe1bc0c7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:24.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.691+0000 7fe1bbfff700 1 -- 172.21.15.7:0/817779528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1ac003a60 msgr2=0x7fe1bc139b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:24.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.691+0000 7fe1bbfff700 1 --2- 172.21.15.7:0/817779528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1ac003a60 0x7fe1bc139b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:24.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.691+0000 7fe1bbfff700 1 -- 172.21.15.7:0/817779528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc0c5e30 con 0x7fe1bc14c7b0 2024-01-27T02:33:24.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.691+0000 7fe1bbfff700 1 --2- 172.21.15.7:0/817779528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc14c7b0 0x7fe1bc13c1d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8011890 tx=0x7fe1a8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:24.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.691+0000 7fe1b8ff9700 1 -- 172.21.15.7:0/817779528 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8015670 con 0x7fe1bc14c7b0 2024-01-27T02:33:24.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.691+0000 7fe1b8ff9700 1 -- 172.21.15.7:0/817779528 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a80157d0 con 0x7fe1bc14c7b0 2024-01-27T02:33:24.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.691+0000 7fe1c306e700 1 -- 172.21.15.7:0/817779528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1bc0c6110 con 0x7fe1bc14c7b0 2024-01-27T02:33:24.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.691+0000 7fe1b8ff9700 1 -- 172.21.15.7:0/817779528 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8015b90 con 0x7fe1bc14c7b0 2024-01-27T02:33:24.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.691+0000 7fe1c306e700 1 -- 172.21.15.7:0/817779528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1bc13a1d0 con 0x7fe1bc14c7b0 2024-01-27T02:33:24.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.692+0000 7fe1c306e700 1 -- 172.21.15.7:0/817779528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1bc072060 con 0x7fe1bc14c7b0 2024-01-27T02:33:24.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.693+0000 7fe1b8ff9700 1 -- 172.21.15.7:0/817779528 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe1a8013410 con 0x7fe1bc14c7b0 2024-01-27T02:33:24.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.693+0000 7fe1b8ff9700 1 --2- 172.21.15.7:0/817779528 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe1a0066f30 0x7fe1a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:24.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.693+0000 7fe1b8ff9700 1 -- 172.21.15.7:0/817779528 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fe1a8084bb0 con 0x7fe1bc14c7b0 2024-01-27T02:33:24.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.693+0000 7fe1c0e0a700 1 --2- 172.21.15.7:0/817779528 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe1a0066f30 0x7fe1a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:24.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.694+0000 7fe1c0e0a700 1 --2- 172.21.15.7:0/817779528 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe1a0066f30 0x7fe1a00693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0008fd0 tx=0x7fe1b0015000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:24.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.698+0000 7fe1b8ff9700 1 -- 172.21.15.7:0/817779528 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe1a8051f30 con 0x7fe1bc14c7b0 2024-01-27T02:33:24.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:24.947+0000 7fe1c306e700 1 -- 172.21.15.7:0/817779528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe1bc15c180 con 0x7fe1bc14c7b0 2024-01-27T02:33:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.809+0000 7fe1b8ff9700 1 -- 172.21.15.7:0/817779528 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v22) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe1a8055550 con 0x7fe1bc14c7b0 2024-01-27T02:33:25.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.813+0000 7fe1c306e700 1 -- 172.21.15.7:0/817779528 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe1a0066f30 msgr2=0x7fe1a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:25.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.813+0000 7fe1c306e700 1 --2- 172.21.15.7:0/817779528 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe1a0066f30 0x7fe1a00693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0008fd0 tx=0x7fe1b0015000 comp rx=0 tx=0).stop 2024-01-27T02:33:25.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.813+0000 7fe1c306e700 1 -- 172.21.15.7:0/817779528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc14c7b0 msgr2=0x7fe1bc13c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:25.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.813+0000 7fe1c306e700 1 --2- 172.21.15.7:0/817779528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc14c7b0 0x7fe1bc13c1d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8011890 tx=0x7fe1a8008040 comp rx=0 tx=0).stop 2024-01-27T02:33:25.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.816+0000 7fe1c306e700 1 -- 172.21.15.7:0/817779528 shutdown_connections 2024-01-27T02:33:25.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.816+0000 7fe1c306e700 1 --2- 172.21.15.7:0/817779528 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe1a0066f30 0x7fe1a00693e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:25.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.816+0000 7fe1c306e700 1 --2- 172.21.15.7:0/817779528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1bc0c6bf0 0x7fe1bc0c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:25.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.816+0000 7fe1c306e700 1 --2- 172.21.15.7:0/817779528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1ac003a60 0x7fe1bc139b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:25.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.816+0000 7fe1c306e700 1 --2- 172.21.15.7:0/817779528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc14c7b0 0x7fe1bc13c1d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:25.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.816+0000 7fe1c306e700 1 -- 172.21.15.7:0/817779528 >> 172.21.15.7:0/817779528 conn(0x7fe1bc063580 msgr2=0x7fe1bc13f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:25.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.816+0000 7fe1c306e700 1 -- 172.21.15.7:0/817779528 shutdown_connections 2024-01-27T02:33:25.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.816+0000 7fe1c306e700 1 -- 172.21.15.7:0/817779528 wait complete. 2024-01-27T02:33:25.819 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 6 and data pool 7 2024-01-27T02:33:25.833 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:33:25.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.969+0000 7fca40c6b700 1 Processor -- start 2024-01-27T02:33:25.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.969+0000 7fca40c6b700 1 -- start start 2024-01-27T02:33:25.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.970+0000 7fca40c6b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fca3c0c4cc0 0x7fca3c0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:25.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.970+0000 7fca40c6b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca3c0ca540 0x7fca3c163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:25.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.970+0000 7fca40c6b700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3c0ce5e0 con 0x7fca3c15a100 2024-01-27T02:33:25.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.970+0000 7fca40c6b700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca3c0cd9d0 con 0x7fca3c0c4cc0 2024-01-27T02:33:25.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.970+0000 7fca40c6b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c0cdcb0 con 0x7fca3c0ca540 2024-01-27T02:33:25.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.970+0000 7fca39d9b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fca3c0c4cc0 0x7fca3c0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:25.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.970+0000 7fca3ad9d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca3c0ca540 0x7fca3c163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:25.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.970+0000 7fca3a59c700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fca3c15a100 0x7fca3c0c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:50290/0 (socket says 172.21.15.7:50290) 2024-01-27T02:33:25.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.970+0000 7fca3a59c700 1 -- 172.21.15.7:0/4240583582 learned_addr learned my addr 172.21.15.7:0/4240583582 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:25.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.971+0000 7fca39d9b700 1 -- 172.21.15.7:0/4240583582 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca3c0ca540 msgr2=0x7fca3c163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:25.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.971+0000 7fca39d9b700 1 --2- 172.21.15.7:0/4240583582 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca3c0ca540 0x7fca3c163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:25.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.971+0000 7fca39d9b700 1 -- 172.21.15.7:0/4240583582 >> v1:172.21.15.188:6789/0 conn(0x7fca3c15a100 legacy=0x7fca3c0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:25.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.971+0000 7fca39d9b700 1 -- 172.21.15.7:0/4240583582 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c0cdf90 con 0x7fca3c0c4cc0 2024-01-27T02:33:25.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.971+0000 7fca39d9b700 1 --2- 172.21.15.7:0/4240583582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca3c0c4cc0 0x7fca3c0cd2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fca240098e0 tx=0x7fca240091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61d61644d549ad4 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:25.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.971+0000 7fca3959a700 1 -- 172.21.15.7:0/4240583582 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2400b640 con 0x7fca3c0c4cc0 2024-01-27T02:33:25.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.971+0000 7fca3959a700 1 -- 172.21.15.7:0/4240583582 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2400b7a0 con 0x7fca3c0c4cc0 2024-01-27T02:33:25.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.971+0000 7fca3959a700 1 -- 172.21.15.7:0/4240583582 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2400bb40 con 0x7fca3c0c4cc0 2024-01-27T02:33:25.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.972+0000 7fca40c6b700 1 -- 172.21.15.7:0/4240583582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca3c0c4cc0 msgr2=0x7fca3c0cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:25.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.972+0000 7fca40c6b700 1 --2- 172.21.15.7:0/4240583582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca3c0c4cc0 0x7fca3c0cd2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fca240098e0 tx=0x7fca240091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:25.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.972+0000 7fca40c6b700 1 -- 172.21.15.7:0/4240583582 shutdown_connections 2024-01-27T02:33:25.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.972+0000 7fca40c6b700 1 --2- 172.21.15.7:0/4240583582 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca3c0ca540 0x7fca3c163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:25.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.972+0000 7fca40c6b700 1 --2- 172.21.15.7:0/4240583582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca3c0c4cc0 0x7fca3c0cd2f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:25.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.972+0000 7fca40c6b700 1 -- 172.21.15.7:0/4240583582 >> 172.21.15.7:0/4240583582 conn(0x7fca3c098020 msgr2=0x7fca3c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:25.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.972+0000 7fca40c6b700 1 -- 172.21.15.7:0/4240583582 shutdown_connections 2024-01-27T02:33:25.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.972+0000 7fca40c6b700 1 -- 172.21.15.7:0/4240583582 wait complete. 2024-01-27T02:33:25.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.973+0000 7fca40c6b700 1 Processor -- start 2024-01-27T02:33:25.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.973+0000 7fca40c6b700 1 -- start start 2024-01-27T02:33:25.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.974+0000 7fca40c6b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca3c0c4cc0 0x7fca3c152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:25.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.974+0000 7fca40c6b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca3c0ca540 0x7fca3c15c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:25.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.974+0000 7fca40c6b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca3c15a100 0x7fca3c06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:25.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.974+0000 7fca40c6b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c15b530 con 0x7fca3c15a100 2024-01-27T02:33:25.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.974+0000 7fca40c6b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c15a920 con 0x7fca3c0c4cc0 2024-01-27T02:33:25.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.974+0000 7fca40c6b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c15ac00 con 0x7fca3c0ca540 2024-01-27T02:33:25.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.974+0000 7fca3ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca3c15a100 0x7fca3c06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:25.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.974+0000 7fca3ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca3c15a100 0x7fca3c06beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37900/0 (socket says 172.21.15.7:37900) 2024-01-27T02:33:25.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.974+0000 7fca3ad9d700 1 -- 172.21.15.7:0/988743487 learned_addr learned my addr 172.21.15.7:0/988743487 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:25.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.974+0000 7fca3a59c700 1 --2- 172.21.15.7:0/988743487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca3c0c4cc0 0x7fca3c152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:25.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.975+0000 7fca39d9b700 1 --2- 172.21.15.7:0/988743487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca3c0ca540 0x7fca3c15c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:25.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.975+0000 7fca3ad9d700 1 -- 172.21.15.7:0/988743487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca3c0ca540 msgr2=0x7fca3c15c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:25.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.975+0000 7fca3ad9d700 1 --2- 172.21.15.7:0/988743487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca3c0ca540 0x7fca3c15c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:25.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.975+0000 7fca3ad9d700 1 -- 172.21.15.7:0/988743487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca3c0c4cc0 msgr2=0x7fca3c152680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:25.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.975+0000 7fca3ad9d700 1 --2- 172.21.15.7:0/988743487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca3c0c4cc0 0x7fca3c152680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:25.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.975+0000 7fca3ad9d700 1 -- 172.21.15.7:0/988743487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca24009420 con 0x7fca3c15a100 2024-01-27T02:33:25.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.975+0000 7fca3ad9d700 1 --2- 172.21.15.7:0/988743487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca3c15a100 0x7fca3c06beb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fca34009e90 tx=0x7fca3400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:25.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.975+0000 7fca327fc700 1 -- 172.21.15.7:0/988743487 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3400cb70 con 0x7fca3c15a100 2024-01-27T02:33:25.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.975+0000 7fca327fc700 1 -- 172.21.15.7:0/988743487 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3400f040 con 0x7fca3c15a100 2024-01-27T02:33:25.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.975+0000 7fca327fc700 1 -- 172.21.15.7:0/988743487 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34014480 con 0x7fca3c15a100 2024-01-27T02:33:25.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.975+0000 7fca40c6b700 1 -- 172.21.15.7:0/988743487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3c15aee0 con 0x7fca3c15a100 2024-01-27T02:33:25.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.975+0000 7fca40c6b700 1 -- 172.21.15.7:0/988743487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3c152bc0 con 0x7fca3c15a100 2024-01-27T02:33:25.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.976+0000 7fca40c6b700 1 -- 172.21.15.7:0/988743487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3c15b670 con 0x7fca3c15a100 2024-01-27T02:33:25.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.977+0000 7fca327fc700 1 -- 172.21.15.7:0/988743487 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fca34014830 con 0x7fca3c15a100 2024-01-27T02:33:25.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.977+0000 7fca327fc700 1 --2- 172.21.15.7:0/988743487 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fca1c066f30 0x7fca1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:25.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.977+0000 7fca327fc700 1 -- 172.21.15.7:0/988743487 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fca34084f00 con 0x7fca3c15a100 2024-01-27T02:33:25.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.977+0000 7fca3a59c700 1 --2- 172.21.15.7:0/988743487 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fca1c066f30 0x7fca1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:25.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.978+0000 7fca3a59c700 1 --2- 172.21.15.7:0/988743487 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fca1c066f30 0x7fca1c0693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fca2c007130 tx=0x7fca2c007eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:25.982+0000 7fca327fc700 1 -- 172.21.15.7:0/988743487 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca3c15b670 con 0x7fca3c15a100 2024-01-27T02:33:26.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.185+0000 7fca40c6b700 1 -- 172.21.15.7:0/988743487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca3c15b670 con 0x7fca3c15a100 2024-01-27T02:33:26.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.186+0000 7fca327fc700 1 -- 172.21.15.7:0/988743487 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+15315 (secure 0 0 0) 0x7fca3c15b670 con 0x7fca3c15a100 2024-01-27T02:33:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:26.188 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":44,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:33:24.949973+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:33:23.445643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-27T02:33:23.921995+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:33:26.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.188+0000 7fca40c6b700 1 -- 172.21.15.7:0/988743487 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fca1c066f30 msgr2=0x7fca1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:26.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.188+0000 7fca40c6b700 1 --2- 172.21.15.7:0/988743487 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fca1c066f30 0x7fca1c0693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fca2c007130 tx=0x7fca2c007eb0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.188+0000 7fca40c6b700 1 -- 172.21.15.7:0/988743487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca3c15a100 msgr2=0x7fca3c06beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:26.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.188+0000 7fca40c6b700 1 --2- 172.21.15.7:0/988743487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca3c15a100 0x7fca3c06beb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fca34009e90 tx=0x7fca3400b040 comp rx=0 tx=0).stop 2024-01-27T02:33:26.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.189+0000 7fca40c6b700 1 -- 172.21.15.7:0/988743487 shutdown_connections 2024-01-27T02:33:26.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.189+0000 7fca40c6b700 1 --2- 172.21.15.7:0/988743487 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fca1c066f30 0x7fca1c0693e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.189+0000 7fca40c6b700 1 --2- 172.21.15.7:0/988743487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca3c15a100 0x7fca3c06beb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.189+0000 7fca40c6b700 1 --2- 172.21.15.7:0/988743487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca3c0c4cc0 0x7fca3c152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.189+0000 7fca40c6b700 1 --2- 172.21.15.7:0/988743487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca3c0ca540 0x7fca3c15c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.189+0000 7fca40c6b700 1 -- 172.21.15.7:0/988743487 >> 172.21.15.7:0/988743487 conn(0x7fca3c098020 msgr2=0x7fca3c1544e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:26.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.189+0000 7fca40c6b700 1 -- 172.21.15.7:0/988743487 shutdown_connections 2024-01-27T02:33:26.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.189+0000 7fca40c6b700 1 -- 172.21.15.7:0/988743487 wait complete. 2024-01-27T02:33:26.204 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:33:26.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.334+0000 7f8dbe04d700 1 Processor -- start 2024-01-27T02:33:26.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.335+0000 7f8dbe04d700 1 -- start start 2024-01-27T02:33:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.335+0000 7f8dbe04d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8db80a7c10 0x7f8db80ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.335+0000 7f8dbe04d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8db80ae880 0x7f8db806f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.335+0000 7f8dbe04d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8db80a7fe0 0x7f8db806f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.335+0000 7f8dbe04d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db80afde0 con 0x7f8db80a7fe0 2024-01-27T02:33:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.335+0000 7f8dbe04d700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db815a880 con 0x7f8db80ae880 2024-01-27T02:33:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.335+0000 7f8dbe04d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db80af950 con 0x7f8db80a7c10 2024-01-27T02:33:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.336+0000 7f8db7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8db80a7fe0 0x7f8db806f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.336+0000 7f8db77fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8db80a7c10 0x7f8db80ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.336+0000 7f8db7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8db80a7fe0 0x7f8db806f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37906/0 (socket says 172.21.15.7:37906) 2024-01-27T02:33:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.336+0000 7f8db7fff700 1 -- 172.21.15.7:0/3105006558 learned_addr learned my addr 172.21.15.7:0/3105006558 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.336+0000 7f8db6ffd700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8db80ae880 0x7f8db806f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.336+0000 7f8db7fff700 1 -- 172.21.15.7:0/3105006558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8db80a7c10 msgr2=0x7f8db80ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.336+0000 7f8db7fff700 1 --2- 172.21.15.7:0/3105006558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8db80a7c10 0x7f8db80ae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.336+0000 7f8db7fff700 1 -- 172.21.15.7:0/3105006558 >> v2:172.21.15.188:3300/0 conn(0x7f8db80ae880 msgr2=0x7f8db806f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.336+0000 7f8db7fff700 1 --2- 172.21.15.7:0/3105006558 >> v2:172.21.15.188:3300/0 conn(0x7f8db80ae880 0x7f8db806f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.336+0000 7f8db7fff700 1 -- 172.21.15.7:0/3105006558 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db80aed40 con 0x7f8db80a7fe0 2024-01-27T02:33:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.336+0000 7f8db7fff700 1 --2- 172.21.15.7:0/3105006558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8db80a7fe0 0x7f8db806f8a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8db806d4f0 tx=0x7f8db0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9332e84a9997a791 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.336+0000 7f8db67fc700 1 -- 172.21.15.7:0/3105006558 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db000e070 con 0x7f8db80a7fe0 2024-01-27T02:33:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.336+0000 7f8db67fc700 1 -- 172.21.15.7:0/3105006558 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db000b970 con 0x7f8db80a7fe0 2024-01-27T02:33:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.336+0000 7f8db67fc700 1 -- 172.21.15.7:0/3105006558 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db00092e0 con 0x7f8db80a7fe0 2024-01-27T02:33:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.337+0000 7f8dbe04d700 1 -- 172.21.15.7:0/3105006558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8db80a7fe0 msgr2=0x7f8db806f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.337+0000 7f8dbe04d700 1 --2- 172.21.15.7:0/3105006558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8db80a7fe0 0x7f8db806f8a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8db806d4f0 tx=0x7f8db0009840 comp rx=0 tx=0).stop 2024-01-27T02:33:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.337+0000 7f8dbe04d700 1 -- 172.21.15.7:0/3105006558 shutdown_connections 2024-01-27T02:33:26.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.337+0000 7f8dbe04d700 1 --2- 172.21.15.7:0/3105006558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8db80a7fe0 0x7f8db806f8a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.337+0000 7f8dbe04d700 1 --2- 172.21.15.7:0/3105006558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8db80a7c10 0x7f8db80ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.337+0000 7f8dbe04d700 1 --2- 172.21.15.7:0/3105006558 >> v2:172.21.15.188:3300/0 conn(0x7f8db80ae880 0x7f8db806f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.337+0000 7f8dbe04d700 1 -- 172.21.15.7:0/3105006558 >> 172.21.15.7:0/3105006558 conn(0x7f8db80679b0 msgr2=0x7f8db8067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:26.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.337+0000 7f8dbe04d700 1 -- 172.21.15.7:0/3105006558 shutdown_connections 2024-01-27T02:33:26.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.337+0000 7f8dbe04d700 1 -- 172.21.15.7:0/3105006558 wait complete. 2024-01-27T02:33:26.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.339+0000 7f8dbe04d700 1 Processor -- start 2024-01-27T02:33:26.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.339+0000 7f8dbe04d700 1 -- start start 2024-01-27T02:33:26.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.339+0000 7f8dbe04d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8db80a7c10 0x7f8db815b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:26.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.339+0000 7f8dbe04d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8db80a7fe0 0x7f8db8156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:26.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.339+0000 7f8dbe04d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8db80ae880 0x7f8db8097f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:26.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.339+0000 7f8dbe04d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db815b910 con 0x7f8db80a7c10 2024-01-27T02:33:26.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.339+0000 7f8dbe04d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db815c3a0 con 0x7f8db80ae880 2024-01-27T02:33:26.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.339+0000 7f8dbe04d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db816aa60 con 0x7f8db80a7fe0 2024-01-27T02:33:26.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.340+0000 7f8db77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8db80a7c10 0x7f8db815b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:26.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.340+0000 7f8db77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8db80a7c10 0x7f8db815b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37922/0 (socket says 172.21.15.7:37922) 2024-01-27T02:33:26.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.340+0000 7f8db77fe700 1 -- 172.21.15.7:0/3549489478 learned_addr learned my addr 172.21.15.7:0/3549489478 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:26.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.340+0000 7f8db7fff700 1 --2- 172.21.15.7:0/3549489478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8db80ae880 0x7f8db8097f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:26.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.340+0000 7f8db6ffd700 1 --2- 172.21.15.7:0/3549489478 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8db80a7fe0 0x7f8db8156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:26.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.340+0000 7f8db77fe700 1 -- 172.21.15.7:0/3549489478 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8db80a7fe0 msgr2=0x7f8db8156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:26.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.340+0000 7f8db77fe700 1 --2- 172.21.15.7:0/3549489478 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8db80a7fe0 0x7f8db8156450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.340+0000 7f8db77fe700 1 -- 172.21.15.7:0/3549489478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8db80ae880 msgr2=0x7f8db8097f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:26.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.340+0000 7f8db77fe700 1 --2- 172.21.15.7:0/3549489478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8db80ae880 0x7f8db8097f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.340+0000 7f8db77fe700 1 -- 172.21.15.7:0/3549489478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db0009ae0 con 0x7f8db80a7c10 2024-01-27T02:33:26.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.340+0000 7f8db77fe700 1 --2- 172.21.15.7:0/3549489478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8db80a7c10 0x7f8db815b3d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8da800ec80 tx=0x7f8da80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:26.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.340+0000 7f8d9f7fe700 1 -- 172.21.15.7:0/3549489478 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da800c850 con 0x7f8db80a7c10 2024-01-27T02:33:26.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.340+0000 7f8d9f7fe700 1 -- 172.21.15.7:0/3549489478 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da800c9b0 con 0x7f8db80a7c10 2024-01-27T02:33:26.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.340+0000 7f8d9f7fe700 1 -- 172.21.15.7:0/3549489478 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da800cd50 con 0x7f8db80a7c10 2024-01-27T02:33:26.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.340+0000 7f8dbe04d700 1 -- 172.21.15.7:0/3549489478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db816a770 con 0x7f8db80a7c10 2024-01-27T02:33:26.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.340+0000 7f8dbe04d700 1 -- 172.21.15.7:0/3549489478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db815bde0 con 0x7f8db80a7c10 2024-01-27T02:33:26.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.341+0000 7f8dbe04d700 1 -- 172.21.15.7:0/3549489478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db804c710 con 0x7f8db80a7c10 2024-01-27T02:33:26.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.342+0000 7f8d9f7fe700 1 -- 172.21.15.7:0/3549489478 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8da8013680 con 0x7f8db80a7c10 2024-01-27T02:33:26.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.342+0000 7f8d9f7fe700 1 --2- 172.21.15.7:0/3549489478 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8da4066f30 0x7f8da40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:26.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.342+0000 7f8d9f7fe700 1 -- 172.21.15.7:0/3549489478 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8da8084750 con 0x7f8db80a7c10 2024-01-27T02:33:26.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.343+0000 7f8db6ffd700 1 --2- 172.21.15.7:0/3549489478 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8da4066f30 0x7f8da40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:26.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.343+0000 7f8db6ffd700 1 --2- 172.21.15.7:0/3549489478 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8da4066f30 0x7f8da40693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8db80a7690 tx=0x7f8dac0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:26.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.347+0000 7f8d9f7fe700 1 -- 172.21.15.7:0/3549489478 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8da80519c0 con 0x7f8db80a7c10 2024-01-27T02:33:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.554+0000 7f8dbe04d700 1 -- 172.21.15.7:0/3549489478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8db80af020 con 0x7f8db80a7c10 2024-01-27T02:33:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.555+0000 7f8d9f7fe700 1 -- 172.21.15.7:0/3549489478 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f8da8054fe0 con 0x7f8db80a7c10 2024-01-27T02:33:26.556 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:26.556 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":45,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:33:26.519256+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:33:23.445643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-27T02:33:23.921995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:33:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.557+0000 7f8dbe04d700 1 -- 172.21.15.7:0/3549489478 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8da4066f30 msgr2=0x7f8da40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.557+0000 7f8dbe04d700 1 --2- 172.21.15.7:0/3549489478 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8da4066f30 0x7f8da40693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8db80a7690 tx=0x7f8dac0091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.557+0000 7f8dbe04d700 1 -- 172.21.15.7:0/3549489478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8db80a7c10 msgr2=0x7f8db815b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.557+0000 7f8dbe04d700 1 --2- 172.21.15.7:0/3549489478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8db80a7c10 0x7f8db815b3d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8da800ec80 tx=0x7f8da80091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.557+0000 7f8dbe04d700 1 -- 172.21.15.7:0/3549489478 shutdown_connections 2024-01-27T02:33:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.557+0000 7f8dbe04d700 1 --2- 172.21.15.7:0/3549489478 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8da4066f30 0x7f8da40693e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.557+0000 7f8dbe04d700 1 --2- 172.21.15.7:0/3549489478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8db80ae880 0x7f8db8097f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.557+0000 7f8dbe04d700 1 --2- 172.21.15.7:0/3549489478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8db80a7c10 0x7f8db815b3d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.557+0000 7f8dbe04d700 1 --2- 172.21.15.7:0/3549489478 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8db80a7fe0 0x7f8db8156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.557+0000 7f8dbe04d700 1 -- 172.21.15.7:0/3549489478 >> 172.21.15.7:0/3549489478 conn(0x7f8db80679b0 msgr2=0x7f8db81583f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.557+0000 7f8dbe04d700 1 -- 172.21.15.7:0/3549489478 shutdown_connections 2024-01-27T02:33:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.557+0000 7f8dbe04d700 1 -- 172.21.15.7:0/3549489478 wait complete. 2024-01-27T02:33:26.572 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-27T02:33:26.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.705+0000 7f631097c700 1 Processor -- start 2024-01-27T02:33:26.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.705+0000 7f631097c700 1 -- start start 2024-01-27T02:33:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.706+0000 7f631097c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f630c0cb950 0x7f630c0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.706+0000 7f631097c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f630c1412d0 0x7f630c141730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.706+0000 7f631097c700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f630c0c9660 con 0x7f630c0cbd20 2024-01-27T02:33:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.706+0000 7f631097c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f630c15c980 con 0x7f630c1412d0 2024-01-27T02:33:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.706+0000 7f631097c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630c141c70 con 0x7f630c0cb950 2024-01-27T02:33:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.707+0000 7f630ad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f630c1412d0 0x7f630c141730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.707+0000 7f630ad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f630c1412d0 0x7f630c141730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37926/0 (socket says 172.21.15.7:37926) 2024-01-27T02:33:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.707+0000 7f630ad9d700 1 -- 172.21.15.7:0/2898535745 learned_addr learned my addr 172.21.15.7:0/2898535745 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.707+0000 7f630a59c700 1 --2- 172.21.15.7:0/2898535745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f630c0cb950 0x7f630c0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.707+0000 7f630ad9d700 1 -- 172.21.15.7:0/2898535745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f630c0cb950 msgr2=0x7f630c0b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.707+0000 7f630ad9d700 1 --2- 172.21.15.7:0/2898535745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f630c0cb950 0x7f630c0b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.707+0000 7f630ad9d700 1 -- 172.21.15.7:0/2898535745 >> v1:172.21.15.188:6789/0 conn(0x7f630c0cbd20 legacy=0x7f630c0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.707+0000 7f630ad9d700 1 -- 172.21.15.7:0/2898535745 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c15be00 con 0x7f630c1412d0 2024-01-27T02:33:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.707+0000 7f630ad9d700 1 --2- 172.21.15.7:0/2898535745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f630c1412d0 0x7f630c141730 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6304009f20 tx=0x7f6304009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2979beb14dc50e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:26.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.707+0000 7f630959a700 1 -- 172.21.15.7:0/2898535745 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630400e070 con 0x7f630c1412d0 2024-01-27T02:33:26.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.707+0000 7f630959a700 1 -- 172.21.15.7:0/2898535745 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630400ba80 con 0x7f630c1412d0 2024-01-27T02:33:26.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.707+0000 7f630959a700 1 -- 172.21.15.7:0/2898535745 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63040092e0 con 0x7f630c1412d0 2024-01-27T02:33:26.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.708+0000 7f631097c700 1 -- 172.21.15.7:0/2898535745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f630c1412d0 msgr2=0x7f630c141730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:26.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.708+0000 7f631097c700 1 --2- 172.21.15.7:0/2898535745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f630c1412d0 0x7f630c141730 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6304009f20 tx=0x7f6304009840 comp rx=0 tx=0).stop 2024-01-27T02:33:26.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.708+0000 7f631097c700 1 -- 172.21.15.7:0/2898535745 shutdown_connections 2024-01-27T02:33:26.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.708+0000 7f631097c700 1 --2- 172.21.15.7:0/2898535745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f630c1412d0 0x7f630c141730 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.708+0000 7f631097c700 1 --2- 172.21.15.7:0/2898535745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f630c0cb950 0x7f630c0b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.708+0000 7f631097c700 1 -- 172.21.15.7:0/2898535745 >> 172.21.15.7:0/2898535745 conn(0x7f630c063580 msgr2=0x7f630c063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:26.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.708+0000 7f631097c700 1 -- 172.21.15.7:0/2898535745 shutdown_connections 2024-01-27T02:33:26.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.708+0000 7f631097c700 1 -- 172.21.15.7:0/2898535745 wait complete. 2024-01-27T02:33:26.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.709+0000 7f631097c700 1 Processor -- start 2024-01-27T02:33:26.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.709+0000 7f631097c700 1 -- start start 2024-01-27T02:33:26.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.710+0000 7f631097c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f630c0cb950 0x7f630c1354d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:26.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.710+0000 7f631097c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f630c0cbd20 0x7f630c0679b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:26.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.710+0000 7f631097c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f630c1412d0 0x7f630c067ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:26.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.710+0000 7f631097c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c0c6d80 con 0x7f630c1412d0 2024-01-27T02:33:26.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.710+0000 7f631097c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c1407e0 con 0x7f630c0cbd20 2024-01-27T02:33:26.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.710+0000 7f631097c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630c159350 con 0x7f630c0cb950 2024-01-27T02:33:26.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.710+0000 7f630ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f630c1412d0 0x7f630c067ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:26.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.710+0000 7f630ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f630c1412d0 0x7f630c067ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37936/0 (socket says 172.21.15.7:37936) 2024-01-27T02:33:26.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.710+0000 7f630ad9d700 1 -- 172.21.15.7:0/3033460723 learned_addr learned my addr 172.21.15.7:0/3033460723 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:26.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.711+0000 7f630a59c700 1 --2- 172.21.15.7:0/3033460723 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f630c0cb950 0x7f630c1354d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:26.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.711+0000 7f6309d9b700 1 --2- 172.21.15.7:0/3033460723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f630c0cbd20 0x7f630c0679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:26.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.711+0000 7f630ad9d700 1 -- 172.21.15.7:0/3033460723 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f630c0cb950 msgr2=0x7f630c1354d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:26.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.711+0000 7f630ad9d700 1 --2- 172.21.15.7:0/3033460723 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f630c0cb950 0x7f630c1354d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.711+0000 7f630ad9d700 1 -- 172.21.15.7:0/3033460723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f630c0cbd20 msgr2=0x7f630c0679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:26.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.711+0000 7f630ad9d700 1 --2- 172.21.15.7:0/3033460723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f630c0cbd20 0x7f630c0679b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:26.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.711+0000 7f630ad9d700 1 -- 172.21.15.7:0/3033460723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6304009ae0 con 0x7f630c1412d0 2024-01-27T02:33:26.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.711+0000 7f630a59c700 1 --2- 172.21.15.7:0/3033460723 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f630c0cb950 0x7f630c1354d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:33:26.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.711+0000 7f6309d9b700 1 --2- 172.21.15.7:0/3033460723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f630c0cbd20 0x7f630c0679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:33:26.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.711+0000 7f630ad9d700 1 --2- 172.21.15.7:0/3033460723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f630c1412d0 0x7f630c067ef0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f630400b6c0 tx=0x7f6304018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:26.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.711+0000 7f63027fc700 1 -- 172.21.15.7:0/3033460723 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63040094e0 con 0x7f630c1412d0 2024-01-27T02:33:26.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.711+0000 7f63027fc700 1 -- 172.21.15.7:0/3033460723 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63040032f0 con 0x7f630c1412d0 2024-01-27T02:33:26.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.711+0000 7f631097c700 1 -- 172.21.15.7:0/3033460723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f630c0c6fd0 con 0x7f630c1412d0 2024-01-27T02:33:26.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.711+0000 7f63027fc700 1 -- 172.21.15.7:0/3033460723 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630401e660 con 0x7f630c1412d0 2024-01-27T02:33:26.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.711+0000 7f631097c700 1 -- 172.21.15.7:0/3033460723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f630c0c7590 con 0x7f630c1412d0 2024-01-27T02:33:26.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.712+0000 7f631097c700 1 -- 172.21.15.7:0/3033460723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f630c072060 con 0x7f630c1412d0 2024-01-27T02:33:26.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.712+0000 7f63027fc700 1 -- 172.21.15.7:0/3033460723 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6304003040 con 0x7f630c1412d0 2024-01-27T02:33:26.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.713+0000 7f63027fc700 1 --2- 172.21.15.7:0/3033460723 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f62ec066f30 0x7f62ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:26.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.713+0000 7f63027fc700 1 -- 172.21.15.7:0/3033460723 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6304086760 con 0x7f630c1412d0 2024-01-27T02:33:26.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.713+0000 7f630a59c700 1 --2- 172.21.15.7:0/3033460723 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f62ec066f30 0x7f62ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:26.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.714+0000 7f630a59c700 1 --2- 172.21.15.7:0/3033460723 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f62ec066f30 0x7f62ec0693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f62f4009990 tx=0x7f62f4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:26.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.717+0000 7f63027fc700 1 -- 172.21.15.7:0/3033460723 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f63040539d0 con 0x7f630c1412d0 2024-01-27T02:33:26.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:26.970+0000 7f631097c700 1 -- 172.21.15.7:0/3033460723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f630c06d3e0 con 0x7f630c1412d0 2024-01-27T02:33:27.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.815+0000 7f63027fc700 1 -- 172.21.15.7:0/3033460723 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6304056ff0 con 0x7f630c1412d0 2024-01-27T02:33:27.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.818+0000 7f631097c700 1 -- 172.21.15.7:0/3033460723 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f62ec066f30 msgr2=0x7f62ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:27.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.818+0000 7f631097c700 1 --2- 172.21.15.7:0/3033460723 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f62ec066f30 0x7f62ec0693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f62f4009990 tx=0x7f62f4008040 comp rx=0 tx=0).stop 2024-01-27T02:33:27.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.818+0000 7f631097c700 1 -- 172.21.15.7:0/3033460723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f630c1412d0 msgr2=0x7f630c067ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:27.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.818+0000 7f631097c700 1 --2- 172.21.15.7:0/3033460723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f630c1412d0 0x7f630c067ef0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f630400b6c0 tx=0x7f6304018000 comp rx=0 tx=0).stop 2024-01-27T02:33:27.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.818+0000 7f631097c700 1 -- 172.21.15.7:0/3033460723 shutdown_connections 2024-01-27T02:33:27.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.818+0000 7f631097c700 1 --2- 172.21.15.7:0/3033460723 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f62ec066f30 0x7f62ec0693e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:27.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.818+0000 7f631097c700 1 --2- 172.21.15.7:0/3033460723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f630c1412d0 0x7f630c067ef0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:27.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.818+0000 7f631097c700 1 --2- 172.21.15.7:0/3033460723 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f630c0cb950 0x7f630c1354d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:27.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.818+0000 7f631097c700 1 --2- 172.21.15.7:0/3033460723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f630c0cbd20 0x7f630c0679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:27.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.818+0000 7f631097c700 1 -- 172.21.15.7:0/3033460723 >> 172.21.15.7:0/3033460723 conn(0x7f630c063580 msgr2=0x7f630c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:27.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.818+0000 7f631097c700 1 -- 172.21.15.7:0/3033460723 shutdown_connections 2024-01-27T02:33:27.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.818+0000 7f631097c700 1 -- 172.21.15.7:0/3033460723 wait complete. 2024-01-27T02:33:27.835 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-27T02:33:27.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.972+0000 7f90a6c99700 1 Processor -- start 2024-01-27T02:33:27.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.972+0000 7f90a6c99700 1 -- start start 2024-01-27T02:33:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.973+0000 7f90a6c99700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90a015a180 0x7f90a00c3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.973+0000 7f90a6c99700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f90a015a550 0x7f90a00c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.973+0000 7f90a6c99700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a00b1450 con 0x7f90a00c3c90 2024-01-27T02:33:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.973+0000 7f90a6c99700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a00b18e0 con 0x7f90a015a550 2024-01-27T02:33:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.973+0000 7f90a6c99700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a00b0840 con 0x7f90a015a180 2024-01-27T02:33:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.973+0000 7f909ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f90a015a550 0x7f90a00c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.973+0000 7f909ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f90a015a550 0x7f90a00c9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37938/0 (socket says 172.21.15.7:37938) 2024-01-27T02:33:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.973+0000 7f909ffff700 1 -- 172.21.15.7:0/1655753674 learned_addr learned my addr 172.21.15.7:0/1655753674 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.973+0000 7f90a4a35700 1 --2- 172.21.15.7:0/1655753674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90a015a180 0x7f90a00c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.973+0000 7f909ffff700 1 -- 172.21.15.7:0/1655753674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90a015a180 msgr2=0x7f90a00c3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.973+0000 7f909ffff700 1 --2- 172.21.15.7:0/1655753674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90a015a180 0x7f90a00c3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.973+0000 7f909ffff700 1 -- 172.21.15.7:0/1655753674 >> v1:172.21.15.188:6789/0 conn(0x7f90a00c3c90 legacy=0x7f90a00aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.973+0000 7f909ffff700 1 -- 172.21.15.7:0/1655753674 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a00b0b20 con 0x7f90a015a550 2024-01-27T02:33:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.973+0000 7f909ffff700 1 --2- 172.21.15.7:0/1655753674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a015a550 0x7f90a00c9ca0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f90940099b0 tx=0x7f9094009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2feda45075f1b54 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.973+0000 7f909f7fe700 1 -- 172.21.15.7:0/1655753674 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909400b820 con 0x7f90a015a550 2024-01-27T02:33:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.974+0000 7f909f7fe700 1 -- 172.21.15.7:0/1655753674 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909400b980 con 0x7f90a015a550 2024-01-27T02:33:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.974+0000 7f909f7fe700 1 -- 172.21.15.7:0/1655753674 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909400bd20 con 0x7f90a015a550 2024-01-27T02:33:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.974+0000 7f90a6c99700 1 -- 172.21.15.7:0/1655753674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a015a550 msgr2=0x7f90a00c9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.974+0000 7f90a6c99700 1 --2- 172.21.15.7:0/1655753674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a015a550 0x7f90a00c9ca0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f90940099b0 tx=0x7f9094009250 comp rx=0 tx=0).stop 2024-01-27T02:33:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.974+0000 7f90a6c99700 1 -- 172.21.15.7:0/1655753674 shutdown_connections 2024-01-27T02:33:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.974+0000 7f90a6c99700 1 --2- 172.21.15.7:0/1655753674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90a015a180 0x7f90a00c3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.974+0000 7f90a6c99700 1 --2- 172.21.15.7:0/1655753674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a015a550 0x7f90a00c9ca0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.974+0000 7f90a6c99700 1 -- 172.21.15.7:0/1655753674 >> 172.21.15.7:0/1655753674 conn(0x7f90a00679b0 msgr2=0x7f90a0067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.974+0000 7f90a6c99700 1 -- 172.21.15.7:0/1655753674 shutdown_connections 2024-01-27T02:33:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.974+0000 7f90a6c99700 1 -- 172.21.15.7:0/1655753674 wait complete. 2024-01-27T02:33:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.975+0000 7f90a6c99700 1 Processor -- start 2024-01-27T02:33:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.976+0000 7f90a6c99700 1 -- start start 2024-01-27T02:33:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.976+0000 7f90a6c99700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a00c3c90 0x7f90a015ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.976+0000 7f90a6c99700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90a015a180 0x7f90a016acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.976+0000 7f90a6c99700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90a015a550 0x7f90a0169ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.976+0000 7f90a6c99700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a015ad50 con 0x7f90a00c3c90 2024-01-27T02:33:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.976+0000 7f90a6c99700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a015dd00 con 0x7f90a015a550 2024-01-27T02:33:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.976+0000 7f90a6c99700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a016b200 con 0x7f90a015a180 2024-01-27T02:33:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.976+0000 7f90a4a35700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a00c3c90 0x7f90a015ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.977+0000 7f90a4a35700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a00c3c90 0x7f90a015ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37942/0 (socket says 172.21.15.7:37942) 2024-01-27T02:33:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.977+0000 7f90a4a35700 1 -- 172.21.15.7:0/2161917896 learned_addr learned my addr 172.21.15.7:0/2161917896 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.977+0000 7f90a5236700 1 --2- 172.21.15.7:0/2161917896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90a015a550 0x7f90a0169ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:27.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.977+0000 7f909ffff700 1 --2- 172.21.15.7:0/2161917896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90a015a180 0x7f90a016acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:27.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.977+0000 7f90a4a35700 1 -- 172.21.15.7:0/2161917896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90a015a180 msgr2=0x7f90a016acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:27.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.977+0000 7f90a4a35700 1 --2- 172.21.15.7:0/2161917896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90a015a180 0x7f90a016acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:27.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.977+0000 7f90a4a35700 1 -- 172.21.15.7:0/2161917896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90a015a550 msgr2=0x7f90a0169ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:27.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.977+0000 7f90a4a35700 1 --2- 172.21.15.7:0/2161917896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90a015a550 0x7f90a0169ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:27.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.977+0000 7f90a4a35700 1 -- 172.21.15.7:0/2161917896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90940094f0 con 0x7f90a00c3c90 2024-01-27T02:33:27.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.977+0000 7f90a5236700 1 --2- 172.21.15.7:0/2161917896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90a015a550 0x7f90a0169ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:27.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.977+0000 7f90a4a35700 1 --2- 172.21.15.7:0/2161917896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a00c3c90 0x7f90a015ca80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f909000d890 tx=0x7f9090008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:27.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.977+0000 7f909cff9700 1 -- 172.21.15.7:0/2161917896 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9090018070 con 0x7f90a00c3c90 2024-01-27T02:33:27.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.977+0000 7f909cff9700 1 -- 172.21.15.7:0/2161917896 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9090005a00 con 0x7f90a00c3c90 2024-01-27T02:33:27.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.977+0000 7f90a6c99700 1 -- 172.21.15.7:0/2161917896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a015d0f0 con 0x7f90a00c3c90 2024-01-27T02:33:27.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.977+0000 7f909cff9700 1 -- 172.21.15.7:0/2161917896 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9090005dc0 con 0x7f90a00c3c90 2024-01-27T02:33:27.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.977+0000 7f90a6c99700 1 -- 172.21.15.7:0/2161917896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a015d6b0 con 0x7f90a00c3c90 2024-01-27T02:33:27.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.978+0000 7f90a6c99700 1 -- 172.21.15.7:0/2161917896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a0072060 con 0x7f90a00c3c90 2024-01-27T02:33:27.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.979+0000 7f909cff9700 1 -- 172.21.15.7:0/2161917896 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9090012800 con 0x7f90a00c3c90 2024-01-27T02:33:27.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.979+0000 7f909cff9700 1 --2- 172.21.15.7:0/2161917896 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f908c066f30 0x7f908c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:27.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.979+0000 7f909cff9700 1 -- 172.21.15.7:0/2161917896 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9090084640 con 0x7f90a00c3c90 2024-01-27T02:33:27.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.980+0000 7f909ffff700 1 --2- 172.21.15.7:0/2161917896 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f908c066f30 0x7f908c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:27.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.980+0000 7f909ffff700 1 --2- 172.21.15.7:0/2161917896 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f908c066f30 0x7f908c0693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f90940099b0 tx=0x7f9094016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:27.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:27.984+0000 7f909cff9700 1 -- 172.21.15.7:0/2161917896 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9090055340 con 0x7f90a00c3c90 2024-01-27T02:33:28.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.235+0000 7f90a6c99700 1 -- 172.21.15.7:0/2161917896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f90a00cacb0 con 0x7f90a00c3c90 2024-01-27T02:33:28.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.816+0000 7f909cff9700 1 -- 172.21.15.7:0/2161917896 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9090054ed0 con 0x7f90a00c3c90 2024-01-27T02:33:28.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.819+0000 7f90a6c99700 1 -- 172.21.15.7:0/2161917896 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f908c066f30 msgr2=0x7f908c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:28.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.819+0000 7f90a6c99700 1 --2- 172.21.15.7:0/2161917896 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f908c066f30 0x7f908c0693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f90940099b0 tx=0x7f9094016000 comp rx=0 tx=0).stop 2024-01-27T02:33:28.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.819+0000 7f90a6c99700 1 -- 172.21.15.7:0/2161917896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a00c3c90 msgr2=0x7f90a015ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:28.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.819+0000 7f90a6c99700 1 --2- 172.21.15.7:0/2161917896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a00c3c90 0x7f90a015ca80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f909000d890 tx=0x7f9090008040 comp rx=0 tx=0).stop 2024-01-27T02:33:28.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.819+0000 7f90a6c99700 1 -- 172.21.15.7:0/2161917896 shutdown_connections 2024-01-27T02:33:28.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.819+0000 7f90a6c99700 1 --2- 172.21.15.7:0/2161917896 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f908c066f30 0x7f908c0693e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:28.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.819+0000 7f90a6c99700 1 --2- 172.21.15.7:0/2161917896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90a015a550 0x7f90a0169ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:28.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.819+0000 7f90a6c99700 1 --2- 172.21.15.7:0/2161917896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a00c3c90 0x7f90a015ca80 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:28.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.819+0000 7f90a6c99700 1 --2- 172.21.15.7:0/2161917896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90a015a180 0x7f90a016acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:28.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.819+0000 7f90a6c99700 1 -- 172.21.15.7:0/2161917896 >> 172.21.15.7:0/2161917896 conn(0x7f90a00679b0 msgr2=0x7f90a006f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:28.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.819+0000 7f90a6c99700 1 -- 172.21.15.7:0/2161917896 shutdown_connections 2024-01-27T02:33:28.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.819+0000 7f90a6c99700 1 -- 172.21.15.7:0/2161917896 wait complete. 2024-01-27T02:33:28.837 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:33:28.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.972+0000 7f0decb97700 1 Processor -- start 2024-01-27T02:33:28.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.972+0000 7f0decb97700 1 -- start start 2024-01-27T02:33:28.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.973+0000 7f0decb97700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0de80b1c00 0x7f0de806e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:28.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.973+0000 7f0decb97700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0de815b5f0 0x7f0de81566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:28.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.973+0000 7f0decb97700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0de8157610 con 0x7f0de80b1fd0 2024-01-27T02:33:28.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.973+0000 7f0decb97700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0de8157310 con 0x7f0de80b1c00 2024-01-27T02:33:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.973+0000 7f0decb97700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de806ee00 con 0x7f0de815b5f0 2024-01-27T02:33:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.973+0000 7f0de659c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0de80b1c00 0x7f0de806e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.973+0000 7f0de5d9b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0de815b5f0 0x7f0de81566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.973+0000 7f0de6d9d700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f0de80b1fd0 0x7f0de8156c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:50322/0 (socket says 172.21.15.7:50322) 2024-01-27T02:33:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.973+0000 7f0de5d9b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0de815b5f0 0x7f0de81566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:33502/0 (socket says 172.21.15.7:33502) 2024-01-27T02:33:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.973+0000 7f0de659c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0de80b1c00 0x7f0de806e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:56166/0 (socket says 172.21.15.7:56166) 2024-01-27T02:33:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.973+0000 7f0de6d9d700 1 -- 172.21.15.7:0/1418723489 learned_addr learned my addr 172.21.15.7:0/1418723489 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.974+0000 7f0de5d9b700 1 -- 172.21.15.7:0/1418723489 >> v2:172.21.15.188:3300/0 conn(0x7f0de80b1c00 msgr2=0x7f0de806e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.974+0000 7f0de5d9b700 1 --2- 172.21.15.7:0/1418723489 >> v2:172.21.15.188:3300/0 conn(0x7f0de80b1c00 0x7f0de806e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.974+0000 7f0de5d9b700 1 -- 172.21.15.7:0/1418723489 >> v1:172.21.15.188:6789/0 conn(0x7f0de80b1fd0 legacy=0x7f0de8156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.974+0000 7f0de5d9b700 1 -- 172.21.15.7:0/1418723489 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de806f0e0 con 0x7f0de815b5f0 2024-01-27T02:33:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.974+0000 7f0de659c700 1 --2- 172.21.15.7:0/1418723489 >> v2:172.21.15.188:3300/0 conn(0x7f0de80b1c00 0x7f0de806e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:33:28.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.974+0000 7f0de5d9b700 1 --2- 172.21.15.7:0/1418723489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0de815b5f0 0x7f0de81566c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0098e0 tx=0x7f0ddc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db710a11c6d790c9 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.974+0000 7f0de559a700 1 -- 172.21.15.7:0/1418723489 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc00b640 con 0x7f0de815b5f0 2024-01-27T02:33:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.974+0000 7f0de559a700 1 -- 172.21.15.7:0/1418723489 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ddc00b7a0 con 0x7f0de815b5f0 2024-01-27T02:33:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.974+0000 7f0de559a700 1 -- 172.21.15.7:0/1418723489 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc00bb40 con 0x7f0de815b5f0 2024-01-27T02:33:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.975+0000 7f0decb97700 1 -- 172.21.15.7:0/1418723489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0de815b5f0 msgr2=0x7f0de81566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.975+0000 7f0decb97700 1 --2- 172.21.15.7:0/1418723489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0de815b5f0 0x7f0de81566c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0098e0 tx=0x7f0ddc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.975+0000 7f0decb97700 1 -- 172.21.15.7:0/1418723489 shutdown_connections 2024-01-27T02:33:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.975+0000 7f0decb97700 1 --2- 172.21.15.7:0/1418723489 >> v2:172.21.15.188:3300/0 conn(0x7f0de80b1c00 0x7f0de806e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.975+0000 7f0decb97700 1 --2- 172.21.15.7:0/1418723489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0de815b5f0 0x7f0de81566c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.975+0000 7f0decb97700 1 -- 172.21.15.7:0/1418723489 >> 172.21.15.7:0/1418723489 conn(0x7f0de8068430 msgr2=0x7f0de806be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.975+0000 7f0decb97700 1 -- 172.21.15.7:0/1418723489 shutdown_connections 2024-01-27T02:33:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.975+0000 7f0decb97700 1 -- 172.21.15.7:0/1418723489 wait complete. 2024-01-27T02:33:28.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.976+0000 7f0decb97700 1 Processor -- start 2024-01-27T02:33:28.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.976+0000 7f0decb97700 1 -- start start 2024-01-27T02:33:28.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.977+0000 7f0decb97700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0de80b1c00 0x7f0de813f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:28.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.977+0000 7f0decb97700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0de80b1fd0 0x7f0de815ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:28.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.977+0000 7f0decb97700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0de815b5f0 0x7f0de815d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:28.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.977+0000 7f0decb97700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de8151470 con 0x7f0de80b1c00 2024-01-27T02:33:28.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.977+0000 7f0decb97700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de81503d0 con 0x7f0de815b5f0 2024-01-27T02:33:28.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.977+0000 7f0decb97700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de81506b0 con 0x7f0de80b1fd0 2024-01-27T02:33:28.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.977+0000 7f0de659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0de80b1c00 0x7f0de813f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:28.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.977+0000 7f0de659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0de80b1c00 0x7f0de813f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37946/0 (socket says 172.21.15.7:37946) 2024-01-27T02:33:28.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.977+0000 7f0de659c700 1 -- 172.21.15.7:0/1599211136 learned_addr learned my addr 172.21.15.7:0/1599211136 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:28.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.977+0000 7f0de6d9d700 1 --2- 172.21.15.7:0/1599211136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0de815b5f0 0x7f0de815d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:28.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.977+0000 7f0de5d9b700 1 --2- 172.21.15.7:0/1599211136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0de80b1fd0 0x7f0de815ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:28.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.977+0000 7f0de659c700 1 -- 172.21.15.7:0/1599211136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0de80b1fd0 msgr2=0x7f0de815ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:28.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.977+0000 7f0de659c700 1 --2- 172.21.15.7:0/1599211136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0de80b1fd0 0x7f0de815ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:28.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.977+0000 7f0de659c700 1 -- 172.21.15.7:0/1599211136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0de815b5f0 msgr2=0x7f0de815d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:28.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.977+0000 7f0de659c700 1 --2- 172.21.15.7:0/1599211136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0de815b5f0 0x7f0de815d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:28.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.977+0000 7f0de659c700 1 -- 172.21.15.7:0/1599211136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd8009420 con 0x7f0de80b1c00 2024-01-27T02:33:28.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.978+0000 7f0de659c700 1 --2- 172.21.15.7:0/1599211136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0de80b1c00 0x7f0de813f5f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800ec80 tx=0x7f0dd80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:28.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.978+0000 7f0dd67fc700 1 -- 172.21.15.7:0/1599211136 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd800c920 con 0x7f0de80b1c00 2024-01-27T02:33:28.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.978+0000 7f0dd67fc700 1 -- 172.21.15.7:0/1599211136 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd800ca80 con 0x7f0de80b1c00 2024-01-27T02:33:28.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.978+0000 7f0decb97700 1 -- 172.21.15.7:0/1599211136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ddc009420 con 0x7f0de80b1c00 2024-01-27T02:33:28.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.978+0000 7f0dd67fc700 1 -- 172.21.15.7:0/1599211136 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd800ce00 con 0x7f0de80b1c00 2024-01-27T02:33:28.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.978+0000 7f0decb97700 1 -- 172.21.15.7:0/1599211136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de8150c70 con 0x7f0de80b1c00 2024-01-27T02:33:28.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.979+0000 7f0decb97700 1 -- 172.21.15.7:0/1599211136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de81512c0 con 0x7f0de80b1c00 2024-01-27T02:33:28.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.979+0000 7f0dd67fc700 1 -- 172.21.15.7:0/1599211136 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0dd8013680 con 0x7f0de80b1c00 2024-01-27T02:33:28.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.980+0000 7f0dd67fc700 1 --2- 172.21.15.7:0/1599211136 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0dc8066f30 0x7f0dc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:28.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.980+0000 7f0dd67fc700 1 -- 172.21.15.7:0/1599211136 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0dd8084930 con 0x7f0de80b1c00 2024-01-27T02:33:28.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.980+0000 7f0de5d9b700 1 --2- 172.21.15.7:0/1599211136 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0dc8066f30 0x7f0dc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:28.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.981+0000 7f0de5d9b700 1 --2- 172.21.15.7:0/1599211136 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0dc8066f30 0x7f0dc80693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0093f0 tx=0x7f0ddc017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:28.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:28.984+0000 7f0dd67fc700 1 -- 172.21.15.7:0/1599211136 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0dd8051ba0 con 0x7f0de80b1c00 2024-01-27T02:33:29.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.238+0000 7f0decb97700 1 -- 172.21.15.7:0/1599211136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0de804c710 con 0x7f0de80b1c00 2024-01-27T02:33:29.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.239+0000 7f0dd67fc700 1 -- 172.21.15.7:0/1599211136 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f0dd8017090 con 0x7f0de80b1c00 2024-01-27T02:33:29.240 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:29.240 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4883,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/198511042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":198511042},{"type":"v1","addr":"172.21.15.188:6836","nonce":198511042}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":4886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1222999254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1222999254},{"type":"v1","addr":"172.21.15.188:6837","nonce":1222999254}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2474016526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2474016526},{"type":"v1","addr":"172.21.15.7:6834","nonce":2474016526}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:33:24.950288+0000","modified":"2024-01-27T02:33:28.814599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5262":{"gid":5262,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/999728998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":999728998},{"type":"v1","addr":"172.21.15.7:6835","nonce":999728998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-27T02:33:29.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.241+0000 7f0decb97700 1 -- 172.21.15.7:0/1599211136 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0dc8066f30 msgr2=0x7f0dc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:29.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.241+0000 7f0decb97700 1 --2- 172.21.15.7:0/1599211136 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0dc8066f30 0x7f0dc80693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0093f0 tx=0x7f0ddc017000 comp rx=0 tx=0).stop 2024-01-27T02:33:29.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.241+0000 7f0decb97700 1 -- 172.21.15.7:0/1599211136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0de80b1c00 msgr2=0x7f0de813f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:29.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.241+0000 7f0decb97700 1 --2- 172.21.15.7:0/1599211136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0de80b1c00 0x7f0de813f5f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800ec80 tx=0x7f0dd80091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.241+0000 7f0decb97700 1 -- 172.21.15.7:0/1599211136 shutdown_connections 2024-01-27T02:33:29.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.241+0000 7f0decb97700 1 --2- 172.21.15.7:0/1599211136 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0dc8066f30 0x7f0dc80693e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.241+0000 7f0decb97700 1 --2- 172.21.15.7:0/1599211136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0de815b5f0 0x7f0de815d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.241+0000 7f0decb97700 1 --2- 172.21.15.7:0/1599211136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0de80b1c00 0x7f0de813f5f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.241+0000 7f0decb97700 1 --2- 172.21.15.7:0/1599211136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0de80b1fd0 0x7f0de815ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.241+0000 7f0decb97700 1 -- 172.21.15.7:0/1599211136 >> 172.21.15.7:0/1599211136 conn(0x7f0de8068430 msgr2=0x7f0de80cded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:29.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.241+0000 7f0decb97700 1 -- 172.21.15.7:0/1599211136 shutdown_connections 2024-01-27T02:33:29.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.241+0000 7f0decb97700 1 -- 172.21.15.7:0/1599211136 wait complete. 2024-01-27T02:33:29.243 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-01-27T02:33:29.256 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:33:29.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.387+0000 7f1dcf3db700 1 Processor -- start 2024-01-27T02:33:29.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.388+0000 7f1dcf3db700 1 -- start start 2024-01-27T02:33:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.388+0000 7f1dcf3db700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1dc8159f40 0x7f1dc80c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.389+0000 7f1dcf3db700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc80c30c0 con 0x7f1dc80c3a90 2024-01-27T02:33:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.389+0000 7f1dcf3db700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc815e960 con 0x7f1dc80aafc0 2024-01-27T02:33:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.389+0000 7f1dcf3db700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc80c4550 con 0x7f1dc8159f40 2024-01-27T02:33:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.389+0000 7f1dcd177700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1dc8159f40 0x7f1dc80c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.389+0000 7f1dcc976700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1dc80c3a90 0x7f1dc80aa8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44716/0 (socket says 172.21.15.7:44716) 2024-01-27T02:33:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.389+0000 7f1dcc976700 1 -- 172.21.15.7:0/2936607762 learned_addr learned my addr 172.21.15.7:0/2936607762 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.389+0000 7f1dcd177700 1 -- 172.21.15.7:0/2936607762 >> v1:172.21.15.188:6789/0 conn(0x7f1dc80aafc0 legacy=0x7f1dc8063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.389+0000 7f1dcd177700 1 -- 172.21.15.7:0/2936607762 >> v1:172.21.15.7:6789/0 conn(0x7f1dc80c3a90 legacy=0x7f1dc80aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:29.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.389+0000 7f1dcd177700 1 -- 172.21.15.7:0/2936607762 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc815ec40 con 0x7f1dc8159f40 2024-01-27T02:33:29.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.389+0000 7f1dcd177700 1 --2- 172.21.15.7:0/2936607762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc8159f40 0x7f1dc80c3550 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40098e0 tx=0x7f1dc40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92bfc9b50faa4f64 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:29.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.389+0000 7f1dbffff700 1 -- 172.21.15.7:0/2936607762 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4009e80 con 0x7f1dc8159f40 2024-01-27T02:33:29.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.390+0000 7f1dbffff700 1 -- 172.21.15.7:0/2936607762 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc400b760 con 0x7f1dc8159f40 2024-01-27T02:33:29.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.390+0000 7f1dbffff700 1 -- 172.21.15.7:0/2936607762 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc400bae0 con 0x7f1dc8159f40 2024-01-27T02:33:29.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.390+0000 7f1dcf3db700 1 -- 172.21.15.7:0/2936607762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc8159f40 msgr2=0x7f1dc80c3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:29.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.390+0000 7f1dcf3db700 1 --2- 172.21.15.7:0/2936607762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc8159f40 0x7f1dc80c3550 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40098e0 tx=0x7f1dc40091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.390+0000 7f1dcf3db700 1 -- 172.21.15.7:0/2936607762 shutdown_connections 2024-01-27T02:33:29.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.390+0000 7f1dcf3db700 1 --2- 172.21.15.7:0/2936607762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc8159f40 0x7f1dc80c3550 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.390+0000 7f1dcf3db700 1 -- 172.21.15.7:0/2936607762 >> 172.21.15.7:0/2936607762 conn(0x7f1dc80cdc30 msgr2=0x7f1dc80ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:29.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.390+0000 7f1dcf3db700 1 -- 172.21.15.7:0/2936607762 shutdown_connections 2024-01-27T02:33:29.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.390+0000 7f1dcf3db700 1 -- 172.21.15.7:0/2936607762 wait complete. 2024-01-27T02:33:29.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.391+0000 7f1dcf3db700 1 Processor -- start 2024-01-27T02:33:29.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.392+0000 7f1dcf3db700 1 -- start start 2024-01-27T02:33:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.392+0000 7f1dcf3db700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1dc80aafc0 0x7f1dc80b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.392+0000 7f1dcf3db700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc80c3a90 0x7f1dc815cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.392+0000 7f1dcf3db700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1dc8159f40 0x7f1dc816df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.393+0000 7f1dcf3db700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc80b2930 con 0x7f1dc80c3a90 2024-01-27T02:33:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.393+0000 7f1dcf3db700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc80b2360 con 0x7f1dc80aafc0 2024-01-27T02:33:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.393+0000 7f1dcf3db700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc80b3300 con 0x7f1dc8159f40 2024-01-27T02:33:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.393+0000 7f1dcc976700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc80c3a90 0x7f1dc815cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.393+0000 7f1dcc976700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc80c3a90 0x7f1dc815cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37976/0 (socket says 172.21.15.7:37976) 2024-01-27T02:33:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.393+0000 7f1dcc976700 1 -- 172.21.15.7:0/2456773187 learned_addr learned my addr 172.21.15.7:0/2456773187 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.393+0000 7f1dcd177700 1 --2- 172.21.15.7:0/2456773187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1dc80aafc0 0x7f1dc80b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.393+0000 7f1dcd978700 1 --2- 172.21.15.7:0/2456773187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1dc8159f40 0x7f1dc816df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.393+0000 7f1dcc976700 1 -- 172.21.15.7:0/2456773187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1dc8159f40 msgr2=0x7f1dc816df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.393+0000 7f1dcc976700 1 --2- 172.21.15.7:0/2456773187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1dc8159f40 0x7f1dc816df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.393+0000 7f1dcc976700 1 -- 172.21.15.7:0/2456773187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1dc80aafc0 msgr2=0x7f1dc80b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.393+0000 7f1dcc976700 1 --2- 172.21.15.7:0/2456773187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1dc80aafc0 0x7f1dc80b2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.393+0000 7f1dcc976700 1 -- 172.21.15.7:0/2456773187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc4009420 con 0x7f1dc80c3a90 2024-01-27T02:33:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.393+0000 7f1dcc976700 1 --2- 172.21.15.7:0/2456773187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc80c3a90 0x7f1dc815cd10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1db800c7e0 tx=0x7f1db8007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.393+0000 7f1dbd7fa700 1 -- 172.21.15.7:0/2456773187 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db8005b30 con 0x7f1dc80c3a90 2024-01-27T02:33:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.393+0000 7f1dbd7fa700 1 -- 172.21.15.7:0/2456773187 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db8005c90 con 0x7f1dc80c3a90 2024-01-27T02:33:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.393+0000 7f1dcf3db700 1 -- 172.21.15.7:0/2456773187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc80c7a80 con 0x7f1dc80c3a90 2024-01-27T02:33:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.393+0000 7f1dbd7fa700 1 -- 172.21.15.7:0/2456773187 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db8007300 con 0x7f1dc80c3a90 2024-01-27T02:33:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.393+0000 7f1dcf3db700 1 -- 172.21.15.7:0/2456773187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc80b25c0 con 0x7f1dc80c3a90 2024-01-27T02:33:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.395+0000 7f1dcf3db700 1 -- 172.21.15.7:0/2456773187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc804c710 con 0x7f1dc80c3a90 2024-01-27T02:33:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.395+0000 7f1dbd7fa700 1 -- 172.21.15.7:0/2456773187 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1db801b3b0 con 0x7f1dc80c3a90 2024-01-27T02:33:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.395+0000 7f1dbd7fa700 1 --2- 172.21.15.7:0/2456773187 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1dac066f30 0x7f1dac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.396+0000 7f1dbd7fa700 1 -- 172.21.15.7:0/2456773187 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1db8083a80 con 0x7f1dc80c3a90 2024-01-27T02:33:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.396+0000 7f1dcd177700 1 --2- 172.21.15.7:0/2456773187 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1dac066f30 0x7f1dac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.396+0000 7f1dcd177700 1 --2- 172.21.15.7:0/2456773187 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1dac066f30 0x7f1dac0693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40030c0 tx=0x7f1dc4016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.400+0000 7f1dbd7fa700 1 -- 172.21.15.7:0/2456773187 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1db8050cf0 con 0x7f1dc80c3a90 2024-01-27T02:33:29.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.605+0000 7f1dcf3db700 1 -- 172.21.15.7:0/2456773187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1dc80b2a70 con 0x7f1dc80c3a90 2024-01-27T02:33:29.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.606+0000 7f1dbd7fa700 1 -- 172.21.15.7:0/2456773187 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f1db8017020 con 0x7f1dc80c3a90 2024-01-27T02:33:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:29.608 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":45,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:33:26.519256+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:33:23.445643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-27T02:33:23.921995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:33:29.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.608+0000 7f1dcf3db700 1 -- 172.21.15.7:0/2456773187 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1dac066f30 msgr2=0x7f1dac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:29.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.608+0000 7f1dcf3db700 1 --2- 172.21.15.7:0/2456773187 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1dac066f30 0x7f1dac0693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40030c0 tx=0x7f1dc4016000 comp rx=0 tx=0).stop 2024-01-27T02:33:29.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.608+0000 7f1dcf3db700 1 -- 172.21.15.7:0/2456773187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc80c3a90 msgr2=0x7f1dc815cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:29.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.608+0000 7f1dcf3db700 1 --2- 172.21.15.7:0/2456773187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc80c3a90 0x7f1dc815cd10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1db800c7e0 tx=0x7f1db8007f00 comp rx=0 tx=0).stop 2024-01-27T02:33:29.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.608+0000 7f1dcf3db700 1 -- 172.21.15.7:0/2456773187 shutdown_connections 2024-01-27T02:33:29.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.608+0000 7f1dcf3db700 1 --2- 172.21.15.7:0/2456773187 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1dac066f30 0x7f1dac0693e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.608+0000 7f1dcf3db700 1 --2- 172.21.15.7:0/2456773187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1dc8159f40 0x7f1dc816df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.608+0000 7f1dcf3db700 1 --2- 172.21.15.7:0/2456773187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1dc80aafc0 0x7f1dc80b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.608+0000 7f1dcf3db700 1 --2- 172.21.15.7:0/2456773187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc80c3a90 0x7f1dc815cd10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.608+0000 7f1dcf3db700 1 -- 172.21.15.7:0/2456773187 >> 172.21.15.7:0/2456773187 conn(0x7f1dc80cdc30 msgr2=0x7f1dc80aa8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:29.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.608+0000 7f1dcf3db700 1 -- 172.21.15.7:0/2456773187 shutdown_connections 2024-01-27T02:33:29.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.608+0000 7f1dcf3db700 1 -- 172.21.15.7:0/2456773187 wait complete. 2024-01-27T02:33:29.623 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-27T02:33:29.624 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:33:29.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.754+0000 7f9cd3d2d700 1 Processor -- start 2024-01-27T02:33:29.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.755+0000 7f9cd3d2d700 1 -- start start 2024-01-27T02:33:29.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.755+0000 7f9cd3d2d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9ccc0c7520 0x7f9ccc0c1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:29.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.755+0000 7f9cd3d2d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9ccc0c78f0 0x7f9ccc0c1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:29.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.755+0000 7f9cd3d2d700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ccc0c2380 con 0x7f9ccc0c94f0 2024-01-27T02:33:29.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.755+0000 7f9cd3d2d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ccc0c2630 con 0x7f9ccc0c78f0 2024-01-27T02:33:29.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.755+0000 7f9cd3d2d700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ccc0c2910 con 0x7f9ccc0c7520 2024-01-27T02:33:29.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.756+0000 7f9cd12c8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9ccc0c78f0 0x7f9ccc0c1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:29.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.756+0000 7f9cd12c8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9ccc0c78f0 0x7f9ccc0c1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37982/0 (socket says 172.21.15.7:37982) 2024-01-27T02:33:29.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.756+0000 7f9cd12c8700 1 -- 172.21.15.7:0/4217405745 learned_addr learned my addr 172.21.15.7:0/4217405745 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:29.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.756+0000 7f9cd1ac9700 1 --2- 172.21.15.7:0/4217405745 >> v2:172.21.15.188:3300/0 conn(0x7f9ccc0c7520 0x7f9ccc0c1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:29.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.756+0000 7f9cd12c8700 1 -- 172.21.15.7:0/4217405745 >> v2:172.21.15.188:3300/0 conn(0x7f9ccc0c7520 msgr2=0x7f9ccc0c1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:29.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.756+0000 7f9cd12c8700 1 --2- 172.21.15.7:0/4217405745 >> v2:172.21.15.188:3300/0 conn(0x7f9ccc0c7520 0x7f9ccc0c1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.756+0000 7f9cd12c8700 1 -- 172.21.15.7:0/4217405745 >> v1:172.21.15.188:6789/0 conn(0x7f9ccc0c94f0 legacy=0x7f9ccc0ab4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:29.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.756+0000 7f9cd12c8700 1 -- 172.21.15.7:0/4217405745 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ccc0c2bf0 con 0x7f9ccc0c78f0 2024-01-27T02:33:29.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.756+0000 7f9cd12c8700 1 --2- 172.21.15.7:0/4217405745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc0c78f0 0x7f9ccc0c1e40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc06c1d0 tx=0x7f9cc8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349c181c95764700 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:29.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.756+0000 7f9cd0ac7700 1 -- 172.21.15.7:0/4217405745 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8009e70 con 0x7f9ccc0c78f0 2024-01-27T02:33:29.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.756+0000 7f9cd0ac7700 1 -- 172.21.15.7:0/4217405745 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc800b860 con 0x7f9ccc0c78f0 2024-01-27T02:33:29.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.756+0000 7f9cd0ac7700 1 -- 172.21.15.7:0/4217405745 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc800bc00 con 0x7f9ccc0c78f0 2024-01-27T02:33:29.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.757+0000 7f9cd3d2d700 1 -- 172.21.15.7:0/4217405745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc0c78f0 msgr2=0x7f9ccc0c1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:29.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.757+0000 7f9cd3d2d700 1 --2- 172.21.15.7:0/4217405745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc0c78f0 0x7f9ccc0c1e40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc06c1d0 tx=0x7f9cc8009250 comp rx=0 tx=0).stop 2024-01-27T02:33:29.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.757+0000 7f9cd3d2d700 1 -- 172.21.15.7:0/4217405745 shutdown_connections 2024-01-27T02:33:29.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.757+0000 7f9cd3d2d700 1 --2- 172.21.15.7:0/4217405745 >> v2:172.21.15.188:3300/0 conn(0x7f9ccc0c7520 0x7f9ccc0c1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.757+0000 7f9cd3d2d700 1 --2- 172.21.15.7:0/4217405745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc0c78f0 0x7f9ccc0c1e40 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.757+0000 7f9cd3d2d700 1 -- 172.21.15.7:0/4217405745 >> 172.21.15.7:0/4217405745 conn(0x7f9ccc063580 msgr2=0x7f9ccc063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:29.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.757+0000 7f9cd3d2d700 1 -- 172.21.15.7:0/4217405745 shutdown_connections 2024-01-27T02:33:29.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.757+0000 7f9cd3d2d700 1 -- 172.21.15.7:0/4217405745 wait complete. 2024-01-27T02:33:29.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.759+0000 7f9cd3d2d700 1 Processor -- start 2024-01-27T02:33:29.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.759+0000 7f9cd3d2d700 1 -- start start 2024-01-27T02:33:29.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.760+0000 7f9cd3d2d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc0c7520 0x7f9ccc13a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:29.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.760+0000 7f9cd3d2d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ccc0c78f0 0x7f9ccc0b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:29.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.760+0000 7f9cd3d2d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ccc0c94f0 0x7f9ccc0c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:29.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.760+0000 7f9cd3d2d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc14d460 con 0x7f9ccc0c7520 2024-01-27T02:33:29.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.760+0000 7f9cd3d2d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc14e260 con 0x7f9ccc0c78f0 2024-01-27T02:33:29.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.760+0000 7f9cd3d2d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc14d650 con 0x7f9ccc0c94f0 2024-01-27T02:33:29.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.760+0000 7f9cd1ac9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc0c7520 0x7f9ccc13a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:29.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.760+0000 7f9cd1ac9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc0c7520 0x7f9ccc13a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37994/0 (socket says 172.21.15.7:37994) 2024-01-27T02:33:29.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.760+0000 7f9cd1ac9700 1 -- 172.21.15.7:0/1574248468 learned_addr learned my addr 172.21.15.7:0/1574248468 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:29.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.760+0000 7f9cd12c8700 1 --2- 172.21.15.7:0/1574248468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ccc0c78f0 0x7f9ccc0b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:29.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.760+0000 7f9cd1ac9700 1 -- 172.21.15.7:0/1574248468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ccc0c94f0 msgr2=0x7f9ccc0c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:29.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.760+0000 7f9cd22ca700 1 --2- 172.21.15.7:0/1574248468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ccc0c94f0 0x7f9ccc0c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:29.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.760+0000 7f9cd1ac9700 1 --2- 172.21.15.7:0/1574248468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ccc0c94f0 0x7f9ccc0c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.760+0000 7f9cd1ac9700 1 -- 172.21.15.7:0/1574248468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ccc0c78f0 msgr2=0x7f9ccc0b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:29.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.760+0000 7f9cd1ac9700 1 --2- 172.21.15.7:0/1574248468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ccc0c78f0 0x7f9ccc0b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.760+0000 7f9cd1ac9700 1 -- 172.21.15.7:0/1574248468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc80094f0 con 0x7f9ccc0c7520 2024-01-27T02:33:29.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.760+0000 7f9cd12c8700 1 --2- 172.21.15.7:0/1574248468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ccc0c78f0 0x7f9ccc0b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:33:29.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.760+0000 7f9cd22ca700 1 --2- 172.21.15.7:0/1574248468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ccc0c94f0 0x7f9ccc0c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:33:29.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.761+0000 7f9cd1ac9700 1 --2- 172.21.15.7:0/1574248468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc0c7520 0x7f9ccc13a720 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000e930 tx=0x7f9cc00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:29.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.761+0000 7f9cbdffb700 1 -- 172.21.15.7:0/1574248468 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc000eed0 con 0x7f9ccc0c7520 2024-01-27T02:33:29.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.761+0000 7f9cbdffb700 1 -- 172.21.15.7:0/1574248468 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc000c850 con 0x7f9ccc0c7520 2024-01-27T02:33:29.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.761+0000 7f9cd3d2d700 1 -- 172.21.15.7:0/1574248468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ccc14d930 con 0x7f9ccc0c7520 2024-01-27T02:33:29.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.761+0000 7f9cbdffb700 1 -- 172.21.15.7:0/1574248468 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc000cbf0 con 0x7f9ccc0c7520 2024-01-27T02:33:29.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.761+0000 7f9cd3d2d700 1 -- 172.21.15.7:0/1574248468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ccc14def0 con 0x7f9ccc0c7520 2024-01-27T02:33:29.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.762+0000 7f9cd3d2d700 1 -- 172.21.15.7:0/1574248468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ccc04c710 con 0x7f9ccc0c7520 2024-01-27T02:33:29.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.762+0000 7f9cbdffb700 1 -- 172.21.15.7:0/1574248468 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9cc0013470 con 0x7f9ccc0c7520 2024-01-27T02:33:29.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.763+0000 7f9cbdffb700 1 --2- 172.21.15.7:0/1574248468 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9cb0066f30 0x7f9cb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:29.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.763+0000 7f9cbdffb700 1 -- 172.21.15.7:0/1574248468 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9cc00847a0 con 0x7f9ccc0c7520 2024-01-27T02:33:29.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.763+0000 7f9cd12c8700 1 --2- 172.21.15.7:0/1574248468 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9cb0066f30 0x7f9cb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:29.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.764+0000 7f9cd12c8700 1 --2- 172.21.15.7:0/1574248468 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9cb0066f30 0x7f9cb00693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc06c1d0 tx=0x7f9cc8017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:29.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.768+0000 7f9cbdffb700 1 -- 172.21.15.7:0/1574248468 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9cc0051a10 con 0x7f9ccc0c7520 2024-01-27T02:33:29.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.965+0000 7f9cd3d2d700 1 -- 172.21.15.7:0/1574248468 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ccc14e3f0 con 0x7f9cb0066f30 2024-01-27T02:33:29.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.974+0000 7f9cbdffb700 1 -- 172.21.15.7:0/1574248468 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285368 (secure 0 0 0) 0x7f9ccc14e3f0 con 0x7f9cb0066f30 2024-01-27T02:33:29.979 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:29.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.978+0000 7f9cd3d2d700 1 -- 172.21.15.7:0/1574248468 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9cb0066f30 msgr2=0x7f9cb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:29.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.978+0000 7f9cd3d2d700 1 --2- 172.21.15.7:0/1574248468 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9cb0066f30 0x7f9cb00693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc06c1d0 tx=0x7f9cc8017000 comp rx=0 tx=0).stop 2024-01-27T02:33:29.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.978+0000 7f9cd3d2d700 1 -- 172.21.15.7:0/1574248468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc0c7520 msgr2=0x7f9ccc13a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:29.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.978+0000 7f9cd3d2d700 1 --2- 172.21.15.7:0/1574248468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc0c7520 0x7f9ccc13a720 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000e930 tx=0x7f9cc00091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.978+0000 7f9cd3d2d700 1 -- 172.21.15.7:0/1574248468 shutdown_connections 2024-01-27T02:33:29.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.978+0000 7f9cd3d2d700 1 --2- 172.21.15.7:0/1574248468 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9cb0066f30 0x7f9cb00693e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.978+0000 7f9cd3d2d700 1 --2- 172.21.15.7:0/1574248468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ccc0c94f0 0x7f9ccc0c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.978+0000 7f9cd3d2d700 1 --2- 172.21.15.7:0/1574248468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc0c7520 0x7f9ccc13a720 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.978+0000 7f9cd3d2d700 1 --2- 172.21.15.7:0/1574248468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ccc0c78f0 0x7f9ccc0b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:29.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.978+0000 7f9cd3d2d700 1 -- 172.21.15.7:0/1574248468 >> 172.21.15.7:0/1574248468 conn(0x7f9ccc063580 msgr2=0x7f9ccc06a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:29.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.978+0000 7f9cd3d2d700 1 -- 172.21.15.7:0/1574248468 shutdown_connections 2024-01-27T02:33:29.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:29.978+0000 7f9cd3d2d700 1 -- 172.21.15.7:0/1574248468 wait complete. 2024-01-27T02:33:29.982 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:33:29.993 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-01-27T02:33:29.591925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2771,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559728,"statfs":{"total":1600000000,"available":1597161978,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001500"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.532064+0000","last_change":"2024-01-27T02:33:25.532064+0000","last_active":"2024-01-27T02:33:25.532064+0000","last_peered":"2024-01-27T02:33:25.532064+0000","last_clean":"2024-01-27T02:33:25.532064+0000","last_became_active":"2024-01-27T02:33:25.531475+0000","last_became_peered":"2024-01-27T02:33:25.531475+0000","last_unstale":"2024-01-27T02:33:25.532064+0000","last_undegraded":"2024-01-27T02:33:25.532064+0000","last_fullsized":"2024-01-27T02:33:25.532064+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:00:05.977370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537241+0000","last_change":"2024-01-27T02:33:25.537241+0000","last_active":"2024-01-27T02:33:25.537241+0000","last_peered":"2024-01-27T02:33:25.537241+0000","last_clean":"2024-01-27T02:33:25.537241+0000","last_became_active":"2024-01-27T02:33:25.537004+0000","last_became_peered":"2024-01-27T02:33:25.537004+0000","last_unstale":"2024-01-27T02:33:25.537241+0000","last_undegraded":"2024-01-27T02:33:25.537241+0000","last_fullsized":"2024-01-27T02:33:25.537241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:00:05.977370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.543304+0000","last_change":"2024-01-27T02:33:25.543236+0000","last_active":"2024-01-27T02:33:25.543304+0000","last_peered":"2024-01-27T02:33:25.543304+0000","last_clean":"2024-01-27T02:33:25.543304+0000","last_became_active":"2024-01-27T02:33:25.543040+0000","last_became_peered":"2024-01-27T02:33:25.543040+0000","last_unstale":"2024-01-27T02:33:25.543304+0000","last_undegraded":"2024-01-27T02:33:25.543304+0000","last_fullsized":"2024-01-27T02:33:25.543304+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:30:47.765282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.533729+0000","last_change":"2024-01-27T02:33:25.533729+0000","last_active":"2024-01-27T02:33:25.533729+0000","last_peered":"2024-01-27T02:33:25.533729+0000","last_clean":"2024-01-27T02:33:25.533729+0000","last_became_active":"2024-01-27T02:33:25.533415+0000","last_became_peered":"2024-01-27T02:33:25.533415+0000","last_unstale":"2024-01-27T02:33:25.533729+0000","last_undegraded":"2024-01-27T02:33:25.533729+0000","last_fullsized":"2024-01-27T02:33:25.533729+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:02:17.320801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.534561+0000","last_change":"2024-01-27T02:33:25.534561+0000","last_active":"2024-01-27T02:33:25.534561+0000","last_peered":"2024-01-27T02:33:25.534561+0000","last_clean":"2024-01-27T02:33:25.534561+0000","last_became_active":"2024-01-27T02:33:25.534344+0000","last_became_peered":"2024-01-27T02:33:25.534344+0000","last_unstale":"2024-01-27T02:33:25.534561+0000","last_undegraded":"2024-01-27T02:33:25.534561+0000","last_fullsized":"2024-01-27T02:33:25.534561+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:29:58.352940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.524145+0000","last_change":"2024-01-27T02:33:25.524101+0000","last_active":"2024-01-27T02:33:25.524145+0000","last_peered":"2024-01-27T02:33:25.524145+0000","last_clean":"2024-01-27T02:33:25.524145+0000","last_became_active":"2024-01-27T02:33:25.523933+0000","last_became_peered":"2024-01-27T02:33:25.523933+0000","last_unstale":"2024-01-27T02:33:25.524145+0000","last_undegraded":"2024-01-27T02:33:25.524145+0000","last_fullsized":"2024-01-27T02:33:25.524145+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:40:49.272206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.531764+0000","last_change":"2024-01-27T02:33:25.531764+0000","last_active":"2024-01-27T02:33:25.531764+0000","last_peered":"2024-01-27T02:33:25.531764+0000","last_clean":"2024-01-27T02:33:25.531764+0000","last_became_active":"2024-01-27T02:33:25.531514+0000","last_became_peered":"2024-01-27T02:33:25.531514+0000","last_unstale":"2024-01-27T02:33:25.531764+0000","last_undegraded":"2024-01-27T02:33:25.531764+0000","last_fullsized":"2024-01-27T02:33:25.531764+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:02:17.320801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.532253+0000","last_change":"2024-01-27T02:33:25.532253+0000","last_active":"2024-01-27T02:33:25.532253+0000","last_peered":"2024-01-27T02:33:25.532253+0000","last_clean":"2024-01-27T02:33:25.532253+0000","last_became_active":"2024-01-27T02:33:25.531784+0000","last_became_peered":"2024-01-27T02:33:25.531784+0000","last_unstale":"2024-01-27T02:33:25.532253+0000","last_undegraded":"2024-01-27T02:33:25.532253+0000","last_fullsized":"2024-01-27T02:33:25.532253+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:00:05.977370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.540430+0000","last_change":"2024-01-27T02:33:25.540430+0000","last_active":"2024-01-27T02:33:25.540430+0000","last_peered":"2024-01-27T02:33:25.540430+0000","last_clean":"2024-01-27T02:33:25.540430+0000","last_became_active":"2024-01-27T02:33:25.540012+0000","last_became_peered":"2024-01-27T02:33:25.540012+0000","last_unstale":"2024-01-27T02:33:25.540430+0000","last_undegraded":"2024-01-27T02:33:25.540430+0000","last_fullsized":"2024-01-27T02:33:25.540430+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:29:58.352940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537762+0000","last_change":"2024-01-27T02:33:25.537762+0000","last_active":"2024-01-27T02:33:25.537762+0000","last_peered":"2024-01-27T02:33:25.537762+0000","last_clean":"2024-01-27T02:33:25.537762+0000","last_became_active":"2024-01-27T02:33:25.534910+0000","last_became_peered":"2024-01-27T02:33:25.534910+0000","last_unstale":"2024-01-27T02:33:25.537762+0000","last_undegraded":"2024-01-27T02:33:25.537762+0000","last_fullsized":"2024-01-27T02:33:25.537762+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:40:48.634883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.536294+0000","last_change":"2024-01-27T02:33:25.536294+0000","last_active":"2024-01-27T02:33:25.536294+0000","last_peered":"2024-01-27T02:33:25.536294+0000","last_clean":"2024-01-27T02:33:25.536294+0000","last_became_active":"2024-01-27T02:33:25.536038+0000","last_became_peered":"2024-01-27T02:33:25.536038+0000","last_unstale":"2024-01-27T02:33:25.536294+0000","last_undegraded":"2024-01-27T02:33:25.536294+0000","last_fullsized":"2024-01-27T02:33:25.536294+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:26:06.584436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.535327+0000","last_change":"2024-01-27T02:33:25.535327+0000","last_active":"2024-01-27T02:33:25.535327+0000","last_peered":"2024-01-27T02:33:25.535327+0000","last_clean":"2024-01-27T02:33:25.535327+0000","last_became_active":"2024-01-27T02:33:25.535092+0000","last_became_peered":"2024-01-27T02:33:25.535092+0000","last_unstale":"2024-01-27T02:33:25.535327+0000","last_undegraded":"2024-01-27T02:33:25.535327+0000","last_fullsized":"2024-01-27T02:33:25.535327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:35:50.150354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.540782+0000","last_change":"2024-01-27T02:33:25.540782+0000","last_active":"2024-01-27T02:33:25.540782+0000","last_peered":"2024-01-27T02:33:25.540782+0000","last_clean":"2024-01-27T02:33:25.540782+0000","last_became_active":"2024-01-27T02:33:25.536407+0000","last_became_peered":"2024-01-27T02:33:25.536407+0000","last_unstale":"2024-01-27T02:33:25.540782+0000","last_undegraded":"2024-01-27T02:33:25.540782+0000","last_fullsized":"2024-01-27T02:33:25.540782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:26:02.647398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.540262+0000","last_change":"2024-01-27T02:33:25.540262+0000","last_active":"2024-01-27T02:33:25.540262+0000","last_peered":"2024-01-27T02:33:25.540262+0000","last_clean":"2024-01-27T02:33:25.540262+0000","last_became_active":"2024-01-27T02:33:25.539622+0000","last_became_peered":"2024-01-27T02:33:25.539622+0000","last_unstale":"2024-01-27T02:33:25.540262+0000","last_undegraded":"2024-01-27T02:33:25.540262+0000","last_fullsized":"2024-01-27T02:33:25.540262+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:05:53.123337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537289+0000","last_change":"2024-01-27T02:33:25.536568+0000","last_active":"2024-01-27T02:33:25.537289+0000","last_peered":"2024-01-27T02:33:25.537289+0000","last_clean":"2024-01-27T02:33:25.537289+0000","last_became_active":"2024-01-27T02:33:25.536316+0000","last_became_peered":"2024-01-27T02:33:25.536316+0000","last_unstale":"2024-01-27T02:33:25.537289+0000","last_undegraded":"2024-01-27T02:33:25.537289+0000","last_fullsized":"2024-01-27T02:33:25.537289+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:36:49.947070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.530270+0000","last_change":"2024-01-27T02:33:25.530175+0000","last_active":"2024-01-27T02:33:25.530270+0000","last_peered":"2024-01-27T02:33:25.530270+0000","last_clean":"2024-01-27T02:33:25.530270+0000","last_became_active":"2024-01-27T02:33:25.529759+0000","last_became_peered":"2024-01-27T02:33:25.529759+0000","last_unstale":"2024-01-27T02:33:25.530270+0000","last_undegraded":"2024-01-27T02:33:25.530270+0000","last_fullsized":"2024-01-27T02:33:25.530270+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:23:23.599278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.531503+0000","last_change":"2024-01-27T02:33:25.531405+0000","last_active":"2024-01-27T02:33:25.531503+0000","last_peered":"2024-01-27T02:33:25.531503+0000","last_clean":"2024-01-27T02:33:25.531503+0000","last_became_active":"2024-01-27T02:33:25.531024+0000","last_became_peered":"2024-01-27T02:33:25.531024+0000","last_unstale":"2024-01-27T02:33:25.531503+0000","last_undegraded":"2024-01-27T02:33:25.531503+0000","last_fullsized":"2024-01-27T02:33:25.531503+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:18:21.974137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.530928+0000","last_change":"2024-01-27T02:33:25.530865+0000","last_active":"2024-01-27T02:33:25.530928+0000","last_peered":"2024-01-27T02:33:25.530928+0000","last_clean":"2024-01-27T02:33:25.530928+0000","last_became_active":"2024-01-27T02:33:25.530601+0000","last_became_peered":"2024-01-27T02:33:25.530601+0000","last_unstale":"2024-01-27T02:33:25.530928+0000","last_undegraded":"2024-01-27T02:33:25.530928+0000","last_fullsized":"2024-01-27T02:33:25.530928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:13:06.294428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.533899+0000","last_change":"2024-01-27T02:33:25.533823+0000","last_active":"2024-01-27T02:33:25.533899+0000","last_peered":"2024-01-27T02:33:25.533899+0000","last_clean":"2024-01-27T02:33:25.533899+0000","last_became_active":"2024-01-27T02:33:25.533551+0000","last_became_peered":"2024-01-27T02:33:25.533551+0000","last_unstale":"2024-01-27T02:33:25.533899+0000","last_undegraded":"2024-01-27T02:33:25.533899+0000","last_fullsized":"2024-01-27T02:33:25.533899+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:02:44.519562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.541102+0000","last_change":"2024-01-27T02:33:25.541102+0000","last_active":"2024-01-27T02:33:25.541102+0000","last_peered":"2024-01-27T02:33:25.541102+0000","last_clean":"2024-01-27T02:33:25.541102+0000","last_became_active":"2024-01-27T02:33:25.540798+0000","last_became_peered":"2024-01-27T02:33:25.540798+0000","last_unstale":"2024-01-27T02:33:25.541102+0000","last_undegraded":"2024-01-27T02:33:25.541102+0000","last_fullsized":"2024-01-27T02:33:25.541102+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:24:20.892845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.523839+0000","last_change":"2024-01-27T02:33:25.523778+0000","last_active":"2024-01-27T02:33:25.523839+0000","last_peered":"2024-01-27T02:33:25.523839+0000","last_clean":"2024-01-27T02:33:25.523839+0000","last_became_active":"2024-01-27T02:33:25.523554+0000","last_became_peered":"2024-01-27T02:33:25.523554+0000","last_unstale":"2024-01-27T02:33:25.523839+0000","last_undegraded":"2024-01-27T02:33:25.523839+0000","last_fullsized":"2024-01-27T02:33:25.523839+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:39:33.075890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537347+0000","last_change":"2024-01-27T02:33:25.537347+0000","last_active":"2024-01-27T02:33:25.537347+0000","last_peered":"2024-01-27T02:33:25.537347+0000","last_clean":"2024-01-27T02:33:25.537347+0000","last_became_active":"2024-01-27T02:33:25.536341+0000","last_became_peered":"2024-01-27T02:33:25.536341+0000","last_unstale":"2024-01-27T02:33:25.537347+0000","last_undegraded":"2024-01-27T02:33:25.537347+0000","last_fullsized":"2024-01-27T02:33:25.537347+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:30:47.765282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.525026+0000","last_change":"2024-01-27T02:33:25.524952+0000","last_active":"2024-01-27T02:33:25.525026+0000","last_peered":"2024-01-27T02:33:25.525026+0000","last_clean":"2024-01-27T02:33:25.525026+0000","last_became_active":"2024-01-27T02:33:25.524721+0000","last_became_peered":"2024-01-27T02:33:25.524721+0000","last_unstale":"2024-01-27T02:33:25.525026+0000","last_undegraded":"2024-01-27T02:33:25.525026+0000","last_fullsized":"2024-01-27T02:33:25.525026+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:26:06.584436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.525413+0000","last_change":"2024-01-27T02:33:25.525335+0000","last_active":"2024-01-27T02:33:25.525413+0000","last_peered":"2024-01-27T02:33:25.525413+0000","last_clean":"2024-01-27T02:33:25.525413+0000","last_became_active":"2024-01-27T02:33:25.524885+0000","last_became_peered":"2024-01-27T02:33:25.524885+0000","last_unstale":"2024-01-27T02:33:25.525413+0000","last_undegraded":"2024-01-27T02:33:25.525413+0000","last_fullsized":"2024-01-27T02:33:25.525413+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:32:41.154409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.535317+0000","last_change":"2024-01-27T02:33:25.535199+0000","last_active":"2024-01-27T02:33:25.535317+0000","last_peered":"2024-01-27T02:33:25.535317+0000","last_clean":"2024-01-27T02:33:25.535317+0000","last_became_active":"2024-01-27T02:33:25.534928+0000","last_became_peered":"2024-01-27T02:33:25.534928+0000","last_unstale":"2024-01-27T02:33:25.535317+0000","last_undegraded":"2024-01-27T02:33:25.535317+0000","last_fullsized":"2024-01-27T02:33:25.535317+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:11:01.017228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.535285+0000","last_change":"2024-01-27T02:33:25.535285+0000","last_active":"2024-01-27T02:33:25.535285+0000","last_peered":"2024-01-27T02:33:25.535285+0000","last_clean":"2024-01-27T02:33:25.535285+0000","last_became_active":"2024-01-27T02:33:25.534119+0000","last_became_peered":"2024-01-27T02:33:25.534119+0000","last_unstale":"2024-01-27T02:33:25.535285+0000","last_undegraded":"2024-01-27T02:33:25.535285+0000","last_fullsized":"2024-01-27T02:33:25.535285+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:03:32.136426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.531483+0000","last_change":"2024-01-27T02:33:25.531258+0000","last_active":"2024-01-27T02:33:25.531483+0000","last_peered":"2024-01-27T02:33:25.531483+0000","last_clean":"2024-01-27T02:33:25.531483+0000","last_became_active":"2024-01-27T02:33:25.530710+0000","last_became_peered":"2024-01-27T02:33:25.530710+0000","last_unstale":"2024-01-27T02:33:25.531483+0000","last_undegraded":"2024-01-27T02:33:25.531483+0000","last_fullsized":"2024-01-27T02:33:25.531483+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:36:14.220063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537083+0000","last_change":"2024-01-27T02:33:25.537083+0000","last_active":"2024-01-27T02:33:25.537083+0000","last_peered":"2024-01-27T02:33:25.537083+0000","last_clean":"2024-01-27T02:33:25.537083+0000","last_became_active":"2024-01-27T02:33:25.536841+0000","last_became_peered":"2024-01-27T02:33:25.536841+0000","last_unstale":"2024-01-27T02:33:25.537083+0000","last_undegraded":"2024-01-27T02:33:25.537083+0000","last_fullsized":"2024-01-27T02:33:25.537083+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:38:34.204456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.535334+0000","last_change":"2024-01-27T02:33:25.535223+0000","last_active":"2024-01-27T02:33:25.535334+0000","last_peered":"2024-01-27T02:33:25.535334+0000","last_clean":"2024-01-27T02:33:25.535334+0000","last_became_active":"2024-01-27T02:33:25.534954+0000","last_became_peered":"2024-01-27T02:33:25.534954+0000","last_unstale":"2024-01-27T02:33:25.535334+0000","last_undegraded":"2024-01-27T02:33:25.535334+0000","last_fullsized":"2024-01-27T02:33:25.535334+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:37:56.187364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.530210+0000","last_change":"2024-01-27T02:33:25.529298+0000","last_active":"2024-01-27T02:33:25.530210+0000","last_peered":"2024-01-27T02:33:25.530210+0000","last_clean":"2024-01-27T02:33:25.530210+0000","last_became_active":"2024-01-27T02:33:25.528998+0000","last_became_peered":"2024-01-27T02:33:25.528998+0000","last_unstale":"2024-01-27T02:33:25.530210+0000","last_undegraded":"2024-01-27T02:33:25.530210+0000","last_fullsized":"2024-01-27T02:33:25.530210+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:06:10.291533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.532284+0000","last_change":"2024-01-27T02:33:25.532044+0000","last_active":"2024-01-27T02:33:25.532284+0000","last_peered":"2024-01-27T02:33:25.532284+0000","last_clean":"2024-01-27T02:33:25.532284+0000","last_became_active":"2024-01-27T02:33:25.531294+0000","last_became_peered":"2024-01-27T02:33:25.531294+0000","last_unstale":"2024-01-27T02:33:25.532284+0000","last_undegraded":"2024-01-27T02:33:25.532284+0000","last_fullsized":"2024-01-27T02:33:25.532284+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:15:10.938188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.541373+0000","last_change":"2024-01-27T02:33:25.541373+0000","last_active":"2024-01-27T02:33:25.541373+0000","last_peered":"2024-01-27T02:33:25.541373+0000","last_clean":"2024-01-27T02:33:25.541373+0000","last_became_active":"2024-01-27T02:33:25.540939+0000","last_became_peered":"2024-01-27T02:33:25.540939+0000","last_unstale":"2024-01-27T02:33:25.541373+0000","last_undegraded":"2024-01-27T02:33:25.541373+0000","last_fullsized":"2024-01-27T02:33:25.541373+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:50:16.570468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.532792+0000","last_change":"2024-01-27T02:33:25.532792+0000","last_active":"2024-01-27T02:33:25.532792+0000","last_peered":"2024-01-27T02:33:25.532792+0000","last_clean":"2024-01-27T02:33:25.532792+0000","last_became_active":"2024-01-27T02:33:25.532511+0000","last_became_peered":"2024-01-27T02:33:25.532511+0000","last_unstale":"2024-01-27T02:33:25.532792+0000","last_undegraded":"2024-01-27T02:33:25.532792+0000","last_fullsized":"2024-01-27T02:33:25.532792+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:34:40.708292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.817720+0000","last_change":"2024-01-27T02:33:24.531980+0000","last_active":"2024-01-27T02:33:25.817720+0000","last_peered":"2024-01-27T02:33:25.817720+0000","last_clean":"2024-01-27T02:33:25.817720+0000","last_became_active":"2024-01-27T02:33:24.531377+0000","last_became_peered":"2024-01-27T02:33:24.531377+0000","last_unstale":"2024-01-27T02:33:25.817720+0000","last_undegraded":"2024-01-27T02:33:25.817720+0000","last_fullsized":"2024-01-27T02:33:25.817720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:29:57.350543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.530203+0000","last_change":"2024-01-27T02:33:25.530117+0000","last_active":"2024-01-27T02:33:25.530203+0000","last_peered":"2024-01-27T02:33:25.530203+0000","last_clean":"2024-01-27T02:33:25.530203+0000","last_became_active":"2024-01-27T02:33:25.529807+0000","last_became_peered":"2024-01-27T02:33:25.529807+0000","last_unstale":"2024-01-27T02:33:25.530203+0000","last_undegraded":"2024-01-27T02:33:25.530203+0000","last_fullsized":"2024-01-27T02:33:25.530203+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:34:40.708292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.814415+0000","last_change":"2024-01-27T02:33:24.532094+0000","last_active":"2024-01-27T02:33:25.814415+0000","last_peered":"2024-01-27T02:33:25.814415+0000","last_clean":"2024-01-27T02:33:25.814415+0000","last_became_active":"2024-01-27T02:33:24.531848+0000","last_became_peered":"2024-01-27T02:33:24.531848+0000","last_unstale":"2024-01-27T02:33:25.814415+0000","last_undegraded":"2024-01-27T02:33:25.814415+0000","last_fullsized":"2024-01-27T02:33:25.814415+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:23:22.596881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.533818+0000","last_change":"2024-01-27T02:33:25.533818+0000","last_active":"2024-01-27T02:33:25.533818+0000","last_peered":"2024-01-27T02:33:25.533818+0000","last_clean":"2024-01-27T02:33:25.533818+0000","last_became_active":"2024-01-27T02:33:25.533583+0000","last_became_peered":"2024-01-27T02:33:25.533583+0000","last_unstale":"2024-01-27T02:33:25.533818+0000","last_undegraded":"2024-01-27T02:33:25.533818+0000","last_fullsized":"2024-01-27T02:33:25.533818+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:06:10.291533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.815739+0000","last_change":"2024-01-27T02:33:24.529476+0000","last_active":"2024-01-27T02:33:25.815739+0000","last_peered":"2024-01-27T02:33:25.815739+0000","last_clean":"2024-01-27T02:33:25.815739+0000","last_became_active":"2024-01-27T02:33:24.528796+0000","last_became_peered":"2024-01-27T02:33:24.528796+0000","last_unstale":"2024-01-27T02:33:25.815739+0000","last_undegraded":"2024-01-27T02:33:25.815739+0000","last_fullsized":"2024-01-27T02:33:25.815739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:23:22.596881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537969+0000","last_change":"2024-01-27T02:33:25.537969+0000","last_active":"2024-01-27T02:33:25.537969+0000","last_peered":"2024-01-27T02:33:25.537969+0000","last_clean":"2024-01-27T02:33:25.537969+0000","last_became_active":"2024-01-27T02:33:25.537435+0000","last_became_peered":"2024-01-27T02:33:25.537435+0000","last_unstale":"2024-01-27T02:33:25.537969+0000","last_undegraded":"2024-01-27T02:33:25.537969+0000","last_fullsized":"2024-01-27T02:33:25.537969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:32:41.154409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.815929+0000","last_change":"2024-01-27T02:33:24.531045+0000","last_active":"2024-01-27T02:33:25.815929+0000","last_peered":"2024-01-27T02:33:25.815929+0000","last_clean":"2024-01-27T02:33:25.815929+0000","last_became_active":"2024-01-27T02:33:24.530529+0000","last_became_peered":"2024-01-27T02:33:24.530529+0000","last_unstale":"2024-01-27T02:33:25.815929+0000","last_undegraded":"2024-01-27T02:33:25.815929+0000","last_fullsized":"2024-01-27T02:33:25.815929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:30:46.762885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.530192+0000","last_change":"2024-01-27T02:33:25.528896+0000","last_active":"2024-01-27T02:33:25.530192+0000","last_peered":"2024-01-27T02:33:25.530192+0000","last_clean":"2024-01-27T02:33:25.530192+0000","last_became_active":"2024-01-27T02:33:25.528515+0000","last_became_peered":"2024-01-27T02:33:25.528515+0000","last_unstale":"2024-01-27T02:33:25.530192+0000","last_undegraded":"2024-01-27T02:33:25.530192+0000","last_fullsized":"2024-01-27T02:33:25.530192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:05:53.123337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.816392+0000","last_change":"2024-01-27T02:33:24.530355+0000","last_active":"2024-01-27T02:33:25.816392+0000","last_peered":"2024-01-27T02:33:25.816392+0000","last_clean":"2024-01-27T02:33:25.816392+0000","last_became_active":"2024-01-27T02:33:24.530071+0000","last_became_peered":"2024-01-27T02:33:24.530071+0000","last_unstale":"2024-01-27T02:33:25.816392+0000","last_undegraded":"2024-01-27T02:33:25.816392+0000","last_fullsized":"2024-01-27T02:33:25.816392+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:05:52.120939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537562+0000","last_change":"2024-01-27T02:33:25.537562+0000","last_active":"2024-01-27T02:33:25.537562+0000","last_peered":"2024-01-27T02:33:25.537562+0000","last_clean":"2024-01-27T02:33:25.537562+0000","last_became_active":"2024-01-27T02:33:25.537040+0000","last_became_peered":"2024-01-27T02:33:25.537040+0000","last_unstale":"2024-01-27T02:33:25.537562+0000","last_undegraded":"2024-01-27T02:33:25.537562+0000","last_fullsized":"2024-01-27T02:33:25.537562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:06:10.291533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.814333+0000","last_change":"2024-01-27T02:33:24.527658+0000","last_active":"2024-01-27T02:33:25.814333+0000","last_peered":"2024-01-27T02:33:25.814333+0000","last_clean":"2024-01-27T02:33:25.814333+0000","last_became_active":"2024-01-27T02:33:24.527438+0000","last_became_peered":"2024-01-27T02:33:24.527438+0000","last_unstale":"2024-01-27T02:33:25.814333+0000","last_undegraded":"2024-01-27T02:33:25.814333+0000","last_fullsized":"2024-01-27T02:33:25.814333+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:11:00.014830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537400+0000","last_change":"2024-01-27T02:33:25.537400+0000","last_active":"2024-01-27T02:33:25.537400+0000","last_peered":"2024-01-27T02:33:25.537400+0000","last_clean":"2024-01-27T02:33:25.537400+0000","last_became_active":"2024-01-27T02:33:25.534594+0000","last_became_peered":"2024-01-27T02:33:25.534594+0000","last_unstale":"2024-01-27T02:33:25.537400+0000","last_undegraded":"2024-01-27T02:33:25.537400+0000","last_fullsized":"2024-01-27T02:33:25.537400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:11:20.939912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.529449+0000","last_change":"2024-01-27T02:33:24.529141+0000","last_active":"2024-01-27T02:33:24.529449+0000","last_peered":"2024-01-27T02:33:24.529449+0000","last_clean":"2024-01-27T02:33:24.529449+0000","last_became_active":"2024-01-27T02:33:24.528970+0000","last_became_peered":"2024-01-27T02:33:24.528970+0000","last_unstale":"2024-01-27T02:33:24.529449+0000","last_undegraded":"2024-01-27T02:33:24.529449+0000","last_fullsized":"2024-01-27T02:33:24.529449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:22:19.787837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.525553+0000","last_change":"2024-01-27T02:33:25.525486+0000","last_active":"2024-01-27T02:33:25.525553+0000","last_peered":"2024-01-27T02:33:25.525553+0000","last_clean":"2024-01-27T02:33:25.525553+0000","last_became_active":"2024-01-27T02:33:25.525140+0000","last_became_peered":"2024-01-27T02:33:25.525140+0000","last_unstale":"2024-01-27T02:33:25.525553+0000","last_undegraded":"2024-01-27T02:33:25.525553+0000","last_fullsized":"2024-01-27T02:33:25.525553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:54:35.139188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.527171+0000","last_change":"2024-01-27T02:33:24.525988+0000","last_active":"2024-01-27T02:33:24.527171+0000","last_peered":"2024-01-27T02:33:24.527171+0000","last_clean":"2024-01-27T02:33:24.527171+0000","last_became_active":"2024-01-27T02:33:24.525584+0000","last_became_peered":"2024-01-27T02:33:24.525584+0000","last_unstale":"2024-01-27T02:33:24.527171+0000","last_undegraded":"2024-01-27T02:33:24.527171+0000","last_fullsized":"2024-01-27T02:33:24.527171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:03:31.134029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537890+0000","last_change":"2024-01-27T02:33:25.537815+0000","last_active":"2024-01-27T02:33:25.537890+0000","last_peered":"2024-01-27T02:33:25.537890+0000","last_clean":"2024-01-27T02:33:25.537890+0000","last_became_active":"2024-01-27T02:33:25.537581+0000","last_became_peered":"2024-01-27T02:33:25.537581+0000","last_unstale":"2024-01-27T02:33:25.537890+0000","last_undegraded":"2024-01-27T02:33:25.537890+0000","last_fullsized":"2024-01-27T02:33:25.537890+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:03:50.797870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.528431+0000","last_change":"2024-01-27T02:33:24.525405+0000","last_active":"2024-01-27T02:33:25.528431+0000","last_peered":"2024-01-27T02:33:25.528431+0000","last_clean":"2024-01-27T02:33:25.528431+0000","last_became_active":"2024-01-27T02:33:24.525090+0000","last_became_peered":"2024-01-27T02:33:24.525090+0000","last_unstale":"2024-01-27T02:33:25.528431+0000","last_undegraded":"2024-01-27T02:33:25.528431+0000","last_fullsized":"2024-01-27T02:33:25.528431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:02:43.517165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.532207+0000","last_change":"2024-01-27T02:33:25.532207+0000","last_active":"2024-01-27T02:33:25.532207+0000","last_peered":"2024-01-27T02:33:25.532207+0000","last_clean":"2024-01-27T02:33:25.532207+0000","last_became_active":"2024-01-27T02:33:25.531823+0000","last_became_peered":"2024-01-27T02:33:25.531823+0000","last_unstale":"2024-01-27T02:33:25.532207+0000","last_undegraded":"2024-01-27T02:33:25.532207+0000","last_fullsized":"2024-01-27T02:33:25.532207+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:32:19.700585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.529755+0000","last_change":"2024-01-27T02:33:24.529629+0000","last_active":"2024-01-27T02:33:24.529755+0000","last_peered":"2024-01-27T02:33:24.529755+0000","last_clean":"2024-01-27T02:33:24.529755+0000","last_became_active":"2024-01-27T02:33:24.529352+0000","last_became_peered":"2024-01-27T02:33:24.529352+0000","last_unstale":"2024-01-27T02:33:24.529755+0000","last_undegraded":"2024-01-27T02:33:24.529755+0000","last_fullsized":"2024-01-27T02:33:24.529755+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:24:19.890448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537338+0000","last_change":"2024-01-27T02:33:25.537206+0000","last_active":"2024-01-27T02:33:25.537338+0000","last_peered":"2024-01-27T02:33:25.537338+0000","last_clean":"2024-01-27T02:33:25.537338+0000","last_became_active":"2024-01-27T02:33:25.536768+0000","last_became_peered":"2024-01-27T02:33:25.536768+0000","last_unstale":"2024-01-27T02:33:25.537338+0000","last_undegraded":"2024-01-27T02:33:25.537338+0000","last_fullsized":"2024-01-27T02:33:25.537338+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:37:33.304637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.816448+0000","last_change":"2024-01-27T02:33:24.532755+0000","last_active":"2024-01-27T02:33:25.816448+0000","last_peered":"2024-01-27T02:33:25.816448+0000","last_clean":"2024-01-27T02:33:25.816448+0000","last_became_active":"2024-01-27T02:33:24.532515+0000","last_became_peered":"2024-01-27T02:33:24.532515+0000","last_unstale":"2024-01-27T02:33:25.816448+0000","last_undegraded":"2024-01-27T02:33:25.816448+0000","last_fullsized":"2024-01-27T02:33:25.816448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:03:49.795473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.532055+0000","last_change":"2024-01-27T02:33:25.532055+0000","last_active":"2024-01-27T02:33:25.532055+0000","last_peered":"2024-01-27T02:33:25.532055+0000","last_clean":"2024-01-27T02:33:25.532055+0000","last_became_active":"2024-01-27T02:33:25.531570+0000","last_became_peered":"2024-01-27T02:33:25.531570+0000","last_unstale":"2024-01-27T02:33:25.532055+0000","last_undegraded":"2024-01-27T02:33:25.532055+0000","last_fullsized":"2024-01-27T02:33:25.532055+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:51:40.287635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.816998+0000","last_change":"2024-01-27T02:33:24.526912+0000","last_active":"2024-01-27T02:33:25.816998+0000","last_peered":"2024-01-27T02:33:25.816998+0000","last_clean":"2024-01-27T02:33:25.816998+0000","last_became_active":"2024-01-27T02:33:24.526500+0000","last_became_peered":"2024-01-27T02:33:24.526500+0000","last_unstale":"2024-01-27T02:33:25.816998+0000","last_undegraded":"2024-01-27T02:33:25.816998+0000","last_fullsized":"2024-01-27T02:33:25.816998+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:01:36.659231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.530185+0000","last_change":"2024-01-27T02:33:25.530010+0000","last_active":"2024-01-27T02:33:25.530185+0000","last_peered":"2024-01-27T02:33:25.530185+0000","last_clean":"2024-01-27T02:33:25.530185+0000","last_became_active":"2024-01-27T02:33:25.529375+0000","last_became_peered":"2024-01-27T02:33:25.529375+0000","last_unstale":"2024-01-27T02:33:25.530185+0000","last_undegraded":"2024-01-27T02:33:25.530185+0000","last_fullsized":"2024-01-27T02:33:25.530185+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:56:29.935410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.816390+0000","last_change":"2024-01-27T02:33:24.532027+0000","last_active":"2024-01-27T02:33:25.816390+0000","last_peered":"2024-01-27T02:33:25.816390+0000","last_clean":"2024-01-27T02:33:25.816390+0000","last_became_active":"2024-01-27T02:33:24.531751+0000","last_became_peered":"2024-01-27T02:33:24.531751+0000","last_unstale":"2024-01-27T02:33:25.816390+0000","last_undegraded":"2024-01-27T02:33:25.816390+0000","last_fullsized":"2024-01-27T02:33:25.816390+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:38:33.202059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537187+0000","last_change":"2024-01-27T02:33:25.537187+0000","last_active":"2024-01-27T02:33:25.537187+0000","last_peered":"2024-01-27T02:33:25.537187+0000","last_clean":"2024-01-27T02:33:25.537187+0000","last_became_active":"2024-01-27T02:33:25.533699+0000","last_became_peered":"2024-01-27T02:33:25.533699+0000","last_unstale":"2024-01-27T02:33:25.537187+0000","last_undegraded":"2024-01-27T02:33:25.537187+0000","last_fullsized":"2024-01-27T02:33:25.537187+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:36:14.220063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.530826+0000","last_change":"2024-01-27T02:33:24.530758+0000","last_active":"2024-01-27T02:33:24.530826+0000","last_peered":"2024-01-27T02:33:24.530826+0000","last_clean":"2024-01-27T02:33:24.530826+0000","last_became_active":"2024-01-27T02:33:24.529888+0000","last_became_peered":"2024-01-27T02:33:24.529888+0000","last_unstale":"2024-01-27T02:33:24.530826+0000","last_undegraded":"2024-01-27T02:33:24.530826+0000","last_fullsized":"2024-01-27T02:33:24.530826+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:38:33.202059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.539205+0000","last_change":"2024-01-27T02:33:25.538969+0000","last_active":"2024-01-27T02:33:25.539205+0000","last_peered":"2024-01-27T02:33:25.539205+0000","last_clean":"2024-01-27T02:33:25.539205+0000","last_became_active":"2024-01-27T02:33:25.538613+0000","last_became_peered":"2024-01-27T02:33:25.538613+0000","last_unstale":"2024-01-27T02:33:25.539205+0000","last_undegraded":"2024-01-27T02:33:25.539205+0000","last_fullsized":"2024-01-27T02:33:25.539205+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:33:01.629270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.520876+0000","last_change":"2024-01-27T02:33:24.534143+0000","last_active":"2024-01-27T02:33:25.520876+0000","last_peered":"2024-01-27T02:33:25.520876+0000","last_clean":"2024-01-27T02:33:25.520876+0000","last_became_active":"2024-01-27T02:33:24.533715+0000","last_became_peered":"2024-01-27T02:33:24.533715+0000","last_unstale":"2024-01-27T02:33:25.520876+0000","last_undegraded":"2024-01-27T02:33:25.520876+0000","last_fullsized":"2024-01-27T02:33:25.520876+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:13:05.292031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.523567+0000","last_change":"2024-01-27T02:33:25.523567+0000","last_active":"2024-01-27T02:33:25.523567+0000","last_peered":"2024-01-27T02:33:25.523567+0000","last_clean":"2024-01-27T02:33:25.523567+0000","last_became_active":"2024-01-27T02:33:25.523376+0000","last_became_peered":"2024-01-27T02:33:25.523376+0000","last_unstale":"2024-01-27T02:33:25.523567+0000","last_undegraded":"2024-01-27T02:33:25.523567+0000","last_fullsized":"2024-01-27T02:33:25.523567+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:31:58.230972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.819146+0000","last_change":"2024-01-27T02:33:24.528941+0000","last_active":"2024-01-27T02:33:25.819146+0000","last_peered":"2024-01-27T02:33:25.819146+0000","last_clean":"2024-01-27T02:33:25.819146+0000","last_became_active":"2024-01-27T02:33:24.528659+0000","last_became_peered":"2024-01-27T02:33:24.528659+0000","last_unstale":"2024-01-27T02:33:25.819146+0000","last_undegraded":"2024-01-27T02:33:25.819146+0000","last_fullsized":"2024-01-27T02:33:25.819146+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:26:01.645000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.535995+0000","last_change":"2024-01-27T02:33:25.535995+0000","last_active":"2024-01-27T02:33:25.535995+0000","last_peered":"2024-01-27T02:33:25.535995+0000","last_clean":"2024-01-27T02:33:25.535995+0000","last_became_active":"2024-01-27T02:33:25.535605+0000","last_became_peered":"2024-01-27T02:33:25.535605+0000","last_unstale":"2024-01-27T02:33:25.535995+0000","last_undegraded":"2024-01-27T02:33:25.535995+0000","last_fullsized":"2024-01-27T02:33:25.535995+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:38:22.996505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.530491+0000","last_change":"2024-01-27T02:33:24.526776+0000","last_active":"2024-01-27T02:33:25.530491+0000","last_peered":"2024-01-27T02:33:25.530491+0000","last_clean":"2024-01-27T02:33:25.530491+0000","last_became_active":"2024-01-27T02:33:24.526136+0000","last_became_peered":"2024-01-27T02:33:24.526136+0000","last_unstale":"2024-01-27T02:33:25.530491+0000","last_undegraded":"2024-01-27T02:33:25.530491+0000","last_fullsized":"2024-01-27T02:33:25.530491+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:41:00.526127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.542378+0000","last_change":"2024-01-27T02:33:25.542378+0000","last_active":"2024-01-27T02:33:25.542378+0000","last_peered":"2024-01-27T02:33:25.542378+0000","last_clean":"2024-01-27T02:33:25.542378+0000","last_became_active":"2024-01-27T02:33:25.542175+0000","last_became_peered":"2024-01-27T02:33:25.542175+0000","last_unstale":"2024-01-27T02:33:25.542378+0000","last_undegraded":"2024-01-27T02:33:25.542378+0000","last_fullsized":"2024-01-27T02:33:25.542378+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:52:44.112812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.815680+0000","last_change":"2024-01-27T02:33:24.524053+0000","last_active":"2024-01-27T02:33:25.815680+0000","last_peered":"2024-01-27T02:33:25.815680+0000","last_clean":"2024-01-27T02:33:25.815680+0000","last_became_active":"2024-01-27T02:33:24.523818+0000","last_became_peered":"2024-01-27T02:33:24.523818+0000","last_unstale":"2024-01-27T02:33:25.815680+0000","last_undegraded":"2024-01-27T02:33:25.815680+0000","last_fullsized":"2024-01-27T02:33:25.815680+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:26:01.645000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.523310+0000","last_change":"2024-01-27T02:33:25.523105+0000","last_active":"2024-01-27T02:33:25.523310+0000","last_peered":"2024-01-27T02:33:25.523310+0000","last_clean":"2024-01-27T02:33:25.523310+0000","last_became_active":"2024-01-27T02:33:25.522821+0000","last_became_peered":"2024-01-27T02:33:25.522821+0000","last_unstale":"2024-01-27T02:33:25.523310+0000","last_undegraded":"2024-01-27T02:33:25.523310+0000","last_fullsized":"2024-01-27T02:33:25.523310+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:38:52.328572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.529237+0000","last_change":"2024-01-27T02:33:24.529323+0000","last_active":"2024-01-27T02:33:25.529237+0000","last_peered":"2024-01-27T02:33:25.529237+0000","last_clean":"2024-01-27T02:33:25.529237+0000","last_became_active":"2024-01-27T02:33:24.529085+0000","last_became_peered":"2024-01-27T02:33:24.529085+0000","last_unstale":"2024-01-27T02:33:25.529237+0000","last_undegraded":"2024-01-27T02:33:25.529237+0000","last_fullsized":"2024-01-27T02:33:25.529237+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:32:40.152012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.535296+0000","last_change":"2024-01-27T02:33:25.534775+0000","last_active":"2024-01-27T02:33:25.535296+0000","last_peered":"2024-01-27T02:33:25.535296+0000","last_clean":"2024-01-27T02:33:25.535296+0000","last_became_active":"2024-01-27T02:33:25.534434+0000","last_became_peered":"2024-01-27T02:33:25.534434+0000","last_unstale":"2024-01-27T02:33:25.535296+0000","last_undegraded":"2024-01-27T02:33:25.535296+0000","last_fullsized":"2024-01-27T02:33:25.535296+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:31:18.212950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.520987+0000","last_change":"2024-01-27T02:33:24.533989+0000","last_active":"2024-01-27T02:33:25.520987+0000","last_peered":"2024-01-27T02:33:25.520987+0000","last_clean":"2024-01-27T02:33:25.520987+0000","last_became_active":"2024-01-27T02:33:24.533376+0000","last_became_peered":"2024-01-27T02:33:24.533376+0000","last_unstale":"2024-01-27T02:33:25.520987+0000","last_undegraded":"2024-01-27T02:33:25.520987+0000","last_fullsized":"2024-01-27T02:33:25.520987+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:30:46.762885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537321+0000","last_change":"2024-01-27T02:33:25.537043+0000","last_active":"2024-01-27T02:33:25.537321+0000","last_peered":"2024-01-27T02:33:25.537321+0000","last_clean":"2024-01-27T02:33:25.537321+0000","last_became_active":"2024-01-27T02:33:25.536423+0000","last_became_peered":"2024-01-27T02:33:25.536423+0000","last_unstale":"2024-01-27T02:33:25.537321+0000","last_undegraded":"2024-01-27T02:33:25.537321+0000","last_fullsized":"2024-01-27T02:33:25.537321+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:01:18.238537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.817338+0000","last_change":"2024-01-27T02:33:24.531842+0000","last_active":"2024-01-27T02:33:25.817338+0000","last_peered":"2024-01-27T02:33:25.817338+0000","last_clean":"2024-01-27T02:33:25.817338+0000","last_became_active":"2024-01-27T02:33:24.531031+0000","last_became_peered":"2024-01-27T02:33:24.531031+0000","last_unstale":"2024-01-27T02:33:25.817338+0000","last_undegraded":"2024-01-27T02:33:25.817338+0000","last_fullsized":"2024-01-27T02:33:25.817338+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:18:20.971740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.538247+0000","last_change":"2024-01-27T02:33:25.538247+0000","last_active":"2024-01-27T02:33:25.538247+0000","last_peered":"2024-01-27T02:33:25.538247+0000","last_clean":"2024-01-27T02:33:25.538247+0000","last_became_active":"2024-01-27T02:33:25.536842+0000","last_became_peered":"2024-01-27T02:33:25.536842+0000","last_unstale":"2024-01-27T02:33:25.538247+0000","last_undegraded":"2024-01-27T02:33:25.538247+0000","last_fullsized":"2024-01-27T02:33:25.538247+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:54:35.139188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.532068+0000","last_change":"2024-01-27T02:33:24.531957+0000","last_active":"2024-01-27T02:33:24.532068+0000","last_peered":"2024-01-27T02:33:24.532068+0000","last_clean":"2024-01-27T02:33:24.532068+0000","last_became_active":"2024-01-27T02:33:24.531702+0000","last_became_peered":"2024-01-27T02:33:24.531702+0000","last_unstale":"2024-01-27T02:33:24.532068+0000","last_undegraded":"2024-01-27T02:33:24.532068+0000","last_fullsized":"2024-01-27T02:33:24.532068+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:54:34.136790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.535497+0000","last_change":"2024-01-27T02:33:25.535497+0000","last_active":"2024-01-27T02:33:25.535497+0000","last_peered":"2024-01-27T02:33:25.535497+0000","last_clean":"2024-01-27T02:33:25.535497+0000","last_became_active":"2024-01-27T02:33:25.535003+0000","last_became_peered":"2024-01-27T02:33:25.535003+0000","last_unstale":"2024-01-27T02:33:25.535497+0000","last_undegraded":"2024-01-27T02:33:25.535497+0000","last_fullsized":"2024-01-27T02:33:25.535497+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:24:55.577678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.527992+0000","last_change":"2024-01-27T02:33:24.527645+0000","last_active":"2024-01-27T02:33:24.527992+0000","last_peered":"2024-01-27T02:33:24.527992+0000","last_clean":"2024-01-27T02:33:24.527992+0000","last_became_active":"2024-01-27T02:33:24.526947+0000","last_became_peered":"2024-01-27T02:33:24.526947+0000","last_unstale":"2024-01-27T02:33:24.527992+0000","last_undegraded":"2024-01-27T02:33:24.527992+0000","last_fullsized":"2024-01-27T02:33:24.527992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:38:21.994108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.539875+0000","last_change":"2024-01-27T02:33:25.539875+0000","last_active":"2024-01-27T02:33:25.539875+0000","last_peered":"2024-01-27T02:33:25.539875+0000","last_clean":"2024-01-27T02:33:25.539875+0000","last_became_active":"2024-01-27T02:33:25.539610+0000","last_became_peered":"2024-01-27T02:33:25.539610+0000","last_unstale":"2024-01-27T02:33:25.539875+0000","last_undegraded":"2024-01-27T02:33:25.539875+0000","last_fullsized":"2024-01-27T02:33:25.539875+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:18:21.974137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.531003+0000","last_change":"2024-01-27T02:33:24.530896+0000","last_active":"2024-01-27T02:33:24.531003+0000","last_peered":"2024-01-27T02:33:24.531003+0000","last_clean":"2024-01-27T02:33:24.531003+0000","last_became_active":"2024-01-27T02:33:24.530208+0000","last_became_peered":"2024-01-27T02:33:24.530208+0000","last_unstale":"2024-01-27T02:33:24.531003+0000","last_undegraded":"2024-01-27T02:33:24.531003+0000","last_fullsized":"2024-01-27T02:33:24.531003+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:36:13.217666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.531907+0000","last_change":"2024-01-27T02:33:25.531825+0000","last_active":"2024-01-27T02:33:25.531907+0000","last_peered":"2024-01-27T02:33:25.531907+0000","last_clean":"2024-01-27T02:33:25.531907+0000","last_became_active":"2024-01-27T02:33:25.531592+0000","last_became_peered":"2024-01-27T02:33:25.531592+0000","last_unstale":"2024-01-27T02:33:25.531907+0000","last_undegraded":"2024-01-27T02:33:25.531907+0000","last_fullsized":"2024-01-27T02:33:25.531907+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:12:38.630136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.520691+0000","last_change":"2024-01-27T02:33:24.525847+0000","last_active":"2024-01-27T02:33:25.520691+0000","last_peered":"2024-01-27T02:33:25.520691+0000","last_clean":"2024-01-27T02:33:25.520691+0000","last_became_active":"2024-01-27T02:33:24.525484+0000","last_became_peered":"2024-01-27T02:33:24.525484+0000","last_unstale":"2024-01-27T02:33:25.520691+0000","last_undegraded":"2024-01-27T02:33:25.520691+0000","last_fullsized":"2024-01-27T02:33:25.520691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:36:13.217666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537015+0000","last_change":"2024-01-27T02:33:25.537015+0000","last_active":"2024-01-27T02:33:25.537015+0000","last_peered":"2024-01-27T02:33:25.537015+0000","last_clean":"2024-01-27T02:33:25.537015+0000","last_became_active":"2024-01-27T02:33:25.533348+0000","last_became_peered":"2024-01-27T02:33:25.533348+0000","last_unstale":"2024-01-27T02:33:25.537015+0000","last_undegraded":"2024-01-27T02:33:25.537015+0000","last_fullsized":"2024-01-27T02:33:25.537015+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:38:34.204456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.533423+0000","last_change":"2024-01-27T02:33:24.533343+0000","last_active":"2024-01-27T02:33:24.533423+0000","last_peered":"2024-01-27T02:33:24.533423+0000","last_clean":"2024-01-27T02:33:24.533423+0000","last_became_active":"2024-01-27T02:33:24.533124+0000","last_became_peered":"2024-01-27T02:33:24.533124+0000","last_unstale":"2024-01-27T02:33:24.533423+0000","last_undegraded":"2024-01-27T02:33:24.533423+0000","last_fullsized":"2024-01-27T02:33:24.533423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:02:33.613233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.534220+0000","last_change":"2024-01-27T02:33:25.534220+0000","last_active":"2024-01-27T02:33:25.534220+0000","last_peered":"2024-01-27T02:33:25.534220+0000","last_clean":"2024-01-27T02:33:25.534220+0000","last_became_active":"2024-01-27T02:33:25.533985+0000","last_became_peered":"2024-01-27T02:33:25.533985+0000","last_unstale":"2024-01-27T02:33:25.534220+0000","last_undegraded":"2024-01-27T02:33:25.534220+0000","last_fullsized":"2024-01-27T02:33:25.534220+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:52:44.112812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.529727+0000","last_change":"2024-01-27T02:33:24.529656+0000","last_active":"2024-01-27T02:33:24.529727+0000","last_peered":"2024-01-27T02:33:24.529727+0000","last_clean":"2024-01-27T02:33:24.529727+0000","last_became_active":"2024-01-27T02:33:24.529187+0000","last_became_peered":"2024-01-27T02:33:24.529187+0000","last_unstale":"2024-01-27T02:33:24.529727+0000","last_undegraded":"2024-01-27T02:33:24.529727+0000","last_fullsized":"2024-01-27T02:33:24.529727+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:56:28.933013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.532305+0000","last_change":"2024-01-27T02:33:25.532102+0000","last_active":"2024-01-27T02:33:25.532305+0000","last_peered":"2024-01-27T02:33:25.532305+0000","last_clean":"2024-01-27T02:33:25.532305+0000","last_became_active":"2024-01-27T02:33:25.531730+0000","last_became_peered":"2024-01-27T02:33:25.531730+0000","last_unstale":"2024-01-27T02:33:25.532305+0000","last_undegraded":"2024-01-27T02:33:25.532305+0000","last_fullsized":"2024-01-27T02:33:25.532305+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:54:00.310977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.532570+0000","last_change":"2024-01-27T02:33:24.531537+0000","last_active":"2024-01-27T02:33:24.532570+0000","last_peered":"2024-01-27T02:33:24.532570+0000","last_clean":"2024-01-27T02:33:24.532570+0000","last_became_active":"2024-01-27T02:33:24.531304+0000","last_became_peered":"2024-01-27T02:33:24.531304+0000","last_unstale":"2024-01-27T02:33:24.532570+0000","last_undegraded":"2024-01-27T02:33:24.532570+0000","last_fullsized":"2024-01-27T02:33:24.532570+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:06:09.289136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.538867+0000","last_change":"2024-01-27T02:33:25.538750+0000","last_active":"2024-01-27T02:33:25.538867+0000","last_peered":"2024-01-27T02:33:25.538867+0000","last_clean":"2024-01-27T02:33:25.538867+0000","last_became_active":"2024-01-27T02:33:25.538258+0000","last_became_peered":"2024-01-27T02:33:25.538258+0000","last_unstale":"2024-01-27T02:33:25.538867+0000","last_undegraded":"2024-01-27T02:33:25.538867+0000","last_fullsized":"2024-01-27T02:33:25.538867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:41:39.374936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.528500+0000","last_change":"2024-01-27T02:33:24.521497+0000","last_active":"2024-01-27T02:33:25.528500+0000","last_peered":"2024-01-27T02:33:25.528500+0000","last_clean":"2024-01-27T02:33:25.528500+0000","last_became_active":"2024-01-27T02:33:24.521258+0000","last_became_peered":"2024-01-27T02:33:24.521258+0000","last_unstale":"2024-01-27T02:33:25.528500+0000","last_undegraded":"2024-01-27T02:33:25.528500+0000","last_fullsized":"2024-01-27T02:33:25.528500+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:56:28.933013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.539841+0000","last_change":"2024-01-27T02:33:25.539841+0000","last_active":"2024-01-27T02:33:25.539841+0000","last_peered":"2024-01-27T02:33:25.539841+0000","last_clean":"2024-01-27T02:33:25.539841+0000","last_became_active":"2024-01-27T02:33:25.539662+0000","last_became_peered":"2024-01-27T02:33:25.539662+0000","last_unstale":"2024-01-27T02:33:25.539841+0000","last_undegraded":"2024-01-27T02:33:25.539841+0000","last_fullsized":"2024-01-27T02:33:25.539841+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:11:20.939912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.533124+0000","last_change":"2024-01-27T02:33:24.533048+0000","last_active":"2024-01-27T02:33:24.533124+0000","last_peered":"2024-01-27T02:33:24.533124+0000","last_clean":"2024-01-27T02:33:24.533124+0000","last_became_active":"2024-01-27T02:33:24.532787+0000","last_became_peered":"2024-01-27T02:33:24.532787+0000","last_unstale":"2024-01-27T02:33:24.533124+0000","last_undegraded":"2024-01-27T02:33:24.533124+0000","last_fullsized":"2024-01-27T02:33:24.533124+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:12:37.627739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.539181+0000","last_change":"2024-01-27T02:33:25.539115+0000","last_active":"2024-01-27T02:33:25.539181+0000","last_peered":"2024-01-27T02:33:25.539181+0000","last_clean":"2024-01-27T02:33:25.539181+0000","last_became_active":"2024-01-27T02:33:25.538926+0000","last_became_peered":"2024-01-27T02:33:25.538926+0000","last_unstale":"2024-01-27T02:33:25.539181+0000","last_undegraded":"2024-01-27T02:33:25.539181+0000","last_fullsized":"2024-01-27T02:33:25.539181+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:38:34.204456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.529675+0000","last_change":"2024-01-27T02:33:24.529505+0000","last_active":"2024-01-27T02:33:24.529675+0000","last_peered":"2024-01-27T02:33:24.529675+0000","last_clean":"2024-01-27T02:33:24.529675+0000","last_became_active":"2024-01-27T02:33:24.528817+0000","last_became_peered":"2024-01-27T02:33:24.528817+0000","last_unstale":"2024-01-27T02:33:24.529675+0000","last_undegraded":"2024-01-27T02:33:24.529675+0000","last_fullsized":"2024-01-27T02:33:24.529675+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:50:15.568071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.529959+0000","last_change":"2024-01-27T02:33:25.529959+0000","last_active":"2024-01-27T02:33:25.529959+0000","last_peered":"2024-01-27T02:33:25.529959+0000","last_clean":"2024-01-27T02:33:25.529959+0000","last_became_active":"2024-01-27T02:33:25.529673+0000","last_became_peered":"2024-01-27T02:33:25.529673+0000","last_unstale":"2024-01-27T02:33:25.529959+0000","last_undegraded":"2024-01-27T02:33:25.529959+0000","last_fullsized":"2024-01-27T02:33:25.529959+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:13:06.294428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.817096+0000","last_change":"2024-01-27T02:33:24.528853+0000","last_active":"2024-01-27T02:33:25.817096+0000","last_peered":"2024-01-27T02:33:25.817096+0000","last_clean":"2024-01-27T02:33:25.817096+0000","last_became_active":"2024-01-27T02:33:24.528580+0000","last_became_peered":"2024-01-27T02:33:24.528580+0000","last_unstale":"2024-01-27T02:33:25.817096+0000","last_undegraded":"2024-01-27T02:33:25.817096+0000","last_fullsized":"2024-01-27T02:33:25.817096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:58:32.272587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":181,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.526836+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:33:24.526836+0000","last_peered":"2024-01-27T02:33:24.526836+0000","last_clean":"2024-01-27T02:33:24.526836+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:33:24.526836+0000","last_undegraded":"2024-01-27T02:33:24.526836+0000","last_fullsized":"2024-01-27T02:33:24.526836+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":778,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194533,"statfs":{"total":200000000,"available":199202763,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.74199999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.751},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.58199999999999996}]},{"osd":2,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.625}]},{"osd":3,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.56999999999999995}]},{"osd":4,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.43099999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.53600000000000003}]},{"osd":6,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.47899999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":202,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195109,"statfs":{"total":200000000,"available":199792577,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.51700000000000002}]},{"osd":1,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.503},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.42999999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.47799999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.46800000000000003}]},{"osd":4,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.379}]},{"osd":5,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.52600000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.36499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":202,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195110,"statfs":{"total":200000000,"available":199793151,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.376},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.41599999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.45600000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.503},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.53000000000000003}]},{"osd":4,"last update":"Sat Jan 27 02:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.78500000000000003}]},{"osd":5,"last update":"Sat Jan 27 02:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59899999999999998}]},{"osd":6,"last update":"Sat Jan 27 02:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.57599999999999996}]},{"osd":7,"last update":"Sat Jan 27 02:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.66000000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":778,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194534,"statfs":{"total":200000000,"available":199202819,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.433}]},{"osd":2,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"last":0.379},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.46999999999999997}]},{"osd":3,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.41999999999999998}]},{"osd":4,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.66900000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.747},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.80500000000000005}]},{"osd":6,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.78200000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.59799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":201,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195110,"statfs":{"total":200000000,"available":199793221,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":6.2729999999999997,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":0.41399999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.314}]},{"osd":3,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.371}]},{"osd":4,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":3.581,"5min":3.581,"15min":3.581},"last":0.57499999999999996}]},{"osd":5,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":3.5369999999999999,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.622},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.53400000000000003}]},{"osd":6,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.52700000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.59299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":202,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195109,"statfs":{"total":200000000,"available":199792611,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.69499999999999995}]},{"osd":1,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.35199999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.52300000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.70799999999999996}]},{"osd":5,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.56200000000000006}]},{"osd":6,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.53900000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.68300000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":203,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195109,"statfs":{"total":200000000,"available":199791685,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.61299999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.81499999999999995}]},{"osd":2,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.73999999999999999}]},{"osd":3,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.72099999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.66700000000000004}]},{"osd":6,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.69299999999999995}]},{"osd":7,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.76100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":202,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195110,"statfs":{"total":200000000,"available":199793151,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.50800000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.68000000000000005}]},{"osd":2,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.73099999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.66600000000000004}]},{"osd":4,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.439}]},{"osd":6,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.36699999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.45700000000000002}]}]}],"pool_statfs":[]}} 2024-01-27T02:33:29.998 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:33:30.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.166+0000 7fcc5937c700 1 Processor -- start 2024-01-27T02:33:30.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.166+0000 7fcc5937c700 1 -- start start 2024-01-27T02:33:30.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.167+0000 7fcc5937c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc540d10a0 0x7fcc540b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:30.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.167+0000 7fcc5937c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc540b4650 con 0x7fcc54159b40 2024-01-27T02:33:30.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.167+0000 7fcc5937c700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc540b3d60 con 0x7fcc540d1470 2024-01-27T02:33:30.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.167+0000 7fcc5937c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc540b34d0 con 0x7fcc540d10a0 2024-01-27T02:33:30.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.167+0000 7fcc537fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcc54159b40 0x7fcc54159f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44720/0 (socket says 172.21.15.7:44720) 2024-01-27T02:33:30.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.167+0000 7fcc537fe700 1 -- 172.21.15.7:0/22412355 learned_addr learned my addr 172.21.15.7:0/22412355 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:30.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.167+0000 7fcc52ffd700 1 --2- 172.21.15.7:0/22412355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc540d10a0 0x7fcc540b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:30.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.168+0000 7fcc51ffb700 1 -- 172.21.15.7:0/22412355 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1167678910 0 0) 0x7fcc540b4650 con 0x7fcc54159b40 2024-01-27T02:33:30.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.168+0000 7fcc51ffb700 1 -- 172.21.15.7:0/22412355 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc3c003570 con 0x7fcc54159b40 2024-01-27T02:33:30.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.168+0000 7fcc51ffb700 1 -- 172.21.15.7:0/22412355 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3367487312 0 0) 0x7fcc540b3d60 con 0x7fcc540d1470 2024-01-27T02:33:30.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.168+0000 7fcc51ffb700 1 -- 172.21.15.7:0/22412355 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc540b4650 con 0x7fcc540d1470 2024-01-27T02:33:30.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.168+0000 7fcc52ffd700 1 -- 172.21.15.7:0/22412355 >> v1:172.21.15.188:6789/0 conn(0x7fcc540d1470 legacy=0x7fcc540b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:30.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.168+0000 7fcc52ffd700 1 -- 172.21.15.7:0/22412355 >> v1:172.21.15.7:6789/0 conn(0x7fcc54159b40 legacy=0x7fcc54159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:30.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.168+0000 7fcc52ffd700 1 -- 172.21.15.7:0/22412355 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc540b4000 con 0x7fcc540d10a0 2024-01-27T02:33:30.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.168+0000 7fcc52ffd700 1 --2- 172.21.15.7:0/22412355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc540d10a0 0x7fcc540b1590 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009910 tx=0x7fcc440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eec5fcb4d58f7261 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:30.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.168+0000 7fcc51ffb700 1 -- 172.21.15.7:0/22412355 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc44011630 con 0x7fcc540d10a0 2024-01-27T02:33:30.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.168+0000 7fcc51ffb700 1 -- 172.21.15.7:0/22412355 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc44011790 con 0x7fcc540d10a0 2024-01-27T02:33:30.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.168+0000 7fcc51ffb700 1 -- 172.21.15.7:0/22412355 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc44011b30 con 0x7fcc540d10a0 2024-01-27T02:33:30.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.169+0000 7fcc5937c700 1 -- 172.21.15.7:0/22412355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc540d10a0 msgr2=0x7fcc540b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:30.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.169+0000 7fcc5937c700 1 --2- 172.21.15.7:0/22412355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc540d10a0 0x7fcc540b1590 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009910 tx=0x7fcc440091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:30.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.169+0000 7fcc5937c700 1 -- 172.21.15.7:0/22412355 shutdown_connections 2024-01-27T02:33:30.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.169+0000 7fcc5937c700 1 --2- 172.21.15.7:0/22412355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc540d10a0 0x7fcc540b1590 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:30.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.169+0000 7fcc5937c700 1 -- 172.21.15.7:0/22412355 >> 172.21.15.7:0/22412355 conn(0x7fcc540679b0 msgr2=0x7fcc54067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:30.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.169+0000 7fcc5937c700 1 -- 172.21.15.7:0/22412355 shutdown_connections 2024-01-27T02:33:30.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.169+0000 7fcc5937c700 1 -- 172.21.15.7:0/22412355 wait complete. 2024-01-27T02:33:30.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.171+0000 7fcc5937c700 1 Processor -- start 2024-01-27T02:33:30.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.171+0000 7fcc5937c700 1 -- start start 2024-01-27T02:33:30.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.172+0000 7fcc5937c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc540d10a0 0x7fcc5416b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:30.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.172+0000 7fcc5937c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc540d1470 0x7fcc540c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:30.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.172+0000 7fcc5937c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc54159b40 0x7fcc54156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:30.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.172+0000 7fcc5937c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5415b7c0 con 0x7fcc540d1470 2024-01-27T02:33:30.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.172+0000 7fcc5937c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc540c8750 con 0x7fcc54159b40 2024-01-27T02:33:30.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.172+0000 7fcc5937c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc540c8a30 con 0x7fcc540d10a0 2024-01-27T02:33:30.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.172+0000 7fcc527fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc540d1470 0x7fcc540c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:30.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.172+0000 7fcc527fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc540d1470 0x7fcc540c7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38000/0 (socket says 172.21.15.7:38000) 2024-01-27T02:33:30.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.172+0000 7fcc527fc700 1 -- 172.21.15.7:0/2024857727 learned_addr learned my addr 172.21.15.7:0/2024857727 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:30.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.172+0000 7fcc52ffd700 1 --2- 172.21.15.7:0/2024857727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc540d10a0 0x7fcc5416b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:30.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.172+0000 7fcc537fe700 1 --2- 172.21.15.7:0/2024857727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc54159b40 0x7fcc54156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:30.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.172+0000 7fcc527fc700 1 -- 172.21.15.7:0/2024857727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc540d10a0 msgr2=0x7fcc5416b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:30.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.172+0000 7fcc527fc700 1 --2- 172.21.15.7:0/2024857727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc540d10a0 0x7fcc5416b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:30.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.172+0000 7fcc527fc700 1 -- 172.21.15.7:0/2024857727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc54159b40 msgr2=0x7fcc54156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:30.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.172+0000 7fcc527fc700 1 --2- 172.21.15.7:0/2024857727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc54159b40 0x7fcc54156450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:30.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.172+0000 7fcc527fc700 1 -- 172.21.15.7:0/2024857727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc44009420 con 0x7fcc540d1470 2024-01-27T02:33:30.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.173+0000 7fcc527fc700 1 --2- 172.21.15.7:0/2024857727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc540d1470 0x7fcc540c7f00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800c590 tx=0x7fcc48007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:30.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.173+0000 7fcc3affd700 1 -- 172.21.15.7:0/2024857727 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48010510 con 0x7fcc540d1470 2024-01-27T02:33:30.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.173+0000 7fcc3affd700 1 -- 172.21.15.7:0/2024857727 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc48010690 con 0x7fcc540d1470 2024-01-27T02:33:30.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.173+0000 7fcc5937c700 1 -- 172.21.15.7:0/2024857727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc540c8d10 con 0x7fcc540d1470 2024-01-27T02:33:30.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.173+0000 7fcc3affd700 1 -- 172.21.15.7:0/2024857727 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48010a50 con 0x7fcc540d1470 2024-01-27T02:33:30.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.173+0000 7fcc5937c700 1 -- 172.21.15.7:0/2024857727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc540c92d0 con 0x7fcc540d1470 2024-01-27T02:33:30.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.174+0000 7fcc5937c700 1 -- 172.21.15.7:0/2024857727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc540c97d0 con 0x7fcc540d1470 2024-01-27T02:33:30.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.174+0000 7fcc3affd700 1 -- 172.21.15.7:0/2024857727 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcc4801b400 con 0x7fcc540d1470 2024-01-27T02:33:30.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.175+0000 7fcc3affd700 1 --2- 172.21.15.7:0/2024857727 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcc40066f30 0x7fcc400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:30.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.175+0000 7fcc3affd700 1 -- 172.21.15.7:0/2024857727 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcc480841f0 con 0x7fcc540d1470 2024-01-27T02:33:30.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.175+0000 7fcc52ffd700 1 --2- 172.21.15.7:0/2024857727 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcc40066f30 0x7fcc400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:30.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.176+0000 7fcc52ffd700 1 --2- 172.21.15.7:0/2024857727 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcc40066f30 0x7fcc400693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009860 tx=0x7fcc44017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:30.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.179+0000 7fcc3affd700 1 -- 172.21.15.7:0/2024857727 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc540c97d0 con 0x7fcc540d1470 2024-01-27T02:33:30.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.377+0000 7fcc5937c700 1 -- 172.21.15.7:0/2024857727 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcc540c97d0 con 0x7fcc40066f30 2024-01-27T02:33:30.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.385+0000 7fcc3affd700 1 -- 172.21.15.7:0/2024857727 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285368 (secure 0 0 0) 0x7fcc540c97d0 con 0x7fcc40066f30 2024-01-27T02:33:30.391 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:30.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.388+0000 7fcc5937c700 1 -- 172.21.15.7:0/2024857727 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcc40066f30 msgr2=0x7fcc400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:30.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.388+0000 7fcc5937c700 1 --2- 172.21.15.7:0/2024857727 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcc40066f30 0x7fcc400693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009860 tx=0x7fcc44017000 comp rx=0 tx=0).stop 2024-01-27T02:33:30.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.388+0000 7fcc5937c700 1 -- 172.21.15.7:0/2024857727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc540d1470 msgr2=0x7fcc540c7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:30.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.388+0000 7fcc5937c700 1 --2- 172.21.15.7:0/2024857727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc540d1470 0x7fcc540c7f00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800c590 tx=0x7fcc48007dd0 comp rx=0 tx=0).stop 2024-01-27T02:33:30.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.388+0000 7fcc5937c700 1 -- 172.21.15.7:0/2024857727 shutdown_connections 2024-01-27T02:33:30.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.388+0000 7fcc5937c700 1 --2- 172.21.15.7:0/2024857727 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcc40066f30 0x7fcc400693e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:30.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.388+0000 7fcc5937c700 1 --2- 172.21.15.7:0/2024857727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc54159b40 0x7fcc54156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:30.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.388+0000 7fcc5937c700 1 --2- 172.21.15.7:0/2024857727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc540d10a0 0x7fcc5416b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:30.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.388+0000 7fcc5937c700 1 --2- 172.21.15.7:0/2024857727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc540d1470 0x7fcc540c7f00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:30.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.388+0000 7fcc5937c700 1 -- 172.21.15.7:0/2024857727 >> 172.21.15.7:0/2024857727 conn(0x7fcc540679b0 msgr2=0x7fcc54157070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:30.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.388+0000 7fcc5937c700 1 -- 172.21.15.7:0/2024857727 shutdown_connections 2024-01-27T02:33:30.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.389+0000 7fcc5937c700 1 -- 172.21.15.7:0/2024857727 wait complete. 2024-01-27T02:33:30.396 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:33:30.404 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-01-27T02:33:29.591925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2771,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559728,"statfs":{"total":1600000000,"available":1597161978,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001500"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.532064+0000","last_change":"2024-01-27T02:33:25.532064+0000","last_active":"2024-01-27T02:33:25.532064+0000","last_peered":"2024-01-27T02:33:25.532064+0000","last_clean":"2024-01-27T02:33:25.532064+0000","last_became_active":"2024-01-27T02:33:25.531475+0000","last_became_peered":"2024-01-27T02:33:25.531475+0000","last_unstale":"2024-01-27T02:33:25.532064+0000","last_undegraded":"2024-01-27T02:33:25.532064+0000","last_fullsized":"2024-01-27T02:33:25.532064+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:00:05.977370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537241+0000","last_change":"2024-01-27T02:33:25.537241+0000","last_active":"2024-01-27T02:33:25.537241+0000","last_peered":"2024-01-27T02:33:25.537241+0000","last_clean":"2024-01-27T02:33:25.537241+0000","last_became_active":"2024-01-27T02:33:25.537004+0000","last_became_peered":"2024-01-27T02:33:25.537004+0000","last_unstale":"2024-01-27T02:33:25.537241+0000","last_undegraded":"2024-01-27T02:33:25.537241+0000","last_fullsized":"2024-01-27T02:33:25.537241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:00:05.977370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.543304+0000","last_change":"2024-01-27T02:33:25.543236+0000","last_active":"2024-01-27T02:33:25.543304+0000","last_peered":"2024-01-27T02:33:25.543304+0000","last_clean":"2024-01-27T02:33:25.543304+0000","last_became_active":"2024-01-27T02:33:25.543040+0000","last_became_peered":"2024-01-27T02:33:25.543040+0000","last_unstale":"2024-01-27T02:33:25.543304+0000","last_undegraded":"2024-01-27T02:33:25.543304+0000","last_fullsized":"2024-01-27T02:33:25.543304+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:30:47.765282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.533729+0000","last_change":"2024-01-27T02:33:25.533729+0000","last_active":"2024-01-27T02:33:25.533729+0000","last_peered":"2024-01-27T02:33:25.533729+0000","last_clean":"2024-01-27T02:33:25.533729+0000","last_became_active":"2024-01-27T02:33:25.533415+0000","last_became_peered":"2024-01-27T02:33:25.533415+0000","last_unstale":"2024-01-27T02:33:25.533729+0000","last_undegraded":"2024-01-27T02:33:25.533729+0000","last_fullsized":"2024-01-27T02:33:25.533729+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:02:17.320801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.534561+0000","last_change":"2024-01-27T02:33:25.534561+0000","last_active":"2024-01-27T02:33:25.534561+0000","last_peered":"2024-01-27T02:33:25.534561+0000","last_clean":"2024-01-27T02:33:25.534561+0000","last_became_active":"2024-01-27T02:33:25.534344+0000","last_became_peered":"2024-01-27T02:33:25.534344+0000","last_unstale":"2024-01-27T02:33:25.534561+0000","last_undegraded":"2024-01-27T02:33:25.534561+0000","last_fullsized":"2024-01-27T02:33:25.534561+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:29:58.352940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.524145+0000","last_change":"2024-01-27T02:33:25.524101+0000","last_active":"2024-01-27T02:33:25.524145+0000","last_peered":"2024-01-27T02:33:25.524145+0000","last_clean":"2024-01-27T02:33:25.524145+0000","last_became_active":"2024-01-27T02:33:25.523933+0000","last_became_peered":"2024-01-27T02:33:25.523933+0000","last_unstale":"2024-01-27T02:33:25.524145+0000","last_undegraded":"2024-01-27T02:33:25.524145+0000","last_fullsized":"2024-01-27T02:33:25.524145+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:40:49.272206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.531764+0000","last_change":"2024-01-27T02:33:25.531764+0000","last_active":"2024-01-27T02:33:25.531764+0000","last_peered":"2024-01-27T02:33:25.531764+0000","last_clean":"2024-01-27T02:33:25.531764+0000","last_became_active":"2024-01-27T02:33:25.531514+0000","last_became_peered":"2024-01-27T02:33:25.531514+0000","last_unstale":"2024-01-27T02:33:25.531764+0000","last_undegraded":"2024-01-27T02:33:25.531764+0000","last_fullsized":"2024-01-27T02:33:25.531764+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:02:17.320801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.532253+0000","last_change":"2024-01-27T02:33:25.532253+0000","last_active":"2024-01-27T02:33:25.532253+0000","last_peered":"2024-01-27T02:33:25.532253+0000","last_clean":"2024-01-27T02:33:25.532253+0000","last_became_active":"2024-01-27T02:33:25.531784+0000","last_became_peered":"2024-01-27T02:33:25.531784+0000","last_unstale":"2024-01-27T02:33:25.532253+0000","last_undegraded":"2024-01-27T02:33:25.532253+0000","last_fullsized":"2024-01-27T02:33:25.532253+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:00:05.977370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.540430+0000","last_change":"2024-01-27T02:33:25.540430+0000","last_active":"2024-01-27T02:33:25.540430+0000","last_peered":"2024-01-27T02:33:25.540430+0000","last_clean":"2024-01-27T02:33:25.540430+0000","last_became_active":"2024-01-27T02:33:25.540012+0000","last_became_peered":"2024-01-27T02:33:25.540012+0000","last_unstale":"2024-01-27T02:33:25.540430+0000","last_undegraded":"2024-01-27T02:33:25.540430+0000","last_fullsized":"2024-01-27T02:33:25.540430+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:29:58.352940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537762+0000","last_change":"2024-01-27T02:33:25.537762+0000","last_active":"2024-01-27T02:33:25.537762+0000","last_peered":"2024-01-27T02:33:25.537762+0000","last_clean":"2024-01-27T02:33:25.537762+0000","last_became_active":"2024-01-27T02:33:25.534910+0000","last_became_peered":"2024-01-27T02:33:25.534910+0000","last_unstale":"2024-01-27T02:33:25.537762+0000","last_undegraded":"2024-01-27T02:33:25.537762+0000","last_fullsized":"2024-01-27T02:33:25.537762+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:40:48.634883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.536294+0000","last_change":"2024-01-27T02:33:25.536294+0000","last_active":"2024-01-27T02:33:25.536294+0000","last_peered":"2024-01-27T02:33:25.536294+0000","last_clean":"2024-01-27T02:33:25.536294+0000","last_became_active":"2024-01-27T02:33:25.536038+0000","last_became_peered":"2024-01-27T02:33:25.536038+0000","last_unstale":"2024-01-27T02:33:25.536294+0000","last_undegraded":"2024-01-27T02:33:25.536294+0000","last_fullsized":"2024-01-27T02:33:25.536294+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:26:06.584436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.535327+0000","last_change":"2024-01-27T02:33:25.535327+0000","last_active":"2024-01-27T02:33:25.535327+0000","last_peered":"2024-01-27T02:33:25.535327+0000","last_clean":"2024-01-27T02:33:25.535327+0000","last_became_active":"2024-01-27T02:33:25.535092+0000","last_became_peered":"2024-01-27T02:33:25.535092+0000","last_unstale":"2024-01-27T02:33:25.535327+0000","last_undegraded":"2024-01-27T02:33:25.535327+0000","last_fullsized":"2024-01-27T02:33:25.535327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:35:50.150354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.540782+0000","last_change":"2024-01-27T02:33:25.540782+0000","last_active":"2024-01-27T02:33:25.540782+0000","last_peered":"2024-01-27T02:33:25.540782+0000","last_clean":"2024-01-27T02:33:25.540782+0000","last_became_active":"2024-01-27T02:33:25.536407+0000","last_became_peered":"2024-01-27T02:33:25.536407+0000","last_unstale":"2024-01-27T02:33:25.540782+0000","last_undegraded":"2024-01-27T02:33:25.540782+0000","last_fullsized":"2024-01-27T02:33:25.540782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:26:02.647398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.540262+0000","last_change":"2024-01-27T02:33:25.540262+0000","last_active":"2024-01-27T02:33:25.540262+0000","last_peered":"2024-01-27T02:33:25.540262+0000","last_clean":"2024-01-27T02:33:25.540262+0000","last_became_active":"2024-01-27T02:33:25.539622+0000","last_became_peered":"2024-01-27T02:33:25.539622+0000","last_unstale":"2024-01-27T02:33:25.540262+0000","last_undegraded":"2024-01-27T02:33:25.540262+0000","last_fullsized":"2024-01-27T02:33:25.540262+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:05:53.123337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537289+0000","last_change":"2024-01-27T02:33:25.536568+0000","last_active":"2024-01-27T02:33:25.537289+0000","last_peered":"2024-01-27T02:33:25.537289+0000","last_clean":"2024-01-27T02:33:25.537289+0000","last_became_active":"2024-01-27T02:33:25.536316+0000","last_became_peered":"2024-01-27T02:33:25.536316+0000","last_unstale":"2024-01-27T02:33:25.537289+0000","last_undegraded":"2024-01-27T02:33:25.537289+0000","last_fullsized":"2024-01-27T02:33:25.537289+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:36:49.947070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.530270+0000","last_change":"2024-01-27T02:33:25.530175+0000","last_active":"2024-01-27T02:33:25.530270+0000","last_peered":"2024-01-27T02:33:25.530270+0000","last_clean":"2024-01-27T02:33:25.530270+0000","last_became_active":"2024-01-27T02:33:25.529759+0000","last_became_peered":"2024-01-27T02:33:25.529759+0000","last_unstale":"2024-01-27T02:33:25.530270+0000","last_undegraded":"2024-01-27T02:33:25.530270+0000","last_fullsized":"2024-01-27T02:33:25.530270+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:23:23.599278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.531503+0000","last_change":"2024-01-27T02:33:25.531405+0000","last_active":"2024-01-27T02:33:25.531503+0000","last_peered":"2024-01-27T02:33:25.531503+0000","last_clean":"2024-01-27T02:33:25.531503+0000","last_became_active":"2024-01-27T02:33:25.531024+0000","last_became_peered":"2024-01-27T02:33:25.531024+0000","last_unstale":"2024-01-27T02:33:25.531503+0000","last_undegraded":"2024-01-27T02:33:25.531503+0000","last_fullsized":"2024-01-27T02:33:25.531503+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:18:21.974137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.530928+0000","last_change":"2024-01-27T02:33:25.530865+0000","last_active":"2024-01-27T02:33:25.530928+0000","last_peered":"2024-01-27T02:33:25.530928+0000","last_clean":"2024-01-27T02:33:25.530928+0000","last_became_active":"2024-01-27T02:33:25.530601+0000","last_became_peered":"2024-01-27T02:33:25.530601+0000","last_unstale":"2024-01-27T02:33:25.530928+0000","last_undegraded":"2024-01-27T02:33:25.530928+0000","last_fullsized":"2024-01-27T02:33:25.530928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:13:06.294428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.533899+0000","last_change":"2024-01-27T02:33:25.533823+0000","last_active":"2024-01-27T02:33:25.533899+0000","last_peered":"2024-01-27T02:33:25.533899+0000","last_clean":"2024-01-27T02:33:25.533899+0000","last_became_active":"2024-01-27T02:33:25.533551+0000","last_became_peered":"2024-01-27T02:33:25.533551+0000","last_unstale":"2024-01-27T02:33:25.533899+0000","last_undegraded":"2024-01-27T02:33:25.533899+0000","last_fullsized":"2024-01-27T02:33:25.533899+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:02:44.519562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.541102+0000","last_change":"2024-01-27T02:33:25.541102+0000","last_active":"2024-01-27T02:33:25.541102+0000","last_peered":"2024-01-27T02:33:25.541102+0000","last_clean":"2024-01-27T02:33:25.541102+0000","last_became_active":"2024-01-27T02:33:25.540798+0000","last_became_peered":"2024-01-27T02:33:25.540798+0000","last_unstale":"2024-01-27T02:33:25.541102+0000","last_undegraded":"2024-01-27T02:33:25.541102+0000","last_fullsized":"2024-01-27T02:33:25.541102+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:24:20.892845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.523839+0000","last_change":"2024-01-27T02:33:25.523778+0000","last_active":"2024-01-27T02:33:25.523839+0000","last_peered":"2024-01-27T02:33:25.523839+0000","last_clean":"2024-01-27T02:33:25.523839+0000","last_became_active":"2024-01-27T02:33:25.523554+0000","last_became_peered":"2024-01-27T02:33:25.523554+0000","last_unstale":"2024-01-27T02:33:25.523839+0000","last_undegraded":"2024-01-27T02:33:25.523839+0000","last_fullsized":"2024-01-27T02:33:25.523839+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:39:33.075890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537347+0000","last_change":"2024-01-27T02:33:25.537347+0000","last_active":"2024-01-27T02:33:25.537347+0000","last_peered":"2024-01-27T02:33:25.537347+0000","last_clean":"2024-01-27T02:33:25.537347+0000","last_became_active":"2024-01-27T02:33:25.536341+0000","last_became_peered":"2024-01-27T02:33:25.536341+0000","last_unstale":"2024-01-27T02:33:25.537347+0000","last_undegraded":"2024-01-27T02:33:25.537347+0000","last_fullsized":"2024-01-27T02:33:25.537347+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:30:47.765282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.525026+0000","last_change":"2024-01-27T02:33:25.524952+0000","last_active":"2024-01-27T02:33:25.525026+0000","last_peered":"2024-01-27T02:33:25.525026+0000","last_clean":"2024-01-27T02:33:25.525026+0000","last_became_active":"2024-01-27T02:33:25.524721+0000","last_became_peered":"2024-01-27T02:33:25.524721+0000","last_unstale":"2024-01-27T02:33:25.525026+0000","last_undegraded":"2024-01-27T02:33:25.525026+0000","last_fullsized":"2024-01-27T02:33:25.525026+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:26:06.584436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.525413+0000","last_change":"2024-01-27T02:33:25.525335+0000","last_active":"2024-01-27T02:33:25.525413+0000","last_peered":"2024-01-27T02:33:25.525413+0000","last_clean":"2024-01-27T02:33:25.525413+0000","last_became_active":"2024-01-27T02:33:25.524885+0000","last_became_peered":"2024-01-27T02:33:25.524885+0000","last_unstale":"2024-01-27T02:33:25.525413+0000","last_undegraded":"2024-01-27T02:33:25.525413+0000","last_fullsized":"2024-01-27T02:33:25.525413+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:32:41.154409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.535317+0000","last_change":"2024-01-27T02:33:25.535199+0000","last_active":"2024-01-27T02:33:25.535317+0000","last_peered":"2024-01-27T02:33:25.535317+0000","last_clean":"2024-01-27T02:33:25.535317+0000","last_became_active":"2024-01-27T02:33:25.534928+0000","last_became_peered":"2024-01-27T02:33:25.534928+0000","last_unstale":"2024-01-27T02:33:25.535317+0000","last_undegraded":"2024-01-27T02:33:25.535317+0000","last_fullsized":"2024-01-27T02:33:25.535317+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:11:01.017228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.535285+0000","last_change":"2024-01-27T02:33:25.535285+0000","last_active":"2024-01-27T02:33:25.535285+0000","last_peered":"2024-01-27T02:33:25.535285+0000","last_clean":"2024-01-27T02:33:25.535285+0000","last_became_active":"2024-01-27T02:33:25.534119+0000","last_became_peered":"2024-01-27T02:33:25.534119+0000","last_unstale":"2024-01-27T02:33:25.535285+0000","last_undegraded":"2024-01-27T02:33:25.535285+0000","last_fullsized":"2024-01-27T02:33:25.535285+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:03:32.136426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.531483+0000","last_change":"2024-01-27T02:33:25.531258+0000","last_active":"2024-01-27T02:33:25.531483+0000","last_peered":"2024-01-27T02:33:25.531483+0000","last_clean":"2024-01-27T02:33:25.531483+0000","last_became_active":"2024-01-27T02:33:25.530710+0000","last_became_peered":"2024-01-27T02:33:25.530710+0000","last_unstale":"2024-01-27T02:33:25.531483+0000","last_undegraded":"2024-01-27T02:33:25.531483+0000","last_fullsized":"2024-01-27T02:33:25.531483+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:36:14.220063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537083+0000","last_change":"2024-01-27T02:33:25.537083+0000","last_active":"2024-01-27T02:33:25.537083+0000","last_peered":"2024-01-27T02:33:25.537083+0000","last_clean":"2024-01-27T02:33:25.537083+0000","last_became_active":"2024-01-27T02:33:25.536841+0000","last_became_peered":"2024-01-27T02:33:25.536841+0000","last_unstale":"2024-01-27T02:33:25.537083+0000","last_undegraded":"2024-01-27T02:33:25.537083+0000","last_fullsized":"2024-01-27T02:33:25.537083+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:38:34.204456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.535334+0000","last_change":"2024-01-27T02:33:25.535223+0000","last_active":"2024-01-27T02:33:25.535334+0000","last_peered":"2024-01-27T02:33:25.535334+0000","last_clean":"2024-01-27T02:33:25.535334+0000","last_became_active":"2024-01-27T02:33:25.534954+0000","last_became_peered":"2024-01-27T02:33:25.534954+0000","last_unstale":"2024-01-27T02:33:25.535334+0000","last_undegraded":"2024-01-27T02:33:25.535334+0000","last_fullsized":"2024-01-27T02:33:25.535334+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:37:56.187364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.530210+0000","last_change":"2024-01-27T02:33:25.529298+0000","last_active":"2024-01-27T02:33:25.530210+0000","last_peered":"2024-01-27T02:33:25.530210+0000","last_clean":"2024-01-27T02:33:25.530210+0000","last_became_active":"2024-01-27T02:33:25.528998+0000","last_became_peered":"2024-01-27T02:33:25.528998+0000","last_unstale":"2024-01-27T02:33:25.530210+0000","last_undegraded":"2024-01-27T02:33:25.530210+0000","last_fullsized":"2024-01-27T02:33:25.530210+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:06:10.291533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.532284+0000","last_change":"2024-01-27T02:33:25.532044+0000","last_active":"2024-01-27T02:33:25.532284+0000","last_peered":"2024-01-27T02:33:25.532284+0000","last_clean":"2024-01-27T02:33:25.532284+0000","last_became_active":"2024-01-27T02:33:25.531294+0000","last_became_peered":"2024-01-27T02:33:25.531294+0000","last_unstale":"2024-01-27T02:33:25.532284+0000","last_undegraded":"2024-01-27T02:33:25.532284+0000","last_fullsized":"2024-01-27T02:33:25.532284+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:15:10.938188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.541373+0000","last_change":"2024-01-27T02:33:25.541373+0000","last_active":"2024-01-27T02:33:25.541373+0000","last_peered":"2024-01-27T02:33:25.541373+0000","last_clean":"2024-01-27T02:33:25.541373+0000","last_became_active":"2024-01-27T02:33:25.540939+0000","last_became_peered":"2024-01-27T02:33:25.540939+0000","last_unstale":"2024-01-27T02:33:25.541373+0000","last_undegraded":"2024-01-27T02:33:25.541373+0000","last_fullsized":"2024-01-27T02:33:25.541373+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:50:16.570468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.532792+0000","last_change":"2024-01-27T02:33:25.532792+0000","last_active":"2024-01-27T02:33:25.532792+0000","last_peered":"2024-01-27T02:33:25.532792+0000","last_clean":"2024-01-27T02:33:25.532792+0000","last_became_active":"2024-01-27T02:33:25.532511+0000","last_became_peered":"2024-01-27T02:33:25.532511+0000","last_unstale":"2024-01-27T02:33:25.532792+0000","last_undegraded":"2024-01-27T02:33:25.532792+0000","last_fullsized":"2024-01-27T02:33:25.532792+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:34:40.708292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.817720+0000","last_change":"2024-01-27T02:33:24.531980+0000","last_active":"2024-01-27T02:33:25.817720+0000","last_peered":"2024-01-27T02:33:25.817720+0000","last_clean":"2024-01-27T02:33:25.817720+0000","last_became_active":"2024-01-27T02:33:24.531377+0000","last_became_peered":"2024-01-27T02:33:24.531377+0000","last_unstale":"2024-01-27T02:33:25.817720+0000","last_undegraded":"2024-01-27T02:33:25.817720+0000","last_fullsized":"2024-01-27T02:33:25.817720+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:29:57.350543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.530203+0000","last_change":"2024-01-27T02:33:25.530117+0000","last_active":"2024-01-27T02:33:25.530203+0000","last_peered":"2024-01-27T02:33:25.530203+0000","last_clean":"2024-01-27T02:33:25.530203+0000","last_became_active":"2024-01-27T02:33:25.529807+0000","last_became_peered":"2024-01-27T02:33:25.529807+0000","last_unstale":"2024-01-27T02:33:25.530203+0000","last_undegraded":"2024-01-27T02:33:25.530203+0000","last_fullsized":"2024-01-27T02:33:25.530203+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:34:40.708292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.814415+0000","last_change":"2024-01-27T02:33:24.532094+0000","last_active":"2024-01-27T02:33:25.814415+0000","last_peered":"2024-01-27T02:33:25.814415+0000","last_clean":"2024-01-27T02:33:25.814415+0000","last_became_active":"2024-01-27T02:33:24.531848+0000","last_became_peered":"2024-01-27T02:33:24.531848+0000","last_unstale":"2024-01-27T02:33:25.814415+0000","last_undegraded":"2024-01-27T02:33:25.814415+0000","last_fullsized":"2024-01-27T02:33:25.814415+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:23:22.596881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.533818+0000","last_change":"2024-01-27T02:33:25.533818+0000","last_active":"2024-01-27T02:33:25.533818+0000","last_peered":"2024-01-27T02:33:25.533818+0000","last_clean":"2024-01-27T02:33:25.533818+0000","last_became_active":"2024-01-27T02:33:25.533583+0000","last_became_peered":"2024-01-27T02:33:25.533583+0000","last_unstale":"2024-01-27T02:33:25.533818+0000","last_undegraded":"2024-01-27T02:33:25.533818+0000","last_fullsized":"2024-01-27T02:33:25.533818+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:06:10.291533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.815739+0000","last_change":"2024-01-27T02:33:24.529476+0000","last_active":"2024-01-27T02:33:25.815739+0000","last_peered":"2024-01-27T02:33:25.815739+0000","last_clean":"2024-01-27T02:33:25.815739+0000","last_became_active":"2024-01-27T02:33:24.528796+0000","last_became_peered":"2024-01-27T02:33:24.528796+0000","last_unstale":"2024-01-27T02:33:25.815739+0000","last_undegraded":"2024-01-27T02:33:25.815739+0000","last_fullsized":"2024-01-27T02:33:25.815739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:23:22.596881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537969+0000","last_change":"2024-01-27T02:33:25.537969+0000","last_active":"2024-01-27T02:33:25.537969+0000","last_peered":"2024-01-27T02:33:25.537969+0000","last_clean":"2024-01-27T02:33:25.537969+0000","last_became_active":"2024-01-27T02:33:25.537435+0000","last_became_peered":"2024-01-27T02:33:25.537435+0000","last_unstale":"2024-01-27T02:33:25.537969+0000","last_undegraded":"2024-01-27T02:33:25.537969+0000","last_fullsized":"2024-01-27T02:33:25.537969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:32:41.154409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.815929+0000","last_change":"2024-01-27T02:33:24.531045+0000","last_active":"2024-01-27T02:33:25.815929+0000","last_peered":"2024-01-27T02:33:25.815929+0000","last_clean":"2024-01-27T02:33:25.815929+0000","last_became_active":"2024-01-27T02:33:24.530529+0000","last_became_peered":"2024-01-27T02:33:24.530529+0000","last_unstale":"2024-01-27T02:33:25.815929+0000","last_undegraded":"2024-01-27T02:33:25.815929+0000","last_fullsized":"2024-01-27T02:33:25.815929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:30:46.762885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.530192+0000","last_change":"2024-01-27T02:33:25.528896+0000","last_active":"2024-01-27T02:33:25.530192+0000","last_peered":"2024-01-27T02:33:25.530192+0000","last_clean":"2024-01-27T02:33:25.530192+0000","last_became_active":"2024-01-27T02:33:25.528515+0000","last_became_peered":"2024-01-27T02:33:25.528515+0000","last_unstale":"2024-01-27T02:33:25.530192+0000","last_undegraded":"2024-01-27T02:33:25.530192+0000","last_fullsized":"2024-01-27T02:33:25.530192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:05:53.123337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.816392+0000","last_change":"2024-01-27T02:33:24.530355+0000","last_active":"2024-01-27T02:33:25.816392+0000","last_peered":"2024-01-27T02:33:25.816392+0000","last_clean":"2024-01-27T02:33:25.816392+0000","last_became_active":"2024-01-27T02:33:24.530071+0000","last_became_peered":"2024-01-27T02:33:24.530071+0000","last_unstale":"2024-01-27T02:33:25.816392+0000","last_undegraded":"2024-01-27T02:33:25.816392+0000","last_fullsized":"2024-01-27T02:33:25.816392+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:05:52.120939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537562+0000","last_change":"2024-01-27T02:33:25.537562+0000","last_active":"2024-01-27T02:33:25.537562+0000","last_peered":"2024-01-27T02:33:25.537562+0000","last_clean":"2024-01-27T02:33:25.537562+0000","last_became_active":"2024-01-27T02:33:25.537040+0000","last_became_peered":"2024-01-27T02:33:25.537040+0000","last_unstale":"2024-01-27T02:33:25.537562+0000","last_undegraded":"2024-01-27T02:33:25.537562+0000","last_fullsized":"2024-01-27T02:33:25.537562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:06:10.291533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.814333+0000","last_change":"2024-01-27T02:33:24.527658+0000","last_active":"2024-01-27T02:33:25.814333+0000","last_peered":"2024-01-27T02:33:25.814333+0000","last_clean":"2024-01-27T02:33:25.814333+0000","last_became_active":"2024-01-27T02:33:24.527438+0000","last_became_peered":"2024-01-27T02:33:24.527438+0000","last_unstale":"2024-01-27T02:33:25.814333+0000","last_undegraded":"2024-01-27T02:33:25.814333+0000","last_fullsized":"2024-01-27T02:33:25.814333+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:11:00.014830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537400+0000","last_change":"2024-01-27T02:33:25.537400+0000","last_active":"2024-01-27T02:33:25.537400+0000","last_peered":"2024-01-27T02:33:25.537400+0000","last_clean":"2024-01-27T02:33:25.537400+0000","last_became_active":"2024-01-27T02:33:25.534594+0000","last_became_peered":"2024-01-27T02:33:25.534594+0000","last_unstale":"2024-01-27T02:33:25.537400+0000","last_undegraded":"2024-01-27T02:33:25.537400+0000","last_fullsized":"2024-01-27T02:33:25.537400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:11:20.939912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.529449+0000","last_change":"2024-01-27T02:33:24.529141+0000","last_active":"2024-01-27T02:33:24.529449+0000","last_peered":"2024-01-27T02:33:24.529449+0000","last_clean":"2024-01-27T02:33:24.529449+0000","last_became_active":"2024-01-27T02:33:24.528970+0000","last_became_peered":"2024-01-27T02:33:24.528970+0000","last_unstale":"2024-01-27T02:33:24.529449+0000","last_undegraded":"2024-01-27T02:33:24.529449+0000","last_fullsized":"2024-01-27T02:33:24.529449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:22:19.787837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.525553+0000","last_change":"2024-01-27T02:33:25.525486+0000","last_active":"2024-01-27T02:33:25.525553+0000","last_peered":"2024-01-27T02:33:25.525553+0000","last_clean":"2024-01-27T02:33:25.525553+0000","last_became_active":"2024-01-27T02:33:25.525140+0000","last_became_peered":"2024-01-27T02:33:25.525140+0000","last_unstale":"2024-01-27T02:33:25.525553+0000","last_undegraded":"2024-01-27T02:33:25.525553+0000","last_fullsized":"2024-01-27T02:33:25.525553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:54:35.139188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.527171+0000","last_change":"2024-01-27T02:33:24.525988+0000","last_active":"2024-01-27T02:33:24.527171+0000","last_peered":"2024-01-27T02:33:24.527171+0000","last_clean":"2024-01-27T02:33:24.527171+0000","last_became_active":"2024-01-27T02:33:24.525584+0000","last_became_peered":"2024-01-27T02:33:24.525584+0000","last_unstale":"2024-01-27T02:33:24.527171+0000","last_undegraded":"2024-01-27T02:33:24.527171+0000","last_fullsized":"2024-01-27T02:33:24.527171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:03:31.134029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537890+0000","last_change":"2024-01-27T02:33:25.537815+0000","last_active":"2024-01-27T02:33:25.537890+0000","last_peered":"2024-01-27T02:33:25.537890+0000","last_clean":"2024-01-27T02:33:25.537890+0000","last_became_active":"2024-01-27T02:33:25.537581+0000","last_became_peered":"2024-01-27T02:33:25.537581+0000","last_unstale":"2024-01-27T02:33:25.537890+0000","last_undegraded":"2024-01-27T02:33:25.537890+0000","last_fullsized":"2024-01-27T02:33:25.537890+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:03:50.797870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.528431+0000","last_change":"2024-01-27T02:33:24.525405+0000","last_active":"2024-01-27T02:33:25.528431+0000","last_peered":"2024-01-27T02:33:25.528431+0000","last_clean":"2024-01-27T02:33:25.528431+0000","last_became_active":"2024-01-27T02:33:24.525090+0000","last_became_peered":"2024-01-27T02:33:24.525090+0000","last_unstale":"2024-01-27T02:33:25.528431+0000","last_undegraded":"2024-01-27T02:33:25.528431+0000","last_fullsized":"2024-01-27T02:33:25.528431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:02:43.517165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.532207+0000","last_change":"2024-01-27T02:33:25.532207+0000","last_active":"2024-01-27T02:33:25.532207+0000","last_peered":"2024-01-27T02:33:25.532207+0000","last_clean":"2024-01-27T02:33:25.532207+0000","last_became_active":"2024-01-27T02:33:25.531823+0000","last_became_peered":"2024-01-27T02:33:25.531823+0000","last_unstale":"2024-01-27T02:33:25.532207+0000","last_undegraded":"2024-01-27T02:33:25.532207+0000","last_fullsized":"2024-01-27T02:33:25.532207+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:32:19.700585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.529755+0000","last_change":"2024-01-27T02:33:24.529629+0000","last_active":"2024-01-27T02:33:24.529755+0000","last_peered":"2024-01-27T02:33:24.529755+0000","last_clean":"2024-01-27T02:33:24.529755+0000","last_became_active":"2024-01-27T02:33:24.529352+0000","last_became_peered":"2024-01-27T02:33:24.529352+0000","last_unstale":"2024-01-27T02:33:24.529755+0000","last_undegraded":"2024-01-27T02:33:24.529755+0000","last_fullsized":"2024-01-27T02:33:24.529755+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:24:19.890448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537338+0000","last_change":"2024-01-27T02:33:25.537206+0000","last_active":"2024-01-27T02:33:25.537338+0000","last_peered":"2024-01-27T02:33:25.537338+0000","last_clean":"2024-01-27T02:33:25.537338+0000","last_became_active":"2024-01-27T02:33:25.536768+0000","last_became_peered":"2024-01-27T02:33:25.536768+0000","last_unstale":"2024-01-27T02:33:25.537338+0000","last_undegraded":"2024-01-27T02:33:25.537338+0000","last_fullsized":"2024-01-27T02:33:25.537338+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:37:33.304637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.816448+0000","last_change":"2024-01-27T02:33:24.532755+0000","last_active":"2024-01-27T02:33:25.816448+0000","last_peered":"2024-01-27T02:33:25.816448+0000","last_clean":"2024-01-27T02:33:25.816448+0000","last_became_active":"2024-01-27T02:33:24.532515+0000","last_became_peered":"2024-01-27T02:33:24.532515+0000","last_unstale":"2024-01-27T02:33:25.816448+0000","last_undegraded":"2024-01-27T02:33:25.816448+0000","last_fullsized":"2024-01-27T02:33:25.816448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:03:49.795473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.532055+0000","last_change":"2024-01-27T02:33:25.532055+0000","last_active":"2024-01-27T02:33:25.532055+0000","last_peered":"2024-01-27T02:33:25.532055+0000","last_clean":"2024-01-27T02:33:25.532055+0000","last_became_active":"2024-01-27T02:33:25.531570+0000","last_became_peered":"2024-01-27T02:33:25.531570+0000","last_unstale":"2024-01-27T02:33:25.532055+0000","last_undegraded":"2024-01-27T02:33:25.532055+0000","last_fullsized":"2024-01-27T02:33:25.532055+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:51:40.287635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.816998+0000","last_change":"2024-01-27T02:33:24.526912+0000","last_active":"2024-01-27T02:33:25.816998+0000","last_peered":"2024-01-27T02:33:25.816998+0000","last_clean":"2024-01-27T02:33:25.816998+0000","last_became_active":"2024-01-27T02:33:24.526500+0000","last_became_peered":"2024-01-27T02:33:24.526500+0000","last_unstale":"2024-01-27T02:33:25.816998+0000","last_undegraded":"2024-01-27T02:33:25.816998+0000","last_fullsized":"2024-01-27T02:33:25.816998+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:01:36.659231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.530185+0000","last_change":"2024-01-27T02:33:25.530010+0000","last_active":"2024-01-27T02:33:25.530185+0000","last_peered":"2024-01-27T02:33:25.530185+0000","last_clean":"2024-01-27T02:33:25.530185+0000","last_became_active":"2024-01-27T02:33:25.529375+0000","last_became_peered":"2024-01-27T02:33:25.529375+0000","last_unstale":"2024-01-27T02:33:25.530185+0000","last_undegraded":"2024-01-27T02:33:25.530185+0000","last_fullsized":"2024-01-27T02:33:25.530185+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:56:29.935410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.816390+0000","last_change":"2024-01-27T02:33:24.532027+0000","last_active":"2024-01-27T02:33:25.816390+0000","last_peered":"2024-01-27T02:33:25.816390+0000","last_clean":"2024-01-27T02:33:25.816390+0000","last_became_active":"2024-01-27T02:33:24.531751+0000","last_became_peered":"2024-01-27T02:33:24.531751+0000","last_unstale":"2024-01-27T02:33:25.816390+0000","last_undegraded":"2024-01-27T02:33:25.816390+0000","last_fullsized":"2024-01-27T02:33:25.816390+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:38:33.202059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537187+0000","last_change":"2024-01-27T02:33:25.537187+0000","last_active":"2024-01-27T02:33:25.537187+0000","last_peered":"2024-01-27T02:33:25.537187+0000","last_clean":"2024-01-27T02:33:25.537187+0000","last_became_active":"2024-01-27T02:33:25.533699+0000","last_became_peered":"2024-01-27T02:33:25.533699+0000","last_unstale":"2024-01-27T02:33:25.537187+0000","last_undegraded":"2024-01-27T02:33:25.537187+0000","last_fullsized":"2024-01-27T02:33:25.537187+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:36:14.220063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.530826+0000","last_change":"2024-01-27T02:33:24.530758+0000","last_active":"2024-01-27T02:33:24.530826+0000","last_peered":"2024-01-27T02:33:24.530826+0000","last_clean":"2024-01-27T02:33:24.530826+0000","last_became_active":"2024-01-27T02:33:24.529888+0000","last_became_peered":"2024-01-27T02:33:24.529888+0000","last_unstale":"2024-01-27T02:33:24.530826+0000","last_undegraded":"2024-01-27T02:33:24.530826+0000","last_fullsized":"2024-01-27T02:33:24.530826+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:38:33.202059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.539205+0000","last_change":"2024-01-27T02:33:25.538969+0000","last_active":"2024-01-27T02:33:25.539205+0000","last_peered":"2024-01-27T02:33:25.539205+0000","last_clean":"2024-01-27T02:33:25.539205+0000","last_became_active":"2024-01-27T02:33:25.538613+0000","last_became_peered":"2024-01-27T02:33:25.538613+0000","last_unstale":"2024-01-27T02:33:25.539205+0000","last_undegraded":"2024-01-27T02:33:25.539205+0000","last_fullsized":"2024-01-27T02:33:25.539205+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:33:01.629270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.520876+0000","last_change":"2024-01-27T02:33:24.534143+0000","last_active":"2024-01-27T02:33:25.520876+0000","last_peered":"2024-01-27T02:33:25.520876+0000","last_clean":"2024-01-27T02:33:25.520876+0000","last_became_active":"2024-01-27T02:33:24.533715+0000","last_became_peered":"2024-01-27T02:33:24.533715+0000","last_unstale":"2024-01-27T02:33:25.520876+0000","last_undegraded":"2024-01-27T02:33:25.520876+0000","last_fullsized":"2024-01-27T02:33:25.520876+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:13:05.292031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.523567+0000","last_change":"2024-01-27T02:33:25.523567+0000","last_active":"2024-01-27T02:33:25.523567+0000","last_peered":"2024-01-27T02:33:25.523567+0000","last_clean":"2024-01-27T02:33:25.523567+0000","last_became_active":"2024-01-27T02:33:25.523376+0000","last_became_peered":"2024-01-27T02:33:25.523376+0000","last_unstale":"2024-01-27T02:33:25.523567+0000","last_undegraded":"2024-01-27T02:33:25.523567+0000","last_fullsized":"2024-01-27T02:33:25.523567+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:31:58.230972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.819146+0000","last_change":"2024-01-27T02:33:24.528941+0000","last_active":"2024-01-27T02:33:25.819146+0000","last_peered":"2024-01-27T02:33:25.819146+0000","last_clean":"2024-01-27T02:33:25.819146+0000","last_became_active":"2024-01-27T02:33:24.528659+0000","last_became_peered":"2024-01-27T02:33:24.528659+0000","last_unstale":"2024-01-27T02:33:25.819146+0000","last_undegraded":"2024-01-27T02:33:25.819146+0000","last_fullsized":"2024-01-27T02:33:25.819146+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:26:01.645000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.535995+0000","last_change":"2024-01-27T02:33:25.535995+0000","last_active":"2024-01-27T02:33:25.535995+0000","last_peered":"2024-01-27T02:33:25.535995+0000","last_clean":"2024-01-27T02:33:25.535995+0000","last_became_active":"2024-01-27T02:33:25.535605+0000","last_became_peered":"2024-01-27T02:33:25.535605+0000","last_unstale":"2024-01-27T02:33:25.535995+0000","last_undegraded":"2024-01-27T02:33:25.535995+0000","last_fullsized":"2024-01-27T02:33:25.535995+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:38:22.996505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.530491+0000","last_change":"2024-01-27T02:33:24.526776+0000","last_active":"2024-01-27T02:33:25.530491+0000","last_peered":"2024-01-27T02:33:25.530491+0000","last_clean":"2024-01-27T02:33:25.530491+0000","last_became_active":"2024-01-27T02:33:24.526136+0000","last_became_peered":"2024-01-27T02:33:24.526136+0000","last_unstale":"2024-01-27T02:33:25.530491+0000","last_undegraded":"2024-01-27T02:33:25.530491+0000","last_fullsized":"2024-01-27T02:33:25.530491+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:41:00.526127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.542378+0000","last_change":"2024-01-27T02:33:25.542378+0000","last_active":"2024-01-27T02:33:25.542378+0000","last_peered":"2024-01-27T02:33:25.542378+0000","last_clean":"2024-01-27T02:33:25.542378+0000","last_became_active":"2024-01-27T02:33:25.542175+0000","last_became_peered":"2024-01-27T02:33:25.542175+0000","last_unstale":"2024-01-27T02:33:25.542378+0000","last_undegraded":"2024-01-27T02:33:25.542378+0000","last_fullsized":"2024-01-27T02:33:25.542378+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:52:44.112812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.815680+0000","last_change":"2024-01-27T02:33:24.524053+0000","last_active":"2024-01-27T02:33:25.815680+0000","last_peered":"2024-01-27T02:33:25.815680+0000","last_clean":"2024-01-27T02:33:25.815680+0000","last_became_active":"2024-01-27T02:33:24.523818+0000","last_became_peered":"2024-01-27T02:33:24.523818+0000","last_unstale":"2024-01-27T02:33:25.815680+0000","last_undegraded":"2024-01-27T02:33:25.815680+0000","last_fullsized":"2024-01-27T02:33:25.815680+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:26:01.645000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.523310+0000","last_change":"2024-01-27T02:33:25.523105+0000","last_active":"2024-01-27T02:33:25.523310+0000","last_peered":"2024-01-27T02:33:25.523310+0000","last_clean":"2024-01-27T02:33:25.523310+0000","last_became_active":"2024-01-27T02:33:25.522821+0000","last_became_peered":"2024-01-27T02:33:25.522821+0000","last_unstale":"2024-01-27T02:33:25.523310+0000","last_undegraded":"2024-01-27T02:33:25.523310+0000","last_fullsized":"2024-01-27T02:33:25.523310+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:38:52.328572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.529237+0000","last_change":"2024-01-27T02:33:24.529323+0000","last_active":"2024-01-27T02:33:25.529237+0000","last_peered":"2024-01-27T02:33:25.529237+0000","last_clean":"2024-01-27T02:33:25.529237+0000","last_became_active":"2024-01-27T02:33:24.529085+0000","last_became_peered":"2024-01-27T02:33:24.529085+0000","last_unstale":"2024-01-27T02:33:25.529237+0000","last_undegraded":"2024-01-27T02:33:25.529237+0000","last_fullsized":"2024-01-27T02:33:25.529237+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:32:40.152012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.535296+0000","last_change":"2024-01-27T02:33:25.534775+0000","last_active":"2024-01-27T02:33:25.535296+0000","last_peered":"2024-01-27T02:33:25.535296+0000","last_clean":"2024-01-27T02:33:25.535296+0000","last_became_active":"2024-01-27T02:33:25.534434+0000","last_became_peered":"2024-01-27T02:33:25.534434+0000","last_unstale":"2024-01-27T02:33:25.535296+0000","last_undegraded":"2024-01-27T02:33:25.535296+0000","last_fullsized":"2024-01-27T02:33:25.535296+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:31:18.212950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.520987+0000","last_change":"2024-01-27T02:33:24.533989+0000","last_active":"2024-01-27T02:33:25.520987+0000","last_peered":"2024-01-27T02:33:25.520987+0000","last_clean":"2024-01-27T02:33:25.520987+0000","last_became_active":"2024-01-27T02:33:24.533376+0000","last_became_peered":"2024-01-27T02:33:24.533376+0000","last_unstale":"2024-01-27T02:33:25.520987+0000","last_undegraded":"2024-01-27T02:33:25.520987+0000","last_fullsized":"2024-01-27T02:33:25.520987+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:30:46.762885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537321+0000","last_change":"2024-01-27T02:33:25.537043+0000","last_active":"2024-01-27T02:33:25.537321+0000","last_peered":"2024-01-27T02:33:25.537321+0000","last_clean":"2024-01-27T02:33:25.537321+0000","last_became_active":"2024-01-27T02:33:25.536423+0000","last_became_peered":"2024-01-27T02:33:25.536423+0000","last_unstale":"2024-01-27T02:33:25.537321+0000","last_undegraded":"2024-01-27T02:33:25.537321+0000","last_fullsized":"2024-01-27T02:33:25.537321+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:01:18.238537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.817338+0000","last_change":"2024-01-27T02:33:24.531842+0000","last_active":"2024-01-27T02:33:25.817338+0000","last_peered":"2024-01-27T02:33:25.817338+0000","last_clean":"2024-01-27T02:33:25.817338+0000","last_became_active":"2024-01-27T02:33:24.531031+0000","last_became_peered":"2024-01-27T02:33:24.531031+0000","last_unstale":"2024-01-27T02:33:25.817338+0000","last_undegraded":"2024-01-27T02:33:25.817338+0000","last_fullsized":"2024-01-27T02:33:25.817338+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:18:20.971740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.538247+0000","last_change":"2024-01-27T02:33:25.538247+0000","last_active":"2024-01-27T02:33:25.538247+0000","last_peered":"2024-01-27T02:33:25.538247+0000","last_clean":"2024-01-27T02:33:25.538247+0000","last_became_active":"2024-01-27T02:33:25.536842+0000","last_became_peered":"2024-01-27T02:33:25.536842+0000","last_unstale":"2024-01-27T02:33:25.538247+0000","last_undegraded":"2024-01-27T02:33:25.538247+0000","last_fullsized":"2024-01-27T02:33:25.538247+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:54:35.139188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.532068+0000","last_change":"2024-01-27T02:33:24.531957+0000","last_active":"2024-01-27T02:33:24.532068+0000","last_peered":"2024-01-27T02:33:24.532068+0000","last_clean":"2024-01-27T02:33:24.532068+0000","last_became_active":"2024-01-27T02:33:24.531702+0000","last_became_peered":"2024-01-27T02:33:24.531702+0000","last_unstale":"2024-01-27T02:33:24.532068+0000","last_undegraded":"2024-01-27T02:33:24.532068+0000","last_fullsized":"2024-01-27T02:33:24.532068+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:54:34.136790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.535497+0000","last_change":"2024-01-27T02:33:25.535497+0000","last_active":"2024-01-27T02:33:25.535497+0000","last_peered":"2024-01-27T02:33:25.535497+0000","last_clean":"2024-01-27T02:33:25.535497+0000","last_became_active":"2024-01-27T02:33:25.535003+0000","last_became_peered":"2024-01-27T02:33:25.535003+0000","last_unstale":"2024-01-27T02:33:25.535497+0000","last_undegraded":"2024-01-27T02:33:25.535497+0000","last_fullsized":"2024-01-27T02:33:25.535497+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:24:55.577678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.527992+0000","last_change":"2024-01-27T02:33:24.527645+0000","last_active":"2024-01-27T02:33:24.527992+0000","last_peered":"2024-01-27T02:33:24.527992+0000","last_clean":"2024-01-27T02:33:24.527992+0000","last_became_active":"2024-01-27T02:33:24.526947+0000","last_became_peered":"2024-01-27T02:33:24.526947+0000","last_unstale":"2024-01-27T02:33:24.527992+0000","last_undegraded":"2024-01-27T02:33:24.527992+0000","last_fullsized":"2024-01-27T02:33:24.527992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:38:21.994108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.539875+0000","last_change":"2024-01-27T02:33:25.539875+0000","last_active":"2024-01-27T02:33:25.539875+0000","last_peered":"2024-01-27T02:33:25.539875+0000","last_clean":"2024-01-27T02:33:25.539875+0000","last_became_active":"2024-01-27T02:33:25.539610+0000","last_became_peered":"2024-01-27T02:33:25.539610+0000","last_unstale":"2024-01-27T02:33:25.539875+0000","last_undegraded":"2024-01-27T02:33:25.539875+0000","last_fullsized":"2024-01-27T02:33:25.539875+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:18:21.974137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.531003+0000","last_change":"2024-01-27T02:33:24.530896+0000","last_active":"2024-01-27T02:33:24.531003+0000","last_peered":"2024-01-27T02:33:24.531003+0000","last_clean":"2024-01-27T02:33:24.531003+0000","last_became_active":"2024-01-27T02:33:24.530208+0000","last_became_peered":"2024-01-27T02:33:24.530208+0000","last_unstale":"2024-01-27T02:33:24.531003+0000","last_undegraded":"2024-01-27T02:33:24.531003+0000","last_fullsized":"2024-01-27T02:33:24.531003+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:36:13.217666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.531907+0000","last_change":"2024-01-27T02:33:25.531825+0000","last_active":"2024-01-27T02:33:25.531907+0000","last_peered":"2024-01-27T02:33:25.531907+0000","last_clean":"2024-01-27T02:33:25.531907+0000","last_became_active":"2024-01-27T02:33:25.531592+0000","last_became_peered":"2024-01-27T02:33:25.531592+0000","last_unstale":"2024-01-27T02:33:25.531907+0000","last_undegraded":"2024-01-27T02:33:25.531907+0000","last_fullsized":"2024-01-27T02:33:25.531907+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:12:38.630136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.520691+0000","last_change":"2024-01-27T02:33:24.525847+0000","last_active":"2024-01-27T02:33:25.520691+0000","last_peered":"2024-01-27T02:33:25.520691+0000","last_clean":"2024-01-27T02:33:25.520691+0000","last_became_active":"2024-01-27T02:33:24.525484+0000","last_became_peered":"2024-01-27T02:33:24.525484+0000","last_unstale":"2024-01-27T02:33:25.520691+0000","last_undegraded":"2024-01-27T02:33:25.520691+0000","last_fullsized":"2024-01-27T02:33:25.520691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:36:13.217666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.537015+0000","last_change":"2024-01-27T02:33:25.537015+0000","last_active":"2024-01-27T02:33:25.537015+0000","last_peered":"2024-01-27T02:33:25.537015+0000","last_clean":"2024-01-27T02:33:25.537015+0000","last_became_active":"2024-01-27T02:33:25.533348+0000","last_became_peered":"2024-01-27T02:33:25.533348+0000","last_unstale":"2024-01-27T02:33:25.537015+0000","last_undegraded":"2024-01-27T02:33:25.537015+0000","last_fullsized":"2024-01-27T02:33:25.537015+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:38:34.204456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.533423+0000","last_change":"2024-01-27T02:33:24.533343+0000","last_active":"2024-01-27T02:33:24.533423+0000","last_peered":"2024-01-27T02:33:24.533423+0000","last_clean":"2024-01-27T02:33:24.533423+0000","last_became_active":"2024-01-27T02:33:24.533124+0000","last_became_peered":"2024-01-27T02:33:24.533124+0000","last_unstale":"2024-01-27T02:33:24.533423+0000","last_undegraded":"2024-01-27T02:33:24.533423+0000","last_fullsized":"2024-01-27T02:33:24.533423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:02:33.613233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.534220+0000","last_change":"2024-01-27T02:33:25.534220+0000","last_active":"2024-01-27T02:33:25.534220+0000","last_peered":"2024-01-27T02:33:25.534220+0000","last_clean":"2024-01-27T02:33:25.534220+0000","last_became_active":"2024-01-27T02:33:25.533985+0000","last_became_peered":"2024-01-27T02:33:25.533985+0000","last_unstale":"2024-01-27T02:33:25.534220+0000","last_undegraded":"2024-01-27T02:33:25.534220+0000","last_fullsized":"2024-01-27T02:33:25.534220+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:52:44.112812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.529727+0000","last_change":"2024-01-27T02:33:24.529656+0000","last_active":"2024-01-27T02:33:24.529727+0000","last_peered":"2024-01-27T02:33:24.529727+0000","last_clean":"2024-01-27T02:33:24.529727+0000","last_became_active":"2024-01-27T02:33:24.529187+0000","last_became_peered":"2024-01-27T02:33:24.529187+0000","last_unstale":"2024-01-27T02:33:24.529727+0000","last_undegraded":"2024-01-27T02:33:24.529727+0000","last_fullsized":"2024-01-27T02:33:24.529727+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:56:28.933013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.532305+0000","last_change":"2024-01-27T02:33:25.532102+0000","last_active":"2024-01-27T02:33:25.532305+0000","last_peered":"2024-01-27T02:33:25.532305+0000","last_clean":"2024-01-27T02:33:25.532305+0000","last_became_active":"2024-01-27T02:33:25.531730+0000","last_became_peered":"2024-01-27T02:33:25.531730+0000","last_unstale":"2024-01-27T02:33:25.532305+0000","last_undegraded":"2024-01-27T02:33:25.532305+0000","last_fullsized":"2024-01-27T02:33:25.532305+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:54:00.310977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.532570+0000","last_change":"2024-01-27T02:33:24.531537+0000","last_active":"2024-01-27T02:33:24.532570+0000","last_peered":"2024-01-27T02:33:24.532570+0000","last_clean":"2024-01-27T02:33:24.532570+0000","last_became_active":"2024-01-27T02:33:24.531304+0000","last_became_peered":"2024-01-27T02:33:24.531304+0000","last_unstale":"2024-01-27T02:33:24.532570+0000","last_undegraded":"2024-01-27T02:33:24.532570+0000","last_fullsized":"2024-01-27T02:33:24.532570+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:06:09.289136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.538867+0000","last_change":"2024-01-27T02:33:25.538750+0000","last_active":"2024-01-27T02:33:25.538867+0000","last_peered":"2024-01-27T02:33:25.538867+0000","last_clean":"2024-01-27T02:33:25.538867+0000","last_became_active":"2024-01-27T02:33:25.538258+0000","last_became_peered":"2024-01-27T02:33:25.538258+0000","last_unstale":"2024-01-27T02:33:25.538867+0000","last_undegraded":"2024-01-27T02:33:25.538867+0000","last_fullsized":"2024-01-27T02:33:25.538867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:41:39.374936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.528500+0000","last_change":"2024-01-27T02:33:24.521497+0000","last_active":"2024-01-27T02:33:25.528500+0000","last_peered":"2024-01-27T02:33:25.528500+0000","last_clean":"2024-01-27T02:33:25.528500+0000","last_became_active":"2024-01-27T02:33:24.521258+0000","last_became_peered":"2024-01-27T02:33:24.521258+0000","last_unstale":"2024-01-27T02:33:25.528500+0000","last_undegraded":"2024-01-27T02:33:25.528500+0000","last_fullsized":"2024-01-27T02:33:25.528500+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:56:28.933013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.539841+0000","last_change":"2024-01-27T02:33:25.539841+0000","last_active":"2024-01-27T02:33:25.539841+0000","last_peered":"2024-01-27T02:33:25.539841+0000","last_clean":"2024-01-27T02:33:25.539841+0000","last_became_active":"2024-01-27T02:33:25.539662+0000","last_became_peered":"2024-01-27T02:33:25.539662+0000","last_unstale":"2024-01-27T02:33:25.539841+0000","last_undegraded":"2024-01-27T02:33:25.539841+0000","last_fullsized":"2024-01-27T02:33:25.539841+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:11:20.939912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.533124+0000","last_change":"2024-01-27T02:33:24.533048+0000","last_active":"2024-01-27T02:33:24.533124+0000","last_peered":"2024-01-27T02:33:24.533124+0000","last_clean":"2024-01-27T02:33:24.533124+0000","last_became_active":"2024-01-27T02:33:24.532787+0000","last_became_peered":"2024-01-27T02:33:24.532787+0000","last_unstale":"2024-01-27T02:33:24.533124+0000","last_undegraded":"2024-01-27T02:33:24.533124+0000","last_fullsized":"2024-01-27T02:33:24.533124+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:12:37.627739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.539181+0000","last_change":"2024-01-27T02:33:25.539115+0000","last_active":"2024-01-27T02:33:25.539181+0000","last_peered":"2024-01-27T02:33:25.539181+0000","last_clean":"2024-01-27T02:33:25.539181+0000","last_became_active":"2024-01-27T02:33:25.538926+0000","last_became_peered":"2024-01-27T02:33:25.538926+0000","last_unstale":"2024-01-27T02:33:25.539181+0000","last_undegraded":"2024-01-27T02:33:25.539181+0000","last_fullsized":"2024-01-27T02:33:25.539181+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:38:34.204456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.529675+0000","last_change":"2024-01-27T02:33:24.529505+0000","last_active":"2024-01-27T02:33:24.529675+0000","last_peered":"2024-01-27T02:33:24.529675+0000","last_clean":"2024-01-27T02:33:24.529675+0000","last_became_active":"2024-01-27T02:33:24.528817+0000","last_became_peered":"2024-01-27T02:33:24.528817+0000","last_unstale":"2024-01-27T02:33:24.529675+0000","last_undegraded":"2024-01-27T02:33:24.529675+0000","last_fullsized":"2024-01-27T02:33:24.529675+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:50:15.568071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":8,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.529959+0000","last_change":"2024-01-27T02:33:25.529959+0000","last_active":"2024-01-27T02:33:25.529959+0000","last_peered":"2024-01-27T02:33:25.529959+0000","last_clean":"2024-01-27T02:33:25.529959+0000","last_became_active":"2024-01-27T02:33:25.529673+0000","last_became_peered":"2024-01-27T02:33:25.529673+0000","last_unstale":"2024-01-27T02:33:25.529959+0000","last_undegraded":"2024-01-27T02:33:25.529959+0000","last_fullsized":"2024-01-27T02:33:25.529959+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:24.511976+0000","last_clean_scrub_stamp":"2024-01-27T02:33:24.511976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:13:06.294428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-01-27T02:33:25.817096+0000","last_change":"2024-01-27T02:33:24.528853+0000","last_active":"2024-01-27T02:33:25.817096+0000","last_peered":"2024-01-27T02:33:25.817096+0000","last_clean":"2024-01-27T02:33:25.817096+0000","last_became_active":"2024-01-27T02:33:24.528580+0000","last_became_peered":"2024-01-27T02:33:24.528580+0000","last_unstale":"2024-01-27T02:33:25.817096+0000","last_undegraded":"2024-01-27T02:33:25.817096+0000","last_fullsized":"2024-01-27T02:33:25.817096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:33:23.509579+0000","last_clean_scrub_stamp":"2024-01-27T02:33:23.509579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:58:32.272587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":181,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-27T02:33:24.526836+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:33:24.526836+0000","last_peered":"2024-01-27T02:33:24.526836+0000","last_clean":"2024-01-27T02:33:24.526836+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:33:24.526836+0000","last_undegraded":"2024-01-27T02:33:24.526836+0000","last_fullsized":"2024-01-27T02:33:24.526836+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":778,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194533,"statfs":{"total":200000000,"available":199202763,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.74199999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.751},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.58199999999999996}]},{"osd":2,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.625}]},{"osd":3,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.56999999999999995}]},{"osd":4,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.43099999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.53600000000000003}]},{"osd":6,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.47899999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":202,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195109,"statfs":{"total":200000000,"available":199792577,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.51700000000000002}]},{"osd":1,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.503},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.42999999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.47799999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.46800000000000003}]},{"osd":4,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.379}]},{"osd":5,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.52600000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.36499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":202,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195110,"statfs":{"total":200000000,"available":199793151,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.376},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.41599999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.45600000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.503},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.53000000000000003}]},{"osd":4,"last update":"Sat Jan 27 02:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.78500000000000003}]},{"osd":5,"last update":"Sat Jan 27 02:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59899999999999998}]},{"osd":6,"last update":"Sat Jan 27 02:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.57599999999999996}]},{"osd":7,"last update":"Sat Jan 27 02:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.66000000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":778,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194534,"statfs":{"total":200000000,"available":199202819,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.433}]},{"osd":2,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"last":0.379},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.46999999999999997}]},{"osd":3,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.41999999999999998}]},{"osd":4,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.66900000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.747},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.80500000000000005}]},{"osd":6,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.78200000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.59799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":201,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195110,"statfs":{"total":200000000,"available":199793221,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":6.2729999999999997,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":0.41399999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.314}]},{"osd":3,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.371}]},{"osd":4,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":3.581,"5min":3.581,"15min":3.581},"last":0.57499999999999996}]},{"osd":5,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":3.5369999999999999,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.622},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.53400000000000003}]},{"osd":6,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.52700000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.59299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":202,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195109,"statfs":{"total":200000000,"available":199792611,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.69499999999999995}]},{"osd":1,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.35199999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.52300000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.70799999999999996}]},{"osd":5,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.56200000000000006}]},{"osd":6,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.53900000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.68300000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":203,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195109,"statfs":{"total":200000000,"available":199791685,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.61299999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.81499999999999995}]},{"osd":2,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.73999999999999999}]},{"osd":3,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.72099999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.66700000000000004}]},{"osd":6,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.69299999999999995}]},{"osd":7,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.76100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":202,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195110,"statfs":{"total":200000000,"available":199793151,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.50800000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.68000000000000005}]},{"osd":2,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.73099999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.66600000000000004}]},{"osd":4,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.439}]},{"osd":6,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.36699999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:32:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.45700000000000002}]}]}],"pool_statfs":[]}} 2024-01-27T02:33:30.410 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-27T02:33:30.410 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-27T02:33:30.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.585+0000 7f380ca00700 1 Processor -- start 2024-01-27T02:33:30.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.585+0000 7f380ca00700 1 -- start start 2024-01-27T02:33:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.586+0000 7f380ca00700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3808159e40 0x7f3808068880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.586+0000 7f380ca00700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3808068dc0 0x7f38080c4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.586+0000 7f380ca00700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38080ab960 con 0x7f380815a210 2024-01-27T02:33:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.586+0000 7f380ca00700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38080ab660 con 0x7f3808159e40 2024-01-27T02:33:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.586+0000 7f380ca00700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3808069350 con 0x7f3808068dc0 2024-01-27T02:33:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.587+0000 7f3806d9d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3808159e40 0x7f3808068880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.587+0000 7f380659c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3808068dc0 0x7f38080c4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.587+0000 7f380759e700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f380815a210 0x7f38080cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:50348/0 (socket says 172.21.15.7:50348) 2024-01-27T02:33:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.587+0000 7f3806d9d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3808159e40 0x7f3808068880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:56214/0 (socket says 172.21.15.7:56214) 2024-01-27T02:33:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.587+0000 7f380659c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3808068dc0 0x7f38080c4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:33552/0 (socket says 172.21.15.7:33552) 2024-01-27T02:33:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.587+0000 7f380759e700 1 -- 172.21.15.7:0/2159125837 learned_addr learned my addr 172.21.15.7:0/2159125837 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.587+0000 7f3806d9d700 1 -- 172.21.15.7:0/2159125837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3808068dc0 msgr2=0x7f38080c4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.587+0000 7f3806d9d700 1 --2- 172.21.15.7:0/2159125837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3808068dc0 0x7f38080c4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:30.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.587+0000 7f3806d9d700 1 -- 172.21.15.7:0/2159125837 >> v1:172.21.15.188:6789/0 conn(0x7f380815a210 legacy=0x7f38080cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:30.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.587+0000 7f3806d9d700 1 -- 172.21.15.7:0/2159125837 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3808069630 con 0x7f3808159e40 2024-01-27T02:33:30.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.587+0000 7f3806d9d700 1 --2- 172.21.15.7:0/2159125837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3808159e40 0x7f3808068880 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f37f80098e0 tx=0x7f37f80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a51479a1b22b83b4 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:30.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.587+0000 7f3805d9b700 1 -- 172.21.15.7:0/2159125837 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f800b640 con 0x7f3808159e40 2024-01-27T02:33:30.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.587+0000 7f3805d9b700 1 -- 172.21.15.7:0/2159125837 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f800b7a0 con 0x7f3808159e40 2024-01-27T02:33:30.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.587+0000 7f3805d9b700 1 -- 172.21.15.7:0/2159125837 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f800bb40 con 0x7f3808159e40 2024-01-27T02:33:30.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.588+0000 7f380ca00700 1 -- 172.21.15.7:0/2159125837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3808159e40 msgr2=0x7f3808068880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:30.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.588+0000 7f380ca00700 1 --2- 172.21.15.7:0/2159125837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3808159e40 0x7f3808068880 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f37f80098e0 tx=0x7f37f80091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:30.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.588+0000 7f380ca00700 1 -- 172.21.15.7:0/2159125837 shutdown_connections 2024-01-27T02:33:30.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.588+0000 7f380ca00700 1 --2- 172.21.15.7:0/2159125837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3808159e40 0x7f3808068880 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:30.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.588+0000 7f380ca00700 1 --2- 172.21.15.7:0/2159125837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3808068dc0 0x7f38080c4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:30.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.588+0000 7f380ca00700 1 -- 172.21.15.7:0/2159125837 >> 172.21.15.7:0/2159125837 conn(0x7f3808097e60 msgr2=0x7f38080711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:30.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.588+0000 7f380ca00700 1 -- 172.21.15.7:0/2159125837 shutdown_connections 2024-01-27T02:33:30.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.588+0000 7f380ca00700 1 -- 172.21.15.7:0/2159125837 wait complete. 2024-01-27T02:33:30.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.589+0000 7f380ca00700 1 Processor -- start 2024-01-27T02:33:30.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.589+0000 7f380ca00700 1 -- start start 2024-01-27T02:33:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.590+0000 7f380ca00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3808068dc0 0x7f38080ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.590+0000 7f380ca00700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3808159e40 0x7f38080aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.590+0000 7f380ca00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f380815a210 0x7f38081543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.590+0000 7f380ca00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38080c9d80 con 0x7f3808068dc0 2024-01-27T02:33:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.590+0000 7f380ca00700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38080cb4c0 con 0x7f380815a210 2024-01-27T02:33:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.590+0000 7f380ca00700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38080acde0 con 0x7f3808159e40 2024-01-27T02:33:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.590+0000 7f3806d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3808068dc0 0x7f38080ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.590+0000 7f3806d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3808068dc0 0x7f38080ac7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38010/0 (socket says 172.21.15.7:38010) 2024-01-27T02:33:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.590+0000 7f3806d9d700 1 -- 172.21.15.7:0/3133586300 learned_addr learned my addr 172.21.15.7:0/3133586300 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.591+0000 7f380759e700 1 --2- 172.21.15.7:0/3133586300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f380815a210 0x7f38081543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.591+0000 7f380659c700 1 --2- 172.21.15.7:0/3133586300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3808159e40 0x7f38080aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.591+0000 7f3806d9d700 1 -- 172.21.15.7:0/3133586300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3808159e40 msgr2=0x7f38080aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.591+0000 7f3806d9d700 1 --2- 172.21.15.7:0/3133586300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3808159e40 0x7f38080aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.591+0000 7f3806d9d700 1 -- 172.21.15.7:0/3133586300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f380815a210 msgr2=0x7f38081543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.591+0000 7f3806d9d700 1 --2- 172.21.15.7:0/3133586300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f380815a210 0x7f38081543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.591+0000 7f3806d9d700 1 -- 172.21.15.7:0/3133586300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f8009420 con 0x7f3808068dc0 2024-01-27T02:33:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.591+0000 7f3806d9d700 1 --2- 172.21.15.7:0/3133586300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3808068dc0 0x7f38080ac7d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f380806ab20 tx=0x7f37f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.591+0000 7f37eeffd700 1 -- 172.21.15.7:0/3133586300 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f801bb70 con 0x7f3808068dc0 2024-01-27T02:33:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.591+0000 7f37eeffd700 1 -- 172.21.15.7:0/3133586300 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f801bcd0 con 0x7f3808068dc0 2024-01-27T02:33:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.591+0000 7f380ca00700 1 -- 172.21.15.7:0/3133586300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38080ad0c0 con 0x7f3808068dc0 2024-01-27T02:33:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.591+0000 7f37eeffd700 1 -- 172.21.15.7:0/3133586300 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f8011670 con 0x7f3808068dc0 2024-01-27T02:33:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.591+0000 7f380ca00700 1 -- 172.21.15.7:0/3133586300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38080ad680 con 0x7f3808068dc0 2024-01-27T02:33:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.592+0000 7f380ca00700 1 -- 172.21.15.7:0/3133586300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f380804c710 con 0x7f3808068dc0 2024-01-27T02:33:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.593+0000 7f37eeffd700 1 -- 172.21.15.7:0/3133586300 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f37f8011a20 con 0x7f3808068dc0 2024-01-27T02:33:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.593+0000 7f37eeffd700 1 --2- 172.21.15.7:0/3133586300 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f37f4066f30 0x7f37f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.593+0000 7f37eeffd700 1 -- 172.21.15.7:0/3133586300 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f37f80859f0 con 0x7f3808068dc0 2024-01-27T02:33:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.593+0000 7f380659c700 1 --2- 172.21.15.7:0/3133586300 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f37f4066f30 0x7f37f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.594+0000 7f380659c700 1 --2- 172.21.15.7:0/3133586300 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f37f4066f30 0x7f37f40693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f38080acd10 tx=0x7f37fc0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:30.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.598+0000 7f37eeffd700 1 -- 172.21.15.7:0/3133586300 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37f8052bc0 con 0x7f3808068dc0 2024-01-27T02:33:30.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.886+0000 7f380ca00700 1 -- 172.21.15.7:0/3133586300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3808069910 con 0x7f3808068dc0 2024-01-27T02:33:30.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.890+0000 7f37eeffd700 1 -- 172.21.15.7:0/3133586300 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7f37f80561e0 con 0x7f3808068dc0 2024-01-27T02:33:30.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.893+0000 7f380ca00700 1 -- 172.21.15.7:0/3133586300 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f37f4066f30 msgr2=0x7f37f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:30.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.893+0000 7f380ca00700 1 --2- 172.21.15.7:0/3133586300 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f37f4066f30 0x7f37f40693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f38080acd10 tx=0x7f37fc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:30.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.893+0000 7f380ca00700 1 -- 172.21.15.7:0/3133586300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3808068dc0 msgr2=0x7f38080ac7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:30.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.893+0000 7f380ca00700 1 --2- 172.21.15.7:0/3133586300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3808068dc0 0x7f38080ac7d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f380806ab20 tx=0x7f37f8017000 comp rx=0 tx=0).stop 2024-01-27T02:33:30.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.893+0000 7f380ca00700 1 -- 172.21.15.7:0/3133586300 shutdown_connections 2024-01-27T02:33:30.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.893+0000 7f380ca00700 1 --2- 172.21.15.7:0/3133586300 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f37f4066f30 0x7f37f40693e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:30.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.893+0000 7f380ca00700 1 --2- 172.21.15.7:0/3133586300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f380815a210 0x7f38081543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:30.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.893+0000 7f380ca00700 1 --2- 172.21.15.7:0/3133586300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3808068dc0 0x7f38080ac7d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:30.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.893+0000 7f380ca00700 1 --2- 172.21.15.7:0/3133586300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3808159e40 0x7f38080aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:30.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.893+0000 7f380ca00700 1 -- 172.21.15.7:0/3133586300 >> 172.21.15.7:0/3133586300 conn(0x7f3808097e60 msgr2=0x7f38080c4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.893+0000 7f380ca00700 1 -- 172.21.15.7:0/3133586300 shutdown_connections 2024-01-27T02:33:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:30.893+0000 7f380ca00700 1 -- 172.21.15.7:0/3133586300 wait complete. 2024-01-27T02:33:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-01-27T02:33:30.907 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:33:31.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.038+0000 7f6f68d75700 1 Processor -- start 2024-01-27T02:33:31.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.038+0000 7f6f68d75700 1 -- start start 2024-01-27T02:33:31.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.038+0000 7f6f68d75700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6f6415a100 0x7f6f640cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.038+0000 7f6f68d75700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6f640cb5b0 0x7f6f640c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.038+0000 7f6f68d75700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f640c4f50 con 0x7f6f640c4af0 2024-01-27T02:33:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.038+0000 7f6f68d75700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f640ce5e0 con 0x7f6f640cb5b0 2024-01-27T02:33:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.038+0000 7f6f68d75700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f640cd9d0 con 0x7f6f6415a100 2024-01-27T02:33:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.038+0000 7f6f61d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6f640cb5b0 0x7f6f640c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.038+0000 7f6f61d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6f640cb5b0 0x7f6f640c45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38018/0 (socket says 172.21.15.7:38018) 2024-01-27T02:33:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.038+0000 7f6f61d9b700 1 -- 172.21.15.7:0/517040446 learned_addr learned my addr 172.21.15.7:0/517040446 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.039+0000 7f6f6259c700 1 --2- 172.21.15.7:0/517040446 >> v2:172.21.15.188:3300/0 conn(0x7f6f6415a100 0x7f6f640cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.039+0000 7f6f61d9b700 1 -- 172.21.15.7:0/517040446 >> v2:172.21.15.188:3300/0 conn(0x7f6f6415a100 msgr2=0x7f6f640cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.039+0000 7f6f61d9b700 1 --2- 172.21.15.7:0/517040446 >> v2:172.21.15.188:3300/0 conn(0x7f6f6415a100 0x7f6f640cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.039+0000 7f6f61d9b700 1 -- 172.21.15.7:0/517040446 >> v1:172.21.15.188:6789/0 conn(0x7f6f640c4af0 legacy=0x7f6f640c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.039+0000 7f6f61d9b700 1 -- 172.21.15.7:0/517040446 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f640cdcb0 con 0x7f6f640cb5b0 2024-01-27T02:33:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.039+0000 7f6f61d9b700 1 --2- 172.21.15.7:0/517040446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f640cb5b0 0x7f6f640c45b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6f6406d710 tx=0x7f6f58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fab98af36f1cb2 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.039+0000 7f6f6159a700 1 -- 172.21.15.7:0/517040446 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5800b860 con 0x7f6f640cb5b0 2024-01-27T02:33:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.039+0000 7f6f6159a700 1 -- 172.21.15.7:0/517040446 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5800b9c0 con 0x7f6f640cb5b0 2024-01-27T02:33:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.039+0000 7f6f6159a700 1 -- 172.21.15.7:0/517040446 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5800bd60 con 0x7f6f640cb5b0 2024-01-27T02:33:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.041+0000 7f6f68d75700 1 -- 172.21.15.7:0/517040446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f640cb5b0 msgr2=0x7f6f640c45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.041+0000 7f6f68d75700 1 --2- 172.21.15.7:0/517040446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f640cb5b0 0x7f6f640c45b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6f6406d710 tx=0x7f6f58009250 comp rx=0 tx=0).stop 2024-01-27T02:33:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.041+0000 7f6f68d75700 1 -- 172.21.15.7:0/517040446 shutdown_connections 2024-01-27T02:33:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.041+0000 7f6f68d75700 1 --2- 172.21.15.7:0/517040446 >> v2:172.21.15.188:3300/0 conn(0x7f6f6415a100 0x7f6f640cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.041+0000 7f6f68d75700 1 --2- 172.21.15.7:0/517040446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f640cb5b0 0x7f6f640c45b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.041+0000 7f6f68d75700 1 -- 172.21.15.7:0/517040446 >> 172.21.15.7:0/517040446 conn(0x7f6f64098020 msgr2=0x7f6f640711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.041+0000 7f6f68d75700 1 -- 172.21.15.7:0/517040446 shutdown_connections 2024-01-27T02:33:31.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.041+0000 7f6f68d75700 1 -- 172.21.15.7:0/517040446 wait complete. 2024-01-27T02:33:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.041+0000 7f6f68d75700 1 Processor -- start 2024-01-27T02:33:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.041+0000 7f6f68d75700 1 -- start start 2024-01-27T02:33:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.043+0000 7f6f68d75700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f640c4af0 0x7f6f6414ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.043+0000 7f6f68d75700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f640cb5b0 0x7f6f6415c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.043+0000 7f6f68d75700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f6415a100 0x7f6f6415b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.043+0000 7f6f68d75700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f640c6e30 con 0x7f6f6415a100 2024-01-27T02:33:31.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.043+0000 7f6f68d75700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6414f1d0 con 0x7f6f640cb5b0 2024-01-27T02:33:31.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.043+0000 7f6f68d75700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f640c6220 con 0x7f6f640c4af0 2024-01-27T02:33:31.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.043+0000 7f6f62d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f6415a100 0x7f6f6415b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.043+0000 7f6f62d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f6415a100 0x7f6f6415b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38022/0 (socket says 172.21.15.7:38022) 2024-01-27T02:33:31.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.043+0000 7f6f62d9d700 1 -- 172.21.15.7:0/2693077093 learned_addr learned my addr 172.21.15.7:0/2693077093 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:31.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.043+0000 7f6f61d9b700 1 --2- 172.21.15.7:0/2693077093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f640cb5b0 0x7f6f6415c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.043+0000 7f6f6259c700 1 --2- 172.21.15.7:0/2693077093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f640c4af0 0x7f6f6414ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.044+0000 7f6f62d9d700 1 -- 172.21.15.7:0/2693077093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f640c4af0 msgr2=0x7f6f6414ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.044+0000 7f6f62d9d700 1 --2- 172.21.15.7:0/2693077093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f640c4af0 0x7f6f6414ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.044+0000 7f6f62d9d700 1 -- 172.21.15.7:0/2693077093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f640cb5b0 msgr2=0x7f6f6415c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.044+0000 7f6f62d9d700 1 --2- 172.21.15.7:0/2693077093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f640cb5b0 0x7f6f6415c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.044+0000 7f6f62d9d700 1 -- 172.21.15.7:0/2693077093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f580094f0 con 0x7f6f6415a100 2024-01-27T02:33:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.044+0000 7f6f62d9d700 1 --2- 172.21.15.7:0/2693077093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f6415a100 0x7f6f6415b430 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00c8e0 tx=0x7f6f5c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.044+0000 7f6f527fc700 1 -- 172.21.15.7:0/2693077093 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c012630 con 0x7f6f6415a100 2024-01-27T02:33:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.044+0000 7f6f527fc700 1 -- 172.21.15.7:0/2693077093 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5c012790 con 0x7f6f6415a100 2024-01-27T02:33:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.044+0000 7f6f527fc700 1 -- 172.21.15.7:0/2693077093 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c012b50 con 0x7f6f6415a100 2024-01-27T02:33:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.044+0000 7f6f68d75700 1 -- 172.21.15.7:0/2693077093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f640c6500 con 0x7f6f6415a100 2024-01-27T02:33:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.044+0000 7f6f68d75700 1 -- 172.21.15.7:0/2693077093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f640c6ac0 con 0x7f6f6415a100 2024-01-27T02:33:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.045+0000 7f6f68d75700 1 -- 172.21.15.7:0/2693077093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f640c6fa0 con 0x7f6f6415a100 2024-01-27T02:33:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.046+0000 7f6f527fc700 1 -- 172.21.15.7:0/2693077093 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6f5c00a3b0 con 0x7f6f6415a100 2024-01-27T02:33:31.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.046+0000 7f6f527fc700 1 --2- 172.21.15.7:0/2693077093 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6f44066f30 0x7f6f440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.046+0000 7f6f527fc700 1 -- 172.21.15.7:0/2693077093 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6f5c084640 con 0x7f6f6415a100 2024-01-27T02:33:31.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.046+0000 7f6f6259c700 1 --2- 172.21.15.7:0/2693077093 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6f44066f30 0x7f6f440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.047+0000 7f6f6259c700 1 --2- 172.21.15.7:0/2693077093 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6f44066f30 0x7f6f440693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6f54006fd0 tx=0x7f6f54008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:31.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.051+0000 7f6f527fc700 1 -- 172.21.15.7:0/2693077093 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f5c055340 con 0x7f6f6415a100 2024-01-27T02:33:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.302+0000 7f6f68d75700 1 -- 172.21.15.7:0/2693077093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f640c4f50 con 0x7f6f6415a100 2024-01-27T02:33:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.303+0000 7f6f527fc700 1 -- 172.21.15.7:0/2693077093 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f6f640c4f50 con 0x7f6f6415a100 2024-01-27T02:33:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:31.304 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4883,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/198511042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":198511042},{"type":"v1","addr":"172.21.15.188:6836","nonce":198511042}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":4886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1222999254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1222999254},{"type":"v1","addr":"172.21.15.188:6837","nonce":1222999254}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2474016526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2474016526},{"type":"v1","addr":"172.21.15.7:6834","nonce":2474016526}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:33:24.950288+0000","modified":"2024-01-27T02:33:28.814599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5262":{"gid":5262,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/999728998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":999728998},{"type":"v1","addr":"172.21.15.7:6835","nonce":999728998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-27T02:33:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.305+0000 7f6f68d75700 1 -- 172.21.15.7:0/2693077093 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6f44066f30 msgr2=0x7f6f440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.305+0000 7f6f68d75700 1 --2- 172.21.15.7:0/2693077093 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6f44066f30 0x7f6f440693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6f54006fd0 tx=0x7f6f54008040 comp rx=0 tx=0).stop 2024-01-27T02:33:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.305+0000 7f6f68d75700 1 -- 172.21.15.7:0/2693077093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f6415a100 msgr2=0x7f6f6415b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.305+0000 7f6f68d75700 1 --2- 172.21.15.7:0/2693077093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f6415a100 0x7f6f6415b430 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00c8e0 tx=0x7f6f5c001710 comp rx=0 tx=0).stop 2024-01-27T02:33:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.305+0000 7f6f68d75700 1 -- 172.21.15.7:0/2693077093 shutdown_connections 2024-01-27T02:33:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.305+0000 7f6f68d75700 1 --2- 172.21.15.7:0/2693077093 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6f44066f30 0x7f6f440693e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.305+0000 7f6f68d75700 1 --2- 172.21.15.7:0/2693077093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f6415a100 0x7f6f6415b430 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.305+0000 7f6f68d75700 1 --2- 172.21.15.7:0/2693077093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f640c4af0 0x7f6f6414ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.305+0000 7f6f68d75700 1 --2- 172.21.15.7:0/2693077093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f640cb5b0 0x7f6f6415c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.305+0000 7f6f68d75700 1 -- 172.21.15.7:0/2693077093 >> 172.21.15.7:0/2693077093 conn(0x7f6f64098020 msgr2=0x7f6f640cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.305+0000 7f6f68d75700 1 -- 172.21.15.7:0/2693077093 shutdown_connections 2024-01-27T02:33:31.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.305+0000 7f6f68d75700 1 -- 172.21.15.7:0/2693077093 wait complete. 2024-01-27T02:33:31.307 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-01-27T02:33:31.485 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-01-27T02:33:31.485+0000 7fc79e073700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-27T02:33:31.722 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-27T02:33:24.950288+0000', 'modified': '2024-01-27T02:33:28.814599+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5262}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5262': {'gid': 5262, 'name': 'c', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6835/999728998', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6833', 'nonce': 999728998}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 999728998}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-27T02:33:31.722 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-27T02:33:31.723 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-27T02:33:31.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.866+0000 7ff072990700 1 Processor -- start 2024-01-27T02:33:31.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.866+0000 7ff072990700 1 -- start start 2024-01-27T02:33:31.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.867+0000 7ff072990700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff06c0c4cc0 0x7ff06c0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.867+0000 7ff072990700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff06c0ca540 0x7ff06c163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.867+0000 7ff072990700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff06c0ce5e0 con 0x7ff06c15a100 2024-01-27T02:33:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.867+0000 7ff072990700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff06c0cd9d0 con 0x7ff06c0c4cc0 2024-01-27T02:33:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.867+0000 7ff072990700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff06c0cdcb0 con 0x7ff06c0ca540 2024-01-27T02:33:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.867+0000 7ff06b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff06c0c4cc0 0x7ff06c0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.867+0000 7ff06b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff06c0c4cc0 0x7ff06c0cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48170/0 (socket says 172.21.15.7:48170) 2024-01-27T02:33:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.867+0000 7ff06b7fe700 1 -- 172.21.15.7:0/368747939 learned_addr learned my addr 172.21.15.7:0/368747939 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.867+0000 7ff070f2d700 1 --2- 172.21.15.7:0/368747939 >> v2:172.21.15.188:3300/0 conn(0x7ff06c0ca540 0x7ff06c163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.868+0000 7ff06b7fe700 1 -- 172.21.15.7:0/368747939 >> v2:172.21.15.188:3300/0 conn(0x7ff06c0ca540 msgr2=0x7ff06c163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:31.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.868+0000 7ff06b7fe700 1 --2- 172.21.15.7:0/368747939 >> v2:172.21.15.188:3300/0 conn(0x7ff06c0ca540 0x7ff06c163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.868+0000 7ff06b7fe700 1 -- 172.21.15.7:0/368747939 >> v1:172.21.15.188:6789/0 conn(0x7ff06c15a100 legacy=0x7ff06c0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:31.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.868+0000 7ff06b7fe700 1 -- 172.21.15.7:0/368747939 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06c0cdf90 con 0x7ff06c0c4cc0 2024-01-27T02:33:31.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.868+0000 7ff06b7fe700 1 --2- 172.21.15.7:0/368747939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c0c4cc0 0x7ff06c0cd2f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff06c069b70 tx=0x7ff060009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bffadc77dd57f4cd server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:31.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.868+0000 7ff06affd700 1 -- 172.21.15.7:0/368747939 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060009e70 con 0x7ff06c0c4cc0 2024-01-27T02:33:31.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.868+0000 7ff06affd700 1 -- 172.21.15.7:0/368747939 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06000b860 con 0x7ff06c0c4cc0 2024-01-27T02:33:31.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.868+0000 7ff06affd700 1 -- 172.21.15.7:0/368747939 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06000bc00 con 0x7ff06c0c4cc0 2024-01-27T02:33:31.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.869+0000 7ff072990700 1 -- 172.21.15.7:0/368747939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c0c4cc0 msgr2=0x7ff06c0cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:31.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.869+0000 7ff072990700 1 --2- 172.21.15.7:0/368747939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c0c4cc0 0x7ff06c0cd2f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff06c069b70 tx=0x7ff060009250 comp rx=0 tx=0).stop 2024-01-27T02:33:31.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.869+0000 7ff072990700 1 -- 172.21.15.7:0/368747939 shutdown_connections 2024-01-27T02:33:31.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.869+0000 7ff072990700 1 --2- 172.21.15.7:0/368747939 >> v2:172.21.15.188:3300/0 conn(0x7ff06c0ca540 0x7ff06c163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.869+0000 7ff072990700 1 --2- 172.21.15.7:0/368747939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c0c4cc0 0x7ff06c0cd2f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.869+0000 7ff072990700 1 -- 172.21.15.7:0/368747939 >> 172.21.15.7:0/368747939 conn(0x7ff06c098020 msgr2=0x7ff06c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:31.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.869+0000 7ff072990700 1 -- 172.21.15.7:0/368747939 shutdown_connections 2024-01-27T02:33:31.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.869+0000 7ff072990700 1 -- 172.21.15.7:0/368747939 wait complete. 2024-01-27T02:33:31.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.870+0000 7ff072990700 1 Processor -- start 2024-01-27T02:33:31.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.871+0000 7ff072990700 1 -- start start 2024-01-27T02:33:31.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.872+0000 7ff072990700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff06c0c4cc0 0x7ff06c0cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.872+0000 7ff072990700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c0ca540 0x7ff06c0cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.872+0000 7ff072990700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff06c15a100 0x7ff06c14d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.872+0000 7ff072990700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c153f80 con 0x7ff06c0ca540 2024-01-27T02:33:31.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.872+0000 7ff072990700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c153650 con 0x7ff06c15a100 2024-01-27T02:33:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.872+0000 7ff072990700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c153930 con 0x7ff06c0c4cc0 2024-01-27T02:33:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.872+0000 7ff06b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c0ca540 0x7ff06c0cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.872+0000 7ff06b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c0ca540 0x7ff06c0cc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48178/0 (socket says 172.21.15.7:48178) 2024-01-27T02:33:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.872+0000 7ff06b7fe700 1 -- 172.21.15.7:0/980114898 learned_addr learned my addr 172.21.15.7:0/980114898 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.872+0000 7ff06bfff700 1 --2- 172.21.15.7:0/980114898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff06c0c4cc0 0x7ff06c0cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.872+0000 7ff070f2d700 1 --2- 172.21.15.7:0/980114898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff06c15a100 0x7ff06c14d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.872+0000 7ff06b7fe700 1 -- 172.21.15.7:0/980114898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff06c0c4cc0 msgr2=0x7ff06c0cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.872+0000 7ff06b7fe700 1 --2- 172.21.15.7:0/980114898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff06c0c4cc0 0x7ff06c0cb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.872+0000 7ff06b7fe700 1 -- 172.21.15.7:0/980114898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff06c15a100 msgr2=0x7ff06c14d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.872+0000 7ff06b7fe700 1 --2- 172.21.15.7:0/980114898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff06c15a100 0x7ff06c14d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.872+0000 7ff06b7fe700 1 -- 172.21.15.7:0/980114898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0600094f0 con 0x7ff06c0ca540 2024-01-27T02:33:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.872+0000 7ff070f2d700 1 --2- 172.21.15.7:0/980114898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff06c15a100 0x7ff06c14d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.872+0000 7ff06b7fe700 1 --2- 172.21.15.7:0/980114898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c0ca540 0x7ff06c0cc030 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff06c06ea80 tx=0x7ff060017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:31.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.872+0000 7ff053fff700 1 -- 172.21.15.7:0/980114898 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06000b9d0 con 0x7ff06c0ca540 2024-01-27T02:33:31.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.873+0000 7ff053fff700 1 -- 172.21.15.7:0/980114898 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06000bb30 con 0x7ff06c0ca540 2024-01-27T02:33:31.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.873+0000 7ff053fff700 1 -- 172.21.15.7:0/980114898 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060025670 con 0x7ff06c0ca540 2024-01-27T02:33:31.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.873+0000 7ff072990700 1 -- 172.21.15.7:0/980114898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff06c153c10 con 0x7ff06c0ca540 2024-01-27T02:33:31.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.873+0000 7ff072990700 1 -- 172.21.15.7:0/980114898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06c0cc730 con 0x7ff06c0ca540 2024-01-27T02:33:31.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.874+0000 7ff053fff700 1 -- 172.21.15.7:0/980114898 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff060023070 con 0x7ff06c0ca540 2024-01-27T02:33:31.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.874+0000 7ff053fff700 1 --2- 172.21.15.7:0/980114898 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff058066f30 0x7ff0580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.875+0000 7ff053fff700 1 -- 172.21.15.7:0/980114898 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff060085a80 con 0x7ff06c0ca540 2024-01-27T02:33:31.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.875+0000 7ff06bfff700 1 --2- 172.21.15.7:0/980114898 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff058066f30 0x7ff0580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.875+0000 7ff06bfff700 1 --2- 172.21.15.7:0/980114898 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff058066f30 0x7ff0580693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0039e0 tx=0x7ff05c007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:31.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.877+0000 7ff07bc93b80 1 Processor -- start 2024-01-27T02:33:31.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.877+0000 7ff07bc93b80 1 -- start start 2024-01-27T02:33:31.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.878+0000 7ff07bc93b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb72d77010 0x55cb72d773e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.878+0000 7ff07bc93b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72dbf360 0x55cb72da1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.878+0000 7ff07bc93b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb72d765d0 0x55cb72da1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.878+0000 7ff07bc93b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb72c7a9f0 con 0x55cb72dbf360 2024-01-27T02:33:31.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.878+0000 7ff07bc93b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb72ccc130 con 0x55cb72d765d0 2024-01-27T02:33:31.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.878+0000 7ff07bc93b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb72cc9930 con 0x55cb72d77010 2024-01-27T02:33:31.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.878+0000 7ff06b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72dbf360 0x55cb72da1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.878+0000 7ff06b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72dbf360 0x55cb72da1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48190/0 (socket says 172.21.15.7:48190) 2024-01-27T02:33:31.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.878+0000 7ff06b7fe700 1 -- 172.21.15.7:0/3508319364 learned_addr learned my addr 172.21.15.7:0/3508319364 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:31.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.878+0000 7ff070f2d700 1 --2- 172.21.15.7:0/3508319364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb72d765d0 0x55cb72da1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.878+0000 7ff06bfff700 1 --2- 172.21.15.7:0/3508319364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb72d77010 0x55cb72d773e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.878+0000 7ff06b7fe700 1 -- 172.21.15.7:0/3508319364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb72d77010 msgr2=0x55cb72d773e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:31.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.878+0000 7ff06b7fe700 1 --2- 172.21.15.7:0/3508319364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb72d77010 0x55cb72d773e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.878+0000 7ff06b7fe700 1 -- 172.21.15.7:0/3508319364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb72d765d0 msgr2=0x55cb72da1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:31.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.878+0000 7ff06b7fe700 1 --2- 172.21.15.7:0/3508319364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb72d765d0 0x55cb72da1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.878+0000 7ff06b7fe700 1 -- 172.21.15.7:0/3508319364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb72b04b10 con 0x55cb72dbf360 2024-01-27T02:33:31.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.878+0000 7ff070f2d700 1 --2- 172.21.15.7:0/3508319364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb72d765d0 0x55cb72da1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:31.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.878+0000 7ff06b7fe700 1 --2- 172.21.15.7:0/3508319364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72dbf360 0x55cb72da1840 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff060083f70 tx=0x7ff060083f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28247602c2ee10b2 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:31.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.878+0000 7ff0517fa700 1 -- 172.21.15.7:0/3508319364 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0600570b0 con 0x55cb72dbf360 2024-01-27T02:33:31.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.878+0000 7ff0517fa700 1 -- 172.21.15.7:0/3508319364 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0600569b0 con 0x55cb72dbf360 2024-01-27T02:33:31.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.878+0000 7ff0517fa700 1 -- 172.21.15.7:0/3508319364 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060059c00 con 0x55cb72dbf360 2024-01-27T02:33:31.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.879+0000 7ff07bc93b80 1 -- 172.21.15.7:0/3508319364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72dbf360 msgr2=0x55cb72da1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:31.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.879+0000 7ff07bc93b80 1 --2- 172.21.15.7:0/3508319364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72dbf360 0x55cb72da1840 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff060083f70 tx=0x7ff060083f00 comp rx=0 tx=0).stop 2024-01-27T02:33:31.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.879+0000 7ff07bc93b80 1 -- 172.21.15.7:0/3508319364 shutdown_connections 2024-01-27T02:33:31.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.879+0000 7ff07bc93b80 1 --2- 172.21.15.7:0/3508319364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb72d765d0 0x55cb72da1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.879+0000 7ff07bc93b80 1 --2- 172.21.15.7:0/3508319364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb72d77010 0x55cb72d773e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.879+0000 7ff07bc93b80 1 --2- 172.21.15.7:0/3508319364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72dbf360 0x55cb72da1840 secure :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff060083f70 tx=0x7ff060083f00 comp rx=0 tx=0).stop 2024-01-27T02:33:31.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.879+0000 7ff07bc93b80 1 -- 172.21.15.7:0/3508319364 >> 172.21.15.7:0/3508319364 conn(0x55cb72d76a70 msgr2=0x55cb72dbf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:31.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.879+0000 7ff07bc93b80 1 -- 172.21.15.7:0/3508319364 shutdown_connections 2024-01-27T02:33:31.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.880+0000 7ff07bc93b80 1 -- 172.21.15.7:0/3508319364 wait complete. 2024-01-27T02:33:31.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.880+0000 7ff07bc93b80 1 -- start start 2024-01-27T02:33:31.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.880+0000 7ff07bc93b80 1 Processor -- start 2024-01-27T02:33:31.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.881+0000 7ff07bc93b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72d765d0 0x55cb72d9e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.881+0000 7ff07bc93b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb72d77010 0x55cb72dcf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.881+0000 7ff07bc93b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb72db6830 0x55cb72dcb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.881+0000 7ff07bc93b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb72cc5720 con 0x55cb72d765d0 2024-01-27T02:33:31.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.881+0000 7ff07bc93b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb72c711b0 con 0x55cb72d77010 2024-01-27T02:33:31.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.881+0000 7ff07bc93b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb72c70e50 con 0x55cb72db6830 2024-01-27T02:33:31.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.881+0000 7ff07bc93b80 10 client.0 _ll_register_callbacks cb 0x55cb72ad37b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-27T02:33:31.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.881+0000 7ff06bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72d765d0 0x55cb72d9e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.881+0000 7ff06bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72d765d0 0x55cb72d9e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48192/0 (socket says 172.21.15.7:48192) 2024-01-27T02:33:31.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.881+0000 7ff06bfff700 1 -- 172.21.15.7:0/2381681345 learned_addr learned my addr 172.21.15.7:0/2381681345 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:31.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.881+0000 7ff06b7fe700 1 --2- 172.21.15.7:0/2381681345 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb72d77010 0x55cb72dcf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.881+0000 7ff070f2d700 1 --2- 172.21.15.7:0/2381681345 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb72db6830 0x55cb72dcb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.882+0000 7ff06bfff700 1 -- 172.21.15.7:0/2381681345 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb72db6830 msgr2=0x55cb72dcb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:31.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.882+0000 7ff06bfff700 1 --2- 172.21.15.7:0/2381681345 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb72db6830 0x55cb72dcb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.882+0000 7ff06bfff700 1 -- 172.21.15.7:0/2381681345 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb72d77010 msgr2=0x55cb72dcf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:31.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.882+0000 7ff06bfff700 1 --2- 172.21.15.7:0/2381681345 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb72d77010 0x55cb72dcf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.882+0000 7ff06bfff700 1 -- 172.21.15.7:0/2381681345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06c0cc730 con 0x55cb72d765d0 2024-01-27T02:33:31.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.882+0000 7ff06bfff700 1 --2- 172.21.15.7:0/2381681345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72d765d0 0x55cb72d9e2b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff05c014830 tx=0x7ff05c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a41d15085479a5 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:31.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.882+0000 7ff04a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-27T02:33:31.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.882+0000 7ff04a7fc700 1 -- 172.21.15.7:0/2381681345 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c014e80 con 0x55cb72d765d0 2024-01-27T02:33:31.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.882+0000 7ff04a7fc700 1 -- 172.21.15.7:0/2381681345 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff05c01c040 con 0x55cb72d765d0 2024-01-27T02:33:31.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.882+0000 7ff07bc93b80 1 -- 172.21.15.7:0/2381681345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb72d6a190 con 0x55cb72d765d0 2024-01-27T02:33:31.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.882+0000 7ff04a7fc700 1 -- 172.21.15.7:0/2381681345 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff05c021640 con 0x55cb72d765d0 2024-01-27T02:33:31.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.882+0000 7ff07bc93b80 10 client.5373 fetch_fsmap learned FSMap version 26 2024-01-27T02:33:31.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.882+0000 7ff07bc93b80 1 -- 172.21.15.7:0/2381681345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55cb72c93290 con 0x55cb72d765d0 2024-01-27T02:33:31.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.882+0000 7ff04a7fc700 1 -- 172.21.15.7:0/2381681345 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cb72d6a190 con 0x55cb72d765d0 2024-01-27T02:33:31.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.882+0000 7ff04a7fc700 1 -- 172.21.15.7:0/2381681345 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) v1 ==== 3100+0+0 (secure 0 0 0) 0x7ff05c0217a0 con 0x55cb72d765d0 2024-01-27T02:33:31.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.882+0000 7ff07bc93b80 10 client.5373 fetch_fsmap finished waiting for FSMap version 26 2024-01-27T02:33:31.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.882+0000 7ff07bc93b80 10 client.5373 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-27T02:33:31.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.882+0000 7ff07bc93b80 20 client.5373 populate_metadata read hostname 'smithi007' 2024-01-27T02:33:31.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.882+0000 7ff07bc93b80 1 --2- 172.21.15.7:0/2381681345 >> [v2:172.21.15.7:6833/999728998,v1:172.21.15.7:6835/999728998] conn(0x55cb72da0640 0x55cb72db7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.882+0000 7ff07bc93b80 4 client.5373 mds_command: new command op to 5262 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-27T02:33:31.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.882+0000 7ff07bc93b80 1 -- 172.21.15.7:0/2381681345 --> [v2:172.21.15.7:6833/999728998,v1:172.21.15.7:6835/999728998] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb72c67ee0 con 0x55cb72da0640 2024-01-27T02:33:31.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.883+0000 7ff06b7fe700 1 --2- 172.21.15.7:0/2381681345 >> [v2:172.21.15.7:6833/999728998,v1:172.21.15.7:6835/999728998] conn(0x55cb72da0640 0x55cb72db7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.883+0000 7ff06b7fe700 1 --2- 172.21.15.7:0/2381681345 >> [v2:172.21.15.7:6833/999728998,v1:172.21.15.7:6835/999728998] conn(0x55cb72da0640 0x55cb72db7380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57da4ddc9af3b7f3 server_cookie=ce10213951ad6afb in_seq=0 out_seq=0 2024-01-27T02:33:31.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.883+0000 7ff04a7fc700 10 client.5373 ms_handle_connect on v2:172.21.15.7:6833/999728998 2024-01-27T02:33:31.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.883+0000 7ff04a7fc700 1 -- 172.21.15.7:0/2381681345 <== mds.0 v2:172.21.15.7:6833/999728998 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17146 (crc 0 0 0) 0x55cb72c67ee0 con 0x55cb72da0640 2024-01-27T02:33:31.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.883+0000 7ff04a7fc700 10 client.5373 handle_command_reply: tid=0 2024-01-27T02:33:31.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.883+0000 7ff07bc93b80 1 client.5373 shutdown 2024-01-27T02:33:31.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.884+0000 7ff07bc93b80 1 -- 172.21.15.7:0/2381681345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72d765d0 msgr2=0x55cb72d9e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:31.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.884+0000 7ff07bc93b80 1 --2- 172.21.15.7:0/2381681345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72d765d0 0x55cb72d9e2b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff05c014830 tx=0x7ff05c015000 comp rx=0 tx=0).stop 2024-01-27T02:33:31.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.884+0000 7ff07bc93b80 1 -- 172.21.15.7:0/2381681345 shutdown_connections 2024-01-27T02:33:31.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.884+0000 7ff07bc93b80 1 --2- 172.21.15.7:0/2381681345 >> [v2:172.21.15.7:6833/999728998,v1:172.21.15.7:6835/999728998] conn(0x55cb72da0640 0x55cb72db7380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.884+0000 7ff07bc93b80 1 --2- 172.21.15.7:0/2381681345 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb72db6830 0x55cb72dcb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.884+0000 7ff07bc93b80 1 --2- 172.21.15.7:0/2381681345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72d765d0 0x55cb72d9e2b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.884+0000 7ff07bc93b80 1 --2- 172.21.15.7:0/2381681345 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb72d77010 0x55cb72dcf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.884+0000 7ff04a7fc700 0 client.5373 ms_handle_reset on v2:172.21.15.7:6833/999728998 2024-01-27T02:33:31.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.884+0000 7ff07bc93b80 1 -- 172.21.15.7:0/2381681345 >> 172.21.15.7:0/2381681345 conn(0x55cb72d76a70 msgr2=0x55cb72d70710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:31.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.884+0000 7ff07bc93b80 1 -- 172.21.15.7:0/2381681345 shutdown_connections 2024-01-27T02:33:31.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.884+0000 7ff07bc93b80 1 -- 172.21.15.7:0/2381681345 wait complete. 2024-01-27T02:33:31.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.884+0000 7ff07bc93b80 20 client.5373 trim_cache size 0 max 16384 2024-01-27T02:33:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.902+0000 7ff07bc93b80 1 Processor -- start 2024-01-27T02:33:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.903+0000 7ff07bc93b80 1 -- start start 2024-01-27T02:33:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.903+0000 7ff07bc93b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb72d765d0 0x55cb72d731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.903+0000 7ff07bc93b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72d77010 0x55cb72d9c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.903+0000 7ff07bc93b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb72db6830 0x55cb72d9cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.903+0000 7ff07bc93b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb72bbde40 con 0x55cb72d77010 2024-01-27T02:33:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.903+0000 7ff07bc93b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb72d9e060 con 0x55cb72db6830 2024-01-27T02:33:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.903+0000 7ff07bc93b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb72dbb570 con 0x55cb72d765d0 2024-01-27T02:33:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.904+0000 7ff06b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72d77010 0x55cb72d9c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.904+0000 7ff06b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72d77010 0x55cb72d9c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48194/0 (socket says 172.21.15.7:48194) 2024-01-27T02:33:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.904+0000 7ff06b7fe700 1 -- 172.21.15.7:0/3839290856 learned_addr learned my addr 172.21.15.7:0/3839290856 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.904+0000 7ff070f2d700 1 --2- 172.21.15.7:0/3839290856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb72db6830 0x55cb72d9cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.904+0000 7ff06bfff700 1 --2- 172.21.15.7:0/3839290856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb72d765d0 0x55cb72d731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.904+0000 7ff06b7fe700 1 -- 172.21.15.7:0/3839290856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb72d765d0 msgr2=0x55cb72d731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.904+0000 7ff06b7fe700 1 --2- 172.21.15.7:0/3839290856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb72d765d0 0x55cb72d731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.904+0000 7ff06b7fe700 1 -- 172.21.15.7:0/3839290856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb72db6830 msgr2=0x55cb72d9cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.904+0000 7ff06b7fe700 1 --2- 172.21.15.7:0/3839290856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb72db6830 0x55cb72d9cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.904+0000 7ff06b7fe700 1 -- 172.21.15.7:0/3839290856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb72d77e60 con 0x55cb72d77010 2024-01-27T02:33:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.904+0000 7ff06bfff700 1 --2- 172.21.15.7:0/3839290856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb72d765d0 0x55cb72d731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.904+0000 7ff06b7fe700 1 --2- 172.21.15.7:0/3839290856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72d77010 0x55cb72d9c9e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff06000b620 tx=0x7ff060026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d823d5e4d0d37164 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.904+0000 7ff051ffb700 1 -- 172.21.15.7:0/3839290856 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060072050 con 0x55cb72d77010 2024-01-27T02:33:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.904+0000 7ff051ffb700 1 -- 172.21.15.7:0/3839290856 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff060059c00 con 0x55cb72d77010 2024-01-27T02:33:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.904+0000 7ff051ffb700 1 -- 172.21.15.7:0/3839290856 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06005d950 con 0x55cb72d77010 2024-01-27T02:33:31.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.905+0000 7ff07bc93b80 1 -- 172.21.15.7:0/3839290856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72d77010 msgr2=0x55cb72d9c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.905+0000 7ff07bc93b80 1 --2- 172.21.15.7:0/3839290856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72d77010 0x55cb72d9c9e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff06000b620 tx=0x7ff060026000 comp rx=0 tx=0).stop 2024-01-27T02:33:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.905+0000 7ff07bc93b80 1 -- 172.21.15.7:0/3839290856 shutdown_connections 2024-01-27T02:33:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.905+0000 7ff07bc93b80 1 --2- 172.21.15.7:0/3839290856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb72db6830 0x55cb72d9cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.905+0000 7ff07bc93b80 1 --2- 172.21.15.7:0/3839290856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb72d765d0 0x55cb72d731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.905+0000 7ff07bc93b80 1 --2- 172.21.15.7:0/3839290856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72d77010 0x55cb72d9c9e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.905+0000 7ff07bc93b80 1 -- 172.21.15.7:0/3839290856 >> 172.21.15.7:0/3839290856 conn(0x55cb72d76a70 msgr2=0x55cb72dbf7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.905+0000 7ff07bc93b80 1 -- 172.21.15.7:0/3839290856 shutdown_connections 2024-01-27T02:33:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.905+0000 7ff07bc93b80 1 -- 172.21.15.7:0/3839290856 wait complete. 2024-01-27T02:33:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.906+0000 7ff07bc93b80 1 -- start start 2024-01-27T02:33:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.906+0000 7ff07bc93b80 1 Processor -- start 2024-01-27T02:33:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.907+0000 7ff07bc93b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb72d765d0 0x55cb72e1abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.907+0000 7ff07bc93b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb72d77010 0x55cb72dbf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.907+0000 7ff07bc93b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72db6830 0x55cb72dbfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.907+0000 7ff07bc93b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb72e20af0 con 0x55cb72db6830 2024-01-27T02:33:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.907+0000 7ff07bc93b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb72dd64d0 con 0x55cb72d77010 2024-01-27T02:33:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.907+0000 7ff07bc93b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb72dcb6a0 con 0x55cb72d765d0 2024-01-27T02:33:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.907+0000 7ff07bc93b80 10 client.0 _ll_register_callbacks cb 0x55cb72ad37b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-27T02:33:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.907+0000 7ff070f2d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72db6830 0x55cb72dbfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.907+0000 7ff070f2d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72db6830 0x55cb72dbfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48196/0 (socket says 172.21.15.7:48196) 2024-01-27T02:33:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.907+0000 7ff070f2d700 1 -- 172.21.15.7:0/1582940859 learned_addr learned my addr 172.21.15.7:0/1582940859 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.907+0000 7ff06bfff700 1 --2- 172.21.15.7:0/1582940859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb72d765d0 0x55cb72e1abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.907+0000 7ff06b7fe700 1 --2- 172.21.15.7:0/1582940859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb72d77010 0x55cb72dbf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.907+0000 7ff070f2d700 1 -- 172.21.15.7:0/1582940859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb72d765d0 msgr2=0x55cb72e1abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.907+0000 7ff070f2d700 1 --2- 172.21.15.7:0/1582940859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb72d765d0 0x55cb72e1abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.907+0000 7ff070f2d700 1 -- 172.21.15.7:0/1582940859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb72d77010 msgr2=0x55cb72dbf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.907+0000 7ff070f2d700 1 --2- 172.21.15.7:0/1582940859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb72d77010 0x55cb72dbf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.907+0000 7ff070f2d700 1 -- 172.21.15.7:0/1582940859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb72b04b10 con 0x55cb72db6830 2024-01-27T02:33:31.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.907+0000 7ff070f2d700 1 --2- 172.21.15.7:0/1582940859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72db6830 0x55cb72dbfd10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff064019760 tx=0x7ff06400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e03574d62ccc9d server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:31.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.907+0000 7ff04a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-27T02:33:31.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.907+0000 7ff04a7fc700 1 -- 172.21.15.7:0/1582940859 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06400e040 con 0x55cb72db6830 2024-01-27T02:33:31.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.908+0000 7ff04a7fc700 1 -- 172.21.15.7:0/1582940859 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff064024ac0 con 0x55cb72db6830 2024-01-27T02:33:31.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.908+0000 7ff07bc93b80 1 -- 172.21.15.7:0/1582940859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0600569b0 con 0x55cb72db6830 2024-01-27T02:33:31.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.908+0000 7ff04a7fc700 1 -- 172.21.15.7:0/1582940859 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff064024e90 con 0x55cb72db6830 2024-01-27T02:33:31.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.908+0000 7ff07bc93b80 10 client.5379 fetch_fsmap learned FSMap version 26 2024-01-27T02:33:31.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.908+0000 7ff07bc93b80 1 -- 172.21.15.7:0/1582940859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55cb72b5cc80 con 0x55cb72db6830 2024-01-27T02:33:31.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.908+0000 7ff04a7fc700 1 -- 172.21.15.7:0/1582940859 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0600569b0 con 0x55cb72db6830 2024-01-27T02:33:31.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.908+0000 7ff04a7fc700 1 -- 172.21.15.7:0/1582940859 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) v1 ==== 3100+0+0 (secure 0 0 0) 0x7ff0640175c0 con 0x55cb72db6830 2024-01-27T02:33:31.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.908+0000 7ff07bc93b80 10 client.5379 fetch_fsmap finished waiting for FSMap version 26 2024-01-27T02:33:31.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.908+0000 7ff07bc93b80 10 client.5379 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-27T02:33:31.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.908+0000 7ff07bc93b80 20 client.5379 populate_metadata read hostname 'smithi007' 2024-01-27T02:33:31.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.908+0000 7ff07bc93b80 1 --2- 172.21.15.7:0/1582940859 >> [v2:172.21.15.7:6833/999728998,v1:172.21.15.7:6835/999728998] conn(0x55cb72d75890 0x55cb72d75c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:31.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.908+0000 7ff07bc93b80 4 client.5379 mds_command: new command op to 5262 tid=0[{"prefix": "status"}] 2024-01-27T02:33:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.908+0000 7ff07bc93b80 1 -- 172.21.15.7:0/1582940859 --> [v2:172.21.15.7:6833/999728998,v1:172.21.15.7:6835/999728998] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cb72c67c00 con 0x55cb72d75890 2024-01-27T02:33:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.908+0000 7ff06bfff700 1 --2- 172.21.15.7:0/1582940859 >> [v2:172.21.15.7:6833/999728998,v1:172.21.15.7:6835/999728998] conn(0x55cb72d75890 0x55cb72d75c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.909+0000 7ff06bfff700 1 --2- 172.21.15.7:0/1582940859 >> [v2:172.21.15.7:6833/999728998,v1:172.21.15.7:6835/999728998] conn(0x55cb72d75890 0x55cb72d75c60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12638b334101ebe7 server_cookie=d16376366c417686 in_seq=0 out_seq=0 2024-01-27T02:33:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.909+0000 7ff04a7fc700 10 client.5379 ms_handle_connect on v2:172.21.15.7:6833/999728998 2024-01-27T02:33:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.909+0000 7ff04a7fc700 1 -- 172.21.15.7:0/1582940859 <== mds.0 v2:172.21.15.7:6833/999728998 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55cb72c67c00 con 0x55cb72d75890 2024-01-27T02:33:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.909+0000 7ff04a7fc700 10 client.5379 handle_command_reply: tid=0 2024-01-27T02:33:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.909+0000 7ff07bc93b80 1 client.5379 shutdown 2024-01-27T02:33:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.909+0000 7ff07bc93b80 1 -- 172.21.15.7:0/1582940859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72db6830 msgr2=0x55cb72dbfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.909+0000 7ff07bc93b80 1 --2- 172.21.15.7:0/1582940859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72db6830 0x55cb72dbfd10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff064019760 tx=0x7ff06400b040 comp rx=0 tx=0).stop 2024-01-27T02:33:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.909+0000 7ff07bc93b80 1 -- 172.21.15.7:0/1582940859 shutdown_connections 2024-01-27T02:33:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.909+0000 7ff07bc93b80 1 --2- 172.21.15.7:0/1582940859 >> [v2:172.21.15.7:6833/999728998,v1:172.21.15.7:6835/999728998] conn(0x55cb72d75890 0x55cb72d75c60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.909+0000 7ff07bc93b80 1 --2- 172.21.15.7:0/1582940859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb72db6830 0x55cb72dbfd10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.909+0000 7ff07bc93b80 1 --2- 172.21.15.7:0/1582940859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cb72d765d0 0x55cb72e1abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.909+0000 7ff04a7fc700 0 client.5379 ms_handle_reset on v2:172.21.15.7:6833/999728998 2024-01-27T02:33:31.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.909+0000 7ff07bc93b80 1 --2- 172.21.15.7:0/1582940859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb72d77010 0x55cb72dbf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.909+0000 7ff07bc93b80 1 -- 172.21.15.7:0/1582940859 >> 172.21.15.7:0/1582940859 conn(0x55cb72d76a70 msgr2=0x55cb72dbb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:31.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.909+0000 7ff07bc93b80 1 -- 172.21.15.7:0/1582940859 shutdown_connections 2024-01-27T02:33:31.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.909+0000 7ff07bc93b80 1 -- 172.21.15.7:0/1582940859 wait complete. 2024-01-27T02:33:31.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.909+0000 7ff07bc93b80 20 client.5379 trim_cache size 0 max 16384 2024-01-27T02:33:31.916 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:33:31.916 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:33:31.916 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-27T02:33:31.916 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5262, 2024-01-27T02:33:31.916 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-27T02:33:31.917 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-27T02:33:31.917 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-27T02:33:31.917 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.1012756460000004, 2024-01-27T02:33:31.917 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 26, 2024-01-27T02:33:31.917 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 45, 2024-01-27T02:33:31.917 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 45, 2024-01-27T02:33:31.917 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 11.222079323999999 2024-01-27T02:33:31.917 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:33:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.910+0000 7ff072990700 1 -- 172.21.15.7:0/980114898 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff058066f30 msgr2=0x7ff0580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.910+0000 7ff072990700 1 --2- 172.21.15.7:0/980114898 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff058066f30 0x7ff0580693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0039e0 tx=0x7ff05c007ec0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.910+0000 7ff072990700 1 -- 172.21.15.7:0/980114898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c0ca540 msgr2=0x7ff06c0cc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.910+0000 7ff072990700 1 --2- 172.21.15.7:0/980114898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c0ca540 0x7ff06c0cc030 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff06c06ea80 tx=0x7ff060017000 comp rx=0 tx=0).stop 2024-01-27T02:33:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.910+0000 7ff072990700 1 -- 172.21.15.7:0/980114898 shutdown_connections 2024-01-27T02:33:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.910+0000 7ff072990700 1 --2- 172.21.15.7:0/980114898 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff058066f30 0x7ff0580693e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.910+0000 7ff072990700 1 --2- 172.21.15.7:0/980114898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff06c15a100 0x7ff06c14d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.910+0000 7ff072990700 1 --2- 172.21.15.7:0/980114898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff06c0c4cc0 0x7ff06c0cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.910+0000 7ff072990700 1 --2- 172.21.15.7:0/980114898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c0ca540 0x7ff06c0cc030 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.910+0000 7ff072990700 1 -- 172.21.15.7:0/980114898 >> 172.21.15.7:0/980114898 conn(0x7ff06c098020 msgr2=0x7ff06c14ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.910+0000 7ff072990700 1 -- 172.21.15.7:0/980114898 shutdown_connections 2024-01-27T02:33:31.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:31.910+0000 7ff072990700 1 -- 172.21.15.7:0/980114898 wait complete. 2024-01-27T02:33:31.924 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:33:32.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.056+0000 7f93f847b700 1 Processor -- start 2024-01-27T02:33:32.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.056+0000 7f93f847b700 1 -- start start 2024-01-27T02:33:32.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.057+0000 7f93f847b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f00c4cc0 0x7f93f00ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:32.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.057+0000 7f93f847b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f93f00cd2f0 0x7f93f00679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:32.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.057+0000 7f93f847b700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93f0067ef0 con 0x7f93f015a230 2024-01-27T02:33:32.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.057+0000 7f93f847b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93f00cd790 con 0x7f93f00cd2f0 2024-01-27T02:33:32.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.057+0000 7f93f847b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f00c5320 con 0x7f93f00c4cc0 2024-01-27T02:33:32.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.057+0000 7f93f6a18700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f93f00cd2f0 0x7f93f00679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:32.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.057+0000 7f93f6a18700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f93f00cd2f0 0x7f93f00679b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48200/0 (socket says 172.21.15.7:48200) 2024-01-27T02:33:32.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.057+0000 7f93f5a16700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f00c4cc0 0x7f93f00ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:32.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.057+0000 7f93f6217700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f93f015a230 0x7f93f00c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:51376/0 (socket says 172.21.15.7:51376) 2024-01-27T02:33:32.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.057+0000 7f93f6a18700 1 -- 172.21.15.7:0/3641795253 learned_addr learned my addr 172.21.15.7:0/3641795253 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:32.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.058+0000 7f93f6a18700 1 -- 172.21.15.7:0/3641795253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f00c4cc0 msgr2=0x7f93f00ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:32.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.058+0000 7f93f6a18700 1 --2- 172.21.15.7:0/3641795253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f00c4cc0 0x7f93f00ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:32.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.058+0000 7f93f6a18700 1 -- 172.21.15.7:0/3641795253 >> v1:172.21.15.188:6789/0 conn(0x7f93f015a230 legacy=0x7f93f00c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:32.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.058+0000 7f93f6a18700 1 -- 172.21.15.7:0/3641795253 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f00c9520 con 0x7f93f00cd2f0 2024-01-27T02:33:32.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.058+0000 7f93f5a16700 1 --2- 172.21.15.7:0/3641795253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f00c4cc0 0x7f93f00ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:33:32.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.058+0000 7f93f6a18700 1 --2- 172.21.15.7:0/3641795253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93f00cd2f0 0x7f93f00679b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009f20 tx=0x7f93e8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8426fb9429aa81 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:32.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.058+0000 7f93f5215700 1 -- 172.21.15.7:0/3641795253 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e800e070 con 0x7f93f00cd2f0 2024-01-27T02:33:32.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.058+0000 7f93f5215700 1 -- 172.21.15.7:0/3641795253 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e800b8d0 con 0x7f93f00cd2f0 2024-01-27T02:33:32.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.058+0000 7f93f5215700 1 -- 172.21.15.7:0/3641795253 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e80092e0 con 0x7f93f00cd2f0 2024-01-27T02:33:32.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.058+0000 7f93f847b700 1 -- 172.21.15.7:0/3641795253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93f00cd2f0 msgr2=0x7f93f00679b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:32.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.058+0000 7f93f847b700 1 --2- 172.21.15.7:0/3641795253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93f00cd2f0 0x7f93f00679b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009f20 tx=0x7f93e8009840 comp rx=0 tx=0).stop 2024-01-27T02:33:32.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.059+0000 7f93f847b700 1 -- 172.21.15.7:0/3641795253 shutdown_connections 2024-01-27T02:33:32.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.059+0000 7f93f847b700 1 --2- 172.21.15.7:0/3641795253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93f00cd2f0 0x7f93f00679b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:32.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.059+0000 7f93f847b700 1 --2- 172.21.15.7:0/3641795253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f00c4cc0 0x7f93f00ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:32.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.059+0000 7f93f847b700 1 -- 172.21.15.7:0/3641795253 >> 172.21.15.7:0/3641795253 conn(0x7f93f0098020 msgr2=0x7f93f00711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:32.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.059+0000 7f93f847b700 1 -- 172.21.15.7:0/3641795253 shutdown_connections 2024-01-27T02:33:32.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.059+0000 7f93f847b700 1 -- 172.21.15.7:0/3641795253 wait complete. 2024-01-27T02:33:32.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.060+0000 7f93f847b700 1 Processor -- start 2024-01-27T02:33:32.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.060+0000 7f93f847b700 1 -- start start 2024-01-27T02:33:32.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.061+0000 7f93f847b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f00c4cc0 0x7f93f01529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:32.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.061+0000 7f93f847b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f00cd2f0 0x7f93f00b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:32.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.061+0000 7f93f847b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93f015a230 0x7f93f00c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:32.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.061+0000 7f93f847b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f00c8c20 con 0x7f93f015a230 2024-01-27T02:33:32.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.061+0000 7f93f847b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f00b2f00 con 0x7f93f00cd2f0 2024-01-27T02:33:32.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.061+0000 7f93f847b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f00b31a0 con 0x7f93f00c4cc0 2024-01-27T02:33:32.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.061+0000 7f93f6a18700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93f015a230 0x7f93f00c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:32.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.061+0000 7f93f6a18700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93f015a230 0x7f93f00c67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48206/0 (socket says 172.21.15.7:48206) 2024-01-27T02:33:32.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.061+0000 7f93f6a18700 1 -- 172.21.15.7:0/3053125597 learned_addr learned my addr 172.21.15.7:0/3053125597 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:32.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.061+0000 7f93f5a16700 1 --2- 172.21.15.7:0/3053125597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f00cd2f0 0x7f93f00b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:32.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.061+0000 7f93f6217700 1 --2- 172.21.15.7:0/3053125597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f00c4cc0 0x7f93f01529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:32.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.061+0000 7f93f6a18700 1 -- 172.21.15.7:0/3053125597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f00c4cc0 msgr2=0x7f93f01529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:32.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.061+0000 7f93f6a18700 1 --2- 172.21.15.7:0/3053125597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f00c4cc0 0x7f93f01529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:32.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.061+0000 7f93f6a18700 1 -- 172.21.15.7:0/3053125597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f00cd2f0 msgr2=0x7f93f00b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:32.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.061+0000 7f93f6a18700 1 --2- 172.21.15.7:0/3053125597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f00cd2f0 0x7f93f00b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:32.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.061+0000 7f93f6a18700 1 -- 172.21.15.7:0/3053125597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e0009420 con 0x7f93f015a230 2024-01-27T02:33:32.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.061+0000 7f93f5a16700 1 --2- 172.21.15.7:0/3053125597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f00cd2f0 0x7f93f00b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:33:32.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.061+0000 7f93f6217700 1 --2- 172.21.15.7:0/3053125597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f00c4cc0 0x7f93f01529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:33:32.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.061+0000 7f93f6a18700 1 --2- 172.21.15.7:0/3053125597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93f015a230 0x7f93f00c67b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f93f006ea80 tx=0x7f93e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:32.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.061+0000 7f93e67fc700 1 -- 172.21.15.7:0/3053125597 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e8022070 con 0x7f93f015a230 2024-01-27T02:33:32.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.062+0000 7f93e67fc700 1 -- 172.21.15.7:0/3053125597 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e8009840 con 0x7f93f015a230 2024-01-27T02:33:32.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.062+0000 7f93e67fc700 1 -- 172.21.15.7:0/3053125597 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e8012e80 con 0x7f93f015a230 2024-01-27T02:33:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.062+0000 7f93f847b700 1 -- 172.21.15.7:0/3053125597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e8009ae0 con 0x7f93f015a230 2024-01-27T02:33:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.062+0000 7f93f847b700 1 -- 172.21.15.7:0/3053125597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f00b3760 con 0x7f93f015a230 2024-01-27T02:33:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.062+0000 7f93f847b700 1 -- 172.21.15.7:0/3053125597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f004c710 con 0x7f93f015a230 2024-01-27T02:33:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.063+0000 7f93e67fc700 1 -- 172.21.15.7:0/3053125597 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f93e801e640 con 0x7f93f015a230 2024-01-27T02:33:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.064+0000 7f93e67fc700 1 --2- 172.21.15.7:0/3053125597 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f93d4066f30 0x7f93d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.064+0000 7f93e67fc700 1 -- 172.21.15.7:0/3053125597 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f93e80858f0 con 0x7f93f015a230 2024-01-27T02:33:32.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.064+0000 7f93f6217700 1 --2- 172.21.15.7:0/3053125597 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f93d4066f30 0x7f93d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:32.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.064+0000 7f93f6217700 1 --2- 172.21.15.7:0/3053125597 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f93d4066f30 0x7f93d40693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f93ec001910 tx=0x7f93ec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:32.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.068+0000 7f93e67fc700 1 -- 172.21.15.7:0/3053125597 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93e8052b60 con 0x7f93f015a230 2024-01-27T02:33:32.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.322+0000 7f93f847b700 1 -- 172.21.15.7:0/3053125597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f93f00c9800 con 0x7f93f015a230 2024-01-27T02:33:32.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.322+0000 7f93e67fc700 1 -- 172.21.15.7:0/3053125597 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7f93e8056180 con 0x7f93f015a230 2024-01-27T02:33:32.323 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:33:32.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.324+0000 7f93f847b700 1 -- 172.21.15.7:0/3053125597 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f93d4066f30 msgr2=0x7f93d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:32.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.324+0000 7f93f847b700 1 --2- 172.21.15.7:0/3053125597 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f93d4066f30 0x7f93d40693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f93ec001910 tx=0x7f93ec009040 comp rx=0 tx=0).stop 2024-01-27T02:33:32.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.324+0000 7f93f847b700 1 -- 172.21.15.7:0/3053125597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93f015a230 msgr2=0x7f93f00c67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:32.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.324+0000 7f93f847b700 1 --2- 172.21.15.7:0/3053125597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93f015a230 0x7f93f00c67b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f93f006ea80 tx=0x7f93e8017000 comp rx=0 tx=0).stop 2024-01-27T02:33:32.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.324+0000 7f93f847b700 1 -- 172.21.15.7:0/3053125597 shutdown_connections 2024-01-27T02:33:32.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.324+0000 7f93f847b700 1 --2- 172.21.15.7:0/3053125597 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f93d4066f30 0x7f93d40693e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:32.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.324+0000 7f93f847b700 1 --2- 172.21.15.7:0/3053125597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93f015a230 0x7f93f00c67b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:32.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.324+0000 7f93f847b700 1 --2- 172.21.15.7:0/3053125597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f00c4cc0 0x7f93f01529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:32.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.324+0000 7f93f847b700 1 --2- 172.21.15.7:0/3053125597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f00cd2f0 0x7f93f00b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:32.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.324+0000 7f93f847b700 1 -- 172.21.15.7:0/3053125597 >> 172.21.15.7:0/3053125597 conn(0x7f93f0098020 msgr2=0x7f93f0156a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:32.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.325+0000 7f93f847b700 1 -- 172.21.15.7:0/3053125597 shutdown_connections 2024-01-27T02:33:32.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:32.325+0000 7f93f847b700 1 -- 172.21.15.7:0/3053125597 wait complete. 2024-01-27T02:33:32.338 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-27T02:33:32.338 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:33:32.338 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-27T02:33:32.339 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-27T02:33:32.339 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-27T02:33:32.339 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-27T02:33:32.339 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-27T02:33:32.339 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:33:32.339 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:33:32.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:32.339 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-27T02:33:32.360 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:33:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:33:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:33:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-27T02:33:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:32.361 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:33:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-27T02:33:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-27T02:33:32.361 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:33:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-27T02:33:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-27T02:33:32.361 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:33:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-27T02:33:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-27T02:33:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-27T02:33:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42299sec preferred_lft 42299sec 2024-01-27T02:33:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-27T02:33:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:32.362 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:33:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-27T02:33:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-27T02:33:32.362 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:33:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 4a:57:d9:3c:c5:a0 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:33:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-27T02:33:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::2477:9eff:fe39:433c/64 scope link 2024-01-27T02:33:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:32.362 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:33:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 4a:57:d9:3c:c5:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-27T02:33:32.363 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::4857:d9ff:fe3c:c5a0/64 scope link 2024-01-27T02:33:32.363 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:32.363 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:33:32.363 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ca:6e:b8:df:53:bf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-27T02:33:32.363 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c86e:b8ff:fedf:53bf/64 scope link 2024-01-27T02:33:32.363 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:32.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:32.363 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:33:32.432 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-27T02:33:32.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:32.434 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-27T02:33:32.498 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-27T02:33:32.499 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:33:32.499 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-27T02:33:32.499 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-27T02:33:32.499 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:33:32.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:32.499 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-27T02:33:32.499 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-27T02:33:32.499 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-27T02:33:32.499 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-27T02:33:32.500 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-27T02:33:32.500 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-27T02:33:32.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:32.623 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-27T02:33:32.643 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-27T02:33:32.643 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-27T02:33:32.643 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:33:32.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:32.644 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-27T02:33:32.719 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:33:32.719 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-27T02:33:32.781 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:33:32.782 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:33:32.782 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-27T02:33:32.844 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:33:32.844 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:33:32.845 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:33:32.940 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:33:32.940 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:33:32.940 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:33:32.950 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:33:32.950 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:33:32.950 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:33:32.950 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:33:32.951 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:33:32.968 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:33:32.968 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:33:32.968 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:33:32.968 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:33:32.968 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:33:32.969 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:33:32.969 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:33:32.969 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:33:32.972 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:33:32.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:32.972 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-27T02:33:33.005 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:33:33.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.186+0000 7f803ac74700 1 Processor -- start 2024-01-27T02:33:33.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.186+0000 7f803ac74700 1 -- start start 2024-01-27T02:33:33.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.187+0000 7f803ac74700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80340c4cc0 0x7f80340cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:33.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.187+0000 7f803ac74700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80340ce5e0 con 0x7f803415a100 2024-01-27T02:33:33.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.187+0000 7f803ac74700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80340cd9d0 con 0x7f80340ca540 2024-01-27T02:33:33.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.187+0000 7f803ac74700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80340cdcb0 con 0x7f80340c4cc0 2024-01-27T02:33:33.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.187+0000 7f8038a10700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f803415a100 0x7f80340c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44456/0 (socket says 172.21.15.7:44456) 2024-01-27T02:33:33.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.187+0000 7f8038a10700 1 -- 172.21.15.7:0/3461399315 learned_addr learned my addr 172.21.15.7:0/3461399315 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:33.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.187+0000 7f8033fff700 1 --2- 172.21.15.7:0/3461399315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80340c4cc0 0x7f80340cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:33.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.188+0000 7f80337fe700 1 -- 172.21.15.7:0/3461399315 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1521034507 0 0) 0x7f80340ce5e0 con 0x7f803415a100 2024-01-27T02:33:33.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.188+0000 7f80337fe700 1 -- 172.21.15.7:0/3461399315 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8020003570 con 0x7f803415a100 2024-01-27T02:33:33.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.188+0000 7f80337fe700 1 -- 172.21.15.7:0/3461399315 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836586226 0 0) 0x7f80340cd9d0 con 0x7f80340ca540 2024-01-27T02:33:33.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.188+0000 7f80337fe700 1 -- 172.21.15.7:0/3461399315 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80340ce5e0 con 0x7f80340ca540 2024-01-27T02:33:33.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.188+0000 7f8033fff700 1 -- 172.21.15.7:0/3461399315 >> v1:172.21.15.188:6789/0 conn(0x7f80340ca540 legacy=0x7f8034163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:33.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.188+0000 7f8033fff700 1 -- 172.21.15.7:0/3461399315 >> v1:172.21.15.7:6789/0 conn(0x7f803415a100 legacy=0x7f80340c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:33.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.188+0000 7f8033fff700 1 -- 172.21.15.7:0/3461399315 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80340cdf90 con 0x7f80340c4cc0 2024-01-27T02:33:33.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.188+0000 7f8033fff700 1 --2- 172.21.15.7:0/3461399315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80340c4cc0 0x7f80340cd2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f801c009910 tx=0x7f801c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37825ade402edc0e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:33.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.188+0000 7f80337fe700 1 -- 172.21.15.7:0/3461399315 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c011630 con 0x7f80340c4cc0 2024-01-27T02:33:33.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.188+0000 7f80337fe700 1 -- 172.21.15.7:0/3461399315 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801c011790 con 0x7f80340c4cc0 2024-01-27T02:33:33.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.188+0000 7f80337fe700 1 -- 172.21.15.7:0/3461399315 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c011b30 con 0x7f80340c4cc0 2024-01-27T02:33:33.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.189+0000 7f803ac74700 1 -- 172.21.15.7:0/3461399315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80340c4cc0 msgr2=0x7f80340cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:33.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.189+0000 7f803ac74700 1 --2- 172.21.15.7:0/3461399315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80340c4cc0 0x7f80340cd2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f801c009910 tx=0x7f801c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:33.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.189+0000 7f803ac74700 1 -- 172.21.15.7:0/3461399315 shutdown_connections 2024-01-27T02:33:33.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.189+0000 7f803ac74700 1 --2- 172.21.15.7:0/3461399315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80340c4cc0 0x7f80340cd2f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:33.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.189+0000 7f803ac74700 1 -- 172.21.15.7:0/3461399315 >> 172.21.15.7:0/3461399315 conn(0x7f8034098020 msgr2=0x7f80340711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:33.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.189+0000 7f803ac74700 1 -- 172.21.15.7:0/3461399315 shutdown_connections 2024-01-27T02:33:33.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.189+0000 7f803ac74700 1 -- 172.21.15.7:0/3461399315 wait complete. 2024-01-27T02:33:33.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.190+0000 7f803ac74700 1 Processor -- start 2024-01-27T02:33:33.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.190+0000 7f803ac74700 1 -- start start 2024-01-27T02:33:33.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.191+0000 7f803ac74700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80340c4cc0 0x7f80340b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:33.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.191+0000 7f803ac74700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80340ca540 0x7f803414ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:33.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.191+0000 7f803ac74700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f803415a100 0x7f80340c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:33.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.191+0000 7f803ac74700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803415a6a0 con 0x7f80340c4cc0 2024-01-27T02:33:33.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.191+0000 7f803ac74700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803415af00 con 0x7f803415a100 2024-01-27T02:33:33.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.191+0000 7f803ac74700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80340c6090 con 0x7f80340ca540 2024-01-27T02:33:33.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.191+0000 7f8038a10700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80340c4cc0 0x7f80340b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:33.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.191+0000 7f8038a10700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80340c4cc0 0x7f80340b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48220/0 (socket says 172.21.15.7:48220) 2024-01-27T02:33:33.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.191+0000 7f8038a10700 1 -- 172.21.15.7:0/836286712 learned_addr learned my addr 172.21.15.7:0/836286712 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:33.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.191+0000 7f8033fff700 1 --2- 172.21.15.7:0/836286712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80340ca540 0x7f803414ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:33.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.191+0000 7f8039211700 1 --2- 172.21.15.7:0/836286712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f803415a100 0x7f80340c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:33.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.192+0000 7f8038a10700 1 -- 172.21.15.7:0/836286712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80340ca540 msgr2=0x7f803414ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:33.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.192+0000 7f8038a10700 1 --2- 172.21.15.7:0/836286712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80340ca540 0x7f803414ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:33.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.192+0000 7f8038a10700 1 -- 172.21.15.7:0/836286712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f803415a100 msgr2=0x7f80340c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:33.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.192+0000 7f8038a10700 1 --2- 172.21.15.7:0/836286712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f803415a100 0x7f80340c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:33.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.192+0000 7f8038a10700 1 -- 172.21.15.7:0/836286712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c009420 con 0x7f80340c4cc0 2024-01-27T02:33:33.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.192+0000 7f8038a10700 1 --2- 172.21.15.7:0/836286712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80340c4cc0 0x7f80340b1f00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f802400c790 tx=0x7f8024007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:33.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.192+0000 7f8030ff9700 1 -- 172.21.15.7:0/836286712 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024005b30 con 0x7f80340c4cc0 2024-01-27T02:33:33.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.192+0000 7f8030ff9700 1 -- 172.21.15.7:0/836286712 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8024005c90 con 0x7f80340c4cc0 2024-01-27T02:33:33.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.192+0000 7f8030ff9700 1 -- 172.21.15.7:0/836286712 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024007300 con 0x7f80340c4cc0 2024-01-27T02:33:33.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.192+0000 7f803ac74700 1 -- 172.21.15.7:0/836286712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80340c8fb0 con 0x7f80340c4cc0 2024-01-27T02:33:33.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.192+0000 7f803ac74700 1 -- 172.21.15.7:0/836286712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803415ab90 con 0x7f80340c4cc0 2024-01-27T02:33:33.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.193+0000 7f803ac74700 1 -- 172.21.15.7:0/836286712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f803404c710 con 0x7f80340c4cc0 2024-01-27T02:33:33.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.194+0000 7f8030ff9700 1 -- 172.21.15.7:0/836286712 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f802401b3b0 con 0x7f80340c4cc0 2024-01-27T02:33:33.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.194+0000 7f8030ff9700 1 --2- 172.21.15.7:0/836286712 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8014066f30 0x7f80140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:33.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.194+0000 7f8030ff9700 1 -- 172.21.15.7:0/836286712 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8024083a10 con 0x7f80340c4cc0 2024-01-27T02:33:33.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.194+0000 7f8033fff700 1 --2- 172.21.15.7:0/836286712 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8014066f30 0x7f80140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:33.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.195+0000 7f8033fff700 1 --2- 172.21.15.7:0/836286712 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8014066f30 0x7f80140693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f801c0091e0 tx=0x7f801c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:33.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.198+0000 7f8030ff9700 1 -- 172.21.15.7:0/836286712 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8024050c80 con 0x7f80340c4cc0 2024-01-27T02:33:33.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.450+0000 7f803ac74700 1 -- 172.21.15.7:0/836286712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f80340ce270 con 0x7f80340c4cc0 2024-01-27T02:33:33.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.450+0000 7f8030ff9700 1 -- 172.21.15.7:0/836286712 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8024017020 con 0x7f80340c4cc0 2024-01-27T02:33:33.452 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:33:33.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.453+0000 7f803ac74700 1 -- 172.21.15.7:0/836286712 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8014066f30 msgr2=0x7f80140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:33.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.453+0000 7f803ac74700 1 --2- 172.21.15.7:0/836286712 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8014066f30 0x7f80140693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f801c0091e0 tx=0x7f801c017000 comp rx=0 tx=0).stop 2024-01-27T02:33:33.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.453+0000 7f803ac74700 1 -- 172.21.15.7:0/836286712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80340c4cc0 msgr2=0x7f80340b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:33.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.453+0000 7f803ac74700 1 --2- 172.21.15.7:0/836286712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80340c4cc0 0x7f80340b1f00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f802400c790 tx=0x7f8024007ea0 comp rx=0 tx=0).stop 2024-01-27T02:33:33.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.453+0000 7f803ac74700 1 -- 172.21.15.7:0/836286712 shutdown_connections 2024-01-27T02:33:33.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.453+0000 7f803ac74700 1 --2- 172.21.15.7:0/836286712 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8014066f30 0x7f80140693e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:33.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.453+0000 7f803ac74700 1 --2- 172.21.15.7:0/836286712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f803415a100 0x7f80340c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:33.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.453+0000 7f803ac74700 1 --2- 172.21.15.7:0/836286712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80340c4cc0 0x7f80340b1f00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:33.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.453+0000 7f803ac74700 1 --2- 172.21.15.7:0/836286712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80340ca540 0x7f803414ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:33.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.453+0000 7f803ac74700 1 -- 172.21.15.7:0/836286712 >> 172.21.15.7:0/836286712 conn(0x7f8034098020 msgr2=0x7f80340c9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:33.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.453+0000 7f803ac74700 1 -- 172.21.15.7:0/836286712 shutdown_connections 2024-01-27T02:33:33.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:33.453+0000 7f803ac74700 1 -- 172.21.15.7:0/836286712 wait complete. 2024-01-27T02:33:33.467 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-27T02:33:33.467 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:33:33.467 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-01-27T02:33:33.467 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-27T02:33:33.467 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-27T02:33:33.467 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-27T02:33:33.467 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-27T02:33:33.467 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:33:33.467 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:33:33.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:33.468 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-01-27T02:33:33.490 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:33:33.490 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:33:33.490 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:33:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-01-27T02:33:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:33.491 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:33:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:33:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-01-27T02:33:33.491 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:33:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:33:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-01-27T02:33:33.491 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:33:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:33:33.491 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:33:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:33:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-27T02:33:33.491 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42306sec preferred_lft 42306sec 2024-01-27T02:33:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b1:d6f7:baf0:bc17/64 scope link noprefixroute 2024-01-27T02:33:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:33.492 INFO:teuthology.orchestra.run.smithi188.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:33:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b0:92:71:90:9d brd ff:ff:ff:ff:ff:ff 2024-01-27T02:33:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-27T02:33:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::10f8:32ff:feed:a752/64 scope link 2024-01-27T02:33:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:33.492 INFO:teuthology.orchestra.run.smithi188.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:33:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b0:92:71:90:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-27T02:33:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b8b0:92ff:fe71:909d/64 scope link 2024-01-27T02:33:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:33.492 INFO:teuthology.orchestra.run.smithi188.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:33:33.492 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ee:78:03:20:3a:a9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-27T02:33:33.493 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec78:3ff:fe20:3aa9/64 scope link 2024-01-27T02:33:33.493 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:33:33.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:33.493 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:33:33.563 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-01-27T02:33:33.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:33.565 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-01-27T02:33:33.627 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-01-27T02:33:33.627 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:33:33.627 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-27T02:33:33.627 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-27T02:33:33.627 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:33:33.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:33.627 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-27T02:33:33.627 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-27T02:33:33.627 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-27T02:33:33.628 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-27T02:33:33.628 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-27T02:33:33.628 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-27T02:33:33.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:33.738 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-01-27T02:33:33.755 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-27T02:33:33.755 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-27T02:33:33.756 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:33:33.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:33.756 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2024-01-27T02:33:33.824 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:33:33.824 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-27T02:33:33.881 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:33:33.881 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:33:33.881 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-27T02:33:33.938 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:33:33.938 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:33:33.938 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:33:34.019 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:33:34.020 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:33:34.020 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:33:34.028 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:33:34.028 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:33:34.028 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:33:34.028 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:33:34.028 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:33:34.046 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:33:34.046 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:33:34.047 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:33:34.047 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:33:34.047 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:33:34.047 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:33:34.047 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:33:34.047 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:33:34.048 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:33:34.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:34.049 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-27T02:33:34.116 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:33:34.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.257+0000 7f8c0113e700 1 Processor -- start 2024-01-27T02:33:34.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.257+0000 7f8c0113e700 1 -- start start 2024-01-27T02:33:34.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.258+0000 7f8c0113e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8bfc0cdbf0 0x7f8bfc162c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:34.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.258+0000 7f8c0113e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bfc0ca020 con 0x7f8bfc157ca0 2024-01-27T02:33:34.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.258+0000 7f8c0113e700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bfc0c5210 con 0x7f8bfc0c3ae0 2024-01-27T02:33:34.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.258+0000 7f8c0113e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bfc0c54f0 con 0x7f8bfc0cdbf0 2024-01-27T02:33:34.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.258+0000 7f8bfb59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8bfc0cdbf0 0x7f8bfc162c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:34.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.258+0000 7f8bfad9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8bfc157ca0 0x7f8bfc0c4a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44458/0 (socket says 172.21.15.7:44458) 2024-01-27T02:33:34.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.258+0000 7f8bfad9d700 1 -- 172.21.15.7:0/4033690608 learned_addr learned my addr 172.21.15.7:0/4033690608 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:34.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.258+0000 7f8bfb59e700 1 -- 172.21.15.7:0/4033690608 >> v1:172.21.15.188:6789/0 conn(0x7f8bfc0c3ae0 legacy=0x7f8bfc0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:34.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.258+0000 7f8bfb59e700 1 -- 172.21.15.7:0/4033690608 >> v1:172.21.15.7:6789/0 conn(0x7f8bfc157ca0 legacy=0x7f8bfc0c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:34.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.258+0000 7f8bfb59e700 1 -- 172.21.15.7:0/4033690608 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc0c57d0 con 0x7f8bfc0cdbf0 2024-01-27T02:33:34.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.258+0000 7f8bf9d9b700 1 -- 172.21.15.7:0/4033690608 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 622631707 0 0) 0x7f8bfc0ca020 con 0x7f8bfc157ca0 2024-01-27T02:33:34.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.259+0000 7f8bfb59e700 1 --2- 172.21.15.7:0/4033690608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bfc0cdbf0 0x7f8bfc162c00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009e60 tx=0x7f8bf4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3d969359ded3f5 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:34.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.259+0000 7f8bf9d9b700 1 -- 172.21.15.7:0/4033690608 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf400e070 con 0x7f8bfc0cdbf0 2024-01-27T02:33:34.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.259+0000 7f8bf9d9b700 1 -- 172.21.15.7:0/4033690608 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf400bc30 con 0x7f8bfc0cdbf0 2024-01-27T02:33:34.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.259+0000 7f8bf9d9b700 1 -- 172.21.15.7:0/4033690608 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4012430 con 0x7f8bfc0cdbf0 2024-01-27T02:33:34.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.259+0000 7f8c0113e700 1 -- 172.21.15.7:0/4033690608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bfc0cdbf0 msgr2=0x7f8bfc162c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:34.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.259+0000 7f8c0113e700 1 --2- 172.21.15.7:0/4033690608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bfc0cdbf0 0x7f8bfc162c00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009e60 tx=0x7f8bf4009780 comp rx=0 tx=0).stop 2024-01-27T02:33:34.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.259+0000 7f8c0113e700 1 -- 172.21.15.7:0/4033690608 shutdown_connections 2024-01-27T02:33:34.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.259+0000 7f8c0113e700 1 --2- 172.21.15.7:0/4033690608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bfc0cdbf0 0x7f8bfc162c00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:34.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.259+0000 7f8c0113e700 1 -- 172.21.15.7:0/4033690608 >> 172.21.15.7:0/4033690608 conn(0x7f8bfc098020 msgr2=0x7f8bfc0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:34.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.259+0000 7f8c0113e700 1 -- 172.21.15.7:0/4033690608 shutdown_connections 2024-01-27T02:33:34.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.259+0000 7f8c0113e700 1 -- 172.21.15.7:0/4033690608 wait complete. 2024-01-27T02:33:34.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.261+0000 7f8c0113e700 1 Processor -- start 2024-01-27T02:33:34.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.261+0000 7f8c0113e700 1 -- start start 2024-01-27T02:33:34.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.262+0000 7f8c0113e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bfc0c3ae0 0x7f8bfc156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:34.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.262+0000 7f8c0113e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bfc0cdbf0 0x7f8bfc153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:34.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.262+0000 7f8c0113e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bfc157ca0 0x7f8bfc152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:34.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.262+0000 7f8c0113e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc157020 con 0x7f8bfc0cdbf0 2024-01-27T02:33:34.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.262+0000 7f8c0113e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc156650 con 0x7f8bfc157ca0 2024-01-27T02:33:34.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.262+0000 7f8c0113e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc06ca90 con 0x7f8bfc0c3ae0 2024-01-27T02:33:34.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.262+0000 7f8bfa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bfc0cdbf0 0x7f8bfc153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:34.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.262+0000 7f8bfa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bfc0cdbf0 0x7f8bfc153c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48232/0 (socket says 172.21.15.7:48232) 2024-01-27T02:33:34.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.263+0000 7f8bfa59c700 1 -- 172.21.15.7:0/2060054306 learned_addr learned my addr 172.21.15.7:0/2060054306 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:34.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.263+0000 7f8bfb59e700 1 --2- 172.21.15.7:0/2060054306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bfc157ca0 0x7f8bfc152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:34.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.263+0000 7f8bfad9d700 1 --2- 172.21.15.7:0/2060054306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bfc0c3ae0 0x7f8bfc156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:34.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.263+0000 7f8bfa59c700 1 -- 172.21.15.7:0/2060054306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bfc0c3ae0 msgr2=0x7f8bfc156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:34.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.263+0000 7f8bfa59c700 1 --2- 172.21.15.7:0/2060054306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bfc0c3ae0 0x7f8bfc156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:34.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.263+0000 7f8bfa59c700 1 -- 172.21.15.7:0/2060054306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bfc157ca0 msgr2=0x7f8bfc152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:34.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.263+0000 7f8bfa59c700 1 --2- 172.21.15.7:0/2060054306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bfc157ca0 0x7f8bfc152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:34.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.263+0000 7f8bfa59c700 1 -- 172.21.15.7:0/2060054306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf4009a20 con 0x7f8bfc0cdbf0 2024-01-27T02:33:34.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.263+0000 7f8bfa59c700 1 --2- 172.21.15.7:0/2060054306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bfc0cdbf0 0x7f8bfc153c90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8be400c4e0 tx=0x7f8be400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:34.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.263+0000 7f8bf2ffd700 1 -- 172.21.15.7:0/2060054306 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4011590 con 0x7f8bfc0cdbf0 2024-01-27T02:33:34.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.263+0000 7f8bf2ffd700 1 -- 172.21.15.7:0/2060054306 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be4011710 con 0x7f8bfc0cdbf0 2024-01-27T02:33:34.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.263+0000 7f8c0113e700 1 -- 172.21.15.7:0/2060054306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bfc1541d0 con 0x7f8bfc0cdbf0 2024-01-27T02:33:34.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.263+0000 7f8bf2ffd700 1 -- 172.21.15.7:0/2060054306 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4011ad0 con 0x7f8bfc0cdbf0 2024-01-27T02:33:34.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.263+0000 7f8c0113e700 1 -- 172.21.15.7:0/2060054306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bfc06d010 con 0x7f8bfc0cdbf0 2024-01-27T02:33:34.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.264+0000 7f8c0113e700 1 -- 172.21.15.7:0/2060054306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bfc156800 con 0x7f8bfc0cdbf0 2024-01-27T02:33:34.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.265+0000 7f8bf2ffd700 1 -- 172.21.15.7:0/2060054306 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8be401c440 con 0x7f8bfc0cdbf0 2024-01-27T02:33:34.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.265+0000 7f8bf2ffd700 1 --2- 172.21.15.7:0/2060054306 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8bdc066f30 0x7f8bdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:34.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.265+0000 7f8bf2ffd700 1 -- 172.21.15.7:0/2060054306 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8be4083980 con 0x7f8bfc0cdbf0 2024-01-27T02:33:34.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.265+0000 7f8bfad9d700 1 --2- 172.21.15.7:0/2060054306 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8bdc066f30 0x7f8bdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:34.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.266+0000 7f8bfad9d700 1 --2- 172.21.15.7:0/2060054306 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8bdc066f30 0x7f8bdc0693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc0c8380 tx=0x7f8bec007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:34.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.270+0000 7f8bf2ffd700 1 -- 172.21.15.7:0/2060054306 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8bfc156800 con 0x7f8bfc0cdbf0 2024-01-27T02:33:34.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.521+0000 7f8c0113e700 1 -- 172.21.15.7:0/2060054306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bfc1567c0 con 0x7f8bfc0cdbf0 2024-01-27T02:33:34.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.521+0000 7f8bf2ffd700 1 -- 172.21.15.7:0/2060054306 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f8be4017020 con 0x7f8bfc0cdbf0 2024-01-27T02:33:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:34.522 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4883,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/198511042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":198511042},{"type":"v1","addr":"172.21.15.188:6836","nonce":198511042}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":4886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1222999254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1222999254},{"type":"v1","addr":"172.21.15.188:6837","nonce":1222999254}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2474016526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2474016526},{"type":"v1","addr":"172.21.15.7:6834","nonce":2474016526}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:33:24.950288+0000","modified":"2024-01-27T02:33:28.814599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5262":{"gid":5262,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/999728998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":999728998},{"type":"v1","addr":"172.21.15.7:6835","nonce":999728998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-27T02:33:34.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.524+0000 7f8c0113e700 1 -- 172.21.15.7:0/2060054306 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8bdc066f30 msgr2=0x7f8bdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:34.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.524+0000 7f8c0113e700 1 --2- 172.21.15.7:0/2060054306 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8bdc066f30 0x7f8bdc0693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc0c8380 tx=0x7f8bec007d30 comp rx=0 tx=0).stop 2024-01-27T02:33:34.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.524+0000 7f8c0113e700 1 -- 172.21.15.7:0/2060054306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bfc0cdbf0 msgr2=0x7f8bfc153c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:34.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.524+0000 7f8c0113e700 1 --2- 172.21.15.7:0/2060054306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bfc0cdbf0 0x7f8bfc153c90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8be400c4e0 tx=0x7f8be400c7f0 comp rx=0 tx=0).stop 2024-01-27T02:33:34.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.524+0000 7f8c0113e700 1 -- 172.21.15.7:0/2060054306 shutdown_connections 2024-01-27T02:33:34.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.524+0000 7f8c0113e700 1 --2- 172.21.15.7:0/2060054306 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8bdc066f30 0x7f8bdc0693e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:34.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.524+0000 7f8c0113e700 1 --2- 172.21.15.7:0/2060054306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bfc157ca0 0x7f8bfc152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:34.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.524+0000 7f8c0113e700 1 --2- 172.21.15.7:0/2060054306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bfc0c3ae0 0x7f8bfc156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:34.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.524+0000 7f8c0113e700 1 --2- 172.21.15.7:0/2060054306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bfc0cdbf0 0x7f8bfc153c90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:34.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.524+0000 7f8c0113e700 1 -- 172.21.15.7:0/2060054306 >> 172.21.15.7:0/2060054306 conn(0x7f8bfc098020 msgr2=0x7f8bfc0ca4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:34.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.524+0000 7f8c0113e700 1 -- 172.21.15.7:0/2060054306 shutdown_connections 2024-01-27T02:33:34.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:34.524+0000 7f8c0113e700 1 -- 172.21.15.7:0/2060054306 wait complete. 2024-01-27T02:33:34.528 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-01-27T02:33:34.538 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-01-27T02:33:34.691 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","whoami":0,"id":5262,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.8709050129999998,"mdsmap_epoch":26,"osdmap_epoch":45,"osdmap_epoch_barrier":45,"uptime":13.991709668} 2024-01-27T02:33:34.691 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", "fs_name": "cephfs", "id": 5262, "mdsmap_epoch": 26, "osdmap_epoch": 45, "osdmap_epoch_barrier": 45, "rank_uptime": 8.870905013, "state": "up:active", "uptime": 13.991709668, "want_state": "up:active", "whoami": 0 } 2024-01-27T02:33:34.691 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-01-27T02:33:34.841 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_mon_report_interval":"5"} 2024-01-27T02:33:34.841 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-01-27T02:33:34.841 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2024-01-27T02:33:34.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:33:34.842 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2024-01-27T02:33:36.523 INFO:teuthology.orchestra.run.smithi007.stderr:265+0 records in 2024-01-27T02:33:36.523 INFO:teuthology.orchestra.run.smithi007.stderr:265+0 records out 2024-01-27T02:33:36.524 INFO:teuthology.orchestra.run.smithi007.stderr:277872640 bytes (278 MB, 265 MiB) copied, 1.65979 s, 167 MB/s 2024-01-27T02:33:36.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:33:36.525 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2024-01-27T02:33:37.976 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:33:37.979 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:33:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:292+0 records in 2024-01-27T02:33:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:292+0 records out 2024-01-27T02:33:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:306184192 bytes (306 MB, 292 MiB) copied, 1.65872 s, 185 MB/s 2024-01-27T02:33:38.206 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-01-27T02:33:38.207 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:33:38.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.341+0000 7f22666ea700 1 Processor -- start 2024-01-27T02:33:38.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.341+0000 7f22666ea700 1 -- start start 2024-01-27T02:33:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.342+0000 7f22666ea700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f226015a100 0x7f22600cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.342+0000 7f22666ea700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22600c4cc0 0x7f2260163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.342+0000 7f22666ea700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22600ce5e0 con 0x7f22600cb5b0 2024-01-27T02:33:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.342+0000 7f22666ea700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22600cd9d0 con 0x7f22600c4cc0 2024-01-27T02:33:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.342+0000 7f22666ea700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22600cdcb0 con 0x7f226015a100 2024-01-27T02:33:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.342+0000 7f2264c87700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22600c4cc0 0x7f2260163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.342+0000 7f2264c87700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22600c4cc0 0x7f2260163d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48238/0 (socket says 172.21.15.7:48238) 2024-01-27T02:33:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.342+0000 7f2264c87700 1 -- 172.21.15.7:0/2229056656 learned_addr learned my addr 172.21.15.7:0/2229056656 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.342+0000 7f225ffff700 1 --2- 172.21.15.7:0/2229056656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f226015a100 0x7f22600cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.342+0000 7f2264c87700 1 -- 172.21.15.7:0/2229056656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f226015a100 msgr2=0x7f22600cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.342+0000 7f2264c87700 1 --2- 172.21.15.7:0/2229056656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f226015a100 0x7f22600cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.342+0000 7f2264c87700 1 -- 172.21.15.7:0/2229056656 >> v1:172.21.15.188:6789/0 conn(0x7f22600cb5b0 legacy=0x7f22600c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.343+0000 7f2264c87700 1 -- 172.21.15.7:0/2229056656 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22600cdf90 con 0x7f22600c4cc0 2024-01-27T02:33:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.343+0000 7f2264c87700 1 --2- 172.21.15.7:0/2229056656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22600c4cc0 0x7f2260163d90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2258009f20 tx=0x7f2258009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a32ffbeab9d06bd server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.343+0000 7f225effd700 1 -- 172.21.15.7:0/2229056656 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225800e070 con 0x7f22600c4cc0 2024-01-27T02:33:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.343+0000 7f225effd700 1 -- 172.21.15.7:0/2229056656 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225800b9d0 con 0x7f22600c4cc0 2024-01-27T02:33:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.343+0000 7f225effd700 1 -- 172.21.15.7:0/2229056656 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22580092e0 con 0x7f22600c4cc0 2024-01-27T02:33:38.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.343+0000 7f22666ea700 1 -- 172.21.15.7:0/2229056656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22600c4cc0 msgr2=0x7f2260163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.343+0000 7f22666ea700 1 --2- 172.21.15.7:0/2229056656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22600c4cc0 0x7f2260163d90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2258009f20 tx=0x7f2258009840 comp rx=0 tx=0).stop 2024-01-27T02:33:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.344+0000 7f22666ea700 1 -- 172.21.15.7:0/2229056656 shutdown_connections 2024-01-27T02:33:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.344+0000 7f22666ea700 1 --2- 172.21.15.7:0/2229056656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22600c4cc0 0x7f2260163d90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.344+0000 7f22666ea700 1 --2- 172.21.15.7:0/2229056656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f226015a100 0x7f22600cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.344+0000 7f22666ea700 1 -- 172.21.15.7:0/2229056656 >> 172.21.15.7:0/2229056656 conn(0x7f2260098020 msgr2=0x7f22600711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.344+0000 7f22666ea700 1 -- 172.21.15.7:0/2229056656 shutdown_connections 2024-01-27T02:33:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.344+0000 7f22666ea700 1 -- 172.21.15.7:0/2229056656 wait complete. 2024-01-27T02:33:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.345+0000 7f22666ea700 1 Processor -- start 2024-01-27T02:33:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.345+0000 7f22666ea700 1 -- start start 2024-01-27T02:33:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.346+0000 7f22666ea700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22600c4cc0 0x7f226009c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:38.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.346+0000 7f22666ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22600cb5b0 0x7f2260169f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:38.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.346+0000 7f22666ea700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f226015a100 0x7f2260068610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:38.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.346+0000 7f22666ea700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22600c75d0 con 0x7f22600cb5b0 2024-01-27T02:33:38.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.346+0000 7f22666ea700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22600c8150 con 0x7f22600c4cc0 2024-01-27T02:33:38.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.346+0000 7f22666ea700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22600c7820 con 0x7f226015a100 2024-01-27T02:33:38.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.346+0000 7f225f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22600cb5b0 0x7f2260169f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:38.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.346+0000 7f225f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22600cb5b0 0x7f2260169f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48246/0 (socket says 172.21.15.7:48246) 2024-01-27T02:33:38.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.346+0000 7f225f7fe700 1 -- 172.21.15.7:0/508207811 learned_addr learned my addr 172.21.15.7:0/508207811 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:38.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.346+0000 7f2264c87700 1 --2- 172.21.15.7:0/508207811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f226015a100 0x7f2260068610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:38.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.346+0000 7f225ffff700 1 --2- 172.21.15.7:0/508207811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22600c4cc0 0x7f226009c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:38.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.346+0000 7f225f7fe700 1 -- 172.21.15.7:0/508207811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f226015a100 msgr2=0x7f2260068610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:38.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.346+0000 7f225f7fe700 1 --2- 172.21.15.7:0/508207811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f226015a100 0x7f2260068610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:38.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.346+0000 7f225f7fe700 1 -- 172.21.15.7:0/508207811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22600c4cc0 msgr2=0x7f226009c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:38.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.346+0000 7f225f7fe700 1 --2- 172.21.15.7:0/508207811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22600c4cc0 0x7f226009c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:38.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.346+0000 7f225f7fe700 1 -- 172.21.15.7:0/508207811 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2258009ae0 con 0x7f22600cb5b0 2024-01-27T02:33:38.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.346+0000 7f225ffff700 1 --2- 172.21.15.7:0/508207811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22600c4cc0 0x7f226009c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:33:38.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.347+0000 7f225f7fe700 1 --2- 172.21.15.7:0/508207811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22600cb5b0 0x7f2260169f00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f224800c590 tx=0x7f224800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:38.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.347+0000 7f2257fff700 1 -- 172.21.15.7:0/508207811 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2248011590 con 0x7f22600cb5b0 2024-01-27T02:33:38.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.347+0000 7f2257fff700 1 -- 172.21.15.7:0/508207811 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2248011710 con 0x7f22600cb5b0 2024-01-27T02:33:38.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.347+0000 7f22666ea700 1 -- 172.21.15.7:0/508207811 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22600c7b00 con 0x7f22600cb5b0 2024-01-27T02:33:38.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.347+0000 7f2257fff700 1 -- 172.21.15.7:0/508207811 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2248011ad0 con 0x7f22600cb5b0 2024-01-27T02:33:38.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.347+0000 7f22666ea700 1 -- 172.21.15.7:0/508207811 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f226009c6b0 con 0x7f22600cb5b0 2024-01-27T02:33:38.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.348+0000 7f2257fff700 1 -- 172.21.15.7:0/508207811 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f224801b3b0 con 0x7f22600cb5b0 2024-01-27T02:33:38.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.348+0000 7f22666ea700 1 -- 172.21.15.7:0/508207811 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22600c82c0 con 0x7f22600cb5b0 2024-01-27T02:33:38.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.349+0000 7f2257fff700 1 --2- 172.21.15.7:0/508207811 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2240066f30 0x7f22400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:38.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.349+0000 7f2257fff700 1 -- 172.21.15.7:0/508207811 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f22480839e0 con 0x7f22600cb5b0 2024-01-27T02:33:38.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.349+0000 7f225ffff700 1 --2- 172.21.15.7:0/508207811 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2240066f30 0x7f22400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:38.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.350+0000 7f225ffff700 1 --2- 172.21.15.7:0/508207811 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2240066f30 0x7f22400693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f22500113e0 tx=0x7f22500091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:38.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.353+0000 7f2257fff700 1 -- 172.21.15.7:0/508207811 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2248050bb0 con 0x7f22600cb5b0 2024-01-27T02:33:38.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.560+0000 7f22666ea700 1 -- 172.21.15.7:0/508207811 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f226004c710 con 0x7f22600cb5b0 2024-01-27T02:33:38.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.561+0000 7f2257fff700 1 -- 172.21.15.7:0/508207811 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v45) v1 ==== 81+0+32212 (secure 0 0 0) 0x7f2248017070 con 0x7f22600cb5b0 2024-01-27T02:33:38.562 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:38.562 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:33:38.562 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 45, 2024-01-27T02:33:38.562 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:33:38.562 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:33:38.562 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:33:26.519256+0000", 2024-01-27T02:33:38.562 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:33:38.562 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:33:38.562 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:33:38.562 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:33:38.563 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:33:38.563 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:33:38.563 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:33:38.563 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:33:38.563 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:33:38.563 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:38.563 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:33:38.563 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:33:38.563 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:33:38.563 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:33:38.563 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:33:38.563 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 7, 2024-01-27T02:33:38.563 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:33:38.563 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:33:38.563 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:33:38.564 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:33:38.564 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:33:38.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.564 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:33:38.564 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:33:38.564 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:33:38.564 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:33:38.564 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:33:38.564 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:33:38.564 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:33:38.564 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:33:38.564 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:33:38.564 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:33:38.564 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:33:38.565 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:33:38.565 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:33:38.565 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:33:38.565 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:33:38.565 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:33:38.565 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:33:38.565 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:33:38.565 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:33:38.565 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:33:38.565 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:33:38.565 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:33:38.565 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:33:38.565 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:33:38.565 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:33:38.566 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:33:38.566 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:33:38.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.566 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "17", 2024-01-27T02:33:38.566 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:33:38.566 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:33:38.566 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:33:38.566 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:33:38.566 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:33:38.566 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:33:38.566 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:33:38.566 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:33:38.566 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:33:38.566 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:33:38.566 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:33:38.567 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:33:38.567 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:33:38.567 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:33:38.567 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:33:38.567 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:33:38.567 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:33:38.567 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:33:38.567 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:33:38.567 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:33:38.567 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:33:38.567 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:33:38.567 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:33:38.567 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:33:38.567 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:33:38.568 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:33:38.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.568 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:33:38.568 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:33:38.568 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:33:38.568 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:33:38.568 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:33:38.568 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:33:38.568 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:33:38.568 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:33:38.568 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:33:38.569 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:33:38.569 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:33:38.569 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:33:38.569 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:33:38.569 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:33:38.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.569 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:33:38.569 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:33:38.569 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.569 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 6, 2024-01-27T02:33:38.569 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:33:38.569 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:33:23.445643+0000", 2024-01-27T02:33:38.569 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:33:38.570 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:33:38.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:33:38.570 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:33:38.570 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:33:38.570 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:33:38.570 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:33:38.570 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:33:38.570 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:33:38.570 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:33:38.570 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:33:38.570 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:33:38.570 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:33:38.570 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:33:38.570 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:33:38.571 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:33:38.571 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:33:38.571 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:33:38.571 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:33:38.571 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:33:38.571 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:33:38.571 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:33:38.571 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:33:38.571 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:33:38.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.571 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "44", 2024-01-27T02:33:38.571 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:33:38.571 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:33:38.571 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:33:38.571 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:33:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:33:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:33:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:33:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:33:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:33:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:33:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:33:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:33:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:33:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:33:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:33:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:33:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:33:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:33:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:33:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:33:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:33:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:33:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:33:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:33:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:33:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:33:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:33:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:33:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:33:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:33:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:33:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:33:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:33:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:33:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:33:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:33:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:33:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:33:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:33:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:33:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:33:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:33:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:33:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:33:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 7, 2024-01-27T02:33:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:33:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:33:23.921995+0000", 2024-01-27T02:33:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:33:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:33:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:33:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:33:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:33:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:33:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:33:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:33:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:33:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:33:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:33:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:33:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:33:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:33:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:33:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:33:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:33:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:33:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:33:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:33:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:33:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:33:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:33:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:33:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "45", 2024-01-27T02:33:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:33:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:33:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:33:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:33:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:33:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:33:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:33:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:33:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:33:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:33:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:33:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:33:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:33:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:33:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:33:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:33:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:33:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:33:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:33:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:33:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:33:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:33:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:33:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:33:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:33:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:33:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:33:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:33:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:33:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:33:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:33:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:33:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:33:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:33:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:33:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:33:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:33:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:33:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:33:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:33:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:33:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:33:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:33:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:33:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:33:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:33:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:33:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:33:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:33:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:33:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:33:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:33:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:33:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:33:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:33:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:33:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:33:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:33:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:33:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:33:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:33:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:33:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:33:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:33:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:33:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:33:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:33:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:33:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:33:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:33:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:33:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:33:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:33:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:33:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:33:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:33:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:33:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:33:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:33:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:33:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:33:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:33:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:33:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:33:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:33:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:33:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:33:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:33:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:33:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:33:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:33:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:33:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:33:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:33:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:33:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:33:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:33:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:33:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:33:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:33:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:33:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:33:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:33:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:33:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:33:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:33:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:33:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:33:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:33:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:33:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:33:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:33:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:33:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:33:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:33:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:33:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:33:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:33:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:33:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:33:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:33:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:33:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:33:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:33:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:33:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:33:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:33:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:33:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:33:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:33:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:33:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:33:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:33:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:33:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:33:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:33:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:33:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:33:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:33:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:33:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:33:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:33:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:33:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:33:38.622 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:38.622 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:33:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:33:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:33:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:33:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:33:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:33:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:33:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:33:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:33:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:33:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:33:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:33:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:33:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:33:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:33:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:33:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:33:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:33:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:33:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:33:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:33:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:33:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:33:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:33:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:33:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:33:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:33:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:33:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:33:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:33:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:33:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:33:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:33:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:33:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:33:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:33:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:33:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:33:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:33:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:33:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:33:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:33:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:33:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:33:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:33:38.631 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:38.631 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:33:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.563+0000 7f22666ea700 1 -- 172.21.15.7:0/508207811 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2240066f30 msgr2=0x7f22400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.563+0000 7f22666ea700 1 --2- 172.21.15.7:0/508207811 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2240066f30 0x7f22400693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f22500113e0 tx=0x7f22500091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.563+0000 7f22666ea700 1 -- 172.21.15.7:0/508207811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22600cb5b0 msgr2=0x7f2260169f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.563+0000 7f22666ea700 1 --2- 172.21.15.7:0/508207811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22600cb5b0 0x7f2260169f00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f224800c590 tx=0x7f224800c8a0 comp rx=0 tx=0).stop 2024-01-27T02:33:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.563+0000 7f22666ea700 1 -- 172.21.15.7:0/508207811 shutdown_connections 2024-01-27T02:33:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.563+0000 7f22666ea700 1 --2- 172.21.15.7:0/508207811 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2240066f30 0x7f22400693e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.563+0000 7f22666ea700 1 --2- 172.21.15.7:0/508207811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f226015a100 0x7f2260068610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.563+0000 7f22666ea700 1 --2- 172.21.15.7:0/508207811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22600c4cc0 0x7f226009c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.563+0000 7f22666ea700 1 --2- 172.21.15.7:0/508207811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22600cb5b0 0x7f2260169f00 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.563+0000 7f22666ea700 1 -- 172.21.15.7:0/508207811 >> 172.21.15.7:0/508207811 conn(0x7f2260098020 msgr2=0x7f22600ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.563+0000 7f22666ea700 1 -- 172.21.15.7:0/508207811 shutdown_connections 2024-01-27T02:33:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:38.563+0000 7f22666ea700 1 -- 172.21.15.7:0/508207811 wait complete. 2024-01-27T02:33:38.634 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-27T02:33:39.906 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-27T02:33:39.905+0000 7fb5c783f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2024-01-27T02:33:43.634 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:33:43.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.773+0000 7fc777185700 1 Processor -- start 2024-01-27T02:33:43.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.773+0000 7fc777185700 1 -- start start 2024-01-27T02:33:43.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.775+0000 7fc777185700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc770157ca0 0x7fc7700c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:43.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.775+0000 7fc777185700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc7700c3ae0 0x7fc7700cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:43.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.775+0000 7fc777185700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7700ca020 con 0x7fc770158070 2024-01-27T02:33:43.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.775+0000 7fc777185700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7700c4f40 con 0x7fc7700c3ae0 2024-01-27T02:33:43.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.775+0000 7fc777185700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7700c5210 con 0x7fc770157ca0 2024-01-27T02:33:43.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.775+0000 7fc76ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc7700c3ae0 0x7fc7700cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:43.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.775+0000 7fc775722700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc770158070 0x7fc77015bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47476/0 (socket says 172.21.15.7:47476) 2024-01-27T02:33:43.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.775+0000 7fc775722700 1 -- 172.21.15.7:0/1938583367 learned_addr learned my addr 172.21.15.7:0/1938583367 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:43.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.775+0000 7fc774f21700 1 --2- 172.21.15.7:0/1938583367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc770157ca0 0x7fc7700c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.775+0000 7fc76ffff700 1 -- 172.21.15.7:0/1938583367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc770157ca0 msgr2=0x7fc7700c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.775+0000 7fc76ffff700 1 --2- 172.21.15.7:0/1938583367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc770157ca0 0x7fc7700c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.775+0000 7fc76ffff700 1 -- 172.21.15.7:0/1938583367 >> v1:172.21.15.7:6789/0 conn(0x7fc770158070 legacy=0x7fc77015bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.775+0000 7fc76ffff700 1 -- 172.21.15.7:0/1938583367 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7700c54f0 con 0x7fc7700c3ae0 2024-01-27T02:33:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.775+0000 7fc76f7fe700 1 -- 172.21.15.7:0/1938583367 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136734612 0 0) 0x7fc7700ca020 con 0x7fc770158070 2024-01-27T02:33:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.775+0000 7fc76ffff700 1 --2- 172.21.15.7:0/1938583367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7700c3ae0 0x7fc7700cd4e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc7580098e0 tx=0x7fc7580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633de462dbc41a03 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.776+0000 7fc76f7fe700 1 -- 172.21.15.7:0/1938583367 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75800b640 con 0x7fc7700c3ae0 2024-01-27T02:33:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.776+0000 7fc76f7fe700 1 -- 172.21.15.7:0/1938583367 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75800b7a0 con 0x7fc7700c3ae0 2024-01-27T02:33:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.776+0000 7fc76f7fe700 1 -- 172.21.15.7:0/1938583367 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75800bb40 con 0x7fc7700c3ae0 2024-01-27T02:33:43.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.776+0000 7fc777185700 1 -- 172.21.15.7:0/1938583367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7700c3ae0 msgr2=0x7fc7700cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:43.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.776+0000 7fc777185700 1 --2- 172.21.15.7:0/1938583367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7700c3ae0 0x7fc7700cd4e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc7580098e0 tx=0x7fc7580091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:43.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.776+0000 7fc777185700 1 -- 172.21.15.7:0/1938583367 shutdown_connections 2024-01-27T02:33:43.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.776+0000 7fc777185700 1 --2- 172.21.15.7:0/1938583367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc770157ca0 0x7fc7700c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:43.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.776+0000 7fc777185700 1 --2- 172.21.15.7:0/1938583367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7700c3ae0 0x7fc7700cd4e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:43.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.776+0000 7fc777185700 1 -- 172.21.15.7:0/1938583367 >> 172.21.15.7:0/1938583367 conn(0x7fc770098020 msgr2=0x7fc7700711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:43.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.776+0000 7fc777185700 1 -- 172.21.15.7:0/1938583367 shutdown_connections 2024-01-27T02:33:43.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.776+0000 7fc777185700 1 -- 172.21.15.7:0/1938583367 wait complete. 2024-01-27T02:33:43.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.778+0000 7fc777185700 1 Processor -- start 2024-01-27T02:33:43.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.778+0000 7fc777185700 1 -- start start 2024-01-27T02:33:43.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.779+0000 7fc777185700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7700c3ae0 0x7fc7700b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:43.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.779+0000 7fc777185700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc770157ca0 0x7fc7700c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:43.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.779+0000 7fc777185700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc770158070 0x7fc77013f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:43.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.779+0000 7fc777185700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7700b3530 con 0x7fc770157ca0 2024-01-27T02:33:43.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.779+0000 7fc777185700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7701553b0 con 0x7fc7700c3ae0 2024-01-27T02:33:43.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.779+0000 7fc777185700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7700c5df0 con 0x7fc770158070 2024-01-27T02:33:43.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.779+0000 7fc76ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc770157ca0 0x7fc7700c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:43.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.779+0000 7fc76ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc770157ca0 0x7fc7700c8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44576/0 (socket says 172.21.15.7:44576) 2024-01-27T02:33:43.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.779+0000 7fc76ffff700 1 -- 172.21.15.7:0/533187086 learned_addr learned my addr 172.21.15.7:0/533187086 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:43.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.779+0000 7fc774f21700 1 --2- 172.21.15.7:0/533187086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7700c3ae0 0x7fc7700b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:43.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.779+0000 7fc775722700 1 --2- 172.21.15.7:0/533187086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc770158070 0x7fc77013f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:43.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.780+0000 7fc76ffff700 1 -- 172.21.15.7:0/533187086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc770158070 msgr2=0x7fc77013f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:43.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.780+0000 7fc76ffff700 1 --2- 172.21.15.7:0/533187086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc770158070 0x7fc77013f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:43.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.780+0000 7fc76ffff700 1 -- 172.21.15.7:0/533187086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7700c3ae0 msgr2=0x7fc7700b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:43.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.780+0000 7fc76ffff700 1 --2- 172.21.15.7:0/533187086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7700c3ae0 0x7fc7700b2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:43.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.780+0000 7fc76ffff700 1 -- 172.21.15.7:0/533187086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc758009420 con 0x7fc770157ca0 2024-01-27T02:33:43.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.780+0000 7fc774f21700 1 --2- 172.21.15.7:0/533187086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7700c3ae0 0x7fc7700b2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:43.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.780+0000 7fc76ffff700 1 --2- 172.21.15.7:0/533187086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc770157ca0 0x7fc7700c8570 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc77006ea80 tx=0x7fc758017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:43.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.780+0000 7fc76cff9700 1 -- 172.21.15.7:0/533187086 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc758000d10 con 0x7fc770157ca0 2024-01-27T02:33:43.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.780+0000 7fc76cff9700 1 -- 172.21.15.7:0/533187086 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75800bb20 con 0x7fc770157ca0 2024-01-27T02:33:43.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.780+0000 7fc777185700 1 -- 172.21.15.7:0/533187086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7700b3250 con 0x7fc770157ca0 2024-01-27T02:33:43.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.780+0000 7fc76cff9700 1 -- 172.21.15.7:0/533187086 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc758011650 con 0x7fc770157ca0 2024-01-27T02:33:43.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.780+0000 7fc777185700 1 -- 172.21.15.7:0/533187086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc770155930 con 0x7fc770157ca0 2024-01-27T02:33:43.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.781+0000 7fc777185700 1 -- 172.21.15.7:0/533187086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77004c710 con 0x7fc770157ca0 2024-01-27T02:33:43.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.782+0000 7fc76cff9700 1 -- 172.21.15.7:0/533187086 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc758011a00 con 0x7fc770157ca0 2024-01-27T02:33:43.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.782+0000 7fc76cff9700 1 --2- 172.21.15.7:0/533187086 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc750066f30 0x7fc7500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:43.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.782+0000 7fc76cff9700 1 -- 172.21.15.7:0/533187086 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc758085b60 con 0x7fc770157ca0 2024-01-27T02:33:43.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.782+0000 7fc76cff9700 1 -- 172.21.15.7:0/533187086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=49}) v3 -- 0x7fc75006ea50 con 0x7fc770157ca0 2024-01-27T02:33:43.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.782+0000 7fc774f21700 1 --2- 172.21.15.7:0/533187086 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc750066f30 0x7fc7500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:43.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.783+0000 7fc774f21700 1 --2- 172.21.15.7:0/533187086 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc750066f30 0x7fc7500693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc760009d80 tx=0x7fc760008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:43.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.787+0000 7fc76cff9700 1 -- 172.21.15.7:0/533187086 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc758052dd0 con 0x7fc770157ca0 2024-01-27T02:33:43.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.993+0000 7fc777185700 1 -- 172.21.15.7:0/533187086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fc7700c57d0 con 0x7fc770157ca0 2024-01-27T02:33:43.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.994+0000 7fc76cff9700 1 -- 172.21.15.7:0/533187086 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v48) v1 ==== 81+0+32355 (secure 0 0 0) 0x7fc758029020 con 0x7fc770157ca0 2024-01-27T02:33:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:43.995 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:33:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 48, 2024-01-27T02:33:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:33:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:33:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:33:41.912319+0000", 2024-01-27T02:33:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:33:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:33:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:33:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:33:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:33:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:33:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:33:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:33:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:33:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:33:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:33:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:33:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:33:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:33:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 7, 2024-01-27T02:33:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:33:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:33:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:33:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:33:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:33:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:33:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:33:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:33:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:33:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:33:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:33:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:33:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:33:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:33:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:33:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:33:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:33:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:33:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:33:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:33:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:33:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:33:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:33:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:33:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:33:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:33:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:33:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:33:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:33:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:33:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:33:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:33:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "46", 2024-01-27T02:33:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:33:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:33:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:33:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:33:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:33:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:33:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:33:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:33:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:33:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:33:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:33:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:33:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:33:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:33:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:33:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:33:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:33:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:33:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:33:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:33:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:33:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:33:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:33:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:33:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:33:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:33:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:33:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:33:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:33:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:33:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:33:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:33:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:33:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:33:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:33:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:33:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:33:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:33:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:33:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:33:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:33:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:33:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 6, 2024-01-27T02:33:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:33:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:33:23.445643+0000", 2024-01-27T02:33:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:33:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:33:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:33:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:33:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:33:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:33:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:33:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:33:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:33:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:33:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:33:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:33:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:33:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:33:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:33:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:33:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:33:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:33:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:33:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:33:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:33:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:33:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:33:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:33:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "46", 2024-01-27T02:33:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:33:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:33:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:33:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:33:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:33:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:33:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:33:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:33:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:33:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:33:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:33:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:33:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:33:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:33:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:33:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:33:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:33:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:33:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:33:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:33:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:33:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:33:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:33:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:33:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:33:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:33:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:33:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:33:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:33:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:33:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:33:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:33:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:33:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:33:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:33:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:33:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:33:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:33:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:33:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:33:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:33:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:33:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:33:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:33:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 7, 2024-01-27T02:33:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:33:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:33:23.921995+0000", 2024-01-27T02:33:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:33:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:33:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:33:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:33:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:33:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:33:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:33:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:33:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:33:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:33:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:33:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:33:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:33:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:33:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:33:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:33:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:33:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:33:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:33:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:33:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:33:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:33:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:33:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:33:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "46", 2024-01-27T02:33:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:33:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:33:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:33:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:33:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:33:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:33:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:33:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:33:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:33:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:33:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:33:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:33:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:33:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:33:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:33:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:33:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:33:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:33:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:33:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:33:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:33:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:33:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:33:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:33:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:33:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:33:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:33:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:33:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:33:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:33:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:33:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:33:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:33:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:33:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:33:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:33:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:33:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:33:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:33:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:33:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:33:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:33:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:33:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:33:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:33:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:33:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:33:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:33:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:33:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:33:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:33:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:33:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:33:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:33:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:33:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:33:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:33:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:33:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:33:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:33:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:33:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:33:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:33:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:33:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:33:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:33:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:33:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:33:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:33:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:33:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:33:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:33:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:33:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:33:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:33:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:33:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:33:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:33:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:33:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:33:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:33:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:33:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:33:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:33:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:33:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:33:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:33:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:33:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:33:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:33:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:33:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:33:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:33:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:33:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:33:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:33:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:33:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:33:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull", 2024-01-27T02:33:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:33:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:33:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:33:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:33:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:33:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:33:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:33:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:33:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:33:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:33:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:33:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:33:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:33:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:33:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:33:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:33:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:33:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:33:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:33:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:33:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:33:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:33:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:33:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:33:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:33:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:33:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:33:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:33:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:33:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:33:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:33:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:33:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:44.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.050 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:33:44.050 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:44.050 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.050 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.050 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:44.050 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.050 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:33:44.050 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:44.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.051 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.051 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:33:44.051 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:44.051 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.051 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.051 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:33:44.051 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:33:44.051 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:33:44.051 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:33:44.051 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:44.051 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:44.051 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:33:44.051 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:44.052 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.052 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:33:44.052 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:44.052 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:44.052 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:44.052 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:44.052 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:44.052 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:44.052 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:44.052 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:44.052 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:44.053 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:44.053 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:44.053 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:44.053 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.053 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:33:44.053 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:44.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.053 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:33:44.053 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:44.054 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.054 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.054 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:44.054 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.054 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:33:44.054 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:44.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.054 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:33:44.054 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:44.055 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.055 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.055 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:44.055 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.055 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.055 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:33:44.055 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:44.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.055 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.055 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:33:44.055 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:44.056 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.056 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.056 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:44.056 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.056 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:33:44.056 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:44.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.056 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:33:44.056 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:44.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.057 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.057 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:33:44.057 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:33:44.057 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:33:44.057 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:33:44.057 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:44.057 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:44.057 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:33:44.057 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:44.057 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.057 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:44.057 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:33:44.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.058 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:33:44.058 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:44.058 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:44.058 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:44.058 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:44.058 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:44.058 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:33:44.058 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:44.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.058 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:33:44.058 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:44.058 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:44.059 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:44.059 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:44.059 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:44.059 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:33:44.059 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:44.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.059 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:33:44.059 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:44.059 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:44.059 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:44.059 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:44.059 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:44.059 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:33:44.060 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:44.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.060 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:33:44.060 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:44.060 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:44.060 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:44.060 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:44.060 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:44.060 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:33:44.060 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:44.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.060 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:33:44.061 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:44.061 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:44.061 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:44.061 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:44.061 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:44.061 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:33:44.061 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:44.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.061 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:33:44.061 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:44.061 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:44.061 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:44.061 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:44.061 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:44.062 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:33:44.062 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:44.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.062 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:33:44.062 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:44.062 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:44.062 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:44.062 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:44.062 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:44.062 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:33:44.062 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:44.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.062 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:33:44.063 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:44.063 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:44.063 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:44.063 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:44.063 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:44.063 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:33:44.063 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:44.063 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.063 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:44.063 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:33:44.063 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:33:44.063 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:33:44.063 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:33:44.063 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:33:44.064 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:33:44.064 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:33:44.064 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:33:44.064 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:33:44.064 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:33:44.064 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:33:44.064 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:33:44.064 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:33:44.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.064 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:33:44.064 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:33:44.064 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:33:44.064 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:33:44.065 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:33:44.065 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:33:44.065 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:33:44.065 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:33:44.065 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:33:44.065 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:33:44.065 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:33:44.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.065 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:33:44.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.996+0000 7fc777185700 1 -- 172.21.15.7:0/533187086 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc750066f30 msgr2=0x7fc7500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:44.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.996+0000 7fc777185700 1 --2- 172.21.15.7:0/533187086 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc750066f30 0x7fc7500693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc760009d80 tx=0x7fc760008040 comp rx=0 tx=0).stop 2024-01-27T02:33:44.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.996+0000 7fc777185700 1 -- 172.21.15.7:0/533187086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc770157ca0 msgr2=0x7fc7700c8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:44.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.996+0000 7fc777185700 1 --2- 172.21.15.7:0/533187086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc770157ca0 0x7fc7700c8570 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc77006ea80 tx=0x7fc758017000 comp rx=0 tx=0).stop 2024-01-27T02:33:44.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.996+0000 7fc777185700 1 -- 172.21.15.7:0/533187086 shutdown_connections 2024-01-27T02:33:44.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.996+0000 7fc777185700 1 --2- 172.21.15.7:0/533187086 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc750066f30 0x7fc7500693e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:44.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.996+0000 7fc777185700 1 --2- 172.21.15.7:0/533187086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc770158070 0x7fc77013f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:44.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.996+0000 7fc777185700 1 --2- 172.21.15.7:0/533187086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7700c3ae0 0x7fc7700b2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:44.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.996+0000 7fc777185700 1 --2- 172.21.15.7:0/533187086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc770157ca0 0x7fc7700c8570 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:44.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.996+0000 7fc777185700 1 -- 172.21.15.7:0/533187086 >> 172.21.15.7:0/533187086 conn(0x7fc770098020 msgr2=0x7fc7700ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:44.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.996+0000 7fc777185700 1 -- 172.21.15.7:0/533187086 shutdown_connections 2024-01-27T02:33:44.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:43.996+0000 7fc777185700 1 -- 172.21.15.7:0/533187086 wait complete. 2024-01-27T02:33:44.068 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-01-27T02:33:44.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:33:44.068 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2024-01-27T02:33:44.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-27T02:33:44.355 INFO:teuthology.orchestra.run.smithi007.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2024-01-27T02:33:44.355 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records in 2024-01-27T02:33:44.355 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records out 2024-01-27T02:33:44.355 INFO:teuthology.orchestra.run.smithi007.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.260652 s, 201 MB/s 2024-01-27T02:33:44.356 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:33:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.486+0000 7fd0a8657700 1 Processor -- start 2024-01-27T02:33:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.486+0000 7fd0a8657700 1 -- start start 2024-01-27T02:33:44.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.487+0000 7fd0a8657700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd0a00cfee0 0x7fd0a00b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:44.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.487+0000 7fd0a8657700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0a00b2e70 0x7fd0a015bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:44.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.487+0000 7fd0a8657700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a015ba30 con 0x7fd0a0157c00 2024-01-27T02:33:44.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.487+0000 7fd0a8657700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0a00b32f0 con 0x7fd0a00cfee0 2024-01-27T02:33:44.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.487+0000 7fd0a8657700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a00d0390 con 0x7fd0a00b2e70 2024-01-27T02:33:44.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.487+0000 7fd0a5bf2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd0a00cfee0 0x7fd0a00b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:44.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.487+0000 7fd0a63f3700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd0a0157c00 0x7fd0a00cf7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47480/0 (socket says 172.21.15.7:47480) 2024-01-27T02:33:44.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.488+0000 7fd0a63f3700 1 -- 172.21.15.7:0/207547087 learned_addr learned my addr 172.21.15.7:0/207547087 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:44.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.488+0000 7fd0a6bf4700 1 --2- 172.21.15.7:0/207547087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0a00b2e70 0x7fd0a015bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:44.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.488+0000 7fd0a5bf2700 1 -- 172.21.15.7:0/207547087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0a00b2e70 msgr2=0x7fd0a015bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:44.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.488+0000 7fd0a5bf2700 1 --2- 172.21.15.7:0/207547087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0a00b2e70 0x7fd0a015bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:44.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.488+0000 7fd0a5bf2700 1 -- 172.21.15.7:0/207547087 >> v1:172.21.15.7:6789/0 conn(0x7fd0a0157c00 legacy=0x7fd0a00cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:44.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.488+0000 7fd0a5bf2700 1 -- 172.21.15.7:0/207547087 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a00d0670 con 0x7fd0a00cfee0 2024-01-27T02:33:44.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.488+0000 7fd0a53f1700 1 -- 172.21.15.7:0/207547087 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386792217 0 0) 0x7fd0a015ba30 con 0x7fd0a0157c00 2024-01-27T02:33:44.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.488+0000 7fd0a5bf2700 1 --2- 172.21.15.7:0/207547087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0a00cfee0 0x7fd0a00b2930 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd0900098e0 tx=0x7fd0900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4102e39f4555096 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:44.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.488+0000 7fd0a53f1700 1 -- 172.21.15.7:0/207547087 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd090009e80 con 0x7fd0a00cfee0 2024-01-27T02:33:44.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.488+0000 7fd0a53f1700 1 -- 172.21.15.7:0/207547087 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09000b860 con 0x7fd0a00cfee0 2024-01-27T02:33:44.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.488+0000 7fd0a53f1700 1 -- 172.21.15.7:0/207547087 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09000bbe0 con 0x7fd0a00cfee0 2024-01-27T02:33:44.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.489+0000 7fd0a8657700 1 -- 172.21.15.7:0/207547087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0a00cfee0 msgr2=0x7fd0a00b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:44.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.489+0000 7fd0a8657700 1 --2- 172.21.15.7:0/207547087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0a00cfee0 0x7fd0a00b2930 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd0900098e0 tx=0x7fd0900091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:44.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.489+0000 7fd0a8657700 1 -- 172.21.15.7:0/207547087 shutdown_connections 2024-01-27T02:33:44.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.489+0000 7fd0a8657700 1 --2- 172.21.15.7:0/207547087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0a00b2e70 0x7fd0a015bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:44.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.489+0000 7fd0a8657700 1 --2- 172.21.15.7:0/207547087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0a00cfee0 0x7fd0a00b2930 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:44.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.489+0000 7fd0a8657700 1 -- 172.21.15.7:0/207547087 >> 172.21.15.7:0/207547087 conn(0x7fd0a0068450 msgr2=0x7fd0a00709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:44.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.489+0000 7fd0a8657700 1 -- 172.21.15.7:0/207547087 shutdown_connections 2024-01-27T02:33:44.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.489+0000 7fd0a8657700 1 -- 172.21.15.7:0/207547087 wait complete. 2024-01-27T02:33:44.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.490+0000 7fd0a8657700 1 Processor -- start 2024-01-27T02:33:44.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.490+0000 7fd0a8657700 1 -- start start 2024-01-27T02:33:44.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.491+0000 7fd0a8657700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0a00b2e70 0x7fd0a015eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:44.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.491+0000 7fd0a8657700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0a00cfee0 0x7fd0a0152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:44.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.491+0000 7fd0a8657700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0a0157c00 0x7fd0a01529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:44.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.491+0000 7fd0a8657700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a015dc00 con 0x7fd0a00cfee0 2024-01-27T02:33:44.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.491+0000 7fd0a8657700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a015d030 con 0x7fd0a00b2e70 2024-01-27T02:33:44.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.491+0000 7fd0a8657700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a015d2d0 con 0x7fd0a0157c00 2024-01-27T02:33:44.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.491+0000 7fd0a5bf2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0a00cfee0 0x7fd0a0152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:44.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.491+0000 7fd0a5bf2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0a00cfee0 0x7fd0a0152460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44596/0 (socket says 172.21.15.7:44596) 2024-01-27T02:33:44.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.491+0000 7fd0a5bf2700 1 -- 172.21.15.7:0/1643359688 learned_addr learned my addr 172.21.15.7:0/1643359688 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:44.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.491+0000 7fd0a6bf4700 1 --2- 172.21.15.7:0/1643359688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0a0157c00 0x7fd0a01529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:44.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.491+0000 7fd0a5bf2700 1 -- 172.21.15.7:0/1643359688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0a0157c00 msgr2=0x7fd0a01529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.491+0000 7fd0a63f3700 1 --2- 172.21.15.7:0/1643359688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0a00b2e70 0x7fd0a015eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.491+0000 7fd0a5bf2700 1 --2- 172.21.15.7:0/1643359688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0a0157c00 0x7fd0a01529a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.492+0000 7fd0a5bf2700 1 -- 172.21.15.7:0/1643359688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0a00b2e70 msgr2=0x7fd0a015eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.492+0000 7fd0a5bf2700 1 --2- 172.21.15.7:0/1643359688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0a00b2e70 0x7fd0a015eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.492+0000 7fd0a5bf2700 1 -- 172.21.15.7:0/1643359688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd090009420 con 0x7fd0a00cfee0 2024-01-27T02:33:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.492+0000 7fd0a63f3700 1 --2- 172.21.15.7:0/1643359688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0a00b2e70 0x7fd0a015eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:33:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.492+0000 7fd0a5bf2700 1 --2- 172.21.15.7:0/1643359688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0a00cfee0 0x7fd0a0152460 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd0a006eb10 tx=0x7fd090017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.492+0000 7fd0967fc700 1 -- 172.21.15.7:0/1643359688 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09000bbe0 con 0x7fd0a00cfee0 2024-01-27T02:33:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.492+0000 7fd0967fc700 1 -- 172.21.15.7:0/1643359688 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd090011420 con 0x7fd0a00cfee0 2024-01-27T02:33:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.492+0000 7fd0a8657700 1 -- 172.21.15.7:0/1643359688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a015d5b0 con 0x7fd0a00cfee0 2024-01-27T02:33:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.492+0000 7fd0967fc700 1 -- 172.21.15.7:0/1643359688 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0900117c0 con 0x7fd0a00cfee0 2024-01-27T02:33:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.492+0000 7fd0a8657700 1 -- 172.21.15.7:0/1643359688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a0153690 con 0x7fd0a00cfee0 2024-01-27T02:33:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.493+0000 7fd0a8657700 1 -- 172.21.15.7:0/1643359688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a004c710 con 0x7fd0a00cfee0 2024-01-27T02:33:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.493+0000 7fd0967fc700 1 -- 172.21.15.7:0/1643359688 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd090011bb0 con 0x7fd0a00cfee0 2024-01-27T02:33:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.494+0000 7fd0967fc700 1 --2- 172.21.15.7:0/1643359688 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd084066f30 0x7fd0840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:44.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.494+0000 7fd0967fc700 1 -- 172.21.15.7:0/1643359688 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd090085f00 con 0x7fd0a00cfee0 2024-01-27T02:33:44.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.494+0000 7fd0967fc700 1 -- 172.21.15.7:0/1643359688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=49}) v3 -- 0x7fd08406ea50 con 0x7fd0a00cfee0 2024-01-27T02:33:44.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.494+0000 7fd0a63f3700 1 --2- 172.21.15.7:0/1643359688 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd084066f30 0x7fd0840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:44.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.495+0000 7fd0a63f3700 1 --2- 172.21.15.7:0/1643359688 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd084066f30 0x7fd0840693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd09c001810 tx=0x7fd09c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:44.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.499+0000 7fd0967fc700 1 -- 172.21.15.7:0/1643359688 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd090011e60 con 0x7fd0a00cfee0 2024-01-27T02:33:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.706+0000 7fd0a8657700 1 -- 172.21.15.7:0/1643359688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd0a00d0950 con 0x7fd0a00cfee0 2024-01-27T02:33:44.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.707+0000 7fd0967fc700 1 -- 172.21.15.7:0/1643359688 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v48) v1 ==== 81+0+32355 (secure 0 0 0) 0x7fd090015070 con 0x7fd0a00cfee0 2024-01-27T02:33:44.707 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:44.708 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:33:44.708 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 48, 2024-01-27T02:33:44.708 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:33:44.708 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:33:44.708 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:33:41.912319+0000", 2024-01-27T02:33:44.708 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:33:44.708 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:33:44.708 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:33:44.708 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:33:44.708 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:33:44.708 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:33:44.708 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:33:44.708 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:33:44.708 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:33:44.709 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:44.709 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:33:44.709 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:33:44.709 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:33:44.709 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:33:44.709 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:33:44.709 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 7, 2024-01-27T02:33:44.709 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:33:44.709 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:33:44.709 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:33:44.709 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:33:44.709 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:33:44.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.709 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:33:44.710 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:33:44.710 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:33:44.710 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:33:44.710 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:33:44.710 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:33:44.710 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:33:44.710 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:33:44.710 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:33:44.710 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:33:44.710 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:33:44.710 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:33:44.710 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:33:44.710 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:33:44.710 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:33:44.711 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:33:44.711 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:33:44.711 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:33:44.711 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:33:44.711 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:33:44.711 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:33:44.711 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:33:44.711 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:33:44.711 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:33:44.711 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:33:44.711 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:33:44.711 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:33:44.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.711 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "46", 2024-01-27T02:33:44.711 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:33:44.711 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:33:44.712 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:33:44.712 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:33:44.712 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:33:44.712 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:33:44.712 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:33:44.712 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:33:44.712 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:33:44.712 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:33:44.712 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:33:44.712 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:33:44.712 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:33:44.712 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:33:44.712 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:33:44.712 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:33:44.713 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:33:44.713 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:33:44.713 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:33:44.713 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:33:44.713 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:33:44.713 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:33:44.713 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:33:44.713 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:33:44.713 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:33:44.713 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:33:44.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.713 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:33:44.713 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:33:44.713 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:33:44.714 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:33:44.714 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:33:44.714 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:33:44.714 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:33:44.714 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:33:44.714 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:33:44.714 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:33:44.714 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:33:44.714 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:33:44.714 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:33:44.714 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:33:44.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.714 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:33:44.714 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:33:44.714 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.715 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 6, 2024-01-27T02:33:44.715 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:33:44.715 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:33:23.445643+0000", 2024-01-27T02:33:44.715 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:33:44.715 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:33:44.715 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:33:44.715 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:33:44.715 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:33:44.715 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:33:44.715 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:33:44.715 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:33:44.715 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:33:44.715 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:33:44.716 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:33:44.716 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:33:44.716 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:33:44.716 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:33:44.716 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:33:44.716 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:33:44.716 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:33:44.716 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:33:44.716 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:33:44.716 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:33:44.716 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:33:44.716 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:33:44.716 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:33:44.717 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:33:44.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.717 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "46", 2024-01-27T02:33:44.717 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:33:44.717 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:33:44.717 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:33:44.717 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:33:44.717 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:33:44.717 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:33:44.717 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:33:44.717 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:33:44.717 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:33:44.717 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:33:44.717 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:33:44.718 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:33:44.718 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:33:44.718 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:33:44.718 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:33:44.718 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:33:44.718 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:33:44.718 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:33:44.718 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:33:44.718 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:33:44.718 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:33:44.718 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:33:44.718 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:33:44.718 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:33:44.718 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:33:44.718 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:33:44.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.719 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:33:44.719 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:33:44.719 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:33:44.719 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:33:44.719 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:33:44.719 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:33:44.719 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:33:44.719 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:33:44.719 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:33:44.719 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:33:44.719 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:33:44.719 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:33:44.719 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:33:44.719 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:33:44.720 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:33:44.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.720 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:33:44.720 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:33:44.720 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:33:44.720 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.720 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.720 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 7, 2024-01-27T02:33:44.720 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:33:44.720 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:33:23.921995+0000", 2024-01-27T02:33:44.720 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:33:44.720 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:33:44.721 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:33:44.721 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:33:44.721 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:33:44.721 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:33:44.721 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:33:44.721 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:33:44.721 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:33:44.721 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:33:44.721 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:33:44.721 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:33:44.721 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:33:44.721 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:33:44.721 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:33:44.721 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:33:44.721 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:33:44.722 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:33:44.722 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:33:44.722 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:33:44.722 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:33:44.722 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:33:44.722 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:33:44.722 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:33:44.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.722 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "46", 2024-01-27T02:33:44.722 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:33:44.722 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:33:44.722 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:33:44.722 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:33:44.722 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:33:44.722 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:33:44.723 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:33:44.723 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:33:44.723 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:33:44.723 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:33:44.723 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:33:44.723 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:33:44.723 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:33:44.723 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:33:44.723 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:33:44.723 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:33:44.723 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:33:44.723 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:33:44.723 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:33:44.723 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:33:44.723 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:33:44.724 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:33:44.724 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:33:44.724 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:33:44.724 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:33:44.724 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:33:44.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.724 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:33:44.724 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:33:44.724 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:33:44.724 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:33:44.724 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:33:44.724 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:33:44.724 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:33:44.724 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:33:44.725 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:33:44.725 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:33:44.725 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:33:44.725 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:33:44.725 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:33:44.725 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:33:44.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.725 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:33:44.725 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:33:44.725 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:33:44.725 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.725 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.725 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.725 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:44.725 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:33:44.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.726 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:33:44.726 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:44.726 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:44.726 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:44.726 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:44.726 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:44.726 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:44.726 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:44.726 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:44.726 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:44.726 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:44.726 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:44.726 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:44.726 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.727 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.727 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:33:44.727 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:44.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.727 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.727 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:33:44.727 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:44.727 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.727 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.727 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:44.727 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.728 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.728 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:33:44.728 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:44.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.728 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.728 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:33:44.728 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:44.728 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.728 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.728 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:44.728 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.729 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.729 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:33:44.729 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:44.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.729 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.729 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:33:44.729 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:44.729 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.729 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.729 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:44.729 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.730 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.730 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:33:44.730 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:44.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.730 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.730 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:33:44.730 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:44.730 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.730 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.730 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:33:44.730 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:33:44.730 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:33:44.730 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:33:44.731 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:44.731 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:44.731 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:33:44.731 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:44.731 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.731 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:33:44.731 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:44.731 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:44.731 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:44.731 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:44.731 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:44.731 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:44.731 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:44.732 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:44.732 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:44.732 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:44.732 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:44.732 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:44.732 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.732 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.732 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:33:44.732 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:44.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.732 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.732 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:33:44.733 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:44.733 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.733 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.733 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:44.733 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.733 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.733 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:33:44.733 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:44.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.733 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.733 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:33:44.734 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:44.734 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.734 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.734 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:44.734 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.734 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.734 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:33:44.734 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:44.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.734 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.734 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:33:44.734 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:44.735 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.735 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.735 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:44.735 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.735 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.735 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:33:44.735 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:44.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.735 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.735 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:33:44.735 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:44.735 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.736 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.736 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:33:44.736 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:33:44.736 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:33:44.736 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:33:44.736 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:44.736 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:44.736 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:44.736 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.736 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:33:44.736 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:44.737 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:44.737 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:44.737 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:44.737 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:44.737 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:44.737 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:44.737 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:44.737 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:44.737 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:44.737 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:44.737 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:44.737 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.737 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.737 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:33:44.738 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:44.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.738 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.738 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:33:44.738 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:44.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.738 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.738 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:44.738 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.738 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.738 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:33:44.739 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:44.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.739 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.739 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:33:44.739 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:44.739 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.739 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.739 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:44.739 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.739 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.739 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:33:44.739 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:44.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.740 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.740 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:33:44.740 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:44.740 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.740 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.740 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:44.740 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.740 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.740 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:33:44.740 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:44.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.741 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.741 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:33:44.741 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:44.741 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.741 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.741 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:33:44.741 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:33:44.741 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:33:44.741 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:33:44.741 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:44.741 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:44.741 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:33:44.741 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:44.742 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.742 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:33:44.742 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:44.742 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:44.742 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:44.742 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:44.742 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:44.742 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:44.742 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:44.742 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:44.742 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:44.742 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:44.742 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:44.742 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:44.743 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.743 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.743 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:33:44.743 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:44.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.743 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.743 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:33:44.743 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:44.743 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.743 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.743 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:44.743 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.744 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.744 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:33:44.744 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:44.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.744 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.744 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:33:44.744 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:44.744 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.744 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.744 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:44.744 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.745 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.745 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:33:44.745 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:44.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.745 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.745 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:33:44.745 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:44.745 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.745 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.745 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:44.745 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.746 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.746 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:33:44.746 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:44.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.746 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.746 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:33:44.746 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:44.746 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.746 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.746 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:33:44.746 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:33:44.746 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:33:44.746 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:33:44.746 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:44.747 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull", 2024-01-27T02:33:44.747 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:44.747 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:44.747 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.747 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:33:44.747 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:44.747 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:44.747 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:44.747 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:44.747 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:44.747 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:44.747 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:44.747 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:44.748 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:44.748 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:44.748 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:44.748 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:44.748 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.748 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.748 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:33:44.748 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:44.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.748 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.748 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:33:44.748 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:44.748 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.749 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.749 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:44.749 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.749 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:33:44.749 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:44.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.749 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:33:44.749 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:44.749 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.749 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.750 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:44.750 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.750 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.750 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:33:44.750 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:44.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.750 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.750 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:33:44.750 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:44.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.750 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.750 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:44.751 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.751 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.751 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:33:44.751 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:44.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.751 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.751 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:33:44.751 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:44.751 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.751 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.751 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:33:44.751 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:33:44.752 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:33:44.752 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:33:44.752 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:44.752 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:44.752 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:44.752 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.752 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:33:44.752 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:44.752 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:44.752 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:44.752 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:44.752 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:44.752 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:44.753 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:44.753 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:44.753 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:44.753 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:44.753 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:44.753 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:44.753 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.753 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.753 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:33:44.753 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:44.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.753 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.753 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:33:44.754 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:44.754 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.754 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.754 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:44.754 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.754 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:33:44.754 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:44.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.754 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:33:44.754 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:44.755 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.755 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.755 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:44.755 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.755 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.755 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:33:44.755 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:44.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.755 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.755 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:33:44.755 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:44.756 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.756 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.756 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:44.756 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.756 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.756 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:33:44.756 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:44.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.756 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.756 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:33:44.756 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:44.756 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.757 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.757 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:33:44.757 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:33:44.757 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:33:44.757 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:33:44.757 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:44.757 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:44.757 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:44.757 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.757 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:33:44.757 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:44.757 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:44.757 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:44.758 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:44.758 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:44.758 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:44.758 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:44.758 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:44.758 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:44.758 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:44.758 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:44.758 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:44.758 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.758 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.758 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:33:44.758 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:44.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.759 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.759 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:33:44.759 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:44.759 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.759 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.759 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:44.759 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.759 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.759 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:33:44.759 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:44.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.760 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.760 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:33:44.760 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:44.760 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.760 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.760 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:44.760 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.760 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.760 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:33:44.760 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:44.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.760 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.761 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:33:44.761 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:44.761 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.761 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.761 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:44.761 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.761 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.761 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:33:44.761 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:44.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.761 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.762 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:33:44.762 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:44.762 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.762 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.762 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:33:44.762 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:33:44.762 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:33:44.762 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:33:44.762 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:44.762 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:44.762 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:33:44.762 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:44.762 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.763 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:33:44.763 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:44.763 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:44.763 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:44.763 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:44.763 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:44.763 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:44.763 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:44.763 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:44.763 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:44.763 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:44.763 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:44.763 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:44.763 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.764 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:33:44.764 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:44.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.764 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:33:44.764 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:44.764 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.764 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.764 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:44.764 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.765 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:33:44.765 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:44.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.765 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:33:44.765 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:44.765 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.765 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.765 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:44.765 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.766 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:33:44.766 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:44.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.766 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.766 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:33:44.766 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:44.766 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.766 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.766 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:44.766 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:44.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.766 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:44.767 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:33:44.767 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:44.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.767 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:44.767 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:33:44.767 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:44.767 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.767 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.767 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:33:44.767 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:33:44.767 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:33:44.767 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:33:44.767 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:44.768 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:44.768 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:33:44.768 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:44.768 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:44.768 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.768 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:44.768 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:33:44.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.768 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:33:44.768 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:44.768 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:44.768 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:44.768 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:44.768 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:44.768 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:33:44.769 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:44.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.769 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:33:44.769 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:44.769 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:44.769 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:44.769 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:44.769 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:44.769 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:33:44.769 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:44.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.769 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:33:44.769 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:44.770 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:44.770 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:44.770 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:44.770 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:44.770 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:33:44.770 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:44.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.770 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:33:44.770 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:44.770 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:44.770 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:44.770 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:44.770 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:44.770 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:33:44.770 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:44.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.771 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:33:44.771 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:44.771 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:44.771 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:44.771 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:44.771 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:44.771 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:33:44.771 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:44.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.771 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:33:44.771 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:44.771 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:44.772 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:44.772 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:44.772 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:44.772 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:33:44.772 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:44.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.772 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:33:44.772 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:44.772 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:44.772 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:44.772 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:44.772 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:44.773 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:33:44.773 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:44.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:44.773 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:33:44.773 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:44.773 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:44.773 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:44.773 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:44.773 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:44.773 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:33:44.773 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:44.773 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.773 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:44.773 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:33:44.773 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:33:44.774 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:33:44.774 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:33:44.774 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:33:44.774 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:33:44.774 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:33:44.774 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:33:44.774 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:33:44.774 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:33:44.774 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:33:44.774 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:33:44.774 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:33:44.774 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:44.774 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:33:44.774 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:33:44.774 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:33:44.775 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:33:44.775 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:33:44.775 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:33:44.775 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:33:44.775 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:33:44.775 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:33:44.775 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:33:44.775 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:33:44.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:44.775 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:33:44.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.709+0000 7fd0a8657700 1 -- 172.21.15.7:0/1643359688 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd084066f30 msgr2=0x7fd0840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.709+0000 7fd0a8657700 1 --2- 172.21.15.7:0/1643359688 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd084066f30 0x7fd0840693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd09c001810 tx=0x7fd09c009040 comp rx=0 tx=0).stop 2024-01-27T02:33:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.709+0000 7fd0a8657700 1 -- 172.21.15.7:0/1643359688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0a00cfee0 msgr2=0x7fd0a0152460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.709+0000 7fd0a8657700 1 --2- 172.21.15.7:0/1643359688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0a00cfee0 0x7fd0a0152460 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd0a006eb10 tx=0x7fd090017000 comp rx=0 tx=0).stop 2024-01-27T02:33:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.709+0000 7fd0a8657700 1 -- 172.21.15.7:0/1643359688 shutdown_connections 2024-01-27T02:33:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.709+0000 7fd0a8657700 1 --2- 172.21.15.7:0/1643359688 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd084066f30 0x7fd0840693e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.709+0000 7fd0a8657700 1 --2- 172.21.15.7:0/1643359688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0a0157c00 0x7fd0a01529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.709+0000 7fd0a8657700 1 --2- 172.21.15.7:0/1643359688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0a00b2e70 0x7fd0a015eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.709+0000 7fd0a8657700 1 --2- 172.21.15.7:0/1643359688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0a00cfee0 0x7fd0a0152460 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.709+0000 7fd0a8657700 1 -- 172.21.15.7:0/1643359688 >> 172.21.15.7:0/1643359688 conn(0x7fd0a0068450 msgr2=0x7fd0a0159e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.709+0000 7fd0a8657700 1 -- 172.21.15.7:0/1643359688 shutdown_connections 2024-01-27T02:33:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.709+0000 7fd0a8657700 1 -- 172.21.15.7:0/1643359688 wait complete. 2024-01-27T02:33:44.780 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:33:44.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.914+0000 7ff77694c700 1 Processor -- start 2024-01-27T02:33:44.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.914+0000 7ff77694c700 1 -- start start 2024-01-27T02:33:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.915+0000 7ff77694c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff770158ab0 0x7ff770156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.915+0000 7ff77694c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff770157610 con 0x7ff77015b5f0 2024-01-27T02:33:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.915+0000 7ff77694c700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff770157310 con 0x7ff7700b1c00 2024-01-27T02:33:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.915+0000 7ff77694c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff77006ee00 con 0x7ff770158ab0 2024-01-27T02:33:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.915+0000 7ff774ee9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff770158ab0 0x7ff770156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.915+0000 7ff774ee9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff770158ab0 0x7ff770156dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44610/0 (socket says 172.21.15.7:44610) 2024-01-27T02:33:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.915+0000 7ff76f7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff77015b5f0 0x7ff7701566c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47482/0 (socket says 172.21.15.7:47482) 2024-01-27T02:33:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.915+0000 7ff774ee9700 1 -- 172.21.15.7:0/2097145328 learned_addr learned my addr 172.21.15.7:0/2097145328 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.915+0000 7ff774ee9700 1 -- 172.21.15.7:0/2097145328 >> v1:172.21.15.188:6789/0 conn(0x7ff7700b1c00 legacy=0x7ff77006e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.915+0000 7ff774ee9700 1 -- 172.21.15.7:0/2097145328 >> v1:172.21.15.7:6789/0 conn(0x7ff77015b5f0 legacy=0x7ff7701566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.915+0000 7ff774ee9700 1 -- 172.21.15.7:0/2097145328 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77006f0e0 con 0x7ff770158ab0 2024-01-27T02:33:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.916+0000 7ff774ee9700 1 --2- 172.21.15.7:0/2097145328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff770158ab0 0x7ff770156dd0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff76800d1a0 tx=0x7ff76800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f93a8feb02bc5f1 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.916+0000 7ff76effd700 1 -- 172.21.15.7:0/2097145328 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff768010070 con 0x7ff770158ab0 2024-01-27T02:33:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.916+0000 7ff76effd700 1 -- 172.21.15.7:0/2097145328 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76800b0d0 con 0x7ff770158ab0 2024-01-27T02:33:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.916+0000 7ff76effd700 1 -- 172.21.15.7:0/2097145328 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76800b490 con 0x7ff770158ab0 2024-01-27T02:33:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.916+0000 7ff77694c700 1 -- 172.21.15.7:0/2097145328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff770158ab0 msgr2=0x7ff770156dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.916+0000 7ff77694c700 1 --2- 172.21.15.7:0/2097145328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff770158ab0 0x7ff770156dd0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff76800d1a0 tx=0x7ff76800ba50 comp rx=0 tx=0).stop 2024-01-27T02:33:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.916+0000 7ff77694c700 1 -- 172.21.15.7:0/2097145328 shutdown_connections 2024-01-27T02:33:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.916+0000 7ff77694c700 1 --2- 172.21.15.7:0/2097145328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff770158ab0 0x7ff770156dd0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.916+0000 7ff77694c700 1 -- 172.21.15.7:0/2097145328 >> 172.21.15.7:0/2097145328 conn(0x7ff770068430 msgr2=0x7ff77006be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.916+0000 7ff77694c700 1 -- 172.21.15.7:0/2097145328 shutdown_connections 2024-01-27T02:33:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.916+0000 7ff77694c700 1 -- 172.21.15.7:0/2097145328 wait complete. 2024-01-27T02:33:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.918+0000 7ff77694c700 1 Processor -- start 2024-01-27T02:33:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.918+0000 7ff77694c700 1 -- start start 2024-01-27T02:33:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.919+0000 7ff77694c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7700b1c00 0x7ff77014f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.919+0000 7ff77694c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff770158ab0 0x7ff7701554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.919+0000 7ff77694c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff77015b5f0 0x7ff7701559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.919+0000 7ff77694c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7700ab7a0 con 0x7ff770158ab0 2024-01-27T02:33:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.919+0000 7ff77694c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7700aba60 con 0x7ff77015b5f0 2024-01-27T02:33:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.919+0000 7ff77694c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7700abd40 con 0x7ff7700b1c00 2024-01-27T02:33:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.919+0000 7ff76f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff770158ab0 0x7ff7701554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.919+0000 7ff76f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff770158ab0 0x7ff7701554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44618/0 (socket says 172.21.15.7:44618) 2024-01-27T02:33:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.919+0000 7ff76f7fe700 1 -- 172.21.15.7:0/3740998118 learned_addr learned my addr 172.21.15.7:0/3740998118 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.919+0000 7ff76ffff700 1 --2- 172.21.15.7:0/3740998118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7700b1c00 0x7ff77014f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.919+0000 7ff774ee9700 1 --2- 172.21.15.7:0/3740998118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff77015b5f0 0x7ff7701559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.919+0000 7ff76f7fe700 1 -- 172.21.15.7:0/3740998118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7700b1c00 msgr2=0x7ff77014f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.919+0000 7ff76f7fe700 1 --2- 172.21.15.7:0/3740998118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7700b1c00 0x7ff77014f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.919+0000 7ff76f7fe700 1 -- 172.21.15.7:0/3740998118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff77015b5f0 msgr2=0x7ff7701559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.919+0000 7ff76f7fe700 1 --2- 172.21.15.7:0/3740998118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff77015b5f0 0x7ff7701559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.919+0000 7ff76f7fe700 1 -- 172.21.15.7:0/3740998118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76800bcf0 con 0x7ff770158ab0 2024-01-27T02:33:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.920+0000 7ff76f7fe700 1 --2- 172.21.15.7:0/3740998118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff770158ab0 0x7ff7701554b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff75800c7c0 tx=0x7ff758007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:44.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.921+0000 7ff767fff700 1 -- 172.21.15.7:0/3740998118 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758005a70 con 0x7ff770158ab0 2024-01-27T02:33:44.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.921+0000 7ff767fff700 1 -- 172.21.15.7:0/3740998118 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff758005bd0 con 0x7ff770158ab0 2024-01-27T02:33:44.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.921+0000 7ff77694c700 1 -- 172.21.15.7:0/3740998118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7700ac020 con 0x7ff770158ab0 2024-01-27T02:33:44.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.921+0000 7ff767fff700 1 -- 172.21.15.7:0/3740998118 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7580070d0 con 0x7ff770158ab0 2024-01-27T02:33:44.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.921+0000 7ff77694c700 1 -- 172.21.15.7:0/3740998118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7700ac820 con 0x7ff770158ab0 2024-01-27T02:33:44.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.922+0000 7ff77694c700 1 -- 172.21.15.7:0/3740998118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7700ac540 con 0x7ff770158ab0 2024-01-27T02:33:44.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.923+0000 7ff767fff700 1 -- 172.21.15.7:0/3740998118 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff758007480 con 0x7ff770158ab0 2024-01-27T02:33:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.923+0000 7ff767fff700 1 --2- 172.21.15.7:0/3740998118 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff750066f30 0x7ff7500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.923+0000 7ff767fff700 1 -- 172.21.15.7:0/3740998118 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff7580839e0 con 0x7ff770158ab0 2024-01-27T02:33:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.923+0000 7ff76ffff700 1 --2- 172.21.15.7:0/3740998118 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff750066f30 0x7ff7500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.924+0000 7ff767fff700 1 -- 172.21.15.7:0/3740998118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=49}) v3 -- 0x7ff75006ea50 con 0x7ff770158ab0 2024-01-27T02:33:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.924+0000 7ff76ffff700 1 --2- 172.21.15.7:0/3740998118 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff750066f30 0x7ff7500693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff7600039e0 tx=0x7ff760007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:44.928+0000 7ff767fff700 1 -- 172.21.15.7:0/3740998118 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff758017070 con 0x7ff770158ab0 2024-01-27T02:33:45.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.177+0000 7ff77694c700 1 -- 172.21.15.7:0/3740998118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff77004c710 con 0x7ff770158ab0 2024-01-27T02:33:45.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.178+0000 7ff767fff700 1 -- 172.21.15.7:0/3740998118 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4591 (secure 0 0 0) 0x7ff7700ac540 con 0x7ff770158ab0 2024-01-27T02:33:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:45.179 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4883,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/198511042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":198511042},{"type":"v1","addr":"172.21.15.188:6836","nonce":198511042}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":4886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1222999254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1222999254},{"type":"v1","addr":"172.21.15.188:6837","nonce":1222999254}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2474016526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2474016526},{"type":"v1","addr":"172.21.15.7:6834","nonce":2474016526}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:33:24.950288+0000","modified":"2024-01-27T02:33:28.814599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5262":{"gid":5262,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/999728998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":999728998},{"type":"v1","addr":"172.21.15.7:6835","nonce":999728998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-27T02:33:45.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.180+0000 7ff77694c700 1 -- 172.21.15.7:0/3740998118 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff750066f30 msgr2=0x7ff7500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:45.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.180+0000 7ff77694c700 1 --2- 172.21.15.7:0/3740998118 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff750066f30 0x7ff7500693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff7600039e0 tx=0x7ff760007ec0 comp rx=0 tx=0).stop 2024-01-27T02:33:45.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.180+0000 7ff77694c700 1 -- 172.21.15.7:0/3740998118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff770158ab0 msgr2=0x7ff7701554b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:45.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.180+0000 7ff77694c700 1 --2- 172.21.15.7:0/3740998118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff770158ab0 0x7ff7701554b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff75800c7c0 tx=0x7ff758007f70 comp rx=0 tx=0).stop 2024-01-27T02:33:45.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.181+0000 7ff77694c700 1 -- 172.21.15.7:0/3740998118 shutdown_connections 2024-01-27T02:33:45.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.181+0000 7ff77694c700 1 --2- 172.21.15.7:0/3740998118 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff750066f30 0x7ff7500693e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:45.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.181+0000 7ff77694c700 1 --2- 172.21.15.7:0/3740998118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff77015b5f0 0x7ff7701559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:45.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.181+0000 7ff77694c700 1 --2- 172.21.15.7:0/3740998118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7700b1c00 0x7ff77014f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:45.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.181+0000 7ff77694c700 1 --2- 172.21.15.7:0/3740998118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff770158ab0 0x7ff7701554b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:45.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.181+0000 7ff77694c700 1 -- 172.21.15.7:0/3740998118 >> 172.21.15.7:0/3740998118 conn(0x7ff770068430 msgr2=0x7ff7701534a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:45.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.181+0000 7ff77694c700 1 -- 172.21.15.7:0/3740998118 shutdown_connections 2024-01-27T02:33:45.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.181+0000 7ff77694c700 1 -- 172.21.15.7:0/3740998118 wait complete. 2024-01-27T02:33:45.182 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-01-27T02:33:45.195 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-01-27T02:33:45.346 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","whoami":0,"id":5262,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":19.526947442000001,"mdsmap_epoch":26,"osdmap_epoch":48,"osdmap_epoch_barrier":48,"uptime":24.647754856999999} 2024-01-27T02:33:45.347 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", "fs_name": "cephfs", "id": 5262, "mdsmap_epoch": 26, "osdmap_epoch": 48, "osdmap_epoch_barrier": 48, "rank_uptime": 19.526947442, "state": "up:active", "uptime": 24.647754857, "want_state": "up:active", "whoami": 0 } 2024-01-27T02:33:45.347 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-01-27T02:33:45.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:33:45.347 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2024-01-27T02:33:45.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-27T02:33:45.366 INFO:teuthology.orchestra.run.smithi007.stderr:dd: failed to open 'small_file_1': Permission denied 2024-01-27T02:33:45.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:33:45.367 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-01-27T02:33:45.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:33:45.429 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-01-27T02:33:45.496 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:33:45.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.675+0000 7f4b4d90a700 1 Processor -- start 2024-01-27T02:33:45.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.676+0000 7f4b4d90a700 1 -- start start 2024-01-27T02:33:45.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.676+0000 7f4b4d90a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b48156c00 0x7f4b48156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:45.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.676+0000 7f4b4d90a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4b4806e040 0x7f4b480c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:45.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.676+0000 7f4b4d90a700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b4806dbb0 con 0x7f4b4806e4b0 2024-01-27T02:33:45.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.676+0000 7f4b4d90a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b480c8bc0 con 0x7f4b48156c00 2024-01-27T02:33:45.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.676+0000 7f4b4d90a700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b480c88e0 con 0x7f4b4806e040 2024-01-27T02:33:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.677+0000 7f4b46ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b48156c00 0x7f4b48156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.677+0000 7f4b46ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b48156c00 0x7f4b48156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44632/0 (socket says 172.21.15.7:44632) 2024-01-27T02:33:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.677+0000 7f4b46ffd700 1 -- 172.21.15.7:0/952272098 learned_addr learned my addr 172.21.15.7:0/952272098 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.677+0000 7f4b467fc700 1 --2- 172.21.15.7:0/952272098 >> v2:172.21.15.188:3300/0 conn(0x7f4b4806e040 0x7f4b480c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.677+0000 7f4b46ffd700 1 -- 172.21.15.7:0/952272098 >> v2:172.21.15.188:3300/0 conn(0x7f4b4806e040 msgr2=0x7f4b480c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.677+0000 7f4b46ffd700 1 --2- 172.21.15.7:0/952272098 >> v2:172.21.15.188:3300/0 conn(0x7f4b4806e040 0x7f4b480c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.677+0000 7f4b46ffd700 1 -- 172.21.15.7:0/952272098 >> v1:172.21.15.188:6789/0 conn(0x7f4b4806e4b0 legacy=0x7f4b480c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.677+0000 7f4b46ffd700 1 -- 172.21.15.7:0/952272098 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4806d560 con 0x7f4b48156c00 2024-01-27T02:33:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.677+0000 7f4b46ffd700 1 --2- 172.21.15.7:0/952272098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b48156c00 0x7f4b48156fd0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4b480cdab0 tx=0x7f4b3800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b43d28176672a2 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.677+0000 7f4b45ffb700 1 -- 172.21.15.7:0/952272098 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3800dba0 con 0x7f4b48156c00 2024-01-27T02:33:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.677+0000 7f4b45ffb700 1 -- 172.21.15.7:0/952272098 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3800dd00 con 0x7f4b48156c00 2024-01-27T02:33:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.677+0000 7f4b45ffb700 1 -- 172.21.15.7:0/952272098 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b380196d0 con 0x7f4b48156c00 2024-01-27T02:33:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.678+0000 7f4b4d90a700 1 -- 172.21.15.7:0/952272098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b48156c00 msgr2=0x7f4b48156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.678+0000 7f4b4d90a700 1 --2- 172.21.15.7:0/952272098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b48156c00 0x7f4b48156fd0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4b480cdab0 tx=0x7f4b3800b4f0 comp rx=0 tx=0).stop 2024-01-27T02:33:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.678+0000 7f4b4d90a700 1 -- 172.21.15.7:0/952272098 shutdown_connections 2024-01-27T02:33:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.678+0000 7f4b4d90a700 1 --2- 172.21.15.7:0/952272098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b48156c00 0x7f4b48156fd0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.678+0000 7f4b4d90a700 1 --2- 172.21.15.7:0/952272098 >> v2:172.21.15.188:3300/0 conn(0x7f4b4806e040 0x7f4b480c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.678+0000 7f4b4d90a700 1 -- 172.21.15.7:0/952272098 >> 172.21.15.7:0/952272098 conn(0x7f4b48063580 msgr2=0x7f4b48063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.678+0000 7f4b4d90a700 1 -- 172.21.15.7:0/952272098 shutdown_connections 2024-01-27T02:33:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.678+0000 7f4b4d90a700 1 -- 172.21.15.7:0/952272098 wait complete. 2024-01-27T02:33:45.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.679+0000 7f4b4d90a700 1 Processor -- start 2024-01-27T02:33:45.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.679+0000 7f4b4d90a700 1 -- start start 2024-01-27T02:33:45.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.680+0000 7f4b4d90a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b4806e040 0x7f4b480b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:45.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.680+0000 7f4b4d90a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b4806e4b0 0x7f4b48156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:45.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.680+0000 7f4b4d90a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b48156c00 0x7f4b48150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:45.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.680+0000 7f4b4d90a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b480affa0 con 0x7f4b4806e040 2024-01-27T02:33:45.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.680+0000 7f4b4d90a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b48157510 con 0x7f4b48156c00 2024-01-27T02:33:45.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.680+0000 7f4b4d90a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b48154620 con 0x7f4b4806e4b0 2024-01-27T02:33:45.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.680+0000 7f4b46ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b4806e040 0x7f4b480b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:45.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.680+0000 7f4b46ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b4806e040 0x7f4b480b02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44642/0 (socket says 172.21.15.7:44642) 2024-01-27T02:33:45.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.680+0000 7f4b46ffd700 1 -- 172.21.15.7:0/3492200034 learned_addr learned my addr 172.21.15.7:0/3492200034 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:45.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.681+0000 7f4b477fe700 1 --2- 172.21.15.7:0/3492200034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b48156c00 0x7f4b48150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:45.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.681+0000 7f4b467fc700 1 --2- 172.21.15.7:0/3492200034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b4806e4b0 0x7f4b48156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:45.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.681+0000 7f4b46ffd700 1 -- 172.21.15.7:0/3492200034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b4806e4b0 msgr2=0x7f4b48156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:45.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.681+0000 7f4b46ffd700 1 --2- 172.21.15.7:0/3492200034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b4806e4b0 0x7f4b48156fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:45.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.681+0000 7f4b46ffd700 1 -- 172.21.15.7:0/3492200034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b48156c00 msgr2=0x7f4b48150270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:45.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.681+0000 7f4b46ffd700 1 --2- 172.21.15.7:0/3492200034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b48156c00 0x7f4b48150270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:45.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.681+0000 7f4b46ffd700 1 -- 172.21.15.7:0/3492200034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3800b790 con 0x7f4b4806e040 2024-01-27T02:33:45.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.681+0000 7f4b467fc700 1 --2- 172.21.15.7:0/3492200034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b4806e4b0 0x7f4b48156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:45.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.681+0000 7f4b46ffd700 1 --2- 172.21.15.7:0/3492200034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b4806e040 0x7f4b480b02c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4b38006fd0 tx=0x7f4b3801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:45.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.681+0000 7f4b2effd700 1 -- 172.21.15.7:0/3492200034 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b38010680 con 0x7f4b4806e040 2024-01-27T02:33:45.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.681+0000 7f4b2effd700 1 -- 172.21.15.7:0/3492200034 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b380107e0 con 0x7f4b4806e040 2024-01-27T02:33:45.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.681+0000 7f4b4d90a700 1 -- 172.21.15.7:0/3492200034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b48154900 con 0x7f4b4806e040 2024-01-27T02:33:45.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.681+0000 7f4b2effd700 1 -- 172.21.15.7:0/3492200034 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b38010ba0 con 0x7f4b4806e040 2024-01-27T02:33:45.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.681+0000 7f4b4d90a700 1 -- 172.21.15.7:0/3492200034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b48154ec0 con 0x7f4b4806e040 2024-01-27T02:33:45.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.682+0000 7f4b4d90a700 1 -- 172.21.15.7:0/3492200034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b4804c710 con 0x7f4b4806e040 2024-01-27T02:33:45.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.683+0000 7f4b2effd700 1 -- 172.21.15.7:0/3492200034 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4b380183e0 con 0x7f4b4806e040 2024-01-27T02:33:45.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.683+0000 7f4b2effd700 1 --2- 172.21.15.7:0/3492200034 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4b34066f30 0x7f4b340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:45.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.683+0000 7f4b2effd700 1 -- 172.21.15.7:0/3492200034 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4b38014070 con 0x7f4b4806e040 2024-01-27T02:33:45.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.683+0000 7f4b2effd700 1 -- 172.21.15.7:0/3492200034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=49}) v3 -- 0x7f4b3406ea50 con 0x7f4b4806e040 2024-01-27T02:33:45.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.683+0000 7f4b467fc700 1 --2- 172.21.15.7:0/3492200034 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4b34066f30 0x7f4b340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:45.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.684+0000 7f4b467fc700 1 --2- 172.21.15.7:0/3492200034 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4b34066f30 0x7f4b340693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4b480cd770 tx=0x7f4b3c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:45.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.687+0000 7f4b2effd700 1 -- 172.21.15.7:0/3492200034 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b380533f0 con 0x7f4b4806e040 2024-01-27T02:33:45.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.889+0000 7f4b4d90a700 1 -- 172.21.15.7:0/3492200034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f4b4806d840 con 0x7f4b4806e040 2024-01-27T02:33:45.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.890+0000 7f4b2effd700 1 -- 172.21.15.7:0/3492200034 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v48) v1 ==== 81+0+32355 (secure 0 0 0) 0x7f4b38056a10 con 0x7f4b4806e040 2024-01-27T02:33:45.891 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:45.891 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:33:45.891 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 48, 2024-01-27T02:33:45.891 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:33:45.891 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:33:45.891 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:33:41.912319+0000", 2024-01-27T02:33:45.891 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:33:45.891 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:33:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:33:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:33:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:33:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:33:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:33:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:33:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:33:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:33:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:33:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:33:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:33:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:33:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 7, 2024-01-27T02:33:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:33:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:33:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:33:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:33:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:33:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:33:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:33:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:33:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:33:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:33:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:33:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:33:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:33:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:33:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:33:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:33:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:33:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:33:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:33:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:33:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:33:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:33:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:33:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:33:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:33:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:33:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:33:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:33:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:33:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:33:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:33:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:33:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "46", 2024-01-27T02:33:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:33:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:33:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:33:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:33:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:33:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:33:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:33:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:33:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:33:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:33:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:33:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:33:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:33:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:33:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:33:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:33:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:33:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:33:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:33:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:33:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:33:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:33:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:33:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:33:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:33:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:33:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:33:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:33:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:33:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:33:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:33:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:33:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:33:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:33:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:33:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:33:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:33:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:33:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:33:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:33:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:33:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:33:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 6, 2024-01-27T02:33:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:33:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:33:23.445643+0000", 2024-01-27T02:33:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:33:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:33:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:33:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:33:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:33:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:33:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:33:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:33:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:33:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:33:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:33:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:33:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:33:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:33:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:33:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:33:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:33:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:33:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:33:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:33:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:33:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:33:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:33:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:33:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "46", 2024-01-27T02:33:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:33:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:33:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:33:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:33:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:33:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:33:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:33:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:33:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:33:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:33:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:33:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:33:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:33:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:33:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:33:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:33:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:33:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:33:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:33:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:33:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:33:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:33:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:33:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:33:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:33:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:33:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:33:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:33:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:33:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:33:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:33:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:33:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:33:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:33:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:33:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:33:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:33:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:33:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:33:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:33:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:33:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:33:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:33:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:33:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 7, 2024-01-27T02:33:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:33:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:33:23.921995+0000", 2024-01-27T02:33:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:33:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:33:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:33:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:33:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:33:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:33:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:33:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:33:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:33:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:33:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:33:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:33:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:33:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:33:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:33:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:33:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:33:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:33:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:33:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:33:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:33:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:33:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:33:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:33:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "46", 2024-01-27T02:33:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:33:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:33:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:33:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:33:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:33:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:33:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:33:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:33:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:33:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:33:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:33:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:33:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:33:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:33:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:33:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:33:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:33:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:33:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:33:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:33:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:33:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:33:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:33:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:33:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:33:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:33:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:33:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:33:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:33:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:33:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:33:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:33:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:33:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:33:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:33:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:33:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:33:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:33:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:33:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:33:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:33:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:33:45.910 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:33:45.910 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.910 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.910 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.910 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:45.910 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:33:45.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.910 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:33:45.910 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:45.910 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:45.910 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:45.910 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:45.910 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:45.910 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:45.911 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:45.911 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:45.911 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:45.911 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:45.911 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:45.911 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:45.911 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.911 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.911 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:33:45.911 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:45.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.911 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.912 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:33:45.912 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:45.912 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.912 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.912 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:45.912 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.912 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:33:45.912 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:45.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.913 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:33:45.913 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:45.913 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.913 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.913 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:45.913 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.913 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.913 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:33:45.913 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:45.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.913 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.914 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:33:45.914 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:45.914 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.914 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.914 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:45.914 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.914 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:33:45.914 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:45.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.915 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.915 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:33:45.915 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:45.915 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.915 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.915 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:33:45.915 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:33:45.915 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:33:45.915 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:33:45.915 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:45.915 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:45.916 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:33:45.916 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:45.916 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.916 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:33:45.916 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:45.916 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:45.916 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:45.916 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:45.916 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:45.916 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:45.916 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:45.916 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:45.917 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:45.917 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:45.917 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:45.917 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:45.917 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.917 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.917 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:33:45.917 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:45.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.917 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.917 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:33:45.917 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:45.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.918 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.918 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:45.918 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.918 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:33:45.918 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:45.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.918 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:33:45.918 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:45.919 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.919 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.919 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:45.919 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.919 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:33:45.919 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:45.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.919 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:33:45.919 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:45.919 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.920 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.920 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:45.920 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.920 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.920 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:33:45.920 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:45.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.920 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.920 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:33:45.920 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:45.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.921 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.921 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:33:45.921 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:33:45.921 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:33:45.921 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:33:45.921 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:45.921 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:45.921 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:45.921 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.922 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:33:45.922 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:45.922 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:45.922 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:45.922 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:45.922 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:45.922 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:45.922 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:45.922 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:45.922 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:45.922 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:45.922 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:45.922 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:45.922 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.923 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:33:45.923 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:45.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.923 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:33:45.923 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:45.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.923 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.923 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:45.923 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.924 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:33:45.924 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:45.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.924 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:33:45.924 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:45.924 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.924 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.924 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:45.924 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.925 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:33:45.925 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:45.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.925 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.925 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:33:45.925 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:45.925 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.925 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.925 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:45.926 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.926 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.926 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:33:45.926 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:45.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.926 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.926 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:33:45.926 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:45.926 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.926 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.926 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:33:45.927 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:33:45.927 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:33:45.927 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:33:45.927 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:45.927 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:45.927 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:33:45.927 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:45.927 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.927 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:33:45.927 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:45.927 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:45.927 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:45.928 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:45.928 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:45.928 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:45.928 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:45.928 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:45.928 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:45.928 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:45.928 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:45.928 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:45.928 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.928 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.928 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:33:45.928 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:45.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.929 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:33:45.929 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:45.929 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.929 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.929 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:45.929 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.929 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:33:45.929 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:45.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.930 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.930 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:33:45.930 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:45.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.930 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.930 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:45.930 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.930 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.930 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:33:45.930 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:45.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.931 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:33:45.931 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:45.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.931 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.931 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:45.931 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.931 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:33:45.931 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:45.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.932 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:33:45.932 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:45.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.932 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.932 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:33:45.932 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:33:45.932 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:33:45.932 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:33:45.932 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:45.932 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull", 2024-01-27T02:33:45.932 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:45.933 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:45.933 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.933 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:33:45.933 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:45.933 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:45.933 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:45.933 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:45.934 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:45.934 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:45.934 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:45.934 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:45.934 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:45.934 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:45.934 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:45.934 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:45.934 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.934 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.935 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:33:45.935 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:45.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.935 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:33:45.935 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:45.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.935 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.936 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:45.936 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.936 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:33:45.936 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:45.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.936 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:33:45.936 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:45.936 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.936 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.937 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:45.937 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.937 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:33:45.937 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:45.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.937 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:33:45.937 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:45.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.937 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.937 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:45.938 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.938 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:33:45.938 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:45.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.938 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:33:45.938 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:45.938 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.938 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.939 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:33:45.939 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:33:45.939 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:33:45.939 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:33:45.939 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:45.939 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:45.939 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:45.939 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.939 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:33:45.940 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:45.940 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:45.940 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:45.940 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:45.940 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:45.940 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:45.940 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:45.940 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:45.940 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:45.940 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:45.940 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:33:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:33:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.942 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:45.942 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.942 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:33:45.942 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:45.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.942 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:33:45.942 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:45.942 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.942 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.943 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:45.943 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.943 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:33:45.943 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:45.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.943 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:33:45.943 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:45.943 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.943 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.944 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:45.944 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.944 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:33:45.944 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:45.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.944 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:33:45.944 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:45.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.944 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.945 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:33:45.945 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:33:45.945 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:33:45.945 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:33:45.945 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:45.945 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:45.945 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:45.945 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.945 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:33:45.945 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:45.945 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:45.945 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:45.946 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:45.946 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:45.946 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:45.946 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:45.946 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:45.946 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:45.946 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:45.946 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:45.946 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:45.946 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.946 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:33:45.946 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:45.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.947 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:33:45.947 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:45.947 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.947 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.947 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:45.947 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.947 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:33:45.947 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:45.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.948 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:33:45.948 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:45.948 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.948 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.948 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:45.948 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.949 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:33:45.949 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:45.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.949 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:33:45.949 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:45.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.949 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.950 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:45.950 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.950 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:33:45.950 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:45.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.950 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:33:45.950 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:45.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.951 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.951 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:33:45.951 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:33:45.951 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:33:45.951 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:33:45.951 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:45.951 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:45.951 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:33:45.951 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:45.951 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.951 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:33:45.952 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:45.952 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:45.952 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:45.952 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:45.952 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:45.952 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:45.952 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:45.952 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:45.952 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:45.952 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:45.952 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:45.952 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:45.952 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.953 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:33:45.953 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:45.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.953 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:33:45.953 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:45.953 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.953 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.953 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:45.953 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.954 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:33:45.954 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:45.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.954 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:33:45.954 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:45.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.954 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.954 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:45.954 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:33:45.955 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:45.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:33:45.955 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:45.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.955 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.955 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:45.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:45.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:45.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:33:45.956 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:45.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:45.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:33:45.956 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:45.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.956 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.956 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:33:45.956 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:33:45.956 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:33:45.957 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:33:45.957 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:45.957 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:45.957 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:33:45.957 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:45.957 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:45.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.957 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:45.957 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:33:45.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.957 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:33:45.957 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:45.957 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:45.957 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:45.957 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:45.958 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:45.958 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:33:45.958 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:45.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.958 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:33:45.958 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:45.958 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:45.958 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:45.958 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:45.958 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:45.958 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:33:45.958 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:45.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.959 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:33:45.959 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:45.959 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:45.959 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:45.959 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:45.959 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:45.959 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:33:45.959 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:45.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.959 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:33:45.959 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:45.959 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:45.959 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:45.960 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:45.960 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:45.960 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:33:45.960 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:45.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.960 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:33:45.960 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:45.960 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:45.960 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:45.960 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:45.960 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:45.960 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:33:45.960 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:45.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.961 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:33:45.961 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:45.961 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:45.961 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:45.961 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:45.961 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:45.961 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:33:45.961 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:45.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.961 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:33:45.961 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:45.961 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:45.962 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:45.962 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:45.962 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:45.962 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:33:45.962 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:45.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:45.962 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:33:45.962 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:45.962 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:45.962 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:45.962 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:45.962 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:45.962 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:33:45.962 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:45.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.963 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:45.963 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:33:45.963 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:33:45.963 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:33:45.963 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:33:45.963 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:33:45.963 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:33:45.963 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:33:45.963 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:33:45.963 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:33:45.963 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:33:45.963 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:33:45.963 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:33:45.963 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:33:45.964 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:45.964 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:33:45.964 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:33:45.964 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:33:45.964 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:33:45.964 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:33:45.964 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:33:45.964 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:33:45.964 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:33:45.964 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:33:45.964 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:33:45.964 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:33:45.964 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:45.965 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:33:45.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.892+0000 7f4b4d90a700 1 -- 172.21.15.7:0/3492200034 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4b34066f30 msgr2=0x7f4b340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:45.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.892+0000 7f4b4d90a700 1 --2- 172.21.15.7:0/3492200034 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4b34066f30 0x7f4b340693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4b480cd770 tx=0x7f4b3c008040 comp rx=0 tx=0).stop 2024-01-27T02:33:45.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.892+0000 7f4b4d90a700 1 -- 172.21.15.7:0/3492200034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b4806e040 msgr2=0x7f4b480b02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:45.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.892+0000 7f4b4d90a700 1 --2- 172.21.15.7:0/3492200034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b4806e040 0x7f4b480b02c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4b38006fd0 tx=0x7f4b3801b000 comp rx=0 tx=0).stop 2024-01-27T02:33:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.892+0000 7f4b4d90a700 1 -- 172.21.15.7:0/3492200034 shutdown_connections 2024-01-27T02:33:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.892+0000 7f4b4d90a700 1 --2- 172.21.15.7:0/3492200034 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4b34066f30 0x7f4b340693e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.892+0000 7f4b4d90a700 1 --2- 172.21.15.7:0/3492200034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b48156c00 0x7f4b48150270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.892+0000 7f4b4d90a700 1 --2- 172.21.15.7:0/3492200034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b4806e040 0x7f4b480b02c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.892+0000 7f4b4d90a700 1 --2- 172.21.15.7:0/3492200034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b4806e4b0 0x7f4b48156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.892+0000 7f4b4d90a700 1 -- 172.21.15.7:0/3492200034 >> 172.21.15.7:0/3492200034 conn(0x7f4b48063580 msgr2=0x7f4b48069a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.892+0000 7f4b4d90a700 1 -- 172.21.15.7:0/3492200034 shutdown_connections 2024-01-27T02:33:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:45.892+0000 7f4b4d90a700 1 -- 172.21.15.7:0/3492200034 wait complete. 2024-01-27T02:33:45.968 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-27T02:33:49.809 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-27T02:33:49.809+0000 7fb5c783f700 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2024-01-27T02:33:50.969 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:33:51.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.109+0000 7ff243284700 1 Processor -- start 2024-01-27T02:33:51.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.109+0000 7ff243284700 1 -- start start 2024-01-27T02:33:51.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.110+0000 7ff243284700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff23c157ca0 0x7ff23c0c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:51.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.110+0000 7ff243284700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff23c0c3ae0 0x7ff23c0cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:51.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.110+0000 7ff243284700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23c0ca020 con 0x7ff23c158070 2024-01-27T02:33:51.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.110+0000 7ff243284700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff23c0c4f40 con 0x7ff23c157ca0 2024-01-27T02:33:51.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.110+0000 7ff243284700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff23c0c5210 con 0x7ff23c0c3ae0 2024-01-27T02:33:51.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.110+0000 7ff241020700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff23c157ca0 0x7ff23c0c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:51.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.110+0000 7ff241020700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff23c157ca0 0x7ff23c0c4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44654/0 (socket says 172.21.15.7:44654) 2024-01-27T02:33:51.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.110+0000 7ff241821700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff23c158070 0x7ff23c15bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47488/0 (socket says 172.21.15.7:47488) 2024-01-27T02:33:51.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.110+0000 7ff241020700 1 -- 172.21.15.7:0/2036218225 learned_addr learned my addr 172.21.15.7:0/2036218225 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:51.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.110+0000 7ff24081f700 1 --2- 172.21.15.7:0/2036218225 >> v2:172.21.15.188:3300/0 conn(0x7ff23c0c3ae0 0x7ff23c0cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:51.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.111+0000 7ff241020700 1 -- 172.21.15.7:0/2036218225 >> v2:172.21.15.188:3300/0 conn(0x7ff23c0c3ae0 msgr2=0x7ff23c0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:51.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.111+0000 7ff241020700 1 --2- 172.21.15.7:0/2036218225 >> v2:172.21.15.188:3300/0 conn(0x7ff23c0c3ae0 0x7ff23c0cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:51.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.111+0000 7ff241020700 1 -- 172.21.15.7:0/2036218225 >> v1:172.21.15.7:6789/0 conn(0x7ff23c158070 legacy=0x7ff23c15bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:51.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.111+0000 7ff233fff700 1 -- 172.21.15.7:0/2036218225 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245988528 0 0) 0x7ff23c0ca020 con 0x7ff23c158070 2024-01-27T02:33:51.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.111+0000 7ff241020700 1 -- 172.21.15.7:0/2036218225 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c0c54f0 con 0x7ff23c157ca0 2024-01-27T02:33:51.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.111+0000 7ff241020700 1 --2- 172.21.15.7:0/2036218225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff23c157ca0 0x7ff23c0c4a00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff23c06e170 tx=0x7ff238009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6841d2d951edb0e9 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:51.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.111+0000 7ff233fff700 1 -- 172.21.15.7:0/2036218225 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff238009e70 con 0x7ff23c157ca0 2024-01-27T02:33:51.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.111+0000 7ff233fff700 1 -- 172.21.15.7:0/2036218225 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23800b860 con 0x7ff23c157ca0 2024-01-27T02:33:51.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.111+0000 7ff233fff700 1 -- 172.21.15.7:0/2036218225 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23800bc00 con 0x7ff23c157ca0 2024-01-27T02:33:51.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.111+0000 7ff243284700 1 -- 172.21.15.7:0/2036218225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff23c157ca0 msgr2=0x7ff23c0c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:51.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.111+0000 7ff243284700 1 --2- 172.21.15.7:0/2036218225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff23c157ca0 0x7ff23c0c4a00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff23c06e170 tx=0x7ff238009250 comp rx=0 tx=0).stop 2024-01-27T02:33:51.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.112+0000 7ff243284700 1 -- 172.21.15.7:0/2036218225 shutdown_connections 2024-01-27T02:33:51.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.112+0000 7ff243284700 1 --2- 172.21.15.7:0/2036218225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff23c157ca0 0x7ff23c0c4a00 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:51.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.112+0000 7ff243284700 1 --2- 172.21.15.7:0/2036218225 >> v2:172.21.15.188:3300/0 conn(0x7ff23c0c3ae0 0x7ff23c0cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:51.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.112+0000 7ff243284700 1 -- 172.21.15.7:0/2036218225 >> 172.21.15.7:0/2036218225 conn(0x7ff23c098020 msgr2=0x7ff23c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:51.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.112+0000 7ff243284700 1 -- 172.21.15.7:0/2036218225 shutdown_connections 2024-01-27T02:33:51.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.112+0000 7ff243284700 1 -- 172.21.15.7:0/2036218225 wait complete. 2024-01-27T02:33:51.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.113+0000 7ff243284700 1 Processor -- start 2024-01-27T02:33:51.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.113+0000 7ff243284700 1 -- start start 2024-01-27T02:33:51.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.115+0000 7ff243284700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff23c0c3ae0 0x7ff23c0b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:51.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.115+0000 7ff243284700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff23c157ca0 0x7ff23c0b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:51.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.115+0000 7ff243284700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff23c158070 0x7ff23c0c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:51.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.115+0000 7ff243284700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c14ff10 con 0x7ff23c157ca0 2024-01-27T02:33:51.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.115+0000 7ff243284700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0c5f20 con 0x7ff23c158070 2024-01-27T02:33:51.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.115+0000 7ff243284700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c0b3400 con 0x7ff23c0c3ae0 2024-01-27T02:33:51.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.115+0000 7ff24081f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff23c157ca0 0x7ff23c0b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:51.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.115+0000 7ff24081f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff23c157ca0 0x7ff23c0b24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44662/0 (socket says 172.21.15.7:44662) 2024-01-27T02:33:51.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.115+0000 7ff24081f700 1 -- 172.21.15.7:0/2967927962 learned_addr learned my addr 172.21.15.7:0/2967927962 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:51.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.115+0000 7ff241821700 1 --2- 172.21.15.7:0/2967927962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff23c158070 0x7ff23c0c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:51.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.115+0000 7ff241020700 1 --2- 172.21.15.7:0/2967927962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff23c0c3ae0 0x7ff23c0b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:51.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.115+0000 7ff24081f700 1 -- 172.21.15.7:0/2967927962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff23c0c3ae0 msgr2=0x7ff23c0b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:51.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.115+0000 7ff24081f700 1 --2- 172.21.15.7:0/2967927962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff23c0c3ae0 0x7ff23c0b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:51.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.115+0000 7ff24081f700 1 -- 172.21.15.7:0/2967927962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff23c158070 msgr2=0x7ff23c0c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:51.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.115+0000 7ff24081f700 1 --2- 172.21.15.7:0/2967927962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff23c158070 0x7ff23c0c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:51.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.115+0000 7ff24081f700 1 -- 172.21.15.7:0/2967927962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2380094f0 con 0x7ff23c157ca0 2024-01-27T02:33:51.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.115+0000 7ff241821700 1 --2- 172.21.15.7:0/2967927962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff23c158070 0x7ff23c0c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:51.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.115+0000 7ff241020700 1 --2- 172.21.15.7:0/2967927962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff23c0c3ae0 0x7ff23c0b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:51.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.115+0000 7ff24081f700 1 --2- 172.21.15.7:0/2967927962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff23c157ca0 0x7ff23c0b24c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00ec80 tx=0x7ff22c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:51.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.116+0000 7ff2317fa700 1 -- 172.21.15.7:0/2967927962 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c00c9d0 con 0x7ff23c157ca0 2024-01-27T02:33:51.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.116+0000 7ff2317fa700 1 -- 172.21.15.7:0/2967927962 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22c00cb30 con 0x7ff23c157ca0 2024-01-27T02:33:51.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.116+0000 7ff243284700 1 -- 172.21.15.7:0/2967927962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff23c0b2b60 con 0x7ff23c157ca0 2024-01-27T02:33:51.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.116+0000 7ff2317fa700 1 -- 172.21.15.7:0/2967927962 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c013470 con 0x7ff23c157ca0 2024-01-27T02:33:51.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.116+0000 7ff243284700 1 -- 172.21.15.7:0/2967927962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23c0b3120 con 0x7ff23c157ca0 2024-01-27T02:33:51.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.117+0000 7ff243284700 1 -- 172.21.15.7:0/2967927962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff23c0c60b0 con 0x7ff23c157ca0 2024-01-27T02:33:51.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.117+0000 7ff2317fa700 1 -- 172.21.15.7:0/2967927962 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff22c013820 con 0x7ff23c157ca0 2024-01-27T02:33:51.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.118+0000 7ff2317fa700 1 --2- 172.21.15.7:0/2967927962 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff220066f30 0x7ff2200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:51.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.118+0000 7ff2317fa700 1 -- 172.21.15.7:0/2967927962 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff22c0848c0 con 0x7ff23c157ca0 2024-01-27T02:33:51.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.118+0000 7ff2317fa700 1 -- 172.21.15.7:0/2967927962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=50}) v3 -- 0x7ff22006ea50 con 0x7ff23c157ca0 2024-01-27T02:33:51.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.118+0000 7ff241020700 1 --2- 172.21.15.7:0/2967927962 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff220066f30 0x7ff2200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:51.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.119+0000 7ff241020700 1 --2- 172.21.15.7:0/2967927962 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff220066f30 0x7ff2200693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff23c06e170 tx=0x7ff238017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:51.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.122+0000 7ff2317fa700 1 -- 172.21.15.7:0/2967927962 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff22c017060 con 0x7ff23c157ca0 2024-01-27T02:33:51.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.329+0000 7ff243284700 1 -- 172.21.15.7:0/2967927962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7ff23c04c710 con 0x7ff23c157ca0 2024-01-27T02:33:51.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.330+0000 7ff2317fa700 1 -- 172.21.15.7:0/2967927962 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v49) v1 ==== 81+0+32331 (secure 0 0 0) 0x7ff22c021090 con 0x7ff23c157ca0 2024-01-27T02:33:51.331 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:51.332 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:33:51.332 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 49, 2024-01-27T02:33:51.332 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:33:51.332 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:33:51.332 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:33:50.935985+0000", 2024-01-27T02:33:51.332 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:33:51.332 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:33:51.332 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:33:51.332 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:33:51.332 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:33:51.332 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:33:51.332 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:33:51.332 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:33:51.332 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:33:51.333 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:51.333 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:33:51.333 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:33:51.333 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:33:51.333 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:33:51.333 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:33:51.333 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 7, 2024-01-27T02:33:51.333 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:33:51.333 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:33:51.333 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:33:51.333 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:33:51.333 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:33:51.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.334 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:33:51.334 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:33:51.334 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:33:51.334 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:33:51.334 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:33:51.334 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:33:51.334 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:33:51.334 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:33:51.334 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:33:51.334 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:33:51.334 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:33:51.334 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:33:51.335 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:33:51.335 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:33:51.335 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:33:51.335 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:33:51.335 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:33:51.335 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:33:51.335 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:33:51.335 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:33:51.335 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:33:51.335 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:33:51.335 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:33:51.335 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:33:51.335 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:33:51.335 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:33:51.335 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:33:51.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.336 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "46", 2024-01-27T02:33:51.336 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:33:51.336 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:33:51.336 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:33:51.336 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:33:51.336 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:33:51.336 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:33:51.336 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:33:51.336 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:33:51.336 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:33:51.336 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:33:51.336 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:33:51.337 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:33:51.337 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:33:51.337 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:33:51.337 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:33:51.337 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:33:51.337 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:33:51.337 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:33:51.337 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:33:51.337 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:33:51.337 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:33:51.337 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:33:51.337 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:33:51.337 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:33:51.338 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:33:51.338 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:33:51.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.338 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:33:51.338 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:33:51.338 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:33:51.338 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:33:51.338 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:33:51.338 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:33:51.338 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:33:51.338 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:33:51.338 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:33:51.338 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:33:51.338 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:33:51.338 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:33:51.339 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:33:51.339 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:33:51.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.339 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:33:51.339 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:33:51.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.339 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 6, 2024-01-27T02:33:51.339 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:33:51.339 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:33:23.445643+0000", 2024-01-27T02:33:51.339 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:33:51.339 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:33:51.340 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:33:51.340 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:33:51.340 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:33:51.340 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:33:51.340 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:33:51.340 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:33:51.340 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:33:51.340 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:33:51.340 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:33:51.340 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:33:51.340 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:33:51.340 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:33:51.340 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:33:51.341 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:33:51.341 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:33:51.341 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:33:51.341 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:33:51.341 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:33:51.341 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:33:51.341 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:33:51.341 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:33:51.341 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:33:51.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.341 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "46", 2024-01-27T02:33:51.341 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:33:51.341 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:33:51.341 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:33:51.342 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:33:51.342 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:33:51.342 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:33:51.342 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:33:51.342 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:33:51.342 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:33:51.342 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:33:51.342 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:33:51.342 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:33:51.342 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:33:51.342 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:33:51.342 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:33:51.342 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:33:51.342 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:33:51.342 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:33:51.343 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:33:51.343 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:33:51.343 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:33:51.343 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:33:51.343 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:33:51.343 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:33:51.343 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:33:51.343 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:33:51.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.343 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:33:51.343 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:33:51.343 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:33:51.343 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:33:51.343 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:33:51.343 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:33:51.344 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:33:51.344 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:33:51.344 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:33:51.344 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:33:51.344 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:33:51.344 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:33:51.344 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:33:51.344 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:33:51.344 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:33:51.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.344 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:33:51.344 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:33:51.344 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:33:51.344 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.345 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 7, 2024-01-27T02:33:51.345 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:33:51.345 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:33:23.921995+0000", 2024-01-27T02:33:51.345 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:33:51.345 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:33:51.345 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:33:51.345 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:33:51.345 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:33:51.345 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:33:51.345 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:33:51.345 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:33:51.345 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:33:51.346 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:33:51.346 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:33:51.346 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:33:51.346 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:33:51.346 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:33:51.346 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:33:51.346 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:33:51.346 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:33:51.346 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:33:51.346 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:33:51.346 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:33:51.346 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:33:51.346 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:33:51.347 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:33:51.347 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:33:51.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.347 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "46", 2024-01-27T02:33:51.347 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:33:51.347 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:33:51.347 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:33:51.347 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:33:51.347 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:33:51.347 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:33:51.347 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:33:51.347 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:33:51.347 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:33:51.347 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:33:51.348 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:33:51.348 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:33:51.348 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:33:51.348 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:33:51.348 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:33:51.348 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:33:51.348 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:33:51.348 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:33:51.348 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:33:51.348 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:33:51.348 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:33:51.348 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:33:51.348 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:33:51.349 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:33:51.349 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:33:51.349 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:33:51.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.349 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:33:51.349 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:33:51.349 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:33:51.349 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:33:51.349 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:33:51.349 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:33:51.349 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:33:51.349 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:33:51.349 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:33:51.349 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:33:51.350 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:33:51.350 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:33:51.350 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:33:51.350 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:33:51.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.350 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:33:51.350 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:33:51.350 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:33:51.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.350 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:51.350 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:33:51.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.351 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:33:51.351 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:51.351 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:51.351 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:51.351 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:51.351 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:51.351 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:51.351 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:51.351 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:51.351 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:51.351 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:51.351 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:51.351 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:51.351 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.352 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.352 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:33:51.352 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:51.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.352 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.352 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:33:51.352 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:51.352 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.352 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.352 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:51.352 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.353 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.353 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:33:51.353 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:51.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.353 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.353 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:33:51.353 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:51.353 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.353 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.353 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:51.353 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.354 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.354 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:33:51.354 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:51.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.354 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.354 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:33:51.354 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:51.354 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.354 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.354 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:51.354 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.355 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.355 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:33:51.355 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:51.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.355 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.355 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:33:51.355 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:51.355 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.355 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.355 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:33:51.355 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:33:51.355 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:33:51.356 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:33:51.356 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:51.356 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:51.356 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:51.356 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.356 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:33:51.356 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:51.356 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:51.356 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:51.356 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:51.356 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:51.357 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:51.357 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:51.357 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:51.357 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:51.357 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:51.357 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:51.357 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:51.357 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.357 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.357 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:33:51.357 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:51.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.357 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.358 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:33:51.358 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:51.358 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.358 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.358 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:51.358 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.358 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.358 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:33:51.358 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:51.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.358 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.359 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:33:51.359 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:51.359 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.359 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.359 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:51.359 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.359 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.359 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:33:51.359 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:51.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.359 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.360 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:33:51.360 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:51.360 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.360 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.360 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:51.360 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.360 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.360 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:33:51.360 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:51.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.360 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.361 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:33:51.361 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:51.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.361 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.361 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:33:51.361 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:33:51.361 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:33:51.361 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:33:51.361 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:51.361 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:51.361 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:51.361 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.362 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:33:51.362 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:51.362 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:51.362 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:51.362 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:51.362 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:51.362 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:51.362 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:51.362 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:51.362 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:51.362 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:51.362 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:51.362 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:51.362 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.363 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.363 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:33:51.363 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:51.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.363 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.363 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:33:51.363 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:51.363 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.363 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.363 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:51.363 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.364 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.364 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:33:51.364 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:51.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.364 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.364 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:33:51.364 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:51.364 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.364 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.364 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:51.364 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.365 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.365 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:33:51.365 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:51.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.365 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.365 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:33:51.365 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:51.365 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.365 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.365 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:51.365 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.366 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.366 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:33:51.366 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:51.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.366 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.366 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:33:51.366 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:51.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.366 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.366 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:33:51.366 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:33:51.366 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:33:51.367 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:33:51.367 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:51.367 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:51.367 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:33:51.367 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:51.367 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.367 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:33:51.367 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:51.367 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:51.367 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:51.367 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:51.367 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:51.368 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:51.368 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:51.368 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:51.368 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:51.368 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:51.368 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:51.368 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:51.368 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.368 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.368 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:33:51.368 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:51.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.369 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.369 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:33:51.369 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:51.369 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.369 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.369 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:51.369 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.369 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.369 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:33:51.369 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:51.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.370 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.370 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:33:51.370 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:51.370 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.370 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.370 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:51.370 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.370 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.370 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:33:51.370 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:51.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.371 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:33:51.371 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:51.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.371 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.371 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:51.371 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.371 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:33:51.371 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:51.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.372 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:33:51.372 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:51.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.372 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.372 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:33:51.372 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:33:51.372 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:33:51.372 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:33:51.372 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:51.372 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull", 2024-01-27T02:33:51.373 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:51.373 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:51.373 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.373 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:33:51.373 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:51.373 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:51.373 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:51.373 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:51.373 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:51.373 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:51.373 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:51.373 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:51.374 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:51.374 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:51.374 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:51.374 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:51.374 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.374 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.374 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:33:51.374 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:51.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.374 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.374 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:33:51.374 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:51.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.375 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.375 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:51.375 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.375 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.375 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:33:51.375 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:51.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.375 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.375 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:33:51.375 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:51.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.376 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.376 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:51.376 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.376 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:33:51.376 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:51.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.376 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:33:51.376 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:51.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.377 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.377 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:51.377 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.377 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.377 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:33:51.377 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:51.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.377 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.377 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:33:51.377 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:51.377 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.378 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.378 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:33:51.378 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:33:51.378 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:33:51.378 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:33:51.378 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:51.378 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:51.378 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:51.378 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.378 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:33:51.379 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:51.379 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:51.379 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:51.379 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:51.379 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:51.379 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:51.379 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:51.379 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:51.379 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:51.379 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:51.379 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:51.379 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:51.379 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.380 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:33:51.380 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:51.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.380 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:33:51.380 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:51.380 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.380 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.380 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:51.380 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.381 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:33:51.381 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:51.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.381 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:33:51.381 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:51.381 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.381 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.381 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:51.381 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.382 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.382 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:33:51.382 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:51.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.382 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.382 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:33:51.382 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:51.382 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.382 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.382 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:51.382 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.382 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.383 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:33:51.383 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:51.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.383 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.383 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:33:51.383 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:51.383 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.383 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.383 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:33:51.383 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:33:51.383 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:33:51.383 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:33:51.384 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:51.384 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:51.384 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:51.384 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.384 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:33:51.384 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:51.384 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:51.384 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:51.384 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:51.384 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:51.384 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:51.384 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:51.385 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:51.385 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:51.385 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:51.385 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:51.385 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:51.385 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.385 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:33:51.385 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:51.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.385 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:33:51.385 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:51.386 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.386 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.386 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:51.386 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.386 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:33:51.386 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:51.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.386 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:33:51.386 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:51.387 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.387 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.387 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:51.387 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.387 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.387 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:33:51.387 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:51.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.387 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.387 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:33:51.387 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:51.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.388 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.388 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:51.388 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.388 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.388 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:33:51.388 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:51.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.388 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.388 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:33:51.388 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:51.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.389 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.389 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:33:51.389 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:33:51.389 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:33:51.389 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:33:51.389 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:51.389 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:51.389 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:33:51.389 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:51.389 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.389 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:33:51.390 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:51.390 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:51.390 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:51.390 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:51.390 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:51.390 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:51.390 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:51.390 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:51.390 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:51.390 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:51.390 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:51.390 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:51.390 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.391 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:33:51.391 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:51.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.391 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:33:51.391 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:51.391 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.391 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.391 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:51.391 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.392 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:33:51.392 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:51.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.392 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.392 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:33:51.392 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:51.392 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.392 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.392 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:51.392 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.393 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.393 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:33:51.393 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:51.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.393 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.393 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:33:51.393 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:51.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.393 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.393 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:51.393 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:51.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.393 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:51.394 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:33:51.394 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:51.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:51.394 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:33:51.394 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:51.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.394 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.394 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:33:51.394 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:33:51.394 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:33:51.394 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:33:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:33:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:33:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:33:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:33:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:33:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:33:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:51.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.397 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:33:51.397 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:51.397 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:51.397 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:51.397 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:51.397 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:51.397 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:33:51.397 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:51.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.397 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:33:51.397 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:51.397 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:51.398 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:51.398 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:51.398 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:51.398 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:33:51.398 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:51.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.398 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:33:51.398 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:51.398 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:51.398 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:51.398 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:51.398 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:51.398 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:33:51.398 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:51.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.399 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:33:51.399 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:51.399 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:51.399 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:51.399 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:51.399 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:51.399 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:33:51.399 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:51.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.399 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:33:51.399 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:51.399 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:51.400 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:51.400 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:51.400 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:51.400 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:33:51.400 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:51.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:51.400 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:33:51.400 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:51.400 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:51.400 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:51.400 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:51.400 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:51.400 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:33:51.401 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:51.401 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.401 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:51.401 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:33:51.401 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:33:51.401 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:33:51.401 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:33:51.401 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:33:51.401 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:33:51.401 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:33:51.401 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:33:51.401 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:33:51.401 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:33:51.401 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:33:51.401 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:33:51.402 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:33:51.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:51.402 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:33:51.402 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:33:51.402 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:33:51.402 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:33:51.402 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:33:51.402 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:33:51.402 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:33:51.402 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:33:51.402 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:33:51.402 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:33:51.403 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:33:51.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:51.403 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:33:51.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.333+0000 7ff243284700 1 -- 172.21.15.7:0/2967927962 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff220066f30 msgr2=0x7ff2200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:51.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.333+0000 7ff243284700 1 --2- 172.21.15.7:0/2967927962 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff220066f30 0x7ff2200693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff23c06e170 tx=0x7ff238017000 comp rx=0 tx=0).stop 2024-01-27T02:33:51.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.333+0000 7ff243284700 1 -- 172.21.15.7:0/2967927962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff23c157ca0 msgr2=0x7ff23c0b24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:51.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.333+0000 7ff243284700 1 --2- 172.21.15.7:0/2967927962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff23c157ca0 0x7ff23c0b24c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00ec80 tx=0x7ff22c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:51.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.333+0000 7ff243284700 1 -- 172.21.15.7:0/2967927962 shutdown_connections 2024-01-27T02:33:51.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.333+0000 7ff243284700 1 --2- 172.21.15.7:0/2967927962 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff220066f30 0x7ff2200693e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:51.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.333+0000 7ff243284700 1 --2- 172.21.15.7:0/2967927962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff23c158070 0x7ff23c0c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:51.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.333+0000 7ff243284700 1 --2- 172.21.15.7:0/2967927962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff23c0c3ae0 0x7ff23c0b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:51.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.333+0000 7ff243284700 1 --2- 172.21.15.7:0/2967927962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff23c157ca0 0x7ff23c0b24c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:51.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.333+0000 7ff243284700 1 -- 172.21.15.7:0/2967927962 >> 172.21.15.7:0/2967927962 conn(0x7ff23c098020 msgr2=0x7ff23c06bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:51.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.333+0000 7ff243284700 1 -- 172.21.15.7:0/2967927962 shutdown_connections 2024-01-27T02:33:51.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:51.333+0000 7ff243284700 1 -- 172.21.15.7:0/2967927962 wait complete. 2024-01-27T02:33:51.406 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-27T02:33:56.407 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:33:56.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.549+0000 7efe47cd9700 1 Processor -- start 2024-01-27T02:33:56.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.550+0000 7efe47cd9700 1 -- start start 2024-01-27T02:33:56.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.551+0000 7efe47cd9700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efe4006ea40 0x7efe40157f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:56.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.551+0000 7efe47cd9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4006c880 con 0x7efe400c9f80 2024-01-27T02:33:56.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.551+0000 7efe47cd9700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4006bc70 con 0x7efe400c9bb0 2024-01-27T02:33:56.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.551+0000 7efe47cd9700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe4006cd10 con 0x7efe4006ea40 2024-01-27T02:33:56.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.551+0000 7efe45274700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efe400c9f80 0x7efe4006e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59242/0 (socket says 172.21.15.7:59242) 2024-01-27T02:33:56.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.551+0000 7efe45274700 1 -- 172.21.15.7:0/3255218993 learned_addr learned my addr 172.21.15.7:0/3255218993 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:56.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.551+0000 7efe44a73700 1 -- 172.21.15.7:0/3255218993 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3625481855 0 0) 0x7efe4006c880 con 0x7efe400c9f80 2024-01-27T02:33:56.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.551+0000 7efe46276700 1 --2- 172.21.15.7:0/3255218993 >> v2:172.21.15.188:3300/0 conn(0x7efe4006ea40 0x7efe40157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:56.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.551+0000 7efe44a73700 1 -- 172.21.15.7:0/3255218993 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe2c003570 con 0x7efe400c9f80 2024-01-27T02:33:56.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.551+0000 7efe44a73700 1 -- 172.21.15.7:0/3255218993 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1595386544 0 0) 0x7efe2c003570 con 0x7efe400c9f80 2024-01-27T02:33:56.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.551+0000 7efe44a73700 1 -- 172.21.15.7:0/3255218993 >> v2:172.21.15.188:3300/0 conn(0x7efe4006ea40 msgr2=0x7efe40157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:56.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.551+0000 7efe44a73700 1 --2- 172.21.15.7:0/3255218993 >> v2:172.21.15.188:3300/0 conn(0x7efe4006ea40 0x7efe40157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:56.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.551+0000 7efe44a73700 1 -- 172.21.15.7:0/3255218993 >> v1:172.21.15.188:6789/0 conn(0x7efe400c9bb0 legacy=0x7efe4006b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:56.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.551+0000 7efe44a73700 1 -- 172.21.15.7:0/3255218993 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4006bf50 con 0x7efe400c9f80 2024-01-27T02:33:56.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.551+0000 7efe44a73700 1 -- 172.21.15.7:0/3255218993 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7efe34001470 con 0x7efe400c9f80 2024-01-27T02:33:56.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.551+0000 7efe44a73700 1 -- 172.21.15.7:0/3255218993 >> v1:172.21.15.7:6789/0 conn(0x7efe400c9f80 legacy=0x7efe4006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:56.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.551+0000 7efe44a73700 1 --2- 172.21.15.7:0/3255218993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe2c003ab0 0x7efe2c003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:56.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.551+0000 7efe44a73700 1 --2- 172.21.15.7:0/3255218993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe2c004810 0x7efe2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:56.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.551+0000 7efe44a73700 1 --2- 172.21.15.7:0/3255218993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe2c005210 0x7efe2c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:56.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.551+0000 7efe44a73700 1 -- 172.21.15.7:0/3255218993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4006cd10 con 0x7efe2c003ab0 2024-01-27T02:33:56.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.551+0000 7efe44a73700 1 -- 172.21.15.7:0/3255218993 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c00c130 con 0x7efe2c004810 2024-01-27T02:33:56.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.551+0000 7efe44a73700 1 -- 172.21.15.7:0/3255218993 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c00c2f0 con 0x7efe2c005210 2024-01-27T02:33:56.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.552+0000 7efe45a75700 1 --2- 172.21.15.7:0/3255218993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe2c003ab0 0x7efe2c003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:56.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.552+0000 7efe45274700 1 --2- 172.21.15.7:0/3255218993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe2c004810 0x7efe2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:56.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.552+0000 7efe46276700 1 --2- 172.21.15.7:0/3255218993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe2c005210 0x7efe2c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:56.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.552+0000 7efe45a75700 1 -- 172.21.15.7:0/3255218993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe2c005210 msgr2=0x7efe2c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:56.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.552+0000 7efe45a75700 1 --2- 172.21.15.7:0/3255218993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe2c005210 0x7efe2c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:56.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.552+0000 7efe45a75700 1 -- 172.21.15.7:0/3255218993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe2c004810 msgr2=0x7efe2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:56.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.552+0000 7efe45a75700 1 --2- 172.21.15.7:0/3255218993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe2c004810 0x7efe2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:56.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.552+0000 7efe45a75700 1 -- 172.21.15.7:0/3255218993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2c003570 con 0x7efe2c003ab0 2024-01-27T02:33:56.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.552+0000 7efe46276700 1 -- 172.21.15.7:0/3255218993 reap_dead start 2024-01-27T02:33:56.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.552+0000 7efe45a75700 1 --2- 172.21.15.7:0/3255218993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe2c003ab0 0x7efe2c003ec0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0018e0 tx=0x7efe3c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c84b6e527f6baef server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:56.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.552+0000 7efe44a73700 1 -- 172.21.15.7:0/3255218993 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c012420 con 0x7efe2c003ab0 2024-01-27T02:33:56.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.552+0000 7efe44a73700 1 -- 172.21.15.7:0/3255218993 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c0125a0 con 0x7efe2c003ab0 2024-01-27T02:33:56.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.552+0000 7efe44a73700 1 -- 172.21.15.7:0/3255218993 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c012960 con 0x7efe2c003ab0 2024-01-27T02:33:56.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.553+0000 7efe47cd9700 1 -- 172.21.15.7:0/3255218993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe2c003ab0 msgr2=0x7efe2c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:56.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.553+0000 7efe47cd9700 1 --2- 172.21.15.7:0/3255218993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe2c003ab0 0x7efe2c003ec0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0018e0 tx=0x7efe3c007d30 comp rx=0 tx=0).stop 2024-01-27T02:33:56.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.553+0000 7efe47cd9700 1 -- 172.21.15.7:0/3255218993 shutdown_connections 2024-01-27T02:33:56.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.553+0000 7efe47cd9700 1 --2- 172.21.15.7:0/3255218993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe2c003ab0 0x7efe2c003ec0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:56.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.553+0000 7efe47cd9700 1 -- 172.21.15.7:0/3255218993 >> 172.21.15.7:0/3255218993 conn(0x7efe400cdc30 msgr2=0x7efe400ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:56.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.553+0000 7efe47cd9700 1 -- 172.21.15.7:0/3255218993 shutdown_connections 2024-01-27T02:33:56.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.553+0000 7efe47cd9700 1 -- 172.21.15.7:0/3255218993 wait complete. 2024-01-27T02:33:56.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.554+0000 7efe47cd9700 1 Processor -- start 2024-01-27T02:33:56.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.554+0000 7efe47cd9700 1 -- start start 2024-01-27T02:33:56.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.555+0000 7efe47cd9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe2c003ab0 0x7efe40168be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:56.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.555+0000 7efe47cd9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe401566c0 0x7efe4014e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:56.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.555+0000 7efe47cd9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe4014ecc0 0x7efe4014f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.555+0000 7efe47cd9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe400c5c40 con 0x7efe4014ecc0 2024-01-27T02:33:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.555+0000 7efe47cd9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe400c4ce0 con 0x7efe2c003ab0 2024-01-27T02:33:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.555+0000 7efe47cd9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe400c5610 con 0x7efe401566c0 2024-01-27T02:33:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.555+0000 7efe46276700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe4014ecc0 0x7efe4014f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.555+0000 7efe46276700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe4014ecc0 0x7efe4014f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34784/0 (socket says 172.21.15.7:34784) 2024-01-27T02:33:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.555+0000 7efe46276700 1 -- 172.21.15.7:0/3767417848 learned_addr learned my addr 172.21.15.7:0/3767417848 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.555+0000 7efe45a75700 1 --2- 172.21.15.7:0/3767417848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe2c003ab0 0x7efe40168be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.555+0000 7efe45274700 1 --2- 172.21.15.7:0/3767417848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe401566c0 0x7efe4014e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.555+0000 7efe46276700 1 -- 172.21.15.7:0/3767417848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe401566c0 msgr2=0x7efe4014e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.555+0000 7efe46276700 1 --2- 172.21.15.7:0/3767417848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe401566c0 0x7efe4014e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.555+0000 7efe46276700 1 -- 172.21.15.7:0/3767417848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe2c003ab0 msgr2=0x7efe40168be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.555+0000 7efe46276700 1 --2- 172.21.15.7:0/3767417848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe2c003ab0 0x7efe40168be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.555+0000 7efe46276700 1 -- 172.21.15.7:0/3767417848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe400c4ef0 con 0x7efe4014ecc0 2024-01-27T02:33:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.555+0000 7efe45274700 1 --2- 172.21.15.7:0/3767417848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe401566c0 0x7efe4014e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.555+0000 7efe45a75700 1 --2- 172.21.15.7:0/3767417848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe2c003ab0 0x7efe40168be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:56.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.556+0000 7efe46276700 1 --2- 172.21.15.7:0/3767417848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe4014ecc0 0x7efe4014f520 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7efe380094d0 tx=0x7efe3800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:56.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.556+0000 7efe31ffb700 1 -- 172.21.15.7:0/3767417848 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3800e040 con 0x7efe4014ecc0 2024-01-27T02:33:56.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.556+0000 7efe31ffb700 1 -- 172.21.15.7:0/3767417848 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3800f040 con 0x7efe4014ecc0 2024-01-27T02:33:56.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.556+0000 7efe47cd9700 1 -- 172.21.15.7:0/3767417848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe400c5860 con 0x7efe4014ecc0 2024-01-27T02:33:56.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.556+0000 7efe31ffb700 1 -- 172.21.15.7:0/3767417848 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3800fe20 con 0x7efe4014ecc0 2024-01-27T02:33:56.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.556+0000 7efe47cd9700 1 -- 172.21.15.7:0/3767417848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe4014e840 con 0x7efe4014ecc0 2024-01-27T02:33:56.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.557+0000 7efe47cd9700 1 -- 172.21.15.7:0/3767417848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe4004c710 con 0x7efe4014ecc0 2024-01-27T02:33:56.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.558+0000 7efe31ffb700 1 -- 172.21.15.7:0/3767417848 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7efe38015640 con 0x7efe4014ecc0 2024-01-27T02:33:56.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.558+0000 7efe31ffb700 1 --2- 172.21.15.7:0/3767417848 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7efe24066f30 0x7efe240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:56.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.558+0000 7efe31ffb700 1 -- 172.21.15.7:0/3767417848 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5763+0+0 (secure 0 0 0) 0x7efe38084eb0 con 0x7efe4014ecc0 2024-01-27T02:33:56.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.558+0000 7efe45a75700 1 --2- 172.21.15.7:0/3767417848 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7efe24066f30 0x7efe240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:56.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.559+0000 7efe45a75700 1 --2- 172.21.15.7:0/3767417848 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7efe24066f30 0x7efe240693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0016c0 tx=0x7efe3c015000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:56.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.563+0000 7efe31ffb700 1 -- 172.21.15.7:0/3767417848 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe380520e0 con 0x7efe4014ecc0 2024-01-27T02:33:56.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.767+0000 7efe47cd9700 1 -- 172.21.15.7:0/3767417848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7efe400cc250 con 0x7efe4014ecc0 2024-01-27T02:33:56.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.768+0000 7efe31ffb700 1 -- 172.21.15.7:0/3767417848 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v52) v1 ==== 81+0+32587 (secure 0 0 0) 0x7efe38055700 con 0x7efe4014ecc0 2024-01-27T02:33:56.769 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:56.769 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:33:56.769 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 52, 2024-01-27T02:33:56.769 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:33:56.769 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:33:56.769 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:33:55.960856+0000", 2024-01-27T02:33:56.769 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:33:56.769 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:33:56.769 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:33:56.770 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:33:56.770 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:33:56.770 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:33:56.770 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:33:56.770 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:33:56.770 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:33:56.770 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:56.770 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:33:56.770 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:33:56.770 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:33:56.770 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:33:56.770 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:33:56.770 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 7, 2024-01-27T02:33:56.770 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:33:56.771 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:33:56.771 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-01-27T02:33:56.771 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:33:56.771 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:33:56.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.771 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:33:56.771 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:33:56.771 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:33:56.771 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:33:56.771 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:33:56.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:33:56.771 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:33:56.771 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:33:56.771 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:33:56.772 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:33:56.772 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:33:56.772 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:33:56.772 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:33:56.772 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:33:56.772 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:33:56.772 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:33:56.772 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:33:56.772 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:33:56.772 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:33:56.772 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:33:56.772 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:33:56.772 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:33:56.772 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:33:56.773 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:33:56.773 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:33:56.773 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:33:56.773 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:33:56.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.773 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "51", 2024-01-27T02:33:56.773 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:33:56.773 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:33:56.773 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:33:56.773 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:33:56.773 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:33:56.773 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:33:56.773 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:33:56.773 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:33:56.773 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:33:56.774 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:33:56.774 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:33:56.774 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:33:56.774 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:33:56.774 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:33:56.774 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:33:56.774 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:33:56.774 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:33:56.774 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:33:56.774 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:33:56.774 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:33:56.774 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:33:56.774 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:33:56.774 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:33:56.774 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:33:56.775 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:33:56.775 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:33:56.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.775 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:33:56.775 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:33:56.775 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:33:56.775 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:33:56.775 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:33:56.775 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:33:56.775 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:33:56.775 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:33:56.776 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:33:56.776 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:33:56.776 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:33:56.776 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:33:56.776 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:33:56.776 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:33:56.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.776 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:33:56.776 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:33:56.776 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.776 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 6, 2024-01-27T02:33:56.776 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:33:56.777 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:33:23.445643+0000", 2024-01-27T02:33:56.777 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:33:56.777 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:33:56.777 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:33:56.777 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:33:56.777 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:33:56.777 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:33:56.777 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:33:56.777 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:33:56.777 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:33:56.777 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:33:56.777 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:33:56.777 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:33:56.777 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:33:56.777 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:33:56.778 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:33:56.778 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:33:56.778 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:33:56.778 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:33:56.778 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:33:56.778 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:33:56.778 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:33:56.778 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:33:56.778 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:33:56.778 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:33:56.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.778 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "51", 2024-01-27T02:33:56.778 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:33:56.778 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:33:56.778 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:33:56.779 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:33:56.779 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:33:56.779 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:33:56.779 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:33:56.779 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:33:56.779 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:33:56.779 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:33:56.779 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:33:56.779 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:33:56.779 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:33:56.779 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:33:56.779 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:33:56.779 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:33:56.779 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:33:56.780 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:33:56.780 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:33:56.780 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:33:56.780 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:33:56.780 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:33:56.780 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:33:56.780 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:33:56.780 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:33:56.780 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:33:56.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.780 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:33:56.780 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:33:56.780 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:33:56.781 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:33:56.781 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:33:56.781 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:33:56.781 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:33:56.781 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:33:56.781 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:33:56.781 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:33:56.781 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:33:56.781 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:33:56.781 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:33:56.781 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:33:56.781 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:33:56.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.781 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:33:56.781 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:33:56.781 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:33:56.782 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.782 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.782 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 7, 2024-01-27T02:33:56.782 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:33:56.782 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:33:23.921995+0000", 2024-01-27T02:33:56.782 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:33:56.782 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:33:56.782 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:33:56.782 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:33:56.782 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:33:56.782 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:33:56.782 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:33:56.783 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:33:56.783 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:33:56.783 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:33:56.783 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:33:56.783 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:33:56.783 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:33:56.783 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:33:56.783 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:33:56.783 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:33:56.783 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:33:56.783 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:33:56.783 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:33:56.783 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:33:56.783 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:33:56.783 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:33:56.784 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:33:56.784 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:33:56.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.784 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "51", 2024-01-27T02:33:56.784 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:33:56.784 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:33:56.784 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:33:56.784 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:33:56.784 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:33:56.784 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:33:56.784 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:33:56.784 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:33:56.784 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:33:56.784 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:33:56.785 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:33:56.785 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:33:56.785 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:33:56.785 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:33:56.785 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:33:56.785 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:33:56.785 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:33:56.785 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:33:56.785 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:33:56.785 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:33:56.785 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:33:56.785 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:33:56.785 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:33:56.786 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:33:56.786 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:33:56.786 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:33:56.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.786 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:33:56.786 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:33:56.786 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:33:56.786 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:33:56.786 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:33:56.786 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:33:56.786 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:33:56.786 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:33:56.786 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:33:56.786 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:33:56.786 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:33:56.787 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:33:56.787 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:33:56.787 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:33:56.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.787 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:33:56.787 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:33:56.787 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:33:56.787 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.787 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.787 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.787 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:56.787 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:33:56.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.787 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:33:56.787 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:56.787 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:56.788 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:56.788 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:56.788 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:56.788 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:56.788 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:56.788 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:56.788 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:56.788 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:56.788 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:56.788 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:56.788 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.788 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.788 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:33:56.788 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:56.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.789 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.789 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:33:56.789 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:56.789 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.789 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.789 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:56.789 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.789 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.789 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:33:56.789 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:56.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.790 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.790 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:33:56.790 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:56.790 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.790 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.790 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:56.790 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.790 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.790 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:33:56.790 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:56.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.791 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:33:56.791 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:56.791 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.791 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.791 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:56.791 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.791 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:33:56.791 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:56.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.791 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:33:56.792 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:56.792 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.792 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.792 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:33:56.792 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:33:56.792 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:33:56.792 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:33:56.792 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:56.792 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:56.792 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:56.792 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.793 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:33:56.793 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:56.793 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:56.793 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:56.793 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:56.793 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:56.793 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:56.793 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:56.793 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:56.793 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:56.793 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:56.793 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:56.793 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:56.793 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.794 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:33:56.794 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:56.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.794 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:33:56.794 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:56.794 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.794 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.794 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:56.794 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.795 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:33:56.795 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:56.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.795 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.795 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:33:56.795 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:56.795 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.795 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.795 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:56.795 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.795 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.795 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:33:56.796 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:56.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.796 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.796 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:33:56.796 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:56.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.796 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.796 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:56.796 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.796 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.796 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:33:56.797 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:56.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.797 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:33:56.797 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:56.797 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.797 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.797 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:33:56.797 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:33:56.797 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:33:56.797 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:33:56.797 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:56.798 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:56.798 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:56.798 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.798 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:33:56.798 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:56.798 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:56.798 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:56.798 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:56.798 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:56.798 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:56.798 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:56.798 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:56.798 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:56.799 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:56.799 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:56.799 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:56.799 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.799 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.799 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:33:56.799 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:56.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.799 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.799 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:33:56.799 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:56.799 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.800 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.800 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:56.800 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.800 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:33:56.800 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:56.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.800 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:33:56.800 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:56.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.800 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.801 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:56.801 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.801 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:33:56.801 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:56.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.801 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:33:56.801 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:56.801 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.801 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.802 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:56.802 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.802 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:33:56.802 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:56.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.802 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:33:56.802 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:56.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.802 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.802 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:33:56.803 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:33:56.803 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:33:56.803 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:33:56.803 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:56.803 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:56.803 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:56.803 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.803 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:33:56.803 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:56.803 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:56.803 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:56.804 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:56.804 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:56.804 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:56.804 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:56.804 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:56.804 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:56.804 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:56.804 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:56.804 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:56.804 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.804 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:33:56.804 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:56.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.805 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:33:56.805 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:56.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.805 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.805 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:56.805 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.805 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:33:56.805 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:56.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.806 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:33:56.806 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:56.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.806 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.806 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:56.806 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.806 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:33:56.806 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:56.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.807 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.807 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:33:56.807 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:56.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.807 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.807 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:56.807 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.807 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.807 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:33:56.807 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:56.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.807 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.808 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:33:56.808 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:56.808 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.808 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.808 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:33:56.808 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:33:56.808 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:33:56.808 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:33:56.808 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:56.808 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:56.808 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:56.808 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.809 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:33:56.809 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:56.809 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:56.809 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:56.809 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:56.809 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:56.809 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:56.809 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:56.809 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:56.809 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:56.809 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:56.809 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:56.809 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:56.809 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.810 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.810 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:33:56.810 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:56.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.810 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.810 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:33:56.810 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:56.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.810 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.810 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:56.810 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.811 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:33:56.811 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:56.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.811 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:33:56.811 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:56.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.811 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.811 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:56.811 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.812 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:33:56.812 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:56.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.812 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:33:56.812 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:56.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.812 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.812 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:56.812 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.813 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:33:56.813 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:56.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.813 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.813 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:33:56.813 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:56.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.813 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.813 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:33:56.813 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:33:56.813 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:33:56.813 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:33:56.814 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:56.814 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:56.814 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:56.814 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.814 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:33:56.814 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:56.814 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:56.814 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:56.814 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:56.814 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:56.814 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:56.814 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:56.815 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:56.815 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:56.815 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:56.815 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:56.815 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:56.815 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.815 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:33:56.815 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:56.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.815 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:33:56.816 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:56.816 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.816 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.816 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:56.816 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.816 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:33:56.816 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:56.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.816 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:33:56.816 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:56.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.817 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.817 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:56.817 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.817 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:33:56.817 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:56.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.817 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:33:56.817 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:56.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.818 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.818 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:56.818 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.818 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.818 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:33:56.818 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:56.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.818 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.818 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:33:56.818 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:56.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.819 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.819 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:33:56.819 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:33:56.819 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:33:56.819 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:33:56.819 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:56.819 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:56.819 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:56.819 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.819 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:33:56.819 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:56.820 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:56.820 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:56.820 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:56.820 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:56.820 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:56.820 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:56.820 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:56.820 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:56.820 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:56.820 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:56.820 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:56.820 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.821 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:33:56.821 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:56.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.821 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.821 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:33:56.821 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:56.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.821 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.821 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:56.821 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.821 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.822 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:33:56.822 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:56.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.822 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.822 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:33:56.822 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:56.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.822 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.822 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:56.822 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.822 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.822 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:33:56.823 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:56.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.823 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:33:56.823 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:56.823 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.823 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.823 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:56.823 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.823 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:33:56.824 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:56.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.824 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.824 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:33:56.824 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:56.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.824 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.824 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:33:56.824 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:33:56.824 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:33:56.824 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:33:56.824 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:56.825 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:56.825 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:56.825 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.825 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:33:56.825 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:56.825 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:56.825 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:56.825 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:56.825 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:56.825 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:56.825 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:56.825 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:56.825 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:56.826 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:56.826 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:56.826 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:56.826 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.826 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.826 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:33:56.826 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:56.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.826 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.826 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:33:56.826 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:56.826 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.827 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.827 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:56.827 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.827 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:33:56.827 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:56.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.827 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:33:56.827 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:56.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.827 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.828 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:56.828 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.828 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:33:56.828 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:56.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.828 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:33:56.828 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:56.828 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.828 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.829 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:56.829 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:56.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.829 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:56.829 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:33:56.829 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:56.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.829 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:56.829 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:33:56.829 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:56.829 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.829 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.830 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:33:56.830 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:33:56.830 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:33:56.830 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:33:56.830 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:56.830 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:56.830 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:56.830 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.830 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:56.830 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:33:56.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.830 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:33:56.830 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:56.830 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:56.831 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:56.831 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:56.831 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:56.831 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:33:56.831 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:56.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.831 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:33:56.831 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:56.831 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:56.831 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:56.831 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:56.831 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:56.831 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:33:56.832 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:56.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.832 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:33:56.832 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:56.832 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:56.832 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:56.832 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:56.832 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:56.832 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:33:56.832 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:56.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.833 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:33:56.833 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:56.833 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:56.833 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:56.833 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:56.833 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:56.833 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:33:56.833 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:56.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.833 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:33:56.833 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:56.833 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:56.833 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:56.833 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:56.834 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:56.834 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:33:56.834 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:56.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.834 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:33:56.834 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:56.834 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:56.834 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:56.834 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:56.834 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:56.834 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:33:56.834 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:56.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.835 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:33:56.835 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:56.835 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:56.835 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:56.835 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:56.835 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:56.835 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:33:56.835 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:56.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.835 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:33:56.835 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:56.835 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:56.836 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:56.836 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:56.836 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:56.836 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:33:56.836 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:56.836 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.836 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:56.836 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:33:56.836 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [ 2024-01-27T02:33:56.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.836 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "7.30", 2024-01-27T02:33:56.836 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-01-27T02:33:56.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.836 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 0, 2024-01-27T02:33:56.836 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 4 2024-01-27T02:33:56.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.837 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.837 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "7.35", 2024-01-27T02:33:56.837 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-01-27T02:33:56.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:56.837 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 0, 2024-01-27T02:33:56.837 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 5 2024-01-27T02:33:56.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.837 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:56.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.837 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:56.837 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:33:56.837 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:33:56.838 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:33:56.838 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:33:56.838 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:33:56.838 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:33:56.838 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:33:56.838 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:33:56.838 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:33:56.838 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:33:56.838 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:33:56.838 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:56.838 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:33:56.838 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:33:56.838 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:33:56.838 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:33:56.838 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:33:56.839 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:33:56.839 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:33:56.839 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:33:56.839 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:33:56.839 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:33:56.839 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:33:56.839 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:56.839 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:33:56.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.770+0000 7efe47cd9700 1 -- 172.21.15.7:0/3767417848 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7efe24066f30 msgr2=0x7efe240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:56.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.770+0000 7efe47cd9700 1 --2- 172.21.15.7:0/3767417848 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7efe24066f30 0x7efe240693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0016c0 tx=0x7efe3c015000 comp rx=0 tx=0).stop 2024-01-27T02:33:56.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.770+0000 7efe47cd9700 1 -- 172.21.15.7:0/3767417848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe4014ecc0 msgr2=0x7efe4014f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:56.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.770+0000 7efe47cd9700 1 --2- 172.21.15.7:0/3767417848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe4014ecc0 0x7efe4014f520 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7efe380094d0 tx=0x7efe3800b040 comp rx=0 tx=0).stop 2024-01-27T02:33:56.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.770+0000 7efe47cd9700 1 -- 172.21.15.7:0/3767417848 shutdown_connections 2024-01-27T02:33:56.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.770+0000 7efe47cd9700 1 --2- 172.21.15.7:0/3767417848 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7efe24066f30 0x7efe240693e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:56.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.770+0000 7efe47cd9700 1 --2- 172.21.15.7:0/3767417848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe4014ecc0 0x7efe4014f520 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:56.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.770+0000 7efe47cd9700 1 --2- 172.21.15.7:0/3767417848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe2c003ab0 0x7efe40168be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:56.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.770+0000 7efe47cd9700 1 --2- 172.21.15.7:0/3767417848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe401566c0 0x7efe4014e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:56.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.770+0000 7efe47cd9700 1 -- 172.21.15.7:0/3767417848 >> 172.21.15.7:0/3767417848 conn(0x7efe400cdc30 msgr2=0x7efe400ccda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:56.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.770+0000 7efe47cd9700 1 -- 172.21.15.7:0/3767417848 shutdown_connections 2024-01-27T02:33:56.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.771+0000 7efe47cd9700 1 -- 172.21.15.7:0/3767417848 wait complete. 2024-01-27T02:33:56.842 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-01-27T02:33:56.842 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:33:56.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.980+0000 7f01b9509700 1 Processor -- start 2024-01-27T02:33:56.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.980+0000 7f01b9509700 1 -- start start 2024-01-27T02:33:56.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.980+0000 7f01b9509700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f01b4157ca0 0x7f01b40c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:56.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.981+0000 7f01b9509700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01b40cdbf0 0x7f01b40ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:56.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.981+0000 7f01b9509700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b40ca020 con 0x7f01b40c3ae0 2024-01-27T02:33:56.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.981+0000 7f01b9509700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b40c4f40 con 0x7f01b4157ca0 2024-01-27T02:33:56.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.981+0000 7f01b9509700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40c5210 con 0x7f01b40cdbf0 2024-01-27T02:33:56.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.981+0000 7f01b2ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f01b4157ca0 0x7f01b40c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:56.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.981+0000 7f01b2ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f01b4157ca0 0x7f01b40c4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34792/0 (socket says 172.21.15.7:34792) 2024-01-27T02:33:56.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.981+0000 7f01b2ffd700 1 -- 172.21.15.7:0/1551861581 learned_addr learned my addr 172.21.15.7:0/1551861581 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:56.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.981+0000 7f01b37fe700 1 --2- 172.21.15.7:0/1551861581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01b40cdbf0 0x7f01b40ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:56.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.981+0000 7f01b2ffd700 1 -- 172.21.15.7:0/1551861581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01b40cdbf0 msgr2=0x7f01b40ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:56.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.981+0000 7f01b2ffd700 1 --2- 172.21.15.7:0/1551861581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01b40cdbf0 0x7f01b40ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:56.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.981+0000 7f01b2ffd700 1 -- 172.21.15.7:0/1551861581 >> v1:172.21.15.188:6789/0 conn(0x7f01b40c3ae0 legacy=0x7f01b40cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:56.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.981+0000 7f01b2ffd700 1 -- 172.21.15.7:0/1551861581 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b40c54f0 con 0x7f01b4157ca0 2024-01-27T02:33:56.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.981+0000 7f01b37fe700 1 --2- 172.21.15.7:0/1551861581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01b40cdbf0 0x7f01b40ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:33:56.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.981+0000 7f01b2ffd700 1 --2- 172.21.15.7:0/1551861581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01b4157ca0 0x7f01b40c4a00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f01a40099b0 tx=0x7f01a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f6f7e1f03eb1ac server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:56.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.981+0000 7f01b1ffb700 1 -- 172.21.15.7:0/1551861581 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a400b860 con 0x7f01b4157ca0 2024-01-27T02:33:56.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.982+0000 7f01b1ffb700 1 -- 172.21.15.7:0/1551861581 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a400b9c0 con 0x7f01b4157ca0 2024-01-27T02:33:56.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.982+0000 7f01b1ffb700 1 -- 172.21.15.7:0/1551861581 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a400bd60 con 0x7f01b4157ca0 2024-01-27T02:33:56.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.982+0000 7f01b9509700 1 -- 172.21.15.7:0/1551861581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01b4157ca0 msgr2=0x7f01b40c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:56.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.982+0000 7f01b9509700 1 --2- 172.21.15.7:0/1551861581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01b4157ca0 0x7f01b40c4a00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f01a40099b0 tx=0x7f01a4009250 comp rx=0 tx=0).stop 2024-01-27T02:33:56.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.982+0000 7f01b9509700 1 -- 172.21.15.7:0/1551861581 shutdown_connections 2024-01-27T02:33:56.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.982+0000 7f01b9509700 1 --2- 172.21.15.7:0/1551861581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01b40cdbf0 0x7f01b40ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:56.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.982+0000 7f01b9509700 1 --2- 172.21.15.7:0/1551861581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01b4157ca0 0x7f01b40c4a00 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:56.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.982+0000 7f01b9509700 1 -- 172.21.15.7:0/1551861581 >> 172.21.15.7:0/1551861581 conn(0x7f01b4098020 msgr2=0x7f01b40711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:56.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.982+0000 7f01b9509700 1 -- 172.21.15.7:0/1551861581 shutdown_connections 2024-01-27T02:33:56.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.982+0000 7f01b9509700 1 -- 172.21.15.7:0/1551861581 wait complete. 2024-01-27T02:33:56.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.984+0000 7f01b9509700 1 Processor -- start 2024-01-27T02:33:56.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.984+0000 7f01b9509700 1 -- start start 2024-01-27T02:33:56.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.984+0000 7f01b9509700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01b40c3ae0 0x7f01b4068310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:56.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.984+0000 7f01b9509700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01b40cdbf0 0x7f01b416d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:56.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.984+0000 7f01b9509700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01b4157ca0 0x7f01b40ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:56.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.984+0000 7f01b9509700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b4068850 con 0x7f01b40cdbf0 2024-01-27T02:33:56.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.985+0000 7f01b9509700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b4068af0 con 0x7f01b4157ca0 2024-01-27T02:33:56.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.985+0000 7f01b9509700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b4068dd0 con 0x7f01b40c3ae0 2024-01-27T02:33:56.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.985+0000 7f01b27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01b40cdbf0 0x7f01b416d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:56.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.985+0000 7f01b27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01b40cdbf0 0x7f01b416d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34796/0 (socket says 172.21.15.7:34796) 2024-01-27T02:33:56.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.985+0000 7f01b27fc700 1 -- 172.21.15.7:0/3440140201 learned_addr learned my addr 172.21.15.7:0/3440140201 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:56.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.985+0000 7f01b37fe700 1 --2- 172.21.15.7:0/3440140201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01b4157ca0 0x7f01b40ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:56.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.985+0000 7f01b2ffd700 1 --2- 172.21.15.7:0/3440140201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01b40c3ae0 0x7f01b4068310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:56.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.985+0000 7f01b27fc700 1 -- 172.21.15.7:0/3440140201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01b40c3ae0 msgr2=0x7f01b4068310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:56.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.985+0000 7f01b27fc700 1 --2- 172.21.15.7:0/3440140201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01b40c3ae0 0x7f01b4068310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:56.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.985+0000 7f01b27fc700 1 -- 172.21.15.7:0/3440140201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01b4157ca0 msgr2=0x7f01b40ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:56.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.985+0000 7f01b27fc700 1 --2- 172.21.15.7:0/3440140201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01b4157ca0 0x7f01b40ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:56.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.985+0000 7f01b27fc700 1 -- 172.21.15.7:0/3440140201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a40094f0 con 0x7f01b40cdbf0 2024-01-27T02:33:56.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.985+0000 7f01b37fe700 1 --2- 172.21.15.7:0/3440140201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01b4157ca0 0x7f01b40ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:33:56.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.985+0000 7f01b27fc700 1 --2- 172.21.15.7:0/3440140201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01b40cdbf0 0x7f01b416d190 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f01a800c590 tx=0x7f01a800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:56.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.985+0000 7f019affd700 1 -- 172.21.15.7:0/3440140201 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a8011590 con 0x7f01b40cdbf0 2024-01-27T02:33:56.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.985+0000 7f019affd700 1 -- 172.21.15.7:0/3440140201 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a80116f0 con 0x7f01b40cdbf0 2024-01-27T02:33:56.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.985+0000 7f019affd700 1 -- 172.21.15.7:0/3440140201 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a8011ab0 con 0x7f01b40cdbf0 2024-01-27T02:33:56.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.985+0000 7f01b9509700 1 -- 172.21.15.7:0/3440140201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b40690b0 con 0x7f01b40cdbf0 2024-01-27T02:33:56.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.985+0000 7f01b9509700 1 -- 172.21.15.7:0/3440140201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b416d6d0 con 0x7f01b40cdbf0 2024-01-27T02:33:56.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.986+0000 7f01b9509700 1 -- 172.21.15.7:0/3440140201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b40698b0 con 0x7f01b40cdbf0 2024-01-27T02:33:56.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.987+0000 7f019affd700 1 -- 172.21.15.7:0/3440140201 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f01a801b3b0 con 0x7f01b40cdbf0 2024-01-27T02:33:56.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.987+0000 7f019affd700 1 --2- 172.21.15.7:0/3440140201 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f01a0066f30 0x7f01a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:56.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.987+0000 7f019affd700 1 -- 172.21.15.7:0/3440140201 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f01a8083bc0 con 0x7f01b40cdbf0 2024-01-27T02:33:56.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.988+0000 7f01b2ffd700 1 --2- 172.21.15.7:0/3440140201 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f01a0066f30 0x7f01a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:56.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.988+0000 7f01b2ffd700 1 --2- 172.21.15.7:0/3440140201 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f01a0066f30 0x7f01a00693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f01a40099b0 tx=0x7f01a4017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:56.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:56.992+0000 7f019affd700 1 -- 172.21.15.7:0/3440140201 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01b40698b0 con 0x7f01b40cdbf0 2024-01-27T02:33:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.195+0000 7f01b9509700 1 -- 172.21.15.7:0/3440140201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f01b40698b0 con 0x7f01b40cdbf0 2024-01-27T02:33:57.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.196+0000 7f019affd700 1 -- 172.21.15.7:0/3440140201 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v53) v1 ==== 81+0+32587 (secure 0 0 0) 0x7f01b40698b0 con 0x7f01b40cdbf0 2024-01-27T02:33:57.197 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:57.197 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:33:57.197 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 53, 2024-01-27T02:33:57.197 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:33:57.198 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:33:57.198 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:33:56.965932+0000", 2024-01-27T02:33:57.198 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:33:57.198 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:33:57.198 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:33:57.198 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:33:57.198 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:33:57.198 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:33:57.198 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:33:57.198 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:33:57.198 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:33:57.198 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:57.198 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:33:57.198 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:33:57.198 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:33:57.199 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:33:57.199 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:33:57.199 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 7, 2024-01-27T02:33:57.199 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:33:57.199 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:33:57.199 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-01-27T02:33:57.199 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:33:57.199 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:33:57.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.199 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:33:57.199 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:33:57.199 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:33:57.199 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:33:57.199 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:33:57.200 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:33:57.200 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:33:57.200 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:33:57.200 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:33:57.200 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:33:57.200 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:33:57.200 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:33:57.200 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:33:57.200 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:33:57.200 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:33:57.200 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:33:57.200 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:33:57.200 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:33:57.200 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:33:57.201 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:33:57.201 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:33:57.201 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:33:57.201 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:33:57.201 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:33:57.201 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:33:57.201 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:33:57.201 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:33:57.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.201 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "51", 2024-01-27T02:33:57.201 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:33:57.201 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:33:57.201 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:33:57.201 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:33:57.202 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:33:57.202 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:33:57.202 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:33:57.202 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:33:57.202 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:33:57.202 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:33:57.202 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:33:57.202 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:33:57.202 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:33:57.202 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:33:57.202 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:33:57.202 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:33:57.202 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:33:57.202 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:33:57.202 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:33:57.202 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:33:57.203 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:33:57.203 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:33:57.203 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:33:57.203 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:33:57.203 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:33:57.203 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:33:57.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.203 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:33:57.203 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:33:57.203 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:33:57.203 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:33:57.203 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:33:57.203 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:33:57.203 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:33:57.204 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:33:57.204 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:33:57.204 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:33:57.204 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:33:57.204 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:33:57.204 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:33:57.204 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:33:57.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.204 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:33:57.204 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:33:57.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.204 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 6, 2024-01-27T02:33:57.204 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:33:57.205 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:33:23.445643+0000", 2024-01-27T02:33:57.205 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:33:57.205 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:33:57.205 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:33:57.205 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:33:57.205 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:33:57.205 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:33:57.205 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:33:57.205 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:33:57.205 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:33:57.205 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:33:57.205 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:33:57.205 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:33:57.205 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:33:57.206 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:33:57.206 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:33:57.206 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:33:57.206 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:33:57.206 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:33:57.206 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:33:57.206 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:33:57.206 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:33:57.206 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:33:57.206 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:33:57.206 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:33:57.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.206 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "51", 2024-01-27T02:33:57.206 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:33:57.206 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:33:57.207 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:33:57.207 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:33:57.207 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:33:57.207 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:33:57.207 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:33:57.207 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:33:57.207 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:33:57.207 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:33:57.207 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:33:57.207 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:33:57.207 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:33:57.207 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:33:57.207 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:33:57.207 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:33:57.207 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:33:57.208 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:33:57.208 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:33:57.208 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:33:57.208 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:33:57.208 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:33:57.208 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:33:57.208 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:33:57.208 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:33:57.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:33:57.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.208 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:33:57.208 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:33:57.208 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:33:57.208 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:33:57.209 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:33:57.209 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:33:57.209 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:33:57.209 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:33:57.209 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:33:57.209 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:33:57.209 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:33:57.209 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:33:57.209 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:33:57.209 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:33:57.209 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:33:57.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.209 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:33:57.209 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:33:57.209 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:33:57.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.210 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 7, 2024-01-27T02:33:57.210 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:33:57.210 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:33:23.921995+0000", 2024-01-27T02:33:57.210 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:33:57.210 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:33:57.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:33:57.210 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:33:57.210 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:33:57.210 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:33:57.210 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:33:57.210 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:33:57.211 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:33:57.211 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:33:57.211 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:33:57.211 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:33:57.211 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:33:57.211 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:33:57.211 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:33:57.211 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:33:57.211 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:33:57.211 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:33:57.211 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:33:57.211 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:33:57.211 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:33:57.211 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:33:57.212 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:33:57.212 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:33:57.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.212 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "51", 2024-01-27T02:33:57.212 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:33:57.212 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:33:57.212 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:33:57.212 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:33:57.212 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:33:57.212 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:33:57.212 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:33:57.212 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:33:57.212 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:33:57.213 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:33:57.213 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:33:57.213 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:33:57.213 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:33:57.213 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:33:57.213 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:33:57.213 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:33:57.213 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:33:57.213 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:33:57.213 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:33:57.213 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:33:57.213 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:33:57.213 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:33:57.213 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:33:57.213 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:33:57.214 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:33:57.214 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:33:57.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.214 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:33:57.214 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:33:57.214 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:33:57.214 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:33:57.214 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:33:57.214 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:33:57.214 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:33:57.214 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:33:57.214 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:33:57.214 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:33:57.214 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:33:57.214 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:33:57.215 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:33:57.215 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:33:57.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.215 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:33:57.215 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:33:57.215 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:33:57.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.215 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:57.215 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:33:57.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.215 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:33:57.216 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:57.216 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:57.216 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:57.216 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:57.216 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:57.216 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:57.216 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:57.216 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:57.216 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:57.216 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:57.216 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:57.216 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:57.216 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.217 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:33:57.217 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:57.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.217 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:33:57.217 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:57.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.217 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.217 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:57.217 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:33:57.218 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:57.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:33:57.218 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:57.218 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.218 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.218 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:57.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:33:57.219 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:57.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.219 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:33:57.219 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:57.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.219 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.219 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:57.219 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.219 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:33:57.219 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:57.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.220 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:33:57.220 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:33:57.220 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.220 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.220 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:33:57.220 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:33:57.220 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:33:57.220 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:33:57.220 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:57.220 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:57.221 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:57.221 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.221 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:33:57.221 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:57.221 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:57.221 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:57.221 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:57.221 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:57.221 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:57.221 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:57.221 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:57.221 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:57.221 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:57.222 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:57.222 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:57.222 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.222 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.222 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:33:57.222 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:57.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.222 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.222 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:33:57.222 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:57.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.222 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.223 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:57.223 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.223 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:33:57.223 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:57.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.223 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:33:57.223 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:57.223 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.223 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.224 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:57.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:33:57.224 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:57.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:33:57.224 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:57.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.224 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.225 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:57.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:33:57.225 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:57.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:33:57.225 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:33:57.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.225 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.225 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:33:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:33:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:33:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:33:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:33:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:33:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:33:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:33:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:33:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:33:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:33:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:33:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:33:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:33:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:33:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:33:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:33:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:33:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:33:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:33:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:33:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:33:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:33:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:33:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:33:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:33:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:33:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:33:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:33:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:33:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:33:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:33:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:33:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:33:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:33:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:33:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:33:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:33:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:33:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:33:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:33:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:33:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:33:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:33:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:33:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:33:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:33:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 52, 2024-01-27T02:33:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:33:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:33:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:33:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:33:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:33:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:33:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:33:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:33:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:33:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:33:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:33:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:33:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:33:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:33:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 52, 2024-01-27T02:33:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:33:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:33:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:33:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:33:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:33:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:33:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:33:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:33:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:33:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:33:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:33:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:33:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:33:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:33:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:33:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:33:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:33:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:33:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:33:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:33:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:33:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:33:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 43, 2024-01-27T02:33:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:33:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:33:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:33:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:33:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:33:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:33:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:33:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:33:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:33:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:33:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:33:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:33:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:33:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:33:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:33:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:33:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:33:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:33:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:33:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:33:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:33:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:33:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:33:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:33:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:33:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:33:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:33:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:33:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:33:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:33:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:33:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:33:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:33:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:33:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:33:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:33:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:33:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:33:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:33:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:33:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:33:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:33:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:33:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:33:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:33:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:33:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:33:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:33:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:33:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [ 2024-01-27T02:33:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "7.30", 2024-01-27T02:33:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-01-27T02:33:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 0, 2024-01-27T02:33:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 4 2024-01-27T02:33:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "7.35", 2024-01-27T02:33:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-01-27T02:33:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:33:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 0, 2024-01-27T02:33:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 5 2024-01-27T02:33:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:33:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:33:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:33:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:33:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:33:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:33:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:33:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:33:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:33:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:33:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:33:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:33:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:33:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:33:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:33:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:33:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:33:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:33:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:33:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:33:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:33:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:33:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:33:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:33:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:33:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:33:57.267 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:33:57.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.198+0000 7f01b9509700 1 -- 172.21.15.7:0/3440140201 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f01a0066f30 msgr2=0x7f01a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:57.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.198+0000 7f01b9509700 1 --2- 172.21.15.7:0/3440140201 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f01a0066f30 0x7f01a00693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f01a40099b0 tx=0x7f01a4017000 comp rx=0 tx=0).stop 2024-01-27T02:33:57.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.198+0000 7f01b9509700 1 -- 172.21.15.7:0/3440140201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01b40cdbf0 msgr2=0x7f01b416d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:57.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.198+0000 7f01b9509700 1 --2- 172.21.15.7:0/3440140201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01b40cdbf0 0x7f01b416d190 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f01a800c590 tx=0x7f01a800c950 comp rx=0 tx=0).stop 2024-01-27T02:33:57.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.198+0000 7f01b9509700 1 -- 172.21.15.7:0/3440140201 shutdown_connections 2024-01-27T02:33:57.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.198+0000 7f01b9509700 1 --2- 172.21.15.7:0/3440140201 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f01a0066f30 0x7f01a00693e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:57.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.198+0000 7f01b9509700 1 --2- 172.21.15.7:0/3440140201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01b4157ca0 0x7f01b40ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:57.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.198+0000 7f01b9509700 1 --2- 172.21.15.7:0/3440140201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01b40c3ae0 0x7f01b4068310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.198+0000 7f01b9509700 1 --2- 172.21.15.7:0/3440140201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01b40cdbf0 0x7f01b416d190 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.198+0000 7f01b9509700 1 -- 172.21.15.7:0/3440140201 >> 172.21.15.7:0/3440140201 conn(0x7f01b4098020 msgr2=0x7f01b40cf5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.198+0000 7f01b9509700 1 -- 172.21.15.7:0/3440140201 shutdown_connections 2024-01-27T02:33:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.199+0000 7f01b9509700 1 -- 172.21.15.7:0/3440140201 wait complete. 2024-01-27T02:33:57.270 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:33:57.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.405+0000 7fae8a74c700 1 Processor -- start 2024-01-27T02:33:57.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.405+0000 7fae8a74c700 1 -- start start 2024-01-27T02:33:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.406+0000 7fae8a74c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fae840cbf10 0x7fae8406e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.406+0000 7fae8a74c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae840b09f0 0x7fae8406e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.406+0000 7fae8a74c700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae840cce30 con 0x7fae841404e0 2024-01-27T02:33:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.406+0000 7fae8a74c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae8406b3b0 con 0x7fae840cbf10 2024-01-27T02:33:57.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.406+0000 7fae8a74c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae840cc3e0 con 0x7fae840b09f0 2024-01-27T02:33:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.406+0000 7fae837fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fae840cbf10 0x7fae8406e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.406+0000 7fae837fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fae840cbf10 0x7fae8406e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34806/0 (socket says 172.21.15.7:34806) 2024-01-27T02:33:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.406+0000 7fae837fe700 1 -- 172.21.15.7:0/1782045893 learned_addr learned my addr 172.21.15.7:0/1782045893 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.406+0000 7fae88ce9700 1 --2- 172.21.15.7:0/1782045893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae840b09f0 0x7fae8406e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.406+0000 7fae837fe700 1 -- 172.21.15.7:0/1782045893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae840b09f0 msgr2=0x7fae8406e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.406+0000 7fae837fe700 1 --2- 172.21.15.7:0/1782045893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae840b09f0 0x7fae8406e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.406+0000 7fae837fe700 1 -- 172.21.15.7:0/1782045893 >> v1:172.21.15.188:6789/0 conn(0x7fae841404e0 legacy=0x7fae840cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.406+0000 7fae837fe700 1 -- 172.21.15.7:0/1782045893 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae840cc6c0 con 0x7fae840cbf10 2024-01-27T02:33:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.407+0000 7fae837fe700 1 --2- 172.21.15.7:0/1782045893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae840cbf10 0x7fae8406e2e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fae780099b0 tx=0x7fae78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583921943cb41385 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.407+0000 7fae82ffd700 1 -- 172.21.15.7:0/1782045893 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae78009ea0 con 0x7fae840cbf10 2024-01-27T02:33:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.407+0000 7fae82ffd700 1 -- 172.21.15.7:0/1782045893 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7800b860 con 0x7fae840cbf10 2024-01-27T02:33:57.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.407+0000 7fae82ffd700 1 -- 172.21.15.7:0/1782045893 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7800bc00 con 0x7fae840cbf10 2024-01-27T02:33:57.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.407+0000 7fae8a74c700 1 -- 172.21.15.7:0/1782045893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae840cbf10 msgr2=0x7fae8406e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:57.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.407+0000 7fae8a74c700 1 --2- 172.21.15.7:0/1782045893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae840cbf10 0x7fae8406e2e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fae780099b0 tx=0x7fae78009250 comp rx=0 tx=0).stop 2024-01-27T02:33:57.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.407+0000 7fae8a74c700 1 -- 172.21.15.7:0/1782045893 shutdown_connections 2024-01-27T02:33:57.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.407+0000 7fae8a74c700 1 --2- 172.21.15.7:0/1782045893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae840b09f0 0x7fae8406e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:57.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.407+0000 7fae8a74c700 1 --2- 172.21.15.7:0/1782045893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae840cbf10 0x7fae8406e2e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:57.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.407+0000 7fae8a74c700 1 -- 172.21.15.7:0/1782045893 >> 172.21.15.7:0/1782045893 conn(0x7fae840cdc30 msgr2=0x7fae840ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:57.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.408+0000 7fae8a74c700 1 -- 172.21.15.7:0/1782045893 shutdown_connections 2024-01-27T02:33:57.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.408+0000 7fae8a74c700 1 -- 172.21.15.7:0/1782045893 wait complete. 2024-01-27T02:33:57.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.409+0000 7fae8a74c700 1 Processor -- start 2024-01-27T02:33:57.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.409+0000 7fae8a74c700 1 -- start start 2024-01-27T02:33:57.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.410+0000 7fae8a74c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae840b09f0 0x7fae84169920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:57.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.410+0000 7fae8a74c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae840cbf10 0x7fae84150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:57.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.410+0000 7fae8a74c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae841404e0 0x7fae8414efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:57.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.410+0000 7fae8a74c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae84151bc0 con 0x7fae840b09f0 2024-01-27T02:33:57.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.410+0000 7fae8a74c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8414d3b0 con 0x7fae840cbf10 2024-01-27T02:33:57.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.410+0000 7fae8a74c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae841518e0 con 0x7fae841404e0 2024-01-27T02:33:57.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.410+0000 7fae83fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae840b09f0 0x7fae84169920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:57.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.410+0000 7fae83fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae840b09f0 0x7fae84169920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34818/0 (socket says 172.21.15.7:34818) 2024-01-27T02:33:57.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.410+0000 7fae83fff700 1 -- 172.21.15.7:0/1925371100 learned_addr learned my addr 172.21.15.7:0/1925371100 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:57.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.410+0000 7fae837fe700 1 --2- 172.21.15.7:0/1925371100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae840cbf10 0x7fae84150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:57.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.410+0000 7fae88ce9700 1 --2- 172.21.15.7:0/1925371100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae841404e0 0x7fae8414efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:57.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.410+0000 7fae83fff700 1 -- 172.21.15.7:0/1925371100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae841404e0 msgr2=0x7fae8414efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:57.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.410+0000 7fae83fff700 1 --2- 172.21.15.7:0/1925371100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae841404e0 0x7fae8414efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:57.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.410+0000 7fae83fff700 1 -- 172.21.15.7:0/1925371100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae840cbf10 msgr2=0x7fae84150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:57.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.410+0000 7fae83fff700 1 --2- 172.21.15.7:0/1925371100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae840cbf10 0x7fae84150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:57.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.410+0000 7fae83fff700 1 -- 172.21.15.7:0/1925371100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae780094f0 con 0x7fae840b09f0 2024-01-27T02:33:57.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.411+0000 7fae83fff700 1 --2- 172.21.15.7:0/1925371100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae840b09f0 0x7fae84169920 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fae7400c4e0 tx=0x7fae7400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:57.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.411+0000 7fae6bfff700 1 -- 172.21.15.7:0/1925371100 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae74011590 con 0x7fae840b09f0 2024-01-27T02:33:57.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.411+0000 7fae6bfff700 1 -- 172.21.15.7:0/1925371100 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae74011710 con 0x7fae840b09f0 2024-01-27T02:33:57.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.411+0000 7fae8a74c700 1 -- 172.21.15.7:0/1925371100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae8414cd60 con 0x7fae840b09f0 2024-01-27T02:33:57.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.411+0000 7fae6bfff700 1 -- 172.21.15.7:0/1925371100 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae74011ad0 con 0x7fae840b09f0 2024-01-27T02:33:57.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.411+0000 7fae8a74c700 1 -- 172.21.15.7:0/1925371100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae84169e60 con 0x7fae840b09f0 2024-01-27T02:33:57.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.412+0000 7fae8a74c700 1 -- 172.21.15.7:0/1925371100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae8404c710 con 0x7fae840b09f0 2024-01-27T02:33:57.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.412+0000 7fae6bfff700 1 -- 172.21.15.7:0/1925371100 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fae7401b3b0 con 0x7fae840b09f0 2024-01-27T02:33:57.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.413+0000 7fae6bfff700 1 --2- 172.21.15.7:0/1925371100 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fae70066f30 0x7fae700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:57.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.413+0000 7fae6bfff700 1 -- 172.21.15.7:0/1925371100 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fae74083a10 con 0x7fae840b09f0 2024-01-27T02:33:57.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.413+0000 7fae837fe700 1 --2- 172.21.15.7:0/1925371100 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fae70066f30 0x7fae700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:57.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.413+0000 7fae837fe700 1 --2- 172.21.15.7:0/1925371100 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fae70066f30 0x7fae700693e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fae780099b0 tx=0x7fae78017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:57.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.417+0000 7fae6bfff700 1 -- 172.21.15.7:0/1925371100 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae74050ba0 con 0x7fae840b09f0 2024-01-27T02:33:57.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.674+0000 7fae8a74c700 1 -- 172.21.15.7:0/1925371100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae840c4670 con 0x7fae840b09f0 2024-01-27T02:33:57.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.674+0000 7fae6bfff700 1 -- 172.21.15.7:0/1925371100 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4591 (secure 0 0 0) 0x7fae74017070 con 0x7fae840b09f0 2024-01-27T02:33:57.675 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:57.675 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4883,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/198511042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":198511042},{"type":"v1","addr":"172.21.15.188:6836","nonce":198511042}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":4886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1222999254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1222999254},{"type":"v1","addr":"172.21.15.188:6837","nonce":1222999254}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2474016526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2474016526},{"type":"v1","addr":"172.21.15.7:6834","nonce":2474016526}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:33:24.950288+0000","modified":"2024-01-27T02:33:28.814599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5262":{"gid":5262,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/999728998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":999728998},{"type":"v1","addr":"172.21.15.7:6835","nonce":999728998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-27T02:33:57.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.677+0000 7fae8a74c700 1 -- 172.21.15.7:0/1925371100 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fae70066f30 msgr2=0x7fae700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:57.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.677+0000 7fae8a74c700 1 --2- 172.21.15.7:0/1925371100 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fae70066f30 0x7fae700693e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fae780099b0 tx=0x7fae78017000 comp rx=0 tx=0).stop 2024-01-27T02:33:57.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.677+0000 7fae8a74c700 1 -- 172.21.15.7:0/1925371100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae840b09f0 msgr2=0x7fae84169920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:57.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.677+0000 7fae8a74c700 1 --2- 172.21.15.7:0/1925371100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae840b09f0 0x7fae84169920 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fae7400c4e0 tx=0x7fae7400c7f0 comp rx=0 tx=0).stop 2024-01-27T02:33:57.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.677+0000 7fae8a74c700 1 -- 172.21.15.7:0/1925371100 shutdown_connections 2024-01-27T02:33:57.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.677+0000 7fae8a74c700 1 --2- 172.21.15.7:0/1925371100 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fae70066f30 0x7fae700693e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:57.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.677+0000 7fae8a74c700 1 --2- 172.21.15.7:0/1925371100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae841404e0 0x7fae8414efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:57.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.677+0000 7fae8a74c700 1 --2- 172.21.15.7:0/1925371100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae840b09f0 0x7fae84169920 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:57.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.677+0000 7fae8a74c700 1 --2- 172.21.15.7:0/1925371100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae840cbf10 0x7fae84150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:57.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.677+0000 7fae8a74c700 1 -- 172.21.15.7:0/1925371100 >> 172.21.15.7:0/1925371100 conn(0x7fae840cdc30 msgr2=0x7fae8413a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:57.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.677+0000 7fae8a74c700 1 -- 172.21.15.7:0/1925371100 shutdown_connections 2024-01-27T02:33:57.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:57.677+0000 7fae8a74c700 1 -- 172.21.15.7:0/1925371100 wait complete. 2024-01-27T02:33:57.679 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-01-27T02:33:57.693 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-01-27T02:33:57.845 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","whoami":0,"id":5262,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":32.025188442000001,"mdsmap_epoch":26,"osdmap_epoch":53,"osdmap_epoch_barrier":48,"uptime":37.145992192999998} 2024-01-27T02:33:57.845 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", "fs_name": "cephfs", "id": 5262, "mdsmap_epoch": 26, "osdmap_epoch": 53, "osdmap_epoch_barrier": 48, "rank_uptime": 32.025188442, "state": "up:active", "uptime": 37.145992193, "want_state": "up:active", "whoami": 0 } 2024-01-27T02:33:57.845 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-01-27T02:33:57.845 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:33:57.846 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-01-27T02:33:58.156 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records in 2024-01-27T02:33:58.156 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records out 2024-01-27T02:33:58.156 INFO:teuthology.orchestra.run.smithi007.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.290579 s, 180 MB/s 2024-01-27T02:33:58.157 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-27T02:33:58.157 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:33:58.212 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:33:58.261 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-27T02:33:58.262 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-27T02:33:58.262 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:33:58.262 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-27T02:33:58.344 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:33:58.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:58.344 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-27T02:33:58.368 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-27T02:33:58.369 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:33:58.369 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-01-27T02:33:58.413 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:33:58.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:33:58.413 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-27T02:33:58.470 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:33:58.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.609+0000 7fb6dfbe5700 1 Processor -- start 2024-01-27T02:33:58.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.609+0000 7fb6dfbe5700 1 -- start start 2024-01-27T02:33:58.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.610+0000 7fb6dfbe5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb6d80c4cc0 0x7fb6d80ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:58.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.610+0000 7fb6dfbe5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d80cd710 con 0x7fb6d815a230 2024-01-27T02:33:58.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.610+0000 7fb6dfbe5700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d80c5320 con 0x7fb6d80cd2f0 2024-01-27T02:33:58.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.610+0000 7fb6dfbe5700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d80c9520 con 0x7fb6d80c4cc0 2024-01-27T02:33:58.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.610+0000 7fb6dd180700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb6d80c4cc0 0x7fb6d80ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:58.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.610+0000 7fb6dd180700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb6d80c4cc0 0x7fb6d80ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34824/0 (socket says 172.21.15.7:34824) 2024-01-27T02:33:58.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.610+0000 7fb6dd981700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb6d815a230 0x7fb6d80c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59252/0 (socket says 172.21.15.7:59252) 2024-01-27T02:33:58.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.610+0000 7fb6dd180700 1 -- 172.21.15.7:0/1243394724 learned_addr learned my addr 172.21.15.7:0/1243394724 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:58.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.611+0000 7fb6dd180700 1 -- 172.21.15.7:0/1243394724 >> v1:172.21.15.188:6789/0 conn(0x7fb6d80cd2f0 legacy=0x7fb6d80ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:58.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.611+0000 7fb6dd180700 1 -- 172.21.15.7:0/1243394724 >> v1:172.21.15.7:6789/0 conn(0x7fb6d815a230 legacy=0x7fb6d80c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:58.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.611+0000 7fb6dd180700 1 -- 172.21.15.7:0/1243394724 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d80c9800 con 0x7fb6d80c4cc0 2024-01-27T02:33:58.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.611+0000 7fb6dc97f700 1 -- 172.21.15.7:0/1243394724 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573384672 0 0) 0x7fb6d80cd710 con 0x7fb6d815a230 2024-01-27T02:33:58.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.611+0000 7fb6dd180700 1 --2- 172.21.15.7:0/1243394724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d80c4cc0 0x7fb6d80ce310 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80099b0 tx=0x7fb6c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17767e39ead8847c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:58.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.611+0000 7fb6dc97f700 1 -- 172.21.15.7:0/1243394724 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c800b860 con 0x7fb6d80c4cc0 2024-01-27T02:33:58.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.611+0000 7fb6dc97f700 1 -- 172.21.15.7:0/1243394724 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c800b9c0 con 0x7fb6d80c4cc0 2024-01-27T02:33:58.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.611+0000 7fb6dc97f700 1 -- 172.21.15.7:0/1243394724 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c800bd60 con 0x7fb6d80c4cc0 2024-01-27T02:33:58.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.612+0000 7fb6dfbe5700 1 -- 172.21.15.7:0/1243394724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d80c4cc0 msgr2=0x7fb6d80ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:58.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.612+0000 7fb6dfbe5700 1 --2- 172.21.15.7:0/1243394724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d80c4cc0 0x7fb6d80ce310 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80099b0 tx=0x7fb6c8009250 comp rx=0 tx=0).stop 2024-01-27T02:33:58.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.612+0000 7fb6dfbe5700 1 -- 172.21.15.7:0/1243394724 shutdown_connections 2024-01-27T02:33:58.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.612+0000 7fb6dfbe5700 1 --2- 172.21.15.7:0/1243394724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d80c4cc0 0x7fb6d80ce310 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:58.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.612+0000 7fb6dfbe5700 1 -- 172.21.15.7:0/1243394724 >> 172.21.15.7:0/1243394724 conn(0x7fb6d8098020 msgr2=0x7fb6d80711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:58.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.612+0000 7fb6dfbe5700 1 -- 172.21.15.7:0/1243394724 shutdown_connections 2024-01-27T02:33:58.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.612+0000 7fb6dfbe5700 1 -- 172.21.15.7:0/1243394724 wait complete. 2024-01-27T02:33:58.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.613+0000 7fb6dfbe5700 1 Processor -- start 2024-01-27T02:33:58.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.613+0000 7fb6dfbe5700 1 -- start start 2024-01-27T02:33:58.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.614+0000 7fb6dfbe5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d80c4cc0 0x7fb6d80b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:58.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.614+0000 7fb6dfbe5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d80cd2f0 0x7fb6d814f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:58.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.614+0000 7fb6dfbe5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6d815a230 0x7fb6d80cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:58.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.614+0000 7fb6dfbe5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d814f6a0 con 0x7fb6d80c4cc0 2024-01-27T02:33:58.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.614+0000 7fb6dfbe5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d816a3f0 con 0x7fb6d815a230 2024-01-27T02:33:58.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.614+0000 7fb6dfbe5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d806cf80 con 0x7fb6d80cd2f0 2024-01-27T02:33:58.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.614+0000 7fb6dd981700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d80c4cc0 0x7fb6d80b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:58.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.614+0000 7fb6dd981700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d80c4cc0 0x7fb6d80b36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34840/0 (socket says 172.21.15.7:34840) 2024-01-27T02:33:58.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.614+0000 7fb6dd981700 1 -- 172.21.15.7:0/2008052420 learned_addr learned my addr 172.21.15.7:0/2008052420 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:58.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.614+0000 7fb6de182700 1 --2- 172.21.15.7:0/2008052420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6d815a230 0x7fb6d80cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:58.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.614+0000 7fb6dd180700 1 --2- 172.21.15.7:0/2008052420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d80cd2f0 0x7fb6d814f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:58.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.614+0000 7fb6dd981700 1 -- 172.21.15.7:0/2008052420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d80cd2f0 msgr2=0x7fb6d814f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:58.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.614+0000 7fb6dd981700 1 --2- 172.21.15.7:0/2008052420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d80cd2f0 0x7fb6d814f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:58.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.614+0000 7fb6dd981700 1 -- 172.21.15.7:0/2008052420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6d815a230 msgr2=0x7fb6d80cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:58.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.614+0000 7fb6dd981700 1 --2- 172.21.15.7:0/2008052420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6d815a230 0x7fb6d80cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:58.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.614+0000 7fb6dd981700 1 -- 172.21.15.7:0/2008052420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c80094f0 con 0x7fb6d80c4cc0 2024-01-27T02:33:58.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.615+0000 7fb6de182700 1 --2- 172.21.15.7:0/2008052420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6d815a230 0x7fb6d80cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:33:58.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.615+0000 7fb6dd981700 1 --2- 172.21.15.7:0/2008052420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d80c4cc0 0x7fb6d80b36b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400c5e0 tx=0x7fb6d4007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:58.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.615+0000 7fb6cdffb700 1 -- 172.21.15.7:0/2008052420 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d40072e0 con 0x7fb6d80c4cc0 2024-01-27T02:33:58.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.615+0000 7fb6cdffb700 1 -- 172.21.15.7:0/2008052420 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d4007460 con 0x7fb6d80c4cc0 2024-01-27T02:33:58.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.615+0000 7fb6cdffb700 1 -- 172.21.15.7:0/2008052420 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d40057f0 con 0x7fb6d80c4cc0 2024-01-27T02:33:58.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.615+0000 7fb6dfbe5700 1 -- 172.21.15.7:0/2008052420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d80b3d40 con 0x7fb6d80c4cc0 2024-01-27T02:33:58.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.615+0000 7fb6dfbe5700 1 -- 172.21.15.7:0/2008052420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d80b4300 con 0x7fb6d80c4cc0 2024-01-27T02:33:58.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.616+0000 7fb6dfbe5700 1 -- 172.21.15.7:0/2008052420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d804c710 con 0x7fb6d80c4cc0 2024-01-27T02:33:58.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.616+0000 7fb6cdffb700 1 -- 172.21.15.7:0/2008052420 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb6d4005ba0 con 0x7fb6d80c4cc0 2024-01-27T02:33:58.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.617+0000 7fb6cdffb700 1 --2- 172.21.15.7:0/2008052420 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb6bc066f30 0x7fb6bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:58.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.617+0000 7fb6cdffb700 1 -- 172.21.15.7:0/2008052420 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fb6d4083ab0 con 0x7fb6d80c4cc0 2024-01-27T02:33:58.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.617+0000 7fb6dd180700 1 --2- 172.21.15.7:0/2008052420 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb6bc066f30 0x7fb6bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:58.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.618+0000 7fb6dd180700 1 --2- 172.21.15.7:0/2008052420 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb6bc066f30 0x7fb6bc0693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80099b0 tx=0x7fb6c8017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:58.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.622+0000 7fb6cdffb700 1 -- 172.21.15.7:0/2008052420 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6d4050ce0 con 0x7fb6d80c4cc0 2024-01-27T02:33:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.874+0000 7fb6dfbe5700 1 -- 172.21.15.7:0/2008052420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6d80c9ae0 con 0x7fb6d80c4cc0 2024-01-27T02:33:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.875+0000 7fb6cdffb700 1 -- 172.21.15.7:0/2008052420 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4591 (secure 0 0 0) 0x7fb6d4017070 con 0x7fb6d80c4cc0 2024-01-27T02:33:58.875 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:58.876 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4883,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/198511042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":198511042},{"type":"v1","addr":"172.21.15.188:6836","nonce":198511042}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":4886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1222999254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1222999254},{"type":"v1","addr":"172.21.15.188:6837","nonce":1222999254}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2474016526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2474016526},{"type":"v1","addr":"172.21.15.7:6834","nonce":2474016526}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:33:24.950288+0000","modified":"2024-01-27T02:33:28.814599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5262":{"gid":5262,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/999728998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":999728998},{"type":"v1","addr":"172.21.15.7:6835","nonce":999728998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-27T02:33:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.877+0000 7fb6dfbe5700 1 -- 172.21.15.7:0/2008052420 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb6bc066f30 msgr2=0x7fb6bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.877+0000 7fb6dfbe5700 1 --2- 172.21.15.7:0/2008052420 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb6bc066f30 0x7fb6bc0693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80099b0 tx=0x7fb6c8017000 comp rx=0 tx=0).stop 2024-01-27T02:33:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.877+0000 7fb6dfbe5700 1 -- 172.21.15.7:0/2008052420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d80c4cc0 msgr2=0x7fb6d80b36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.877+0000 7fb6dfbe5700 1 --2- 172.21.15.7:0/2008052420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d80c4cc0 0x7fb6d80b36b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400c5e0 tx=0x7fb6d4007ef0 comp rx=0 tx=0).stop 2024-01-27T02:33:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.877+0000 7fb6dfbe5700 1 -- 172.21.15.7:0/2008052420 shutdown_connections 2024-01-27T02:33:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.877+0000 7fb6dfbe5700 1 --2- 172.21.15.7:0/2008052420 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb6bc066f30 0x7fb6bc0693e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.877+0000 7fb6dfbe5700 1 --2- 172.21.15.7:0/2008052420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6d815a230 0x7fb6d80cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.877+0000 7fb6dfbe5700 1 --2- 172.21.15.7:0/2008052420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d80c4cc0 0x7fb6d80b36b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.877+0000 7fb6dfbe5700 1 --2- 172.21.15.7:0/2008052420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d80cd2f0 0x7fb6d814f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.877+0000 7fb6dfbe5700 1 -- 172.21.15.7:0/2008052420 >> 172.21.15.7:0/2008052420 conn(0x7fb6d8098020 msgr2=0x7fb6d8067f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.877+0000 7fb6dfbe5700 1 -- 172.21.15.7:0/2008052420 shutdown_connections 2024-01-27T02:33:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:58.877+0000 7fb6dfbe5700 1 -- 172.21.15.7:0/2008052420 wait complete. 2024-01-27T02:33:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-01-27T02:33:58.891 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:33:59.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.022+0000 7fb83e36f700 1 Processor -- start 2024-01-27T02:33:59.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.022+0000 7fb83e36f700 1 -- start start 2024-01-27T02:33:59.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.023+0000 7fb83e36f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb83806fa70 0x7fb838165280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:59.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.023+0000 7fb83e36f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb83815a880 con 0x7fb8380a7c10 2024-01-27T02:33:59.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.023+0000 7fb83e36f700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8380af950 con 0x7fb8380a7fe0 2024-01-27T02:33:59.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.023+0000 7fb83e36f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8380afde0 con 0x7fb83806fa70 2024-01-27T02:33:59.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.023+0000 7fb837fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb8380a7c10 0x7fb8380ae340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59256/0 (socket says 172.21.15.7:59256) 2024-01-27T02:33:59.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.023+0000 7fb837fff700 1 -- 172.21.15.7:0/1582479260 learned_addr learned my addr 172.21.15.7:0/1582479260 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:59.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.023+0000 7fb83c90c700 1 --2- 172.21.15.7:0/1582479260 >> v2:172.21.15.7:3300/0 conn(0x7fb83806fa70 0x7fb838165280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:59.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.023+0000 7fb836ffd700 1 -- 172.21.15.7:0/1582479260 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3953517242 0 0) 0x7fb83815a880 con 0x7fb8380a7c10 2024-01-27T02:33:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.023+0000 7fb836ffd700 1 -- 172.21.15.7:0/1582479260 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb820003570 con 0x7fb8380a7c10 2024-01-27T02:33:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.023+0000 7fb836ffd700 1 -- 172.21.15.7:0/1582479260 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1274843509 0 0) 0x7fb8380af950 con 0x7fb8380a7fe0 2024-01-27T02:33:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.023+0000 7fb836ffd700 1 -- 172.21.15.7:0/1582479260 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb83815a880 con 0x7fb8380a7fe0 2024-01-27T02:33:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.023+0000 7fb83c90c700 1 -- 172.21.15.7:0/1582479260 >> v1:172.21.15.188:6789/0 conn(0x7fb8380a7fe0 legacy=0x7fb83806f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.023+0000 7fb83c90c700 1 -- 172.21.15.7:0/1582479260 >> v1:172.21.15.7:6789/0 conn(0x7fb8380a7c10 legacy=0x7fb8380ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.023+0000 7fb83c90c700 1 -- 172.21.15.7:0/1582479260 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8380aed40 con 0x7fb83806fa70 2024-01-27T02:33:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.024+0000 7fb83c90c700 1 --2- 172.21.15.7:0/1582479260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb83806fa70 0x7fb838165280 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb830009e60 tx=0x7fb830009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4e9178592a8a3f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.024+0000 7fb836ffd700 1 -- 172.21.15.7:0/1582479260 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83000e070 con 0x7fb83806fa70 2024-01-27T02:33:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.024+0000 7fb836ffd700 1 -- 172.21.15.7:0/1582479260 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83000bab0 con 0x7fb83806fa70 2024-01-27T02:33:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.024+0000 7fb836ffd700 1 -- 172.21.15.7:0/1582479260 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83000be70 con 0x7fb83806fa70 2024-01-27T02:33:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.024+0000 7fb83e36f700 1 -- 172.21.15.7:0/1582479260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb83806fa70 msgr2=0x7fb838165280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.024+0000 7fb83e36f700 1 --2- 172.21.15.7:0/1582479260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb83806fa70 0x7fb838165280 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb830009e60 tx=0x7fb830009780 comp rx=0 tx=0).stop 2024-01-27T02:33:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.024+0000 7fb83e36f700 1 -- 172.21.15.7:0/1582479260 shutdown_connections 2024-01-27T02:33:59.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.024+0000 7fb83e36f700 1 --2- 172.21.15.7:0/1582479260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb83806fa70 0x7fb838165280 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.024+0000 7fb83e36f700 1 -- 172.21.15.7:0/1582479260 >> 172.21.15.7:0/1582479260 conn(0x7fb8380679b0 msgr2=0x7fb838067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:59.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.024+0000 7fb83e36f700 1 -- 172.21.15.7:0/1582479260 shutdown_connections 2024-01-27T02:33:59.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.024+0000 7fb83e36f700 1 -- 172.21.15.7:0/1582479260 wait complete. 2024-01-27T02:33:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.026+0000 7fb83e36f700 1 Processor -- start 2024-01-27T02:33:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.026+0000 7fb83e36f700 1 -- start start 2024-01-27T02:33:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.027+0000 7fb83e36f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb83806fa70 0x7fb83806dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.027+0000 7fb83e36f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8380a7c10 0x7fb83815ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.027+0000 7fb83e36f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8380a7fe0 0x7fb83815b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.027+0000 7fb83e36f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83815dc50 con 0x7fb83806fa70 2024-01-27T02:33:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.027+0000 7fb83e36f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83806e2e0 con 0x7fb8380a7c10 2024-01-27T02:33:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.027+0000 7fb83e36f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb83806e510 con 0x7fb8380a7fe0 2024-01-27T02:33:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.027+0000 7fb837fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb83806fa70 0x7fb83806dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.027+0000 7fb837fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb83806fa70 0x7fb83806dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34866/0 (socket says 172.21.15.7:34866) 2024-01-27T02:33:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.027+0000 7fb837fff700 1 -- 172.21.15.7:0/3321554597 learned_addr learned my addr 172.21.15.7:0/3321554597 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.027+0000 7fb83c90c700 1 --2- 172.21.15.7:0/3321554597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8380a7fe0 0x7fb83815b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.027+0000 7fb8377fe700 1 --2- 172.21.15.7:0/3321554597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8380a7c10 0x7fb83815ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.027+0000 7fb837fff700 1 -- 172.21.15.7:0/3321554597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8380a7fe0 msgr2=0x7fb83815b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.027+0000 7fb837fff700 1 --2- 172.21.15.7:0/3321554597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8380a7fe0 0x7fb83815b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.027+0000 7fb837fff700 1 -- 172.21.15.7:0/3321554597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8380a7c10 msgr2=0x7fb83815ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.027+0000 7fb837fff700 1 --2- 172.21.15.7:0/3321554597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8380a7c10 0x7fb83815ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.027+0000 7fb837fff700 1 -- 172.21.15.7:0/3321554597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb830009a20 con 0x7fb83806fa70 2024-01-27T02:33:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.027+0000 7fb8377fe700 1 --2- 172.21.15.7:0/3321554597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8380a7c10 0x7fb83815ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:33:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.027+0000 7fb837fff700 1 --2- 172.21.15.7:0/3321554597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb83806fa70 0x7fb83806dda0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb82800c7c0 tx=0x7fb828007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.027+0000 7fb81ffff700 1 -- 172.21.15.7:0/3321554597 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb828005b30 con 0x7fb83806fa70 2024-01-27T02:33:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.027+0000 7fb81ffff700 1 -- 172.21.15.7:0/3321554597 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb828005c90 con 0x7fb83806fa70 2024-01-27T02:33:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.027+0000 7fb83e36f700 1 -- 172.21.15.7:0/3321554597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb83806e7f0 con 0x7fb83806fa70 2024-01-27T02:33:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.027+0000 7fb81ffff700 1 -- 172.21.15.7:0/3321554597 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb828007300 con 0x7fb83806fa70 2024-01-27T02:33:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.028+0000 7fb83e36f700 1 -- 172.21.15.7:0/3321554597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb83806edb0 con 0x7fb83806fa70 2024-01-27T02:33:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.028+0000 7fb83e36f700 1 -- 172.21.15.7:0/3321554597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb83804c710 con 0x7fb83806fa70 2024-01-27T02:33:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.029+0000 7fb81ffff700 1 -- 172.21.15.7:0/3321554597 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb82801b400 con 0x7fb83806fa70 2024-01-27T02:33:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.029+0000 7fb81ffff700 1 --2- 172.21.15.7:0/3321554597 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb824066f30 0x7fb8240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.029+0000 7fb81ffff700 1 -- 172.21.15.7:0/3321554597 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fb8280840d0 con 0x7fb83806fa70 2024-01-27T02:33:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.029+0000 7fb8377fe700 1 --2- 172.21.15.7:0/3321554597 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb824066f30 0x7fb8240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.030+0000 7fb8377fe700 1 --2- 172.21.15.7:0/3321554597 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb824066f30 0x7fb8240693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb82c007fd0 tx=0x7fb82c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.034+0000 7fb81ffff700 1 -- 172.21.15.7:0/3321554597 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8280512e0 con 0x7fb83806fa70 2024-01-27T02:33:59.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.235+0000 7fb83e36f700 1 -- 172.21.15.7:0/3321554597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8380af020 con 0x7fb83806fa70 2024-01-27T02:33:59.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.236+0000 7fb81ffff700 1 -- 172.21.15.7:0/3321554597 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fb828054900 con 0x7fb83806fa70 2024-01-27T02:33:59.237 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:33:59.237 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":53,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:33:56.965932+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:33:23.445643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-27T02:33:23.921995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.30","mappings":[{"from":0,"to":4}]},{"pgid":"7.35","mappings":[{"from":0,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:33:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.238+0000 7fb83e36f700 1 -- 172.21.15.7:0/3321554597 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb824066f30 msgr2=0x7fb8240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.238+0000 7fb83e36f700 1 --2- 172.21.15.7:0/3321554597 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb824066f30 0x7fb8240693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb82c007fd0 tx=0x7fb82c009040 comp rx=0 tx=0).stop 2024-01-27T02:33:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.238+0000 7fb83e36f700 1 -- 172.21.15.7:0/3321554597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb83806fa70 msgr2=0x7fb83806dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.238+0000 7fb83e36f700 1 --2- 172.21.15.7:0/3321554597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb83806fa70 0x7fb83806dda0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb82800c7c0 tx=0x7fb828007ea0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.238+0000 7fb83e36f700 1 -- 172.21.15.7:0/3321554597 shutdown_connections 2024-01-27T02:33:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.238+0000 7fb83e36f700 1 --2- 172.21.15.7:0/3321554597 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb824066f30 0x7fb8240693e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.238+0000 7fb83e36f700 1 --2- 172.21.15.7:0/3321554597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8380a7fe0 0x7fb83815b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.238+0000 7fb83e36f700 1 --2- 172.21.15.7:0/3321554597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb83806fa70 0x7fb83806dda0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.238+0000 7fb83e36f700 1 --2- 172.21.15.7:0/3321554597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8380a7c10 0x7fb83815ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.238+0000 7fb83e36f700 1 -- 172.21.15.7:0/3321554597 >> 172.21.15.7:0/3321554597 conn(0x7fb8380679b0 msgr2=0x7fb8381573e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.239+0000 7fb83e36f700 1 -- 172.21.15.7:0/3321554597 shutdown_connections 2024-01-27T02:33:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.239+0000 7fb83e36f700 1 -- 172.21.15.7:0/3321554597 wait complete. 2024-01-27T02:33:59.253 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-27T02:33:23.445643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-01-27T02:33:23.921995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-27T02:33:59.253 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:33:59.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.388+0000 7f0121cd4700 1 Processor -- start 2024-01-27T02:33:59.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.388+0000 7f0121cd4700 1 -- start start 2024-01-27T02:33:59.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.389+0000 7f0121cd4700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f011c154920 0x7f011c0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:59.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.389+0000 7f0121cd4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f011c0ad7c0 0x7f011c063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:59.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.389+0000 7f0121cd4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f011c0adec0 con 0x7f011c154550 2024-01-27T02:33:59.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.389+0000 7f0121cd4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f011c0ae1a0 con 0x7f011c0ad7c0 2024-01-27T02:33:59.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.389+0000 7f0121cd4700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f011c15a850 con 0x7f011c154920 2024-01-27T02:33:59.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.389+0000 7f011b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f011c154550 0x7f011c15b050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59270/0 (socket says 172.21.15.7:59270) 2024-01-27T02:33:59.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.389+0000 7f011bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f011c0ad7c0 0x7f011c063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:59.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.389+0000 7f011b7fe700 1 -- 172.21.15.7:0/2461993021 learned_addr learned my addr 172.21.15.7:0/2461993021 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:59.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.389+0000 7f011affd700 1 --2- 172.21.15.7:0/2461993021 >> v2:172.21.15.188:3300/0 conn(0x7f011c154920 0x7f011c0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:59.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.389+0000 7f011bfff700 1 -- 172.21.15.7:0/2461993021 >> v2:172.21.15.188:3300/0 conn(0x7f011c154920 msgr2=0x7f011c0ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.389+0000 7f011bfff700 1 --2- 172.21.15.7:0/2461993021 >> v2:172.21.15.188:3300/0 conn(0x7f011c154920 0x7f011c0ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.389+0000 7f011bfff700 1 -- 172.21.15.7:0/2461993021 >> v1:172.21.15.7:6789/0 conn(0x7f011c154550 legacy=0x7f011c15b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.390+0000 7f011bfff700 1 -- 172.21.15.7:0/2461993021 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011c15ab30 con 0x7f011c0ad7c0 2024-01-27T02:33:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.389+0000 7f011a7fc700 1 -- 172.21.15.7:0/2461993021 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513869854 0 0) 0x7f011c0adec0 con 0x7f011c154550 2024-01-27T02:33:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.390+0000 7f011bfff700 1 --2- 172.21.15.7:0/2461993021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f011c0ad7c0 0x7f011c063580 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f011c0b32f0 tx=0x7f0114009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f882885ca186d2ee server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.390+0000 7f011a7fc700 1 -- 172.21.15.7:0/2461993021 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011400e070 con 0x7f011c0ad7c0 2024-01-27T02:33:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.390+0000 7f011a7fc700 1 -- 172.21.15.7:0/2461993021 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f011400bb60 con 0x7f011c0ad7c0 2024-01-27T02:33:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.390+0000 7f011a7fc700 1 -- 172.21.15.7:0/2461993021 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0114012430 con 0x7f011c0ad7c0 2024-01-27T02:33:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.390+0000 7f0121cd4700 1 -- 172.21.15.7:0/2461993021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f011c0ad7c0 msgr2=0x7f011c063580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.390+0000 7f0121cd4700 1 --2- 172.21.15.7:0/2461993021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f011c0ad7c0 0x7f011c063580 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f011c0b32f0 tx=0x7f0114009780 comp rx=0 tx=0).stop 2024-01-27T02:33:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.390+0000 7f0121cd4700 1 -- 172.21.15.7:0/2461993021 shutdown_connections 2024-01-27T02:33:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.390+0000 7f0121cd4700 1 --2- 172.21.15.7:0/2461993021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f011c0ad7c0 0x7f011c063580 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.390+0000 7f0121cd4700 1 --2- 172.21.15.7:0/2461993021 >> v2:172.21.15.188:3300/0 conn(0x7f011c154920 0x7f011c0ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.390+0000 7f0121cd4700 1 -- 172.21.15.7:0/2461993021 >> 172.21.15.7:0/2461993021 conn(0x7f011c068480 msgr2=0x7f011c0cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:59.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.390+0000 7f0121cd4700 1 -- 172.21.15.7:0/2461993021 shutdown_connections 2024-01-27T02:33:59.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.391+0000 7f0121cd4700 1 -- 172.21.15.7:0/2461993021 wait complete. 2024-01-27T02:33:59.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.392+0000 7f0121cd4700 1 Processor -- start 2024-01-27T02:33:59.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.392+0000 7f0121cd4700 1 -- start start 2024-01-27T02:33:59.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.393+0000 7f0121cd4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f011c0ad7c0 0x7f011c155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:59.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.393+0000 7f0121cd4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f011c154550 0x7f011c150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:59.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.393+0000 7f0121cd4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f011c154920 0x7f011c151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:59.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.393+0000 7f0121cd4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c156af0 con 0x7f011c0ad7c0 2024-01-27T02:33:59.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.393+0000 7f0121cd4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c155f70 con 0x7f011c154920 2024-01-27T02:33:59.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.393+0000 7f0121cd4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f011c156250 con 0x7f011c154550 2024-01-27T02:33:59.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.393+0000 7f011b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f011c0ad7c0 0x7f011c155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:59.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.393+0000 7f011b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f011c0ad7c0 0x7f011c155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34882/0 (socket says 172.21.15.7:34882) 2024-01-27T02:33:59.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.393+0000 7f011b7fe700 1 -- 172.21.15.7:0/635985231 learned_addr learned my addr 172.21.15.7:0/635985231 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.393+0000 7f011bfff700 1 --2- 172.21.15.7:0/635985231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f011c154920 0x7f011c151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.393+0000 7f011affd700 1 --2- 172.21.15.7:0/635985231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f011c154550 0x7f011c150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.393+0000 7f011b7fe700 1 -- 172.21.15.7:0/635985231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f011c154550 msgr2=0x7f011c150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.393+0000 7f011b7fe700 1 --2- 172.21.15.7:0/635985231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f011c154550 0x7f011c150480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.393+0000 7f011b7fe700 1 -- 172.21.15.7:0/635985231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f011c154920 msgr2=0x7f011c151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.393+0000 7f011b7fe700 1 --2- 172.21.15.7:0/635985231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f011c154920 0x7f011c151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.393+0000 7f011b7fe700 1 -- 172.21.15.7:0/635985231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0114009a20 con 0x7f011c0ad7c0 2024-01-27T02:33:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.393+0000 7f011bfff700 1 --2- 172.21.15.7:0/635985231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f011c154920 0x7f011c151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:33:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.393+0000 7f011affd700 1 --2- 172.21.15.7:0/635985231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f011c154550 0x7f011c150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:33:59.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.393+0000 7f011b7fe700 1 --2- 172.21.15.7:0/635985231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f011c0ad7c0 0x7f011c155720 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f010c00c810 tx=0x7f010c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:59.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.393+0000 7f01037fe700 1 -- 172.21.15.7:0/635985231 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c005b30 con 0x7f011c0ad7c0 2024-01-27T02:33:59.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.393+0000 7f01037fe700 1 -- 172.21.15.7:0/635985231 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010c005c90 con 0x7f011c0ad7c0 2024-01-27T02:33:59.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.394+0000 7f0121cd4700 1 -- 172.21.15.7:0/635985231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f011c156530 con 0x7f011c0ad7c0 2024-01-27T02:33:59.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.394+0000 7f01037fe700 1 -- 172.21.15.7:0/635985231 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c007300 con 0x7f011c0ad7c0 2024-01-27T02:33:59.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.394+0000 7f0121cd4700 1 -- 172.21.15.7:0/635985231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f011c155c60 con 0x7f011c0ad7c0 2024-01-27T02:33:59.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.395+0000 7f0121cd4700 1 -- 172.21.15.7:0/635985231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f011c072060 con 0x7f011c0ad7c0 2024-01-27T02:33:59.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.395+0000 7f01037fe700 1 -- 172.21.15.7:0/635985231 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f010c01b400 con 0x7f011c0ad7c0 2024-01-27T02:33:59.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.395+0000 7f01037fe700 1 --2- 172.21.15.7:0/635985231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0108066f30 0x7f01080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:59.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.395+0000 7f01037fe700 1 -- 172.21.15.7:0/635985231 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f010c084100 con 0x7f011c0ad7c0 2024-01-27T02:33:59.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.396+0000 7f011affd700 1 --2- 172.21.15.7:0/635985231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0108066f30 0x7f01080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:59.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.396+0000 7f011affd700 1 --2- 172.21.15.7:0/635985231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0108066f30 0x7f01080693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0110006f00 tx=0x7f0110008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:59.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.400+0000 7f01037fe700 1 -- 172.21.15.7:0/635985231 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f010c051310 con 0x7f011c0ad7c0 2024-01-27T02:33:59.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.602+0000 7f0121cd4700 1 -- 172.21.15.7:0/635985231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f011c06dcd0 con 0x7f011c0ad7c0 2024-01-27T02:33:59.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.603+0000 7f01037fe700 1 -- 172.21.15.7:0/635985231 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7f010c054930 con 0x7f011c0ad7c0 2024-01-27T02:33:59.603 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:33:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.605+0000 7f0121cd4700 1 -- 172.21.15.7:0/635985231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0108066f30 msgr2=0x7f01080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.605+0000 7f0121cd4700 1 --2- 172.21.15.7:0/635985231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0108066f30 0x7f01080693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0110006f00 tx=0x7f0110008040 comp rx=0 tx=0).stop 2024-01-27T02:33:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.605+0000 7f0121cd4700 1 -- 172.21.15.7:0/635985231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f011c0ad7c0 msgr2=0x7f011c155720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.605+0000 7f0121cd4700 1 --2- 172.21.15.7:0/635985231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f011c0ad7c0 0x7f011c155720 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f010c00c810 tx=0x7f010c007e00 comp rx=0 tx=0).stop 2024-01-27T02:33:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.605+0000 7f0121cd4700 1 -- 172.21.15.7:0/635985231 shutdown_connections 2024-01-27T02:33:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.605+0000 7f0121cd4700 1 --2- 172.21.15.7:0/635985231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0108066f30 0x7f01080693e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.605+0000 7f0121cd4700 1 --2- 172.21.15.7:0/635985231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f011c154920 0x7f011c151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.605+0000 7f0121cd4700 1 --2- 172.21.15.7:0/635985231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f011c0ad7c0 0x7f011c155720 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.605+0000 7f0121cd4700 1 --2- 172.21.15.7:0/635985231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f011c154550 0x7f011c150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.605+0000 7f0121cd4700 1 -- 172.21.15.7:0/635985231 >> 172.21.15.7:0/635985231 conn(0x7f011c068480 msgr2=0x7f011c1534a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.605+0000 7f0121cd4700 1 -- 172.21.15.7:0/635985231 shutdown_connections 2024-01-27T02:33:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.605+0000 7f0121cd4700 1 -- 172.21.15.7:0/635985231 wait complete. 2024-01-27T02:33:59.619 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-27T02:33:59.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.753+0000 7f37c0f21700 1 Processor -- start 2024-01-27T02:33:59.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.753+0000 7f37c0f21700 1 -- start start 2024-01-27T02:33:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.754+0000 7f37c0f21700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37bc1416c0 0x7f37bc0cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.754+0000 7f37c0f21700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f37bc141a90 0x7f37bc0cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.754+0000 7f37c0f21700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37bc15c7d0 con 0x7f37bc0cab00 2024-01-27T02:33:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.754+0000 7f37c0f21700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37bc0cd9c0 con 0x7f37bc141a90 2024-01-27T02:33:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.754+0000 7f37c0f21700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc0c9640 con 0x7f37bc1416c0 2024-01-27T02:33:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.755+0000 7f37b3fff700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f37bc141a90 0x7f37bc0cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.755+0000 7f37ba59c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37bc1416c0 0x7f37bc0cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.755+0000 7f37bad9d700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f37bc0cab00 0x7f37bc0caf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:59532/0 (socket says 172.21.15.7:59532) 2024-01-27T02:33:59.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.755+0000 7f37bad9d700 1 -- 172.21.15.7:0/1077502747 learned_addr learned my addr 172.21.15.7:0/1077502747 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:33:59.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.755+0000 7f37ba59c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37bc1416c0 0x7f37bc0cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:57798/0 (socket says 172.21.15.7:57798) 2024-01-27T02:33:59.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.755+0000 7f37b3fff700 1 -- 172.21.15.7:0/1077502747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37bc1416c0 msgr2=0x7f37bc0cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:33:59.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.755+0000 7f37b3fff700 1 --2- 172.21.15.7:0/1077502747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37bc1416c0 0x7f37bc0cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.755+0000 7f37b3fff700 1 -- 172.21.15.7:0/1077502747 >> v1:172.21.15.188:6789/0 conn(0x7f37bc0cab00 legacy=0x7f37bc0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:59.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.755+0000 7f37b3fff700 1 -- 172.21.15.7:0/1077502747 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37bc15bbc0 con 0x7f37bc141a90 2024-01-27T02:33:59.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.755+0000 7f37b3fff700 1 --2- 172.21.15.7:0/1077502747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37bc141a90 0x7f37bc0cbc20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f37a40098e0 tx=0x7f37a40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16e78a971ad9adcb server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:59.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.756+0000 7f37b9d9b700 1 -- 172.21.15.7:0/1077502747 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4011630 con 0x7f37bc141a90 2024-01-27T02:33:59.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.756+0000 7f37b9d9b700 1 -- 172.21.15.7:0/1077502747 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a4011790 con 0x7f37bc141a90 2024-01-27T02:33:59.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.756+0000 7f37b9d9b700 1 -- 172.21.15.7:0/1077502747 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4011b30 con 0x7f37bc141a90 2024-01-27T02:33:59.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.756+0000 7f37c0f21700 1 -- 172.21.15.7:0/1077502747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37bc141a90 msgr2=0x7f37bc0cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:59.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.756+0000 7f37c0f21700 1 --2- 172.21.15.7:0/1077502747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37bc141a90 0x7f37bc0cbc20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f37a40098e0 tx=0x7f37a40091a0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.756+0000 7f37c0f21700 1 -- 172.21.15.7:0/1077502747 shutdown_connections 2024-01-27T02:33:59.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.756+0000 7f37c0f21700 1 --2- 172.21.15.7:0/1077502747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37bc1416c0 0x7f37bc0cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.756+0000 7f37c0f21700 1 --2- 172.21.15.7:0/1077502747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37bc141a90 0x7f37bc0cbc20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.756+0000 7f37c0f21700 1 -- 172.21.15.7:0/1077502747 >> 172.21.15.7:0/1077502747 conn(0x7f37bc063580 msgr2=0x7f37bc063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:59.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.756+0000 7f37c0f21700 1 -- 172.21.15.7:0/1077502747 shutdown_connections 2024-01-27T02:33:59.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.756+0000 7f37c0f21700 1 -- 172.21.15.7:0/1077502747 wait complete. 2024-01-27T02:33:59.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.758+0000 7f37c0f21700 1 Processor -- start 2024-01-27T02:33:59.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.758+0000 7f37c0f21700 1 -- start start 2024-01-27T02:33:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.759+0000 7f37c0f21700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37bc0cab00 0x7f37bc157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.759+0000 7f37c0f21700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37bc1416c0 0x7f37bc15a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.759+0000 7f37c0f21700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37bc141a90 0x7f37bc15a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.759+0000 7f37c0f21700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc0b1ed0 con 0x7f37bc0cab00 2024-01-27T02:33:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.759+0000 7f37c0f21700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc0b21b0 con 0x7f37bc1416c0 2024-01-27T02:33:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.759+0000 7f37c0f21700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc15ad70 con 0x7f37bc141a90 2024-01-27T02:33:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.759+0000 7f37ba59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37bc0cab00 0x7f37bc157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.759+0000 7f37ba59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37bc0cab00 0x7f37bc157830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34896/0 (socket says 172.21.15.7:34896) 2024-01-27T02:33:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.759+0000 7f37ba59c700 1 -- 172.21.15.7:0/2349009623 learned_addr learned my addr 172.21.15.7:0/2349009623 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:33:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.759+0000 7f37bad9d700 1 --2- 172.21.15.7:0/2349009623 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37bc141a90 0x7f37bc15a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.759+0000 7f37b3fff700 1 --2- 172.21.15.7:0/2349009623 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37bc1416c0 0x7f37bc15a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.759+0000 7f37ba59c700 1 -- 172.21.15.7:0/2349009623 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37bc141a90 msgr2=0x7f37bc15a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.759+0000 7f37ba59c700 1 --2- 172.21.15.7:0/2349009623 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37bc141a90 0x7f37bc15a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.759+0000 7f37ba59c700 1 -- 172.21.15.7:0/2349009623 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37bc1416c0 msgr2=0x7f37bc15a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.759+0000 7f37ba59c700 1 --2- 172.21.15.7:0/2349009623 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37bc1416c0 0x7f37bc15a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.759+0000 7f37ba59c700 1 -- 172.21.15.7:0/2349009623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a4009420 con 0x7f37bc0cab00 2024-01-27T02:33:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.759+0000 7f37bad9d700 1 --2- 172.21.15.7:0/2349009623 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37bc141a90 0x7f37bc15a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:33:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.759+0000 7f37ba59c700 1 --2- 172.21.15.7:0/2349009623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37bc0cab00 0x7f37bc157830 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00ecb0 tx=0x7f37ac009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.760+0000 7f37b27fc700 1 -- 172.21.15.7:0/2349009623 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac00c9d0 con 0x7f37bc0cab00 2024-01-27T02:33:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.760+0000 7f37b27fc700 1 -- 172.21.15.7:0/2349009623 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac00cb30 con 0x7f37bc0cab00 2024-01-27T02:33:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.760+0000 7f37c0f21700 1 -- 172.21.15.7:0/2349009623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37bc15b000 con 0x7f37bc0cab00 2024-01-27T02:33:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.760+0000 7f37b27fc700 1 -- 172.21.15.7:0/2349009623 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac00ceb0 con 0x7f37bc0cab00 2024-01-27T02:33:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.760+0000 7f37c0f21700 1 -- 172.21.15.7:0/2349009623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37bc15b5c0 con 0x7f37bc0cab00 2024-01-27T02:33:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.761+0000 7f37c0f21700 1 -- 172.21.15.7:0/2349009623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37bc072060 con 0x7f37bc0cab00 2024-01-27T02:33:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.761+0000 7f37b27fc700 1 -- 172.21.15.7:0/2349009623 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f37ac013680 con 0x7f37bc0cab00 2024-01-27T02:33:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.762+0000 7f37b27fc700 1 --2- 172.21.15.7:0/2349009623 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f379c066f30 0x7f379c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:33:59.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.762+0000 7f37b27fc700 1 -- 172.21.15.7:0/2349009623 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f37ac084c70 con 0x7f37bc0cab00 2024-01-27T02:33:59.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.762+0000 7f37b3fff700 1 --2- 172.21.15.7:0/2349009623 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f379c066f30 0x7f379c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:33:59.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.762+0000 7f37b3fff700 1 --2- 172.21.15.7:0/2349009623 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f379c066f30 0x7f379c0693e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f37a40093f0 tx=0x7f37a4016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:33:59.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.766+0000 7f37b27fc700 1 -- 172.21.15.7:0/2349009623 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37ac051ea0 con 0x7f37bc0cab00 2024-01-27T02:33:59.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.970+0000 7f37c0f21700 1 -- 172.21.15.7:0/2349009623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f37bc15bea0 con 0x7f37bc0cab00 2024-01-27T02:33:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.970+0000 7f37b27fc700 1 -- 172.21.15.7:0/2349009623 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7f37ac0170a0 con 0x7f37bc0cab00 2024-01-27T02:33:59.971 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-27T02:33:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.972+0000 7f37c0f21700 1 -- 172.21.15.7:0/2349009623 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f379c066f30 msgr2=0x7f379c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.972+0000 7f37c0f21700 1 --2- 172.21.15.7:0/2349009623 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f379c066f30 0x7f379c0693e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f37a40093f0 tx=0x7f37a4016000 comp rx=0 tx=0).stop 2024-01-27T02:33:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.972+0000 7f37c0f21700 1 -- 172.21.15.7:0/2349009623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37bc0cab00 msgr2=0x7f37bc157830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:33:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.972+0000 7f37c0f21700 1 --2- 172.21.15.7:0/2349009623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37bc0cab00 0x7f37bc157830 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00ecb0 tx=0x7f37ac009120 comp rx=0 tx=0).stop 2024-01-27T02:33:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.972+0000 7f37c0f21700 1 -- 172.21.15.7:0/2349009623 shutdown_connections 2024-01-27T02:33:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.972+0000 7f37c0f21700 1 --2- 172.21.15.7:0/2349009623 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f379c066f30 0x7f379c0693e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.972+0000 7f37c0f21700 1 --2- 172.21.15.7:0/2349009623 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37bc141a90 0x7f37bc15a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.972+0000 7f37c0f21700 1 --2- 172.21.15.7:0/2349009623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37bc0cab00 0x7f37bc157830 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.972+0000 7f37c0f21700 1 --2- 172.21.15.7:0/2349009623 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37bc1416c0 0x7f37bc15a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:33:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.972+0000 7f37c0f21700 1 -- 172.21.15.7:0/2349009623 >> 172.21.15.7:0/2349009623 conn(0x7f37bc063580 msgr2=0x7f37bc0c4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:33:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.972+0000 7f37c0f21700 1 -- 172.21.15.7:0/2349009623 shutdown_connections 2024-01-27T02:33:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:33:59.972+0000 7f37c0f21700 1 -- 172.21.15.7:0/2349009623 wait complete. 2024-01-27T02:33:59.988 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-27T02:34:00.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.122+0000 7f131c5c2700 1 Processor -- start 2024-01-27T02:34:00.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.122+0000 7f131c5c2700 1 -- start start 2024-01-27T02:34:00.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.123+0000 7f131c5c2700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f13140cb950 0x7f13140b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:00.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.123+0000 7f131c5c2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f13140cbd20 0x7f13140ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:00.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.123+0000 7f131c5c2700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13140c9660 con 0x7f13141412d0 2024-01-27T02:34:00.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.123+0000 7f131c5c2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f131415c980 con 0x7f13140cbd20 2024-01-27T02:34:00.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.123+0000 7f131c5c2700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f131415be00 con 0x7f13140cb950 2024-01-27T02:34:00.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.123+0000 7f1319b5d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f13140cbd20 0x7f13140ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:00.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.123+0000 7f1319b5d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f13140cbd20 0x7f13140ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34900/0 (socket says 172.21.15.7:34900) 2024-01-27T02:34:00.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.123+0000 7f1319b5d700 1 -- 172.21.15.7:0/3069651962 learned_addr learned my addr 172.21.15.7:0/3069651962 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:00.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.123+0000 7f131a35e700 1 --2- 172.21.15.7:0/3069651962 >> v2:172.21.15.188:3300/0 conn(0x7f13140cb950 0x7f13140b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:00.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.124+0000 7f1319b5d700 1 -- 172.21.15.7:0/3069651962 >> v2:172.21.15.188:3300/0 conn(0x7f13140cb950 msgr2=0x7f13140b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:00.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.124+0000 7f1319b5d700 1 --2- 172.21.15.7:0/3069651962 >> v2:172.21.15.188:3300/0 conn(0x7f13140cb950 0x7f13140b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.124+0000 7f1319b5d700 1 -- 172.21.15.7:0/3069651962 >> v1:172.21.15.188:6789/0 conn(0x7f13141412d0 legacy=0x7f1314141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:00.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.124+0000 7f1319b5d700 1 -- 172.21.15.7:0/3069651962 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131415c0e0 con 0x7f13140cbd20 2024-01-27T02:34:00.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.124+0000 7f131a35e700 1 --2- 172.21.15.7:0/3069651962 >> v2:172.21.15.188:3300/0 conn(0x7f13140cb950 0x7f13140b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:00.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.124+0000 7f1319b5d700 1 --2- 172.21.15.7:0/3069651962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13140cbd20 0x7f13140ce4a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f13140cfd30 tx=0x7f1304009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6578cb0225016d0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:00.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.124+0000 7f131935c700 1 -- 172.21.15.7:0/3069651962 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130400b860 con 0x7f13140cbd20 2024-01-27T02:34:00.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.124+0000 7f131935c700 1 -- 172.21.15.7:0/3069651962 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130400b9c0 con 0x7f13140cbd20 2024-01-27T02:34:00.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.124+0000 7f131935c700 1 -- 172.21.15.7:0/3069651962 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130400bd60 con 0x7f13140cbd20 2024-01-27T02:34:00.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.124+0000 7f131c5c2700 1 -- 172.21.15.7:0/3069651962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13140cbd20 msgr2=0x7f13140ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:00.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.124+0000 7f131c5c2700 1 --2- 172.21.15.7:0/3069651962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13140cbd20 0x7f13140ce4a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f13140cfd30 tx=0x7f1304009250 comp rx=0 tx=0).stop 2024-01-27T02:34:00.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.125+0000 7f131c5c2700 1 -- 172.21.15.7:0/3069651962 shutdown_connections 2024-01-27T02:34:00.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.125+0000 7f131c5c2700 1 --2- 172.21.15.7:0/3069651962 >> v2:172.21.15.188:3300/0 conn(0x7f13140cb950 0x7f13140b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.125+0000 7f131c5c2700 1 --2- 172.21.15.7:0/3069651962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13140cbd20 0x7f13140ce4a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.125+0000 7f131c5c2700 1 -- 172.21.15.7:0/3069651962 >> 172.21.15.7:0/3069651962 conn(0x7f1314063580 msgr2=0x7f1314063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:00.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.125+0000 7f131c5c2700 1 -- 172.21.15.7:0/3069651962 shutdown_connections 2024-01-27T02:34:00.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.125+0000 7f131c5c2700 1 -- 172.21.15.7:0/3069651962 wait complete. 2024-01-27T02:34:00.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.126+0000 7f131c5c2700 1 Processor -- start 2024-01-27T02:34:00.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.126+0000 7f131c5c2700 1 -- start start 2024-01-27T02:34:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.127+0000 7f131c5c2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13140cb950 0x7f131413e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.127+0000 7f131c5c2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13140cbd20 0x7f13140a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.127+0000 7f131c5c2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13141412d0 0x7f1314167e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.127+0000 7f131c5c2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131415afb0 con 0x7f13140cbd20 2024-01-27T02:34:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.127+0000 7f131c5c2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131415a3a0 con 0x7f13141412d0 2024-01-27T02:34:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.127+0000 7f131c5c2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131415a680 con 0x7f13140cb950 2024-01-27T02:34:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.127+0000 7f1319b5d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13140cbd20 0x7f13140a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.127+0000 7f1319b5d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13140cbd20 0x7f13140a64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34904/0 (socket says 172.21.15.7:34904) 2024-01-27T02:34:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.127+0000 7f1319b5d700 1 -- 172.21.15.7:0/992067340 learned_addr learned my addr 172.21.15.7:0/992067340 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.127+0000 7f131a35e700 1 --2- 172.21.15.7:0/992067340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13140cb950 0x7f131413e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.127+0000 7f131ab5f700 1 --2- 172.21.15.7:0/992067340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13141412d0 0x7f1314167e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.127+0000 7f1319b5d700 1 -- 172.21.15.7:0/992067340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13140cb950 msgr2=0x7f131413e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.127+0000 7f1319b5d700 1 --2- 172.21.15.7:0/992067340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13140cb950 0x7f131413e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.127+0000 7f1319b5d700 1 -- 172.21.15.7:0/992067340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13141412d0 msgr2=0x7f1314167e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.127+0000 7f1319b5d700 1 --2- 172.21.15.7:0/992067340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13141412d0 0x7f1314167e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.127+0000 7f1319b5d700 1 -- 172.21.15.7:0/992067340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13040094f0 con 0x7f13140cbd20 2024-01-27T02:34:00.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.127+0000 7f131ab5f700 1 --2- 172.21.15.7:0/992067340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13141412d0 0x7f1314167e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:34:00.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.127+0000 7f131a35e700 1 --2- 172.21.15.7:0/992067340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13140cb950 0x7f131413e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:34:00.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.127+0000 7f1319b5d700 1 --2- 172.21.15.7:0/992067340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13140cbd20 0x7f13140a64a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1304009290 tx=0x7f1304017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:00.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.128+0000 7f130a7fc700 1 -- 172.21.15.7:0/992067340 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1304011e60 con 0x7f13140cbd20 2024-01-27T02:34:00.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.128+0000 7f130a7fc700 1 -- 172.21.15.7:0/992067340 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130401a660 con 0x7f13140cbd20 2024-01-27T02:34:00.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.128+0000 7f131c5c2700 1 -- 172.21.15.7:0/992067340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131415a960 con 0x7f13140cbd20 2024-01-27T02:34:00.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.128+0000 7f130a7fc700 1 -- 172.21.15.7:0/992067340 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130401aa20 con 0x7f13140cbd20 2024-01-27T02:34:00.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.128+0000 7f131c5c2700 1 -- 172.21.15.7:0/992067340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13140a69e0 con 0x7f13140cbd20 2024-01-27T02:34:00.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.129+0000 7f131c5c2700 1 -- 172.21.15.7:0/992067340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f131415b120 con 0x7f13140cbd20 2024-01-27T02:34:00.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.129+0000 7f130a7fc700 1 -- 172.21.15.7:0/992067340 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f130401b430 con 0x7f13140cbd20 2024-01-27T02:34:00.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.129+0000 7f130a7fc700 1 --2- 172.21.15.7:0/992067340 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f12f8066f30 0x7f12f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:00.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.130+0000 7f130a7fc700 1 -- 172.21.15.7:0/992067340 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f1304085d10 con 0x7f13140cbd20 2024-01-27T02:34:00.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.130+0000 7f131a35e700 1 --2- 172.21.15.7:0/992067340 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f12f8066f30 0x7f12f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:00.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.130+0000 7f131a35e700 1 --2- 172.21.15.7:0/992067340 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f12f8066f30 0x7f12f80693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1310006f00 tx=0x7f1310008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:00.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.134+0000 7f130a7fc700 1 -- 172.21.15.7:0/992067340 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1304052f40 con 0x7f13140cbd20 2024-01-27T02:34:00.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.336+0000 7f131c5c2700 1 -- 172.21.15.7:0/992067340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1314072060 con 0x7f13140cbd20 2024-01-27T02:34:00.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.336+0000 7f130a7fc700 1 -- 172.21.15.7:0/992067340 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7f13040150a0 con 0x7f13140cbd20 2024-01-27T02:34:00.337 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-27T02:34:00.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.338+0000 7f131c5c2700 1 -- 172.21.15.7:0/992067340 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f12f8066f30 msgr2=0x7f12f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:00.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.338+0000 7f131c5c2700 1 --2- 172.21.15.7:0/992067340 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f12f8066f30 0x7f12f80693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1310006f00 tx=0x7f1310008040 comp rx=0 tx=0).stop 2024-01-27T02:34:00.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.338+0000 7f131c5c2700 1 -- 172.21.15.7:0/992067340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13140cbd20 msgr2=0x7f13140a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:00.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.338+0000 7f131c5c2700 1 --2- 172.21.15.7:0/992067340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13140cbd20 0x7f13140a64a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1304009290 tx=0x7f1304017000 comp rx=0 tx=0).stop 2024-01-27T02:34:00.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.338+0000 7f131c5c2700 1 -- 172.21.15.7:0/992067340 shutdown_connections 2024-01-27T02:34:00.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.338+0000 7f131c5c2700 1 --2- 172.21.15.7:0/992067340 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f12f8066f30 0x7f12f80693e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.338+0000 7f131c5c2700 1 --2- 172.21.15.7:0/992067340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13141412d0 0x7f1314167e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.338+0000 7f131c5c2700 1 --2- 172.21.15.7:0/992067340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13140cb950 0x7f131413e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.338+0000 7f131c5c2700 1 --2- 172.21.15.7:0/992067340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13140cbd20 0x7f13140a64a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.338+0000 7f131c5c2700 1 -- 172.21.15.7:0/992067340 >> 172.21.15.7:0/992067340 conn(0x7f1314063580 msgr2=0x7f13140cf460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:00.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.339+0000 7f131c5c2700 1 -- 172.21.15.7:0/992067340 shutdown_connections 2024-01-27T02:34:00.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.339+0000 7f131c5c2700 1 -- 172.21.15.7:0/992067340 wait complete. 2024-01-27T02:34:00.352 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:34:00.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.484+0000 7f382e6cb700 1 Processor -- start 2024-01-27T02:34:00.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.484+0000 7f382e6cb700 1 -- start start 2024-01-27T02:34:00.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.485+0000 7f382e6cb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f38280a7fe0 0x7f382806f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:00.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.485+0000 7f382e6cb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382815dfb0 0x7f382806f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:00.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.485+0000 7f382e6cb700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f382815a880 con 0x7f38280a7c10 2024-01-27T02:34:00.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.485+0000 7f382e6cb700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38280afde0 con 0x7f38280a7fe0 2024-01-27T02:34:00.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.485+0000 7f382e6cb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38280af950 con 0x7f382815dfb0 2024-01-27T02:34:00.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.485+0000 7f38277fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f38280a7fe0 0x7f382806f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:00.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.485+0000 7f38277fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f38280a7fe0 0x7f382806f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34920/0 (socket says 172.21.15.7:34920) 2024-01-27T02:34:00.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.485+0000 7f38277fe700 1 -- 172.21.15.7:0/190551044 learned_addr learned my addr 172.21.15.7:0/190551044 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:00.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.485+0000 7f382cc68700 1 --2- 172.21.15.7:0/190551044 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382815dfb0 0x7f382806f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:00.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.485+0000 7f38277fe700 1 -- 172.21.15.7:0/190551044 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382815dfb0 msgr2=0x7f382806f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:00.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.485+0000 7f38277fe700 1 --2- 172.21.15.7:0/190551044 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382815dfb0 0x7f382806f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.485+0000 7f38277fe700 1 -- 172.21.15.7:0/190551044 >> v1:172.21.15.188:6789/0 conn(0x7f38280a7c10 legacy=0x7f38280ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:00.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.485+0000 7f38277fe700 1 -- 172.21.15.7:0/190551044 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38280aed40 con 0x7f38280a7fe0 2024-01-27T02:34:00.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.485+0000 7f38277fe700 1 --2- 172.21.15.7:0/190551044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38280a7fe0 0x7f382806f360 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f38100099b0 tx=0x7f3810009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596604bf8becc027 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:00.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.486+0000 7f3826ffd700 1 -- 172.21.15.7:0/190551044 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381000b860 con 0x7f38280a7fe0 2024-01-27T02:34:00.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.486+0000 7f3826ffd700 1 -- 172.21.15.7:0/190551044 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381000b9c0 con 0x7f38280a7fe0 2024-01-27T02:34:00.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.486+0000 7f3826ffd700 1 -- 172.21.15.7:0/190551044 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381000bd60 con 0x7f38280a7fe0 2024-01-27T02:34:00.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.486+0000 7f382e6cb700 1 -- 172.21.15.7:0/190551044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38280a7fe0 msgr2=0x7f382806f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:00.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.486+0000 7f382e6cb700 1 --2- 172.21.15.7:0/190551044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38280a7fe0 0x7f382806f360 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f38100099b0 tx=0x7f3810009250 comp rx=0 tx=0).stop 2024-01-27T02:34:00.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.486+0000 7f382e6cb700 1 -- 172.21.15.7:0/190551044 shutdown_connections 2024-01-27T02:34:00.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.486+0000 7f382e6cb700 1 --2- 172.21.15.7:0/190551044 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382815dfb0 0x7f382806f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.486+0000 7f382e6cb700 1 --2- 172.21.15.7:0/190551044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38280a7fe0 0x7f382806f360 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.486+0000 7f382e6cb700 1 -- 172.21.15.7:0/190551044 >> 172.21.15.7:0/190551044 conn(0x7f38280679b0 msgr2=0x7f3828067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:00.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.486+0000 7f382e6cb700 1 -- 172.21.15.7:0/190551044 shutdown_connections 2024-01-27T02:34:00.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.486+0000 7f382e6cb700 1 -- 172.21.15.7:0/190551044 wait complete. 2024-01-27T02:34:00.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.487+0000 7f382e6cb700 1 Processor -- start 2024-01-27T02:34:00.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.488+0000 7f382e6cb700 1 -- start start 2024-01-27T02:34:00.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.488+0000 7f382e6cb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38280a7c10 0x7f3828160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:00.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.488+0000 7f382e6cb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38280a7fe0 0x7f382815f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:00.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.488+0000 7f382e6cb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f382815dfb0 0x7f382815f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:00.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.488+0000 7f382e6cb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3828144c70 con 0x7f382815dfb0 2024-01-27T02:34:00.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.488+0000 7f382e6cb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3828144f50 con 0x7f38280a7fe0 2024-01-27T02:34:00.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.488+0000 7f382e6cb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3828145230 con 0x7f38280a7c10 2024-01-27T02:34:00.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.489+0000 7f382cc68700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f382815dfb0 0x7f382815f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:00.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.489+0000 7f382cc68700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f382815dfb0 0x7f382815f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34922/0 (socket says 172.21.15.7:34922) 2024-01-27T02:34:00.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.489+0000 7f382cc68700 1 -- 172.21.15.7:0/3189033217 learned_addr learned my addr 172.21.15.7:0/3189033217 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:00.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.489+0000 7f38277fe700 1 --2- 172.21.15.7:0/3189033217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38280a7fe0 0x7f382815f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:00.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.489+0000 7f3827fff700 1 --2- 172.21.15.7:0/3189033217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38280a7c10 0x7f3828160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:00.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.489+0000 7f382cc68700 1 -- 172.21.15.7:0/3189033217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38280a7c10 msgr2=0x7f3828160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.489+0000 7f382cc68700 1 --2- 172.21.15.7:0/3189033217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38280a7c10 0x7f3828160a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.489+0000 7f382cc68700 1 -- 172.21.15.7:0/3189033217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38280a7fe0 msgr2=0x7f382815f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.489+0000 7f382cc68700 1 --2- 172.21.15.7:0/3189033217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38280a7fe0 0x7f382815f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.489+0000 7f382cc68700 1 -- 172.21.15.7:0/3189033217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38100094f0 con 0x7f382815dfb0 2024-01-27T02:34:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.489+0000 7f382cc68700 1 --2- 172.21.15.7:0/3189033217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f382815dfb0 0x7f382815f930 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3820007ef0 tx=0x7f3820009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.489+0000 7f381ffff700 1 -- 172.21.15.7:0/3189033217 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382000eb30 con 0x7f382815dfb0 2024-01-27T02:34:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.489+0000 7f381ffff700 1 -- 172.21.15.7:0/3189033217 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382000ec90 con 0x7f382815dfb0 2024-01-27T02:34:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.489+0000 7f381ffff700 1 -- 172.21.15.7:0/3189033217 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3820014670 con 0x7f382815dfb0 2024-01-27T02:34:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.489+0000 7f382e6cb700 1 -- 172.21.15.7:0/3189033217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3828145510 con 0x7f382815dfb0 2024-01-27T02:34:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.490+0000 7f382e6cb700 1 -- 172.21.15.7:0/3189033217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3828160fc0 con 0x7f382815dfb0 2024-01-27T02:34:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.491+0000 7f382e6cb700 1 -- 172.21.15.7:0/3189033217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3828145a30 con 0x7f382815dfb0 2024-01-27T02:34:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.491+0000 7f381ffff700 1 -- 172.21.15.7:0/3189033217 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3820014a60 con 0x7f382815dfb0 2024-01-27T02:34:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.491+0000 7f381ffff700 1 --2- 172.21.15.7:0/3189033217 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3808066f30 0x7f38080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.492+0000 7f381ffff700 1 -- 172.21.15.7:0/3189033217 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f3820084b10 con 0x7f382815dfb0 2024-01-27T02:34:00.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.492+0000 7f3827fff700 1 --2- 172.21.15.7:0/3189033217 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3808066f30 0x7f38080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:00.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.492+0000 7f3827fff700 1 --2- 172.21.15.7:0/3189033217 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3808066f30 0x7f38080693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3818003a80 tx=0x7f3818007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:00.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.496+0000 7f381ffff700 1 -- 172.21.15.7:0/3189033217 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3828145a30 con 0x7f382815dfb0 2024-01-27T02:34:00.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.747+0000 7f382e6cb700 1 -- 172.21.15.7:0/3189033217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3828145a30 con 0x7f382815dfb0 2024-01-27T02:34:00.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.748+0000 7f381ffff700 1 -- 172.21.15.7:0/3189033217 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f3828145a30 con 0x7f382815dfb0 2024-01-27T02:34:00.748 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:34:00.748 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4883,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/198511042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":198511042},{"type":"v1","addr":"172.21.15.188:6836","nonce":198511042}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":4886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1222999254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1222999254},{"type":"v1","addr":"172.21.15.188:6837","nonce":1222999254}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2474016526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2474016526},{"type":"v1","addr":"172.21.15.7:6834","nonce":2474016526}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:33:24.950288+0000","modified":"2024-01-27T02:33:28.814599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5262":{"gid":5262,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/999728998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":999728998},{"type":"v1","addr":"172.21.15.7:6835","nonce":999728998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-27T02:34:00.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.750+0000 7f382e6cb700 1 -- 172.21.15.7:0/3189033217 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3808066f30 msgr2=0x7f38080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:00.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.750+0000 7f382e6cb700 1 --2- 172.21.15.7:0/3189033217 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3808066f30 0x7f38080693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3818003a80 tx=0x7f3818007ec0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.750+0000 7f382e6cb700 1 -- 172.21.15.7:0/3189033217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f382815dfb0 msgr2=0x7f382815f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:00.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.750+0000 7f382e6cb700 1 --2- 172.21.15.7:0/3189033217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f382815dfb0 0x7f382815f930 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3820007ef0 tx=0x7f3820009840 comp rx=0 tx=0).stop 2024-01-27T02:34:00.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.750+0000 7f382e6cb700 1 -- 172.21.15.7:0/3189033217 shutdown_connections 2024-01-27T02:34:00.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.750+0000 7f382e6cb700 1 --2- 172.21.15.7:0/3189033217 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3808066f30 0x7f38080693e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.750+0000 7f382e6cb700 1 --2- 172.21.15.7:0/3189033217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f382815dfb0 0x7f382815f930 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.750+0000 7f382e6cb700 1 --2- 172.21.15.7:0/3189033217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38280a7c10 0x7f3828160a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.750+0000 7f382e6cb700 1 --2- 172.21.15.7:0/3189033217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38280a7fe0 0x7f382815f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.750+0000 7f382e6cb700 1 -- 172.21.15.7:0/3189033217 >> 172.21.15.7:0/3189033217 conn(0x7f38280679b0 msgr2=0x7f382815ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:00.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.750+0000 7f382e6cb700 1 -- 172.21.15.7:0/3189033217 shutdown_connections 2024-01-27T02:34:00.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.750+0000 7f382e6cb700 1 -- 172.21.15.7:0/3189033217 wait complete. 2024-01-27T02:34:00.752 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-01-27T02:34:00.764 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:34:00.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.895+0000 7f462c490700 1 Processor -- start 2024-01-27T02:34:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.895+0000 7f462c490700 1 -- start start 2024-01-27T02:34:00.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.896+0000 7f462c490700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4624159f40 0x7f46240c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:00.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.896+0000 7f462c490700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f46240c3a90 0x7f46240aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.896+0000 7f462c490700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46240c30c0 con 0x7f462415a310 2024-01-27T02:34:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.896+0000 7f462c490700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46240c4550 con 0x7f46240c3a90 2024-01-27T02:34:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.896+0000 7f462c490700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462415e960 con 0x7f4624159f40 2024-01-27T02:34:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.896+0000 7f4629a2b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f46240c3a90 0x7f46240aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.896+0000 7f462aa2d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f462415a310 0x7f4624063580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59278/0 (socket says 172.21.15.7:59278) 2024-01-27T02:34:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.896+0000 7f4629a2b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f46240c3a90 0x7f46240aa8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34934/0 (socket says 172.21.15.7:34934) 2024-01-27T02:34:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.896+0000 7f462aa2d700 1 -- 172.21.15.7:0/859830822 learned_addr learned my addr 172.21.15.7:0/859830822 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:34:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.896+0000 7f462a22c700 1 --2- 172.21.15.7:0/859830822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4624159f40 0x7f46240c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.897+0000 7f4629a2b700 1 -- 172.21.15.7:0/859830822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4624159f40 msgr2=0x7f46240c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.897+0000 7f4629a2b700 1 --2- 172.21.15.7:0/859830822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4624159f40 0x7f46240c3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.897+0000 7f4629a2b700 1 -- 172.21.15.7:0/859830822 >> v1:172.21.15.7:6789/0 conn(0x7f462415a310 legacy=0x7f4624063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.897+0000 7f4629a2b700 1 -- 172.21.15.7:0/859830822 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462415ec40 con 0x7f46240c3a90 2024-01-27T02:34:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.897+0000 7f462922a700 1 -- 172.21.15.7:0/859830822 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1988111706 0 0) 0x7f46240c30c0 con 0x7f462415a310 2024-01-27T02:34:00.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.897+0000 7f4629a2b700 1 --2- 172.21.15.7:0/859830822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46240c3a90 0x7f46240aa8b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f46200098e0 tx=0x7f46200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f73a6da65e8d9a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:00.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.897+0000 7f462922a700 1 -- 172.21.15.7:0/859830822 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462000b640 con 0x7f46240c3a90 2024-01-27T02:34:00.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.897+0000 7f462922a700 1 -- 172.21.15.7:0/859830822 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462000b7a0 con 0x7f46240c3a90 2024-01-27T02:34:00.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.897+0000 7f462922a700 1 -- 172.21.15.7:0/859830822 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462000bb40 con 0x7f46240c3a90 2024-01-27T02:34:00.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.898+0000 7f462c490700 1 -- 172.21.15.7:0/859830822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46240c3a90 msgr2=0x7f46240aa8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:00.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.898+0000 7f462c490700 1 --2- 172.21.15.7:0/859830822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46240c3a90 0x7f46240aa8b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f46200098e0 tx=0x7f46200091a0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.898+0000 7f462c490700 1 -- 172.21.15.7:0/859830822 shutdown_connections 2024-01-27T02:34:00.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.898+0000 7f462c490700 1 --2- 172.21.15.7:0/859830822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4624159f40 0x7f46240c3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.898+0000 7f462c490700 1 --2- 172.21.15.7:0/859830822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46240c3a90 0x7f46240aa8b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.898+0000 7f462c490700 1 -- 172.21.15.7:0/859830822 >> 172.21.15.7:0/859830822 conn(0x7f46240cdc30 msgr2=0x7f46240ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:00.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.898+0000 7f462c490700 1 -- 172.21.15.7:0/859830822 shutdown_connections 2024-01-27T02:34:00.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.898+0000 7f462c490700 1 -- 172.21.15.7:0/859830822 wait complete. 2024-01-27T02:34:00.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.899+0000 7f462c490700 1 Processor -- start 2024-01-27T02:34:00.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.899+0000 7f462c490700 1 -- start start 2024-01-27T02:34:00.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.900+0000 7f462c490700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46240c3a90 0x7f46240c7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:00.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.900+0000 7f462c490700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4624159f40 0x7f462415be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:00.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.900+0000 7f462c490700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f462415a310 0x7f46240cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:00.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.900+0000 7f462c490700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462416ba10 con 0x7f462415a310 2024-01-27T02:34:00.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.900+0000 7f462c490700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462415c370 con 0x7f46240c3a90 2024-01-27T02:34:00.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.900+0000 7f462c490700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462416ae90 con 0x7f4624159f40 2024-01-27T02:34:00.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.900+0000 7f462aa2d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f462415a310 0x7f46240cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:00.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.900+0000 7f462aa2d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f462415a310 0x7f46240cbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34938/0 (socket says 172.21.15.7:34938) 2024-01-27T02:34:00.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.900+0000 7f462aa2d700 1 -- 172.21.15.7:0/1052443878 learned_addr learned my addr 172.21.15.7:0/1052443878 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:00.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.900+0000 7f4629a2b700 1 --2- 172.21.15.7:0/1052443878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4624159f40 0x7f462415be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:00.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.900+0000 7f462a22c700 1 --2- 172.21.15.7:0/1052443878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46240c3a90 0x7f46240c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:00.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.900+0000 7f462aa2d700 1 -- 172.21.15.7:0/1052443878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4624159f40 msgr2=0x7f462415be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:00.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.900+0000 7f462aa2d700 1 --2- 172.21.15.7:0/1052443878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4624159f40 0x7f462415be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.900+0000 7f462aa2d700 1 -- 172.21.15.7:0/1052443878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46240c3a90 msgr2=0x7f46240c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:00.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.900+0000 7f462aa2d700 1 --2- 172.21.15.7:0/1052443878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46240c3a90 0x7f46240c7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:00.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.900+0000 7f462aa2d700 1 -- 172.21.15.7:0/1052443878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4620009420 con 0x7f462415a310 2024-01-27T02:34:00.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.900+0000 7f462aa2d700 1 --2- 172.21.15.7:0/1052443878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f462415a310 0x7f46240cbe60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f461c00cbf0 tx=0x7f461c001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:00.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.901+0000 7f461a7fc700 1 -- 172.21.15.7:0/1052443878 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c00a980 con 0x7f462415a310 2024-01-27T02:34:00.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.901+0000 7f461a7fc700 1 -- 172.21.15.7:0/1052443878 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461c00aae0 con 0x7f462415a310 2024-01-27T02:34:00.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.901+0000 7f462c490700 1 -- 172.21.15.7:0/1052443878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462416b170 con 0x7f462415a310 2024-01-27T02:34:00.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.901+0000 7f461a7fc700 1 -- 172.21.15.7:0/1052443878 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c00aea0 con 0x7f462415a310 2024-01-27T02:34:00.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.901+0000 7f462c490700 1 -- 172.21.15.7:0/1052443878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462416b7b0 con 0x7f462415a310 2024-01-27T02:34:00.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.902+0000 7f462c490700 1 -- 172.21.15.7:0/1052443878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46240c8d10 con 0x7f462415a310 2024-01-27T02:34:00.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.902+0000 7f461a7fc700 1 -- 172.21.15.7:0/1052443878 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f461c007300 con 0x7f462415a310 2024-01-27T02:34:00.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.903+0000 7f461a7fc700 1 --2- 172.21.15.7:0/1052443878 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4608066f30 0x7f46080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:00.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.903+0000 7f461a7fc700 1 -- 172.21.15.7:0/1052443878 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f461c084930 con 0x7f462415a310 2024-01-27T02:34:00.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.903+0000 7f462a22c700 1 --2- 172.21.15.7:0/1052443878 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4608066f30 0x7f46080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:00.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.903+0000 7f462a22c700 1 --2- 172.21.15.7:0/1052443878 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4608066f30 0x7f46080693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4614009d40 tx=0x7f4614008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:00.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:00.907+0000 7f461a7fc700 1 -- 172.21.15.7:0/1052443878 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f461c051be0 con 0x7f462415a310 2024-01-27T02:34:01.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.112+0000 7f462c490700 1 -- 172.21.15.7:0/1052443878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46240c37c0 con 0x7f462415a310 2024-01-27T02:34:01.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.113+0000 7f461a7fc700 1 -- 172.21.15.7:0/1052443878 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f461c055200 con 0x7f462415a310 2024-01-27T02:34:01.113 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:34:01.113 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":53,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:33:56.965932+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:33:23.445643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-27T02:33:23.921995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.30","mappings":[{"from":0,"to":4}]},{"pgid":"7.35","mappings":[{"from":0,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:34:01.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.115+0000 7f462c490700 1 -- 172.21.15.7:0/1052443878 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4608066f30 msgr2=0x7f46080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:01.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.115+0000 7f462c490700 1 --2- 172.21.15.7:0/1052443878 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4608066f30 0x7f46080693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4614009d40 tx=0x7f4614008040 comp rx=0 tx=0).stop 2024-01-27T02:34:01.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.115+0000 7f462c490700 1 -- 172.21.15.7:0/1052443878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f462415a310 msgr2=0x7f46240cbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:01.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.115+0000 7f462c490700 1 --2- 172.21.15.7:0/1052443878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f462415a310 0x7f46240cbe60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f461c00cbf0 tx=0x7f461c001980 comp rx=0 tx=0).stop 2024-01-27T02:34:01.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.115+0000 7f462c490700 1 -- 172.21.15.7:0/1052443878 shutdown_connections 2024-01-27T02:34:01.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.115+0000 7f462c490700 1 --2- 172.21.15.7:0/1052443878 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4608066f30 0x7f46080693e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.115+0000 7f462c490700 1 --2- 172.21.15.7:0/1052443878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f462415a310 0x7f46240cbe60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.115+0000 7f462c490700 1 --2- 172.21.15.7:0/1052443878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46240c3a90 0x7f46240c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.115+0000 7f462c490700 1 --2- 172.21.15.7:0/1052443878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4624159f40 0x7f462415be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.115+0000 7f462c490700 1 -- 172.21.15.7:0/1052443878 >> 172.21.15.7:0/1052443878 conn(0x7f46240cdc30 msgr2=0x7f462406a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:01.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.115+0000 7f462c490700 1 -- 172.21.15.7:0/1052443878 shutdown_connections 2024-01-27T02:34:01.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.115+0000 7f462c490700 1 -- 172.21.15.7:0/1052443878 wait complete. 2024-01-27T02:34:01.129 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-27T02:34:01.129 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:34:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.260+0000 7fb9a7ddf700 1 Processor -- start 2024-01-27T02:34:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.260+0000 7fb9a7ddf700 1 -- start start 2024-01-27T02:34:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.261+0000 7fb9a7ddf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb9a00b1c00 0x7fb9a006e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.261+0000 7fb9a7ddf700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9a00b1fd0 0x7fb9a0156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.261+0000 7fb9a7ddf700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9a0157610 con 0x7fb9a015b5f0 2024-01-27T02:34:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.261+0000 7fb9a7ddf700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9a0157310 con 0x7fb9a00b1c00 2024-01-27T02:34:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.261+0000 7fb9a7ddf700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a006ee00 con 0x7fb9a00b1fd0 2024-01-27T02:34:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.261+0000 7fb9a5b7b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb9a00b1c00 0x7fb9a006e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.261+0000 7fb9a5b7b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb9a00b1c00 0x7fb9a006e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34950/0 (socket says 172.21.15.7:34950) 2024-01-27T02:34:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.261+0000 7fb9a5b7b700 1 -- 172.21.15.7:0/3523672236 learned_addr learned my addr 172.21.15.7:0/3523672236 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.261+0000 7fb9a537a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb9a015b5f0 0x7fb9a01566c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59284/0 (socket says 172.21.15.7:59284) 2024-01-27T02:34:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.261+0000 7fb9a637c700 1 --2- 172.21.15.7:0/3523672236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9a00b1fd0 0x7fb9a0156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.262+0000 7fb9a5b7b700 1 -- 172.21.15.7:0/3523672236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9a00b1fd0 msgr2=0x7fb9a0156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.262+0000 7fb9a5b7b700 1 --2- 172.21.15.7:0/3523672236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9a00b1fd0 0x7fb9a0156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.262+0000 7fb9a5b7b700 1 -- 172.21.15.7:0/3523672236 >> v1:172.21.15.7:6789/0 conn(0x7fb9a015b5f0 legacy=0x7fb9a01566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.262+0000 7fb9a4b79700 1 -- 172.21.15.7:0/3523672236 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2235550258 0 0) 0x7fb9a0157610 con 0x7fb9a015b5f0 2024-01-27T02:34:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.262+0000 7fb9a5b7b700 1 -- 172.21.15.7:0/3523672236 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a006f0e0 con 0x7fb9a00b1c00 2024-01-27T02:34:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.262+0000 7fb9a5b7b700 1 --2- 172.21.15.7:0/3523672236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9a00b1c00 0x7fb9a006e540 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00bc50 tx=0x7fb99c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0925320ead9b371 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.262+0000 7fb9a4b79700 1 -- 172.21.15.7:0/3523672236 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c00be60 con 0x7fb9a00b1c00 2024-01-27T02:34:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.262+0000 7fb9a4b79700 1 -- 172.21.15.7:0/3523672236 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99c00de00 con 0x7fb9a00b1c00 2024-01-27T02:34:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.262+0000 7fb9a4b79700 1 -- 172.21.15.7:0/3523672236 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0196b0 con 0x7fb9a00b1c00 2024-01-27T02:34:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.263+0000 7fb9a7ddf700 1 -- 172.21.15.7:0/3523672236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9a00b1c00 msgr2=0x7fb9a006e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.263+0000 7fb9a7ddf700 1 --2- 172.21.15.7:0/3523672236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9a00b1c00 0x7fb9a006e540 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00bc50 tx=0x7fb99c00b4f0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.263+0000 7fb9a7ddf700 1 -- 172.21.15.7:0/3523672236 shutdown_connections 2024-01-27T02:34:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.263+0000 7fb9a7ddf700 1 --2- 172.21.15.7:0/3523672236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9a00b1fd0 0x7fb9a0156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.263+0000 7fb9a7ddf700 1 --2- 172.21.15.7:0/3523672236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9a00b1c00 0x7fb9a006e540 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.263+0000 7fb9a7ddf700 1 -- 172.21.15.7:0/3523672236 >> 172.21.15.7:0/3523672236 conn(0x7fb9a0068430 msgr2=0x7fb9a006be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.263+0000 7fb9a7ddf700 1 -- 172.21.15.7:0/3523672236 shutdown_connections 2024-01-27T02:34:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.263+0000 7fb9a7ddf700 1 -- 172.21.15.7:0/3523672236 wait complete. 2024-01-27T02:34:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.264+0000 7fb9a7ddf700 1 Processor -- start 2024-01-27T02:34:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.264+0000 7fb9a7ddf700 1 -- start start 2024-01-27T02:34:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.265+0000 7fb9a7ddf700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9a00b1c00 0x7fb9a00a9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.265+0000 7fb9a7ddf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9a00b1fd0 0x7fb9a013dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.265+0000 7fb9a7ddf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9a015b5f0 0x7fb9a00a7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.265+0000 7fb9a7ddf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a00a64a0 con 0x7fb9a00b1fd0 2024-01-27T02:34:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.265+0000 7fb9a7ddf700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a00a72c0 con 0x7fb9a015b5f0 2024-01-27T02:34:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.265+0000 7fb9a7ddf700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a00a66b0 con 0x7fb9a00b1c00 2024-01-27T02:34:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.265+0000 7fb9a537a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9a00b1fd0 0x7fb9a013dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.265+0000 7fb9a537a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9a00b1fd0 0x7fb9a013dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34956/0 (socket says 172.21.15.7:34956) 2024-01-27T02:34:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.265+0000 7fb9a537a700 1 -- 172.21.15.7:0/1318552067 learned_addr learned my addr 172.21.15.7:0/1318552067 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.265+0000 7fb9a5b7b700 1 --2- 172.21.15.7:0/1318552067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9a00b1c00 0x7fb9a00a9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.265+0000 7fb9a537a700 1 -- 172.21.15.7:0/1318552067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9a00b1c00 msgr2=0x7fb9a00a9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.265+0000 7fb9a537a700 1 --2- 172.21.15.7:0/1318552067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9a00b1c00 0x7fb9a00a9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.265+0000 7fb9a537a700 1 -- 172.21.15.7:0/1318552067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9a015b5f0 msgr2=0x7fb9a00a7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.265+0000 7fb9a637c700 1 --2- 172.21.15.7:0/1318552067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9a015b5f0 0x7fb9a00a7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.265+0000 7fb9a537a700 1 --2- 172.21.15.7:0/1318552067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9a015b5f0 0x7fb9a00a7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.265+0000 7fb9a537a700 1 -- 172.21.15.7:0/1318552067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c00b790 con 0x7fb9a00b1fd0 2024-01-27T02:34:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.265+0000 7fb9a537a700 1 --2- 172.21.15.7:0/1318552067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9a00b1fd0 0x7fb9a013dd60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb99400c5b0 tx=0x7fb994001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.266+0000 7fb991ffb700 1 -- 172.21.15.7:0/1318552067 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb994005a70 con 0x7fb9a00b1fd0 2024-01-27T02:34:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.266+0000 7fb991ffb700 1 -- 172.21.15.7:0/1318552067 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb994005bd0 con 0x7fb9a00b1fd0 2024-01-27T02:34:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.266+0000 7fb9a7ddf700 1 -- 172.21.15.7:0/1318552067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a00a6990 con 0x7fb9a00b1fd0 2024-01-27T02:34:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.266+0000 7fb991ffb700 1 -- 172.21.15.7:0/1318552067 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9940070d0 con 0x7fb9a00b1fd0 2024-01-27T02:34:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.266+0000 7fb9a7ddf700 1 -- 172.21.15.7:0/1318552067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a00a6f50 con 0x7fb9a00b1fd0 2024-01-27T02:34:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.267+0000 7fb9a7ddf700 1 -- 172.21.15.7:0/1318552067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9a015ada0 con 0x7fb9a00b1fd0 2024-01-27T02:34:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.267+0000 7fb991ffb700 1 -- 172.21.15.7:0/1318552067 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb994007480 con 0x7fb9a00b1fd0 2024-01-27T02:34:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.268+0000 7fb991ffb700 1 --2- 172.21.15.7:0/1318552067 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb984066f30 0x7fb9840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.268+0000 7fb991ffb700 1 -- 172.21.15.7:0/1318552067 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fb994083a90 con 0x7fb9a00b1fd0 2024-01-27T02:34:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.268+0000 7fb9a5b7b700 1 --2- 172.21.15.7:0/1318552067 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb984066f30 0x7fb9840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.269+0000 7fb9a5b7b700 1 --2- 172.21.15.7:0/1318552067 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb984066f30 0x7fb9840693e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00bc50 tx=0x7fb99c01b000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.272+0000 7fb991ffb700 1 -- 172.21.15.7:0/1318552067 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb994050cc0 con 0x7fb9a00b1fd0 2024-01-27T02:34:01.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.523+0000 7fb9a7ddf700 1 -- 172.21.15.7:0/1318552067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9a006c6d0 con 0x7fb9a00b1fd0 2024-01-27T02:34:01.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.524+0000 7fb991ffb700 1 -- 172.21.15.7:0/1318552067 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4591 (secure 0 0 0) 0x7fb994017020 con 0x7fb9a00b1fd0 2024-01-27T02:34:01.525 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:34:01.525 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4883,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/198511042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":198511042},{"type":"v1","addr":"172.21.15.188:6836","nonce":198511042}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":4886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1222999254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1222999254},{"type":"v1","addr":"172.21.15.188:6837","nonce":1222999254}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2474016526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2474016526},{"type":"v1","addr":"172.21.15.7:6834","nonce":2474016526}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:33:24.950288+0000","modified":"2024-01-27T02:33:28.814599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5262":{"gid":5262,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/999728998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":999728998},{"type":"v1","addr":"172.21.15.7:6835","nonce":999728998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-27T02:34:01.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.526+0000 7fb9a7ddf700 1 -- 172.21.15.7:0/1318552067 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb984066f30 msgr2=0x7fb9840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:01.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.526+0000 7fb9a7ddf700 1 --2- 172.21.15.7:0/1318552067 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb984066f30 0x7fb9840693e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00bc50 tx=0x7fb99c01b000 comp rx=0 tx=0).stop 2024-01-27T02:34:01.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.526+0000 7fb9a7ddf700 1 -- 172.21.15.7:0/1318552067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9a00b1fd0 msgr2=0x7fb9a013dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:01.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.526+0000 7fb9a7ddf700 1 --2- 172.21.15.7:0/1318552067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9a00b1fd0 0x7fb9a013dd60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb99400c5b0 tx=0x7fb994001040 comp rx=0 tx=0).stop 2024-01-27T02:34:01.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.526+0000 7fb9a7ddf700 1 -- 172.21.15.7:0/1318552067 shutdown_connections 2024-01-27T02:34:01.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.526+0000 7fb9a7ddf700 1 --2- 172.21.15.7:0/1318552067 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb984066f30 0x7fb9840693e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.526+0000 7fb9a7ddf700 1 --2- 172.21.15.7:0/1318552067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9a015b5f0 0x7fb9a00a7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.526+0000 7fb9a7ddf700 1 --2- 172.21.15.7:0/1318552067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9a00b1c00 0x7fb9a00a9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.526+0000 7fb9a7ddf700 1 --2- 172.21.15.7:0/1318552067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9a00b1fd0 0x7fb9a013dd60 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.526+0000 7fb9a7ddf700 1 -- 172.21.15.7:0/1318552067 >> 172.21.15.7:0/1318552067 conn(0x7fb9a0068430 msgr2=0x7fb9a00b1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:01.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.527+0000 7fb9a7ddf700 1 -- 172.21.15.7:0/1318552067 shutdown_connections 2024-01-27T02:34:01.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.527+0000 7fb9a7ddf700 1 -- 172.21.15.7:0/1318552067 wait complete. 2024-01-27T02:34:01.528 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-01-27T02:34:01.540 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:34:01.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.671+0000 7f6c2b299700 1 Processor -- start 2024-01-27T02:34:01.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.671+0000 7f6c2b299700 1 -- start start 2024-01-27T02:34:01.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.672+0000 7f6c2b299700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2415c3d0 0x7f6c240b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:01.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.672+0000 7f6c2b299700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6c240710f0 0x7f6c240b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:01.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.672+0000 7f6c2b299700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2415ed60 con 0x7f6c2415c000 2024-01-27T02:34:01.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.672+0000 7f6c2b299700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c2415f040 con 0x7f6c240710f0 2024-01-27T02:34:01.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.672+0000 7f6c2b299700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2415f320 con 0x7f6c2415c3d0 2024-01-27T02:34:01.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.672+0000 7f6c29836700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6c240710f0 0x7f6c240b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:01.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.672+0000 7f6c29836700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6c240710f0 0x7f6c240b2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58016/0 (socket says 172.21.15.7:58016) 2024-01-27T02:34:01.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.672+0000 7f6c29836700 1 -- 172.21.15.7:0/2961580563 learned_addr learned my addr 172.21.15.7:0/2961580563 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:01.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.673+0000 7f6c28834700 1 --2- 172.21.15.7:0/2961580563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2415c3d0 0x7f6c240b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:01.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.673+0000 7f6c29836700 1 -- 172.21.15.7:0/2961580563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2415c3d0 msgr2=0x7f6c240b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:01.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.673+0000 7f6c29836700 1 --2- 172.21.15.7:0/2961580563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2415c3d0 0x7f6c240b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.673+0000 7f6c29836700 1 -- 172.21.15.7:0/2961580563 >> v1:172.21.15.188:6789/0 conn(0x7f6c2415c000 legacy=0x7f6c240b6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:01.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.673+0000 7f6c29836700 1 -- 172.21.15.7:0/2961580563 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2415f600 con 0x7f6c240710f0 2024-01-27T02:34:01.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.673+0000 7f6c28834700 1 --2- 172.21.15.7:0/2961580563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2415c3d0 0x7f6c240b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:01.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.673+0000 7f6c29836700 1 --2- 172.21.15.7:0/2961580563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c240710f0 0x7f6c240b2e70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009f20 tx=0x7f6c1c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad81d0e59583579 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:01.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.673+0000 7f6c17fff700 1 -- 172.21.15.7:0/2961580563 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c00e070 con 0x7f6c240710f0 2024-01-27T02:34:01.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.673+0000 7f6c17fff700 1 -- 172.21.15.7:0/2961580563 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c00ba80 con 0x7f6c240710f0 2024-01-27T02:34:01.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.673+0000 7f6c17fff700 1 -- 172.21.15.7:0/2961580563 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0092e0 con 0x7f6c240710f0 2024-01-27T02:34:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.674+0000 7f6c2b299700 1 -- 172.21.15.7:0/2961580563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c240710f0 msgr2=0x7f6c240b2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.674+0000 7f6c2b299700 1 --2- 172.21.15.7:0/2961580563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c240710f0 0x7f6c240b2e70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009f20 tx=0x7f6c1c009840 comp rx=0 tx=0).stop 2024-01-27T02:34:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.674+0000 7f6c2b299700 1 -- 172.21.15.7:0/2961580563 shutdown_connections 2024-01-27T02:34:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.674+0000 7f6c2b299700 1 --2- 172.21.15.7:0/2961580563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c240710f0 0x7f6c240b2e70 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.674+0000 7f6c2b299700 1 --2- 172.21.15.7:0/2961580563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2415c3d0 0x7f6c240b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.674+0000 7f6c2b299700 1 -- 172.21.15.7:0/2961580563 >> 172.21.15.7:0/2961580563 conn(0x7f6c24068480 msgr2=0x7f6c240709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.674+0000 7f6c2b299700 1 -- 172.21.15.7:0/2961580563 shutdown_connections 2024-01-27T02:34:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.674+0000 7f6c2b299700 1 -- 172.21.15.7:0/2961580563 wait complete. 2024-01-27T02:34:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.675+0000 7f6c2b299700 1 Processor -- start 2024-01-27T02:34:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.675+0000 7f6c2b299700 1 -- start start 2024-01-27T02:34:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.676+0000 7f6c2b299700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c240710f0 0x7f6c24159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:01.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.676+0000 7f6c2b299700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2415c000 0x7f6c241611b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.676+0000 7f6c2b299700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c2415c3d0 0x7f6c241582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.676+0000 7f6c2b299700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c24159920 con 0x7f6c240710f0 2024-01-27T02:34:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.676+0000 7f6c2b299700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c24158d10 con 0x7f6c2415c3d0 2024-01-27T02:34:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.676+0000 7f6c2b299700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c24158ff0 con 0x7f6c2415c000 2024-01-27T02:34:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.676+0000 7f6c29035700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c240710f0 0x7f6c24159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.676+0000 7f6c29035700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c240710f0 0x7f6c24159db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58020/0 (socket says 172.21.15.7:58020) 2024-01-27T02:34:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.676+0000 7f6c29035700 1 -- 172.21.15.7:0/526878531 learned_addr learned my addr 172.21.15.7:0/526878531 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.676+0000 7f6c29836700 1 --2- 172.21.15.7:0/526878531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c2415c3d0 0x7f6c241582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.676+0000 7f6c28834700 1 --2- 172.21.15.7:0/526878531 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2415c000 0x7f6c241611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.677+0000 7f6c29035700 1 -- 172.21.15.7:0/526878531 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2415c000 msgr2=0x7f6c241611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.677+0000 7f6c29035700 1 --2- 172.21.15.7:0/526878531 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2415c000 0x7f6c241611b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.677+0000 7f6c29035700 1 -- 172.21.15.7:0/526878531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c2415c3d0 msgr2=0x7f6c241582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:01.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.677+0000 7f6c29035700 1 --2- 172.21.15.7:0/526878531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c2415c3d0 0x7f6c241582c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.677+0000 7f6c29035700 1 -- 172.21.15.7:0/526878531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c009ae0 con 0x7f6c240710f0 2024-01-27T02:34:01.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.677+0000 7f6c29035700 1 --2- 172.21.15.7:0/526878531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c240710f0 0x7f6c24159db0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000c590 tx=0x7f6c2000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:01.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.677+0000 7f6c157fa700 1 -- 172.21.15.7:0/526878531 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20011590 con 0x7f6c240710f0 2024-01-27T02:34:01.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.677+0000 7f6c157fa700 1 -- 172.21.15.7:0/526878531 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c20011710 con 0x7f6c240710f0 2024-01-27T02:34:01.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.677+0000 7f6c2b299700 1 -- 172.21.15.7:0/526878531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c241592d0 con 0x7f6c240710f0 2024-01-27T02:34:01.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.677+0000 7f6c157fa700 1 -- 172.21.15.7:0/526878531 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20011ad0 con 0x7f6c240710f0 2024-01-27T02:34:01.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.677+0000 7f6c2b299700 1 -- 172.21.15.7:0/526878531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2415a2f0 con 0x7f6c240710f0 2024-01-27T02:34:01.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.678+0000 7f6c2b299700 1 -- 172.21.15.7:0/526878531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c24159c00 con 0x7f6c240710f0 2024-01-27T02:34:01.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.678+0000 7f6c157fa700 1 -- 172.21.15.7:0/526878531 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6c2001b3b0 con 0x7f6c240710f0 2024-01-27T02:34:01.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.679+0000 7f6c157fa700 1 --2- 172.21.15.7:0/526878531 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6c08066f30 0x7f6c080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:01.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.679+0000 7f6c157fa700 1 -- 172.21.15.7:0/526878531 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f6c20083a00 con 0x7f6c240710f0 2024-01-27T02:34:01.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.679+0000 7f6c28834700 1 --2- 172.21.15.7:0/526878531 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6c08066f30 0x7f6c080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:01.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.680+0000 7f6c28834700 1 --2- 172.21.15.7:0/526878531 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6c08066f30 0x7f6c080693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6c240d04a0 tx=0x7f6c18008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:01.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.683+0000 7f6c157fa700 1 -- 172.21.15.7:0/526878531 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c20050c30 con 0x7f6c240710f0 2024-01-27T02:34:01.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.933+0000 7f6c2b299700 1 -- 172.21.15.7:0/526878531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c2415f8e0 con 0x7f6c240710f0 2024-01-27T02:34:01.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.934+0000 7f6c157fa700 1 -- 172.21.15.7:0/526878531 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f6c20017070 con 0x7f6c240710f0 2024-01-27T02:34:01.935 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:34:01.935 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4883,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/198511042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":198511042},{"type":"v1","addr":"172.21.15.188:6836","nonce":198511042}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":4886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1222999254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1222999254},{"type":"v1","addr":"172.21.15.188:6837","nonce":1222999254}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2474016526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2474016526},{"type":"v1","addr":"172.21.15.7:6834","nonce":2474016526}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:33:24.950288+0000","modified":"2024-01-27T02:33:28.814599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5262":{"gid":5262,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/999728998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":999728998},{"type":"v1","addr":"172.21.15.7:6835","nonce":999728998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-27T02:34:01.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.936+0000 7f6c2b299700 1 -- 172.21.15.7:0/526878531 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6c08066f30 msgr2=0x7f6c080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:01.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.936+0000 7f6c2b299700 1 --2- 172.21.15.7:0/526878531 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6c08066f30 0x7f6c080693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6c240d04a0 tx=0x7f6c18008040 comp rx=0 tx=0).stop 2024-01-27T02:34:01.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.936+0000 7f6c2b299700 1 -- 172.21.15.7:0/526878531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c240710f0 msgr2=0x7f6c24159db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.936+0000 7f6c2b299700 1 --2- 172.21.15.7:0/526878531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c240710f0 0x7f6c24159db0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000c590 tx=0x7f6c2000c8a0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.937+0000 7f6c2b299700 1 -- 172.21.15.7:0/526878531 shutdown_connections 2024-01-27T02:34:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.937+0000 7f6c2b299700 1 --2- 172.21.15.7:0/526878531 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6c08066f30 0x7f6c080693e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.937+0000 7f6c2b299700 1 --2- 172.21.15.7:0/526878531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c2415c3d0 0x7f6c241582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.937+0000 7f6c2b299700 1 --2- 172.21.15.7:0/526878531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c240710f0 0x7f6c24159db0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.937+0000 7f6c2b299700 1 --2- 172.21.15.7:0/526878531 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2415c000 0x7f6c241611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.937+0000 7f6c2b299700 1 -- 172.21.15.7:0/526878531 >> 172.21.15.7:0/526878531 conn(0x7f6c24068480 msgr2=0x7f6c24160850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.937+0000 7f6c2b299700 1 -- 172.21.15.7:0/526878531 shutdown_connections 2024-01-27T02:34:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:01.937+0000 7f6c2b299700 1 -- 172.21.15.7:0/526878531 wait complete. 2024-01-27T02:34:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-01-27T02:34:01.950 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:34:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.082+0000 7f871c315700 1 Processor -- start 2024-01-27T02:34:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.082+0000 7f871c315700 1 -- start start 2024-01-27T02:34:02.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.083+0000 7f871c315700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8714143520 0x7f8714063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:02.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.083+0000 7f871c315700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f87141438f0 0x7f8714063ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:02.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.083+0000 7f871c315700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87140cb7a0 con 0x7f87140aa8b0 2024-01-27T02:34:02.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.083+0000 7f871c315700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87140cb9f0 con 0x7f8714143520 2024-01-27T02:34:02.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.083+0000 7f871c315700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87140cbcd0 con 0x7f87141438f0 2024-01-27T02:34:02.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.083+0000 7f871a0b1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8714143520 0x7f8714063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:02.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.084+0000 7f871a0b1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8714143520 0x7f8714063580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58030/0 (socket says 172.21.15.7:58030) 2024-01-27T02:34:02.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.084+0000 7f871a0b1700 1 -- 172.21.15.7:0/3475132564 learned_addr learned my addr 172.21.15.7:0/3475132564 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:02.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.084+0000 7f87198b0700 1 --2- 172.21.15.7:0/3475132564 >> v2:172.21.15.188:3300/0 conn(0x7f87141438f0 0x7f8714063ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:02.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.084+0000 7f871a0b1700 1 -- 172.21.15.7:0/3475132564 >> v2:172.21.15.188:3300/0 conn(0x7f87141438f0 msgr2=0x7f8714063ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:02.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.084+0000 7f871a0b1700 1 --2- 172.21.15.7:0/3475132564 >> v2:172.21.15.188:3300/0 conn(0x7f87141438f0 0x7f8714063ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.084+0000 7f871a0b1700 1 -- 172.21.15.7:0/3475132564 >> v1:172.21.15.188:6789/0 conn(0x7f87140aa8b0 legacy=0x7f87140c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:02.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.084+0000 7f871a0b1700 1 -- 172.21.15.7:0/3475132564 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87140cbfb0 con 0x7f8714143520 2024-01-27T02:34:02.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.084+0000 7f871a0b1700 1 --2- 172.21.15.7:0/3475132564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8714143520 0x7f8714063580 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f87140ca0e0 tx=0x7f8710009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0232fba6b9c03de server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:02.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.084+0000 7f87190af700 1 -- 172.21.15.7:0/3475132564 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8710005c10 con 0x7f8714143520 2024-01-27T02:34:02.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.084+0000 7f87190af700 1 -- 172.21.15.7:0/3475132564 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8710005d70 con 0x7f8714143520 2024-01-27T02:34:02.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.084+0000 7f87190af700 1 -- 172.21.15.7:0/3475132564 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8710006280 con 0x7f8714143520 2024-01-27T02:34:02.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.085+0000 7f871c315700 1 -- 172.21.15.7:0/3475132564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8714143520 msgr2=0x7f8714063580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:02.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.085+0000 7f871c315700 1 --2- 172.21.15.7:0/3475132564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8714143520 0x7f8714063580 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f87140ca0e0 tx=0x7f8710009220 comp rx=0 tx=0).stop 2024-01-27T02:34:02.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.085+0000 7f871c315700 1 -- 172.21.15.7:0/3475132564 shutdown_connections 2024-01-27T02:34:02.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.085+0000 7f871c315700 1 --2- 172.21.15.7:0/3475132564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8714143520 0x7f8714063580 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.085+0000 7f871c315700 1 --2- 172.21.15.7:0/3475132564 >> v2:172.21.15.188:3300/0 conn(0x7f87141438f0 0x7f8714063ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.085+0000 7f871c315700 1 -- 172.21.15.7:0/3475132564 >> 172.21.15.7:0/3475132564 conn(0x7f87140cdbd0 msgr2=0x7f87140cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:02.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.085+0000 7f871c315700 1 -- 172.21.15.7:0/3475132564 shutdown_connections 2024-01-27T02:34:02.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.085+0000 7f871c315700 1 -- 172.21.15.7:0/3475132564 wait complete. 2024-01-27T02:34:02.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.086+0000 7f871c315700 1 Processor -- start 2024-01-27T02:34:02.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.086+0000 7f871c315700 1 -- start start 2024-01-27T02:34:02.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.087+0000 7f871c315700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87140aa8b0 0x7f8714158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:02.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.087+0000 7f871c315700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8714143520 0x7f871415a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:02.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.087+0000 7f871c315700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87141438f0 0x7f871415ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:02.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.087+0000 7f871c315700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871415c020 con 0x7f87140aa8b0 2024-01-27T02:34:02.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.087+0000 7f871c315700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871416a870 con 0x7f8714143520 2024-01-27T02:34:02.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.087+0000 7f871c315700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87140afe80 con 0x7f87141438f0 2024-01-27T02:34:02.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.087+0000 7f871a0b1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87140aa8b0 0x7f8714158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:02.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.087+0000 7f871a0b1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87140aa8b0 0x7f8714158ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58036/0 (socket says 172.21.15.7:58036) 2024-01-27T02:34:02.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.087+0000 7f871a0b1700 1 -- 172.21.15.7:0/2963556180 learned_addr learned my addr 172.21.15.7:0/2963556180 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:02.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.087+0000 7f871a8b2700 1 --2- 172.21.15.7:0/2963556180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87141438f0 0x7f871415ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:02.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.087+0000 7f87198b0700 1 --2- 172.21.15.7:0/2963556180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8714143520 0x7f871415a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:02.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.087+0000 7f871a0b1700 1 -- 172.21.15.7:0/2963556180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87141438f0 msgr2=0x7f871415ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:02.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.087+0000 7f871a0b1700 1 --2- 172.21.15.7:0/2963556180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87141438f0 0x7f871415ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.087+0000 7f871a0b1700 1 -- 172.21.15.7:0/2963556180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8714143520 msgr2=0x7f871415a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:02.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.087+0000 7f871a0b1700 1 --2- 172.21.15.7:0/2963556180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8714143520 0x7f871415a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.087+0000 7f871a0b1700 1 -- 172.21.15.7:0/2963556180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8710009480 con 0x7f87140aa8b0 2024-01-27T02:34:02.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.087+0000 7f87198b0700 1 --2- 172.21.15.7:0/2963556180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8714143520 0x7f871415a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:34:02.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.088+0000 7f871a0b1700 1 --2- 172.21.15.7:0/2963556180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87140aa8b0 0x7f8714158ea0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f87140c49d0 tx=0x7f8710016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:02.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.088+0000 7f87067fc700 1 -- 172.21.15.7:0/2963556180 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871000b4a0 con 0x7f87140aa8b0 2024-01-27T02:34:02.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.088+0000 7f87067fc700 1 -- 172.21.15.7:0/2963556180 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871000b620 con 0x7f87140aa8b0 2024-01-27T02:34:02.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.088+0000 7f871c315700 1 -- 172.21.15.7:0/2963556180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f871415b1b0 con 0x7f87140aa8b0 2024-01-27T02:34:02.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.088+0000 7f87067fc700 1 -- 172.21.15.7:0/2963556180 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871000ba20 con 0x7f87140aa8b0 2024-01-27T02:34:02.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.088+0000 7f871c315700 1 -- 172.21.15.7:0/2963556180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871415b9d0 con 0x7f87140aa8b0 2024-01-27T02:34:02.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.089+0000 7f871c315700 1 -- 172.21.15.7:0/2963556180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f871404c710 con 0x7f87140aa8b0 2024-01-27T02:34:02.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.089+0000 7f87067fc700 1 -- 172.21.15.7:0/2963556180 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f871001a420 con 0x7f87140aa8b0 2024-01-27T02:34:02.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.090+0000 7f87067fc700 1 --2- 172.21.15.7:0/2963556180 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f86f8066f30 0x7f86f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:02.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.090+0000 7f87067fc700 1 -- 172.21.15.7:0/2963556180 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f87100859b0 con 0x7f87140aa8b0 2024-01-27T02:34:02.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.090+0000 7f87198b0700 1 --2- 172.21.15.7:0/2963556180 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f86f8066f30 0x7f86f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:02.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.091+0000 7f87198b0700 1 --2- 172.21.15.7:0/2963556180 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f86f8066f30 0x7f86f80693e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8708006f00 tx=0x7f8708008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:02.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.094+0000 7f87067fc700 1 -- 172.21.15.7:0/2963556180 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8710052c90 con 0x7f87140aa8b0 2024-01-27T02:34:02.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.299+0000 7f871c315700 1 -- 172.21.15.7:0/2963556180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87140cc290 con 0x7f87140aa8b0 2024-01-27T02:34:02.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.300+0000 7f87067fc700 1 -- 172.21.15.7:0/2963556180 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f8710029070 con 0x7f87140aa8b0 2024-01-27T02:34:02.301 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:34:02.301 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":53,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:33:56.965932+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:33:23.445643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-27T02:33:23.921995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.30","mappings":[{"from":0,"to":4}]},{"pgid":"7.35","mappings":[{"from":0,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:34:02.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.303+0000 7f871c315700 1 -- 172.21.15.7:0/2963556180 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f86f8066f30 msgr2=0x7f86f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:02.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.303+0000 7f871c315700 1 --2- 172.21.15.7:0/2963556180 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f86f8066f30 0x7f86f80693e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8708006f00 tx=0x7f8708008040 comp rx=0 tx=0).stop 2024-01-27T02:34:02.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.303+0000 7f871c315700 1 -- 172.21.15.7:0/2963556180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87140aa8b0 msgr2=0x7f8714158ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:02.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.303+0000 7f871c315700 1 --2- 172.21.15.7:0/2963556180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87140aa8b0 0x7f8714158ea0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f87140c49d0 tx=0x7f8710016000 comp rx=0 tx=0).stop 2024-01-27T02:34:02.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.303+0000 7f871c315700 1 -- 172.21.15.7:0/2963556180 shutdown_connections 2024-01-27T02:34:02.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.303+0000 7f871c315700 1 --2- 172.21.15.7:0/2963556180 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f86f8066f30 0x7f86f80693e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.303+0000 7f871c315700 1 --2- 172.21.15.7:0/2963556180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87141438f0 0x7f871415ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.303+0000 7f871c315700 1 --2- 172.21.15.7:0/2963556180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87140aa8b0 0x7f8714158ea0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.303+0000 7f871c315700 1 --2- 172.21.15.7:0/2963556180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8714143520 0x7f871415a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.303+0000 7f871c315700 1 -- 172.21.15.7:0/2963556180 >> 172.21.15.7:0/2963556180 conn(0x7f87140cdbd0 msgr2=0x7f871406ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:02.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.303+0000 7f871c315700 1 -- 172.21.15.7:0/2963556180 shutdown_connections 2024-01-27T02:34:02.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.303+0000 7f871c315700 1 -- 172.21.15.7:0/2963556180 wait complete. 2024-01-27T02:34:02.317 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-27T02:34:02.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.449+0000 7ff49869e700 1 Processor -- start 2024-01-27T02:34:02.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.449+0000 7ff49869e700 1 -- start start 2024-01-27T02:34:02.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.450+0000 7ff49869e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4900b2f80 0x7ff49006a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:02.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.450+0000 7ff49869e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4901553e0 0x7ff490155830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:02.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.450+0000 7ff49869e700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49006a900 con 0x7ff4900b3350 2024-01-27T02:34:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.450+0000 7ff49869e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff490159390 con 0x7ff4900b2f80 2024-01-27T02:34:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.450+0000 7ff49869e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff490158780 con 0x7ff4901553e0 2024-01-27T02:34:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.450+0000 7ff49643a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4900b2f80 0x7ff49006a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.450+0000 7ff49643a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4900b2f80 0x7ff49006a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58050/0 (socket says 172.21.15.7:58050) 2024-01-27T02:34:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.450+0000 7ff49643a700 1 -- 172.21.15.7:0/50578038 learned_addr learned my addr 172.21.15.7:0/50578038 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.451+0000 7ff496c3b700 1 --2- 172.21.15.7:0/50578038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4901553e0 0x7ff490155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.451+0000 7ff49643a700 1 -- 172.21.15.7:0/50578038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4901553e0 msgr2=0x7ff490155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.451+0000 7ff49643a700 1 --2- 172.21.15.7:0/50578038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4901553e0 0x7ff490155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.451+0000 7ff49643a700 1 -- 172.21.15.7:0/50578038 >> v1:172.21.15.188:6789/0 conn(0x7ff4900b3350 legacy=0x7ff4900ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.451+0000 7ff49643a700 1 -- 172.21.15.7:0/50578038 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff490158a60 con 0x7ff4900b2f80 2024-01-27T02:34:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.451+0000 7ff49643a700 1 --2- 172.21.15.7:0/50578038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4900b2f80 0x7ff49006a3c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff4800099b0 tx=0x7ff480009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39553a262f6540bd server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.451+0000 7ff495438700 1 -- 172.21.15.7:0/50578038 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480009ea0 con 0x7ff4900b2f80 2024-01-27T02:34:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.451+0000 7ff495438700 1 -- 172.21.15.7:0/50578038 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48000b860 con 0x7ff4900b2f80 2024-01-27T02:34:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.451+0000 7ff495438700 1 -- 172.21.15.7:0/50578038 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48000bc00 con 0x7ff4900b2f80 2024-01-27T02:34:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.452+0000 7ff49869e700 1 -- 172.21.15.7:0/50578038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4900b2f80 msgr2=0x7ff49006a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.452+0000 7ff49869e700 1 --2- 172.21.15.7:0/50578038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4900b2f80 0x7ff49006a3c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff4800099b0 tx=0x7ff480009250 comp rx=0 tx=0).stop 2024-01-27T02:34:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.452+0000 7ff49869e700 1 -- 172.21.15.7:0/50578038 shutdown_connections 2024-01-27T02:34:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.452+0000 7ff49869e700 1 --2- 172.21.15.7:0/50578038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4901553e0 0x7ff490155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.452+0000 7ff49869e700 1 --2- 172.21.15.7:0/50578038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4900b2f80 0x7ff49006a3c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.452+0000 7ff49869e700 1 -- 172.21.15.7:0/50578038 >> 172.21.15.7:0/50578038 conn(0x7ff490068430 msgr2=0x7ff49006be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.452+0000 7ff49869e700 1 -- 172.21.15.7:0/50578038 shutdown_connections 2024-01-27T02:34:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.452+0000 7ff49869e700 1 -- 172.21.15.7:0/50578038 wait complete. 2024-01-27T02:34:02.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.453+0000 7ff49869e700 1 Processor -- start 2024-01-27T02:34:02.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.453+0000 7ff49869e700 1 -- start start 2024-01-27T02:34:02.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.454+0000 7ff49869e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4900b2f80 0x7ff4901457e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:02.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.454+0000 7ff49869e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4900b3350 0x7ff4900c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:02.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.454+0000 7ff49869e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4901553e0 0x7ff4900c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:02.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.454+0000 7ff49869e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49015d020 con 0x7ff4900b2f80 2024-01-27T02:34:02.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.454+0000 7ff49869e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4900c35f0 con 0x7ff4900b3350 2024-01-27T02:34:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.454+0000 7ff49869e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4900c29e0 con 0x7ff4901553e0 2024-01-27T02:34:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.454+0000 7ff49643a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4900b2f80 0x7ff4901457e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.454+0000 7ff49643a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4900b2f80 0x7ff4901457e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58062/0 (socket says 172.21.15.7:58062) 2024-01-27T02:34:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.454+0000 7ff49643a700 1 -- 172.21.15.7:0/258789231 learned_addr learned my addr 172.21.15.7:0/258789231 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.454+0000 7ff49643a700 1 -- 172.21.15.7:0/258789231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4901553e0 msgr2=0x7ff4900c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.454+0000 7ff49643a700 1 --2- 172.21.15.7:0/258789231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4901553e0 0x7ff4900c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.454+0000 7ff495c39700 1 --2- 172.21.15.7:0/258789231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4900b3350 0x7ff4900c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.454+0000 7ff49643a700 1 -- 172.21.15.7:0/258789231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4900b3350 msgr2=0x7ff4900c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.454+0000 7ff49643a700 1 --2- 172.21.15.7:0/258789231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4900b3350 0x7ff4900c7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.454+0000 7ff49643a700 1 -- 172.21.15.7:0/258789231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4800094f0 con 0x7ff4900b2f80 2024-01-27T02:34:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.455+0000 7ff49643a700 1 --2- 172.21.15.7:0/258789231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4900b2f80 0x7ff4901457e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff49006def0 tx=0x7ff480017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.455+0000 7ff4867fc700 1 -- 172.21.15.7:0/258789231 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48000bbe0 con 0x7ff4900b2f80 2024-01-27T02:34:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.455+0000 7ff4867fc700 1 -- 172.21.15.7:0/258789231 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff480011420 con 0x7ff4900b2f80 2024-01-27T02:34:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.455+0000 7ff49869e700 1 -- 172.21.15.7:0/258789231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4900c2cc0 con 0x7ff4900b2f80 2024-01-27T02:34:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.455+0000 7ff4867fc700 1 -- 172.21.15.7:0/258789231 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4800117c0 con 0x7ff4900b2f80 2024-01-27T02:34:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.455+0000 7ff49869e700 1 -- 172.21.15.7:0/258789231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4900c3280 con 0x7ff4900b2f80 2024-01-27T02:34:02.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.456+0000 7ff49869e700 1 -- 172.21.15.7:0/258789231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff49004c710 con 0x7ff4900b2f80 2024-01-27T02:34:02.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.456+0000 7ff4867fc700 1 -- 172.21.15.7:0/258789231 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff480011bf0 con 0x7ff4900b2f80 2024-01-27T02:34:02.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.457+0000 7ff4867fc700 1 --2- 172.21.15.7:0/258789231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff474066f30 0x7ff4740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:02.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.457+0000 7ff4867fc700 1 -- 172.21.15.7:0/258789231 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7ff480085ed0 con 0x7ff4900b2f80 2024-01-27T02:34:02.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.457+0000 7ff495c39700 1 --2- 172.21.15.7:0/258789231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff474066f30 0x7ff4740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:02.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.458+0000 7ff495c39700 1 --2- 172.21.15.7:0/258789231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff474066f30 0x7ff4740693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff4901577a0 tx=0x7ff48c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.461+0000 7ff4867fc700 1 -- 172.21.15.7:0/258789231 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff480053100 con 0x7ff4900b2f80 2024-01-27T02:34:02.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.712+0000 7ff49869e700 1 -- 172.21.15.7:0/258789231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff490158d40 con 0x7ff4900b2f80 2024-01-27T02:34:02.713 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-27T02:34:02.712+0000 7fb5c503a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-27T02:34:02.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.717+0000 7ff4867fc700 1 -- 172.21.15.7:0/258789231 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff480056720 con 0x7ff4900b2f80 2024-01-27T02:34:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.719+0000 7ff49869e700 1 -- 172.21.15.7:0/258789231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff474066f30 msgr2=0x7ff4740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.719+0000 7ff49869e700 1 --2- 172.21.15.7:0/258789231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff474066f30 0x7ff4740693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff4901577a0 tx=0x7ff48c009040 comp rx=0 tx=0).stop 2024-01-27T02:34:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.719+0000 7ff49869e700 1 -- 172.21.15.7:0/258789231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4900b2f80 msgr2=0x7ff4901457e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.719+0000 7ff49869e700 1 --2- 172.21.15.7:0/258789231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4900b2f80 0x7ff4901457e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff49006def0 tx=0x7ff480017000 comp rx=0 tx=0).stop 2024-01-27T02:34:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.719+0000 7ff49869e700 1 -- 172.21.15.7:0/258789231 shutdown_connections 2024-01-27T02:34:02.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.719+0000 7ff49869e700 1 --2- 172.21.15.7:0/258789231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff474066f30 0x7ff4740693e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.719+0000 7ff49869e700 1 --2- 172.21.15.7:0/258789231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4901553e0 0x7ff4900c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.719+0000 7ff49869e700 1 --2- 172.21.15.7:0/258789231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4900b2f80 0x7ff4901457e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.719+0000 7ff49869e700 1 --2- 172.21.15.7:0/258789231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4900b3350 0x7ff4900c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.719+0000 7ff49869e700 1 -- 172.21.15.7:0/258789231 >> 172.21.15.7:0/258789231 conn(0x7ff490068430 msgr2=0x7ff4900ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:02.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.719+0000 7ff49869e700 1 -- 172.21.15.7:0/258789231 shutdown_connections 2024-01-27T02:34:02.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.719+0000 7ff49869e700 1 -- 172.21.15.7:0/258789231 wait complete. 2024-01-27T02:34:02.721 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-27T02:34:02.722 INFO:tasks.ceph.mds.c.smithi007.stderr: -9196> 2024-01-27T02:33:31.485+0000 7fc79e073700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-27T02:34:02.734 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-27T02:34:02.754 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-27T02:34:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.867+0000 7f21d5c94700 1 Processor -- start 2024-01-27T02:34:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.867+0000 7f21d5c94700 1 -- start start 2024-01-27T02:34:02.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.868+0000 7f21d5c94700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f21d0156b60 0x7f21d00ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:02.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.868+0000 7f21d5c94700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f21d0156f30 0x7f21d015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:02.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.868+0000 7f21d5c94700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d00aeaa0 con 0x7f21d00ad9f0 2024-01-27T02:34:02.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.868+0000 7f21d5c94700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d00ade90 con 0x7f21d0156f30 2024-01-27T02:34:02.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.868+0000 7f21d5c94700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d00ae170 con 0x7f21d0156b60 2024-01-27T02:34:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.868+0000 7f21cffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f21d0156f30 0x7f21d015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.868+0000 7f21cffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f21d0156f30 0x7f21d015a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58090/0 (socket says 172.21.15.7:58090) 2024-01-27T02:34:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.868+0000 7f21cffff700 1 -- 172.21.15.7:0/951800578 learned_addr learned my addr 172.21.15.7:0/951800578 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.868+0000 7f21cf7fe700 1 --2- 172.21.15.7:0/951800578 >> v2:172.21.15.188:3300/0 conn(0x7f21d0156b60 0x7f21d00ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.869+0000 7f21cffff700 1 -- 172.21.15.7:0/951800578 >> v2:172.21.15.188:3300/0 conn(0x7f21d0156b60 msgr2=0x7f21d00ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.869+0000 7f21cffff700 1 --2- 172.21.15.7:0/951800578 >> v2:172.21.15.188:3300/0 conn(0x7f21d0156b60 0x7f21d00ad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.869+0000 7f21cffff700 1 -- 172.21.15.7:0/951800578 >> v1:172.21.15.188:6789/0 conn(0x7f21d00ad9f0 legacy=0x7f21d00aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.869+0000 7f21cffff700 1 -- 172.21.15.7:0/951800578 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d00ae450 con 0x7f21d0156f30 2024-01-27T02:34:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.869+0000 7f21cffff700 1 --2- 172.21.15.7:0/951800578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d0156f30 0x7f21d015a9b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f21d0064f00 tx=0x7f21c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aae93f30c2c7a43 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.869+0000 7f21ce7fc700 1 -- 172.21.15.7:0/951800578 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c800e070 con 0x7f21d0156f30 2024-01-27T02:34:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.869+0000 7f21ce7fc700 1 -- 172.21.15.7:0/951800578 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c800ba50 con 0x7f21d0156f30 2024-01-27T02:34:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.869+0000 7f21ce7fc700 1 -- 172.21.15.7:0/951800578 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c80092e0 con 0x7f21d0156f30 2024-01-27T02:34:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.870+0000 7f21d5c94700 1 -- 172.21.15.7:0/951800578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d0156f30 msgr2=0x7f21d015a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.870+0000 7f21d5c94700 1 --2- 172.21.15.7:0/951800578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d0156f30 0x7f21d015a9b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f21d0064f00 tx=0x7f21c8009840 comp rx=0 tx=0).stop 2024-01-27T02:34:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.870+0000 7f21d5c94700 1 -- 172.21.15.7:0/951800578 shutdown_connections 2024-01-27T02:34:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.870+0000 7f21d5c94700 1 --2- 172.21.15.7:0/951800578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d0156f30 0x7f21d015a9b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.870+0000 7f21d5c94700 1 --2- 172.21.15.7:0/951800578 >> v2:172.21.15.188:3300/0 conn(0x7f21d0156b60 0x7f21d00ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.870+0000 7f21d5c94700 1 -- 172.21.15.7:0/951800578 >> 172.21.15.7:0/951800578 conn(0x7f21d0068480 msgr2=0x7f21d00cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.870+0000 7f21d5c94700 1 -- 172.21.15.7:0/951800578 shutdown_connections 2024-01-27T02:34:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.870+0000 7f21d5c94700 1 -- 172.21.15.7:0/951800578 wait complete. 2024-01-27T02:34:02.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.871+0000 7f21d5c94700 1 Processor -- start 2024-01-27T02:34:02.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.871+0000 7f21d5c94700 1 -- start start 2024-01-27T02:34:02.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.872+0000 7f21d5c94700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d00ad9f0 0x7f21d01509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:02.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.872+0000 7f21d5c94700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d0156b60 0x7f21d00cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:02.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.872+0000 7f21d5c94700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d0156f30 0x7f21d01682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.872+0000 7f21d5c94700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d0150f20 con 0x7f21d00ad9f0 2024-01-27T02:34:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.872+0000 7f21d5c94700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d00abc60 con 0x7f21d0156b60 2024-01-27T02:34:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.872+0000 7f21d5c94700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d00abf40 con 0x7f21d0156f30 2024-01-27T02:34:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.872+0000 7f21cf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d00ad9f0 0x7f21d01509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.872+0000 7f21cf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d00ad9f0 0x7f21d01509e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58094/0 (socket says 172.21.15.7:58094) 2024-01-27T02:34:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.872+0000 7f21cf7fe700 1 -- 172.21.15.7:0/2535520016 learned_addr learned my addr 172.21.15.7:0/2535520016 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.872+0000 7f21ceffd700 1 --2- 172.21.15.7:0/2535520016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d0156b60 0x7f21d00cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.872+0000 7f21cffff700 1 --2- 172.21.15.7:0/2535520016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d0156f30 0x7f21d01682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.872+0000 7f21cf7fe700 1 -- 172.21.15.7:0/2535520016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d0156f30 msgr2=0x7f21d01682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.872+0000 7f21cf7fe700 1 --2- 172.21.15.7:0/2535520016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d0156f30 0x7f21d01682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.872+0000 7f21cf7fe700 1 -- 172.21.15.7:0/2535520016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d0156b60 msgr2=0x7f21d00cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.872+0000 7f21cf7fe700 1 --2- 172.21.15.7:0/2535520016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d0156b60 0x7f21d00cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:02.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.872+0000 7f21cf7fe700 1 -- 172.21.15.7:0/2535520016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c8009ae0 con 0x7f21d00ad9f0 2024-01-27T02:34:02.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.872+0000 7f21ceffd700 1 --2- 172.21.15.7:0/2535520016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d0156b60 0x7f21d00cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:02.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.873+0000 7f21cf7fe700 1 --2- 172.21.15.7:0/2535520016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d00ad9f0 0x7f21d01509e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f21c000d890 tx=0x7f21c0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:02.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.873+0000 7f21c77fe700 1 -- 172.21.15.7:0/2535520016 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c000b600 con 0x7f21d00ad9f0 2024-01-27T02:34:02.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.873+0000 7f21c77fe700 1 -- 172.21.15.7:0/2535520016 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c000b760 con 0x7f21d00ad9f0 2024-01-27T02:34:02.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.873+0000 7f21d5c94700 1 -- 172.21.15.7:0/2535520016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d00ac220 con 0x7f21d00ad9f0 2024-01-27T02:34:02.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.873+0000 7f21c77fe700 1 -- 172.21.15.7:0/2535520016 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c000bae0 con 0x7f21d00ad9f0 2024-01-27T02:34:02.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.873+0000 7f21d5c94700 1 -- 172.21.15.7:0/2535520016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d00ac7e0 con 0x7f21d00ad9f0 2024-01-27T02:34:02.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.874+0000 7f21d5c94700 1 -- 172.21.15.7:0/2535520016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d0072060 con 0x7f21d00ad9f0 2024-01-27T02:34:02.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.874+0000 7f21c77fe700 1 -- 172.21.15.7:0/2535520016 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f21c001e460 con 0x7f21d00ad9f0 2024-01-27T02:34:02.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.875+0000 7f21c77fe700 1 --2- 172.21.15.7:0/2535520016 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f21b0066f30 0x7f21b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:02.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.875+0000 7f21c77fe700 1 -- 172.21.15.7:0/2535520016 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f21c0084d10 con 0x7f21d00ad9f0 2024-01-27T02:34:02.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.875+0000 7f21ceffd700 1 --2- 172.21.15.7:0/2535520016 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f21b0066f30 0x7f21b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:02.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.876+0000 7f21ceffd700 1 --2- 172.21.15.7:0/2535520016 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f21b0066f30 0x7f21b00693e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f21b8007fd0 tx=0x7f21b8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:02.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:02.879+0000 7f21c77fe700 1 -- 172.21.15.7:0/2535520016 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f21c008a020 con 0x7f21d00ad9f0 2024-01-27T02:34:03.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.133+0000 7f21d5c94700 1 -- 172.21.15.7:0/2535520016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f21d00acb50 con 0x7f21d00ad9f0 2024-01-27T02:34:03.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.718+0000 7f21c77fe700 1 -- 172.21.15.7:0/2535520016 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f21c0051e50 con 0x7f21d00ad9f0 2024-01-27T02:34:03.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.721+0000 7f21d5c94700 1 -- 172.21.15.7:0/2535520016 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f21b0066f30 msgr2=0x7f21b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:03.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.721+0000 7f21d5c94700 1 --2- 172.21.15.7:0/2535520016 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f21b0066f30 0x7f21b00693e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f21b8007fd0 tx=0x7f21b8009040 comp rx=0 tx=0).stop 2024-01-27T02:34:03.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.721+0000 7f21d5c94700 1 -- 172.21.15.7:0/2535520016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d00ad9f0 msgr2=0x7f21d01509e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:03.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.721+0000 7f21d5c94700 1 --2- 172.21.15.7:0/2535520016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d00ad9f0 0x7f21d01509e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f21c000d890 tx=0x7f21c0008040 comp rx=0 tx=0).stop 2024-01-27T02:34:03.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.721+0000 7f21d5c94700 1 -- 172.21.15.7:0/2535520016 shutdown_connections 2024-01-27T02:34:03.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.721+0000 7f21d5c94700 1 --2- 172.21.15.7:0/2535520016 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f21b0066f30 0x7f21b00693e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:03.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.721+0000 7f21d5c94700 1 --2- 172.21.15.7:0/2535520016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d0156f30 0x7f21d01682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:03.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.721+0000 7f21d5c94700 1 --2- 172.21.15.7:0/2535520016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21d00ad9f0 0x7f21d01509e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:03.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.721+0000 7f21d5c94700 1 --2- 172.21.15.7:0/2535520016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d0156b60 0x7f21d00cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:03.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.721+0000 7f21d5c94700 1 -- 172.21.15.7:0/2535520016 >> 172.21.15.7:0/2535520016 conn(0x7f21d0068480 msgr2=0x7f21d00a7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:03.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.721+0000 7f21d5c94700 1 -- 172.21.15.7:0/2535520016 shutdown_connections 2024-01-27T02:34:03.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.721+0000 7f21d5c94700 1 -- 172.21.15.7:0/2535520016 wait complete. 2024-01-27T02:34:03.737 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-27T02:34:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.920+0000 7f40f056e700 1 Processor -- start 2024-01-27T02:34:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.921+0000 7f40f056e700 1 -- start start 2024-01-27T02:34:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.922+0000 7f40f056e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f40e8157f40 0x7f40e80cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.922+0000 7f40f056e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40e80cdb70 0x7f40e80cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.922+0000 7f40f056e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40e80cbc00 0x7f40e8158310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.922+0000 7f40f056e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e80ca670 con 0x7f40e80cdb70 2024-01-27T02:34:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.922+0000 7f40f056e700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e80cdfc0 con 0x7f40e8157f40 2024-01-27T02:34:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.922+0000 7f40f056e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e80cb470 con 0x7f40e80cbc00 2024-01-27T02:34:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.922+0000 7f40edb09700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40e80cdb70 0x7f40e80cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.922+0000 7f40edb09700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40e80cdb70 0x7f40e80cb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58098/0 (socket says 172.21.15.7:58098) 2024-01-27T02:34:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.922+0000 7f40edb09700 1 -- 172.21.15.7:0/2504937116 learned_addr learned my addr 172.21.15.7:0/2504937116 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.923+0000 7f40ee30a700 1 --2- 172.21.15.7:0/2504937116 >> v2:172.21.15.188:3300/0 conn(0x7f40e8157f40 0x7f40e80cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.923+0000 7f40eeb0b700 1 --2- 172.21.15.7:0/2504937116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40e80cbc00 0x7f40e8158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.923+0000 7f40edb09700 1 -- 172.21.15.7:0/2504937116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40e80cbc00 msgr2=0x7f40e8158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.923+0000 7f40edb09700 1 --2- 172.21.15.7:0/2504937116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40e80cbc00 0x7f40e8158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.923+0000 7f40edb09700 1 -- 172.21.15.7:0/2504937116 >> v2:172.21.15.188:3300/0 conn(0x7f40e8157f40 msgr2=0x7f40e80cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.923+0000 7f40edb09700 1 --2- 172.21.15.7:0/2504937116 >> v2:172.21.15.188:3300/0 conn(0x7f40e8157f40 0x7f40e80cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.923+0000 7f40edb09700 1 -- 172.21.15.7:0/2504937116 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e80cc080 con 0x7f40e80cdb70 2024-01-27T02:34:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.923+0000 7f40eeb0b700 1 --2- 172.21.15.7:0/2504937116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40e80cbc00 0x7f40e8158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:34:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.923+0000 7f40ee30a700 1 --2- 172.21.15.7:0/2504937116 >> v2:172.21.15.188:3300/0 conn(0x7f40e8157f40 0x7f40e80cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:34:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.923+0000 7f40edb09700 1 --2- 172.21.15.7:0/2504937116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e80cdb70 0x7f40e80cb6c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f40e8159650 tx=0x7f40dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78601e99fbada07 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.923+0000 7f40ed308700 1 -- 172.21.15.7:0/2504937116 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc00b860 con 0x7f40e80cdb70 2024-01-27T02:34:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.923+0000 7f40ed308700 1 -- 172.21.15.7:0/2504937116 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40dc00b9c0 con 0x7f40e80cdb70 2024-01-27T02:34:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.923+0000 7f40ed308700 1 -- 172.21.15.7:0/2504937116 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc00bd60 con 0x7f40e80cdb70 2024-01-27T02:34:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.924+0000 7f40f056e700 1 -- 172.21.15.7:0/2504937116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e80cdb70 msgr2=0x7f40e80cb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.924+0000 7f40f056e700 1 --2- 172.21.15.7:0/2504937116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e80cdb70 0x7f40e80cb6c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f40e8159650 tx=0x7f40dc009250 comp rx=0 tx=0).stop 2024-01-27T02:34:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.924+0000 7f40f056e700 1 -- 172.21.15.7:0/2504937116 shutdown_connections 2024-01-27T02:34:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.924+0000 7f40f056e700 1 --2- 172.21.15.7:0/2504937116 >> v2:172.21.15.188:3300/0 conn(0x7f40e8157f40 0x7f40e80cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.924+0000 7f40f056e700 1 --2- 172.21.15.7:0/2504937116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40e80cbc00 0x7f40e8158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.924+0000 7f40f056e700 1 --2- 172.21.15.7:0/2504937116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e80cdb70 0x7f40e80cb6c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.924+0000 7f40f056e700 1 -- 172.21.15.7:0/2504937116 >> 172.21.15.7:0/2504937116 conn(0x7f40e8098020 msgr2=0x7f40e80711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.924+0000 7f40f056e700 1 -- 172.21.15.7:0/2504937116 shutdown_connections 2024-01-27T02:34:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.924+0000 7f40f056e700 1 -- 172.21.15.7:0/2504937116 wait complete. 2024-01-27T02:34:03.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.925+0000 7f40f056e700 1 Processor -- start 2024-01-27T02:34:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.925+0000 7f40f056e700 1 -- start start 2024-01-27T02:34:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.926+0000 7f40f056e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40e80cbc00 0x7f40e8068410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.926+0000 7f40f056e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e80cdb70 0x7f40e8156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.926+0000 7f40f056e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40e8157f40 0x7f40e80b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.926+0000 7f40f056e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e8156300 con 0x7f40e80cdb70 2024-01-27T02:34:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.926+0000 7f40f056e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e81556f0 con 0x7f40e8157f40 2024-01-27T02:34:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.926+0000 7f40f056e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e81559d0 con 0x7f40e80cbc00 2024-01-27T02:34:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.926+0000 7f40edb09700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e80cdb70 0x7f40e8156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.926+0000 7f40edb09700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e80cdb70 0x7f40e8156790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58110/0 (socket says 172.21.15.7:58110) 2024-01-27T02:34:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.926+0000 7f40edb09700 1 -- 172.21.15.7:0/2891933249 learned_addr learned my addr 172.21.15.7:0/2891933249 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.926+0000 7f40eeb0b700 1 --2- 172.21.15.7:0/2891933249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40e8157f40 0x7f40e80b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.926+0000 7f40ee30a700 1 --2- 172.21.15.7:0/2891933249 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40e80cbc00 0x7f40e8068410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.926+0000 7f40edb09700 1 -- 172.21.15.7:0/2891933249 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40e80cbc00 msgr2=0x7f40e8068410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.926+0000 7f40edb09700 1 --2- 172.21.15.7:0/2891933249 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40e80cbc00 0x7f40e8068410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.926+0000 7f40edb09700 1 -- 172.21.15.7:0/2891933249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40e8157f40 msgr2=0x7f40e80b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.926+0000 7f40edb09700 1 --2- 172.21.15.7:0/2891933249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40e8157f40 0x7f40e80b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.926+0000 7f40edb09700 1 -- 172.21.15.7:0/2891933249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40dc0094f0 con 0x7f40e80cdb70 2024-01-27T02:34:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.926+0000 7f40edb09700 1 --2- 172.21.15.7:0/2891933249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e80cdb70 0x7f40e8156790 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f40e806ea80 tx=0x7f40dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.927+0000 7f40da7fc700 1 -- 172.21.15.7:0/2891933249 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc024440 con 0x7f40e80cdb70 2024-01-27T02:34:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.927+0000 7f40da7fc700 1 -- 172.21.15.7:0/2891933249 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40dc0245a0 con 0x7f40e80cdb70 2024-01-27T02:34:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.927+0000 7f40f056e700 1 -- 172.21.15.7:0/2891933249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e8155cb0 con 0x7f40e80cdb70 2024-01-27T02:34:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.927+0000 7f40da7fc700 1 -- 172.21.15.7:0/2891933249 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc024960 con 0x7f40e80cdb70 2024-01-27T02:34:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.927+0000 7f40f056e700 1 -- 172.21.15.7:0/2891933249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e8154a50 con 0x7f40e80cdb70 2024-01-27T02:34:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.928+0000 7f40f056e700 1 -- 172.21.15.7:0/2891933249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e8156470 con 0x7f40e80cdb70 2024-01-27T02:34:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.928+0000 7f40da7fc700 1 -- 172.21.15.7:0/2891933249 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f40dc01a420 con 0x7f40e80cdb70 2024-01-27T02:34:03.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.928+0000 7f40da7fc700 1 --2- 172.21.15.7:0/2891933249 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f40cc066f60 0x7f40cc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:03.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.929+0000 7f40da7fc700 1 -- 172.21.15.7:0/2891933249 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f40dc0863c0 con 0x7f40e80cdb70 2024-01-27T02:34:03.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.929+0000 7f40ee30a700 1 --2- 172.21.15.7:0/2891933249 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f40cc066f60 0x7f40cc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:03.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.929+0000 7f40ee30a700 1 --2- 172.21.15.7:0/2891933249 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f40cc066f60 0x7f40cc069410 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f40e8063b50 tx=0x7f40e4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:03.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:03.933+0000 7f40da7fc700 1 -- 172.21.15.7:0/2891933249 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40dc053620 con 0x7f40e80cdb70 2024-01-27T02:34:04.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:04.142+0000 7f40f056e700 1 -- 172.21.15.7:0/2891933249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f40e804c710 con 0x7f40e80cdb70 2024-01-27T02:34:05.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.002+0000 7f40da7fc700 1 -- 172.21.15.7:0/2891933249 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v55) v1 ==== 181+0+0 (secure 0 0 0) 0x7f40dc056c40 con 0x7f40e80cdb70 2024-01-27T02:34:05.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.007+0000 7f40f056e700 1 -- 172.21.15.7:0/2891933249 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f40cc066f60 msgr2=0x7f40cc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:05.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.007+0000 7f40f056e700 1 --2- 172.21.15.7:0/2891933249 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f40cc066f60 0x7f40cc069410 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f40e8063b50 tx=0x7f40e4008040 comp rx=0 tx=0).stop 2024-01-27T02:34:05.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.007+0000 7f40f056e700 1 -- 172.21.15.7:0/2891933249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e80cdb70 msgr2=0x7f40e8156790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:05.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.007+0000 7f40f056e700 1 --2- 172.21.15.7:0/2891933249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e80cdb70 0x7f40e8156790 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f40e806ea80 tx=0x7f40dc017000 comp rx=0 tx=0).stop 2024-01-27T02:34:05.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.007+0000 7f40f056e700 1 -- 172.21.15.7:0/2891933249 shutdown_connections 2024-01-27T02:34:05.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.007+0000 7f40f056e700 1 --2- 172.21.15.7:0/2891933249 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f40cc066f60 0x7f40cc069410 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:05.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.007+0000 7f40f056e700 1 --2- 172.21.15.7:0/2891933249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40e8157f40 0x7f40e80b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:05.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.007+0000 7f40f056e700 1 --2- 172.21.15.7:0/2891933249 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40e80cbc00 0x7f40e8068410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:05.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.007+0000 7f40f056e700 1 --2- 172.21.15.7:0/2891933249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e80cdb70 0x7f40e8156790 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:05.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.007+0000 7f40f056e700 1 -- 172.21.15.7:0/2891933249 >> 172.21.15.7:0/2891933249 conn(0x7f40e8098020 msgr2=0x7f40e80cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:05.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.008+0000 7f40f056e700 1 -- 172.21.15.7:0/2891933249 shutdown_connections 2024-01-27T02:34:05.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.008+0000 7f40f056e700 1 -- 172.21.15.7:0/2891933249 wait complete. 2024-01-27T02:34:05.009 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-27T02:34:05.023 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-27T02:34:05.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.165+0000 7ff72f8a0700 1 Processor -- start 2024-01-27T02:34:05.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.165+0000 7ff72f8a0700 1 -- start start 2024-01-27T02:34:05.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.166+0000 7ff72f8a0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff7280679b0 0x7ff7280c4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:05.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.166+0000 7ff72f8a0700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff7280c4790 0x7ff7280ca420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.166+0000 7ff72f8a0700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7280c9f90 con 0x7ff72815e640 2024-01-27T02:34:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.166+0000 7ff72f8a0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7280c4bd0 con 0x7ff7280679b0 2024-01-27T02:34:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.166+0000 7ff72f8a0700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7280c9380 con 0x7ff7280c4790 2024-01-27T02:34:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.166+0000 7ff72ce3b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff7280679b0 0x7ff7280c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.166+0000 7ff72ce3b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff7280679b0 0x7ff7280c4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58118/0 (socket says 172.21.15.7:58118) 2024-01-27T02:34:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.167+0000 7ff72ce3b700 1 -- 172.21.15.7:0/3492464040 learned_addr learned my addr 172.21.15.7:0/3492464040 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.167+0000 7ff72de3d700 1 --2- 172.21.15.7:0/3492464040 >> v2:172.21.15.188:3300/0 conn(0x7ff7280c4790 0x7ff7280ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.167+0000 7ff72ce3b700 1 -- 172.21.15.7:0/3492464040 >> v2:172.21.15.188:3300/0 conn(0x7ff7280c4790 msgr2=0x7ff7280ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.167+0000 7ff72ce3b700 1 --2- 172.21.15.7:0/3492464040 >> v2:172.21.15.188:3300/0 conn(0x7ff7280c4790 0x7ff7280ca420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.167+0000 7ff72ce3b700 1 -- 172.21.15.7:0/3492464040 >> v1:172.21.15.188:6789/0 conn(0x7ff72815e640 legacy=0x7ff7280cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.167+0000 7ff72ce3b700 1 -- 172.21.15.7:0/3492464040 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7280c9660 con 0x7ff7280679b0 2024-01-27T02:34:05.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.167+0000 7ff72de3d700 1 --2- 172.21.15.7:0/3492464040 >> v2:172.21.15.188:3300/0 conn(0x7ff7280c4790 0x7ff7280ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:34:05.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.167+0000 7ff72ce3b700 1 --2- 172.21.15.7:0/3492464040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7280679b0 0x7ff7280c4250 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff72806e7a0 tx=0x7ff71c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191a1ff89fea0461 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:05.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.167+0000 7ff71bfff700 1 -- 172.21.15.7:0/3492464040 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c00b820 con 0x7ff7280679b0 2024-01-27T02:34:05.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.167+0000 7ff71bfff700 1 -- 172.21.15.7:0/3492464040 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71c00b980 con 0x7ff7280679b0 2024-01-27T02:34:05.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.167+0000 7ff71bfff700 1 -- 172.21.15.7:0/3492464040 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c00bd20 con 0x7ff7280679b0 2024-01-27T02:34:05.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.168+0000 7ff72f8a0700 1 -- 172.21.15.7:0/3492464040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7280679b0 msgr2=0x7ff7280c4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:05.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.168+0000 7ff72f8a0700 1 --2- 172.21.15.7:0/3492464040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7280679b0 0x7ff7280c4250 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff72806e7a0 tx=0x7ff71c009250 comp rx=0 tx=0).stop 2024-01-27T02:34:05.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.168+0000 7ff72f8a0700 1 -- 172.21.15.7:0/3492464040 shutdown_connections 2024-01-27T02:34:05.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.168+0000 7ff72f8a0700 1 --2- 172.21.15.7:0/3492464040 >> v2:172.21.15.188:3300/0 conn(0x7ff7280c4790 0x7ff7280ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:05.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.168+0000 7ff72f8a0700 1 --2- 172.21.15.7:0/3492464040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7280679b0 0x7ff7280c4250 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:05.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.168+0000 7ff72f8a0700 1 -- 172.21.15.7:0/3492464040 >> 172.21.15.7:0/3492464040 conn(0x7ff728098020 msgr2=0x7ff7280711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:05.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.168+0000 7ff72f8a0700 1 -- 172.21.15.7:0/3492464040 shutdown_connections 2024-01-27T02:34:05.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.168+0000 7ff72f8a0700 1 -- 172.21.15.7:0/3492464040 wait complete. 2024-01-27T02:34:05.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.169+0000 7ff72f8a0700 1 Processor -- start 2024-01-27T02:34:05.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.169+0000 7ff72f8a0700 1 -- start start 2024-01-27T02:34:05.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.170+0000 7ff72f8a0700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7280679b0 0x7ff728163940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:05.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.170+0000 7ff72f8a0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7280c4790 0x7ff7280c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:05.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.170+0000 7ff72f8a0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff72815e640 0x7ff728161650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:05.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.170+0000 7ff72f8a0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7280b2800 con 0x7ff72815e640 2024-01-27T02:34:05.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.170+0000 7ff72f8a0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7280b1c30 con 0x7ff7280c4790 2024-01-27T02:34:05.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.170+0000 7ff72f8a0700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff728163e80 con 0x7ff7280679b0 2024-01-27T02:34:05.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.170+0000 7ff72de3d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff72815e640 0x7ff728161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:05.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.170+0000 7ff72de3d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff72815e640 0x7ff728161650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58128/0 (socket says 172.21.15.7:58128) 2024-01-27T02:34:05.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.170+0000 7ff72de3d700 1 -- 172.21.15.7:0/4206784786 learned_addr learned my addr 172.21.15.7:0/4206784786 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:05.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.170+0000 7ff72ce3b700 1 --2- 172.21.15.7:0/4206784786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7280c4790 0x7ff7280c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:05.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.170+0000 7ff72d63c700 1 --2- 172.21.15.7:0/4206784786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7280679b0 0x7ff728163940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:05.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.171+0000 7ff72de3d700 1 -- 172.21.15.7:0/4206784786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7280679b0 msgr2=0x7ff728163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:05.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.171+0000 7ff72de3d700 1 --2- 172.21.15.7:0/4206784786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7280679b0 0x7ff728163940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:05.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.171+0000 7ff72de3d700 1 -- 172.21.15.7:0/4206784786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7280c4790 msgr2=0x7ff7280c67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:05.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.171+0000 7ff72de3d700 1 --2- 172.21.15.7:0/4206784786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7280c4790 0x7ff7280c67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:05.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.171+0000 7ff72de3d700 1 -- 172.21.15.7:0/4206784786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71c0094f0 con 0x7ff72815e640 2024-01-27T02:34:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.171+0000 7ff72de3d700 1 --2- 172.21.15.7:0/4206784786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff72815e640 0x7ff728161650 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff72000a380 tx=0x7ff720007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.171+0000 7ff7197fa700 1 -- 172.21.15.7:0/4206784786 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720015440 con 0x7ff72815e640 2024-01-27T02:34:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.171+0000 7ff7197fa700 1 -- 172.21.15.7:0/4206784786 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7200155a0 con 0x7ff72815e640 2024-01-27T02:34:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.171+0000 7ff72f8a0700 1 -- 172.21.15.7:0/4206784786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7280b1ed0 con 0x7ff72815e640 2024-01-27T02:34:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.171+0000 7ff7197fa700 1 -- 172.21.15.7:0/4206784786 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720015960 con 0x7ff72815e640 2024-01-27T02:34:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.171+0000 7ff72f8a0700 1 -- 172.21.15.7:0/4206784786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7280b2490 con 0x7ff72815e640 2024-01-27T02:34:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.172+0000 7ff72f8a0700 1 -- 172.21.15.7:0/4206784786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7280b29b0 con 0x7ff72815e640 2024-01-27T02:34:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.173+0000 7ff7197fa700 1 -- 172.21.15.7:0/4206784786 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff72000d440 con 0x7ff72815e640 2024-01-27T02:34:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.173+0000 7ff7197fa700 1 --2- 172.21.15.7:0/4206784786 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff70c066f30 0x7ff70c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:05.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.173+0000 7ff7197fa700 1 -- 172.21.15.7:0/4206784786 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5411+0+0 (secure 0 0 0) 0x7ff720084ed0 con 0x7ff72815e640 2024-01-27T02:34:05.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.173+0000 7ff72d63c700 1 --2- 172.21.15.7:0/4206784786 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff70c066f30 0x7ff70c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:05.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.174+0000 7ff72d63c700 1 --2- 172.21.15.7:0/4206784786 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff70c066f30 0x7ff70c0693e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff72815c880 tx=0x7ff724007ef0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:05.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.178+0000 7ff7197fa700 1 -- 172.21.15.7:0/4206784786 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7280b29b0 con 0x7ff72815e640 2024-01-27T02:34:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:05.384+0000 7ff72f8a0700 1 -- 172.21.15.7:0/4206784786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff72804c710 con 0x7ff72815e640 2024-01-27T02:34:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.011+0000 7ff7197fa700 1 -- 172.21.15.7:0/4206784786 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v56) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff72001a0d0 con 0x7ff72815e640 2024-01-27T02:34:06.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.017+0000 7ff72f8a0700 1 -- 172.21.15.7:0/4206784786 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff70c066f30 msgr2=0x7ff70c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:06.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.017+0000 7ff72f8a0700 1 --2- 172.21.15.7:0/4206784786 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff70c066f30 0x7ff70c0693e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff72815c880 tx=0x7ff724007ef0 comp rx=0 tx=0).stop 2024-01-27T02:34:06.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.017+0000 7ff72f8a0700 1 -- 172.21.15.7:0/4206784786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff72815e640 msgr2=0x7ff728161650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:06.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.017+0000 7ff72f8a0700 1 --2- 172.21.15.7:0/4206784786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff72815e640 0x7ff728161650 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff72000a380 tx=0x7ff720007900 comp rx=0 tx=0).stop 2024-01-27T02:34:06.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.017+0000 7ff72f8a0700 1 -- 172.21.15.7:0/4206784786 shutdown_connections 2024-01-27T02:34:06.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.017+0000 7ff72f8a0700 1 --2- 172.21.15.7:0/4206784786 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff70c066f30 0x7ff70c0693e0 secure :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff72815c880 tx=0x7ff724007ef0 comp rx=0 tx=0).stop 2024-01-27T02:34:06.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.017+0000 7ff72f8a0700 1 --2- 172.21.15.7:0/4206784786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff72815e640 0x7ff728161650 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:06.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.017+0000 7ff72f8a0700 1 --2- 172.21.15.7:0/4206784786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7280679b0 0x7ff728163940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:06.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.017+0000 7ff72f8a0700 1 --2- 172.21.15.7:0/4206784786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7280c4790 0x7ff7280c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:06.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.017+0000 7ff72f8a0700 1 -- 172.21.15.7:0/4206784786 >> 172.21.15.7:0/4206784786 conn(0x7ff728098020 msgr2=0x7ff72806c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:06.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.018+0000 7ff72f8a0700 1 -- 172.21.15.7:0/4206784786 shutdown_connections 2024-01-27T02:34:06.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.018+0000 7ff72f8a0700 1 -- 172.21.15.7:0/4206784786 wait complete. 2024-01-27T02:34:06.020 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-27T02:34:06.034 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-27T02:34:06.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.170+0000 7fdd09bcd700 1 Processor -- start 2024-01-27T02:34:06.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.170+0000 7fdd09bcd700 1 -- start start 2024-01-27T02:34:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.171+0000 7fdd09bcd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdd040cee40 0x7fdd0415b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.171+0000 7fdd09bcd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd040ca420 0x7fdd04157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.171+0000 7fdd09bcd700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0415c3e0 con 0x7fdd0415be80 2024-01-27T02:34:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.171+0000 7fdd09bcd700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd0415c6c0 con 0x7fdd040cee40 2024-01-27T02:34:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.171+0000 7fdd09bcd700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd040afc40 con 0x7fdd040ca420 2024-01-27T02:34:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.171+0000 7fdd037fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdd040cee40 0x7fdd0415b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.171+0000 7fdd037fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdd040cee40 0x7fdd0415b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58144/0 (socket says 172.21.15.7:58144) 2024-01-27T02:34:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.171+0000 7fdd037fe700 1 -- 172.21.15.7:0/2691163380 learned_addr learned my addr 172.21.15.7:0/2691163380 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.171+0000 7fdd03fff700 1 --2- 172.21.15.7:0/2691163380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd040ca420 0x7fdd04157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.172+0000 7fdd037fe700 1 -- 172.21.15.7:0/2691163380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd040ca420 msgr2=0x7fdd04157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.172+0000 7fdd037fe700 1 --2- 172.21.15.7:0/2691163380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd040ca420 0x7fdd04157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.172+0000 7fdd037fe700 1 -- 172.21.15.7:0/2691163380 >> v1:172.21.15.188:6789/0 conn(0x7fdd0415be80 legacy=0x7fdd04157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.172+0000 7fdd037fe700 1 -- 172.21.15.7:0/2691163380 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd040aff20 con 0x7fdd040cee40 2024-01-27T02:34:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.172+0000 7fdd037fe700 1 --2- 172.21.15.7:0/2691163380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd040cee40 0x7fdd0415b940 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40099b0 tx=0x7fdcf4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311500576f3f9570 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.172+0000 7fdd027fc700 1 -- 172.21.15.7:0/2691163380 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4009ea0 con 0x7fdd040cee40 2024-01-27T02:34:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.172+0000 7fdd027fc700 1 -- 172.21.15.7:0/2691163380 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf400b860 con 0x7fdd040cee40 2024-01-27T02:34:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.172+0000 7fdd027fc700 1 -- 172.21.15.7:0/2691163380 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf400bc00 con 0x7fdd040cee40 2024-01-27T02:34:06.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.173+0000 7fdd09bcd700 1 -- 172.21.15.7:0/2691163380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd040cee40 msgr2=0x7fdd0415b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:06.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.173+0000 7fdd09bcd700 1 --2- 172.21.15.7:0/2691163380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd040cee40 0x7fdd0415b940 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40099b0 tx=0x7fdcf4009250 comp rx=0 tx=0).stop 2024-01-27T02:34:06.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.173+0000 7fdd09bcd700 1 -- 172.21.15.7:0/2691163380 shutdown_connections 2024-01-27T02:34:06.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.173+0000 7fdd09bcd700 1 --2- 172.21.15.7:0/2691163380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd040ca420 0x7fdd04157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:06.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.173+0000 7fdd09bcd700 1 --2- 172.21.15.7:0/2691163380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd040cee40 0x7fdd0415b940 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:06.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.173+0000 7fdd09bcd700 1 -- 172.21.15.7:0/2691163380 >> 172.21.15.7:0/2691163380 conn(0x7fdd0406f2c0 msgr2=0x7fdd0406f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:06.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.173+0000 7fdd09bcd700 1 -- 172.21.15.7:0/2691163380 shutdown_connections 2024-01-27T02:34:06.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.173+0000 7fdd09bcd700 1 -- 172.21.15.7:0/2691163380 wait complete. 2024-01-27T02:34:06.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.174+0000 7fdd09bcd700 1 Processor -- start 2024-01-27T02:34:06.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.174+0000 7fdd09bcd700 1 -- start start 2024-01-27T02:34:06.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.175+0000 7fdd09bcd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd040ca420 0x7fdd040c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:06.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.175+0000 7fdd09bcd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd040cee40 0x7fdd040c6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:06.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.175+0000 7fdd09bcd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd0415be80 0x7fdd040caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.175+0000 7fdd09bcd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd040c52d0 con 0x7fdd040cee40 2024-01-27T02:34:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.175+0000 7fdd09bcd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd040c7fc0 con 0x7fdd040ca420 2024-01-27T02:34:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.175+0000 7fdd09bcd700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd040c73b0 con 0x7fdd0415be80 2024-01-27T02:34:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.175+0000 7fdd02ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd040cee40 0x7fdd040c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.175+0000 7fdd02ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd040cee40 0x7fdd040c6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58146/0 (socket says 172.21.15.7:58146) 2024-01-27T02:34:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.175+0000 7fdd02ffd700 1 -- 172.21.15.7:0/3344831781 learned_addr learned my addr 172.21.15.7:0/3344831781 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.175+0000 7fdd03fff700 1 --2- 172.21.15.7:0/3344831781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd0415be80 0x7fdd040caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.175+0000 7fdd037fe700 1 --2- 172.21.15.7:0/3344831781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd040ca420 0x7fdd040c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.175+0000 7fdd02ffd700 1 -- 172.21.15.7:0/3344831781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd0415be80 msgr2=0x7fdd040caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.175+0000 7fdd02ffd700 1 --2- 172.21.15.7:0/3344831781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd0415be80 0x7fdd040caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.175+0000 7fdd02ffd700 1 -- 172.21.15.7:0/3344831781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd040ca420 msgr2=0x7fdd040c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.175+0000 7fdd02ffd700 1 --2- 172.21.15.7:0/3344831781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd040ca420 0x7fdd040c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.175+0000 7fdd02ffd700 1 -- 172.21.15.7:0/3344831781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf40094f0 con 0x7fdd040cee40 2024-01-27T02:34:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.176+0000 7fdd02ffd700 1 --2- 172.21.15.7:0/3344831781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd040cee40 0x7fdd040c6650 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800c4e0 tx=0x7fdcf800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:06.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.176+0000 7fdceb7fe700 1 -- 172.21.15.7:0/3344831781 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8011590 con 0x7fdd040cee40 2024-01-27T02:34:06.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.176+0000 7fdceb7fe700 1 -- 172.21.15.7:0/3344831781 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf8011710 con 0x7fdd040cee40 2024-01-27T02:34:06.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.176+0000 7fdceb7fe700 1 -- 172.21.15.7:0/3344831781 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8011ad0 con 0x7fdd040cee40 2024-01-27T02:34:06.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.176+0000 7fdd09bcd700 1 -- 172.21.15.7:0/3344831781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd040c7690 con 0x7fdd040cee40 2024-01-27T02:34:06.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.176+0000 7fdd09bcd700 1 -- 172.21.15.7:0/3344831781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd040c7c50 con 0x7fdd040cee40 2024-01-27T02:34:06.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.177+0000 7fdd09bcd700 1 -- 172.21.15.7:0/3344831781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd04072060 con 0x7fdd040cee40 2024-01-27T02:34:06.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.177+0000 7fdceb7fe700 1 -- 172.21.15.7:0/3344831781 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fdcf801b3b0 con 0x7fdd040cee40 2024-01-27T02:34:06.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.178+0000 7fdceb7fe700 1 --2- 172.21.15.7:0/3344831781 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdcf0066f30 0x7fdcf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:06.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.178+0000 7fdceb7fe700 1 -- 172.21.15.7:0/3344831781 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fdcf80836f0 con 0x7fdd040cee40 2024-01-27T02:34:06.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.178+0000 7fdd037fe700 1 --2- 172.21.15.7:0/3344831781 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdcf0066f30 0x7fdcf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:06.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.179+0000 7fdd037fe700 1 --2- 172.21.15.7:0/3344831781 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdcf0066f30 0x7fdcf00693e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40099b0 tx=0x7fdcf4017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:06.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.183+0000 7fdceb7fe700 1 -- 172.21.15.7:0/3344831781 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdcf8050c60 con 0x7fdd040cee40 2024-01-27T02:34:06.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.389+0000 7fdd09bcd700 1 -- 172.21.15.7:0/3344831781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fdd040c8100 con 0x7fdd040cee40 2024-01-27T02:34:06.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.400+0000 7fdceb7fe700 1 -- 172.21.15.7:0/3344831781 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdcf80170a0 con 0x7fdd040cee40 2024-01-27T02:34:06.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.403+0000 7fdd09bcd700 1 -- 172.21.15.7:0/3344831781 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdcf0066f30 msgr2=0x7fdcf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:06.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.403+0000 7fdd09bcd700 1 --2- 172.21.15.7:0/3344831781 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdcf0066f30 0x7fdcf00693e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40099b0 tx=0x7fdcf4017000 comp rx=0 tx=0).stop 2024-01-27T02:34:06.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.403+0000 7fdd09bcd700 1 -- 172.21.15.7:0/3344831781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd040cee40 msgr2=0x7fdd040c6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:06.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.403+0000 7fdd09bcd700 1 --2- 172.21.15.7:0/3344831781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd040cee40 0x7fdd040c6650 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800c4e0 tx=0x7fdcf800c8a0 comp rx=0 tx=0).stop 2024-01-27T02:34:06.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.403+0000 7fdd09bcd700 1 -- 172.21.15.7:0/3344831781 shutdown_connections 2024-01-27T02:34:06.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.403+0000 7fdd09bcd700 1 --2- 172.21.15.7:0/3344831781 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdcf0066f30 0x7fdcf00693e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:06.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.403+0000 7fdd09bcd700 1 --2- 172.21.15.7:0/3344831781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd0415be80 0x7fdd040caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:06.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.403+0000 7fdd09bcd700 1 --2- 172.21.15.7:0/3344831781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd040ca420 0x7fdd040c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:06.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.403+0000 7fdd09bcd700 1 --2- 172.21.15.7:0/3344831781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd040cee40 0x7fdd040c6650 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:06.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.403+0000 7fdd09bcd700 1 -- 172.21.15.7:0/3344831781 >> 172.21.15.7:0/3344831781 conn(0x7fdd0406f2c0 msgr2=0x7fdd04151b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:06.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.403+0000 7fdd09bcd700 1 -- 172.21.15.7:0/3344831781 shutdown_connections 2024-01-27T02:34:06.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.403+0000 7fdd09bcd700 1 -- 172.21.15.7:0/3344831781 wait complete. 2024-01-27T02:34:06.417 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2024-01-27T02:34:06.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.549+0000 7f958e2f1700 1 Processor -- start 2024-01-27T02:34:06.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.549+0000 7f958e2f1700 1 -- start start 2024-01-27T02:34:06.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.550+0000 7f958e2f1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9588143520 0x7f9588063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:06.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.550+0000 7f958e2f1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95880aa8b0 0x7f95880c8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:06.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.550+0000 7f958e2f1700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95880cb7a0 con 0x7f95881438f0 2024-01-27T02:34:06.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.550+0000 7f958e2f1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95880cb9f0 con 0x7f9588143520 2024-01-27T02:34:06.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.550+0000 7f958e2f1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95880cbcd0 con 0x7f95880aa8b0 2024-01-27T02:34:06.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.550+0000 7f9587fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9588143520 0x7f9588063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:06.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.550+0000 7f9587fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9588143520 0x7f9588063580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58148/0 (socket says 172.21.15.7:58148) 2024-01-27T02:34:06.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.550+0000 7f9587fff700 1 -- 172.21.15.7:0/3280519205 learned_addr learned my addr 172.21.15.7:0/3280519205 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:06.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.550+0000 7f958c88e700 1 --2- 172.21.15.7:0/3280519205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95880aa8b0 0x7f95880c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:06.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.550+0000 7f9587fff700 1 -- 172.21.15.7:0/3280519205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95880aa8b0 msgr2=0x7f95880c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:06.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.550+0000 7f9587fff700 1 --2- 172.21.15.7:0/3280519205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95880aa8b0 0x7f95880c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:06.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.550+0000 7f9587fff700 1 -- 172.21.15.7:0/3280519205 >> v1:172.21.15.188:6789/0 conn(0x7f95881438f0 legacy=0x7f95880c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:06.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.550+0000 7f9587fff700 1 -- 172.21.15.7:0/3280519205 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95880cbfb0 con 0x7f9588143520 2024-01-27T02:34:06.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.550+0000 7f958c88e700 1 --2- 172.21.15.7:0/3280519205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95880aa8b0 0x7f95880c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:06.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.551+0000 7f9587fff700 1 --2- 172.21.15.7:0/3280519205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9588143520 0x7f9588063580 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9578009990 tx=0x7f9578009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f90d7dcbe7c5cd server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:06.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.551+0000 7f9586ffd700 1 -- 172.21.15.7:0/3280519205 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578005c10 con 0x7f9588143520 2024-01-27T02:34:06.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.551+0000 7f9586ffd700 1 -- 172.21.15.7:0/3280519205 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9578005d70 con 0x7f9588143520 2024-01-27T02:34:06.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.551+0000 7f9586ffd700 1 -- 172.21.15.7:0/3280519205 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957800b8a0 con 0x7f9588143520 2024-01-27T02:34:06.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.551+0000 7f958e2f1700 1 -- 172.21.15.7:0/3280519205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9588143520 msgr2=0x7f9588063580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:06.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.551+0000 7f958e2f1700 1 --2- 172.21.15.7:0/3280519205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9588143520 0x7f9588063580 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9578009990 tx=0x7f9578009220 comp rx=0 tx=0).stop 2024-01-27T02:34:06.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.551+0000 7f958e2f1700 1 -- 172.21.15.7:0/3280519205 shutdown_connections 2024-01-27T02:34:06.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.551+0000 7f958e2f1700 1 --2- 172.21.15.7:0/3280519205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95880aa8b0 0x7f95880c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:06.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.551+0000 7f958e2f1700 1 --2- 172.21.15.7:0/3280519205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9588143520 0x7f9588063580 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:06.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.551+0000 7f958e2f1700 1 -- 172.21.15.7:0/3280519205 >> 172.21.15.7:0/3280519205 conn(0x7f95880cdbd0 msgr2=0x7f95880cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:06.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.552+0000 7f958e2f1700 1 -- 172.21.15.7:0/3280519205 shutdown_connections 2024-01-27T02:34:06.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.552+0000 7f958e2f1700 1 -- 172.21.15.7:0/3280519205 wait complete. 2024-01-27T02:34:06.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.553+0000 7f958e2f1700 1 Processor -- start 2024-01-27T02:34:06.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.553+0000 7f958e2f1700 1 -- start start 2024-01-27T02:34:06.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.554+0000 7f958e2f1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95880aa8b0 0x7f95880b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:06.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.554+0000 7f958e2f1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9588143520 0x7f9588140d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:06.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.554+0000 7f958e2f1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95881438f0 0x7f958816a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:06.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.554+0000 7f958e2f1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95880ad740 con 0x7f95881438f0 2024-01-27T02:34:06.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.554+0000 7f958e2f1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95880add00 con 0x7f9588143520 2024-01-27T02:34:06.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.554+0000 7f958e2f1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958816b130 con 0x7f95880aa8b0 2024-01-27T02:34:06.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.554+0000 7f958c88e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95881438f0 0x7f958816a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:06.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.554+0000 7f958c88e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95881438f0 0x7f958816a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58160/0 (socket says 172.21.15.7:58160) 2024-01-27T02:34:06.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.554+0000 7f958c88e700 1 -- 172.21.15.7:0/1179743159 learned_addr learned my addr 172.21.15.7:0/1179743159 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:06.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.554+0000 7f95877fe700 1 --2- 172.21.15.7:0/1179743159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9588143520 0x7f9588140d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:06.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.554+0000 7f9587fff700 1 --2- 172.21.15.7:0/1179743159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95880aa8b0 0x7f95880b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:06.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.554+0000 7f958c88e700 1 -- 172.21.15.7:0/1179743159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95880aa8b0 msgr2=0x7f95880b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:06.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.554+0000 7f958c88e700 1 --2- 172.21.15.7:0/1179743159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95880aa8b0 0x7f95880b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:06.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.554+0000 7f958c88e700 1 -- 172.21.15.7:0/1179743159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9588143520 msgr2=0x7f9588140d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:06.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.554+0000 7f958c88e700 1 --2- 172.21.15.7:0/1179743159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9588143520 0x7f9588140d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:06.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.554+0000 7f958c88e700 1 -- 172.21.15.7:0/1179743159 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9578009480 con 0x7f95881438f0 2024-01-27T02:34:06.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.554+0000 7f95877fe700 1 --2- 172.21.15.7:0/1179743159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9588143520 0x7f9588140d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:34:06.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.554+0000 7f958c88e700 1 --2- 172.21.15.7:0/1179743159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95881438f0 0x7f958816a600 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f958000d180 tx=0x7f9580009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:06.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.554+0000 7f957ffff700 1 -- 172.21.15.7:0/1179743159 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958000de20 con 0x7f95881438f0 2024-01-27T02:34:06.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.555+0000 7f957ffff700 1 -- 172.21.15.7:0/1179743159 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958000f040 con 0x7f95881438f0 2024-01-27T02:34:06.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.555+0000 7f958e2f1700 1 -- 172.21.15.7:0/1179743159 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f958816b410 con 0x7f95881438f0 2024-01-27T02:34:06.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.555+0000 7f957ffff700 1 -- 172.21.15.7:0/1179743159 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9580015690 con 0x7f95881438f0 2024-01-27T02:34:06.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.555+0000 7f958e2f1700 1 -- 172.21.15.7:0/1179743159 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9588141260 con 0x7f95881438f0 2024-01-27T02:34:06.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.556+0000 7f958e2f1700 1 -- 172.21.15.7:0/1179743159 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95880ade70 con 0x7f95881438f0 2024-01-27T02:34:06.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.556+0000 7f957ffff700 1 -- 172.21.15.7:0/1179743159 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f958001e3b0 con 0x7f95881438f0 2024-01-27T02:34:06.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.556+0000 7f957ffff700 1 --2- 172.21.15.7:0/1179743159 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9568066f30 0x7f95680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:06.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.557+0000 7f957ffff700 1 -- 172.21.15.7:0/1179743159 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f9580084810 con 0x7f95881438f0 2024-01-27T02:34:06.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.557+0000 7f9587fff700 1 --2- 172.21.15.7:0/1179743159 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9568066f30 0x7f95680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:06.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.558+0000 7f9587fff700 1 --2- 172.21.15.7:0/1179743159 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9568066f30 0x7f95680693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9578009fd0 tx=0x7f9578016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:06.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.561+0000 7f957ffff700 1 -- 172.21.15.7:0/1179743159 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9580051ce0 con 0x7f95881438f0 2024-01-27T02:34:06.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:06.850+0000 7f958e2f1700 1 -- 172.21.15.7:0/1179743159 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) v1 -- 0x7f958804c710 con 0x7f95881438f0 2024-01-27T02:34:07.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.012+0000 7f957ffff700 1 -- 172.21.15.7:0/1179743159 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v130) v1 ==== 142+0+0 (secure 0 0 0) 0x7f9580055300 con 0x7f95881438f0 2024-01-27T02:34:07.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.014+0000 7f958e2f1700 1 -- 172.21.15.7:0/1179743159 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9568066f30 msgr2=0x7f95680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:07.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.014+0000 7f958e2f1700 1 --2- 172.21.15.7:0/1179743159 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9568066f30 0x7f95680693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9578009fd0 tx=0x7f9578016000 comp rx=0 tx=0).stop 2024-01-27T02:34:07.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.014+0000 7f958e2f1700 1 -- 172.21.15.7:0/1179743159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95881438f0 msgr2=0x7f958816a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:07.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.014+0000 7f958e2f1700 1 --2- 172.21.15.7:0/1179743159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95881438f0 0x7f958816a600 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f958000d180 tx=0x7f9580009a50 comp rx=0 tx=0).stop 2024-01-27T02:34:07.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.014+0000 7f958e2f1700 1 -- 172.21.15.7:0/1179743159 shutdown_connections 2024-01-27T02:34:07.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.014+0000 7f958e2f1700 1 --2- 172.21.15.7:0/1179743159 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9568066f30 0x7f95680693e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:07.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.014+0000 7f958e2f1700 1 --2- 172.21.15.7:0/1179743159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95881438f0 0x7f958816a600 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:07.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.014+0000 7f958e2f1700 1 --2- 172.21.15.7:0/1179743159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95880aa8b0 0x7f95880b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:07.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.014+0000 7f958e2f1700 1 --2- 172.21.15.7:0/1179743159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9588143520 0x7f9588140d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:07.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.014+0000 7f958e2f1700 1 -- 172.21.15.7:0/1179743159 >> 172.21.15.7:0/1179743159 conn(0x7f95880cdbd0 msgr2=0x7f95880cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:07.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.015+0000 7f958e2f1700 1 -- 172.21.15.7:0/1179743159 shutdown_connections 2024-01-27T02:34:07.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.015+0000 7f958e2f1700 1 -- 172.21.15.7:0/1179743159 wait complete. 2024-01-27T02:34:07.030 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2024-01-27T02:34:07.030 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestClusterFull) 2024-01-27T02:34:07.031 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2024-01-27T02:34:07.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.166+0000 7f8745759700 1 Processor -- start 2024-01-27T02:34:07.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.166+0000 7f8745759700 1 -- start start 2024-01-27T02:34:07.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.167+0000 7f8745759700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87400d10a0 0x7f87400b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:07.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.167+0000 7f8745759700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87400b4650 con 0x7f87400d1470 2024-01-27T02:34:07.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.167+0000 7f8745759700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87400b3d60 con 0x7f8740159b40 2024-01-27T02:34:07.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.167+0000 7f8745759700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87400b34d0 con 0x7f87400d10a0 2024-01-27T02:34:07.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.167+0000 7f873e7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f87400d1470 0x7f87400b2dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58604/0 (socket says 172.21.15.7:58604) 2024-01-27T02:34:07.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.167+0000 7f873e7fc700 1 -- 172.21.15.7:0/2251976605 learned_addr learned my addr 172.21.15.7:0/2251976605 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:34:07.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.167+0000 7f873effd700 1 --2- 172.21.15.7:0/2251976605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87400d10a0 0x7f87400b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:07.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.167+0000 7f873dffb700 1 -- 172.21.15.7:0/2251976605 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972275507 0 0) 0x7f87400b4650 con 0x7f87400d1470 2024-01-27T02:34:07.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.167+0000 7f873dffb700 1 -- 172.21.15.7:0/2251976605 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8728003570 con 0x7f87400d1470 2024-01-27T02:34:07.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.168+0000 7f873dffb700 1 -- 172.21.15.7:0/2251976605 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724516735 0 0) 0x7f87400b3d60 con 0x7f8740159b40 2024-01-27T02:34:07.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.168+0000 7f873dffb700 1 -- 172.21.15.7:0/2251976605 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87400b4650 con 0x7f8740159b40 2024-01-27T02:34:07.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.168+0000 7f873dffb700 1 -- 172.21.15.7:0/2251976605 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3123918258 0 0) 0x7f8728003570 con 0x7f87400d1470 2024-01-27T02:34:07.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.168+0000 7f873effd700 1 -- 172.21.15.7:0/2251976605 >> v1:172.21.15.188:6789/0 conn(0x7f8740159b40 legacy=0x7f8740159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:07.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.168+0000 7f873effd700 1 -- 172.21.15.7:0/2251976605 >> v1:172.21.15.7:6789/0 conn(0x7f87400d1470 legacy=0x7f87400b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:07.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.168+0000 7f873effd700 1 -- 172.21.15.7:0/2251976605 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87400b4000 con 0x7f87400d10a0 2024-01-27T02:34:07.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.168+0000 7f873effd700 1 --2- 172.21.15.7:0/2251976605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87400d10a0 0x7f87400b1590 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8730009910 tx=0x7f87300091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6edf7a0be80f2571 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:07.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.168+0000 7f873dffb700 1 -- 172.21.15.7:0/2251976605 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8734001470 con 0x7f87400d10a0 2024-01-27T02:34:07.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.168+0000 7f873dffb700 1 -- 172.21.15.7:0/2251976605 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8730011420 con 0x7f87400d10a0 2024-01-27T02:34:07.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.168+0000 7f873dffb700 1 -- 172.21.15.7:0/2251976605 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87300117c0 con 0x7f87400d10a0 2024-01-27T02:34:07.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.169+0000 7f8745759700 1 -- 172.21.15.7:0/2251976605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87400d10a0 msgr2=0x7f87400b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:07.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.169+0000 7f8745759700 1 --2- 172.21.15.7:0/2251976605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87400d10a0 0x7f87400b1590 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8730009910 tx=0x7f87300091a0 comp rx=0 tx=0).stop 2024-01-27T02:34:07.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.169+0000 7f8745759700 1 -- 172.21.15.7:0/2251976605 shutdown_connections 2024-01-27T02:34:07.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.169+0000 7f8745759700 1 --2- 172.21.15.7:0/2251976605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87400d10a0 0x7f87400b1590 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:07.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.169+0000 7f8745759700 1 -- 172.21.15.7:0/2251976605 >> 172.21.15.7:0/2251976605 conn(0x7f87400679b0 msgr2=0x7f8740067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:07.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.169+0000 7f8745759700 1 -- 172.21.15.7:0/2251976605 shutdown_connections 2024-01-27T02:34:07.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.169+0000 7f8745759700 1 -- 172.21.15.7:0/2251976605 wait complete. 2024-01-27T02:34:07.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.170+0000 7f8745759700 1 Processor -- start 2024-01-27T02:34:07.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.170+0000 7f8745759700 1 -- start start 2024-01-27T02:34:07.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.171+0000 7f8745759700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87400d10a0 0x7f874016a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:07.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.171+0000 7f8745759700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87400d1470 0x7f874015a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:07.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.171+0000 7f8745759700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8740159b40 0x7f87400c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.171+0000 7f8745759700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87400c9920 con 0x7f8740159b40 2024-01-27T02:34:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.171+0000 7f8745759700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874015aef0 con 0x7f87400d1470 2024-01-27T02:34:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.171+0000 7f8745759700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87400c8090 con 0x7f87400d10a0 2024-01-27T02:34:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.171+0000 7f873f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8740159b40 0x7f87400c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.171+0000 7f873f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8740159b40 0x7f87400c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58166/0 (socket says 172.21.15.7:58166) 2024-01-27T02:34:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.171+0000 7f873f7fe700 1 -- 172.21.15.7:0/1996915221 learned_addr learned my addr 172.21.15.7:0/1996915221 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.171+0000 7f873effd700 1 --2- 172.21.15.7:0/1996915221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87400d10a0 0x7f874016a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.171+0000 7f873e7fc700 1 --2- 172.21.15.7:0/1996915221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87400d1470 0x7f874015a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.172+0000 7f873f7fe700 1 -- 172.21.15.7:0/1996915221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87400d10a0 msgr2=0x7f874016a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.172+0000 7f873f7fe700 1 --2- 172.21.15.7:0/1996915221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87400d10a0 0x7f874016a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.172+0000 7f873f7fe700 1 -- 172.21.15.7:0/1996915221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87400d1470 msgr2=0x7f874015a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.172+0000 7f873f7fe700 1 --2- 172.21.15.7:0/1996915221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87400d1470 0x7f874015a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.172+0000 7f873f7fe700 1 -- 172.21.15.7:0/1996915221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8730009420 con 0x7f8740159b40 2024-01-27T02:34:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.172+0000 7f873effd700 1 --2- 172.21.15.7:0/1996915221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87400d10a0 0x7f874016a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.172+0000 7f873f7fe700 1 --2- 172.21.15.7:0/1996915221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8740159b40 0x7f87400c8a20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f873800c910 tx=0x7f8738007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.172+0000 7f8726ffd700 1 -- 172.21.15.7:0/1996915221 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873800a7e0 con 0x7f8740159b40 2024-01-27T02:34:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.172+0000 7f8726ffd700 1 -- 172.21.15.7:0/1996915221 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873800a960 con 0x7f8740159b40 2024-01-27T02:34:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.172+0000 7f8726ffd700 1 -- 172.21.15.7:0/1996915221 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873800ad20 con 0x7f8740159b40 2024-01-27T02:34:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.172+0000 7f8745759700 1 -- 172.21.15.7:0/1996915221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874015acc0 con 0x7f8740159b40 2024-01-27T02:34:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.172+0000 7f8745759700 1 -- 172.21.15.7:0/1996915221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874015d090 con 0x7f8740159b40 2024-01-27T02:34:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.173+0000 7f8745759700 1 -- 172.21.15.7:0/1996915221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f874015d220 con 0x7f8740159b40 2024-01-27T02:34:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.174+0000 7f8726ffd700 1 -- 172.21.15.7:0/1996915221 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8738012630 con 0x7f8740159b40 2024-01-27T02:34:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.174+0000 7f8726ffd700 1 --2- 172.21.15.7:0/1996915221 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f872c066f30 0x7f872c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.174+0000 7f8726ffd700 1 -- 172.21.15.7:0/1996915221 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8738084650 con 0x7f8740159b40 2024-01-27T02:34:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.174+0000 7f873effd700 1 --2- 172.21.15.7:0/1996915221 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f872c066f30 0x7f872c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.175+0000 7f873effd700 1 --2- 172.21.15.7:0/1996915221 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f872c066f30 0x7f872c0693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f87300091e0 tx=0x7f8730016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.178+0000 7f8726ffd700 1 -- 172.21.15.7:0/1996915221 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8738051bc0 con 0x7f8740159b40 2024-01-27T02:34:07.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:07.468+0000 7f8745759700 1 -- 172.21.15.7:0/1996915221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) v1 -- 0x7f874004c710 con 0x7f8740159b40 2024-01-27T02:34:08.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.017+0000 7f8726ffd700 1 -- 172.21.15.7:0/1996915221 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v131) v1 ==== 137+0+0 (secure 0 0 0) 0x7f87380551e0 con 0x7f8740159b40 2024-01-27T02:34:08.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.020+0000 7f8745759700 1 -- 172.21.15.7:0/1996915221 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f872c066f30 msgr2=0x7f872c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:08.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.020+0000 7f8745759700 1 --2- 172.21.15.7:0/1996915221 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f872c066f30 0x7f872c0693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f87300091e0 tx=0x7f8730016000 comp rx=0 tx=0).stop 2024-01-27T02:34:08.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.020+0000 7f8745759700 1 -- 172.21.15.7:0/1996915221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8740159b40 msgr2=0x7f87400c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:08.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.020+0000 7f8745759700 1 --2- 172.21.15.7:0/1996915221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8740159b40 0x7f87400c8a20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f873800c910 tx=0x7f8738007f90 comp rx=0 tx=0).stop 2024-01-27T02:34:08.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.020+0000 7f8745759700 1 -- 172.21.15.7:0/1996915221 shutdown_connections 2024-01-27T02:34:08.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.020+0000 7f8745759700 1 --2- 172.21.15.7:0/1996915221 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f872c066f30 0x7f872c0693e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:08.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.020+0000 7f8745759700 1 --2- 172.21.15.7:0/1996915221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8740159b40 0x7f87400c8a20 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:08.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.020+0000 7f8745759700 1 --2- 172.21.15.7:0/1996915221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87400d10a0 0x7f874016a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:08.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.020+0000 7f8745759700 1 --2- 172.21.15.7:0/1996915221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87400d1470 0x7f874015a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:08.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.020+0000 7f8745759700 1 -- 172.21.15.7:0/1996915221 >> 172.21.15.7:0/1996915221 conn(0x7f87400679b0 msgr2=0x7f8740158e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:08.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.020+0000 7f8745759700 1 -- 172.21.15.7:0/1996915221 shutdown_connections 2024-01-27T02:34:08.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.020+0000 7f8745759700 1 -- 172.21.15.7:0/1996915221 wait complete. 2024-01-27T02:34:08.036 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-01-27T02:34:08.073 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:34:08.082 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:34:08.192 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_objectstore":"memstore"} 2024-01-27T02:34:08.192 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-01-27T02:34:08.193 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-27T02:34:08.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.369+0000 7f7b95f70700 1 Processor -- start 2024-01-27T02:34:08.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.369+0000 7f7b95f70700 1 -- start start 2024-01-27T02:34:08.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.370+0000 7f7b95f70700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7b9015c9b0 0x7f7b90157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:08.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.370+0000 7f7b95f70700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b9015c3b0 0x7f7b90158210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:08.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.371+0000 7f7b95f70700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b900cd0d0 con 0x7f7b9015bfe0 2024-01-27T02:34:08.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.371+0000 7f7b95f70700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b900cc4c0 con 0x7f7b9015c9b0 2024-01-27T02:34:08.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.371+0000 7f7b95f70700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b900cc7a0 con 0x7f7b9015c3b0 2024-01-27T02:34:08.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.371+0000 7f7b8f7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7b9015bfe0 0x7f7b900cbc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58614/0 (socket says 172.21.15.7:58614) 2024-01-27T02:34:08.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.371+0000 7f7b8f7fe700 1 -- 172.21.15.7:0/3080440051 learned_addr learned my addr 172.21.15.7:0/3080440051 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:34:08.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.371+0000 7f7b8effd700 1 --2- 172.21.15.7:0/3080440051 >> v2:172.21.15.7:3300/0 conn(0x7f7b9015c9b0 0x7f7b90157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:08.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.371+0000 7f7b8ffff700 1 --2- 172.21.15.7:0/3080440051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b9015c3b0 0x7f7b90158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:08.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.371+0000 7f7b8e7fc700 1 -- 172.21.15.7:0/3080440051 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594790952 0 0) 0x7f7b900cd0d0 con 0x7f7b9015bfe0 2024-01-27T02:34:08.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.371+0000 7f7b8e7fc700 1 -- 172.21.15.7:0/3080440051 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b7c003570 con 0x7f7b9015bfe0 2024-01-27T02:34:08.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.371+0000 7f7b8effd700 1 -- 172.21.15.7:0/3080440051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b9015c3b0 msgr2=0x7f7b90158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:08.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.371+0000 7f7b8effd700 1 --2- 172.21.15.7:0/3080440051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b9015c3b0 0x7f7b90158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:08.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.371+0000 7f7b8effd700 1 -- 172.21.15.7:0/3080440051 >> v1:172.21.15.7:6789/0 conn(0x7f7b9015bfe0 legacy=0x7f7b900cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:08.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.371+0000 7f7b8effd700 1 -- 172.21.15.7:0/3080440051 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b900cca80 con 0x7f7b9015c9b0 2024-01-27T02:34:08.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.371+0000 7f7b8effd700 1 --2- 172.21.15.7:0/3080440051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9015c9b0 0x7f7b90157cd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7b780098e0 tx=0x7f7b780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182f5ce13a389825 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:08.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.372+0000 7f7b8e7fc700 1 -- 172.21.15.7:0/3080440051 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7800b640 con 0x7f7b9015c9b0 2024-01-27T02:34:08.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.372+0000 7f7b8e7fc700 1 -- 172.21.15.7:0/3080440051 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7800b7a0 con 0x7f7b9015c9b0 2024-01-27T02:34:08.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.372+0000 7f7b8e7fc700 1 -- 172.21.15.7:0/3080440051 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7800bb40 con 0x7f7b9015c9b0 2024-01-27T02:34:08.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.372+0000 7f7b95f70700 1 -- 172.21.15.7:0/3080440051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9015c9b0 msgr2=0x7f7b90157cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:08.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.372+0000 7f7b95f70700 1 --2- 172.21.15.7:0/3080440051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9015c9b0 0x7f7b90157cd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7b780098e0 tx=0x7f7b780091a0 comp rx=0 tx=0).stop 2024-01-27T02:34:08.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.372+0000 7f7b95f70700 1 -- 172.21.15.7:0/3080440051 shutdown_connections 2024-01-27T02:34:08.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.372+0000 7f7b95f70700 1 --2- 172.21.15.7:0/3080440051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b9015c3b0 0x7f7b90158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:08.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.372+0000 7f7b95f70700 1 --2- 172.21.15.7:0/3080440051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9015c9b0 0x7f7b90157cd0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:08.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.372+0000 7f7b95f70700 1 -- 172.21.15.7:0/3080440051 >> 172.21.15.7:0/3080440051 conn(0x7f7b900679b0 msgr2=0x7f7b90067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:08.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.372+0000 7f7b95f70700 1 -- 172.21.15.7:0/3080440051 shutdown_connections 2024-01-27T02:34:08.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.372+0000 7f7b95f70700 1 -- 172.21.15.7:0/3080440051 wait complete. 2024-01-27T02:34:08.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.374+0000 7f7b95f70700 1 Processor -- start 2024-01-27T02:34:08.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.374+0000 7f7b95f70700 1 -- start start 2024-01-27T02:34:08.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.375+0000 7f7b95f70700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b9015bfe0 0x7f7b9016b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:08.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.375+0000 7f7b95f70700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9015c3b0 0x7f7b901568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:08.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.375+0000 7f7b95f70700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b9015c9b0 0x7f7b900c2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:08.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.375+0000 7f7b95f70700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b900c44d0 con 0x7f7b9015c3b0 2024-01-27T02:34:08.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.375+0000 7f7b95f70700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b900c47b0 con 0x7f7b9015c9b0 2024-01-27T02:34:08.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.375+0000 7f7b95f70700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b900c4a90 con 0x7f7b9015bfe0 2024-01-27T02:34:08.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.375+0000 7f7b8effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9015c3b0 0x7f7b901568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:08.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.375+0000 7f7b8effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9015c3b0 0x7f7b901568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58184/0 (socket says 172.21.15.7:58184) 2024-01-27T02:34:08.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.375+0000 7f7b8effd700 1 -- 172.21.15.7:0/3981440521 learned_addr learned my addr 172.21.15.7:0/3981440521 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:08.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.375+0000 7f7b8f7fe700 1 --2- 172.21.15.7:0/3981440521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b9015bfe0 0x7f7b9016b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:08.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.376+0000 7f7b8ffff700 1 --2- 172.21.15.7:0/3981440521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b9015c9b0 0x7f7b900c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:08.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.376+0000 7f7b8effd700 1 -- 172.21.15.7:0/3981440521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b9015bfe0 msgr2=0x7f7b9016b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:08.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.376+0000 7f7b8effd700 1 --2- 172.21.15.7:0/3981440521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b9015bfe0 0x7f7b9016b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:08.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.376+0000 7f7b8effd700 1 -- 172.21.15.7:0/3981440521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b9015c9b0 msgr2=0x7f7b900c2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:08.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.376+0000 7f7b8effd700 1 --2- 172.21.15.7:0/3981440521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b9015c9b0 0x7f7b900c2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:08.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.376+0000 7f7b8effd700 1 -- 172.21.15.7:0/3981440521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b78009420 con 0x7f7b9015c3b0 2024-01-27T02:34:08.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.376+0000 7f7b8effd700 1 --2- 172.21.15.7:0/3981440521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9015c3b0 0x7f7b901568d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800a010 tx=0x7f7b78017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:08.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.376+0000 7f7b877fe700 1 -- 172.21.15.7:0/3981440521 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7801be20 con 0x7f7b9015c3b0 2024-01-27T02:34:08.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.376+0000 7f7b877fe700 1 -- 172.21.15.7:0/3981440521 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b78011420 con 0x7f7b9015c3b0 2024-01-27T02:34:08.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.376+0000 7f7b95f70700 1 -- 172.21.15.7:0/3981440521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b900c4d70 con 0x7f7b9015c3b0 2024-01-27T02:34:08.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.376+0000 7f7b877fe700 1 -- 172.21.15.7:0/3981440521 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b780117e0 con 0x7f7b9015c3b0 2024-01-27T02:34:08.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.376+0000 7f7b95f70700 1 -- 172.21.15.7:0/3981440521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9016be10 con 0x7f7b9015c3b0 2024-01-27T02:34:08.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.377+0000 7f7b95f70700 1 -- 172.21.15.7:0/3981440521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b90072060 con 0x7f7b9015c3b0 2024-01-27T02:34:08.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.378+0000 7f7b877fe700 1 -- 172.21.15.7:0/3981440521 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7b78011b90 con 0x7f7b9015c3b0 2024-01-27T02:34:08.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.378+0000 7f7b877fe700 1 --2- 172.21.15.7:0/3981440521 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7b70066f30 0x7f7b700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:08.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.378+0000 7f7b877fe700 1 -- 172.21.15.7:0/3981440521 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7b78085b00 con 0x7f7b9015c3b0 2024-01-27T02:34:08.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.378+0000 7f7b8f7fe700 1 --2- 172.21.15.7:0/3981440521 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7b70066f30 0x7f7b700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:08.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.379+0000 7f7b8f7fe700 1 --2- 172.21.15.7:0/3981440521 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7b70066f30 0x7f7b700693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7b80001360 tx=0x7f7b80009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:08.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.382+0000 7f7b877fe700 1 -- 172.21.15.7:0/3981440521 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b78053070 con 0x7f7b9015c3b0 2024-01-27T02:34:08.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.587+0000 7f7b95f70700 1 -- 172.21.15.7:0/3981440521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7b900cabd0 con 0x7f7b9015c3b0 2024-01-27T02:34:08.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.591+0000 7f7b877fe700 1 -- 172.21.15.7:0/3981440521 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7b78056690 con 0x7f7b9015c3b0 2024-01-27T02:34:08.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.593+0000 7f7b95f70700 1 -- 172.21.15.7:0/3981440521 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7b70066f30 msgr2=0x7f7b700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:08.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.593+0000 7f7b95f70700 1 --2- 172.21.15.7:0/3981440521 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7b70066f30 0x7f7b700693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7b80001360 tx=0x7f7b80009040 comp rx=0 tx=0).stop 2024-01-27T02:34:08.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.593+0000 7f7b95f70700 1 -- 172.21.15.7:0/3981440521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9015c3b0 msgr2=0x7f7b901568d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:08.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.593+0000 7f7b95f70700 1 --2- 172.21.15.7:0/3981440521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9015c3b0 0x7f7b901568d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800a010 tx=0x7f7b78017000 comp rx=0 tx=0).stop 2024-01-27T02:34:08.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.593+0000 7f7b95f70700 1 -- 172.21.15.7:0/3981440521 shutdown_connections 2024-01-27T02:34:08.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.593+0000 7f7b95f70700 1 --2- 172.21.15.7:0/3981440521 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7b70066f30 0x7f7b700693e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:08.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.593+0000 7f7b95f70700 1 --2- 172.21.15.7:0/3981440521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b9015c9b0 0x7f7b900c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:08.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.593+0000 7f7b95f70700 1 --2- 172.21.15.7:0/3981440521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b9015bfe0 0x7f7b9016b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:08.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.593+0000 7f7b95f70700 1 --2- 172.21.15.7:0/3981440521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9015c3b0 0x7f7b901568d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:08.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.593+0000 7f7b95f70700 1 -- 172.21.15.7:0/3981440521 >> 172.21.15.7:0/3981440521 conn(0x7f7b900679b0 msgr2=0x7f7b900b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:08.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.593+0000 7f7b95f70700 1 -- 172.21.15.7:0/3981440521 shutdown_connections 2024-01-27T02:34:08.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.594+0000 7f7b95f70700 1 -- 172.21.15.7:0/3981440521 wait complete. 2024-01-27T02:34:08.607 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-27T02:34:08.608 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:34:08.654 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:34:08.698 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-27T02:34:08.698 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-27T02:34:08.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:34:08.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.833+0000 7f5d4a32c700 1 Processor -- start 2024-01-27T02:34:08.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.833+0000 7f5d4a32c700 1 -- start start 2024-01-27T02:34:08.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.834+0000 7f5d4a32c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5d440c4cc0 0x7f5d440cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:08.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.834+0000 7f5d4a32c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d440ce5e0 con 0x7f5d4415a100 2024-01-27T02:34:08.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.834+0000 7f5d4a32c700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d440cd9d0 con 0x7f5d440ca540 2024-01-27T02:34:08.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.834+0000 7f5d4a32c700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d440cdcb0 con 0x7f5d440c4cc0 2024-01-27T02:34:08.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.834+0000 7f5d43fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5d4415a100 0x7f5d440c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58616/0 (socket says 172.21.15.7:58616) 2024-01-27T02:34:08.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.834+0000 7f5d43fff700 1 -- 172.21.15.7:0/1108307487 learned_addr learned my addr 172.21.15.7:0/1108307487 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:34:08.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.834+0000 7f5d437fe700 1 --2- 172.21.15.7:0/1108307487 >> v2:172.21.15.188:3300/0 conn(0x7f5d440c4cc0 0x7f5d440cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.834+0000 7f5d42ffd700 1 -- 172.21.15.7:0/1108307487 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4016596851 0 0) 0x7f5d440ce5e0 con 0x7f5d4415a100 2024-01-27T02:34:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.834+0000 7f5d42ffd700 1 -- 172.21.15.7:0/1108307487 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d2c003570 con 0x7f5d4415a100 2024-01-27T02:34:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.834+0000 7f5d42ffd700 1 -- 172.21.15.7:0/1108307487 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 709178234 0 0) 0x7f5d2c003570 con 0x7f5d4415a100 2024-01-27T02:34:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.834+0000 7f5d42ffd700 1 -- 172.21.15.7:0/1108307487 >> v2:172.21.15.188:3300/0 conn(0x7f5d440c4cc0 msgr2=0x7f5d440cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.834+0000 7f5d42ffd700 1 --2- 172.21.15.7:0/1108307487 >> v2:172.21.15.188:3300/0 conn(0x7f5d440c4cc0 0x7f5d440cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.834+0000 7f5d42ffd700 1 -- 172.21.15.7:0/1108307487 >> v1:172.21.15.188:6789/0 conn(0x7f5d440ca540 legacy=0x7f5d44163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.834+0000 7f5d42ffd700 1 -- 172.21.15.7:0/1108307487 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d440cdf90 con 0x7f5d4415a100 2024-01-27T02:34:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.834+0000 7f5d42ffd700 1 -- 172.21.15.7:0/1108307487 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f5d34001470 con 0x7f5d4415a100 2024-01-27T02:34:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.834+0000 7f5d42ffd700 1 -- 172.21.15.7:0/1108307487 >> v1:172.21.15.7:6789/0 conn(0x7f5d4415a100 legacy=0x7f5d440c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.834+0000 7f5d42ffd700 1 --2- 172.21.15.7:0/1108307487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d2c003a60 0x7f5d2c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.835+0000 7f5d42ffd700 1 --2- 172.21.15.7:0/1108307487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d2c004810 0x7f5d2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.835+0000 7f5d42ffd700 1 --2- 172.21.15.7:0/1108307487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d2c0051c0 0x7f5d2c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.835+0000 7f5d42ffd700 1 -- 172.21.15.7:0/1108307487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d440cdcb0 con 0x7f5d2c0051c0 2024-01-27T02:34:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.835+0000 7f5d42ffd700 1 -- 172.21.15.7:0/1108307487 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c00c0b0 con 0x7f5d2c004810 2024-01-27T02:34:08.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.835+0000 7f5d42ffd700 1 -- 172.21.15.7:0/1108307487 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c00c270 con 0x7f5d2c003a60 2024-01-27T02:34:08.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.835+0000 7f5d488c9700 1 --2- 172.21.15.7:0/1108307487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d2c0051c0 0x7f5d2c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:08.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.835+0000 7f5d43fff700 1 --2- 172.21.15.7:0/1108307487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d2c003a60 0x7f5d2c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:08.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.835+0000 7f5d437fe700 1 --2- 172.21.15.7:0/1108307487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d2c004810 0x7f5d2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:08.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.835+0000 7f5d488c9700 1 -- 172.21.15.7:0/1108307487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d2c003a60 msgr2=0x7f5d2c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:08.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.835+0000 7f5d488c9700 1 --2- 172.21.15.7:0/1108307487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d2c003a60 0x7f5d2c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:08.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.835+0000 7f5d488c9700 1 -- 172.21.15.7:0/1108307487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d2c004810 msgr2=0x7f5d2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:08.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.835+0000 7f5d488c9700 1 --2- 172.21.15.7:0/1108307487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d2c004810 0x7f5d2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:08.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.835+0000 7f5d488c9700 1 -- 172.21.15.7:0/1108307487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c003570 con 0x7f5d2c0051c0 2024-01-27T02:34:08.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.835+0000 7f5d488c9700 1 -- 172.21.15.7:0/1108307487 reap_dead start 2024-01-27T02:34:08.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.835+0000 7f5d488c9700 1 --2- 172.21.15.7:0/1108307487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d2c0051c0 0x7f5d2c00ba60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0020a0 tx=0x7f5d3c0016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6601841bc30bf353 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:08.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.835+0000 7f5d42ffd700 1 -- 172.21.15.7:0/1108307487 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c0019e0 con 0x7f5d2c0051c0 2024-01-27T02:34:08.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.835+0000 7f5d42ffd700 1 -- 172.21.15.7:0/1108307487 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3c00c8b0 con 0x7f5d2c0051c0 2024-01-27T02:34:08.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.835+0000 7f5d42ffd700 1 -- 172.21.15.7:0/1108307487 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c00cc90 con 0x7f5d2c0051c0 2024-01-27T02:34:08.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.836+0000 7f5d4a32c700 1 -- 172.21.15.7:0/1108307487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d2c0051c0 msgr2=0x7f5d2c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:08.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.836+0000 7f5d4a32c700 1 --2- 172.21.15.7:0/1108307487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d2c0051c0 0x7f5d2c00ba60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0020a0 tx=0x7f5d3c0016d0 comp rx=0 tx=0).stop 2024-01-27T02:34:08.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.836+0000 7f5d4a32c700 1 -- 172.21.15.7:0/1108307487 shutdown_connections 2024-01-27T02:34:08.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.836+0000 7f5d4a32c700 1 --2- 172.21.15.7:0/1108307487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d2c0051c0 0x7f5d2c00ba60 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:08.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.836+0000 7f5d4a32c700 1 -- 172.21.15.7:0/1108307487 >> 172.21.15.7:0/1108307487 conn(0x7f5d44098020 msgr2=0x7f5d440711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:08.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.836+0000 7f5d4a32c700 1 -- 172.21.15.7:0/1108307487 shutdown_connections 2024-01-27T02:34:08.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.836+0000 7f5d4a32c700 1 -- 172.21.15.7:0/1108307487 wait complete. 2024-01-27T02:34:08.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.837+0000 7f5d4a32c700 1 Processor -- start 2024-01-27T02:34:08.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.837+0000 7f5d4a32c700 1 -- start start 2024-01-27T02:34:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.838+0000 7f5d4a32c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d2c0051c0 0x7f5d44143d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.838+0000 7f5d4a32c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d44143290 0x7f5d44143700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.838+0000 7f5d4a32c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d440c76b0 0x7f5d440c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.838+0000 7f5d4a32c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d440c8090 con 0x7f5d44143290 2024-01-27T02:34:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.838+0000 7f5d4a32c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d440b2ad0 con 0x7f5d440c76b0 2024-01-27T02:34:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.838+0000 7f5d4a32c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d440b38d0 con 0x7f5d2c0051c0 2024-01-27T02:34:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.838+0000 7f5d437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d44143290 0x7f5d44143700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.838+0000 7f5d437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d44143290 0x7f5d44143700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58194/0 (socket says 172.21.15.7:58194) 2024-01-27T02:34:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.838+0000 7f5d437fe700 1 -- 172.21.15.7:0/1109315645 learned_addr learned my addr 172.21.15.7:0/1109315645 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.839+0000 7f5d488c9700 1 --2- 172.21.15.7:0/1109315645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d440c76b0 0x7f5d440c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.839+0000 7f5d43fff700 1 --2- 172.21.15.7:0/1109315645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d2c0051c0 0x7f5d44143d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.839+0000 7f5d437fe700 1 -- 172.21.15.7:0/1109315645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d2c0051c0 msgr2=0x7f5d44143d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.839+0000 7f5d437fe700 1 --2- 172.21.15.7:0/1109315645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d2c0051c0 0x7f5d44143d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.839+0000 7f5d437fe700 1 -- 172.21.15.7:0/1109315645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d440c76b0 msgr2=0x7f5d440c7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.839+0000 7f5d437fe700 1 --2- 172.21.15.7:0/1109315645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d440c76b0 0x7f5d440c7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.839+0000 7f5d437fe700 1 -- 172.21.15.7:0/1109315645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d441442d0 con 0x7f5d44143290 2024-01-27T02:34:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.839+0000 7f5d43fff700 1 --2- 172.21.15.7:0/1109315645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d2c0051c0 0x7f5d44143d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.839+0000 7f5d437fe700 1 --2- 172.21.15.7:0/1109315645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d44143290 0x7f5d44143700 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5d38011b10 tx=0x7f5d380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.839+0000 7f5d2bfff700 1 -- 172.21.15.7:0/1109315645 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3800f940 con 0x7f5d44143290 2024-01-27T02:34:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.839+0000 7f5d2bfff700 1 -- 172.21.15.7:0/1109315645 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3800faa0 con 0x7f5d44143290 2024-01-27T02:34:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.839+0000 7f5d2bfff700 1 -- 172.21.15.7:0/1109315645 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3800fe20 con 0x7f5d44143290 2024-01-27T02:34:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.839+0000 7f5d4a32c700 1 -- 172.21.15.7:0/1109315645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d440b27f0 con 0x7f5d44143290 2024-01-27T02:34:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.839+0000 7f5d4a32c700 1 -- 172.21.15.7:0/1109315645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d440b3560 con 0x7f5d44143290 2024-01-27T02:34:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.840+0000 7f5d4a32c700 1 -- 172.21.15.7:0/1109315645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d440b3a80 con 0x7f5d44143290 2024-01-27T02:34:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.841+0000 7f5d2bfff700 1 -- 172.21.15.7:0/1109315645 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5d38014650 con 0x7f5d44143290 2024-01-27T02:34:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.841+0000 7f5d2bfff700 1 --2- 172.21.15.7:0/1109315645 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5d30066f30 0x7f5d300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.841+0000 7f5d2bfff700 1 -- 172.21.15.7:0/1109315645 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f5d38018030 con 0x7f5d44143290 2024-01-27T02:34:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.841+0000 7f5d43fff700 1 --2- 172.21.15.7:0/1109315645 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5d30066f30 0x7f5d300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.842+0000 7f5d43fff700 1 --2- 172.21.15.7:0/1109315645 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5d30066f30 0x7f5d300693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5d34001c30 tx=0x7f5d34009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:08.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:08.846+0000 7f5d2bfff700 1 -- 172.21.15.7:0/1109315645 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d440b3a80 con 0x7f5d44143290 2024-01-27T02:34:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.096+0000 7f5d4a32c700 1 -- 172.21.15.7:0/1109315645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d440b36f0 con 0x7f5d44143290 2024-01-27T02:34:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.097+0000 7f5d2bfff700 1 -- 172.21.15.7:0/1109315645 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f5d38022090 con 0x7f5d44143290 2024-01-27T02:34:09.147 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:34:09.147 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4883,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/198511042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":198511042},{"type":"v1","addr":"172.21.15.188:6836","nonce":198511042}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":4886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1222999254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1222999254},{"type":"v1","addr":"172.21.15.188:6837","nonce":1222999254}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2474016526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2474016526},{"type":"v1","addr":"172.21.15.7:6834","nonce":2474016526}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5556,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3873822002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3873822002},{"type":"v1","addr":"172.21.15.7:6835","nonce":3873822002}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[]} 2024-01-27T02:34:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.099+0000 7f5d4a32c700 1 -- 172.21.15.7:0/1109315645 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5d30066f30 msgr2=0x7f5d300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.099+0000 7f5d4a32c700 1 --2- 172.21.15.7:0/1109315645 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5d30066f30 0x7f5d300693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5d34001c30 tx=0x7f5d34009040 comp rx=0 tx=0).stop 2024-01-27T02:34:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.099+0000 7f5d4a32c700 1 -- 172.21.15.7:0/1109315645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d44143290 msgr2=0x7f5d44143700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.099+0000 7f5d4a32c700 1 --2- 172.21.15.7:0/1109315645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d44143290 0x7f5d44143700 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5d38011b10 tx=0x7f5d380091a0 comp rx=0 tx=0).stop 2024-01-27T02:34:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.099+0000 7f5d4a32c700 1 -- 172.21.15.7:0/1109315645 shutdown_connections 2024-01-27T02:34:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.099+0000 7f5d4a32c700 1 --2- 172.21.15.7:0/1109315645 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5d30066f30 0x7f5d300693e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.099+0000 7f5d4a32c700 1 --2- 172.21.15.7:0/1109315645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d440c76b0 0x7f5d440c7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.099+0000 7f5d4a32c700 1 --2- 172.21.15.7:0/1109315645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d2c0051c0 0x7f5d44143d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.099+0000 7f5d4a32c700 1 --2- 172.21.15.7:0/1109315645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d44143290 0x7f5d44143700 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.099+0000 7f5d4a32c700 1 -- 172.21.15.7:0/1109315645 >> 172.21.15.7:0/1109315645 conn(0x7f5d44098020 msgr2=0x7f5d4406bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.099+0000 7f5d4a32c700 1 -- 172.21.15.7:0/1109315645 shutdown_connections 2024-01-27T02:34:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.099+0000 7f5d4a32c700 1 -- 172.21.15.7:0/1109315645 wait complete. 2024-01-27T02:34:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-01-27T02:34:09.150 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-27T02:34:09.150 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-27T02:34:09.150 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-27T02:34:09.151 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:34:09.151 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-27T02:34:09.151 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-27T02:34:09.151 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-27T02:34:09.151 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:34:09.151 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-27T02:34:09.151 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-27T02:34:09.152 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-27T02:34:09.152 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:34:09.152 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-27T02:34:09.152 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-27T02:34:09.152 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-27T02:34:09.152 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:34:09.163 INFO:tasks.ceph.mds.a:Stopped 2024-01-27T02:34:09.163 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-27T02:34:09.164 INFO:tasks.ceph.mds.c:Stopped 2024-01-27T02:34:09.164 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-27T02:34:09.165 INFO:tasks.ceph.mds.b:Stopped 2024-01-27T02:34:09.165 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-27T02:34:09.166 INFO:tasks.ceph.mds.d:Stopped 2024-01-27T02:34:09.166 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-27T02:34:09.206 INFO:tasks.ceph.mds.a:Started 2024-01-27T02:34:09.206 DEBUG:teuthology.parallel:result is None 2024-01-27T02:34:09.207 INFO:tasks.ceph.mds.c:Started 2024-01-27T02:34:09.207 DEBUG:teuthology.parallel:result is None 2024-01-27T02:34:09.208 INFO:tasks.ceph.mds.b:Started 2024-01-27T02:34:09.208 DEBUG:teuthology.parallel:result is None 2024-01-27T02:34:09.209 INFO:tasks.ceph.mds.d:Started 2024-01-27T02:34:09.209 DEBUG:teuthology.parallel:result is None 2024-01-27T02:34:09.209 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-27T02:34:09.294 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-01-27T02:34:09.294 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-01-27T02:34:09.295 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-27T02:34:09.295 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-27T02:34:09.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.351+0000 7f7bb7a59700 1 Processor -- start 2024-01-27T02:34:09.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.351+0000 7f7bb7a59700 1 -- start start 2024-01-27T02:34:09.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.352+0000 7f7bb7a59700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7bb0156fd0 0x7f7bb00c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:09.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.352+0000 7f7bb7a59700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb006dbb0 con 0x7f7bb0156c00 2024-01-27T02:34:09.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.352+0000 7f7bb7a59700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb00c9c50 con 0x7f7bb00c9830 2024-01-27T02:34:09.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.352+0000 7f7bb7a59700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb00c8bc0 con 0x7f7bb0156fd0 2024-01-27T02:34:09.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.352+0000 7f7bb57f5700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7bb0156c00 0x7f7bb006e040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58622/0 (socket says 172.21.15.7:58622) 2024-01-27T02:34:09.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.352+0000 7f7bb57f5700 1 -- 172.21.15.7:0/1447417195 learned_addr learned my addr 172.21.15.7:0/1447417195 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:34:09.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.352+0000 7f7bb4ff4700 1 --2- 172.21.15.7:0/1447417195 >> v2:172.21.15.188:3300/0 conn(0x7f7bb0156fd0 0x7f7bb00c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:09.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.352+0000 7f7ba3fff700 1 -- 172.21.15.7:0/1447417195 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3290487832 0 0) 0x7f7bb006dbb0 con 0x7f7bb0156c00 2024-01-27T02:34:09.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.352+0000 7f7ba3fff700 1 -- 172.21.15.7:0/1447417195 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b9c003570 con 0x7f7bb0156c00 2024-01-27T02:34:09.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.352+0000 7f7ba3fff700 1 -- 172.21.15.7:0/1447417195 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 68167083 0 0) 0x7f7bb00c9c50 con 0x7f7bb00c9830 2024-01-27T02:34:09.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.352+0000 7f7ba3fff700 1 -- 172.21.15.7:0/1447417195 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bb006dbb0 con 0x7f7bb00c9830 2024-01-27T02:34:09.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.353+0000 7f7ba3fff700 1 -- 172.21.15.7:0/1447417195 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 892813951 0 0) 0x7f7b9c003570 con 0x7f7bb0156c00 2024-01-27T02:34:09.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.353+0000 7f7bb4ff4700 1 -- 172.21.15.7:0/1447417195 >> v1:172.21.15.188:6789/0 conn(0x7f7bb00c9830 legacy=0x7f7bb00c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:09.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.353+0000 7f7bb4ff4700 1 -- 172.21.15.7:0/1447417195 >> v1:172.21.15.7:6789/0 conn(0x7f7bb0156c00 legacy=0x7f7bb006e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:09.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.353+0000 7f7bb4ff4700 1 -- 172.21.15.7:0/1447417195 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb006e750 con 0x7f7bb0156fd0 2024-01-27T02:34:09.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.353+0000 7f7bb4ff4700 1 --2- 172.21.15.7:0/1447417195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bb0156fd0 0x7f7bb00c92f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0098e0 tx=0x7f7bac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edbed93ae257c20f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:09.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.353+0000 7f7ba3fff700 1 -- 172.21.15.7:0/1447417195 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4001470 con 0x7f7bb0156fd0 2024-01-27T02:34:09.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.353+0000 7f7ba3fff700 1 -- 172.21.15.7:0/1447417195 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac011420 con 0x7f7bb0156fd0 2024-01-27T02:34:09.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.353+0000 7f7ba3fff700 1 -- 172.21.15.7:0/1447417195 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac0117c0 con 0x7f7bb0156fd0 2024-01-27T02:34:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.354+0000 7f7bb7a59700 1 -- 172.21.15.7:0/1447417195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bb0156fd0 msgr2=0x7f7bb00c92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.354+0000 7f7bb7a59700 1 --2- 172.21.15.7:0/1447417195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bb0156fd0 0x7f7bb00c92f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0098e0 tx=0x7f7bac0091a0 comp rx=0 tx=0).stop 2024-01-27T02:34:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.354+0000 7f7bb7a59700 1 -- 172.21.15.7:0/1447417195 shutdown_connections 2024-01-27T02:34:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.354+0000 7f7bb7a59700 1 --2- 172.21.15.7:0/1447417195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bb0156fd0 0x7f7bb00c92f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.354+0000 7f7bb7a59700 1 -- 172.21.15.7:0/1447417195 >> 172.21.15.7:0/1447417195 conn(0x7f7bb0063580 msgr2=0x7f7bb0063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.354+0000 7f7bb7a59700 1 -- 172.21.15.7:0/1447417195 shutdown_connections 2024-01-27T02:34:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.354+0000 7f7bb7a59700 1 -- 172.21.15.7:0/1447417195 wait complete. 2024-01-27T02:34:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.355+0000 7f7bb7a59700 1 Processor -- start 2024-01-27T02:34:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.355+0000 7f7bb7a59700 1 -- start start 2024-01-27T02:34:09.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.356+0000 7f7bb7a59700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb00c9830 0x7f7bb00b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:09.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.356+0000 7f7bb7a59700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bb0156c00 0x7f7bb0158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:09.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.356+0000 7f7bb7a59700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bb0156fd0 0x7f7bb0154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:09.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.356+0000 7f7bb7a59700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb01594e0 con 0x7f7bb00c9830 2024-01-27T02:34:09.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.356+0000 7f7bb7a59700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb00b1bb0 con 0x7f7bb0156fd0 2024-01-27T02:34:09.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.356+0000 7f7bb7a59700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb01588d0 con 0x7f7bb0156c00 2024-01-27T02:34:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.356+0000 7f7bb57f5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb00c9830 0x7f7bb00b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.356+0000 7f7bb57f5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb00c9830 0x7f7bb00b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58238/0 (socket says 172.21.15.7:58238) 2024-01-27T02:34:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.356+0000 7f7bb57f5700 1 -- 172.21.15.7:0/1480408621 learned_addr learned my addr 172.21.15.7:0/1480408621 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.356+0000 7f7bb4ff4700 1 --2- 172.21.15.7:0/1480408621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bb0156c00 0x7f7bb0158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.356+0000 7f7bb5ff6700 1 --2- 172.21.15.7:0/1480408621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bb0156fd0 0x7f7bb0154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.356+0000 7f7bb57f5700 1 -- 172.21.15.7:0/1480408621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bb0156c00 msgr2=0x7f7bb0158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.356+0000 7f7bb57f5700 1 --2- 172.21.15.7:0/1480408621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bb0156c00 0x7f7bb0158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.356+0000 7f7bb57f5700 1 -- 172.21.15.7:0/1480408621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bb0156fd0 msgr2=0x7f7bb0154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.356+0000 7f7bb57f5700 1 --2- 172.21.15.7:0/1480408621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bb0156fd0 0x7f7bb0154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.356+0000 7f7bb57f5700 1 -- 172.21.15.7:0/1480408621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac009420 con 0x7f7bb00c9830 2024-01-27T02:34:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.356+0000 7f7bb5ff6700 1 --2- 172.21.15.7:0/1480408621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bb0156fd0 0x7f7bb0154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.356+0000 7f7bb4ff4700 1 --2- 172.21.15.7:0/1480408621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bb0156c00 0x7f7bb0158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.357+0000 7f7bb57f5700 1 --2- 172.21.15.7:0/1480408621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb00c9830 0x7f7bb00b1e50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400cb70 tx=0x7f7ba4001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.357+0000 7f7ba17fa700 1 -- 172.21.15.7:0/1480408621 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4007360 con 0x7f7bb00c9830 2024-01-27T02:34:09.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.357+0000 7f7ba17fa700 1 -- 172.21.15.7:0/1480408621 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba40074e0 con 0x7f7bb00c9830 2024-01-27T02:34:09.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.357+0000 7f7ba17fa700 1 -- 172.21.15.7:0/1480408621 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4004830 con 0x7f7bb00c9830 2024-01-27T02:34:09.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.357+0000 7f7bb7a59700 1 -- 172.21.15.7:0/1480408621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb0158bb0 con 0x7f7bb00c9830 2024-01-27T02:34:09.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.357+0000 7f7bb7a59700 1 -- 172.21.15.7:0/1480408621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb0159170 con 0x7f7bb00c9830 2024-01-27T02:34:09.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.358+0000 7f7bb7a59700 1 -- 172.21.15.7:0/1480408621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb004c710 con 0x7f7bb00c9830 2024-01-27T02:34:09.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.358+0000 7f7ba17fa700 1 -- 172.21.15.7:0/1480408621 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7ba4004ba0 con 0x7f7bb00c9830 2024-01-27T02:34:09.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.359+0000 7f7ba17fa700 1 --2- 172.21.15.7:0/1480408621 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7b94066f30 0x7f7b940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:09.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.359+0000 7f7ba17fa700 1 -- 172.21.15.7:0/1480408621 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7ba4083660 con 0x7f7bb00c9830 2024-01-27T02:34:09.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.359+0000 7f7bb4ff4700 1 --2- 172.21.15.7:0/1480408621 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7b94066f30 0x7f7b940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:09.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.360+0000 7f7bb4ff4700 1 --2- 172.21.15.7:0/1480408621 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7b94066f30 0x7f7b940693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0093f0 tx=0x7f7bac016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:09.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.363+0000 7f7ba17fa700 1 -- 172.21.15.7:0/1480408621 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ba4050c50 con 0x7f7bb00c9830 2024-01-27T02:34:09.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:09.589+0000 7f7bb7a59700 1 -- 172.21.15.7:0/1480408621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7bb00c88e0 con 0x7f7bb00c9830 2024-01-27T02:34:10.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.830+0000 7f7ba17fa700 1 -- 172.21.15.7:0/1480408621 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v61) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7ba4018080 con 0x7f7bb00c9830 2024-01-27T02:34:10.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.833+0000 7f7bb7a59700 1 -- 172.21.15.7:0/1480408621 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7b94066f30 msgr2=0x7f7b940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:10.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.833+0000 7f7bb7a59700 1 --2- 172.21.15.7:0/1480408621 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7b94066f30 0x7f7b940693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0093f0 tx=0x7f7bac016000 comp rx=0 tx=0).stop 2024-01-27T02:34:10.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.833+0000 7f7bb7a59700 1 -- 172.21.15.7:0/1480408621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb00c9830 msgr2=0x7f7bb00b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:10.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.833+0000 7f7bb7a59700 1 --2- 172.21.15.7:0/1480408621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb00c9830 0x7f7bb00b1e50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400cb70 tx=0x7f7ba4001b50 comp rx=0 tx=0).stop 2024-01-27T02:34:10.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.833+0000 7f7bb7a59700 1 -- 172.21.15.7:0/1480408621 shutdown_connections 2024-01-27T02:34:10.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.833+0000 7f7bb7a59700 1 --2- 172.21.15.7:0/1480408621 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7b94066f30 0x7f7b940693e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:10.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.833+0000 7f7bb7a59700 1 --2- 172.21.15.7:0/1480408621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bb0156fd0 0x7f7bb0154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:10.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.833+0000 7f7bb7a59700 1 --2- 172.21.15.7:0/1480408621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb00c9830 0x7f7bb00b1e50 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:10.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.833+0000 7f7bb7a59700 1 --2- 172.21.15.7:0/1480408621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bb0156c00 0x7f7bb0158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:10.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.833+0000 7f7bb7a59700 1 -- 172.21.15.7:0/1480408621 >> 172.21.15.7:0/1480408621 conn(0x7f7bb0063580 msgr2=0x7f7bb00cc060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:10.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.833+0000 7f7bb7a59700 1 -- 172.21.15.7:0/1480408621 shutdown_connections 2024-01-27T02:34:10.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.833+0000 7f7bb7a59700 1 -- 172.21.15.7:0/1480408621 wait complete. 2024-01-27T02:34:10.835 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-01-27T02:34:10.849 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-27T02:34:10.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.984+0000 7effaabcf700 1 Processor -- start 2024-01-27T02:34:10.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.984+0000 7effaabcf700 1 -- start start 2024-01-27T02:34:10.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.985+0000 7effaabcf700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effa40c3ae0 0x7effa40cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:10.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.985+0000 7effaabcf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7effa4158070 0x7effa40cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:10.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.985+0000 7effaabcf700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa40ca020 con 0x7effa4157ca0 2024-01-27T02:34:10.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.985+0000 7effaabcf700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa40c5210 con 0x7effa4158070 2024-01-27T02:34:10.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.985+0000 7effaabcf700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa40c54f0 con 0x7effa40c3ae0 2024-01-27T02:34:10.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.985+0000 7effa916c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7effa4158070 0x7effa40cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:10.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.985+0000 7effa916c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7effa4158070 0x7effa40cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58252/0 (socket says 172.21.15.7:58252) 2024-01-27T02:34:10.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.985+0000 7effa916c700 1 -- 172.21.15.7:0/3997660150 learned_addr learned my addr 172.21.15.7:0/3997660150 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:10.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.985+0000 7effa3fff700 1 --2- 172.21.15.7:0/3997660150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effa40c3ae0 0x7effa40cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:10.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.985+0000 7effa916c700 1 -- 172.21.15.7:0/3997660150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effa40c3ae0 msgr2=0x7effa40cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:10.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.985+0000 7effa916c700 1 --2- 172.21.15.7:0/3997660150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effa40c3ae0 0x7effa40cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:10.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.985+0000 7effa916c700 1 -- 172.21.15.7:0/3997660150 >> v1:172.21.15.188:6789/0 conn(0x7effa4157ca0 legacy=0x7effa40c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:10.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.985+0000 7effa916c700 1 -- 172.21.15.7:0/3997660150 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa40c57d0 con 0x7effa4158070 2024-01-27T02:34:10.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.986+0000 7effa916c700 1 --2- 172.21.15.7:0/3997660150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effa4158070 0x7effa40cda20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7eff9c009f20 tx=0x7eff9c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cdb824aab4ab9f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:10.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.986+0000 7effa37fe700 1 -- 172.21.15.7:0/3997660150 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c00e070 con 0x7effa4158070 2024-01-27T02:34:10.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.986+0000 7effa37fe700 1 -- 172.21.15.7:0/3997660150 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9c00b9d0 con 0x7effa4158070 2024-01-27T02:34:10.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.986+0000 7effa37fe700 1 -- 172.21.15.7:0/3997660150 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c0092e0 con 0x7effa4158070 2024-01-27T02:34:10.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.986+0000 7effaabcf700 1 -- 172.21.15.7:0/3997660150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effa4158070 msgr2=0x7effa40cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:10.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.986+0000 7effaabcf700 1 --2- 172.21.15.7:0/3997660150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effa4158070 0x7effa40cda20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7eff9c009f20 tx=0x7eff9c009840 comp rx=0 tx=0).stop 2024-01-27T02:34:10.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.986+0000 7effaabcf700 1 -- 172.21.15.7:0/3997660150 shutdown_connections 2024-01-27T02:34:10.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.986+0000 7effaabcf700 1 --2- 172.21.15.7:0/3997660150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effa4158070 0x7effa40cda20 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:10.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.986+0000 7effaabcf700 1 --2- 172.21.15.7:0/3997660150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effa40c3ae0 0x7effa40cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:10.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.986+0000 7effaabcf700 1 -- 172.21.15.7:0/3997660150 >> 172.21.15.7:0/3997660150 conn(0x7effa4098020 msgr2=0x7effa40711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:10.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.986+0000 7effaabcf700 1 -- 172.21.15.7:0/3997660150 shutdown_connections 2024-01-27T02:34:10.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.987+0000 7effaabcf700 1 -- 172.21.15.7:0/3997660150 wait complete. 2024-01-27T02:34:10.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.988+0000 7effaabcf700 1 Processor -- start 2024-01-27T02:34:10.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.988+0000 7effaabcf700 1 -- start start 2024-01-27T02:34:10.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.989+0000 7effaabcf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effa40c3ae0 0x7effa40c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:10.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.989+0000 7effaabcf700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effa4157ca0 0x7effa41517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:10.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.989+0000 7effaabcf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effa4158070 0x7effa4154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:10.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.989+0000 7effaabcf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa40c7cf0 con 0x7effa4158070 2024-01-27T02:34:10.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.989+0000 7effaabcf700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa40c7730 con 0x7effa40c3ae0 2024-01-27T02:34:10.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.989+0000 7effaabcf700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa40c7a10 con 0x7effa4157ca0 2024-01-27T02:34:10.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.989+0000 7effa916c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effa4158070 0x7effa4154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:10.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.989+0000 7effa916c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effa4158070 0x7effa4154510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58254/0 (socket says 172.21.15.7:58254) 2024-01-27T02:34:10.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.989+0000 7effa916c700 1 -- 172.21.15.7:0/4081390695 learned_addr learned my addr 172.21.15.7:0/4081390695 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:10.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.989+0000 7effa3fff700 1 --2- 172.21.15.7:0/4081390695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effa4157ca0 0x7effa41517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:10.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.989+0000 7effa896b700 1 --2- 172.21.15.7:0/4081390695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effa40c3ae0 0x7effa40c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:10.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.989+0000 7effa916c700 1 -- 172.21.15.7:0/4081390695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effa4157ca0 msgr2=0x7effa41517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:10.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.989+0000 7effa916c700 1 --2- 172.21.15.7:0/4081390695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effa4157ca0 0x7effa41517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:10.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.989+0000 7effa916c700 1 -- 172.21.15.7:0/4081390695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effa40c3ae0 msgr2=0x7effa40c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:10.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.989+0000 7effa916c700 1 --2- 172.21.15.7:0/4081390695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effa40c3ae0 0x7effa40c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:10.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.989+0000 7effa916c700 1 -- 172.21.15.7:0/4081390695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c009ae0 con 0x7effa4158070 2024-01-27T02:34:10.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.989+0000 7effa3fff700 1 --2- 172.21.15.7:0/4081390695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effa4157ca0 0x7effa41517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:10.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.989+0000 7effa916c700 1 --2- 172.21.15.7:0/4081390695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effa4158070 0x7effa4154510 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00b610 tx=0x7eff9c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:10.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.989+0000 7effa0ff9700 1 -- 172.21.15.7:0/4081390695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c008c30 con 0x7effa4158070 2024-01-27T02:34:10.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.990+0000 7effa0ff9700 1 -- 172.21.15.7:0/4081390695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9c008d90 con 0x7effa4158070 2024-01-27T02:34:10.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.990+0000 7effaabcf700 1 -- 172.21.15.7:0/4081390695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa4153d10 con 0x7effa4158070 2024-01-27T02:34:10.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.990+0000 7effa0ff9700 1 -- 172.21.15.7:0/4081390695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c01c640 con 0x7effa4158070 2024-01-27T02:34:10.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.990+0000 7effaabcf700 1 -- 172.21.15.7:0/4081390695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa40c7300 con 0x7effa4158070 2024-01-27T02:34:10.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.991+0000 7effaabcf700 1 -- 172.21.15.7:0/4081390695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa404c710 con 0x7effa4158070 2024-01-27T02:34:10.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.991+0000 7effa0ff9700 1 -- 172.21.15.7:0/4081390695 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7eff9c003040 con 0x7effa4158070 2024-01-27T02:34:10.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.991+0000 7effa0ff9700 1 --2- 172.21.15.7:0/4081390695 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7eff88066f30 0x7eff880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:10.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.992+0000 7effa0ff9700 1 -- 172.21.15.7:0/4081390695 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4853+0+0 (secure 0 0 0) 0x7eff9c085aa0 con 0x7effa4158070 2024-01-27T02:34:10.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.992+0000 7effa896b700 1 --2- 172.21.15.7:0/4081390695 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7eff88066f30 0x7eff880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:10.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.992+0000 7effa896b700 1 --2- 172.21.15.7:0/4081390695 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7eff88066f30 0x7eff880693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7eff98003860 tx=0x7eff98009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:10.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:10.996+0000 7effa0ff9700 1 -- 172.21.15.7:0/4081390695 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff9c053060 con 0x7effa4158070 2024-01-27T02:34:11.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.279+0000 7effaabcf700 1 -- 172.21.15.7:0/4081390695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7effa40c9bb0 con 0x7effa4158070 2024-01-27T02:34:11.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.280+0000 7effa0ff9700 1 -- 172.21.15.7:0/4081390695 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7eff9c056680 con 0x7effa4158070 2024-01-27T02:34:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:34:11.281 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:34:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-01-27T02:34:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:34:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-01-27T02:34:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlHdcPKBAAfhnh1I3ZSQEXWJXlXCs98Q==", 2024-01-27T02:34:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:34:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:34:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:34:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:34:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:34:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:34:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:34:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:34:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-01-27T02:34:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rl8eVYNRAAfP8rOqO50RpoNZ1+1j2DXQ==", 2024-01-27T02:34:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:34:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:34:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:34:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:34:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:34:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:34:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:34:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:34:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-01-27T02:34:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlJxRKMBAAS0j7/zZeqGglOxCK1pg0WA==", 2024-01-27T02:34:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:34:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:34:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:34:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:34:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:34:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:34:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:34:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:34:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-01-27T02:34:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlNwfUABAALRQAoHoGhniqCvTQOqbbug==", 2024-01-27T02:34:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:34:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:34:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:34:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:34:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:34:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:34:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:34:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:34:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-01-27T02:34:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7Rlh6VSJhAAttuCte1WhNEvUkv5x/IKQQ==", 2024-01-27T02:34:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:34:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:34:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:34:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:34:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:34:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:34:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:34:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-01-27T02:34:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7Rlu/RGLhAAKzaCNhzYKhsNhxkMvY++gw==", 2024-01-27T02:34:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:34:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:34:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:34:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:34:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:34:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:34:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:34:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-01-27T02:34:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7RliRMyNhAAr0tG3evmVV7ZlGxbxfWPMw==", 2024-01-27T02:34:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:34:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:34:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:34:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:34:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:34:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:34:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:34:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-01-27T02:34:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAPa7RlRuSNAhAA3e9atlb+ffvlnLhRYbWZPw==", 2024-01-27T02:34:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:34:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:34:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:34:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:34:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:34:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:34:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:34:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-01-27T02:34:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7RlfeA6DBAAXpklgbNHRO+5G+fdIERjRw==", 2024-01-27T02:34:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:34:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:34:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:34:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:34:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:34:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:34:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:34:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-01-27T02:34:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7Rl77ItExAABB7uABm63ubE/YUs2B5pGw==", 2024-01-27T02:34:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:34:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:34:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:34:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:34:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:34:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:34:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:34:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-01-27T02:34:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7RljDuwHBAAPPAlhoqS03qPpq58jmevLA==", 2024-01-27T02:34:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:34:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:34:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:34:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:34:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:34:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:34:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:34:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-01-27T02:34:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7Rl1YElJhAAqHP/QEdl1SEalGcbQwPwAg==", 2024-01-27T02:34:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:34:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:34:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:34:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:34:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:34:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:34:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:34:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-01-27T02:34:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlaEqaBRAATFiIeW8lT9BdNuN0m3x9dQ==", 2024-01-27T02:34:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:34:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:34:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-01-27T02:34:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-27T02:34:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:34:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:34:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:34:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-01-27T02:34:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlNADmEBAAJXwdJ35ylP5coy1dtHoD3g==", 2024-01-27T02:34:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:34:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:34:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:34:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:34:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:34:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:34:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:34:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:34:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-01-27T02:34:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7Rl5fRqCRAA/5swTGknnpP64TAh+2X41A==", 2024-01-27T02:34:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:34:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:34:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:34:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:34:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:34:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:34:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:34:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:34:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-01-27T02:34:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7Rl0SdwFhAAhCmw9ViiJbHVfS5a2r/D+w==", 2024-01-27T02:34:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:34:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:34:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:34:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:34:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:34:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:34:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:34:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:34:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-01-27T02:34:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlsNgPARAA0Qm0oJgA8GO1oUT3ndT8Tg==", 2024-01-27T02:34:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:34:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:34:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:34:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:34:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:34:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:34:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:34:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:34:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-01-27T02:34:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rlfb7qIxAAlsJYiSolJ7D8EykGWP62bA==", 2024-01-27T02:34:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:34:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:34:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-27T02:34:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:34:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:34:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:34:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:34:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-01-27T02:34:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rls7TaHxAALS3ghxKtfmgQfIELiD9+Bg==", 2024-01-27T02:34:11.293 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:34:11.293 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:34:11.293 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-27T02:34:11.293 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:34:11.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:34:11.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:34:11.293 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:34:11.293 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:34:11.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.282+0000 7effaabcf700 1 -- 172.21.15.7:0/4081390695 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7eff88066f30 msgr2=0x7eff880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:11.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.282+0000 7effaabcf700 1 --2- 172.21.15.7:0/4081390695 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7eff88066f30 0x7eff880693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7eff98003860 tx=0x7eff98009040 comp rx=0 tx=0).stop 2024-01-27T02:34:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.282+0000 7effaabcf700 1 -- 172.21.15.7:0/4081390695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effa4158070 msgr2=0x7effa4154510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.282+0000 7effaabcf700 1 --2- 172.21.15.7:0/4081390695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effa4158070 0x7effa4154510 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00b610 tx=0x7eff9c018000 comp rx=0 tx=0).stop 2024-01-27T02:34:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.282+0000 7effaabcf700 1 -- 172.21.15.7:0/4081390695 shutdown_connections 2024-01-27T02:34:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.282+0000 7effaabcf700 1 --2- 172.21.15.7:0/4081390695 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7eff88066f30 0x7eff880693e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.282+0000 7effaabcf700 1 --2- 172.21.15.7:0/4081390695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effa4158070 0x7effa4154510 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.282+0000 7effaabcf700 1 --2- 172.21.15.7:0/4081390695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effa40c3ae0 0x7effa40c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.282+0000 7effaabcf700 1 --2- 172.21.15.7:0/4081390695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effa4157ca0 0x7effa41517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.282+0000 7effaabcf700 1 -- 172.21.15.7:0/4081390695 >> 172.21.15.7:0/4081390695 conn(0x7effa4098020 msgr2=0x7effa40b3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.282+0000 7effaabcf700 1 -- 172.21.15.7:0/4081390695 shutdown_connections 2024-01-27T02:34:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.282+0000 7effaabcf700 1 -- 172.21.15.7:0/4081390695 wait complete. 2024-01-27T02:34:11.297 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:34:11.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.427+0000 7f19848a7700 1 Processor -- start 2024-01-27T02:34:11.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.427+0000 7f19848a7700 1 -- start start 2024-01-27T02:34:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.428+0000 7f19848a7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198006ea40 0x7f1980157f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.428+0000 7f19848a7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198006c880 con 0x7f19800c9f80 2024-01-27T02:34:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.428+0000 7f19848a7700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198006bc70 con 0x7f19800c9bb0 2024-01-27T02:34:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.428+0000 7f19848a7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198006cd10 con 0x7f198006ea40 2024-01-27T02:34:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.429+0000 7f197dd9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f19800c9f80 0x7f198006e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42102/0 (socket says 172.21.15.7:42102) 2024-01-27T02:34:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.429+0000 7f197dd9b700 1 -- 172.21.15.7:0/3606113803 learned_addr learned my addr 172.21.15.7:0/3606113803 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:34:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.429+0000 7f197ed9d700 1 --2- 172.21.15.7:0/3606113803 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198006ea40 0x7f1980157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.429+0000 7f197d59a700 1 -- 172.21.15.7:0/3606113803 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4427650 0 0) 0x7f198006c880 con 0x7f19800c9f80 2024-01-27T02:34:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.429+0000 7f197d59a700 1 -- 172.21.15.7:0/3606113803 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1968003570 con 0x7f19800c9f80 2024-01-27T02:34:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.429+0000 7f197d59a700 1 -- 172.21.15.7:0/3606113803 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1554174425 0 0) 0x7f198006bc70 con 0x7f19800c9bb0 2024-01-27T02:34:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.429+0000 7f197d59a700 1 -- 172.21.15.7:0/3606113803 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f198006c880 con 0x7f19800c9bb0 2024-01-27T02:34:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.429+0000 7f197ed9d700 1 -- 172.21.15.7:0/3606113803 >> v1:172.21.15.188:6789/0 conn(0x7f19800c9bb0 legacy=0x7f198006b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.429+0000 7f197ed9d700 1 -- 172.21.15.7:0/3606113803 >> v1:172.21.15.7:6789/0 conn(0x7f19800c9f80 legacy=0x7f198006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.429+0000 7f197ed9d700 1 -- 172.21.15.7:0/3606113803 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198006bf50 con 0x7f198006ea40 2024-01-27T02:34:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.429+0000 7f197ed9d700 1 --2- 172.21.15.7:0/3606113803 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198006ea40 0x7f1980157f70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1980157480 tx=0x7f1978009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2005ca81384d306f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.430+0000 7f197d59a700 1 -- 172.21.15.7:0/3606113803 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978016070 con 0x7f198006ea40 2024-01-27T02:34:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.430+0000 7f197d59a700 1 -- 172.21.15.7:0/3606113803 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197800bab0 con 0x7f198006ea40 2024-01-27T02:34:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.430+0000 7f197d59a700 1 -- 172.21.15.7:0/3606113803 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197800e070 con 0x7f198006ea40 2024-01-27T02:34:11.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.430+0000 7f19848a7700 1 -- 172.21.15.7:0/3606113803 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198006ea40 msgr2=0x7f1980157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:11.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.430+0000 7f19848a7700 1 --2- 172.21.15.7:0/3606113803 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198006ea40 0x7f1980157f70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1980157480 tx=0x7f1978009780 comp rx=0 tx=0).stop 2024-01-27T02:34:11.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.430+0000 7f19848a7700 1 -- 172.21.15.7:0/3606113803 shutdown_connections 2024-01-27T02:34:11.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.430+0000 7f19848a7700 1 --2- 172.21.15.7:0/3606113803 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198006ea40 0x7f1980157f70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:11.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.430+0000 7f19848a7700 1 -- 172.21.15.7:0/3606113803 >> 172.21.15.7:0/3606113803 conn(0x7f19800cdc30 msgr2=0x7f19800ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:11.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.430+0000 7f19848a7700 1 -- 172.21.15.7:0/3606113803 shutdown_connections 2024-01-27T02:34:11.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.430+0000 7f19848a7700 1 -- 172.21.15.7:0/3606113803 wait complete. 2024-01-27T02:34:11.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.432+0000 7f19848a7700 1 Processor -- start 2024-01-27T02:34:11.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.432+0000 7f19848a7700 1 -- start start 2024-01-27T02:34:11.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.432+0000 7f19848a7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198006ea40 0x7f19800aff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:11.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.432+0000 7f19848a7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19800c9bb0 0x7f19800b2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:11.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.432+0000 7f19848a7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19800c9f80 0x7f198015a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:11.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.432+0000 7f19848a7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19800c5af0 con 0x7f19800c9f80 2024-01-27T02:34:11.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.432+0000 7f19848a7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19800b2e00 con 0x7f19800c9bb0 2024-01-27T02:34:11.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.432+0000 7f19848a7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19800b30e0 con 0x7f198006ea40 2024-01-27T02:34:11.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.432+0000 7f197ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19800c9f80 0x7f198015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:11.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.433+0000 7f197ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19800c9f80 0x7f198015a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52648/0 (socket says 172.21.15.7:52648) 2024-01-27T02:34:11.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.433+0000 7f197ed9d700 1 -- 172.21.15.7:0/1548240401 learned_addr learned my addr 172.21.15.7:0/1548240401 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:11.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.433+0000 7f197dd9b700 1 --2- 172.21.15.7:0/1548240401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19800c9bb0 0x7f19800b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:11.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.433+0000 7f197e59c700 1 --2- 172.21.15.7:0/1548240401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198006ea40 0x7f19800aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.433+0000 7f197ed9d700 1 -- 172.21.15.7:0/1548240401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198006ea40 msgr2=0x7f19800aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.433+0000 7f197ed9d700 1 --2- 172.21.15.7:0/1548240401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198006ea40 0x7f19800aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.433+0000 7f197ed9d700 1 -- 172.21.15.7:0/1548240401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19800c9bb0 msgr2=0x7f19800b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.433+0000 7f197ed9d700 1 --2- 172.21.15.7:0/1548240401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19800c9bb0 0x7f19800b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.433+0000 7f197ed9d700 1 -- 172.21.15.7:0/1548240401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1978009a20 con 0x7f19800c9f80 2024-01-27T02:34:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.433+0000 7f197dd9b700 1 --2- 172.21.15.7:0/1548240401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19800c9bb0 0x7f19800b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.433+0000 7f197e59c700 1 --2- 172.21.15.7:0/1548240401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198006ea40 0x7f19800aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.433+0000 7f197ed9d700 1 --2- 172.21.15.7:0/1548240401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19800c9f80 0x7f198015a830 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f19800ced40 tx=0x7f1978018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.433+0000 7f196e7fc700 1 -- 172.21.15.7:0/1548240401 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978027470 con 0x7f19800c9f80 2024-01-27T02:34:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.433+0000 7f196e7fc700 1 -- 172.21.15.7:0/1548240401 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19780275d0 con 0x7f19800c9f80 2024-01-27T02:34:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.433+0000 7f19848a7700 1 -- 172.21.15.7:0/1548240401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19800c5780 con 0x7f19800c9f80 2024-01-27T02:34:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.433+0000 7f196e7fc700 1 -- 172.21.15.7:0/1548240401 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978027990 con 0x7f19800c9f80 2024-01-27T02:34:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.433+0000 7f19848a7700 1 -- 172.21.15.7:0/1548240401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19800b28a0 con 0x7f19800c9f80 2024-01-27T02:34:11.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.434+0000 7f19848a7700 1 -- 172.21.15.7:0/1548240401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19800b3320 con 0x7f19800c9f80 2024-01-27T02:34:11.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.435+0000 7f196e7fc700 1 -- 172.21.15.7:0/1548240401 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1978024070 con 0x7f19800c9f80 2024-01-27T02:34:11.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.435+0000 7f196e7fc700 1 --2- 172.21.15.7:0/1548240401 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1960066f30 0x7f19600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.435+0000 7f196e7fc700 1 -- 172.21.15.7:0/1548240401 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1978086920 con 0x7f19800c9f80 2024-01-27T02:34:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.436+0000 7f197e59c700 1 --2- 172.21.15.7:0/1548240401 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1960066f30 0x7f19600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.436+0000 7f197e59c700 1 --2- 172.21.15.7:0/1548240401 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1960066f30 0x7f19600693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f19800cd640 tx=0x7f1970009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:11.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.440+0000 7f196e7fc700 1 -- 172.21.15.7:0/1548240401 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f19800b3320 con 0x7f19800c9f80 2024-01-27T02:34:11.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.643+0000 7f19848a7700 1 -- 172.21.15.7:0/1548240401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19800c5ca0 con 0x7f19800c9f80 2024-01-27T02:34:11.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.644+0000 7f196e7fc700 1 -- 172.21.15.7:0/1548240401 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+12086 (secure 0 0 0) 0x7f19800c5ca0 con 0x7f19800c9f80 2024-01-27T02:34:11.645 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:34:11.645 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":61,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:34:10.826870+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:34:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.647+0000 7f19848a7700 1 -- 172.21.15.7:0/1548240401 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1960066f30 msgr2=0x7f19600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.647+0000 7f19848a7700 1 --2- 172.21.15.7:0/1548240401 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1960066f30 0x7f19600693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f19800cd640 tx=0x7f1970009040 comp rx=0 tx=0).stop 2024-01-27T02:34:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.647+0000 7f19848a7700 1 -- 172.21.15.7:0/1548240401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19800c9f80 msgr2=0x7f198015a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.647+0000 7f19848a7700 1 --2- 172.21.15.7:0/1548240401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19800c9f80 0x7f198015a830 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f19800ced40 tx=0x7f1978018000 comp rx=0 tx=0).stop 2024-01-27T02:34:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.647+0000 7f19848a7700 1 -- 172.21.15.7:0/1548240401 shutdown_connections 2024-01-27T02:34:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.647+0000 7f19848a7700 1 --2- 172.21.15.7:0/1548240401 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1960066f30 0x7f19600693e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.647+0000 7f19848a7700 1 --2- 172.21.15.7:0/1548240401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19800c9f80 0x7f198015a830 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.647+0000 7f19848a7700 1 --2- 172.21.15.7:0/1548240401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198006ea40 0x7f19800aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.647+0000 7f19848a7700 1 --2- 172.21.15.7:0/1548240401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19800c9bb0 0x7f19800b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.647+0000 7f19848a7700 1 -- 172.21.15.7:0/1548240401 >> 172.21.15.7:0/1548240401 conn(0x7f19800cdc30 msgr2=0x7f1980158540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.647+0000 7f19848a7700 1 -- 172.21.15.7:0/1548240401 shutdown_connections 2024-01-27T02:34:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.647+0000 7f19848a7700 1 -- 172.21.15.7:0/1548240401 wait complete. 2024-01-27T02:34:11.661 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-27T02:34:11.661 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:34:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.795+0000 7f01039b9700 1 Processor -- start 2024-01-27T02:34:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.796+0000 7f01039b9700 1 -- start start 2024-01-27T02:34:11.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.796+0000 7f01039b9700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f00fc1416c0 0x7f00fc0cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:11.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.797+0000 7f01039b9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f00fc0cb210 0x7f00fc0cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:11.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.797+0000 7f01039b9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00fc15c7d0 con 0x7f00fc141a90 2024-01-27T02:34:11.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.797+0000 7f01039b9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00fc0cd9c0 con 0x7f00fc0cb210 2024-01-27T02:34:11.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.797+0000 7f01039b9700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00fc0c9640 con 0x7f00fc1416c0 2024-01-27T02:34:11.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.797+0000 7f0101f56700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f00fc0cb210 0x7f00fc0cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:11.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.797+0000 7f0100f54700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f00fc141a90 0x7f00fc0cab00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42116/0 (socket says 172.21.15.7:42116) 2024-01-27T02:34:11.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.797+0000 7f0101f56700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f00fc0cb210 0x7f00fc0cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52650/0 (socket says 172.21.15.7:52650) 2024-01-27T02:34:11.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.797+0000 7f0100f54700 1 -- 172.21.15.7:0/4034225594 learned_addr learned my addr 172.21.15.7:0/4034225594 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:34:11.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.797+0000 7f0101755700 1 --2- 172.21.15.7:0/4034225594 >> v2:172.21.15.188:3300/0 conn(0x7f00fc1416c0 0x7f00fc0cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:11.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.797+0000 7f0101f56700 1 -- 172.21.15.7:0/4034225594 >> v2:172.21.15.188:3300/0 conn(0x7f00fc1416c0 msgr2=0x7f00fc0cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:11.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.797+0000 7f0101f56700 1 --2- 172.21.15.7:0/4034225594 >> v2:172.21.15.188:3300/0 conn(0x7f00fc1416c0 0x7f00fc0cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:11.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.797+0000 7f0101f56700 1 -- 172.21.15.7:0/4034225594 >> v1:172.21.15.7:6789/0 conn(0x7f00fc141a90 legacy=0x7f00fc0cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:11.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.797+0000 7f0101f56700 1 -- 172.21.15.7:0/4034225594 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc15bbc0 con 0x7f00fc0cb210 2024-01-27T02:34:11.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.797+0000 7f00effff700 1 -- 172.21.15.7:0/4034225594 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519172375 0 0) 0x7f00fc15c7d0 con 0x7f00fc141a90 2024-01-27T02:34:11.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.797+0000 7f0101f56700 1 --2- 172.21.15.7:0/4034225594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00fc0cb210 0x7f00fc0cbc20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0cb6b0 tx=0x7f00f4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfd516bebe58649 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:11.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.797+0000 7f00effff700 1 -- 172.21.15.7:0/4034225594 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f400e070 con 0x7f00fc0cb210 2024-01-27T02:34:11.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.798+0000 7f00effff700 1 -- 172.21.15.7:0/4034225594 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f400bc30 con 0x7f00fc0cb210 2024-01-27T02:34:11.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.798+0000 7f00effff700 1 -- 172.21.15.7:0/4034225594 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4012430 con 0x7f00fc0cb210 2024-01-27T02:34:11.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.798+0000 7f01039b9700 1 -- 172.21.15.7:0/4034225594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00fc0cb210 msgr2=0x7f00fc0cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:11.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.798+0000 7f01039b9700 1 --2- 172.21.15.7:0/4034225594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00fc0cb210 0x7f00fc0cbc20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0cb6b0 tx=0x7f00f4009780 comp rx=0 tx=0).stop 2024-01-27T02:34:11.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.798+0000 7f01039b9700 1 -- 172.21.15.7:0/4034225594 shutdown_connections 2024-01-27T02:34:11.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.798+0000 7f01039b9700 1 --2- 172.21.15.7:0/4034225594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00fc0cb210 0x7f00fc0cbc20 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.798+0000 7f01039b9700 1 --2- 172.21.15.7:0/4034225594 >> v2:172.21.15.188:3300/0 conn(0x7f00fc1416c0 0x7f00fc0cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.798+0000 7f01039b9700 1 -- 172.21.15.7:0/4034225594 >> 172.21.15.7:0/4034225594 conn(0x7f00fc063580 msgr2=0x7f00fc063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.798+0000 7f01039b9700 1 -- 172.21.15.7:0/4034225594 shutdown_connections 2024-01-27T02:34:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.798+0000 7f01039b9700 1 -- 172.21.15.7:0/4034225594 wait complete. 2024-01-27T02:34:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.799+0000 7f01039b9700 1 Processor -- start 2024-01-27T02:34:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.799+0000 7f01039b9700 1 -- start start 2024-01-27T02:34:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.800+0000 7f01039b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00fc0cb210 0x7f00fc0b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.800+0000 7f01039b9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00fc1416c0 0x7f00fc140090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:11.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.800+0000 7f01039b9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00fc141a90 0x7f00fc0b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:11.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.800+0000 7f01039b9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc1405d0 con 0x7f00fc0cb210 2024-01-27T02:34:11.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.800+0000 7f01039b9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc0b29f0 con 0x7f00fc1416c0 2024-01-27T02:34:11.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.800+0000 7f01039b9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc0b3480 con 0x7f00fc141a90 2024-01-27T02:34:11.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.801+0000 7f0101755700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00fc0cb210 0x7f00fc0b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:11.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.801+0000 7f0101755700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00fc0cb210 0x7f00fc0b2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52666/0 (socket says 172.21.15.7:52666) 2024-01-27T02:34:11.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.801+0000 7f0101755700 1 -- 172.21.15.7:0/3162345886 learned_addr learned my addr 172.21.15.7:0/3162345886 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:11.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.801+0000 7f0101f56700 1 --2- 172.21.15.7:0/3162345886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00fc141a90 0x7f00fc0b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:11.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.801+0000 7f0100f54700 1 --2- 172.21.15.7:0/3162345886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00fc1416c0 0x7f00fc140090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:11.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.801+0000 7f0101755700 1 -- 172.21.15.7:0/3162345886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00fc141a90 msgr2=0x7f00fc0b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:11.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.801+0000 7f0101755700 1 --2- 172.21.15.7:0/3162345886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00fc141a90 0x7f00fc0b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:11.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.801+0000 7f0101755700 1 -- 172.21.15.7:0/3162345886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00fc1416c0 msgr2=0x7f00fc140090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.801+0000 7f0101755700 1 --2- 172.21.15.7:0/3162345886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00fc1416c0 0x7f00fc140090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.801+0000 7f0101755700 1 -- 172.21.15.7:0/3162345886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f4009a20 con 0x7f00fc0cb210 2024-01-27T02:34:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.801+0000 7f0100f54700 1 --2- 172.21.15.7:0/3162345886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00fc1416c0 0x7f00fc140090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:34:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.801+0000 7f0101f56700 1 --2- 172.21.15.7:0/3162345886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00fc141a90 0x7f00fc0b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:34:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.801+0000 7f0101755700 1 --2- 172.21.15.7:0/3162345886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00fc0cb210 0x7f00fc0b2df0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f00f000d810 tx=0x7f00f0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.801+0000 7f00ed7fa700 1 -- 172.21.15.7:0/3162345886 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f00125f0 con 0x7f00fc0cb210 2024-01-27T02:34:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.801+0000 7f00ed7fa700 1 -- 172.21.15.7:0/3162345886 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f0012750 con 0x7f00fc0cb210 2024-01-27T02:34:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.801+0000 7f00ed7fa700 1 -- 172.21.15.7:0/3162345886 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0012b10 con 0x7f00fc0cb210 2024-01-27T02:34:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.801+0000 7f01039b9700 1 -- 172.21.15.7:0/3162345886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00fc0b3760 con 0x7f00fc0cb210 2024-01-27T02:34:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.801+0000 7f01039b9700 1 -- 172.21.15.7:0/3162345886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00fc0b3d20 con 0x7f00fc0cb210 2024-01-27T02:34:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.802+0000 7f01039b9700 1 -- 172.21.15.7:0/3162345886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00fc0b4370 con 0x7f00fc0cb210 2024-01-27T02:34:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.803+0000 7f00ed7fa700 1 -- 172.21.15.7:0/3162345886 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f00f001e460 con 0x7f00fc0cb210 2024-01-27T02:34:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.803+0000 7f00ed7fa700 1 --2- 172.21.15.7:0/3162345886 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f00e0066f30 0x7f00e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.803+0000 7f00ed7fa700 1 -- 172.21.15.7:0/3162345886 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f00f0084850 con 0x7f00fc0cb210 2024-01-27T02:34:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.804+0000 7f0100f54700 1 --2- 172.21.15.7:0/3162345886 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f00e0066f30 0x7f00e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.804+0000 7f0100f54700 1 --2- 172.21.15.7:0/3162345886 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f00e0066f30 0x7f00e00693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f00f8001810 tx=0x7f00f8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:11.808+0000 7f00ed7fa700 1 -- 172.21.15.7:0/3162345886 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00f0051e10 con 0x7f00fc0cb210 2024-01-27T02:34:12.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.009+0000 7f01039b9700 1 -- 172.21.15.7:0/3162345886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f00fc072060 con 0x7f00fc0cb210 2024-01-27T02:34:12.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.010+0000 7f00ed7fa700 1 -- 172.21.15.7:0/3162345886 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7f00f0055430 con 0x7f00fc0cb210 2024-01-27T02:34:12.010 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:34:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.012+0000 7f01039b9700 1 -- 172.21.15.7:0/3162345886 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f00e0066f30 msgr2=0x7f00e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.012+0000 7f01039b9700 1 --2- 172.21.15.7:0/3162345886 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f00e0066f30 0x7f00e00693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f00f8001810 tx=0x7f00f8009040 comp rx=0 tx=0).stop 2024-01-27T02:34:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.012+0000 7f01039b9700 1 -- 172.21.15.7:0/3162345886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00fc0cb210 msgr2=0x7f00fc0b2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.012+0000 7f01039b9700 1 --2- 172.21.15.7:0/3162345886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00fc0cb210 0x7f00fc0b2df0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f00f000d810 tx=0x7f00f0008040 comp rx=0 tx=0).stop 2024-01-27T02:34:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.012+0000 7f01039b9700 1 -- 172.21.15.7:0/3162345886 shutdown_connections 2024-01-27T02:34:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.012+0000 7f01039b9700 1 --2- 172.21.15.7:0/3162345886 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f00e0066f30 0x7f00e00693e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.012+0000 7f01039b9700 1 --2- 172.21.15.7:0/3162345886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00fc141a90 0x7f00fc0b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.012+0000 7f01039b9700 1 --2- 172.21.15.7:0/3162345886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00fc0cb210 0x7f00fc0b2df0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.012+0000 7f01039b9700 1 --2- 172.21.15.7:0/3162345886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00fc1416c0 0x7f00fc140090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.012+0000 7f01039b9700 1 -- 172.21.15.7:0/3162345886 >> 172.21.15.7:0/3162345886 conn(0x7f00fc063580 msgr2=0x7f00fc0cb650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:12.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.012+0000 7f01039b9700 1 -- 172.21.15.7:0/3162345886 shutdown_connections 2024-01-27T02:34:12.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.012+0000 7f01039b9700 1 -- 172.21.15.7:0/3162345886 wait complete. 2024-01-27T02:34:12.026 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-27T02:34:12.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.157+0000 7f99d322a700 1 Processor -- start 2024-01-27T02:34:12.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.157+0000 7f99d322a700 1 -- start start 2024-01-27T02:34:12.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.158+0000 7f99d322a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f99cc0c3ae0 0x7f99cc0cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:12.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.158+0000 7f99d322a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99cc158070 0x7f99cc0cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:12.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.158+0000 7f99d322a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99cc0ca020 con 0x7f99cc157ca0 2024-01-27T02:34:12.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.159+0000 7f99d322a700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99cc0c5210 con 0x7f99cc0c3ae0 2024-01-27T02:34:12.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.159+0000 7f99d322a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc0c54f0 con 0x7f99cc158070 2024-01-27T02:34:12.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.159+0000 7f99d0fc6700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f99cc157ca0 0x7f99cc0c4a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42124/0 (socket says 172.21.15.7:42124) 2024-01-27T02:34:12.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.159+0000 7f99d0fc6700 1 -- 172.21.15.7:0/3634672238 learned_addr learned my addr 172.21.15.7:0/3634672238 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:34:12.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.159+0000 7f99cbfff700 1 --2- 172.21.15.7:0/3634672238 >> v2:172.21.15.188:3300/0 conn(0x7f99cc0c3ae0 0x7f99cc0cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:12.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.159+0000 7f99d17c7700 1 --2- 172.21.15.7:0/3634672238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99cc158070 0x7f99cc0cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.159+0000 7f99cb7fe700 1 -- 172.21.15.7:0/3634672238 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 417695396 0 0) 0x7f99cc0ca020 con 0x7f99cc157ca0 2024-01-27T02:34:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.159+0000 7f99cb7fe700 1 -- 172.21.15.7:0/3634672238 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99b8003570 con 0x7f99cc157ca0 2024-01-27T02:34:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.159+0000 7f99cbfff700 1 -- 172.21.15.7:0/3634672238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99cc158070 msgr2=0x7f99cc0cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.159+0000 7f99cbfff700 1 --2- 172.21.15.7:0/3634672238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99cc158070 0x7f99cc0cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.159+0000 7f99cbfff700 1 -- 172.21.15.7:0/3634672238 >> v1:172.21.15.7:6789/0 conn(0x7f99cc157ca0 legacy=0x7f99cc0c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.159+0000 7f99cb7fe700 1 -- 172.21.15.7:0/3634672238 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4230775521 0 0) 0x7f99b8003570 con 0x7f99cc157ca0 2024-01-27T02:34:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.159+0000 7f99cbfff700 1 -- 172.21.15.7:0/3634672238 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99cc0c57d0 con 0x7f99cc0c3ae0 2024-01-27T02:34:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.159+0000 7f99d17c7700 1 --2- 172.21.15.7:0/3634672238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99cc158070 0x7f99cc0cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:34:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.160+0000 7f99cbfff700 1 --2- 172.21.15.7:0/3634672238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99cc0c3ae0 0x7f99cc0cd4e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f99b40098e0 tx=0x7f99b40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d2a69fc828134d5 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.160+0000 7f99cb7fe700 1 -- 172.21.15.7:0/3634672238 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc001470 con 0x7f99cc0c3ae0 2024-01-27T02:34:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.160+0000 7f99cb7fe700 1 -- 172.21.15.7:0/3634672238 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b4011420 con 0x7f99cc0c3ae0 2024-01-27T02:34:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.160+0000 7f99cb7fe700 1 -- 172.21.15.7:0/3634672238 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b40117c0 con 0x7f99cc0c3ae0 2024-01-27T02:34:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.160+0000 7f99d322a700 1 -- 172.21.15.7:0/3634672238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99cc0c3ae0 msgr2=0x7f99cc0cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.160+0000 7f99d322a700 1 --2- 172.21.15.7:0/3634672238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99cc0c3ae0 0x7f99cc0cd4e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f99b40098e0 tx=0x7f99b40091a0 comp rx=0 tx=0).stop 2024-01-27T02:34:12.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.160+0000 7f99d322a700 1 -- 172.21.15.7:0/3634672238 shutdown_connections 2024-01-27T02:34:12.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.160+0000 7f99d322a700 1 --2- 172.21.15.7:0/3634672238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99cc158070 0x7f99cc0cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:12.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.160+0000 7f99d322a700 1 --2- 172.21.15.7:0/3634672238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99cc0c3ae0 0x7f99cc0cd4e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:12.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.160+0000 7f99d322a700 1 -- 172.21.15.7:0/3634672238 >> 172.21.15.7:0/3634672238 conn(0x7f99cc098020 msgr2=0x7f99cc0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:12.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.160+0000 7f99d322a700 1 -- 172.21.15.7:0/3634672238 shutdown_connections 2024-01-27T02:34:12.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.160+0000 7f99d322a700 1 -- 172.21.15.7:0/3634672238 wait complete. 2024-01-27T02:34:12.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.162+0000 7f99d322a700 1 Processor -- start 2024-01-27T02:34:12.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.162+0000 7f99d322a700 1 -- start start 2024-01-27T02:34:12.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.163+0000 7f99d322a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99cc0c3ae0 0x7f99cc155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:12.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.163+0000 7f99d322a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99cc157ca0 0x7f99cc0b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:12.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.163+0000 7f99d322a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99cc158070 0x7f99cc1540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:12.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.163+0000 7f99d322a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc1559a0 con 0x7f99cc158070 2024-01-27T02:34:12.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.163+0000 7f99d322a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc154d90 con 0x7f99cc157ca0 2024-01-27T02:34:12.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.163+0000 7f99d322a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc155070 con 0x7f99cc0c3ae0 2024-01-27T02:34:12.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.163+0000 7f99d17c7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99cc158070 0x7f99cc1540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:12.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.163+0000 7f99d17c7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99cc158070 0x7f99cc1540d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52676/0 (socket says 172.21.15.7:52676) 2024-01-27T02:34:12.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.164+0000 7f99d0fc6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99cc0c3ae0 0x7f99cc155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:12.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.164+0000 7f99d17c7700 1 -- 172.21.15.7:0/3306404817 learned_addr learned my addr 172.21.15.7:0/3306404817 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:12.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.164+0000 7f99cbfff700 1 --2- 172.21.15.7:0/3306404817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99cc157ca0 0x7f99cc0b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:12.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.164+0000 7f99d17c7700 1 -- 172.21.15.7:0/3306404817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99cc0c3ae0 msgr2=0x7f99cc155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:12.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.164+0000 7f99d17c7700 1 --2- 172.21.15.7:0/3306404817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99cc0c3ae0 0x7f99cc155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:12.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.164+0000 7f99d17c7700 1 -- 172.21.15.7:0/3306404817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99cc157ca0 msgr2=0x7f99cc0b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:12.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.164+0000 7f99d17c7700 1 --2- 172.21.15.7:0/3306404817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99cc157ca0 0x7f99cc0b33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:12.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.164+0000 7f99d17c7700 1 -- 172.21.15.7:0/3306404817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c4009a20 con 0x7f99cc158070 2024-01-27T02:34:12.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.164+0000 7f99d17c7700 1 --2- 172.21.15.7:0/3306404817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99cc158070 0x7f99cc1540d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009f20 tx=0x7f99c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:12.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.164+0000 7f99c8ff9700 1 -- 172.21.15.7:0/3306404817 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c400cc00 con 0x7f99cc158070 2024-01-27T02:34:12.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.164+0000 7f99c8ff9700 1 -- 172.21.15.7:0/3306404817 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c400f040 con 0x7f99cc158070 2024-01-27T02:34:12.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.164+0000 7f99d322a700 1 -- 172.21.15.7:0/3306404817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b4009420 con 0x7f99cc158070 2024-01-27T02:34:12.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.164+0000 7f99c8ff9700 1 -- 172.21.15.7:0/3306404817 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4014480 con 0x7f99cc158070 2024-01-27T02:34:12.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.164+0000 7f99d322a700 1 -- 172.21.15.7:0/3306404817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99cc155630 con 0x7f99cc158070 2024-01-27T02:34:12.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.165+0000 7f99d322a700 1 -- 172.21.15.7:0/3306404817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99cc155b10 con 0x7f99cc158070 2024-01-27T02:34:12.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.166+0000 7f99c8ff9700 1 -- 172.21.15.7:0/3306404817 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f99c4003020 con 0x7f99cc158070 2024-01-27T02:34:12.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.166+0000 7f99c8ff9700 1 --2- 172.21.15.7:0/3306404817 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f99ac066f60 0x7f99ac069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:12.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.166+0000 7f99c8ff9700 1 -- 172.21.15.7:0/3306404817 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f99c4084be0 con 0x7f99cc158070 2024-01-27T02:34:12.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.166+0000 7f99d0fc6700 1 --2- 172.21.15.7:0/3306404817 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f99ac066f60 0x7f99ac069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:12.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.167+0000 7f99d0fc6700 1 --2- 172.21.15.7:0/3306404817 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f99ac066f60 0x7f99ac069410 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0011e0 tx=0x7f99bc001aa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:12.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.171+0000 7f99c8ff9700 1 -- 172.21.15.7:0/3306404817 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f99c4052100 con 0x7f99cc158070 2024-01-27T02:34:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.377+0000 7f99d322a700 1 -- 172.21.15.7:0/3306404817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f99cc04c710 con 0x7f99cc158070 2024-01-27T02:34:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.377+0000 7f99c8ff9700 1 -- 172.21.15.7:0/3306404817 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v61) v1 ==== 84+0+65 (secure 0 0 0) 0x7f99c4055720 con 0x7f99cc158070 2024-01-27T02:34:12.378 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:34:12.378 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-27T02:34:12.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:34:12.378 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-27T02:34:12.378 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-27T02:34:12.378 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:34:12.378 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-27T02:34:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.380+0000 7f99d322a700 1 -- 172.21.15.7:0/3306404817 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f99ac066f60 msgr2=0x7f99ac069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.380+0000 7f99d322a700 1 --2- 172.21.15.7:0/3306404817 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f99ac066f60 0x7f99ac069410 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0011e0 tx=0x7f99bc001aa0 comp rx=0 tx=0).stop 2024-01-27T02:34:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.380+0000 7f99d322a700 1 -- 172.21.15.7:0/3306404817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99cc158070 msgr2=0x7f99cc1540d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.380+0000 7f99d322a700 1 --2- 172.21.15.7:0/3306404817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99cc158070 0x7f99cc1540d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009f20 tx=0x7f99c400b040 comp rx=0 tx=0).stop 2024-01-27T02:34:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.380+0000 7f99d322a700 1 -- 172.21.15.7:0/3306404817 shutdown_connections 2024-01-27T02:34:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.380+0000 7f99d322a700 1 --2- 172.21.15.7:0/3306404817 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f99ac066f60 0x7f99ac069410 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.380+0000 7f99d322a700 1 --2- 172.21.15.7:0/3306404817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99cc158070 0x7f99cc1540d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.380+0000 7f99d322a700 1 --2- 172.21.15.7:0/3306404817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99cc0c3ae0 0x7f99cc155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.380+0000 7f99d322a700 1 --2- 172.21.15.7:0/3306404817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99cc157ca0 0x7f99cc0b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.380+0000 7f99d322a700 1 -- 172.21.15.7:0/3306404817 >> 172.21.15.7:0/3306404817 conn(0x7f99cc098020 msgr2=0x7f99cc06c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.380+0000 7f99d322a700 1 -- 172.21.15.7:0/3306404817 shutdown_connections 2024-01-27T02:34:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.380+0000 7f99d322a700 1 -- 172.21.15.7:0/3306404817 wait complete. 2024-01-27T02:34:12.394 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-27T02:34:12.394 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-27T02:34:12.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.530+0000 7faba5bd1700 1 Processor -- start 2024-01-27T02:34:12.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.530+0000 7faba5bd1700 1 -- start start 2024-01-27T02:34:12.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.531+0000 7faba5bd1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faba0157dd0 0x7faba00adf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:12.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.531+0000 7faba5bd1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faba01581a0 0x7faba00cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:12.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.531+0000 7faba5bd1700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faba00c5630 0x7faba00cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:12.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.531+0000 7faba5bd1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba00c51a0 con 0x7faba0157dd0 2024-01-27T02:34:12.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.531+0000 7faba5bd1700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba00c4590 con 0x7faba00c5630 2024-01-27T02:34:12.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.531+0000 7faba5bd1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba00c4870 con 0x7faba01581a0 2024-01-27T02:34:12.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.531+0000 7fab9f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faba0157dd0 0x7faba00adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:12.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.531+0000 7fab9effd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faba01581a0 0x7faba00cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:12.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.531+0000 7fab9f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faba0157dd0 0x7faba00adf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52682/0 (socket says 172.21.15.7:52682) 2024-01-27T02:34:12.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.531+0000 7fab9f7fe700 1 -- 172.21.15.7:0/2076683068 learned_addr learned my addr 172.21.15.7:0/2076683068 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:12.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.531+0000 7fab9ffff700 1 --2- 172.21.15.7:0/2076683068 >> v2:172.21.15.188:3300/0 conn(0x7faba00c5630 0x7faba00cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:12.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.531+0000 7fab9f7fe700 1 -- 172.21.15.7:0/2076683068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faba01581a0 msgr2=0x7faba00cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:12.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.531+0000 7fab9f7fe700 1 --2- 172.21.15.7:0/2076683068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faba01581a0 0x7faba00cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:12.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.532+0000 7fab9f7fe700 1 -- 172.21.15.7:0/2076683068 >> v2:172.21.15.188:3300/0 conn(0x7faba00c5630 msgr2=0x7faba00cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:12.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.532+0000 7fab9f7fe700 1 --2- 172.21.15.7:0/2076683068 >> v2:172.21.15.188:3300/0 conn(0x7faba00c5630 0x7faba00cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:12.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.532+0000 7fab9f7fe700 1 -- 172.21.15.7:0/2076683068 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba00c4b50 con 0x7faba0157dd0 2024-01-27T02:34:12.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.532+0000 7fab9f7fe700 1 --2- 172.21.15.7:0/2076683068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba0157dd0 0x7faba00adf20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7faba00c9310 tx=0x7fab90009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a26e447f619734 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:12.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.532+0000 7fab9e7fc700 1 -- 172.21.15.7:0/2076683068 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab90005c10 con 0x7faba0157dd0 2024-01-27T02:34:12.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.532+0000 7fab9e7fc700 1 -- 172.21.15.7:0/2076683068 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab90005d70 con 0x7faba0157dd0 2024-01-27T02:34:12.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.532+0000 7fab9e7fc700 1 -- 172.21.15.7:0/2076683068 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab90006260 con 0x7faba0157dd0 2024-01-27T02:34:12.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.532+0000 7faba5bd1700 1 -- 172.21.15.7:0/2076683068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba0157dd0 msgr2=0x7faba00adf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:12.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.532+0000 7faba5bd1700 1 --2- 172.21.15.7:0/2076683068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba0157dd0 0x7faba00adf20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7faba00c9310 tx=0x7fab90009220 comp rx=0 tx=0).stop 2024-01-27T02:34:12.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.533+0000 7faba5bd1700 1 -- 172.21.15.7:0/2076683068 shutdown_connections 2024-01-27T02:34:12.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.533+0000 7faba5bd1700 1 --2- 172.21.15.7:0/2076683068 >> v2:172.21.15.188:3300/0 conn(0x7faba00c5630 0x7faba00cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:12.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.533+0000 7faba5bd1700 1 --2- 172.21.15.7:0/2076683068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba0157dd0 0x7faba00adf20 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:12.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.533+0000 7faba5bd1700 1 --2- 172.21.15.7:0/2076683068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faba01581a0 0x7faba00cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:12.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.533+0000 7faba5bd1700 1 -- 172.21.15.7:0/2076683068 >> 172.21.15.7:0/2076683068 conn(0x7faba0098020 msgr2=0x7faba00711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:12.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.533+0000 7faba5bd1700 1 -- 172.21.15.7:0/2076683068 shutdown_connections 2024-01-27T02:34:12.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.533+0000 7faba5bd1700 1 -- 172.21.15.7:0/2076683068 wait complete. 2024-01-27T02:34:12.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.534+0000 7faba5bd1700 1 Processor -- start 2024-01-27T02:34:12.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.534+0000 7faba5bd1700 1 -- start start 2024-01-27T02:34:12.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.535+0000 7faba5bd1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba00c5630 0x7faba00c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:12.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.535+0000 7faba5bd1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faba0157dd0 0x7faba0069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:12.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.535+0000 7faba5bd1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faba01581a0 0x7faba015bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:12.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.535+0000 7faba5bd1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba00c86f0 con 0x7faba00c5630 2024-01-27T02:34:12.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.535+0000 7faba5bd1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba00c8b80 con 0x7faba01581a0 2024-01-27T02:34:12.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.535+0000 7faba5bd1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba00c7ae0 con 0x7faba0157dd0 2024-01-27T02:34:12.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.535+0000 7fab9f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba00c5630 0x7faba00c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:12.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.535+0000 7fab9f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba00c5630 0x7faba00c74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52692/0 (socket says 172.21.15.7:52692) 2024-01-27T02:34:12.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.535+0000 7fab9f7fe700 1 -- 172.21.15.7:0/819160982 learned_addr learned my addr 172.21.15.7:0/819160982 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:12.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.535+0000 7fab9ffff700 1 --2- 172.21.15.7:0/819160982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faba01581a0 0x7faba015bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:12.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.535+0000 7fab9effd700 1 --2- 172.21.15.7:0/819160982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faba0157dd0 0x7faba0069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:12.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.535+0000 7fab9f7fe700 1 -- 172.21.15.7:0/819160982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faba0157dd0 msgr2=0x7faba0069350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:12.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.535+0000 7fab9f7fe700 1 --2- 172.21.15.7:0/819160982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faba0157dd0 0x7faba0069350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:12.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.535+0000 7fab9f7fe700 1 -- 172.21.15.7:0/819160982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faba01581a0 msgr2=0x7faba015bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:12.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.535+0000 7fab9f7fe700 1 --2- 172.21.15.7:0/819160982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faba01581a0 0x7faba015bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:12.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.535+0000 7fab9f7fe700 1 -- 172.21.15.7:0/819160982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab90009480 con 0x7faba00c5630 2024-01-27T02:34:12.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.535+0000 7fab9ffff700 1 --2- 172.21.15.7:0/819160982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faba01581a0 0x7faba015bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:34:12.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.535+0000 7fab9effd700 1 --2- 172.21.15.7:0/819160982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faba0157dd0 0x7faba0069350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:34:12.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.535+0000 7fab9f7fe700 1 --2- 172.21.15.7:0/819160982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba00c5630 0x7faba00c74b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fab90009f50 tx=0x7fab90016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:12.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.536+0000 7fab877fe700 1 -- 172.21.15.7:0/819160982 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab90011800 con 0x7faba00c5630 2024-01-27T02:34:12.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.536+0000 7fab877fe700 1 -- 172.21.15.7:0/819160982 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab90020070 con 0x7faba00c5630 2024-01-27T02:34:12.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.536+0000 7faba5bd1700 1 -- 172.21.15.7:0/819160982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba00c7dc0 con 0x7faba00c5630 2024-01-27T02:34:12.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.536+0000 7fab877fe700 1 -- 172.21.15.7:0/819160982 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab90011b90 con 0x7faba00c5630 2024-01-27T02:34:12.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.536+0000 7faba5bd1700 1 -- 172.21.15.7:0/819160982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba00c8380 con 0x7faba00c5630 2024-01-27T02:34:12.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.537+0000 7faba5bd1700 1 -- 172.21.15.7:0/819160982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba004c710 con 0x7faba00c5630 2024-01-27T02:34:12.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.538+0000 7fab877fe700 1 -- 172.21.15.7:0/819160982 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fab90022030 con 0x7faba00c5630 2024-01-27T02:34:12.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.538+0000 7fab877fe700 1 --2- 172.21.15.7:0/819160982 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fab8c066f30 0x7fab8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:12.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.538+0000 7fab877fe700 1 -- 172.21.15.7:0/819160982 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fab90086120 con 0x7faba00c5630 2024-01-27T02:34:12.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.538+0000 7fab9effd700 1 --2- 172.21.15.7:0/819160982 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fab8c066f30 0x7fab8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:12.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.539+0000 7fab9effd700 1 --2- 172.21.15.7:0/819160982 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fab8c066f30 0x7fab8c0693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fab94006f00 tx=0x7fab94008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.542+0000 7fab877fe700 1 -- 172.21.15.7:0/819160982 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab900536c0 con 0x7faba00c5630 2024-01-27T02:34:12.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:12.743+0000 7faba5bd1700 1 -- 172.21.15.7:0/819160982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7faba00c4e30 con 0x7faba00c5630 2024-01-27T02:34:13.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.032+0000 7fab877fe700 1 -- 172.21.15.7:0/819160982 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v62) v1 ==== 138+0+0 (secure 0 0 0) 0x7fab90056ce0 con 0x7faba00c5630 2024-01-27T02:34:13.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.036+0000 7faba5bd1700 1 -- 172.21.15.7:0/819160982 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fab8c066f30 msgr2=0x7fab8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:13.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.036+0000 7faba5bd1700 1 --2- 172.21.15.7:0/819160982 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fab8c066f30 0x7fab8c0693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fab94006f00 tx=0x7fab94008040 comp rx=0 tx=0).stop 2024-01-27T02:34:13.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.037+0000 7faba5bd1700 1 -- 172.21.15.7:0/819160982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba00c5630 msgr2=0x7faba00c74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:13.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.037+0000 7faba5bd1700 1 --2- 172.21.15.7:0/819160982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba00c5630 0x7faba00c74b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fab90009f50 tx=0x7fab90016000 comp rx=0 tx=0).stop 2024-01-27T02:34:13.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.037+0000 7faba5bd1700 1 -- 172.21.15.7:0/819160982 shutdown_connections 2024-01-27T02:34:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.037+0000 7faba5bd1700 1 --2- 172.21.15.7:0/819160982 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fab8c066f30 0x7fab8c0693e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.037+0000 7faba5bd1700 1 --2- 172.21.15.7:0/819160982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faba01581a0 0x7faba015bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.037+0000 7faba5bd1700 1 --2- 172.21.15.7:0/819160982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba00c5630 0x7faba00c74b0 secure :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fab90009f50 tx=0x7fab90016000 comp rx=0 tx=0).stop 2024-01-27T02:34:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.037+0000 7faba5bd1700 1 --2- 172.21.15.7:0/819160982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faba0157dd0 0x7faba0069350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.037+0000 7faba5bd1700 1 -- 172.21.15.7:0/819160982 >> 172.21.15.7:0/819160982 conn(0x7faba0098020 msgr2=0x7faba01567f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.038+0000 7faba5bd1700 1 -- 172.21.15.7:0/819160982 shutdown_connections 2024-01-27T02:34:13.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.038+0000 7faba5bd1700 1 -- 172.21.15.7:0/819160982 wait complete. 2024-01-27T02:34:13.040 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-27T02:34:13.055 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-27T02:34:13.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.198+0000 7fab1d1e2700 1 Processor -- start 2024-01-27T02:34:13.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.198+0000 7fab1d1e2700 1 -- start start 2024-01-27T02:34:13.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.199+0000 7fab1d1e2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab180cba60 0x7fab180ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:13.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.199+0000 7fab1d1e2700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fab1806b4e0 0x7fab1806b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:13.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.199+0000 7fab1d1e2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab180679b0 con 0x7fab180cbe30 2024-01-27T02:34:13.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.199+0000 7fab1d1e2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab18067c90 con 0x7fab180cba60 2024-01-27T02:34:13.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.199+0000 7fab1d1e2700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab1806be20 con 0x7fab1806b4e0 2024-01-27T02:34:13.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.199+0000 7fab16d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab180cba60 0x7fab180ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:13.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.199+0000 7fab1659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fab180cbe30 0x7fab1815b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42128/0 (socket says 172.21.15.7:42128) 2024-01-27T02:34:13.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.199+0000 7fab1659c700 1 -- 172.21.15.7:0/2366067470 learned_addr learned my addr 172.21.15.7:0/2366067470 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:34:13.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.200+0000 7fab1759e700 1 --2- 172.21.15.7:0/2366067470 >> v2:172.21.15.188:3300/0 conn(0x7fab1806b4e0 0x7fab1806b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:13.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.200+0000 7fab16d9d700 1 -- 172.21.15.7:0/2366067470 >> v2:172.21.15.188:3300/0 conn(0x7fab1806b4e0 msgr2=0x7fab1806b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:13.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.200+0000 7fab16d9d700 1 --2- 172.21.15.7:0/2366067470 >> v2:172.21.15.188:3300/0 conn(0x7fab1806b4e0 0x7fab1806b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:13.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.200+0000 7fab16d9d700 1 -- 172.21.15.7:0/2366067470 >> v1:172.21.15.7:6789/0 conn(0x7fab180cbe30 legacy=0x7fab1815b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:13.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.200+0000 7fab16d9d700 1 -- 172.21.15.7:0/2366067470 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab180cca50 con 0x7fab180cba60 2024-01-27T02:34:13.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.200+0000 7fab15d9b700 1 -- 172.21.15.7:0/2366067470 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695460066 0 0) 0x7fab180679b0 con 0x7fab180cbe30 2024-01-27T02:34:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.200+0000 7fab16d9d700 1 --2- 172.21.15.7:0/2366067470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab180cba60 0x7fab180ce410 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fab180b1800 tx=0x7fab08009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820cd41a284236c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.200+0000 7fab15d9b700 1 -- 172.21.15.7:0/2366067470 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08005c10 con 0x7fab180cba60 2024-01-27T02:34:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.200+0000 7fab15d9b700 1 -- 172.21.15.7:0/2366067470 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab08005d70 con 0x7fab180cba60 2024-01-27T02:34:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.200+0000 7fab15d9b700 1 -- 172.21.15.7:0/2366067470 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08006260 con 0x7fab180cba60 2024-01-27T02:34:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.201+0000 7fab1d1e2700 1 -- 172.21.15.7:0/2366067470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab180cba60 msgr2=0x7fab180ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.201+0000 7fab1d1e2700 1 --2- 172.21.15.7:0/2366067470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab180cba60 0x7fab180ce410 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fab180b1800 tx=0x7fab08009120 comp rx=0 tx=0).stop 2024-01-27T02:34:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.201+0000 7fab1d1e2700 1 -- 172.21.15.7:0/2366067470 shutdown_connections 2024-01-27T02:34:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.201+0000 7fab1d1e2700 1 --2- 172.21.15.7:0/2366067470 >> v2:172.21.15.188:3300/0 conn(0x7fab1806b4e0 0x7fab1806b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.201+0000 7fab1d1e2700 1 --2- 172.21.15.7:0/2366067470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab180cba60 0x7fab180ce410 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.201+0000 7fab1d1e2700 1 -- 172.21.15.7:0/2366067470 >> 172.21.15.7:0/2366067470 conn(0x7fab18063580 msgr2=0x7fab18063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.201+0000 7fab1d1e2700 1 -- 172.21.15.7:0/2366067470 shutdown_connections 2024-01-27T02:34:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.201+0000 7fab1d1e2700 1 -- 172.21.15.7:0/2366067470 wait complete. 2024-01-27T02:34:13.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.202+0000 7fab1d1e2700 1 Processor -- start 2024-01-27T02:34:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.202+0000 7fab1d1e2700 1 -- start start 2024-01-27T02:34:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.203+0000 7fab1d1e2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1806b4e0 0x7fab1814a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.203+0000 7fab1d1e2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab180cba60 0x7fab180b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.203+0000 7fab1d1e2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab180cbe30 0x7fab1815e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.203+0000 7fab1d1e2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1814ad70 con 0x7fab180cba60 2024-01-27T02:34:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.203+0000 7fab1d1e2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab180b20e0 con 0x7fab180cbe30 2024-01-27T02:34:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.203+0000 7fab1d1e2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab180ac3e0 con 0x7fab1806b4e0 2024-01-27T02:34:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.203+0000 7fab1659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab180cba60 0x7fab180b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.203+0000 7fab1659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab180cba60 0x7fab180b1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52712/0 (socket says 172.21.15.7:52712) 2024-01-27T02:34:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.203+0000 7fab1659c700 1 -- 172.21.15.7:0/3851146677 learned_addr learned my addr 172.21.15.7:0/3851146677 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.203+0000 7fab16d9d700 1 --2- 172.21.15.7:0/3851146677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1806b4e0 0x7fab1814a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.203+0000 7fab1759e700 1 --2- 172.21.15.7:0/3851146677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab180cbe30 0x7fab1815e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:13.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.204+0000 7fab1659c700 1 -- 172.21.15.7:0/3851146677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1806b4e0 msgr2=0x7fab1814a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.204+0000 7fab1659c700 1 --2- 172.21.15.7:0/3851146677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1806b4e0 0x7fab1814a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.204+0000 7fab1659c700 1 -- 172.21.15.7:0/3851146677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab180cbe30 msgr2=0x7fab1815e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.204+0000 7fab1659c700 1 --2- 172.21.15.7:0/3851146677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab180cbe30 0x7fab1815e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.204+0000 7fab1659c700 1 -- 172.21.15.7:0/3851146677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab080093c0 con 0x7fab180cba60 2024-01-27T02:34:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.204+0000 7fab1759e700 1 --2- 172.21.15.7:0/3851146677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab180cbe30 0x7fab1815e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.204+0000 7fab1659c700 1 --2- 172.21.15.7:0/3851146677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab180cba60 0x7fab180b1ba0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00c610 tx=0x7fab0c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.204+0000 7faafeffd700 1 -- 172.21.15.7:0/3851146677 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c005b30 con 0x7fab180cba60 2024-01-27T02:34:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.204+0000 7faafeffd700 1 -- 172.21.15.7:0/3851146677 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0c005c90 con 0x7fab180cba60 2024-01-27T02:34:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.204+0000 7fab1d1e2700 1 -- 172.21.15.7:0/3851146677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab180ac7d0 con 0x7fab180cba60 2024-01-27T02:34:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.204+0000 7faafeffd700 1 -- 172.21.15.7:0/3851146677 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c007300 con 0x7fab180cba60 2024-01-27T02:34:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.204+0000 7fab1d1e2700 1 -- 172.21.15.7:0/3851146677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab180acd90 con 0x7fab180cba60 2024-01-27T02:34:13.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.205+0000 7fab1d1e2700 1 -- 172.21.15.7:0/3851146677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab18072060 con 0x7fab180cba60 2024-01-27T02:34:13.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.206+0000 7faafeffd700 1 -- 172.21.15.7:0/3851146677 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fab0c01b3b0 con 0x7fab180cba60 2024-01-27T02:34:13.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.206+0000 7faafeffd700 1 --2- 172.21.15.7:0/3851146677 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fab04066f30 0x7fab040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:13.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.207+0000 7faafeffd700 1 -- 172.21.15.7:0/3851146677 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fab0c083ce0 con 0x7fab180cba60 2024-01-27T02:34:13.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.207+0000 7fab16d9d700 1 --2- 172.21.15.7:0/3851146677 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fab04066f30 0x7fab040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:13.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.207+0000 7fab16d9d700 1 --2- 172.21.15.7:0/3851146677 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fab04066f30 0x7fab040693e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fab08009160 tx=0x7fab08016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:13.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.211+0000 7faafeffd700 1 -- 172.21.15.7:0/3851146677 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab0c051110 con 0x7fab180cba60 2024-01-27T02:34:13.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:13.412+0000 7fab1d1e2700 1 -- 172.21.15.7:0/3851146677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fab1806d5c0 con 0x7fab180cba60 2024-01-27T02:34:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.035+0000 7faafeffd700 1 -- 172.21.15.7:0/3851146677 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v63) v1 ==== 170+0+0 (secure 0 0 0) 0x7fab0c0170d0 con 0x7fab180cba60 2024-01-27T02:34:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.047+0000 7fab1d1e2700 1 -- 172.21.15.7:0/3851146677 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fab04066f30 msgr2=0x7fab040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.047+0000 7fab1d1e2700 1 --2- 172.21.15.7:0/3851146677 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fab04066f30 0x7fab040693e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fab08009160 tx=0x7fab08016000 comp rx=0 tx=0).stop 2024-01-27T02:34:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.047+0000 7fab1d1e2700 1 -- 172.21.15.7:0/3851146677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab180cba60 msgr2=0x7fab180b1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.047+0000 7fab1d1e2700 1 --2- 172.21.15.7:0/3851146677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab180cba60 0x7fab180b1ba0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00c610 tx=0x7fab0c000fe0 comp rx=0 tx=0).stop 2024-01-27T02:34:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.047+0000 7fab1d1e2700 1 -- 172.21.15.7:0/3851146677 shutdown_connections 2024-01-27T02:34:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.047+0000 7fab1d1e2700 1 --2- 172.21.15.7:0/3851146677 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fab04066f30 0x7fab040693e0 secure :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fab08009160 tx=0x7fab08016000 comp rx=0 tx=0).stop 2024-01-27T02:34:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.047+0000 7fab1d1e2700 1 --2- 172.21.15.7:0/3851146677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab180cbe30 0x7fab1815e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.047+0000 7fab1d1e2700 1 --2- 172.21.15.7:0/3851146677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1806b4e0 0x7fab1814a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.047+0000 7fab1d1e2700 1 --2- 172.21.15.7:0/3851146677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab180cba60 0x7fab180b1ba0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.047+0000 7fab1d1e2700 1 -- 172.21.15.7:0/3851146677 >> 172.21.15.7:0/3851146677 conn(0x7fab18063580 msgr2=0x7fab181586c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.048+0000 7fab1d1e2700 1 -- 172.21.15.7:0/3851146677 shutdown_connections 2024-01-27T02:34:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.049+0000 7fab1d1e2700 1 -- 172.21.15.7:0/3851146677 wait complete. 2024-01-27T02:34:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-27T02:34:14.067 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-27T02:34:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.207+0000 7f65374cd700 1 Processor -- start 2024-01-27T02:34:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.207+0000 7f65374cd700 1 -- start start 2024-01-27T02:34:14.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.208+0000 7f65374cd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f653015a180 0x7f65300c3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:14.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.208+0000 7f65374cd700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f65300ca3b0 0x7f65300aef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:14.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.208+0000 7f65374cd700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65300b1450 con 0x7f653015a550 2024-01-27T02:34:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.208+0000 7f65374cd700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65300b18e0 con 0x7f65300ca3b0 2024-01-27T02:34:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.208+0000 7f65374cd700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65300b0840 con 0x7f653015a180 2024-01-27T02:34:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.208+0000 7f6535a6a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f65300ca3b0 0x7f65300aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.208+0000 7f6535269700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f653015a180 0x7f65300c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.208+0000 7f6534a68700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f653015a550 0x7f65300c9ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:46002/0 (socket says 172.21.15.7:46002) 2024-01-27T02:34:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.208+0000 7f6534a68700 1 -- 172.21.15.7:0/3199666377 learned_addr learned my addr 172.21.15.7:0/3199666377 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:34:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.208+0000 7f6535a6a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f65300ca3b0 0x7f65300aef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:44056/0 (socket says 172.21.15.7:44056) 2024-01-27T02:34:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.208+0000 7f6535a6a700 1 -- 172.21.15.7:0/3199666377 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f653015a180 msgr2=0x7f65300c3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.208+0000 7f6535a6a700 1 --2- 172.21.15.7:0/3199666377 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f653015a180 0x7f65300c3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.208+0000 7f6535a6a700 1 -- 172.21.15.7:0/3199666377 >> v1:172.21.15.188:6789/0 conn(0x7f653015a550 legacy=0x7f65300c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.208+0000 7f6535a6a700 1 -- 172.21.15.7:0/3199666377 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65300b0b20 con 0x7f65300ca3b0 2024-01-27T02:34:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.208+0000 7f6527fff700 1 -- 172.21.15.7:0/3199666377 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 244019462 0 0) 0x7f65300b1450 con 0x7f653015a550 2024-01-27T02:34:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.209+0000 7f6535269700 1 --2- 172.21.15.7:0/3199666377 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f653015a180 0x7f65300c3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:34:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.209+0000 7f6535a6a700 1 --2- 172.21.15.7:0/3199666377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65300ca3b0 0x7f65300aef20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6528009e60 tx=0x7f6528009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d40f4c8120b763f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.209+0000 7f6527fff700 1 -- 172.21.15.7:0/3199666377 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652800e070 con 0x7f65300ca3b0 2024-01-27T02:34:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.209+0000 7f6527fff700 1 -- 172.21.15.7:0/3199666377 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652800bb60 con 0x7f65300ca3b0 2024-01-27T02:34:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.209+0000 7f6527fff700 1 -- 172.21.15.7:0/3199666377 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6528012430 con 0x7f65300ca3b0 2024-01-27T02:34:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.210+0000 7f65374cd700 1 -- 172.21.15.7:0/3199666377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65300ca3b0 msgr2=0x7f65300aef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.210+0000 7f65374cd700 1 --2- 172.21.15.7:0/3199666377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65300ca3b0 0x7f65300aef20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6528009e60 tx=0x7f6528009780 comp rx=0 tx=0).stop 2024-01-27T02:34:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.210+0000 7f65374cd700 1 -- 172.21.15.7:0/3199666377 shutdown_connections 2024-01-27T02:34:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.210+0000 7f65374cd700 1 --2- 172.21.15.7:0/3199666377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65300ca3b0 0x7f65300aef20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.210+0000 7f65374cd700 1 --2- 172.21.15.7:0/3199666377 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f653015a180 0x7f65300c3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.210+0000 7f65374cd700 1 -- 172.21.15.7:0/3199666377 >> 172.21.15.7:0/3199666377 conn(0x7f65300679b0 msgr2=0x7f6530067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.210+0000 7f65374cd700 1 -- 172.21.15.7:0/3199666377 shutdown_connections 2024-01-27T02:34:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.210+0000 7f65374cd700 1 -- 172.21.15.7:0/3199666377 wait complete. 2024-01-27T02:34:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.211+0000 7f65374cd700 1 Processor -- start 2024-01-27T02:34:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.211+0000 7f65374cd700 1 -- start start 2024-01-27T02:34:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.212+0000 7f65374cd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65300ca3b0 0x7f653016aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.212+0000 7f65374cd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f653015a180 0x7f6530159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.212+0000 7f65374cd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f653015a550 0x7f653015fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.212+0000 7f65374cd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6530097db0 con 0x7f653015a180 2024-01-27T02:34:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.212+0000 7f65374cd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6530098090 con 0x7f653015a550 2024-01-27T02:34:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.212+0000 7f65374cd700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6530098370 con 0x7f65300ca3b0 2024-01-27T02:34:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.212+0000 7f6534a68700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f653015a180 0x7f6530159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.212+0000 7f6534a68700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f653015a180 0x7f6530159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52714/0 (socket says 172.21.15.7:52714) 2024-01-27T02:34:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.212+0000 7f6534a68700 1 -- 172.21.15.7:0/329953202 learned_addr learned my addr 172.21.15.7:0/329953202 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.212+0000 7f6535a6a700 1 --2- 172.21.15.7:0/329953202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f653015a550 0x7f653015fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.212+0000 7f6535269700 1 --2- 172.21.15.7:0/329953202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65300ca3b0 0x7f653016aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.213+0000 7f6534a68700 1 -- 172.21.15.7:0/329953202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65300ca3b0 msgr2=0x7f653016aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.213+0000 7f6534a68700 1 --2- 172.21.15.7:0/329953202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65300ca3b0 0x7f653016aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.213+0000 7f6534a68700 1 -- 172.21.15.7:0/329953202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f653015a550 msgr2=0x7f653015fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.213+0000 7f6534a68700 1 --2- 172.21.15.7:0/329953202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f653015a550 0x7f653015fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.213+0000 7f6534a68700 1 -- 172.21.15.7:0/329953202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652c0093c0 con 0x7f653015a180 2024-01-27T02:34:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.213+0000 7f6534a68700 1 --2- 172.21.15.7:0/329953202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f653015a180 0x7f6530159410 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f652000c7e0 tx=0x7f6520007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.213+0000 7f65257fa700 1 -- 172.21.15.7:0/329953202 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6520005b30 con 0x7f653015a180 2024-01-27T02:34:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.213+0000 7f65257fa700 1 -- 172.21.15.7:0/329953202 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6520005c90 con 0x7f653015a180 2024-01-27T02:34:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.213+0000 7f65374cd700 1 -- 172.21.15.7:0/329953202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6528009a20 con 0x7f653015a180 2024-01-27T02:34:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.213+0000 7f65257fa700 1 -- 172.21.15.7:0/329953202 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6520007300 con 0x7f653015a180 2024-01-27T02:34:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.213+0000 7f65374cd700 1 -- 172.21.15.7:0/329953202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6530098930 con 0x7f653015a180 2024-01-27T02:34:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.214+0000 7f65374cd700 1 -- 172.21.15.7:0/329953202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6530072060 con 0x7f653015a180 2024-01-27T02:34:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.215+0000 7f65257fa700 1 -- 172.21.15.7:0/329953202 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f652001b3b0 con 0x7f653015a180 2024-01-27T02:34:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.215+0000 7f65257fa700 1 --2- 172.21.15.7:0/329953202 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6514066f30 0x7f65140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.215+0000 7f65257fa700 1 -- 172.21.15.7:0/329953202 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f6520083a90 con 0x7f653015a180 2024-01-27T02:34:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.215+0000 7f6535269700 1 --2- 172.21.15.7:0/329953202 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6514066f30 0x7f65140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.216+0000 7f6535269700 1 --2- 172.21.15.7:0/329953202 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6514066f30 0x7f65140693e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f65300ccea0 tx=0x7f652c009120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.220+0000 7f65257fa700 1 -- 172.21.15.7:0/329953202 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6520050cc0 con 0x7f653015a180 2024-01-27T02:34:14.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:14.469+0000 7f65374cd700 1 -- 172.21.15.7:0/329953202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f65300cacb0 con 0x7f653015a180 2024-01-27T02:34:15.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.055+0000 7f65257fa700 1 -- 172.21.15.7:0/329953202 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v30) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6520017020 con 0x7f653015a180 2024-01-27T02:34:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.062+0000 7f65374cd700 1 -- 172.21.15.7:0/329953202 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6514066f30 msgr2=0x7f65140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.062+0000 7f65374cd700 1 --2- 172.21.15.7:0/329953202 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6514066f30 0x7f65140693e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f65300ccea0 tx=0x7f652c009120 comp rx=0 tx=0).stop 2024-01-27T02:34:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.062+0000 7f65374cd700 1 -- 172.21.15.7:0/329953202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f653015a180 msgr2=0x7f6530159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.062+0000 7f65374cd700 1 --2- 172.21.15.7:0/329953202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f653015a180 0x7f6530159410 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f652000c7e0 tx=0x7f6520007df0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.062+0000 7f65374cd700 1 -- 172.21.15.7:0/329953202 shutdown_connections 2024-01-27T02:34:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.062+0000 7f65374cd700 1 --2- 172.21.15.7:0/329953202 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6514066f30 0x7f65140693e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.062+0000 7f65374cd700 1 --2- 172.21.15.7:0/329953202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f653015a550 0x7f653015fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.062+0000 7f65374cd700 1 --2- 172.21.15.7:0/329953202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65300ca3b0 0x7f653016aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.062+0000 7f65374cd700 1 --2- 172.21.15.7:0/329953202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f653015a180 0x7f6530159410 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.062+0000 7f65374cd700 1 -- 172.21.15.7:0/329953202 >> 172.21.15.7:0/329953202 conn(0x7f65300679b0 msgr2=0x7f653006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.062+0000 7f65374cd700 1 -- 172.21.15.7:0/329953202 shutdown_connections 2024-01-27T02:34:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.062+0000 7f65374cd700 1 -- 172.21.15.7:0/329953202 wait complete. 2024-01-27T02:34:15.065 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 8 and data pool 9 2024-01-27T02:34:15.080 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:34:15.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.215+0000 7feca15ff700 1 Processor -- start 2024-01-27T02:34:15.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.215+0000 7feca15ff700 1 -- start start 2024-01-27T02:34:15.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.216+0000 7feca15ff700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fec9c15a450 0x7fec9c0b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:15.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.216+0000 7feca15ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec9c0cc980 con 0x7fec9c0b2030 2024-01-27T02:34:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.216+0000 7feca15ff700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec9c0ccc20 con 0x7fec9c0ce0f0 2024-01-27T02:34:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.216+0000 7feca15ff700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec9c0ccf00 con 0x7fec9c15a450 2024-01-27T02:34:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.216+0000 7fec9a7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fec9c0b2030 0x7fec9c0cd9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42140/0 (socket says 172.21.15.7:42140) 2024-01-27T02:34:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.216+0000 7fec9a7fc700 1 -- 172.21.15.7:0/2039211391 learned_addr learned my addr 172.21.15.7:0/2039211391 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:34:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.216+0000 7fec9affd700 1 --2- 172.21.15.7:0/2039211391 >> v2:172.21.15.188:3300/0 conn(0x7fec9c15a450 0x7fec9c0b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.216+0000 7fec99ffb700 1 -- 172.21.15.7:0/2039211391 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 587718612 0 0) 0x7fec9c0cc980 con 0x7fec9c0b2030 2024-01-27T02:34:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.217+0000 7fec99ffb700 1 -- 172.21.15.7:0/2039211391 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec84003570 con 0x7fec9c0b2030 2024-01-27T02:34:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.217+0000 7fec99ffb700 1 -- 172.21.15.7:0/2039211391 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 203540301 0 0) 0x7fec9c0ccc20 con 0x7fec9c0ce0f0 2024-01-27T02:34:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.217+0000 7fec99ffb700 1 -- 172.21.15.7:0/2039211391 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec9c0cc980 con 0x7fec9c0ce0f0 2024-01-27T02:34:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.217+0000 7fec9affd700 1 -- 172.21.15.7:0/2039211391 >> v1:172.21.15.188:6789/0 conn(0x7fec9c0ce0f0 legacy=0x7fec9c0c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.217+0000 7fec9affd700 1 -- 172.21.15.7:0/2039211391 >> v1:172.21.15.7:6789/0 conn(0x7fec9c0b2030 legacy=0x7fec9c0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.217+0000 7fec9affd700 1 -- 172.21.15.7:0/2039211391 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c0cd1e0 con 0x7fec9c15a450 2024-01-27T02:34:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.217+0000 7fec9affd700 1 --2- 172.21.15.7:0/2039211391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec9c15a450 0x7fec9c0b1af0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0098d0 tx=0x7fec8c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=497c96f27f430b93 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.217+0000 7fec99ffb700 1 -- 172.21.15.7:0/2039211391 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c005c10 con 0x7fec9c15a450 2024-01-27T02:34:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.217+0000 7fec99ffb700 1 -- 172.21.15.7:0/2039211391 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8c005d70 con 0x7fec9c15a450 2024-01-27T02:34:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.217+0000 7fec99ffb700 1 -- 172.21.15.7:0/2039211391 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c006260 con 0x7fec9c15a450 2024-01-27T02:34:15.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.218+0000 7feca15ff700 1 -- 172.21.15.7:0/2039211391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec9c15a450 msgr2=0x7fec9c0b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:15.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.218+0000 7feca15ff700 1 --2- 172.21.15.7:0/2039211391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec9c15a450 0x7fec9c0b1af0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0098d0 tx=0x7fec8c009120 comp rx=0 tx=0).stop 2024-01-27T02:34:15.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.218+0000 7feca15ff700 1 -- 172.21.15.7:0/2039211391 shutdown_connections 2024-01-27T02:34:15.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.218+0000 7feca15ff700 1 --2- 172.21.15.7:0/2039211391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec9c15a450 0x7fec9c0b1af0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.218+0000 7feca15ff700 1 -- 172.21.15.7:0/2039211391 >> 172.21.15.7:0/2039211391 conn(0x7fec9c0711c0 msgr2=0x7fec9c0ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:15.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.218+0000 7feca15ff700 1 -- 172.21.15.7:0/2039211391 shutdown_connections 2024-01-27T02:34:15.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.218+0000 7feca15ff700 1 -- 172.21.15.7:0/2039211391 wait complete. 2024-01-27T02:34:15.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.219+0000 7feca15ff700 1 Processor -- start 2024-01-27T02:34:15.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.219+0000 7feca15ff700 1 -- start start 2024-01-27T02:34:15.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.220+0000 7feca15ff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec9c0b2030 0x7fec9c0c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:15.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.220+0000 7feca15ff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec9c0ce0f0 0x7fec9c0c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:15.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.220+0000 7feca15ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec9c15a450 0x7fec9c0c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:15.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.220+0000 7feca15ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0b2ee0 con 0x7fec9c15a450 2024-01-27T02:34:15.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.220+0000 7feca15ff700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c156a00 con 0x7fec9c0ce0f0 2024-01-27T02:34:15.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.220+0000 7feca15ff700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0c56a0 con 0x7fec9c0b2030 2024-01-27T02:34:15.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.220+0000 7fec9b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec9c15a450 0x7fec9c0c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:15.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.220+0000 7fec9b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec9c15a450 0x7fec9c0c6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52728/0 (socket says 172.21.15.7:52728) 2024-01-27T02:34:15.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.220+0000 7fec9b7fe700 1 -- 172.21.15.7:0/1163640585 learned_addr learned my addr 172.21.15.7:0/1163640585 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:15.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.220+0000 7fec9affd700 1 --2- 172.21.15.7:0/1163640585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec9c0b2030 0x7fec9c0c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:15.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.220+0000 7fec9a7fc700 1 --2- 172.21.15.7:0/1163640585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec9c0ce0f0 0x7fec9c0c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:15.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.221+0000 7fec9b7fe700 1 -- 172.21.15.7:0/1163640585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec9c0b2030 msgr2=0x7fec9c0c50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:15.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.221+0000 7fec9b7fe700 1 --2- 172.21.15.7:0/1163640585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec9c0b2030 0x7fec9c0c50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.221+0000 7fec9b7fe700 1 -- 172.21.15.7:0/1163640585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec9c0ce0f0 msgr2=0x7fec9c0c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:15.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.221+0000 7fec9b7fe700 1 --2- 172.21.15.7:0/1163640585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec9c0ce0f0 0x7fec9c0c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.221+0000 7fec9b7fe700 1 -- 172.21.15.7:0/1163640585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8c0093c0 con 0x7fec9c15a450 2024-01-27T02:34:15.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.221+0000 7fec9b7fe700 1 --2- 172.21.15.7:0/1163640585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec9c15a450 0x7fec9c0c6a40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fec9400c910 tx=0x7fec940027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:15.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.221+0000 7fec82ffd700 1 -- 172.21.15.7:0/1163640585 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94012630 con 0x7fec9c15a450 2024-01-27T02:34:15.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.221+0000 7fec82ffd700 1 -- 172.21.15.7:0/1163640585 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec940127b0 con 0x7fec9c15a450 2024-01-27T02:34:15.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.221+0000 7fec82ffd700 1 -- 172.21.15.7:0/1163640585 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94012b70 con 0x7fec9c15a450 2024-01-27T02:34:15.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.221+0000 7feca15ff700 1 -- 172.21.15.7:0/1163640585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9c0b25b0 con 0x7fec9c15a450 2024-01-27T02:34:15.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.221+0000 7feca15ff700 1 -- 172.21.15.7:0/1163640585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9c0b2b70 con 0x7fec9c15a450 2024-01-27T02:34:15.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.222+0000 7feca15ff700 1 -- 172.21.15.7:0/1163640585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec9c0b3090 con 0x7fec9c15a450 2024-01-27T02:34:15.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.223+0000 7fec82ffd700 1 -- 172.21.15.7:0/1163640585 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fec9401b400 con 0x7fec9c15a450 2024-01-27T02:34:15.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.223+0000 7fec82ffd700 1 --2- 172.21.15.7:0/1163640585 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fec88066f30 0x7fec880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:15.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.223+0000 7fec82ffd700 1 -- 172.21.15.7:0/1163640585 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fec94084840 con 0x7fec9c15a450 2024-01-27T02:34:15.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.223+0000 7fec9affd700 1 --2- 172.21.15.7:0/1163640585 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fec88066f30 0x7fec880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:15.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.224+0000 7fec9affd700 1 --2- 172.21.15.7:0/1163640585 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fec88066f30 0x7fec880693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fec8c009160 tx=0x7fec8c016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.228+0000 7fec82ffd700 1 -- 172.21.15.7:0/1163640585 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec9c0b3090 con 0x7fec9c15a450 2024-01-27T02:34:15.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.432+0000 7feca15ff700 1 -- 172.21.15.7:0/1163640585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec9c04c710 con 0x7fec9c15a450 2024-01-27T02:34:15.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.433+0000 7fec82ffd700 1 -- 172.21.15.7:0/1163640585 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+15315 (secure 0 0 0) 0x7fec94051ab0 con 0x7fec9c15a450 2024-01-27T02:34:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:34:15.435 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":64,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:34:14.471110+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:34:12.775862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-27T02:34:13.445542+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:34:15.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.436+0000 7feca15ff700 1 -- 172.21.15.7:0/1163640585 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fec88066f30 msgr2=0x7fec880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:15.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.436+0000 7feca15ff700 1 --2- 172.21.15.7:0/1163640585 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fec88066f30 0x7fec880693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fec8c009160 tx=0x7fec8c016000 comp rx=0 tx=0).stop 2024-01-27T02:34:15.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.436+0000 7feca15ff700 1 -- 172.21.15.7:0/1163640585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec9c15a450 msgr2=0x7fec9c0c6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:15.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.436+0000 7feca15ff700 1 --2- 172.21.15.7:0/1163640585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec9c15a450 0x7fec9c0c6a40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fec9400c910 tx=0x7fec940027a0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.436+0000 7feca15ff700 1 -- 172.21.15.7:0/1163640585 shutdown_connections 2024-01-27T02:34:15.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.436+0000 7feca15ff700 1 --2- 172.21.15.7:0/1163640585 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fec88066f30 0x7fec880693e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.436+0000 7feca15ff700 1 --2- 172.21.15.7:0/1163640585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec9c15a450 0x7fec9c0c6a40 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.436+0000 7feca15ff700 1 --2- 172.21.15.7:0/1163640585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec9c0b2030 0x7fec9c0c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.436+0000 7feca15ff700 1 --2- 172.21.15.7:0/1163640585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec9c0ce0f0 0x7fec9c0c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.436+0000 7feca15ff700 1 -- 172.21.15.7:0/1163640585 >> 172.21.15.7:0/1163640585 conn(0x7fec9c0711c0 msgr2=0x7fec9c0ca8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:15.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.436+0000 7feca15ff700 1 -- 172.21.15.7:0/1163640585 shutdown_connections 2024-01-27T02:34:15.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.436+0000 7feca15ff700 1 -- 172.21.15.7:0/1163640585 wait complete. 2024-01-27T02:34:15.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:34:15.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.581+0000 7f64ce993700 1 Processor -- start 2024-01-27T02:34:15.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.581+0000 7f64ce993700 1 -- start start 2024-01-27T02:34:15.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.582+0000 7f64ce993700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f64c80c9bb0 0x7f64c8157f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:15.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.582+0000 7f64ce993700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f64c806e330 0x7f64c806e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:15.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.582+0000 7f64ce993700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c806c880 con 0x7f64c80c9f80 2024-01-27T02:34:15.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.582+0000 7f64ce993700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c806cd10 con 0x7f64c806e330 2024-01-27T02:34:15.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.582+0000 7f64ce993700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c806ec90 con 0x7f64c80c9bb0 2024-01-27T02:34:15.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.582+0000 7f64ccf30700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f64c806e330 0x7f64c806e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:15.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.582+0000 7f64ccf30700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f64c806e330 0x7f64c806e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52744/0 (socket says 172.21.15.7:52744) 2024-01-27T02:34:15.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.582+0000 7f64ccf30700 1 -- 172.21.15.7:0/359405281 learned_addr learned my addr 172.21.15.7:0/359405281 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:15.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.582+0000 7f64c77fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f64c80c9f80 0x7f64c806b400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42156/0 (socket says 172.21.15.7:42156) 2024-01-27T02:34:15.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.583+0000 7f64c7fff700 1 --2- 172.21.15.7:0/359405281 >> v2:172.21.15.188:3300/0 conn(0x7f64c80c9bb0 0x7f64c8157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:15.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.583+0000 7f64ccf30700 1 -- 172.21.15.7:0/359405281 >> v2:172.21.15.188:3300/0 conn(0x7f64c80c9bb0 msgr2=0x7f64c8157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:15.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.583+0000 7f64ccf30700 1 --2- 172.21.15.7:0/359405281 >> v2:172.21.15.188:3300/0 conn(0x7f64c80c9bb0 0x7f64c8157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.583+0000 7f64ccf30700 1 -- 172.21.15.7:0/359405281 >> v1:172.21.15.7:6789/0 conn(0x7f64c80c9f80 legacy=0x7f64c806b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:15.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.583+0000 7f64ccf30700 1 -- 172.21.15.7:0/359405281 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c806bc70 con 0x7f64c806e330 2024-01-27T02:34:15.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.583+0000 7f64c6ffd700 1 -- 172.21.15.7:0/359405281 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888307252 0 0) 0x7f64c806c880 con 0x7f64c80c9f80 2024-01-27T02:34:15.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.583+0000 7f64ccf30700 1 --2- 172.21.15.7:0/359405281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c806e330 0x7f64c806e750 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f64c806e020 tx=0x7f64c000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78cdddc04426ea6 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:15.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.583+0000 7f64c6ffd700 1 -- 172.21.15.7:0/359405281 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c0010070 con 0x7f64c806e330 2024-01-27T02:34:15.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.583+0000 7f64c6ffd700 1 -- 172.21.15.7:0/359405281 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c000de90 con 0x7f64c806e330 2024-01-27T02:34:15.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.583+0000 7f64c6ffd700 1 -- 172.21.15.7:0/359405281 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c000b320 con 0x7f64c806e330 2024-01-27T02:34:15.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.584+0000 7f64ce993700 1 -- 172.21.15.7:0/359405281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c806e330 msgr2=0x7f64c806e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:15.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.584+0000 7f64ce993700 1 --2- 172.21.15.7:0/359405281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c806e330 0x7f64c806e750 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f64c806e020 tx=0x7f64c000ba50 comp rx=0 tx=0).stop 2024-01-27T02:34:15.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.584+0000 7f64ce993700 1 -- 172.21.15.7:0/359405281 shutdown_connections 2024-01-27T02:34:15.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.584+0000 7f64ce993700 1 --2- 172.21.15.7:0/359405281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c806e330 0x7f64c806e750 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.584+0000 7f64ce993700 1 --2- 172.21.15.7:0/359405281 >> v2:172.21.15.188:3300/0 conn(0x7f64c80c9bb0 0x7f64c8157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.584+0000 7f64ce993700 1 -- 172.21.15.7:0/359405281 >> 172.21.15.7:0/359405281 conn(0x7f64c80cdc30 msgr2=0x7f64c80ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:15.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.584+0000 7f64ce993700 1 -- 172.21.15.7:0/359405281 shutdown_connections 2024-01-27T02:34:15.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.584+0000 7f64ce993700 1 -- 172.21.15.7:0/359405281 wait complete. 2024-01-27T02:34:15.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.585+0000 7f64ce993700 1 Processor -- start 2024-01-27T02:34:15.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.585+0000 7f64ce993700 1 -- start start 2024-01-27T02:34:15.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.586+0000 7f64ce993700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c806e330 0x7f64c80c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:15.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.586+0000 7f64ce993700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64c80c9bb0 0x7f64c814fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:15.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.586+0000 7f64ce993700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64c80c9f80 0x7f64c80b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:15.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.586+0000 7f64ce993700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c80c7fd0 con 0x7f64c806e330 2024-01-27T02:34:15.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.586+0000 7f64ce993700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c8150fb0 con 0x7f64c80c9bb0 2024-01-27T02:34:15.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.586+0000 7f64ce993700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c80c4e80 con 0x7f64c80c9f80 2024-01-27T02:34:15.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.586+0000 7f64c7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c806e330 0x7f64c80c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:15.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.586+0000 7f64c7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c806e330 0x7f64c80c78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52748/0 (socket says 172.21.15.7:52748) 2024-01-27T02:34:15.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.586+0000 7f64c7fff700 1 -- 172.21.15.7:0/153357567 learned_addr learned my addr 172.21.15.7:0/153357567 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:15.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.586+0000 7f64c77fe700 1 --2- 172.21.15.7:0/153357567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64c80c9bb0 0x7f64c814fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:15.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.586+0000 7f64ccf30700 1 --2- 172.21.15.7:0/153357567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64c80c9f80 0x7f64c80b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:15.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.586+0000 7f64c7fff700 1 -- 172.21.15.7:0/153357567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64c80c9f80 msgr2=0x7f64c80b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:15.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.586+0000 7f64c7fff700 1 --2- 172.21.15.7:0/153357567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64c80c9f80 0x7f64c80b09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.586+0000 7f64c7fff700 1 -- 172.21.15.7:0/153357567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64c80c9bb0 msgr2=0x7f64c814fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:15.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.586+0000 7f64c7fff700 1 --2- 172.21.15.7:0/153357567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64c80c9bb0 0x7f64c814fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.586+0000 7f64c7fff700 1 -- 172.21.15.7:0/153357567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c000bcf0 con 0x7f64c806e330 2024-01-27T02:34:15.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.586+0000 7f64c77fe700 1 --2- 172.21.15.7:0/153357567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64c80c9bb0 0x7f64c814fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:15.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.586+0000 7f64ccf30700 1 --2- 172.21.15.7:0/153357567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64c80c9f80 0x7f64c80b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:15.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.587+0000 7f64c7fff700 1 --2- 172.21.15.7:0/153357567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c806e330 0x7f64c80c78f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f64b800d890 tx=0x7f64b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:15.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.587+0000 7f64bffff700 1 -- 172.21.15.7:0/153357567 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b800b600 con 0x7f64c806e330 2024-01-27T02:34:15.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.587+0000 7f64bffff700 1 -- 172.21.15.7:0/153357567 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b800b760 con 0x7f64c806e330 2024-01-27T02:34:15.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.587+0000 7f64bffff700 1 -- 172.21.15.7:0/153357567 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b800bae0 con 0x7f64c806e330 2024-01-27T02:34:15.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.587+0000 7f64ce993700 1 -- 172.21.15.7:0/153357567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c80c5160 con 0x7f64c806e330 2024-01-27T02:34:15.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.587+0000 7f64ce993700 1 -- 172.21.15.7:0/153357567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c80c5720 con 0x7f64c806e330 2024-01-27T02:34:15.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.588+0000 7f64ce993700 1 -- 172.21.15.7:0/153357567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c804c710 con 0x7f64c806e330 2024-01-27T02:34:15.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.588+0000 7f64bffff700 1 -- 172.21.15.7:0/153357567 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f64b801e460 con 0x7f64c806e330 2024-01-27T02:34:15.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.589+0000 7f64bffff700 1 --2- 172.21.15.7:0/153357567 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f64a8066f30 0x7f64a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:15.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.589+0000 7f64bffff700 1 -- 172.21.15.7:0/153357567 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f64b8084c10 con 0x7f64c806e330 2024-01-27T02:34:15.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.589+0000 7f64c77fe700 1 --2- 172.21.15.7:0/153357567 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f64a8066f30 0x7f64a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:15.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.590+0000 7f64c77fe700 1 --2- 172.21.15.7:0/153357567 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f64a8066f30 0x7f64a80693e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f64b0007fd0 tx=0x7f64b0007e30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:15.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.593+0000 7f64bffff700 1 -- 172.21.15.7:0/153357567 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64b8051e80 con 0x7f64c806e330 2024-01-27T02:34:15.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.796+0000 7f64ce993700 1 -- 172.21.15.7:0/153357567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64c80c58e0 con 0x7f64c806e330 2024-01-27T02:34:15.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.797+0000 7f64bffff700 1 -- 172.21.15.7:0/153357567 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+15315 (secure 0 0 0) 0x7f64b80554a0 con 0x7f64c806e330 2024-01-27T02:34:15.799 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:34:15.799 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":64,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:34:14.471110+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:34:12.775862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-27T02:34:13.445542+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:34:15.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.799+0000 7f64ce993700 1 -- 172.21.15.7:0/153357567 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f64a8066f30 msgr2=0x7f64a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:15.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.799+0000 7f64ce993700 1 --2- 172.21.15.7:0/153357567 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f64a8066f30 0x7f64a80693e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f64b0007fd0 tx=0x7f64b0007e30 comp rx=0 tx=0).stop 2024-01-27T02:34:15.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.799+0000 7f64ce993700 1 -- 172.21.15.7:0/153357567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c806e330 msgr2=0x7f64c80c78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:15.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.799+0000 7f64ce993700 1 --2- 172.21.15.7:0/153357567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c806e330 0x7f64c80c78f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f64b800d890 tx=0x7f64b8008040 comp rx=0 tx=0).stop 2024-01-27T02:34:15.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.799+0000 7f64ce993700 1 -- 172.21.15.7:0/153357567 shutdown_connections 2024-01-27T02:34:15.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.799+0000 7f64ce993700 1 --2- 172.21.15.7:0/153357567 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f64a8066f30 0x7f64a80693e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.799+0000 7f64ce993700 1 --2- 172.21.15.7:0/153357567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64c80c9f80 0x7f64c80b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.799+0000 7f64ce993700 1 --2- 172.21.15.7:0/153357567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c806e330 0x7f64c80c78f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.799+0000 7f64ce993700 1 --2- 172.21.15.7:0/153357567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64c80c9bb0 0x7f64c814fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.799+0000 7f64ce993700 1 -- 172.21.15.7:0/153357567 >> 172.21.15.7:0/153357567 conn(0x7f64c80cdc30 msgr2=0x7f64c80cd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:15.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.799+0000 7f64ce993700 1 -- 172.21.15.7:0/153357567 shutdown_connections 2024-01-27T02:34:15.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.799+0000 7f64ce993700 1 -- 172.21.15.7:0/153357567 wait complete. 2024-01-27T02:34:15.815 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-27T02:34:15.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.948+0000 7fd0d5fef700 1 Processor -- start 2024-01-27T02:34:15.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.948+0000 7fd0d5fef700 1 -- start start 2024-01-27T02:34:15.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.949+0000 7fd0d5fef700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd0d00cba60 0x7fd0d00ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.949+0000 7fd0d5fef700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd0d00cbe30 0x7fd0d015b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.949+0000 7fd0d5fef700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d00679b0 con 0x7fd0d006b4e0 2024-01-27T02:34:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.949+0000 7fd0d5fef700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d0067c90 con 0x7fd0d00cbe30 2024-01-27T02:34:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.949+0000 7fd0d5fef700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d015b7b0 con 0x7fd0d00cba60 2024-01-27T02:34:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.949+0000 7fd0ceffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd0d00cbe30 0x7fd0d015b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.949+0000 7fd0cffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd0d006b4e0 0x7fd0d006b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42164/0 (socket says 172.21.15.7:42164) 2024-01-27T02:34:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.950+0000 7fd0cffff700 1 -- 172.21.15.7:0/3121081757 learned_addr learned my addr 172.21.15.7:0/3121081757 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:34:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.950+0000 7fd0ceffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd0d00cbe30 0x7fd0d015b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52754/0 (socket says 172.21.15.7:52754) 2024-01-27T02:34:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.950+0000 7fd0cf7fe700 1 --2- 172.21.15.7:0/3121081757 >> v2:172.21.15.188:3300/0 conn(0x7fd0d00cba60 0x7fd0d00ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.950+0000 7fd0ceffd700 1 -- 172.21.15.7:0/3121081757 >> v2:172.21.15.188:3300/0 conn(0x7fd0d00cba60 msgr2=0x7fd0d00ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.950+0000 7fd0ceffd700 1 --2- 172.21.15.7:0/3121081757 >> v2:172.21.15.188:3300/0 conn(0x7fd0d00cba60 0x7fd0d00ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.950+0000 7fd0ceffd700 1 -- 172.21.15.7:0/3121081757 >> v1:172.21.15.7:6789/0 conn(0x7fd0d006b4e0 legacy=0x7fd0d006b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.950+0000 7fd0ceffd700 1 -- 172.21.15.7:0/3121081757 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d00cca50 con 0x7fd0d00cbe30 2024-01-27T02:34:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.950+0000 7fd0ce7fc700 1 -- 172.21.15.7:0/3121081757 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 349503973 0 0) 0x7fd0d00679b0 con 0x7fd0d006b4e0 2024-01-27T02:34:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.950+0000 7fd0cf7fe700 1 --2- 172.21.15.7:0/3121081757 >> v2:172.21.15.188:3300/0 conn(0x7fd0d00cba60 0x7fd0d00ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.950+0000 7fd0ceffd700 1 --2- 172.21.15.7:0/3121081757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0d00cbe30 0x7fd0d015b270 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd0d00cfd50 tx=0x7fd0c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2c55ded4c561cb server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.950+0000 7fd0ce7fc700 1 -- 172.21.15.7:0/3121081757 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c400b6b0 con 0x7fd0d00cbe30 2024-01-27T02:34:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.950+0000 7fd0ce7fc700 1 -- 172.21.15.7:0/3121081757 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c400b810 con 0x7fd0d00cbe30 2024-01-27T02:34:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.950+0000 7fd0ce7fc700 1 -- 172.21.15.7:0/3121081757 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c400bbb0 con 0x7fd0d00cbe30 2024-01-27T02:34:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.951+0000 7fd0d5fef700 1 -- 172.21.15.7:0/3121081757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0d00cbe30 msgr2=0x7fd0d015b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.951+0000 7fd0d5fef700 1 --2- 172.21.15.7:0/3121081757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0d00cbe30 0x7fd0d015b270 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd0d00cfd50 tx=0x7fd0c40091a0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.951+0000 7fd0d5fef700 1 -- 172.21.15.7:0/3121081757 shutdown_connections 2024-01-27T02:34:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.951+0000 7fd0d5fef700 1 --2- 172.21.15.7:0/3121081757 >> v2:172.21.15.188:3300/0 conn(0x7fd0d00cba60 0x7fd0d00ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.951+0000 7fd0d5fef700 1 --2- 172.21.15.7:0/3121081757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0d00cbe30 0x7fd0d015b270 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.951+0000 7fd0d5fef700 1 -- 172.21.15.7:0/3121081757 >> 172.21.15.7:0/3121081757 conn(0x7fd0d0063580 msgr2=0x7fd0d0063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.951+0000 7fd0d5fef700 1 -- 172.21.15.7:0/3121081757 shutdown_connections 2024-01-27T02:34:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.951+0000 7fd0d5fef700 1 -- 172.21.15.7:0/3121081757 wait complete. 2024-01-27T02:34:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.952+0000 7fd0d5fef700 1 Processor -- start 2024-01-27T02:34:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.952+0000 7fd0d5fef700 1 -- start start 2024-01-27T02:34:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.953+0000 7fd0d5fef700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0d006b4e0 0x7fd0d00ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.953+0000 7fd0d5fef700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0d00cba60 0x7fd0d0156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.953+0000 7fd0d5fef700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0d00cbe30 0x7fd0d016b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.953+0000 7fd0d5fef700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d0157060 con 0x7fd0d00cbe30 2024-01-27T02:34:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.953+0000 7fd0d5fef700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d00b1a80 con 0x7fd0d006b4e0 2024-01-27T02:34:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.953+0000 7fd0d5fef700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d016b9a0 con 0x7fd0d00cba60 2024-01-27T02:34:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.953+0000 7fd0cffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0d00cbe30 0x7fd0d016b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.953+0000 7fd0cffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0d00cbe30 0x7fd0d016b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52764/0 (socket says 172.21.15.7:52764) 2024-01-27T02:34:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.953+0000 7fd0cffff700 1 -- 172.21.15.7:0/3900061234 learned_addr learned my addr 172.21.15.7:0/3900061234 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.953+0000 7fd0cf7fe700 1 --2- 172.21.15.7:0/3900061234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0d006b4e0 0x7fd0d00ac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.953+0000 7fd0ceffd700 1 --2- 172.21.15.7:0/3900061234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0d00cba60 0x7fd0d0156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:15.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.954+0000 7fd0cffff700 1 -- 172.21.15.7:0/3900061234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0d00cba60 msgr2=0x7fd0d0156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:15.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.954+0000 7fd0cffff700 1 --2- 172.21.15.7:0/3900061234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0d00cba60 0x7fd0d0156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.954+0000 7fd0cffff700 1 -- 172.21.15.7:0/3900061234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0d006b4e0 msgr2=0x7fd0d00ac0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:15.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.954+0000 7fd0cffff700 1 --2- 172.21.15.7:0/3900061234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0d006b4e0 0x7fd0d00ac0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:15.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.954+0000 7fd0cffff700 1 -- 172.21.15.7:0/3900061234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c4009420 con 0x7fd0d00cbe30 2024-01-27T02:34:15.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.954+0000 7fd0cffff700 1 --2- 172.21.15.7:0/3900061234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0d00cbe30 0x7fd0d016b460 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800ccc0 tx=0x7fd0c8001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:15.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.954+0000 7fd0b77fe700 1 -- 172.21.15.7:0/3900061234 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c800a880 con 0x7fd0d00cbe30 2024-01-27T02:34:15.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.954+0000 7fd0b77fe700 1 -- 172.21.15.7:0/3900061234 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c800a9e0 con 0x7fd0d00cbe30 2024-01-27T02:34:15.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.954+0000 7fd0d5fef700 1 -- 172.21.15.7:0/3900061234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d00ac740 con 0x7fd0d00cbe30 2024-01-27T02:34:15.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.954+0000 7fd0b77fe700 1 -- 172.21.15.7:0/3900061234 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c800ad80 con 0x7fd0d00cbe30 2024-01-27T02:34:15.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.954+0000 7fd0d5fef700 1 -- 172.21.15.7:0/3900061234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d00acd00 con 0x7fd0d00cbe30 2024-01-27T02:34:15.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.955+0000 7fd0d5fef700 1 -- 172.21.15.7:0/3900061234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d00b1c30 con 0x7fd0d00cbe30 2024-01-27T02:34:15.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.956+0000 7fd0b77fe700 1 -- 172.21.15.7:0/3900061234 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd0c8007300 con 0x7fd0d00cbe30 2024-01-27T02:34:15.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.956+0000 7fd0b77fe700 1 --2- 172.21.15.7:0/3900061234 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd0bc066f30 0x7fd0bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:15.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.956+0000 7fd0b77fe700 1 -- 172.21.15.7:0/3900061234 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd0c8084800 con 0x7fd0d00cbe30 2024-01-27T02:34:15.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.956+0000 7fd0cf7fe700 1 --2- 172.21.15.7:0/3900061234 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd0bc066f30 0x7fd0bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:15.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.957+0000 7fd0cf7fe700 1 --2- 172.21.15.7:0/3900061234 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd0bc066f30 0x7fd0bc0693e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd0d00b08a0 tx=0x7fd0c0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:15.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:15.961+0000 7fd0b77fe700 1 -- 172.21.15.7:0/3900061234 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0d00b1c30 con 0x7fd0d00cbe30 2024-01-27T02:34:16.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:16.217+0000 7fd0d5fef700 1 -- 172.21.15.7:0/3900061234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd0d00b1c30 con 0x7fd0d00cbe30 2024-01-27T02:34:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.068+0000 7fd0b77fe700 1 -- 172.21.15.7:0/3900061234 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd0d00b1c30 con 0x7fd0d00cbe30 2024-01-27T02:34:17.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.071+0000 7fd0d5fef700 1 -- 172.21.15.7:0/3900061234 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd0bc066f30 msgr2=0x7fd0bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:17.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.071+0000 7fd0d5fef700 1 --2- 172.21.15.7:0/3900061234 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd0bc066f30 0x7fd0bc0693e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd0d00b08a0 tx=0x7fd0c0008040 comp rx=0 tx=0).stop 2024-01-27T02:34:17.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.071+0000 7fd0d5fef700 1 -- 172.21.15.7:0/3900061234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0d00cbe30 msgr2=0x7fd0d016b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:17.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.071+0000 7fd0d5fef700 1 --2- 172.21.15.7:0/3900061234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0d00cbe30 0x7fd0d016b460 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800ccc0 tx=0x7fd0c8001a20 comp rx=0 tx=0).stop 2024-01-27T02:34:17.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.071+0000 7fd0d5fef700 1 -- 172.21.15.7:0/3900061234 shutdown_connections 2024-01-27T02:34:17.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.071+0000 7fd0d5fef700 1 --2- 172.21.15.7:0/3900061234 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd0bc066f30 0x7fd0bc0693e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:17.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.071+0000 7fd0d5fef700 1 --2- 172.21.15.7:0/3900061234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0d00cbe30 0x7fd0d016b460 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:17.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.071+0000 7fd0d5fef700 1 --2- 172.21.15.7:0/3900061234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0d006b4e0 0x7fd0d00ac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:17.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.071+0000 7fd0d5fef700 1 --2- 172.21.15.7:0/3900061234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0d00cba60 0x7fd0d0156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:17.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.071+0000 7fd0d5fef700 1 -- 172.21.15.7:0/3900061234 >> 172.21.15.7:0/3900061234 conn(0x7fd0d0063580 msgr2=0x7fd0d0158860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:17.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.071+0000 7fd0d5fef700 1 -- 172.21.15.7:0/3900061234 shutdown_connections 2024-01-27T02:34:17.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.071+0000 7fd0d5fef700 1 -- 172.21.15.7:0/3900061234 wait complete. 2024-01-27T02:34:17.089 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-27T02:34:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.226+0000 7f05c36a2700 1 Processor -- start 2024-01-27T02:34:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.226+0000 7f05c36a2700 1 -- start start 2024-01-27T02:34:17.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.227+0000 7f05c36a2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f05bc141a90 0x7f05bc0cab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:17.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.227+0000 7f05c36a2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05bc0cbdf0 0x7f05bc0cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.227+0000 7f05c36a2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05bc15c7d0 con 0x7f05bc1416c0 2024-01-27T02:34:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.227+0000 7f05c36a2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05bc0cd9c0 con 0x7f05bc141a90 2024-01-27T02:34:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.227+0000 7f05c36a2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc0c9640 con 0x7f05bc0cbdf0 2024-01-27T02:34:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.227+0000 7f05c0c3d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f05bc141a90 0x7f05bc0cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.227+0000 7f05c143e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f05bc1416c0 0x7f05bc0cb6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42176/0 (socket says 172.21.15.7:42176) 2024-01-27T02:34:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.227+0000 7f05c143e700 1 -- 172.21.15.7:0/2690325167 learned_addr learned my addr 172.21.15.7:0/2690325167 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:34:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.228+0000 7f05c1c3f700 1 --2- 172.21.15.7:0/2690325167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05bc0cbdf0 0x7f05bc0cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.228+0000 7f05c0c3d700 1 -- 172.21.15.7:0/2690325167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05bc0cbdf0 msgr2=0x7f05bc0cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.228+0000 7f05c0c3d700 1 --2- 172.21.15.7:0/2690325167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05bc0cbdf0 0x7f05bc0cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.228+0000 7f05c0c3d700 1 -- 172.21.15.7:0/2690325167 >> v1:172.21.15.7:6789/0 conn(0x7f05bc1416c0 legacy=0x7f05bc0cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.228+0000 7f05c0c3d700 1 -- 172.21.15.7:0/2690325167 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc15bbc0 con 0x7f05bc141a90 2024-01-27T02:34:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.228+0000 7f05affff700 1 -- 172.21.15.7:0/2690325167 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912577456 0 0) 0x7f05bc15c7d0 con 0x7f05bc1416c0 2024-01-27T02:34:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.228+0000 7f05c0c3d700 1 --2- 172.21.15.7:0/2690325167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05bc141a90 0x7f05bc0cab00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f05b80098e0 tx=0x7f05b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5788e407da4362b3 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.228+0000 7f05affff700 1 -- 172.21.15.7:0/2690325167 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b800b640 con 0x7f05bc141a90 2024-01-27T02:34:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.228+0000 7f05affff700 1 -- 172.21.15.7:0/2690325167 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b800b7a0 con 0x7f05bc141a90 2024-01-27T02:34:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.228+0000 7f05affff700 1 -- 172.21.15.7:0/2690325167 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b800bb40 con 0x7f05bc141a90 2024-01-27T02:34:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.229+0000 7f05c36a2700 1 -- 172.21.15.7:0/2690325167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05bc141a90 msgr2=0x7f05bc0cab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.229+0000 7f05c36a2700 1 --2- 172.21.15.7:0/2690325167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05bc141a90 0x7f05bc0cab00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f05b80098e0 tx=0x7f05b80091a0 comp rx=0 tx=0).stop 2024-01-27T02:34:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.229+0000 7f05c36a2700 1 -- 172.21.15.7:0/2690325167 shutdown_connections 2024-01-27T02:34:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.229+0000 7f05c36a2700 1 --2- 172.21.15.7:0/2690325167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05bc0cbdf0 0x7f05bc0cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.229+0000 7f05c36a2700 1 --2- 172.21.15.7:0/2690325167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05bc141a90 0x7f05bc0cab00 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.229+0000 7f05c36a2700 1 -- 172.21.15.7:0/2690325167 >> 172.21.15.7:0/2690325167 conn(0x7f05bc063580 msgr2=0x7f05bc063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.229+0000 7f05c36a2700 1 -- 172.21.15.7:0/2690325167 shutdown_connections 2024-01-27T02:34:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.229+0000 7f05c36a2700 1 -- 172.21.15.7:0/2690325167 wait complete. 2024-01-27T02:34:17.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.230+0000 7f05c36a2700 1 Processor -- start 2024-01-27T02:34:17.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.230+0000 7f05c36a2700 1 -- start start 2024-01-27T02:34:17.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.231+0000 7f05c36a2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05bc0cbdf0 0x7f05bc1577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:17.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.231+0000 7f05c36a2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05bc1416c0 0x7f05bc0679b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:17.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.231+0000 7f05c36a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05bc141a90 0x7f05bc159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:17.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.231+0000 7f05c36a2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc1590a0 con 0x7f05bc141a90 2024-01-27T02:34:17.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.231+0000 7f05c36a2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc068040 con 0x7f05bc1416c0 2024-01-27T02:34:17.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.231+0000 7f05c36a2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc068320 con 0x7f05bc0cbdf0 2024-01-27T02:34:17.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.231+0000 7f05c1c3f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05bc141a90 0x7f05bc159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:17.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.231+0000 7f05c1c3f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05bc141a90 0x7f05bc159300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52792/0 (socket says 172.21.15.7:52792) 2024-01-27T02:34:17.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.231+0000 7f05c1c3f700 1 -- 172.21.15.7:0/2858739511 learned_addr learned my addr 172.21.15.7:0/2858739511 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.231+0000 7f05c0c3d700 1 --2- 172.21.15.7:0/2858739511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05bc1416c0 0x7f05bc0679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.231+0000 7f05c143e700 1 --2- 172.21.15.7:0/2858739511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05bc0cbdf0 0x7f05bc1577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.232+0000 7f05c1c3f700 1 -- 172.21.15.7:0/2858739511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05bc0cbdf0 msgr2=0x7f05bc1577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.232+0000 7f05c1c3f700 1 --2- 172.21.15.7:0/2858739511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05bc0cbdf0 0x7f05bc1577e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.232+0000 7f05c1c3f700 1 -- 172.21.15.7:0/2858739511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05bc1416c0 msgr2=0x7f05bc0679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.232+0000 7f05c1c3f700 1 --2- 172.21.15.7:0/2858739511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05bc1416c0 0x7f05bc0679b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.232+0000 7f05c1c3f700 1 -- 172.21.15.7:0/2858739511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b8009420 con 0x7f05bc141a90 2024-01-27T02:34:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.232+0000 7f05c0c3d700 1 --2- 172.21.15.7:0/2858739511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05bc1416c0 0x7f05bc0679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:34:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.232+0000 7f05c143e700 1 --2- 172.21.15.7:0/2858739511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05bc0cbdf0 0x7f05bc1577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:34:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.232+0000 7f05c1c3f700 1 --2- 172.21.15.7:0/2858739511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05bc141a90 0x7f05bc159300 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009df0 tx=0x7f05b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.232+0000 7f05ad7fa700 1 -- 172.21.15.7:0/2858739511 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b400cb50 con 0x7f05bc141a90 2024-01-27T02:34:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.232+0000 7f05ad7fa700 1 -- 172.21.15.7:0/2858739511 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b400f040 con 0x7f05bc141a90 2024-01-27T02:34:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.232+0000 7f05c36a2700 1 -- 172.21.15.7:0/2858739511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05bc068600 con 0x7f05bc141a90 2024-01-27T02:34:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.232+0000 7f05ad7fa700 1 -- 172.21.15.7:0/2858739511 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b4014480 con 0x7f05bc141a90 2024-01-27T02:34:17.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.232+0000 7f05c36a2700 1 -- 172.21.15.7:0/2858739511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05bc068bc0 con 0x7f05bc141a90 2024-01-27T02:34:17.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.233+0000 7f05c36a2700 1 -- 172.21.15.7:0/2858739511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05bc068d50 con 0x7f05bc141a90 2024-01-27T02:34:17.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.233+0000 7f05ad7fa700 1 -- 172.21.15.7:0/2858739511 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f05b4014830 con 0x7f05bc141a90 2024-01-27T02:34:17.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.234+0000 7f05ad7fa700 1 --2- 172.21.15.7:0/2858739511 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f05a0066f30 0x7f05a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:17.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.234+0000 7f05ad7fa700 1 -- 172.21.15.7:0/2858739511 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f05b4085230 con 0x7f05bc141a90 2024-01-27T02:34:17.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.234+0000 7f05c143e700 1 --2- 172.21.15.7:0/2858739511 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f05a0066f30 0x7f05a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:17.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.235+0000 7f05c143e700 1 --2- 172.21.15.7:0/2858739511 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f05a0066f30 0x7f05a00693e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f05b0001810 tx=0x7f05b0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.239+0000 7f05ad7fa700 1 -- 172.21.15.7:0/2858739511 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05b40524a0 con 0x7f05bc141a90 2024-01-27T02:34:17.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:17.497+0000 7f05c36a2700 1 -- 172.21.15.7:0/2858739511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f05bc072060 con 0x7f05bc141a90 2024-01-27T02:34:18.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.074+0000 7f05ad7fa700 1 -- 172.21.15.7:0/2858739511 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f05b4055ac0 con 0x7f05bc141a90 2024-01-27T02:34:18.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.076+0000 7f05c36a2700 1 -- 172.21.15.7:0/2858739511 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f05a0066f30 msgr2=0x7f05a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:18.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.076+0000 7f05c36a2700 1 --2- 172.21.15.7:0/2858739511 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f05a0066f30 0x7f05a00693e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f05b0001810 tx=0x7f05b0009040 comp rx=0 tx=0).stop 2024-01-27T02:34:18.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.076+0000 7f05c36a2700 1 -- 172.21.15.7:0/2858739511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05bc141a90 msgr2=0x7f05bc159300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:18.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.076+0000 7f05c36a2700 1 --2- 172.21.15.7:0/2858739511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05bc141a90 0x7f05bc159300 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009df0 tx=0x7f05b400b040 comp rx=0 tx=0).stop 2024-01-27T02:34:18.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.077+0000 7f05c36a2700 1 -- 172.21.15.7:0/2858739511 shutdown_connections 2024-01-27T02:34:18.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.077+0000 7f05c36a2700 1 --2- 172.21.15.7:0/2858739511 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f05a0066f30 0x7f05a00693e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:18.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.077+0000 7f05c36a2700 1 --2- 172.21.15.7:0/2858739511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05bc141a90 0x7f05bc159300 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:18.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.077+0000 7f05c36a2700 1 --2- 172.21.15.7:0/2858739511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05bc0cbdf0 0x7f05bc1577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:18.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.077+0000 7f05c36a2700 1 --2- 172.21.15.7:0/2858739511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05bc1416c0 0x7f05bc0679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:18.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.077+0000 7f05c36a2700 1 -- 172.21.15.7:0/2858739511 >> 172.21.15.7:0/2858739511 conn(0x7f05bc063580 msgr2=0x7f05bc0b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:18.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.077+0000 7f05c36a2700 1 -- 172.21.15.7:0/2858739511 shutdown_connections 2024-01-27T02:34:18.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.077+0000 7f05c36a2700 1 -- 172.21.15.7:0/2858739511 wait complete. 2024-01-27T02:34:18.094 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:34:18.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.228+0000 7fbb2fc91700 1 Processor -- start 2024-01-27T02:34:18.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.229+0000 7fbb2fc91700 1 -- start start 2024-01-27T02:34:18.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.230+0000 7fbb2fc91700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb280c4cc0 0x7fbb280cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:18.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.230+0000 7fbb2fc91700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb280ce5e0 con 0x7fbb280ca540 2024-01-27T02:34:18.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.230+0000 7fbb2fc91700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb280cd9d0 con 0x7fbb2815a100 2024-01-27T02:34:18.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.230+0000 7fbb2fc91700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb280cdcb0 con 0x7fbb280c4cc0 2024-01-27T02:34:18.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.230+0000 7fbb2d22c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb280c4cc0 0x7fbb280cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:18.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.230+0000 7fbb2e22e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbb280ca540 0x7fbb28163d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42188/0 (socket says 172.21.15.7:42188) 2024-01-27T02:34:18.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.230+0000 7fbb2e22e700 1 -- 172.21.15.7:0/847196848 learned_addr learned my addr 172.21.15.7:0/847196848 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:34:18.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.230+0000 7fbb2d22c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb280c4cc0 0x7fbb280cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52802/0 (socket says 172.21.15.7:52802) 2024-01-27T02:34:18.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.230+0000 7fbb2d22c700 1 -- 172.21.15.7:0/847196848 >> v1:172.21.15.188:6789/0 conn(0x7fbb2815a100 legacy=0x7fbb280c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:18.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.230+0000 7fbb2d22c700 1 -- 172.21.15.7:0/847196848 >> v1:172.21.15.7:6789/0 conn(0x7fbb280ca540 legacy=0x7fbb28163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:18.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.230+0000 7fbb2d22c700 1 -- 172.21.15.7:0/847196848 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb280cdf90 con 0x7fbb280c4cc0 2024-01-27T02:34:18.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.230+0000 7fbb2ca2b700 1 -- 172.21.15.7:0/847196848 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2737392713 0 0) 0x7fbb280ce5e0 con 0x7fbb280ca540 2024-01-27T02:34:18.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.230+0000 7fbb2d22c700 1 --2- 172.21.15.7:0/847196848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb280c4cc0 0x7fbb280cd2f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbb180098e0 tx=0x7fbb180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94c16bcdfa50b13 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:18.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.230+0000 7fbb2ca2b700 1 -- 172.21.15.7:0/847196848 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18009e80 con 0x7fbb280c4cc0 2024-01-27T02:34:18.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.230+0000 7fbb2ca2b700 1 -- 172.21.15.7:0/847196848 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1800b820 con 0x7fbb280c4cc0 2024-01-27T02:34:18.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.231+0000 7fbb2ca2b700 1 -- 172.21.15.7:0/847196848 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1800bba0 con 0x7fbb280c4cc0 2024-01-27T02:34:18.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.231+0000 7fbb2fc91700 1 -- 172.21.15.7:0/847196848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb280c4cc0 msgr2=0x7fbb280cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:18.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.231+0000 7fbb2fc91700 1 --2- 172.21.15.7:0/847196848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb280c4cc0 0x7fbb280cd2f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbb180098e0 tx=0x7fbb180091a0 comp rx=0 tx=0).stop 2024-01-27T02:34:18.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.231+0000 7fbb2fc91700 1 -- 172.21.15.7:0/847196848 shutdown_connections 2024-01-27T02:34:18.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.231+0000 7fbb2fc91700 1 --2- 172.21.15.7:0/847196848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb280c4cc0 0x7fbb280cd2f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:18.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.231+0000 7fbb2fc91700 1 -- 172.21.15.7:0/847196848 >> 172.21.15.7:0/847196848 conn(0x7fbb28098020 msgr2=0x7fbb280711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:18.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.231+0000 7fbb2fc91700 1 -- 172.21.15.7:0/847196848 shutdown_connections 2024-01-27T02:34:18.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.231+0000 7fbb2fc91700 1 -- 172.21.15.7:0/847196848 wait complete. 2024-01-27T02:34:18.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.232+0000 7fbb2fc91700 1 Processor -- start 2024-01-27T02:34:18.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.232+0000 7fbb2fc91700 1 -- start start 2024-01-27T02:34:18.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.233+0000 7fbb2fc91700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb280c4cc0 0x7fbb280b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:18.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.233+0000 7fbb2fc91700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb280ca540 0x7fbb280688a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:18.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.233+0000 7fbb2fc91700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb2815a100 0x7fbb281519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:18.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.233+0000 7fbb2fc91700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2815ca40 con 0x7fbb2815a100 2024-01-27T02:34:18.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.233+0000 7fbb2fc91700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb280b4920 con 0x7fbb280ca540 2024-01-27T02:34:18.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.233+0000 7fbb2fc91700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2815bb50 con 0x7fbb280c4cc0 2024-01-27T02:34:18.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.233+0000 7fbb2e22e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb2815a100 0x7fbb281519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:18.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.233+0000 7fbb2e22e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb2815a100 0x7fbb281519b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52812/0 (socket says 172.21.15.7:52812) 2024-01-27T02:34:18.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.233+0000 7fbb2e22e700 1 -- 172.21.15.7:0/3307234927 learned_addr learned my addr 172.21.15.7:0/3307234927 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:18.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.233+0000 7fbb2d22c700 1 --2- 172.21.15.7:0/3307234927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb280ca540 0x7fbb280688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:18.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.233+0000 7fbb2da2d700 1 --2- 172.21.15.7:0/3307234927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb280c4cc0 0x7fbb280b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:18.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.234+0000 7fbb2e22e700 1 -- 172.21.15.7:0/3307234927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb280c4cc0 msgr2=0x7fbb280b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:18.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.234+0000 7fbb2e22e700 1 --2- 172.21.15.7:0/3307234927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb280c4cc0 0x7fbb280b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:18.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.234+0000 7fbb2e22e700 1 -- 172.21.15.7:0/3307234927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb280ca540 msgr2=0x7fbb280688a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:18.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.234+0000 7fbb2e22e700 1 --2- 172.21.15.7:0/3307234927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb280ca540 0x7fbb280688a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:18.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.234+0000 7fbb2e22e700 1 -- 172.21.15.7:0/3307234927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb18009420 con 0x7fbb2815a100 2024-01-27T02:34:18.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.234+0000 7fbb2da2d700 1 --2- 172.21.15.7:0/3307234927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb280c4cc0 0x7fbb280b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:18.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.234+0000 7fbb2e22e700 1 --2- 172.21.15.7:0/3307234927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb2815a100 0x7fbb281519b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000cc90 tx=0x7fbb20001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:18.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.234+0000 7fbb1dffb700 1 -- 172.21.15.7:0/3307234927 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2000a960 con 0x7fbb2815a100 2024-01-27T02:34:18.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.234+0000 7fbb1dffb700 1 -- 172.21.15.7:0/3307234927 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2000aac0 con 0x7fbb2815a100 2024-01-27T02:34:18.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.234+0000 7fbb2fc91700 1 -- 172.21.15.7:0/3307234927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb2815be30 con 0x7fbb2815a100 2024-01-27T02:34:18.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.234+0000 7fbb1dffb700 1 -- 172.21.15.7:0/3307234927 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2000ae80 con 0x7fbb2815a100 2024-01-27T02:34:18.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.234+0000 7fbb2fc91700 1 -- 172.21.15.7:0/3307234927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb2815c3f0 con 0x7fbb2815a100 2024-01-27T02:34:18.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.235+0000 7fbb2fc91700 1 -- 172.21.15.7:0/3307234927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb280c6870 con 0x7fbb2815a100 2024-01-27T02:34:18.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.236+0000 7fbb1dffb700 1 -- 172.21.15.7:0/3307234927 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbb20007300 con 0x7fbb2815a100 2024-01-27T02:34:18.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.236+0000 7fbb1dffb700 1 --2- 172.21.15.7:0/3307234927 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb0c066f30 0x7fbb0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:18.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.236+0000 7fbb1dffb700 1 -- 172.21.15.7:0/3307234927 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbb200849a0 con 0x7fbb2815a100 2024-01-27T02:34:18.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.236+0000 7fbb2da2d700 1 --2- 172.21.15.7:0/3307234927 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb0c066f30 0x7fbb0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:18.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.237+0000 7fbb2da2d700 1 --2- 172.21.15.7:0/3307234927 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb0c066f30 0x7fbb0c0693e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbb24006fd0 tx=0x7fbb24008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:18.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.241+0000 7fbb1dffb700 1 -- 172.21.15.7:0/3307234927 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb20051c90 con 0x7fbb2815a100 2024-01-27T02:34:18.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.489+0000 7fbb2fc91700 1 -- 172.21.15.7:0/3307234927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb2804c710 con 0x7fbb2815a100 2024-01-27T02:34:18.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.490+0000 7fbb1dffb700 1 -- 172.21.15.7:0/3307234927 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4597 (secure 0 0 0) 0x7fbb200552b0 con 0x7fbb2815a100 2024-01-27T02:34:18.491 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:34:18.491 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5114,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/2839704447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":2839704447},{"type":"v1","addr":"172.21.15.188:6837","nonce":2839704447}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5117,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3964291855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3964291855},{"type":"v1","addr":"172.21.15.7:6834","nonce":3964291855}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/3332325272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3332325272},{"type":"v1","addr":"172.21.15.188:6836","nonce":3332325272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:34:14.471429+0000","modified":"2024-01-27T02:34:18.068858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1399055016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1399055016},{"type":"v1","addr":"172.21.15.7:6835","nonce":1399055016}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2024-01-27T02:34:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.492+0000 7fbb2fc91700 1 -- 172.21.15.7:0/3307234927 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb0c066f30 msgr2=0x7fbb0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.492+0000 7fbb2fc91700 1 --2- 172.21.15.7:0/3307234927 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb0c066f30 0x7fbb0c0693e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbb24006fd0 tx=0x7fbb24008040 comp rx=0 tx=0).stop 2024-01-27T02:34:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.492+0000 7fbb2fc91700 1 -- 172.21.15.7:0/3307234927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb2815a100 msgr2=0x7fbb281519b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.492+0000 7fbb2fc91700 1 --2- 172.21.15.7:0/3307234927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb2815a100 0x7fbb281519b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000cc90 tx=0x7fbb20001a80 comp rx=0 tx=0).stop 2024-01-27T02:34:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.492+0000 7fbb2fc91700 1 -- 172.21.15.7:0/3307234927 shutdown_connections 2024-01-27T02:34:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.492+0000 7fbb2fc91700 1 --2- 172.21.15.7:0/3307234927 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb0c066f30 0x7fbb0c0693e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.492+0000 7fbb2fc91700 1 --2- 172.21.15.7:0/3307234927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb2815a100 0x7fbb281519b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.492+0000 7fbb2fc91700 1 --2- 172.21.15.7:0/3307234927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb280c4cc0 0x7fbb280b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.492+0000 7fbb2fc91700 1 --2- 172.21.15.7:0/3307234927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb280ca540 0x7fbb280688a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.492+0000 7fbb2fc91700 1 -- 172.21.15.7:0/3307234927 >> 172.21.15.7:0/3307234927 conn(0x7fbb28098020 msgr2=0x7fbb2806cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.492+0000 7fbb2fc91700 1 -- 172.21.15.7:0/3307234927 shutdown_connections 2024-01-27T02:34:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.492+0000 7fbb2fc91700 1 -- 172.21.15.7:0/3307234927 wait complete. 2024-01-27T02:34:18.495 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2024-01-27T02:34:18.507 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:34:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.636+0000 7f9020163700 1 Processor -- start 2024-01-27T02:34:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.637+0000 7f9020163700 1 -- start start 2024-01-27T02:34:18.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.638+0000 7f9020163700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90180c3ae0 0x7f90180cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:18.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.638+0000 7f9020163700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9018158070 0x7f90180cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:18.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.638+0000 7f9020163700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90180ca020 con 0x7f9018157ca0 2024-01-27T02:34:18.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.638+0000 7f9020163700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90180c5210 con 0x7f9018158070 2024-01-27T02:34:18.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.638+0000 7f9020163700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90180c54f0 con 0x7f90180c3ae0 2024-01-27T02:34:18.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.638+0000 7f901e700700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9018158070 0x7f90180cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:18.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.638+0000 7f901deff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9018157ca0 0x7f90180c4a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42192/0 (socket says 172.21.15.7:42192) 2024-01-27T02:34:18.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.638+0000 7f901e700700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9018158070 0x7f90180cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52824/0 (socket says 172.21.15.7:52824) 2024-01-27T02:34:18.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.638+0000 7f901deff700 1 -- 172.21.15.7:0/3069185265 learned_addr learned my addr 172.21.15.7:0/3069185265 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:34:18.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.638+0000 7f901d6fe700 1 --2- 172.21.15.7:0/3069185265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90180c3ae0 0x7f90180cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:18.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.638+0000 7f901e700700 1 -- 172.21.15.7:0/3069185265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90180c3ae0 msgr2=0x7f90180cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:18.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.638+0000 7f901e700700 1 --2- 172.21.15.7:0/3069185265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90180c3ae0 0x7f90180cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:18.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.638+0000 7f901e700700 1 -- 172.21.15.7:0/3069185265 >> v1:172.21.15.7:6789/0 conn(0x7f9018157ca0 legacy=0x7f90180c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.638+0000 7f901e700700 1 -- 172.21.15.7:0/3069185265 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90180c57d0 con 0x7f9018158070 2024-01-27T02:34:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.638+0000 7f901cefd700 1 -- 172.21.15.7:0/3069185265 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982399303 0 0) 0x7f90180ca020 con 0x7f9018157ca0 2024-01-27T02:34:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.638+0000 7f901e700700 1 --2- 172.21.15.7:0/3069185265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9018158070 0x7f90180cda20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9010009e60 tx=0x7f9010009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff956ac8d6d99167 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.639+0000 7f901cefd700 1 -- 172.21.15.7:0/3069185265 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901000e070 con 0x7f9018158070 2024-01-27T02:34:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.639+0000 7f901cefd700 1 -- 172.21.15.7:0/3069185265 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901000bb60 con 0x7f9018158070 2024-01-27T02:34:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.639+0000 7f901cefd700 1 -- 172.21.15.7:0/3069185265 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010012430 con 0x7f9018158070 2024-01-27T02:34:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.639+0000 7f9020163700 1 -- 172.21.15.7:0/3069185265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9018158070 msgr2=0x7f90180cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.639+0000 7f9020163700 1 --2- 172.21.15.7:0/3069185265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9018158070 0x7f90180cda20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9010009e60 tx=0x7f9010009780 comp rx=0 tx=0).stop 2024-01-27T02:34:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.639+0000 7f9020163700 1 -- 172.21.15.7:0/3069185265 shutdown_connections 2024-01-27T02:34:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.639+0000 7f9020163700 1 --2- 172.21.15.7:0/3069185265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9018158070 0x7f90180cda20 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.639+0000 7f9020163700 1 --2- 172.21.15.7:0/3069185265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90180c3ae0 0x7f90180cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.639+0000 7f9020163700 1 -- 172.21.15.7:0/3069185265 >> 172.21.15.7:0/3069185265 conn(0x7f9018098020 msgr2=0x7f90180711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.639+0000 7f9020163700 1 -- 172.21.15.7:0/3069185265 shutdown_connections 2024-01-27T02:34:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.639+0000 7f9020163700 1 -- 172.21.15.7:0/3069185265 wait complete. 2024-01-27T02:34:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.640+0000 7f9020163700 1 Processor -- start 2024-01-27T02:34:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.641+0000 7f9020163700 1 -- start start 2024-01-27T02:34:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.641+0000 7f9020163700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90180c3ae0 0x7f901816a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:18.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.641+0000 7f9020163700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9018157ca0 0x7f90180c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:18.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.641+0000 7f9020163700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9018158070 0x7f90180c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:18.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.641+0000 7f9020163700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9018169c20 con 0x7f90180c3ae0 2024-01-27T02:34:18.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.641+0000 7f9020163700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901816ab50 con 0x7f9018157ca0 2024-01-27T02:34:18.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.641+0000 7f9020163700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90180c8920 con 0x7f9018158070 2024-01-27T02:34:18.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.641+0000 7f901deff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90180c3ae0 0x7f901816a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:18.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.641+0000 7f901deff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90180c3ae0 0x7f901816a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52826/0 (socket says 172.21.15.7:52826) 2024-01-27T02:34:18.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.641+0000 7f901deff700 1 -- 172.21.15.7:0/3800525231 learned_addr learned my addr 172.21.15.7:0/3800525231 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:18.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.642+0000 7f901d6fe700 1 --2- 172.21.15.7:0/3800525231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9018157ca0 0x7f90180c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.642+0000 7f901e700700 1 --2- 172.21.15.7:0/3800525231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9018158070 0x7f90180c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.642+0000 7f901deff700 1 -- 172.21.15.7:0/3800525231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9018158070 msgr2=0x7f90180c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.642+0000 7f901deff700 1 --2- 172.21.15.7:0/3800525231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9018158070 0x7f90180c8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.642+0000 7f901deff700 1 -- 172.21.15.7:0/3800525231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9018157ca0 msgr2=0x7f90180c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.642+0000 7f901deff700 1 --2- 172.21.15.7:0/3800525231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9018157ca0 0x7f90180c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.642+0000 7f901deff700 1 -- 172.21.15.7:0/3800525231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9010009a20 con 0x7f90180c3ae0 2024-01-27T02:34:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.642+0000 7f901e700700 1 --2- 172.21.15.7:0/3800525231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9018158070 0x7f90180c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.642+0000 7f901d6fe700 1 --2- 172.21.15.7:0/3800525231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9018157ca0 0x7f90180c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.642+0000 7f901deff700 1 --2- 172.21.15.7:0/3800525231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90180c3ae0 0x7f901816a610 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f901400c7e0 tx=0x7f9014007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.642+0000 7f900dffb700 1 -- 172.21.15.7:0/3800525231 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9014005b30 con 0x7f90180c3ae0 2024-01-27T02:34:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.642+0000 7f900dffb700 1 -- 172.21.15.7:0/3800525231 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9014005c90 con 0x7f90180c3ae0 2024-01-27T02:34:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.642+0000 7f900dffb700 1 -- 172.21.15.7:0/3800525231 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9014007300 con 0x7f90180c3ae0 2024-01-27T02:34:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.642+0000 7f9020163700 1 -- 172.21.15.7:0/3800525231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90180c8c00 con 0x7f90180c3ae0 2024-01-27T02:34:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.642+0000 7f9020163700 1 -- 172.21.15.7:0/3800525231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901816a0f0 con 0x7f90180c3ae0 2024-01-27T02:34:18.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.643+0000 7f9020163700 1 -- 172.21.15.7:0/3800525231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90180c8e40 con 0x7f90180c3ae0 2024-01-27T02:34:18.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.644+0000 7f900dffb700 1 -- 172.21.15.7:0/3800525231 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f901401b400 con 0x7f90180c3ae0 2024-01-27T02:34:18.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.644+0000 7f900dffb700 1 --2- 172.21.15.7:0/3800525231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8ffc066f30 0x7f8ffc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:18.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.644+0000 7f900dffb700 1 -- 172.21.15.7:0/3800525231 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f90140840f0 con 0x7f90180c3ae0 2024-01-27T02:34:18.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.645+0000 7f901d6fe700 1 --2- 172.21.15.7:0/3800525231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8ffc066f30 0x7f8ffc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:18.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.645+0000 7f901d6fe700 1 --2- 172.21.15.7:0/3800525231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8ffc066f30 0x7f8ffc0693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9018152440 tx=0x7f9008008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:18.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.649+0000 7f900dffb700 1 -- 172.21.15.7:0/3800525231 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90180c8e40 con 0x7f90180c3ae0 2024-01-27T02:34:18.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.850+0000 7f9020163700 1 -- 172.21.15.7:0/3800525231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90180c8e40 con 0x7f90180c3ae0 2024-01-27T02:34:18.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.851+0000 7f900dffb700 1 -- 172.21.15.7:0/3800525231 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f90180c8e40 con 0x7f90180c3ae0 2024-01-27T02:34:18.853 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:34:18.853 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":65,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:34:15.816418+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:34:12.775862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-27T02:34:13.445542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:34:18.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.854+0000 7f9020163700 1 -- 172.21.15.7:0/3800525231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8ffc066f30 msgr2=0x7f8ffc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:18.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.854+0000 7f9020163700 1 --2- 172.21.15.7:0/3800525231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8ffc066f30 0x7f8ffc0693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9018152440 tx=0x7f9008008040 comp rx=0 tx=0).stop 2024-01-27T02:34:18.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.854+0000 7f9020163700 1 -- 172.21.15.7:0/3800525231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90180c3ae0 msgr2=0x7f901816a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:18.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.854+0000 7f9020163700 1 --2- 172.21.15.7:0/3800525231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90180c3ae0 0x7f901816a610 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f901400c7e0 tx=0x7f9014007df0 comp rx=0 tx=0).stop 2024-01-27T02:34:18.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.854+0000 7f9020163700 1 -- 172.21.15.7:0/3800525231 shutdown_connections 2024-01-27T02:34:18.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.854+0000 7f9020163700 1 --2- 172.21.15.7:0/3800525231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8ffc066f30 0x7f8ffc0693e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:18.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.854+0000 7f9020163700 1 --2- 172.21.15.7:0/3800525231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9018158070 0x7f90180c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:18.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.854+0000 7f9020163700 1 --2- 172.21.15.7:0/3800525231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90180c3ae0 0x7f901816a610 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:18.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.854+0000 7f9020163700 1 --2- 172.21.15.7:0/3800525231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9018157ca0 0x7f90180c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:18.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.854+0000 7f9020163700 1 -- 172.21.15.7:0/3800525231 >> 172.21.15.7:0/3800525231 conn(0x7f9018098020 msgr2=0x7f90180cd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:18.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.854+0000 7f9020163700 1 -- 172.21.15.7:0/3800525231 shutdown_connections 2024-01-27T02:34:18.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:18.854+0000 7f9020163700 1 -- 172.21.15.7:0/3800525231 wait complete. 2024-01-27T02:34:18.870 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-27T02:34:18.870 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:34:19.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.000+0000 7f03594fd700 1 Processor -- start 2024-01-27T02:34:19.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.000+0000 7f03594fd700 1 -- start start 2024-01-27T02:34:19.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.001+0000 7f03594fd700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f03541409e0 0x7f03540b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:19.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.001+0000 7f03594fd700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03540c30c0 con 0x7f0354140610 2024-01-27T02:34:19.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.001+0000 7f03594fd700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f035406ccf0 con 0x7f035406e330 2024-01-27T02:34:19.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.001+0000 7f03594fd700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f035406ca10 con 0x7f03541409e0 2024-01-27T02:34:19.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.001+0000 7f0352ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0354140610 0x7f035406bef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42202/0 (socket says 172.21.15.7:42202) 2024-01-27T02:34:19.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.002+0000 7f0352ffd700 1 -- 172.21.15.7:0/1949563167 learned_addr learned my addr 172.21.15.7:0/1949563167 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:34:19.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.002+0000 7f03527fc700 1 --2- 172.21.15.7:0/1949563167 >> v2:172.21.15.188:3300/0 conn(0x7f03541409e0 0x7f03540b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:19.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.002+0000 7f0351ffb700 1 -- 172.21.15.7:0/1949563167 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4216346524 0 0) 0x7f03540c30c0 con 0x7f0354140610 2024-01-27T02:34:19.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.002+0000 7f0351ffb700 1 -- 172.21.15.7:0/1949563167 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0340003570 con 0x7f0354140610 2024-01-27T02:34:19.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.002+0000 7f03527fc700 1 -- 172.21.15.7:0/1949563167 >> v1:172.21.15.188:6789/0 conn(0x7f035406e330 legacy=0x7f03540c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:19.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.002+0000 7f0351ffb700 1 -- 172.21.15.7:0/1949563167 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1100909635 0 0) 0x7f0340003570 con 0x7f0354140610 2024-01-27T02:34:19.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.002+0000 7f03527fc700 1 -- 172.21.15.7:0/1949563167 >> v1:172.21.15.7:6789/0 conn(0x7f0354140610 legacy=0x7f035406bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:19.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.002+0000 7f03527fc700 1 -- 172.21.15.7:0/1949563167 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03540c2a70 con 0x7f03541409e0 2024-01-27T02:34:19.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.002+0000 7f03527fc700 1 --2- 172.21.15.7:0/1949563167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03541409e0 0x7f03540b09f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f033c0098e0 tx=0x7f033c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cf884c62bc864a9 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:19.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.002+0000 7f0351ffb700 1 -- 172.21.15.7:0/1949563167 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344001470 con 0x7f03541409e0 2024-01-27T02:34:19.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.003+0000 7f0351ffb700 1 -- 172.21.15.7:0/1949563167 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c011420 con 0x7f03541409e0 2024-01-27T02:34:19.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.003+0000 7f0351ffb700 1 -- 172.21.15.7:0/1949563167 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c0117c0 con 0x7f03541409e0 2024-01-27T02:34:19.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.003+0000 7f03594fd700 1 -- 172.21.15.7:0/1949563167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03541409e0 msgr2=0x7f03540b09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:19.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.003+0000 7f03594fd700 1 --2- 172.21.15.7:0/1949563167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03541409e0 0x7f03540b09f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f033c0098e0 tx=0x7f033c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:34:19.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.003+0000 7f03594fd700 1 -- 172.21.15.7:0/1949563167 shutdown_connections 2024-01-27T02:34:19.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.003+0000 7f03594fd700 1 --2- 172.21.15.7:0/1949563167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03541409e0 0x7f03540b09f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:19.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.003+0000 7f03594fd700 1 -- 172.21.15.7:0/1949563167 >> 172.21.15.7:0/1949563167 conn(0x7f03540cdc30 msgr2=0x7f03540ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:19.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.003+0000 7f03594fd700 1 -- 172.21.15.7:0/1949563167 shutdown_connections 2024-01-27T02:34:19.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.003+0000 7f03594fd700 1 -- 172.21.15.7:0/1949563167 wait complete. 2024-01-27T02:34:19.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.005+0000 7f03594fd700 1 Processor -- start 2024-01-27T02:34:19.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.005+0000 7f03594fd700 1 -- start start 2024-01-27T02:34:19.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.005+0000 7f03594fd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f035406e330 0x7f035415ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:19.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.005+0000 7f03594fd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0354140610 0x7f0354156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:19.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.005+0000 7f03594fd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03541409e0 0x7f03540b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:19.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.005+0000 7f03594fd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03540c8030 con 0x7f03541409e0 2024-01-27T02:34:19.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.005+0000 7f03594fd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035415bfb0 con 0x7f035406e330 2024-01-27T02:34:19.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.005+0000 7f03594fd700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0354158170 con 0x7f0354140610 2024-01-27T02:34:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.006+0000 7f03537fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03541409e0 0x7f03540b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.006+0000 7f03537fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03541409e0 0x7f03540b0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52838/0 (socket says 172.21.15.7:52838) 2024-01-27T02:34:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.006+0000 7f03537fe700 1 -- 172.21.15.7:0/1350650886 learned_addr learned my addr 172.21.15.7:0/1350650886 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.006+0000 7f0352ffd700 1 --2- 172.21.15.7:0/1350650886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f035406e330 0x7f035415ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.006+0000 7f03527fc700 1 --2- 172.21.15.7:0/1350650886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0354140610 0x7f0354156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.006+0000 7f03537fe700 1 -- 172.21.15.7:0/1350650886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0354140610 msgr2=0x7f0354156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.006+0000 7f03537fe700 1 --2- 172.21.15.7:0/1350650886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0354140610 0x7f0354156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.006+0000 7f03537fe700 1 -- 172.21.15.7:0/1350650886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f035406e330 msgr2=0x7f035415ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.006+0000 7f03537fe700 1 --2- 172.21.15.7:0/1350650886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f035406e330 0x7f035415ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.006+0000 7f03537fe700 1 -- 172.21.15.7:0/1350650886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c009420 con 0x7f03541409e0 2024-01-27T02:34:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.006+0000 7f03527fc700 1 --2- 172.21.15.7:0/1350650886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0354140610 0x7f0354156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.006+0000 7f0352ffd700 1 --2- 172.21.15.7:0/1350650886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f035406e330 0x7f035415ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.006+0000 7f03537fe700 1 --2- 172.21.15.7:0/1350650886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03541409e0 0x7f03540b0f70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f034c00c8e0 tx=0x7f034c0016c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:19.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.006+0000 7f034affd700 1 -- 172.21.15.7:0/1350650886 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c00ce80 con 0x7f03541409e0 2024-01-27T02:34:19.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.006+0000 7f034affd700 1 -- 172.21.15.7:0/1350650886 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034c00a7e0 con 0x7f03541409e0 2024-01-27T02:34:19.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.006+0000 7f03594fd700 1 -- 172.21.15.7:0/1350650886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f035415c1e0 con 0x7f03541409e0 2024-01-27T02:34:19.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.006+0000 7f034affd700 1 -- 172.21.15.7:0/1350650886 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c00ab80 con 0x7f03541409e0 2024-01-27T02:34:19.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.006+0000 7f03594fd700 1 -- 172.21.15.7:0/1350650886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0354157b20 con 0x7f03541409e0 2024-01-27T02:34:19.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.008+0000 7f03594fd700 1 -- 172.21.15.7:0/1350650886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03541582e0 con 0x7f03541409e0 2024-01-27T02:34:19.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.008+0000 7f034affd700 1 -- 172.21.15.7:0/1350650886 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f034c012420 con 0x7f03541409e0 2024-01-27T02:34:19.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.008+0000 7f034affd700 1 --2- 172.21.15.7:0/1350650886 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0334066f30 0x7f03340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:19.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.008+0000 7f034affd700 1 -- 172.21.15.7:0/1350650886 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f034c084880 con 0x7f03541409e0 2024-01-27T02:34:19.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.008+0000 7f0352ffd700 1 --2- 172.21.15.7:0/1350650886 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0334066f30 0x7f03340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:19.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.009+0000 7f0352ffd700 1 --2- 172.21.15.7:0/1350650886 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0334066f30 0x7f03340693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0344000c00 tx=0x7f0344001980 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:19.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.013+0000 7f034affd700 1 -- 172.21.15.7:0/1350650886 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f034c051af0 con 0x7f03541409e0 2024-01-27T02:34:19.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.210+0000 7f03594fd700 1 -- 172.21.15.7:0/1350650886 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f035404c710 con 0x7f0334066f30 2024-01-27T02:34:19.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.218+0000 7f034affd700 1 -- 172.21.15.7:0/1350650886 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285240 (secure 0 0 0) 0x7f035404c710 con 0x7f0334066f30 2024-01-27T02:34:19.224 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:34:19.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.222+0000 7f03594fd700 1 -- 172.21.15.7:0/1350650886 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0334066f30 msgr2=0x7f03340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:19.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.222+0000 7f03594fd700 1 --2- 172.21.15.7:0/1350650886 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0334066f30 0x7f03340693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0344000c00 tx=0x7f0344001980 comp rx=0 tx=0).stop 2024-01-27T02:34:19.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.222+0000 7f03594fd700 1 -- 172.21.15.7:0/1350650886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03541409e0 msgr2=0x7f03540b0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:19.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.222+0000 7f03594fd700 1 --2- 172.21.15.7:0/1350650886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03541409e0 0x7f03540b0f70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f034c00c8e0 tx=0x7f034c0016c0 comp rx=0 tx=0).stop 2024-01-27T02:34:19.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.222+0000 7f03594fd700 1 -- 172.21.15.7:0/1350650886 shutdown_connections 2024-01-27T02:34:19.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.222+0000 7f03594fd700 1 --2- 172.21.15.7:0/1350650886 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0334066f30 0x7f03340693e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:19.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.222+0000 7f03594fd700 1 --2- 172.21.15.7:0/1350650886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03541409e0 0x7f03540b0f70 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:19.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.222+0000 7f03594fd700 1 --2- 172.21.15.7:0/1350650886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f035406e330 0x7f035415ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:19.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.222+0000 7f03594fd700 1 --2- 172.21.15.7:0/1350650886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0354140610 0x7f0354156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:19.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.222+0000 7f03594fd700 1 -- 172.21.15.7:0/1350650886 >> 172.21.15.7:0/1350650886 conn(0x7f03540cdc30 msgr2=0x7f03540c3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:19.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.222+0000 7f03594fd700 1 -- 172.21.15.7:0/1350650886 shutdown_connections 2024-01-27T02:34:19.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.222+0000 7f03594fd700 1 -- 172.21.15.7:0/1350650886 wait complete. 2024-01-27T02:34:19.227 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:34:19.237 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-01-27T02:34:17.603844+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":102,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3709,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558790,"statfs":{"total":1600000000,"available":1596201420,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.570691"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.832172+0000","last_change":"2024-01-27T02:34:14.832008+0000","last_active":"2024-01-27T02:34:14.832172+0000","last_peered":"2024-01-27T02:34:14.832172+0000","last_clean":"2024-01-27T02:34:14.832172+0000","last_became_active":"2024-01-27T02:34:14.831727+0000","last_became_peered":"2024-01-27T02:34:14.831727+0000","last_unstale":"2024-01-27T02:34:14.832172+0000","last_undegraded":"2024-01-27T02:34:14.832172+0000","last_fullsized":"2024-01-27T02:34:14.832172+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:31:01.741897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.829074+0000","last_change":"2024-01-27T02:34:14.828797+0000","last_active":"2024-01-27T02:34:14.829074+0000","last_peered":"2024-01-27T02:34:14.829074+0000","last_clean":"2024-01-27T02:34:14.829074+0000","last_became_active":"2024-01-27T02:34:14.828497+0000","last_became_peered":"2024-01-27T02:34:14.828497+0000","last_unstale":"2024-01-27T02:34:14.829074+0000","last_undegraded":"2024-01-27T02:34:14.829074+0000","last_fullsized":"2024-01-27T02:34:14.829074+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:18:33.953475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.829016+0000","last_change":"2024-01-27T02:34:14.828841+0000","last_active":"2024-01-27T02:34:14.829016+0000","last_peered":"2024-01-27T02:34:14.829016+0000","last_clean":"2024-01-27T02:34:14.829016+0000","last_became_active":"2024-01-27T02:34:14.828110+0000","last_became_peered":"2024-01-27T02:34:14.828110+0000","last_unstale":"2024-01-27T02:34:14.829016+0000","last_undegraded":"2024-01-27T02:34:14.829016+0000","last_fullsized":"2024-01-27T02:34:14.829016+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:44:29.126424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.830679+0000","last_change":"2024-01-27T02:34:14.830679+0000","last_active":"2024-01-27T02:34:14.830679+0000","last_peered":"2024-01-27T02:34:14.830679+0000","last_clean":"2024-01-27T02:34:14.830679+0000","last_became_active":"2024-01-27T02:34:14.830445+0000","last_became_peered":"2024-01-27T02:34:14.830445+0000","last_unstale":"2024-01-27T02:34:14.830679+0000","last_undegraded":"2024-01-27T02:34:14.830679+0000","last_fullsized":"2024-01-27T02:34:14.830679+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:32:49.387479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.830531+0000","last_change":"2024-01-27T02:34:14.830215+0000","last_active":"2024-01-27T02:34:14.830531+0000","last_peered":"2024-01-27T02:34:14.830531+0000","last_clean":"2024-01-27T02:34:14.830531+0000","last_became_active":"2024-01-27T02:34:14.829958+0000","last_became_peered":"2024-01-27T02:34:14.829958+0000","last_unstale":"2024-01-27T02:34:14.830531+0000","last_undegraded":"2024-01-27T02:34:14.830531+0000","last_fullsized":"2024-01-27T02:34:14.830531+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:38:26.502127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.829701+0000","last_change":"2024-01-27T02:34:14.829701+0000","last_active":"2024-01-27T02:34:14.829701+0000","last_peered":"2024-01-27T02:34:14.829701+0000","last_clean":"2024-01-27T02:34:14.829701+0000","last_became_active":"2024-01-27T02:34:14.828723+0000","last_became_peered":"2024-01-27T02:34:14.828723+0000","last_unstale":"2024-01-27T02:34:14.829701+0000","last_undegraded":"2024-01-27T02:34:14.829701+0000","last_fullsized":"2024-01-27T02:34:14.829701+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:58:33.213596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.824463+0000","last_change":"2024-01-27T02:34:14.824379+0000","last_active":"2024-01-27T02:34:14.824463+0000","last_peered":"2024-01-27T02:34:14.824463+0000","last_clean":"2024-01-27T02:34:14.824463+0000","last_became_active":"2024-01-27T02:34:14.824122+0000","last_became_peered":"2024-01-27T02:34:14.824122+0000","last_unstale":"2024-01-27T02:34:14.824463+0000","last_undegraded":"2024-01-27T02:34:14.824463+0000","last_fullsized":"2024-01-27T02:34:14.824463+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:27:31.764056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.821615+0000","last_change":"2024-01-27T02:34:14.821539+0000","last_active":"2024-01-27T02:34:14.821615+0000","last_peered":"2024-01-27T02:34:14.821615+0000","last_clean":"2024-01-27T02:34:14.821615+0000","last_became_active":"2024-01-27T02:34:14.821291+0000","last_became_peered":"2024-01-27T02:34:14.821291+0000","last_unstale":"2024-01-27T02:34:14.821615+0000","last_undegraded":"2024-01-27T02:34:14.821615+0000","last_fullsized":"2024-01-27T02:34:14.821615+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:49:13.682800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.825084+0000","last_change":"2024-01-27T02:34:14.824740+0000","last_active":"2024-01-27T02:34:14.825084+0000","last_peered":"2024-01-27T02:34:14.825084+0000","last_clean":"2024-01-27T02:34:14.825084+0000","last_became_active":"2024-01-27T02:34:14.824470+0000","last_became_peered":"2024-01-27T02:34:14.824470+0000","last_unstale":"2024-01-27T02:34:14.825084+0000","last_undegraded":"2024-01-27T02:34:14.825084+0000","last_fullsized":"2024-01-27T02:34:14.825084+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:38:06.611401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.834061+0000","last_change":"2024-01-27T02:34:14.834061+0000","last_active":"2024-01-27T02:34:14.834061+0000","last_peered":"2024-01-27T02:34:14.834061+0000","last_clean":"2024-01-27T02:34:14.834061+0000","last_became_active":"2024-01-27T02:34:14.833423+0000","last_became_peered":"2024-01-27T02:34:14.833423+0000","last_unstale":"2024-01-27T02:34:14.834061+0000","last_undegraded":"2024-01-27T02:34:14.834061+0000","last_fullsized":"2024-01-27T02:34:14.834061+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:19:58.242716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.836298+0000","last_change":"2024-01-27T02:34:14.836298+0000","last_active":"2024-01-27T02:34:14.836298+0000","last_peered":"2024-01-27T02:34:14.836298+0000","last_clean":"2024-01-27T02:34:14.836298+0000","last_became_active":"2024-01-27T02:34:14.836049+0000","last_became_peered":"2024-01-27T02:34:14.836049+0000","last_unstale":"2024-01-27T02:34:14.836298+0000","last_undegraded":"2024-01-27T02:34:14.836298+0000","last_fullsized":"2024-01-27T02:34:14.836298+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:36:32.916928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.826633+0000","last_change":"2024-01-27T02:34:14.826633+0000","last_active":"2024-01-27T02:34:14.826633+0000","last_peered":"2024-01-27T02:34:14.826633+0000","last_clean":"2024-01-27T02:34:14.826633+0000","last_became_active":"2024-01-27T02:34:14.826298+0000","last_became_peered":"2024-01-27T02:34:14.826298+0000","last_unstale":"2024-01-27T02:34:14.826633+0000","last_undegraded":"2024-01-27T02:34:14.826633+0000","last_fullsized":"2024-01-27T02:34:14.826633+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:38:24.525603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.829594+0000","last_change":"2024-01-27T02:34:14.829594+0000","last_active":"2024-01-27T02:34:14.829594+0000","last_peered":"2024-01-27T02:34:14.829594+0000","last_clean":"2024-01-27T02:34:14.829594+0000","last_became_active":"2024-01-27T02:34:14.829267+0000","last_became_peered":"2024-01-27T02:34:14.829267+0000","last_unstale":"2024-01-27T02:34:14.829594+0000","last_undegraded":"2024-01-27T02:34:14.829594+0000","last_fullsized":"2024-01-27T02:34:14.829594+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:10:26.114865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.825559+0000","last_change":"2024-01-27T02:34:14.825216+0000","last_active":"2024-01-27T02:34:14.825559+0000","last_peered":"2024-01-27T02:34:14.825559+0000","last_clean":"2024-01-27T02:34:14.825559+0000","last_became_active":"2024-01-27T02:34:14.824976+0000","last_became_peered":"2024-01-27T02:34:14.824976+0000","last_unstale":"2024-01-27T02:34:14.825559+0000","last_undegraded":"2024-01-27T02:34:14.825559+0000","last_fullsized":"2024-01-27T02:34:14.825559+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:48:22.462890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.830810+0000","last_change":"2024-01-27T02:34:14.830698+0000","last_active":"2024-01-27T02:34:14.830810+0000","last_peered":"2024-01-27T02:34:14.830810+0000","last_clean":"2024-01-27T02:34:14.830810+0000","last_became_active":"2024-01-27T02:34:14.829984+0000","last_became_peered":"2024-01-27T02:34:14.829984+0000","last_unstale":"2024-01-27T02:34:14.830810+0000","last_undegraded":"2024-01-27T02:34:14.830810+0000","last_fullsized":"2024-01-27T02:34:14.830810+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:02:24.823736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.829364+0000","last_change":"2024-01-27T02:34:14.829364+0000","last_active":"2024-01-27T02:34:14.829364+0000","last_peered":"2024-01-27T02:34:14.829364+0000","last_clean":"2024-01-27T02:34:14.829364+0000","last_became_active":"2024-01-27T02:34:14.828289+0000","last_became_peered":"2024-01-27T02:34:14.828289+0000","last_unstale":"2024-01-27T02:34:14.829364+0000","last_undegraded":"2024-01-27T02:34:14.829364+0000","last_fullsized":"2024-01-27T02:34:14.829364+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:42:30.170573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.828985+0000","last_change":"2024-01-27T02:34:14.825531+0000","last_active":"2024-01-27T02:34:14.828985+0000","last_peered":"2024-01-27T02:34:14.828985+0000","last_clean":"2024-01-27T02:34:14.828985+0000","last_became_active":"2024-01-27T02:34:14.825038+0000","last_became_peered":"2024-01-27T02:34:14.825038+0000","last_unstale":"2024-01-27T02:34:14.828985+0000","last_undegraded":"2024-01-27T02:34:14.828985+0000","last_fullsized":"2024-01-27T02:34:14.828985+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:52:30.964086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.834577+0000","last_change":"2024-01-27T02:34:14.834507+0000","last_active":"2024-01-27T02:34:14.834577+0000","last_peered":"2024-01-27T02:34:14.834577+0000","last_clean":"2024-01-27T02:34:14.834577+0000","last_became_active":"2024-01-27T02:34:14.833556+0000","last_became_peered":"2024-01-27T02:34:14.833556+0000","last_unstale":"2024-01-27T02:34:14.834577+0000","last_undegraded":"2024-01-27T02:34:14.834577+0000","last_fullsized":"2024-01-27T02:34:14.834577+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:22:21.128760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.832472+0000","last_change":"2024-01-27T02:34:14.832371+0000","last_active":"2024-01-27T02:34:14.832472+0000","last_peered":"2024-01-27T02:34:14.832472+0000","last_clean":"2024-01-27T02:34:14.832472+0000","last_became_active":"2024-01-27T02:34:14.832110+0000","last_became_peered":"2024-01-27T02:34:14.832110+0000","last_unstale":"2024-01-27T02:34:14.832472+0000","last_undegraded":"2024-01-27T02:34:14.832472+0000","last_fullsized":"2024-01-27T02:34:14.832472+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:55:19.182457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.828440+0000","last_change":"2024-01-27T02:34:14.824527+0000","last_active":"2024-01-27T02:34:14.828440+0000","last_peered":"2024-01-27T02:34:14.828440+0000","last_clean":"2024-01-27T02:34:14.828440+0000","last_became_active":"2024-01-27T02:34:14.824033+0000","last_became_peered":"2024-01-27T02:34:14.824033+0000","last_unstale":"2024-01-27T02:34:14.828440+0000","last_undegraded":"2024-01-27T02:34:14.828440+0000","last_fullsized":"2024-01-27T02:34:14.828440+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:32:14.609662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.828292+0000","last_change":"2024-01-27T02:34:14.826918+0000","last_active":"2024-01-27T02:34:14.828292+0000","last_peered":"2024-01-27T02:34:14.828292+0000","last_clean":"2024-01-27T02:34:14.828292+0000","last_became_active":"2024-01-27T02:34:14.826773+0000","last_became_peered":"2024-01-27T02:34:14.826773+0000","last_unstale":"2024-01-27T02:34:14.828292+0000","last_undegraded":"2024-01-27T02:34:14.828292+0000","last_fullsized":"2024-01-27T02:34:14.828292+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:53:19.368477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.828516+0000","last_change":"2024-01-27T02:34:14.825368+0000","last_active":"2024-01-27T02:34:14.828516+0000","last_peered":"2024-01-27T02:34:14.828516+0000","last_clean":"2024-01-27T02:34:14.828516+0000","last_became_active":"2024-01-27T02:34:14.824445+0000","last_became_peered":"2024-01-27T02:34:14.824445+0000","last_unstale":"2024-01-27T02:34:14.828516+0000","last_undegraded":"2024-01-27T02:34:14.828516+0000","last_fullsized":"2024-01-27T02:34:14.828516+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:37:02.591325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.834865+0000","last_change":"2024-01-27T02:34:14.834803+0000","last_active":"2024-01-27T02:34:14.834865+0000","last_peered":"2024-01-27T02:34:14.834865+0000","last_clean":"2024-01-27T02:34:14.834865+0000","last_became_active":"2024-01-27T02:34:14.834270+0000","last_became_peered":"2024-01-27T02:34:14.834270+0000","last_unstale":"2024-01-27T02:34:14.834865+0000","last_undegraded":"2024-01-27T02:34:14.834865+0000","last_fullsized":"2024-01-27T02:34:14.834865+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:46:48.013304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.829022+0000","last_change":"2024-01-27T02:34:14.829022+0000","last_active":"2024-01-27T02:34:14.829022+0000","last_peered":"2024-01-27T02:34:14.829022+0000","last_clean":"2024-01-27T02:34:14.829022+0000","last_became_active":"2024-01-27T02:34:14.828758+0000","last_became_peered":"2024-01-27T02:34:14.828758+0000","last_unstale":"2024-01-27T02:34:14.829022+0000","last_undegraded":"2024-01-27T02:34:14.829022+0000","last_fullsized":"2024-01-27T02:34:14.829022+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:50:30.225349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.833187+0000","last_change":"2024-01-27T02:34:14.833012+0000","last_active":"2024-01-27T02:34:14.833187+0000","last_peered":"2024-01-27T02:34:14.833187+0000","last_clean":"2024-01-27T02:34:14.833187+0000","last_became_active":"2024-01-27T02:34:14.832758+0000","last_became_peered":"2024-01-27T02:34:14.832758+0000","last_unstale":"2024-01-27T02:34:14.833187+0000","last_undegraded":"2024-01-27T02:34:14.833187+0000","last_fullsized":"2024-01-27T02:34:14.833187+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:26:45.431640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.825531+0000","last_change":"2024-01-27T02:34:14.825531+0000","last_active":"2024-01-27T02:34:14.825531+0000","last_peered":"2024-01-27T02:34:14.825531+0000","last_clean":"2024-01-27T02:34:14.825531+0000","last_became_active":"2024-01-27T02:34:14.825229+0000","last_became_peered":"2024-01-27T02:34:14.825229+0000","last_unstale":"2024-01-27T02:34:14.825531+0000","last_undegraded":"2024-01-27T02:34:14.825531+0000","last_fullsized":"2024-01-27T02:34:14.825531+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:35:26.921373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.827457+0000","last_change":"2024-01-27T02:34:14.827457+0000","last_active":"2024-01-27T02:34:14.827457+0000","last_peered":"2024-01-27T02:34:14.827457+0000","last_clean":"2024-01-27T02:34:14.827457+0000","last_became_active":"2024-01-27T02:34:14.827196+0000","last_became_peered":"2024-01-27T02:34:14.827196+0000","last_unstale":"2024-01-27T02:34:14.827457+0000","last_undegraded":"2024-01-27T02:34:14.827457+0000","last_fullsized":"2024-01-27T02:34:14.827457+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:51:43.116654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.831118+0000","last_change":"2024-01-27T02:34:14.830990+0000","last_active":"2024-01-27T02:34:14.831118+0000","last_peered":"2024-01-27T02:34:14.831118+0000","last_clean":"2024-01-27T02:34:14.831118+0000","last_became_active":"2024-01-27T02:34:14.830615+0000","last_became_peered":"2024-01-27T02:34:14.830615+0000","last_unstale":"2024-01-27T02:34:14.831118+0000","last_undegraded":"2024-01-27T02:34:14.831118+0000","last_fullsized":"2024-01-27T02:34:14.831118+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:25:03.230683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.829030+0000","last_change":"2024-01-27T02:34:14.827218+0000","last_active":"2024-01-27T02:34:14.829030+0000","last_peered":"2024-01-27T02:34:14.829030+0000","last_clean":"2024-01-27T02:34:14.829030+0000","last_became_active":"2024-01-27T02:34:14.826428+0000","last_became_peered":"2024-01-27T02:34:14.826428+0000","last_unstale":"2024-01-27T02:34:14.829030+0000","last_undegraded":"2024-01-27T02:34:14.829030+0000","last_fullsized":"2024-01-27T02:34:14.829030+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:37:00.713869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.830509+0000","last_change":"2024-01-27T02:34:14.830415+0000","last_active":"2024-01-27T02:34:14.830509+0000","last_peered":"2024-01-27T02:34:14.830509+0000","last_clean":"2024-01-27T02:34:14.830509+0000","last_became_active":"2024-01-27T02:34:14.830139+0000","last_became_peered":"2024-01-27T02:34:14.830139+0000","last_unstale":"2024-01-27T02:34:14.830509+0000","last_undegraded":"2024-01-27T02:34:14.830509+0000","last_fullsized":"2024-01-27T02:34:14.830509+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:51:43.116654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.832994+0000","last_change":"2024-01-27T02:34:14.832559+0000","last_active":"2024-01-27T02:34:14.832994+0000","last_peered":"2024-01-27T02:34:14.832994+0000","last_clean":"2024-01-27T02:34:14.832994+0000","last_became_active":"2024-01-27T02:34:14.832310+0000","last_became_peered":"2024-01-27T02:34:14.832310+0000","last_unstale":"2024-01-27T02:34:14.832994+0000","last_undegraded":"2024-01-27T02:34:14.832994+0000","last_fullsized":"2024-01-27T02:34:14.832994+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:13:32.756245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.829991+0000","last_change":"2024-01-27T02:34:14.829991+0000","last_active":"2024-01-27T02:34:14.829991+0000","last_peered":"2024-01-27T02:34:14.829991+0000","last_clean":"2024-01-27T02:34:14.829991+0000","last_became_active":"2024-01-27T02:34:14.829668+0000","last_became_peered":"2024-01-27T02:34:14.829668+0000","last_unstale":"2024-01-27T02:34:14.829991+0000","last_undegraded":"2024-01-27T02:34:14.829991+0000","last_fullsized":"2024-01-27T02:34:14.829991+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:25:34.499048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.831396+0000","last_change":"2024-01-27T02:34:14.831396+0000","last_active":"2024-01-27T02:34:14.831396+0000","last_peered":"2024-01-27T02:34:14.831396+0000","last_clean":"2024-01-27T02:34:14.831396+0000","last_became_active":"2024-01-27T02:34:14.831159+0000","last_became_peered":"2024-01-27T02:34:14.831159+0000","last_unstale":"2024-01-27T02:34:14.831396+0000","last_undegraded":"2024-01-27T02:34:14.831396+0000","last_fullsized":"2024-01-27T02:34:14.831396+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:22:21.128760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"64'4","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.065038+0000","last_change":"2024-01-27T02:34:14.044359+0000","last_active":"2024-01-27T02:34:15.065038+0000","last_peered":"2024-01-27T02:34:15.065038+0000","last_clean":"2024-01-27T02:34:15.065038+0000","last_became_active":"2024-01-27T02:34:14.043760+0000","last_became_peered":"2024-01-27T02:34:14.043760+0000","last_unstale":"2024-01-27T02:34:15.065038+0000","last_undegraded":"2024-01-27T02:34:15.065038+0000","last_fullsized":"2024-01-27T02:34:15.065038+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:20:21.119379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.835494+0000","last_change":"2024-01-27T02:34:14.835494+0000","last_active":"2024-01-27T02:34:14.835494+0000","last_peered":"2024-01-27T02:34:14.835494+0000","last_clean":"2024-01-27T02:34:14.835494+0000","last_became_active":"2024-01-27T02:34:14.834923+0000","last_became_peered":"2024-01-27T02:34:14.834923+0000","last_unstale":"2024-01-27T02:34:14.835494+0000","last_undegraded":"2024-01-27T02:34:14.835494+0000","last_fullsized":"2024-01-27T02:34:14.835494+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:01:15.315520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.064859+0000","last_change":"2024-01-27T02:34:14.038365+0000","last_active":"2024-01-27T02:34:15.064859+0000","last_peered":"2024-01-27T02:34:15.064859+0000","last_clean":"2024-01-27T02:34:15.064859+0000","last_became_active":"2024-01-27T02:34:14.038029+0000","last_became_peered":"2024-01-27T02:34:14.038029+0000","last_unstale":"2024-01-27T02:34:15.064859+0000","last_undegraded":"2024-01-27T02:34:15.064859+0000","last_fullsized":"2024-01-27T02:34:15.064859+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:42:29.168670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.829956+0000","last_change":"2024-01-27T02:34:14.829956+0000","last_active":"2024-01-27T02:34:14.829956+0000","last_peered":"2024-01-27T02:34:14.829956+0000","last_clean":"2024-01-27T02:34:14.829956+0000","last_became_active":"2024-01-27T02:34:14.829118+0000","last_became_peered":"2024-01-27T02:34:14.829118+0000","last_unstale":"2024-01-27T02:34:14.829956+0000","last_undegraded":"2024-01-27T02:34:14.829956+0000","last_fullsized":"2024-01-27T02:34:14.829956+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:32:49.387479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"64'2","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.064157+0000","last_change":"2024-01-27T02:34:14.046453+0000","last_active":"2024-01-27T02:34:15.064157+0000","last_peered":"2024-01-27T02:34:15.064157+0000","last_clean":"2024-01-27T02:34:15.064157+0000","last_became_active":"2024-01-27T02:34:14.046051+0000","last_became_peered":"2024-01-27T02:34:14.046051+0000","last_unstale":"2024-01-27T02:34:15.064157+0000","last_undegraded":"2024-01-27T02:34:15.064157+0000","last_fullsized":"2024-01-27T02:34:15.064157+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:02:23.821834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.826929+0000","last_change":"2024-01-27T02:34:14.826929+0000","last_active":"2024-01-27T02:34:14.826929+0000","last_peered":"2024-01-27T02:34:14.826929+0000","last_clean":"2024-01-27T02:34:14.826929+0000","last_became_active":"2024-01-27T02:34:14.826656+0000","last_became_peered":"2024-01-27T02:34:14.826656+0000","last_unstale":"2024-01-27T02:34:14.826929+0000","last_undegraded":"2024-01-27T02:34:14.826929+0000","last_fullsized":"2024-01-27T02:34:14.826929+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:03:33.933530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"64'2","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.065620+0000","last_change":"2024-01-27T02:34:14.047224+0000","last_active":"2024-01-27T02:34:15.065620+0000","last_peered":"2024-01-27T02:34:15.065620+0000","last_clean":"2024-01-27T02:34:15.065620+0000","last_became_active":"2024-01-27T02:34:14.046857+0000","last_became_peered":"2024-01-27T02:34:14.046857+0000","last_unstale":"2024-01-27T02:34:15.065620+0000","last_undegraded":"2024-01-27T02:34:15.065620+0000","last_fullsized":"2024-01-27T02:34:15.065620+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:48:21.460988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.828459+0000","last_change":"2024-01-27T02:34:14.824673+0000","last_active":"2024-01-27T02:34:14.828459+0000","last_peered":"2024-01-27T02:34:14.828459+0000","last_clean":"2024-01-27T02:34:14.828459+0000","last_became_active":"2024-01-27T02:34:14.823879+0000","last_became_peered":"2024-01-27T02:34:14.823879+0000","last_unstale":"2024-01-27T02:34:14.828459+0000","last_undegraded":"2024-01-27T02:34:14.828459+0000","last_fullsized":"2024-01-27T02:34:14.828459+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:11:18.711696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.062136+0000","last_change":"2024-01-27T02:34:14.047567+0000","last_active":"2024-01-27T02:34:15.062136+0000","last_peered":"2024-01-27T02:34:15.062136+0000","last_clean":"2024-01-27T02:34:15.062136+0000","last_became_active":"2024-01-27T02:34:14.046851+0000","last_became_peered":"2024-01-27T02:34:14.046851+0000","last_unstale":"2024-01-27T02:34:15.062136+0000","last_undegraded":"2024-01-27T02:34:15.062136+0000","last_fullsized":"2024-01-27T02:34:15.062136+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:50:29.223446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.828479+0000","last_change":"2024-01-27T02:34:14.824727+0000","last_active":"2024-01-27T02:34:14.828479+0000","last_peered":"2024-01-27T02:34:14.828479+0000","last_clean":"2024-01-27T02:34:14.828479+0000","last_became_active":"2024-01-27T02:34:14.823493+0000","last_became_peered":"2024-01-27T02:34:14.823493+0000","last_unstale":"2024-01-27T02:34:14.828479+0000","last_undegraded":"2024-01-27T02:34:14.828479+0000","last_fullsized":"2024-01-27T02:34:14.828479+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:22:44.449010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"64'1","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.820536+0000","last_change":"2024-01-27T02:34:14.042534+0000","last_active":"2024-01-27T02:34:14.820536+0000","last_peered":"2024-01-27T02:34:14.820536+0000","last_clean":"2024-01-27T02:34:14.820536+0000","last_became_active":"2024-01-27T02:34:14.042118+0000","last_became_peered":"2024-01-27T02:34:14.042118+0000","last_unstale":"2024-01-27T02:34:14.820536+0000","last_undegraded":"2024-01-27T02:34:14.820536+0000","last_fullsized":"2024-01-27T02:34:14.820536+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:09:52.420262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.832702+0000","last_change":"2024-01-27T02:34:14.832702+0000","last_active":"2024-01-27T02:34:14.832702+0000","last_peered":"2024-01-27T02:34:14.832702+0000","last_clean":"2024-01-27T02:34:14.832702+0000","last_became_active":"2024-01-27T02:34:14.832399+0000","last_became_peered":"2024-01-27T02:34:14.832399+0000","last_unstale":"2024-01-27T02:34:14.832702+0000","last_undegraded":"2024-01-27T02:34:14.832702+0000","last_fullsized":"2024-01-27T02:34:14.832702+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:27:10.780868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.046845+0000","last_change":"2024-01-27T02:34:14.046758+0000","last_active":"2024-01-27T02:34:14.046845+0000","last_peered":"2024-01-27T02:34:14.046845+0000","last_clean":"2024-01-27T02:34:14.046845+0000","last_became_active":"2024-01-27T02:34:14.046536+0000","last_became_peered":"2024-01-27T02:34:14.046536+0000","last_unstale":"2024-01-27T02:34:14.046845+0000","last_undegraded":"2024-01-27T02:34:14.046845+0000","last_fullsized":"2024-01-27T02:34:14.046845+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:02:23.821834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.829048+0000","last_change":"2024-01-27T02:34:14.827374+0000","last_active":"2024-01-27T02:34:14.829048+0000","last_peered":"2024-01-27T02:34:14.829048+0000","last_clean":"2024-01-27T02:34:14.829048+0000","last_became_active":"2024-01-27T02:34:14.826793+0000","last_became_peered":"2024-01-27T02:34:14.826793+0000","last_unstale":"2024-01-27T02:34:14.829048+0000","last_undegraded":"2024-01-27T02:34:14.829048+0000","last_fullsized":"2024-01-27T02:34:14.829048+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:39:52.794096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.818831+0000","last_change":"2024-01-27T02:34:14.051391+0000","last_active":"2024-01-27T02:34:14.818831+0000","last_peered":"2024-01-27T02:34:14.818831+0000","last_clean":"2024-01-27T02:34:14.818831+0000","last_became_active":"2024-01-27T02:34:14.051237+0000","last_became_peered":"2024-01-27T02:34:14.051237+0000","last_unstale":"2024-01-27T02:34:14.818831+0000","last_undegraded":"2024-01-27T02:34:14.818831+0000","last_fullsized":"2024-01-27T02:34:14.818831+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:45:52.767110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.828275+0000","last_change":"2024-01-27T02:34:14.827958+0000","last_active":"2024-01-27T02:34:14.828275+0000","last_peered":"2024-01-27T02:34:14.828275+0000","last_clean":"2024-01-27T02:34:14.828275+0000","last_became_active":"2024-01-27T02:34:14.827687+0000","last_became_peered":"2024-01-27T02:34:14.827687+0000","last_unstale":"2024-01-27T02:34:14.828275+0000","last_undegraded":"2024-01-27T02:34:14.828275+0000","last_fullsized":"2024-01-27T02:34:14.828275+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:37:33.100835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.826582+0000","last_change":"2024-01-27T02:34:14.044833+0000","last_active":"2024-01-27T02:34:14.826582+0000","last_peered":"2024-01-27T02:34:14.826582+0000","last_clean":"2024-01-27T02:34:14.826582+0000","last_became_active":"2024-01-27T02:34:14.044611+0000","last_became_peered":"2024-01-27T02:34:14.044611+0000","last_unstale":"2024-01-27T02:34:14.826582+0000","last_undegraded":"2024-01-27T02:34:14.826582+0000","last_fullsized":"2024-01-27T02:34:14.826582+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:59:15.966990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.833237+0000","last_change":"2024-01-27T02:34:14.833190+0000","last_active":"2024-01-27T02:34:14.833237+0000","last_peered":"2024-01-27T02:34:14.833237+0000","last_clean":"2024-01-27T02:34:14.833237+0000","last_became_active":"2024-01-27T02:34:14.832551+0000","last_became_peered":"2024-01-27T02:34:14.832551+0000","last_unstale":"2024-01-27T02:34:14.833237+0000","last_undegraded":"2024-01-27T02:34:14.833237+0000","last_fullsized":"2024-01-27T02:34:14.833237+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:32:49.387479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.819961+0000","last_change":"2024-01-27T02:34:14.051488+0000","last_active":"2024-01-27T02:34:14.819961+0000","last_peered":"2024-01-27T02:34:14.819961+0000","last_clean":"2024-01-27T02:34:14.819961+0000","last_became_active":"2024-01-27T02:34:14.049919+0000","last_became_peered":"2024-01-27T02:34:14.049919+0000","last_unstale":"2024-01-27T02:34:14.819961+0000","last_undegraded":"2024-01-27T02:34:14.819961+0000","last_fullsized":"2024-01-27T02:34:14.819961+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:58:25.986996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.833953+0000","last_change":"2024-01-27T02:34:14.833953+0000","last_active":"2024-01-27T02:34:14.833953+0000","last_peered":"2024-01-27T02:34:14.833953+0000","last_clean":"2024-01-27T02:34:14.833953+0000","last_became_active":"2024-01-27T02:34:14.830746+0000","last_became_peered":"2024-01-27T02:34:14.830746+0000","last_unstale":"2024-01-27T02:34:14.833953+0000","last_undegraded":"2024-01-27T02:34:14.833953+0000","last_fullsized":"2024-01-27T02:34:14.833953+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:50:30.225349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"64'3","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.065709+0000","last_change":"2024-01-27T02:34:14.045955+0000","last_active":"2024-01-27T02:34:15.065709+0000","last_peered":"2024-01-27T02:34:15.065709+0000","last_clean":"2024-01-27T02:34:15.065709+0000","last_became_active":"2024-01-27T02:34:14.045731+0000","last_became_peered":"2024-01-27T02:34:14.045731+0000","last_unstale":"2024-01-27T02:34:15.065709+0000","last_undegraded":"2024-01-27T02:34:15.065709+0000","last_fullsized":"2024-01-27T02:34:15.065709+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:37:32.098933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.834729+0000","last_change":"2024-01-27T02:34:14.834657+0000","last_active":"2024-01-27T02:34:14.834729+0000","last_peered":"2024-01-27T02:34:14.834729+0000","last_clean":"2024-01-27T02:34:14.834729+0000","last_became_active":"2024-01-27T02:34:14.833967+0000","last_became_peered":"2024-01-27T02:34:14.833967+0000","last_unstale":"2024-01-27T02:34:14.834729+0000","last_undegraded":"2024-01-27T02:34:14.834729+0000","last_fullsized":"2024-01-27T02:34:14.834729+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:25:34.499048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"64'6","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.065665+0000","last_change":"2024-01-27T02:34:14.045550+0000","last_active":"2024-01-27T02:34:15.065665+0000","last_peered":"2024-01-27T02:34:15.065665+0000","last_clean":"2024-01-27T02:34:15.065665+0000","last_became_active":"2024-01-27T02:34:14.044663+0000","last_became_peered":"2024-01-27T02:34:14.044663+0000","last_unstale":"2024-01-27T02:34:15.065665+0000","last_undegraded":"2024-01-27T02:34:15.065665+0000","last_fullsized":"2024-01-27T02:34:15.065665+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:03:28.482115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.828833+0000","last_change":"2024-01-27T02:34:14.828833+0000","last_active":"2024-01-27T02:34:14.828833+0000","last_peered":"2024-01-27T02:34:14.828833+0000","last_clean":"2024-01-27T02:34:14.828833+0000","last_became_active":"2024-01-27T02:34:14.828554+0000","last_became_peered":"2024-01-27T02:34:14.828554+0000","last_unstale":"2024-01-27T02:34:14.828833+0000","last_undegraded":"2024-01-27T02:34:14.828833+0000","last_fullsized":"2024-01-27T02:34:14.828833+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:42:30.170573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"64'2","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.063612+0000","last_change":"2024-01-27T02:34:14.050502+0000","last_active":"2024-01-27T02:34:15.063612+0000","last_peered":"2024-01-27T02:34:15.063612+0000","last_clean":"2024-01-27T02:34:15.063612+0000","last_became_active":"2024-01-27T02:34:14.049898+0000","last_became_peered":"2024-01-27T02:34:14.049898+0000","last_unstale":"2024-01-27T02:34:15.063612+0000","last_undegraded":"2024-01-27T02:34:15.063612+0000","last_fullsized":"2024-01-27T02:34:15.063612+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:13:56.673742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.831742+0000","last_change":"2024-01-27T02:34:14.831645+0000","last_active":"2024-01-27T02:34:14.831742+0000","last_peered":"2024-01-27T02:34:14.831742+0000","last_clean":"2024-01-27T02:34:14.831742+0000","last_became_active":"2024-01-27T02:34:14.830976+0000","last_became_peered":"2024-01-27T02:34:14.830976+0000","last_unstale":"2024-01-27T02:34:14.831742+0000","last_undegraded":"2024-01-27T02:34:14.831742+0000","last_fullsized":"2024-01-27T02:34:14.831742+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:23.803292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.041437+0000","last_change":"2024-01-27T02:34:14.041362+0000","last_active":"2024-01-27T02:34:14.041437+0000","last_peered":"2024-01-27T02:34:14.041437+0000","last_clean":"2024-01-27T02:34:14.041437+0000","last_became_active":"2024-01-27T02:34:14.041131+0000","last_became_peered":"2024-01-27T02:34:14.041131+0000","last_unstale":"2024-01-27T02:34:14.041437+0000","last_undegraded":"2024-01-27T02:34:14.041437+0000","last_fullsized":"2024-01-27T02:34:14.041437+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:45:45.390108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.825582+0000","last_change":"2024-01-27T02:34:14.825448+0000","last_active":"2024-01-27T02:34:14.825582+0000","last_peered":"2024-01-27T02:34:14.825582+0000","last_clean":"2024-01-27T02:34:14.825582+0000","last_became_active":"2024-01-27T02:34:14.824986+0000","last_became_peered":"2024-01-27T02:34:14.824986+0000","last_unstale":"2024-01-27T02:34:14.825582+0000","last_undegraded":"2024-01-27T02:34:14.825582+0000","last_fullsized":"2024-01-27T02:34:14.825582+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:20:22.121281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.046317+0000","last_change":"2024-01-27T02:34:14.045205+0000","last_active":"2024-01-27T02:34:14.046317+0000","last_peered":"2024-01-27T02:34:14.046317+0000","last_clean":"2024-01-27T02:34:14.046317+0000","last_became_active":"2024-01-27T02:34:14.044332+0000","last_became_peered":"2024-01-27T02:34:14.044332+0000","last_unstale":"2024-01-27T02:34:14.046317+0000","last_undegraded":"2024-01-27T02:34:14.046317+0000","last_fullsized":"2024-01-27T02:34:14.046317+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:09:52.420262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.830480+0000","last_change":"2024-01-27T02:34:14.829392+0000","last_active":"2024-01-27T02:34:14.830480+0000","last_peered":"2024-01-27T02:34:14.830480+0000","last_clean":"2024-01-27T02:34:14.830480+0000","last_became_active":"2024-01-27T02:34:14.829129+0000","last_became_peered":"2024-01-27T02:34:14.829129+0000","last_unstale":"2024-01-27T02:34:14.830480+0000","last_undegraded":"2024-01-27T02:34:14.830480+0000","last_fullsized":"2024-01-27T02:34:14.830480+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:02:25.313682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"64'8","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.066477+0000","last_change":"2024-01-27T02:34:14.041138+0000","last_active":"2024-01-27T02:34:15.066477+0000","last_peered":"2024-01-27T02:34:15.066477+0000","last_clean":"2024-01-27T02:34:15.066477+0000","last_became_active":"2024-01-27T02:34:14.040706+0000","last_became_peered":"2024-01-27T02:34:14.040706+0000","last_unstale":"2024-01-27T02:34:15.066477+0000","last_undegraded":"2024-01-27T02:34:15.066477+0000","last_fullsized":"2024-01-27T02:34:15.066477+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:09:52.420262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.831912+0000","last_change":"2024-01-27T02:34:14.831839+0000","last_active":"2024-01-27T02:34:14.831912+0000","last_peered":"2024-01-27T02:34:14.831912+0000","last_clean":"2024-01-27T02:34:14.831912+0000","last_became_active":"2024-01-27T02:34:14.831410+0000","last_became_peered":"2024-01-27T02:34:14.831410+0000","last_unstale":"2024-01-27T02:34:14.831912+0000","last_undegraded":"2024-01-27T02:34:14.831912+0000","last_fullsized":"2024-01-27T02:34:14.831912+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:58:33.213596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.063001+0000","last_change":"2024-01-27T02:34:14.048893+0000","last_active":"2024-01-27T02:34:15.063001+0000","last_peered":"2024-01-27T02:34:15.063001+0000","last_clean":"2024-01-27T02:34:15.063001+0000","last_became_active":"2024-01-27T02:34:14.047262+0000","last_became_peered":"2024-01-27T02:34:14.047262+0000","last_unstale":"2024-01-27T02:34:15.063001+0000","last_undegraded":"2024-01-27T02:34:15.063001+0000","last_fullsized":"2024-01-27T02:34:15.063001+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:38:23.523701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.828699+0000","last_change":"2024-01-27T02:34:14.828699+0000","last_active":"2024-01-27T02:34:14.828699+0000","last_peered":"2024-01-27T02:34:14.828699+0000","last_clean":"2024-01-27T02:34:14.828699+0000","last_became_active":"2024-01-27T02:34:14.828432+0000","last_became_peered":"2024-01-27T02:34:14.828432+0000","last_unstale":"2024-01-27T02:34:14.828699+0000","last_undegraded":"2024-01-27T02:34:14.828699+0000","last_fullsized":"2024-01-27T02:34:14.828699+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:51:43.116654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"64'2","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.064694+0000","last_change":"2024-01-27T02:34:14.046312+0000","last_active":"2024-01-27T02:34:15.064694+0000","last_peered":"2024-01-27T02:34:15.064694+0000","last_clean":"2024-01-27T02:34:15.064694+0000","last_became_active":"2024-01-27T02:34:14.045709+0000","last_became_peered":"2024-01-27T02:34:14.045709+0000","last_unstale":"2024-01-27T02:34:15.064694+0000","last_undegraded":"2024-01-27T02:34:15.064694+0000","last_fullsized":"2024-01-27T02:34:15.064694+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:59:15.966990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.821061+0000","last_change":"2024-01-27T02:34:14.820829+0000","last_active":"2024-01-27T02:34:14.821061+0000","last_peered":"2024-01-27T02:34:14.821061+0000","last_clean":"2024-01-27T02:34:14.821061+0000","last_became_active":"2024-01-27T02:34:14.820496+0000","last_became_peered":"2024-01-27T02:34:14.820496+0000","last_unstale":"2024-01-27T02:34:14.821061+0000","last_undegraded":"2024-01-27T02:34:14.821061+0000","last_fullsized":"2024-01-27T02:34:14.821061+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:25:24.180251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"64'1","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.824476+0000","last_change":"2024-01-27T02:34:14.047081+0000","last_active":"2024-01-27T02:34:14.824476+0000","last_peered":"2024-01-27T02:34:14.824476+0000","last_clean":"2024-01-27T02:34:14.824476+0000","last_became_active":"2024-01-27T02:34:14.046575+0000","last_became_peered":"2024-01-27T02:34:14.046575+0000","last_unstale":"2024-01-27T02:34:14.824476+0000","last_undegraded":"2024-01-27T02:34:14.824476+0000","last_fullsized":"2024-01-27T02:34:14.824476+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:20:21.119379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.833339+0000","last_change":"2024-01-27T02:34:14.833295+0000","last_active":"2024-01-27T02:34:14.833339+0000","last_peered":"2024-01-27T02:34:14.833339+0000","last_clean":"2024-01-27T02:34:14.833339+0000","last_became_active":"2024-01-27T02:34:14.832860+0000","last_became_peered":"2024-01-27T02:34:14.832860+0000","last_unstale":"2024-01-27T02:34:14.833339+0000","last_undegraded":"2024-01-27T02:34:14.833339+0000","last_fullsized":"2024-01-27T02:34:14.833339+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:35:26.921373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.046424+0000","last_change":"2024-01-27T02:34:14.045691+0000","last_active":"2024-01-27T02:34:14.046424+0000","last_peered":"2024-01-27T02:34:14.046424+0000","last_clean":"2024-01-27T02:34:14.046424+0000","last_became_active":"2024-01-27T02:34:14.045396+0000","last_became_peered":"2024-01-27T02:34:14.045396+0000","last_unstale":"2024-01-27T02:34:14.046424+0000","last_undegraded":"2024-01-27T02:34:14.046424+0000","last_fullsized":"2024-01-27T02:34:14.046424+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:37:32.098933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.832207+0000","last_change":"2024-01-27T02:34:14.832207+0000","last_active":"2024-01-27T02:34:14.832207+0000","last_peered":"2024-01-27T02:34:14.832207+0000","last_clean":"2024-01-27T02:34:14.832207+0000","last_became_active":"2024-01-27T02:34:14.831949+0000","last_became_peered":"2024-01-27T02:34:14.831949+0000","last_unstale":"2024-01-27T02:34:14.832207+0000","last_undegraded":"2024-01-27T02:34:14.832207+0000","last_fullsized":"2024-01-27T02:34:14.832207+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:58:33.213596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"64'2","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.063302+0000","last_change":"2024-01-27T02:34:14.046458+0000","last_active":"2024-01-27T02:34:15.063302+0000","last_peered":"2024-01-27T02:34:15.063302+0000","last_clean":"2024-01-27T02:34:15.063302+0000","last_became_active":"2024-01-27T02:34:14.046165+0000","last_became_peered":"2024-01-27T02:34:14.046165+0000","last_unstale":"2024-01-27T02:34:15.063302+0000","last_undegraded":"2024-01-27T02:34:15.063302+0000","last_fullsized":"2024-01-27T02:34:15.063302+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:03:32.931628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.827960+0000","last_change":"2024-01-27T02:34:14.827960+0000","last_active":"2024-01-27T02:34:14.827960+0000","last_peered":"2024-01-27T02:34:14.827960+0000","last_clean":"2024-01-27T02:34:14.827960+0000","last_became_active":"2024-01-27T02:34:14.827141+0000","last_became_peered":"2024-01-27T02:34:14.827141+0000","last_unstale":"2024-01-27T02:34:14.827960+0000","last_undegraded":"2024-01-27T02:34:14.827960+0000","last_fullsized":"2024-01-27T02:34:14.827960+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:44:29.126424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.048236+0000","last_change":"2024-01-27T02:34:14.048043+0000","last_active":"2024-01-27T02:34:14.048236+0000","last_peered":"2024-01-27T02:34:14.048236+0000","last_clean":"2024-01-27T02:34:14.048236+0000","last_became_active":"2024-01-27T02:34:14.047691+0000","last_became_peered":"2024-01-27T02:34:14.047691+0000","last_unstale":"2024-01-27T02:34:14.048236+0000","last_undegraded":"2024-01-27T02:34:14.048236+0000","last_fullsized":"2024-01-27T02:34:14.048236+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:10.895520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.835543+0000","last_change":"2024-01-27T02:34:14.835543+0000","last_active":"2024-01-27T02:34:14.835543+0000","last_peered":"2024-01-27T02:34:14.835543+0000","last_clean":"2024-01-27T02:34:14.835543+0000","last_became_active":"2024-01-27T02:34:14.835281+0000","last_became_peered":"2024-01-27T02:34:14.835281+0000","last_unstale":"2024-01-27T02:34:14.835543+0000","last_undegraded":"2024-01-27T02:34:14.835543+0000","last_fullsized":"2024-01-27T02:34:14.835543+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:35:26.921373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.046139+0000","last_change":"2024-01-27T02:34:14.043434+0000","last_active":"2024-01-27T02:34:14.046139+0000","last_peered":"2024-01-27T02:34:14.046139+0000","last_clean":"2024-01-27T02:34:14.046139+0000","last_became_active":"2024-01-27T02:34:14.043174+0000","last_became_peered":"2024-01-27T02:34:14.043174+0000","last_unstale":"2024-01-27T02:34:14.046139+0000","last_undegraded":"2024-01-27T02:34:14.046139+0000","last_fullsized":"2024-01-27T02:34:14.046139+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:37:32.098933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.831862+0000","last_change":"2024-01-27T02:34:14.831862+0000","last_active":"2024-01-27T02:34:14.831862+0000","last_peered":"2024-01-27T02:34:14.831862+0000","last_clean":"2024-01-27T02:34:14.831862+0000","last_became_active":"2024-01-27T02:34:14.831444+0000","last_became_peered":"2024-01-27T02:34:14.831444+0000","last_unstale":"2024-01-27T02:34:14.831862+0000","last_undegraded":"2024-01-27T02:34:14.831862+0000","last_fullsized":"2024-01-27T02:34:14.831862+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:02:25.313682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"64'2","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.063617+0000","last_change":"2024-01-27T02:34:14.051497+0000","last_active":"2024-01-27T02:34:15.063617+0000","last_peered":"2024-01-27T02:34:15.063617+0000","last_clean":"2024-01-27T02:34:15.063617+0000","last_became_active":"2024-01-27T02:34:14.051241+0000","last_became_peered":"2024-01-27T02:34:14.051241+0000","last_unstale":"2024-01-27T02:34:15.063617+0000","last_undegraded":"2024-01-27T02:34:15.063617+0000","last_fullsized":"2024-01-27T02:34:15.063617+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:44:04.272285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.826511+0000","last_change":"2024-01-27T02:34:14.826511+0000","last_active":"2024-01-27T02:34:14.826511+0000","last_peered":"2024-01-27T02:34:14.826511+0000","last_clean":"2024-01-27T02:34:14.826511+0000","last_became_active":"2024-01-27T02:34:14.826082+0000","last_became_peered":"2024-01-27T02:34:14.826082+0000","last_unstale":"2024-01-27T02:34:14.826511+0000","last_undegraded":"2024-01-27T02:34:14.826511+0000","last_fullsized":"2024-01-27T02:34:14.826511+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:13:57.675645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.042629+0000","last_change":"2024-01-27T02:34:14.042366+0000","last_active":"2024-01-27T02:34:14.042629+0000","last_peered":"2024-01-27T02:34:14.042629+0000","last_clean":"2024-01-27T02:34:14.042629+0000","last_became_active":"2024-01-27T02:34:14.041847+0000","last_became_peered":"2024-01-27T02:34:14.041847+0000","last_unstale":"2024-01-27T02:34:14.042629+0000","last_undegraded":"2024-01-27T02:34:14.042629+0000","last_fullsized":"2024-01-27T02:34:14.042629+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:33:19.626854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.833677+0000","last_change":"2024-01-27T02:34:14.833677+0000","last_active":"2024-01-27T02:34:14.833677+0000","last_peered":"2024-01-27T02:34:14.833677+0000","last_clean":"2024-01-27T02:34:14.833677+0000","last_became_active":"2024-01-27T02:34:14.833026+0000","last_became_peered":"2024-01-27T02:34:14.833026+0000","last_unstale":"2024-01-27T02:34:14.833677+0000","last_undegraded":"2024-01-27T02:34:14.833677+0000","last_fullsized":"2024-01-27T02:34:14.833677+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:35:50.281851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.042591+0000","last_change":"2024-01-27T02:34:14.042532+0000","last_active":"2024-01-27T02:34:14.042591+0000","last_peered":"2024-01-27T02:34:14.042591+0000","last_clean":"2024-01-27T02:34:14.042591+0000","last_became_active":"2024-01-27T02:34:14.042302+0000","last_became_peered":"2024-01-27T02:34:14.042302+0000","last_unstale":"2024-01-27T02:34:14.042591+0000","last_undegraded":"2024-01-27T02:34:14.042591+0000","last_fullsized":"2024-01-27T02:34:14.042591+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:45:13.750457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.826738+0000","last_change":"2024-01-27T02:34:14.826738+0000","last_active":"2024-01-27T02:34:14.826738+0000","last_peered":"2024-01-27T02:34:14.826738+0000","last_clean":"2024-01-27T02:34:14.826738+0000","last_became_active":"2024-01-27T02:34:14.826236+0000","last_became_peered":"2024-01-27T02:34:14.826236+0000","last_unstale":"2024-01-27T02:34:14.826738+0000","last_undegraded":"2024-01-27T02:34:14.826738+0000","last_fullsized":"2024-01-27T02:34:14.826738+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:53:19.368477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.046158+0000","last_change":"2024-01-27T02:34:14.043678+0000","last_active":"2024-01-27T02:34:14.046158+0000","last_peered":"2024-01-27T02:34:14.046158+0000","last_clean":"2024-01-27T02:34:14.046158+0000","last_became_active":"2024-01-27T02:34:14.043451+0000","last_became_peered":"2024-01-27T02:34:14.043451+0000","last_unstale":"2024-01-27T02:34:14.046158+0000","last_undegraded":"2024-01-27T02:34:14.046158+0000","last_fullsized":"2024-01-27T02:34:14.046158+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:38:05.609498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.829004+0000","last_change":"2024-01-27T02:34:14.826838+0000","last_active":"2024-01-27T02:34:14.829004+0000","last_peered":"2024-01-27T02:34:14.829004+0000","last_clean":"2024-01-27T02:34:14.829004+0000","last_became_active":"2024-01-27T02:34:14.826571+0000","last_became_peered":"2024-01-27T02:34:14.826571+0000","last_unstale":"2024-01-27T02:34:14.829004+0000","last_undegraded":"2024-01-27T02:34:14.829004+0000","last_fullsized":"2024-01-27T02:34:14.829004+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:58:27.917764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.044312+0000","last_change":"2024-01-27T02:34:14.044223+0000","last_active":"2024-01-27T02:34:14.044312+0000","last_peered":"2024-01-27T02:34:14.044312+0000","last_clean":"2024-01-27T02:34:14.044312+0000","last_became_active":"2024-01-27T02:34:14.043938+0000","last_became_peered":"2024-01-27T02:34:14.043938+0000","last_unstale":"2024-01-27T02:34:14.044312+0000","last_undegraded":"2024-01-27T02:34:14.044312+0000","last_fullsized":"2024-01-27T02:34:14.044312+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:38:59.335007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.833150+0000","last_change":"2024-01-27T02:34:14.833090+0000","last_active":"2024-01-27T02:34:14.833150+0000","last_peered":"2024-01-27T02:34:14.833150+0000","last_clean":"2024-01-27T02:34:14.833150+0000","last_became_active":"2024-01-27T02:34:14.832225+0000","last_became_peered":"2024-01-27T02:34:14.832225+0000","last_unstale":"2024-01-27T02:34:14.833150+0000","last_undegraded":"2024-01-27T02:34:14.833150+0000","last_fullsized":"2024-01-27T02:34:14.833150+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:22:37.496411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.819436+0000","last_change":"2024-01-27T02:34:14.047656+0000","last_active":"2024-01-27T02:34:14.819436+0000","last_peered":"2024-01-27T02:34:14.819436+0000","last_clean":"2024-01-27T02:34:14.819436+0000","last_became_active":"2024-01-27T02:34:14.047384+0000","last_became_peered":"2024-01-27T02:34:14.047384+0000","last_unstale":"2024-01-27T02:34:14.819436+0000","last_undegraded":"2024-01-27T02:34:14.819436+0000","last_fullsized":"2024-01-27T02:34:14.819436+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:10:25.112962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.834694+0000","last_change":"2024-01-27T02:34:14.834694+0000","last_active":"2024-01-27T02:34:14.834694+0000","last_peered":"2024-01-27T02:34:14.834694+0000","last_clean":"2024-01-27T02:34:14.834694+0000","last_became_active":"2024-01-27T02:34:14.834220+0000","last_became_peered":"2024-01-27T02:34:14.834220+0000","last_unstale":"2024-01-27T02:34:14.834694+0000","last_undegraded":"2024-01-27T02:34:14.834694+0000","last_fullsized":"2024-01-27T02:34:14.834694+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:03:29.484017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.046467+0000","last_change":"2024-01-27T02:34:14.046346+0000","last_active":"2024-01-27T02:34:14.046467+0000","last_peered":"2024-01-27T02:34:14.046467+0000","last_clean":"2024-01-27T02:34:14.046467+0000","last_became_active":"2024-01-27T02:34:14.045803+0000","last_became_peered":"2024-01-27T02:34:14.045803+0000","last_unstale":"2024-01-27T02:34:14.046467+0000","last_undegraded":"2024-01-27T02:34:14.046467+0000","last_fullsized":"2024-01-27T02:34:14.046467+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:45:45.390108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.828497+0000","last_change":"2024-01-27T02:34:14.825239+0000","last_active":"2024-01-27T02:34:14.828497+0000","last_peered":"2024-01-27T02:34:14.828497+0000","last_clean":"2024-01-27T02:34:14.828497+0000","last_became_active":"2024-01-27T02:34:14.824940+0000","last_became_peered":"2024-01-27T02:34:14.824940+0000","last_unstale":"2024-01-27T02:34:14.828497+0000","last_undegraded":"2024-01-27T02:34:14.828497+0000","last_fullsized":"2024-01-27T02:34:14.828497+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:46:48.013304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.819108+0000","last_change":"2024-01-27T02:34:14.049093+0000","last_active":"2024-01-27T02:34:14.819108+0000","last_peered":"2024-01-27T02:34:14.819108+0000","last_clean":"2024-01-27T02:34:14.819108+0000","last_became_active":"2024-01-27T02:34:14.048624+0000","last_became_peered":"2024-01-27T02:34:14.048624+0000","last_unstale":"2024-01-27T02:34:14.819108+0000","last_undegraded":"2024-01-27T02:34:14.819108+0000","last_fullsized":"2024-01-27T02:34:14.819108+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:22.801389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.824286+0000","last_change":"2024-01-27T02:34:14.823216+0000","last_active":"2024-01-27T02:34:14.824286+0000","last_peered":"2024-01-27T02:34:14.824286+0000","last_clean":"2024-01-27T02:34:14.824286+0000","last_became_active":"2024-01-27T02:34:14.822948+0000","last_became_peered":"2024-01-27T02:34:14.822948+0000","last_unstale":"2024-01-27T02:34:14.824286+0000","last_undegraded":"2024-01-27T02:34:14.824286+0000","last_fullsized":"2024-01-27T02:34:14.824286+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:12:49.632475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.044937+0000","last_change":"2024-01-27T02:34:14.044095+0000","last_active":"2024-01-27T02:34:14.044937+0000","last_peered":"2024-01-27T02:34:14.044937+0000","last_clean":"2024-01-27T02:34:14.044937+0000","last_became_active":"2024-01-27T02:34:14.043773+0000","last_became_peered":"2024-01-27T02:34:14.043773+0000","last_unstale":"2024-01-27T02:34:14.044937+0000","last_undegraded":"2024-01-27T02:34:14.044937+0000","last_fullsized":"2024-01-27T02:34:14.044937+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:44:04.272285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":199,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.823745+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:34:14.823745+0000","last_peered":"2024-01-27T02:34:14.823745+0000","last_clean":"2024-01-27T02:34:14.823745+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:34:14.823745+0000","last_undegraded":"2024-01-27T02:34:14.823745+0000","last_fullsized":"2024-01-27T02:34:14.823745+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":894,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194418,"statfs":{"total":200000000,"available":199084390,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.41899999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58499999999999996,"15min":0.58799999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.67300000000000004}]},{"osd":1,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.624},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55100000000000005,"15min":0.55000000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.57499999999999996}]},{"osd":2,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58099999999999996,"15min":0.57999999999999996},"min":{"1min":0.45600000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58499999999999996,"15min":0.58099999999999996},"min":{"1min":0.44900000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.71499999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.55200000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.48199999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.69199999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.76300000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.50800000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.496,"15min":0.49399999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.49399999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.32100000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.63900000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80300000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.752,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.439}]}]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":317,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194995,"statfs":{"total":200000000,"available":199674918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.51400000000000001,"15min":0.497},"min":{"1min":0.41899999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.52200000000000002,"15min":0.50800000000000001},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.53200000000000003}]},{"osd":1,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.51000000000000001,"15min":0.49399999999999999},"min":{"1min":0.48799999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.54800000000000004,"15min":0.53600000000000003},"min":{"1min":0.39900000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.57399999999999995}]},{"osd":2,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.41599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.441,"5min":0.438,"15min":0.438},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.69299999999999995}]},{"osd":3,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.54700000000000004,"15min":0.53900000000000003},"min":{"1min":0.435,"5min":0.376,"15min":0.376},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.436,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.64800000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.39500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.60699999999999998,"5min":0.748,"15min":0.748},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55600000000000005,"15min":0.56200000000000006},"min":{"1min":0.379,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70499999999999996,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.627}]},{"osd":5,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.67900000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.60599999999999998,"15min":0.61499999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.72299999999999998}]},{"osd":7,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.61899999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.754,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.746},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69999999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.73399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":317,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194994,"statfs":{"total":200000000,"available":199674828,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.371,"15min":0.36799999999999999},"min":{"1min":0.28199999999999997,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.30199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.42099999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.36199999999999999,"15min":0.35299999999999998},"min":{"1min":0.32200000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.42299999999999999,"15min":0.42099999999999999},"min":{"1min":0.32200000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.60099999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.45000000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.46400000000000002,"15min":0.46000000000000002},"min":{"1min":0.36299999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69299999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.38500000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.621,"15min":0.621},"min":{"1min":0.46300000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.60099999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.46200000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66200000000000003,"15min":0.66600000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.629}]},{"osd":6,"last update":"Sat Jan 27 02:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66600000000000004,"15min":0.66400000000000003},"min":{"1min":0.52500000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.64500000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.505,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68000000000000005,"15min":0.67700000000000005},"min":{"1min":0.58299999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.67700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":896,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194416,"statfs":{"total":200000000,"available":199082460,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.27600000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.54600000000000004,"5min":0.626,"15min":0.626},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.40899999999999997,"15min":0.41299999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.54400000000000004,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.42299999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.41799999999999998,"15min":0.41599999999999998},"min":{"1min":0.36099999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.34699999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63100000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.36799999999999999}]},{"osd":3,"last update":"Sat Jan 27 02:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.42399999999999999,"15min":0.42999999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.50700000000000001,"5min":0.625,"15min":0.625},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.46400000000000002,"15min":0.46800000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.60199999999999998,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.45700000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.81699999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.65300000000000002,"15min":0.65700000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.76300000000000001}]},{"osd":5,"last update":"Sat Jan 27 02:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.747,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67400000000000004,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.84599999999999997,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.69099999999999995}]},{"osd":6,"last update":"Sat Jan 27 02:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.65500000000000003,"15min":0.66000000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.83099999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.625}]},{"osd":7,"last update":"Sat Jan 27 02:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.68600000000000005,"15min":0.69099999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.78200000000000003,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.68300000000000005,"15min":0.69099999999999995},"min":{"1min":0.50800000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.80900000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.70099999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":317,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194994,"statfs":{"total":200000000,"available":199674862,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40400000000000003,"15min":0.40600000000000003},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.64700000000000002,"15min":0.68300000000000005},"min":{"1min":0.34300000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.64500000000000002,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":0.47799999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.41599999999999998,"15min":0.41799999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.54400000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.443,"15min":0.44800000000000001},"min":{"1min":0.308,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.51000000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.38900000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.40799999999999997,"15min":0.40999999999999998},"min":{"1min":0.29299999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.48399999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.48199999999999998,"15min":0.48399999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.64800000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.46100000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64600000000000002,"15min":0.64400000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88800000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.80500000000000005,"15min":0.82299999999999995},"min":{"1min":0.56799999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.96499999999999997,"5min":3.581,"15min":3.581},"last":0.82299999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.746,"15min":0.76600000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88400000000000001,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67100000000000004,"15min":0.66900000000000004},"min":{"1min":0.51600000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96599999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.80200000000000005}]},{"osd":6,"last update":"Sat Jan 27 02:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.82699999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.875},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84099999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.78800000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67100000000000004,"15min":0.67400000000000004},"min":{"1min":0.52500000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.76600000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.68400000000000005,"15min":0.68500000000000005},"min":{"1min":0.46800000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73099999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":323,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194988,"statfs":{"total":200000000,"available":199668574,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.35899999999999999,"15min":0.34899999999999998},"min":{"1min":0.28899999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.42999999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.38700000000000001,"15min":0.38400000000000001},"min":{"1min":0.30299999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.56100000000000005,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.33400000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.52000000000000002,"5min":0.628,"15min":0.628},"last":0.443}]},{"osd":2,"last update":"Sat Jan 27 02:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.42099999999999999,"15min":0.41799999999999998},"min":{"1min":0.34399999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.53400000000000003,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.443,"15min":0.437},"min":{"1min":0.36699999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.39700000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79500000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.49099999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.79100000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.45100000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66400000000000003,"15min":0.66300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.81299999999999994,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.69999999999999996}]},{"osd":6,"last update":"Sat Jan 27 02:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.53000000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.78200000000000003,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.751},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.48499999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.81899999999999995,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.64100000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.628,"15min":0.624},"min":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.80400000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65600000000000003,"15min":0.65700000000000003},"min":{"1min":0.496,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.77000000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":325,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194987,"statfs":{"total":200000000,"available":199667094,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58399999999999996,"15min":0.58699999999999997},"min":{"1min":0.33400000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.98199999999999998,"5min":1.099,"15min":1.099},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.129,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.56399999999999995}]},{"osd":1,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.59799999999999998,"15min":0.59199999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81499999999999995,"5min":1.133,"15min":1.133},"last":0.54300000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.45300000000000001,"5min":0.434,"15min":0.434},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.437,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0009999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.58699999999999997}]},{"osd":3,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79600000000000004,"5min":1.115,"15min":1.115},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.5,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.2839999999999998,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.57699999999999996}]},{"osd":5,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48599999999999999,"15min":0.48699999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.495,"15min":0.49199999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72899999999999998,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.42699999999999999}]},{"osd":6,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.34899999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.48399999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.496},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.40999999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.44600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":318,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194994,"statfs":{"total":200000000,"available":199674294,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.60499999999999998,"15min":0.57799999999999996},"min":{"1min":0.40999999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.242,"5min":3.242,"15min":3.242},"last":0.47199999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.61599999999999999,"15min":0.60499999999999998},"min":{"1min":0.46000000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65800000000000003,"15min":0.66000000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.88500000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.55500000000000005}]},{"osd":2,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.62,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":3.04,"5min":3.04,"15min":3.04},"last":0.5},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.85599999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.52100000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.71499999999999997,"15min":0.69499999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":4.032,"5min":4.032,"15min":4.032},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.71199999999999997,"15min":0.68100000000000005},"min":{"1min":0.51300000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":4.282,"5min":4.282,"15min":4.282},"last":0.69899999999999995}]},{"osd":4,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53200000000000003,"15min":0.52900000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.499,"5min":0.57299999999999995,"15min":0.58499999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73099999999999998,"5min":1.101,"15min":1.101},"last":0.68100000000000005}]},{"osd":6,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56699999999999995,"15min":0.57399999999999995},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59499999999999997,"15min":0.59199999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.72099999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91100000000000003,"5min":1.008,"15min":1.008},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2,"5min":2,"15min":2},"last":0.67200000000000004}]}]}],"pool_statfs":[]}} 2024-01-27T02:34:19.243 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:34:19.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.410+0000 7fcbd4842700 1 Processor -- start 2024-01-27T02:34:19.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.410+0000 7fcbd4842700 1 -- start start 2024-01-27T02:34:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.411+0000 7fcbd4842700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcbd015a100 0x7fcbd00cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.411+0000 7fcbd4842700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd00ce5e0 con 0x7fcbd00cb5b0 2024-01-27T02:34:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.411+0000 7fcbd4842700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd00cd9d0 con 0x7fcbd00c4cc0 2024-01-27T02:34:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.411+0000 7fcbd4842700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd00cdcb0 con 0x7fcbd015a100 2024-01-27T02:34:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.411+0000 7fcbce59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcbd015a100 0x7fcbd00cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.411+0000 7fcbce59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcbd015a100 0x7fcbd00cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52852/0 (socket says 172.21.15.7:52852) 2024-01-27T02:34:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.411+0000 7fcbce59c700 1 -- 172.21.15.7:0/2807528136 learned_addr learned my addr 172.21.15.7:0/2807528136 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.411+0000 7fcbcdd9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcbd00cb5b0 0x7fcbd00c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42216/0 (socket says 172.21.15.7:42216) 2024-01-27T02:34:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.411+0000 7fcbce59c700 1 -- 172.21.15.7:0/2807528136 >> v1:172.21.15.188:6789/0 conn(0x7fcbd00c4cc0 legacy=0x7fcbd0163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.411+0000 7fcbce59c700 1 -- 172.21.15.7:0/2807528136 >> v1:172.21.15.7:6789/0 conn(0x7fcbd00cb5b0 legacy=0x7fcbd00c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.411+0000 7fcbce59c700 1 -- 172.21.15.7:0/2807528136 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd00cdf90 con 0x7fcbd015a100 2024-01-27T02:34:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.411+0000 7fcbcd59a700 1 -- 172.21.15.7:0/2807528136 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2529994788 0 0) 0x7fcbd00ce5e0 con 0x7fcbd00cb5b0 2024-01-27T02:34:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.412+0000 7fcbce59c700 1 --2- 172.21.15.7:0/2807528136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd015a100 0x7fcbd00cd2f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00099b0 tx=0x7fcbc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb72499c8ab82062 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.412+0000 7fcbcd59a700 1 -- 172.21.15.7:0/2807528136 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc000b820 con 0x7fcbd015a100 2024-01-27T02:34:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.412+0000 7fcbcd59a700 1 -- 172.21.15.7:0/2807528136 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc000b980 con 0x7fcbd015a100 2024-01-27T02:34:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.412+0000 7fcbcd59a700 1 -- 172.21.15.7:0/2807528136 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc000bd20 con 0x7fcbd015a100 2024-01-27T02:34:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.412+0000 7fcbd4842700 1 -- 172.21.15.7:0/2807528136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd015a100 msgr2=0x7fcbd00cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.412+0000 7fcbd4842700 1 --2- 172.21.15.7:0/2807528136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd015a100 0x7fcbd00cd2f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00099b0 tx=0x7fcbc0009250 comp rx=0 tx=0).stop 2024-01-27T02:34:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.412+0000 7fcbd4842700 1 -- 172.21.15.7:0/2807528136 shutdown_connections 2024-01-27T02:34:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.412+0000 7fcbd4842700 1 --2- 172.21.15.7:0/2807528136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd015a100 0x7fcbd00cd2f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.412+0000 7fcbd4842700 1 -- 172.21.15.7:0/2807528136 >> 172.21.15.7:0/2807528136 conn(0x7fcbd0098020 msgr2=0x7fcbd00711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.412+0000 7fcbd4842700 1 -- 172.21.15.7:0/2807528136 shutdown_connections 2024-01-27T02:34:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.412+0000 7fcbd4842700 1 -- 172.21.15.7:0/2807528136 wait complete. 2024-01-27T02:34:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.414+0000 7fcbd4842700 1 Processor -- start 2024-01-27T02:34:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.414+0000 7fcbd4842700 1 -- start start 2024-01-27T02:34:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.414+0000 7fcbd4842700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbd00c4cc0 0x7fcbd01545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.414+0000 7fcbd4842700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbd00cb5b0 0x7fcbd015ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.414+0000 7fcbd4842700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd015a100 0x7fcbd0068870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.414+0000 7fcbd4842700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd01607c0 con 0x7fcbd015a100 2024-01-27T02:34:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.414+0000 7fcbd4842700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd0154b30 con 0x7fcbd00c4cc0 2024-01-27T02:34:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.414+0000 7fcbd4842700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd015c4d0 con 0x7fcbd00cb5b0 2024-01-27T02:34:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.414+0000 7fcbced9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd015a100 0x7fcbd0068870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.415+0000 7fcbced9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd015a100 0x7fcbd0068870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52864/0 (socket says 172.21.15.7:52864) 2024-01-27T02:34:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.415+0000 7fcbced9d700 1 -- 172.21.15.7:0/3616826131 learned_addr learned my addr 172.21.15.7:0/3616826131 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.415+0000 7fcbce59c700 1 --2- 172.21.15.7:0/3616826131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbd00c4cc0 0x7fcbd01545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.415+0000 7fcbcdd9b700 1 --2- 172.21.15.7:0/3616826131 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbd00cb5b0 0x7fcbd015ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.415+0000 7fcbced9d700 1 -- 172.21.15.7:0/3616826131 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbd00cb5b0 msgr2=0x7fcbd015ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:19.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.415+0000 7fcbced9d700 1 --2- 172.21.15.7:0/3616826131 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbd00cb5b0 0x7fcbd015ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:19.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.415+0000 7fcbced9d700 1 -- 172.21.15.7:0/3616826131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbd00c4cc0 msgr2=0x7fcbd01545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:19.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.415+0000 7fcbced9d700 1 --2- 172.21.15.7:0/3616826131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbd00c4cc0 0x7fcbd01545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:19.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.415+0000 7fcbced9d700 1 -- 172.21.15.7:0/3616826131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc00094f0 con 0x7fcbd015a100 2024-01-27T02:34:19.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.415+0000 7fcbce59c700 1 --2- 172.21.15.7:0/3616826131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbd00c4cc0 0x7fcbd01545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:19.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.415+0000 7fcbcdd9b700 1 --2- 172.21.15.7:0/3616826131 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbd00cb5b0 0x7fcbd015ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:19.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.415+0000 7fcbced9d700 1 --2- 172.21.15.7:0/3616826131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd015a100 0x7fcbd0068870 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800c860 tx=0x7fcbc8002680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:19.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.415+0000 7fcbbe7fc700 1 -- 172.21.15.7:0/3616826131 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8012630 con 0x7fcbd015a100 2024-01-27T02:34:19.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.415+0000 7fcbbe7fc700 1 -- 172.21.15.7:0/3616826131 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc80127b0 con 0x7fcbd015a100 2024-01-27T02:34:19.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.415+0000 7fcbbe7fc700 1 -- 172.21.15.7:0/3616826131 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8012b70 con 0x7fcbd015a100 2024-01-27T02:34:19.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.415+0000 7fcbd4842700 1 -- 172.21.15.7:0/3616826131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd015b8c0 con 0x7fcbd015a100 2024-01-27T02:34:19.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.415+0000 7fcbd4842700 1 -- 172.21.15.7:0/3616826131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd015be80 con 0x7fcbd015a100 2024-01-27T02:34:19.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.416+0000 7fcbd4842700 1 -- 172.21.15.7:0/3616826131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd015c610 con 0x7fcbd015a100 2024-01-27T02:34:19.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.417+0000 7fcbbe7fc700 1 -- 172.21.15.7:0/3616826131 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcbc801b400 con 0x7fcbd015a100 2024-01-27T02:34:19.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.417+0000 7fcbbe7fc700 1 --2- 172.21.15.7:0/3616826131 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcbb0066f30 0x7fcbb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:19.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.417+0000 7fcbbe7fc700 1 -- 172.21.15.7:0/3616826131 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcbc8084930 con 0x7fcbd015a100 2024-01-27T02:34:19.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.417+0000 7fcbce59c700 1 --2- 172.21.15.7:0/3616826131 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcbb0066f30 0x7fcbb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:19.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.418+0000 7fcbce59c700 1 --2- 172.21.15.7:0/3616826131 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcbb0066f30 0x7fcbb00693e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00099b0 tx=0x7fcbc0016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:19.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.422+0000 7fcbbe7fc700 1 -- 172.21.15.7:0/3616826131 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcbc8051c50 con 0x7fcbd015a100 2024-01-27T02:34:19.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.620+0000 7fcbd4842700 1 -- 172.21.15.7:0/3616826131 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcbd004c710 con 0x7fcbb0066f30 2024-01-27T02:34:19.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.628+0000 7fcbbe7fc700 1 -- 172.21.15.7:0/3616826131 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285240 (secure 0 0 0) 0x7fcbd004c710 con 0x7fcbb0066f30 2024-01-27T02:34:19.634 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:34:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.631+0000 7fcbd4842700 1 -- 172.21.15.7:0/3616826131 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcbb0066f30 msgr2=0x7fcbb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.631+0000 7fcbd4842700 1 --2- 172.21.15.7:0/3616826131 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcbb0066f30 0x7fcbb00693e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00099b0 tx=0x7fcbc0016000 comp rx=0 tx=0).stop 2024-01-27T02:34:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.631+0000 7fcbd4842700 1 -- 172.21.15.7:0/3616826131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd015a100 msgr2=0x7fcbd0068870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.631+0000 7fcbd4842700 1 --2- 172.21.15.7:0/3616826131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd015a100 0x7fcbd0068870 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800c860 tx=0x7fcbc8002680 comp rx=0 tx=0).stop 2024-01-27T02:34:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.632+0000 7fcbd4842700 1 -- 172.21.15.7:0/3616826131 shutdown_connections 2024-01-27T02:34:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.632+0000 7fcbd4842700 1 --2- 172.21.15.7:0/3616826131 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcbb0066f30 0x7fcbb00693e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.632+0000 7fcbd4842700 1 --2- 172.21.15.7:0/3616826131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd015a100 0x7fcbd0068870 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.632+0000 7fcbd4842700 1 --2- 172.21.15.7:0/3616826131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbd00c4cc0 0x7fcbd01545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.632+0000 7fcbd4842700 1 --2- 172.21.15.7:0/3616826131 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbd00cb5b0 0x7fcbd015ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:19.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.632+0000 7fcbd4842700 1 -- 172.21.15.7:0/3616826131 >> 172.21.15.7:0/3616826131 conn(0x7fcbd0098020 msgr2=0x7fcbd006b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:19.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.632+0000 7fcbd4842700 1 -- 172.21.15.7:0/3616826131 shutdown_connections 2024-01-27T02:34:19.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.632+0000 7fcbd4842700 1 -- 172.21.15.7:0/3616826131 wait complete. 2024-01-27T02:34:19.637 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:34:19.647 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-01-27T02:34:19.604460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":102,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3709,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558790,"statfs":{"total":1600000000,"available":1596201420,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001486"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.832172+0000","last_change":"2024-01-27T02:34:14.832008+0000","last_active":"2024-01-27T02:34:14.832172+0000","last_peered":"2024-01-27T02:34:14.832172+0000","last_clean":"2024-01-27T02:34:14.832172+0000","last_became_active":"2024-01-27T02:34:14.831727+0000","last_became_peered":"2024-01-27T02:34:14.831727+0000","last_unstale":"2024-01-27T02:34:14.832172+0000","last_undegraded":"2024-01-27T02:34:14.832172+0000","last_fullsized":"2024-01-27T02:34:14.832172+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:31:01.741897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.829074+0000","last_change":"2024-01-27T02:34:14.828797+0000","last_active":"2024-01-27T02:34:14.829074+0000","last_peered":"2024-01-27T02:34:14.829074+0000","last_clean":"2024-01-27T02:34:14.829074+0000","last_became_active":"2024-01-27T02:34:14.828497+0000","last_became_peered":"2024-01-27T02:34:14.828497+0000","last_unstale":"2024-01-27T02:34:14.829074+0000","last_undegraded":"2024-01-27T02:34:14.829074+0000","last_fullsized":"2024-01-27T02:34:14.829074+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:18:33.953475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.829016+0000","last_change":"2024-01-27T02:34:14.828841+0000","last_active":"2024-01-27T02:34:14.829016+0000","last_peered":"2024-01-27T02:34:14.829016+0000","last_clean":"2024-01-27T02:34:14.829016+0000","last_became_active":"2024-01-27T02:34:14.828110+0000","last_became_peered":"2024-01-27T02:34:14.828110+0000","last_unstale":"2024-01-27T02:34:14.829016+0000","last_undegraded":"2024-01-27T02:34:14.829016+0000","last_fullsized":"2024-01-27T02:34:14.829016+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:44:29.126424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.830679+0000","last_change":"2024-01-27T02:34:14.830679+0000","last_active":"2024-01-27T02:34:14.830679+0000","last_peered":"2024-01-27T02:34:14.830679+0000","last_clean":"2024-01-27T02:34:14.830679+0000","last_became_active":"2024-01-27T02:34:14.830445+0000","last_became_peered":"2024-01-27T02:34:14.830445+0000","last_unstale":"2024-01-27T02:34:14.830679+0000","last_undegraded":"2024-01-27T02:34:14.830679+0000","last_fullsized":"2024-01-27T02:34:14.830679+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:32:49.387479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.830531+0000","last_change":"2024-01-27T02:34:14.830215+0000","last_active":"2024-01-27T02:34:14.830531+0000","last_peered":"2024-01-27T02:34:14.830531+0000","last_clean":"2024-01-27T02:34:14.830531+0000","last_became_active":"2024-01-27T02:34:14.829958+0000","last_became_peered":"2024-01-27T02:34:14.829958+0000","last_unstale":"2024-01-27T02:34:14.830531+0000","last_undegraded":"2024-01-27T02:34:14.830531+0000","last_fullsized":"2024-01-27T02:34:14.830531+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:38:26.502127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.829701+0000","last_change":"2024-01-27T02:34:14.829701+0000","last_active":"2024-01-27T02:34:14.829701+0000","last_peered":"2024-01-27T02:34:14.829701+0000","last_clean":"2024-01-27T02:34:14.829701+0000","last_became_active":"2024-01-27T02:34:14.828723+0000","last_became_peered":"2024-01-27T02:34:14.828723+0000","last_unstale":"2024-01-27T02:34:14.829701+0000","last_undegraded":"2024-01-27T02:34:14.829701+0000","last_fullsized":"2024-01-27T02:34:14.829701+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:58:33.213596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.824463+0000","last_change":"2024-01-27T02:34:14.824379+0000","last_active":"2024-01-27T02:34:14.824463+0000","last_peered":"2024-01-27T02:34:14.824463+0000","last_clean":"2024-01-27T02:34:14.824463+0000","last_became_active":"2024-01-27T02:34:14.824122+0000","last_became_peered":"2024-01-27T02:34:14.824122+0000","last_unstale":"2024-01-27T02:34:14.824463+0000","last_undegraded":"2024-01-27T02:34:14.824463+0000","last_fullsized":"2024-01-27T02:34:14.824463+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:27:31.764056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.821615+0000","last_change":"2024-01-27T02:34:14.821539+0000","last_active":"2024-01-27T02:34:14.821615+0000","last_peered":"2024-01-27T02:34:14.821615+0000","last_clean":"2024-01-27T02:34:14.821615+0000","last_became_active":"2024-01-27T02:34:14.821291+0000","last_became_peered":"2024-01-27T02:34:14.821291+0000","last_unstale":"2024-01-27T02:34:14.821615+0000","last_undegraded":"2024-01-27T02:34:14.821615+0000","last_fullsized":"2024-01-27T02:34:14.821615+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:49:13.682800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.825084+0000","last_change":"2024-01-27T02:34:14.824740+0000","last_active":"2024-01-27T02:34:14.825084+0000","last_peered":"2024-01-27T02:34:14.825084+0000","last_clean":"2024-01-27T02:34:14.825084+0000","last_became_active":"2024-01-27T02:34:14.824470+0000","last_became_peered":"2024-01-27T02:34:14.824470+0000","last_unstale":"2024-01-27T02:34:14.825084+0000","last_undegraded":"2024-01-27T02:34:14.825084+0000","last_fullsized":"2024-01-27T02:34:14.825084+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:38:06.611401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.834061+0000","last_change":"2024-01-27T02:34:14.834061+0000","last_active":"2024-01-27T02:34:14.834061+0000","last_peered":"2024-01-27T02:34:14.834061+0000","last_clean":"2024-01-27T02:34:14.834061+0000","last_became_active":"2024-01-27T02:34:14.833423+0000","last_became_peered":"2024-01-27T02:34:14.833423+0000","last_unstale":"2024-01-27T02:34:14.834061+0000","last_undegraded":"2024-01-27T02:34:14.834061+0000","last_fullsized":"2024-01-27T02:34:14.834061+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:19:58.242716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.836298+0000","last_change":"2024-01-27T02:34:14.836298+0000","last_active":"2024-01-27T02:34:14.836298+0000","last_peered":"2024-01-27T02:34:14.836298+0000","last_clean":"2024-01-27T02:34:14.836298+0000","last_became_active":"2024-01-27T02:34:14.836049+0000","last_became_peered":"2024-01-27T02:34:14.836049+0000","last_unstale":"2024-01-27T02:34:14.836298+0000","last_undegraded":"2024-01-27T02:34:14.836298+0000","last_fullsized":"2024-01-27T02:34:14.836298+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:36:32.916928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.826633+0000","last_change":"2024-01-27T02:34:14.826633+0000","last_active":"2024-01-27T02:34:14.826633+0000","last_peered":"2024-01-27T02:34:14.826633+0000","last_clean":"2024-01-27T02:34:14.826633+0000","last_became_active":"2024-01-27T02:34:14.826298+0000","last_became_peered":"2024-01-27T02:34:14.826298+0000","last_unstale":"2024-01-27T02:34:14.826633+0000","last_undegraded":"2024-01-27T02:34:14.826633+0000","last_fullsized":"2024-01-27T02:34:14.826633+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:38:24.525603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.829594+0000","last_change":"2024-01-27T02:34:14.829594+0000","last_active":"2024-01-27T02:34:14.829594+0000","last_peered":"2024-01-27T02:34:14.829594+0000","last_clean":"2024-01-27T02:34:14.829594+0000","last_became_active":"2024-01-27T02:34:14.829267+0000","last_became_peered":"2024-01-27T02:34:14.829267+0000","last_unstale":"2024-01-27T02:34:14.829594+0000","last_undegraded":"2024-01-27T02:34:14.829594+0000","last_fullsized":"2024-01-27T02:34:14.829594+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:10:26.114865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.825559+0000","last_change":"2024-01-27T02:34:14.825216+0000","last_active":"2024-01-27T02:34:14.825559+0000","last_peered":"2024-01-27T02:34:14.825559+0000","last_clean":"2024-01-27T02:34:14.825559+0000","last_became_active":"2024-01-27T02:34:14.824976+0000","last_became_peered":"2024-01-27T02:34:14.824976+0000","last_unstale":"2024-01-27T02:34:14.825559+0000","last_undegraded":"2024-01-27T02:34:14.825559+0000","last_fullsized":"2024-01-27T02:34:14.825559+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:48:22.462890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.830810+0000","last_change":"2024-01-27T02:34:14.830698+0000","last_active":"2024-01-27T02:34:14.830810+0000","last_peered":"2024-01-27T02:34:14.830810+0000","last_clean":"2024-01-27T02:34:14.830810+0000","last_became_active":"2024-01-27T02:34:14.829984+0000","last_became_peered":"2024-01-27T02:34:14.829984+0000","last_unstale":"2024-01-27T02:34:14.830810+0000","last_undegraded":"2024-01-27T02:34:14.830810+0000","last_fullsized":"2024-01-27T02:34:14.830810+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:02:24.823736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.829364+0000","last_change":"2024-01-27T02:34:14.829364+0000","last_active":"2024-01-27T02:34:14.829364+0000","last_peered":"2024-01-27T02:34:14.829364+0000","last_clean":"2024-01-27T02:34:14.829364+0000","last_became_active":"2024-01-27T02:34:14.828289+0000","last_became_peered":"2024-01-27T02:34:14.828289+0000","last_unstale":"2024-01-27T02:34:14.829364+0000","last_undegraded":"2024-01-27T02:34:14.829364+0000","last_fullsized":"2024-01-27T02:34:14.829364+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:42:30.170573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.828985+0000","last_change":"2024-01-27T02:34:14.825531+0000","last_active":"2024-01-27T02:34:14.828985+0000","last_peered":"2024-01-27T02:34:14.828985+0000","last_clean":"2024-01-27T02:34:14.828985+0000","last_became_active":"2024-01-27T02:34:14.825038+0000","last_became_peered":"2024-01-27T02:34:14.825038+0000","last_unstale":"2024-01-27T02:34:14.828985+0000","last_undegraded":"2024-01-27T02:34:14.828985+0000","last_fullsized":"2024-01-27T02:34:14.828985+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:52:30.964086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.834577+0000","last_change":"2024-01-27T02:34:14.834507+0000","last_active":"2024-01-27T02:34:14.834577+0000","last_peered":"2024-01-27T02:34:14.834577+0000","last_clean":"2024-01-27T02:34:14.834577+0000","last_became_active":"2024-01-27T02:34:14.833556+0000","last_became_peered":"2024-01-27T02:34:14.833556+0000","last_unstale":"2024-01-27T02:34:14.834577+0000","last_undegraded":"2024-01-27T02:34:14.834577+0000","last_fullsized":"2024-01-27T02:34:14.834577+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:22:21.128760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.832472+0000","last_change":"2024-01-27T02:34:14.832371+0000","last_active":"2024-01-27T02:34:14.832472+0000","last_peered":"2024-01-27T02:34:14.832472+0000","last_clean":"2024-01-27T02:34:14.832472+0000","last_became_active":"2024-01-27T02:34:14.832110+0000","last_became_peered":"2024-01-27T02:34:14.832110+0000","last_unstale":"2024-01-27T02:34:14.832472+0000","last_undegraded":"2024-01-27T02:34:14.832472+0000","last_fullsized":"2024-01-27T02:34:14.832472+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:55:19.182457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.828440+0000","last_change":"2024-01-27T02:34:14.824527+0000","last_active":"2024-01-27T02:34:14.828440+0000","last_peered":"2024-01-27T02:34:14.828440+0000","last_clean":"2024-01-27T02:34:14.828440+0000","last_became_active":"2024-01-27T02:34:14.824033+0000","last_became_peered":"2024-01-27T02:34:14.824033+0000","last_unstale":"2024-01-27T02:34:14.828440+0000","last_undegraded":"2024-01-27T02:34:14.828440+0000","last_fullsized":"2024-01-27T02:34:14.828440+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:32:14.609662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.828292+0000","last_change":"2024-01-27T02:34:14.826918+0000","last_active":"2024-01-27T02:34:14.828292+0000","last_peered":"2024-01-27T02:34:14.828292+0000","last_clean":"2024-01-27T02:34:14.828292+0000","last_became_active":"2024-01-27T02:34:14.826773+0000","last_became_peered":"2024-01-27T02:34:14.826773+0000","last_unstale":"2024-01-27T02:34:14.828292+0000","last_undegraded":"2024-01-27T02:34:14.828292+0000","last_fullsized":"2024-01-27T02:34:14.828292+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:53:19.368477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.828516+0000","last_change":"2024-01-27T02:34:14.825368+0000","last_active":"2024-01-27T02:34:14.828516+0000","last_peered":"2024-01-27T02:34:14.828516+0000","last_clean":"2024-01-27T02:34:14.828516+0000","last_became_active":"2024-01-27T02:34:14.824445+0000","last_became_peered":"2024-01-27T02:34:14.824445+0000","last_unstale":"2024-01-27T02:34:14.828516+0000","last_undegraded":"2024-01-27T02:34:14.828516+0000","last_fullsized":"2024-01-27T02:34:14.828516+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:37:02.591325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.834865+0000","last_change":"2024-01-27T02:34:14.834803+0000","last_active":"2024-01-27T02:34:14.834865+0000","last_peered":"2024-01-27T02:34:14.834865+0000","last_clean":"2024-01-27T02:34:14.834865+0000","last_became_active":"2024-01-27T02:34:14.834270+0000","last_became_peered":"2024-01-27T02:34:14.834270+0000","last_unstale":"2024-01-27T02:34:14.834865+0000","last_undegraded":"2024-01-27T02:34:14.834865+0000","last_fullsized":"2024-01-27T02:34:14.834865+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:46:48.013304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.829022+0000","last_change":"2024-01-27T02:34:14.829022+0000","last_active":"2024-01-27T02:34:14.829022+0000","last_peered":"2024-01-27T02:34:14.829022+0000","last_clean":"2024-01-27T02:34:14.829022+0000","last_became_active":"2024-01-27T02:34:14.828758+0000","last_became_peered":"2024-01-27T02:34:14.828758+0000","last_unstale":"2024-01-27T02:34:14.829022+0000","last_undegraded":"2024-01-27T02:34:14.829022+0000","last_fullsized":"2024-01-27T02:34:14.829022+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:50:30.225349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.833187+0000","last_change":"2024-01-27T02:34:14.833012+0000","last_active":"2024-01-27T02:34:14.833187+0000","last_peered":"2024-01-27T02:34:14.833187+0000","last_clean":"2024-01-27T02:34:14.833187+0000","last_became_active":"2024-01-27T02:34:14.832758+0000","last_became_peered":"2024-01-27T02:34:14.832758+0000","last_unstale":"2024-01-27T02:34:14.833187+0000","last_undegraded":"2024-01-27T02:34:14.833187+0000","last_fullsized":"2024-01-27T02:34:14.833187+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:26:45.431640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.825531+0000","last_change":"2024-01-27T02:34:14.825531+0000","last_active":"2024-01-27T02:34:14.825531+0000","last_peered":"2024-01-27T02:34:14.825531+0000","last_clean":"2024-01-27T02:34:14.825531+0000","last_became_active":"2024-01-27T02:34:14.825229+0000","last_became_peered":"2024-01-27T02:34:14.825229+0000","last_unstale":"2024-01-27T02:34:14.825531+0000","last_undegraded":"2024-01-27T02:34:14.825531+0000","last_fullsized":"2024-01-27T02:34:14.825531+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:35:26.921373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.827457+0000","last_change":"2024-01-27T02:34:14.827457+0000","last_active":"2024-01-27T02:34:14.827457+0000","last_peered":"2024-01-27T02:34:14.827457+0000","last_clean":"2024-01-27T02:34:14.827457+0000","last_became_active":"2024-01-27T02:34:14.827196+0000","last_became_peered":"2024-01-27T02:34:14.827196+0000","last_unstale":"2024-01-27T02:34:14.827457+0000","last_undegraded":"2024-01-27T02:34:14.827457+0000","last_fullsized":"2024-01-27T02:34:14.827457+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:51:43.116654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.831118+0000","last_change":"2024-01-27T02:34:14.830990+0000","last_active":"2024-01-27T02:34:14.831118+0000","last_peered":"2024-01-27T02:34:14.831118+0000","last_clean":"2024-01-27T02:34:14.831118+0000","last_became_active":"2024-01-27T02:34:14.830615+0000","last_became_peered":"2024-01-27T02:34:14.830615+0000","last_unstale":"2024-01-27T02:34:14.831118+0000","last_undegraded":"2024-01-27T02:34:14.831118+0000","last_fullsized":"2024-01-27T02:34:14.831118+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:25:03.230683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.829030+0000","last_change":"2024-01-27T02:34:14.827218+0000","last_active":"2024-01-27T02:34:14.829030+0000","last_peered":"2024-01-27T02:34:14.829030+0000","last_clean":"2024-01-27T02:34:14.829030+0000","last_became_active":"2024-01-27T02:34:14.826428+0000","last_became_peered":"2024-01-27T02:34:14.826428+0000","last_unstale":"2024-01-27T02:34:14.829030+0000","last_undegraded":"2024-01-27T02:34:14.829030+0000","last_fullsized":"2024-01-27T02:34:14.829030+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:37:00.713869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.830509+0000","last_change":"2024-01-27T02:34:14.830415+0000","last_active":"2024-01-27T02:34:14.830509+0000","last_peered":"2024-01-27T02:34:14.830509+0000","last_clean":"2024-01-27T02:34:14.830509+0000","last_became_active":"2024-01-27T02:34:14.830139+0000","last_became_peered":"2024-01-27T02:34:14.830139+0000","last_unstale":"2024-01-27T02:34:14.830509+0000","last_undegraded":"2024-01-27T02:34:14.830509+0000","last_fullsized":"2024-01-27T02:34:14.830509+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:51:43.116654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.832994+0000","last_change":"2024-01-27T02:34:14.832559+0000","last_active":"2024-01-27T02:34:14.832994+0000","last_peered":"2024-01-27T02:34:14.832994+0000","last_clean":"2024-01-27T02:34:14.832994+0000","last_became_active":"2024-01-27T02:34:14.832310+0000","last_became_peered":"2024-01-27T02:34:14.832310+0000","last_unstale":"2024-01-27T02:34:14.832994+0000","last_undegraded":"2024-01-27T02:34:14.832994+0000","last_fullsized":"2024-01-27T02:34:14.832994+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:13:32.756245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.829991+0000","last_change":"2024-01-27T02:34:14.829991+0000","last_active":"2024-01-27T02:34:14.829991+0000","last_peered":"2024-01-27T02:34:14.829991+0000","last_clean":"2024-01-27T02:34:14.829991+0000","last_became_active":"2024-01-27T02:34:14.829668+0000","last_became_peered":"2024-01-27T02:34:14.829668+0000","last_unstale":"2024-01-27T02:34:14.829991+0000","last_undegraded":"2024-01-27T02:34:14.829991+0000","last_fullsized":"2024-01-27T02:34:14.829991+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:25:34.499048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.831396+0000","last_change":"2024-01-27T02:34:14.831396+0000","last_active":"2024-01-27T02:34:14.831396+0000","last_peered":"2024-01-27T02:34:14.831396+0000","last_clean":"2024-01-27T02:34:14.831396+0000","last_became_active":"2024-01-27T02:34:14.831159+0000","last_became_peered":"2024-01-27T02:34:14.831159+0000","last_unstale":"2024-01-27T02:34:14.831396+0000","last_undegraded":"2024-01-27T02:34:14.831396+0000","last_fullsized":"2024-01-27T02:34:14.831396+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:22:21.128760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"64'4","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.065038+0000","last_change":"2024-01-27T02:34:14.044359+0000","last_active":"2024-01-27T02:34:15.065038+0000","last_peered":"2024-01-27T02:34:15.065038+0000","last_clean":"2024-01-27T02:34:15.065038+0000","last_became_active":"2024-01-27T02:34:14.043760+0000","last_became_peered":"2024-01-27T02:34:14.043760+0000","last_unstale":"2024-01-27T02:34:15.065038+0000","last_undegraded":"2024-01-27T02:34:15.065038+0000","last_fullsized":"2024-01-27T02:34:15.065038+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:20:21.119379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.835494+0000","last_change":"2024-01-27T02:34:14.835494+0000","last_active":"2024-01-27T02:34:14.835494+0000","last_peered":"2024-01-27T02:34:14.835494+0000","last_clean":"2024-01-27T02:34:14.835494+0000","last_became_active":"2024-01-27T02:34:14.834923+0000","last_became_peered":"2024-01-27T02:34:14.834923+0000","last_unstale":"2024-01-27T02:34:14.835494+0000","last_undegraded":"2024-01-27T02:34:14.835494+0000","last_fullsized":"2024-01-27T02:34:14.835494+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:01:15.315520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.064859+0000","last_change":"2024-01-27T02:34:14.038365+0000","last_active":"2024-01-27T02:34:15.064859+0000","last_peered":"2024-01-27T02:34:15.064859+0000","last_clean":"2024-01-27T02:34:15.064859+0000","last_became_active":"2024-01-27T02:34:14.038029+0000","last_became_peered":"2024-01-27T02:34:14.038029+0000","last_unstale":"2024-01-27T02:34:15.064859+0000","last_undegraded":"2024-01-27T02:34:15.064859+0000","last_fullsized":"2024-01-27T02:34:15.064859+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:42:29.168670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.829956+0000","last_change":"2024-01-27T02:34:14.829956+0000","last_active":"2024-01-27T02:34:14.829956+0000","last_peered":"2024-01-27T02:34:14.829956+0000","last_clean":"2024-01-27T02:34:14.829956+0000","last_became_active":"2024-01-27T02:34:14.829118+0000","last_became_peered":"2024-01-27T02:34:14.829118+0000","last_unstale":"2024-01-27T02:34:14.829956+0000","last_undegraded":"2024-01-27T02:34:14.829956+0000","last_fullsized":"2024-01-27T02:34:14.829956+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:32:49.387479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"64'2","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.064157+0000","last_change":"2024-01-27T02:34:14.046453+0000","last_active":"2024-01-27T02:34:15.064157+0000","last_peered":"2024-01-27T02:34:15.064157+0000","last_clean":"2024-01-27T02:34:15.064157+0000","last_became_active":"2024-01-27T02:34:14.046051+0000","last_became_peered":"2024-01-27T02:34:14.046051+0000","last_unstale":"2024-01-27T02:34:15.064157+0000","last_undegraded":"2024-01-27T02:34:15.064157+0000","last_fullsized":"2024-01-27T02:34:15.064157+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:02:23.821834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.826929+0000","last_change":"2024-01-27T02:34:14.826929+0000","last_active":"2024-01-27T02:34:14.826929+0000","last_peered":"2024-01-27T02:34:14.826929+0000","last_clean":"2024-01-27T02:34:14.826929+0000","last_became_active":"2024-01-27T02:34:14.826656+0000","last_became_peered":"2024-01-27T02:34:14.826656+0000","last_unstale":"2024-01-27T02:34:14.826929+0000","last_undegraded":"2024-01-27T02:34:14.826929+0000","last_fullsized":"2024-01-27T02:34:14.826929+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:03:33.933530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"64'2","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.065620+0000","last_change":"2024-01-27T02:34:14.047224+0000","last_active":"2024-01-27T02:34:15.065620+0000","last_peered":"2024-01-27T02:34:15.065620+0000","last_clean":"2024-01-27T02:34:15.065620+0000","last_became_active":"2024-01-27T02:34:14.046857+0000","last_became_peered":"2024-01-27T02:34:14.046857+0000","last_unstale":"2024-01-27T02:34:15.065620+0000","last_undegraded":"2024-01-27T02:34:15.065620+0000","last_fullsized":"2024-01-27T02:34:15.065620+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:48:21.460988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.828459+0000","last_change":"2024-01-27T02:34:14.824673+0000","last_active":"2024-01-27T02:34:14.828459+0000","last_peered":"2024-01-27T02:34:14.828459+0000","last_clean":"2024-01-27T02:34:14.828459+0000","last_became_active":"2024-01-27T02:34:14.823879+0000","last_became_peered":"2024-01-27T02:34:14.823879+0000","last_unstale":"2024-01-27T02:34:14.828459+0000","last_undegraded":"2024-01-27T02:34:14.828459+0000","last_fullsized":"2024-01-27T02:34:14.828459+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:11:18.711696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.062136+0000","last_change":"2024-01-27T02:34:14.047567+0000","last_active":"2024-01-27T02:34:15.062136+0000","last_peered":"2024-01-27T02:34:15.062136+0000","last_clean":"2024-01-27T02:34:15.062136+0000","last_became_active":"2024-01-27T02:34:14.046851+0000","last_became_peered":"2024-01-27T02:34:14.046851+0000","last_unstale":"2024-01-27T02:34:15.062136+0000","last_undegraded":"2024-01-27T02:34:15.062136+0000","last_fullsized":"2024-01-27T02:34:15.062136+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:50:29.223446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.828479+0000","last_change":"2024-01-27T02:34:14.824727+0000","last_active":"2024-01-27T02:34:14.828479+0000","last_peered":"2024-01-27T02:34:14.828479+0000","last_clean":"2024-01-27T02:34:14.828479+0000","last_became_active":"2024-01-27T02:34:14.823493+0000","last_became_peered":"2024-01-27T02:34:14.823493+0000","last_unstale":"2024-01-27T02:34:14.828479+0000","last_undegraded":"2024-01-27T02:34:14.828479+0000","last_fullsized":"2024-01-27T02:34:14.828479+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:22:44.449010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"64'1","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.820536+0000","last_change":"2024-01-27T02:34:14.042534+0000","last_active":"2024-01-27T02:34:14.820536+0000","last_peered":"2024-01-27T02:34:14.820536+0000","last_clean":"2024-01-27T02:34:14.820536+0000","last_became_active":"2024-01-27T02:34:14.042118+0000","last_became_peered":"2024-01-27T02:34:14.042118+0000","last_unstale":"2024-01-27T02:34:14.820536+0000","last_undegraded":"2024-01-27T02:34:14.820536+0000","last_fullsized":"2024-01-27T02:34:14.820536+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:09:52.420262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.832702+0000","last_change":"2024-01-27T02:34:14.832702+0000","last_active":"2024-01-27T02:34:14.832702+0000","last_peered":"2024-01-27T02:34:14.832702+0000","last_clean":"2024-01-27T02:34:14.832702+0000","last_became_active":"2024-01-27T02:34:14.832399+0000","last_became_peered":"2024-01-27T02:34:14.832399+0000","last_unstale":"2024-01-27T02:34:14.832702+0000","last_undegraded":"2024-01-27T02:34:14.832702+0000","last_fullsized":"2024-01-27T02:34:14.832702+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:27:10.780868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.046845+0000","last_change":"2024-01-27T02:34:14.046758+0000","last_active":"2024-01-27T02:34:14.046845+0000","last_peered":"2024-01-27T02:34:14.046845+0000","last_clean":"2024-01-27T02:34:14.046845+0000","last_became_active":"2024-01-27T02:34:14.046536+0000","last_became_peered":"2024-01-27T02:34:14.046536+0000","last_unstale":"2024-01-27T02:34:14.046845+0000","last_undegraded":"2024-01-27T02:34:14.046845+0000","last_fullsized":"2024-01-27T02:34:14.046845+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:02:23.821834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.829048+0000","last_change":"2024-01-27T02:34:14.827374+0000","last_active":"2024-01-27T02:34:14.829048+0000","last_peered":"2024-01-27T02:34:14.829048+0000","last_clean":"2024-01-27T02:34:14.829048+0000","last_became_active":"2024-01-27T02:34:14.826793+0000","last_became_peered":"2024-01-27T02:34:14.826793+0000","last_unstale":"2024-01-27T02:34:14.829048+0000","last_undegraded":"2024-01-27T02:34:14.829048+0000","last_fullsized":"2024-01-27T02:34:14.829048+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:39:52.794096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.818831+0000","last_change":"2024-01-27T02:34:14.051391+0000","last_active":"2024-01-27T02:34:14.818831+0000","last_peered":"2024-01-27T02:34:14.818831+0000","last_clean":"2024-01-27T02:34:14.818831+0000","last_became_active":"2024-01-27T02:34:14.051237+0000","last_became_peered":"2024-01-27T02:34:14.051237+0000","last_unstale":"2024-01-27T02:34:14.818831+0000","last_undegraded":"2024-01-27T02:34:14.818831+0000","last_fullsized":"2024-01-27T02:34:14.818831+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:45:52.767110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.828275+0000","last_change":"2024-01-27T02:34:14.827958+0000","last_active":"2024-01-27T02:34:14.828275+0000","last_peered":"2024-01-27T02:34:14.828275+0000","last_clean":"2024-01-27T02:34:14.828275+0000","last_became_active":"2024-01-27T02:34:14.827687+0000","last_became_peered":"2024-01-27T02:34:14.827687+0000","last_unstale":"2024-01-27T02:34:14.828275+0000","last_undegraded":"2024-01-27T02:34:14.828275+0000","last_fullsized":"2024-01-27T02:34:14.828275+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:37:33.100835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.826582+0000","last_change":"2024-01-27T02:34:14.044833+0000","last_active":"2024-01-27T02:34:14.826582+0000","last_peered":"2024-01-27T02:34:14.826582+0000","last_clean":"2024-01-27T02:34:14.826582+0000","last_became_active":"2024-01-27T02:34:14.044611+0000","last_became_peered":"2024-01-27T02:34:14.044611+0000","last_unstale":"2024-01-27T02:34:14.826582+0000","last_undegraded":"2024-01-27T02:34:14.826582+0000","last_fullsized":"2024-01-27T02:34:14.826582+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:59:15.966990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.833237+0000","last_change":"2024-01-27T02:34:14.833190+0000","last_active":"2024-01-27T02:34:14.833237+0000","last_peered":"2024-01-27T02:34:14.833237+0000","last_clean":"2024-01-27T02:34:14.833237+0000","last_became_active":"2024-01-27T02:34:14.832551+0000","last_became_peered":"2024-01-27T02:34:14.832551+0000","last_unstale":"2024-01-27T02:34:14.833237+0000","last_undegraded":"2024-01-27T02:34:14.833237+0000","last_fullsized":"2024-01-27T02:34:14.833237+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:32:49.387479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.819961+0000","last_change":"2024-01-27T02:34:14.051488+0000","last_active":"2024-01-27T02:34:14.819961+0000","last_peered":"2024-01-27T02:34:14.819961+0000","last_clean":"2024-01-27T02:34:14.819961+0000","last_became_active":"2024-01-27T02:34:14.049919+0000","last_became_peered":"2024-01-27T02:34:14.049919+0000","last_unstale":"2024-01-27T02:34:14.819961+0000","last_undegraded":"2024-01-27T02:34:14.819961+0000","last_fullsized":"2024-01-27T02:34:14.819961+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:58:25.986996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.833953+0000","last_change":"2024-01-27T02:34:14.833953+0000","last_active":"2024-01-27T02:34:14.833953+0000","last_peered":"2024-01-27T02:34:14.833953+0000","last_clean":"2024-01-27T02:34:14.833953+0000","last_became_active":"2024-01-27T02:34:14.830746+0000","last_became_peered":"2024-01-27T02:34:14.830746+0000","last_unstale":"2024-01-27T02:34:14.833953+0000","last_undegraded":"2024-01-27T02:34:14.833953+0000","last_fullsized":"2024-01-27T02:34:14.833953+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:50:30.225349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"64'3","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.065709+0000","last_change":"2024-01-27T02:34:14.045955+0000","last_active":"2024-01-27T02:34:15.065709+0000","last_peered":"2024-01-27T02:34:15.065709+0000","last_clean":"2024-01-27T02:34:15.065709+0000","last_became_active":"2024-01-27T02:34:14.045731+0000","last_became_peered":"2024-01-27T02:34:14.045731+0000","last_unstale":"2024-01-27T02:34:15.065709+0000","last_undegraded":"2024-01-27T02:34:15.065709+0000","last_fullsized":"2024-01-27T02:34:15.065709+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:37:32.098933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.834729+0000","last_change":"2024-01-27T02:34:14.834657+0000","last_active":"2024-01-27T02:34:14.834729+0000","last_peered":"2024-01-27T02:34:14.834729+0000","last_clean":"2024-01-27T02:34:14.834729+0000","last_became_active":"2024-01-27T02:34:14.833967+0000","last_became_peered":"2024-01-27T02:34:14.833967+0000","last_unstale":"2024-01-27T02:34:14.834729+0000","last_undegraded":"2024-01-27T02:34:14.834729+0000","last_fullsized":"2024-01-27T02:34:14.834729+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:25:34.499048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"64'6","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.065665+0000","last_change":"2024-01-27T02:34:14.045550+0000","last_active":"2024-01-27T02:34:15.065665+0000","last_peered":"2024-01-27T02:34:15.065665+0000","last_clean":"2024-01-27T02:34:15.065665+0000","last_became_active":"2024-01-27T02:34:14.044663+0000","last_became_peered":"2024-01-27T02:34:14.044663+0000","last_unstale":"2024-01-27T02:34:15.065665+0000","last_undegraded":"2024-01-27T02:34:15.065665+0000","last_fullsized":"2024-01-27T02:34:15.065665+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:03:28.482115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.828833+0000","last_change":"2024-01-27T02:34:14.828833+0000","last_active":"2024-01-27T02:34:14.828833+0000","last_peered":"2024-01-27T02:34:14.828833+0000","last_clean":"2024-01-27T02:34:14.828833+0000","last_became_active":"2024-01-27T02:34:14.828554+0000","last_became_peered":"2024-01-27T02:34:14.828554+0000","last_unstale":"2024-01-27T02:34:14.828833+0000","last_undegraded":"2024-01-27T02:34:14.828833+0000","last_fullsized":"2024-01-27T02:34:14.828833+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:42:30.170573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"64'2","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.063612+0000","last_change":"2024-01-27T02:34:14.050502+0000","last_active":"2024-01-27T02:34:15.063612+0000","last_peered":"2024-01-27T02:34:15.063612+0000","last_clean":"2024-01-27T02:34:15.063612+0000","last_became_active":"2024-01-27T02:34:14.049898+0000","last_became_peered":"2024-01-27T02:34:14.049898+0000","last_unstale":"2024-01-27T02:34:15.063612+0000","last_undegraded":"2024-01-27T02:34:15.063612+0000","last_fullsized":"2024-01-27T02:34:15.063612+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:13:56.673742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.831742+0000","last_change":"2024-01-27T02:34:14.831645+0000","last_active":"2024-01-27T02:34:14.831742+0000","last_peered":"2024-01-27T02:34:14.831742+0000","last_clean":"2024-01-27T02:34:14.831742+0000","last_became_active":"2024-01-27T02:34:14.830976+0000","last_became_peered":"2024-01-27T02:34:14.830976+0000","last_unstale":"2024-01-27T02:34:14.831742+0000","last_undegraded":"2024-01-27T02:34:14.831742+0000","last_fullsized":"2024-01-27T02:34:14.831742+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:23.803292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.041437+0000","last_change":"2024-01-27T02:34:14.041362+0000","last_active":"2024-01-27T02:34:14.041437+0000","last_peered":"2024-01-27T02:34:14.041437+0000","last_clean":"2024-01-27T02:34:14.041437+0000","last_became_active":"2024-01-27T02:34:14.041131+0000","last_became_peered":"2024-01-27T02:34:14.041131+0000","last_unstale":"2024-01-27T02:34:14.041437+0000","last_undegraded":"2024-01-27T02:34:14.041437+0000","last_fullsized":"2024-01-27T02:34:14.041437+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:45:45.390108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.825582+0000","last_change":"2024-01-27T02:34:14.825448+0000","last_active":"2024-01-27T02:34:14.825582+0000","last_peered":"2024-01-27T02:34:14.825582+0000","last_clean":"2024-01-27T02:34:14.825582+0000","last_became_active":"2024-01-27T02:34:14.824986+0000","last_became_peered":"2024-01-27T02:34:14.824986+0000","last_unstale":"2024-01-27T02:34:14.825582+0000","last_undegraded":"2024-01-27T02:34:14.825582+0000","last_fullsized":"2024-01-27T02:34:14.825582+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:20:22.121281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.046317+0000","last_change":"2024-01-27T02:34:14.045205+0000","last_active":"2024-01-27T02:34:14.046317+0000","last_peered":"2024-01-27T02:34:14.046317+0000","last_clean":"2024-01-27T02:34:14.046317+0000","last_became_active":"2024-01-27T02:34:14.044332+0000","last_became_peered":"2024-01-27T02:34:14.044332+0000","last_unstale":"2024-01-27T02:34:14.046317+0000","last_undegraded":"2024-01-27T02:34:14.046317+0000","last_fullsized":"2024-01-27T02:34:14.046317+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:09:52.420262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.830480+0000","last_change":"2024-01-27T02:34:14.829392+0000","last_active":"2024-01-27T02:34:14.830480+0000","last_peered":"2024-01-27T02:34:14.830480+0000","last_clean":"2024-01-27T02:34:14.830480+0000","last_became_active":"2024-01-27T02:34:14.829129+0000","last_became_peered":"2024-01-27T02:34:14.829129+0000","last_unstale":"2024-01-27T02:34:14.830480+0000","last_undegraded":"2024-01-27T02:34:14.830480+0000","last_fullsized":"2024-01-27T02:34:14.830480+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:02:25.313682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"64'8","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.066477+0000","last_change":"2024-01-27T02:34:14.041138+0000","last_active":"2024-01-27T02:34:15.066477+0000","last_peered":"2024-01-27T02:34:15.066477+0000","last_clean":"2024-01-27T02:34:15.066477+0000","last_became_active":"2024-01-27T02:34:14.040706+0000","last_became_peered":"2024-01-27T02:34:14.040706+0000","last_unstale":"2024-01-27T02:34:15.066477+0000","last_undegraded":"2024-01-27T02:34:15.066477+0000","last_fullsized":"2024-01-27T02:34:15.066477+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:09:52.420262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.831912+0000","last_change":"2024-01-27T02:34:14.831839+0000","last_active":"2024-01-27T02:34:14.831912+0000","last_peered":"2024-01-27T02:34:14.831912+0000","last_clean":"2024-01-27T02:34:14.831912+0000","last_became_active":"2024-01-27T02:34:14.831410+0000","last_became_peered":"2024-01-27T02:34:14.831410+0000","last_unstale":"2024-01-27T02:34:14.831912+0000","last_undegraded":"2024-01-27T02:34:14.831912+0000","last_fullsized":"2024-01-27T02:34:14.831912+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:58:33.213596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.063001+0000","last_change":"2024-01-27T02:34:14.048893+0000","last_active":"2024-01-27T02:34:15.063001+0000","last_peered":"2024-01-27T02:34:15.063001+0000","last_clean":"2024-01-27T02:34:15.063001+0000","last_became_active":"2024-01-27T02:34:14.047262+0000","last_became_peered":"2024-01-27T02:34:14.047262+0000","last_unstale":"2024-01-27T02:34:15.063001+0000","last_undegraded":"2024-01-27T02:34:15.063001+0000","last_fullsized":"2024-01-27T02:34:15.063001+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:38:23.523701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.828699+0000","last_change":"2024-01-27T02:34:14.828699+0000","last_active":"2024-01-27T02:34:14.828699+0000","last_peered":"2024-01-27T02:34:14.828699+0000","last_clean":"2024-01-27T02:34:14.828699+0000","last_became_active":"2024-01-27T02:34:14.828432+0000","last_became_peered":"2024-01-27T02:34:14.828432+0000","last_unstale":"2024-01-27T02:34:14.828699+0000","last_undegraded":"2024-01-27T02:34:14.828699+0000","last_fullsized":"2024-01-27T02:34:14.828699+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:51:43.116654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"64'2","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.064694+0000","last_change":"2024-01-27T02:34:14.046312+0000","last_active":"2024-01-27T02:34:15.064694+0000","last_peered":"2024-01-27T02:34:15.064694+0000","last_clean":"2024-01-27T02:34:15.064694+0000","last_became_active":"2024-01-27T02:34:14.045709+0000","last_became_peered":"2024-01-27T02:34:14.045709+0000","last_unstale":"2024-01-27T02:34:15.064694+0000","last_undegraded":"2024-01-27T02:34:15.064694+0000","last_fullsized":"2024-01-27T02:34:15.064694+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:59:15.966990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.821061+0000","last_change":"2024-01-27T02:34:14.820829+0000","last_active":"2024-01-27T02:34:14.821061+0000","last_peered":"2024-01-27T02:34:14.821061+0000","last_clean":"2024-01-27T02:34:14.821061+0000","last_became_active":"2024-01-27T02:34:14.820496+0000","last_became_peered":"2024-01-27T02:34:14.820496+0000","last_unstale":"2024-01-27T02:34:14.821061+0000","last_undegraded":"2024-01-27T02:34:14.821061+0000","last_fullsized":"2024-01-27T02:34:14.821061+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:25:24.180251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"64'1","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.824476+0000","last_change":"2024-01-27T02:34:14.047081+0000","last_active":"2024-01-27T02:34:14.824476+0000","last_peered":"2024-01-27T02:34:14.824476+0000","last_clean":"2024-01-27T02:34:14.824476+0000","last_became_active":"2024-01-27T02:34:14.046575+0000","last_became_peered":"2024-01-27T02:34:14.046575+0000","last_unstale":"2024-01-27T02:34:14.824476+0000","last_undegraded":"2024-01-27T02:34:14.824476+0000","last_fullsized":"2024-01-27T02:34:14.824476+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:20:21.119379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.833339+0000","last_change":"2024-01-27T02:34:14.833295+0000","last_active":"2024-01-27T02:34:14.833339+0000","last_peered":"2024-01-27T02:34:14.833339+0000","last_clean":"2024-01-27T02:34:14.833339+0000","last_became_active":"2024-01-27T02:34:14.832860+0000","last_became_peered":"2024-01-27T02:34:14.832860+0000","last_unstale":"2024-01-27T02:34:14.833339+0000","last_undegraded":"2024-01-27T02:34:14.833339+0000","last_fullsized":"2024-01-27T02:34:14.833339+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:35:26.921373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.046424+0000","last_change":"2024-01-27T02:34:14.045691+0000","last_active":"2024-01-27T02:34:14.046424+0000","last_peered":"2024-01-27T02:34:14.046424+0000","last_clean":"2024-01-27T02:34:14.046424+0000","last_became_active":"2024-01-27T02:34:14.045396+0000","last_became_peered":"2024-01-27T02:34:14.045396+0000","last_unstale":"2024-01-27T02:34:14.046424+0000","last_undegraded":"2024-01-27T02:34:14.046424+0000","last_fullsized":"2024-01-27T02:34:14.046424+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:37:32.098933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.832207+0000","last_change":"2024-01-27T02:34:14.832207+0000","last_active":"2024-01-27T02:34:14.832207+0000","last_peered":"2024-01-27T02:34:14.832207+0000","last_clean":"2024-01-27T02:34:14.832207+0000","last_became_active":"2024-01-27T02:34:14.831949+0000","last_became_peered":"2024-01-27T02:34:14.831949+0000","last_unstale":"2024-01-27T02:34:14.832207+0000","last_undegraded":"2024-01-27T02:34:14.832207+0000","last_fullsized":"2024-01-27T02:34:14.832207+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:58:33.213596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"64'2","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.063302+0000","last_change":"2024-01-27T02:34:14.046458+0000","last_active":"2024-01-27T02:34:15.063302+0000","last_peered":"2024-01-27T02:34:15.063302+0000","last_clean":"2024-01-27T02:34:15.063302+0000","last_became_active":"2024-01-27T02:34:14.046165+0000","last_became_peered":"2024-01-27T02:34:14.046165+0000","last_unstale":"2024-01-27T02:34:15.063302+0000","last_undegraded":"2024-01-27T02:34:15.063302+0000","last_fullsized":"2024-01-27T02:34:15.063302+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:03:32.931628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.827960+0000","last_change":"2024-01-27T02:34:14.827960+0000","last_active":"2024-01-27T02:34:14.827960+0000","last_peered":"2024-01-27T02:34:14.827960+0000","last_clean":"2024-01-27T02:34:14.827960+0000","last_became_active":"2024-01-27T02:34:14.827141+0000","last_became_peered":"2024-01-27T02:34:14.827141+0000","last_unstale":"2024-01-27T02:34:14.827960+0000","last_undegraded":"2024-01-27T02:34:14.827960+0000","last_fullsized":"2024-01-27T02:34:14.827960+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:44:29.126424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.048236+0000","last_change":"2024-01-27T02:34:14.048043+0000","last_active":"2024-01-27T02:34:14.048236+0000","last_peered":"2024-01-27T02:34:14.048236+0000","last_clean":"2024-01-27T02:34:14.048236+0000","last_became_active":"2024-01-27T02:34:14.047691+0000","last_became_peered":"2024-01-27T02:34:14.047691+0000","last_unstale":"2024-01-27T02:34:14.048236+0000","last_undegraded":"2024-01-27T02:34:14.048236+0000","last_fullsized":"2024-01-27T02:34:14.048236+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:31:10.895520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.835543+0000","last_change":"2024-01-27T02:34:14.835543+0000","last_active":"2024-01-27T02:34:14.835543+0000","last_peered":"2024-01-27T02:34:14.835543+0000","last_clean":"2024-01-27T02:34:14.835543+0000","last_became_active":"2024-01-27T02:34:14.835281+0000","last_became_peered":"2024-01-27T02:34:14.835281+0000","last_unstale":"2024-01-27T02:34:14.835543+0000","last_undegraded":"2024-01-27T02:34:14.835543+0000","last_fullsized":"2024-01-27T02:34:14.835543+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:35:26.921373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.046139+0000","last_change":"2024-01-27T02:34:14.043434+0000","last_active":"2024-01-27T02:34:14.046139+0000","last_peered":"2024-01-27T02:34:14.046139+0000","last_clean":"2024-01-27T02:34:14.046139+0000","last_became_active":"2024-01-27T02:34:14.043174+0000","last_became_peered":"2024-01-27T02:34:14.043174+0000","last_unstale":"2024-01-27T02:34:14.046139+0000","last_undegraded":"2024-01-27T02:34:14.046139+0000","last_fullsized":"2024-01-27T02:34:14.046139+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:37:32.098933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.831862+0000","last_change":"2024-01-27T02:34:14.831862+0000","last_active":"2024-01-27T02:34:14.831862+0000","last_peered":"2024-01-27T02:34:14.831862+0000","last_clean":"2024-01-27T02:34:14.831862+0000","last_became_active":"2024-01-27T02:34:14.831444+0000","last_became_peered":"2024-01-27T02:34:14.831444+0000","last_unstale":"2024-01-27T02:34:14.831862+0000","last_undegraded":"2024-01-27T02:34:14.831862+0000","last_fullsized":"2024-01-27T02:34:14.831862+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:02:25.313682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"64'2","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:15.063617+0000","last_change":"2024-01-27T02:34:14.051497+0000","last_active":"2024-01-27T02:34:15.063617+0000","last_peered":"2024-01-27T02:34:15.063617+0000","last_clean":"2024-01-27T02:34:15.063617+0000","last_became_active":"2024-01-27T02:34:14.051241+0000","last_became_peered":"2024-01-27T02:34:14.051241+0000","last_unstale":"2024-01-27T02:34:15.063617+0000","last_undegraded":"2024-01-27T02:34:15.063617+0000","last_fullsized":"2024-01-27T02:34:15.063617+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:44:04.272285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.826511+0000","last_change":"2024-01-27T02:34:14.826511+0000","last_active":"2024-01-27T02:34:14.826511+0000","last_peered":"2024-01-27T02:34:14.826511+0000","last_clean":"2024-01-27T02:34:14.826511+0000","last_became_active":"2024-01-27T02:34:14.826082+0000","last_became_peered":"2024-01-27T02:34:14.826082+0000","last_unstale":"2024-01-27T02:34:14.826511+0000","last_undegraded":"2024-01-27T02:34:14.826511+0000","last_fullsized":"2024-01-27T02:34:14.826511+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:13:57.675645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.042629+0000","last_change":"2024-01-27T02:34:14.042366+0000","last_active":"2024-01-27T02:34:14.042629+0000","last_peered":"2024-01-27T02:34:14.042629+0000","last_clean":"2024-01-27T02:34:14.042629+0000","last_became_active":"2024-01-27T02:34:14.041847+0000","last_became_peered":"2024-01-27T02:34:14.041847+0000","last_unstale":"2024-01-27T02:34:14.042629+0000","last_undegraded":"2024-01-27T02:34:14.042629+0000","last_fullsized":"2024-01-27T02:34:14.042629+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:33:19.626854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.833677+0000","last_change":"2024-01-27T02:34:14.833677+0000","last_active":"2024-01-27T02:34:14.833677+0000","last_peered":"2024-01-27T02:34:14.833677+0000","last_clean":"2024-01-27T02:34:14.833677+0000","last_became_active":"2024-01-27T02:34:14.833026+0000","last_became_peered":"2024-01-27T02:34:14.833026+0000","last_unstale":"2024-01-27T02:34:14.833677+0000","last_undegraded":"2024-01-27T02:34:14.833677+0000","last_fullsized":"2024-01-27T02:34:14.833677+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:35:50.281851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.042591+0000","last_change":"2024-01-27T02:34:14.042532+0000","last_active":"2024-01-27T02:34:14.042591+0000","last_peered":"2024-01-27T02:34:14.042591+0000","last_clean":"2024-01-27T02:34:14.042591+0000","last_became_active":"2024-01-27T02:34:14.042302+0000","last_became_peered":"2024-01-27T02:34:14.042302+0000","last_unstale":"2024-01-27T02:34:14.042591+0000","last_undegraded":"2024-01-27T02:34:14.042591+0000","last_fullsized":"2024-01-27T02:34:14.042591+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:45:13.750457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.826738+0000","last_change":"2024-01-27T02:34:14.826738+0000","last_active":"2024-01-27T02:34:14.826738+0000","last_peered":"2024-01-27T02:34:14.826738+0000","last_clean":"2024-01-27T02:34:14.826738+0000","last_became_active":"2024-01-27T02:34:14.826236+0000","last_became_peered":"2024-01-27T02:34:14.826236+0000","last_unstale":"2024-01-27T02:34:14.826738+0000","last_undegraded":"2024-01-27T02:34:14.826738+0000","last_fullsized":"2024-01-27T02:34:14.826738+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:53:19.368477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.046158+0000","last_change":"2024-01-27T02:34:14.043678+0000","last_active":"2024-01-27T02:34:14.046158+0000","last_peered":"2024-01-27T02:34:14.046158+0000","last_clean":"2024-01-27T02:34:14.046158+0000","last_became_active":"2024-01-27T02:34:14.043451+0000","last_became_peered":"2024-01-27T02:34:14.043451+0000","last_unstale":"2024-01-27T02:34:14.046158+0000","last_undegraded":"2024-01-27T02:34:14.046158+0000","last_fullsized":"2024-01-27T02:34:14.046158+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:38:05.609498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.829004+0000","last_change":"2024-01-27T02:34:14.826838+0000","last_active":"2024-01-27T02:34:14.829004+0000","last_peered":"2024-01-27T02:34:14.829004+0000","last_clean":"2024-01-27T02:34:14.829004+0000","last_became_active":"2024-01-27T02:34:14.826571+0000","last_became_peered":"2024-01-27T02:34:14.826571+0000","last_unstale":"2024-01-27T02:34:14.829004+0000","last_undegraded":"2024-01-27T02:34:14.829004+0000","last_fullsized":"2024-01-27T02:34:14.829004+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:58:27.917764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.044312+0000","last_change":"2024-01-27T02:34:14.044223+0000","last_active":"2024-01-27T02:34:14.044312+0000","last_peered":"2024-01-27T02:34:14.044312+0000","last_clean":"2024-01-27T02:34:14.044312+0000","last_became_active":"2024-01-27T02:34:14.043938+0000","last_became_peered":"2024-01-27T02:34:14.043938+0000","last_unstale":"2024-01-27T02:34:14.044312+0000","last_undegraded":"2024-01-27T02:34:14.044312+0000","last_fullsized":"2024-01-27T02:34:14.044312+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:38:59.335007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.833150+0000","last_change":"2024-01-27T02:34:14.833090+0000","last_active":"2024-01-27T02:34:14.833150+0000","last_peered":"2024-01-27T02:34:14.833150+0000","last_clean":"2024-01-27T02:34:14.833150+0000","last_became_active":"2024-01-27T02:34:14.832225+0000","last_became_peered":"2024-01-27T02:34:14.832225+0000","last_unstale":"2024-01-27T02:34:14.833150+0000","last_undegraded":"2024-01-27T02:34:14.833150+0000","last_fullsized":"2024-01-27T02:34:14.833150+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:22:37.496411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.819436+0000","last_change":"2024-01-27T02:34:14.047656+0000","last_active":"2024-01-27T02:34:14.819436+0000","last_peered":"2024-01-27T02:34:14.819436+0000","last_clean":"2024-01-27T02:34:14.819436+0000","last_became_active":"2024-01-27T02:34:14.047384+0000","last_became_peered":"2024-01-27T02:34:14.047384+0000","last_unstale":"2024-01-27T02:34:14.819436+0000","last_undegraded":"2024-01-27T02:34:14.819436+0000","last_fullsized":"2024-01-27T02:34:14.819436+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:10:25.112962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.834694+0000","last_change":"2024-01-27T02:34:14.834694+0000","last_active":"2024-01-27T02:34:14.834694+0000","last_peered":"2024-01-27T02:34:14.834694+0000","last_clean":"2024-01-27T02:34:14.834694+0000","last_became_active":"2024-01-27T02:34:14.834220+0000","last_became_peered":"2024-01-27T02:34:14.834220+0000","last_unstale":"2024-01-27T02:34:14.834694+0000","last_undegraded":"2024-01-27T02:34:14.834694+0000","last_fullsized":"2024-01-27T02:34:14.834694+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:03:29.484017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.046467+0000","last_change":"2024-01-27T02:34:14.046346+0000","last_active":"2024-01-27T02:34:14.046467+0000","last_peered":"2024-01-27T02:34:14.046467+0000","last_clean":"2024-01-27T02:34:14.046467+0000","last_became_active":"2024-01-27T02:34:14.045803+0000","last_became_peered":"2024-01-27T02:34:14.045803+0000","last_unstale":"2024-01-27T02:34:14.046467+0000","last_undegraded":"2024-01-27T02:34:14.046467+0000","last_fullsized":"2024-01-27T02:34:14.046467+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:45:45.390108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.828497+0000","last_change":"2024-01-27T02:34:14.825239+0000","last_active":"2024-01-27T02:34:14.828497+0000","last_peered":"2024-01-27T02:34:14.828497+0000","last_clean":"2024-01-27T02:34:14.828497+0000","last_became_active":"2024-01-27T02:34:14.824940+0000","last_became_peered":"2024-01-27T02:34:14.824940+0000","last_unstale":"2024-01-27T02:34:14.828497+0000","last_undegraded":"2024-01-27T02:34:14.828497+0000","last_fullsized":"2024-01-27T02:34:14.828497+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:46:48.013304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.819108+0000","last_change":"2024-01-27T02:34:14.049093+0000","last_active":"2024-01-27T02:34:14.819108+0000","last_peered":"2024-01-27T02:34:14.819108+0000","last_clean":"2024-01-27T02:34:14.819108+0000","last_became_active":"2024-01-27T02:34:14.048624+0000","last_became_peered":"2024-01-27T02:34:14.048624+0000","last_unstale":"2024-01-27T02:34:14.819108+0000","last_undegraded":"2024-01-27T02:34:14.819108+0000","last_fullsized":"2024-01-27T02:34:14.819108+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:22.801389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.824286+0000","last_change":"2024-01-27T02:34:14.823216+0000","last_active":"2024-01-27T02:34:14.824286+0000","last_peered":"2024-01-27T02:34:14.824286+0000","last_clean":"2024-01-27T02:34:14.824286+0000","last_became_active":"2024-01-27T02:34:14.822948+0000","last_became_peered":"2024-01-27T02:34:14.822948+0000","last_unstale":"2024-01-27T02:34:14.824286+0000","last_undegraded":"2024-01-27T02:34:14.824286+0000","last_fullsized":"2024-01-27T02:34:14.824286+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:14.030048+0000","last_clean_scrub_stamp":"2024-01-27T02:34:14.030048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:12:49.632475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.044937+0000","last_change":"2024-01-27T02:34:14.044095+0000","last_active":"2024-01-27T02:34:14.044937+0000","last_peered":"2024-01-27T02:34:14.044937+0000","last_clean":"2024-01-27T02:34:14.044937+0000","last_became_active":"2024-01-27T02:34:14.043773+0000","last_became_peered":"2024-01-27T02:34:14.043773+0000","last_unstale":"2024-01-27T02:34:14.044937+0000","last_undegraded":"2024-01-27T02:34:14.044937+0000","last_fullsized":"2024-01-27T02:34:14.044937+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:34:13.028146+0000","last_clean_scrub_stamp":"2024-01-27T02:34:13.028146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:44:04.272285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":199,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-27T02:34:14.823745+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:34:14.823745+0000","last_peered":"2024-01-27T02:34:14.823745+0000","last_clean":"2024-01-27T02:34:14.823745+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:34:14.823745+0000","last_undegraded":"2024-01-27T02:34:14.823745+0000","last_fullsized":"2024-01-27T02:34:14.823745+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":894,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194418,"statfs":{"total":200000000,"available":199084390,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.41899999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58499999999999996,"15min":0.58799999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.67300000000000004}]},{"osd":1,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59499999999999997,"15min":0.59699999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.624},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55100000000000005,"15min":0.55000000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.57499999999999996}]},{"osd":2,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58099999999999996,"15min":0.57999999999999996},"min":{"1min":0.45600000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58499999999999996,"15min":0.58099999999999996},"min":{"1min":0.44900000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.71499999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.55200000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.48199999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.69199999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.76300000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.50800000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.496,"15min":0.49399999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.49399999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.32100000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.63900000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80300000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.752,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.439}]}]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":317,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194995,"statfs":{"total":200000000,"available":199674918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.51400000000000001,"15min":0.497},"min":{"1min":0.41899999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.52200000000000002,"15min":0.50800000000000001},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.53200000000000003}]},{"osd":1,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.51000000000000001,"15min":0.49399999999999999},"min":{"1min":0.48799999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.54800000000000004,"15min":0.53600000000000003},"min":{"1min":0.39900000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.57399999999999995}]},{"osd":2,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.41599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.441,"5min":0.438,"15min":0.438},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.69299999999999995}]},{"osd":3,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.54700000000000004,"15min":0.53900000000000003},"min":{"1min":0.435,"5min":0.376,"15min":0.376},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.436,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.64800000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.39500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.60699999999999998,"5min":0.748,"15min":0.748},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55600000000000005,"15min":0.56200000000000006},"min":{"1min":0.379,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70499999999999996,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.627}]},{"osd":5,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.67900000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.60599999999999998,"15min":0.61499999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.72299999999999998}]},{"osd":7,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.61899999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.754,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.746},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69999999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.73399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":317,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194994,"statfs":{"total":200000000,"available":199674828,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.371,"15min":0.36799999999999999},"min":{"1min":0.28199999999999997,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.30199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.42099999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.36199999999999999,"15min":0.35299999999999998},"min":{"1min":0.32200000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.42299999999999999,"15min":0.42099999999999999},"min":{"1min":0.32200000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.60099999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.45000000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.46400000000000002,"15min":0.46000000000000002},"min":{"1min":0.36299999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69299999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.38500000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.621,"15min":0.621},"min":{"1min":0.46300000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.60099999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.46200000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66200000000000003,"15min":0.66600000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.629}]},{"osd":6,"last update":"Sat Jan 27 02:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66600000000000004,"15min":0.66400000000000003},"min":{"1min":0.52500000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.64500000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.505,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68000000000000005,"15min":0.67700000000000005},"min":{"1min":0.58299999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.67700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":896,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194416,"statfs":{"total":200000000,"available":199082460,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.27600000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.54600000000000004,"5min":0.626,"15min":0.626},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.40899999999999997,"15min":0.41299999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.54400000000000004,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.42299999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.41799999999999998,"15min":0.41599999999999998},"min":{"1min":0.36099999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.34699999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.63100000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.36799999999999999}]},{"osd":3,"last update":"Sat Jan 27 02:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.42399999999999999,"15min":0.42999999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.50700000000000001,"5min":0.625,"15min":0.625},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.46400000000000002,"15min":0.46800000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.60199999999999998,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.45700000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.81699999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.65300000000000002,"15min":0.65700000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.76300000000000001}]},{"osd":5,"last update":"Sat Jan 27 02:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.747,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67400000000000004,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.84599999999999997,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.69099999999999995}]},{"osd":6,"last update":"Sat Jan 27 02:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.65500000000000003,"15min":0.66000000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.83099999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.625}]},{"osd":7,"last update":"Sat Jan 27 02:34:02 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.68600000000000005,"15min":0.69099999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.78200000000000003,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.68300000000000005,"15min":0.69099999999999995},"min":{"1min":0.50800000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.80900000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.70099999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":317,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194994,"statfs":{"total":200000000,"available":199674862,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.40400000000000003,"15min":0.40600000000000003},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.64700000000000002,"15min":0.68300000000000005},"min":{"1min":0.34300000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.64500000000000002,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":0.47799999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.41599999999999998,"15min":0.41799999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.54400000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.443,"15min":0.44800000000000001},"min":{"1min":0.308,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.51000000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.38900000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.40799999999999997,"15min":0.40999999999999998},"min":{"1min":0.29299999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.48399999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.48199999999999998,"15min":0.48399999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.64800000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.46100000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64600000000000002,"15min":0.64400000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88800000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.80500000000000005,"15min":0.82299999999999995},"min":{"1min":0.56799999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.96499999999999997,"5min":3.581,"15min":3.581},"last":0.82299999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.746,"15min":0.76600000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88400000000000001,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67100000000000004,"15min":0.66900000000000004},"min":{"1min":0.51600000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96599999999999997,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.80200000000000005}]},{"osd":6,"last update":"Sat Jan 27 02:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.82699999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.875},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84099999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.78800000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67100000000000004,"15min":0.67400000000000004},"min":{"1min":0.52500000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.76600000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.68400000000000005,"15min":0.68500000000000005},"min":{"1min":0.46800000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73099999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":323,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194988,"statfs":{"total":200000000,"available":199668574,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.35899999999999999,"15min":0.34899999999999998},"min":{"1min":0.28899999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.42999999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.38700000000000001,"15min":0.38400000000000001},"min":{"1min":0.30299999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.56100000000000005,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.33400000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.52000000000000002,"5min":0.628,"15min":0.628},"last":0.443}]},{"osd":2,"last update":"Sat Jan 27 02:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.42099999999999999,"15min":0.41799999999999998},"min":{"1min":0.34399999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.53400000000000003,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.443,"15min":0.437},"min":{"1min":0.36699999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.39700000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79500000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.49099999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.79100000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.45100000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66400000000000003,"15min":0.66300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.81299999999999994,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.69999999999999996}]},{"osd":6,"last update":"Sat Jan 27 02:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.53000000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.78200000000000003,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.751},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.48499999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.81899999999999995,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.64100000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:34:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.628,"15min":0.624},"min":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.80400000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65600000000000003,"15min":0.65700000000000003},"min":{"1min":0.496,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.77000000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":325,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194987,"statfs":{"total":200000000,"available":199667094,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58399999999999996,"15min":0.58699999999999997},"min":{"1min":0.33400000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.98199999999999998,"5min":1.099,"15min":1.099},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.129,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.56399999999999995}]},{"osd":1,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.59799999999999998,"15min":0.59199999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81499999999999995,"5min":1.133,"15min":1.133},"last":0.54300000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.45300000000000001,"5min":0.434,"15min":0.434},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.437,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0009999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.58699999999999997}]},{"osd":3,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79600000000000004,"5min":1.115,"15min":1.115},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.5,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.2839999999999998,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.57699999999999996}]},{"osd":5,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48599999999999999,"15min":0.48699999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.495,"15min":0.49199999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72899999999999998,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.42699999999999999}]},{"osd":6,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.50700000000000001,"15min":0.501},"min":{"1min":0.34899999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.48399999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:34:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.496},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.40999999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.44600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":318,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194994,"statfs":{"total":200000000,"available":199674294,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.60499999999999998,"15min":0.57799999999999996},"min":{"1min":0.40999999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.242,"5min":3.242,"15min":3.242},"last":0.47199999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.61599999999999999,"15min":0.60499999999999998},"min":{"1min":0.46000000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65800000000000003,"15min":0.66000000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.88500000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.55500000000000005}]},{"osd":2,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.62,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":3.04,"5min":3.04,"15min":3.04},"last":0.5},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.85599999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.52100000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.71499999999999997,"15min":0.69499999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":4.032,"5min":4.032,"15min":4.032},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.71199999999999997,"15min":0.68100000000000005},"min":{"1min":0.51300000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":4.282,"5min":4.282,"15min":4.282},"last":0.69899999999999995}]},{"osd":4,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53200000000000003,"15min":0.52900000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.499,"5min":0.57299999999999995,"15min":0.58499999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73099999999999998,"5min":1.101,"15min":1.101},"last":0.68100000000000005}]},{"osd":6,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.56699999999999995,"15min":0.57399999999999995},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59499999999999997,"15min":0.59199999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.72099999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91100000000000003,"5min":1.008,"15min":1.008},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2,"5min":2,"15min":2},"last":0.67200000000000004}]}]}],"pool_statfs":[]}} 2024-01-27T02:34:19.652 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-27T02:34:19.652 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-27T02:34:19.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.828+0000 7f8c3560e700 1 Processor -- start 2024-01-27T02:34:19.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.829+0000 7f8c3560e700 1 -- start start 2024-01-27T02:34:19.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.829+0000 7f8c3560e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8c30154550 0x7f8c300ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:19.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.829+0000 7f8c3560e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8c30154920 0x7f8c300ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:19.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.829+0000 7f8c3560e700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c300adec0 con 0x7f8c3015b050 2024-01-27T02:34:19.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.829+0000 7f8c3560e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c300ae1a0 con 0x7f8c30154920 2024-01-27T02:34:19.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.829+0000 7f8c3560e700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c3015a850 con 0x7f8c30154550 2024-01-27T02:34:19.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.830+0000 7f8c2e7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8c30154920 0x7f8c300ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:19.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.830+0000 7f8c2e7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8c30154920 0x7f8c300ad7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52866/0 (socket says 172.21.15.7:52866) 2024-01-27T02:34:19.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.830+0000 7f8c2e7fc700 1 -- 172.21.15.7:0/3901821546 learned_addr learned my addr 172.21.15.7:0/3901821546 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:19.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.830+0000 7f8c2effd700 1 --2- 172.21.15.7:0/3901821546 >> v2:172.21.15.188:3300/0 conn(0x7f8c30154550 0x7f8c300ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:19.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.830+0000 7f8c2e7fc700 1 -- 172.21.15.7:0/3901821546 >> v2:172.21.15.188:3300/0 conn(0x7f8c30154550 msgr2=0x7f8c300ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:19.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.830+0000 7f8c2e7fc700 1 --2- 172.21.15.7:0/3901821546 >> v2:172.21.15.188:3300/0 conn(0x7f8c30154550 0x7f8c300ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:19.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.830+0000 7f8c2e7fc700 1 -- 172.21.15.7:0/3901821546 >> v1:172.21.15.188:6789/0 conn(0x7f8c3015b050 legacy=0x7f8c30063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:19.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.830+0000 7f8c2e7fc700 1 -- 172.21.15.7:0/3901821546 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3015ab30 con 0x7f8c30154920 2024-01-27T02:34:19.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.830+0000 7f8c2e7fc700 1 --2- 172.21.15.7:0/3901821546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c30154920 0x7f8c300ad7c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8c30068f50 tx=0x7f8c20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51381d6d3185aac server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:19.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.830+0000 7f8c2dffb700 1 -- 172.21.15.7:0/3901821546 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c20009e70 con 0x7f8c30154920 2024-01-27T02:34:19.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.830+0000 7f8c2dffb700 1 -- 172.21.15.7:0/3901821546 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2000b860 con 0x7f8c30154920 2024-01-27T02:34:19.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.830+0000 7f8c2dffb700 1 -- 172.21.15.7:0/3901821546 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2000bc00 con 0x7f8c30154920 2024-01-27T02:34:19.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.831+0000 7f8c3560e700 1 -- 172.21.15.7:0/3901821546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c30154920 msgr2=0x7f8c300ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:19.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.831+0000 7f8c3560e700 1 --2- 172.21.15.7:0/3901821546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c30154920 0x7f8c300ad7c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8c30068f50 tx=0x7f8c20009250 comp rx=0 tx=0).stop 2024-01-27T02:34:19.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.831+0000 7f8c3560e700 1 -- 172.21.15.7:0/3901821546 shutdown_connections 2024-01-27T02:34:19.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.831+0000 7f8c3560e700 1 --2- 172.21.15.7:0/3901821546 >> v2:172.21.15.188:3300/0 conn(0x7f8c30154550 0x7f8c300ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:19.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.831+0000 7f8c3560e700 1 --2- 172.21.15.7:0/3901821546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c30154920 0x7f8c300ad7c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:19.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.831+0000 7f8c3560e700 1 -- 172.21.15.7:0/3901821546 >> 172.21.15.7:0/3901821546 conn(0x7f8c30068480 msgr2=0x7f8c300cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:19.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.831+0000 7f8c3560e700 1 -- 172.21.15.7:0/3901821546 shutdown_connections 2024-01-27T02:34:19.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.831+0000 7f8c3560e700 1 -- 172.21.15.7:0/3901821546 wait complete. 2024-01-27T02:34:19.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.832+0000 7f8c3560e700 1 Processor -- start 2024-01-27T02:34:19.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.833+0000 7f8c3560e700 1 -- start start 2024-01-27T02:34:19.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.833+0000 7f8c3560e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c30154550 0x7f8c3015d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:19.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.833+0000 7f8c3560e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c30154920 0x7f8c301526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:19.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.833+0000 7f8c3560e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c3015b050 0x7f8c3015f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:19.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.833+0000 7f8c3560e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3015be30 con 0x7f8c30154550 2024-01-27T02:34:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.833+0000 7f8c3560e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c30152c20 con 0x7f8c30154920 2024-01-27T02:34:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.833+0000 7f8c3560e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c301602a0 con 0x7f8c3015b050 2024-01-27T02:34:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.833+0000 7f8c2effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c30154550 0x7f8c3015d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.833+0000 7f8c2effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c30154550 0x7f8c3015d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52870/0 (socket says 172.21.15.7:52870) 2024-01-27T02:34:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.833+0000 7f8c2effd700 1 -- 172.21.15.7:0/790295635 learned_addr learned my addr 172.21.15.7:0/790295635 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.834+0000 7f8c2e7fc700 1 --2- 172.21.15.7:0/790295635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c30154920 0x7f8c301526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.834+0000 7f8c2f7fe700 1 --2- 172.21.15.7:0/790295635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c3015b050 0x7f8c3015f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.834+0000 7f8c2effd700 1 -- 172.21.15.7:0/790295635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c3015b050 msgr2=0x7f8c3015f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.834+0000 7f8c2effd700 1 --2- 172.21.15.7:0/790295635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c3015b050 0x7f8c3015f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.834+0000 7f8c2effd700 1 -- 172.21.15.7:0/790295635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c30154920 msgr2=0x7f8c301526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.834+0000 7f8c2effd700 1 --2- 172.21.15.7:0/790295635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c30154920 0x7f8c301526e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.834+0000 7f8c2effd700 1 -- 172.21.15.7:0/790295635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c200094f0 con 0x7f8c30154550 2024-01-27T02:34:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.834+0000 7f8c2effd700 1 --2- 172.21.15.7:0/790295635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c30154550 0x7f8c3015d700 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800ec80 tx=0x7f8c180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.834+0000 7f8c26ffd700 1 -- 172.21.15.7:0/790295635 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1800c850 con 0x7f8c30154550 2024-01-27T02:34:19.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.834+0000 7f8c26ffd700 1 -- 172.21.15.7:0/790295635 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1800c9b0 con 0x7f8c30154550 2024-01-27T02:34:19.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.834+0000 7f8c3560e700 1 -- 172.21.15.7:0/790295635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3015b500 con 0x7f8c30154550 2024-01-27T02:34:19.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.834+0000 7f8c26ffd700 1 -- 172.21.15.7:0/790295635 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1800cd50 con 0x7f8c30154550 2024-01-27T02:34:19.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.834+0000 7f8c3560e700 1 -- 172.21.15.7:0/790295635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c3015bac0 con 0x7f8c30154550 2024-01-27T02:34:19.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.835+0000 7f8c3560e700 1 -- 172.21.15.7:0/790295635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c30072060 con 0x7f8c30154550 2024-01-27T02:34:19.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.836+0000 7f8c26ffd700 1 -- 172.21.15.7:0/790295635 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8c18013680 con 0x7f8c30154550 2024-01-27T02:34:19.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.836+0000 7f8c26ffd700 1 --2- 172.21.15.7:0/790295635 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8c10066f30 0x7f8c100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:19.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.836+0000 7f8c26ffd700 1 -- 172.21.15.7:0/790295635 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8c18084760 con 0x7f8c30154550 2024-01-27T02:34:19.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.836+0000 7f8c2e7fc700 1 --2- 172.21.15.7:0/790295635 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8c10066f30 0x7f8c100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:19.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.837+0000 7f8c2e7fc700 1 --2- 172.21.15.7:0/790295635 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8c10066f30 0x7f8c100693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8c3006e330 tx=0x7f8c20017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:19.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:19.841+0000 7f8c26ffd700 1 -- 172.21.15.7:0/790295635 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c180519d0 con 0x7f8c30154550 2024-01-27T02:34:20.029 INFO:tasks.ceph.mds.a.smithi007.stderr:2024-01-27T02:34:20.027+0000 7f4d3ee48700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-27T02:34:20.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.131+0000 7f8c3560e700 1 -- 172.21.15.7:0/790295635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8c3015c110 con 0x7f8c30154550 2024-01-27T02:34:20.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.135+0000 7f8c26ffd700 1 -- 172.21.15.7:0/790295635 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7f8c18054ff0 con 0x7f8c30154550 2024-01-27T02:34:20.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.137+0000 7f8c3560e700 1 -- 172.21.15.7:0/790295635 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8c10066f30 msgr2=0x7f8c100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:20.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.137+0000 7f8c3560e700 1 --2- 172.21.15.7:0/790295635 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8c10066f30 0x7f8c100693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8c3006e330 tx=0x7f8c20017000 comp rx=0 tx=0).stop 2024-01-27T02:34:20.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.137+0000 7f8c3560e700 1 -- 172.21.15.7:0/790295635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c30154550 msgr2=0x7f8c3015d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:20.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.137+0000 7f8c3560e700 1 --2- 172.21.15.7:0/790295635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c30154550 0x7f8c3015d700 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800ec80 tx=0x7f8c180091a0 comp rx=0 tx=0).stop 2024-01-27T02:34:20.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.137+0000 7f8c3560e700 1 -- 172.21.15.7:0/790295635 shutdown_connections 2024-01-27T02:34:20.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.137+0000 7f8c3560e700 1 --2- 172.21.15.7:0/790295635 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8c10066f30 0x7f8c100693e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:20.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.137+0000 7f8c3560e700 1 --2- 172.21.15.7:0/790295635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c3015b050 0x7f8c3015f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:20.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.137+0000 7f8c3560e700 1 --2- 172.21.15.7:0/790295635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c30154550 0x7f8c3015d700 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:20.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.137+0000 7f8c3560e700 1 --2- 172.21.15.7:0/790295635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c30154920 0x7f8c301526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:20.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.137+0000 7f8c3560e700 1 -- 172.21.15.7:0/790295635 >> 172.21.15.7:0/790295635 conn(0x7f8c30068480 msgr2=0x7f8c30159850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:20.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.137+0000 7f8c3560e700 1 -- 172.21.15.7:0/790295635 shutdown_connections 2024-01-27T02:34:20.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.137+0000 7f8c3560e700 1 -- 172.21.15.7:0/790295635 wait complete. 2024-01-27T02:34:20.140 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-01-27T02:34:20.153 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:34:20.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.283+0000 7fb059b6f700 1 Processor -- start 2024-01-27T02:34:20.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.283+0000 7fb059b6f700 1 -- start start 2024-01-27T02:34:20.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.284+0000 7fb059b6f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb05415a100 0x7fb0540cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:20.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.284+0000 7fb059b6f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0540ce5e0 con 0x7fb0540cb5b0 2024-01-27T02:34:20.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.284+0000 7fb059b6f700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0540cd9d0 con 0x7fb0540c4cc0 2024-01-27T02:34:20.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.284+0000 7fb059b6f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0540cdcb0 con 0x7fb05415a100 2024-01-27T02:34:20.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.284+0000 7fb052ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb0540cb5b0 0x7fb0540c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42220/0 (socket says 172.21.15.7:42220) 2024-01-27T02:34:20.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.284+0000 7fb052ffd700 1 -- 172.21.15.7:0/3339390763 learned_addr learned my addr 172.21.15.7:0/3339390763 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:34:20.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.284+0000 7fb0537fe700 1 --2- 172.21.15.7:0/3339390763 >> v2:172.21.15.188:3300/0 conn(0x7fb05415a100 0x7fb0540cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:20.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.284+0000 7fb0527fc700 1 -- 172.21.15.7:0/3339390763 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3561576200 0 0) 0x7fb0540ce5e0 con 0x7fb0540cb5b0 2024-01-27T02:34:20.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.284+0000 7fb0527fc700 1 -- 172.21.15.7:0/3339390763 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb03c003570 con 0x7fb0540cb5b0 2024-01-27T02:34:20.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.285+0000 7fb0527fc700 1 -- 172.21.15.7:0/3339390763 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596383247 0 0) 0x7fb0540cd9d0 con 0x7fb0540c4cc0 2024-01-27T02:34:20.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.285+0000 7fb0527fc700 1 -- 172.21.15.7:0/3339390763 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0540ce5e0 con 0x7fb0540c4cc0 2024-01-27T02:34:20.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.285+0000 7fb0537fe700 1 -- 172.21.15.7:0/3339390763 >> v1:172.21.15.188:6789/0 conn(0x7fb0540c4cc0 legacy=0x7fb054163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:20.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.285+0000 7fb0527fc700 1 -- 172.21.15.7:0/3339390763 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3958462841 0 0) 0x7fb03c003570 con 0x7fb0540cb5b0 2024-01-27T02:34:20.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.285+0000 7fb0537fe700 1 -- 172.21.15.7:0/3339390763 >> v1:172.21.15.7:6789/0 conn(0x7fb0540cb5b0 legacy=0x7fb0540c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:20.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.285+0000 7fb0537fe700 1 -- 172.21.15.7:0/3339390763 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0540cdf90 con 0x7fb05415a100 2024-01-27T02:34:20.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.285+0000 7fb0537fe700 1 --2- 172.21.15.7:0/3339390763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb05415a100 0x7fb0540cd2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb0440098e0 tx=0x7fb0440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94b2eed20bf0e2c6 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:20.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.285+0000 7fb0527fc700 1 -- 172.21.15.7:0/3339390763 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb048001470 con 0x7fb05415a100 2024-01-27T02:34:20.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.285+0000 7fb0527fc700 1 -- 172.21.15.7:0/3339390763 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb044011420 con 0x7fb05415a100 2024-01-27T02:34:20.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.285+0000 7fb0527fc700 1 -- 172.21.15.7:0/3339390763 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0440117c0 con 0x7fb05415a100 2024-01-27T02:34:20.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.286+0000 7fb059b6f700 1 -- 172.21.15.7:0/3339390763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb05415a100 msgr2=0x7fb0540cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:20.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.286+0000 7fb059b6f700 1 --2- 172.21.15.7:0/3339390763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb05415a100 0x7fb0540cd2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb0440098e0 tx=0x7fb0440091a0 comp rx=0 tx=0).stop 2024-01-27T02:34:20.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.286+0000 7fb059b6f700 1 -- 172.21.15.7:0/3339390763 shutdown_connections 2024-01-27T02:34:20.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.286+0000 7fb059b6f700 1 --2- 172.21.15.7:0/3339390763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb05415a100 0x7fb0540cd2f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:20.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.286+0000 7fb059b6f700 1 -- 172.21.15.7:0/3339390763 >> 172.21.15.7:0/3339390763 conn(0x7fb054098020 msgr2=0x7fb0540711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:20.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.286+0000 7fb059b6f700 1 -- 172.21.15.7:0/3339390763 shutdown_connections 2024-01-27T02:34:20.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.286+0000 7fb059b6f700 1 -- 172.21.15.7:0/3339390763 wait complete. 2024-01-27T02:34:20.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.287+0000 7fb059b6f700 1 Processor -- start 2024-01-27T02:34:20.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.287+0000 7fb059b6f700 1 -- start start 2024-01-27T02:34:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.288+0000 7fb059b6f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0540c4cc0 0x7fb05415ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.288+0000 7fb059b6f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0540cb5b0 0x7fb054154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.288+0000 7fb059b6f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb05415a100 0x7fb054068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.288+0000 7fb059b6f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb054154b40 con 0x7fb0540cb5b0 2024-01-27T02:34:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.288+0000 7fb059b6f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540b1f00 con 0x7fb0540c4cc0 2024-01-27T02:34:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.288+0000 7fb059b6f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540b21e0 con 0x7fb05415a100 2024-01-27T02:34:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.288+0000 7fb052ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0540cb5b0 0x7fb054154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.288+0000 7fb052ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0540cb5b0 0x7fb054154600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52878/0 (socket says 172.21.15.7:52878) 2024-01-27T02:34:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.288+0000 7fb052ffd700 1 -- 172.21.15.7:0/2314242231 learned_addr learned my addr 172.21.15.7:0/2314242231 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.288+0000 7fb0537fe700 1 --2- 172.21.15.7:0/2314242231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0540c4cc0 0x7fb05415ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.288+0000 7fb053fff700 1 --2- 172.21.15.7:0/2314242231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb05415a100 0x7fb054068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.288+0000 7fb052ffd700 1 -- 172.21.15.7:0/2314242231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb05415a100 msgr2=0x7fb054068990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.288+0000 7fb052ffd700 1 --2- 172.21.15.7:0/2314242231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb05415a100 0x7fb054068990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.288+0000 7fb052ffd700 1 -- 172.21.15.7:0/2314242231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0540c4cc0 msgr2=0x7fb05415ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.288+0000 7fb052ffd700 1 --2- 172.21.15.7:0/2314242231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0540c4cc0 0x7fb05415ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.289+0000 7fb052ffd700 1 -- 172.21.15.7:0/2314242231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb044009420 con 0x7fb0540cb5b0 2024-01-27T02:34:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.289+0000 7fb0537fe700 1 --2- 172.21.15.7:0/2314242231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0540c4cc0 0x7fb05415ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.289+0000 7fb053fff700 1 --2- 172.21.15.7:0/2314242231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb05415a100 0x7fb054068990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.289+0000 7fb052ffd700 1 --2- 172.21.15.7:0/2314242231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0540cb5b0 0x7fb054154600 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb04800cb70 tx=0x7fb048001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.289+0000 7fb03b7fe700 1 -- 172.21.15.7:0/2314242231 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb048001d00 con 0x7fb0540cb5b0 2024-01-27T02:34:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.289+0000 7fb03b7fe700 1 -- 172.21.15.7:0/2314242231 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb048007360 con 0x7fb0540cb5b0 2024-01-27T02:34:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.289+0000 7fb059b6f700 1 -- 172.21.15.7:0/2314242231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0540b24c0 con 0x7fb0540cb5b0 2024-01-27T02:34:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.289+0000 7fb03b7fe700 1 -- 172.21.15.7:0/2314242231 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb048004830 con 0x7fb0540cb5b0 2024-01-27T02:34:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.289+0000 7fb059b6f700 1 -- 172.21.15.7:0/2314242231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0540c6590 con 0x7fb0540cb5b0 2024-01-27T02:34:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.290+0000 7fb059b6f700 1 -- 172.21.15.7:0/2314242231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb05406d9f0 con 0x7fb0540cb5b0 2024-01-27T02:34:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.290+0000 7fb03b7fe700 1 -- 172.21.15.7:0/2314242231 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb048004ba0 con 0x7fb0540cb5b0 2024-01-27T02:34:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.291+0000 7fb03b7fe700 1 --2- 172.21.15.7:0/2314242231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb040066f30 0x7fb0400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.291+0000 7fb03b7fe700 1 -- 172.21.15.7:0/2314242231 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb048083e00 con 0x7fb0540cb5b0 2024-01-27T02:34:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.291+0000 7fb0537fe700 1 --2- 172.21.15.7:0/2314242231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb040066f30 0x7fb0400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.292+0000 7fb0537fe700 1 --2- 172.21.15.7:0/2314242231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb040066f30 0x7fb0400693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb0440093f0 tx=0x7fb044016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:20.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.295+0000 7fb03b7fe700 1 -- 172.21.15.7:0/2314242231 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0480510f0 con 0x7fb0540cb5b0 2024-01-27T02:34:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.548+0000 7fb059b6f700 1 -- 172.21.15.7:0/2314242231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb05404c710 con 0x7fb0540cb5b0 2024-01-27T02:34:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.548+0000 7fb03b7fe700 1 -- 172.21.15.7:0/2314242231 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4597 (secure 0 0 0) 0x7fb048018020 con 0x7fb0540cb5b0 2024-01-27T02:34:20.550 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:34:20.550 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5114,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/2839704447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":2839704447},{"type":"v1","addr":"172.21.15.188:6837","nonce":2839704447}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5117,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3964291855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3964291855},{"type":"v1","addr":"172.21.15.7:6834","nonce":3964291855}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/3332325272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3332325272},{"type":"v1","addr":"172.21.15.188:6836","nonce":3332325272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:34:14.471429+0000","modified":"2024-01-27T02:34:18.068858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1399055016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1399055016},{"type":"v1","addr":"172.21.15.7:6835","nonce":1399055016}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2024-01-27T02:34:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.550+0000 7fb059b6f700 1 -- 172.21.15.7:0/2314242231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb040066f30 msgr2=0x7fb0400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.550+0000 7fb059b6f700 1 --2- 172.21.15.7:0/2314242231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb040066f30 0x7fb0400693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb0440093f0 tx=0x7fb044016000 comp rx=0 tx=0).stop 2024-01-27T02:34:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.550+0000 7fb059b6f700 1 -- 172.21.15.7:0/2314242231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0540cb5b0 msgr2=0x7fb054154600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:20.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.550+0000 7fb059b6f700 1 --2- 172.21.15.7:0/2314242231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0540cb5b0 0x7fb054154600 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb04800cb70 tx=0x7fb048001ab0 comp rx=0 tx=0).stop 2024-01-27T02:34:20.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.551+0000 7fb059b6f700 1 -- 172.21.15.7:0/2314242231 shutdown_connections 2024-01-27T02:34:20.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.551+0000 7fb059b6f700 1 --2- 172.21.15.7:0/2314242231 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb040066f30 0x7fb0400693e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:20.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.551+0000 7fb059b6f700 1 --2- 172.21.15.7:0/2314242231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb05415a100 0x7fb054068990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:20.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.551+0000 7fb059b6f700 1 --2- 172.21.15.7:0/2314242231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0540c4cc0 0x7fb05415ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:20.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.551+0000 7fb059b6f700 1 --2- 172.21.15.7:0/2314242231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0540cb5b0 0x7fb054154600 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:20.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.551+0000 7fb059b6f700 1 -- 172.21.15.7:0/2314242231 >> 172.21.15.7:0/2314242231 conn(0x7fb054098020 msgr2=0x7fb054154f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:20.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.551+0000 7fb059b6f700 1 -- 172.21.15.7:0/2314242231 shutdown_connections 2024-01-27T02:34:20.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:20.551+0000 7fb059b6f700 1 -- 172.21.15.7:0/2314242231 wait complete. 2024-01-27T02:34:20.554 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2024-01-27T02:34:20.968 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-27T02:34:14.471429+0000', 'modified': '2024-01-27T02:34:18.068858+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5631}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5631': {'gid': 5631, 'name': 'a', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6835/1399055016', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6833', 'nonce': 1399055016}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 1399055016}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-27T02:34:20.969 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-27T02:34:20.969 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-27T02:34:21.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.109+0000 7f746ba2f700 1 Processor -- start 2024-01-27T02:34:21.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.110+0000 7f746ba2f700 1 -- start start 2024-01-27T02:34:21.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.111+0000 7f746ba2f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7464157ca0 0x7f74640c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.111+0000 7f746ba2f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74640c3ae0 0x7f74640cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.111+0000 7f746ba2f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74640ca020 con 0x7f7464158070 2024-01-27T02:34:21.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.111+0000 7f746ba2f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74640c4f40 con 0x7f7464157ca0 2024-01-27T02:34:21.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.111+0000 7f746ba2f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74640c5210 con 0x7f74640c3ae0 2024-01-27T02:34:21.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.111+0000 7f74697cb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7464157ca0 0x7f74640c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.111+0000 7f7469fcc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7464158070 0x7f746415bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42230/0 (socket says 172.21.15.7:42230) 2024-01-27T02:34:21.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.111+0000 7f74697cb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7464157ca0 0x7f74640c4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52888/0 (socket says 172.21.15.7:52888) 2024-01-27T02:34:21.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.111+0000 7f7469fcc700 1 -- 172.21.15.7:0/2922831545 learned_addr learned my addr 172.21.15.7:0/2922831545 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:34:21.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.111+0000 7f7468fca700 1 --2- 172.21.15.7:0/2922831545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74640c3ae0 0x7f74640cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.111+0000 7f74697cb700 1 -- 172.21.15.7:0/2922831545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74640c3ae0 msgr2=0x7f74640cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:21.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.111+0000 7f74697cb700 1 --2- 172.21.15.7:0/2922831545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74640c3ae0 0x7f74640cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.111+0000 7f74697cb700 1 -- 172.21.15.7:0/2922831545 >> v1:172.21.15.7:6789/0 conn(0x7f7464158070 legacy=0x7f746415bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:21.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.111+0000 7f74697cb700 1 -- 172.21.15.7:0/2922831545 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74640c54f0 con 0x7f7464157ca0 2024-01-27T02:34:21.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.111+0000 7f74697cb700 1 --2- 172.21.15.7:0/2922831545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7464157ca0 0x7f74640c4a00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f74600098e0 tx=0x7f74600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae41a189f4cd110 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:21.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.111+0000 7f745bfff700 1 -- 172.21.15.7:0/2922831545 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7460009e80 con 0x7f7464157ca0 2024-01-27T02:34:21.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.111+0000 7f745bfff700 1 -- 172.21.15.7:0/2922831545 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746000b860 con 0x7f7464157ca0 2024-01-27T02:34:21.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.112+0000 7f745bfff700 1 -- 172.21.15.7:0/2922831545 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746000bbe0 con 0x7f7464157ca0 2024-01-27T02:34:21.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.112+0000 7f746ba2f700 1 -- 172.21.15.7:0/2922831545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7464157ca0 msgr2=0x7f74640c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:21.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.112+0000 7f746ba2f700 1 --2- 172.21.15.7:0/2922831545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7464157ca0 0x7f74640c4a00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f74600098e0 tx=0x7f74600091a0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.112+0000 7f746ba2f700 1 -- 172.21.15.7:0/2922831545 shutdown_connections 2024-01-27T02:34:21.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.112+0000 7f746ba2f700 1 --2- 172.21.15.7:0/2922831545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7464157ca0 0x7f74640c4a00 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.112+0000 7f746ba2f700 1 --2- 172.21.15.7:0/2922831545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74640c3ae0 0x7f74640cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.112+0000 7f746ba2f700 1 -- 172.21.15.7:0/2922831545 >> 172.21.15.7:0/2922831545 conn(0x7f7464098020 msgr2=0x7f74640711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:21.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.112+0000 7f746ba2f700 1 -- 172.21.15.7:0/2922831545 shutdown_connections 2024-01-27T02:34:21.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.112+0000 7f746ba2f700 1 -- 172.21.15.7:0/2922831545 wait complete. 2024-01-27T02:34:21.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.113+0000 7f746ba2f700 1 Processor -- start 2024-01-27T02:34:21.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.114+0000 7f746ba2f700 1 -- start start 2024-01-27T02:34:21.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.115+0000 7f746ba2f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74640c3ae0 0x7f746414d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.115+0000 7f746ba2f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7464157ca0 0x7f74640b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.115+0000 7f746ba2f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7464158070 0x7f7464068eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.115+0000 7f746ba2f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74640b4530 con 0x7f7464158070 2024-01-27T02:34:21.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.115+0000 7f746ba2f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74640b3920 con 0x7f74640c3ae0 2024-01-27T02:34:21.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.115+0000 7f746ba2f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74640b3c00 con 0x7f7464157ca0 2024-01-27T02:34:21.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.115+0000 7f7469fcc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7464158070 0x7f7464068eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.115+0000 7f7469fcc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7464158070 0x7f7464068eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52892/0 (socket says 172.21.15.7:52892) 2024-01-27T02:34:21.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.115+0000 7f7469fcc700 1 -- 172.21.15.7:0/2655769801 learned_addr learned my addr 172.21.15.7:0/2655769801 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:21.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.115+0000 7f74697cb700 1 --2- 172.21.15.7:0/2655769801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74640c3ae0 0x7f746414d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.115+0000 7f7468fca700 1 --2- 172.21.15.7:0/2655769801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7464157ca0 0x7f74640b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.115+0000 7f7469fcc700 1 -- 172.21.15.7:0/2655769801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7464157ca0 msgr2=0x7f74640b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:21.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.115+0000 7f7469fcc700 1 --2- 172.21.15.7:0/2655769801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7464157ca0 0x7f74640b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.115+0000 7f7469fcc700 1 -- 172.21.15.7:0/2655769801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74640c3ae0 msgr2=0x7f746414d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:21.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.115+0000 7f7469fcc700 1 --2- 172.21.15.7:0/2655769801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74640c3ae0 0x7f746414d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.115+0000 7f7469fcc700 1 -- 172.21.15.7:0/2655769801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7460009420 con 0x7f7464158070 2024-01-27T02:34:21.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.115+0000 7f7469fcc700 1 --2- 172.21.15.7:0/2655769801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7464158070 0x7f7464068eb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f745c00cbc0 tx=0x7f745c0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:21.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.116+0000 7f74597fa700 1 -- 172.21.15.7:0/2655769801 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c00a940 con 0x7f7464158070 2024-01-27T02:34:21.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.116+0000 7f74597fa700 1 -- 172.21.15.7:0/2655769801 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745c00aaa0 con 0x7f7464158070 2024-01-27T02:34:21.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.116+0000 7f746ba2f700 1 -- 172.21.15.7:0/2655769801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74640b3ee0 con 0x7f7464158070 2024-01-27T02:34:21.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.116+0000 7f74597fa700 1 -- 172.21.15.7:0/2655769801 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c00ae20 con 0x7f7464158070 2024-01-27T02:34:21.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.116+0000 7f746ba2f700 1 -- 172.21.15.7:0/2655769801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746414df20 con 0x7f7464158070 2024-01-27T02:34:21.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.117+0000 7f74597fa700 1 -- 172.21.15.7:0/2655769801 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f745c007300 con 0x7f7464158070 2024-01-27T02:34:21.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.118+0000 7f74597fa700 1 --2- 172.21.15.7:0/2655769801 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7448066f30 0x7f74480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.118+0000 7f74597fa700 1 -- 172.21.15.7:0/2655769801 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f745c084920 con 0x7f7464158070 2024-01-27T02:34:21.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.118+0000 7f74697cb700 1 --2- 172.21.15.7:0/2655769801 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7448066f30 0x7f74480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.119+0000 7f74697cb700 1 --2- 172.21.15.7:0/2655769801 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7448066f30 0x7f74480693e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f74600091e0 tx=0x7f7460017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:21.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.120+0000 7f7474d32b80 1 Processor -- start 2024-01-27T02:34:21.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.120+0000 7f7474d32b80 1 -- start start 2024-01-27T02:34:21.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.121+0000 7f7474d32b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b56d1fe010 0x55b56d1fe3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.121+0000 7f7474d32b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b56d246360 0x55b56d228840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.121+0000 7f7474d32b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d1fd5d0 0x55b56d228d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.121+0000 7f7474d32b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b56d1019f0 con 0x55b56d1fd5d0 2024-01-27T02:34:21.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.121+0000 7f7474d32b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b56d153130 con 0x55b56d1fe010 2024-01-27T02:34:21.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.121+0000 7f7474d32b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b56d150930 con 0x55b56d246360 2024-01-27T02:34:21.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.121+0000 7f7469fcc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d1fd5d0 0x55b56d228d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.121+0000 7f7469fcc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d1fd5d0 0x55b56d228d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52900/0 (socket says 172.21.15.7:52900) 2024-01-27T02:34:21.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.121+0000 7f7469fcc700 1 -- 172.21.15.7:0/3779880399 learned_addr learned my addr 172.21.15.7:0/3779880399 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.121+0000 7f7468fca700 1 --2- 172.21.15.7:0/3779880399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b56d246360 0x55b56d228840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.121+0000 7f74697cb700 1 --2- 172.21.15.7:0/3779880399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b56d1fe010 0x55b56d1fe3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.121+0000 7f7469fcc700 1 -- 172.21.15.7:0/3779880399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b56d246360 msgr2=0x55b56d228840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.121+0000 7f7469fcc700 1 --2- 172.21.15.7:0/3779880399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b56d246360 0x55b56d228840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.121+0000 7f7469fcc700 1 -- 172.21.15.7:0/3779880399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b56d1fe010 msgr2=0x55b56d1fe3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.121+0000 7f7469fcc700 1 --2- 172.21.15.7:0/3779880399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b56d1fe010 0x55b56d1fe3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.121+0000 7f7469fcc700 1 -- 172.21.15.7:0/3779880399 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b56cf8bb10 con 0x55b56d1fd5d0 2024-01-27T02:34:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.121+0000 7f7469fcc700 1 --2- 172.21.15.7:0/3779880399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d1fd5d0 0x55b56d228d80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f745c051d30 tx=0x7f745c082d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7627498644b4d0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.121+0000 7f744e7fc700 1 -- 172.21.15.7:0/3779880399 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c058a80 con 0x55b56d1fd5d0 2024-01-27T02:34:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.122+0000 7f744e7fc700 1 -- 172.21.15.7:0/3779880399 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745c05c7d0 con 0x55b56d1fd5d0 2024-01-27T02:34:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.122+0000 7f744e7fc700 1 -- 172.21.15.7:0/3779880399 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c053bd0 con 0x55b56d1fd5d0 2024-01-27T02:34:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.122+0000 7f7474d32b80 1 -- 172.21.15.7:0/3779880399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d1fd5d0 msgr2=0x55b56d228d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.122+0000 7f7474d32b80 1 --2- 172.21.15.7:0/3779880399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d1fd5d0 0x55b56d228d80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f745c051d30 tx=0x7f745c082d00 comp rx=0 tx=0).stop 2024-01-27T02:34:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.122+0000 7f7474d32b80 1 -- 172.21.15.7:0/3779880399 shutdown_connections 2024-01-27T02:34:21.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.122+0000 7f7474d32b80 1 --2- 172.21.15.7:0/3779880399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d1fd5d0 0x55b56d228d80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.122+0000 7f7474d32b80 1 --2- 172.21.15.7:0/3779880399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b56d1fe010 0x55b56d1fe3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.122+0000 7f7474d32b80 1 --2- 172.21.15.7:0/3779880399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b56d246360 0x55b56d228840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.122+0000 7f7474d32b80 1 -- 172.21.15.7:0/3779880399 >> 172.21.15.7:0/3779880399 conn(0x55b56d1fda70 msgr2=0x55b56d246820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:21.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.122+0000 7f7474d32b80 1 -- 172.21.15.7:0/3779880399 shutdown_connections 2024-01-27T02:34:21.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.122+0000 7f7474d32b80 1 -- 172.21.15.7:0/3779880399 wait complete. 2024-01-27T02:34:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.123+0000 7f7474d32b80 1 -- start start 2024-01-27T02:34:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.123+0000 7f7474d32b80 1 Processor -- start 2024-01-27T02:34:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.124+0000 7f7474d32b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b56d1fd5d0 0x55b56d252fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.124+0000 7f7474d32b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d1fe010 0x55b56d1fc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.124+0000 7f7474d32b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b56d246360 0x55b56d240520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.124+0000 7f7474d32b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b56d14c720 con 0x55b56d1fe010 2024-01-27T02:34:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.124+0000 7f7474d32b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b56d0f81b0 con 0x55b56d1fd5d0 2024-01-27T02:34:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.124+0000 7f7474d32b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b56d0f7e50 con 0x55b56d246360 2024-01-27T02:34:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.124+0000 7f7474d32b80 10 client.0 _ll_register_callbacks cb 0x55b56cf5a7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-27T02:34:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.124+0000 7f7468fca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d1fe010 0x55b56d1fc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.124+0000 7f7468fca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d1fe010 0x55b56d1fc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52902/0 (socket says 172.21.15.7:52902) 2024-01-27T02:34:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.124+0000 7f7468fca700 1 -- 172.21.15.7:0/3078261487 learned_addr learned my addr 172.21.15.7:0/3078261487 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.124+0000 7f74697cb700 1 --2- 172.21.15.7:0/3078261487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b56d1fd5d0 0x55b56d252fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.124+0000 7f7469fcc700 1 --2- 172.21.15.7:0/3078261487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b56d246360 0x55b56d240520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.124+0000 7f7468fca700 1 -- 172.21.15.7:0/3078261487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b56d246360 msgr2=0x55b56d240520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.124+0000 7f7468fca700 1 --2- 172.21.15.7:0/3078261487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b56d246360 0x55b56d240520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.124+0000 7f7468fca700 1 -- 172.21.15.7:0/3078261487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b56d1fd5d0 msgr2=0x55b56d252fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.124+0000 7f7468fca700 1 --2- 172.21.15.7:0/3078261487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b56d1fd5d0 0x55b56d252fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.125+0000 7f7468fca700 1 -- 172.21.15.7:0/3078261487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746414df20 con 0x55b56d1fe010 2024-01-27T02:34:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.125+0000 7f7468fca700 1 --2- 172.21.15.7:0/3078261487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d1fe010 0x55b56d1fc990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f74540126b0 tx=0x7f7454008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d677d3cf5328b server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.125+0000 7f74437fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-27T02:34:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.125+0000 7f74437fe700 1 -- 172.21.15.7:0/3078261487 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7454007700 con 0x55b56d1fe010 2024-01-27T02:34:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.125+0000 7f7474d32b80 1 -- 172.21.15.7:0/3078261487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b56d1f1190 con 0x55b56d1fe010 2024-01-27T02:34:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.125+0000 7f74437fe700 1 -- 172.21.15.7:0/3078261487 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7454007e00 con 0x55b56d1fe010 2024-01-27T02:34:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.125+0000 7f74437fe700 1 -- 172.21.15.7:0/3078261487 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7454015870 con 0x55b56d1fe010 2024-01-27T02:34:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.125+0000 7f7474d32b80 10 client.5775 fetch_fsmap learned FSMap version 34 2024-01-27T02:34:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.125+0000 7f7474d32b80 1 -- 172.21.15.7:0/3078261487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55b56d11a290 con 0x55b56d1fe010 2024-01-27T02:34:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.125+0000 7f74437fe700 1 -- 172.21.15.7:0/3078261487 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b56d1f1190 con 0x55b56d1fe010 2024-01-27T02:34:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.125+0000 7f74437fe700 1 -- 172.21.15.7:0/3078261487 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 34) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f74540159d0 con 0x55b56d1fe010 2024-01-27T02:34:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.125+0000 7f7474d32b80 10 client.5775 fetch_fsmap finished waiting for FSMap version 34 2024-01-27T02:34:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.125+0000 7f7474d32b80 10 client.5775 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-27T02:34:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.125+0000 7f7474d32b80 20 client.5775 populate_metadata read hostname 'smithi007' 2024-01-27T02:34:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.125+0000 7f7474d32b80 1 --2- 172.21.15.7:0/3078261487 >> [v2:172.21.15.7:6833/1399055016,v1:172.21.15.7:6835/1399055016] conn(0x55b56d2253c0 0x55b56d225790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.125+0000 7f7474d32b80 4 client.5775 mds_command: new command op to 5631 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-27T02:34:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.125+0000 7f7474d32b80 1 -- 172.21.15.7:0/3078261487 --> [v2:172.21.15.7:6833/1399055016,v1:172.21.15.7:6835/1399055016] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b56d0eeee0 con 0x55b56d2253c0 2024-01-27T02:34:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.126+0000 7f74697cb700 1 --2- 172.21.15.7:0/3078261487 >> [v2:172.21.15.7:6833/1399055016,v1:172.21.15.7:6835/1399055016] conn(0x55b56d2253c0 0x55b56d225790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.126+0000 7f74437fe700 10 client.5775 ms_handle_connect on v2:172.21.15.7:6833/1399055016 2024-01-27T02:34:21.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.126+0000 7f74697cb700 1 --2- 172.21.15.7:0/3078261487 >> [v2:172.21.15.7:6833/1399055016,v1:172.21.15.7:6835/1399055016] conn(0x55b56d2253c0 0x55b56d225790 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b6ce9d6a9260187 server_cookie=9f6f7e27a3cb5ea6 in_seq=0 out_seq=0 2024-01-27T02:34:21.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.127+0000 7f74437fe700 1 -- 172.21.15.7:0/3078261487 <== mds.0 v2:172.21.15.7:6833/1399055016 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17146 (crc 0 0 0) 0x55b56d0eeee0 con 0x55b56d2253c0 2024-01-27T02:34:21.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.127+0000 7f74437fe700 10 client.5775 handle_command_reply: tid=0 2024-01-27T02:34:21.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.127+0000 7f7474d32b80 1 client.5775 shutdown 2024-01-27T02:34:21.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.127+0000 7f7474d32b80 1 -- 172.21.15.7:0/3078261487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d1fe010 msgr2=0x55b56d1fc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:21.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.127+0000 7f7474d32b80 1 --2- 172.21.15.7:0/3078261487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d1fe010 0x55b56d1fc990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f74540126b0 tx=0x7f7454008040 comp rx=0 tx=0).stop 2024-01-27T02:34:21.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.127+0000 7f7474d32b80 1 -- 172.21.15.7:0/3078261487 shutdown_connections 2024-01-27T02:34:21.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.127+0000 7f7474d32b80 1 --2- 172.21.15.7:0/3078261487 >> [v2:172.21.15.7:6833/1399055016,v1:172.21.15.7:6835/1399055016] conn(0x55b56d2253c0 0x55b56d225790 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.127+0000 7f7474d32b80 1 --2- 172.21.15.7:0/3078261487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b56d246360 0x55b56d240520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.127+0000 7f7474d32b80 1 --2- 172.21.15.7:0/3078261487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b56d1fd5d0 0x55b56d252fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.127+0000 7f7474d32b80 1 --2- 172.21.15.7:0/3078261487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d1fe010 0x55b56d1fc990 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.127+0000 7f7474d32b80 1 -- 172.21.15.7:0/3078261487 >> 172.21.15.7:0/3078261487 conn(0x55b56d1fda70 msgr2=0x55b56d1f7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:21.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.127+0000 7f74437fe700 0 client.5775 ms_handle_reset on v2:172.21.15.7:6833/1399055016 2024-01-27T02:34:21.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.127+0000 7f7474d32b80 1 -- 172.21.15.7:0/3078261487 shutdown_connections 2024-01-27T02:34:21.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.127+0000 7f7474d32b80 1 -- 172.21.15.7:0/3078261487 wait complete. 2024-01-27T02:34:21.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.127+0000 7f7474d32b80 20 client.5775 trim_cache size 0 max 16384 2024-01-27T02:34:21.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.145+0000 7f7474d32b80 1 Processor -- start 2024-01-27T02:34:21.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.145+0000 7f7474d32b80 1 -- start start 2024-01-27T02:34:21.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.146+0000 7f7474d32b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b56d1fd5d0 0x55b56d241fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.146+0000 7f7474d32b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b56d1fe010 0x55b56d23c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.146+0000 7f7474d32b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d246360 0x55b56d25c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.146+0000 7f7474d32b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b56d044e40 con 0x55b56d246360 2024-01-27T02:34:21.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.146+0000 7f7474d32b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b56d2521f0 con 0x55b56d1fd5d0 2024-01-27T02:34:21.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.146+0000 7f7474d32b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b56d23dca0 con 0x55b56d1fe010 2024-01-27T02:34:21.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.146+0000 7f7469fcc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d246360 0x55b56d25c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.146+0000 7f7469fcc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d246360 0x55b56d25c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52918/0 (socket says 172.21.15.7:52918) 2024-01-27T02:34:21.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.146+0000 7f7469fcc700 1 -- 172.21.15.7:0/373633047 learned_addr learned my addr 172.21.15.7:0/373633047 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:21.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.146+0000 7f7468fca700 1 --2- 172.21.15.7:0/373633047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b56d1fe010 0x55b56d23c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.146+0000 7f74697cb700 1 --2- 172.21.15.7:0/373633047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b56d1fd5d0 0x55b56d241fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.146+0000 7f7469fcc700 1 -- 172.21.15.7:0/373633047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b56d1fe010 msgr2=0x55b56d23c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:21.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.146+0000 7f7469fcc700 1 --2- 172.21.15.7:0/373633047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b56d1fe010 0x55b56d23c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.146+0000 7f7469fcc700 1 -- 172.21.15.7:0/373633047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b56d1fd5d0 msgr2=0x55b56d241fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:21.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.146+0000 7f7469fcc700 1 --2- 172.21.15.7:0/373633047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b56d1fd5d0 0x55b56d241fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.146+0000 7f7469fcc700 1 -- 172.21.15.7:0/373633047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b56cfe3c80 con 0x55b56d246360 2024-01-27T02:34:21.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.146+0000 7f7468fca700 1 --2- 172.21.15.7:0/373633047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b56d1fe010 0x55b56d23c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:21.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.146+0000 7f74697cb700 1 --2- 172.21.15.7:0/373633047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b56d1fd5d0 0x55b56d241fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:21.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.146+0000 7f7469fcc700 1 --2- 172.21.15.7:0/373633047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d246360 0x55b56d25c370 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f745c0560e0 tx=0x7f745c082000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186ad068b786540d server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:21.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.147+0000 7f744effd700 1 -- 172.21.15.7:0/373633047 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c055370 con 0x55b56d246360 2024-01-27T02:34:21.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.147+0000 7f744effd700 1 -- 172.21.15.7:0/373633047 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745c058a80 con 0x55b56d246360 2024-01-27T02:34:21.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.147+0000 7f744effd700 1 -- 172.21.15.7:0/373633047 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c055ca0 con 0x55b56d246360 2024-01-27T02:34:21.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.147+0000 7f7474d32b80 1 -- 172.21.15.7:0/373633047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d246360 msgr2=0x55b56d25c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:21.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.147+0000 7f7474d32b80 1 --2- 172.21.15.7:0/373633047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d246360 0x55b56d25c370 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f745c0560e0 tx=0x7f745c082000 comp rx=0 tx=0).stop 2024-01-27T02:34:21.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.147+0000 7f7474d32b80 1 -- 172.21.15.7:0/373633047 shutdown_connections 2024-01-27T02:34:21.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.147+0000 7f7474d32b80 1 --2- 172.21.15.7:0/373633047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d246360 0x55b56d25c370 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.147+0000 7f7474d32b80 1 --2- 172.21.15.7:0/373633047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b56d1fd5d0 0x55b56d241fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.147+0000 7f7474d32b80 1 --2- 172.21.15.7:0/373633047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b56d1fe010 0x55b56d23c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.147+0000 7f7474d32b80 1 -- 172.21.15.7:0/373633047 >> 172.21.15.7:0/373633047 conn(0x55b56d1fda70 msgr2=0x55b56d1fc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:21.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.147+0000 7f7474d32b80 1 -- 172.21.15.7:0/373633047 shutdown_connections 2024-01-27T02:34:21.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.147+0000 7f7474d32b80 1 -- 172.21.15.7:0/373633047 wait complete. 2024-01-27T02:34:21.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.148+0000 7f7474d32b80 1 -- start start 2024-01-27T02:34:21.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.148+0000 7f7474d32b80 1 Processor -- start 2024-01-27T02:34:21.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.149+0000 7f7474d32b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b56d1fd5d0 0x55b56d228580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.149+0000 7f7474d32b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d1fe010 0x55b56d228ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.149+0000 7f7474d32b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b56d246360 0x55b56d2a78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.149+0000 7f7474d32b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b56d245850 con 0x55b56d1fe010 2024-01-27T02:34:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.149+0000 7f7474d32b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b56d2a73d0 con 0x55b56d1fd5d0 2024-01-27T02:34:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.149+0000 7f7474d32b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b56d223a20 con 0x55b56d246360 2024-01-27T02:34:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.149+0000 7f7474d32b80 10 client.0 _ll_register_callbacks cb 0x55b56cf5a7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-27T02:34:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.149+0000 7f7468fca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d1fe010 0x55b56d228ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.149+0000 7f7468fca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d1fe010 0x55b56d228ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52922/0 (socket says 172.21.15.7:52922) 2024-01-27T02:34:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.149+0000 7f7468fca700 1 -- 172.21.15.7:0/3137851668 learned_addr learned my addr 172.21.15.7:0/3137851668 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.149+0000 7f7469fcc700 1 --2- 172.21.15.7:0/3137851668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b56d246360 0x55b56d2a78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.149+0000 7f74697cb700 1 --2- 172.21.15.7:0/3137851668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b56d1fd5d0 0x55b56d228580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.149+0000 7f7468fca700 1 -- 172.21.15.7:0/3137851668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b56d246360 msgr2=0x55b56d2a78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.149+0000 7f7468fca700 1 --2- 172.21.15.7:0/3137851668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b56d246360 0x55b56d2a78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.149+0000 7f7468fca700 1 -- 172.21.15.7:0/3137851668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b56d1fd5d0 msgr2=0x55b56d228580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:21.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.149+0000 7f7468fca700 1 --2- 172.21.15.7:0/3137851668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b56d1fd5d0 0x55b56d228580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.149+0000 7f7468fca700 1 -- 172.21.15.7:0/3137851668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b56cf8bb10 con 0x55b56d1fe010 2024-01-27T02:34:21.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.149+0000 7f7469fcc700 1 --2- 172.21.15.7:0/3137851668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b56d246360 0x55b56d2a78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:21.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.149+0000 7f74697cb700 1 --2- 172.21.15.7:0/3137851668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b56d1fd5d0 0x55b56d228580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:21.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.150+0000 7f7468fca700 1 --2- 172.21.15.7:0/3137851668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d1fe010 0x55b56d228ac0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x55b56d1fd170 tx=0x7f7454022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f89d2272486870 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:21.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.150+0000 7f74437fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-27T02:34:21.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.150+0000 7f74437fe700 1 -- 172.21.15.7:0/3137851668 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74540075e0 con 0x55b56d1fe010 2024-01-27T02:34:21.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.150+0000 7f74437fe700 1 -- 172.21.15.7:0/3137851668 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7454000bc0 con 0x55b56d1fe010 2024-01-27T02:34:21.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.150+0000 7f7474d32b80 1 -- 172.21.15.7:0/3137851668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f745c05c7d0 con 0x55b56d1fe010 2024-01-27T02:34:21.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.150+0000 7f74437fe700 1 -- 172.21.15.7:0/3137851668 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f745401e530 con 0x55b56d1fe010 2024-01-27T02:34:21.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.150+0000 7f7474d32b80 10 client.5781 fetch_fsmap learned FSMap version 34 2024-01-27T02:34:21.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.150+0000 7f7474d32b80 1 -- 172.21.15.7:0/3137851668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55b56cf89fe0 con 0x55b56d1fe010 2024-01-27T02:34:21.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.150+0000 7f74437fe700 1 -- 172.21.15.7:0/3137851668 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745c05c7d0 con 0x55b56d1fe010 2024-01-27T02:34:21.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.150+0000 7f74437fe700 1 -- 172.21.15.7:0/3137851668 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 34) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f745401e920 con 0x55b56d1fe010 2024-01-27T02:34:21.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.150+0000 7f7474d32b80 10 client.5781 fetch_fsmap finished waiting for FSMap version 34 2024-01-27T02:34:21.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.150+0000 7f7474d32b80 10 client.5781 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-27T02:34:21.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.150+0000 7f7474d32b80 20 client.5781 populate_metadata read hostname 'smithi007' 2024-01-27T02:34:21.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.150+0000 7f7474d32b80 1 --2- 172.21.15.7:0/3137851668 >> [v2:172.21.15.7:6833/1399055016,v1:172.21.15.7:6835/1399055016] conn(0x55b56d1f9280 0x55b56d2a7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.150+0000 7f7474d32b80 4 client.5781 mds_command: new command op to 5631 tid=0[{"prefix": "status"}] 2024-01-27T02:34:21.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.150+0000 7f7474d32b80 1 -- 172.21.15.7:0/3137851668 --> [v2:172.21.15.7:6833/1399055016,v1:172.21.15.7:6835/1399055016] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b56d0eec00 con 0x55b56d1f9280 2024-01-27T02:34:21.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.150+0000 7f74697cb700 1 --2- 172.21.15.7:0/3137851668 >> [v2:172.21.15.7:6833/1399055016,v1:172.21.15.7:6835/1399055016] conn(0x55b56d1f9280 0x55b56d2a7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.151+0000 7f74437fe700 10 client.5781 ms_handle_connect on v2:172.21.15.7:6833/1399055016 2024-01-27T02:34:21.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.151+0000 7f74697cb700 1 --2- 172.21.15.7:0/3137851668 >> [v2:172.21.15.7:6833/1399055016,v1:172.21.15.7:6835/1399055016] conn(0x55b56d1f9280 0x55b56d2a7e10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ba3e57c1bd0d28f server_cookie=87d8142f56cc3978 in_seq=0 out_seq=0 2024-01-27T02:34:21.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.151+0000 7f74437fe700 1 -- 172.21.15.7:0/3137851668 <== mds.0 v2:172.21.15.7:6833/1399055016 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55b56d0eec00 con 0x55b56d1f9280 2024-01-27T02:34:21.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.151+0000 7f74437fe700 10 client.5781 handle_command_reply: tid=0 2024-01-27T02:34:21.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.151+0000 7f7474d32b80 1 client.5781 shutdown 2024-01-27T02:34:21.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.151+0000 7f7474d32b80 1 -- 172.21.15.7:0/3137851668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d1fe010 msgr2=0x55b56d228ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:21.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.151+0000 7f7474d32b80 1 --2- 172.21.15.7:0/3137851668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d1fe010 0x55b56d228ac0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x55b56d1fd170 tx=0x7f7454022000 comp rx=0 tx=0).stop 2024-01-27T02:34:21.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.151+0000 7f7474d32b80 1 -- 172.21.15.7:0/3137851668 shutdown_connections 2024-01-27T02:34:21.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.151+0000 7f7474d32b80 1 --2- 172.21.15.7:0/3137851668 >> [v2:172.21.15.7:6833/1399055016,v1:172.21.15.7:6835/1399055016] conn(0x55b56d1f9280 0x55b56d2a7e10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.151+0000 7f7474d32b80 1 --2- 172.21.15.7:0/3137851668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b56d246360 0x55b56d2a78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.151+0000 7f7474d32b80 1 --2- 172.21.15.7:0/3137851668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b56d1fd5d0 0x55b56d228580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.151+0000 7f7474d32b80 1 --2- 172.21.15.7:0/3137851668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b56d1fe010 0x55b56d228ac0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.151+0000 7f74437fe700 0 client.5781 ms_handle_reset on v2:172.21.15.7:6833/1399055016 2024-01-27T02:34:21.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.151+0000 7f7474d32b80 1 -- 172.21.15.7:0/3137851668 >> 172.21.15.7:0/3137851668 conn(0x55b56d1fda70 msgr2=0x55b56d25c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:21.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.151+0000 7f7474d32b80 1 -- 172.21.15.7:0/3137851668 shutdown_connections 2024-01-27T02:34:21.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.151+0000 7f7474d32b80 1 -- 172.21.15.7:0/3137851668 wait complete. 2024-01-27T02:34:21.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.151+0000 7f7474d32b80 20 client.5781 trim_cache size 0 max 16384 2024-01-27T02:34:21.160 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:34:21.160 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:34:21.160 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-27T02:34:21.160 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5631, 2024-01-27T02:34:21.160 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-27T02:34:21.160 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-27T02:34:21.160 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-27T02:34:21.160 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.0976611780000001, 2024-01-27T02:34:21.160 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 34, 2024-01-27T02:34:21.161 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 65, 2024-01-27T02:34:21.161 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 65, 2024-01-27T02:34:21.161 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 11.855419291 2024-01-27T02:34:21.161 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:34:21.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.152+0000 7f746ba2f700 1 -- 172.21.15.7:0/2655769801 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7448066f30 msgr2=0x7f74480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:21.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.152+0000 7f746ba2f700 1 --2- 172.21.15.7:0/2655769801 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7448066f30 0x7f74480693e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f74600091e0 tx=0x7f7460017000 comp rx=0 tx=0).stop 2024-01-27T02:34:21.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.152+0000 7f746ba2f700 1 -- 172.21.15.7:0/2655769801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7464158070 msgr2=0x7f7464068eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:21.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.152+0000 7f746ba2f700 1 --2- 172.21.15.7:0/2655769801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7464158070 0x7f7464068eb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f745c00cbc0 tx=0x7f745c0019c0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.152+0000 7f746ba2f700 1 -- 172.21.15.7:0/2655769801 shutdown_connections 2024-01-27T02:34:21.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.152+0000 7f746ba2f700 1 --2- 172.21.15.7:0/2655769801 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7448066f30 0x7f74480693e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.152+0000 7f746ba2f700 1 --2- 172.21.15.7:0/2655769801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7464158070 0x7f7464068eb0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.152+0000 7f746ba2f700 1 --2- 172.21.15.7:0/2655769801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74640c3ae0 0x7f746414d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.152+0000 7f746ba2f700 1 --2- 172.21.15.7:0/2655769801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7464157ca0 0x7f74640b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.152+0000 7f746ba2f700 1 -- 172.21.15.7:0/2655769801 >> 172.21.15.7:0/2655769801 conn(0x7f7464098020 msgr2=0x7f74640ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:21.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.152+0000 7f746ba2f700 1 -- 172.21.15.7:0/2655769801 shutdown_connections 2024-01-27T02:34:21.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.152+0000 7f746ba2f700 1 -- 172.21.15.7:0/2655769801 wait complete. 2024-01-27T02:34:21.167 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:34:21.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.298+0000 7f7ab6a10700 1 Processor -- start 2024-01-27T02:34:21.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.298+0000 7f7ab6a10700 1 -- start start 2024-01-27T02:34:21.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.299+0000 7f7ab6a10700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ab01438f0 0x7f7ab00c8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.299+0000 7f7ab6a10700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab00cb7a0 con 0x7f7ab0063c90 2024-01-27T02:34:21.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.299+0000 7f7ab6a10700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab00cb9f0 con 0x7f7ab0143520 2024-01-27T02:34:21.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.299+0000 7f7ab6a10700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ab00cbcd0 con 0x7f7ab01438f0 2024-01-27T02:34:21.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.299+0000 7f7ab4fad700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ab01438f0 0x7f7ab00c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.299+0000 7f7aaf7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7ab0063c90 0x7f7ab00c8210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42240/0 (socket says 172.21.15.7:42240) 2024-01-27T02:34:21.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.299+0000 7f7ab4fad700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ab01438f0 0x7f7ab00c8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52936/0 (socket says 172.21.15.7:52936) 2024-01-27T02:34:21.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.299+0000 7f7aaf7fe700 1 -- 172.21.15.7:0/3058913108 learned_addr learned my addr 172.21.15.7:0/3058913108 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:34:21.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.299+0000 7f7ab4fad700 1 -- 172.21.15.7:0/3058913108 >> v1:172.21.15.188:6789/0 conn(0x7f7ab0143520 legacy=0x7f7ab0063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:21.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.299+0000 7f7ab4fad700 1 -- 172.21.15.7:0/3058913108 >> v1:172.21.15.7:6789/0 conn(0x7f7ab0063c90 legacy=0x7f7ab00c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.299+0000 7f7ab4fad700 1 -- 172.21.15.7:0/3058913108 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab00cbfb0 con 0x7f7ab01438f0 2024-01-27T02:34:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.299+0000 7f7aaeffd700 1 -- 172.21.15.7:0/3058913108 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795291529 0 0) 0x7f7ab00cb7a0 con 0x7f7ab0063c90 2024-01-27T02:34:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.299+0000 7f7ab4fad700 1 --2- 172.21.15.7:0/3058913108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab01438f0 0x7f7ab00c8920 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009e60 tx=0x7f7aa8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe5054381e8e487 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.299+0000 7f7aaeffd700 1 -- 172.21.15.7:0/3058913108 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa800e070 con 0x7f7ab01438f0 2024-01-27T02:34:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.300+0000 7f7aaeffd700 1 -- 172.21.15.7:0/3058913108 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa800bc30 con 0x7f7ab01438f0 2024-01-27T02:34:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.300+0000 7f7aaeffd700 1 -- 172.21.15.7:0/3058913108 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8012430 con 0x7f7ab01438f0 2024-01-27T02:34:21.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.300+0000 7f7ab6a10700 1 -- 172.21.15.7:0/3058913108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab01438f0 msgr2=0x7f7ab00c8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:21.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.300+0000 7f7ab6a10700 1 --2- 172.21.15.7:0/3058913108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab01438f0 0x7f7ab00c8920 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009e60 tx=0x7f7aa8009780 comp rx=0 tx=0).stop 2024-01-27T02:34:21.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.300+0000 7f7ab6a10700 1 -- 172.21.15.7:0/3058913108 shutdown_connections 2024-01-27T02:34:21.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.300+0000 7f7ab6a10700 1 --2- 172.21.15.7:0/3058913108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab01438f0 0x7f7ab00c8920 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.300+0000 7f7ab6a10700 1 -- 172.21.15.7:0/3058913108 >> 172.21.15.7:0/3058913108 conn(0x7f7ab00cdbd0 msgr2=0x7f7ab00cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:21.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.300+0000 7f7ab6a10700 1 -- 172.21.15.7:0/3058913108 shutdown_connections 2024-01-27T02:34:21.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.300+0000 7f7ab6a10700 1 -- 172.21.15.7:0/3058913108 wait complete. 2024-01-27T02:34:21.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.301+0000 7f7ab6a10700 1 Processor -- start 2024-01-27T02:34:21.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.302+0000 7f7ab6a10700 1 -- start start 2024-01-27T02:34:21.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.302+0000 7f7ab6a10700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab0063c90 0x7f7ab01408c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.302+0000 7f7ab6a10700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab0143520 0x7f7ab00ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.302+0000 7f7ab6a10700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ab01438f0 0x7f7ab00cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.303+0000 7f7ab6a10700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab00b19c0 con 0x7f7ab0143520 2024-01-27T02:34:21.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.303+0000 7f7ab6a10700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab00b16e0 con 0x7f7ab01438f0 2024-01-27T02:34:21.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.303+0000 7f7ab6a10700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab00ad0a0 con 0x7f7ab0063c90 2024-01-27T02:34:21.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.303+0000 7f7aaf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab0143520 0x7f7ab00ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.303+0000 7f7aaf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab0143520 0x7f7ab00ca5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52938/0 (socket says 172.21.15.7:52938) 2024-01-27T02:34:21.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.303+0000 7f7aaf7fe700 1 -- 172.21.15.7:0/1104943011 learned_addr learned my addr 172.21.15.7:0/1104943011 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:21.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.303+0000 7f7ab4fad700 1 --2- 172.21.15.7:0/1104943011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ab01438f0 0x7f7ab00cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.303+0000 7f7aaffff700 1 --2- 172.21.15.7:0/1104943011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab0063c90 0x7f7ab01408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.303+0000 7f7aaf7fe700 1 -- 172.21.15.7:0/1104943011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab0063c90 msgr2=0x7f7ab01408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:21.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.303+0000 7f7aaf7fe700 1 --2- 172.21.15.7:0/1104943011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab0063c90 0x7f7ab01408c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.303+0000 7f7aaf7fe700 1 -- 172.21.15.7:0/1104943011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ab01438f0 msgr2=0x7f7ab00cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:21.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.303+0000 7f7aaf7fe700 1 --2- 172.21.15.7:0/1104943011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ab01438f0 0x7f7ab00cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.303+0000 7f7aaf7fe700 1 -- 172.21.15.7:0/1104943011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa8009a20 con 0x7f7ab0143520 2024-01-27T02:34:21.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.303+0000 7f7aaffff700 1 --2- 172.21.15.7:0/1104943011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab0063c90 0x7f7ab01408c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:21.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.303+0000 7f7ab4fad700 1 --2- 172.21.15.7:0/1104943011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ab01438f0 0x7f7ab00cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:21.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.303+0000 7f7aaf7fe700 1 --2- 172.21.15.7:0/1104943011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab0143520 0x7f7ab00ca5d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400c5e0 tx=0x7f7aa4007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:21.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.303+0000 7f7a97fff700 1 -- 172.21.15.7:0/1104943011 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4005c00 con 0x7f7ab0143520 2024-01-27T02:34:21.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.303+0000 7f7a97fff700 1 -- 172.21.15.7:0/1104943011 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa4005d60 con 0x7f7ab0143520 2024-01-27T02:34:21.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.303+0000 7f7ab6a10700 1 -- 172.21.15.7:0/1104943011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab00ad380 con 0x7f7ab0143520 2024-01-27T02:34:21.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.303+0000 7f7a97fff700 1 -- 172.21.15.7:0/1104943011 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4007300 con 0x7f7ab0143520 2024-01-27T02:34:21.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.303+0000 7f7ab6a10700 1 -- 172.21.15.7:0/1104943011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab0140e00 con 0x7f7ab0143520 2024-01-27T02:34:21.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.305+0000 7f7ab6a10700 1 -- 172.21.15.7:0/1104943011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab004c710 con 0x7f7ab0143520 2024-01-27T02:34:21.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.305+0000 7f7a97fff700 1 -- 172.21.15.7:0/1104943011 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7aa401b3b0 con 0x7f7ab0143520 2024-01-27T02:34:21.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.306+0000 7f7a97fff700 1 --2- 172.21.15.7:0/1104943011 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7a9c066f30 0x7f7a9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:21.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.306+0000 7f7a97fff700 1 -- 172.21.15.7:0/1104943011 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7aa4083ac0 con 0x7f7ab0143520 2024-01-27T02:34:21.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.306+0000 7f7aaffff700 1 --2- 172.21.15.7:0/1104943011 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7a9c066f30 0x7f7a9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:21.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.306+0000 7f7aaffff700 1 --2- 172.21.15.7:0/1104943011 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7a9c066f30 0x7f7a9c0693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7aa00018f0 tx=0x7f7aa0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:21.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.310+0000 7f7a97fff700 1 -- 172.21.15.7:0/1104943011 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7aa4050d30 con 0x7f7ab0143520 2024-01-27T02:34:21.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.562+0000 7f7ab6a10700 1 -- 172.21.15.7:0/1104943011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7ab00cc290 con 0x7f7ab0143520 2024-01-27T02:34:21.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.562+0000 7f7a97fff700 1 -- 172.21.15.7:0/1104943011 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7aa4083e60 con 0x7f7ab0143520 2024-01-27T02:34:21.564 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:34:21.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.565+0000 7f7ab6a10700 1 -- 172.21.15.7:0/1104943011 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7a9c066f30 msgr2=0x7f7a9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:21.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.565+0000 7f7ab6a10700 1 --2- 172.21.15.7:0/1104943011 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7a9c066f30 0x7f7a9c0693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7aa00018f0 tx=0x7f7aa0009040 comp rx=0 tx=0).stop 2024-01-27T02:34:21.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.565+0000 7f7ab6a10700 1 -- 172.21.15.7:0/1104943011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab0143520 msgr2=0x7f7ab00ca5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:21.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.565+0000 7f7ab6a10700 1 --2- 172.21.15.7:0/1104943011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab0143520 0x7f7ab00ca5d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400c5e0 tx=0x7f7aa4007d50 comp rx=0 tx=0).stop 2024-01-27T02:34:21.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.565+0000 7f7ab6a10700 1 -- 172.21.15.7:0/1104943011 shutdown_connections 2024-01-27T02:34:21.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.565+0000 7f7ab6a10700 1 --2- 172.21.15.7:0/1104943011 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7a9c066f30 0x7f7a9c0693e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.565+0000 7f7ab6a10700 1 --2- 172.21.15.7:0/1104943011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ab01438f0 0x7f7ab00cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.565+0000 7f7ab6a10700 1 --2- 172.21.15.7:0/1104943011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab0063c90 0x7f7ab01408c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.565+0000 7f7ab6a10700 1 --2- 172.21.15.7:0/1104943011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab0143520 0x7f7ab00ca5d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:21.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.565+0000 7f7ab6a10700 1 -- 172.21.15.7:0/1104943011 >> 172.21.15.7:0/1104943011 conn(0x7f7ab00cdbd0 msgr2=0x7f7ab00aa930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:21.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.565+0000 7f7ab6a10700 1 -- 172.21.15.7:0/1104943011 shutdown_connections 2024-01-27T02:34:21.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:21.565+0000 7f7ab6a10700 1 -- 172.21.15.7:0/1104943011 wait complete. 2024-01-27T02:34:21.580 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-27T02:34:21.580 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:34:21.580 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-27T02:34:21.580 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-27T02:34:21.580 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-27T02:34:21.580 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-27T02:34:21.580 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-27T02:34:21.580 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:34:21.581 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:34:21.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:34:21.581 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-27T02:34:21.601 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:34:21.601 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:34:21.601 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:34:21.601 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:34:21.601 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-27T02:34:21.601 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:34:21.601 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:34:21.602 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-27T02:34:21.602 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-27T02:34:21.602 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:34:21.602 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-27T02:34:21.602 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-27T02:34:21.602 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:34:21.602 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-27T02:34:21.602 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-27T02:34:21.602 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-27T02:34:21.602 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42250sec preferred_lft 42250sec 2024-01-27T02:34:21.602 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-27T02:34:21.602 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:34:21.602 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:34:21.602 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-27T02:34:21.603 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-27T02:34:21.603 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:34:21.603 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 4a:57:d9:3c:c5:a0 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:34:21.603 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-27T02:34:21.603 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:34:21.603 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::2477:9eff:fe39:433c/64 scope link 2024-01-27T02:34:21.603 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:34:21.603 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:34:21.603 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 4a:57:d9:3c:c5:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-27T02:34:21.603 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::4857:d9ff:fe3c:c5a0/64 scope link 2024-01-27T02:34:21.603 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:34:21.603 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:34:21.603 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ca:6e:b8:df:53:bf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-27T02:34:21.603 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c86e:b8ff:fedf:53bf/64 scope link 2024-01-27T02:34:21.603 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:34:21.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:34:21.604 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:34:21.672 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-27T02:34:21.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:34:21.674 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-27T02:34:21.738 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-27T02:34:21.738 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:34:21.738 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-27T02:34:21.738 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-27T02:34:21.738 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:34:21.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:34:21.739 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-27T02:34:21.739 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-27T02:34:21.739 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-27T02:34:21.739 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-27T02:34:21.739 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-27T02:34:21.739 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-27T02:34:21.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:34:21.864 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-27T02:34:21.884 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-27T02:34:21.884 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-27T02:34:21.885 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:34:21.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:34:21.885 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-27T02:34:21.959 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:34:21.960 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-27T02:34:22.022 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:34:22.022 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:34:22.022 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-27T02:34:22.084 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:34:22.084 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:34:22.084 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:34:22.178 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:34:22.178 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:34:22.178 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:34:22.188 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:34:22.188 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:34:22.188 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:34:22.188 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:34:22.189 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:34:22.207 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:34:22.207 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:34:22.207 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:34:22.207 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:34:22.207 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:34:22.207 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:34:22.207 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:34:22.207 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:34:22.209 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:34:22.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:34:22.209 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-27T02:34:22.242 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:34:22.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.420+0000 7f5174546700 1 Processor -- start 2024-01-27T02:34:22.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.420+0000 7f5174546700 1 -- start start 2024-01-27T02:34:22.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.421+0000 7f5174546700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f516c15b740 0x7f516c0c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:22.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.421+0000 7f5174546700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f516c0c4a30 con 0x7f516c0cc8e0 2024-01-27T02:34:22.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.421+0000 7f5174546700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f516c15bb10 con 0x7f516c0c55d0 2024-01-27T02:34:22.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.421+0000 7f5174546700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f516c0c3e20 con 0x7f516c15b740 2024-01-27T02:34:22.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.421+0000 7f51722e2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f516c15b740 0x7f516c0c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:22.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.421+0000 7f51722e2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f516c15b740 0x7f516c0c5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56868/0 (socket says 172.21.15.7:56868) 2024-01-27T02:34:22.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.421+0000 7f5171ae1700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f516c0cc8e0 0x7f516c0c4ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40584/0 (socket says 172.21.15.7:40584) 2024-01-27T02:34:22.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.421+0000 7f51722e2700 1 -- 172.21.15.7:0/3192209591 learned_addr learned my addr 172.21.15.7:0/3192209591 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:22.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.421+0000 7f51722e2700 1 -- 172.21.15.7:0/3192209591 >> v1:172.21.15.188:6789/0 conn(0x7f516c0c55d0 legacy=0x7f516c0cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:34:22.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.422+0000 7f51722e2700 1 -- 172.21.15.7:0/3192209591 >> v1:172.21.15.7:6789/0 conn(0x7f516c0cc8e0 legacy=0x7f516c0c4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:22.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.422+0000 7f51712e0700 1 -- 172.21.15.7:0/3192209591 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 96302739 0 0) 0x7f516c0c4a30 con 0x7f516c0cc8e0 2024-01-27T02:34:22.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.422+0000 7f51722e2700 1 -- 172.21.15.7:0/3192209591 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516c0c4100 con 0x7f516c15b740 2024-01-27T02:34:22.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.422+0000 7f51722e2700 1 --2- 172.21.15.7:0/3192209591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f516c15b740 0x7f516c0c5dc0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f51600099e0 tx=0x7f5160009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d44771e9ebfbc0c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:22.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.422+0000 7f51712e0700 1 -- 172.21.15.7:0/3192209591 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516000b820 con 0x7f516c15b740 2024-01-27T02:34:22.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.422+0000 7f51712e0700 1 -- 172.21.15.7:0/3192209591 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516000b980 con 0x7f516c15b740 2024-01-27T02:34:22.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.422+0000 7f51712e0700 1 -- 172.21.15.7:0/3192209591 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516000bd20 con 0x7f516c15b740 2024-01-27T02:34:22.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.422+0000 7f5174546700 1 -- 172.21.15.7:0/3192209591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f516c15b740 msgr2=0x7f516c0c5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:22.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.422+0000 7f5174546700 1 --2- 172.21.15.7:0/3192209591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f516c15b740 0x7f516c0c5dc0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f51600099e0 tx=0x7f5160009250 comp rx=0 tx=0).stop 2024-01-27T02:34:22.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.423+0000 7f5174546700 1 -- 172.21.15.7:0/3192209591 shutdown_connections 2024-01-27T02:34:22.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.423+0000 7f5174546700 1 --2- 172.21.15.7:0/3192209591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f516c15b740 0x7f516c0c5dc0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:22.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.423+0000 7f5174546700 1 -- 172.21.15.7:0/3192209591 >> 172.21.15.7:0/3192209591 conn(0x7f516c063d60 msgr2=0x7f516c0ced10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:22.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.423+0000 7f5174546700 1 -- 172.21.15.7:0/3192209591 shutdown_connections 2024-01-27T02:34:22.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.423+0000 7f5174546700 1 -- 172.21.15.7:0/3192209591 wait complete. 2024-01-27T02:34:22.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.424+0000 7f5174546700 1 Processor -- start 2024-01-27T02:34:22.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.424+0000 7f5174546700 1 -- start start 2024-01-27T02:34:22.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.425+0000 7f5174546700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f516c0c55d0 0x7f516c0c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:22.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.425+0000 7f5174546700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f516c0cc8e0 0x7f516c14d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:22.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.425+0000 7f5174546700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f516c15b740 0x7f516c1555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.425+0000 7f5174546700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c0c8c20 con 0x7f516c15b740 2024-01-27T02:34:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.425+0000 7f5174546700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c0c75d0 con 0x7f516c0c55d0 2024-01-27T02:34:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.425+0000 7f5174546700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516c0b26c0 con 0x7f516c0cc8e0 2024-01-27T02:34:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.425+0000 7f5172ae3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f516c15b740 0x7f516c1555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.425+0000 7f5172ae3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f516c15b740 0x7f516c1555a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56884/0 (socket says 172.21.15.7:56884) 2024-01-27T02:34:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.425+0000 7f5172ae3700 1 -- 172.21.15.7:0/2694693144 learned_addr learned my addr 172.21.15.7:0/2694693144 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.425+0000 7f5171ae1700 1 --2- 172.21.15.7:0/2694693144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f516c0cc8e0 0x7f516c14d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.425+0000 7f51722e2700 1 --2- 172.21.15.7:0/2694693144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f516c0c55d0 0x7f516c0c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.425+0000 7f5172ae3700 1 -- 172.21.15.7:0/2694693144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f516c0cc8e0 msgr2=0x7f516c14d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.425+0000 7f5172ae3700 1 --2- 172.21.15.7:0/2694693144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f516c0cc8e0 0x7f516c14d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.425+0000 7f5172ae3700 1 -- 172.21.15.7:0/2694693144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f516c0c55d0 msgr2=0x7f516c0c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.425+0000 7f5172ae3700 1 --2- 172.21.15.7:0/2694693144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f516c0c55d0 0x7f516c0c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.425+0000 7f5172ae3700 1 -- 172.21.15.7:0/2694693144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51600094f0 con 0x7f516c15b740 2024-01-27T02:34:22.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.425+0000 7f5172ae3700 1 --2- 172.21.15.7:0/2694693144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f516c15b740 0x7f516c1555a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f516400c8e0 tx=0x7f5164001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:22.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.425+0000 7f515e7fc700 1 -- 172.21.15.7:0/2694693144 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516400ce80 con 0x7f516c15b740 2024-01-27T02:34:22.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.425+0000 7f515e7fc700 1 -- 172.21.15.7:0/2694693144 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516400a7e0 con 0x7f516c15b740 2024-01-27T02:34:22.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.426+0000 7f5174546700 1 -- 172.21.15.7:0/2694693144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516c0c6f80 con 0x7f516c15b740 2024-01-27T02:34:22.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.426+0000 7f515e7fc700 1 -- 172.21.15.7:0/2694693144 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516400ab80 con 0x7f516c15b740 2024-01-27T02:34:22.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.426+0000 7f5174546700 1 -- 172.21.15.7:0/2694693144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516c0b29a0 con 0x7f516c15b740 2024-01-27T02:34:22.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.427+0000 7f5174546700 1 -- 172.21.15.7:0/2694693144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f516c04c710 con 0x7f516c15b740 2024-01-27T02:34:22.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.427+0000 7f515e7fc700 1 -- 172.21.15.7:0/2694693144 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5164012420 con 0x7f516c15b740 2024-01-27T02:34:22.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.428+0000 7f515e7fc700 1 --2- 172.21.15.7:0/2694693144 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5150066f30 0x7f51500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:22.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.428+0000 7f515e7fc700 1 -- 172.21.15.7:0/2694693144 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5164084880 con 0x7f516c15b740 2024-01-27T02:34:22.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.428+0000 7f51722e2700 1 --2- 172.21.15.7:0/2694693144 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5150066f30 0x7f51500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:22.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.428+0000 7f51722e2700 1 --2- 172.21.15.7:0/2694693144 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5150066f30 0x7f51500693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f51600099e0 tx=0x7f5160017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:22.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.432+0000 7f515e7fc700 1 -- 172.21.15.7:0/2694693144 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5164051af0 con 0x7f516c15b740 2024-01-27T02:34:22.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.689+0000 7f5174546700 1 -- 172.21.15.7:0/2694693144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f516c0c43e0 con 0x7f516c15b740 2024-01-27T02:34:22.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.689+0000 7f515e7fc700 1 -- 172.21.15.7:0/2694693144 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5164055110 con 0x7f516c15b740 2024-01-27T02:34:22.692 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:34:22.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.691+0000 7f5174546700 1 -- 172.21.15.7:0/2694693144 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5150066f30 msgr2=0x7f51500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:22.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.691+0000 7f5174546700 1 --2- 172.21.15.7:0/2694693144 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5150066f30 0x7f51500693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f51600099e0 tx=0x7f5160017000 comp rx=0 tx=0).stop 2024-01-27T02:34:22.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.691+0000 7f5174546700 1 -- 172.21.15.7:0/2694693144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f516c15b740 msgr2=0x7f516c1555a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:22.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.691+0000 7f5174546700 1 --2- 172.21.15.7:0/2694693144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f516c15b740 0x7f516c1555a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f516400c8e0 tx=0x7f5164001710 comp rx=0 tx=0).stop 2024-01-27T02:34:22.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.691+0000 7f5174546700 1 -- 172.21.15.7:0/2694693144 shutdown_connections 2024-01-27T02:34:22.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.691+0000 7f5174546700 1 --2- 172.21.15.7:0/2694693144 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5150066f30 0x7f51500693e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:22.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.691+0000 7f5174546700 1 --2- 172.21.15.7:0/2694693144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f516c15b740 0x7f516c1555a0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:22.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.691+0000 7f5174546700 1 --2- 172.21.15.7:0/2694693144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f516c0c55d0 0x7f516c0c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:22.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.691+0000 7f5174546700 1 --2- 172.21.15.7:0/2694693144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f516c0cc8e0 0x7f516c14d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:22.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.691+0000 7f5174546700 1 -- 172.21.15.7:0/2694693144 >> 172.21.15.7:0/2694693144 conn(0x7f516c063d60 msgr2=0x7f516c159460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:22.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.692+0000 7f5174546700 1 -- 172.21.15.7:0/2694693144 shutdown_connections 2024-01-27T02:34:22.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:22.692+0000 7f5174546700 1 -- 172.21.15.7:0/2694693144 wait complete. 2024-01-27T02:34:22.706 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-27T02:34:22.707 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:34:22.707 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-01-27T02:34:22.707 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-27T02:34:22.707 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-27T02:34:22.707 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-27T02:34:22.707 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-27T02:34:22.707 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:34:22.707 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:34:22.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:34:22.707 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-01-27T02:34:22.729 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:34:22.729 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:34:22.729 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:34:22.729 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:34:22.729 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-01-27T02:34:22.729 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:34:22.729 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:34:22.729 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:34:22.729 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-01-27T02:34:22.729 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:34:22.730 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:34:22.730 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-01-27T02:34:22.730 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:34:22.730 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:34:22.730 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:34:22.730 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:34:22.730 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-27T02:34:22.730 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42256sec preferred_lft 42256sec 2024-01-27T02:34:22.730 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b1:d6f7:baf0:bc17/64 scope link noprefixroute 2024-01-27T02:34:22.730 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:34:22.730 INFO:teuthology.orchestra.run.smithi188.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:34:22.730 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b0:92:71:90:9d brd ff:ff:ff:ff:ff:ff 2024-01-27T02:34:22.730 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-27T02:34:22.730 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:34:22.730 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::10f8:32ff:feed:a752/64 scope link 2024-01-27T02:34:22.731 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:34:22.731 INFO:teuthology.orchestra.run.smithi188.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:34:22.731 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b0:92:71:90:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-27T02:34:22.731 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b8b0:92ff:fe71:909d/64 scope link 2024-01-27T02:34:22.731 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:34:22.731 INFO:teuthology.orchestra.run.smithi188.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:34:22.731 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ee:78:03:20:3a:a9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-27T02:34:22.731 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec78:3ff:fe20:3aa9/64 scope link 2024-01-27T02:34:22.731 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:34:22.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:34:22.731 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:34:22.800 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-01-27T02:34:22.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:34:22.802 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-01-27T02:34:22.862 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-01-27T02:34:22.863 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:34:22.863 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-27T02:34:22.863 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-27T02:34:22.863 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:34:22.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:34:22.864 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-27T02:34:22.864 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-27T02:34:22.864 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-27T02:34:22.864 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-27T02:34:22.864 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-27T02:34:22.864 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-27T02:34:22.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:34:22.975 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-01-27T02:34:22.992 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-27T02:34:22.992 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-27T02:34:22.992 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:34:22.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:34:22.993 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2024-01-27T02:34:23.060 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:34:23.060 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-27T02:34:23.117 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:34:23.117 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:34:23.117 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-27T02:34:23.173 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:34:23.174 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:34:23.174 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:34:23.256 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:34:23.256 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:34:23.256 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:34:23.264 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:34:23.264 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:34:23.264 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:34:23.264 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:34:23.264 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:34:23.282 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:34:23.282 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:34:23.282 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:34:23.282 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:34:23.282 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:34:23.282 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:34:23.282 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:34:23.282 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:34:23.284 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:34:23.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:34:23.285 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-27T02:34:23.353 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:34:23.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.492+0000 7f245895e700 1 Processor -- start 2024-01-27T02:34:23.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.492+0000 7f245895e700 1 -- start start 2024-01-27T02:34:23.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.493+0000 7f245895e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f245406fa70 0x7f2454165280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:23.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.493+0000 7f245895e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f245415a880 con 0x7f24540a7fe0 2024-01-27T02:34:23.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.493+0000 7f245895e700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24540af950 con 0x7f24540a7c10 2024-01-27T02:34:23.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.493+0000 7f245895e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24540afde0 con 0x7f245406fa70 2024-01-27T02:34:23.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.493+0000 7f2451d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f24540a7fe0 0x7f245406f360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40598/0 (socket says 172.21.15.7:40598) 2024-01-27T02:34:23.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.493+0000 7f2451d9b700 1 -- 172.21.15.7:0/2940387204 learned_addr learned my addr 172.21.15.7:0/2940387204 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:34:23.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.493+0000 7f2452d9d700 1 --2- 172.21.15.7:0/2940387204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f245406fa70 0x7f2454165280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:23.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.493+0000 7f245159a700 1 -- 172.21.15.7:0/2940387204 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2960587952 0 0) 0x7f245415a880 con 0x7f24540a7fe0 2024-01-27T02:34:23.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.493+0000 7f245159a700 1 -- 172.21.15.7:0/2940387204 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2440003570 con 0x7f24540a7fe0 2024-01-27T02:34:23.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.493+0000 7f245159a700 1 -- 172.21.15.7:0/2940387204 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4091213236 0 0) 0x7f24540af950 con 0x7f24540a7c10 2024-01-27T02:34:23.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.494+0000 7f245159a700 1 -- 172.21.15.7:0/2940387204 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f245415a880 con 0x7f24540a7c10 2024-01-27T02:34:23.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.494+0000 7f2452d9d700 1 -- 172.21.15.7:0/2940387204 >> v1:172.21.15.188:6789/0 conn(0x7f24540a7c10 legacy=0x7f24540ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:23.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.494+0000 7f2452d9d700 1 -- 172.21.15.7:0/2940387204 >> v1:172.21.15.7:6789/0 conn(0x7f24540a7fe0 legacy=0x7f245406f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:23.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.494+0000 7f2452d9d700 1 -- 172.21.15.7:0/2940387204 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24540aed40 con 0x7f245406fa70 2024-01-27T02:34:23.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.494+0000 7f2452d9d700 1 --2- 172.21.15.7:0/2940387204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f245406fa70 0x7f2454165280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f244c009e90 tx=0x7f244c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=457aa079d5942c6 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:23.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.494+0000 7f245159a700 1 -- 172.21.15.7:0/2940387204 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c00e070 con 0x7f245406fa70 2024-01-27T02:34:23.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.494+0000 7f245159a700 1 -- 172.21.15.7:0/2940387204 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244c00bb80 con 0x7f245406fa70 2024-01-27T02:34:23.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.494+0000 7f245159a700 1 -- 172.21.15.7:0/2940387204 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c012430 con 0x7f245406fa70 2024-01-27T02:34:23.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.495+0000 7f245895e700 1 -- 172.21.15.7:0/2940387204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f245406fa70 msgr2=0x7f2454165280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:23.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.495+0000 7f245895e700 1 --2- 172.21.15.7:0/2940387204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f245406fa70 0x7f2454165280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f244c009e90 tx=0x7f244c009780 comp rx=0 tx=0).stop 2024-01-27T02:34:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.495+0000 7f245895e700 1 -- 172.21.15.7:0/2940387204 shutdown_connections 2024-01-27T02:34:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.495+0000 7f245895e700 1 --2- 172.21.15.7:0/2940387204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f245406fa70 0x7f2454165280 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.495+0000 7f245895e700 1 -- 172.21.15.7:0/2940387204 >> 172.21.15.7:0/2940387204 conn(0x7f24540679b0 msgr2=0x7f2454067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.495+0000 7f245895e700 1 -- 172.21.15.7:0/2940387204 shutdown_connections 2024-01-27T02:34:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.495+0000 7f245895e700 1 -- 172.21.15.7:0/2940387204 wait complete. 2024-01-27T02:34:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.496+0000 7f245895e700 1 Processor -- start 2024-01-27T02:34:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.496+0000 7f245895e700 1 -- start start 2024-01-27T02:34:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.497+0000 7f245895e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f245406fa70 0x7f245415aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.497+0000 7f245895e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24540a7c10 0x7f2454098aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.497+0000 7f245895e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24540a7fe0 0x7f2454155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.497+0000 7f245895e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245409a810 con 0x7f24540a7c10 2024-01-27T02:34:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.497+0000 7f245895e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2454155a70 con 0x7f24540a7fe0 2024-01-27T02:34:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.497+0000 7f245895e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f245415b0a0 con 0x7f245406fa70 2024-01-27T02:34:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.497+0000 7f2451d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24540a7c10 0x7f2454098aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.497+0000 7f2451d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24540a7c10 0x7f2454098aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56900/0 (socket says 172.21.15.7:56900) 2024-01-27T02:34:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.497+0000 7f2451d9b700 1 -- 172.21.15.7:0/2840032236 learned_addr learned my addr 172.21.15.7:0/2840032236 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:34:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.497+0000 7f2452d9d700 1 --2- 172.21.15.7:0/2840032236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24540a7fe0 0x7f2454155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.497+0000 7f245259c700 1 --2- 172.21.15.7:0/2840032236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f245406fa70 0x7f245415aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.497+0000 7f2451d9b700 1 -- 172.21.15.7:0/2840032236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f245406fa70 msgr2=0x7f245415aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.497+0000 7f2451d9b700 1 --2- 172.21.15.7:0/2840032236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f245406fa70 0x7f245415aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.497+0000 7f2451d9b700 1 -- 172.21.15.7:0/2840032236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24540a7fe0 msgr2=0x7f2454155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.497+0000 7f2451d9b700 1 --2- 172.21.15.7:0/2840032236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24540a7fe0 0x7f2454155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.497+0000 7f2451d9b700 1 -- 172.21.15.7:0/2840032236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c009a20 con 0x7f24540a7c10 2024-01-27T02:34:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.497+0000 7f245259c700 1 --2- 172.21.15.7:0/2840032236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f245406fa70 0x7f245415aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:34:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.497+0000 7f2451d9b700 1 --2- 172.21.15.7:0/2840032236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24540a7c10 0x7f2454098aa0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f244400c7c0 tx=0x7f2444007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.498+0000 7f244a7fc700 1 -- 172.21.15.7:0/2840032236 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2444005b30 con 0x7f24540a7c10 2024-01-27T02:34:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.498+0000 7f244a7fc700 1 -- 172.21.15.7:0/2840032236 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2444005c90 con 0x7f24540a7c10 2024-01-27T02:34:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.498+0000 7f245895e700 1 -- 172.21.15.7:0/2840032236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245415b380 con 0x7f24540a7c10 2024-01-27T02:34:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.498+0000 7f244a7fc700 1 -- 172.21.15.7:0/2840032236 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24440072e0 con 0x7f24540a7c10 2024-01-27T02:34:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.498+0000 7f245895e700 1 -- 172.21.15.7:0/2840032236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f245415b940 con 0x7f24540a7c10 2024-01-27T02:34:23.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.499+0000 7f245895e700 1 -- 172.21.15.7:0/2840032236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2454158ac0 con 0x7f24540a7c10 2024-01-27T02:34:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.499+0000 7f244a7fc700 1 -- 172.21.15.7:0/2840032236 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f244401b3b0 con 0x7f24540a7c10 2024-01-27T02:34:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.500+0000 7f244a7fc700 1 --2- 172.21.15.7:0/2840032236 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2434066f30 0x7f24340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:34:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.500+0000 7f244a7fc700 1 -- 172.21.15.7:0/2840032236 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2444083aa0 con 0x7f24540a7c10 2024-01-27T02:34:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.500+0000 7f245259c700 1 --2- 172.21.15.7:0/2840032236 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2434066f30 0x7f24340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:34:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.500+0000 7f245259c700 1 --2- 172.21.15.7:0/2840032236 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2434066f30 0x7f24340693e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f243c000c00 tx=0x7f243c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:34:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.504+0000 7f244a7fc700 1 -- 172.21.15.7:0/2840032236 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f244404b5a0 con 0x7f24540a7c10 2024-01-27T02:34:23.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.754+0000 7f245895e700 1 -- 172.21.15.7:0/2840032236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f245404c710 con 0x7f24540a7c10 2024-01-27T02:34:23.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.754+0000 7f244a7fc700 1 -- 172.21.15.7:0/2840032236 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f2444017020 con 0x7f24540a7c10 2024-01-27T02:34:23.756 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:34:23.756 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5114,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/2839704447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":2839704447},{"type":"v1","addr":"172.21.15.188:6837","nonce":2839704447}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5117,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3964291855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3964291855},{"type":"v1","addr":"172.21.15.7:6834","nonce":3964291855}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/3332325272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3332325272},{"type":"v1","addr":"172.21.15.188:6836","nonce":3332325272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:34:14.471429+0000","modified":"2024-01-27T02:34:18.068858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1399055016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1399055016},{"type":"v1","addr":"172.21.15.7:6835","nonce":1399055016}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2024-01-27T02:34:23.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.756+0000 7f245895e700 1 -- 172.21.15.7:0/2840032236 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2434066f30 msgr2=0x7f24340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:23.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.756+0000 7f245895e700 1 --2- 172.21.15.7:0/2840032236 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2434066f30 0x7f24340693e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f243c000c00 tx=0x7f243c009040 comp rx=0 tx=0).stop 2024-01-27T02:34:23.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.756+0000 7f245895e700 1 -- 172.21.15.7:0/2840032236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24540a7c10 msgr2=0x7f2454098aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:34:23.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.756+0000 7f245895e700 1 --2- 172.21.15.7:0/2840032236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24540a7c10 0x7f2454098aa0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f244400c7c0 tx=0x7f2444007e40 comp rx=0 tx=0).stop 2024-01-27T02:34:23.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.757+0000 7f245895e700 1 -- 172.21.15.7:0/2840032236 shutdown_connections 2024-01-27T02:34:23.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.757+0000 7f245895e700 1 --2- 172.21.15.7:0/2840032236 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2434066f30 0x7f24340693e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:23.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.757+0000 7f245895e700 1 --2- 172.21.15.7:0/2840032236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24540a7fe0 0x7f2454155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:23.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.757+0000 7f245895e700 1 --2- 172.21.15.7:0/2840032236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f245406fa70 0x7f245415aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:23.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.757+0000 7f245895e700 1 --2- 172.21.15.7:0/2840032236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24540a7c10 0x7f2454098aa0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:34:23.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.757+0000 7f245895e700 1 -- 172.21.15.7:0/2840032236 >> 172.21.15.7:0/2840032236 conn(0x7f24540679b0 msgr2=0x7f2454069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:34:23.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.757+0000 7f245895e700 1 -- 172.21.15.7:0/2840032236 shutdown_connections 2024-01-27T02:34:23.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:34:23.757+0000 7f245895e700 1 -- 172.21.15.7:0/2840032236 wait complete. 2024-01-27T02:34:23.760 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2024-01-27T02:34:23.772 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-01-27T02:34:23.926 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","whoami":0,"id":5631,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.8593690620000007,"mdsmap_epoch":34,"osdmap_epoch":65,"osdmap_epoch_barrier":65,"uptime":14.617130081999999} 2024-01-27T02:34:23.926 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", "fs_name": "cephfs", "id": 5631, "mdsmap_epoch": 34, "osdmap_epoch": 65, "osdmap_epoch_barrier": 65, "rank_uptime": 8.859369062, "state": "up:active", "uptime": 14.617130082, "want_state": "up:active", "whoami": 0 } 2024-01-27T02:34:23.927 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-01-27T02:34:24.075 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_mon_report_interval":"5"} 2024-01-27T02:34:24.075 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-01-27T02:34:24.076 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-01-27T02:34:24.218 INFO:teuthology.orchestra.run.smithi007.stdout:{"mon_tick_interval":"5"} 2024-01-27T02:34:24.218 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-01-27T02:34:24.218 INFO:tasks.cephfs.test_full:pool capacity 557082176, 531MB should be enough to fill it 2024-01-27T02:34:24.219 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-01-27T02:34:24.219 DEBUG:teuthology.orchestra.run.smithi007:> import time 2024-01-27T02:34:24.219 DEBUG:teuthology.orchestra.run.smithi007:> import datetime 2024-01-27T02:34:24.219 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2024-01-27T02:34:24.219 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-01-27T02:34:24.219 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:34:24.219 DEBUG:teuthology.orchestra.run.smithi007:> # Write some buffered data through before going full, all should be well 2024-01-27T02:34:24.219 DEBUG:teuthology.orchestra.run.smithi007:> print("writing some data through which we expect to succeed") 2024-01-27T02:34:24.219 DEBUG:teuthology.orchestra.run.smithi007:> bytes = 0 2024-01-27T02:34:24.219 DEBUG:teuthology.orchestra.run.smithi007:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-01-27T02:34:24.219 DEBUG:teuthology.orchestra.run.smithi007:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-01-27T02:34:24.220 DEBUG:teuthology.orchestra.run.smithi007:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2024-01-27T02:34:24.220 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f) 2024-01-27T02:34:24.220 DEBUG:teuthology.orchestra.run.smithi007:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-01-27T02:34:24.220 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:34:24.220 DEBUG:teuthology.orchestra.run.smithi007:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-01-27T02:34:24.220 DEBUG:teuthology.orchestra.run.smithi007:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-01-27T02:34:24.220 DEBUG:teuthology.orchestra.run.smithi007:> # buffered IO, the error should always only appear from fsync and not 2024-01-27T02:34:24.220 DEBUG:teuthology.orchestra.run.smithi007:> # from write 2024-01-27T02:34:24.220 DEBUG:teuthology.orchestra.run.smithi007:> full = False 2024-01-27T02:34:24.220 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:34:24.220 DEBUG:teuthology.orchestra.run.smithi007:> for n in range(0, int(531 * 0.9)): 2024-01-27T02:34:24.220 DEBUG:teuthology.orchestra.run.smithi007:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-01-27T02:34:24.220 DEBUG:teuthology.orchestra.run.smithi007:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2024-01-27T02:34:24.221 DEBUG:teuthology.orchestra.run.smithi007:> print("done writing {0} bytes".format(bytes)) 2024-01-27T02:34:24.221 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:34:24.221 DEBUG:teuthology.orchestra.run.smithi007:> # OK, now we should sneak in under the full condition 2024-01-27T02:34:24.221 DEBUG:teuthology.orchestra.run.smithi007:> # due to the time it takes the OSDs to report to the 2024-01-27T02:34:24.221 DEBUG:teuthology.orchestra.run.smithi007:> # mons, and get a successful fsync on our full-making data 2024-01-27T02:34:24.221 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f) 2024-01-27T02:34:24.221 DEBUG:teuthology.orchestra.run.smithi007:> print("successfully fsync'"'"'ed prior to getting full state reported") 2024-01-27T02:34:24.221 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:34:24.221 DEBUG:teuthology.orchestra.run.smithi007:> # buffered write, add more dirty data to the buffer 2024-01-27T02:34:24.221 DEBUG:teuthology.orchestra.run.smithi007:> print("starting buffered write") 2024-01-27T02:34:24.221 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-01-27T02:34:24.221 DEBUG:teuthology.orchestra.run.smithi007:> for n in range(0, int(531 * 0.2)): 2024-01-27T02:34:24.221 DEBUG:teuthology.orchestra.run.smithi007:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-01-27T02:34:24.221 DEBUG:teuthology.orchestra.run.smithi007:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-01-27T02:34:24.222 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(15.0) 2024-01-27T02:34:24.222 DEBUG:teuthology.orchestra.run.smithi007:> except OSError: 2024-01-27T02:34:24.222 DEBUG:teuthology.orchestra.run.smithi007:> pass; 2024-01-27T02:34:24.222 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:34:24.222 DEBUG:teuthology.orchestra.run.smithi007:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2024-01-27T02:34:24.222 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:34:24.222 DEBUG:teuthology.orchestra.run.smithi007:> # Wait long enough for a background flush that should fail 2024-01-27T02:34:24.222 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(30) 2024-01-27T02:34:24.222 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:34:24.222 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-01-27T02:34:24.222 DEBUG:teuthology.orchestra.run.smithi007:> # ...and check that the failed background flush is reflected in fclose 2024-01-27T02:34:24.222 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-01-27T02:34:24.222 DEBUG:teuthology.orchestra.run.smithi007:> os.close(f) 2024-01-27T02:34:24.223 DEBUG:teuthology.orchestra.run.smithi007:> except OSError: 2024-01-27T02:34:24.223 DEBUG:teuthology.orchestra.run.smithi007:> print("close() returned an error as expected") 2024-01-27T02:34:24.223 DEBUG:teuthology.orchestra.run.smithi007:> else: 2024-01-27T02:34:24.223 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("close() failed to raise error") 2024-01-27T02:34:24.223 DEBUG:teuthology.orchestra.run.smithi007:> else: 2024-01-27T02:34:24.223 DEBUG:teuthology.orchestra.run.smithi007:> # The kernel cephfs client does not raise errors on fclose 2024-01-27T02:34:24.223 DEBUG:teuthology.orchestra.run.smithi007:> os.close(f) 2024-01-27T02:34:24.223 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:34:24.223 DEBUG:teuthology.orchestra.run.smithi007:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-01-27T02:34:24.223 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-01-27T02:34:29.379 INFO:teuthology.orchestra.run.smithi007.stdout:writing some data through which we expect to succeed 2024-01-27T02:34:29.379 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-01-27T02:34:29.379 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 1572864 bytes via buffered write, may repeat 2024-01-27T02:34:29.380 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 2621440 bytes via buffered write, may repeat 2024-01-27T02:34:29.380 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 3670016 bytes via buffered write, may repeat 2024-01-27T02:34:29.380 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 4718592 bytes via buffered write, may repeat 2024-01-27T02:34:29.380 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 5767168 bytes via buffered write, may repeat 2024-01-27T02:34:29.380 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 6815744 bytes via buffered write, may repeat 2024-01-27T02:34:29.380 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 7864320 bytes via buffered write, may repeat 2024-01-27T02:34:29.380 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 8912896 bytes via buffered write, may repeat 2024-01-27T02:34:29.380 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 9961472 bytes via buffered write, may repeat 2024-01-27T02:34:29.380 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 11010048 bytes via buffered write, may repeat 2024-01-27T02:34:29.380 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 12058624 bytes via buffered write, may repeat 2024-01-27T02:34:29.380 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 13107200 bytes via buffered write, may repeat 2024-01-27T02:34:29.380 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 14155776 bytes via buffered write, may repeat 2024-01-27T02:34:29.380 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 15204352 bytes via buffered write, may repeat 2024-01-27T02:34:29.381 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 16252928 bytes via buffered write, may repeat 2024-01-27T02:34:29.381 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 17301504 bytes via buffered write, may repeat 2024-01-27T02:34:29.381 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 18350080 bytes via buffered write, may repeat 2024-01-27T02:34:29.381 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 19398656 bytes via buffered write, may repeat 2024-01-27T02:34:29.381 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 20447232 bytes via buffered write, may repeat 2024-01-27T02:34:29.381 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 21495808 bytes via buffered write, may repeat 2024-01-27T02:34:29.381 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 22544384 bytes via buffered write, may repeat 2024-01-27T02:34:29.381 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 23592960 bytes via buffered write, may repeat 2024-01-27T02:34:29.381 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 24641536 bytes via buffered write, may repeat 2024-01-27T02:34:29.381 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 25690112 bytes via buffered write, may repeat 2024-01-27T02:34:29.381 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 26738688 bytes via buffered write, may repeat 2024-01-27T02:34:29.381 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 27787264 bytes via buffered write, may repeat 2024-01-27T02:34:29.381 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 28835840 bytes via buffered write, may repeat 2024-01-27T02:34:29.381 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 29884416 bytes via buffered write, may repeat 2024-01-27T02:34:29.381 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 30932992 bytes via buffered write, may repeat 2024-01-27T02:34:29.382 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 31981568 bytes via buffered write, may repeat 2024-01-27T02:34:29.382 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 33030144 bytes via buffered write, may repeat 2024-01-27T02:34:29.382 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 34078720 bytes via buffered write, may repeat 2024-01-27T02:34:29.382 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 35127296 bytes via buffered write, may repeat 2024-01-27T02:34:29.382 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 36175872 bytes via buffered write, may repeat 2024-01-27T02:34:29.382 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 37224448 bytes via buffered write, may repeat 2024-01-27T02:34:29.382 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 38273024 bytes via buffered write, may repeat 2024-01-27T02:34:29.382 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 39321600 bytes via buffered write, may repeat 2024-01-27T02:34:29.382 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 40370176 bytes via buffered write, may repeat 2024-01-27T02:34:29.382 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 41418752 bytes via buffered write, may repeat 2024-01-27T02:34:29.382 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 42467328 bytes via buffered write, may repeat 2024-01-27T02:34:29.382 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 43515904 bytes via buffered write, may repeat 2024-01-27T02:34:29.382 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 44564480 bytes via buffered write, may repeat 2024-01-27T02:34:29.383 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 45613056 bytes via buffered write, may repeat 2024-01-27T02:34:29.383 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 46661632 bytes via buffered write, may repeat 2024-01-27T02:34:29.383 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 47710208 bytes via buffered write, may repeat 2024-01-27T02:34:29.383 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 48758784 bytes via buffered write, may repeat 2024-01-27T02:34:29.383 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 49807360 bytes via buffered write, may repeat 2024-01-27T02:34:29.383 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 50855936 bytes via buffered write, may repeat 2024-01-27T02:34:29.383 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 51904512 bytes via buffered write, may repeat 2024-01-27T02:34:29.383 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 52953088 bytes via buffered write, may repeat 2024-01-27T02:34:29.383 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 54001664 bytes via buffered write, may repeat 2024-01-27T02:34:29.383 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 55050240 bytes via buffered write, may repeat 2024-01-27T02:34:29.383 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 56098816 bytes via buffered write, may repeat 2024-01-27T02:34:29.383 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 57147392 bytes via buffered write, may repeat 2024-01-27T02:34:29.383 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 58195968 bytes via buffered write, may repeat 2024-01-27T02:34:29.383 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 59244544 bytes via buffered write, may repeat 2024-01-27T02:34:29.383 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 60293120 bytes via buffered write, may repeat 2024-01-27T02:34:29.383 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 61341696 bytes via buffered write, may repeat 2024-01-27T02:34:29.384 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 62390272 bytes via buffered write, may repeat 2024-01-27T02:34:29.384 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 63438848 bytes via buffered write, may repeat 2024-01-27T02:34:29.384 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 64487424 bytes via buffered write, may repeat 2024-01-27T02:34:29.384 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 65536000 bytes via buffered write, may repeat 2024-01-27T02:34:29.384 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 66584576 bytes via buffered write, may repeat 2024-01-27T02:34:29.384 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 67633152 bytes via buffered write, may repeat 2024-01-27T02:34:29.384 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 68681728 bytes via buffered write, may repeat 2024-01-27T02:34:29.384 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 69730304 bytes via buffered write, may repeat 2024-01-27T02:34:29.384 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 70778880 bytes via buffered write, may repeat 2024-01-27T02:34:29.384 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 71827456 bytes via buffered write, may repeat 2024-01-27T02:34:29.384 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 72876032 bytes via buffered write, may repeat 2024-01-27T02:34:29.384 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 73924608 bytes via buffered write, may repeat 2024-01-27T02:34:29.384 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 74973184 bytes via buffered write, may repeat 2024-01-27T02:34:29.385 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 76021760 bytes via buffered write, may repeat 2024-01-27T02:34:29.385 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 77070336 bytes via buffered write, may repeat 2024-01-27T02:34:29.385 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 78118912 bytes via buffered write, may repeat 2024-01-27T02:34:29.385 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 79167488 bytes via buffered write, may repeat 2024-01-27T02:34:29.385 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 80216064 bytes via buffered write, may repeat 2024-01-27T02:34:29.385 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 81264640 bytes via buffered write, may repeat 2024-01-27T02:34:29.385 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 82313216 bytes via buffered write, may repeat 2024-01-27T02:34:29.385 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 83361792 bytes via buffered write, may repeat 2024-01-27T02:34:29.385 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 84410368 bytes via buffered write, may repeat 2024-01-27T02:34:29.385 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 85458944 bytes via buffered write, may repeat 2024-01-27T02:34:29.386 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 86507520 bytes via buffered write, may repeat 2024-01-27T02:34:29.386 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 87556096 bytes via buffered write, may repeat 2024-01-27T02:34:29.386 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 88604672 bytes via buffered write, may repeat 2024-01-27T02:34:29.386 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 89653248 bytes via buffered write, may repeat 2024-01-27T02:34:29.386 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 90701824 bytes via buffered write, may repeat 2024-01-27T02:34:29.386 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 91750400 bytes via buffered write, may repeat 2024-01-27T02:34:29.386 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 92798976 bytes via buffered write, may repeat 2024-01-27T02:34:29.386 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 93847552 bytes via buffered write, may repeat 2024-01-27T02:34:29.386 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 94896128 bytes via buffered write, may repeat 2024-01-27T02:34:29.387 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 95944704 bytes via buffered write, may repeat 2024-01-27T02:34:29.387 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 96993280 bytes via buffered write, may repeat 2024-01-27T02:34:29.387 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 98041856 bytes via buffered write, may repeat 2024-01-27T02:34:29.387 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 99090432 bytes via buffered write, may repeat 2024-01-27T02:34:29.387 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 100139008 bytes via buffered write, may repeat 2024-01-27T02:34:29.387 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 101187584 bytes via buffered write, may repeat 2024-01-27T02:34:29.387 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 102236160 bytes via buffered write, may repeat 2024-01-27T02:34:29.387 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 103284736 bytes via buffered write, may repeat 2024-01-27T02:34:29.387 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 104333312 bytes via buffered write, may repeat 2024-01-27T02:34:29.387 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 105381888 bytes via buffered write, may repeat 2024-01-27T02:34:29.387 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 106430464 bytes via buffered write, may repeat 2024-01-27T02:34:29.387 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 107479040 bytes via buffered write, may repeat 2024-01-27T02:34:29.387 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 108527616 bytes via buffered write, may repeat 2024-01-27T02:34:29.387 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 109576192 bytes via buffered write, may repeat 2024-01-27T02:34:29.388 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 110624768 bytes via buffered write, may repeat 2024-01-27T02:34:29.388 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 111673344 bytes via buffered write, may repeat 2024-01-27T02:34:29.388 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 112721920 bytes via buffered write, may repeat 2024-01-27T02:34:29.388 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 113770496 bytes via buffered write, may repeat 2024-01-27T02:34:29.388 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 114819072 bytes via buffered write, may repeat 2024-01-27T02:34:29.388 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 115867648 bytes via buffered write, may repeat 2024-01-27T02:34:29.388 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 116916224 bytes via buffered write, may repeat 2024-01-27T02:34:29.388 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 117964800 bytes via buffered write, may repeat 2024-01-27T02:34:29.388 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 119013376 bytes via buffered write, may repeat 2024-01-27T02:34:29.388 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 120061952 bytes via buffered write, may repeat 2024-01-27T02:34:29.388 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 121110528 bytes via buffered write, may repeat 2024-01-27T02:34:29.388 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 122159104 bytes via buffered write, may repeat 2024-01-27T02:34:29.388 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 123207680 bytes via buffered write, may repeat 2024-01-27T02:34:29.388 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 124256256 bytes via buffered write, may repeat 2024-01-27T02:34:29.389 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 125304832 bytes via buffered write, may repeat 2024-01-27T02:34:29.389 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 126353408 bytes via buffered write, may repeat 2024-01-27T02:34:29.389 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 127401984 bytes via buffered write, may repeat 2024-01-27T02:34:29.389 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 128450560 bytes via buffered write, may repeat 2024-01-27T02:34:29.389 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 129499136 bytes via buffered write, may repeat 2024-01-27T02:34:29.389 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 130547712 bytes via buffered write, may repeat 2024-01-27T02:34:29.389 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 131596288 bytes via buffered write, may repeat 2024-01-27T02:34:29.389 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 132644864 bytes via buffered write, may repeat 2024-01-27T02:34:29.389 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 133693440 bytes via buffered write, may repeat 2024-01-27T02:34:29.389 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 134742016 bytes via buffered write, may repeat 2024-01-27T02:34:29.389 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 135790592 bytes via buffered write, may repeat 2024-01-27T02:34:29.389 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 136839168 bytes via buffered write, may repeat 2024-01-27T02:34:29.389 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 137887744 bytes via buffered write, may repeat 2024-01-27T02:34:29.389 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 138936320 bytes via buffered write, may repeat 2024-01-27T02:34:29.390 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 139984896 bytes via buffered write, may repeat 2024-01-27T02:34:29.390 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 141033472 bytes via buffered write, may repeat 2024-01-27T02:34:29.390 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 142082048 bytes via buffered write, may repeat 2024-01-27T02:34:29.390 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 143130624 bytes via buffered write, may repeat 2024-01-27T02:34:29.390 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 144179200 bytes via buffered write, may repeat 2024-01-27T02:34:29.390 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 145227776 bytes via buffered write, may repeat 2024-01-27T02:34:29.390 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 146276352 bytes via buffered write, may repeat 2024-01-27T02:34:29.390 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 147324928 bytes via buffered write, may repeat 2024-01-27T02:34:29.390 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 148373504 bytes via buffered write, may repeat 2024-01-27T02:34:29.390 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 149422080 bytes via buffered write, may repeat 2024-01-27T02:34:29.390 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 150470656 bytes via buffered write, may repeat 2024-01-27T02:34:29.390 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 151519232 bytes via buffered write, may repeat 2024-01-27T02:34:29.390 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 152567808 bytes via buffered write, may repeat 2024-01-27T02:34:29.390 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 153616384 bytes via buffered write, may repeat 2024-01-27T02:34:29.391 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 154664960 bytes via buffered write, may repeat 2024-01-27T02:34:29.391 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 155713536 bytes via buffered write, may repeat 2024-01-27T02:34:29.391 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 156762112 bytes via buffered write, may repeat 2024-01-27T02:34:29.391 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 157810688 bytes via buffered write, may repeat 2024-01-27T02:34:29.391 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 158859264 bytes via buffered write, may repeat 2024-01-27T02:34:29.391 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 159907840 bytes via buffered write, may repeat 2024-01-27T02:34:29.391 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 160956416 bytes via buffered write, may repeat 2024-01-27T02:34:29.391 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 162004992 bytes via buffered write, may repeat 2024-01-27T02:34:29.434 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 163053568 bytes via buffered write, may repeat 2024-01-27T02:34:29.435 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 164102144 bytes via buffered write, may repeat 2024-01-27T02:34:29.435 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 165150720 bytes via buffered write, may repeat 2024-01-27T02:34:29.435 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 166199296 bytes via buffered write, may repeat 2024-01-27T02:34:29.435 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 167247872 bytes via buffered write, may repeat 2024-01-27T02:34:29.435 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 168296448 bytes via buffered write, may repeat 2024-01-27T02:34:29.435 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 169345024 bytes via buffered write, may repeat 2024-01-27T02:34:29.435 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 170393600 bytes via buffered write, may repeat 2024-01-27T02:34:29.435 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 171442176 bytes via buffered write, may repeat 2024-01-27T02:34:29.435 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 172490752 bytes via buffered write, may repeat 2024-01-27T02:34:29.435 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 173539328 bytes via buffered write, may repeat 2024-01-27T02:34:29.435 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 174587904 bytes via buffered write, may repeat 2024-01-27T02:34:29.436 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 175636480 bytes via buffered write, may repeat 2024-01-27T02:34:29.436 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 176685056 bytes via buffered write, may repeat 2024-01-27T02:34:29.436 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 177733632 bytes via buffered write, may repeat 2024-01-27T02:34:29.436 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 178782208 bytes via buffered write, may repeat 2024-01-27T02:34:29.436 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 179830784 bytes via buffered write, may repeat 2024-01-27T02:34:29.436 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 180879360 bytes via buffered write, may repeat 2024-01-27T02:34:29.436 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 181927936 bytes via buffered write, may repeat 2024-01-27T02:34:29.436 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 182976512 bytes via buffered write, may repeat 2024-01-27T02:34:29.436 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 184025088 bytes via buffered write, may repeat 2024-01-27T02:34:29.436 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 185073664 bytes via buffered write, may repeat 2024-01-27T02:34:29.436 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 186122240 bytes via buffered write, may repeat 2024-01-27T02:34:29.436 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 187170816 bytes via buffered write, may repeat 2024-01-27T02:34:29.436 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 188219392 bytes via buffered write, may repeat 2024-01-27T02:34:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 189267968 bytes via buffered write, may repeat 2024-01-27T02:34:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 190316544 bytes via buffered write, may repeat 2024-01-27T02:34:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 191365120 bytes via buffered write, may repeat 2024-01-27T02:34:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 192413696 bytes via buffered write, may repeat 2024-01-27T02:34:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 193462272 bytes via buffered write, may repeat 2024-01-27T02:34:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 194510848 bytes via buffered write, may repeat 2024-01-27T02:34:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 195559424 bytes via buffered write, may repeat 2024-01-27T02:34:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 196608000 bytes via buffered write, may repeat 2024-01-27T02:34:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 197656576 bytes via buffered write, may repeat 2024-01-27T02:34:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 198705152 bytes via buffered write, may repeat 2024-01-27T02:34:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 199753728 bytes via buffered write, may repeat 2024-01-27T02:34:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 200802304 bytes via buffered write, may repeat 2024-01-27T02:34:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 201850880 bytes via buffered write, may repeat 2024-01-27T02:34:29.437 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 202899456 bytes via buffered write, may repeat 2024-01-27T02:34:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 203948032 bytes via buffered write, may repeat 2024-01-27T02:34:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 204996608 bytes via buffered write, may repeat 2024-01-27T02:34:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 206045184 bytes via buffered write, may repeat 2024-01-27T02:34:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 207093760 bytes via buffered write, may repeat 2024-01-27T02:34:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 208142336 bytes via buffered write, may repeat 2024-01-27T02:34:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 209190912 bytes via buffered write, may repeat 2024-01-27T02:34:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 210239488 bytes via buffered write, may repeat 2024-01-27T02:34:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 211288064 bytes via buffered write, may repeat 2024-01-27T02:34:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 212336640 bytes via buffered write, may repeat 2024-01-27T02:34:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 213385216 bytes via buffered write, may repeat 2024-01-27T02:34:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 214433792 bytes via buffered write, may repeat 2024-01-27T02:34:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 215482368 bytes via buffered write, may repeat 2024-01-27T02:34:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 216530944 bytes via buffered write, may repeat 2024-01-27T02:34:29.438 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 217579520 bytes via buffered write, may repeat 2024-01-27T02:34:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 218628096 bytes via buffered write, may repeat 2024-01-27T02:34:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 219676672 bytes via buffered write, may repeat 2024-01-27T02:34:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 220725248 bytes via buffered write, may repeat 2024-01-27T02:34:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 221773824 bytes via buffered write, may repeat 2024-01-27T02:34:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 222822400 bytes via buffered write, may repeat 2024-01-27T02:34:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 223870976 bytes via buffered write, may repeat 2024-01-27T02:34:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 224919552 bytes via buffered write, may repeat 2024-01-27T02:34:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 225968128 bytes via buffered write, may repeat 2024-01-27T02:34:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 227016704 bytes via buffered write, may repeat 2024-01-27T02:34:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 228065280 bytes via buffered write, may repeat 2024-01-27T02:34:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 229113856 bytes via buffered write, may repeat 2024-01-27T02:34:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 230162432 bytes via buffered write, may repeat 2024-01-27T02:34:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 231211008 bytes via buffered write, may repeat 2024-01-27T02:34:29.439 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 232259584 bytes via buffered write, may repeat 2024-01-27T02:34:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 233308160 bytes via buffered write, may repeat 2024-01-27T02:34:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 234356736 bytes via buffered write, may repeat 2024-01-27T02:34:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 235405312 bytes via buffered write, may repeat 2024-01-27T02:34:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 236453888 bytes via buffered write, may repeat 2024-01-27T02:34:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 237502464 bytes via buffered write, may repeat 2024-01-27T02:34:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 238551040 bytes via buffered write, may repeat 2024-01-27T02:34:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 239599616 bytes via buffered write, may repeat 2024-01-27T02:34:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 240648192 bytes via buffered write, may repeat 2024-01-27T02:34:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 241696768 bytes via buffered write, may repeat 2024-01-27T02:34:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 242745344 bytes via buffered write, may repeat 2024-01-27T02:34:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 243793920 bytes via buffered write, may repeat 2024-01-27T02:34:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 244842496 bytes via buffered write, may repeat 2024-01-27T02:34:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 245891072 bytes via buffered write, may repeat 2024-01-27T02:34:29.440 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 246939648 bytes via buffered write, may repeat 2024-01-27T02:34:29.441 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 247988224 bytes via buffered write, may repeat 2024-01-27T02:34:29.441 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 249036800 bytes via buffered write, may repeat 2024-01-27T02:34:29.441 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 250085376 bytes via buffered write, may repeat 2024-01-27T02:34:29.441 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 251133952 bytes via buffered write, may repeat 2024-01-27T02:34:29.441 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 252182528 bytes via buffered write, may repeat 2024-01-27T02:34:29.441 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 253231104 bytes via buffered write, may repeat 2024-01-27T02:34:29.441 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 254279680 bytes via buffered write, may repeat 2024-01-27T02:34:29.441 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 255328256 bytes via buffered write, may repeat 2024-01-27T02:34:29.441 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 256376832 bytes via buffered write, may repeat 2024-01-27T02:34:29.441 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 257425408 bytes via buffered write, may repeat 2024-01-27T02:34:29.441 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 258473984 bytes via buffered write, may repeat 2024-01-27T02:34:29.441 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 259522560 bytes via buffered write, may repeat 2024-01-27T02:34:29.441 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 260571136 bytes via buffered write, may repeat 2024-01-27T02:34:29.441 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 261619712 bytes via buffered write, may repeat 2024-01-27T02:34:29.441 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 262668288 bytes via buffered write, may repeat 2024-01-27T02:34:29.442 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 263716864 bytes via buffered write, may repeat 2024-01-27T02:34:29.442 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 264765440 bytes via buffered write, may repeat 2024-01-27T02:34:29.442 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 265814016 bytes via buffered write, may repeat 2024-01-27T02:34:29.442 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 266862592 bytes via buffered write, may repeat 2024-01-27T02:34:29.442 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 267911168 bytes via buffered write, may repeat 2024-01-27T02:34:29.442 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 268959744 bytes via buffered write, may repeat 2024-01-27T02:34:29.442 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 270008320 bytes via buffered write, may repeat 2024-01-27T02:34:29.442 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 271056896 bytes via buffered write, may repeat 2024-01-27T02:34:29.442 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 272105472 bytes via buffered write, may repeat 2024-01-27T02:34:29.442 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 273154048 bytes via buffered write, may repeat 2024-01-27T02:34:29.442 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 274202624 bytes via buffered write, may repeat 2024-01-27T02:34:29.442 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 275251200 bytes via buffered write, may repeat 2024-01-27T02:34:29.442 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 276299776 bytes via buffered write, may repeat 2024-01-27T02:34:29.442 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 277348352 bytes via buffered write, may repeat 2024-01-27T02:34:29.443 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 278396928 bytes via buffered write, may repeat 2024-01-27T02:34:29.443 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 279445504 bytes via buffered write, may repeat 2024-01-27T02:34:29.443 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 280494080 bytes via buffered write, may repeat 2024-01-27T02:34:29.443 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 281542656 bytes via buffered write, may repeat 2024-01-27T02:34:29.443 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 282591232 bytes via buffered write, may repeat 2024-01-27T02:34:29.443 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 283639808 bytes via buffered write, may repeat 2024-01-27T02:34:29.443 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 284688384 bytes via buffered write, may repeat 2024-01-27T02:34:29.443 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 285736960 bytes via buffered write, may repeat 2024-01-27T02:34:29.443 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 286785536 bytes via buffered write, may repeat 2024-01-27T02:34:29.443 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 287834112 bytes via buffered write, may repeat 2024-01-27T02:34:29.443 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 288882688 bytes via buffered write, may repeat 2024-01-27T02:34:29.443 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 289931264 bytes via buffered write, may repeat 2024-01-27T02:34:29.443 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 290979840 bytes via buffered write, may repeat 2024-01-27T02:34:29.443 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 292028416 bytes via buffered write, may repeat 2024-01-27T02:34:29.444 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 293076992 bytes via buffered write, may repeat 2024-01-27T02:34:29.444 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 294125568 bytes via buffered write, may repeat 2024-01-27T02:34:29.444 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 295174144 bytes via buffered write, may repeat 2024-01-27T02:34:29.444 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 296222720 bytes via buffered write, may repeat 2024-01-27T02:34:29.444 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 297271296 bytes via buffered write, may repeat 2024-01-27T02:34:29.444 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 298319872 bytes via buffered write, may repeat 2024-01-27T02:34:29.444 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 299368448 bytes via buffered write, may repeat 2024-01-27T02:34:29.444 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 300417024 bytes via buffered write, may repeat 2024-01-27T02:34:29.444 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 301465600 bytes via buffered write, may repeat 2024-01-27T02:34:29.444 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 302514176 bytes via buffered write, may repeat 2024-01-27T02:34:29.444 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 303562752 bytes via buffered write, may repeat 2024-01-27T02:34:29.444 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 304611328 bytes via buffered write, may repeat 2024-01-27T02:34:29.444 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 305659904 bytes via buffered write, may repeat 2024-01-27T02:34:29.444 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 306708480 bytes via buffered write, may repeat 2024-01-27T02:34:29.445 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 307757056 bytes via buffered write, may repeat 2024-01-27T02:34:29.445 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 308805632 bytes via buffered write, may repeat 2024-01-27T02:34:29.445 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 309854208 bytes via buffered write, may repeat 2024-01-27T02:34:29.445 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 310902784 bytes via buffered write, may repeat 2024-01-27T02:34:29.445 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 311951360 bytes via buffered write, may repeat 2024-01-27T02:34:29.445 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 312999936 bytes via buffered write, may repeat 2024-01-27T02:34:29.445 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 314048512 bytes via buffered write, may repeat 2024-01-27T02:34:29.445 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 315097088 bytes via buffered write, may repeat 2024-01-27T02:34:29.445 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 316145664 bytes via buffered write, may repeat 2024-01-27T02:34:29.445 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 317194240 bytes via buffered write, may repeat 2024-01-27T02:34:29.445 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 318242816 bytes via buffered write, may repeat 2024-01-27T02:34:29.445 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 319291392 bytes via buffered write, may repeat 2024-01-27T02:34:29.445 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 320339968 bytes via buffered write, may repeat 2024-01-27T02:34:29.445 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 321388544 bytes via buffered write, may repeat 2024-01-27T02:34:29.446 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 322437120 bytes via buffered write, may repeat 2024-01-27T02:34:29.446 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 323485696 bytes via buffered write, may repeat 2024-01-27T02:34:29.446 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 324534272 bytes via buffered write, may repeat 2024-01-27T02:34:29.488 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 325582848 bytes via buffered write, may repeat 2024-01-27T02:34:29.489 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 326631424 bytes via buffered write, may repeat 2024-01-27T02:34:29.489 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 327680000 bytes via buffered write, may repeat 2024-01-27T02:34:29.489 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 328728576 bytes via buffered write, may repeat 2024-01-27T02:34:29.489 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 329777152 bytes via buffered write, may repeat 2024-01-27T02:34:29.489 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 330825728 bytes via buffered write, may repeat 2024-01-27T02:34:29.489 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 331874304 bytes via buffered write, may repeat 2024-01-27T02:34:29.489 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 332922880 bytes via buffered write, may repeat 2024-01-27T02:34:29.489 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 333971456 bytes via buffered write, may repeat 2024-01-27T02:34:29.489 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 335020032 bytes via buffered write, may repeat 2024-01-27T02:34:29.489 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 336068608 bytes via buffered write, may repeat 2024-01-27T02:34:29.489 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 337117184 bytes via buffered write, may repeat 2024-01-27T02:34:29.489 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 338165760 bytes via buffered write, may repeat 2024-01-27T02:34:29.489 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 339214336 bytes via buffered write, may repeat 2024-01-27T02:34:29.490 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 340262912 bytes via buffered write, may repeat 2024-01-27T02:34:29.490 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 341311488 bytes via buffered write, may repeat 2024-01-27T02:34:29.490 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 342360064 bytes via buffered write, may repeat 2024-01-27T02:34:29.490 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 343408640 bytes via buffered write, may repeat 2024-01-27T02:34:29.490 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 344457216 bytes via buffered write, may repeat 2024-01-27T02:34:29.490 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 345505792 bytes via buffered write, may repeat 2024-01-27T02:34:29.490 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 346554368 bytes via buffered write, may repeat 2024-01-27T02:34:29.490 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 347602944 bytes via buffered write, may repeat 2024-01-27T02:34:29.490 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 348651520 bytes via buffered write, may repeat 2024-01-27T02:34:29.490 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 349700096 bytes via buffered write, may repeat 2024-01-27T02:34:29.490 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 350748672 bytes via buffered write, may repeat 2024-01-27T02:34:29.490 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 351797248 bytes via buffered write, may repeat 2024-01-27T02:34:29.490 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 352845824 bytes via buffered write, may repeat 2024-01-27T02:34:29.490 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 353894400 bytes via buffered write, may repeat 2024-01-27T02:34:29.491 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 354942976 bytes via buffered write, may repeat 2024-01-27T02:34:29.491 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 355991552 bytes via buffered write, may repeat 2024-01-27T02:34:29.491 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 357040128 bytes via buffered write, may repeat 2024-01-27T02:34:29.491 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 358088704 bytes via buffered write, may repeat 2024-01-27T02:34:29.491 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 359137280 bytes via buffered write, may repeat 2024-01-27T02:34:29.491 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 360185856 bytes via buffered write, may repeat 2024-01-27T02:34:29.491 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 361234432 bytes via buffered write, may repeat 2024-01-27T02:34:29.491 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 362283008 bytes via buffered write, may repeat 2024-01-27T02:34:29.491 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 363331584 bytes via buffered write, may repeat 2024-01-27T02:34:29.491 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 364380160 bytes via buffered write, may repeat 2024-01-27T02:34:29.491 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 365428736 bytes via buffered write, may repeat 2024-01-27T02:34:29.491 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 366477312 bytes via buffered write, may repeat 2024-01-27T02:34:29.491 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 367525888 bytes via buffered write, may repeat 2024-01-27T02:34:29.491 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 368574464 bytes via buffered write, may repeat 2024-01-27T02:34:29.491 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 369623040 bytes via buffered write, may repeat 2024-01-27T02:34:29.492 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 370671616 bytes via buffered write, may repeat 2024-01-27T02:34:29.492 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 371720192 bytes via buffered write, may repeat 2024-01-27T02:34:29.492 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 372768768 bytes via buffered write, may repeat 2024-01-27T02:34:29.492 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 373817344 bytes via buffered write, may repeat 2024-01-27T02:34:29.492 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 374865920 bytes via buffered write, may repeat 2024-01-27T02:34:29.492 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 375914496 bytes via buffered write, may repeat 2024-01-27T02:34:29.492 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 376963072 bytes via buffered write, may repeat 2024-01-27T02:34:29.492 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 378011648 bytes via buffered write, may repeat 2024-01-27T02:34:29.492 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 379060224 bytes via buffered write, may repeat 2024-01-27T02:34:29.492 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 380108800 bytes via buffered write, may repeat 2024-01-27T02:34:29.492 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 381157376 bytes via buffered write, may repeat 2024-01-27T02:34:29.492 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 382205952 bytes via buffered write, may repeat 2024-01-27T02:34:29.492 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 383254528 bytes via buffered write, may repeat 2024-01-27T02:34:29.492 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 384303104 bytes via buffered write, may repeat 2024-01-27T02:34:29.493 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 385351680 bytes via buffered write, may repeat 2024-01-27T02:34:29.493 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 386400256 bytes via buffered write, may repeat 2024-01-27T02:34:29.493 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 387448832 bytes via buffered write, may repeat 2024-01-27T02:34:29.493 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 388497408 bytes via buffered write, may repeat 2024-01-27T02:34:29.493 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 389545984 bytes via buffered write, may repeat 2024-01-27T02:34:29.493 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 390594560 bytes via buffered write, may repeat 2024-01-27T02:34:29.493 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 391643136 bytes via buffered write, may repeat 2024-01-27T02:34:29.493 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 392691712 bytes via buffered write, may repeat 2024-01-27T02:34:29.493 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 393740288 bytes via buffered write, may repeat 2024-01-27T02:34:29.493 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 394788864 bytes via buffered write, may repeat 2024-01-27T02:34:29.493 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 395837440 bytes via buffered write, may repeat 2024-01-27T02:34:29.493 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 396886016 bytes via buffered write, may repeat 2024-01-27T02:34:29.493 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 397934592 bytes via buffered write, may repeat 2024-01-27T02:34:29.493 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 398983168 bytes via buffered write, may repeat 2024-01-27T02:34:29.493 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 400031744 bytes via buffered write, may repeat 2024-01-27T02:34:29.494 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 401080320 bytes via buffered write, may repeat 2024-01-27T02:34:29.494 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 402128896 bytes via buffered write, may repeat 2024-01-27T02:34:29.494 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 403177472 bytes via buffered write, may repeat 2024-01-27T02:34:29.494 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 404226048 bytes via buffered write, may repeat 2024-01-27T02:34:29.494 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 405274624 bytes via buffered write, may repeat 2024-01-27T02:34:29.494 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 406323200 bytes via buffered write, may repeat 2024-01-27T02:34:29.494 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 407371776 bytes via buffered write, may repeat 2024-01-27T02:34:29.494 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 408420352 bytes via buffered write, may repeat 2024-01-27T02:34:29.494 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 409468928 bytes via buffered write, may repeat 2024-01-27T02:34:29.494 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 410517504 bytes via buffered write, may repeat 2024-01-27T02:34:29.494 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 411566080 bytes via buffered write, may repeat 2024-01-27T02:34:29.494 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 412614656 bytes via buffered write, may repeat 2024-01-27T02:34:29.494 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 413663232 bytes via buffered write, may repeat 2024-01-27T02:34:29.494 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 414711808 bytes via buffered write, may repeat 2024-01-27T02:34:29.494 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 415760384 bytes via buffered write, may repeat 2024-01-27T02:34:29.495 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 416808960 bytes via buffered write, may repeat 2024-01-27T02:34:29.495 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 417857536 bytes via buffered write, may repeat 2024-01-27T02:34:29.495 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 418906112 bytes via buffered write, may repeat 2024-01-27T02:34:29.495 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 419954688 bytes via buffered write, may repeat 2024-01-27T02:34:29.495 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 421003264 bytes via buffered write, may repeat 2024-01-27T02:34:29.495 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 422051840 bytes via buffered write, may repeat 2024-01-27T02:34:29.495 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 423100416 bytes via buffered write, may repeat 2024-01-27T02:34:29.495 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 424148992 bytes via buffered write, may repeat 2024-01-27T02:34:29.495 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 425197568 bytes via buffered write, may repeat 2024-01-27T02:34:29.495 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 426246144 bytes via buffered write, may repeat 2024-01-27T02:34:29.495 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 427294720 bytes via buffered write, may repeat 2024-01-27T02:34:29.495 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 428343296 bytes via buffered write, may repeat 2024-01-27T02:34:29.495 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 429391872 bytes via buffered write, may repeat 2024-01-27T02:34:29.495 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 430440448 bytes via buffered write, may repeat 2024-01-27T02:34:29.496 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 431489024 bytes via buffered write, may repeat 2024-01-27T02:34:29.496 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 432537600 bytes via buffered write, may repeat 2024-01-27T02:34:29.496 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 433586176 bytes via buffered write, may repeat 2024-01-27T02:34:29.496 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 434634752 bytes via buffered write, may repeat 2024-01-27T02:34:29.496 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 435683328 bytes via buffered write, may repeat 2024-01-27T02:34:29.496 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 436731904 bytes via buffered write, may repeat 2024-01-27T02:34:29.496 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 437780480 bytes via buffered write, may repeat 2024-01-27T02:34:29.496 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 438829056 bytes via buffered write, may repeat 2024-01-27T02:34:29.496 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 439877632 bytes via buffered write, may repeat 2024-01-27T02:34:29.496 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 440926208 bytes via buffered write, may repeat 2024-01-27T02:34:29.496 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 441974784 bytes via buffered write, may repeat 2024-01-27T02:34:29.496 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 443023360 bytes via buffered write, may repeat 2024-01-27T02:34:29.497 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 444071936 bytes via buffered write, may repeat 2024-01-27T02:34:29.497 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 445120512 bytes via buffered write, may repeat 2024-01-27T02:34:29.497 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 446169088 bytes via buffered write, may repeat 2024-01-27T02:34:29.497 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 447217664 bytes via buffered write, may repeat 2024-01-27T02:34:29.497 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 448266240 bytes via buffered write, may repeat 2024-01-27T02:34:29.497 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 449314816 bytes via buffered write, may repeat 2024-01-27T02:34:29.497 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 450363392 bytes via buffered write, may repeat 2024-01-27T02:34:29.497 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 451411968 bytes via buffered write, may repeat 2024-01-27T02:34:29.497 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 452460544 bytes via buffered write, may repeat 2024-01-27T02:34:29.497 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 453509120 bytes via buffered write, may repeat 2024-01-27T02:34:29.497 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 454557696 bytes via buffered write, may repeat 2024-01-27T02:34:29.497 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 455606272 bytes via buffered write, may repeat 2024-01-27T02:34:29.497 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 456654848 bytes via buffered write, may repeat 2024-01-27T02:34:29.497 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 457703424 bytes via buffered write, may repeat 2024-01-27T02:34:29.497 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 458752000 bytes via buffered write, may repeat 2024-01-27T02:34:29.498 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 459800576 bytes via buffered write, may repeat 2024-01-27T02:34:29.498 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 460849152 bytes via buffered write, may repeat 2024-01-27T02:34:29.498 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 461897728 bytes via buffered write, may repeat 2024-01-27T02:34:29.498 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 462946304 bytes via buffered write, may repeat 2024-01-27T02:34:29.498 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 463994880 bytes via buffered write, may repeat 2024-01-27T02:34:29.498 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 465043456 bytes via buffered write, may repeat 2024-01-27T02:34:29.498 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 466092032 bytes via buffered write, may repeat 2024-01-27T02:34:29.498 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 467140608 bytes via buffered write, may repeat 2024-01-27T02:34:29.498 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 468189184 bytes via buffered write, may repeat 2024-01-27T02:34:29.498 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 469237760 bytes via buffered write, may repeat 2024-01-27T02:34:29.498 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 470286336 bytes via buffered write, may repeat 2024-01-27T02:34:29.498 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 471334912 bytes via buffered write, may repeat 2024-01-27T02:34:29.498 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 472383488 bytes via buffered write, may repeat 2024-01-27T02:34:29.498 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 473432064 bytes via buffered write, may repeat 2024-01-27T02:34:29.499 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 474480640 bytes via buffered write, may repeat 2024-01-27T02:34:29.499 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 475529216 bytes via buffered write, may repeat 2024-01-27T02:34:29.499 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 476577792 bytes via buffered write, may repeat 2024-01-27T02:34:29.499 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 477626368 bytes via buffered write, may repeat 2024-01-27T02:34:29.499 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 478674944 bytes via buffered write, may repeat 2024-01-27T02:34:29.499 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 479723520 bytes via buffered write, may repeat 2024-01-27T02:34:29.499 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 480772096 bytes via buffered write, may repeat 2024-01-27T02:34:29.499 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 481820672 bytes via buffered write, may repeat 2024-01-27T02:34:29.499 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 482869248 bytes via buffered write, may repeat 2024-01-27T02:34:29.499 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 483917824 bytes via buffered write, may repeat 2024-01-27T02:34:29.499 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 484966400 bytes via buffered write, may repeat 2024-01-27T02:34:29.499 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 486014976 bytes via buffered write, may repeat 2024-01-27T02:34:29.499 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 487063552 bytes via buffered write, may repeat 2024-01-27T02:34:31.839 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-27T02:34:31.837+0000 7fb5c783f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2024-01-27T02:34:38.145 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:34:38.148 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:34:39.819 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-27T02:34:39.818+0000 7fb5c783f700 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2024-01-27T02:35:08.237 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:35:08.240 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:35:38.340 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:35:38.344 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:35:44.942 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 488112128 bytes via buffered write, may repeat 2024-01-27T02:35:44.942 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 489160704 bytes via buffered write, may repeat 2024-01-27T02:35:44.942 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 490209280 bytes via buffered write, may repeat 2024-01-27T02:35:44.942 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 491257856 bytes via buffered write, may repeat 2024-01-27T02:35:44.943 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 492306432 bytes via buffered write, may repeat 2024-01-27T02:35:44.943 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 493355008 bytes via buffered write, may repeat 2024-01-27T02:35:44.943 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 494403584 bytes via buffered write, may repeat 2024-01-27T02:35:44.943 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 495452160 bytes via buffered write, may repeat 2024-01-27T02:35:44.943 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 496500736 bytes via buffered write, may repeat 2024-01-27T02:35:44.943 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 497549312 bytes via buffered write, may repeat 2024-01-27T02:35:44.943 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 498597888 bytes via buffered write, may repeat 2024-01-27T02:35:44.943 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 499646464 bytes via buffered write, may repeat 2024-01-27T02:35:44.943 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 500695040 bytes via buffered write, may repeat 2024-01-27T02:35:44.943 INFO:teuthology.orchestra.run.smithi007.stdout:done writing 500695040 bytes 2024-01-27T02:35:44.943 INFO:teuthology.orchestra.run.smithi007.stdout:successfully fsync'ed prior to getting full state reported 2024-01-27T02:35:44.943 INFO:teuthology.orchestra.run.smithi007.stdout:starting buffered write 2024-01-27T02:35:44.943 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-27T02:35:44.943 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-27T02:35:44.944 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-27T02:35:44.944 INFO:teuthology.orchestra.run.smithi007.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2024-01-27T02:35:45.004 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-27T02:35:45.005 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:35:45.057 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:35:45.103 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-27T02:35:45.103 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-27T02:35:45.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:35:45.103 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-27T02:35:45.158 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:35:45.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:35:45.159 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-27T02:35:45.180 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-27T02:35:45.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:35:45.181 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-01-27T02:35:45.224 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:35:45.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:35:45.225 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-27T02:35:45.284 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:35:45.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.426+0000 7fbe20454700 1 Processor -- start 2024-01-27T02:35:45.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.426+0000 7fbe20454700 1 -- start start 2024-01-27T02:35:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.427+0000 7fbe20454700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbe181566a0 0x7fbe180a7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.427+0000 7fbe20454700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe18156a70 0x7fbe180aee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.427+0000 7fbe20454700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe180afd30 con 0x7fbe180acc80 2024-01-27T02:35:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.427+0000 7fbe20454700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe1806df80 con 0x7fbe181566a0 2024-01-27T02:35:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.427+0000 7fbe20454700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe180af770 con 0x7fbe18156a70 2024-01-27T02:35:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.427+0000 7fbe1e1f0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbe181566a0 0x7fbe180a7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.427+0000 7fbe1e1f0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbe181566a0 0x7fbe180a7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43986/0 (socket says 172.21.15.7:43986) 2024-01-27T02:35:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.427+0000 7fbe1e1f0700 1 -- 172.21.15.7:0/961952955 learned_addr learned my addr 172.21.15.7:0/961952955 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:45.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.427+0000 7fbe1d9ef700 1 --2- 172.21.15.7:0/961952955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe18156a70 0x7fbe180aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:45.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.427+0000 7fbe1e1f0700 1 -- 172.21.15.7:0/961952955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe18156a70 msgr2=0x7fbe180aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:45.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.427+0000 7fbe1e1f0700 1 --2- 172.21.15.7:0/961952955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe18156a70 0x7fbe180aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:45.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.427+0000 7fbe1e1f0700 1 -- 172.21.15.7:0/961952955 >> v1:172.21.15.188:6789/0 conn(0x7fbe180acc80 legacy=0x7fbe180b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:45.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.427+0000 7fbe1e1f0700 1 -- 172.21.15.7:0/961952955 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe180afa50 con 0x7fbe181566a0 2024-01-27T02:35:45.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.427+0000 7fbe1d9ef700 1 --2- 172.21.15.7:0/961952955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe18156a70 0x7fbe180aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:35:45.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.428+0000 7fbe1e1f0700 1 --2- 172.21.15.7:0/961952955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe181566a0 0x7fbe180a7e20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400bc40 tx=0x7fbe1400b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82badfaeea7d4594 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:45.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.428+0000 7fbe1d1ee700 1 -- 172.21.15.7:0/961952955 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1400db40 con 0x7fbe181566a0 2024-01-27T02:35:45.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.428+0000 7fbe1d1ee700 1 -- 172.21.15.7:0/961952955 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1400dca0 con 0x7fbe181566a0 2024-01-27T02:35:45.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.428+0000 7fbe1d1ee700 1 -- 172.21.15.7:0/961952955 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe14017640 con 0x7fbe181566a0 2024-01-27T02:35:45.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.428+0000 7fbe20454700 1 -- 172.21.15.7:0/961952955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe181566a0 msgr2=0x7fbe180a7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:45.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.428+0000 7fbe20454700 1 --2- 172.21.15.7:0/961952955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe181566a0 0x7fbe180a7e20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400bc40 tx=0x7fbe1400b4e0 comp rx=0 tx=0).stop 2024-01-27T02:35:45.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.428+0000 7fbe20454700 1 -- 172.21.15.7:0/961952955 shutdown_connections 2024-01-27T02:35:45.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.428+0000 7fbe20454700 1 --2- 172.21.15.7:0/961952955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe181566a0 0x7fbe180a7e20 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:45.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.428+0000 7fbe20454700 1 --2- 172.21.15.7:0/961952955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe18156a70 0x7fbe180aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:45.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.428+0000 7fbe20454700 1 -- 172.21.15.7:0/961952955 >> 172.21.15.7:0/961952955 conn(0x7fbe180679b0 msgr2=0x7fbe18067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:45.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.428+0000 7fbe20454700 1 -- 172.21.15.7:0/961952955 shutdown_connections 2024-01-27T02:35:45.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.429+0000 7fbe20454700 1 -- 172.21.15.7:0/961952955 wait complete. 2024-01-27T02:35:45.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.430+0000 7fbe20454700 1 Processor -- start 2024-01-27T02:35:45.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.430+0000 7fbe20454700 1 -- start start 2024-01-27T02:35:45.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.430+0000 7fbe20454700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe180acc80 0x7fbe18165940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:45.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.430+0000 7fbe20454700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe181566a0 0x7fbe180aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:45.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.430+0000 7fbe20454700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe18156a70 0x7fbe18099b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:45.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.430+0000 7fbe20454700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe180abaa0 con 0x7fbe18156a70 2024-01-27T02:35:45.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.430+0000 7fbe20454700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe180abd80 con 0x7fbe181566a0 2024-01-27T02:35:45.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.430+0000 7fbe20454700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe180ac060 con 0x7fbe180acc80 2024-01-27T02:35:45.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.431+0000 7fbe1e9f1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe18156a70 0x7fbe18099b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:45.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.431+0000 7fbe1e9f1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe18156a70 0x7fbe18099b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44002/0 (socket says 172.21.15.7:44002) 2024-01-27T02:35:45.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.431+0000 7fbe1e9f1700 1 -- 172.21.15.7:0/1500643114 learned_addr learned my addr 172.21.15.7:0/1500643114 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:45.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.431+0000 7fbe1d9ef700 1 --2- 172.21.15.7:0/1500643114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe181566a0 0x7fbe180aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:45.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.431+0000 7fbe1e1f0700 1 --2- 172.21.15.7:0/1500643114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe180acc80 0x7fbe18165940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:45.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.431+0000 7fbe1e9f1700 1 -- 172.21.15.7:0/1500643114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe180acc80 msgr2=0x7fbe18165940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:45.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.431+0000 7fbe1e9f1700 1 --2- 172.21.15.7:0/1500643114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe180acc80 0x7fbe18165940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:45.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.431+0000 7fbe1e9f1700 1 -- 172.21.15.7:0/1500643114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe181566a0 msgr2=0x7fbe180aafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:45.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.431+0000 7fbe1e9f1700 1 --2- 172.21.15.7:0/1500643114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe181566a0 0x7fbe180aafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:45.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.431+0000 7fbe1e9f1700 1 -- 172.21.15.7:0/1500643114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1400b780 con 0x7fbe18156a70 2024-01-27T02:35:45.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.431+0000 7fbe1d9ef700 1 --2- 172.21.15.7:0/1500643114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe181566a0 0x7fbe180aafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:35:45.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.431+0000 7fbe1e9f1700 1 --2- 172.21.15.7:0/1500643114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe18156a70 0x7fbe18099b90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000c8e0 tx=0x7fbe10001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:45.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.431+0000 7fbe0e7fc700 1 -- 172.21.15.7:0/1500643114 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1000ce80 con 0x7fbe18156a70 2024-01-27T02:35:45.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.431+0000 7fbe0e7fc700 1 -- 172.21.15.7:0/1500643114 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1000a7e0 con 0x7fbe18156a70 2024-01-27T02:35:45.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.431+0000 7fbe20454700 1 -- 172.21.15.7:0/1500643114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe18148610 con 0x7fbe18156a70 2024-01-27T02:35:45.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.431+0000 7fbe0e7fc700 1 -- 172.21.15.7:0/1500643114 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1000aba0 con 0x7fbe18156a70 2024-01-27T02:35:45.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.431+0000 7fbe20454700 1 -- 172.21.15.7:0/1500643114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe180a8620 con 0x7fbe18156a70 2024-01-27T02:35:45.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.432+0000 7fbe20454700 1 -- 172.21.15.7:0/1500643114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe1804c710 con 0x7fbe18156a70 2024-01-27T02:35:45.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.433+0000 7fbe0e7fc700 1 -- 172.21.15.7:0/1500643114 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbe10003030 con 0x7fbe18156a70 2024-01-27T02:35:45.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.433+0000 7fbe0e7fc700 1 --2- 172.21.15.7:0/1500643114 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbdfc066f30 0x7fbdfc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:45.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.433+0000 7fbe0e7fc700 1 -- 172.21.15.7:0/1500643114 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbe100844f0 con 0x7fbe18156a70 2024-01-27T02:35:45.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.434+0000 7fbe1e1f0700 1 --2- 172.21.15.7:0/1500643114 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbdfc066f30 0x7fbdfc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:45.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.434+0000 7fbe0e7fc700 1 -- 172.21.15.7:0/1500643114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=69}) v3 -- 0x7fbdfc06ea50 con 0x7fbe18156a70 2024-01-27T02:35:45.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.434+0000 7fbe1e1f0700 1 --2- 172.21.15.7:0/1500643114 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbdfc066f30 0x7fbdfc0693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400b520 tx=0x7fbe14019000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:45.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.438+0000 7fbe0e7fc700 1 -- 172.21.15.7:0/1500643114 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe10051760 con 0x7fbe18156a70 2024-01-27T02:35:45.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.688+0000 7fbe20454700 1 -- 172.21.15.7:0/1500643114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe1806dc10 con 0x7fbe18156a70 2024-01-27T02:35:45.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.689+0000 7fbe0e7fc700 1 -- 172.21.15.7:0/1500643114 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4597 (secure 0 0 0) 0x7fbe10054d80 con 0x7fbe18156a70 2024-01-27T02:35:45.689 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:35:45.689 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5114,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/2839704447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":2839704447},{"type":"v1","addr":"172.21.15.188:6837","nonce":2839704447}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5117,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3964291855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3964291855},{"type":"v1","addr":"172.21.15.7:6834","nonce":3964291855}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/3332325272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3332325272},{"type":"v1","addr":"172.21.15.188:6836","nonce":3332325272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:34:14.471429+0000","modified":"2024-01-27T02:34:18.068858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1399055016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1399055016},{"type":"v1","addr":"172.21.15.7:6835","nonce":1399055016}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2024-01-27T02:35:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.691+0000 7fbe20454700 1 -- 172.21.15.7:0/1500643114 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbdfc066f30 msgr2=0x7fbdfc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.691+0000 7fbe20454700 1 --2- 172.21.15.7:0/1500643114 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbdfc066f30 0x7fbdfc0693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400b520 tx=0x7fbe14019000 comp rx=0 tx=0).stop 2024-01-27T02:35:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.691+0000 7fbe20454700 1 -- 172.21.15.7:0/1500643114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe18156a70 msgr2=0x7fbe18099b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.691+0000 7fbe20454700 1 --2- 172.21.15.7:0/1500643114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe18156a70 0x7fbe18099b90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000c8e0 tx=0x7fbe10001710 comp rx=0 tx=0).stop 2024-01-27T02:35:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.691+0000 7fbe20454700 1 -- 172.21.15.7:0/1500643114 shutdown_connections 2024-01-27T02:35:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.691+0000 7fbe20454700 1 --2- 172.21.15.7:0/1500643114 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbdfc066f30 0x7fbdfc0693e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.691+0000 7fbe20454700 1 --2- 172.21.15.7:0/1500643114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe18156a70 0x7fbe18099b90 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.691+0000 7fbe20454700 1 --2- 172.21.15.7:0/1500643114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe180acc80 0x7fbe18165940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.691+0000 7fbe20454700 1 --2- 172.21.15.7:0/1500643114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe181566a0 0x7fbe180aafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.691+0000 7fbe20454700 1 -- 172.21.15.7:0/1500643114 >> 172.21.15.7:0/1500643114 conn(0x7fbe180679b0 msgr2=0x7fbe1806aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.691+0000 7fbe20454700 1 -- 172.21.15.7:0/1500643114 shutdown_connections 2024-01-27T02:35:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.691+0000 7fbe20454700 1 -- 172.21.15.7:0/1500643114 wait complete. 2024-01-27T02:35:45.693 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2024-01-27T02:35:45.705 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:35:45.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.836+0000 7f9683dfc700 1 Processor -- start 2024-01-27T02:35:45.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.837+0000 7f9683dfc700 1 -- start start 2024-01-27T02:35:45.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.837+0000 7f9683dfc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f967c15a100 0x7f967c0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:45.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.838+0000 7f9683dfc700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f967c0ce5e0 con 0x7f967c0c4cc0 2024-01-27T02:35:45.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.838+0000 7f9683dfc700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f967c0cd9d0 con 0x7f967c0cb5b0 2024-01-27T02:35:45.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.838+0000 7f9683dfc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f967c0cdcb0 con 0x7f967c15a100 2024-01-27T02:35:45.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.838+0000 7f9682399700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f967c0c4cc0 0x7f967c163d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56022/0 (socket says 172.21.15.7:56022) 2024-01-27T02:35:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.838+0000 7f9682399700 1 -- 172.21.15.7:0/168181142 learned_addr learned my addr 172.21.15.7:0/168181142 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:35:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.838+0000 7f9681b98700 1 --2- 172.21.15.7:0/168181142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f967c15a100 0x7f967c0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.838+0000 7f9680b96700 1 -- 172.21.15.7:0/168181142 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2200630706 0 0) 0x7f967c0ce5e0 con 0x7f967c0c4cc0 2024-01-27T02:35:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.838+0000 7f9680b96700 1 -- 172.21.15.7:0/168181142 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9668003570 con 0x7f967c0c4cc0 2024-01-27T02:35:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.838+0000 7f9680b96700 1 -- 172.21.15.7:0/168181142 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302485787 0 0) 0x7f967c0cd9d0 con 0x7f967c0cb5b0 2024-01-27T02:35:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.838+0000 7f9680b96700 1 -- 172.21.15.7:0/168181142 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f967c0ce5e0 con 0x7f967c0cb5b0 2024-01-27T02:35:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.838+0000 7f9681b98700 1 -- 172.21.15.7:0/168181142 >> v1:172.21.15.188:6789/0 conn(0x7f967c0cb5b0 legacy=0x7f967c0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.838+0000 7f9681b98700 1 -- 172.21.15.7:0/168181142 >> v1:172.21.15.7:6789/0 conn(0x7f967c0c4cc0 legacy=0x7f967c163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.838+0000 7f9681b98700 1 -- 172.21.15.7:0/168181142 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967c0cdf90 con 0x7f967c15a100 2024-01-27T02:35:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.839+0000 7f9681b98700 1 --2- 172.21.15.7:0/168181142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f967c15a100 0x7f967c0cd2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9678009910 tx=0x7f96780091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5938817db49a4a8a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.839+0000 7f9680b96700 1 -- 172.21.15.7:0/168181142 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678011630 con 0x7f967c15a100 2024-01-27T02:35:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.839+0000 7f9680b96700 1 -- 172.21.15.7:0/168181142 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9678011790 con 0x7f967c15a100 2024-01-27T02:35:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.839+0000 7f9680b96700 1 -- 172.21.15.7:0/168181142 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678011b10 con 0x7f967c15a100 2024-01-27T02:35:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.839+0000 7f9683dfc700 1 -- 172.21.15.7:0/168181142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f967c15a100 msgr2=0x7f967c0cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.839+0000 7f9683dfc700 1 --2- 172.21.15.7:0/168181142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f967c15a100 0x7f967c0cd2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9678009910 tx=0x7f96780091a0 comp rx=0 tx=0).stop 2024-01-27T02:35:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.839+0000 7f9683dfc700 1 -- 172.21.15.7:0/168181142 shutdown_connections 2024-01-27T02:35:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.839+0000 7f9683dfc700 1 --2- 172.21.15.7:0/168181142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f967c15a100 0x7f967c0cd2f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.839+0000 7f9683dfc700 1 -- 172.21.15.7:0/168181142 >> 172.21.15.7:0/168181142 conn(0x7f967c098020 msgr2=0x7f967c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.839+0000 7f9683dfc700 1 -- 172.21.15.7:0/168181142 shutdown_connections 2024-01-27T02:35:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.839+0000 7f9683dfc700 1 -- 172.21.15.7:0/168181142 wait complete. 2024-01-27T02:35:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.841+0000 7f9683dfc700 1 Processor -- start 2024-01-27T02:35:45.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.841+0000 7f9683dfc700 1 -- start start 2024-01-27T02:35:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.842+0000 7f9683dfc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f967c0c4cc0 0x7f967c153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.842+0000 7f9683dfc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f967c0cb5b0 0x7f967c06b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.842+0000 7f9683dfc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f967c15a100 0x7f967c06ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.842+0000 7f9683dfc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967c154330 con 0x7f967c0cb5b0 2024-01-27T02:35:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.842+0000 7f9683dfc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967c0c5df0 con 0x7f967c0c4cc0 2024-01-27T02:35:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.842+0000 7f9683dfc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f967c155ca0 con 0x7f967c15a100 2024-01-27T02:35:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.842+0000 7f9681397700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f967c0cb5b0 0x7f967c06b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.842+0000 7f9681397700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f967c0cb5b0 0x7f967c06b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44016/0 (socket says 172.21.15.7:44016) 2024-01-27T02:35:45.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.842+0000 7f9681397700 1 -- 172.21.15.7:0/4027383341 learned_addr learned my addr 172.21.15.7:0/4027383341 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:45.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.843+0000 7f9681b98700 1 --2- 172.21.15.7:0/4027383341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f967c0c4cc0 0x7f967c153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:45.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.843+0000 7f9682399700 1 --2- 172.21.15.7:0/4027383341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f967c15a100 0x7f967c06ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:45.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.843+0000 7f9681397700 1 -- 172.21.15.7:0/4027383341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f967c15a100 msgr2=0x7f967c06ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:45.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.843+0000 7f9681397700 1 --2- 172.21.15.7:0/4027383341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f967c15a100 0x7f967c06ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:45.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.843+0000 7f9681397700 1 -- 172.21.15.7:0/4027383341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f967c0c4cc0 msgr2=0x7f967c153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:45.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.843+0000 7f9681397700 1 --2- 172.21.15.7:0/4027383341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f967c0c4cc0 0x7f967c153df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:45.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.843+0000 7f9681397700 1 -- 172.21.15.7:0/4027383341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9678009420 con 0x7f967c0cb5b0 2024-01-27T02:35:45.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.843+0000 7f9681397700 1 --2- 172.21.15.7:0/4027383341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f967c0cb5b0 0x7f967c06b4c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f967000c580 tx=0x7f9670007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:45.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.843+0000 7f966dffb700 1 -- 172.21.15.7:0/4027383341 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670010510 con 0x7f967c0cb5b0 2024-01-27T02:35:45.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.843+0000 7f966dffb700 1 -- 172.21.15.7:0/4027383341 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9670010670 con 0x7f967c0cb5b0 2024-01-27T02:35:45.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.843+0000 7f966dffb700 1 -- 172.21.15.7:0/4027383341 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670010a30 con 0x7f967c0cb5b0 2024-01-27T02:35:45.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.843+0000 7f9683dfc700 1 -- 172.21.15.7:0/4027383341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f967c155f80 con 0x7f967c0cb5b0 2024-01-27T02:35:45.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.843+0000 7f9683dfc700 1 -- 172.21.15.7:0/4027383341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f967c156540 con 0x7f967c0cb5b0 2024-01-27T02:35:45.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.844+0000 7f9683dfc700 1 -- 172.21.15.7:0/4027383341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967c0c5f80 con 0x7f967c0cb5b0 2024-01-27T02:35:45.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.845+0000 7f966dffb700 1 -- 172.21.15.7:0/4027383341 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f967001b400 con 0x7f967c0cb5b0 2024-01-27T02:35:45.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.845+0000 7f966dffb700 1 --2- 172.21.15.7:0/4027383341 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9660066f30 0x7f96600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:45.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.846+0000 7f966dffb700 1 -- 172.21.15.7:0/4027383341 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9670084260 con 0x7f967c0cb5b0 2024-01-27T02:35:45.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.846+0000 7f966dffb700 1 -- 172.21.15.7:0/4027383341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=69}) v3 -- 0x7f966006ea50 con 0x7f967c0cb5b0 2024-01-27T02:35:45.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.846+0000 7f9681b98700 1 --2- 172.21.15.7:0/4027383341 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9660066f30 0x7f96600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:45.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.846+0000 7f9681b98700 1 --2- 172.21.15.7:0/4027383341 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9660066f30 0x7f96600693e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9678009860 tx=0x7f9678017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:45.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:45.850+0000 7f966dffb700 1 -- 172.21.15.7:0/4027383341 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9670051430 con 0x7f967c0cb5b0 2024-01-27T02:35:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.057+0000 7f9683dfc700 1 -- 172.21.15.7:0/4027383341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f967c04c710 con 0x7f967c0cb5b0 2024-01-27T02:35:46.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.058+0000 7f966dffb700 1 -- 172.21.15.7:0/4027383341 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+15360 (secure 0 0 0) 0x7f9670054a50 con 0x7f967c0cb5b0 2024-01-27T02:35:46.059 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:35:46.059 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":68,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:34:34.902046+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:34:12.775862+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-27T02:34:13.445542+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:35:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.061+0000 7f9683dfc700 1 -- 172.21.15.7:0/4027383341 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9660066f30 msgr2=0x7f96600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.061+0000 7f9683dfc700 1 --2- 172.21.15.7:0/4027383341 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9660066f30 0x7f96600693e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9678009860 tx=0x7f9678017000 comp rx=0 tx=0).stop 2024-01-27T02:35:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.061+0000 7f9683dfc700 1 -- 172.21.15.7:0/4027383341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f967c0cb5b0 msgr2=0x7f967c06b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.061+0000 7f9683dfc700 1 --2- 172.21.15.7:0/4027383341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f967c0cb5b0 0x7f967c06b4c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f967000c580 tx=0x7f9670007e10 comp rx=0 tx=0).stop 2024-01-27T02:35:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.061+0000 7f9683dfc700 1 -- 172.21.15.7:0/4027383341 shutdown_connections 2024-01-27T02:35:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.061+0000 7f9683dfc700 1 --2- 172.21.15.7:0/4027383341 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9660066f30 0x7f96600693e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.061+0000 7f9683dfc700 1 --2- 172.21.15.7:0/4027383341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f967c15a100 0x7f967c06ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.061+0000 7f9683dfc700 1 --2- 172.21.15.7:0/4027383341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f967c0c4cc0 0x7f967c153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.061+0000 7f9683dfc700 1 --2- 172.21.15.7:0/4027383341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f967c0cb5b0 0x7f967c06b4c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.061+0000 7f9683dfc700 1 -- 172.21.15.7:0/4027383341 >> 172.21.15.7:0/4027383341 conn(0x7f967c098020 msgr2=0x7f967c15fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.061+0000 7f9683dfc700 1 -- 172.21.15.7:0/4027383341 shutdown_connections 2024-01-27T02:35:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.061+0000 7f9683dfc700 1 -- 172.21.15.7:0/4027383341 wait complete. 2024-01-27T02:35:46.075 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-27T02:34:12.775862+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-01-27T02:34:13.445542+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-27T02:35:46.075 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:35:46.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.265+0000 7fbec12a9700 1 Processor -- start 2024-01-27T02:35:46.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.265+0000 7fbec12a9700 1 -- start start 2024-01-27T02:35:46.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.266+0000 7fbec12a9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbebc15d460 0x7fbebc0d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:46.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.266+0000 7fbec12a9700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbebc0cf280 0x7fbebc15e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:46.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.266+0000 7fbec12a9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbebc15d830 0x7fbebc0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:46.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.266+0000 7fbec12a9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbebc0b38d0 con 0x7fbebc15d460 2024-01-27T02:35:46.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.266+0000 7fbec12a9700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbebc0d0390 con 0x7fbebc0cf280 2024-01-27T02:35:46.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.266+0000 7fbec12a9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc0cf780 con 0x7fbebc15d830 2024-01-27T02:35:46.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.266+0000 7fbebaffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbebc15d460 0x7fbebc0d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:46.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.266+0000 7fbebaffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbebc15d460 0x7fbebc0d0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44028/0 (socket says 172.21.15.7:44028) 2024-01-27T02:35:46.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.266+0000 7fbebaffd700 1 -- 172.21.15.7:0/1650262427 learned_addr learned my addr 172.21.15.7:0/1650262427 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:46.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.266+0000 7fbeba7fc700 1 --2- 172.21.15.7:0/1650262427 >> v2:172.21.15.188:3300/0 conn(0x7fbebc0cf280 0x7fbebc15e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:46.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.266+0000 7fbebb7fe700 1 --2- 172.21.15.7:0/1650262427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbebc15d830 0x7fbebc0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:46.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.266+0000 7fbebaffd700 1 -- 172.21.15.7:0/1650262427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbebc15d830 msgr2=0x7fbebc0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:46.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.266+0000 7fbebaffd700 1 --2- 172.21.15.7:0/1650262427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbebc15d830 0x7fbebc0cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.267+0000 7fbebaffd700 1 -- 172.21.15.7:0/1650262427 >> v2:172.21.15.188:3300/0 conn(0x7fbebc0cf280 msgr2=0x7fbebc15e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:46.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.267+0000 7fbebaffd700 1 --2- 172.21.15.7:0/1650262427 >> v2:172.21.15.188:3300/0 conn(0x7fbebc0cf280 0x7fbebc15e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.267+0000 7fbebaffd700 1 -- 172.21.15.7:0/1650262427 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc0cfa60 con 0x7fbebc15d460 2024-01-27T02:35:46.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.267+0000 7fbeba7fc700 1 --2- 172.21.15.7:0/1650262427 >> v2:172.21.15.188:3300/0 conn(0x7fbebc0cf280 0x7fbebc15e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:35:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.267+0000 7fbebb7fe700 1 --2- 172.21.15.7:0/1650262427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbebc15d830 0x7fbebc0cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:35:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.267+0000 7fbebaffd700 1 --2- 172.21.15.7:0/1650262427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbebc15d460 0x7fbebc0d0820 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbebc08a1e0 tx=0x7fbeac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f81857582bc687e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.267+0000 7fbeb9ffb700 1 -- 172.21.15.7:0/1650262427 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac00b860 con 0x7fbebc15d460 2024-01-27T02:35:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.267+0000 7fbeb9ffb700 1 -- 172.21.15.7:0/1650262427 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeac00b9c0 con 0x7fbebc15d460 2024-01-27T02:35:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.267+0000 7fbeb9ffb700 1 -- 172.21.15.7:0/1650262427 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac00bd60 con 0x7fbebc15d460 2024-01-27T02:35:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.267+0000 7fbec12a9700 1 -- 172.21.15.7:0/1650262427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbebc15d460 msgr2=0x7fbebc0d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.267+0000 7fbec12a9700 1 --2- 172.21.15.7:0/1650262427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbebc15d460 0x7fbebc0d0820 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbebc08a1e0 tx=0x7fbeac009250 comp rx=0 tx=0).stop 2024-01-27T02:35:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.268+0000 7fbec12a9700 1 -- 172.21.15.7:0/1650262427 shutdown_connections 2024-01-27T02:35:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.268+0000 7fbec12a9700 1 --2- 172.21.15.7:0/1650262427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbebc15d830 0x7fbebc0cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.268+0000 7fbec12a9700 1 --2- 172.21.15.7:0/1650262427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbebc15d460 0x7fbebc0d0820 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.268+0000 7fbec12a9700 1 --2- 172.21.15.7:0/1650262427 >> v2:172.21.15.188:3300/0 conn(0x7fbebc0cf280 0x7fbebc15e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.268+0000 7fbec12a9700 1 -- 172.21.15.7:0/1650262427 >> 172.21.15.7:0/1650262427 conn(0x7fbebc063580 msgr2=0x7fbebc063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.268+0000 7fbec12a9700 1 -- 172.21.15.7:0/1650262427 shutdown_connections 2024-01-27T02:35:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.268+0000 7fbec12a9700 1 -- 172.21.15.7:0/1650262427 wait complete. 2024-01-27T02:35:46.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.269+0000 7fbec12a9700 1 Processor -- start 2024-01-27T02:35:46.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.269+0000 7fbec12a9700 1 -- start start 2024-01-27T02:35:46.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.270+0000 7fbec12a9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbebc0cf280 0x7fbebc155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:46.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.270+0000 7fbec12a9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbebc15d460 0x7fbebc141050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:46.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.270+0000 7fbec12a9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbebc15d830 0x7fbebc16a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:46.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.270+0000 7fbec12a9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc1555c0 con 0x7fbebc15d830 2024-01-27T02:35:46.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.270+0000 7fbec12a9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc16b040 con 0x7fbebc15d460 2024-01-27T02:35:46.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.270+0000 7fbec12a9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc16b320 con 0x7fbebc0cf280 2024-01-27T02:35:46.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.270+0000 7fbebb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbebc15d830 0x7fbebc16a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:46.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.270+0000 7fbebb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbebc15d830 0x7fbebc16a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44034/0 (socket says 172.21.15.7:44034) 2024-01-27T02:35:46.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.270+0000 7fbebb7fe700 1 -- 172.21.15.7:0/603911040 learned_addr learned my addr 172.21.15.7:0/603911040 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:46.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.270+0000 7fbeba7fc700 1 --2- 172.21.15.7:0/603911040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbebc15d460 0x7fbebc141050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:46.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.270+0000 7fbebaffd700 1 --2- 172.21.15.7:0/603911040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbebc0cf280 0x7fbebc155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:46.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.270+0000 7fbebb7fe700 1 -- 172.21.15.7:0/603911040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbebc0cf280 msgr2=0x7fbebc155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:46.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.270+0000 7fbebb7fe700 1 --2- 172.21.15.7:0/603911040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbebc0cf280 0x7fbebc155a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.270+0000 7fbebb7fe700 1 -- 172.21.15.7:0/603911040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbebc15d460 msgr2=0x7fbebc141050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:46.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.270+0000 7fbebb7fe700 1 --2- 172.21.15.7:0/603911040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbebc15d460 0x7fbebc141050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.270+0000 7fbebb7fe700 1 -- 172.21.15.7:0/603911040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac0094f0 con 0x7fbebc15d830 2024-01-27T02:35:46.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.270+0000 7fbebaffd700 1 --2- 172.21.15.7:0/603911040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbebc0cf280 0x7fbebc155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:35:46.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.270+0000 7fbebb7fe700 1 --2- 172.21.15.7:0/603911040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbebc15d830 0x7fbebc16a4b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400cef0 tx=0x7fbeb4007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:46.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.271+0000 7fbeb2ffd700 1 -- 172.21.15.7:0/603911040 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb400add0 con 0x7fbebc15d830 2024-01-27T02:35:46.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.271+0000 7fbeb2ffd700 1 -- 172.21.15.7:0/603911040 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb4016420 con 0x7fbebc15d830 2024-01-27T02:35:46.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.271+0000 7fbec12a9700 1 -- 172.21.15.7:0/603911040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbebc155250 con 0x7fbebc15d830 2024-01-27T02:35:46.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.271+0000 7fbeb2ffd700 1 -- 172.21.15.7:0/603911040 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb40167e0 con 0x7fbebc15d830 2024-01-27T02:35:46.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.271+0000 7fbec12a9700 1 -- 172.21.15.7:0/603911040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbebc155f90 con 0x7fbebc15d830 2024-01-27T02:35:46.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.272+0000 7fbec12a9700 1 -- 172.21.15.7:0/603911040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbebc0d0cc0 con 0x7fbebc15d830 2024-01-27T02:35:46.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.272+0000 7fbeb2ffd700 1 -- 172.21.15.7:0/603911040 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbeb4016bd0 con 0x7fbebc15d830 2024-01-27T02:35:46.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.273+0000 7fbeb2ffd700 1 --2- 172.21.15.7:0/603911040 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbe9c066f30 0x7fbe9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:46.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.273+0000 7fbeb2ffd700 1 -- 172.21.15.7:0/603911040 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbeb4012070 con 0x7fbebc15d830 2024-01-27T02:35:46.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.273+0000 7fbeb2ffd700 1 -- 172.21.15.7:0/603911040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=69}) v3 -- 0x7fbe9c06ea50 con 0x7fbebc15d830 2024-01-27T02:35:46.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.273+0000 7fbebaffd700 1 --2- 172.21.15.7:0/603911040 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbe9c066f30 0x7fbe9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:46.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.274+0000 7fbebaffd700 1 --2- 172.21.15.7:0/603911040 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbe9c066f30 0x7fbe9c0693e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbebc08a1e0 tx=0x7fbeac017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:46.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.277+0000 7fbeb2ffd700 1 -- 172.21.15.7:0/603911040 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbeb404e5a0 con 0x7fbebc15d830 2024-01-27T02:35:46.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.479+0000 7fbec12a9700 1 -- 172.21.15.7:0/603911040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbebc072060 con 0x7fbebc15d830 2024-01-27T02:35:46.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.480+0000 7fbeb2ffd700 1 -- 172.21.15.7:0/603911040 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbeb40520a0 con 0x7fbebc15d830 2024-01-27T02:35:46.481 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:35:46.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.482+0000 7fbec12a9700 1 -- 172.21.15.7:0/603911040 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbe9c066f30 msgr2=0x7fbe9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:46.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.482+0000 7fbec12a9700 1 --2- 172.21.15.7:0/603911040 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbe9c066f30 0x7fbe9c0693e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbebc08a1e0 tx=0x7fbeac017000 comp rx=0 tx=0).stop 2024-01-27T02:35:46.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.482+0000 7fbec12a9700 1 -- 172.21.15.7:0/603911040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbebc15d830 msgr2=0x7fbebc16a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:46.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.482+0000 7fbec12a9700 1 --2- 172.21.15.7:0/603911040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbebc15d830 0x7fbebc16a4b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400cef0 tx=0x7fbeb4007860 comp rx=0 tx=0).stop 2024-01-27T02:35:46.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.482+0000 7fbec12a9700 1 -- 172.21.15.7:0/603911040 shutdown_connections 2024-01-27T02:35:46.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.482+0000 7fbec12a9700 1 --2- 172.21.15.7:0/603911040 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbe9c066f30 0x7fbe9c0693e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.482+0000 7fbec12a9700 1 --2- 172.21.15.7:0/603911040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbebc15d830 0x7fbebc16a4b0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.482+0000 7fbec12a9700 1 --2- 172.21.15.7:0/603911040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbebc0cf280 0x7fbebc155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.482+0000 7fbec12a9700 1 --2- 172.21.15.7:0/603911040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbebc15d460 0x7fbebc141050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.482+0000 7fbec12a9700 1 -- 172.21.15.7:0/603911040 >> 172.21.15.7:0/603911040 conn(0x7fbebc063580 msgr2=0x7fbebc0b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:46.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.483+0000 7fbec12a9700 1 -- 172.21.15.7:0/603911040 shutdown_connections 2024-01-27T02:35:46.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.483+0000 7fbec12a9700 1 -- 172.21.15.7:0/603911040 wait complete. 2024-01-27T02:35:46.496 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-27T02:35:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.631+0000 7fd875dab700 1 Processor -- start 2024-01-27T02:35:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.631+0000 7fd875dab700 1 -- start start 2024-01-27T02:35:46.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.632+0000 7fd875dab700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd8700af7e0 0x7fd87006e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.632+0000 7fd875dab700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd8700ca4c0 0x7fd8700cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.632+0000 7fd875dab700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd87006ed10 con 0x7fd8700b13d0 2024-01-27T02:35:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.632+0000 7fd875dab700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd87006eff0 con 0x7fd8700af7e0 2024-01-27T02:35:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.632+0000 7fd875dab700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd87006f2d0 con 0x7fd8700ca4c0 2024-01-27T02:35:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.632+0000 7fd86f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd8700af7e0 0x7fd87006e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.632+0000 7fd86effd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd8700b13d0 0x7fd8700c9db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56026/0 (socket says 172.21.15.7:56026) 2024-01-27T02:35:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.632+0000 7fd86effd700 1 -- 172.21.15.7:0/2799999606 learned_addr learned my addr 172.21.15.7:0/2799999606 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:35:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.633+0000 7fd86ffff700 1 --2- 172.21.15.7:0/2799999606 >> v2:172.21.15.188:3300/0 conn(0x7fd8700ca4c0 0x7fd8700cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.633+0000 7fd86f7fe700 1 -- 172.21.15.7:0/2799999606 >> v2:172.21.15.188:3300/0 conn(0x7fd8700ca4c0 msgr2=0x7fd8700cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.633+0000 7fd86f7fe700 1 --2- 172.21.15.7:0/2799999606 >> v2:172.21.15.188:3300/0 conn(0x7fd8700ca4c0 0x7fd8700cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.633+0000 7fd86f7fe700 1 -- 172.21.15.7:0/2799999606 >> v1:172.21.15.7:6789/0 conn(0x7fd8700b13d0 legacy=0x7fd8700c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.633+0000 7fd86f7fe700 1 -- 172.21.15.7:0/2799999606 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd87006f5b0 con 0x7fd8700af7e0 2024-01-27T02:35:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.633+0000 7fd86e7fc700 1 -- 172.21.15.7:0/2799999606 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801135307 0 0) 0x7fd87006ed10 con 0x7fd8700b13d0 2024-01-27T02:35:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.633+0000 7fd86f7fe700 1 --2- 172.21.15.7:0/2799999606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8700af7e0 0x7fd87006e630 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd8700cd360 tx=0x7fd8600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff913cc9321b0a6 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.633+0000 7fd86e7fc700 1 -- 172.21.15.7:0/2799999606 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860011630 con 0x7fd8700af7e0 2024-01-27T02:35:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.633+0000 7fd86e7fc700 1 -- 172.21.15.7:0/2799999606 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd860011790 con 0x7fd8700af7e0 2024-01-27T02:35:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.633+0000 7fd86e7fc700 1 -- 172.21.15.7:0/2799999606 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860011b30 con 0x7fd8700af7e0 2024-01-27T02:35:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.634+0000 7fd875dab700 1 -- 172.21.15.7:0/2799999606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8700af7e0 msgr2=0x7fd87006e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.634+0000 7fd875dab700 1 --2- 172.21.15.7:0/2799999606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8700af7e0 0x7fd87006e630 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd8700cd360 tx=0x7fd8600091a0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.634+0000 7fd875dab700 1 -- 172.21.15.7:0/2799999606 shutdown_connections 2024-01-27T02:35:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.634+0000 7fd875dab700 1 --2- 172.21.15.7:0/2799999606 >> v2:172.21.15.188:3300/0 conn(0x7fd8700ca4c0 0x7fd8700cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.634+0000 7fd875dab700 1 --2- 172.21.15.7:0/2799999606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8700af7e0 0x7fd87006e630 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.634+0000 7fd875dab700 1 -- 172.21.15.7:0/2799999606 >> 172.21.15.7:0/2799999606 conn(0x7fd8700cd3b0 msgr2=0x7fd8700cd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.634+0000 7fd875dab700 1 -- 172.21.15.7:0/2799999606 shutdown_connections 2024-01-27T02:35:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.634+0000 7fd875dab700 1 -- 172.21.15.7:0/2799999606 wait complete. 2024-01-27T02:35:46.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.636+0000 7fd875dab700 1 Processor -- start 2024-01-27T02:35:46.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.636+0000 7fd875dab700 1 -- start start 2024-01-27T02:35:46.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.637+0000 7fd875dab700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8700af7e0 0x7fd87016d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:46.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.637+0000 7fd875dab700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8700b13d0 0x7fd87015c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:46.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.637+0000 7fd875dab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8700ca4c0 0x7fd87016ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:46.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.637+0000 7fd875dab700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87016d5c0 con 0x7fd8700ca4c0 2024-01-27T02:35:46.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.637+0000 7fd875dab700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87015cad0 con 0x7fd8700b13d0 2024-01-27T02:35:46.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.637+0000 7fd875dab700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd87015cd00 con 0x7fd8700af7e0 2024-01-27T02:35:46.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.637+0000 7fd86ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8700ca4c0 0x7fd87016ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:46.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.637+0000 7fd86ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8700ca4c0 0x7fd87016ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44062/0 (socket says 172.21.15.7:44062) 2024-01-27T02:35:46.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.637+0000 7fd86ffff700 1 -- 172.21.15.7:0/270448828 learned_addr learned my addr 172.21.15.7:0/270448828 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:46.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.637+0000 7fd86f7fe700 1 --2- 172.21.15.7:0/270448828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8700af7e0 0x7fd87016d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:46.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.637+0000 7fd86effd700 1 --2- 172.21.15.7:0/270448828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8700b13d0 0x7fd87015c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:46.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.637+0000 7fd86ffff700 1 -- 172.21.15.7:0/270448828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8700af7e0 msgr2=0x7fd87016d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:46.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.637+0000 7fd86ffff700 1 --2- 172.21.15.7:0/270448828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8700af7e0 0x7fd87016d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.637+0000 7fd86ffff700 1 -- 172.21.15.7:0/270448828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8700b13d0 msgr2=0x7fd87015c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.637+0000 7fd86ffff700 1 --2- 172.21.15.7:0/270448828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8700b13d0 0x7fd87015c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.637+0000 7fd86ffff700 1 -- 172.21.15.7:0/270448828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd860009420 con 0x7fd8700ca4c0 2024-01-27T02:35:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.637+0000 7fd86ffff700 1 --2- 172.21.15.7:0/270448828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8700ca4c0 0x7fd87016ab20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd868009ef0 tx=0x7fd86800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.638+0000 7fd8577fe700 1 -- 172.21.15.7:0/270448828 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86800cc50 con 0x7fd8700ca4c0 2024-01-27T02:35:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.638+0000 7fd8577fe700 1 -- 172.21.15.7:0/270448828 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86800f040 con 0x7fd8700ca4c0 2024-01-27T02:35:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.638+0000 7fd875dab700 1 -- 172.21.15.7:0/270448828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd87015cfe0 con 0x7fd8700ca4c0 2024-01-27T02:35:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.638+0000 7fd8577fe700 1 -- 172.21.15.7:0/270448828 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868014480 con 0x7fd8700ca4c0 2024-01-27T02:35:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.638+0000 7fd875dab700 1 -- 172.21.15.7:0/270448828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd87015d5a0 con 0x7fd8700ca4c0 2024-01-27T02:35:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.639+0000 7fd875dab700 1 -- 172.21.15.7:0/270448828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd87004c7b0 con 0x7fd8700ca4c0 2024-01-27T02:35:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.639+0000 7fd8577fe700 1 -- 172.21.15.7:0/270448828 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd8680147f0 con 0x7fd8700ca4c0 2024-01-27T02:35:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.640+0000 7fd8577fe700 1 --2- 172.21.15.7:0/270448828 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd85c066f30 0x7fd85c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.640+0000 7fd8577fe700 1 -- 172.21.15.7:0/270448828 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd868085370 con 0x7fd8700ca4c0 2024-01-27T02:35:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.640+0000 7fd86f7fe700 1 --2- 172.21.15.7:0/270448828 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd85c066f30 0x7fd85c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.640+0000 7fd8577fe700 1 -- 172.21.15.7:0/270448828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=69}) v3 -- 0x7fd85c06ea50 con 0x7fd8700ca4c0 2024-01-27T02:35:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.641+0000 7fd86f7fe700 1 --2- 172.21.15.7:0/270448828 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd85c066f30 0x7fd85c0693e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd8600091e0 tx=0x7fd860017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.644+0000 7fd8577fe700 1 -- 172.21.15.7:0/270448828 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd868018060 con 0x7fd8700ca4c0 2024-01-27T02:35:46.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.848+0000 7fd875dab700 1 -- 172.21.15.7:0/270448828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd870072060 con 0x7fd8700ca4c0 2024-01-27T02:35:46.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.849+0000 7fd8577fe700 1 -- 172.21.15.7:0/270448828 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v68) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd868052540 con 0x7fd8700ca4c0 2024-01-27T02:35:46.852 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-27T02:35:46.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.851+0000 7fd875dab700 1 -- 172.21.15.7:0/270448828 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd85c066f30 msgr2=0x7fd85c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:46.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.851+0000 7fd875dab700 1 --2- 172.21.15.7:0/270448828 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd85c066f30 0x7fd85c0693e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd8600091e0 tx=0x7fd860017000 comp rx=0 tx=0).stop 2024-01-27T02:35:46.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.851+0000 7fd875dab700 1 -- 172.21.15.7:0/270448828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8700ca4c0 msgr2=0x7fd87016ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:46.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.851+0000 7fd875dab700 1 --2- 172.21.15.7:0/270448828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8700ca4c0 0x7fd87016ab20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd868009ef0 tx=0x7fd86800b040 comp rx=0 tx=0).stop 2024-01-27T02:35:46.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.851+0000 7fd875dab700 1 -- 172.21.15.7:0/270448828 shutdown_connections 2024-01-27T02:35:46.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.851+0000 7fd875dab700 1 --2- 172.21.15.7:0/270448828 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd85c066f30 0x7fd85c0693e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.851+0000 7fd875dab700 1 --2- 172.21.15.7:0/270448828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8700ca4c0 0x7fd87016ab20 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.851+0000 7fd875dab700 1 --2- 172.21.15.7:0/270448828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8700af7e0 0x7fd87016d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.851+0000 7fd875dab700 1 --2- 172.21.15.7:0/270448828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8700b13d0 0x7fd87015c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:46.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.851+0000 7fd875dab700 1 -- 172.21.15.7:0/270448828 >> 172.21.15.7:0/270448828 conn(0x7fd8700cd3b0 msgr2=0x7fd8700697e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:46.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.851+0000 7fd875dab700 1 -- 172.21.15.7:0/270448828 shutdown_connections 2024-01-27T02:35:46.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.851+0000 7fd875dab700 1 -- 172.21.15.7:0/270448828 wait complete. 2024-01-27T02:35:46.865 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-27T02:35:47.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.999+0000 7fef14e42700 1 Processor -- start 2024-01-27T02:35:47.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:46.999+0000 7fef14e42700 1 -- start start 2024-01-27T02:35:47.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.000+0000 7fef14e42700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fef10156b60 0x7fef100ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:47.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.000+0000 7fef14e42700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef100ad9f0 0x7fef100aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:47.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.000+0000 7fef14e42700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fef100af470 0x7fef1015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:47.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.000+0000 7fef14e42700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef100aeaa0 con 0x7fef100af470 2024-01-27T02:35:47.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.000+0000 7fef14e42700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef100ade90 con 0x7fef10156b60 2024-01-27T02:35:47.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.000+0000 7fef14e42700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef100ae170 con 0x7fef100ad9f0 2024-01-27T02:35:47.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.001+0000 7fef0ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fef100af470 0x7fef1015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:47.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.001+0000 7fef0ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fef100af470 0x7fef1015a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44074/0 (socket says 172.21.15.7:44074) 2024-01-27T02:35:47.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.001+0000 7fef0ed9d700 1 -- 172.21.15.7:0/2199417900 learned_addr learned my addr 172.21.15.7:0/2199417900 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:47.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.001+0000 7fef07fff700 1 --2- 172.21.15.7:0/2199417900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef100ad9f0 0x7fef100aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:47.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.001+0000 7fef0e59c700 1 --2- 172.21.15.7:0/2199417900 >> v2:172.21.15.188:3300/0 conn(0x7fef10156b60 0x7fef100ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:47.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.001+0000 7fef0ed9d700 1 -- 172.21.15.7:0/2199417900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef100ad9f0 msgr2=0x7fef100aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:47.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.001+0000 7fef0ed9d700 1 --2- 172.21.15.7:0/2199417900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef100ad9f0 0x7fef100aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.001+0000 7fef0ed9d700 1 -- 172.21.15.7:0/2199417900 >> v2:172.21.15.188:3300/0 conn(0x7fef10156b60 msgr2=0x7fef100ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:47.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.001+0000 7fef0ed9d700 1 --2- 172.21.15.7:0/2199417900 >> v2:172.21.15.188:3300/0 conn(0x7fef10156b60 0x7fef100ad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.001+0000 7fef0ed9d700 1 -- 172.21.15.7:0/2199417900 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef100ae450 con 0x7fef100af470 2024-01-27T02:35:47.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.001+0000 7fef0ed9d700 1 --2- 172.21.15.7:0/2199417900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef100af470 0x7fef1015a9b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fef10064f00 tx=0x7fef08009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee8b57d24262dbc server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:47.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.001+0000 7fef0dd9b700 1 -- 172.21.15.7:0/2199417900 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0800e070 con 0x7fef100af470 2024-01-27T02:35:47.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.001+0000 7fef0dd9b700 1 -- 172.21.15.7:0/2199417900 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0800ba50 con 0x7fef100af470 2024-01-27T02:35:47.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.002+0000 7fef0dd9b700 1 -- 172.21.15.7:0/2199417900 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef080092e0 con 0x7fef100af470 2024-01-27T02:35:47.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.002+0000 7fef14e42700 1 -- 172.21.15.7:0/2199417900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef100af470 msgr2=0x7fef1015a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:47.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.002+0000 7fef14e42700 1 --2- 172.21.15.7:0/2199417900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef100af470 0x7fef1015a9b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fef10064f00 tx=0x7fef08009840 comp rx=0 tx=0).stop 2024-01-27T02:35:47.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.002+0000 7fef14e42700 1 -- 172.21.15.7:0/2199417900 shutdown_connections 2024-01-27T02:35:47.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.002+0000 7fef14e42700 1 --2- 172.21.15.7:0/2199417900 >> v2:172.21.15.188:3300/0 conn(0x7fef10156b60 0x7fef100ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.002+0000 7fef14e42700 1 --2- 172.21.15.7:0/2199417900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef100af470 0x7fef1015a9b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.002+0000 7fef14e42700 1 --2- 172.21.15.7:0/2199417900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef100ad9f0 0x7fef100aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.002+0000 7fef14e42700 1 -- 172.21.15.7:0/2199417900 >> 172.21.15.7:0/2199417900 conn(0x7fef10068480 msgr2=0x7fef100cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:47.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.002+0000 7fef14e42700 1 -- 172.21.15.7:0/2199417900 shutdown_connections 2024-01-27T02:35:47.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.002+0000 7fef14e42700 1 -- 172.21.15.7:0/2199417900 wait complete. 2024-01-27T02:35:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.003+0000 7fef14e42700 1 Processor -- start 2024-01-27T02:35:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.003+0000 7fef14e42700 1 -- start start 2024-01-27T02:35:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.004+0000 7fef14e42700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef100ad9f0 0x7fef1016ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.004+0000 7fef14e42700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef100af470 0x7fef1014fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.004+0000 7fef14e42700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef10156b60 0x7fef1014a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.004+0000 7fef14e42700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1014e900 con 0x7fef100af470 2024-01-27T02:35:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.004+0000 7fef14e42700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1014ebe0 con 0x7fef100ad9f0 2024-01-27T02:35:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.004+0000 7fef14e42700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef10150070 con 0x7fef10156b60 2024-01-27T02:35:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.004+0000 7fef07fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef100af470 0x7fef1014fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.004+0000 7fef07fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef100af470 0x7fef1014fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44090/0 (socket says 172.21.15.7:44090) 2024-01-27T02:35:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.004+0000 7fef07fff700 1 -- 172.21.15.7:0/1367825846 learned_addr learned my addr 172.21.15.7:0/1367825846 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.004+0000 7fef0e59c700 1 --2- 172.21.15.7:0/1367825846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef100ad9f0 0x7fef1016ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.004+0000 7fef0ed9d700 1 --2- 172.21.15.7:0/1367825846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef10156b60 0x7fef1014a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.005+0000 7fef07fff700 1 -- 172.21.15.7:0/1367825846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef10156b60 msgr2=0x7fef1014a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.005+0000 7fef07fff700 1 --2- 172.21.15.7:0/1367825846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef10156b60 0x7fef1014a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.005+0000 7fef07fff700 1 -- 172.21.15.7:0/1367825846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef100ad9f0 msgr2=0x7fef1016ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.005+0000 7fef07fff700 1 --2- 172.21.15.7:0/1367825846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef100ad9f0 0x7fef1016ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.005+0000 7fef07fff700 1 -- 172.21.15.7:0/1367825846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef08009ae0 con 0x7fef100af470 2024-01-27T02:35:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.005+0000 7fef07fff700 1 --2- 172.21.15.7:0/1367825846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef100af470 0x7fef1014fb30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7feef800d890 tx=0x7feef8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.005+0000 7fef067fc700 1 -- 172.21.15.7:0/1367825846 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef80125f0 con 0x7fef100af470 2024-01-27T02:35:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.005+0000 7fef067fc700 1 -- 172.21.15.7:0/1367825846 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef8012750 con 0x7fef100af470 2024-01-27T02:35:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.005+0000 7fef067fc700 1 -- 172.21.15.7:0/1367825846 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef8012af0 con 0x7fef100af470 2024-01-27T02:35:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.005+0000 7fef14e42700 1 -- 172.21.15.7:0/1367825846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef100cdcf0 con 0x7fef100af470 2024-01-27T02:35:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.005+0000 7fef14e42700 1 -- 172.21.15.7:0/1367825846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef100ce2b0 con 0x7fef100af470 2024-01-27T02:35:47.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.006+0000 7fef14e42700 1 -- 172.21.15.7:0/1367825846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef1014ed70 con 0x7fef100af470 2024-01-27T02:35:47.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.006+0000 7fef067fc700 1 -- 172.21.15.7:0/1367825846 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7feef801e460 con 0x7fef100af470 2024-01-27T02:35:47.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.007+0000 7fef067fc700 1 --2- 172.21.15.7:0/1367825846 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7feef0066f30 0x7feef00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:47.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.007+0000 7fef067fc700 1 -- 172.21.15.7:0/1367825846 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feef8084a60 con 0x7fef100af470 2024-01-27T02:35:47.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.007+0000 7fef067fc700 1 -- 172.21.15.7:0/1367825846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=69}) v3 -- 0x7feef006ea50 con 0x7fef100af470 2024-01-27T02:35:47.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.007+0000 7fef0e59c700 1 --2- 172.21.15.7:0/1367825846 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7feef0066f30 0x7feef00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:47.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.008+0000 7fef0e59c700 1 --2- 172.21.15.7:0/1367825846 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7feef0066f30 0x7feef00693e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fef00006fd0 tx=0x7fef00008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:47.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.011+0000 7fef067fc700 1 -- 172.21.15.7:0/1367825846 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feef8055760 con 0x7fef100af470 2024-01-27T02:35:47.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.214+0000 7fef14e42700 1 -- 172.21.15.7:0/1367825846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fef10072060 con 0x7fef100af470 2024-01-27T02:35:47.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.215+0000 7fef067fc700 1 -- 172.21.15.7:0/1367825846 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v68) v1 ==== 100+0+11 (secure 0 0 0) 0x7feef80552f0 con 0x7fef100af470 2024-01-27T02:35:47.215 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-27T02:35:47.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.217+0000 7fef14e42700 1 -- 172.21.15.7:0/1367825846 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7feef0066f30 msgr2=0x7feef00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:47.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.217+0000 7fef14e42700 1 --2- 172.21.15.7:0/1367825846 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7feef0066f30 0x7feef00693e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fef00006fd0 tx=0x7fef00008040 comp rx=0 tx=0).stop 2024-01-27T02:35:47.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.217+0000 7fef14e42700 1 -- 172.21.15.7:0/1367825846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef100af470 msgr2=0x7fef1014fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:47.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.217+0000 7fef14e42700 1 --2- 172.21.15.7:0/1367825846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef100af470 0x7fef1014fb30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7feef800d890 tx=0x7feef8008040 comp rx=0 tx=0).stop 2024-01-27T02:35:47.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.217+0000 7fef14e42700 1 -- 172.21.15.7:0/1367825846 shutdown_connections 2024-01-27T02:35:47.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.217+0000 7fef14e42700 1 --2- 172.21.15.7:0/1367825846 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7feef0066f30 0x7feef00693e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.217+0000 7fef14e42700 1 --2- 172.21.15.7:0/1367825846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef10156b60 0x7fef1014a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.217+0000 7fef14e42700 1 --2- 172.21.15.7:0/1367825846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef100ad9f0 0x7fef1016ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.217+0000 7fef14e42700 1 --2- 172.21.15.7:0/1367825846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef100af470 0x7fef1014fb30 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.217+0000 7fef14e42700 1 -- 172.21.15.7:0/1367825846 >> 172.21.15.7:0/1367825846 conn(0x7fef10068480 msgr2=0x7fef10153030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:47.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.217+0000 7fef14e42700 1 -- 172.21.15.7:0/1367825846 shutdown_connections 2024-01-27T02:35:47.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.217+0000 7fef14e42700 1 -- 172.21.15.7:0/1367825846 wait complete. 2024-01-27T02:35:47.231 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:35:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.362+0000 7f116c46d700 1 Processor -- start 2024-01-27T02:35:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.362+0000 7f116c46d700 1 -- start start 2024-01-27T02:35:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.363+0000 7f116c46d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1164157ca0 0x7f11640c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.363+0000 7f116c46d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f11640c3ae0 0x7f11640cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.363+0000 7f116c46d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1164158070 0x7f11640cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.363+0000 7f116c46d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11640ca020 con 0x7f11640c3ae0 2024-01-27T02:35:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.363+0000 7f116c46d700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11640c4f40 con 0x7f1164157ca0 2024-01-27T02:35:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.363+0000 7f116c46d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11640c5210 con 0x7f1164158070 2024-01-27T02:35:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.364+0000 7f1169a08700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f11640c3ae0 0x7f11640cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:47.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.364+0000 7f1169a08700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f11640c3ae0 0x7f11640cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44104/0 (socket says 172.21.15.7:44104) 2024-01-27T02:35:47.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.364+0000 7f1169a08700 1 -- 172.21.15.7:0/3823729958 learned_addr learned my addr 172.21.15.7:0/3823729958 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:47.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.364+0000 7f116aa0a700 1 --2- 172.21.15.7:0/3823729958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1164158070 0x7f11640cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:47.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.364+0000 7f116a209700 1 --2- 172.21.15.7:0/3823729958 >> v2:172.21.15.188:3300/0 conn(0x7f1164157ca0 0x7f11640c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:47.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.364+0000 7f1169a08700 1 -- 172.21.15.7:0/3823729958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1164158070 msgr2=0x7f11640cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:47.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.364+0000 7f1169a08700 1 --2- 172.21.15.7:0/3823729958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1164158070 0x7f11640cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.364+0000 7f1169a08700 1 -- 172.21.15.7:0/3823729958 >> v2:172.21.15.188:3300/0 conn(0x7f1164157ca0 msgr2=0x7f11640c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:47.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.364+0000 7f1169a08700 1 --2- 172.21.15.7:0/3823729958 >> v2:172.21.15.188:3300/0 conn(0x7f1164157ca0 0x7f11640c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.364+0000 7f1169a08700 1 -- 172.21.15.7:0/3823729958 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11640c54f0 con 0x7f11640c3ae0 2024-01-27T02:35:47.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.364+0000 7f1169a08700 1 --2- 172.21.15.7:0/3823729958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11640c3ae0 0x7f11640cd4e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f116406d710 tx=0x7f1154009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2b34e60092d148 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:47.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.364+0000 7f1169207700 1 -- 172.21.15.7:0/3823729958 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115400b860 con 0x7f11640c3ae0 2024-01-27T02:35:47.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.364+0000 7f1169207700 1 -- 172.21.15.7:0/3823729958 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115400b9c0 con 0x7f11640c3ae0 2024-01-27T02:35:47.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.364+0000 7f1169207700 1 -- 172.21.15.7:0/3823729958 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115400bd60 con 0x7f11640c3ae0 2024-01-27T02:35:47.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.365+0000 7f116c46d700 1 -- 172.21.15.7:0/3823729958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11640c3ae0 msgr2=0x7f11640cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:47.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.365+0000 7f116c46d700 1 --2- 172.21.15.7:0/3823729958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11640c3ae0 0x7f11640cd4e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f116406d710 tx=0x7f1154009250 comp rx=0 tx=0).stop 2024-01-27T02:35:47.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.365+0000 7f116c46d700 1 -- 172.21.15.7:0/3823729958 shutdown_connections 2024-01-27T02:35:47.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.365+0000 7f116c46d700 1 --2- 172.21.15.7:0/3823729958 >> v2:172.21.15.188:3300/0 conn(0x7f1164157ca0 0x7f11640c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.365+0000 7f116c46d700 1 --2- 172.21.15.7:0/3823729958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1164158070 0x7f11640cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.365+0000 7f116c46d700 1 --2- 172.21.15.7:0/3823729958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11640c3ae0 0x7f11640cd4e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.365+0000 7f116c46d700 1 -- 172.21.15.7:0/3823729958 >> 172.21.15.7:0/3823729958 conn(0x7f1164098020 msgr2=0x7f11640711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:47.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.365+0000 7f116c46d700 1 -- 172.21.15.7:0/3823729958 shutdown_connections 2024-01-27T02:35:47.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.365+0000 7f116c46d700 1 -- 172.21.15.7:0/3823729958 wait complete. 2024-01-27T02:35:47.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.366+0000 7f116c46d700 1 Processor -- start 2024-01-27T02:35:47.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.366+0000 7f116c46d700 1 -- start start 2024-01-27T02:35:47.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.367+0000 7f116c46d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11640c3ae0 0x7f11640b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:47.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.367+0000 7f116c46d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1164157ca0 0x7f11640c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.367+0000 7f116c46d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1164158070 0x7f116416b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.367+0000 7f116c46d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11640c8c90 con 0x7f1164157ca0 2024-01-27T02:35:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.367+0000 7f116c46d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11640c6300 con 0x7f11640c3ae0 2024-01-27T02:35:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.367+0000 7f116c46d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11640b2040 con 0x7f1164158070 2024-01-27T02:35:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.367+0000 7f1169a08700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1164157ca0 0x7f11640c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.367+0000 7f1169a08700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1164157ca0 0x7f11640c7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44106/0 (socket says 172.21.15.7:44106) 2024-01-27T02:35:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.367+0000 7f1169a08700 1 -- 172.21.15.7:0/1743124749 learned_addr learned my addr 172.21.15.7:0/1743124749 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.367+0000 7f116a209700 1 --2- 172.21.15.7:0/1743124749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11640c3ae0 0x7f11640b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.367+0000 7f116aa0a700 1 --2- 172.21.15.7:0/1743124749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1164158070 0x7f116416b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.368+0000 7f1169a08700 1 -- 172.21.15.7:0/1743124749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1164158070 msgr2=0x7f116416b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.368+0000 7f1169a08700 1 --2- 172.21.15.7:0/1743124749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1164158070 0x7f116416b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.368+0000 7f1169a08700 1 -- 172.21.15.7:0/1743124749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11640c3ae0 msgr2=0x7f11640b1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.368+0000 7f1169a08700 1 --2- 172.21.15.7:0/1743124749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11640c3ae0 0x7f11640b1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.368+0000 7f1169a08700 1 -- 172.21.15.7:0/1743124749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11540094f0 con 0x7f1164157ca0 2024-01-27T02:35:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.368+0000 7f116a209700 1 --2- 172.21.15.7:0/1743124749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11640c3ae0 0x7f11640b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:35:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.368+0000 7f1169a08700 1 --2- 172.21.15.7:0/1743124749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1164157ca0 0x7f11640c7f50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f116406ea80 tx=0x7f1154017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.368+0000 7f115a7fc700 1 -- 172.21.15.7:0/1743124749 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1154024440 con 0x7f1164157ca0 2024-01-27T02:35:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.368+0000 7f115a7fc700 1 -- 172.21.15.7:0/1743124749 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11540245a0 con 0x7f1164157ca0 2024-01-27T02:35:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.368+0000 7f116c46d700 1 -- 172.21.15.7:0/1743124749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f116416c190 con 0x7f1164157ca0 2024-01-27T02:35:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.368+0000 7f115a7fc700 1 -- 172.21.15.7:0/1743124749 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1154024960 con 0x7f1164157ca0 2024-01-27T02:35:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.368+0000 7f116c46d700 1 -- 172.21.15.7:0/1743124749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11640c5f90 con 0x7f1164157ca0 2024-01-27T02:35:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.369+0000 7f116c46d700 1 -- 172.21.15.7:0/1743124749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116416c6b0 con 0x7f1164157ca0 2024-01-27T02:35:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.370+0000 7f115a7fc700 1 -- 172.21.15.7:0/1743124749 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f115401a420 con 0x7f1164157ca0 2024-01-27T02:35:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.370+0000 7f115a7fc700 1 --2- 172.21.15.7:0/1743124749 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1148066f30 0x7f11480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.370+0000 7f115a7fc700 1 -- 172.21.15.7:0/1743124749 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1154086350 con 0x7f1164157ca0 2024-01-27T02:35:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.370+0000 7f115a7fc700 1 -- 172.21.15.7:0/1743124749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=69}) v3 -- 0x7f114806ea50 con 0x7f1164157ca0 2024-01-27T02:35:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.370+0000 7f116a209700 1 --2- 172.21.15.7:0/1743124749 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1148066f30 0x7f11480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.371+0000 7f116a209700 1 --2- 172.21.15.7:0/1743124749 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1148066f30 0x7f11480693e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1160006fd0 tx=0x7f1160008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:47.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.374+0000 7f115a7fc700 1 -- 172.21.15.7:0/1743124749 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f116416c6b0 con 0x7f1164157ca0 2024-01-27T02:35:47.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.634+0000 7f116c46d700 1 -- 172.21.15.7:0/1743124749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f116416c6b0 con 0x7f1164157ca0 2024-01-27T02:35:47.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.634+0000 7f115a7fc700 1 -- 172.21.15.7:0/1743124749 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f116416c6b0 con 0x7f1164157ca0 2024-01-27T02:35:47.635 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:35:47.635 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5114,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/2839704447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":2839704447},{"type":"v1","addr":"172.21.15.188:6837","nonce":2839704447}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5117,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3964291855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3964291855},{"type":"v1","addr":"172.21.15.7:6834","nonce":3964291855}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/3332325272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3332325272},{"type":"v1","addr":"172.21.15.188:6836","nonce":3332325272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:34:14.471429+0000","modified":"2024-01-27T02:34:18.068858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1399055016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1399055016},{"type":"v1","addr":"172.21.15.7:6835","nonce":1399055016}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2024-01-27T02:35:47.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.637+0000 7f116c46d700 1 -- 172.21.15.7:0/1743124749 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1148066f30 msgr2=0x7f11480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:47.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.637+0000 7f116c46d700 1 --2- 172.21.15.7:0/1743124749 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1148066f30 0x7f11480693e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1160006fd0 tx=0x7f1160008040 comp rx=0 tx=0).stop 2024-01-27T02:35:47.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.637+0000 7f116c46d700 1 -- 172.21.15.7:0/1743124749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1164157ca0 msgr2=0x7f11640c7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:47.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.637+0000 7f116c46d700 1 --2- 172.21.15.7:0/1743124749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1164157ca0 0x7f11640c7f50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f116406ea80 tx=0x7f1154017000 comp rx=0 tx=0).stop 2024-01-27T02:35:47.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.637+0000 7f116c46d700 1 -- 172.21.15.7:0/1743124749 shutdown_connections 2024-01-27T02:35:47.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.637+0000 7f116c46d700 1 --2- 172.21.15.7:0/1743124749 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1148066f30 0x7f11480693e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.637+0000 7f116c46d700 1 --2- 172.21.15.7:0/1743124749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1164158070 0x7f116416b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.637+0000 7f116c46d700 1 --2- 172.21.15.7:0/1743124749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11640c3ae0 0x7f11640b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.637+0000 7f116c46d700 1 --2- 172.21.15.7:0/1743124749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1164157ca0 0x7f11640c7f50 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.637+0000 7f116c46d700 1 -- 172.21.15.7:0/1743124749 >> 172.21.15.7:0/1743124749 conn(0x7f1164098020 msgr2=0x7f11640ca4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:47.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.637+0000 7f116c46d700 1 -- 172.21.15.7:0/1743124749 shutdown_connections 2024-01-27T02:35:47.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.637+0000 7f116c46d700 1 -- 172.21.15.7:0/1743124749 wait complete. 2024-01-27T02:35:47.639 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2024-01-27T02:35:47.651 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:35:47.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.782+0000 7f14921f1700 1 Processor -- start 2024-01-27T02:35:47.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.782+0000 7f14921f1700 1 -- start start 2024-01-27T02:35:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.783+0000 7f14921f1700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f148c06ea40 0x7f148c161a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.783+0000 7f14921f1700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f148c151940 con 0x7f148c157f50 2024-01-27T02:35:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.783+0000 7f14921f1700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f148c06c7f0 con 0x7f148c151570 2024-01-27T02:35:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.783+0000 7f14921f1700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148c13a1a0 con 0x7f148c06ea40 2024-01-27T02:35:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.783+0000 7f148affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f148c157f50 0x7f148c06e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56036/0 (socket says 172.21.15.7:56036) 2024-01-27T02:35:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.783+0000 7f148affd700 1 -- 172.21.15.7:0/3522083798 learned_addr learned my addr 172.21.15.7:0/3522083798 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:35:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.783+0000 7f148bfff700 1 --2- 172.21.15.7:0/3522083798 >> v2:172.21.15.188:3300/0 conn(0x7f148c06ea40 0x7f148c161a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.783+0000 7f148a7fc700 1 -- 172.21.15.7:0/3522083798 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 731017385 0 0) 0x7f148c151940 con 0x7f148c157f50 2024-01-27T02:35:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.784+0000 7f148a7fc700 1 -- 172.21.15.7:0/3522083798 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1474003570 con 0x7f148c157f50 2024-01-27T02:35:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.784+0000 7f148a7fc700 1 -- 172.21.15.7:0/3522083798 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3093548384 0 0) 0x7f148c06c7f0 con 0x7f148c151570 2024-01-27T02:35:47.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.784+0000 7f148a7fc700 1 -- 172.21.15.7:0/3522083798 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f148c151940 con 0x7f148c151570 2024-01-27T02:35:47.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.784+0000 7f148bfff700 1 -- 172.21.15.7:0/3522083798 >> v1:172.21.15.188:6789/0 conn(0x7f148c151570 legacy=0x7f148c06b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:47.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.784+0000 7f148bfff700 1 -- 172.21.15.7:0/3522083798 >> v1:172.21.15.7:6789/0 conn(0x7f148c157f50 legacy=0x7f148c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:47.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.784+0000 7f148bfff700 1 -- 172.21.15.7:0/3522083798 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c1583a0 con 0x7f148c06ea40 2024-01-27T02:35:47.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.784+0000 7f148bfff700 1 --2- 172.21.15.7:0/3522083798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f148c06ea40 0x7f148c161a60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1484009e60 tx=0x7f1484009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e51aef670260db0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:47.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.784+0000 7f148a7fc700 1 -- 172.21.15.7:0/3522083798 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148400e070 con 0x7f148c06ea40 2024-01-27T02:35:47.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.784+0000 7f148a7fc700 1 -- 172.21.15.7:0/3522083798 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148400bab0 con 0x7f148c06ea40 2024-01-27T02:35:47.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.784+0000 7f148a7fc700 1 -- 172.21.15.7:0/3522083798 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148400be70 con 0x7f148c06ea40 2024-01-27T02:35:47.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.785+0000 7f14921f1700 1 -- 172.21.15.7:0/3522083798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f148c06ea40 msgr2=0x7f148c161a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:47.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.785+0000 7f14921f1700 1 --2- 172.21.15.7:0/3522083798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f148c06ea40 0x7f148c161a60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1484009e60 tx=0x7f1484009780 comp rx=0 tx=0).stop 2024-01-27T02:35:47.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.785+0000 7f14921f1700 1 -- 172.21.15.7:0/3522083798 shutdown_connections 2024-01-27T02:35:47.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.785+0000 7f14921f1700 1 --2- 172.21.15.7:0/3522083798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f148c06ea40 0x7f148c161a60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.785+0000 7f14921f1700 1 -- 172.21.15.7:0/3522083798 >> 172.21.15.7:0/3522083798 conn(0x7f148c0cdc30 msgr2=0x7f148c0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:47.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.785+0000 7f14921f1700 1 -- 172.21.15.7:0/3522083798 shutdown_connections 2024-01-27T02:35:47.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.785+0000 7f14921f1700 1 -- 172.21.15.7:0/3522083798 wait complete. 2024-01-27T02:35:47.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.786+0000 7f14921f1700 1 Processor -- start 2024-01-27T02:35:47.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.786+0000 7f14921f1700 1 -- start start 2024-01-27T02:35:47.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.787+0000 7f14921f1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f148c06ea40 0x7f148c14f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.787+0000 7f14921f1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f148c151570 0x7f148c15b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.787+0000 7f14921f1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f148c157f50 0x7f148c15b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.787+0000 7f14921f1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c0b24a0 con 0x7f148c157f50 2024-01-27T02:35:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.787+0000 7f14921f1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c15be80 con 0x7f148c151570 2024-01-27T02:35:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.787+0000 7f14921f1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148c15c160 con 0x7f148c06ea40 2024-01-27T02:35:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.787+0000 7f148bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f148c157f50 0x7f148c15b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.787+0000 7f148bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f148c157f50 0x7f148c15b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44110/0 (socket says 172.21.15.7:44110) 2024-01-27T02:35:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.787+0000 7f148bfff700 1 -- 172.21.15.7:0/1713728966 learned_addr learned my addr 172.21.15.7:0/1713728966 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.787+0000 7f148affd700 1 --2- 172.21.15.7:0/1713728966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f148c151570 0x7f148c15b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.787+0000 7f148b7fe700 1 --2- 172.21.15.7:0/1713728966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f148c06ea40 0x7f148c14f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.788+0000 7f148bfff700 1 -- 172.21.15.7:0/1713728966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f148c06ea40 msgr2=0x7f148c14f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.788+0000 7f148bfff700 1 --2- 172.21.15.7:0/1713728966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f148c06ea40 0x7f148c14f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.788+0000 7f148bfff700 1 -- 172.21.15.7:0/1713728966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f148c151570 msgr2=0x7f148c15b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.788+0000 7f148bfff700 1 --2- 172.21.15.7:0/1713728966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f148c151570 0x7f148c15b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.788+0000 7f148bfff700 1 -- 172.21.15.7:0/1713728966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1484009a20 con 0x7f148c157f50 2024-01-27T02:35:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.788+0000 7f148bfff700 1 --2- 172.21.15.7:0/1713728966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f148c157f50 0x7f148c15b890 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1484009460 tx=0x7f1484019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.788+0000 7f14737fe700 1 -- 172.21.15.7:0/1713728966 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1484012dd0 con 0x7f148c157f50 2024-01-27T02:35:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.788+0000 7f14737fe700 1 -- 172.21.15.7:0/1713728966 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1484026480 con 0x7f148c157f50 2024-01-27T02:35:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.788+0000 7f14921f1700 1 -- 172.21.15.7:0/1713728966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148c0b2130 con 0x7f148c157f50 2024-01-27T02:35:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.788+0000 7f14737fe700 1 -- 172.21.15.7:0/1713728966 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1484026840 con 0x7f148c157f50 2024-01-27T02:35:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.788+0000 7f14921f1700 1 -- 172.21.15.7:0/1713728966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148c15c440 con 0x7f148c157f50 2024-01-27T02:35:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.789+0000 7f14921f1700 1 -- 172.21.15.7:0/1713728966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f148c04c710 con 0x7f148c157f50 2024-01-27T02:35:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.790+0000 7f14737fe700 1 -- 172.21.15.7:0/1713728966 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f148401c400 con 0x7f148c157f50 2024-01-27T02:35:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.790+0000 7f14737fe700 1 --2- 172.21.15.7:0/1713728966 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1478066f30 0x7f14780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.790+0000 7f14737fe700 1 -- 172.21.15.7:0/1713728966 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1484086480 con 0x7f148c157f50 2024-01-27T02:35:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.790+0000 7f14737fe700 1 -- 172.21.15.7:0/1713728966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=69}) v3 -- 0x7f147806ea50 con 0x7f148c157f50 2024-01-27T02:35:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.790+0000 7f148b7fe700 1 --2- 172.21.15.7:0/1713728966 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1478066f30 0x7f14780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.791+0000 7f148b7fe700 1 --2- 172.21.15.7:0/1713728966 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1478066f30 0x7f14780693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f148c0c7a80 tx=0x7f147c007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.795+0000 7f14737fe700 1 -- 172.21.15.7:0/1713728966 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14840536f0 con 0x7f148c157f50 2024-01-27T02:35:47.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.998+0000 7f14921f1700 1 -- 172.21.15.7:0/1713728966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f148c15a290 con 0x7f148c157f50 2024-01-27T02:35:47.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:47.999+0000 7f14737fe700 1 -- 172.21.15.7:0/1713728966 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+15360 (secure 0 0 0) 0x7f148402b030 con 0x7f148c157f50 2024-01-27T02:35:48.000 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:35:48.000 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":68,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:34:34.902046+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:34:12.775862+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-27T02:34:13.445542+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:35:48.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.001+0000 7f14921f1700 1 -- 172.21.15.7:0/1713728966 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1478066f30 msgr2=0x7f14780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:48.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.001+0000 7f14921f1700 1 --2- 172.21.15.7:0/1713728966 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1478066f30 0x7f14780693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f148c0c7a80 tx=0x7f147c007d50 comp rx=0 tx=0).stop 2024-01-27T02:35:48.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.001+0000 7f14921f1700 1 -- 172.21.15.7:0/1713728966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f148c157f50 msgr2=0x7f148c15b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:48.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.001+0000 7f14921f1700 1 --2- 172.21.15.7:0/1713728966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f148c157f50 0x7f148c15b890 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1484009460 tx=0x7f1484019000 comp rx=0 tx=0).stop 2024-01-27T02:35:48.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.001+0000 7f14921f1700 1 -- 172.21.15.7:0/1713728966 shutdown_connections 2024-01-27T02:35:48.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.001+0000 7f14921f1700 1 --2- 172.21.15.7:0/1713728966 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1478066f30 0x7f14780693e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.001+0000 7f14921f1700 1 --2- 172.21.15.7:0/1713728966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f148c157f50 0x7f148c15b890 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.001+0000 7f14921f1700 1 --2- 172.21.15.7:0/1713728966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f148c06ea40 0x7f148c14f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.001+0000 7f14921f1700 1 --2- 172.21.15.7:0/1713728966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f148c151570 0x7f148c15b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.001+0000 7f14921f1700 1 -- 172.21.15.7:0/1713728966 >> 172.21.15.7:0/1713728966 conn(0x7f148c0cdc30 msgr2=0x7f148c158750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:48.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.001+0000 7f14921f1700 1 -- 172.21.15.7:0/1713728966 shutdown_connections 2024-01-27T02:35:48.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.001+0000 7f14921f1700 1 -- 172.21.15.7:0/1713728966 wait complete. 2024-01-27T02:35:48.016 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-27T02:35:48.016 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:35:48.019 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-27T02:35:48.019+0000 7fb5c783f700 -1 log_channel(cluster) log [ERR] : Health check update: 2 full osd(s) (OSD_FULL) 2024-01-27T02:35:48.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.148+0000 7f89ca8d8700 1 Processor -- start 2024-01-27T02:35:48.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.148+0000 7f89ca8d8700 1 -- start start 2024-01-27T02:35:48.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.149+0000 7f89ca8d8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89c40c9bb0 0x7f89c4157f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:48.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.149+0000 7f89ca8d8700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f89c406e330 0x7f89c406e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:48.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.149+0000 7f89ca8d8700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89c406c880 con 0x7f89c40c9f80 2024-01-27T02:35:48.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.149+0000 7f89ca8d8700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c406cd10 con 0x7f89c406e330 2024-01-27T02:35:48.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.149+0000 7f89ca8d8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c406ecb0 con 0x7f89c40c9bb0 2024-01-27T02:35:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.150+0000 7f89c8e75700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f89c406e330 0x7f89c406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.150+0000 7f89c3fff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89c40c9bb0 0x7f89c4157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.150+0000 7f89c37fe700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f89c40c9f80 0x7f89c406b400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:51206/0 (socket says 172.21.15.7:51206) 2024-01-27T02:35:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.150+0000 7f89c37fe700 1 -- 172.21.15.7:0/3192742528 learned_addr learned my addr 172.21.15.7:0/3192742528 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:35:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.150+0000 7f89c8e75700 1 -- 172.21.15.7:0/3192742528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89c40c9bb0 msgr2=0x7f89c4157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.150+0000 7f89c8e75700 1 --2- 172.21.15.7:0/3192742528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89c40c9bb0 0x7f89c4157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.150+0000 7f89c8e75700 1 -- 172.21.15.7:0/3192742528 >> v1:172.21.15.188:6789/0 conn(0x7f89c40c9f80 legacy=0x7f89c406b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.150+0000 7f89c8e75700 1 -- 172.21.15.7:0/3192742528 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c406bc70 con 0x7f89c406e330 2024-01-27T02:35:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.150+0000 7f89c8e75700 1 --2- 172.21.15.7:0/3192742528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89c406e330 0x7f89c406e770 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009e60 tx=0x7f89bc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=653e4cf5a5658f8f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.150+0000 7f89c2ffd700 1 -- 172.21.15.7:0/3192742528 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc00e070 con 0x7f89c406e330 2024-01-27T02:35:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.151+0000 7f89c2ffd700 1 -- 172.21.15.7:0/3192742528 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89bc00ba60 con 0x7f89c406e330 2024-01-27T02:35:48.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.151+0000 7f89c2ffd700 1 -- 172.21.15.7:0/3192742528 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc00be20 con 0x7f89c406e330 2024-01-27T02:35:48.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.151+0000 7f89ca8d8700 1 -- 172.21.15.7:0/3192742528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89c406e330 msgr2=0x7f89c406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:48.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.151+0000 7f89ca8d8700 1 --2- 172.21.15.7:0/3192742528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89c406e330 0x7f89c406e770 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009e60 tx=0x7f89bc009780 comp rx=0 tx=0).stop 2024-01-27T02:35:48.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.151+0000 7f89ca8d8700 1 -- 172.21.15.7:0/3192742528 shutdown_connections 2024-01-27T02:35:48.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.151+0000 7f89ca8d8700 1 --2- 172.21.15.7:0/3192742528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89c406e330 0x7f89c406e770 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.151+0000 7f89ca8d8700 1 --2- 172.21.15.7:0/3192742528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89c40c9bb0 0x7f89c4157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.151+0000 7f89ca8d8700 1 -- 172.21.15.7:0/3192742528 >> 172.21.15.7:0/3192742528 conn(0x7f89c40cdc30 msgr2=0x7f89c40ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:48.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.151+0000 7f89ca8d8700 1 -- 172.21.15.7:0/3192742528 shutdown_connections 2024-01-27T02:35:48.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.151+0000 7f89ca8d8700 1 -- 172.21.15.7:0/3192742528 wait complete. 2024-01-27T02:35:48.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.152+0000 7f89ca8d8700 1 Processor -- start 2024-01-27T02:35:48.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.153+0000 7f89ca8d8700 1 -- start start 2024-01-27T02:35:48.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.153+0000 7f89ca8d8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89c406e330 0x7f89c40c7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:48.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.153+0000 7f89ca8d8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89c40c9bb0 0x7f89c40c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:48.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.153+0000 7f89ca8d8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89c40c9f80 0x7f89c40b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:48.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.153+0000 7f89ca8d8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c41691e0 con 0x7f89c406e330 2024-01-27T02:35:48.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.153+0000 7f89ca8d8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c415bcb0 con 0x7f89c40c9bb0 2024-01-27T02:35:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.153+0000 7f89ca8d8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c415bf90 con 0x7f89c40c9f80 2024-01-27T02:35:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.154+0000 7f89c3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89c406e330 0x7f89c40c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.154+0000 7f89c3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89c406e330 0x7f89c40c7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44116/0 (socket says 172.21.15.7:44116) 2024-01-27T02:35:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.154+0000 7f89c3fff700 1 -- 172.21.15.7:0/2718687764 learned_addr learned my addr 172.21.15.7:0/2718687764 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.154+0000 7f89c37fe700 1 --2- 172.21.15.7:0/2718687764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89c40c9bb0 0x7f89c40c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.154+0000 7f89c8e75700 1 --2- 172.21.15.7:0/2718687764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89c40c9f80 0x7f89c40b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.154+0000 7f89c3fff700 1 -- 172.21.15.7:0/2718687764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89c40c9f80 msgr2=0x7f89c40b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.154+0000 7f89c3fff700 1 --2- 172.21.15.7:0/2718687764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89c40c9f80 0x7f89c40b2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.154+0000 7f89c3fff700 1 -- 172.21.15.7:0/2718687764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89c40c9bb0 msgr2=0x7f89c40c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.154+0000 7f89c3fff700 1 --2- 172.21.15.7:0/2718687764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89c40c9bb0 0x7f89c40c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.154+0000 7f89c3fff700 1 -- 172.21.15.7:0/2718687764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89bc009a20 con 0x7f89c406e330 2024-01-27T02:35:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.154+0000 7f89c3fff700 1 --2- 172.21.15.7:0/2718687764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89c406e330 0x7f89c40c7730 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f89b400ec80 tx=0x7f89b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.154+0000 7f89abfff700 1 -- 172.21.15.7:0/2718687764 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b400c850 con 0x7f89c406e330 2024-01-27T02:35:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.154+0000 7f89abfff700 1 -- 172.21.15.7:0/2718687764 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b400c9b0 con 0x7f89c406e330 2024-01-27T02:35:48.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.154+0000 7f89abfff700 1 -- 172.21.15.7:0/2718687764 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b400cd50 con 0x7f89c406e330 2024-01-27T02:35:48.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.154+0000 7f89ca8d8700 1 -- 172.21.15.7:0/2718687764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c415c270 con 0x7f89c406e330 2024-01-27T02:35:48.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.154+0000 7f89ca8d8700 1 -- 172.21.15.7:0/2718687764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c40c7c70 con 0x7f89c406e330 2024-01-27T02:35:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.155+0000 7f89ca8d8700 1 -- 172.21.15.7:0/2718687764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c404c710 con 0x7f89c406e330 2024-01-27T02:35:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.156+0000 7f89abfff700 1 -- 172.21.15.7:0/2718687764 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f89b4013680 con 0x7f89c406e330 2024-01-27T02:35:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.156+0000 7f89abfff700 1 --2- 172.21.15.7:0/2718687764 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f89b0066f30 0x7f89b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.156+0000 7f89abfff700 1 -- 172.21.15.7:0/2718687764 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f89b40847b0 con 0x7f89c406e330 2024-01-27T02:35:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.156+0000 7f89abfff700 1 -- 172.21.15.7:0/2718687764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f89b006ea50 con 0x7f89c406e330 2024-01-27T02:35:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.157+0000 7f89c37fe700 1 --2- 172.21.15.7:0/2718687764 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f89b0066f30 0x7f89b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.157+0000 7f89c37fe700 1 --2- 172.21.15.7:0/2718687764 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f89b0066f30 0x7f89b00693e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f89c414fe50 tx=0x7f89b8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:48.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.161+0000 7f89abfff700 1 -- 172.21.15.7:0/2718687764 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89b4017060 con 0x7f89c406e330 2024-01-27T02:35:48.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.409+0000 7f89ca8d8700 1 -- 172.21.15.7:0/2718687764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89c415c8c0 con 0x7f89c406e330 2024-01-27T02:35:48.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.410+0000 7f89abfff700 1 -- 172.21.15.7:0/2718687764 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f89b4051a20 con 0x7f89c406e330 2024-01-27T02:35:48.410 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:35:48.410 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5114,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/2839704447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":2839704447},{"type":"v1","addr":"172.21.15.188:6837","nonce":2839704447}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5117,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3964291855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3964291855},{"type":"v1","addr":"172.21.15.7:6834","nonce":3964291855}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/3332325272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3332325272},{"type":"v1","addr":"172.21.15.188:6836","nonce":3332325272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:34:14.471429+0000","modified":"2024-01-27T02:34:18.068858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1399055016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1399055016},{"type":"v1","addr":"172.21.15.7:6835","nonce":1399055016}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2024-01-27T02:35:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.412+0000 7f89ca8d8700 1 -- 172.21.15.7:0/2718687764 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f89b0066f30 msgr2=0x7f89b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:48.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.412+0000 7f89ca8d8700 1 --2- 172.21.15.7:0/2718687764 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f89b0066f30 0x7f89b00693e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f89c414fe50 tx=0x7f89b8009040 comp rx=0 tx=0).stop 2024-01-27T02:35:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.412+0000 7f89ca8d8700 1 -- 172.21.15.7:0/2718687764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89c406e330 msgr2=0x7f89c40c7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.412+0000 7f89ca8d8700 1 --2- 172.21.15.7:0/2718687764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89c406e330 0x7f89c40c7730 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f89b400ec80 tx=0x7f89b40091a0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.412+0000 7f89ca8d8700 1 -- 172.21.15.7:0/2718687764 shutdown_connections 2024-01-27T02:35:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.412+0000 7f89ca8d8700 1 --2- 172.21.15.7:0/2718687764 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f89b0066f30 0x7f89b00693e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.412+0000 7f89ca8d8700 1 --2- 172.21.15.7:0/2718687764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89c40c9f80 0x7f89c40b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.412+0000 7f89ca8d8700 1 --2- 172.21.15.7:0/2718687764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89c406e330 0x7f89c40c7730 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.412+0000 7f89ca8d8700 1 --2- 172.21.15.7:0/2718687764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89c40c9bb0 0x7f89c40c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.412+0000 7f89ca8d8700 1 -- 172.21.15.7:0/2718687764 >> 172.21.15.7:0/2718687764 conn(0x7f89c40cdc30 msgr2=0x7f89c4159480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.412+0000 7f89ca8d8700 1 -- 172.21.15.7:0/2718687764 shutdown_connections 2024-01-27T02:35:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.412+0000 7f89ca8d8700 1 -- 172.21.15.7:0/2718687764 wait complete. 2024-01-27T02:35:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2024-01-27T02:35:48.426 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:35:48.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.599+0000 7fdf6a54d700 1 Processor -- start 2024-01-27T02:35:48.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.599+0000 7fdf6a54d700 1 -- start start 2024-01-27T02:35:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.600+0000 7fdf6a54d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf6415a100 0x7fdf640cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.600+0000 7fdf6a54d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdf640c4cc0 0x7fdf64163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.600+0000 7fdf6a54d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf640ce5e0 con 0x7fdf640cb5b0 2024-01-27T02:35:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.600+0000 7fdf6a54d700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf640cd9d0 con 0x7fdf640c4cc0 2024-01-27T02:35:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.600+0000 7fdf6a54d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf640cdcb0 con 0x7fdf6415a100 2024-01-27T02:35:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.600+0000 7fdf637fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdf640cb5b0 0x7fdf640c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56050/0 (socket says 172.21.15.7:56050) 2024-01-27T02:35:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.600+0000 7fdf637fe700 1 -- 172.21.15.7:0/640550892 learned_addr learned my addr 172.21.15.7:0/640550892 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:35:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.600+0000 7fdf68aea700 1 --2- 172.21.15.7:0/640550892 >> v2:172.21.15.188:3300/0 conn(0x7fdf640c4cc0 0x7fdf64163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.600+0000 7fdf63fff700 1 --2- 172.21.15.7:0/640550892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf6415a100 0x7fdf640cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.600+0000 7fdf62ffd700 1 -- 172.21.15.7:0/640550892 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3517318486 0 0) 0x7fdf640ce5e0 con 0x7fdf640cb5b0 2024-01-27T02:35:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.600+0000 7fdf62ffd700 1 -- 172.21.15.7:0/640550892 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf4c003570 con 0x7fdf640cb5b0 2024-01-27T02:35:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.600+0000 7fdf62ffd700 1 -- 172.21.15.7:0/640550892 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1987437124 0 0) 0x7fdf4c003570 con 0x7fdf640cb5b0 2024-01-27T02:35:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.600+0000 7fdf62ffd700 1 -- 172.21.15.7:0/640550892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf6415a100 msgr2=0x7fdf640cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.600+0000 7fdf62ffd700 1 --2- 172.21.15.7:0/640550892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf6415a100 0x7fdf640cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.600+0000 7fdf62ffd700 1 -- 172.21.15.7:0/640550892 >> v2:172.21.15.188:3300/0 conn(0x7fdf640c4cc0 msgr2=0x7fdf64163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.600+0000 7fdf62ffd700 1 --2- 172.21.15.7:0/640550892 >> v2:172.21.15.188:3300/0 conn(0x7fdf640c4cc0 0x7fdf64163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.600+0000 7fdf62ffd700 1 -- 172.21.15.7:0/640550892 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf640cdf90 con 0x7fdf640cb5b0 2024-01-27T02:35:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.600+0000 7fdf62ffd700 1 -- 172.21.15.7:0/640550892 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7fdf58001470 con 0x7fdf640cb5b0 2024-01-27T02:35:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.600+0000 7fdf62ffd700 1 -- 172.21.15.7:0/640550892 >> v1:172.21.15.7:6789/0 conn(0x7fdf640cb5b0 legacy=0x7fdf640c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.601+0000 7fdf62ffd700 1 --2- 172.21.15.7:0/640550892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf6415a100 0x7fdf4c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.601+0000 7fdf62ffd700 1 --2- 172.21.15.7:0/640550892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf4c003fe0 0x7fdf4c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.601+0000 7fdf62ffd700 1 --2- 172.21.15.7:0/640550892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf4c004d50 0x7fdf4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.601+0000 7fdf62ffd700 1 -- 172.21.15.7:0/640550892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf640cd9d0 con 0x7fdf4c004d50 2024-01-27T02:35:48.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.601+0000 7fdf62ffd700 1 -- 172.21.15.7:0/640550892 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf640cdcb0 con 0x7fdf4c003fe0 2024-01-27T02:35:48.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.601+0000 7fdf62ffd700 1 -- 172.21.15.7:0/640550892 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c00ba90 con 0x7fdf6415a100 2024-01-27T02:35:48.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.601+0000 7fdf68aea700 1 --2- 172.21.15.7:0/640550892 >> v2:172.21.15.188:3300/0 conn(0x7fdf640c4cc0 0x7fdf64163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:35:48.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.601+0000 7fdf63fff700 1 --2- 172.21.15.7:0/640550892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf6415a100 0x7fdf4c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:48.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.601+0000 7fdf637fe700 1 --2- 172.21.15.7:0/640550892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf4c003fe0 0x7fdf4c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:48.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.601+0000 7fdf68aea700 1 --2- 172.21.15.7:0/640550892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf4c004d50 0x7fdf4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:48.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.601+0000 7fdf68aea700 1 -- 172.21.15.7:0/640550892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf6415a100 msgr2=0x7fdf4c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:48.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.601+0000 7fdf68aea700 1 --2- 172.21.15.7:0/640550892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf6415a100 0x7fdf4c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.601+0000 7fdf68aea700 1 -- 172.21.15.7:0/640550892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf4c003fe0 msgr2=0x7fdf4c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:48.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.601+0000 7fdf68aea700 1 --2- 172.21.15.7:0/640550892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf4c003fe0 0x7fdf4c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.601+0000 7fdf68aea700 1 -- 172.21.15.7:0/640550892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4c003570 con 0x7fdf4c004d50 2024-01-27T02:35:48.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.601+0000 7fdf68aea700 1 --2- 172.21.15.7:0/640550892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf4c004d50 0x7fdf4c0051c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdf640c50c0 tx=0x7fdf5c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c159e588e47489c6 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:48.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.601+0000 7fdf62ffd700 1 -- 172.21.15.7:0/640550892 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c00ebe0 con 0x7fdf4c004d50 2024-01-27T02:35:48.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.601+0000 7fdf62ffd700 1 -- 172.21.15.7:0/640550892 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5c00f040 con 0x7fdf4c004d50 2024-01-27T02:35:48.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.601+0000 7fdf62ffd700 1 -- 172.21.15.7:0/640550892 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c014480 con 0x7fdf4c004d50 2024-01-27T02:35:48.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.602+0000 7fdf6a54d700 1 -- 172.21.15.7:0/640550892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf4c004d50 msgr2=0x7fdf4c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:48.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.602+0000 7fdf6a54d700 1 --2- 172.21.15.7:0/640550892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf4c004d50 0x7fdf4c0051c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdf640c50c0 tx=0x7fdf5c00b040 comp rx=0 tx=0).stop 2024-01-27T02:35:48.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.602+0000 7fdf68aea700 1 -- 172.21.15.7:0/640550892 reap_dead start 2024-01-27T02:35:48.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.602+0000 7fdf6a54d700 1 -- 172.21.15.7:0/640550892 shutdown_connections 2024-01-27T02:35:48.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.602+0000 7fdf6a54d700 1 -- 172.21.15.7:0/640550892 >> 172.21.15.7:0/640550892 conn(0x7fdf64098020 msgr2=0x7fdf640711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:48.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.602+0000 7fdf6a54d700 1 -- 172.21.15.7:0/640550892 shutdown_connections 2024-01-27T02:35:48.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.602+0000 7fdf6a54d700 1 -- 172.21.15.7:0/640550892 wait complete. 2024-01-27T02:35:48.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.603+0000 7fdf6a54d700 1 Processor -- start 2024-01-27T02:35:48.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.604+0000 7fdf6a54d700 1 -- start start 2024-01-27T02:35:48.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.604+0000 7fdf6a54d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf64158fb0 0x7fdf6415a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:48.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.604+0000 7fdf6a54d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf640c62a0 0x7fdf6415aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:48.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.604+0000 7fdf6a54d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6415e080 0x7fdf6415e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:48.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.604+0000 7fdf6a54d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf640c7340 con 0x7fdf6415e080 2024-01-27T02:35:48.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.604+0000 7fdf6a54d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf64159380 con 0x7fdf640c62a0 2024-01-27T02:35:48.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.604+0000 7fdf6a54d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf640c6730 con 0x7fdf64158fb0 2024-01-27T02:35:48.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.604+0000 7fdf68aea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6415e080 0x7fdf6415e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:48.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.604+0000 7fdf68aea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6415e080 0x7fdf6415e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44142/0 (socket says 172.21.15.7:44142) 2024-01-27T02:35:48.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.604+0000 7fdf68aea700 1 -- 172.21.15.7:0/1809767532 learned_addr learned my addr 172.21.15.7:0/1809767532 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:48.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.605+0000 7fdf637fe700 1 --2- 172.21.15.7:0/1809767532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf640c62a0 0x7fdf6415aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:48.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.605+0000 7fdf63fff700 1 --2- 172.21.15.7:0/1809767532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf64158fb0 0x7fdf6415a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:48.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.605+0000 7fdf68aea700 1 -- 172.21.15.7:0/1809767532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf64158fb0 msgr2=0x7fdf6415a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:48.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.605+0000 7fdf68aea700 1 --2- 172.21.15.7:0/1809767532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf64158fb0 0x7fdf6415a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.605+0000 7fdf68aea700 1 -- 172.21.15.7:0/1809767532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf640c62a0 msgr2=0x7fdf6415aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:48.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.605+0000 7fdf68aea700 1 --2- 172.21.15.7:0/1809767532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf640c62a0 0x7fdf6415aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.605+0000 7fdf68aea700 1 -- 172.21.15.7:0/1809767532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf640c6a10 con 0x7fdf6415e080 2024-01-27T02:35:48.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.605+0000 7fdf63fff700 1 --2- 172.21.15.7:0/1809767532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf64158fb0 0x7fdf6415a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:35:48.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.605+0000 7fdf68aea700 1 --2- 172.21.15.7:0/1809767532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6415e080 0x7fdf6415e590 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c002230 tx=0x7fdf5c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:48.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.605+0000 7fdf4bfff700 1 -- 172.21.15.7:0/1809767532 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c014480 con 0x7fdf6415e080 2024-01-27T02:35:48.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.605+0000 7fdf4bfff700 1 -- 172.21.15.7:0/1809767532 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5c01e440 con 0x7fdf6415e080 2024-01-27T02:35:48.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.605+0000 7fdf6a54d700 1 -- 172.21.15.7:0/1809767532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf640c6cf0 con 0x7fdf6415e080 2024-01-27T02:35:48.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.605+0000 7fdf4bfff700 1 -- 172.21.15.7:0/1809767532 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c01e800 con 0x7fdf6415e080 2024-01-27T02:35:48.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.605+0000 7fdf6a54d700 1 -- 172.21.15.7:0/1809767532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf640c77d0 con 0x7fdf6415e080 2024-01-27T02:35:48.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.606+0000 7fdf6a54d700 1 -- 172.21.15.7:0/1809767532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf6404c710 con 0x7fdf6415e080 2024-01-27T02:35:48.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.607+0000 7fdf4bfff700 1 -- 172.21.15.7:0/1809767532 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fdf5c01eb70 con 0x7fdf6415e080 2024-01-27T02:35:48.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.607+0000 7fdf4bfff700 1 --2- 172.21.15.7:0/1809767532 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdf50066f30 0x7fdf500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:48.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.607+0000 7fdf4bfff700 1 -- 172.21.15.7:0/1809767532 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdf5c086750 con 0x7fdf6415e080 2024-01-27T02:35:48.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.607+0000 7fdf63fff700 1 --2- 172.21.15.7:0/1809767532 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdf50066f30 0x7fdf500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:48.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.608+0000 7fdf4bfff700 1 -- 172.21.15.7:0/1809767532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7fdf5006ea50 con 0x7fdf6415e080 2024-01-27T02:35:48.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.608+0000 7fdf63fff700 1 --2- 172.21.15.7:0/1809767532 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdf50066f30 0x7fdf500693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdf54006f00 tx=0x7fdf54008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:48.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.612+0000 7fdf4bfff700 1 -- 172.21.15.7:0/1809767532 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf5c002020 con 0x7fdf6415e080 2024-01-27T02:35:48.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.860+0000 7fdf6a54d700 1 -- 172.21.15.7:0/1809767532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf640ce270 con 0x7fdf6415e080 2024-01-27T02:35:48.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.861+0000 7fdf4bfff700 1 -- 172.21.15.7:0/1809767532 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4597 (secure 0 0 0) 0x7fdf5c02d020 con 0x7fdf6415e080 2024-01-27T02:35:48.861 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:35:48.861 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5114,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/2839704447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":2839704447},{"type":"v1","addr":"172.21.15.188:6837","nonce":2839704447}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5117,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3964291855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3964291855},{"type":"v1","addr":"172.21.15.7:6834","nonce":3964291855}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/3332325272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3332325272},{"type":"v1","addr":"172.21.15.188:6836","nonce":3332325272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:34:14.471429+0000","modified":"2024-01-27T02:34:18.068858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1399055016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1399055016},{"type":"v1","addr":"172.21.15.7:6835","nonce":1399055016}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2024-01-27T02:35:48.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.863+0000 7fdf6a54d700 1 -- 172.21.15.7:0/1809767532 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdf50066f30 msgr2=0x7fdf500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:48.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.863+0000 7fdf6a54d700 1 --2- 172.21.15.7:0/1809767532 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdf50066f30 0x7fdf500693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdf54006f00 tx=0x7fdf54008040 comp rx=0 tx=0).stop 2024-01-27T02:35:48.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.863+0000 7fdf6a54d700 1 -- 172.21.15.7:0/1809767532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6415e080 msgr2=0x7fdf6415e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:48.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.863+0000 7fdf6a54d700 1 --2- 172.21.15.7:0/1809767532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6415e080 0x7fdf6415e590 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c002230 tx=0x7fdf5c019000 comp rx=0 tx=0).stop 2024-01-27T02:35:48.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.863+0000 7fdf6a54d700 1 -- 172.21.15.7:0/1809767532 shutdown_connections 2024-01-27T02:35:48.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.863+0000 7fdf6a54d700 1 --2- 172.21.15.7:0/1809767532 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdf50066f30 0x7fdf500693e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.863+0000 7fdf6a54d700 1 --2- 172.21.15.7:0/1809767532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf6415e080 0x7fdf6415e590 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.863+0000 7fdf6a54d700 1 --2- 172.21.15.7:0/1809767532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf64158fb0 0x7fdf6415a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.863+0000 7fdf6a54d700 1 --2- 172.21.15.7:0/1809767532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf640c62a0 0x7fdf6415aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:48.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.863+0000 7fdf6a54d700 1 -- 172.21.15.7:0/1809767532 >> 172.21.15.7:0/1809767532 conn(0x7fdf64098020 msgr2=0x7fdf6415bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:48.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.863+0000 7fdf6a54d700 1 -- 172.21.15.7:0/1809767532 shutdown_connections 2024-01-27T02:35:48.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:48.863+0000 7fdf6a54d700 1 -- 172.21.15.7:0/1809767532 wait complete. 2024-01-27T02:35:48.865 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2024-01-27T02:35:48.877 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:35:49.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.008+0000 7f4d2852f700 1 Processor -- start 2024-01-27T02:35:49.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.008+0000 7f4d2852f700 1 -- start start 2024-01-27T02:35:49.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.009+0000 7f4d2852f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4d200a7c10 0x7f4d200ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:49.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.009+0000 7f4d2852f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4d200a7fe0 0x7f4d200b0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:49.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.009+0000 7f4d2852f700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d2015a880 con 0x7f4d200ae880 2024-01-27T02:35:49.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.009+0000 7f4d2852f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d200afde0 con 0x7f4d200a7fe0 2024-01-27T02:35:49.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.009+0000 7f4d2852f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d200af950 con 0x7f4d200a7c10 2024-01-27T02:35:49.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.009+0000 7f4d26acc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4d200a7fe0 0x7f4d200b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:49.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.009+0000 7f4d26acc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4d200a7fe0 0x7f4d200b0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44150/0 (socket says 172.21.15.7:44150) 2024-01-27T02:35:49.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.009+0000 7f4d26acc700 1 -- 172.21.15.7:0/1786116039 learned_addr learned my addr 172.21.15.7:0/1786116039 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:49.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.010+0000 7f4d262cb700 1 --2- 172.21.15.7:0/1786116039 >> v2:172.21.15.188:3300/0 conn(0x7f4d200a7c10 0x7f4d200ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:49.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.010+0000 7f4d26acc700 1 -- 172.21.15.7:0/1786116039 >> v2:172.21.15.188:3300/0 conn(0x7f4d200a7c10 msgr2=0x7f4d200ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:49.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.010+0000 7f4d26acc700 1 --2- 172.21.15.7:0/1786116039 >> v2:172.21.15.188:3300/0 conn(0x7f4d200a7c10 0x7f4d200ae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.010+0000 7f4d26acc700 1 -- 172.21.15.7:0/1786116039 >> v1:172.21.15.188:6789/0 conn(0x7f4d200ae880 legacy=0x7f4d2006f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:49.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.010+0000 7f4d26acc700 1 -- 172.21.15.7:0/1786116039 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d200aed40 con 0x7f4d200a7fe0 2024-01-27T02:35:49.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.010+0000 7f4d262cb700 1 --2- 172.21.15.7:0/1786116039 >> v2:172.21.15.188:3300/0 conn(0x7f4d200a7c10 0x7f4d200ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:35:49.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.010+0000 7f4d26acc700 1 --2- 172.21.15.7:0/1786116039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d200a7fe0 0x7f4d200b0180 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4d200ac950 tx=0x7f4d18009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e082e3b1605d1c97 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:49.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.010+0000 7f4d252c9700 1 -- 172.21.15.7:0/1786116039 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1800e070 con 0x7f4d200a7fe0 2024-01-27T02:35:49.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.010+0000 7f4d252c9700 1 -- 172.21.15.7:0/1786116039 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1800ba50 con 0x7f4d200a7fe0 2024-01-27T02:35:49.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.010+0000 7f4d252c9700 1 -- 172.21.15.7:0/1786116039 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d180092e0 con 0x7f4d200a7fe0 2024-01-27T02:35:49.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.011+0000 7f4d2852f700 1 -- 172.21.15.7:0/1786116039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d200a7fe0 msgr2=0x7f4d200b0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:49.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.011+0000 7f4d2852f700 1 --2- 172.21.15.7:0/1786116039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d200a7fe0 0x7f4d200b0180 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4d200ac950 tx=0x7f4d18009840 comp rx=0 tx=0).stop 2024-01-27T02:35:49.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.011+0000 7f4d2852f700 1 -- 172.21.15.7:0/1786116039 shutdown_connections 2024-01-27T02:35:49.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.011+0000 7f4d2852f700 1 --2- 172.21.15.7:0/1786116039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d200a7fe0 0x7f4d200b0180 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.011+0000 7f4d2852f700 1 --2- 172.21.15.7:0/1786116039 >> v2:172.21.15.188:3300/0 conn(0x7f4d200a7c10 0x7f4d200ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.011+0000 7f4d2852f700 1 -- 172.21.15.7:0/1786116039 >> 172.21.15.7:0/1786116039 conn(0x7f4d200679b0 msgr2=0x7f4d20067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:49.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.011+0000 7f4d2852f700 1 -- 172.21.15.7:0/1786116039 shutdown_connections 2024-01-27T02:35:49.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.011+0000 7f4d2852f700 1 -- 172.21.15.7:0/1786116039 wait complete. 2024-01-27T02:35:49.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.012+0000 7f4d2852f700 1 Processor -- start 2024-01-27T02:35:49.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.012+0000 7f4d2852f700 1 -- start start 2024-01-27T02:35:49.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.013+0000 7f4d2852f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d200a7c10 0x7f4d20157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:49.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.013+0000 7f4d2852f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d200a7fe0 0x7f4d2015cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:49.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.013+0000 7f4d2852f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d200ae880 0x7f4d20152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:49.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.013+0000 7f4d2852f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d20156c40 con 0x7f4d200ae880 2024-01-27T02:35:49.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.013+0000 7f4d2852f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d20156f20 con 0x7f4d200a7c10 2024-01-27T02:35:49.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.013+0000 7f4d2852f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d20157200 con 0x7f4d200a7fe0 2024-01-27T02:35:49.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.013+0000 7f4d26acc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d200ae880 0x7f4d20152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:49.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.013+0000 7f4d26acc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d200ae880 0x7f4d20152940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44166/0 (socket says 172.21.15.7:44166) 2024-01-27T02:35:49.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.013+0000 7f4d26acc700 1 -- 172.21.15.7:0/1797603325 learned_addr learned my addr 172.21.15.7:0/1797603325 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:49.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.013+0000 7f4d262cb700 1 --2- 172.21.15.7:0/1797603325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d200a7c10 0x7f4d20157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:49.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.013+0000 7f4d25aca700 1 --2- 172.21.15.7:0/1797603325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d200a7fe0 0x7f4d2015cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:49.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.013+0000 7f4d26acc700 1 -- 172.21.15.7:0/1797603325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d200a7fe0 msgr2=0x7f4d2015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:49.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.013+0000 7f4d26acc700 1 --2- 172.21.15.7:0/1797603325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d200a7fe0 0x7f4d2015cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.013+0000 7f4d26acc700 1 -- 172.21.15.7:0/1797603325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d200a7c10 msgr2=0x7f4d20157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:49.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.013+0000 7f4d26acc700 1 --2- 172.21.15.7:0/1797603325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d200a7c10 0x7f4d20157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.013+0000 7f4d26acc700 1 -- 172.21.15.7:0/1797603325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d18009ae0 con 0x7f4d200ae880 2024-01-27T02:35:49.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.014+0000 7f4d26acc700 1 --2- 172.21.15.7:0/1797603325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d200ae880 0x7f4d20152940 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4d18012000 tx=0x7f4d18018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:49.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.014+0000 7f4d167fc700 1 -- 172.21.15.7:0/1797603325 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1800be60 con 0x7f4d200ae880 2024-01-27T02:35:49.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.014+0000 7f4d167fc700 1 -- 172.21.15.7:0/1797603325 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d18008ce0 con 0x7f4d200ae880 2024-01-27T02:35:49.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.014+0000 7f4d2852f700 1 -- 172.21.15.7:0/1797603325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d201574e0 con 0x7f4d200ae880 2024-01-27T02:35:49.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.014+0000 7f4d167fc700 1 -- 172.21.15.7:0/1797603325 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d180266b0 con 0x7f4d200ae880 2024-01-27T02:35:49.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.014+0000 7f4d2852f700 1 -- 172.21.15.7:0/1797603325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d20158220 con 0x7f4d200ae880 2024-01-27T02:35:49.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.015+0000 7f4d2852f700 1 -- 172.21.15.7:0/1797603325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d2004c710 con 0x7f4d200ae880 2024-01-27T02:35:49.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.015+0000 7f4d167fc700 1 -- 172.21.15.7:0/1797603325 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4d18003040 con 0x7f4d200ae880 2024-01-27T02:35:49.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.016+0000 7f4d167fc700 1 --2- 172.21.15.7:0/1797603325 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4d04066f30 0x7f4d040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:49.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.016+0000 7f4d167fc700 1 -- 172.21.15.7:0/1797603325 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4d180863d0 con 0x7f4d200ae880 2024-01-27T02:35:49.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.016+0000 7f4d167fc700 1 -- 172.21.15.7:0/1797603325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f4d0406ea50 con 0x7f4d200ae880 2024-01-27T02:35:49.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.016+0000 7f4d262cb700 1 --2- 172.21.15.7:0/1797603325 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4d04066f30 0x7f4d040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:49.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.016+0000 7f4d262cb700 1 --2- 172.21.15.7:0/1797603325 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4d04066f30 0x7f4d040693e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c006f00 tx=0x7f4d1c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:49.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.020+0000 7f4d167fc700 1 -- 172.21.15.7:0/1797603325 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d180535a0 con 0x7f4d200ae880 2024-01-27T02:35:49.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.224+0000 7f4d2852f700 1 -- 172.21.15.7:0/1797603325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d200af020 con 0x7f4d200ae880 2024-01-27T02:35:49.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.225+0000 7f4d167fc700 1 -- 172.21.15.7:0/1797603325 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+15331 (secure 0 0 0) 0x7f4d18021090 con 0x7f4d200ae880 2024-01-27T02:35:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:35:49.226 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":69,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:35:48.019624+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:34:12.775862+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-27T02:34:13.445542+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:35:49.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.228+0000 7f4d2852f700 1 -- 172.21.15.7:0/1797603325 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4d04066f30 msgr2=0x7f4d040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:49.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.228+0000 7f4d2852f700 1 --2- 172.21.15.7:0/1797603325 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4d04066f30 0x7f4d040693e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c006f00 tx=0x7f4d1c008040 comp rx=0 tx=0).stop 2024-01-27T02:35:49.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.228+0000 7f4d2852f700 1 -- 172.21.15.7:0/1797603325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d200ae880 msgr2=0x7f4d20152940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:49.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.228+0000 7f4d2852f700 1 --2- 172.21.15.7:0/1797603325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d200ae880 0x7f4d20152940 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4d18012000 tx=0x7f4d18018000 comp rx=0 tx=0).stop 2024-01-27T02:35:49.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.228+0000 7f4d2852f700 1 -- 172.21.15.7:0/1797603325 shutdown_connections 2024-01-27T02:35:49.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.228+0000 7f4d2852f700 1 --2- 172.21.15.7:0/1797603325 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4d04066f30 0x7f4d040693e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.228+0000 7f4d2852f700 1 --2- 172.21.15.7:0/1797603325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d200ae880 0x7f4d20152940 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.228+0000 7f4d2852f700 1 --2- 172.21.15.7:0/1797603325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d200a7c10 0x7f4d20157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.228+0000 7f4d2852f700 1 --2- 172.21.15.7:0/1797603325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d200a7fe0 0x7f4d2015cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.228+0000 7f4d2852f700 1 -- 172.21.15.7:0/1797603325 >> 172.21.15.7:0/1797603325 conn(0x7f4d200679b0 msgr2=0x7f4d2014f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:49.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.228+0000 7f4d2852f700 1 -- 172.21.15.7:0/1797603325 shutdown_connections 2024-01-27T02:35:49.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.228+0000 7f4d2852f700 1 -- 172.21.15.7:0/1797603325 wait complete. 2024-01-27T02:35:49.242 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-27T02:35:49.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.374+0000 7f6a91023700 1 Processor -- start 2024-01-27T02:35:49.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.374+0000 7f6a91023700 1 -- start start 2024-01-27T02:35:49.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.375+0000 7f6a91023700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6a8c0b2f80 0x7f6a8c06a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:49.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.375+0000 7f6a91023700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6a8c1553e0 0x7f6a8c155860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:49.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.375+0000 7f6a91023700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a8c06a900 con 0x7f6a8c0b3350 2024-01-27T02:35:49.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.375+0000 7f6a91023700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a8c159390 con 0x7f6a8c1553e0 2024-01-27T02:35:49.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.375+0000 7f6a91023700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a8c158780 con 0x7f6a8c0b2f80 2024-01-27T02:35:49.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.375+0000 7f6a8b59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6a8c1553e0 0x7f6a8c155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:49.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.375+0000 7f6a8b59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6a8c1553e0 0x7f6a8c155860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44170/0 (socket says 172.21.15.7:44170) 2024-01-27T02:35:49.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.376+0000 7f6a8b59e700 1 -- 172.21.15.7:0/2966345211 learned_addr learned my addr 172.21.15.7:0/2966345211 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:49.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.376+0000 7f6a8ad9d700 1 --2- 172.21.15.7:0/2966345211 >> v2:172.21.15.188:3300/0 conn(0x7f6a8c0b2f80 0x7f6a8c06a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:49.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.376+0000 7f6a8b59e700 1 -- 172.21.15.7:0/2966345211 >> v2:172.21.15.188:3300/0 conn(0x7f6a8c0b2f80 msgr2=0x7f6a8c06a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:49.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.376+0000 7f6a8b59e700 1 --2- 172.21.15.7:0/2966345211 >> v2:172.21.15.188:3300/0 conn(0x7f6a8c0b2f80 0x7f6a8c06a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.376+0000 7f6a8b59e700 1 -- 172.21.15.7:0/2966345211 >> v1:172.21.15.188:6789/0 conn(0x7f6a8c0b3350 legacy=0x7f6a8c0ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:49.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.376+0000 7f6a8b59e700 1 -- 172.21.15.7:0/2966345211 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8c158a60 con 0x7f6a8c1553e0 2024-01-27T02:35:49.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.376+0000 7f6a8b59e700 1 --2- 172.21.15.7:0/2966345211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c1553e0 0x7f6a8c155860 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c06dec0 tx=0x7f6a84009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4844f2cdac2ea8a1 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:49.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.376+0000 7f6a89d9b700 1 -- 172.21.15.7:0/2966345211 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8400e070 con 0x7f6a8c1553e0 2024-01-27T02:35:49.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.376+0000 7f6a89d9b700 1 -- 172.21.15.7:0/2966345211 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8400ba10 con 0x7f6a8c1553e0 2024-01-27T02:35:49.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.376+0000 7f6a89d9b700 1 -- 172.21.15.7:0/2966345211 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a840092e0 con 0x7f6a8c1553e0 2024-01-27T02:35:49.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.377+0000 7f6a91023700 1 -- 172.21.15.7:0/2966345211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c1553e0 msgr2=0x7f6a8c155860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:49.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.377+0000 7f6a91023700 1 --2- 172.21.15.7:0/2966345211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c1553e0 0x7f6a8c155860 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c06dec0 tx=0x7f6a84009840 comp rx=0 tx=0).stop 2024-01-27T02:35:49.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.377+0000 7f6a91023700 1 -- 172.21.15.7:0/2966345211 shutdown_connections 2024-01-27T02:35:49.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.377+0000 7f6a91023700 1 --2- 172.21.15.7:0/2966345211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c1553e0 0x7f6a8c155860 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.377+0000 7f6a91023700 1 --2- 172.21.15.7:0/2966345211 >> v2:172.21.15.188:3300/0 conn(0x7f6a8c0b2f80 0x7f6a8c06a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.377+0000 7f6a91023700 1 -- 172.21.15.7:0/2966345211 >> 172.21.15.7:0/2966345211 conn(0x7f6a8c068430 msgr2=0x7f6a8c06be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:49.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.377+0000 7f6a91023700 1 -- 172.21.15.7:0/2966345211 shutdown_connections 2024-01-27T02:35:49.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.377+0000 7f6a91023700 1 -- 172.21.15.7:0/2966345211 wait complete. 2024-01-27T02:35:49.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.378+0000 7f6a91023700 1 Processor -- start 2024-01-27T02:35:49.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.378+0000 7f6a91023700 1 -- start start 2024-01-27T02:35:49.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.379+0000 7f6a91023700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a8c0b2f80 0x7f6a8c14e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:49.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.379+0000 7f6a91023700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c0b3350 0x7f6a8c15a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:49.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.379+0000 7f6a91023700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a8c1553e0 0x7f6a8c15aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:49.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.379+0000 7f6a91023700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c169a70 con 0x7f6a8c0b3350 2024-01-27T02:35:49.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.379+0000 7f6a91023700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c169140 con 0x7f6a8c1553e0 2024-01-27T02:35:49.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.379+0000 7f6a91023700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c169420 con 0x7f6a8c0b2f80 2024-01-27T02:35:49.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.379+0000 7f6a8a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c0b3350 0x7f6a8c15a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:49.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.379+0000 7f6a8a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c0b3350 0x7f6a8c15a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44182/0 (socket says 172.21.15.7:44182) 2024-01-27T02:35:49.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.379+0000 7f6a8a59c700 1 -- 172.21.15.7:0/833520403 learned_addr learned my addr 172.21.15.7:0/833520403 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:49.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.380+0000 7f6a8b59e700 1 --2- 172.21.15.7:0/833520403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a8c1553e0 0x7f6a8c15aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:49.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.380+0000 7f6a8ad9d700 1 --2- 172.21.15.7:0/833520403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a8c0b2f80 0x7f6a8c14e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:49.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.380+0000 7f6a8a59c700 1 -- 172.21.15.7:0/833520403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a8c0b2f80 msgr2=0x7f6a8c14e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:49.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.380+0000 7f6a8a59c700 1 --2- 172.21.15.7:0/833520403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a8c0b2f80 0x7f6a8c14e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.380+0000 7f6a8a59c700 1 -- 172.21.15.7:0/833520403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a8c1553e0 msgr2=0x7f6a8c15aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:49.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.380+0000 7f6a8a59c700 1 --2- 172.21.15.7:0/833520403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a8c1553e0 0x7f6a8c15aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.380+0000 7f6a8a59c700 1 -- 172.21.15.7:0/833520403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a84009ae0 con 0x7f6a8c0b3350 2024-01-27T02:35:49.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.380+0000 7f6a8b59e700 1 --2- 172.21.15.7:0/833520403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a8c1553e0 0x7f6a8c15aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:35:49.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.380+0000 7f6a8a59c700 1 --2- 172.21.15.7:0/833520403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c0b3350 0x7f6a8c15a530 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400c5d0 tx=0x7f6a74007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:49.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.380+0000 7f6a82ffd700 1 -- 172.21.15.7:0/833520403 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74011590 con 0x7f6a8c0b3350 2024-01-27T02:35:49.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.380+0000 7f6a82ffd700 1 -- 172.21.15.7:0/833520403 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a74011710 con 0x7f6a8c0b3350 2024-01-27T02:35:49.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.380+0000 7f6a82ffd700 1 -- 172.21.15.7:0/833520403 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74011ad0 con 0x7f6a8c0b3350 2024-01-27T02:35:49.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.380+0000 7f6a91023700 1 -- 172.21.15.7:0/833520403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8c169700 con 0x7f6a8c0b3350 2024-01-27T02:35:49.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.380+0000 7f6a91023700 1 -- 172.21.15.7:0/833520403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8c151200 con 0x7f6a8c0b3350 2024-01-27T02:35:49.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.381+0000 7f6a91023700 1 -- 172.21.15.7:0/833520403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a8c151420 con 0x7f6a8c0b3350 2024-01-27T02:35:49.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.382+0000 7f6a82ffd700 1 -- 172.21.15.7:0/833520403 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6a7401c460 con 0x7f6a8c0b3350 2024-01-27T02:35:49.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.382+0000 7f6a82ffd700 1 --2- 172.21.15.7:0/833520403 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6a6c066f30 0x7f6a6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:49.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.382+0000 7f6a82ffd700 1 -- 172.21.15.7:0/833520403 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6a74084290 con 0x7f6a8c0b3350 2024-01-27T02:35:49.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.382+0000 7f6a82ffd700 1 -- 172.21.15.7:0/833520403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f6a6c06ea50 con 0x7f6a8c0b3350 2024-01-27T02:35:49.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.382+0000 7f6a8ad9d700 1 --2- 172.21.15.7:0/833520403 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6a6c066f30 0x7f6a6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:49.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.383+0000 7f6a8ad9d700 1 --2- 172.21.15.7:0/833520403 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6a6c066f30 0x7f6a6c0693e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c006fd0 tx=0x7f6a7c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:49.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.386+0000 7f6a82ffd700 1 -- 172.21.15.7:0/833520403 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a8c151420 con 0x7f6a8c0b3350 2024-01-27T02:35:49.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.635+0000 7f6a91023700 1 -- 172.21.15.7:0/833520403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6a8c151420 con 0x7f6a8c0b3350 2024-01-27T02:35:49.636 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-27T02:35:49.636+0000 7fb5c503a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-27T02:35:49.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.638+0000 7f6a82ffd700 1 -- 172.21.15.7:0/833520403 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(70..70 src has 1..70) v4 ==== 374+0+0 (secure 0 0 0) 0x7f6a740840d0 con 0x7f6a8c0b3350 2024-01-27T02:35:49.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.638+0000 7f6a82ffd700 1 -- 172.21.15.7:0/833520403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=71}) v3 -- 0x7f6a6c06f3a0 con 0x7f6a8c0b3350 2024-01-27T02:35:49.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.640+0000 7f6a82ffd700 1 -- 172.21.15.7:0/833520403 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6a8c151420 con 0x7f6a8c0b3350 2024-01-27T02:35:49.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.644+0000 7f6a91023700 1 -- 172.21.15.7:0/833520403 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6a6c066f30 msgr2=0x7f6a6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:49.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.644+0000 7f6a91023700 1 --2- 172.21.15.7:0/833520403 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6a6c066f30 0x7f6a6c0693e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c006fd0 tx=0x7f6a7c008040 comp rx=0 tx=0).stop 2024-01-27T02:35:49.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.644+0000 7f6a91023700 1 -- 172.21.15.7:0/833520403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c0b3350 msgr2=0x7f6a8c15a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:49.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.644+0000 7f6a91023700 1 --2- 172.21.15.7:0/833520403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c0b3350 0x7f6a8c15a530 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400c5d0 tx=0x7f6a74007f60 comp rx=0 tx=0).stop 2024-01-27T02:35:49.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.644+0000 7f6a91023700 1 -- 172.21.15.7:0/833520403 shutdown_connections 2024-01-27T02:35:49.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.644+0000 7f6a91023700 1 --2- 172.21.15.7:0/833520403 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6a6c066f30 0x7f6a6c0693e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.644+0000 7f6a91023700 1 --2- 172.21.15.7:0/833520403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a8c1553e0 0x7f6a8c15aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.644+0000 7f6a91023700 1 --2- 172.21.15.7:0/833520403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a8c0b2f80 0x7f6a8c14e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.644+0000 7f6a91023700 1 --2- 172.21.15.7:0/833520403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c0b3350 0x7f6a8c15a530 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.644+0000 7f6a91023700 1 -- 172.21.15.7:0/833520403 >> 172.21.15.7:0/833520403 conn(0x7f6a8c068430 msgr2=0x7f6a8c0cb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:49.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.644+0000 7f6a91023700 1 -- 172.21.15.7:0/833520403 shutdown_connections 2024-01-27T02:35:49.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.644+0000 7f6a91023700 1 -- 172.21.15.7:0/833520403 wait complete. 2024-01-27T02:35:49.646 INFO:tasks.ceph.mds.a.smithi007.stderr: -9581> 2024-01-27T02:34:20.027+0000 7f4d3ee48700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-27T02:35:49.646 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-27T02:35:49.658 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-27T02:35:49.680 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-27T02:35:49.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.793+0000 7fc27698a700 1 Processor -- start 2024-01-27T02:35:49.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.793+0000 7fc27698a700 1 -- start start 2024-01-27T02:35:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.794+0000 7fc27698a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc2700cba60 0x7fc2700ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.794+0000 7fc27698a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2700cbe30 0x7fc27015b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.794+0000 7fc27698a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc27006b4e0 0x7fc27006b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.794+0000 7fc27698a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc27006be80 con 0x7fc2700cba60 2024-01-27T02:35:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.794+0000 7fc27698a700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2700679b0 con 0x7fc27006b4e0 2024-01-27T02:35:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.794+0000 7fc27698a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc270067c90 con 0x7fc2700cbe30 2024-01-27T02:35:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.794+0000 7fc26ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc2700cba60 0x7fc2700ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.794+0000 7fc26ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc2700cba60 0x7fc2700ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44212/0 (socket says 172.21.15.7:44212) 2024-01-27T02:35:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.794+0000 7fc26ffff700 1 -- 172.21.15.7:0/3709375068 learned_addr learned my addr 172.21.15.7:0/3709375068 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.794+0000 7fc274f27700 1 --2- 172.21.15.7:0/3709375068 >> v2:172.21.15.188:3300/0 conn(0x7fc27006b4e0 0x7fc27006b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.794+0000 7fc26f7fe700 1 --2- 172.21.15.7:0/3709375068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2700cbe30 0x7fc27015b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.794+0000 7fc26ffff700 1 -- 172.21.15.7:0/3709375068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2700cbe30 msgr2=0x7fc27015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.794+0000 7fc26ffff700 1 --2- 172.21.15.7:0/3709375068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2700cbe30 0x7fc27015b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.794+0000 7fc26ffff700 1 -- 172.21.15.7:0/3709375068 >> v2:172.21.15.188:3300/0 conn(0x7fc27006b4e0 msgr2=0x7fc27006b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.794+0000 7fc26ffff700 1 --2- 172.21.15.7:0/3709375068 >> v2:172.21.15.188:3300/0 conn(0x7fc27006b4e0 0x7fc27006b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.794+0000 7fc26ffff700 1 -- 172.21.15.7:0/3709375068 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27015b7b0 con 0x7fc2700cba60 2024-01-27T02:35:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.794+0000 7fc274f27700 1 --2- 172.21.15.7:0/3709375068 >> v2:172.21.15.188:3300/0 conn(0x7fc27006b4e0 0x7fc27006b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:35:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.795+0000 7fc26ffff700 1 --2- 172.21.15.7:0/3709375068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2700cba60 0x7fc2700ce410 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc27006e490 tx=0x7fc260009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44e98f176ff2429 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.795+0000 7fc26effd700 1 -- 172.21.15.7:0/3709375068 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc260005c10 con 0x7fc2700cba60 2024-01-27T02:35:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.795+0000 7fc26effd700 1 -- 172.21.15.7:0/3709375068 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc260005d70 con 0x7fc2700cba60 2024-01-27T02:35:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.795+0000 7fc26effd700 1 -- 172.21.15.7:0/3709375068 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc260006260 con 0x7fc2700cba60 2024-01-27T02:35:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.795+0000 7fc27698a700 1 -- 172.21.15.7:0/3709375068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2700cba60 msgr2=0x7fc2700ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:49.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.795+0000 7fc27698a700 1 --2- 172.21.15.7:0/3709375068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2700cba60 0x7fc2700ce410 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc27006e490 tx=0x7fc260009220 comp rx=0 tx=0).stop 2024-01-27T02:35:49.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.795+0000 7fc27698a700 1 -- 172.21.15.7:0/3709375068 shutdown_connections 2024-01-27T02:35:49.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.795+0000 7fc27698a700 1 --2- 172.21.15.7:0/3709375068 >> v2:172.21.15.188:3300/0 conn(0x7fc27006b4e0 0x7fc27006b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.795+0000 7fc27698a700 1 --2- 172.21.15.7:0/3709375068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2700cba60 0x7fc2700ce410 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.795+0000 7fc27698a700 1 --2- 172.21.15.7:0/3709375068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2700cbe30 0x7fc27015b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.795+0000 7fc27698a700 1 -- 172.21.15.7:0/3709375068 >> 172.21.15.7:0/3709375068 conn(0x7fc270063580 msgr2=0x7fc270063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:49.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.795+0000 7fc27698a700 1 -- 172.21.15.7:0/3709375068 shutdown_connections 2024-01-27T02:35:49.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.795+0000 7fc27698a700 1 -- 172.21.15.7:0/3709375068 wait complete. 2024-01-27T02:35:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.797+0000 7fc27698a700 1 Processor -- start 2024-01-27T02:35:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.797+0000 7fc27698a700 1 -- start start 2024-01-27T02:35:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.798+0000 7fc27698a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc27006b4e0 0x7fc27016aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.798+0000 7fc27698a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2700cba60 0x7fc2700b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.798+0000 7fc27698a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2700cbe30 0x7fc2701556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.798+0000 7fc27698a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2700af410 con 0x7fc2700cbe30 2024-01-27T02:35:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.798+0000 7fc27698a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2700b14f0 con 0x7fc27006b4e0 2024-01-27T02:35:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.798+0000 7fc27698a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2700ad0a0 con 0x7fc2700cba60 2024-01-27T02:35:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.798+0000 7fc274f27700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2700cbe30 0x7fc2701556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.798+0000 7fc274f27700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2700cbe30 0x7fc2701556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44222/0 (socket says 172.21.15.7:44222) 2024-01-27T02:35:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.798+0000 7fc274f27700 1 -- 172.21.15.7:0/3088946651 learned_addr learned my addr 172.21.15.7:0/3088946651 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.798+0000 7fc26ffff700 1 --2- 172.21.15.7:0/3088946651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc27006b4e0 0x7fc27016aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.798+0000 7fc26f7fe700 1 --2- 172.21.15.7:0/3088946651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2700cba60 0x7fc2700b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.798+0000 7fc274f27700 1 -- 172.21.15.7:0/3088946651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2700cba60 msgr2=0x7fc2700b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.798+0000 7fc274f27700 1 --2- 172.21.15.7:0/3088946651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2700cba60 0x7fc2700b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.798+0000 7fc274f27700 1 -- 172.21.15.7:0/3088946651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc27006b4e0 msgr2=0x7fc27016aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.798+0000 7fc274f27700 1 --2- 172.21.15.7:0/3088946651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc27006b4e0 0x7fc27016aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.798+0000 7fc274f27700 1 -- 172.21.15.7:0/3088946651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc268009a20 con 0x7fc2700cbe30 2024-01-27T02:35:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.798+0000 7fc26ffff700 1 --2- 172.21.15.7:0/3088946651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc27006b4e0 0x7fc27016aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:35:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.798+0000 7fc274f27700 1 --2- 172.21.15.7:0/3088946651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2700cbe30 0x7fc2701556e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc268009f20 tx=0x7fc26800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.799+0000 7fc257fff700 1 -- 172.21.15.7:0/3088946651 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26800cc80 con 0x7fc2700cbe30 2024-01-27T02:35:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.799+0000 7fc257fff700 1 -- 172.21.15.7:0/3088946651 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26800f040 con 0x7fc2700cbe30 2024-01-27T02:35:49.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.799+0000 7fc257fff700 1 -- 172.21.15.7:0/3088946651 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268014480 con 0x7fc2700cbe30 2024-01-27T02:35:49.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.799+0000 7fc27698a700 1 -- 172.21.15.7:0/3088946651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc260009480 con 0x7fc2700cbe30 2024-01-27T02:35:49.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.799+0000 7fc27698a700 1 -- 172.21.15.7:0/3088946651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2700af6b0 con 0x7fc2700cbe30 2024-01-27T02:35:49.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.800+0000 7fc27698a700 1 -- 172.21.15.7:0/3088946651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2700af270 con 0x7fc2700cbe30 2024-01-27T02:35:49.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.800+0000 7fc257fff700 1 -- 172.21.15.7:0/3088946651 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc268003020 con 0x7fc2700cbe30 2024-01-27T02:35:49.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.800+0000 7fc257fff700 1 --2- 172.21.15.7:0/3088946651 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc25c066f30 0x7fc25c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:49.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.801+0000 7fc257fff700 1 -- 172.21.15.7:0/3088946651 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fc268085020 con 0x7fc2700cbe30 2024-01-27T02:35:49.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.801+0000 7fc257fff700 1 -- 172.21.15.7:0/3088946651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=71}) v3 -- 0x7fc25c06eaf0 con 0x7fc2700cbe30 2024-01-27T02:35:49.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.801+0000 7fc26ffff700 1 --2- 172.21.15.7:0/3088946651 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc25c066f30 0x7fc25c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:49.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.801+0000 7fc26ffff700 1 --2- 172.21.15.7:0/3088946651 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc25c066f30 0x7fc25c0693e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc260009260 tx=0x7fc260016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:49.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:49.805+0000 7fc257fff700 1 -- 172.21.15.7:0/3088946651 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc2700af270 con 0x7fc2700cbe30 2024-01-27T02:35:50.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.058+0000 7fc27698a700 1 -- 172.21.15.7:0/3088946651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc2700af270 con 0x7fc2700cbe30 2024-01-27T02:35:50.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.641+0000 7fc257fff700 1 -- 172.21.15.7:0/3088946651 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc2700af270 con 0x7fc2700cbe30 2024-01-27T02:35:50.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.645+0000 7fc27698a700 1 -- 172.21.15.7:0/3088946651 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc25c066f30 msgr2=0x7fc25c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:50.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.645+0000 7fc27698a700 1 --2- 172.21.15.7:0/3088946651 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc25c066f30 0x7fc25c0693e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc260009260 tx=0x7fc260016000 comp rx=0 tx=0).stop 2024-01-27T02:35:50.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.645+0000 7fc27698a700 1 -- 172.21.15.7:0/3088946651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2700cbe30 msgr2=0x7fc2701556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:50.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.645+0000 7fc27698a700 1 --2- 172.21.15.7:0/3088946651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2700cbe30 0x7fc2701556e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc268009f20 tx=0x7fc26800b040 comp rx=0 tx=0).stop 2024-01-27T02:35:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.645+0000 7fc27698a700 1 -- 172.21.15.7:0/3088946651 shutdown_connections 2024-01-27T02:35:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.645+0000 7fc27698a700 1 --2- 172.21.15.7:0/3088946651 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc25c066f30 0x7fc25c0693e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.645+0000 7fc27698a700 1 --2- 172.21.15.7:0/3088946651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2700cbe30 0x7fc2701556e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.645+0000 7fc27698a700 1 --2- 172.21.15.7:0/3088946651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc27006b4e0 0x7fc27016aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.645+0000 7fc27698a700 1 --2- 172.21.15.7:0/3088946651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2700cba60 0x7fc2700b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.645+0000 7fc27698a700 1 -- 172.21.15.7:0/3088946651 >> 172.21.15.7:0/3088946651 conn(0x7fc270063580 msgr2=0x7fc2700c4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.646+0000 7fc27698a700 1 -- 172.21.15.7:0/3088946651 shutdown_connections 2024-01-27T02:35:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.646+0000 7fc27698a700 1 -- 172.21.15.7:0/3088946651 wait complete. 2024-01-27T02:35:50.663 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-27T02:35:50.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.849+0000 7fd9fcf8f700 1 Processor -- start 2024-01-27T02:35:50.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.849+0000 7fd9fcf8f700 1 -- start start 2024-01-27T02:35:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.850+0000 7fd9fcf8f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd9f80697f0 0x7fd9f80679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.850+0000 7fd9fcf8f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd9f80cf0a0 0x7fd9f80ca460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.850+0000 7fd9fcf8f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f80cf540 0x7fd9f80ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.850+0000 7fd9fcf8f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9f8067ef0 con 0x7fd9f80cf0a0 2024-01-27T02:35:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.850+0000 7fd9fcf8f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9f80c9400 con 0x7fd9f80697f0 2024-01-27T02:35:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.850+0000 7fd9fcf8f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f80c96a0 con 0x7fd9f80cf540 2024-01-27T02:35:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.851+0000 7fd9f5d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd9f80cf0a0 0x7fd9f80ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:50.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.851+0000 7fd9f5d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd9f80cf0a0 0x7fd9f80ca460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44230/0 (socket says 172.21.15.7:44230) 2024-01-27T02:35:50.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.851+0000 7fd9f5d9b700 1 -- 172.21.15.7:0/2353957314 learned_addr learned my addr 172.21.15.7:0/2353957314 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:50.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.851+0000 7fd9f659c700 1 --2- 172.21.15.7:0/2353957314 >> v2:172.21.15.188:3300/0 conn(0x7fd9f80697f0 0x7fd9f80679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:50.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.851+0000 7fd9f6d9d700 1 --2- 172.21.15.7:0/2353957314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f80cf540 0x7fd9f80ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:50.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.851+0000 7fd9f5d9b700 1 -- 172.21.15.7:0/2353957314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f80cf540 msgr2=0x7fd9f80ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:50.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.851+0000 7fd9f5d9b700 1 --2- 172.21.15.7:0/2353957314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f80cf540 0x7fd9f80ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:50.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.851+0000 7fd9f5d9b700 1 -- 172.21.15.7:0/2353957314 >> v2:172.21.15.188:3300/0 conn(0x7fd9f80697f0 msgr2=0x7fd9f80679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:50.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.851+0000 7fd9f5d9b700 1 --2- 172.21.15.7:0/2353957314 >> v2:172.21.15.188:3300/0 conn(0x7fd9f80697f0 0x7fd9f80679b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:50.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.851+0000 7fd9f5d9b700 1 -- 172.21.15.7:0/2353957314 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f80c9980 con 0x7fd9f80cf0a0 2024-01-27T02:35:50.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.851+0000 7fd9f5d9b700 1 --2- 172.21.15.7:0/2353957314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9f80cf0a0 0x7fd9f80ca460 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd9f806a060 tx=0x7fd9e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b85c1f1a6e9a64f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:50.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.851+0000 7fd9f559a700 1 -- 172.21.15.7:0/2353957314 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e000b860 con 0x7fd9f80cf0a0 2024-01-27T02:35:50.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.851+0000 7fd9f559a700 1 -- 172.21.15.7:0/2353957314 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e000b9c0 con 0x7fd9f80cf0a0 2024-01-27T02:35:50.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.851+0000 7fd9f559a700 1 -- 172.21.15.7:0/2353957314 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e000bd60 con 0x7fd9f80cf0a0 2024-01-27T02:35:50.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.852+0000 7fd9fcf8f700 1 -- 172.21.15.7:0/2353957314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9f80cf0a0 msgr2=0x7fd9f80ca460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:50.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.852+0000 7fd9fcf8f700 1 --2- 172.21.15.7:0/2353957314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9f80cf0a0 0x7fd9f80ca460 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd9f806a060 tx=0x7fd9e0009250 comp rx=0 tx=0).stop 2024-01-27T02:35:50.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.852+0000 7fd9fcf8f700 1 -- 172.21.15.7:0/2353957314 shutdown_connections 2024-01-27T02:35:50.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.852+0000 7fd9fcf8f700 1 --2- 172.21.15.7:0/2353957314 >> v2:172.21.15.188:3300/0 conn(0x7fd9f80697f0 0x7fd9f80679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:50.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.852+0000 7fd9fcf8f700 1 --2- 172.21.15.7:0/2353957314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f80cf540 0x7fd9f80ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:50.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.852+0000 7fd9fcf8f700 1 --2- 172.21.15.7:0/2353957314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9f80cf0a0 0x7fd9f80ca460 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:50.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.852+0000 7fd9fcf8f700 1 -- 172.21.15.7:0/2353957314 >> 172.21.15.7:0/2353957314 conn(0x7fd9f806fdd0 msgr2=0x7fd9f8063580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:50.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.852+0000 7fd9fcf8f700 1 -- 172.21.15.7:0/2353957314 shutdown_connections 2024-01-27T02:35:50.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.852+0000 7fd9fcf8f700 1 -- 172.21.15.7:0/2353957314 wait complete. 2024-01-27T02:35:50.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.853+0000 7fd9fcf8f700 1 Processor -- start 2024-01-27T02:35:50.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.853+0000 7fd9fcf8f700 1 -- start start 2024-01-27T02:35:50.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.854+0000 7fd9fcf8f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f80697f0 0x7fd9f80acd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:50.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.854+0000 7fd9fcf8f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f80cf0a0 0x7fd9f81571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:50.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.854+0000 7fd9fcf8f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9f80cf540 0x7fd9f80b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:50.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.854+0000 7fd9fcf8f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f816afa0 con 0x7fd9f80cf540 2024-01-27T02:35:50.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.854+0000 7fd9fcf8f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f816a3b0 con 0x7fd9f80697f0 2024-01-27T02:35:50.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.854+0000 7fd9fcf8f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f816a670 con 0x7fd9f80cf0a0 2024-01-27T02:35:50.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.854+0000 7fd9f6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9f80cf540 0x7fd9f80b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:50.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.854+0000 7fd9f6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9f80cf540 0x7fd9f80b08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44240/0 (socket says 172.21.15.7:44240) 2024-01-27T02:35:50.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.854+0000 7fd9f6d9d700 1 -- 172.21.15.7:0/2410060297 learned_addr learned my addr 172.21.15.7:0/2410060297 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:50.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.854+0000 7fd9f659c700 1 --2- 172.21.15.7:0/2410060297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f80697f0 0x7fd9f80acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:50.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.854+0000 7fd9f5d9b700 1 --2- 172.21.15.7:0/2410060297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f80cf0a0 0x7fd9f81571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:50.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.854+0000 7fd9f6d9d700 1 -- 172.21.15.7:0/2410060297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f80cf0a0 msgr2=0x7fd9f81571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:50.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.854+0000 7fd9f6d9d700 1 --2- 172.21.15.7:0/2410060297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f80cf0a0 0x7fd9f81571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:50.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.854+0000 7fd9f6d9d700 1 -- 172.21.15.7:0/2410060297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f80697f0 msgr2=0x7fd9f80acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:50.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.854+0000 7fd9f6d9d700 1 --2- 172.21.15.7:0/2410060297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f80697f0 0x7fd9f80acd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:50.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.855+0000 7fd9f6d9d700 1 -- 172.21.15.7:0/2410060297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e00094f0 con 0x7fd9f80cf540 2024-01-27T02:35:50.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.855+0000 7fd9f5d9b700 1 --2- 172.21.15.7:0/2410060297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f80cf0a0 0x7fd9f81571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:35:50.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.855+0000 7fd9f6d9d700 1 --2- 172.21.15.7:0/2410060297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9f80cf540 0x7fd9f80b08d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009fa0 tx=0x7fd9f0009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:50.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.855+0000 7fd9ee7fc700 1 -- 172.21.15.7:0/2410060297 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f000ec70 con 0x7fd9f80cf540 2024-01-27T02:35:50.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.855+0000 7fd9ee7fc700 1 -- 172.21.15.7:0/2410060297 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f000edd0 con 0x7fd9f80cf540 2024-01-27T02:35:50.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.855+0000 7fd9ee7fc700 1 -- 172.21.15.7:0/2410060297 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0014670 con 0x7fd9f80cf540 2024-01-27T02:35:50.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.855+0000 7fd9fcf8f700 1 -- 172.21.15.7:0/2410060297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9f816a950 con 0x7fd9f80cf540 2024-01-27T02:35:50.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.855+0000 7fd9fcf8f700 1 -- 172.21.15.7:0/2410060297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f80ad2a0 con 0x7fd9f80cf540 2024-01-27T02:35:50.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.856+0000 7fd9fcf8f700 1 -- 172.21.15.7:0/2410060297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9f8069bc0 con 0x7fd9f80cf540 2024-01-27T02:35:50.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.857+0000 7fd9ee7fc700 1 -- 172.21.15.7:0/2410060297 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd9f00149e0 con 0x7fd9f80cf540 2024-01-27T02:35:50.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.857+0000 7fd9ee7fc700 1 --2- 172.21.15.7:0/2410060297 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd9d8066f30 0x7fd9d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:50.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.857+0000 7fd9ee7fc700 1 -- 172.21.15.7:0/2410060297 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fd9f0084c70 con 0x7fd9f80cf540 2024-01-27T02:35:50.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.857+0000 7fd9ee7fc700 1 -- 172.21.15.7:0/2410060297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=71}) v3 -- 0x7fd9d806eaf0 con 0x7fd9f80cf540 2024-01-27T02:35:50.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.857+0000 7fd9f659c700 1 --2- 172.21.15.7:0/2410060297 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd9d8066f30 0x7fd9d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:50.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.858+0000 7fd9f659c700 1 --2- 172.21.15.7:0/2410060297 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd9d8066f30 0x7fd9d80693e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80061f0 tx=0x7fd9e8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:50.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:50.861+0000 7fd9ee7fc700 1 -- 172.21.15.7:0/2410060297 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9f0051e90 con 0x7fd9f80cf540 2024-01-27T02:35:51.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.065+0000 7fd9fcf8f700 1 -- 172.21.15.7:0/2410060297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd9f804c710 con 0x7fd9f80cf540 2024-01-27T02:35:51.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.641+0000 7fd9ee7fc700 1 -- 172.21.15.7:0/2410060297 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(71..71 src has 1..71) v4 ==== 1057+0+0 (secure 0 0 0) 0x7fd9f008a020 con 0x7fd9f80cf540 2024-01-27T02:35:51.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.641+0000 7fd9ee7fc700 1 -- 172.21.15.7:0/2410060297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7fd9d806cf20 con 0x7fd9f80cf540 2024-01-27T02:35:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.646+0000 7fd9ee7fc700 1 -- 172.21.15.7:0/2410060297 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v71) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd9f00554b0 con 0x7fd9f80cf540 2024-01-27T02:35:51.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.651+0000 7fd9fcf8f700 1 -- 172.21.15.7:0/2410060297 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd9d8066f30 msgr2=0x7fd9d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:51.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.651+0000 7fd9fcf8f700 1 --2- 172.21.15.7:0/2410060297 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd9d8066f30 0x7fd9d80693e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80061f0 tx=0x7fd9e8008040 comp rx=0 tx=0).stop 2024-01-27T02:35:51.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.651+0000 7fd9fcf8f700 1 -- 172.21.15.7:0/2410060297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9f80cf540 msgr2=0x7fd9f80b08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:51.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.651+0000 7fd9fcf8f700 1 --2- 172.21.15.7:0/2410060297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9f80cf540 0x7fd9f80b08d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009fa0 tx=0x7fd9f0009900 comp rx=0 tx=0).stop 2024-01-27T02:35:51.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.651+0000 7fd9fcf8f700 1 -- 172.21.15.7:0/2410060297 shutdown_connections 2024-01-27T02:35:51.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.651+0000 7fd9fcf8f700 1 --2- 172.21.15.7:0/2410060297 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd9d8066f30 0x7fd9d80693e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:51.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.651+0000 7fd9fcf8f700 1 --2- 172.21.15.7:0/2410060297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9f80cf540 0x7fd9f80b08d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:51.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.651+0000 7fd9fcf8f700 1 --2- 172.21.15.7:0/2410060297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f80697f0 0x7fd9f80acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:51.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.651+0000 7fd9fcf8f700 1 --2- 172.21.15.7:0/2410060297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f80cf0a0 0x7fd9f81571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:51.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.651+0000 7fd9fcf8f700 1 -- 172.21.15.7:0/2410060297 >> 172.21.15.7:0/2410060297 conn(0x7fd9f806fdd0 msgr2=0x7fd9f80c47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:51.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.651+0000 7fd9fcf8f700 1 -- 172.21.15.7:0/2410060297 shutdown_connections 2024-01-27T02:35:51.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.651+0000 7fd9fcf8f700 1 -- 172.21.15.7:0/2410060297 wait complete. 2024-01-27T02:35:51.653 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-27T02:35:51.668 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-27T02:35:51.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.807+0000 7f99868c2700 1 Processor -- start 2024-01-27T02:35:51.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.807+0000 7f99868c2700 1 -- start start 2024-01-27T02:35:51.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.808+0000 7f99868c2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99800cdd40 0x7f99800cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:51.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.808+0000 7f99868c2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99800cb470 con 0x7f9980157f40 2024-01-27T02:35:51.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.808+0000 7f99868c2700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99800cc0a0 con 0x7f99800cbc00 2024-01-27T02:35:51.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.808+0000 7f99868c2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99800ca670 con 0x7f99800cdd40 2024-01-27T02:35:51.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.808+0000 7f997ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9980157f40 0x7f99800cd630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34206/0 (socket says 172.21.15.7:34206) 2024-01-27T02:35:51.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.808+0000 7f997ffff700 1 -- 172.21.15.7:0/531665373 learned_addr learned my addr 172.21.15.7:0/531665373 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:35:51.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997f7fe700 1 --2- 172.21.15.7:0/531665373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99800cdd40 0x7f99800cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:51.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997effd700 1 -- 172.21.15.7:0/531665373 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2247086553 0 0) 0x7f99800cb470 con 0x7f9980157f40 2024-01-27T02:35:51.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997effd700 1 -- 172.21.15.7:0/531665373 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f996c003570 con 0x7f9980157f40 2024-01-27T02:35:51.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997effd700 1 -- 172.21.15.7:0/531665373 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 566436651 0 0) 0x7f996c003570 con 0x7f9980157f40 2024-01-27T02:35:51.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997effd700 1 -- 172.21.15.7:0/531665373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99800cdd40 msgr2=0x7f99800cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:51.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997effd700 1 --2- 172.21.15.7:0/531665373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99800cdd40 0x7f99800cb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:51.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997effd700 1 -- 172.21.15.7:0/531665373 >> v1:172.21.15.188:6789/0 conn(0x7f99800cbc00 legacy=0x7f99800c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:51.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997effd700 1 -- 172.21.15.7:0/531665373 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99800ca860 con 0x7f9980157f40 2024-01-27T02:35:51.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997effd700 1 -- 172.21.15.7:0/531665373 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f99700015f0 con 0x7f9980157f40 2024-01-27T02:35:51.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997effd700 1 -- 172.21.15.7:0/531665373 >> v1:172.21.15.7:6789/0 conn(0x7f9980157f40 legacy=0x7f99800cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:51.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997effd700 1 --2- 172.21.15.7:0/531665373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99800cdd40 0x7f996c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:51.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997effd700 1 --2- 172.21.15.7:0/531665373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f996c004010 0x7f996c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:51.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997effd700 1 --2- 172.21.15.7:0/531665373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f996c004d50 0x7f996c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:51.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997effd700 1 -- 172.21.15.7:0/531665373 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c00bbf0 con 0x7f996c004010 2024-01-27T02:35:51.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997effd700 1 -- 172.21.15.7:0/531665373 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c00bd30 con 0x7f996c004d50 2024-01-27T02:35:51.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997effd700 1 -- 172.21.15.7:0/531665373 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996c00bef0 con 0x7f99800cdd40 2024-01-27T02:35:51.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997f7fe700 1 --2- 172.21.15.7:0/531665373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f996c004010 0x7f996c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:51.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997ffff700 1 --2- 172.21.15.7:0/531665373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99800cdd40 0x7f996c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:51.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f9984e5f700 1 --2- 172.21.15.7:0/531665373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f996c004d50 0x7f996c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:51.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997f7fe700 1 -- 172.21.15.7:0/531665373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99800cdd40 msgr2=0x7f996c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:51.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997f7fe700 1 --2- 172.21.15.7:0/531665373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99800cdd40 0x7f996c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:51.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997f7fe700 1 -- 172.21.15.7:0/531665373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f996c004d50 msgr2=0x7f996c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:51.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997f7fe700 1 --2- 172.21.15.7:0/531665373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f996c004d50 0x7f996c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:51.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f997f7fe700 1 -- 172.21.15.7:0/531665373 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c003570 con 0x7f996c004010 2024-01-27T02:35:51.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.809+0000 7f9984e5f700 1 --2- 172.21.15.7:0/531665373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f996c004d50 0x7f996c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:35:51.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.810+0000 7f997f7fe700 1 --2- 172.21.15.7:0/531665373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f996c004010 0x7f996c004810 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9968006f00 tx=0x7f99680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5bd4104c0fdd697 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:51.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.810+0000 7f997effd700 1 -- 172.21.15.7:0/531665373 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996800e710 con 0x7f996c004010 2024-01-27T02:35:51.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.810+0000 7f997effd700 1 -- 172.21.15.7:0/531665373 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996800e870 con 0x7f996c004010 2024-01-27T02:35:51.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.810+0000 7f997effd700 1 -- 172.21.15.7:0/531665373 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996800ec30 con 0x7f996c004010 2024-01-27T02:35:51.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.810+0000 7f99868c2700 1 -- 172.21.15.7:0/531665373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f996c004010 msgr2=0x7f996c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:51.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.810+0000 7f99868c2700 1 --2- 172.21.15.7:0/531665373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f996c004010 0x7f996c004810 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9968006f00 tx=0x7f99680091a0 comp rx=0 tx=0).stop 2024-01-27T02:35:51.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.810+0000 7f9984e5f700 1 -- 172.21.15.7:0/531665373 reap_dead start 2024-01-27T02:35:51.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.810+0000 7f99868c2700 1 -- 172.21.15.7:0/531665373 shutdown_connections 2024-01-27T02:35:51.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.810+0000 7f99868c2700 1 -- 172.21.15.7:0/531665373 >> 172.21.15.7:0/531665373 conn(0x7f9980098020 msgr2=0x7f99800711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:51.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.811+0000 7f99868c2700 1 -- 172.21.15.7:0/531665373 shutdown_connections 2024-01-27T02:35:51.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.811+0000 7f99868c2700 1 -- 172.21.15.7:0/531665373 wait complete. 2024-01-27T02:35:51.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.812+0000 7f99868c2700 1 Processor -- start 2024-01-27T02:35:51.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.812+0000 7f99868c2700 1 -- start start 2024-01-27T02:35:51.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.813+0000 7f99868c2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f996c004010 0x7f998015a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:51.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.813+0000 7f99868c2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998014dea0 0x7f99800b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:51.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.813+0000 7f99868c2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99800b3010 0x7f99800c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:51.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.813+0000 7f99868c2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99800c67f0 con 0x7f998014dea0 2024-01-27T02:35:51.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.813+0000 7f99868c2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998015aa50 con 0x7f996c004010 2024-01-27T02:35:51.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.813+0000 7f99868c2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99800c78c0 con 0x7f99800b3010 2024-01-27T02:35:51.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.813+0000 7f997f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998014dea0 0x7f99800b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:51.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.813+0000 7f997f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998014dea0 0x7f99800b2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42310/0 (socket says 172.21.15.7:42310) 2024-01-27T02:35:51.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.813+0000 7f997f7fe700 1 -- 172.21.15.7:0/4197035344 learned_addr learned my addr 172.21.15.7:0/4197035344 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:51.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.813+0000 7f997f7fe700 1 -- 172.21.15.7:0/4197035344 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99800b3010 msgr2=0x7f99800c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:51.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.813+0000 7f9984e5f700 1 --2- 172.21.15.7:0/4197035344 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99800b3010 0x7f99800c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:51.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.813+0000 7f997ffff700 1 --2- 172.21.15.7:0/4197035344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f996c004010 0x7f998015a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:51.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.813+0000 7f997f7fe700 1 --2- 172.21.15.7:0/4197035344 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99800b3010 0x7f99800c6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:51.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.813+0000 7f997f7fe700 1 -- 172.21.15.7:0/4197035344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f996c004010 msgr2=0x7f998015a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:51.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.813+0000 7f997f7fe700 1 --2- 172.21.15.7:0/4197035344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f996c004010 0x7f998015a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:51.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.813+0000 7f997f7fe700 1 -- 172.21.15.7:0/4197035344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99800c6510 con 0x7f998014dea0 2024-01-27T02:35:51.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.813+0000 7f997f7fe700 1 --2- 172.21.15.7:0/4197035344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998014dea0 0x7f99800b2ad0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f996800e4b0 tx=0x7f996801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:51.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.813+0000 7f9977fff700 1 -- 172.21.15.7:0/4197035344 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9968023070 con 0x7f998014dea0 2024-01-27T02:35:51.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.813+0000 7f9977fff700 1 -- 172.21.15.7:0/4197035344 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996800e9e0 con 0x7f998014dea0 2024-01-27T02:35:51.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.814+0000 7f99868c2700 1 -- 172.21.15.7:0/4197035344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99800c8340 con 0x7f998014dea0 2024-01-27T02:35:51.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.814+0000 7f9977fff700 1 -- 172.21.15.7:0/4197035344 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996801e030 con 0x7f998014dea0 2024-01-27T02:35:51.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.814+0000 7f99868c2700 1 -- 172.21.15.7:0/4197035344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998014e310 con 0x7f998014dea0 2024-01-27T02:35:51.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.815+0000 7f99868c2700 1 -- 172.21.15.7:0/4197035344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f998015cb30 con 0x7f998014dea0 2024-01-27T02:35:51.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.815+0000 7f9977fff700 1 -- 172.21.15.7:0/4197035344 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9968013470 con 0x7f998014dea0 2024-01-27T02:35:51.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.815+0000 7f9977fff700 1 --2- 172.21.15.7:0/4197035344 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9960066f30 0x7f99600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:51.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.815+0000 7f9977fff700 1 -- 172.21.15.7:0/4197035344 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f9968085af0 con 0x7f998014dea0 2024-01-27T02:35:51.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.816+0000 7f997ffff700 1 --2- 172.21.15.7:0/4197035344 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9960066f30 0x7f99600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:51.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.816+0000 7f997ffff700 1 --2- 172.21.15.7:0/4197035344 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9960066f30 0x7f99600693e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9970000ef0 tx=0x7f9970009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:51.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:51.820+0000 7f9977fff700 1 -- 172.21.15.7:0/4197035344 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9968052ec0 con 0x7f998014dea0 2024-01-27T02:35:52.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.023+0000 7f99868c2700 1 -- 172.21.15.7:0/4197035344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f998004c710 con 0x7f998014dea0 2024-01-27T02:35:52.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.652+0000 7f9977fff700 1 -- 172.21.15.7:0/4197035344 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v72) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9968017020 con 0x7f998014dea0 2024-01-27T02:35:52.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.658+0000 7f99868c2700 1 -- 172.21.15.7:0/4197035344 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9960066f30 msgr2=0x7f99600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:52.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.658+0000 7f99868c2700 1 --2- 172.21.15.7:0/4197035344 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9960066f30 0x7f99600693e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9970000ef0 tx=0x7f9970009040 comp rx=0 tx=0).stop 2024-01-27T02:35:52.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.658+0000 7f99868c2700 1 -- 172.21.15.7:0/4197035344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998014dea0 msgr2=0x7f99800b2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:52.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.658+0000 7f99868c2700 1 --2- 172.21.15.7:0/4197035344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998014dea0 0x7f99800b2ad0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f996800e4b0 tx=0x7f996801a000 comp rx=0 tx=0).stop 2024-01-27T02:35:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.658+0000 7f99868c2700 1 -- 172.21.15.7:0/4197035344 shutdown_connections 2024-01-27T02:35:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.658+0000 7f99868c2700 1 --2- 172.21.15.7:0/4197035344 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9960066f30 0x7f99600693e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.658+0000 7f99868c2700 1 --2- 172.21.15.7:0/4197035344 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99800b3010 0x7f99800c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.658+0000 7f99868c2700 1 --2- 172.21.15.7:0/4197035344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f996c004010 0x7f998015a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.658+0000 7f99868c2700 1 --2- 172.21.15.7:0/4197035344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998014dea0 0x7f99800b2ad0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.658+0000 7f99868c2700 1 -- 172.21.15.7:0/4197035344 >> 172.21.15.7:0/4197035344 conn(0x7f9980098020 msgr2=0x7f99800c9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.658+0000 7f99868c2700 1 -- 172.21.15.7:0/4197035344 shutdown_connections 2024-01-27T02:35:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.658+0000 7f99868c2700 1 -- 172.21.15.7:0/4197035344 wait complete. 2024-01-27T02:35:52.660 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-27T02:35:52.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-27T02:35:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.812+0000 7fb33edae700 1 Processor -- start 2024-01-27T02:35:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.812+0000 7fb33edae700 1 -- start start 2024-01-27T02:35:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.813+0000 7fb33edae700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb338156f30 0x7fb3380aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.813+0000 7fb33edae700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb3380af470 0x7fb33815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.813+0000 7fb33edae700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3380aeaa0 con 0x7fb338156b60 2024-01-27T02:35:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.813+0000 7fb33edae700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3380adbc0 con 0x7fb3380af470 2024-01-27T02:35:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.813+0000 7fb33edae700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3380ade90 con 0x7fb338156f30 2024-01-27T02:35:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.814+0000 7fb33cb4a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb338156b60 0x7fb3380ad4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34222/0 (socket says 172.21.15.7:34222) 2024-01-27T02:35:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.814+0000 7fb33cb4a700 1 -- 172.21.15.7:0/3350800492 learned_addr learned my addr 172.21.15.7:0/3350800492 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:35:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.814+0000 7fb337fff700 1 --2- 172.21.15.7:0/3350800492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb338156f30 0x7fb3380aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.814+0000 7fb33d34b700 1 --2- 172.21.15.7:0/3350800492 >> v2:172.21.15.188:3300/0 conn(0x7fb3380af470 0x7fb33815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.814+0000 7fb3377fe700 1 -- 172.21.15.7:0/3350800492 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3918636629 0 0) 0x7fb3380aeaa0 con 0x7fb338156b60 2024-01-27T02:35:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.814+0000 7fb3377fe700 1 -- 172.21.15.7:0/3350800492 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb328003570 con 0x7fb338156b60 2024-01-27T02:35:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.814+0000 7fb3377fe700 1 -- 172.21.15.7:0/3350800492 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2060930438 0 0) 0x7fb328003570 con 0x7fb338156b60 2024-01-27T02:35:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.814+0000 7fb3377fe700 1 -- 172.21.15.7:0/3350800492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb338156f30 msgr2=0x7fb3380aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.814+0000 7fb3377fe700 1 --2- 172.21.15.7:0/3350800492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb338156f30 0x7fb3380aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.814+0000 7fb3377fe700 1 -- 172.21.15.7:0/3350800492 >> v2:172.21.15.188:3300/0 conn(0x7fb3380af470 msgr2=0x7fb33815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.814+0000 7fb3377fe700 1 --2- 172.21.15.7:0/3350800492 >> v2:172.21.15.188:3300/0 conn(0x7fb3380af470 0x7fb33815a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.814+0000 7fb3377fe700 1 -- 172.21.15.7:0/3350800492 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3380ae170 con 0x7fb338156b60 2024-01-27T02:35:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.814+0000 7fb3377fe700 1 -- 172.21.15.7:0/3350800492 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7fb32c001470 con 0x7fb338156b60 2024-01-27T02:35:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.814+0000 7fb3377fe700 1 -- 172.21.15.7:0/3350800492 >> v1:172.21.15.7:6789/0 conn(0x7fb338156b60 legacy=0x7fb3380ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.814+0000 7fb3377fe700 1 --2- 172.21.15.7:0/3350800492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb338156f30 0x7fb328003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.814+0000 7fb3377fe700 1 --2- 172.21.15.7:0/3350800492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3280040b0 0x7fb328004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.814+0000 7fb3377fe700 1 --2- 172.21.15.7:0/3350800492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb328004d50 0x7fb3280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.814+0000 7fb3377fe700 1 -- 172.21.15.7:0/3350800492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3380adbc0 con 0x7fb3280040b0 2024-01-27T02:35:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.814+0000 7fb3377fe700 1 -- 172.21.15.7:0/3350800492 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3380ade90 con 0x7fb328004d50 2024-01-27T02:35:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.814+0000 7fb3377fe700 1 -- 172.21.15.7:0/3350800492 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32800dd70 con 0x7fb338156f30 2024-01-27T02:35:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.814+0000 7fb337fff700 1 --2- 172.21.15.7:0/3350800492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3280040b0 0x7fb328004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.814+0000 7fb33cb4a700 1 --2- 172.21.15.7:0/3350800492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb338156f30 0x7fb328003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.815+0000 7fb33d34b700 1 --2- 172.21.15.7:0/3350800492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb328004d50 0x7fb3280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.815+0000 7fb337fff700 1 -- 172.21.15.7:0/3350800492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb338156f30 msgr2=0x7fb328003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.815+0000 7fb337fff700 1 --2- 172.21.15.7:0/3350800492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb338156f30 0x7fb328003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.815+0000 7fb337fff700 1 -- 172.21.15.7:0/3350800492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb328004d50 msgr2=0x7fb3280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.815+0000 7fb337fff700 1 --2- 172.21.15.7:0/3350800492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb328004d50 0x7fb3280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.815+0000 7fb337fff700 1 -- 172.21.15.7:0/3350800492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb328003570 con 0x7fb3280040b0 2024-01-27T02:35:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.815+0000 7fb337fff700 1 --2- 172.21.15.7:0/3350800492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3280040b0 0x7fb328004810 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb324006f00 tx=0x7fb3240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abeb7e16339a71e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.815+0000 7fb3377fe700 1 -- 172.21.15.7:0/3350800492 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32400e710 con 0x7fb3280040b0 2024-01-27T02:35:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.815+0000 7fb3377fe700 1 -- 172.21.15.7:0/3350800492 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32400e870 con 0x7fb3280040b0 2024-01-27T02:35:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.815+0000 7fb3377fe700 1 -- 172.21.15.7:0/3350800492 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32400ec30 con 0x7fb3280040b0 2024-01-27T02:35:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.816+0000 7fb33edae700 1 -- 172.21.15.7:0/3350800492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3280040b0 msgr2=0x7fb328004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.816+0000 7fb33edae700 1 --2- 172.21.15.7:0/3350800492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3280040b0 0x7fb328004810 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb324006f00 tx=0x7fb3240091a0 comp rx=0 tx=0).stop 2024-01-27T02:35:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.816+0000 7fb33d34b700 1 -- 172.21.15.7:0/3350800492 reap_dead start 2024-01-27T02:35:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.816+0000 7fb33edae700 1 -- 172.21.15.7:0/3350800492 shutdown_connections 2024-01-27T02:35:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.816+0000 7fb33edae700 1 -- 172.21.15.7:0/3350800492 >> 172.21.15.7:0/3350800492 conn(0x7fb338068480 msgr2=0x7fb3380cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.816+0000 7fb33edae700 1 -- 172.21.15.7:0/3350800492 shutdown_connections 2024-01-27T02:35:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.816+0000 7fb33edae700 1 -- 172.21.15.7:0/3350800492 wait complete. 2024-01-27T02:35:52.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.817+0000 7fb33edae700 1 Processor -- start 2024-01-27T02:35:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.817+0000 7fb33edae700 1 -- start start 2024-01-27T02:35:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.818+0000 7fb33edae700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3280040b0 0x7fb33813d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.818+0000 7fb33edae700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb33814bd80 0x7fb33814c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.818+0000 7fb33edae700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb338154a70 0x7fb338154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.818+0000 7fb33edae700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb338155450 con 0x7fb33814bd80 2024-01-27T02:35:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.818+0000 7fb33edae700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb338152e40 con 0x7fb3280040b0 2024-01-27T02:35:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.818+0000 7fb33edae700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33813ae10 con 0x7fb338154a70 2024-01-27T02:35:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.818+0000 7fb337fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb33814bd80 0x7fb33814c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.818+0000 7fb337fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb33814bd80 0x7fb33814c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42336/0 (socket says 172.21.15.7:42336) 2024-01-27T02:35:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.818+0000 7fb337fff700 1 -- 172.21.15.7:0/1382712579 learned_addr learned my addr 172.21.15.7:0/1382712579 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.818+0000 7fb33cb4a700 1 --2- 172.21.15.7:0/1382712579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3280040b0 0x7fb33813d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.818+0000 7fb33d34b700 1 --2- 172.21.15.7:0/1382712579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb338154a70 0x7fb338154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.818+0000 7fb337fff700 1 -- 172.21.15.7:0/1382712579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb338154a70 msgr2=0x7fb338154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.818+0000 7fb337fff700 1 --2- 172.21.15.7:0/1382712579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb338154a70 0x7fb338154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.818+0000 7fb337fff700 1 -- 172.21.15.7:0/1382712579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3280040b0 msgr2=0x7fb33813d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.818+0000 7fb337fff700 1 --2- 172.21.15.7:0/1382712579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3280040b0 0x7fb33813d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.818+0000 7fb337fff700 1 -- 172.21.15.7:0/1382712579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33814c7d0 con 0x7fb33814bd80 2024-01-27T02:35:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.818+0000 7fb33cb4a700 1 --2- 172.21.15.7:0/1382712579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3280040b0 0x7fb33813d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:35:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.818+0000 7fb33d34b700 1 --2- 172.21.15.7:0/1382712579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb338154a70 0x7fb338154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:35:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.819+0000 7fb337fff700 1 --2- 172.21.15.7:0/1382712579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb33814bd80 0x7fb33814c1f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb328004760 tx=0x7fb32401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.819+0000 7fb334ff9700 1 -- 172.21.15.7:0/1382712579 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb324022070 con 0x7fb33814bd80 2024-01-27T02:35:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.819+0000 7fb334ff9700 1 -- 172.21.15.7:0/1382712579 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32400e9e0 con 0x7fb33814bd80 2024-01-27T02:35:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.819+0000 7fb33edae700 1 -- 172.21.15.7:0/1382712579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33814cab0 con 0x7fb33814bd80 2024-01-27T02:35:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.819+0000 7fb334ff9700 1 -- 172.21.15.7:0/1382712579 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb324027470 con 0x7fb33814bd80 2024-01-27T02:35:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.819+0000 7fb33edae700 1 -- 172.21.15.7:0/1382712579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33813dd40 con 0x7fb33814bd80 2024-01-27T02:35:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.820+0000 7fb33edae700 1 -- 172.21.15.7:0/1382712579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33814ccf0 con 0x7fb33814bd80 2024-01-27T02:35:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.821+0000 7fb334ff9700 1 -- 172.21.15.7:0/1382712579 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb32401c450 con 0x7fb33814bd80 2024-01-27T02:35:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.821+0000 7fb334ff9700 1 --2- 172.21.15.7:0/1382712579 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb31c066f30 0x7fb31c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.821+0000 7fb334ff9700 1 -- 172.21.15.7:0/1382712579 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb324085e80 con 0x7fb33814bd80 2024-01-27T02:35:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.821+0000 7fb33cb4a700 1 --2- 172.21.15.7:0/1382712579 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb31c066f30 0x7fb31c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.822+0000 7fb33cb4a700 1 --2- 172.21.15.7:0/1382712579 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb31c066f30 0x7fb31c0693e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb32c007dd0 tx=0x7fb32c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:52.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:52.826+0000 7fb334ff9700 1 -- 172.21.15.7:0/1382712579 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb33814ccf0 con 0x7fb33814bd80 2024-01-27T02:35:53.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.029+0000 7fb33edae700 1 -- 172.21.15.7:0/1382712579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb33814ccf0 con 0x7fb33814bd80 2024-01-27T02:35:53.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.033+0000 7fb334ff9700 1 -- 172.21.15.7:0/1382712579 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb33814ccf0 con 0x7fb33814bd80 2024-01-27T02:35:53.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.035+0000 7fb33edae700 1 -- 172.21.15.7:0/1382712579 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb31c066f30 msgr2=0x7fb31c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:53.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.035+0000 7fb33edae700 1 --2- 172.21.15.7:0/1382712579 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb31c066f30 0x7fb31c0693e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb32c007dd0 tx=0x7fb32c009040 comp rx=0 tx=0).stop 2024-01-27T02:35:53.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.035+0000 7fb33edae700 1 -- 172.21.15.7:0/1382712579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb33814bd80 msgr2=0x7fb33814c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:53.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.035+0000 7fb33edae700 1 --2- 172.21.15.7:0/1382712579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb33814bd80 0x7fb33814c1f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb328004760 tx=0x7fb32401a000 comp rx=0 tx=0).stop 2024-01-27T02:35:53.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.035+0000 7fb33edae700 1 -- 172.21.15.7:0/1382712579 shutdown_connections 2024-01-27T02:35:53.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.035+0000 7fb33edae700 1 --2- 172.21.15.7:0/1382712579 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb31c066f30 0x7fb31c0693e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:53.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.035+0000 7fb33edae700 1 --2- 172.21.15.7:0/1382712579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb338154a70 0x7fb338154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:53.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.035+0000 7fb33edae700 1 --2- 172.21.15.7:0/1382712579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3280040b0 0x7fb33813d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:53.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.035+0000 7fb33edae700 1 --2- 172.21.15.7:0/1382712579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb33814bd80 0x7fb33814c1f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:53.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.035+0000 7fb33edae700 1 -- 172.21.15.7:0/1382712579 >> 172.21.15.7:0/1382712579 conn(0x7fb338068480 msgr2=0x7fb3380a64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:53.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.035+0000 7fb33edae700 1 -- 172.21.15.7:0/1382712579 shutdown_connections 2024-01-27T02:35:53.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.035+0000 7fb33edae700 1 -- 172.21.15.7:0/1382712579 wait complete. 2024-01-27T02:35:53.050 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2024-01-27T02:35:53.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.180+0000 7f995393d700 1 Processor -- start 2024-01-27T02:35:53.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.181+0000 7f995393d700 1 -- start start 2024-01-27T02:35:53.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.182+0000 7f995393d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f994c0a7bd0 0x7f994c0af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.182+0000 7f995393d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f994c06f870 con 0x7f994c0a7fa0 2024-01-27T02:35:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.182+0000 7f995393d700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f994c06ec60 con 0x7f994c0b1aa0 2024-01-27T02:35:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.182+0000 7f995393d700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f994c06fe40 con 0x7f994c0a7bd0 2024-01-27T02:35:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.182+0000 7f9950ed8700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f994c0a7fa0 0x7f994c0b1390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34226/0 (socket says 172.21.15.7:34226) 2024-01-27T02:35:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.182+0000 7f9950ed8700 1 -- 172.21.15.7:0/3019315020 learned_addr learned my addr 172.21.15.7:0/3019315020 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:35:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.182+0000 7f99516d9700 1 --2- 172.21.15.7:0/3019315020 >> v2:172.21.15.188:3300/0 conn(0x7f994c0a7bd0 0x7f994c0af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.182+0000 7f9943fff700 1 -- 172.21.15.7:0/3019315020 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3800775730 0 0) 0x7f994c06f870 con 0x7f994c0a7fa0 2024-01-27T02:35:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.182+0000 7f9943fff700 1 -- 172.21.15.7:0/3019315020 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9938003570 con 0x7f994c0a7fa0 2024-01-27T02:35:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.182+0000 7f9943fff700 1 -- 172.21.15.7:0/3019315020 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4294106933 0 0) 0x7f994c06ec60 con 0x7f994c0b1aa0 2024-01-27T02:35:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.182+0000 7f9943fff700 1 -- 172.21.15.7:0/3019315020 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f994c06f870 con 0x7f994c0b1aa0 2024-01-27T02:35:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.182+0000 7f9943fff700 1 -- 172.21.15.7:0/3019315020 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3399651141 0 0) 0x7f9938003570 con 0x7f994c0a7fa0 2024-01-27T02:35:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.182+0000 7f9943fff700 1 -- 172.21.15.7:0/3019315020 >> v2:172.21.15.188:3300/0 conn(0x7f994c0a7bd0 msgr2=0x7f994c0af6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.182+0000 7f9943fff700 1 --2- 172.21.15.7:0/3019315020 >> v2:172.21.15.188:3300/0 conn(0x7f994c0a7bd0 0x7f994c0af6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:53.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.182+0000 7f9943fff700 1 -- 172.21.15.7:0/3019315020 >> v1:172.21.15.188:6789/0 conn(0x7f994c0b1aa0 legacy=0x7f994c097bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:53.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.182+0000 7f9943fff700 1 -- 172.21.15.7:0/3019315020 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994c06ef40 con 0x7f994c0a7fa0 2024-01-27T02:35:53.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.182+0000 7f9943fff700 1 -- 172.21.15.7:0/3019315020 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f993c001470 con 0x7f994c0a7fa0 2024-01-27T02:35:53.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.182+0000 7f9943fff700 1 -- 172.21.15.7:0/3019315020 >> v1:172.21.15.7:6789/0 conn(0x7f994c0a7fa0 legacy=0x7f994c0b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:53.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.183+0000 7f9943fff700 1 --2- 172.21.15.7:0/3019315020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9938003bf0 0x7f9938003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:53.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.183+0000 7f9943fff700 1 --2- 172.21.15.7:0/3019315020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9938004810 0x7f9938004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:53.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.183+0000 7f9943fff700 1 --2- 172.21.15.7:0/3019315020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99380051c0 0x7f993800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:53.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.183+0000 7f9943fff700 1 -- 172.21.15.7:0/3019315020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c06fe40 con 0x7f99380051c0 2024-01-27T02:35:53.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.183+0000 7f9943fff700 1 -- 172.21.15.7:0/3019315020 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993800c0b0 con 0x7f9938004810 2024-01-27T02:35:53.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.183+0000 7f9943fff700 1 -- 172.21.15.7:0/3019315020 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993800c270 con 0x7f9938003bf0 2024-01-27T02:35:53.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.183+0000 7f99516d9700 1 --2- 172.21.15.7:0/3019315020 >> v2:172.21.15.188:3300/0 conn(0x7f994c0a7bd0 0x7f994c0af6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:35:53.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.183+0000 7f9951eda700 1 --2- 172.21.15.7:0/3019315020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99380051c0 0x7f993800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:53.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.183+0000 7f9950ed8700 1 --2- 172.21.15.7:0/3019315020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9938004810 0x7f9938004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:53.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.183+0000 7f99516d9700 1 --2- 172.21.15.7:0/3019315020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9938003bf0 0x7f9938003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:53.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.183+0000 7f9951eda700 1 -- 172.21.15.7:0/3019315020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9938003bf0 msgr2=0x7f9938003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:53.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.183+0000 7f9951eda700 1 --2- 172.21.15.7:0/3019315020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9938003bf0 0x7f9938003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:53.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.183+0000 7f9951eda700 1 -- 172.21.15.7:0/3019315020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9938004810 msgr2=0x7f9938004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:53.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.183+0000 7f9951eda700 1 --2- 172.21.15.7:0/3019315020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9938004810 0x7f9938004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:53.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.183+0000 7f9951eda700 1 -- 172.21.15.7:0/3019315020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9938003570 con 0x7f99380051c0 2024-01-27T02:35:53.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.183+0000 7f9951eda700 1 -- 172.21.15.7:0/3019315020 reap_dead start 2024-01-27T02:35:53.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.183+0000 7f9951eda700 1 --2- 172.21.15.7:0/3019315020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99380051c0 0x7f993800ba60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f994c0b1ea0 tx=0x7f99440017c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f277b42b59b8e8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:53.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.183+0000 7f9943fff700 1 -- 172.21.15.7:0/3019315020 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9944001ad0 con 0x7f99380051c0 2024-01-27T02:35:53.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.183+0000 7f9943fff700 1 -- 172.21.15.7:0/3019315020 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994400c8b0 con 0x7f99380051c0 2024-01-27T02:35:53.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.183+0000 7f9943fff700 1 -- 172.21.15.7:0/3019315020 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994400cc50 con 0x7f99380051c0 2024-01-27T02:35:53.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.184+0000 7f995393d700 1 -- 172.21.15.7:0/3019315020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99380051c0 msgr2=0x7f993800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:53.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.184+0000 7f995393d700 1 --2- 172.21.15.7:0/3019315020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99380051c0 0x7f993800ba60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f994c0b1ea0 tx=0x7f99440017c0 comp rx=0 tx=0).stop 2024-01-27T02:35:53.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.184+0000 7f995393d700 1 -- 172.21.15.7:0/3019315020 shutdown_connections 2024-01-27T02:35:53.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.184+0000 7f995393d700 1 --2- 172.21.15.7:0/3019315020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99380051c0 0x7f993800ba60 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:53.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.184+0000 7f995393d700 1 -- 172.21.15.7:0/3019315020 >> 172.21.15.7:0/3019315020 conn(0x7f994c0679b0 msgr2=0x7f994c067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:53.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.184+0000 7f995393d700 1 -- 172.21.15.7:0/3019315020 shutdown_connections 2024-01-27T02:35:53.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.184+0000 7f995393d700 1 -- 172.21.15.7:0/3019315020 wait complete. 2024-01-27T02:35:53.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.185+0000 7f995393d700 1 Processor -- start 2024-01-27T02:35:53.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.185+0000 7f995393d700 1 -- start start 2024-01-27T02:35:53.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.186+0000 7f995393d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99380051c0 0x7f994c169f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:53.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.186+0000 7f995393d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f994c1525d0 0x7f994c1421e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:53.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.186+0000 7f995393d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994c1501f0 0x7f994c142720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:53.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.186+0000 7f995393d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c0adea0 con 0x7f994c1501f0 2024-01-27T02:35:53.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.186+0000 7f995393d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c152140 con 0x7f994c1525d0 2024-01-27T02:35:53.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.186+0000 7f995393d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f994c151290 con 0x7f99380051c0 2024-01-27T02:35:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.186+0000 7f9951eda700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994c1501f0 0x7f994c142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.187+0000 7f9951eda700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994c1501f0 0x7f994c142720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42346/0 (socket says 172.21.15.7:42346) 2024-01-27T02:35:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.187+0000 7f9951eda700 1 -- 172.21.15.7:0/3115055207 learned_addr learned my addr 172.21.15.7:0/3115055207 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.187+0000 7f99516d9700 1 --2- 172.21.15.7:0/3115055207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99380051c0 0x7f994c169f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.187+0000 7f9950ed8700 1 --2- 172.21.15.7:0/3115055207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f994c1525d0 0x7f994c1421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.187+0000 7f9951eda700 1 -- 172.21.15.7:0/3115055207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99380051c0 msgr2=0x7f994c169f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.187+0000 7f9951eda700 1 --2- 172.21.15.7:0/3115055207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99380051c0 0x7f994c169f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.187+0000 7f9951eda700 1 -- 172.21.15.7:0/3115055207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f994c1525d0 msgr2=0x7f994c1421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.187+0000 7f9951eda700 1 --2- 172.21.15.7:0/3115055207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f994c1525d0 0x7f994c1421e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.187+0000 7f9951eda700 1 -- 172.21.15.7:0/3115055207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994c151530 con 0x7f994c1501f0 2024-01-27T02:35:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.187+0000 7f9951eda700 1 --2- 172.21.15.7:0/3115055207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994c1501f0 0x7f994c142720 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f994c15abf0 tx=0x7f9944016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.187+0000 7f99417fa700 1 -- 172.21.15.7:0/3115055207 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9944001ad0 con 0x7f994c1501f0 2024-01-27T02:35:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.187+0000 7f99417fa700 1 -- 172.21.15.7:0/3115055207 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994400cea0 con 0x7f994c1501f0 2024-01-27T02:35:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.187+0000 7f995393d700 1 -- 172.21.15.7:0/3115055207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f994c151810 con 0x7f994c1501f0 2024-01-27T02:35:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.187+0000 7f99417fa700 1 -- 172.21.15.7:0/3115055207 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994401e890 con 0x7f994c1501f0 2024-01-27T02:35:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.187+0000 7f995393d700 1 -- 172.21.15.7:0/3115055207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f994c151dd0 con 0x7f994c1501f0 2024-01-27T02:35:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.188+0000 7f995393d700 1 -- 172.21.15.7:0/3115055207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f994c1522f0 con 0x7f994c1501f0 2024-01-27T02:35:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.189+0000 7f99417fa700 1 -- 172.21.15.7:0/3115055207 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9944003030 con 0x7f994c1501f0 2024-01-27T02:35:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.189+0000 7f99417fa700 1 --2- 172.21.15.7:0/3115055207 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9930066f60 0x7f9930069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.189+0000 7f99417fa700 1 -- 172.21.15.7:0/3115055207 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f9944085d70 con 0x7f994c1501f0 2024-01-27T02:35:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.189+0000 7f99516d9700 1 --2- 172.21.15.7:0/3115055207 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9930066f60 0x7f9930069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.190+0000 7f99516d9700 1 --2- 172.21.15.7:0/3115055207 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9930066f60 0x7f9930069410 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9948006f00 tx=0x7f99480091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:53.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.194+0000 7f99417fa700 1 -- 172.21.15.7:0/3115055207 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f994c1522f0 con 0x7f994c1501f0 2024-01-27T02:35:53.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:53.485+0000 7f995393d700 1 -- 172.21.15.7:0/3115055207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) v1 -- 0x7f994c1522b0 con 0x7f994c1501f0 2024-01-27T02:35:54.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.043+0000 7f99417fa700 1 -- 172.21.15.7:0/3115055207 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v200) v1 ==== 134+0+0 (secure 0 0 0) 0x7f9944012e60 con 0x7f994c1501f0 2024-01-27T02:35:54.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.046+0000 7f995393d700 1 -- 172.21.15.7:0/3115055207 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9930066f60 msgr2=0x7f9930069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:54.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.046+0000 7f995393d700 1 --2- 172.21.15.7:0/3115055207 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9930066f60 0x7f9930069410 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9948006f00 tx=0x7f99480091a0 comp rx=0 tx=0).stop 2024-01-27T02:35:54.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.046+0000 7f995393d700 1 -- 172.21.15.7:0/3115055207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994c1501f0 msgr2=0x7f994c142720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:54.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.046+0000 7f995393d700 1 --2- 172.21.15.7:0/3115055207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994c1501f0 0x7f994c142720 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f994c15abf0 tx=0x7f9944016000 comp rx=0 tx=0).stop 2024-01-27T02:35:54.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.046+0000 7f995393d700 1 -- 172.21.15.7:0/3115055207 shutdown_connections 2024-01-27T02:35:54.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.046+0000 7f995393d700 1 --2- 172.21.15.7:0/3115055207 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9930066f60 0x7f9930069410 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:54.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.046+0000 7f995393d700 1 --2- 172.21.15.7:0/3115055207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994c1501f0 0x7f994c142720 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:54.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.046+0000 7f995393d700 1 --2- 172.21.15.7:0/3115055207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99380051c0 0x7f994c169f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:54.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.046+0000 7f995393d700 1 --2- 172.21.15.7:0/3115055207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f994c1525d0 0x7f994c1421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:54.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.046+0000 7f995393d700 1 -- 172.21.15.7:0/3115055207 >> 172.21.15.7:0/3115055207 conn(0x7f994c0679b0 msgr2=0x7f994c098880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:54.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.046+0000 7f995393d700 1 -- 172.21.15.7:0/3115055207 shutdown_connections 2024-01-27T02:35:54.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.046+0000 7f995393d700 1 -- 172.21.15.7:0/3115055207 wait complete. 2024-01-27T02:35:54.062 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestClusterFull) ... ok 2024-01-27T02:35:54.062 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) That when the full flag is encountered during asynchronous 2024-01-27T02:35:54.062 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestClusterFull) 2024-01-27T02:35:54.063 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2024-01-27T02:35:54.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.198+0000 7f1405175700 1 Processor -- start 2024-01-27T02:35:54.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.198+0000 7f1405175700 1 -- start start 2024-01-27T02:35:54.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.199+0000 7f1405175700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1400157ca0 0x7f14000c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:54.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.199+0000 7f1405175700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f14000cdbf0 0x7f14000ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:54.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.199+0000 7f1405175700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14000ca020 con 0x7f14000c3ae0 2024-01-27T02:35:54.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.199+0000 7f1405175700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14000c4f40 con 0x7f14000cdbf0 2024-01-27T02:35:54.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.199+0000 7f1405175700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14000c5210 con 0x7f1400157ca0 2024-01-27T02:35:54.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.199+0000 7f13ff59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f14000cdbf0 0x7f14000ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:54.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.199+0000 7f13ff59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f14000cdbf0 0x7f14000ca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42354/0 (socket says 172.21.15.7:42354) 2024-01-27T02:35:54.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.199+0000 7f13ff59e700 1 -- 172.21.15.7:0/2474637279 learned_addr learned my addr 172.21.15.7:0/2474637279 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:54.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.199+0000 7f13fed9d700 1 --2- 172.21.15.7:0/2474637279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1400157ca0 0x7f14000c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:54.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.199+0000 7f13ff59e700 1 -- 172.21.15.7:0/2474637279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1400157ca0 msgr2=0x7f14000c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:54.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.199+0000 7f13ff59e700 1 --2- 172.21.15.7:0/2474637279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1400157ca0 0x7f14000c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:54.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.199+0000 7f13ff59e700 1 -- 172.21.15.7:0/2474637279 >> v1:172.21.15.188:6789/0 conn(0x7f14000c3ae0 legacy=0x7f14000cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:54.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.200+0000 7f13ff59e700 1 -- 172.21.15.7:0/2474637279 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14000c54f0 con 0x7f14000cdbf0 2024-01-27T02:35:54.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.200+0000 7f13fed9d700 1 --2- 172.21.15.7:0/2474637279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1400157ca0 0x7f14000c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:35:54.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.200+0000 7f13ff59e700 1 --2- 172.21.15.7:0/2474637279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14000cdbf0 0x7f14000ca4b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009f20 tx=0x7f13f8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e243bb4a11f0ad server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:54.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.200+0000 7f13fdd9b700 1 -- 172.21.15.7:0/2474637279 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f800e070 con 0x7f14000cdbf0 2024-01-27T02:35:54.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.200+0000 7f13fdd9b700 1 -- 172.21.15.7:0/2474637279 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f800ba80 con 0x7f14000cdbf0 2024-01-27T02:35:54.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.200+0000 7f13fdd9b700 1 -- 172.21.15.7:0/2474637279 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f80092e0 con 0x7f14000cdbf0 2024-01-27T02:35:54.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.200+0000 7f1405175700 1 -- 172.21.15.7:0/2474637279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14000cdbf0 msgr2=0x7f14000ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:54.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.200+0000 7f1405175700 1 --2- 172.21.15.7:0/2474637279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14000cdbf0 0x7f14000ca4b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009f20 tx=0x7f13f8009840 comp rx=0 tx=0).stop 2024-01-27T02:35:54.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.201+0000 7f1405175700 1 -- 172.21.15.7:0/2474637279 shutdown_connections 2024-01-27T02:35:54.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.201+0000 7f1405175700 1 --2- 172.21.15.7:0/2474637279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14000cdbf0 0x7f14000ca4b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:54.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.201+0000 7f1405175700 1 --2- 172.21.15.7:0/2474637279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1400157ca0 0x7f14000c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:54.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.201+0000 7f1405175700 1 -- 172.21.15.7:0/2474637279 >> 172.21.15.7:0/2474637279 conn(0x7f1400098020 msgr2=0x7f14000711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:54.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.201+0000 7f1405175700 1 -- 172.21.15.7:0/2474637279 shutdown_connections 2024-01-27T02:35:54.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.201+0000 7f1405175700 1 -- 172.21.15.7:0/2474637279 wait complete. 2024-01-27T02:35:54.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.202+0000 7f1405175700 1 Processor -- start 2024-01-27T02:35:54.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.202+0000 7f1405175700 1 -- start start 2024-01-27T02:35:54.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.203+0000 7f1405175700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14000c3ae0 0x7f140014e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:54.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.203+0000 7f1405175700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14000cdbf0 0x7f14000c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:54.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.203+0000 7f1405175700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1400157ca0 0x7f14000c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:54.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.203+0000 7f1405175700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140014eae0 con 0x7f14000c3ae0 2024-01-27T02:35:54.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.203+0000 7f1405175700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14000b28a0 con 0x7f1400157ca0 2024-01-27T02:35:54.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.203+0000 7f1405175700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14000b22c0 con 0x7f14000cdbf0 2024-01-27T02:35:54.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.203+0000 7f13fed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14000c3ae0 0x7f140014e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:54.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.203+0000 7f13fed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14000c3ae0 0x7f140014e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42368/0 (socket says 172.21.15.7:42368) 2024-01-27T02:35:54.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.203+0000 7f13fed9d700 1 -- 172.21.15.7:0/1267378430 learned_addr learned my addr 172.21.15.7:0/1267378430 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:54.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.203+0000 7f13ff59e700 1 --2- 172.21.15.7:0/1267378430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1400157ca0 0x7f14000c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:54.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.203+0000 7f13fe59c700 1 --2- 172.21.15.7:0/1267378430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14000cdbf0 0x7f14000c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:54.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.203+0000 7f13fed9d700 1 -- 172.21.15.7:0/1267378430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14000cdbf0 msgr2=0x7f14000c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:54.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.203+0000 7f13fed9d700 1 --2- 172.21.15.7:0/1267378430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14000cdbf0 0x7f14000c7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:54.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.203+0000 7f13fed9d700 1 -- 172.21.15.7:0/1267378430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1400157ca0 msgr2=0x7f14000c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:54.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.203+0000 7f13fed9d700 1 --2- 172.21.15.7:0/1267378430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1400157ca0 0x7f14000c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:54.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.203+0000 7f13fed9d700 1 -- 172.21.15.7:0/1267378430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f8009ae0 con 0x7f14000c3ae0 2024-01-27T02:35:54.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.204+0000 7f13fed9d700 1 --2- 172.21.15.7:0/1267378430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14000c3ae0 0x7f140014e5a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f13f000c8b0 tx=0x7f13f0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:54.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.204+0000 7f13f6ffd700 1 -- 172.21.15.7:0/1267378430 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f0011600 con 0x7f14000c3ae0 2024-01-27T02:35:54.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.204+0000 7f13f6ffd700 1 -- 172.21.15.7:0/1267378430 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f0011760 con 0x7f14000c3ae0 2024-01-27T02:35:54.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.204+0000 7f1405175700 1 -- 172.21.15.7:0/1267378430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f140014efa0 con 0x7f14000c3ae0 2024-01-27T02:35:54.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.204+0000 7f13f6ffd700 1 -- 172.21.15.7:0/1267378430 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f0011b00 con 0x7f14000c3ae0 2024-01-27T02:35:54.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.204+0000 7f1405175700 1 -- 172.21.15.7:0/1267378430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14000b25b0 con 0x7f14000c3ae0 2024-01-27T02:35:54.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.205+0000 7f1405175700 1 -- 172.21.15.7:0/1267378430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f140004c710 con 0x7f14000c3ae0 2024-01-27T02:35:54.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.206+0000 7f13f6ffd700 1 -- 172.21.15.7:0/1267378430 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f13f0023480 con 0x7f14000c3ae0 2024-01-27T02:35:54.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.206+0000 7f13f6ffd700 1 --2- 172.21.15.7:0/1267378430 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f13e0066f30 0x7f13e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:54.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.206+0000 7f13f6ffd700 1 -- 172.21.15.7:0/1267378430 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f13f00845c0 con 0x7f14000c3ae0 2024-01-27T02:35:54.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.206+0000 7f13fe59c700 1 --2- 172.21.15.7:0/1267378430 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f13e0066f30 0x7f13e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:54.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.207+0000 7f13fe59c700 1 --2- 172.21.15.7:0/1267378430 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f13e0066f30 0x7f13e00693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f13e8003860 tx=0x7f13e8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:54.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.210+0000 7f13f6ffd700 1 -- 172.21.15.7:0/1267378430 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f13f0055640 con 0x7f14000c3ae0 2024-01-27T02:35:54.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:54.498+0000 7f1405175700 1 -- 172.21.15.7:0/1267378430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) v1 -- 0x7f1400154770 con 0x7f14000c3ae0 2024-01-27T02:35:55.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.044+0000 7f13f6ffd700 1 -- 172.21.15.7:0/1267378430 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v201) v1 ==== 136+0+0 (secure 0 0 0) 0x7f13f00551d0 con 0x7f14000c3ae0 2024-01-27T02:35:55.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.047+0000 7f1405175700 1 -- 172.21.15.7:0/1267378430 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f13e0066f30 msgr2=0x7f13e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:55.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.047+0000 7f1405175700 1 --2- 172.21.15.7:0/1267378430 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f13e0066f30 0x7f13e00693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f13e8003860 tx=0x7f13e8009040 comp rx=0 tx=0).stop 2024-01-27T02:35:55.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.047+0000 7f1405175700 1 -- 172.21.15.7:0/1267378430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14000c3ae0 msgr2=0x7f140014e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:55.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.047+0000 7f1405175700 1 --2- 172.21.15.7:0/1267378430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14000c3ae0 0x7f140014e5a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f13f000c8b0 tx=0x7f13f0008040 comp rx=0 tx=0).stop 2024-01-27T02:35:55.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.047+0000 7f1405175700 1 -- 172.21.15.7:0/1267378430 shutdown_connections 2024-01-27T02:35:55.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.047+0000 7f1405175700 1 --2- 172.21.15.7:0/1267378430 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f13e0066f30 0x7f13e00693e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:55.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.047+0000 7f1405175700 1 --2- 172.21.15.7:0/1267378430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1400157ca0 0x7f14000c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:55.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.047+0000 7f1405175700 1 --2- 172.21.15.7:0/1267378430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14000c3ae0 0x7f140014e5a0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:55.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.047+0000 7f1405175700 1 --2- 172.21.15.7:0/1267378430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14000cdbf0 0x7f14000c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:55.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.047+0000 7f1405175700 1 -- 172.21.15.7:0/1267378430 >> 172.21.15.7:0/1267378430 conn(0x7f1400098020 msgr2=0x7f140006c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:55.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.048+0000 7f1405175700 1 -- 172.21.15.7:0/1267378430 shutdown_connections 2024-01-27T02:35:55.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.048+0000 7f1405175700 1 -- 172.21.15.7:0/1267378430 wait complete. 2024-01-27T02:35:55.064 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-01-27T02:35:55.216 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_objectstore":"memstore"} 2024-01-27T02:35:55.216 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-01-27T02:35:55.216 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-27T02:35:55.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.350+0000 7f2f5adf5700 1 Processor -- start 2024-01-27T02:35:55.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.350+0000 7f2f5adf5700 1 -- start start 2024-01-27T02:35:55.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.351+0000 7f2f5adf5700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2f54154550 0x7f2f540ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:55.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.351+0000 7f2f5adf5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f540adec0 con 0x7f2f54154920 2024-01-27T02:35:55.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.351+0000 7f2f5adf5700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f540ae1a0 con 0x7f2f540ad7c0 2024-01-27T02:35:55.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.351+0000 7f2f5adf5700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f5415a850 con 0x7f2f54154550 2024-01-27T02:35:55.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.352+0000 7f2f53fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2f54154920 0x7f2f5415b050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34228/0 (socket says 172.21.15.7:34228) 2024-01-27T02:35:55.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.352+0000 7f2f53fff700 1 -- 172.21.15.7:0/2843028894 learned_addr learned my addr 172.21.15.7:0/2843028894 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:35:55.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.352+0000 7f2f58b91700 1 --2- 172.21.15.7:0/2843028894 >> v2:172.21.15.188:3300/0 conn(0x7f2f54154550 0x7f2f540ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:55.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.352+0000 7f2f537fe700 1 -- 172.21.15.7:0/2843028894 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746653665 0 0) 0x7f2f540adec0 con 0x7f2f54154920 2024-01-27T02:35:55.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.352+0000 7f2f537fe700 1 -- 172.21.15.7:0/2843028894 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f44003570 con 0x7f2f54154920 2024-01-27T02:35:55.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.352+0000 7f2f537fe700 1 -- 172.21.15.7:0/2843028894 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001927742 0 0) 0x7f2f540ae1a0 con 0x7f2f540ad7c0 2024-01-27T02:35:55.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.352+0000 7f2f537fe700 1 -- 172.21.15.7:0/2843028894 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f540adec0 con 0x7f2f540ad7c0 2024-01-27T02:35:55.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.352+0000 7f2f58b91700 1 -- 172.21.15.7:0/2843028894 >> v1:172.21.15.188:6789/0 conn(0x7f2f540ad7c0 legacy=0x7f2f54063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:55.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.352+0000 7f2f537fe700 1 -- 172.21.15.7:0/2843028894 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 534428600 0 0) 0x7f2f44003570 con 0x7f2f54154920 2024-01-27T02:35:55.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.352+0000 7f2f58b91700 1 -- 172.21.15.7:0/2843028894 >> v1:172.21.15.7:6789/0 conn(0x7f2f54154920 legacy=0x7f2f5415b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:55.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.352+0000 7f2f58b91700 1 -- 172.21.15.7:0/2843028894 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5415ab30 con 0x7f2f54154550 2024-01-27T02:35:55.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.352+0000 7f2f58b91700 1 --2- 172.21.15.7:0/2843028894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f54154550 0x7f2f540ce6b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2f480098e0 tx=0x7f2f480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b42544c124f768c3 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:55.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.353+0000 7f2f537fe700 1 -- 172.21.15.7:0/2843028894 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40001470 con 0x7f2f54154550 2024-01-27T02:35:55.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.353+0000 7f2f537fe700 1 -- 172.21.15.7:0/2843028894 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f48011420 con 0x7f2f54154550 2024-01-27T02:35:55.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.353+0000 7f2f537fe700 1 -- 172.21.15.7:0/2843028894 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f480117c0 con 0x7f2f54154550 2024-01-27T02:35:55.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.353+0000 7f2f5adf5700 1 -- 172.21.15.7:0/2843028894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f54154550 msgr2=0x7f2f540ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:55.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.353+0000 7f2f5adf5700 1 --2- 172.21.15.7:0/2843028894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f54154550 0x7f2f540ce6b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2f480098e0 tx=0x7f2f480091a0 comp rx=0 tx=0).stop 2024-01-27T02:35:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.353+0000 7f2f5adf5700 1 -- 172.21.15.7:0/2843028894 shutdown_connections 2024-01-27T02:35:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.353+0000 7f2f5adf5700 1 --2- 172.21.15.7:0/2843028894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f54154550 0x7f2f540ce6b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.353+0000 7f2f5adf5700 1 -- 172.21.15.7:0/2843028894 >> 172.21.15.7:0/2843028894 conn(0x7f2f54068480 msgr2=0x7f2f540cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.353+0000 7f2f5adf5700 1 -- 172.21.15.7:0/2843028894 shutdown_connections 2024-01-27T02:35:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.353+0000 7f2f5adf5700 1 -- 172.21.15.7:0/2843028894 wait complete. 2024-01-27T02:35:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.355+0000 7f2f5adf5700 1 Processor -- start 2024-01-27T02:35:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.355+0000 7f2f5adf5700 1 -- start start 2024-01-27T02:35:55.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.356+0000 7f2f5adf5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f540ad7c0 0x7f2f54151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:55.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.356+0000 7f2f5adf5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f54154550 0x7f2f54156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:55.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.356+0000 7f2f5adf5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f54154920 0x7f2f540637d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:55.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.356+0000 7f2f5adf5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f54157700 con 0x7f2f540ad7c0 2024-01-27T02:35:55.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.356+0000 7f2f5adf5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f54156c70 con 0x7f2f54154920 2024-01-27T02:35:55.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.356+0000 7f2f5adf5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5416e130 con 0x7f2f54154550 2024-01-27T02:35:55.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.356+0000 7f2f58b91700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f540ad7c0 0x7f2f54151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.356+0000 7f2f58b91700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f540ad7c0 0x7f2f54151890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42382/0 (socket says 172.21.15.7:42382) 2024-01-27T02:35:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.356+0000 7f2f58b91700 1 -- 172.21.15.7:0/2284474021 learned_addr learned my addr 172.21.15.7:0/2284474021 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.357+0000 7f2f59392700 1 --2- 172.21.15.7:0/2284474021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f54154920 0x7f2f540637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.357+0000 7f2f53fff700 1 --2- 172.21.15.7:0/2284474021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f54154550 0x7f2f54156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.357+0000 7f2f58b91700 1 -- 172.21.15.7:0/2284474021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f54154550 msgr2=0x7f2f54156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.357+0000 7f2f58b91700 1 --2- 172.21.15.7:0/2284474021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f54154550 0x7f2f54156730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.357+0000 7f2f58b91700 1 -- 172.21.15.7:0/2284474021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f54154920 msgr2=0x7f2f540637d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.357+0000 7f2f58b91700 1 --2- 172.21.15.7:0/2284474021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f54154920 0x7f2f540637d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.357+0000 7f2f58b91700 1 -- 172.21.15.7:0/2284474021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f48009420 con 0x7f2f540ad7c0 2024-01-27T02:35:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.357+0000 7f2f59392700 1 --2- 172.21.15.7:0/2284474021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f54154920 0x7f2f540637d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:35:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.357+0000 7f2f53fff700 1 --2- 172.21.15.7:0/2284474021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f54154550 0x7f2f54156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:35:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.357+0000 7f2f58b91700 1 --2- 172.21.15.7:0/2284474021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f540ad7c0 0x7f2f54151890 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2f540700c0 tx=0x7f2f48016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.357+0000 7f2f50ff9700 1 -- 172.21.15.7:0/2284474021 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4801cc90 con 0x7f2f540ad7c0 2024-01-27T02:35:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.357+0000 7f2f50ff9700 1 -- 172.21.15.7:0/2284474021 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4801cdf0 con 0x7f2f540ad7c0 2024-01-27T02:35:55.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.357+0000 7f2f5adf5700 1 -- 172.21.15.7:0/2284474021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f54156e60 con 0x7f2f540ad7c0 2024-01-27T02:35:55.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.357+0000 7f2f50ff9700 1 -- 172.21.15.7:0/2284474021 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f48025680 con 0x7f2f540ad7c0 2024-01-27T02:35:55.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.357+0000 7f2f5adf5700 1 -- 172.21.15.7:0/2284474021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f54157420 con 0x7f2f540ad7c0 2024-01-27T02:35:55.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.358+0000 7f2f5adf5700 1 -- 172.21.15.7:0/2284474021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f54072060 con 0x7f2f540ad7c0 2024-01-27T02:35:55.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.359+0000 7f2f50ff9700 1 -- 172.21.15.7:0/2284474021 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2f480193f0 con 0x7f2f540ad7c0 2024-01-27T02:35:55.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.359+0000 7f2f50ff9700 1 --2- 172.21.15.7:0/2284474021 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2f38066f30 0x7f2f380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:55.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.359+0000 7f2f50ff9700 1 -- 172.21.15.7:0/2284474021 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2f48085640 con 0x7f2f540ad7c0 2024-01-27T02:35:55.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.360+0000 7f2f53fff700 1 --2- 172.21.15.7:0/2284474021 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2f38066f30 0x7f2f380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:55.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.360+0000 7f2f53fff700 1 --2- 172.21.15.7:0/2284474021 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2f38066f30 0x7f2f380693e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2f5415a100 tx=0x7f2f40009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:55.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.364+0000 7f2f50ff9700 1 -- 172.21.15.7:0/2284474021 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2f48052b10 con 0x7f2f540ad7c0 2024-01-27T02:35:55.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.569+0000 7f2f5adf5700 1 -- 172.21.15.7:0/2284474021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2f5406f780 con 0x7f2f540ad7c0 2024-01-27T02:35:55.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.573+0000 7f2f50ff9700 1 -- 172.21.15.7:0/2284474021 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v8)=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2f48056130 con 0x7f2f540ad7c0 2024-01-27T02:35:55.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.575+0000 7f2f5adf5700 1 -- 172.21.15.7:0/2284474021 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2f38066f30 msgr2=0x7f2f380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:55.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.575+0000 7f2f5adf5700 1 --2- 172.21.15.7:0/2284474021 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2f38066f30 0x7f2f380693e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2f5415a100 tx=0x7f2f40009040 comp rx=0 tx=0).stop 2024-01-27T02:35:55.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.575+0000 7f2f5adf5700 1 -- 172.21.15.7:0/2284474021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f540ad7c0 msgr2=0x7f2f54151890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:55.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.575+0000 7f2f5adf5700 1 --2- 172.21.15.7:0/2284474021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f540ad7c0 0x7f2f54151890 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2f540700c0 tx=0x7f2f48016000 comp rx=0 tx=0).stop 2024-01-27T02:35:55.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.575+0000 7f2f5adf5700 1 -- 172.21.15.7:0/2284474021 shutdown_connections 2024-01-27T02:35:55.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.575+0000 7f2f5adf5700 1 --2- 172.21.15.7:0/2284474021 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2f38066f30 0x7f2f380693e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:55.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.575+0000 7f2f5adf5700 1 --2- 172.21.15.7:0/2284474021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f54154920 0x7f2f540637d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:55.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.575+0000 7f2f5adf5700 1 --2- 172.21.15.7:0/2284474021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f540ad7c0 0x7f2f54151890 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:55.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.575+0000 7f2f5adf5700 1 --2- 172.21.15.7:0/2284474021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f54154550 0x7f2f54156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:55.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.575+0000 7f2f5adf5700 1 -- 172.21.15.7:0/2284474021 >> 172.21.15.7:0/2284474021 conn(0x7f2f54068480 msgr2=0x7f2f5406ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:55.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.576+0000 7f2f5adf5700 1 -- 172.21.15.7:0/2284474021 shutdown_connections 2024-01-27T02:35:55.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.576+0000 7f2f5adf5700 1 -- 172.21.15.7:0/2284474021 wait complete. 2024-01-27T02:35:55.589 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-27T02:35:55.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:35:55.634 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:35:55.680 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-27T02:35:55.681 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-27T02:35:55.681 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:35:55.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.817+0000 7f30a53b7700 1 Processor -- start 2024-01-27T02:35:55.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.817+0000 7f30a53b7700 1 -- start start 2024-01-27T02:35:55.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.818+0000 7f30a53b7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f30a00b2ff0 0x7f30a00b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:55.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.818+0000 7f30a53b7700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f30a015b8e0 0x7f30a00ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:55.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.818+0000 7f30a53b7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a00b3900 con 0x7f30a015ddd0 2024-01-27T02:35:55.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.818+0000 7f30a53b7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a00ce7e0 con 0x7f30a00b2ff0 2024-01-27T02:35:55.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.818+0000 7f30a53b7700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a015ebe0 con 0x7f30a015b8e0 2024-01-27T02:35:55.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.818+0000 7f309effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f30a00b2ff0 0x7f30a00b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:55.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.818+0000 7f309f7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f30a015ddd0 0x7f30a015ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34232/0 (socket says 172.21.15.7:34232) 2024-01-27T02:35:55.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.818+0000 7f309f7fe700 1 -- 172.21.15.7:0/2385931376 learned_addr learned my addr 172.21.15.7:0/2385931376 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:35:55.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.818+0000 7f309e7fc700 1 --2- 172.21.15.7:0/2385931376 >> v2:172.21.15.188:3300/0 conn(0x7f30a015b8e0 0x7f30a00ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:55.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.818+0000 7f309dffb700 1 -- 172.21.15.7:0/2385931376 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1763716247 0 0) 0x7f30a00b3900 con 0x7f30a015ddd0 2024-01-27T02:35:55.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.818+0000 7f309dffb700 1 -- 172.21.15.7:0/2385931376 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f308c003570 con 0x7f30a015ddd0 2024-01-27T02:35:55.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.818+0000 7f309effd700 1 -- 172.21.15.7:0/2385931376 >> v2:172.21.15.188:3300/0 conn(0x7f30a015b8e0 msgr2=0x7f30a00ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:55.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.818+0000 7f309effd700 1 --2- 172.21.15.7:0/2385931376 >> v2:172.21.15.188:3300/0 conn(0x7f30a015b8e0 0x7f30a00ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:55.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.818+0000 7f309effd700 1 -- 172.21.15.7:0/2385931376 >> v1:172.21.15.7:6789/0 conn(0x7f30a015ddd0 legacy=0x7f30a015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:55.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.819+0000 7f309effd700 1 -- 172.21.15.7:0/2385931376 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a01587b0 con 0x7f30a00b2ff0 2024-01-27T02:35:55.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.819+0000 7f309effd700 1 --2- 172.21.15.7:0/2385931376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30a00b2ff0 0x7f30a00b33c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f30a0154d80 tx=0x7f3090009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724aaab2bef0ae39 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:55.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.819+0000 7f309dffb700 1 -- 172.21.15.7:0/2385931376 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090005c10 con 0x7f30a00b2ff0 2024-01-27T02:35:55.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.819+0000 7f309dffb700 1 -- 172.21.15.7:0/2385931376 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3090005d70 con 0x7f30a00b2ff0 2024-01-27T02:35:55.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.819+0000 7f309dffb700 1 -- 172.21.15.7:0/2385931376 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090006260 con 0x7f30a00b2ff0 2024-01-27T02:35:55.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.819+0000 7f30a53b7700 1 -- 172.21.15.7:0/2385931376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30a00b2ff0 msgr2=0x7f30a00b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:55.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.819+0000 7f30a53b7700 1 --2- 172.21.15.7:0/2385931376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30a00b2ff0 0x7f30a00b33c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f30a0154d80 tx=0x7f3090009120 comp rx=0 tx=0).stop 2024-01-27T02:35:55.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.820+0000 7f30a53b7700 1 -- 172.21.15.7:0/2385931376 shutdown_connections 2024-01-27T02:35:55.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.820+0000 7f30a53b7700 1 --2- 172.21.15.7:0/2385931376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30a00b2ff0 0x7f30a00b33c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:55.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.820+0000 7f30a53b7700 1 --2- 172.21.15.7:0/2385931376 >> v2:172.21.15.188:3300/0 conn(0x7f30a015b8e0 0x7f30a00ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:55.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.820+0000 7f30a53b7700 1 -- 172.21.15.7:0/2385931376 >> 172.21.15.7:0/2385931376 conn(0x7f30a00679b0 msgr2=0x7f30a0067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:55.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.820+0000 7f30a53b7700 1 -- 172.21.15.7:0/2385931376 shutdown_connections 2024-01-27T02:35:55.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.820+0000 7f30a53b7700 1 -- 172.21.15.7:0/2385931376 wait complete. 2024-01-27T02:35:55.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.821+0000 7f30a53b7700 1 Processor -- start 2024-01-27T02:35:55.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.821+0000 7f30a53b7700 1 -- start start 2024-01-27T02:35:55.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.822+0000 7f30a53b7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30a00b2ff0 0x7f30a0156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:55.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.822+0000 7f30a53b7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30a015b8e0 0x7f30a015a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:55.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.822+0000 7f30a53b7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30a015ddd0 0x7f30a0150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:55.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.822+0000 7f30a53b7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a0150cc0 con 0x7f30a015b8e0 2024-01-27T02:35:55.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.822+0000 7f30a53b7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a016d930 con 0x7f30a00b2ff0 2024-01-27T02:35:55.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.822+0000 7f30a53b7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a016dc10 con 0x7f30a015ddd0 2024-01-27T02:35:55.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.822+0000 7f309e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30a015b8e0 0x7f30a015a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:55.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.822+0000 7f309e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30a015b8e0 0x7f30a015a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42410/0 (socket says 172.21.15.7:42410) 2024-01-27T02:35:55.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.822+0000 7f309e7fc700 1 -- 172.21.15.7:0/2680561845 learned_addr learned my addr 172.21.15.7:0/2680561845 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:55.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.822+0000 7f309effd700 1 --2- 172.21.15.7:0/2680561845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30a00b2ff0 0x7f30a0156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:55.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.822+0000 7f309f7fe700 1 --2- 172.21.15.7:0/2680561845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30a015ddd0 0x7f30a0150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:55.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.822+0000 7f309e7fc700 1 -- 172.21.15.7:0/2680561845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30a015ddd0 msgr2=0x7f30a0150780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:55.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.822+0000 7f309e7fc700 1 --2- 172.21.15.7:0/2680561845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30a015ddd0 0x7f30a0150780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:55.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.822+0000 7f309e7fc700 1 -- 172.21.15.7:0/2680561845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30a00b2ff0 msgr2=0x7f30a0156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:55.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.822+0000 7f309e7fc700 1 --2- 172.21.15.7:0/2680561845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30a00b2ff0 0x7f30a0156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:55.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.822+0000 7f309e7fc700 1 -- 172.21.15.7:0/2680561845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30900093c0 con 0x7f30a015b8e0 2024-01-27T02:35:55.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.822+0000 7f309e7fc700 1 --2- 172.21.15.7:0/2680561845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30a015b8e0 0x7f30a015a7d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f308800d860 tx=0x7f3088008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:55.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.823+0000 7f3096ffd700 1 -- 172.21.15.7:0/2680561845 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088018070 con 0x7f30a015b8e0 2024-01-27T02:35:55.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.823+0000 7f3096ffd700 1 -- 172.21.15.7:0/2680561845 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30880125f0 con 0x7f30a015b8e0 2024-01-27T02:35:55.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.823+0000 7f30a53b7700 1 -- 172.21.15.7:0/2680561845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a016def0 con 0x7f30a015b8e0 2024-01-27T02:35:55.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.823+0000 7f3096ffd700 1 -- 172.21.15.7:0/2680561845 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30880129b0 con 0x7f30a015b8e0 2024-01-27T02:35:55.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.823+0000 7f30a53b7700 1 -- 172.21.15.7:0/2680561845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a015a270 con 0x7f30a015b8e0 2024-01-27T02:35:55.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.824+0000 7f30a53b7700 1 -- 172.21.15.7:0/2680561845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a016e390 con 0x7f30a015b8e0 2024-01-27T02:35:55.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.824+0000 7f3096ffd700 1 -- 172.21.15.7:0/2680561845 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3088012d20 con 0x7f30a015b8e0 2024-01-27T02:35:55.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.824+0000 7f3096ffd700 1 --2- 172.21.15.7:0/2680561845 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3080066f30 0x7f30800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:55.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.825+0000 7f3096ffd700 1 -- 172.21.15.7:0/2680561845 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f3088084500 con 0x7f30a015b8e0 2024-01-27T02:35:55.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.825+0000 7f309effd700 1 --2- 172.21.15.7:0/2680561845 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3080066f30 0x7f30800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:55.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.825+0000 7f309effd700 1 --2- 172.21.15.7:0/2680561845 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3080066f30 0x7f30800693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3090009160 tx=0x7f3090016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:55.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:55.829+0000 7f3096ffd700 1 -- 172.21.15.7:0/2680561845 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3088055500 con 0x7f30a015b8e0 2024-01-27T02:35:56.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.079+0000 7f30a53b7700 1 -- 172.21.15.7:0/2680561845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30a004c710 con 0x7f30a015b8e0 2024-01-27T02:35:56.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.079+0000 7f3096ffd700 1 -- 172.21.15.7:0/2680561845 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f3088055090 con 0x7f30a015b8e0 2024-01-27T02:35:56.080 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:35:56.080 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5114,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/2839704447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":2839704447},{"type":"v1","addr":"172.21.15.188:6837","nonce":2839704447}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5117,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3964291855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3964291855},{"type":"v1","addr":"172.21.15.7:6834","nonce":3964291855}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/3332325272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3332325272},{"type":"v1","addr":"172.21.15.188:6836","nonce":3332325272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5877,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3297236842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3297236842},{"type":"v1","addr":"172.21.15.7:6835","nonce":3297236842}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[]} 2024-01-27T02:35:56.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.082+0000 7f30a53b7700 1 -- 172.21.15.7:0/2680561845 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3080066f30 msgr2=0x7f30800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:56.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.082+0000 7f30a53b7700 1 --2- 172.21.15.7:0/2680561845 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3080066f30 0x7f30800693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3090009160 tx=0x7f3090016000 comp rx=0 tx=0).stop 2024-01-27T02:35:56.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.082+0000 7f30a53b7700 1 -- 172.21.15.7:0/2680561845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30a015b8e0 msgr2=0x7f30a015a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:56.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.082+0000 7f30a53b7700 1 --2- 172.21.15.7:0/2680561845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30a015b8e0 0x7f30a015a7d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f308800d860 tx=0x7f3088008040 comp rx=0 tx=0).stop 2024-01-27T02:35:56.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.082+0000 7f30a53b7700 1 -- 172.21.15.7:0/2680561845 shutdown_connections 2024-01-27T02:35:56.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.082+0000 7f30a53b7700 1 --2- 172.21.15.7:0/2680561845 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3080066f30 0x7f30800693e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:56.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.082+0000 7f30a53b7700 1 --2- 172.21.15.7:0/2680561845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30a015ddd0 0x7f30a0150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:56.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.082+0000 7f30a53b7700 1 --2- 172.21.15.7:0/2680561845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30a00b2ff0 0x7f30a0156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:56.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.082+0000 7f30a53b7700 1 --2- 172.21.15.7:0/2680561845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30a015b8e0 0x7f30a015a7d0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:56.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.082+0000 7f30a53b7700 1 -- 172.21.15.7:0/2680561845 >> 172.21.15.7:0/2680561845 conn(0x7f30a00679b0 msgr2=0x7f30a015cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:56.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.082+0000 7f30a53b7700 1 -- 172.21.15.7:0/2680561845 shutdown_connections 2024-01-27T02:35:56.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.082+0000 7f30a53b7700 1 -- 172.21.15.7:0/2680561845 wait complete. 2024-01-27T02:35:56.084 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-01-27T02:35:56.097 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-27T02:35:56.097 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-27T02:35:56.097 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-27T02:35:56.097 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:35:56.097 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-27T02:35:56.097 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-27T02:35:56.098 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-27T02:35:56.098 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:35:56.098 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-27T02:35:56.098 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-27T02:35:56.098 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-27T02:35:56.098 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:35:56.098 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-27T02:35:56.098 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-27T02:35:56.099 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-27T02:35:56.099 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:35:56.107 INFO:tasks.ceph.mds.c:Stopped 2024-01-27T02:35:56.108 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-27T02:35:56.109 INFO:tasks.ceph.mds.a:Stopped 2024-01-27T02:35:56.109 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-27T02:35:56.111 INFO:tasks.ceph.mds.d:Stopped 2024-01-27T02:35:56.111 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-27T02:35:56.112 INFO:tasks.ceph.mds.b:Stopped 2024-01-27T02:35:56.113 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-27T02:35:56.150 INFO:tasks.ceph.mds.c:Started 2024-01-27T02:35:56.150 DEBUG:teuthology.parallel:result is None 2024-01-27T02:35:56.151 INFO:tasks.ceph.mds.a:Started 2024-01-27T02:35:56.151 DEBUG:teuthology.parallel:result is None 2024-01-27T02:35:56.153 INFO:tasks.ceph.mds.d:Started 2024-01-27T02:35:56.154 DEBUG:teuthology.parallel:result is None 2024-01-27T02:35:56.154 INFO:tasks.ceph.mds.b:Started 2024-01-27T02:35:56.154 DEBUG:teuthology.parallel:result is None 2024-01-27T02:35:56.155 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-27T02:35:56.236 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-27T02:35:56.237 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-27T02:35:56.240 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-01-27T02:35:56.241 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-01-27T02:35:56.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.293+0000 7f55d44a4700 1 Processor -- start 2024-01-27T02:35:56.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.294+0000 7f55d44a4700 1 -- start start 2024-01-27T02:35:56.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.295+0000 7f55d44a4700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f55cc06fa70 0x7f55cc165280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:56.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.295+0000 7f55d44a4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55cc15a880 con 0x7f55cc0a7c10 2024-01-27T02:35:56.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.295+0000 7f55d44a4700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55cc0af950 con 0x7f55cc0a7fe0 2024-01-27T02:35:56.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.295+0000 7f55d44a4700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55cc0afde0 con 0x7f55cc06fa70 2024-01-27T02:35:56.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.295+0000 7f55d2240700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f55cc0a7c10 0x7f55cc0ae340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34246/0 (socket says 172.21.15.7:34246) 2024-01-27T02:35:56.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.295+0000 7f55d2240700 1 -- 172.21.15.7:0/1626902881 learned_addr learned my addr 172.21.15.7:0/1626902881 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:35:56.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.295+0000 7f55d2a41700 1 --2- 172.21.15.7:0/1626902881 >> v2:172.21.15.188:3300/0 conn(0x7f55cc06fa70 0x7f55cc165280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:56.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.295+0000 7f55d123e700 1 -- 172.21.15.7:0/1626902881 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2213093524 0 0) 0x7f55cc15a880 con 0x7f55cc0a7c10 2024-01-27T02:35:56.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.295+0000 7f55d123e700 1 -- 172.21.15.7:0/1626902881 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55b8003570 con 0x7f55cc0a7c10 2024-01-27T02:35:56.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.295+0000 7f55d2a41700 1 -- 172.21.15.7:0/1626902881 >> v1:172.21.15.188:6789/0 conn(0x7f55cc0a7fe0 legacy=0x7f55cc06f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:56.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.295+0000 7f55d2a41700 1 -- 172.21.15.7:0/1626902881 >> v1:172.21.15.7:6789/0 conn(0x7f55cc0a7c10 legacy=0x7f55cc0ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:56.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.295+0000 7f55d2a41700 1 -- 172.21.15.7:0/1626902881 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc0aed40 con 0x7f55cc06fa70 2024-01-27T02:35:56.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.295+0000 7f55d2a41700 1 --2- 172.21.15.7:0/1626902881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55cc06fa70 0x7f55cc165280 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009e60 tx=0x7f55c4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fcebfd17dec0956 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:56.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.296+0000 7f55d123e700 1 -- 172.21.15.7:0/1626902881 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c400e070 con 0x7f55cc06fa70 2024-01-27T02:35:56.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.296+0000 7f55d123e700 1 -- 172.21.15.7:0/1626902881 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c400bb60 con 0x7f55cc06fa70 2024-01-27T02:35:56.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.296+0000 7f55d123e700 1 -- 172.21.15.7:0/1626902881 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c4012430 con 0x7f55cc06fa70 2024-01-27T02:35:56.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.296+0000 7f55d44a4700 1 -- 172.21.15.7:0/1626902881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55cc06fa70 msgr2=0x7f55cc165280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:56.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.296+0000 7f55d44a4700 1 --2- 172.21.15.7:0/1626902881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55cc06fa70 0x7f55cc165280 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009e60 tx=0x7f55c4009780 comp rx=0 tx=0).stop 2024-01-27T02:35:56.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.296+0000 7f55d44a4700 1 -- 172.21.15.7:0/1626902881 shutdown_connections 2024-01-27T02:35:56.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.296+0000 7f55d44a4700 1 --2- 172.21.15.7:0/1626902881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55cc06fa70 0x7f55cc165280 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:56.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.296+0000 7f55d44a4700 1 -- 172.21.15.7:0/1626902881 >> 172.21.15.7:0/1626902881 conn(0x7f55cc0679b0 msgr2=0x7f55cc067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:56.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.296+0000 7f55d44a4700 1 -- 172.21.15.7:0/1626902881 shutdown_connections 2024-01-27T02:35:56.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.296+0000 7f55d44a4700 1 -- 172.21.15.7:0/1626902881 wait complete. 2024-01-27T02:35:56.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.298+0000 7f55d44a4700 1 Processor -- start 2024-01-27T02:35:56.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.298+0000 7f55d44a4700 1 -- start start 2024-01-27T02:35:56.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.299+0000 7f55d44a4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55cc06fa70 0x7f55cc146220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:56.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.299+0000 7f55d44a4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc0a7c10 0x7f55cc15d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:56.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.299+0000 7f55d44a4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55cc0a7fe0 0x7f55cc1457c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:56.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.299+0000 7f55d44a4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc098d30 con 0x7f55cc06fa70 2024-01-27T02:35:56.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.299+0000 7f55d44a4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc099010 con 0x7f55cc0a7fe0 2024-01-27T02:35:56.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.299+0000 7f55d44a4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc0992f0 con 0x7f55cc0a7c10 2024-01-27T02:35:56.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.299+0000 7f55d2240700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55cc06fa70 0x7f55cc146220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:56.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.299+0000 7f55d2240700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55cc06fa70 0x7f55cc146220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42440/0 (socket says 172.21.15.7:42440) 2024-01-27T02:35:56.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.299+0000 7f55d2240700 1 -- 172.21.15.7:0/540447909 learned_addr learned my addr 172.21.15.7:0/540447909 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:56.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.299+0000 7f55d2a41700 1 --2- 172.21.15.7:0/540447909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55cc0a7fe0 0x7f55cc1457c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:56.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.299+0000 7f55d1a3f700 1 --2- 172.21.15.7:0/540447909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc0a7c10 0x7f55cc15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:56.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.300+0000 7f55d2240700 1 -- 172.21.15.7:0/540447909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc0a7c10 msgr2=0x7f55cc15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:56.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.300+0000 7f55d2240700 1 --2- 172.21.15.7:0/540447909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc0a7c10 0x7f55cc15d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:56.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.300+0000 7f55d2240700 1 -- 172.21.15.7:0/540447909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55cc0a7fe0 msgr2=0x7f55cc1457c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:56.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.300+0000 7f55d2240700 1 --2- 172.21.15.7:0/540447909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55cc0a7fe0 0x7f55cc1457c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:56.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.300+0000 7f55d2240700 1 -- 172.21.15.7:0/540447909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c4009a20 con 0x7f55cc06fa70 2024-01-27T02:35:56.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.300+0000 7f55d1a3f700 1 --2- 172.21.15.7:0/540447909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc0a7c10 0x7f55cc15d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:35:56.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.300+0000 7f55d2240700 1 --2- 172.21.15.7:0/540447909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55cc06fa70 0x7f55cc146220 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f55c800c7c0 tx=0x7f55c8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:56.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.300+0000 7f55be7fc700 1 -- 172.21.15.7:0/540447909 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c8005b30 con 0x7f55cc06fa70 2024-01-27T02:35:56.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.300+0000 7f55be7fc700 1 -- 172.21.15.7:0/540447909 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c8005cb0 con 0x7f55cc06fa70 2024-01-27T02:35:56.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.300+0000 7f55d44a4700 1 -- 172.21.15.7:0/540447909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55cc0995d0 con 0x7f55cc06fa70 2024-01-27T02:35:56.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.300+0000 7f55be7fc700 1 -- 172.21.15.7:0/540447909 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c8007320 con 0x7f55cc06fa70 2024-01-27T02:35:56.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.300+0000 7f55d44a4700 1 -- 172.21.15.7:0/540447909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55cc145d00 con 0x7f55cc06fa70 2024-01-27T02:35:56.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.301+0000 7f55d44a4700 1 -- 172.21.15.7:0/540447909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55cc145f80 con 0x7f55cc06fa70 2024-01-27T02:35:56.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.302+0000 7f55be7fc700 1 -- 172.21.15.7:0/540447909 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f55c801b3b0 con 0x7f55cc06fa70 2024-01-27T02:35:56.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.302+0000 7f55be7fc700 1 --2- 172.21.15.7:0/540447909 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f55b0066f30 0x7f55b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:56.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.302+0000 7f55be7fc700 1 -- 172.21.15.7:0/540447909 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f55c8083720 con 0x7f55cc06fa70 2024-01-27T02:35:56.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.302+0000 7f55d1a3f700 1 --2- 172.21.15.7:0/540447909 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f55b0066f30 0x7f55b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:56.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.303+0000 7f55d1a3f700 1 --2- 172.21.15.7:0/540447909 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f55b0066f30 0x7f55b00693e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f55c0007fd0 tx=0x7f55c0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:56.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.306+0000 7f55be7fc700 1 -- 172.21.15.7:0/540447909 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55cc145f80 con 0x7f55cc06fa70 2024-01-27T02:35:56.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:56.525+0000 7f55d44a4700 1 -- 172.21.15.7:0/540447909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f55cc145f80 con 0x7f55cc06fa70 2024-01-27T02:35:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.584+0000 7f55be7fc700 1 -- 172.21.15.7:0/540447909 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v77) v1 ==== 97+0+0 (secure 0 0 0) 0x7f55cc145f80 con 0x7f55cc06fa70 2024-01-27T02:35:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.587+0000 7f55d44a4700 1 -- 172.21.15.7:0/540447909 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f55b0066f30 msgr2=0x7f55b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.587+0000 7f55d44a4700 1 --2- 172.21.15.7:0/540447909 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f55b0066f30 0x7f55b00693e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f55c0007fd0 tx=0x7f55c0009040 comp rx=0 tx=0).stop 2024-01-27T02:35:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.587+0000 7f55d44a4700 1 -- 172.21.15.7:0/540447909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55cc06fa70 msgr2=0x7f55cc146220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.587+0000 7f55d44a4700 1 --2- 172.21.15.7:0/540447909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55cc06fa70 0x7f55cc146220 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f55c800c7c0 tx=0x7f55c8007e00 comp rx=0 tx=0).stop 2024-01-27T02:35:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.587+0000 7f55d44a4700 1 -- 172.21.15.7:0/540447909 shutdown_connections 2024-01-27T02:35:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.587+0000 7f55d44a4700 1 --2- 172.21.15.7:0/540447909 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f55b0066f30 0x7f55b00693e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.587+0000 7f55d44a4700 1 --2- 172.21.15.7:0/540447909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55cc0a7fe0 0x7f55cc1457c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.587+0000 7f55d44a4700 1 --2- 172.21.15.7:0/540447909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55cc06fa70 0x7f55cc146220 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.587+0000 7f55d44a4700 1 --2- 172.21.15.7:0/540447909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc0a7c10 0x7f55cc15d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.587+0000 7f55d44a4700 1 -- 172.21.15.7:0/540447909 >> 172.21.15.7:0/540447909 conn(0x7f55cc0679b0 msgr2=0x7f55cc154e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.587+0000 7f55d44a4700 1 -- 172.21.15.7:0/540447909 shutdown_connections 2024-01-27T02:35:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.587+0000 7f55d44a4700 1 -- 172.21.15.7:0/540447909 wait complete. 2024-01-27T02:35:57.589 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-01-27T02:35:57.603 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-27T02:35:57.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.739+0000 7f8ff093b700 1 Processor -- start 2024-01-27T02:35:57.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.739+0000 7f8ff093b700 1 -- start start 2024-01-27T02:35:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.740+0000 7f8ff093b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fec157ca0 0x7f8fec0c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.741+0000 7f8ff093b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8fec0cdbf0 0x7f8fec0ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.741+0000 7f8ff093b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fec0ca020 con 0x7f8fec0c3ae0 2024-01-27T02:35:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.741+0000 7f8ff093b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fec0c4f40 con 0x7f8fec0cdbf0 2024-01-27T02:35:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.741+0000 7f8ff093b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec0c5210 con 0x7f8fec157ca0 2024-01-27T02:35:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.741+0000 7f8fead9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8fec0cdbf0 0x7f8fec0ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.741+0000 7f8fe9d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8fec0c3ae0 0x7f8fec0cd4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34250/0 (socket says 172.21.15.7:34250) 2024-01-27T02:35:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.741+0000 7f8fe9d9b700 1 -- 172.21.15.7:0/2228839830 learned_addr learned my addr 172.21.15.7:0/2228839830 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:35:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.741+0000 7f8fea59c700 1 --2- 172.21.15.7:0/2228839830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fec157ca0 0x7f8fec0c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.741+0000 7f8fead9d700 1 -- 172.21.15.7:0/2228839830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fec157ca0 msgr2=0x7f8fec0c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.741+0000 7f8fead9d700 1 --2- 172.21.15.7:0/2228839830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fec157ca0 0x7f8fec0c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.741+0000 7f8fead9d700 1 -- 172.21.15.7:0/2228839830 >> v1:172.21.15.7:6789/0 conn(0x7f8fec0c3ae0 legacy=0x7f8fec0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.741+0000 7f8fead9d700 1 -- 172.21.15.7:0/2228839830 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec0c54f0 con 0x7f8fec0cdbf0 2024-01-27T02:35:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.741+0000 7f8fe959a700 1 -- 172.21.15.7:0/2228839830 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832795443 0 0) 0x7f8fec0ca020 con 0x7f8fec0c3ae0 2024-01-27T02:35:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.741+0000 7f8fead9d700 1 --2- 172.21.15.7:0/2228839830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fec0cdbf0 0x7f8fec0ca4b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009e60 tx=0x7f8fe4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46e35ea4d75ffad server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.742+0000 7f8fe959a700 1 -- 172.21.15.7:0/2228839830 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400e070 con 0x7f8fec0cdbf0 2024-01-27T02:35:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.742+0000 7f8fe959a700 1 -- 172.21.15.7:0/2228839830 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe400bb60 con 0x7f8fec0cdbf0 2024-01-27T02:35:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.742+0000 7f8fe959a700 1 -- 172.21.15.7:0/2228839830 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4012430 con 0x7f8fec0cdbf0 2024-01-27T02:35:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.742+0000 7f8ff093b700 1 -- 172.21.15.7:0/2228839830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fec0cdbf0 msgr2=0x7f8fec0ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.742+0000 7f8ff093b700 1 --2- 172.21.15.7:0/2228839830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fec0cdbf0 0x7f8fec0ca4b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009e60 tx=0x7f8fe4009780 comp rx=0 tx=0).stop 2024-01-27T02:35:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.742+0000 7f8ff093b700 1 -- 172.21.15.7:0/2228839830 shutdown_connections 2024-01-27T02:35:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.742+0000 7f8ff093b700 1 --2- 172.21.15.7:0/2228839830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fec0cdbf0 0x7f8fec0ca4b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.742+0000 7f8ff093b700 1 --2- 172.21.15.7:0/2228839830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fec157ca0 0x7f8fec0c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:57.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.742+0000 7f8ff093b700 1 -- 172.21.15.7:0/2228839830 >> 172.21.15.7:0/2228839830 conn(0x7f8fec098020 msgr2=0x7f8fec0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:57.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.742+0000 7f8ff093b700 1 -- 172.21.15.7:0/2228839830 shutdown_connections 2024-01-27T02:35:57.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.742+0000 7f8ff093b700 1 -- 172.21.15.7:0/2228839830 wait complete. 2024-01-27T02:35:57.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.744+0000 7f8ff093b700 1 Processor -- start 2024-01-27T02:35:57.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.744+0000 7f8ff093b700 1 -- start start 2024-01-27T02:35:57.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.745+0000 7f8ff093b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fec0c3ae0 0x7f8fec0b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:57.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.745+0000 7f8ff093b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fec0cdbf0 0x7f8fec0ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:57.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.745+0000 7f8ff093b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fec157ca0 0x7f8fec0cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:57.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.745+0000 7f8ff093b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec0c6ee0 con 0x7f8fec0cdbf0 2024-01-27T02:35:57.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.745+0000 7f8ff093b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec0c62d0 con 0x7f8fec0c3ae0 2024-01-27T02:35:57.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.745+0000 7f8ff093b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec0c65b0 con 0x7f8fec157ca0 2024-01-27T02:35:57.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.745+0000 7f8fe9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fec0cdbf0 0x7f8fec0ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:57.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.745+0000 7f8fe9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fec0cdbf0 0x7f8fec0ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42454/0 (socket says 172.21.15.7:42454) 2024-01-27T02:35:57.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.745+0000 7f8fe9d9b700 1 -- 172.21.15.7:0/670487056 learned_addr learned my addr 172.21.15.7:0/670487056 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:57.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.745+0000 7f8fead9d700 1 --2- 172.21.15.7:0/670487056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fec157ca0 0x7f8fec0cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:57.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.745+0000 7f8fea59c700 1 --2- 172.21.15.7:0/670487056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fec0c3ae0 0x7f8fec0b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:57.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.745+0000 7f8fe9d9b700 1 -- 172.21.15.7:0/670487056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fec157ca0 msgr2=0x7f8fec0cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:57.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.745+0000 7f8fe9d9b700 1 --2- 172.21.15.7:0/670487056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fec157ca0 0x7f8fec0cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:57.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.745+0000 7f8fe9d9b700 1 -- 172.21.15.7:0/670487056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fec0c3ae0 msgr2=0x7f8fec0b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:57.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.746+0000 7f8fe9d9b700 1 --2- 172.21.15.7:0/670487056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fec0c3ae0 0x7f8fec0b2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:57.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.746+0000 7f8fe9d9b700 1 -- 172.21.15.7:0/670487056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe4009a20 con 0x7f8fec0cdbf0 2024-01-27T02:35:57.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.746+0000 7f8fead9d700 1 --2- 172.21.15.7:0/670487056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fec157ca0 0x7f8fec0cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:35:57.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.746+0000 7f8fe9d9b700 1 --2- 172.21.15.7:0/670487056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fec0cdbf0 0x7f8fec0ca830 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8fd400c7e0 tx=0x7f8fd4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:57.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.746+0000 7f8fe27fc700 1 -- 172.21.15.7:0/670487056 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4005b30 con 0x7f8fec0cdbf0 2024-01-27T02:35:57.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.746+0000 7f8fe27fc700 1 -- 172.21.15.7:0/670487056 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd4005c90 con 0x7f8fec0cdbf0 2024-01-27T02:35:57.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.746+0000 7f8ff093b700 1 -- 172.21.15.7:0/670487056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fec0c6890 con 0x7f8fec0cdbf0 2024-01-27T02:35:57.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.746+0000 7f8fe27fc700 1 -- 172.21.15.7:0/670487056 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4007300 con 0x7f8fec0cdbf0 2024-01-27T02:35:57.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.746+0000 7f8ff093b700 1 -- 172.21.15.7:0/670487056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fec0b2a50 con 0x7f8fec0cdbf0 2024-01-27T02:35:57.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.747+0000 7f8ff093b700 1 -- 172.21.15.7:0/670487056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fec04c710 con 0x7f8fec0cdbf0 2024-01-27T02:35:57.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.748+0000 7f8fe27fc700 1 -- 172.21.15.7:0/670487056 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8fd401b400 con 0x7f8fec0cdbf0 2024-01-27T02:35:57.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.748+0000 7f8fe27fc700 1 --2- 172.21.15.7:0/670487056 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8fcc066f30 0x7f8fcc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:57.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.748+0000 7f8fe27fc700 1 -- 172.21.15.7:0/670487056 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8fd4083d90 con 0x7f8fec0cdbf0 2024-01-27T02:35:57.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.748+0000 7f8fea59c700 1 --2- 172.21.15.7:0/670487056 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8fcc066f30 0x7f8fcc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:57.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.749+0000 7f8fea59c700 1 --2- 172.21.15.7:0/670487056 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8fcc066f30 0x7f8fcc0693e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009ca0 tx=0x7f8fdc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:57.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:57.752+0000 7f8fe27fc700 1 -- 172.21.15.7:0/670487056 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8fd4051330 con 0x7f8fec0cdbf0 2024-01-27T02:35:58.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.037+0000 7f8ff093b700 1 -- 172.21.15.7:0/670487056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8fec0c57d0 con 0x7f8fec0cdbf0 2024-01-27T02:35:58.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.037+0000 7f8fe27fc700 1 -- 172.21.15.7:0/670487056 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f8fd4054950 con 0x7f8fec0cdbf0 2024-01-27T02:35:58.038 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:35:58.038 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:35:58.038 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-01-27T02:35:58.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:35:58.038 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-01-27T02:35:58.038 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlHdcPKBAAfhnh1I3ZSQEXWJXlXCs98Q==", 2024-01-27T02:35:58.038 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:35:58.038 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:35:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:35:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:35:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:35:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:35:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:35:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:35:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-01-27T02:35:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rl8eVYNRAAfP8rOqO50RpoNZ1+1j2DXQ==", 2024-01-27T02:35:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:35:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:35:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:35:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:35:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:35:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:35:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:35:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:35:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-01-27T02:35:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlJxRKMBAAS0j7/zZeqGglOxCK1pg0WA==", 2024-01-27T02:35:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:35:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:35:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:35:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:35:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:35:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:35:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:35:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:35:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-01-27T02:35:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlNwfUABAALRQAoHoGhniqCvTQOqbbug==", 2024-01-27T02:35:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:35:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:35:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:35:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:35:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:35:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:35:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:35:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:35:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-01-27T02:35:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7Rlh6VSJhAAttuCte1WhNEvUkv5x/IKQQ==", 2024-01-27T02:35:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:35:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:35:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:35:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:35:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:35:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:35:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:35:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-01-27T02:35:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7Rlu/RGLhAAKzaCNhzYKhsNhxkMvY++gw==", 2024-01-27T02:35:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:35:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:35:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:35:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:35:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:35:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:35:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:35:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-01-27T02:35:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7RliRMyNhAAr0tG3evmVV7ZlGxbxfWPMw==", 2024-01-27T02:35:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:35:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:35:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:35:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:35:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:35:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:35:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:35:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-01-27T02:35:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAPa7RlRuSNAhAA3e9atlb+ffvlnLhRYbWZPw==", 2024-01-27T02:35:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:35:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:35:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:35:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:35:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:35:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:35:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:35:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-01-27T02:35:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7RlfeA6DBAAXpklgbNHRO+5G+fdIERjRw==", 2024-01-27T02:35:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:35:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:35:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:35:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:35:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:35:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:35:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:35:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-01-27T02:35:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7Rl77ItExAABB7uABm63ubE/YUs2B5pGw==", 2024-01-27T02:35:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:35:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:35:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:35:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:35:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:35:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:35:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:35:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-01-27T02:35:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7RljDuwHBAAPPAlhoqS03qPpq58jmevLA==", 2024-01-27T02:35:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:35:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:35:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:35:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:35:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:35:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:35:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:35:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-01-27T02:35:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7Rl1YElJhAAqHP/QEdl1SEalGcbQwPwAg==", 2024-01-27T02:35:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:35:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:35:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:35:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:35:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:35:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:35:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:35:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-01-27T02:35:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlaEqaBRAATFiIeW8lT9BdNuN0m3x9dQ==", 2024-01-27T02:35:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:35:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:35:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-01-27T02:35:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-27T02:35:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:35:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:35:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:35:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-01-27T02:35:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlNADmEBAAJXwdJ35ylP5coy1dtHoD3g==", 2024-01-27T02:35:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:35:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:35:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:35:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:35:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:35:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:35:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:35:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:35:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-01-27T02:35:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7Rl5fRqCRAA/5swTGknnpP64TAh+2X41A==", 2024-01-27T02:35:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:35:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:35:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:35:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:35:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:35:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:35:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:35:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:35:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-01-27T02:35:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7Rl0SdwFhAAhCmw9ViiJbHVfS5a2r/D+w==", 2024-01-27T02:35:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:35:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:35:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:35:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:35:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:35:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:35:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:35:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:35:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-01-27T02:35:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlsNgPARAA0Qm0oJgA8GO1oUT3ndT8Tg==", 2024-01-27T02:35:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:35:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:35:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:35:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:35:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:35:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:35:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:35:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:35:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-01-27T02:35:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rlfb7qIxAAlsJYiSolJ7D8EykGWP62bA==", 2024-01-27T02:35:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:35:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:35:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-27T02:35:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:35:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:35:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:35:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:35:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-01-27T02:35:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rls7TaHxAALS3ghxKtfmgQfIELiD9+Bg==", 2024-01-27T02:35:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:35:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:35:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-27T02:35:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:35:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:35:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:35:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:35:58.051 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:35:58.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.039+0000 7f8ff093b700 1 -- 172.21.15.7:0/670487056 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8fcc066f30 msgr2=0x7f8fcc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:58.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.039+0000 7f8ff093b700 1 --2- 172.21.15.7:0/670487056 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8fcc066f30 0x7f8fcc0693e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009ca0 tx=0x7f8fdc008040 comp rx=0 tx=0).stop 2024-01-27T02:35:58.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.039+0000 7f8ff093b700 1 -- 172.21.15.7:0/670487056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fec0cdbf0 msgr2=0x7f8fec0ca830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:58.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.039+0000 7f8ff093b700 1 --2- 172.21.15.7:0/670487056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fec0cdbf0 0x7f8fec0ca830 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8fd400c7e0 tx=0x7f8fd4007df0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.039+0000 7f8ff093b700 1 -- 172.21.15.7:0/670487056 shutdown_connections 2024-01-27T02:35:58.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.039+0000 7f8ff093b700 1 --2- 172.21.15.7:0/670487056 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8fcc066f30 0x7f8fcc0693e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.039+0000 7f8ff093b700 1 --2- 172.21.15.7:0/670487056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fec157ca0 0x7f8fec0cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.039+0000 7f8ff093b700 1 --2- 172.21.15.7:0/670487056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fec0c3ae0 0x7f8fec0b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.039+0000 7f8ff093b700 1 --2- 172.21.15.7:0/670487056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fec0cdbf0 0x7f8fec0ca830 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.039+0000 7f8ff093b700 1 -- 172.21.15.7:0/670487056 >> 172.21.15.7:0/670487056 conn(0x7f8fec098020 msgr2=0x7f8fec1514e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:58.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.039+0000 7f8ff093b700 1 -- 172.21.15.7:0/670487056 shutdown_connections 2024-01-27T02:35:58.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.039+0000 7f8ff093b700 1 -- 172.21.15.7:0/670487056 wait complete. 2024-01-27T02:35:58.054 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:35:58.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.184+0000 7fb0bb61a700 1 Processor -- start 2024-01-27T02:35:58.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.184+0000 7fb0bb61a700 1 -- start start 2024-01-27T02:35:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.185+0000 7fb0bb61a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb0b40637d0 0x7fb0b406ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.185+0000 7fb0bb61a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0b4063ba0 0x7fb0b40c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.185+0000 7fb0bb61a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0b415b6c0 con 0x7fb0b40c41e0 2024-01-27T02:35:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.185+0000 7fb0bb61a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b415ae50 con 0x7fb0b4063ba0 2024-01-27T02:35:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.185+0000 7fb0bb61a700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b415b100 con 0x7fb0b40637d0 2024-01-27T02:35:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.186+0000 7fb0b9bb7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0b4063ba0 0x7fb0b40c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.186+0000 7fb0b8bb5700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb0b40c41e0 0x7fb0b415a740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34266/0 (socket says 172.21.15.7:34266) 2024-01-27T02:35:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.186+0000 7fb0b8bb5700 1 -- 172.21.15.7:0/3422964162 learned_addr learned my addr 172.21.15.7:0/3422964162 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:35:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.186+0000 7fb0b93b6700 1 --2- 172.21.15.7:0/3422964162 >> v2:172.21.15.188:3300/0 conn(0x7fb0b40637d0 0x7fb0b406ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.186+0000 7fb0b9bb7700 1 -- 172.21.15.7:0/3422964162 >> v2:172.21.15.188:3300/0 conn(0x7fb0b40637d0 msgr2=0x7fb0b406ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:35:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.186+0000 7fb0b9bb7700 1 --2- 172.21.15.7:0/3422964162 >> v2:172.21.15.188:3300/0 conn(0x7fb0b40637d0 0x7fb0b406ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.186+0000 7fb0b9bb7700 1 -- 172.21.15.7:0/3422964162 >> v1:172.21.15.7:6789/0 conn(0x7fb0b40c41e0 legacy=0x7fb0b415a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:58.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.186+0000 7fb0abfff700 1 -- 172.21.15.7:0/3422964162 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 764095325 0 0) 0x7fb0b415b6c0 con 0x7fb0b40c41e0 2024-01-27T02:35:58.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.186+0000 7fb0b9bb7700 1 -- 172.21.15.7:0/3422964162 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b415b3e0 con 0x7fb0b4063ba0 2024-01-27T02:35:58.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.186+0000 7fb0b9bb7700 1 --2- 172.21.15.7:0/3422964162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b4063ba0 0x7fb0b40c86b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40b06f0 tx=0x7fb0ac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596a3578b211c24b server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:58.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.186+0000 7fb0abfff700 1 -- 172.21.15.7:0/3422964162 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac00e070 con 0x7fb0b4063ba0 2024-01-27T02:35:58.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.186+0000 7fb0abfff700 1 -- 172.21.15.7:0/3422964162 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ac00bb60 con 0x7fb0b4063ba0 2024-01-27T02:35:58.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.186+0000 7fb0abfff700 1 -- 172.21.15.7:0/3422964162 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac012430 con 0x7fb0b4063ba0 2024-01-27T02:35:58.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.187+0000 7fb0bb61a700 1 -- 172.21.15.7:0/3422964162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b4063ba0 msgr2=0x7fb0b40c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:58.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.187+0000 7fb0bb61a700 1 --2- 172.21.15.7:0/3422964162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b4063ba0 0x7fb0b40c86b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40b06f0 tx=0x7fb0ac009780 comp rx=0 tx=0).stop 2024-01-27T02:35:58.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.187+0000 7fb0bb61a700 1 -- 172.21.15.7:0/3422964162 shutdown_connections 2024-01-27T02:35:58.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.187+0000 7fb0bb61a700 1 --2- 172.21.15.7:0/3422964162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b4063ba0 0x7fb0b40c86b0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.187+0000 7fb0bb61a700 1 --2- 172.21.15.7:0/3422964162 >> v2:172.21.15.188:3300/0 conn(0x7fb0b40637d0 0x7fb0b406ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.187+0000 7fb0bb61a700 1 -- 172.21.15.7:0/3422964162 >> 172.21.15.7:0/3422964162 conn(0x7fb0b40cdc30 msgr2=0x7fb0b40ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:58.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.187+0000 7fb0bb61a700 1 -- 172.21.15.7:0/3422964162 shutdown_connections 2024-01-27T02:35:58.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.187+0000 7fb0bb61a700 1 -- 172.21.15.7:0/3422964162 wait complete. 2024-01-27T02:35:58.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.188+0000 7fb0bb61a700 1 Processor -- start 2024-01-27T02:35:58.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.188+0000 7fb0bb61a700 1 -- start start 2024-01-27T02:35:58.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.189+0000 7fb0bb61a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0b40637d0 0x7fb0b40b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:58.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.189+0000 7fb0bb61a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b4063ba0 0x7fb0b4152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:58.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.189+0000 7fb0bb61a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0b40c41e0 0x7fb0b4153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:58.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.189+0000 7fb0bb61a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b40b2c00 con 0x7fb0b4063ba0 2024-01-27T02:35:58.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.189+0000 7fb0bb61a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b40b1ff0 con 0x7fb0b40637d0 2024-01-27T02:35:58.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.189+0000 7fb0bb61a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b40b22d0 con 0x7fb0b40c41e0 2024-01-27T02:35:58.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.189+0000 7fb0b8bb5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b4063ba0 0x7fb0b4152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:58.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.190+0000 7fb0b8bb5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b4063ba0 0x7fb0b4152c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42472/0 (socket says 172.21.15.7:42472) 2024-01-27T02:35:58.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.190+0000 7fb0b8bb5700 1 -- 172.21.15.7:0/2482452615 learned_addr learned my addr 172.21.15.7:0/2482452615 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:58.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.190+0000 7fb0b9bb7700 1 --2- 172.21.15.7:0/2482452615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0b40c41e0 0x7fb0b4153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:58.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.190+0000 7fb0b93b6700 1 --2- 172.21.15.7:0/2482452615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0b40637d0 0x7fb0b40b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:58.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.190+0000 7fb0b8bb5700 1 -- 172.21.15.7:0/2482452615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0b40c41e0 msgr2=0x7fb0b4153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:58.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.190+0000 7fb0b8bb5700 1 --2- 172.21.15.7:0/2482452615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0b40c41e0 0x7fb0b4153160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.190+0000 7fb0b8bb5700 1 -- 172.21.15.7:0/2482452615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0b40637d0 msgr2=0x7fb0b40b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:58.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.190+0000 7fb0b8bb5700 1 --2- 172.21.15.7:0/2482452615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0b40637d0 0x7fb0b40b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.190+0000 7fb0b8bb5700 1 -- 172.21.15.7:0/2482452615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ac009a20 con 0x7fb0b4063ba0 2024-01-27T02:35:58.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.190+0000 7fb0b8bb5700 1 --2- 172.21.15.7:0/2482452615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b4063ba0 0x7fb0b4152c20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400c5b0 tx=0x7fb0a4000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:58.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.190+0000 7fb0a97fa700 1 -- 172.21.15.7:0/2482452615 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4005b30 con 0x7fb0b4063ba0 2024-01-27T02:35:58.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.190+0000 7fb0a97fa700 1 -- 172.21.15.7:0/2482452615 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a4005c90 con 0x7fb0b4063ba0 2024-01-27T02:35:58.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.190+0000 7fb0bb61a700 1 -- 172.21.15.7:0/2482452615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b40b25b0 con 0x7fb0b4063ba0 2024-01-27T02:35:58.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.190+0000 7fb0a97fa700 1 -- 172.21.15.7:0/2482452615 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a40072e0 con 0x7fb0b4063ba0 2024-01-27T02:35:58.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.190+0000 7fb0bb61a700 1 -- 172.21.15.7:0/2482452615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b40b3090 con 0x7fb0b4063ba0 2024-01-27T02:35:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.191+0000 7fb0bb61a700 1 -- 172.21.15.7:0/2482452615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b404c710 con 0x7fb0b4063ba0 2024-01-27T02:35:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.192+0000 7fb0a97fa700 1 -- 172.21.15.7:0/2482452615 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb0a401b3b0 con 0x7fb0b4063ba0 2024-01-27T02:35:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.192+0000 7fb0a97fa700 1 --2- 172.21.15.7:0/2482452615 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb098066f30 0x7fb0980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.192+0000 7fb0a97fa700 1 -- 172.21.15.7:0/2482452615 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb0a4083510 con 0x7fb0b4063ba0 2024-01-27T02:35:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.193+0000 7fb0b93b6700 1 --2- 172.21.15.7:0/2482452615 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb098066f30 0x7fb0980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.193+0000 7fb0b93b6700 1 --2- 172.21.15.7:0/2482452615 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb098066f30 0x7fb0980693e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40cab80 tx=0x7fb0b00091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.197+0000 7fb0a97fa700 1 -- 172.21.15.7:0/2482452615 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0a404b5a0 con 0x7fb0b4063ba0 2024-01-27T02:35:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.400+0000 7fb0bb61a700 1 -- 172.21.15.7:0/2482452615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0b40b2d40 con 0x7fb0b4063ba0 2024-01-27T02:35:58.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.401+0000 7fb0a97fa700 1 -- 172.21.15.7:0/2482452615 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+12086 (secure 0 0 0) 0x7fb0a4050ae0 con 0x7fb0b4063ba0 2024-01-27T02:35:58.401 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:35:58.401 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":77,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:35:57.581354+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:35:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.403+0000 7fb0bb61a700 1 -- 172.21.15.7:0/2482452615 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb098066f30 msgr2=0x7fb0980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.403+0000 7fb0bb61a700 1 --2- 172.21.15.7:0/2482452615 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb098066f30 0x7fb0980693e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40cab80 tx=0x7fb0b00091a0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.403+0000 7fb0bb61a700 1 -- 172.21.15.7:0/2482452615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b4063ba0 msgr2=0x7fb0b4152c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.403+0000 7fb0bb61a700 1 --2- 172.21.15.7:0/2482452615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b4063ba0 0x7fb0b4152c20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400c5b0 tx=0x7fb0a4000fe0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.403+0000 7fb0bb61a700 1 -- 172.21.15.7:0/2482452615 shutdown_connections 2024-01-27T02:35:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.403+0000 7fb0bb61a700 1 --2- 172.21.15.7:0/2482452615 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb098066f30 0x7fb0980693e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.403+0000 7fb0bb61a700 1 --2- 172.21.15.7:0/2482452615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0b40c41e0 0x7fb0b4153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.403+0000 7fb0bb61a700 1 --2- 172.21.15.7:0/2482452615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0b40637d0 0x7fb0b40b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.403+0000 7fb0bb61a700 1 --2- 172.21.15.7:0/2482452615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b4063ba0 0x7fb0b4152c20 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.403+0000 7fb0bb61a700 1 -- 172.21.15.7:0/2482452615 >> 172.21.15.7:0/2482452615 conn(0x7fb0b40cdc30 msgr2=0x7fb0b41524c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:58.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.403+0000 7fb0bb61a700 1 -- 172.21.15.7:0/2482452615 shutdown_connections 2024-01-27T02:35:58.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.403+0000 7fb0bb61a700 1 -- 172.21.15.7:0/2482452615 wait complete. 2024-01-27T02:35:58.417 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-27T02:35:58.418 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:35:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.552+0000 7f5ccf7e5700 1 Processor -- start 2024-01-27T02:35:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.552+0000 7f5ccf7e5700 1 -- start start 2024-01-27T02:35:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.553+0000 7f5ccf7e5700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5cc8156f30 0x7f5cc80aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.553+0000 7f5ccf7e5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc80aeaa0 con 0x7f5cc8156b60 2024-01-27T02:35:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.553+0000 7f5ccf7e5700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc80adbc0 con 0x7f5cc80af470 2024-01-27T02:35:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.553+0000 7f5ccf7e5700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc80ade90 con 0x7f5cc8156f30 2024-01-27T02:35:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.553+0000 7f5ccd581700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5cc8156b60 0x7f5cc80ad4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34276/0 (socket says 172.21.15.7:34276) 2024-01-27T02:35:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.553+0000 7f5ccd581700 1 -- 172.21.15.7:0/3046845982 learned_addr learned my addr 172.21.15.7:0/3046845982 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:35:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.554+0000 7f5cccd80700 1 --2- 172.21.15.7:0/3046845982 >> v2:172.21.15.188:3300/0 conn(0x7f5cc8156f30 0x7f5cc80aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.554+0000 7f5cbbfff700 1 -- 172.21.15.7:0/3046845982 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 957691043 0 0) 0x7f5cc80aeaa0 con 0x7f5cc8156b60 2024-01-27T02:35:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.554+0000 7f5cbbfff700 1 -- 172.21.15.7:0/3046845982 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cb4003570 con 0x7f5cc8156b60 2024-01-27T02:35:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.554+0000 7f5cccd80700 1 -- 172.21.15.7:0/3046845982 >> v1:172.21.15.188:6789/0 conn(0x7f5cc80af470 legacy=0x7f5cc815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.554+0000 7f5cbbfff700 1 -- 172.21.15.7:0/3046845982 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 419088264 0 0) 0x7f5cb4003570 con 0x7f5cc8156b60 2024-01-27T02:35:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.554+0000 7f5cccd80700 1 -- 172.21.15.7:0/3046845982 >> v1:172.21.15.7:6789/0 conn(0x7f5cc8156b60 legacy=0x7f5cc80ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.554+0000 7f5cccd80700 1 -- 172.21.15.7:0/3046845982 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc80ae170 con 0x7f5cc8156f30 2024-01-27T02:35:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.554+0000 7f5cccd80700 1 --2- 172.21.15.7:0/3046845982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc8156f30 0x7f5cc80aef30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009910 tx=0x7f5cc40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb87e29f3e42f4c0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.554+0000 7f5cbbfff700 1 -- 172.21.15.7:0/3046845982 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc001470 con 0x7f5cc8156f30 2024-01-27T02:35:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.554+0000 7f5cbbfff700 1 -- 172.21.15.7:0/3046845982 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc4011420 con 0x7f5cc8156f30 2024-01-27T02:35:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.555+0000 7f5cbbfff700 1 -- 172.21.15.7:0/3046845982 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc40117c0 con 0x7f5cc8156f30 2024-01-27T02:35:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.555+0000 7f5ccf7e5700 1 -- 172.21.15.7:0/3046845982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc8156f30 msgr2=0x7f5cc80aef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.555+0000 7f5ccf7e5700 1 --2- 172.21.15.7:0/3046845982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc8156f30 0x7f5cc80aef30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009910 tx=0x7f5cc40091a0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.555+0000 7f5ccf7e5700 1 -- 172.21.15.7:0/3046845982 shutdown_connections 2024-01-27T02:35:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.555+0000 7f5ccf7e5700 1 --2- 172.21.15.7:0/3046845982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc8156f30 0x7f5cc80aef30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.555+0000 7f5ccf7e5700 1 -- 172.21.15.7:0/3046845982 >> 172.21.15.7:0/3046845982 conn(0x7f5cc8068480 msgr2=0x7f5cc80cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.555+0000 7f5ccf7e5700 1 -- 172.21.15.7:0/3046845982 shutdown_connections 2024-01-27T02:35:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.555+0000 7f5ccf7e5700 1 -- 172.21.15.7:0/3046845982 wait complete. 2024-01-27T02:35:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.556+0000 7f5ccf7e5700 1 Processor -- start 2024-01-27T02:35:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.556+0000 7f5ccf7e5700 1 -- start start 2024-01-27T02:35:58.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.557+0000 7f5ccf7e5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc80af470 0x7f5cc81411b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.557+0000 7f5ccf7e5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cc8156b60 0x7f5cc8142180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.557+0000 7f5ccf7e5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc8156f30 0x7f5cc806ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.557+0000 7f5ccf7e5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc81429b0 con 0x7f5cc80af470 2024-01-27T02:35:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.557+0000 7f5ccf7e5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc8142c90 con 0x7f5cc8156f30 2024-01-27T02:35:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.557+0000 7f5ccf7e5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc8143240 con 0x7f5cc8156b60 2024-01-27T02:35:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.557+0000 7f5ccd581700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc80af470 0x7f5cc81411b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.558+0000 7f5ccd581700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc80af470 0x7f5cc81411b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42484/0 (socket says 172.21.15.7:42484) 2024-01-27T02:35:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.558+0000 7f5ccd581700 1 -- 172.21.15.7:0/2418564953 learned_addr learned my addr 172.21.15.7:0/2418564953 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.558+0000 7f5cccd80700 1 --2- 172.21.15.7:0/2418564953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cc8156b60 0x7f5cc8142180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.558+0000 7f5ccdd82700 1 --2- 172.21.15.7:0/2418564953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc8156f30 0x7f5cc806ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.558+0000 7f5ccd581700 1 -- 172.21.15.7:0/2418564953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cc8156b60 msgr2=0x7f5cc8142180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.558+0000 7f5ccd581700 1 --2- 172.21.15.7:0/2418564953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cc8156b60 0x7f5cc8142180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.558+0000 7f5ccd581700 1 -- 172.21.15.7:0/2418564953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc8156f30 msgr2=0x7f5cc806ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.558+0000 7f5ccd581700 1 --2- 172.21.15.7:0/2418564953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc8156f30 0x7f5cc806ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.558+0000 7f5ccd581700 1 -- 172.21.15.7:0/2418564953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc4009420 con 0x7f5cc80af470 2024-01-27T02:35:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.558+0000 7f5cccd80700 1 --2- 172.21.15.7:0/2418564953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cc8156b60 0x7f5cc8142180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:35:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.558+0000 7f5ccdd82700 1 --2- 172.21.15.7:0/2418564953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc8156f30 0x7f5cc806ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:35:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.558+0000 7f5ccd581700 1 --2- 172.21.15.7:0/2418564953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc80af470 0x7f5cc81411b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00cb90 tx=0x7f5cbc001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.558+0000 7f5cb97fa700 1 -- 172.21.15.7:0/2418564953 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc007360 con 0x7f5cc80af470 2024-01-27T02:35:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.558+0000 7f5cb97fa700 1 -- 172.21.15.7:0/2418564953 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cbc0074e0 con 0x7f5cc80af470 2024-01-27T02:35:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.558+0000 7f5ccf7e5700 1 -- 172.21.15.7:0/2418564953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc8143520 con 0x7f5cc80af470 2024-01-27T02:35:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.558+0000 7f5cb97fa700 1 -- 172.21.15.7:0/2418564953 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc004830 con 0x7f5cc80af470 2024-01-27T02:35:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.558+0000 7f5ccf7e5700 1 -- 172.21.15.7:0/2418564953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc81426c0 con 0x7f5cc80af470 2024-01-27T02:35:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.559+0000 7f5ccf7e5700 1 -- 172.21.15.7:0/2418564953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc8143a40 con 0x7f5cc80af470 2024-01-27T02:35:58.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.560+0000 7f5cb97fa700 1 -- 172.21.15.7:0/2418564953 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5cbc004c00 con 0x7f5cc80af470 2024-01-27T02:35:58.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.560+0000 7f5cb97fa700 1 --2- 172.21.15.7:0/2418564953 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5cac066f30 0x7f5cac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:58.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.560+0000 7f5cb97fa700 1 -- 172.21.15.7:0/2418564953 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5cbc083a30 con 0x7f5cc80af470 2024-01-27T02:35:58.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.561+0000 7f5cccd80700 1 --2- 172.21.15.7:0/2418564953 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5cac066f30 0x7f5cac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:58.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.561+0000 7f5cccd80700 1 --2- 172.21.15.7:0/2418564953 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5cac066f30 0x7f5cac0693e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5cc40093f0 tx=0x7f5cc4016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:58.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.565+0000 7f5cb97fa700 1 -- 172.21.15.7:0/2418564953 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5cc8143a40 con 0x7f5cc80af470 2024-01-27T02:35:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.769+0000 7f5ccf7e5700 1 -- 172.21.15.7:0/2418564953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5cc8143a40 con 0x7f5cc80af470 2024-01-27T02:35:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.769+0000 7f5cb97fa700 1 -- 172.21.15.7:0/2418564953 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5cc8143a40 con 0x7f5cc80af470 2024-01-27T02:35:58.771 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:35:58.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.771+0000 7f5ccf7e5700 1 -- 172.21.15.7:0/2418564953 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5cac066f30 msgr2=0x7f5cac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:58.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.771+0000 7f5ccf7e5700 1 --2- 172.21.15.7:0/2418564953 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5cac066f30 0x7f5cac0693e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5cc40093f0 tx=0x7f5cc4016000 comp rx=0 tx=0).stop 2024-01-27T02:35:58.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.771+0000 7f5ccf7e5700 1 -- 172.21.15.7:0/2418564953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc80af470 msgr2=0x7f5cc81411b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:58.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.771+0000 7f5ccf7e5700 1 --2- 172.21.15.7:0/2418564953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc80af470 0x7f5cc81411b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00cb90 tx=0x7f5cbc001b50 comp rx=0 tx=0).stop 2024-01-27T02:35:58.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.772+0000 7f5ccf7e5700 1 -- 172.21.15.7:0/2418564953 shutdown_connections 2024-01-27T02:35:58.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.772+0000 7f5ccf7e5700 1 --2- 172.21.15.7:0/2418564953 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5cac066f30 0x7f5cac0693e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.772+0000 7f5ccf7e5700 1 --2- 172.21.15.7:0/2418564953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc8156f30 0x7f5cc806ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.772+0000 7f5ccf7e5700 1 --2- 172.21.15.7:0/2418564953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc80af470 0x7f5cc81411b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.772+0000 7f5ccf7e5700 1 --2- 172.21.15.7:0/2418564953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cc8156b60 0x7f5cc8142180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.772+0000 7f5ccf7e5700 1 -- 172.21.15.7:0/2418564953 >> 172.21.15.7:0/2418564953 conn(0x7f5cc8068480 msgr2=0x7f5cc814ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:58.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.772+0000 7f5ccf7e5700 1 -- 172.21.15.7:0/2418564953 shutdown_connections 2024-01-27T02:35:58.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.772+0000 7f5ccf7e5700 1 -- 172.21.15.7:0/2418564953 wait complete. 2024-01-27T02:35:58.786 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-27T02:35:58.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.917+0000 7fce3820c700 1 Processor -- start 2024-01-27T02:35:58.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.918+0000 7fce3820c700 1 -- start start 2024-01-27T02:35:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.919+0000 7fce3820c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fce30063c90 0x7fce30164cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.919+0000 7fce3820c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce300b0e30 con 0x7fce30159fb0 2024-01-27T02:35:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.919+0000 7fce3820c700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce300acab0 con 0x7fce300ac640 2024-01-27T02:35:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.919+0000 7fce3820c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce300ad2e0 con 0x7fce30063c90 2024-01-27T02:35:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.919+0000 7fce35fa8700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fce30159fb0 0x7fce3006b030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34278/0 (socket says 172.21.15.7:34278) 2024-01-27T02:35:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.919+0000 7fce35fa8700 1 -- 172.21.15.7:0/259282942 learned_addr learned my addr 172.21.15.7:0/259282942 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:35:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.919+0000 7fce367a9700 1 --2- 172.21.15.7:0/259282942 >> v2:172.21.15.7:3300/0 conn(0x7fce30063c90 0x7fce30164cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.919+0000 7fce34fa6700 1 -- 172.21.15.7:0/259282942 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4013676088 0 0) 0x7fce300b0e30 con 0x7fce30159fb0 2024-01-27T02:35:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.919+0000 7fce34fa6700 1 -- 172.21.15.7:0/259282942 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce1c003570 con 0x7fce30159fb0 2024-01-27T02:35:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.919+0000 7fce367a9700 1 -- 172.21.15.7:0/259282942 >> v1:172.21.15.188:6789/0 conn(0x7fce300ac640 legacy=0x7fce30063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.919+0000 7fce367a9700 1 -- 172.21.15.7:0/259282942 >> v1:172.21.15.7:6789/0 conn(0x7fce30159fb0 legacy=0x7fce3006b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.919+0000 7fce367a9700 1 -- 172.21.15.7:0/259282942 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce300acd20 con 0x7fce30063c90 2024-01-27T02:35:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.919+0000 7fce367a9700 1 --2- 172.21.15.7:0/259282942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce30063c90 0x7fce30164cb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fce28009e60 tx=0x7fce28009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f839ad10443f4906 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.919+0000 7fce34fa6700 1 -- 172.21.15.7:0/259282942 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2800e070 con 0x7fce30063c90 2024-01-27T02:35:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.920+0000 7fce34fa6700 1 -- 172.21.15.7:0/259282942 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2800bb60 con 0x7fce30063c90 2024-01-27T02:35:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.920+0000 7fce34fa6700 1 -- 172.21.15.7:0/259282942 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce28012430 con 0x7fce30063c90 2024-01-27T02:35:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.920+0000 7fce3820c700 1 -- 172.21.15.7:0/259282942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce30063c90 msgr2=0x7fce30164cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.920+0000 7fce3820c700 1 --2- 172.21.15.7:0/259282942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce30063c90 0x7fce30164cb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fce28009e60 tx=0x7fce28009780 comp rx=0 tx=0).stop 2024-01-27T02:35:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.920+0000 7fce3820c700 1 -- 172.21.15.7:0/259282942 shutdown_connections 2024-01-27T02:35:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.920+0000 7fce3820c700 1 --2- 172.21.15.7:0/259282942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce30063c90 0x7fce30164cb0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.920+0000 7fce3820c700 1 -- 172.21.15.7:0/259282942 >> 172.21.15.7:0/259282942 conn(0x7fce30068430 msgr2=0x7fce3006be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.920+0000 7fce3820c700 1 -- 172.21.15.7:0/259282942 shutdown_connections 2024-01-27T02:35:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.920+0000 7fce3820c700 1 -- 172.21.15.7:0/259282942 wait complete. 2024-01-27T02:35:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.921+0000 7fce3820c700 1 Processor -- start 2024-01-27T02:35:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.922+0000 7fce3820c700 1 -- start start 2024-01-27T02:35:58.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.922+0000 7fce3820c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce30063c90 0x7fce30150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:58.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.922+0000 7fce3820c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce300ac640 0x7fce30153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:58.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.922+0000 7fce3820c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce30159fb0 0x7fce30151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:58.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.922+0000 7fce3820c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3015af30 con 0x7fce30063c90 2024-01-27T02:35:58.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.922+0000 7fce3820c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3015b1d0 con 0x7fce300ac640 2024-01-27T02:35:58.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.922+0000 7fce3820c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3015b4b0 con 0x7fce30159fb0 2024-01-27T02:35:58.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.922+0000 7fce35fa8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce30063c90 0x7fce30150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:58.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.922+0000 7fce35fa8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce30063c90 0x7fce30150790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42506/0 (socket says 172.21.15.7:42506) 2024-01-27T02:35:58.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.922+0000 7fce35fa8700 1 -- 172.21.15.7:0/325471553 learned_addr learned my addr 172.21.15.7:0/325471553 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:58.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.923+0000 7fce367a9700 1 --2- 172.21.15.7:0/325471553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce30159fb0 0x7fce30151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:58.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.923+0000 7fce357a7700 1 --2- 172.21.15.7:0/325471553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce300ac640 0x7fce30153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:58.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.923+0000 7fce35fa8700 1 -- 172.21.15.7:0/325471553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce30159fb0 msgr2=0x7fce30151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:58.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.923+0000 7fce35fa8700 1 --2- 172.21.15.7:0/325471553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce30159fb0 0x7fce30151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.923+0000 7fce35fa8700 1 -- 172.21.15.7:0/325471553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce300ac640 msgr2=0x7fce30153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:58.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.923+0000 7fce35fa8700 1 --2- 172.21.15.7:0/325471553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce300ac640 0x7fce30153c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:58.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.923+0000 7fce35fa8700 1 -- 172.21.15.7:0/325471553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce28009a20 con 0x7fce30063c90 2024-01-27T02:35:58.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.923+0000 7fce35fa8700 1 --2- 172.21.15.7:0/325471553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce30063c90 0x7fce30150790 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00c590 tx=0x7fce2c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:58.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.923+0000 7fce25ffb700 1 -- 172.21.15.7:0/325471553 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c005b30 con 0x7fce30063c90 2024-01-27T02:35:58.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.923+0000 7fce25ffb700 1 -- 172.21.15.7:0/325471553 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2c005cb0 con 0x7fce30063c90 2024-01-27T02:35:58.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.923+0000 7fce25ffb700 1 -- 172.21.15.7:0/325471553 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c007320 con 0x7fce30063c90 2024-01-27T02:35:58.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.923+0000 7fce3820c700 1 -- 172.21.15.7:0/325471553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3015b790 con 0x7fce30063c90 2024-01-27T02:35:58.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.924+0000 7fce3820c700 1 -- 172.21.15.7:0/325471553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce30150cd0 con 0x7fce30063c90 2024-01-27T02:35:58.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.925+0000 7fce3820c700 1 -- 172.21.15.7:0/325471553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce3004c710 con 0x7fce30063c90 2024-01-27T02:35:58.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.925+0000 7fce25ffb700 1 -- 172.21.15.7:0/325471553 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fce2c01b3b0 con 0x7fce30063c90 2024-01-27T02:35:58.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.925+0000 7fce25ffb700 1 --2- 172.21.15.7:0/325471553 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fce14066f30 0x7fce140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:58.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.925+0000 7fce25ffb700 1 -- 172.21.15.7:0/325471553 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fce2c083660 con 0x7fce30063c90 2024-01-27T02:35:58.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.925+0000 7fce357a7700 1 --2- 172.21.15.7:0/325471553 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fce14066f30 0x7fce140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:58.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.926+0000 7fce357a7700 1 --2- 172.21.15.7:0/325471553 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fce14066f30 0x7fce140693e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fce300b2a80 tx=0x7fce20007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:58.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:58.930+0000 7fce25ffb700 1 -- 172.21.15.7:0/325471553 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce2c050c20 con 0x7fce30063c90 2024-01-27T02:35:59.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.137+0000 7fce3820c700 1 -- 172.21.15.7:0/325471553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fce300ad000 con 0x7fce30063c90 2024-01-27T02:35:59.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.137+0000 7fce25ffb700 1 -- 172.21.15.7:0/325471553 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) v1 ==== 84+0+65 (secure 0 0 0) 0x7fce2c017090 con 0x7fce30063c90 2024-01-27T02:35:59.138 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:35:59.138 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-27T02:35:59.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:35:59.138 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-27T02:35:59.138 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-27T02:35:59.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:35:59.139 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-27T02:35:59.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.140+0000 7fce3820c700 1 -- 172.21.15.7:0/325471553 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fce14066f30 msgr2=0x7fce140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:59.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.140+0000 7fce3820c700 1 --2- 172.21.15.7:0/325471553 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fce14066f30 0x7fce140693e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fce300b2a80 tx=0x7fce20007ec0 comp rx=0 tx=0).stop 2024-01-27T02:35:59.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.140+0000 7fce3820c700 1 -- 172.21.15.7:0/325471553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce30063c90 msgr2=0x7fce30150790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:59.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.140+0000 7fce3820c700 1 --2- 172.21.15.7:0/325471553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce30063c90 0x7fce30150790 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00c590 tx=0x7fce2c007e00 comp rx=0 tx=0).stop 2024-01-27T02:35:59.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.140+0000 7fce3820c700 1 -- 172.21.15.7:0/325471553 shutdown_connections 2024-01-27T02:35:59.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.140+0000 7fce3820c700 1 --2- 172.21.15.7:0/325471553 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fce14066f30 0x7fce140693e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:59.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.140+0000 7fce3820c700 1 --2- 172.21.15.7:0/325471553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce30159fb0 0x7fce30151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:59.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.140+0000 7fce3820c700 1 --2- 172.21.15.7:0/325471553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce30063c90 0x7fce30150790 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:59.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.140+0000 7fce3820c700 1 --2- 172.21.15.7:0/325471553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce300ac640 0x7fce30153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:59.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.140+0000 7fce3820c700 1 -- 172.21.15.7:0/325471553 >> 172.21.15.7:0/325471553 conn(0x7fce30068430 msgr2=0x7fce30157150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:59.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.140+0000 7fce3820c700 1 -- 172.21.15.7:0/325471553 shutdown_connections 2024-01-27T02:35:59.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.140+0000 7fce3820c700 1 -- 172.21.15.7:0/325471553 wait complete. 2024-01-27T02:35:59.154 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-27T02:35:59.154 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-27T02:35:59.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.290+0000 7f0001046700 1 Processor -- start 2024-01-27T02:35:59.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.290+0000 7f0001046700 1 -- start start 2024-01-27T02:35:59.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.291+0000 7f0001046700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efffc06bde0 0x7efffc06c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:59.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.291+0000 7f0001046700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efffc06c790 con 0x7efffc0adeb0 2024-01-27T02:35:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.291+0000 7f0001046700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efffc0c96a0 con 0x7efffc156f50 2024-01-27T02:35:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.291+0000 7f0001046700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efffc06e2d0 con 0x7efffc06bde0 2024-01-27T02:35:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.291+0000 7efffb59e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efffc0adeb0 0x7efffc0ae330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34282/0 (socket says 172.21.15.7:34282) 2024-01-27T02:35:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.291+0000 7efffb59e700 1 -- 172.21.15.7:0/293314110 learned_addr learned my addr 172.21.15.7:0/293314110 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:35:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.291+0000 7efffa59c700 1 --2- 172.21.15.7:0/293314110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efffc06bde0 0x7efffc06c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.291+0000 7efff9d9b700 1 -- 172.21.15.7:0/293314110 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4075096242 0 0) 0x7efffc06c790 con 0x7efffc0adeb0 2024-01-27T02:35:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.291+0000 7efff9d9b700 1 -- 172.21.15.7:0/293314110 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effe4003570 con 0x7efffc0adeb0 2024-01-27T02:35:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.291+0000 7efff9d9b700 1 -- 172.21.15.7:0/293314110 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3420265547 0 0) 0x7efffc0c96a0 con 0x7efffc156f50 2024-01-27T02:35:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.291+0000 7efff9d9b700 1 -- 172.21.15.7:0/293314110 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efffc06c790 con 0x7efffc156f50 2024-01-27T02:35:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.291+0000 7efff9d9b700 1 -- 172.21.15.7:0/293314110 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1828869176 0 0) 0x7effe4003570 con 0x7efffc0adeb0 2024-01-27T02:35:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.291+0000 7efffa59c700 1 -- 172.21.15.7:0/293314110 >> v1:172.21.15.188:6789/0 conn(0x7efffc156f50 legacy=0x7efffc06edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.291+0000 7efffa59c700 1 -- 172.21.15.7:0/293314110 >> v1:172.21.15.7:6789/0 conn(0x7efffc0adeb0 legacy=0x7efffc0ae330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.291+0000 7efffa59c700 1 -- 172.21.15.7:0/293314110 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efffc06e5b0 con 0x7efffc06bde0 2024-01-27T02:35:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.292+0000 7efffa59c700 1 --2- 172.21.15.7:0/293314110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efffc06bde0 0x7efffc06c250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efff0009910 tx=0x7efff00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=319af8e224284d82 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:59.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.292+0000 7efff9d9b700 1 -- 172.21.15.7:0/293314110 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff40017f0 con 0x7efffc06bde0 2024-01-27T02:35:59.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.292+0000 7efff9d9b700 1 -- 172.21.15.7:0/293314110 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff0011420 con 0x7efffc06bde0 2024-01-27T02:35:59.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.292+0000 7efff9d9b700 1 -- 172.21.15.7:0/293314110 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff00117c0 con 0x7efffc06bde0 2024-01-27T02:35:59.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.292+0000 7f0001046700 1 -- 172.21.15.7:0/293314110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efffc06bde0 msgr2=0x7efffc06c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:59.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.292+0000 7f0001046700 1 --2- 172.21.15.7:0/293314110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efffc06bde0 0x7efffc06c250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efff0009910 tx=0x7efff00091a0 comp rx=0 tx=0).stop 2024-01-27T02:35:59.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.293+0000 7f0001046700 1 -- 172.21.15.7:0/293314110 shutdown_connections 2024-01-27T02:35:59.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.293+0000 7f0001046700 1 --2- 172.21.15.7:0/293314110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efffc06bde0 0x7efffc06c250 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:59.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.293+0000 7f0001046700 1 -- 172.21.15.7:0/293314110 >> 172.21.15.7:0/293314110 conn(0x7efffc068480 msgr2=0x7efffc0cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:35:59.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.293+0000 7f0001046700 1 -- 172.21.15.7:0/293314110 shutdown_connections 2024-01-27T02:35:59.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.293+0000 7f0001046700 1 -- 172.21.15.7:0/293314110 wait complete. 2024-01-27T02:35:59.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.294+0000 7f0001046700 1 Processor -- start 2024-01-27T02:35:59.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.294+0000 7f0001046700 1 -- start start 2024-01-27T02:35:59.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.295+0000 7f0001046700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efffc06bde0 0x7efffc14b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:59.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.295+0000 7f0001046700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efffc0adeb0 0x7efffc0cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:59.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.295+0000 7f0001046700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efffc156f50 0x7efffc0c16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:59.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.295+0000 7f0001046700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efffc14bac0 con 0x7efffc156f50 2024-01-27T02:35:59.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.295+0000 7f0001046700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efffc14f820 con 0x7efffc0adeb0 2024-01-27T02:35:59.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.295+0000 7f0001046700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efffc14ec10 con 0x7efffc06bde0 2024-01-27T02:35:59.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.295+0000 7efffb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efffc156f50 0x7efffc0c16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:59.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.295+0000 7efffb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efffc156f50 0x7efffc0c16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42520/0 (socket says 172.21.15.7:42520) 2024-01-27T02:35:59.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.295+0000 7efffb59e700 1 -- 172.21.15.7:0/890932846 learned_addr learned my addr 172.21.15.7:0/890932846 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:35:59.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.295+0000 7efffad9d700 1 --2- 172.21.15.7:0/890932846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efffc06bde0 0x7efffc14b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:59.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.295+0000 7efffa59c700 1 --2- 172.21.15.7:0/890932846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efffc0adeb0 0x7efffc0cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:59.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.295+0000 7efffb59e700 1 -- 172.21.15.7:0/890932846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efffc06bde0 msgr2=0x7efffc14b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:59.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.295+0000 7efffb59e700 1 --2- 172.21.15.7:0/890932846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efffc06bde0 0x7efffc14b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:59.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.295+0000 7efffb59e700 1 -- 172.21.15.7:0/890932846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efffc0adeb0 msgr2=0x7efffc0cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:35:59.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.295+0000 7efffb59e700 1 --2- 172.21.15.7:0/890932846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efffc0adeb0 0x7efffc0cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:35:59.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.295+0000 7efffb59e700 1 -- 172.21.15.7:0/890932846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff0009420 con 0x7efffc156f50 2024-01-27T02:35:59.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.295+0000 7efffa59c700 1 --2- 172.21.15.7:0/890932846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efffc0adeb0 0x7efffc0cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:35:59.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.296+0000 7efffb59e700 1 --2- 172.21.15.7:0/890932846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efffc156f50 0x7efffc0c16a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7efff400cc90 tx=0x7efff4007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:59.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.296+0000 7effe2ffd700 1 -- 172.21.15.7:0/890932846 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff400ce40 con 0x7efffc156f50 2024-01-27T02:35:59.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.296+0000 7effe2ffd700 1 -- 172.21.15.7:0/890932846 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff4004dc0 con 0x7efffc156f50 2024-01-27T02:35:59.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.296+0000 7f0001046700 1 -- 172.21.15.7:0/890932846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efffc14eef0 con 0x7efffc156f50 2024-01-27T02:35:59.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.296+0000 7effe2ffd700 1 -- 172.21.15.7:0/890932846 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff400ab10 con 0x7efffc156f50 2024-01-27T02:35:59.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.296+0000 7f0001046700 1 -- 172.21.15.7:0/890932846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efffc14f4b0 con 0x7efffc156f50 2024-01-27T02:35:59.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.297+0000 7f0001046700 1 -- 172.21.15.7:0/890932846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efffc04c710 con 0x7efffc156f50 2024-01-27T02:35:59.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.297+0000 7effe2ffd700 1 -- 172.21.15.7:0/890932846 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7efff4007150 con 0x7efffc156f50 2024-01-27T02:35:59.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.298+0000 7effe2ffd700 1 --2- 172.21.15.7:0/890932846 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7effe8066f30 0x7effe80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:35:59.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.298+0000 7effe2ffd700 1 -- 172.21.15.7:0/890932846 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4853+0+0 (secure 0 0 0) 0x7efff4083e40 con 0x7efffc156f50 2024-01-27T02:35:59.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.298+0000 7efffad9d700 1 --2- 172.21.15.7:0/890932846 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7effe8066f30 0x7effe80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:35:59.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.299+0000 7efffad9d700 1 --2- 172.21.15.7:0/890932846 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7effe8066f30 0x7effe80693e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7effec0037e0 tx=0x7effec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:35:59.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.302+0000 7effe2ffd700 1 -- 172.21.15.7:0/890932846 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efff40513e0 con 0x7efffc156f50 2024-01-27T02:35:59.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:35:59.505+0000 7f0001046700 1 -- 172.21.15.7:0/890932846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7efffc14f960 con 0x7efffc156f50 2024-01-27T02:36:00.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.065+0000 7effe2ffd700 1 -- 172.21.15.7:0/890932846 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) v1 ==== 138+0+0 (secure 0 0 0) 0x7efff4018070 con 0x7efffc156f50 2024-01-27T02:36:00.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.070+0000 7f0001046700 1 -- 172.21.15.7:0/890932846 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7effe8066f30 msgr2=0x7effe80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:00.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.070+0000 7f0001046700 1 --2- 172.21.15.7:0/890932846 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7effe8066f30 0x7effe80693e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7effec0037e0 tx=0x7effec009040 comp rx=0 tx=0).stop 2024-01-27T02:36:00.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.070+0000 7f0001046700 1 -- 172.21.15.7:0/890932846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efffc156f50 msgr2=0x7efffc0c16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:00.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.070+0000 7f0001046700 1 --2- 172.21.15.7:0/890932846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efffc156f50 0x7efffc0c16a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7efff400cc90 tx=0x7efff4007d20 comp rx=0 tx=0).stop 2024-01-27T02:36:00.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.070+0000 7f0001046700 1 -- 172.21.15.7:0/890932846 shutdown_connections 2024-01-27T02:36:00.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.070+0000 7f0001046700 1 --2- 172.21.15.7:0/890932846 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7effe8066f30 0x7effe80693e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:00.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.070+0000 7f0001046700 1 --2- 172.21.15.7:0/890932846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efffc156f50 0x7efffc0c16a0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:00.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.070+0000 7f0001046700 1 --2- 172.21.15.7:0/890932846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efffc06bde0 0x7efffc14b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:00.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.070+0000 7f0001046700 1 --2- 172.21.15.7:0/890932846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efffc0adeb0 0x7efffc0cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:00.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.070+0000 7f0001046700 1 -- 172.21.15.7:0/890932846 >> 172.21.15.7:0/890932846 conn(0x7efffc068480 msgr2=0x7efffc06ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:00.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.070+0000 7f0001046700 1 -- 172.21.15.7:0/890932846 shutdown_connections 2024-01-27T02:36:00.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.070+0000 7f0001046700 1 -- 172.21.15.7:0/890932846 wait complete. 2024-01-27T02:36:00.072 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-27T02:36:00.086 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-27T02:36:00.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.229+0000 7f4c840c8700 1 Processor -- start 2024-01-27T02:36:00.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.229+0000 7f4c840c8700 1 -- start start 2024-01-27T02:36:00.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.230+0000 7f4c840c8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c7c0cf280 0x7f4c7c0d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:00.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.230+0000 7f4c840c8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c7c0b38d0 con 0x7f4c7c15d460 2024-01-27T02:36:00.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.230+0000 7f4c840c8700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c7c0d0390 con 0x7f4c7c15d830 2024-01-27T02:36:00.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.230+0000 7f4c840c8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c0cf780 con 0x7f4c7c0cf280 2024-01-27T02:36:00.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.230+0000 7f4c81e64700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4c7c15d460 0x7f4c7c15e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34290/0 (socket says 172.21.15.7:34290) 2024-01-27T02:36:00.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.230+0000 7f4c81e64700 1 -- 172.21.15.7:0/2478823313 learned_addr learned my addr 172.21.15.7:0/2478823313 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:00.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.230+0000 7f4c81663700 1 --2- 172.21.15.7:0/2478823313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c7c0cf280 0x7f4c7c0d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:00.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.231+0000 7f4c80e62700 1 -- 172.21.15.7:0/2478823313 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840012410 0 0) 0x7f4c78001110 con 0x7f4c7c15d460 2024-01-27T02:36:00.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.231+0000 7f4c80e62700 1 -- 172.21.15.7:0/2478823313 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c68003570 con 0x7f4c7c15d460 2024-01-27T02:36:00.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.231+0000 7f4c80e62700 1 -- 172.21.15.7:0/2478823313 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558274561 0 0) 0x7f4c7c0d0390 con 0x7f4c7c15d830 2024-01-27T02:36:00.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.231+0000 7f4c80e62700 1 -- 172.21.15.7:0/2478823313 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c78001110 con 0x7f4c7c15d830 2024-01-27T02:36:00.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.231+0000 7f4c80e62700 1 -- 172.21.15.7:0/2478823313 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3953808337 0 0) 0x7f4c68003570 con 0x7f4c7c15d460 2024-01-27T02:36:00.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.231+0000 7f4c80e62700 1 -- 172.21.15.7:0/2478823313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c7c0cf280 msgr2=0x7f4c7c0d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:00.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.231+0000 7f4c80e62700 1 --2- 172.21.15.7:0/2478823313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c7c0cf280 0x7f4c7c0d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:00.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.231+0000 7f4c80e62700 1 -- 172.21.15.7:0/2478823313 >> v1:172.21.15.188:6789/0 conn(0x7f4c7c15d830 legacy=0x7f4c7c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:00.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.231+0000 7f4c80e62700 1 -- 172.21.15.7:0/2478823313 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7c0cfa60 con 0x7f4c7c15d460 2024-01-27T02:36:00.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.231+0000 7f4c80e62700 1 -- 172.21.15.7:0/2478823313 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f4c78001600 con 0x7f4c7c15d460 2024-01-27T02:36:00.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.231+0000 7f4c81663700 1 --2- 172.21.15.7:0/2478823313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c7c0cf280 0x7f4c7c0d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:36:00.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.231+0000 7f4c80e62700 1 -- 172.21.15.7:0/2478823313 >> v1:172.21.15.7:6789/0 conn(0x7f4c7c15d460 legacy=0x7f4c7c15e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:00.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.231+0000 7f4c80e62700 1 --2- 172.21.15.7:0/2478823313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c68003c20 0x7f4c68004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:00.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.231+0000 7f4c80e62700 1 --2- 172.21.15.7:0/2478823313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c7c0cf280 0x7f4c68004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:00.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.231+0000 7f4c80e62700 1 --2- 172.21.15.7:0/2478823313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c68004d50 0x7f4c680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:00.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.231+0000 7f4c80e62700 1 -- 172.21.15.7:0/2478823313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c0cf780 con 0x7f4c68004d50 2024-01-27T02:36:00.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.231+0000 7f4c80e62700 1 -- 172.21.15.7:0/2478823313 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6800bb10 con 0x7f4c68003c20 2024-01-27T02:36:00.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.231+0000 7f4c80e62700 1 -- 172.21.15.7:0/2478823313 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6800bcd0 con 0x7f4c7c0cf280 2024-01-27T02:36:00.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.231+0000 7f4c82665700 1 --2- 172.21.15.7:0/2478823313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c68004d50 0x7f4c680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:00.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.231+0000 7f4c81e64700 1 --2- 172.21.15.7:0/2478823313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c68003c20 0x7f4c68004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:00.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.231+0000 7f4c81663700 1 --2- 172.21.15.7:0/2478823313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c7c0cf280 0x7f4c68004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:00.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.232+0000 7f4c82665700 1 -- 172.21.15.7:0/2478823313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c7c0cf280 msgr2=0x7f4c68004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:00.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.232+0000 7f4c82665700 1 --2- 172.21.15.7:0/2478823313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c7c0cf280 0x7f4c68004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:00.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.232+0000 7f4c82665700 1 -- 172.21.15.7:0/2478823313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c68003c20 msgr2=0x7f4c68004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:00.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.232+0000 7f4c82665700 1 --2- 172.21.15.7:0/2478823313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c68003c20 0x7f4c68004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:00.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.232+0000 7f4c82665700 1 -- 172.21.15.7:0/2478823313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c70009420 con 0x7f4c68004d50 2024-01-27T02:36:00.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.232+0000 7f4c82665700 1 --2- 172.21.15.7:0/2478823313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c68004d50 0x7f4c680051c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4c74002160 tx=0x7f4c740017e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18a7928ce0a1638 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:00.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.232+0000 7f4c80e62700 1 -- 172.21.15.7:0/2478823313 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c74001af0 con 0x7f4c68004d50 2024-01-27T02:36:00.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.232+0000 7f4c80e62700 1 -- 172.21.15.7:0/2478823313 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c7400c980 con 0x7f4c68004d50 2024-01-27T02:36:00.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.232+0000 7f4c80e62700 1 -- 172.21.15.7:0/2478823313 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7400cd20 con 0x7f4c68004d50 2024-01-27T02:36:00.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.232+0000 7f4c840c8700 1 -- 172.21.15.7:0/2478823313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c68004d50 msgr2=0x7f4c680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:00.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.232+0000 7f4c840c8700 1 --2- 172.21.15.7:0/2478823313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c68004d50 0x7f4c680051c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4c74002160 tx=0x7f4c740017e0 comp rx=0 tx=0).stop 2024-01-27T02:36:00.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.232+0000 7f4c82665700 1 -- 172.21.15.7:0/2478823313 reap_dead start 2024-01-27T02:36:00.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.233+0000 7f4c840c8700 1 -- 172.21.15.7:0/2478823313 shutdown_connections 2024-01-27T02:36:00.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.233+0000 7f4c840c8700 1 -- 172.21.15.7:0/2478823313 >> 172.21.15.7:0/2478823313 conn(0x7f4c7c063580 msgr2=0x7f4c7c063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:00.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.233+0000 7f4c840c8700 1 -- 172.21.15.7:0/2478823313 shutdown_connections 2024-01-27T02:36:00.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.233+0000 7f4c840c8700 1 -- 172.21.15.7:0/2478823313 wait complete. 2024-01-27T02:36:00.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.234+0000 7f4c840c8700 1 Processor -- start 2024-01-27T02:36:00.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.234+0000 7f4c840c8700 1 -- start start 2024-01-27T02:36:00.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.235+0000 7f4c840c8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c68004d50 0x7f4c7c0c9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:00.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.235+0000 7f4c840c8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c7c16b1b0 0x7f4c7c0b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:00.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.235+0000 7f4c840c8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c7c0b23a0 0x7f4c7c0b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:00.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.235+0000 7f4c840c8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c0cafa0 con 0x7f4c7c16b1b0 2024-01-27T02:36:00.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.235+0000 7f4c840c8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c0cab10 con 0x7f4c7c0b23a0 2024-01-27T02:36:00.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.235+0000 7f4c840c8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c0b2d50 con 0x7f4c68004d50 2024-01-27T02:36:00.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.235+0000 7f4c81663700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c7c16b1b0 0x7f4c7c0b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:00.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.235+0000 7f4c81663700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c7c16b1b0 0x7f4c7c0b5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42540/0 (socket says 172.21.15.7:42540) 2024-01-27T02:36:00.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.235+0000 7f4c81663700 1 -- 172.21.15.7:0/3281079009 learned_addr learned my addr 172.21.15.7:0/3281079009 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:00.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.235+0000 7f4c81e64700 1 --2- 172.21.15.7:0/3281079009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c68004d50 0x7f4c7c0c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:00.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.235+0000 7f4c82665700 1 --2- 172.21.15.7:0/3281079009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c7c0b23a0 0x7f4c7c0b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:00.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.235+0000 7f4c81663700 1 -- 172.21.15.7:0/3281079009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c68004d50 msgr2=0x7f4c7c0c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:00.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.235+0000 7f4c81663700 1 --2- 172.21.15.7:0/3281079009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c68004d50 0x7f4c7c0c9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:00.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.235+0000 7f4c81663700 1 -- 172.21.15.7:0/3281079009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c7c0b23a0 msgr2=0x7f4c7c0b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:00.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.235+0000 7f4c81663700 1 --2- 172.21.15.7:0/3281079009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c7c0b23a0 0x7f4c7c0b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:00.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.235+0000 7f4c81663700 1 -- 172.21.15.7:0/3281079009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7c0c9d90 con 0x7f4c7c16b1b0 2024-01-27T02:36:00.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.235+0000 7f4c81e64700 1 --2- 172.21.15.7:0/3281079009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c68004d50 0x7f4c7c0c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:36:00.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.235+0000 7f4c81663700 1 --2- 172.21.15.7:0/3281079009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c7c16b1b0 0x7f4c7c0b5260 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4c70011da0 tx=0x7f4c700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:00.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.235+0000 7f4c6dffb700 1 -- 172.21.15.7:0/3281079009 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7000fcd0 con 0x7f4c7c16b1b0 2024-01-27T02:36:00.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.235+0000 7f4c6dffb700 1 -- 172.21.15.7:0/3281079009 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c7000fe30 con 0x7f4c7c16b1b0 2024-01-27T02:36:00.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.235+0000 7f4c840c8700 1 -- 172.21.15.7:0/3281079009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c7c16b620 con 0x7f4c7c16b1b0 2024-01-27T02:36:00.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.235+0000 7f4c6dffb700 1 -- 172.21.15.7:0/3281079009 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c70014690 con 0x7f4c7c16b1b0 2024-01-27T02:36:00.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.236+0000 7f4c840c8700 1 -- 172.21.15.7:0/3281079009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7c0ca1e0 con 0x7f4c7c16b1b0 2024-01-27T02:36:00.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.237+0000 7f4c840c8700 1 -- 172.21.15.7:0/3281079009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c7c0cacc0 con 0x7f4c7c16b1b0 2024-01-27T02:36:00.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.237+0000 7f4c6dffb700 1 -- 172.21.15.7:0/3281079009 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4c70014a80 con 0x7f4c7c16b1b0 2024-01-27T02:36:00.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.238+0000 7f4c6dffb700 1 --2- 172.21.15.7:0/3281079009 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4c60066f30 0x7f4c600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:00.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.238+0000 7f4c6dffb700 1 -- 172.21.15.7:0/3281079009 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f4c70018030 con 0x7f4c7c16b1b0 2024-01-27T02:36:00.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.238+0000 7f4c81e64700 1 --2- 172.21.15.7:0/3281079009 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4c60066f30 0x7f4c600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:00.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.238+0000 7f4c81e64700 1 --2- 172.21.15.7:0/3281079009 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4c60066f30 0x7f4c600693e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4c78000e10 tx=0x7f4c78009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:00.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.242+0000 7f4c6dffb700 1 -- 172.21.15.7:0/3281079009 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c7c0cacc0 con 0x7f4c7c16b1b0 2024-01-27T02:36:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:00.448+0000 7f4c840c8700 1 -- 172.21.15.7:0/3281079009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4c7c0cacc0 con 0x7f4c7c16b1b0 2024-01-27T02:36:01.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.069+0000 7f4c6dffb700 1 -- 172.21.15.7:0/3281079009 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4c7c0cacc0 con 0x7f4c7c16b1b0 2024-01-27T02:36:01.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.074+0000 7f4c840c8700 1 -- 172.21.15.7:0/3281079009 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4c60066f30 msgr2=0x7f4c600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:01.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.074+0000 7f4c840c8700 1 --2- 172.21.15.7:0/3281079009 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4c60066f30 0x7f4c600693e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4c78000e10 tx=0x7f4c78009040 comp rx=0 tx=0).stop 2024-01-27T02:36:01.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.074+0000 7f4c840c8700 1 -- 172.21.15.7:0/3281079009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c7c16b1b0 msgr2=0x7f4c7c0b5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:01.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.074+0000 7f4c840c8700 1 --2- 172.21.15.7:0/3281079009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c7c16b1b0 0x7f4c7c0b5260 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4c70011da0 tx=0x7f4c700091a0 comp rx=0 tx=0).stop 2024-01-27T02:36:01.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.074+0000 7f4c840c8700 1 -- 172.21.15.7:0/3281079009 shutdown_connections 2024-01-27T02:36:01.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.074+0000 7f4c840c8700 1 --2- 172.21.15.7:0/3281079009 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4c60066f30 0x7f4c600693e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:01.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.074+0000 7f4c840c8700 1 --2- 172.21.15.7:0/3281079009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c7c0b23a0 0x7f4c7c0b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:01.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.074+0000 7f4c840c8700 1 --2- 172.21.15.7:0/3281079009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c68004d50 0x7f4c7c0c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:01.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.074+0000 7f4c840c8700 1 --2- 172.21.15.7:0/3281079009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c7c16b1b0 0x7f4c7c0b5260 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:01.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.074+0000 7f4c840c8700 1 -- 172.21.15.7:0/3281079009 >> 172.21.15.7:0/3281079009 conn(0x7f4c7c063580 msgr2=0x7f4c7c0c6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:01.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.074+0000 7f4c840c8700 1 -- 172.21.15.7:0/3281079009 shutdown_connections 2024-01-27T02:36:01.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.074+0000 7f4c840c8700 1 -- 172.21.15.7:0/3281079009 wait complete. 2024-01-27T02:36:01.077 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-27T02:36:01.098 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-27T02:36:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.238+0000 7ff59c3ea700 1 Processor -- start 2024-01-27T02:36:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.238+0000 7ff59c3ea700 1 -- start start 2024-01-27T02:36:01.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.239+0000 7ff59c3ea700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff5941416c0 0x7ff5940cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:01.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.239+0000 7ff59c3ea700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff5940cb210 0x7ff5940cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.239+0000 7ff59c3ea700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff59415c7d0 con 0x7ff594141a90 2024-01-27T02:36:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.239+0000 7ff59c3ea700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5940cd9c0 con 0x7ff5940cb210 2024-01-27T02:36:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.239+0000 7ff59c3ea700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5940c9640 con 0x7ff5941416c0 2024-01-27T02:36:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.239+0000 7ff59a987700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff5940cb210 0x7ff5940cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.239+0000 7ff599985700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff594141a90 0x7ff5940cab00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34292/0 (socket says 172.21.15.7:34292) 2024-01-27T02:36:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.239+0000 7ff599985700 1 -- 172.21.15.7:0/1576906946 learned_addr learned my addr 172.21.15.7:0/1576906946 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.240+0000 7ff59a186700 1 --2- 172.21.15.7:0/1576906946 >> v2:172.21.15.188:3300/0 conn(0x7ff5941416c0 0x7ff5940cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.240+0000 7ff59a987700 1 -- 172.21.15.7:0/1576906946 >> v2:172.21.15.188:3300/0 conn(0x7ff5941416c0 msgr2=0x7ff5940cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.240+0000 7ff59a987700 1 --2- 172.21.15.7:0/1576906946 >> v2:172.21.15.188:3300/0 conn(0x7ff5941416c0 0x7ff5940cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.240+0000 7ff59a987700 1 -- 172.21.15.7:0/1576906946 >> v1:172.21.15.7:6789/0 conn(0x7ff594141a90 legacy=0x7ff5940cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.240+0000 7ff59a987700 1 -- 172.21.15.7:0/1576906946 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59415bbc0 con 0x7ff5940cb210 2024-01-27T02:36:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.240+0000 7ff59a987700 1 --2- 172.21.15.7:0/1576906946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5940cb210 0x7ff5940cbc20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff5940cb6b0 tx=0x7ff58c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365e341eebe7a907 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.240+0000 7ff599184700 1 -- 172.21.15.7:0/1576906946 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c00e070 con 0x7ff5940cb210 2024-01-27T02:36:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.240+0000 7ff599184700 1 -- 172.21.15.7:0/1576906946 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c00bc30 con 0x7ff5940cb210 2024-01-27T02:36:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.240+0000 7ff599184700 1 -- 172.21.15.7:0/1576906946 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c012430 con 0x7ff5940cb210 2024-01-27T02:36:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.241+0000 7ff59c3ea700 1 -- 172.21.15.7:0/1576906946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5940cb210 msgr2=0x7ff5940cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.241+0000 7ff59c3ea700 1 --2- 172.21.15.7:0/1576906946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5940cb210 0x7ff5940cbc20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff5940cb6b0 tx=0x7ff58c009780 comp rx=0 tx=0).stop 2024-01-27T02:36:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.241+0000 7ff59c3ea700 1 -- 172.21.15.7:0/1576906946 shutdown_connections 2024-01-27T02:36:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.241+0000 7ff59c3ea700 1 --2- 172.21.15.7:0/1576906946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5940cb210 0x7ff5940cbc20 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.241+0000 7ff59c3ea700 1 --2- 172.21.15.7:0/1576906946 >> v2:172.21.15.188:3300/0 conn(0x7ff5941416c0 0x7ff5940cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.241+0000 7ff59c3ea700 1 -- 172.21.15.7:0/1576906946 >> 172.21.15.7:0/1576906946 conn(0x7ff594063580 msgr2=0x7ff594063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.241+0000 7ff59c3ea700 1 -- 172.21.15.7:0/1576906946 shutdown_connections 2024-01-27T02:36:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.241+0000 7ff59c3ea700 1 -- 172.21.15.7:0/1576906946 wait complete. 2024-01-27T02:36:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.242+0000 7ff59c3ea700 1 Processor -- start 2024-01-27T02:36:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.242+0000 7ff59c3ea700 1 -- start start 2024-01-27T02:36:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.243+0000 7ff59c3ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5940cb210 0x7ff59413d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.243+0000 7ff59c3ea700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5941416c0 0x7ff5940b2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.243+0000 7ff59c3ea700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff594141a90 0x7ff5940cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.243+0000 7ff59c3ea700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff594168300 con 0x7ff5940cb210 2024-01-27T02:36:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.243+0000 7ff59c3ea700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5940b2ee0 con 0x7ff5941416c0 2024-01-27T02:36:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.243+0000 7ff59c3ea700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5940b31c0 con 0x7ff594141a90 2024-01-27T02:36:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.243+0000 7ff59a186700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5940cb210 0x7ff59413d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.243+0000 7ff59a186700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5940cb210 0x7ff59413d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42554/0 (socket says 172.21.15.7:42554) 2024-01-27T02:36:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.243+0000 7ff59a186700 1 -- 172.21.15.7:0/436944133 learned_addr learned my addr 172.21.15.7:0/436944133 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.243+0000 7ff599985700 1 --2- 172.21.15.7:0/436944133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5941416c0 0x7ff5940b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.243+0000 7ff59a987700 1 --2- 172.21.15.7:0/436944133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff594141a90 0x7ff5940cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.243+0000 7ff59a186700 1 -- 172.21.15.7:0/436944133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff594141a90 msgr2=0x7ff5940cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.243+0000 7ff59a186700 1 --2- 172.21.15.7:0/436944133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff594141a90 0x7ff5940cc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.243+0000 7ff59a186700 1 -- 172.21.15.7:0/436944133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5941416c0 msgr2=0x7ff5940b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.243+0000 7ff59a186700 1 --2- 172.21.15.7:0/436944133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5941416c0 0x7ff5940b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.243+0000 7ff59a186700 1 -- 172.21.15.7:0/436944133 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58c009a20 con 0x7ff5940cb210 2024-01-27T02:36:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.244+0000 7ff59a186700 1 --2- 172.21.15.7:0/436944133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5940cb210 0x7ff59413d2a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff58800d810 tx=0x7ff588008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.244+0000 7ff5867fc700 1 -- 172.21.15.7:0/436944133 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5880125f0 con 0x7ff5940cb210 2024-01-27T02:36:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.244+0000 7ff5867fc700 1 -- 172.21.15.7:0/436944133 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff588012750 con 0x7ff5940cb210 2024-01-27T02:36:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.244+0000 7ff5867fc700 1 -- 172.21.15.7:0/436944133 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff588012b10 con 0x7ff5940cb210 2024-01-27T02:36:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.244+0000 7ff59c3ea700 1 -- 172.21.15.7:0/436944133 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5940b34a0 con 0x7ff5940cb210 2024-01-27T02:36:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.244+0000 7ff59c3ea700 1 -- 172.21.15.7:0/436944133 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5940c7e30 con 0x7ff5940cb210 2024-01-27T02:36:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.245+0000 7ff59c3ea700 1 -- 172.21.15.7:0/436944133 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff594168650 con 0x7ff5940cb210 2024-01-27T02:36:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.245+0000 7ff5867fc700 1 -- 172.21.15.7:0/436944133 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff58801e460 con 0x7ff5940cb210 2024-01-27T02:36:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.246+0000 7ff5867fc700 1 --2- 172.21.15.7:0/436944133 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff578066f30 0x7ff5780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.246+0000 7ff5867fc700 1 -- 172.21.15.7:0/436944133 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5605+0+0 (secure 0 0 0) 0x7ff588084a90 con 0x7ff5940cb210 2024-01-27T02:36:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.246+0000 7ff599985700 1 --2- 172.21.15.7:0/436944133 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff578066f30 0x7ff5780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.247+0000 7ff599985700 1 --2- 172.21.15.7:0/436944133 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff578066f30 0x7ff5780693e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff59406b830 tx=0x7ff590009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:01.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.250+0000 7ff5867fc700 1 -- 172.21.15.7:0/436944133 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff588051d60 con 0x7ff5940cb210 2024-01-27T02:36:01.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:01.501+0000 7ff59c3ea700 1 -- 172.21.15.7:0/436944133 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff594168440 con 0x7ff5940cb210 2024-01-27T02:36:02.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.102+0000 7ff5867fc700 1 -- 172.21.15.7:0/436944133 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v38) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff588055380 con 0x7ff5940cb210 2024-01-27T02:36:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.105+0000 7ff59c3ea700 1 -- 172.21.15.7:0/436944133 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff578066f30 msgr2=0x7ff5780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.105+0000 7ff59c3ea700 1 --2- 172.21.15.7:0/436944133 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff578066f30 0x7ff5780693e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff59406b830 tx=0x7ff590009040 comp rx=0 tx=0).stop 2024-01-27T02:36:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.105+0000 7ff59c3ea700 1 -- 172.21.15.7:0/436944133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5940cb210 msgr2=0x7ff59413d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.105+0000 7ff59c3ea700 1 --2- 172.21.15.7:0/436944133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5940cb210 0x7ff59413d2a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff58800d810 tx=0x7ff588008040 comp rx=0 tx=0).stop 2024-01-27T02:36:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.105+0000 7ff59c3ea700 1 -- 172.21.15.7:0/436944133 shutdown_connections 2024-01-27T02:36:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.105+0000 7ff59c3ea700 1 --2- 172.21.15.7:0/436944133 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff578066f30 0x7ff5780693e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.105+0000 7ff59c3ea700 1 --2- 172.21.15.7:0/436944133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff594141a90 0x7ff5940cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.105+0000 7ff59c3ea700 1 --2- 172.21.15.7:0/436944133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5940cb210 0x7ff59413d2a0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.105+0000 7ff59c3ea700 1 --2- 172.21.15.7:0/436944133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5941416c0 0x7ff5940b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.105+0000 7ff59c3ea700 1 -- 172.21.15.7:0/436944133 >> 172.21.15.7:0/436944133 conn(0x7ff594063580 msgr2=0x7ff59413e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:02.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.106+0000 7ff59c3ea700 1 -- 172.21.15.7:0/436944133 shutdown_connections 2024-01-27T02:36:02.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.106+0000 7ff59c3ea700 1 -- 172.21.15.7:0/436944133 wait complete. 2024-01-27T02:36:02.109 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 10 and data pool 11 2024-01-27T02:36:02.125 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:36:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.260+0000 7fc95b195700 1 Processor -- start 2024-01-27T02:36:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.260+0000 7fc95b195700 1 -- start start 2024-01-27T02:36:02.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.260+0000 7fc95b195700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc95415a100 0x7fc9540cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.261+0000 7fc95b195700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9540c4cc0 0x7fc954163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.261+0000 7fc95b195700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9540ce5e0 con 0x7fc9540cb5b0 2024-01-27T02:36:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.261+0000 7fc95b195700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9540cd9d0 con 0x7fc95415a100 2024-01-27T02:36:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.261+0000 7fc95b195700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9540cdcb0 con 0x7fc9540c4cc0 2024-01-27T02:36:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.261+0000 7fc953fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc9540cb5b0 0x7fc9540c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40602/0 (socket says 172.21.15.7:40602) 2024-01-27T02:36:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.261+0000 7fc953fff700 1 -- 172.21.15.7:0/2591311754 learned_addr learned my addr 172.21.15.7:0/2591311754 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.261+0000 7fc958f31700 1 --2- 172.21.15.7:0/2591311754 >> v2:172.21.15.188:3300/0 conn(0x7fc95415a100 0x7fc9540cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.261+0000 7fc959732700 1 --2- 172.21.15.7:0/2591311754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9540c4cc0 0x7fc954163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.261+0000 7fc9537fe700 1 -- 172.21.15.7:0/2591311754 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926985751 0 0) 0x7fc9540ce5e0 con 0x7fc9540cb5b0 2024-01-27T02:36:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.261+0000 7fc9537fe700 1 -- 172.21.15.7:0/2591311754 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc93c003570 con 0x7fc9540cb5b0 2024-01-27T02:36:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.261+0000 7fc9537fe700 1 -- 172.21.15.7:0/2591311754 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 9892355 0 0) 0x7fc93c003570 con 0x7fc9540cb5b0 2024-01-27T02:36:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.261+0000 7fc9537fe700 1 -- 172.21.15.7:0/2591311754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9540c4cc0 msgr2=0x7fc954163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.261+0000 7fc9537fe700 1 --2- 172.21.15.7:0/2591311754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9540c4cc0 0x7fc954163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.261+0000 7fc9537fe700 1 -- 172.21.15.7:0/2591311754 >> v2:172.21.15.188:3300/0 conn(0x7fc95415a100 msgr2=0x7fc9540cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.261+0000 7fc9537fe700 1 --2- 172.21.15.7:0/2591311754 >> v2:172.21.15.188:3300/0 conn(0x7fc95415a100 0x7fc9540cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.261+0000 7fc9537fe700 1 -- 172.21.15.7:0/2591311754 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9540cdf90 con 0x7fc9540cb5b0 2024-01-27T02:36:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.261+0000 7fc9537fe700 1 -- 172.21.15.7:0/2591311754 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7fc948001470 con 0x7fc9540cb5b0 2024-01-27T02:36:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.261+0000 7fc9537fe700 1 -- 172.21.15.7:0/2591311754 >> v1:172.21.15.7:6789/0 conn(0x7fc9540cb5b0 legacy=0x7fc9540c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.261+0000 7fc9537fe700 1 --2- 172.21.15.7:0/2591311754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9540c4cc0 0x7fc93c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.261+0000 7fc9537fe700 1 --2- 172.21.15.7:0/2591311754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc93c003fe0 0x7fc93c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.261+0000 7fc9537fe700 1 --2- 172.21.15.7:0/2591311754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc93c004d50 0x7fc93c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.262+0000 7fc9537fe700 1 -- 172.21.15.7:0/2591311754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9540cd9d0 con 0x7fc93c003fe0 2024-01-27T02:36:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.262+0000 7fc9537fe700 1 -- 172.21.15.7:0/2591311754 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9540cdcb0 con 0x7fc93c004d50 2024-01-27T02:36:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.262+0000 7fc9537fe700 1 -- 172.21.15.7:0/2591311754 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c00ba90 con 0x7fc9540c4cc0 2024-01-27T02:36:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.262+0000 7fc958f31700 1 --2- 172.21.15.7:0/2591311754 >> v2:172.21.15.188:3300/0 conn(0x7fc95415a100 0x7fc9540cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:36:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.262+0000 7fc953fff700 1 --2- 172.21.15.7:0/2591311754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc93c003fe0 0x7fc93c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.262+0000 7fc959732700 1 --2- 172.21.15.7:0/2591311754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc93c004d50 0x7fc93c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.262+0000 7fc958f31700 1 --2- 172.21.15.7:0/2591311754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9540c4cc0 0x7fc93c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.262+0000 7fc953fff700 1 -- 172.21.15.7:0/2591311754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9540c4cc0 msgr2=0x7fc93c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.262+0000 7fc953fff700 1 --2- 172.21.15.7:0/2591311754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9540c4cc0 0x7fc93c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.262+0000 7fc953fff700 1 -- 172.21.15.7:0/2591311754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc93c004d50 msgr2=0x7fc93c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.262+0000 7fc953fff700 1 --2- 172.21.15.7:0/2591311754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc93c004d50 0x7fc93c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.262+0000 7fc953fff700 1 -- 172.21.15.7:0/2591311754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93c003570 con 0x7fc93c003fe0 2024-01-27T02:36:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.262+0000 7fc953fff700 1 --2- 172.21.15.7:0/2591311754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc93c003fe0 0x7fc93c004810 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc94800c3c0 tx=0x7fc948001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23780534ab94b2b server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.262+0000 7fc9537fe700 1 -- 172.21.15.7:0/2591311754 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94800cc30 con 0x7fc93c003fe0 2024-01-27T02:36:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.262+0000 7fc9537fe700 1 -- 172.21.15.7:0/2591311754 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94800cdd0 con 0x7fc93c003fe0 2024-01-27T02:36:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.262+0000 7fc9537fe700 1 -- 172.21.15.7:0/2591311754 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948005b70 con 0x7fc93c003fe0 2024-01-27T02:36:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.263+0000 7fc95b195700 1 -- 172.21.15.7:0/2591311754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc93c003fe0 msgr2=0x7fc93c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.263+0000 7fc95b195700 1 --2- 172.21.15.7:0/2591311754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc93c003fe0 0x7fc93c004810 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc94800c3c0 tx=0x7fc948001e90 comp rx=0 tx=0).stop 2024-01-27T02:36:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.263+0000 7fc959732700 1 -- 172.21.15.7:0/2591311754 reap_dead start 2024-01-27T02:36:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.263+0000 7fc95b195700 1 -- 172.21.15.7:0/2591311754 shutdown_connections 2024-01-27T02:36:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.263+0000 7fc95b195700 1 -- 172.21.15.7:0/2591311754 >> 172.21.15.7:0/2591311754 conn(0x7fc954098020 msgr2=0x7fc9540711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.263+0000 7fc95b195700 1 -- 172.21.15.7:0/2591311754 shutdown_connections 2024-01-27T02:36:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.263+0000 7fc95b195700 1 -- 172.21.15.7:0/2591311754 wait complete. 2024-01-27T02:36:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.264+0000 7fc95b195700 1 Processor -- start 2024-01-27T02:36:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.264+0000 7fc95b195700 1 -- start start 2024-01-27T02:36:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.265+0000 7fc95b195700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc93c003fe0 0x7fc95416b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.265+0000 7fc95b195700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9540b3910 0x7fc954154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.265+0000 7fc95b195700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc954068a80 0x7fc954068ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.265+0000 7fc95b195700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95416b700 con 0x7fc954068a80 2024-01-27T02:36:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.265+0000 7fc95b195700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95415a4d0 con 0x7fc9540b3910 2024-01-27T02:36:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.265+0000 7fc95b195700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc954069430 con 0x7fc93c003fe0 2024-01-27T02:36:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.265+0000 7fc959732700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc954068a80 0x7fc954068ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.265+0000 7fc959732700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc954068a80 0x7fc954068ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33368/0 (socket says 172.21.15.7:33368) 2024-01-27T02:36:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.265+0000 7fc959732700 1 -- 172.21.15.7:0/2695767901 learned_addr learned my addr 172.21.15.7:0/2695767901 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.266+0000 7fc953fff700 1 --2- 172.21.15.7:0/2695767901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9540b3910 0x7fc954154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.266+0000 7fc958f31700 1 --2- 172.21.15.7:0/2695767901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc93c003fe0 0x7fc95416b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.266+0000 7fc959732700 1 -- 172.21.15.7:0/2695767901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc93c003fe0 msgr2=0x7fc95416b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.266+0000 7fc959732700 1 --2- 172.21.15.7:0/2695767901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc93c003fe0 0x7fc95416b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.266+0000 7fc959732700 1 -- 172.21.15.7:0/2695767901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9540b3910 msgr2=0x7fc954154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.266+0000 7fc959732700 1 --2- 172.21.15.7:0/2695767901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9540b3910 0x7fc954154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.266+0000 7fc959732700 1 -- 172.21.15.7:0/2695767901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9540b4530 con 0x7fc954068a80 2024-01-27T02:36:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.266+0000 7fc959732700 1 --2- 172.21.15.7:0/2695767901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc954068a80 0x7fc954068ef0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc9540c9330 tx=0x7fc94c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.266+0000 7fc950ff9700 1 -- 172.21.15.7:0/2695767901 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c00e040 con 0x7fc954068a80 2024-01-27T02:36:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.266+0000 7fc950ff9700 1 -- 172.21.15.7:0/2695767901 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94c00f040 con 0x7fc954068a80 2024-01-27T02:36:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.266+0000 7fc950ff9700 1 -- 172.21.15.7:0/2695767901 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c00fe50 con 0x7fc954068a80 2024-01-27T02:36:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.266+0000 7fc95b195700 1 -- 172.21.15.7:0/2695767901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc95416c570 con 0x7fc954068a80 2024-01-27T02:36:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.266+0000 7fc95b195700 1 -- 172.21.15.7:0/2695767901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95416b960 con 0x7fc954068a80 2024-01-27T02:36:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.267+0000 7fc95b195700 1 -- 172.21.15.7:0/2695767901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc95416c700 con 0x7fc954068a80 2024-01-27T02:36:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.268+0000 7fc950ff9700 1 -- 172.21.15.7:0/2695767901 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc94c015640 con 0x7fc954068a80 2024-01-27T02:36:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.268+0000 7fc950ff9700 1 --2- 172.21.15.7:0/2695767901 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc940066f30 0x7fc9400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.268+0000 7fc950ff9700 1 -- 172.21.15.7:0/2695767901 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc94c084bd0 con 0x7fc954068a80 2024-01-27T02:36:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.268+0000 7fc958f31700 1 --2- 172.21.15.7:0/2695767901 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc940066f30 0x7fc9400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.269+0000 7fc958f31700 1 --2- 172.21.15.7:0/2695767901 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc940066f30 0x7fc9400693e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc9440093f0 tx=0x7fc944008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.273+0000 7fc950ff9700 1 -- 172.21.15.7:0/2695767901 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc94c0558d0 con 0x7fc954068a80 2024-01-27T02:36:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.476+0000 7fc95b195700 1 -- 172.21.15.7:0/2695767901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9540b4530 con 0x7fc954068a80 2024-01-27T02:36:02.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.477+0000 7fc950ff9700 1 -- 172.21.15.7:0/2695767901 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+15318 (secure 0 0 0) 0x7fc9540b4530 con 0x7fc954068a80 2024-01-27T02:36:02.478 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:36:02.478 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":80,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:36:01.502486+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:35:59.537952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-27T02:36:00.480234+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:36:02.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.480+0000 7fc95b195700 1 -- 172.21.15.7:0/2695767901 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc940066f30 msgr2=0x7fc9400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:02.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.480+0000 7fc95b195700 1 --2- 172.21.15.7:0/2695767901 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc940066f30 0x7fc9400693e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc9440093f0 tx=0x7fc944008040 comp rx=0 tx=0).stop 2024-01-27T02:36:02.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.480+0000 7fc95b195700 1 -- 172.21.15.7:0/2695767901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc954068a80 msgr2=0x7fc954068ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:02.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.480+0000 7fc95b195700 1 --2- 172.21.15.7:0/2695767901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc954068a80 0x7fc954068ef0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc9540c9330 tx=0x7fc94c00b040 comp rx=0 tx=0).stop 2024-01-27T02:36:02.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.480+0000 7fc95b195700 1 -- 172.21.15.7:0/2695767901 shutdown_connections 2024-01-27T02:36:02.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.480+0000 7fc95b195700 1 --2- 172.21.15.7:0/2695767901 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc940066f30 0x7fc9400693e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.480+0000 7fc95b195700 1 --2- 172.21.15.7:0/2695767901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc954068a80 0x7fc954068ef0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.480+0000 7fc95b195700 1 --2- 172.21.15.7:0/2695767901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc93c003fe0 0x7fc95416b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.480+0000 7fc95b195700 1 --2- 172.21.15.7:0/2695767901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9540b3910 0x7fc954154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.480+0000 7fc95b195700 1 -- 172.21.15.7:0/2695767901 >> 172.21.15.7:0/2695767901 conn(0x7fc954098020 msgr2=0x7fc9540cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:02.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.480+0000 7fc95b195700 1 -- 172.21.15.7:0/2695767901 shutdown_connections 2024-01-27T02:36:02.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.480+0000 7fc95b195700 1 -- 172.21.15.7:0/2695767901 wait complete. 2024-01-27T02:36:02.494 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:36:02.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.626+0000 7fde525fe700 1 Processor -- start 2024-01-27T02:36:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.626+0000 7fde525fe700 1 -- start start 2024-01-27T02:36:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.627+0000 7fde525fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde4c15a100 0x7fde4c0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.627+0000 7fde525fe700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fde4c0cb5b0 0x7fde4c0c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.627+0000 7fde525fe700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde4c0ce5e0 con 0x7fde4c0c4af0 2024-01-27T02:36:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.627+0000 7fde525fe700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde4c0c4f70 con 0x7fde4c0cb5b0 2024-01-27T02:36:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.627+0000 7fde525fe700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c0cd9d0 con 0x7fde4c15a100 2024-01-27T02:36:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.627+0000 7fde50b9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fde4c0c4af0 0x7fde4c0c9530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40604/0 (socket says 172.21.15.7:40604) 2024-01-27T02:36:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.627+0000 7fde50b9b700 1 -- 172.21.15.7:0/2973305811 learned_addr learned my addr 172.21.15.7:0/2973305811 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.627+0000 7fde4b7fe700 1 --2- 172.21.15.7:0/2973305811 >> v2:172.21.15.188:3300/0 conn(0x7fde4c0cb5b0 0x7fde4c0c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.627+0000 7fde4bfff700 1 --2- 172.21.15.7:0/2973305811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde4c15a100 0x7fde4c0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.627+0000 7fde4affd700 1 -- 172.21.15.7:0/2973305811 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194950083 0 0) 0x7fde4c0ce5e0 con 0x7fde4c0c4af0 2024-01-27T02:36:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.627+0000 7fde4affd700 1 -- 172.21.15.7:0/2973305811 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde34003570 con 0x7fde4c0c4af0 2024-01-27T02:36:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.628+0000 7fde4bfff700 1 -- 172.21.15.7:0/2973305811 >> v2:172.21.15.188:3300/0 conn(0x7fde4c0cb5b0 msgr2=0x7fde4c0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:02.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.628+0000 7fde4bfff700 1 --2- 172.21.15.7:0/2973305811 >> v2:172.21.15.188:3300/0 conn(0x7fde4c0cb5b0 0x7fde4c0c45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.628+0000 7fde4bfff700 1 -- 172.21.15.7:0/2973305811 >> v1:172.21.15.7:6789/0 conn(0x7fde4c0c4af0 legacy=0x7fde4c0c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:02.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.628+0000 7fde4affd700 1 -- 172.21.15.7:0/2973305811 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3901785516 0 0) 0x7fde34003570 con 0x7fde4c0c4af0 2024-01-27T02:36:02.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.628+0000 7fde4bfff700 1 -- 172.21.15.7:0/2973305811 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c0cdcb0 con 0x7fde4c15a100 2024-01-27T02:36:02.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.628+0000 7fde4b7fe700 1 --2- 172.21.15.7:0/2973305811 >> v2:172.21.15.188:3300/0 conn(0x7fde4c0cb5b0 0x7fde4c0c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:36:02.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.628+0000 7fde4bfff700 1 --2- 172.21.15.7:0/2973305811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde4c15a100 0x7fde4c0cd2f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0098e0 tx=0x7fde3c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d2903140a6fd7ff server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:02.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.628+0000 7fde4affd700 1 -- 172.21.15.7:0/2973305811 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c011630 con 0x7fde4c15a100 2024-01-27T02:36:02.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.628+0000 7fde4affd700 1 -- 172.21.15.7:0/2973305811 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3c011790 con 0x7fde4c15a100 2024-01-27T02:36:02.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.628+0000 7fde4affd700 1 -- 172.21.15.7:0/2973305811 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c011b30 con 0x7fde4c15a100 2024-01-27T02:36:02.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.629+0000 7fde525fe700 1 -- 172.21.15.7:0/2973305811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde4c15a100 msgr2=0x7fde4c0cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:02.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.629+0000 7fde525fe700 1 --2- 172.21.15.7:0/2973305811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde4c15a100 0x7fde4c0cd2f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0098e0 tx=0x7fde3c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.629+0000 7fde525fe700 1 -- 172.21.15.7:0/2973305811 shutdown_connections 2024-01-27T02:36:02.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.629+0000 7fde525fe700 1 --2- 172.21.15.7:0/2973305811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde4c15a100 0x7fde4c0cd2f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.629+0000 7fde525fe700 1 --2- 172.21.15.7:0/2973305811 >> v2:172.21.15.188:3300/0 conn(0x7fde4c0cb5b0 0x7fde4c0c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.629+0000 7fde525fe700 1 -- 172.21.15.7:0/2973305811 >> 172.21.15.7:0/2973305811 conn(0x7fde4c098020 msgr2=0x7fde4c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.629+0000 7fde525fe700 1 -- 172.21.15.7:0/2973305811 shutdown_connections 2024-01-27T02:36:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.629+0000 7fde525fe700 1 -- 172.21.15.7:0/2973305811 wait complete. 2024-01-27T02:36:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.630+0000 7fde525fe700 1 Processor -- start 2024-01-27T02:36:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.630+0000 7fde525fe700 1 -- start start 2024-01-27T02:36:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.631+0000 7fde525fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde4c0c4af0 0x7fde4c0b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.631+0000 7fde525fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde4c0cb5b0 0x7fde4c15c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.631+0000 7fde525fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde4c15a100 0x7fde4c157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.631+0000 7fde525fe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c0b3570 con 0x7fde4c0c4af0 2024-01-27T02:36:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.631+0000 7fde525fe700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c0c6d00 con 0x7fde4c0cb5b0 2024-01-27T02:36:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.631+0000 7fde525fe700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c0b2a50 con 0x7fde4c15a100 2024-01-27T02:36:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.631+0000 7fde4bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde4c0c4af0 0x7fde4c0b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.631+0000 7fde4bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde4c0c4af0 0x7fde4c0b2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33384/0 (socket says 172.21.15.7:33384) 2024-01-27T02:36:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.631+0000 7fde4bfff700 1 -- 172.21.15.7:0/665113567 learned_addr learned my addr 172.21.15.7:0/665113567 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.631+0000 7fde50b9b700 1 --2- 172.21.15.7:0/665113567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde4c15a100 0x7fde4c157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.631+0000 7fde4b7fe700 1 --2- 172.21.15.7:0/665113567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde4c0cb5b0 0x7fde4c15c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.631+0000 7fde4bfff700 1 -- 172.21.15.7:0/665113567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde4c15a100 msgr2=0x7fde4c157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.631+0000 7fde4bfff700 1 --2- 172.21.15.7:0/665113567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde4c15a100 0x7fde4c157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.631+0000 7fde4bfff700 1 -- 172.21.15.7:0/665113567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde4c0cb5b0 msgr2=0x7fde4c15c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.631+0000 7fde4bfff700 1 --2- 172.21.15.7:0/665113567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde4c0cb5b0 0x7fde4c15c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.631+0000 7fde4bfff700 1 -- 172.21.15.7:0/665113567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde40009420 con 0x7fde4c0c4af0 2024-01-27T02:36:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.632+0000 7fde4bfff700 1 --2- 172.21.15.7:0/665113567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde4c0c4af0 0x7fde4c0b2510 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0cdb10 tx=0x7fde3c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.632+0000 7fde33fff700 1 -- 172.21.15.7:0/665113567 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c01f070 con 0x7fde4c0c4af0 2024-01-27T02:36:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.632+0000 7fde33fff700 1 -- 172.21.15.7:0/665113567 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3c002f70 con 0x7fde4c0c4af0 2024-01-27T02:36:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.632+0000 7fde525fe700 1 -- 172.21.15.7:0/665113567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde3c009420 con 0x7fde4c0c4af0 2024-01-27T02:36:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.632+0000 7fde33fff700 1 -- 172.21.15.7:0/665113567 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c011900 con 0x7fde4c0c4af0 2024-01-27T02:36:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.632+0000 7fde525fe700 1 -- 172.21.15.7:0/665113567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4c0b2d80 con 0x7fde4c0c4af0 2024-01-27T02:36:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.633+0000 7fde525fe700 1 -- 172.21.15.7:0/665113567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4c04c710 con 0x7fde4c0c4af0 2024-01-27T02:36:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.633+0000 7fde33fff700 1 -- 172.21.15.7:0/665113567 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fde3c023070 con 0x7fde4c0c4af0 2024-01-27T02:36:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.634+0000 7fde33fff700 1 --2- 172.21.15.7:0/665113567 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fde38066f30 0x7fde380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.634+0000 7fde33fff700 1 -- 172.21.15.7:0/665113567 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fde3c085850 con 0x7fde4c0c4af0 2024-01-27T02:36:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.634+0000 7fde4b7fe700 1 --2- 172.21.15.7:0/665113567 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fde38066f30 0x7fde380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.635+0000 7fde4b7fe700 1 --2- 172.21.15.7:0/665113567 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fde38066f30 0x7fde380693e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fde4c16d8e0 tx=0x7fde400091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:02.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.639+0000 7fde33fff700 1 -- 172.21.15.7:0/665113567 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde3c052ac0 con 0x7fde4c0c4af0 2024-01-27T02:36:02.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.842+0000 7fde525fe700 1 -- 172.21.15.7:0/665113567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde4c069950 con 0x7fde4c0c4af0 2024-01-27T02:36:02.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.843+0000 7fde33fff700 1 -- 172.21.15.7:0/665113567 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+15318 (secure 0 0 0) 0x7fde3c02a070 con 0x7fde4c0c4af0 2024-01-27T02:36:02.843 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:36:02.844 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":80,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:36:01.502486+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:35:59.537952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-27T02:36:00.480234+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:36:02.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.845+0000 7fde525fe700 1 -- 172.21.15.7:0/665113567 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fde38066f30 msgr2=0x7fde380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:02.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.845+0000 7fde525fe700 1 --2- 172.21.15.7:0/665113567 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fde38066f30 0x7fde380693e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fde4c16d8e0 tx=0x7fde400091a0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.845+0000 7fde525fe700 1 -- 172.21.15.7:0/665113567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde4c0c4af0 msgr2=0x7fde4c0b2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:02.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.845+0000 7fde525fe700 1 --2- 172.21.15.7:0/665113567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde4c0c4af0 0x7fde4c0b2510 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0cdb10 tx=0x7fde3c017000 comp rx=0 tx=0).stop 2024-01-27T02:36:02.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.845+0000 7fde525fe700 1 -- 172.21.15.7:0/665113567 shutdown_connections 2024-01-27T02:36:02.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.845+0000 7fde525fe700 1 --2- 172.21.15.7:0/665113567 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fde38066f30 0x7fde380693e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.845+0000 7fde525fe700 1 --2- 172.21.15.7:0/665113567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde4c15a100 0x7fde4c157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.845+0000 7fde525fe700 1 --2- 172.21.15.7:0/665113567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde4c0c4af0 0x7fde4c0b2510 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.845+0000 7fde525fe700 1 --2- 172.21.15.7:0/665113567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde4c0cb5b0 0x7fde4c15c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.845+0000 7fde525fe700 1 -- 172.21.15.7:0/665113567 >> 172.21.15.7:0/665113567 conn(0x7fde4c098020 msgr2=0x7fde4c159370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:02.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.845+0000 7fde525fe700 1 -- 172.21.15.7:0/665113567 shutdown_connections 2024-01-27T02:36:02.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.845+0000 7fde525fe700 1 -- 172.21.15.7:0/665113567 wait complete. 2024-01-27T02:36:02.860 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-27T02:36:02.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.995+0000 7f20f00cb700 1 Processor -- start 2024-01-27T02:36:02.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.995+0000 7f20f00cb700 1 -- start start 2024-01-27T02:36:02.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.996+0000 7f20f00cb700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f20e8156b90 0x7f20e806b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:02.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.996+0000 7f20f00cb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f20e806e950 0x7f20e80c8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:02.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.996+0000 7f20f00cb700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e806c820 con 0x7f20e806b5b0 2024-01-27T02:36:02.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.996+0000 7f20f00cb700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e806ba20 con 0x7f20e806e950 2024-01-27T02:36:02.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.996+0000 7f20f00cb700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e806ccb0 con 0x7f20e8156b90 2024-01-27T02:36:02.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.996+0000 7f20ed666700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f20e806b5b0 0x7f20e806e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40618/0 (socket says 172.21.15.7:40618) 2024-01-27T02:36:02.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.996+0000 7f20ed666700 1 -- 172.21.15.7:0/907373320 learned_addr learned my addr 172.21.15.7:0/907373320 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.996+0000 7f20ee668700 1 --2- 172.21.15.7:0/907373320 >> v2:172.21.15.7:3300/0 conn(0x7f20e806e950 0x7f20e80c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.996+0000 7f20ede67700 1 --2- 172.21.15.7:0/907373320 >> v2:172.21.15.188:3300/0 conn(0x7f20e8156b90 0x7f20e806b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.996+0000 7f20ece65700 1 -- 172.21.15.7:0/907373320 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3322155386 0 0) 0x7f20e806c820 con 0x7f20e806b5b0 2024-01-27T02:36:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.996+0000 7f20ece65700 1 -- 172.21.15.7:0/907373320 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20d4003570 con 0x7f20e806b5b0 2024-01-27T02:36:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.996+0000 7f20ee668700 1 -- 172.21.15.7:0/907373320 >> v2:172.21.15.188:3300/0 conn(0x7f20e8156b90 msgr2=0x7f20e806b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.996+0000 7f20ee668700 1 --2- 172.21.15.7:0/907373320 >> v2:172.21.15.188:3300/0 conn(0x7f20e8156b90 0x7f20e806b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.996+0000 7f20ee668700 1 -- 172.21.15.7:0/907373320 >> v1:172.21.15.7:6789/0 conn(0x7f20e806b5b0 legacy=0x7f20e806e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.996+0000 7f20ee668700 1 -- 172.21.15.7:0/907373320 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e806bc10 con 0x7f20e806e950 2024-01-27T02:36:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.997+0000 7f20ee668700 1 --2- 172.21.15.7:0/907373320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20e806e950 0x7f20e80c8140 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f20e806dc50 tx=0x7f20e0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9ce34f97af7b68 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.997+0000 7f20ece65700 1 -- 172.21.15.7:0/907373320 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e000e070 con 0x7f20e806e950 2024-01-27T02:36:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.997+0000 7f20ece65700 1 -- 172.21.15.7:0/907373320 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e000bab0 con 0x7f20e806e950 2024-01-27T02:36:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.997+0000 7f20ece65700 1 -- 172.21.15.7:0/907373320 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e000be30 con 0x7f20e806e950 2024-01-27T02:36:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.997+0000 7f20f00cb700 1 -- 172.21.15.7:0/907373320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20e806e950 msgr2=0x7f20e80c8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.997+0000 7f20f00cb700 1 --2- 172.21.15.7:0/907373320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20e806e950 0x7f20e80c8140 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f20e806dc50 tx=0x7f20e0009780 comp rx=0 tx=0).stop 2024-01-27T02:36:03.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.997+0000 7f20f00cb700 1 -- 172.21.15.7:0/907373320 shutdown_connections 2024-01-27T02:36:03.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.997+0000 7f20f00cb700 1 --2- 172.21.15.7:0/907373320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20e806e950 0x7f20e80c8140 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:03.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.997+0000 7f20f00cb700 1 --2- 172.21.15.7:0/907373320 >> v2:172.21.15.188:3300/0 conn(0x7f20e8156b90 0x7f20e806b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:03.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.997+0000 7f20f00cb700 1 -- 172.21.15.7:0/907373320 >> 172.21.15.7:0/907373320 conn(0x7f20e80ca570 msgr2=0x7f20e80ca970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:03.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.998+0000 7f20f00cb700 1 -- 172.21.15.7:0/907373320 shutdown_connections 2024-01-27T02:36:03.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.998+0000 7f20f00cb700 1 -- 172.21.15.7:0/907373320 wait complete. 2024-01-27T02:36:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.999+0000 7f20f00cb700 1 Processor -- start 2024-01-27T02:36:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.999+0000 7f20f00cb700 1 -- start start 2024-01-27T02:36:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.999+0000 7f20f00cb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20e806b5b0 0x7f20e80b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.999+0000 7f20f00cb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20e806e950 0x7f20e8142e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:02.999+0000 7f20f00cb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20e8156b90 0x7f20e8147a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.000+0000 7f20f00cb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e8159890 con 0x7f20e806e950 2024-01-27T02:36:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.000+0000 7f20f00cb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e80b2a30 con 0x7f20e8156b90 2024-01-27T02:36:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.000+0000 7f20f00cb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e80b2d10 con 0x7f20e806b5b0 2024-01-27T02:36:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.000+0000 7f20ed666700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20e806e950 0x7f20e8142e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.000+0000 7f20ed666700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20e806e950 0x7f20e8142e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33404/0 (socket says 172.21.15.7:33404) 2024-01-27T02:36:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.000+0000 7f20ed666700 1 -- 172.21.15.7:0/1069701224 learned_addr learned my addr 172.21.15.7:0/1069701224 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.000+0000 7f20ede67700 1 --2- 172.21.15.7:0/1069701224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20e806b5b0 0x7f20e80b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:03.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.000+0000 7f20ee668700 1 --2- 172.21.15.7:0/1069701224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20e8156b90 0x7f20e8147a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:03.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.000+0000 7f20ed666700 1 -- 172.21.15.7:0/1069701224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20e806b5b0 msgr2=0x7f20e80b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:03.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.000+0000 7f20ed666700 1 --2- 172.21.15.7:0/1069701224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20e806b5b0 0x7f20e80b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:03.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.000+0000 7f20ed666700 1 -- 172.21.15.7:0/1069701224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20e8156b90 msgr2=0x7f20e8147a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:03.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.000+0000 7f20ed666700 1 --2- 172.21.15.7:0/1069701224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20e8156b90 0x7f20e8147a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:03.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.000+0000 7f20ed666700 1 -- 172.21.15.7:0/1069701224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e0009a20 con 0x7f20e806e950 2024-01-27T02:36:03.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.000+0000 7f20ed666700 1 --2- 172.21.15.7:0/1069701224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20e806e950 0x7f20e8142e50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f20d80011e0 tx=0x7f20d8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:03.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.000+0000 7f20ddffb700 1 -- 172.21.15.7:0/1069701224 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8005b30 con 0x7f20e806e950 2024-01-27T02:36:03.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.000+0000 7f20ddffb700 1 -- 172.21.15.7:0/1069701224 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d8005c90 con 0x7f20e806e950 2024-01-27T02:36:03.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.000+0000 7f20f00cb700 1 -- 172.21.15.7:0/1069701224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e80b2ff0 con 0x7f20e806e950 2024-01-27T02:36:03.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.000+0000 7f20ddffb700 1 -- 172.21.15.7:0/1069701224 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8007300 con 0x7f20e806e950 2024-01-27T02:36:03.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.001+0000 7f20f00cb700 1 -- 172.21.15.7:0/1069701224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e8149030 con 0x7f20e806e950 2024-01-27T02:36:03.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.002+0000 7f20f00cb700 1 -- 172.21.15.7:0/1069701224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e8148e90 con 0x7f20e806e950 2024-01-27T02:36:03.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.002+0000 7f20ddffb700 1 -- 172.21.15.7:0/1069701224 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f20d8007670 con 0x7f20e806e950 2024-01-27T02:36:03.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.002+0000 7f20ddffb700 1 --2- 172.21.15.7:0/1069701224 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f20cc066f30 0x7f20cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:03.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.002+0000 7f20ddffb700 1 -- 172.21.15.7:0/1069701224 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f20d8084090 con 0x7f20e806e950 2024-01-27T02:36:03.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.003+0000 7f20ede67700 1 --2- 172.21.15.7:0/1069701224 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f20cc066f30 0x7f20cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:03.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.003+0000 7f20ede67700 1 --2- 172.21.15.7:0/1069701224 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f20cc066f30 0x7f20cc0693e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009ca0 tx=0x7f20e4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:03.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.007+0000 7f20ddffb700 1 -- 172.21.15.7:0/1069701224 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f20e8148e90 con 0x7f20e806e950 2024-01-27T02:36:03.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:03.259+0000 7f20f00cb700 1 -- 172.21.15.7:0/1069701224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f20e8148e90 con 0x7f20e806e950 2024-01-27T02:36:04.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.110+0000 7f20ddffb700 1 -- 172.21.15.7:0/1069701224 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v41) v1 ==== 118+0+0 (secure 0 0 0) 0x7f20e8148e90 con 0x7f20e806e950 2024-01-27T02:36:04.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.113+0000 7f20f00cb700 1 -- 172.21.15.7:0/1069701224 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f20cc066f30 msgr2=0x7f20cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:04.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.113+0000 7f20f00cb700 1 --2- 172.21.15.7:0/1069701224 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f20cc066f30 0x7f20cc0693e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009ca0 tx=0x7f20e4008040 comp rx=0 tx=0).stop 2024-01-27T02:36:04.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.113+0000 7f20f00cb700 1 -- 172.21.15.7:0/1069701224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20e806e950 msgr2=0x7f20e8142e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:04.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.113+0000 7f20f00cb700 1 --2- 172.21.15.7:0/1069701224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20e806e950 0x7f20e8142e50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f20d80011e0 tx=0x7f20d8007df0 comp rx=0 tx=0).stop 2024-01-27T02:36:04.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.113+0000 7f20f00cb700 1 -- 172.21.15.7:0/1069701224 shutdown_connections 2024-01-27T02:36:04.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.113+0000 7f20f00cb700 1 --2- 172.21.15.7:0/1069701224 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f20cc066f30 0x7f20cc0693e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:04.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.113+0000 7f20f00cb700 1 --2- 172.21.15.7:0/1069701224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20e8156b90 0x7f20e8147a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:04.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.113+0000 7f20f00cb700 1 --2- 172.21.15.7:0/1069701224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20e806b5b0 0x7f20e80b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:04.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.113+0000 7f20f00cb700 1 --2- 172.21.15.7:0/1069701224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20e806e950 0x7f20e8142e50 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:04.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.113+0000 7f20f00cb700 1 -- 172.21.15.7:0/1069701224 >> 172.21.15.7:0/1069701224 conn(0x7f20e80ca570 msgr2=0x7f20e80cbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:04.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.113+0000 7f20f00cb700 1 -- 172.21.15.7:0/1069701224 shutdown_connections 2024-01-27T02:36:04.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.113+0000 7f20f00cb700 1 -- 172.21.15.7:0/1069701224 wait complete. 2024-01-27T02:36:04.129 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-27T02:36:04.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.268+0000 7f4fc8c16700 1 Processor -- start 2024-01-27T02:36:04.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.268+0000 7f4fc8c16700 1 -- start start 2024-01-27T02:36:04.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.269+0000 7f4fc8c16700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4fc40ca420 0x7f4fc4157790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:04.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.269+0000 7f4fc8c16700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc40cf210 0x7f4fc4157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:04.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.269+0000 7f4fc8c16700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc4158210 con 0x7f4fc40cee40 2024-01-27T02:36:04.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.269+0000 7f4fc8c16700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc415c3e0 con 0x7f4fc40ca420 2024-01-27T02:36:04.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.269+0000 7f4fc8c16700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc415c6c0 con 0x7f4fc40cf210 2024-01-27T02:36:04.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.269+0000 7f4fc1d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4fc40ca420 0x7f4fc4157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:04.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.269+0000 7f4fc259c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4fc40cee40 0x7f4fc415b940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40628/0 (socket says 172.21.15.7:40628) 2024-01-27T02:36:04.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.269+0000 7f4fc259c700 1 -- 172.21.15.7:0/2702733810 learned_addr learned my addr 172.21.15.7:0/2702733810 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:04.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.269+0000 7f4fc2d9d700 1 --2- 172.21.15.7:0/2702733810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc40cf210 0x7f4fc4157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:04.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.269+0000 7f4fc1d9b700 1 -- 172.21.15.7:0/2702733810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc40cf210 msgr2=0x7f4fc4157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:04.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.269+0000 7f4fc1d9b700 1 --2- 172.21.15.7:0/2702733810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc40cf210 0x7f4fc4157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:04.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.269+0000 7f4fc1d9b700 1 -- 172.21.15.7:0/2702733810 >> v1:172.21.15.7:6789/0 conn(0x7f4fc40cee40 legacy=0x7f4fc415b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:04.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.269+0000 7f4fc1d9b700 1 -- 172.21.15.7:0/2702733810 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc40afc40 con 0x7f4fc40ca420 2024-01-27T02:36:04.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.269+0000 7f4fc159a700 1 -- 172.21.15.7:0/2702733810 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567648124 0 0) 0x7f4fc4158210 con 0x7f4fc40cee40 2024-01-27T02:36:04.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.270+0000 7f4fc1d9b700 1 --2- 172.21.15.7:0/2702733810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fc40ca420 0x7f4fc4157790 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40098e0 tx=0x7f4fb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2870de817547f5 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:04.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.270+0000 7f4fc159a700 1 -- 172.21.15.7:0/2702733810 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4009e80 con 0x7f4fc40ca420 2024-01-27T02:36:04.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.270+0000 7f4fc159a700 1 -- 172.21.15.7:0/2702733810 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb400b860 con 0x7f4fc40ca420 2024-01-27T02:36:04.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.270+0000 7f4fc159a700 1 -- 172.21.15.7:0/2702733810 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb400bbe0 con 0x7f4fc40ca420 2024-01-27T02:36:04.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.270+0000 7f4fc8c16700 1 -- 172.21.15.7:0/2702733810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fc40ca420 msgr2=0x7f4fc4157790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.270+0000 7f4fc8c16700 1 --2- 172.21.15.7:0/2702733810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fc40ca420 0x7f4fc4157790 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40098e0 tx=0x7f4fb40091a0 comp rx=0 tx=0).stop 2024-01-27T02:36:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.271+0000 7f4fc8c16700 1 -- 172.21.15.7:0/2702733810 shutdown_connections 2024-01-27T02:36:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.271+0000 7f4fc8c16700 1 --2- 172.21.15.7:0/2702733810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc40cf210 0x7f4fc4157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.271+0000 7f4fc8c16700 1 --2- 172.21.15.7:0/2702733810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fc40ca420 0x7f4fc4157790 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.271+0000 7f4fc8c16700 1 -- 172.21.15.7:0/2702733810 >> 172.21.15.7:0/2702733810 conn(0x7f4fc406f2c0 msgr2=0x7f4fc406f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.271+0000 7f4fc8c16700 1 -- 172.21.15.7:0/2702733810 shutdown_connections 2024-01-27T02:36:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.271+0000 7f4fc8c16700 1 -- 172.21.15.7:0/2702733810 wait complete. 2024-01-27T02:36:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.272+0000 7f4fc8c16700 1 Processor -- start 2024-01-27T02:36:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.272+0000 7f4fc8c16700 1 -- start start 2024-01-27T02:36:04.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.273+0000 7f4fc8c16700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc40ca420 0x7f4fc41519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:04.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.273+0000 7f4fc8c16700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fc40cee40 0x7f4fc416bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:04.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.273+0000 7f4fc8c16700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc40cf210 0x7f4fc4150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:04.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.273+0000 7f4fc8c16700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc416c740 con 0x7f4fc40cee40 2024-01-27T02:36:04.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.273+0000 7f4fc8c16700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc40c77d0 con 0x7f4fc40ca420 2024-01-27T02:36:04.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.273+0000 7f4fc8c16700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc40b2d50 con 0x7f4fc40cf210 2024-01-27T02:36:04.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.273+0000 7f4fc1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fc40cee40 0x7f4fc416bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:04.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.273+0000 7f4fc1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fc40cee40 0x7f4fc416bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33426/0 (socket says 172.21.15.7:33426) 2024-01-27T02:36:04.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.273+0000 7f4fc1d9b700 1 -- 172.21.15.7:0/2197353514 learned_addr learned my addr 172.21.15.7:0/2197353514 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:04.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.273+0000 7f4fc2d9d700 1 --2- 172.21.15.7:0/2197353514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc40cf210 0x7f4fc4150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:04.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.273+0000 7f4fc259c700 1 --2- 172.21.15.7:0/2197353514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc40ca420 0x7f4fc41519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:04.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.273+0000 7f4fc1d9b700 1 -- 172.21.15.7:0/2197353514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc40cf210 msgr2=0x7f4fc4150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:04.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.273+0000 7f4fc1d9b700 1 --2- 172.21.15.7:0/2197353514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc40cf210 0x7f4fc4150d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:04.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.273+0000 7f4fc1d9b700 1 -- 172.21.15.7:0/2197353514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc40ca420 msgr2=0x7f4fc41519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:04.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.273+0000 7f4fc1d9b700 1 --2- 172.21.15.7:0/2197353514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc40ca420 0x7f4fc41519c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:04.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.273+0000 7f4fc1d9b700 1 -- 172.21.15.7:0/2197353514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb4009420 con 0x7f4fc40cee40 2024-01-27T02:36:04.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.273+0000 7f4fc1d9b700 1 --2- 172.21.15.7:0/2197353514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fc40cee40 0x7f4fc416bb20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40719d0 tx=0x7f4fb4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:04.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.273+0000 7f4fba7fc700 1 -- 172.21.15.7:0/2197353514 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb400bbe0 con 0x7f4fc40cee40 2024-01-27T02:36:04.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.273+0000 7f4fba7fc700 1 -- 172.21.15.7:0/2197353514 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb4011420 con 0x7f4fc40cee40 2024-01-27T02:36:04.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.273+0000 7f4fba7fc700 1 -- 172.21.15.7:0/2197353514 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb40117c0 con 0x7f4fc40cee40 2024-01-27T02:36:04.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.273+0000 7f4fc8c16700 1 -- 172.21.15.7:0/2197353514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc4151f00 con 0x7f4fc40cee40 2024-01-27T02:36:04.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.274+0000 7f4fc8c16700 1 -- 172.21.15.7:0/2197353514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc40c7210 con 0x7f4fc40cee40 2024-01-27T02:36:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.275+0000 7f4fc8c16700 1 -- 172.21.15.7:0/2197353514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc4072060 con 0x7f4fc40cee40 2024-01-27T02:36:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.275+0000 7f4fba7fc700 1 -- 172.21.15.7:0/2197353514 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4fb4011bb0 con 0x7f4fc40cee40 2024-01-27T02:36:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.275+0000 7f4fba7fc700 1 --2- 172.21.15.7:0/2197353514 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4fa4066f30 0x7f4fa40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.275+0000 7f4fba7fc700 1 -- 172.21.15.7:0/2197353514 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4fb4085f00 con 0x7f4fc40cee40 2024-01-27T02:36:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.275+0000 7f4fc259c700 1 --2- 172.21.15.7:0/2197353514 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4fa4066f30 0x7f4fa40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.276+0000 7f4fc259c700 1 --2- 172.21.15.7:0/2197353514 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4fa4066f30 0x7f4fa40693e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4fac007fd0 tx=0x7f4fac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.280+0000 7f4fba7fc700 1 -- 172.21.15.7:0/2197353514 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4fb4053170 con 0x7f4fc40cee40 2024-01-27T02:36:04.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:04.536+0000 7f4fc8c16700 1 -- 172.21.15.7:0/2197353514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4fc40aff20 con 0x7f4fc40cee40 2024-01-27T02:36:05.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.112+0000 7f4fba7fc700 1 -- 172.21.15.7:0/2197353514 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v42) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4fb4011e60 con 0x7f4fc40cee40 2024-01-27T02:36:05.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.114+0000 7f4fc8c16700 1 -- 172.21.15.7:0/2197353514 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4fa4066f30 msgr2=0x7f4fa40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:05.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.114+0000 7f4fc8c16700 1 --2- 172.21.15.7:0/2197353514 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4fa4066f30 0x7f4fa40693e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4fac007fd0 tx=0x7f4fac009040 comp rx=0 tx=0).stop 2024-01-27T02:36:05.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.114+0000 7f4fc8c16700 1 -- 172.21.15.7:0/2197353514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fc40cee40 msgr2=0x7f4fc416bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:05.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.114+0000 7f4fc8c16700 1 --2- 172.21.15.7:0/2197353514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fc40cee40 0x7f4fc416bb20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40719d0 tx=0x7f4fb4017000 comp rx=0 tx=0).stop 2024-01-27T02:36:05.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.114+0000 7f4fc8c16700 1 -- 172.21.15.7:0/2197353514 shutdown_connections 2024-01-27T02:36:05.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.114+0000 7f4fc8c16700 1 --2- 172.21.15.7:0/2197353514 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4fa4066f30 0x7f4fa40693e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.115+0000 7f4fc8c16700 1 --2- 172.21.15.7:0/2197353514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc40cf210 0x7f4fc4150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.115+0000 7f4fc8c16700 1 --2- 172.21.15.7:0/2197353514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc40ca420 0x7f4fc41519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.115+0000 7f4fc8c16700 1 --2- 172.21.15.7:0/2197353514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fc40cee40 0x7f4fc416bb20 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.115+0000 7f4fc8c16700 1 -- 172.21.15.7:0/2197353514 >> 172.21.15.7:0/2197353514 conn(0x7f4fc406f2c0 msgr2=0x7f4fc40b0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:05.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.115+0000 7f4fc8c16700 1 -- 172.21.15.7:0/2197353514 shutdown_connections 2024-01-27T02:36:05.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.115+0000 7f4fc8c16700 1 -- 172.21.15.7:0/2197353514 wait complete. 2024-01-27T02:36:05.130 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:36:05.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.265+0000 7f1261a1a700 1 Processor -- start 2024-01-27T02:36:05.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.265+0000 7f1261a1a700 1 -- start start 2024-01-27T02:36:05.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.266+0000 7f1261a1a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f125c15a070 0x7f125c15b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:05.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.266+0000 7f1261a1a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f125c0b0670 0x7f125c163d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:05.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.266+0000 7f1261a1a700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f125c0689c0 con 0x7f125c0caf10 2024-01-27T02:36:05.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.266+0000 7f1261a1a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f125c0ae5f0 con 0x7f125c15a070 2024-01-27T02:36:05.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.266+0000 7f1261a1a700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f125c0ae8b0 con 0x7f125c0b0670 2024-01-27T02:36:05.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.266+0000 7f125affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f125c15a070 0x7f125c15b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:05.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.266+0000 7f125affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f125c15a070 0x7f125c15b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33428/0 (socket says 172.21.15.7:33428) 2024-01-27T02:36:05.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.266+0000 7f125affd700 1 -- 172.21.15.7:0/208089504 learned_addr learned my addr 172.21.15.7:0/208089504 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:05.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.266+0000 7f125b7fe700 1 --2- 172.21.15.7:0/208089504 >> v2:172.21.15.188:3300/0 conn(0x7f125c0b0670 0x7f125c163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:05.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.266+0000 7f125affd700 1 -- 172.21.15.7:0/208089504 >> v2:172.21.15.188:3300/0 conn(0x7f125c0b0670 msgr2=0x7f125c163d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:05.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.266+0000 7f125affd700 1 --2- 172.21.15.7:0/208089504 >> v2:172.21.15.188:3300/0 conn(0x7f125c0b0670 0x7f125c163d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.266+0000 7f125affd700 1 -- 172.21.15.7:0/208089504 >> v1:172.21.15.188:6789/0 conn(0x7f125c0caf10 legacy=0x7f125c0aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:05.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.266+0000 7f125affd700 1 -- 172.21.15.7:0/208089504 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f125c0aeb90 con 0x7f125c15a070 2024-01-27T02:36:05.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.267+0000 7f125affd700 1 --2- 172.21.15.7:0/208089504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f125c15a070 0x7f125c15b2a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f125c06e540 tx=0x7f124c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1f2b23a81958bd server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:05.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.267+0000 7f1259ffb700 1 -- 172.21.15.7:0/208089504 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c00b860 con 0x7f125c15a070 2024-01-27T02:36:05.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.267+0000 7f1259ffb700 1 -- 172.21.15.7:0/208089504 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124c00b9c0 con 0x7f125c15a070 2024-01-27T02:36:05.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.267+0000 7f1259ffb700 1 -- 172.21.15.7:0/208089504 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c00bd60 con 0x7f125c15a070 2024-01-27T02:36:05.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.267+0000 7f1261a1a700 1 -- 172.21.15.7:0/208089504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f125c15a070 msgr2=0x7f125c15b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:05.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.267+0000 7f1261a1a700 1 --2- 172.21.15.7:0/208089504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f125c15a070 0x7f125c15b2a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f125c06e540 tx=0x7f124c009250 comp rx=0 tx=0).stop 2024-01-27T02:36:05.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.267+0000 7f1261a1a700 1 -- 172.21.15.7:0/208089504 shutdown_connections 2024-01-27T02:36:05.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.267+0000 7f1261a1a700 1 --2- 172.21.15.7:0/208089504 >> v2:172.21.15.188:3300/0 conn(0x7f125c0b0670 0x7f125c163d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.267+0000 7f1261a1a700 1 --2- 172.21.15.7:0/208089504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f125c15a070 0x7f125c15b2a0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.267+0000 7f1261a1a700 1 -- 172.21.15.7:0/208089504 >> 172.21.15.7:0/208089504 conn(0x7f125c098020 msgr2=0x7f125c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:05.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.267+0000 7f1261a1a700 1 -- 172.21.15.7:0/208089504 shutdown_connections 2024-01-27T02:36:05.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.267+0000 7f1261a1a700 1 -- 172.21.15.7:0/208089504 wait complete. 2024-01-27T02:36:05.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.269+0000 7f1261a1a700 1 Processor -- start 2024-01-27T02:36:05.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.269+0000 7f1261a1a700 1 -- start start 2024-01-27T02:36:05.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.270+0000 7f1261a1a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f125c0b0670 0x7f125c145190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:05.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.270+0000 7f1261a1a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f125c0caf10 0x7f125c09a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:05.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.270+0000 7f1261a1a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f125c15a070 0x7f125c09aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:05.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.270+0000 7f1261a1a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125c146860 con 0x7f125c0caf10 2024-01-27T02:36:05.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.270+0000 7f1261a1a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125c145be0 con 0x7f125c15a070 2024-01-27T02:36:05.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.270+0000 7f1261a1a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f125c145ec0 con 0x7f125c0b0670 2024-01-27T02:36:05.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.270+0000 7f125a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f125c0caf10 0x7f125c09a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:05.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.270+0000 7f125a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f125c0caf10 0x7f125c09a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33432/0 (socket says 172.21.15.7:33432) 2024-01-27T02:36:05.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.270+0000 7f125a7fc700 1 -- 172.21.15.7:0/1964682886 learned_addr learned my addr 172.21.15.7:0/1964682886 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:05.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.270+0000 7f125affd700 1 --2- 172.21.15.7:0/1964682886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f125c0b0670 0x7f125c145190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:05.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.270+0000 7f125b7fe700 1 --2- 172.21.15.7:0/1964682886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f125c15a070 0x7f125c09aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:05.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.270+0000 7f125a7fc700 1 -- 172.21.15.7:0/1964682886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f125c0b0670 msgr2=0x7f125c145190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:05.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.270+0000 7f125a7fc700 1 --2- 172.21.15.7:0/1964682886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f125c0b0670 0x7f125c145190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.270+0000 7f125a7fc700 1 -- 172.21.15.7:0/1964682886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f125c15a070 msgr2=0x7f125c09aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:05.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.270+0000 7f125a7fc700 1 --2- 172.21.15.7:0/1964682886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f125c15a070 0x7f125c09aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.270+0000 7f125a7fc700 1 -- 172.21.15.7:0/1964682886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c0094f0 con 0x7f125c0caf10 2024-01-27T02:36:05.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.270+0000 7f125b7fe700 1 --2- 172.21.15.7:0/1964682886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f125c15a070 0x7f125c09aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:36:05.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.270+0000 7f125a7fc700 1 --2- 172.21.15.7:0/1964682886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f125c0caf10 0x7f125c09a9a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f125000c590 tx=0x7f125000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:05.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.270+0000 7f1242ffd700 1 -- 172.21.15.7:0/1964682886 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1250011590 con 0x7f125c0caf10 2024-01-27T02:36:05.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.270+0000 7f1242ffd700 1 -- 172.21.15.7:0/1964682886 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1250011710 con 0x7f125c0caf10 2024-01-27T02:36:05.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.271+0000 7f1261a1a700 1 -- 172.21.15.7:0/1964682886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f125c1461a0 con 0x7f125c0caf10 2024-01-27T02:36:05.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.271+0000 7f1242ffd700 1 -- 172.21.15.7:0/1964682886 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1250011ad0 con 0x7f125c0caf10 2024-01-27T02:36:05.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.271+0000 7f1261a1a700 1 -- 172.21.15.7:0/1964682886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f125c144e20 con 0x7f125c0caf10 2024-01-27T02:36:05.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.272+0000 7f1261a1a700 1 -- 172.21.15.7:0/1964682886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f125c1469a0 con 0x7f125c0caf10 2024-01-27T02:36:05.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.272+0000 7f1242ffd700 1 -- 172.21.15.7:0/1964682886 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f125001b3b0 con 0x7f125c0caf10 2024-01-27T02:36:05.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.273+0000 7f1242ffd700 1 --2- 172.21.15.7:0/1964682886 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1248066f30 0x7f12480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:05.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.273+0000 7f1242ffd700 1 -- 172.21.15.7:0/1964682886 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1250083af0 con 0x7f125c0caf10 2024-01-27T02:36:05.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.273+0000 7f125affd700 1 --2- 172.21.15.7:0/1964682886 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1248066f30 0x7f12480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:05.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.274+0000 7f125affd700 1 --2- 172.21.15.7:0/1964682886 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1248066f30 0x7f12480693e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f125c06e540 tx=0x7f124c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:05.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.278+0000 7f1242ffd700 1 -- 172.21.15.7:0/1964682886 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f125c1469a0 con 0x7f125c0caf10 2024-01-27T02:36:05.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.529+0000 7f1261a1a700 1 -- 172.21.15.7:0/1964682886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f125c1469a0 con 0x7f125c0caf10 2024-01-27T02:36:05.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.530+0000 7f1242ffd700 1 -- 172.21.15.7:0/1964682886 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4590 (secure 0 0 0) 0x7f125c1469a0 con 0x7f125c0caf10 2024-01-27T02:36:05.530 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:36:05.531 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5839,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/100533817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":100533817},{"type":"v1","addr":"172.21.15.7:6834","nonce":100533817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5842,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/223425097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":223425097},{"type":"v1","addr":"172.21.15.7:6835","nonce":223425097}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5848,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/950040140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":950040140},{"type":"v1","addr":"172.21.15.188:6837","nonce":950040140}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:36:01.502818+0000","modified":"2024-01-27T02:36:05.109352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5946":{"gid":5946,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6836/2269430612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2269430612},{"type":"v1","addr":"172.21.15.188:6836","nonce":2269430612}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2024-01-27T02:36:05.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.532+0000 7f1261a1a700 1 -- 172.21.15.7:0/1964682886 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1248066f30 msgr2=0x7f12480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:05.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.532+0000 7f1261a1a700 1 --2- 172.21.15.7:0/1964682886 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1248066f30 0x7f12480693e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f125c06e540 tx=0x7f124c017000 comp rx=0 tx=0).stop 2024-01-27T02:36:05.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.532+0000 7f1261a1a700 1 -- 172.21.15.7:0/1964682886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f125c0caf10 msgr2=0x7f125c09a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:05.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.532+0000 7f1261a1a700 1 --2- 172.21.15.7:0/1964682886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f125c0caf10 0x7f125c09a9a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f125000c590 tx=0x7f125000c8a0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.532+0000 7f1261a1a700 1 -- 172.21.15.7:0/1964682886 shutdown_connections 2024-01-27T02:36:05.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.532+0000 7f1261a1a700 1 --2- 172.21.15.7:0/1964682886 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1248066f30 0x7f12480693e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.532+0000 7f1261a1a700 1 --2- 172.21.15.7:0/1964682886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f125c15a070 0x7f125c09aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.532+0000 7f1261a1a700 1 --2- 172.21.15.7:0/1964682886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f125c0b0670 0x7f125c145190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.532+0000 7f1261a1a700 1 --2- 172.21.15.7:0/1964682886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f125c0caf10 0x7f125c09a9a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.532+0000 7f1261a1a700 1 -- 172.21.15.7:0/1964682886 >> 172.21.15.7:0/1964682886 conn(0x7f125c098020 msgr2=0x7f125c06ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:05.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.532+0000 7f1261a1a700 1 -- 172.21.15.7:0/1964682886 shutdown_connections 2024-01-27T02:36:05.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.532+0000 7f1261a1a700 1 -- 172.21.15.7:0/1964682886 wait complete. 2024-01-27T02:36:05.534 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 42 2024-01-27T02:36:05.546 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:36:05.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.678+0000 7fb8da4f9700 1 Processor -- start 2024-01-27T02:36:05.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.678+0000 7fb8da4f9700 1 -- start start 2024-01-27T02:36:05.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.679+0000 7fb8da4f9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d4158ab0 0x7fb8d4156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:05.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.679+0000 7fb8da4f9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d4157610 con 0x7fb8d40b1c00 2024-01-27T02:36:05.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.679+0000 7fb8da4f9700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d4157310 con 0x7fb8d415b5f0 2024-01-27T02:36:05.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.679+0000 7fb8da4f9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d406ee00 con 0x7fb8d4158ab0 2024-01-27T02:36:05.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.679+0000 7fb8d3fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb8d40b1c00 0x7fb8d406e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40630/0 (socket says 172.21.15.7:40630) 2024-01-27T02:36:05.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.679+0000 7fb8d3fff700 1 -- 172.21.15.7:0/3183757574 learned_addr learned my addr 172.21.15.7:0/3183757574 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:05.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.679+0000 7fb8d2ffd700 1 -- 172.21.15.7:0/3183757574 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151923178 0 0) 0x7fb8d4157610 con 0x7fb8d40b1c00 2024-01-27T02:36:05.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.679+0000 7fb8d2ffd700 1 -- 172.21.15.7:0/3183757574 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8c0003570 con 0x7fb8d40b1c00 2024-01-27T02:36:05.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.679+0000 7fb8d8a96700 1 --2- 172.21.15.7:0/3183757574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d4158ab0 0x7fb8d4156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:05.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.680+0000 7fb8d2ffd700 1 -- 172.21.15.7:0/3183757574 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3079878773 0 0) 0x7fb8d4157310 con 0x7fb8d415b5f0 2024-01-27T02:36:05.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.680+0000 7fb8d2ffd700 1 -- 172.21.15.7:0/3183757574 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8d4157610 con 0x7fb8d415b5f0 2024-01-27T02:36:05.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.680+0000 7fb8d2ffd700 1 -- 172.21.15.7:0/3183757574 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 905165130 0 0) 0x7fb8c0003570 con 0x7fb8d40b1c00 2024-01-27T02:36:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.680+0000 7fb8d2ffd700 1 -- 172.21.15.7:0/3183757574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d4158ab0 msgr2=0x7fb8d4156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.680+0000 7fb8d2ffd700 1 --2- 172.21.15.7:0/3183757574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d4158ab0 0x7fb8d4156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.680+0000 7fb8d2ffd700 1 -- 172.21.15.7:0/3183757574 >> v1:172.21.15.188:6789/0 conn(0x7fb8d415b5f0 legacy=0x7fb8d41566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.680+0000 7fb8d2ffd700 1 -- 172.21.15.7:0/3183757574 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d406f0e0 con 0x7fb8d40b1c00 2024-01-27T02:36:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.680+0000 7fb8d2ffd700 1 -- 172.21.15.7:0/3183757574 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7fb8c4001470 con 0x7fb8d40b1c00 2024-01-27T02:36:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.680+0000 7fb8d2ffd700 1 -- 172.21.15.7:0/3183757574 >> v1:172.21.15.7:6789/0 conn(0x7fb8d40b1c00 legacy=0x7fb8d406e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.680+0000 7fb8d2ffd700 1 --2- 172.21.15.7:0/3183757574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d4158ab0 0x7fb8c0003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.680+0000 7fb8d2ffd700 1 --2- 172.21.15.7:0/3183757574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c0004180 0x7fb8c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.680+0000 7fb8d2ffd700 1 --2- 172.21.15.7:0/3183757574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c0004d50 0x7fb8c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.680+0000 7fb8d2ffd700 1 -- 172.21.15.7:0/3183757574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d406ee00 con 0x7fb8c0004180 2024-01-27T02:36:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.680+0000 7fb8d2ffd700 1 -- 172.21.15.7:0/3183757574 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c000bba0 con 0x7fb8c0004d50 2024-01-27T02:36:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.680+0000 7fb8d2ffd700 1 -- 172.21.15.7:0/3183757574 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c000bd60 con 0x7fb8d4158ab0 2024-01-27T02:36:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.680+0000 7fb8d37fe700 1 --2- 172.21.15.7:0/3183757574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c0004180 0x7fb8c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.680+0000 7fb8d37fe700 1 -- 172.21.15.7:0/3183757574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d4158ab0 msgr2=0x7fb8c0003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:05.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.680+0000 7fb8d37fe700 1 --2- 172.21.15.7:0/3183757574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d4158ab0 0x7fb8c0003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.680+0000 7fb8d37fe700 1 -- 172.21.15.7:0/3183757574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c0004d50 msgr2=0x7fb8c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:05.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.680+0000 7fb8d37fe700 1 --2- 172.21.15.7:0/3183757574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c0004d50 0x7fb8c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.680+0000 7fb8d37fe700 1 -- 172.21.15.7:0/3183757574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c0003570 con 0x7fb8c0004180 2024-01-27T02:36:05.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.680+0000 7fb8d37fe700 1 --2- 172.21.15.7:0/3183757574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c0004180 0x7fb8c0004810 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc001a10 tx=0x7fb8bc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fc93cd28a70085 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:05.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.681+0000 7fb8d2ffd700 1 -- 172.21.15.7:0/3183757574 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc00c5d0 con 0x7fb8c0004180 2024-01-27T02:36:05.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.681+0000 7fb8d2ffd700 1 -- 172.21.15.7:0/3183757574 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8bc00c730 con 0x7fb8c0004180 2024-01-27T02:36:05.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.681+0000 7fb8d2ffd700 1 -- 172.21.15.7:0/3183757574 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc00cad0 con 0x7fb8c0004180 2024-01-27T02:36:05.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.681+0000 7fb8da4f9700 1 -- 172.21.15.7:0/3183757574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c0004180 msgr2=0x7fb8c0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:05.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.681+0000 7fb8da4f9700 1 --2- 172.21.15.7:0/3183757574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c0004180 0x7fb8c0004810 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc001a10 tx=0x7fb8bc007d30 comp rx=0 tx=0).stop 2024-01-27T02:36:05.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.681+0000 7fb8d8a96700 1 -- 172.21.15.7:0/3183757574 reap_dead start 2024-01-27T02:36:05.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.681+0000 7fb8da4f9700 1 -- 172.21.15.7:0/3183757574 shutdown_connections 2024-01-27T02:36:05.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.681+0000 7fb8da4f9700 1 -- 172.21.15.7:0/3183757574 >> 172.21.15.7:0/3183757574 conn(0x7fb8d4068430 msgr2=0x7fb8d406be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:05.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.681+0000 7fb8da4f9700 1 -- 172.21.15.7:0/3183757574 shutdown_connections 2024-01-27T02:36:05.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.681+0000 7fb8da4f9700 1 -- 172.21.15.7:0/3183757574 wait complete. 2024-01-27T02:36:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.682+0000 7fb8da4f9700 1 Processor -- start 2024-01-27T02:36:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.683+0000 7fb8da4f9700 1 -- start start 2024-01-27T02:36:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.683+0000 7fb8da4f9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c0004180 0x7fb8d4158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.683+0000 7fb8da4f9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d40aec50 0x7fb8d416df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.683+0000 7fb8da4f9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8d414b330 0x7fb8d414b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.683+0000 7fb8da4f9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d414d380 con 0x7fb8c0004180 2024-01-27T02:36:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.683+0000 7fb8da4f9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d414d900 con 0x7fb8d414b330 2024-01-27T02:36:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.683+0000 7fb8da4f9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d40af760 con 0x7fb8d40aec50 2024-01-27T02:36:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.684+0000 7fb8d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c0004180 0x7fb8d4158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.684+0000 7fb8d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c0004180 0x7fb8d4158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33448/0 (socket says 172.21.15.7:33448) 2024-01-27T02:36:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.684+0000 7fb8d3fff700 1 -- 172.21.15.7:0/3013912099 learned_addr learned my addr 172.21.15.7:0/3013912099 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.684+0000 7fb8d37fe700 1 --2- 172.21.15.7:0/3013912099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d40aec50 0x7fb8d416df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.684+0000 7fb8d8a96700 1 --2- 172.21.15.7:0/3013912099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8d414b330 0x7fb8d414b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.684+0000 7fb8d3fff700 1 -- 172.21.15.7:0/3013912099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d40aec50 msgr2=0x7fb8d416df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.684+0000 7fb8d3fff700 1 --2- 172.21.15.7:0/3013912099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d40aec50 0x7fb8d416df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.684+0000 7fb8d3fff700 1 -- 172.21.15.7:0/3013912099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8d414b330 msgr2=0x7fb8d414b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.684+0000 7fb8d3fff700 1 --2- 172.21.15.7:0/3013912099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8d414b330 0x7fb8d414b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.684+0000 7fb8d3fff700 1 -- 172.21.15.7:0/3013912099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d414bcc0 con 0x7fb8c0004180 2024-01-27T02:36:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.684+0000 7fb8d3fff700 1 --2- 172.21.15.7:0/3013912099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c0004180 0x7fb8d4158390 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400c7a0 tx=0x7fb8d40b2fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.684+0000 7fb8cbfff700 1 -- 172.21.15.7:0/3013912099 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4007360 con 0x7fb8c0004180 2024-01-27T02:36:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.684+0000 7fb8cbfff700 1 -- 172.21.15.7:0/3013912099 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c40074c0 con 0x7fb8c0004180 2024-01-27T02:36:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.684+0000 7fb8cbfff700 1 -- 172.21.15.7:0/3013912099 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4004830 con 0x7fb8c0004180 2024-01-27T02:36:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.684+0000 7fb8da4f9700 1 -- 172.21.15.7:0/3013912099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d40adce0 con 0x7fb8c0004180 2024-01-27T02:36:05.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.684+0000 7fb8da4f9700 1 -- 172.21.15.7:0/3013912099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d40af480 con 0x7fb8c0004180 2024-01-27T02:36:05.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.685+0000 7fb8da4f9700 1 -- 172.21.15.7:0/3013912099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d404c710 con 0x7fb8c0004180 2024-01-27T02:36:05.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.686+0000 7fb8cbfff700 1 -- 172.21.15.7:0/3013912099 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb8c4004ba0 con 0x7fb8c0004180 2024-01-27T02:36:05.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.686+0000 7fb8cbfff700 1 --2- 172.21.15.7:0/3013912099 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb8b4066f30 0x7fb8b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:05.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.686+0000 7fb8cbfff700 1 -- 172.21.15.7:0/3013912099 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb8c4084220 con 0x7fb8c0004180 2024-01-27T02:36:05.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.687+0000 7fb8d37fe700 1 --2- 172.21.15.7:0/3013912099 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb8b4066f30 0x7fb8b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:05.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.687+0000 7fb8d37fe700 1 --2- 172.21.15.7:0/3013912099 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb8b4066f30 0x7fb8b40693e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb8d415ab50 tx=0x7fb8bc015000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:05.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.691+0000 7fb8cbfff700 1 -- 172.21.15.7:0/3013912099 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8c4051510 con 0x7fb8c0004180 2024-01-27T02:36:05.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.895+0000 7fb8da4f9700 1 -- 172.21.15.7:0/3013912099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8d40b24f0 con 0x7fb8c0004180 2024-01-27T02:36:05.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.896+0000 7fb8cbfff700 1 -- 172.21.15.7:0/3013912099 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+15305 (secure 0 0 0) 0x7fb8c4054b30 con 0x7fb8c0004180 2024-01-27T02:36:05.896 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:36:05.896 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:36:03.085405+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:35:59.537952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-27T02:36:00.480234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:36:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.898+0000 7fb8da4f9700 1 -- 172.21.15.7:0/3013912099 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb8b4066f30 msgr2=0x7fb8b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.898+0000 7fb8da4f9700 1 --2- 172.21.15.7:0/3013912099 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb8b4066f30 0x7fb8b40693e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb8d415ab50 tx=0x7fb8bc015000 comp rx=0 tx=0).stop 2024-01-27T02:36:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.898+0000 7fb8da4f9700 1 -- 172.21.15.7:0/3013912099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c0004180 msgr2=0x7fb8d4158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.898+0000 7fb8da4f9700 1 --2- 172.21.15.7:0/3013912099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c0004180 0x7fb8d4158390 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400c7a0 tx=0x7fb8d40b2fe0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.898+0000 7fb8da4f9700 1 -- 172.21.15.7:0/3013912099 shutdown_connections 2024-01-27T02:36:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.898+0000 7fb8da4f9700 1 --2- 172.21.15.7:0/3013912099 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb8b4066f30 0x7fb8b40693e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.898+0000 7fb8da4f9700 1 --2- 172.21.15.7:0/3013912099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8d414b330 0x7fb8d414b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.898+0000 7fb8da4f9700 1 --2- 172.21.15.7:0/3013912099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c0004180 0x7fb8d4158390 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.898+0000 7fb8da4f9700 1 --2- 172.21.15.7:0/3013912099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d40aec50 0x7fb8d416df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.898+0000 7fb8da4f9700 1 -- 172.21.15.7:0/3013912099 >> 172.21.15.7:0/3013912099 conn(0x7fb8d4068430 msgr2=0x7fb8d40cdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.898+0000 7fb8da4f9700 1 -- 172.21.15.7:0/3013912099 shutdown_connections 2024-01-27T02:36:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:05.898+0000 7fb8da4f9700 1 -- 172.21.15.7:0/3013912099 wait complete. 2024-01-27T02:36:05.912 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-27T02:36:05.913 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:36:06.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.044+0000 7f6fb9080700 1 Processor -- start 2024-01-27T02:36:06.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.044+0000 7f6fb9080700 1 -- start start 2024-01-27T02:36:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.045+0000 7f6fb9080700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fb40ca540 0x7f6fb40cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.045+0000 7f6fb9080700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb40ce5e0 con 0x7f6fb40c4cc0 2024-01-27T02:36:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.045+0000 7f6fb9080700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb40cd9d0 con 0x7f6fb415a100 2024-01-27T02:36:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.045+0000 7f6fb9080700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb40cdcb0 con 0x7f6fb40ca540 2024-01-27T02:36:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.045+0000 7f6fb259c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6fb40c4cc0 0x7f6fb4161d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40632/0 (socket says 172.21.15.7:40632) 2024-01-27T02:36:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.045+0000 7f6fb259c700 1 -- 172.21.15.7:0/1756273993 learned_addr learned my addr 172.21.15.7:0/1756273993 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.045+0000 7f6fb359e700 1 --2- 172.21.15.7:0/1756273993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fb40ca540 0x7f6fb40cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.045+0000 7f6fb1d9b700 1 -- 172.21.15.7:0/1756273993 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373062818 0 0) 0x7f6fb40ce5e0 con 0x7f6fb40c4cc0 2024-01-27T02:36:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.045+0000 7f6fb1d9b700 1 -- 172.21.15.7:0/1756273993 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f9c003570 con 0x7f6fb40c4cc0 2024-01-27T02:36:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.045+0000 7f6fb1d9b700 1 -- 172.21.15.7:0/1756273993 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217555394 0 0) 0x7f6fb40cd9d0 con 0x7f6fb415a100 2024-01-27T02:36:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.045+0000 7f6fb1d9b700 1 -- 172.21.15.7:0/1756273993 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fb40ce5e0 con 0x7f6fb415a100 2024-01-27T02:36:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.045+0000 7f6fb1d9b700 1 -- 172.21.15.7:0/1756273993 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 829964332 0 0) 0x7f6f9c003570 con 0x7f6fb40c4cc0 2024-01-27T02:36:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.045+0000 7f6fb1d9b700 1 -- 172.21.15.7:0/1756273993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fb40ca540 msgr2=0x7f6fb40cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:06.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.045+0000 7f6fb1d9b700 1 --2- 172.21.15.7:0/1756273993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fb40ca540 0x7f6fb40cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.045+0000 7f6fb1d9b700 1 -- 172.21.15.7:0/1756273993 >> v1:172.21.15.188:6789/0 conn(0x7f6fb415a100 legacy=0x7f6fb40c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:06.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.045+0000 7f6fb1d9b700 1 -- 172.21.15.7:0/1756273993 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb40cdf90 con 0x7f6fb40c4cc0 2024-01-27T02:36:06.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.045+0000 7f6fb1d9b700 1 -- 172.21.15.7:0/1756273993 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f6fa4001470 con 0x7f6fb40c4cc0 2024-01-27T02:36:06.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.046+0000 7f6fb1d9b700 1 -- 172.21.15.7:0/1756273993 >> v1:172.21.15.7:6789/0 conn(0x7f6fb40c4cc0 legacy=0x7f6fb4161d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:06.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.046+0000 7f6fb1d9b700 1 --2- 172.21.15.7:0/1756273993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fb40ca540 0x7f6f9c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:06.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.046+0000 7f6fb1d9b700 1 --2- 172.21.15.7:0/1756273993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f9c0041d0 0x7f6f9c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:06.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.046+0000 7f6fb1d9b700 1 --2- 172.21.15.7:0/1756273993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f9c004d50 0x7f6f9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:06.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.046+0000 7f6fb1d9b700 1 -- 172.21.15.7:0/1756273993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb40cdcb0 con 0x7f6f9c004d50 2024-01-27T02:36:06.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.046+0000 7f6fb1d9b700 1 -- 172.21.15.7:0/1756273993 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c00bbf0 con 0x7f6f9c0041d0 2024-01-27T02:36:06.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.046+0000 7f6fb1d9b700 1 -- 172.21.15.7:0/1756273993 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c00bdb0 con 0x7f6fb40ca540 2024-01-27T02:36:06.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.046+0000 7f6fb359e700 1 --2- 172.21.15.7:0/1756273993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f9c004d50 0x7f6f9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:06.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.046+0000 7f6fb2d9d700 1 --2- 172.21.15.7:0/1756273993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fb40ca540 0x7f6f9c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:06.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.046+0000 7f6fb259c700 1 --2- 172.21.15.7:0/1756273993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f9c0041d0 0x7f6f9c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.046+0000 7f6fb359e700 1 -- 172.21.15.7:0/1756273993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fb40ca540 msgr2=0x7f6f9c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.046+0000 7f6fb359e700 1 --2- 172.21.15.7:0/1756273993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fb40ca540 0x7f6f9c003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.046+0000 7f6fb359e700 1 -- 172.21.15.7:0/1756273993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f9c0041d0 msgr2=0x7f6f9c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.046+0000 7f6fb359e700 1 --2- 172.21.15.7:0/1756273993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f9c0041d0 0x7f6f9c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.046+0000 7f6fb359e700 1 -- 172.21.15.7:0/1756273993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c003570 con 0x7f6f9c004d50 2024-01-27T02:36:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.046+0000 7f6fb359e700 1 --2- 172.21.15.7:0/1756273993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f9c004d50 0x7f6f9c0051c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00e210 tx=0x7f6fac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3a38f25ffaf757 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.046+0000 7f6fb1d9b700 1 -- 172.21.15.7:0/1756273993 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac00eac0 con 0x7f6f9c004d50 2024-01-27T02:36:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.046+0000 7f6fb1d9b700 1 -- 172.21.15.7:0/1756273993 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fac00f040 con 0x7f6f9c004d50 2024-01-27T02:36:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.046+0000 7f6fb1d9b700 1 -- 172.21.15.7:0/1756273993 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac00ee70 con 0x7f6f9c004d50 2024-01-27T02:36:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.047+0000 7f6fb9080700 1 -- 172.21.15.7:0/1756273993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f9c004d50 msgr2=0x7f6f9c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.047+0000 7f6fb9080700 1 --2- 172.21.15.7:0/1756273993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f9c004d50 0x7f6f9c0051c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00e210 tx=0x7f6fac00b040 comp rx=0 tx=0).stop 2024-01-27T02:36:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.047+0000 7f6fb359e700 1 -- 172.21.15.7:0/1756273993 reap_dead start 2024-01-27T02:36:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.047+0000 7f6fb9080700 1 -- 172.21.15.7:0/1756273993 shutdown_connections 2024-01-27T02:36:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.047+0000 7f6fb9080700 1 -- 172.21.15.7:0/1756273993 >> 172.21.15.7:0/1756273993 conn(0x7f6fb4098020 msgr2=0x7f6fb40711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.047+0000 7f6fb9080700 1 -- 172.21.15.7:0/1756273993 shutdown_connections 2024-01-27T02:36:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.047+0000 7f6fb9080700 1 -- 172.21.15.7:0/1756273993 wait complete. 2024-01-27T02:36:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.049+0000 7f6fb9080700 1 Processor -- start 2024-01-27T02:36:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.049+0000 7f6fb9080700 1 -- start start 2024-01-27T02:36:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.050+0000 7f6fb9080700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fb414ebb0 0x7f6fb40b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.050+0000 7f6fb9080700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fb40c8710 0x7f6fb416b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.050+0000 7f6fb9080700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb416b550 0x7f6fb40b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.050+0000 7f6fb9080700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb40c6aa0 con 0x7f6fb416b550 2024-01-27T02:36:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.050+0000 7f6fb9080700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb40c5e90 con 0x7f6fb414ebb0 2024-01-27T02:36:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.050+0000 7f6fb9080700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb40c6170 con 0x7f6fb40c8710 2024-01-27T02:36:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.050+0000 7f6fb359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb416b550 0x7f6fb40b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.050+0000 7f6fb359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb416b550 0x7f6fb40b3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33458/0 (socket says 172.21.15.7:33458) 2024-01-27T02:36:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.050+0000 7f6fb359e700 1 -- 172.21.15.7:0/2167761701 learned_addr learned my addr 172.21.15.7:0/2167761701 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.050+0000 7f6fb259c700 1 --2- 172.21.15.7:0/2167761701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fb40c8710 0x7f6fb416b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.050+0000 7f6fb2d9d700 1 --2- 172.21.15.7:0/2167761701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fb414ebb0 0x7f6fb40b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.050+0000 7f6fb359e700 1 -- 172.21.15.7:0/2167761701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fb40c8710 msgr2=0x7f6fb416b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.050+0000 7f6fb359e700 1 --2- 172.21.15.7:0/2167761701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fb40c8710 0x7f6fb416b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.050+0000 7f6fb359e700 1 -- 172.21.15.7:0/2167761701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fb414ebb0 msgr2=0x7f6fb40b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.050+0000 7f6fb359e700 1 --2- 172.21.15.7:0/2167761701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fb414ebb0 0x7f6fb40b24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.050+0000 7f6fb359e700 1 -- 172.21.15.7:0/2167761701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb40c6450 con 0x7f6fb416b550 2024-01-27T02:36:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.050+0000 7f6fb2d9d700 1 --2- 172.21.15.7:0/2167761701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fb414ebb0 0x7f6fb40b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:36:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.051+0000 7f6fb359e700 1 --2- 172.21.15.7:0/2167761701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb416b550 0x7f6fb40b3250 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6fac016000 tx=0x7f6fac01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.051+0000 7f6f9affd700 1 -- 172.21.15.7:0/2167761701 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac027430 con 0x7f6fb416b550 2024-01-27T02:36:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.051+0000 7f6f9affd700 1 -- 172.21.15.7:0/2167761701 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fac022070 con 0x7f6fb416b550 2024-01-27T02:36:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.051+0000 7f6f9affd700 1 -- 172.21.15.7:0/2167761701 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac027800 con 0x7f6fb416b550 2024-01-27T02:36:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.051+0000 7f6fb9080700 1 -- 172.21.15.7:0/2167761701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb40c6730 con 0x7f6fb416b550 2024-01-27T02:36:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.051+0000 7f6fb9080700 1 -- 172.21.15.7:0/2167761701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb414ef80 con 0x7f6fb416b550 2024-01-27T02:36:06.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.052+0000 7f6fb9080700 1 -- 172.21.15.7:0/2167761701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb414f170 con 0x7f6fb416b550 2024-01-27T02:36:06.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.052+0000 7f6f9affd700 1 -- 172.21.15.7:0/2167761701 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6fac024030 con 0x7f6fb416b550 2024-01-27T02:36:06.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.053+0000 7f6f9affd700 1 --2- 172.21.15.7:0/2167761701 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6fa0066f30 0x7f6fa00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:06.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.053+0000 7f6f9affd700 1 -- 172.21.15.7:0/2167761701 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6fac086a70 con 0x7f6fb416b550 2024-01-27T02:36:06.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.053+0000 7f6fb2d9d700 1 --2- 172.21.15.7:0/2167761701 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6fa0066f30 0x7f6fa00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:06.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.054+0000 7f6fb2d9d700 1 --2- 172.21.15.7:0/2167761701 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6fa0066f30 0x7f6fa00693e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8000f60 tx=0x7f6fa8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:06.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.057+0000 7f6f9affd700 1 -- 172.21.15.7:0/2167761701 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6fac053ce0 con 0x7f6fb416b550 2024-01-27T02:36:06.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.259+0000 7f6fb9080700 1 -- 172.21.15.7:0/2167761701 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6fb404c710 con 0x7f6fa0066f30 2024-01-27T02:36:06.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.268+0000 7f6f9affd700 1 -- 172.21.15.7:0/2167761701 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282722 (secure 0 0 0) 0x7f6fb404c710 con 0x7f6fa0066f30 2024-01-27T02:36:06.273 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:36:06.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.271+0000 7f6fb9080700 1 -- 172.21.15.7:0/2167761701 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6fa0066f30 msgr2=0x7f6fa00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:06.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.271+0000 7f6fb9080700 1 --2- 172.21.15.7:0/2167761701 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6fa0066f30 0x7f6fa00693e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8000f60 tx=0x7f6fa8009040 comp rx=0 tx=0).stop 2024-01-27T02:36:06.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.271+0000 7f6fb9080700 1 -- 172.21.15.7:0/2167761701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb416b550 msgr2=0x7f6fb40b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:06.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.271+0000 7f6fb9080700 1 --2- 172.21.15.7:0/2167761701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb416b550 0x7f6fb40b3250 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6fac016000 tx=0x7f6fac01a000 comp rx=0 tx=0).stop 2024-01-27T02:36:06.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.272+0000 7f6fb9080700 1 -- 172.21.15.7:0/2167761701 shutdown_connections 2024-01-27T02:36:06.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.272+0000 7f6fb9080700 1 --2- 172.21.15.7:0/2167761701 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6fa0066f30 0x7f6fa00693e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.272+0000 7f6fb9080700 1 --2- 172.21.15.7:0/2167761701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fb416b550 0x7f6fb40b3250 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.272+0000 7f6fb9080700 1 --2- 172.21.15.7:0/2167761701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fb414ebb0 0x7f6fb40b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.272+0000 7f6fb9080700 1 --2- 172.21.15.7:0/2167761701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fb40c8710 0x7f6fb416b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.272+0000 7f6fb9080700 1 -- 172.21.15.7:0/2167761701 >> 172.21.15.7:0/2167761701 conn(0x7f6fb4098020 msgr2=0x7f6fb4154f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:06.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.272+0000 7f6fb9080700 1 -- 172.21.15.7:0/2167761701 shutdown_connections 2024-01-27T02:36:06.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.272+0000 7f6fb9080700 1 -- 172.21.15.7:0/2167761701 wait complete. 2024-01-27T02:36:06.276 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:36:06.286 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2024-01-27T02:36:05.631498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4343,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558156,"statfs":{"total":1600000000,"available":1595552251,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.567011"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077060+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:19:59.227343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078576+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:09.241354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.071538+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:51:24.783765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078657+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:45:48.321027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079475+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:17:28.880573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.080893+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:25:43.791658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078520+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:57:20.844278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077646+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:16:42.015521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078957+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:25.098588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.081944+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:21:41.100136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076679+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:57:20.844278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078947+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:39:19.173475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.080386+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:32:03.288747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077576+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:41:44.104324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079892+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:26:17.309915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079382+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:28:19.604756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.080873+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:47:56.363317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.080422+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:10:26.712434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.075169+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:15:05.885263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079455+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:19:38.789133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078990+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:18:04.141154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079678+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:25:43.791658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.075028+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:26:17.309915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076931+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:12:02.170281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.081562+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:10:52.422378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076922+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:21.383627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.071521+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:51:11.452504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078331+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:09.241354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.071456+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:37:39.458923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076978+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:48:39.299949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076528+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:41:50.546172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078193+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:27:01.184977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076537+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:48:39.299949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.077741+0000","last_change":"2024-01-27T02:36:01.077741+0000","last_active":"2024-01-27T02:36:01.077741+0000","last_peered":"2024-01-27T02:36:01.077741+0000","last_clean":"2024-01-27T02:36:01.077741+0000","last_became_active":"2024-01-27T02:36:01.077423+0000","last_became_peered":"2024-01-27T02:36:01.077423+0000","last_unstale":"2024-01-27T02:36:01.077741+0000","last_undegraded":"2024-01-27T02:36:01.077741+0000","last_fullsized":"2024-01-27T02:36:01.077741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:15.272806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077778+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:09:23.740404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.084751+0000","last_change":"2024-01-27T02:36:01.084751+0000","last_active":"2024-01-27T02:36:01.084751+0000","last_peered":"2024-01-27T02:36:01.084751+0000","last_clean":"2024-01-27T02:36:01.084751+0000","last_became_active":"2024-01-27T02:36:01.084351+0000","last_became_peered":"2024-01-27T02:36:01.084351+0000","last_unstale":"2024-01-27T02:36:01.084751+0000","last_undegraded":"2024-01-27T02:36:01.084751+0000","last_fullsized":"2024-01-27T02:36:01.084751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:42:38.497144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.080521+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:31:44.339745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.087593+0000","last_change":"2024-01-27T02:36:01.087593+0000","last_active":"2024-01-27T02:36:01.087593+0000","last_peered":"2024-01-27T02:36:01.087593+0000","last_clean":"2024-01-27T02:36:01.087593+0000","last_became_active":"2024-01-27T02:36:01.086695+0000","last_became_peered":"2024-01-27T02:36:01.086695+0000","last_unstale":"2024-01-27T02:36:01.087593+0000","last_undegraded":"2024-01-27T02:36:01.087593+0000","last_fullsized":"2024-01-27T02:36:01.087593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:15:04.879041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077553+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:56:33.979247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.082918+0000","last_change":"2024-01-27T02:36:01.082918+0000","last_active":"2024-01-27T02:36:01.082918+0000","last_peered":"2024-01-27T02:36:01.082918+0000","last_clean":"2024-01-27T02:36:01.082918+0000","last_became_active":"2024-01-27T02:36:01.081169+0000","last_became_peered":"2024-01-27T02:36:01.081169+0000","last_unstale":"2024-01-27T02:36:01.082918+0000","last_undegraded":"2024-01-27T02:36:01.082918+0000","last_fullsized":"2024-01-27T02:36:01.082918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:39:18.167253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.075988+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:32:03.288747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.083508+0000","last_change":"2024-01-27T02:36:01.083508+0000","last_active":"2024-01-27T02:36:01.083508+0000","last_peered":"2024-01-27T02:36:01.083508+0000","last_clean":"2024-01-27T02:36:01.083508+0000","last_became_active":"2024-01-27T02:36:01.083244+0000","last_became_peered":"2024-01-27T02:36:01.083244+0000","last_unstale":"2024-01-27T02:36:01.083508+0000","last_undegraded":"2024-01-27T02:36:01.083508+0000","last_fullsized":"2024-01-27T02:36:01.083508+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:42:06.107055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079272+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:31:44.339745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.081011+0000","last_change":"2024-01-27T02:36:01.081011+0000","last_active":"2024-01-27T02:36:01.081011+0000","last_peered":"2024-01-27T02:36:01.081011+0000","last_clean":"2024-01-27T02:36:01.081011+0000","last_became_active":"2024-01-27T02:36:01.080166+0000","last_became_peered":"2024-01-27T02:36:01.080166+0000","last_unstale":"2024-01-27T02:36:01.081011+0000","last_undegraded":"2024-01-27T02:36:01.081011+0000","last_fullsized":"2024-01-27T02:36:01.081011+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:42:38.578182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079075+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:03:33.449715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.086427+0000","last_change":"2024-01-27T02:36:01.086427+0000","last_active":"2024-01-27T02:36:01.086427+0000","last_peered":"2024-01-27T02:36:01.086427+0000","last_clean":"2024-01-27T02:36:01.086427+0000","last_became_active":"2024-01-27T02:36:01.085912+0000","last_became_peered":"2024-01-27T02:36:01.085912+0000","last_unstale":"2024-01-27T02:36:01.086427+0000","last_undegraded":"2024-01-27T02:36:01.086427+0000","last_fullsized":"2024-01-27T02:36:01.086427+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:12:59.343148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079659+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:21:28.748243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.084880+0000","last_change":"2024-01-27T02:36:01.084880+0000","last_active":"2024-01-27T02:36:01.084880+0000","last_peered":"2024-01-27T02:36:01.084880+0000","last_clean":"2024-01-27T02:36:01.084880+0000","last_became_active":"2024-01-27T02:36:01.084532+0000","last_became_peered":"2024-01-27T02:36:01.084532+0000","last_unstale":"2024-01-27T02:36:01.084880+0000","last_undegraded":"2024-01-27T02:36:01.084880+0000","last_fullsized":"2024-01-27T02:36:01.084880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:58:04.302291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076109+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:32:03.273755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.083170+0000","last_change":"2024-01-27T02:36:01.083170+0000","last_active":"2024-01-27T02:36:01.083170+0000","last_peered":"2024-01-27T02:36:01.083170+0000","last_clean":"2024-01-27T02:36:01.083170+0000","last_became_active":"2024-01-27T02:36:01.082730+0000","last_became_peered":"2024-01-27T02:36:01.082730+0000","last_unstale":"2024-01-27T02:36:01.083170+0000","last_undegraded":"2024-01-27T02:36:01.083170+0000","last_fullsized":"2024-01-27T02:36:01.083170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:07:19.984794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076273+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:21.383627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.079850+0000","last_change":"2024-01-27T02:36:01.079850+0000","last_active":"2024-01-27T02:36:01.079850+0000","last_peered":"2024-01-27T02:36:01.079850+0000","last_clean":"2024-01-27T02:36:01.079850+0000","last_became_active":"2024-01-27T02:36:01.079488+0000","last_became_peered":"2024-01-27T02:36:01.079488+0000","last_unstale":"2024-01-27T02:36:01.079850+0000","last_undegraded":"2024-01-27T02:36:01.079850+0000","last_fullsized":"2024-01-27T02:36:01.079850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:36:51.217544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.074770+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:35:14.198571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.075921+0000","last_change":"2024-01-27T02:36:01.075921+0000","last_active":"2024-01-27T02:36:01.075921+0000","last_peered":"2024-01-27T02:36:01.075921+0000","last_clean":"2024-01-27T02:36:01.075921+0000","last_became_active":"2024-01-27T02:36:01.075647+0000","last_became_peered":"2024-01-27T02:36:01.075647+0000","last_unstale":"2024-01-27T02:36:01.075921+0000","last_undegraded":"2024-01-27T02:36:01.075921+0000","last_fullsized":"2024-01-27T02:36:01.075921+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:54:58.712360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077216+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:22:30.371211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.085134+0000","last_change":"2024-01-27T02:36:01.085134+0000","last_active":"2024-01-27T02:36:01.085134+0000","last_peered":"2024-01-27T02:36:01.085134+0000","last_clean":"2024-01-27T02:36:01.085134+0000","last_became_active":"2024-01-27T02:36:01.084887+0000","last_became_peered":"2024-01-27T02:36:01.084887+0000","last_unstale":"2024-01-27T02:36:01.085134+0000","last_undegraded":"2024-01-27T02:36:01.085134+0000","last_fullsized":"2024-01-27T02:36:01.085134+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:24:30.352994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.075834+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:14:13.651138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.081854+0000","last_change":"2024-01-27T02:36:01.081854+0000","last_active":"2024-01-27T02:36:01.081854+0000","last_peered":"2024-01-27T02:36:01.081854+0000","last_clean":"2024-01-27T02:36:01.081854+0000","last_became_active":"2024-01-27T02:36:01.081553+0000","last_became_peered":"2024-01-27T02:36:01.081553+0000","last_unstale":"2024-01-27T02:36:01.081854+0000","last_undegraded":"2024-01-27T02:36:01.081854+0000","last_fullsized":"2024-01-27T02:36:01.081854+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:36:51.217544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076474+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:14:13.651138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.087113+0000","last_change":"2024-01-27T02:36:01.087113+0000","last_active":"2024-01-27T02:36:01.087113+0000","last_peered":"2024-01-27T02:36:01.087113+0000","last_clean":"2024-01-27T02:36:01.087113+0000","last_became_active":"2024-01-27T02:36:01.086735+0000","last_became_peered":"2024-01-27T02:36:01.086735+0000","last_unstale":"2024-01-27T02:36:01.087113+0000","last_undegraded":"2024-01-27T02:36:01.087113+0000","last_fullsized":"2024-01-27T02:36:01.087113+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:51:30.355507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079910+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:04:07.439211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.079455+0000","last_change":"2024-01-27T02:36:01.079455+0000","last_active":"2024-01-27T02:36:01.079455+0000","last_peered":"2024-01-27T02:36:01.079455+0000","last_clean":"2024-01-27T02:36:01.079455+0000","last_became_active":"2024-01-27T02:36:01.078810+0000","last_became_peered":"2024-01-27T02:36:01.078810+0000","last_unstale":"2024-01-27T02:36:01.079455+0000","last_undegraded":"2024-01-27T02:36:01.079455+0000","last_fullsized":"2024-01-27T02:36:01.079455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:32:57.446408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078346+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:32:55.342184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.083387+0000","last_change":"2024-01-27T02:36:01.083387+0000","last_active":"2024-01-27T02:36:01.083387+0000","last_peered":"2024-01-27T02:36:01.083387+0000","last_clean":"2024-01-27T02:36:01.083387+0000","last_became_active":"2024-01-27T02:36:01.083023+0000","last_became_peered":"2024-01-27T02:36:01.083023+0000","last_unstale":"2024-01-27T02:36:01.083387+0000","last_undegraded":"2024-01-27T02:36:01.083387+0000","last_fullsized":"2024-01-27T02:36:01.083387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:16:41.009300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079871+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:42.090153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.075386+0000","last_change":"2024-01-27T02:36:01.075386+0000","last_active":"2024-01-27T02:36:01.075386+0000","last_peered":"2024-01-27T02:36:01.075386+0000","last_clean":"2024-01-27T02:36:01.075386+0000","last_became_active":"2024-01-27T02:36:01.074276+0000","last_became_peered":"2024-01-27T02:36:01.074276+0000","last_unstale":"2024-01-27T02:36:01.075386+0000","last_undegraded":"2024-01-27T02:36:01.075386+0000","last_fullsized":"2024-01-27T02:36:01.075386+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:46:34.597904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076123+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:41:50.546172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.087262+0000","last_change":"2024-01-27T02:36:01.087262+0000","last_active":"2024-01-27T02:36:01.087262+0000","last_peered":"2024-01-27T02:36:01.087262+0000","last_clean":"2024-01-27T02:36:01.087262+0000","last_became_active":"2024-01-27T02:36:01.086163+0000","last_became_peered":"2024-01-27T02:36:01.086163+0000","last_unstale":"2024-01-27T02:36:01.087262+0000","last_undegraded":"2024-01-27T02:36:01.087262+0000","last_fullsized":"2024-01-27T02:36:01.087262+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:35:13.192349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078527+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:23:57.404805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.086968+0000","last_change":"2024-01-27T02:36:01.086968+0000","last_active":"2024-01-27T02:36:01.086968+0000","last_peered":"2024-01-27T02:36:01.086968+0000","last_clean":"2024-01-27T02:36:01.086968+0000","last_became_active":"2024-01-27T02:36:01.086112+0000","last_became_peered":"2024-01-27T02:36:01.086112+0000","last_unstale":"2024-01-27T02:36:01.086968+0000","last_undegraded":"2024-01-27T02:36:01.086968+0000","last_fullsized":"2024-01-27T02:36:01.086968+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:03:55.891925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078774+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:19:43.900445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.085172+0000","last_change":"2024-01-27T02:36:01.085172+0000","last_active":"2024-01-27T02:36:01.085172+0000","last_peered":"2024-01-27T02:36:01.085172+0000","last_clean":"2024-01-27T02:36:01.085172+0000","last_became_active":"2024-01-27T02:36:01.084894+0000","last_became_peered":"2024-01-27T02:36:01.084894+0000","last_unstale":"2024-01-27T02:36:01.085172+0000","last_undegraded":"2024-01-27T02:36:01.085172+0000","last_fullsized":"2024-01-27T02:36:01.085172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:19:37.782912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.070899+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:05:31.415970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.080657+0000","last_change":"2024-01-27T02:36:01.080657+0000","last_active":"2024-01-27T02:36:01.080657+0000","last_peered":"2024-01-27T02:36:01.080657+0000","last_clean":"2024-01-27T02:36:01.080657+0000","last_became_active":"2024-01-27T02:36:01.078442+0000","last_became_peered":"2024-01-27T02:36:01.078442+0000","last_unstale":"2024-01-27T02:36:01.080657+0000","last_undegraded":"2024-01-27T02:36:01.080657+0000","last_fullsized":"2024-01-27T02:36:01.080657+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:45:47.314805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.070938+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:22:09.618511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.085914+0000","last_change":"2024-01-27T02:36:01.085914+0000","last_active":"2024-01-27T02:36:01.085914+0000","last_peered":"2024-01-27T02:36:01.085914+0000","last_clean":"2024-01-27T02:36:01.085914+0000","last_became_active":"2024-01-27T02:36:01.085674+0000","last_became_peered":"2024-01-27T02:36:01.085674+0000","last_unstale":"2024-01-27T02:36:01.085914+0000","last_undegraded":"2024-01-27T02:36:01.085914+0000","last_fullsized":"2024-01-27T02:36:01.085914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:19:37.782912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.070886+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:47:18.259755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.086128+0000","last_change":"2024-01-27T02:36:01.086128+0000","last_active":"2024-01-27T02:36:01.086128+0000","last_peered":"2024-01-27T02:36:01.086128+0000","last_clean":"2024-01-27T02:36:01.086128+0000","last_became_active":"2024-01-27T02:36:01.085891+0000","last_became_peered":"2024-01-27T02:36:01.085891+0000","last_unstale":"2024-01-27T02:36:01.086128+0000","last_undegraded":"2024-01-27T02:36:01.086128+0000","last_fullsized":"2024-01-27T02:36:01.086128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:05:15.222635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079413+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:16.279028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.079712+0000","last_change":"2024-01-27T02:36:01.079712+0000","last_active":"2024-01-27T02:36:01.079712+0000","last_peered":"2024-01-27T02:36:01.079712+0000","last_clean":"2024-01-27T02:36:01.079712+0000","last_became_active":"2024-01-27T02:36:01.079176+0000","last_became_peered":"2024-01-27T02:36:01.079176+0000","last_unstale":"2024-01-27T02:36:01.079712+0000","last_undegraded":"2024-01-27T02:36:01.079712+0000","last_fullsized":"2024-01-27T02:36:01.079712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:12:59.343148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.075688+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:26:17.309915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.084252+0000","last_change":"2024-01-27T02:36:01.084252+0000","last_active":"2024-01-27T02:36:01.084252+0000","last_peered":"2024-01-27T02:36:01.084252+0000","last_clean":"2024-01-27T02:36:01.084252+0000","last_became_active":"2024-01-27T02:36:01.083785+0000","last_became_peered":"2024-01-27T02:36:01.083785+0000","last_unstale":"2024-01-27T02:36:01.084252+0000","last_undegraded":"2024-01-27T02:36:01.084252+0000","last_fullsized":"2024-01-27T02:36:01.084252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:15.272806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.075405+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:32:03.288747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.083357+0000","last_change":"2024-01-27T02:36:01.083357+0000","last_active":"2024-01-27T02:36:01.083357+0000","last_peered":"2024-01-27T02:36:01.083357+0000","last_clean":"2024-01-27T02:36:01.083357+0000","last_became_active":"2024-01-27T02:36:01.083009+0000","last_became_peered":"2024-01-27T02:36:01.083009+0000","last_unstale":"2024-01-27T02:36:01.083357+0000","last_undegraded":"2024-01-27T02:36:01.083357+0000","last_fullsized":"2024-01-27T02:36:01.083357+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:17:17.877802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.075713+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:32:03.273755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.083453+0000","last_change":"2024-01-27T02:36:01.083453+0000","last_active":"2024-01-27T02:36:01.083453+0000","last_peered":"2024-01-27T02:36:01.083453+0000","last_clean":"2024-01-27T02:36:01.083453+0000","last_became_active":"2024-01-27T02:36:01.083173+0000","last_became_peered":"2024-01-27T02:36:01.083173+0000","last_unstale":"2024-01-27T02:36:01.083453+0000","last_undegraded":"2024-01-27T02:36:01.083453+0000","last_fullsized":"2024-01-27T02:36:01.083453+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:21:40.093915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077155+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:11:23.338931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.086016+0000","last_change":"2024-01-27T02:36:01.086016+0000","last_active":"2024-01-27T02:36:01.086016+0000","last_peered":"2024-01-27T02:36:01.086016+0000","last_clean":"2024-01-27T02:36:01.086016+0000","last_became_active":"2024-01-27T02:36:01.085747+0000","last_became_peered":"2024-01-27T02:36:01.085747+0000","last_unstale":"2024-01-27T02:36:01.086016+0000","last_undegraded":"2024-01-27T02:36:01.086016+0000","last_fullsized":"2024-01-27T02:36:01.086016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:03:04.731782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.080121+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:18:04.141154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.085778+0000","last_change":"2024-01-27T02:36:01.085778+0000","last_active":"2024-01-27T02:36:01.085778+0000","last_peered":"2024-01-27T02:36:01.085778+0000","last_clean":"2024-01-27T02:36:01.085778+0000","last_became_active":"2024-01-27T02:36:01.085504+0000","last_became_peered":"2024-01-27T02:36:01.085504+0000","last_unstale":"2024-01-27T02:36:01.085778+0000","last_undegraded":"2024-01-27T02:36:01.085778+0000","last_fullsized":"2024-01-27T02:36:01.085778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:34:00.107552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078206+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:03:56.898146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.083238+0000","last_change":"2024-01-27T02:36:01.083238+0000","last_active":"2024-01-27T02:36:01.083238+0000","last_peered":"2024-01-27T02:36:01.083238+0000","last_clean":"2024-01-27T02:36:01.083238+0000","last_became_active":"2024-01-27T02:36:01.082768+0000","last_became_peered":"2024-01-27T02:36:01.082768+0000","last_unstale":"2024-01-27T02:36:01.083238+0000","last_undegraded":"2024-01-27T02:36:01.083238+0000","last_fullsized":"2024-01-27T02:36:01.083238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:34:52.370887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077264+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:34:53.377109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.078683+0000","last_change":"2024-01-27T02:36:01.078683+0000","last_active":"2024-01-27T02:36:01.078683+0000","last_peered":"2024-01-27T02:36:01.078683+0000","last_clean":"2024-01-27T02:36:01.078683+0000","last_became_active":"2024-01-27T02:36:01.078126+0000","last_became_peered":"2024-01-27T02:36:01.078126+0000","last_unstale":"2024-01-27T02:36:01.078683+0000","last_undegraded":"2024-01-27T02:36:01.078683+0000","last_fullsized":"2024-01-27T02:36:01.078683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:10:25.706212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.070886+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:09:00.736774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.080741+0000","last_change":"2024-01-27T02:36:01.080741+0000","last_active":"2024-01-27T02:36:01.080741+0000","last_peered":"2024-01-27T02:36:01.080741+0000","last_clean":"2024-01-27T02:36:01.080741+0000","last_became_active":"2024-01-27T02:36:01.079822+0000","last_became_peered":"2024-01-27T02:36:01.079822+0000","last_unstale":"2024-01-27T02:36:01.080741+0000","last_undegraded":"2024-01-27T02:36:01.080741+0000","last_fullsized":"2024-01-27T02:36:01.080741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:23:56.398583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":211,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-01-27T02:36:00.065172+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:36:00.065172+0000","last_peered":"2024-01-27T02:36:00.065172+0000","last_clean":"2024-01-27T02:36:00.065172+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:36:00.065172+0000","last_undegraded":"2024-01-27T02:36:00.065172+0000","last_fullsized":"2024-01-27T02:36:00.065172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":973,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194339,"statfs":{"total":200000000,"available":199003604,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.50900000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.41599999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.46899999999999997}]},{"osd":2,"last update":"Sat Jan 27 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.499,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.59299999999999997,"15min":0.58399999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.41499999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59799999999999998,"15min":0.59099999999999997},"min":{"1min":0.48199999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.59799999999999998,"15min":0.59099999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.45500000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48999999999999999,"15min":0.49199999999999999},"min":{"1min":0.317,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69699999999999995,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.57799999999999996,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.43099999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.53600000000000003,"15min":0.54200000000000004},"min":{"1min":0.39300000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.59099999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.56200000000000006,"15min":0.56799999999999995},"min":{"1min":0.36899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.63900000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.60699999999999998}]},{"osd":6,"last update":"Sat Jan 27 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59399999999999997,"15min":0.60099999999999998},"min":{"1min":0.438,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.67700000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.40100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.68000000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.48399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574899,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":402,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194909,"statfs":{"total":200000000,"available":199587709,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.56000000000000005,"15min":0.51200000000000001},"min":{"1min":0.434,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79700000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55800000000000005,"15min":0.52000000000000002},"min":{"1min":0.35999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.59999999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.55700000000000005,"15min":0.51000000000000001},"min":{"1min":0.42599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76500000000000001,"5min":0.874,"15min":0.874},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58699999999999997,"15min":0.54900000000000004},"min":{"1min":0.44500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82199999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61499999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56599999999999995,"15min":0.53900000000000003},"min":{"1min":0.46999999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73999999999999999,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.61099999999999999,"15min":0.57799999999999996},"min":{"1min":0.52200000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80400000000000005,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.63800000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.48799999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.74299999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.56699999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.78400000000000003,"5min":1.262,"15min":1.262},"last":0.57099999999999995}]},{"osd":4,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.36599999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70299999999999996,"5min":0.748,"15min":0.748},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53600000000000003,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70299999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.54600000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52900000000000003,"15min":0.53400000000000003},"min":{"1min":0.42199999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65600000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58699999999999997,"15min":0.60899999999999999},"min":{"1min":0.41799999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76700000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.438}]},{"osd":7,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.61699999999999999},"min":{"1min":0.39300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.73499999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.5},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57799999999999996,"15min":0.59199999999999997},"min":{"1min":0.38500000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71599999999999997,"5min":1.038,"15min":1.038},"last":0.41999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":396,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194915,"statfs":{"total":200000000,"available":199593972,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.379,"15min":0.371},"min":{"1min":0.29699999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39900000000000002,"15min":0.39300000000000002},"min":{"1min":0.28599999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.53700000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.373,"15min":0.35699999999999998},"min":{"1min":0.26600000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.53800000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.42999999999999999,"15min":0.42299999999999999},"min":{"1min":0.34899999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.55700000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.47199999999999998,"15min":0.46300000000000002},"min":{"1min":0.39700000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65600000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.625},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.34999999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.60799999999999998}]},{"osd":4,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.62,"15min":0.62},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.80100000000000005,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.51300000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.77100000000000002,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.71999999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.628,"15min":0.61799999999999999},"min":{"1min":0.46400000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.88300000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.80100000000000005}]},{"osd":6,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69199999999999995,"15min":0.68999999999999995},"min":{"1min":0.501,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90000000000000002,"5min":1.024,"15min":1.024},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67100000000000004,"15min":0.66500000000000004},"min":{"1min":0.53300000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.879,"5min":1.151,"15min":1.151},"last":0.83599999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68300000000000005,"15min":0.67300000000000004},"min":{"1min":0.58499999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91800000000000004,"5min":1.097,"15min":1.097},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89700000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.88100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":979,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194333,"statfs":{"total":200000000,"available":198997341,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.38900000000000001,"15min":0.38800000000000001},"min":{"1min":0.24399999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.40899999999999997,"15min":0.41299999999999998},"min":{"1min":0.29399999999999998,"5min":0.223,"15min":0.223},"max":{"1min":0.58699999999999997,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.44}]},{"osd":2,"last update":"Sat Jan 27 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42099999999999999,"15min":0.41699999999999998},"min":{"1min":0.28699999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.30499999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.54300000000000004}]},{"osd":3,"last update":"Sat Jan 27 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.31900000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46700000000000003,"15min":0.46899999999999997},"min":{"1min":0.33600000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.628,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.48799999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95499999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.44,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.59499999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.49399999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67700000000000005,"15min":0.68200000000000005},"min":{"1min":0.47799999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.65000000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.51300000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67400000000000004,"15min":0.67300000000000004},"min":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.63400000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.69099999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.51000000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.93999999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.52000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":396,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194915,"statfs":{"total":200000000,"available":199593972,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41799999999999998,"15min":0.40999999999999998},"min":{"1min":0.26400000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56499999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.55500000000000005,"15min":0.65300000000000002},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61799999999999999,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":0.38500000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42099999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.626,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.443,"15min":0.44800000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.62,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.318}]},{"osd":3,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.42099999999999999,"15min":0.41499999999999998},"min":{"1min":0.313,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.56299999999999994,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.64900000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.42099999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.51200000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89800000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75700000000000001,"15min":0.80700000000000005},"min":{"1min":0.56399999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.91000000000000003,"5min":3.581,"15min":3.581},"last":0.61599999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69999999999999996,"15min":0.751},"min":{"1min":0.52500000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84799999999999998,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67800000000000005,"15min":0.67100000000000004},"min":{"1min":0.52700000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92000000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.63}]},{"osd":6,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.52600000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89100000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.67800000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94999999999999996,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.51100000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.79900000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69199999999999995,"15min":0.68799999999999994},"min":{"1min":0.47499999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.53900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":396,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194915,"statfs":{"total":200000000,"available":199593972,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36399999999999999,"15min":0.35099999999999998},"min":{"1min":0.27400000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.49399999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.39500000000000002,"15min":0.39100000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56000000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.31900000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39200000000000002,"15min":0.38600000000000001},"min":{"1min":0.30099999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.51500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.316,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.57899999999999996,"5min":0.628,"15min":0.628},"last":0.48799999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.42999999999999999,"15min":0.42099999999999999},"min":{"1min":0.33600000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45100000000000001,"15min":0.439},"min":{"1min":0.37,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.51400000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.498,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80900000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60799999999999998,"15min":0.59499999999999997},"min":{"1min":0.49099999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.75}]},{"osd":5,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61699999999999999,"15min":0.60299999999999998},"min":{"1min":0.44800000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.63300000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65100000000000002,"15min":0.65000000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.89000000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64600000000000002,"15min":0.64200000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.76300000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.57399999999999995}]},{"osd":7,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64100000000000001,"15min":0.629},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.54600000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.64700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574899,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":396,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194915,"statfs":{"total":200000000,"available":199593972,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.38100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.70899999999999996,"5min":1.099,"15min":1.099},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.45800000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77500000000000002,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.72299999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60299999999999998,"15min":0.59399999999999997},"min":{"1min":0.439,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86899999999999999,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.44,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79800000000000004,"5min":1.133,"15min":1.133},"last":0.81999999999999995}]},{"osd":2,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.44600000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.81299999999999994,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.3400000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.61499999999999999,"15min":0.61399999999999999},"min":{"1min":0.45600000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82799999999999996,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.69899999999999995}]},{"osd":3,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.57699999999999996},"min":{"1min":0.435,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83199999999999996,"5min":1.115,"15min":1.115},"last":1.4059999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65400000000000003,"15min":0.64800000000000002},"min":{"1min":0.47799999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93799999999999994,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.747}]},{"osd":5,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.33300000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68600000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":1.1799999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.312,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66500000000000004,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.77900000000000003}]},{"osd":6,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.503,"15min":0.5},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72299999999999998,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77700000000000002,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.95099999999999996}]},{"osd":7,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55300000000000005,"15min":0.55400000000000005},"min":{"1min":0.41599999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58299999999999996,"15min":0.57399999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77500000000000002,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.79000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":402,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194909,"statfs":{"total":200000000,"available":199587709,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.82599999999999996,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.60499999999999998,"15min":0.57799999999999996},"min":{"1min":0.39800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.75600000000000001,"5min":3.242,"15min":3.242},"last":0.55300000000000005}]},{"osd":1,"last update":"Sat Jan 27 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.622,"15min":0.60699999999999998},"min":{"1min":0.45700000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.78900000000000003,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.65200000000000002,"15min":0.65800000000000003},"min":{"1min":0.437,"5min":0.436,"15min":0.436},"max":{"1min":0.84399999999999997,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.621}]},{"osd":2,"last update":"Sat Jan 27 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.39800000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.83099999999999996,"5min":3.04,"15min":3.04},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85899999999999999,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66200000000000003}]},{"osd":3,"last update":"Sat Jan 27 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.69999999999999996,"15min":0.68999999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88900000000000001,"5min":4.032,"15min":4.032},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.71099999999999997,"15min":0.68100000000000005},"min":{"1min":0.47699999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.86199999999999999,"5min":4.282,"15min":4.282},"last":0.69799999999999995}]},{"osd":4,"last update":"Sat Jan 27 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.30599999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.79000000000000004,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.55300000000000005,"15min":0.57899999999999996},"min":{"1min":0.318,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79800000000000004,"5min":1.101,"15min":1.101},"last":0.51500000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.55000000000000004,"15min":0.56799999999999995},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.73199999999999998,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85799999999999998,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.47199999999999998}]},{"osd":7,"last update":"Sat Jan 27 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.34799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77100000000000002,"5min":1.008,"15min":1.008},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.35899999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82799999999999996,"5min":2,"15min":2},"last":0.59199999999999997}]}]}],"pool_statfs":[]}} 2024-01-27T02:36:06.292 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:36:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.460+0000 7fb112e36700 1 Processor -- start 2024-01-27T02:36:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.460+0000 7fb112e36700 1 -- start start 2024-01-27T02:36:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.461+0000 7fb112e36700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb10c0b2b80 0x7fb10c157810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.461+0000 7fb112e36700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb10c0cff50 0x7fb10c157d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.461+0000 7fb112e36700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10c15bb40 con 0x7fb10c0b2f50 2024-01-27T02:36:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.461+0000 7fb112e36700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb10c0d0da0 con 0x7fb10c0b2b80 2024-01-27T02:36:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.461+0000 7fb112e36700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb10c158290 con 0x7fb10c0cff50 2024-01-27T02:36:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.461+0000 7fb110bd2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb10c0b2b80 0x7fb10c157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.461+0000 7fb10bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb10c0b2f50 0x7fb10c0cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40642/0 (socket says 172.21.15.7:40642) 2024-01-27T02:36:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.461+0000 7fb10bfff700 1 -- 172.21.15.7:0/2066409513 learned_addr learned my addr 172.21.15.7:0/2066409513 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.461+0000 7fb1113d3700 1 --2- 172.21.15.7:0/2066409513 >> v2:172.21.15.188:3300/0 conn(0x7fb10c0cff50 0x7fb10c157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.461+0000 7fb110bd2700 1 -- 172.21.15.7:0/2066409513 >> v2:172.21.15.188:3300/0 conn(0x7fb10c0cff50 msgr2=0x7fb10c157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.461+0000 7fb110bd2700 1 --2- 172.21.15.7:0/2066409513 >> v2:172.21.15.188:3300/0 conn(0x7fb10c0cff50 0x7fb10c157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.461+0000 7fb110bd2700 1 -- 172.21.15.7:0/2066409513 >> v1:172.21.15.7:6789/0 conn(0x7fb10c0b2f50 legacy=0x7fb10c0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.462+0000 7fb110bd2700 1 -- 172.21.15.7:0/2066409513 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c15ab30 con 0x7fb10c0b2b80 2024-01-27T02:36:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.461+0000 7fb10b7fe700 1 -- 172.21.15.7:0/2066409513 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1798844466 0 0) 0x7fb10c15bb40 con 0x7fb10c0b2f50 2024-01-27T02:36:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.462+0000 7fb1113d3700 1 --2- 172.21.15.7:0/2066409513 >> v2:172.21.15.188:3300/0 conn(0x7fb10c0cff50 0x7fb10c157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:36:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.462+0000 7fb110bd2700 1 --2- 172.21.15.7:0/2066409513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb10c0b2b80 0x7fb10c157810 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb10c15bab0 tx=0x7fb1000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc8cdaad14c5376 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.462+0000 7fb10b7fe700 1 -- 172.21.15.7:0/2066409513 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10000b6b0 con 0x7fb10c0b2b80 2024-01-27T02:36:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.462+0000 7fb10b7fe700 1 -- 172.21.15.7:0/2066409513 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10000b810 con 0x7fb10c0b2b80 2024-01-27T02:36:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.462+0000 7fb10b7fe700 1 -- 172.21.15.7:0/2066409513 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10000bbb0 con 0x7fb10c0b2b80 2024-01-27T02:36:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.462+0000 7fb112e36700 1 -- 172.21.15.7:0/2066409513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb10c0b2b80 msgr2=0x7fb10c157810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.462+0000 7fb112e36700 1 --2- 172.21.15.7:0/2066409513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb10c0b2b80 0x7fb10c157810 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb10c15bab0 tx=0x7fb1000091a0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.462+0000 7fb112e36700 1 -- 172.21.15.7:0/2066409513 shutdown_connections 2024-01-27T02:36:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.462+0000 7fb112e36700 1 --2- 172.21.15.7:0/2066409513 >> v2:172.21.15.188:3300/0 conn(0x7fb10c0cff50 0x7fb10c157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.462+0000 7fb112e36700 1 --2- 172.21.15.7:0/2066409513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb10c0b2b80 0x7fb10c157810 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.462+0000 7fb112e36700 1 -- 172.21.15.7:0/2066409513 >> 172.21.15.7:0/2066409513 conn(0x7fb10c068450 msgr2=0x7fb10c0709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.463+0000 7fb112e36700 1 -- 172.21.15.7:0/2066409513 shutdown_connections 2024-01-27T02:36:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.463+0000 7fb112e36700 1 -- 172.21.15.7:0/2066409513 wait complete. 2024-01-27T02:36:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.464+0000 7fb112e36700 1 Processor -- start 2024-01-27T02:36:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.464+0000 7fb112e36700 1 -- start start 2024-01-27T02:36:06.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.465+0000 7fb112e36700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb10c0b2b80 0x7fb10c0c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:06.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.465+0000 7fb112e36700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb10c0b2f50 0x7fb10c156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:06.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.465+0000 7fb112e36700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb10c0cff50 0x7fb10c0c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:06.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.465+0000 7fb112e36700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c0c92b0 con 0x7fb10c0b2f50 2024-01-27T02:36:06.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.465+0000 7fb112e36700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c0c8a10 con 0x7fb10c0b2b80 2024-01-27T02:36:06.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.465+0000 7fb112e36700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c0c8cf0 con 0x7fb10c0cff50 2024-01-27T02:36:06.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.465+0000 7fb10bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb10c0b2f50 0x7fb10c156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:06.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.465+0000 7fb10bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb10c0b2f50 0x7fb10c156830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33486/0 (socket says 172.21.15.7:33486) 2024-01-27T02:36:06.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.465+0000 7fb10bfff700 1 -- 172.21.15.7:0/1267940351 learned_addr learned my addr 172.21.15.7:0/1267940351 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:06.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.465+0000 7fb110bd2700 1 --2- 172.21.15.7:0/1267940351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb10c0b2b80 0x7fb10c0c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:06.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.465+0000 7fb1113d3700 1 --2- 172.21.15.7:0/1267940351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb10c0cff50 0x7fb10c0c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:06.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.465+0000 7fb10bfff700 1 -- 172.21.15.7:0/1267940351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb10c0cff50 msgr2=0x7fb10c0c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:06.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.465+0000 7fb10bfff700 1 --2- 172.21.15.7:0/1267940351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb10c0cff50 0x7fb10c0c81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.465+0000 7fb10bfff700 1 -- 172.21.15.7:0/1267940351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb10c0b2b80 msgr2=0x7fb10c0c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:06.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.465+0000 7fb10bfff700 1 --2- 172.21.15.7:0/1267940351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb10c0b2b80 0x7fb10c0c6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.465+0000 7fb10bfff700 1 -- 172.21.15.7:0/1267940351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb100009420 con 0x7fb10c0b2f50 2024-01-27T02:36:06.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.465+0000 7fb1113d3700 1 --2- 172.21.15.7:0/1267940351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb10c0cff50 0x7fb10c0c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:36:06.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.466+0000 7fb10bfff700 1 --2- 172.21.15.7:0/1267940351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb10c0b2f50 0x7fb10c156830 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800c7e0 tx=0x7fb0f8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:06.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.466+0000 7fb108ff9700 1 -- 172.21.15.7:0/1267940351 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8005b30 con 0x7fb10c0b2f50 2024-01-27T02:36:06.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.466+0000 7fb108ff9700 1 -- 172.21.15.7:0/1267940351 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f8005c90 con 0x7fb10c0b2f50 2024-01-27T02:36:06.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.466+0000 7fb112e36700 1 -- 172.21.15.7:0/1267940351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10c0c8fd0 con 0x7fb10c0b2f50 2024-01-27T02:36:06.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.466+0000 7fb108ff9700 1 -- 172.21.15.7:0/1267940351 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8007300 con 0x7fb10c0b2f50 2024-01-27T02:36:06.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.466+0000 7fb112e36700 1 -- 172.21.15.7:0/1267940351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10c156d70 con 0x7fb10c0b2f50 2024-01-27T02:36:06.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.467+0000 7fb112e36700 1 -- 172.21.15.7:0/1267940351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10c0cc1d0 con 0x7fb10c0b2f50 2024-01-27T02:36:06.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.467+0000 7fb108ff9700 1 -- 172.21.15.7:0/1267940351 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb0f80076b0 con 0x7fb10c0b2f50 2024-01-27T02:36:06.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.468+0000 7fb108ff9700 1 --2- 172.21.15.7:0/1267940351 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb0f0066f30 0x7fb0f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:06.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.468+0000 7fb108ff9700 1 -- 172.21.15.7:0/1267940351 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb0f8083aa0 con 0x7fb10c0b2f50 2024-01-27T02:36:06.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.468+0000 7fb110bd2700 1 --2- 172.21.15.7:0/1267940351 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb0f0066f30 0x7fb0f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:06.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.469+0000 7fb110bd2700 1 --2- 172.21.15.7:0/1267940351 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb0f0066f30 0x7fb0f00693e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb1000093f0 tx=0x7fb100017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:06.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.472+0000 7fb108ff9700 1 -- 172.21.15.7:0/1267940351 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb10c0cc1d0 con 0x7fb10c0b2f50 2024-01-27T02:36:06.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.669+0000 7fb112e36700 1 -- 172.21.15.7:0/1267940351 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb10c0cc5b0 con 0x7fb0f0066f30 2024-01-27T02:36:06.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.677+0000 7fb108ff9700 1 -- 172.21.15.7:0/1267940351 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282722 (secure 0 0 0) 0x7fb10c0cc5b0 con 0x7fb0f0066f30 2024-01-27T02:36:06.681 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:36:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.680+0000 7fb112e36700 1 -- 172.21.15.7:0/1267940351 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb0f0066f30 msgr2=0x7fb0f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.680+0000 7fb112e36700 1 --2- 172.21.15.7:0/1267940351 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb0f0066f30 0x7fb0f00693e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb1000093f0 tx=0x7fb100017000 comp rx=0 tx=0).stop 2024-01-27T02:36:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.680+0000 7fb112e36700 1 -- 172.21.15.7:0/1267940351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb10c0b2f50 msgr2=0x7fb10c156830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.680+0000 7fb112e36700 1 --2- 172.21.15.7:0/1267940351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb10c0b2f50 0x7fb10c156830 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800c7e0 tx=0x7fb0f8007df0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.680+0000 7fb112e36700 1 -- 172.21.15.7:0/1267940351 shutdown_connections 2024-01-27T02:36:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.680+0000 7fb112e36700 1 --2- 172.21.15.7:0/1267940351 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb0f0066f30 0x7fb0f00693e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.680+0000 7fb112e36700 1 --2- 172.21.15.7:0/1267940351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb10c0cff50 0x7fb10c0c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.680+0000 7fb112e36700 1 --2- 172.21.15.7:0/1267940351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb10c0b2b80 0x7fb10c0c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.680+0000 7fb112e36700 1 --2- 172.21.15.7:0/1267940351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb10c0b2f50 0x7fb10c156830 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.681+0000 7fb112e36700 1 -- 172.21.15.7:0/1267940351 >> 172.21.15.7:0/1267940351 conn(0x7fb10c068450 msgr2=0x7fb10c0b65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.681+0000 7fb112e36700 1 -- 172.21.15.7:0/1267940351 shutdown_connections 2024-01-27T02:36:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.681+0000 7fb112e36700 1 -- 172.21.15.7:0/1267940351 wait complete. 2024-01-27T02:36:06.685 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:36:06.695 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2024-01-27T02:36:05.631498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4343,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558156,"statfs":{"total":1600000000,"available":1595552251,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.567011"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077060+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:19:59.227343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078576+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:09.241354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.071538+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:51:24.783765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078657+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:45:48.321027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079475+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:17:28.880573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.080893+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:25:43.791658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078520+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:57:20.844278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077646+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:16:42.015521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078957+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:25.098588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.081944+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:21:41.100136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076679+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:57:20.844278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078947+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:39:19.173475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.080386+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:32:03.288747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077576+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:41:44.104324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079892+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:26:17.309915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079382+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:28:19.604756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.080873+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:47:56.363317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.080422+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:10:26.712434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.075169+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:15:05.885263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079455+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:19:38.789133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078990+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:18:04.141154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079678+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:25:43.791658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.075028+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:26:17.309915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076931+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:12:02.170281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.081562+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:10:52.422378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076922+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:21.383627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.071521+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:51:11.452504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078331+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:09.241354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.071456+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:37:39.458923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076978+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:48:39.299949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076528+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:41:50.546172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078193+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:27:01.184977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076537+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:48:39.299949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.077741+0000","last_change":"2024-01-27T02:36:01.077741+0000","last_active":"2024-01-27T02:36:01.077741+0000","last_peered":"2024-01-27T02:36:01.077741+0000","last_clean":"2024-01-27T02:36:01.077741+0000","last_became_active":"2024-01-27T02:36:01.077423+0000","last_became_peered":"2024-01-27T02:36:01.077423+0000","last_unstale":"2024-01-27T02:36:01.077741+0000","last_undegraded":"2024-01-27T02:36:01.077741+0000","last_fullsized":"2024-01-27T02:36:01.077741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:15.272806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077778+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:09:23.740404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.084751+0000","last_change":"2024-01-27T02:36:01.084751+0000","last_active":"2024-01-27T02:36:01.084751+0000","last_peered":"2024-01-27T02:36:01.084751+0000","last_clean":"2024-01-27T02:36:01.084751+0000","last_became_active":"2024-01-27T02:36:01.084351+0000","last_became_peered":"2024-01-27T02:36:01.084351+0000","last_unstale":"2024-01-27T02:36:01.084751+0000","last_undegraded":"2024-01-27T02:36:01.084751+0000","last_fullsized":"2024-01-27T02:36:01.084751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:42:38.497144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.080521+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:31:44.339745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.087593+0000","last_change":"2024-01-27T02:36:01.087593+0000","last_active":"2024-01-27T02:36:01.087593+0000","last_peered":"2024-01-27T02:36:01.087593+0000","last_clean":"2024-01-27T02:36:01.087593+0000","last_became_active":"2024-01-27T02:36:01.086695+0000","last_became_peered":"2024-01-27T02:36:01.086695+0000","last_unstale":"2024-01-27T02:36:01.087593+0000","last_undegraded":"2024-01-27T02:36:01.087593+0000","last_fullsized":"2024-01-27T02:36:01.087593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:15:04.879041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077553+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:56:33.979247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.082918+0000","last_change":"2024-01-27T02:36:01.082918+0000","last_active":"2024-01-27T02:36:01.082918+0000","last_peered":"2024-01-27T02:36:01.082918+0000","last_clean":"2024-01-27T02:36:01.082918+0000","last_became_active":"2024-01-27T02:36:01.081169+0000","last_became_peered":"2024-01-27T02:36:01.081169+0000","last_unstale":"2024-01-27T02:36:01.082918+0000","last_undegraded":"2024-01-27T02:36:01.082918+0000","last_fullsized":"2024-01-27T02:36:01.082918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:39:18.167253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.075988+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:32:03.288747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.083508+0000","last_change":"2024-01-27T02:36:01.083508+0000","last_active":"2024-01-27T02:36:01.083508+0000","last_peered":"2024-01-27T02:36:01.083508+0000","last_clean":"2024-01-27T02:36:01.083508+0000","last_became_active":"2024-01-27T02:36:01.083244+0000","last_became_peered":"2024-01-27T02:36:01.083244+0000","last_unstale":"2024-01-27T02:36:01.083508+0000","last_undegraded":"2024-01-27T02:36:01.083508+0000","last_fullsized":"2024-01-27T02:36:01.083508+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:42:06.107055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079272+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:31:44.339745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.081011+0000","last_change":"2024-01-27T02:36:01.081011+0000","last_active":"2024-01-27T02:36:01.081011+0000","last_peered":"2024-01-27T02:36:01.081011+0000","last_clean":"2024-01-27T02:36:01.081011+0000","last_became_active":"2024-01-27T02:36:01.080166+0000","last_became_peered":"2024-01-27T02:36:01.080166+0000","last_unstale":"2024-01-27T02:36:01.081011+0000","last_undegraded":"2024-01-27T02:36:01.081011+0000","last_fullsized":"2024-01-27T02:36:01.081011+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:42:38.578182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079075+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:03:33.449715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.086427+0000","last_change":"2024-01-27T02:36:01.086427+0000","last_active":"2024-01-27T02:36:01.086427+0000","last_peered":"2024-01-27T02:36:01.086427+0000","last_clean":"2024-01-27T02:36:01.086427+0000","last_became_active":"2024-01-27T02:36:01.085912+0000","last_became_peered":"2024-01-27T02:36:01.085912+0000","last_unstale":"2024-01-27T02:36:01.086427+0000","last_undegraded":"2024-01-27T02:36:01.086427+0000","last_fullsized":"2024-01-27T02:36:01.086427+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:12:59.343148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079659+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:21:28.748243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.084880+0000","last_change":"2024-01-27T02:36:01.084880+0000","last_active":"2024-01-27T02:36:01.084880+0000","last_peered":"2024-01-27T02:36:01.084880+0000","last_clean":"2024-01-27T02:36:01.084880+0000","last_became_active":"2024-01-27T02:36:01.084532+0000","last_became_peered":"2024-01-27T02:36:01.084532+0000","last_unstale":"2024-01-27T02:36:01.084880+0000","last_undegraded":"2024-01-27T02:36:01.084880+0000","last_fullsized":"2024-01-27T02:36:01.084880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:58:04.302291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076109+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:32:03.273755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.083170+0000","last_change":"2024-01-27T02:36:01.083170+0000","last_active":"2024-01-27T02:36:01.083170+0000","last_peered":"2024-01-27T02:36:01.083170+0000","last_clean":"2024-01-27T02:36:01.083170+0000","last_became_active":"2024-01-27T02:36:01.082730+0000","last_became_peered":"2024-01-27T02:36:01.082730+0000","last_unstale":"2024-01-27T02:36:01.083170+0000","last_undegraded":"2024-01-27T02:36:01.083170+0000","last_fullsized":"2024-01-27T02:36:01.083170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:07:19.984794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076273+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:21.383627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.079850+0000","last_change":"2024-01-27T02:36:01.079850+0000","last_active":"2024-01-27T02:36:01.079850+0000","last_peered":"2024-01-27T02:36:01.079850+0000","last_clean":"2024-01-27T02:36:01.079850+0000","last_became_active":"2024-01-27T02:36:01.079488+0000","last_became_peered":"2024-01-27T02:36:01.079488+0000","last_unstale":"2024-01-27T02:36:01.079850+0000","last_undegraded":"2024-01-27T02:36:01.079850+0000","last_fullsized":"2024-01-27T02:36:01.079850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:36:51.217544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.074770+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:35:14.198571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.075921+0000","last_change":"2024-01-27T02:36:01.075921+0000","last_active":"2024-01-27T02:36:01.075921+0000","last_peered":"2024-01-27T02:36:01.075921+0000","last_clean":"2024-01-27T02:36:01.075921+0000","last_became_active":"2024-01-27T02:36:01.075647+0000","last_became_peered":"2024-01-27T02:36:01.075647+0000","last_unstale":"2024-01-27T02:36:01.075921+0000","last_undegraded":"2024-01-27T02:36:01.075921+0000","last_fullsized":"2024-01-27T02:36:01.075921+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:54:58.712360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077216+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:22:30.371211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.085134+0000","last_change":"2024-01-27T02:36:01.085134+0000","last_active":"2024-01-27T02:36:01.085134+0000","last_peered":"2024-01-27T02:36:01.085134+0000","last_clean":"2024-01-27T02:36:01.085134+0000","last_became_active":"2024-01-27T02:36:01.084887+0000","last_became_peered":"2024-01-27T02:36:01.084887+0000","last_unstale":"2024-01-27T02:36:01.085134+0000","last_undegraded":"2024-01-27T02:36:01.085134+0000","last_fullsized":"2024-01-27T02:36:01.085134+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:24:30.352994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.075834+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:14:13.651138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.081854+0000","last_change":"2024-01-27T02:36:01.081854+0000","last_active":"2024-01-27T02:36:01.081854+0000","last_peered":"2024-01-27T02:36:01.081854+0000","last_clean":"2024-01-27T02:36:01.081854+0000","last_became_active":"2024-01-27T02:36:01.081553+0000","last_became_peered":"2024-01-27T02:36:01.081553+0000","last_unstale":"2024-01-27T02:36:01.081854+0000","last_undegraded":"2024-01-27T02:36:01.081854+0000","last_fullsized":"2024-01-27T02:36:01.081854+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:36:51.217544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076474+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:14:13.651138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.087113+0000","last_change":"2024-01-27T02:36:01.087113+0000","last_active":"2024-01-27T02:36:01.087113+0000","last_peered":"2024-01-27T02:36:01.087113+0000","last_clean":"2024-01-27T02:36:01.087113+0000","last_became_active":"2024-01-27T02:36:01.086735+0000","last_became_peered":"2024-01-27T02:36:01.086735+0000","last_unstale":"2024-01-27T02:36:01.087113+0000","last_undegraded":"2024-01-27T02:36:01.087113+0000","last_fullsized":"2024-01-27T02:36:01.087113+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:51:30.355507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079910+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:04:07.439211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.079455+0000","last_change":"2024-01-27T02:36:01.079455+0000","last_active":"2024-01-27T02:36:01.079455+0000","last_peered":"2024-01-27T02:36:01.079455+0000","last_clean":"2024-01-27T02:36:01.079455+0000","last_became_active":"2024-01-27T02:36:01.078810+0000","last_became_peered":"2024-01-27T02:36:01.078810+0000","last_unstale":"2024-01-27T02:36:01.079455+0000","last_undegraded":"2024-01-27T02:36:01.079455+0000","last_fullsized":"2024-01-27T02:36:01.079455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:32:57.446408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078346+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:32:55.342184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.083387+0000","last_change":"2024-01-27T02:36:01.083387+0000","last_active":"2024-01-27T02:36:01.083387+0000","last_peered":"2024-01-27T02:36:01.083387+0000","last_clean":"2024-01-27T02:36:01.083387+0000","last_became_active":"2024-01-27T02:36:01.083023+0000","last_became_peered":"2024-01-27T02:36:01.083023+0000","last_unstale":"2024-01-27T02:36:01.083387+0000","last_undegraded":"2024-01-27T02:36:01.083387+0000","last_fullsized":"2024-01-27T02:36:01.083387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:16:41.009300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079871+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:42.090153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.075386+0000","last_change":"2024-01-27T02:36:01.075386+0000","last_active":"2024-01-27T02:36:01.075386+0000","last_peered":"2024-01-27T02:36:01.075386+0000","last_clean":"2024-01-27T02:36:01.075386+0000","last_became_active":"2024-01-27T02:36:01.074276+0000","last_became_peered":"2024-01-27T02:36:01.074276+0000","last_unstale":"2024-01-27T02:36:01.075386+0000","last_undegraded":"2024-01-27T02:36:01.075386+0000","last_fullsized":"2024-01-27T02:36:01.075386+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:46:34.597904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076123+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:41:50.546172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.087262+0000","last_change":"2024-01-27T02:36:01.087262+0000","last_active":"2024-01-27T02:36:01.087262+0000","last_peered":"2024-01-27T02:36:01.087262+0000","last_clean":"2024-01-27T02:36:01.087262+0000","last_became_active":"2024-01-27T02:36:01.086163+0000","last_became_peered":"2024-01-27T02:36:01.086163+0000","last_unstale":"2024-01-27T02:36:01.087262+0000","last_undegraded":"2024-01-27T02:36:01.087262+0000","last_fullsized":"2024-01-27T02:36:01.087262+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:35:13.192349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078527+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:23:57.404805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.086968+0000","last_change":"2024-01-27T02:36:01.086968+0000","last_active":"2024-01-27T02:36:01.086968+0000","last_peered":"2024-01-27T02:36:01.086968+0000","last_clean":"2024-01-27T02:36:01.086968+0000","last_became_active":"2024-01-27T02:36:01.086112+0000","last_became_peered":"2024-01-27T02:36:01.086112+0000","last_unstale":"2024-01-27T02:36:01.086968+0000","last_undegraded":"2024-01-27T02:36:01.086968+0000","last_fullsized":"2024-01-27T02:36:01.086968+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:03:55.891925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078774+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:19:43.900445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.085172+0000","last_change":"2024-01-27T02:36:01.085172+0000","last_active":"2024-01-27T02:36:01.085172+0000","last_peered":"2024-01-27T02:36:01.085172+0000","last_clean":"2024-01-27T02:36:01.085172+0000","last_became_active":"2024-01-27T02:36:01.084894+0000","last_became_peered":"2024-01-27T02:36:01.084894+0000","last_unstale":"2024-01-27T02:36:01.085172+0000","last_undegraded":"2024-01-27T02:36:01.085172+0000","last_fullsized":"2024-01-27T02:36:01.085172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:19:37.782912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.070899+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:05:31.415970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.080657+0000","last_change":"2024-01-27T02:36:01.080657+0000","last_active":"2024-01-27T02:36:01.080657+0000","last_peered":"2024-01-27T02:36:01.080657+0000","last_clean":"2024-01-27T02:36:01.080657+0000","last_became_active":"2024-01-27T02:36:01.078442+0000","last_became_peered":"2024-01-27T02:36:01.078442+0000","last_unstale":"2024-01-27T02:36:01.080657+0000","last_undegraded":"2024-01-27T02:36:01.080657+0000","last_fullsized":"2024-01-27T02:36:01.080657+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:45:47.314805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.070938+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:22:09.618511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.085914+0000","last_change":"2024-01-27T02:36:01.085914+0000","last_active":"2024-01-27T02:36:01.085914+0000","last_peered":"2024-01-27T02:36:01.085914+0000","last_clean":"2024-01-27T02:36:01.085914+0000","last_became_active":"2024-01-27T02:36:01.085674+0000","last_became_peered":"2024-01-27T02:36:01.085674+0000","last_unstale":"2024-01-27T02:36:01.085914+0000","last_undegraded":"2024-01-27T02:36:01.085914+0000","last_fullsized":"2024-01-27T02:36:01.085914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:19:37.782912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.070886+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:47:18.259755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.086128+0000","last_change":"2024-01-27T02:36:01.086128+0000","last_active":"2024-01-27T02:36:01.086128+0000","last_peered":"2024-01-27T02:36:01.086128+0000","last_clean":"2024-01-27T02:36:01.086128+0000","last_became_active":"2024-01-27T02:36:01.085891+0000","last_became_peered":"2024-01-27T02:36:01.085891+0000","last_unstale":"2024-01-27T02:36:01.086128+0000","last_undegraded":"2024-01-27T02:36:01.086128+0000","last_fullsized":"2024-01-27T02:36:01.086128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:05:15.222635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079413+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:16.279028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.079712+0000","last_change":"2024-01-27T02:36:01.079712+0000","last_active":"2024-01-27T02:36:01.079712+0000","last_peered":"2024-01-27T02:36:01.079712+0000","last_clean":"2024-01-27T02:36:01.079712+0000","last_became_active":"2024-01-27T02:36:01.079176+0000","last_became_peered":"2024-01-27T02:36:01.079176+0000","last_unstale":"2024-01-27T02:36:01.079712+0000","last_undegraded":"2024-01-27T02:36:01.079712+0000","last_fullsized":"2024-01-27T02:36:01.079712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:12:59.343148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.075688+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:26:17.309915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.084252+0000","last_change":"2024-01-27T02:36:01.084252+0000","last_active":"2024-01-27T02:36:01.084252+0000","last_peered":"2024-01-27T02:36:01.084252+0000","last_clean":"2024-01-27T02:36:01.084252+0000","last_became_active":"2024-01-27T02:36:01.083785+0000","last_became_peered":"2024-01-27T02:36:01.083785+0000","last_unstale":"2024-01-27T02:36:01.084252+0000","last_undegraded":"2024-01-27T02:36:01.084252+0000","last_fullsized":"2024-01-27T02:36:01.084252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:15.272806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.075405+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:32:03.288747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.083357+0000","last_change":"2024-01-27T02:36:01.083357+0000","last_active":"2024-01-27T02:36:01.083357+0000","last_peered":"2024-01-27T02:36:01.083357+0000","last_clean":"2024-01-27T02:36:01.083357+0000","last_became_active":"2024-01-27T02:36:01.083009+0000","last_became_peered":"2024-01-27T02:36:01.083009+0000","last_unstale":"2024-01-27T02:36:01.083357+0000","last_undegraded":"2024-01-27T02:36:01.083357+0000","last_fullsized":"2024-01-27T02:36:01.083357+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:17:17.877802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.075713+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:32:03.273755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.083453+0000","last_change":"2024-01-27T02:36:01.083453+0000","last_active":"2024-01-27T02:36:01.083453+0000","last_peered":"2024-01-27T02:36:01.083453+0000","last_clean":"2024-01-27T02:36:01.083453+0000","last_became_active":"2024-01-27T02:36:01.083173+0000","last_became_peered":"2024-01-27T02:36:01.083173+0000","last_unstale":"2024-01-27T02:36:01.083453+0000","last_undegraded":"2024-01-27T02:36:01.083453+0000","last_fullsized":"2024-01-27T02:36:01.083453+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:21:40.093915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077155+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:11:23.338931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.086016+0000","last_change":"2024-01-27T02:36:01.086016+0000","last_active":"2024-01-27T02:36:01.086016+0000","last_peered":"2024-01-27T02:36:01.086016+0000","last_clean":"2024-01-27T02:36:01.086016+0000","last_became_active":"2024-01-27T02:36:01.085747+0000","last_became_peered":"2024-01-27T02:36:01.085747+0000","last_unstale":"2024-01-27T02:36:01.086016+0000","last_undegraded":"2024-01-27T02:36:01.086016+0000","last_fullsized":"2024-01-27T02:36:01.086016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:03:04.731782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.080121+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:18:04.141154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.085778+0000","last_change":"2024-01-27T02:36:01.085778+0000","last_active":"2024-01-27T02:36:01.085778+0000","last_peered":"2024-01-27T02:36:01.085778+0000","last_clean":"2024-01-27T02:36:01.085778+0000","last_became_active":"2024-01-27T02:36:01.085504+0000","last_became_peered":"2024-01-27T02:36:01.085504+0000","last_unstale":"2024-01-27T02:36:01.085778+0000","last_undegraded":"2024-01-27T02:36:01.085778+0000","last_fullsized":"2024-01-27T02:36:01.085778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:34:00.107552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078206+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:03:56.898146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.083238+0000","last_change":"2024-01-27T02:36:01.083238+0000","last_active":"2024-01-27T02:36:01.083238+0000","last_peered":"2024-01-27T02:36:01.083238+0000","last_clean":"2024-01-27T02:36:01.083238+0000","last_became_active":"2024-01-27T02:36:01.082768+0000","last_became_peered":"2024-01-27T02:36:01.082768+0000","last_unstale":"2024-01-27T02:36:01.083238+0000","last_undegraded":"2024-01-27T02:36:01.083238+0000","last_fullsized":"2024-01-27T02:36:01.083238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:34:52.370887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077264+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:34:53.377109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.078683+0000","last_change":"2024-01-27T02:36:01.078683+0000","last_active":"2024-01-27T02:36:01.078683+0000","last_peered":"2024-01-27T02:36:01.078683+0000","last_clean":"2024-01-27T02:36:01.078683+0000","last_became_active":"2024-01-27T02:36:01.078126+0000","last_became_peered":"2024-01-27T02:36:01.078126+0000","last_unstale":"2024-01-27T02:36:01.078683+0000","last_undegraded":"2024-01-27T02:36:01.078683+0000","last_fullsized":"2024-01-27T02:36:01.078683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:10:25.706212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.070886+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:09:00.736774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.080741+0000","last_change":"2024-01-27T02:36:01.080741+0000","last_active":"2024-01-27T02:36:01.080741+0000","last_peered":"2024-01-27T02:36:01.080741+0000","last_clean":"2024-01-27T02:36:01.080741+0000","last_became_active":"2024-01-27T02:36:01.079822+0000","last_became_peered":"2024-01-27T02:36:01.079822+0000","last_unstale":"2024-01-27T02:36:01.080741+0000","last_undegraded":"2024-01-27T02:36:01.080741+0000","last_fullsized":"2024-01-27T02:36:01.080741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:23:56.398583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":211,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-01-27T02:36:00.065172+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:36:00.065172+0000","last_peered":"2024-01-27T02:36:00.065172+0000","last_clean":"2024-01-27T02:36:00.065172+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:36:00.065172+0000","last_undegraded":"2024-01-27T02:36:00.065172+0000","last_fullsized":"2024-01-27T02:36:00.065172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":973,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194339,"statfs":{"total":200000000,"available":199003604,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.50900000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.41599999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.46899999999999997}]},{"osd":2,"last update":"Sat Jan 27 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.499,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.59299999999999997,"15min":0.58399999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.41499999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59799999999999998,"15min":0.59099999999999997},"min":{"1min":0.48199999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.59799999999999998,"15min":0.59099999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.45500000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48999999999999999,"15min":0.49199999999999999},"min":{"1min":0.317,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69699999999999995,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.57799999999999996,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.43099999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.53600000000000003,"15min":0.54200000000000004},"min":{"1min":0.39300000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.59099999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.56200000000000006,"15min":0.56799999999999995},"min":{"1min":0.36899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.63900000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.60699999999999998}]},{"osd":6,"last update":"Sat Jan 27 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59399999999999997,"15min":0.60099999999999998},"min":{"1min":0.438,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.67700000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.40100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.68000000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.48399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574899,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":402,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194909,"statfs":{"total":200000000,"available":199587709,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.56000000000000005,"15min":0.51200000000000001},"min":{"1min":0.434,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79700000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55800000000000005,"15min":0.52000000000000002},"min":{"1min":0.35999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.59999999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.55700000000000005,"15min":0.51000000000000001},"min":{"1min":0.42599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76500000000000001,"5min":0.874,"15min":0.874},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58699999999999997,"15min":0.54900000000000004},"min":{"1min":0.44500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82199999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61499999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56599999999999995,"15min":0.53900000000000003},"min":{"1min":0.46999999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73999999999999999,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.61099999999999999,"15min":0.57799999999999996},"min":{"1min":0.52200000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80400000000000005,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.63800000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.48799999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.74299999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.56699999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.78400000000000003,"5min":1.262,"15min":1.262},"last":0.57099999999999995}]},{"osd":4,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.36599999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70299999999999996,"5min":0.748,"15min":0.748},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53600000000000003,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70299999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.54600000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52900000000000003,"15min":0.53400000000000003},"min":{"1min":0.42199999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65600000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58699999999999997,"15min":0.60899999999999999},"min":{"1min":0.41799999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76700000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.438}]},{"osd":7,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.61699999999999999},"min":{"1min":0.39300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.73499999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.5},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57799999999999996,"15min":0.59199999999999997},"min":{"1min":0.38500000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71599999999999997,"5min":1.038,"15min":1.038},"last":0.41999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":396,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194915,"statfs":{"total":200000000,"available":199593972,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.379,"15min":0.371},"min":{"1min":0.29699999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39900000000000002,"15min":0.39300000000000002},"min":{"1min":0.28599999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.53700000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.373,"15min":0.35699999999999998},"min":{"1min":0.26600000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.53800000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.42999999999999999,"15min":0.42299999999999999},"min":{"1min":0.34899999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.55700000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.47199999999999998,"15min":0.46300000000000002},"min":{"1min":0.39700000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65600000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.625},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.34999999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.60799999999999998}]},{"osd":4,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.62,"15min":0.62},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.80100000000000005,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.51300000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.77100000000000002,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.71999999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.628,"15min":0.61799999999999999},"min":{"1min":0.46400000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.88300000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.80100000000000005}]},{"osd":6,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69199999999999995,"15min":0.68999999999999995},"min":{"1min":0.501,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90000000000000002,"5min":1.024,"15min":1.024},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67100000000000004,"15min":0.66500000000000004},"min":{"1min":0.53300000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.879,"5min":1.151,"15min":1.151},"last":0.83599999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68300000000000005,"15min":0.67300000000000004},"min":{"1min":0.58499999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91800000000000004,"5min":1.097,"15min":1.097},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89700000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.88100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":979,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194333,"statfs":{"total":200000000,"available":198997341,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.38900000000000001,"15min":0.38800000000000001},"min":{"1min":0.24399999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.40899999999999997,"15min":0.41299999999999998},"min":{"1min":0.29399999999999998,"5min":0.223,"15min":0.223},"max":{"1min":0.58699999999999997,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.44}]},{"osd":2,"last update":"Sat Jan 27 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42099999999999999,"15min":0.41699999999999998},"min":{"1min":0.28699999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.30499999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.54300000000000004}]},{"osd":3,"last update":"Sat Jan 27 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.31900000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46700000000000003,"15min":0.46899999999999997},"min":{"1min":0.33600000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.628,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.48799999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95499999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.44,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.59499999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.49399999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67700000000000005,"15min":0.68200000000000005},"min":{"1min":0.47799999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.65000000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.51300000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67400000000000004,"15min":0.67300000000000004},"min":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.63400000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.69099999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.51000000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.93999999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.52000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":396,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194915,"statfs":{"total":200000000,"available":199593972,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41799999999999998,"15min":0.40999999999999998},"min":{"1min":0.26400000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56499999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.55500000000000005,"15min":0.65300000000000002},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61799999999999999,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":0.38500000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42099999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.626,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.443,"15min":0.44800000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.62,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.318}]},{"osd":3,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.42099999999999999,"15min":0.41499999999999998},"min":{"1min":0.313,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.56299999999999994,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.64900000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.42099999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.51200000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89800000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75700000000000001,"15min":0.80700000000000005},"min":{"1min":0.56399999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.91000000000000003,"5min":3.581,"15min":3.581},"last":0.61599999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69999999999999996,"15min":0.751},"min":{"1min":0.52500000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84799999999999998,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67800000000000005,"15min":0.67100000000000004},"min":{"1min":0.52700000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92000000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.63}]},{"osd":6,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.52600000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89100000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.67800000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94999999999999996,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.51100000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.79900000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69199999999999995,"15min":0.68799999999999994},"min":{"1min":0.47499999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.53900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":396,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194915,"statfs":{"total":200000000,"available":199593972,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36399999999999999,"15min":0.35099999999999998},"min":{"1min":0.27400000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.49399999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.39500000000000002,"15min":0.39100000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56000000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.31900000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39200000000000002,"15min":0.38600000000000001},"min":{"1min":0.30099999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.51500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.316,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.57899999999999996,"5min":0.628,"15min":0.628},"last":0.48799999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.42999999999999999,"15min":0.42099999999999999},"min":{"1min":0.33600000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45100000000000001,"15min":0.439},"min":{"1min":0.37,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.51400000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.498,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80900000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60799999999999998,"15min":0.59499999999999997},"min":{"1min":0.49099999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.75}]},{"osd":5,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61699999999999999,"15min":0.60299999999999998},"min":{"1min":0.44800000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.63300000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65100000000000002,"15min":0.65000000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.89000000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64600000000000002,"15min":0.64200000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.76300000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.57399999999999995}]},{"osd":7,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64100000000000001,"15min":0.629},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.54600000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.64700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574899,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":396,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194915,"statfs":{"total":200000000,"available":199593972,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.38100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.70899999999999996,"5min":1.099,"15min":1.099},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.45800000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77500000000000002,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.72299999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60299999999999998,"15min":0.59399999999999997},"min":{"1min":0.439,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86899999999999999,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.44,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79800000000000004,"5min":1.133,"15min":1.133},"last":0.81999999999999995}]},{"osd":2,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.44600000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.81299999999999994,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.3400000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.61499999999999999,"15min":0.61399999999999999},"min":{"1min":0.45600000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82799999999999996,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.69899999999999995}]},{"osd":3,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.57699999999999996},"min":{"1min":0.435,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83199999999999996,"5min":1.115,"15min":1.115},"last":1.4059999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65400000000000003,"15min":0.64800000000000002},"min":{"1min":0.47799999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93799999999999994,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.747}]},{"osd":5,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.33300000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68600000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":1.1799999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.312,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66500000000000004,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.77900000000000003}]},{"osd":6,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.503,"15min":0.5},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72299999999999998,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77700000000000002,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.95099999999999996}]},{"osd":7,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55300000000000005,"15min":0.55400000000000005},"min":{"1min":0.41599999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58299999999999996,"15min":0.57399999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77500000000000002,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.79000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":402,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194909,"statfs":{"total":200000000,"available":199587709,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.82599999999999996,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.60499999999999998,"15min":0.57799999999999996},"min":{"1min":0.39800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.75600000000000001,"5min":3.242,"15min":3.242},"last":0.55300000000000005}]},{"osd":1,"last update":"Sat Jan 27 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.622,"15min":0.60699999999999998},"min":{"1min":0.45700000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.78900000000000003,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.65200000000000002,"15min":0.65800000000000003},"min":{"1min":0.437,"5min":0.436,"15min":0.436},"max":{"1min":0.84399999999999997,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.621}]},{"osd":2,"last update":"Sat Jan 27 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.39800000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.83099999999999996,"5min":3.04,"15min":3.04},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85899999999999999,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66200000000000003}]},{"osd":3,"last update":"Sat Jan 27 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.69999999999999996,"15min":0.68999999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88900000000000001,"5min":4.032,"15min":4.032},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.71099999999999997,"15min":0.68100000000000005},"min":{"1min":0.47699999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.86199999999999999,"5min":4.282,"15min":4.282},"last":0.69799999999999995}]},{"osd":4,"last update":"Sat Jan 27 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.30599999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.79000000000000004,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.55300000000000005,"15min":0.57899999999999996},"min":{"1min":0.318,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79800000000000004,"5min":1.101,"15min":1.101},"last":0.51500000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.55000000000000004,"15min":0.56799999999999995},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.73199999999999998,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85799999999999998,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.47199999999999998}]},{"osd":7,"last update":"Sat Jan 27 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.34799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77100000000000002,"5min":1.008,"15min":1.008},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.35899999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82799999999999996,"5min":2,"15min":2},"last":0.59199999999999997}]}]}],"pool_statfs":[]}} 2024-01-27T02:36:06.699 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:36:06.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.870+0000 7fb88fa53700 1 Processor -- start 2024-01-27T02:36:06.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.870+0000 7fb88fa53700 1 -- start start 2024-01-27T02:36:06.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.871+0000 7fb88fa53700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb8880c9f80 0x7fb888157f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:06.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.871+0000 7fb88fa53700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb88806c880 con 0x7fb88806e330 2024-01-27T02:36:06.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.871+0000 7fb88fa53700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb88806bc70 con 0x7fb8880c9bb0 2024-01-27T02:36:06.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.871+0000 7fb88fa53700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb88806cd10 con 0x7fb8880c9f80 2024-01-27T02:36:06.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.871+0000 7fb88dff0700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb88806e330 0x7fb88806e750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40656/0 (socket says 172.21.15.7:40656) 2024-01-27T02:36:06.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.871+0000 7fb88dff0700 1 -- 172.21.15.7:0/671978139 learned_addr learned my addr 172.21.15.7:0/671978139 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:06.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.872+0000 7fb88cfee700 1 --2- 172.21.15.7:0/671978139 >> v2:172.21.15.188:3300/0 conn(0x7fb8880c9f80 0x7fb888157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:06.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.872+0000 7fb87ffff700 1 -- 172.21.15.7:0/671978139 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3689399460 0 0) 0x7fb88806c880 con 0x7fb88806e330 2024-01-27T02:36:06.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.872+0000 7fb87ffff700 1 -- 172.21.15.7:0/671978139 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb874003570 con 0x7fb88806e330 2024-01-27T02:36:06.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.872+0000 7fb88cfee700 1 -- 172.21.15.7:0/671978139 >> v1:172.21.15.188:6789/0 conn(0x7fb8880c9bb0 legacy=0x7fb88806b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:06.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.872+0000 7fb88cfee700 1 -- 172.21.15.7:0/671978139 >> v1:172.21.15.7:6789/0 conn(0x7fb88806e330 legacy=0x7fb88806e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:06.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.872+0000 7fb87ffff700 1 -- 172.21.15.7:0/671978139 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2216203862 0 0) 0x7fb874003570 con 0x7fb88806e330 2024-01-27T02:36:06.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.872+0000 7fb88cfee700 1 -- 172.21.15.7:0/671978139 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88806bf50 con 0x7fb8880c9f80 2024-01-27T02:36:06.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.872+0000 7fb88cfee700 1 --2- 172.21.15.7:0/671978139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8880c9f80 0x7fb888157f70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb8780098e0 tx=0x7fb8780091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4efbc988acea7292 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:06.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.873+0000 7fb87ffff700 1 -- 172.21.15.7:0/671978139 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8800017f0 con 0x7fb8880c9f80 2024-01-27T02:36:06.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.873+0000 7fb87ffff700 1 -- 172.21.15.7:0/671978139 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb878011420 con 0x7fb8880c9f80 2024-01-27T02:36:06.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.873+0000 7fb87ffff700 1 -- 172.21.15.7:0/671978139 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8780117c0 con 0x7fb8880c9f80 2024-01-27T02:36:06.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.873+0000 7fb88fa53700 1 -- 172.21.15.7:0/671978139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8880c9f80 msgr2=0x7fb888157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:06.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.873+0000 7fb88fa53700 1 --2- 172.21.15.7:0/671978139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8880c9f80 0x7fb888157f70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb8780098e0 tx=0x7fb8780091a0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.873+0000 7fb88fa53700 1 -- 172.21.15.7:0/671978139 shutdown_connections 2024-01-27T02:36:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.873+0000 7fb88fa53700 1 --2- 172.21.15.7:0/671978139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8880c9f80 0x7fb888157f70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.873+0000 7fb88fa53700 1 -- 172.21.15.7:0/671978139 >> 172.21.15.7:0/671978139 conn(0x7fb8880cdc30 msgr2=0x7fb8880ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.873+0000 7fb88fa53700 1 -- 172.21.15.7:0/671978139 shutdown_connections 2024-01-27T02:36:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.873+0000 7fb88fa53700 1 -- 172.21.15.7:0/671978139 wait complete. 2024-01-27T02:36:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.874+0000 7fb88fa53700 1 Processor -- start 2024-01-27T02:36:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.875+0000 7fb88fa53700 1 -- start start 2024-01-27T02:36:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.875+0000 7fb88fa53700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb88806e330 0x7fb8880c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.875+0000 7fb88fa53700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8880c9bb0 0x7fb8880b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.875+0000 7fb88fa53700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8880c9f80 0x7fb8880b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.875+0000 7fb88fa53700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880c54c0 con 0x7fb8880c9bb0 2024-01-27T02:36:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.875+0000 7fb88fa53700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88815c1a0 con 0x7fb8880c9f80 2024-01-27T02:36:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.875+0000 7fb88fa53700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880c5220 con 0x7fb88806e330 2024-01-27T02:36:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.876+0000 7fb88cfee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8880c9bb0 0x7fb8880b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.876+0000 7fb88cfee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8880c9bb0 0x7fb8880b1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33498/0 (socket says 172.21.15.7:33498) 2024-01-27T02:36:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.876+0000 7fb88cfee700 1 -- 172.21.15.7:0/1441091673 learned_addr learned my addr 172.21.15.7:0/1441091673 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.876+0000 7fb88dff0700 1 --2- 172.21.15.7:0/1441091673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8880c9f80 0x7fb8880b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.876+0000 7fb88d7ef700 1 --2- 172.21.15.7:0/1441091673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb88806e330 0x7fb8880c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.876+0000 7fb88cfee700 1 -- 172.21.15.7:0/1441091673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb88806e330 msgr2=0x7fb8880c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.876+0000 7fb88cfee700 1 --2- 172.21.15.7:0/1441091673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb88806e330 0x7fb8880c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.876+0000 7fb88cfee700 1 -- 172.21.15.7:0/1441091673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8880c9f80 msgr2=0x7fb8880b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.876+0000 7fb88cfee700 1 --2- 172.21.15.7:0/1441091673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8880c9f80 0x7fb8880b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.876+0000 7fb88cfee700 1 -- 172.21.15.7:0/1441091673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb878009420 con 0x7fb8880c9bb0 2024-01-27T02:36:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.876+0000 7fb88cfee700 1 --2- 172.21.15.7:0/1441091673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8880c9bb0 0x7fb8880b1090 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb8780091e0 tx=0x7fb878016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.876+0000 7fb87d7fa700 1 -- 172.21.15.7:0/1441091673 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87801cb30 con 0x7fb8880c9bb0 2024-01-27T02:36:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.876+0000 7fb87d7fa700 1 -- 172.21.15.7:0/1441091673 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87801cc90 con 0x7fb8880c9bb0 2024-01-27T02:36:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.876+0000 7fb88fa53700 1 -- 172.21.15.7:0/1441091673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88815b3e0 con 0x7fb8880c9bb0 2024-01-27T02:36:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.876+0000 7fb87d7fa700 1 -- 172.21.15.7:0/1441091673 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87801a6a0 con 0x7fb8880c9bb0 2024-01-27T02:36:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.876+0000 7fb88fa53700 1 -- 172.21.15.7:0/1441091673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88815b9a0 con 0x7fb8880c9bb0 2024-01-27T02:36:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.878+0000 7fb88fa53700 1 -- 172.21.15.7:0/1441091673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88804c710 con 0x7fb8880c9bb0 2024-01-27T02:36:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.878+0000 7fb87d7fa700 1 -- 172.21.15.7:0/1441091673 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb87801aa50 con 0x7fb8880c9bb0 2024-01-27T02:36:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.878+0000 7fb87d7fa700 1 --2- 172.21.15.7:0/1441091673 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb86c066f30 0x7fb86c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.878+0000 7fb87d7fa700 1 -- 172.21.15.7:0/1441091673 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb878085b30 con 0x7fb8880c9bb0 2024-01-27T02:36:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.879+0000 7fb88d7ef700 1 --2- 172.21.15.7:0/1441091673 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb86c066f30 0x7fb86c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.879+0000 7fb88d7ef700 1 --2- 172.21.15.7:0/1441091673 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb86c066f30 0x7fb86c0693e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb884003ac0 tx=0x7fb884007ea0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:06.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:06.883+0000 7fb87d7fa700 1 -- 172.21.15.7:0/1441091673 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb878052da0 con 0x7fb8880c9bb0 2024-01-27T02:36:07.052 INFO:tasks.ceph.mds.d.smithi188.stderr:2024-01-27T02:36:07.051+0000 7fd330de8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-27T02:36:07.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.084+0000 7fb88fa53700 1 -- 172.21.15.7:0/1441091673 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb8881518b0 con 0x7fb86c066f30 2024-01-27T02:36:07.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.093+0000 7fb87d7fa700 1 -- 172.21.15.7:0/1441091673 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282722 (secure 0 0 0) 0x7fb8881518b0 con 0x7fb86c066f30 2024-01-27T02:36:07.097 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:36:07.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.097+0000 7fb88fa53700 1 -- 172.21.15.7:0/1441091673 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb86c066f30 msgr2=0x7fb86c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:07.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.097+0000 7fb88fa53700 1 --2- 172.21.15.7:0/1441091673 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb86c066f30 0x7fb86c0693e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb884003ac0 tx=0x7fb884007ea0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.097+0000 7fb88fa53700 1 -- 172.21.15.7:0/1441091673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8880c9bb0 msgr2=0x7fb8880b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.097+0000 7fb88fa53700 1 --2- 172.21.15.7:0/1441091673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8880c9bb0 0x7fb8880b1090 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb8780091e0 tx=0x7fb878016000 comp rx=0 tx=0).stop 2024-01-27T02:36:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.097+0000 7fb88fa53700 1 -- 172.21.15.7:0/1441091673 shutdown_connections 2024-01-27T02:36:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.097+0000 7fb88fa53700 1 --2- 172.21.15.7:0/1441091673 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb86c066f30 0x7fb86c0693e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.097+0000 7fb88fa53700 1 --2- 172.21.15.7:0/1441091673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8880c9f80 0x7fb8880b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.097+0000 7fb88fa53700 1 --2- 172.21.15.7:0/1441091673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb88806e330 0x7fb8880c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.097+0000 7fb88fa53700 1 --2- 172.21.15.7:0/1441091673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8880c9bb0 0x7fb8880b1090 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.097+0000 7fb88fa53700 1 -- 172.21.15.7:0/1441091673 >> 172.21.15.7:0/1441091673 conn(0x7fb8880cdc30 msgr2=0x7fb8880ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.097+0000 7fb88fa53700 1 -- 172.21.15.7:0/1441091673 shutdown_connections 2024-01-27T02:36:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.097+0000 7fb88fa53700 1 -- 172.21.15.7:0/1441091673 wait complete. 2024-01-27T02:36:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:36:07.111 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2024-01-27T02:36:05.631498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4343,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558156,"statfs":{"total":1600000000,"available":1595552251,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.567011"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077060+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:19:59.227343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078576+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:09.241354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.071538+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:51:24.783765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078657+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:45:48.321027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079475+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:17:28.880573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.080893+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:25:43.791658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078520+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:57:20.844278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077646+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:16:42.015521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078957+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:25.098588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.081944+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:21:41.100136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076679+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:57:20.844278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078947+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:39:19.173475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.080386+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:32:03.288747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077576+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:41:44.104324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079892+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:26:17.309915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079382+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:28:19.604756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.080873+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:47:56.363317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.080422+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:10:26.712434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.075169+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:15:05.885263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079455+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:19:38.789133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078990+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:18:04.141154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079678+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:25:43.791658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.075028+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:26:17.309915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076931+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:12:02.170281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.081562+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:10:52.422378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076922+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:21.383627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.071521+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:51:11.452504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078331+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:09.241354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.071456+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:37:39.458923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076978+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:48:39.299949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076528+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:41:50.546172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078193+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:27:01.184977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076537+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:48:39.299949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.077741+0000","last_change":"2024-01-27T02:36:01.077741+0000","last_active":"2024-01-27T02:36:01.077741+0000","last_peered":"2024-01-27T02:36:01.077741+0000","last_clean":"2024-01-27T02:36:01.077741+0000","last_became_active":"2024-01-27T02:36:01.077423+0000","last_became_peered":"2024-01-27T02:36:01.077423+0000","last_unstale":"2024-01-27T02:36:01.077741+0000","last_undegraded":"2024-01-27T02:36:01.077741+0000","last_fullsized":"2024-01-27T02:36:01.077741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:15.272806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077778+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:09:23.740404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.084751+0000","last_change":"2024-01-27T02:36:01.084751+0000","last_active":"2024-01-27T02:36:01.084751+0000","last_peered":"2024-01-27T02:36:01.084751+0000","last_clean":"2024-01-27T02:36:01.084751+0000","last_became_active":"2024-01-27T02:36:01.084351+0000","last_became_peered":"2024-01-27T02:36:01.084351+0000","last_unstale":"2024-01-27T02:36:01.084751+0000","last_undegraded":"2024-01-27T02:36:01.084751+0000","last_fullsized":"2024-01-27T02:36:01.084751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:42:38.497144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.080521+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:31:44.339745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.087593+0000","last_change":"2024-01-27T02:36:01.087593+0000","last_active":"2024-01-27T02:36:01.087593+0000","last_peered":"2024-01-27T02:36:01.087593+0000","last_clean":"2024-01-27T02:36:01.087593+0000","last_became_active":"2024-01-27T02:36:01.086695+0000","last_became_peered":"2024-01-27T02:36:01.086695+0000","last_unstale":"2024-01-27T02:36:01.087593+0000","last_undegraded":"2024-01-27T02:36:01.087593+0000","last_fullsized":"2024-01-27T02:36:01.087593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:15:04.879041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077553+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:56:33.979247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.082918+0000","last_change":"2024-01-27T02:36:01.082918+0000","last_active":"2024-01-27T02:36:01.082918+0000","last_peered":"2024-01-27T02:36:01.082918+0000","last_clean":"2024-01-27T02:36:01.082918+0000","last_became_active":"2024-01-27T02:36:01.081169+0000","last_became_peered":"2024-01-27T02:36:01.081169+0000","last_unstale":"2024-01-27T02:36:01.082918+0000","last_undegraded":"2024-01-27T02:36:01.082918+0000","last_fullsized":"2024-01-27T02:36:01.082918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:39:18.167253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.075988+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:32:03.288747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.083508+0000","last_change":"2024-01-27T02:36:01.083508+0000","last_active":"2024-01-27T02:36:01.083508+0000","last_peered":"2024-01-27T02:36:01.083508+0000","last_clean":"2024-01-27T02:36:01.083508+0000","last_became_active":"2024-01-27T02:36:01.083244+0000","last_became_peered":"2024-01-27T02:36:01.083244+0000","last_unstale":"2024-01-27T02:36:01.083508+0000","last_undegraded":"2024-01-27T02:36:01.083508+0000","last_fullsized":"2024-01-27T02:36:01.083508+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:42:06.107055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079272+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:31:44.339745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.081011+0000","last_change":"2024-01-27T02:36:01.081011+0000","last_active":"2024-01-27T02:36:01.081011+0000","last_peered":"2024-01-27T02:36:01.081011+0000","last_clean":"2024-01-27T02:36:01.081011+0000","last_became_active":"2024-01-27T02:36:01.080166+0000","last_became_peered":"2024-01-27T02:36:01.080166+0000","last_unstale":"2024-01-27T02:36:01.081011+0000","last_undegraded":"2024-01-27T02:36:01.081011+0000","last_fullsized":"2024-01-27T02:36:01.081011+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:42:38.578182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079075+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:03:33.449715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.086427+0000","last_change":"2024-01-27T02:36:01.086427+0000","last_active":"2024-01-27T02:36:01.086427+0000","last_peered":"2024-01-27T02:36:01.086427+0000","last_clean":"2024-01-27T02:36:01.086427+0000","last_became_active":"2024-01-27T02:36:01.085912+0000","last_became_peered":"2024-01-27T02:36:01.085912+0000","last_unstale":"2024-01-27T02:36:01.086427+0000","last_undegraded":"2024-01-27T02:36:01.086427+0000","last_fullsized":"2024-01-27T02:36:01.086427+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:12:59.343148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079659+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:21:28.748243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.084880+0000","last_change":"2024-01-27T02:36:01.084880+0000","last_active":"2024-01-27T02:36:01.084880+0000","last_peered":"2024-01-27T02:36:01.084880+0000","last_clean":"2024-01-27T02:36:01.084880+0000","last_became_active":"2024-01-27T02:36:01.084532+0000","last_became_peered":"2024-01-27T02:36:01.084532+0000","last_unstale":"2024-01-27T02:36:01.084880+0000","last_undegraded":"2024-01-27T02:36:01.084880+0000","last_fullsized":"2024-01-27T02:36:01.084880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:58:04.302291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076109+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:32:03.273755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.083170+0000","last_change":"2024-01-27T02:36:01.083170+0000","last_active":"2024-01-27T02:36:01.083170+0000","last_peered":"2024-01-27T02:36:01.083170+0000","last_clean":"2024-01-27T02:36:01.083170+0000","last_became_active":"2024-01-27T02:36:01.082730+0000","last_became_peered":"2024-01-27T02:36:01.082730+0000","last_unstale":"2024-01-27T02:36:01.083170+0000","last_undegraded":"2024-01-27T02:36:01.083170+0000","last_fullsized":"2024-01-27T02:36:01.083170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:07:19.984794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076273+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:21.383627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.079850+0000","last_change":"2024-01-27T02:36:01.079850+0000","last_active":"2024-01-27T02:36:01.079850+0000","last_peered":"2024-01-27T02:36:01.079850+0000","last_clean":"2024-01-27T02:36:01.079850+0000","last_became_active":"2024-01-27T02:36:01.079488+0000","last_became_peered":"2024-01-27T02:36:01.079488+0000","last_unstale":"2024-01-27T02:36:01.079850+0000","last_undegraded":"2024-01-27T02:36:01.079850+0000","last_fullsized":"2024-01-27T02:36:01.079850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:36:51.217544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.074770+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:35:14.198571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.075921+0000","last_change":"2024-01-27T02:36:01.075921+0000","last_active":"2024-01-27T02:36:01.075921+0000","last_peered":"2024-01-27T02:36:01.075921+0000","last_clean":"2024-01-27T02:36:01.075921+0000","last_became_active":"2024-01-27T02:36:01.075647+0000","last_became_peered":"2024-01-27T02:36:01.075647+0000","last_unstale":"2024-01-27T02:36:01.075921+0000","last_undegraded":"2024-01-27T02:36:01.075921+0000","last_fullsized":"2024-01-27T02:36:01.075921+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:54:58.712360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077216+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:22:30.371211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.085134+0000","last_change":"2024-01-27T02:36:01.085134+0000","last_active":"2024-01-27T02:36:01.085134+0000","last_peered":"2024-01-27T02:36:01.085134+0000","last_clean":"2024-01-27T02:36:01.085134+0000","last_became_active":"2024-01-27T02:36:01.084887+0000","last_became_peered":"2024-01-27T02:36:01.084887+0000","last_unstale":"2024-01-27T02:36:01.085134+0000","last_undegraded":"2024-01-27T02:36:01.085134+0000","last_fullsized":"2024-01-27T02:36:01.085134+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:24:30.352994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.075834+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:14:13.651138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.081854+0000","last_change":"2024-01-27T02:36:01.081854+0000","last_active":"2024-01-27T02:36:01.081854+0000","last_peered":"2024-01-27T02:36:01.081854+0000","last_clean":"2024-01-27T02:36:01.081854+0000","last_became_active":"2024-01-27T02:36:01.081553+0000","last_became_peered":"2024-01-27T02:36:01.081553+0000","last_unstale":"2024-01-27T02:36:01.081854+0000","last_undegraded":"2024-01-27T02:36:01.081854+0000","last_fullsized":"2024-01-27T02:36:01.081854+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:36:51.217544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076474+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:14:13.651138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.087113+0000","last_change":"2024-01-27T02:36:01.087113+0000","last_active":"2024-01-27T02:36:01.087113+0000","last_peered":"2024-01-27T02:36:01.087113+0000","last_clean":"2024-01-27T02:36:01.087113+0000","last_became_active":"2024-01-27T02:36:01.086735+0000","last_became_peered":"2024-01-27T02:36:01.086735+0000","last_unstale":"2024-01-27T02:36:01.087113+0000","last_undegraded":"2024-01-27T02:36:01.087113+0000","last_fullsized":"2024-01-27T02:36:01.087113+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:51:30.355507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079910+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:04:07.439211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.079455+0000","last_change":"2024-01-27T02:36:01.079455+0000","last_active":"2024-01-27T02:36:01.079455+0000","last_peered":"2024-01-27T02:36:01.079455+0000","last_clean":"2024-01-27T02:36:01.079455+0000","last_became_active":"2024-01-27T02:36:01.078810+0000","last_became_peered":"2024-01-27T02:36:01.078810+0000","last_unstale":"2024-01-27T02:36:01.079455+0000","last_undegraded":"2024-01-27T02:36:01.079455+0000","last_fullsized":"2024-01-27T02:36:01.079455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:32:57.446408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078346+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:32:55.342184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.083387+0000","last_change":"2024-01-27T02:36:01.083387+0000","last_active":"2024-01-27T02:36:01.083387+0000","last_peered":"2024-01-27T02:36:01.083387+0000","last_clean":"2024-01-27T02:36:01.083387+0000","last_became_active":"2024-01-27T02:36:01.083023+0000","last_became_peered":"2024-01-27T02:36:01.083023+0000","last_unstale":"2024-01-27T02:36:01.083387+0000","last_undegraded":"2024-01-27T02:36:01.083387+0000","last_fullsized":"2024-01-27T02:36:01.083387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:16:41.009300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079871+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:42.090153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.075386+0000","last_change":"2024-01-27T02:36:01.075386+0000","last_active":"2024-01-27T02:36:01.075386+0000","last_peered":"2024-01-27T02:36:01.075386+0000","last_clean":"2024-01-27T02:36:01.075386+0000","last_became_active":"2024-01-27T02:36:01.074276+0000","last_became_peered":"2024-01-27T02:36:01.074276+0000","last_unstale":"2024-01-27T02:36:01.075386+0000","last_undegraded":"2024-01-27T02:36:01.075386+0000","last_fullsized":"2024-01-27T02:36:01.075386+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:46:34.597904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.076123+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:41:50.546172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.087262+0000","last_change":"2024-01-27T02:36:01.087262+0000","last_active":"2024-01-27T02:36:01.087262+0000","last_peered":"2024-01-27T02:36:01.087262+0000","last_clean":"2024-01-27T02:36:01.087262+0000","last_became_active":"2024-01-27T02:36:01.086163+0000","last_became_peered":"2024-01-27T02:36:01.086163+0000","last_unstale":"2024-01-27T02:36:01.087262+0000","last_undegraded":"2024-01-27T02:36:01.087262+0000","last_fullsized":"2024-01-27T02:36:01.087262+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:35:13.192349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078527+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:23:57.404805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.086968+0000","last_change":"2024-01-27T02:36:01.086968+0000","last_active":"2024-01-27T02:36:01.086968+0000","last_peered":"2024-01-27T02:36:01.086968+0000","last_clean":"2024-01-27T02:36:01.086968+0000","last_became_active":"2024-01-27T02:36:01.086112+0000","last_became_peered":"2024-01-27T02:36:01.086112+0000","last_unstale":"2024-01-27T02:36:01.086968+0000","last_undegraded":"2024-01-27T02:36:01.086968+0000","last_fullsized":"2024-01-27T02:36:01.086968+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:03:55.891925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078774+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:19:43.900445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.085172+0000","last_change":"2024-01-27T02:36:01.085172+0000","last_active":"2024-01-27T02:36:01.085172+0000","last_peered":"2024-01-27T02:36:01.085172+0000","last_clean":"2024-01-27T02:36:01.085172+0000","last_became_active":"2024-01-27T02:36:01.084894+0000","last_became_peered":"2024-01-27T02:36:01.084894+0000","last_unstale":"2024-01-27T02:36:01.085172+0000","last_undegraded":"2024-01-27T02:36:01.085172+0000","last_fullsized":"2024-01-27T02:36:01.085172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:19:37.782912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.070899+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:05:31.415970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.080657+0000","last_change":"2024-01-27T02:36:01.080657+0000","last_active":"2024-01-27T02:36:01.080657+0000","last_peered":"2024-01-27T02:36:01.080657+0000","last_clean":"2024-01-27T02:36:01.080657+0000","last_became_active":"2024-01-27T02:36:01.078442+0000","last_became_peered":"2024-01-27T02:36:01.078442+0000","last_unstale":"2024-01-27T02:36:01.080657+0000","last_undegraded":"2024-01-27T02:36:01.080657+0000","last_fullsized":"2024-01-27T02:36:01.080657+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:45:47.314805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.070938+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:22:09.618511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.085914+0000","last_change":"2024-01-27T02:36:01.085914+0000","last_active":"2024-01-27T02:36:01.085914+0000","last_peered":"2024-01-27T02:36:01.085914+0000","last_clean":"2024-01-27T02:36:01.085914+0000","last_became_active":"2024-01-27T02:36:01.085674+0000","last_became_peered":"2024-01-27T02:36:01.085674+0000","last_unstale":"2024-01-27T02:36:01.085914+0000","last_undegraded":"2024-01-27T02:36:01.085914+0000","last_fullsized":"2024-01-27T02:36:01.085914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:19:37.782912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.070886+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:47:18.259755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.086128+0000","last_change":"2024-01-27T02:36:01.086128+0000","last_active":"2024-01-27T02:36:01.086128+0000","last_peered":"2024-01-27T02:36:01.086128+0000","last_clean":"2024-01-27T02:36:01.086128+0000","last_became_active":"2024-01-27T02:36:01.085891+0000","last_became_peered":"2024-01-27T02:36:01.085891+0000","last_unstale":"2024-01-27T02:36:01.086128+0000","last_undegraded":"2024-01-27T02:36:01.086128+0000","last_fullsized":"2024-01-27T02:36:01.086128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:05:15.222635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.079413+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:16.279028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.079712+0000","last_change":"2024-01-27T02:36:01.079712+0000","last_active":"2024-01-27T02:36:01.079712+0000","last_peered":"2024-01-27T02:36:01.079712+0000","last_clean":"2024-01-27T02:36:01.079712+0000","last_became_active":"2024-01-27T02:36:01.079176+0000","last_became_peered":"2024-01-27T02:36:01.079176+0000","last_unstale":"2024-01-27T02:36:01.079712+0000","last_undegraded":"2024-01-27T02:36:01.079712+0000","last_fullsized":"2024-01-27T02:36:01.079712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:12:59.343148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.075688+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:26:17.309915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.084252+0000","last_change":"2024-01-27T02:36:01.084252+0000","last_active":"2024-01-27T02:36:01.084252+0000","last_peered":"2024-01-27T02:36:01.084252+0000","last_clean":"2024-01-27T02:36:01.084252+0000","last_became_active":"2024-01-27T02:36:01.083785+0000","last_became_peered":"2024-01-27T02:36:01.083785+0000","last_unstale":"2024-01-27T02:36:01.084252+0000","last_undegraded":"2024-01-27T02:36:01.084252+0000","last_fullsized":"2024-01-27T02:36:01.084252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:15.272806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.075405+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:32:03.288747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.083357+0000","last_change":"2024-01-27T02:36:01.083357+0000","last_active":"2024-01-27T02:36:01.083357+0000","last_peered":"2024-01-27T02:36:01.083357+0000","last_clean":"2024-01-27T02:36:01.083357+0000","last_became_active":"2024-01-27T02:36:01.083009+0000","last_became_peered":"2024-01-27T02:36:01.083009+0000","last_unstale":"2024-01-27T02:36:01.083357+0000","last_undegraded":"2024-01-27T02:36:01.083357+0000","last_fullsized":"2024-01-27T02:36:01.083357+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:17:17.877802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.075713+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:32:03.273755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.083453+0000","last_change":"2024-01-27T02:36:01.083453+0000","last_active":"2024-01-27T02:36:01.083453+0000","last_peered":"2024-01-27T02:36:01.083453+0000","last_clean":"2024-01-27T02:36:01.083453+0000","last_became_active":"2024-01-27T02:36:01.083173+0000","last_became_peered":"2024-01-27T02:36:01.083173+0000","last_unstale":"2024-01-27T02:36:01.083453+0000","last_undegraded":"2024-01-27T02:36:01.083453+0000","last_fullsized":"2024-01-27T02:36:01.083453+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:21:40.093915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077155+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:11:23.338931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.086016+0000","last_change":"2024-01-27T02:36:01.086016+0000","last_active":"2024-01-27T02:36:01.086016+0000","last_peered":"2024-01-27T02:36:01.086016+0000","last_clean":"2024-01-27T02:36:01.086016+0000","last_became_active":"2024-01-27T02:36:01.085747+0000","last_became_peered":"2024-01-27T02:36:01.085747+0000","last_unstale":"2024-01-27T02:36:01.086016+0000","last_undegraded":"2024-01-27T02:36:01.086016+0000","last_fullsized":"2024-01-27T02:36:01.086016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:03:04.731782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.080121+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:18:04.141154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.085778+0000","last_change":"2024-01-27T02:36:01.085778+0000","last_active":"2024-01-27T02:36:01.085778+0000","last_peered":"2024-01-27T02:36:01.085778+0000","last_clean":"2024-01-27T02:36:01.085778+0000","last_became_active":"2024-01-27T02:36:01.085504+0000","last_became_peered":"2024-01-27T02:36:01.085504+0000","last_unstale":"2024-01-27T02:36:01.085778+0000","last_undegraded":"2024-01-27T02:36:01.085778+0000","last_fullsized":"2024-01-27T02:36:01.085778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:34:00.107552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.078206+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:03:56.898146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.083238+0000","last_change":"2024-01-27T02:36:01.083238+0000","last_active":"2024-01-27T02:36:01.083238+0000","last_peered":"2024-01-27T02:36:01.083238+0000","last_clean":"2024-01-27T02:36:01.083238+0000","last_became_active":"2024-01-27T02:36:01.082768+0000","last_became_peered":"2024-01-27T02:36:01.082768+0000","last_unstale":"2024-01-27T02:36:01.083238+0000","last_undegraded":"2024-01-27T02:36:01.083238+0000","last_fullsized":"2024-01-27T02:36:01.083238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:34:52.370887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.077264+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:34:53.377109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.078683+0000","last_change":"2024-01-27T02:36:01.078683+0000","last_active":"2024-01-27T02:36:01.078683+0000","last_peered":"2024-01-27T02:36:01.078683+0000","last_clean":"2024-01-27T02:36:01.078683+0000","last_became_active":"2024-01-27T02:36:01.078126+0000","last_became_peered":"2024-01-27T02:36:01.078126+0000","last_unstale":"2024-01-27T02:36:01.078683+0000","last_undegraded":"2024-01-27T02:36:01.078683+0000","last_fullsized":"2024-01-27T02:36:01.078683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:10:25.706212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:36:01.063602+0000","last_change":"2024-01-27T02:36:01.070886+0000","last_active":"2024-01-27T02:36:01.063602+0000","last_peered":"2024-01-27T02:36:01.063602+0000","last_clean":"2024-01-27T02:36:01.063602+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:36:01.063602+0000","last_undegraded":"2024-01-27T02:36:01.063602+0000","last_fullsized":"2024-01-27T02:36:01.063602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:09:00.736774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":8,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-01-27T02:36:01.080741+0000","last_change":"2024-01-27T02:36:01.080741+0000","last_active":"2024-01-27T02:36:01.080741+0000","last_peered":"2024-01-27T02:36:01.080741+0000","last_clean":"2024-01-27T02:36:01.080741+0000","last_became_active":"2024-01-27T02:36:01.079822+0000","last_became_peered":"2024-01-27T02:36:01.079822+0000","last_unstale":"2024-01-27T02:36:01.080741+0000","last_undegraded":"2024-01-27T02:36:01.080741+0000","last_fullsized":"2024-01-27T02:36:01.080741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:23:56.398583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":211,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-01-27T02:36:00.065172+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:36:00.065172+0000","last_peered":"2024-01-27T02:36:00.065172+0000","last_clean":"2024-01-27T02:36:00.065172+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:36:00.065172+0000","last_undegraded":"2024-01-27T02:36:00.065172+0000","last_fullsized":"2024-01-27T02:36:00.065172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":973,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194339,"statfs":{"total":200000000,"available":199003604,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.50900000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.41599999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.46899999999999997}]},{"osd":2,"last update":"Sat Jan 27 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.499,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.59299999999999997,"15min":0.58399999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.41499999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59799999999999998,"15min":0.59099999999999997},"min":{"1min":0.48199999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.59799999999999998,"15min":0.59099999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.45500000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48999999999999999,"15min":0.49199999999999999},"min":{"1min":0.317,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69699999999999995,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.57799999999999996,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.43099999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.53600000000000003,"15min":0.54200000000000004},"min":{"1min":0.39300000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.59099999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.56200000000000006,"15min":0.56799999999999995},"min":{"1min":0.36899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.63900000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.60699999999999998}]},{"osd":6,"last update":"Sat Jan 27 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59399999999999997,"15min":0.60099999999999998},"min":{"1min":0.438,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.67700000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.40100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.68000000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.48399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574899,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":402,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194909,"statfs":{"total":200000000,"available":199587709,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.56000000000000005,"15min":0.51200000000000001},"min":{"1min":0.434,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79700000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55800000000000005,"15min":0.52000000000000002},"min":{"1min":0.35999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.59999999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.55700000000000005,"15min":0.51000000000000001},"min":{"1min":0.42599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76500000000000001,"5min":0.874,"15min":0.874},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58699999999999997,"15min":0.54900000000000004},"min":{"1min":0.44500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82199999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61499999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56599999999999995,"15min":0.53900000000000003},"min":{"1min":0.46999999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73999999999999999,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.61099999999999999,"15min":0.57799999999999996},"min":{"1min":0.52200000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80400000000000005,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.63800000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.48799999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.74299999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.56699999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.78400000000000003,"5min":1.262,"15min":1.262},"last":0.57099999999999995}]},{"osd":4,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.36599999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70299999999999996,"5min":0.748,"15min":0.748},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53600000000000003,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70299999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.54600000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52900000000000003,"15min":0.53400000000000003},"min":{"1min":0.42199999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65600000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58699999999999997,"15min":0.60899999999999999},"min":{"1min":0.41799999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76700000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.438}]},{"osd":7,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.61699999999999999},"min":{"1min":0.39300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.73499999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.5},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57799999999999996,"15min":0.59199999999999997},"min":{"1min":0.38500000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71599999999999997,"5min":1.038,"15min":1.038},"last":0.41999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":396,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194915,"statfs":{"total":200000000,"available":199593972,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.379,"15min":0.371},"min":{"1min":0.29699999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39900000000000002,"15min":0.39300000000000002},"min":{"1min":0.28599999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.53700000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.373,"15min":0.35699999999999998},"min":{"1min":0.26600000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.53800000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.42999999999999999,"15min":0.42299999999999999},"min":{"1min":0.34899999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.55700000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.47199999999999998,"15min":0.46300000000000002},"min":{"1min":0.39700000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65600000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.625},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.34999999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.60799999999999998}]},{"osd":4,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.62,"15min":0.62},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.80100000000000005,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.51300000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.77100000000000002,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.71999999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.628,"15min":0.61799999999999999},"min":{"1min":0.46400000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.88300000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.80100000000000005}]},{"osd":6,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69199999999999995,"15min":0.68999999999999995},"min":{"1min":0.501,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90000000000000002,"5min":1.024,"15min":1.024},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67100000000000004,"15min":0.66500000000000004},"min":{"1min":0.53300000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.879,"5min":1.151,"15min":1.151},"last":0.83599999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68300000000000005,"15min":0.67300000000000004},"min":{"1min":0.58499999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91800000000000004,"5min":1.097,"15min":1.097},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89700000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.88100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":979,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194333,"statfs":{"total":200000000,"available":198997341,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.38900000000000001,"15min":0.38800000000000001},"min":{"1min":0.24399999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.40899999999999997,"15min":0.41299999999999998},"min":{"1min":0.29399999999999998,"5min":0.223,"15min":0.223},"max":{"1min":0.58699999999999997,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.44}]},{"osd":2,"last update":"Sat Jan 27 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42099999999999999,"15min":0.41699999999999998},"min":{"1min":0.28699999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.30499999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.54300000000000004}]},{"osd":3,"last update":"Sat Jan 27 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.43099999999999999},"min":{"1min":0.31900000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46700000000000003,"15min":0.46899999999999997},"min":{"1min":0.33600000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.628,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.48799999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95499999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.44,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.59499999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.49399999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67700000000000005,"15min":0.68200000000000005},"min":{"1min":0.47799999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.65000000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.51300000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67400000000000004,"15min":0.67300000000000004},"min":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.63400000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.69099999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.51000000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.93999999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.52000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":396,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194915,"statfs":{"total":200000000,"available":199593972,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41799999999999998,"15min":0.40999999999999998},"min":{"1min":0.26400000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56499999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.55500000000000005,"15min":0.65300000000000002},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61799999999999999,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":0.38500000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42099999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.626,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.443,"15min":0.44800000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.62,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.318}]},{"osd":3,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.42099999999999999,"15min":0.41499999999999998},"min":{"1min":0.313,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.56299999999999994,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.64900000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.42099999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.51200000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89800000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75700000000000001,"15min":0.80700000000000005},"min":{"1min":0.56399999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.91000000000000003,"5min":3.581,"15min":3.581},"last":0.61599999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69999999999999996,"15min":0.751},"min":{"1min":0.52500000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84799999999999998,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67800000000000005,"15min":0.67100000000000004},"min":{"1min":0.52700000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92000000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.63}]},{"osd":6,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.52600000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89100000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.67800000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94999999999999996,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.51100000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.79900000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69199999999999995,"15min":0.68799999999999994},"min":{"1min":0.47499999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.53900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":396,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194915,"statfs":{"total":200000000,"available":199593972,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36399999999999999,"15min":0.35099999999999998},"min":{"1min":0.27400000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.49399999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.39500000000000002,"15min":0.39100000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.56000000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.31900000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39200000000000002,"15min":0.38600000000000001},"min":{"1min":0.30099999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.51500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.316,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.57899999999999996,"5min":0.628,"15min":0.628},"last":0.48799999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.42999999999999999,"15min":0.42099999999999999},"min":{"1min":0.33600000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45100000000000001,"15min":0.439},"min":{"1min":0.37,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.51400000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.498,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80900000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60799999999999998,"15min":0.59499999999999997},"min":{"1min":0.49099999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.75}]},{"osd":5,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61699999999999999,"15min":0.60299999999999998},"min":{"1min":0.44800000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.63300000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65100000000000002,"15min":0.65000000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.89000000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64600000000000002,"15min":0.64200000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.76300000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.57399999999999995}]},{"osd":7,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64100000000000001,"15min":0.629},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.54600000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.64700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574899,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":396,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194915,"statfs":{"total":200000000,"available":199593972,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.38100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.70899999999999996,"5min":1.099,"15min":1.099},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.45800000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77500000000000002,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.72299999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60299999999999998,"15min":0.59399999999999997},"min":{"1min":0.439,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86899999999999999,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.44,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79800000000000004,"5min":1.133,"15min":1.133},"last":0.81999999999999995}]},{"osd":2,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.44600000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.81299999999999994,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.3400000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.61499999999999999,"15min":0.61399999999999999},"min":{"1min":0.45600000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82799999999999996,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.69899999999999995}]},{"osd":3,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.57699999999999996},"min":{"1min":0.435,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83199999999999996,"5min":1.115,"15min":1.115},"last":1.4059999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65400000000000003,"15min":0.64800000000000002},"min":{"1min":0.47799999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93799999999999994,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.747}]},{"osd":5,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.33300000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68600000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":1.1799999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.312,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66500000000000004,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.77900000000000003}]},{"osd":6,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.503,"15min":0.5},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72299999999999998,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77700000000000002,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.95099999999999996}]},{"osd":7,"last update":"Sat Jan 27 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55300000000000005,"15min":0.55400000000000005},"min":{"1min":0.41599999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58299999999999996,"15min":0.57399999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77500000000000002,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.79000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":402,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194909,"statfs":{"total":200000000,"available":199587709,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.82599999999999996,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.60499999999999998,"15min":0.57799999999999996},"min":{"1min":0.39800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.75600000000000001,"5min":3.242,"15min":3.242},"last":0.55300000000000005}]},{"osd":1,"last update":"Sat Jan 27 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.622,"15min":0.60699999999999998},"min":{"1min":0.45700000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.78900000000000003,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.65200000000000002,"15min":0.65800000000000003},"min":{"1min":0.437,"5min":0.436,"15min":0.436},"max":{"1min":0.84399999999999997,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.621}]},{"osd":2,"last update":"Sat Jan 27 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.39800000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.83099999999999996,"5min":3.04,"15min":3.04},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85899999999999999,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66200000000000003}]},{"osd":3,"last update":"Sat Jan 27 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.69999999999999996,"15min":0.68999999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88900000000000001,"5min":4.032,"15min":4.032},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.71099999999999997,"15min":0.68100000000000005},"min":{"1min":0.47699999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.86199999999999999,"5min":4.282,"15min":4.282},"last":0.69799999999999995}]},{"osd":4,"last update":"Sat Jan 27 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.30599999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.79000000000000004,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.55300000000000005,"15min":0.57899999999999996},"min":{"1min":0.318,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79800000000000004,"5min":1.101,"15min":1.101},"last":0.51500000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.55000000000000004,"15min":0.56799999999999995},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.73199999999999998,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85799999999999998,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.47199999999999998}]},{"osd":7,"last update":"Sat Jan 27 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.34799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77100000000000002,"5min":1.008,"15min":1.008},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.35899999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82799999999999996,"5min":2,"15min":2},"last":0.59199999999999997}]}]}],"pool_statfs":[]}} 2024-01-27T02:36:07.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2024-01-27T02:36:07.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.077060+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T05:19:59.227343+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:36:07.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3e is not active+clean 2024-01-27T02:36:07.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.078576+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T14:12:09.241354+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:36:07.117 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3d is not active+clean 2024-01-27T02:36:07.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.071538+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:51:24.783765+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:36:07.117 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2024-01-27T02:36:07.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.078657+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T09:45:48.321027+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:36:07.117 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3b is not active+clean 2024-01-27T02:36:07.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.079475+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:17:28.880573+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:36:07.117 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3a is not active+clean 2024-01-27T02:36:07.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.080893+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T05:25:43.791658+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:36:07.117 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2024-01-27T02:36:07.117 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.078520+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T03:57:20.844278+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:36:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.38 is not active+clean 2024-01-27T02:36:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.077646+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T03:16:42.015521+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:36:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.37 is not active+clean 2024-01-27T02:36:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.078957+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T02:38:25.098588+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:36:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2024-01-27T02:36:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.081944+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T05:21:41.100136+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:36:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2024-01-27T02:36:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.076679+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T03:57:20.844278+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:36:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2024-01-27T02:36:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.078947+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T12:39:19.173475+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:36:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2024-01-27T02:36:07.118 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.080386+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T04:32:03.288747+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:36:07.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.32 is not active+clean 2024-01-27T02:36:07.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.077576+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:41:44.104324+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:36:07.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2024-01-27T02:36:07.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.079892+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T04:26:17.309915+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:36:07.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2024-01-27T02:36:07.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.079382+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:28:19.604756+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:36:07.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2f is not active+clean 2024-01-27T02:36:07.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.080873+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T06:47:56.363317+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:36:07.119 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2024-01-27T02:36:07.119 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.080422+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:10:26.712434+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:36:07.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2d is not active+clean 2024-01-27T02:36:07.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.075169+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:15:05.885263+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:36:07.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2024-01-27T02:36:07.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.079455+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:19:38.789133+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:36:07.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2024-01-27T02:36:07.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.078990+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:18:04.141154+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:36:07.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2024-01-27T02:36:07.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.079678+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T05:25:43.791658+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:36:07.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2024-01-27T02:36:07.120 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.075028+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T04:26:17.309915+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:36:07.120 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2024-01-27T02:36:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.076931+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:12:02.170281+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:36:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2024-01-27T02:36:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.081562+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T14:10:52.422378+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:36:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2024-01-27T02:36:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.076922+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:35:21.383627+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:36:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.25 is not active+clean 2024-01-27T02:36:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.071521+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:51:11.452504+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:36:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2024-01-27T02:36:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.078331+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T14:12:09.241354+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:36:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.23 is not active+clean 2024-01-27T02:36:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.071456+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T09:37:39.458923+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:36:07.121 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2024-01-27T02:36:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.076978+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T05:48:39.299949+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:36:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2024-01-27T02:36:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.076528+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:41:50.546172+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:36:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.20 is not active+clean 2024-01-27T02:36:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.078193+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T14:27:01.184977+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:36:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2024-01-27T02:36:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.076537+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T05:48:39.299949+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:36:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2024-01-27T02:36:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.077778+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T12:09:23.740404+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:36:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2024-01-27T02:36:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.080521+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T04:31:44.339745+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:36:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2024-01-27T02:36:07.122 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.077553+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:56:33.979247+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:36:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2024-01-27T02:36:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.075988+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T04:32:03.288747+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:36:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2024-01-27T02:36:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.079272+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T04:31:44.339745+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:36:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2024-01-27T02:36:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.079075+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T13:03:33.449715+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:36:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2024-01-27T02:36:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.079659+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:21:28.748243+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:36:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2024-01-27T02:36:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.076109+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T13:32:03.273755+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:36:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2024-01-27T02:36:07.123 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.076273+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:35:21.383627+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:36:07.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2024-01-27T02:36:07.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.074770+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T06:35:14.198571+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:36:07.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2024-01-27T02:36:07.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.077216+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T06:22:30.371211+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:36:07.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2024-01-27T02:36:07.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.075834+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T09:14:13.651138+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:36:07.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2024-01-27T02:36:07.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.076474+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T09:14:13.651138+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:36:07.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2024-01-27T02:36:07.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.079910+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T13:04:07.439211+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:36:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2024-01-27T02:36:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.078346+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T05:32:55.342184+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:36:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2024-01-27T02:36:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.079871+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:03:42.090153+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:36:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2024-01-27T02:36:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.076123+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:41:50.546172+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:36:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2024-01-27T02:36:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.078527+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:23:57.404805+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:36:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2024-01-27T02:36:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.078774+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T14:19:43.900445+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:36:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2024-01-27T02:36:07.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.070899+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T09:05:31.415970+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:36:07.126 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2024-01-27T02:36:07.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.070938+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:22:09.618511+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:36:07.126 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2024-01-27T02:36:07.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.070886+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:47:18.259755+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:36:07.126 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2024-01-27T02:36:07.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.079413+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:03:16.279028+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:36:07.126 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2024-01-27T02:36:07.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.075688+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T04:26:17.309915+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:36:07.126 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2024-01-27T02:36:07.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.075405+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T04:32:03.288747+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:36:07.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2024-01-27T02:36:07.127 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.075713+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T13:32:03.273755+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:36:07.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2024-01-27T02:36:07.127 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.077155+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T12:11:23.338931+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:36:07.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2024-01-27T02:36:07.127 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.080121+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:18:04.141154+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:36:07.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2024-01-27T02:36:07.127 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.078206+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T03:03:56.898146+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:36:07.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2024-01-27T02:36:07.127 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.077264+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T12:34:53.377109+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:36:07.127 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2024-01-27T02:36:07.128 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:36:01.063602+0000', 'last_change': '2024-01-27T02:36:01.070886+0000', 'last_active': '2024-01-27T02:36:01.063602+0000', 'last_peered': '2024-01-27T02:36:01.063602+0000', 'last_clean': '2024-01-27T02:36:01.063602+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:36:01.063602+0000', 'last_undegraded': '2024-01-27T02:36:01.063602+0000', 'last_fullsized': '2024-01-27T02:36:01.063602+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 79, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'last_clean_scrub_stamp': '2024-01-27T02:36:01.063602+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T08:09:00.736774+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:36:07.128 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-01-27T02:36:07.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.284+0000 7f388a905700 1 Processor -- start 2024-01-27T02:36:07.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.284+0000 7f388a905700 1 -- start start 2024-01-27T02:36:07.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.285+0000 7f388a905700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f38840cd120 0x7f388415b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:07.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.285+0000 7f388a905700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3884063c90 0x7f38840cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:07.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.285+0000 7f388a905700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38840cb7c0 con 0x7f388406e2a0 2024-01-27T02:36:07.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.285+0000 7f388a905700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38840cda30 con 0x7f38840cd120 2024-01-27T02:36:07.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.285+0000 7f388a905700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388406e740 con 0x7f3884063c90 2024-01-27T02:36:07.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.286+0000 7f38837fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f388406e2a0 0x7f3884063580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40668/0 (socket says 172.21.15.7:40668) 2024-01-27T02:36:07.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.286+0000 7f38837fe700 1 -- 172.21.15.7:0/2119311282 learned_addr learned my addr 172.21.15.7:0/2119311282 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:07.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.286+0000 7f3888ea2700 1 --2- 172.21.15.7:0/2119311282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3884063c90 0x7f38840cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:07.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.286+0000 7f3883fff700 1 --2- 172.21.15.7:0/2119311282 >> v2:172.21.15.188:3300/0 conn(0x7f38840cd120 0x7f388415b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:07.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.286+0000 7f3882ffd700 1 -- 172.21.15.7:0/2119311282 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1337325356 0 0) 0x7f38840cb7c0 con 0x7f388406e2a0 2024-01-27T02:36:07.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.286+0000 7f3882ffd700 1 -- 172.21.15.7:0/2119311282 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f386c003570 con 0x7f388406e2a0 2024-01-27T02:36:07.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.286+0000 7f3882ffd700 1 -- 172.21.15.7:0/2119311282 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3069060244 0 0) 0x7f386c003570 con 0x7f388406e2a0 2024-01-27T02:36:07.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.286+0000 7f3882ffd700 1 -- 172.21.15.7:0/2119311282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3884063c90 msgr2=0x7f38840cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.286+0000 7f3882ffd700 1 --2- 172.21.15.7:0/2119311282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3884063c90 0x7f38840cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.286+0000 7f3882ffd700 1 -- 172.21.15.7:0/2119311282 >> v2:172.21.15.188:3300/0 conn(0x7f38840cd120 msgr2=0x7f388415b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.286+0000 7f3882ffd700 1 --2- 172.21.15.7:0/2119311282 >> v2:172.21.15.188:3300/0 conn(0x7f38840cd120 0x7f388415b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.286+0000 7f3882ffd700 1 -- 172.21.15.7:0/2119311282 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388406e9f0 con 0x7f388406e2a0 2024-01-27T02:36:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.286+0000 7f3882ffd700 1 -- 172.21.15.7:0/2119311282 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f3878001470 con 0x7f388406e2a0 2024-01-27T02:36:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.286+0000 7f3882ffd700 1 -- 172.21.15.7:0/2119311282 >> v1:172.21.15.7:6789/0 conn(0x7f388406e2a0 legacy=0x7f3884063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.286+0000 7f3882ffd700 1 --2- 172.21.15.7:0/2119311282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f386c003a60 0x7f386c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.286+0000 7f3882ffd700 1 --2- 172.21.15.7:0/2119311282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f386c004810 0x7f386c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.286+0000 7f3882ffd700 1 --2- 172.21.15.7:0/2119311282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3884063c90 0x7f386c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.286+0000 7f3882ffd700 1 -- 172.21.15.7:0/2119311282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38840cda30 con 0x7f386c003a60 2024-01-27T02:36:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.286+0000 7f3882ffd700 1 -- 172.21.15.7:0/2119311282 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388406e740 con 0x7f386c004810 2024-01-27T02:36:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.286+0000 7f3882ffd700 1 -- 172.21.15.7:0/2119311282 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386c00ba60 con 0x7f3884063c90 2024-01-27T02:36:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.286+0000 7f3883fff700 1 --2- 172.21.15.7:0/2119311282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f386c003a60 0x7f386c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.287+0000 7f38837fe700 1 --2- 172.21.15.7:0/2119311282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f386c004810 0x7f386c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.287+0000 7f3888ea2700 1 --2- 172.21.15.7:0/2119311282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3884063c90 0x7f386c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.287+0000 7f3883fff700 1 -- 172.21.15.7:0/2119311282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3884063c90 msgr2=0x7f386c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.287+0000 7f3883fff700 1 --2- 172.21.15.7:0/2119311282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3884063c90 0x7f386c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.287+0000 7f3883fff700 1 -- 172.21.15.7:0/2119311282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f386c004810 msgr2=0x7f386c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.287+0000 7f3883fff700 1 --2- 172.21.15.7:0/2119311282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f386c004810 0x7f386c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.287+0000 7f3883fff700 1 -- 172.21.15.7:0/2119311282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386c003570 con 0x7f386c003a60 2024-01-27T02:36:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.287+0000 7f3883fff700 1 --2- 172.21.15.7:0/2119311282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f386c003a60 0x7f386c003e70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3874009c20 tx=0x7f3874009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36208e8a4e8e59c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.287+0000 7f3882ffd700 1 -- 172.21.15.7:0/2119311282 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3874005a00 con 0x7f386c003a60 2024-01-27T02:36:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.287+0000 7f3882ffd700 1 -- 172.21.15.7:0/2119311282 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3874005b60 con 0x7f386c003a60 2024-01-27T02:36:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.287+0000 7f3882ffd700 1 -- 172.21.15.7:0/2119311282 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387400e710 con 0x7f386c003a60 2024-01-27T02:36:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.288+0000 7f388a905700 1 -- 172.21.15.7:0/2119311282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f386c003a60 msgr2=0x7f386c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.288+0000 7f388a905700 1 --2- 172.21.15.7:0/2119311282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f386c003a60 0x7f386c003e70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3874009c20 tx=0x7f3874009120 comp rx=0 tx=0).stop 2024-01-27T02:36:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.288+0000 7f3888ea2700 1 -- 172.21.15.7:0/2119311282 reap_dead start 2024-01-27T02:36:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.288+0000 7f388a905700 1 -- 172.21.15.7:0/2119311282 shutdown_connections 2024-01-27T02:36:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.288+0000 7f388a905700 1 -- 172.21.15.7:0/2119311282 >> 172.21.15.7:0/2119311282 conn(0x7f38840cdbf0 msgr2=0x7f38840cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.288+0000 7f388a905700 1 -- 172.21.15.7:0/2119311282 shutdown_connections 2024-01-27T02:36:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.288+0000 7f388a905700 1 -- 172.21.15.7:0/2119311282 wait complete. 2024-01-27T02:36:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.289+0000 7f388a905700 1 Processor -- start 2024-01-27T02:36:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.289+0000 7f388a905700 1 -- start start 2024-01-27T02:36:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.290+0000 7f388a905700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f386c003a60 0x7f388414fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.290+0000 7f388a905700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38841534a0 0x7f388415e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.290+0000 7f388a905700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3884153910 0x7f38840add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.290+0000 7f388a905700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38841502e0 con 0x7f38841534a0 2024-01-27T02:36:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.290+0000 7f388a905700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3884153010 con 0x7f3884153910 2024-01-27T02:36:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.290+0000 7f388a905700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3884152120 con 0x7f386c003a60 2024-01-27T02:36:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.290+0000 7f38837fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38841534a0 0x7f388415e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.290+0000 7f38837fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38841534a0 0x7f388415e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33522/0 (socket says 172.21.15.7:33522) 2024-01-27T02:36:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.290+0000 7f38837fe700 1 -- 172.21.15.7:0/3717249727 learned_addr learned my addr 172.21.15.7:0/3717249727 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.290+0000 7f3883fff700 1 --2- 172.21.15.7:0/3717249727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f386c003a60 0x7f388414fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.290+0000 7f3888ea2700 1 --2- 172.21.15.7:0/3717249727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3884153910 0x7f38840add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.290+0000 7f38837fe700 1 -- 172.21.15.7:0/3717249727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f386c003a60 msgr2=0x7f388414fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.290+0000 7f38837fe700 1 --2- 172.21.15.7:0/3717249727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f386c003a60 0x7f388414fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.290+0000 7f38837fe700 1 -- 172.21.15.7:0/3717249727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3884153910 msgr2=0x7f38840add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:07.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.290+0000 7f38837fe700 1 --2- 172.21.15.7:0/3717249727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3884153910 0x7f38840add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.290+0000 7f38837fe700 1 -- 172.21.15.7:0/3717249727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3884152400 con 0x7f38841534a0 2024-01-27T02:36:07.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.290+0000 7f3883fff700 1 --2- 172.21.15.7:0/3717249727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f386c003a60 0x7f388414fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:36:07.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.290+0000 7f3888ea2700 1 --2- 172.21.15.7:0/3717249727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3884153910 0x7f38840add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:36:07.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.290+0000 7f38837fe700 1 --2- 172.21.15.7:0/3717249727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38841534a0 0x7f388415e290 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f387800eb90 tx=0x7f3878000d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:07.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.291+0000 7f386bfff700 1 -- 172.21.15.7:0/3717249727 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3878005e70 con 0x7f38841534a0 2024-01-27T02:36:07.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.291+0000 7f386bfff700 1 -- 172.21.15.7:0/3717249727 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387800c7e0 con 0x7f38841534a0 2024-01-27T02:36:07.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.291+0000 7f388a905700 1 -- 172.21.15.7:0/3717249727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38841526e0 con 0x7f38841534a0 2024-01-27T02:36:07.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.291+0000 7f386bfff700 1 -- 172.21.15.7:0/3717249727 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387800cbc0 con 0x7f38841534a0 2024-01-27T02:36:07.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.291+0000 7f388a905700 1 -- 172.21.15.7:0/3717249727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3884152ca0 con 0x7f38841534a0 2024-01-27T02:36:07.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.292+0000 7f388a905700 1 -- 172.21.15.7:0/3717249727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f388404c710 con 0x7f38841534a0 2024-01-27T02:36:07.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.293+0000 7f386bfff700 1 -- 172.21.15.7:0/3717249727 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f387801d460 con 0x7f38841534a0 2024-01-27T02:36:07.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.293+0000 7f386bfff700 1 --2- 172.21.15.7:0/3717249727 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3870066f30 0x7f38700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:07.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.293+0000 7f386bfff700 1 -- 172.21.15.7:0/3717249727 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3878084f40 con 0x7f38841534a0 2024-01-27T02:36:07.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.293+0000 7f3883fff700 1 --2- 172.21.15.7:0/3717249727 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3870066f30 0x7f38700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:07.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.294+0000 7f3883fff700 1 --2- 172.21.15.7:0/3717249727 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3870066f30 0x7f38700693e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f387400a010 tx=0x7f3874019000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:07.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.297+0000 7f386bfff700 1 -- 172.21.15.7:0/3717249727 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3878052230 con 0x7f38841534a0 2024-01-27T02:36:07.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.585+0000 7f388a905700 1 -- 172.21.15.7:0/3717249727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f388406ecd0 con 0x7f38841534a0 2024-01-27T02:36:07.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.585+0000 7f386bfff700 1 -- 172.21.15.7:0/3717249727 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1029 (secure 0 0 0) 0x7f3878055850 con 0x7f38841534a0 2024-01-27T02:36:07.586 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:36:07.586 INFO:teuthology.orchestra.run.smithi007.stdout:{"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":257,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":81,"num_osds":8,"num_up_osds":8,"osd_up_since":1706322715,"num_in_osds":8,"osd_in_since":1706322712,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":4447749,"bytes_avail":1595552251,"bytes_total":1600000000,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":42,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"d","status":"up:active","gid":5946}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-01-27T02:35:21.620383+0000","services":{}},"progress_events":{}} 2024-01-27T02:36:07.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.588+0000 7f388a905700 1 -- 172.21.15.7:0/3717249727 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3870066f30 msgr2=0x7f38700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:07.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.588+0000 7f388a905700 1 --2- 172.21.15.7:0/3717249727 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3870066f30 0x7f38700693e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f387400a010 tx=0x7f3874019000 comp rx=0 tx=0).stop 2024-01-27T02:36:07.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.588+0000 7f388a905700 1 -- 172.21.15.7:0/3717249727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38841534a0 msgr2=0x7f388415e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:07.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.588+0000 7f388a905700 1 --2- 172.21.15.7:0/3717249727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38841534a0 0x7f388415e290 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f387800eb90 tx=0x7f3878000d00 comp rx=0 tx=0).stop 2024-01-27T02:36:07.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.588+0000 7f388a905700 1 -- 172.21.15.7:0/3717249727 shutdown_connections 2024-01-27T02:36:07.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.588+0000 7f388a905700 1 --2- 172.21.15.7:0/3717249727 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3870066f30 0x7f38700693e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.588+0000 7f388a905700 1 --2- 172.21.15.7:0/3717249727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3884153910 0x7f38840add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.588+0000 7f388a905700 1 --2- 172.21.15.7:0/3717249727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f386c003a60 0x7f388414fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.588+0000 7f388a905700 1 --2- 172.21.15.7:0/3717249727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38841534a0 0x7f388415e290 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.588+0000 7f388a905700 1 -- 172.21.15.7:0/3717249727 >> 172.21.15.7:0/3717249727 conn(0x7f38840cdbf0 msgr2=0x7f38840cc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:07.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.588+0000 7f388a905700 1 -- 172.21.15.7:0/3717249727 shutdown_connections 2024-01-27T02:36:07.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.588+0000 7f388a905700 1 -- 172.21.15.7:0/3717249727 wait complete. 2024-01-27T02:36:07.602 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-01-27T02:36:07.602 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:36:07.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.734+0000 7faa2cf9f700 1 Processor -- start 2024-01-27T02:36:07.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.734+0000 7faa2cf9f700 1 -- start start 2024-01-27T02:36:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.735+0000 7faa2cf9f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa2815a310 0x7faa280aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.735+0000 7faa2cf9f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faa280aadf0 0x7faa28063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.735+0000 7faa2cf9f700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa280c30c0 con 0x7faa28159f40 2024-01-27T02:36:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.735+0000 7faa2cf9f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa280c4550 con 0x7faa280aadf0 2024-01-27T02:36:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.735+0000 7faa2cf9f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa280ab270 con 0x7faa2815a310 2024-01-27T02:36:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.735+0000 7faa25d9b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa2815a310 0x7faa280aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.735+0000 7faa26d9d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faa280aadf0 0x7faa28063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.735+0000 7faa25d9b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa2815a310 0x7faa280aa8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:56030/0 (socket says 172.21.15.7:56030) 2024-01-27T02:36:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.735+0000 7faa26d9d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faa280aadf0 0x7faa28063580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:39296/0 (socket says 172.21.15.7:39296) 2024-01-27T02:36:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.735+0000 7faa25d9b700 1 -- 172.21.15.7:0/711016933 learned_addr learned my addr 172.21.15.7:0/711016933 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.735+0000 7faa25d9b700 1 -- 172.21.15.7:0/711016933 >> v2:172.21.15.188:3300/0 conn(0x7faa280aadf0 msgr2=0x7faa28063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.735+0000 7faa25d9b700 1 --2- 172.21.15.7:0/711016933 >> v2:172.21.15.188:3300/0 conn(0x7faa280aadf0 0x7faa28063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.735+0000 7faa25d9b700 1 -- 172.21.15.7:0/711016933 >> v1:172.21.15.188:6789/0 conn(0x7faa28159f40 legacy=0x7faa280c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.735+0000 7faa25d9b700 1 -- 172.21.15.7:0/711016933 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2815e960 con 0x7faa2815a310 2024-01-27T02:36:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.735+0000 7faa26d9d700 1 --2- 172.21.15.7:0/711016933 >> v2:172.21.15.188:3300/0 conn(0x7faa280aadf0 0x7faa28063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:36:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.736+0000 7faa25d9b700 1 --2- 172.21.15.7:0/711016933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa2815a310 0x7faa280aa8b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0099b0 tx=0x7faa1c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99e8750a71189df3 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.736+0000 7faa2559a700 1 -- 172.21.15.7:0/711016933 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c009ea0 con 0x7faa2815a310 2024-01-27T02:36:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.736+0000 7faa2559a700 1 -- 172.21.15.7:0/711016933 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1c00b860 con 0x7faa2815a310 2024-01-27T02:36:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.736+0000 7faa2559a700 1 -- 172.21.15.7:0/711016933 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c00bc00 con 0x7faa2815a310 2024-01-27T02:36:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.736+0000 7faa2cf9f700 1 -- 172.21.15.7:0/711016933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa2815a310 msgr2=0x7faa280aa8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.736+0000 7faa2cf9f700 1 --2- 172.21.15.7:0/711016933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa2815a310 0x7faa280aa8b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0099b0 tx=0x7faa1c009250 comp rx=0 tx=0).stop 2024-01-27T02:36:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.737+0000 7faa2cf9f700 1 -- 172.21.15.7:0/711016933 shutdown_connections 2024-01-27T02:36:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.737+0000 7faa2cf9f700 1 --2- 172.21.15.7:0/711016933 >> v2:172.21.15.188:3300/0 conn(0x7faa280aadf0 0x7faa28063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.737+0000 7faa2cf9f700 1 --2- 172.21.15.7:0/711016933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa2815a310 0x7faa280aa8b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.737+0000 7faa2cf9f700 1 -- 172.21.15.7:0/711016933 >> 172.21.15.7:0/711016933 conn(0x7faa280cdc30 msgr2=0x7faa280ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:07.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.737+0000 7faa2cf9f700 1 -- 172.21.15.7:0/711016933 shutdown_connections 2024-01-27T02:36:07.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.737+0000 7faa2cf9f700 1 -- 172.21.15.7:0/711016933 wait complete. 2024-01-27T02:36:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.738+0000 7faa2cf9f700 1 Processor -- start 2024-01-27T02:36:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.738+0000 7faa2cf9f700 1 -- start start 2024-01-27T02:36:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.739+0000 7faa2cf9f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa280aadf0 0x7faa2815d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.739+0000 7faa2cf9f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa28159f40 0x7faa280c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.739+0000 7faa2cf9f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa2815a310 0x7faa280c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.739+0000 7faa2cf9f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2815b120 con 0x7faa280aadf0 2024-01-27T02:36:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.739+0000 7faa2cf9f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2815a860 con 0x7faa2815a310 2024-01-27T02:36:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.739+0000 7faa2cf9f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2815aad0 con 0x7faa28159f40 2024-01-27T02:36:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.739+0000 7faa2659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa280aadf0 0x7faa2815d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.739+0000 7faa2659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa280aadf0 0x7faa2815d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33530/0 (socket says 172.21.15.7:33530) 2024-01-27T02:36:07.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.739+0000 7faa2659c700 1 -- 172.21.15.7:0/2554509404 learned_addr learned my addr 172.21.15.7:0/2554509404 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:07.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.739+0000 7faa25d9b700 1 --2- 172.21.15.7:0/2554509404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa28159f40 0x7faa280c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:07.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.739+0000 7faa26d9d700 1 --2- 172.21.15.7:0/2554509404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa2815a310 0x7faa280c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:07.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.739+0000 7faa2659c700 1 -- 172.21.15.7:0/2554509404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa28159f40 msgr2=0x7faa280c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:07.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.739+0000 7faa2659c700 1 --2- 172.21.15.7:0/2554509404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa28159f40 0x7faa280c7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.739+0000 7faa2659c700 1 -- 172.21.15.7:0/2554509404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa2815a310 msgr2=0x7faa280c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:07.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.739+0000 7faa2659c700 1 --2- 172.21.15.7:0/2554509404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa2815a310 0x7faa280c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.739+0000 7faa2659c700 1 -- 172.21.15.7:0/2554509404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa20009a20 con 0x7faa280aadf0 2024-01-27T02:36:07.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.739+0000 7faa26d9d700 1 --2- 172.21.15.7:0/2554509404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa2815a310 0x7faa280c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:36:07.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.739+0000 7faa2659c700 1 --2- 172.21.15.7:0/2554509404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa280aadf0 0x7faa2815d2a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7faa1800c5c0 tx=0x7faa1800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:07.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.739+0000 7faa167fc700 1 -- 172.21.15.7:0/2554509404 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa18011590 con 0x7faa280aadf0 2024-01-27T02:36:07.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.739+0000 7faa167fc700 1 -- 172.21.15.7:0/2554509404 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa18011710 con 0x7faa280aadf0 2024-01-27T02:36:07.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.739+0000 7faa2cf9f700 1 -- 172.21.15.7:0/2554509404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1c0094f0 con 0x7faa280aadf0 2024-01-27T02:36:07.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.740+0000 7faa167fc700 1 -- 172.21.15.7:0/2554509404 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa18011ad0 con 0x7faa280aadf0 2024-01-27T02:36:07.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.740+0000 7faa2cf9f700 1 -- 172.21.15.7:0/2554509404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa280af4a0 con 0x7faa280aadf0 2024-01-27T02:36:07.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.741+0000 7faa2cf9f700 1 -- 172.21.15.7:0/2554509404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa2804c710 con 0x7faa280aadf0 2024-01-27T02:36:07.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.741+0000 7faa167fc700 1 -- 172.21.15.7:0/2554509404 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7faa1801b3b0 con 0x7faa280aadf0 2024-01-27T02:36:07.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.741+0000 7faa167fc700 1 --2- 172.21.15.7:0/2554509404 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faa08066f30 0x7faa080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:07.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.741+0000 7faa167fc700 1 -- 172.21.15.7:0/2554509404 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faa18083ad0 con 0x7faa280aadf0 2024-01-27T02:36:07.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.742+0000 7faa25d9b700 1 --2- 172.21.15.7:0/2554509404 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faa08066f30 0x7faa080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:07.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.742+0000 7faa25d9b700 1 --2- 172.21.15.7:0/2554509404 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faa08066f30 0x7faa080693e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0099b0 tx=0x7faa1c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:07.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.746+0000 7faa167fc700 1 -- 172.21.15.7:0/2554509404 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa18050d40 con 0x7faa280aadf0 2024-01-27T02:36:07.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.944+0000 7faa2cf9f700 1 -- 172.21.15.7:0/2554509404 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faa2806f770 con 0x7faa08066f30 2024-01-27T02:36:07.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.952+0000 7faa167fc700 1 -- 172.21.15.7:0/2554509404 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285449 (secure 0 0 0) 0x7faa2806f770 con 0x7faa08066f30 2024-01-27T02:36:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:36:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.956+0000 7faa2cf9f700 1 -- 172.21.15.7:0/2554509404 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faa08066f30 msgr2=0x7faa080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.956+0000 7faa2cf9f700 1 --2- 172.21.15.7:0/2554509404 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faa08066f30 0x7faa080693e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0099b0 tx=0x7faa1c017000 comp rx=0 tx=0).stop 2024-01-27T02:36:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.956+0000 7faa2cf9f700 1 -- 172.21.15.7:0/2554509404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa280aadf0 msgr2=0x7faa2815d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.956+0000 7faa2cf9f700 1 --2- 172.21.15.7:0/2554509404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa280aadf0 0x7faa2815d2a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7faa1800c5c0 tx=0x7faa1800c980 comp rx=0 tx=0).stop 2024-01-27T02:36:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.956+0000 7faa2cf9f700 1 -- 172.21.15.7:0/2554509404 shutdown_connections 2024-01-27T02:36:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.956+0000 7faa2cf9f700 1 --2- 172.21.15.7:0/2554509404 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faa08066f30 0x7faa080693e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.956+0000 7faa2cf9f700 1 --2- 172.21.15.7:0/2554509404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa2815a310 0x7faa280c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.956+0000 7faa2cf9f700 1 --2- 172.21.15.7:0/2554509404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa280aadf0 0x7faa2815d2a0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.956+0000 7faa2cf9f700 1 --2- 172.21.15.7:0/2554509404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa28159f40 0x7faa280c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:07.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.956+0000 7faa2cf9f700 1 -- 172.21.15.7:0/2554509404 >> 172.21.15.7:0/2554509404 conn(0x7faa280cdc30 msgr2=0x7faa2806edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:07.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.956+0000 7faa2cf9f700 1 -- 172.21.15.7:0/2554509404 shutdown_connections 2024-01-27T02:36:07.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:07.956+0000 7faa2cf9f700 1 -- 172.21.15.7:0/2554509404 wait complete. 2024-01-27T02:36:07.960 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:36:07.971 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-01-27T02:36:07.632020+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4480,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558019,"statfs":{"total":1600000000,"available":1595411916,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.557969"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.099700+0000","last_change":"2024-01-27T02:36:02.098841+0000","last_active":"2024-01-27T02:36:02.099700+0000","last_peered":"2024-01-27T02:36:02.099700+0000","last_clean":"2024-01-27T02:36:02.099700+0000","last_became_active":"2024-01-27T02:36:02.098660+0000","last_became_peered":"2024-01-27T02:36:02.098660+0000","last_unstale":"2024-01-27T02:36:02.099700+0000","last_undegraded":"2024-01-27T02:36:02.099700+0000","last_fullsized":"2024-01-27T02:36:02.099700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:47:18.259755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.096517+0000","last_change":"2024-01-27T02:36:02.096444+0000","last_active":"2024-01-27T02:36:02.096517+0000","last_peered":"2024-01-27T02:36:02.096517+0000","last_clean":"2024-01-27T02:36:02.096517+0000","last_became_active":"2024-01-27T02:36:02.096187+0000","last_became_peered":"2024-01-27T02:36:02.096187+0000","last_unstale":"2024-01-27T02:36:02.096517+0000","last_undegraded":"2024-01-27T02:36:02.096517+0000","last_fullsized":"2024-01-27T02:36:02.096517+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:42.090153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.095761+0000","last_change":"2024-01-27T02:36:02.095654+0000","last_active":"2024-01-27T02:36:02.095761+0000","last_peered":"2024-01-27T02:36:02.095761+0000","last_clean":"2024-01-27T02:36:02.095761+0000","last_became_active":"2024-01-27T02:36:02.094876+0000","last_became_peered":"2024-01-27T02:36:02.094876+0000","last_unstale":"2024-01-27T02:36:02.095761+0000","last_undegraded":"2024-01-27T02:36:02.095761+0000","last_fullsized":"2024-01-27T02:36:02.095761+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:46:46.374799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.093079+0000","last_change":"2024-01-27T02:36:02.093003+0000","last_active":"2024-01-27T02:36:02.093079+0000","last_peered":"2024-01-27T02:36:02.093079+0000","last_clean":"2024-01-27T02:36:02.093079+0000","last_became_active":"2024-01-27T02:36:02.092682+0000","last_became_peered":"2024-01-27T02:36:02.092682+0000","last_unstale":"2024-01-27T02:36:02.093079+0000","last_undegraded":"2024-01-27T02:36:02.093079+0000","last_fullsized":"2024-01-27T02:36:02.093079+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:07:20.991016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.101504+0000","last_change":"2024-01-27T02:36:02.101043+0000","last_active":"2024-01-27T02:36:02.101504+0000","last_peered":"2024-01-27T02:36:02.101504+0000","last_clean":"2024-01-27T02:36:02.101504+0000","last_became_active":"2024-01-27T02:36:02.100405+0000","last_became_peered":"2024-01-27T02:36:02.100405+0000","last_unstale":"2024-01-27T02:36:02.101504+0000","last_undegraded":"2024-01-27T02:36:02.101504+0000","last_fullsized":"2024-01-27T02:36:02.101504+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:54:00.073605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.098495+0000","last_change":"2024-01-27T02:36:02.098420+0000","last_active":"2024-01-27T02:36:02.098495+0000","last_peered":"2024-01-27T02:36:02.098495+0000","last_clean":"2024-01-27T02:36:02.098495+0000","last_became_active":"2024-01-27T02:36:02.097919+0000","last_became_peered":"2024-01-27T02:36:02.097919+0000","last_unstale":"2024-01-27T02:36:02.098495+0000","last_undegraded":"2024-01-27T02:36:02.098495+0000","last_fullsized":"2024-01-27T02:36:02.098495+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:16.279028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.101173+0000","last_change":"2024-01-27T02:36:02.101093+0000","last_active":"2024-01-27T02:36:02.101173+0000","last_peered":"2024-01-27T02:36:02.101173+0000","last_clean":"2024-01-27T02:36:02.101173+0000","last_became_active":"2024-01-27T02:36:02.100737+0000","last_became_peered":"2024-01-27T02:36:02.100737+0000","last_unstale":"2024-01-27T02:36:02.101173+0000","last_undegraded":"2024-01-27T02:36:02.101173+0000","last_fullsized":"2024-01-27T02:36:02.101173+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:56:33.979247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.094681+0000","last_change":"2024-01-27T02:36:02.093938+0000","last_active":"2024-01-27T02:36:02.094681+0000","last_peered":"2024-01-27T02:36:02.094681+0000","last_clean":"2024-01-27T02:36:02.094681+0000","last_became_active":"2024-01-27T02:36:02.093239+0000","last_became_peered":"2024-01-27T02:36:02.093239+0000","last_unstale":"2024-01-27T02:36:02.094681+0000","last_undegraded":"2024-01-27T02:36:02.094681+0000","last_fullsized":"2024-01-27T02:36:02.094681+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:10:52.422378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.096863+0000","last_change":"2024-01-27T02:36:02.096790+0000","last_active":"2024-01-27T02:36:02.096863+0000","last_peered":"2024-01-27T02:36:02.096863+0000","last_clean":"2024-01-27T02:36:02.096863+0000","last_became_active":"2024-01-27T02:36:02.096488+0000","last_became_peered":"2024-01-27T02:36:02.096488+0000","last_unstale":"2024-01-27T02:36:02.096863+0000","last_undegraded":"2024-01-27T02:36:02.096863+0000","last_fullsized":"2024-01-27T02:36:02.096863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:11.427024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.103462+0000","last_change":"2024-01-27T02:36:02.102823+0000","last_active":"2024-01-27T02:36:02.103462+0000","last_peered":"2024-01-27T02:36:02.103462+0000","last_clean":"2024-01-27T02:36:02.103462+0000","last_became_active":"2024-01-27T02:36:02.102183+0000","last_became_peered":"2024-01-27T02:36:02.102183+0000","last_unstale":"2024-01-27T02:36:02.103462+0000","last_undegraded":"2024-01-27T02:36:02.103462+0000","last_fullsized":"2024-01-27T02:36:02.103462+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:26:17.309915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.099658+0000","last_change":"2024-01-27T02:36:02.098268+0000","last_active":"2024-01-27T02:36:02.099658+0000","last_peered":"2024-01-27T02:36:02.099658+0000","last_clean":"2024-01-27T02:36:02.099658+0000","last_became_active":"2024-01-27T02:36:02.097955+0000","last_became_peered":"2024-01-27T02:36:02.097955+0000","last_unstale":"2024-01-27T02:36:02.099658+0000","last_undegraded":"2024-01-27T02:36:02.099658+0000","last_fullsized":"2024-01-27T02:36:02.099658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:54:59.718581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.098980+0000","last_change":"2024-01-27T02:36:02.098873+0000","last_active":"2024-01-27T02:36:02.098980+0000","last_peered":"2024-01-27T02:36:02.098980+0000","last_clean":"2024-01-27T02:36:02.098980+0000","last_became_active":"2024-01-27T02:36:02.098497+0000","last_became_peered":"2024-01-27T02:36:02.098497+0000","last_unstale":"2024-01-27T02:36:02.098980+0000","last_undegraded":"2024-01-27T02:36:02.098980+0000","last_fullsized":"2024-01-27T02:36:02.098980+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:12:41.850276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.097417+0000","last_change":"2024-01-27T02:36:02.097351+0000","last_active":"2024-01-27T02:36:02.097417+0000","last_peered":"2024-01-27T02:36:02.097417+0000","last_clean":"2024-01-27T02:36:02.097417+0000","last_became_active":"2024-01-27T02:36:02.096406+0000","last_became_peered":"2024-01-27T02:36:02.096406+0000","last_unstale":"2024-01-27T02:36:02.097417+0000","last_undegraded":"2024-01-27T02:36:02.097417+0000","last_fullsized":"2024-01-27T02:36:02.097417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:57:20.844278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.094795+0000","last_change":"2024-01-27T02:36:02.094721+0000","last_active":"2024-01-27T02:36:02.094795+0000","last_peered":"2024-01-27T02:36:02.094795+0000","last_clean":"2024-01-27T02:36:02.094795+0000","last_became_active":"2024-01-27T02:36:02.094463+0000","last_became_peered":"2024-01-27T02:36:02.094463+0000","last_unstale":"2024-01-27T02:36:02.094795+0000","last_undegraded":"2024-01-27T02:36:02.094795+0000","last_fullsized":"2024-01-27T02:36:02.094795+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:37:23.189235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.095468+0000","last_change":"2024-01-27T02:36:02.095410+0000","last_active":"2024-01-27T02:36:02.095468+0000","last_peered":"2024-01-27T02:36:02.095468+0000","last_clean":"2024-01-27T02:36:02.095468+0000","last_became_active":"2024-01-27T02:36:02.095027+0000","last_became_peered":"2024-01-27T02:36:02.095027+0000","last_unstale":"2024-01-27T02:36:02.095468+0000","last_undegraded":"2024-01-27T02:36:02.095468+0000","last_fullsized":"2024-01-27T02:36:02.095468+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:14:13.651138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.100950+0000","last_change":"2024-01-27T02:36:02.100887+0000","last_active":"2024-01-27T02:36:02.100950+0000","last_peered":"2024-01-27T02:36:02.100950+0000","last_clean":"2024-01-27T02:36:02.100950+0000","last_became_active":"2024-01-27T02:36:02.100554+0000","last_became_peered":"2024-01-27T02:36:02.100554+0000","last_unstale":"2024-01-27T02:36:02.100950+0000","last_undegraded":"2024-01-27T02:36:02.100950+0000","last_fullsized":"2024-01-27T02:36:02.100950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:21:28.748243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.101558+0000","last_change":"2024-01-27T02:36:02.100548+0000","last_active":"2024-01-27T02:36:02.101558+0000","last_peered":"2024-01-27T02:36:02.101558+0000","last_clean":"2024-01-27T02:36:02.101558+0000","last_became_active":"2024-01-27T02:36:02.099461+0000","last_became_peered":"2024-01-27T02:36:02.099461+0000","last_unstale":"2024-01-27T02:36:02.101558+0000","last_undegraded":"2024-01-27T02:36:02.101558+0000","last_fullsized":"2024-01-27T02:36:02.101558+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:07:58.814010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.094832+0000","last_change":"2024-01-27T02:36:02.093803+0000","last_active":"2024-01-27T02:36:02.094832+0000","last_peered":"2024-01-27T02:36:02.094832+0000","last_clean":"2024-01-27T02:36:02.094832+0000","last_became_active":"2024-01-27T02:36:02.093304+0000","last_became_peered":"2024-01-27T02:36:02.093304+0000","last_unstale":"2024-01-27T02:36:02.094832+0000","last_undegraded":"2024-01-27T02:36:02.094832+0000","last_fullsized":"2024-01-27T02:36:02.094832+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:35:14.198571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.090855+0000","last_change":"2024-01-27T02:36:02.090798+0000","last_active":"2024-01-27T02:36:02.090855+0000","last_peered":"2024-01-27T02:36:02.090855+0000","last_clean":"2024-01-27T02:36:02.090855+0000","last_became_active":"2024-01-27T02:36:02.090569+0000","last_became_peered":"2024-01-27T02:36:02.090569+0000","last_unstale":"2024-01-27T02:36:02.090855+0000","last_undegraded":"2024-01-27T02:36:02.090855+0000","last_fullsized":"2024-01-27T02:36:02.090855+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:21:41.100136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.095382+0000","last_change":"2024-01-27T02:36:02.095258+0000","last_active":"2024-01-27T02:36:02.095382+0000","last_peered":"2024-01-27T02:36:02.095382+0000","last_clean":"2024-01-27T02:36:02.095382+0000","last_became_active":"2024-01-27T02:36:02.094714+0000","last_became_peered":"2024-01-27T02:36:02.094714+0000","last_unstale":"2024-01-27T02:36:02.095382+0000","last_undegraded":"2024-01-27T02:36:02.095382+0000","last_fullsized":"2024-01-27T02:36:02.095382+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:32:03.273755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.099797+0000","last_change":"2024-01-27T02:36:02.099691+0000","last_active":"2024-01-27T02:36:02.099797+0000","last_peered":"2024-01-27T02:36:02.099797+0000","last_clean":"2024-01-27T02:36:02.099797+0000","last_became_active":"2024-01-27T02:36:02.099281+0000","last_became_peered":"2024-01-27T02:36:02.099281+0000","last_unstale":"2024-01-27T02:36:02.099797+0000","last_undegraded":"2024-01-27T02:36:02.099797+0000","last_fullsized":"2024-01-27T02:36:02.099797+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:32:58.452630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.096155+0000","last_change":"2024-01-27T02:36:02.096081+0000","last_active":"2024-01-27T02:36:02.096155+0000","last_peered":"2024-01-27T02:36:02.096155+0000","last_clean":"2024-01-27T02:36:02.096155+0000","last_became_active":"2024-01-27T02:36:02.095770+0000","last_became_peered":"2024-01-27T02:36:02.095770+0000","last_unstale":"2024-01-27T02:36:02.096155+0000","last_undegraded":"2024-01-27T02:36:02.096155+0000","last_fullsized":"2024-01-27T02:36:02.096155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:16.279028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.099677+0000","last_change":"2024-01-27T02:36:02.093040+0000","last_active":"2024-01-27T02:36:02.099677+0000","last_peered":"2024-01-27T02:36:02.099677+0000","last_clean":"2024-01-27T02:36:02.099677+0000","last_became_active":"2024-01-27T02:36:02.092769+0000","last_became_peered":"2024-01-27T02:36:02.092769+0000","last_unstale":"2024-01-27T02:36:02.099677+0000","last_undegraded":"2024-01-27T02:36:02.099677+0000","last_fullsized":"2024-01-27T02:36:02.099677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:42:07.113277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.105510+0000","last_change":"2024-01-27T02:36:02.105443+0000","last_active":"2024-01-27T02:36:02.105510+0000","last_peered":"2024-01-27T02:36:02.105510+0000","last_clean":"2024-01-27T02:36:02.105510+0000","last_became_active":"2024-01-27T02:36:02.105056+0000","last_became_peered":"2024-01-27T02:36:02.105056+0000","last_unstale":"2024-01-27T02:36:02.105510+0000","last_undegraded":"2024-01-27T02:36:02.105510+0000","last_fullsized":"2024-01-27T02:36:02.105510+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:22:09.618511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.103480+0000","last_change":"2024-01-27T02:36:02.103171+0000","last_active":"2024-01-27T02:36:02.103480+0000","last_peered":"2024-01-27T02:36:02.103480+0000","last_clean":"2024-01-27T02:36:02.103480+0000","last_became_active":"2024-01-27T02:36:02.102570+0000","last_became_peered":"2024-01-27T02:36:02.102570+0000","last_unstale":"2024-01-27T02:36:02.103480+0000","last_undegraded":"2024-01-27T02:36:02.103480+0000","last_fullsized":"2024-01-27T02:36:02.103480+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:05:16.228857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.096047+0000","last_change":"2024-01-27T02:36:02.095825+0000","last_active":"2024-01-27T02:36:02.096047+0000","last_peered":"2024-01-27T02:36:02.096047+0000","last_clean":"2024-01-27T02:36:02.096047+0000","last_became_active":"2024-01-27T02:36:02.095571+0000","last_became_peered":"2024-01-27T02:36:02.095571+0000","last_unstale":"2024-01-27T02:36:02.096047+0000","last_undegraded":"2024-01-27T02:36:02.096047+0000","last_fullsized":"2024-01-27T02:36:02.096047+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:18:51.540850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.098459+0000","last_change":"2024-01-27T02:36:02.098328+0000","last_active":"2024-01-27T02:36:02.098459+0000","last_peered":"2024-01-27T02:36:02.098459+0000","last_clean":"2024-01-27T02:36:02.098459+0000","last_became_active":"2024-01-27T02:36:02.097173+0000","last_became_peered":"2024-01-27T02:36:02.097173+0000","last_unstale":"2024-01-27T02:36:02.098459+0000","last_undegraded":"2024-01-27T02:36:02.098459+0000","last_fullsized":"2024-01-27T02:36:02.098459+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:18:11.870531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.095708+0000","last_change":"2024-01-27T02:36:02.095646+0000","last_active":"2024-01-27T02:36:02.095708+0000","last_peered":"2024-01-27T02:36:02.095708+0000","last_clean":"2024-01-27T02:36:02.095708+0000","last_became_active":"2024-01-27T02:36:02.095377+0000","last_became_peered":"2024-01-27T02:36:02.095377+0000","last_unstale":"2024-01-27T02:36:02.095708+0000","last_undegraded":"2024-01-27T02:36:02.095708+0000","last_fullsized":"2024-01-27T02:36:02.095708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:42:39.503366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.093718+0000","last_change":"2024-01-27T02:36:02.093524+0000","last_active":"2024-01-27T02:36:02.093718+0000","last_peered":"2024-01-27T02:36:02.093718+0000","last_clean":"2024-01-27T02:36:02.093718+0000","last_became_active":"2024-01-27T02:36:02.093075+0000","last_became_peered":"2024-01-27T02:36:02.093075+0000","last_unstale":"2024-01-27T02:36:02.093718+0000","last_undegraded":"2024-01-27T02:36:02.093718+0000","last_fullsized":"2024-01-27T02:36:02.093718+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:33:36.526133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.095241+0000","last_change":"2024-01-27T02:36:02.095166+0000","last_active":"2024-01-27T02:36:02.095241+0000","last_peered":"2024-01-27T02:36:02.095241+0000","last_clean":"2024-01-27T02:36:02.095241+0000","last_became_active":"2024-01-27T02:36:02.094776+0000","last_became_peered":"2024-01-27T02:36:02.094776+0000","last_unstale":"2024-01-27T02:36:02.095241+0000","last_undegraded":"2024-01-27T02:36:02.095241+0000","last_fullsized":"2024-01-27T02:36:02.095241+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:51:06.991922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.095316+0000","last_change":"2024-01-27T02:36:02.095226+0000","last_active":"2024-01-27T02:36:02.095316+0000","last_peered":"2024-01-27T02:36:02.095316+0000","last_clean":"2024-01-27T02:36:02.095316+0000","last_became_active":"2024-01-27T02:36:02.094967+0000","last_became_peered":"2024-01-27T02:36:02.094967+0000","last_unstale":"2024-01-27T02:36:02.095316+0000","last_undegraded":"2024-01-27T02:36:02.095316+0000","last_fullsized":"2024-01-27T02:36:02.095316+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:37:14.931166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.090835+0000","last_change":"2024-01-27T02:36:02.090232+0000","last_active":"2024-01-27T02:36:02.090835+0000","last_peered":"2024-01-27T02:36:02.090835+0000","last_clean":"2024-01-27T02:36:02.090835+0000","last_became_active":"2024-01-27T02:36:02.089987+0000","last_became_peered":"2024-01-27T02:36:02.089987+0000","last_unstale":"2024-01-27T02:36:02.090835+0000","last_undegraded":"2024-01-27T02:36:02.090835+0000","last_fullsized":"2024-01-27T02:36:02.090835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:28:19.604756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.105360+0000","last_change":"2024-01-27T02:36:02.105289+0000","last_active":"2024-01-27T02:36:02.105360+0000","last_peered":"2024-01-27T02:36:02.105360+0000","last_clean":"2024-01-27T02:36:02.105360+0000","last_became_active":"2024-01-27T02:36:02.104736+0000","last_became_peered":"2024-01-27T02:36:02.104736+0000","last_unstale":"2024-01-27T02:36:02.105360+0000","last_undegraded":"2024-01-27T02:36:02.105360+0000","last_fullsized":"2024-01-27T02:36:02.105360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:05:31.415970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"80'4","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.111953+0000","last_change":"2024-01-27T02:36:01.077741+0000","last_active":"2024-01-27T02:36:02.111953+0000","last_peered":"2024-01-27T02:36:02.111953+0000","last_clean":"2024-01-27T02:36:02.111953+0000","last_became_active":"2024-01-27T02:36:01.077423+0000","last_became_peered":"2024-01-27T02:36:01.077423+0000","last_unstale":"2024-01-27T02:36:02.111953+0000","last_undegraded":"2024-01-27T02:36:02.111953+0000","last_fullsized":"2024-01-27T02:36:02.111953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:15.272806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.097995+0000","last_change":"2024-01-27T02:36:02.097414+0000","last_active":"2024-01-27T02:36:02.097995+0000","last_peered":"2024-01-27T02:36:02.097995+0000","last_clean":"2024-01-27T02:36:02.097995+0000","last_became_active":"2024-01-27T02:36:02.096356+0000","last_became_peered":"2024-01-27T02:36:02.096356+0000","last_unstale":"2024-01-27T02:36:02.097995+0000","last_undegraded":"2024-01-27T02:36:02.097995+0000","last_fullsized":"2024-01-27T02:36:02.097995+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:18:33.881224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"80'1","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.109117+0000","last_change":"2024-01-27T02:36:01.084751+0000","last_active":"2024-01-27T02:36:02.109117+0000","last_peered":"2024-01-27T02:36:02.109117+0000","last_clean":"2024-01-27T02:36:02.109117+0000","last_became_active":"2024-01-27T02:36:01.084351+0000","last_became_peered":"2024-01-27T02:36:01.084351+0000","last_unstale":"2024-01-27T02:36:02.109117+0000","last_undegraded":"2024-01-27T02:36:02.109117+0000","last_fullsized":"2024-01-27T02:36:02.109117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:42:38.497144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.101483+0000","last_change":"2024-01-27T02:36:02.101386+0000","last_active":"2024-01-27T02:36:02.101483+0000","last_peered":"2024-01-27T02:36:02.101483+0000","last_clean":"2024-01-27T02:36:02.101483+0000","last_became_active":"2024-01-27T02:36:02.100967+0000","last_became_peered":"2024-01-27T02:36:02.100967+0000","last_unstale":"2024-01-27T02:36:02.101483+0000","last_undegraded":"2024-01-27T02:36:02.101483+0000","last_fullsized":"2024-01-27T02:36:02.101483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:42:39.503366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"80'2","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.111416+0000","last_change":"2024-01-27T02:36:01.087593+0000","last_active":"2024-01-27T02:36:02.111416+0000","last_peered":"2024-01-27T02:36:02.111416+0000","last_clean":"2024-01-27T02:36:02.111416+0000","last_became_active":"2024-01-27T02:36:01.086695+0000","last_became_peered":"2024-01-27T02:36:01.086695+0000","last_unstale":"2024-01-27T02:36:02.111416+0000","last_undegraded":"2024-01-27T02:36:02.111416+0000","last_fullsized":"2024-01-27T02:36:02.111416+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:15:04.879041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.101667+0000","last_change":"2024-01-27T02:36:02.101583+0000","last_active":"2024-01-27T02:36:02.101667+0000","last_peered":"2024-01-27T02:36:02.101667+0000","last_clean":"2024-01-27T02:36:02.101667+0000","last_became_active":"2024-01-27T02:36:02.101339+0000","last_became_peered":"2024-01-27T02:36:02.101339+0000","last_unstale":"2024-01-27T02:36:02.101667+0000","last_undegraded":"2024-01-27T02:36:02.101667+0000","last_fullsized":"2024-01-27T02:36:02.101667+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:19:18.008697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"80'2","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.108615+0000","last_change":"2024-01-27T02:36:01.082918+0000","last_active":"2024-01-27T02:36:02.108615+0000","last_peered":"2024-01-27T02:36:02.108615+0000","last_clean":"2024-01-27T02:36:02.108615+0000","last_became_active":"2024-01-27T02:36:01.081169+0000","last_became_peered":"2024-01-27T02:36:01.081169+0000","last_unstale":"2024-01-27T02:36:02.108615+0000","last_undegraded":"2024-01-27T02:36:02.108615+0000","last_fullsized":"2024-01-27T02:36:02.108615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:39:18.167253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.096712+0000","last_change":"2024-01-27T02:36:02.096644+0000","last_active":"2024-01-27T02:36:02.096712+0000","last_peered":"2024-01-27T02:36:02.096712+0000","last_clean":"2024-01-27T02:36:02.096712+0000","last_became_active":"2024-01-27T02:36:02.096421+0000","last_became_peered":"2024-01-27T02:36:02.096421+0000","last_unstale":"2024-01-27T02:36:02.096712+0000","last_undegraded":"2024-01-27T02:36:02.096712+0000","last_fullsized":"2024-01-27T02:36:02.096712+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:54:59.718581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"80'1","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.109955+0000","last_change":"2024-01-27T02:36:01.083508+0000","last_active":"2024-01-27T02:36:02.109955+0000","last_peered":"2024-01-27T02:36:02.109955+0000","last_clean":"2024-01-27T02:36:02.109955+0000","last_became_active":"2024-01-27T02:36:01.083244+0000","last_became_peered":"2024-01-27T02:36:01.083244+0000","last_unstale":"2024-01-27T02:36:02.109955+0000","last_undegraded":"2024-01-27T02:36:02.109955+0000","last_fullsized":"2024-01-27T02:36:02.109955+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:42:06.107055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.101711+0000","last_change":"2024-01-27T02:36:02.101628+0000","last_active":"2024-01-27T02:36:02.101711+0000","last_peered":"2024-01-27T02:36:02.101711+0000","last_clean":"2024-01-27T02:36:02.101711+0000","last_became_active":"2024-01-27T02:36:02.101390+0000","last_became_peered":"2024-01-27T02:36:02.101390+0000","last_unstale":"2024-01-27T02:36:02.101711+0000","last_undegraded":"2024-01-27T02:36:02.101711+0000","last_fullsized":"2024-01-27T02:36:02.101711+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:04:39.785751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"80'1","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.108475+0000","last_change":"2024-01-27T02:36:01.081011+0000","last_active":"2024-01-27T02:36:02.108475+0000","last_peered":"2024-01-27T02:36:02.108475+0000","last_clean":"2024-01-27T02:36:02.108475+0000","last_became_active":"2024-01-27T02:36:01.080166+0000","last_became_peered":"2024-01-27T02:36:01.080166+0000","last_unstale":"2024-01-27T02:36:02.108475+0000","last_undegraded":"2024-01-27T02:36:02.108475+0000","last_fullsized":"2024-01-27T02:36:02.108475+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:42:38.578182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.101540+0000","last_change":"2024-01-27T02:36:02.101196+0000","last_active":"2024-01-27T02:36:02.101540+0000","last_peered":"2024-01-27T02:36:02.101540+0000","last_clean":"2024-01-27T02:36:02.101540+0000","last_became_active":"2024-01-27T02:36:02.100779+0000","last_became_peered":"2024-01-27T02:36:02.100779+0000","last_unstale":"2024-01-27T02:36:02.101540+0000","last_undegraded":"2024-01-27T02:36:02.101540+0000","last_fullsized":"2024-01-27T02:36:02.101540+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:30:24.870081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.088074+0000","last_change":"2024-01-27T02:36:01.086427+0000","last_active":"2024-01-27T02:36:02.088074+0000","last_peered":"2024-01-27T02:36:02.088074+0000","last_clean":"2024-01-27T02:36:02.088074+0000","last_became_active":"2024-01-27T02:36:01.085912+0000","last_became_peered":"2024-01-27T02:36:01.085912+0000","last_unstale":"2024-01-27T02:36:02.088074+0000","last_undegraded":"2024-01-27T02:36:02.088074+0000","last_fullsized":"2024-01-27T02:36:02.088074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:12:59.343148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.101257+0000","last_change":"2024-01-27T02:36:02.101173+0000","last_active":"2024-01-27T02:36:02.101257+0000","last_peered":"2024-01-27T02:36:02.101257+0000","last_clean":"2024-01-27T02:36:02.101257+0000","last_became_active":"2024-01-27T02:36:02.100885+0000","last_became_peered":"2024-01-27T02:36:02.100885+0000","last_unstale":"2024-01-27T02:36:02.101257+0000","last_undegraded":"2024-01-27T02:36:02.101257+0000","last_fullsized":"2024-01-27T02:36:02.101257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:21.383627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.094047+0000","last_change":"2024-01-27T02:36:01.084880+0000","last_active":"2024-01-27T02:36:02.094047+0000","last_peered":"2024-01-27T02:36:02.094047+0000","last_clean":"2024-01-27T02:36:02.094047+0000","last_became_active":"2024-01-27T02:36:01.084532+0000","last_became_peered":"2024-01-27T02:36:01.084532+0000","last_unstale":"2024-01-27T02:36:02.094047+0000","last_undegraded":"2024-01-27T02:36:02.094047+0000","last_fullsized":"2024-01-27T02:36:02.094047+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:58:04.302291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.103068+0000","last_change":"2024-01-27T02:36:02.103007+0000","last_active":"2024-01-27T02:36:02.103068+0000","last_peered":"2024-01-27T02:36:02.103068+0000","last_clean":"2024-01-27T02:36:02.103068+0000","last_became_active":"2024-01-27T02:36:02.102782+0000","last_became_peered":"2024-01-27T02:36:02.102782+0000","last_unstale":"2024-01-27T02:36:02.103068+0000","last_undegraded":"2024-01-27T02:36:02.103068+0000","last_fullsized":"2024-01-27T02:36:02.103068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:19:59.227343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.094505+0000","last_change":"2024-01-27T02:36:01.083170+0000","last_active":"2024-01-27T02:36:02.094505+0000","last_peered":"2024-01-27T02:36:02.094505+0000","last_clean":"2024-01-27T02:36:02.094505+0000","last_became_active":"2024-01-27T02:36:01.082730+0000","last_became_peered":"2024-01-27T02:36:01.082730+0000","last_unstale":"2024-01-27T02:36:02.094505+0000","last_undegraded":"2024-01-27T02:36:02.094505+0000","last_fullsized":"2024-01-27T02:36:02.094505+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:07:19.984794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.104269+0000","last_change":"2024-01-27T02:36:02.104204+0000","last_active":"2024-01-27T02:36:02.104269+0000","last_peered":"2024-01-27T02:36:02.104269+0000","last_clean":"2024-01-27T02:36:02.104269+0000","last_became_active":"2024-01-27T02:36:02.103860+0000","last_became_peered":"2024-01-27T02:36:02.103860+0000","last_unstale":"2024-01-27T02:36:02.104269+0000","last_undegraded":"2024-01-27T02:36:02.104269+0000","last_fullsized":"2024-01-27T02:36:02.104269+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:09:00.736774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.088250+0000","last_change":"2024-01-27T02:36:01.079850+0000","last_active":"2024-01-27T02:36:02.088250+0000","last_peered":"2024-01-27T02:36:02.088250+0000","last_clean":"2024-01-27T02:36:02.088250+0000","last_became_active":"2024-01-27T02:36:01.079488+0000","last_became_peered":"2024-01-27T02:36:01.079488+0000","last_unstale":"2024-01-27T02:36:02.088250+0000","last_undegraded":"2024-01-27T02:36:02.088250+0000","last_fullsized":"2024-01-27T02:36:02.088250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:36:51.217544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.094636+0000","last_change":"2024-01-27T02:36:02.093902+0000","last_active":"2024-01-27T02:36:02.094636+0000","last_peered":"2024-01-27T02:36:02.094636+0000","last_clean":"2024-01-27T02:36:02.094636+0000","last_became_active":"2024-01-27T02:36:02.093390+0000","last_became_peered":"2024-01-27T02:36:02.093390+0000","last_unstale":"2024-01-27T02:36:02.094636+0000","last_undegraded":"2024-01-27T02:36:02.094636+0000","last_fullsized":"2024-01-27T02:36:02.094636+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:04:39.785751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"80'3","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.108543+0000","last_change":"2024-01-27T02:36:01.075921+0000","last_active":"2024-01-27T02:36:02.108543+0000","last_peered":"2024-01-27T02:36:02.108543+0000","last_clean":"2024-01-27T02:36:02.108543+0000","last_became_active":"2024-01-27T02:36:01.075647+0000","last_became_peered":"2024-01-27T02:36:01.075647+0000","last_unstale":"2024-01-27T02:36:02.108543+0000","last_undegraded":"2024-01-27T02:36:02.108543+0000","last_fullsized":"2024-01-27T02:36:02.108543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:54:58.712360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.098013+0000","last_change":"2024-01-27T02:36:02.097710+0000","last_active":"2024-01-27T02:36:02.098013+0000","last_peered":"2024-01-27T02:36:02.098013+0000","last_clean":"2024-01-27T02:36:02.098013+0000","last_became_active":"2024-01-27T02:36:02.096707+0000","last_became_peered":"2024-01-27T02:36:02.096707+0000","last_unstale":"2024-01-27T02:36:02.098013+0000","last_undegraded":"2024-01-27T02:36:02.098013+0000","last_fullsized":"2024-01-27T02:36:02.098013+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:13:16.927847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"80'6","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.111701+0000","last_change":"2024-01-27T02:36:01.085134+0000","last_active":"2024-01-27T02:36:02.111701+0000","last_peered":"2024-01-27T02:36:02.111701+0000","last_clean":"2024-01-27T02:36:02.111701+0000","last_became_active":"2024-01-27T02:36:01.084887+0000","last_became_peered":"2024-01-27T02:36:01.084887+0000","last_unstale":"2024-01-27T02:36:02.111701+0000","last_undegraded":"2024-01-27T02:36:02.111701+0000","last_fullsized":"2024-01-27T02:36:02.111701+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:24:30.352994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.102647+0000","last_change":"2024-01-27T02:36:02.102590+0000","last_active":"2024-01-27T02:36:02.102647+0000","last_peered":"2024-01-27T02:36:02.102647+0000","last_clean":"2024-01-27T02:36:02.102647+0000","last_became_active":"2024-01-27T02:36:02.102401+0000","last_became_peered":"2024-01-27T02:36:02.102401+0000","last_unstale":"2024-01-27T02:36:02.102647+0000","last_undegraded":"2024-01-27T02:36:02.102647+0000","last_fullsized":"2024-01-27T02:36:02.102647+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:37:50.389419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"80'2","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.109329+0000","last_change":"2024-01-27T02:36:01.081854+0000","last_active":"2024-01-27T02:36:02.109329+0000","last_peered":"2024-01-27T02:36:02.109329+0000","last_clean":"2024-01-27T02:36:02.109329+0000","last_became_active":"2024-01-27T02:36:01.081553+0000","last_became_peered":"2024-01-27T02:36:01.081553+0000","last_unstale":"2024-01-27T02:36:02.109329+0000","last_undegraded":"2024-01-27T02:36:02.109329+0000","last_fullsized":"2024-01-27T02:36:02.109329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:36:51.217544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.102016+0000","last_change":"2024-01-27T02:36:02.101950+0000","last_active":"2024-01-27T02:36:02.102016+0000","last_peered":"2024-01-27T02:36:02.102016+0000","last_clean":"2024-01-27T02:36:02.102016+0000","last_became_active":"2024-01-27T02:36:02.101724+0000","last_became_peered":"2024-01-27T02:36:02.101724+0000","last_unstale":"2024-01-27T02:36:02.102016+0000","last_undegraded":"2024-01-27T02:36:02.102016+0000","last_fullsized":"2024-01-27T02:36:02.102016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:42:07.113277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.088999+0000","last_change":"2024-01-27T02:36:01.087113+0000","last_active":"2024-01-27T02:36:02.088999+0000","last_peered":"2024-01-27T02:36:02.088999+0000","last_clean":"2024-01-27T02:36:02.088999+0000","last_became_active":"2024-01-27T02:36:01.086735+0000","last_became_peered":"2024-01-27T02:36:01.086735+0000","last_unstale":"2024-01-27T02:36:02.088999+0000","last_undegraded":"2024-01-27T02:36:02.088999+0000","last_fullsized":"2024-01-27T02:36:02.088999+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:51:30.355507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.096777+0000","last_change":"2024-01-27T02:36:02.096670+0000","last_active":"2024-01-27T02:36:02.096777+0000","last_peered":"2024-01-27T02:36:02.096777+0000","last_clean":"2024-01-27T02:36:02.096777+0000","last_became_active":"2024-01-27T02:36:02.096070+0000","last_became_peered":"2024-01-27T02:36:02.096070+0000","last_unstale":"2024-01-27T02:36:02.096777+0000","last_undegraded":"2024-01-27T02:36:02.096777+0000","last_fullsized":"2024-01-27T02:36:02.096777+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:48:39.299949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.094742+0000","last_change":"2024-01-27T02:36:01.079455+0000","last_active":"2024-01-27T02:36:02.094742+0000","last_peered":"2024-01-27T02:36:02.094742+0000","last_clean":"2024-01-27T02:36:02.094742+0000","last_became_active":"2024-01-27T02:36:01.078810+0000","last_became_peered":"2024-01-27T02:36:01.078810+0000","last_unstale":"2024-01-27T02:36:02.094742+0000","last_undegraded":"2024-01-27T02:36:02.094742+0000","last_fullsized":"2024-01-27T02:36:02.094742+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:32:57.446408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.100189+0000","last_change":"2024-01-27T02:36:02.099873+0000","last_active":"2024-01-27T02:36:02.100189+0000","last_peered":"2024-01-27T02:36:02.100189+0000","last_clean":"2024-01-27T02:36:02.100189+0000","last_became_active":"2024-01-27T02:36:02.099126+0000","last_became_peered":"2024-01-27T02:36:02.099126+0000","last_unstale":"2024-01-27T02:36:02.100189+0000","last_undegraded":"2024-01-27T02:36:02.100189+0000","last_fullsized":"2024-01-27T02:36:02.100189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:43:40.714127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"80'8","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.112850+0000","last_change":"2024-01-27T02:36:01.083387+0000","last_active":"2024-01-27T02:36:02.112850+0000","last_peered":"2024-01-27T02:36:02.112850+0000","last_clean":"2024-01-27T02:36:02.112850+0000","last_became_active":"2024-01-27T02:36:01.083023+0000","last_became_peered":"2024-01-27T02:36:01.083023+0000","last_unstale":"2024-01-27T02:36:02.112850+0000","last_undegraded":"2024-01-27T02:36:02.112850+0000","last_fullsized":"2024-01-27T02:36:02.112850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:16:41.009300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.097713+0000","last_change":"2024-01-27T02:36:02.097609+0000","last_active":"2024-01-27T02:36:02.097713+0000","last_peered":"2024-01-27T02:36:02.097713+0000","last_clean":"2024-01-27T02:36:02.097713+0000","last_became_active":"2024-01-27T02:36:02.097211+0000","last_became_peered":"2024-01-27T02:36:02.097211+0000","last_unstale":"2024-01-27T02:36:02.097713+0000","last_undegraded":"2024-01-27T02:36:02.097713+0000","last_fullsized":"2024-01-27T02:36:02.097713+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:04:07.439211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"80'1","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.110341+0000","last_change":"2024-01-27T02:36:01.075386+0000","last_active":"2024-01-27T02:36:02.110341+0000","last_peered":"2024-01-27T02:36:02.110341+0000","last_clean":"2024-01-27T02:36:02.110341+0000","last_became_active":"2024-01-27T02:36:01.074276+0000","last_became_peered":"2024-01-27T02:36:01.074276+0000","last_unstale":"2024-01-27T02:36:02.110341+0000","last_undegraded":"2024-01-27T02:36:02.110341+0000","last_fullsized":"2024-01-27T02:36:02.110341+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:46:34.597904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.100508+0000","last_change":"2024-01-27T02:36:02.100373+0000","last_active":"2024-01-27T02:36:02.100508+0000","last_peered":"2024-01-27T02:36:02.100508+0000","last_clean":"2024-01-27T02:36:02.100508+0000","last_became_active":"2024-01-27T02:36:02.100122+0000","last_became_peered":"2024-01-27T02:36:02.100122+0000","last_unstale":"2024-01-27T02:36:02.100508+0000","last_undegraded":"2024-01-27T02:36:02.100508+0000","last_fullsized":"2024-01-27T02:36:02.100508+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:51:06.991922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"80'2","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.111523+0000","last_change":"2024-01-27T02:36:01.087262+0000","last_active":"2024-01-27T02:36:02.111523+0000","last_peered":"2024-01-27T02:36:02.111523+0000","last_clean":"2024-01-27T02:36:02.111523+0000","last_became_active":"2024-01-27T02:36:01.086163+0000","last_became_peered":"2024-01-27T02:36:01.086163+0000","last_unstale":"2024-01-27T02:36:02.111523+0000","last_undegraded":"2024-01-27T02:36:02.111523+0000","last_fullsized":"2024-01-27T02:36:02.111523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:35:13.192349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.098961+0000","last_change":"2024-01-27T02:36:02.098722+0000","last_active":"2024-01-27T02:36:02.098961+0000","last_peered":"2024-01-27T02:36:02.098961+0000","last_clean":"2024-01-27T02:36:02.098961+0000","last_became_active":"2024-01-27T02:36:02.098241+0000","last_became_peered":"2024-01-27T02:36:02.098241+0000","last_unstale":"2024-01-27T02:36:02.098961+0000","last_undegraded":"2024-01-27T02:36:02.098961+0000","last_fullsized":"2024-01-27T02:36:02.098961+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:51:31.361728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"80'1","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.110945+0000","last_change":"2024-01-27T02:36:01.086968+0000","last_active":"2024-01-27T02:36:02.110945+0000","last_peered":"2024-01-27T02:36:02.110945+0000","last_clean":"2024-01-27T02:36:02.110945+0000","last_became_active":"2024-01-27T02:36:01.086112+0000","last_became_peered":"2024-01-27T02:36:01.086112+0000","last_unstale":"2024-01-27T02:36:02.110945+0000","last_undegraded":"2024-01-27T02:36:02.110945+0000","last_fullsized":"2024-01-27T02:36:02.110945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:03:55.891925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.097350+0000","last_change":"2024-01-27T02:36:02.097268+0000","last_active":"2024-01-27T02:36:02.097350+0000","last_peered":"2024-01-27T02:36:02.097350+0000","last_clean":"2024-01-27T02:36:02.097350+0000","last_became_active":"2024-01-27T02:36:02.097011+0000","last_became_peered":"2024-01-27T02:36:02.097011+0000","last_unstale":"2024-01-27T02:36:02.097350+0000","last_undegraded":"2024-01-27T02:36:02.097350+0000","last_fullsized":"2024-01-27T02:36:02.097350+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:14:57.057715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.090638+0000","last_change":"2024-01-27T02:36:01.085172+0000","last_active":"2024-01-27T02:36:02.090638+0000","last_peered":"2024-01-27T02:36:02.090638+0000","last_clean":"2024-01-27T02:36:02.090638+0000","last_became_active":"2024-01-27T02:36:01.084894+0000","last_became_peered":"2024-01-27T02:36:01.084894+0000","last_unstale":"2024-01-27T02:36:02.090638+0000","last_undegraded":"2024-01-27T02:36:02.090638+0000","last_fullsized":"2024-01-27T02:36:02.090638+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:19:37.782912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.091427+0000","last_change":"2024-01-27T02:36:02.091309+0000","last_active":"2024-01-27T02:36:02.091427+0000","last_peered":"2024-01-27T02:36:02.091427+0000","last_clean":"2024-01-27T02:36:02.091427+0000","last_became_active":"2024-01-27T02:36:02.091070+0000","last_became_peered":"2024-01-27T02:36:02.091070+0000","last_unstale":"2024-01-27T02:36:02.091427+0000","last_undegraded":"2024-01-27T02:36:02.091427+0000","last_fullsized":"2024-01-27T02:36:02.091427+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:55.624218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.088430+0000","last_change":"2024-01-27T02:36:01.080657+0000","last_active":"2024-01-27T02:36:02.088430+0000","last_peered":"2024-01-27T02:36:02.088430+0000","last_clean":"2024-01-27T02:36:02.088430+0000","last_became_active":"2024-01-27T02:36:01.078442+0000","last_became_peered":"2024-01-27T02:36:01.078442+0000","last_unstale":"2024-01-27T02:36:02.088430+0000","last_undegraded":"2024-01-27T02:36:02.088430+0000","last_fullsized":"2024-01-27T02:36:02.088430+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:45:47.314805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.096241+0000","last_change":"2024-01-27T02:36:02.096166+0000","last_active":"2024-01-27T02:36:02.096241+0000","last_peered":"2024-01-27T02:36:02.096241+0000","last_clean":"2024-01-27T02:36:02.096241+0000","last_became_active":"2024-01-27T02:36:02.095902+0000","last_became_peered":"2024-01-27T02:36:02.095902+0000","last_unstale":"2024-01-27T02:36:02.096241+0000","last_undegraded":"2024-01-27T02:36:02.096241+0000","last_fullsized":"2024-01-27T02:36:02.096241+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:48:08.841511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.092336+0000","last_change":"2024-01-27T02:36:01.085914+0000","last_active":"2024-01-27T02:36:02.092336+0000","last_peered":"2024-01-27T02:36:02.092336+0000","last_clean":"2024-01-27T02:36:02.092336+0000","last_became_active":"2024-01-27T02:36:01.085674+0000","last_became_peered":"2024-01-27T02:36:01.085674+0000","last_unstale":"2024-01-27T02:36:02.092336+0000","last_undegraded":"2024-01-27T02:36:02.092336+0000","last_fullsized":"2024-01-27T02:36:02.092336+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:19:37.782912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.093670+0000","last_change":"2024-01-27T02:36:02.092888+0000","last_active":"2024-01-27T02:36:02.093670+0000","last_peered":"2024-01-27T02:36:02.093670+0000","last_clean":"2024-01-27T02:36:02.093670+0000","last_became_active":"2024-01-27T02:36:02.092614+0000","last_became_peered":"2024-01-27T02:36:02.092614+0000","last_unstale":"2024-01-27T02:36:02.093670+0000","last_undegraded":"2024-01-27T02:36:02.093670+0000","last_fullsized":"2024-01-27T02:36:02.093670+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:18:39.999190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.091927+0000","last_change":"2024-01-27T02:36:01.086128+0000","last_active":"2024-01-27T02:36:02.091927+0000","last_peered":"2024-01-27T02:36:02.091927+0000","last_clean":"2024-01-27T02:36:02.091927+0000","last_became_active":"2024-01-27T02:36:01.085891+0000","last_became_peered":"2024-01-27T02:36:01.085891+0000","last_unstale":"2024-01-27T02:36:02.091927+0000","last_undegraded":"2024-01-27T02:36:02.091927+0000","last_fullsized":"2024-01-27T02:36:02.091927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:05:15.222635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.097780+0000","last_change":"2024-01-27T02:36:02.097572+0000","last_active":"2024-01-27T02:36:02.097780+0000","last_peered":"2024-01-27T02:36:02.097780+0000","last_clean":"2024-01-27T02:36:02.097780+0000","last_became_active":"2024-01-27T02:36:02.096958+0000","last_became_peered":"2024-01-27T02:36:02.096958+0000","last_unstale":"2024-01-27T02:36:02.097780+0000","last_undegraded":"2024-01-27T02:36:02.097780+0000","last_fullsized":"2024-01-27T02:36:02.097780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:41:50.546172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"80'2","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.109071+0000","last_change":"2024-01-27T02:36:01.079712+0000","last_active":"2024-01-27T02:36:02.109071+0000","last_peered":"2024-01-27T02:36:02.109071+0000","last_clean":"2024-01-27T02:36:02.109071+0000","last_became_active":"2024-01-27T02:36:01.079176+0000","last_became_peered":"2024-01-27T02:36:01.079176+0000","last_unstale":"2024-01-27T02:36:02.109071+0000","last_undegraded":"2024-01-27T02:36:02.109071+0000","last_fullsized":"2024-01-27T02:36:02.109071+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:12:59.343148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.101156+0000","last_change":"2024-01-27T02:36:02.100955+0000","last_active":"2024-01-27T02:36:02.101156+0000","last_peered":"2024-01-27T02:36:02.101156+0000","last_clean":"2024-01-27T02:36:02.101156+0000","last_became_active":"2024-01-27T02:36:02.100449+0000","last_became_peered":"2024-01-27T02:36:02.100449+0000","last_unstale":"2024-01-27T02:36:02.101156+0000","last_undegraded":"2024-01-27T02:36:02.101156+0000","last_fullsized":"2024-01-27T02:36:02.101156+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:12:02.170281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"81'1","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-01-27T02:36:03.406250+0000","last_change":"2024-01-27T02:36:01.084252+0000","last_active":"2024-01-27T02:36:03.406250+0000","last_peered":"2024-01-27T02:36:03.406250+0000","last_clean":"2024-01-27T02:36:03.406250+0000","last_became_active":"2024-01-27T02:36:01.083785+0000","last_became_peered":"2024-01-27T02:36:01.083785+0000","last_unstale":"2024-01-27T02:36:03.406250+0000","last_undegraded":"2024-01-27T02:36:03.406250+0000","last_fullsized":"2024-01-27T02:36:03.406250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:15.272806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.103977+0000","last_change":"2024-01-27T02:36:02.103785+0000","last_active":"2024-01-27T02:36:02.103977+0000","last_peered":"2024-01-27T02:36:02.103977+0000","last_clean":"2024-01-27T02:36:02.103977+0000","last_became_active":"2024-01-27T02:36:02.103538+0000","last_became_peered":"2024-01-27T02:36:02.103538+0000","last_unstale":"2024-01-27T02:36:02.103977+0000","last_undegraded":"2024-01-27T02:36:02.103977+0000","last_fullsized":"2024-01-27T02:36:02.103977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:18:51.540850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.092358+0000","last_change":"2024-01-27T02:36:01.083357+0000","last_active":"2024-01-27T02:36:02.092358+0000","last_peered":"2024-01-27T02:36:02.092358+0000","last_clean":"2024-01-27T02:36:02.092358+0000","last_became_active":"2024-01-27T02:36:01.083009+0000","last_became_peered":"2024-01-27T02:36:01.083009+0000","last_unstale":"2024-01-27T02:36:02.092358+0000","last_undegraded":"2024-01-27T02:36:02.092358+0000","last_fullsized":"2024-01-27T02:36:02.092358+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:17:17.877802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.100492+0000","last_change":"2024-01-27T02:36:02.099832+0000","last_active":"2024-01-27T02:36:02.100492+0000","last_peered":"2024-01-27T02:36:02.100492+0000","last_clean":"2024-01-27T02:36:02.100492+0000","last_became_active":"2024-01-27T02:36:02.099223+0000","last_became_peered":"2024-01-27T02:36:02.099223+0000","last_unstale":"2024-01-27T02:36:02.100492+0000","last_undegraded":"2024-01-27T02:36:02.100492+0000","last_fullsized":"2024-01-27T02:36:02.100492+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:37:14.931166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.092093+0000","last_change":"2024-01-27T02:36:01.083453+0000","last_active":"2024-01-27T02:36:02.092093+0000","last_peered":"2024-01-27T02:36:02.092093+0000","last_clean":"2024-01-27T02:36:02.092093+0000","last_became_active":"2024-01-27T02:36:01.083173+0000","last_became_peered":"2024-01-27T02:36:01.083173+0000","last_unstale":"2024-01-27T02:36:02.092093+0000","last_undegraded":"2024-01-27T02:36:02.092093+0000","last_fullsized":"2024-01-27T02:36:02.092093+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:21:40.093915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.100167+0000","last_change":"2024-01-27T02:36:02.099715+0000","last_active":"2024-01-27T02:36:02.100167+0000","last_peered":"2024-01-27T02:36:02.100167+0000","last_clean":"2024-01-27T02:36:02.100167+0000","last_became_active":"2024-01-27T02:36:02.098796+0000","last_became_peered":"2024-01-27T02:36:02.098796+0000","last_unstale":"2024-01-27T02:36:02.100167+0000","last_undegraded":"2024-01-27T02:36:02.100167+0000","last_fullsized":"2024-01-27T02:36:02.100167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:40:44.218390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.093872+0000","last_change":"2024-01-27T02:36:01.086016+0000","last_active":"2024-01-27T02:36:02.093872+0000","last_peered":"2024-01-27T02:36:02.093872+0000","last_clean":"2024-01-27T02:36:02.093872+0000","last_became_active":"2024-01-27T02:36:01.085747+0000","last_became_peered":"2024-01-27T02:36:01.085747+0000","last_unstale":"2024-01-27T02:36:02.093872+0000","last_undegraded":"2024-01-27T02:36:02.093872+0000","last_fullsized":"2024-01-27T02:36:02.093872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:03:04.731782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.101466+0000","last_change":"2024-01-27T02:36:02.101216+0000","last_active":"2024-01-27T02:36:02.101466+0000","last_peered":"2024-01-27T02:36:02.101466+0000","last_clean":"2024-01-27T02:36:02.101466+0000","last_became_active":"2024-01-27T02:36:02.100632+0000","last_became_peered":"2024-01-27T02:36:02.100632+0000","last_unstale":"2024-01-27T02:36:02.101466+0000","last_undegraded":"2024-01-27T02:36:02.101466+0000","last_fullsized":"2024-01-27T02:36:02.101466+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:05:16.228857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.088965+0000","last_change":"2024-01-27T02:36:01.085778+0000","last_active":"2024-01-27T02:36:02.088965+0000","last_peered":"2024-01-27T02:36:02.088965+0000","last_clean":"2024-01-27T02:36:02.088965+0000","last_became_active":"2024-01-27T02:36:01.085504+0000","last_became_peered":"2024-01-27T02:36:01.085504+0000","last_unstale":"2024-01-27T02:36:02.088965+0000","last_undegraded":"2024-01-27T02:36:02.088965+0000","last_fullsized":"2024-01-27T02:36:02.088965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:34:00.107552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.089533+0000","last_change":"2024-01-27T02:36:02.089392+0000","last_active":"2024-01-27T02:36:02.089533+0000","last_peered":"2024-01-27T02:36:02.089533+0000","last_clean":"2024-01-27T02:36:02.089533+0000","last_became_active":"2024-01-27T02:36:02.089117+0000","last_became_peered":"2024-01-27T02:36:02.089117+0000","last_unstale":"2024-01-27T02:36:02.089533+0000","last_undegraded":"2024-01-27T02:36:02.089533+0000","last_fullsized":"2024-01-27T02:36:02.089533+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:15:05.885263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.091855+0000","last_change":"2024-01-27T02:36:01.083238+0000","last_active":"2024-01-27T02:36:02.091855+0000","last_peered":"2024-01-27T02:36:02.091855+0000","last_clean":"2024-01-27T02:36:02.091855+0000","last_became_active":"2024-01-27T02:36:01.082768+0000","last_became_peered":"2024-01-27T02:36:01.082768+0000","last_unstale":"2024-01-27T02:36:02.091855+0000","last_undegraded":"2024-01-27T02:36:02.091855+0000","last_fullsized":"2024-01-27T02:36:02.091855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:34:52.370887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.090788+0000","last_change":"2024-01-27T02:36:02.089283+0000","last_active":"2024-01-27T02:36:02.090788+0000","last_peered":"2024-01-27T02:36:02.090788+0000","last_clean":"2024-01-27T02:36:02.090788+0000","last_became_active":"2024-01-27T02:36:02.089044+0000","last_became_peered":"2024-01-27T02:36:02.089044+0000","last_unstale":"2024-01-27T02:36:02.090788+0000","last_undegraded":"2024-01-27T02:36:02.090788+0000","last_fullsized":"2024-01-27T02:36:02.090788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:32:58.452630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.088515+0000","last_change":"2024-01-27T02:36:01.078683+0000","last_active":"2024-01-27T02:36:02.088515+0000","last_peered":"2024-01-27T02:36:02.088515+0000","last_clean":"2024-01-27T02:36:02.088515+0000","last_became_active":"2024-01-27T02:36:01.078126+0000","last_became_peered":"2024-01-27T02:36:01.078126+0000","last_unstale":"2024-01-27T02:36:02.088515+0000","last_undegraded":"2024-01-27T02:36:02.088515+0000","last_fullsized":"2024-01-27T02:36:02.088515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:10:25.706212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.093699+0000","last_change":"2024-01-27T02:36:02.093378+0000","last_active":"2024-01-27T02:36:02.093699+0000","last_peered":"2024-01-27T02:36:02.093699+0000","last_clean":"2024-01-27T02:36:02.093699+0000","last_became_active":"2024-01-27T02:36:02.092243+0000","last_became_peered":"2024-01-27T02:36:02.092243+0000","last_unstale":"2024-01-27T02:36:02.093699+0000","last_undegraded":"2024-01-27T02:36:02.093699+0000","last_fullsized":"2024-01-27T02:36:02.093699+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:44:17.898667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"80'2","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.110306+0000","last_change":"2024-01-27T02:36:01.080741+0000","last_active":"2024-01-27T02:36:02.110306+0000","last_peered":"2024-01-27T02:36:02.110306+0000","last_clean":"2024-01-27T02:36:02.110306+0000","last_became_active":"2024-01-27T02:36:01.079822+0000","last_became_peered":"2024-01-27T02:36:01.079822+0000","last_unstale":"2024-01-27T02:36:02.110306+0000","last_undegraded":"2024-01-27T02:36:02.110306+0000","last_fullsized":"2024-01-27T02:36:02.110306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:23:56.398583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":213,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.096068+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:36:02.096068+0000","last_peered":"2024-01-27T02:36:02.096068+0000","last_clean":"2024-01-27T02:36:02.096068+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:36:02.096068+0000","last_undegraded":"2024-01-27T02:36:02.096068+0000","last_fullsized":"2024-01-27T02:36:02.096068+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":992,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194320,"statfs":{"total":200000000,"available":198983702,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.45000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71599999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.46800000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.78400000000000003}]},{"osd":1,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.435,"5min":0.373,"15min":0.373},"max":{"1min":0.71899999999999997,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.438,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.70599999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.66300000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.45800000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.72299999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.41499999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70499999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.64300000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.73499999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.72499999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.67200000000000004}]},{"osd":4,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48299999999999998,"15min":0.48999999999999999},"min":{"1min":0.29299999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66500000000000004,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.502},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.504,"15min":0.51600000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73399999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.56399999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.52300000000000002,"15min":0.53700000000000003},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.72099999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55400000000000005,"15min":0.56499999999999995},"min":{"1min":0.38700000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73499999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.58899999999999997}]},{"osd":6,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.58399999999999996,"15min":0.59799999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73199999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56799999999999995,"15min":0.57799999999999996},"min":{"1min":0.40500000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76700000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":415,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194897,"statfs":{"total":200000000,"available":199574566,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.56000000000000005,"15min":0.51200000000000001},"min":{"1min":0.434,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79700000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55800000000000005,"15min":0.52000000000000002},"min":{"1min":0.35999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.61599999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.55700000000000005,"15min":0.51000000000000001},"min":{"1min":0.42599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76500000000000001,"5min":0.874,"15min":0.874},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58699999999999997,"15min":0.54900000000000004},"min":{"1min":0.44500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82199999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.52100000000000002}]},{"osd":2,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56599999999999995,"15min":0.53900000000000003},"min":{"1min":0.46999999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73999999999999999,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.61099999999999999,"15min":0.57799999999999996},"min":{"1min":0.52200000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80400000000000005,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.57499999999999996}]},{"osd":3,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.48799999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.74299999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.56699999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.78400000000000003,"5min":1.262,"15min":1.262},"last":0.63600000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.36599999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70299999999999996,"5min":0.748,"15min":0.748},"last":0.75},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53600000000000003,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70299999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.65000000000000002}]},{"osd":5,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52900000000000003,"15min":0.53400000000000003},"min":{"1min":0.42199999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65600000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58699999999999997,"15min":0.60899999999999999},"min":{"1min":0.41799999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76700000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.69299999999999995}]},{"osd":7,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.61699999999999999},"min":{"1min":0.39300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.73499999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57799999999999996,"15min":0.59199999999999997},"min":{"1min":0.38500000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71599999999999997,"5min":1.038,"15min":1.038},"last":0.68200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":417,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194895,"statfs":{"total":200000000,"available":199572534,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.38600000000000001,"15min":0.373},"min":{"1min":0.307,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.55900000000000005,"5min":0.626,"15min":0.626},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40699999999999997,"15min":0.39500000000000002},"min":{"1min":0.28699999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.57099999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.38300000000000001}]},{"osd":2,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.36099999999999999},"min":{"1min":0.29599999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.60099999999999998,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.442,"15min":0.42699999999999999},"min":{"1min":0.35499999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65900000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.434}]},{"osd":3,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48099999999999998,"15min":0.46600000000000003},"min":{"1min":0.36599999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.501,"5min":0.46300000000000002,"15min":0.45400000000000001},"min":{"1min":0.34399999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.68700000000000006,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.45100000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.63200000000000001,"15min":0.624},"min":{"1min":0.48599999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.96799999999999997,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.64600000000000002,"15min":0.63200000000000001},"min":{"1min":0.51700000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88800000000000001,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.625}]},{"osd":5,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63800000000000001,"15min":0.622},"min":{"1min":0.47499999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.93400000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.66300000000000003,"15min":0.66700000000000004},"min":{"1min":0.47299999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.70499999999999996}]},{"osd":6,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.70299999999999996,"15min":0.69399999999999995},"min":{"1min":0.55100000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.95699999999999996,"5min":1.024,"15min":1.024},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.68400000000000005,"15min":0.67000000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.90300000000000002,"5min":1.151,"15min":1.151},"last":0.67900000000000005}]},{"osd":7,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.69799999999999995,"15min":0.67800000000000005},"min":{"1min":0.495,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88,"5min":1.097,"15min":1.097},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.70299999999999996,"15min":0.68500000000000005},"min":{"1min":0.54600000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.65900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":991,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194320,"statfs":{"total":200000000,"available":198984290,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39100000000000001,"15min":0.38900000000000001},"min":{"1min":0.28799999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.59799999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.30299999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.624,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.48399999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.42799999999999999,"15min":0.41899999999999998},"min":{"1min":0.33900000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.441,"15min":0.432},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58699999999999997,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.40300000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.42899999999999999,"15min":0.43099999999999999},"min":{"1min":0.33100000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61799999999999999,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.377},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.47499999999999998,"15min":0.47099999999999997},"min":{"1min":0.40600000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42399999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.51200000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.878,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65600000000000003,"15min":0.65800000000000003},"min":{"1min":0.51900000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54700000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66500000000000004,"15min":0.65300000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0029999999999999,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.56999999999999995,"5min":0.44,"15min":0.44},"max":{"1min":0.90500000000000003,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.61499999999999999}]},{"osd":6,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.66400000000000003,"15min":0.66300000000000003},"min":{"1min":0.52500000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89200000000000002,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68300000000000005,"15min":0.67600000000000005},"min":{"1min":0.54900000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.65900000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.54300000000000004,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.98599999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68700000000000006,"15min":0.69299999999999995},"min":{"1min":0.499,"5min":0.498,"15min":0.498},"max":{"1min":0.92300000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.499}]}]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":416,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194895,"statfs":{"total":200000000,"available":199573122,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41799999999999998,"15min":0.40999999999999998},"min":{"1min":0.26400000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56499999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.55500000000000005,"15min":0.65300000000000002},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61799999999999999,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":0.47899999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42099999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.626,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.443,"15min":0.44800000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.62,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.39400000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.42099999999999999,"15min":0.41499999999999998},"min":{"1min":0.313,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.56299999999999994,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.502},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.64900000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.44800000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.51200000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89800000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75700000000000001,"15min":0.80700000000000005},"min":{"1min":0.56399999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.91000000000000003,"5min":3.581,"15min":3.581},"last":0.71599999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69999999999999996,"15min":0.751},"min":{"1min":0.52500000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84799999999999998,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67800000000000005,"15min":0.67100000000000004},"min":{"1min":0.52700000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92000000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.621}]},{"osd":6,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.52600000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89100000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.67800000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94999999999999996,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.70199999999999996}]},{"osd":7,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.79900000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69199999999999995,"15min":0.68799999999999994},"min":{"1min":0.47499999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.67300000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":415,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194897,"statfs":{"total":200000000,"available":199574546,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.376,"15min":0.35499999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.57299999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.255},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40000000000000002,"15min":0.39300000000000002},"min":{"1min":0.29099999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66900000000000004,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.318}]},{"osd":1,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.40699999999999997,"15min":0.39100000000000001},"min":{"1min":0.317,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.437,"15min":0.41799999999999998},"min":{"1min":0.38800000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.30199999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.44600000000000001,"15min":0.42599999999999999},"min":{"1min":0.36199999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.46899999999999997,"15min":0.44500000000000001},"min":{"1min":0.35299999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.436}]},{"osd":4,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.433,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.628,"15min":0.60199999999999998},"min":{"1min":0.504,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.47899999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.63400000000000001,"15min":0.60899999999999999},"min":{"1min":0.55700000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.51000000000000001,"5min":0.496,"15min":0.496},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.46899999999999997}]},{"osd":6,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.65900000000000003,"15min":0.65300000000000002},"min":{"1min":0.496,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64800000000000002,"15min":0.64300000000000002},"min":{"1min":0.503,"5min":0.435,"15min":0.435},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.49299999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.65800000000000003,"15min":0.63400000000000001},"min":{"1min":0.52900000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.49399999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94299999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.50900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":415,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194897,"statfs":{"total":200000000,"available":199574588,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.47299999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.73499999999999999,"5min":1.099,"15min":1.099},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.45000000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73499999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.58899999999999997}]},{"osd":1,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.48999999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0309999999999999,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81999999999999995,"5min":1.133,"15min":1.133},"last":0.60299999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60999999999999999,"15min":0.59699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.3400000000000001,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.626},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.46400000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79800000000000004,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.52000000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.47699999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.624,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.44900000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82099999999999995,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.57299999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.33700000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32800000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.495}]},{"osd":6,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.501,"15min":0.499},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68500000000000005,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38600000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.64200000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54600000000000004,"15min":0.55200000000000005},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79000000000000004,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.66000000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":415,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194897,"statfs":{"total":200000000,"available":199574568,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:36:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58799999999999997,"15min":0.59999999999999998},"min":{"1min":0.41799999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.86499999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.443},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60699999999999998,"15min":0.57899999999999996},"min":{"1min":0.44600000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71699999999999997,"5min":3.242,"15min":3.242},"last":0.52900000000000003}]},{"osd":1,"last update":"Sat Jan 27 02:36:02 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.629,"15min":0.60899999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77200000000000002,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.64400000000000002,"15min":0.65500000000000003},"min":{"1min":0.48399999999999999,"5min":0.436,"15min":0.436},"max":{"1min":0.73699999999999999,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.58699999999999997}]},{"osd":2,"last update":"Sat Jan 27 02:36:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.62,"15min":0.59799999999999998},"min":{"1min":0.46100000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.76600000000000001,"5min":3.04,"15min":3.04},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59599999999999997,"15min":0.57799999999999996},"min":{"1min":0.441,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72199999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.46899999999999997}]},{"osd":3,"last update":"Sat Jan 27 02:36:02 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.68899999999999995,"15min":0.68600000000000005},"min":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76600000000000001,"5min":4.032,"15min":4.032},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.71099999999999997,"15min":0.68100000000000005},"min":{"1min":0.55500000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.77900000000000003,"5min":4.282,"15min":4.282},"last":0.63900000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:36:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52400000000000002,"15min":0.52600000000000002},"min":{"1min":0.34399999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.69999999999999996,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.53400000000000003,"15min":0.57199999999999995},"min":{"1min":0.33200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.75,"5min":1.101,"15min":1.101},"last":0.51200000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:36:02 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53400000000000003,"15min":0.56299999999999994},"min":{"1min":0.38200000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65400000000000003,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.75900000000000001,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.55000000000000004}]},{"osd":7,"last update":"Sat Jan 27 02:36:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.46200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.70299999999999996,"5min":1.008,"15min":1.008},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64700000000000002,"15min":0.66500000000000004},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75,"5min":2,"15min":2},"last":0.66100000000000003}]}]}],"pool_statfs":[]}} 2024-01-27T02:36:08.413 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:36:08.416 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:36:10.976 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:36:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.117+0000 7f754a5fb700 1 Processor -- start 2024-01-27T02:36:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.117+0000 7f754a5fb700 1 -- start start 2024-01-27T02:36:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.118+0000 7f754a5fb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f754415a100 0x7f75440cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.118+0000 7f754a5fb700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f75440cb5b0 0x7f75440c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.118+0000 7f754a5fb700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75440c4f50 con 0x7f75440c4af0 2024-01-27T02:36:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.118+0000 7f754a5fb700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75440ce5e0 con 0x7f754415a100 2024-01-27T02:36:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.118+0000 7f754a5fb700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75440cd9d0 con 0x7f75440cb5b0 2024-01-27T02:36:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.118+0000 7f7543fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f754415a100 0x7f75440cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.118+0000 7f7548b98700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f75440c4af0 0x7f75440c9530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40684/0 (socket says 172.21.15.7:40684) 2024-01-27T02:36:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.118+0000 7f7548b98700 1 -- 172.21.15.7:0/634665937 learned_addr learned my addr 172.21.15.7:0/634665937 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.118+0000 7f75437fe700 1 --2- 172.21.15.7:0/634665937 >> v2:172.21.15.188:3300/0 conn(0x7f75440cb5b0 0x7f75440c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.118+0000 7f7543fff700 1 -- 172.21.15.7:0/634665937 >> v2:172.21.15.188:3300/0 conn(0x7f75440cb5b0 msgr2=0x7f75440c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.118+0000 7f7543fff700 1 --2- 172.21.15.7:0/634665937 >> v2:172.21.15.188:3300/0 conn(0x7f75440cb5b0 0x7f75440c45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.118+0000 7f7543fff700 1 -- 172.21.15.7:0/634665937 >> v1:172.21.15.7:6789/0 conn(0x7f75440c4af0 legacy=0x7f75440c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.118+0000 7f7543fff700 1 -- 172.21.15.7:0/634665937 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75440cdcb0 con 0x7f754415a100 2024-01-27T02:36:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.118+0000 7f7542ffd700 1 -- 172.21.15.7:0/634665937 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3843346841 0 0) 0x7f753c001490 con 0x7f75440c4af0 2024-01-27T02:36:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.118+0000 7f7543fff700 1 --2- 172.21.15.7:0/634665937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f754415a100 0x7f75440cd2f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f754406e170 tx=0x7f75340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90120a6de08fad1f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.119+0000 7f7542ffd700 1 -- 172.21.15.7:0/634665937 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753400b6b0 con 0x7f754415a100 2024-01-27T02:36:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.119+0000 7f7542ffd700 1 -- 172.21.15.7:0/634665937 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753400b810 con 0x7f754415a100 2024-01-27T02:36:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.119+0000 7f7542ffd700 1 -- 172.21.15.7:0/634665937 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753400bbb0 con 0x7f754415a100 2024-01-27T02:36:11.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.119+0000 7f754a5fb700 1 -- 172.21.15.7:0/634665937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f754415a100 msgr2=0x7f75440cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.119+0000 7f754a5fb700 1 --2- 172.21.15.7:0/634665937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f754415a100 0x7f75440cd2f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f754406e170 tx=0x7f75340091a0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.119+0000 7f754a5fb700 1 -- 172.21.15.7:0/634665937 shutdown_connections 2024-01-27T02:36:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.119+0000 7f754a5fb700 1 --2- 172.21.15.7:0/634665937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f754415a100 0x7f75440cd2f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.119+0000 7f754a5fb700 1 --2- 172.21.15.7:0/634665937 >> v2:172.21.15.188:3300/0 conn(0x7f75440cb5b0 0x7f75440c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.119+0000 7f754a5fb700 1 -- 172.21.15.7:0/634665937 >> 172.21.15.7:0/634665937 conn(0x7f7544098020 msgr2=0x7f75440711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.119+0000 7f754a5fb700 1 -- 172.21.15.7:0/634665937 shutdown_connections 2024-01-27T02:36:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.119+0000 7f754a5fb700 1 -- 172.21.15.7:0/634665937 wait complete. 2024-01-27T02:36:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.121+0000 7f754a5fb700 1 Processor -- start 2024-01-27T02:36:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.121+0000 7f754a5fb700 1 -- start start 2024-01-27T02:36:11.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.122+0000 7f754a5fb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75440c4af0 0x7f754415c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:11.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.122+0000 7f754a5fb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75440cb5b0 0x7f754415ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:11.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.122+0000 7f754a5fb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f754415a100 0x7f754414aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:11.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.122+0000 7f754a5fb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75440b2500 con 0x7f75440cb5b0 2024-01-27T02:36:11.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.122+0000 7f754a5fb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754415b650 con 0x7f75440c4af0 2024-01-27T02:36:11.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.122+0000 7f754a5fb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754415b3c0 con 0x7f754415a100 2024-01-27T02:36:11.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.122+0000 7f75437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75440cb5b0 0x7f754415ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:11.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.122+0000 7f75437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75440cb5b0 0x7f754415ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33554/0 (socket says 172.21.15.7:33554) 2024-01-27T02:36:11.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.122+0000 7f75437fe700 1 -- 172.21.15.7:0/1585900386 learned_addr learned my addr 172.21.15.7:0/1585900386 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:11.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.122+0000 7f7543fff700 1 --2- 172.21.15.7:0/1585900386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75440c4af0 0x7f754415c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:11.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.122+0000 7f7548b98700 1 --2- 172.21.15.7:0/1585900386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f754415a100 0x7f754414aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:11.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.122+0000 7f75437fe700 1 -- 172.21.15.7:0/1585900386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f754415a100 msgr2=0x7f754414aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:11.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.122+0000 7f75437fe700 1 --2- 172.21.15.7:0/1585900386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f754415a100 0x7f754414aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.122+0000 7f75437fe700 1 -- 172.21.15.7:0/1585900386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75440c4af0 msgr2=0x7f754415c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:11.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.122+0000 7f75437fe700 1 --2- 172.21.15.7:0/1585900386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75440c4af0 0x7f754415c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.122+0000 7f75437fe700 1 -- 172.21.15.7:0/1585900386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7534009420 con 0x7f75440cb5b0 2024-01-27T02:36:11.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.122+0000 7f75437fe700 1 --2- 172.21.15.7:0/1585900386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75440cb5b0 0x7f754415ae80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f752c00d890 tx=0x7f752c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:11.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.122+0000 7f753bfff700 1 -- 172.21.15.7:0/1585900386 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c0125f0 con 0x7f75440cb5b0 2024-01-27T02:36:11.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.122+0000 7f753bfff700 1 -- 172.21.15.7:0/1585900386 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752c012750 con 0x7f75440cb5b0 2024-01-27T02:36:11.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.122+0000 7f754a5fb700 1 -- 172.21.15.7:0/1585900386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75440b1c10 con 0x7f75440cb5b0 2024-01-27T02:36:11.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.122+0000 7f753bfff700 1 -- 172.21.15.7:0/1585900386 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c012ad0 con 0x7f75440cb5b0 2024-01-27T02:36:11.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.123+0000 7f754a5fb700 1 -- 172.21.15.7:0/1585900386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75440b2190 con 0x7f75440cb5b0 2024-01-27T02:36:11.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.124+0000 7f754a5fb700 1 -- 172.21.15.7:0/1585900386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f754404c710 con 0x7f75440cb5b0 2024-01-27T02:36:11.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.124+0000 7f753bfff700 1 -- 172.21.15.7:0/1585900386 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f752c01e460 con 0x7f75440cb5b0 2024-01-27T02:36:11.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.124+0000 7f753bfff700 1 --2- 172.21.15.7:0/1585900386 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7524066f30 0x7f75240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:11.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.125+0000 7f753bfff700 1 -- 172.21.15.7:0/1585900386 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f752c084b70 con 0x7f75440cb5b0 2024-01-27T02:36:11.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.125+0000 7f7543fff700 1 --2- 172.21.15.7:0/1585900386 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7524066f30 0x7f75240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:11.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.125+0000 7f7543fff700 1 --2- 172.21.15.7:0/1585900386 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7524066f30 0x7f75240693e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f75340093f0 tx=0x7f7534017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:11.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.129+0000 7f753bfff700 1 -- 172.21.15.7:0/1585900386 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f752c055870 con 0x7f75440cb5b0 2024-01-27T02:36:11.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.328+0000 7f754a5fb700 1 -- 172.21.15.7:0/1585900386 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f75440cdf90 con 0x7f7524066f30 2024-01-27T02:36:11.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.337+0000 7f753bfff700 1 -- 172.21.15.7:0/1585900386 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285449 (secure 0 0 0) 0x7f75440cdf90 con 0x7f7524066f30 2024-01-27T02:36:11.341 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:36:11.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.341+0000 7f754a5fb700 1 -- 172.21.15.7:0/1585900386 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7524066f30 msgr2=0x7f75240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:11.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.341+0000 7f754a5fb700 1 --2- 172.21.15.7:0/1585900386 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7524066f30 0x7f75240693e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f75340093f0 tx=0x7f7534017000 comp rx=0 tx=0).stop 2024-01-27T02:36:11.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.341+0000 7f754a5fb700 1 -- 172.21.15.7:0/1585900386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75440cb5b0 msgr2=0x7f754415ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:11.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.341+0000 7f754a5fb700 1 --2- 172.21.15.7:0/1585900386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75440cb5b0 0x7f754415ae80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f752c00d890 tx=0x7f752c008040 comp rx=0 tx=0).stop 2024-01-27T02:36:11.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.341+0000 7f754a5fb700 1 -- 172.21.15.7:0/1585900386 shutdown_connections 2024-01-27T02:36:11.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.341+0000 7f754a5fb700 1 --2- 172.21.15.7:0/1585900386 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7524066f30 0x7f75240693e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.341+0000 7f754a5fb700 1 --2- 172.21.15.7:0/1585900386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f754415a100 0x7f754414aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.341+0000 7f754a5fb700 1 --2- 172.21.15.7:0/1585900386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75440c4af0 0x7f754415c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.341+0000 7f754a5fb700 1 --2- 172.21.15.7:0/1585900386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75440cb5b0 0x7f754415ae80 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.341+0000 7f754a5fb700 1 -- 172.21.15.7:0/1585900386 >> 172.21.15.7:0/1585900386 conn(0x7f7544098020 msgr2=0x7f75440cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:11.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.341+0000 7f754a5fb700 1 -- 172.21.15.7:0/1585900386 shutdown_connections 2024-01-27T02:36:11.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.341+0000 7f754a5fb700 1 -- 172.21.15.7:0/1585900386 wait complete. 2024-01-27T02:36:11.345 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:36:11.355 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-01-27T02:36:09.632581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4480,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558019,"statfs":{"total":1600000000,"available":1595411916,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002062"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.099700+0000","last_change":"2024-01-27T02:36:02.098841+0000","last_active":"2024-01-27T02:36:02.099700+0000","last_peered":"2024-01-27T02:36:02.099700+0000","last_clean":"2024-01-27T02:36:02.099700+0000","last_became_active":"2024-01-27T02:36:02.098660+0000","last_became_peered":"2024-01-27T02:36:02.098660+0000","last_unstale":"2024-01-27T02:36:02.099700+0000","last_undegraded":"2024-01-27T02:36:02.099700+0000","last_fullsized":"2024-01-27T02:36:02.099700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:47:18.259755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.096517+0000","last_change":"2024-01-27T02:36:02.096444+0000","last_active":"2024-01-27T02:36:02.096517+0000","last_peered":"2024-01-27T02:36:02.096517+0000","last_clean":"2024-01-27T02:36:02.096517+0000","last_became_active":"2024-01-27T02:36:02.096187+0000","last_became_peered":"2024-01-27T02:36:02.096187+0000","last_unstale":"2024-01-27T02:36:02.096517+0000","last_undegraded":"2024-01-27T02:36:02.096517+0000","last_fullsized":"2024-01-27T02:36:02.096517+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:42.090153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.095761+0000","last_change":"2024-01-27T02:36:02.095654+0000","last_active":"2024-01-27T02:36:02.095761+0000","last_peered":"2024-01-27T02:36:02.095761+0000","last_clean":"2024-01-27T02:36:02.095761+0000","last_became_active":"2024-01-27T02:36:02.094876+0000","last_became_peered":"2024-01-27T02:36:02.094876+0000","last_unstale":"2024-01-27T02:36:02.095761+0000","last_undegraded":"2024-01-27T02:36:02.095761+0000","last_fullsized":"2024-01-27T02:36:02.095761+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:46:46.374799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.093079+0000","last_change":"2024-01-27T02:36:02.093003+0000","last_active":"2024-01-27T02:36:02.093079+0000","last_peered":"2024-01-27T02:36:02.093079+0000","last_clean":"2024-01-27T02:36:02.093079+0000","last_became_active":"2024-01-27T02:36:02.092682+0000","last_became_peered":"2024-01-27T02:36:02.092682+0000","last_unstale":"2024-01-27T02:36:02.093079+0000","last_undegraded":"2024-01-27T02:36:02.093079+0000","last_fullsized":"2024-01-27T02:36:02.093079+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:07:20.991016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.101504+0000","last_change":"2024-01-27T02:36:02.101043+0000","last_active":"2024-01-27T02:36:02.101504+0000","last_peered":"2024-01-27T02:36:02.101504+0000","last_clean":"2024-01-27T02:36:02.101504+0000","last_became_active":"2024-01-27T02:36:02.100405+0000","last_became_peered":"2024-01-27T02:36:02.100405+0000","last_unstale":"2024-01-27T02:36:02.101504+0000","last_undegraded":"2024-01-27T02:36:02.101504+0000","last_fullsized":"2024-01-27T02:36:02.101504+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:54:00.073605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.098495+0000","last_change":"2024-01-27T02:36:02.098420+0000","last_active":"2024-01-27T02:36:02.098495+0000","last_peered":"2024-01-27T02:36:02.098495+0000","last_clean":"2024-01-27T02:36:02.098495+0000","last_became_active":"2024-01-27T02:36:02.097919+0000","last_became_peered":"2024-01-27T02:36:02.097919+0000","last_unstale":"2024-01-27T02:36:02.098495+0000","last_undegraded":"2024-01-27T02:36:02.098495+0000","last_fullsized":"2024-01-27T02:36:02.098495+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:16.279028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.101173+0000","last_change":"2024-01-27T02:36:02.101093+0000","last_active":"2024-01-27T02:36:02.101173+0000","last_peered":"2024-01-27T02:36:02.101173+0000","last_clean":"2024-01-27T02:36:02.101173+0000","last_became_active":"2024-01-27T02:36:02.100737+0000","last_became_peered":"2024-01-27T02:36:02.100737+0000","last_unstale":"2024-01-27T02:36:02.101173+0000","last_undegraded":"2024-01-27T02:36:02.101173+0000","last_fullsized":"2024-01-27T02:36:02.101173+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:56:33.979247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.094681+0000","last_change":"2024-01-27T02:36:02.093938+0000","last_active":"2024-01-27T02:36:02.094681+0000","last_peered":"2024-01-27T02:36:02.094681+0000","last_clean":"2024-01-27T02:36:02.094681+0000","last_became_active":"2024-01-27T02:36:02.093239+0000","last_became_peered":"2024-01-27T02:36:02.093239+0000","last_unstale":"2024-01-27T02:36:02.094681+0000","last_undegraded":"2024-01-27T02:36:02.094681+0000","last_fullsized":"2024-01-27T02:36:02.094681+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:10:52.422378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.096863+0000","last_change":"2024-01-27T02:36:02.096790+0000","last_active":"2024-01-27T02:36:02.096863+0000","last_peered":"2024-01-27T02:36:02.096863+0000","last_clean":"2024-01-27T02:36:02.096863+0000","last_became_active":"2024-01-27T02:36:02.096488+0000","last_became_peered":"2024-01-27T02:36:02.096488+0000","last_unstale":"2024-01-27T02:36:02.096863+0000","last_undegraded":"2024-01-27T02:36:02.096863+0000","last_fullsized":"2024-01-27T02:36:02.096863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:11.427024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.103462+0000","last_change":"2024-01-27T02:36:02.102823+0000","last_active":"2024-01-27T02:36:02.103462+0000","last_peered":"2024-01-27T02:36:02.103462+0000","last_clean":"2024-01-27T02:36:02.103462+0000","last_became_active":"2024-01-27T02:36:02.102183+0000","last_became_peered":"2024-01-27T02:36:02.102183+0000","last_unstale":"2024-01-27T02:36:02.103462+0000","last_undegraded":"2024-01-27T02:36:02.103462+0000","last_fullsized":"2024-01-27T02:36:02.103462+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:26:17.309915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.099658+0000","last_change":"2024-01-27T02:36:02.098268+0000","last_active":"2024-01-27T02:36:02.099658+0000","last_peered":"2024-01-27T02:36:02.099658+0000","last_clean":"2024-01-27T02:36:02.099658+0000","last_became_active":"2024-01-27T02:36:02.097955+0000","last_became_peered":"2024-01-27T02:36:02.097955+0000","last_unstale":"2024-01-27T02:36:02.099658+0000","last_undegraded":"2024-01-27T02:36:02.099658+0000","last_fullsized":"2024-01-27T02:36:02.099658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:54:59.718581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.098980+0000","last_change":"2024-01-27T02:36:02.098873+0000","last_active":"2024-01-27T02:36:02.098980+0000","last_peered":"2024-01-27T02:36:02.098980+0000","last_clean":"2024-01-27T02:36:02.098980+0000","last_became_active":"2024-01-27T02:36:02.098497+0000","last_became_peered":"2024-01-27T02:36:02.098497+0000","last_unstale":"2024-01-27T02:36:02.098980+0000","last_undegraded":"2024-01-27T02:36:02.098980+0000","last_fullsized":"2024-01-27T02:36:02.098980+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:12:41.850276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.097417+0000","last_change":"2024-01-27T02:36:02.097351+0000","last_active":"2024-01-27T02:36:02.097417+0000","last_peered":"2024-01-27T02:36:02.097417+0000","last_clean":"2024-01-27T02:36:02.097417+0000","last_became_active":"2024-01-27T02:36:02.096406+0000","last_became_peered":"2024-01-27T02:36:02.096406+0000","last_unstale":"2024-01-27T02:36:02.097417+0000","last_undegraded":"2024-01-27T02:36:02.097417+0000","last_fullsized":"2024-01-27T02:36:02.097417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:57:20.844278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.094795+0000","last_change":"2024-01-27T02:36:02.094721+0000","last_active":"2024-01-27T02:36:02.094795+0000","last_peered":"2024-01-27T02:36:02.094795+0000","last_clean":"2024-01-27T02:36:02.094795+0000","last_became_active":"2024-01-27T02:36:02.094463+0000","last_became_peered":"2024-01-27T02:36:02.094463+0000","last_unstale":"2024-01-27T02:36:02.094795+0000","last_undegraded":"2024-01-27T02:36:02.094795+0000","last_fullsized":"2024-01-27T02:36:02.094795+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:37:23.189235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.095468+0000","last_change":"2024-01-27T02:36:02.095410+0000","last_active":"2024-01-27T02:36:02.095468+0000","last_peered":"2024-01-27T02:36:02.095468+0000","last_clean":"2024-01-27T02:36:02.095468+0000","last_became_active":"2024-01-27T02:36:02.095027+0000","last_became_peered":"2024-01-27T02:36:02.095027+0000","last_unstale":"2024-01-27T02:36:02.095468+0000","last_undegraded":"2024-01-27T02:36:02.095468+0000","last_fullsized":"2024-01-27T02:36:02.095468+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:14:13.651138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.100950+0000","last_change":"2024-01-27T02:36:02.100887+0000","last_active":"2024-01-27T02:36:02.100950+0000","last_peered":"2024-01-27T02:36:02.100950+0000","last_clean":"2024-01-27T02:36:02.100950+0000","last_became_active":"2024-01-27T02:36:02.100554+0000","last_became_peered":"2024-01-27T02:36:02.100554+0000","last_unstale":"2024-01-27T02:36:02.100950+0000","last_undegraded":"2024-01-27T02:36:02.100950+0000","last_fullsized":"2024-01-27T02:36:02.100950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:21:28.748243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.101558+0000","last_change":"2024-01-27T02:36:02.100548+0000","last_active":"2024-01-27T02:36:02.101558+0000","last_peered":"2024-01-27T02:36:02.101558+0000","last_clean":"2024-01-27T02:36:02.101558+0000","last_became_active":"2024-01-27T02:36:02.099461+0000","last_became_peered":"2024-01-27T02:36:02.099461+0000","last_unstale":"2024-01-27T02:36:02.101558+0000","last_undegraded":"2024-01-27T02:36:02.101558+0000","last_fullsized":"2024-01-27T02:36:02.101558+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:07:58.814010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.094832+0000","last_change":"2024-01-27T02:36:02.093803+0000","last_active":"2024-01-27T02:36:02.094832+0000","last_peered":"2024-01-27T02:36:02.094832+0000","last_clean":"2024-01-27T02:36:02.094832+0000","last_became_active":"2024-01-27T02:36:02.093304+0000","last_became_peered":"2024-01-27T02:36:02.093304+0000","last_unstale":"2024-01-27T02:36:02.094832+0000","last_undegraded":"2024-01-27T02:36:02.094832+0000","last_fullsized":"2024-01-27T02:36:02.094832+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:35:14.198571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.090855+0000","last_change":"2024-01-27T02:36:02.090798+0000","last_active":"2024-01-27T02:36:02.090855+0000","last_peered":"2024-01-27T02:36:02.090855+0000","last_clean":"2024-01-27T02:36:02.090855+0000","last_became_active":"2024-01-27T02:36:02.090569+0000","last_became_peered":"2024-01-27T02:36:02.090569+0000","last_unstale":"2024-01-27T02:36:02.090855+0000","last_undegraded":"2024-01-27T02:36:02.090855+0000","last_fullsized":"2024-01-27T02:36:02.090855+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:21:41.100136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.095382+0000","last_change":"2024-01-27T02:36:02.095258+0000","last_active":"2024-01-27T02:36:02.095382+0000","last_peered":"2024-01-27T02:36:02.095382+0000","last_clean":"2024-01-27T02:36:02.095382+0000","last_became_active":"2024-01-27T02:36:02.094714+0000","last_became_peered":"2024-01-27T02:36:02.094714+0000","last_unstale":"2024-01-27T02:36:02.095382+0000","last_undegraded":"2024-01-27T02:36:02.095382+0000","last_fullsized":"2024-01-27T02:36:02.095382+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:32:03.273755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.099797+0000","last_change":"2024-01-27T02:36:02.099691+0000","last_active":"2024-01-27T02:36:02.099797+0000","last_peered":"2024-01-27T02:36:02.099797+0000","last_clean":"2024-01-27T02:36:02.099797+0000","last_became_active":"2024-01-27T02:36:02.099281+0000","last_became_peered":"2024-01-27T02:36:02.099281+0000","last_unstale":"2024-01-27T02:36:02.099797+0000","last_undegraded":"2024-01-27T02:36:02.099797+0000","last_fullsized":"2024-01-27T02:36:02.099797+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:32:58.452630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.096155+0000","last_change":"2024-01-27T02:36:02.096081+0000","last_active":"2024-01-27T02:36:02.096155+0000","last_peered":"2024-01-27T02:36:02.096155+0000","last_clean":"2024-01-27T02:36:02.096155+0000","last_became_active":"2024-01-27T02:36:02.095770+0000","last_became_peered":"2024-01-27T02:36:02.095770+0000","last_unstale":"2024-01-27T02:36:02.096155+0000","last_undegraded":"2024-01-27T02:36:02.096155+0000","last_fullsized":"2024-01-27T02:36:02.096155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:16.279028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.099677+0000","last_change":"2024-01-27T02:36:02.093040+0000","last_active":"2024-01-27T02:36:02.099677+0000","last_peered":"2024-01-27T02:36:02.099677+0000","last_clean":"2024-01-27T02:36:02.099677+0000","last_became_active":"2024-01-27T02:36:02.092769+0000","last_became_peered":"2024-01-27T02:36:02.092769+0000","last_unstale":"2024-01-27T02:36:02.099677+0000","last_undegraded":"2024-01-27T02:36:02.099677+0000","last_fullsized":"2024-01-27T02:36:02.099677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:42:07.113277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.105510+0000","last_change":"2024-01-27T02:36:02.105443+0000","last_active":"2024-01-27T02:36:02.105510+0000","last_peered":"2024-01-27T02:36:02.105510+0000","last_clean":"2024-01-27T02:36:02.105510+0000","last_became_active":"2024-01-27T02:36:02.105056+0000","last_became_peered":"2024-01-27T02:36:02.105056+0000","last_unstale":"2024-01-27T02:36:02.105510+0000","last_undegraded":"2024-01-27T02:36:02.105510+0000","last_fullsized":"2024-01-27T02:36:02.105510+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:22:09.618511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.103480+0000","last_change":"2024-01-27T02:36:02.103171+0000","last_active":"2024-01-27T02:36:02.103480+0000","last_peered":"2024-01-27T02:36:02.103480+0000","last_clean":"2024-01-27T02:36:02.103480+0000","last_became_active":"2024-01-27T02:36:02.102570+0000","last_became_peered":"2024-01-27T02:36:02.102570+0000","last_unstale":"2024-01-27T02:36:02.103480+0000","last_undegraded":"2024-01-27T02:36:02.103480+0000","last_fullsized":"2024-01-27T02:36:02.103480+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:05:16.228857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.096047+0000","last_change":"2024-01-27T02:36:02.095825+0000","last_active":"2024-01-27T02:36:02.096047+0000","last_peered":"2024-01-27T02:36:02.096047+0000","last_clean":"2024-01-27T02:36:02.096047+0000","last_became_active":"2024-01-27T02:36:02.095571+0000","last_became_peered":"2024-01-27T02:36:02.095571+0000","last_unstale":"2024-01-27T02:36:02.096047+0000","last_undegraded":"2024-01-27T02:36:02.096047+0000","last_fullsized":"2024-01-27T02:36:02.096047+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:18:51.540850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.098459+0000","last_change":"2024-01-27T02:36:02.098328+0000","last_active":"2024-01-27T02:36:02.098459+0000","last_peered":"2024-01-27T02:36:02.098459+0000","last_clean":"2024-01-27T02:36:02.098459+0000","last_became_active":"2024-01-27T02:36:02.097173+0000","last_became_peered":"2024-01-27T02:36:02.097173+0000","last_unstale":"2024-01-27T02:36:02.098459+0000","last_undegraded":"2024-01-27T02:36:02.098459+0000","last_fullsized":"2024-01-27T02:36:02.098459+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:18:11.870531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.095708+0000","last_change":"2024-01-27T02:36:02.095646+0000","last_active":"2024-01-27T02:36:02.095708+0000","last_peered":"2024-01-27T02:36:02.095708+0000","last_clean":"2024-01-27T02:36:02.095708+0000","last_became_active":"2024-01-27T02:36:02.095377+0000","last_became_peered":"2024-01-27T02:36:02.095377+0000","last_unstale":"2024-01-27T02:36:02.095708+0000","last_undegraded":"2024-01-27T02:36:02.095708+0000","last_fullsized":"2024-01-27T02:36:02.095708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:42:39.503366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.093718+0000","last_change":"2024-01-27T02:36:02.093524+0000","last_active":"2024-01-27T02:36:02.093718+0000","last_peered":"2024-01-27T02:36:02.093718+0000","last_clean":"2024-01-27T02:36:02.093718+0000","last_became_active":"2024-01-27T02:36:02.093075+0000","last_became_peered":"2024-01-27T02:36:02.093075+0000","last_unstale":"2024-01-27T02:36:02.093718+0000","last_undegraded":"2024-01-27T02:36:02.093718+0000","last_fullsized":"2024-01-27T02:36:02.093718+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:33:36.526133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.095241+0000","last_change":"2024-01-27T02:36:02.095166+0000","last_active":"2024-01-27T02:36:02.095241+0000","last_peered":"2024-01-27T02:36:02.095241+0000","last_clean":"2024-01-27T02:36:02.095241+0000","last_became_active":"2024-01-27T02:36:02.094776+0000","last_became_peered":"2024-01-27T02:36:02.094776+0000","last_unstale":"2024-01-27T02:36:02.095241+0000","last_undegraded":"2024-01-27T02:36:02.095241+0000","last_fullsized":"2024-01-27T02:36:02.095241+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:51:06.991922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.095316+0000","last_change":"2024-01-27T02:36:02.095226+0000","last_active":"2024-01-27T02:36:02.095316+0000","last_peered":"2024-01-27T02:36:02.095316+0000","last_clean":"2024-01-27T02:36:02.095316+0000","last_became_active":"2024-01-27T02:36:02.094967+0000","last_became_peered":"2024-01-27T02:36:02.094967+0000","last_unstale":"2024-01-27T02:36:02.095316+0000","last_undegraded":"2024-01-27T02:36:02.095316+0000","last_fullsized":"2024-01-27T02:36:02.095316+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:37:14.931166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.090835+0000","last_change":"2024-01-27T02:36:02.090232+0000","last_active":"2024-01-27T02:36:02.090835+0000","last_peered":"2024-01-27T02:36:02.090835+0000","last_clean":"2024-01-27T02:36:02.090835+0000","last_became_active":"2024-01-27T02:36:02.089987+0000","last_became_peered":"2024-01-27T02:36:02.089987+0000","last_unstale":"2024-01-27T02:36:02.090835+0000","last_undegraded":"2024-01-27T02:36:02.090835+0000","last_fullsized":"2024-01-27T02:36:02.090835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:28:19.604756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.105360+0000","last_change":"2024-01-27T02:36:02.105289+0000","last_active":"2024-01-27T02:36:02.105360+0000","last_peered":"2024-01-27T02:36:02.105360+0000","last_clean":"2024-01-27T02:36:02.105360+0000","last_became_active":"2024-01-27T02:36:02.104736+0000","last_became_peered":"2024-01-27T02:36:02.104736+0000","last_unstale":"2024-01-27T02:36:02.105360+0000","last_undegraded":"2024-01-27T02:36:02.105360+0000","last_fullsized":"2024-01-27T02:36:02.105360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:05:31.415970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"80'4","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.111953+0000","last_change":"2024-01-27T02:36:01.077741+0000","last_active":"2024-01-27T02:36:02.111953+0000","last_peered":"2024-01-27T02:36:02.111953+0000","last_clean":"2024-01-27T02:36:02.111953+0000","last_became_active":"2024-01-27T02:36:01.077423+0000","last_became_peered":"2024-01-27T02:36:01.077423+0000","last_unstale":"2024-01-27T02:36:02.111953+0000","last_undegraded":"2024-01-27T02:36:02.111953+0000","last_fullsized":"2024-01-27T02:36:02.111953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:15.272806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.097995+0000","last_change":"2024-01-27T02:36:02.097414+0000","last_active":"2024-01-27T02:36:02.097995+0000","last_peered":"2024-01-27T02:36:02.097995+0000","last_clean":"2024-01-27T02:36:02.097995+0000","last_became_active":"2024-01-27T02:36:02.096356+0000","last_became_peered":"2024-01-27T02:36:02.096356+0000","last_unstale":"2024-01-27T02:36:02.097995+0000","last_undegraded":"2024-01-27T02:36:02.097995+0000","last_fullsized":"2024-01-27T02:36:02.097995+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:18:33.881224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"80'1","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.109117+0000","last_change":"2024-01-27T02:36:01.084751+0000","last_active":"2024-01-27T02:36:02.109117+0000","last_peered":"2024-01-27T02:36:02.109117+0000","last_clean":"2024-01-27T02:36:02.109117+0000","last_became_active":"2024-01-27T02:36:01.084351+0000","last_became_peered":"2024-01-27T02:36:01.084351+0000","last_unstale":"2024-01-27T02:36:02.109117+0000","last_undegraded":"2024-01-27T02:36:02.109117+0000","last_fullsized":"2024-01-27T02:36:02.109117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:42:38.497144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.101483+0000","last_change":"2024-01-27T02:36:02.101386+0000","last_active":"2024-01-27T02:36:02.101483+0000","last_peered":"2024-01-27T02:36:02.101483+0000","last_clean":"2024-01-27T02:36:02.101483+0000","last_became_active":"2024-01-27T02:36:02.100967+0000","last_became_peered":"2024-01-27T02:36:02.100967+0000","last_unstale":"2024-01-27T02:36:02.101483+0000","last_undegraded":"2024-01-27T02:36:02.101483+0000","last_fullsized":"2024-01-27T02:36:02.101483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:42:39.503366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"80'2","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.111416+0000","last_change":"2024-01-27T02:36:01.087593+0000","last_active":"2024-01-27T02:36:02.111416+0000","last_peered":"2024-01-27T02:36:02.111416+0000","last_clean":"2024-01-27T02:36:02.111416+0000","last_became_active":"2024-01-27T02:36:01.086695+0000","last_became_peered":"2024-01-27T02:36:01.086695+0000","last_unstale":"2024-01-27T02:36:02.111416+0000","last_undegraded":"2024-01-27T02:36:02.111416+0000","last_fullsized":"2024-01-27T02:36:02.111416+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:15:04.879041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.101667+0000","last_change":"2024-01-27T02:36:02.101583+0000","last_active":"2024-01-27T02:36:02.101667+0000","last_peered":"2024-01-27T02:36:02.101667+0000","last_clean":"2024-01-27T02:36:02.101667+0000","last_became_active":"2024-01-27T02:36:02.101339+0000","last_became_peered":"2024-01-27T02:36:02.101339+0000","last_unstale":"2024-01-27T02:36:02.101667+0000","last_undegraded":"2024-01-27T02:36:02.101667+0000","last_fullsized":"2024-01-27T02:36:02.101667+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:19:18.008697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"80'2","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.108615+0000","last_change":"2024-01-27T02:36:01.082918+0000","last_active":"2024-01-27T02:36:02.108615+0000","last_peered":"2024-01-27T02:36:02.108615+0000","last_clean":"2024-01-27T02:36:02.108615+0000","last_became_active":"2024-01-27T02:36:01.081169+0000","last_became_peered":"2024-01-27T02:36:01.081169+0000","last_unstale":"2024-01-27T02:36:02.108615+0000","last_undegraded":"2024-01-27T02:36:02.108615+0000","last_fullsized":"2024-01-27T02:36:02.108615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:39:18.167253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.096712+0000","last_change":"2024-01-27T02:36:02.096644+0000","last_active":"2024-01-27T02:36:02.096712+0000","last_peered":"2024-01-27T02:36:02.096712+0000","last_clean":"2024-01-27T02:36:02.096712+0000","last_became_active":"2024-01-27T02:36:02.096421+0000","last_became_peered":"2024-01-27T02:36:02.096421+0000","last_unstale":"2024-01-27T02:36:02.096712+0000","last_undegraded":"2024-01-27T02:36:02.096712+0000","last_fullsized":"2024-01-27T02:36:02.096712+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:54:59.718581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"80'1","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.109955+0000","last_change":"2024-01-27T02:36:01.083508+0000","last_active":"2024-01-27T02:36:02.109955+0000","last_peered":"2024-01-27T02:36:02.109955+0000","last_clean":"2024-01-27T02:36:02.109955+0000","last_became_active":"2024-01-27T02:36:01.083244+0000","last_became_peered":"2024-01-27T02:36:01.083244+0000","last_unstale":"2024-01-27T02:36:02.109955+0000","last_undegraded":"2024-01-27T02:36:02.109955+0000","last_fullsized":"2024-01-27T02:36:02.109955+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:42:06.107055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.101711+0000","last_change":"2024-01-27T02:36:02.101628+0000","last_active":"2024-01-27T02:36:02.101711+0000","last_peered":"2024-01-27T02:36:02.101711+0000","last_clean":"2024-01-27T02:36:02.101711+0000","last_became_active":"2024-01-27T02:36:02.101390+0000","last_became_peered":"2024-01-27T02:36:02.101390+0000","last_unstale":"2024-01-27T02:36:02.101711+0000","last_undegraded":"2024-01-27T02:36:02.101711+0000","last_fullsized":"2024-01-27T02:36:02.101711+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:04:39.785751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"80'1","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.108475+0000","last_change":"2024-01-27T02:36:01.081011+0000","last_active":"2024-01-27T02:36:02.108475+0000","last_peered":"2024-01-27T02:36:02.108475+0000","last_clean":"2024-01-27T02:36:02.108475+0000","last_became_active":"2024-01-27T02:36:01.080166+0000","last_became_peered":"2024-01-27T02:36:01.080166+0000","last_unstale":"2024-01-27T02:36:02.108475+0000","last_undegraded":"2024-01-27T02:36:02.108475+0000","last_fullsized":"2024-01-27T02:36:02.108475+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:42:38.578182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.101540+0000","last_change":"2024-01-27T02:36:02.101196+0000","last_active":"2024-01-27T02:36:02.101540+0000","last_peered":"2024-01-27T02:36:02.101540+0000","last_clean":"2024-01-27T02:36:02.101540+0000","last_became_active":"2024-01-27T02:36:02.100779+0000","last_became_peered":"2024-01-27T02:36:02.100779+0000","last_unstale":"2024-01-27T02:36:02.101540+0000","last_undegraded":"2024-01-27T02:36:02.101540+0000","last_fullsized":"2024-01-27T02:36:02.101540+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:30:24.870081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.088074+0000","last_change":"2024-01-27T02:36:01.086427+0000","last_active":"2024-01-27T02:36:02.088074+0000","last_peered":"2024-01-27T02:36:02.088074+0000","last_clean":"2024-01-27T02:36:02.088074+0000","last_became_active":"2024-01-27T02:36:01.085912+0000","last_became_peered":"2024-01-27T02:36:01.085912+0000","last_unstale":"2024-01-27T02:36:02.088074+0000","last_undegraded":"2024-01-27T02:36:02.088074+0000","last_fullsized":"2024-01-27T02:36:02.088074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:12:59.343148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.101257+0000","last_change":"2024-01-27T02:36:02.101173+0000","last_active":"2024-01-27T02:36:02.101257+0000","last_peered":"2024-01-27T02:36:02.101257+0000","last_clean":"2024-01-27T02:36:02.101257+0000","last_became_active":"2024-01-27T02:36:02.100885+0000","last_became_peered":"2024-01-27T02:36:02.100885+0000","last_unstale":"2024-01-27T02:36:02.101257+0000","last_undegraded":"2024-01-27T02:36:02.101257+0000","last_fullsized":"2024-01-27T02:36:02.101257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:21.383627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.094047+0000","last_change":"2024-01-27T02:36:01.084880+0000","last_active":"2024-01-27T02:36:02.094047+0000","last_peered":"2024-01-27T02:36:02.094047+0000","last_clean":"2024-01-27T02:36:02.094047+0000","last_became_active":"2024-01-27T02:36:01.084532+0000","last_became_peered":"2024-01-27T02:36:01.084532+0000","last_unstale":"2024-01-27T02:36:02.094047+0000","last_undegraded":"2024-01-27T02:36:02.094047+0000","last_fullsized":"2024-01-27T02:36:02.094047+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:58:04.302291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.103068+0000","last_change":"2024-01-27T02:36:02.103007+0000","last_active":"2024-01-27T02:36:02.103068+0000","last_peered":"2024-01-27T02:36:02.103068+0000","last_clean":"2024-01-27T02:36:02.103068+0000","last_became_active":"2024-01-27T02:36:02.102782+0000","last_became_peered":"2024-01-27T02:36:02.102782+0000","last_unstale":"2024-01-27T02:36:02.103068+0000","last_undegraded":"2024-01-27T02:36:02.103068+0000","last_fullsized":"2024-01-27T02:36:02.103068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:19:59.227343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.094505+0000","last_change":"2024-01-27T02:36:01.083170+0000","last_active":"2024-01-27T02:36:02.094505+0000","last_peered":"2024-01-27T02:36:02.094505+0000","last_clean":"2024-01-27T02:36:02.094505+0000","last_became_active":"2024-01-27T02:36:01.082730+0000","last_became_peered":"2024-01-27T02:36:01.082730+0000","last_unstale":"2024-01-27T02:36:02.094505+0000","last_undegraded":"2024-01-27T02:36:02.094505+0000","last_fullsized":"2024-01-27T02:36:02.094505+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:07:19.984794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.104269+0000","last_change":"2024-01-27T02:36:02.104204+0000","last_active":"2024-01-27T02:36:02.104269+0000","last_peered":"2024-01-27T02:36:02.104269+0000","last_clean":"2024-01-27T02:36:02.104269+0000","last_became_active":"2024-01-27T02:36:02.103860+0000","last_became_peered":"2024-01-27T02:36:02.103860+0000","last_unstale":"2024-01-27T02:36:02.104269+0000","last_undegraded":"2024-01-27T02:36:02.104269+0000","last_fullsized":"2024-01-27T02:36:02.104269+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:09:00.736774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.088250+0000","last_change":"2024-01-27T02:36:01.079850+0000","last_active":"2024-01-27T02:36:02.088250+0000","last_peered":"2024-01-27T02:36:02.088250+0000","last_clean":"2024-01-27T02:36:02.088250+0000","last_became_active":"2024-01-27T02:36:01.079488+0000","last_became_peered":"2024-01-27T02:36:01.079488+0000","last_unstale":"2024-01-27T02:36:02.088250+0000","last_undegraded":"2024-01-27T02:36:02.088250+0000","last_fullsized":"2024-01-27T02:36:02.088250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:36:51.217544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.094636+0000","last_change":"2024-01-27T02:36:02.093902+0000","last_active":"2024-01-27T02:36:02.094636+0000","last_peered":"2024-01-27T02:36:02.094636+0000","last_clean":"2024-01-27T02:36:02.094636+0000","last_became_active":"2024-01-27T02:36:02.093390+0000","last_became_peered":"2024-01-27T02:36:02.093390+0000","last_unstale":"2024-01-27T02:36:02.094636+0000","last_undegraded":"2024-01-27T02:36:02.094636+0000","last_fullsized":"2024-01-27T02:36:02.094636+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:04:39.785751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"80'3","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.108543+0000","last_change":"2024-01-27T02:36:01.075921+0000","last_active":"2024-01-27T02:36:02.108543+0000","last_peered":"2024-01-27T02:36:02.108543+0000","last_clean":"2024-01-27T02:36:02.108543+0000","last_became_active":"2024-01-27T02:36:01.075647+0000","last_became_peered":"2024-01-27T02:36:01.075647+0000","last_unstale":"2024-01-27T02:36:02.108543+0000","last_undegraded":"2024-01-27T02:36:02.108543+0000","last_fullsized":"2024-01-27T02:36:02.108543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:54:58.712360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.098013+0000","last_change":"2024-01-27T02:36:02.097710+0000","last_active":"2024-01-27T02:36:02.098013+0000","last_peered":"2024-01-27T02:36:02.098013+0000","last_clean":"2024-01-27T02:36:02.098013+0000","last_became_active":"2024-01-27T02:36:02.096707+0000","last_became_peered":"2024-01-27T02:36:02.096707+0000","last_unstale":"2024-01-27T02:36:02.098013+0000","last_undegraded":"2024-01-27T02:36:02.098013+0000","last_fullsized":"2024-01-27T02:36:02.098013+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:13:16.927847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"80'6","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.111701+0000","last_change":"2024-01-27T02:36:01.085134+0000","last_active":"2024-01-27T02:36:02.111701+0000","last_peered":"2024-01-27T02:36:02.111701+0000","last_clean":"2024-01-27T02:36:02.111701+0000","last_became_active":"2024-01-27T02:36:01.084887+0000","last_became_peered":"2024-01-27T02:36:01.084887+0000","last_unstale":"2024-01-27T02:36:02.111701+0000","last_undegraded":"2024-01-27T02:36:02.111701+0000","last_fullsized":"2024-01-27T02:36:02.111701+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:24:30.352994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.102647+0000","last_change":"2024-01-27T02:36:02.102590+0000","last_active":"2024-01-27T02:36:02.102647+0000","last_peered":"2024-01-27T02:36:02.102647+0000","last_clean":"2024-01-27T02:36:02.102647+0000","last_became_active":"2024-01-27T02:36:02.102401+0000","last_became_peered":"2024-01-27T02:36:02.102401+0000","last_unstale":"2024-01-27T02:36:02.102647+0000","last_undegraded":"2024-01-27T02:36:02.102647+0000","last_fullsized":"2024-01-27T02:36:02.102647+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:37:50.389419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"80'2","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.109329+0000","last_change":"2024-01-27T02:36:01.081854+0000","last_active":"2024-01-27T02:36:02.109329+0000","last_peered":"2024-01-27T02:36:02.109329+0000","last_clean":"2024-01-27T02:36:02.109329+0000","last_became_active":"2024-01-27T02:36:01.081553+0000","last_became_peered":"2024-01-27T02:36:01.081553+0000","last_unstale":"2024-01-27T02:36:02.109329+0000","last_undegraded":"2024-01-27T02:36:02.109329+0000","last_fullsized":"2024-01-27T02:36:02.109329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:36:51.217544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.102016+0000","last_change":"2024-01-27T02:36:02.101950+0000","last_active":"2024-01-27T02:36:02.102016+0000","last_peered":"2024-01-27T02:36:02.102016+0000","last_clean":"2024-01-27T02:36:02.102016+0000","last_became_active":"2024-01-27T02:36:02.101724+0000","last_became_peered":"2024-01-27T02:36:02.101724+0000","last_unstale":"2024-01-27T02:36:02.102016+0000","last_undegraded":"2024-01-27T02:36:02.102016+0000","last_fullsized":"2024-01-27T02:36:02.102016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:42:07.113277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.088999+0000","last_change":"2024-01-27T02:36:01.087113+0000","last_active":"2024-01-27T02:36:02.088999+0000","last_peered":"2024-01-27T02:36:02.088999+0000","last_clean":"2024-01-27T02:36:02.088999+0000","last_became_active":"2024-01-27T02:36:01.086735+0000","last_became_peered":"2024-01-27T02:36:01.086735+0000","last_unstale":"2024-01-27T02:36:02.088999+0000","last_undegraded":"2024-01-27T02:36:02.088999+0000","last_fullsized":"2024-01-27T02:36:02.088999+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:51:30.355507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.096777+0000","last_change":"2024-01-27T02:36:02.096670+0000","last_active":"2024-01-27T02:36:02.096777+0000","last_peered":"2024-01-27T02:36:02.096777+0000","last_clean":"2024-01-27T02:36:02.096777+0000","last_became_active":"2024-01-27T02:36:02.096070+0000","last_became_peered":"2024-01-27T02:36:02.096070+0000","last_unstale":"2024-01-27T02:36:02.096777+0000","last_undegraded":"2024-01-27T02:36:02.096777+0000","last_fullsized":"2024-01-27T02:36:02.096777+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:48:39.299949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.094742+0000","last_change":"2024-01-27T02:36:01.079455+0000","last_active":"2024-01-27T02:36:02.094742+0000","last_peered":"2024-01-27T02:36:02.094742+0000","last_clean":"2024-01-27T02:36:02.094742+0000","last_became_active":"2024-01-27T02:36:01.078810+0000","last_became_peered":"2024-01-27T02:36:01.078810+0000","last_unstale":"2024-01-27T02:36:02.094742+0000","last_undegraded":"2024-01-27T02:36:02.094742+0000","last_fullsized":"2024-01-27T02:36:02.094742+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:32:57.446408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.100189+0000","last_change":"2024-01-27T02:36:02.099873+0000","last_active":"2024-01-27T02:36:02.100189+0000","last_peered":"2024-01-27T02:36:02.100189+0000","last_clean":"2024-01-27T02:36:02.100189+0000","last_became_active":"2024-01-27T02:36:02.099126+0000","last_became_peered":"2024-01-27T02:36:02.099126+0000","last_unstale":"2024-01-27T02:36:02.100189+0000","last_undegraded":"2024-01-27T02:36:02.100189+0000","last_fullsized":"2024-01-27T02:36:02.100189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:43:40.714127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"80'8","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.112850+0000","last_change":"2024-01-27T02:36:01.083387+0000","last_active":"2024-01-27T02:36:02.112850+0000","last_peered":"2024-01-27T02:36:02.112850+0000","last_clean":"2024-01-27T02:36:02.112850+0000","last_became_active":"2024-01-27T02:36:01.083023+0000","last_became_peered":"2024-01-27T02:36:01.083023+0000","last_unstale":"2024-01-27T02:36:02.112850+0000","last_undegraded":"2024-01-27T02:36:02.112850+0000","last_fullsized":"2024-01-27T02:36:02.112850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:16:41.009300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.097713+0000","last_change":"2024-01-27T02:36:02.097609+0000","last_active":"2024-01-27T02:36:02.097713+0000","last_peered":"2024-01-27T02:36:02.097713+0000","last_clean":"2024-01-27T02:36:02.097713+0000","last_became_active":"2024-01-27T02:36:02.097211+0000","last_became_peered":"2024-01-27T02:36:02.097211+0000","last_unstale":"2024-01-27T02:36:02.097713+0000","last_undegraded":"2024-01-27T02:36:02.097713+0000","last_fullsized":"2024-01-27T02:36:02.097713+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:04:07.439211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"80'1","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.110341+0000","last_change":"2024-01-27T02:36:01.075386+0000","last_active":"2024-01-27T02:36:02.110341+0000","last_peered":"2024-01-27T02:36:02.110341+0000","last_clean":"2024-01-27T02:36:02.110341+0000","last_became_active":"2024-01-27T02:36:01.074276+0000","last_became_peered":"2024-01-27T02:36:01.074276+0000","last_unstale":"2024-01-27T02:36:02.110341+0000","last_undegraded":"2024-01-27T02:36:02.110341+0000","last_fullsized":"2024-01-27T02:36:02.110341+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:46:34.597904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.100508+0000","last_change":"2024-01-27T02:36:02.100373+0000","last_active":"2024-01-27T02:36:02.100508+0000","last_peered":"2024-01-27T02:36:02.100508+0000","last_clean":"2024-01-27T02:36:02.100508+0000","last_became_active":"2024-01-27T02:36:02.100122+0000","last_became_peered":"2024-01-27T02:36:02.100122+0000","last_unstale":"2024-01-27T02:36:02.100508+0000","last_undegraded":"2024-01-27T02:36:02.100508+0000","last_fullsized":"2024-01-27T02:36:02.100508+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:51:06.991922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"80'2","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.111523+0000","last_change":"2024-01-27T02:36:01.087262+0000","last_active":"2024-01-27T02:36:02.111523+0000","last_peered":"2024-01-27T02:36:02.111523+0000","last_clean":"2024-01-27T02:36:02.111523+0000","last_became_active":"2024-01-27T02:36:01.086163+0000","last_became_peered":"2024-01-27T02:36:01.086163+0000","last_unstale":"2024-01-27T02:36:02.111523+0000","last_undegraded":"2024-01-27T02:36:02.111523+0000","last_fullsized":"2024-01-27T02:36:02.111523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:35:13.192349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.098961+0000","last_change":"2024-01-27T02:36:02.098722+0000","last_active":"2024-01-27T02:36:02.098961+0000","last_peered":"2024-01-27T02:36:02.098961+0000","last_clean":"2024-01-27T02:36:02.098961+0000","last_became_active":"2024-01-27T02:36:02.098241+0000","last_became_peered":"2024-01-27T02:36:02.098241+0000","last_unstale":"2024-01-27T02:36:02.098961+0000","last_undegraded":"2024-01-27T02:36:02.098961+0000","last_fullsized":"2024-01-27T02:36:02.098961+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:51:31.361728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"80'1","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.110945+0000","last_change":"2024-01-27T02:36:01.086968+0000","last_active":"2024-01-27T02:36:02.110945+0000","last_peered":"2024-01-27T02:36:02.110945+0000","last_clean":"2024-01-27T02:36:02.110945+0000","last_became_active":"2024-01-27T02:36:01.086112+0000","last_became_peered":"2024-01-27T02:36:01.086112+0000","last_unstale":"2024-01-27T02:36:02.110945+0000","last_undegraded":"2024-01-27T02:36:02.110945+0000","last_fullsized":"2024-01-27T02:36:02.110945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:03:55.891925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.097350+0000","last_change":"2024-01-27T02:36:02.097268+0000","last_active":"2024-01-27T02:36:02.097350+0000","last_peered":"2024-01-27T02:36:02.097350+0000","last_clean":"2024-01-27T02:36:02.097350+0000","last_became_active":"2024-01-27T02:36:02.097011+0000","last_became_peered":"2024-01-27T02:36:02.097011+0000","last_unstale":"2024-01-27T02:36:02.097350+0000","last_undegraded":"2024-01-27T02:36:02.097350+0000","last_fullsized":"2024-01-27T02:36:02.097350+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:14:57.057715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.090638+0000","last_change":"2024-01-27T02:36:01.085172+0000","last_active":"2024-01-27T02:36:02.090638+0000","last_peered":"2024-01-27T02:36:02.090638+0000","last_clean":"2024-01-27T02:36:02.090638+0000","last_became_active":"2024-01-27T02:36:01.084894+0000","last_became_peered":"2024-01-27T02:36:01.084894+0000","last_unstale":"2024-01-27T02:36:02.090638+0000","last_undegraded":"2024-01-27T02:36:02.090638+0000","last_fullsized":"2024-01-27T02:36:02.090638+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:19:37.782912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.091427+0000","last_change":"2024-01-27T02:36:02.091309+0000","last_active":"2024-01-27T02:36:02.091427+0000","last_peered":"2024-01-27T02:36:02.091427+0000","last_clean":"2024-01-27T02:36:02.091427+0000","last_became_active":"2024-01-27T02:36:02.091070+0000","last_became_peered":"2024-01-27T02:36:02.091070+0000","last_unstale":"2024-01-27T02:36:02.091427+0000","last_undegraded":"2024-01-27T02:36:02.091427+0000","last_fullsized":"2024-01-27T02:36:02.091427+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:55.624218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.088430+0000","last_change":"2024-01-27T02:36:01.080657+0000","last_active":"2024-01-27T02:36:02.088430+0000","last_peered":"2024-01-27T02:36:02.088430+0000","last_clean":"2024-01-27T02:36:02.088430+0000","last_became_active":"2024-01-27T02:36:01.078442+0000","last_became_peered":"2024-01-27T02:36:01.078442+0000","last_unstale":"2024-01-27T02:36:02.088430+0000","last_undegraded":"2024-01-27T02:36:02.088430+0000","last_fullsized":"2024-01-27T02:36:02.088430+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:45:47.314805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.096241+0000","last_change":"2024-01-27T02:36:02.096166+0000","last_active":"2024-01-27T02:36:02.096241+0000","last_peered":"2024-01-27T02:36:02.096241+0000","last_clean":"2024-01-27T02:36:02.096241+0000","last_became_active":"2024-01-27T02:36:02.095902+0000","last_became_peered":"2024-01-27T02:36:02.095902+0000","last_unstale":"2024-01-27T02:36:02.096241+0000","last_undegraded":"2024-01-27T02:36:02.096241+0000","last_fullsized":"2024-01-27T02:36:02.096241+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:48:08.841511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.092336+0000","last_change":"2024-01-27T02:36:01.085914+0000","last_active":"2024-01-27T02:36:02.092336+0000","last_peered":"2024-01-27T02:36:02.092336+0000","last_clean":"2024-01-27T02:36:02.092336+0000","last_became_active":"2024-01-27T02:36:01.085674+0000","last_became_peered":"2024-01-27T02:36:01.085674+0000","last_unstale":"2024-01-27T02:36:02.092336+0000","last_undegraded":"2024-01-27T02:36:02.092336+0000","last_fullsized":"2024-01-27T02:36:02.092336+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:19:37.782912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.093670+0000","last_change":"2024-01-27T02:36:02.092888+0000","last_active":"2024-01-27T02:36:02.093670+0000","last_peered":"2024-01-27T02:36:02.093670+0000","last_clean":"2024-01-27T02:36:02.093670+0000","last_became_active":"2024-01-27T02:36:02.092614+0000","last_became_peered":"2024-01-27T02:36:02.092614+0000","last_unstale":"2024-01-27T02:36:02.093670+0000","last_undegraded":"2024-01-27T02:36:02.093670+0000","last_fullsized":"2024-01-27T02:36:02.093670+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:18:39.999190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.091927+0000","last_change":"2024-01-27T02:36:01.086128+0000","last_active":"2024-01-27T02:36:02.091927+0000","last_peered":"2024-01-27T02:36:02.091927+0000","last_clean":"2024-01-27T02:36:02.091927+0000","last_became_active":"2024-01-27T02:36:01.085891+0000","last_became_peered":"2024-01-27T02:36:01.085891+0000","last_unstale":"2024-01-27T02:36:02.091927+0000","last_undegraded":"2024-01-27T02:36:02.091927+0000","last_fullsized":"2024-01-27T02:36:02.091927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:05:15.222635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.097780+0000","last_change":"2024-01-27T02:36:02.097572+0000","last_active":"2024-01-27T02:36:02.097780+0000","last_peered":"2024-01-27T02:36:02.097780+0000","last_clean":"2024-01-27T02:36:02.097780+0000","last_became_active":"2024-01-27T02:36:02.096958+0000","last_became_peered":"2024-01-27T02:36:02.096958+0000","last_unstale":"2024-01-27T02:36:02.097780+0000","last_undegraded":"2024-01-27T02:36:02.097780+0000","last_fullsized":"2024-01-27T02:36:02.097780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:41:50.546172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"80'2","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.109071+0000","last_change":"2024-01-27T02:36:01.079712+0000","last_active":"2024-01-27T02:36:02.109071+0000","last_peered":"2024-01-27T02:36:02.109071+0000","last_clean":"2024-01-27T02:36:02.109071+0000","last_became_active":"2024-01-27T02:36:01.079176+0000","last_became_peered":"2024-01-27T02:36:01.079176+0000","last_unstale":"2024-01-27T02:36:02.109071+0000","last_undegraded":"2024-01-27T02:36:02.109071+0000","last_fullsized":"2024-01-27T02:36:02.109071+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:12:59.343148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.101156+0000","last_change":"2024-01-27T02:36:02.100955+0000","last_active":"2024-01-27T02:36:02.101156+0000","last_peered":"2024-01-27T02:36:02.101156+0000","last_clean":"2024-01-27T02:36:02.101156+0000","last_became_active":"2024-01-27T02:36:02.100449+0000","last_became_peered":"2024-01-27T02:36:02.100449+0000","last_unstale":"2024-01-27T02:36:02.101156+0000","last_undegraded":"2024-01-27T02:36:02.101156+0000","last_fullsized":"2024-01-27T02:36:02.101156+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:12:02.170281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"81'1","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-01-27T02:36:03.406250+0000","last_change":"2024-01-27T02:36:01.084252+0000","last_active":"2024-01-27T02:36:03.406250+0000","last_peered":"2024-01-27T02:36:03.406250+0000","last_clean":"2024-01-27T02:36:03.406250+0000","last_became_active":"2024-01-27T02:36:01.083785+0000","last_became_peered":"2024-01-27T02:36:01.083785+0000","last_unstale":"2024-01-27T02:36:03.406250+0000","last_undegraded":"2024-01-27T02:36:03.406250+0000","last_fullsized":"2024-01-27T02:36:03.406250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:03:15.272806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.103977+0000","last_change":"2024-01-27T02:36:02.103785+0000","last_active":"2024-01-27T02:36:02.103977+0000","last_peered":"2024-01-27T02:36:02.103977+0000","last_clean":"2024-01-27T02:36:02.103977+0000","last_became_active":"2024-01-27T02:36:02.103538+0000","last_became_peered":"2024-01-27T02:36:02.103538+0000","last_unstale":"2024-01-27T02:36:02.103977+0000","last_undegraded":"2024-01-27T02:36:02.103977+0000","last_fullsized":"2024-01-27T02:36:02.103977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:18:51.540850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.092358+0000","last_change":"2024-01-27T02:36:01.083357+0000","last_active":"2024-01-27T02:36:02.092358+0000","last_peered":"2024-01-27T02:36:02.092358+0000","last_clean":"2024-01-27T02:36:02.092358+0000","last_became_active":"2024-01-27T02:36:01.083009+0000","last_became_peered":"2024-01-27T02:36:01.083009+0000","last_unstale":"2024-01-27T02:36:02.092358+0000","last_undegraded":"2024-01-27T02:36:02.092358+0000","last_fullsized":"2024-01-27T02:36:02.092358+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:17:17.877802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.100492+0000","last_change":"2024-01-27T02:36:02.099832+0000","last_active":"2024-01-27T02:36:02.100492+0000","last_peered":"2024-01-27T02:36:02.100492+0000","last_clean":"2024-01-27T02:36:02.100492+0000","last_became_active":"2024-01-27T02:36:02.099223+0000","last_became_peered":"2024-01-27T02:36:02.099223+0000","last_unstale":"2024-01-27T02:36:02.100492+0000","last_undegraded":"2024-01-27T02:36:02.100492+0000","last_fullsized":"2024-01-27T02:36:02.100492+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:37:14.931166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.092093+0000","last_change":"2024-01-27T02:36:01.083453+0000","last_active":"2024-01-27T02:36:02.092093+0000","last_peered":"2024-01-27T02:36:02.092093+0000","last_clean":"2024-01-27T02:36:02.092093+0000","last_became_active":"2024-01-27T02:36:01.083173+0000","last_became_peered":"2024-01-27T02:36:01.083173+0000","last_unstale":"2024-01-27T02:36:02.092093+0000","last_undegraded":"2024-01-27T02:36:02.092093+0000","last_fullsized":"2024-01-27T02:36:02.092093+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:21:40.093915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.100167+0000","last_change":"2024-01-27T02:36:02.099715+0000","last_active":"2024-01-27T02:36:02.100167+0000","last_peered":"2024-01-27T02:36:02.100167+0000","last_clean":"2024-01-27T02:36:02.100167+0000","last_became_active":"2024-01-27T02:36:02.098796+0000","last_became_peered":"2024-01-27T02:36:02.098796+0000","last_unstale":"2024-01-27T02:36:02.100167+0000","last_undegraded":"2024-01-27T02:36:02.100167+0000","last_fullsized":"2024-01-27T02:36:02.100167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:40:44.218390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.093872+0000","last_change":"2024-01-27T02:36:01.086016+0000","last_active":"2024-01-27T02:36:02.093872+0000","last_peered":"2024-01-27T02:36:02.093872+0000","last_clean":"2024-01-27T02:36:02.093872+0000","last_became_active":"2024-01-27T02:36:01.085747+0000","last_became_peered":"2024-01-27T02:36:01.085747+0000","last_unstale":"2024-01-27T02:36:02.093872+0000","last_undegraded":"2024-01-27T02:36:02.093872+0000","last_fullsized":"2024-01-27T02:36:02.093872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:03:04.731782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.101466+0000","last_change":"2024-01-27T02:36:02.101216+0000","last_active":"2024-01-27T02:36:02.101466+0000","last_peered":"2024-01-27T02:36:02.101466+0000","last_clean":"2024-01-27T02:36:02.101466+0000","last_became_active":"2024-01-27T02:36:02.100632+0000","last_became_peered":"2024-01-27T02:36:02.100632+0000","last_unstale":"2024-01-27T02:36:02.101466+0000","last_undegraded":"2024-01-27T02:36:02.101466+0000","last_fullsized":"2024-01-27T02:36:02.101466+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:05:16.228857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.088965+0000","last_change":"2024-01-27T02:36:01.085778+0000","last_active":"2024-01-27T02:36:02.088965+0000","last_peered":"2024-01-27T02:36:02.088965+0000","last_clean":"2024-01-27T02:36:02.088965+0000","last_became_active":"2024-01-27T02:36:01.085504+0000","last_became_peered":"2024-01-27T02:36:01.085504+0000","last_unstale":"2024-01-27T02:36:02.088965+0000","last_undegraded":"2024-01-27T02:36:02.088965+0000","last_fullsized":"2024-01-27T02:36:02.088965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:34:00.107552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.089533+0000","last_change":"2024-01-27T02:36:02.089392+0000","last_active":"2024-01-27T02:36:02.089533+0000","last_peered":"2024-01-27T02:36:02.089533+0000","last_clean":"2024-01-27T02:36:02.089533+0000","last_became_active":"2024-01-27T02:36:02.089117+0000","last_became_peered":"2024-01-27T02:36:02.089117+0000","last_unstale":"2024-01-27T02:36:02.089533+0000","last_undegraded":"2024-01-27T02:36:02.089533+0000","last_fullsized":"2024-01-27T02:36:02.089533+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:15:05.885263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.091855+0000","last_change":"2024-01-27T02:36:01.083238+0000","last_active":"2024-01-27T02:36:02.091855+0000","last_peered":"2024-01-27T02:36:02.091855+0000","last_clean":"2024-01-27T02:36:02.091855+0000","last_became_active":"2024-01-27T02:36:01.082768+0000","last_became_peered":"2024-01-27T02:36:01.082768+0000","last_unstale":"2024-01-27T02:36:02.091855+0000","last_undegraded":"2024-01-27T02:36:02.091855+0000","last_fullsized":"2024-01-27T02:36:02.091855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:34:52.370887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.090788+0000","last_change":"2024-01-27T02:36:02.089283+0000","last_active":"2024-01-27T02:36:02.090788+0000","last_peered":"2024-01-27T02:36:02.090788+0000","last_clean":"2024-01-27T02:36:02.090788+0000","last_became_active":"2024-01-27T02:36:02.089044+0000","last_became_peered":"2024-01-27T02:36:02.089044+0000","last_unstale":"2024-01-27T02:36:02.090788+0000","last_undegraded":"2024-01-27T02:36:02.090788+0000","last_fullsized":"2024-01-27T02:36:02.090788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:32:58.452630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.088515+0000","last_change":"2024-01-27T02:36:01.078683+0000","last_active":"2024-01-27T02:36:02.088515+0000","last_peered":"2024-01-27T02:36:02.088515+0000","last_clean":"2024-01-27T02:36:02.088515+0000","last_became_active":"2024-01-27T02:36:01.078126+0000","last_became_peered":"2024-01-27T02:36:01.078126+0000","last_unstale":"2024-01-27T02:36:02.088515+0000","last_undegraded":"2024-01-27T02:36:02.088515+0000","last_fullsized":"2024-01-27T02:36:02.088515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:10:25.706212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.093699+0000","last_change":"2024-01-27T02:36:02.093378+0000","last_active":"2024-01-27T02:36:02.093699+0000","last_peered":"2024-01-27T02:36:02.093699+0000","last_clean":"2024-01-27T02:36:02.093699+0000","last_became_active":"2024-01-27T02:36:02.092243+0000","last_became_peered":"2024-01-27T02:36:02.092243+0000","last_unstale":"2024-01-27T02:36:02.093699+0000","last_undegraded":"2024-01-27T02:36:02.093699+0000","last_fullsized":"2024-01-27T02:36:02.093699+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:01.063602+0000","last_clean_scrub_stamp":"2024-01-27T02:36:01.063602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:44:17.898667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"80'2","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.110306+0000","last_change":"2024-01-27T02:36:01.080741+0000","last_active":"2024-01-27T02:36:02.110306+0000","last_peered":"2024-01-27T02:36:02.110306+0000","last_clean":"2024-01-27T02:36:02.110306+0000","last_became_active":"2024-01-27T02:36:01.079822+0000","last_became_peered":"2024-01-27T02:36:01.079822+0000","last_unstale":"2024-01-27T02:36:02.110306+0000","last_undegraded":"2024-01-27T02:36:02.110306+0000","last_fullsized":"2024-01-27T02:36:02.110306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:36:00.057381+0000","last_clean_scrub_stamp":"2024-01-27T02:36:00.057381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:23:56.398583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":213,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-01-27T02:36:02.096068+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:36:02.096068+0000","last_peered":"2024-01-27T02:36:02.096068+0000","last_clean":"2024-01-27T02:36:02.096068+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:36:02.096068+0000","last_undegraded":"2024-01-27T02:36:02.096068+0000","last_fullsized":"2024-01-27T02:36:02.096068+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":992,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194320,"statfs":{"total":200000000,"available":198983702,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.45000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71599999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.46800000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.78400000000000003}]},{"osd":1,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.435,"5min":0.373,"15min":0.373},"max":{"1min":0.71899999999999997,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.438,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.70599999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.66300000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.45800000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.72299999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.41499999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70499999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.64300000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.73499999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.72499999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.67200000000000004}]},{"osd":4,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48299999999999998,"15min":0.48999999999999999},"min":{"1min":0.29299999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66500000000000004,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.502},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.504,"15min":0.51600000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73399999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.56399999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.52300000000000002,"15min":0.53700000000000003},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.72099999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55400000000000005,"15min":0.56499999999999995},"min":{"1min":0.38700000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73499999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.58899999999999997}]},{"osd":6,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.58399999999999996,"15min":0.59799999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73199999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56799999999999995,"15min":0.57799999999999996},"min":{"1min":0.40500000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76700000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":415,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194897,"statfs":{"total":200000000,"available":199574566,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.56000000000000005,"15min":0.51200000000000001},"min":{"1min":0.434,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79700000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55800000000000005,"15min":0.52000000000000002},"min":{"1min":0.35999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.61599999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.55700000000000005,"15min":0.51000000000000001},"min":{"1min":0.42599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76500000000000001,"5min":0.874,"15min":0.874},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58699999999999997,"15min":0.54900000000000004},"min":{"1min":0.44500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82199999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.52100000000000002}]},{"osd":2,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56599999999999995,"15min":0.53900000000000003},"min":{"1min":0.46999999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73999999999999999,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.61099999999999999,"15min":0.57799999999999996},"min":{"1min":0.52200000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80400000000000005,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.57499999999999996}]},{"osd":3,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.48799999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.74299999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.56699999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.78400000000000003,"5min":1.262,"15min":1.262},"last":0.63600000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.36599999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70299999999999996,"5min":0.748,"15min":0.748},"last":0.75},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53600000000000003,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70299999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.65000000000000002}]},{"osd":5,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52900000000000003,"15min":0.53400000000000003},"min":{"1min":0.42199999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65600000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58699999999999997,"15min":0.60899999999999999},"min":{"1min":0.41799999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76700000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.69299999999999995}]},{"osd":7,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.61699999999999999},"min":{"1min":0.39300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.73499999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57799999999999996,"15min":0.59199999999999997},"min":{"1min":0.38500000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71599999999999997,"5min":1.038,"15min":1.038},"last":0.68200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":417,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194895,"statfs":{"total":200000000,"available":199572534,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.38600000000000001,"15min":0.373},"min":{"1min":0.307,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.55900000000000005,"5min":0.626,"15min":0.626},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40699999999999997,"15min":0.39500000000000002},"min":{"1min":0.28699999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.57099999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.38300000000000001}]},{"osd":2,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.36099999999999999},"min":{"1min":0.29599999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.60099999999999998,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.442,"15min":0.42699999999999999},"min":{"1min":0.35499999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65900000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.434}]},{"osd":3,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48099999999999998,"15min":0.46600000000000003},"min":{"1min":0.36599999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.501,"5min":0.46300000000000002,"15min":0.45400000000000001},"min":{"1min":0.34399999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.68700000000000006,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.45100000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.63200000000000001,"15min":0.624},"min":{"1min":0.48599999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.96799999999999997,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.64600000000000002,"15min":0.63200000000000001},"min":{"1min":0.51700000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88800000000000001,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.625}]},{"osd":5,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63800000000000001,"15min":0.622},"min":{"1min":0.47499999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.93400000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.66300000000000003,"15min":0.66700000000000004},"min":{"1min":0.47299999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.70499999999999996}]},{"osd":6,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.70299999999999996,"15min":0.69399999999999995},"min":{"1min":0.55100000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.95699999999999996,"5min":1.024,"15min":1.024},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.68400000000000005,"15min":0.67000000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.90300000000000002,"5min":1.151,"15min":1.151},"last":0.67900000000000005}]},{"osd":7,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.69799999999999995,"15min":0.67800000000000005},"min":{"1min":0.495,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88,"5min":1.097,"15min":1.097},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.70299999999999996,"15min":0.68500000000000005},"min":{"1min":0.54600000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.65900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":991,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194320,"statfs":{"total":200000000,"available":198984290,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.39100000000000001,"15min":0.38900000000000001},"min":{"1min":0.28799999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.59799999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.30299999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.624,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.48399999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.42799999999999999,"15min":0.41899999999999998},"min":{"1min":0.33900000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.441,"15min":0.432},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58699999999999997,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.40300000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.42899999999999999,"15min":0.43099999999999999},"min":{"1min":0.33100000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61799999999999999,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.377},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.47499999999999998,"15min":0.47099999999999997},"min":{"1min":0.40600000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42399999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.51200000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.878,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65600000000000003,"15min":0.65800000000000003},"min":{"1min":0.51900000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54700000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66500000000000004,"15min":0.65300000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0029999999999999,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.56999999999999995,"5min":0.44,"15min":0.44},"max":{"1min":0.90500000000000003,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.61499999999999999}]},{"osd":6,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.66400000000000003,"15min":0.66300000000000003},"min":{"1min":0.52500000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89200000000000002,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68300000000000005,"15min":0.67600000000000005},"min":{"1min":0.54900000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.65900000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68700000000000006,"15min":0.69099999999999995},"min":{"1min":0.54300000000000004,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.98599999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68700000000000006,"15min":0.69299999999999995},"min":{"1min":0.499,"5min":0.498,"15min":0.498},"max":{"1min":0.92300000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.499}]}]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":416,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194895,"statfs":{"total":200000000,"available":199573122,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41799999999999998,"15min":0.40999999999999998},"min":{"1min":0.26400000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56499999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.55500000000000005,"15min":0.65300000000000002},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61799999999999999,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":0.47899999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42099999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.626,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.443,"15min":0.44800000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.62,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.39400000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.42099999999999999,"15min":0.41499999999999998},"min":{"1min":0.313,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.56299999999999994,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.502},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.64900000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.44800000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.51200000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89800000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75700000000000001,"15min":0.80700000000000005},"min":{"1min":0.56399999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.91000000000000003,"5min":3.581,"15min":3.581},"last":0.71599999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69999999999999996,"15min":0.751},"min":{"1min":0.52500000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84799999999999998,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67800000000000005,"15min":0.67100000000000004},"min":{"1min":0.52700000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92000000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.621}]},{"osd":6,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.52600000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89100000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.67800000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94999999999999996,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.70199999999999996}]},{"osd":7,"last update":"Sat Jan 27 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.79900000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69199999999999995,"15min":0.68799999999999994},"min":{"1min":0.47499999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.67300000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":415,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194897,"statfs":{"total":200000000,"available":199574546,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.376,"15min":0.35499999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.57299999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.255},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40000000000000002,"15min":0.39300000000000002},"min":{"1min":0.29099999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66900000000000004,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.318}]},{"osd":1,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.40699999999999997,"15min":0.39100000000000001},"min":{"1min":0.317,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.437,"15min":0.41799999999999998},"min":{"1min":0.38800000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.30199999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.44600000000000001,"15min":0.42599999999999999},"min":{"1min":0.36199999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.46899999999999997,"15min":0.44500000000000001},"min":{"1min":0.35299999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.436}]},{"osd":4,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.433,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.628,"15min":0.60199999999999998},"min":{"1min":0.504,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.47899999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.63400000000000001,"15min":0.60899999999999999},"min":{"1min":0.55700000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.51000000000000001,"5min":0.496,"15min":0.496},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.46899999999999997}]},{"osd":6,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.65900000000000003,"15min":0.65300000000000002},"min":{"1min":0.496,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64800000000000002,"15min":0.64300000000000002},"min":{"1min":0.503,"5min":0.435,"15min":0.435},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.49299999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.65800000000000003,"15min":0.63400000000000001},"min":{"1min":0.52900000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.49399999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94299999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.50900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":415,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194897,"statfs":{"total":200000000,"available":199574588,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.47299999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.73499999999999999,"5min":1.099,"15min":1.099},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.45000000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73499999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.58899999999999997}]},{"osd":1,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.48999999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0309999999999999,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81999999999999995,"5min":1.133,"15min":1.133},"last":0.60299999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60999999999999999,"15min":0.59699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.3400000000000001,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.626},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.46400000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79800000000000004,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.52000000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.47699999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.624,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.44900000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82099999999999995,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.57299999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.33700000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32800000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.495}]},{"osd":6,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.501,"15min":0.499},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68500000000000005,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38600000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.64200000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54600000000000004,"15min":0.55200000000000005},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79000000000000004,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.66000000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":415,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194897,"statfs":{"total":200000000,"available":199574568,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:36:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58799999999999997,"15min":0.59999999999999998},"min":{"1min":0.41799999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.86499999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.443},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60699999999999998,"15min":0.57899999999999996},"min":{"1min":0.44600000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71699999999999997,"5min":3.242,"15min":3.242},"last":0.52900000000000003}]},{"osd":1,"last update":"Sat Jan 27 02:36:02 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.629,"15min":0.60899999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77200000000000002,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.64400000000000002,"15min":0.65500000000000003},"min":{"1min":0.48399999999999999,"5min":0.436,"15min":0.436},"max":{"1min":0.73699999999999999,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.58699999999999997}]},{"osd":2,"last update":"Sat Jan 27 02:36:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.62,"15min":0.59799999999999998},"min":{"1min":0.46100000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.76600000000000001,"5min":3.04,"15min":3.04},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59599999999999997,"15min":0.57799999999999996},"min":{"1min":0.441,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72199999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.46899999999999997}]},{"osd":3,"last update":"Sat Jan 27 02:36:02 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.68899999999999995,"15min":0.68600000000000005},"min":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76600000000000001,"5min":4.032,"15min":4.032},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.71099999999999997,"15min":0.68100000000000005},"min":{"1min":0.55500000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.77900000000000003,"5min":4.282,"15min":4.282},"last":0.63900000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:36:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52400000000000002,"15min":0.52600000000000002},"min":{"1min":0.34399999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.69999999999999996,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.53400000000000003,"15min":0.57199999999999995},"min":{"1min":0.33200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.75,"5min":1.101,"15min":1.101},"last":0.51200000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:36:02 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53400000000000003,"15min":0.56299999999999994},"min":{"1min":0.38200000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65400000000000003,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.75900000000000001,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.55000000000000004}]},{"osd":7,"last update":"Sat Jan 27 02:36:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.46200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.70299999999999996,"5min":1.008,"15min":1.008},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64700000000000002,"15min":0.66500000000000004},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75,"5min":2,"15min":2},"last":0.66100000000000003}]}]}],"pool_statfs":[]}} 2024-01-27T02:36:11.360 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-27T02:36:11.360 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-27T02:36:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.537+0000 7f28d0830700 1 Processor -- start 2024-01-27T02:36:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.537+0000 7f28d0830700 1 -- start start 2024-01-27T02:36:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.538+0000 7f28d0830700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f28cc156b60 0x7f28cc0ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.538+0000 7f28d0830700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28cc156f30 0x7f28cc15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.538+0000 7f28d0830700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28cc0aeaa0 con 0x7f28cc0ad9f0 2024-01-27T02:36:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.538+0000 7f28d0830700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28cc0ade90 con 0x7f28cc156b60 2024-01-27T02:36:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.538+0000 7f28d0830700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc0ae170 con 0x7f28cc156f30 2024-01-27T02:36:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.538+0000 7f28c3fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f28cc0ad9f0 0x7f28cc0aef30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39234/0 (socket says 172.21.15.7:39234) 2024-01-27T02:36:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.538+0000 7f28c3fff700 1 -- 172.21.15.7:0/3785785684 learned_addr learned my addr 172.21.15.7:0/3785785684 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.538+0000 7f28cad9d700 1 --2- 172.21.15.7:0/3785785684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28cc156f30 0x7f28cc15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.538+0000 7f28ca59c700 1 --2- 172.21.15.7:0/3785785684 >> v2:172.21.15.188:3300/0 conn(0x7f28cc156b60 0x7f28cc0ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28c9d9b700 1 -- 172.21.15.7:0/3785785684 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1555007345 0 0) 0x7f28cc0aeaa0 con 0x7f28cc0ad9f0 2024-01-27T02:36:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28c9d9b700 1 -- 172.21.15.7:0/3785785684 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28b8003570 con 0x7f28cc0ad9f0 2024-01-27T02:36:11.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28c9d9b700 1 -- 172.21.15.7:0/3785785684 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3591263715 0 0) 0x7f28b8003570 con 0x7f28cc0ad9f0 2024-01-27T02:36:11.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28c9d9b700 1 -- 172.21.15.7:0/3785785684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28cc156f30 msgr2=0x7f28cc15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:11.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28c9d9b700 1 --2- 172.21.15.7:0/3785785684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28cc156f30 0x7f28cc15a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28c9d9b700 1 -- 172.21.15.7:0/3785785684 >> v2:172.21.15.188:3300/0 conn(0x7f28cc156b60 msgr2=0x7f28cc0ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:11.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28c9d9b700 1 --2- 172.21.15.7:0/3785785684 >> v2:172.21.15.188:3300/0 conn(0x7f28cc156b60 0x7f28cc0ad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28c9d9b700 1 -- 172.21.15.7:0/3785785684 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc0ae450 con 0x7f28cc0ad9f0 2024-01-27T02:36:11.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28c9d9b700 1 -- 172.21.15.7:0/3785785684 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f28b4001470 con 0x7f28cc0ad9f0 2024-01-27T02:36:11.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28c9d9b700 1 -- 172.21.15.7:0/3785785684 >> v1:172.21.15.7:6789/0 conn(0x7f28cc0ad9f0 legacy=0x7f28cc0aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:11.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28c9d9b700 1 --2- 172.21.15.7:0/3785785684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28cc156f30 0x7f28b8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:11.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28c9d9b700 1 --2- 172.21.15.7:0/3785785684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28b8003fe0 0x7f28b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:11.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28c9d9b700 1 --2- 172.21.15.7:0/3785785684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28b8004d50 0x7f28b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:11.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28c9d9b700 1 -- 172.21.15.7:0/3785785684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc0ade90 con 0x7f28b8004d50 2024-01-27T02:36:11.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28c9d9b700 1 -- 172.21.15.7:0/3785785684 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc0ae170 con 0x7f28b8003fe0 2024-01-27T02:36:11.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28c9d9b700 1 -- 172.21.15.7:0/3785785684 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b800ba90 con 0x7f28cc156f30 2024-01-27T02:36:11.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28cad9d700 1 --2- 172.21.15.7:0/3785785684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28b8004d50 0x7f28b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:11.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28ca59c700 1 --2- 172.21.15.7:0/3785785684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28cc156f30 0x7f28b8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:11.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28c3fff700 1 --2- 172.21.15.7:0/3785785684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28b8003fe0 0x7f28b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:11.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28cad9d700 1 -- 172.21.15.7:0/3785785684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28cc156f30 msgr2=0x7f28b8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:11.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28cad9d700 1 --2- 172.21.15.7:0/3785785684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28cc156f30 0x7f28b8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28cad9d700 1 -- 172.21.15.7:0/3785785684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28b8003fe0 msgr2=0x7f28b8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:11.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28cad9d700 1 --2- 172.21.15.7:0/3785785684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28b8003fe0 0x7f28b8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.539+0000 7f28cad9d700 1 -- 172.21.15.7:0/3785785684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b8003570 con 0x7f28b8004d50 2024-01-27T02:36:11.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.540+0000 7f28cad9d700 1 --2- 172.21.15.7:0/3785785684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28b8004d50 0x7f28b80051c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f28c400e210 tx=0x7f28c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1084afd02c5294b8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:11.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.540+0000 7f28c9d9b700 1 -- 172.21.15.7:0/3785785684 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c400ea60 con 0x7f28b8004d50 2024-01-27T02:36:11.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.540+0000 7f28c9d9b700 1 -- 172.21.15.7:0/3785785684 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c400f040 con 0x7f28b8004d50 2024-01-27T02:36:11.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.540+0000 7f28c9d9b700 1 -- 172.21.15.7:0/3785785684 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c400ee10 con 0x7f28b8004d50 2024-01-27T02:36:11.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.540+0000 7f28d0830700 1 -- 172.21.15.7:0/3785785684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28b8004d50 msgr2=0x7f28b80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:11.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.540+0000 7f28d0830700 1 --2- 172.21.15.7:0/3785785684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28b8004d50 0x7f28b80051c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f28c400e210 tx=0x7f28c400b040 comp rx=0 tx=0).stop 2024-01-27T02:36:11.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.540+0000 7f28cad9d700 1 -- 172.21.15.7:0/3785785684 reap_dead start 2024-01-27T02:36:11.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.540+0000 7f28d0830700 1 -- 172.21.15.7:0/3785785684 shutdown_connections 2024-01-27T02:36:11.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.540+0000 7f28d0830700 1 -- 172.21.15.7:0/3785785684 >> 172.21.15.7:0/3785785684 conn(0x7f28cc068480 msgr2=0x7f28cc0cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:11.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.541+0000 7f28d0830700 1 -- 172.21.15.7:0/3785785684 shutdown_connections 2024-01-27T02:36:11.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.541+0000 7f28d0830700 1 -- 172.21.15.7:0/3785785684 wait complete. 2024-01-27T02:36:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.542+0000 7f28d0830700 1 Processor -- start 2024-01-27T02:36:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.542+0000 7f28d0830700 1 -- start start 2024-01-27T02:36:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.542+0000 7f28d0830700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28b8004d50 0x7f28cc14f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.542+0000 7f28d0830700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28cc13b4c0 0x7f28cc0a7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.542+0000 7f28d0830700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28cc13b930 0x7f28cc0a79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.543+0000 7f28d0830700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc15a7c0 con 0x7f28b8004d50 2024-01-27T02:36:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.543+0000 7f28d0830700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc14b1d0 con 0x7f28cc13b930 2024-01-27T02:36:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.543+0000 7f28d0830700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc15a200 con 0x7f28cc13b4c0 2024-01-27T02:36:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.543+0000 7f28ca59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28b8004d50 0x7f28cc14f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.543+0000 7f28ca59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28b8004d50 0x7f28cc14f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41634/0 (socket says 172.21.15.7:41634) 2024-01-27T02:36:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.543+0000 7f28ca59c700 1 -- 172.21.15.7:0/924446818 learned_addr learned my addr 172.21.15.7:0/924446818 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.543+0000 7f28c3fff700 1 --2- 172.21.15.7:0/924446818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28cc13b4c0 0x7f28cc0a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.543+0000 7f28cad9d700 1 --2- 172.21.15.7:0/924446818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28cc13b930 0x7f28cc0a79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.543+0000 7f28ca59c700 1 -- 172.21.15.7:0/924446818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28cc13b4c0 msgr2=0x7f28cc0a7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.543+0000 7f28ca59c700 1 --2- 172.21.15.7:0/924446818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28cc13b4c0 0x7f28cc0a7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.543+0000 7f28ca59c700 1 -- 172.21.15.7:0/924446818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28cc13b930 msgr2=0x7f28cc0a79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.543+0000 7f28ca59c700 1 --2- 172.21.15.7:0/924446818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28cc13b930 0x7f28cc0a79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.543+0000 7f28ca59c700 1 -- 172.21.15.7:0/924446818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc15a4e0 con 0x7f28b8004d50 2024-01-27T02:36:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.543+0000 7f28ca59c700 1 --2- 172.21.15.7:0/924446818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28b8004d50 0x7f28cc14f2d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f28bc006ac0 tx=0x7f28bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.543+0000 7f28c27fc700 1 -- 172.21.15.7:0/924446818 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc00fa10 con 0x7f28b8004d50 2024-01-27T02:36:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.543+0000 7f28c27fc700 1 -- 172.21.15.7:0/924446818 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc00fb70 con 0x7f28b8004d50 2024-01-27T02:36:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.543+0000 7f28c27fc700 1 -- 172.21.15.7:0/924446818 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc014440 con 0x7f28b8004d50 2024-01-27T02:36:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.543+0000 7f28d0830700 1 -- 172.21.15.7:0/924446818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28cc14b470 con 0x7f28b8004d50 2024-01-27T02:36:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.543+0000 7f28d0830700 1 -- 172.21.15.7:0/924446818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28cc159780 con 0x7f28b8004d50 2024-01-27T02:36:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.544+0000 7f28d0830700 1 -- 172.21.15.7:0/924446818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28cc072060 con 0x7f28b8004d50 2024-01-27T02:36:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.545+0000 7f28c27fc700 1 -- 172.21.15.7:0/924446818 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f28bc014830 con 0x7f28b8004d50 2024-01-27T02:36:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.545+0000 7f28c27fc700 1 --2- 172.21.15.7:0/924446818 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f28ac066f30 0x7f28ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.545+0000 7f28c27fc700 1 -- 172.21.15.7:0/924446818 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f28bc018030 con 0x7f28b8004d50 2024-01-27T02:36:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.545+0000 7f28c3fff700 1 --2- 172.21.15.7:0/924446818 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f28ac066f30 0x7f28ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.546+0000 7f28c3fff700 1 --2- 172.21.15.7:0/924446818 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f28ac066f30 0x7f28ac0693e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f28b40079e0 tx=0x7f28b4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.550+0000 7f28c27fc700 1 -- 172.21.15.7:0/924446818 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28bc0521d0 con 0x7f28b8004d50 2024-01-27T02:36:11.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.840+0000 7f28d0830700 1 -- 172.21.15.7:0/924446818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f28cc0ae730 con 0x7f28b8004d50 2024-01-27T02:36:11.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.845+0000 7f28c27fc700 1 -- 172.21.15.7:0/924446818 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7f28bc0220e0 con 0x7f28b8004d50 2024-01-27T02:36:11.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.847+0000 7f28d0830700 1 -- 172.21.15.7:0/924446818 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f28ac066f30 msgr2=0x7f28ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:11.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.847+0000 7f28d0830700 1 --2- 172.21.15.7:0/924446818 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f28ac066f30 0x7f28ac0693e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f28b40079e0 tx=0x7f28b4009040 comp rx=0 tx=0).stop 2024-01-27T02:36:11.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.847+0000 7f28d0830700 1 -- 172.21.15.7:0/924446818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28b8004d50 msgr2=0x7f28cc14f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:11.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.847+0000 7f28d0830700 1 --2- 172.21.15.7:0/924446818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28b8004d50 0x7f28cc14f2d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f28bc006ac0 tx=0x7f28bc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.847+0000 7f28d0830700 1 -- 172.21.15.7:0/924446818 shutdown_connections 2024-01-27T02:36:11.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.847+0000 7f28d0830700 1 --2- 172.21.15.7:0/924446818 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f28ac066f30 0x7f28ac0693e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.847+0000 7f28d0830700 1 --2- 172.21.15.7:0/924446818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28cc13b930 0x7f28cc0a79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.847+0000 7f28d0830700 1 --2- 172.21.15.7:0/924446818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28b8004d50 0x7f28cc14f2d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.847+0000 7f28d0830700 1 --2- 172.21.15.7:0/924446818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28cc13b4c0 0x7f28cc0a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.847+0000 7f28d0830700 1 -- 172.21.15.7:0/924446818 >> 172.21.15.7:0/924446818 conn(0x7f28cc068480 msgr2=0x7f28cc063580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:11.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.847+0000 7f28d0830700 1 -- 172.21.15.7:0/924446818 shutdown_connections 2024-01-27T02:36:11.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.847+0000 7f28d0830700 1 -- 172.21.15.7:0/924446818 wait complete. 2024-01-27T02:36:11.849 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-01-27T02:36:11.861 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:36:11.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.992+0000 7f7a41fe4700 1 Processor -- start 2024-01-27T02:36:11.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.993+0000 7f7a41fe4700 1 -- start start 2024-01-27T02:36:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.993+0000 7f7a41fe4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7a3c159f40 0x7f7a3c0c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.993+0000 7f7a41fe4700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7a3c0aafc0 0x7f7a3c063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.993+0000 7f7a41fe4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a3c0c30c0 con 0x7f7a3c0c3a90 2024-01-27T02:36:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.994+0000 7f7a41fe4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a3c0c4550 con 0x7f7a3c159f40 2024-01-27T02:36:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.994+0000 7f7a41fe4700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a3c15e960 con 0x7f7a3c0aafc0 2024-01-27T02:36:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.994+0000 7f7a3b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7a3c159f40 0x7f7a3c0c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.994+0000 7f7a3affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7a3c0c3a90 0x7f7a3c0aa8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39244/0 (socket says 172.21.15.7:39244) 2024-01-27T02:36:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.994+0000 7f7a3affd700 1 -- 172.21.15.7:0/1296104145 learned_addr learned my addr 172.21.15.7:0/1296104145 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.994+0000 7f7a3b7fe700 1 -- 172.21.15.7:0/1296104145 >> v2:172.21.15.188:3300/0 conn(0x7f7a3c0aafc0 msgr2=0x7f7a3c063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.994+0000 7f7a3b7fe700 1 --2- 172.21.15.7:0/1296104145 >> v2:172.21.15.188:3300/0 conn(0x7f7a3c0aafc0 0x7f7a3c063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.994+0000 7f7a3b7fe700 1 -- 172.21.15.7:0/1296104145 >> v1:172.21.15.7:6789/0 conn(0x7f7a3c0c3a90 legacy=0x7f7a3c0aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.994+0000 7f7a3b7fe700 1 -- 172.21.15.7:0/1296104145 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3c15ec40 con 0x7f7a3c159f40 2024-01-27T02:36:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.994+0000 7f7a3a7fc700 1 -- 172.21.15.7:0/1296104145 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1002324046 0 0) 0x7f7a3c0c30c0 con 0x7f7a3c0c3a90 2024-01-27T02:36:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.994+0000 7f7a3b7fe700 1 --2- 172.21.15.7:0/1296104145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a3c159f40 0x7f7a3c0c3550 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0c3030 tx=0x7f7a2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efd60d83dce4087 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.994+0000 7f7a3a7fc700 1 -- 172.21.15.7:0/1296104145 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c00b6b0 con 0x7f7a3c159f40 2024-01-27T02:36:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.994+0000 7f7a3a7fc700 1 -- 172.21.15.7:0/1296104145 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2c00b830 con 0x7f7a3c159f40 2024-01-27T02:36:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.994+0000 7f7a3a7fc700 1 -- 172.21.15.7:0/1296104145 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c00bbf0 con 0x7f7a3c159f40 2024-01-27T02:36:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.995+0000 7f7a41fe4700 1 -- 172.21.15.7:0/1296104145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a3c159f40 msgr2=0x7f7a3c0c3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.995+0000 7f7a41fe4700 1 --2- 172.21.15.7:0/1296104145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a3c159f40 0x7f7a3c0c3550 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0c3030 tx=0x7f7a2c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.995+0000 7f7a41fe4700 1 -- 172.21.15.7:0/1296104145 shutdown_connections 2024-01-27T02:36:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.995+0000 7f7a41fe4700 1 --2- 172.21.15.7:0/1296104145 >> v2:172.21.15.188:3300/0 conn(0x7f7a3c0aafc0 0x7f7a3c063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.995+0000 7f7a41fe4700 1 --2- 172.21.15.7:0/1296104145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a3c159f40 0x7f7a3c0c3550 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.995+0000 7f7a41fe4700 1 -- 172.21.15.7:0/1296104145 >> 172.21.15.7:0/1296104145 conn(0x7f7a3c0cdc30 msgr2=0x7f7a3c0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.995+0000 7f7a41fe4700 1 -- 172.21.15.7:0/1296104145 shutdown_connections 2024-01-27T02:36:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.995+0000 7f7a41fe4700 1 -- 172.21.15.7:0/1296104145 wait complete. 2024-01-27T02:36:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.996+0000 7f7a41fe4700 1 Processor -- start 2024-01-27T02:36:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.996+0000 7f7a41fe4700 1 -- start start 2024-01-27T02:36:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.997+0000 7f7a41fe4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a3c0aafc0 0x7f7a3c156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.997+0000 7f7a41fe4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a3c0c3a90 0x7f7a3c0b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.997+0000 7f7a41fe4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a3c159f40 0x7f7a3c06c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.997+0000 7f7a41fe4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c16aa10 con 0x7f7a3c0c3a90 2024-01-27T02:36:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.997+0000 7f7a41fe4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c0b2e80 con 0x7f7a3c159f40 2024-01-27T02:36:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.997+0000 7f7a41fe4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c16acd0 con 0x7f7a3c0aafc0 2024-01-27T02:36:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.997+0000 7f7a3affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a3c0c3a90 0x7f7a3c0b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.997+0000 7f7a3affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a3c0c3a90 0x7f7a3c0b23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41656/0 (socket says 172.21.15.7:41656) 2024-01-27T02:36:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.997+0000 7f7a3affd700 1 -- 172.21.15.7:0/3323040113 learned_addr learned my addr 172.21.15.7:0/3323040113 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.997+0000 7f7a3bfff700 1 --2- 172.21.15.7:0/3323040113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a3c159f40 0x7f7a3c06c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.998+0000 7f7a3b7fe700 1 --2- 172.21.15.7:0/3323040113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a3c0aafc0 0x7f7a3c156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.998+0000 7f7a3affd700 1 -- 172.21.15.7:0/3323040113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a3c0aafc0 msgr2=0x7f7a3c156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.998+0000 7f7a3affd700 1 --2- 172.21.15.7:0/3323040113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a3c0aafc0 0x7f7a3c156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.998+0000 7f7a3affd700 1 -- 172.21.15.7:0/3323040113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a3c159f40 msgr2=0x7f7a3c06c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.998+0000 7f7a3affd700 1 --2- 172.21.15.7:0/3323040113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a3c159f40 0x7f7a3c06c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.998+0000 7f7a3affd700 1 -- 172.21.15.7:0/3323040113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2c009420 con 0x7f7a3c0c3a90 2024-01-27T02:36:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.998+0000 7f7a3affd700 1 --2- 172.21.15.7:0/3323040113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a3c0c3a90 0x7f7a3c0b23a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000c810 tx=0x7f7a30007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.998+0000 7f7a237fe700 1 -- 172.21.15.7:0/3323040113 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a30005c00 con 0x7f7a3c0c3a90 2024-01-27T02:36:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.998+0000 7f7a237fe700 1 -- 172.21.15.7:0/3323040113 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a30005d60 con 0x7f7a3c0c3a90 2024-01-27T02:36:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.998+0000 7f7a41fe4700 1 -- 172.21.15.7:0/3323040113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3c16afb0 con 0x7f7a3c0c3a90 2024-01-27T02:36:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.998+0000 7f7a237fe700 1 -- 172.21.15.7:0/3323040113 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a30007300 con 0x7f7a3c0c3a90 2024-01-27T02:36:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.998+0000 7f7a41fe4700 1 -- 172.21.15.7:0/3323040113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3c0af910 con 0x7f7a3c0c3a90 2024-01-27T02:36:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:11.999+0000 7f7a41fe4700 1 -- 172.21.15.7:0/3323040113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a3c16b4d0 con 0x7f7a3c0c3a90 2024-01-27T02:36:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.000+0000 7f7a237fe700 1 -- 172.21.15.7:0/3323040113 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7a3001b3b0 con 0x7f7a3c0c3a90 2024-01-27T02:36:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.000+0000 7f7a237fe700 1 --2- 172.21.15.7:0/3323040113 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7a28066f30 0x7f7a280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.000+0000 7f7a237fe700 1 -- 172.21.15.7:0/3323040113 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7a300839f0 con 0x7f7a3c0c3a90 2024-01-27T02:36:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.000+0000 7f7a3b7fe700 1 --2- 172.21.15.7:0/3323040113 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7a28066f30 0x7f7a280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.001+0000 7f7a3b7fe700 1 --2- 172.21.15.7:0/3323040113 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7a28066f30 0x7f7a280693e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0c3030 tx=0x7f7a2c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:12.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.005+0000 7f7a237fe700 1 -- 172.21.15.7:0/3323040113 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a3c16b4d0 con 0x7f7a3c0c3a90 2024-01-27T02:36:12.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.253+0000 7f7a41fe4700 1 -- 172.21.15.7:0/3323040113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a3c16b4d0 con 0x7f7a3c0c3a90 2024-01-27T02:36:12.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.254+0000 7f7a237fe700 1 -- 172.21.15.7:0/3323040113 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4590 (secure 0 0 0) 0x7f7a3c16b4d0 con 0x7f7a3c0c3a90 2024-01-27T02:36:12.254 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:36:12.254 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5839,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/100533817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":100533817},{"type":"v1","addr":"172.21.15.7:6834","nonce":100533817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5842,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/223425097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":223425097},{"type":"v1","addr":"172.21.15.7:6835","nonce":223425097}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5848,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/950040140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":950040140},{"type":"v1","addr":"172.21.15.188:6837","nonce":950040140}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:36:01.502818+0000","modified":"2024-01-27T02:36:05.109352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5946":{"gid":5946,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6836/2269430612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2269430612},{"type":"v1","addr":"172.21.15.188:6836","nonce":2269430612}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2024-01-27T02:36:12.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.256+0000 7f7a41fe4700 1 -- 172.21.15.7:0/3323040113 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7a28066f30 msgr2=0x7f7a280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.256+0000 7f7a41fe4700 1 --2- 172.21.15.7:0/3323040113 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7a28066f30 0x7f7a280693e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0c3030 tx=0x7f7a2c017000 comp rx=0 tx=0).stop 2024-01-27T02:36:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.256+0000 7f7a41fe4700 1 -- 172.21.15.7:0/3323040113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a3c0c3a90 msgr2=0x7f7a3c0b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.256+0000 7f7a41fe4700 1 --2- 172.21.15.7:0/3323040113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a3c0c3a90 0x7f7a3c0b23a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000c810 tx=0x7f7a30007db0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.256+0000 7f7a41fe4700 1 -- 172.21.15.7:0/3323040113 shutdown_connections 2024-01-27T02:36:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.256+0000 7f7a41fe4700 1 --2- 172.21.15.7:0/3323040113 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7a28066f30 0x7f7a280693e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.256+0000 7f7a41fe4700 1 --2- 172.21.15.7:0/3323040113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a3c159f40 0x7f7a3c06c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.256+0000 7f7a41fe4700 1 --2- 172.21.15.7:0/3323040113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a3c0aafc0 0x7f7a3c156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.256+0000 7f7a41fe4700 1 --2- 172.21.15.7:0/3323040113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a3c0c3a90 0x7f7a3c0b23a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.256+0000 7f7a41fe4700 1 -- 172.21.15.7:0/3323040113 >> 172.21.15.7:0/3323040113 conn(0x7f7a3c0cdc30 msgr2=0x7f7a3c157240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.256+0000 7f7a41fe4700 1 -- 172.21.15.7:0/3323040113 shutdown_connections 2024-01-27T02:36:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.256+0000 7f7a41fe4700 1 -- 172.21.15.7:0/3323040113 wait complete. 2024-01-27T02:36:12.258 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 42 2024-01-27T02:36:12.673 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-27T02:36:01.502818+0000', 'modified': '2024-01-27T02:36:05.109352+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5946}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5946': {'gid': 5946, 'name': 'd', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6836/2269430612', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 2269430612}, {'type': 'v1', 'addr': '172.21.15.188:6836', 'nonce': 2269430612}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-27T02:36:12.674 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-27T02:36:12.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-01-27T02:36:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.815+0000 7f0768d6e700 1 Processor -- start 2024-01-27T02:36:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.816+0000 7f0768d6e700 1 -- start start 2024-01-27T02:36:12.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.817+0000 7f0768d6e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f07640cb7b0 0x7f0764063950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.817+0000 7f0768d6e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f07640b0b50 0x7f07640b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.817+0000 7f0768d6e700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07640cc8f0 con 0x7f0764063580 2024-01-27T02:36:12.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.817+0000 7f0768d6e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0764063e90 con 0x7f07640b0b50 2024-01-27T02:36:12.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.817+0000 7f0768d6e700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07640cbce0 con 0x7f07640cb7b0 2024-01-27T02:36:12.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.817+0000 7f0762d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f07640b0b50 0x7f07640b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.817+0000 7f0762d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f07640b0b50 0x7f07640b0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41670/0 (socket says 172.21.15.7:41670) 2024-01-27T02:36:12.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.817+0000 7f0762d9d700 1 -- 172.21.15.7:0/3430373962 learned_addr learned my addr 172.21.15.7:0/3430373962 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:12.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.817+0000 7f0761d9b700 1 --2- 172.21.15.7:0/3430373962 >> v2:172.21.15.188:3300/0 conn(0x7f07640cb7b0 0x7f0764063950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.818+0000 7f0762d9d700 1 -- 172.21.15.7:0/3430373962 >> v2:172.21.15.188:3300/0 conn(0x7f07640cb7b0 msgr2=0x7f0764063950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:12.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.818+0000 7f0762d9d700 1 --2- 172.21.15.7:0/3430373962 >> v2:172.21.15.188:3300/0 conn(0x7f07640cb7b0 0x7f0764063950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.818+0000 7f0762d9d700 1 -- 172.21.15.7:0/3430373962 >> v1:172.21.15.188:6789/0 conn(0x7f0764063580 legacy=0x7f07640cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:12.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.818+0000 7f0762d9d700 1 -- 172.21.15.7:0/3430373962 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07640cbfc0 con 0x7f07640b0b50 2024-01-27T02:36:12.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.818+0000 7f0761d9b700 1 --2- 172.21.15.7:0/3430373962 >> v2:172.21.15.188:3300/0 conn(0x7f07640cb7b0 0x7f0764063950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:36:12.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.818+0000 7f0762d9d700 1 --2- 172.21.15.7:0/3430373962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07640b0b50 0x7f07640b0fb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f07640ce980 tx=0x7f075c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c275567841e6e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:12.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.818+0000 7f076159a700 1 -- 172.21.15.7:0/3430373962 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c00e070 con 0x7f07640b0b50 2024-01-27T02:36:12.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.818+0000 7f076159a700 1 -- 172.21.15.7:0/3430373962 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075c00ba50 con 0x7f07640b0b50 2024-01-27T02:36:12.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.818+0000 7f076159a700 1 -- 172.21.15.7:0/3430373962 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c0092e0 con 0x7f07640b0b50 2024-01-27T02:36:12.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.819+0000 7f0768d6e700 1 -- 172.21.15.7:0/3430373962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07640b0b50 msgr2=0x7f07640b0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:12.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.819+0000 7f0768d6e700 1 --2- 172.21.15.7:0/3430373962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07640b0b50 0x7f07640b0fb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f07640ce980 tx=0x7f075c009840 comp rx=0 tx=0).stop 2024-01-27T02:36:12.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.819+0000 7f0768d6e700 1 -- 172.21.15.7:0/3430373962 shutdown_connections 2024-01-27T02:36:12.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.819+0000 7f0768d6e700 1 --2- 172.21.15.7:0/3430373962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07640b0b50 0x7f07640b0fb0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.819+0000 7f0768d6e700 1 --2- 172.21.15.7:0/3430373962 >> v2:172.21.15.188:3300/0 conn(0x7f07640cb7b0 0x7f0764063950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.819+0000 7f0768d6e700 1 -- 172.21.15.7:0/3430373962 >> 172.21.15.7:0/3430373962 conn(0x7f0764068480 msgr2=0x7f07640cdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:12.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.819+0000 7f0768d6e700 1 -- 172.21.15.7:0/3430373962 shutdown_connections 2024-01-27T02:36:12.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.819+0000 7f0768d6e700 1 -- 172.21.15.7:0/3430373962 wait complete. 2024-01-27T02:36:12.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.820+0000 7f0768d6e700 1 Processor -- start 2024-01-27T02:36:12.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.820+0000 7f0768d6e700 1 -- start start 2024-01-27T02:36:12.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.821+0000 7f0768d6e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0764063580 0x7f0764154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.821+0000 7f0768d6e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07640b0b50 0x7f07640c7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.821+0000 7f0768d6e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07640cb7b0 0x7f0764155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.821+0000 7f0768d6e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07640c7e20 con 0x7f07640b0b50 2024-01-27T02:36:12.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.821+0000 7f0768d6e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076416e210 con 0x7f07640cb7b0 2024-01-27T02:36:12.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.821+0000 7f0768d6e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076416d8e0 con 0x7f0764063580 2024-01-27T02:36:12.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.821+0000 7f0761d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07640b0b50 0x7f07640c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.821+0000 7f0761d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07640b0b50 0x7f07640c7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41678/0 (socket says 172.21.15.7:41678) 2024-01-27T02:36:12.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.821+0000 7f0761d9b700 1 -- 172.21.15.7:0/3831199691 learned_addr learned my addr 172.21.15.7:0/3831199691 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:12.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.821+0000 7f0762d9d700 1 --2- 172.21.15.7:0/3831199691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07640cb7b0 0x7f0764155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.821+0000 7f076259c700 1 --2- 172.21.15.7:0/3831199691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0764063580 0x7f0764154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.821+0000 7f0761d9b700 1 -- 172.21.15.7:0/3831199691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0764063580 msgr2=0x7f0764154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:12.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.821+0000 7f0761d9b700 1 --2- 172.21.15.7:0/3831199691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0764063580 0x7f0764154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.821+0000 7f0761d9b700 1 -- 172.21.15.7:0/3831199691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07640cb7b0 msgr2=0x7f0764155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:12.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.821+0000 7f0761d9b700 1 --2- 172.21.15.7:0/3831199691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07640cb7b0 0x7f0764155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.821+0000 7f0761d9b700 1 -- 172.21.15.7:0/3831199691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075c009ae0 con 0x7f07640b0b50 2024-01-27T02:36:12.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.821+0000 7f0762d9d700 1 --2- 172.21.15.7:0/3831199691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07640cb7b0 0x7f0764155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:36:12.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.821+0000 7f076259c700 1 --2- 172.21.15.7:0/3831199691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0764063580 0x7f0764154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:36:12.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.821+0000 7f0761d9b700 1 --2- 172.21.15.7:0/3831199691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07640b0b50 0x7f07640c7110 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f074c00d890 tx=0x7f074c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:12.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.822+0000 7f075a7fc700 1 -- 172.21.15.7:0/3831199691 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c0125f0 con 0x7f07640b0b50 2024-01-27T02:36:12.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.822+0000 7f075a7fc700 1 -- 172.21.15.7:0/3831199691 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074c012750 con 0x7f07640b0b50 2024-01-27T02:36:12.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.822+0000 7f0768d6e700 1 -- 172.21.15.7:0/3831199691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f076416dbc0 con 0x7f07640b0b50 2024-01-27T02:36:12.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.822+0000 7f075a7fc700 1 -- 172.21.15.7:0/3831199691 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c012af0 con 0x7f07640b0b50 2024-01-27T02:36:12.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.822+0000 7f0768d6e700 1 -- 172.21.15.7:0/3831199691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07640c7aa0 con 0x7f07640b0b50 2024-01-27T02:36:12.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.823+0000 7f075a7fc700 1 -- 172.21.15.7:0/3831199691 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f074c014430 con 0x7f07640b0b50 2024-01-27T02:36:12.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.824+0000 7f075a7fc700 1 --2- 172.21.15.7:0/3831199691 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0744066f30 0x7f07440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.824+0000 7f075a7fc700 1 -- 172.21.15.7:0/3831199691 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f074c084b10 con 0x7f07640b0b50 2024-01-27T02:36:12.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.824+0000 7f076259c700 1 --2- 172.21.15.7:0/3831199691 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0744066f30 0x7f07440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.824+0000 7f076259c700 1 --2- 172.21.15.7:0/3831199691 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0744066f30 0x7f07440693e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0754001840 tx=0x7f0754009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:12.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.826+0000 7f0772071b80 1 Processor -- start 2024-01-27T02:36:12.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.826+0000 7f0772071b80 1 -- start start 2024-01-27T02:36:12.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.827+0000 7f0772071b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa3f06c010 0x55aa3f06c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.827+0000 7f0772071b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aa3f0b4360 0x55aa3f096840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.827+0000 7f0772071b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f06b5d0 0x55aa3f096d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.827+0000 7f0772071b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3ef6f9f0 con 0x55aa3f06b5d0 2024-01-27T02:36:12.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.827+0000 7f0772071b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3efc1130 con 0x55aa3f06c010 2024-01-27T02:36:12.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.827+0000 7f0772071b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa3efbe930 con 0x55aa3f0b4360 2024-01-27T02:36:12.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.827+0000 7f0762d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f06b5d0 0x55aa3f096d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.827+0000 7f0762d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f06b5d0 0x55aa3f096d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41682/0 (socket says 172.21.15.7:41682) 2024-01-27T02:36:12.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.827+0000 7f0762d9d700 1 -- 172.21.15.7:0/4055508422 learned_addr learned my addr 172.21.15.7:0/4055508422 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:12.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.827+0000 7f0761d9b700 1 --2- 172.21.15.7:0/4055508422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aa3f0b4360 0x55aa3f096840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.827+0000 7f076259c700 1 --2- 172.21.15.7:0/4055508422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa3f06c010 0x55aa3f06c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.828+0000 7f0762d9d700 1 -- 172.21.15.7:0/4055508422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aa3f0b4360 msgr2=0x55aa3f096840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:12.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.828+0000 7f0762d9d700 1 --2- 172.21.15.7:0/4055508422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aa3f0b4360 0x55aa3f096840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.828+0000 7f0762d9d700 1 -- 172.21.15.7:0/4055508422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa3f06c010 msgr2=0x55aa3f06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:12.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.828+0000 7f0762d9d700 1 --2- 172.21.15.7:0/4055508422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa3f06c010 0x55aa3f06c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.828+0000 7f0762d9d700 1 -- 172.21.15.7:0/4055508422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa3edf9b10 con 0x55aa3f06b5d0 2024-01-27T02:36:12.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.828+0000 7f0762d9d700 1 --2- 172.21.15.7:0/4055508422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f06b5d0 0x55aa3f096d80 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f075c0092e0 tx=0x7f075c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53a7e97b1b2d4f7 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:12.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.828+0000 7f07437fe700 1 -- 172.21.15.7:0/4055508422 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c009940 con 0x55aa3f06b5d0 2024-01-27T02:36:12.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.828+0000 7f07437fe700 1 -- 172.21.15.7:0/4055508422 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075c00be60 con 0x55aa3f06b5d0 2024-01-27T02:36:12.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.828+0000 7f07437fe700 1 -- 172.21.15.7:0/4055508422 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c007200 con 0x55aa3f06b5d0 2024-01-27T02:36:12.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.829+0000 7f0772071b80 1 -- 172.21.15.7:0/4055508422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f06b5d0 msgr2=0x55aa3f096d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:12.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.829+0000 7f0772071b80 1 --2- 172.21.15.7:0/4055508422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f06b5d0 0x55aa3f096d80 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f075c0092e0 tx=0x7f075c018000 comp rx=0 tx=0).stop 2024-01-27T02:36:12.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.829+0000 7f0772071b80 1 -- 172.21.15.7:0/4055508422 shutdown_connections 2024-01-27T02:36:12.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.829+0000 7f0772071b80 1 --2- 172.21.15.7:0/4055508422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f06b5d0 0x55aa3f096d80 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.829+0000 7f0772071b80 1 --2- 172.21.15.7:0/4055508422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa3f06c010 0x55aa3f06c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.829+0000 7f0772071b80 1 --2- 172.21.15.7:0/4055508422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aa3f0b4360 0x55aa3f096840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.829+0000 7f0772071b80 1 -- 172.21.15.7:0/4055508422 >> 172.21.15.7:0/4055508422 conn(0x55aa3f06ba70 msgr2=0x55aa3f0b4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:12.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.829+0000 7f0772071b80 1 -- 172.21.15.7:0/4055508422 shutdown_connections 2024-01-27T02:36:12.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.829+0000 7f0772071b80 1 -- 172.21.15.7:0/4055508422 wait complete. 2024-01-27T02:36:12.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.830+0000 7f0772071b80 1 -- start start 2024-01-27T02:36:12.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.830+0000 7f0772071b80 1 Processor -- start 2024-01-27T02:36:12.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.831+0000 7f0772071b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa3f06b5d0 0x55aa3f0b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.831+0000 7f0772071b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f06c010 0x55aa3f0b1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.831+0000 7f0772071b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aa3f0b4360 0x55aa3f0b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.831+0000 7f0772071b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3efba720 con 0x55aa3f06c010 2024-01-27T02:36:12.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.831+0000 7f0772071b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3ef661b0 con 0x55aa3f06b5d0 2024-01-27T02:36:12.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.831+0000 7f0772071b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa3ef65e50 con 0x55aa3f0b4360 2024-01-27T02:36:12.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.832+0000 7f0772071b80 10 client.0 _ll_register_callbacks cb 0x55aa3edc87b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-27T02:36:12.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.832+0000 7f0761d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f06c010 0x55aa3f0b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.832+0000 7f0761d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f06c010 0x55aa3f0b1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41698/0 (socket says 172.21.15.7:41698) 2024-01-27T02:36:12.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.832+0000 7f0761d9b700 1 -- 172.21.15.7:0/3298322177 learned_addr learned my addr 172.21.15.7:0/3298322177 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:12.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.832+0000 7f076259c700 1 --2- 172.21.15.7:0/3298322177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa3f06b5d0 0x55aa3f0b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.832+0000 7f0762d9d700 1 --2- 172.21.15.7:0/3298322177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aa3f0b4360 0x55aa3f0b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.832+0000 7f0761d9b700 1 -- 172.21.15.7:0/3298322177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aa3f0b4360 msgr2=0x55aa3f0b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:12.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.832+0000 7f0761d9b700 1 --2- 172.21.15.7:0/3298322177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aa3f0b4360 0x55aa3f0b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.832+0000 7f0761d9b700 1 -- 172.21.15.7:0/3298322177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa3f06b5d0 msgr2=0x55aa3f0b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:12.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.832+0000 7f0761d9b700 1 --2- 172.21.15.7:0/3298322177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa3f06b5d0 0x55aa3f0b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.832+0000 7f0761d9b700 1 -- 172.21.15.7:0/3298322177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07640cbfc0 con 0x55aa3f06c010 2024-01-27T02:36:12.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.832+0000 7f0761d9b700 1 --2- 172.21.15.7:0/3298322177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f06c010 0x55aa3f0b1340 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f074c007f40 tx=0x7f074c082ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a100298da2009b server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.832+0000 7f0740ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-27T02:36:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.832+0000 7f0740ff9700 1 -- 172.21.15.7:0/3298322177 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c05c940 con 0x55aa3f06c010 2024-01-27T02:36:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.832+0000 7f0740ff9700 1 -- 172.21.15.7:0/3298322177 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f074c07ed90 con 0x55aa3f06c010 2024-01-27T02:36:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.832+0000 7f0772071b80 1 -- 172.21.15.7:0/3298322177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa3f05f190 con 0x55aa3f06c010 2024-01-27T02:36:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.832+0000 7f0740ff9700 1 -- 172.21.15.7:0/3298322177 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f074c055e10 con 0x55aa3f06c010 2024-01-27T02:36:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.832+0000 7f0772071b80 10 client.6117 fetch_fsmap learned FSMap version 42 2024-01-27T02:36:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.832+0000 7f0772071b80 1 -- 172.21.15.7:0/3298322177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x55aa3ef88290 con 0x55aa3f06c010 2024-01-27T02:36:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.833+0000 7f0740ff9700 1 -- 172.21.15.7:0/3298322177 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aa3f05f190 con 0x55aa3f06c010 2024-01-27T02:36:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.833+0000 7f0740ff9700 1 -- 172.21.15.7:0/3298322177 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 42) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f074c054e90 con 0x55aa3f06c010 2024-01-27T02:36:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.833+0000 7f0772071b80 10 client.6117 fetch_fsmap finished waiting for FSMap version 42 2024-01-27T02:36:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.833+0000 7f0772071b80 10 client.6117 resolve_mds: resolved name 'd' to daemon mds.d 2024-01-27T02:36:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.833+0000 7f0772071b80 20 client.6117 populate_metadata read hostname 'smithi007' 2024-01-27T02:36:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.833+0000 7f0772071b80 1 --2- 172.21.15.7:0/3298322177 >> [v2:172.21.15.188:6834/2269430612,v1:172.21.15.188:6836/2269430612] conn(0x55aa3f0abb40 0x55aa3f0abf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.833+0000 7f0772071b80 4 client.6117 mds_command: new command op to 5946 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-27T02:36:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.833+0000 7f0772071b80 1 -- 172.21.15.7:0/3298322177 --> [v2:172.21.15.188:6834/2269430612,v1:172.21.15.188:6836/2269430612] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa3ef5cee0 con 0x55aa3f0abb40 2024-01-27T02:36:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.833+0000 7f076259c700 1 --2- 172.21.15.7:0/3298322177 >> [v2:172.21.15.188:6834/2269430612,v1:172.21.15.188:6836/2269430612] conn(0x55aa3f0abb40 0x55aa3f0abf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.833+0000 7f0740ff9700 10 client.6117 ms_handle_connect on v2:172.21.15.188:6834/2269430612 2024-01-27T02:36:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.833+0000 7f076259c700 1 --2- 172.21.15.7:0/3298322177 >> [v2:172.21.15.188:6834/2269430612,v1:172.21.15.188:6836/2269430612] conn(0x55aa3f0abb40 0x55aa3f0abf10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7926c5735c40f267 server_cookie=29630d182655beb0 in_seq=0 out_seq=0 2024-01-27T02:36:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.834+0000 7f0740ff9700 1 -- 172.21.15.7:0/3298322177 <== mds.0 v2:172.21.15.188:6834/2269430612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17146 (crc 0 0 0) 0x55aa3ef5cee0 con 0x55aa3f0abb40 2024-01-27T02:36:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.834+0000 7f0740ff9700 10 client.6117 handle_command_reply: tid=0 2024-01-27T02:36:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.834+0000 7f0772071b80 1 client.6117 shutdown 2024-01-27T02:36:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.834+0000 7f0772071b80 1 -- 172.21.15.7:0/3298322177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f06c010 msgr2=0x55aa3f0b1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.834+0000 7f0772071b80 1 --2- 172.21.15.7:0/3298322177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f06c010 0x55aa3f0b1340 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f074c007f40 tx=0x7f074c082ef0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.834+0000 7f0772071b80 1 -- 172.21.15.7:0/3298322177 shutdown_connections 2024-01-27T02:36:12.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.834+0000 7f0772071b80 1 --2- 172.21.15.7:0/3298322177 >> [v2:172.21.15.188:6834/2269430612,v1:172.21.15.188:6836/2269430612] conn(0x55aa3f0abb40 0x55aa3f0abf10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.834+0000 7f0772071b80 1 --2- 172.21.15.7:0/3298322177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aa3f0b4360 0x55aa3f0b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.834+0000 7f0772071b80 1 --2- 172.21.15.7:0/3298322177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa3f06b5d0 0x55aa3f0b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.834+0000 7f0740ff9700 0 client.6117 ms_handle_reset on v2:172.21.15.188:6834/2269430612 2024-01-27T02:36:12.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.834+0000 7f0772071b80 1 --2- 172.21.15.7:0/3298322177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f06c010 0x55aa3f0b1340 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.834+0000 7f0772071b80 1 -- 172.21.15.7:0/3298322177 >> 172.21.15.7:0/3298322177 conn(0x55aa3f06ba70 msgr2=0x55aa3f096610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:12.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.835+0000 7f0772071b80 1 -- 172.21.15.7:0/3298322177 shutdown_connections 2024-01-27T02:36:12.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.835+0000 7f0772071b80 1 -- 172.21.15.7:0/3298322177 wait complete. 2024-01-27T02:36:12.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.835+0000 7f0772071b80 20 client.6117 trim_cache size 0 max 16384 2024-01-27T02:36:12.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.853+0000 7f0772071b80 1 Processor -- start 2024-01-27T02:36:12.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.853+0000 7f0772071b80 1 -- start start 2024-01-27T02:36:12.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.854+0000 7f0772071b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa3f06b5d0 0x55aa3f0967c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.854+0000 7f0772071b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aa3f06c010 0x55aa3f096d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.854+0000 7f0772071b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f0b4360 0x55aa3f0aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.854+0000 7f0772071b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3eeb2e40 con 0x55aa3f0b4360 2024-01-27T02:36:12.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.854+0000 7f0772071b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3f09a1c0 con 0x55aa3f06b5d0 2024-01-27T02:36:12.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.854+0000 7f0772071b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa3f0652e0 con 0x55aa3f06c010 2024-01-27T02:36:12.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.854+0000 7f0762d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f0b4360 0x55aa3f0aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.854+0000 7f0762d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f0b4360 0x55aa3f0aabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41710/0 (socket says 172.21.15.7:41710) 2024-01-27T02:36:12.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.854+0000 7f0762d9d700 1 -- 172.21.15.7:0/4173526818 learned_addr learned my addr 172.21.15.7:0/4173526818 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:12.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.854+0000 7f076259c700 1 --2- 172.21.15.7:0/4173526818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa3f06b5d0 0x55aa3f0967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.854+0000 7f0761d9b700 1 --2- 172.21.15.7:0/4173526818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aa3f06c010 0x55aa3f096d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.854+0000 7f0762d9d700 1 -- 172.21.15.7:0/4173526818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aa3f06c010 msgr2=0x55aa3f096d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:12.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.854+0000 7f0762d9d700 1 --2- 172.21.15.7:0/4173526818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aa3f06c010 0x55aa3f096d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.854+0000 7f0762d9d700 1 -- 172.21.15.7:0/4173526818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa3f06b5d0 msgr2=0x55aa3f0967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:12.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.854+0000 7f0762d9d700 1 --2- 172.21.15.7:0/4173526818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa3f06b5d0 0x55aa3f0967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.854+0000 7f0762d9d700 1 -- 172.21.15.7:0/4173526818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa3f06ce60 con 0x55aa3f0b4360 2024-01-27T02:36:12.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.854+0000 7f0762d9d700 1 --2- 172.21.15.7:0/4173526818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f0b4360 0x55aa3f0aabe0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f075c009310 tx=0x7f075c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d665664b5caba0bb server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:12.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.855+0000 7f0743fff700 1 -- 172.21.15.7:0/4173526818 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c02c070 con 0x55aa3f0b4360 2024-01-27T02:36:12.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.855+0000 7f0743fff700 1 -- 172.21.15.7:0/4173526818 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075c02d040 con 0x55aa3f0b4360 2024-01-27T02:36:12.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.855+0000 7f0743fff700 1 -- 172.21.15.7:0/4173526818 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c007200 con 0x55aa3f0b4360 2024-01-27T02:36:12.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.855+0000 7f0772071b80 1 -- 172.21.15.7:0/4173526818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f0b4360 msgr2=0x55aa3f0aabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:12.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.855+0000 7f0772071b80 1 --2- 172.21.15.7:0/4173526818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f0b4360 0x55aa3f0aabe0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f075c009310 tx=0x7f075c01f000 comp rx=0 tx=0).stop 2024-01-27T02:36:12.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.855+0000 7f0772071b80 1 -- 172.21.15.7:0/4173526818 shutdown_connections 2024-01-27T02:36:12.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.855+0000 7f0772071b80 1 --2- 172.21.15.7:0/4173526818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f0b4360 0x55aa3f0aabe0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.855+0000 7f0772071b80 1 --2- 172.21.15.7:0/4173526818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa3f06b5d0 0x55aa3f0967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.855+0000 7f0772071b80 1 --2- 172.21.15.7:0/4173526818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aa3f06c010 0x55aa3f096d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.855+0000 7f0772071b80 1 -- 172.21.15.7:0/4173526818 >> 172.21.15.7:0/4173526818 conn(0x55aa3f06ba70 msgr2=0x55aa3f0af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:12.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.855+0000 7f0772071b80 1 -- 172.21.15.7:0/4173526818 shutdown_connections 2024-01-27T02:36:12.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.855+0000 7f0772071b80 1 -- 172.21.15.7:0/4173526818 wait complete. 2024-01-27T02:36:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.856+0000 7f0772071b80 1 -- start start 2024-01-27T02:36:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.856+0000 7f0772071b80 1 Processor -- start 2024-01-27T02:36:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.857+0000 7f0772071b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aa3f06b5d0 0x55aa3f0aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.857+0000 7f0772071b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa3f06c010 0x55aa3f0ab120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.857+0000 7f0772071b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f0b4360 0x55aa3f1156d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.857+0000 7f0772071b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3f116990 con 0x55aa3f0b4360 2024-01-27T02:36:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.857+0000 7f0772071b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3f0c3780 con 0x55aa3f06c010 2024-01-27T02:36:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.857+0000 7f0772071b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa3f0c4bd0 con 0x55aa3f06b5d0 2024-01-27T02:36:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.857+0000 7f0772071b80 10 client.0 _ll_register_callbacks cb 0x7f074c0560a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-27T02:36:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.857+0000 7f0762d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f0b4360 0x55aa3f1156d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.857+0000 7f0762d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f0b4360 0x55aa3f1156d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41714/0 (socket says 172.21.15.7:41714) 2024-01-27T02:36:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.857+0000 7f0762d9d700 1 -- 172.21.15.7:0/1536018928 learned_addr learned my addr 172.21.15.7:0/1536018928 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.857+0000 7f0761d9b700 1 --2- 172.21.15.7:0/1536018928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa3f06c010 0x55aa3f0ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.857+0000 7f076259c700 1 --2- 172.21.15.7:0/1536018928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aa3f06b5d0 0x55aa3f0aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.858+0000 7f0762d9d700 1 -- 172.21.15.7:0/1536018928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aa3f06b5d0 msgr2=0x55aa3f0aabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.858+0000 7f0762d9d700 1 --2- 172.21.15.7:0/1536018928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aa3f06b5d0 0x55aa3f0aabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.858+0000 7f0762d9d700 1 -- 172.21.15.7:0/1536018928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa3f06c010 msgr2=0x55aa3f0ab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.858+0000 7f0762d9d700 1 --2- 172.21.15.7:0/1536018928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa3f06c010 0x55aa3f0ab120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.858+0000 7f0762d9d700 1 -- 172.21.15.7:0/1536018928 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa3edf9b10 con 0x55aa3f0b4360 2024-01-27T02:36:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.858+0000 7f0762d9d700 1 --2- 172.21.15.7:0/1536018928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f0b4360 0x55aa3f1156d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f075c000f80 tx=0x7f075c02f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc29b40bbb498ec server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.858+0000 7f0740ff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-27T02:36:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.858+0000 7f0740ff9700 1 -- 172.21.15.7:0/1536018928 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c02c050 con 0x55aa3f0b4360 2024-01-27T02:36:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.858+0000 7f0740ff9700 1 -- 172.21.15.7:0/1536018928 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f075c00bbc0 con 0x55aa3f0b4360 2024-01-27T02:36:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.858+0000 7f0772071b80 1 -- 172.21.15.7:0/1536018928 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f075c00be60 con 0x55aa3f0b4360 2024-01-27T02:36:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.858+0000 7f0740ff9700 1 -- 172.21.15.7:0/1536018928 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f075c009940 con 0x55aa3f0b4360 2024-01-27T02:36:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.858+0000 7f0772071b80 10 client.6123 fetch_fsmap learned FSMap version 42 2024-01-27T02:36:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.858+0000 7f0772071b80 1 -- 172.21.15.7:0/1536018928 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x55aa3ee51c80 con 0x55aa3f0b4360 2024-01-27T02:36:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.858+0000 7f0740ff9700 1 -- 172.21.15.7:0/1536018928 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075c00be60 con 0x55aa3f0b4360 2024-01-27T02:36:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.858+0000 7f0740ff9700 1 -- 172.21.15.7:0/1536018928 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 42) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f075c024030 con 0x55aa3f0b4360 2024-01-27T02:36:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.858+0000 7f0772071b80 10 client.6123 fetch_fsmap finished waiting for FSMap version 42 2024-01-27T02:36:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.858+0000 7f0772071b80 10 client.6123 resolve_mds: resolved name 'd' to daemon mds.d 2024-01-27T02:36:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.858+0000 7f0772071b80 20 client.6123 populate_metadata read hostname 'smithi007' 2024-01-27T02:36:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.859+0000 7f0772071b80 1 --2- 172.21.15.7:0/1536018928 >> [v2:172.21.15.188:6834/2269430612,v1:172.21.15.188:6836/2269430612] conn(0x55aa3f06c3e0 0x55aa3f115c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.859+0000 7f0772071b80 4 client.6123 mds_command: new command op to 5946 tid=0[{"prefix": "status"}] 2024-01-27T02:36:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.859+0000 7f0772071b80 1 -- 172.21.15.7:0/1536018928 --> [v2:172.21.15.188:6834/2269430612,v1:172.21.15.188:6836/2269430612] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55aa3ef5cc00 con 0x55aa3f06c3e0 2024-01-27T02:36:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.859+0000 7f076259c700 1 --2- 172.21.15.7:0/1536018928 >> [v2:172.21.15.188:6834/2269430612,v1:172.21.15.188:6836/2269430612] conn(0x55aa3f06c3e0 0x55aa3f115c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.859+0000 7f076259c700 1 --2- 172.21.15.7:0/1536018928 >> [v2:172.21.15.188:6834/2269430612,v1:172.21.15.188:6836/2269430612] conn(0x55aa3f06c3e0 0x55aa3f115c10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8b490dcca56c821 server_cookie=fde7937c656d2e11 in_seq=0 out_seq=0 2024-01-27T02:36:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.859+0000 7f0740ff9700 10 client.6123 ms_handle_connect on v2:172.21.15.188:6834/2269430612 2024-01-27T02:36:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.859+0000 7f0740ff9700 1 -- 172.21.15.7:0/1536018928 <== mds.0 v2:172.21.15.188:6834/2269430612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55aa3ef5cc00 con 0x55aa3f06c3e0 2024-01-27T02:36:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.859+0000 7f0740ff9700 10 client.6123 handle_command_reply: tid=0 2024-01-27T02:36:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.859+0000 7f0772071b80 1 client.6123 shutdown 2024-01-27T02:36:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.860+0000 7f0772071b80 1 -- 172.21.15.7:0/1536018928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f0b4360 msgr2=0x55aa3f1156d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.860+0000 7f0772071b80 1 --2- 172.21.15.7:0/1536018928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f0b4360 0x55aa3f1156d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f075c000f80 tx=0x7f075c02f000 comp rx=0 tx=0).stop 2024-01-27T02:36:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.860+0000 7f0772071b80 1 -- 172.21.15.7:0/1536018928 shutdown_connections 2024-01-27T02:36:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.860+0000 7f0772071b80 1 --2- 172.21.15.7:0/1536018928 >> [v2:172.21.15.188:6834/2269430612,v1:172.21.15.188:6836/2269430612] conn(0x55aa3f06c3e0 0x55aa3f115c10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.860+0000 7f0772071b80 1 --2- 172.21.15.7:0/1536018928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aa3f0b4360 0x55aa3f1156d0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.860+0000 7f0772071b80 1 --2- 172.21.15.7:0/1536018928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aa3f06b5d0 0x55aa3f0aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.860+0000 7f0740ff9700 0 client.6123 ms_handle_reset on v2:172.21.15.188:6834/2269430612 2024-01-27T02:36:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.860+0000 7f0772071b80 1 --2- 172.21.15.7:0/1536018928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa3f06c010 0x55aa3f0ab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.860+0000 7f0772071b80 1 -- 172.21.15.7:0/1536018928 >> 172.21.15.7:0/1536018928 conn(0x55aa3f06ba70 msgr2=0x55aa3f0967c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:12.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.860+0000 7f0772071b80 1 -- 172.21.15.7:0/1536018928 shutdown_connections 2024-01-27T02:36:12.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.860+0000 7f0772071b80 1 -- 172.21.15.7:0/1536018928 wait complete. 2024-01-27T02:36:12.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.860+0000 7f0772071b80 20 client.6123 trim_cache size 0 max 16384 2024-01-27T02:36:12.868 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:36:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:36:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-27T02:36:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5946, 2024-01-27T02:36:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-27T02:36:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-27T02:36:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-27T02:36:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 10.761495121999999, 2024-01-27T02:36:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 42, 2024-01-27T02:36:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 81, 2024-01-27T02:36:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 81, 2024-01-27T02:36:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 16.617693688999999 2024-01-27T02:36:12.869 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:36:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.860+0000 7f0768d6e700 1 -- 172.21.15.7:0/3831199691 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0744066f30 msgr2=0x7f07440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.860+0000 7f0768d6e700 1 --2- 172.21.15.7:0/3831199691 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0744066f30 0x7f07440693e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0754001840 tx=0x7f0754009040 comp rx=0 tx=0).stop 2024-01-27T02:36:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.860+0000 7f0768d6e700 1 -- 172.21.15.7:0/3831199691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07640b0b50 msgr2=0x7f07640c7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.860+0000 7f0768d6e700 1 --2- 172.21.15.7:0/3831199691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07640b0b50 0x7f07640c7110 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f074c00d890 tx=0x7f074c008040 comp rx=0 tx=0).stop 2024-01-27T02:36:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.860+0000 7f0768d6e700 1 -- 172.21.15.7:0/3831199691 shutdown_connections 2024-01-27T02:36:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.860+0000 7f0768d6e700 1 --2- 172.21.15.7:0/3831199691 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0744066f30 0x7f07440693e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.860+0000 7f0768d6e700 1 --2- 172.21.15.7:0/3831199691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07640cb7b0 0x7f0764155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.860+0000 7f0768d6e700 1 --2- 172.21.15.7:0/3831199691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0764063580 0x7f0764154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.860+0000 7f0768d6e700 1 --2- 172.21.15.7:0/3831199691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07640b0b50 0x7f07640c7110 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.860+0000 7f0768d6e700 1 -- 172.21.15.7:0/3831199691 >> 172.21.15.7:0/3831199691 conn(0x7f0764068480 msgr2=0x7f0764155370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.861+0000 7f0768d6e700 1 -- 172.21.15.7:0/3831199691 shutdown_connections 2024-01-27T02:36:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:12.861+0000 7f0768d6e700 1 -- 172.21.15.7:0/3831199691 wait complete. 2024-01-27T02:36:12.874 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:36:13.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.006+0000 7fe2b5aa7700 1 Processor -- start 2024-01-27T02:36:13.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.006+0000 7fe2b5aa7700 1 -- start start 2024-01-27T02:36:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.007+0000 7fe2b5aa7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2b00b1aa0 0x7fe2b00af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.007+0000 7fe2b5aa7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b006f870 con 0x7fe2b00a7bd0 2024-01-27T02:36:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.007+0000 7fe2b5aa7700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b006ec60 con 0x7fe2b00a7fa0 2024-01-27T02:36:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.007+0000 7fe2b5aa7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b006fe40 con 0x7fe2b00b1aa0 2024-01-27T02:36:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.007+0000 7fe2af7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe2b00a7bd0 0x7fe2b00b1390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39256/0 (socket says 172.21.15.7:39256) 2024-01-27T02:36:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.007+0000 7fe2af7fe700 1 -- 172.21.15.7:0/1795208359 learned_addr learned my addr 172.21.15.7:0/1795208359 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.008+0000 7fe2aeffd700 1 --2- 172.21.15.7:0/1795208359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2b00b1aa0 0x7fe2b00af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.008+0000 7fe2ae7fc700 1 -- 172.21.15.7:0/1795208359 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393647081 0 0) 0x7fe2b006f870 con 0x7fe2b00a7bd0 2024-01-27T02:36:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.008+0000 7fe2ae7fc700 1 -- 172.21.15.7:0/1795208359 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe298003570 con 0x7fe2b00a7bd0 2024-01-27T02:36:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.008+0000 7fe2ae7fc700 1 -- 172.21.15.7:0/1795208359 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3043872609 0 0) 0x7fe2b006ec60 con 0x7fe2b00a7fa0 2024-01-27T02:36:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.008+0000 7fe2ae7fc700 1 -- 172.21.15.7:0/1795208359 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2b006f870 con 0x7fe2b00a7fa0 2024-01-27T02:36:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.008+0000 7fe2aeffd700 1 -- 172.21.15.7:0/1795208359 >> v1:172.21.15.188:6789/0 conn(0x7fe2b00a7fa0 legacy=0x7fe2b0097bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.008+0000 7fe2aeffd700 1 -- 172.21.15.7:0/1795208359 >> v1:172.21.15.7:6789/0 conn(0x7fe2b00a7bd0 legacy=0x7fe2b00b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.008+0000 7fe2aeffd700 1 -- 172.21.15.7:0/1795208359 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b006ef40 con 0x7fe2b00b1aa0 2024-01-27T02:36:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.008+0000 7fe2aeffd700 1 --2- 172.21.15.7:0/1795208359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2b00b1aa0 0x7fe2b00af6d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009910 tx=0x7fe2a40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcdc0964b06c351c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.008+0000 7fe2ae7fc700 1 -- 172.21.15.7:0/1795208359 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4011630 con 0x7fe2b00b1aa0 2024-01-27T02:36:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.008+0000 7fe2ae7fc700 1 -- 172.21.15.7:0/1795208359 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a4011790 con 0x7fe2b00b1aa0 2024-01-27T02:36:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.008+0000 7fe2ae7fc700 1 -- 172.21.15.7:0/1795208359 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4011b30 con 0x7fe2b00b1aa0 2024-01-27T02:36:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.009+0000 7fe2b5aa7700 1 -- 172.21.15.7:0/1795208359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2b00b1aa0 msgr2=0x7fe2b00af6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.009+0000 7fe2b5aa7700 1 --2- 172.21.15.7:0/1795208359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2b00b1aa0 0x7fe2b00af6d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009910 tx=0x7fe2a40091a0 comp rx=0 tx=0).stop 2024-01-27T02:36:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.009+0000 7fe2b5aa7700 1 -- 172.21.15.7:0/1795208359 shutdown_connections 2024-01-27T02:36:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.009+0000 7fe2b5aa7700 1 --2- 172.21.15.7:0/1795208359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2b00b1aa0 0x7fe2b00af6d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.009+0000 7fe2b5aa7700 1 -- 172.21.15.7:0/1795208359 >> 172.21.15.7:0/1795208359 conn(0x7fe2b00679b0 msgr2=0x7fe2b0067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.009+0000 7fe2b5aa7700 1 -- 172.21.15.7:0/1795208359 shutdown_connections 2024-01-27T02:36:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.009+0000 7fe2b5aa7700 1 -- 172.21.15.7:0/1795208359 wait complete. 2024-01-27T02:36:13.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.010+0000 7fe2b5aa7700 1 Processor -- start 2024-01-27T02:36:13.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.010+0000 7fe2b5aa7700 1 -- start start 2024-01-27T02:36:13.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.011+0000 7fe2b5aa7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2b00a7bd0 0x7fe2b015a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:13.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.011+0000 7fe2b5aa7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2b00a7fa0 0x7fe2b015d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:13.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.011+0000 7fe2b5aa7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2b00b1aa0 0x7fe2b01525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:13.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.011+0000 7fe2b5aa7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b015d020 con 0x7fe2b00b1aa0 2024-01-27T02:36:13.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.011+0000 7fe2b5aa7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b015d9f0 con 0x7fe2b00a7bd0 2024-01-27T02:36:13.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.011+0000 7fe2b5aa7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b015c410 con 0x7fe2b00a7fa0 2024-01-27T02:36:13.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.011+0000 7fe2affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2b00b1aa0 0x7fe2b01525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.011+0000 7fe2affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2b00b1aa0 0x7fe2b01525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41726/0 (socket says 172.21.15.7:41726) 2024-01-27T02:36:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.011+0000 7fe2affff700 1 -- 172.21.15.7:0/4242955128 learned_addr learned my addr 172.21.15.7:0/4242955128 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.012+0000 7fe2aeffd700 1 --2- 172.21.15.7:0/4242955128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2b00a7fa0 0x7fe2b015d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.012+0000 7fe2af7fe700 1 --2- 172.21.15.7:0/4242955128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2b00a7bd0 0x7fe2b015a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.012+0000 7fe2affff700 1 -- 172.21.15.7:0/4242955128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2b00a7fa0 msgr2=0x7fe2b015d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.012+0000 7fe2affff700 1 --2- 172.21.15.7:0/4242955128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2b00a7fa0 0x7fe2b015d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.012+0000 7fe2affff700 1 -- 172.21.15.7:0/4242955128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2b00a7bd0 msgr2=0x7fe2b015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.012+0000 7fe2affff700 1 --2- 172.21.15.7:0/4242955128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2b00a7bd0 0x7fe2b015a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.012+0000 7fe2affff700 1 -- 172.21.15.7:0/4242955128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a4009420 con 0x7fe2b00b1aa0 2024-01-27T02:36:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.012+0000 7fe2affff700 1 --2- 172.21.15.7:0/4242955128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2b00b1aa0 0x7fe2b01525e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800c910 tx=0x7fe2a80027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.012+0000 7fe2977fe700 1 -- 172.21.15.7:0/4242955128 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a800a7e0 con 0x7fe2b00b1aa0 2024-01-27T02:36:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.012+0000 7fe2977fe700 1 -- 172.21.15.7:0/4242955128 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a800a960 con 0x7fe2b00b1aa0 2024-01-27T02:36:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.012+0000 7fe2b5aa7700 1 -- 172.21.15.7:0/4242955128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b015c6f0 con 0x7fe2b00b1aa0 2024-01-27T02:36:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.012+0000 7fe2977fe700 1 -- 172.21.15.7:0/4242955128 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a800ad20 con 0x7fe2b00b1aa0 2024-01-27T02:36:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.012+0000 7fe2b5aa7700 1 -- 172.21.15.7:0/4242955128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b015ccb0 con 0x7fe2b00b1aa0 2024-01-27T02:36:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.013+0000 7fe2b5aa7700 1 -- 172.21.15.7:0/4242955128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b015d190 con 0x7fe2b00b1aa0 2024-01-27T02:36:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.014+0000 7fe2977fe700 1 -- 172.21.15.7:0/4242955128 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe2a8012630 con 0x7fe2b00b1aa0 2024-01-27T02:36:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.014+0000 7fe2977fe700 1 --2- 172.21.15.7:0/4242955128 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe29c066f30 0x7fe29c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.014+0000 7fe2977fe700 1 -- 172.21.15.7:0/4242955128 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe2a80849f0 con 0x7fe2b00b1aa0 2024-01-27T02:36:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.014+0000 7fe2af7fe700 1 --2- 172.21.15.7:0/4242955128 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe29c066f30 0x7fe29c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.015+0000 7fe2af7fe700 1 --2- 172.21.15.7:0/4242955128 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe29c066f30 0x7fe29c0693e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe2b01577f0 tx=0x7fe2a0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:13.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.018+0000 7fe2977fe700 1 -- 172.21.15.7:0/4242955128 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2a8051d10 con 0x7fe2b00b1aa0 2024-01-27T02:36:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.274+0000 7fe2b5aa7700 1 -- 172.21.15.7:0/4242955128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe2b004c710 con 0x7fe2b00b1aa0 2024-01-27T02:36:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.275+0000 7fe2977fe700 1 -- 172.21.15.7:0/4242955128 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v42) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe2a8055330 con 0x7fe2b00b1aa0 2024-01-27T02:36:13.275 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:36:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.277+0000 7fe2b5aa7700 1 -- 172.21.15.7:0/4242955128 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe29c066f30 msgr2=0x7fe29c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.277+0000 7fe2b5aa7700 1 --2- 172.21.15.7:0/4242955128 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe29c066f30 0x7fe29c0693e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe2b01577f0 tx=0x7fe2a0009040 comp rx=0 tx=0).stop 2024-01-27T02:36:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.277+0000 7fe2b5aa7700 1 -- 172.21.15.7:0/4242955128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2b00b1aa0 msgr2=0x7fe2b01525e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.277+0000 7fe2b5aa7700 1 --2- 172.21.15.7:0/4242955128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2b00b1aa0 0x7fe2b01525e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800c910 tx=0x7fe2a80027a0 comp rx=0 tx=0).stop 2024-01-27T02:36:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.277+0000 7fe2b5aa7700 1 -- 172.21.15.7:0/4242955128 shutdown_connections 2024-01-27T02:36:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.277+0000 7fe2b5aa7700 1 --2- 172.21.15.7:0/4242955128 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe29c066f30 0x7fe29c0693e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.277+0000 7fe2b5aa7700 1 --2- 172.21.15.7:0/4242955128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2b00b1aa0 0x7fe2b01525e0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.277+0000 7fe2b5aa7700 1 --2- 172.21.15.7:0/4242955128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2b00a7bd0 0x7fe2b015a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.277+0000 7fe2b5aa7700 1 --2- 172.21.15.7:0/4242955128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2b00a7fa0 0x7fe2b015d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.277+0000 7fe2b5aa7700 1 -- 172.21.15.7:0/4242955128 >> 172.21.15.7:0/4242955128 conn(0x7fe2b00679b0 msgr2=0x7fe2b00b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:13.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.277+0000 7fe2b5aa7700 1 -- 172.21.15.7:0/4242955128 shutdown_connections 2024-01-27T02:36:13.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:13.277+0000 7fe2b5aa7700 1 -- 172.21.15.7:0/4242955128 wait complete. 2024-01-27T02:36:13.291 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-27T02:36:13.291 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:36:13.292 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-27T02:36:13.292 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-27T02:36:13.292 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-27T02:36:13.292 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-27T02:36:13.292 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-27T02:36:13.292 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:36:13.292 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:36:13.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:36:13.292 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-27T02:36:13.312 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:36:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:36:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:36:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:36:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-27T02:36:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:36:13.313 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:36:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-27T02:36:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-27T02:36:13.313 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:36:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-27T02:36:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-27T02:36:13.313 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:36:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-27T02:36:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-27T02:36:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-27T02:36:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42138sec preferred_lft 42138sec 2024-01-27T02:36:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-27T02:36:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:36:13.314 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:36:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-27T02:36:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-27T02:36:13.314 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:36:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 4a:57:d9:3c:c5:a0 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:36:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-27T02:36:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:36:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::2477:9eff:fe39:433c/64 scope link 2024-01-27T02:36:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:36:13.314 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:36:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 4a:57:d9:3c:c5:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-27T02:36:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::4857:d9ff:fe3c:c5a0/64 scope link 2024-01-27T02:36:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:36:13.315 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:36:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ca:6e:b8:df:53:bf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-27T02:36:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c86e:b8ff:fedf:53bf/64 scope link 2024-01-27T02:36:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:36:13.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:36:13.315 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:36:13.384 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-27T02:36:13.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:36:13.386 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-27T02:36:13.450 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-27T02:36:13.450 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:36:13.450 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-27T02:36:13.450 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-27T02:36:13.450 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:36:13.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:36:13.450 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-27T02:36:13.451 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-27T02:36:13.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-27T02:36:13.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-27T02:36:13.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-27T02:36:13.451 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-27T02:36:13.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:36:13.575 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-27T02:36:13.595 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-27T02:36:13.595 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-27T02:36:13.595 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:36:13.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:36:13.595 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-27T02:36:13.672 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:36:13.672 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-27T02:36:13.734 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:36:13.735 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:36:13.735 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-27T02:36:13.796 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:36:13.796 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:36:13.797 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:36:13.891 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:36:13.891 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:36:13.891 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:36:13.901 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:36:13.901 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:36:13.901 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:36:13.901 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:36:13.901 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:36:13.920 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:36:13.920 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:36:13.920 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:36:13.920 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:36:13.920 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:36:13.920 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:36:13.920 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:36:13.921 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:36:13.923 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:36:13.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:36:13.923 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-27T02:36:13.955 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:36:14.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.134+0000 7faa66ff4700 1 Processor -- start 2024-01-27T02:36:14.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.134+0000 7faa66ff4700 1 -- start start 2024-01-27T02:36:14.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.135+0000 7faa66ff4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faa60143520 0x7faa60063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.135+0000 7faa66ff4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa600cb7a0 con 0x7faa600aa8b0 2024-01-27T02:36:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.135+0000 7faa66ff4700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa600cb9f0 con 0x7faa601438f0 2024-01-27T02:36:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.135+0000 7faa66ff4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa600cbcd0 con 0x7faa60143520 2024-01-27T02:36:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.135+0000 7faa64d90700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faa60143520 0x7faa60063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.135+0000 7faa65591700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7faa600aa8b0 0x7faa6006e2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39266/0 (socket says 172.21.15.7:39266) 2024-01-27T02:36:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.135+0000 7faa65591700 1 -- 172.21.15.7:0/4040748523 learned_addr learned my addr 172.21.15.7:0/4040748523 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.136+0000 7faa64d90700 1 -- 172.21.15.7:0/4040748523 >> v1:172.21.15.188:6789/0 conn(0x7faa601438f0 legacy=0x7faa600c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.136+0000 7faa64d90700 1 -- 172.21.15.7:0/4040748523 >> v1:172.21.15.7:6789/0 conn(0x7faa600aa8b0 legacy=0x7faa6006e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.136+0000 7faa64d90700 1 -- 172.21.15.7:0/4040748523 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa600cbfb0 con 0x7faa60143520 2024-01-27T02:36:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.136+0000 7faa64d90700 1 --2- 172.21.15.7:0/4040748523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa60143520 0x7faa60063580 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7faa540098d0 tx=0x7faa54009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7181ff9c5562b56a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.136+0000 7faa5f7fe700 1 -- 172.21.15.7:0/4040748523 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa54009e70 con 0x7faa60143520 2024-01-27T02:36:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.136+0000 7faa5f7fe700 1 -- 172.21.15.7:0/4040748523 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa54005c10 con 0x7faa60143520 2024-01-27T02:36:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.136+0000 7faa5f7fe700 1 -- 172.21.15.7:0/4040748523 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5400b650 con 0x7faa60143520 2024-01-27T02:36:14.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.137+0000 7faa66ff4700 1 -- 172.21.15.7:0/4040748523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa60143520 msgr2=0x7faa60063580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:14.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.137+0000 7faa66ff4700 1 --2- 172.21.15.7:0/4040748523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa60143520 0x7faa60063580 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7faa540098d0 tx=0x7faa54009120 comp rx=0 tx=0).stop 2024-01-27T02:36:14.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.137+0000 7faa66ff4700 1 -- 172.21.15.7:0/4040748523 shutdown_connections 2024-01-27T02:36:14.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.137+0000 7faa66ff4700 1 --2- 172.21.15.7:0/4040748523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa60143520 0x7faa60063580 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:14.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.137+0000 7faa66ff4700 1 -- 172.21.15.7:0/4040748523 >> 172.21.15.7:0/4040748523 conn(0x7faa600cdbd0 msgr2=0x7faa600cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:14.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.137+0000 7faa66ff4700 1 -- 172.21.15.7:0/4040748523 shutdown_connections 2024-01-27T02:36:14.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.137+0000 7faa66ff4700 1 -- 172.21.15.7:0/4040748523 wait complete. 2024-01-27T02:36:14.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.138+0000 7faa66ff4700 1 Processor -- start 2024-01-27T02:36:14.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.138+0000 7faa66ff4700 1 -- start start 2024-01-27T02:36:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.139+0000 7faa66ff4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa600aa8b0 0x7faa6016b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.139+0000 7faa66ff4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa60143520 0x7faa60159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.139+0000 7faa66ff4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa601438f0 0x7faa600b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.139+0000 7faa66ff4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6016e130 con 0x7faa600aa8b0 2024-01-27T02:36:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.139+0000 7faa66ff4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6016e410 con 0x7faa60143520 2024-01-27T02:36:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.139+0000 7faa66ff4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6016e6f0 con 0x7faa601438f0 2024-01-27T02:36:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.139+0000 7faa64d90700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa600aa8b0 0x7faa6016b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.139+0000 7faa64d90700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa600aa8b0 0x7faa6016b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41746/0 (socket says 172.21.15.7:41746) 2024-01-27T02:36:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.139+0000 7faa64d90700 1 -- 172.21.15.7:0/4121560728 learned_addr learned my addr 172.21.15.7:0/4121560728 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.139+0000 7faa5ffff700 1 --2- 172.21.15.7:0/4121560728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa60143520 0x7faa60159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.139+0000 7faa64d90700 1 -- 172.21.15.7:0/4121560728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa601438f0 msgr2=0x7faa600b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.139+0000 7faa65591700 1 --2- 172.21.15.7:0/4121560728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa601438f0 0x7faa600b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.139+0000 7faa64d90700 1 --2- 172.21.15.7:0/4121560728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa601438f0 0x7faa600b2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.139+0000 7faa64d90700 1 -- 172.21.15.7:0/4121560728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa60143520 msgr2=0x7faa60159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.139+0000 7faa64d90700 1 --2- 172.21.15.7:0/4121560728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa60143520 0x7faa60159680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.139+0000 7faa64d90700 1 -- 172.21.15.7:0/4121560728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa540093c0 con 0x7faa600aa8b0 2024-01-27T02:36:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.140+0000 7faa5ffff700 1 --2- 172.21.15.7:0/4121560728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa60143520 0x7faa60159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:36:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.140+0000 7faa65591700 1 --2- 172.21.15.7:0/4121560728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa601438f0 0x7faa600b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:36:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.140+0000 7faa64d90700 1 --2- 172.21.15.7:0/4121560728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa600aa8b0 0x7faa6016b1f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7faa600c49d0 tx=0x7faa54016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.140+0000 7faa5cff9700 1 -- 172.21.15.7:0/4121560728 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5401dc80 con 0x7faa600aa8b0 2024-01-27T02:36:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.140+0000 7faa5cff9700 1 -- 172.21.15.7:0/4121560728 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5401dde0 con 0x7faa600aa8b0 2024-01-27T02:36:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.140+0000 7faa66ff4700 1 -- 172.21.15.7:0/4121560728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa6016e9d0 con 0x7faa600aa8b0 2024-01-27T02:36:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.140+0000 7faa5cff9700 1 -- 172.21.15.7:0/4121560728 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa54026680 con 0x7faa600aa8b0 2024-01-27T02:36:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.140+0000 7faa66ff4700 1 -- 172.21.15.7:0/4121560728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa6016dd70 con 0x7faa600aa8b0 2024-01-27T02:36:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.141+0000 7faa66ff4700 1 -- 172.21.15.7:0/4121560728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa6004c710 con 0x7faa600aa8b0 2024-01-27T02:36:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.141+0000 7faa5cff9700 1 -- 172.21.15.7:0/4121560728 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7faa5401a420 con 0x7faa600aa8b0 2024-01-27T02:36:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.142+0000 7faa5cff9700 1 --2- 172.21.15.7:0/4121560728 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faa44066f30 0x7faa440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.142+0000 7faa5cff9700 1 -- 172.21.15.7:0/4121560728 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faa54085ed0 con 0x7faa600aa8b0 2024-01-27T02:36:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.142+0000 7faa5ffff700 1 --2- 172.21.15.7:0/4121560728 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faa44066f30 0x7faa440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.143+0000 7faa5ffff700 1 --2- 172.21.15.7:0/4121560728 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faa44066f30 0x7faa440693e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faa6015d640 tx=0x7faa4c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:14.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.146+0000 7faa5cff9700 1 -- 172.21.15.7:0/4121560728 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa540531c0 con 0x7faa600aa8b0 2024-01-27T02:36:14.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.399+0000 7faa66ff4700 1 -- 172.21.15.7:0/4121560728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7faa600cc290 con 0x7faa600aa8b0 2024-01-27T02:36:14.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.400+0000 7faa5cff9700 1 -- 172.21.15.7:0/4121560728 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v42) v1 ==== 53+0+73 (secure 0 0 0) 0x7faa5402b070 con 0x7faa600aa8b0 2024-01-27T02:36:14.400 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:36:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.402+0000 7faa66ff4700 1 -- 172.21.15.7:0/4121560728 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faa44066f30 msgr2=0x7faa440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.402+0000 7faa66ff4700 1 --2- 172.21.15.7:0/4121560728 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faa44066f30 0x7faa440693e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faa6015d640 tx=0x7faa4c009040 comp rx=0 tx=0).stop 2024-01-27T02:36:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.402+0000 7faa66ff4700 1 -- 172.21.15.7:0/4121560728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa600aa8b0 msgr2=0x7faa6016b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.402+0000 7faa66ff4700 1 --2- 172.21.15.7:0/4121560728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa600aa8b0 0x7faa6016b1f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7faa600c49d0 tx=0x7faa54016000 comp rx=0 tx=0).stop 2024-01-27T02:36:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.402+0000 7faa66ff4700 1 -- 172.21.15.7:0/4121560728 shutdown_connections 2024-01-27T02:36:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.402+0000 7faa66ff4700 1 --2- 172.21.15.7:0/4121560728 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faa44066f30 0x7faa440693e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.402+0000 7faa66ff4700 1 --2- 172.21.15.7:0/4121560728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa601438f0 0x7faa600b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.402+0000 7faa66ff4700 1 --2- 172.21.15.7:0/4121560728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa600aa8b0 0x7faa6016b1f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.402+0000 7faa66ff4700 1 --2- 172.21.15.7:0/4121560728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa60143520 0x7faa60159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.402+0000 7faa66ff4700 1 -- 172.21.15.7:0/4121560728 >> 172.21.15.7:0/4121560728 conn(0x7faa600cdbd0 msgr2=0x7faa6006c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:14.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.402+0000 7faa66ff4700 1 -- 172.21.15.7:0/4121560728 shutdown_connections 2024-01-27T02:36:14.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:14.402+0000 7faa66ff4700 1 -- 172.21.15.7:0/4121560728 wait complete. 2024-01-27T02:36:14.416 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-27T02:36:14.416 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:36:14.416 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-01-27T02:36:14.416 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-27T02:36:14.416 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-27T02:36:14.416 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-27T02:36:14.416 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-27T02:36:14.416 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:36:14.417 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:36:14.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:36:14.417 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-01-27T02:36:14.438 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:36:14.439 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:36:14.439 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:36:14.439 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:36:14.439 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-01-27T02:36:14.439 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:36:14.439 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:36:14.439 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:36:14.439 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-01-27T02:36:14.439 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:36:14.439 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:36:14.439 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-01-27T02:36:14.439 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:36:14.439 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:36:14.439 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:36:14.440 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:36:14.440 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-27T02:36:14.440 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42144sec preferred_lft 42144sec 2024-01-27T02:36:14.440 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b1:d6f7:baf0:bc17/64 scope link noprefixroute 2024-01-27T02:36:14.440 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:36:14.440 INFO:teuthology.orchestra.run.smithi188.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:36:14.440 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b0:92:71:90:9d brd ff:ff:ff:ff:ff:ff 2024-01-27T02:36:14.440 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-27T02:36:14.440 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:36:14.440 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::10f8:32ff:feed:a752/64 scope link 2024-01-27T02:36:14.440 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:36:14.440 INFO:teuthology.orchestra.run.smithi188.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:36:14.440 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b0:92:71:90:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-27T02:36:14.440 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b8b0:92ff:fe71:909d/64 scope link 2024-01-27T02:36:14.441 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:36:14.441 INFO:teuthology.orchestra.run.smithi188.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:36:14.441 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ee:78:03:20:3a:a9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-27T02:36:14.441 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec78:3ff:fe20:3aa9/64 scope link 2024-01-27T02:36:14.441 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:36:14.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:36:14.441 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:36:14.510 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-01-27T02:36:14.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:36:14.512 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-01-27T02:36:14.575 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-01-27T02:36:14.575 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:36:14.575 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-27T02:36:14.575 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-27T02:36:14.575 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:36:14.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:36:14.576 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-27T02:36:14.576 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-27T02:36:14.576 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-27T02:36:14.576 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-27T02:36:14.576 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-27T02:36:14.576 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-27T02:36:14.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:36:14.687 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-01-27T02:36:14.704 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-27T02:36:14.705 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-27T02:36:14.705 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:36:14.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:36:14.705 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2024-01-27T02:36:14.773 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:36:14.773 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-27T02:36:14.830 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:36:14.830 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:36:14.830 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-27T02:36:14.886 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:36:14.887 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:36:14.887 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:36:14.969 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:36:14.969 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:36:14.969 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:36:14.977 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:36:14.978 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:36:14.978 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:36:14.978 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:36:14.978 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:36:14.995 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:36:14.995 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:36:14.995 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:36:14.995 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:36:14.995 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:36:14.995 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:36:14.995 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:36:14.995 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:36:14.997 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:36:14.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:36:14.997 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-27T02:36:15.063 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:36:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.204+0000 7f6e235b2700 1 Processor -- start 2024-01-27T02:36:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.204+0000 7f6e235b2700 1 -- start start 2024-01-27T02:36:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.205+0000 7f6e235b2700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6e1c15a100 0x7f6e1c0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.205+0000 7f6e235b2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6e1c0c4cc0 0x7f6e1c163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.205+0000 7f6e235b2700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e1c0ce5e0 con 0x7f6e1c0cb5b0 2024-01-27T02:36:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.205+0000 7f6e235b2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e1c0cd9d0 con 0x7f6e1c0c4cc0 2024-01-27T02:36:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.205+0000 7f6e235b2700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e1c0cdcb0 con 0x7f6e1c15a100 2024-01-27T02:36:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.205+0000 7f6e21b4f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6e1c0c4cc0 0x7f6e1c163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.205+0000 7f6e21b4f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6e1c0c4cc0 0x7f6e1c163d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41758/0 (socket says 172.21.15.7:41758) 2024-01-27T02:36:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.205+0000 7f6e21b4f700 1 -- 172.21.15.7:0/98232300 learned_addr learned my addr 172.21.15.7:0/98232300 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.206+0000 7f6e2134e700 1 --2- 172.21.15.7:0/98232300 >> v2:172.21.15.188:3300/0 conn(0x7f6e1c15a100 0x7f6e1c0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.206+0000 7f6e21b4f700 1 -- 172.21.15.7:0/98232300 >> v2:172.21.15.188:3300/0 conn(0x7f6e1c15a100 msgr2=0x7f6e1c0cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.206+0000 7f6e21b4f700 1 --2- 172.21.15.7:0/98232300 >> v2:172.21.15.188:3300/0 conn(0x7f6e1c15a100 0x7f6e1c0cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.206+0000 7f6e21b4f700 1 -- 172.21.15.7:0/98232300 >> v1:172.21.15.188:6789/0 conn(0x7f6e1c0cb5b0 legacy=0x7f6e1c0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.206+0000 7f6e21b4f700 1 -- 172.21.15.7:0/98232300 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1c0cdf90 con 0x7f6e1c0c4cc0 2024-01-27T02:36:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.206+0000 7f6e21b4f700 1 --2- 172.21.15.7:0/98232300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e1c0c4cc0 0x7f6e1c163d90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c06d710 tx=0x7f6e14009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e55b9f083088b3 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.206+0000 7f6e13fff700 1 -- 172.21.15.7:0/98232300 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1400e070 con 0x7f6e1c0c4cc0 2024-01-27T02:36:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.206+0000 7f6e13fff700 1 -- 172.21.15.7:0/98232300 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e1400ba10 con 0x7f6e1c0c4cc0 2024-01-27T02:36:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.206+0000 7f6e13fff700 1 -- 172.21.15.7:0/98232300 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e140092e0 con 0x7f6e1c0c4cc0 2024-01-27T02:36:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.207+0000 7f6e235b2700 1 -- 172.21.15.7:0/98232300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e1c0c4cc0 msgr2=0x7f6e1c163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.207+0000 7f6e235b2700 1 --2- 172.21.15.7:0/98232300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e1c0c4cc0 0x7f6e1c163d90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c06d710 tx=0x7f6e14009840 comp rx=0 tx=0).stop 2024-01-27T02:36:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.207+0000 7f6e235b2700 1 -- 172.21.15.7:0/98232300 shutdown_connections 2024-01-27T02:36:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.207+0000 7f6e235b2700 1 --2- 172.21.15.7:0/98232300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e1c0c4cc0 0x7f6e1c163d90 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.207+0000 7f6e235b2700 1 --2- 172.21.15.7:0/98232300 >> v2:172.21.15.188:3300/0 conn(0x7f6e1c15a100 0x7f6e1c0cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.207+0000 7f6e235b2700 1 -- 172.21.15.7:0/98232300 >> 172.21.15.7:0/98232300 conn(0x7f6e1c098020 msgr2=0x7f6e1c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.207+0000 7f6e235b2700 1 -- 172.21.15.7:0/98232300 shutdown_connections 2024-01-27T02:36:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.207+0000 7f6e235b2700 1 -- 172.21.15.7:0/98232300 wait complete. 2024-01-27T02:36:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.208+0000 7f6e235b2700 1 Processor -- start 2024-01-27T02:36:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.208+0000 7f6e235b2700 1 -- start start 2024-01-27T02:36:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.209+0000 7f6e235b2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e1c0c4cc0 0x7f6e1c16a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.209+0000 7f6e235b2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e1c0cb5b0 0x7f6e1c1535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.209+0000 7f6e235b2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e1c15a100 0x7f6e1c16e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.209+0000 7f6e235b2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c153b20 con 0x7f6e1c0c4cc0 2024-01-27T02:36:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.209+0000 7f6e235b2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c15a6e0 con 0x7f6e1c15a100 2024-01-27T02:36:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.209+0000 7f6e235b2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c16e690 con 0x7f6e1c0cb5b0 2024-01-27T02:36:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.209+0000 7f6e2134e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e1c0c4cc0 0x7f6e1c16a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.209+0000 7f6e2134e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e1c0c4cc0 0x7f6e1c16a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41770/0 (socket says 172.21.15.7:41770) 2024-01-27T02:36:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.209+0000 7f6e2134e700 1 -- 172.21.15.7:0/3356251074 learned_addr learned my addr 172.21.15.7:0/3356251074 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.209+0000 7f6e21b4f700 1 --2- 172.21.15.7:0/3356251074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e1c15a100 0x7f6e1c16e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.209+0000 7f6e20b4d700 1 --2- 172.21.15.7:0/3356251074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e1c0cb5b0 0x7f6e1c1535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.210+0000 7f6e2134e700 1 -- 172.21.15.7:0/3356251074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e1c0cb5b0 msgr2=0x7f6e1c1535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.210+0000 7f6e2134e700 1 --2- 172.21.15.7:0/3356251074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e1c0cb5b0 0x7f6e1c1535e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.210+0000 7f6e2134e700 1 -- 172.21.15.7:0/3356251074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e1c15a100 msgr2=0x7f6e1c16e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.210+0000 7f6e2134e700 1 --2- 172.21.15.7:0/3356251074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e1c15a100 0x7f6e1c16e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.210+0000 7f6e2134e700 1 -- 172.21.15.7:0/3356251074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e14009ae0 con 0x7f6e1c0c4cc0 2024-01-27T02:36:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.210+0000 7f6e2134e700 1 --2- 172.21.15.7:0/3356251074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e1c0c4cc0 0x7f6e1c16a0b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800d890 tx=0x7f6e18008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.210+0000 7f6e117fa700 1 -- 172.21.15.7:0/3356251074 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e180125f0 con 0x7f6e1c0c4cc0 2024-01-27T02:36:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.210+0000 7f6e117fa700 1 -- 172.21.15.7:0/3356251074 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e18012750 con 0x7f6e1c0c4cc0 2024-01-27T02:36:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.210+0000 7f6e117fa700 1 -- 172.21.15.7:0/3356251074 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18012b10 con 0x7f6e1c0c4cc0 2024-01-27T02:36:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.210+0000 7f6e235b2700 1 -- 172.21.15.7:0/3356251074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e1c154010 con 0x7f6e1c0c4cc0 2024-01-27T02:36:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.210+0000 7f6e235b2700 1 -- 172.21.15.7:0/3356251074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e1c16a9f0 con 0x7f6e1c0c4cc0 2024-01-27T02:36:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.211+0000 7f6e235b2700 1 -- 172.21.15.7:0/3356251074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e1c15a9c0 con 0x7f6e1c0c4cc0 2024-01-27T02:36:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.211+0000 7f6e117fa700 1 -- 172.21.15.7:0/3356251074 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6e1801e460 con 0x7f6e1c0c4cc0 2024-01-27T02:36:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.212+0000 7f6e117fa700 1 --2- 172.21.15.7:0/3356251074 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6e00066f30 0x7f6e000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.212+0000 7f6e117fa700 1 -- 172.21.15.7:0/3356251074 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6e180849a0 con 0x7f6e1c0c4cc0 2024-01-27T02:36:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.212+0000 7f6e20b4d700 1 --2- 172.21.15.7:0/3356251074 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6e00066f30 0x7f6e000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:15.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.213+0000 7f6e20b4d700 1 --2- 172.21.15.7:0/3356251074 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6e00066f30 0x7f6e000693e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c158f60 tx=0x7f6e0c007ea0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:15.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.216+0000 7f6e117fa700 1 -- 172.21.15.7:0/3356251074 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e180556a0 con 0x7f6e1c0c4cc0 2024-01-27T02:36:15.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.465+0000 7f6e235b2700 1 -- 172.21.15.7:0/3356251074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e1c04c710 con 0x7f6e1c0c4cc0 2024-01-27T02:36:15.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.466+0000 7f6e117fa700 1 -- 172.21.15.7:0/3356251074 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4590 (secure 0 0 0) 0x7f6e18055230 con 0x7f6e1c0c4cc0 2024-01-27T02:36:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:36:15.467 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5839,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/100533817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":100533817},{"type":"v1","addr":"172.21.15.7:6834","nonce":100533817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5842,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/223425097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":223425097},{"type":"v1","addr":"172.21.15.7:6835","nonce":223425097}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5848,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/950040140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":950040140},{"type":"v1","addr":"172.21.15.188:6837","nonce":950040140}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:36:01.502818+0000","modified":"2024-01-27T02:36:05.109352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5946":{"gid":5946,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6836/2269430612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2269430612},{"type":"v1","addr":"172.21.15.188:6836","nonce":2269430612}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2024-01-27T02:36:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.468+0000 7f6e235b2700 1 -- 172.21.15.7:0/3356251074 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6e00066f30 msgr2=0x7f6e000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.468+0000 7f6e235b2700 1 --2- 172.21.15.7:0/3356251074 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6e00066f30 0x7f6e000693e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c158f60 tx=0x7f6e0c007ea0 comp rx=0 tx=0).stop 2024-01-27T02:36:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.468+0000 7f6e235b2700 1 -- 172.21.15.7:0/3356251074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e1c0c4cc0 msgr2=0x7f6e1c16a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.468+0000 7f6e235b2700 1 --2- 172.21.15.7:0/3356251074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e1c0c4cc0 0x7f6e1c16a0b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800d890 tx=0x7f6e18008040 comp rx=0 tx=0).stop 2024-01-27T02:36:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.468+0000 7f6e235b2700 1 -- 172.21.15.7:0/3356251074 shutdown_connections 2024-01-27T02:36:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.468+0000 7f6e235b2700 1 --2- 172.21.15.7:0/3356251074 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6e00066f30 0x7f6e000693e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.468+0000 7f6e235b2700 1 --2- 172.21.15.7:0/3356251074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e1c15a100 0x7f6e1c16e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.468+0000 7f6e235b2700 1 --2- 172.21.15.7:0/3356251074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e1c0c4cc0 0x7f6e1c16a0b0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.468+0000 7f6e235b2700 1 --2- 172.21.15.7:0/3356251074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e1c0cb5b0 0x7f6e1c1535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.468+0000 7f6e235b2700 1 -- 172.21.15.7:0/3356251074 >> 172.21.15.7:0/3356251074 conn(0x7f6e1c098020 msgr2=0x7f6e1c0ca200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.468+0000 7f6e235b2700 1 -- 172.21.15.7:0/3356251074 shutdown_connections 2024-01-27T02:36:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:15.468+0000 7f6e235b2700 1 -- 172.21.15.7:0/3356251074 wait complete. 2024-01-27T02:36:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 42 2024-01-27T02:36:15.482 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2024-01-27T02:36:15.620 INFO:teuthology.orchestra.run.smithi188.stdout:{"cluster_fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","whoami":0,"id":5946,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":13.511294117,"mdsmap_epoch":42,"osdmap_epoch":81,"osdmap_epoch_barrier":81,"uptime":19.367491843} 2024-01-27T02:36:15.620 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", "fs_name": "cephfs", "id": 5946, "mdsmap_epoch": 42, "osdmap_epoch": 81, "osdmap_epoch_barrier": 81, "rank_uptime": 13.511294117, "state": "up:active", "uptime": 19.367491843, "want_state": "up:active", "whoami": 0 } 2024-01-27T02:36:15.621 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-01-27T02:36:15.776 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_mon_report_interval":"5"} 2024-01-27T02:36:15.777 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-01-27T02:36:15.777 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-01-27T02:36:15.918 INFO:teuthology.orchestra.run.smithi007.stdout:{"mon_tick_interval":"5"} 2024-01-27T02:36:15.919 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-01-27T02:36:15.919 INFO:tasks.cephfs.test_full:pool capacity 557082176, 531MB should be enough to fill it 2024-01-27T02:36:15.919 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-01-27T02:36:15.919 DEBUG:teuthology.orchestra.run.smithi007:> import time 2024-01-27T02:36:15.919 DEBUG:teuthology.orchestra.run.smithi007:> import datetime 2024-01-27T02:36:15.919 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2024-01-27T02:36:15.919 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-01-27T02:36:15.919 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:36:15.919 DEBUG:teuthology.orchestra.run.smithi007:> # Write some buffered data through before going full, all should be well 2024-01-27T02:36:15.920 DEBUG:teuthology.orchestra.run.smithi007:> print("writing some data through which we expect to succeed") 2024-01-27T02:36:15.920 DEBUG:teuthology.orchestra.run.smithi007:> bytes = 0 2024-01-27T02:36:15.920 DEBUG:teuthology.orchestra.run.smithi007:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-01-27T02:36:15.920 DEBUG:teuthology.orchestra.run.smithi007:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-01-27T02:36:15.920 DEBUG:teuthology.orchestra.run.smithi007:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2024-01-27T02:36:15.920 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f) 2024-01-27T02:36:15.920 DEBUG:teuthology.orchestra.run.smithi007:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-01-27T02:36:15.920 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:36:15.920 DEBUG:teuthology.orchestra.run.smithi007:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-01-27T02:36:15.920 DEBUG:teuthology.orchestra.run.smithi007:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-01-27T02:36:15.920 DEBUG:teuthology.orchestra.run.smithi007:> # buffered IO, the error should always only appear from fsync and not 2024-01-27T02:36:15.920 DEBUG:teuthology.orchestra.run.smithi007:> # from write 2024-01-27T02:36:15.920 DEBUG:teuthology.orchestra.run.smithi007:> full = False 2024-01-27T02:36:15.920 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:36:15.920 DEBUG:teuthology.orchestra.run.smithi007:> for n in range(0, int(531 * 1.1)): 2024-01-27T02:36:15.920 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-01-27T02:36:15.921 DEBUG:teuthology.orchestra.run.smithi007:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-01-27T02:36:15.921 DEBUG:teuthology.orchestra.run.smithi007:> print("wrote bytes via buffered write, moving on to fsync") 2024-01-27T02:36:15.921 DEBUG:teuthology.orchestra.run.smithi007:> except OSError as e: 2024-01-27T02:36:15.921 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-01-27T02:36:15.921 DEBUG:teuthology.orchestra.run.smithi007:> print("Unexpected error %s from write() instead of fsync()" % e) 2024-01-27T02:36:15.921 DEBUG:teuthology.orchestra.run.smithi007:> raise 2024-01-27T02:36:15.921 DEBUG:teuthology.orchestra.run.smithi007:> else: 2024-01-27T02:36:15.921 DEBUG:teuthology.orchestra.run.smithi007:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-01-27T02:36:15.921 DEBUG:teuthology.orchestra.run.smithi007:> full = True 2024-01-27T02:36:15.921 DEBUG:teuthology.orchestra.run.smithi007:> break 2024-01-27T02:36:15.921 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:36:15.921 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-01-27T02:36:15.921 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f) 2024-01-27T02:36:15.921 DEBUG:teuthology.orchestra.run.smithi007:> print("fsync'"'"'ed successfully") 2024-01-27T02:36:15.921 DEBUG:teuthology.orchestra.run.smithi007:> except OSError as e: 2024-01-27T02:36:15.921 DEBUG:teuthology.orchestra.run.smithi007:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-01-27T02:36:15.922 DEBUG:teuthology.orchestra.run.smithi007:> full = True 2024-01-27T02:36:15.922 DEBUG:teuthology.orchestra.run.smithi007:> break 2024-01-27T02:36:15.922 DEBUG:teuthology.orchestra.run.smithi007:> else: 2024-01-27T02:36:15.922 DEBUG:teuthology.orchestra.run.smithi007:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-01-27T02:36:15.922 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:36:15.922 DEBUG:teuthology.orchestra.run.smithi007:> if n > 531 * 0.9: 2024-01-27T02:36:15.922 DEBUG:teuthology.orchestra.run.smithi007:> # Be cautious in the last region where we expect to hit 2024-01-27T02:36:15.922 DEBUG:teuthology.orchestra.run.smithi007:> # the full condition, so that we don'"'"'t overshoot too dramatically 2024-01-27T02:36:15.922 DEBUG:teuthology.orchestra.run.smithi007:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-01-27T02:36:15.922 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(15.0) 2024-01-27T02:36:15.922 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:36:15.922 DEBUG:teuthology.orchestra.run.smithi007:> if not full: 2024-01-27T02:36:15.922 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2024-01-27T02:36:15.922 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:36:15.922 DEBUG:teuthology.orchestra.run.smithi007:> # close() should not raise an error because we already caught it in 2024-01-27T02:36:15.922 DEBUG:teuthology.orchestra.run.smithi007:> # fsync. There shouldn'"'"'t have been any more writeback errors 2024-01-27T02:36:15.923 DEBUG:teuthology.orchestra.run.smithi007:> # since then because all IOs got cancelled on the full flag. 2024-01-27T02:36:15.923 DEBUG:teuthology.orchestra.run.smithi007:> print("calling close") 2024-01-27T02:36:15.923 DEBUG:teuthology.orchestra.run.smithi007:> os.close(f) 2024-01-27T02:36:15.923 DEBUG:teuthology.orchestra.run.smithi007:> print("close() did not raise error") 2024-01-27T02:36:15.923 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:36:15.923 DEBUG:teuthology.orchestra.run.smithi007:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-01-27T02:36:15.923 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-01-27T02:36:16.523 INFO:teuthology.orchestra.run.smithi007.stdout:writing some data through which we expect to succeed 2024-01-27T02:36:16.523 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-01-27T02:36:16.524 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.524 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.524 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 1.00 MB 2024-01-27T02:36:16.524 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.524 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.524 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 2.00 MB 2024-01-27T02:36:16.524 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.524 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.524 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 3.00 MB 2024-01-27T02:36:16.524 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.524 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.525 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 4.00 MB 2024-01-27T02:36:16.525 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.525 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.525 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 5.00 MB 2024-01-27T02:36:16.525 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.525 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.525 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 6.00 MB 2024-01-27T02:36:16.525 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.525 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.525 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 7.00 MB 2024-01-27T02:36:16.525 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.525 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.526 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 8.00 MB 2024-01-27T02:36:16.526 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.526 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.526 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 9.00 MB 2024-01-27T02:36:16.526 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.526 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.526 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 10.00 MB 2024-01-27T02:36:16.526 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.526 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.526 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 11.00 MB 2024-01-27T02:36:16.526 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.526 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.526 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 12.00 MB 2024-01-27T02:36:16.527 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.527 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.527 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 13.00 MB 2024-01-27T02:36:16.527 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.527 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.527 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 14.00 MB 2024-01-27T02:36:16.527 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.527 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.527 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 15.00 MB 2024-01-27T02:36:16.527 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.527 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.527 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 16.00 MB 2024-01-27T02:36:16.528 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.528 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.528 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 17.00 MB 2024-01-27T02:36:16.528 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.528 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.528 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 18.00 MB 2024-01-27T02:36:16.528 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.528 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.528 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 19.00 MB 2024-01-27T02:36:16.528 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.528 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.528 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 20.00 MB 2024-01-27T02:36:16.529 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.529 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.529 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 21.00 MB 2024-01-27T02:36:16.529 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.529 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.529 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 22.00 MB 2024-01-27T02:36:16.529 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.529 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.529 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 23.00 MB 2024-01-27T02:36:16.529 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.529 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.529 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 24.00 MB 2024-01-27T02:36:16.529 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.530 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.530 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 25.00 MB 2024-01-27T02:36:16.530 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.530 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.530 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 26.00 MB 2024-01-27T02:36:16.530 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.530 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.530 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 27.00 MB 2024-01-27T02:36:16.530 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.530 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.530 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 28.00 MB 2024-01-27T02:36:16.530 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.530 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.531 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 29.00 MB 2024-01-27T02:36:16.531 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.531 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.531 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 30.00 MB 2024-01-27T02:36:16.531 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.531 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.531 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 31.00 MB 2024-01-27T02:36:16.531 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.531 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.531 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 32.00 MB 2024-01-27T02:36:16.531 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.531 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.532 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 33.00 MB 2024-01-27T02:36:16.532 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.532 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.532 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 34.00 MB 2024-01-27T02:36:16.532 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.532 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.532 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 35.00 MB 2024-01-27T02:36:16.532 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.532 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.532 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 36.00 MB 2024-01-27T02:36:16.532 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.532 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.533 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 37.00 MB 2024-01-27T02:36:16.533 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.533 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.533 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 38.00 MB 2024-01-27T02:36:16.533 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.533 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.533 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 39.00 MB 2024-01-27T02:36:16.533 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.533 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.533 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 40.00 MB 2024-01-27T02:36:16.533 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.533 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.533 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 41.00 MB 2024-01-27T02:36:16.534 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.534 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.534 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 42.00 MB 2024-01-27T02:36:16.534 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.534 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.534 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 43.00 MB 2024-01-27T02:36:16.534 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.534 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.534 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 44.00 MB 2024-01-27T02:36:16.534 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.534 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.535 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 45.00 MB 2024-01-27T02:36:16.535 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.535 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.535 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 46.00 MB 2024-01-27T02:36:16.535 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.535 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.535 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 47.00 MB 2024-01-27T02:36:16.535 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.535 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.535 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 48.00 MB 2024-01-27T02:36:16.535 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.535 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.535 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 49.00 MB 2024-01-27T02:36:16.536 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.536 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.536 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 50.00 MB 2024-01-27T02:36:16.536 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.536 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.536 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 51.00 MB 2024-01-27T02:36:16.536 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.536 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.536 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 52.00 MB 2024-01-27T02:36:16.536 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.536 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.536 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 53.00 MB 2024-01-27T02:36:16.537 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.537 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.537 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 54.00 MB 2024-01-27T02:36:16.537 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.537 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.537 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 55.00 MB 2024-01-27T02:36:16.537 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.537 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.537 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 56.00 MB 2024-01-27T02:36:16.537 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.537 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.537 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 57.00 MB 2024-01-27T02:36:16.538 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.538 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.538 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 58.00 MB 2024-01-27T02:36:16.538 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.538 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.538 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 59.00 MB 2024-01-27T02:36:16.538 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.538 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.538 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 60.00 MB 2024-01-27T02:36:16.539 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.539 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.539 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 61.00 MB 2024-01-27T02:36:16.539 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.539 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.539 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 62.00 MB 2024-01-27T02:36:16.539 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.539 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.539 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 63.00 MB 2024-01-27T02:36:16.540 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.540 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.540 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 64.00 MB 2024-01-27T02:36:16.540 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.540 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.540 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 65.00 MB 2024-01-27T02:36:16.540 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.540 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.540 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 66.00 MB 2024-01-27T02:36:16.541 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.541 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.541 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 67.00 MB 2024-01-27T02:36:16.541 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.541 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.541 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 68.00 MB 2024-01-27T02:36:16.541 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.541 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.541 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 69.00 MB 2024-01-27T02:36:16.541 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.541 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.541 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 70.00 MB 2024-01-27T02:36:16.541 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.542 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.542 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 71.00 MB 2024-01-27T02:36:16.542 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.542 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.542 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 72.00 MB 2024-01-27T02:36:16.542 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.542 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.542 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 73.00 MB 2024-01-27T02:36:16.542 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.542 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.542 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 74.00 MB 2024-01-27T02:36:16.542 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.543 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.543 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 75.00 MB 2024-01-27T02:36:16.543 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.543 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.543 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 76.00 MB 2024-01-27T02:36:16.543 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.543 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.543 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 77.00 MB 2024-01-27T02:36:16.543 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.543 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.543 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 78.00 MB 2024-01-27T02:36:16.543 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.544 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.544 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 79.00 MB 2024-01-27T02:36:16.544 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.544 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.544 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 80.00 MB 2024-01-27T02:36:16.764 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.764 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.764 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 81.00 MB 2024-01-27T02:36:16.765 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.765 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.765 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 82.00 MB 2024-01-27T02:36:16.765 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.765 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.765 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 83.00 MB 2024-01-27T02:36:16.765 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.765 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.765 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 84.00 MB 2024-01-27T02:36:16.765 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.765 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.765 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 85.00 MB 2024-01-27T02:36:16.765 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.765 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.766 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 86.00 MB 2024-01-27T02:36:16.766 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.766 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.766 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 87.00 MB 2024-01-27T02:36:16.766 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.766 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.766 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 88.00 MB 2024-01-27T02:36:16.766 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.766 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.766 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 89.00 MB 2024-01-27T02:36:16.766 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.766 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.766 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 90.00 MB 2024-01-27T02:36:16.767 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.767 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.767 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 91.00 MB 2024-01-27T02:36:16.767 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.767 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.767 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 92.00 MB 2024-01-27T02:36:16.767 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.767 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.767 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 93.00 MB 2024-01-27T02:36:16.767 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.767 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.767 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 94.00 MB 2024-01-27T02:36:16.767 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.767 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.768 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 95.00 MB 2024-01-27T02:36:16.768 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.768 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.768 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 96.00 MB 2024-01-27T02:36:16.768 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.768 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.768 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 97.00 MB 2024-01-27T02:36:16.768 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.768 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.768 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 98.00 MB 2024-01-27T02:36:16.768 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.768 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.768 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 99.00 MB 2024-01-27T02:36:16.769 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.769 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.769 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 100.00 MB 2024-01-27T02:36:16.769 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.769 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.769 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 101.00 MB 2024-01-27T02:36:16.769 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.769 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.769 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 102.00 MB 2024-01-27T02:36:16.769 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.769 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.769 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 103.00 MB 2024-01-27T02:36:16.769 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.770 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.770 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 104.00 MB 2024-01-27T02:36:16.770 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.770 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.770 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 105.00 MB 2024-01-27T02:36:16.770 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.770 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.770 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 106.00 MB 2024-01-27T02:36:16.770 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.770 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.770 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 107.00 MB 2024-01-27T02:36:16.770 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.770 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.770 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 108.00 MB 2024-01-27T02:36:16.771 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.771 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.771 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 109.00 MB 2024-01-27T02:36:16.771 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.771 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.771 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 110.00 MB 2024-01-27T02:36:16.771 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.771 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.771 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 111.00 MB 2024-01-27T02:36:16.771 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.771 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.771 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 112.00 MB 2024-01-27T02:36:16.771 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.771 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.772 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 113.00 MB 2024-01-27T02:36:16.772 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.772 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.772 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 114.00 MB 2024-01-27T02:36:16.772 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.772 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.772 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 115.00 MB 2024-01-27T02:36:16.772 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.772 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.772 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 116.00 MB 2024-01-27T02:36:16.772 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.772 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.772 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 117.00 MB 2024-01-27T02:36:16.772 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.773 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.773 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 118.00 MB 2024-01-27T02:36:16.773 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.773 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.773 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 119.00 MB 2024-01-27T02:36:16.773 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.773 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.773 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 120.00 MB 2024-01-27T02:36:16.773 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.773 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.773 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 121.00 MB 2024-01-27T02:36:16.773 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.773 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.773 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 122.00 MB 2024-01-27T02:36:16.773 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.774 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.774 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 123.00 MB 2024-01-27T02:36:16.774 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.774 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.774 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 124.00 MB 2024-01-27T02:36:16.774 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.774 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.774 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 125.00 MB 2024-01-27T02:36:16.774 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.774 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.774 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 126.00 MB 2024-01-27T02:36:16.774 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.774 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.774 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 127.00 MB 2024-01-27T02:36:16.775 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.775 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.775 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 128.00 MB 2024-01-27T02:36:16.775 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.775 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.775 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 129.00 MB 2024-01-27T02:36:16.775 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.775 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.775 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 130.00 MB 2024-01-27T02:36:16.775 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.775 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.775 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 131.00 MB 2024-01-27T02:36:16.775 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.775 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.775 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 132.00 MB 2024-01-27T02:36:16.776 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.776 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.776 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 133.00 MB 2024-01-27T02:36:16.776 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.776 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.776 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 134.00 MB 2024-01-27T02:36:16.776 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.776 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.776 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 135.00 MB 2024-01-27T02:36:16.776 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.776 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.776 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 136.00 MB 2024-01-27T02:36:16.776 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.776 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.777 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 137.00 MB 2024-01-27T02:36:16.777 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.777 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.777 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 138.00 MB 2024-01-27T02:36:16.777 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.777 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.777 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 139.00 MB 2024-01-27T02:36:16.777 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.777 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.777 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 140.00 MB 2024-01-27T02:36:16.777 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.777 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.777 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 141.00 MB 2024-01-27T02:36:16.777 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.778 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.778 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 142.00 MB 2024-01-27T02:36:16.778 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.778 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.778 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 143.00 MB 2024-01-27T02:36:16.778 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.778 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.778 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 144.00 MB 2024-01-27T02:36:16.778 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.778 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.778 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 145.00 MB 2024-01-27T02:36:16.778 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.778 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.778 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 146.00 MB 2024-01-27T02:36:16.778 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.779 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.779 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 147.00 MB 2024-01-27T02:36:16.779 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.779 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.779 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 148.00 MB 2024-01-27T02:36:16.779 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.779 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.779 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 149.00 MB 2024-01-27T02:36:16.779 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.779 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.779 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 150.00 MB 2024-01-27T02:36:16.779 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.779 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.779 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 151.00 MB 2024-01-27T02:36:16.780 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.780 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.780 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 152.00 MB 2024-01-27T02:36:16.780 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.780 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.780 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 153.00 MB 2024-01-27T02:36:16.780 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.780 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.780 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 154.00 MB 2024-01-27T02:36:16.780 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.780 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.780 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 155.00 MB 2024-01-27T02:36:16.780 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.780 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.781 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 156.00 MB 2024-01-27T02:36:16.781 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.781 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.781 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 157.00 MB 2024-01-27T02:36:16.781 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.781 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.781 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 158.00 MB 2024-01-27T02:36:16.781 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.781 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.781 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 159.00 MB 2024-01-27T02:36:16.781 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.781 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.781 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 160.00 MB 2024-01-27T02:36:16.781 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:16.782 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:16.782 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 161.00 MB 2024-01-27T02:36:17.000 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.000 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.000 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 162.00 MB 2024-01-27T02:36:17.000 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.000 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.001 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 163.00 MB 2024-01-27T02:36:17.001 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.001 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.001 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 164.00 MB 2024-01-27T02:36:17.001 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.001 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.001 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 165.00 MB 2024-01-27T02:36:17.001 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.001 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.001 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 166.00 MB 2024-01-27T02:36:17.001 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.001 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.001 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 167.00 MB 2024-01-27T02:36:17.001 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.002 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.002 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 168.00 MB 2024-01-27T02:36:17.002 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.002 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.002 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 169.00 MB 2024-01-27T02:36:17.002 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.002 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.002 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 170.00 MB 2024-01-27T02:36:17.002 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.002 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.002 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 171.00 MB 2024-01-27T02:36:17.002 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.002 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.003 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 172.00 MB 2024-01-27T02:36:17.003 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.003 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.003 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 173.00 MB 2024-01-27T02:36:17.003 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.003 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.003 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 174.00 MB 2024-01-27T02:36:17.003 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.003 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.003 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 175.00 MB 2024-01-27T02:36:17.003 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.003 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.003 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 176.00 MB 2024-01-27T02:36:17.004 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.004 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.004 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 177.00 MB 2024-01-27T02:36:17.004 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.004 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.004 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 178.00 MB 2024-01-27T02:36:17.004 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.004 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.004 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 179.00 MB 2024-01-27T02:36:17.004 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.004 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.004 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 180.00 MB 2024-01-27T02:36:17.004 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.005 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.005 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 181.00 MB 2024-01-27T02:36:17.005 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.005 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.005 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 182.00 MB 2024-01-27T02:36:17.005 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.005 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.005 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 183.00 MB 2024-01-27T02:36:17.005 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.005 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.005 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 184.00 MB 2024-01-27T02:36:17.005 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.005 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.005 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 185.00 MB 2024-01-27T02:36:17.006 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.006 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.006 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 186.00 MB 2024-01-27T02:36:17.006 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.006 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.006 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 187.00 MB 2024-01-27T02:36:17.006 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.006 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.006 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 188.00 MB 2024-01-27T02:36:17.006 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.006 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.006 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 189.00 MB 2024-01-27T02:36:17.006 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.006 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.006 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 190.00 MB 2024-01-27T02:36:17.006 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.007 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.007 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 191.00 MB 2024-01-27T02:36:17.007 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.007 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.007 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 192.00 MB 2024-01-27T02:36:17.007 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.007 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.007 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 193.00 MB 2024-01-27T02:36:17.007 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.007 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.007 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 194.00 MB 2024-01-27T02:36:17.007 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.007 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.007 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 195.00 MB 2024-01-27T02:36:17.008 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.008 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.008 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 196.00 MB 2024-01-27T02:36:17.008 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.008 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.008 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 197.00 MB 2024-01-27T02:36:17.008 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.008 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.008 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 198.00 MB 2024-01-27T02:36:17.008 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.008 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.008 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 199.00 MB 2024-01-27T02:36:17.008 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.008 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.009 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 200.00 MB 2024-01-27T02:36:17.009 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.009 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.009 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 201.00 MB 2024-01-27T02:36:17.009 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.009 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.009 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 202.00 MB 2024-01-27T02:36:17.009 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.009 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.009 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 203.00 MB 2024-01-27T02:36:17.009 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.009 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.009 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 204.00 MB 2024-01-27T02:36:17.009 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.010 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.010 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 205.00 MB 2024-01-27T02:36:17.010 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.010 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.010 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 206.00 MB 2024-01-27T02:36:17.010 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.010 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.010 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 207.00 MB 2024-01-27T02:36:17.010 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.010 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.010 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 208.00 MB 2024-01-27T02:36:17.010 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.010 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.010 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 209.00 MB 2024-01-27T02:36:17.011 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.011 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.011 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 210.00 MB 2024-01-27T02:36:17.011 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.011 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.011 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 211.00 MB 2024-01-27T02:36:17.011 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.011 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.011 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 212.00 MB 2024-01-27T02:36:17.011 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.011 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.011 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 213.00 MB 2024-01-27T02:36:17.011 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.012 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.012 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 214.00 MB 2024-01-27T02:36:17.012 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.012 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.012 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 215.00 MB 2024-01-27T02:36:17.012 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.012 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.012 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 216.00 MB 2024-01-27T02:36:17.012 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.012 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.012 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 217.00 MB 2024-01-27T02:36:17.012 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.012 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.012 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 218.00 MB 2024-01-27T02:36:17.013 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.013 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.013 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 219.00 MB 2024-01-27T02:36:17.013 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.013 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.013 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 220.00 MB 2024-01-27T02:36:17.013 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.013 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.013 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 221.00 MB 2024-01-27T02:36:17.013 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.013 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.013 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 222.00 MB 2024-01-27T02:36:17.013 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.013 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.014 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 223.00 MB 2024-01-27T02:36:17.014 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.014 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.014 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 224.00 MB 2024-01-27T02:36:17.014 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.014 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.014 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 225.00 MB 2024-01-27T02:36:17.014 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.014 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.014 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 226.00 MB 2024-01-27T02:36:17.014 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.014 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.014 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 227.00 MB 2024-01-27T02:36:17.015 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.015 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.015 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 228.00 MB 2024-01-27T02:36:17.015 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.015 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.015 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 229.00 MB 2024-01-27T02:36:17.015 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.015 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.015 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 230.00 MB 2024-01-27T02:36:17.015 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.015 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.015 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 231.00 MB 2024-01-27T02:36:17.015 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.016 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.016 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 232.00 MB 2024-01-27T02:36:17.016 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.016 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.016 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 233.00 MB 2024-01-27T02:36:17.016 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.016 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.016 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 234.00 MB 2024-01-27T02:36:17.016 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.017 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.017 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 235.00 MB 2024-01-27T02:36:17.017 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.017 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.017 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 236.00 MB 2024-01-27T02:36:17.017 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.017 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.017 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 237.00 MB 2024-01-27T02:36:17.017 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.017 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.017 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 238.00 MB 2024-01-27T02:36:17.018 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.018 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.018 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 239.00 MB 2024-01-27T02:36:17.018 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.018 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.018 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 240.00 MB 2024-01-27T02:36:17.018 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.018 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.018 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 241.00 MB 2024-01-27T02:36:17.228 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.228 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.228 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 242.00 MB 2024-01-27T02:36:17.228 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.228 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.228 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 243.00 MB 2024-01-27T02:36:17.228 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.229 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.229 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 244.00 MB 2024-01-27T02:36:17.229 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.229 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.229 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 245.00 MB 2024-01-27T02:36:17.229 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.229 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.229 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 246.00 MB 2024-01-27T02:36:17.229 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.229 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.229 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 247.00 MB 2024-01-27T02:36:17.229 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.229 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.229 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 248.00 MB 2024-01-27T02:36:17.230 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.230 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.230 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 249.00 MB 2024-01-27T02:36:17.230 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.230 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.230 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 250.00 MB 2024-01-27T02:36:17.230 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.230 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.230 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 251.00 MB 2024-01-27T02:36:17.230 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.230 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.230 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 252.00 MB 2024-01-27T02:36:17.230 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.230 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.230 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 253.00 MB 2024-01-27T02:36:17.231 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.231 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.231 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 254.00 MB 2024-01-27T02:36:17.231 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.231 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.231 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 255.00 MB 2024-01-27T02:36:17.231 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.231 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.231 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 256.00 MB 2024-01-27T02:36:17.231 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.231 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.231 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 257.00 MB 2024-01-27T02:36:17.231 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.231 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.232 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 258.00 MB 2024-01-27T02:36:17.232 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.232 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.232 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 259.00 MB 2024-01-27T02:36:17.232 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.232 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.232 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 260.00 MB 2024-01-27T02:36:17.232 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.232 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.232 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 261.00 MB 2024-01-27T02:36:17.232 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.232 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.232 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 262.00 MB 2024-01-27T02:36:17.232 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.233 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.233 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 263.00 MB 2024-01-27T02:36:17.233 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.233 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.233 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 264.00 MB 2024-01-27T02:36:17.233 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.233 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.233 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 265.00 MB 2024-01-27T02:36:17.233 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.233 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.233 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 266.00 MB 2024-01-27T02:36:17.233 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.233 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.233 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 267.00 MB 2024-01-27T02:36:17.233 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.234 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.234 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 268.00 MB 2024-01-27T02:36:17.234 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.234 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.234 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 269.00 MB 2024-01-27T02:36:17.234 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.234 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.234 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 270.00 MB 2024-01-27T02:36:17.234 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.234 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.234 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 271.00 MB 2024-01-27T02:36:17.234 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.234 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.235 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 272.00 MB 2024-01-27T02:36:17.235 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.235 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.235 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 273.00 MB 2024-01-27T02:36:17.235 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.235 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.235 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 274.00 MB 2024-01-27T02:36:17.235 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.235 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.235 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 275.00 MB 2024-01-27T02:36:17.235 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.235 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.235 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 276.00 MB 2024-01-27T02:36:17.236 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.236 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.236 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 277.00 MB 2024-01-27T02:36:17.236 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.236 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.236 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 278.00 MB 2024-01-27T02:36:17.236 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.236 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.236 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 279.00 MB 2024-01-27T02:36:17.236 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 280.00 MB 2024-01-27T02:36:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 281.00 MB 2024-01-27T02:36:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 282.00 MB 2024-01-27T02:36:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 283.00 MB 2024-01-27T02:36:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 284.00 MB 2024-01-27T02:36:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 285.00 MB 2024-01-27T02:36:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 286.00 MB 2024-01-27T02:36:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 287.00 MB 2024-01-27T02:36:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 288.00 MB 2024-01-27T02:36:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 289.00 MB 2024-01-27T02:36:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 290.00 MB 2024-01-27T02:36:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 291.00 MB 2024-01-27T02:36:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 292.00 MB 2024-01-27T02:36:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 293.00 MB 2024-01-27T02:36:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 294.00 MB 2024-01-27T02:36:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.240 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.240 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 295.00 MB 2024-01-27T02:36:17.240 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.240 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.240 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 296.00 MB 2024-01-27T02:36:17.240 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.240 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.240 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 297.00 MB 2024-01-27T02:36:17.240 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.240 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.240 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 298.00 MB 2024-01-27T02:36:17.240 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.240 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.240 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 299.00 MB 2024-01-27T02:36:17.241 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.241 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.241 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 300.00 MB 2024-01-27T02:36:17.241 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.241 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.241 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 301.00 MB 2024-01-27T02:36:17.241 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.241 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.241 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 302.00 MB 2024-01-27T02:36:17.241 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.241 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.241 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 303.00 MB 2024-01-27T02:36:17.241 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.241 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.242 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 304.00 MB 2024-01-27T02:36:17.242 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.242 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.242 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 305.00 MB 2024-01-27T02:36:17.242 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.242 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.242 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 306.00 MB 2024-01-27T02:36:17.242 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.242 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.242 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 307.00 MB 2024-01-27T02:36:17.242 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.242 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.242 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 308.00 MB 2024-01-27T02:36:17.243 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.243 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.243 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 309.00 MB 2024-01-27T02:36:17.243 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.243 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.243 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 310.00 MB 2024-01-27T02:36:17.243 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.243 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.243 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 311.00 MB 2024-01-27T02:36:17.243 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.243 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.243 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 312.00 MB 2024-01-27T02:36:17.243 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.243 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.244 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 313.00 MB 2024-01-27T02:36:17.244 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.244 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.244 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 314.00 MB 2024-01-27T02:36:17.244 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.244 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.244 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 315.00 MB 2024-01-27T02:36:17.244 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.244 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.244 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 316.00 MB 2024-01-27T02:36:17.244 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.244 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.244 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 317.00 MB 2024-01-27T02:36:17.245 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.245 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.245 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 318.00 MB 2024-01-27T02:36:17.245 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.245 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.245 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 319.00 MB 2024-01-27T02:36:17.245 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.245 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.245 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 320.00 MB 2024-01-27T02:36:17.245 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.245 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.245 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 321.00 MB 2024-01-27T02:36:17.245 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.245 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.480 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 322.00 MB 2024-01-27T02:36:17.480 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.480 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.480 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 323.00 MB 2024-01-27T02:36:17.480 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 324.00 MB 2024-01-27T02:36:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 325.00 MB 2024-01-27T02:36:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 326.00 MB 2024-01-27T02:36:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 327.00 MB 2024-01-27T02:36:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 328.00 MB 2024-01-27T02:36:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 329.00 MB 2024-01-27T02:36:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 330.00 MB 2024-01-27T02:36:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 331.00 MB 2024-01-27T02:36:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 332.00 MB 2024-01-27T02:36:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 333.00 MB 2024-01-27T02:36:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 334.00 MB 2024-01-27T02:36:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 335.00 MB 2024-01-27T02:36:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 336.00 MB 2024-01-27T02:36:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 337.00 MB 2024-01-27T02:36:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 338.00 MB 2024-01-27T02:36:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 339.00 MB 2024-01-27T02:36:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 340.00 MB 2024-01-27T02:36:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 341.00 MB 2024-01-27T02:36:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 342.00 MB 2024-01-27T02:36:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 343.00 MB 2024-01-27T02:36:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 344.00 MB 2024-01-27T02:36:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 345.00 MB 2024-01-27T02:36:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 346.00 MB 2024-01-27T02:36:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 347.00 MB 2024-01-27T02:36:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 348.00 MB 2024-01-27T02:36:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 349.00 MB 2024-01-27T02:36:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 350.00 MB 2024-01-27T02:36:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 351.00 MB 2024-01-27T02:36:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 352.00 MB 2024-01-27T02:36:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 353.00 MB 2024-01-27T02:36:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 354.00 MB 2024-01-27T02:36:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 355.00 MB 2024-01-27T02:36:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 356.00 MB 2024-01-27T02:36:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 357.00 MB 2024-01-27T02:36:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 358.00 MB 2024-01-27T02:36:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 359.00 MB 2024-01-27T02:36:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 360.00 MB 2024-01-27T02:36:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 361.00 MB 2024-01-27T02:36:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 362.00 MB 2024-01-27T02:36:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 363.00 MB 2024-01-27T02:36:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 364.00 MB 2024-01-27T02:36:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 365.00 MB 2024-01-27T02:36:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 366.00 MB 2024-01-27T02:36:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 367.00 MB 2024-01-27T02:36:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 368.00 MB 2024-01-27T02:36:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 369.00 MB 2024-01-27T02:36:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 370.00 MB 2024-01-27T02:36:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 371.00 MB 2024-01-27T02:36:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 372.00 MB 2024-01-27T02:36:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 373.00 MB 2024-01-27T02:36:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 374.00 MB 2024-01-27T02:36:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 375.00 MB 2024-01-27T02:36:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 376.00 MB 2024-01-27T02:36:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 377.00 MB 2024-01-27T02:36:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 378.00 MB 2024-01-27T02:36:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 379.00 MB 2024-01-27T02:36:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 380.00 MB 2024-01-27T02:36:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 381.00 MB 2024-01-27T02:36:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 382.00 MB 2024-01-27T02:36:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 383.00 MB 2024-01-27T02:36:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 384.00 MB 2024-01-27T02:36:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 385.00 MB 2024-01-27T02:36:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 386.00 MB 2024-01-27T02:36:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 387.00 MB 2024-01-27T02:36:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 388.00 MB 2024-01-27T02:36:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 389.00 MB 2024-01-27T02:36:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 390.00 MB 2024-01-27T02:36:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 391.00 MB 2024-01-27T02:36:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 392.00 MB 2024-01-27T02:36:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 393.00 MB 2024-01-27T02:36:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 394.00 MB 2024-01-27T02:36:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 395.00 MB 2024-01-27T02:36:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 396.00 MB 2024-01-27T02:36:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 397.00 MB 2024-01-27T02:36:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 398.00 MB 2024-01-27T02:36:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 399.00 MB 2024-01-27T02:36:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 400.00 MB 2024-01-27T02:36:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 401.00 MB 2024-01-27T02:36:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 402.00 MB 2024-01-27T02:36:19.855 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-27T02:36:19.855+0000 7fb5c783f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2024-01-27T02:36:29.841 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-27T02:36:29.840+0000 7fb5c783f700 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2024-01-27T02:36:38.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:36:38.483 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:36:47.768 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.768 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.768 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 403.00 MB 2024-01-27T02:36:47.768 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.768 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.769 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 404.00 MB 2024-01-27T02:36:47.769 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.769 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.769 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 405.00 MB 2024-01-27T02:36:47.769 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.769 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.769 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 406.00 MB 2024-01-27T02:36:47.769 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.769 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.769 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 407.00 MB 2024-01-27T02:36:47.769 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.769 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.769 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 408.00 MB 2024-01-27T02:36:47.769 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.769 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.770 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 409.00 MB 2024-01-27T02:36:47.770 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.770 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.770 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 410.00 MB 2024-01-27T02:36:47.770 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.770 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.770 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 411.00 MB 2024-01-27T02:36:47.770 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.770 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.770 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 412.00 MB 2024-01-27T02:36:47.770 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.770 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.770 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 413.00 MB 2024-01-27T02:36:47.770 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.770 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.771 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 414.00 MB 2024-01-27T02:36:47.771 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.771 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.771 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 415.00 MB 2024-01-27T02:36:47.771 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.771 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.771 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 416.00 MB 2024-01-27T02:36:47.771 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.771 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.771 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 417.00 MB 2024-01-27T02:36:47.771 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.771 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.771 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 418.00 MB 2024-01-27T02:36:47.771 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.771 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.772 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 419.00 MB 2024-01-27T02:36:47.772 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.772 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.772 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 420.00 MB 2024-01-27T02:36:47.772 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.772 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.772 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 421.00 MB 2024-01-27T02:36:47.772 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.772 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.772 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 422.00 MB 2024-01-27T02:36:47.772 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.772 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.772 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 423.00 MB 2024-01-27T02:36:47.772 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.773 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.773 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 424.00 MB 2024-01-27T02:36:47.773 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.773 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.773 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 425.00 MB 2024-01-27T02:36:47.773 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.773 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.773 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 426.00 MB 2024-01-27T02:36:47.773 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.773 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.773 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 427.00 MB 2024-01-27T02:36:47.773 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.773 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.773 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 428.00 MB 2024-01-27T02:36:47.773 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.774 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.774 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 429.00 MB 2024-01-27T02:36:47.774 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.774 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.774 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 430.00 MB 2024-01-27T02:36:47.774 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.774 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.774 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 431.00 MB 2024-01-27T02:36:47.774 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.774 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.774 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 432.00 MB 2024-01-27T02:36:47.774 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.774 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.774 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 433.00 MB 2024-01-27T02:36:47.774 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.775 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.775 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 434.00 MB 2024-01-27T02:36:47.775 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.775 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.775 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 435.00 MB 2024-01-27T02:36:47.775 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.775 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.775 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 436.00 MB 2024-01-27T02:36:47.775 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.775 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.775 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 437.00 MB 2024-01-27T02:36:47.775 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.775 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.775 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 438.00 MB 2024-01-27T02:36:47.775 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.776 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.776 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 439.00 MB 2024-01-27T02:36:47.776 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.776 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.776 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 440.00 MB 2024-01-27T02:36:47.776 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.776 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.776 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 441.00 MB 2024-01-27T02:36:47.776 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.776 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.776 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 442.00 MB 2024-01-27T02:36:47.776 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.776 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.776 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 443.00 MB 2024-01-27T02:36:47.777 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.777 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.777 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 444.00 MB 2024-01-27T02:36:47.777 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.777 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.777 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 445.00 MB 2024-01-27T02:36:47.777 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.777 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.777 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 446.00 MB 2024-01-27T02:36:47.777 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.777 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.778 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 447.00 MB 2024-01-27T02:36:47.778 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.778 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.778 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 448.00 MB 2024-01-27T02:36:47.778 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.778 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.778 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 449.00 MB 2024-01-27T02:36:47.778 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.778 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.778 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 450.00 MB 2024-01-27T02:36:47.778 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.778 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.779 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 451.00 MB 2024-01-27T02:36:47.779 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.779 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.779 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 452.00 MB 2024-01-27T02:36:47.779 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.779 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.779 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 453.00 MB 2024-01-27T02:36:47.779 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.779 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.779 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 454.00 MB 2024-01-27T02:36:47.779 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.779 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.779 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 455.00 MB 2024-01-27T02:36:47.779 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.779 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.780 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 456.00 MB 2024-01-27T02:36:47.780 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.780 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.780 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 457.00 MB 2024-01-27T02:36:47.780 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.780 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.780 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 458.00 MB 2024-01-27T02:36:47.780 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.780 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.780 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 459.00 MB 2024-01-27T02:36:47.780 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.780 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.780 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 460.00 MB 2024-01-27T02:36:47.780 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.781 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.781 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 461.00 MB 2024-01-27T02:36:47.781 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.781 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.781 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 462.00 MB 2024-01-27T02:36:47.781 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.781 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.781 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 463.00 MB 2024-01-27T02:36:47.781 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.781 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.781 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 464.00 MB 2024-01-27T02:36:47.781 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.781 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.781 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 465.00 MB 2024-01-27T02:36:47.781 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.782 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.782 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 466.00 MB 2024-01-27T02:36:47.782 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.782 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.782 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 467.00 MB 2024-01-27T02:36:47.782 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.782 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.782 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 468.00 MB 2024-01-27T02:36:47.782 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.782 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.782 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 469.00 MB 2024-01-27T02:36:47.782 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.782 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.782 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 470.00 MB 2024-01-27T02:36:47.782 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.783 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.783 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 471.00 MB 2024-01-27T02:36:47.783 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.783 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.783 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 472.00 MB 2024-01-27T02:36:47.783 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.783 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.783 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 473.00 MB 2024-01-27T02:36:47.783 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.783 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.783 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 474.00 MB 2024-01-27T02:36:47.783 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.783 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.783 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 475.00 MB 2024-01-27T02:36:47.783 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.784 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.784 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 476.00 MB 2024-01-27T02:36:47.784 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.784 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.784 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 477.00 MB 2024-01-27T02:36:47.784 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.784 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.784 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 478.00 MB 2024-01-27T02:36:47.784 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.784 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.785 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 479.00 MB 2024-01-27T02:36:47.785 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-27T02:36:47.785 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:36:47.785 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:36:47.785 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 480.00 MB 2024-01-27T02:36:47.785 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-27T02:36:47.785 INFO:teuthology.orchestra.run.smithi007.stdout:Reached fullness after 480.00 MB 2024-01-27T02:36:47.785 INFO:teuthology.orchestra.run.smithi007.stdout:calling close 2024-01-27T02:36:47.785 INFO:teuthology.orchestra.run.smithi007.stdout:close() did not raise error 2024-01-27T02:36:47.833 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-27T02:36:47.833 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:36:47.897 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:36:47.943 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-27T02:36:47.943 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-27T02:36:47.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:36:47.943 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-27T02:36:47.998 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:36:47.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:36:47.999 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-27T02:36:48.020 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-27T02:36:48.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:36:48.021 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-01-27T02:36:48.070 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:36:48.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:36:48.070 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-27T02:36:48.088 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:36:48.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.277+0000 7f02958d5700 1 Processor -- start 2024-01-27T02:36:48.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.277+0000 7f02958d5700 1 -- start start 2024-01-27T02:36:48.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.277+0000 7f02958d5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f029015a390 0x7f029015e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:48.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.277+0000 7f02958d5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02900cc830 con 0x7f02900d0da0 2024-01-27T02:36:48.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.277+0000 7f02958d5700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02900b35e0 con 0x7f0290159fc0 2024-01-27T02:36:48.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.277+0000 7f02958d5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02900cbf00 con 0x7f029015a390 2024-01-27T02:36:48.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.278+0000 7f028f7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f02900d0da0 0x7f02900af8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54892/0 (socket says 172.21.15.7:54892) 2024-01-27T02:36:48.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.278+0000 7f028f7fe700 1 -- 172.21.15.7:0/3991625116 learned_addr learned my addr 172.21.15.7:0/3991625116 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.278+0000 7f028e7fc700 1 --2- 172.21.15.7:0/3991625116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f029015a390 0x7f029015e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.278+0000 7f028dffb700 1 -- 172.21.15.7:0/3991625116 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4241714119 0 0) 0x7f02900cc830 con 0x7f02900d0da0 2024-01-27T02:36:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.278+0000 7f028dffb700 1 -- 172.21.15.7:0/3991625116 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0278003570 con 0x7f02900d0da0 2024-01-27T02:36:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.278+0000 7f028dffb700 1 -- 172.21.15.7:0/3991625116 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 254674726 0 0) 0x7f02900b35e0 con 0x7f0290159fc0 2024-01-27T02:36:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.278+0000 7f028dffb700 1 -- 172.21.15.7:0/3991625116 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02900cc830 con 0x7f0290159fc0 2024-01-27T02:36:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.278+0000 7f028e7fc700 1 -- 172.21.15.7:0/3991625116 >> v1:172.21.15.188:6789/0 conn(0x7f0290159fc0 legacy=0x7f02900b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.278+0000 7f028e7fc700 1 -- 172.21.15.7:0/3991625116 >> v1:172.21.15.7:6789/0 conn(0x7f02900d0da0 legacy=0x7f02900af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.278+0000 7f028dffb700 1 -- 172.21.15.7:0/3991625116 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4110676947 0 0) 0x7f0278003570 con 0x7f02900d0da0 2024-01-27T02:36:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.278+0000 7f028e7fc700 1 -- 172.21.15.7:0/3991625116 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02900cc1e0 con 0x7f029015a390 2024-01-27T02:36:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.278+0000 7f028e7fc700 1 --2- 172.21.15.7:0/3991625116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f029015a390 0x7f029015e8d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0284009910 tx=0x7f02840091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4b578cd9949e10c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.279+0000 7f028dffb700 1 -- 172.21.15.7:0/3991625116 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02880017f0 con 0x7f029015a390 2024-01-27T02:36:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.279+0000 7f028dffb700 1 -- 172.21.15.7:0/3991625116 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0284011420 con 0x7f029015a390 2024-01-27T02:36:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.279+0000 7f028dffb700 1 -- 172.21.15.7:0/3991625116 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02840117c0 con 0x7f029015a390 2024-01-27T02:36:48.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.279+0000 7f02958d5700 1 -- 172.21.15.7:0/3991625116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f029015a390 msgr2=0x7f029015e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:48.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.279+0000 7f02958d5700 1 --2- 172.21.15.7:0/3991625116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f029015a390 0x7f029015e8d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0284009910 tx=0x7f02840091a0 comp rx=0 tx=0).stop 2024-01-27T02:36:48.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.279+0000 7f02958d5700 1 -- 172.21.15.7:0/3991625116 shutdown_connections 2024-01-27T02:36:48.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.279+0000 7f02958d5700 1 --2- 172.21.15.7:0/3991625116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f029015a390 0x7f029015e8d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:48.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.279+0000 7f02958d5700 1 -- 172.21.15.7:0/3991625116 >> 172.21.15.7:0/3991625116 conn(0x7f02900679b0 msgr2=0x7f0290067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:48.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.279+0000 7f02958d5700 1 -- 172.21.15.7:0/3991625116 shutdown_connections 2024-01-27T02:36:48.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.279+0000 7f02958d5700 1 -- 172.21.15.7:0/3991625116 wait complete. 2024-01-27T02:36:48.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.281+0000 7f02958d5700 1 Processor -- start 2024-01-27T02:36:48.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.281+0000 7f02958d5700 1 -- start start 2024-01-27T02:36:48.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.282+0000 7f02958d5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02900d0da0 0x7f029015b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:48.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.282+0000 7f02958d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0290159fc0 0x7f029015c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:48.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.282+0000 7f02958d5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f029015a390 0x7f0290149f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:48.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.282+0000 7f02958d5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029015c0e0 con 0x7f0290159fc0 2024-01-27T02:36:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.282+0000 7f02958d5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02900c4bf0 con 0x7f029015a390 2024-01-27T02:36:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.282+0000 7f02958d5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f029015bb20 con 0x7f02900d0da0 2024-01-27T02:36:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.282+0000 7f028e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0290159fc0 0x7f029015c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.282+0000 7f028e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0290159fc0 0x7f029015c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42902/0 (socket says 172.21.15.7:42902) 2024-01-27T02:36:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.282+0000 7f028e7fc700 1 -- 172.21.15.7:0/1267339443 learned_addr learned my addr 172.21.15.7:0/1267339443 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.282+0000 7f028f7fe700 1 --2- 172.21.15.7:0/1267339443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f029015a390 0x7f0290149f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.282+0000 7f028effd700 1 --2- 172.21.15.7:0/1267339443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02900d0da0 0x7f029015b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.282+0000 7f028e7fc700 1 -- 172.21.15.7:0/1267339443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02900d0da0 msgr2=0x7f029015b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.282+0000 7f028e7fc700 1 --2- 172.21.15.7:0/1267339443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02900d0da0 0x7f029015b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.282+0000 7f028e7fc700 1 -- 172.21.15.7:0/1267339443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f029015a390 msgr2=0x7f0290149f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.282+0000 7f028e7fc700 1 --2- 172.21.15.7:0/1267339443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f029015a390 0x7f0290149f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.282+0000 7f028e7fc700 1 -- 172.21.15.7:0/1267339443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0284009420 con 0x7f0290159fc0 2024-01-27T02:36:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.282+0000 7f028f7fe700 1 --2- 172.21.15.7:0/1267339443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f029015a390 0x7f0290149f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:36:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.282+0000 7f028e7fc700 1 --2- 172.21.15.7:0/1267339443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0290159fc0 0x7f029015c790 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f02840030c0 tx=0x7f0284016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.283+0000 7f0276ffd700 1 -- 172.21.15.7:0/1267339443 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028401cc50 con 0x7f0290159fc0 2024-01-27T02:36:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.283+0000 7f0276ffd700 1 -- 172.21.15.7:0/1267339443 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028401cdb0 con 0x7f0290159fc0 2024-01-27T02:36:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.283+0000 7f02958d5700 1 -- 172.21.15.7:0/1267339443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029015be00 con 0x7f0290159fc0 2024-01-27T02:36:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.283+0000 7f0276ffd700 1 -- 172.21.15.7:0/1267339443 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028401a6a0 con 0x7f0290159fc0 2024-01-27T02:36:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.283+0000 7f02958d5700 1 -- 172.21.15.7:0/1267339443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029015b6a0 con 0x7f0290159fc0 2024-01-27T02:36:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.284+0000 7f02958d5700 1 -- 172.21.15.7:0/1267339443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f029006dfd0 con 0x7f0290159fc0 2024-01-27T02:36:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.284+0000 7f0276ffd700 1 -- 172.21.15.7:0/1267339443 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f028401aad0 con 0x7f0290159fc0 2024-01-27T02:36:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.284+0000 7f0276ffd700 1 --2- 172.21.15.7:0/1267339443 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f027c066f30 0x7f027c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.285+0000 7f0276ffd700 1 -- 172.21.15.7:0/1267339443 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0284086120 con 0x7f0290159fc0 2024-01-27T02:36:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.285+0000 7f0276ffd700 1 -- 172.21.15.7:0/1267339443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=85}) v3 -- 0x7f027c06ea50 con 0x7f0290159fc0 2024-01-27T02:36:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.285+0000 7f028effd700 1 --2- 172.21.15.7:0/1267339443 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f027c066f30 0x7f027c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.285+0000 7f028effd700 1 --2- 172.21.15.7:0/1267339443 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f027c066f30 0x7f027c0693e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0280003870 tx=0x7f0280009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.289+0000 7f0276ffd700 1 -- 172.21.15.7:0/1267339443 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0284053390 con 0x7f0290159fc0 2024-01-27T02:36:48.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.539+0000 7f02958d5700 1 -- 172.21.15.7:0/1267339443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02900c4d80 con 0x7f0290159fc0 2024-01-27T02:36:48.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.540+0000 7f0276ffd700 1 -- 172.21.15.7:0/1267339443 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4590 (secure 0 0 0) 0x7f02840569b0 con 0x7f0290159fc0 2024-01-27T02:36:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:36:48.542 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5839,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/100533817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":100533817},{"type":"v1","addr":"172.21.15.7:6834","nonce":100533817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5842,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/223425097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":223425097},{"type":"v1","addr":"172.21.15.7:6835","nonce":223425097}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5848,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/950040140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":950040140},{"type":"v1","addr":"172.21.15.188:6837","nonce":950040140}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:36:01.502818+0000","modified":"2024-01-27T02:36:05.109352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5946":{"gid":5946,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6836/2269430612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2269430612},{"type":"v1","addr":"172.21.15.188:6836","nonce":2269430612}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2024-01-27T02:36:48.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.542+0000 7f02958d5700 1 -- 172.21.15.7:0/1267339443 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f027c066f30 msgr2=0x7f027c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:48.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.542+0000 7f02958d5700 1 --2- 172.21.15.7:0/1267339443 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f027c066f30 0x7f027c0693e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0280003870 tx=0x7f0280009040 comp rx=0 tx=0).stop 2024-01-27T02:36:48.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.542+0000 7f02958d5700 1 -- 172.21.15.7:0/1267339443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0290159fc0 msgr2=0x7f029015c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:48.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.542+0000 7f02958d5700 1 --2- 172.21.15.7:0/1267339443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0290159fc0 0x7f029015c790 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f02840030c0 tx=0x7f0284016000 comp rx=0 tx=0).stop 2024-01-27T02:36:48.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.542+0000 7f02958d5700 1 -- 172.21.15.7:0/1267339443 shutdown_connections 2024-01-27T02:36:48.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.542+0000 7f02958d5700 1 --2- 172.21.15.7:0/1267339443 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f027c066f30 0x7f027c0693e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:48.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.542+0000 7f02958d5700 1 --2- 172.21.15.7:0/1267339443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f029015a390 0x7f0290149f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:48.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.542+0000 7f02958d5700 1 --2- 172.21.15.7:0/1267339443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02900d0da0 0x7f029015b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:48.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.542+0000 7f02958d5700 1 --2- 172.21.15.7:0/1267339443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0290159fc0 0x7f029015c790 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:48.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.542+0000 7f02958d5700 1 -- 172.21.15.7:0/1267339443 >> 172.21.15.7:0/1267339443 conn(0x7f02900679b0 msgr2=0x7f0290159470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:48.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.542+0000 7f02958d5700 1 -- 172.21.15.7:0/1267339443 shutdown_connections 2024-01-27T02:36:48.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.542+0000 7f02958d5700 1 -- 172.21.15.7:0/1267339443 wait complete. 2024-01-27T02:36:48.546 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 42 2024-01-27T02:36:48.558 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:36:48.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.688+0000 7f3e67afb700 1 Processor -- start 2024-01-27T02:36:48.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.688+0000 7f3e67afb700 1 -- start start 2024-01-27T02:36:48.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.689+0000 7f3e67afb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e60147d50 0x7f3e600c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:48.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.689+0000 7f3e67afb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3e60148120 0x7f3e600c8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:48.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.689+0000 7f3e67afb700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3e600c89c0 0x7f3e60063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:48.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.689+0000 7f3e67afb700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e600c8e60 con 0x7f3e60148120 2024-01-27T02:36:48.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.689+0000 7f3e67afb700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e601630c0 con 0x7f3e600c89c0 2024-01-27T02:36:48.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.689+0000 7f3e67afb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e600c3970 con 0x7f3e60147d50 2024-01-27T02:36:48.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.689+0000 7f3e65096700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3e60148120 0x7f3e600c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:48.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.689+0000 7f3e65096700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3e60148120 0x7f3e600c8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42914/0 (socket says 172.21.15.7:42914) 2024-01-27T02:36:48.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.689+0000 7f3e65096700 1 -- 172.21.15.7:0/439361312 learned_addr learned my addr 172.21.15.7:0/439361312 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:48.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.689+0000 7f3e66098700 1 --2- 172.21.15.7:0/439361312 >> v2:172.21.15.188:3300/0 conn(0x7f3e600c89c0 0x7f3e60063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:48.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.689+0000 7f3e65897700 1 --2- 172.21.15.7:0/439361312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e60147d50 0x7f3e600c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:48.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.690+0000 7f3e65096700 1 -- 172.21.15.7:0/439361312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e60147d50 msgr2=0x7f3e600c3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:48.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.690+0000 7f3e65096700 1 --2- 172.21.15.7:0/439361312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e60147d50 0x7f3e600c3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:48.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.690+0000 7f3e65096700 1 -- 172.21.15.7:0/439361312 >> v2:172.21.15.188:3300/0 conn(0x7f3e600c89c0 msgr2=0x7f3e60063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.690+0000 7f3e65096700 1 --2- 172.21.15.7:0/439361312 >> v2:172.21.15.188:3300/0 conn(0x7f3e600c89c0 0x7f3e60063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.690+0000 7f3e65096700 1 -- 172.21.15.7:0/439361312 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e600c3040 con 0x7f3e60148120 2024-01-27T02:36:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.690+0000 7f3e65096700 1 --2- 172.21.15.7:0/439361312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e60148120 0x7f3e600c8480 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3e600c4c70 tx=0x7f3e50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5eacc5e3c794a7 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.690+0000 7f3e64895700 1 -- 172.21.15.7:0/439361312 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5000b640 con 0x7f3e60148120 2024-01-27T02:36:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.690+0000 7f3e64895700 1 -- 172.21.15.7:0/439361312 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5000b7a0 con 0x7f3e60148120 2024-01-27T02:36:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.690+0000 7f3e64895700 1 -- 172.21.15.7:0/439361312 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5000bb60 con 0x7f3e60148120 2024-01-27T02:36:48.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.691+0000 7f3e67afb700 1 -- 172.21.15.7:0/439361312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e60148120 msgr2=0x7f3e600c8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:48.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.691+0000 7f3e67afb700 1 --2- 172.21.15.7:0/439361312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e60148120 0x7f3e600c8480 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3e600c4c70 tx=0x7f3e50009250 comp rx=0 tx=0).stop 2024-01-27T02:36:48.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.691+0000 7f3e67afb700 1 -- 172.21.15.7:0/439361312 shutdown_connections 2024-01-27T02:36:48.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.691+0000 7f3e67afb700 1 --2- 172.21.15.7:0/439361312 >> v2:172.21.15.188:3300/0 conn(0x7f3e600c89c0 0x7f3e60063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:48.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.691+0000 7f3e67afb700 1 --2- 172.21.15.7:0/439361312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e60147d50 0x7f3e600c3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:48.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.691+0000 7f3e67afb700 1 --2- 172.21.15.7:0/439361312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e60148120 0x7f3e600c8480 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:48.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.691+0000 7f3e67afb700 1 -- 172.21.15.7:0/439361312 >> 172.21.15.7:0/439361312 conn(0x7f3e600cdc30 msgr2=0x7f3e600ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:48.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.691+0000 7f3e67afb700 1 -- 172.21.15.7:0/439361312 shutdown_connections 2024-01-27T02:36:48.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.691+0000 7f3e67afb700 1 -- 172.21.15.7:0/439361312 wait complete. 2024-01-27T02:36:48.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.692+0000 7f3e67afb700 1 Processor -- start 2024-01-27T02:36:48.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.692+0000 7f3e67afb700 1 -- start start 2024-01-27T02:36:48.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.693+0000 7f3e67afb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e600c89c0 0x7f3e601446b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:48.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.693+0000 7f3e67afb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e60147d50 0x7f3e60145ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.693+0000 7f3e67afb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e60148120 0x7f3e601461e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.693+0000 7f3e67afb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e600c7aa0 con 0x7f3e60147d50 2024-01-27T02:36:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.693+0000 7f3e67afb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e60144bf0 con 0x7f3e60148120 2024-01-27T02:36:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.693+0000 7f3e67afb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e60171e30 con 0x7f3e600c89c0 2024-01-27T02:36:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.693+0000 7f3e65096700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e60147d50 0x7f3e60145ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.693+0000 7f3e65096700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e60147d50 0x7f3e60145ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42918/0 (socket says 172.21.15.7:42918) 2024-01-27T02:36:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.693+0000 7f3e65096700 1 -- 172.21.15.7:0/338742569 learned_addr learned my addr 172.21.15.7:0/338742569 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.693+0000 7f3e65897700 1 --2- 172.21.15.7:0/338742569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e600c89c0 0x7f3e601446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.693+0000 7f3e66098700 1 --2- 172.21.15.7:0/338742569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e60148120 0x7f3e601461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.693+0000 7f3e65096700 1 -- 172.21.15.7:0/338742569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e600c89c0 msgr2=0x7f3e601446b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.693+0000 7f3e65096700 1 --2- 172.21.15.7:0/338742569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e600c89c0 0x7f3e601446b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.693+0000 7f3e65096700 1 -- 172.21.15.7:0/338742569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e60148120 msgr2=0x7f3e601461e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:48.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.693+0000 7f3e65096700 1 --2- 172.21.15.7:0/338742569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e60148120 0x7f3e601461e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:48.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.693+0000 7f3e65096700 1 -- 172.21.15.7:0/338742569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e500094f0 con 0x7f3e60147d50 2024-01-27T02:36:48.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.693+0000 7f3e65096700 1 --2- 172.21.15.7:0/338742569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e60147d50 0x7f3e60145ca0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3e6006dcc0 tx=0x7f3e50017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:48.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.694+0000 7f3e55ffb700 1 -- 172.21.15.7:0/338742569 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e50000d10 con 0x7f3e60147d50 2024-01-27T02:36:48.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.694+0000 7f3e55ffb700 1 -- 172.21.15.7:0/338742569 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5000bb20 con 0x7f3e60147d50 2024-01-27T02:36:48.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.694+0000 7f3e67afb700 1 -- 172.21.15.7:0/338742569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e60172110 con 0x7f3e60147d50 2024-01-27T02:36:48.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.694+0000 7f3e55ffb700 1 -- 172.21.15.7:0/338742569 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e50011670 con 0x7f3e60147d50 2024-01-27T02:36:48.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.694+0000 7f3e67afb700 1 -- 172.21.15.7:0/338742569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e601726d0 con 0x7f3e60147d50 2024-01-27T02:36:48.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.695+0000 7f3e67afb700 1 -- 172.21.15.7:0/338742569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e60171f70 con 0x7f3e60147d50 2024-01-27T02:36:48.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.695+0000 7f3e55ffb700 1 -- 172.21.15.7:0/338742569 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3e500119e0 con 0x7f3e60147d50 2024-01-27T02:36:48.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.696+0000 7f3e55ffb700 1 --2- 172.21.15.7:0/338742569 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3e44066f30 0x7f3e440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:48.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.696+0000 7f3e55ffb700 1 -- 172.21.15.7:0/338742569 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3e50085d60 con 0x7f3e60147d50 2024-01-27T02:36:48.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.696+0000 7f3e55ffb700 1 -- 172.21.15.7:0/338742569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=85}) v3 -- 0x7f3e4406ea50 con 0x7f3e60147d50 2024-01-27T02:36:48.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.696+0000 7f3e65897700 1 --2- 172.21.15.7:0/338742569 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3e44066f30 0x7f3e440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:48.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.696+0000 7f3e65897700 1 --2- 172.21.15.7:0/338742569 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3e44066f30 0x7f3e440693e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009d40 tx=0x7f3e5c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:48.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.700+0000 7f3e55ffb700 1 -- 172.21.15.7:0/338742569 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e60171f70 con 0x7f3e60147d50 2024-01-27T02:36:48.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.904+0000 7f3e67afb700 1 -- 172.21.15.7:0/338742569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e60171f70 con 0x7f3e60147d50 2024-01-27T02:36:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.905+0000 7f3e55ffb700 1 -- 172.21.15.7:0/338742569 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+15348 (secure 0 0 0) 0x7f3e60171f70 con 0x7f3e60147d50 2024-01-27T02:36:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:36:48.907 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:36:22.866314+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:35:59.537952+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-27T02:36:00.480234+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:36:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.907+0000 7f3e67afb700 1 -- 172.21.15.7:0/338742569 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3e44066f30 msgr2=0x7f3e440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.907+0000 7f3e67afb700 1 --2- 172.21.15.7:0/338742569 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3e44066f30 0x7f3e440693e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009d40 tx=0x7f3e5c008040 comp rx=0 tx=0).stop 2024-01-27T02:36:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.907+0000 7f3e67afb700 1 -- 172.21.15.7:0/338742569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e60147d50 msgr2=0x7f3e60145ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.907+0000 7f3e67afb700 1 --2- 172.21.15.7:0/338742569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e60147d50 0x7f3e60145ca0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3e6006dcc0 tx=0x7f3e50017000 comp rx=0 tx=0).stop 2024-01-27T02:36:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.907+0000 7f3e67afb700 1 -- 172.21.15.7:0/338742569 shutdown_connections 2024-01-27T02:36:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.907+0000 7f3e67afb700 1 --2- 172.21.15.7:0/338742569 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3e44066f30 0x7f3e440693e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.907+0000 7f3e67afb700 1 --2- 172.21.15.7:0/338742569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e60148120 0x7f3e601461e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.907+0000 7f3e67afb700 1 --2- 172.21.15.7:0/338742569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e600c89c0 0x7f3e601446b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.907+0000 7f3e67afb700 1 --2- 172.21.15.7:0/338742569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e60147d50 0x7f3e60145ca0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:48.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.907+0000 7f3e67afb700 1 -- 172.21.15.7:0/338742569 >> 172.21.15.7:0/338742569 conn(0x7f3e600cdc30 msgr2=0x7f3e601620e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:48.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.907+0000 7f3e67afb700 1 -- 172.21.15.7:0/338742569 shutdown_connections 2024-01-27T02:36:48.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:48.907+0000 7f3e67afb700 1 -- 172.21.15.7:0/338742569 wait complete. 2024-01-27T02:36:48.925 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-27T02:35:59.537952+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-01-27T02:36:00.480234+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-27T02:36:48.925 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:36:49.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.058+0000 7fb0eebdb700 1 Processor -- start 2024-01-27T02:36:49.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.058+0000 7fb0eebdb700 1 -- start start 2024-01-27T02:36:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.059+0000 7fb0eebdb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0e80cb950 0x7fb0e80b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.059+0000 7fb0eebdb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0e80cbd20 0x7fb0e80ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.059+0000 7fb0eebdb700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e80c9660 con 0x7fb0e81412d0 2024-01-27T02:36:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.059+0000 7fb0eebdb700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e815c980 con 0x7fb0e80cbd20 2024-01-27T02:36:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.059+0000 7fb0eebdb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e815be00 con 0x7fb0e80cb950 2024-01-27T02:36:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.060+0000 7fb0e7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0e80cbd20 0x7fb0e80ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.060+0000 7fb0e7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0e80cbd20 0x7fb0e80ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42932/0 (socket says 172.21.15.7:42932) 2024-01-27T02:36:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.060+0000 7fb0e7fff700 1 -- 172.21.15.7:0/3151862842 learned_addr learned my addr 172.21.15.7:0/3151862842 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.060+0000 7fb0ec977700 1 --2- 172.21.15.7:0/3151862842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0e80cb950 0x7fb0e80b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.060+0000 7fb0e7fff700 1 -- 172.21.15.7:0/3151862842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0e80cb950 msgr2=0x7fb0e80b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.060+0000 7fb0e7fff700 1 --2- 172.21.15.7:0/3151862842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0e80cb950 0x7fb0e80b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.060+0000 7fb0e7fff700 1 -- 172.21.15.7:0/3151862842 >> v1:172.21.15.188:6789/0 conn(0x7fb0e81412d0 legacy=0x7fb0e8141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:49.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.060+0000 7fb0e7fff700 1 -- 172.21.15.7:0/3151862842 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e815c0e0 con 0x7fb0e80cbd20 2024-01-27T02:36:49.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.060+0000 7fb0e7fff700 1 --2- 172.21.15.7:0/3151862842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e80cbd20 0x7fb0e80ce4a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb0d40099b0 tx=0x7fb0d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b149bda5fb63a17 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:49.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.060+0000 7fb0e77fe700 1 -- 172.21.15.7:0/3151862842 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d400b860 con 0x7fb0e80cbd20 2024-01-27T02:36:49.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.060+0000 7fb0e77fe700 1 -- 172.21.15.7:0/3151862842 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d400b9c0 con 0x7fb0e80cbd20 2024-01-27T02:36:49.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.060+0000 7fb0e77fe700 1 -- 172.21.15.7:0/3151862842 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d400bd60 con 0x7fb0e80cbd20 2024-01-27T02:36:49.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.061+0000 7fb0eebdb700 1 -- 172.21.15.7:0/3151862842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e80cbd20 msgr2=0x7fb0e80ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:49.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.061+0000 7fb0eebdb700 1 --2- 172.21.15.7:0/3151862842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e80cbd20 0x7fb0e80ce4a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb0d40099b0 tx=0x7fb0d4009250 comp rx=0 tx=0).stop 2024-01-27T02:36:49.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.061+0000 7fb0eebdb700 1 -- 172.21.15.7:0/3151862842 shutdown_connections 2024-01-27T02:36:49.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.061+0000 7fb0eebdb700 1 --2- 172.21.15.7:0/3151862842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0e80cb950 0x7fb0e80b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.061+0000 7fb0eebdb700 1 --2- 172.21.15.7:0/3151862842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e80cbd20 0x7fb0e80ce4a0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.061+0000 7fb0eebdb700 1 -- 172.21.15.7:0/3151862842 >> 172.21.15.7:0/3151862842 conn(0x7fb0e8063580 msgr2=0x7fb0e8063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:49.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.061+0000 7fb0eebdb700 1 -- 172.21.15.7:0/3151862842 shutdown_connections 2024-01-27T02:36:49.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.061+0000 7fb0eebdb700 1 -- 172.21.15.7:0/3151862842 wait complete. 2024-01-27T02:36:49.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.062+0000 7fb0eebdb700 1 Processor -- start 2024-01-27T02:36:49.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.062+0000 7fb0eebdb700 1 -- start start 2024-01-27T02:36:49.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.063+0000 7fb0eebdb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0e80cb950 0x7fb0e8159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:49.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.063+0000 7fb0eebdb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e80cbd20 0x7fb0e80b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:49.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.063+0000 7fb0eebdb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0e81412d0 0x7fb0e80b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:49.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.063+0000 7fb0eebdb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e816b190 con 0x7fb0e80cbd20 2024-01-27T02:36:49.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.063+0000 7fb0eebdb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e816dfc0 con 0x7fb0e80cb950 2024-01-27T02:36:49.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.063+0000 7fb0eebdb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e813ff40 con 0x7fb0e81412d0 2024-01-27T02:36:49.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.063+0000 7fb0e7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e80cbd20 0x7fb0e80b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:49.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.063+0000 7fb0e7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e80cbd20 0x7fb0e80b33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42942/0 (socket says 172.21.15.7:42942) 2024-01-27T02:36:49.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.063+0000 7fb0e7fff700 1 -- 172.21.15.7:0/1525424583 learned_addr learned my addr 172.21.15.7:0/1525424583 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:49.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.064+0000 7fb0ec977700 1 --2- 172.21.15.7:0/1525424583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0e80cb950 0x7fb0e8159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:49.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.064+0000 7fb0ed178700 1 --2- 172.21.15.7:0/1525424583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0e81412d0 0x7fb0e80b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:49.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.064+0000 7fb0e7fff700 1 -- 172.21.15.7:0/1525424583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0e81412d0 msgr2=0x7fb0e80b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:49.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.064+0000 7fb0e7fff700 1 --2- 172.21.15.7:0/1525424583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0e81412d0 0x7fb0e80b38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.064+0000 7fb0e7fff700 1 -- 172.21.15.7:0/1525424583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0e80cb950 msgr2=0x7fb0e8159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:49.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.064+0000 7fb0e7fff700 1 --2- 172.21.15.7:0/1525424583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0e80cb950 0x7fb0e8159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.064+0000 7fb0e7fff700 1 -- 172.21.15.7:0/1525424583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d40094f0 con 0x7fb0e80cbd20 2024-01-27T02:36:49.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.064+0000 7fb0ec977700 1 --2- 172.21.15.7:0/1525424583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0e80cb950 0x7fb0e8159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:36:49.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.064+0000 7fb0e7fff700 1 --2- 172.21.15.7:0/1525424583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e80cbd20 0x7fb0e80b33b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80c97f0 tx=0x7fb0d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:49.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.064+0000 7fb0e4ff9700 1 -- 172.21.15.7:0/1525424583 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4024440 con 0x7fb0e80cbd20 2024-01-27T02:36:49.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.064+0000 7fb0e4ff9700 1 -- 172.21.15.7:0/1525424583 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d40245a0 con 0x7fb0e80cbd20 2024-01-27T02:36:49.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.064+0000 7fb0eebdb700 1 -- 172.21.15.7:0/1525424583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e816aeb0 con 0x7fb0e80cbd20 2024-01-27T02:36:49.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.064+0000 7fb0e4ff9700 1 -- 172.21.15.7:0/1525424583 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4024940 con 0x7fb0e80cbd20 2024-01-27T02:36:49.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.064+0000 7fb0eebdb700 1 -- 172.21.15.7:0/1525424583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e816dc50 con 0x7fb0e80cbd20 2024-01-27T02:36:49.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.065+0000 7fb0eebdb700 1 -- 172.21.15.7:0/1525424583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e8072060 con 0x7fb0e80cbd20 2024-01-27T02:36:49.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.066+0000 7fb0e4ff9700 1 -- 172.21.15.7:0/1525424583 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb0d401a420 con 0x7fb0e80cbd20 2024-01-27T02:36:49.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.066+0000 7fb0e4ff9700 1 --2- 172.21.15.7:0/1525424583 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb0cc066f30 0x7fb0cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:49.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.066+0000 7fb0e4ff9700 1 -- 172.21.15.7:0/1525424583 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb0d40863e0 con 0x7fb0e80cbd20 2024-01-27T02:36:49.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.066+0000 7fb0e4ff9700 1 -- 172.21.15.7:0/1525424583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=85}) v3 -- 0x7fb0cc06ea50 con 0x7fb0e80cbd20 2024-01-27T02:36:49.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.066+0000 7fb0ec977700 1 --2- 172.21.15.7:0/1525424583 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb0cc066f30 0x7fb0cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:49.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.067+0000 7fb0ec977700 1 --2- 172.21.15.7:0/1525424583 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb0cc066f30 0x7fb0cc0693e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc006fd0 tx=0x7fb0dc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:49.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.071+0000 7fb0e4ff9700 1 -- 172.21.15.7:0/1525424583 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0d4053650 con 0x7fb0e80cbd20 2024-01-27T02:36:49.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.276+0000 7fb0eebdb700 1 -- 172.21.15.7:0/1525424583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb0e806d3e0 con 0x7fb0e80cbd20 2024-01-27T02:36:49.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.276+0000 7fb0e4ff9700 1 -- 172.21.15.7:0/1525424583 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb0d4015020 con 0x7fb0e80cbd20 2024-01-27T02:36:49.278 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:36:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.279+0000 7fb0eebdb700 1 -- 172.21.15.7:0/1525424583 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb0cc066f30 msgr2=0x7fb0cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.279+0000 7fb0eebdb700 1 --2- 172.21.15.7:0/1525424583 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb0cc066f30 0x7fb0cc0693e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc006fd0 tx=0x7fb0dc008040 comp rx=0 tx=0).stop 2024-01-27T02:36:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.279+0000 7fb0eebdb700 1 -- 172.21.15.7:0/1525424583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e80cbd20 msgr2=0x7fb0e80b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:49.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.279+0000 7fb0eebdb700 1 --2- 172.21.15.7:0/1525424583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e80cbd20 0x7fb0e80b33b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80c97f0 tx=0x7fb0d4017000 comp rx=0 tx=0).stop 2024-01-27T02:36:49.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.279+0000 7fb0eebdb700 1 -- 172.21.15.7:0/1525424583 shutdown_connections 2024-01-27T02:36:49.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.279+0000 7fb0eebdb700 1 --2- 172.21.15.7:0/1525424583 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb0cc066f30 0x7fb0cc0693e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.279+0000 7fb0eebdb700 1 --2- 172.21.15.7:0/1525424583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0e81412d0 0x7fb0e80b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.279+0000 7fb0eebdb700 1 --2- 172.21.15.7:0/1525424583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0e80cb950 0x7fb0e8159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.279+0000 7fb0eebdb700 1 --2- 172.21.15.7:0/1525424583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e80cbd20 0x7fb0e80b33b0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.279+0000 7fb0eebdb700 1 -- 172.21.15.7:0/1525424583 >> 172.21.15.7:0/1525424583 conn(0x7fb0e8063580 msgr2=0x7fb0e80cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:49.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.279+0000 7fb0eebdb700 1 -- 172.21.15.7:0/1525424583 shutdown_connections 2024-01-27T02:36:49.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.279+0000 7fb0eebdb700 1 -- 172.21.15.7:0/1525424583 wait complete. 2024-01-27T02:36:49.294 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-27T02:36:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.428+0000 7f15136f3700 1 Processor -- start 2024-01-27T02:36:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.428+0000 7f15136f3700 1 -- start start 2024-01-27T02:36:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.429+0000 7f15136f3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f150c15e7a0 0x7f150c0a8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.429+0000 7f15136f3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f150c063c90 0x7f150c0afa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.429+0000 7f15136f3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f150c0a80c0 con 0x7f150c15e3d0 2024-01-27T02:36:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.429+0000 7f15136f3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f150c0ce6b0 con 0x7f150c063c90 2024-01-27T02:36:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.429+0000 7f15136f3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150c0a7790 con 0x7f150c15e7a0 2024-01-27T02:36:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.429+0000 7f1511c90700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f150c063c90 0x7f150c0afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.429+0000 7f151148f700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f150c15e3d0 0x7f150c063580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54906/0 (socket says 172.21.15.7:54906) 2024-01-27T02:36:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.429+0000 7f151148f700 1 -- 172.21.15.7:0/1946412837 learned_addr learned my addr 172.21.15.7:0/1946412837 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.430+0000 7f1510c8e700 1 --2- 172.21.15.7:0/1946412837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f150c15e7a0 0x7f150c0a8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.430+0000 7f1511c90700 1 -- 172.21.15.7:0/1946412837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f150c15e7a0 msgr2=0x7f150c0a8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.430+0000 7f1511c90700 1 --2- 172.21.15.7:0/1946412837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f150c15e7a0 0x7f150c0a8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.430+0000 7f1511c90700 1 -- 172.21.15.7:0/1946412837 >> v1:172.21.15.7:6789/0 conn(0x7f150c15e3d0 legacy=0x7f150c063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.430+0000 7f1511c90700 1 -- 172.21.15.7:0/1946412837 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150c0a7a70 con 0x7f150c063c90 2024-01-27T02:36:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.430+0000 7f1503fff700 1 -- 172.21.15.7:0/1946412837 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048995882 0 0) 0x7f150c0a80c0 con 0x7f150c15e3d0 2024-01-27T02:36:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.430+0000 7f1511c90700 1 --2- 172.21.15.7:0/1946412837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f150c063c90 0x7f150c0afa40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1504009e60 tx=0x7f1504009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db763ad9704dc0f9 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.430+0000 7f1503fff700 1 -- 172.21.15.7:0/1946412837 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150400e070 con 0x7f150c063c90 2024-01-27T02:36:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.430+0000 7f1503fff700 1 -- 172.21.15.7:0/1946412837 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150400bb60 con 0x7f150c063c90 2024-01-27T02:36:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.430+0000 7f1503fff700 1 -- 172.21.15.7:0/1946412837 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1504012430 con 0x7f150c063c90 2024-01-27T02:36:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.431+0000 7f15136f3700 1 -- 172.21.15.7:0/1946412837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f150c063c90 msgr2=0x7f150c0afa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.431+0000 7f15136f3700 1 --2- 172.21.15.7:0/1946412837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f150c063c90 0x7f150c0afa40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1504009e60 tx=0x7f1504009780 comp rx=0 tx=0).stop 2024-01-27T02:36:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.431+0000 7f15136f3700 1 -- 172.21.15.7:0/1946412837 shutdown_connections 2024-01-27T02:36:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.431+0000 7f15136f3700 1 --2- 172.21.15.7:0/1946412837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f150c063c90 0x7f150c0afa40 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.431+0000 7f15136f3700 1 --2- 172.21.15.7:0/1946412837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f150c15e7a0 0x7f150c0a8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.431+0000 7f15136f3700 1 -- 172.21.15.7:0/1946412837 >> 172.21.15.7:0/1946412837 conn(0x7f150c068480 msgr2=0x7f150c0cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.431+0000 7f15136f3700 1 -- 172.21.15.7:0/1946412837 shutdown_connections 2024-01-27T02:36:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.431+0000 7f15136f3700 1 -- 172.21.15.7:0/1946412837 wait complete. 2024-01-27T02:36:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.432+0000 7f15136f3700 1 Processor -- start 2024-01-27T02:36:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.432+0000 7f15136f3700 1 -- start start 2024-01-27T02:36:49.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.433+0000 7f15136f3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f150c063c90 0x7f150c16e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:49.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.433+0000 7f15136f3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f150c15e3d0 0x7f150c0ac930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:49.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.433+0000 7f15136f3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f150c15e7a0 0x7f150c161050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.433+0000 7f15136f3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c160540 con 0x7f150c15e7a0 2024-01-27T02:36:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.433+0000 7f15136f3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c16eee0 con 0x7f150c063c90 2024-01-27T02:36:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.433+0000 7f15136f3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150c15eea0 con 0x7f150c15e3d0 2024-01-27T02:36:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.433+0000 7f1511c90700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f150c15e7a0 0x7f150c161050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.433+0000 7f1511c90700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f150c15e7a0 0x7f150c161050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42974/0 (socket says 172.21.15.7:42974) 2024-01-27T02:36:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.433+0000 7f1511c90700 1 -- 172.21.15.7:0/3203731767 learned_addr learned my addr 172.21.15.7:0/3203731767 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.433+0000 7f1510c8e700 1 --2- 172.21.15.7:0/3203731767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f150c15e3d0 0x7f150c0ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.433+0000 7f151148f700 1 --2- 172.21.15.7:0/3203731767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f150c063c90 0x7f150c16e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.434+0000 7f1511c90700 1 -- 172.21.15.7:0/3203731767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f150c15e3d0 msgr2=0x7f150c0ac930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.434+0000 7f1511c90700 1 --2- 172.21.15.7:0/3203731767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f150c15e3d0 0x7f150c0ac930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.434+0000 7f1511c90700 1 -- 172.21.15.7:0/3203731767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f150c063c90 msgr2=0x7f150c16e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.434+0000 7f1511c90700 1 --2- 172.21.15.7:0/3203731767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f150c063c90 0x7f150c16e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.434+0000 7f1511c90700 1 -- 172.21.15.7:0/3203731767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1504009a20 con 0x7f150c15e7a0 2024-01-27T02:36:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.434+0000 7f1511c90700 1 --2- 172.21.15.7:0/3203731767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f150c15e7a0 0x7f150c161050 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1504013000 tx=0x7f1504019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:49.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.434+0000 7f15017fa700 1 -- 172.21.15.7:0/3203731767 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1504012ea0 con 0x7f150c15e7a0 2024-01-27T02:36:49.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.434+0000 7f15017fa700 1 -- 172.21.15.7:0/3203731767 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1504027470 con 0x7f150c15e7a0 2024-01-27T02:36:49.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.434+0000 7f15017fa700 1 -- 172.21.15.7:0/3203731767 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1504027830 con 0x7f150c15e7a0 2024-01-27T02:36:49.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.434+0000 7f15136f3700 1 -- 172.21.15.7:0/3203731767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150c15f930 con 0x7f150c15e7a0 2024-01-27T02:36:49.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.434+0000 7f15136f3700 1 -- 172.21.15.7:0/3203731767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150c15fef0 con 0x7f150c15e7a0 2024-01-27T02:36:49.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.435+0000 7f15136f3700 1 -- 172.21.15.7:0/3203731767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150c072060 con 0x7f150c15e7a0 2024-01-27T02:36:49.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.436+0000 7f15017fa700 1 -- 172.21.15.7:0/3203731767 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1504024070 con 0x7f150c15e7a0 2024-01-27T02:36:49.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.436+0000 7f15017fa700 1 --2- 172.21.15.7:0/3203731767 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f14f0066f30 0x7f14f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:49.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.436+0000 7f15017fa700 1 -- 172.21.15.7:0/3203731767 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1504086c60 con 0x7f150c15e7a0 2024-01-27T02:36:49.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.436+0000 7f15017fa700 1 -- 172.21.15.7:0/3203731767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=85}) v3 -- 0x7f14f006ea50 con 0x7f150c15e7a0 2024-01-27T02:36:49.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.436+0000 7f151148f700 1 --2- 172.21.15.7:0/3203731767 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f14f0066f30 0x7f14f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:49.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.437+0000 7f151148f700 1 --2- 172.21.15.7:0/3203731767 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f14f0066f30 0x7f14f00693e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f14fc007fd0 tx=0x7f14fc007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:49.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.441+0000 7f15017fa700 1 -- 172.21.15.7:0/3203731767 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f150402c060 con 0x7f150c15e7a0 2024-01-27T02:36:49.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.640+0000 7f15136f3700 1 -- 172.21.15.7:0/3203731767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f150c0cde40 con 0x7f150c15e7a0 2024-01-27T02:36:49.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.640+0000 7f15017fa700 1 -- 172.21.15.7:0/3203731767 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1504053ed0 con 0x7f150c15e7a0 2024-01-27T02:36:49.642 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-27T02:36:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.642+0000 7f15136f3700 1 -- 172.21.15.7:0/3203731767 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f14f0066f30 msgr2=0x7f14f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.642+0000 7f15136f3700 1 --2- 172.21.15.7:0/3203731767 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f14f0066f30 0x7f14f00693e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f14fc007fd0 tx=0x7f14fc007d30 comp rx=0 tx=0).stop 2024-01-27T02:36:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.642+0000 7f15136f3700 1 -- 172.21.15.7:0/3203731767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f150c15e7a0 msgr2=0x7f150c161050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.642+0000 7f15136f3700 1 --2- 172.21.15.7:0/3203731767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f150c15e7a0 0x7f150c161050 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1504013000 tx=0x7f1504019000 comp rx=0 tx=0).stop 2024-01-27T02:36:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.642+0000 7f15136f3700 1 -- 172.21.15.7:0/3203731767 shutdown_connections 2024-01-27T02:36:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.642+0000 7f15136f3700 1 --2- 172.21.15.7:0/3203731767 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f14f0066f30 0x7f14f00693e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.642+0000 7f15136f3700 1 --2- 172.21.15.7:0/3203731767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f150c15e7a0 0x7f150c161050 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.642+0000 7f15136f3700 1 --2- 172.21.15.7:0/3203731767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f150c063c90 0x7f150c16e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.642+0000 7f15136f3700 1 --2- 172.21.15.7:0/3203731767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f150c15e3d0 0x7f150c0ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.642+0000 7f15136f3700 1 -- 172.21.15.7:0/3203731767 >> 172.21.15.7:0/3203731767 conn(0x7f150c068480 msgr2=0x7f150c15baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:49.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.642+0000 7f15136f3700 1 -- 172.21.15.7:0/3203731767 shutdown_connections 2024-01-27T02:36:49.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.642+0000 7f15136f3700 1 -- 172.21.15.7:0/3203731767 wait complete. 2024-01-27T02:36:49.658 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-27T02:36:49.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.792+0000 7fe0c56e8700 1 Processor -- start 2024-01-27T02:36:49.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.792+0000 7fe0c56e8700 1 -- start start 2024-01-27T02:36:49.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.792+0000 7fe0c56e8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0c00680c0 0x7fe0c00693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:49.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.792+0000 7fe0c56e8700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe0c00cc8d0 0x7fe0c00c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:49.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.792+0000 7fe0c56e8700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c0069930 con 0x7fe0c00b0f70 2024-01-27T02:36:49.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.792+0000 7fe0c56e8700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c0068840 con 0x7fe0c00cc8d0 2024-01-27T02:36:49.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.792+0000 7fe0c56e8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c015e820 con 0x7fe0c00680c0 2024-01-27T02:36:49.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.793+0000 7fe0bf7fe700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe0c00cc8d0 0x7fe0c00c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.793+0000 7fe0be7fc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0c00680c0 0x7fe0c00693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.793+0000 7fe0be7fc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0c00680c0 0x7fe0c00693f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:38034/0 (socket says 172.21.15.7:38034) 2024-01-27T02:36:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.793+0000 7fe0bf7fe700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe0c00cc8d0 0x7fe0c00c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:44638/0 (socket says 172.21.15.7:44638) 2024-01-27T02:36:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.793+0000 7fe0be7fc700 1 -- 172.21.15.7:0/333804702 learned_addr learned my addr 172.21.15.7:0/333804702 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.793+0000 7fe0be7fc700 1 -- 172.21.15.7:0/333804702 >> v2:172.21.15.188:3300/0 conn(0x7fe0c00cc8d0 msgr2=0x7fe0c00c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.793+0000 7fe0be7fc700 1 --2- 172.21.15.7:0/333804702 >> v2:172.21.15.188:3300/0 conn(0x7fe0c00cc8d0 0x7fe0c00c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.793+0000 7fe0be7fc700 1 -- 172.21.15.7:0/333804702 >> v1:172.21.15.188:6789/0 conn(0x7fe0c00b0f70 legacy=0x7fe0c00679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.793+0000 7fe0be7fc700 1 -- 172.21.15.7:0/333804702 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c015dc10 con 0x7fe0c00680c0 2024-01-27T02:36:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.793+0000 7fe0be7fc700 1 --2- 172.21.15.7:0/333804702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0c00680c0 0x7fe0c00693f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00099b0 tx=0x7fe0b0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f2847cbbe639d96 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.794+0000 7fe0bdffb700 1 -- 172.21.15.7:0/333804702 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0009ea0 con 0x7fe0c00680c0 2024-01-27T02:36:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.794+0000 7fe0bdffb700 1 -- 172.21.15.7:0/333804702 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b000b860 con 0x7fe0c00680c0 2024-01-27T02:36:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.794+0000 7fe0bdffb700 1 -- 172.21.15.7:0/333804702 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b000bc00 con 0x7fe0c00680c0 2024-01-27T02:36:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.794+0000 7fe0c56e8700 1 -- 172.21.15.7:0/333804702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0c00680c0 msgr2=0x7fe0c00693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.794+0000 7fe0c56e8700 1 --2- 172.21.15.7:0/333804702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0c00680c0 0x7fe0c00693f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00099b0 tx=0x7fe0b0009250 comp rx=0 tx=0).stop 2024-01-27T02:36:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.794+0000 7fe0c56e8700 1 -- 172.21.15.7:0/333804702 shutdown_connections 2024-01-27T02:36:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.794+0000 7fe0c56e8700 1 --2- 172.21.15.7:0/333804702 >> v2:172.21.15.188:3300/0 conn(0x7fe0c00cc8d0 0x7fe0c00c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.794+0000 7fe0c56e8700 1 --2- 172.21.15.7:0/333804702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0c00680c0 0x7fe0c00693f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.794+0000 7fe0c56e8700 1 -- 172.21.15.7:0/333804702 >> 172.21.15.7:0/333804702 conn(0x7fe0c0063580 msgr2=0x7fe0c0063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.794+0000 7fe0c56e8700 1 -- 172.21.15.7:0/333804702 shutdown_connections 2024-01-27T02:36:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.794+0000 7fe0c56e8700 1 -- 172.21.15.7:0/333804702 wait complete. 2024-01-27T02:36:49.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.796+0000 7fe0c56e8700 1 Processor -- start 2024-01-27T02:36:49.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.796+0000 7fe0c56e8700 1 -- start start 2024-01-27T02:36:49.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.796+0000 7fe0c56e8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0c00680c0 0x7fe0c00c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:49.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.796+0000 7fe0c56e8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0c00b0f70 0x7fe0c0157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:49.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.796+0000 7fe0c56e8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0c00cc8d0 0x7fe0c00b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:49.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.796+0000 7fe0c56e8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00c7260 con 0x7fe0c00cc8d0 2024-01-27T02:36:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.796+0000 7fe0c56e8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00c74e0 con 0x7fe0c00680c0 2024-01-27T02:36:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.796+0000 7fe0c56e8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c00ca880 con 0x7fe0c00b0f70 2024-01-27T02:36:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.797+0000 7fe0bf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0c00cc8d0 0x7fe0c00b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.797+0000 7fe0bf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0c00cc8d0 0x7fe0c00b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42978/0 (socket says 172.21.15.7:42978) 2024-01-27T02:36:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.797+0000 7fe0bf7fe700 1 -- 172.21.15.7:0/3922276969 learned_addr learned my addr 172.21.15.7:0/3922276969 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.797+0000 7fe0beffd700 1 --2- 172.21.15.7:0/3922276969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0c00680c0 0x7fe0c00c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.797+0000 7fe0be7fc700 1 --2- 172.21.15.7:0/3922276969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0c00b0f70 0x7fe0c0157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.797+0000 7fe0bf7fe700 1 -- 172.21.15.7:0/3922276969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0c00b0f70 msgr2=0x7fe0c0157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.797+0000 7fe0bf7fe700 1 --2- 172.21.15.7:0/3922276969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0c00b0f70 0x7fe0c0157600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.797+0000 7fe0bf7fe700 1 -- 172.21.15.7:0/3922276969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0c00680c0 msgr2=0x7fe0c00c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.797+0000 7fe0bf7fe700 1 --2- 172.21.15.7:0/3922276969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0c00680c0 0x7fe0c00c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.797+0000 7fe0bf7fe700 1 -- 172.21.15.7:0/3922276969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b00094f0 con 0x7fe0c00cc8d0 2024-01-27T02:36:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.797+0000 7fe0bf7fe700 1 --2- 172.21.15.7:0/3922276969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0c00cc8d0 0x7fe0c00b3a20 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009ec0 tx=0x7fe0b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.797+0000 7fe0b6ffd700 1 -- 172.21.15.7:0/3922276969 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b800cb70 con 0x7fe0c00cc8d0 2024-01-27T02:36:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.797+0000 7fe0b6ffd700 1 -- 172.21.15.7:0/3922276969 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b800f040 con 0x7fe0c00cc8d0 2024-01-27T02:36:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.797+0000 7fe0c56e8700 1 -- 172.21.15.7:0/3922276969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c00c9fe0 con 0x7fe0c00cc8d0 2024-01-27T02:36:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.797+0000 7fe0b6ffd700 1 -- 172.21.15.7:0/3922276969 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8014480 con 0x7fe0c00cc8d0 2024-01-27T02:36:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.797+0000 7fe0c56e8700 1 -- 172.21.15.7:0/3922276969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c00ca620 con 0x7fe0c00cc8d0 2024-01-27T02:36:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.798+0000 7fe0c56e8700 1 -- 172.21.15.7:0/3922276969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c00c7670 con 0x7fe0c00cc8d0 2024-01-27T02:36:49.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.799+0000 7fe0b6ffd700 1 -- 172.21.15.7:0/3922276969 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe0b8014830 con 0x7fe0c00cc8d0 2024-01-27T02:36:49.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.799+0000 7fe0b6ffd700 1 --2- 172.21.15.7:0/3922276969 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe0a0066f30 0x7fe0a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:49.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.800+0000 7fe0b6ffd700 1 -- 172.21.15.7:0/3922276969 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe0b8084e70 con 0x7fe0c00cc8d0 2024-01-27T02:36:49.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.800+0000 7fe0b6ffd700 1 -- 172.21.15.7:0/3922276969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=85}) v3 -- 0x7fe0a006ea50 con 0x7fe0c00cc8d0 2024-01-27T02:36:49.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.800+0000 7fe0beffd700 1 --2- 172.21.15.7:0/3922276969 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe0a0066f30 0x7fe0a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:49.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.800+0000 7fe0beffd700 1 --2- 172.21.15.7:0/3922276969 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe0a0066f30 0x7fe0a00693e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8003a80 tx=0x7fe0a8007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:49.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:49.805+0000 7fe0b6ffd700 1 -- 172.21.15.7:0/3922276969 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe0b8052040 con 0x7fe0c00cc8d0 2024-01-27T02:36:50.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.008+0000 7fe0c56e8700 1 -- 172.21.15.7:0/3922276969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe0c0072060 con 0x7fe0c00cc8d0 2024-01-27T02:36:50.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.008+0000 7fe0b6ffd700 1 -- 172.21.15.7:0/3922276969 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe0b8018030 con 0x7fe0c00cc8d0 2024-01-27T02:36:50.010 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-27T02:36:50.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.010+0000 7fe0c56e8700 1 -- 172.21.15.7:0/3922276969 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe0a0066f30 msgr2=0x7fe0a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:50.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.010+0000 7fe0c56e8700 1 --2- 172.21.15.7:0/3922276969 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe0a0066f30 0x7fe0a00693e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8003a80 tx=0x7fe0a8007ec0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.010+0000 7fe0c56e8700 1 -- 172.21.15.7:0/3922276969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0c00cc8d0 msgr2=0x7fe0c00b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:50.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.010+0000 7fe0c56e8700 1 --2- 172.21.15.7:0/3922276969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0c00cc8d0 0x7fe0c00b3a20 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009ec0 tx=0x7fe0b800b040 comp rx=0 tx=0).stop 2024-01-27T02:36:50.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.011+0000 7fe0c56e8700 1 -- 172.21.15.7:0/3922276969 shutdown_connections 2024-01-27T02:36:50.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.011+0000 7fe0c56e8700 1 --2- 172.21.15.7:0/3922276969 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe0a0066f30 0x7fe0a00693e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.011+0000 7fe0c56e8700 1 --2- 172.21.15.7:0/3922276969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0c00cc8d0 0x7fe0c00b3a20 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.011+0000 7fe0c56e8700 1 --2- 172.21.15.7:0/3922276969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0c00680c0 0x7fe0c00c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.011+0000 7fe0c56e8700 1 --2- 172.21.15.7:0/3922276969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0c00b0f70 0x7fe0c0157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.011+0000 7fe0c56e8700 1 -- 172.21.15.7:0/3922276969 >> 172.21.15.7:0/3922276969 conn(0x7fe0c0063580 msgr2=0x7fe0c0159600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:50.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.011+0000 7fe0c56e8700 1 -- 172.21.15.7:0/3922276969 shutdown_connections 2024-01-27T02:36:50.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.011+0000 7fe0c56e8700 1 -- 172.21.15.7:0/3922276969 wait complete. 2024-01-27T02:36:50.026 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:36:50.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.159+0000 7f3443ae4700 1 Processor -- start 2024-01-27T02:36:50.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.159+0000 7f3443ae4700 1 -- start start 2024-01-27T02:36:50.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.160+0000 7f3443ae4700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f343c0c3ae0 0x7f343c0cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.160+0000 7f3443ae4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f343c158070 0x7f343c0cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.160+0000 7f3443ae4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f343c0ca020 con 0x7f343c157ca0 2024-01-27T02:36:50.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.160+0000 7f3443ae4700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f343c0c5210 con 0x7f343c0c3ae0 2024-01-27T02:36:50.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.160+0000 7f3443ae4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343c0c54f0 con 0x7f343c158070 2024-01-27T02:36:50.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.160+0000 7f3441880700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f343c157ca0 0x7f343c0c4a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54916/0 (socket says 172.21.15.7:54916) 2024-01-27T02:36:50.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.160+0000 7f3441880700 1 -- 172.21.15.7:0/1892401590 learned_addr learned my addr 172.21.15.7:0/1892401590 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:50.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.161+0000 7f344107f700 1 --2- 172.21.15.7:0/1892401590 >> v2:172.21.15.188:3300/0 conn(0x7f343c0c3ae0 0x7f343c0cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.161+0000 7f344087e700 1 -- 172.21.15.7:0/1892401590 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3988104794 0 0) 0x7f343c0ca020 con 0x7f343c157ca0 2024-01-27T02:36:50.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.161+0000 7f3442081700 1 --2- 172.21.15.7:0/1892401590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f343c158070 0x7f343c0cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.161+0000 7f344087e700 1 -- 172.21.15.7:0/1892401590 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3428003570 con 0x7f343c157ca0 2024-01-27T02:36:50.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.161+0000 7f344087e700 1 -- 172.21.15.7:0/1892401590 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 297102563 0 0) 0x7f3428003570 con 0x7f343c157ca0 2024-01-27T02:36:50.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.161+0000 7f344087e700 1 -- 172.21.15.7:0/1892401590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f343c158070 msgr2=0x7f343c0cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:50.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.161+0000 7f344087e700 1 --2- 172.21.15.7:0/1892401590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f343c158070 0x7f343c0cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.161+0000 7f344087e700 1 -- 172.21.15.7:0/1892401590 >> v2:172.21.15.188:3300/0 conn(0x7f343c0c3ae0 msgr2=0x7f343c0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:50.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.161+0000 7f344087e700 1 --2- 172.21.15.7:0/1892401590 >> v2:172.21.15.188:3300/0 conn(0x7f343c0c3ae0 0x7f343c0cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.161+0000 7f344087e700 1 -- 172.21.15.7:0/1892401590 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c0c57d0 con 0x7f343c157ca0 2024-01-27T02:36:50.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.161+0000 7f344087e700 1 -- 172.21.15.7:0/1892401590 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f3438001470 con 0x7f343c157ca0 2024-01-27T02:36:50.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.161+0000 7f344087e700 1 -- 172.21.15.7:0/1892401590 >> v1:172.21.15.7:6789/0 conn(0x7f343c157ca0 legacy=0x7f343c0c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:50.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.161+0000 7f344087e700 1 --2- 172.21.15.7:0/1892401590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f343c158070 0x7f3428003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.161+0000 7f344087e700 1 --2- 172.21.15.7:0/1892401590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3428003fe0 0x7f3428004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.161+0000 7f344087e700 1 --2- 172.21.15.7:0/1892401590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3428004d50 0x7f34280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.161+0000 7f344087e700 1 -- 172.21.15.7:0/1892401590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c0c5210 con 0x7f3428004d50 2024-01-27T02:36:50.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.161+0000 7f344087e700 1 -- 172.21.15.7:0/1892401590 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c0c54f0 con 0x7f3428003fe0 2024-01-27T02:36:50.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.161+0000 7f344087e700 1 -- 172.21.15.7:0/1892401590 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342800ba90 con 0x7f343c158070 2024-01-27T02:36:50.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.161+0000 7f3442081700 1 --2- 172.21.15.7:0/1892401590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3428004d50 0x7f34280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.161+0000 7f3441880700 1 --2- 172.21.15.7:0/1892401590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f343c158070 0x7f3428003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.161+0000 7f344107f700 1 --2- 172.21.15.7:0/1892401590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3428003fe0 0x7f3428004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.162+0000 7f3442081700 1 -- 172.21.15.7:0/1892401590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f343c158070 msgr2=0x7f3428003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:50.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.162+0000 7f3442081700 1 --2- 172.21.15.7:0/1892401590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f343c158070 0x7f3428003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.162+0000 7f3442081700 1 -- 172.21.15.7:0/1892401590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3428003fe0 msgr2=0x7f3428004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:50.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.162+0000 7f3442081700 1 --2- 172.21.15.7:0/1892401590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3428003fe0 0x7f3428004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.162+0000 7f3442081700 1 -- 172.21.15.7:0/1892401590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3428003570 con 0x7f3428004d50 2024-01-27T02:36:50.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.162+0000 7f3442081700 1 --2- 172.21.15.7:0/1892401590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3428004d50 0x7f34280051c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f343400e210 tx=0x7f343400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746a7fad26202eaa server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:50.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.162+0000 7f344087e700 1 -- 172.21.15.7:0/1892401590 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343400ea60 con 0x7f3428004d50 2024-01-27T02:36:50.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.162+0000 7f344087e700 1 -- 172.21.15.7:0/1892401590 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343400f040 con 0x7f3428004d50 2024-01-27T02:36:50.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.162+0000 7f344087e700 1 -- 172.21.15.7:0/1892401590 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343400ee10 con 0x7f3428004d50 2024-01-27T02:36:50.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.162+0000 7f3443ae4700 1 -- 172.21.15.7:0/1892401590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3428004d50 msgr2=0x7f34280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:50.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.162+0000 7f3443ae4700 1 --2- 172.21.15.7:0/1892401590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3428004d50 0x7f34280051c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f343400e210 tx=0x7f343400b040 comp rx=0 tx=0).stop 2024-01-27T02:36:50.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.162+0000 7f3442081700 1 -- 172.21.15.7:0/1892401590 reap_dead start 2024-01-27T02:36:50.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.163+0000 7f3443ae4700 1 -- 172.21.15.7:0/1892401590 shutdown_connections 2024-01-27T02:36:50.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.163+0000 7f3443ae4700 1 -- 172.21.15.7:0/1892401590 >> 172.21.15.7:0/1892401590 conn(0x7f343c098020 msgr2=0x7f343c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:50.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.163+0000 7f3443ae4700 1 -- 172.21.15.7:0/1892401590 shutdown_connections 2024-01-27T02:36:50.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.163+0000 7f3443ae4700 1 -- 172.21.15.7:0/1892401590 wait complete. 2024-01-27T02:36:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.164+0000 7f3443ae4700 1 Processor -- start 2024-01-27T02:36:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.164+0000 7f3443ae4700 1 -- start start 2024-01-27T02:36:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.165+0000 7f3443ae4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3428004d50 0x7f343c0c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.165+0000 7f3443ae4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f343c0c76e0 0x7f343c169e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.165+0000 7f3443ae4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343c16a390 0x7f343c16a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.165+0000 7f3443ae4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c0c7250 con 0x7f343c16a390 2024-01-27T02:36:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.165+0000 7f3443ae4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c0c6640 con 0x7f343c0c76e0 2024-01-27T02:36:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.165+0000 7f3443ae4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343c0c6920 con 0x7f3428004d50 2024-01-27T02:36:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.165+0000 7f3442081700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343c16a390 0x7f343c16a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.165+0000 7f3442081700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343c16a390 0x7f343c16a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42996/0 (socket says 172.21.15.7:42996) 2024-01-27T02:36:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.165+0000 7f3442081700 1 -- 172.21.15.7:0/3485650557 learned_addr learned my addr 172.21.15.7:0/3485650557 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.165+0000 7f3441880700 1 --2- 172.21.15.7:0/3485650557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3428004d50 0x7f343c0c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.165+0000 7f344107f700 1 --2- 172.21.15.7:0/3485650557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f343c0c76e0 0x7f343c169e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.165+0000 7f3442081700 1 -- 172.21.15.7:0/3485650557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3428004d50 msgr2=0x7f343c0c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.165+0000 7f3442081700 1 --2- 172.21.15.7:0/3485650557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3428004d50 0x7f343c0c5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.165+0000 7f3442081700 1 -- 172.21.15.7:0/3485650557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f343c0c76e0 msgr2=0x7f343c169e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.165+0000 7f3442081700 1 --2- 172.21.15.7:0/3485650557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f343c0c76e0 0x7f343c169e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.165+0000 7f3442081700 1 -- 172.21.15.7:0/3485650557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c0c6c00 con 0x7f343c16a390 2024-01-27T02:36:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.165+0000 7f344107f700 1 --2- 172.21.15.7:0/3485650557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f343c0c76e0 0x7f343c169e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:36:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.165+0000 7f3442081700 1 --2- 172.21.15.7:0/3485650557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343c16a390 0x7f343c16a830 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f343400afd0 tx=0x7f343401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.166+0000 7f3431ffb700 1 -- 172.21.15.7:0/3485650557 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343401e640 con 0x7f343c16a390 2024-01-27T02:36:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.166+0000 7f3431ffb700 1 -- 172.21.15.7:0/3485650557 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3434022070 con 0x7f343c16a390 2024-01-27T02:36:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.166+0000 7f3443ae4700 1 -- 172.21.15.7:0/3485650557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f343c0c6ee0 con 0x7f343c16a390 2024-01-27T02:36:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.166+0000 7f3431ffb700 1 -- 172.21.15.7:0/3485650557 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343401e9d0 con 0x7f343c16a390 2024-01-27T02:36:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.166+0000 7f3443ae4700 1 -- 172.21.15.7:0/3485650557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343c0afbc0 con 0x7f343c16a390 2024-01-27T02:36:50.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.167+0000 7f3443ae4700 1 -- 172.21.15.7:0/3485650557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f343c0c73c0 con 0x7f343c16a390 2024-01-27T02:36:50.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.167+0000 7f3431ffb700 1 -- 172.21.15.7:0/3485650557 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3434003040 con 0x7f343c16a390 2024-01-27T02:36:50.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.168+0000 7f3431ffb700 1 --2- 172.21.15.7:0/3485650557 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3420066fe0 0x7f3420069490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.168+0000 7f3431ffb700 1 -- 172.21.15.7:0/3485650557 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3434024070 con 0x7f343c16a390 2024-01-27T02:36:50.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.168+0000 7f3431ffb700 1 -- 172.21.15.7:0/3485650557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=85}) v3 -- 0x7f342006eb00 con 0x7f343c16a390 2024-01-27T02:36:50.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.168+0000 7f3441880700 1 --2- 172.21.15.7:0/3485650557 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3420066fe0 0x7f3420069490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.168+0000 7f3441880700 1 --2- 172.21.15.7:0/3485650557 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3420066fe0 0x7f3420069490 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3438001200 tx=0x7f3438009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:50.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.172+0000 7f3431ffb700 1 -- 172.21.15.7:0/3485650557 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3434018060 con 0x7f343c16a390 2024-01-27T02:36:50.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.421+0000 7f3443ae4700 1 -- 172.21.15.7:0/3485650557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f343c04c710 con 0x7f343c16a390 2024-01-27T02:36:50.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.421+0000 7f3431ffb700 1 -- 172.21.15.7:0/3485650557 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4590 (secure 0 0 0) 0x7f343402e030 con 0x7f343c16a390 2024-01-27T02:36:50.422 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:36:50.422 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5839,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/100533817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":100533817},{"type":"v1","addr":"172.21.15.7:6834","nonce":100533817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5842,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/223425097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":223425097},{"type":"v1","addr":"172.21.15.7:6835","nonce":223425097}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5848,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/950040140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":950040140},{"type":"v1","addr":"172.21.15.188:6837","nonce":950040140}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:36:01.502818+0000","modified":"2024-01-27T02:36:05.109352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5946":{"gid":5946,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6836/2269430612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2269430612},{"type":"v1","addr":"172.21.15.188:6836","nonce":2269430612}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2024-01-27T02:36:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.423+0000 7f3443ae4700 1 -- 172.21.15.7:0/3485650557 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3420066fe0 msgr2=0x7f3420069490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.423+0000 7f3443ae4700 1 --2- 172.21.15.7:0/3485650557 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3420066fe0 0x7f3420069490 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3438001200 tx=0x7f3438009040 comp rx=0 tx=0).stop 2024-01-27T02:36:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.423+0000 7f3443ae4700 1 -- 172.21.15.7:0/3485650557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343c16a390 msgr2=0x7f343c16a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.423+0000 7f3443ae4700 1 --2- 172.21.15.7:0/3485650557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343c16a390 0x7f343c16a830 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f343400afd0 tx=0x7f343401a000 comp rx=0 tx=0).stop 2024-01-27T02:36:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.423+0000 7f3443ae4700 1 -- 172.21.15.7:0/3485650557 shutdown_connections 2024-01-27T02:36:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.423+0000 7f3443ae4700 1 --2- 172.21.15.7:0/3485650557 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3420066fe0 0x7f3420069490 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.423+0000 7f3443ae4700 1 --2- 172.21.15.7:0/3485650557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343c16a390 0x7f343c16a830 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.423+0000 7f3443ae4700 1 --2- 172.21.15.7:0/3485650557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3428004d50 0x7f343c0c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.423+0000 7f3443ae4700 1 --2- 172.21.15.7:0/3485650557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f343c0c76e0 0x7f343c169e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.423+0000 7f3443ae4700 1 -- 172.21.15.7:0/3485650557 >> 172.21.15.7:0/3485650557 conn(0x7f343c098020 msgr2=0x7f343c0b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.424+0000 7f3443ae4700 1 -- 172.21.15.7:0/3485650557 shutdown_connections 2024-01-27T02:36:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.424+0000 7f3443ae4700 1 -- 172.21.15.7:0/3485650557 wait complete. 2024-01-27T02:36:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 42 2024-01-27T02:36:50.438 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:36:50.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.569+0000 7fc416e07700 1 Processor -- start 2024-01-27T02:36:50.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.569+0000 7fc416e07700 1 -- start start 2024-01-27T02:36:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.570+0000 7fc416e07700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc4100c4cc0 0x7fc4100cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.570+0000 7fc416e07700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc4100ca540 0x7fc410163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.570+0000 7fc416e07700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4100ce5e0 con 0x7fc41015a100 2024-01-27T02:36:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.570+0000 7fc416e07700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4100cd9d0 con 0x7fc4100ca540 2024-01-27T02:36:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.570+0000 7fc416e07700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4100cdcb0 con 0x7fc4100c4cc0 2024-01-27T02:36:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.570+0000 7fc4153a4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc4100ca540 0x7fc410163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.570+0000 7fc4153a4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc4100ca540 0x7fc410163d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43010/0 (socket says 172.21.15.7:43010) 2024-01-27T02:36:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.570+0000 7fc4153a4700 1 -- 172.21.15.7:0/2667055724 learned_addr learned my addr 172.21.15.7:0/2667055724 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.570+0000 7fc40ffff700 1 --2- 172.21.15.7:0/2667055724 >> v2:172.21.15.188:3300/0 conn(0x7fc4100c4cc0 0x7fc4100cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.570+0000 7fc4153a4700 1 -- 172.21.15.7:0/2667055724 >> v2:172.21.15.188:3300/0 conn(0x7fc4100c4cc0 msgr2=0x7fc4100cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.570+0000 7fc4153a4700 1 --2- 172.21.15.7:0/2667055724 >> v2:172.21.15.188:3300/0 conn(0x7fc4100c4cc0 0x7fc4100cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.570+0000 7fc4153a4700 1 -- 172.21.15.7:0/2667055724 >> v1:172.21.15.188:6789/0 conn(0x7fc41015a100 legacy=0x7fc4100c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.570+0000 7fc4153a4700 1 -- 172.21.15.7:0/2667055724 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4100cdf90 con 0x7fc4100ca540 2024-01-27T02:36:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.570+0000 7fc4153a4700 1 --2- 172.21.15.7:0/2667055724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4100ca540 0x7fc410163d90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc41006e170 tx=0x7fc408009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92101a7a766e1c6 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.571+0000 7fc40f7fe700 1 -- 172.21.15.7:0/2667055724 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40800e070 con 0x7fc4100ca540 2024-01-27T02:36:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.571+0000 7fc40f7fe700 1 -- 172.21.15.7:0/2667055724 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40800ba10 con 0x7fc4100ca540 2024-01-27T02:36:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.571+0000 7fc40f7fe700 1 -- 172.21.15.7:0/2667055724 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4080092e0 con 0x7fc4100ca540 2024-01-27T02:36:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.571+0000 7fc416e07700 1 -- 172.21.15.7:0/2667055724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4100ca540 msgr2=0x7fc410163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.571+0000 7fc416e07700 1 --2- 172.21.15.7:0/2667055724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4100ca540 0x7fc410163d90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc41006e170 tx=0x7fc408009840 comp rx=0 tx=0).stop 2024-01-27T02:36:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.571+0000 7fc416e07700 1 -- 172.21.15.7:0/2667055724 shutdown_connections 2024-01-27T02:36:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.571+0000 7fc416e07700 1 --2- 172.21.15.7:0/2667055724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4100ca540 0x7fc410163d90 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.571+0000 7fc416e07700 1 --2- 172.21.15.7:0/2667055724 >> v2:172.21.15.188:3300/0 conn(0x7fc4100c4cc0 0x7fc4100cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.571+0000 7fc416e07700 1 -- 172.21.15.7:0/2667055724 >> 172.21.15.7:0/2667055724 conn(0x7fc410098020 msgr2=0x7fc4100711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.571+0000 7fc416e07700 1 -- 172.21.15.7:0/2667055724 shutdown_connections 2024-01-27T02:36:50.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.571+0000 7fc416e07700 1 -- 172.21.15.7:0/2667055724 wait complete. 2024-01-27T02:36:50.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.572+0000 7fc416e07700 1 Processor -- start 2024-01-27T02:36:50.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.573+0000 7fc416e07700 1 -- start start 2024-01-27T02:36:50.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.573+0000 7fc416e07700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4100c4cc0 0x7fc4100b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.573+0000 7fc416e07700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4100ca540 0x7fc41014b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.573+0000 7fc416e07700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc41015a100 0x7fc41016b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.573+0000 7fc416e07700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41016e9b0 con 0x7fc4100c4cc0 2024-01-27T02:36:50.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.573+0000 7fc416e07700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41014baa0 con 0x7fc41015a100 2024-01-27T02:36:50.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.573+0000 7fc416e07700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41016dda0 con 0x7fc4100ca540 2024-01-27T02:36:50.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.574+0000 7fc414ba3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4100c4cc0 0x7fc4100b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.574+0000 7fc414ba3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4100c4cc0 0x7fc4100b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43018/0 (socket says 172.21.15.7:43018) 2024-01-27T02:36:50.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.574+0000 7fc414ba3700 1 -- 172.21.15.7:0/2338769451 learned_addr learned my addr 172.21.15.7:0/2338769451 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:50.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.574+0000 7fc40ffff700 1 --2- 172.21.15.7:0/2338769451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4100ca540 0x7fc41014b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.574+0000 7fc4153a4700 1 --2- 172.21.15.7:0/2338769451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc41015a100 0x7fc41016b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.574+0000 7fc414ba3700 1 -- 172.21.15.7:0/2338769451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4100ca540 msgr2=0x7fc41014b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:50.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.574+0000 7fc414ba3700 1 --2- 172.21.15.7:0/2338769451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4100ca540 0x7fc41014b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.574+0000 7fc414ba3700 1 -- 172.21.15.7:0/2338769451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc41015a100 msgr2=0x7fc41016b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:50.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.574+0000 7fc414ba3700 1 --2- 172.21.15.7:0/2338769451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc41015a100 0x7fc41016b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.574+0000 7fc414ba3700 1 -- 172.21.15.7:0/2338769451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc408009ae0 con 0x7fc4100c4cc0 2024-01-27T02:36:50.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.574+0000 7fc414ba3700 1 --2- 172.21.15.7:0/2338769451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4100c4cc0 0x7fc4100b1bc0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc40400b5e0 tx=0x7fc404009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:50.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.574+0000 7fc40cff9700 1 -- 172.21.15.7:0/2338769451 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40400f510 con 0x7fc4100c4cc0 2024-01-27T02:36:50.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.574+0000 7fc40cff9700 1 -- 172.21.15.7:0/2338769451 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40400f670 con 0x7fc4100c4cc0 2024-01-27T02:36:50.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.574+0000 7fc416e07700 1 -- 172.21.15.7:0/2338769451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41016e080 con 0x7fc4100c4cc0 2024-01-27T02:36:50.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.574+0000 7fc40cff9700 1 -- 172.21.15.7:0/2338769451 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40400fa30 con 0x7fc4100c4cc0 2024-01-27T02:36:50.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.574+0000 7fc416e07700 1 -- 172.21.15.7:0/2338769451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41016e640 con 0x7fc4100c4cc0 2024-01-27T02:36:50.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.575+0000 7fc416e07700 1 -- 172.21.15.7:0/2338769451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc41004c710 con 0x7fc4100c4cc0 2024-01-27T02:36:50.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.576+0000 7fc40cff9700 1 -- 172.21.15.7:0/2338769451 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc40401a400 con 0x7fc4100c4cc0 2024-01-27T02:36:50.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.576+0000 7fc40cff9700 1 --2- 172.21.15.7:0/2338769451 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc3f4066f30 0x7fc3f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.576+0000 7fc40cff9700 1 -- 172.21.15.7:0/2338769451 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc404083fd0 con 0x7fc4100c4cc0 2024-01-27T02:36:50.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.576+0000 7fc40cff9700 1 -- 172.21.15.7:0/2338769451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=85}) v3 -- 0x7fc3f406ea50 con 0x7fc4100c4cc0 2024-01-27T02:36:50.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.576+0000 7fc40ffff700 1 --2- 172.21.15.7:0/2338769451 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc3f4066f30 0x7fc3f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.577+0000 7fc40ffff700 1 --2- 172.21.15.7:0/2338769451 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc3f4066f30 0x7fc3f40693e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc006fd0 tx=0x7fc3fc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.581+0000 7fc40cff9700 1 -- 172.21.15.7:0/2338769451 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc4040512c0 con 0x7fc4100c4cc0 2024-01-27T02:36:50.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.786+0000 7fc416e07700 1 -- 172.21.15.7:0/2338769451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4100ce270 con 0x7fc4100c4cc0 2024-01-27T02:36:50.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.787+0000 7fc40cff9700 1 -- 172.21.15.7:0/2338769451 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+15348 (secure 0 0 0) 0x7fc4040548e0 con 0x7fc4100c4cc0 2024-01-27T02:36:50.787 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:36:50.788 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:36:22.866314+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:35:59.537952+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-27T02:36:00.480234+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:36:50.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.789+0000 7fc416e07700 1 -- 172.21.15.7:0/2338769451 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc3f4066f30 msgr2=0x7fc3f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.789+0000 7fc416e07700 1 --2- 172.21.15.7:0/2338769451 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc3f4066f30 0x7fc3f40693e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc006fd0 tx=0x7fc3fc008040 comp rx=0 tx=0).stop 2024-01-27T02:36:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.789+0000 7fc416e07700 1 -- 172.21.15.7:0/2338769451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4100c4cc0 msgr2=0x7fc4100b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.789+0000 7fc416e07700 1 --2- 172.21.15.7:0/2338769451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4100c4cc0 0x7fc4100b1bc0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc40400b5e0 tx=0x7fc404009120 comp rx=0 tx=0).stop 2024-01-27T02:36:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.789+0000 7fc416e07700 1 -- 172.21.15.7:0/2338769451 shutdown_connections 2024-01-27T02:36:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.789+0000 7fc416e07700 1 --2- 172.21.15.7:0/2338769451 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc3f4066f30 0x7fc3f40693e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.789+0000 7fc416e07700 1 --2- 172.21.15.7:0/2338769451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc41015a100 0x7fc41016b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.789+0000 7fc416e07700 1 --2- 172.21.15.7:0/2338769451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4100c4cc0 0x7fc4100b1bc0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.789+0000 7fc416e07700 1 --2- 172.21.15.7:0/2338769451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4100ca540 0x7fc41014b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.789+0000 7fc416e07700 1 -- 172.21.15.7:0/2338769451 >> 172.21.15.7:0/2338769451 conn(0x7fc410098020 msgr2=0x7fc410154c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.789+0000 7fc416e07700 1 -- 172.21.15.7:0/2338769451 shutdown_connections 2024-01-27T02:36:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.789+0000 7fc416e07700 1 -- 172.21.15.7:0/2338769451 wait complete. 2024-01-27T02:36:50.803 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-27T02:36:50.804 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:36:50.921 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-27T02:36:50.921+0000 7fb5c783f700 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2024-01-27T02:36:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.938+0000 7f482a7b2700 1 Processor -- start 2024-01-27T02:36:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.938+0000 7f482a7b2700 1 -- start start 2024-01-27T02:36:50.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.939+0000 7f482a7b2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4824143520 0x7f4824063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.939+0000 7f482a7b2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48240cb7a0 con 0x7f48241438f0 2024-01-27T02:36:50.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.939+0000 7f482a7b2700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48240cb9f0 con 0x7f48240aa8b0 2024-01-27T02:36:50.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.939+0000 7f482a7b2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48240cbcd0 con 0x7f4824143520 2024-01-27T02:36:50.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.939+0000 7f48237fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f48241438f0 0x7f48240c8210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54928/0 (socket says 172.21.15.7:54928) 2024-01-27T02:36:50.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.939+0000 7f48237fe700 1 -- 172.21.15.7:0/1181691166 learned_addr learned my addr 172.21.15.7:0/1181691166 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:50.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.939+0000 7f4823fff700 1 --2- 172.21.15.7:0/1181691166 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4824143520 0x7f4824063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.939+0000 7f4822ffd700 1 -- 172.21.15.7:0/1181691166 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3052496793 0 0) 0x7f48240cb7a0 con 0x7f48241438f0 2024-01-27T02:36:50.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.939+0000 7f4822ffd700 1 -- 172.21.15.7:0/1181691166 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f480c003570 con 0x7f48241438f0 2024-01-27T02:36:50.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f4822ffd700 1 -- 172.21.15.7:0/1181691166 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2660983735 0 0) 0x7f480c003570 con 0x7f48241438f0 2024-01-27T02:36:50.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f4822ffd700 1 -- 172.21.15.7:0/1181691166 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4824143520 msgr2=0x7f4824063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:50.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f4822ffd700 1 --2- 172.21.15.7:0/1181691166 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4824143520 0x7f4824063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f4822ffd700 1 -- 172.21.15.7:0/1181691166 >> v1:172.21.15.188:6789/0 conn(0x7f48240aa8b0 legacy=0x7f482406e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:50.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f4822ffd700 1 -- 172.21.15.7:0/1181691166 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48240cbfb0 con 0x7f48241438f0 2024-01-27T02:36:50.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f4822ffd700 1 -- 172.21.15.7:0/1181691166 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f4818001470 con 0x7f48241438f0 2024-01-27T02:36:50.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f4823fff700 1 --2- 172.21.15.7:0/1181691166 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4824143520 0x7f4824063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:36:50.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f4822ffd700 1 -- 172.21.15.7:0/1181691166 >> v1:172.21.15.7:6789/0 conn(0x7f48241438f0 legacy=0x7f48240c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:50.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f4822ffd700 1 --2- 172.21.15.7:0/1181691166 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4824143520 0x7f480c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f4822ffd700 1 --2- 172.21.15.7:0/1181691166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f480c003fd0 0x7f480c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f4822ffd700 1 --2- 172.21.15.7:0/1181691166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f480c004d50 0x7f480c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f4822ffd700 1 -- 172.21.15.7:0/1181691166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48240cbcd0 con 0x7f480c004d50 2024-01-27T02:36:50.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f4822ffd700 1 -- 172.21.15.7:0/1181691166 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480c00bc20 con 0x7f480c003fd0 2024-01-27T02:36:50.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f4822ffd700 1 -- 172.21.15.7:0/1181691166 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480c00bde0 con 0x7f4824143520 2024-01-27T02:36:50.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f4828d4f700 1 --2- 172.21.15.7:0/1181691166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f480c004d50 0x7f480c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f48237fe700 1 --2- 172.21.15.7:0/1181691166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f480c003fd0 0x7f480c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f4823fff700 1 --2- 172.21.15.7:0/1181691166 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4824143520 0x7f480c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f4828d4f700 1 -- 172.21.15.7:0/1181691166 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4824143520 msgr2=0x7f480c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:50.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f4828d4f700 1 --2- 172.21.15.7:0/1181691166 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4824143520 0x7f480c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f4828d4f700 1 -- 172.21.15.7:0/1181691166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f480c003fd0 msgr2=0x7f480c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:50.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f4828d4f700 1 --2- 172.21.15.7:0/1181691166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f480c003fd0 0x7f480c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f4828d4f700 1 -- 172.21.15.7:0/1181691166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48140093c0 con 0x7f480c004d50 2024-01-27T02:36:50.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.940+0000 7f4828d4f700 1 --2- 172.21.15.7:0/1181691166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f480c004d50 0x7f480c0051c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f481c001fc0 tx=0x7f481c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481de84e2686611e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:50.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.941+0000 7f4822ffd700 1 -- 172.21.15.7:0/1181691166 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c001950 con 0x7f480c004d50 2024-01-27T02:36:50.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.941+0000 7f4822ffd700 1 -- 172.21.15.7:0/1181691166 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481c00c8b0 con 0x7f480c004d50 2024-01-27T02:36:50.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.941+0000 7f4822ffd700 1 -- 172.21.15.7:0/1181691166 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c00cc50 con 0x7f480c004d50 2024-01-27T02:36:50.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.941+0000 7f482a7b2700 1 -- 172.21.15.7:0/1181691166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f480c004d50 msgr2=0x7f480c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.941+0000 7f482a7b2700 1 --2- 172.21.15.7:0/1181691166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f480c004d50 0x7f480c0051c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f481c001fc0 tx=0x7f481c001640 comp rx=0 tx=0).stop 2024-01-27T02:36:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.941+0000 7f4828d4f700 1 -- 172.21.15.7:0/1181691166 reap_dead start 2024-01-27T02:36:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.941+0000 7f482a7b2700 1 -- 172.21.15.7:0/1181691166 shutdown_connections 2024-01-27T02:36:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.941+0000 7f482a7b2700 1 -- 172.21.15.7:0/1181691166 >> 172.21.15.7:0/1181691166 conn(0x7f48240cdbd0 msgr2=0x7f48240cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.941+0000 7f482a7b2700 1 -- 172.21.15.7:0/1181691166 shutdown_connections 2024-01-27T02:36:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.941+0000 7f482a7b2700 1 -- 172.21.15.7:0/1181691166 wait complete. 2024-01-27T02:36:50.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.943+0000 7f482a7b2700 1 Processor -- start 2024-01-27T02:36:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.943+0000 7f482a7b2700 1 -- start start 2024-01-27T02:36:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.944+0000 7f482a7b2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f482415eaf0 0x7f48240b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.944+0000 7f482a7b2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482415ffa0 0x7f4824160410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.944+0000 7f482a7b2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f482406b540 0x7f482406b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.944+0000 7f482a7b2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4824160950 con 0x7f482406b540 2024-01-27T02:36:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.944+0000 7f482a7b2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482415f880 con 0x7f482415ffa0 2024-01-27T02:36:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.944+0000 7f482a7b2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4824140ae0 con 0x7f482415eaf0 2024-01-27T02:36:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.944+0000 7f4828d4f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f482406b540 0x7f482406b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.944+0000 7f4828d4f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f482406b540 0x7f482406b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43036/0 (socket says 172.21.15.7:43036) 2024-01-27T02:36:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.944+0000 7f4828d4f700 1 -- 172.21.15.7:0/1523089574 learned_addr learned my addr 172.21.15.7:0/1523089574 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.944+0000 7f4823fff700 1 --2- 172.21.15.7:0/1523089574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f482415eaf0 0x7f48240b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.944+0000 7f48237fe700 1 --2- 172.21.15.7:0/1523089574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482415ffa0 0x7f4824160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.945+0000 7f4828d4f700 1 -- 172.21.15.7:0/1523089574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f482415eaf0 msgr2=0x7f48240b09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.945+0000 7f4828d4f700 1 --2- 172.21.15.7:0/1523089574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f482415eaf0 0x7f48240b09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.945+0000 7f4828d4f700 1 -- 172.21.15.7:0/1523089574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482415ffa0 msgr2=0x7f4824160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.945+0000 7f4828d4f700 1 --2- 172.21.15.7:0/1523089574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482415ffa0 0x7f4824160410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.945+0000 7f4828d4f700 1 -- 172.21.15.7:0/1523089574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482415f2c0 con 0x7f482406b540 2024-01-27T02:36:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.945+0000 7f48237fe700 1 --2- 172.21.15.7:0/1523089574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482415ffa0 0x7f4824160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:36:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.945+0000 7f4828d4f700 1 --2- 172.21.15.7:0/1523089574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f482406b540 0x7f482406b9c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f480c00bbf0 tx=0x7f481c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.945+0000 7f480bfff700 1 -- 172.21.15.7:0/1523089574 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c00c6d0 con 0x7f482406b540 2024-01-27T02:36:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.945+0000 7f480bfff700 1 -- 172.21.15.7:0/1523089574 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481c012e80 con 0x7f482406b540 2024-01-27T02:36:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.945+0000 7f480bfff700 1 -- 172.21.15.7:0/1523089574 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c006df0 con 0x7f482406b540 2024-01-27T02:36:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.945+0000 7f482a7b2700 1 -- 172.21.15.7:0/1523089574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f482415f5a0 con 0x7f482406b540 2024-01-27T02:36:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.945+0000 7f482a7b2700 1 -- 172.21.15.7:0/1523089574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48240b17c0 con 0x7f482406b540 2024-01-27T02:36:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.946+0000 7f482a7b2700 1 -- 172.21.15.7:0/1523089574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f482404c710 con 0x7f482406b540 2024-01-27T02:36:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.947+0000 7f480bfff700 1 -- 172.21.15.7:0/1523089574 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f481c019070 con 0x7f482406b540 2024-01-27T02:36:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.947+0000 7f480bfff700 1 --2- 172.21.15.7:0/1523089574 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4810066f30 0x7f48100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.947+0000 7f480bfff700 1 -- 172.21.15.7:0/1523089574 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f481c085cc0 con 0x7f482406b540 2024-01-27T02:36:50.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.947+0000 7f480bfff700 1 -- 172.21.15.7:0/1523089574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=86}) v3 -- 0x7f481006ea50 con 0x7f482406b540 2024-01-27T02:36:50.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.947+0000 7f4823fff700 1 --2- 172.21.15.7:0/1523089574 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4810066f30 0x7f48100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:50.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.948+0000 7f4823fff700 1 --2- 172.21.15.7:0/1523089574 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4810066f30 0x7f48100693e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4814006ea0 tx=0x7f4814009120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:50.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:50.952+0000 7f480bfff700 1 -- 172.21.15.7:0/1523089574 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f481c017030 con 0x7f482406b540 2024-01-27T02:36:51.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.203+0000 7f482a7b2700 1 -- 172.21.15.7:0/1523089574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48240cc290 con 0x7f482406b540 2024-01-27T02:36:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.203+0000 7f480bfff700 1 -- 172.21.15.7:0/1523089574 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4590 (secure 0 0 0) 0x7f481c017030 con 0x7f482406b540 2024-01-27T02:36:51.204 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:36:51.204 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5839,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/100533817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":100533817},{"type":"v1","addr":"172.21.15.7:6834","nonce":100533817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5842,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/223425097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":223425097},{"type":"v1","addr":"172.21.15.7:6835","nonce":223425097}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5848,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/950040140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":950040140},{"type":"v1","addr":"172.21.15.188:6837","nonce":950040140}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:36:01.502818+0000","modified":"2024-01-27T02:36:05.109352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5946":{"gid":5946,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6836/2269430612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2269430612},{"type":"v1","addr":"172.21.15.188:6836","nonce":2269430612}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2024-01-27T02:36:51.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.205+0000 7f482a7b2700 1 -- 172.21.15.7:0/1523089574 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4810066f30 msgr2=0x7f48100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:51.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.205+0000 7f482a7b2700 1 --2- 172.21.15.7:0/1523089574 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4810066f30 0x7f48100693e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4814006ea0 tx=0x7f4814009120 comp rx=0 tx=0).stop 2024-01-27T02:36:51.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.205+0000 7f482a7b2700 1 -- 172.21.15.7:0/1523089574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f482406b540 msgr2=0x7f482406b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:51.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.205+0000 7f482a7b2700 1 --2- 172.21.15.7:0/1523089574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f482406b540 0x7f482406b9c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f480c00bbf0 tx=0x7f481c015000 comp rx=0 tx=0).stop 2024-01-27T02:36:51.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.205+0000 7f482a7b2700 1 -- 172.21.15.7:0/1523089574 shutdown_connections 2024-01-27T02:36:51.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.205+0000 7f482a7b2700 1 --2- 172.21.15.7:0/1523089574 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4810066f30 0x7f48100693e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:51.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.205+0000 7f482a7b2700 1 --2- 172.21.15.7:0/1523089574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f482406b540 0x7f482406b9c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:51.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.205+0000 7f482a7b2700 1 --2- 172.21.15.7:0/1523089574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f482415eaf0 0x7f48240b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:51.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.205+0000 7f482a7b2700 1 --2- 172.21.15.7:0/1523089574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482415ffa0 0x7f4824160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:51.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.205+0000 7f482a7b2700 1 -- 172.21.15.7:0/1523089574 >> 172.21.15.7:0/1523089574 conn(0x7f48240cdbd0 msgr2=0x7f4824063580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:51.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.206+0000 7f482a7b2700 1 -- 172.21.15.7:0/1523089574 shutdown_connections 2024-01-27T02:36:51.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.206+0000 7f482a7b2700 1 -- 172.21.15.7:0/1523089574 wait complete. 2024-01-27T02:36:51.207 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 42 2024-01-27T02:36:51.220 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:36:51.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.351+0000 7f8d4f52e700 1 Processor -- start 2024-01-27T02:36:51.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.351+0000 7f8d4f52e700 1 -- start start 2024-01-27T02:36:51.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.352+0000 7f8d4f52e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8d480c3ae0 0x7f8d480cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:51.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.352+0000 7f8d4f52e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d48158070 0x7f8d480cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:51.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.352+0000 7f8d4f52e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d480ca020 con 0x7f8d48157ca0 2024-01-27T02:36:51.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.352+0000 7f8d4f52e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d480c5210 con 0x7f8d480c3ae0 2024-01-27T02:36:51.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.352+0000 7f8d4f52e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d480c54f0 con 0x7f8d48158070 2024-01-27T02:36:51.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.352+0000 7f8d4cac9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8d480c3ae0 0x7f8d480cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:51.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.352+0000 7f8d4d2ca700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8d48157ca0 0x7f8d480c4a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54938/0 (socket says 172.21.15.7:54938) 2024-01-27T02:36:51.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.352+0000 7f8d4d2ca700 1 -- 172.21.15.7:0/4054012326 learned_addr learned my addr 172.21.15.7:0/4054012326 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:51.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.352+0000 7f8d4dacb700 1 --2- 172.21.15.7:0/4054012326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d48158070 0x7f8d480cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:51.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.352+0000 7f8d4cac9700 1 -- 172.21.15.7:0/4054012326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d48158070 msgr2=0x7f8d480cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:51.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.352+0000 7f8d4cac9700 1 --2- 172.21.15.7:0/4054012326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d48158070 0x7f8d480cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:51.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.352+0000 7f8d4cac9700 1 -- 172.21.15.7:0/4054012326 >> v1:172.21.15.7:6789/0 conn(0x7f8d48157ca0 legacy=0x7f8d480c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:51.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.352+0000 7f8d4cac9700 1 -- 172.21.15.7:0/4054012326 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d480c57d0 con 0x7f8d480c3ae0 2024-01-27T02:36:51.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.352+0000 7f8d3ffff700 1 -- 172.21.15.7:0/4054012326 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331223431 0 0) 0x7f8d480ca020 con 0x7f8d48157ca0 2024-01-27T02:36:51.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.352+0000 7f8d4cac9700 1 --2- 172.21.15.7:0/4054012326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d480c3ae0 0x7f8d480cd4e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f8d380098e0 tx=0x7f8d380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1148293eed7e8f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:51.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.353+0000 7f8d3ffff700 1 -- 172.21.15.7:0/4054012326 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38009e80 con 0x7f8d480c3ae0 2024-01-27T02:36:51.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.353+0000 7f8d3ffff700 1 -- 172.21.15.7:0/4054012326 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3800b860 con 0x7f8d480c3ae0 2024-01-27T02:36:51.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.353+0000 7f8d3ffff700 1 -- 172.21.15.7:0/4054012326 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3800bbe0 con 0x7f8d480c3ae0 2024-01-27T02:36:51.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.353+0000 7f8d4f52e700 1 -- 172.21.15.7:0/4054012326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d480c3ae0 msgr2=0x7f8d480cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.353+0000 7f8d4f52e700 1 --2- 172.21.15.7:0/4054012326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d480c3ae0 0x7f8d480cd4e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f8d380098e0 tx=0x7f8d380091a0 comp rx=0 tx=0).stop 2024-01-27T02:36:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.353+0000 7f8d4f52e700 1 -- 172.21.15.7:0/4054012326 shutdown_connections 2024-01-27T02:36:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.353+0000 7f8d4f52e700 1 --2- 172.21.15.7:0/4054012326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d48158070 0x7f8d480cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.353+0000 7f8d4f52e700 1 --2- 172.21.15.7:0/4054012326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d480c3ae0 0x7f8d480cd4e0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.353+0000 7f8d4f52e700 1 -- 172.21.15.7:0/4054012326 >> 172.21.15.7:0/4054012326 conn(0x7f8d48098020 msgr2=0x7f8d480711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.353+0000 7f8d4f52e700 1 -- 172.21.15.7:0/4054012326 shutdown_connections 2024-01-27T02:36:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.353+0000 7f8d4f52e700 1 -- 172.21.15.7:0/4054012326 wait complete. 2024-01-27T02:36:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.355+0000 7f8d4f52e700 1 Processor -- start 2024-01-27T02:36:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.355+0000 7f8d4f52e700 1 -- start start 2024-01-27T02:36:51.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.356+0000 7f8d4f52e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d480c3ae0 0x7f8d48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:51.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.356+0000 7f8d4f52e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d48157ca0 0x7f8d48152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:51.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.356+0000 7f8d4f52e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d48158070 0x7f8d480b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:51.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.356+0000 7f8d4f52e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d480c7050 con 0x7f8d48157ca0 2024-01-27T02:36:51.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.356+0000 7f8d4f52e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d48069960 con 0x7f8d480c3ae0 2024-01-27T02:36:51.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.356+0000 7f8d4f52e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d480c67b0 con 0x7f8d48158070 2024-01-27T02:36:51.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.356+0000 7f8d4cac9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d48157ca0 0x7f8d48152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:51.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.356+0000 7f8d4cac9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d48157ca0 0x7f8d48152420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43058/0 (socket says 172.21.15.7:43058) 2024-01-27T02:36:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.356+0000 7f8d4cac9700 1 -- 172.21.15.7:0/3131476850 learned_addr learned my addr 172.21.15.7:0/3131476850 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.356+0000 7f8d4dacb700 1 --2- 172.21.15.7:0/3131476850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d48158070 0x7f8d480b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.356+0000 7f8d4d2ca700 1 --2- 172.21.15.7:0/3131476850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d480c3ae0 0x7f8d48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.356+0000 7f8d4cac9700 1 -- 172.21.15.7:0/3131476850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d48158070 msgr2=0x7f8d480b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.356+0000 7f8d4cac9700 1 --2- 172.21.15.7:0/3131476850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d48158070 0x7f8d480b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.356+0000 7f8d4cac9700 1 -- 172.21.15.7:0/3131476850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d480c3ae0 msgr2=0x7f8d48069420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.356+0000 7f8d4cac9700 1 --2- 172.21.15.7:0/3131476850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d480c3ae0 0x7f8d48069420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.356+0000 7f8d4cac9700 1 -- 172.21.15.7:0/3131476850 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d38009420 con 0x7f8d48157ca0 2024-01-27T02:36:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.356+0000 7f8d4d2ca700 1 --2- 172.21.15.7:0/3131476850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d480c3ae0 0x7f8d48069420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:36:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.356+0000 7f8d4cac9700 1 --2- 172.21.15.7:0/3131476850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d48157ca0 0x7f8d48152420 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8d4806ea80 tx=0x7f8d38017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.357+0000 7f8d3d7fa700 1 -- 172.21.15.7:0/3131476850 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3800b9d0 con 0x7f8d48157ca0 2024-01-27T02:36:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.357+0000 7f8d3d7fa700 1 -- 172.21.15.7:0/3131476850 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3800bb30 con 0x7f8d48157ca0 2024-01-27T02:36:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.357+0000 7f8d4f52e700 1 -- 172.21.15.7:0/3131476850 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d480c6a90 con 0x7f8d48157ca0 2024-01-27T02:36:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.357+0000 7f8d3d7fa700 1 -- 172.21.15.7:0/3131476850 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38025670 con 0x7f8d48157ca0 2024-01-27T02:36:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.357+0000 7f8d4f52e700 1 -- 172.21.15.7:0/3131476850 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d48067b50 con 0x7f8d48157ca0 2024-01-27T02:36:51.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.358+0000 7f8d4f52e700 1 -- 172.21.15.7:0/3131476850 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d480679b0 con 0x7f8d48157ca0 2024-01-27T02:36:51.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.358+0000 7f8d3d7fa700 1 -- 172.21.15.7:0/3131476850 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8d38023070 con 0x7f8d48157ca0 2024-01-27T02:36:51.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.358+0000 7f8d3d7fa700 1 --2- 172.21.15.7:0/3131476850 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8d2c066f30 0x7f8d2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.359+0000 7f8d3d7fa700 1 -- 172.21.15.7:0/3131476850 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8d38085b20 con 0x7f8d48157ca0 2024-01-27T02:36:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.359+0000 7f8d3d7fa700 1 -- 172.21.15.7:0/3131476850 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=86}) v3 -- 0x7f8d2c06ea50 con 0x7f8d48157ca0 2024-01-27T02:36:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.359+0000 7f8d4d2ca700 1 --2- 172.21.15.7:0/3131476850 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8d2c066f30 0x7f8d2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.359+0000 7f8d4d2ca700 1 --2- 172.21.15.7:0/3131476850 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8d2c066f30 0x7f8d2c0693e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8d440011e0 tx=0x7f8d44009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:51.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.363+0000 7f8d3d7fa700 1 -- 172.21.15.7:0/3131476850 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d480679b0 con 0x7f8d48157ca0 2024-01-27T02:36:51.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.614+0000 7f8d4f52e700 1 -- 172.21.15.7:0/3131476850 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d480679b0 con 0x7f8d48157ca0 2024-01-27T02:36:51.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.615+0000 7f8d3d7fa700 1 -- 172.21.15.7:0/3131476850 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4590 (secure 0 0 0) 0x7f8d480679b0 con 0x7f8d48157ca0 2024-01-27T02:36:51.616 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:36:51.616 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5839,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/100533817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":100533817},{"type":"v1","addr":"172.21.15.7:6834","nonce":100533817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5842,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/223425097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":223425097},{"type":"v1","addr":"172.21.15.7:6835","nonce":223425097}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5848,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/950040140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":950040140},{"type":"v1","addr":"172.21.15.188:6837","nonce":950040140}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:36:01.502818+0000","modified":"2024-01-27T02:36:05.109352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5946":{"gid":5946,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6836/2269430612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2269430612},{"type":"v1","addr":"172.21.15.188:6836","nonce":2269430612}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2024-01-27T02:36:51.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.617+0000 7f8d4f52e700 1 -- 172.21.15.7:0/3131476850 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8d2c066f30 msgr2=0x7f8d2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.617+0000 7f8d4f52e700 1 --2- 172.21.15.7:0/3131476850 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8d2c066f30 0x7f8d2c0693e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8d440011e0 tx=0x7f8d44009040 comp rx=0 tx=0).stop 2024-01-27T02:36:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.617+0000 7f8d4f52e700 1 -- 172.21.15.7:0/3131476850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d48157ca0 msgr2=0x7f8d48152420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.617+0000 7f8d4f52e700 1 --2- 172.21.15.7:0/3131476850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d48157ca0 0x7f8d48152420 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8d4806ea80 tx=0x7f8d38017000 comp rx=0 tx=0).stop 2024-01-27T02:36:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.617+0000 7f8d4f52e700 1 -- 172.21.15.7:0/3131476850 shutdown_connections 2024-01-27T02:36:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.617+0000 7f8d4f52e700 1 --2- 172.21.15.7:0/3131476850 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8d2c066f30 0x7f8d2c0693e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.617+0000 7f8d4f52e700 1 --2- 172.21.15.7:0/3131476850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d48158070 0x7f8d480b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.617+0000 7f8d4f52e700 1 --2- 172.21.15.7:0/3131476850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d480c3ae0 0x7f8d48069420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.617+0000 7f8d4f52e700 1 --2- 172.21.15.7:0/3131476850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d48157ca0 0x7f8d48152420 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.617+0000 7f8d4f52e700 1 -- 172.21.15.7:0/3131476850 >> 172.21.15.7:0/3131476850 conn(0x7f8d48098020 msgr2=0x7f8d4806b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.617+0000 7f8d4f52e700 1 -- 172.21.15.7:0/3131476850 shutdown_connections 2024-01-27T02:36:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.617+0000 7f8d4f52e700 1 -- 172.21.15.7:0/3131476850 wait complete. 2024-01-27T02:36:51.619 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 42 2024-01-27T02:36:51.631 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:36:51.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.762+0000 7fd198550700 1 Processor -- start 2024-01-27T02:36:51.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.762+0000 7fd198550700 1 -- start start 2024-01-27T02:36:51.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.763+0000 7fd198550700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd190157ca0 0x7fd1900c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:51.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.763+0000 7fd198550700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1900ca020 con 0x7fd1900c3ae0 2024-01-27T02:36:51.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.763+0000 7fd198550700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1900c4f40 con 0x7fd1900cdbf0 2024-01-27T02:36:51.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.763+0000 7fd198550700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1900c5210 con 0x7fd190157ca0 2024-01-27T02:36:51.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.763+0000 7fd1962ec700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd190157ca0 0x7fd1900c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:51.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.763+0000 7fd195aeb700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd1900c3ae0 0x7fd1900cd4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60754/0 (socket says 172.21.15.7:60754) 2024-01-27T02:36:51.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.763+0000 7fd195aeb700 1 -- 172.21.15.7:0/65959833 learned_addr learned my addr 172.21.15.7:0/65959833 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:51.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.764+0000 7fd1962ec700 1 -- 172.21.15.7:0/65959833 >> v1:172.21.15.188:6789/0 conn(0x7fd1900cdbf0 legacy=0x7fd1900ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:51.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.764+0000 7fd1962ec700 1 -- 172.21.15.7:0/65959833 >> v1:172.21.15.7:6789/0 conn(0x7fd1900c3ae0 legacy=0x7fd1900cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:51.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.764+0000 7fd1962ec700 1 -- 172.21.15.7:0/65959833 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1900c54f0 con 0x7fd190157ca0 2024-01-27T02:36:51.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.764+0000 7fd1952ea700 1 -- 172.21.15.7:0/65959833 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 293384106 0 0) 0x7fd1900ca020 con 0x7fd1900c3ae0 2024-01-27T02:36:51.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.764+0000 7fd1962ec700 1 --2- 172.21.15.7:0/65959833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd190157ca0 0x7fd1900c4a00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0098e0 tx=0x7fd18c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ddee99c342e052 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:51.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.764+0000 7fd1952ea700 1 -- 172.21.15.7:0/65959833 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c009e80 con 0x7fd190157ca0 2024-01-27T02:36:51.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.764+0000 7fd1952ea700 1 -- 172.21.15.7:0/65959833 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18c00b860 con 0x7fd190157ca0 2024-01-27T02:36:51.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.764+0000 7fd1952ea700 1 -- 172.21.15.7:0/65959833 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c00bbe0 con 0x7fd190157ca0 2024-01-27T02:36:51.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.765+0000 7fd198550700 1 -- 172.21.15.7:0/65959833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd190157ca0 msgr2=0x7fd1900c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:51.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.765+0000 7fd198550700 1 --2- 172.21.15.7:0/65959833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd190157ca0 0x7fd1900c4a00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0098e0 tx=0x7fd18c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:36:51.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.765+0000 7fd198550700 1 -- 172.21.15.7:0/65959833 shutdown_connections 2024-01-27T02:36:51.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.765+0000 7fd198550700 1 --2- 172.21.15.7:0/65959833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd190157ca0 0x7fd1900c4a00 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:51.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.765+0000 7fd198550700 1 -- 172.21.15.7:0/65959833 >> 172.21.15.7:0/65959833 conn(0x7fd190098020 msgr2=0x7fd1900711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:51.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.765+0000 7fd198550700 1 -- 172.21.15.7:0/65959833 shutdown_connections 2024-01-27T02:36:51.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.765+0000 7fd198550700 1 -- 172.21.15.7:0/65959833 wait complete. 2024-01-27T02:36:51.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.766+0000 7fd198550700 1 Processor -- start 2024-01-27T02:36:51.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.766+0000 7fd198550700 1 -- start start 2024-01-27T02:36:51.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.767+0000 7fd198550700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1900c3ae0 0x7fd19016b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:51.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.767+0000 7fd198550700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1900cdbf0 0x7fd190168230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:51.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.767+0000 7fd198550700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd190157ca0 0x7fd190168770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:51.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.767+0000 7fd198550700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1900c7740 con 0x7fd1900c3ae0 2024-01-27T02:36:51.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.767+0000 7fd198550700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1900c6bc0 con 0x7fd1900cdbf0 2024-01-27T02:36:51.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.767+0000 7fd198550700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1900c6ea0 con 0x7fd190157ca0 2024-01-27T02:36:51.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.767+0000 7fd1962ec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1900c3ae0 0x7fd19016b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:51.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.767+0000 7fd1962ec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1900c3ae0 0x7fd19016b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49842/0 (socket says 172.21.15.7:49842) 2024-01-27T02:36:51.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.767+0000 7fd1962ec700 1 -- 172.21.15.7:0/2384330198 learned_addr learned my addr 172.21.15.7:0/2384330198 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:51.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.767+0000 7fd195aeb700 1 --2- 172.21.15.7:0/2384330198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1900cdbf0 0x7fd190168230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:51.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.767+0000 7fd196aed700 1 --2- 172.21.15.7:0/2384330198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd190157ca0 0x7fd190168770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:51.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.767+0000 7fd1962ec700 1 -- 172.21.15.7:0/2384330198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd190157ca0 msgr2=0x7fd190168770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:51.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.767+0000 7fd1962ec700 1 --2- 172.21.15.7:0/2384330198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd190157ca0 0x7fd190168770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:51.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.767+0000 7fd1962ec700 1 -- 172.21.15.7:0/2384330198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1900cdbf0 msgr2=0x7fd190168230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:51.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.767+0000 7fd1962ec700 1 --2- 172.21.15.7:0/2384330198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1900cdbf0 0x7fd190168230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:51.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.767+0000 7fd1962ec700 1 -- 172.21.15.7:0/2384330198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c009420 con 0x7fd1900c3ae0 2024-01-27T02:36:51.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.767+0000 7fd196aed700 1 --2- 172.21.15.7:0/2384330198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd190157ca0 0x7fd190168770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:36:51.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.767+0000 7fd1962ec700 1 --2- 172.21.15.7:0/2384330198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1900c3ae0 0x7fd19016b1c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd19006ea80 tx=0x7fd18c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:51.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.768+0000 7fd1867fc700 1 -- 172.21.15.7:0/2384330198 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c01f070 con 0x7fd1900c3ae0 2024-01-27T02:36:51.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.768+0000 7fd1867fc700 1 -- 172.21.15.7:0/2384330198 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18c00b9d0 con 0x7fd1900c3ae0 2024-01-27T02:36:51.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.768+0000 7fd198550700 1 -- 172.21.15.7:0/2384330198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1900c7180 con 0x7fd1900c3ae0 2024-01-27T02:36:51.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.768+0000 7fd1867fc700 1 -- 172.21.15.7:0/2384330198 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c025670 con 0x7fd1900c3ae0 2024-01-27T02:36:51.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.768+0000 7fd198550700 1 -- 172.21.15.7:0/2384330198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1900c67b0 con 0x7fd1900c3ae0 2024-01-27T02:36:51.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.769+0000 7fd198550700 1 -- 172.21.15.7:0/2384330198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd190169360 con 0x7fd1900c3ae0 2024-01-27T02:36:51.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.769+0000 7fd1867fc700 1 -- 172.21.15.7:0/2384330198 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd18c023070 con 0x7fd1900c3ae0 2024-01-27T02:36:51.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.769+0000 7fd1867fc700 1 --2- 172.21.15.7:0/2384330198 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd174066f30 0x7fd1740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:51.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.770+0000 7fd1867fc700 1 -- 172.21.15.7:0/2384330198 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd18c085af0 con 0x7fd1900c3ae0 2024-01-27T02:36:51.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.770+0000 7fd195aeb700 1 --2- 172.21.15.7:0/2384330198 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd174066f30 0x7fd1740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:51.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.770+0000 7fd1867fc700 1 -- 172.21.15.7:0/2384330198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=86}) v3 -- 0x7fd17406ea50 con 0x7fd1900c3ae0 2024-01-27T02:36:51.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.770+0000 7fd195aeb700 1 --2- 172.21.15.7:0/2384330198 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd174066f30 0x7fd1740693e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd1900c87f0 tx=0x7fd180009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:51.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.774+0000 7fd1867fc700 1 -- 172.21.15.7:0/2384330198 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd18c02a060 con 0x7fd1900c3ae0 2024-01-27T02:36:51.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.929+0000 7fd1867fc700 1 -- 172.21.15.7:0/2384330198 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(86..86 src has 1..86) v4 ==== 288+0+0 (secure 0 0 0) 0x7fd18c057120 con 0x7fd1900c3ae0 2024-01-27T02:36:51.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.929+0000 7fd1867fc700 1 -- 172.21.15.7:0/2384330198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=87}) v3 -- 0x7fd17406f340 con 0x7fd1900c3ae0 2024-01-27T02:36:51.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.983+0000 7fd198550700 1 -- 172.21.15.7:0/2384330198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd19004c710 con 0x7fd1900c3ae0 2024-01-27T02:36:51.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.984+0000 7fd1867fc700 1 -- 172.21.15.7:0/2384330198 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+15334 (secure 0 0 0) 0x7fd18c02a070 con 0x7fd1900c3ae0 2024-01-27T02:36:51.984 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:36:51.984 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:36:51.926737+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:35:59.537952+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-27T02:36:00.480234+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:36:51.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.986+0000 7fd198550700 1 -- 172.21.15.7:0/2384330198 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd174066f30 msgr2=0x7fd1740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:51.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.986+0000 7fd198550700 1 --2- 172.21.15.7:0/2384330198 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd174066f30 0x7fd1740693e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd1900c87f0 tx=0x7fd180009040 comp rx=0 tx=0).stop 2024-01-27T02:36:51.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.986+0000 7fd198550700 1 -- 172.21.15.7:0/2384330198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1900c3ae0 msgr2=0x7fd19016b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:51.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.986+0000 7fd198550700 1 --2- 172.21.15.7:0/2384330198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1900c3ae0 0x7fd19016b1c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd19006ea80 tx=0x7fd18c017000 comp rx=0 tx=0).stop 2024-01-27T02:36:51.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.986+0000 7fd198550700 1 -- 172.21.15.7:0/2384330198 shutdown_connections 2024-01-27T02:36:51.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.986+0000 7fd198550700 1 --2- 172.21.15.7:0/2384330198 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd174066f30 0x7fd1740693e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:51.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.986+0000 7fd198550700 1 --2- 172.21.15.7:0/2384330198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd190157ca0 0x7fd190168770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:51.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.986+0000 7fd198550700 1 --2- 172.21.15.7:0/2384330198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1900c3ae0 0x7fd19016b1c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:51.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.986+0000 7fd198550700 1 --2- 172.21.15.7:0/2384330198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1900cdbf0 0x7fd190168230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:51.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.986+0000 7fd198550700 1 -- 172.21.15.7:0/2384330198 >> 172.21.15.7:0/2384330198 conn(0x7fd190098020 msgr2=0x7fd1900cbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:51.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.986+0000 7fd198550700 1 -- 172.21.15.7:0/2384330198 shutdown_connections 2024-01-27T02:36:51.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:51.986+0000 7fd198550700 1 -- 172.21.15.7:0/2384330198 wait complete. 2024-01-27T02:36:52.003 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-27T02:36:52.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.136+0000 7f0325072700 1 Processor -- start 2024-01-27T02:36:52.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.136+0000 7f0325072700 1 -- start start 2024-01-27T02:36:52.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.137+0000 7f0325072700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0320157c00 0x7f03200cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:52.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.137+0000 7f0325072700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f03200b3040 0x7f032015bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:52.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.137+0000 7f0325072700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f032015ba30 con 0x7f03200cfd10 2024-01-27T02:36:52.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.137+0000 7f0325072700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03200d0140 con 0x7f03200b3040 2024-01-27T02:36:52.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.137+0000 7f0325072700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03200d0390 con 0x7f0320157c00 2024-01-27T02:36:52.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.137+0000 7f031f59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f03200b3040 0x7f032015bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:52.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.137+0000 7f031f59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f03200b3040 0x7f032015bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49850/0 (socket says 172.21.15.7:49850) 2024-01-27T02:36:52.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.137+0000 7f031f59e700 1 -- 172.21.15.7:0/1284549097 learned_addr learned my addr 172.21.15.7:0/1284549097 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:52.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.137+0000 7f031ed9d700 1 --2- 172.21.15.7:0/1284549097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0320157c00 0x7f03200cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:52.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.137+0000 7f031f59e700 1 -- 172.21.15.7:0/1284549097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0320157c00 msgr2=0x7f03200cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:52.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.137+0000 7f031f59e700 1 --2- 172.21.15.7:0/1284549097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0320157c00 0x7f03200cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:52.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.137+0000 7f031f59e700 1 -- 172.21.15.7:0/1284549097 >> v1:172.21.15.188:6789/0 conn(0x7f03200cfd10 legacy=0x7f03200b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:52.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.137+0000 7f031f59e700 1 -- 172.21.15.7:0/1284549097 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03200d0670 con 0x7f03200b3040 2024-01-27T02:36:52.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.137+0000 7f031f59e700 1 --2- 172.21.15.7:0/1284549097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03200b3040 0x7f032015bec0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0318009f20 tx=0x7f0318009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fda4a9d694a998a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:52.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.137+0000 7f031dd9b700 1 -- 172.21.15.7:0/1284549097 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031800e070 con 0x7f03200b3040 2024-01-27T02:36:52.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.138+0000 7f031dd9b700 1 -- 172.21.15.7:0/1284549097 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031800ba80 con 0x7f03200b3040 2024-01-27T02:36:52.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.138+0000 7f031dd9b700 1 -- 172.21.15.7:0/1284549097 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03180092e0 con 0x7f03200b3040 2024-01-27T02:36:52.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.138+0000 7f0325072700 1 -- 172.21.15.7:0/1284549097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03200b3040 msgr2=0x7f032015bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:52.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.138+0000 7f0325072700 1 --2- 172.21.15.7:0/1284549097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03200b3040 0x7f032015bec0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0318009f20 tx=0x7f0318009840 comp rx=0 tx=0).stop 2024-01-27T02:36:52.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.138+0000 7f0325072700 1 -- 172.21.15.7:0/1284549097 shutdown_connections 2024-01-27T02:36:52.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.138+0000 7f0325072700 1 --2- 172.21.15.7:0/1284549097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03200b3040 0x7f032015bec0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:52.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.138+0000 7f0325072700 1 --2- 172.21.15.7:0/1284549097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0320157c00 0x7f03200cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:52.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.138+0000 7f0325072700 1 -- 172.21.15.7:0/1284549097 >> 172.21.15.7:0/1284549097 conn(0x7f0320068450 msgr2=0x7f03200709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:52.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.138+0000 7f0325072700 1 -- 172.21.15.7:0/1284549097 shutdown_connections 2024-01-27T02:36:52.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.138+0000 7f0325072700 1 -- 172.21.15.7:0/1284549097 wait complete. 2024-01-27T02:36:52.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.140+0000 7f0325072700 1 Processor -- start 2024-01-27T02:36:52.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.140+0000 7f0325072700 1 -- start start 2024-01-27T02:36:52.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.141+0000 7f0325072700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03200b3040 0x7f0320155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:52.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.141+0000 7f0325072700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03200cfd10 0x7f0320151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:52.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.141+0000 7f0325072700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0320157c00 0x7f0320150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:52.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.141+0000 7f0325072700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03201521b0 con 0x7f03200cfd10 2024-01-27T02:36:52.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.141+0000 7f0325072700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0320155e80 con 0x7f03200b3040 2024-01-27T02:36:52.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.141+0000 7f0325072700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032015e010 con 0x7f0320157c00 2024-01-27T02:36:52.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.141+0000 7f031e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03200cfd10 0x7f0320151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:52.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.141+0000 7f031e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03200cfd10 0x7f0320151530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49866/0 (socket says 172.21.15.7:49866) 2024-01-27T02:36:52.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.141+0000 7f031e59c700 1 -- 172.21.15.7:0/2104414059 learned_addr learned my addr 172.21.15.7:0/2104414059 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:52.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.141+0000 7f031ed9d700 1 --2- 172.21.15.7:0/2104414059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03200b3040 0x7f0320155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:52.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.141+0000 7f031f59e700 1 --2- 172.21.15.7:0/2104414059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0320157c00 0x7f0320150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:52.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.142+0000 7f031e59c700 1 -- 172.21.15.7:0/2104414059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0320157c00 msgr2=0x7f0320150710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:52.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.142+0000 7f031e59c700 1 --2- 172.21.15.7:0/2104414059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0320157c00 0x7f0320150710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:52.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.142+0000 7f031e59c700 1 -- 172.21.15.7:0/2104414059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03200b3040 msgr2=0x7f0320155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:52.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.142+0000 7f031e59c700 1 --2- 172.21.15.7:0/2104414059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03200b3040 0x7f0320155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:52.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.142+0000 7f031e59c700 1 -- 172.21.15.7:0/2104414059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0318009ae0 con 0x7f03200cfd10 2024-01-27T02:36:52.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.142+0000 7f031e59c700 1 --2- 172.21.15.7:0/2104414059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03200cfd10 0x7f0320151530 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f031400c590 tx=0x7f031400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:52.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.142+0000 7f0306ffd700 1 -- 172.21.15.7:0/2104414059 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0314011590 con 0x7f03200cfd10 2024-01-27T02:36:52.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.142+0000 7f0306ffd700 1 -- 172.21.15.7:0/2104414059 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0314011710 con 0x7f03200cfd10 2024-01-27T02:36:52.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.142+0000 7f0325072700 1 -- 172.21.15.7:0/2104414059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03201560f0 con 0x7f03200cfd10 2024-01-27T02:36:52.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.142+0000 7f0306ffd700 1 -- 172.21.15.7:0/2104414059 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0314011ad0 con 0x7f03200cfd10 2024-01-27T02:36:52.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.142+0000 7f0325072700 1 -- 172.21.15.7:0/2104414059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f032015e540 con 0x7f03200cfd10 2024-01-27T02:36:52.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.143+0000 7f0325072700 1 -- 172.21.15.7:0/2104414059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f032015e760 con 0x7f03200cfd10 2024-01-27T02:36:52.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.143+0000 7f0306ffd700 1 -- 172.21.15.7:0/2104414059 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f031401b3b0 con 0x7f03200cfd10 2024-01-27T02:36:52.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.144+0000 7f0306ffd700 1 --2- 172.21.15.7:0/2104414059 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f030c066f30 0x7f030c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:52.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.144+0000 7f0306ffd700 1 -- 172.21.15.7:0/2104414059 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f03140839c0 con 0x7f03200cfd10 2024-01-27T02:36:52.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.144+0000 7f0306ffd700 1 -- 172.21.15.7:0/2104414059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=87}) v3 -- 0x7f030c06ea50 con 0x7f03200cfd10 2024-01-27T02:36:52.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.144+0000 7f031ed9d700 1 --2- 172.21.15.7:0/2104414059 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f030c066f30 0x7f030c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:52.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.145+0000 7f031ed9d700 1 --2- 172.21.15.7:0/2104414059 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f030c066f30 0x7f030c0693e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0310006fd0 tx=0x7f0310008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:52.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.148+0000 7f0306ffd700 1 -- 172.21.15.7:0/2104414059 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0314017070 con 0x7f03200cfd10 2024-01-27T02:36:52.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.401+0000 7f0325072700 1 -- 172.21.15.7:0/2104414059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f032004c710 con 0x7f03200cfd10 2024-01-27T02:36:52.402 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-27T02:36:52.402+0000 7fb5c503a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-27T02:36:52.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.404+0000 7f0306ffd700 1 -- 172.21.15.7:0/2104414059 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(87..87 src has 1..87) v4 ==== 366+0+0 (secure 0 0 0) 0x7f0314054ff0 con 0x7f03200cfd10 2024-01-27T02:36:52.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.404+0000 7f0306ffd700 1 -- 172.21.15.7:0/2104414059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=88}) v3 -- 0x7f030c06f3a0 con 0x7f03200cfd10 2024-01-27T02:36:52.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.406+0000 7f0306ffd700 1 -- 172.21.15.7:0/2104414059 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0314050c30 con 0x7f03200cfd10 2024-01-27T02:36:52.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.410+0000 7f0325072700 1 -- 172.21.15.7:0/2104414059 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f030c066f30 msgr2=0x7f030c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:52.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.410+0000 7f0325072700 1 --2- 172.21.15.7:0/2104414059 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f030c066f30 0x7f030c0693e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0310006fd0 tx=0x7f0310008040 comp rx=0 tx=0).stop 2024-01-27T02:36:52.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.410+0000 7f0325072700 1 -- 172.21.15.7:0/2104414059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03200cfd10 msgr2=0x7f0320151530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:52.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.410+0000 7f0325072700 1 --2- 172.21.15.7:0/2104414059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03200cfd10 0x7f0320151530 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f031400c590 tx=0x7f031400c8a0 comp rx=0 tx=0).stop 2024-01-27T02:36:52.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.410+0000 7f0325072700 1 -- 172.21.15.7:0/2104414059 shutdown_connections 2024-01-27T02:36:52.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.410+0000 7f0325072700 1 --2- 172.21.15.7:0/2104414059 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f030c066f30 0x7f030c0693e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:52.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.410+0000 7f0325072700 1 --2- 172.21.15.7:0/2104414059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0320157c00 0x7f0320150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:52.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.410+0000 7f0325072700 1 --2- 172.21.15.7:0/2104414059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03200b3040 0x7f0320155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:52.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.410+0000 7f0325072700 1 --2- 172.21.15.7:0/2104414059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03200cfd10 0x7f0320151530 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:52.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.410+0000 7f0325072700 1 -- 172.21.15.7:0/2104414059 >> 172.21.15.7:0/2104414059 conn(0x7f0320068450 msgr2=0x7f03201593b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:52.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.410+0000 7f0325072700 1 -- 172.21.15.7:0/2104414059 shutdown_connections 2024-01-27T02:36:52.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.410+0000 7f0325072700 1 -- 172.21.15.7:0/2104414059 wait complete. 2024-01-27T02:36:52.413 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-27T02:36:52.424 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-27T02:36:52.443 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-01-27T02:36:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.558+0000 7f30cdc13700 1 Processor -- start 2024-01-27T02:36:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.558+0000 7f30cdc13700 1 -- start start 2024-01-27T02:36:52.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.559+0000 7f30cdc13700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f30c81416c0 0x7f30c80cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.559+0000 7f30cdc13700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c815c7d0 con 0x7f30c80cb210 2024-01-27T02:36:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.559+0000 7f30cdc13700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c80c9640 con 0x7f30c8141a90 2024-01-27T02:36:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.559+0000 7f30cdc13700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c815bbc0 con 0x7f30c81416c0 2024-01-27T02:36:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.559+0000 7f30c77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f30c81416c0 0x7f30c80cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.559+0000 7f30c7fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f30c80cb210 0x7f30c80cc9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60770/0 (socket says 172.21.15.7:60770) 2024-01-27T02:36:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.559+0000 7f30c7fff700 1 -- 172.21.15.7:0/2519267762 learned_addr learned my addr 172.21.15.7:0/2519267762 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.559+0000 7f30c77fe700 1 -- 172.21.15.7:0/2519267762 >> v1:172.21.15.188:6789/0 conn(0x7f30c8141a90 legacy=0x7f30c80cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.559+0000 7f30c77fe700 1 -- 172.21.15.7:0/2519267762 >> v1:172.21.15.7:6789/0 conn(0x7f30c80cb210 legacy=0x7f30c80cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.559+0000 7f30c77fe700 1 -- 172.21.15.7:0/2519267762 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c815bea0 con 0x7f30c81416c0 2024-01-27T02:36:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.559+0000 7f30c77fe700 1 --2- 172.21.15.7:0/2519267762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30c81416c0 0x7f30c80cb6e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f30b80098d0 tx=0x7f30b8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244d113d6e19db2a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.559+0000 7f30c67fc700 1 -- 172.21.15.7:0/2519267762 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8009e70 con 0x7f30c81416c0 2024-01-27T02:36:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.560+0000 7f30c67fc700 1 -- 172.21.15.7:0/2519267762 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b8005c10 con 0x7f30c81416c0 2024-01-27T02:36:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.560+0000 7f30c67fc700 1 -- 172.21.15.7:0/2519267762 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b800b650 con 0x7f30c81416c0 2024-01-27T02:36:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.560+0000 7f30cdc13700 1 -- 172.21.15.7:0/2519267762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30c81416c0 msgr2=0x7f30c80cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.560+0000 7f30cdc13700 1 --2- 172.21.15.7:0/2519267762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30c81416c0 0x7f30c80cb6e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f30b80098d0 tx=0x7f30b8009120 comp rx=0 tx=0).stop 2024-01-27T02:36:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.560+0000 7f30cdc13700 1 -- 172.21.15.7:0/2519267762 shutdown_connections 2024-01-27T02:36:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.560+0000 7f30cdc13700 1 --2- 172.21.15.7:0/2519267762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30c81416c0 0x7f30c80cb6e0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.560+0000 7f30cdc13700 1 -- 172.21.15.7:0/2519267762 >> 172.21.15.7:0/2519267762 conn(0x7f30c8063580 msgr2=0x7f30c8063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.560+0000 7f30cdc13700 1 -- 172.21.15.7:0/2519267762 shutdown_connections 2024-01-27T02:36:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.560+0000 7f30cdc13700 1 -- 172.21.15.7:0/2519267762 wait complete. 2024-01-27T02:36:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.562+0000 7f30cdc13700 1 Processor -- start 2024-01-27T02:36:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.562+0000 7f30cdc13700 1 -- start start 2024-01-27T02:36:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.562+0000 7f30cdc13700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30c80cb210 0x7f30c81396e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.562+0000 7f30cdc13700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30c81416c0 0x7f30c80c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.562+0000 7f30cdc13700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30c8141a90 0x7f30c813a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.562+0000 7f30cdc13700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c80b2a50 con 0x7f30c81416c0 2024-01-27T02:36:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.562+0000 7f30cdc13700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c8068fb0 con 0x7f30c80cb210 2024-01-27T02:36:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.562+0000 7f30cdc13700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c8069290 con 0x7f30c8141a90 2024-01-27T02:36:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.562+0000 7f30c6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30c81416c0 0x7f30c80c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.563+0000 7f30c6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30c81416c0 0x7f30c80c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49880/0 (socket says 172.21.15.7:49880) 2024-01-27T02:36:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.563+0000 7f30c6ffd700 1 -- 172.21.15.7:0/956740612 learned_addr learned my addr 172.21.15.7:0/956740612 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.563+0000 7f30c7fff700 1 --2- 172.21.15.7:0/956740612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30c8141a90 0x7f30c813a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.563+0000 7f30c77fe700 1 --2- 172.21.15.7:0/956740612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30c80cb210 0x7f30c81396e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.563+0000 7f30c6ffd700 1 -- 172.21.15.7:0/956740612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30c8141a90 msgr2=0x7f30c813a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.563+0000 7f30c6ffd700 1 --2- 172.21.15.7:0/956740612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30c8141a90 0x7f30c813a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.563+0000 7f30c6ffd700 1 -- 172.21.15.7:0/956740612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30c80cb210 msgr2=0x7f30c81396e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.563+0000 7f30c6ffd700 1 --2- 172.21.15.7:0/956740612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30c80cb210 0x7f30c81396e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:52.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.563+0000 7f30c6ffd700 1 -- 172.21.15.7:0/956740612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b80093c0 con 0x7f30c81416c0 2024-01-27T02:36:52.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.563+0000 7f30c6ffd700 1 --2- 172.21.15.7:0/956740612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30c81416c0 0x7f30c80c83d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00b5e0 tx=0x7f30bc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:52.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.563+0000 7f30af7fe700 1 -- 172.21.15.7:0/956740612 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc00f510 con 0x7f30c81416c0 2024-01-27T02:36:52.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.563+0000 7f30af7fe700 1 -- 172.21.15.7:0/956740612 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30bc00f670 con 0x7f30c81416c0 2024-01-27T02:36:52.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.563+0000 7f30cdc13700 1 -- 172.21.15.7:0/956740612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c8069570 con 0x7f30c81416c0 2024-01-27T02:36:52.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.563+0000 7f30af7fe700 1 -- 172.21.15.7:0/956740612 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc00fa30 con 0x7f30c81416c0 2024-01-27T02:36:52.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.563+0000 7f30cdc13700 1 -- 172.21.15.7:0/956740612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c8139c20 con 0x7f30c81416c0 2024-01-27T02:36:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.564+0000 7f30cdc13700 1 -- 172.21.15.7:0/956740612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c8072060 con 0x7f30c81416c0 2024-01-27T02:36:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.565+0000 7f30af7fe700 1 -- 172.21.15.7:0/956740612 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f30bc01a400 con 0x7f30c81416c0 2024-01-27T02:36:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.565+0000 7f30af7fe700 1 --2- 172.21.15.7:0/956740612 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f30b4066f30 0x7f30b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.565+0000 7f30af7fe700 1 -- 172.21.15.7:0/956740612 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f30bc083fc0 con 0x7f30c81416c0 2024-01-27T02:36:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.565+0000 7f30c77fe700 1 --2- 172.21.15.7:0/956740612 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f30b4066f30 0x7f30b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.565+0000 7f30af7fe700 1 -- 172.21.15.7:0/956740612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=88}) v3 -- 0x7f30b406eaf0 con 0x7f30c81416c0 2024-01-27T02:36:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.566+0000 7f30c77fe700 1 --2- 172.21.15.7:0/956740612 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f30b4066f30 0x7f30b40693e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f30b8003080 tx=0x7f30b8015000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.570+0000 7f30af7fe700 1 -- 172.21.15.7:0/956740612 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f30bc0511c0 con 0x7f30c81416c0 2024-01-27T02:36:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:52.824+0000 7f30cdc13700 1 -- 172.21.15.7:0/956740612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f30c815c180 con 0x7f30c81416c0 2024-01-27T02:36:53.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.407+0000 7f30af7fe700 1 -- 172.21.15.7:0/956740612 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) v1 ==== 104+0+0 (secure 0 0 0) 0x7f30bc0547e0 con 0x7f30c81416c0 2024-01-27T02:36:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.409+0000 7f30cdc13700 1 -- 172.21.15.7:0/956740612 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f30b4066f30 msgr2=0x7f30b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.409+0000 7f30cdc13700 1 --2- 172.21.15.7:0/956740612 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f30b4066f30 0x7f30b40693e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f30b8003080 tx=0x7f30b8015000 comp rx=0 tx=0).stop 2024-01-27T02:36:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.410+0000 7f30cdc13700 1 -- 172.21.15.7:0/956740612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30c81416c0 msgr2=0x7f30c80c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.410+0000 7f30cdc13700 1 --2- 172.21.15.7:0/956740612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30c81416c0 0x7f30c80c83d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00b5e0 tx=0x7f30bc009120 comp rx=0 tx=0).stop 2024-01-27T02:36:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.410+0000 7f30cdc13700 1 -- 172.21.15.7:0/956740612 shutdown_connections 2024-01-27T02:36:53.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.410+0000 7f30cdc13700 1 --2- 172.21.15.7:0/956740612 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f30b4066f30 0x7f30b40693e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:53.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.410+0000 7f30cdc13700 1 --2- 172.21.15.7:0/956740612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30c8141a90 0x7f30c813a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:53.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.410+0000 7f30cdc13700 1 --2- 172.21.15.7:0/956740612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30c80cb210 0x7f30c81396e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:53.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.410+0000 7f30cdc13700 1 --2- 172.21.15.7:0/956740612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30c81416c0 0x7f30c80c83d0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:53.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.410+0000 7f30cdc13700 1 -- 172.21.15.7:0/956740612 >> 172.21.15.7:0/956740612 conn(0x7f30c8063580 msgr2=0x7f30c81577d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:53.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.410+0000 7f30cdc13700 1 -- 172.21.15.7:0/956740612 shutdown_connections 2024-01-27T02:36:53.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.410+0000 7f30cdc13700 1 -- 172.21.15.7:0/956740612 wait complete. 2024-01-27T02:36:53.426 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-27T02:36:53.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.565+0000 7f2a15426700 1 Processor -- start 2024-01-27T02:36:53.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.565+0000 7f2a15426700 1 -- start start 2024-01-27T02:36:53.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.566+0000 7f2a15426700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a10157be0 0x7f2a100cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:53.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.566+0000 7f2a15426700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a1015b0b0 con 0x7f2a100cc310 2024-01-27T02:36:53.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.566+0000 7f2a15426700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a100ce2b0 con 0x7f2a100cde40 2024-01-27T02:36:53.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.566+0000 7f2a15426700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a100afe70 con 0x7f2a10157be0 2024-01-27T02:36:53.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.566+0000 7f2a0f7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2a100cc310 0x7f2a1015b920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60782/0 (socket says 172.21.15.7:60782) 2024-01-27T02:36:53.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.566+0000 7f2a0f7fe700 1 -- 172.21.15.7:0/922005500 learned_addr learned my addr 172.21.15.7:0/922005500 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:53.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.566+0000 7f2a0effd700 1 --2- 172.21.15.7:0/922005500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a10157be0 0x7f2a100cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:53.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.567+0000 7f2a0dffb700 1 -- 172.21.15.7:0/922005500 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2666173872 0 0) 0x7f2a1015b0b0 con 0x7f2a100cc310 2024-01-27T02:36:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.567+0000 7f2a0dffb700 1 -- 172.21.15.7:0/922005500 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29fc003570 con 0x7f2a100cc310 2024-01-27T02:36:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.567+0000 7f2a0dffb700 1 -- 172.21.15.7:0/922005500 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3401006886 0 0) 0x7f2a100ce2b0 con 0x7f2a100cde40 2024-01-27T02:36:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.567+0000 7f2a0effd700 1 -- 172.21.15.7:0/922005500 >> v1:172.21.15.188:6789/0 conn(0x7f2a100cde40 legacy=0x7f2a100cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.567+0000 7f2a0effd700 1 -- 172.21.15.7:0/922005500 >> v1:172.21.15.7:6789/0 conn(0x7f2a100cc310 legacy=0x7f2a1015b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.567+0000 7f2a0effd700 1 -- 172.21.15.7:0/922005500 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a100cc880 con 0x7f2a10157be0 2024-01-27T02:36:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.567+0000 7f2a0effd700 1 --2- 172.21.15.7:0/922005500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a10157be0 0x7f2a100cd900 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2a00009910 tx=0x7f2a000091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d7dc12c4035b765 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.567+0000 7f2a0dffb700 1 -- 172.21.15.7:0/922005500 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0000b680 con 0x7f2a10157be0 2024-01-27T02:36:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.567+0000 7f2a0dffb700 1 -- 172.21.15.7:0/922005500 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0000b7e0 con 0x7f2a10157be0 2024-01-27T02:36:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.567+0000 7f2a0dffb700 1 -- 172.21.15.7:0/922005500 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0000bba0 con 0x7f2a10157be0 2024-01-27T02:36:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.568+0000 7f2a15426700 1 -- 172.21.15.7:0/922005500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a10157be0 msgr2=0x7f2a100cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.568+0000 7f2a15426700 1 --2- 172.21.15.7:0/922005500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a10157be0 0x7f2a100cd900 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2a00009910 tx=0x7f2a000091a0 comp rx=0 tx=0).stop 2024-01-27T02:36:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.568+0000 7f2a15426700 1 -- 172.21.15.7:0/922005500 shutdown_connections 2024-01-27T02:36:53.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.568+0000 7f2a15426700 1 --2- 172.21.15.7:0/922005500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a10157be0 0x7f2a100cd900 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:53.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.568+0000 7f2a15426700 1 -- 172.21.15.7:0/922005500 >> 172.21.15.7:0/922005500 conn(0x7f2a10068440 msgr2=0x7f2a10063580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:53.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.568+0000 7f2a15426700 1 -- 172.21.15.7:0/922005500 shutdown_connections 2024-01-27T02:36:53.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.568+0000 7f2a15426700 1 -- 172.21.15.7:0/922005500 wait complete. 2024-01-27T02:36:53.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.570+0000 7f2a15426700 1 Processor -- start 2024-01-27T02:36:53.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.570+0000 7f2a15426700 1 -- start start 2024-01-27T02:36:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.571+0000 7f2a15426700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a100cc310 0x7f2a100cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.571+0000 7f2a15426700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a100cde40 0x7f2a10149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.571+0000 7f2a15426700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a10157be0 0x7f2a100b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.571+0000 7f2a15426700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1016b680 con 0x7f2a100cc310 2024-01-27T02:36:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.571+0000 7f2a15426700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a100d0510 con 0x7f2a100cde40 2024-01-27T02:36:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.571+0000 7f2a15426700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1014a020 con 0x7f2a10157be0 2024-01-27T02:36:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.571+0000 7f2a0effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a100cc310 0x7f2a100cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.571+0000 7f2a0effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a100cc310 0x7f2a100cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49882/0 (socket says 172.21.15.7:49882) 2024-01-27T02:36:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.571+0000 7f2a0effd700 1 -- 172.21.15.7:0/675637440 learned_addr learned my addr 172.21.15.7:0/675637440 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.571+0000 7f2a0f7fe700 1 --2- 172.21.15.7:0/675637440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a10157be0 0x7f2a100b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.571+0000 7f2a0e7fc700 1 --2- 172.21.15.7:0/675637440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a100cde40 0x7f2a10149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.571+0000 7f2a0effd700 1 -- 172.21.15.7:0/675637440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a10157be0 msgr2=0x7f2a100b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.571+0000 7f2a0effd700 1 --2- 172.21.15.7:0/675637440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a10157be0 0x7f2a100b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.571+0000 7f2a0effd700 1 -- 172.21.15.7:0/675637440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a100cde40 msgr2=0x7f2a10149ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.571+0000 7f2a0effd700 1 --2- 172.21.15.7:0/675637440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a100cde40 0x7f2a10149ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.571+0000 7f2a0effd700 1 -- 172.21.15.7:0/675637440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a00009420 con 0x7f2a100cc310 2024-01-27T02:36:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.571+0000 7f2a0e7fc700 1 --2- 172.21.15.7:0/675637440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a100cde40 0x7f2a10149ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:36:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.571+0000 7f2a0f7fe700 1 --2- 172.21.15.7:0/675637440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a10157be0 0x7f2a100b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:36:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.572+0000 7f2a0effd700 1 --2- 172.21.15.7:0/675637440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a100cc310 0x7f2a100cffd0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2a00009fb0 tx=0x7f2a00017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.572+0000 7f2a06ffd700 1 -- 172.21.15.7:0/675637440 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0000bb60 con 0x7f2a100cc310 2024-01-27T02:36:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.572+0000 7f2a06ffd700 1 -- 172.21.15.7:0/675637440 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a00011420 con 0x7f2a100cc310 2024-01-27T02:36:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.572+0000 7f2a06ffd700 1 -- 172.21.15.7:0/675637440 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a000117e0 con 0x7f2a100cc310 2024-01-27T02:36:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.572+0000 7f2a15426700 1 -- 172.21.15.7:0/675637440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a1016aa70 con 0x7f2a100cc310 2024-01-27T02:36:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.572+0000 7f2a15426700 1 -- 172.21.15.7:0/675637440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1016b030 con 0x7f2a100cc310 2024-01-27T02:36:53.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.573+0000 7f2a15426700 1 -- 172.21.15.7:0/675637440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a1006dfd0 con 0x7f2a100cc310 2024-01-27T02:36:53.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.573+0000 7f2a06ffd700 1 -- 172.21.15.7:0/675637440 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2a00011b50 con 0x7f2a100cc310 2024-01-27T02:36:53.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.574+0000 7f2a06ffd700 1 --2- 172.21.15.7:0/675637440 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f29f0066f30 0x7f29f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:53.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.574+0000 7f2a06ffd700 1 -- 172.21.15.7:0/675637440 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f2a00086280 con 0x7f2a100cc310 2024-01-27T02:36:53.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.574+0000 7f2a06ffd700 1 -- 172.21.15.7:0/675637440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=88}) v3 -- 0x7f29f006eaf0 con 0x7f2a100cc310 2024-01-27T02:36:53.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.574+0000 7f2a0e7fc700 1 --2- 172.21.15.7:0/675637440 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f29f0066f30 0x7f29f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:53.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.575+0000 7f2a0e7fc700 1 --2- 172.21.15.7:0/675637440 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f29f0066f30 0x7f29f00693e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f29f80011e0 tx=0x7f29f8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:53.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.578+0000 7f2a06ffd700 1 -- 172.21.15.7:0/675637440 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a000534a0 con 0x7f2a100cc310 2024-01-27T02:36:53.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.787+0000 7f2a15426700 1 -- 172.21.15.7:0/675637440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2a1004c710 con 0x7f2a100cc310 2024-01-27T02:36:53.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.932+0000 7f2a06ffd700 1 -- 172.21.15.7:0/675637440 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(88..88 src has 1..88) v4 ==== 296+0+0 (secure 0 0 0) 0x7f2a000860c0 con 0x7f2a100cc310 2024-01-27T02:36:53.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.932+0000 7f2a06ffd700 1 -- 172.21.15.7:0/675637440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7f29f006cf20 con 0x7f2a100cc310 2024-01-27T02:36:53.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.935+0000 7f2a06ffd700 1 -- 172.21.15.7:0/675637440 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v88) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2a00059030 con 0x7f2a100cc310 2024-01-27T02:36:53.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.942+0000 7f2a15426700 1 -- 172.21.15.7:0/675637440 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f29f0066f30 msgr2=0x7f29f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:53.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.942+0000 7f2a15426700 1 --2- 172.21.15.7:0/675637440 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f29f0066f30 0x7f29f00693e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f29f80011e0 tx=0x7f29f8009040 comp rx=0 tx=0).stop 2024-01-27T02:36:53.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.942+0000 7f2a15426700 1 -- 172.21.15.7:0/675637440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a100cc310 msgr2=0x7f2a100cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:53.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.942+0000 7f2a15426700 1 --2- 172.21.15.7:0/675637440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a100cc310 0x7f2a100cffd0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2a00009fb0 tx=0x7f2a00017000 comp rx=0 tx=0).stop 2024-01-27T02:36:53.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.942+0000 7f2a15426700 1 -- 172.21.15.7:0/675637440 shutdown_connections 2024-01-27T02:36:53.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.942+0000 7f2a15426700 1 --2- 172.21.15.7:0/675637440 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f29f0066f30 0x7f29f00693e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:53.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.942+0000 7f2a15426700 1 --2- 172.21.15.7:0/675637440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a10157be0 0x7f2a100b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:53.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.942+0000 7f2a15426700 1 --2- 172.21.15.7:0/675637440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a100cc310 0x7f2a100cffd0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:53.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.942+0000 7f2a15426700 1 --2- 172.21.15.7:0/675637440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a100cde40 0x7f2a10149ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:53.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.942+0000 7f2a15426700 1 -- 172.21.15.7:0/675637440 >> 172.21.15.7:0/675637440 conn(0x7f2a10068440 msgr2=0x7f2a100cb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:53.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.942+0000 7f2a15426700 1 -- 172.21.15.7:0/675637440 shutdown_connections 2024-01-27T02:36:53.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:53.942+0000 7f2a15426700 1 -- 172.21.15.7:0/675637440 wait complete. 2024-01-27T02:36:53.945 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-27T02:36:53.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-27T02:36:54.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.094+0000 7fba6d194700 1 Processor -- start 2024-01-27T02:36:54.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.094+0000 7fba6d194700 1 -- start start 2024-01-27T02:36:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.095+0000 7fba6d194700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fba68157f40 0x7fba680cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.095+0000 7fba6d194700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fba680cdb70 0x7fba680cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.095+0000 7fba6d194700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba680cbc00 0x7fba68158310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.095+0000 7fba6d194700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba680ca670 con 0x7fba68157f40 2024-01-27T02:36:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.095+0000 7fba6d194700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba680cdfc0 con 0x7fba680cdb70 2024-01-27T02:36:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.095+0000 7fba6d194700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba680cb470 con 0x7fba680cbc00 2024-01-27T02:36:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.095+0000 7fba66d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fba68157f40 0x7fba680cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.095+0000 7fba66d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fba68157f40 0x7fba680cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49890/0 (socket says 172.21.15.7:49890) 2024-01-27T02:36:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.095+0000 7fba66d9d700 1 -- 172.21.15.7:0/923810112 learned_addr learned my addr 172.21.15.7:0/923810112 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.095+0000 7fba6759e700 1 --2- 172.21.15.7:0/923810112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba680cbc00 0x7fba68158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:54.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.095+0000 7fba6659c700 1 --2- 172.21.15.7:0/923810112 >> v2:172.21.15.188:3300/0 conn(0x7fba680cdb70 0x7fba680cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:54.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.095+0000 7fba66d9d700 1 -- 172.21.15.7:0/923810112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba680cbc00 msgr2=0x7fba68158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:54.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.095+0000 7fba66d9d700 1 --2- 172.21.15.7:0/923810112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba680cbc00 0x7fba68158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:54.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.095+0000 7fba66d9d700 1 -- 172.21.15.7:0/923810112 >> v2:172.21.15.188:3300/0 conn(0x7fba680cdb70 msgr2=0x7fba680cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:54.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.095+0000 7fba66d9d700 1 --2- 172.21.15.7:0/923810112 >> v2:172.21.15.188:3300/0 conn(0x7fba680cdb70 0x7fba680cb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:54.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.095+0000 7fba66d9d700 1 -- 172.21.15.7:0/923810112 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba680cc080 con 0x7fba68157f40 2024-01-27T02:36:54.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.095+0000 7fba6759e700 1 --2- 172.21.15.7:0/923810112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba680cbc00 0x7fba68158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:36:54.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.095+0000 7fba6659c700 1 --2- 172.21.15.7:0/923810112 >> v2:172.21.15.188:3300/0 conn(0x7fba680cdb70 0x7fba680cb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:36:54.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.096+0000 7fba66d9d700 1 --2- 172.21.15.7:0/923810112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba68157f40 0x7fba680cd630 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fba680b1880 tx=0x7fba58009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba35b69b5011fa6 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:54.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.096+0000 7fba65d9b700 1 -- 172.21.15.7:0/923810112 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba58005c10 con 0x7fba68157f40 2024-01-27T02:36:54.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.096+0000 7fba65d9b700 1 -- 172.21.15.7:0/923810112 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba58005d70 con 0x7fba68157f40 2024-01-27T02:36:54.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.096+0000 7fba65d9b700 1 -- 172.21.15.7:0/923810112 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5800b880 con 0x7fba68157f40 2024-01-27T02:36:54.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.096+0000 7fba6d194700 1 -- 172.21.15.7:0/923810112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba68157f40 msgr2=0x7fba680cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:54.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.096+0000 7fba6d194700 1 --2- 172.21.15.7:0/923810112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba68157f40 0x7fba680cd630 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fba680b1880 tx=0x7fba58009220 comp rx=0 tx=0).stop 2024-01-27T02:36:54.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.096+0000 7fba6d194700 1 -- 172.21.15.7:0/923810112 shutdown_connections 2024-01-27T02:36:54.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.096+0000 7fba6d194700 1 --2- 172.21.15.7:0/923810112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba680cbc00 0x7fba68158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:54.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.096+0000 7fba6d194700 1 --2- 172.21.15.7:0/923810112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba68157f40 0x7fba680cd630 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:54.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.096+0000 7fba6d194700 1 --2- 172.21.15.7:0/923810112 >> v2:172.21.15.188:3300/0 conn(0x7fba680cdb70 0x7fba680cb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:54.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.096+0000 7fba6d194700 1 -- 172.21.15.7:0/923810112 >> 172.21.15.7:0/923810112 conn(0x7fba68098020 msgr2=0x7fba680711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:54.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.096+0000 7fba6d194700 1 -- 172.21.15.7:0/923810112 shutdown_connections 2024-01-27T02:36:54.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.096+0000 7fba6d194700 1 -- 172.21.15.7:0/923810112 wait complete. 2024-01-27T02:36:54.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.098+0000 7fba6d194700 1 Processor -- start 2024-01-27T02:36:54.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.098+0000 7fba6d194700 1 -- start start 2024-01-27T02:36:54.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.099+0000 7fba6d194700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba680cbc00 0x7fba6815c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:54.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.099+0000 7fba6d194700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba680cdb70 0x7fba68149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:54.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.099+0000 7fba6d194700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba68157f40 0x7fba68168280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:54.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.099+0000 7fba6d194700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba680b2890 con 0x7fba68157f40 2024-01-27T02:36:54.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.099+0000 7fba6d194700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba680b1c80 con 0x7fba680cbc00 2024-01-27T02:36:54.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.099+0000 7fba6d194700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba680b1f60 con 0x7fba680cdb70 2024-01-27T02:36:54.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.099+0000 7fba6759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba68157f40 0x7fba68168280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:54.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.099+0000 7fba6759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba68157f40 0x7fba68168280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49898/0 (socket says 172.21.15.7:49898) 2024-01-27T02:36:54.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.099+0000 7fba6759e700 1 -- 172.21.15.7:0/1602699746 learned_addr learned my addr 172.21.15.7:0/1602699746 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.099+0000 7fba66d9d700 1 --2- 172.21.15.7:0/1602699746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba680cbc00 0x7fba6815c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.099+0000 7fba6659c700 1 --2- 172.21.15.7:0/1602699746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba680cdb70 0x7fba68149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.099+0000 7fba6759e700 1 -- 172.21.15.7:0/1602699746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba680cdb70 msgr2=0x7fba68149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.099+0000 7fba6759e700 1 --2- 172.21.15.7:0/1602699746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba680cdb70 0x7fba68149770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.099+0000 7fba6759e700 1 -- 172.21.15.7:0/1602699746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba680cbc00 msgr2=0x7fba6815c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.099+0000 7fba6759e700 1 --2- 172.21.15.7:0/1602699746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba680cbc00 0x7fba6815c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.099+0000 7fba6759e700 1 -- 172.21.15.7:0/1602699746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba58009480 con 0x7fba68157f40 2024-01-27T02:36:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.099+0000 7fba6759e700 1 --2- 172.21.15.7:0/1602699746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba68157f40 0x7fba68168280 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fba6000cf60 tx=0x7fba60007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.100+0000 7fba5effd700 1 -- 172.21.15.7:0/1602699746 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6000ae40 con 0x7fba68157f40 2024-01-27T02:36:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.100+0000 7fba5effd700 1 -- 172.21.15.7:0/1602699746 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba60015440 con 0x7fba68157f40 2024-01-27T02:36:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.100+0000 7fba6d194700 1 -- 172.21.15.7:0/1602699746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba680b2240 con 0x7fba68157f40 2024-01-27T02:36:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.100+0000 7fba5effd700 1 -- 172.21.15.7:0/1602699746 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba60015800 con 0x7fba68157f40 2024-01-27T02:36:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.100+0000 7fba6d194700 1 -- 172.21.15.7:0/1602699746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba6815c9b0 con 0x7fba68157f40 2024-01-27T02:36:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.101+0000 7fba6d194700 1 -- 172.21.15.7:0/1602699746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba6804c710 con 0x7fba68157f40 2024-01-27T02:36:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.101+0000 7fba5effd700 1 -- 172.21.15.7:0/1602699746 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fba6000d440 con 0x7fba68157f40 2024-01-27T02:36:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.102+0000 7fba5effd700 1 --2- 172.21.15.7:0/1602699746 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fba48066f30 0x7fba480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.102+0000 7fba5effd700 1 -- 172.21.15.7:0/1602699746 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5353+0+0 (secure 0 0 0) 0x7fba60084f60 con 0x7fba68157f40 2024-01-27T02:36:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.102+0000 7fba5effd700 1 -- 172.21.15.7:0/1602699746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7fba4806dcf0 con 0x7fba68157f40 2024-01-27T02:36:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.102+0000 7fba66d9d700 1 --2- 172.21.15.7:0/1602699746 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fba48066f30 0x7fba480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.102+0000 7fba66d9d700 1 --2- 172.21.15.7:0/1602699746 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fba48066f30 0x7fba480693e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fba58009260 tx=0x7fba58016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:54.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.106+0000 7fba5effd700 1 -- 172.21.15.7:0/1602699746 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba6001a060 con 0x7fba68157f40 2024-01-27T02:36:54.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.308+0000 7fba6d194700 1 -- 172.21.15.7:0/1602699746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fba680ca860 con 0x7fba68157f40 2024-01-27T02:36:54.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.952+0000 7fba5effd700 1 -- 172.21.15.7:0/1602699746 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(89..89 src has 1..89) v4 ==== 666+0+0 (secure 0 0 0) 0x7fba600566f0 con 0x7fba68157f40 2024-01-27T02:36:54.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.952+0000 7fba5effd700 1 -- 172.21.15.7:0/1602699746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=90}) v3 -- 0x7fba4806ca80 con 0x7fba68157f40 2024-01-27T02:36:54.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.954+0000 7fba5effd700 1 -- 172.21.15.7:0/1602699746 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v89) v1 ==== 169+0+0 (secure 0 0 0) 0x7fba6001a030 con 0x7fba68157f40 2024-01-27T02:36:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.968+0000 7fba6d194700 1 -- 172.21.15.7:0/1602699746 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fba48066f30 msgr2=0x7fba480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.968+0000 7fba6d194700 1 --2- 172.21.15.7:0/1602699746 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fba48066f30 0x7fba480693e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fba58009260 tx=0x7fba58016000 comp rx=0 tx=0).stop 2024-01-27T02:36:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.968+0000 7fba6d194700 1 -- 172.21.15.7:0/1602699746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba68157f40 msgr2=0x7fba68168280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.968+0000 7fba6d194700 1 --2- 172.21.15.7:0/1602699746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba68157f40 0x7fba68168280 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fba6000cf60 tx=0x7fba60007870 comp rx=0 tx=0).stop 2024-01-27T02:36:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.968+0000 7fba6d194700 1 -- 172.21.15.7:0/1602699746 shutdown_connections 2024-01-27T02:36:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.968+0000 7fba6d194700 1 --2- 172.21.15.7:0/1602699746 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fba48066f30 0x7fba480693e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:54.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.968+0000 7fba6d194700 1 --2- 172.21.15.7:0/1602699746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba68157f40 0x7fba68168280 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:54.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.968+0000 7fba6d194700 1 --2- 172.21.15.7:0/1602699746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba680cbc00 0x7fba6815c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:54.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.968+0000 7fba6d194700 1 --2- 172.21.15.7:0/1602699746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba680cdb70 0x7fba68149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:54.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.968+0000 7fba6d194700 1 -- 172.21.15.7:0/1602699746 >> 172.21.15.7:0/1602699746 conn(0x7fba68098020 msgr2=0x7fba6806cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:54.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.968+0000 7fba6d194700 1 -- 172.21.15.7:0/1602699746 shutdown_connections 2024-01-27T02:36:54.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:54.968+0000 7fba6d194700 1 -- 172.21.15.7:0/1602699746 wait complete. 2024-01-27T02:36:54.970 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-27T02:36:54.984 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-27T02:36:55.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.122+0000 7f6d74c34700 1 Processor -- start 2024-01-27T02:36:55.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.122+0000 7f6d74c34700 1 -- start start 2024-01-27T02:36:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.123+0000 7f6d74c34700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6d70156b60 0x7f6d700ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.123+0000 7f6d74c34700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d700aeaa0 con 0x7f6d700ad9f0 2024-01-27T02:36:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.123+0000 7f6d74c34700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d700ade90 con 0x7f6d70156f30 2024-01-27T02:36:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.123+0000 7f6d74c34700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d700ae170 con 0x7f6d70156b60 2024-01-27T02:36:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.123+0000 7f6d6dd9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6d700ad9f0 0x7f6d700aef30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60792/0 (socket says 172.21.15.7:60792) 2024-01-27T02:36:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.123+0000 7f6d6dd9b700 1 -- 172.21.15.7:0/583146887 learned_addr learned my addr 172.21.15.7:0/583146887 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.123+0000 7f6d6e59c700 1 --2- 172.21.15.7:0/583146887 >> v2:172.21.15.188:3300/0 conn(0x7f6d70156b60 0x7f6d700ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.123+0000 7f6d6d59a700 1 -- 172.21.15.7:0/583146887 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 843478730 0 0) 0x7f6d700aeaa0 con 0x7f6d700ad9f0 2024-01-27T02:36:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.123+0000 7f6d6d59a700 1 -- 172.21.15.7:0/583146887 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d58003570 con 0x7f6d700ad9f0 2024-01-27T02:36:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.123+0000 7f6d6d59a700 1 -- 172.21.15.7:0/583146887 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2267578140 0 0) 0x7f6d58003570 con 0x7f6d700ad9f0 2024-01-27T02:36:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.123+0000 7f6d6d59a700 1 -- 172.21.15.7:0/583146887 >> v2:172.21.15.188:3300/0 conn(0x7f6d70156b60 msgr2=0x7f6d700ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.123+0000 7f6d6d59a700 1 --2- 172.21.15.7:0/583146887 >> v2:172.21.15.188:3300/0 conn(0x7f6d70156b60 0x7f6d700ad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.124+0000 7f6d6d59a700 1 -- 172.21.15.7:0/583146887 >> v1:172.21.15.188:6789/0 conn(0x7f6d70156f30 legacy=0x7f6d7015a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.124+0000 7f6d6d59a700 1 -- 172.21.15.7:0/583146887 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d700ae450 con 0x7f6d700ad9f0 2024-01-27T02:36:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.124+0000 7f6d6d59a700 1 -- 172.21.15.7:0/583146887 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f6d64001470 con 0x7f6d700ad9f0 2024-01-27T02:36:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.124+0000 7f6d6d59a700 1 -- 172.21.15.7:0/583146887 >> v1:172.21.15.7:6789/0 conn(0x7f6d700ad9f0 legacy=0x7f6d700aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.124+0000 7f6d6d59a700 1 --2- 172.21.15.7:0/583146887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d58003a60 0x7f6d58003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.124+0000 7f6d6d59a700 1 --2- 172.21.15.7:0/583146887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d58004810 0x7f6d58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.124+0000 7f6d6d59a700 1 --2- 172.21.15.7:0/583146887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d580051c0 0x7f6d5800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.124+0000 7f6d6d59a700 1 -- 172.21.15.7:0/583146887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d700ae170 con 0x7f6d580051c0 2024-01-27T02:36:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.124+0000 7f6d6d59a700 1 -- 172.21.15.7:0/583146887 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5800c0d0 con 0x7f6d58004810 2024-01-27T02:36:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.124+0000 7f6d6d59a700 1 -- 172.21.15.7:0/583146887 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5800c290 con 0x7f6d58003a60 2024-01-27T02:36:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.124+0000 7f6d6ed9d700 1 --2- 172.21.15.7:0/583146887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d580051c0 0x7f6d5800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.124+0000 7f6d6e59c700 1 --2- 172.21.15.7:0/583146887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d58003a60 0x7f6d58003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.124+0000 7f6d6dd9b700 1 --2- 172.21.15.7:0/583146887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d58004810 0x7f6d58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.124+0000 7f6d6ed9d700 1 -- 172.21.15.7:0/583146887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d58003a60 msgr2=0x7f6d58003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.124+0000 7f6d6ed9d700 1 --2- 172.21.15.7:0/583146887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d58003a60 0x7f6d58003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.124+0000 7f6d6ed9d700 1 -- 172.21.15.7:0/583146887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d58004810 msgr2=0x7f6d58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.124+0000 7f6d6ed9d700 1 --2- 172.21.15.7:0/583146887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d58004810 0x7f6d58004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:55.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.124+0000 7f6d6ed9d700 1 -- 172.21.15.7:0/583146887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d58003570 con 0x7f6d580051c0 2024-01-27T02:36:55.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.124+0000 7f6d6ed9d700 1 -- 172.21.15.7:0/583146887 reap_dead start 2024-01-27T02:36:55.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.124+0000 7f6d6ed9d700 1 --2- 172.21.15.7:0/583146887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d580051c0 0x7f6d5800ba60 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6d68001fc0 tx=0x7f6d68001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8432b3baa8a2b68e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:55.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.125+0000 7f6d6d59a700 1 -- 172.21.15.7:0/583146887 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d68001950 con 0x7f6d580051c0 2024-01-27T02:36:55.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.125+0000 7f6d6d59a700 1 -- 172.21.15.7:0/583146887 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6800c8b0 con 0x7f6d580051c0 2024-01-27T02:36:55.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.125+0000 7f6d6d59a700 1 -- 172.21.15.7:0/583146887 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6800cc50 con 0x7f6d580051c0 2024-01-27T02:36:55.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.125+0000 7f6d74c34700 1 -- 172.21.15.7:0/583146887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d580051c0 msgr2=0x7f6d5800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:55.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.125+0000 7f6d74c34700 1 --2- 172.21.15.7:0/583146887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d580051c0 0x7f6d5800ba60 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6d68001fc0 tx=0x7f6d68001640 comp rx=0 tx=0).stop 2024-01-27T02:36:55.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.125+0000 7f6d74c34700 1 -- 172.21.15.7:0/583146887 shutdown_connections 2024-01-27T02:36:55.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.125+0000 7f6d74c34700 1 --2- 172.21.15.7:0/583146887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d580051c0 0x7f6d5800ba60 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:55.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.125+0000 7f6d74c34700 1 -- 172.21.15.7:0/583146887 >> 172.21.15.7:0/583146887 conn(0x7f6d70068480 msgr2=0x7f6d700cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:55.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.125+0000 7f6d74c34700 1 -- 172.21.15.7:0/583146887 shutdown_connections 2024-01-27T02:36:55.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.125+0000 7f6d74c34700 1 -- 172.21.15.7:0/583146887 wait complete. 2024-01-27T02:36:55.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.126+0000 7f6d74c34700 1 Processor -- start 2024-01-27T02:36:55.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.127+0000 7f6d74c34700 1 -- start start 2024-01-27T02:36:55.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.127+0000 7f6d74c34700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d580051c0 0x7f6d70155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:55.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.127+0000 7f6d74c34700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d70153690 0x7f6d7014d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:55.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.127+0000 7f6d74c34700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d700b1700 0x7f6d700b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:55.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.127+0000 7f6d74c34700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d700b20b0 con 0x7f6d70153690 2024-01-27T02:36:55.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.127+0000 7f6d74c34700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d70157610 con 0x7f6d580051c0 2024-01-27T02:36:55.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.127+0000 7f6d74c34700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d70153e00 con 0x7f6d700b1700 2024-01-27T02:36:55.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.127+0000 7f6d6dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d70153690 0x7f6d7014d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:55.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.127+0000 7f6d6dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d70153690 0x7f6d7014d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49922/0 (socket says 172.21.15.7:49922) 2024-01-27T02:36:55.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.127+0000 7f6d6dd9b700 1 -- 172.21.15.7:0/3614344511 learned_addr learned my addr 172.21.15.7:0/3614344511 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:55.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.127+0000 7f6d6ed9d700 1 --2- 172.21.15.7:0/3614344511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d700b1700 0x7f6d700b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:55.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.127+0000 7f6d6e59c700 1 --2- 172.21.15.7:0/3614344511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d580051c0 0x7f6d70155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:55.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.128+0000 7f6d6dd9b700 1 -- 172.21.15.7:0/3614344511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d700b1700 msgr2=0x7f6d700b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:55.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.128+0000 7f6d6dd9b700 1 --2- 172.21.15.7:0/3614344511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d700b1700 0x7f6d700b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:55.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.128+0000 7f6d6dd9b700 1 -- 172.21.15.7:0/3614344511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d580051c0 msgr2=0x7f6d70155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:55.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.128+0000 7f6d6dd9b700 1 --2- 172.21.15.7:0/3614344511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d580051c0 0x7f6d70155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:55.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.128+0000 7f6d6dd9b700 1 -- 172.21.15.7:0/3614344511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d701540e0 con 0x7f6d70153690 2024-01-27T02:36:55.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.128+0000 7f6d6dd9b700 1 --2- 172.21.15.7:0/3614344511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d70153690 0x7f6d7014d780 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6d64007390 tx=0x7f6d64001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:55.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.128+0000 7f6d5e7fc700 1 -- 172.21.15.7:0/3614344511 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6400f6e0 con 0x7f6d70153690 2024-01-27T02:36:55.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.128+0000 7f6d5e7fc700 1 -- 172.21.15.7:0/3614344511 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6400f880 con 0x7f6d70153690 2024-01-27T02:36:55.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.128+0000 7f6d74c34700 1 -- 172.21.15.7:0/3614344511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d701543c0 con 0x7f6d70153690 2024-01-27T02:36:55.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.128+0000 7f6d5e7fc700 1 -- 172.21.15.7:0/3614344511 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6400fc40 con 0x7f6d70153690 2024-01-27T02:36:55.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.128+0000 7f6d74c34700 1 -- 172.21.15.7:0/3614344511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7014dcc0 con 0x7f6d70153690 2024-01-27T02:36:55.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.129+0000 7f6d74c34700 1 -- 172.21.15.7:0/3614344511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d70154600 con 0x7f6d70153690 2024-01-27T02:36:55.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.130+0000 7f6d5e7fc700 1 -- 172.21.15.7:0/3614344511 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6d6401d460 con 0x7f6d70153690 2024-01-27T02:36:55.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.130+0000 7f6d5e7fc700 1 --2- 172.21.15.7:0/3614344511 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6d50066f30 0x7f6d500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:55.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.130+0000 7f6d5e7fc700 1 -- 172.21.15.7:0/3614344511 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6d64084310 con 0x7f6d70153690 2024-01-27T02:36:55.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.130+0000 7f6d6e59c700 1 --2- 172.21.15.7:0/3614344511 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6d50066f30 0x7f6d500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:55.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.131+0000 7f6d6e59c700 1 --2- 172.21.15.7:0/3614344511 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6d50066f30 0x7f6d500693e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6d60006f00 tx=0x7f6d600091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:55.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.135+0000 7f6d5e7fc700 1 -- 172.21.15.7:0/3614344511 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d70154600 con 0x7f6d70153690 2024-01-27T02:36:55.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.339+0000 7f6d74c34700 1 -- 172.21.15.7:0/3614344511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6d701573e0 con 0x7f6d70153690 2024-01-27T02:36:55.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.343+0000 7f6d5e7fc700 1 -- 172.21.15.7:0/3614344511 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6d701573e0 con 0x7f6d70153690 2024-01-27T02:36:55.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.345+0000 7f6d74c34700 1 -- 172.21.15.7:0/3614344511 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6d50066f30 msgr2=0x7f6d500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:55.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.345+0000 7f6d74c34700 1 --2- 172.21.15.7:0/3614344511 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6d50066f30 0x7f6d500693e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6d60006f00 tx=0x7f6d600091a0 comp rx=0 tx=0).stop 2024-01-27T02:36:55.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.345+0000 7f6d74c34700 1 -- 172.21.15.7:0/3614344511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d70153690 msgr2=0x7f6d7014d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:55.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.345+0000 7f6d74c34700 1 --2- 172.21.15.7:0/3614344511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d70153690 0x7f6d7014d780 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6d64007390 tx=0x7f6d64001e50 comp rx=0 tx=0).stop 2024-01-27T02:36:55.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.345+0000 7f6d74c34700 1 -- 172.21.15.7:0/3614344511 shutdown_connections 2024-01-27T02:36:55.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.345+0000 7f6d74c34700 1 --2- 172.21.15.7:0/3614344511 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6d50066f30 0x7f6d500693e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:55.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.345+0000 7f6d74c34700 1 --2- 172.21.15.7:0/3614344511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d700b1700 0x7f6d700b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:55.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.345+0000 7f6d74c34700 1 --2- 172.21.15.7:0/3614344511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d580051c0 0x7f6d70155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:55.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.345+0000 7f6d74c34700 1 --2- 172.21.15.7:0/3614344511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d70153690 0x7f6d7014d780 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:55.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.345+0000 7f6d74c34700 1 -- 172.21.15.7:0/3614344511 >> 172.21.15.7:0/3614344511 conn(0x7f6d70068480 msgr2=0x7f6d7006a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:55.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.345+0000 7f6d74c34700 1 -- 172.21.15.7:0/3614344511 shutdown_connections 2024-01-27T02:36:55.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.346+0000 7f6d74c34700 1 -- 172.21.15.7:0/3614344511 wait complete. 2024-01-27T02:36:55.359 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2024-01-27T02:36:55.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.491+0000 7fc53c8d7700 1 Processor -- start 2024-01-27T02:36:55.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.491+0000 7fc53c8d7700 1 -- start start 2024-01-27T02:36:55.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.492+0000 7fc53c8d7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc5380c89a0 0x7fc53806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:55.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.492+0000 7fc53c8d7700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc5380c8d70 0x7fc538063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:55.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.492+0000 7fc53c8d7700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5380c4450 con 0x7fc538063ac0 2024-01-27T02:36:55.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.492+0000 7fc53c8d7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc53806e9e0 con 0x7fc5380c89a0 2024-01-27T02:36:55.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.492+0000 7fc53c8d7700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc538063f00 con 0x7fc5380c8d70 2024-01-27T02:36:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.492+0000 7fc53659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc5380c89a0 0x7fc53806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.492+0000 7fc53659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc5380c89a0 0x7fc53806ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49930/0 (socket says 172.21.15.7:49930) 2024-01-27T02:36:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.492+0000 7fc53659c700 1 -- 172.21.15.7:0/2414660732 learned_addr learned my addr 172.21.15.7:0/2414660732 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.493+0000 7fc535d9b700 1 --2- 172.21.15.7:0/2414660732 >> v2:172.21.15.188:3300/0 conn(0x7fc5380c8d70 0x7fc538063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.493+0000 7fc53659c700 1 -- 172.21.15.7:0/2414660732 >> v2:172.21.15.188:3300/0 conn(0x7fc5380c8d70 msgr2=0x7fc538063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.493+0000 7fc53659c700 1 --2- 172.21.15.7:0/2414660732 >> v2:172.21.15.188:3300/0 conn(0x7fc5380c8d70 0x7fc538063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.493+0000 7fc53659c700 1 -- 172.21.15.7:0/2414660732 >> v1:172.21.15.188:6789/0 conn(0x7fc538063ac0 legacy=0x7fc53815a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.493+0000 7fc53659c700 1 -- 172.21.15.7:0/2414660732 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53806e3d0 con 0x7fc5380c89a0 2024-01-27T02:36:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.493+0000 7fc53659c700 1 --2- 172.21.15.7:0/2414660732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5380c89a0 0x7fc53806ee70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc53806cee0 tx=0x7fc528009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efa4ee3caba3dce server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.493+0000 7fc53559a700 1 -- 172.21.15.7:0/2414660732 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52800b820 con 0x7fc5380c89a0 2024-01-27T02:36:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.493+0000 7fc53559a700 1 -- 172.21.15.7:0/2414660732 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52800b980 con 0x7fc5380c89a0 2024-01-27T02:36:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.493+0000 7fc53559a700 1 -- 172.21.15.7:0/2414660732 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52800bd20 con 0x7fc5380c89a0 2024-01-27T02:36:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.494+0000 7fc53c8d7700 1 -- 172.21.15.7:0/2414660732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5380c89a0 msgr2=0x7fc53806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.494+0000 7fc53c8d7700 1 --2- 172.21.15.7:0/2414660732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5380c89a0 0x7fc53806ee70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc53806cee0 tx=0x7fc528009250 comp rx=0 tx=0).stop 2024-01-27T02:36:55.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.494+0000 7fc53c8d7700 1 -- 172.21.15.7:0/2414660732 shutdown_connections 2024-01-27T02:36:55.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.494+0000 7fc53c8d7700 1 --2- 172.21.15.7:0/2414660732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5380c89a0 0x7fc53806ee70 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:55.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.494+0000 7fc53c8d7700 1 --2- 172.21.15.7:0/2414660732 >> v2:172.21.15.188:3300/0 conn(0x7fc5380c8d70 0x7fc538063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:55.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.494+0000 7fc53c8d7700 1 -- 172.21.15.7:0/2414660732 >> 172.21.15.7:0/2414660732 conn(0x7fc5380cdc30 msgr2=0x7fc5380ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:55.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.494+0000 7fc53c8d7700 1 -- 172.21.15.7:0/2414660732 shutdown_connections 2024-01-27T02:36:55.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.494+0000 7fc53c8d7700 1 -- 172.21.15.7:0/2414660732 wait complete. 2024-01-27T02:36:55.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.495+0000 7fc53c8d7700 1 Processor -- start 2024-01-27T02:36:55.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.495+0000 7fc53c8d7700 1 -- start start 2024-01-27T02:36:55.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.496+0000 7fc53c8d7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc538063ac0 0x7fc5380abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:55.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.496+0000 7fc53c8d7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5380c89a0 0x7fc53816aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:55.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.496+0000 7fc53c8d7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5380c8d70 0x7fc53816d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:55.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.496+0000 7fc53c8d7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5380ac160 con 0x7fc5380c89a0 2024-01-27T02:36:55.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.496+0000 7fc53c8d7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53816b020 con 0x7fc538063ac0 2024-01-27T02:36:55.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.496+0000 7fc53c8d7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc53816bad0 con 0x7fc5380c8d70 2024-01-27T02:36:55.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.496+0000 7fc535d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5380c89a0 0x7fc53816aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:55.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.496+0000 7fc535d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5380c89a0 0x7fc53816aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49938/0 (socket says 172.21.15.7:49938) 2024-01-27T02:36:55.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.496+0000 7fc535d9b700 1 -- 172.21.15.7:0/316998528 learned_addr learned my addr 172.21.15.7:0/316998528 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:55.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.496+0000 7fc53659c700 1 --2- 172.21.15.7:0/316998528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc538063ac0 0x7fc5380abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:55.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.496+0000 7fc536d9d700 1 --2- 172.21.15.7:0/316998528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5380c8d70 0x7fc53816d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:55.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.496+0000 7fc535d9b700 1 -- 172.21.15.7:0/316998528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5380c8d70 msgr2=0x7fc53816d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:55.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.496+0000 7fc535d9b700 1 --2- 172.21.15.7:0/316998528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5380c8d70 0x7fc53816d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:55.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.496+0000 7fc535d9b700 1 -- 172.21.15.7:0/316998528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc538063ac0 msgr2=0x7fc5380abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:55.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.496+0000 7fc535d9b700 1 --2- 172.21.15.7:0/316998528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc538063ac0 0x7fc5380abc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:55.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.496+0000 7fc535d9b700 1 -- 172.21.15.7:0/316998528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5280094f0 con 0x7fc5380c89a0 2024-01-27T02:36:55.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.497+0000 7fc535d9b700 1 --2- 172.21.15.7:0/316998528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5380c89a0 0x7fc53816aae0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc52000d890 tx=0x7fc520008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:55.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.497+0000 7fc52e7fc700 1 -- 172.21.15.7:0/316998528 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5200125f0 con 0x7fc5380c89a0 2024-01-27T02:36:55.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.497+0000 7fc52e7fc700 1 -- 172.21.15.7:0/316998528 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc520012770 con 0x7fc5380c89a0 2024-01-27T02:36:55.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.497+0000 7fc53c8d7700 1 -- 172.21.15.7:0/316998528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5380ad090 con 0x7fc5380c89a0 2024-01-27T02:36:55.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.497+0000 7fc52e7fc700 1 -- 172.21.15.7:0/316998528 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc520012b30 con 0x7fc5380c89a0 2024-01-27T02:36:55.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.497+0000 7fc53c8d7700 1 -- 172.21.15.7:0/316998528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc53816b510 con 0x7fc5380c89a0 2024-01-27T02:36:55.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.498+0000 7fc53c8d7700 1 -- 172.21.15.7:0/316998528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5380ad220 con 0x7fc5380c89a0 2024-01-27T02:36:55.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.498+0000 7fc52e7fc700 1 -- 172.21.15.7:0/316998528 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc52001e460 con 0x7fc5380c89a0 2024-01-27T02:36:55.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.499+0000 7fc52e7fc700 1 --2- 172.21.15.7:0/316998528 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc518066f30 0x7fc5180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:55.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.499+0000 7fc52e7fc700 1 -- 172.21.15.7:0/316998528 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fc520084800 con 0x7fc5380c89a0 2024-01-27T02:36:55.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.499+0000 7fc53659c700 1 --2- 172.21.15.7:0/316998528 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc518066f30 0x7fc5180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:55.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.499+0000 7fc53659c700 1 --2- 172.21.15.7:0/316998528 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc518066f30 0x7fc5180693e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc53806cee0 tx=0x7fc528016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:55.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.503+0000 7fc52e7fc700 1 -- 172.21.15.7:0/316998528 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc520051d70 con 0x7fc5380c89a0 2024-01-27T02:36:55.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.792+0000 7fc53c8d7700 1 -- 172.21.15.7:0/316998528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) v1 -- 0x7fc5380c48a0 con 0x7fc5380c89a0 2024-01-27T02:36:55.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.945+0000 7fc52e7fc700 1 -- 172.21.15.7:0/316998528 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v246) v1 ==== 133+0+0 (secure 0 0 0) 0x7fc520055390 con 0x7fc5380c89a0 2024-01-27T02:36:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.948+0000 7fc53c8d7700 1 -- 172.21.15.7:0/316998528 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc518066f30 msgr2=0x7fc5180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.948+0000 7fc53c8d7700 1 --2- 172.21.15.7:0/316998528 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc518066f30 0x7fc5180693e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc53806cee0 tx=0x7fc528016000 comp rx=0 tx=0).stop 2024-01-27T02:36:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.948+0000 7fc53c8d7700 1 -- 172.21.15.7:0/316998528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5380c89a0 msgr2=0x7fc53816aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.948+0000 7fc53c8d7700 1 --2- 172.21.15.7:0/316998528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5380c89a0 0x7fc53816aae0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc52000d890 tx=0x7fc520008040 comp rx=0 tx=0).stop 2024-01-27T02:36:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.948+0000 7fc53c8d7700 1 -- 172.21.15.7:0/316998528 shutdown_connections 2024-01-27T02:36:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.948+0000 7fc53c8d7700 1 --2- 172.21.15.7:0/316998528 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc518066f30 0x7fc5180693e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.948+0000 7fc53c8d7700 1 --2- 172.21.15.7:0/316998528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5380c8d70 0x7fc53816d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.948+0000 7fc53c8d7700 1 --2- 172.21.15.7:0/316998528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc538063ac0 0x7fc5380abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.948+0000 7fc53c8d7700 1 --2- 172.21.15.7:0/316998528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5380c89a0 0x7fc53816aae0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.948+0000 7fc53c8d7700 1 -- 172.21.15.7:0/316998528 >> 172.21.15.7:0/316998528 conn(0x7fc5380cdc30 msgr2=0x7fc5381579d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.948+0000 7fc53c8d7700 1 -- 172.21.15.7:0/316998528 shutdown_connections 2024-01-27T02:36:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:55.948+0000 7fc53c8d7700 1 -- 172.21.15.7:0/316998528 wait complete. 2024-01-27T02:36:55.963 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2024-01-27T02:36:55.963 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestClusterFull) 2024-01-27T02:36:55.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2024-01-27T02:36:56.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.099+0000 7f374588b700 1 Processor -- start 2024-01-27T02:36:56.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.099+0000 7f374588b700 1 -- start start 2024-01-27T02:36:56.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.100+0000 7f374588b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f374015d770 0x7f37400cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:56.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.100+0000 7f374588b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f374006ea60 0x7f374006eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:56.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.100+0000 7f374588b700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f374006e5d0 con 0x7f374015d3a0 2024-01-27T02:36:56.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.100+0000 7f374588b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37400cbe10 con 0x7f374006ea60 2024-01-27T02:36:56.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.100+0000 7f374588b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37400ce860 con 0x7f374015d770 2024-01-27T02:36:56.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.100+0000 7f373f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f374006ea60 0x7f374006eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:56.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.100+0000 7f373f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f374006ea60 0x7f374006eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49944/0 (socket says 172.21.15.7:49944) 2024-01-27T02:36:56.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.100+0000 7f373f7fe700 1 -- 172.21.15.7:0/1043942352 learned_addr learned my addr 172.21.15.7:0/1043942352 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:56.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.100+0000 7f373e7fc700 1 --2- 172.21.15.7:0/1043942352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f374015d770 0x7f37400cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:56.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.100+0000 7f373f7fe700 1 -- 172.21.15.7:0/1043942352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f374015d770 msgr2=0x7f37400cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:56.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.100+0000 7f373f7fe700 1 --2- 172.21.15.7:0/1043942352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f374015d770 0x7f37400cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:56.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.100+0000 7f373f7fe700 1 -- 172.21.15.7:0/1043942352 >> v1:172.21.15.188:6789/0 conn(0x7f374015d3a0 legacy=0x7f37400c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:56.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.100+0000 7f373f7fe700 1 -- 172.21.15.7:0/1043942352 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37400cd8f0 con 0x7f374006ea60 2024-01-27T02:36:56.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.100+0000 7f373e7fc700 1 --2- 172.21.15.7:0/1043942352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f374015d770 0x7f37400cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:36:56.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.101+0000 7f373f7fe700 1 --2- 172.21.15.7:0/1043942352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f374006ea60 0x7f374006eee0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3738009f20 tx=0x7f3738009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7379532c76a38f0e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:56.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.101+0000 7f373dffb700 1 -- 172.21.15.7:0/1043942352 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373800e070 con 0x7f374006ea60 2024-01-27T02:36:56.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.101+0000 7f373dffb700 1 -- 172.21.15.7:0/1043942352 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373800ba40 con 0x7f374006ea60 2024-01-27T02:36:56.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.101+0000 7f373dffb700 1 -- 172.21.15.7:0/1043942352 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37380092e0 con 0x7f374006ea60 2024-01-27T02:36:56.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.101+0000 7f374588b700 1 -- 172.21.15.7:0/1043942352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f374006ea60 msgr2=0x7f374006eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:56.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.101+0000 7f374588b700 1 --2- 172.21.15.7:0/1043942352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f374006ea60 0x7f374006eee0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3738009f20 tx=0x7f3738009840 comp rx=0 tx=0).stop 2024-01-27T02:36:56.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.101+0000 7f374588b700 1 -- 172.21.15.7:0/1043942352 shutdown_connections 2024-01-27T02:36:56.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.101+0000 7f374588b700 1 --2- 172.21.15.7:0/1043942352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f374006ea60 0x7f374006eee0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:56.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.101+0000 7f374588b700 1 --2- 172.21.15.7:0/1043942352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f374015d770 0x7f37400cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:56.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.101+0000 7f374588b700 1 -- 172.21.15.7:0/1043942352 >> 172.21.15.7:0/1043942352 conn(0x7f3740068480 msgr2=0x7f37400cdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:56.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.102+0000 7f374588b700 1 -- 172.21.15.7:0/1043942352 shutdown_connections 2024-01-27T02:36:56.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.102+0000 7f374588b700 1 -- 172.21.15.7:0/1043942352 wait complete. 2024-01-27T02:36:56.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.103+0000 7f374588b700 1 Processor -- start 2024-01-27T02:36:56.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.103+0000 7f374588b700 1 -- start start 2024-01-27T02:36:56.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.104+0000 7f374588b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f374006ea60 0x7f374014fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:56.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.104+0000 7f374588b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f374015d3a0 0x7f37400aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:56.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.104+0000 7f374588b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f374015d770 0x7f37400aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:56.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.104+0000 7f374588b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37400ad410 con 0x7f374015d770 2024-01-27T02:36:56.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.104+0000 7f374588b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3740150980 con 0x7f374015d3a0 2024-01-27T02:36:56.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.104+0000 7f374588b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37400ab960 con 0x7f374006ea60 2024-01-27T02:36:56.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.104+0000 7f373f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f374015d770 0x7f37400aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:56.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.104+0000 7f373f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f374015d770 0x7f37400aadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49958/0 (socket says 172.21.15.7:49958) 2024-01-27T02:36:56.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.104+0000 7f373f7fe700 1 -- 172.21.15.7:0/935342715 learned_addr learned my addr 172.21.15.7:0/935342715 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.104+0000 7f373effd700 1 --2- 172.21.15.7:0/935342715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f374006ea60 0x7f374014fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.104+0000 7f373e7fc700 1 --2- 172.21.15.7:0/935342715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f374015d3a0 0x7f37400aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.104+0000 7f373f7fe700 1 -- 172.21.15.7:0/935342715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f374006ea60 msgr2=0x7f374014fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.104+0000 7f373f7fe700 1 --2- 172.21.15.7:0/935342715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f374006ea60 0x7f374014fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.104+0000 7f373f7fe700 1 -- 172.21.15.7:0/935342715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f374015d3a0 msgr2=0x7f37400aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.104+0000 7f373f7fe700 1 --2- 172.21.15.7:0/935342715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f374015d3a0 0x7f37400aa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.104+0000 7f373f7fe700 1 -- 172.21.15.7:0/935342715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3738009ae0 con 0x7f374015d770 2024-01-27T02:36:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.104+0000 7f373e7fc700 1 --2- 172.21.15.7:0/935342715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f374015d3a0 0x7f37400aa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:36:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.104+0000 7f373f7fe700 1 --2- 172.21.15.7:0/935342715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f374015d770 0x7f37400aadf0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3740069ca0 tx=0x7f3738017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.105+0000 7f3736ffd700 1 -- 172.21.15.7:0/935342715 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3738012db0 con 0x7f374015d770 2024-01-27T02:36:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.105+0000 7f3736ffd700 1 -- 172.21.15.7:0/935342715 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373801e430 con 0x7f374015d770 2024-01-27T02:36:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.105+0000 7f374588b700 1 -- 172.21.15.7:0/935342715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37400abc40 con 0x7f374015d770 2024-01-27T02:36:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.105+0000 7f3736ffd700 1 -- 172.21.15.7:0/935342715 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373801e7f0 con 0x7f374015d770 2024-01-27T02:36:56.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.105+0000 7f374588b700 1 -- 172.21.15.7:0/935342715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37400ac200 con 0x7f374015d770 2024-01-27T02:36:56.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.106+0000 7f374588b700 1 -- 172.21.15.7:0/935342715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f374006dba0 con 0x7f374015d770 2024-01-27T02:36:56.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.106+0000 7f3736ffd700 1 -- 172.21.15.7:0/935342715 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f373801ebe0 con 0x7f374015d770 2024-01-27T02:36:56.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.106+0000 7f3736ffd700 1 --2- 172.21.15.7:0/935342715 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3720066f30 0x7f37200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:56.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.107+0000 7f3736ffd700 1 -- 172.21.15.7:0/935342715 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f3738085e60 con 0x7f374015d770 2024-01-27T02:36:56.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.107+0000 7f373effd700 1 --2- 172.21.15.7:0/935342715 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3720066f30 0x7f37200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:56.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.107+0000 7f373effd700 1 --2- 172.21.15.7:0/935342715 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3720066f30 0x7f37200693e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f37400c3810 tx=0x7f3730007ea0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:56.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.111+0000 7f3736ffd700 1 -- 172.21.15.7:0/935342715 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37380533d0 con 0x7f374015d770 2024-01-27T02:36:56.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.400+0000 7f374588b700 1 -- 172.21.15.7:0/935342715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) v1 -- 0x7f374004c710 con 0x7f374015d770 2024-01-27T02:36:56.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.947+0000 7f3736ffd700 1 -- 172.21.15.7:0/935342715 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v247) v1 ==== 140+0+0 (secure 0 0 0) 0x7f37380569f0 con 0x7f374015d770 2024-01-27T02:36:56.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.950+0000 7f374588b700 1 -- 172.21.15.7:0/935342715 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3720066f30 msgr2=0x7f37200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:56.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.950+0000 7f374588b700 1 --2- 172.21.15.7:0/935342715 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3720066f30 0x7f37200693e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f37400c3810 tx=0x7f3730007ea0 comp rx=0 tx=0).stop 2024-01-27T02:36:56.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.950+0000 7f374588b700 1 -- 172.21.15.7:0/935342715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f374015d770 msgr2=0x7f37400aadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:56.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.950+0000 7f374588b700 1 --2- 172.21.15.7:0/935342715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f374015d770 0x7f37400aadf0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3740069ca0 tx=0x7f3738017000 comp rx=0 tx=0).stop 2024-01-27T02:36:56.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.950+0000 7f374588b700 1 -- 172.21.15.7:0/935342715 shutdown_connections 2024-01-27T02:36:56.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.950+0000 7f374588b700 1 --2- 172.21.15.7:0/935342715 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3720066f30 0x7f37200693e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:56.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.950+0000 7f374588b700 1 --2- 172.21.15.7:0/935342715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f374015d770 0x7f37400aadf0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:56.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.950+0000 7f374588b700 1 --2- 172.21.15.7:0/935342715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f374006ea60 0x7f374014fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:56.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.950+0000 7f374588b700 1 --2- 172.21.15.7:0/935342715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f374015d3a0 0x7f37400aa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:56.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.950+0000 7f374588b700 1 -- 172.21.15.7:0/935342715 >> 172.21.15.7:0/935342715 conn(0x7f3740068480 msgr2=0x7f37400c2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:56.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.950+0000 7f374588b700 1 -- 172.21.15.7:0/935342715 shutdown_connections 2024-01-27T02:36:56.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:56.950+0000 7f374588b700 1 -- 172.21.15.7:0/935342715 wait complete. 2024-01-27T02:36:56.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-01-27T02:36:57.119 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_objectstore":"memstore"} 2024-01-27T02:36:57.119 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-01-27T02:36:57.119 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-27T02:36:57.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.253+0000 7fe2ed899700 1 Processor -- start 2024-01-27T02:36:57.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.254+0000 7fe2ed899700 1 -- start start 2024-01-27T02:36:57.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.254+0000 7fe2ed899700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e80ca3b0 0x7fe2e80aef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:57.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.254+0000 7fe2ed899700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e80b1450 con 0x7fe2e815a550 2024-01-27T02:36:57.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.254+0000 7fe2ed899700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e80b0840 con 0x7fe2e815a180 2024-01-27T02:36:57.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.254+0000 7fe2ed899700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e80b18e0 con 0x7fe2e80ca3b0 2024-01-27T02:36:57.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.255+0000 7fe2e67fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe2e815a550 0x7fe2e80c9ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60802/0 (socket says 172.21.15.7:60802) 2024-01-27T02:36:57.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.255+0000 7fe2e67fc700 1 -- 172.21.15.7:0/2653636067 learned_addr learned my addr 172.21.15.7:0/2653636067 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:57.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.255+0000 7fe2e77fe700 1 --2- 172.21.15.7:0/2653636067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e80ca3b0 0x7fe2e80aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:57.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.255+0000 7fe2e5ffb700 1 -- 172.21.15.7:0/2653636067 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2479020991 0 0) 0x7fe2e80b1450 con 0x7fe2e815a550 2024-01-27T02:36:57.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.255+0000 7fe2e5ffb700 1 -- 172.21.15.7:0/2653636067 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2d0003570 con 0x7fe2e815a550 2024-01-27T02:36:57.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.255+0000 7fe2e5ffb700 1 -- 172.21.15.7:0/2653636067 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276589552 0 0) 0x7fe2e80b0840 con 0x7fe2e815a180 2024-01-27T02:36:57.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.255+0000 7fe2e77fe700 1 -- 172.21.15.7:0/2653636067 >> v1:172.21.15.188:6789/0 conn(0x7fe2e815a180 legacy=0x7fe2e80c3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:57.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.255+0000 7fe2e77fe700 1 -- 172.21.15.7:0/2653636067 >> v1:172.21.15.7:6789/0 conn(0x7fe2e815a550 legacy=0x7fe2e80c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:57.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.255+0000 7fe2e77fe700 1 -- 172.21.15.7:0/2653636067 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e80b0b20 con 0x7fe2e80ca3b0 2024-01-27T02:36:57.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.255+0000 7fe2e77fe700 1 --2- 172.21.15.7:0/2653636067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e80ca3b0 0x7fe2e80aef20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0009e90 tx=0x7fe2e0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b9b985779f12a8a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:57.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.256+0000 7fe2e5ffb700 1 -- 172.21.15.7:0/2653636067 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e000e070 con 0x7fe2e80ca3b0 2024-01-27T02:36:57.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.256+0000 7fe2e5ffb700 1 -- 172.21.15.7:0/2653636067 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e000bc30 con 0x7fe2e80ca3b0 2024-01-27T02:36:57.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.256+0000 7fe2e5ffb700 1 -- 172.21.15.7:0/2653636067 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0012430 con 0x7fe2e80ca3b0 2024-01-27T02:36:57.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.256+0000 7fe2ed899700 1 -- 172.21.15.7:0/2653636067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e80ca3b0 msgr2=0x7fe2e80aef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:57.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.256+0000 7fe2ed899700 1 --2- 172.21.15.7:0/2653636067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e80ca3b0 0x7fe2e80aef20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0009e90 tx=0x7fe2e0009780 comp rx=0 tx=0).stop 2024-01-27T02:36:57.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.256+0000 7fe2ed899700 1 -- 172.21.15.7:0/2653636067 shutdown_connections 2024-01-27T02:36:57.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.256+0000 7fe2ed899700 1 --2- 172.21.15.7:0/2653636067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e80ca3b0 0x7fe2e80aef20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:57.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.256+0000 7fe2ed899700 1 -- 172.21.15.7:0/2653636067 >> 172.21.15.7:0/2653636067 conn(0x7fe2e80679b0 msgr2=0x7fe2e8067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:57.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.256+0000 7fe2ed899700 1 -- 172.21.15.7:0/2653636067 shutdown_connections 2024-01-27T02:36:57.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.256+0000 7fe2ed899700 1 -- 172.21.15.7:0/2653636067 wait complete. 2024-01-27T02:36:57.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.258+0000 7fe2ed899700 1 Processor -- start 2024-01-27T02:36:57.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.258+0000 7fe2ed899700 1 -- start start 2024-01-27T02:36:57.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.258+0000 7fe2ed899700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e80ca3b0 0x7fe2e809a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:57.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.258+0000 7fe2ed899700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e815a180 0x7fe2e80c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:57.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.258+0000 7fe2ed899700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2e815a550 0x7fe2e8098450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:57.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.258+0000 7fe2ed899700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e809acf0 con 0x7fe2e815a180 2024-01-27T02:36:57.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.258+0000 7fe2ed899700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e80c9220 con 0x7fe2e815a550 2024-01-27T02:36:57.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.258+0000 7fe2ed899700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e8099710 con 0x7fe2e80ca3b0 2024-01-27T02:36:57.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.258+0000 7fe2e67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e815a180 0x7fe2e80c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:57.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.259+0000 7fe2e67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e815a180 0x7fe2e80c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49974/0 (socket says 172.21.15.7:49974) 2024-01-27T02:36:57.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.259+0000 7fe2e67fc700 1 -- 172.21.15.7:0/2372236123 learned_addr learned my addr 172.21.15.7:0/2372236123 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:57.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.259+0000 7fe2e6ffd700 1 --2- 172.21.15.7:0/2372236123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e80ca3b0 0x7fe2e809a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:57.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.259+0000 7fe2e77fe700 1 --2- 172.21.15.7:0/2372236123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2e815a550 0x7fe2e8098450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:57.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.259+0000 7fe2e67fc700 1 -- 172.21.15.7:0/2372236123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e80ca3b0 msgr2=0x7fe2e809a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:57.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.259+0000 7fe2e67fc700 1 --2- 172.21.15.7:0/2372236123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e80ca3b0 0x7fe2e809a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:57.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.259+0000 7fe2e67fc700 1 -- 172.21.15.7:0/2372236123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2e815a550 msgr2=0x7fe2e8098450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:57.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.259+0000 7fe2e67fc700 1 --2- 172.21.15.7:0/2372236123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2e815a550 0x7fe2e8098450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:57.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.259+0000 7fe2e67fc700 1 -- 172.21.15.7:0/2372236123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e0009a20 con 0x7fe2e815a180 2024-01-27T02:36:57.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.259+0000 7fe2e67fc700 1 --2- 172.21.15.7:0/2372236123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e815a180 0x7fe2e80c8ce0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00c590 tx=0x7fe2dc007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:57.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.259+0000 7fe2ceffd700 1 -- 172.21.15.7:0/2372236123 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc005b30 con 0x7fe2e815a180 2024-01-27T02:36:57.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.259+0000 7fe2ceffd700 1 -- 172.21.15.7:0/2372236123 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2dc005c90 con 0x7fe2e815a180 2024-01-27T02:36:57.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.259+0000 7fe2ceffd700 1 -- 172.21.15.7:0/2372236123 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0072e0 con 0x7fe2e815a180 2024-01-27T02:36:57.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.259+0000 7fe2ed899700 1 -- 172.21.15.7:0/2372236123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e80999f0 con 0x7fe2e815a180 2024-01-27T02:36:57.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.259+0000 7fe2ed899700 1 -- 172.21.15.7:0/2372236123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e8099fb0 con 0x7fe2e815a180 2024-01-27T02:36:57.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.260+0000 7fe2ed899700 1 -- 172.21.15.7:0/2372236123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e8072060 con 0x7fe2e815a180 2024-01-27T02:36:57.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.261+0000 7fe2ceffd700 1 -- 172.21.15.7:0/2372236123 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe2dc007690 con 0x7fe2e815a180 2024-01-27T02:36:57.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.261+0000 7fe2ceffd700 1 --2- 172.21.15.7:0/2372236123 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe2d4066f30 0x7fe2d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:57.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.261+0000 7fe2ceffd700 1 -- 172.21.15.7:0/2372236123 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fe2dc083560 con 0x7fe2e815a180 2024-01-27T02:36:57.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.261+0000 7fe2e6ffd700 1 --2- 172.21.15.7:0/2372236123 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe2d4066f30 0x7fe2d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:57.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.262+0000 7fe2e6ffd700 1 --2- 172.21.15.7:0/2372236123 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe2d4066f30 0x7fe2d40693e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8007fd0 tx=0x7fe2d8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:57.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.265+0000 7fe2ceffd700 1 -- 172.21.15.7:0/2372236123 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2dc050ad0 con 0x7fe2e815a180 2024-01-27T02:36:57.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.467+0000 7fe2ed899700 1 -- 172.21.15.7:0/2372236123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe2e80cacb0 con 0x7fe2e815a180 2024-01-27T02:36:57.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.471+0000 7fe2ceffd700 1 -- 172.21.15.7:0/2372236123 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe2dc0170c0 con 0x7fe2e815a180 2024-01-27T02:36:57.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.473+0000 7fe2ed899700 1 -- 172.21.15.7:0/2372236123 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe2d4066f30 msgr2=0x7fe2d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:57.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.473+0000 7fe2ed899700 1 --2- 172.21.15.7:0/2372236123 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe2d4066f30 0x7fe2d40693e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8007fd0 tx=0x7fe2d8009040 comp rx=0 tx=0).stop 2024-01-27T02:36:57.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.473+0000 7fe2ed899700 1 -- 172.21.15.7:0/2372236123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e815a180 msgr2=0x7fe2e80c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:57.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.473+0000 7fe2ed899700 1 --2- 172.21.15.7:0/2372236123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e815a180 0x7fe2e80c8ce0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00c590 tx=0x7fe2dc007d90 comp rx=0 tx=0).stop 2024-01-27T02:36:57.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.473+0000 7fe2ed899700 1 -- 172.21.15.7:0/2372236123 shutdown_connections 2024-01-27T02:36:57.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.473+0000 7fe2ed899700 1 --2- 172.21.15.7:0/2372236123 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe2d4066f30 0x7fe2d40693e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:57.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.473+0000 7fe2ed899700 1 --2- 172.21.15.7:0/2372236123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2e815a550 0x7fe2e8098450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:57.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.473+0000 7fe2ed899700 1 --2- 172.21.15.7:0/2372236123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2e80ca3b0 0x7fe2e809a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:57.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.473+0000 7fe2ed899700 1 --2- 172.21.15.7:0/2372236123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e815a180 0x7fe2e80c8ce0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:57.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.473+0000 7fe2ed899700 1 -- 172.21.15.7:0/2372236123 >> 172.21.15.7:0/2372236123 conn(0x7fe2e80679b0 msgr2=0x7fe2e806dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:57.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.474+0000 7fe2ed899700 1 -- 172.21.15.7:0/2372236123 shutdown_connections 2024-01-27T02:36:57.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.474+0000 7fe2ed899700 1 -- 172.21.15.7:0/2372236123 wait complete. 2024-01-27T02:36:57.488 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-27T02:36:57.488 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:36:57.531 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:36:57.577 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-27T02:36:57.577 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-27T02:36:57.577 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:36:57.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.712+0000 7f239ec88700 1 Processor -- start 2024-01-27T02:36:57.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.712+0000 7f239ec88700 1 -- start start 2024-01-27T02:36:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.713+0000 7f239ec88700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f23980c9bb0 0x7f2398157f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.713+0000 7f239ec88700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f239806e330 0x7f239806e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.713+0000 7f239ec88700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f239806c880 con 0x7f23980c9f80 2024-01-27T02:36:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.713+0000 7f239ec88700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f239806cd10 con 0x7f23980c9bb0 2024-01-27T02:36:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.713+0000 7f239ec88700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f239806ec90 con 0x7f239806e330 2024-01-27T02:36:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.713+0000 7f239ca24700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f23980c9bb0 0x7f2398157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.713+0000 7f2397fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f23980c9f80 0x7f239806b400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60804/0 (socket says 172.21.15.7:60804) 2024-01-27T02:36:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.713+0000 7f2397fff700 1 -- 172.21.15.7:0/2276198655 learned_addr learned my addr 172.21.15.7:0/2276198655 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.713+0000 7f239d225700 1 --2- 172.21.15.7:0/2276198655 >> v2:172.21.15.188:3300/0 conn(0x7f239806e330 0x7f239806e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.714+0000 7f23977fe700 1 -- 172.21.15.7:0/2276198655 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3672838855 0 0) 0x7f239806c880 con 0x7f23980c9f80 2024-01-27T02:36:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.714+0000 7f23977fe700 1 -- 172.21.15.7:0/2276198655 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2380003570 con 0x7f23980c9f80 2024-01-27T02:36:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.714+0000 7f239ca24700 1 -- 172.21.15.7:0/2276198655 >> v2:172.21.15.188:3300/0 conn(0x7f239806e330 msgr2=0x7f239806e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.714+0000 7f239ca24700 1 --2- 172.21.15.7:0/2276198655 >> v2:172.21.15.188:3300/0 conn(0x7f239806e330 0x7f239806e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.714+0000 7f239ca24700 1 -- 172.21.15.7:0/2276198655 >> v1:172.21.15.7:6789/0 conn(0x7f23980c9f80 legacy=0x7f239806b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.714+0000 7f239ca24700 1 -- 172.21.15.7:0/2276198655 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239806bc70 con 0x7f23980c9bb0 2024-01-27T02:36:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.714+0000 7f239ca24700 1 --2- 172.21.15.7:0/2276198655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23980c9bb0 0x7f2398157f70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f239806db30 tx=0x7f23880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c2150cb2e1380b server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.714+0000 7f23977fe700 1 -- 172.21.15.7:0/2276198655 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388011630 con 0x7f23980c9bb0 2024-01-27T02:36:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.714+0000 7f23977fe700 1 -- 172.21.15.7:0/2276198655 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2388011790 con 0x7f23980c9bb0 2024-01-27T02:36:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.714+0000 7f23977fe700 1 -- 172.21.15.7:0/2276198655 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388011b30 con 0x7f23980c9bb0 2024-01-27T02:36:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.715+0000 7f239ec88700 1 -- 172.21.15.7:0/2276198655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23980c9bb0 msgr2=0x7f2398157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.715+0000 7f239ec88700 1 --2- 172.21.15.7:0/2276198655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23980c9bb0 0x7f2398157f70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f239806db30 tx=0x7f23880091a0 comp rx=0 tx=0).stop 2024-01-27T02:36:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.715+0000 7f239ec88700 1 -- 172.21.15.7:0/2276198655 shutdown_connections 2024-01-27T02:36:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.715+0000 7f239ec88700 1 --2- 172.21.15.7:0/2276198655 >> v2:172.21.15.188:3300/0 conn(0x7f239806e330 0x7f239806e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.715+0000 7f239ec88700 1 --2- 172.21.15.7:0/2276198655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23980c9bb0 0x7f2398157f70 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.715+0000 7f239ec88700 1 -- 172.21.15.7:0/2276198655 >> 172.21.15.7:0/2276198655 conn(0x7f23980cdc30 msgr2=0x7f23980ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.715+0000 7f239ec88700 1 -- 172.21.15.7:0/2276198655 shutdown_connections 2024-01-27T02:36:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.715+0000 7f239ec88700 1 -- 172.21.15.7:0/2276198655 wait complete. 2024-01-27T02:36:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.716+0000 7f239ec88700 1 Processor -- start 2024-01-27T02:36:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.717+0000 7f239ec88700 1 -- start start 2024-01-27T02:36:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.718+0000 7f239ec88700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f239806e330 0x7f23980b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.718+0000 7f239ec88700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23980c9bb0 0x7f2398166fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.718+0000 7f239ec88700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23980c9f80 0x7f23980c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.718+0000 7f239ec88700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2398139560 con 0x7f23980c9f80 2024-01-27T02:36:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.718+0000 7f239ec88700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23980c65a0 con 0x7f239806e330 2024-01-27T02:36:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.718+0000 7f239ec88700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23980c6880 con 0x7f23980c9bb0 2024-01-27T02:36:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.718+0000 7f239d225700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23980c9f80 0x7f23980c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.718+0000 7f239d225700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23980c9f80 0x7f23980c5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49982/0 (socket says 172.21.15.7:49982) 2024-01-27T02:36:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.718+0000 7f239d225700 1 -- 172.21.15.7:0/1722150943 learned_addr learned my addr 172.21.15.7:0/1722150943 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.718+0000 7f2397fff700 1 --2- 172.21.15.7:0/1722150943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23980c9bb0 0x7f2398166fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.718+0000 7f239ca24700 1 --2- 172.21.15.7:0/1722150943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f239806e330 0x7f23980b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.718+0000 7f239d225700 1 -- 172.21.15.7:0/1722150943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23980c9bb0 msgr2=0x7f2398166fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.718+0000 7f239d225700 1 --2- 172.21.15.7:0/1722150943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23980c9bb0 0x7f2398166fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.718+0000 7f239d225700 1 -- 172.21.15.7:0/1722150943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f239806e330 msgr2=0x7f23980b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.718+0000 7f239d225700 1 --2- 172.21.15.7:0/1722150943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f239806e330 0x7f23980b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.718+0000 7f239d225700 1 -- 172.21.15.7:0/1722150943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2388009420 con 0x7f23980c9f80 2024-01-27T02:36:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.718+0000 7f239d225700 1 --2- 172.21.15.7:0/1722150943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23980c9f80 0x7f23980c5b50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f2390009fa0 tx=0x7f2390009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.718+0000 7f2394ff9700 1 -- 172.21.15.7:0/1722150943 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239000ed20 con 0x7f23980c9f80 2024-01-27T02:36:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.718+0000 7f2394ff9700 1 -- 172.21.15.7:0/1722150943 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239000ee80 con 0x7f23980c9f80 2024-01-27T02:36:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.718+0000 7f239ec88700 1 -- 172.21.15.7:0/1722150943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23980c6b60 con 0x7f23980c9f80 2024-01-27T02:36:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.719+0000 7f2394ff9700 1 -- 172.21.15.7:0/1722150943 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2390014670 con 0x7f23980c9f80 2024-01-27T02:36:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.719+0000 7f239ec88700 1 -- 172.21.15.7:0/1722150943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23980c71a0 con 0x7f23980c9f80 2024-01-27T02:36:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.719+0000 7f239ec88700 1 -- 172.21.15.7:0/1722150943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f239804c710 con 0x7f23980c9f80 2024-01-27T02:36:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.720+0000 7f2394ff9700 1 -- 172.21.15.7:0/1722150943 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f23900149e0 con 0x7f23980c9f80 2024-01-27T02:36:57.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.720+0000 7f2394ff9700 1 --2- 172.21.15.7:0/1722150943 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2384066f30 0x7f23840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:57.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.720+0000 7f2394ff9700 1 -- 172.21.15.7:0/1722150943 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2390084a70 con 0x7f23980c9f80 2024-01-27T02:36:57.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.721+0000 7f239ca24700 1 --2- 172.21.15.7:0/1722150943 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2384066f30 0x7f23840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:57.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.721+0000 7f239ca24700 1 --2- 172.21.15.7:0/1722150943 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2384066f30 0x7f23840693e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f23880093f0 tx=0x7f2388017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:57.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.725+0000 7f2394ff9700 1 -- 172.21.15.7:0/1722150943 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2390051fe0 con 0x7f23980c9f80 2024-01-27T02:36:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.976+0000 7f239ec88700 1 -- 172.21.15.7:0/1722150943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23981503a0 con 0x7f23980c9f80 2024-01-27T02:36:57.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.976+0000 7f2394ff9700 1 -- 172.21.15.7:0/1722150943 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+3520 (secure 0 0 0) 0x7f2390055600 con 0x7f23980c9f80 2024-01-27T02:36:57.977 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:36:57.977 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":44,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2000106193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2000106193},{"type":"v1","addr":"172.21.15.188:6836","nonce":2000106193}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5839,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/100533817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":100533817},{"type":"v1","addr":"172.21.15.7:6834","nonce":100533817}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5842,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/223425097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":223425097},{"type":"v1","addr":"172.21.15.7:6835","nonce":223425097}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5848,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/950040140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":950040140},{"type":"v1","addr":"172.21.15.188:6837","nonce":950040140}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[]} 2024-01-27T02:36:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.978+0000 7f239ec88700 1 -- 172.21.15.7:0/1722150943 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2384066f30 msgr2=0x7f23840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.978+0000 7f239ec88700 1 --2- 172.21.15.7:0/1722150943 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2384066f30 0x7f23840693e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f23880093f0 tx=0x7f2388017000 comp rx=0 tx=0).stop 2024-01-27T02:36:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.978+0000 7f239ec88700 1 -- 172.21.15.7:0/1722150943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23980c9f80 msgr2=0x7f23980c5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.978+0000 7f239ec88700 1 --2- 172.21.15.7:0/1722150943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23980c9f80 0x7f23980c5b50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f2390009fa0 tx=0x7f2390009900 comp rx=0 tx=0).stop 2024-01-27T02:36:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.979+0000 7f239ec88700 1 -- 172.21.15.7:0/1722150943 shutdown_connections 2024-01-27T02:36:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.979+0000 7f239ec88700 1 --2- 172.21.15.7:0/1722150943 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2384066f30 0x7f23840693e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.979+0000 7f239ec88700 1 --2- 172.21.15.7:0/1722150943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23980c9f80 0x7f23980c5b50 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.979+0000 7f239ec88700 1 --2- 172.21.15.7:0/1722150943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f239806e330 0x7f23980b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.979+0000 7f239ec88700 1 --2- 172.21.15.7:0/1722150943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23980c9bb0 0x7f2398166fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.979+0000 7f239ec88700 1 -- 172.21.15.7:0/1722150943 >> 172.21.15.7:0/1722150943 conn(0x7f23980cdc30 msgr2=0x7f2398158540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.979+0000 7f239ec88700 1 -- 172.21.15.7:0/1722150943 shutdown_connections 2024-01-27T02:36:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:57.979+0000 7f239ec88700 1 -- 172.21.15.7:0/1722150943 wait complete. 2024-01-27T02:36:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 44 2024-01-27T02:36:57.993 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-27T02:36:57.993 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-27T02:36:57.993 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-27T02:36:57.993 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:36:57.993 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-27T02:36:57.993 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-27T02:36:57.994 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-27T02:36:57.994 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:36:57.994 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-27T02:36:57.994 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-27T02:36:57.994 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-27T02:36:57.994 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:36:57.994 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-27T02:36:57.994 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-27T02:36:57.995 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-27T02:36:57.995 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:36:58.003 INFO:tasks.ceph.mds.c:Stopped 2024-01-27T02:36:58.003 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-27T02:36:58.003 INFO:tasks.ceph.mds.a:Stopped 2024-01-27T02:36:58.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-27T02:36:58.005 INFO:tasks.ceph.mds.b:Stopped 2024-01-27T02:36:58.005 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-27T02:36:58.006 INFO:tasks.ceph.mds.d:Stopped 2024-01-27T02:36:58.006 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-27T02:36:58.046 INFO:tasks.ceph.mds.c:Started 2024-01-27T02:36:58.046 DEBUG:teuthology.parallel:result is None 2024-01-27T02:36:58.047 INFO:tasks.ceph.mds.a:Started 2024-01-27T02:36:58.047 DEBUG:teuthology.parallel:result is None 2024-01-27T02:36:58.049 INFO:tasks.ceph.mds.b:Started 2024-01-27T02:36:58.049 DEBUG:teuthology.parallel:result is None 2024-01-27T02:36:58.049 INFO:tasks.ceph.mds.d:Started 2024-01-27T02:36:58.050 DEBUG:teuthology.parallel:result is None 2024-01-27T02:36:58.050 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-27T02:36:58.133 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-27T02:36:58.134 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-27T02:36:58.136 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-01-27T02:36:58.136 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-01-27T02:36:58.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.189+0000 7f90a9483700 1 Processor -- start 2024-01-27T02:36:58.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.189+0000 7f90a9483700 1 -- start start 2024-01-27T02:36:58.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.190+0000 7f90a9483700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f90a40d1470 0x7f90a40b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:58.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.190+0000 7f90a9483700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a40b4650 con 0x7f90a40d10a0 2024-01-27T02:36:58.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.190+0000 7f90a9483700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a40b3d60 con 0x7f90a4159b40 2024-01-27T02:36:58.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.190+0000 7f90a9483700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a40b34d0 con 0x7f90a40d1470 2024-01-27T02:36:58.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.190+0000 7f90a2ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f90a40d10a0 0x7f90a40b2dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60826/0 (socket says 172.21.15.7:60826) 2024-01-27T02:36:58.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.190+0000 7f90a2ffd700 1 -- 172.21.15.7:0/1671312837 learned_addr learned my addr 172.21.15.7:0/1671312837 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:36:58.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.190+0000 7f90a27fc700 1 --2- 172.21.15.7:0/1671312837 >> v2:172.21.15.188:3300/0 conn(0x7f90a40d1470 0x7f90a40b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:58.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.190+0000 7f90a1ffb700 1 -- 172.21.15.7:0/1671312837 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1943187079 0 0) 0x7f90a40b4650 con 0x7f90a40d10a0 2024-01-27T02:36:58.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.190+0000 7f90a1ffb700 1 -- 172.21.15.7:0/1671312837 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f908c003570 con 0x7f90a40d10a0 2024-01-27T02:36:58.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.190+0000 7f90a27fc700 1 -- 172.21.15.7:0/1671312837 >> v1:172.21.15.188:6789/0 conn(0x7f90a4159b40 legacy=0x7f90a4159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:58.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.190+0000 7f90a1ffb700 1 -- 172.21.15.7:0/1671312837 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1808058516 0 0) 0x7f90a40b3d60 con 0x7f90a4159b40 2024-01-27T02:36:58.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.190+0000 7f90a27fc700 1 -- 172.21.15.7:0/1671312837 >> v1:172.21.15.7:6789/0 conn(0x7f90a40d10a0 legacy=0x7f90a40b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:58.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.190+0000 7f90a27fc700 1 -- 172.21.15.7:0/1671312837 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a40b4000 con 0x7f90a40d1470 2024-01-27T02:36:58.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.191+0000 7f90a27fc700 1 --2- 172.21.15.7:0/1671312837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90a40d1470 0x7f90a40b1590 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f90980098e0 tx=0x7f90980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f259ec3b11e444c8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:58.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.191+0000 7f90a1ffb700 1 -- 172.21.15.7:0/1671312837 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909800b640 con 0x7f90a40d1470 2024-01-27T02:36:58.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.191+0000 7f90a1ffb700 1 -- 172.21.15.7:0/1671312837 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909800b7a0 con 0x7f90a40d1470 2024-01-27T02:36:58.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.191+0000 7f90a1ffb700 1 -- 172.21.15.7:0/1671312837 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909800bb40 con 0x7f90a40d1470 2024-01-27T02:36:58.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.191+0000 7f90a9483700 1 -- 172.21.15.7:0/1671312837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90a40d1470 msgr2=0x7f90a40b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:58.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.191+0000 7f90a9483700 1 --2- 172.21.15.7:0/1671312837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90a40d1470 0x7f90a40b1590 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f90980098e0 tx=0x7f90980091a0 comp rx=0 tx=0).stop 2024-01-27T02:36:58.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.191+0000 7f90a9483700 1 -- 172.21.15.7:0/1671312837 shutdown_connections 2024-01-27T02:36:58.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.191+0000 7f90a9483700 1 --2- 172.21.15.7:0/1671312837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90a40d1470 0x7f90a40b1590 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:58.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.191+0000 7f90a9483700 1 -- 172.21.15.7:0/1671312837 >> 172.21.15.7:0/1671312837 conn(0x7f90a40679b0 msgr2=0x7f90a4067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.191+0000 7f90a9483700 1 -- 172.21.15.7:0/1671312837 shutdown_connections 2024-01-27T02:36:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.191+0000 7f90a9483700 1 -- 172.21.15.7:0/1671312837 wait complete. 2024-01-27T02:36:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.193+0000 7f90a9483700 1 Processor -- start 2024-01-27T02:36:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.193+0000 7f90a9483700 1 -- start start 2024-01-27T02:36:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.194+0000 7f90a9483700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90a40d10a0 0x7f90a40ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.194+0000 7f90a9483700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90a40d1470 0x7f90a41503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.194+0000 7f90a9483700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a4159b40 0x7f90a40cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.194+0000 7f90a9483700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a40c4e80 con 0x7f90a4159b40 2024-01-27T02:36:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.194+0000 7f90a9483700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a40c5160 con 0x7f90a40d10a0 2024-01-27T02:36:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.194+0000 7f90a9483700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a40c5440 con 0x7f90a40d1470 2024-01-27T02:36:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.194+0000 7f90a37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a4159b40 0x7f90a40cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.194+0000 7f90a37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a4159b40 0x7f90a40cfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50016/0 (socket says 172.21.15.7:50016) 2024-01-27T02:36:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.194+0000 7f90a37fe700 1 -- 172.21.15.7:0/2962301215 learned_addr learned my addr 172.21.15.7:0/2962301215 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.194+0000 7f90a2ffd700 1 --2- 172.21.15.7:0/2962301215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90a40d10a0 0x7f90a40ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.194+0000 7f90a27fc700 1 --2- 172.21.15.7:0/2962301215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90a40d1470 0x7f90a41503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:58.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.194+0000 7f90a37fe700 1 -- 172.21.15.7:0/2962301215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90a40d1470 msgr2=0x7f90a41503f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:58.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.194+0000 7f90a37fe700 1 --2- 172.21.15.7:0/2962301215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90a40d1470 0x7f90a41503f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:58.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.194+0000 7f90a37fe700 1 -- 172.21.15.7:0/2962301215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90a40d10a0 msgr2=0x7f90a40ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:58.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.194+0000 7f90a37fe700 1 --2- 172.21.15.7:0/2962301215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90a40d10a0 0x7f90a40ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:58.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.194+0000 7f90a37fe700 1 -- 172.21.15.7:0/2962301215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9098009420 con 0x7f90a4159b40 2024-01-27T02:36:58.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.194+0000 7f90a2ffd700 1 --2- 172.21.15.7:0/2962301215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90a40d10a0 0x7f90a40ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:36:58.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.194+0000 7f90a37fe700 1 --2- 172.21.15.7:0/2962301215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a4159b40 0x7f90a40cfc30 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f909c00c890 tx=0x7f909c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:58.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.194+0000 7f908affd700 1 -- 172.21.15.7:0/2962301215 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c017070 con 0x7f90a4159b40 2024-01-27T02:36:58.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.195+0000 7f908affd700 1 -- 172.21.15.7:0/2962301215 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909c006ba0 con 0x7f90a4159b40 2024-01-27T02:36:58.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.195+0000 7f90a9483700 1 -- 172.21.15.7:0/2962301215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a40c5720 con 0x7f90a4159b40 2024-01-27T02:36:58.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.195+0000 7f908affd700 1 -- 172.21.15.7:0/2962301215 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c012630 con 0x7f90a4159b40 2024-01-27T02:36:58.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.195+0000 7f90a9483700 1 -- 172.21.15.7:0/2962301215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a4150930 con 0x7f90a4159b40 2024-01-27T02:36:58.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.196+0000 7f90a9483700 1 -- 172.21.15.7:0/2962301215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a404c710 con 0x7f90a4159b40 2024-01-27T02:36:58.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.196+0000 7f908affd700 1 -- 172.21.15.7:0/2962301215 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f909c012a60 con 0x7f90a4159b40 2024-01-27T02:36:58.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.197+0000 7f908affd700 1 --2- 172.21.15.7:0/2962301215 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9090066f30 0x7f90900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:58.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.197+0000 7f908affd700 1 -- 172.21.15.7:0/2962301215 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f909c084040 con 0x7f90a4159b40 2024-01-27T02:36:58.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.197+0000 7f90a2ffd700 1 --2- 172.21.15.7:0/2962301215 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9090066f30 0x7f90900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:58.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.197+0000 7f90a2ffd700 1 --2- 172.21.15.7:0/2962301215 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9090066f30 0x7f90900693e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f90940058d0 tx=0x7f9094009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:58.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.201+0000 7f908affd700 1 -- 172.21.15.7:0/2962301215 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f909c0515b0 con 0x7f90a4159b40 2024-01-27T02:36:58.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:58.423+0000 7f90a9483700 1 -- 172.21.15.7:0/2962301215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f90a40b42e0 con 0x7f90a4159b40 2024-01-27T02:36:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.495+0000 7f908affd700 1 -- 172.21.15.7:0/2962301215 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v94) v1 ==== 97+0+0 (secure 0 0 0) 0x7f909c054bd0 con 0x7f90a4159b40 2024-01-27T02:36:59.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.497+0000 7f90a9483700 1 -- 172.21.15.7:0/2962301215 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9090066f30 msgr2=0x7f90900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:59.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.497+0000 7f90a9483700 1 --2- 172.21.15.7:0/2962301215 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9090066f30 0x7f90900693e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f90940058d0 tx=0x7f9094009040 comp rx=0 tx=0).stop 2024-01-27T02:36:59.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.497+0000 7f90a9483700 1 -- 172.21.15.7:0/2962301215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a4159b40 msgr2=0x7f90a40cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:59.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.497+0000 7f90a9483700 1 --2- 172.21.15.7:0/2962301215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a4159b40 0x7f90a40cfc30 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f909c00c890 tx=0x7f909c001670 comp rx=0 tx=0).stop 2024-01-27T02:36:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.498+0000 7f90a9483700 1 -- 172.21.15.7:0/2962301215 shutdown_connections 2024-01-27T02:36:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.498+0000 7f90a9483700 1 --2- 172.21.15.7:0/2962301215 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9090066f30 0x7f90900693e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.498+0000 7f90a9483700 1 --2- 172.21.15.7:0/2962301215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a4159b40 0x7f90a40cfc30 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.498+0000 7f90a9483700 1 --2- 172.21.15.7:0/2962301215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90a40d10a0 0x7f90a40ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.498+0000 7f90a9483700 1 --2- 172.21.15.7:0/2962301215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90a40d1470 0x7f90a41503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.498+0000 7f90a9483700 1 -- 172.21.15.7:0/2962301215 >> 172.21.15.7:0/2962301215 conn(0x7f90a40679b0 msgr2=0x7f90a4159f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.498+0000 7f90a9483700 1 -- 172.21.15.7:0/2962301215 shutdown_connections 2024-01-27T02:36:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.498+0000 7f90a9483700 1 -- 172.21.15.7:0/2962301215 wait complete. 2024-01-27T02:36:59.499 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-01-27T02:36:59.513 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-27T02:36:59.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.649+0000 7fef96ab8700 1 Processor -- start 2024-01-27T02:36:59.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.649+0000 7fef96ab8700 1 -- start start 2024-01-27T02:36:59.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.650+0000 7fef96ab8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef9015a100 0x7fef900cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:59.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.650+0000 7fef96ab8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fef900c4cc0 0x7fef90163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:59.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.650+0000 7fef96ab8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef900ce5e0 con 0x7fef900cb5b0 2024-01-27T02:36:59.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.650+0000 7fef96ab8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef900cd9d0 con 0x7fef900c4cc0 2024-01-27T02:36:59.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.650+0000 7fef96ab8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef900cdcb0 con 0x7fef9015a100 2024-01-27T02:36:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.650+0000 7fef95055700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fef900c4cc0 0x7fef90163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.650+0000 7fef95055700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fef900c4cc0 0x7fef90163d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50032/0 (socket says 172.21.15.7:50032) 2024-01-27T02:36:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.650+0000 7fef95055700 1 -- 172.21.15.7:0/4038646206 learned_addr learned my addr 172.21.15.7:0/4038646206 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.650+0000 7fef8ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fef900cb5b0 0x7fef900c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60842/0 (socket says 172.21.15.7:60842) 2024-01-27T02:36:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.650+0000 7fef94854700 1 --2- 172.21.15.7:0/4038646206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef9015a100 0x7fef900cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.651+0000 7fef95055700 1 -- 172.21.15.7:0/4038646206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef9015a100 msgr2=0x7fef900cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:36:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.651+0000 7fef95055700 1 --2- 172.21.15.7:0/4038646206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef9015a100 0x7fef900cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.651+0000 7fef95055700 1 -- 172.21.15.7:0/4038646206 >> v1:172.21.15.7:6789/0 conn(0x7fef900cb5b0 legacy=0x7fef900c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.651+0000 7fef95055700 1 -- 172.21.15.7:0/4038646206 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef900cdf90 con 0x7fef900c4cc0 2024-01-27T02:36:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.651+0000 7fef8f7fe700 1 -- 172.21.15.7:0/4038646206 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986611572 0 0) 0x7fef900ce5e0 con 0x7fef900cb5b0 2024-01-27T02:36:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.651+0000 7fef94854700 1 --2- 172.21.15.7:0/4038646206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef9015a100 0x7fef900cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:36:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.651+0000 7fef95055700 1 --2- 172.21.15.7:0/4038646206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef900c4cc0 0x7fef90163d90 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fef88009f20 tx=0x7fef88009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b09c6211137eefd server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.651+0000 7fef8f7fe700 1 -- 172.21.15.7:0/4038646206 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8800e070 con 0x7fef900c4cc0 2024-01-27T02:36:59.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.651+0000 7fef8f7fe700 1 -- 172.21.15.7:0/4038646206 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8800ba80 con 0x7fef900c4cc0 2024-01-27T02:36:59.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.651+0000 7fef8f7fe700 1 -- 172.21.15.7:0/4038646206 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef880092e0 con 0x7fef900c4cc0 2024-01-27T02:36:59.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.651+0000 7fef96ab8700 1 -- 172.21.15.7:0/4038646206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef900c4cc0 msgr2=0x7fef90163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:59.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.651+0000 7fef96ab8700 1 --2- 172.21.15.7:0/4038646206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef900c4cc0 0x7fef90163d90 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fef88009f20 tx=0x7fef88009840 comp rx=0 tx=0).stop 2024-01-27T02:36:59.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.651+0000 7fef96ab8700 1 -- 172.21.15.7:0/4038646206 shutdown_connections 2024-01-27T02:36:59.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.651+0000 7fef96ab8700 1 --2- 172.21.15.7:0/4038646206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef900c4cc0 0x7fef90163d90 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:59.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.651+0000 7fef96ab8700 1 --2- 172.21.15.7:0/4038646206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef9015a100 0x7fef900cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:59.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.652+0000 7fef96ab8700 1 -- 172.21.15.7:0/4038646206 >> 172.21.15.7:0/4038646206 conn(0x7fef90098020 msgr2=0x7fef900711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:59.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.652+0000 7fef96ab8700 1 -- 172.21.15.7:0/4038646206 shutdown_connections 2024-01-27T02:36:59.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.652+0000 7fef96ab8700 1 -- 172.21.15.7:0/4038646206 wait complete. 2024-01-27T02:36:59.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.653+0000 7fef96ab8700 1 Processor -- start 2024-01-27T02:36:59.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.653+0000 7fef96ab8700 1 -- start start 2024-01-27T02:36:59.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.654+0000 7fef96ab8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef900c4cc0 0x7fef90150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:59.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.654+0000 7fef96ab8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef900cb5b0 0x7fef9016aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:59.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.654+0000 7fef96ab8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef9015a100 0x7fef9014dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:59.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.654+0000 7fef96ab8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9016af70 con 0x7fef900cb5b0 2024-01-27T02:36:59.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.654+0000 7fef96ab8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9014f9d0 con 0x7fef9015a100 2024-01-27T02:36:59.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.654+0000 7fef96ab8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9014fcb0 con 0x7fef900c4cc0 2024-01-27T02:36:59.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.654+0000 7fef8ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef900cb5b0 0x7fef9016aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:59.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.654+0000 7fef8ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef900cb5b0 0x7fef9016aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50040/0 (socket says 172.21.15.7:50040) 2024-01-27T02:36:59.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.654+0000 7fef8ffff700 1 -- 172.21.15.7:0/2968463860 learned_addr learned my addr 172.21.15.7:0/2968463860 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:36:59.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.654+0000 7fef95055700 1 --2- 172.21.15.7:0/2968463860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef9015a100 0x7fef9014dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:59.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.654+0000 7fef94854700 1 --2- 172.21.15.7:0/2968463860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef900c4cc0 0x7fef90150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:59.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.654+0000 7fef8ffff700 1 -- 172.21.15.7:0/2968463860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef900c4cc0 msgr2=0x7fef90150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:59.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.654+0000 7fef8ffff700 1 --2- 172.21.15.7:0/2968463860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef900c4cc0 0x7fef90150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:59.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.654+0000 7fef8ffff700 1 -- 172.21.15.7:0/2968463860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef9015a100 msgr2=0x7fef9014dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:59.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.654+0000 7fef8ffff700 1 --2- 172.21.15.7:0/2968463860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef9015a100 0x7fef9014dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:59.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.654+0000 7fef8ffff700 1 -- 172.21.15.7:0/2968463860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef88009ae0 con 0x7fef900cb5b0 2024-01-27T02:36:59.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.655+0000 7fef8ffff700 1 --2- 172.21.15.7:0/2968463860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef900cb5b0 0x7fef9016aa30 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0011e0 tx=0x7fef7c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:59.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.655+0000 7fef8cff9700 1 -- 172.21.15.7:0/2968463860 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c005a70 con 0x7fef900cb5b0 2024-01-27T02:36:59.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.655+0000 7fef8cff9700 1 -- 172.21.15.7:0/2968463860 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7c005bd0 con 0x7fef900cb5b0 2024-01-27T02:36:59.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.655+0000 7fef96ab8700 1 -- 172.21.15.7:0/2968463860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9014ff90 con 0x7fef900cb5b0 2024-01-27T02:36:59.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.655+0000 7fef8cff9700 1 -- 172.21.15.7:0/2968463860 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c0070d0 con 0x7fef900cb5b0 2024-01-27T02:36:59.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.655+0000 7fef96ab8700 1 -- 172.21.15.7:0/2968463860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef90150550 con 0x7fef900cb5b0 2024-01-27T02:36:59.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.656+0000 7fef96ab8700 1 -- 172.21.15.7:0/2968463860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9004c710 con 0x7fef900cb5b0 2024-01-27T02:36:59.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.656+0000 7fef8cff9700 1 -- 172.21.15.7:0/2968463860 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fef7c007480 con 0x7fef900cb5b0 2024-01-27T02:36:59.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.657+0000 7fef8cff9700 1 --2- 172.21.15.7:0/2968463860 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fef74066f30 0x7fef740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:36:59.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.657+0000 7fef8cff9700 1 -- 172.21.15.7:0/2968463860 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fef7c083690 con 0x7fef900cb5b0 2024-01-27T02:36:59.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.657+0000 7fef94854700 1 --2- 172.21.15.7:0/2968463860 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fef74066f30 0x7fef740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:36:59.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.658+0000 7fef94854700 1 --2- 172.21.15.7:0/2968463860 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fef74066f30 0x7fef740693e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fef84006fd0 tx=0x7fef84008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:36:59.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.661+0000 7fef8cff9700 1 -- 172.21.15.7:0/2968463860 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef7c050c50 con 0x7fef900cb5b0 2024-01-27T02:36:59.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.944+0000 7fef96ab8700 1 -- 172.21.15.7:0/2968463860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fef900ce270 con 0x7fef900cb5b0 2024-01-27T02:36:59.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.945+0000 7fef8cff9700 1 -- 172.21.15.7:0/2968463860 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fef7c017020 con 0x7fef900cb5b0 2024-01-27T02:36:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:36:59.946 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:36:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-01-27T02:36:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:36:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-01-27T02:36:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlHdcPKBAAfhnh1I3ZSQEXWJXlXCs98Q==", 2024-01-27T02:36:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:36:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:36:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:36:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:36:59.947 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:36:59.947 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:36:59.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:36:59.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:36:59.947 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-01-27T02:36:59.947 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rl8eVYNRAAfP8rOqO50RpoNZ1+1j2DXQ==", 2024-01-27T02:36:59.947 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:36:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:36:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:36:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:36:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:36:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:36:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:36:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:36:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-01-27T02:36:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlJxRKMBAAS0j7/zZeqGglOxCK1pg0WA==", 2024-01-27T02:36:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:36:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:36:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:36:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:36:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:36:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:36:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:36:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:36:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-01-27T02:36:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlNwfUABAALRQAoHoGhniqCvTQOqbbug==", 2024-01-27T02:36:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:36:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:36:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:36:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:36:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:36:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:36:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:36:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:36:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-01-27T02:36:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7Rlh6VSJhAAttuCte1WhNEvUkv5x/IKQQ==", 2024-01-27T02:36:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:36:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:36:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:36:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:36:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:36:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:36:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:36:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-01-27T02:36:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7Rlu/RGLhAAKzaCNhzYKhsNhxkMvY++gw==", 2024-01-27T02:36:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:36:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:36:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:36:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:36:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:36:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:36:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:36:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-01-27T02:36:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7RliRMyNhAAr0tG3evmVV7ZlGxbxfWPMw==", 2024-01-27T02:36:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:36:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:36:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:36:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:36:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:36:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:36:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:36:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-01-27T02:36:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAPa7RlRuSNAhAA3e9atlb+ffvlnLhRYbWZPw==", 2024-01-27T02:36:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:36:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:36:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:36:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:36:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:36:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:36:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:36:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-01-27T02:36:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7RlfeA6DBAAXpklgbNHRO+5G+fdIERjRw==", 2024-01-27T02:36:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:36:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:36:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:36:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:36:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:36:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:36:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:36:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-01-27T02:36:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7Rl77ItExAABB7uABm63ubE/YUs2B5pGw==", 2024-01-27T02:36:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:36:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:36:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:36:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:36:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:36:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:36:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:36:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-01-27T02:36:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7RljDuwHBAAPPAlhoqS03qPpq58jmevLA==", 2024-01-27T02:36:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:36:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:36:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:36:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:36:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:36:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:36:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:36:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-01-27T02:36:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7Rl1YElJhAAqHP/QEdl1SEalGcbQwPwAg==", 2024-01-27T02:36:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:36:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:36:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:36:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:36:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:36:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:36:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:36:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-01-27T02:36:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlaEqaBRAATFiIeW8lT9BdNuN0m3x9dQ==", 2024-01-27T02:36:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:36:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:36:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-01-27T02:36:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-27T02:36:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:36:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:36:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:36:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-01-27T02:36:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlNADmEBAAJXwdJ35ylP5coy1dtHoD3g==", 2024-01-27T02:36:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:36:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:36:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:36:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:36:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:36:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:36:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:36:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:36:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-01-27T02:36:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7Rl5fRqCRAA/5swTGknnpP64TAh+2X41A==", 2024-01-27T02:36:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:36:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:36:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:36:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:36:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:36:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:36:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:36:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:36:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-01-27T02:36:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7Rl0SdwFhAAhCmw9ViiJbHVfS5a2r/D+w==", 2024-01-27T02:36:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:36:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:36:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:36:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:36:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:36:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:36:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:36:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:36:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-01-27T02:36:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlsNgPARAA0Qm0oJgA8GO1oUT3ndT8Tg==", 2024-01-27T02:36:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:36:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:36:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:36:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:36:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:36:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:36:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:36:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:36:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-01-27T02:36:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rlfb7qIxAAlsJYiSolJ7D8EykGWP62bA==", 2024-01-27T02:36:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:36:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:36:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-27T02:36:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:36:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:36:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:36:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:36:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-01-27T02:36:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rls7TaHxAALS3ghxKtfmgQfIELiD9+Bg==", 2024-01-27T02:36:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:36:59.960 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:36:59.960 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-27T02:36:59.960 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:36:59.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:36:59.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:36:59.960 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:36:59.960 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:36:59.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.947+0000 7fef96ab8700 1 -- 172.21.15.7:0/2968463860 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fef74066f30 msgr2=0x7fef740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:59.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.947+0000 7fef96ab8700 1 --2- 172.21.15.7:0/2968463860 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fef74066f30 0x7fef740693e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fef84006fd0 tx=0x7fef84008040 comp rx=0 tx=0).stop 2024-01-27T02:36:59.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.947+0000 7fef96ab8700 1 -- 172.21.15.7:0/2968463860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef900cb5b0 msgr2=0x7fef9016aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:36:59.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.947+0000 7fef96ab8700 1 --2- 172.21.15.7:0/2968463860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef900cb5b0 0x7fef9016aa30 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0011e0 tx=0x7fef7c007e10 comp rx=0 tx=0).stop 2024-01-27T02:36:59.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.947+0000 7fef96ab8700 1 -- 172.21.15.7:0/2968463860 shutdown_connections 2024-01-27T02:36:59.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.947+0000 7fef96ab8700 1 --2- 172.21.15.7:0/2968463860 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fef74066f30 0x7fef740693e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:59.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.947+0000 7fef96ab8700 1 --2- 172.21.15.7:0/2968463860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef9015a100 0x7fef9014dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:59.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.947+0000 7fef96ab8700 1 --2- 172.21.15.7:0/2968463860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef900c4cc0 0x7fef90150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:59.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.947+0000 7fef96ab8700 1 --2- 172.21.15.7:0/2968463860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef900cb5b0 0x7fef9016aa30 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:36:59.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.947+0000 7fef96ab8700 1 -- 172.21.15.7:0/2968463860 >> 172.21.15.7:0/2968463860 conn(0x7fef90098020 msgr2=0x7fef900c9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:36:59.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.947+0000 7fef96ab8700 1 -- 172.21.15.7:0/2968463860 shutdown_connections 2024-01-27T02:36:59.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:36:59.947+0000 7fef96ab8700 1 -- 172.21.15.7:0/2968463860 wait complete. 2024-01-27T02:36:59.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:37:00.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.095+0000 7f8c70e2e700 1 Processor -- start 2024-01-27T02:37:00.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.095+0000 7f8c70e2e700 1 -- start start 2024-01-27T02:37:00.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.096+0000 7f8c70e2e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c6c1438f0 0x7f8c6c0c8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:00.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.096+0000 7f8c70e2e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c6c0cb7a0 con 0x7f8c6c063c90 2024-01-27T02:37:00.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.096+0000 7f8c70e2e700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c6c0cb9f0 con 0x7f8c6c143520 2024-01-27T02:37:00.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.096+0000 7f8c70e2e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0cbcd0 con 0x7f8c6c1438f0 2024-01-27T02:37:00.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.096+0000 7f8c69d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8c6c063c90 0x7f8c6c0c8210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60858/0 (socket says 172.21.15.7:60858) 2024-01-27T02:37:00.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.096+0000 7f8c69d9b700 1 -- 172.21.15.7:0/837357237 learned_addr learned my addr 172.21.15.7:0/837357237 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:00.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.096+0000 7f8c6ad9d700 1 --2- 172.21.15.7:0/837357237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c6c1438f0 0x7f8c6c0c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:00.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.096+0000 7f8c6959a700 1 -- 172.21.15.7:0/837357237 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2619154573 0 0) 0x7f8c6c0cb7a0 con 0x7f8c6c063c90 2024-01-27T02:37:00.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.096+0000 7f8c6959a700 1 -- 172.21.15.7:0/837357237 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c54003570 con 0x7f8c6c063c90 2024-01-27T02:37:00.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.096+0000 7f8c6959a700 1 -- 172.21.15.7:0/837357237 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 368841058 0 0) 0x7f8c6c0cb9f0 con 0x7f8c6c143520 2024-01-27T02:37:00.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.096+0000 7f8c6959a700 1 -- 172.21.15.7:0/837357237 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c6c0cb7a0 con 0x7f8c6c143520 2024-01-27T02:37:00.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.097+0000 7f8c6ad9d700 1 -- 172.21.15.7:0/837357237 >> v1:172.21.15.188:6789/0 conn(0x7f8c6c143520 legacy=0x7f8c6c063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:00.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.097+0000 7f8c6ad9d700 1 -- 172.21.15.7:0/837357237 >> v1:172.21.15.7:6789/0 conn(0x7f8c6c063c90 legacy=0x7f8c6c0c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:00.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.097+0000 7f8c6ad9d700 1 -- 172.21.15.7:0/837357237 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6c0cbfb0 con 0x7f8c6c1438f0 2024-01-27T02:37:00.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.097+0000 7f8c6ad9d700 1 --2- 172.21.15.7:0/837357237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c6c1438f0 0x7f8c6c0c8920 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8c64009e90 tx=0x7f8c64009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=610b8829f4044448 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:00.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.097+0000 7f8c6959a700 1 -- 172.21.15.7:0/837357237 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6400e070 con 0x7f8c6c1438f0 2024-01-27T02:37:00.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.097+0000 7f8c6959a700 1 -- 172.21.15.7:0/837357237 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6400bb80 con 0x7f8c6c1438f0 2024-01-27T02:37:00.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.097+0000 7f8c6959a700 1 -- 172.21.15.7:0/837357237 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64012430 con 0x7f8c6c1438f0 2024-01-27T02:37:00.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.098+0000 7f8c70e2e700 1 -- 172.21.15.7:0/837357237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c6c1438f0 msgr2=0x7f8c6c0c8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:00.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.098+0000 7f8c70e2e700 1 --2- 172.21.15.7:0/837357237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c6c1438f0 0x7f8c6c0c8920 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8c64009e90 tx=0x7f8c64009780 comp rx=0 tx=0).stop 2024-01-27T02:37:00.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.098+0000 7f8c70e2e700 1 -- 172.21.15.7:0/837357237 shutdown_connections 2024-01-27T02:37:00.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.098+0000 7f8c70e2e700 1 --2- 172.21.15.7:0/837357237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c6c1438f0 0x7f8c6c0c8920 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:00.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.098+0000 7f8c70e2e700 1 -- 172.21.15.7:0/837357237 >> 172.21.15.7:0/837357237 conn(0x7f8c6c0cdbd0 msgr2=0x7f8c6c0cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:00.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.098+0000 7f8c70e2e700 1 -- 172.21.15.7:0/837357237 shutdown_connections 2024-01-27T02:37:00.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.098+0000 7f8c70e2e700 1 -- 172.21.15.7:0/837357237 wait complete. 2024-01-27T02:37:00.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.099+0000 7f8c70e2e700 1 Processor -- start 2024-01-27T02:37:00.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.099+0000 7f8c70e2e700 1 -- start start 2024-01-27T02:37:00.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.100+0000 7f8c70e2e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c6c063c90 0x7f8c6c140840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:00.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.100+0000 7f8c70e2e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c6c143520 0x7f8c6c0afc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:00.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.100+0000 7f8c70e2e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c6c1438f0 0x7f8c6c13f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:00.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.100+0000 7f8c70e2e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0b01b0 con 0x7f8c6c063c90 2024-01-27T02:37:00.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.100+0000 7f8c70e2e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0b1d90 con 0x7f8c6c143520 2024-01-27T02:37:00.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.100+0000 7f8c70e2e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c140ed0 con 0x7f8c6c1438f0 2024-01-27T02:37:00.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.100+0000 7f8c6a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c6c063c90 0x7f8c6c140840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:00.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.100+0000 7f8c6a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c6c063c90 0x7f8c6c140840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50042/0 (socket says 172.21.15.7:50042) 2024-01-27T02:37:00.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.100+0000 7f8c6a59c700 1 -- 172.21.15.7:0/118712688 learned_addr learned my addr 172.21.15.7:0/118712688 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:00.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.100+0000 7f8c69d9b700 1 --2- 172.21.15.7:0/118712688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c6c143520 0x7f8c6c0afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:00.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.100+0000 7f8c6ad9d700 1 --2- 172.21.15.7:0/118712688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c6c1438f0 0x7f8c6c13f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:00.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.100+0000 7f8c6a59c700 1 -- 172.21.15.7:0/118712688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c6c1438f0 msgr2=0x7f8c6c13f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:00.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.100+0000 7f8c6a59c700 1 --2- 172.21.15.7:0/118712688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c6c1438f0 0x7f8c6c13f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:00.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.100+0000 7f8c6a59c700 1 -- 172.21.15.7:0/118712688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c6c143520 msgr2=0x7f8c6c0afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:00.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.100+0000 7f8c6a59c700 1 --2- 172.21.15.7:0/118712688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c6c143520 0x7f8c6c0afc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:00.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.100+0000 7f8c6a59c700 1 -- 172.21.15.7:0/118712688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c64009a20 con 0x7f8c6c063c90 2024-01-27T02:37:00.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.100+0000 7f8c69d9b700 1 --2- 172.21.15.7:0/118712688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c6c143520 0x7f8c6c0afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:00.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.100+0000 7f8c6ad9d700 1 --2- 172.21.15.7:0/118712688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c6c1438f0 0x7f8c6c13f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:00.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.100+0000 7f8c6a59c700 1 --2- 172.21.15.7:0/118712688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c6c063c90 0x7f8c6c140840 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00c7c0 tx=0x7f8c5c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:00.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.101+0000 7f8c5a7fc700 1 -- 172.21.15.7:0/118712688 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c011680 con 0x7f8c6c063c90 2024-01-27T02:37:00.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.101+0000 7f8c5a7fc700 1 -- 172.21.15.7:0/118712688 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5c0117e0 con 0x7f8c6c063c90 2024-01-27T02:37:00.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.101+0000 7f8c70e2e700 1 -- 172.21.15.7:0/118712688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6c0b1460 con 0x7f8c6c063c90 2024-01-27T02:37:00.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.101+0000 7f8c5a7fc700 1 -- 172.21.15.7:0/118712688 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c011ba0 con 0x7f8c6c063c90 2024-01-27T02:37:00.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.101+0000 7f8c70e2e700 1 -- 172.21.15.7:0/118712688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c6c0b1a20 con 0x7f8c6c063c90 2024-01-27T02:37:00.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.102+0000 7f8c70e2e700 1 -- 172.21.15.7:0/118712688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c6c04c710 con 0x7f8c6c063c90 2024-01-27T02:37:00.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.102+0000 7f8c5a7fc700 1 -- 172.21.15.7:0/118712688 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8c5c012440 con 0x7f8c6c063c90 2024-01-27T02:37:00.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.103+0000 7f8c5a7fc700 1 --2- 172.21.15.7:0/118712688 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8c4c066f30 0x7f8c4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:00.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.103+0000 7f8c5a7fc700 1 -- 172.21.15.7:0/118712688 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8c5c0840d0 con 0x7f8c6c063c90 2024-01-27T02:37:00.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.103+0000 7f8c69d9b700 1 --2- 172.21.15.7:0/118712688 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8c4c066f30 0x7f8c4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:00.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.104+0000 7f8c69d9b700 1 --2- 172.21.15.7:0/118712688 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8c4c066f30 0x7f8c4c0693e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c16e120 tx=0x7f8c60009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:00.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.107+0000 7f8c5a7fc700 1 -- 172.21.15.7:0/118712688 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c5c051690 con 0x7f8c6c063c90 2024-01-27T02:37:00.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.313+0000 7f8c70e2e700 1 -- 172.21.15.7:0/118712688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c6c0cc290 con 0x7f8c6c063c90 2024-01-27T02:37:00.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.313+0000 7f8c5a7fc700 1 -- 172.21.15.7:0/118712688 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+12087 (secure 0 0 0) 0x7f8c5c054cb0 con 0x7f8c6c063c90 2024-01-27T02:37:00.314 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:00.314 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":94,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:36:59.491643+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:37:00.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.316+0000 7f8c70e2e700 1 -- 172.21.15.7:0/118712688 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8c4c066f30 msgr2=0x7f8c4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:00.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.316+0000 7f8c70e2e700 1 --2- 172.21.15.7:0/118712688 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8c4c066f30 0x7f8c4c0693e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c16e120 tx=0x7f8c60009040 comp rx=0 tx=0).stop 2024-01-27T02:37:00.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.316+0000 7f8c70e2e700 1 -- 172.21.15.7:0/118712688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c6c063c90 msgr2=0x7f8c6c140840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:00.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.316+0000 7f8c70e2e700 1 --2- 172.21.15.7:0/118712688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c6c063c90 0x7f8c6c140840 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00c7c0 tx=0x7f8c5c007e00 comp rx=0 tx=0).stop 2024-01-27T02:37:00.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.316+0000 7f8c70e2e700 1 -- 172.21.15.7:0/118712688 shutdown_connections 2024-01-27T02:37:00.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.316+0000 7f8c70e2e700 1 --2- 172.21.15.7:0/118712688 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8c4c066f30 0x7f8c4c0693e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:00.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.316+0000 7f8c70e2e700 1 --2- 172.21.15.7:0/118712688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c6c1438f0 0x7f8c6c13f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:00.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.316+0000 7f8c70e2e700 1 --2- 172.21.15.7:0/118712688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c6c063c90 0x7f8c6c140840 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:00.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.316+0000 7f8c70e2e700 1 --2- 172.21.15.7:0/118712688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c6c143520 0x7f8c6c0afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:00.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.316+0000 7f8c70e2e700 1 -- 172.21.15.7:0/118712688 >> 172.21.15.7:0/118712688 conn(0x7f8c6c0cdbd0 msgr2=0x7f8c6c06e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:00.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.316+0000 7f8c70e2e700 1 -- 172.21.15.7:0/118712688 shutdown_connections 2024-01-27T02:37:00.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.316+0000 7f8c70e2e700 1 -- 172.21.15.7:0/118712688 wait complete. 2024-01-27T02:37:00.330 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-27T02:37:00.330 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:37:00.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.464+0000 7f9738c09700 1 Processor -- start 2024-01-27T02:37:00.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.464+0000 7f9738c09700 1 -- start start 2024-01-27T02:37:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.465+0000 7f9738c09700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97340abb20 0x7f9734161e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.465+0000 7f9738c09700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9734070d00 con 0x7f9734157250 2024-01-27T02:37:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.465+0000 7f9738c09700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97340c29c0 con 0x7f9734156e80 2024-01-27T02:37:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.465+0000 7f9738c09700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97340cc0a0 con 0x7f97340abb20 2024-01-27T02:37:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.466+0000 7f9731d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9734157250 0x7f97340cb990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60860/0 (socket says 172.21.15.7:60860) 2024-01-27T02:37:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.466+0000 7f9731d9b700 1 -- 172.21.15.7:0/3260930215 learned_addr learned my addr 172.21.15.7:0/3260930215 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.466+0000 7f9732d9d700 1 --2- 172.21.15.7:0/3260930215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97340abb20 0x7f9734161e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.466+0000 7f973159a700 1 -- 172.21.15.7:0/3260930215 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718692499 0 0) 0x7f9734070d00 con 0x7f9734157250 2024-01-27T02:37:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.466+0000 7f973159a700 1 -- 172.21.15.7:0/3260930215 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9720003570 con 0x7f9734157250 2024-01-27T02:37:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.466+0000 7f973159a700 1 -- 172.21.15.7:0/3260930215 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2606766247 0 0) 0x7f97340c29c0 con 0x7f9734156e80 2024-01-27T02:37:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.466+0000 7f973159a700 1 -- 172.21.15.7:0/3260930215 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9734070d00 con 0x7f9734156e80 2024-01-27T02:37:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.466+0000 7f9732d9d700 1 -- 172.21.15.7:0/3260930215 >> v1:172.21.15.188:6789/0 conn(0x7f9734156e80 legacy=0x7f97340cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.466+0000 7f9732d9d700 1 -- 172.21.15.7:0/3260930215 >> v1:172.21.15.7:6789/0 conn(0x7f9734157250 legacy=0x7f97340cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.466+0000 7f973159a700 1 -- 172.21.15.7:0/3260930215 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 144946550 0 0) 0x7f9720003570 con 0x7f9734157250 2024-01-27T02:37:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.466+0000 7f9732d9d700 1 -- 172.21.15.7:0/3260930215 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97340aa050 con 0x7f97340abb20 2024-01-27T02:37:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.466+0000 7f9732d9d700 1 --2- 172.21.15.7:0/3260930215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97340abb20 0x7f9734161e10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f97340aa240 tx=0x7f972c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad7388ebc278d5ae server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.467+0000 7f973159a700 1 -- 172.21.15.7:0/3260930215 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c00e070 con 0x7f97340abb20 2024-01-27T02:37:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.467+0000 7f973159a700 1 -- 172.21.15.7:0/3260930215 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972c00b7a0 con 0x7f97340abb20 2024-01-27T02:37:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.467+0000 7f973159a700 1 -- 172.21.15.7:0/3260930215 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c001470 con 0x7f97340abb20 2024-01-27T02:37:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.467+0000 7f9738c09700 1 -- 172.21.15.7:0/3260930215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97340abb20 msgr2=0x7f9734161e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.467+0000 7f9738c09700 1 --2- 172.21.15.7:0/3260930215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97340abb20 0x7f9734161e10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f97340aa240 tx=0x7f972c009780 comp rx=0 tx=0).stop 2024-01-27T02:37:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.467+0000 7f9738c09700 1 -- 172.21.15.7:0/3260930215 shutdown_connections 2024-01-27T02:37:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.467+0000 7f9738c09700 1 --2- 172.21.15.7:0/3260930215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97340abb20 0x7f9734161e10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:00.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.467+0000 7f9738c09700 1 -- 172.21.15.7:0/3260930215 >> 172.21.15.7:0/3260930215 conn(0x7f97340679b0 msgr2=0x7f9734067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:00.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.467+0000 7f9738c09700 1 -- 172.21.15.7:0/3260930215 shutdown_connections 2024-01-27T02:37:00.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.467+0000 7f9738c09700 1 -- 172.21.15.7:0/3260930215 wait complete. 2024-01-27T02:37:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.469+0000 7f9738c09700 1 Processor -- start 2024-01-27T02:37:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.469+0000 7f9738c09700 1 -- start start 2024-01-27T02:37:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.470+0000 7f9738c09700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97340abb20 0x7f9734169350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.470+0000 7f9738c09700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9734156e80 0x7f9734168b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.470+0000 7f9738c09700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9734157250 0x7f9734099de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.470+0000 7f9738c09700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97340c5400 con 0x7f9734157250 2024-01-27T02:37:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.470+0000 7f9738c09700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97340c6260 con 0x7f97340abb20 2024-01-27T02:37:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.470+0000 7f9738c09700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9734169890 con 0x7f9734156e80 2024-01-27T02:37:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.470+0000 7f9732d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9734157250 0x7f9734099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:00.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.470+0000 7f9732d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9734157250 0x7f9734099de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50046/0 (socket says 172.21.15.7:50046) 2024-01-27T02:37:00.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.470+0000 7f9732d9d700 1 -- 172.21.15.7:0/2683030282 learned_addr learned my addr 172.21.15.7:0/2683030282 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:00.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.470+0000 7f973259c700 1 --2- 172.21.15.7:0/2683030282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97340abb20 0x7f9734169350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:00.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.470+0000 7f9731d9b700 1 --2- 172.21.15.7:0/2683030282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9734156e80 0x7f9734168b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:00.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.471+0000 7f9732d9d700 1 -- 172.21.15.7:0/2683030282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9734156e80 msgr2=0x7f9734168b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:00.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.471+0000 7f9732d9d700 1 --2- 172.21.15.7:0/2683030282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9734156e80 0x7f9734168b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:00.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.471+0000 7f9732d9d700 1 -- 172.21.15.7:0/2683030282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97340abb20 msgr2=0x7f9734169350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:00.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.471+0000 7f9732d9d700 1 --2- 172.21.15.7:0/2683030282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97340abb20 0x7f9734169350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:00.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.471+0000 7f9732d9d700 1 -- 172.21.15.7:0/2683030282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972c009a20 con 0x7f9734157250 2024-01-27T02:37:00.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.471+0000 7f9732d9d700 1 --2- 172.21.15.7:0/2683030282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9734157250 0x7f9734099de0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f97340c2b50 tx=0x7f972c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:00.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.471+0000 7f972a7fc700 1 -- 172.21.15.7:0/2683030282 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c012b30 con 0x7f9734157250 2024-01-27T02:37:00.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.471+0000 7f972a7fc700 1 -- 172.21.15.7:0/2683030282 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972c012c90 con 0x7f9734157250 2024-01-27T02:37:00.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.471+0000 7f9738c09700 1 -- 172.21.15.7:0/2683030282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97340c5650 con 0x7f9734157250 2024-01-27T02:37:00.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.471+0000 7f972a7fc700 1 -- 172.21.15.7:0/2683030282 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c01e680 con 0x7f9734157250 2024-01-27T02:37:00.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.471+0000 7f9738c09700 1 -- 172.21.15.7:0/2683030282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97340c5c10 con 0x7f9734157250 2024-01-27T02:37:00.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.472+0000 7f9738c09700 1 -- 172.21.15.7:0/2683030282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f973406fdf0 con 0x7f9734157250 2024-01-27T02:37:00.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.473+0000 7f972a7fc700 1 -- 172.21.15.7:0/2683030282 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f972c01ea70 con 0x7f9734157250 2024-01-27T02:37:00.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.473+0000 7f972a7fc700 1 --2- 172.21.15.7:0/2683030282 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9714066f30 0x7f97140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:00.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.473+0000 7f972a7fc700 1 -- 172.21.15.7:0/2683030282 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f972c0863f0 con 0x7f9734157250 2024-01-27T02:37:00.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.473+0000 7f973259c700 1 --2- 172.21.15.7:0/2683030282 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9714066f30 0x7f97140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:00.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.474+0000 7f973259c700 1 --2- 172.21.15.7:0/2683030282 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9714066f30 0x7f97140693e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f97340ad100 tx=0x7f9724007d40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:00.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.477+0000 7f972a7fc700 1 -- 172.21.15.7:0/2683030282 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f972c053910 con 0x7f9734157250 2024-01-27T02:37:00.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.679+0000 7f9738c09700 1 -- 172.21.15.7:0/2683030282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9734072060 con 0x7f9734157250 2024-01-27T02:37:00.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.679+0000 7f972a7fc700 1 -- 172.21.15.7:0/2683030282 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f972c056f30 con 0x7f9734157250 2024-01-27T02:37:00.680 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:37:00.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.681+0000 7f9738c09700 1 -- 172.21.15.7:0/2683030282 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9714066f30 msgr2=0x7f97140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:00.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.681+0000 7f9738c09700 1 --2- 172.21.15.7:0/2683030282 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9714066f30 0x7f97140693e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f97340ad100 tx=0x7f9724007d40 comp rx=0 tx=0).stop 2024-01-27T02:37:00.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.681+0000 7f9738c09700 1 -- 172.21.15.7:0/2683030282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9734157250 msgr2=0x7f9734099de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:00.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.681+0000 7f9738c09700 1 --2- 172.21.15.7:0/2683030282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9734157250 0x7f9734099de0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f97340c2b50 tx=0x7f972c017000 comp rx=0 tx=0).stop 2024-01-27T02:37:00.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.681+0000 7f9738c09700 1 -- 172.21.15.7:0/2683030282 shutdown_connections 2024-01-27T02:37:00.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.681+0000 7f9738c09700 1 --2- 172.21.15.7:0/2683030282 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9714066f30 0x7f97140693e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:00.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.681+0000 7f9738c09700 1 --2- 172.21.15.7:0/2683030282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9734157250 0x7f9734099de0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:00.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.681+0000 7f9738c09700 1 --2- 172.21.15.7:0/2683030282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97340abb20 0x7f9734169350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:00.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.681+0000 7f9738c09700 1 --2- 172.21.15.7:0/2683030282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9734156e80 0x7f9734168b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:00.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.681+0000 7f9738c09700 1 -- 172.21.15.7:0/2683030282 >> 172.21.15.7:0/2683030282 conn(0x7f97340679b0 msgr2=0x7f9734155f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:00.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.681+0000 7f9738c09700 1 -- 172.21.15.7:0/2683030282 shutdown_connections 2024-01-27T02:37:00.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.681+0000 7f9738c09700 1 -- 172.21.15.7:0/2683030282 wait complete. 2024-01-27T02:37:00.695 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-27T02:37:00.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.826+0000 7f43fbad1700 1 Processor -- start 2024-01-27T02:37:00.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.827+0000 7f43fbad1700 1 -- start start 2024-01-27T02:37:00.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.828+0000 7f43fbad1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43f40c9bb0 0x7f43f4157f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:00.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.828+0000 7f43fbad1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f43f40c9f80 0x7f43f406b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:00.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.828+0000 7f43fbad1700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f406c880 con 0x7f43f406e330 2024-01-27T02:37:00.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.828+0000 7f43fbad1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f406cd10 con 0x7f43f40c9f80 2024-01-27T02:37:00.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.828+0000 7f43fbad1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f406bc70 con 0x7f43f40c9bb0 2024-01-27T02:37:00.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.828+0000 7f43f906c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f43f40c9f80 0x7f43f406b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:00.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.828+0000 7f43f906c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f43f40c9f80 0x7f43f406b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50058/0 (socket says 172.21.15.7:50058) 2024-01-27T02:37:00.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.828+0000 7f43f906c700 1 -- 172.21.15.7:0/3758197703 learned_addr learned my addr 172.21.15.7:0/3758197703 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:00.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.828+0000 7f43f986d700 1 --2- 172.21.15.7:0/3758197703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43f40c9bb0 0x7f43f4157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:00.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.828+0000 7f43f906c700 1 -- 172.21.15.7:0/3758197703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43f40c9bb0 msgr2=0x7f43f4157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:00.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.828+0000 7f43f906c700 1 --2- 172.21.15.7:0/3758197703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43f40c9bb0 0x7f43f4157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:00.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.828+0000 7f43f906c700 1 -- 172.21.15.7:0/3758197703 >> v1:172.21.15.188:6789/0 conn(0x7f43f406e330 legacy=0x7f43f40c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:00.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.828+0000 7f43f906c700 1 -- 172.21.15.7:0/3758197703 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f406bf50 con 0x7f43f40c9f80 2024-01-27T02:37:00.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.828+0000 7f43f906c700 1 --2- 172.21.15.7:0/3758197703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f40c9f80 0x7f43f406b400 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f43e400bc50 tx=0x7f43e400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fd00c5f288fa8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:00.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.828+0000 7f43f886b700 1 -- 172.21.15.7:0/3758197703 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e400be60 con 0x7f43f40c9f80 2024-01-27T02:37:00.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.829+0000 7f43f886b700 1 -- 172.21.15.7:0/3758197703 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e400ddc0 con 0x7f43f40c9f80 2024-01-27T02:37:00.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.829+0000 7f43f886b700 1 -- 172.21.15.7:0/3758197703 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e40106a0 con 0x7f43f40c9f80 2024-01-27T02:37:00.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.829+0000 7f43fbad1700 1 -- 172.21.15.7:0/3758197703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f40c9f80 msgr2=0x7f43f406b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:00.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.829+0000 7f43fbad1700 1 --2- 172.21.15.7:0/3758197703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f40c9f80 0x7f43f406b400 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f43e400bc50 tx=0x7f43e400b4f0 comp rx=0 tx=0).stop 2024-01-27T02:37:00.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.829+0000 7f43fbad1700 1 -- 172.21.15.7:0/3758197703 shutdown_connections 2024-01-27T02:37:00.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.829+0000 7f43fbad1700 1 --2- 172.21.15.7:0/3758197703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43f40c9bb0 0x7f43f4157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:00.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.829+0000 7f43fbad1700 1 --2- 172.21.15.7:0/3758197703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f40c9f80 0x7f43f406b400 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:00.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.829+0000 7f43fbad1700 1 -- 172.21.15.7:0/3758197703 >> 172.21.15.7:0/3758197703 conn(0x7f43f40cdc30 msgr2=0x7f43f40ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:00.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.829+0000 7f43fbad1700 1 -- 172.21.15.7:0/3758197703 shutdown_connections 2024-01-27T02:37:00.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.829+0000 7f43fbad1700 1 -- 172.21.15.7:0/3758197703 wait complete. 2024-01-27T02:37:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.830+0000 7f43fbad1700 1 Processor -- start 2024-01-27T02:37:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.830+0000 7f43fbad1700 1 -- start start 2024-01-27T02:37:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.831+0000 7f43fbad1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43f406e330 0x7f43f4156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.831+0000 7f43fbad1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f40c9bb0 0x7f43f414fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.831+0000 7f43fbad1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43f40c9f80 0x7f43f41501d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.831+0000 7f43fbad1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f40b15c0 con 0x7f43f40c9bb0 2024-01-27T02:37:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.831+0000 7f43fbad1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f4150800 con 0x7f43f406e330 2024-01-27T02:37:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.831+0000 7f43fbad1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f4150ae0 con 0x7f43f40c9f80 2024-01-27T02:37:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.831+0000 7f43f906c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f40c9bb0 0x7f43f414fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.831+0000 7f43f906c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f40c9bb0 0x7f43f414fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50074/0 (socket says 172.21.15.7:50074) 2024-01-27T02:37:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.831+0000 7f43f906c700 1 -- 172.21.15.7:0/755472371 learned_addr learned my addr 172.21.15.7:0/755472371 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.832+0000 7f43f986d700 1 --2- 172.21.15.7:0/755472371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43f406e330 0x7f43f4156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.832+0000 7f43fa06e700 1 --2- 172.21.15.7:0/755472371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43f40c9f80 0x7f43f41501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.832+0000 7f43f906c700 1 -- 172.21.15.7:0/755472371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43f40c9f80 msgr2=0x7f43f41501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.832+0000 7f43f906c700 1 --2- 172.21.15.7:0/755472371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43f40c9f80 0x7f43f41501d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.832+0000 7f43f906c700 1 -- 172.21.15.7:0/755472371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43f406e330 msgr2=0x7f43f4156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.832+0000 7f43f906c700 1 --2- 172.21.15.7:0/755472371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43f406e330 0x7f43f4156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.832+0000 7f43f906c700 1 -- 172.21.15.7:0/755472371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e400b790 con 0x7f43f40c9bb0 2024-01-27T02:37:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.832+0000 7f43f906c700 1 --2- 172.21.15.7:0/755472371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f40c9bb0 0x7f43f414fc90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f43e400b760 tx=0x7f43e401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.832+0000 7f43e9ffb700 1 -- 172.21.15.7:0/755472371 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e4018700 con 0x7f43f40c9bb0 2024-01-27T02:37:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.832+0000 7f43e9ffb700 1 -- 172.21.15.7:0/755472371 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e4018860 con 0x7f43f40c9bb0 2024-01-27T02:37:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.832+0000 7f43fbad1700 1 -- 172.21.15.7:0/755472371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f4150dc0 con 0x7f43f40c9bb0 2024-01-27T02:37:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.832+0000 7f43e9ffb700 1 -- 172.21.15.7:0/755472371 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e4018c20 con 0x7f43f40c9bb0 2024-01-27T02:37:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.832+0000 7f43fbad1700 1 -- 172.21.15.7:0/755472371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f4151380 con 0x7f43f40c9bb0 2024-01-27T02:37:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.833+0000 7f43fbad1700 1 -- 172.21.15.7:0/755472371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f40ce000 con 0x7f43f40c9bb0 2024-01-27T02:37:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.834+0000 7f43e9ffb700 1 -- 172.21.15.7:0/755472371 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f43e400f3d0 con 0x7f43f40c9bb0 2024-01-27T02:37:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.834+0000 7f43e9ffb700 1 --2- 172.21.15.7:0/755472371 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f43d8066f30 0x7f43d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.834+0000 7f43e9ffb700 1 -- 172.21.15.7:0/755472371 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f43e4014070 con 0x7f43f40c9bb0 2024-01-27T02:37:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.835+0000 7f43f986d700 1 --2- 172.21.15.7:0/755472371 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f43d8066f30 0x7f43d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.835+0000 7f43f986d700 1 --2- 172.21.15.7:0/755472371 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f43d8066f30 0x7f43d80693e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f43f0006fd0 tx=0x7f43f0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:00.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:00.839+0000 7f43e9ffb700 1 -- 172.21.15.7:0/755472371 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43e4053310 con 0x7f43f40c9bb0 2024-01-27T02:37:01.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.043+0000 7f43fbad1700 1 -- 172.21.15.7:0/755472371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f43f404c710 con 0x7f43f40c9bb0 2024-01-27T02:37:01.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.044+0000 7f43e9ffb700 1 -- 172.21.15.7:0/755472371 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v94) v1 ==== 84+0+65 (secure 0 0 0) 0x7f43e4027090 con 0x7f43f40c9bb0 2024-01-27T02:37:01.044 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:01.044 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-27T02:37:01.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:01.045 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-27T02:37:01.045 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-27T02:37:01.045 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:01.045 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-27T02:37:01.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.046+0000 7f43fbad1700 1 -- 172.21.15.7:0/755472371 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f43d8066f30 msgr2=0x7f43d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:01.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.046+0000 7f43fbad1700 1 --2- 172.21.15.7:0/755472371 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f43d8066f30 0x7f43d80693e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f43f0006fd0 tx=0x7f43f0008040 comp rx=0 tx=0).stop 2024-01-27T02:37:01.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.046+0000 7f43fbad1700 1 -- 172.21.15.7:0/755472371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f40c9bb0 msgr2=0x7f43f414fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:01.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.046+0000 7f43fbad1700 1 --2- 172.21.15.7:0/755472371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f40c9bb0 0x7f43f414fc90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f43e400b760 tx=0x7f43e401a000 comp rx=0 tx=0).stop 2024-01-27T02:37:01.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.046+0000 7f43fbad1700 1 -- 172.21.15.7:0/755472371 shutdown_connections 2024-01-27T02:37:01.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.046+0000 7f43fbad1700 1 --2- 172.21.15.7:0/755472371 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f43d8066f30 0x7f43d80693e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:01.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.046+0000 7f43fbad1700 1 --2- 172.21.15.7:0/755472371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43f40c9f80 0x7f43f41501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:01.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.046+0000 7f43fbad1700 1 --2- 172.21.15.7:0/755472371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43f406e330 0x7f43f4156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:01.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.046+0000 7f43fbad1700 1 --2- 172.21.15.7:0/755472371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f40c9bb0 0x7f43f414fc90 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:01.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.046+0000 7f43fbad1700 1 -- 172.21.15.7:0/755472371 >> 172.21.15.7:0/755472371 conn(0x7f43f40cdc30 msgr2=0x7f43f40ca5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:01.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.046+0000 7f43fbad1700 1 -- 172.21.15.7:0/755472371 shutdown_connections 2024-01-27T02:37:01.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.046+0000 7f43fbad1700 1 -- 172.21.15.7:0/755472371 wait complete. 2024-01-27T02:37:01.060 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-27T02:37:01.060 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-27T02:37:01.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.195+0000 7fe91a04b700 1 Processor -- start 2024-01-27T02:37:01.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.195+0000 7fe91a04b700 1 -- start start 2024-01-27T02:37:01.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.196+0000 7fe91a04b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe91415d7c0 0x7fe914063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:01.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.196+0000 7fe91a04b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe914063ac0 0x7fe9140aacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:01.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.196+0000 7fe91a04b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9140cbf10 con 0x7fe91415d3f0 2024-01-27T02:37:01.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.196+0000 7fe9137fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe91415d3f0 0x7fe9140cb800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60872/0 (socket says 172.21.15.7:60872) 2024-01-27T02:37:01.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.196+0000 7fe9137fe700 1 -- 172.21.15.7:0/1351656273 learned_addr learned my addr 172.21.15.7:0/1351656273 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:01.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.196+0000 7fe91a04b700 1 -- 172.21.15.7:0/1351656273 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe914063f60 con 0x7fe91415d7c0 2024-01-27T02:37:01.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.196+0000 7fe91a04b700 1 -- 172.21.15.7:0/1351656273 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9140cc1e0 con 0x7fe914063ac0 2024-01-27T02:37:01.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.196+0000 7fe912ffd700 1 --2- 172.21.15.7:0/1351656273 >> v2:172.21.15.188:3300/0 conn(0x7fe91415d7c0 0x7fe914063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:01.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.197+0000 7fe913fff700 1 --2- 172.21.15.7:0/1351656273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe914063ac0 0x7fe9140aacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:01.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.197+0000 7fe9127fc700 1 -- 172.21.15.7:0/1351656273 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81030113 0 0) 0x7fe9140cbf10 con 0x7fe91415d3f0 2024-01-27T02:37:01.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.197+0000 7fe9127fc700 1 -- 172.21.15.7:0/1351656273 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe900003570 con 0x7fe91415d3f0 2024-01-27T02:37:01.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.197+0000 7fe912ffd700 1 -- 172.21.15.7:0/1351656273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe914063ac0 msgr2=0x7fe9140aacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:01.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.197+0000 7fe912ffd700 1 --2- 172.21.15.7:0/1351656273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe914063ac0 0x7fe9140aacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:01.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.197+0000 7fe912ffd700 1 -- 172.21.15.7:0/1351656273 >> v1:172.21.15.7:6789/0 conn(0x7fe91415d3f0 legacy=0x7fe9140cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.197+0000 7fe912ffd700 1 -- 172.21.15.7:0/1351656273 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9140cc4c0 con 0x7fe91415d7c0 2024-01-27T02:37:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.197+0000 7fe912ffd700 1 --2- 172.21.15.7:0/1351656273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe91415d7c0 0x7fe914063580 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc009910 tx=0x7fe8fc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16c3e42a52620cca server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.197+0000 7fe9127fc700 1 -- 172.21.15.7:0/1351656273 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc011630 con 0x7fe91415d7c0 2024-01-27T02:37:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.197+0000 7fe9127fc700 1 -- 172.21.15.7:0/1351656273 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc011790 con 0x7fe91415d7c0 2024-01-27T02:37:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.197+0000 7fe9127fc700 1 -- 172.21.15.7:0/1351656273 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc011b30 con 0x7fe91415d7c0 2024-01-27T02:37:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.198+0000 7fe91a04b700 1 -- 172.21.15.7:0/1351656273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe91415d7c0 msgr2=0x7fe914063580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.198+0000 7fe91a04b700 1 --2- 172.21.15.7:0/1351656273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe91415d7c0 0x7fe914063580 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc009910 tx=0x7fe8fc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.198+0000 7fe91a04b700 1 -- 172.21.15.7:0/1351656273 shutdown_connections 2024-01-27T02:37:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.198+0000 7fe91a04b700 1 --2- 172.21.15.7:0/1351656273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe914063ac0 0x7fe9140aacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.198+0000 7fe91a04b700 1 --2- 172.21.15.7:0/1351656273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe91415d7c0 0x7fe914063580 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.198+0000 7fe91a04b700 1 -- 172.21.15.7:0/1351656273 >> 172.21.15.7:0/1351656273 conn(0x7fe9140cdc30 msgr2=0x7fe9140ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.198+0000 7fe91a04b700 1 -- 172.21.15.7:0/1351656273 shutdown_connections 2024-01-27T02:37:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.198+0000 7fe91a04b700 1 -- 172.21.15.7:0/1351656273 wait complete. 2024-01-27T02:37:01.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.199+0000 7fe91a04b700 1 Processor -- start 2024-01-27T02:37:01.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.200+0000 7fe91a04b700 1 -- start start 2024-01-27T02:37:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.200+0000 7fe91a04b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe914063ac0 0x7fe91406c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.200+0000 7fe91a04b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe91415d3f0 0x7fe9140c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.200+0000 7fe91a04b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe91415d7c0 0x7fe9140c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.200+0000 7fe91a04b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9140aea60 con 0x7fe91415d7c0 2024-01-27T02:37:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.200+0000 7fe91a04b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91406bcc0 con 0x7fe914063ac0 2024-01-27T02:37:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.200+0000 7fe91a04b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91406b1a0 con 0x7fe91415d3f0 2024-01-27T02:37:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.200+0000 7fe913fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe91415d7c0 0x7fe9140c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.201+0000 7fe913fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe91415d7c0 0x7fe9140c7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50086/0 (socket says 172.21.15.7:50086) 2024-01-27T02:37:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.201+0000 7fe913fff700 1 -- 172.21.15.7:0/1367086390 learned_addr learned my addr 172.21.15.7:0/1367086390 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.201+0000 7fe912ffd700 1 --2- 172.21.15.7:0/1367086390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe91415d3f0 0x7fe9140c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.201+0000 7fe9137fe700 1 --2- 172.21.15.7:0/1367086390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe914063ac0 0x7fe91406c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.201+0000 7fe913fff700 1 -- 172.21.15.7:0/1367086390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe91415d3f0 msgr2=0x7fe9140c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.201+0000 7fe913fff700 1 --2- 172.21.15.7:0/1367086390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe91415d3f0 0x7fe9140c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.201+0000 7fe913fff700 1 -- 172.21.15.7:0/1367086390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe914063ac0 msgr2=0x7fe91406c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.201+0000 7fe913fff700 1 --2- 172.21.15.7:0/1367086390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe914063ac0 0x7fe91406c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.201+0000 7fe913fff700 1 -- 172.21.15.7:0/1367086390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8fc009420 con 0x7fe91415d7c0 2024-01-27T02:37:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.201+0000 7fe913fff700 1 --2- 172.21.15.7:0/1367086390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe91415d7c0 0x7fe9140c7f20 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009df0 tx=0x7fe90c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.201+0000 7fe90b7fe700 1 -- 172.21.15.7:0/1367086390 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c00caa0 con 0x7fe91415d7c0 2024-01-27T02:37:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.201+0000 7fe90b7fe700 1 -- 172.21.15.7:0/1367086390 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90c00f040 con 0x7fe91415d7c0 2024-01-27T02:37:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.201+0000 7fe91a04b700 1 -- 172.21.15.7:0/1367086390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe91406c880 con 0x7fe91415d7c0 2024-01-27T02:37:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.201+0000 7fe90b7fe700 1 -- 172.21.15.7:0/1367086390 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c00ce50 con 0x7fe91415d7c0 2024-01-27T02:37:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.201+0000 7fe91a04b700 1 -- 172.21.15.7:0/1367086390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe91406b700 con 0x7fe91415d7c0 2024-01-27T02:37:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.202+0000 7fe91a04b700 1 -- 172.21.15.7:0/1367086390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe91404c710 con 0x7fe91415d7c0 2024-01-27T02:37:01.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.203+0000 7fe90b7fe700 1 -- 172.21.15.7:0/1367086390 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe90c014690 con 0x7fe91415d7c0 2024-01-27T02:37:01.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.203+0000 7fe90b7fe700 1 --2- 172.21.15.7:0/1367086390 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe8f4066f30 0x7fe8f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:01.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.203+0000 7fe90b7fe700 1 -- 172.21.15.7:0/1367086390 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe90c084a90 con 0x7fe91415d7c0 2024-01-27T02:37:01.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.203+0000 7fe9137fe700 1 --2- 172.21.15.7:0/1367086390 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe8f4066f30 0x7fe8f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:01.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.204+0000 7fe9137fe700 1 --2- 172.21.15.7:0/1367086390 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe8f4066f30 0x7fe8f40693e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe904007fd0 tx=0x7fe904009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:01.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.207+0000 7fe90b7fe700 1 -- 172.21.15.7:0/1367086390 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe90c055ae0 con 0x7fe91415d7c0 2024-01-27T02:37:01.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.413+0000 7fe91a04b700 1 -- 172.21.15.7:0/1367086390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe9140aeba0 con 0x7fe91415d7c0 2024-01-27T02:37:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.962+0000 7fe90b7fe700 1 -- 172.21.15.7:0/1367086390 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v95) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe90c055670 con 0x7fe91415d7c0 2024-01-27T02:37:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.967+0000 7fe91a04b700 1 -- 172.21.15.7:0/1367086390 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe8f4066f30 msgr2=0x7fe8f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.967+0000 7fe91a04b700 1 --2- 172.21.15.7:0/1367086390 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe8f4066f30 0x7fe8f40693e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe904007fd0 tx=0x7fe904009040 comp rx=0 tx=0).stop 2024-01-27T02:37:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.967+0000 7fe91a04b700 1 -- 172.21.15.7:0/1367086390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe91415d7c0 msgr2=0x7fe9140c7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.967+0000 7fe91a04b700 1 --2- 172.21.15.7:0/1367086390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe91415d7c0 0x7fe9140c7f20 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009df0 tx=0x7fe90c00b040 comp rx=0 tx=0).stop 2024-01-27T02:37:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.968+0000 7fe91a04b700 1 -- 172.21.15.7:0/1367086390 shutdown_connections 2024-01-27T02:37:01.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.968+0000 7fe91a04b700 1 --2- 172.21.15.7:0/1367086390 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe8f4066f30 0x7fe8f40693e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:01.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.968+0000 7fe91a04b700 1 --2- 172.21.15.7:0/1367086390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe91415d7c0 0x7fe9140c7f20 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:01.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.968+0000 7fe91a04b700 1 --2- 172.21.15.7:0/1367086390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe914063ac0 0x7fe91406c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:01.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.968+0000 7fe91a04b700 1 --2- 172.21.15.7:0/1367086390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe91415d3f0 0x7fe9140c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:01.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.968+0000 7fe91a04b700 1 -- 172.21.15.7:0/1367086390 >> 172.21.15.7:0/1367086390 conn(0x7fe9140cdc30 msgr2=0x7fe9140c2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:01.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.968+0000 7fe91a04b700 1 -- 172.21.15.7:0/1367086390 shutdown_connections 2024-01-27T02:37:01.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:01.968+0000 7fe91a04b700 1 -- 172.21.15.7:0/1367086390 wait complete. 2024-01-27T02:37:01.969 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-27T02:37:01.984 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-27T02:37:02.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.128+0000 7f1a32497700 1 Processor -- start 2024-01-27T02:37:02.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.128+0000 7f1a32497700 1 -- start start 2024-01-27T02:37:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.128+0000 7f1a32497700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1a2c156e80 0x7f1a2c0cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.128+0000 7f1a32497700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2c0c2120 0x7f1a2c0aab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.129+0000 7f1a32497700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a2c070d00 con 0x7f1a2c157250 2024-01-27T02:37:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.129+0000 7f1a32497700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a2c0c29c0 con 0x7f1a2c156e80 2024-01-27T02:37:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.129+0000 7f1a32497700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c0aa050 con 0x7f1a2c0c2120 2024-01-27T02:37:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.129+0000 7f1a2bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1a2c156e80 0x7f1a2c0cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.129+0000 7f1a2bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1a2c156e80 0x7f1a2c0cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37442/0 (socket says 172.21.15.7:37442) 2024-01-27T02:37:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.129+0000 7f1a2bfff700 1 -- 172.21.15.7:0/350704320 learned_addr learned my addr 172.21.15.7:0/350704320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.129+0000 7f1a30a34700 1 --2- 172.21.15.7:0/350704320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2c0c2120 0x7f1a2c0aab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.129+0000 7f1a2bfff700 1 -- 172.21.15.7:0/350704320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2c0c2120 msgr2=0x7f1a2c0aab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.129+0000 7f1a2bfff700 1 --2- 172.21.15.7:0/350704320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2c0c2120 0x7f1a2c0aab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.129+0000 7f1a2bfff700 1 -- 172.21.15.7:0/350704320 >> v1:172.21.15.188:6789/0 conn(0x7f1a2c157250 legacy=0x7f1a2c0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.129+0000 7f1a2bfff700 1 -- 172.21.15.7:0/350704320 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2c0c26e0 con 0x7f1a2c156e80 2024-01-27T02:37:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.129+0000 7f1a2bfff700 1 --2- 172.21.15.7:0/350704320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a2c156e80 0x7f1a2c0cb280 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00bc30 tx=0x7f1a1c00b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52c448fe5e4fd30 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.129+0000 7f1a2affd700 1 -- 172.21.15.7:0/350704320 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c00dc50 con 0x7f1a2c156e80 2024-01-27T02:37:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.129+0000 7f1a2affd700 1 -- 172.21.15.7:0/350704320 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1c00ddb0 con 0x7f1a2c156e80 2024-01-27T02:37:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.129+0000 7f1a2affd700 1 -- 172.21.15.7:0/350704320 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c005c30 con 0x7f1a2c156e80 2024-01-27T02:37:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.130+0000 7f1a32497700 1 -- 172.21.15.7:0/350704320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a2c156e80 msgr2=0x7f1a2c0cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.130+0000 7f1a32497700 1 --2- 172.21.15.7:0/350704320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a2c156e80 0x7f1a2c0cb280 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00bc30 tx=0x7f1a1c00b4c0 comp rx=0 tx=0).stop 2024-01-27T02:37:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.130+0000 7f1a32497700 1 -- 172.21.15.7:0/350704320 shutdown_connections 2024-01-27T02:37:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.130+0000 7f1a32497700 1 --2- 172.21.15.7:0/350704320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2c0c2120 0x7f1a2c0aab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.130+0000 7f1a32497700 1 --2- 172.21.15.7:0/350704320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a2c156e80 0x7f1a2c0cb280 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.130+0000 7f1a32497700 1 -- 172.21.15.7:0/350704320 >> 172.21.15.7:0/350704320 conn(0x7f1a2c0679b0 msgr2=0x7f1a2c067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.130+0000 7f1a32497700 1 -- 172.21.15.7:0/350704320 shutdown_connections 2024-01-27T02:37:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.130+0000 7f1a32497700 1 -- 172.21.15.7:0/350704320 wait complete. 2024-01-27T02:37:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.131+0000 7f1a32497700 1 Processor -- start 2024-01-27T02:37:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.132+0000 7f1a32497700 1 -- start start 2024-01-27T02:37:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.132+0000 7f1a32497700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2c0c2120 0x7f1a2c154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.132+0000 7f1a32497700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a2c156e80 0x7f1a2c16adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.132+0000 7f1a32497700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a2c157250 0x7f1a2c0aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.132+0000 7f1a32497700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c16bdd0 con 0x7f1a2c156e80 2024-01-27T02:37:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.132+0000 7f1a32497700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c16c500 con 0x7f1a2c157250 2024-01-27T02:37:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.132+0000 7f1a32497700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c16b810 con 0x7f1a2c0c2120 2024-01-27T02:37:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.132+0000 7f1a2b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a2c156e80 0x7f1a2c16adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.133+0000 7f1a2b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a2c156e80 0x7f1a2c16adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37450/0 (socket says 172.21.15.7:37450) 2024-01-27T02:37:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.133+0000 7f1a2b7fe700 1 -- 172.21.15.7:0/2571782169 learned_addr learned my addr 172.21.15.7:0/2571782169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.133+0000 7f1a2bfff700 1 --2- 172.21.15.7:0/2571782169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2c0c2120 0x7f1a2c154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.133+0000 7f1a30a34700 1 --2- 172.21.15.7:0/2571782169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a2c157250 0x7f1a2c0aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:02.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.133+0000 7f1a2b7fe700 1 -- 172.21.15.7:0/2571782169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2c0c2120 msgr2=0x7f1a2c154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:02.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.133+0000 7f1a2b7fe700 1 --2- 172.21.15.7:0/2571782169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2c0c2120 0x7f1a2c154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:02.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.133+0000 7f1a2b7fe700 1 -- 172.21.15.7:0/2571782169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a2c157250 msgr2=0x7f1a2c0aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:02.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.133+0000 7f1a2b7fe700 1 --2- 172.21.15.7:0/2571782169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a2c157250 0x7f1a2c0aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:02.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.133+0000 7f1a2b7fe700 1 -- 172.21.15.7:0/2571782169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a1c00b720 con 0x7f1a2c156e80 2024-01-27T02:37:02.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.133+0000 7f1a2bfff700 1 --2- 172.21.15.7:0/2571782169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2c0c2120 0x7f1a2c154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:37:02.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.133+0000 7f1a2b7fe700 1 --2- 172.21.15.7:0/2571782169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a2c156e80 0x7f1a2c16adb0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000c4e0 tx=0x7f1a2000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:02.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.133+0000 7f1a13fff700 1 -- 172.21.15.7:0/2571782169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a20011590 con 0x7f1a2c156e80 2024-01-27T02:37:02.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.133+0000 7f1a13fff700 1 -- 172.21.15.7:0/2571782169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a20011710 con 0x7f1a2c156e80 2024-01-27T02:37:02.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.133+0000 7f1a32497700 1 -- 172.21.15.7:0/2571782169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a2c16baf0 con 0x7f1a2c156e80 2024-01-27T02:37:02.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.133+0000 7f1a13fff700 1 -- 172.21.15.7:0/2571782169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a20011ad0 con 0x7f1a2c156e80 2024-01-27T02:37:02.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.133+0000 7f1a32497700 1 -- 172.21.15.7:0/2571782169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2c1516f0 con 0x7f1a2c156e80 2024-01-27T02:37:02.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.134+0000 7f1a32497700 1 -- 172.21.15.7:0/2571782169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2c16bf80 con 0x7f1a2c156e80 2024-01-27T02:37:02.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.135+0000 7f1a13fff700 1 -- 172.21.15.7:0/2571782169 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1a2001b3b0 con 0x7f1a2c156e80 2024-01-27T02:37:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.135+0000 7f1a13fff700 1 --2- 172.21.15.7:0/2571782169 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a18066f00 0x7f1a180693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.135+0000 7f1a13fff700 1 -- 172.21.15.7:0/2571782169 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f1a20083880 con 0x7f1a2c156e80 2024-01-27T02:37:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.135+0000 7f1a2bfff700 1 --2- 172.21.15.7:0/2571782169 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a18066f00 0x7f1a180693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.136+0000 7f1a2bfff700 1 --2- 172.21.15.7:0/2571782169 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a18066f00 0x7f1a180693b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00b500 tx=0x7f1a1c01a000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:02.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.140+0000 7f1a13fff700 1 -- 172.21.15.7:0/2571782169 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a2c16bf80 con 0x7f1a2c156e80 2024-01-27T02:37:02.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.344+0000 7f1a32497700 1 -- 172.21.15.7:0/2571782169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1a2c16c640 con 0x7f1a2c156e80 2024-01-27T02:37:02.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.968+0000 7f1a13fff700 1 -- 172.21.15.7:0/2571782169 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v96) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1a2c16c640 con 0x7f1a2c156e80 2024-01-27T02:37:02.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.975+0000 7f1a32497700 1 -- 172.21.15.7:0/2571782169 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a18066f00 msgr2=0x7f1a180693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:02.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.975+0000 7f1a32497700 1 --2- 172.21.15.7:0/2571782169 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a18066f00 0x7f1a180693b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00b500 tx=0x7f1a1c01a000 comp rx=0 tx=0).stop 2024-01-27T02:37:02.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.975+0000 7f1a32497700 1 -- 172.21.15.7:0/2571782169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a2c156e80 msgr2=0x7f1a2c16adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:02.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.975+0000 7f1a32497700 1 --2- 172.21.15.7:0/2571782169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a2c156e80 0x7f1a2c16adb0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000c4e0 tx=0x7f1a2000c8a0 comp rx=0 tx=0).stop 2024-01-27T02:37:02.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.975+0000 7f1a32497700 1 -- 172.21.15.7:0/2571782169 shutdown_connections 2024-01-27T02:37:02.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.975+0000 7f1a32497700 1 --2- 172.21.15.7:0/2571782169 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a18066f00 0x7f1a180693b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:02.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.975+0000 7f1a32497700 1 --2- 172.21.15.7:0/2571782169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a2c157250 0x7f1a2c0aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:02.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.975+0000 7f1a32497700 1 --2- 172.21.15.7:0/2571782169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2c0c2120 0x7f1a2c154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:02.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.975+0000 7f1a32497700 1 --2- 172.21.15.7:0/2571782169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a2c156e80 0x7f1a2c16adb0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:02.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.975+0000 7f1a32497700 1 -- 172.21.15.7:0/2571782169 >> 172.21.15.7:0/2571782169 conn(0x7f1a2c0679b0 msgr2=0x7f1a2c0cbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:02.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.976+0000 7f1a32497700 1 -- 172.21.15.7:0/2571782169 shutdown_connections 2024-01-27T02:37:02.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:02.976+0000 7f1a32497700 1 -- 172.21.15.7:0/2571782169 wait complete. 2024-01-27T02:37:02.981 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-27T02:37:02.998 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-27T02:37:03.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.135+0000 7f0439655700 1 Processor -- start 2024-01-27T02:37:03.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.135+0000 7f0439655700 1 -- start start 2024-01-27T02:37:03.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.136+0000 7f0439655700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0434154810 0x7f04340aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:03.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.136+0000 7f0439655700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f04340aadf0 0x7f04340c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:03.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.136+0000 7f0439655700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04340c72f0 con 0x7f0434154440 2024-01-27T02:37:03.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.136+0000 7f0439655700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04340cb6e0 con 0x7f04340aadf0 2024-01-27T02:37:03.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.136+0000 7f0439655700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04340ab210 con 0x7f0434154810 2024-01-27T02:37:03.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.136+0000 7f0432ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0434154440 0x7f04340cafd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48172/0 (socket says 172.21.15.7:48172) 2024-01-27T02:37:03.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.136+0000 7f0432ffd700 1 -- 172.21.15.7:0/2264275259 learned_addr learned my addr 172.21.15.7:0/2264275259 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.136+0000 7f04327fc700 1 --2- 172.21.15.7:0/2264275259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0434154810 0x7f04340aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.136+0000 7f04337fe700 1 --2- 172.21.15.7:0/2264275259 >> v2:172.21.15.188:3300/0 conn(0x7f04340aadf0 0x7f04340c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.136+0000 7f0431ffb700 1 -- 172.21.15.7:0/2264275259 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1839102711 0 0) 0x7f04340c72f0 con 0x7f0434154440 2024-01-27T02:37:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.137+0000 7f0431ffb700 1 -- 172.21.15.7:0/2264275259 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f041c003570 con 0x7f0434154440 2024-01-27T02:37:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.137+0000 7f0431ffb700 1 -- 172.21.15.7:0/2264275259 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1461560536 0 0) 0x7f041c003570 con 0x7f0434154440 2024-01-27T02:37:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.137+0000 7f0431ffb700 1 -- 172.21.15.7:0/2264275259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0434154810 msgr2=0x7f04340aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.137+0000 7f0431ffb700 1 --2- 172.21.15.7:0/2264275259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0434154810 0x7f04340aa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.137+0000 7f0431ffb700 1 -- 172.21.15.7:0/2264275259 >> v2:172.21.15.188:3300/0 conn(0x7f04340aadf0 msgr2=0x7f04340c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.137+0000 7f0431ffb700 1 --2- 172.21.15.7:0/2264275259 >> v2:172.21.15.188:3300/0 conn(0x7f04340aadf0 0x7f04340c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.137+0000 7f0431ffb700 1 -- 172.21.15.7:0/2264275259 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04340c7520 con 0x7f0434154440 2024-01-27T02:37:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.137+0000 7f0431ffb700 1 -- 172.21.15.7:0/2264275259 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f0424001470 con 0x7f0434154440 2024-01-27T02:37:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.137+0000 7f0431ffb700 1 -- 172.21.15.7:0/2264275259 >> v1:172.21.15.7:6789/0 conn(0x7f0434154440 legacy=0x7f04340cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.137+0000 7f0431ffb700 1 --2- 172.21.15.7:0/2264275259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0434154810 0x7f041c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:03.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.137+0000 7f0431ffb700 1 --2- 172.21.15.7:0/2264275259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f041c003fe0 0x7f041c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:03.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.137+0000 7f0431ffb700 1 --2- 172.21.15.7:0/2264275259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c004d50 0x7f041c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:03.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.137+0000 7f0431ffb700 1 -- 172.21.15.7:0/2264275259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04340cb6e0 con 0x7f041c004d50 2024-01-27T02:37:03.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.137+0000 7f0431ffb700 1 -- 172.21.15.7:0/2264275259 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04340ab210 con 0x7f041c003fe0 2024-01-27T02:37:03.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.137+0000 7f0431ffb700 1 -- 172.21.15.7:0/2264275259 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041c00ba90 con 0x7f0434154810 2024-01-27T02:37:03.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.137+0000 7f04337fe700 1 --2- 172.21.15.7:0/2264275259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c004d50 0x7f041c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:03.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.137+0000 7f0432ffd700 1 --2- 172.21.15.7:0/2264275259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0434154810 0x7f041c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:03.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.137+0000 7f04327fc700 1 --2- 172.21.15.7:0/2264275259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f041c003fe0 0x7f041c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:03.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.137+0000 7f04337fe700 1 -- 172.21.15.7:0/2264275259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0434154810 msgr2=0x7f041c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:03.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.137+0000 7f04337fe700 1 --2- 172.21.15.7:0/2264275259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0434154810 0x7f041c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:03.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.137+0000 7f04337fe700 1 -- 172.21.15.7:0/2264275259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f041c003fe0 msgr2=0x7f041c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:03.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.137+0000 7f04337fe700 1 --2- 172.21.15.7:0/2264275259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f041c003fe0 0x7f041c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:03.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.138+0000 7f04337fe700 1 -- 172.21.15.7:0/2264275259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c003570 con 0x7f041c004d50 2024-01-27T02:37:03.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.138+0000 7f04337fe700 1 --2- 172.21.15.7:0/2264275259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c004d50 0x7f041c0051c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f042c00e210 tx=0x7f042c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766d4068a38bd96d server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:03.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.138+0000 7f0431ffb700 1 -- 172.21.15.7:0/2264275259 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c00ea60 con 0x7f041c004d50 2024-01-27T02:37:03.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.138+0000 7f0431ffb700 1 -- 172.21.15.7:0/2264275259 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042c00f040 con 0x7f041c004d50 2024-01-27T02:37:03.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.138+0000 7f0431ffb700 1 -- 172.21.15.7:0/2264275259 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c00ee10 con 0x7f041c004d50 2024-01-27T02:37:03.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.138+0000 7f0439655700 1 -- 172.21.15.7:0/2264275259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c004d50 msgr2=0x7f041c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:03.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.138+0000 7f0439655700 1 --2- 172.21.15.7:0/2264275259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c004d50 0x7f041c0051c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f042c00e210 tx=0x7f042c00b040 comp rx=0 tx=0).stop 2024-01-27T02:37:03.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.138+0000 7f04337fe700 1 -- 172.21.15.7:0/2264275259 reap_dead start 2024-01-27T02:37:03.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.139+0000 7f0439655700 1 -- 172.21.15.7:0/2264275259 shutdown_connections 2024-01-27T02:37:03.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.139+0000 7f0439655700 1 -- 172.21.15.7:0/2264275259 >> 172.21.15.7:0/2264275259 conn(0x7f0434063580 msgr2=0x7f0434063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:03.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.139+0000 7f0439655700 1 -- 172.21.15.7:0/2264275259 shutdown_connections 2024-01-27T02:37:03.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.139+0000 7f0439655700 1 -- 172.21.15.7:0/2264275259 wait complete. 2024-01-27T02:37:03.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.140+0000 7f0439655700 1 Processor -- start 2024-01-27T02:37:03.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.140+0000 7f0439655700 1 -- start start 2024-01-27T02:37:03.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.141+0000 7f0439655700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c004d50 0x7f04341501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:03.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.141+0000 7f0439655700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04340b2250 0x7f043406ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:03.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.141+0000 7f0439655700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04340afd60 0x7f04340b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:03.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.141+0000 7f0439655700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04340b0740 con 0x7f041c004d50 2024-01-27T02:37:03.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.141+0000 7f0439655700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043415c360 con 0x7f04340afd60 2024-01-27T02:37:03.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.141+0000 7f0439655700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f043415c640 con 0x7f04340b2250 2024-01-27T02:37:03.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.141+0000 7f0432ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c004d50 0x7f04341501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:03.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.141+0000 7f0432ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c004d50 0x7f04341501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37470/0 (socket says 172.21.15.7:37470) 2024-01-27T02:37:03.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.141+0000 7f0432ffd700 1 -- 172.21.15.7:0/2846919194 learned_addr learned my addr 172.21.15.7:0/2846919194 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:03.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.141+0000 7f04327fc700 1 --2- 172.21.15.7:0/2846919194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04340b2250 0x7f043406ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:03.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.141+0000 7f04337fe700 1 --2- 172.21.15.7:0/2846919194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04340afd60 0x7f04340b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:03.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.141+0000 7f0432ffd700 1 -- 172.21.15.7:0/2846919194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04340b2250 msgr2=0x7f043406ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:03.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.141+0000 7f0432ffd700 1 --2- 172.21.15.7:0/2846919194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04340b2250 0x7f043406ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:03.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.141+0000 7f0432ffd700 1 -- 172.21.15.7:0/2846919194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04340afd60 msgr2=0x7f04340b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:03.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.141+0000 7f0432ffd700 1 --2- 172.21.15.7:0/2846919194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04340afd60 0x7f04340b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:03.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.141+0000 7f0432ffd700 1 -- 172.21.15.7:0/2846919194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043415c920 con 0x7f041c004d50 2024-01-27T02:37:03.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.141+0000 7f0432ffd700 1 --2- 172.21.15.7:0/2846919194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c004d50 0x7f04341501e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f04240079e0 tx=0x7f0424001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:03.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.141+0000 7f041affd700 1 -- 172.21.15.7:0/2846919194 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0424018030 con 0x7f041c004d50 2024-01-27T02:37:03.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.142+0000 7f041affd700 1 -- 172.21.15.7:0/2846919194 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042400f970 con 0x7f041c004d50 2024-01-27T02:37:03.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.142+0000 7f0439655700 1 -- 172.21.15.7:0/2846919194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043415cc00 con 0x7f041c004d50 2024-01-27T02:37:03.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.142+0000 7f041affd700 1 -- 172.21.15.7:0/2846919194 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042400fd10 con 0x7f041c004d50 2024-01-27T02:37:03.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.142+0000 7f0439655700 1 -- 172.21.15.7:0/2846919194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04340b1080 con 0x7f041c004d50 2024-01-27T02:37:03.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.143+0000 7f0439655700 1 -- 172.21.15.7:0/2846919194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0434072060 con 0x7f041c004d50 2024-01-27T02:37:03.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.143+0000 7f041affd700 1 -- 172.21.15.7:0/2846919194 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f042401b070 con 0x7f041c004d50 2024-01-27T02:37:03.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.144+0000 7f041affd700 1 --2- 172.21.15.7:0/2846919194 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0420066f30 0x7f04200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:03.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.144+0000 7f041affd700 1 -- 172.21.15.7:0/2846919194 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f0424019070 con 0x7f041c004d50 2024-01-27T02:37:03.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.144+0000 7f04327fc700 1 --2- 172.21.15.7:0/2846919194 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0420066f30 0x7f04200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:03.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.145+0000 7f04327fc700 1 --2- 172.21.15.7:0/2846919194 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0420066f30 0x7f04200693e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f04340b29b0 tx=0x7f0428008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:03.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.149+0000 7f041affd700 1 -- 172.21.15.7:0/2846919194 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0424051720 con 0x7f041c004d50 2024-01-27T02:37:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.403+0000 7f0439655700 1 -- 172.21.15.7:0/2846919194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f043406d100 con 0x7f041c004d50 2024-01-27T02:37:03.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.986+0000 7f041affd700 1 -- 172.21.15.7:0/2846919194 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v46) v1 ==== 174+0+0 (secure 0 0 0) 0x7f0424054d40 con 0x7f041c004d50 2024-01-27T02:37:03.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.993+0000 7f0439655700 1 -- 172.21.15.7:0/2846919194 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0420066f30 msgr2=0x7f04200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:03.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.993+0000 7f0439655700 1 --2- 172.21.15.7:0/2846919194 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0420066f30 0x7f04200693e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f04340b29b0 tx=0x7f0428008040 comp rx=0 tx=0).stop 2024-01-27T02:37:03.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.993+0000 7f0439655700 1 -- 172.21.15.7:0/2846919194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c004d50 msgr2=0x7f04341501e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:03.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.993+0000 7f0439655700 1 --2- 172.21.15.7:0/2846919194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c004d50 0x7f04341501e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f04240079e0 tx=0x7f0424001e90 comp rx=0 tx=0).stop 2024-01-27T02:37:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.994+0000 7f0439655700 1 -- 172.21.15.7:0/2846919194 shutdown_connections 2024-01-27T02:37:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.994+0000 7f0439655700 1 --2- 172.21.15.7:0/2846919194 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0420066f30 0x7f04200693e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.994+0000 7f0439655700 1 --2- 172.21.15.7:0/2846919194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04340afd60 0x7f04340b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.994+0000 7f0439655700 1 --2- 172.21.15.7:0/2846919194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c004d50 0x7f04341501e0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.994+0000 7f0439655700 1 --2- 172.21.15.7:0/2846919194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04340b2250 0x7f043406ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.994+0000 7f0439655700 1 -- 172.21.15.7:0/2846919194 >> 172.21.15.7:0/2846919194 conn(0x7f0434063580 msgr2=0x7f04340692d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:03.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.996+0000 7f0439655700 1 -- 172.21.15.7:0/2846919194 shutdown_connections 2024-01-27T02:37:03.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:03.996+0000 7f0439655700 1 -- 172.21.15.7:0/2846919194 wait complete. 2024-01-27T02:37:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 12 and data pool 13 2024-01-27T02:37:04.013 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:37:04.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.146+0000 7fb59bc3e700 1 Processor -- start 2024-01-27T02:37:04.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.146+0000 7fb59bc3e700 1 -- start start 2024-01-27T02:37:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.147+0000 7fb59bc3e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb594157ca0 0x7fb5940c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.147+0000 7fb59bc3e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb5940c3ae0 0x7fb5940cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.147+0000 7fb59bc3e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb594158070 0x7fb5940cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.147+0000 7fb59bc3e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5940ca020 con 0x7fb594157ca0 2024-01-27T02:37:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.147+0000 7fb59bc3e700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5940c4f40 con 0x7fb5940c3ae0 2024-01-27T02:37:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.147+0000 7fb59bc3e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5940c5210 con 0x7fb594158070 2024-01-27T02:37:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.147+0000 7fb5999da700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb594157ca0 0x7fb5940c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.147+0000 7fb5999da700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb594157ca0 0x7fb5940c4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37472/0 (socket says 172.21.15.7:37472) 2024-01-27T02:37:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.147+0000 7fb5999da700 1 -- 172.21.15.7:0/3377050790 learned_addr learned my addr 172.21.15.7:0/3377050790 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.148+0000 7fb5991d9700 1 --2- 172.21.15.7:0/3377050790 >> v2:172.21.15.188:3300/0 conn(0x7fb5940c3ae0 0x7fb5940cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.148+0000 7fb59a1db700 1 --2- 172.21.15.7:0/3377050790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb594158070 0x7fb5940cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.148+0000 7fb5999da700 1 -- 172.21.15.7:0/3377050790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb594158070 msgr2=0x7fb5940cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.148+0000 7fb5999da700 1 --2- 172.21.15.7:0/3377050790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb594158070 0x7fb5940cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.148+0000 7fb5999da700 1 -- 172.21.15.7:0/3377050790 >> v2:172.21.15.188:3300/0 conn(0x7fb5940c3ae0 msgr2=0x7fb5940cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.148+0000 7fb5999da700 1 --2- 172.21.15.7:0/3377050790 >> v2:172.21.15.188:3300/0 conn(0x7fb5940c3ae0 0x7fb5940cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.148+0000 7fb5999da700 1 -- 172.21.15.7:0/3377050790 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5940c54f0 con 0x7fb594157ca0 2024-01-27T02:37:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.148+0000 7fb5999da700 1 --2- 172.21.15.7:0/3377050790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb594157ca0 0x7fb5940c4a00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb59406e170 tx=0x7fb590009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad949ba338a7f13 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.148+0000 7fb5989d8700 1 -- 172.21.15.7:0/3377050790 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59000b860 con 0x7fb594157ca0 2024-01-27T02:37:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.148+0000 7fb5989d8700 1 -- 172.21.15.7:0/3377050790 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb59000b9c0 con 0x7fb594157ca0 2024-01-27T02:37:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.148+0000 7fb5989d8700 1 -- 172.21.15.7:0/3377050790 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59000bd60 con 0x7fb594157ca0 2024-01-27T02:37:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.149+0000 7fb59bc3e700 1 -- 172.21.15.7:0/3377050790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb594157ca0 msgr2=0x7fb5940c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.149+0000 7fb59bc3e700 1 --2- 172.21.15.7:0/3377050790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb594157ca0 0x7fb5940c4a00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb59406e170 tx=0x7fb590009250 comp rx=0 tx=0).stop 2024-01-27T02:37:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.149+0000 7fb59bc3e700 1 -- 172.21.15.7:0/3377050790 shutdown_connections 2024-01-27T02:37:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.149+0000 7fb59bc3e700 1 --2- 172.21.15.7:0/3377050790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb594158070 0x7fb5940cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.149+0000 7fb59bc3e700 1 --2- 172.21.15.7:0/3377050790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb594157ca0 0x7fb5940c4a00 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.149+0000 7fb59bc3e700 1 --2- 172.21.15.7:0/3377050790 >> v2:172.21.15.188:3300/0 conn(0x7fb5940c3ae0 0x7fb5940cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.149+0000 7fb59bc3e700 1 -- 172.21.15.7:0/3377050790 >> 172.21.15.7:0/3377050790 conn(0x7fb594098020 msgr2=0x7fb5940711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.149+0000 7fb59bc3e700 1 -- 172.21.15.7:0/3377050790 shutdown_connections 2024-01-27T02:37:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.149+0000 7fb59bc3e700 1 -- 172.21.15.7:0/3377050790 wait complete. 2024-01-27T02:37:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.150+0000 7fb59bc3e700 1 Processor -- start 2024-01-27T02:37:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.150+0000 7fb59bc3e700 1 -- start start 2024-01-27T02:37:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.151+0000 7fb59bc3e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5940c3ae0 0x7fb5940b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.151+0000 7fb59bc3e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb594157ca0 0x7fb5940b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.151+0000 7fb59bc3e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb594158070 0x7fb59414d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.151+0000 7fb59bc3e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5940b2af0 con 0x7fb594158070 2024-01-27T02:37:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.151+0000 7fb59bc3e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5940b3440 con 0x7fb5940c3ae0 2024-01-27T02:37:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.151+0000 7fb59bc3e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5940b3720 con 0x7fb594157ca0 2024-01-27T02:37:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.151+0000 7fb59a1db700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb594158070 0x7fb59414d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.151+0000 7fb59a1db700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb594158070 0x7fb59414d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37474/0 (socket says 172.21.15.7:37474) 2024-01-27T02:37:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.151+0000 7fb59a1db700 1 -- 172.21.15.7:0/2704783492 learned_addr learned my addr 172.21.15.7:0/2704783492 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.151+0000 7fb5999da700 1 --2- 172.21.15.7:0/2704783492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5940c3ae0 0x7fb5940b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.151+0000 7fb5991d9700 1 --2- 172.21.15.7:0/2704783492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb594157ca0 0x7fb5940b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.151+0000 7fb59a1db700 1 -- 172.21.15.7:0/2704783492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb594157ca0 msgr2=0x7fb5940b25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.151+0000 7fb59a1db700 1 --2- 172.21.15.7:0/2704783492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb594157ca0 0x7fb5940b25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.151+0000 7fb59a1db700 1 -- 172.21.15.7:0/2704783492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5940c3ae0 msgr2=0x7fb5940b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.151+0000 7fb59a1db700 1 --2- 172.21.15.7:0/2704783492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5940c3ae0 0x7fb5940b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.151+0000 7fb59a1db700 1 -- 172.21.15.7:0/2704783492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5900094f0 con 0x7fb594158070 2024-01-27T02:37:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.151+0000 7fb5991d9700 1 --2- 172.21.15.7:0/2704783492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb594157ca0 0x7fb5940b25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.152+0000 7fb59a1db700 1 --2- 172.21.15.7:0/2704783492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb594158070 0x7fb59414d8b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb58c007f20 tx=0x7fb58c009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.152+0000 7fb589ffb700 1 -- 172.21.15.7:0/2704783492 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c00eca0 con 0x7fb594158070 2024-01-27T02:37:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.152+0000 7fb589ffb700 1 -- 172.21.15.7:0/2704783492 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58c00ee20 con 0x7fb594158070 2024-01-27T02:37:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.152+0000 7fb59bc3e700 1 -- 172.21.15.7:0/2704783492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5940b3a00 con 0x7fb594158070 2024-01-27T02:37:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.152+0000 7fb589ffb700 1 -- 172.21.15.7:0/2704783492 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c014670 con 0x7fb594158070 2024-01-27T02:37:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.152+0000 7fb59bc3e700 1 -- 172.21.15.7:0/2704783492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59414ddf0 con 0x7fb594158070 2024-01-27T02:37:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.153+0000 7fb59bc3e700 1 -- 172.21.15.7:0/2704783492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb59404c710 con 0x7fb594158070 2024-01-27T02:37:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.154+0000 7fb589ffb700 1 -- 172.21.15.7:0/2704783492 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb58c0149e0 con 0x7fb594158070 2024-01-27T02:37:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.154+0000 7fb589ffb700 1 --2- 172.21.15.7:0/2704783492 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb578066f30 0x7fb5780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.154+0000 7fb589ffb700 1 -- 172.21.15.7:0/2704783492 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb58c084c60 con 0x7fb594158070 2024-01-27T02:37:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.154+0000 7fb5999da700 1 --2- 172.21.15.7:0/2704783492 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb578066f30 0x7fb5780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.155+0000 7fb5999da700 1 --2- 172.21.15.7:0/2704783492 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb578066f30 0x7fb5780693e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb59406e170 tx=0x7fb590017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:04.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.158+0000 7fb589ffb700 1 -- 172.21.15.7:0/2704783492 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb58c051ed0 con 0x7fb594158070 2024-01-27T02:37:04.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.366+0000 7fb59bc3e700 1 -- 172.21.15.7:0/2704783492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5940c57d0 con 0x7fb594158070 2024-01-27T02:37:04.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.367+0000 7fb589ffb700 1 -- 172.21.15.7:0/2704783492 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+15318 (secure 0 0 0) 0x7fb58c0554f0 con 0x7fb594158070 2024-01-27T02:37:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:04.367 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":97,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:37:03.404031+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:37:01.445734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-27T02:37:02.377151+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:37:04.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.369+0000 7fb59bc3e700 1 -- 172.21.15.7:0/2704783492 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb578066f30 msgr2=0x7fb5780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:04.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.369+0000 7fb59bc3e700 1 --2- 172.21.15.7:0/2704783492 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb578066f30 0x7fb5780693e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb59406e170 tx=0x7fb590017000 comp rx=0 tx=0).stop 2024-01-27T02:37:04.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.369+0000 7fb59bc3e700 1 -- 172.21.15.7:0/2704783492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb594158070 msgr2=0x7fb59414d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:04.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.369+0000 7fb59bc3e700 1 --2- 172.21.15.7:0/2704783492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb594158070 0x7fb59414d8b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb58c007f20 tx=0x7fb58c009860 comp rx=0 tx=0).stop 2024-01-27T02:37:04.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.369+0000 7fb59bc3e700 1 -- 172.21.15.7:0/2704783492 shutdown_connections 2024-01-27T02:37:04.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.369+0000 7fb59bc3e700 1 --2- 172.21.15.7:0/2704783492 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb578066f30 0x7fb5780693e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.369+0000 7fb59bc3e700 1 --2- 172.21.15.7:0/2704783492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb594158070 0x7fb59414d8b0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.369+0000 7fb59bc3e700 1 --2- 172.21.15.7:0/2704783492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5940c3ae0 0x7fb5940b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.369+0000 7fb59bc3e700 1 --2- 172.21.15.7:0/2704783492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb594157ca0 0x7fb5940b25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.369+0000 7fb59bc3e700 1 -- 172.21.15.7:0/2704783492 >> 172.21.15.7:0/2704783492 conn(0x7fb594098020 msgr2=0x7fb5940cd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:04.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.369+0000 7fb59bc3e700 1 -- 172.21.15.7:0/2704783492 shutdown_connections 2024-01-27T02:37:04.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.369+0000 7fb59bc3e700 1 -- 172.21.15.7:0/2704783492 wait complete. 2024-01-27T02:37:04.384 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:37:04.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.514+0000 7ff1d2ef2700 1 Processor -- start 2024-01-27T02:37:04.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.515+0000 7ff1d2ef2700 1 -- start start 2024-01-27T02:37:04.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.516+0000 7ff1d2ef2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff1cc0a7c10 0x7ff1cc0ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:04.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.516+0000 7ff1d2ef2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1cc0a7fe0 0x7ff1cc0b0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:04.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.516+0000 7ff1d2ef2700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1cc15a880 con 0x7ff1cc0ae880 2024-01-27T02:37:04.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.516+0000 7ff1d2ef2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1cc0afde0 con 0x7ff1cc0a7c10 2024-01-27T02:37:04.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.516+0000 7ff1d2ef2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc0af950 con 0x7ff1cc0a7fe0 2024-01-27T02:37:04.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.516+0000 7ff1d0c8e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff1cc0a7c10 0x7ff1cc0ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:04.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.516+0000 7ff1d0c8e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff1cc0a7c10 0x7ff1cc0ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37476/0 (socket says 172.21.15.7:37476) 2024-01-27T02:37:04.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.516+0000 7ff1d0c8e700 1 -- 172.21.15.7:0/4151536352 learned_addr learned my addr 172.21.15.7:0/4151536352 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:04.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.516+0000 7ff1d148f700 1 --2- 172.21.15.7:0/4151536352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1cc0a7fe0 0x7ff1cc0b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:04.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.516+0000 7ff1d0c8e700 1 -- 172.21.15.7:0/4151536352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1cc0a7fe0 msgr2=0x7ff1cc0b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:04.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.516+0000 7ff1d0c8e700 1 --2- 172.21.15.7:0/4151536352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1cc0a7fe0 0x7ff1cc0b0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.516+0000 7ff1d0c8e700 1 -- 172.21.15.7:0/4151536352 >> v1:172.21.15.188:6789/0 conn(0x7ff1cc0ae880 legacy=0x7ff1cc06f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:04.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.516+0000 7ff1d0c8e700 1 -- 172.21.15.7:0/4151536352 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc0aed40 con 0x7ff1cc0a7c10 2024-01-27T02:37:04.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.517+0000 7ff1d0c8e700 1 --2- 172.21.15.7:0/4151536352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1cc0a7c10 0x7ff1cc0ae340 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0099b0 tx=0x7ff1bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0763a80221fbee8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:04.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.517+0000 7ff1cb7fe700 1 -- 172.21.15.7:0/4151536352 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc00b860 con 0x7ff1cc0a7c10 2024-01-27T02:37:04.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.517+0000 7ff1cb7fe700 1 -- 172.21.15.7:0/4151536352 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1bc00b9c0 con 0x7ff1cc0a7c10 2024-01-27T02:37:04.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.517+0000 7ff1cb7fe700 1 -- 172.21.15.7:0/4151536352 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc00bd60 con 0x7ff1cc0a7c10 2024-01-27T02:37:04.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.517+0000 7ff1d2ef2700 1 -- 172.21.15.7:0/4151536352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1cc0a7c10 msgr2=0x7ff1cc0ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:04.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.517+0000 7ff1d2ef2700 1 --2- 172.21.15.7:0/4151536352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1cc0a7c10 0x7ff1cc0ae340 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0099b0 tx=0x7ff1bc009250 comp rx=0 tx=0).stop 2024-01-27T02:37:04.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.517+0000 7ff1d2ef2700 1 -- 172.21.15.7:0/4151536352 shutdown_connections 2024-01-27T02:37:04.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.517+0000 7ff1d2ef2700 1 --2- 172.21.15.7:0/4151536352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1cc0a7fe0 0x7ff1cc0b0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.517+0000 7ff1d2ef2700 1 --2- 172.21.15.7:0/4151536352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1cc0a7c10 0x7ff1cc0ae340 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.517+0000 7ff1d2ef2700 1 -- 172.21.15.7:0/4151536352 >> 172.21.15.7:0/4151536352 conn(0x7ff1cc0679b0 msgr2=0x7ff1cc067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:04.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.518+0000 7ff1d2ef2700 1 -- 172.21.15.7:0/4151536352 shutdown_connections 2024-01-27T02:37:04.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.518+0000 7ff1d2ef2700 1 -- 172.21.15.7:0/4151536352 wait complete. 2024-01-27T02:37:04.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.519+0000 7ff1d2ef2700 1 Processor -- start 2024-01-27T02:37:04.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.519+0000 7ff1d2ef2700 1 -- start start 2024-01-27T02:37:04.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.520+0000 7ff1d2ef2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1cc0a7c10 0x7ff1cc150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:04.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.520+0000 7ff1d2ef2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1cc0a7fe0 0x7ff1cc15eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:04.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.520+0000 7ff1d2ef2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1cc0ae880 0x7ff1cc15f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:04.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.520+0000 7ff1d2ef2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc16ba90 con 0x7ff1cc0a7fe0 2024-01-27T02:37:04.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.520+0000 7ff1d2ef2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc097b70 con 0x7ff1cc0ae880 2024-01-27T02:37:04.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.520+0000 7ff1d2ef2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc097e10 con 0x7ff1cc0a7c10 2024-01-27T02:37:04.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.520+0000 7ff1cbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1cc0a7fe0 0x7ff1cc15eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:04.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.520+0000 7ff1cbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1cc0a7fe0 0x7ff1cc15eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37484/0 (socket says 172.21.15.7:37484) 2024-01-27T02:37:04.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.520+0000 7ff1cbfff700 1 -- 172.21.15.7:0/1171869666 learned_addr learned my addr 172.21.15.7:0/1171869666 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:04.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.520+0000 7ff1d0c8e700 1 --2- 172.21.15.7:0/1171869666 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1cc0a7c10 0x7ff1cc150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:04.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.520+0000 7ff1d148f700 1 --2- 172.21.15.7:0/1171869666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1cc0ae880 0x7ff1cc15f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:04.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.520+0000 7ff1cbfff700 1 -- 172.21.15.7:0/1171869666 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1cc0a7c10 msgr2=0x7ff1cc150cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:04.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.520+0000 7ff1cbfff700 1 --2- 172.21.15.7:0/1171869666 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1cc0a7c10 0x7ff1cc150cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.520+0000 7ff1cbfff700 1 -- 172.21.15.7:0/1171869666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1cc0ae880 msgr2=0x7ff1cc15f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:04.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.520+0000 7ff1cbfff700 1 --2- 172.21.15.7:0/1171869666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1cc0ae880 0x7ff1cc15f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.520+0000 7ff1cbfff700 1 -- 172.21.15.7:0/1171869666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1bc0094f0 con 0x7ff1cc0a7fe0 2024-01-27T02:37:04.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.520+0000 7ff1cbfff700 1 --2- 172.21.15.7:0/1171869666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1cc0a7fe0 0x7ff1cc15eea0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff1b400c590 tx=0x7ff1b4007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:04.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.521+0000 7ff1c8ff9700 1 -- 172.21.15.7:0/1171869666 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4011590 con 0x7ff1cc0a7fe0 2024-01-27T02:37:04.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.521+0000 7ff1c8ff9700 1 -- 172.21.15.7:0/1171869666 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b4011710 con 0x7ff1cc0a7fe0 2024-01-27T02:37:04.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.521+0000 7ff1d2ef2700 1 -- 172.21.15.7:0/1171869666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1cc0980f0 con 0x7ff1cc0a7fe0 2024-01-27T02:37:04.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.521+0000 7ff1c8ff9700 1 -- 172.21.15.7:0/1171869666 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4011ad0 con 0x7ff1cc0a7fe0 2024-01-27T02:37:04.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.521+0000 7ff1d2ef2700 1 -- 172.21.15.7:0/1171869666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1cc151210 con 0x7ff1cc0a7fe0 2024-01-27T02:37:04.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.522+0000 7ff1d2ef2700 1 -- 172.21.15.7:0/1171869666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1cc04c710 con 0x7ff1cc0a7fe0 2024-01-27T02:37:04.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.522+0000 7ff1c8ff9700 1 -- 172.21.15.7:0/1171869666 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff1b401b3b0 con 0x7ff1cc0a7fe0 2024-01-27T02:37:04.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.523+0000 7ff1c8ff9700 1 --2- 172.21.15.7:0/1171869666 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff1ac066f30 0x7ff1ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:04.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.523+0000 7ff1c8ff9700 1 -- 172.21.15.7:0/1171869666 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff1b4083910 con 0x7ff1cc0a7fe0 2024-01-27T02:37:04.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.523+0000 7ff1d0c8e700 1 --2- 172.21.15.7:0/1171869666 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff1ac066f30 0x7ff1ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:04.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.523+0000 7ff1d0c8e700 1 --2- 172.21.15.7:0/1171869666 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff1ac066f30 0x7ff1ac0693e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0099b0 tx=0x7ff1bc017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:04.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.527+0000 7ff1c8ff9700 1 -- 172.21.15.7:0/1171869666 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1b4050b80 con 0x7ff1cc0a7fe0 2024-01-27T02:37:04.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.729+0000 7ff1d2ef2700 1 -- 172.21.15.7:0/1171869666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1cc0af020 con 0x7ff1cc0a7fe0 2024-01-27T02:37:04.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.730+0000 7ff1c8ff9700 1 -- 172.21.15.7:0/1171869666 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+15318 (secure 0 0 0) 0x7ff1b4017070 con 0x7ff1cc0a7fe0 2024-01-27T02:37:04.730 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:04.730 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":97,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:37:03.404031+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:37:01.445734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-27T02:37:02.377151+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:37:04.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.732+0000 7ff1d2ef2700 1 -- 172.21.15.7:0/1171869666 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff1ac066f30 msgr2=0x7ff1ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:04.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.732+0000 7ff1d2ef2700 1 --2- 172.21.15.7:0/1171869666 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff1ac066f30 0x7ff1ac0693e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0099b0 tx=0x7ff1bc017000 comp rx=0 tx=0).stop 2024-01-27T02:37:04.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.732+0000 7ff1d2ef2700 1 -- 172.21.15.7:0/1171869666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1cc0a7fe0 msgr2=0x7ff1cc15eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:04.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.732+0000 7ff1d2ef2700 1 --2- 172.21.15.7:0/1171869666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1cc0a7fe0 0x7ff1cc15eea0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff1b400c590 tx=0x7ff1b4007f50 comp rx=0 tx=0).stop 2024-01-27T02:37:04.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.732+0000 7ff1d2ef2700 1 -- 172.21.15.7:0/1171869666 shutdown_connections 2024-01-27T02:37:04.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.732+0000 7ff1d2ef2700 1 --2- 172.21.15.7:0/1171869666 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff1ac066f30 0x7ff1ac0693e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.732+0000 7ff1d2ef2700 1 --2- 172.21.15.7:0/1171869666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1cc0ae880 0x7ff1cc15f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.732+0000 7ff1d2ef2700 1 --2- 172.21.15.7:0/1171869666 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1cc0a7c10 0x7ff1cc150cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.732+0000 7ff1d2ef2700 1 --2- 172.21.15.7:0/1171869666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1cc0a7fe0 0x7ff1cc15eea0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.732+0000 7ff1d2ef2700 1 -- 172.21.15.7:0/1171869666 >> 172.21.15.7:0/1171869666 conn(0x7ff1cc0679b0 msgr2=0x7ff1cc1599f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:04.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.732+0000 7ff1d2ef2700 1 -- 172.21.15.7:0/1171869666 shutdown_connections 2024-01-27T02:37:04.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.732+0000 7ff1d2ef2700 1 -- 172.21.15.7:0/1171869666 wait complete. 2024-01-27T02:37:04.746 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-27T02:37:04.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.882+0000 7fbbb5ea3700 1 Processor -- start 2024-01-27T02:37:04.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.882+0000 7fbbb5ea3700 1 -- start start 2024-01-27T02:37:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.883+0000 7fbbb5ea3700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbbb015d460 0x7fbbb00d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.884+0000 7fbbb5ea3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbbb00cf280 0x7fbbb015e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.884+0000 7fbbb5ea3700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb00b38d0 con 0x7fbbb015d830 2024-01-27T02:37:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.884+0000 7fbbb5ea3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb00d0390 con 0x7fbbb00cf280 2024-01-27T02:37:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.884+0000 7fbbb5ea3700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb00cf780 con 0x7fbbb015d460 2024-01-27T02:37:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.884+0000 7fbbaeffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbbb00cf280 0x7fbbb015e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.884+0000 7fbbaeffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbbb00cf280 0x7fbbb015e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37488/0 (socket says 172.21.15.7:37488) 2024-01-27T02:37:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.884+0000 7fbbaeffd700 1 -- 172.21.15.7:0/3868918044 learned_addr learned my addr 172.21.15.7:0/3868918044 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.884+0000 7fbbaf7fe700 1 --2- 172.21.15.7:0/3868918044 >> v2:172.21.15.188:3300/0 conn(0x7fbbb015d460 0x7fbbb00d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.884+0000 7fbbaeffd700 1 -- 172.21.15.7:0/3868918044 >> v2:172.21.15.188:3300/0 conn(0x7fbbb015d460 msgr2=0x7fbbb00d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:04.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.884+0000 7fbbaeffd700 1 --2- 172.21.15.7:0/3868918044 >> v2:172.21.15.188:3300/0 conn(0x7fbbb015d460 0x7fbbb00d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.884+0000 7fbbaeffd700 1 -- 172.21.15.7:0/3868918044 >> v1:172.21.15.188:6789/0 conn(0x7fbbb015d830 legacy=0x7fbbb00cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:04.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.884+0000 7fbbaeffd700 1 -- 172.21.15.7:0/3868918044 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb00cfa60 con 0x7fbbb00cf280 2024-01-27T02:37:04.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.884+0000 7fbbaeffd700 1 --2- 172.21.15.7:0/3868918044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbb00cf280 0x7fbbb015e6b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00ce640 tx=0x7fbb98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29355f843986bd2 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:04.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.884+0000 7fbbae7fc700 1 -- 172.21.15.7:0/3868918044 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98009e70 con 0x7fbbb00cf280 2024-01-27T02:37:04.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.884+0000 7fbbae7fc700 1 -- 172.21.15.7:0/3868918044 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9800b860 con 0x7fbbb00cf280 2024-01-27T02:37:04.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.884+0000 7fbbae7fc700 1 -- 172.21.15.7:0/3868918044 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9800bc00 con 0x7fbbb00cf280 2024-01-27T02:37:04.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.885+0000 7fbbb5ea3700 1 -- 172.21.15.7:0/3868918044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbb00cf280 msgr2=0x7fbbb015e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:04.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.885+0000 7fbbb5ea3700 1 --2- 172.21.15.7:0/3868918044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbb00cf280 0x7fbbb015e6b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00ce640 tx=0x7fbb98009250 comp rx=0 tx=0).stop 2024-01-27T02:37:04.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.885+0000 7fbbb5ea3700 1 -- 172.21.15.7:0/3868918044 shutdown_connections 2024-01-27T02:37:04.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.885+0000 7fbbb5ea3700 1 --2- 172.21.15.7:0/3868918044 >> v2:172.21.15.188:3300/0 conn(0x7fbbb015d460 0x7fbbb00d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.885+0000 7fbbb5ea3700 1 --2- 172.21.15.7:0/3868918044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbb00cf280 0x7fbbb015e6b0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.885+0000 7fbbb5ea3700 1 -- 172.21.15.7:0/3868918044 >> 172.21.15.7:0/3868918044 conn(0x7fbbb0063580 msgr2=0x7fbbb0063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:04.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.885+0000 7fbbb5ea3700 1 -- 172.21.15.7:0/3868918044 shutdown_connections 2024-01-27T02:37:04.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.885+0000 7fbbb5ea3700 1 -- 172.21.15.7:0/3868918044 wait complete. 2024-01-27T02:37:04.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.887+0000 7fbbb5ea3700 1 Processor -- start 2024-01-27T02:37:04.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.887+0000 7fbbb5ea3700 1 -- start start 2024-01-27T02:37:04.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.888+0000 7fbbb5ea3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbb00cf280 0x7fbbb00c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:04.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.888+0000 7fbbb5ea3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbb015d460 0x7fbbb01515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:04.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.888+0000 7fbbb5ea3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbb015d830 0x7fbbb00b5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:04.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.888+0000 7fbbb5ea3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb016b7d0 con 0x7fbbb015d830 2024-01-27T02:37:04.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.888+0000 7fbbb5ea3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb016abc0 con 0x7fbbb00cf280 2024-01-27T02:37:04.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.888+0000 7fbbb5ea3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb016aea0 con 0x7fbbb015d460 2024-01-27T02:37:04.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.888+0000 7fbbaffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbb015d830 0x7fbbb00b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:04.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.888+0000 7fbbaffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbb015d830 0x7fbbb00b5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37500/0 (socket says 172.21.15.7:37500) 2024-01-27T02:37:04.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.888+0000 7fbbaffff700 1 -- 172.21.15.7:0/4066390186 learned_addr learned my addr 172.21.15.7:0/4066390186 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:04.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.888+0000 7fbbaf7fe700 1 --2- 172.21.15.7:0/4066390186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbb00cf280 0x7fbbb00c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:04.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.888+0000 7fbbaeffd700 1 --2- 172.21.15.7:0/4066390186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbb015d460 0x7fbbb01515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:04.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.889+0000 7fbbaffff700 1 -- 172.21.15.7:0/4066390186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbb015d460 msgr2=0x7fbbb01515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:04.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.889+0000 7fbbaffff700 1 --2- 172.21.15.7:0/4066390186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbb015d460 0x7fbbb01515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.889+0000 7fbbaffff700 1 -- 172.21.15.7:0/4066390186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbb00cf280 msgr2=0x7fbbb00c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:04.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.889+0000 7fbbaffff700 1 --2- 172.21.15.7:0/4066390186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbb00cf280 0x7fbbb00c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:04.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.889+0000 7fbbaffff700 1 -- 172.21.15.7:0/4066390186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb980094f0 con 0x7fbbb015d830 2024-01-27T02:37:04.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.889+0000 7fbbaffff700 1 --2- 172.21.15.7:0/4066390186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbb015d830 0x7fbbb00b5570 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fbba800c840 tx=0x7fbba8001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:04.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.889+0000 7fbba77fe700 1 -- 172.21.15.7:0/4066390186 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8012630 con 0x7fbbb015d830 2024-01-27T02:37:04.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.889+0000 7fbba77fe700 1 -- 172.21.15.7:0/4066390186 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba80127b0 con 0x7fbbb015d830 2024-01-27T02:37:04.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.889+0000 7fbbb5ea3700 1 -- 172.21.15.7:0/4066390186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb016b180 con 0x7fbbb015d830 2024-01-27T02:37:04.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.889+0000 7fbba77fe700 1 -- 172.21.15.7:0/4066390186 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8012b70 con 0x7fbbb015d830 2024-01-27T02:37:04.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.889+0000 7fbbb5ea3700 1 -- 172.21.15.7:0/4066390186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb016a880 con 0x7fbbb015d830 2024-01-27T02:37:04.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.890+0000 7fbbb5ea3700 1 -- 172.21.15.7:0/4066390186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb00b3ca0 con 0x7fbbb015d830 2024-01-27T02:37:04.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.891+0000 7fbba77fe700 1 -- 172.21.15.7:0/4066390186 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbba801a3a0 con 0x7fbbb015d830 2024-01-27T02:37:04.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.891+0000 7fbba77fe700 1 --2- 172.21.15.7:0/4066390186 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb90066f30 0x7fbb900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:04.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.891+0000 7fbba77fe700 1 -- 172.21.15.7:0/4066390186 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbba8084510 con 0x7fbbb015d830 2024-01-27T02:37:04.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.891+0000 7fbbaf7fe700 1 --2- 172.21.15.7:0/4066390186 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb90066f30 0x7fbb900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:04.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.892+0000 7fbbaf7fe700 1 --2- 172.21.15.7:0/4066390186 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb90066f30 0x7fbb900693e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009db0 tx=0x7fbba00091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:04.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:04.895+0000 7fbba77fe700 1 -- 172.21.15.7:0/4066390186 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbbb00b3ca0 con 0x7fbbb015d830 2024-01-27T02:37:05.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:05.145+0000 7fbbb5ea3700 1 -- 172.21.15.7:0/4066390186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbbb00b38d0 con 0x7fbbb015d830 2024-01-27T02:37:05.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:05.989+0000 7fbba77fe700 1 -- 172.21.15.7:0/4066390186 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v48) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbbb00b38d0 con 0x7fbbb015d830 2024-01-27T02:37:05.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:05.992+0000 7fbbb5ea3700 1 -- 172.21.15.7:0/4066390186 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb90066f30 msgr2=0x7fbb900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:05.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:05.992+0000 7fbbb5ea3700 1 --2- 172.21.15.7:0/4066390186 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb90066f30 0x7fbb900693e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009db0 tx=0x7fbba00091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:05.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:05.992+0000 7fbbb5ea3700 1 -- 172.21.15.7:0/4066390186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbb015d830 msgr2=0x7fbbb00b5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:05.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:05.992+0000 7fbbb5ea3700 1 --2- 172.21.15.7:0/4066390186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbb015d830 0x7fbbb00b5570 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fbba800c840 tx=0x7fbba8001670 comp rx=0 tx=0).stop 2024-01-27T02:37:05.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:05.992+0000 7fbbb5ea3700 1 -- 172.21.15.7:0/4066390186 shutdown_connections 2024-01-27T02:37:05.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:05.992+0000 7fbbb5ea3700 1 --2- 172.21.15.7:0/4066390186 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb90066f30 0x7fbb900693e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:05.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:05.992+0000 7fbbb5ea3700 1 --2- 172.21.15.7:0/4066390186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbb015d830 0x7fbbb00b5570 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:05.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:05.992+0000 7fbbb5ea3700 1 --2- 172.21.15.7:0/4066390186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbb00cf280 0x7fbbb00c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:05.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:05.992+0000 7fbbb5ea3700 1 --2- 172.21.15.7:0/4066390186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbb015d460 0x7fbbb01515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:05.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:05.992+0000 7fbbb5ea3700 1 -- 172.21.15.7:0/4066390186 >> 172.21.15.7:0/4066390186 conn(0x7fbbb0063580 msgr2=0x7fbbb00d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:05.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:05.993+0000 7fbbb5ea3700 1 -- 172.21.15.7:0/4066390186 shutdown_connections 2024-01-27T02:37:05.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:05.993+0000 7fbbb5ea3700 1 -- 172.21.15.7:0/4066390186 wait complete. 2024-01-27T02:37:06.009 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-27T02:37:06.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.147+0000 7f60779d9700 1 Processor -- start 2024-01-27T02:37:06.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.147+0000 7f60779d9700 1 -- start start 2024-01-27T02:37:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.149+0000 7f60779d9700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f607006ea30 0x7f607015a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.149+0000 7f60779d9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60701408e0 0x7f607015aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.149+0000 7f60779d9700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60700cb7c0 con 0x7f6070140510 2024-01-27T02:37:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.149+0000 7f60779d9700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60700cc060 con 0x7f607006ea30 2024-01-27T02:37:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.149+0000 7f60779d9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f607015b410 con 0x7f60701408e0 2024-01-27T02:37:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.149+0000 7f6074f74700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f607006ea30 0x7f607015a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.149+0000 7f6075f76700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60701408e0 0x7f607015aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.149+0000 7f6074f74700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f607006ea30 0x7f607015a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:60868/0 (socket says 172.21.15.7:60868) 2024-01-27T02:37:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.149+0000 7f6074f74700 1 -- 172.21.15.7:0/3245374839 learned_addr learned my addr 172.21.15.7:0/3245374839 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.149+0000 7f6075775700 1 --1- 172.21.15.7:0/3245374839 >> v1:172.21.15.188:6789/0 conn(0x7f6070140510 0x7f607006e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:55590/0 (socket says 172.21.15.7:55590) 2024-01-27T02:37:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.149+0000 7f6075f76700 1 -- 172.21.15.7:0/3245374839 >> v2:172.21.15.188:3300/0 conn(0x7f607006ea30 msgr2=0x7f607015a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:06.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.149+0000 7f6075f76700 1 --2- 172.21.15.7:0/3245374839 >> v2:172.21.15.188:3300/0 conn(0x7f607006ea30 0x7f607015a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:06.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.149+0000 7f6075f76700 1 -- 172.21.15.7:0/3245374839 >> v1:172.21.15.188:6789/0 conn(0x7f6070140510 legacy=0x7f607006e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:06.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.149+0000 7f6075f76700 1 -- 172.21.15.7:0/3245374839 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607015b6c0 con 0x7f60701408e0 2024-01-27T02:37:06.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.149+0000 7f6063fff700 1 -- 172.21.15.7:0/3245374839 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3499069834 0 0) 0x7f60700cb7c0 con 0x7f6070140510 2024-01-27T02:37:06.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.149+0000 7f6074f74700 1 --2- 172.21.15.7:0/3245374839 >> v2:172.21.15.188:3300/0 conn(0x7f607006ea30 0x7f607015a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:37:06.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.150+0000 7f6075f76700 1 --2- 172.21.15.7:0/3245374839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60701408e0 0x7f607015aed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6068009e60 tx=0x7f6068009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25800c705d619223 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:06.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.150+0000 7f6063fff700 1 -- 172.21.15.7:0/3245374839 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606800e070 con 0x7f60701408e0 2024-01-27T02:37:06.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.150+0000 7f6063fff700 1 -- 172.21.15.7:0/3245374839 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606800bb60 con 0x7f60701408e0 2024-01-27T02:37:06.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.150+0000 7f6063fff700 1 -- 172.21.15.7:0/3245374839 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6068012430 con 0x7f60701408e0 2024-01-27T02:37:06.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.150+0000 7f60779d9700 1 -- 172.21.15.7:0/3245374839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60701408e0 msgr2=0x7f607015aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:06.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.150+0000 7f60779d9700 1 --2- 172.21.15.7:0/3245374839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60701408e0 0x7f607015aed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6068009e60 tx=0x7f6068009780 comp rx=0 tx=0).stop 2024-01-27T02:37:06.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.150+0000 7f60779d9700 1 -- 172.21.15.7:0/3245374839 shutdown_connections 2024-01-27T02:37:06.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.150+0000 7f60779d9700 1 --2- 172.21.15.7:0/3245374839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60701408e0 0x7f607015aed0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:06.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.150+0000 7f60779d9700 1 --2- 172.21.15.7:0/3245374839 >> v2:172.21.15.188:3300/0 conn(0x7f607006ea30 0x7f607015a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:06.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.150+0000 7f60779d9700 1 -- 172.21.15.7:0/3245374839 >> 172.21.15.7:0/3245374839 conn(0x7f60700cdbf0 msgr2=0x7f60700cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:06.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.150+0000 7f60779d9700 1 -- 172.21.15.7:0/3245374839 shutdown_connections 2024-01-27T02:37:06.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.151+0000 7f60779d9700 1 -- 172.21.15.7:0/3245374839 wait complete. 2024-01-27T02:37:06.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.152+0000 7f60779d9700 1 Processor -- start 2024-01-27T02:37:06.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.152+0000 7f60779d9700 1 -- start start 2024-01-27T02:37:06.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.153+0000 7f60779d9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f607006ea30 0x7f60700b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:06.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.153+0000 7f60779d9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6070140510 0x7f60700b1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:06.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.153+0000 7f60779d9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60701408e0 0x7f60701567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:06.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.153+0000 7f60779d9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60700b2a50 con 0x7f60701408e0 2024-01-27T02:37:06.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.153+0000 7f60779d9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60700b1c70 con 0x7f6070140510 2024-01-27T02:37:06.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.153+0000 7f60779d9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60700b2770 con 0x7f607006ea30 2024-01-27T02:37:06.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.153+0000 7f6075f76700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60701408e0 0x7f60701567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:06.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.153+0000 7f6075f76700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60701408e0 0x7f60701567a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37514/0 (socket says 172.21.15.7:37514) 2024-01-27T02:37:06.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.153+0000 7f6075f76700 1 -- 172.21.15.7:0/1287272825 learned_addr learned my addr 172.21.15.7:0/1287272825 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:06.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.153+0000 7f6075775700 1 --2- 172.21.15.7:0/1287272825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f607006ea30 0x7f60700b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:06.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.153+0000 7f6074f74700 1 --2- 172.21.15.7:0/1287272825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6070140510 0x7f60700b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:06.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.153+0000 7f6075f76700 1 -- 172.21.15.7:0/1287272825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f607006ea30 msgr2=0x7f60700b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:06.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.153+0000 7f6075f76700 1 --2- 172.21.15.7:0/1287272825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f607006ea30 0x7f60700b20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:06.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.153+0000 7f6075f76700 1 -- 172.21.15.7:0/1287272825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6070140510 msgr2=0x7f60700b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:06.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.153+0000 7f6075f76700 1 --2- 172.21.15.7:0/1287272825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6070140510 0x7f60700b1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:06.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.153+0000 7f6075f76700 1 -- 172.21.15.7:0/1287272825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606c0094f0 con 0x7f60701408e0 2024-01-27T02:37:06.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.153+0000 7f6075775700 1 --2- 172.21.15.7:0/1287272825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f607006ea30 0x7f60700b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:06.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.153+0000 7f6074f74700 1 --2- 172.21.15.7:0/1287272825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6070140510 0x7f60700b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:06.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.153+0000 7f6075f76700 1 --2- 172.21.15.7:0/1287272825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60701408e0 0x7f60701567a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f607015a880 tx=0x7f6068019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:06.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.154+0000 7f60617fa700 1 -- 172.21.15.7:0/1287272825 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6068027470 con 0x7f60701408e0 2024-01-27T02:37:06.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.154+0000 7f60617fa700 1 -- 172.21.15.7:0/1287272825 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6068027610 con 0x7f60701408e0 2024-01-27T02:37:06.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.154+0000 7f60779d9700 1 -- 172.21.15.7:0/1287272825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6068009a20 con 0x7f60701408e0 2024-01-27T02:37:06.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.154+0000 7f60617fa700 1 -- 172.21.15.7:0/1287272825 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60680279d0 con 0x7f60701408e0 2024-01-27T02:37:06.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.154+0000 7f60779d9700 1 -- 172.21.15.7:0/1287272825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60701587a0 con 0x7f60701408e0 2024-01-27T02:37:06.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.155+0000 7f60779d9700 1 -- 172.21.15.7:0/1287272825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6070072060 con 0x7f60701408e0 2024-01-27T02:37:06.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.155+0000 7f60617fa700 1 -- 172.21.15.7:0/1287272825 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6068024070 con 0x7f60701408e0 2024-01-27T02:37:06.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.155+0000 7f60617fa700 1 --2- 172.21.15.7:0/1287272825 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6054066f30 0x7f60540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:06.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.156+0000 7f60617fa700 1 -- 172.21.15.7:0/1287272825 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6068087040 con 0x7f60701408e0 2024-01-27T02:37:06.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.156+0000 7f6075775700 1 --2- 172.21.15.7:0/1287272825 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6054066f30 0x7f60540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:06.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.156+0000 7f6075775700 1 --2- 172.21.15.7:0/1287272825 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6054066f30 0x7f60540693e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f60640018d0 tx=0x7f6064009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:06.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.160+0000 7f60617fa700 1 -- 172.21.15.7:0/1287272825 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f60680542b0 con 0x7f60701408e0 2024-01-27T02:37:06.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.412+0000 7f60779d9700 1 -- 172.21.15.7:0/1287272825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f60700b38a0 con 0x7f60701408e0 2024-01-27T02:37:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.990+0000 7f60617fa700 1 -- 172.21.15.7:0/1287272825 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v49) v1 ==== 122+0+0 (secure 0 0 0) 0x7f60680578d0 con 0x7f60701408e0 2024-01-27T02:37:06.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.993+0000 7f60779d9700 1 -- 172.21.15.7:0/1287272825 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6054066f30 msgr2=0x7f60540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:06.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.993+0000 7f60779d9700 1 --2- 172.21.15.7:0/1287272825 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6054066f30 0x7f60540693e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f60640018d0 tx=0x7f6064009040 comp rx=0 tx=0).stop 2024-01-27T02:37:06.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.993+0000 7f60779d9700 1 -- 172.21.15.7:0/1287272825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60701408e0 msgr2=0x7f60701567a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:06.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.993+0000 7f60779d9700 1 --2- 172.21.15.7:0/1287272825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60701408e0 0x7f60701567a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f607015a880 tx=0x7f6068019000 comp rx=0 tx=0).stop 2024-01-27T02:37:06.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.993+0000 7f60779d9700 1 -- 172.21.15.7:0/1287272825 shutdown_connections 2024-01-27T02:37:06.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.993+0000 7f60779d9700 1 --2- 172.21.15.7:0/1287272825 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6054066f30 0x7f60540693e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.993+0000 7f60779d9700 1 --2- 172.21.15.7:0/1287272825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60701408e0 0x7f60701567a0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.993+0000 7f60779d9700 1 --2- 172.21.15.7:0/1287272825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f607006ea30 0x7f60700b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.993+0000 7f60779d9700 1 --2- 172.21.15.7:0/1287272825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6070140510 0x7f60700b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.993+0000 7f60779d9700 1 -- 172.21.15.7:0/1287272825 >> 172.21.15.7:0/1287272825 conn(0x7f60700cdbf0 msgr2=0x7f60700ca470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.994+0000 7f60779d9700 1 -- 172.21.15.7:0/1287272825 shutdown_connections 2024-01-27T02:37:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:06.994+0000 7f60779d9700 1 -- 172.21.15.7:0/1287272825 wait complete. 2024-01-27T02:37:07.010 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:37:07.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.145+0000 7f46c9852700 1 Processor -- start 2024-01-27T02:37:07.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.145+0000 7f46c9852700 1 -- start start 2024-01-27T02:37:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.146+0000 7f46c9852700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f46c415a450 0x7f46c40b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.146+0000 7f46c9852700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c40cc980 con 0x7f46c40b2030 2024-01-27T02:37:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.146+0000 7f46c9852700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c40ccc20 con 0x7f46c40ce0f0 2024-01-27T02:37:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.146+0000 7f46c9852700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46c40ccf00 con 0x7f46c415a450 2024-01-27T02:37:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.146+0000 7f46c2ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f46c415a450 0x7f46c40b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.146+0000 7f46c2ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f46c415a450 0x7f46c40b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37518/0 (socket says 172.21.15.7:37518) 2024-01-27T02:37:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.146+0000 7f46c2ffd700 1 -- 172.21.15.7:0/2598307008 learned_addr learned my addr 172.21.15.7:0/2598307008 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.146+0000 7f46c27fc700 1 --1- 172.21.15.7:0/2598307008 >> v1:172.21.15.7:6789/0 conn(0x7f46c40b2030 0x7f46c40cd9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48182/0 (socket says 172.21.15.7:48182) 2024-01-27T02:37:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.146+0000 7f46c2ffd700 1 -- 172.21.15.7:0/2598307008 >> v1:172.21.15.188:6789/0 conn(0x7f46c40ce0f0 legacy=0x7f46c40c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.146+0000 7f46c2ffd700 1 -- 172.21.15.7:0/2598307008 >> v1:172.21.15.7:6789/0 conn(0x7f46c40b2030 legacy=0x7f46c40cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.147+0000 7f46c2ffd700 1 -- 172.21.15.7:0/2598307008 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c40cd1e0 con 0x7f46c415a450 2024-01-27T02:37:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.147+0000 7f46c2ffd700 1 --2- 172.21.15.7:0/2598307008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46c415a450 0x7f46c40b1af0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009990 tx=0x7f46b4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a89f3db63aa7ee2 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.147+0000 7f46c1ffb700 1 -- 172.21.15.7:0/2598307008 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b400b760 con 0x7f46c415a450 2024-01-27T02:37:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.147+0000 7f46c1ffb700 1 -- 172.21.15.7:0/2598307008 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b400b8c0 con 0x7f46c415a450 2024-01-27T02:37:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.147+0000 7f46c1ffb700 1 -- 172.21.15.7:0/2598307008 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b400bc60 con 0x7f46c415a450 2024-01-27T02:37:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.148+0000 7f46c9852700 1 -- 172.21.15.7:0/2598307008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46c415a450 msgr2=0x7f46c40b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.148+0000 7f46c9852700 1 --2- 172.21.15.7:0/2598307008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46c415a450 0x7f46c40b1af0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009990 tx=0x7f46b4009220 comp rx=0 tx=0).stop 2024-01-27T02:37:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.148+0000 7f46c9852700 1 -- 172.21.15.7:0/2598307008 shutdown_connections 2024-01-27T02:37:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.148+0000 7f46c9852700 1 --2- 172.21.15.7:0/2598307008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46c415a450 0x7f46c40b1af0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.148+0000 7f46c9852700 1 -- 172.21.15.7:0/2598307008 >> 172.21.15.7:0/2598307008 conn(0x7f46c40711c0 msgr2=0x7f46c40ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.148+0000 7f46c9852700 1 -- 172.21.15.7:0/2598307008 shutdown_connections 2024-01-27T02:37:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.148+0000 7f46c9852700 1 -- 172.21.15.7:0/2598307008 wait complete. 2024-01-27T02:37:07.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.149+0000 7f46c9852700 1 Processor -- start 2024-01-27T02:37:07.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.149+0000 7f46c9852700 1 -- start start 2024-01-27T02:37:07.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.150+0000 7f46c9852700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46c40b2030 0x7f46c40b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:07.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.150+0000 7f46c9852700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46c40ce0f0 0x7f46c40c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:07.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.150+0000 7f46c9852700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46c415a450 0x7f46c41550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:07.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.150+0000 7f46c9852700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c416e130 con 0x7f46c415a450 2024-01-27T02:37:07.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.150+0000 7f46c9852700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c40b45e0 con 0x7f46c40ce0f0 2024-01-27T02:37:07.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.150+0000 7f46c9852700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c40b48c0 con 0x7f46c40b2030 2024-01-27T02:37:07.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.150+0000 7f46c37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46c415a450 0x7f46c41550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:07.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.150+0000 7f46c37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46c415a450 0x7f46c41550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37532/0 (socket says 172.21.15.7:37532) 2024-01-27T02:37:07.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.150+0000 7f46c37fe700 1 -- 172.21.15.7:0/3596255171 learned_addr learned my addr 172.21.15.7:0/3596255171 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:07.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.150+0000 7f46c27fc700 1 --2- 172.21.15.7:0/3596255171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46c40ce0f0 0x7f46c40c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:07.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.150+0000 7f46c2ffd700 1 --2- 172.21.15.7:0/3596255171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46c40b2030 0x7f46c40b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:07.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.150+0000 7f46c37fe700 1 -- 172.21.15.7:0/3596255171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46c40b2030 msgr2=0x7f46c40b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:07.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.150+0000 7f46c37fe700 1 --2- 172.21.15.7:0/3596255171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46c40b2030 0x7f46c40b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:07.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.150+0000 7f46c37fe700 1 -- 172.21.15.7:0/3596255171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46c40ce0f0 msgr2=0x7f46c40c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:07.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.150+0000 7f46c37fe700 1 --2- 172.21.15.7:0/3596255171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46c40ce0f0 0x7f46c40c5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:07.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.150+0000 7f46c37fe700 1 -- 172.21.15.7:0/3596255171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b4009480 con 0x7f46c415a450 2024-01-27T02:37:07.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.150+0000 7f46c2ffd700 1 --2- 172.21.15.7:0/3596255171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46c40b2030 0x7f46c40b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:07.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.150+0000 7f46c37fe700 1 --2- 172.21.15.7:0/3596255171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46c415a450 0x7f46c41550e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00c8e0 tx=0x7f46bc001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:07.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.151+0000 7f46aaffd700 1 -- 172.21.15.7:0/3596255171 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc00ce80 con 0x7f46c415a450 2024-01-27T02:37:07.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.151+0000 7f46aaffd700 1 -- 172.21.15.7:0/3596255171 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46bc00a7e0 con 0x7f46c415a450 2024-01-27T02:37:07.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.151+0000 7f46c9852700 1 -- 172.21.15.7:0/3596255171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c416e3f0 con 0x7f46c415a450 2024-01-27T02:37:07.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.151+0000 7f46aaffd700 1 -- 172.21.15.7:0/3596255171 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc00ab80 con 0x7f46c415a450 2024-01-27T02:37:07.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.151+0000 7f46c9852700 1 -- 172.21.15.7:0/3596255171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c4155860 con 0x7f46c415a450 2024-01-27T02:37:07.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.152+0000 7f46c9852700 1 -- 172.21.15.7:0/3596255171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c416e760 con 0x7f46c415a450 2024-01-27T02:37:07.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.152+0000 7f46aaffd700 1 -- 172.21.15.7:0/3596255171 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f46bc012420 con 0x7f46c415a450 2024-01-27T02:37:07.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.153+0000 7f46aaffd700 1 --2- 172.21.15.7:0/3596255171 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f46b0066f30 0x7f46b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:07.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.153+0000 7f46aaffd700 1 -- 172.21.15.7:0/3596255171 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f46bc084840 con 0x7f46c415a450 2024-01-27T02:37:07.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.153+0000 7f46c2ffd700 1 --2- 172.21.15.7:0/3596255171 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f46b0066f30 0x7f46b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:07.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.154+0000 7f46c2ffd700 1 --2- 172.21.15.7:0/3596255171 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f46b0066f30 0x7f46b00693e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f46b400c010 tx=0x7f46b4016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:07.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.158+0000 7f46aaffd700 1 -- 172.21.15.7:0/3596255171 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46bc051ab0 con 0x7f46c415a450 2024-01-27T02:37:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.406+0000 7f46c9852700 1 -- 172.21.15.7:0/3596255171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46c404c710 con 0x7f46c415a450 2024-01-27T02:37:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.407+0000 7f46aaffd700 1 -- 172.21.15.7:0/3596255171 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4584 (secure 0 0 0) 0x7f46bc0550d0 con 0x7f46c415a450 2024-01-27T02:37:07.408 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:07.408 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/450556982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":450556982},{"type":"v1","addr":"172.21.15.188:6836","nonce":450556982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1711422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1711422},{"type":"v1","addr":"172.21.15.188:6837","nonce":1711422}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6294,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1110306600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1110306600},{"type":"v1","addr":"172.21.15.7:6833","nonce":1110306600}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:37:03.404355+0000","modified":"2024-01-27T02:37:06.988798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6297":{"gid":6297,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/328992853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":328992853},{"type":"v1","addr":"172.21.15.7:6835","nonce":328992853}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-27T02:37:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.409+0000 7f46c9852700 1 -- 172.21.15.7:0/3596255171 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f46b0066f30 msgr2=0x7f46b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.409+0000 7f46c9852700 1 --2- 172.21.15.7:0/3596255171 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f46b0066f30 0x7f46b00693e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f46b400c010 tx=0x7f46b4016000 comp rx=0 tx=0).stop 2024-01-27T02:37:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.409+0000 7f46c9852700 1 -- 172.21.15.7:0/3596255171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46c415a450 msgr2=0x7f46c41550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.409+0000 7f46c9852700 1 --2- 172.21.15.7:0/3596255171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46c415a450 0x7f46c41550e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00c8e0 tx=0x7f46bc001710 comp rx=0 tx=0).stop 2024-01-27T02:37:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.409+0000 7f46c9852700 1 -- 172.21.15.7:0/3596255171 shutdown_connections 2024-01-27T02:37:07.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.409+0000 7f46c9852700 1 --2- 172.21.15.7:0/3596255171 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f46b0066f30 0x7f46b00693e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:07.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.409+0000 7f46c9852700 1 --2- 172.21.15.7:0/3596255171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46c415a450 0x7f46c41550e0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:07.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.409+0000 7f46c9852700 1 --2- 172.21.15.7:0/3596255171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46c40b2030 0x7f46c40b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:07.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.409+0000 7f46c9852700 1 --2- 172.21.15.7:0/3596255171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46c40ce0f0 0x7f46c40c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:07.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.409+0000 7f46c9852700 1 -- 172.21.15.7:0/3596255171 >> 172.21.15.7:0/3596255171 conn(0x7f46c40711c0 msgr2=0x7f46c40caa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:07.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.410+0000 7f46c9852700 1 -- 172.21.15.7:0/3596255171 shutdown_connections 2024-01-27T02:37:07.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.410+0000 7f46c9852700 1 -- 172.21.15.7:0/3596255171 wait complete. 2024-01-27T02:37:07.411 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 49 2024-01-27T02:37:07.424 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:37:07.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.555+0000 7fdc204aa700 1 Processor -- start 2024-01-27T02:37:07.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.555+0000 7fdc204aa700 1 -- start start 2024-01-27T02:37:07.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.556+0000 7fdc204aa700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc1806a960 0x7fdc18063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:07.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.556+0000 7fdc204aa700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdc18158c30 0x7fdc18063ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:07.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.556+0000 7fdc204aa700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc18159450 con 0x7fdc1815c730 2024-01-27T02:37:07.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.556+0000 7fdc204aa700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1806c8a0 con 0x7fdc18158c30 2024-01-27T02:37:07.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.556+0000 7fdc204aa700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1815c2a0 con 0x7fdc1806a960 2024-01-27T02:37:07.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.556+0000 7fdc1da45700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdc1815c730 0x7fdc18158520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48196/0 (socket says 172.21.15.7:48196) 2024-01-27T02:37:07.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.556+0000 7fdc1da45700 1 -- 172.21.15.7:0/3123957718 learned_addr learned my addr 172.21.15.7:0/3123957718 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:07.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.556+0000 7fdc1e246700 1 --2- 172.21.15.7:0/3123957718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc1806a960 0x7fdc18063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:07.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.556+0000 7fdc1ea47700 1 --2- 172.21.15.7:0/3123957718 >> v2:172.21.15.188:3300/0 conn(0x7fdc18158c30 0x7fdc18063ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:07.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.556+0000 7fdc1d244700 1 -- 172.21.15.7:0/3123957718 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285317221 0 0) 0x7fdc18159450 con 0x7fdc1815c730 2024-01-27T02:37:07.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.556+0000 7fdc1d244700 1 -- 172.21.15.7:0/3123957718 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc04003570 con 0x7fdc1815c730 2024-01-27T02:37:07.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.557+0000 7fdc1d244700 1 -- 172.21.15.7:0/3123957718 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1403840948 0 0) 0x7fdc04003570 con 0x7fdc1815c730 2024-01-27T02:37:07.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.557+0000 7fdc1e246700 1 -- 172.21.15.7:0/3123957718 >> v2:172.21.15.188:3300/0 conn(0x7fdc18158c30 msgr2=0x7fdc18063ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:07.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.557+0000 7fdc1e246700 1 --2- 172.21.15.7:0/3123957718 >> v2:172.21.15.188:3300/0 conn(0x7fdc18158c30 0x7fdc18063ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:07.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.557+0000 7fdc1e246700 1 -- 172.21.15.7:0/3123957718 >> v1:172.21.15.7:6789/0 conn(0x7fdc1815c730 legacy=0x7fdc18158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:07.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.557+0000 7fdc1e246700 1 -- 172.21.15.7:0/3123957718 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc180ca080 con 0x7fdc1806a960 2024-01-27T02:37:07.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.557+0000 7fdc1ea47700 1 --2- 172.21.15.7:0/3123957718 >> v2:172.21.15.188:3300/0 conn(0x7fdc18158c30 0x7fdc18063ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:37:07.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.557+0000 7fdc1e246700 1 --2- 172.21.15.7:0/3123957718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc1806a960 0x7fdc18063580 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdc140098e0 tx=0x7fdc140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc6a5538a05c400f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:07.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.557+0000 7fdc1d244700 1 -- 172.21.15.7:0/3123957718 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc08001470 con 0x7fdc1806a960 2024-01-27T02:37:07.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.557+0000 7fdc1d244700 1 -- 172.21.15.7:0/3123957718 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc14011420 con 0x7fdc1806a960 2024-01-27T02:37:07.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.557+0000 7fdc1d244700 1 -- 172.21.15.7:0/3123957718 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc140117a0 con 0x7fdc1806a960 2024-01-27T02:37:07.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.558+0000 7fdc204aa700 1 -- 172.21.15.7:0/3123957718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc1806a960 msgr2=0x7fdc18063580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:07.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.558+0000 7fdc204aa700 1 --2- 172.21.15.7:0/3123957718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc1806a960 0x7fdc18063580 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdc140098e0 tx=0x7fdc140091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:07.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.558+0000 7fdc204aa700 1 -- 172.21.15.7:0/3123957718 shutdown_connections 2024-01-27T02:37:07.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.558+0000 7fdc204aa700 1 --2- 172.21.15.7:0/3123957718 >> v2:172.21.15.188:3300/0 conn(0x7fdc18158c30 0x7fdc18063ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:07.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.558+0000 7fdc204aa700 1 --2- 172.21.15.7:0/3123957718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc1806a960 0x7fdc18063580 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:07.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.558+0000 7fdc204aa700 1 -- 172.21.15.7:0/3123957718 >> 172.21.15.7:0/3123957718 conn(0x7fdc180cdc30 msgr2=0x7fdc180ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:07.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.558+0000 7fdc204aa700 1 -- 172.21.15.7:0/3123957718 shutdown_connections 2024-01-27T02:37:07.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.558+0000 7fdc204aa700 1 -- 172.21.15.7:0/3123957718 wait complete. 2024-01-27T02:37:07.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.559+0000 7fdc204aa700 1 Processor -- start 2024-01-27T02:37:07.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.559+0000 7fdc204aa700 1 -- start start 2024-01-27T02:37:07.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.560+0000 7fdc204aa700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc1806a960 0x7fdc18137a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:07.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.560+0000 7fdc204aa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc18158c30 0x7fdc1815a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:07.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.560+0000 7fdc204aa700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc1815c730 0x7fdc1815a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:07.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.560+0000 7fdc204aa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc18143170 con 0x7fdc18158c30 2024-01-27T02:37:07.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.560+0000 7fdc204aa700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc18137f60 con 0x7fdc1815c730 2024-01-27T02:37:07.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.560+0000 7fdc204aa700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc18142560 con 0x7fdc1806a960 2024-01-27T02:37:07.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.560+0000 7fdc1da45700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc18158c30 0x7fdc1815a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:07.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.560+0000 7fdc1da45700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc18158c30 0x7fdc1815a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37544/0 (socket says 172.21.15.7:37544) 2024-01-27T02:37:07.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.560+0000 7fdc1da45700 1 -- 172.21.15.7:0/2074011556 learned_addr learned my addr 172.21.15.7:0/2074011556 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:07.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.560+0000 7fdc1e246700 1 --2- 172.21.15.7:0/2074011556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc1806a960 0x7fdc18137a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:07.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.560+0000 7fdc1ea47700 1 --2- 172.21.15.7:0/2074011556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc1815c730 0x7fdc1815a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:07.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.560+0000 7fdc1da45700 1 -- 172.21.15.7:0/2074011556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc1806a960 msgr2=0x7fdc18137a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:07.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.560+0000 7fdc1da45700 1 --2- 172.21.15.7:0/2074011556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc1806a960 0x7fdc18137a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:07.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.560+0000 7fdc1da45700 1 -- 172.21.15.7:0/2074011556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc1815c730 msgr2=0x7fdc1815a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:07.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.560+0000 7fdc1da45700 1 --2- 172.21.15.7:0/2074011556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc1815c730 0x7fdc1815a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:07.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.560+0000 7fdc1da45700 1 -- 172.21.15.7:0/2074011556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc10009a20 con 0x7fdc18158c30 2024-01-27T02:37:07.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.561+0000 7fdc1da45700 1 --2- 172.21.15.7:0/2074011556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc18158c30 0x7fdc1815a400 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800cb90 tx=0x7fdc08001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:07.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.561+0000 7fdc0e7fc700 1 -- 172.21.15.7:0/2074011556 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc08001d60 con 0x7fdc18158c30 2024-01-27T02:37:07.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.561+0000 7fdc0e7fc700 1 -- 172.21.15.7:0/2074011556 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc08007360 con 0x7fdc18158c30 2024-01-27T02:37:07.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.561+0000 7fdc0e7fc700 1 -- 172.21.15.7:0/2074011556 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc08007700 con 0x7fdc18158c30 2024-01-27T02:37:07.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.561+0000 7fdc204aa700 1 -- 172.21.15.7:0/2074011556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc14009420 con 0x7fdc18158c30 2024-01-27T02:37:07.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.561+0000 7fdc204aa700 1 -- 172.21.15.7:0/2074011556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc18142b20 con 0x7fdc18158c30 2024-01-27T02:37:07.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.562+0000 7fdc204aa700 1 -- 172.21.15.7:0/2074011556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc1804c710 con 0x7fdc18158c30 2024-01-27T02:37:07.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.562+0000 7fdc0e7fc700 1 -- 172.21.15.7:0/2074011556 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fdc08004a40 con 0x7fdc18158c30 2024-01-27T02:37:07.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.563+0000 7fdc0e7fc700 1 --2- 172.21.15.7:0/2074011556 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdbfc066f30 0x7fdbfc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:07.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.563+0000 7fdc0e7fc700 1 -- 172.21.15.7:0/2074011556 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdc08083f10 con 0x7fdc18158c30 2024-01-27T02:37:07.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.563+0000 7fdc1e246700 1 --2- 172.21.15.7:0/2074011556 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdbfc066f30 0x7fdbfc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:07.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.564+0000 7fdc1e246700 1 --2- 172.21.15.7:0/2074011556 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdbfc066f30 0x7fdbfc0693e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdc140091e0 tx=0x7fdc14016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:07.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.567+0000 7fdc0e7fc700 1 -- 172.21.15.7:0/2074011556 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc08051160 con 0x7fdc18158c30 2024-01-27T02:37:07.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.772+0000 7fdc204aa700 1 -- 172.21.15.7:0/2074011556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc18143500 con 0x7fdc18158c30 2024-01-27T02:37:07.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.773+0000 7fdc0e7fc700 1 -- 172.21.15.7:0/2074011556 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+15305 (secure 0 0 0) 0x7fdc08018020 con 0x7fdc18158c30 2024-01-27T02:37:07.774 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:07.774 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":98,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:37:04.968738+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:37:01.445734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-27T02:37:02.377151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:37:07.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.775+0000 7fdc204aa700 1 -- 172.21.15.7:0/2074011556 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdbfc066f30 msgr2=0x7fdbfc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:07.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.775+0000 7fdc204aa700 1 --2- 172.21.15.7:0/2074011556 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdbfc066f30 0x7fdbfc0693e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdc140091e0 tx=0x7fdc14016000 comp rx=0 tx=0).stop 2024-01-27T02:37:07.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.775+0000 7fdc204aa700 1 -- 172.21.15.7:0/2074011556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc18158c30 msgr2=0x7fdc1815a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:07.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.775+0000 7fdc204aa700 1 --2- 172.21.15.7:0/2074011556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc18158c30 0x7fdc1815a400 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800cb90 tx=0x7fdc08001ad0 comp rx=0 tx=0).stop 2024-01-27T02:37:07.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.775+0000 7fdc204aa700 1 -- 172.21.15.7:0/2074011556 shutdown_connections 2024-01-27T02:37:07.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.775+0000 7fdc204aa700 1 --2- 172.21.15.7:0/2074011556 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdbfc066f30 0x7fdbfc0693e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:07.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.775+0000 7fdc204aa700 1 --2- 172.21.15.7:0/2074011556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc1815c730 0x7fdc1815a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:07.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.775+0000 7fdc204aa700 1 --2- 172.21.15.7:0/2074011556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc1806a960 0x7fdc18137a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:07.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.775+0000 7fdc204aa700 1 --2- 172.21.15.7:0/2074011556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc18158c30 0x7fdc1815a400 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:07.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.775+0000 7fdc204aa700 1 -- 172.21.15.7:0/2074011556 >> 172.21.15.7:0/2074011556 conn(0x7fdc180cdc30 msgr2=0x7fdc1814ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:07.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.775+0000 7fdc204aa700 1 -- 172.21.15.7:0/2074011556 shutdown_connections 2024-01-27T02:37:07.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.775+0000 7fdc204aa700 1 -- 172.21.15.7:0/2074011556 wait complete. 2024-01-27T02:37:07.790 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-27T02:37:07.790 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:37:07.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.921+0000 7f6e168e5700 1 Processor -- start 2024-01-27T02:37:07.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.921+0000 7f6e168e5700 1 -- start start 2024-01-27T02:37:07.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.922+0000 7f6e168e5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6e101438f0 0x7f6e100c8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:07.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.922+0000 7f6e168e5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e100cb7a0 con 0x7f6e10063c90 2024-01-27T02:37:07.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.922+0000 7f6e168e5700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e100cb9f0 con 0x7f6e10143520 2024-01-27T02:37:07.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.922+0000 7f6e168e5700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e100cbcd0 con 0x7f6e101438f0 2024-01-27T02:37:07.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.922+0000 7f6e14e82700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6e101438f0 0x7f6e100c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.922+0000 7f6e0f7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6e10063c90 0x7f6e100c8210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48206/0 (socket says 172.21.15.7:48206) 2024-01-27T02:37:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.922+0000 7f6e0f7fe700 1 -- 172.21.15.7:0/3701373265 learned_addr learned my addr 172.21.15.7:0/3701373265 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.922+0000 7f6e14e82700 1 -- 172.21.15.7:0/3701373265 >> v1:172.21.15.188:6789/0 conn(0x7f6e10143520 legacy=0x7f6e10063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.922+0000 7f6e14e82700 1 -- 172.21.15.7:0/3701373265 >> v1:172.21.15.7:6789/0 conn(0x7f6e10063c90 legacy=0x7f6e100c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.922+0000 7f6e0effd700 1 -- 172.21.15.7:0/3701373265 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2181139061 0 0) 0x7f6e100cb7a0 con 0x7f6e10063c90 2024-01-27T02:37:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.922+0000 7f6e14e82700 1 -- 172.21.15.7:0/3701373265 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e100cbfb0 con 0x7f6e101438f0 2024-01-27T02:37:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.923+0000 7f6e14e82700 1 --2- 172.21.15.7:0/3701373265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e101438f0 0x7f6e100c8920 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6e08009e60 tx=0x7f6e08009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5702739367db42fc server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.923+0000 7f6e0effd700 1 -- 172.21.15.7:0/3701373265 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0800e070 con 0x7f6e101438f0 2024-01-27T02:37:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.923+0000 7f6e0effd700 1 -- 172.21.15.7:0/3701373265 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0800bc30 con 0x7f6e101438f0 2024-01-27T02:37:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.923+0000 7f6e0effd700 1 -- 172.21.15.7:0/3701373265 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08012430 con 0x7f6e101438f0 2024-01-27T02:37:07.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.923+0000 7f6e168e5700 1 -- 172.21.15.7:0/3701373265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e101438f0 msgr2=0x7f6e100c8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:07.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.923+0000 7f6e168e5700 1 --2- 172.21.15.7:0/3701373265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e101438f0 0x7f6e100c8920 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6e08009e60 tx=0x7f6e08009780 comp rx=0 tx=0).stop 2024-01-27T02:37:07.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.923+0000 7f6e168e5700 1 -- 172.21.15.7:0/3701373265 shutdown_connections 2024-01-27T02:37:07.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.923+0000 7f6e168e5700 1 --2- 172.21.15.7:0/3701373265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e101438f0 0x7f6e100c8920 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:07.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.923+0000 7f6e168e5700 1 -- 172.21.15.7:0/3701373265 >> 172.21.15.7:0/3701373265 conn(0x7f6e100cdbd0 msgr2=0x7f6e100cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:07.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.923+0000 7f6e168e5700 1 -- 172.21.15.7:0/3701373265 shutdown_connections 2024-01-27T02:37:07.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.923+0000 7f6e168e5700 1 -- 172.21.15.7:0/3701373265 wait complete. 2024-01-27T02:37:07.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.925+0000 7f6e168e5700 1 Processor -- start 2024-01-27T02:37:07.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.925+0000 7f6e168e5700 1 -- start start 2024-01-27T02:37:07.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.926+0000 7f6e168e5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e10063c90 0x7f6e101408c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:07.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.926+0000 7f6e168e5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e10143520 0x7f6e100ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:07.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.926+0000 7f6e168e5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e101438f0 0x7f6e100cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:07.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.926+0000 7f6e168e5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e100b19c0 con 0x7f6e10143520 2024-01-27T02:37:07.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.926+0000 7f6e168e5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e100b16e0 con 0x7f6e101438f0 2024-01-27T02:37:07.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.926+0000 7f6e168e5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e100ad0a0 con 0x7f6e10063c90 2024-01-27T02:37:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.926+0000 7f6e0f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e10143520 0x7f6e100ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.926+0000 7f6e0f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e10143520 0x7f6e100ca5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37556/0 (socket says 172.21.15.7:37556) 2024-01-27T02:37:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.926+0000 7f6e0f7fe700 1 -- 172.21.15.7:0/2952192905 learned_addr learned my addr 172.21.15.7:0/2952192905 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.926+0000 7f6e14e82700 1 --2- 172.21.15.7:0/2952192905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e101438f0 0x7f6e100cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.926+0000 7f6e0ffff700 1 --2- 172.21.15.7:0/2952192905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e10063c90 0x7f6e101408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.926+0000 7f6e0f7fe700 1 -- 172.21.15.7:0/2952192905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e10063c90 msgr2=0x7f6e101408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.926+0000 7f6e0f7fe700 1 --2- 172.21.15.7:0/2952192905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e10063c90 0x7f6e101408c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.926+0000 7f6e0f7fe700 1 -- 172.21.15.7:0/2952192905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e101438f0 msgr2=0x7f6e100cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.926+0000 7f6e0f7fe700 1 --2- 172.21.15.7:0/2952192905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e101438f0 0x7f6e100cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.926+0000 7f6e0f7fe700 1 -- 172.21.15.7:0/2952192905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e08009a20 con 0x7f6e10143520 2024-01-27T02:37:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.926+0000 7f6e14e82700 1 --2- 172.21.15.7:0/2952192905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e101438f0 0x7f6e100cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:37:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.926+0000 7f6e0f7fe700 1 --2- 172.21.15.7:0/2952192905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e10143520 0x7f6e100ca5d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000c5e0 tx=0x7f6e00007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.926+0000 7f6e07fff700 1 -- 172.21.15.7:0/2952192905 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e00005c00 con 0x7f6e10143520 2024-01-27T02:37:07.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.927+0000 7f6e07fff700 1 -- 172.21.15.7:0/2952192905 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e00005d60 con 0x7f6e10143520 2024-01-27T02:37:07.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.927+0000 7f6e07fff700 1 -- 172.21.15.7:0/2952192905 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e00007300 con 0x7f6e10143520 2024-01-27T02:37:07.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.927+0000 7f6e168e5700 1 -- 172.21.15.7:0/2952192905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e100ad380 con 0x7f6e10143520 2024-01-27T02:37:07.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.927+0000 7f6e168e5700 1 -- 172.21.15.7:0/2952192905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e10140e00 con 0x7f6e10143520 2024-01-27T02:37:07.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.928+0000 7f6e168e5700 1 -- 172.21.15.7:0/2952192905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e1004c710 con 0x7f6e10143520 2024-01-27T02:37:07.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.928+0000 7f6e07fff700 1 -- 172.21.15.7:0/2952192905 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6e0001b3b0 con 0x7f6e10143520 2024-01-27T02:37:07.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.929+0000 7f6e07fff700 1 --2- 172.21.15.7:0/2952192905 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6df0066f30 0x7f6df00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:07.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.929+0000 7f6e07fff700 1 -- 172.21.15.7:0/2952192905 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6e00083a40 con 0x7f6e10143520 2024-01-27T02:37:07.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.929+0000 7f6e0ffff700 1 --2- 172.21.15.7:0/2952192905 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6df0066f30 0x7f6df00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:07.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.930+0000 7f6e0ffff700 1 --2- 172.21.15.7:0/2952192905 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6df0066f30 0x7f6df00693e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6df8007fd0 tx=0x7f6df8007eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:07.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:07.933+0000 7f6e07fff700 1 -- 172.21.15.7:0/2952192905 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e00085e60 con 0x7f6e10143520 2024-01-27T02:37:08.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.132+0000 7f6e168e5700 1 -- 172.21.15.7:0/2952192905 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6e100cc290 con 0x7f6df0066f30 2024-01-27T02:37:08.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.141+0000 7f6e07fff700 1 -- 172.21.15.7:0/2952192905 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285449 (secure 0 0 0) 0x7f6e100cc290 con 0x7f6df0066f30 2024-01-27T02:37:08.145 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:08.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.144+0000 7f6e168e5700 1 -- 172.21.15.7:0/2952192905 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6df0066f30 msgr2=0x7f6df00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:08.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.144+0000 7f6e168e5700 1 --2- 172.21.15.7:0/2952192905 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6df0066f30 0x7f6df00693e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6df8007fd0 tx=0x7f6df8007eb0 comp rx=0 tx=0).stop 2024-01-27T02:37:08.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.144+0000 7f6e168e5700 1 -- 172.21.15.7:0/2952192905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e10143520 msgr2=0x7f6e100ca5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:08.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.144+0000 7f6e168e5700 1 --2- 172.21.15.7:0/2952192905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e10143520 0x7f6e100ca5d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000c5e0 tx=0x7f6e00007d50 comp rx=0 tx=0).stop 2024-01-27T02:37:08.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.144+0000 7f6e168e5700 1 -- 172.21.15.7:0/2952192905 shutdown_connections 2024-01-27T02:37:08.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.144+0000 7f6e168e5700 1 --2- 172.21.15.7:0/2952192905 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6df0066f30 0x7f6df00693e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:08.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.144+0000 7f6e168e5700 1 --2- 172.21.15.7:0/2952192905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e101438f0 0x7f6e100cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:08.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.144+0000 7f6e168e5700 1 --2- 172.21.15.7:0/2952192905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e10063c90 0x7f6e101408c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:08.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.145+0000 7f6e168e5700 1 --2- 172.21.15.7:0/2952192905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e10143520 0x7f6e100ca5d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:08.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.145+0000 7f6e168e5700 1 -- 172.21.15.7:0/2952192905 >> 172.21.15.7:0/2952192905 conn(0x7f6e100cdbd0 msgr2=0x7f6e100aa930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:08.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.145+0000 7f6e168e5700 1 -- 172.21.15.7:0/2952192905 shutdown_connections 2024-01-27T02:37:08.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.145+0000 7f6e168e5700 1 -- 172.21.15.7:0/2952192905 wait complete. 2024-01-27T02:37:08.149 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:37:08.159 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-01-27T02:37:07.646453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5257,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557242,"statfs":{"total":1600000000,"available":1594616220,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.683138"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.981928+0000","last_change":"2024-01-27T02:37:03.981539+0000","last_active":"2024-01-27T02:37:03.981928+0000","last_peered":"2024-01-27T02:37:03.981928+0000","last_clean":"2024-01-27T02:37:03.981928+0000","last_became_active":"2024-01-27T02:37:03.981297+0000","last_became_peered":"2024-01-27T02:37:03.981297+0000","last_unstale":"2024-01-27T02:37:03.981928+0000","last_undegraded":"2024-01-27T02:37:03.981928+0000","last_fullsized":"2024-01-27T02:37:03.981928+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:00:23.179794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.974722+0000","last_change":"2024-01-27T02:37:03.974633+0000","last_active":"2024-01-27T02:37:03.974722+0000","last_peered":"2024-01-27T02:37:03.974722+0000","last_clean":"2024-01-27T02:37:03.974722+0000","last_became_active":"2024-01-27T02:37:03.974323+0000","last_became_peered":"2024-01-27T02:37:03.974323+0000","last_unstale":"2024-01-27T02:37:03.974722+0000","last_undegraded":"2024-01-27T02:37:03.974722+0000","last_fullsized":"2024-01-27T02:37:03.974722+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:48:17.966842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.984565+0000","last_change":"2024-01-27T02:37:03.984493+0000","last_active":"2024-01-27T02:37:03.984565+0000","last_peered":"2024-01-27T02:37:03.984565+0000","last_clean":"2024-01-27T02:37:03.984565+0000","last_became_active":"2024-01-27T02:37:03.984227+0000","last_became_peered":"2024-01-27T02:37:03.984227+0000","last_unstale":"2024-01-27T02:37:03.984565+0000","last_undegraded":"2024-01-27T02:37:03.984565+0000","last_fullsized":"2024-01-27T02:37:03.984565+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:13.752690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.989618+0000","last_change":"2024-01-27T02:37:03.989503+0000","last_active":"2024-01-27T02:37:03.989618+0000","last_peered":"2024-01-27T02:37:03.989618+0000","last_clean":"2024-01-27T02:37:03.989618+0000","last_became_active":"2024-01-27T02:37:03.989059+0000","last_became_peered":"2024-01-27T02:37:03.989059+0000","last_unstale":"2024-01-27T02:37:03.989618+0000","last_undegraded":"2024-01-27T02:37:03.989618+0000","last_fullsized":"2024-01-27T02:37:03.989618+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:40:17.306454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.978681+0000","last_change":"2024-01-27T02:37:03.978549+0000","last_active":"2024-01-27T02:37:03.978681+0000","last_peered":"2024-01-27T02:37:03.978681+0000","last_clean":"2024-01-27T02:37:03.978681+0000","last_became_active":"2024-01-27T02:37:03.978158+0000","last_became_peered":"2024-01-27T02:37:03.978158+0000","last_unstale":"2024-01-27T02:37:03.978681+0000","last_undegraded":"2024-01-27T02:37:03.978681+0000","last_fullsized":"2024-01-27T02:37:03.978681+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:51:37.252954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.983434+0000","last_change":"2024-01-27T02:37:03.983239+0000","last_active":"2024-01-27T02:37:03.983434+0000","last_peered":"2024-01-27T02:37:03.983434+0000","last_clean":"2024-01-27T02:37:03.983434+0000","last_became_active":"2024-01-27T02:37:03.982749+0000","last_became_peered":"2024-01-27T02:37:03.982749+0000","last_unstale":"2024-01-27T02:37:03.983434+0000","last_undegraded":"2024-01-27T02:37:03.983434+0000","last_fullsized":"2024-01-27T02:37:03.983434+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:05:34.176684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.981290+0000","last_change":"2024-01-27T02:37:03.979871+0000","last_active":"2024-01-27T02:37:03.981290+0000","last_peered":"2024-01-27T02:37:03.981290+0000","last_clean":"2024-01-27T02:37:03.981290+0000","last_became_active":"2024-01-27T02:37:03.979564+0000","last_became_peered":"2024-01-27T02:37:03.979564+0000","last_unstale":"2024-01-27T02:37:03.981290+0000","last_undegraded":"2024-01-27T02:37:03.981290+0000","last_fullsized":"2024-01-27T02:37:03.981290+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:26:53.954759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.990787+0000","last_change":"2024-01-27T02:37:03.990696+0000","last_active":"2024-01-27T02:37:03.990787+0000","last_peered":"2024-01-27T02:37:03.990787+0000","last_clean":"2024-01-27T02:37:03.990787+0000","last_became_active":"2024-01-27T02:37:03.990447+0000","last_became_peered":"2024-01-27T02:37:03.990447+0000","last_unstale":"2024-01-27T02:37:03.990787+0000","last_undegraded":"2024-01-27T02:37:03.990787+0000","last_fullsized":"2024-01-27T02:37:03.990787+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:54:16.825640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.984123+0000","last_change":"2024-01-27T02:37:03.984059+0000","last_active":"2024-01-27T02:37:03.984123+0000","last_peered":"2024-01-27T02:37:03.984123+0000","last_clean":"2024-01-27T02:37:03.984123+0000","last_became_active":"2024-01-27T02:37:03.983752+0000","last_became_peered":"2024-01-27T02:37:03.983752+0000","last_unstale":"2024-01-27T02:37:03.984123+0000","last_undegraded":"2024-01-27T02:37:03.984123+0000","last_fullsized":"2024-01-27T02:37:03.984123+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:03:33.502042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.976768+0000","last_change":"2024-01-27T02:37:03.975609+0000","last_active":"2024-01-27T02:37:03.976768+0000","last_peered":"2024-01-27T02:37:03.976768+0000","last_clean":"2024-01-27T02:37:03.976768+0000","last_became_active":"2024-01-27T02:37:03.975279+0000","last_became_peered":"2024-01-27T02:37:03.975279+0000","last_unstale":"2024-01-27T02:37:03.976768+0000","last_undegraded":"2024-01-27T02:37:03.976768+0000","last_fullsized":"2024-01-27T02:37:03.976768+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:51:37.252954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.977507+0000","last_change":"2024-01-27T02:37:03.977434+0000","last_active":"2024-01-27T02:37:03.977507+0000","last_peered":"2024-01-27T02:37:03.977507+0000","last_clean":"2024-01-27T02:37:03.977507+0000","last_became_active":"2024-01-27T02:37:03.977131+0000","last_became_peered":"2024-01-27T02:37:03.977131+0000","last_unstale":"2024-01-27T02:37:03.977507+0000","last_undegraded":"2024-01-27T02:37:03.977507+0000","last_fullsized":"2024-01-27T02:37:03.977507+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:14:22.604751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.980680+0000","last_change":"2024-01-27T02:37:03.976569+0000","last_active":"2024-01-27T02:37:03.980680+0000","last_peered":"2024-01-27T02:37:03.980680+0000","last_clean":"2024-01-27T02:37:03.980680+0000","last_became_active":"2024-01-27T02:37:03.975238+0000","last_became_peered":"2024-01-27T02:37:03.975238+0000","last_unstale":"2024-01-27T02:37:03.980680+0000","last_undegraded":"2024-01-27T02:37:03.980680+0000","last_fullsized":"2024-01-27T02:37:03.980680+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:20:09.835171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.979516+0000","last_change":"2024-01-27T02:37:03.979282+0000","last_active":"2024-01-27T02:37:03.979516+0000","last_peered":"2024-01-27T02:37:03.979516+0000","last_clean":"2024-01-27T02:37:03.979516+0000","last_became_active":"2024-01-27T02:37:03.978995+0000","last_became_peered":"2024-01-27T02:37:03.978995+0000","last_unstale":"2024-01-27T02:37:03.979516+0000","last_undegraded":"2024-01-27T02:37:03.979516+0000","last_fullsized":"2024-01-27T02:37:03.979516+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:03:31.433351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.989594+0000","last_change":"2024-01-27T02:37:03.989339+0000","last_active":"2024-01-27T02:37:03.989594+0000","last_peered":"2024-01-27T02:37:03.989594+0000","last_clean":"2024-01-27T02:37:03.989594+0000","last_became_active":"2024-01-27T02:37:03.988740+0000","last_became_peered":"2024-01-27T02:37:03.988740+0000","last_unstale":"2024-01-27T02:37:03.989594+0000","last_undegraded":"2024-01-27T02:37:03.989594+0000","last_fullsized":"2024-01-27T02:37:03.989594+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:08:53.127784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.982593+0000","last_change":"2024-01-27T02:37:03.979903+0000","last_active":"2024-01-27T02:37:03.982593+0000","last_peered":"2024-01-27T02:37:03.982593+0000","last_clean":"2024-01-27T02:37:03.982593+0000","last_became_active":"2024-01-27T02:37:03.979612+0000","last_became_peered":"2024-01-27T02:37:03.979612+0000","last_unstale":"2024-01-27T02:37:03.982593+0000","last_undegraded":"2024-01-27T02:37:03.982593+0000","last_fullsized":"2024-01-27T02:37:03.982593+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:13:12.836181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.984596+0000","last_change":"2024-01-27T02:37:03.984442+0000","last_active":"2024-01-27T02:37:03.984596+0000","last_peered":"2024-01-27T02:37:03.984596+0000","last_clean":"2024-01-27T02:37:03.984596+0000","last_became_active":"2024-01-27T02:37:03.984162+0000","last_became_peered":"2024-01-27T02:37:03.984162+0000","last_unstale":"2024-01-27T02:37:03.984596+0000","last_undegraded":"2024-01-27T02:37:03.984596+0000","last_fullsized":"2024-01-27T02:37:03.984596+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:03:33.502042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.983275+0000","last_change":"2024-01-27T02:37:03.983201+0000","last_active":"2024-01-27T02:37:03.983275+0000","last_peered":"2024-01-27T02:37:03.983275+0000","last_clean":"2024-01-27T02:37:03.983275+0000","last_became_active":"2024-01-27T02:37:03.982475+0000","last_became_peered":"2024-01-27T02:37:03.982475+0000","last_unstale":"2024-01-27T02:37:03.983275+0000","last_undegraded":"2024-01-27T02:37:03.983275+0000","last_fullsized":"2024-01-27T02:37:03.983275+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:20:09.835171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.982611+0000","last_change":"2024-01-27T02:37:03.982505+0000","last_active":"2024-01-27T02:37:03.982611+0000","last_peered":"2024-01-27T02:37:03.982611+0000","last_clean":"2024-01-27T02:37:03.982611+0000","last_became_active":"2024-01-27T02:37:03.982219+0000","last_became_peered":"2024-01-27T02:37:03.982219+0000","last_unstale":"2024-01-27T02:37:03.982611+0000","last_undegraded":"2024-01-27T02:37:03.982611+0000","last_fullsized":"2024-01-27T02:37:03.982611+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:51:20.295257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.991328+0000","last_change":"2024-01-27T02:37:03.991016+0000","last_active":"2024-01-27T02:37:03.991328+0000","last_peered":"2024-01-27T02:37:03.991328+0000","last_clean":"2024-01-27T02:37:03.991328+0000","last_became_active":"2024-01-27T02:37:03.990747+0000","last_became_peered":"2024-01-27T02:37:03.990747+0000","last_unstale":"2024-01-27T02:37:03.991328+0000","last_undegraded":"2024-01-27T02:37:03.991328+0000","last_fullsized":"2024-01-27T02:37:03.991328+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:20:43.448875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.988281+0000","last_change":"2024-01-27T02:37:03.988205+0000","last_active":"2024-01-27T02:37:03.988281+0000","last_peered":"2024-01-27T02:37:03.988281+0000","last_clean":"2024-01-27T02:37:03.988281+0000","last_became_active":"2024-01-27T02:37:03.987971+0000","last_became_peered":"2024-01-27T02:37:03.987971+0000","last_unstale":"2024-01-27T02:37:03.988281+0000","last_undegraded":"2024-01-27T02:37:03.988281+0000","last_fullsized":"2024-01-27T02:37:03.988281+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:37:43.105081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.994255+0000","last_change":"2024-01-27T02:37:03.991935+0000","last_active":"2024-01-27T02:37:03.994255+0000","last_peered":"2024-01-27T02:37:03.994255+0000","last_clean":"2024-01-27T02:37:03.994255+0000","last_became_active":"2024-01-27T02:37:03.989031+0000","last_became_peered":"2024-01-27T02:37:03.989031+0000","last_unstale":"2024-01-27T02:37:03.994255+0000","last_undegraded":"2024-01-27T02:37:03.994255+0000","last_fullsized":"2024-01-27T02:37:03.994255+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:30:14.882894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.989660+0000","last_change":"2024-01-27T02:37:03.989298+0000","last_active":"2024-01-27T02:37:03.989660+0000","last_peered":"2024-01-27T02:37:03.989660+0000","last_clean":"2024-01-27T02:37:03.989660+0000","last_became_active":"2024-01-27T02:37:03.988975+0000","last_became_peered":"2024-01-27T02:37:03.988975+0000","last_unstale":"2024-01-27T02:37:03.989660+0000","last_undegraded":"2024-01-27T02:37:03.989660+0000","last_fullsized":"2024-01-27T02:37:03.989660+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:06:57.129035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.986597+0000","last_change":"2024-01-27T02:37:03.986558+0000","last_active":"2024-01-27T02:37:03.986597+0000","last_peered":"2024-01-27T02:37:03.986597+0000","last_clean":"2024-01-27T02:37:03.986597+0000","last_became_active":"2024-01-27T02:37:03.986189+0000","last_became_peered":"2024-01-27T02:37:03.986189+0000","last_unstale":"2024-01-27T02:37:03.986597+0000","last_undegraded":"2024-01-27T02:37:03.986597+0000","last_fullsized":"2024-01-27T02:37:03.986597+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:20:43.448875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.985981+0000","last_change":"2024-01-27T02:37:03.985904+0000","last_active":"2024-01-27T02:37:03.985981+0000","last_peered":"2024-01-27T02:37:03.985981+0000","last_clean":"2024-01-27T02:37:03.985981+0000","last_became_active":"2024-01-27T02:37:03.985625+0000","last_became_peered":"2024-01-27T02:37:03.985625+0000","last_unstale":"2024-01-27T02:37:03.985981+0000","last_undegraded":"2024-01-27T02:37:03.985981+0000","last_fullsized":"2024-01-27T02:37:03.985981+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:36:37.195219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.973326+0000","last_change":"2024-01-27T02:37:03.973241+0000","last_active":"2024-01-27T02:37:03.973326+0000","last_peered":"2024-01-27T02:37:03.973326+0000","last_clean":"2024-01-27T02:37:03.973326+0000","last_became_active":"2024-01-27T02:37:03.972965+0000","last_became_peered":"2024-01-27T02:37:03.972965+0000","last_unstale":"2024-01-27T02:37:03.973326+0000","last_undegraded":"2024-01-27T02:37:03.973326+0000","last_fullsized":"2024-01-27T02:37:03.973326+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:39:32.384063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.979943+0000","last_change":"2024-01-27T02:37:03.979831+0000","last_active":"2024-01-27T02:37:03.979943+0000","last_peered":"2024-01-27T02:37:03.979943+0000","last_clean":"2024-01-27T02:37:03.979943+0000","last_became_active":"2024-01-27T02:37:03.979515+0000","last_became_peered":"2024-01-27T02:37:03.979515+0000","last_unstale":"2024-01-27T02:37:03.979943+0000","last_undegraded":"2024-01-27T02:37:03.979943+0000","last_fullsized":"2024-01-27T02:37:03.979943+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:51:37.252954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.983501+0000","last_change":"2024-01-27T02:37:03.983376+0000","last_active":"2024-01-27T02:37:03.983501+0000","last_peered":"2024-01-27T02:37:03.983501+0000","last_clean":"2024-01-27T02:37:03.983501+0000","last_became_active":"2024-01-27T02:37:03.982797+0000","last_became_peered":"2024-01-27T02:37:03.982797+0000","last_unstale":"2024-01-27T02:37:03.983501+0000","last_undegraded":"2024-01-27T02:37:03.983501+0000","last_fullsized":"2024-01-27T02:37:03.983501+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:00:23.179794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.981947+0000","last_change":"2024-01-27T02:37:03.981665+0000","last_active":"2024-01-27T02:37:03.981947+0000","last_peered":"2024-01-27T02:37:03.981947+0000","last_clean":"2024-01-27T02:37:03.981947+0000","last_became_active":"2024-01-27T02:37:03.980985+0000","last_became_peered":"2024-01-27T02:37:03.980985+0000","last_unstale":"2024-01-27T02:37:03.981947+0000","last_undegraded":"2024-01-27T02:37:03.981947+0000","last_fullsized":"2024-01-27T02:37:03.981947+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:03:33.502042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.973890+0000","last_change":"2024-01-27T02:37:03.973788+0000","last_active":"2024-01-27T02:37:03.973890+0000","last_peered":"2024-01-27T02:37:03.973890+0000","last_clean":"2024-01-27T02:37:03.973890+0000","last_became_active":"2024-01-27T02:37:03.973469+0000","last_became_peered":"2024-01-27T02:37:03.973469+0000","last_unstale":"2024-01-27T02:37:03.973890+0000","last_undegraded":"2024-01-27T02:37:03.973890+0000","last_fullsized":"2024-01-27T02:37:03.973890+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:37:36.575409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.988812+0000","last_change":"2024-01-27T02:37:03.988684+0000","last_active":"2024-01-27T02:37:03.988812+0000","last_peered":"2024-01-27T02:37:03.988812+0000","last_clean":"2024-01-27T02:37:03.988812+0000","last_became_active":"2024-01-27T02:37:03.988390+0000","last_became_peered":"2024-01-27T02:37:03.988390+0000","last_unstale":"2024-01-27T02:37:03.988812+0000","last_undegraded":"2024-01-27T02:37:03.988812+0000","last_fullsized":"2024-01-27T02:37:03.988812+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:48:58.110238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.990077+0000","last_change":"2024-01-27T02:37:03.990012+0000","last_active":"2024-01-27T02:37:03.990077+0000","last_peered":"2024-01-27T02:37:03.990077+0000","last_clean":"2024-01-27T02:37:03.990077+0000","last_became_active":"2024-01-27T02:37:03.989541+0000","last_became_peered":"2024-01-27T02:37:03.989541+0000","last_unstale":"2024-01-27T02:37:03.990077+0000","last_undegraded":"2024-01-27T02:37:03.990077+0000","last_fullsized":"2024-01-27T02:37:03.990077+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:33:34.067913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.980704+0000","last_change":"2024-01-27T02:37:03.977028+0000","last_active":"2024-01-27T02:37:03.980704+0000","last_peered":"2024-01-27T02:37:03.980704+0000","last_clean":"2024-01-27T02:37:03.980704+0000","last_became_active":"2024-01-27T02:37:03.976168+0000","last_became_peered":"2024-01-27T02:37:03.976168+0000","last_unstale":"2024-01-27T02:37:03.980704+0000","last_undegraded":"2024-01-27T02:37:03.980704+0000","last_fullsized":"2024-01-27T02:37:03.980704+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:49:53.022320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.976382+0000","last_change":"2024-01-27T02:37:03.976297+0000","last_active":"2024-01-27T02:37:03.976382+0000","last_peered":"2024-01-27T02:37:03.976382+0000","last_clean":"2024-01-27T02:37:03.976382+0000","last_became_active":"2024-01-27T02:37:03.976061+0000","last_became_peered":"2024-01-27T02:37:03.976061+0000","last_unstale":"2024-01-27T02:37:03.976382+0000","last_undegraded":"2024-01-27T02:37:03.976382+0000","last_fullsized":"2024-01-27T02:37:03.976382+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:54:16.825640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"97'4","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.995049+0000","last_change":"2024-01-27T02:37:02.985948+0000","last_active":"2024-01-27T02:37:03.995049+0000","last_peered":"2024-01-27T02:37:03.995049+0000","last_clean":"2024-01-27T02:37:03.995049+0000","last_became_active":"2024-01-27T02:37:02.985748+0000","last_became_peered":"2024-01-27T02:37:02.985748+0000","last_unstale":"2024-01-27T02:37:03.995049+0000","last_undegraded":"2024-01-27T02:37:03.995049+0000","last_fullsized":"2024-01-27T02:37:03.995049+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:46:16.101944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.986503+0000","last_change":"2024-01-27T02:37:03.986440+0000","last_active":"2024-01-27T02:37:03.986503+0000","last_peered":"2024-01-27T02:37:03.986503+0000","last_clean":"2024-01-27T02:37:03.986503+0000","last_became_active":"2024-01-27T02:37:03.985869+0000","last_became_peered":"2024-01-27T02:37:03.985869+0000","last_unstale":"2024-01-27T02:37:03.986503+0000","last_undegraded":"2024-01-27T02:37:03.986503+0000","last_fullsized":"2024-01-27T02:37:03.986503+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:05:34.176684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"97'1","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.993544+0000","last_change":"2024-01-27T02:37:02.981569+0000","last_active":"2024-01-27T02:37:03.993544+0000","last_peered":"2024-01-27T02:37:03.993544+0000","last_clean":"2024-01-27T02:37:03.993544+0000","last_became_active":"2024-01-27T02:37:02.981221+0000","last_became_peered":"2024-01-27T02:37:02.981221+0000","last_unstale":"2024-01-27T02:37:03.993544+0000","last_undegraded":"2024-01-27T02:37:03.993544+0000","last_fullsized":"2024-01-27T02:37:03.993544+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:27:26.563071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.975382+0000","last_change":"2024-01-27T02:37:03.975299+0000","last_active":"2024-01-27T02:37:03.975382+0000","last_peered":"2024-01-27T02:37:03.975382+0000","last_clean":"2024-01-27T02:37:03.975382+0000","last_became_active":"2024-01-27T02:37:03.975030+0000","last_became_peered":"2024-01-27T02:37:03.975030+0000","last_unstale":"2024-01-27T02:37:03.975382+0000","last_undegraded":"2024-01-27T02:37:03.975382+0000","last_fullsized":"2024-01-27T02:37:03.975382+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:09:36.161409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"97'2","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.996157+0000","last_change":"2024-01-27T02:37:02.986148+0000","last_active":"2024-01-27T02:37:03.996157+0000","last_peered":"2024-01-27T02:37:03.996157+0000","last_clean":"2024-01-27T02:37:03.996157+0000","last_became_active":"2024-01-27T02:37:02.985884+0000","last_became_peered":"2024-01-27T02:37:02.985884+0000","last_unstale":"2024-01-27T02:37:03.996157+0000","last_undegraded":"2024-01-27T02:37:03.996157+0000","last_fullsized":"2024-01-27T02:37:03.996157+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:23:02.531377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.982707+0000","last_change":"2024-01-27T02:37:03.982229+0000","last_active":"2024-01-27T02:37:03.982707+0000","last_peered":"2024-01-27T02:37:03.982707+0000","last_clean":"2024-01-27T02:37:03.982707+0000","last_became_active":"2024-01-27T02:37:03.981800+0000","last_became_peered":"2024-01-27T02:37:03.981800+0000","last_unstale":"2024-01-27T02:37:03.982707+0000","last_undegraded":"2024-01-27T02:37:03.982707+0000","last_fullsized":"2024-01-27T02:37:03.982707+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:55:29.044564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"97'2","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.995362+0000","last_change":"2024-01-27T02:37:02.986529+0000","last_active":"2024-01-27T02:37:03.995362+0000","last_peered":"2024-01-27T02:37:03.995362+0000","last_clean":"2024-01-27T02:37:03.995362+0000","last_became_active":"2024-01-27T02:37:02.985853+0000","last_became_peered":"2024-01-27T02:37:02.985853+0000","last_unstale":"2024-01-27T02:37:03.995362+0000","last_undegraded":"2024-01-27T02:37:03.995362+0000","last_fullsized":"2024-01-27T02:37:03.995362+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:04:34.145511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.990511+0000","last_change":"2024-01-27T02:37:03.990250+0000","last_active":"2024-01-27T02:37:03.990511+0000","last_peered":"2024-01-27T02:37:03.990511+0000","last_clean":"2024-01-27T02:37:03.990511+0000","last_became_active":"2024-01-27T02:37:03.989804+0000","last_became_peered":"2024-01-27T02:37:03.989804+0000","last_unstale":"2024-01-27T02:37:03.990511+0000","last_undegraded":"2024-01-27T02:37:03.990511+0000","last_fullsized":"2024-01-27T02:37:03.990511+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:21:14.462607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"97'1","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.997194+0000","last_change":"2024-01-27T02:37:02.981692+0000","last_active":"2024-01-27T02:37:03.997194+0000","last_peered":"2024-01-27T02:37:03.997194+0000","last_clean":"2024-01-27T02:37:03.997194+0000","last_became_active":"2024-01-27T02:37:02.981398+0000","last_became_peered":"2024-01-27T02:37:02.981398+0000","last_unstale":"2024-01-27T02:37:03.997194+0000","last_undegraded":"2024-01-27T02:37:03.997194+0000","last_fullsized":"2024-01-27T02:37:03.997194+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:29:25.334034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.983137+0000","last_change":"2024-01-27T02:37:03.983024+0000","last_active":"2024-01-27T02:37:03.983137+0000","last_peered":"2024-01-27T02:37:03.983137+0000","last_clean":"2024-01-27T02:37:03.983137+0000","last_became_active":"2024-01-27T02:37:03.982163+0000","last_became_peered":"2024-01-27T02:37:03.982163+0000","last_unstale":"2024-01-27T02:37:03.983137+0000","last_undegraded":"2024-01-27T02:37:03.983137+0000","last_fullsized":"2024-01-27T02:37:03.983137+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:09:36.161409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"97'1","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.991439+0000","last_change":"2024-01-27T02:37:02.980720+0000","last_active":"2024-01-27T02:37:03.991439+0000","last_peered":"2024-01-27T02:37:03.991439+0000","last_clean":"2024-01-27T02:37:03.991439+0000","last_became_active":"2024-01-27T02:37:02.980470+0000","last_became_peered":"2024-01-27T02:37:02.980470+0000","last_unstale":"2024-01-27T02:37:03.991439+0000","last_undegraded":"2024-01-27T02:37:03.991439+0000","last_fullsized":"2024-01-27T02:37:03.991439+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:39:31.378671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.990132+0000","last_change":"2024-01-27T02:37:03.990050+0000","last_active":"2024-01-27T02:37:03.990132+0000","last_peered":"2024-01-27T02:37:03.990132+0000","last_clean":"2024-01-27T02:37:03.990132+0000","last_became_active":"2024-01-27T02:37:03.989619+0000","last_became_peered":"2024-01-27T02:37:03.989619+0000","last_unstale":"2024-01-27T02:37:03.990132+0000","last_undegraded":"2024-01-27T02:37:03.990132+0000","last_fullsized":"2024-01-27T02:37:03.990132+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:03:58.690531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.981086+0000","last_change":"2024-01-27T02:37:02.983675+0000","last_active":"2024-01-27T02:37:03.981086+0000","last_peered":"2024-01-27T02:37:03.981086+0000","last_clean":"2024-01-27T02:37:03.981086+0000","last_became_active":"2024-01-27T02:37:02.982951+0000","last_became_peered":"2024-01-27T02:37:02.982951+0000","last_unstale":"2024-01-27T02:37:03.981086+0000","last_undegraded":"2024-01-27T02:37:03.981086+0000","last_fullsized":"2024-01-27T02:37:03.981086+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:04.275415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.976196+0000","last_change":"2024-01-27T02:37:03.976131+0000","last_active":"2024-01-27T02:37:03.976196+0000","last_peered":"2024-01-27T02:37:03.976196+0000","last_clean":"2024-01-27T02:37:03.976196+0000","last_became_active":"2024-01-27T02:37:03.975923+0000","last_became_peered":"2024-01-27T02:37:03.975923+0000","last_unstale":"2024-01-27T02:37:03.976196+0000","last_undegraded":"2024-01-27T02:37:03.976196+0000","last_fullsized":"2024-01-27T02:37:03.976196+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:40:38.576965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.971423+0000","last_change":"2024-01-27T02:37:02.979435+0000","last_active":"2024-01-27T02:37:03.971423+0000","last_peered":"2024-01-27T02:37:03.971423+0000","last_clean":"2024-01-27T02:37:03.971423+0000","last_became_active":"2024-01-27T02:37:02.978795+0000","last_became_peered":"2024-01-27T02:37:02.978795+0000","last_unstale":"2024-01-27T02:37:03.971423+0000","last_undegraded":"2024-01-27T02:37:03.971423+0000","last_fullsized":"2024-01-27T02:37:03.971423+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:50:27.998647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.982575+0000","last_change":"2024-01-27T02:37:03.978063+0000","last_active":"2024-01-27T02:37:03.982575+0000","last_peered":"2024-01-27T02:37:03.982575+0000","last_clean":"2024-01-27T02:37:03.982575+0000","last_became_active":"2024-01-27T02:37:03.977541+0000","last_became_peered":"2024-01-27T02:37:03.977541+0000","last_unstale":"2024-01-27T02:37:03.982575+0000","last_undegraded":"2024-01-27T02:37:03.982575+0000","last_fullsized":"2024-01-27T02:37:03.982575+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:25:59.851526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.972208+0000","last_change":"2024-01-27T02:37:02.972626+0000","last_active":"2024-01-27T02:37:03.972208+0000","last_peered":"2024-01-27T02:37:03.972208+0000","last_clean":"2024-01-27T02:37:03.972208+0000","last_became_active":"2024-01-27T02:37:02.972346+0000","last_became_peered":"2024-01-27T02:37:02.972346+0000","last_unstale":"2024-01-27T02:37:03.972208+0000","last_undegraded":"2024-01-27T02:37:03.972208+0000","last_fullsized":"2024-01-27T02:37:03.972208+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:48:57.104846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.983155+0000","last_change":"2024-01-27T02:37:03.983086+0000","last_active":"2024-01-27T02:37:03.983155+0000","last_peered":"2024-01-27T02:37:03.983155+0000","last_clean":"2024-01-27T02:37:03.983155+0000","last_became_active":"2024-01-27T02:37:03.982440+0000","last_became_peered":"2024-01-27T02:37:03.982440+0000","last_unstale":"2024-01-27T02:37:03.983155+0000","last_undegraded":"2024-01-27T02:37:03.983155+0000","last_fullsized":"2024-01-27T02:37:03.983155+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:51:37.252954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.971380+0000","last_change":"2024-01-27T02:37:02.977998+0000","last_active":"2024-01-27T02:37:03.971380+0000","last_peered":"2024-01-27T02:37:03.971380+0000","last_clean":"2024-01-27T02:37:03.971380+0000","last_became_active":"2024-01-27T02:37:02.977510+0000","last_became_peered":"2024-01-27T02:37:02.977510+0000","last_unstale":"2024-01-27T02:37:03.971380+0000","last_undegraded":"2024-01-27T02:37:03.971380+0000","last_fullsized":"2024-01-27T02:37:03.971380+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:06:56.123644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.985252+0000","last_change":"2024-01-27T02:37:03.985066+0000","last_active":"2024-01-27T02:37:03.985252+0000","last_peered":"2024-01-27T02:37:03.985252+0000","last_clean":"2024-01-27T02:37:03.985252+0000","last_became_active":"2024-01-27T02:37:03.984385+0000","last_became_peered":"2024-01-27T02:37:03.984385+0000","last_unstale":"2024-01-27T02:37:03.985252+0000","last_undegraded":"2024-01-27T02:37:03.985252+0000","last_fullsized":"2024-01-27T02:37:03.985252+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:05:34.176684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"97'3","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.997149+0000","last_change":"2024-01-27T02:37:02.981563+0000","last_active":"2024-01-27T02:37:03.997149+0000","last_peered":"2024-01-27T02:37:03.997149+0000","last_clean":"2024-01-27T02:37:03.997149+0000","last_became_active":"2024-01-27T02:37:02.981298+0000","last_became_peered":"2024-01-27T02:37:02.981298+0000","last_unstale":"2024-01-27T02:37:03.997149+0000","last_undegraded":"2024-01-27T02:37:03.997149+0000","last_fullsized":"2024-01-27T02:37:03.997149+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:48:16.961451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.989027+0000","last_change":"2024-01-27T02:37:03.988969+0000","last_active":"2024-01-27T02:37:03.989027+0000","last_peered":"2024-01-27T02:37:03.989027+0000","last_clean":"2024-01-27T02:37:03.989027+0000","last_became_active":"2024-01-27T02:37:03.988696+0000","last_became_peered":"2024-01-27T02:37:03.988696+0000","last_unstale":"2024-01-27T02:37:03.989027+0000","last_undegraded":"2024-01-27T02:37:03.989027+0000","last_fullsized":"2024-01-27T02:37:03.989027+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:57:41.192776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"97'6","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.998459+0000","last_change":"2024-01-27T02:37:02.981337+0000","last_active":"2024-01-27T02:37:03.998459+0000","last_peered":"2024-01-27T02:37:03.998459+0000","last_clean":"2024-01-27T02:37:03.998459+0000","last_became_active":"2024-01-27T02:37:02.981101+0000","last_became_peered":"2024-01-27T02:37:02.981101+0000","last_unstale":"2024-01-27T02:37:03.998459+0000","last_undegraded":"2024-01-27T02:37:03.998459+0000","last_fullsized":"2024-01-27T02:37:03.998459+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:35:02.399898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.986735+0000","last_change":"2024-01-27T02:37:03.986628+0000","last_active":"2024-01-27T02:37:03.986735+0000","last_peered":"2024-01-27T02:37:03.986735+0000","last_clean":"2024-01-27T02:37:03.986735+0000","last_became_active":"2024-01-27T02:37:03.986106+0000","last_became_peered":"2024-01-27T02:37:03.986106+0000","last_unstale":"2024-01-27T02:37:03.986735+0000","last_undegraded":"2024-01-27T02:37:03.986735+0000","last_fullsized":"2024-01-27T02:37:03.986735+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:03:33.502042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"97'2","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.996625+0000","last_change":"2024-01-27T02:37:02.978230+0000","last_active":"2024-01-27T02:37:03.996625+0000","last_peered":"2024-01-27T02:37:03.996625+0000","last_clean":"2024-01-27T02:37:03.996625+0000","last_became_active":"2024-01-27T02:37:02.977954+0000","last_became_peered":"2024-01-27T02:37:02.977954+0000","last_unstale":"2024-01-27T02:37:03.996625+0000","last_undegraded":"2024-01-27T02:37:03.996625+0000","last_fullsized":"2024-01-27T02:37:03.996625+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:13:17.931874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.989987+0000","last_change":"2024-01-27T02:37:03.989900+0000","last_active":"2024-01-27T02:37:03.989987+0000","last_peered":"2024-01-27T02:37:03.989987+0000","last_clean":"2024-01-27T02:37:03.989987+0000","last_became_active":"2024-01-27T02:37:03.989225+0000","last_became_peered":"2024-01-27T02:37:03.989225+0000","last_unstale":"2024-01-27T02:37:03.989987+0000","last_undegraded":"2024-01-27T02:37:03.989987+0000","last_fullsized":"2024-01-27T02:37:03.989987+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:11:08.055611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.969981+0000","last_change":"2024-01-27T02:37:02.978205+0000","last_active":"2024-01-27T02:37:03.969981+0000","last_peered":"2024-01-27T02:37:03.969981+0000","last_clean":"2024-01-27T02:37:03.969981+0000","last_became_active":"2024-01-27T02:37:02.977960+0000","last_became_peered":"2024-01-27T02:37:02.977960+0000","last_unstale":"2024-01-27T02:37:03.969981+0000","last_undegraded":"2024-01-27T02:37:03.969981+0000","last_fullsized":"2024-01-27T02:37:03.969981+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:08:02.593649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.990489+0000","last_change":"2024-01-27T02:37:03.990080+0000","last_active":"2024-01-27T02:37:03.990489+0000","last_peered":"2024-01-27T02:37:03.990489+0000","last_clean":"2024-01-27T02:37:03.990489+0000","last_became_active":"2024-01-27T02:37:03.988096+0000","last_became_peered":"2024-01-27T02:37:03.988096+0000","last_unstale":"2024-01-27T02:37:03.990489+0000","last_undegraded":"2024-01-27T02:37:03.990489+0000","last_fullsized":"2024-01-27T02:37:03.990489+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:29:26.339425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.976556+0000","last_change":"2024-01-27T02:37:02.979846+0000","last_active":"2024-01-27T02:37:03.976556+0000","last_peered":"2024-01-27T02:37:03.976556+0000","last_clean":"2024-01-27T02:37:03.976556+0000","last_became_active":"2024-01-27T02:37:02.979575+0000","last_became_peered":"2024-01-27T02:37:02.979575+0000","last_unstale":"2024-01-27T02:37:03.976556+0000","last_undegraded":"2024-01-27T02:37:03.976556+0000","last_fullsized":"2024-01-27T02:37:03.976556+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:14:50.580128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.982653+0000","last_change":"2024-01-27T02:37:03.978764+0000","last_active":"2024-01-27T02:37:03.982653+0000","last_peered":"2024-01-27T02:37:03.982653+0000","last_clean":"2024-01-27T02:37:03.982653+0000","last_became_active":"2024-01-27T02:37:03.978430+0000","last_became_peered":"2024-01-27T02:37:03.978430+0000","last_unstale":"2024-01-27T02:37:03.982653+0000","last_undegraded":"2024-01-27T02:37:03.982653+0000","last_fullsized":"2024-01-27T02:37:03.982653+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:27:27.568462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"97'8","reported_seq":18,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:04.001401+0000","last_change":"2024-01-27T02:37:02.980162+0000","last_active":"2024-01-27T02:37:04.001401+0000","last_peered":"2024-01-27T02:37:04.001401+0000","last_clean":"2024-01-27T02:37:04.001401+0000","last_became_active":"2024-01-27T02:37:02.979918+0000","last_became_peered":"2024-01-27T02:37:02.979918+0000","last_unstale":"2024-01-27T02:37:04.001401+0000","last_undegraded":"2024-01-27T02:37:04.001401+0000","last_fullsized":"2024-01-27T02:37:04.001401+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:16:11.230822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.984790+0000","last_change":"2024-01-27T02:37:03.984645+0000","last_active":"2024-01-27T02:37:03.984790+0000","last_peered":"2024-01-27T02:37:03.984790+0000","last_clean":"2024-01-27T02:37:03.984790+0000","last_became_active":"2024-01-27T02:37:03.984412+0000","last_became_peered":"2024-01-27T02:37:03.984412+0000","last_unstale":"2024-01-27T02:37:03.984790+0000","last_undegraded":"2024-01-27T02:37:03.984790+0000","last_fullsized":"2024-01-27T02:37:03.984790+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:19:18.604178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"97'1","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.989113+0000","last_change":"2024-01-27T02:37:02.976792+0000","last_active":"2024-01-27T02:37:03.989113+0000","last_peered":"2024-01-27T02:37:03.989113+0000","last_clean":"2024-01-27T02:37:03.989113+0000","last_became_active":"2024-01-27T02:37:02.976590+0000","last_became_peered":"2024-01-27T02:37:02.976590+0000","last_unstale":"2024-01-27T02:37:03.989113+0000","last_undegraded":"2024-01-27T02:37:03.989113+0000","last_fullsized":"2024-01-27T02:37:03.989113+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:28:59.567547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.980723+0000","last_change":"2024-01-27T02:37:03.978614+0000","last_active":"2024-01-27T02:37:03.980723+0000","last_peered":"2024-01-27T02:37:03.980723+0000","last_clean":"2024-01-27T02:37:03.980723+0000","last_became_active":"2024-01-27T02:37:03.978100+0000","last_became_peered":"2024-01-27T02:37:03.978100+0000","last_unstale":"2024-01-27T02:37:03.980723+0000","last_undegraded":"2024-01-27T02:37:03.980723+0000","last_fullsized":"2024-01-27T02:37:03.980723+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:54:16.825640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"97'2","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.996263+0000","last_change":"2024-01-27T02:37:02.979310+0000","last_active":"2024-01-27T02:37:03.996263+0000","last_peered":"2024-01-27T02:37:03.996263+0000","last_clean":"2024-01-27T02:37:03.996263+0000","last_became_active":"2024-01-27T02:37:02.979006+0000","last_became_peered":"2024-01-27T02:37:02.979006+0000","last_unstale":"2024-01-27T02:37:03.996263+0000","last_undegraded":"2024-01-27T02:37:03.996263+0000","last_fullsized":"2024-01-27T02:37:03.996263+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:11:18.375438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.988332+0000","last_change":"2024-01-27T02:37:03.988214+0000","last_active":"2024-01-27T02:37:03.988332+0000","last_peered":"2024-01-27T02:37:03.988332+0000","last_clean":"2024-01-27T02:37:03.988332+0000","last_became_active":"2024-01-27T02:37:03.987991+0000","last_became_peered":"2024-01-27T02:37:03.987991+0000","last_unstale":"2024-01-27T02:37:03.988332+0000","last_undegraded":"2024-01-27T02:37:03.988332+0000","last_fullsized":"2024-01-27T02:37:03.988332+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:23:03.536769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.979007+0000","last_change":"2024-01-27T02:37:02.977693+0000","last_active":"2024-01-27T02:37:03.979007+0000","last_peered":"2024-01-27T02:37:03.979007+0000","last_clean":"2024-01-27T02:37:03.979007+0000","last_became_active":"2024-01-27T02:37:02.977381+0000","last_became_peered":"2024-01-27T02:37:02.977381+0000","last_unstale":"2024-01-27T02:37:03.979007+0000","last_undegraded":"2024-01-27T02:37:03.979007+0000","last_fullsized":"2024-01-27T02:37:03.979007+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:14:50.580128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.983247+0000","last_change":"2024-01-27T02:37:03.982932+0000","last_active":"2024-01-27T02:37:03.983247+0000","last_peered":"2024-01-27T02:37:03.983247+0000","last_clean":"2024-01-27T02:37:03.983247+0000","last_became_active":"2024-01-27T02:37:03.982683+0000","last_became_peered":"2024-01-27T02:37:03.982683+0000","last_unstale":"2024-01-27T02:37:03.983247+0000","last_undegraded":"2024-01-27T02:37:03.983247+0000","last_fullsized":"2024-01-27T02:37:03.983247+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:19:53.912030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"97'2","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.995276+0000","last_change":"2024-01-27T02:37:02.987475+0000","last_active":"2024-01-27T02:37:03.995276+0000","last_peered":"2024-01-27T02:37:03.995276+0000","last_clean":"2024-01-27T02:37:03.995276+0000","last_became_active":"2024-01-27T02:37:02.987322+0000","last_became_peered":"2024-01-27T02:37:02.987322+0000","last_unstale":"2024-01-27T02:37:03.995276+0000","last_undegraded":"2024-01-27T02:37:03.995276+0000","last_fullsized":"2024-01-27T02:37:03.995276+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:19:46.184238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.976434+0000","last_change":"2024-01-27T02:37:03.976154+0000","last_active":"2024-01-27T02:37:03.976434+0000","last_peered":"2024-01-27T02:37:03.976434+0000","last_clean":"2024-01-27T02:37:03.976434+0000","last_became_active":"2024-01-27T02:37:03.975889+0000","last_became_peered":"2024-01-27T02:37:03.975889+0000","last_unstale":"2024-01-27T02:37:03.976434+0000","last_undegraded":"2024-01-27T02:37:03.976434+0000","last_fullsized":"2024-01-27T02:37:03.976434+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:20:09.835171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.975591+0000","last_change":"2024-01-27T02:37:02.985913+0000","last_active":"2024-01-27T02:37:03.975591+0000","last_peered":"2024-01-27T02:37:03.975591+0000","last_clean":"2024-01-27T02:37:03.975591+0000","last_became_active":"2024-01-27T02:37:02.985569+0000","last_became_peered":"2024-01-27T02:37:02.985569+0000","last_unstale":"2024-01-27T02:37:03.975591+0000","last_undegraded":"2024-01-27T02:37:03.975591+0000","last_fullsized":"2024-01-27T02:37:03.975591+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:55:50.883094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.994201+0000","last_change":"2024-01-27T02:37:03.991778+0000","last_active":"2024-01-27T02:37:03.994201+0000","last_peered":"2024-01-27T02:37:03.994201+0000","last_clean":"2024-01-27T02:37:03.994201+0000","last_became_active":"2024-01-27T02:37:03.984102+0000","last_became_peered":"2024-01-27T02:37:03.984102+0000","last_unstale":"2024-01-27T02:37:03.994201+0000","last_undegraded":"2024-01-27T02:37:03.994201+0000","last_fullsized":"2024-01-27T02:37:03.994201+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:02:05.606454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.976520+0000","last_change":"2024-01-27T02:37:02.985405+0000","last_active":"2024-01-27T02:37:03.976520+0000","last_peered":"2024-01-27T02:37:03.976520+0000","last_clean":"2024-01-27T02:37:03.976520+0000","last_became_active":"2024-01-27T02:37:02.985122+0000","last_became_peered":"2024-01-27T02:37:02.985122+0000","last_unstale":"2024-01-27T02:37:03.976520+0000","last_undegraded":"2024-01-27T02:37:03.976520+0000","last_fullsized":"2024-01-27T02:37:03.976520+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:40:22.882301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.988197+0000","last_change":"2024-01-27T02:37:03.988116+0000","last_active":"2024-01-27T02:37:03.988197+0000","last_peered":"2024-01-27T02:37:03.988197+0000","last_clean":"2024-01-27T02:37:03.988197+0000","last_became_active":"2024-01-27T02:37:03.987868+0000","last_became_peered":"2024-01-27T02:37:03.987868+0000","last_unstale":"2024-01-27T02:37:03.988197+0000","last_undegraded":"2024-01-27T02:37:03.988197+0000","last_fullsized":"2024-01-27T02:37:03.988197+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:09:36.161409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.970170+0000","last_change":"2024-01-27T02:37:02.986387+0000","last_active":"2024-01-27T02:37:03.970170+0000","last_peered":"2024-01-27T02:37:03.970170+0000","last_clean":"2024-01-27T02:37:03.970170+0000","last_became_active":"2024-01-27T02:37:02.986149+0000","last_became_peered":"2024-01-27T02:37:02.986149+0000","last_unstale":"2024-01-27T02:37:03.970170+0000","last_undegraded":"2024-01-27T02:37:03.970170+0000","last_fullsized":"2024-01-27T02:37:03.970170+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:11:07.050220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.984613+0000","last_change":"2024-01-27T02:37:03.984508+0000","last_active":"2024-01-27T02:37:03.984613+0000","last_peered":"2024-01-27T02:37:03.984613+0000","last_clean":"2024-01-27T02:37:03.984613+0000","last_became_active":"2024-01-27T02:37:03.984271+0000","last_became_peered":"2024-01-27T02:37:03.984271+0000","last_unstale":"2024-01-27T02:37:03.984613+0000","last_undegraded":"2024-01-27T02:37:03.984613+0000","last_fullsized":"2024-01-27T02:37:03.984613+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:11:19.380830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"97'2","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.996746+0000","last_change":"2024-01-27T02:37:02.982134+0000","last_active":"2024-01-27T02:37:03.996746+0000","last_peered":"2024-01-27T02:37:03.996746+0000","last_clean":"2024-01-27T02:37:03.996746+0000","last_became_active":"2024-01-27T02:37:02.981540+0000","last_became_peered":"2024-01-27T02:37:02.981540+0000","last_unstale":"2024-01-27T02:37:03.996746+0000","last_undegraded":"2024-01-27T02:37:03.996746+0000","last_fullsized":"2024-01-27T02:37:03.996746+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:14:59.963707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.977931+0000","last_change":"2024-01-27T02:37:03.977858+0000","last_active":"2024-01-27T02:37:03.977931+0000","last_peered":"2024-01-27T02:37:03.977931+0000","last_clean":"2024-01-27T02:37:03.977931+0000","last_became_active":"2024-01-27T02:37:03.977564+0000","last_became_peered":"2024-01-27T02:37:03.977564+0000","last_unstale":"2024-01-27T02:37:03.977931+0000","last_undegraded":"2024-01-27T02:37:03.977931+0000","last_fullsized":"2024-01-27T02:37:03.977931+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:19:47.189629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.971852+0000","last_change":"2024-01-27T02:37:02.979350+0000","last_active":"2024-01-27T02:37:03.971852+0000","last_peered":"2024-01-27T02:37:03.971852+0000","last_clean":"2024-01-27T02:37:03.971852+0000","last_became_active":"2024-01-27T02:37:02.979028+0000","last_became_peered":"2024-01-27T02:37:02.979028+0000","last_unstale":"2024-01-27T02:37:03.971852+0000","last_undegraded":"2024-01-27T02:37:03.971852+0000","last_fullsized":"2024-01-27T02:37:03.971852+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:29:41.047727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.980741+0000","last_change":"2024-01-27T02:37:03.979467+0000","last_active":"2024-01-27T02:37:03.980741+0000","last_peered":"2024-01-27T02:37:03.980741+0000","last_clean":"2024-01-27T02:37:03.980741+0000","last_became_active":"2024-01-27T02:37:03.979218+0000","last_became_peered":"2024-01-27T02:37:03.979218+0000","last_unstale":"2024-01-27T02:37:03.980741+0000","last_undegraded":"2024-01-27T02:37:03.980741+0000","last_fullsized":"2024-01-27T02:37:03.980741+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:11:08.055611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.971611+0000","last_change":"2024-01-27T02:37:02.987022+0000","last_active":"2024-01-27T02:37:03.971611+0000","last_peered":"2024-01-27T02:37:03.971611+0000","last_clean":"2024-01-27T02:37:03.971611+0000","last_became_active":"2024-01-27T02:37:02.986813+0000","last_became_peered":"2024-01-27T02:37:02.986813+0000","last_unstale":"2024-01-27T02:37:03.971611+0000","last_undegraded":"2024-01-27T02:37:03.971611+0000","last_fullsized":"2024-01-27T02:37:03.971611+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:31:48.055150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.983979+0000","last_change":"2024-01-27T02:37:03.983901+0000","last_active":"2024-01-27T02:37:03.983979+0000","last_peered":"2024-01-27T02:37:03.983979+0000","last_clean":"2024-01-27T02:37:03.983979+0000","last_became_active":"2024-01-27T02:37:03.983655+0000","last_became_peered":"2024-01-27T02:37:03.983655+0000","last_unstale":"2024-01-27T02:37:03.983979+0000","last_undegraded":"2024-01-27T02:37:03.983979+0000","last_fullsized":"2024-01-27T02:37:03.983979+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:05:34.176684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.972141+0000","last_change":"2024-01-27T02:37:02.972564+0000","last_active":"2024-01-27T02:37:03.972141+0000","last_peered":"2024-01-27T02:37:03.972141+0000","last_clean":"2024-01-27T02:37:03.972141+0000","last_became_active":"2024-01-27T02:37:02.972302+0000","last_became_peered":"2024-01-27T02:37:02.972302+0000","last_unstale":"2024-01-27T02:37:03.972141+0000","last_undegraded":"2024-01-27T02:37:03.972141+0000","last_fullsized":"2024-01-27T02:37:03.972141+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:21:13.457215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.982259+0000","last_change":"2024-01-27T02:37:03.980359+0000","last_active":"2024-01-27T02:37:03.982259+0000","last_peered":"2024-01-27T02:37:03.982259+0000","last_clean":"2024-01-27T02:37:03.982259+0000","last_became_active":"2024-01-27T02:37:03.976853+0000","last_became_peered":"2024-01-27T02:37:03.976853+0000","last_unstale":"2024-01-27T02:37:03.982259+0000","last_undegraded":"2024-01-27T02:37:03.982259+0000","last_fullsized":"2024-01-27T02:37:03.982259+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:57:41.192776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.980358+0000","last_change":"2024-01-27T02:37:02.978741+0000","last_active":"2024-01-27T02:37:03.980358+0000","last_peered":"2024-01-27T02:37:03.980358+0000","last_clean":"2024-01-27T02:37:03.980358+0000","last_became_active":"2024-01-27T02:37:02.978478+0000","last_became_peered":"2024-01-27T02:37:02.978478+0000","last_unstale":"2024-01-27T02:37:03.980358+0000","last_undegraded":"2024-01-27T02:37:03.980358+0000","last_fullsized":"2024-01-27T02:37:03.980358+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:28:16.622480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.986547+0000","last_change":"2024-01-27T02:37:03.986463+0000","last_active":"2024-01-27T02:37:03.986547+0000","last_peered":"2024-01-27T02:37:03.986547+0000","last_clean":"2024-01-27T02:37:03.986547+0000","last_became_active":"2024-01-27T02:37:03.985851+0000","last_became_peered":"2024-01-27T02:37:03.985851+0000","last_unstale":"2024-01-27T02:37:03.986547+0000","last_undegraded":"2024-01-27T02:37:03.986547+0000","last_fullsized":"2024-01-27T02:37:03.986547+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:09:36.161409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.977915+0000","last_change":"2024-01-27T02:37:02.978252+0000","last_active":"2024-01-27T02:37:03.977915+0000","last_peered":"2024-01-27T02:37:03.977915+0000","last_clean":"2024-01-27T02:37:03.977915+0000","last_became_active":"2024-01-27T02:37:02.977491+0000","last_became_peered":"2024-01-27T02:37:02.977491+0000","last_unstale":"2024-01-27T02:37:03.977915+0000","last_undegraded":"2024-01-27T02:37:03.977915+0000","last_fullsized":"2024-01-27T02:37:03.977915+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:06:56.123644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.979536+0000","last_change":"2024-01-27T02:37:03.979433+0000","last_active":"2024-01-27T02:37:03.979536+0000","last_peered":"2024-01-27T02:37:03.979536+0000","last_clean":"2024-01-27T02:37:03.979536+0000","last_became_active":"2024-01-27T02:37:03.979076+0000","last_became_peered":"2024-01-27T02:37:03.979076+0000","last_unstale":"2024-01-27T02:37:03.979536+0000","last_undegraded":"2024-01-27T02:37:03.979536+0000","last_fullsized":"2024-01-27T02:37:03.979536+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:05:34.176684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.972474+0000","last_change":"2024-01-27T02:37:02.983565+0000","last_active":"2024-01-27T02:37:03.972474+0000","last_peered":"2024-01-27T02:37:03.972474+0000","last_clean":"2024-01-27T02:37:03.972474+0000","last_became_active":"2024-01-27T02:37:02.983313+0000","last_became_peered":"2024-01-27T02:37:02.983313+0000","last_unstale":"2024-01-27T02:37:03.972474+0000","last_undegraded":"2024-01-27T02:37:03.972474+0000","last_fullsized":"2024-01-27T02:37:03.972474+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:29:25.334034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.981962+0000","last_change":"2024-01-27T02:37:03.981805+0000","last_active":"2024-01-27T02:37:03.981962+0000","last_peered":"2024-01-27T02:37:03.981962+0000","last_clean":"2024-01-27T02:37:03.981962+0000","last_became_active":"2024-01-27T02:37:03.981416+0000","last_became_peered":"2024-01-27T02:37:03.981416+0000","last_unstale":"2024-01-27T02:37:03.981962+0000","last_undegraded":"2024-01-27T02:37:03.981962+0000","last_fullsized":"2024-01-27T02:37:03.981962+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:08:03.599041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.974452+0000","last_change":"2024-01-27T02:37:02.978260+0000","last_active":"2024-01-27T02:37:03.974452+0000","last_peered":"2024-01-27T02:37:03.974452+0000","last_clean":"2024-01-27T02:37:03.974452+0000","last_became_active":"2024-01-27T02:37:02.978021+0000","last_became_peered":"2024-01-27T02:37:02.978021+0000","last_unstale":"2024-01-27T02:37:03.974452+0000","last_undegraded":"2024-01-27T02:37:03.974452+0000","last_fullsized":"2024-01-27T02:37:03.974452+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:48:16.961451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.982724+0000","last_change":"2024-01-27T02:37:03.982160+0000","last_active":"2024-01-27T02:37:03.982724+0000","last_peered":"2024-01-27T02:37:03.982724+0000","last_clean":"2024-01-27T02:37:03.982724+0000","last_became_active":"2024-01-27T02:37:03.981913+0000","last_became_peered":"2024-01-27T02:37:03.981913+0000","last_unstale":"2024-01-27T02:37:03.982724+0000","last_undegraded":"2024-01-27T02:37:03.982724+0000","last_fullsized":"2024-01-27T02:37:03.982724+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:35:18.731072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"97'1","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.997265+0000","last_change":"2024-01-27T02:37:02.980076+0000","last_active":"2024-01-27T02:37:03.997265+0000","last_peered":"2024-01-27T02:37:03.997265+0000","last_clean":"2024-01-27T02:37:03.997265+0000","last_became_active":"2024-01-27T02:37:02.979839+0000","last_became_peered":"2024-01-27T02:37:02.979839+0000","last_unstale":"2024-01-27T02:37:03.997265+0000","last_undegraded":"2024-01-27T02:37:03.997265+0000","last_fullsized":"2024-01-27T02:37:03.997265+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:04:34.145511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":230,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.974931+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:37:03.974931+0000","last_peered":"2024-01-27T02:37:03.974931+0000","last_clean":"2024-01-27T02:37:03.974931+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:37:03.974931+0000","last_undegraded":"2024-01-27T02:37:03.974931+0000","last_fullsized":"2024-01-27T02:37:03.974931+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1089,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194223,"statfs":{"total":200000000,"available":198884758,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.45000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71599999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.46800000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.49099999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.435,"5min":0.373,"15min":0.373},"max":{"1min":0.71899999999999997,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.438,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.70599999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.53300000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.45800000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.72299999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.41499999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70499999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.65300000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.73499999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.72499999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.502}]},{"osd":4,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48299999999999998,"15min":0.48999999999999999},"min":{"1min":0.29299999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66500000000000004,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.504,"15min":0.51600000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73399999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.54600000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.52300000000000002,"15min":0.53700000000000003},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.72099999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55400000000000005,"15min":0.56499999999999995},"min":{"1min":0.38700000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73499999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.58599999999999997}]},{"osd":6,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.58399999999999996,"15min":0.59799999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73199999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.629},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56799999999999995,"15min":0.57799999999999996},"min":{"1min":0.40500000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76700000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":513,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194799,"statfs":{"total":200000000,"available":199474530,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58699999999999997,"15min":0.52100000000000002},"min":{"1min":0.45000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.52700000000000002},"min":{"1min":0.434,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71899999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.66300000000000003}]},{"osd":1,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.51800000000000002},"min":{"1min":0.41999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79700000000000004,"5min":0.874,"15min":0.874},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.60599999999999998,"15min":0.55500000000000005},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.68500000000000005}]},{"osd":2,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.41199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.625,"15min":0.58299999999999996},"min":{"1min":0.45100000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79700000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.64900000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58199999999999996,"15min":0.55100000000000005},"min":{"1min":0.47599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.76800000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61499999999999999,"15min":0.57299999999999995},"min":{"1min":0.48499999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.78100000000000003,"5min":1.262,"15min":1.262},"last":0.57999999999999996}]},{"osd":4,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.34200000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52700000000000002,"15min":0.55300000000000005},"min":{"1min":0.41199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68300000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.41199999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53200000000000003,"15min":0.53500000000000003},"min":{"1min":0.38800000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70799999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.60599999999999998},"min":{"1min":0.40899999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70799999999999996,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.47399999999999998}]},{"osd":7,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60599999999999998,"15min":0.61599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73699999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71099999999999997,"5min":1.038,"15min":1.038},"last":0.441}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":512,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194799,"statfs":{"total":200000000,"available":199475002,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.38600000000000001,"15min":0.373},"min":{"1min":0.307,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.55900000000000005,"5min":0.626,"15min":0.626},"last":0.435},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40699999999999997,"15min":0.39500000000000002},"min":{"1min":0.28699999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.57099999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.39800000000000002}]},{"osd":2,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.36099999999999999},"min":{"1min":0.29599999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.60099999999999998,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.442,"15min":0.42699999999999999},"min":{"1min":0.35499999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65900000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.51600000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48099999999999998,"15min":0.46600000000000003},"min":{"1min":0.36599999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.501,"5min":0.46300000000000002,"15min":0.45400000000000001},"min":{"1min":0.34399999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.68700000000000006,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.47999999999999998}]},{"osd":4,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.63200000000000001,"15min":0.624},"min":{"1min":0.48599999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.96799999999999997,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.64600000000000002,"15min":0.63200000000000001},"min":{"1min":0.51700000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88800000000000001,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.88900000000000001}]},{"osd":5,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63800000000000001,"15min":0.622},"min":{"1min":0.47499999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.93400000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.66300000000000003,"15min":0.66700000000000004},"min":{"1min":0.47299999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.81899999999999995}]},{"osd":6,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.70299999999999996,"15min":0.69399999999999995},"min":{"1min":0.55100000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.95699999999999996,"5min":1.024,"15min":1.024},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.68400000000000005,"15min":0.67000000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.90300000000000002,"5min":1.151,"15min":1.151},"last":0.72799999999999998}]},{"osd":7,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.69799999999999995,"15min":0.67800000000000005},"min":{"1min":0.495,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88,"5min":1.097,"15min":1.097},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.70299999999999996,"15min":0.68500000000000005},"min":{"1min":0.54600000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.86799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1089,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194223,"statfs":{"total":200000000,"available":198884794,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.38900000000000001},"min":{"1min":0.28100000000000003,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.66600000000000004,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27800000000000002,"5min":0.223,"15min":0.223},"max":{"1min":0.65200000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.38700000000000001}]},{"osd":2,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.436,"15min":0.42199999999999999},"min":{"1min":0.32800000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56299999999999994,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.44900000000000001,"15min":0.435},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58999999999999997,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.46600000000000003}]},{"osd":3,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42999999999999999,"15min":0.432},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.63600000000000001,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.443},{"interface":"front","average":{"1min":0.502,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.35299999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.63200000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.54900000000000004}]},{"osd":4,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.65500000000000003,"15min":0.64000000000000001},"min":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45900000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89700000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.60199999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67500000000000004,"15min":0.65600000000000003},"min":{"1min":0.48399999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.86499999999999999,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.62},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67700000000000005,"15min":0.68200000000000005},"min":{"1min":0.5,"5min":0.44,"15min":0.44},"max":{"1min":0.96599999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.66700000000000004}]},{"osd":6,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.56200000000000006,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91300000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.754},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69299999999999995,"15min":0.67900000000000005},"min":{"1min":0.56599999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94199999999999995,"5min":1.04,"15min":1.04},"last":0.77000000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.55300000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.52100000000000002,"5min":0.498,"15min":0.498},"max":{"1min":0.95799999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.79300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":513,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194799,"statfs":{"total":200000000,"available":199474642,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.41699999999999998,"15min":0.40999999999999998},"min":{"1min":0.28000000000000003,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56599999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.50900000000000001,"15min":0.63700000000000001},"min":{"1min":0.34499999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72799999999999998,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":0.438}]},{"osd":1,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.434,"15min":0.42399999999999999},"min":{"1min":0.32700000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.44,"15min":0.44700000000000001},"min":{"1min":0.29799999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.65300000000000002,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.39800000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.41499999999999998},"min":{"1min":0.30399999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.65000000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76500000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.372}]},{"osd":4,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68200000000000005,"15min":0.65600000000000003},"min":{"1min":0.53600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73399999999999999,"15min":0.79900000000000004},"min":{"1min":0.54900000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.995,"5min":3.581,"15min":3.581},"last":0.55000000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68000000000000005,"15min":0.74399999999999999},"min":{"1min":0.502,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90500000000000003,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67700000000000005,"15min":0.67100000000000004},"min":{"1min":0.501,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85899999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.54000000000000004}]},{"osd":6,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.53800000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93999999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0189999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.52600000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.53600000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70299999999999996,"15min":0.69099999999999995},"min":{"1min":0.625,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.44700000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":514,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194798,"statfs":{"total":200000000,"available":199473660,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.38800000000000001,"15min":0.35899999999999999},"min":{"1min":0.255,"5min":0.253,"15min":0.253},"max":{"1min":0.56200000000000006,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40600000000000003,"15min":0.39500000000000002},"min":{"1min":0.29199999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.626,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.30399999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.27400000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66300000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.45600000000000002,"15min":0.42399999999999999},"min":{"1min":0.30199999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.85099999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46100000000000002,"15min":0.43099999999999999},"min":{"1min":0.33700000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.48399999999999999,"15min":0.45000000000000001},"min":{"1min":0.378,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.503}]},{"osd":4,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.66900000000000004,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.6660000000000004,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64200000000000002,"15min":0.60599999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93300000000000005,"5min":1.012,"15min":1.012},"last":0.67400000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.61299999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90400000000000003,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.71599999999999997,"15min":0.68000000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":4.6849999999999996,"5min":4.6849999999999996,"15min":4.6849999999999996},"last":0.72699999999999998}]},{"osd":6,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90100000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.47099999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.61299999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.71699999999999997,"15min":0.65400000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":4.7130000000000001,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68500000000000005,"15min":0.66700000000000004},"min":{"1min":0.495,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.71299999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":512,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194799,"statfs":{"total":200000000,"available":199475196,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.47299999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.73499999999999999,"5min":1.099,"15min":1.099},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.45000000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73499999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.51000000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.48999999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0309999999999999,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.496},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81999999999999995,"5min":1.133,"15min":1.133},"last":0.60299999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60999999999999999,"15min":0.59699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.3400000000000001,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.46400000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79800000000000004,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.65200000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.47699999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.44900000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82099999999999995,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.53000000000000003}]},{"osd":5,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.33700000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32800000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.42899999999999999}]},{"osd":6,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.501,"15min":0.499},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68500000000000005,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38600000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.39700000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54600000000000004,"15min":0.55200000000000005},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79000000000000004,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.45400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":514,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194798,"statfs":{"total":200000000,"available":199473638,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56999999999999995,"15min":0.59399999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68700000000000006,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.60099999999999998,"15min":0.57699999999999996},"min":{"1min":0.34499999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67100000000000004,"5min":3.242,"15min":3.242},"last":0.61099999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.625,"15min":0.60799999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70599999999999996,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.627},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.626,"15min":0.64900000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69299999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.67600000000000005}]},{"osd":2,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61599999999999999,"15min":0.59599999999999997},"min":{"1min":0.39100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.71499999999999997,"5min":3.04,"15min":3.04},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59399999999999997,"15min":0.57699999999999996},"min":{"1min":0.40100000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72499999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.63400000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.66500000000000004,"15min":0.67800000000000005},"min":{"1min":0.44400000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.65500000000000003,"5min":4.032,"15min":4.032},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.70399999999999996,"15min":0.67800000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.77000000000000002,"5min":4.282,"15min":4.282},"last":0.71499999999999997}]},{"osd":4,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81399999999999995,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.51100000000000001,"15min":0.56499999999999995},"min":{"1min":0.36799999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.61499999999999999,"5min":1.101,"15min":1.101},"last":0.497}]},{"osd":6,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51300000000000001,"15min":0.55600000000000005},"min":{"1min":0.36499999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.629,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56499999999999995,"15min":0.58199999999999996},"min":{"1min":0.39100000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76800000000000002,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.52500000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.55400000000000005,"15min":0.56699999999999995},"min":{"1min":0.40200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.745,"5min":1.008,"15min":1.008},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.626,"15min":0.65800000000000003},"min":{"1min":0.38400000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84799999999999998,"5min":2,"15min":2},"last":0.65800000000000003}]}]}],"pool_statfs":[]}} 2024-01-27T02:37:08.164 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:37:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.332+0000 7fb7b7127700 1 Processor -- start 2024-01-27T02:37:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.333+0000 7fb7b7127700 1 -- start start 2024-01-27T02:37:08.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.333+0000 7fb7b7127700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb7b00ca140 0x7fb7b00ca510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:08.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.333+0000 7fb7b7127700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb7b00b1f50 0x7fb7b00ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:08.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.333+0000 7fb7b7127700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7b00b2350 con 0x7fb7b00b1af0 2024-01-27T02:37:08.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.333+0000 7fb7b7127700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b00c4bd0 con 0x7fb7b00b1f50 2024-01-27T02:37:08.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.333+0000 7fb7b7127700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b00caea0 con 0x7fb7b00ca140 2024-01-27T02:37:08.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.334+0000 7fb7b56c4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb7b00b1f50 0x7fb7b00ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:08.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.334+0000 7fb7b56c4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb7b00b1f50 0x7fb7b00ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37570/0 (socket says 172.21.15.7:37570) 2024-01-27T02:37:08.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.334+0000 7fb7b56c4700 1 -- 172.21.15.7:0/4009152453 learned_addr learned my addr 172.21.15.7:0/4009152453 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:08.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.334+0000 7fb7b4ec3700 1 --2- 172.21.15.7:0/4009152453 >> v2:172.21.15.188:3300/0 conn(0x7fb7b00ca140 0x7fb7b00ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:08.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.334+0000 7fb7b56c4700 1 -- 172.21.15.7:0/4009152453 >> v2:172.21.15.188:3300/0 conn(0x7fb7b00ca140 msgr2=0x7fb7b00ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:08.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.334+0000 7fb7b56c4700 1 --2- 172.21.15.7:0/4009152453 >> v2:172.21.15.188:3300/0 conn(0x7fb7b00ca140 0x7fb7b00ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:08.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.334+0000 7fb7b56c4700 1 -- 172.21.15.7:0/4009152453 >> v1:172.21.15.188:6789/0 conn(0x7fb7b00b1af0 legacy=0x7fb7b00c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:08.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.334+0000 7fb7b56c4700 1 -- 172.21.15.7:0/4009152453 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b00cb180 con 0x7fb7b00b1f50 2024-01-27T02:37:08.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.334+0000 7fb7b4ec3700 1 --2- 172.21.15.7:0/4009152453 >> v2:172.21.15.188:3300/0 conn(0x7fb7b00ca140 0x7fb7b00ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:37:08.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.334+0000 7fb7b56c4700 1 --2- 172.21.15.7:0/4009152453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b00b1f50 0x7fb7b00ccb10 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00b4e20 tx=0x7fb7a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962ec6f4595705c5 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:08.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.334+0000 7fb7af7fe700 1 -- 172.21.15.7:0/4009152453 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a800e070 con 0x7fb7b00b1f50 2024-01-27T02:37:08.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.334+0000 7fb7af7fe700 1 -- 172.21.15.7:0/4009152453 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a800ba50 con 0x7fb7b00b1f50 2024-01-27T02:37:08.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.334+0000 7fb7af7fe700 1 -- 172.21.15.7:0/4009152453 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a80092e0 con 0x7fb7b00b1f50 2024-01-27T02:37:08.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.335+0000 7fb7b7127700 1 -- 172.21.15.7:0/4009152453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b00b1f50 msgr2=0x7fb7b00ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:08.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.335+0000 7fb7b7127700 1 --2- 172.21.15.7:0/4009152453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b00b1f50 0x7fb7b00ccb10 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00b4e20 tx=0x7fb7a8009840 comp rx=0 tx=0).stop 2024-01-27T02:37:08.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.335+0000 7fb7b7127700 1 -- 172.21.15.7:0/4009152453 shutdown_connections 2024-01-27T02:37:08.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.335+0000 7fb7b7127700 1 --2- 172.21.15.7:0/4009152453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b00b1f50 0x7fb7b00ccb10 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:08.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.335+0000 7fb7b7127700 1 --2- 172.21.15.7:0/4009152453 >> v2:172.21.15.188:3300/0 conn(0x7fb7b00ca140 0x7fb7b00ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:08.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.335+0000 7fb7b7127700 1 -- 172.21.15.7:0/4009152453 >> 172.21.15.7:0/4009152453 conn(0x7fb7b00711c0 msgr2=0x7fb7b00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:08.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.335+0000 7fb7b7127700 1 -- 172.21.15.7:0/4009152453 shutdown_connections 2024-01-27T02:37:08.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.335+0000 7fb7b7127700 1 -- 172.21.15.7:0/4009152453 wait complete. 2024-01-27T02:37:08.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.336+0000 7fb7b7127700 1 Processor -- start 2024-01-27T02:37:08.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.337+0000 7fb7b7127700 1 -- start start 2024-01-27T02:37:08.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.337+0000 7fb7b7127700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b00b1af0 0x7fb7b0156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:08.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.337+0000 7fb7b7127700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7b00b1f50 0x7fb7b013f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.337+0000 7fb7b7127700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7b00ca140 0x7fb7b015cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.337+0000 7fb7b7127700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b013f460 con 0x7fb7b00b1af0 2024-01-27T02:37:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.337+0000 7fb7b7127700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b013e850 con 0x7fb7b00ca140 2024-01-27T02:37:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.337+0000 7fb7b7127700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b013eb30 con 0x7fb7b00b1f50 2024-01-27T02:37:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.337+0000 7fb7b4ec3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b00b1af0 0x7fb7b0156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.337+0000 7fb7b4ec3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b00b1af0 0x7fb7b0156460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37572/0 (socket says 172.21.15.7:37572) 2024-01-27T02:37:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.337+0000 7fb7b4ec3700 1 -- 172.21.15.7:0/1907797966 learned_addr learned my addr 172.21.15.7:0/1907797966 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.338+0000 7fb7b56c4700 1 --2- 172.21.15.7:0/1907797966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7b00ca140 0x7fb7b015cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.338+0000 7fb7affff700 1 --2- 172.21.15.7:0/1907797966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7b00b1f50 0x7fb7b013f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.338+0000 7fb7b4ec3700 1 -- 172.21.15.7:0/1907797966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7b00b1f50 msgr2=0x7fb7b013f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.338+0000 7fb7b4ec3700 1 --2- 172.21.15.7:0/1907797966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7b00b1f50 0x7fb7b013f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.338+0000 7fb7b4ec3700 1 -- 172.21.15.7:0/1907797966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7b00ca140 msgr2=0x7fb7b015cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.338+0000 7fb7b4ec3700 1 --2- 172.21.15.7:0/1907797966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7b00ca140 0x7fb7b015cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.338+0000 7fb7b4ec3700 1 -- 172.21.15.7:0/1907797966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a8009ae0 con 0x7fb7b00b1af0 2024-01-27T02:37:08.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.338+0000 7fb7b4ec3700 1 --2- 172.21.15.7:0/1907797966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b00b1af0 0x7fb7b0156460 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000d890 tx=0x7fb7a0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:08.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.338+0000 7fb7acff9700 1 -- 172.21.15.7:0/1907797966 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0005a00 con 0x7fb7b00b1af0 2024-01-27T02:37:08.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.338+0000 7fb7acff9700 1 -- 172.21.15.7:0/1907797966 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a0005b60 con 0x7fb7b00b1af0 2024-01-27T02:37:08.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.338+0000 7fb7b7127700 1 -- 172.21.15.7:0/1907797966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b013ee10 con 0x7fb7b00b1af0 2024-01-27T02:37:08.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.338+0000 7fb7acff9700 1 -- 172.21.15.7:0/1907797966 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a00125f0 con 0x7fb7b00b1af0 2024-01-27T02:37:08.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.338+0000 7fb7b7127700 1 -- 172.21.15.7:0/1907797966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b015c9d0 con 0x7fb7b00b1af0 2024-01-27T02:37:08.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.339+0000 7fb7b7127700 1 -- 172.21.15.7:0/1907797966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b013f740 con 0x7fb7b00b1af0 2024-01-27T02:37:08.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.340+0000 7fb7acff9700 1 -- 172.21.15.7:0/1907797966 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb7a00129e0 con 0x7fb7b00b1af0 2024-01-27T02:37:08.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.340+0000 7fb7acff9700 1 --2- 172.21.15.7:0/1907797966 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb790066f30 0x7fb7900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:08.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.340+0000 7fb7acff9700 1 -- 172.21.15.7:0/1907797966 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb7a00841b0 con 0x7fb7b00b1af0 2024-01-27T02:37:08.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.340+0000 7fb7affff700 1 --2- 172.21.15.7:0/1907797966 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb790066f30 0x7fb7900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:08.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.341+0000 7fb7affff700 1 --2- 172.21.15.7:0/1907797966 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb790066f30 0x7fb7900693e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb7b01403f0 tx=0x7fb798009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:08.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.344+0000 7fb7acff9700 1 -- 172.21.15.7:0/1907797966 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb7a004d5a0 con 0x7fb7b00b1af0 2024-01-27T02:37:08.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.544+0000 7fb7b7127700 1 -- 172.21.15.7:0/1907797966 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb7b013f5a0 con 0x7fb790066f30 2024-01-27T02:37:08.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.552+0000 7fb7acff9700 1 -- 172.21.15.7:0/1907797966 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285449 (secure 0 0 0) 0x7fb798001ca0 con 0x7fb790066f30 2024-01-27T02:37:08.556 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:08.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.555+0000 7fb7b7127700 1 -- 172.21.15.7:0/1907797966 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb790066f30 msgr2=0x7fb7900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:08.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.555+0000 7fb7b7127700 1 --2- 172.21.15.7:0/1907797966 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb790066f30 0x7fb7900693e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb7b01403f0 tx=0x7fb798009040 comp rx=0 tx=0).stop 2024-01-27T02:37:08.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.555+0000 7fb7b7127700 1 -- 172.21.15.7:0/1907797966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b00b1af0 msgr2=0x7fb7b0156460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:08.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.555+0000 7fb7b7127700 1 --2- 172.21.15.7:0/1907797966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b00b1af0 0x7fb7b0156460 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000d890 tx=0x7fb7a0008040 comp rx=0 tx=0).stop 2024-01-27T02:37:08.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.555+0000 7fb7b7127700 1 -- 172.21.15.7:0/1907797966 shutdown_connections 2024-01-27T02:37:08.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.555+0000 7fb7b7127700 1 --2- 172.21.15.7:0/1907797966 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb790066f30 0x7fb7900693e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:08.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.555+0000 7fb7b7127700 1 --2- 172.21.15.7:0/1907797966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7b00ca140 0x7fb7b015cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:08.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.555+0000 7fb7b7127700 1 --2- 172.21.15.7:0/1907797966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b00b1af0 0x7fb7b0156460 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:08.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.555+0000 7fb7b7127700 1 --2- 172.21.15.7:0/1907797966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7b00b1f50 0x7fb7b013f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:08.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.555+0000 7fb7b7127700 1 -- 172.21.15.7:0/1907797966 >> 172.21.15.7:0/1907797966 conn(0x7fb7b00711c0 msgr2=0x7fb7b0159f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:08.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.555+0000 7fb7b7127700 1 -- 172.21.15.7:0/1907797966 shutdown_connections 2024-01-27T02:37:08.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.555+0000 7fb7b7127700 1 -- 172.21.15.7:0/1907797966 wait complete. 2024-01-27T02:37:08.560 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:37:08.561 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:37:08.563 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:37:08.570 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-01-27T02:37:07.646453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5257,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557242,"statfs":{"total":1600000000,"available":1594616220,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.683138"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.981928+0000","last_change":"2024-01-27T02:37:03.981539+0000","last_active":"2024-01-27T02:37:03.981928+0000","last_peered":"2024-01-27T02:37:03.981928+0000","last_clean":"2024-01-27T02:37:03.981928+0000","last_became_active":"2024-01-27T02:37:03.981297+0000","last_became_peered":"2024-01-27T02:37:03.981297+0000","last_unstale":"2024-01-27T02:37:03.981928+0000","last_undegraded":"2024-01-27T02:37:03.981928+0000","last_fullsized":"2024-01-27T02:37:03.981928+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:00:23.179794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.974722+0000","last_change":"2024-01-27T02:37:03.974633+0000","last_active":"2024-01-27T02:37:03.974722+0000","last_peered":"2024-01-27T02:37:03.974722+0000","last_clean":"2024-01-27T02:37:03.974722+0000","last_became_active":"2024-01-27T02:37:03.974323+0000","last_became_peered":"2024-01-27T02:37:03.974323+0000","last_unstale":"2024-01-27T02:37:03.974722+0000","last_undegraded":"2024-01-27T02:37:03.974722+0000","last_fullsized":"2024-01-27T02:37:03.974722+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:48:17.966842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.984565+0000","last_change":"2024-01-27T02:37:03.984493+0000","last_active":"2024-01-27T02:37:03.984565+0000","last_peered":"2024-01-27T02:37:03.984565+0000","last_clean":"2024-01-27T02:37:03.984565+0000","last_became_active":"2024-01-27T02:37:03.984227+0000","last_became_peered":"2024-01-27T02:37:03.984227+0000","last_unstale":"2024-01-27T02:37:03.984565+0000","last_undegraded":"2024-01-27T02:37:03.984565+0000","last_fullsized":"2024-01-27T02:37:03.984565+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:13.752690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.989618+0000","last_change":"2024-01-27T02:37:03.989503+0000","last_active":"2024-01-27T02:37:03.989618+0000","last_peered":"2024-01-27T02:37:03.989618+0000","last_clean":"2024-01-27T02:37:03.989618+0000","last_became_active":"2024-01-27T02:37:03.989059+0000","last_became_peered":"2024-01-27T02:37:03.989059+0000","last_unstale":"2024-01-27T02:37:03.989618+0000","last_undegraded":"2024-01-27T02:37:03.989618+0000","last_fullsized":"2024-01-27T02:37:03.989618+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:40:17.306454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.978681+0000","last_change":"2024-01-27T02:37:03.978549+0000","last_active":"2024-01-27T02:37:03.978681+0000","last_peered":"2024-01-27T02:37:03.978681+0000","last_clean":"2024-01-27T02:37:03.978681+0000","last_became_active":"2024-01-27T02:37:03.978158+0000","last_became_peered":"2024-01-27T02:37:03.978158+0000","last_unstale":"2024-01-27T02:37:03.978681+0000","last_undegraded":"2024-01-27T02:37:03.978681+0000","last_fullsized":"2024-01-27T02:37:03.978681+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:51:37.252954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.983434+0000","last_change":"2024-01-27T02:37:03.983239+0000","last_active":"2024-01-27T02:37:03.983434+0000","last_peered":"2024-01-27T02:37:03.983434+0000","last_clean":"2024-01-27T02:37:03.983434+0000","last_became_active":"2024-01-27T02:37:03.982749+0000","last_became_peered":"2024-01-27T02:37:03.982749+0000","last_unstale":"2024-01-27T02:37:03.983434+0000","last_undegraded":"2024-01-27T02:37:03.983434+0000","last_fullsized":"2024-01-27T02:37:03.983434+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:05:34.176684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.981290+0000","last_change":"2024-01-27T02:37:03.979871+0000","last_active":"2024-01-27T02:37:03.981290+0000","last_peered":"2024-01-27T02:37:03.981290+0000","last_clean":"2024-01-27T02:37:03.981290+0000","last_became_active":"2024-01-27T02:37:03.979564+0000","last_became_peered":"2024-01-27T02:37:03.979564+0000","last_unstale":"2024-01-27T02:37:03.981290+0000","last_undegraded":"2024-01-27T02:37:03.981290+0000","last_fullsized":"2024-01-27T02:37:03.981290+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:26:53.954759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.990787+0000","last_change":"2024-01-27T02:37:03.990696+0000","last_active":"2024-01-27T02:37:03.990787+0000","last_peered":"2024-01-27T02:37:03.990787+0000","last_clean":"2024-01-27T02:37:03.990787+0000","last_became_active":"2024-01-27T02:37:03.990447+0000","last_became_peered":"2024-01-27T02:37:03.990447+0000","last_unstale":"2024-01-27T02:37:03.990787+0000","last_undegraded":"2024-01-27T02:37:03.990787+0000","last_fullsized":"2024-01-27T02:37:03.990787+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:54:16.825640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.984123+0000","last_change":"2024-01-27T02:37:03.984059+0000","last_active":"2024-01-27T02:37:03.984123+0000","last_peered":"2024-01-27T02:37:03.984123+0000","last_clean":"2024-01-27T02:37:03.984123+0000","last_became_active":"2024-01-27T02:37:03.983752+0000","last_became_peered":"2024-01-27T02:37:03.983752+0000","last_unstale":"2024-01-27T02:37:03.984123+0000","last_undegraded":"2024-01-27T02:37:03.984123+0000","last_fullsized":"2024-01-27T02:37:03.984123+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:03:33.502042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.976768+0000","last_change":"2024-01-27T02:37:03.975609+0000","last_active":"2024-01-27T02:37:03.976768+0000","last_peered":"2024-01-27T02:37:03.976768+0000","last_clean":"2024-01-27T02:37:03.976768+0000","last_became_active":"2024-01-27T02:37:03.975279+0000","last_became_peered":"2024-01-27T02:37:03.975279+0000","last_unstale":"2024-01-27T02:37:03.976768+0000","last_undegraded":"2024-01-27T02:37:03.976768+0000","last_fullsized":"2024-01-27T02:37:03.976768+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:51:37.252954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.977507+0000","last_change":"2024-01-27T02:37:03.977434+0000","last_active":"2024-01-27T02:37:03.977507+0000","last_peered":"2024-01-27T02:37:03.977507+0000","last_clean":"2024-01-27T02:37:03.977507+0000","last_became_active":"2024-01-27T02:37:03.977131+0000","last_became_peered":"2024-01-27T02:37:03.977131+0000","last_unstale":"2024-01-27T02:37:03.977507+0000","last_undegraded":"2024-01-27T02:37:03.977507+0000","last_fullsized":"2024-01-27T02:37:03.977507+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:14:22.604751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.980680+0000","last_change":"2024-01-27T02:37:03.976569+0000","last_active":"2024-01-27T02:37:03.980680+0000","last_peered":"2024-01-27T02:37:03.980680+0000","last_clean":"2024-01-27T02:37:03.980680+0000","last_became_active":"2024-01-27T02:37:03.975238+0000","last_became_peered":"2024-01-27T02:37:03.975238+0000","last_unstale":"2024-01-27T02:37:03.980680+0000","last_undegraded":"2024-01-27T02:37:03.980680+0000","last_fullsized":"2024-01-27T02:37:03.980680+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:20:09.835171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.979516+0000","last_change":"2024-01-27T02:37:03.979282+0000","last_active":"2024-01-27T02:37:03.979516+0000","last_peered":"2024-01-27T02:37:03.979516+0000","last_clean":"2024-01-27T02:37:03.979516+0000","last_became_active":"2024-01-27T02:37:03.978995+0000","last_became_peered":"2024-01-27T02:37:03.978995+0000","last_unstale":"2024-01-27T02:37:03.979516+0000","last_undegraded":"2024-01-27T02:37:03.979516+0000","last_fullsized":"2024-01-27T02:37:03.979516+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:03:31.433351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.989594+0000","last_change":"2024-01-27T02:37:03.989339+0000","last_active":"2024-01-27T02:37:03.989594+0000","last_peered":"2024-01-27T02:37:03.989594+0000","last_clean":"2024-01-27T02:37:03.989594+0000","last_became_active":"2024-01-27T02:37:03.988740+0000","last_became_peered":"2024-01-27T02:37:03.988740+0000","last_unstale":"2024-01-27T02:37:03.989594+0000","last_undegraded":"2024-01-27T02:37:03.989594+0000","last_fullsized":"2024-01-27T02:37:03.989594+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:08:53.127784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.982593+0000","last_change":"2024-01-27T02:37:03.979903+0000","last_active":"2024-01-27T02:37:03.982593+0000","last_peered":"2024-01-27T02:37:03.982593+0000","last_clean":"2024-01-27T02:37:03.982593+0000","last_became_active":"2024-01-27T02:37:03.979612+0000","last_became_peered":"2024-01-27T02:37:03.979612+0000","last_unstale":"2024-01-27T02:37:03.982593+0000","last_undegraded":"2024-01-27T02:37:03.982593+0000","last_fullsized":"2024-01-27T02:37:03.982593+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:13:12.836181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.984596+0000","last_change":"2024-01-27T02:37:03.984442+0000","last_active":"2024-01-27T02:37:03.984596+0000","last_peered":"2024-01-27T02:37:03.984596+0000","last_clean":"2024-01-27T02:37:03.984596+0000","last_became_active":"2024-01-27T02:37:03.984162+0000","last_became_peered":"2024-01-27T02:37:03.984162+0000","last_unstale":"2024-01-27T02:37:03.984596+0000","last_undegraded":"2024-01-27T02:37:03.984596+0000","last_fullsized":"2024-01-27T02:37:03.984596+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:03:33.502042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.983275+0000","last_change":"2024-01-27T02:37:03.983201+0000","last_active":"2024-01-27T02:37:03.983275+0000","last_peered":"2024-01-27T02:37:03.983275+0000","last_clean":"2024-01-27T02:37:03.983275+0000","last_became_active":"2024-01-27T02:37:03.982475+0000","last_became_peered":"2024-01-27T02:37:03.982475+0000","last_unstale":"2024-01-27T02:37:03.983275+0000","last_undegraded":"2024-01-27T02:37:03.983275+0000","last_fullsized":"2024-01-27T02:37:03.983275+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:20:09.835171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.982611+0000","last_change":"2024-01-27T02:37:03.982505+0000","last_active":"2024-01-27T02:37:03.982611+0000","last_peered":"2024-01-27T02:37:03.982611+0000","last_clean":"2024-01-27T02:37:03.982611+0000","last_became_active":"2024-01-27T02:37:03.982219+0000","last_became_peered":"2024-01-27T02:37:03.982219+0000","last_unstale":"2024-01-27T02:37:03.982611+0000","last_undegraded":"2024-01-27T02:37:03.982611+0000","last_fullsized":"2024-01-27T02:37:03.982611+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:51:20.295257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.991328+0000","last_change":"2024-01-27T02:37:03.991016+0000","last_active":"2024-01-27T02:37:03.991328+0000","last_peered":"2024-01-27T02:37:03.991328+0000","last_clean":"2024-01-27T02:37:03.991328+0000","last_became_active":"2024-01-27T02:37:03.990747+0000","last_became_peered":"2024-01-27T02:37:03.990747+0000","last_unstale":"2024-01-27T02:37:03.991328+0000","last_undegraded":"2024-01-27T02:37:03.991328+0000","last_fullsized":"2024-01-27T02:37:03.991328+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:20:43.448875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.988281+0000","last_change":"2024-01-27T02:37:03.988205+0000","last_active":"2024-01-27T02:37:03.988281+0000","last_peered":"2024-01-27T02:37:03.988281+0000","last_clean":"2024-01-27T02:37:03.988281+0000","last_became_active":"2024-01-27T02:37:03.987971+0000","last_became_peered":"2024-01-27T02:37:03.987971+0000","last_unstale":"2024-01-27T02:37:03.988281+0000","last_undegraded":"2024-01-27T02:37:03.988281+0000","last_fullsized":"2024-01-27T02:37:03.988281+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:37:43.105081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.994255+0000","last_change":"2024-01-27T02:37:03.991935+0000","last_active":"2024-01-27T02:37:03.994255+0000","last_peered":"2024-01-27T02:37:03.994255+0000","last_clean":"2024-01-27T02:37:03.994255+0000","last_became_active":"2024-01-27T02:37:03.989031+0000","last_became_peered":"2024-01-27T02:37:03.989031+0000","last_unstale":"2024-01-27T02:37:03.994255+0000","last_undegraded":"2024-01-27T02:37:03.994255+0000","last_fullsized":"2024-01-27T02:37:03.994255+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:30:14.882894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.989660+0000","last_change":"2024-01-27T02:37:03.989298+0000","last_active":"2024-01-27T02:37:03.989660+0000","last_peered":"2024-01-27T02:37:03.989660+0000","last_clean":"2024-01-27T02:37:03.989660+0000","last_became_active":"2024-01-27T02:37:03.988975+0000","last_became_peered":"2024-01-27T02:37:03.988975+0000","last_unstale":"2024-01-27T02:37:03.989660+0000","last_undegraded":"2024-01-27T02:37:03.989660+0000","last_fullsized":"2024-01-27T02:37:03.989660+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:06:57.129035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.986597+0000","last_change":"2024-01-27T02:37:03.986558+0000","last_active":"2024-01-27T02:37:03.986597+0000","last_peered":"2024-01-27T02:37:03.986597+0000","last_clean":"2024-01-27T02:37:03.986597+0000","last_became_active":"2024-01-27T02:37:03.986189+0000","last_became_peered":"2024-01-27T02:37:03.986189+0000","last_unstale":"2024-01-27T02:37:03.986597+0000","last_undegraded":"2024-01-27T02:37:03.986597+0000","last_fullsized":"2024-01-27T02:37:03.986597+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:20:43.448875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.985981+0000","last_change":"2024-01-27T02:37:03.985904+0000","last_active":"2024-01-27T02:37:03.985981+0000","last_peered":"2024-01-27T02:37:03.985981+0000","last_clean":"2024-01-27T02:37:03.985981+0000","last_became_active":"2024-01-27T02:37:03.985625+0000","last_became_peered":"2024-01-27T02:37:03.985625+0000","last_unstale":"2024-01-27T02:37:03.985981+0000","last_undegraded":"2024-01-27T02:37:03.985981+0000","last_fullsized":"2024-01-27T02:37:03.985981+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:36:37.195219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.973326+0000","last_change":"2024-01-27T02:37:03.973241+0000","last_active":"2024-01-27T02:37:03.973326+0000","last_peered":"2024-01-27T02:37:03.973326+0000","last_clean":"2024-01-27T02:37:03.973326+0000","last_became_active":"2024-01-27T02:37:03.972965+0000","last_became_peered":"2024-01-27T02:37:03.972965+0000","last_unstale":"2024-01-27T02:37:03.973326+0000","last_undegraded":"2024-01-27T02:37:03.973326+0000","last_fullsized":"2024-01-27T02:37:03.973326+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:39:32.384063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.979943+0000","last_change":"2024-01-27T02:37:03.979831+0000","last_active":"2024-01-27T02:37:03.979943+0000","last_peered":"2024-01-27T02:37:03.979943+0000","last_clean":"2024-01-27T02:37:03.979943+0000","last_became_active":"2024-01-27T02:37:03.979515+0000","last_became_peered":"2024-01-27T02:37:03.979515+0000","last_unstale":"2024-01-27T02:37:03.979943+0000","last_undegraded":"2024-01-27T02:37:03.979943+0000","last_fullsized":"2024-01-27T02:37:03.979943+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:51:37.252954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.983501+0000","last_change":"2024-01-27T02:37:03.983376+0000","last_active":"2024-01-27T02:37:03.983501+0000","last_peered":"2024-01-27T02:37:03.983501+0000","last_clean":"2024-01-27T02:37:03.983501+0000","last_became_active":"2024-01-27T02:37:03.982797+0000","last_became_peered":"2024-01-27T02:37:03.982797+0000","last_unstale":"2024-01-27T02:37:03.983501+0000","last_undegraded":"2024-01-27T02:37:03.983501+0000","last_fullsized":"2024-01-27T02:37:03.983501+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:00:23.179794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.981947+0000","last_change":"2024-01-27T02:37:03.981665+0000","last_active":"2024-01-27T02:37:03.981947+0000","last_peered":"2024-01-27T02:37:03.981947+0000","last_clean":"2024-01-27T02:37:03.981947+0000","last_became_active":"2024-01-27T02:37:03.980985+0000","last_became_peered":"2024-01-27T02:37:03.980985+0000","last_unstale":"2024-01-27T02:37:03.981947+0000","last_undegraded":"2024-01-27T02:37:03.981947+0000","last_fullsized":"2024-01-27T02:37:03.981947+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:03:33.502042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.973890+0000","last_change":"2024-01-27T02:37:03.973788+0000","last_active":"2024-01-27T02:37:03.973890+0000","last_peered":"2024-01-27T02:37:03.973890+0000","last_clean":"2024-01-27T02:37:03.973890+0000","last_became_active":"2024-01-27T02:37:03.973469+0000","last_became_peered":"2024-01-27T02:37:03.973469+0000","last_unstale":"2024-01-27T02:37:03.973890+0000","last_undegraded":"2024-01-27T02:37:03.973890+0000","last_fullsized":"2024-01-27T02:37:03.973890+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:37:36.575409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.988812+0000","last_change":"2024-01-27T02:37:03.988684+0000","last_active":"2024-01-27T02:37:03.988812+0000","last_peered":"2024-01-27T02:37:03.988812+0000","last_clean":"2024-01-27T02:37:03.988812+0000","last_became_active":"2024-01-27T02:37:03.988390+0000","last_became_peered":"2024-01-27T02:37:03.988390+0000","last_unstale":"2024-01-27T02:37:03.988812+0000","last_undegraded":"2024-01-27T02:37:03.988812+0000","last_fullsized":"2024-01-27T02:37:03.988812+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:48:58.110238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.990077+0000","last_change":"2024-01-27T02:37:03.990012+0000","last_active":"2024-01-27T02:37:03.990077+0000","last_peered":"2024-01-27T02:37:03.990077+0000","last_clean":"2024-01-27T02:37:03.990077+0000","last_became_active":"2024-01-27T02:37:03.989541+0000","last_became_peered":"2024-01-27T02:37:03.989541+0000","last_unstale":"2024-01-27T02:37:03.990077+0000","last_undegraded":"2024-01-27T02:37:03.990077+0000","last_fullsized":"2024-01-27T02:37:03.990077+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:33:34.067913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.980704+0000","last_change":"2024-01-27T02:37:03.977028+0000","last_active":"2024-01-27T02:37:03.980704+0000","last_peered":"2024-01-27T02:37:03.980704+0000","last_clean":"2024-01-27T02:37:03.980704+0000","last_became_active":"2024-01-27T02:37:03.976168+0000","last_became_peered":"2024-01-27T02:37:03.976168+0000","last_unstale":"2024-01-27T02:37:03.980704+0000","last_undegraded":"2024-01-27T02:37:03.980704+0000","last_fullsized":"2024-01-27T02:37:03.980704+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:49:53.022320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.976382+0000","last_change":"2024-01-27T02:37:03.976297+0000","last_active":"2024-01-27T02:37:03.976382+0000","last_peered":"2024-01-27T02:37:03.976382+0000","last_clean":"2024-01-27T02:37:03.976382+0000","last_became_active":"2024-01-27T02:37:03.976061+0000","last_became_peered":"2024-01-27T02:37:03.976061+0000","last_unstale":"2024-01-27T02:37:03.976382+0000","last_undegraded":"2024-01-27T02:37:03.976382+0000","last_fullsized":"2024-01-27T02:37:03.976382+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:54:16.825640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"97'4","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.995049+0000","last_change":"2024-01-27T02:37:02.985948+0000","last_active":"2024-01-27T02:37:03.995049+0000","last_peered":"2024-01-27T02:37:03.995049+0000","last_clean":"2024-01-27T02:37:03.995049+0000","last_became_active":"2024-01-27T02:37:02.985748+0000","last_became_peered":"2024-01-27T02:37:02.985748+0000","last_unstale":"2024-01-27T02:37:03.995049+0000","last_undegraded":"2024-01-27T02:37:03.995049+0000","last_fullsized":"2024-01-27T02:37:03.995049+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:46:16.101944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.986503+0000","last_change":"2024-01-27T02:37:03.986440+0000","last_active":"2024-01-27T02:37:03.986503+0000","last_peered":"2024-01-27T02:37:03.986503+0000","last_clean":"2024-01-27T02:37:03.986503+0000","last_became_active":"2024-01-27T02:37:03.985869+0000","last_became_peered":"2024-01-27T02:37:03.985869+0000","last_unstale":"2024-01-27T02:37:03.986503+0000","last_undegraded":"2024-01-27T02:37:03.986503+0000","last_fullsized":"2024-01-27T02:37:03.986503+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:05:34.176684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"97'1","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.993544+0000","last_change":"2024-01-27T02:37:02.981569+0000","last_active":"2024-01-27T02:37:03.993544+0000","last_peered":"2024-01-27T02:37:03.993544+0000","last_clean":"2024-01-27T02:37:03.993544+0000","last_became_active":"2024-01-27T02:37:02.981221+0000","last_became_peered":"2024-01-27T02:37:02.981221+0000","last_unstale":"2024-01-27T02:37:03.993544+0000","last_undegraded":"2024-01-27T02:37:03.993544+0000","last_fullsized":"2024-01-27T02:37:03.993544+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:27:26.563071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.975382+0000","last_change":"2024-01-27T02:37:03.975299+0000","last_active":"2024-01-27T02:37:03.975382+0000","last_peered":"2024-01-27T02:37:03.975382+0000","last_clean":"2024-01-27T02:37:03.975382+0000","last_became_active":"2024-01-27T02:37:03.975030+0000","last_became_peered":"2024-01-27T02:37:03.975030+0000","last_unstale":"2024-01-27T02:37:03.975382+0000","last_undegraded":"2024-01-27T02:37:03.975382+0000","last_fullsized":"2024-01-27T02:37:03.975382+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:09:36.161409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"97'2","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.996157+0000","last_change":"2024-01-27T02:37:02.986148+0000","last_active":"2024-01-27T02:37:03.996157+0000","last_peered":"2024-01-27T02:37:03.996157+0000","last_clean":"2024-01-27T02:37:03.996157+0000","last_became_active":"2024-01-27T02:37:02.985884+0000","last_became_peered":"2024-01-27T02:37:02.985884+0000","last_unstale":"2024-01-27T02:37:03.996157+0000","last_undegraded":"2024-01-27T02:37:03.996157+0000","last_fullsized":"2024-01-27T02:37:03.996157+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:23:02.531377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.982707+0000","last_change":"2024-01-27T02:37:03.982229+0000","last_active":"2024-01-27T02:37:03.982707+0000","last_peered":"2024-01-27T02:37:03.982707+0000","last_clean":"2024-01-27T02:37:03.982707+0000","last_became_active":"2024-01-27T02:37:03.981800+0000","last_became_peered":"2024-01-27T02:37:03.981800+0000","last_unstale":"2024-01-27T02:37:03.982707+0000","last_undegraded":"2024-01-27T02:37:03.982707+0000","last_fullsized":"2024-01-27T02:37:03.982707+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:55:29.044564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"97'2","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.995362+0000","last_change":"2024-01-27T02:37:02.986529+0000","last_active":"2024-01-27T02:37:03.995362+0000","last_peered":"2024-01-27T02:37:03.995362+0000","last_clean":"2024-01-27T02:37:03.995362+0000","last_became_active":"2024-01-27T02:37:02.985853+0000","last_became_peered":"2024-01-27T02:37:02.985853+0000","last_unstale":"2024-01-27T02:37:03.995362+0000","last_undegraded":"2024-01-27T02:37:03.995362+0000","last_fullsized":"2024-01-27T02:37:03.995362+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:04:34.145511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.990511+0000","last_change":"2024-01-27T02:37:03.990250+0000","last_active":"2024-01-27T02:37:03.990511+0000","last_peered":"2024-01-27T02:37:03.990511+0000","last_clean":"2024-01-27T02:37:03.990511+0000","last_became_active":"2024-01-27T02:37:03.989804+0000","last_became_peered":"2024-01-27T02:37:03.989804+0000","last_unstale":"2024-01-27T02:37:03.990511+0000","last_undegraded":"2024-01-27T02:37:03.990511+0000","last_fullsized":"2024-01-27T02:37:03.990511+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:21:14.462607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"97'1","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.997194+0000","last_change":"2024-01-27T02:37:02.981692+0000","last_active":"2024-01-27T02:37:03.997194+0000","last_peered":"2024-01-27T02:37:03.997194+0000","last_clean":"2024-01-27T02:37:03.997194+0000","last_became_active":"2024-01-27T02:37:02.981398+0000","last_became_peered":"2024-01-27T02:37:02.981398+0000","last_unstale":"2024-01-27T02:37:03.997194+0000","last_undegraded":"2024-01-27T02:37:03.997194+0000","last_fullsized":"2024-01-27T02:37:03.997194+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:29:25.334034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.983137+0000","last_change":"2024-01-27T02:37:03.983024+0000","last_active":"2024-01-27T02:37:03.983137+0000","last_peered":"2024-01-27T02:37:03.983137+0000","last_clean":"2024-01-27T02:37:03.983137+0000","last_became_active":"2024-01-27T02:37:03.982163+0000","last_became_peered":"2024-01-27T02:37:03.982163+0000","last_unstale":"2024-01-27T02:37:03.983137+0000","last_undegraded":"2024-01-27T02:37:03.983137+0000","last_fullsized":"2024-01-27T02:37:03.983137+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:09:36.161409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"97'1","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.991439+0000","last_change":"2024-01-27T02:37:02.980720+0000","last_active":"2024-01-27T02:37:03.991439+0000","last_peered":"2024-01-27T02:37:03.991439+0000","last_clean":"2024-01-27T02:37:03.991439+0000","last_became_active":"2024-01-27T02:37:02.980470+0000","last_became_peered":"2024-01-27T02:37:02.980470+0000","last_unstale":"2024-01-27T02:37:03.991439+0000","last_undegraded":"2024-01-27T02:37:03.991439+0000","last_fullsized":"2024-01-27T02:37:03.991439+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:39:31.378671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.990132+0000","last_change":"2024-01-27T02:37:03.990050+0000","last_active":"2024-01-27T02:37:03.990132+0000","last_peered":"2024-01-27T02:37:03.990132+0000","last_clean":"2024-01-27T02:37:03.990132+0000","last_became_active":"2024-01-27T02:37:03.989619+0000","last_became_peered":"2024-01-27T02:37:03.989619+0000","last_unstale":"2024-01-27T02:37:03.990132+0000","last_undegraded":"2024-01-27T02:37:03.990132+0000","last_fullsized":"2024-01-27T02:37:03.990132+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:03:58.690531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.981086+0000","last_change":"2024-01-27T02:37:02.983675+0000","last_active":"2024-01-27T02:37:03.981086+0000","last_peered":"2024-01-27T02:37:03.981086+0000","last_clean":"2024-01-27T02:37:03.981086+0000","last_became_active":"2024-01-27T02:37:02.982951+0000","last_became_peered":"2024-01-27T02:37:02.982951+0000","last_unstale":"2024-01-27T02:37:03.981086+0000","last_undegraded":"2024-01-27T02:37:03.981086+0000","last_fullsized":"2024-01-27T02:37:03.981086+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:04.275415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.976196+0000","last_change":"2024-01-27T02:37:03.976131+0000","last_active":"2024-01-27T02:37:03.976196+0000","last_peered":"2024-01-27T02:37:03.976196+0000","last_clean":"2024-01-27T02:37:03.976196+0000","last_became_active":"2024-01-27T02:37:03.975923+0000","last_became_peered":"2024-01-27T02:37:03.975923+0000","last_unstale":"2024-01-27T02:37:03.976196+0000","last_undegraded":"2024-01-27T02:37:03.976196+0000","last_fullsized":"2024-01-27T02:37:03.976196+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:40:38.576965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.971423+0000","last_change":"2024-01-27T02:37:02.979435+0000","last_active":"2024-01-27T02:37:03.971423+0000","last_peered":"2024-01-27T02:37:03.971423+0000","last_clean":"2024-01-27T02:37:03.971423+0000","last_became_active":"2024-01-27T02:37:02.978795+0000","last_became_peered":"2024-01-27T02:37:02.978795+0000","last_unstale":"2024-01-27T02:37:03.971423+0000","last_undegraded":"2024-01-27T02:37:03.971423+0000","last_fullsized":"2024-01-27T02:37:03.971423+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:50:27.998647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.982575+0000","last_change":"2024-01-27T02:37:03.978063+0000","last_active":"2024-01-27T02:37:03.982575+0000","last_peered":"2024-01-27T02:37:03.982575+0000","last_clean":"2024-01-27T02:37:03.982575+0000","last_became_active":"2024-01-27T02:37:03.977541+0000","last_became_peered":"2024-01-27T02:37:03.977541+0000","last_unstale":"2024-01-27T02:37:03.982575+0000","last_undegraded":"2024-01-27T02:37:03.982575+0000","last_fullsized":"2024-01-27T02:37:03.982575+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:25:59.851526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.972208+0000","last_change":"2024-01-27T02:37:02.972626+0000","last_active":"2024-01-27T02:37:03.972208+0000","last_peered":"2024-01-27T02:37:03.972208+0000","last_clean":"2024-01-27T02:37:03.972208+0000","last_became_active":"2024-01-27T02:37:02.972346+0000","last_became_peered":"2024-01-27T02:37:02.972346+0000","last_unstale":"2024-01-27T02:37:03.972208+0000","last_undegraded":"2024-01-27T02:37:03.972208+0000","last_fullsized":"2024-01-27T02:37:03.972208+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:48:57.104846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.983155+0000","last_change":"2024-01-27T02:37:03.983086+0000","last_active":"2024-01-27T02:37:03.983155+0000","last_peered":"2024-01-27T02:37:03.983155+0000","last_clean":"2024-01-27T02:37:03.983155+0000","last_became_active":"2024-01-27T02:37:03.982440+0000","last_became_peered":"2024-01-27T02:37:03.982440+0000","last_unstale":"2024-01-27T02:37:03.983155+0000","last_undegraded":"2024-01-27T02:37:03.983155+0000","last_fullsized":"2024-01-27T02:37:03.983155+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:51:37.252954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.971380+0000","last_change":"2024-01-27T02:37:02.977998+0000","last_active":"2024-01-27T02:37:03.971380+0000","last_peered":"2024-01-27T02:37:03.971380+0000","last_clean":"2024-01-27T02:37:03.971380+0000","last_became_active":"2024-01-27T02:37:02.977510+0000","last_became_peered":"2024-01-27T02:37:02.977510+0000","last_unstale":"2024-01-27T02:37:03.971380+0000","last_undegraded":"2024-01-27T02:37:03.971380+0000","last_fullsized":"2024-01-27T02:37:03.971380+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:06:56.123644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.985252+0000","last_change":"2024-01-27T02:37:03.985066+0000","last_active":"2024-01-27T02:37:03.985252+0000","last_peered":"2024-01-27T02:37:03.985252+0000","last_clean":"2024-01-27T02:37:03.985252+0000","last_became_active":"2024-01-27T02:37:03.984385+0000","last_became_peered":"2024-01-27T02:37:03.984385+0000","last_unstale":"2024-01-27T02:37:03.985252+0000","last_undegraded":"2024-01-27T02:37:03.985252+0000","last_fullsized":"2024-01-27T02:37:03.985252+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:05:34.176684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"97'3","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.997149+0000","last_change":"2024-01-27T02:37:02.981563+0000","last_active":"2024-01-27T02:37:03.997149+0000","last_peered":"2024-01-27T02:37:03.997149+0000","last_clean":"2024-01-27T02:37:03.997149+0000","last_became_active":"2024-01-27T02:37:02.981298+0000","last_became_peered":"2024-01-27T02:37:02.981298+0000","last_unstale":"2024-01-27T02:37:03.997149+0000","last_undegraded":"2024-01-27T02:37:03.997149+0000","last_fullsized":"2024-01-27T02:37:03.997149+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:48:16.961451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.989027+0000","last_change":"2024-01-27T02:37:03.988969+0000","last_active":"2024-01-27T02:37:03.989027+0000","last_peered":"2024-01-27T02:37:03.989027+0000","last_clean":"2024-01-27T02:37:03.989027+0000","last_became_active":"2024-01-27T02:37:03.988696+0000","last_became_peered":"2024-01-27T02:37:03.988696+0000","last_unstale":"2024-01-27T02:37:03.989027+0000","last_undegraded":"2024-01-27T02:37:03.989027+0000","last_fullsized":"2024-01-27T02:37:03.989027+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:57:41.192776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"97'6","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.998459+0000","last_change":"2024-01-27T02:37:02.981337+0000","last_active":"2024-01-27T02:37:03.998459+0000","last_peered":"2024-01-27T02:37:03.998459+0000","last_clean":"2024-01-27T02:37:03.998459+0000","last_became_active":"2024-01-27T02:37:02.981101+0000","last_became_peered":"2024-01-27T02:37:02.981101+0000","last_unstale":"2024-01-27T02:37:03.998459+0000","last_undegraded":"2024-01-27T02:37:03.998459+0000","last_fullsized":"2024-01-27T02:37:03.998459+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:35:02.399898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.986735+0000","last_change":"2024-01-27T02:37:03.986628+0000","last_active":"2024-01-27T02:37:03.986735+0000","last_peered":"2024-01-27T02:37:03.986735+0000","last_clean":"2024-01-27T02:37:03.986735+0000","last_became_active":"2024-01-27T02:37:03.986106+0000","last_became_peered":"2024-01-27T02:37:03.986106+0000","last_unstale":"2024-01-27T02:37:03.986735+0000","last_undegraded":"2024-01-27T02:37:03.986735+0000","last_fullsized":"2024-01-27T02:37:03.986735+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:03:33.502042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"97'2","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.996625+0000","last_change":"2024-01-27T02:37:02.978230+0000","last_active":"2024-01-27T02:37:03.996625+0000","last_peered":"2024-01-27T02:37:03.996625+0000","last_clean":"2024-01-27T02:37:03.996625+0000","last_became_active":"2024-01-27T02:37:02.977954+0000","last_became_peered":"2024-01-27T02:37:02.977954+0000","last_unstale":"2024-01-27T02:37:03.996625+0000","last_undegraded":"2024-01-27T02:37:03.996625+0000","last_fullsized":"2024-01-27T02:37:03.996625+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:13:17.931874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.989987+0000","last_change":"2024-01-27T02:37:03.989900+0000","last_active":"2024-01-27T02:37:03.989987+0000","last_peered":"2024-01-27T02:37:03.989987+0000","last_clean":"2024-01-27T02:37:03.989987+0000","last_became_active":"2024-01-27T02:37:03.989225+0000","last_became_peered":"2024-01-27T02:37:03.989225+0000","last_unstale":"2024-01-27T02:37:03.989987+0000","last_undegraded":"2024-01-27T02:37:03.989987+0000","last_fullsized":"2024-01-27T02:37:03.989987+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:11:08.055611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.969981+0000","last_change":"2024-01-27T02:37:02.978205+0000","last_active":"2024-01-27T02:37:03.969981+0000","last_peered":"2024-01-27T02:37:03.969981+0000","last_clean":"2024-01-27T02:37:03.969981+0000","last_became_active":"2024-01-27T02:37:02.977960+0000","last_became_peered":"2024-01-27T02:37:02.977960+0000","last_unstale":"2024-01-27T02:37:03.969981+0000","last_undegraded":"2024-01-27T02:37:03.969981+0000","last_fullsized":"2024-01-27T02:37:03.969981+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:08:02.593649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.990489+0000","last_change":"2024-01-27T02:37:03.990080+0000","last_active":"2024-01-27T02:37:03.990489+0000","last_peered":"2024-01-27T02:37:03.990489+0000","last_clean":"2024-01-27T02:37:03.990489+0000","last_became_active":"2024-01-27T02:37:03.988096+0000","last_became_peered":"2024-01-27T02:37:03.988096+0000","last_unstale":"2024-01-27T02:37:03.990489+0000","last_undegraded":"2024-01-27T02:37:03.990489+0000","last_fullsized":"2024-01-27T02:37:03.990489+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:29:26.339425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.976556+0000","last_change":"2024-01-27T02:37:02.979846+0000","last_active":"2024-01-27T02:37:03.976556+0000","last_peered":"2024-01-27T02:37:03.976556+0000","last_clean":"2024-01-27T02:37:03.976556+0000","last_became_active":"2024-01-27T02:37:02.979575+0000","last_became_peered":"2024-01-27T02:37:02.979575+0000","last_unstale":"2024-01-27T02:37:03.976556+0000","last_undegraded":"2024-01-27T02:37:03.976556+0000","last_fullsized":"2024-01-27T02:37:03.976556+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:14:50.580128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.982653+0000","last_change":"2024-01-27T02:37:03.978764+0000","last_active":"2024-01-27T02:37:03.982653+0000","last_peered":"2024-01-27T02:37:03.982653+0000","last_clean":"2024-01-27T02:37:03.982653+0000","last_became_active":"2024-01-27T02:37:03.978430+0000","last_became_peered":"2024-01-27T02:37:03.978430+0000","last_unstale":"2024-01-27T02:37:03.982653+0000","last_undegraded":"2024-01-27T02:37:03.982653+0000","last_fullsized":"2024-01-27T02:37:03.982653+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:27:27.568462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"97'8","reported_seq":18,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:04.001401+0000","last_change":"2024-01-27T02:37:02.980162+0000","last_active":"2024-01-27T02:37:04.001401+0000","last_peered":"2024-01-27T02:37:04.001401+0000","last_clean":"2024-01-27T02:37:04.001401+0000","last_became_active":"2024-01-27T02:37:02.979918+0000","last_became_peered":"2024-01-27T02:37:02.979918+0000","last_unstale":"2024-01-27T02:37:04.001401+0000","last_undegraded":"2024-01-27T02:37:04.001401+0000","last_fullsized":"2024-01-27T02:37:04.001401+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:16:11.230822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.984790+0000","last_change":"2024-01-27T02:37:03.984645+0000","last_active":"2024-01-27T02:37:03.984790+0000","last_peered":"2024-01-27T02:37:03.984790+0000","last_clean":"2024-01-27T02:37:03.984790+0000","last_became_active":"2024-01-27T02:37:03.984412+0000","last_became_peered":"2024-01-27T02:37:03.984412+0000","last_unstale":"2024-01-27T02:37:03.984790+0000","last_undegraded":"2024-01-27T02:37:03.984790+0000","last_fullsized":"2024-01-27T02:37:03.984790+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:19:18.604178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"97'1","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.989113+0000","last_change":"2024-01-27T02:37:02.976792+0000","last_active":"2024-01-27T02:37:03.989113+0000","last_peered":"2024-01-27T02:37:03.989113+0000","last_clean":"2024-01-27T02:37:03.989113+0000","last_became_active":"2024-01-27T02:37:02.976590+0000","last_became_peered":"2024-01-27T02:37:02.976590+0000","last_unstale":"2024-01-27T02:37:03.989113+0000","last_undegraded":"2024-01-27T02:37:03.989113+0000","last_fullsized":"2024-01-27T02:37:03.989113+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:28:59.567547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.980723+0000","last_change":"2024-01-27T02:37:03.978614+0000","last_active":"2024-01-27T02:37:03.980723+0000","last_peered":"2024-01-27T02:37:03.980723+0000","last_clean":"2024-01-27T02:37:03.980723+0000","last_became_active":"2024-01-27T02:37:03.978100+0000","last_became_peered":"2024-01-27T02:37:03.978100+0000","last_unstale":"2024-01-27T02:37:03.980723+0000","last_undegraded":"2024-01-27T02:37:03.980723+0000","last_fullsized":"2024-01-27T02:37:03.980723+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:54:16.825640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"97'2","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.996263+0000","last_change":"2024-01-27T02:37:02.979310+0000","last_active":"2024-01-27T02:37:03.996263+0000","last_peered":"2024-01-27T02:37:03.996263+0000","last_clean":"2024-01-27T02:37:03.996263+0000","last_became_active":"2024-01-27T02:37:02.979006+0000","last_became_peered":"2024-01-27T02:37:02.979006+0000","last_unstale":"2024-01-27T02:37:03.996263+0000","last_undegraded":"2024-01-27T02:37:03.996263+0000","last_fullsized":"2024-01-27T02:37:03.996263+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:11:18.375438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.988332+0000","last_change":"2024-01-27T02:37:03.988214+0000","last_active":"2024-01-27T02:37:03.988332+0000","last_peered":"2024-01-27T02:37:03.988332+0000","last_clean":"2024-01-27T02:37:03.988332+0000","last_became_active":"2024-01-27T02:37:03.987991+0000","last_became_peered":"2024-01-27T02:37:03.987991+0000","last_unstale":"2024-01-27T02:37:03.988332+0000","last_undegraded":"2024-01-27T02:37:03.988332+0000","last_fullsized":"2024-01-27T02:37:03.988332+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:23:03.536769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.979007+0000","last_change":"2024-01-27T02:37:02.977693+0000","last_active":"2024-01-27T02:37:03.979007+0000","last_peered":"2024-01-27T02:37:03.979007+0000","last_clean":"2024-01-27T02:37:03.979007+0000","last_became_active":"2024-01-27T02:37:02.977381+0000","last_became_peered":"2024-01-27T02:37:02.977381+0000","last_unstale":"2024-01-27T02:37:03.979007+0000","last_undegraded":"2024-01-27T02:37:03.979007+0000","last_fullsized":"2024-01-27T02:37:03.979007+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:14:50.580128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.983247+0000","last_change":"2024-01-27T02:37:03.982932+0000","last_active":"2024-01-27T02:37:03.983247+0000","last_peered":"2024-01-27T02:37:03.983247+0000","last_clean":"2024-01-27T02:37:03.983247+0000","last_became_active":"2024-01-27T02:37:03.982683+0000","last_became_peered":"2024-01-27T02:37:03.982683+0000","last_unstale":"2024-01-27T02:37:03.983247+0000","last_undegraded":"2024-01-27T02:37:03.983247+0000","last_fullsized":"2024-01-27T02:37:03.983247+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:19:53.912030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"97'2","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.995276+0000","last_change":"2024-01-27T02:37:02.987475+0000","last_active":"2024-01-27T02:37:03.995276+0000","last_peered":"2024-01-27T02:37:03.995276+0000","last_clean":"2024-01-27T02:37:03.995276+0000","last_became_active":"2024-01-27T02:37:02.987322+0000","last_became_peered":"2024-01-27T02:37:02.987322+0000","last_unstale":"2024-01-27T02:37:03.995276+0000","last_undegraded":"2024-01-27T02:37:03.995276+0000","last_fullsized":"2024-01-27T02:37:03.995276+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:19:46.184238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.976434+0000","last_change":"2024-01-27T02:37:03.976154+0000","last_active":"2024-01-27T02:37:03.976434+0000","last_peered":"2024-01-27T02:37:03.976434+0000","last_clean":"2024-01-27T02:37:03.976434+0000","last_became_active":"2024-01-27T02:37:03.975889+0000","last_became_peered":"2024-01-27T02:37:03.975889+0000","last_unstale":"2024-01-27T02:37:03.976434+0000","last_undegraded":"2024-01-27T02:37:03.976434+0000","last_fullsized":"2024-01-27T02:37:03.976434+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:20:09.835171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.975591+0000","last_change":"2024-01-27T02:37:02.985913+0000","last_active":"2024-01-27T02:37:03.975591+0000","last_peered":"2024-01-27T02:37:03.975591+0000","last_clean":"2024-01-27T02:37:03.975591+0000","last_became_active":"2024-01-27T02:37:02.985569+0000","last_became_peered":"2024-01-27T02:37:02.985569+0000","last_unstale":"2024-01-27T02:37:03.975591+0000","last_undegraded":"2024-01-27T02:37:03.975591+0000","last_fullsized":"2024-01-27T02:37:03.975591+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:55:50.883094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.994201+0000","last_change":"2024-01-27T02:37:03.991778+0000","last_active":"2024-01-27T02:37:03.994201+0000","last_peered":"2024-01-27T02:37:03.994201+0000","last_clean":"2024-01-27T02:37:03.994201+0000","last_became_active":"2024-01-27T02:37:03.984102+0000","last_became_peered":"2024-01-27T02:37:03.984102+0000","last_unstale":"2024-01-27T02:37:03.994201+0000","last_undegraded":"2024-01-27T02:37:03.994201+0000","last_fullsized":"2024-01-27T02:37:03.994201+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:02:05.606454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.976520+0000","last_change":"2024-01-27T02:37:02.985405+0000","last_active":"2024-01-27T02:37:03.976520+0000","last_peered":"2024-01-27T02:37:03.976520+0000","last_clean":"2024-01-27T02:37:03.976520+0000","last_became_active":"2024-01-27T02:37:02.985122+0000","last_became_peered":"2024-01-27T02:37:02.985122+0000","last_unstale":"2024-01-27T02:37:03.976520+0000","last_undegraded":"2024-01-27T02:37:03.976520+0000","last_fullsized":"2024-01-27T02:37:03.976520+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:40:22.882301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.988197+0000","last_change":"2024-01-27T02:37:03.988116+0000","last_active":"2024-01-27T02:37:03.988197+0000","last_peered":"2024-01-27T02:37:03.988197+0000","last_clean":"2024-01-27T02:37:03.988197+0000","last_became_active":"2024-01-27T02:37:03.987868+0000","last_became_peered":"2024-01-27T02:37:03.987868+0000","last_unstale":"2024-01-27T02:37:03.988197+0000","last_undegraded":"2024-01-27T02:37:03.988197+0000","last_fullsized":"2024-01-27T02:37:03.988197+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:09:36.161409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.970170+0000","last_change":"2024-01-27T02:37:02.986387+0000","last_active":"2024-01-27T02:37:03.970170+0000","last_peered":"2024-01-27T02:37:03.970170+0000","last_clean":"2024-01-27T02:37:03.970170+0000","last_became_active":"2024-01-27T02:37:02.986149+0000","last_became_peered":"2024-01-27T02:37:02.986149+0000","last_unstale":"2024-01-27T02:37:03.970170+0000","last_undegraded":"2024-01-27T02:37:03.970170+0000","last_fullsized":"2024-01-27T02:37:03.970170+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:11:07.050220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.984613+0000","last_change":"2024-01-27T02:37:03.984508+0000","last_active":"2024-01-27T02:37:03.984613+0000","last_peered":"2024-01-27T02:37:03.984613+0000","last_clean":"2024-01-27T02:37:03.984613+0000","last_became_active":"2024-01-27T02:37:03.984271+0000","last_became_peered":"2024-01-27T02:37:03.984271+0000","last_unstale":"2024-01-27T02:37:03.984613+0000","last_undegraded":"2024-01-27T02:37:03.984613+0000","last_fullsized":"2024-01-27T02:37:03.984613+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:11:19.380830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"97'2","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.996746+0000","last_change":"2024-01-27T02:37:02.982134+0000","last_active":"2024-01-27T02:37:03.996746+0000","last_peered":"2024-01-27T02:37:03.996746+0000","last_clean":"2024-01-27T02:37:03.996746+0000","last_became_active":"2024-01-27T02:37:02.981540+0000","last_became_peered":"2024-01-27T02:37:02.981540+0000","last_unstale":"2024-01-27T02:37:03.996746+0000","last_undegraded":"2024-01-27T02:37:03.996746+0000","last_fullsized":"2024-01-27T02:37:03.996746+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:14:59.963707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.977931+0000","last_change":"2024-01-27T02:37:03.977858+0000","last_active":"2024-01-27T02:37:03.977931+0000","last_peered":"2024-01-27T02:37:03.977931+0000","last_clean":"2024-01-27T02:37:03.977931+0000","last_became_active":"2024-01-27T02:37:03.977564+0000","last_became_peered":"2024-01-27T02:37:03.977564+0000","last_unstale":"2024-01-27T02:37:03.977931+0000","last_undegraded":"2024-01-27T02:37:03.977931+0000","last_fullsized":"2024-01-27T02:37:03.977931+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:19:47.189629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.971852+0000","last_change":"2024-01-27T02:37:02.979350+0000","last_active":"2024-01-27T02:37:03.971852+0000","last_peered":"2024-01-27T02:37:03.971852+0000","last_clean":"2024-01-27T02:37:03.971852+0000","last_became_active":"2024-01-27T02:37:02.979028+0000","last_became_peered":"2024-01-27T02:37:02.979028+0000","last_unstale":"2024-01-27T02:37:03.971852+0000","last_undegraded":"2024-01-27T02:37:03.971852+0000","last_fullsized":"2024-01-27T02:37:03.971852+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:29:41.047727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.980741+0000","last_change":"2024-01-27T02:37:03.979467+0000","last_active":"2024-01-27T02:37:03.980741+0000","last_peered":"2024-01-27T02:37:03.980741+0000","last_clean":"2024-01-27T02:37:03.980741+0000","last_became_active":"2024-01-27T02:37:03.979218+0000","last_became_peered":"2024-01-27T02:37:03.979218+0000","last_unstale":"2024-01-27T02:37:03.980741+0000","last_undegraded":"2024-01-27T02:37:03.980741+0000","last_fullsized":"2024-01-27T02:37:03.980741+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:11:08.055611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.971611+0000","last_change":"2024-01-27T02:37:02.987022+0000","last_active":"2024-01-27T02:37:03.971611+0000","last_peered":"2024-01-27T02:37:03.971611+0000","last_clean":"2024-01-27T02:37:03.971611+0000","last_became_active":"2024-01-27T02:37:02.986813+0000","last_became_peered":"2024-01-27T02:37:02.986813+0000","last_unstale":"2024-01-27T02:37:03.971611+0000","last_undegraded":"2024-01-27T02:37:03.971611+0000","last_fullsized":"2024-01-27T02:37:03.971611+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:31:48.055150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.983979+0000","last_change":"2024-01-27T02:37:03.983901+0000","last_active":"2024-01-27T02:37:03.983979+0000","last_peered":"2024-01-27T02:37:03.983979+0000","last_clean":"2024-01-27T02:37:03.983979+0000","last_became_active":"2024-01-27T02:37:03.983655+0000","last_became_peered":"2024-01-27T02:37:03.983655+0000","last_unstale":"2024-01-27T02:37:03.983979+0000","last_undegraded":"2024-01-27T02:37:03.983979+0000","last_fullsized":"2024-01-27T02:37:03.983979+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:05:34.176684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.972141+0000","last_change":"2024-01-27T02:37:02.972564+0000","last_active":"2024-01-27T02:37:03.972141+0000","last_peered":"2024-01-27T02:37:03.972141+0000","last_clean":"2024-01-27T02:37:03.972141+0000","last_became_active":"2024-01-27T02:37:02.972302+0000","last_became_peered":"2024-01-27T02:37:02.972302+0000","last_unstale":"2024-01-27T02:37:03.972141+0000","last_undegraded":"2024-01-27T02:37:03.972141+0000","last_fullsized":"2024-01-27T02:37:03.972141+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:21:13.457215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.982259+0000","last_change":"2024-01-27T02:37:03.980359+0000","last_active":"2024-01-27T02:37:03.982259+0000","last_peered":"2024-01-27T02:37:03.982259+0000","last_clean":"2024-01-27T02:37:03.982259+0000","last_became_active":"2024-01-27T02:37:03.976853+0000","last_became_peered":"2024-01-27T02:37:03.976853+0000","last_unstale":"2024-01-27T02:37:03.982259+0000","last_undegraded":"2024-01-27T02:37:03.982259+0000","last_fullsized":"2024-01-27T02:37:03.982259+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:57:41.192776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.980358+0000","last_change":"2024-01-27T02:37:02.978741+0000","last_active":"2024-01-27T02:37:03.980358+0000","last_peered":"2024-01-27T02:37:03.980358+0000","last_clean":"2024-01-27T02:37:03.980358+0000","last_became_active":"2024-01-27T02:37:02.978478+0000","last_became_peered":"2024-01-27T02:37:02.978478+0000","last_unstale":"2024-01-27T02:37:03.980358+0000","last_undegraded":"2024-01-27T02:37:03.980358+0000","last_fullsized":"2024-01-27T02:37:03.980358+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:28:16.622480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.986547+0000","last_change":"2024-01-27T02:37:03.986463+0000","last_active":"2024-01-27T02:37:03.986547+0000","last_peered":"2024-01-27T02:37:03.986547+0000","last_clean":"2024-01-27T02:37:03.986547+0000","last_became_active":"2024-01-27T02:37:03.985851+0000","last_became_peered":"2024-01-27T02:37:03.985851+0000","last_unstale":"2024-01-27T02:37:03.986547+0000","last_undegraded":"2024-01-27T02:37:03.986547+0000","last_fullsized":"2024-01-27T02:37:03.986547+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:09:36.161409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.977915+0000","last_change":"2024-01-27T02:37:02.978252+0000","last_active":"2024-01-27T02:37:03.977915+0000","last_peered":"2024-01-27T02:37:03.977915+0000","last_clean":"2024-01-27T02:37:03.977915+0000","last_became_active":"2024-01-27T02:37:02.977491+0000","last_became_peered":"2024-01-27T02:37:02.977491+0000","last_unstale":"2024-01-27T02:37:03.977915+0000","last_undegraded":"2024-01-27T02:37:03.977915+0000","last_fullsized":"2024-01-27T02:37:03.977915+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:06:56.123644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.979536+0000","last_change":"2024-01-27T02:37:03.979433+0000","last_active":"2024-01-27T02:37:03.979536+0000","last_peered":"2024-01-27T02:37:03.979536+0000","last_clean":"2024-01-27T02:37:03.979536+0000","last_became_active":"2024-01-27T02:37:03.979076+0000","last_became_peered":"2024-01-27T02:37:03.979076+0000","last_unstale":"2024-01-27T02:37:03.979536+0000","last_undegraded":"2024-01-27T02:37:03.979536+0000","last_fullsized":"2024-01-27T02:37:03.979536+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:05:34.176684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.972474+0000","last_change":"2024-01-27T02:37:02.983565+0000","last_active":"2024-01-27T02:37:03.972474+0000","last_peered":"2024-01-27T02:37:03.972474+0000","last_clean":"2024-01-27T02:37:03.972474+0000","last_became_active":"2024-01-27T02:37:02.983313+0000","last_became_peered":"2024-01-27T02:37:02.983313+0000","last_unstale":"2024-01-27T02:37:03.972474+0000","last_undegraded":"2024-01-27T02:37:03.972474+0000","last_fullsized":"2024-01-27T02:37:03.972474+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:29:25.334034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.981962+0000","last_change":"2024-01-27T02:37:03.981805+0000","last_active":"2024-01-27T02:37:03.981962+0000","last_peered":"2024-01-27T02:37:03.981962+0000","last_clean":"2024-01-27T02:37:03.981962+0000","last_became_active":"2024-01-27T02:37:03.981416+0000","last_became_peered":"2024-01-27T02:37:03.981416+0000","last_unstale":"2024-01-27T02:37:03.981962+0000","last_undegraded":"2024-01-27T02:37:03.981962+0000","last_fullsized":"2024-01-27T02:37:03.981962+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:08:03.599041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.974452+0000","last_change":"2024-01-27T02:37:02.978260+0000","last_active":"2024-01-27T02:37:03.974452+0000","last_peered":"2024-01-27T02:37:03.974452+0000","last_clean":"2024-01-27T02:37:03.974452+0000","last_became_active":"2024-01-27T02:37:02.978021+0000","last_became_peered":"2024-01-27T02:37:02.978021+0000","last_unstale":"2024-01-27T02:37:03.974452+0000","last_undegraded":"2024-01-27T02:37:03.974452+0000","last_fullsized":"2024-01-27T02:37:03.974452+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:48:16.961451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.982724+0000","last_change":"2024-01-27T02:37:03.982160+0000","last_active":"2024-01-27T02:37:03.982724+0000","last_peered":"2024-01-27T02:37:03.982724+0000","last_clean":"2024-01-27T02:37:03.982724+0000","last_became_active":"2024-01-27T02:37:03.981913+0000","last_became_peered":"2024-01-27T02:37:03.981913+0000","last_unstale":"2024-01-27T02:37:03.982724+0000","last_undegraded":"2024-01-27T02:37:03.982724+0000","last_fullsized":"2024-01-27T02:37:03.982724+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:02.961706+0000","last_clean_scrub_stamp":"2024-01-27T02:37:02.961706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:35:18.731072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"97'1","reported_seq":11,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.997265+0000","last_change":"2024-01-27T02:37:02.980076+0000","last_active":"2024-01-27T02:37:03.997265+0000","last_peered":"2024-01-27T02:37:03.997265+0000","last_clean":"2024-01-27T02:37:03.997265+0000","last_became_active":"2024-01-27T02:37:02.979839+0000","last_became_peered":"2024-01-27T02:37:02.979839+0000","last_unstale":"2024-01-27T02:37:03.997265+0000","last_undegraded":"2024-01-27T02:37:03.997265+0000","last_fullsized":"2024-01-27T02:37:03.997265+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:01.956314+0000","last_clean_scrub_stamp":"2024-01-27T02:37:01.956314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:04:34.145511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":230,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-01-27T02:37:03.974931+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:37:03.974931+0000","last_peered":"2024-01-27T02:37:03.974931+0000","last_clean":"2024-01-27T02:37:03.974931+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:37:03.974931+0000","last_undegraded":"2024-01-27T02:37:03.974931+0000","last_fullsized":"2024-01-27T02:37:03.974931+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1089,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194223,"statfs":{"total":200000000,"available":198884758,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.45000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71599999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.46800000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.49099999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.435,"5min":0.373,"15min":0.373},"max":{"1min":0.71899999999999997,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.438,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.70599999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.53300000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.45800000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.72299999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.41499999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70499999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.65300000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.73499999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.72499999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.502}]},{"osd":4,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48299999999999998,"15min":0.48999999999999999},"min":{"1min":0.29299999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66500000000000004,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.504,"15min":0.51600000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73399999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.54600000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.52300000000000002,"15min":0.53700000000000003},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.72099999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55400000000000005,"15min":0.56499999999999995},"min":{"1min":0.38700000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73499999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.58599999999999997}]},{"osd":6,"last update":"Sat Jan 27 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.58399999999999996,"15min":0.59799999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73199999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.629},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56799999999999995,"15min":0.57799999999999996},"min":{"1min":0.40500000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76700000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":513,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194799,"statfs":{"total":200000000,"available":199474530,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58699999999999997,"15min":0.52100000000000002},"min":{"1min":0.45000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.52700000000000002},"min":{"1min":0.434,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71899999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.66300000000000003}]},{"osd":1,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.51800000000000002},"min":{"1min":0.41999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79700000000000004,"5min":0.874,"15min":0.874},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.60599999999999998,"15min":0.55500000000000005},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.68500000000000005}]},{"osd":2,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.41199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.625,"15min":0.58299999999999996},"min":{"1min":0.45100000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79700000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.64900000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58199999999999996,"15min":0.55100000000000005},"min":{"1min":0.47599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.76800000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61499999999999999,"15min":0.57299999999999995},"min":{"1min":0.48499999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.78100000000000003,"5min":1.262,"15min":1.262},"last":0.57999999999999996}]},{"osd":4,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.34200000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52700000000000002,"15min":0.55300000000000005},"min":{"1min":0.41199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68300000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.41199999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53200000000000003,"15min":0.53500000000000003},"min":{"1min":0.38800000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70799999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.60599999999999998},"min":{"1min":0.40899999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70799999999999996,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.47399999999999998}]},{"osd":7,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60599999999999998,"15min":0.61599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73699999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71099999999999997,"5min":1.038,"15min":1.038},"last":0.441}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":512,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194799,"statfs":{"total":200000000,"available":199475002,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.38600000000000001,"15min":0.373},"min":{"1min":0.307,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.55900000000000005,"5min":0.626,"15min":0.626},"last":0.435},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40699999999999997,"15min":0.39500000000000002},"min":{"1min":0.28699999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.57099999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.39800000000000002}]},{"osd":2,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.36099999999999999},"min":{"1min":0.29599999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.60099999999999998,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.442,"15min":0.42699999999999999},"min":{"1min":0.35499999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65900000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.51600000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48099999999999998,"15min":0.46600000000000003},"min":{"1min":0.36599999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.501,"5min":0.46300000000000002,"15min":0.45400000000000001},"min":{"1min":0.34399999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.68700000000000006,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.47999999999999998}]},{"osd":4,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.63200000000000001,"15min":0.624},"min":{"1min":0.48599999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.96799999999999997,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.64600000000000002,"15min":0.63200000000000001},"min":{"1min":0.51700000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88800000000000001,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.88900000000000001}]},{"osd":5,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63800000000000001,"15min":0.622},"min":{"1min":0.47499999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.93400000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.66300000000000003,"15min":0.66700000000000004},"min":{"1min":0.47299999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.81899999999999995}]},{"osd":6,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.70299999999999996,"15min":0.69399999999999995},"min":{"1min":0.55100000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.95699999999999996,"5min":1.024,"15min":1.024},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.68400000000000005,"15min":0.67000000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.90300000000000002,"5min":1.151,"15min":1.151},"last":0.72799999999999998}]},{"osd":7,"last update":"Sat Jan 27 02:36:03 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.69799999999999995,"15min":0.67800000000000005},"min":{"1min":0.495,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88,"5min":1.097,"15min":1.097},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.70299999999999996,"15min":0.68500000000000005},"min":{"1min":0.54600000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.86799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1089,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194223,"statfs":{"total":200000000,"available":198884794,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.38900000000000001},"min":{"1min":0.28100000000000003,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.66600000000000004,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27800000000000002,"5min":0.223,"15min":0.223},"max":{"1min":0.65200000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.38700000000000001}]},{"osd":2,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.436,"15min":0.42199999999999999},"min":{"1min":0.32800000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56299999999999994,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.44900000000000001,"15min":0.435},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58999999999999997,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.46600000000000003}]},{"osd":3,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42999999999999999,"15min":0.432},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.63600000000000001,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.443},{"interface":"front","average":{"1min":0.502,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.35299999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.63200000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.54900000000000004}]},{"osd":4,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.65500000000000003,"15min":0.64000000000000001},"min":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45900000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89700000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.60199999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67500000000000004,"15min":0.65600000000000003},"min":{"1min":0.48399999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.86499999999999999,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.62},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67700000000000005,"15min":0.68200000000000005},"min":{"1min":0.5,"5min":0.44,"15min":0.44},"max":{"1min":0.96599999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.66700000000000004}]},{"osd":6,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.56200000000000006,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91300000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.754},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69299999999999995,"15min":0.67900000000000005},"min":{"1min":0.56599999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94199999999999995,"5min":1.04,"15min":1.04},"last":0.77000000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.55300000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.52100000000000002,"5min":0.498,"15min":0.498},"max":{"1min":0.95799999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.79300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":513,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194799,"statfs":{"total":200000000,"available":199474642,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.41699999999999998,"15min":0.40999999999999998},"min":{"1min":0.28000000000000003,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56599999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.50900000000000001,"15min":0.63700000000000001},"min":{"1min":0.34499999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72799999999999998,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":0.438}]},{"osd":1,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.434,"15min":0.42399999999999999},"min":{"1min":0.32700000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.44,"15min":0.44700000000000001},"min":{"1min":0.29799999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.65300000000000002,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.39800000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.41499999999999998},"min":{"1min":0.30399999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.65000000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76500000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.372}]},{"osd":4,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68200000000000005,"15min":0.65600000000000003},"min":{"1min":0.53600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73399999999999999,"15min":0.79900000000000004},"min":{"1min":0.54900000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.995,"5min":3.581,"15min":3.581},"last":0.55000000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68000000000000005,"15min":0.74399999999999999},"min":{"1min":0.502,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90500000000000003,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67700000000000005,"15min":0.67100000000000004},"min":{"1min":0.501,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85899999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.54000000000000004}]},{"osd":6,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.53800000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93999999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0189999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.52600000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.53600000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70299999999999996,"15min":0.69099999999999995},"min":{"1min":0.625,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.44700000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":514,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194798,"statfs":{"total":200000000,"available":199473660,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.38800000000000001,"15min":0.35899999999999999},"min":{"1min":0.255,"5min":0.253,"15min":0.253},"max":{"1min":0.56200000000000006,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40600000000000003,"15min":0.39500000000000002},"min":{"1min":0.29199999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.626,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.30399999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.27400000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66300000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.45600000000000002,"15min":0.42399999999999999},"min":{"1min":0.30199999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.85099999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46100000000000002,"15min":0.43099999999999999},"min":{"1min":0.33700000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.48399999999999999,"15min":0.45000000000000001},"min":{"1min":0.378,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.503}]},{"osd":4,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.66900000000000004,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.6660000000000004,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64200000000000002,"15min":0.60599999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93300000000000005,"5min":1.012,"15min":1.012},"last":0.67400000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.61299999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90400000000000003,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.71599999999999997,"15min":0.68000000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":4.6849999999999996,"5min":4.6849999999999996,"15min":4.6849999999999996},"last":0.72699999999999998}]},{"osd":6,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90100000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.47099999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.61299999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.71699999999999997,"15min":0.65400000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":4.7130000000000001,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68500000000000005,"15min":0.66700000000000004},"min":{"1min":0.495,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.71299999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":512,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194799,"statfs":{"total":200000000,"available":199475196,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.47299999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.73499999999999999,"5min":1.099,"15min":1.099},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.45000000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.73499999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.51000000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.48999999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0309999999999999,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.496},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81999999999999995,"5min":1.133,"15min":1.133},"last":0.60299999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60999999999999999,"15min":0.59699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.3400000000000001,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.46400000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79800000000000004,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.65200000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.47699999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.44900000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82099999999999995,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.53000000000000003}]},{"osd":5,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.33700000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32800000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.42899999999999999}]},{"osd":6,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.501,"15min":0.499},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68500000000000005,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38600000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.39700000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54600000000000004,"15min":0.55200000000000005},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79000000000000004,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.45400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":514,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194798,"statfs":{"total":200000000,"available":199473638,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56999999999999995,"15min":0.59399999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68700000000000006,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.60099999999999998,"15min":0.57699999999999996},"min":{"1min":0.34499999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67100000000000004,"5min":3.242,"15min":3.242},"last":0.61099999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.625,"15min":0.60799999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70599999999999996,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.627},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.626,"15min":0.64900000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69299999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.67600000000000005}]},{"osd":2,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61599999999999999,"15min":0.59599999999999997},"min":{"1min":0.39100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.71499999999999997,"5min":3.04,"15min":3.04},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59399999999999997,"15min":0.57699999999999996},"min":{"1min":0.40100000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72499999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.63400000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.66500000000000004,"15min":0.67800000000000005},"min":{"1min":0.44400000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.65500000000000003,"5min":4.032,"15min":4.032},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.70399999999999996,"15min":0.67800000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.77000000000000002,"5min":4.282,"15min":4.282},"last":0.71499999999999997}]},{"osd":4,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81399999999999995,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.51100000000000001,"15min":0.56499999999999995},"min":{"1min":0.36799999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.61499999999999999,"5min":1.101,"15min":1.101},"last":0.497}]},{"osd":6,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51300000000000001,"15min":0.55600000000000005},"min":{"1min":0.36499999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.629,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56499999999999995,"15min":0.58199999999999996},"min":{"1min":0.39100000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76800000000000002,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.52500000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.55400000000000005,"15min":0.56699999999999995},"min":{"1min":0.40200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.745,"5min":1.008,"15min":1.008},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.626,"15min":0.65800000000000003},"min":{"1min":0.38400000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84799999999999998,"5min":2,"15min":2},"last":0.65800000000000003}]}]}],"pool_statfs":[]}} 2024-01-27T02:37:08.574 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-27T02:37:08.574 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-27T02:37:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.751+0000 7fba44cbb700 1 Processor -- start 2024-01-27T02:37:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.751+0000 7fba44cbb700 1 -- start start 2024-01-27T02:37:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.752+0000 7fba44cbb700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fba400aa8b0 0x7fba400c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.752+0000 7fba44cbb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba40063c90 0x7fba400c8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.752+0000 7fba44cbb700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba400c8cb0 con 0x7fba40143670 2024-01-27T02:37:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.753+0000 7fba44cbb700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba400cc290 con 0x7fba400aa8b0 2024-01-27T02:37:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.753+0000 7fba44cbb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba400cb960 con 0x7fba40063c90 2024-01-27T02:37:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.753+0000 7fba3dd9b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fba400aa8b0 0x7fba400c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.753+0000 7fba3ed9d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba40063c90 0x7fba400c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.753+0000 7fba3dd9b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fba400aa8b0 0x7fba400c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:60914/0 (socket says 172.21.15.7:60914) 2024-01-27T02:37:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.753+0000 7fba3ed9d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba40063c90 0x7fba400c8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:59788/0 (socket says 172.21.15.7:59788) 2024-01-27T02:37:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.753+0000 7fba3e59c700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fba40143670 0x7fba40063580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:55614/0 (socket says 172.21.15.7:55614) 2024-01-27T02:37:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.753+0000 7fba3dd9b700 1 -- 172.21.15.7:0/777060329 learned_addr learned my addr 172.21.15.7:0/777060329 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.753+0000 7fba3dd9b700 1 -- 172.21.15.7:0/777060329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba40063c90 msgr2=0x7fba400c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.753+0000 7fba3dd9b700 1 --2- 172.21.15.7:0/777060329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba40063c90 0x7fba400c8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.753+0000 7fba3dd9b700 1 -- 172.21.15.7:0/777060329 >> v1:172.21.15.188:6789/0 conn(0x7fba40143670 legacy=0x7fba40063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.753+0000 7fba3dd9b700 1 -- 172.21.15.7:0/777060329 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba400cbc40 con 0x7fba400aa8b0 2024-01-27T02:37:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.753+0000 7fba3d59a700 1 -- 172.21.15.7:0/777060329 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2369522489 0 0) 0x7fba30001110 con 0x7fba40143670 2024-01-27T02:37:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.753+0000 7fba3ed9d700 1 --2- 172.21.15.7:0/777060329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba40063c90 0x7fba400c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:37:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.754+0000 7fba3dd9b700 1 --2- 172.21.15.7:0/777060329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba400aa8b0 0x7fba400c8230 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fba280099b0 tx=0x7fba28009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=239d1b2d305289d6 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.754+0000 7fba3d59a700 1 -- 172.21.15.7:0/777060329 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba28009ea0 con 0x7fba400aa8b0 2024-01-27T02:37:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.754+0000 7fba3d59a700 1 -- 172.21.15.7:0/777060329 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2800b860 con 0x7fba400aa8b0 2024-01-27T02:37:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.754+0000 7fba3d59a700 1 -- 172.21.15.7:0/777060329 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2800bc00 con 0x7fba400aa8b0 2024-01-27T02:37:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.754+0000 7fba44cbb700 1 -- 172.21.15.7:0/777060329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba400aa8b0 msgr2=0x7fba400c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.754+0000 7fba44cbb700 1 --2- 172.21.15.7:0/777060329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba400aa8b0 0x7fba400c8230 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fba280099b0 tx=0x7fba28009250 comp rx=0 tx=0).stop 2024-01-27T02:37:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.755+0000 7fba44cbb700 1 -- 172.21.15.7:0/777060329 shutdown_connections 2024-01-27T02:37:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.755+0000 7fba44cbb700 1 --2- 172.21.15.7:0/777060329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba40063c90 0x7fba400c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.755+0000 7fba44cbb700 1 --2- 172.21.15.7:0/777060329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba400aa8b0 0x7fba400c8230 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.755+0000 7fba44cbb700 1 -- 172.21.15.7:0/777060329 >> 172.21.15.7:0/777060329 conn(0x7fba400cdbf0 msgr2=0x7fba400cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.755+0000 7fba44cbb700 1 -- 172.21.15.7:0/777060329 shutdown_connections 2024-01-27T02:37:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.755+0000 7fba44cbb700 1 -- 172.21.15.7:0/777060329 wait complete. 2024-01-27T02:37:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.756+0000 7fba44cbb700 1 Processor -- start 2024-01-27T02:37:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.756+0000 7fba44cbb700 1 -- start start 2024-01-27T02:37:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.757+0000 7fba44cbb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba40063c90 0x7fba4013f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.757+0000 7fba44cbb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba400aa8b0 0x7fba4015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.757+0000 7fba44cbb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba40143670 0x7fba400cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.757+0000 7fba44cbb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4006afa0 con 0x7fba40143670 2024-01-27T02:37:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.757+0000 7fba44cbb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4006a390 con 0x7fba40063c90 2024-01-27T02:37:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.757+0000 7fba44cbb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba4006a670 con 0x7fba400aa8b0 2024-01-27T02:37:08.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.757+0000 7fba3ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba40143670 0x7fba400cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.757+0000 7fba3ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba40143670 0x7fba400cdfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37576/0 (socket says 172.21.15.7:37576) 2024-01-27T02:37:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.757+0000 7fba3ed9d700 1 -- 172.21.15.7:0/1605915938 learned_addr learned my addr 172.21.15.7:0/1605915938 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.757+0000 7fba3e59c700 1 --2- 172.21.15.7:0/1605915938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba40063c90 0x7fba4013f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.757+0000 7fba3dd9b700 1 --2- 172.21.15.7:0/1605915938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba400aa8b0 0x7fba4015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.757+0000 7fba3ed9d700 1 -- 172.21.15.7:0/1605915938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba400aa8b0 msgr2=0x7fba4015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.757+0000 7fba3ed9d700 1 --2- 172.21.15.7:0/1605915938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba400aa8b0 0x7fba4015ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.757+0000 7fba3ed9d700 1 -- 172.21.15.7:0/1605915938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba40063c90 msgr2=0x7fba4013f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.757+0000 7fba3ed9d700 1 --2- 172.21.15.7:0/1605915938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba40063c90 0x7fba4013f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.757+0000 7fba3ed9d700 1 -- 172.21.15.7:0/1605915938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba38009a20 con 0x7fba40143670 2024-01-27T02:37:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.757+0000 7fba3dd9b700 1 --2- 172.21.15.7:0/1605915938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba400aa8b0 0x7fba4015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:37:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.757+0000 7fba3ed9d700 1 --2- 172.21.15.7:0/1605915938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba40143670 0x7fba400cdfc0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fba38009f90 tx=0x7fba3800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.757+0000 7fba367fc700 1 -- 172.21.15.7:0/1605915938 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3800cc20 con 0x7fba40143670 2024-01-27T02:37:08.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.758+0000 7fba367fc700 1 -- 172.21.15.7:0/1605915938 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3800f040 con 0x7fba40143670 2024-01-27T02:37:08.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.758+0000 7fba44cbb700 1 -- 172.21.15.7:0/1605915938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba280094f0 con 0x7fba40143670 2024-01-27T02:37:08.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.758+0000 7fba367fc700 1 -- 172.21.15.7:0/1605915938 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba38014480 con 0x7fba40143670 2024-01-27T02:37:08.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.758+0000 7fba44cbb700 1 -- 172.21.15.7:0/1605915938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba4006ac30 con 0x7fba40143670 2024-01-27T02:37:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.759+0000 7fba44cbb700 1 -- 172.21.15.7:0/1605915938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba400ab970 con 0x7fba40143670 2024-01-27T02:37:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.759+0000 7fba367fc700 1 -- 172.21.15.7:0/1605915938 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fba380147f0 con 0x7fba40143670 2024-01-27T02:37:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.759+0000 7fba367fc700 1 --2- 172.21.15.7:0/1605915938 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fba20066f30 0x7fba200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.759+0000 7fba367fc700 1 -- 172.21.15.7:0/1605915938 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fba380852f0 con 0x7fba40143670 2024-01-27T02:37:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.760+0000 7fba3e59c700 1 --2- 172.21.15.7:0/1605915938 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fba20066f30 0x7fba200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.760+0000 7fba3e59c700 1 --2- 172.21.15.7:0/1605915938 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fba20066f30 0x7fba200693e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fba4006cbe0 tx=0x7fba30007ef0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:08.764+0000 7fba367fc700 1 -- 172.21.15.7:0/1605915938 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba38055ff0 con 0x7fba40143670 2024-01-27T02:37:08.955 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-01-27T02:37:08.954+0000 7fdcc1d3a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-27T02:37:09.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.052+0000 7fba44cbb700 1 -- 172.21.15.7:0/1605915938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fba40072060 con 0x7fba40143670 2024-01-27T02:37:09.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.055+0000 7fba367fc700 1 -- 172.21.15.7:0/1605915938 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7fba38055b80 con 0x7fba40143670 2024-01-27T02:37:09.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.057+0000 7fba44cbb700 1 -- 172.21.15.7:0/1605915938 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fba20066f30 msgr2=0x7fba200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:09.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.057+0000 7fba44cbb700 1 --2- 172.21.15.7:0/1605915938 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fba20066f30 0x7fba200693e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fba4006cbe0 tx=0x7fba30007ef0 comp rx=0 tx=0).stop 2024-01-27T02:37:09.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.057+0000 7fba44cbb700 1 -- 172.21.15.7:0/1605915938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba40143670 msgr2=0x7fba400cdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:09.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.057+0000 7fba44cbb700 1 --2- 172.21.15.7:0/1605915938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba40143670 0x7fba400cdfc0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fba38009f90 tx=0x7fba3800b040 comp rx=0 tx=0).stop 2024-01-27T02:37:09.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.057+0000 7fba44cbb700 1 -- 172.21.15.7:0/1605915938 shutdown_connections 2024-01-27T02:37:09.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.057+0000 7fba44cbb700 1 --2- 172.21.15.7:0/1605915938 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fba20066f30 0x7fba200693e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:09.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.057+0000 7fba44cbb700 1 --2- 172.21.15.7:0/1605915938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba40143670 0x7fba400cdfc0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:09.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.057+0000 7fba44cbb700 1 --2- 172.21.15.7:0/1605915938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba40063c90 0x7fba4013f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:09.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.057+0000 7fba44cbb700 1 --2- 172.21.15.7:0/1605915938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba400aa8b0 0x7fba4015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:09.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.057+0000 7fba44cbb700 1 -- 172.21.15.7:0/1605915938 >> 172.21.15.7:0/1605915938 conn(0x7fba400cdbf0 msgr2=0x7fba40142740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:09.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.057+0000 7fba44cbb700 1 -- 172.21.15.7:0/1605915938 shutdown_connections 2024-01-27T02:37:09.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.057+0000 7fba44cbb700 1 -- 172.21.15.7:0/1605915938 wait complete. 2024-01-27T02:37:09.059 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-01-27T02:37:09.071 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:37:09.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.202+0000 7f6011073700 1 Processor -- start 2024-01-27T02:37:09.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.202+0000 7f6011073700 1 -- start start 2024-01-27T02:37:09.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.203+0000 7f6011073700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f600c15a100 0x7f600c0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:09.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.203+0000 7f6011073700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f600c0cb5b0 0x7f600c0c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:09.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.203+0000 7f6011073700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c0c4af0 0x7f600c0c9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:09.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.203+0000 7f6011073700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f600c0c4f70 con 0x7f600c15a100 2024-01-27T02:37:09.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.203+0000 7f6011073700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f600c0ce5e0 con 0x7f600c0cb5b0 2024-01-27T02:37:09.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.203+0000 7f6011073700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0cd9d0 con 0x7f600c0c4af0 2024-01-27T02:37:09.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.203+0000 7f600ad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f600c15a100 0x7f600c0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:09.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.204+0000 7f600ad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f600c15a100 0x7f600c0cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37588/0 (socket says 172.21.15.7:37588) 2024-01-27T02:37:09.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.204+0000 7f600ad9d700 1 -- 172.21.15.7:0/1987737196 learned_addr learned my addr 172.21.15.7:0/1987737196 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:09.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.204+0000 7f600a59c700 1 --2- 172.21.15.7:0/1987737196 >> v2:172.21.15.188:3300/0 conn(0x7f600c0cb5b0 0x7f600c0c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:09.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.204+0000 7f600b59e700 1 --2- 172.21.15.7:0/1987737196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c0c4af0 0x7f600c0c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:09.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.204+0000 7f600ad9d700 1 -- 172.21.15.7:0/1987737196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c0c4af0 msgr2=0x7f600c0c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:09.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.204+0000 7f600ad9d700 1 --2- 172.21.15.7:0/1987737196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c0c4af0 0x7f600c0c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:09.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.204+0000 7f600ad9d700 1 -- 172.21.15.7:0/1987737196 >> v2:172.21.15.188:3300/0 conn(0x7f600c0cb5b0 msgr2=0x7f600c0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:09.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.204+0000 7f600ad9d700 1 --2- 172.21.15.7:0/1987737196 >> v2:172.21.15.188:3300/0 conn(0x7f600c0cb5b0 0x7f600c0c45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:09.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.204+0000 7f600ad9d700 1 -- 172.21.15.7:0/1987737196 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c0cdcb0 con 0x7f600c15a100 2024-01-27T02:37:09.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.204+0000 7f600b59e700 1 --2- 172.21.15.7:0/1987737196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c0c4af0 0x7f600c0c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:37:09.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.204+0000 7f600ad9d700 1 --2- 172.21.15.7:0/1987737196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f600c15a100 0x7f600c0cd2f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f600c06e170 tx=0x7f5ffc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54fc77d3c24d7d0e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:09.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.204+0000 7f6009d9b700 1 -- 172.21.15.7:0/1987737196 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc009e70 con 0x7f600c15a100 2024-01-27T02:37:09.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.204+0000 7f6009d9b700 1 -- 172.21.15.7:0/1987737196 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ffc00b860 con 0x7f600c15a100 2024-01-27T02:37:09.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.204+0000 7f6009d9b700 1 -- 172.21.15.7:0/1987737196 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc00bc00 con 0x7f600c15a100 2024-01-27T02:37:09.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.205+0000 7f6011073700 1 -- 172.21.15.7:0/1987737196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f600c15a100 msgr2=0x7f600c0cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:09.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.205+0000 7f6011073700 1 --2- 172.21.15.7:0/1987737196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f600c15a100 0x7f600c0cd2f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f600c06e170 tx=0x7f5ffc009250 comp rx=0 tx=0).stop 2024-01-27T02:37:09.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.205+0000 7f6011073700 1 -- 172.21.15.7:0/1987737196 shutdown_connections 2024-01-27T02:37:09.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.205+0000 7f6011073700 1 --2- 172.21.15.7:0/1987737196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c0c4af0 0x7f600c0c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:09.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.205+0000 7f6011073700 1 --2- 172.21.15.7:0/1987737196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f600c15a100 0x7f600c0cd2f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:09.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.205+0000 7f6011073700 1 --2- 172.21.15.7:0/1987737196 >> v2:172.21.15.188:3300/0 conn(0x7f600c0cb5b0 0x7f600c0c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:09.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.205+0000 7f6011073700 1 -- 172.21.15.7:0/1987737196 >> 172.21.15.7:0/1987737196 conn(0x7f600c098020 msgr2=0x7f600c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:09.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.205+0000 7f6011073700 1 -- 172.21.15.7:0/1987737196 shutdown_connections 2024-01-27T02:37:09.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.205+0000 7f6011073700 1 -- 172.21.15.7:0/1987737196 wait complete. 2024-01-27T02:37:09.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.206+0000 7f6011073700 1 Processor -- start 2024-01-27T02:37:09.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.206+0000 7f6011073700 1 -- start start 2024-01-27T02:37:09.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.207+0000 7f6011073700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c0c4af0 0x7f600c0c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:09.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.207+0000 7f6011073700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f600c0cb5b0 0x7f600c15c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:09.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.207+0000 7f6011073700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f600c15a100 0x7f600c156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:09.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.207+0000 7f6011073700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c15ab80 con 0x7f600c0cb5b0 2024-01-27T02:37:09.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.207+0000 7f6011073700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c15a570 con 0x7f600c15a100 2024-01-27T02:37:09.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.207+0000 7f6011073700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0c7d70 con 0x7f600c0c4af0 2024-01-27T02:37:09.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.207+0000 7f600a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f600c0cb5b0 0x7f600c15c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:09.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.207+0000 7f600a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f600c0cb5b0 0x7f600c15c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37602/0 (socket says 172.21.15.7:37602) 2024-01-27T02:37:09.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.207+0000 7f600a59c700 1 -- 172.21.15.7:0/1860963573 learned_addr learned my addr 172.21.15.7:0/1860963573 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:09.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.207+0000 7f600ad9d700 1 --2- 172.21.15.7:0/1860963573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c0c4af0 0x7f600c0c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:09.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.207+0000 7f600b59e700 1 --2- 172.21.15.7:0/1860963573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f600c15a100 0x7f600c156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:09.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.208+0000 7f600a59c700 1 -- 172.21.15.7:0/1860963573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c0c4af0 msgr2=0x7f600c0c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:09.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.208+0000 7f600a59c700 1 --2- 172.21.15.7:0/1860963573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c0c4af0 0x7f600c0c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:09.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.208+0000 7f600a59c700 1 -- 172.21.15.7:0/1860963573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f600c15a100 msgr2=0x7f600c156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:09.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.208+0000 7f600a59c700 1 --2- 172.21.15.7:0/1860963573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f600c15a100 0x7f600c156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:09.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.208+0000 7f600a59c700 1 -- 172.21.15.7:0/1860963573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ffc0094f0 con 0x7f600c0cb5b0 2024-01-27T02:37:09.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.208+0000 7f600a59c700 1 --2- 172.21.15.7:0/1860963573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f600c0cb5b0 0x7f600c15c960 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f600000e930 tx=0x7f60000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:09.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.208+0000 7f5ff2ffd700 1 -- 172.21.15.7:0/1860963573 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000013680 con 0x7f600c0cb5b0 2024-01-27T02:37:09.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.208+0000 7f5ff2ffd700 1 -- 172.21.15.7:0/1860963573 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60000137e0 con 0x7f600c0cb5b0 2024-01-27T02:37:09.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.208+0000 7f6011073700 1 -- 172.21.15.7:0/1860963573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600c15a810 con 0x7f600c0cb5b0 2024-01-27T02:37:09.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.208+0000 7f5ff2ffd700 1 -- 172.21.15.7:0/1860963573 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000013b80 con 0x7f600c0cb5b0 2024-01-27T02:37:09.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.208+0000 7f6011073700 1 -- 172.21.15.7:0/1860963573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600c0c8050 con 0x7f600c0cb5b0 2024-01-27T02:37:09.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.209+0000 7f6011073700 1 -- 172.21.15.7:0/1860963573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600c04c710 con 0x7f600c0cb5b0 2024-01-27T02:37:09.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.210+0000 7f5ff2ffd700 1 -- 172.21.15.7:0/1860963573 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f600000c3e0 con 0x7f600c0cb5b0 2024-01-27T02:37:09.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.210+0000 7f5ff2ffd700 1 --2- 172.21.15.7:0/1860963573 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5ff8066f30 0x7f5ff80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:09.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.210+0000 7f5ff2ffd700 1 -- 172.21.15.7:0/1860963573 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f60000845e0 con 0x7f600c0cb5b0 2024-01-27T02:37:09.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.210+0000 7f600ad9d700 1 --2- 172.21.15.7:0/1860963573 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5ff8066f30 0x7f5ff80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:09.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.211+0000 7f600ad9d700 1 --2- 172.21.15.7:0/1860963573 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5ff8066f30 0x7f5ff80693e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f600c06e170 tx=0x7f5ffc017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:09.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.215+0000 7f5ff2ffd700 1 -- 172.21.15.7:0/1860963573 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6000051850 con 0x7f600c0cb5b0 2024-01-27T02:37:09.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.466+0000 7f6011073700 1 -- 172.21.15.7:0/1860963573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f600c0cdf90 con 0x7f600c0cb5b0 2024-01-27T02:37:09.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.467+0000 7f5ff2ffd700 1 -- 172.21.15.7:0/1860963573 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4584 (secure 0 0 0) 0x7f6000054e70 con 0x7f600c0cb5b0 2024-01-27T02:37:09.467 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:09.468 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/450556982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":450556982},{"type":"v1","addr":"172.21.15.188:6836","nonce":450556982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1711422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1711422},{"type":"v1","addr":"172.21.15.188:6837","nonce":1711422}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6294,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1110306600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1110306600},{"type":"v1","addr":"172.21.15.7:6833","nonce":1110306600}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:37:03.404355+0000","modified":"2024-01-27T02:37:06.988798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6297":{"gid":6297,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/328992853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":328992853},{"type":"v1","addr":"172.21.15.7:6835","nonce":328992853}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-27T02:37:09.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.469+0000 7f6011073700 1 -- 172.21.15.7:0/1860963573 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5ff8066f30 msgr2=0x7f5ff80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:09.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.469+0000 7f6011073700 1 --2- 172.21.15.7:0/1860963573 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5ff8066f30 0x7f5ff80693e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f600c06e170 tx=0x7f5ffc017000 comp rx=0 tx=0).stop 2024-01-27T02:37:09.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.469+0000 7f6011073700 1 -- 172.21.15.7:0/1860963573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f600c0cb5b0 msgr2=0x7f600c15c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:09.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.469+0000 7f6011073700 1 --2- 172.21.15.7:0/1860963573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f600c0cb5b0 0x7f600c15c960 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f600000e930 tx=0x7f60000091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:09.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.469+0000 7f6011073700 1 -- 172.21.15.7:0/1860963573 shutdown_connections 2024-01-27T02:37:09.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.469+0000 7f6011073700 1 --2- 172.21.15.7:0/1860963573 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5ff8066f30 0x7f5ff80693e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:09.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.469+0000 7f6011073700 1 --2- 172.21.15.7:0/1860963573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f600c15a100 0x7f600c156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:09.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.469+0000 7f6011073700 1 --2- 172.21.15.7:0/1860963573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c0c4af0 0x7f600c0c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:09.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.469+0000 7f6011073700 1 --2- 172.21.15.7:0/1860963573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f600c0cb5b0 0x7f600c15c960 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:09.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.469+0000 7f6011073700 1 -- 172.21.15.7:0/1860963573 >> 172.21.15.7:0/1860963573 conn(0x7f600c098020 msgr2=0x7f600c0ca9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:09.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.469+0000 7f6011073700 1 -- 172.21.15.7:0/1860963573 shutdown_connections 2024-01-27T02:37:09.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:09.469+0000 7f6011073700 1 -- 172.21.15.7:0/1860963573 wait complete. 2024-01-27T02:37:09.471 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 49 2024-01-27T02:37:09.885 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-27T02:37:03.404355+0000', 'modified': '2024-01-27T02:37:06.988798+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6297}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6297': {'gid': 6297, 'name': 'c', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6835/328992853', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 328992853}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 328992853}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-27T02:37:09.885 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-27T02:37:09.885 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-27T02:37:10.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.028+0000 7fd30ebe0700 1 Processor -- start 2024-01-27T02:37:10.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.028+0000 7fd30ebe0700 1 -- start start 2024-01-27T02:37:10.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.029+0000 7fd30ebe0700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3080b1c00 0x7fd30806e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.029+0000 7fd30ebe0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd308157610 con 0x7fd30815b5f0 2024-01-27T02:37:10.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.029+0000 7fd30ebe0700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30806ee00 con 0x7fd3080b1fd0 2024-01-27T02:37:10.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.029+0000 7fd30ebe0700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30806f0e0 con 0x7fd3080b1c00 2024-01-27T02:37:10.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.029+0000 7fd307fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd30815b5f0 0x7fd3081566c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48216/0 (socket says 172.21.15.7:48216) 2024-01-27T02:37:10.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.029+0000 7fd307fff700 1 -- 172.21.15.7:0/798610673 learned_addr learned my addr 172.21.15.7:0/798610673 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:10.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.029+0000 7fd30c97c700 1 --2- 172.21.15.7:0/798610673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3080b1c00 0x7fd30806e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.029+0000 7fd3077fe700 1 -- 172.21.15.7:0/798610673 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2236390944 0 0) 0x7fd308157610 con 0x7fd30815b5f0 2024-01-27T02:37:10.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.029+0000 7fd3077fe700 1 -- 172.21.15.7:0/798610673 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2f8003570 con 0x7fd30815b5f0 2024-01-27T02:37:10.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.029+0000 7fd3077fe700 1 -- 172.21.15.7:0/798610673 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3703107304 0 0) 0x7fd30806ee00 con 0x7fd3080b1fd0 2024-01-27T02:37:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.029+0000 7fd3077fe700 1 -- 172.21.15.7:0/798610673 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd308157610 con 0x7fd3080b1fd0 2024-01-27T02:37:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.029+0000 7fd3077fe700 1 -- 172.21.15.7:0/798610673 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3570040314 0 0) 0x7fd2f8003570 con 0x7fd30815b5f0 2024-01-27T02:37:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.029+0000 7fd30c97c700 1 -- 172.21.15.7:0/798610673 >> v1:172.21.15.188:6789/0 conn(0x7fd3080b1fd0 legacy=0x7fd308156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.029+0000 7fd30c97c700 1 -- 172.21.15.7:0/798610673 >> v1:172.21.15.7:6789/0 conn(0x7fd30815b5f0 legacy=0x7fd3081566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.029+0000 7fd30c97c700 1 -- 172.21.15.7:0/798610673 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30806f3c0 con 0x7fd3080b1c00 2024-01-27T02:37:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.030+0000 7fd30c97c700 1 --2- 172.21.15.7:0/798610673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3080b1c00 0x7fd30806e540 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009910 tx=0x7fd2fc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b1e01f1f1ad2c7c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.030+0000 7fd3077fe700 1 -- 172.21.15.7:0/798610673 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4001470 con 0x7fd3080b1c00 2024-01-27T02:37:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.030+0000 7fd3077fe700 1 -- 172.21.15.7:0/798610673 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc011420 con 0x7fd3080b1c00 2024-01-27T02:37:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.030+0000 7fd3077fe700 1 -- 172.21.15.7:0/798610673 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc0117c0 con 0x7fd3080b1c00 2024-01-27T02:37:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.030+0000 7fd30ebe0700 1 -- 172.21.15.7:0/798610673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3080b1c00 msgr2=0x7fd30806e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.030+0000 7fd30ebe0700 1 --2- 172.21.15.7:0/798610673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3080b1c00 0x7fd30806e540 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009910 tx=0x7fd2fc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.030+0000 7fd30ebe0700 1 -- 172.21.15.7:0/798610673 shutdown_connections 2024-01-27T02:37:10.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.030+0000 7fd30ebe0700 1 --2- 172.21.15.7:0/798610673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3080b1c00 0x7fd30806e540 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.030+0000 7fd30ebe0700 1 -- 172.21.15.7:0/798610673 >> 172.21.15.7:0/798610673 conn(0x7fd308068430 msgr2=0x7fd30806be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.030+0000 7fd30ebe0700 1 -- 172.21.15.7:0/798610673 shutdown_connections 2024-01-27T02:37:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.030+0000 7fd30ebe0700 1 -- 172.21.15.7:0/798610673 wait complete. 2024-01-27T02:37:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.032+0000 7fd30ebe0700 1 Processor -- start 2024-01-27T02:37:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.032+0000 7fd30ebe0700 1 -- start start 2024-01-27T02:37:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.033+0000 7fd30ebe0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3080b1c00 0x7fd30814b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.033+0000 7fd30ebe0700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3080b1fd0 0x7fd30814c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.033+0000 7fd30ebe0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd30815b5f0 0x7fd30813e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.033+0000 7fd30ebe0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30814c520 con 0x7fd30815b5f0 2024-01-27T02:37:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.033+0000 7fd30ebe0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30814cef0 con 0x7fd3080b1c00 2024-01-27T02:37:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.033+0000 7fd30ebe0700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30814b910 con 0x7fd3080b1fd0 2024-01-27T02:37:10.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.033+0000 7fd30d17d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd30815b5f0 0x7fd30813e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.033+0000 7fd30d17d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd30815b5f0 0x7fd30813e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37610/0 (socket says 172.21.15.7:37610) 2024-01-27T02:37:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.033+0000 7fd30d17d700 1 -- 172.21.15.7:0/648717901 learned_addr learned my addr 172.21.15.7:0/648717901 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.033+0000 7fd30c97c700 1 --2- 172.21.15.7:0/648717901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3080b1c00 0x7fd30814b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.033+0000 7fd307fff700 1 --2- 172.21.15.7:0/648717901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3080b1fd0 0x7fd30814c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.033+0000 7fd30d17d700 1 -- 172.21.15.7:0/648717901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3080b1fd0 msgr2=0x7fd30814c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.033+0000 7fd30d17d700 1 --2- 172.21.15.7:0/648717901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3080b1fd0 0x7fd30814c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.033+0000 7fd30d17d700 1 -- 172.21.15.7:0/648717901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3080b1c00 msgr2=0x7fd30814b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.033+0000 7fd30d17d700 1 --2- 172.21.15.7:0/648717901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3080b1c00 0x7fd30814b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.033+0000 7fd30d17d700 1 -- 172.21.15.7:0/648717901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc009420 con 0x7fd30815b5f0 2024-01-27T02:37:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.033+0000 7fd30d17d700 1 --2- 172.21.15.7:0/648717901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd30815b5f0 0x7fd30813e600 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd30000c910 tx=0x7fd3000027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.034+0000 7fd304ff9700 1 -- 172.21.15.7:0/648717901 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30000a7e0 con 0x7fd30815b5f0 2024-01-27T02:37:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.034+0000 7fd304ff9700 1 -- 172.21.15.7:0/648717901 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30000a960 con 0x7fd30815b5f0 2024-01-27T02:37:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.034+0000 7fd30ebe0700 1 -- 172.21.15.7:0/648717901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30814bbf0 con 0x7fd30815b5f0 2024-01-27T02:37:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.034+0000 7fd304ff9700 1 -- 172.21.15.7:0/648717901 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30000ad20 con 0x7fd30815b5f0 2024-01-27T02:37:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.034+0000 7fd30ebe0700 1 -- 172.21.15.7:0/648717901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30814c1b0 con 0x7fd30815b5f0 2024-01-27T02:37:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.035+0000 7fd304ff9700 1 -- 172.21.15.7:0/648717901 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd300012630 con 0x7fd30815b5f0 2024-01-27T02:37:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.035+0000 7fd304ff9700 1 --2- 172.21.15.7:0/648717901 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd2ec066f30 0x7fd2ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.036+0000 7fd304ff9700 1 -- 172.21.15.7:0/648717901 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd300084a00 con 0x7fd30815b5f0 2024-01-27T02:37:10.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.036+0000 7fd30c97c700 1 --2- 172.21.15.7:0/648717901 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd2ec066f30 0x7fd2ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.036+0000 7fd30c97c700 1 --2- 172.21.15.7:0/648717901 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd2ec066f30 0x7fd2ec0693e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc0030c0 tx=0x7fd2fc016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:10.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.038+0000 7fd317ee3b80 1 Processor -- start 2024-01-27T02:37:10.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.038+0000 7fd317ee3b80 1 -- start start 2024-01-27T02:37:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.039+0000 7fd317ee3b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ce7b0f8010 0x55ce7b0f83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.039+0000 7fd317ee3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce7b140360 0x55ce7b122840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.039+0000 7fd317ee3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f75d0 0x55ce7b122d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.039+0000 7fd317ee3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce7affb9f0 con 0x55ce7b0f75d0 2024-01-27T02:37:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.039+0000 7fd317ee3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce7b04d130 con 0x55ce7b140360 2024-01-27T02:37:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.039+0000 7fd317ee3b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce7b04a930 con 0x55ce7b0f8010 2024-01-27T02:37:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.039+0000 7fd30d17d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f75d0 0x55ce7b122d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.039+0000 7fd30d17d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f75d0 0x55ce7b122d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37618/0 (socket says 172.21.15.7:37618) 2024-01-27T02:37:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.039+0000 7fd30c97c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ce7b0f8010 0x55ce7b0f83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.039+0000 7fd30d17d700 1 -- 172.21.15.7:0/3823750201 learned_addr learned my addr 172.21.15.7:0/3823750201 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.039+0000 7fd307fff700 1 --2- 172.21.15.7:0/3823750201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce7b140360 0x55ce7b122840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.039+0000 7fd30d17d700 1 -- 172.21.15.7:0/3823750201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ce7b0f8010 msgr2=0x55ce7b0f83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.039+0000 7fd30d17d700 1 --2- 172.21.15.7:0/3823750201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ce7b0f8010 0x55ce7b0f83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.039+0000 7fd30d17d700 1 -- 172.21.15.7:0/3823750201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce7b140360 msgr2=0x55ce7b122840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.039+0000 7fd30d17d700 1 --2- 172.21.15.7:0/3823750201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce7b140360 0x55ce7b122840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.039+0000 7fd30d17d700 1 -- 172.21.15.7:0/3823750201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce7ae85b10 con 0x55ce7b0f75d0 2024-01-27T02:37:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.039+0000 7fd30d17d700 1 --2- 172.21.15.7:0/3823750201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f75d0 0x55ce7b122d80 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd300082ef0 tx=0x7fd300082e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dedb651430282a83 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.039+0000 7fd2f1ffb700 1 -- 172.21.15.7:0/3823750201 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300055930 con 0x55ce7b0f75d0 2024-01-27T02:37:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.039+0000 7fd2f1ffb700 1 -- 172.21.15.7:0/3823750201 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd300058b80 con 0x55ce7b0f75d0 2024-01-27T02:37:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.039+0000 7fd2f1ffb700 1 -- 172.21.15.7:0/3823750201 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30005c8d0 con 0x55ce7b0f75d0 2024-01-27T02:37:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.040+0000 7fd317ee3b80 1 -- 172.21.15.7:0/3823750201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f75d0 msgr2=0x55ce7b122d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.040+0000 7fd317ee3b80 1 --2- 172.21.15.7:0/3823750201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f75d0 0x55ce7b122d80 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd300082ef0 tx=0x7fd300082e80 comp rx=0 tx=0).stop 2024-01-27T02:37:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.040+0000 7fd317ee3b80 1 -- 172.21.15.7:0/3823750201 shutdown_connections 2024-01-27T02:37:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.040+0000 7fd317ee3b80 1 --2- 172.21.15.7:0/3823750201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f75d0 0x55ce7b122d80 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.040+0000 7fd317ee3b80 1 --2- 172.21.15.7:0/3823750201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ce7b0f8010 0x55ce7b0f83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.040+0000 7fd317ee3b80 1 --2- 172.21.15.7:0/3823750201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce7b140360 0x55ce7b122840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.040+0000 7fd317ee3b80 1 -- 172.21.15.7:0/3823750201 >> 172.21.15.7:0/3823750201 conn(0x55ce7b0f7a70 msgr2=0x55ce7b140820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.040+0000 7fd317ee3b80 1 -- 172.21.15.7:0/3823750201 shutdown_connections 2024-01-27T02:37:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.040+0000 7fd317ee3b80 1 -- 172.21.15.7:0/3823750201 wait complete. 2024-01-27T02:37:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.041+0000 7fd317ee3b80 1 -- start start 2024-01-27T02:37:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.041+0000 7fd317ee3b80 1 Processor -- start 2024-01-27T02:37:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.042+0000 7fd317ee3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f75d0 0x55ce7b11dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.042+0000 7fd317ee3b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ce7b0f8010 0x55ce7b123340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.043+0000 7fd317ee3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce7b140360 0x55ce7b125880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.043+0000 7fd317ee3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce7b046720 con 0x55ce7b0f75d0 2024-01-27T02:37:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.043+0000 7fd317ee3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce7aff21b0 con 0x55ce7b140360 2024-01-27T02:37:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.043+0000 7fd317ee3b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce7aff1e50 con 0x55ce7b0f8010 2024-01-27T02:37:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.043+0000 7fd317ee3b80 10 client.0 _ll_register_callbacks cb 0x55ce7ae547b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-27T02:37:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.043+0000 7fd30c97c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f75d0 0x55ce7b11dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.043+0000 7fd30c97c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f75d0 0x55ce7b11dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37630/0 (socket says 172.21.15.7:37630) 2024-01-27T02:37:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.043+0000 7fd30c97c700 1 -- 172.21.15.7:0/2905786117 learned_addr learned my addr 172.21.15.7:0/2905786117 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.043+0000 7fd30d17d700 1 --2- 172.21.15.7:0/2905786117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce7b140360 0x55ce7b125880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.043+0000 7fd307fff700 1 --2- 172.21.15.7:0/2905786117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ce7b0f8010 0x55ce7b123340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.043+0000 7fd30c97c700 1 -- 172.21.15.7:0/2905786117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ce7b0f8010 msgr2=0x55ce7b123340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.043+0000 7fd30c97c700 1 --2- 172.21.15.7:0/2905786117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ce7b0f8010 0x55ce7b123340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.043+0000 7fd30c97c700 1 -- 172.21.15.7:0/2905786117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce7b140360 msgr2=0x55ce7b125880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.043+0000 7fd30c97c700 1 --2- 172.21.15.7:0/2905786117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce7b140360 0x55ce7b125880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.043+0000 7fd30c97c700 1 -- 172.21.15.7:0/2905786117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30814c1b0 con 0x55ce7b0f75d0 2024-01-27T02:37:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.043+0000 7fd30c97c700 1 --2- 172.21.15.7:0/2905786117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f75d0 0x55ce7b11dc60 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc006d80 tx=0x7fd2fc021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccedf83502d04104 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.043+0000 7fd2e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-27T02:37:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.043+0000 7fd2e6ffd700 1 -- 172.21.15.7:0/2905786117 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc02e070 con 0x55ce7b0f75d0 2024-01-27T02:37:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.043+0000 7fd2e6ffd700 1 -- 172.21.15.7:0/2905786117 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd2fc01cad0 con 0x55ce7b0f75d0 2024-01-27T02:37:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.043+0000 7fd317ee3b80 1 -- 172.21.15.7:0/2905786117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce7b0eb190 con 0x55ce7b0f75d0 2024-01-27T02:37:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.044+0000 7fd2e6ffd700 1 -- 172.21.15.7:0/2905786117 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2fc021070 con 0x55ce7b0f75d0 2024-01-27T02:37:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.044+0000 7fd317ee3b80 10 client.6423 fetch_fsmap learned FSMap version 49 2024-01-27T02:37:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.044+0000 7fd317ee3b80 1 -- 172.21.15.7:0/2905786117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x55ce7b014290 con 0x55ce7b0f75d0 2024-01-27T02:37:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.044+0000 7fd2e6ffd700 1 -- 172.21.15.7:0/2905786117 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ce7b0eb190 con 0x55ce7b0f75d0 2024-01-27T02:37:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.044+0000 7fd2e6ffd700 1 -- 172.21.15.7:0/2905786117 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 49) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd2fc028410 con 0x55ce7b0f75d0 2024-01-27T02:37:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.044+0000 7fd317ee3b80 10 client.6423 fetch_fsmap finished waiting for FSMap version 49 2024-01-27T02:37:10.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.044+0000 7fd317ee3b80 10 client.6423 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-27T02:37:10.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.044+0000 7fd317ee3b80 20 client.6423 populate_metadata read hostname 'smithi007' 2024-01-27T02:37:10.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.044+0000 7fd317ee3b80 1 --2- 172.21.15.7:0/2905786117 >> [v2:172.21.15.7:6834/328992853,v1:172.21.15.7:6835/328992853] conn(0x55ce7b13d6b0 0x55ce7b13da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.044+0000 7fd317ee3b80 4 client.6423 mds_command: new command op to 6297 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-27T02:37:10.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.044+0000 7fd317ee3b80 1 -- 172.21.15.7:0/2905786117 --> [v2:172.21.15.7:6834/328992853,v1:172.21.15.7:6835/328992853] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce7afe8ee0 con 0x55ce7b13d6b0 2024-01-27T02:37:10.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.044+0000 7fd307fff700 1 --2- 172.21.15.7:0/2905786117 >> [v2:172.21.15.7:6834/328992853,v1:172.21.15.7:6835/328992853] conn(0x55ce7b13d6b0 0x55ce7b13da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.045+0000 7fd2e6ffd700 10 client.6423 ms_handle_connect on v2:172.21.15.7:6834/328992853 2024-01-27T02:37:10.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.045+0000 7fd307fff700 1 --2- 172.21.15.7:0/2905786117 >> [v2:172.21.15.7:6834/328992853,v1:172.21.15.7:6835/328992853] conn(0x55ce7b13d6b0 0x55ce7b13da80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9899a13aba1a1c server_cookie=e06e3436449671f4 in_seq=0 out_seq=0 2024-01-27T02:37:10.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.045+0000 7fd2e6ffd700 1 -- 172.21.15.7:0/2905786117 <== mds.0 v2:172.21.15.7:6834/328992853 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17146 (crc 0 0 0) 0x55ce7afe8ee0 con 0x55ce7b13d6b0 2024-01-27T02:37:10.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.045+0000 7fd2e6ffd700 10 client.6423 handle_command_reply: tid=0 2024-01-27T02:37:10.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.045+0000 7fd317ee3b80 1 client.6423 shutdown 2024-01-27T02:37:10.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.046+0000 7fd317ee3b80 1 -- 172.21.15.7:0/2905786117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f75d0 msgr2=0x55ce7b11dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:10.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.046+0000 7fd317ee3b80 1 --2- 172.21.15.7:0/2905786117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f75d0 0x55ce7b11dc60 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc006d80 tx=0x7fd2fc021000 comp rx=0 tx=0).stop 2024-01-27T02:37:10.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.046+0000 7fd317ee3b80 1 -- 172.21.15.7:0/2905786117 shutdown_connections 2024-01-27T02:37:10.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.046+0000 7fd317ee3b80 1 --2- 172.21.15.7:0/2905786117 >> [v2:172.21.15.7:6834/328992853,v1:172.21.15.7:6835/328992853] conn(0x55ce7b13d6b0 0x55ce7b13da80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.046+0000 7fd317ee3b80 1 --2- 172.21.15.7:0/2905786117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce7b140360 0x55ce7b125880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.046+0000 7fd317ee3b80 1 --2- 172.21.15.7:0/2905786117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f75d0 0x55ce7b11dc60 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.046+0000 7fd2e6ffd700 0 client.6423 ms_handle_reset on v2:172.21.15.7:6834/328992853 2024-01-27T02:37:10.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.046+0000 7fd317ee3b80 1 --2- 172.21.15.7:0/2905786117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ce7b0f8010 0x55ce7b123340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.046+0000 7fd317ee3b80 1 -- 172.21.15.7:0/2905786117 >> 172.21.15.7:0/2905786117 conn(0x55ce7b0f7a70 msgr2=0x55ce7b0f37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:10.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.046+0000 7fd317ee3b80 1 -- 172.21.15.7:0/2905786117 shutdown_connections 2024-01-27T02:37:10.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.046+0000 7fd317ee3b80 1 -- 172.21.15.7:0/2905786117 wait complete. 2024-01-27T02:37:10.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.046+0000 7fd317ee3b80 20 client.6423 trim_cache size 0 max 16384 2024-01-27T02:37:10.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.064+0000 7fd317ee3b80 1 Processor -- start 2024-01-27T02:37:10.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.064+0000 7fd317ee3b80 1 -- start start 2024-01-27T02:37:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.065+0000 7fd317ee3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce7b0f75d0 0x55ce7b121ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.065+0000 7fd317ee3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f8010 0x55ce7b14d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.065+0000 7fd317ee3b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ce7b140360 0x55ce7b14db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.065+0000 7fd317ee3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce7af3ee40 con 0x55ce7b0f8010 2024-01-27T02:37:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.065+0000 7fd317ee3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce7b0f0e50 con 0x55ce7b0f75d0 2024-01-27T02:37:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.065+0000 7fd317ee3b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce7b14ebb0 con 0x55ce7b140360 2024-01-27T02:37:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.065+0000 7fd307fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f8010 0x55ce7b14d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.065+0000 7fd307fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f8010 0x55ce7b14d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37634/0 (socket says 172.21.15.7:37634) 2024-01-27T02:37:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.065+0000 7fd307fff700 1 -- 172.21.15.7:0/2310594402 learned_addr learned my addr 172.21.15.7:0/2310594402 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.066+0000 7fd30c97c700 1 --2- 172.21.15.7:0/2310594402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce7b0f75d0 0x55ce7b121ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.066+0000 7fd30d17d700 1 --2- 172.21.15.7:0/2310594402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ce7b140360 0x55ce7b14db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.066+0000 7fd307fff700 1 -- 172.21.15.7:0/2310594402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ce7b140360 msgr2=0x55ce7b14db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.066+0000 7fd307fff700 1 --2- 172.21.15.7:0/2310594402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ce7b140360 0x55ce7b14db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.066+0000 7fd307fff700 1 -- 172.21.15.7:0/2310594402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce7b0f75d0 msgr2=0x55ce7b121ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.066+0000 7fd307fff700 1 --2- 172.21.15.7:0/2310594402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce7b0f75d0 0x55ce7b121ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.066+0000 7fd307fff700 1 -- 172.21.15.7:0/2310594402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce7aeddc80 con 0x55ce7b0f8010 2024-01-27T02:37:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.066+0000 7fd30c97c700 1 --2- 172.21.15.7:0/2310594402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce7b0f75d0 0x55ce7b121ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.066+0000 7fd30d17d700 1 --2- 172.21.15.7:0/2310594402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ce7b140360 0x55ce7b14db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.066+0000 7fd307fff700 1 --2- 172.21.15.7:0/2310594402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f8010 0x55ce7b14d650 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd30816d770 tx=0x7fd2f4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f608be5197ee8690 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.066+0000 7fd2f27fc700 1 -- 172.21.15.7:0/2310594402 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4022030 con 0x55ce7b0f8010 2024-01-27T02:37:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.066+0000 7fd2f27fc700 1 -- 172.21.15.7:0/2310594402 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f4017cf0 con 0x55ce7b0f8010 2024-01-27T02:37:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.066+0000 7fd2f27fc700 1 -- 172.21.15.7:0/2310594402 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4004a60 con 0x55ce7b0f8010 2024-01-27T02:37:10.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.067+0000 7fd317ee3b80 1 -- 172.21.15.7:0/2310594402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f8010 msgr2=0x55ce7b14d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:10.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.067+0000 7fd317ee3b80 1 --2- 172.21.15.7:0/2310594402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f8010 0x55ce7b14d650 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd30816d770 tx=0x7fd2f4000bc0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.067+0000 7fd317ee3b80 1 -- 172.21.15.7:0/2310594402 shutdown_connections 2024-01-27T02:37:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.067+0000 7fd317ee3b80 1 --2- 172.21.15.7:0/2310594402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ce7b140360 0x55ce7b14db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.067+0000 7fd317ee3b80 1 --2- 172.21.15.7:0/2310594402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce7b0f75d0 0x55ce7b121ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.067+0000 7fd317ee3b80 1 --2- 172.21.15.7:0/2310594402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f8010 0x55ce7b14d650 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.067+0000 7fd317ee3b80 1 -- 172.21.15.7:0/2310594402 >> 172.21.15.7:0/2310594402 conn(0x55ce7b0f7a70 msgr2=0x55ce7b1225f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.067+0000 7fd317ee3b80 1 -- 172.21.15.7:0/2310594402 shutdown_connections 2024-01-27T02:37:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.067+0000 7fd317ee3b80 1 -- 172.21.15.7:0/2310594402 wait complete. 2024-01-27T02:37:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.067+0000 7fd317ee3b80 1 -- start start 2024-01-27T02:37:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.068+0000 7fd317ee3b80 1 Processor -- start 2024-01-27T02:37:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.068+0000 7fd317ee3b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce7b0f75d0 0x55ce7b11dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.068+0000 7fd317ee3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f8010 0x55ce7b11e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.069+0000 7fd317ee3b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ce7b140360 0x55ce7b11b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.069+0000 7fd317ee3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce7b1a1280 con 0x55ce7b0f8010 2024-01-27T02:37:10.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.069+0000 7fd317ee3b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce7b0f7270 con 0x55ce7b0f75d0 2024-01-27T02:37:10.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.069+0000 7fd317ee3b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce7b13bdc0 con 0x55ce7b140360 2024-01-27T02:37:10.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.069+0000 7fd317ee3b80 10 client.0 _ll_register_callbacks cb 0x55ce7ae547b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-27T02:37:10.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.069+0000 7fd307fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f8010 0x55ce7b11e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.069+0000 7fd307fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f8010 0x55ce7b11e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37648/0 (socket says 172.21.15.7:37648) 2024-01-27T02:37:10.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.069+0000 7fd307fff700 1 -- 172.21.15.7:0/683832162 learned_addr learned my addr 172.21.15.7:0/683832162 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:10.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.069+0000 7fd30c97c700 1 --2- 172.21.15.7:0/683832162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce7b0f75d0 0x55ce7b11dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.069+0000 7fd30d17d700 1 --2- 172.21.15.7:0/683832162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ce7b140360 0x55ce7b11b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.069+0000 7fd307fff700 1 -- 172.21.15.7:0/683832162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ce7b140360 msgr2=0x55ce7b11b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:10.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.069+0000 7fd307fff700 1 --2- 172.21.15.7:0/683832162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ce7b140360 0x55ce7b11b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.069+0000 7fd307fff700 1 -- 172.21.15.7:0/683832162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce7b0f75d0 msgr2=0x55ce7b11dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:10.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.069+0000 7fd307fff700 1 --2- 172.21.15.7:0/683832162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce7b0f75d0 0x55ce7b11dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.069+0000 7fd307fff700 1 -- 172.21.15.7:0/683832162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f400f040 con 0x55ce7b0f8010 2024-01-27T02:37:10.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.069+0000 7fd307fff700 1 --2- 172.21.15.7:0/683832162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f8010 0x55ce7b11e1a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4005f70 tx=0x7fd2f401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4e0096a4dae44a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:10.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.069+0000 7fd2e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-27T02:37:10.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.069+0000 7fd2e6ffd700 1 -- 172.21.15.7:0/683832162 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f40051f0 con 0x55ce7b0f8010 2024-01-27T02:37:10.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.069+0000 7fd2e6ffd700 1 -- 172.21.15.7:0/683832162 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd2f4023070 con 0x55ce7b0f8010 2024-01-27T02:37:10.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.069+0000 7fd317ee3b80 1 -- 172.21.15.7:0/683832162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd300058b80 con 0x55ce7b0f8010 2024-01-27T02:37:10.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.070+0000 7fd2e6ffd700 1 -- 172.21.15.7:0/683832162 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2f4005560 con 0x55ce7b0f8010 2024-01-27T02:37:10.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.070+0000 7fd317ee3b80 10 client.6429 fetch_fsmap learned FSMap version 49 2024-01-27T02:37:10.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.070+0000 7fd317ee3b80 1 -- 172.21.15.7:0/683832162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x55ce7ae83fe0 con 0x55ce7b0f8010 2024-01-27T02:37:10.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.070+0000 7fd2e6ffd700 1 -- 172.21.15.7:0/683832162 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd300058b80 con 0x55ce7b0f8010 2024-01-27T02:37:10.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.070+0000 7fd2e6ffd700 1 -- 172.21.15.7:0/683832162 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 49) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd2f4033470 con 0x55ce7b0f8010 2024-01-27T02:37:10.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.070+0000 7fd317ee3b80 10 client.6429 fetch_fsmap finished waiting for FSMap version 49 2024-01-27T02:37:10.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.070+0000 7fd317ee3b80 10 client.6429 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-27T02:37:10.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.070+0000 7fd317ee3b80 20 client.6429 populate_metadata read hostname 'smithi007' 2024-01-27T02:37:10.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.070+0000 7fd317ee3b80 1 --2- 172.21.15.7:0/683832162 >> [v2:172.21.15.7:6834/328992853,v1:172.21.15.7:6835/328992853] conn(0x55ce7b11c1b0 0x55ce7b0f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.070+0000 7fd317ee3b80 4 client.6429 mds_command: new command op to 6297 tid=0[{"prefix": "status"}] 2024-01-27T02:37:10.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.070+0000 7fd317ee3b80 1 -- 172.21.15.7:0/683832162 --> [v2:172.21.15.7:6834/328992853,v1:172.21.15.7:6835/328992853] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ce7afe8c00 con 0x55ce7b11c1b0 2024-01-27T02:37:10.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.070+0000 7fd30c97c700 1 --2- 172.21.15.7:0/683832162 >> [v2:172.21.15.7:6834/328992853,v1:172.21.15.7:6835/328992853] conn(0x55ce7b11c1b0 0x55ce7b0f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.070+0000 7fd2e6ffd700 10 client.6429 ms_handle_connect on v2:172.21.15.7:6834/328992853 2024-01-27T02:37:10.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.070+0000 7fd30c97c700 1 --2- 172.21.15.7:0/683832162 >> [v2:172.21.15.7:6834/328992853,v1:172.21.15.7:6835/328992853] conn(0x55ce7b11c1b0 0x55ce7b0f5e60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64ccec47802692cf server_cookie=4424668a53019ba in_seq=0 out_seq=0 2024-01-27T02:37:10.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.071+0000 7fd2e6ffd700 1 -- 172.21.15.7:0/683832162 <== mds.0 v2:172.21.15.7:6834/328992853 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55ce7afe8c00 con 0x55ce7b11c1b0 2024-01-27T02:37:10.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.071+0000 7fd2e6ffd700 10 client.6429 handle_command_reply: tid=0 2024-01-27T02:37:10.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.071+0000 7fd317ee3b80 1 client.6429 shutdown 2024-01-27T02:37:10.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.071+0000 7fd317ee3b80 1 -- 172.21.15.7:0/683832162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f8010 msgr2=0x55ce7b11e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:10.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.071+0000 7fd317ee3b80 1 --2- 172.21.15.7:0/683832162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f8010 0x55ce7b11e1a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4005f70 tx=0x7fd2f401f000 comp rx=0 tx=0).stop 2024-01-27T02:37:10.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.071+0000 7fd317ee3b80 1 -- 172.21.15.7:0/683832162 shutdown_connections 2024-01-27T02:37:10.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.071+0000 7fd317ee3b80 1 --2- 172.21.15.7:0/683832162 >> [v2:172.21.15.7:6834/328992853,v1:172.21.15.7:6835/328992853] conn(0x55ce7b11c1b0 0x55ce7b0f5e60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.071+0000 7fd317ee3b80 1 --2- 172.21.15.7:0/683832162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ce7b140360 0x55ce7b11b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.071+0000 7fd317ee3b80 1 --2- 172.21.15.7:0/683832162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ce7b0f75d0 0x55ce7b11dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.071+0000 7fd317ee3b80 1 --2- 172.21.15.7:0/683832162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce7b0f8010 0x55ce7b11e1a0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.071+0000 7fd2e6ffd700 0 client.6429 ms_handle_reset on v2:172.21.15.7:6834/328992853 2024-01-27T02:37:10.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.071+0000 7fd317ee3b80 1 -- 172.21.15.7:0/683832162 >> 172.21.15.7:0/683832162 conn(0x55ce7b0f7a70 msgr2=0x55ce7b139930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:10.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.071+0000 7fd317ee3b80 1 -- 172.21.15.7:0/683832162 shutdown_connections 2024-01-27T02:37:10.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.071+0000 7fd317ee3b80 1 -- 172.21.15.7:0/683832162 wait complete. 2024-01-27T02:37:10.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.071+0000 7fd317ee3b80 20 client.6429 trim_cache size 0 max 16384 2024-01-27T02:37:10.077 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:37:10.077 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:37:10.077 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-27T02:37:10.077 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 6297, 2024-01-27T02:37:10.078 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-27T02:37:10.078 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-27T02:37:10.078 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-27T02:37:10.078 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.0907592609999996, 2024-01-27T02:37:10.078 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 49, 2024-01-27T02:37:10.078 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 98, 2024-01-27T02:37:10.078 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 98, 2024-01-27T02:37:10.078 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 11.934233157 2024-01-27T02:37:10.078 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:37:10.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.072+0000 7fd30ebe0700 1 -- 172.21.15.7:0/648717901 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd2ec066f30 msgr2=0x7fd2ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:10.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.072+0000 7fd30ebe0700 1 --2- 172.21.15.7:0/648717901 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd2ec066f30 0x7fd2ec0693e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc0030c0 tx=0x7fd2fc016000 comp rx=0 tx=0).stop 2024-01-27T02:37:10.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.072+0000 7fd30ebe0700 1 -- 172.21.15.7:0/648717901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd30815b5f0 msgr2=0x7fd30813e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:10.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.072+0000 7fd30ebe0700 1 --2- 172.21.15.7:0/648717901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd30815b5f0 0x7fd30813e600 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd30000c910 tx=0x7fd3000027a0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.072+0000 7fd30ebe0700 1 -- 172.21.15.7:0/648717901 shutdown_connections 2024-01-27T02:37:10.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.072+0000 7fd30ebe0700 1 --2- 172.21.15.7:0/648717901 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd2ec066f30 0x7fd2ec0693e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.072+0000 7fd30ebe0700 1 --2- 172.21.15.7:0/648717901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd30815b5f0 0x7fd30813e600 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.072+0000 7fd30ebe0700 1 --2- 172.21.15.7:0/648717901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3080b1c00 0x7fd30814b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.072+0000 7fd30ebe0700 1 --2- 172.21.15.7:0/648717901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3080b1fd0 0x7fd30814c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.072+0000 7fd30ebe0700 1 -- 172.21.15.7:0/648717901 >> 172.21.15.7:0/648717901 conn(0x7fd308068430 msgr2=0x7fd308157aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:10.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.072+0000 7fd30ebe0700 1 -- 172.21.15.7:0/648717901 shutdown_connections 2024-01-27T02:37:10.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.072+0000 7fd30ebe0700 1 -- 172.21.15.7:0/648717901 wait complete. 2024-01-27T02:37:10.086 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:37:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.217+0000 7feb9e142700 1 Processor -- start 2024-01-27T02:37:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.217+0000 7feb9e142700 1 -- start start 2024-01-27T02:37:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.218+0000 7feb9e142700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb980a7fe0 0x7feb9806f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.218+0000 7feb9e142700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7feb9815dfb0 0x7feb9806f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.218+0000 7feb9e142700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb9815a880 con 0x7feb980a7c10 2024-01-27T02:37:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.218+0000 7feb9e142700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb980afde0 con 0x7feb9815dfb0 2024-01-27T02:37:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.218+0000 7feb9e142700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb980af950 con 0x7feb980a7fe0 2024-01-27T02:37:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.218+0000 7feb97fff700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7feb9815dfb0 0x7feb9806f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.218+0000 7feb96ffd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb980a7fe0 0x7feb9806f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.218+0000 7feb97fff700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7feb9815dfb0 0x7feb9806f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:32776/0 (socket says 172.21.15.7:32776) 2024-01-27T02:37:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.218+0000 7feb977fe700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7feb980a7c10 0x7feb980ae340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:55632/0 (socket says 172.21.15.7:55632) 2024-01-27T02:37:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.218+0000 7feb97fff700 1 -- 172.21.15.7:0/2068741517 learned_addr learned my addr 172.21.15.7:0/2068741517 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.219+0000 7feb96ffd700 1 -- 172.21.15.7:0/2068741517 >> v2:172.21.15.188:3300/0 conn(0x7feb9815dfb0 msgr2=0x7feb9806f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.219+0000 7feb96ffd700 1 --2- 172.21.15.7:0/2068741517 >> v2:172.21.15.188:3300/0 conn(0x7feb9815dfb0 0x7feb9806f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.219+0000 7feb96ffd700 1 -- 172.21.15.7:0/2068741517 >> v1:172.21.15.188:6789/0 conn(0x7feb980a7c10 legacy=0x7feb980ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.219+0000 7feb967fc700 1 -- 172.21.15.7:0/2068741517 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 642551622 0 0) 0x7feb9815a880 con 0x7feb980a7c10 2024-01-27T02:37:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.219+0000 7feb96ffd700 1 -- 172.21.15.7:0/2068741517 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb980aed40 con 0x7feb980a7fe0 2024-01-27T02:37:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.219+0000 7feb97fff700 1 --2- 172.21.15.7:0/2068741517 >> v2:172.21.15.188:3300/0 conn(0x7feb9815dfb0 0x7feb9806f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:37:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.219+0000 7feb96ffd700 1 --2- 172.21.15.7:0/2068741517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb980a7fe0 0x7feb9806f360 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0098e0 tx=0x7feb8c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e977d0d54c831e2f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.219+0000 7feb967fc700 1 -- 172.21.15.7:0/2068741517 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8c00b640 con 0x7feb980a7fe0 2024-01-27T02:37:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.219+0000 7feb967fc700 1 -- 172.21.15.7:0/2068741517 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8c00b7a0 con 0x7feb980a7fe0 2024-01-27T02:37:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.219+0000 7feb967fc700 1 -- 172.21.15.7:0/2068741517 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8c00bb40 con 0x7feb980a7fe0 2024-01-27T02:37:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.220+0000 7feb9e142700 1 -- 172.21.15.7:0/2068741517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb980a7fe0 msgr2=0x7feb9806f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.220+0000 7feb9e142700 1 --2- 172.21.15.7:0/2068741517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb980a7fe0 0x7feb9806f360 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0098e0 tx=0x7feb8c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.220+0000 7feb9e142700 1 -- 172.21.15.7:0/2068741517 shutdown_connections 2024-01-27T02:37:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.220+0000 7feb9e142700 1 --2- 172.21.15.7:0/2068741517 >> v2:172.21.15.188:3300/0 conn(0x7feb9815dfb0 0x7feb9806f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.220+0000 7feb9e142700 1 --2- 172.21.15.7:0/2068741517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb980a7fe0 0x7feb9806f360 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.220+0000 7feb9e142700 1 -- 172.21.15.7:0/2068741517 >> 172.21.15.7:0/2068741517 conn(0x7feb980679b0 msgr2=0x7feb98067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.220+0000 7feb9e142700 1 -- 172.21.15.7:0/2068741517 shutdown_connections 2024-01-27T02:37:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.220+0000 7feb9e142700 1 -- 172.21.15.7:0/2068741517 wait complete. 2024-01-27T02:37:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.221+0000 7feb9e142700 1 Processor -- start 2024-01-27T02:37:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.221+0000 7feb9e142700 1 -- start start 2024-01-27T02:37:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.222+0000 7feb9e142700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb980a7c10 0x7feb98153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.222+0000 7feb9e142700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb980a7fe0 0x7feb9814ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.222+0000 7feb9e142700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb9815dfb0 0x7feb98099d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.222+0000 7feb9e142700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9816f020 con 0x7feb9815dfb0 2024-01-27T02:37:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.222+0000 7feb9e142700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb98151a10 con 0x7feb980a7c10 2024-01-27T02:37:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.222+0000 7feb9e142700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb98151120 con 0x7feb980a7fe0 2024-01-27T02:37:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.222+0000 7feb97fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb9815dfb0 0x7feb98099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.222+0000 7feb97fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb9815dfb0 0x7feb98099d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37662/0 (socket says 172.21.15.7:37662) 2024-01-27T02:37:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.222+0000 7feb97fff700 1 -- 172.21.15.7:0/2532674073 learned_addr learned my addr 172.21.15.7:0/2532674073 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.223+0000 7feb977fe700 1 --2- 172.21.15.7:0/2532674073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb980a7c10 0x7feb98153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.223+0000 7feb96ffd700 1 --2- 172.21.15.7:0/2532674073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb980a7fe0 0x7feb9814ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.223+0000 7feb97fff700 1 -- 172.21.15.7:0/2532674073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb980a7fe0 msgr2=0x7feb9814ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.223+0000 7feb97fff700 1 --2- 172.21.15.7:0/2532674073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb980a7fe0 0x7feb9814ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.223+0000 7feb97fff700 1 -- 172.21.15.7:0/2532674073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb980a7c10 msgr2=0x7feb98153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.223+0000 7feb97fff700 1 --2- 172.21.15.7:0/2532674073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb980a7c10 0x7feb98153910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.223+0000 7feb97fff700 1 -- 172.21.15.7:0/2532674073 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb90009ae0 con 0x7feb9815dfb0 2024-01-27T02:37:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.223+0000 7feb97fff700 1 --2- 172.21.15.7:0/2532674073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb9815dfb0 0x7feb98099d30 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7feb90009f20 tx=0x7feb9000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.223+0000 7feb7f7fe700 1 -- 172.21.15.7:0/2532674073 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9000c960 con 0x7feb9815dfb0 2024-01-27T02:37:10.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.223+0000 7feb7f7fe700 1 -- 172.21.15.7:0/2532674073 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9000f040 con 0x7feb9815dfb0 2024-01-27T02:37:10.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.223+0000 7feb9e142700 1 -- 172.21.15.7:0/2532674073 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb8c009420 con 0x7feb9815dfb0 2024-01-27T02:37:10.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.223+0000 7feb7f7fe700 1 -- 172.21.15.7:0/2532674073 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9000cd30 con 0x7feb9815dfb0 2024-01-27T02:37:10.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.223+0000 7feb9e142700 1 -- 172.21.15.7:0/2532674073 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb981513c0 con 0x7feb9815dfb0 2024-01-27T02:37:10.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.224+0000 7feb9e142700 1 -- 172.21.15.7:0/2532674073 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb980aea50 con 0x7feb9815dfb0 2024-01-27T02:37:10.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.225+0000 7feb7f7fe700 1 -- 172.21.15.7:0/2532674073 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7feb90014650 con 0x7feb9815dfb0 2024-01-27T02:37:10.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.225+0000 7feb7f7fe700 1 --2- 172.21.15.7:0/2532674073 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7feb84066f30 0x7feb840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:10.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.225+0000 7feb7f7fe700 1 -- 172.21.15.7:0/2532674073 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feb90084c60 con 0x7feb9815dfb0 2024-01-27T02:37:10.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.226+0000 7feb977fe700 1 --2- 172.21.15.7:0/2532674073 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7feb84066f30 0x7feb840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:10.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.226+0000 7feb977fe700 1 --2- 172.21.15.7:0/2532674073 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7feb84066f30 0x7feb840693e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7feb88007fd0 tx=0x7feb88007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.230+0000 7feb7f7fe700 1 -- 172.21.15.7:0/2532674073 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feb90051e30 con 0x7feb9815dfb0 2024-01-27T02:37:10.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.480+0000 7feb9e142700 1 -- 172.21.15.7:0/2532674073 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7feb98159e90 con 0x7feb9815dfb0 2024-01-27T02:37:10.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.480+0000 7feb7f7fe700 1 -- 172.21.15.7:0/2532674073 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) v1 ==== 53+0+73 (secure 0 0 0) 0x7feb90055450 con 0x7feb9815dfb0 2024-01-27T02:37:10.481 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:37:10.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.482+0000 7feb9e142700 1 -- 172.21.15.7:0/2532674073 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7feb84066f30 msgr2=0x7feb840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:10.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.482+0000 7feb9e142700 1 --2- 172.21.15.7:0/2532674073 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7feb84066f30 0x7feb840693e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7feb88007fd0 tx=0x7feb88007d50 comp rx=0 tx=0).stop 2024-01-27T02:37:10.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.482+0000 7feb9e142700 1 -- 172.21.15.7:0/2532674073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb9815dfb0 msgr2=0x7feb98099d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:10.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.482+0000 7feb9e142700 1 --2- 172.21.15.7:0/2532674073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb9815dfb0 0x7feb98099d30 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7feb90009f20 tx=0x7feb9000b040 comp rx=0 tx=0).stop 2024-01-27T02:37:10.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.482+0000 7feb9e142700 1 -- 172.21.15.7:0/2532674073 shutdown_connections 2024-01-27T02:37:10.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.482+0000 7feb9e142700 1 --2- 172.21.15.7:0/2532674073 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7feb84066f30 0x7feb840693e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.482+0000 7feb9e142700 1 --2- 172.21.15.7:0/2532674073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb9815dfb0 0x7feb98099d30 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.482+0000 7feb9e142700 1 --2- 172.21.15.7:0/2532674073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb980a7c10 0x7feb98153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.482+0000 7feb9e142700 1 --2- 172.21.15.7:0/2532674073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb980a7fe0 0x7feb9814ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:10.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.482+0000 7feb9e142700 1 -- 172.21.15.7:0/2532674073 >> 172.21.15.7:0/2532674073 conn(0x7feb980679b0 msgr2=0x7feb9806e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:10.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.482+0000 7feb9e142700 1 -- 172.21.15.7:0/2532674073 shutdown_connections 2024-01-27T02:37:10.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:10.482+0000 7feb9e142700 1 -- 172.21.15.7:0/2532674073 wait complete. 2024-01-27T02:37:10.496 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-27T02:37:10.497 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:37:10.497 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-27T02:37:10.497 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-27T02:37:10.497 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-27T02:37:10.497 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-27T02:37:10.497 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-27T02:37:10.497 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:37:10.497 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:37:10.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:37:10.497 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-27T02:37:10.517 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:37:10.517 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:37:10.517 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:37:10.517 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:37:10.518 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-27T02:37:10.518 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:37:10.518 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:37:10.518 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-27T02:37:10.518 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-27T02:37:10.518 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:37:10.518 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-27T02:37:10.518 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-27T02:37:10.518 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:37:10.518 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-27T02:37:10.518 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-27T02:37:10.518 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-27T02:37:10.518 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42081sec preferred_lft 42081sec 2024-01-27T02:37:10.518 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-27T02:37:10.519 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:37:10.519 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:37:10.519 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-27T02:37:10.519 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-27T02:37:10.519 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:37:10.519 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 4a:57:d9:3c:c5:a0 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:37:10.519 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-27T02:37:10.519 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:37:10.519 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::2477:9eff:fe39:433c/64 scope link 2024-01-27T02:37:10.519 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:37:10.519 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:37:10.519 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 4a:57:d9:3c:c5:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-27T02:37:10.519 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::4857:d9ff:fe3c:c5a0/64 scope link 2024-01-27T02:37:10.519 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:37:10.519 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:37:10.520 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ca:6e:b8:df:53:bf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-27T02:37:10.520 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c86e:b8ff:fedf:53bf/64 scope link 2024-01-27T02:37:10.520 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:37:10.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:37:10.520 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:37:10.588 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-27T02:37:10.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:37:10.590 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-27T02:37:10.654 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-27T02:37:10.654 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:37:10.654 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-27T02:37:10.655 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-27T02:37:10.655 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:37:10.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:37:10.655 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-27T02:37:10.655 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-27T02:37:10.655 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-27T02:37:10.655 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-27T02:37:10.655 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-27T02:37:10.655 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-27T02:37:10.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:37:10.779 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-27T02:37:10.799 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-27T02:37:10.799 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-27T02:37:10.799 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:37:10.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:37:10.800 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-27T02:37:10.877 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:37:10.877 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-27T02:37:10.944 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:37:10.944 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:37:10.944 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-27T02:37:11.009 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:37:11.009 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:37:11.009 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:37:11.109 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:37:11.109 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:37:11.109 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:37:11.119 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:37:11.119 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:37:11.119 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:37:11.119 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:37:11.119 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:37:11.136 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:37:11.136 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:37:11.137 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:37:11.137 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:37:11.137 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:37:11.137 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:37:11.137 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:37:11.137 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:37:11.139 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:37:11.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:37:11.139 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-27T02:37:11.171 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:37:11.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.351+0000 7f8e20edf700 1 Processor -- start 2024-01-27T02:37:11.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.351+0000 7f8e20edf700 1 -- start start 2024-01-27T02:37:11.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.352+0000 7f8e20edf700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e1c0a7c10 0x7f8e1c0ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.352+0000 7f8e20edf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8e1c0a7fe0 0x7f8e1c0b0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.352+0000 7f8e20edf700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e1c15a880 con 0x7f8e1c0ae880 2024-01-27T02:37:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.352+0000 7f8e20edf700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e1c0afde0 con 0x7f8e1c0a7fe0 2024-01-27T02:37:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.352+0000 7f8e20edf700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c0af950 con 0x7f8e1c0a7c10 2024-01-27T02:37:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.352+0000 7f8e1ad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8e1c0a7fe0 0x7f8e1c0b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.352+0000 7f8e1ad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8e1c0a7fe0 0x7f8e1c0b0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37674/0 (socket says 172.21.15.7:37674) 2024-01-27T02:37:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.352+0000 7f8e1ad9d700 1 -- 172.21.15.7:0/3441521695 learned_addr learned my addr 172.21.15.7:0/3441521695 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.352+0000 7f8e19d9b700 1 --1- 172.21.15.7:0/3441521695 >> v1:172.21.15.7:6789/0 conn(0x7f8e1c0ae880 0x7f8e1c06f360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48230/0 (socket says 172.21.15.7:48230) 2024-01-27T02:37:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.353+0000 7f8e1a59c700 1 --2- 172.21.15.7:0/3441521695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e1c0a7c10 0x7f8e1c0ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.353+0000 7f8e1ad9d700 1 -- 172.21.15.7:0/3441521695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e1c0a7c10 msgr2=0x7f8e1c0ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.353+0000 7f8e1ad9d700 1 --2- 172.21.15.7:0/3441521695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e1c0a7c10 0x7f8e1c0ae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.353+0000 7f8e1ad9d700 1 -- 172.21.15.7:0/3441521695 >> v1:172.21.15.7:6789/0 conn(0x7f8e1c0ae880 legacy=0x7f8e1c06f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.353+0000 7f8e1ad9d700 1 -- 172.21.15.7:0/3441521695 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c0aed40 con 0x7f8e1c0a7fe0 2024-01-27T02:37:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.353+0000 7f8e1a59c700 1 --2- 172.21.15.7:0/3441521695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e1c0a7c10 0x7f8e1c0ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.353+0000 7f8e1ad9d700 1 --2- 172.21.15.7:0/3441521695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e1c0a7fe0 0x7f8e1c0b0180 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8e14009f20 tx=0x7f8e14009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab34914e638fe8ff server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.353+0000 7f8e1959a700 1 -- 172.21.15.7:0/3441521695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1400e070 con 0x7f8e1c0a7fe0 2024-01-27T02:37:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.353+0000 7f8e1959a700 1 -- 172.21.15.7:0/3441521695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1400ba80 con 0x7f8e1c0a7fe0 2024-01-27T02:37:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.353+0000 7f8e1959a700 1 -- 172.21.15.7:0/3441521695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e140092e0 con 0x7f8e1c0a7fe0 2024-01-27T02:37:11.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.354+0000 7f8e20edf700 1 -- 172.21.15.7:0/3441521695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e1c0a7fe0 msgr2=0x7f8e1c0b0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:11.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.354+0000 7f8e20edf700 1 --2- 172.21.15.7:0/3441521695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e1c0a7fe0 0x7f8e1c0b0180 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8e14009f20 tx=0x7f8e14009840 comp rx=0 tx=0).stop 2024-01-27T02:37:11.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.354+0000 7f8e20edf700 1 -- 172.21.15.7:0/3441521695 shutdown_connections 2024-01-27T02:37:11.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.354+0000 7f8e20edf700 1 --2- 172.21.15.7:0/3441521695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e1c0a7fe0 0x7f8e1c0b0180 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:11.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.354+0000 7f8e20edf700 1 --2- 172.21.15.7:0/3441521695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e1c0a7c10 0x7f8e1c0ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:11.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.354+0000 7f8e20edf700 1 -- 172.21.15.7:0/3441521695 >> 172.21.15.7:0/3441521695 conn(0x7f8e1c0679b0 msgr2=0x7f8e1c067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:11.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.354+0000 7f8e20edf700 1 -- 172.21.15.7:0/3441521695 shutdown_connections 2024-01-27T02:37:11.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.354+0000 7f8e20edf700 1 -- 172.21.15.7:0/3441521695 wait complete. 2024-01-27T02:37:11.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.355+0000 7f8e20edf700 1 Processor -- start 2024-01-27T02:37:11.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.355+0000 7f8e20edf700 1 -- start start 2024-01-27T02:37:11.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.356+0000 7f8e20edf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e1c0a7c10 0x7f8e1c1567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:11.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.356+0000 7f8e20edf700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e1c0a7fe0 0x7f8e1c150080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:11.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.356+0000 7f8e20edf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e1c0ae880 0x7f8e1c1558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:11.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.356+0000 7f8e20edf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c1505c0 con 0x7f8e1c0a7c10 2024-01-27T02:37:11.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.356+0000 7f8e20edf700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c16db00 con 0x7f8e1c0ae880 2024-01-27T02:37:11.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.356+0000 7f8e20edf700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c16dde0 con 0x7f8e1c0a7fe0 2024-01-27T02:37:11.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.356+0000 7f8e1a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e1c0a7c10 0x7f8e1c1567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:11.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.356+0000 7f8e1a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e1c0a7c10 0x7f8e1c1567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37684/0 (socket says 172.21.15.7:37684) 2024-01-27T02:37:11.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.356+0000 7f8e1a59c700 1 -- 172.21.15.7:0/1841664887 learned_addr learned my addr 172.21.15.7:0/1841664887 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:11.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.356+0000 7f8e19d9b700 1 --2- 172.21.15.7:0/1841664887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e1c0a7fe0 0x7f8e1c150080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:11.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.356+0000 7f8e1a59c700 1 -- 172.21.15.7:0/1841664887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e1c0a7fe0 msgr2=0x7f8e1c150080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:11.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.356+0000 7f8e1ad9d700 1 --2- 172.21.15.7:0/1841664887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e1c0ae880 0x7f8e1c1558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:11.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.356+0000 7f8e1a59c700 1 --2- 172.21.15.7:0/1841664887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e1c0a7fe0 0x7f8e1c150080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:11.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.356+0000 7f8e1a59c700 1 -- 172.21.15.7:0/1841664887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e1c0ae880 msgr2=0x7f8e1c1558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:11.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.356+0000 7f8e1a59c700 1 --2- 172.21.15.7:0/1841664887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e1c0ae880 0x7f8e1c1558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:11.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.356+0000 7f8e1a59c700 1 -- 172.21.15.7:0/1841664887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e14009ae0 con 0x7f8e1c0a7c10 2024-01-27T02:37:11.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.356+0000 7f8e1ad9d700 1 --2- 172.21.15.7:0/1841664887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e1c0ae880 0x7f8e1c1558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:11.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.356+0000 7f8e1a59c700 1 --2- 172.21.15.7:0/1841664887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e1c0a7c10 0x7f8e1c1567e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00c830 tx=0x7f8e0c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:11.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.357+0000 7f8e0a7fc700 1 -- 172.21.15.7:0/1841664887 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c011600 con 0x7f8e1c0a7c10 2024-01-27T02:37:11.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.357+0000 7f8e0a7fc700 1 -- 172.21.15.7:0/1841664887 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c011760 con 0x7f8e1c0a7c10 2024-01-27T02:37:11.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.357+0000 7f8e20edf700 1 -- 172.21.15.7:0/1841664887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e1c16e0c0 con 0x7f8e1c0a7c10 2024-01-27T02:37:11.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.357+0000 7f8e0a7fc700 1 -- 172.21.15.7:0/1841664887 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c011b20 con 0x7f8e1c0a7c10 2024-01-27T02:37:11.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.357+0000 7f8e20edf700 1 -- 172.21.15.7:0/1841664887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1c16e680 con 0x7f8e1c0a7c10 2024-01-27T02:37:11.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.358+0000 7f8e20edf700 1 -- 172.21.15.7:0/1841664887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e1c04c710 con 0x7f8e1c0a7c10 2024-01-27T02:37:11.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.358+0000 7f8e0a7fc700 1 -- 172.21.15.7:0/1841664887 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8e0c00a3e0 con 0x7f8e1c0a7c10 2024-01-27T02:37:11.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.358+0000 7f8e0a7fc700 1 --2- 172.21.15.7:0/1841664887 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8dfc066f30 0x7f8dfc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:11.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.359+0000 7f8e0a7fc700 1 -- 172.21.15.7:0/1841664887 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8e0c0849f0 con 0x7f8e1c0a7c10 2024-01-27T02:37:11.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.359+0000 7f8e19d9b700 1 --2- 172.21.15.7:0/1841664887 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8dfc066f30 0x7f8dfc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:11.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.359+0000 7f8e19d9b700 1 --2- 172.21.15.7:0/1841664887 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8dfc066f30 0x7f8dfc0693e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c156d20 tx=0x7f8e10009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:11.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.363+0000 7f8e0a7fc700 1 -- 172.21.15.7:0/1841664887 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8e0c051c60 con 0x7f8e1c0a7c10 2024-01-27T02:37:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.617+0000 7f8e20edf700 1 -- 172.21.15.7:0/1841664887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8e1c0af020 con 0x7f8e1c0a7c10 2024-01-27T02:37:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.617+0000 7f8e0a7fc700 1 -- 172.21.15.7:0/1841664887 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8e0c015070 con 0x7f8e1c0a7c10 2024-01-27T02:37:11.618 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:37:11.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.619+0000 7f8e20edf700 1 -- 172.21.15.7:0/1841664887 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8dfc066f30 msgr2=0x7f8dfc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:11.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.619+0000 7f8e20edf700 1 --2- 172.21.15.7:0/1841664887 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8dfc066f30 0x7f8dfc0693e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c156d20 tx=0x7f8e10009040 comp rx=0 tx=0).stop 2024-01-27T02:37:11.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.619+0000 7f8e20edf700 1 -- 172.21.15.7:0/1841664887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e1c0a7c10 msgr2=0x7f8e1c1567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:11.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.619+0000 7f8e20edf700 1 --2- 172.21.15.7:0/1841664887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e1c0a7c10 0x7f8e1c1567e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00c830 tx=0x7f8e0c008040 comp rx=0 tx=0).stop 2024-01-27T02:37:11.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.619+0000 7f8e20edf700 1 -- 172.21.15.7:0/1841664887 shutdown_connections 2024-01-27T02:37:11.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.619+0000 7f8e20edf700 1 --2- 172.21.15.7:0/1841664887 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8dfc066f30 0x7f8dfc0693e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:11.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.619+0000 7f8e20edf700 1 --2- 172.21.15.7:0/1841664887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e1c0ae880 0x7f8e1c1558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:11.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.619+0000 7f8e20edf700 1 --2- 172.21.15.7:0/1841664887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e1c0a7c10 0x7f8e1c1567e0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:11.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.619+0000 7f8e20edf700 1 --2- 172.21.15.7:0/1841664887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e1c0a7fe0 0x7f8e1c150080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:11.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.620+0000 7f8e20edf700 1 -- 172.21.15.7:0/1841664887 >> 172.21.15.7:0/1841664887 conn(0x7f8e1c0679b0 msgr2=0x7f8e1c06dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:11.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.620+0000 7f8e20edf700 1 -- 172.21.15.7:0/1841664887 shutdown_connections 2024-01-27T02:37:11.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:11.620+0000 7f8e20edf700 1 -- 172.21.15.7:0/1841664887 wait complete. 2024-01-27T02:37:11.633 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-27T02:37:11.633 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:37:11.633 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-01-27T02:37:11.633 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-27T02:37:11.634 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-27T02:37:11.634 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-27T02:37:11.634 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-27T02:37:11.634 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:37:11.634 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:37:11.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:37:11.634 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-01-27T02:37:11.656 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:37:11.656 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:37:11.656 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:37:11.656 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:37:11.656 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-01-27T02:37:11.656 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:37:11.657 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:37:11.657 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:37:11.657 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-01-27T02:37:11.657 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:37:11.657 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:37:11.657 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-01-27T02:37:11.657 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:37:11.657 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:37:11.657 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:37:11.657 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:37:11.657 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-27T02:37:11.657 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42087sec preferred_lft 42087sec 2024-01-27T02:37:11.657 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b1:d6f7:baf0:bc17/64 scope link noprefixroute 2024-01-27T02:37:11.657 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:37:11.658 INFO:teuthology.orchestra.run.smithi188.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:37:11.658 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b0:92:71:90:9d brd ff:ff:ff:ff:ff:ff 2024-01-27T02:37:11.658 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-27T02:37:11.658 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:37:11.658 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::10f8:32ff:feed:a752/64 scope link 2024-01-27T02:37:11.658 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:37:11.658 INFO:teuthology.orchestra.run.smithi188.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:37:11.658 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b0:92:71:90:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-27T02:37:11.658 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b8b0:92ff:fe71:909d/64 scope link 2024-01-27T02:37:11.658 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:37:11.658 INFO:teuthology.orchestra.run.smithi188.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:37:11.658 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ee:78:03:20:3a:a9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-27T02:37:11.658 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec78:3ff:fe20:3aa9/64 scope link 2024-01-27T02:37:11.658 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:37:11.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:37:11.659 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:37:11.729 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-01-27T02:37:11.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:37:11.731 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-01-27T02:37:11.793 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-01-27T02:37:11.793 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:37:11.793 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-27T02:37:11.793 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-27T02:37:11.793 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:37:11.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:37:11.794 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-27T02:37:11.794 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-27T02:37:11.794 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-27T02:37:11.794 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-27T02:37:11.794 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-27T02:37:11.794 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-27T02:37:11.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:37:11.906 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-01-27T02:37:11.925 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-27T02:37:11.925 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-27T02:37:11.926 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:37:11.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:37:11.926 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2024-01-27T02:37:11.996 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:37:11.996 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-27T02:37:12.053 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:37:12.053 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:37:12.054 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-27T02:37:12.111 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:37:12.111 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:37:12.111 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:37:12.193 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:37:12.193 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:37:12.193 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:37:12.202 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:37:12.202 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:37:12.202 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:37:12.202 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:37:12.202 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:37:12.220 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:37:12.220 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:37:12.220 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:37:12.221 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:37:12.221 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:37:12.221 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:37:12.221 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:37:12.221 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:37:12.222 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:37:12.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:37:12.223 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-27T02:37:12.290 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:37:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.430+0000 7fd3ffd45700 1 Processor -- start 2024-01-27T02:37:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.430+0000 7fd3ffd45700 1 -- start start 2024-01-27T02:37:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.431+0000 7fd3ffd45700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd3f8147d50 0x7fd3f80c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.431+0000 7fd3ffd45700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd3f8148120 0x7fd3f80c8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.431+0000 7fd3ffd45700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f81630c0 con 0x7fd3f80c89c0 2024-01-27T02:37:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.431+0000 7fd3ffd45700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f80c8e20 con 0x7fd3f8147d50 2024-01-27T02:37:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.431+0000 7fd3ffd45700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f80c3970 con 0x7fd3f8148120 2024-01-27T02:37:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.431+0000 7fd3fdae1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd3f8147d50 0x7fd3f80c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.431+0000 7fd3fdae1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd3f8147d50 0x7fd3f80c3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53440/0 (socket says 172.21.15.7:53440) 2024-01-27T02:37:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.431+0000 7fd3fdae1700 1 -- 172.21.15.7:0/752951549 learned_addr learned my addr 172.21.15.7:0/752951549 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.431+0000 7fd3fd2e0700 1 --2- 172.21.15.7:0/752951549 >> v2:172.21.15.188:3300/0 conn(0x7fd3f8148120 0x7fd3f80c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.432+0000 7fd3fdae1700 1 -- 172.21.15.7:0/752951549 >> v2:172.21.15.188:3300/0 conn(0x7fd3f8148120 msgr2=0x7fd3f80c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.432+0000 7fd3fdae1700 1 --2- 172.21.15.7:0/752951549 >> v2:172.21.15.188:3300/0 conn(0x7fd3f8148120 0x7fd3f80c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.432+0000 7fd3fcadf700 1 -- 172.21.15.7:0/752951549 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 86637292 0 0) 0x7fd3f81630c0 con 0x7fd3f80c89c0 2024-01-27T02:37:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.432+0000 7fd3fdae1700 1 -- 172.21.15.7:0/752951549 >> v1:172.21.15.188:6789/0 conn(0x7fd3f80c89c0 legacy=0x7fd3f8063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.432+0000 7fd3fdae1700 1 -- 172.21.15.7:0/752951549 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f80c3040 con 0x7fd3f8147d50 2024-01-27T02:37:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.432+0000 7fd3fdae1700 1 --2- 172.21.15.7:0/752951549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3f8147d50 0x7fd3f80c3e00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80b0770 tx=0x7fd3f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6af167ee75c7c8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:12.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.432+0000 7fd3fcadf700 1 -- 172.21.15.7:0/752951549 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4009e70 con 0x7fd3f8147d50 2024-01-27T02:37:12.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.432+0000 7fd3fcadf700 1 -- 172.21.15.7:0/752951549 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f400b860 con 0x7fd3f8147d50 2024-01-27T02:37:12.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.432+0000 7fd3fcadf700 1 -- 172.21.15.7:0/752951549 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f400bc00 con 0x7fd3f8147d50 2024-01-27T02:37:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.433+0000 7fd3ffd45700 1 -- 172.21.15.7:0/752951549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3f8147d50 msgr2=0x7fd3f80c3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.433+0000 7fd3ffd45700 1 --2- 172.21.15.7:0/752951549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3f8147d50 0x7fd3f80c3e00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80b0770 tx=0x7fd3f4009250 comp rx=0 tx=0).stop 2024-01-27T02:37:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.433+0000 7fd3ffd45700 1 -- 172.21.15.7:0/752951549 shutdown_connections 2024-01-27T02:37:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.433+0000 7fd3ffd45700 1 --2- 172.21.15.7:0/752951549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3f8147d50 0x7fd3f80c3e00 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.433+0000 7fd3ffd45700 1 --2- 172.21.15.7:0/752951549 >> v2:172.21.15.188:3300/0 conn(0x7fd3f8148120 0x7fd3f80c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.433+0000 7fd3ffd45700 1 -- 172.21.15.7:0/752951549 >> 172.21.15.7:0/752951549 conn(0x7fd3f80cdc30 msgr2=0x7fd3f80ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.433+0000 7fd3ffd45700 1 -- 172.21.15.7:0/752951549 shutdown_connections 2024-01-27T02:37:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.433+0000 7fd3ffd45700 1 -- 172.21.15.7:0/752951549 wait complete. 2024-01-27T02:37:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.434+0000 7fd3ffd45700 1 Processor -- start 2024-01-27T02:37:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.434+0000 7fd3ffd45700 1 -- start start 2024-01-27T02:37:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.435+0000 7fd3ffd45700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3f80c89c0 0x7fd3f80b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.435+0000 7fd3ffd45700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3f8147d50 0x7fd3f816faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.435+0000 7fd3ffd45700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3f8148120 0x7fd3f8146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.435+0000 7fd3ffd45700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f80b0ee0 con 0x7fd3f8147d50 2024-01-27T02:37:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.435+0000 7fd3ffd45700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f80b2d90 con 0x7fd3f8148120 2024-01-27T02:37:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.435+0000 7fd3ffd45700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f80b27d0 con 0x7fd3f80c89c0 2024-01-27T02:37:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.435+0000 7fd3fd2e0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3f8147d50 0x7fd3f816faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.435+0000 7fd3fd2e0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3f8147d50 0x7fd3f816faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53446/0 (socket says 172.21.15.7:53446) 2024-01-27T02:37:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.435+0000 7fd3fd2e0700 1 -- 172.21.15.7:0/2150420855 learned_addr learned my addr 172.21.15.7:0/2150420855 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.435+0000 7fd3fe2e2700 1 --2- 172.21.15.7:0/2150420855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3f8148120 0x7fd3f8146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.435+0000 7fd3fdae1700 1 --2- 172.21.15.7:0/2150420855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3f80c89c0 0x7fd3f80b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.435+0000 7fd3fd2e0700 1 -- 172.21.15.7:0/2150420855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3f80c89c0 msgr2=0x7fd3f80b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.435+0000 7fd3fd2e0700 1 --2- 172.21.15.7:0/2150420855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3f80c89c0 0x7fd3f80b21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.435+0000 7fd3fd2e0700 1 -- 172.21.15.7:0/2150420855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3f8148120 msgr2=0x7fd3f8146a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.435+0000 7fd3fd2e0700 1 --2- 172.21.15.7:0/2150420855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3f8148120 0x7fd3f8146a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:12.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.435+0000 7fd3fd2e0700 1 -- 172.21.15.7:0/2150420855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f40094f0 con 0x7fd3f8147d50 2024-01-27T02:37:12.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.436+0000 7fd3fd2e0700 1 --2- 172.21.15.7:0/2150420855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3f8147d50 0x7fd3f816faf0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800ec80 tx=0x7fd3e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:12.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.436+0000 7fd3edffb700 1 -- 172.21.15.7:0/2150420855 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e800c850 con 0x7fd3f8147d50 2024-01-27T02:37:12.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.436+0000 7fd3edffb700 1 -- 172.21.15.7:0/2150420855 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e800c9b0 con 0x7fd3f8147d50 2024-01-27T02:37:12.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.436+0000 7fd3ffd45700 1 -- 172.21.15.7:0/2150420855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f80b2ab0 con 0x7fd3f8147d50 2024-01-27T02:37:12.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.436+0000 7fd3edffb700 1 -- 172.21.15.7:0/2150420855 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e800cd50 con 0x7fd3f8147d50 2024-01-27T02:37:12.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.436+0000 7fd3ffd45700 1 -- 172.21.15.7:0/2150420855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f8146fc0 con 0x7fd3f8147d50 2024-01-27T02:37:12.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.437+0000 7fd3ffd45700 1 -- 172.21.15.7:0/2150420855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f80b09d0 con 0x7fd3f8147d50 2024-01-27T02:37:12.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.438+0000 7fd3edffb700 1 -- 172.21.15.7:0/2150420855 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd3e8013680 con 0x7fd3f8147d50 2024-01-27T02:37:12.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.438+0000 7fd3edffb700 1 --2- 172.21.15.7:0/2150420855 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd3dc066f30 0x7fd3dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:12.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.438+0000 7fd3edffb700 1 -- 172.21.15.7:0/2150420855 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd3e8084720 con 0x7fd3f8147d50 2024-01-27T02:37:12.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.438+0000 7fd3fdae1700 1 --2- 172.21.15.7:0/2150420855 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd3dc066f30 0x7fd3dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:12.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.439+0000 7fd3fdae1700 1 --2- 172.21.15.7:0/2150420855 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd3dc066f30 0x7fd3dc0693e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80b0770 tx=0x7fd3f4017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:12.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.443+0000 7fd3edffb700 1 -- 172.21.15.7:0/2150420855 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3f80b09d0 con 0x7fd3f8147d50 2024-01-27T02:37:12.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.693+0000 7fd3ffd45700 1 -- 172.21.15.7:0/2150420855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3f80b09d0 con 0x7fd3f8147d50 2024-01-27T02:37:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.693+0000 7fd3edffb700 1 -- 172.21.15.7:0/2150420855 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4584 (secure 0 0 0) 0x7fd3f80b09d0 con 0x7fd3f8147d50 2024-01-27T02:37:12.698 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:12.698 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/450556982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":450556982},{"type":"v1","addr":"172.21.15.188:6836","nonce":450556982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1711422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1711422},{"type":"v1","addr":"172.21.15.188:6837","nonce":1711422}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6294,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1110306600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1110306600},{"type":"v1","addr":"172.21.15.7:6833","nonce":1110306600}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:37:03.404355+0000","modified":"2024-01-27T02:37:06.988798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6297":{"gid":6297,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/328992853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":328992853},{"type":"v1","addr":"172.21.15.7:6835","nonce":328992853}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-27T02:37:12.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.695+0000 7fd3ffd45700 1 -- 172.21.15.7:0/2150420855 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd3dc066f30 msgr2=0x7fd3dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.695+0000 7fd3ffd45700 1 --2- 172.21.15.7:0/2150420855 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd3dc066f30 0x7fd3dc0693e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80b0770 tx=0x7fd3f4017000 comp rx=0 tx=0).stop 2024-01-27T02:37:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.695+0000 7fd3ffd45700 1 -- 172.21.15.7:0/2150420855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3f8147d50 msgr2=0x7fd3f816faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.695+0000 7fd3ffd45700 1 --2- 172.21.15.7:0/2150420855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3f8147d50 0x7fd3f816faf0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800ec80 tx=0x7fd3e80091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.696+0000 7fd3ffd45700 1 -- 172.21.15.7:0/2150420855 shutdown_connections 2024-01-27T02:37:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.696+0000 7fd3ffd45700 1 --2- 172.21.15.7:0/2150420855 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd3dc066f30 0x7fd3dc0693e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.696+0000 7fd3ffd45700 1 --2- 172.21.15.7:0/2150420855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3f8148120 0x7fd3f8146a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.696+0000 7fd3ffd45700 1 --2- 172.21.15.7:0/2150420855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3f80c89c0 0x7fd3f80b21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.696+0000 7fd3ffd45700 1 --2- 172.21.15.7:0/2150420855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3f8147d50 0x7fd3f816faf0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.696+0000 7fd3ffd45700 1 -- 172.21.15.7:0/2150420855 >> 172.21.15.7:0/2150420855 conn(0x7fd3f80cdc30 msgr2=0x7fd3f80cd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.696+0000 7fd3ffd45700 1 -- 172.21.15.7:0/2150420855 shutdown_connections 2024-01-27T02:37:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:12.696+0000 7fd3ffd45700 1 -- 172.21.15.7:0/2150420855 wait complete. 2024-01-27T02:37:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 49 2024-01-27T02:37:12.710 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-01-27T02:37:12.861 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","whoami":0,"id":6297,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.8694335849999995,"mdsmap_epoch":49,"osdmap_epoch":98,"osdmap_epoch_barrier":98,"uptime":14.712908134999999} 2024-01-27T02:37:12.862 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", "fs_name": "cephfs", "id": 6297, "mdsmap_epoch": 49, "osdmap_epoch": 98, "osdmap_epoch_barrier": 98, "rank_uptime": 8.869433585, "state": "up:active", "uptime": 14.712908135, "want_state": "up:active", "whoami": 0 } 2024-01-27T02:37:12.862 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-01-27T02:37:13.011 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_mon_report_interval":"5"} 2024-01-27T02:37:13.012 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-01-27T02:37:13.012 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2024-01-27T02:37:13.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:37:13.012 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2024-01-27T02:37:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:265+0 records in 2024-01-27T02:37:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:265+0 records out 2024-01-27T02:37:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:277872640 bytes (278 MB, 265 MiB) copied, 1.50194 s, 185 MB/s 2024-01-27T02:37:14.537 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:37:14.537 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2024-01-27T02:37:16.239 INFO:teuthology.orchestra.run.smithi007.stderr:292+0 records in 2024-01-27T02:37:16.239 INFO:teuthology.orchestra.run.smithi007.stderr:292+0 records out 2024-01-27T02:37:16.239 INFO:teuthology.orchestra.run.smithi007.stderr:306184192 bytes (306 MB, 292 MiB) copied, 1.67962 s, 182 MB/s 2024-01-27T02:37:16.240 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-01-27T02:37:16.240 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:37:16.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.376+0000 7f4a3d9ec700 1 Processor -- start 2024-01-27T02:37:16.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.376+0000 7f4a3d9ec700 1 -- start start 2024-01-27T02:37:16.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.377+0000 7f4a3d9ec700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a380d09f0 0x7f4a3815de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:16.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.377+0000 7f4a3d9ec700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a380b3d90 con 0x7f4a38143c30 2024-01-27T02:37:16.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.377+0000 7f4a3d9ec700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a380d0ea0 con 0x7f4a38143860 2024-01-27T02:37:16.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.377+0000 7f4a3d9ec700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a380d1180 con 0x7f4a380d09f0 2024-01-27T02:37:16.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.377+0000 7f4a377fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4a38143c30 0x7f4a380b4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40760/0 (socket says 172.21.15.7:40760) 2024-01-27T02:37:16.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.377+0000 7f4a377fe700 1 -- 172.21.15.7:0/1266929759 learned_addr learned my addr 172.21.15.7:0/1266929759 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:16.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.377+0000 7f4a367fc700 1 --2- 172.21.15.7:0/1266929759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a380d09f0 0x7f4a3815de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:16.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.377+0000 7f4a35ffb700 1 -- 172.21.15.7:0/1266929759 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82918798 0 0) 0x7f4a380b3d90 con 0x7f4a38143c30 2024-01-27T02:37:16.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.377+0000 7f4a35ffb700 1 -- 172.21.15.7:0/1266929759 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a24003570 con 0x7f4a38143c30 2024-01-27T02:37:16.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.377+0000 7f4a35ffb700 1 -- 172.21.15.7:0/1266929759 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 590881849 0 0) 0x7f4a380d0ea0 con 0x7f4a38143860 2024-01-27T02:37:16.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.377+0000 7f4a35ffb700 1 -- 172.21.15.7:0/1266929759 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a380b3d90 con 0x7f4a38143860 2024-01-27T02:37:16.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.378+0000 7f4a367fc700 1 -- 172.21.15.7:0/1266929759 >> v1:172.21.15.188:6789/0 conn(0x7f4a38143860 legacy=0x7f4a380d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:16.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.378+0000 7f4a35ffb700 1 -- 172.21.15.7:0/1266929759 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 654959165 0 0) 0x7f4a24003570 con 0x7f4a38143c30 2024-01-27T02:37:16.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.378+0000 7f4a367fc700 1 -- 172.21.15.7:0/1266929759 >> v1:172.21.15.7:6789/0 conn(0x7f4a38143c30 legacy=0x7f4a380b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:16.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.378+0000 7f4a367fc700 1 -- 172.21.15.7:0/1266929759 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a380d1460 con 0x7f4a380d09f0 2024-01-27T02:37:16.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.378+0000 7f4a367fc700 1 --2- 172.21.15.7:0/1266929759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a380d09f0 0x7f4a3815de00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009910 tx=0x7f4a280091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=212a0f0a1bdb1c35 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:16.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.378+0000 7f4a35ffb700 1 -- 172.21.15.7:0/1266929759 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a300017f0 con 0x7f4a380d09f0 2024-01-27T02:37:16.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.378+0000 7f4a35ffb700 1 -- 172.21.15.7:0/1266929759 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a28011420 con 0x7f4a380d09f0 2024-01-27T02:37:16.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.378+0000 7f4a35ffb700 1 -- 172.21.15.7:0/1266929759 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a280117c0 con 0x7f4a380d09f0 2024-01-27T02:37:16.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.379+0000 7f4a3d9ec700 1 -- 172.21.15.7:0/1266929759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a380d09f0 msgr2=0x7f4a3815de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:16.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.379+0000 7f4a3d9ec700 1 --2- 172.21.15.7:0/1266929759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a380d09f0 0x7f4a3815de00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009910 tx=0x7f4a280091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:16.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.379+0000 7f4a3d9ec700 1 -- 172.21.15.7:0/1266929759 shutdown_connections 2024-01-27T02:37:16.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.379+0000 7f4a3d9ec700 1 --2- 172.21.15.7:0/1266929759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a380d09f0 0x7f4a3815de00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:16.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.379+0000 7f4a3d9ec700 1 -- 172.21.15.7:0/1266929759 >> 172.21.15.7:0/1266929759 conn(0x7f4a38068440 msgr2=0x7f4a380709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:16.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.379+0000 7f4a3d9ec700 1 -- 172.21.15.7:0/1266929759 shutdown_connections 2024-01-27T02:37:16.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.379+0000 7f4a3d9ec700 1 -- 172.21.15.7:0/1266929759 wait complete. 2024-01-27T02:37:16.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.380+0000 7f4a3d9ec700 1 Processor -- start 2024-01-27T02:37:16.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.380+0000 7f4a3d9ec700 1 -- start start 2024-01-27T02:37:16.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.381+0000 7f4a3d9ec700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a380d09f0 0x7f4a3816b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:16.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.381+0000 7f4a3d9ec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a38143860 0x7f4a3816a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:16.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.381+0000 7f4a3d9ec700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a38143c30 0x7f4a3815c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:16.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.381+0000 7f4a3d9ec700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3816aa70 con 0x7f4a38143860 2024-01-27T02:37:16.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.381+0000 7f4a3d9ec700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3815dd00 con 0x7f4a38143c30 2024-01-27T02:37:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.381+0000 7f4a3d9ec700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3815e190 con 0x7f4a380d09f0 2024-01-27T02:37:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.381+0000 7f4a367fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a38143860 0x7f4a3816a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.381+0000 7f4a367fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a38143860 0x7f4a3816a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53460/0 (socket says 172.21.15.7:53460) 2024-01-27T02:37:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.381+0000 7f4a367fc700 1 -- 172.21.15.7:0/3131962189 learned_addr learned my addr 172.21.15.7:0/3131962189 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.381+0000 7f4a36ffd700 1 --2- 172.21.15.7:0/3131962189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a380d09f0 0x7f4a3816b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.381+0000 7f4a377fe700 1 --2- 172.21.15.7:0/3131962189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a38143c30 0x7f4a3815c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.381+0000 7f4a367fc700 1 -- 172.21.15.7:0/3131962189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a380d09f0 msgr2=0x7f4a3816b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.381+0000 7f4a367fc700 1 --2- 172.21.15.7:0/3131962189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a380d09f0 0x7f4a3816b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.381+0000 7f4a367fc700 1 -- 172.21.15.7:0/3131962189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a38143c30 msgr2=0x7f4a3815c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.381+0000 7f4a367fc700 1 --2- 172.21.15.7:0/3131962189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a38143c30 0x7f4a3815c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.381+0000 7f4a367fc700 1 -- 172.21.15.7:0/3131962189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a28009420 con 0x7f4a38143860 2024-01-27T02:37:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.382+0000 7f4a367fc700 1 --2- 172.21.15.7:0/3131962189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a38143860 0x7f4a3816a530 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4a280091e0 tx=0x7f4a28016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.382+0000 7f4a2effd700 1 -- 172.21.15.7:0/3131962189 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2801cb30 con 0x7f4a38143860 2024-01-27T02:37:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.382+0000 7f4a2effd700 1 -- 172.21.15.7:0/3131962189 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2801cc90 con 0x7f4a38143860 2024-01-27T02:37:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.382+0000 7f4a3d9ec700 1 -- 172.21.15.7:0/3131962189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3815d0f0 con 0x7f4a38143860 2024-01-27T02:37:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.382+0000 7f4a2effd700 1 -- 172.21.15.7:0/3131962189 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2801a6a0 con 0x7f4a38143860 2024-01-27T02:37:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.382+0000 7f4a3d9ec700 1 -- 172.21.15.7:0/3131962189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3815d6b0 con 0x7f4a38143860 2024-01-27T02:37:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.383+0000 7f4a3d9ec700 1 -- 172.21.15.7:0/3131962189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a3804c710 con 0x7f4a38143860 2024-01-27T02:37:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.383+0000 7f4a2effd700 1 -- 172.21.15.7:0/3131962189 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4a2801aad0 con 0x7f4a38143860 2024-01-27T02:37:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.384+0000 7f4a2effd700 1 --2- 172.21.15.7:0/3131962189 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a18066f30 0x7f4a180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.384+0000 7f4a2effd700 1 -- 172.21.15.7:0/3131962189 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4a28086370 con 0x7f4a38143860 2024-01-27T02:37:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.384+0000 7f4a36ffd700 1 --2- 172.21.15.7:0/3131962189 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a18066f30 0x7f4a180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.385+0000 7f4a36ffd700 1 --2- 172.21.15.7:0/3131962189 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a18066f30 0x7f4a180693e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4a20007fd0 tx=0x7f4a20007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.388+0000 7f4a2effd700 1 -- 172.21.15.7:0/3131962189 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a280535f0 con 0x7f4a38143860 2024-01-27T02:37:16.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.591+0000 7f4a3d9ec700 1 -- 172.21.15.7:0/3131962189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f4a3815a090 con 0x7f4a38143860 2024-01-27T02:37:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.592+0000 7f4a2effd700 1 -- 172.21.15.7:0/3131962189 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v98) v1 ==== 81+0+32215 (secure 0 0 0) 0x7f4a28056c10 con 0x7f4a38143860 2024-01-27T02:37:16.593 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:16.594 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:37:16.594 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 98, 2024-01-27T02:37:16.594 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:37:16.594 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:37:16.594 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:37:04.968738+0000", 2024-01-27T02:37:16.594 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:37:16.594 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:37:16.594 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:37:16.594 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:37:16.594 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:37:16.594 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:37:16.595 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:37:16.595 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:37:16.595 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:37:16.595 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:16.595 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:37:16.595 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:37:16.595 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:37:16.595 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:37:16.595 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:37:16.595 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 13, 2024-01-27T02:37:16.595 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:37:16.595 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:37:16.596 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:37:16.596 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:37:16.596 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:37:16.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.596 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:37:16.596 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:37:16.596 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:37:16.596 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:37:16.596 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:37:16.596 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:37:16.596 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:37:16.597 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:37:16.597 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:37:16.597 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:37:16.597 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:37:16.597 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:37:16.597 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:37:16.597 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:37:16.597 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:37:16.597 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:37:16.597 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:37:16.597 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:37:16.597 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:37:16.597 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:37:16.597 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:37:16.598 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:37:16.598 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:37:16.598 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:37:16.598 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:37:16.598 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:37:16.598 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:37:16.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.598 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "89", 2024-01-27T02:37:16.598 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:37:16.598 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:37:16.598 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:37:16.598 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:37:16.599 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:37:16.599 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:37:16.599 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:37:16.599 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:37:16.599 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:37:16.599 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:37:16.599 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:37:16.599 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:37:16.599 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:37:16.599 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:37:16.599 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:37:16.599 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:37:16.599 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:37:16.600 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:37:16.600 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:37:16.600 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:37:16.600 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:37:16.600 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:37:16.600 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:37:16.600 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:37:16.600 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:37:16.600 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:37:16.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.600 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:37:16.601 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:37:16.601 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:37:16.601 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:37:16.601 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:37:16.601 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:37:16.601 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:37:16.601 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:37:16.601 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:37:16.601 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:37:16.601 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:37:16.601 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:37:16.601 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:37:16.601 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:37:16.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.602 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:37:16.602 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:37:16.602 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.602 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 12, 2024-01-27T02:37:16.602 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:37:16.602 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:37:01.445734+0000", 2024-01-27T02:37:16.602 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:37:16.602 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:37:16.602 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:37:16.602 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:37:16.603 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:37:16.603 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:37:16.603 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:37:16.603 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:37:16.603 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:37:16.603 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:37:16.603 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:37:16.603 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:37:16.603 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:37:16.603 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:37:16.603 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:37:16.603 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:37:16.604 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:37:16.604 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:37:16.604 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:37:16.604 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:37:16.604 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:37:16.604 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:37:16.604 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:37:16.604 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:37:16.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.604 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "97", 2024-01-27T02:37:16.604 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:37:16.605 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:37:16.605 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:37:16.605 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:37:16.605 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:37:16.605 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:37:16.605 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:37:16.605 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:37:16.605 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:37:16.605 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:37:16.605 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:37:16.605 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:37:16.606 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:37:16.606 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:37:16.606 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:37:16.606 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:37:16.606 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:37:16.606 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:37:16.606 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:37:16.606 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:37:16.606 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:37:16.606 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:37:16.606 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:37:16.606 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:37:16.606 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:37:16.607 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:37:16.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.607 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:37:16.607 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:37:16.607 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:37:16.607 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:37:16.607 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:37:16.607 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:37:16.607 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:37:16.607 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:37:16.607 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:37:16.608 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:37:16.608 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:37:16.608 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:37:16.608 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:37:16.608 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:37:16.608 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:37:16.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.609 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:37:16.609 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:37:16.609 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:37:16.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.609 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 13, 2024-01-27T02:37:16.609 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:37:16.609 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:37:02.377151+0000", 2024-01-27T02:37:16.609 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:37:16.609 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:37:16.609 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:37:16.610 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:37:16.610 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:37:16.610 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:37:16.610 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:37:16.610 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:37:16.610 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:37:16.610 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:37:16.610 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:37:16.611 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:37:16.611 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:37:16.611 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:37:16.611 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:37:16.611 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:37:16.611 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:37:16.611 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:37:16.611 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:37:16.611 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:37:16.611 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:37:16.611 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:37:16.611 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:37:16.611 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:37:16.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.612 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "98", 2024-01-27T02:37:16.612 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:37:16.612 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:37:16.612 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:37:16.612 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:37:16.612 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:37:16.612 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:37:16.613 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:37:16.613 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:37:16.613 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:37:16.613 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:37:16.613 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:37:16.613 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:37:16.613 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:37:16.613 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:37:16.613 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:37:16.614 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:37:16.614 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:37:16.614 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:37:16.614 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:37:16.614 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:37:16.614 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:37:16.614 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:37:16.614 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:37:16.615 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:37:16.615 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:37:16.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:37:16.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.615 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:37:16.615 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:37:16.615 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:37:16.615 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:37:16.615 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:37:16.615 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:37:16.616 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:37:16.616 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:37:16.616 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:37:16.616 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:37:16.616 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:37:16.616 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:37:16.616 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:37:16.616 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:37:16.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.617 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:37:16.617 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:37:16.617 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:37:16.617 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.617 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.617 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.617 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:16.618 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:37:16.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.618 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:37:16.618 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:16.618 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:16.618 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:16.618 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:16.618 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:16.619 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:16.619 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:16.619 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:16.619 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:16.619 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:16.619 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:16.619 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:16.619 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.620 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.620 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:37:16.620 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:16.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.620 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.620 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:37:16.620 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:16.621 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.621 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.621 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:16.621 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.621 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.621 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:37:16.621 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:16.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.622 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.622 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:37:16.622 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:16.622 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.622 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.622 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:16.622 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.622 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.622 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:37:16.622 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:16.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.623 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.623 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:37:16.623 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:16.623 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.623 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.623 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:16.623 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.624 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.624 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:37:16.624 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:16.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.624 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.624 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:37:16.624 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:16.624 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.624 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.625 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:37:16.625 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:37:16.625 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:37:16.625 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:37:16.625 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:16.625 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:16.625 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:16.625 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.625 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:37:16.626 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:16.626 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:16.626 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:16.626 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:16.626 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:16.626 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:16.626 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:16.626 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:16.626 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:16.626 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:16.626 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:16.626 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:16.627 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.627 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.627 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:37:16.627 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:16.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.627 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.627 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:37:16.627 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:16.627 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.627 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.628 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:16.628 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.628 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.628 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:37:16.628 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:16.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.628 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.629 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:37:16.629 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:16.629 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.629 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.629 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:16.629 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.629 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.629 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:37:16.629 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:16.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.630 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.630 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:37:16.630 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:16.630 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.630 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.630 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:16.630 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.630 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.631 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:37:16.631 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:16.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.631 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.631 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:37:16.631 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:16.631 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.631 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.631 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:37:16.632 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:37:16.632 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:37:16.632 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:37:16.632 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:16.632 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:16.632 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:16.632 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.632 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:37:16.632 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:16.633 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:16.633 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:16.633 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:16.633 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:16.633 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:16.633 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:16.633 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:16.633 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:16.633 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:16.633 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:16.633 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:16.634 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.634 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.634 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:37:16.634 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:16.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.634 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.634 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:37:16.634 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:16.634 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.635 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.635 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:16.635 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.635 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.635 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:37:16.635 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:16.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.635 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.635 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:37:16.636 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:16.636 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.636 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.636 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:16.636 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.636 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.636 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:37:16.636 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:16.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.637 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.637 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:37:16.637 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:16.637 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.637 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.637 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:16.637 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.637 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.637 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:37:16.638 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:16.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.638 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.638 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:37:16.638 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:16.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.638 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.638 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:37:16.638 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:37:16.638 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:37:16.639 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:37:16.639 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:16.639 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:16.639 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:16.639 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.639 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:37:16.639 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:16.639 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:16.639 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:16.639 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:16.640 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:16.640 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:16.640 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:16.640 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:16.640 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:16.640 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:16.640 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:16.640 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:16.640 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.640 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.640 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:37:16.641 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:16.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.641 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.641 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:37:16.641 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:16.641 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.641 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.641 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:16.641 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.642 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.642 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:37:16.642 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:16.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.642 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.642 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:37:16.642 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:16.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.642 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.642 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:16.643 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.643 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.643 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:37:16.643 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:16.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.643 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.643 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:37:16.643 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:16.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.643 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.644 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:16.644 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.644 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.644 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:37:16.644 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:16.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.644 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.644 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:37:16.644 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:16.645 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.645 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.645 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:37:16.645 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:37:16.645 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:37:16.645 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:37:16.645 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:16.645 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:16.645 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:16.645 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.646 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:37:16.646 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:16.646 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:16.646 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:16.646 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:16.646 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:16.646 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:16.647 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:16.647 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:16.647 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:16.647 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:16.647 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:16.647 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:16.647 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.647 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.647 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:37:16.647 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:16.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.648 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.648 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:37:16.648 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:16.648 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.648 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.648 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:16.649 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.649 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.649 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:37:16.649 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:16.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.649 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.649 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:37:16.649 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:16.649 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.649 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.650 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:16.650 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.650 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.650 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:37:16.650 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:16.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.650 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.651 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:37:16.651 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:16.651 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.651 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.651 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:16.651 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.651 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.651 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:37:16.651 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:16.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.652 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.652 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:37:16.652 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:16.652 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.652 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.652 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:37:16.653 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:37:16.653 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:37:16.653 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:37:16.653 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:16.653 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:16.653 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:16.653 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.653 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:37:16.653 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:16.654 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:16.654 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:16.654 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:16.654 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:16.654 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:16.654 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:16.654 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:16.654 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:16.655 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:16.655 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:16.655 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:16.655 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.655 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.655 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:37:16.655 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:16.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.656 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.656 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:37:16.656 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:16.656 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.656 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.656 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:16.656 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.657 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:37:16.657 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:16.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.658 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:37:16.658 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:16.658 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.658 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.658 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:16.658 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.658 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.658 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:37:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:37:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:37:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:37:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:37:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:37:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:37:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:37:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:37:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:37:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:37:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:37:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:37:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:37:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:37:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:37:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:37:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:37:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:37:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:37:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:37:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:37:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:37:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:37:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:37:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:37:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:37:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:37:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:37:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:37:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:37:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:37:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:37:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:37:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:37:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:37:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:37:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:37:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:37:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:37:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:37:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:37:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:37:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:37:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:37:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:37:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:37:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:37:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:37:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:37:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:16.682 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:16.682 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:16.682 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:37:16.682 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:16.682 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.682 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:16.682 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:37:16.682 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:37:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:37:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:37:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:37:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:37:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:37:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:37:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:37:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:37:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:37:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:37:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:37:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:16.684 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:37:16.684 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:37:16.684 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:37:16.684 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:37:16.684 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:37:16.684 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:37:16.685 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:37:16.685 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:37:16.685 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:37:16.685 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:37:16.685 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:37:16.685 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:16.685 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:37:16.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.595+0000 7f4a3d9ec700 1 -- 172.21.15.7:0/3131962189 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a18066f30 msgr2=0x7f4a180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:16.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.595+0000 7f4a3d9ec700 1 --2- 172.21.15.7:0/3131962189 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a18066f30 0x7f4a180693e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4a20007fd0 tx=0x7f4a20007d50 comp rx=0 tx=0).stop 2024-01-27T02:37:16.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.595+0000 7f4a3d9ec700 1 -- 172.21.15.7:0/3131962189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a38143860 msgr2=0x7f4a3816a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:16.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.595+0000 7f4a3d9ec700 1 --2- 172.21.15.7:0/3131962189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a38143860 0x7f4a3816a530 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4a280091e0 tx=0x7f4a28016000 comp rx=0 tx=0).stop 2024-01-27T02:37:16.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.595+0000 7f4a3d9ec700 1 -- 172.21.15.7:0/3131962189 shutdown_connections 2024-01-27T02:37:16.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.595+0000 7f4a3d9ec700 1 --2- 172.21.15.7:0/3131962189 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a18066f30 0x7f4a180693e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:16.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.595+0000 7f4a3d9ec700 1 --2- 172.21.15.7:0/3131962189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a38143c30 0x7f4a3815c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:16.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.595+0000 7f4a3d9ec700 1 --2- 172.21.15.7:0/3131962189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a380d09f0 0x7f4a3816b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:16.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.595+0000 7f4a3d9ec700 1 --2- 172.21.15.7:0/3131962189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a38143860 0x7f4a3816a530 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:16.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.595+0000 7f4a3d9ec700 1 -- 172.21.15.7:0/3131962189 >> 172.21.15.7:0/3131962189 conn(0x7f4a38068440 msgr2=0x7f4a38142820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:16.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.595+0000 7f4a3d9ec700 1 -- 172.21.15.7:0/3131962189 shutdown_connections 2024-01-27T02:37:16.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:16.595+0000 7f4a3d9ec700 1 -- 172.21.15.7:0/3131962189 wait complete. 2024-01-27T02:37:16.690 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-27T02:37:18.073 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-27T02:37:18.072+0000 7fb5c783f700 -1 log_channel(cluster) log [ERR] : Health check failed: 2 full osd(s) (OSD_FULL) 2024-01-27T02:37:21.690 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:37:21.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.831+0000 7f0feb58f700 1 Processor -- start 2024-01-27T02:37:21.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.832+0000 7f0feb58f700 1 -- start start 2024-01-27T02:37:21.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.832+0000 7f0feb58f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0fe415a100 0x7f0fe40cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:21.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.833+0000 7f0feb58f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0fe40c4cc0 0x7f0fe4163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:21.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.833+0000 7f0feb58f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe40ce5e0 con 0x7f0fe40cb5b0 2024-01-27T02:37:21.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.833+0000 7f0feb58f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe40cd9d0 con 0x7f0fe415a100 2024-01-27T02:37:21.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.833+0000 7f0feb58f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe40cdcb0 con 0x7f0fe40c4cc0 2024-01-27T02:37:21.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.833+0000 7f0fe932b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0fe415a100 0x7f0fe40cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:21.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.833+0000 7f0fe8b2a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0fe40cb5b0 0x7f0fe40c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47480/0 (socket says 172.21.15.7:47480) 2024-01-27T02:37:21.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.833+0000 7f0fe8b2a700 1 -- 172.21.15.7:0/3503245673 learned_addr learned my addr 172.21.15.7:0/3503245673 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:21.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.833+0000 7f0fe9b2c700 1 --2- 172.21.15.7:0/3503245673 >> v2:172.21.15.188:3300/0 conn(0x7f0fe40c4cc0 0x7f0fe4163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:21.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.833+0000 7f0fdbfff700 1 -- 172.21.15.7:0/3503245673 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1388399364 0 0) 0x7f0fe40ce5e0 con 0x7f0fe40cb5b0 2024-01-27T02:37:21.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.833+0000 7f0fdbfff700 1 -- 172.21.15.7:0/3503245673 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fd0003570 con 0x7f0fe40cb5b0 2024-01-27T02:37:21.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.833+0000 7f0fe932b700 1 -- 172.21.15.7:0/3503245673 >> v2:172.21.15.188:3300/0 conn(0x7f0fe40c4cc0 msgr2=0x7f0fe4163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:21.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.833+0000 7f0fe932b700 1 --2- 172.21.15.7:0/3503245673 >> v2:172.21.15.188:3300/0 conn(0x7f0fe40c4cc0 0x7f0fe4163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:21.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.833+0000 7f0fe932b700 1 -- 172.21.15.7:0/3503245673 >> v1:172.21.15.7:6789/0 conn(0x7f0fe40cb5b0 legacy=0x7f0fe40c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:21.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.833+0000 7f0fe932b700 1 -- 172.21.15.7:0/3503245673 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe40cdf90 con 0x7f0fe415a100 2024-01-27T02:37:21.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.833+0000 7f0fe9b2c700 1 --2- 172.21.15.7:0/3503245673 >> v2:172.21.15.188:3300/0 conn(0x7f0fe40c4cc0 0x7f0fe4163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:21.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.833+0000 7f0fe932b700 1 --2- 172.21.15.7:0/3503245673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe415a100 0x7f0fe40cd2f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40c9310 tx=0x7f0fe00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77725cce8c8e41f5 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:21.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.833+0000 7f0fdbfff700 1 -- 172.21.15.7:0/3503245673 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0011630 con 0x7f0fe415a100 2024-01-27T02:37:21.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.834+0000 7f0fdbfff700 1 -- 172.21.15.7:0/3503245673 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe0011790 con 0x7f0fe415a100 2024-01-27T02:37:21.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.834+0000 7f0fdbfff700 1 -- 172.21.15.7:0/3503245673 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0011b30 con 0x7f0fe415a100 2024-01-27T02:37:21.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.834+0000 7f0feb58f700 1 -- 172.21.15.7:0/3503245673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe415a100 msgr2=0x7f0fe40cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:21.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.834+0000 7f0feb58f700 1 --2- 172.21.15.7:0/3503245673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe415a100 0x7f0fe40cd2f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40c9310 tx=0x7f0fe00091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:21.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.834+0000 7f0feb58f700 1 -- 172.21.15.7:0/3503245673 shutdown_connections 2024-01-27T02:37:21.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.834+0000 7f0feb58f700 1 --2- 172.21.15.7:0/3503245673 >> v2:172.21.15.188:3300/0 conn(0x7f0fe40c4cc0 0x7f0fe4163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:21.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.834+0000 7f0feb58f700 1 --2- 172.21.15.7:0/3503245673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe415a100 0x7f0fe40cd2f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:21.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.834+0000 7f0feb58f700 1 -- 172.21.15.7:0/3503245673 >> 172.21.15.7:0/3503245673 conn(0x7f0fe4098020 msgr2=0x7f0fe40711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:21.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.834+0000 7f0feb58f700 1 -- 172.21.15.7:0/3503245673 shutdown_connections 2024-01-27T02:37:21.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.834+0000 7f0feb58f700 1 -- 172.21.15.7:0/3503245673 wait complete. 2024-01-27T02:37:21.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.835+0000 7f0feb58f700 1 Processor -- start 2024-01-27T02:37:21.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.835+0000 7f0feb58f700 1 -- start start 2024-01-27T02:37:21.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.836+0000 7f0feb58f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0fe40c4cc0 0x7f0fe40b3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:21.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.836+0000 7f0feb58f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0fe40cb5b0 0x7f0fe40b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:21.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.836+0000 7f0feb58f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe415a100 0x7f0fe40b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:21.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.836+0000 7f0feb58f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe415b3a0 con 0x7f0fe415a100 2024-01-27T02:37:21.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.836+0000 7f0feb58f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe4156630 con 0x7f0fe40c4cc0 2024-01-27T02:37:21.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.836+0000 7f0feb58f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe415be30 con 0x7f0fe40cb5b0 2024-01-27T02:37:21.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.837+0000 7f0fe9b2c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe415a100 0x7f0fe40b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:21.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.837+0000 7f0fe9b2c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe415a100 0x7f0fe40b1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54778/0 (socket says 172.21.15.7:54778) 2024-01-27T02:37:21.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.837+0000 7f0fe9b2c700 1 -- 172.21.15.7:0/3024209202 learned_addr learned my addr 172.21.15.7:0/3024209202 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:21.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.837+0000 7f0fe932b700 1 --2- 172.21.15.7:0/3024209202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0fe40c4cc0 0x7f0fe40b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:21.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.837+0000 7f0fe8b2a700 1 --2- 172.21.15.7:0/3024209202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0fe40cb5b0 0x7f0fe40b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:21.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.837+0000 7f0fe9b2c700 1 -- 172.21.15.7:0/3024209202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0fe40cb5b0 msgr2=0x7f0fe40b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:21.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.837+0000 7f0fe9b2c700 1 --2- 172.21.15.7:0/3024209202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0fe40cb5b0 0x7f0fe40b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:21.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.837+0000 7f0fe9b2c700 1 -- 172.21.15.7:0/3024209202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0fe40c4cc0 msgr2=0x7f0fe40b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:21.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.837+0000 7f0fe9b2c700 1 --2- 172.21.15.7:0/3024209202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0fe40c4cc0 0x7f0fe40b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:21.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.837+0000 7f0fe9b2c700 1 -- 172.21.15.7:0/3024209202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe0009420 con 0x7f0fe415a100 2024-01-27T02:37:21.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.837+0000 7f0fe932b700 1 --2- 172.21.15.7:0/3024209202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0fe40c4cc0 0x7f0fe40b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:37:21.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.837+0000 7f0fe9b2c700 1 --2- 172.21.15.7:0/3024209202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe415a100 0x7f0fe40b1ae0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00cf90 tx=0x7f0fdc0078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:21.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.837+0000 7f0fd97fa700 1 -- 172.21.15.7:0/3024209202 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc015440 con 0x7f0fe415a100 2024-01-27T02:37:21.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.837+0000 7f0fd97fa700 1 -- 172.21.15.7:0/3024209202 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fdc0155a0 con 0x7f0fe415a100 2024-01-27T02:37:21.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.837+0000 7f0feb58f700 1 -- 172.21.15.7:0/3024209202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe415b590 con 0x7f0fe415a100 2024-01-27T02:37:21.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.837+0000 7f0fd97fa700 1 -- 172.21.15.7:0/3024209202 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc015960 con 0x7f0fe415a100 2024-01-27T02:37:21.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.837+0000 7f0feb58f700 1 -- 172.21.15.7:0/3024209202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe415bb50 con 0x7f0fe415a100 2024-01-27T02:37:21.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.839+0000 7f0feb58f700 1 -- 172.21.15.7:0/3024209202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe41567a0 con 0x7f0fe415a100 2024-01-27T02:37:21.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.839+0000 7f0fd97fa700 1 -- 172.21.15.7:0/3024209202 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0fdc015d50 con 0x7f0fe415a100 2024-01-27T02:37:21.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.839+0000 7f0fd97fa700 1 --2- 172.21.15.7:0/3024209202 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0fc8066f30 0x7f0fc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:21.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.839+0000 7f0fd97fa700 1 -- 172.21.15.7:0/3024209202 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0fdc084e90 con 0x7f0fe415a100 2024-01-27T02:37:21.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.839+0000 7f0fd97fa700 1 -- 172.21.15.7:0/3024209202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=102}) v3 -- 0x7f0fc806ea50 con 0x7f0fe415a100 2024-01-27T02:37:21.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.840+0000 7f0fe932b700 1 --2- 172.21.15.7:0/3024209202 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0fc8066f30 0x7f0fc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:21.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.840+0000 7f0fe932b700 1 --2- 172.21.15.7:0/3024209202 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0fc8066f30 0x7f0fc80693e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0fe00093f0 tx=0x7f0fe0017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:21.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:21.844+0000 7f0fd97fa700 1 -- 172.21.15.7:0/3024209202 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0fe41567a0 con 0x7f0fe415a100 2024-01-27T02:37:22.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.047+0000 7f0feb58f700 1 -- 172.21.15.7:0/3024209202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f0fe41567a0 con 0x7f0fe415a100 2024-01-27T02:37:22.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.048+0000 7f0fd97fa700 1 -- 172.21.15.7:0/3024209202 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v101) v1 ==== 81+0+32359 (secure 0 0 0) 0x7f0fe41567a0 con 0x7f0fe415a100 2024-01-27T02:37:22.049 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:22.049 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:37:22.049 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 101, 2024-01-27T02:37:22.049 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:37:22.049 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:37:22.049 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:37:21.127827+0000", 2024-01-27T02:37:22.049 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:37:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:37:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:37:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:37:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:37:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:37:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:37:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:37:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:37:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:37:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:37:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:37:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:37:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:37:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 13, 2024-01-27T02:37:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:37:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:37:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:37:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:37:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:37:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:37:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:37:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:37:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:37:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:37:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:37:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:37:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:37:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:37:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:37:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:37:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:37:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:37:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:37:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:37:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:37:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:37:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:37:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:37:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:37:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:37:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:37:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:37:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:37:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:37:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:37:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:37:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "99", 2024-01-27T02:37:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:37:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:37:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:37:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:37:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:37:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:37:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:37:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:37:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:37:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:37:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:37:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:37:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:37:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:37:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:37:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:37:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:37:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:37:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:37:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:37:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:37:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:37:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:37:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:37:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:37:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:37:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:37:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:37:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:37:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:37:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:37:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:37:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:37:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:37:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:37:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:37:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:37:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:37:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:37:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:37:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:37:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:37:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 12, 2024-01-27T02:37:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:37:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:37:01.445734+0000", 2024-01-27T02:37:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:37:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:37:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:37:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:37:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:37:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:37:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:37:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:37:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:37:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:37:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:37:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:37:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:37:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:37:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:37:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:37:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:37:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:37:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:37:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:37:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:37:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:37:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:37:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:37:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "99", 2024-01-27T02:37:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:37:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:37:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:37:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:37:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:37:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:37:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:37:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:37:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:37:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:37:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:37:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:37:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:37:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:37:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:37:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:37:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:37:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:37:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:37:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:37:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:37:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:37:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:37:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:37:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:37:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:37:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:37:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:37:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:37:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:37:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:37:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:37:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:37:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:37:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:37:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:37:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:37:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:37:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:37:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:37:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:37:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:37:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:37:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:37:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 13, 2024-01-27T02:37:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:37:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:37:02.377151+0000", 2024-01-27T02:37:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:37:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:37:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:37:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:37:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:37:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:37:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:37:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:37:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:37:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:37:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:37:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:37:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:37:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:37:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:37:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:37:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:37:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:37:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:37:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:37:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:37:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:37:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:37:22.065 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:37:22.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.065 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "99", 2024-01-27T02:37:22.065 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:37:22.065 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:37:22.065 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:37:22.065 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:37:22.065 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:37:22.065 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:37:22.065 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:37:22.065 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:37:22.066 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:37:22.066 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:37:22.066 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:37:22.066 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:37:22.066 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:37:22.066 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:37:22.066 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:37:22.066 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:37:22.066 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:37:22.067 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:37:22.067 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:37:22.067 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:37:22.067 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:37:22.067 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:37:22.067 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:37:22.067 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:37:22.067 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:37:22.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:37:22.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.067 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:37:22.067 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:37:22.067 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:37:22.067 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:37:22.068 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:37:22.068 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:37:22.068 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:37:22.068 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:37:22.068 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:37:22.068 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:37:22.068 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:37:22.069 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:37:22.069 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:37:22.069 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:37:22.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.069 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:37:22.069 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:37:22.069 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:37:22.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.069 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:22.069 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:37:22.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.070 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:37:22.070 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:22.070 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:22.070 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:22.070 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:22.070 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:22.070 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:22.071 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:22.071 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:22.071 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:22.071 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:22.071 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:22.071 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:22.071 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.071 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:37:22.071 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:22.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.072 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:37:22.072 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:22.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.072 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.072 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:22.072 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.073 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:37:22.073 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:22.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.073 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:37:22.073 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:22.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.073 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.074 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:22.074 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.074 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:37:22.074 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:22.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.075 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:37:22.075 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:22.075 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.075 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.075 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:22.075 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.075 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:37:22.076 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:22.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.076 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:37:22.076 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:22.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.076 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.076 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:37:22.076 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:37:22.077 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:37:22.077 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:37:22.077 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:22.077 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:22.077 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:37:22.077 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:22.077 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.078 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:37:22.078 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:22.078 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:22.078 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:22.078 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:22.078 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:22.078 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:22.078 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:22.079 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:22.079 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:22.079 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:22.079 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:22.079 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:22.079 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.079 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:37:22.079 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:22.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.079 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:37:22.079 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:22.080 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.080 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.080 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:22.080 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.080 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:37:22.080 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:22.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.080 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:37:22.080 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:22.081 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.081 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.081 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:22.081 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.081 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:37:22.081 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:22.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.081 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:37:22.081 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:22.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.082 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.082 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:22.082 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.082 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:37:22.082 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:22.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.082 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:37:22.082 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:22.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.083 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.083 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:37:22.083 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:37:22.083 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:37:22.083 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:37:22.083 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:22.083 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:22.083 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:37:22.083 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:22.083 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.083 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:37:22.083 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:22.084 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:22.084 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:22.084 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:22.084 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:22.084 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:22.084 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:22.084 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:22.084 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:22.084 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:22.084 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:22.084 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:22.084 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.085 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:37:22.085 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:22.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.085 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:37:22.085 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:22.085 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.085 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.085 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:22.085 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.085 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:37:22.085 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:22.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.086 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.086 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:37:22.086 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:22.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.086 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.086 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:22.086 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.086 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.086 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:37:22.086 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:22.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.087 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:37:22.087 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:22.087 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.087 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.087 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:22.087 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.087 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:37:22.087 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:22.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.088 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:37:22.088 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:22.088 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.088 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.088 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:37:22.088 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:37:22.088 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:37:22.088 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:37:22.088 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:22.088 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:22.088 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:22.088 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.089 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:37:22.089 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:22.089 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:22.089 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:22.089 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:22.089 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:22.089 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:22.089 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:22.089 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:22.089 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:22.089 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:22.089 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:22.089 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:22.089 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.090 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:37:22.090 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:22.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.090 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:37:22.090 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:22.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.090 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.090 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:22.090 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.091 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:37:22.091 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:22.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.091 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:37:22.091 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:22.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.091 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.091 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:22.091 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.092 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:37:22.092 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:22.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.092 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:37:22.092 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:22.092 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.092 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.092 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:22.092 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.093 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.093 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:37:22.093 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:22.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.093 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.093 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:37:22.093 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:22.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.093 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.093 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:37:22.093 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:37:22.093 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:37:22.093 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:37:22.094 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:22.094 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:22.094 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:22.094 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.094 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:37:22.094 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:22.094 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:22.094 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:22.094 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:22.094 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:22.094 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:22.094 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:22.095 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:22.095 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:22.095 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:22.095 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:22.095 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:22.095 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.095 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:37:22.095 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:22.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.096 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:37:22.096 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:22.096 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.096 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.096 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:22.096 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.096 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:37:22.096 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:22.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.096 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:37:22.097 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:22.097 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.097 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.097 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:22.097 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.097 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:37:22.097 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:22.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.097 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:37:22.097 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:22.097 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.098 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.098 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:22.098 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.098 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:37:22.098 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:22.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.098 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:37:22.098 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:22.098 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.098 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.098 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:37:22.099 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:37:22.099 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:37:22.099 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:37:22.099 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:22.099 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:22.099 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:37:22.099 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:22.099 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.099 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:37:22.099 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:22.099 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:22.099 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:22.100 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:22.100 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:22.100 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:22.100 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:22.100 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:22.100 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:22.100 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:22.100 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:22.101 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:22.101 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.101 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:37:22.101 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:22.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.101 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:37:22.101 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:22.101 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.101 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.101 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:22.102 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.102 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:37:22.102 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:22.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.102 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:37:22.102 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:22.102 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.103 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.103 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:22.103 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.103 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:37:22.103 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:22.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.103 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:37:22.104 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:22.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.104 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.104 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:22.104 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.104 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:37:22.104 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:22.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.105 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:37:22.105 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:22.105 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.105 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.105 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:37:22.105 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:37:22.105 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:37:22.105 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:37:22.105 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:22.106 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:22.106 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:22.106 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.106 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:37:22.106 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:22.106 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:22.106 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:22.106 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:22.106 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:22.106 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:22.107 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:22.107 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:22.107 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:22.107 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:22.107 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:22.107 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:22.107 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.107 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:37:22.107 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:22.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.108 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.108 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:37:22.108 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:22.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.108 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.108 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:22.108 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.109 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:37:22.109 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:22.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.109 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:37:22.109 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:22.109 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.109 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.110 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:22.110 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.110 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:37:22.110 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:22.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.111 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:37:22.111 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:22.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.111 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.111 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:22.111 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.111 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:37:22.112 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:22.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.112 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.112 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:37:22.112 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:22.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.112 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.112 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:37:22.112 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:37:22.112 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:37:22.112 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:37:22.113 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:22.113 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:22.113 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:37:22.113 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:22.113 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.113 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:37:22.113 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:22.113 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:22.113 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:22.113 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:22.115 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:22.115 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:22.115 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:22.115 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:22.115 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:22.115 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:22.115 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:22.115 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:22.115 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.116 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:37:22.116 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:22.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.116 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:37:22.116 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:22.116 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.116 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.116 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:22.116 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.117 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.117 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:37:22.117 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:22.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.117 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.117 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:37:22.117 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:22.117 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.117 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.117 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:22.117 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.118 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:37:22.118 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:22.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.118 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:37:22.118 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:22.118 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.118 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.119 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:22.119 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.119 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.119 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:37:22.119 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:22.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.119 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.119 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:37:22.119 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:22.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.120 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.120 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:37:22.120 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:37:22.120 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:37:22.120 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:37:22.120 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:22.120 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull", 2024-01-27T02:37:22.120 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:22.120 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:22.120 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.121 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:22.121 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:37:22.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.121 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:37:22.121 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:22.121 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:22.121 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:22.121 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:22.121 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:22.121 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:37:22.121 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:22.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.121 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:37:22.121 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:22.122 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:22.122 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:22.122 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:22.122 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:22.122 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:37:22.122 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:22.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.122 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:37:22.122 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:22.122 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:22.122 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:22.122 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:22.123 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:22.123 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:37:22.123 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:22.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.123 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:37:22.123 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:22.123 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:22.123 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:22.123 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:22.123 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:22.123 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:37:22.123 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:22.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.124 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:37:22.124 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:22.124 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:22.124 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:22.124 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:22.124 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:22.124 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:37:22.124 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:22.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.124 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:37:22.125 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:22.125 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:22.125 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:22.125 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:22.125 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:22.125 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:37:22.125 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:22.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.125 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:37:22.125 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:22.125 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:22.126 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:22.126 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:22.126 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:22.126 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:37:22.126 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:22.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.126 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:37:22.126 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:22.126 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:22.126 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:22.126 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:22.126 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:22.126 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:37:22.126 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:22.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.127 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:22.127 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:37:22.127 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:37:22.127 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:37:22.127 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:37:22.127 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:37:22.127 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:37:22.127 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:37:22.127 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:37:22.127 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:37:22.127 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:37:22.127 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:37:22.127 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:37:22.127 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:37:22.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.128 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:37:22.128 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:37:22.128 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:37:22.128 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:37:22.128 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:37:22.128 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:37:22.128 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:37:22.128 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:37:22.128 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:37:22.128 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:37:22.128 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:37:22.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.128 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:37:22.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.050+0000 7f0feb58f700 1 -- 172.21.15.7:0/3024209202 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0fc8066f30 msgr2=0x7f0fc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:22.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.050+0000 7f0feb58f700 1 --2- 172.21.15.7:0/3024209202 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0fc8066f30 0x7f0fc80693e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0fe00093f0 tx=0x7f0fe0017000 comp rx=0 tx=0).stop 2024-01-27T02:37:22.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.050+0000 7f0feb58f700 1 -- 172.21.15.7:0/3024209202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe415a100 msgr2=0x7f0fe40b1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:22.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.050+0000 7f0feb58f700 1 --2- 172.21.15.7:0/3024209202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe415a100 0x7f0fe40b1ae0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00cf90 tx=0x7f0fdc0078a0 comp rx=0 tx=0).stop 2024-01-27T02:37:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.050+0000 7f0feb58f700 1 -- 172.21.15.7:0/3024209202 shutdown_connections 2024-01-27T02:37:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.050+0000 7f0feb58f700 1 --2- 172.21.15.7:0/3024209202 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0fc8066f30 0x7f0fc80693e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.050+0000 7f0feb58f700 1 --2- 172.21.15.7:0/3024209202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe415a100 0x7f0fe40b1ae0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.050+0000 7f0feb58f700 1 --2- 172.21.15.7:0/3024209202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0fe40c4cc0 0x7f0fe40b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.050+0000 7f0feb58f700 1 --2- 172.21.15.7:0/3024209202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0fe40cb5b0 0x7f0fe40b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.051+0000 7f0feb58f700 1 -- 172.21.15.7:0/3024209202 >> 172.21.15.7:0/3024209202 conn(0x7f0fe4098020 msgr2=0x7f0fe4155410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.051+0000 7f0feb58f700 1 -- 172.21.15.7:0/3024209202 shutdown_connections 2024-01-27T02:37:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.051+0000 7f0feb58f700 1 -- 172.21.15.7:0/3024209202 wait complete. 2024-01-27T02:37:22.131 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-01-27T02:37:22.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:37:22.132 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2024-01-27T02:37:22.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-27T02:37:22.416 INFO:teuthology.orchestra.run.smithi007.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2024-01-27T02:37:22.417 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records in 2024-01-27T02:37:22.417 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records out 2024-01-27T02:37:22.417 INFO:teuthology.orchestra.run.smithi007.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.2562 s, 205 MB/s 2024-01-27T02:37:22.417 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:37:22.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.548+0000 7f05f1023700 1 Processor -- start 2024-01-27T02:37:22.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.548+0000 7f05f1023700 1 -- start start 2024-01-27T02:37:22.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.549+0000 7f05f1023700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f05ec0b1c00 0x7f05ec06e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:22.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.549+0000 7f05f1023700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05ec0b1fd0 0x7f05ec156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:22.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.549+0000 7f05f1023700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ec157610 con 0x7f05ec15b5f0 2024-01-27T02:37:22.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.549+0000 7f05f1023700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ec157310 con 0x7f05ec0b1c00 2024-01-27T02:37:22.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.549+0000 7f05f1023700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec06ee00 con 0x7f05ec0b1fd0 2024-01-27T02:37:22.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.550+0000 7f05ead9d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f05ec0b1c00 0x7f05ec06e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:22.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.550+0000 7f05eb59e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05ec0b1fd0 0x7f05ec156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.550+0000 7f05eb59e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05ec0b1fd0 0x7f05ec156dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:36262/0 (socket says 172.21.15.7:36262) 2024-01-27T02:37:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.550+0000 7f05eb59e700 1 -- 172.21.15.7:0/4288098003 learned_addr learned my addr 172.21.15.7:0/4288098003 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.550+0000 7f05e3fff700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f05ec15b5f0 0x7f05ec1566c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:35416/0 (socket says 172.21.15.7:35416) 2024-01-27T02:37:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.550+0000 7f05eb59e700 1 -- 172.21.15.7:0/4288098003 >> v2:172.21.15.188:3300/0 conn(0x7f05ec0b1c00 msgr2=0x7f05ec06e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.550+0000 7f05eb59e700 1 --2- 172.21.15.7:0/4288098003 >> v2:172.21.15.188:3300/0 conn(0x7f05ec0b1c00 0x7f05ec06e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.550+0000 7f05eb59e700 1 -- 172.21.15.7:0/4288098003 >> v1:172.21.15.188:6789/0 conn(0x7f05ec15b5f0 legacy=0x7f05ec1566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.550+0000 7f05eb59e700 1 -- 172.21.15.7:0/4288098003 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec06f0e0 con 0x7f05ec0b1fd0 2024-01-27T02:37:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.550+0000 7f05ead9d700 1 --2- 172.21.15.7:0/4288098003 >> v2:172.21.15.188:3300/0 conn(0x7f05ec0b1c00 0x7f05ec06e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:37:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.550+0000 7f05eb59e700 1 --2- 172.21.15.7:0/4288098003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05ec0b1fd0 0x7f05ec156dd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009f20 tx=0x7f05e4009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6025dbed2aba4b1 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.551+0000 7f05ea59c700 1 -- 172.21.15.7:0/4288098003 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e400e070 con 0x7f05ec0b1fd0 2024-01-27T02:37:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.551+0000 7f05ea59c700 1 -- 172.21.15.7:0/4288098003 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e400b9d0 con 0x7f05ec0b1fd0 2024-01-27T02:37:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.551+0000 7f05ea59c700 1 -- 172.21.15.7:0/4288098003 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e40092e0 con 0x7f05ec0b1fd0 2024-01-27T02:37:22.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.551+0000 7f05f1023700 1 -- 172.21.15.7:0/4288098003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05ec0b1fd0 msgr2=0x7f05ec156dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:22.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.551+0000 7f05f1023700 1 --2- 172.21.15.7:0/4288098003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05ec0b1fd0 0x7f05ec156dd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009f20 tx=0x7f05e4009840 comp rx=0 tx=0).stop 2024-01-27T02:37:22.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.551+0000 7f05f1023700 1 -- 172.21.15.7:0/4288098003 shutdown_connections 2024-01-27T02:37:22.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.551+0000 7f05f1023700 1 --2- 172.21.15.7:0/4288098003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05ec0b1fd0 0x7f05ec156dd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:22.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.551+0000 7f05f1023700 1 --2- 172.21.15.7:0/4288098003 >> v2:172.21.15.188:3300/0 conn(0x7f05ec0b1c00 0x7f05ec06e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:22.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.551+0000 7f05f1023700 1 -- 172.21.15.7:0/4288098003 >> 172.21.15.7:0/4288098003 conn(0x7f05ec068430 msgr2=0x7f05ec06be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:22.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.551+0000 7f05f1023700 1 -- 172.21.15.7:0/4288098003 shutdown_connections 2024-01-27T02:37:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.551+0000 7f05f1023700 1 -- 172.21.15.7:0/4288098003 wait complete. 2024-01-27T02:37:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.552+0000 7f05f1023700 1 Processor -- start 2024-01-27T02:37:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.552+0000 7f05f1023700 1 -- start start 2024-01-27T02:37:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.553+0000 7f05f1023700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec0b1c00 0x7f05ec0ab500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.553+0000 7f05f1023700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05ec0b1fd0 0x7f05ec0a6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.553+0000 7f05f1023700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05ec15b5f0 0x7f05ec14fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.553+0000 7f05f1023700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec0ad590 con 0x7f05ec0b1c00 2024-01-27T02:37:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.553+0000 7f05f1023700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec0a7890 con 0x7f05ec15b5f0 2024-01-27T02:37:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.553+0000 7f05f1023700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec0a7b70 con 0x7f05ec0b1fd0 2024-01-27T02:37:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.553+0000 7f05ead9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec0b1c00 0x7f05ec0ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.553+0000 7f05ead9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec0b1c00 0x7f05ec0ab500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54780/0 (socket says 172.21.15.7:54780) 2024-01-27T02:37:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.553+0000 7f05ead9d700 1 -- 172.21.15.7:0/4018921821 learned_addr learned my addr 172.21.15.7:0/4018921821 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.553+0000 7f05e3fff700 1 --2- 172.21.15.7:0/4018921821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05ec0b1fd0 0x7f05ec0a6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.553+0000 7f05eb59e700 1 --2- 172.21.15.7:0/4018921821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05ec15b5f0 0x7f05ec14fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.554+0000 7f05ead9d700 1 -- 172.21.15.7:0/4018921821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05ec0b1fd0 msgr2=0x7f05ec0a6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.554+0000 7f05ead9d700 1 --2- 172.21.15.7:0/4018921821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05ec0b1fd0 0x7f05ec0a6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.554+0000 7f05ead9d700 1 -- 172.21.15.7:0/4018921821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05ec15b5f0 msgr2=0x7f05ec14fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.554+0000 7f05ead9d700 1 --2- 172.21.15.7:0/4018921821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05ec15b5f0 0x7f05ec14fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.554+0000 7f05ead9d700 1 -- 172.21.15.7:0/4018921821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05dc009420 con 0x7f05ec0b1c00 2024-01-27T02:37:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.554+0000 7f05ead9d700 1 --2- 172.21.15.7:0/4018921821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec0b1c00 0x7f05ec0ab500 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00ec80 tx=0x7f05dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.554+0000 7f05e2ffd700 1 -- 172.21.15.7:0/4018921821 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc00c920 con 0x7f05ec0b1c00 2024-01-27T02:37:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.554+0000 7f05e2ffd700 1 -- 172.21.15.7:0/4018921821 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05dc00ca80 con 0x7f05ec0b1c00 2024-01-27T02:37:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.554+0000 7f05f1023700 1 -- 172.21.15.7:0/4018921821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e4009ae0 con 0x7f05ec0b1c00 2024-01-27T02:37:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.554+0000 7f05e2ffd700 1 -- 172.21.15.7:0/4018921821 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc00ce00 con 0x7f05ec0b1c00 2024-01-27T02:37:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.554+0000 7f05f1023700 1 -- 172.21.15.7:0/4018921821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ec0ad810 con 0x7f05ec0b1c00 2024-01-27T02:37:22.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.555+0000 7f05f1023700 1 -- 172.21.15.7:0/4018921821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ec04c710 con 0x7f05ec0b1c00 2024-01-27T02:37:22.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.556+0000 7f05e2ffd700 1 -- 172.21.15.7:0/4018921821 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f05dc013680 con 0x7f05ec0b1c00 2024-01-27T02:37:22.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.556+0000 7f05e2ffd700 1 --2- 172.21.15.7:0/4018921821 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f05cc066f30 0x7f05cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:22.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.556+0000 7f05e2ffd700 1 -- 172.21.15.7:0/4018921821 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f05dc084ca0 con 0x7f05ec0b1c00 2024-01-27T02:37:22.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.556+0000 7f05e2ffd700 1 -- 172.21.15.7:0/4018921821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=103}) v3 -- 0x7f05cc06ea50 con 0x7f05ec0b1c00 2024-01-27T02:37:22.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.557+0000 7f05e3fff700 1 --2- 172.21.15.7:0/4018921821 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f05cc066f30 0x7f05cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:22.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.557+0000 7f05e3fff700 1 --2- 172.21.15.7:0/4018921821 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f05cc066f30 0x7f05cc0693e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f05ec0aa870 tx=0x7f05d4007eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:22.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.561+0000 7f05e2ffd700 1 -- 172.21.15.7:0/4018921821 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05dc051f10 con 0x7f05ec0b1c00 2024-01-27T02:37:22.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.765+0000 7f05f1023700 1 -- 172.21.15.7:0/4018921821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f05ec06f3c0 con 0x7f05ec0b1c00 2024-01-27T02:37:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.766+0000 7f05e2ffd700 1 -- 172.21.15.7:0/4018921821 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v102) v1 ==== 81+0+32383 (secure 0 0 0) 0x7f05dc021020 con 0x7f05ec0b1c00 2024-01-27T02:37:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:22.767 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:37:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 102, 2024-01-27T02:37:22.768 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:37:22.768 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:37:22.768 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:37:22.129521+0000", 2024-01-27T02:37:22.768 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:37:22.768 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:37:22.769 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:37:22.769 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:37:22.769 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:37:22.769 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:37:22.769 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:37:22.769 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:37:22.770 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:37:22.770 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:22.770 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:37:22.770 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:37:22.770 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:37:22.770 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:37:22.771 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:37:22.771 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 13, 2024-01-27T02:37:22.771 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:37:22.771 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:37:22.771 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:37:22.772 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:37:22.772 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:37:22.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.772 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:37:22.772 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:37:22.772 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:37:22.773 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:37:22.773 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:37:22.773 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:37:22.773 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:37:22.773 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:37:22.773 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:37:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:37:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:37:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:37:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:37:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:37:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:37:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:37:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:37:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:37:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:37:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:37:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:37:22.776 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:37:22.776 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:37:22.776 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:37:22.776 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:37:22.776 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:37:22.777 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:37:22.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.777 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "99", 2024-01-27T02:37:22.777 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:37:22.777 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:37:22.777 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:37:22.778 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:37:22.778 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:37:22.778 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:37:22.778 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:37:22.778 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:37:22.778 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:37:22.778 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:37:22.779 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:37:22.779 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:37:22.779 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:37:22.779 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:37:22.779 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:37:22.779 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:37:22.780 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:37:22.780 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:37:22.780 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:37:22.780 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:37:22.780 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:37:22.780 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:37:22.781 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:37:22.781 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:37:22.781 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:37:22.781 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:37:22.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.781 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:37:22.782 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:37:22.782 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:37:22.782 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:37:22.782 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:37:22.782 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:37:22.782 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:37:22.783 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:37:22.783 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:37:22.783 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:37:22.783 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:37:22.783 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:37:22.783 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:37:22.784 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:37:22.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.784 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:37:22.784 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:37:22.784 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.785 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 12, 2024-01-27T02:37:22.785 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:37:22.785 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:37:01.445734+0000", 2024-01-27T02:37:22.785 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:37:22.785 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:37:22.785 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:37:22.785 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:37:22.785 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:37:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:37:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:37:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:37:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:37:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:37:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:37:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:37:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:37:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:37:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:37:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:37:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:37:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:37:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:37:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:37:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:37:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:37:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:37:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:37:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "99", 2024-01-27T02:37:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:37:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:37:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:37:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:37:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:37:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:37:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:37:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:37:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:37:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:37:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:37:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:37:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:37:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:37:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:37:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:37:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:37:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:37:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:37:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:37:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:37:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:37:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:37:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:37:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:37:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:37:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:37:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:37:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:37:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:37:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:37:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:37:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:37:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:37:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:37:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:37:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:37:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:37:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:37:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:37:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:37:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:37:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:37:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:37:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 13, 2024-01-27T02:37:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:37:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:37:02.377151+0000", 2024-01-27T02:37:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:37:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:37:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:37:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:37:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:37:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:37:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:37:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:37:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:37:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:37:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:37:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:37:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:37:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:37:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:37:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:37:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:37:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:37:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:37:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:37:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:37:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:37:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:37:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:37:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "99", 2024-01-27T02:37:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:37:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:37:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:37:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:37:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:37:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:37:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:37:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:37:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:37:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:37:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:37:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:37:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:37:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:37:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:37:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:37:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:37:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:37:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:37:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:37:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:37:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:37:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:37:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:37:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:37:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:37:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:37:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:37:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:37:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:37:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:37:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:37:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:37:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:37:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:37:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:37:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:37:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:37:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:37:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:37:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:37:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:37:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:37:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:37:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:37:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:37:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:37:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:37:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:37:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:37:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:37:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:37:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:37:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:37:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:37:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:37:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:37:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:37:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:37:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:37:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:37:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:37:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:37:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:37:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:37:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:37:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:37:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:37:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:37:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:37:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:37:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:37:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:37:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:37:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:37:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:37:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:37:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:37:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:37:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:37:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:37:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:37:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:37:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:37:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:37:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:37:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:37:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:37:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:37:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:37:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:37:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:37:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:37:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:37:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:37:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:37:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:37:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:37:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:37:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:37:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:37:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:37:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:37:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:37:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:37:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:37:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:37:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:37:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:37:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:37:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:37:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:37:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:37:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:37:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:37:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:37:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:37:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:37:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:37:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:37:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:37:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:37:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:37:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:37:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:37:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:37:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:37:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:37:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:37:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:37:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:37:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:37:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:37:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:37:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:37:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:37:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:37:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:37:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:37:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:37:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:37:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:37:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:37:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:37:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:37:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:37:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:37:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:37:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:37:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:37:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:37:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:37:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:37:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull", 2024-01-27T02:37:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:37:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:37:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:37:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:37:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:37:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:37:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:37:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:37:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:37:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:37:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:37:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:37:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:37:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:37:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:37:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:37:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:37:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:37:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:37:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:37:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:37:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:37:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:37:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:37:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:37:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:37:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:37:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:37:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:37:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:37:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:37:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:37:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:37:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:37:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:37:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:37:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:37:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:37:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:37:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:37:22.847 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:37:22.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:22.847 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:37:22.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.768+0000 7f05f1023700 1 -- 172.21.15.7:0/4018921821 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f05cc066f30 msgr2=0x7f05cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.768+0000 7f05f1023700 1 --2- 172.21.15.7:0/4018921821 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f05cc066f30 0x7f05cc0693e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f05ec0aa870 tx=0x7f05d4007eb0 comp rx=0 tx=0).stop 2024-01-27T02:37:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.768+0000 7f05f1023700 1 -- 172.21.15.7:0/4018921821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec0b1c00 msgr2=0x7f05ec0ab500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.768+0000 7f05f1023700 1 --2- 172.21.15.7:0/4018921821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec0b1c00 0x7f05ec0ab500 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00ec80 tx=0x7f05dc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.768+0000 7f05f1023700 1 -- 172.21.15.7:0/4018921821 shutdown_connections 2024-01-27T02:37:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.768+0000 7f05f1023700 1 --2- 172.21.15.7:0/4018921821 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f05cc066f30 0x7f05cc0693e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.768+0000 7f05f1023700 1 --2- 172.21.15.7:0/4018921821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05ec15b5f0 0x7f05ec14fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.768+0000 7f05f1023700 1 --2- 172.21.15.7:0/4018921821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec0b1c00 0x7f05ec0ab500 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.768+0000 7f05f1023700 1 --2- 172.21.15.7:0/4018921821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05ec0b1fd0 0x7f05ec0a6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.768+0000 7f05f1023700 1 -- 172.21.15.7:0/4018921821 >> 172.21.15.7:0/4018921821 conn(0x7f05ec068430 msgr2=0x7f05ec1520b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.768+0000 7f05f1023700 1 -- 172.21.15.7:0/4018921821 shutdown_connections 2024-01-27T02:37:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.768+0000 7f05f1023700 1 -- 172.21.15.7:0/4018921821 wait complete. 2024-01-27T02:37:22.850 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:37:22.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.984+0000 7fe15d7ac700 1 Processor -- start 2024-01-27T02:37:22.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.984+0000 7fe15d7ac700 1 -- start start 2024-01-27T02:37:22.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.985+0000 7fe15d7ac700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe158156580 0x7fe1580cb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:22.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.985+0000 7fe15d7ac700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe1580c8ce0 0x7fe1580cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:22.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.985+0000 7fe15d7ac700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1580c7a00 con 0x7fe15806d340 2024-01-27T02:37:22.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.985+0000 7fe15d7ac700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1580c8850 con 0x7fe1580c8ce0 2024-01-27T02:37:22.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.985+0000 7fe15d7ac700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1580c7c40 con 0x7fe158156580 2024-01-27T02:37:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.985+0000 7fe1577fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe15806d340 0x7fe1580c0500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47484/0 (socket says 172.21.15.7:47484) 2024-01-27T02:37:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.985+0000 7fe1577fe700 1 -- 172.21.15.7:0/2372132184 learned_addr learned my addr 172.21.15.7:0/2372132184 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.985+0000 7fe1567fc700 1 --2- 172.21.15.7:0/2372132184 >> v2:172.21.15.188:3300/0 conn(0x7fe1580c8ce0 0x7fe1580cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.985+0000 7fe156ffd700 1 --2- 172.21.15.7:0/2372132184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe158156580 0x7fe1580cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.985+0000 7fe155ffb700 1 -- 172.21.15.7:0/2372132184 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1983172444 0 0) 0x7fe150001490 con 0x7fe15806d340 2024-01-27T02:37:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.985+0000 7fe155ffb700 1 -- 172.21.15.7:0/2372132184 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe144003570 con 0x7fe15806d340 2024-01-27T02:37:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.985+0000 7fe156ffd700 1 -- 172.21.15.7:0/2372132184 >> v2:172.21.15.188:3300/0 conn(0x7fe1580c8ce0 msgr2=0x7fe1580cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.985+0000 7fe156ffd700 1 --2- 172.21.15.7:0/2372132184 >> v2:172.21.15.188:3300/0 conn(0x7fe1580c8ce0 0x7fe1580cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.986+0000 7fe156ffd700 1 -- 172.21.15.7:0/2372132184 >> v1:172.21.15.7:6789/0 conn(0x7fe15806d340 legacy=0x7fe1580c0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.986+0000 7fe156ffd700 1 -- 172.21.15.7:0/2372132184 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1580c7f20 con 0x7fe158156580 2024-01-27T02:37:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.986+0000 7fe156ffd700 1 --2- 172.21.15.7:0/2372132184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe158156580 0x7fe1580cb710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe14800aa90 tx=0x7fe14800a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=864035cbaa190493 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.986+0000 7fe155ffb700 1 -- 172.21.15.7:0/2372132184 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14800c970 con 0x7fe158156580 2024-01-27T02:37:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.986+0000 7fe155ffb700 1 -- 172.21.15.7:0/2372132184 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14800cad0 con 0x7fe158156580 2024-01-27T02:37:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.986+0000 7fe155ffb700 1 -- 172.21.15.7:0/2372132184 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14800ce90 con 0x7fe158156580 2024-01-27T02:37:22.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.986+0000 7fe15d7ac700 1 -- 172.21.15.7:0/2372132184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe158156580 msgr2=0x7fe1580cb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:22.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.986+0000 7fe15d7ac700 1 --2- 172.21.15.7:0/2372132184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe158156580 0x7fe1580cb710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe14800aa90 tx=0x7fe14800a330 comp rx=0 tx=0).stop 2024-01-27T02:37:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.987+0000 7fe15d7ac700 1 -- 172.21.15.7:0/2372132184 shutdown_connections 2024-01-27T02:37:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.987+0000 7fe15d7ac700 1 --2- 172.21.15.7:0/2372132184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe158156580 0x7fe1580cb710 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.987+0000 7fe15d7ac700 1 --2- 172.21.15.7:0/2372132184 >> v2:172.21.15.188:3300/0 conn(0x7fe1580c8ce0 0x7fe1580cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.987+0000 7fe15d7ac700 1 -- 172.21.15.7:0/2372132184 >> 172.21.15.7:0/2372132184 conn(0x7fe158063580 msgr2=0x7fe158063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.987+0000 7fe15d7ac700 1 -- 172.21.15.7:0/2372132184 shutdown_connections 2024-01-27T02:37:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.987+0000 7fe15d7ac700 1 -- 172.21.15.7:0/2372132184 wait complete. 2024-01-27T02:37:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.988+0000 7fe15d7ac700 1 Processor -- start 2024-01-27T02:37:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.988+0000 7fe15d7ac700 1 -- start start 2024-01-27T02:37:22.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.989+0000 7fe15d7ac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15806d340 0x7fe158152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:22.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.989+0000 7fe15d7ac700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1580c8ce0 0x7fe1580c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:22.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.989+0000 7fe15d7ac700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe158156580 0x7fe15815c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:22.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.989+0000 7fe15d7ac700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15815bfb0 con 0x7fe15806d340 2024-01-27T02:37:22.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.989+0000 7fe15d7ac700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15815b3a0 con 0x7fe158156580 2024-01-27T02:37:22.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.989+0000 7fe15d7ac700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15815b680 con 0x7fe1580c8ce0 2024-01-27T02:37:22.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.989+0000 7fe156ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15806d340 0x7fe158152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:22.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.989+0000 7fe156ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15806d340 0x7fe158152aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54792/0 (socket says 172.21.15.7:54792) 2024-01-27T02:37:22.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.989+0000 7fe156ffd700 1 -- 172.21.15.7:0/3536682047 learned_addr learned my addr 172.21.15.7:0/3536682047 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:22.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.989+0000 7fe1567fc700 1 --2- 172.21.15.7:0/3536682047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1580c8ce0 0x7fe1580c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:22.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.989+0000 7fe1577fe700 1 --2- 172.21.15.7:0/3536682047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe158156580 0x7fe15815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:22.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.989+0000 7fe156ffd700 1 -- 172.21.15.7:0/3536682047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1580c8ce0 msgr2=0x7fe1580c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:22.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.989+0000 7fe156ffd700 1 --2- 172.21.15.7:0/3536682047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1580c8ce0 0x7fe1580c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:22.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.989+0000 7fe156ffd700 1 -- 172.21.15.7:0/3536682047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe158156580 msgr2=0x7fe15815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:22.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.989+0000 7fe156ffd700 1 --2- 172.21.15.7:0/3536682047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe158156580 0x7fe15815c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:22.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.989+0000 7fe156ffd700 1 -- 172.21.15.7:0/3536682047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14800a5d0 con 0x7fe15806d340 2024-01-27T02:37:22.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.989+0000 7fe1567fc700 1 --2- 172.21.15.7:0/3536682047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1580c8ce0 0x7fe1580c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:37:22.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.989+0000 7fe1577fe700 1 --2- 172.21.15.7:0/3536682047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe158156580 0x7fe15815c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:37:22.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.989+0000 7fe156ffd700 1 --2- 172.21.15.7:0/3536682047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15806d340 0x7fe158152aa0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe1580c7d80 tx=0x7fe14801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.990+0000 7fe14effd700 1 -- 172.21.15.7:0/3536682047 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480213f0 con 0x7fe15806d340 2024-01-27T02:37:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.990+0000 7fe14effd700 1 -- 172.21.15.7:0/3536682047 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe148021550 con 0x7fe15806d340 2024-01-27T02:37:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.990+0000 7fe15d7ac700 1 -- 172.21.15.7:0/3536682047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe15815b960 con 0x7fe15806d340 2024-01-27T02:37:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.990+0000 7fe14effd700 1 -- 172.21.15.7:0/3536682047 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe148021910 con 0x7fe15806d340 2024-01-27T02:37:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.990+0000 7fe15d7ac700 1 -- 172.21.15.7:0/3536682047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe158152fe0 con 0x7fe15806d340 2024-01-27T02:37:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.991+0000 7fe15d7ac700 1 -- 172.21.15.7:0/3536682047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15804c710 con 0x7fe15806d340 2024-01-27T02:37:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.991+0000 7fe14effd700 1 -- 172.21.15.7:0/3536682047 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe148021d00 con 0x7fe15806d340 2024-01-27T02:37:22.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.992+0000 7fe14effd700 1 --2- 172.21.15.7:0/3536682047 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe138066f30 0x7fe1380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:22.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.992+0000 7fe14effd700 1 -- 172.21.15.7:0/3536682047 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe148085910 con 0x7fe15806d340 2024-01-27T02:37:22.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.992+0000 7fe14effd700 1 -- 172.21.15.7:0/3536682047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=103}) v3 -- 0x7fe13806ea50 con 0x7fe15806d340 2024-01-27T02:37:22.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.992+0000 7fe1567fc700 1 --2- 172.21.15.7:0/3536682047 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe138066f30 0x7fe1380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:22.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.993+0000 7fe1567fc700 1 --2- 172.21.15.7:0/3536682047 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe138066f30 0x7fe1380693e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe158154470 tx=0x7fe140008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:22.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:22.996+0000 7fe14effd700 1 -- 172.21.15.7:0/3536682047 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe148052b80 con 0x7fe15806d340 2024-01-27T02:37:23.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.250+0000 7fe15d7ac700 1 -- 172.21.15.7:0/3536682047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1580c8200 con 0x7fe15806d340 2024-01-27T02:37:23.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.251+0000 7fe14effd700 1 -- 172.21.15.7:0/3536682047 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4584 (secure 0 0 0) 0x7fe1480561a0 con 0x7fe15806d340 2024-01-27T02:37:23.251 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:23.251 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/450556982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":450556982},{"type":"v1","addr":"172.21.15.188:6836","nonce":450556982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1711422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1711422},{"type":"v1","addr":"172.21.15.188:6837","nonce":1711422}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6294,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1110306600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1110306600},{"type":"v1","addr":"172.21.15.7:6833","nonce":1110306600}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:37:03.404355+0000","modified":"2024-01-27T02:37:06.988798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6297":{"gid":6297,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/328992853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":328992853},{"type":"v1","addr":"172.21.15.7:6835","nonce":328992853}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-27T02:37:23.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.253+0000 7fe15d7ac700 1 -- 172.21.15.7:0/3536682047 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe138066f30 msgr2=0x7fe1380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:23.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.253+0000 7fe15d7ac700 1 --2- 172.21.15.7:0/3536682047 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe138066f30 0x7fe1380693e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe158154470 tx=0x7fe140008040 comp rx=0 tx=0).stop 2024-01-27T02:37:23.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.253+0000 7fe15d7ac700 1 -- 172.21.15.7:0/3536682047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15806d340 msgr2=0x7fe158152aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:23.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.253+0000 7fe15d7ac700 1 --2- 172.21.15.7:0/3536682047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15806d340 0x7fe158152aa0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe1580c7d80 tx=0x7fe14801a000 comp rx=0 tx=0).stop 2024-01-27T02:37:23.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.253+0000 7fe15d7ac700 1 -- 172.21.15.7:0/3536682047 shutdown_connections 2024-01-27T02:37:23.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.253+0000 7fe15d7ac700 1 --2- 172.21.15.7:0/3536682047 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe138066f30 0x7fe1380693e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:23.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.253+0000 7fe15d7ac700 1 --2- 172.21.15.7:0/3536682047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe158156580 0x7fe15815c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:23.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.253+0000 7fe15d7ac700 1 --2- 172.21.15.7:0/3536682047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15806d340 0x7fe158152aa0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:23.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.253+0000 7fe15d7ac700 1 --2- 172.21.15.7:0/3536682047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1580c8ce0 0x7fe1580c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:23.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.253+0000 7fe15d7ac700 1 -- 172.21.15.7:0/3536682047 >> 172.21.15.7:0/3536682047 conn(0x7fe158063580 msgr2=0x7fe1580c6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.253+0000 7fe15d7ac700 1 -- 172.21.15.7:0/3536682047 shutdown_connections 2024-01-27T02:37:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.253+0000 7fe15d7ac700 1 -- 172.21.15.7:0/3536682047 wait complete. 2024-01-27T02:37:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 49 2024-01-27T02:37:23.267 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-01-27T02:37:23.419 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","whoami":0,"id":6297,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":19.427483528,"mdsmap_epoch":49,"osdmap_epoch":102,"osdmap_epoch_barrier":102,"uptime":25.270960134999999} 2024-01-27T02:37:23.420 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", "fs_name": "cephfs", "id": 6297, "mdsmap_epoch": 49, "osdmap_epoch": 102, "osdmap_epoch_barrier": 102, "rank_uptime": 19.427483528, "state": "up:active", "uptime": 25.270960135, "want_state": "up:active", "whoami": 0 } 2024-01-27T02:37:23.420 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-01-27T02:37:23.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:37:23.420 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2024-01-27T02:37:23.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-27T02:37:23.439 INFO:teuthology.orchestra.run.smithi007.stderr:dd: failed to open 'small_file_1': Permission denied 2024-01-27T02:37:23.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:37:23.440 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-01-27T02:37:23.501 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:37:23.501 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-01-27T02:37:23.585 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:37:23.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.722+0000 7f9d74a20700 1 Processor -- start 2024-01-27T02:37:23.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.723+0000 7f9d74a20700 1 -- start start 2024-01-27T02:37:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.723+0000 7f9d74a20700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d700c4cc0 0x7f9d700cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.723+0000 7f9d74a20700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9d700ca540 0x7f9d70163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.723+0000 7f9d74a20700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d700ce5e0 con 0x7f9d7015a100 2024-01-27T02:37:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.723+0000 7f9d74a20700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d700cd9d0 con 0x7f9d700ca540 2024-01-27T02:37:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.723+0000 7f9d74a20700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d700cdcb0 con 0x7f9d700c4cc0 2024-01-27T02:37:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.724+0000 7f9d6ed9d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9d700ca540 0x7f9d70163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.724+0000 7f9d65d9b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d700c4cc0 0x7f9d700cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:23.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.724+0000 7f9d6e59c700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f9d7015a100 0x7f9d700c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:35426/0 (socket says 172.21.15.7:35426) 2024-01-27T02:37:23.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.724+0000 7f9d6ed9d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9d700ca540 0x7f9d70163d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:55908/0 (socket says 172.21.15.7:55908) 2024-01-27T02:37:23.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.724+0000 7f9d6e59c700 1 -- 172.21.15.7:0/2942074973 learned_addr learned my addr 172.21.15.7:0/2942074973 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:23.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.724+0000 7f9d65d9b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d700c4cc0 0x7f9d700cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:36312/0 (socket says 172.21.15.7:36312) 2024-01-27T02:37:23.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.724+0000 7f9d65d9b700 1 -- 172.21.15.7:0/2942074973 >> v2:172.21.15.188:3300/0 conn(0x7f9d700ca540 msgr2=0x7f9d70163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:23.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.724+0000 7f9d65d9b700 1 --2- 172.21.15.7:0/2942074973 >> v2:172.21.15.188:3300/0 conn(0x7f9d700ca540 0x7f9d70163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:23.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.724+0000 7f9d65d9b700 1 -- 172.21.15.7:0/2942074973 >> v1:172.21.15.188:6789/0 conn(0x7f9d7015a100 legacy=0x7f9d700c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:23.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.724+0000 7f9d65d9b700 1 -- 172.21.15.7:0/2942074973 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d700cdf90 con 0x7f9d700c4cc0 2024-01-27T02:37:23.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.724+0000 7f9d65d9b700 1 --2- 172.21.15.7:0/2942074973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d700c4cc0 0x7f9d700cd2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9d580098e0 tx=0x7f9d580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2286265520653a48 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:23.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.725+0000 7f9d6dd9b700 1 -- 172.21.15.7:0/2942074973 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5800b640 con 0x7f9d700c4cc0 2024-01-27T02:37:23.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.725+0000 7f9d6dd9b700 1 -- 172.21.15.7:0/2942074973 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5800b7a0 con 0x7f9d700c4cc0 2024-01-27T02:37:23.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.725+0000 7f9d6dd9b700 1 -- 172.21.15.7:0/2942074973 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5800bb40 con 0x7f9d700c4cc0 2024-01-27T02:37:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.725+0000 7f9d74a20700 1 -- 172.21.15.7:0/2942074973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d700c4cc0 msgr2=0x7f9d700cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.725+0000 7f9d74a20700 1 --2- 172.21.15.7:0/2942074973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d700c4cc0 0x7f9d700cd2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9d580098e0 tx=0x7f9d580091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.725+0000 7f9d74a20700 1 -- 172.21.15.7:0/2942074973 shutdown_connections 2024-01-27T02:37:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.725+0000 7f9d74a20700 1 --2- 172.21.15.7:0/2942074973 >> v2:172.21.15.188:3300/0 conn(0x7f9d700ca540 0x7f9d70163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.725+0000 7f9d74a20700 1 --2- 172.21.15.7:0/2942074973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d700c4cc0 0x7f9d700cd2f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.725+0000 7f9d74a20700 1 -- 172.21.15.7:0/2942074973 >> 172.21.15.7:0/2942074973 conn(0x7f9d70098020 msgr2=0x7f9d700711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.725+0000 7f9d74a20700 1 -- 172.21.15.7:0/2942074973 shutdown_connections 2024-01-27T02:37:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.725+0000 7f9d74a20700 1 -- 172.21.15.7:0/2942074973 wait complete. 2024-01-27T02:37:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.727+0000 7f9d74a20700 1 Processor -- start 2024-01-27T02:37:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.727+0000 7f9d74a20700 1 -- start start 2024-01-27T02:37:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.727+0000 7f9d74a20700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d700c4cc0 0x7f9d7015bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:23.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.727+0000 7f9d74a20700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d700ca540 0x7f9d70152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:23.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.727+0000 7f9d74a20700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d7015a100 0x7f9d70068170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:23.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.727+0000 7f9d74a20700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7015d0d0 con 0x7f9d700ca540 2024-01-27T02:37:23.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.727+0000 7f9d74a20700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7015c4c0 con 0x7f9d7015a100 2024-01-27T02:37:23.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.727+0000 7f9d74a20700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7015c7a0 con 0x7f9d700c4cc0 2024-01-27T02:37:23.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.728+0000 7f9d65d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d700ca540 0x7f9d70152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:23.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.728+0000 7f9d65d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d700ca540 0x7f9d70152380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54802/0 (socket says 172.21.15.7:54802) 2024-01-27T02:37:23.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.728+0000 7f9d65d9b700 1 -- 172.21.15.7:0/1778856949 learned_addr learned my addr 172.21.15.7:0/1778856949 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:23.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.728+0000 7f9d6e59c700 1 --2- 172.21.15.7:0/1778856949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d700c4cc0 0x7f9d7015bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:23.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.728+0000 7f9d6ed9d700 1 --2- 172.21.15.7:0/1778856949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d7015a100 0x7f9d70068170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:23.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.728+0000 7f9d65d9b700 1 -- 172.21.15.7:0/1778856949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d700c4cc0 msgr2=0x7f9d7015bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:23.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.728+0000 7f9d65d9b700 1 --2- 172.21.15.7:0/1778856949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d700c4cc0 0x7f9d7015bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:23.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.728+0000 7f9d65d9b700 1 -- 172.21.15.7:0/1778856949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d7015a100 msgr2=0x7f9d70068170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:23.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.728+0000 7f9d65d9b700 1 --2- 172.21.15.7:0/1778856949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d7015a100 0x7f9d70068170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:23.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.728+0000 7f9d65d9b700 1 -- 172.21.15.7:0/1778856949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d58009420 con 0x7f9d700ca540 2024-01-27T02:37:23.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.728+0000 7f9d65d9b700 1 --2- 172.21.15.7:0/1778856949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d700ca540 0x7f9d70152380 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800a010 tx=0x7f9d58017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:23.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.728+0000 7f9d66ffd700 1 -- 172.21.15.7:0/1778856949 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5801bc80 con 0x7f9d700ca540 2024-01-27T02:37:23.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.728+0000 7f9d66ffd700 1 -- 172.21.15.7:0/1778856949 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5801be20 con 0x7f9d700ca540 2024-01-27T02:37:23.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.728+0000 7f9d74a20700 1 -- 172.21.15.7:0/1778856949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7015ca80 con 0x7f9d700ca540 2024-01-27T02:37:23.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.728+0000 7f9d66ffd700 1 -- 172.21.15.7:0/1778856949 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d58011670 con 0x7f9d700ca540 2024-01-27T02:37:23.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.728+0000 7f9d74a20700 1 -- 172.21.15.7:0/1778856949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7015c100 con 0x7f9d700ca540 2024-01-27T02:37:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.729+0000 7f9d74a20700 1 -- 172.21.15.7:0/1778856949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d7004c710 con 0x7f9d700ca540 2024-01-27T02:37:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.730+0000 7f9d66ffd700 1 -- 172.21.15.7:0/1778856949 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9d58011aa0 con 0x7f9d700ca540 2024-01-27T02:37:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.730+0000 7f9d66ffd700 1 --2- 172.21.15.7:0/1778856949 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9d50066f30 0x7f9d500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.731+0000 7f9d66ffd700 1 -- 172.21.15.7:0/1778856949 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9d58085c10 con 0x7f9d700ca540 2024-01-27T02:37:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.731+0000 7f9d66ffd700 1 -- 172.21.15.7:0/1778856949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=103}) v3 -- 0x7f9d5006ea50 con 0x7f9d700ca540 2024-01-27T02:37:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.731+0000 7f9d6e59c700 1 --2- 172.21.15.7:0/1778856949 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9d50066f30 0x7f9d500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.731+0000 7f9d6e59c700 1 --2- 172.21.15.7:0/1778856949 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9d50066f30 0x7f9d500693e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9d60007130 tx=0x7f9d60007eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:23.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.735+0000 7f9d66ffd700 1 -- 172.21.15.7:0/1778856949 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d58052e80 con 0x7f9d700ca540 2024-01-27T02:37:23.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.941+0000 7f9d74a20700 1 -- 172.21.15.7:0/1778856949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f9d700ce270 con 0x7f9d700ca540 2024-01-27T02:37:23.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.942+0000 7f9d66ffd700 1 -- 172.21.15.7:0/1778856949 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v102) v1 ==== 81+0+32383 (secure 0 0 0) 0x7f9d580564a0 con 0x7f9d700ca540 2024-01-27T02:37:23.943 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:23.943 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:37:23.943 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 102, 2024-01-27T02:37:23.943 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:37:23.943 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:37:23.943 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:37:22.129521+0000", 2024-01-27T02:37:23.943 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:37:23.943 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:37:23.943 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:37:23.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:37:23.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:37:23.944 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:37:23.944 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:37:23.944 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:37:23.944 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:37:23.944 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:23.944 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:37:23.944 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:37:23.944 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:37:23.944 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:37:23.944 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:37:23.945 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 13, 2024-01-27T02:37:23.945 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:37:23.945 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:37:23.945 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:37:23.945 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:37:23.945 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:37:23.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.945 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:37:23.945 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:37:23.945 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:37:23.945 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:37:23.945 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:37:23.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:37:23.945 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:37:23.946 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:37:23.946 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:37:23.946 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:37:23.946 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:37:23.946 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:37:23.946 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:37:23.946 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:37:23.946 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:37:23.946 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:37:23.946 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:37:23.947 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:37:23.947 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:37:23.947 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:37:23.947 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:37:23.947 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:37:23.947 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:37:23.947 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:37:23.947 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:37:23.947 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:37:23.947 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:37:23.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.947 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "99", 2024-01-27T02:37:23.947 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:37:23.947 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:37:23.947 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:37:23.948 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:37:23.948 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:37:23.948 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:37:23.948 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:37:23.948 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:37:23.948 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:37:23.948 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:37:23.948 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:37:23.948 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:37:23.948 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:37:23.948 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:37:23.948 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:37:23.948 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:37:23.949 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:37:23.949 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:37:23.949 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:37:23.949 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:37:23.949 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:37:23.949 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:37:23.949 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:37:23.949 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:37:23.949 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:37:23.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:37:23.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.949 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:37:23.949 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:37:23.949 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:37:23.950 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:37:23.950 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:37:23.950 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:37:23.950 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:37:23.950 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:37:23.950 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:37:23.950 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:37:23.950 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:37:23.950 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:37:23.950 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:37:23.950 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:37:23.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.950 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:37:23.950 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:37:23.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.951 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 12, 2024-01-27T02:37:23.951 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:37:23.951 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:37:01.445734+0000", 2024-01-27T02:37:23.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:37:23.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:37:23.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:37:23.951 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:37:23.951 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:37:23.951 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:37:23.951 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:37:23.951 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:37:23.952 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:37:23.952 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:37:23.952 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:37:23.952 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:37:23.952 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:37:23.952 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:37:23.952 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:37:23.952 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:37:23.952 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:37:23.952 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:37:23.952 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:37:23.952 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:37:23.952 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:37:23.952 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:37:23.953 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:37:23.953 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:37:23.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.953 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "99", 2024-01-27T02:37:23.953 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:37:23.953 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:37:23.953 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:37:23.953 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:37:23.953 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:37:23.953 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:37:23.953 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:37:23.954 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:37:23.954 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:37:23.954 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:37:23.954 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:37:23.954 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:37:23.954 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:37:23.954 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:37:23.954 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:37:23.954 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:37:23.954 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:37:23.955 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:37:23.955 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:37:23.955 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:37:23.955 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:37:23.955 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:37:23.955 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:37:23.955 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:37:23.955 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:37:23.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:37:23.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.955 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:37:23.956 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:37:23.956 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:37:23.956 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:37:23.956 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:37:23.956 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:37:23.956 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:37:23.956 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:37:23.956 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:37:23.956 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:37:23.957 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:37:23.957 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:37:23.957 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:37:23.957 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:37:23.957 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:37:23.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.957 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:37:23.957 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:37:23.957 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:37:23.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.958 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 13, 2024-01-27T02:37:23.958 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:37:23.958 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:37:02.377151+0000", 2024-01-27T02:37:23.958 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:37:23.958 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:37:23.958 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:37:23.958 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:37:23.958 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:37:23.958 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:37:23.958 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:37:23.958 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:37:23.958 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:37:23.959 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:37:23.959 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:37:23.959 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:37:23.959 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:37:23.959 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:37:23.959 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:37:23.959 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:37:23.959 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:37:23.959 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:37:23.959 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:37:23.959 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:37:23.959 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:37:23.959 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:37:23.960 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:37:23.960 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:37:23.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.960 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "99", 2024-01-27T02:37:23.960 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:37:23.960 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:37:23.960 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:37:23.960 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:37:23.961 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:37:23.961 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:37:23.961 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:37:23.961 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:37:23.961 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:37:23.961 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:37:23.961 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:37:23.961 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:37:23.961 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:37:23.961 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:37:23.961 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:37:23.962 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:37:23.963 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:37:23.963 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:37:23.963 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:37:23.963 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:37:23.963 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:37:23.963 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:37:23.963 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:37:23.963 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:37:23.963 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:37:23.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:37:23.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.963 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:37:23.963 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:37:23.963 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:37:23.964 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:37:23.964 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:37:23.964 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:37:23.964 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:37:23.964 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:37:23.964 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:37:23.964 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:37:23.964 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:37:23.964 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:37:23.964 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:37:23.964 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:37:23.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.964 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:37:23.965 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:37:23.965 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:37:23.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.965 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:23.965 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:37:23.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.965 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:37:23.965 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:23.965 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:23.965 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:23.965 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:23.965 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:23.966 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:23.966 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:23.966 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:23.966 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:23.966 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:23.966 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:23.966 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:23.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:37:23.966 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:23.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:23.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:37:23.967 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:23.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.967 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.967 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:23.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:37:23.968 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:23.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:23.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:37:23.968 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:23.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.969 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.969 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:23.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:37:23.969 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:23.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:23.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:37:23.969 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:23.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.970 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.970 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:23.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:37:23.970 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:23.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:23.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:37:23.970 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:23.970 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.970 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.971 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:37:23.971 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:37:23.971 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:37:23.971 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:37:23.971 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:23.971 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:23.971 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:37:23.971 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:23.971 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.971 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:37:23.972 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:23.972 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:23.972 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:23.972 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:23.972 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:23.972 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:23.972 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:23.972 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:23.972 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:23.972 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:23.973 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:23.973 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:23.973 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.973 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:37:23.973 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:23.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:23.974 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:37:23.974 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:23.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.974 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.974 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:23.974 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.974 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:37:23.974 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:23.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:23.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:37:23.975 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:23.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.975 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.975 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:23.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:37:23.975 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:23.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:23.976 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:37:23.976 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:23.976 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.976 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.976 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:23.976 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.976 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:37:23.976 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:23.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:23.977 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:37:23.977 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:23.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.977 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.977 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:37:23.977 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:37:23.977 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:37:23.977 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:37:23.977 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:23.977 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:23.977 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:37:23.977 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:23.978 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.978 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:37:23.978 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:23.978 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:23.978 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:23.978 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:23.978 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:23.978 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:23.979 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:23.979 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:23.979 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:23.979 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:23.979 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:23.979 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:23.979 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.979 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.979 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:37:23.979 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:23.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.979 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:23.980 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:37:23.980 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:23.980 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.980 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.980 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:23.980 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.980 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:37:23.980 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:23.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:23.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:37:23.981 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:23.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.981 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.981 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:23.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:37:23.981 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:23.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:23.982 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:37:23.982 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:23.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.982 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.982 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:23.982 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.982 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:37:23.982 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:23.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:23.983 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:37:23.983 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:23.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.983 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.984 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:37:23.984 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:37:23.984 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:37:23.984 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:37:23.984 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:23.984 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:23.984 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:37:23.984 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:23.984 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.984 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:37:23.985 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:23.985 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:23.985 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:23.985 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:23.985 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:23.985 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:23.985 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:23.985 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:23.985 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:23.985 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:23.985 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:23.985 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:23.985 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.986 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:37:23.986 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:23.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:23.986 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:37:23.986 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:23.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.986 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.986 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:23.986 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.987 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:37:23.987 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:23.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:23.987 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:37:23.987 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:23.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.987 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.987 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:23.987 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:37:23.988 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:23.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:23.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:37:23.988 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:23.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.988 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.988 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:23.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.989 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:37:23.989 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:23.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:23.989 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:37:23.989 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:23.989 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.989 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.990 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:37:23.990 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:37:23.990 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:37:23.990 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:37:23.990 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:23.990 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:23.990 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:23.990 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.990 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:37:23.990 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:23.990 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:23.991 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:23.991 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:23.991 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:23.991 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:23.991 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:23.991 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:23.991 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:23.991 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:23.991 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:23.991 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:23.991 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.991 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:37:23.992 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:23.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:23.992 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:37:23.992 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:23.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.992 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.992 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:23.992 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.992 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:37:23.993 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:23.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:23.993 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:37:23.993 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:23.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.993 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.993 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:23.993 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.993 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:37:23.994 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:23.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:23.994 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:37:23.994 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:23.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.994 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.994 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:23.994 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.994 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:37:23.994 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:23.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:23.995 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:37:23.995 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:23.995 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.995 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.995 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:37:23.995 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:37:23.995 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:37:23.995 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:37:23.995 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:23.995 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:23.996 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:37:23.996 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:23.996 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.996 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:37:23.996 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:23.996 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:23.996 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:23.996 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:23.996 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:23.996 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:23.996 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:23.996 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:23.997 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:23.997 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:23.997 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:23.997 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:23.997 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.997 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:37:23.997 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:23.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:23.997 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:37:23.997 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:23.998 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.998 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.998 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:23.998 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.998 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:37:23.998 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:23.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:23.998 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:37:23.998 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:23.998 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:23.999 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:23.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.999 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:23.999 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:23.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.999 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:23.999 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:37:23.999 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:23.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:23.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:23.999 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:24.000 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:37:24.000 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:24.000 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:24.000 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:24.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.000 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:24.000 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:24.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:24.000 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:37:24.000 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:24.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.001 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:24.001 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:37:24.001 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:24.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:24.001 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:24.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.001 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:37:24.001 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:37:24.001 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:37:24.001 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:37:24.001 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:24.001 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:24.001 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:24.002 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:24.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.002 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:37:24.002 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:24.002 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:24.002 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:24.002 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:24.002 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:24.002 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:24.002 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:24.002 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:24.002 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:24.002 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:24.002 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:24.003 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:24.003 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:24.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.003 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:24.003 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:37:24.003 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:24.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.003 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:24.003 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:37:24.003 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:24.003 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:24.003 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:24.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.004 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:24.004 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:24.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.004 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:24.004 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:37:24.004 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:24.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.004 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:24.004 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:37:24.004 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:24.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:24.005 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:24.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.005 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:24.005 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:24.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:24.005 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:37:24.005 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:24.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:24.006 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:37:24.006 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:24.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:24.006 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:24.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.006 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:24.006 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:24.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.006 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:24.006 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:37:24.006 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:24.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:24.007 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:37:24.007 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:24.007 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:24.007 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:24.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.007 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:37:24.007 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:37:24.007 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:37:24.007 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:37:24.007 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:24.007 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:24.007 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:37:24.007 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:24.008 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:24.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.008 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:37:24.008 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:24.008 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:24.008 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:24.008 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:24.008 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:24.008 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:24.008 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:24.008 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:24.008 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:24.009 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:24.009 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:24.009 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:24.009 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:24.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.009 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:24.009 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:37:24.009 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:24.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.009 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:24.009 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:37:24.009 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:24.009 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:24.010 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:24.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.010 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:24.010 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:24.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.010 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:24.010 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:37:24.010 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:24.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:24.011 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:37:24.011 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:24.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:24.011 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:24.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.011 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:24.011 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:24.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:24.011 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:37:24.011 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:24.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:24.012 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:37:24.012 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:24.012 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:24.012 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:24.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.012 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:24.012 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:24.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:24.012 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:37:24.012 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:24.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:24.013 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:37:24.013 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:24.013 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:24.013 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:24.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.013 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:37:24.013 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:37:24.013 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:37:24.013 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:37:24.013 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:24.013 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull", 2024-01-27T02:37:24.013 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:24.014 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:24.014 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:24.014 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:24.014 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:24.014 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:37:24.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.014 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:37:24.014 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:24.014 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:24.014 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:24.014 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:24.014 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:24.014 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:37:24.015 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:24.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.015 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:37:24.015 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:24.015 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:24.015 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:24.015 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:24.015 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:24.015 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:37:24.015 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:24.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.016 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:37:24.016 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:24.016 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:24.016 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:24.016 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:24.016 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:24.016 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:37:24.016 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:24.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.016 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:37:24.016 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:24.016 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:24.017 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:24.017 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:24.017 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:24.017 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:37:24.017 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:24.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.017 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:37:24.017 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:24.017 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:24.017 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:24.017 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:24.017 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:24.017 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:37:24.018 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:24.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.018 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:37:24.018 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:24.018 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:24.018 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:24.018 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:24.018 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:24.018 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:37:24.018 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:24.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.018 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:37:24.018 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:24.019 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:24.019 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:24.019 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:24.019 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:24.019 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:37:24.019 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:24.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:24.019 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:37:24.019 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:24.019 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:24.019 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:24.019 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:24.019 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:24.020 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:37:24.020 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:24.020 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:24.020 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:24.020 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:37:24.020 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:37:24.020 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:37:24.020 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:37:24.020 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:37:24.020 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:37:24.021 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:37:24.021 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:37:24.021 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:37:24.021 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:37:24.021 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:37:24.021 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:37:24.021 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:37:24.021 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:24.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:24.021 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:37:24.021 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:37:24.021 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:37:24.022 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:37:24.022 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:37:24.022 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:37:24.022 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:37:24.022 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:37:24.022 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:37:24.022 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:37:24.022 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:37:24.022 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:24.022 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:37:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.944+0000 7f9d74a20700 1 -- 172.21.15.7:0/1778856949 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9d50066f30 msgr2=0x7f9d500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.944+0000 7f9d74a20700 1 --2- 172.21.15.7:0/1778856949 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9d50066f30 0x7f9d500693e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9d60007130 tx=0x7f9d60007eb0 comp rx=0 tx=0).stop 2024-01-27T02:37:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.944+0000 7f9d74a20700 1 -- 172.21.15.7:0/1778856949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d700ca540 msgr2=0x7f9d70152380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.944+0000 7f9d74a20700 1 --2- 172.21.15.7:0/1778856949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d700ca540 0x7f9d70152380 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800a010 tx=0x7f9d58017000 comp rx=0 tx=0).stop 2024-01-27T02:37:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.944+0000 7f9d74a20700 1 -- 172.21.15.7:0/1778856949 shutdown_connections 2024-01-27T02:37:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.944+0000 7f9d74a20700 1 --2- 172.21.15.7:0/1778856949 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9d50066f30 0x7f9d500693e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.944+0000 7f9d74a20700 1 --2- 172.21.15.7:0/1778856949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d7015a100 0x7f9d70068170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.944+0000 7f9d74a20700 1 --2- 172.21.15.7:0/1778856949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d700c4cc0 0x7f9d7015bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.944+0000 7f9d74a20700 1 --2- 172.21.15.7:0/1778856949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d700ca540 0x7f9d70152380 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.944+0000 7f9d74a20700 1 -- 172.21.15.7:0/1778856949 >> 172.21.15.7:0/1778856949 conn(0x7f9d70098020 msgr2=0x7f9d7006c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:24.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.944+0000 7f9d74a20700 1 -- 172.21.15.7:0/1778856949 shutdown_connections 2024-01-27T02:37:24.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:23.944+0000 7f9d74a20700 1 -- 172.21.15.7:0/1778856949 wait complete. 2024-01-27T02:37:24.025 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-27T02:37:24.851 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-27T02:37:24.851+0000 7fb5c783f700 -1 log_channel(cluster) log [ERR] : Health check update: 5 full osd(s) (OSD_FULL) 2024-01-27T02:37:29.026 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:37:29.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.166+0000 7fb74a453700 1 Processor -- start 2024-01-27T02:37:29.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.166+0000 7fb74a453700 1 -- start start 2024-01-27T02:37:29.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.167+0000 7fb74a453700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb7440d1470 0x7fb7440b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:29.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.167+0000 7fb74a453700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7440b4650 con 0x7fb744159b40 2024-01-27T02:37:29.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.167+0000 7fb74a453700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7440b3d60 con 0x7fb7440d10a0 2024-01-27T02:37:29.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.167+0000 7fb74a453700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7440b34d0 con 0x7fb7440d1470 2024-01-27T02:37:29.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.167+0000 7fb7437fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb7440d1470 0x7fb7440b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:29.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.167+0000 7fb7489f0700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb744159b40 0x7fb744159f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47500/0 (socket says 172.21.15.7:47500) 2024-01-27T02:37:29.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.167+0000 7fb7489f0700 1 -- 172.21.15.7:0/594804435 learned_addr learned my addr 172.21.15.7:0/594804435 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:29.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.167+0000 7fb7437fe700 1 -- 172.21.15.7:0/594804435 >> v1:172.21.15.188:6789/0 conn(0x7fb7440d10a0 legacy=0x7fb7440b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:29.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.168+0000 7fb7437fe700 1 -- 172.21.15.7:0/594804435 >> v1:172.21.15.7:6789/0 conn(0x7fb744159b40 legacy=0x7fb744159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:29.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.168+0000 7fb7437fe700 1 -- 172.21.15.7:0/594804435 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7440b4000 con 0x7fb7440d1470 2024-01-27T02:37:29.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.168+0000 7fb742ffd700 1 -- 172.21.15.7:0/594804435 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 478727668 0 0) 0x7fb7440b4650 con 0x7fb744159b40 2024-01-27T02:37:29.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.168+0000 7fb7437fe700 1 --2- 172.21.15.7:0/594804435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7440d1470 0x7fb7440b1590 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb7380098e0 tx=0x7fb7380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529ef53ab9716f67 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.168+0000 7fb742ffd700 1 -- 172.21.15.7:0/594804435 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb738009e80 con 0x7fb7440d1470 2024-01-27T02:37:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.168+0000 7fb742ffd700 1 -- 172.21.15.7:0/594804435 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73800b860 con 0x7fb7440d1470 2024-01-27T02:37:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.168+0000 7fb742ffd700 1 -- 172.21.15.7:0/594804435 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73800bbe0 con 0x7fb7440d1470 2024-01-27T02:37:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.169+0000 7fb74a453700 1 -- 172.21.15.7:0/594804435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7440d1470 msgr2=0x7fb7440b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.169+0000 7fb74a453700 1 --2- 172.21.15.7:0/594804435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7440d1470 0x7fb7440b1590 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb7380098e0 tx=0x7fb7380091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.169+0000 7fb74a453700 1 -- 172.21.15.7:0/594804435 shutdown_connections 2024-01-27T02:37:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.169+0000 7fb74a453700 1 --2- 172.21.15.7:0/594804435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7440d1470 0x7fb7440b1590 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.169+0000 7fb74a453700 1 -- 172.21.15.7:0/594804435 >> 172.21.15.7:0/594804435 conn(0x7fb7440679b0 msgr2=0x7fb744067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.169+0000 7fb74a453700 1 -- 172.21.15.7:0/594804435 shutdown_connections 2024-01-27T02:37:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.169+0000 7fb74a453700 1 -- 172.21.15.7:0/594804435 wait complete. 2024-01-27T02:37:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.170+0000 7fb74a453700 1 Processor -- start 2024-01-27T02:37:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.171+0000 7fb74a453700 1 -- start start 2024-01-27T02:37:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.172+0000 7fb74a453700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7440d10a0 0x7fb7441561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.172+0000 7fb74a453700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7440d1470 0x7fb74415b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.172+0000 7fb74a453700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb744159b40 0x7fb7440c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.172+0000 7fb74a453700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7440c4c80 con 0x7fb744159b40 2024-01-27T02:37:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.172+0000 7fb74a453700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7441566f0 con 0x7fb7440d10a0 2024-01-27T02:37:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.172+0000 7fb74a453700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7440c9920 con 0x7fb7440d1470 2024-01-27T02:37:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.172+0000 7fb7489f0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb744159b40 0x7fb7440c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.172+0000 7fb7489f0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb744159b40 0x7fb7440c8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54812/0 (socket says 172.21.15.7:54812) 2024-01-27T02:37:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.172+0000 7fb7489f0700 1 -- 172.21.15.7:0/2566579005 learned_addr learned my addr 172.21.15.7:0/2566579005 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.172+0000 7fb7437fe700 1 --2- 172.21.15.7:0/2566579005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7440d1470 0x7fb74415b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.172+0000 7fb743fff700 1 --2- 172.21.15.7:0/2566579005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7440d10a0 0x7fb7441561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.172+0000 7fb7489f0700 1 -- 172.21.15.7:0/2566579005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7440d1470 msgr2=0x7fb74415b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.172+0000 7fb7489f0700 1 --2- 172.21.15.7:0/2566579005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7440d1470 0x7fb74415b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.172+0000 7fb7489f0700 1 -- 172.21.15.7:0/2566579005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7440d10a0 msgr2=0x7fb7441561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.172+0000 7fb7489f0700 1 --2- 172.21.15.7:0/2566579005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7440d10a0 0x7fb7441561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.172+0000 7fb7489f0700 1 -- 172.21.15.7:0/2566579005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb738009420 con 0x7fb744159b40 2024-01-27T02:37:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.172+0000 7fb7489f0700 1 --2- 172.21.15.7:0/2566579005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb744159b40 0x7fb7440c8ed0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00cc40 tx=0x7fb73c001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.172+0000 7fb72bfff700 1 -- 172.21.15.7:0/2566579005 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c00a920 con 0x7fb744159b40 2024-01-27T02:37:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.172+0000 7fb72bfff700 1 -- 172.21.15.7:0/2566579005 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73c00aaa0 con 0x7fb744159b40 2024-01-27T02:37:29.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.172+0000 7fb74a453700 1 -- 172.21.15.7:0/2566579005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7440c9c00 con 0x7fb744159b40 2024-01-27T02:37:29.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.172+0000 7fb72bfff700 1 -- 172.21.15.7:0/2566579005 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c00ae60 con 0x7fb744159b40 2024-01-27T02:37:29.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.173+0000 7fb74a453700 1 -- 172.21.15.7:0/2566579005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7440c4910 con 0x7fb744159b40 2024-01-27T02:37:29.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.173+0000 7fb74a453700 1 -- 172.21.15.7:0/2566579005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb74404c710 con 0x7fb744159b40 2024-01-27T02:37:29.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.174+0000 7fb72bfff700 1 -- 172.21.15.7:0/2566579005 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb73c007300 con 0x7fb744159b40 2024-01-27T02:37:29.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.174+0000 7fb72bfff700 1 --2- 172.21.15.7:0/2566579005 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb730066f30 0x7fb7300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:29.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.174+0000 7fb72bfff700 1 -- 172.21.15.7:0/2566579005 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb73c084950 con 0x7fb744159b40 2024-01-27T02:37:29.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.175+0000 7fb72bfff700 1 -- 172.21.15.7:0/2566579005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=104}) v3 -- 0x7fb73006ea50 con 0x7fb744159b40 2024-01-27T02:37:29.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.175+0000 7fb743fff700 1 --2- 172.21.15.7:0/2566579005 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb730066f30 0x7fb7300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:29.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.175+0000 7fb743fff700 1 --2- 172.21.15.7:0/2566579005 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb730066f30 0x7fb7300693e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb7440cd380 tx=0x7fb734009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:29.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.179+0000 7fb72bfff700 1 -- 172.21.15.7:0/2566579005 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb73c051bb0 con 0x7fb744159b40 2024-01-27T02:37:29.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.387+0000 7fb74a453700 1 -- 172.21.15.7:0/2566579005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb7440b42e0 con 0x7fb744159b40 2024-01-27T02:37:29.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.388+0000 7fb72bfff700 1 -- 172.21.15.7:0/2566579005 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v103) v1 ==== 81+0+32391 (secure 0 0 0) 0x7fb73c0551d0 con 0x7fb744159b40 2024-01-27T02:37:29.389 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:29.389 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:37:29.389 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 103, 2024-01-27T02:37:29.389 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:37:29.389 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:37:29.389 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:37:27.145363+0000", 2024-01-27T02:37:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:37:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:37:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:37:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:37:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:37:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:37:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:37:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:37:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:37:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:37:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:37:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:37:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:37:29.391 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:37:29.391 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 13, 2024-01-27T02:37:29.391 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:37:29.391 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:37:29.391 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:37:29.391 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:37:29.391 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:37:29.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.391 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:37:29.391 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:37:29.391 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:37:29.391 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:37:29.391 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:37:29.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:37:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:37:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:37:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:37:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:37:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:37:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:37:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:37:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:37:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:37:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:37:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:37:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:37:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:37:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:37:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:37:29.392 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:37:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:37:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:37:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:37:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:37:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:37:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "99", 2024-01-27T02:37:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:37:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:37:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:37:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:37:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:37:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:37:29.393 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:37:29.394 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:37:29.394 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:37:29.394 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:37:29.394 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:37:29.394 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:37:29.394 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:37:29.394 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:37:29.394 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:37:29.394 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:37:29.394 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:37:29.394 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:37:29.394 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:37:29.394 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:37:29.394 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:37:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:37:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:37:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:37:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:37:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:37:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:37:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:37:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:37:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:37:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:37:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:37:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:37:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:37:29.395 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:37:29.396 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:37:29.396 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:37:29.396 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:37:29.396 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:37:29.396 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:37:29.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.396 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:37:29.396 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:37:29.396 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.396 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 12, 2024-01-27T02:37:29.396 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:37:29.396 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:37:01.445734+0000", 2024-01-27T02:37:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:37:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:37:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:37:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:37:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:37:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:37:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:37:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:37:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:37:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:37:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:37:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:37:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:37:29.397 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:37:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:37:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:37:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:37:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:37:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:37:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:37:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:37:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:37:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:37:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:37:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "99", 2024-01-27T02:37:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:37:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:37:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:37:29.398 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:37:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:37:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:37:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:37:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:37:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:37:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:37:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:37:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:37:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:37:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:37:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:37:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:37:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:37:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:37:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:37:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:37:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:37:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:37:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:37:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:37:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:37:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:37:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:37:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:37:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:37:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:37:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:37:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:37:29.400 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:37:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:37:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:37:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:37:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:37:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:37:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:37:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:37:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:37:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:37:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:37:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:37:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 13, 2024-01-27T02:37:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:37:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:37:02.377151+0000", 2024-01-27T02:37:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-27T02:37:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-27T02:37:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:37:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:37:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:37:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:37:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:37:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:37:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:37:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:37:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:37:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:37:29.402 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:37:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:37:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:37:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:37:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:37:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:37:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:37:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:37:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:37:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:37:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:37:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:37:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "99", 2024-01-27T02:37:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:37:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:37:29.403 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:37:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:37:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:37:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:37:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:37:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:37:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:37:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:37:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:37:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:37:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:37:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:37:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:37:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:37:29.404 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:37:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:37:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:37:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:37:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:37:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:37:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:37:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:37:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:37:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:37:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:37:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:37:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:37:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:37:29.405 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:37:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:37:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:37:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:37:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:37:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:37:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:37:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:37:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:37:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:37:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:37:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:37:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:37:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.406 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:37:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:37:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:29.407 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:37:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:37:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:37:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:37:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.409 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:37:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:37:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:37:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:37:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:37:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:37:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:37:29.411 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:37:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:37:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:37:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:29.412 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:37:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:37:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.414 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:29.414 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.414 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:37:29.414 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:29.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.414 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:37:29.414 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:29.414 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.414 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:37:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:37:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.416 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:29.416 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.416 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.416 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:37:29.416 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:29.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.416 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.416 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:37:29.416 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:29.416 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.416 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:37:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:37:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:37:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:37:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:37:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:37:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:29.417 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:37:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:29.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:37:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:37:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.420 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:37:29.420 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:29.420 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.420 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.420 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:29.420 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.420 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:37:29.420 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:37:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:37:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:29.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:37:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:37:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:37:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:37:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:37:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:37:29.422 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:37:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:29.423 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:29.424 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.424 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.424 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:37:29.424 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:29.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.424 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.424 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:37:29.424 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:29.424 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.424 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.424 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:29.424 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:37:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:37:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.426 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:37:29.426 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:29.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.426 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:37:29.426 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:29.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.426 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.426 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:29.426 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:37:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:37:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:37:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:37:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:37:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:37:29.427 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:37:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:29.428 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:29.429 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:29.429 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:29.429 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:29.429 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:29.429 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.429 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.429 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:37:29.429 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:29.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.429 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.429 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:37:29.429 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:29.430 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.430 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.430 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:29.430 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.430 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.430 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:37:29.430 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:29.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.430 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.430 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:37:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:37:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:37:29.431 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:37:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:37:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:29.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:37:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:37:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:37:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:37:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull", 2024-01-27T02:37:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:37:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:29.433 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:37:29.434 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:29.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.435 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:37:29.435 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:29.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.435 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.435 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:29.435 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.435 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:37:29.435 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:29.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:37:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:37:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:37:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:37:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.437 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:37:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:37:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:37:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:37:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:37:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.438 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:37:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.439 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:37:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:37:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:37:29.440 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:37:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:37:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:37:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:37:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.442 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:37:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:37:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:37:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:37:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:37:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-27T02:37:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:37:29.443 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:29.444 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:29.444 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:29.444 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:29.444 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:29.444 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:29.444 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:29.444 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:29.444 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:29.444 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:29.444 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:29.444 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:29.444 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.444 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:37:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:37:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.445 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:37:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:37:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:37:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:37:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:37:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:29.447 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:37:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:37:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:37:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:37:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:37:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull", 2024-01-27T02:37:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:37:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:37:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:37:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:37:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:29.449 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:37:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:37:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:37:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:29.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:37:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:37:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:37:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:29.451 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:37:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:37:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:37:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:29.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:37:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:37:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:37:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:29.453 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:37:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:37:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:37:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:37:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:37:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:37:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:37:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:37:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:37:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:37:29.454 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:37:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:37:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:37:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:37:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:37:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:37:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:37:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:37:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:37:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:37:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:37:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:37:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:37:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:37:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:37:29.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:29.456 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:37:29.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.390+0000 7fb74a453700 1 -- 172.21.15.7:0/2566579005 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb730066f30 msgr2=0x7fb7300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:29.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.390+0000 7fb74a453700 1 --2- 172.21.15.7:0/2566579005 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb730066f30 0x7fb7300693e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb7440cd380 tx=0x7fb734009040 comp rx=0 tx=0).stop 2024-01-27T02:37:29.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.390+0000 7fb74a453700 1 -- 172.21.15.7:0/2566579005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb744159b40 msgr2=0x7fb7440c8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:29.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.390+0000 7fb74a453700 1 --2- 172.21.15.7:0/2566579005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb744159b40 0x7fb7440c8ed0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00cc40 tx=0x7fb73c001970 comp rx=0 tx=0).stop 2024-01-27T02:37:29.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.391+0000 7fb74a453700 1 -- 172.21.15.7:0/2566579005 shutdown_connections 2024-01-27T02:37:29.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.391+0000 7fb74a453700 1 --2- 172.21.15.7:0/2566579005 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb730066f30 0x7fb7300693e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:29.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.391+0000 7fb74a453700 1 --2- 172.21.15.7:0/2566579005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb744159b40 0x7fb7440c8ed0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:29.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.391+0000 7fb74a453700 1 --2- 172.21.15.7:0/2566579005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7440d10a0 0x7fb7441561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:29.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.391+0000 7fb74a453700 1 --2- 172.21.15.7:0/2566579005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7440d1470 0x7fb74415b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:29.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.391+0000 7fb74a453700 1 -- 172.21.15.7:0/2566579005 >> 172.21.15.7:0/2566579005 conn(0x7fb7440679b0 msgr2=0x7fb7440cc880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:29.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.391+0000 7fb74a453700 1 -- 172.21.15.7:0/2566579005 shutdown_connections 2024-01-27T02:37:29.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:29.391+0000 7fb74a453700 1 -- 172.21.15.7:0/2566579005 wait complete. 2024-01-27T02:37:29.458 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-27T02:37:29.852 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-27T02:37:29.852+0000 7fb5c783f700 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2024-01-27T02:37:34.459 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:37:34.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.598+0000 7fd98329f700 1 Processor -- start 2024-01-27T02:37:34.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.598+0000 7fd98329f700 1 -- start start 2024-01-27T02:37:34.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.599+0000 7fd98329f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd97c157ca0 0x7fd97c0c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:34.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.599+0000 7fd98329f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd97c0cdbf0 0x7fd97c0ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.599+0000 7fd98329f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd97c0ca020 con 0x7fd97c0c3ae0 2024-01-27T02:37:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.599+0000 7fd98329f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd97c0c4f40 con 0x7fd97c157ca0 2024-01-27T02:37:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.599+0000 7fd98329f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0c5210 con 0x7fd97c0cdbf0 2024-01-27T02:37:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.599+0000 7fd98083a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd97c0c3ae0 0x7fd97c0cd4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59542/0 (socket says 172.21.15.7:59542) 2024-01-27T02:37:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.599+0000 7fd98083a700 1 -- 172.21.15.7:0/1573267219 learned_addr learned my addr 172.21.15.7:0/1573267219 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.599+0000 7fd98103b700 1 --2- 172.21.15.7:0/1573267219 >> v2:172.21.15.188:3300/0 conn(0x7fd97c157ca0 0x7fd97c0c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.599+0000 7fd98183c700 1 --2- 172.21.15.7:0/1573267219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd97c0cdbf0 0x7fd97c0ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.599+0000 7fd96ffff700 1 -- 172.21.15.7:0/1573267219 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3373211940 0 0) 0x7fd97c0ca020 con 0x7fd97c0c3ae0 2024-01-27T02:37:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd96ffff700 1 -- 172.21.15.7:0/1573267219 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd968003570 con 0x7fd97c0c3ae0 2024-01-27T02:37:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd96ffff700 1 -- 172.21.15.7:0/1573267219 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 921934116 0 0) 0x7fd968003570 con 0x7fd97c0c3ae0 2024-01-27T02:37:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd96ffff700 1 -- 172.21.15.7:0/1573267219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd97c0cdbf0 msgr2=0x7fd97c0ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd96ffff700 1 --2- 172.21.15.7:0/1573267219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd97c0cdbf0 0x7fd97c0ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd96ffff700 1 -- 172.21.15.7:0/1573267219 >> v2:172.21.15.188:3300/0 conn(0x7fd97c157ca0 msgr2=0x7fd97c0c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd96ffff700 1 --2- 172.21.15.7:0/1573267219 >> v2:172.21.15.188:3300/0 conn(0x7fd97c157ca0 0x7fd97c0c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd96ffff700 1 -- 172.21.15.7:0/1573267219 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97c0c54f0 con 0x7fd97c0c3ae0 2024-01-27T02:37:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd96ffff700 1 -- 172.21.15.7:0/1573267219 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7fd978001470 con 0x7fd97c0c3ae0 2024-01-27T02:37:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd98103b700 1 --2- 172.21.15.7:0/1573267219 >> v2:172.21.15.188:3300/0 conn(0x7fd97c157ca0 0x7fd97c0c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:37:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd96ffff700 1 -- 172.21.15.7:0/1573267219 >> v1:172.21.15.7:6789/0 conn(0x7fd97c0c3ae0 legacy=0x7fd97c0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd96ffff700 1 --2- 172.21.15.7:0/1573267219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd97c0cdbf0 0x7fd968003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd96ffff700 1 --2- 172.21.15.7:0/1573267219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd968003fe0 0x7fd968004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd96ffff700 1 --2- 172.21.15.7:0/1573267219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd968004d50 0x7fd9680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd96ffff700 1 -- 172.21.15.7:0/1573267219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0c4f40 con 0x7fd968003fe0 2024-01-27T02:37:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd96ffff700 1 -- 172.21.15.7:0/1573267219 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0c5210 con 0x7fd968004d50 2024-01-27T02:37:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd96ffff700 1 -- 172.21.15.7:0/1573267219 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd96800ba90 con 0x7fd97c0cdbf0 2024-01-27T02:37:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd98083a700 1 --2- 172.21.15.7:0/1573267219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd968003fe0 0x7fd968004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd98103b700 1 --2- 172.21.15.7:0/1573267219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd97c0cdbf0 0x7fd968003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd98183c700 1 --2- 172.21.15.7:0/1573267219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd968004d50 0x7fd9680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd98083a700 1 -- 172.21.15.7:0/1573267219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd97c0cdbf0 msgr2=0x7fd968003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd98083a700 1 --2- 172.21.15.7:0/1573267219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd97c0cdbf0 0x7fd968003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd98083a700 1 -- 172.21.15.7:0/1573267219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd968004d50 msgr2=0x7fd9680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd98083a700 1 --2- 172.21.15.7:0/1573267219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd968004d50 0x7fd9680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.600+0000 7fd98083a700 1 -- 172.21.15.7:0/1573267219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd968003570 con 0x7fd968003fe0 2024-01-27T02:37:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.601+0000 7fd98083a700 1 --2- 172.21.15.7:0/1573267219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd968003fe0 0x7fd968004810 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd97800c3c0 tx=0x7fd978001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693c419ac0feba59 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.601+0000 7fd96ffff700 1 -- 172.21.15.7:0/1573267219 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97800cc30 con 0x7fd968003fe0 2024-01-27T02:37:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.601+0000 7fd96ffff700 1 -- 172.21.15.7:0/1573267219 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97800cdb0 con 0x7fd968003fe0 2024-01-27T02:37:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.601+0000 7fd96ffff700 1 -- 172.21.15.7:0/1573267219 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd978005b70 con 0x7fd968003fe0 2024-01-27T02:37:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.601+0000 7fd98329f700 1 -- 172.21.15.7:0/1573267219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd968003fe0 msgr2=0x7fd968004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.601+0000 7fd98329f700 1 --2- 172.21.15.7:0/1573267219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd968003fe0 0x7fd968004810 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd97800c3c0 tx=0x7fd978001e90 comp rx=0 tx=0).stop 2024-01-27T02:37:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.601+0000 7fd98183c700 1 -- 172.21.15.7:0/1573267219 reap_dead start 2024-01-27T02:37:34.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.601+0000 7fd98329f700 1 -- 172.21.15.7:0/1573267219 shutdown_connections 2024-01-27T02:37:34.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.601+0000 7fd98329f700 1 -- 172.21.15.7:0/1573267219 >> 172.21.15.7:0/1573267219 conn(0x7fd97c098020 msgr2=0x7fd97c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:34.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.602+0000 7fd98329f700 1 -- 172.21.15.7:0/1573267219 shutdown_connections 2024-01-27T02:37:34.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.602+0000 7fd98329f700 1 -- 172.21.15.7:0/1573267219 wait complete. 2024-01-27T02:37:34.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.603+0000 7fd98329f700 1 Processor -- start 2024-01-27T02:37:34.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.603+0000 7fd98329f700 1 -- start start 2024-01-27T02:37:34.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.604+0000 7fd98329f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd968003fe0 0x7fd97c154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.604+0000 7fd98329f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd97c14db70 0x7fd97c14fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.604+0000 7fd98329f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd97c150120 0x7fd97c153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.604+0000 7fd98329f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0c6a50 con 0x7fd968003fe0 2024-01-27T02:37:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.604+0000 7fd98329f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c151c10 con 0x7fd97c14db70 2024-01-27T02:37:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.604+0000 7fd98329f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0c61b0 con 0x7fd97c150120 2024-01-27T02:37:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.604+0000 7fd98103b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd968003fe0 0x7fd97c154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.604+0000 7fd98103b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd968003fe0 0x7fd97c154000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35386/0 (socket says 172.21.15.7:35386) 2024-01-27T02:37:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.604+0000 7fd98103b700 1 -- 172.21.15.7:0/1239740314 learned_addr learned my addr 172.21.15.7:0/1239740314 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.604+0000 7fd98083a700 1 --2- 172.21.15.7:0/1239740314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd97c14db70 0x7fd97c14fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.604+0000 7fd98183c700 1 --2- 172.21.15.7:0/1239740314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd97c150120 0x7fd97c153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.604+0000 7fd98103b700 1 -- 172.21.15.7:0/1239740314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd97c150120 msgr2=0x7fd97c153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.604+0000 7fd98103b700 1 --2- 172.21.15.7:0/1239740314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd97c150120 0x7fd97c153230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:34.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.604+0000 7fd98103b700 1 -- 172.21.15.7:0/1239740314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd97c14db70 msgr2=0x7fd97c14fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.604+0000 7fd98103b700 1 --2- 172.21.15.7:0/1239740314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd97c14db70 0x7fd97c14fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.604+0000 7fd98103b700 1 -- 172.21.15.7:0/1239740314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97c0c6490 con 0x7fd968003fe0 2024-01-27T02:37:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.604+0000 7fd98083a700 1 --2- 172.21.15.7:0/1239740314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd97c14db70 0x7fd97c14fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.604+0000 7fd98103b700 1 --2- 172.21.15.7:0/1239740314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd968003fe0 0x7fd97c154000 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd970011da0 tx=0x7fd9700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.605+0000 7fd96d7fa700 1 -- 172.21.15.7:0/1239740314 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97000fbf0 con 0x7fd968003fe0 2024-01-27T02:37:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.605+0000 7fd96d7fa700 1 -- 172.21.15.7:0/1239740314 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97000fd50 con 0x7fd968003fe0 2024-01-27T02:37:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.605+0000 7fd96d7fa700 1 -- 172.21.15.7:0/1239740314 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd970014690 con 0x7fd968003fe0 2024-01-27T02:37:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.605+0000 7fd98329f700 1 -- 172.21.15.7:0/1239740314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97c0c6770 con 0x7fd968003fe0 2024-01-27T02:37:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.605+0000 7fd98329f700 1 -- 172.21.15.7:0/1239740314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97c0c5df0 con 0x7fd968003fe0 2024-01-27T02:37:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.606+0000 7fd98329f700 1 -- 172.21.15.7:0/1239740314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97c151ef0 con 0x7fd968003fe0 2024-01-27T02:37:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.606+0000 7fd96d7fa700 1 -- 172.21.15.7:0/1239740314 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd970014a80 con 0x7fd968003fe0 2024-01-27T02:37:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.607+0000 7fd96d7fa700 1 --2- 172.21.15.7:0/1239740314 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd960066f30 0x7fd9600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.607+0000 7fd96d7fa700 1 -- 172.21.15.7:0/1239740314 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd970018030 con 0x7fd968003fe0 2024-01-27T02:37:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.607+0000 7fd98083a700 1 --2- 172.21.15.7:0/1239740314 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd960066f30 0x7fd9600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:34.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.608+0000 7fd98083a700 1 --2- 172.21.15.7:0/1239740314 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd960066f30 0x7fd9600693e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0b2e50 tx=0x7fd978015000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:34.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.611+0000 7fd96d7fa700 1 -- 172.21.15.7:0/1239740314 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd970052410 con 0x7fd968003fe0 2024-01-27T02:37:34.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.814+0000 7fd98329f700 1 -- 172.21.15.7:0/1239740314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd97c04c710 con 0x7fd968003fe0 2024-01-27T02:37:34.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.815+0000 7fd96d7fa700 1 -- 172.21.15.7:0/1239740314 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v106) v1 ==== 81+0+32219 (secure 0 0 0) 0x7fd970022090 con 0x7fd968003fe0 2024-01-27T02:37:34.817 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:34.817 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:37:34.817 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 106, 2024-01-27T02:37:34.817 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:37:34.817 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:37:34.818 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:37:32.458337+0000", 2024-01-27T02:37:34.818 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:37:34.818 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:37:34.818 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:37:34.818 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:37:34.818 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:37:34.818 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:37:34.818 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:37:34.818 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:37:34.818 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:37:34.818 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:34.818 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:37:34.818 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:37:34.818 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:37:34.819 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:37:34.819 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:37:34.819 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 13, 2024-01-27T02:37:34.819 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:37:34.819 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:37:34.819 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:37:34.819 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:37:34.819 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:37:34.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.819 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:37:34.819 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:37:34.819 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:37:34.819 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:37:34.819 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:37:34.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:37:34.820 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:37:34.820 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:37:34.820 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:37:34.820 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:37:34.820 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:37:34.820 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:37:34.820 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:37:34.820 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:37:34.820 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:37:34.820 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:37:34.820 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:37:34.820 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:37:34.820 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:37:34.821 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:37:34.821 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:37:34.821 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:37:34.821 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:37:34.821 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:37:34.821 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:37:34.821 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:37:34.821 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:37:34.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.821 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "106", 2024-01-27T02:37:34.821 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:37:34.821 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:37:34.821 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:37:34.821 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:37:34.821 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:37:34.821 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:37:34.821 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:37:34.822 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:37:34.822 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:37:34.822 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:37:34.822 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:37:34.822 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:37:34.822 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:37:34.822 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:37:34.822 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:37:34.822 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:37:34.822 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:37:34.822 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:37:34.822 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:37:34.822 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:37:34.822 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:37:34.822 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:37:34.823 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:37:34.823 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:37:34.823 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:37:34.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:37:34.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.823 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:37:34.823 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:37:34.823 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:37:34.823 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:37:34.823 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:37:34.823 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:37:34.823 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:37:34.823 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:37:34.823 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:37:34.823 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:37:34.824 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:37:34.824 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:37:34.824 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:37:34.824 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:37:34.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.824 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:37:34.824 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:37:34.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.825 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 12, 2024-01-27T02:37:34.825 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:37:34.825 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:37:01.445734+0000", 2024-01-27T02:37:34.825 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:37:34.825 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:37:34.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:37:34.825 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:37:34.825 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:37:34.825 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:37:34.825 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:37:34.825 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:37:34.825 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:37:34.825 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:37:34.826 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:37:34.826 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:37:34.826 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:37:34.826 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:37:34.826 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:37:34.826 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:37:34.826 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:37:34.826 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:37:34.826 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:37:34.826 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:37:34.826 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:37:34.826 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:37:34.826 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:37:34.826 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:37:34.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.826 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "106", 2024-01-27T02:37:34.827 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:37:34.827 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:37:34.827 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:37:34.827 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:37:34.827 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:37:34.827 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:37:34.827 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:37:34.827 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:37:34.827 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:37:34.827 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:37:34.827 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:37:34.827 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:37:34.827 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:37:34.827 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:37:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:37:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:37:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:37:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:37:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:37:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:37:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:37:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:37:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:37:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:37:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:37:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:37:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:37:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:37:34.829 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:37:34.829 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:37:34.829 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:37:34.829 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:37:34.829 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:37:34.829 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:37:34.829 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:37:34.829 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:37:34.829 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:37:34.829 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:37:34.829 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:37:34.829 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:37:34.829 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:37:34.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.829 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:37:34.830 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:37:34.830 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:37:34.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.830 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 13, 2024-01-27T02:37:34.830 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:37:34.830 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:37:02.377151+0000", 2024-01-27T02:37:34.830 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:37:34.830 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:37:34.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:37:34.831 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:37:34.831 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:37:34.831 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:37:34.831 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:37:34.831 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:37:34.831 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:37:34.831 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:37:34.831 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:37:34.831 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:37:34.832 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:37:34.832 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:37:34.832 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:37:34.832 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:37:34.832 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:37:34.832 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:37:34.832 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:37:34.832 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:37:34.832 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:37:34.832 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:37:34.832 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:37:34.833 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:37:34.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.833 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "106", 2024-01-27T02:37:34.833 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:37:34.833 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:37:34.833 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:37:34.833 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:37:34.833 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:37:34.833 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:37:34.833 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:37:34.833 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:37:34.834 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:37:34.834 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:37:34.834 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:37:34.834 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:37:34.834 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:37:34.834 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:37:34.834 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:37:34.834 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:37:34.834 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:37:34.834 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:37:34.835 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:37:34.835 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:37:34.835 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:37:34.835 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:37:34.835 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:37:34.835 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:37:34.835 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:37:34.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:37:34.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.835 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:37:34.835 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:37:34.836 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:37:34.836 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:37:34.836 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:37:34.836 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:37:34.836 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:37:34.836 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:37:34.836 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:37:34.836 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:37:34.836 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:37:34.836 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:37:34.836 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:37:34.837 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:37:34.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.837 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:37:34.837 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:37:34.837 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:37:34.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.837 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:34.837 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:37:34.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.838 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:37:34.838 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:34.838 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:34.838 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:34.838 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:34.838 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:34.838 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:34.838 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:34.838 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:34.839 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:34.839 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:34.839 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:34.839 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:34.839 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.839 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.839 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:37:34.839 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:34.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.839 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.839 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:37:34.839 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:34.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.840 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.840 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:34.840 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.840 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.840 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:37:34.840 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:34.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.841 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.841 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:37:34.841 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:34.841 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.841 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.841 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:34.841 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.841 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.841 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:37:34.842 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:34.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.842 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.842 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:37:34.842 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:34.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.842 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.842 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:34.843 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.843 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.843 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:37:34.843 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:34.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.843 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.843 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:37:34.843 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:34.843 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.843 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.843 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:37:34.844 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:37:34.844 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:37:34.844 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:37:34.844 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:34.844 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:34.844 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:34.844 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.844 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:37:34.844 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:34.844 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:34.844 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:34.845 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:34.845 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:34.845 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:34.845 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:34.845 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:34.845 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:34.845 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:34.845 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:34.845 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:34.845 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.845 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.845 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:37:34.845 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:34.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.846 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.846 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:37:34.846 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:34.846 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.846 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.846 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:34.846 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.846 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.846 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:37:34.846 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:34.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.847 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:37:34.847 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:34.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.847 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.847 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:34.847 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.847 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:37:34.847 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:34.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.848 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:37:34.848 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:34.848 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.848 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.848 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:34.848 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.848 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:37:34.848 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:34.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.849 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.849 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:37:34.849 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:34.849 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.849 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.849 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:37:34.849 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:37:34.849 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:37:34.849 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:37:34.849 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:34.849 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:34.849 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:34.849 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.850 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:37:34.850 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:34.850 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:34.850 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:34.850 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:34.850 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:34.850 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:34.850 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:34.850 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:34.850 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:34.850 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:34.850 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:34.851 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:34.851 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.851 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.851 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:37:34.851 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:34.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.851 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.851 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:37:34.851 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:34.851 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.851 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.851 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:34.852 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.852 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.852 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:37:34.852 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:34.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.852 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.852 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:37:34.852 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:34.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.852 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.852 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:34.853 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.853 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.853 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:37:34.853 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:34.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.853 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.853 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:37:34.853 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:34.853 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.853 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.853 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:34.854 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.854 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.854 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:37:34.854 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:34.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.854 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.854 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:37:34.854 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:34.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.854 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.854 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:37:34.854 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:37:34.855 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:37:34.855 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:37:34.855 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:34.855 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:34.855 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:34.855 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.855 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:37:34.855 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:34.855 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:34.855 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:34.855 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:34.856 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:34.856 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:34.856 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:34.856 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:34.856 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:34.856 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:34.856 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:34.856 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:34.856 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.856 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:37:34.856 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:34.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.857 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:37:34.857 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:34.857 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.857 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.857 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:34.857 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.857 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:37:34.857 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:34.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.858 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.858 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:37:34.858 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:34.858 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.858 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.858 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:34.858 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.858 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.858 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:37:34.858 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:34.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.859 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:37:34.859 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:34.859 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.859 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.859 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:34.859 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.859 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:37:34.859 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:34.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.860 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:37:34.860 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:34.860 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.860 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.860 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:37:34.860 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:37:34.860 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:37:34.860 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:37:34.860 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:34.860 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:34.860 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:34.860 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.861 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:37:34.861 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:34.861 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:34.861 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:34.861 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:34.861 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:34.861 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:34.861 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:34.861 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:34.861 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:34.861 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:34.862 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:34.862 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:34.862 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.862 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:37:34.862 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:34.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.862 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:37:34.862 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:34.862 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.863 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.863 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:34.863 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.863 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.863 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:37:34.863 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:34.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.863 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.863 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:37:34.863 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:34.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.863 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.864 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:34.864 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.864 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.864 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:37:34.864 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:34.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:37:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:37:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:37:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:37:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:37:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:37:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:37:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:37:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:37:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:37:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:37:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:37:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:37:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:37:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:37:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:37:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:37:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:37:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:37:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:37:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:37:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:37:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:37:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:37:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:37:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:37:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:37:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:37:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:37:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:37:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:37:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:37:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:37:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:37:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.882 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:37:34.882 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:34.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.882 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.882 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:37:34.882 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:34.882 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.882 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.882 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:34.882 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.882 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.882 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:37:34.883 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:34.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.883 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.883 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:37:34.883 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:34.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.883 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.883 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:34.883 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.883 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.883 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:37:34.883 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:34.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.884 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:37:34.884 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:34.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.884 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.884 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:34.884 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:34.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:34.884 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:37:34.884 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:34.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.885 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:34.885 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:37:34.885 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:34.885 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.885 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.885 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:37:34.885 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:37:34.885 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:37:34.885 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:37:34.885 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:34.885 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:34.886 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:34.886 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:34.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.886 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:34.886 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:37:34.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.886 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:37:34.886 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:34.886 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:34.886 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:34.886 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:34.886 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:34.886 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:37:34.886 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:34.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.887 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:37:34.887 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:34.887 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:34.887 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:34.887 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:34.887 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:34.887 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:37:34.887 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:34.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.887 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:37:34.887 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:34.887 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:34.887 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:34.888 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:34.888 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:34.888 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:37:34.888 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:34.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.888 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:37:34.888 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:34.888 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:34.888 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:34.888 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:34.888 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:34.888 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:37:34.888 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:34.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.889 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:37:34.889 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:34.889 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:34.889 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:34.889 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:34.889 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:34.889 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:37:34.889 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:34.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.889 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:37:34.889 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:34.889 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:34.889 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:34.890 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:34.890 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:34.890 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:37:34.890 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:34.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.890 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:37:34.890 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:34.890 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:34.890 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:34.890 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:34.890 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:34.890 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:37:34.890 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:34.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:34.891 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:37:34.891 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:34.891 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:34.891 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:34.891 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:34.891 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:34.891 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:37:34.891 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:34.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.891 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:34.891 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:37:34.891 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:37:34.891 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:37:34.892 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:37:34.892 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:37:34.892 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:37:34.892 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:37:34.892 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:37:34.892 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:37:34.892 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:37:34.892 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:37:34.892 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:37:34.892 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:37:34.892 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:34.892 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:37:34.893 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:37:34.893 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:37:34.893 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:37:34.893 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:37:34.893 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:37:34.893 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:37:34.893 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:37:34.893 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:37:34.893 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:37:34.893 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:37:34.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:34.893 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:37:34.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.818+0000 7fd98329f700 1 -- 172.21.15.7:0/1239740314 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd960066f30 msgr2=0x7fd9600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:34.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.818+0000 7fd98329f700 1 --2- 172.21.15.7:0/1239740314 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd960066f30 0x7fd9600693e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0b2e50 tx=0x7fd978015000 comp rx=0 tx=0).stop 2024-01-27T02:37:34.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.818+0000 7fd98329f700 1 -- 172.21.15.7:0/1239740314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd968003fe0 msgr2=0x7fd97c154000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:34.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.818+0000 7fd98329f700 1 --2- 172.21.15.7:0/1239740314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd968003fe0 0x7fd97c154000 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd970011da0 tx=0x7fd9700091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:34.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.818+0000 7fd98329f700 1 -- 172.21.15.7:0/1239740314 shutdown_connections 2024-01-27T02:37:34.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.818+0000 7fd98329f700 1 --2- 172.21.15.7:0/1239740314 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd960066f30 0x7fd9600693e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:34.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.818+0000 7fd98329f700 1 --2- 172.21.15.7:0/1239740314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd97c150120 0x7fd97c153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:34.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.818+0000 7fd98329f700 1 --2- 172.21.15.7:0/1239740314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd968003fe0 0x7fd97c154000 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:34.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.818+0000 7fd98329f700 1 --2- 172.21.15.7:0/1239740314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd97c14db70 0x7fd97c14fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:34.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.818+0000 7fd98329f700 1 -- 172.21.15.7:0/1239740314 >> 172.21.15.7:0/1239740314 conn(0x7fd97c098020 msgr2=0x7fd97c06b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:34.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.818+0000 7fd98329f700 1 -- 172.21.15.7:0/1239740314 shutdown_connections 2024-01-27T02:37:34.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:34.818+0000 7fd98329f700 1 -- 172.21.15.7:0/1239740314 wait complete. 2024-01-27T02:37:34.896 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-01-27T02:37:34.896 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:37:35.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.031+0000 7f03d54ba700 1 Processor -- start 2024-01-27T02:37:35.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.031+0000 7f03d54ba700 1 -- start start 2024-01-27T02:37:35.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.032+0000 7f03d54ba700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f03d0143860 0x7f03d00d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:35.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.032+0000 7f03d54ba700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f03d0143c30 0x7f03d00d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:35.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.032+0000 7f03d54ba700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03d015de00 0x7f03d015e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:35.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.032+0000 7f03d54ba700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d015e7a0 con 0x7f03d0143c30 2024-01-27T02:37:35.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.032+0000 7f03d54ba700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d00b3d90 con 0x7f03d0143860 2024-01-27T02:37:35.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.032+0000 7f03d54ba700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d00d0ea0 con 0x7f03d015de00 2024-01-27T02:37:35.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.032+0000 7f03ce7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f03d0143c30 0x7f03d00d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:35.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.032+0000 7f03ce7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f03d0143c30 0x7f03d00d0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35396/0 (socket says 172.21.15.7:35396) 2024-01-27T02:37:35.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.032+0000 7f03ce7fc700 1 -- 172.21.15.7:0/946799158 learned_addr learned my addr 172.21.15.7:0/946799158 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:35.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.032+0000 7f03cf7fe700 1 --2- 172.21.15.7:0/946799158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03d015de00 0x7f03d015e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:35.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.032+0000 7f03ceffd700 1 --2- 172.21.15.7:0/946799158 >> v2:172.21.15.188:3300/0 conn(0x7f03d0143860 0x7f03d00d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:35.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.033+0000 7f03ce7fc700 1 -- 172.21.15.7:0/946799158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03d015de00 msgr2=0x7f03d015e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:35.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.033+0000 7f03ce7fc700 1 --2- 172.21.15.7:0/946799158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03d015de00 0x7f03d015e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:35.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.033+0000 7f03ce7fc700 1 -- 172.21.15.7:0/946799158 >> v2:172.21.15.188:3300/0 conn(0x7f03d0143860 msgr2=0x7f03d00d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:35.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.033+0000 7f03ce7fc700 1 --2- 172.21.15.7:0/946799158 >> v2:172.21.15.188:3300/0 conn(0x7f03d0143860 0x7f03d00d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:35.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.033+0000 7f03ce7fc700 1 -- 172.21.15.7:0/946799158 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d00d1180 con 0x7f03d0143c30 2024-01-27T02:37:35.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.033+0000 7f03ce7fc700 1 --2- 172.21.15.7:0/946799158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d0143c30 0x7f03d00d0820 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f03d00b3a80 tx=0x7f03b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36f5c9e7a993a46 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:35.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.033+0000 7f03cdffb700 1 -- 172.21.15.7:0/946799158 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b800b640 con 0x7f03d0143c30 2024-01-27T02:37:35.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.033+0000 7f03cdffb700 1 -- 172.21.15.7:0/946799158 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b800b7a0 con 0x7f03d0143c30 2024-01-27T02:37:35.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.033+0000 7f03cdffb700 1 -- 172.21.15.7:0/946799158 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b800bb60 con 0x7f03d0143c30 2024-01-27T02:37:35.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.034+0000 7f03d54ba700 1 -- 172.21.15.7:0/946799158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d0143c30 msgr2=0x7f03d00d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:35.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.034+0000 7f03d54ba700 1 --2- 172.21.15.7:0/946799158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d0143c30 0x7f03d00d0820 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f03d00b3a80 tx=0x7f03b8009250 comp rx=0 tx=0).stop 2024-01-27T02:37:35.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.034+0000 7f03d54ba700 1 -- 172.21.15.7:0/946799158 shutdown_connections 2024-01-27T02:37:35.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.034+0000 7f03d54ba700 1 --2- 172.21.15.7:0/946799158 >> v2:172.21.15.188:3300/0 conn(0x7f03d0143860 0x7f03d00d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:35.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.034+0000 7f03d54ba700 1 --2- 172.21.15.7:0/946799158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03d015de00 0x7f03d015e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:35.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.034+0000 7f03d54ba700 1 --2- 172.21.15.7:0/946799158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d0143c30 0x7f03d00d0820 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:35.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.034+0000 7f03d54ba700 1 -- 172.21.15.7:0/946799158 >> 172.21.15.7:0/946799158 conn(0x7f03d0068440 msgr2=0x7f03d00709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:35.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.034+0000 7f03d54ba700 1 -- 172.21.15.7:0/946799158 shutdown_connections 2024-01-27T02:37:35.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.034+0000 7f03d54ba700 1 -- 172.21.15.7:0/946799158 wait complete. 2024-01-27T02:37:35.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.036+0000 7f03d54ba700 1 Processor -- start 2024-01-27T02:37:35.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.036+0000 7f03d54ba700 1 -- start start 2024-01-27T02:37:35.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.037+0000 7f03d54ba700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03d0143860 0x7f03d016b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:35.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.037+0000 7f03d54ba700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03d0143c30 0x7f03d0139d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:35.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.037+0000 7f03d54ba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d015de00 0x7f03d0159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:35.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.037+0000 7f03d54ba700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d015c4f0 con 0x7f03d015de00 2024-01-27T02:37:35.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.037+0000 7f03d54ba700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d013a290 con 0x7f03d0143860 2024-01-27T02:37:35.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.037+0000 7f03d54ba700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d013a530 con 0x7f03d0143c30 2024-01-27T02:37:35.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.037+0000 7f03cf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d015de00 0x7f03d0159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:35.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.037+0000 7f03cf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d015de00 0x7f03d0159aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35408/0 (socket says 172.21.15.7:35408) 2024-01-27T02:37:35.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.037+0000 7f03cf7fe700 1 -- 172.21.15.7:0/646272835 learned_addr learned my addr 172.21.15.7:0/646272835 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:35.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.037+0000 7f03ceffd700 1 --2- 172.21.15.7:0/646272835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03d0143860 0x7f03d016b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:35.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.037+0000 7f03ce7fc700 1 --2- 172.21.15.7:0/646272835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03d0143c30 0x7f03d0139d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:35.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.037+0000 7f03cf7fe700 1 -- 172.21.15.7:0/646272835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03d0143c30 msgr2=0x7f03d0139d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:35.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.037+0000 7f03cf7fe700 1 --2- 172.21.15.7:0/646272835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03d0143c30 0x7f03d0139d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:35.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.037+0000 7f03cf7fe700 1 -- 172.21.15.7:0/646272835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03d0143860 msgr2=0x7f03d016b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:35.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.037+0000 7f03cf7fe700 1 --2- 172.21.15.7:0/646272835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03d0143860 0x7f03d016b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:35.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.037+0000 7f03cf7fe700 1 -- 172.21.15.7:0/646272835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b80094f0 con 0x7f03d015de00 2024-01-27T02:37:35.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.037+0000 7f03ceffd700 1 --2- 172.21.15.7:0/646272835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03d0143860 0x7f03d016b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:37:35.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.037+0000 7f03cf7fe700 1 --2- 172.21.15.7:0/646272835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d015de00 0x7f03d0159aa0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009df0 tx=0x7f03c800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:35.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.037+0000 7f03c6ffd700 1 -- 172.21.15.7:0/646272835 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c800cb50 con 0x7f03d015de00 2024-01-27T02:37:35.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.038+0000 7f03c6ffd700 1 -- 172.21.15.7:0/646272835 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c800f040 con 0x7f03d015de00 2024-01-27T02:37:35.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.038+0000 7f03d54ba700 1 -- 172.21.15.7:0/646272835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d015bbc0 con 0x7f03d015de00 2024-01-27T02:37:35.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.038+0000 7f03c6ffd700 1 -- 172.21.15.7:0/646272835 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8014480 con 0x7f03d015de00 2024-01-27T02:37:35.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.038+0000 7f03d54ba700 1 -- 172.21.15.7:0/646272835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d015c180 con 0x7f03d015de00 2024-01-27T02:37:35.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.039+0000 7f03d54ba700 1 -- 172.21.15.7:0/646272835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d015c660 con 0x7f03d015de00 2024-01-27T02:37:35.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.039+0000 7f03c6ffd700 1 -- 172.21.15.7:0/646272835 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f03c8014830 con 0x7f03d015de00 2024-01-27T02:37:35.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.040+0000 7f03c6ffd700 1 --2- 172.21.15.7:0/646272835 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f03b0066f30 0x7f03b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:35.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.040+0000 7f03c6ffd700 1 -- 172.21.15.7:0/646272835 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f03c8084eb0 con 0x7f03d015de00 2024-01-27T02:37:35.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.040+0000 7f03ceffd700 1 --2- 172.21.15.7:0/646272835 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f03b0066f30 0x7f03b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:35.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.040+0000 7f03ceffd700 1 --2- 172.21.15.7:0/646272835 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f03b0066f30 0x7f03b00693e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f03d00b4000 tx=0x7f03c0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:35.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.044+0000 7f03c6ffd700 1 -- 172.21.15.7:0/646272835 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f03c8052120 con 0x7f03d015de00 2024-01-27T02:37:35.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.248+0000 7f03d54ba700 1 -- 172.21.15.7:0/646272835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f03d004c710 con 0x7f03d015de00 2024-01-27T02:37:35.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.249+0000 7f03c6ffd700 1 -- 172.21.15.7:0/646272835 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v106) v1 ==== 81+0+32219 (secure 0 0 0) 0x7f03c8055740 con 0x7f03d015de00 2024-01-27T02:37:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:35.251 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:37:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 106, 2024-01-27T02:37:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:37:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:37:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:37:32.458337+0000", 2024-01-27T02:37:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:37:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:37:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:37:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:37:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:37:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:37:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:37:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:37:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:37:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:37:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:37:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:37:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:37:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:37:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 13, 2024-01-27T02:37:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:37:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:37:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:37:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:37:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:37:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:37:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:37:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:37:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:37:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:37:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:37:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:37:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:37:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:37:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:37:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:37:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:37:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:37:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:37:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:37:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:37:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:37:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:37:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:37:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:37:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:37:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:37:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:37:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:37:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:37:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:37:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:37:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "106", 2024-01-27T02:37:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:37:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:37:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:37:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:37:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:37:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:37:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:37:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:37:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:37:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:37:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:37:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:37:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:37:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:37:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:37:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:37:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:37:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:37:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:37:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:37:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:37:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:37:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:37:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:37:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:37:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:37:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:37:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:37:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:37:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:37:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:37:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:37:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:37:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:37:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:37:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:37:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:37:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:37:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:37:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:37:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:37:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:37:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 12, 2024-01-27T02:37:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:37:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:37:01.445734+0000", 2024-01-27T02:37:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:37:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:37:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:37:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:37:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:37:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:37:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:37:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:37:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:37:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:37:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:37:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:37:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:37:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:37:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:37:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:37:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:37:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:37:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:37:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:37:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:37:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:37:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:37:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:37:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "106", 2024-01-27T02:37:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:37:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:37:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:37:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:37:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:37:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:37:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:37:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:37:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:37:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:37:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:37:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:37:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:37:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:37:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:37:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:37:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:37:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:37:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:37:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:37:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:37:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:37:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:37:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:37:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:37:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:37:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:37:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:37:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:37:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:37:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:37:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:37:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:37:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:37:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:37:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:37:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:37:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:37:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:37:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:37:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:37:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:37:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:37:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:37:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 13, 2024-01-27T02:37:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:37:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:37:02.377151+0000", 2024-01-27T02:37:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:37:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:37:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:37:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:37:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:37:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:37:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:37:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:37:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:37:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:37:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:37:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:37:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:37:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:37:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:37:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:37:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:37:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:37:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:37:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:37:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:37:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:37:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:37:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:37:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "106", 2024-01-27T02:37:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:37:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:37:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:37:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:37:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:37:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:37:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:37:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:37:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:37:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:37:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:37:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:37:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:37:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:37:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:37:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:37:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:37:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:37:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:37:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:37:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:37:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:37:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:37:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:37:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:37:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:37:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:37:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:37:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:37:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:37:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:37:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:37:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:37:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:37:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:37:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:37:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:37:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:37:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:37:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:37:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:37:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:37:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:37:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:37:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:37:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:37:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:37:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:37:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:37:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:37:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:37:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:37:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:37:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:37:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:37:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:37:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:37:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:37:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:37:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:37:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:37:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:37:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:37:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:37:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:37:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:37:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:37:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:37:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:37:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:37:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:37:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:37:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:37:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:37:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:37:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:37:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:37:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:37:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:37:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:37:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:37:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:37:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:37:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:37:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:37:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:37:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:37:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:37:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:37:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:37:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:37:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:37:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:37:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:37:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:37:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:37:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:37:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:37:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:37:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:37:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:37:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:37:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:37:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:37:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:37:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:37:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:37:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:37:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:37:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:37:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:37:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:37:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:37:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:37:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:37:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:37:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:37:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:37:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:37:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:37:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:37:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:37:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:37:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:37:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:37:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:37:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:37:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:37:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:37:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:37:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:37:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:37:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:37:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:37:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:37:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:37:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:37:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:37:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:37:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:37:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:37:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:37:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:37:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:37:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:37:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:37:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:37:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:37:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:37:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:37:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:37:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 96, 2024-01-27T02:37:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:37:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:37:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:37:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:37:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:37:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:37:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:37:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:37:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:37:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:37:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:37:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:37:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:37:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:37:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:37:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:37:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:37:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:37:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:37:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:37:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:37:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:37:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:37:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:37:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:37:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:37:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:37:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:37:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:37:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:37:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:37:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:37:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:37:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:37:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:37:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:37:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:37:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:37:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:37:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:37:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:37:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:37:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:37:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:37:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:37:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:37:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:37:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:37:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:37:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:37:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:37:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:37:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:37:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:37:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:37:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:37:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:37:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:37:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:37:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:37:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:37:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:37:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:37:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:37:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:37:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:37:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:37:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:37:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:37:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:37:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:37:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:37:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:37:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:35.322 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:37:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.252+0000 7f03d54ba700 1 -- 172.21.15.7:0/646272835 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f03b0066f30 msgr2=0x7f03b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.252+0000 7f03d54ba700 1 --2- 172.21.15.7:0/646272835 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f03b0066f30 0x7f03b00693e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f03d00b4000 tx=0x7f03c0008040 comp rx=0 tx=0).stop 2024-01-27T02:37:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.252+0000 7f03d54ba700 1 -- 172.21.15.7:0/646272835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d015de00 msgr2=0x7f03d0159aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.252+0000 7f03d54ba700 1 --2- 172.21.15.7:0/646272835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d015de00 0x7f03d0159aa0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009df0 tx=0x7f03c800b040 comp rx=0 tx=0).stop 2024-01-27T02:37:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.252+0000 7f03d54ba700 1 -- 172.21.15.7:0/646272835 shutdown_connections 2024-01-27T02:37:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.252+0000 7f03d54ba700 1 --2- 172.21.15.7:0/646272835 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f03b0066f30 0x7f03b00693e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.252+0000 7f03d54ba700 1 --2- 172.21.15.7:0/646272835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d015de00 0x7f03d0159aa0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.252+0000 7f03d54ba700 1 --2- 172.21.15.7:0/646272835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03d0143860 0x7f03d016b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.252+0000 7f03d54ba700 1 --2- 172.21.15.7:0/646272835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03d0143c30 0x7f03d0139d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.252+0000 7f03d54ba700 1 -- 172.21.15.7:0/646272835 >> 172.21.15.7:0/646272835 conn(0x7f03d0068440 msgr2=0x7f03d01408b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.252+0000 7f03d54ba700 1 -- 172.21.15.7:0/646272835 shutdown_connections 2024-01-27T02:37:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.252+0000 7f03d54ba700 1 -- 172.21.15.7:0/646272835 wait complete. 2024-01-27T02:37:35.326 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:37:35.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.458+0000 7f763ca15700 1 Processor -- start 2024-01-27T02:37:35.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.458+0000 7f763ca15700 1 -- start start 2024-01-27T02:37:35.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.459+0000 7f763ca15700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7638157ca0 0x7f76380c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:35.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.459+0000 7f763ca15700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f76380cdbf0 0x7f76380ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:35.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.460+0000 7f763ca15700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76380ca020 con 0x7f76380c3ae0 2024-01-27T02:37:35.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.460+0000 7f763ca15700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76380c4f40 con 0x7f76380cdbf0 2024-01-27T02:37:35.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.460+0000 7f763ca15700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76380c5210 con 0x7f7638157ca0 2024-01-27T02:37:35.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.460+0000 7f7636d9d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f76380cdbf0 0x7f76380ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:35.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.460+0000 7f763659c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7638157ca0 0x7f76380c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:35.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.460+0000 7f7635d9b700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f76380c3ae0 0x7f76380cd4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:58448/0 (socket says 172.21.15.7:58448) 2024-01-27T02:37:35.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.460+0000 7f763659c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7638157ca0 0x7f76380c4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:38308/0 (socket says 172.21.15.7:38308) 2024-01-27T02:37:35.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.460+0000 7f7635d9b700 1 -- 172.21.15.7:0/2667898847 learned_addr learned my addr 172.21.15.7:0/2667898847 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:35.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.460+0000 7f7636d9d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f76380cdbf0 0x7f76380ca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:33684/0 (socket says 172.21.15.7:33684) 2024-01-27T02:37:35.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.460+0000 7f7636d9d700 1 -- 172.21.15.7:0/2667898847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7638157ca0 msgr2=0x7f76380c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:35.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.460+0000 7f7636d9d700 1 --2- 172.21.15.7:0/2667898847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7638157ca0 0x7f76380c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:35.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.460+0000 7f7636d9d700 1 -- 172.21.15.7:0/2667898847 >> v1:172.21.15.188:6789/0 conn(0x7f76380c3ae0 legacy=0x7f76380cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:35.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.460+0000 7f763559a700 1 -- 172.21.15.7:0/2667898847 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1457428414 0 0) 0x7f76380ca020 con 0x7f76380c3ae0 2024-01-27T02:37:35.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.460+0000 7f7636d9d700 1 -- 172.21.15.7:0/2667898847 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76380c54f0 con 0x7f76380cdbf0 2024-01-27T02:37:35.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.460+0000 7f7636d9d700 1 --2- 172.21.15.7:0/2667898847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76380cdbf0 0x7f76380ca4b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7630009e60 tx=0x7f7630009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e218912f6d68bea3 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:35.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.461+0000 7f763559a700 1 -- 172.21.15.7:0/2667898847 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763000e070 con 0x7f76380cdbf0 2024-01-27T02:37:35.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.461+0000 7f763559a700 1 -- 172.21.15.7:0/2667898847 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763000bb60 con 0x7f76380cdbf0 2024-01-27T02:37:35.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.461+0000 7f763559a700 1 -- 172.21.15.7:0/2667898847 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7630012430 con 0x7f76380cdbf0 2024-01-27T02:37:35.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.461+0000 7f763ca15700 1 -- 172.21.15.7:0/2667898847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76380cdbf0 msgr2=0x7f76380ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:35.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.461+0000 7f763ca15700 1 --2- 172.21.15.7:0/2667898847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76380cdbf0 0x7f76380ca4b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7630009e60 tx=0x7f7630009780 comp rx=0 tx=0).stop 2024-01-27T02:37:35.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.461+0000 7f763ca15700 1 -- 172.21.15.7:0/2667898847 shutdown_connections 2024-01-27T02:37:35.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.461+0000 7f763ca15700 1 --2- 172.21.15.7:0/2667898847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76380cdbf0 0x7f76380ca4b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:35.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.461+0000 7f763ca15700 1 --2- 172.21.15.7:0/2667898847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7638157ca0 0x7f76380c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:35.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.461+0000 7f763ca15700 1 -- 172.21.15.7:0/2667898847 >> 172.21.15.7:0/2667898847 conn(0x7f7638098020 msgr2=0x7f76380711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:35.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.461+0000 7f763ca15700 1 -- 172.21.15.7:0/2667898847 shutdown_connections 2024-01-27T02:37:35.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.461+0000 7f763ca15700 1 -- 172.21.15.7:0/2667898847 wait complete. 2024-01-27T02:37:35.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.463+0000 7f763ca15700 1 Processor -- start 2024-01-27T02:37:35.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.463+0000 7f763ca15700 1 -- start start 2024-01-27T02:37:35.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.463+0000 7f763ca15700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76380c3ae0 0x7f763814e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:35.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.463+0000 7f763ca15700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76380cdbf0 0x7f7638069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:35.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.463+0000 7f763ca15700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7638157ca0 0x7f7638067e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:35.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.463+0000 7f763ca15700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76380b16a0 con 0x7f76380c3ae0 2024-01-27T02:37:35.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.463+0000 7f763ca15700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7638069100 con 0x7f7638157ca0 2024-01-27T02:37:35.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.463+0000 7f763ca15700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76380684f0 con 0x7f76380cdbf0 2024-01-27T02:37:35.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.463+0000 7f763659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76380c3ae0 0x7f763814e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:35.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.464+0000 7f763659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76380c3ae0 0x7f763814e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35414/0 (socket says 172.21.15.7:35414) 2024-01-27T02:37:35.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.464+0000 7f763659c700 1 -- 172.21.15.7:0/2824240438 learned_addr learned my addr 172.21.15.7:0/2824240438 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:35.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.464+0000 7f7636d9d700 1 --2- 172.21.15.7:0/2824240438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7638157ca0 0x7f7638067e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:35.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.464+0000 7f7635d9b700 1 --2- 172.21.15.7:0/2824240438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76380cdbf0 0x7f7638069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:35.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.464+0000 7f763659c700 1 -- 172.21.15.7:0/2824240438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76380cdbf0 msgr2=0x7f7638069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:35.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.464+0000 7f763659c700 1 --2- 172.21.15.7:0/2824240438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76380cdbf0 0x7f7638069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:35.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.464+0000 7f763659c700 1 -- 172.21.15.7:0/2824240438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7638157ca0 msgr2=0x7f7638067e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:35.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.464+0000 7f763659c700 1 --2- 172.21.15.7:0/2824240438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7638157ca0 0x7f7638067e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:35.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.464+0000 7f763659c700 1 -- 172.21.15.7:0/2824240438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7630009a20 con 0x7f76380c3ae0 2024-01-27T02:37:35.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.464+0000 7f763659c700 1 --2- 172.21.15.7:0/2824240438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76380c3ae0 0x7f763814e0d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f762800e930 tx=0x7f76280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:35.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.464+0000 7f76267fc700 1 -- 172.21.15.7:0/2824240438 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628013680 con 0x7f76380c3ae0 2024-01-27T02:37:35.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.464+0000 7f76267fc700 1 -- 172.21.15.7:0/2824240438 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76280137e0 con 0x7f76380c3ae0 2024-01-27T02:37:35.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.464+0000 7f76267fc700 1 -- 172.21.15.7:0/2824240438 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628013b80 con 0x7f76380c3ae0 2024-01-27T02:37:35.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.464+0000 7f763ca15700 1 -- 172.21.15.7:0/2824240438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76380687d0 con 0x7f76380c3ae0 2024-01-27T02:37:35.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.464+0000 7f763ca15700 1 -- 172.21.15.7:0/2824240438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7638068d90 con 0x7f76380c3ae0 2024-01-27T02:37:35.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.465+0000 7f763ca15700 1 -- 172.21.15.7:0/2824240438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7638068f50 con 0x7f76380c3ae0 2024-01-27T02:37:35.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.466+0000 7f76267fc700 1 -- 172.21.15.7:0/2824240438 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f762800c3e0 con 0x7f76380c3ae0 2024-01-27T02:37:35.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.466+0000 7f76267fc700 1 --2- 172.21.15.7:0/2824240438 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7618066f30 0x7f76180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:35.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.467+0000 7f76267fc700 1 -- 172.21.15.7:0/2824240438 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f76280845b0 con 0x7f76380c3ae0 2024-01-27T02:37:35.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.467+0000 7f7635d9b700 1 --2- 172.21.15.7:0/2824240438 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7618066f30 0x7f76180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:35.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.467+0000 7f7635d9b700 1 --2- 172.21.15.7:0/2824240438 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7618066f30 0x7f76180693e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f76380b0e50 tx=0x7f762c007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.471+0000 7f76267fc700 1 -- 172.21.15.7:0/2824240438 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7628051820 con 0x7f76380c3ae0 2024-01-27T02:37:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.723+0000 7f763ca15700 1 -- 172.21.15.7:0/2824240438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f763804c710 con 0x7f76380c3ae0 2024-01-27T02:37:35.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.723+0000 7f76267fc700 1 -- 172.21.15.7:0/2824240438 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4584 (secure 0 0 0) 0x7f7628054e40 con 0x7f76380c3ae0 2024-01-27T02:37:35.725 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:35.725 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/450556982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":450556982},{"type":"v1","addr":"172.21.15.188:6836","nonce":450556982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1711422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1711422},{"type":"v1","addr":"172.21.15.188:6837","nonce":1711422}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6294,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1110306600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1110306600},{"type":"v1","addr":"172.21.15.7:6833","nonce":1110306600}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:37:03.404355+0000","modified":"2024-01-27T02:37:06.988798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6297":{"gid":6297,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/328992853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":328992853},{"type":"v1","addr":"172.21.15.7:6835","nonce":328992853}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-27T02:37:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.726+0000 7f763ca15700 1 -- 172.21.15.7:0/2824240438 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7618066f30 msgr2=0x7f76180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.726+0000 7f763ca15700 1 --2- 172.21.15.7:0/2824240438 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7618066f30 0x7f76180693e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f76380b0e50 tx=0x7f762c007d30 comp rx=0 tx=0).stop 2024-01-27T02:37:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.726+0000 7f763ca15700 1 -- 172.21.15.7:0/2824240438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76380c3ae0 msgr2=0x7f763814e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.726+0000 7f763ca15700 1 --2- 172.21.15.7:0/2824240438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76380c3ae0 0x7f763814e0d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f762800e930 tx=0x7f76280091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.726+0000 7f763ca15700 1 -- 172.21.15.7:0/2824240438 shutdown_connections 2024-01-27T02:37:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.726+0000 7f763ca15700 1 --2- 172.21.15.7:0/2824240438 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7618066f30 0x7f76180693e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.726+0000 7f763ca15700 1 --2- 172.21.15.7:0/2824240438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7638157ca0 0x7f7638067e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.726+0000 7f763ca15700 1 --2- 172.21.15.7:0/2824240438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76380c3ae0 0x7f763814e0d0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.726+0000 7f763ca15700 1 --2- 172.21.15.7:0/2824240438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76380cdbf0 0x7f7638069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.726+0000 7f763ca15700 1 -- 172.21.15.7:0/2824240438 >> 172.21.15.7:0/2824240438 conn(0x7f7638098020 msgr2=0x7f7638151500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.726+0000 7f763ca15700 1 -- 172.21.15.7:0/2824240438 shutdown_connections 2024-01-27T02:37:35.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:35.726+0000 7f763ca15700 1 -- 172.21.15.7:0/2824240438 wait complete. 2024-01-27T02:37:35.729 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 49 2024-01-27T02:37:35.741 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-01-27T02:37:35.893 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","whoami":0,"id":6297,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":31.901768197999999,"mdsmap_epoch":49,"osdmap_epoch":106,"osdmap_epoch_barrier":102,"uptime":37.745249776000001} 2024-01-27T02:37:35.894 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", "fs_name": "cephfs", "id": 6297, "mdsmap_epoch": 49, "osdmap_epoch": 106, "osdmap_epoch_barrier": 102, "rank_uptime": 31.901768198, "state": "up:active", "uptime": 37.745249776, "want_state": "up:active", "whoami": 0 } 2024-01-27T02:37:35.894 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-01-27T02:37:35.894 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:37:35.894 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-01-27T02:37:36.213 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records in 2024-01-27T02:37:36.214 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records out 2024-01-27T02:37:36.214 INFO:teuthology.orchestra.run.smithi007.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.299632 s, 175 MB/s 2024-01-27T02:37:36.215 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-27T02:37:36.215 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:37:36.261 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:37:36.298 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-27T02:37:36.298 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-27T02:37:36.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:37:36.299 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-27T02:37:36.384 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:37:36.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:37:36.384 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-27T02:37:36.405 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-27T02:37:36.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:37:36.405 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-01-27T02:37:36.445 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:37:36.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:37:36.445 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-27T02:37:36.503 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:37:36.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.643+0000 7f38f3635700 1 Processor -- start 2024-01-27T02:37:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.644+0000 7f38f3635700 1 -- start start 2024-01-27T02:37:36.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.645+0000 7f38f3635700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f38ec151570 0x7f38ec157f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:36.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.645+0000 7f38f3635700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f38ec06b400 0x7f38ec06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:36.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.645+0000 7f38f3635700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38ec151940 con 0x7f38ec06e870 2024-01-27T02:37:36.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.645+0000 7f38f3635700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ec13a1a0 con 0x7f38ec06b400 2024-01-27T02:37:36.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.645+0000 7f38f3635700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ec158490 con 0x7f38ec151570 2024-01-27T02:37:36.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.645+0000 7f38f0bd0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f38ec06b400 0x7f38ec06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:36.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.645+0000 7f38f0bd0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f38ec06b400 0x7f38ec06e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35428/0 (socket says 172.21.15.7:35428) 2024-01-27T02:37:36.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.645+0000 7f38f0bd0700 1 -- 172.21.15.7:0/3769600726 learned_addr learned my addr 172.21.15.7:0/3769600726 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:36.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.645+0000 7f38f13d1700 1 --2- 172.21.15.7:0/3769600726 >> v2:172.21.15.188:3300/0 conn(0x7f38ec151570 0x7f38ec157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:36.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.645+0000 7f38f0bd0700 1 -- 172.21.15.7:0/3769600726 >> v2:172.21.15.188:3300/0 conn(0x7f38ec151570 msgr2=0x7f38ec157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:36.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.645+0000 7f38f0bd0700 1 --2- 172.21.15.7:0/3769600726 >> v2:172.21.15.188:3300/0 conn(0x7f38ec151570 0x7f38ec157f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:36.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.645+0000 7f38f0bd0700 1 -- 172.21.15.7:0/3769600726 >> v1:172.21.15.188:6789/0 conn(0x7f38ec06e870 legacy=0x7f38ec0c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:36.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.645+0000 7f38f0bd0700 1 -- 172.21.15.7:0/3769600726 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec06ecd0 con 0x7f38ec06b400 2024-01-27T02:37:36.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.645+0000 7f38f0bd0700 1 --2- 172.21.15.7:0/3769600726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38ec06b400 0x7f38ec06e330 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f38ec06dc90 tx=0x7f38dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b37425904ee0c7a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.646+0000 7f38e3fff700 1 -- 172.21.15.7:0/3769600726 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc009e70 con 0x7f38ec06b400 2024-01-27T02:37:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.646+0000 7f38e3fff700 1 -- 172.21.15.7:0/3769600726 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc00b860 con 0x7f38ec06b400 2024-01-27T02:37:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.646+0000 7f38e3fff700 1 -- 172.21.15.7:0/3769600726 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc00bc00 con 0x7f38ec06b400 2024-01-27T02:37:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.646+0000 7f38f3635700 1 -- 172.21.15.7:0/3769600726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38ec06b400 msgr2=0x7f38ec06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.646+0000 7f38f3635700 1 --2- 172.21.15.7:0/3769600726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38ec06b400 0x7f38ec06e330 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f38ec06dc90 tx=0x7f38dc009250 comp rx=0 tx=0).stop 2024-01-27T02:37:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.646+0000 7f38f3635700 1 -- 172.21.15.7:0/3769600726 shutdown_connections 2024-01-27T02:37:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.646+0000 7f38f3635700 1 --2- 172.21.15.7:0/3769600726 >> v2:172.21.15.188:3300/0 conn(0x7f38ec151570 0x7f38ec157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.646+0000 7f38f3635700 1 --2- 172.21.15.7:0/3769600726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38ec06b400 0x7f38ec06e330 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.646+0000 7f38f3635700 1 -- 172.21.15.7:0/3769600726 >> 172.21.15.7:0/3769600726 conn(0x7f38ec0cdc30 msgr2=0x7f38ec0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.646+0000 7f38f3635700 1 -- 172.21.15.7:0/3769600726 shutdown_connections 2024-01-27T02:37:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.646+0000 7f38f3635700 1 -- 172.21.15.7:0/3769600726 wait complete. 2024-01-27T02:37:36.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.648+0000 7f38f3635700 1 Processor -- start 2024-01-27T02:37:36.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.648+0000 7f38f3635700 1 -- start start 2024-01-27T02:37:36.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.649+0000 7f38f3635700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38ec06b400 0x7f38ec148fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:36.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.649+0000 7f38f3635700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38ec06e870 0x7f38ec16a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:36.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.649+0000 7f38f3635700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38ec151570 0x7f38ec16a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:36.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.649+0000 7f38f3635700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0c3670 con 0x7f38ec151570 2024-01-27T02:37:36.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.649+0000 7f38f3635700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec148b50 con 0x7f38ec06e870 2024-01-27T02:37:36.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.649+0000 7f38f3635700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0b31e0 con 0x7f38ec06b400 2024-01-27T02:37:36.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.649+0000 7f38f1bd2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38ec151570 0x7f38ec16a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:36.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.649+0000 7f38f1bd2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38ec151570 0x7f38ec16a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35432/0 (socket says 172.21.15.7:35432) 2024-01-27T02:37:36.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.649+0000 7f38f1bd2700 1 -- 172.21.15.7:0/2299994917 learned_addr learned my addr 172.21.15.7:0/2299994917 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:36.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.649+0000 7f38f13d1700 1 --2- 172.21.15.7:0/2299994917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38ec06b400 0x7f38ec148fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:36.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.649+0000 7f38f0bd0700 1 --2- 172.21.15.7:0/2299994917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38ec06e870 0x7f38ec16a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:36.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.649+0000 7f38f1bd2700 1 -- 172.21.15.7:0/2299994917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38ec06b400 msgr2=0x7f38ec148fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:36.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.649+0000 7f38f1bd2700 1 --2- 172.21.15.7:0/2299994917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38ec06b400 0x7f38ec148fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:36.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.649+0000 7f38f1bd2700 1 -- 172.21.15.7:0/2299994917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38ec06e870 msgr2=0x7f38ec16a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:36.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.649+0000 7f38f1bd2700 1 --2- 172.21.15.7:0/2299994917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38ec06e870 0x7f38ec16a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:36.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.649+0000 7f38f1bd2700 1 -- 172.21.15.7:0/2299994917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc0094f0 con 0x7f38ec151570 2024-01-27T02:37:36.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.649+0000 7f38f13d1700 1 --2- 172.21.15.7:0/2299994917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38ec06b400 0x7f38ec148fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:36.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.649+0000 7f38f1bd2700 1 --2- 172.21.15.7:0/2299994917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38ec151570 0x7f38ec16a6e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f38e400c8e0 tx=0x7f38e4001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:36.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.649+0000 7f38e17fa700 1 -- 172.21.15.7:0/2299994917 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e400ce80 con 0x7f38ec151570 2024-01-27T02:37:36.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.650+0000 7f38e17fa700 1 -- 172.21.15.7:0/2299994917 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e400a7e0 con 0x7f38ec151570 2024-01-27T02:37:36.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.650+0000 7f38f3635700 1 -- 172.21.15.7:0/2299994917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ec148500 con 0x7f38ec151570 2024-01-27T02:37:36.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.650+0000 7f38e17fa700 1 -- 172.21.15.7:0/2299994917 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e400ab80 con 0x7f38ec151570 2024-01-27T02:37:36.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.650+0000 7f38f3635700 1 -- 172.21.15.7:0/2299994917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ec1495a0 con 0x7f38ec151570 2024-01-27T02:37:36.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.651+0000 7f38f3635700 1 -- 172.21.15.7:0/2299994917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ec148cc0 con 0x7f38ec151570 2024-01-27T02:37:36.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.651+0000 7f38e17fa700 1 -- 172.21.15.7:0/2299994917 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f38e4012420 con 0x7f38ec151570 2024-01-27T02:37:36.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.652+0000 7f38e17fa700 1 --2- 172.21.15.7:0/2299994917 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f38d0066f30 0x7f38d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:36.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.652+0000 7f38e17fa700 1 -- 172.21.15.7:0/2299994917 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f38e40848c0 con 0x7f38ec151570 2024-01-27T02:37:36.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.652+0000 7f38f13d1700 1 --2- 172.21.15.7:0/2299994917 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f38d0066f30 0x7f38d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:36.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.652+0000 7f38f13d1700 1 --2- 172.21.15.7:0/2299994917 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f38d0066f30 0x7f38d00693e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f38e8006f00 tx=0x7f38e8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:36.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.656+0000 7f38e17fa700 1 -- 172.21.15.7:0/2299994917 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38e4051a90 con 0x7f38ec151570 2024-01-27T02:37:36.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.907+0000 7f38f3635700 1 -- 172.21.15.7:0/2299994917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38ec1482f0 con 0x7f38ec151570 2024-01-27T02:37:36.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.907+0000 7f38e17fa700 1 -- 172.21.15.7:0/2299994917 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4584 (secure 0 0 0) 0x7f38e40550b0 con 0x7f38ec151570 2024-01-27T02:37:36.909 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:36.909 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/450556982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":450556982},{"type":"v1","addr":"172.21.15.188:6836","nonce":450556982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1711422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1711422},{"type":"v1","addr":"172.21.15.188:6837","nonce":1711422}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6294,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1110306600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1110306600},{"type":"v1","addr":"172.21.15.7:6833","nonce":1110306600}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:37:03.404355+0000","modified":"2024-01-27T02:37:06.988798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6297":{"gid":6297,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/328992853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":328992853},{"type":"v1","addr":"172.21.15.7:6835","nonce":328992853}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-27T02:37:36.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.910+0000 7f38f3635700 1 -- 172.21.15.7:0/2299994917 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f38d0066f30 msgr2=0x7f38d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:36.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.910+0000 7f38f3635700 1 --2- 172.21.15.7:0/2299994917 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f38d0066f30 0x7f38d00693e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f38e8006f00 tx=0x7f38e8008040 comp rx=0 tx=0).stop 2024-01-27T02:37:36.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.910+0000 7f38f3635700 1 -- 172.21.15.7:0/2299994917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38ec151570 msgr2=0x7f38ec16a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:36.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.910+0000 7f38f3635700 1 --2- 172.21.15.7:0/2299994917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38ec151570 0x7f38ec16a6e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f38e400c8e0 tx=0x7f38e4001710 comp rx=0 tx=0).stop 2024-01-27T02:37:36.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.910+0000 7f38f3635700 1 -- 172.21.15.7:0/2299994917 shutdown_connections 2024-01-27T02:37:36.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.910+0000 7f38f3635700 1 --2- 172.21.15.7:0/2299994917 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f38d0066f30 0x7f38d00693e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:36.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.910+0000 7f38f3635700 1 --2- 172.21.15.7:0/2299994917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38ec151570 0x7f38ec16a6e0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:36.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.910+0000 7f38f3635700 1 --2- 172.21.15.7:0/2299994917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38ec06b400 0x7f38ec148fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:36.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.910+0000 7f38f3635700 1 --2- 172.21.15.7:0/2299994917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38ec06e870 0x7f38ec16a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:36.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.910+0000 7f38f3635700 1 -- 172.21.15.7:0/2299994917 >> 172.21.15.7:0/2299994917 conn(0x7f38ec0cdc30 msgr2=0x7f38ec0ca350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:36.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.910+0000 7f38f3635700 1 -- 172.21.15.7:0/2299994917 shutdown_connections 2024-01-27T02:37:36.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:36.910+0000 7f38f3635700 1 -- 172.21.15.7:0/2299994917 wait complete. 2024-01-27T02:37:36.913 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 49 2024-01-27T02:37:36.925 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:37:37.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.055+0000 7fa1f76d1700 1 Processor -- start 2024-01-27T02:37:37.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.055+0000 7fa1f76d1700 1 -- start start 2024-01-27T02:37:37.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.056+0000 7fa1f76d1700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa1f015d590 0x7fa1f00cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:37.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.056+0000 7fa1f76d1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f00ce2c0 0x7fa1f00ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:37.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.056+0000 7fa1f76d1700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f00c9e50 con 0x7fa1f00679b0 2024-01-27T02:37:37.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.056+0000 7fa1f76d1700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f00ce780 con 0x7fa1f015d590 2024-01-27T02:37:37.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.056+0000 7fa1f76d1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f00c9520 con 0x7fa1f00ce2c0 2024-01-27T02:37:37.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.056+0000 7fa1f5c6e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f00ce2c0 0x7fa1f00ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:37.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.056+0000 7fa1f546d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa1f015d590 0x7fa1f00cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:37.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.057+0000 7fa1f5c6e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f00ce2c0 0x7fa1f00ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:38344/0 (socket says 172.21.15.7:38344) 2024-01-27T02:37:37.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.057+0000 7fa1f546d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa1f015d590 0x7fa1f00cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:33732/0 (socket says 172.21.15.7:33732) 2024-01-27T02:37:37.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.057+0000 7fa1f5c6e700 1 -- 172.21.15.7:0/529203384 learned_addr learned my addr 172.21.15.7:0/529203384 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:37.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.057+0000 7fa1e7fff700 1 -- 172.21.15.7:0/529203384 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 326773921 0 0) 0x7fa1f00c9e50 con 0x7fa1f00679b0 2024-01-27T02:37:37.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.057+0000 7fa1e7fff700 1 -- 172.21.15.7:0/529203384 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1dc003570 con 0x7fa1f00679b0 2024-01-27T02:37:37.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.057+0000 7fa1f5c6e700 1 -- 172.21.15.7:0/529203384 >> v2:172.21.15.188:3300/0 conn(0x7fa1f015d590 msgr2=0x7fa1f00cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:37.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.057+0000 7fa1f5c6e700 1 --2- 172.21.15.7:0/529203384 >> v2:172.21.15.188:3300/0 conn(0x7fa1f015d590 0x7fa1f00cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.057+0000 7fa1f5c6e700 1 -- 172.21.15.7:0/529203384 >> v1:172.21.15.188:6789/0 conn(0x7fa1f00679b0 legacy=0x7fa1f00c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:37.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.057+0000 7fa1f5c6e700 1 -- 172.21.15.7:0/529203384 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f00c9800 con 0x7fa1f00ce2c0 2024-01-27T02:37:37.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.057+0000 7fa1f546d700 1 --2- 172.21.15.7:0/529203384 >> v2:172.21.15.188:3300/0 conn(0x7fa1f015d590 0x7fa1f00cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:37:37.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.057+0000 7fa1f5c6e700 1 --2- 172.21.15.7:0/529203384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f00ce2c0 0x7fa1f00ca2e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009f20 tx=0x7fa1e8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b42cf3c495beca4e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:37.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.057+0000 7fa1e7fff700 1 -- 172.21.15.7:0/529203384 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e800e070 con 0x7fa1f00ce2c0 2024-01-27T02:37:37.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.057+0000 7fa1e7fff700 1 -- 172.21.15.7:0/529203384 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e800b970 con 0x7fa1f00ce2c0 2024-01-27T02:37:37.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.057+0000 7fa1e7fff700 1 -- 172.21.15.7:0/529203384 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e80092e0 con 0x7fa1f00ce2c0 2024-01-27T02:37:37.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.058+0000 7fa1f76d1700 1 -- 172.21.15.7:0/529203384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f00ce2c0 msgr2=0x7fa1f00ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:37.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.058+0000 7fa1f76d1700 1 --2- 172.21.15.7:0/529203384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f00ce2c0 0x7fa1f00ca2e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009f20 tx=0x7fa1e8009840 comp rx=0 tx=0).stop 2024-01-27T02:37:37.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.058+0000 7fa1f76d1700 1 -- 172.21.15.7:0/529203384 shutdown_connections 2024-01-27T02:37:37.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.058+0000 7fa1f76d1700 1 --2- 172.21.15.7:0/529203384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f00ce2c0 0x7fa1f00ca2e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.058+0000 7fa1f76d1700 1 --2- 172.21.15.7:0/529203384 >> v2:172.21.15.188:3300/0 conn(0x7fa1f015d590 0x7fa1f00cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.058+0000 7fa1f76d1700 1 -- 172.21.15.7:0/529203384 >> 172.21.15.7:0/529203384 conn(0x7fa1f0098020 msgr2=0x7fa1f00711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:37.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.058+0000 7fa1f76d1700 1 -- 172.21.15.7:0/529203384 shutdown_connections 2024-01-27T02:37:37.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.058+0000 7fa1f76d1700 1 -- 172.21.15.7:0/529203384 wait complete. 2024-01-27T02:37:37.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.059+0000 7fa1f76d1700 1 Processor -- start 2024-01-27T02:37:37.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.059+0000 7fa1f76d1700 1 -- start start 2024-01-27T02:37:37.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.060+0000 7fa1f76d1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f00679b0 0x7fa1f00b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:37.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.060+0000 7fa1f76d1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f00ce2c0 0x7fa1f00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:37.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.060+0000 7fa1f76d1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f015d590 0x7fa1f0169930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:37.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.060+0000 7fa1f76d1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f00b34d0 con 0x7fa1f00679b0 2024-01-27T02:37:37.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.060+0000 7fa1f76d1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f00b28c0 con 0x7fa1f00ce2c0 2024-01-27T02:37:37.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.060+0000 7fa1f76d1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f00b2ba0 con 0x7fa1f015d590 2024-01-27T02:37:37.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.060+0000 7fa1f546d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f00679b0 0x7fa1f00b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:37.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.060+0000 7fa1f546d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f00679b0 0x7fa1f00b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35434/0 (socket says 172.21.15.7:35434) 2024-01-27T02:37:37.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.060+0000 7fa1f546d700 1 -- 172.21.15.7:0/2140777637 learned_addr learned my addr 172.21.15.7:0/2140777637 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:37.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.060+0000 7fa1f5c6e700 1 --2- 172.21.15.7:0/2140777637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f015d590 0x7fa1f0169930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:37.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.060+0000 7fa1f4c6c700 1 --2- 172.21.15.7:0/2140777637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f00ce2c0 0x7fa1f00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:37.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.061+0000 7fa1f546d700 1 -- 172.21.15.7:0/2140777637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f015d590 msgr2=0x7fa1f0169930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:37.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.061+0000 7fa1f546d700 1 --2- 172.21.15.7:0/2140777637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f015d590 0x7fa1f0169930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.061+0000 7fa1f546d700 1 -- 172.21.15.7:0/2140777637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f00ce2c0 msgr2=0x7fa1f00b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:37.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.061+0000 7fa1f546d700 1 --2- 172.21.15.7:0/2140777637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f00ce2c0 0x7fa1f00b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.061+0000 7fa1f546d700 1 -- 172.21.15.7:0/2140777637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec009420 con 0x7fa1f00679b0 2024-01-27T02:37:37.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.061+0000 7fa1f546d700 1 --2- 172.21.15.7:0/2140777637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f00679b0 0x7fa1f00b1b50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00ec80 tx=0x7fa1ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:37.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.061+0000 7fa1e57fa700 1 -- 172.21.15.7:0/2140777637 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec00c850 con 0x7fa1f00679b0 2024-01-27T02:37:37.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.061+0000 7fa1e57fa700 1 -- 172.21.15.7:0/2140777637 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ec00c9b0 con 0x7fa1f00679b0 2024-01-27T02:37:37.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.061+0000 7fa1f76d1700 1 -- 172.21.15.7:0/2140777637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e8009ae0 con 0x7fa1f00679b0 2024-01-27T02:37:37.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.061+0000 7fa1e57fa700 1 -- 172.21.15.7:0/2140777637 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec00cd30 con 0x7fa1f00679b0 2024-01-27T02:37:37.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.061+0000 7fa1f76d1700 1 -- 172.21.15.7:0/2140777637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f00b3160 con 0x7fa1f00679b0 2024-01-27T02:37:37.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.062+0000 7fa1f76d1700 1 -- 172.21.15.7:0/2140777637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f004c710 con 0x7fa1f00679b0 2024-01-27T02:37:37.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.063+0000 7fa1e57fa700 1 -- 172.21.15.7:0/2140777637 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa1ec013680 con 0x7fa1f00679b0 2024-01-27T02:37:37.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.063+0000 7fa1e57fa700 1 --2- 172.21.15.7:0/2140777637 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa1d4066f30 0x7fa1d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:37.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.063+0000 7fa1e57fa700 1 -- 172.21.15.7:0/2140777637 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa1ec084810 con 0x7fa1f00679b0 2024-01-27T02:37:37.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.063+0000 7fa1f4c6c700 1 --2- 172.21.15.7:0/2140777637 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa1d4066f30 0x7fa1d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:37.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.064+0000 7fa1f4c6c700 1 --2- 172.21.15.7:0/2140777637 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa1d4066f30 0x7fa1d40693e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00c5380 tx=0x7fa1e0007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:37.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.068+0000 7fa1e57fa700 1 -- 172.21.15.7:0/2140777637 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1ec055510 con 0x7fa1f00679b0 2024-01-27T02:37:37.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.272+0000 7fa1f76d1700 1 -- 172.21.15.7:0/2140777637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1f00c9ae0 con 0x7fa1f00679b0 2024-01-27T02:37:37.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.273+0000 7fa1e57fa700 1 -- 172.21.15.7:0/2140777637 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+15309 (secure 0 0 0) 0x7fa1ec017090 con 0x7fa1f00679b0 2024-01-27T02:37:37.274 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:37.274 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":106,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:37:32.458337+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:37:01.445734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-27T02:37:02.377151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:37:37.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.275+0000 7fa1f76d1700 1 -- 172.21.15.7:0/2140777637 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa1d4066f30 msgr2=0x7fa1d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:37.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.275+0000 7fa1f76d1700 1 --2- 172.21.15.7:0/2140777637 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa1d4066f30 0x7fa1d40693e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00c5380 tx=0x7fa1e0007d30 comp rx=0 tx=0).stop 2024-01-27T02:37:37.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.275+0000 7fa1f76d1700 1 -- 172.21.15.7:0/2140777637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f00679b0 msgr2=0x7fa1f00b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:37.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.275+0000 7fa1f76d1700 1 --2- 172.21.15.7:0/2140777637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f00679b0 0x7fa1f00b1b50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00ec80 tx=0x7fa1ec0091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.275+0000 7fa1f76d1700 1 -- 172.21.15.7:0/2140777637 shutdown_connections 2024-01-27T02:37:37.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.275+0000 7fa1f76d1700 1 --2- 172.21.15.7:0/2140777637 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa1d4066f30 0x7fa1d40693e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.275+0000 7fa1f76d1700 1 --2- 172.21.15.7:0/2140777637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f015d590 0x7fa1f0169930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.275+0000 7fa1f76d1700 1 --2- 172.21.15.7:0/2140777637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f00679b0 0x7fa1f00b1b50 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.275+0000 7fa1f76d1700 1 --2- 172.21.15.7:0/2140777637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f00ce2c0 0x7fa1f00b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.275+0000 7fa1f76d1700 1 -- 172.21.15.7:0/2140777637 >> 172.21.15.7:0/2140777637 conn(0x7fa1f0098020 msgr2=0x7fa1f006ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:37.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.275+0000 7fa1f76d1700 1 -- 172.21.15.7:0/2140777637 shutdown_connections 2024-01-27T02:37:37.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.275+0000 7fa1f76d1700 1 -- 172.21.15.7:0/2140777637 wait complete. 2024-01-27T02:37:37.290 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-27T02:37:01.445734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-01-27T02:37:02.377151+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-27T02:37:37.291 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:37:37.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.423+0000 7f0987b05700 1 Processor -- start 2024-01-27T02:37:37.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.424+0000 7f0987b05700 1 -- start start 2024-01-27T02:37:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.424+0000 7f0987b05700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0980154550 0x7f09800ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.425+0000 7f0987b05700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09800adec0 con 0x7f09800ad7c0 2024-01-27T02:37:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.425+0000 7f0987b05700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09800ae1a0 con 0x7f0980154920 2024-01-27T02:37:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.425+0000 7f0987b05700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f098015a850 con 0x7f0980154550 2024-01-27T02:37:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.425+0000 7f09860a2700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f09800ad7c0 0x7f0980063580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59546/0 (socket says 172.21.15.7:59546) 2024-01-27T02:37:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.425+0000 7f09860a2700 1 -- 172.21.15.7:0/230916007 learned_addr learned my addr 172.21.15.7:0/230916007 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:37.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.425+0000 7f09858a1700 1 --2- 172.21.15.7:0/230916007 >> v2:172.21.15.188:3300/0 conn(0x7f0980154550 0x7f09800ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:37.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.425+0000 7f098489f700 1 -- 172.21.15.7:0/230916007 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 962555620 0 0) 0x7f09800adec0 con 0x7f09800ad7c0 2024-01-27T02:37:37.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.425+0000 7f098489f700 1 -- 172.21.15.7:0/230916007 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f096c003570 con 0x7f09800ad7c0 2024-01-27T02:37:37.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.425+0000 7f098489f700 1 -- 172.21.15.7:0/230916007 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523599781 0 0) 0x7f09800ae1a0 con 0x7f0980154920 2024-01-27T02:37:37.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.425+0000 7f098489f700 1 -- 172.21.15.7:0/230916007 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09800adec0 con 0x7f0980154920 2024-01-27T02:37:37.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.425+0000 7f09858a1700 1 -- 172.21.15.7:0/230916007 >> v1:172.21.15.188:6789/0 conn(0x7f0980154920 legacy=0x7f098015b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:37.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.425+0000 7f09858a1700 1 -- 172.21.15.7:0/230916007 >> v1:172.21.15.7:6789/0 conn(0x7f09800ad7c0 legacy=0x7f0980063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:37.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.425+0000 7f09858a1700 1 -- 172.21.15.7:0/230916007 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098015ab30 con 0x7f0980154550 2024-01-27T02:37:37.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.426+0000 7f09858a1700 1 --2- 172.21.15.7:0/230916007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0980154550 0x7f09800ce6b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f097c0098e0 tx=0x7f097c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d049cbedf8bbdb0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:37.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.426+0000 7f098489f700 1 -- 172.21.15.7:0/230916007 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097c011630 con 0x7f0980154550 2024-01-27T02:37:37.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.426+0000 7f098489f700 1 -- 172.21.15.7:0/230916007 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097c011790 con 0x7f0980154550 2024-01-27T02:37:37.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.426+0000 7f098489f700 1 -- 172.21.15.7:0/230916007 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097c011b30 con 0x7f0980154550 2024-01-27T02:37:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.426+0000 7f0987b05700 1 -- 172.21.15.7:0/230916007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0980154550 msgr2=0x7f09800ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.426+0000 7f0987b05700 1 --2- 172.21.15.7:0/230916007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0980154550 0x7f09800ce6b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f097c0098e0 tx=0x7f097c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.426+0000 7f0987b05700 1 -- 172.21.15.7:0/230916007 shutdown_connections 2024-01-27T02:37:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.426+0000 7f0987b05700 1 --2- 172.21.15.7:0/230916007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0980154550 0x7f09800ce6b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.426+0000 7f0987b05700 1 -- 172.21.15.7:0/230916007 >> 172.21.15.7:0/230916007 conn(0x7f0980068480 msgr2=0x7f09800cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.426+0000 7f0987b05700 1 -- 172.21.15.7:0/230916007 shutdown_connections 2024-01-27T02:37:37.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.426+0000 7f0987b05700 1 -- 172.21.15.7:0/230916007 wait complete. 2024-01-27T02:37:37.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.428+0000 7f0987b05700 1 Processor -- start 2024-01-27T02:37:37.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.428+0000 7f0987b05700 1 -- start start 2024-01-27T02:37:37.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.428+0000 7f0987b05700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09800ad7c0 0x7f0980147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:37.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.428+0000 7f0987b05700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0980154550 0x7f09800b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:37.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.428+0000 7f0987b05700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0980154920 0x7f09800b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:37.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.428+0000 7f0987b05700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09800c3740 con 0x7f09800ad7c0 2024-01-27T02:37:37.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.428+0000 7f0987b05700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09800c2b30 con 0x7f0980154550 2024-01-27T02:37:37.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.428+0000 7f0987b05700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09800c2e10 con 0x7f0980154920 2024-01-27T02:37:37.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.429+0000 7f09858a1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09800ad7c0 0x7f0980147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:37.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.429+0000 7f09858a1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09800ad7c0 0x7f0980147640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35440/0 (socket says 172.21.15.7:35440) 2024-01-27T02:37:37.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.429+0000 7f09858a1700 1 -- 172.21.15.7:0/2571966098 learned_addr learned my addr 172.21.15.7:0/2571966098 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:37.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.429+0000 7f09860a2700 1 --2- 172.21.15.7:0/2571966098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0980154920 0x7f09800b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:37.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.429+0000 7f09850a0700 1 --2- 172.21.15.7:0/2571966098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0980154550 0x7f09800b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.429+0000 7f09858a1700 1 -- 172.21.15.7:0/2571966098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0980154920 msgr2=0x7f09800b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.429+0000 7f09858a1700 1 --2- 172.21.15.7:0/2571966098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0980154920 0x7f09800b25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.429+0000 7f09858a1700 1 -- 172.21.15.7:0/2571966098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0980154550 msgr2=0x7f09800b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.429+0000 7f09858a1700 1 --2- 172.21.15.7:0/2571966098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0980154550 0x7f09800b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.429+0000 7f09858a1700 1 -- 172.21.15.7:0/2571966098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097c009420 con 0x7f09800ad7c0 2024-01-27T02:37:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.429+0000 7f09858a1700 1 --2- 172.21.15.7:0/2571966098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09800ad7c0 0x7f0980147640 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f09800700c0 tx=0x7f097c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.429+0000 7f0975ffb700 1 -- 172.21.15.7:0/2571966098 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097c01f070 con 0x7f09800ad7c0 2024-01-27T02:37:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.429+0000 7f0975ffb700 1 -- 172.21.15.7:0/2571966098 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097c011900 con 0x7f09800ad7c0 2024-01-27T02:37:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.429+0000 7f0987b05700 1 -- 172.21.15.7:0/2571966098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09800c30f0 con 0x7f09800ad7c0 2024-01-27T02:37:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.429+0000 7f0975ffb700 1 -- 172.21.15.7:0/2571966098 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097c023410 con 0x7f09800ad7c0 2024-01-27T02:37:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.430+0000 7f0987b05700 1 -- 172.21.15.7:0/2571966098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0980147b80 con 0x7f09800ad7c0 2024-01-27T02:37:37.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.431+0000 7f0987b05700 1 -- 172.21.15.7:0/2571966098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0980072060 con 0x7f09800ad7c0 2024-01-27T02:37:37.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.431+0000 7f0975ffb700 1 -- 172.21.15.7:0/2571966098 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f097c02c3b0 con 0x7f09800ad7c0 2024-01-27T02:37:37.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.432+0000 7f0975ffb700 1 --2- 172.21.15.7:0/2571966098 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0964066f30 0x7f09640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:37.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.432+0000 7f0975ffb700 1 -- 172.21.15.7:0/2571966098 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f097c0857f0 con 0x7f09800ad7c0 2024-01-27T02:37:37.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.432+0000 7f09850a0700 1 --2- 172.21.15.7:0/2571966098 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0964066f30 0x7f09640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:37.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.432+0000 7f09850a0700 1 --2- 172.21.15.7:0/2571966098 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0964066f30 0x7f09640693e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0980157d10 tx=0x7f0970007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:37.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.436+0000 7f0975ffb700 1 -- 172.21.15.7:0/2571966098 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f097c0529c0 con 0x7f09800ad7c0 2024-01-27T02:37:37.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.638+0000 7f0987b05700 1 -- 172.21.15.7:0/2571966098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f098006f780 con 0x7f09800ad7c0 2024-01-27T02:37:37.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.639+0000 7f0975ffb700 1 -- 172.21.15.7:0/2571966098 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7f097c055fe0 con 0x7f09800ad7c0 2024-01-27T02:37:37.640 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:37:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.641+0000 7f0987b05700 1 -- 172.21.15.7:0/2571966098 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0964066f30 msgr2=0x7f09640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.641+0000 7f0987b05700 1 --2- 172.21.15.7:0/2571966098 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0964066f30 0x7f09640693e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0980157d10 tx=0x7f0970007d50 comp rx=0 tx=0).stop 2024-01-27T02:37:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.641+0000 7f0987b05700 1 -- 172.21.15.7:0/2571966098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09800ad7c0 msgr2=0x7f0980147640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.641+0000 7f0987b05700 1 --2- 172.21.15.7:0/2571966098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09800ad7c0 0x7f0980147640 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f09800700c0 tx=0x7f097c017000 comp rx=0 tx=0).stop 2024-01-27T02:37:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.641+0000 7f0987b05700 1 -- 172.21.15.7:0/2571966098 shutdown_connections 2024-01-27T02:37:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.641+0000 7f0987b05700 1 --2- 172.21.15.7:0/2571966098 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0964066f30 0x7f09640693e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.641+0000 7f0987b05700 1 --2- 172.21.15.7:0/2571966098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0980154920 0x7f09800b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.641+0000 7f0987b05700 1 --2- 172.21.15.7:0/2571966098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09800ad7c0 0x7f0980147640 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.641+0000 7f0987b05700 1 --2- 172.21.15.7:0/2571966098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0980154550 0x7f09800b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.641+0000 7f0987b05700 1 -- 172.21.15.7:0/2571966098 >> 172.21.15.7:0/2571966098 conn(0x7f0980068480 msgr2=0x7f09800a84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:37.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.641+0000 7f0987b05700 1 -- 172.21.15.7:0/2571966098 shutdown_connections 2024-01-27T02:37:37.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.641+0000 7f0987b05700 1 -- 172.21.15.7:0/2571966098 wait complete. 2024-01-27T02:37:37.656 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-27T02:37:37.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.789+0000 7f6ddc0fe700 1 Processor -- start 2024-01-27T02:37:37.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.789+0000 7f6ddc0fe700 1 -- start start 2024-01-27T02:37:37.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.790+0000 7f6ddc0fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dd415e3d0 0x7f6dd40a8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.790+0000 7f6ddc0fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6dd4063c90 0x7f6dd40afa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.790+0000 7f6ddc0fe700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dd40a80c0 con 0x7f6dd415e7a0 2024-01-27T02:37:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.790+0000 7f6ddc0fe700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dd40ce6b0 con 0x7f6dd4063c90 2024-01-27T02:37:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.790+0000 7f6ddc0fe700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd40a7790 con 0x7f6dd415e3d0 2024-01-27T02:37:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.790+0000 7f6dda69b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6dd4063c90 0x7f6dd40afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.790+0000 7f6dda69b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6dd4063c90 0x7f6dd40afa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35452/0 (socket says 172.21.15.7:35452) 2024-01-27T02:37:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.790+0000 7f6dda69b700 1 -- 172.21.15.7:0/4212737503 learned_addr learned my addr 172.21.15.7:0/4212737503 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.790+0000 7f6dd9e9a700 1 --2- 172.21.15.7:0/4212737503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dd415e3d0 0x7f6dd40a8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.791+0000 7f6dda69b700 1 -- 172.21.15.7:0/4212737503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dd415e3d0 msgr2=0x7f6dd40a8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.791+0000 7f6dda69b700 1 --2- 172.21.15.7:0/4212737503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dd415e3d0 0x7f6dd40a8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.791+0000 7f6dda69b700 1 -- 172.21.15.7:0/4212737503 >> v1:172.21.15.188:6789/0 conn(0x7f6dd415e7a0 legacy=0x7f6dd4063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.791+0000 7f6dda69b700 1 -- 172.21.15.7:0/4212737503 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd40a7a70 con 0x7f6dd4063c90 2024-01-27T02:37:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.791+0000 7f6dda69b700 1 --2- 172.21.15.7:0/4212737503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dd4063c90 0x7f6dd40afa40 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009f20 tx=0x7f6dcc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fde79f00eb5b35d server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:37.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.791+0000 7f6dd8e98700 1 -- 172.21.15.7:0/4212737503 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc00e070 con 0x7f6dd4063c90 2024-01-27T02:37:37.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.791+0000 7f6dd8e98700 1 -- 172.21.15.7:0/4212737503 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dcc00b9d0 con 0x7f6dd4063c90 2024-01-27T02:37:37.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.791+0000 7f6dd8e98700 1 -- 172.21.15.7:0/4212737503 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc0092e0 con 0x7f6dd4063c90 2024-01-27T02:37:37.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.792+0000 7f6ddc0fe700 1 -- 172.21.15.7:0/4212737503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dd4063c90 msgr2=0x7f6dd40afa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:37.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.792+0000 7f6ddc0fe700 1 --2- 172.21.15.7:0/4212737503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dd4063c90 0x7f6dd40afa40 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009f20 tx=0x7f6dcc009840 comp rx=0 tx=0).stop 2024-01-27T02:37:37.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.792+0000 7f6ddc0fe700 1 -- 172.21.15.7:0/4212737503 shutdown_connections 2024-01-27T02:37:37.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.792+0000 7f6ddc0fe700 1 --2- 172.21.15.7:0/4212737503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dd4063c90 0x7f6dd40afa40 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.792+0000 7f6ddc0fe700 1 --2- 172.21.15.7:0/4212737503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dd415e3d0 0x7f6dd40a8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.792+0000 7f6ddc0fe700 1 -- 172.21.15.7:0/4212737503 >> 172.21.15.7:0/4212737503 conn(0x7f6dd4068480 msgr2=0x7f6dd40cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:37.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.792+0000 7f6ddc0fe700 1 -- 172.21.15.7:0/4212737503 shutdown_connections 2024-01-27T02:37:37.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.792+0000 7f6ddc0fe700 1 -- 172.21.15.7:0/4212737503 wait complete. 2024-01-27T02:37:37.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.793+0000 7f6ddc0fe700 1 Processor -- start 2024-01-27T02:37:37.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.793+0000 7f6ddc0fe700 1 -- start start 2024-01-27T02:37:37.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.794+0000 7f6ddc0fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6dd4063c90 0x7f6dd4163490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:37.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.794+0000 7f6ddc0fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dd415e3d0 0x7f6dd41639d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:37.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.794+0000 7f6ddc0fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dd415e7a0 0x7f6dd40b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:37.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.794+0000 7f6ddc0fe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd4164dd0 con 0x7f6dd415e7a0 2024-01-27T02:37:37.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.794+0000 7f6ddc0fe700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd4163f10 con 0x7f6dd4063c90 2024-01-27T02:37:37.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.794+0000 7f6ddc0fe700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd41641c0 con 0x7f6dd415e3d0 2024-01-27T02:37:37.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.794+0000 7f6dda69b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dd415e7a0 0x7f6dd40b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:37.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.794+0000 7f6dda69b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dd415e7a0 0x7f6dd40b0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35468/0 (socket says 172.21.15.7:35468) 2024-01-27T02:37:37.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.794+0000 7f6dda69b700 1 -- 172.21.15.7:0/1549112110 learned_addr learned my addr 172.21.15.7:0/1549112110 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:37.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.794+0000 7f6dd9e9a700 1 --2- 172.21.15.7:0/1549112110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6dd4063c90 0x7f6dd4163490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:37.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.794+0000 7f6dd9699700 1 --2- 172.21.15.7:0/1549112110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dd415e3d0 0x7f6dd41639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:37.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.794+0000 7f6dda69b700 1 -- 172.21.15.7:0/1549112110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dd415e3d0 msgr2=0x7f6dd41639d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:37.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.794+0000 7f6dda69b700 1 --2- 172.21.15.7:0/1549112110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dd415e3d0 0x7f6dd41639d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.794+0000 7f6dda69b700 1 -- 172.21.15.7:0/1549112110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6dd4063c90 msgr2=0x7f6dd4163490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:37.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.794+0000 7f6dda69b700 1 --2- 172.21.15.7:0/1549112110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6dd4063c90 0x7f6dd4163490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:37.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.794+0000 7f6dda69b700 1 -- 172.21.15.7:0/1549112110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc009ae0 con 0x7f6dd415e7a0 2024-01-27T02:37:37.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.794+0000 7f6dda69b700 1 --2- 172.21.15.7:0/1549112110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dd415e7a0 0x7f6dd40b0990 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40700c0 tx=0x7f6dcc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:37.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.795+0000 7f6dc9ffb700 1 -- 172.21.15.7:0/1549112110 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc008ad0 con 0x7f6dd415e7a0 2024-01-27T02:37:37.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.795+0000 7f6dc9ffb700 1 -- 172.21.15.7:0/1549112110 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dcc008c30 con 0x7f6dd415e7a0 2024-01-27T02:37:37.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.795+0000 7f6dc9ffb700 1 -- 172.21.15.7:0/1549112110 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc026460 con 0x7f6dd415e7a0 2024-01-27T02:37:37.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.795+0000 7f6ddc0fe700 1 -- 172.21.15.7:0/1549112110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd41644a0 con 0x7f6dd415e7a0 2024-01-27T02:37:37.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.795+0000 7f6ddc0fe700 1 -- 172.21.15.7:0/1549112110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd4164a60 con 0x7f6dd415e7a0 2024-01-27T02:37:37.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.796+0000 7f6ddc0fe700 1 -- 172.21.15.7:0/1549112110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd4164f80 con 0x7f6dd415e7a0 2024-01-27T02:37:37.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.796+0000 7f6dc9ffb700 1 -- 172.21.15.7:0/1549112110 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6dcc003040 con 0x7f6dd415e7a0 2024-01-27T02:37:37.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.797+0000 7f6dc9ffb700 1 --2- 172.21.15.7:0/1549112110 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6db8066f30 0x7f6db80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:37.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.797+0000 7f6dc9ffb700 1 -- 172.21.15.7:0/1549112110 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6dcc085db0 con 0x7f6dd415e7a0 2024-01-27T02:37:37.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.797+0000 7f6dd9e9a700 1 --2- 172.21.15.7:0/1549112110 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6db8066f30 0x7f6db80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:37.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.798+0000 7f6dd9e9a700 1 --2- 172.21.15.7:0/1549112110 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6db8066f30 0x7f6db80693e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009db0 tx=0x7f6dd00091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:37.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:37.801+0000 7f6dc9ffb700 1 -- 172.21.15.7:0/1549112110 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6dcc053020 con 0x7f6dd415e7a0 2024-01-27T02:37:38.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.008+0000 7f6ddc0fe700 1 -- 172.21.15.7:0/1549112110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6dd4072060 con 0x7f6dd415e7a0 2024-01-27T02:37:38.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.008+0000 7f6dc9ffb700 1 -- 172.21.15.7:0/1549112110 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6dcc056640 con 0x7f6dd415e7a0 2024-01-27T02:37:38.010 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-27T02:37:38.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.011+0000 7f6ddc0fe700 1 -- 172.21.15.7:0/1549112110 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6db8066f30 msgr2=0x7f6db80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:38.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.011+0000 7f6ddc0fe700 1 --2- 172.21.15.7:0/1549112110 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6db8066f30 0x7f6db80693e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009db0 tx=0x7f6dd00091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.011+0000 7f6ddc0fe700 1 -- 172.21.15.7:0/1549112110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dd415e7a0 msgr2=0x7f6dd40b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:38.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.011+0000 7f6ddc0fe700 1 --2- 172.21.15.7:0/1549112110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dd415e7a0 0x7f6dd40b0990 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40700c0 tx=0x7f6dcc018000 comp rx=0 tx=0).stop 2024-01-27T02:37:38.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.011+0000 7f6ddc0fe700 1 -- 172.21.15.7:0/1549112110 shutdown_connections 2024-01-27T02:37:38.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.011+0000 7f6ddc0fe700 1 --2- 172.21.15.7:0/1549112110 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6db8066f30 0x7f6db80693e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.011+0000 7f6ddc0fe700 1 --2- 172.21.15.7:0/1549112110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dd415e7a0 0x7f6dd40b0990 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.011+0000 7f6ddc0fe700 1 --2- 172.21.15.7:0/1549112110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6dd4063c90 0x7f6dd4163490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.011+0000 7f6ddc0fe700 1 --2- 172.21.15.7:0/1549112110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dd415e3d0 0x7f6dd41639d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.011+0000 7f6ddc0fe700 1 -- 172.21.15.7:0/1549112110 >> 172.21.15.7:0/1549112110 conn(0x7f6dd4068480 msgr2=0x7f6dd415a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:38.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.011+0000 7f6ddc0fe700 1 -- 172.21.15.7:0/1549112110 shutdown_connections 2024-01-27T02:37:38.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.011+0000 7f6ddc0fe700 1 -- 172.21.15.7:0/1549112110 wait complete. 2024-01-27T02:37:38.026 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-27T02:37:38.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.160+0000 7f4ab2639700 1 Processor -- start 2024-01-27T02:37:38.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.160+0000 7f4ab2639700 1 -- start start 2024-01-27T02:37:38.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.161+0000 7f4ab2639700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4aac15d830 0x7f4aac0d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:38.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.161+0000 7f4ab2639700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aac0b38d0 con 0x7f4aac15d460 2024-01-27T02:37:38.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.161+0000 7f4ab2639700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aac0d0390 con 0x7f4aac0cf280 2024-01-27T02:37:38.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.161+0000 7f4ab2639700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aac0cf780 con 0x7f4aac15d830 2024-01-27T02:37:38.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.161+0000 7f4aabfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4aac15d460 0x7f4aac15e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59550/0 (socket says 172.21.15.7:59550) 2024-01-27T02:37:38.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.161+0000 7f4aabfff700 1 -- 172.21.15.7:0/3860563658 learned_addr learned my addr 172.21.15.7:0/3860563658 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:38.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.161+0000 7f4ab0bd6700 1 --2- 172.21.15.7:0/3860563658 >> v2:172.21.15.188:3300/0 conn(0x7f4aac15d830 0x7f4aac0d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:38.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.161+0000 7f4aaaffd700 1 -- 172.21.15.7:0/3860563658 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 730821401 0 0) 0x7f4a9c001110 con 0x7f4aac15d460 2024-01-27T02:37:38.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.161+0000 7f4aaaffd700 1 -- 172.21.15.7:0/3860563658 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a94003570 con 0x7f4aac15d460 2024-01-27T02:37:38.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.162+0000 7f4aaaffd700 1 -- 172.21.15.7:0/3860563658 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4065778720 0 0) 0x7f4a94003570 con 0x7f4aac15d460 2024-01-27T02:37:38.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.162+0000 7f4ab0bd6700 1 -- 172.21.15.7:0/3860563658 >> v1:172.21.15.188:6789/0 conn(0x7f4aac0cf280 legacy=0x7f4aac0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:38.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.162+0000 7f4ab0bd6700 1 -- 172.21.15.7:0/3860563658 >> v1:172.21.15.7:6789/0 conn(0x7f4aac15d460 legacy=0x7f4aac15e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:38.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.162+0000 7f4ab0bd6700 1 -- 172.21.15.7:0/3860563658 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aac0cfa60 con 0x7f4aac15d830 2024-01-27T02:37:38.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.162+0000 7f4ab0bd6700 1 --2- 172.21.15.7:0/3860563658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4aac15d830 0x7f4aac0d0820 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009e60 tx=0x7f4aa4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c00aec8a1a36535e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:38.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.162+0000 7f4aaaffd700 1 -- 172.21.15.7:0/3860563658 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa400e070 con 0x7f4aac15d830 2024-01-27T02:37:38.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.162+0000 7f4aaaffd700 1 -- 172.21.15.7:0/3860563658 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa400b850 con 0x7f4aac15d830 2024-01-27T02:37:38.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.162+0000 7f4aaaffd700 1 -- 172.21.15.7:0/3860563658 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c001600 con 0x7f4aac15d830 2024-01-27T02:37:38.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.163+0000 7f4ab2639700 1 -- 172.21.15.7:0/3860563658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4aac15d830 msgr2=0x7f4aac0d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:38.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.163+0000 7f4ab2639700 1 --2- 172.21.15.7:0/3860563658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4aac15d830 0x7f4aac0d0820 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009e60 tx=0x7f4aa4009780 comp rx=0 tx=0).stop 2024-01-27T02:37:38.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.163+0000 7f4ab2639700 1 -- 172.21.15.7:0/3860563658 shutdown_connections 2024-01-27T02:37:38.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.163+0000 7f4ab2639700 1 --2- 172.21.15.7:0/3860563658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4aac15d830 0x7f4aac0d0820 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.163+0000 7f4ab2639700 1 -- 172.21.15.7:0/3860563658 >> 172.21.15.7:0/3860563658 conn(0x7f4aac063580 msgr2=0x7f4aac063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:38.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.163+0000 7f4ab2639700 1 -- 172.21.15.7:0/3860563658 shutdown_connections 2024-01-27T02:37:38.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.163+0000 7f4ab2639700 1 -- 172.21.15.7:0/3860563658 wait complete. 2024-01-27T02:37:38.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.164+0000 7f4ab2639700 1 Processor -- start 2024-01-27T02:37:38.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.164+0000 7f4ab2639700 1 -- start start 2024-01-27T02:37:38.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.165+0000 7f4ab2639700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aac0cf280 0x7f4aac1595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:38.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.165+0000 7f4ab2639700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4aac15d460 0x7f4aac0a7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.165+0000 7f4ab2639700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4aac15d830 0x7f4aac0c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.165+0000 7f4ab2639700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac16a3e0 con 0x7f4aac0cf280 2024-01-27T02:37:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.165+0000 7f4ab2639700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac0c9b60 con 0x7f4aac15d460 2024-01-27T02:37:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.165+0000 7f4ab2639700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac0c92c0 con 0x7f4aac15d830 2024-01-27T02:37:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.165+0000 7f4aabfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aac0cf280 0x7f4aac1595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.165+0000 7f4aabfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aac0cf280 0x7f4aac1595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35480/0 (socket says 172.21.15.7:35480) 2024-01-27T02:37:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.165+0000 7f4aabfff700 1 -- 172.21.15.7:0/839043548 learned_addr learned my addr 172.21.15.7:0/839043548 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.165+0000 7f4ab0bd6700 1 --2- 172.21.15.7:0/839043548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4aac15d830 0x7f4aac0c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.165+0000 7f4aab7fe700 1 --2- 172.21.15.7:0/839043548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4aac15d460 0x7f4aac0a7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.166+0000 7f4aabfff700 1 -- 172.21.15.7:0/839043548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4aac15d830 msgr2=0x7f4aac0c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.166+0000 7f4aabfff700 1 --2- 172.21.15.7:0/839043548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4aac15d830 0x7f4aac0c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.166+0000 7f4aabfff700 1 -- 172.21.15.7:0/839043548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4aac15d460 msgr2=0x7f4aac0a7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:38.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.166+0000 7f4aabfff700 1 --2- 172.21.15.7:0/839043548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4aac15d460 0x7f4aac0a7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.166+0000 7f4aabfff700 1 -- 172.21.15.7:0/839043548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa4009a20 con 0x7f4aac0cf280 2024-01-27T02:37:38.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.166+0000 7f4aabfff700 1 --2- 172.21.15.7:0/839043548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aac0cf280 0x7f4aac1595f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00cb90 tx=0x7f4a9c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:38.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.166+0000 7f4a93fff700 1 -- 172.21.15.7:0/839043548 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c007150 con 0x7f4aac0cf280 2024-01-27T02:37:38.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.166+0000 7f4a93fff700 1 -- 172.21.15.7:0/839043548 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9c0072d0 con 0x7f4aac0cf280 2024-01-27T02:37:38.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.166+0000 7f4ab2639700 1 -- 172.21.15.7:0/839043548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aac0c95a0 con 0x7f4aac0cf280 2024-01-27T02:37:38.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.166+0000 7f4a93fff700 1 -- 172.21.15.7:0/839043548 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c007690 con 0x7f4aac0cf280 2024-01-27T02:37:38.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.166+0000 7f4ab2639700 1 -- 172.21.15.7:0/839043548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aac16a5d0 con 0x7f4aac0cf280 2024-01-27T02:37:38.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.167+0000 7f4ab2639700 1 -- 172.21.15.7:0/839043548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aac072060 con 0x7f4aac0cf280 2024-01-27T02:37:38.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.168+0000 7f4a93fff700 1 -- 172.21.15.7:0/839043548 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4a9c004a40 con 0x7f4aac0cf280 2024-01-27T02:37:38.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.168+0000 7f4a93fff700 1 --2- 172.21.15.7:0/839043548 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a98066f30 0x7f4a980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:38.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.168+0000 7f4a93fff700 1 -- 172.21.15.7:0/839043548 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4a9c0836c0 con 0x7f4aac0cf280 2024-01-27T02:37:38.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.168+0000 7f4aab7fe700 1 --2- 172.21.15.7:0/839043548 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a98066f30 0x7f4a980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:38.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.169+0000 7f4aab7fe700 1 --2- 172.21.15.7:0/839043548 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a98066f30 0x7f4a980693e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0ca240 tx=0x7f4aa0007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:38.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.173+0000 7f4a93fff700 1 -- 172.21.15.7:0/839043548 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a9c0509b0 con 0x7f4aac0cf280 2024-01-27T02:37:38.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.374+0000 7f4ab2639700 1 -- 172.21.15.7:0/839043548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4aac0b38d0 con 0x7f4aac0cf280 2024-01-27T02:37:38.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.374+0000 7f4a93fff700 1 -- 172.21.15.7:0/839043548 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4aac0b38d0 con 0x7f4aac0cf280 2024-01-27T02:37:38.376 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-27T02:37:38.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.376+0000 7f4ab2639700 1 -- 172.21.15.7:0/839043548 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a98066f30 msgr2=0x7f4a980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:38.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.376+0000 7f4ab2639700 1 --2- 172.21.15.7:0/839043548 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a98066f30 0x7f4a980693e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0ca240 tx=0x7f4aa0007ec0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.376+0000 7f4ab2639700 1 -- 172.21.15.7:0/839043548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aac0cf280 msgr2=0x7f4aac1595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:38.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.376+0000 7f4ab2639700 1 --2- 172.21.15.7:0/839043548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aac0cf280 0x7f4aac1595f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00cb90 tx=0x7f4a9c001e50 comp rx=0 tx=0).stop 2024-01-27T02:37:38.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.376+0000 7f4ab2639700 1 -- 172.21.15.7:0/839043548 shutdown_connections 2024-01-27T02:37:38.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.376+0000 7f4ab2639700 1 --2- 172.21.15.7:0/839043548 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a98066f30 0x7f4a980693e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.376+0000 7f4ab2639700 1 --2- 172.21.15.7:0/839043548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4aac15d830 0x7f4aac0c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.376+0000 7f4ab2639700 1 --2- 172.21.15.7:0/839043548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aac0cf280 0x7f4aac1595f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.376+0000 7f4ab2639700 1 --2- 172.21.15.7:0/839043548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4aac15d460 0x7f4aac0a7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.376+0000 7f4ab2639700 1 -- 172.21.15.7:0/839043548 >> 172.21.15.7:0/839043548 conn(0x7f4aac063580 msgr2=0x7f4aac063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:38.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.376+0000 7f4ab2639700 1 -- 172.21.15.7:0/839043548 shutdown_connections 2024-01-27T02:37:38.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.377+0000 7f4ab2639700 1 -- 172.21.15.7:0/839043548 wait complete. 2024-01-27T02:37:38.392 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:37:38.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.523+0000 7fe95ced4700 1 Processor -- start 2024-01-27T02:37:38.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.523+0000 7fe95ced4700 1 -- start start 2024-01-27T02:37:38.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.524+0000 7fe95ced4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9580a7fe0 0x7fe95806f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:38.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.524+0000 7fe95ced4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe95815dfb0 0x7fe95806f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:38.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.524+0000 7fe95ced4700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95815a880 con 0x7fe9580a7c10 2024-01-27T02:37:38.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.524+0000 7fe95ced4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9580afde0 con 0x7fe95815dfb0 2024-01-27T02:37:38.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.524+0000 7fe95ced4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580af950 con 0x7fe9580a7fe0 2024-01-27T02:37:38.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.524+0000 7fe956d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe95815dfb0 0x7fe95806f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:38.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.524+0000 7fe956d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe95815dfb0 0x7fe95806f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35494/0 (socket says 172.21.15.7:35494) 2024-01-27T02:37:38.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.524+0000 7fe956d9d700 1 -- 172.21.15.7:0/3886421174 learned_addr learned my addr 172.21.15.7:0/3886421174 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:38.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.524+0000 7fe955d9b700 1 --2- 172.21.15.7:0/3886421174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9580a7fe0 0x7fe95806f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:38.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.524+0000 7fe956d9d700 1 -- 172.21.15.7:0/3886421174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9580a7fe0 msgr2=0x7fe95806f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:38.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.524+0000 7fe956d9d700 1 --2- 172.21.15.7:0/3886421174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9580a7fe0 0x7fe95806f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.524+0000 7fe956d9d700 1 -- 172.21.15.7:0/3886421174 >> v1:172.21.15.188:6789/0 conn(0x7fe9580a7c10 legacy=0x7fe9580ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:38.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.524+0000 7fe956d9d700 1 -- 172.21.15.7:0/3886421174 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9580aed40 con 0x7fe95815dfb0 2024-01-27T02:37:38.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.525+0000 7fe956d9d700 1 --2- 172.21.15.7:0/3886421174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe95815dfb0 0x7fe95806f8a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe950009f20 tx=0x7fe950009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9921fe4a28f609a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:38.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.525+0000 7fe95559a700 1 -- 172.21.15.7:0/3886421174 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95000e070 con 0x7fe95815dfb0 2024-01-27T02:37:38.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.525+0000 7fe95559a700 1 -- 172.21.15.7:0/3886421174 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95000ba80 con 0x7fe95815dfb0 2024-01-27T02:37:38.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.525+0000 7fe95559a700 1 -- 172.21.15.7:0/3886421174 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9500092e0 con 0x7fe95815dfb0 2024-01-27T02:37:38.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.525+0000 7fe95ced4700 1 -- 172.21.15.7:0/3886421174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe95815dfb0 msgr2=0x7fe95806f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:38.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.525+0000 7fe95ced4700 1 --2- 172.21.15.7:0/3886421174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe95815dfb0 0x7fe95806f8a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe950009f20 tx=0x7fe950009840 comp rx=0 tx=0).stop 2024-01-27T02:37:38.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.526+0000 7fe95ced4700 1 -- 172.21.15.7:0/3886421174 shutdown_connections 2024-01-27T02:37:38.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.526+0000 7fe95ced4700 1 --2- 172.21.15.7:0/3886421174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe95815dfb0 0x7fe95806f8a0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.526+0000 7fe95ced4700 1 --2- 172.21.15.7:0/3886421174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9580a7fe0 0x7fe95806f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.526+0000 7fe95ced4700 1 -- 172.21.15.7:0/3886421174 >> 172.21.15.7:0/3886421174 conn(0x7fe9580679b0 msgr2=0x7fe958067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:38.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.526+0000 7fe95ced4700 1 -- 172.21.15.7:0/3886421174 shutdown_connections 2024-01-27T02:37:38.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.526+0000 7fe95ced4700 1 -- 172.21.15.7:0/3886421174 wait complete. 2024-01-27T02:37:38.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.527+0000 7fe95ced4700 1 Processor -- start 2024-01-27T02:37:38.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.527+0000 7fe95ced4700 1 -- start start 2024-01-27T02:37:38.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.528+0000 7fe95ced4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9580a7c10 0x7fe958097a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:38.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.528+0000 7fe95ced4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9580a7fe0 0x7fe95816e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:38.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.528+0000 7fe95ced4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe95815dfb0 0x7fe95806ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:38.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.528+0000 7fe95ced4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95816ead0 con 0x7fe9580a7c10 2024-01-27T02:37:38.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.528+0000 7fe95ced4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe958098020 con 0x7fe95815dfb0 2024-01-27T02:37:38.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.528+0000 7fe95ced4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe958098300 con 0x7fe9580a7fe0 2024-01-27T02:37:38.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.528+0000 7fe95659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9580a7c10 0x7fe958097a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:38.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.528+0000 7fe95659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9580a7c10 0x7fe958097a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35500/0 (socket says 172.21.15.7:35500) 2024-01-27T02:37:38.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.528+0000 7fe95659c700 1 -- 172.21.15.7:0/212715409 learned_addr learned my addr 172.21.15.7:0/212715409 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:38.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.528+0000 7fe955d9b700 1 --2- 172.21.15.7:0/212715409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9580a7fe0 0x7fe95816e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:38.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.528+0000 7fe956d9d700 1 --2- 172.21.15.7:0/212715409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe95815dfb0 0x7fe95806ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:38.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.528+0000 7fe95659c700 1 -- 172.21.15.7:0/212715409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9580a7fe0 msgr2=0x7fe95816e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:38.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.528+0000 7fe95659c700 1 --2- 172.21.15.7:0/212715409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9580a7fe0 0x7fe95816e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.528+0000 7fe95659c700 1 -- 172.21.15.7:0/212715409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe95815dfb0 msgr2=0x7fe95806ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:38.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.528+0000 7fe95659c700 1 --2- 172.21.15.7:0/212715409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe95815dfb0 0x7fe95806ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.528+0000 7fe95659c700 1 -- 172.21.15.7:0/212715409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe950009ae0 con 0x7fe9580a7c10 2024-01-27T02:37:38.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.528+0000 7fe955d9b700 1 --2- 172.21.15.7:0/212715409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9580a7fe0 0x7fe95816e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:38.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.528+0000 7fe956d9d700 1 --2- 172.21.15.7:0/212715409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe95815dfb0 0x7fe95806ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:38.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.528+0000 7fe95659c700 1 --2- 172.21.15.7:0/212715409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9580a7c10 0x7fe958097a40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe94800c590 tx=0x7fe94800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:38.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.528+0000 7fe9467fc700 1 -- 172.21.15.7:0/212715409 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948011590 con 0x7fe9580a7c10 2024-01-27T02:37:38.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.529+0000 7fe9467fc700 1 -- 172.21.15.7:0/212715409 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9480116f0 con 0x7fe9580a7c10 2024-01-27T02:37:38.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.529+0000 7fe95ced4700 1 -- 172.21.15.7:0/212715409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9580985e0 con 0x7fe9580a7c10 2024-01-27T02:37:38.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.529+0000 7fe9467fc700 1 -- 172.21.15.7:0/212715409 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948011ab0 con 0x7fe9580a7c10 2024-01-27T02:37:38.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.529+0000 7fe95ced4700 1 -- 172.21.15.7:0/212715409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95806b250 con 0x7fe9580a7c10 2024-01-27T02:37:38.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.529+0000 7fe95ced4700 1 -- 172.21.15.7:0/212715409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe95804c710 con 0x7fe9580a7c10 2024-01-27T02:37:38.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.530+0000 7fe9467fc700 1 -- 172.21.15.7:0/212715409 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe94801b3b0 con 0x7fe9580a7c10 2024-01-27T02:37:38.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.530+0000 7fe9467fc700 1 --2- 172.21.15.7:0/212715409 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe938066f30 0x7fe9380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:38.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.530+0000 7fe9467fc700 1 -- 172.21.15.7:0/212715409 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe948083ae0 con 0x7fe9580a7c10 2024-01-27T02:37:38.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.531+0000 7fe955d9b700 1 --2- 172.21.15.7:0/212715409 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe938066f30 0x7fe9380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:38.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.531+0000 7fe955d9b700 1 --2- 172.21.15.7:0/212715409 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe938066f30 0x7fe9380693e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe958159b90 tx=0x7fe94c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:38.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.535+0000 7fe9467fc700 1 -- 172.21.15.7:0/212715409 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe948050d50 con 0x7fe9580a7c10 2024-01-27T02:37:38.636 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:37:38.639 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:37:38.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.790+0000 7fe95ced4700 1 -- 172.21.15.7:0/212715409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9580af020 con 0x7fe9580a7c10 2024-01-27T02:37:38.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.790+0000 7fe9467fc700 1 -- 172.21.15.7:0/212715409 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4584 (secure 0 0 0) 0x7fe948017070 con 0x7fe9580a7c10 2024-01-27T02:37:38.792 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:38.792 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/450556982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":450556982},{"type":"v1","addr":"172.21.15.188:6836","nonce":450556982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1711422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1711422},{"type":"v1","addr":"172.21.15.188:6837","nonce":1711422}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6294,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1110306600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1110306600},{"type":"v1","addr":"172.21.15.7:6833","nonce":1110306600}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:37:03.404355+0000","modified":"2024-01-27T02:37:06.988798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6297":{"gid":6297,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/328992853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":328992853},{"type":"v1","addr":"172.21.15.7:6835","nonce":328992853}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-27T02:37:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.792+0000 7fe95ced4700 1 -- 172.21.15.7:0/212715409 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe938066f30 msgr2=0x7fe9380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.793+0000 7fe95ced4700 1 --2- 172.21.15.7:0/212715409 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe938066f30 0x7fe9380693e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe958159b90 tx=0x7fe94c008040 comp rx=0 tx=0).stop 2024-01-27T02:37:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.793+0000 7fe95ced4700 1 -- 172.21.15.7:0/212715409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9580a7c10 msgr2=0x7fe958097a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.793+0000 7fe95ced4700 1 --2- 172.21.15.7:0/212715409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9580a7c10 0x7fe958097a40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe94800c590 tx=0x7fe94800c950 comp rx=0 tx=0).stop 2024-01-27T02:37:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.793+0000 7fe95ced4700 1 -- 172.21.15.7:0/212715409 shutdown_connections 2024-01-27T02:37:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.793+0000 7fe95ced4700 1 --2- 172.21.15.7:0/212715409 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe938066f30 0x7fe9380693e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.793+0000 7fe95ced4700 1 --2- 172.21.15.7:0/212715409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe95815dfb0 0x7fe95806ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.793+0000 7fe95ced4700 1 --2- 172.21.15.7:0/212715409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9580a7c10 0x7fe958097a40 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.793+0000 7fe95ced4700 1 --2- 172.21.15.7:0/212715409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9580a7fe0 0x7fe95816e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.793+0000 7fe95ced4700 1 -- 172.21.15.7:0/212715409 >> 172.21.15.7:0/212715409 conn(0x7fe9580679b0 msgr2=0x7fe9581589b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.793+0000 7fe95ced4700 1 -- 172.21.15.7:0/212715409 shutdown_connections 2024-01-27T02:37:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.793+0000 7fe95ced4700 1 -- 172.21.15.7:0/212715409 wait complete. 2024-01-27T02:37:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 49 2024-01-27T02:37:38.808 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:37:38.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.938+0000 7f8635989700 1 Processor -- start 2024-01-27T02:37:38.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.938+0000 7f8635989700 1 -- start start 2024-01-27T02:37:38.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.939+0000 7f8635989700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f86300cd310 0x7f863015dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:38.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.939+0000 7f8635989700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f863006e5d0 con 0x7f863015d3a0 2024-01-27T02:37:38.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.939+0000 7f8635989700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86300ce860 con 0x7f863015d770 2024-01-27T02:37:38.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.939+0000 7f8635989700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86300cbe10 con 0x7f86300cd310 2024-01-27T02:37:38.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.939+0000 7f862f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f86300cd310 0x7f863015dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.939+0000 7f862effd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f863015d3a0 0x7f86300c4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59554/0 (socket says 172.21.15.7:59554) 2024-01-27T02:37:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.939+0000 7f862effd700 1 -- 172.21.15.7:0/1774783190 learned_addr learned my addr 172.21.15.7:0/1774783190 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.939+0000 7f862f7fe700 1 -- 172.21.15.7:0/1774783190 >> v1:172.21.15.188:6789/0 conn(0x7f863015d770 legacy=0x7f863006ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.939+0000 7f862f7fe700 1 -- 172.21.15.7:0/1774783190 >> v1:172.21.15.7:6789/0 conn(0x7f863015d3a0 legacy=0x7f86300c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.939+0000 7f862f7fe700 1 -- 172.21.15.7:0/1774783190 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86300cd8f0 con 0x7f86300cd310 2024-01-27T02:37:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.940+0000 7f862f7fe700 1 --2- 172.21.15.7:0/1774783190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86300cd310 0x7f863015dc60 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f86300cdae0 tx=0x7f8628009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95834d6968bc466 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.940+0000 7f862dffb700 1 -- 172.21.15.7:0/1774783190 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628016070 con 0x7f86300cd310 2024-01-27T02:37:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.940+0000 7f862dffb700 1 -- 172.21.15.7:0/1774783190 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862800bc30 con 0x7f86300cd310 2024-01-27T02:37:38.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.940+0000 7f862dffb700 1 -- 172.21.15.7:0/1774783190 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862800e070 con 0x7f86300cd310 2024-01-27T02:37:38.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.940+0000 7f8635989700 1 -- 172.21.15.7:0/1774783190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86300cd310 msgr2=0x7f863015dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:38.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.940+0000 7f8635989700 1 --2- 172.21.15.7:0/1774783190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86300cd310 0x7f863015dc60 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f86300cdae0 tx=0x7f8628009780 comp rx=0 tx=0).stop 2024-01-27T02:37:38.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.940+0000 7f8635989700 1 -- 172.21.15.7:0/1774783190 shutdown_connections 2024-01-27T02:37:38.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.940+0000 7f8635989700 1 --2- 172.21.15.7:0/1774783190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86300cd310 0x7f863015dc60 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.940+0000 7f8635989700 1 -- 172.21.15.7:0/1774783190 >> 172.21.15.7:0/1774783190 conn(0x7f8630068480 msgr2=0x7f86300cdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:38.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.941+0000 7f8635989700 1 -- 172.21.15.7:0/1774783190 shutdown_connections 2024-01-27T02:37:38.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.941+0000 7f8635989700 1 -- 172.21.15.7:0/1774783190 wait complete. 2024-01-27T02:37:38.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.942+0000 7f8635989700 1 Processor -- start 2024-01-27T02:37:38.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.942+0000 7f8635989700 1 -- start start 2024-01-27T02:37:38.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.943+0000 7f8635989700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86300cd310 0x7f8630154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:38.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.943+0000 7f8635989700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f863015d3a0 0x7f86300b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:38.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.943+0000 7f8635989700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f863015d770 0x7f863016a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:38.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.943+0000 7f8635989700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86300af570 con 0x7f863015d3a0 2024-01-27T02:37:38.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.943+0000 7f8635989700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86301585f0 con 0x7f86300cd310 2024-01-27T02:37:38.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.943+0000 7f8635989700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86300ad520 con 0x7f863015d770 2024-01-27T02:37:38.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.943+0000 7f862e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f863015d3a0 0x7f86300b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:38.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.943+0000 7f862e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f863015d3a0 0x7f86300b31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35522/0 (socket says 172.21.15.7:35522) 2024-01-27T02:37:38.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.943+0000 7f862e7fc700 1 -- 172.21.15.7:0/1277899301 learned_addr learned my addr 172.21.15.7:0/1277899301 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:38.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.943+0000 7f862effd700 1 --2- 172.21.15.7:0/1277899301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86300cd310 0x7f8630154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:38.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.943+0000 7f862f7fe700 1 --2- 172.21.15.7:0/1277899301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f863015d770 0x7f863016a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:38.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.943+0000 7f862e7fc700 1 -- 172.21.15.7:0/1277899301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f863015d770 msgr2=0x7f863016a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:38.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.943+0000 7f862e7fc700 1 --2- 172.21.15.7:0/1277899301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f863015d770 0x7f863016a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.943+0000 7f862e7fc700 1 -- 172.21.15.7:0/1277899301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86300cd310 msgr2=0x7f8630154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:38.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.943+0000 7f862e7fc700 1 --2- 172.21.15.7:0/1277899301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86300cd310 0x7f8630154770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:38.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.943+0000 7f862e7fc700 1 -- 172.21.15.7:0/1277899301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8628009a20 con 0x7f863015d3a0 2024-01-27T02:37:38.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.943+0000 7f862e7fc700 1 --2- 172.21.15.7:0/1277899301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f863015d3a0 0x7f86300b31d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f861800c4e0 tx=0x7f8618007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:38.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.944+0000 7f8626ffd700 1 -- 172.21.15.7:0/1277899301 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861800ce90 con 0x7f863015d3a0 2024-01-27T02:37:38.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.944+0000 7f8626ffd700 1 -- 172.21.15.7:0/1277899301 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8618011590 con 0x7f863015d3a0 2024-01-27T02:37:38.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.944+0000 7f8635989700 1 -- 172.21.15.7:0/1277899301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86300af810 con 0x7f863015d3a0 2024-01-27T02:37:38.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.944+0000 7f8626ffd700 1 -- 172.21.15.7:0/1277899301 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8618011950 con 0x7f863015d3a0 2024-01-27T02:37:38.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.944+0000 7f8635989700 1 -- 172.21.15.7:0/1277899301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86300afdd0 con 0x7f863015d3a0 2024-01-27T02:37:38.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.945+0000 7f8635989700 1 -- 172.21.15.7:0/1277899301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8630158810 con 0x7f863015d3a0 2024-01-27T02:37:38.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.945+0000 7f8626ffd700 1 -- 172.21.15.7:0/1277899301 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8618011d00 con 0x7f863015d3a0 2024-01-27T02:37:38.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.946+0000 7f8626ffd700 1 --2- 172.21.15.7:0/1277899301 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8610066f30 0x7f86100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:38.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.946+0000 7f8626ffd700 1 -- 172.21.15.7:0/1277899301 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8618083800 con 0x7f863015d3a0 2024-01-27T02:37:38.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.946+0000 7f862effd700 1 --2- 172.21.15.7:0/1277899301 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8610066f30 0x7f86100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:38.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.946+0000 7f862effd700 1 --2- 172.21.15.7:0/1277899301 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8610066f30 0x7f86100693e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8620007130 tx=0x7f8620007eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:38.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:38.950+0000 7f8626ffd700 1 -- 172.21.15.7:0/1277899301 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8630158810 con 0x7f863015d3a0 2024-01-27T02:37:39.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.156+0000 7f8635989700 1 -- 172.21.15.7:0/1277899301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f863006dba0 con 0x7f863015d3a0 2024-01-27T02:37:39.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.157+0000 7f8626ffd700 1 -- 172.21.15.7:0/1277899301 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+15309 (secure 0 0 0) 0x7f8618017020 con 0x7f863015d3a0 2024-01-27T02:37:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:39.158 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":106,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:37:32.458337+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:37:01.445734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-27T02:37:02.377151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:37:39.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.158+0000 7f8635989700 1 -- 172.21.15.7:0/1277899301 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8610066f30 msgr2=0x7f86100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:39.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.158+0000 7f8635989700 1 --2- 172.21.15.7:0/1277899301 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8610066f30 0x7f86100693e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8620007130 tx=0x7f8620007eb0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.158+0000 7f8635989700 1 -- 172.21.15.7:0/1277899301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f863015d3a0 msgr2=0x7f86300b31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:39.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.158+0000 7f8635989700 1 --2- 172.21.15.7:0/1277899301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f863015d3a0 0x7f86300b31d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f861800c4e0 tx=0x7f8618007f00 comp rx=0 tx=0).stop 2024-01-27T02:37:39.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.159+0000 7f8635989700 1 -- 172.21.15.7:0/1277899301 shutdown_connections 2024-01-27T02:37:39.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.159+0000 7f8635989700 1 --2- 172.21.15.7:0/1277899301 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8610066f30 0x7f86100693e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.159+0000 7f8635989700 1 --2- 172.21.15.7:0/1277899301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f863015d770 0x7f863016a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.159+0000 7f8635989700 1 --2- 172.21.15.7:0/1277899301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86300cd310 0x7f8630154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.159+0000 7f8635989700 1 --2- 172.21.15.7:0/1277899301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f863015d3a0 0x7f86300b31d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.159+0000 7f8635989700 1 -- 172.21.15.7:0/1277899301 >> 172.21.15.7:0/1277899301 conn(0x7f8630068480 msgr2=0x7f86300c3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:39.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.159+0000 7f8635989700 1 -- 172.21.15.7:0/1277899301 shutdown_connections 2024-01-27T02:37:39.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.159+0000 7f8635989700 1 -- 172.21.15.7:0/1277899301 wait complete. 2024-01-27T02:37:39.174 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-27T02:37:39.174 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:37:39.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.304+0000 7feac72d3700 1 Processor -- start 2024-01-27T02:37:39.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.304+0000 7feac72d3700 1 -- start start 2024-01-27T02:37:39.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.305+0000 7feac72d3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac00b1c00 0x7feac006e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:39.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.305+0000 7feac72d3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac0157610 con 0x7feac015b5f0 2024-01-27T02:37:39.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.305+0000 7feac72d3700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac006ee00 con 0x7feac00b1fd0 2024-01-27T02:37:39.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.305+0000 7feac72d3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac006f0e0 con 0x7feac00b1c00 2024-01-27T02:37:39.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.305+0000 7feac486e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7feac015b5f0 0x7feac01566c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59558/0 (socket says 172.21.15.7:59558) 2024-01-27T02:37:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.305+0000 7feac486e700 1 -- 172.21.15.7:0/2493427844 learned_addr learned my addr 172.21.15.7:0/2493427844 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.305+0000 7feac506f700 1 --2- 172.21.15.7:0/2493427844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac00b1c00 0x7feac006e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.305+0000 7feab7fff700 1 -- 172.21.15.7:0/2493427844 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3433813741 0 0) 0x7feac0157610 con 0x7feac015b5f0 2024-01-27T02:37:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.305+0000 7feab7fff700 1 -- 172.21.15.7:0/2493427844 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaac003570 con 0x7feac015b5f0 2024-01-27T02:37:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feab7fff700 1 -- 172.21.15.7:0/2493427844 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2567266301 0 0) 0x7feac006ee00 con 0x7feac00b1fd0 2024-01-27T02:37:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feab7fff700 1 -- 172.21.15.7:0/2493427844 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feac0157610 con 0x7feac00b1fd0 2024-01-27T02:37:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feab7fff700 1 -- 172.21.15.7:0/2493427844 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2776736616 0 0) 0x7feaac003570 con 0x7feac015b5f0 2024-01-27T02:37:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feab7fff700 1 -- 172.21.15.7:0/2493427844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac00b1c00 msgr2=0x7feac006e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feab7fff700 1 --2- 172.21.15.7:0/2493427844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac00b1c00 0x7feac006e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feab7fff700 1 -- 172.21.15.7:0/2493427844 >> v1:172.21.15.188:6789/0 conn(0x7feac00b1fd0 legacy=0x7feac0156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feab7fff700 1 -- 172.21.15.7:0/2493427844 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac006f3c0 con 0x7feac015b5f0 2024-01-27T02:37:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feab7fff700 1 -- 172.21.15.7:0/2493427844 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7feab0001470 con 0x7feac015b5f0 2024-01-27T02:37:39.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feab7fff700 1 -- 172.21.15.7:0/2493427844 >> v1:172.21.15.7:6789/0 conn(0x7feac015b5f0 legacy=0x7feac01566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:39.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feab7fff700 1 --2- 172.21.15.7:0/2493427844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feaac003c20 0x7feaac004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:39.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feab7fff700 1 --2- 172.21.15.7:0/2493427844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaac004810 0x7feaac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:39.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feab7fff700 1 --2- 172.21.15.7:0/2493427844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac00b1c00 0x7feaac005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:39.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feab7fff700 1 -- 172.21.15.7:0/2493427844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac006f0e0 con 0x7feaac004810 2024-01-27T02:37:39.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feab7fff700 1 -- 172.21.15.7:0/2493427844 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaac00bbf0 con 0x7feaac003c20 2024-01-27T02:37:39.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feab7fff700 1 -- 172.21.15.7:0/2493427844 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaac00bdb0 con 0x7feac00b1c00 2024-01-27T02:37:39.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feac486e700 1 --2- 172.21.15.7:0/2493427844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaac004810 0x7feaac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:39.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feac506f700 1 --2- 172.21.15.7:0/2493427844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feaac003c20 0x7feaac004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:39.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feac5870700 1 --2- 172.21.15.7:0/2493427844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac00b1c00 0x7feaac005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:39.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feac486e700 1 -- 172.21.15.7:0/2493427844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac00b1c00 msgr2=0x7feaac005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:39.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feac486e700 1 --2- 172.21.15.7:0/2493427844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac00b1c00 0x7feaac005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feac486e700 1 -- 172.21.15.7:0/2493427844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feaac003c20 msgr2=0x7feaac004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:39.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feac486e700 1 --2- 172.21.15.7:0/2493427844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feaac003c20 0x7feaac004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.306+0000 7feac486e700 1 -- 172.21.15.7:0/2493427844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaac003570 con 0x7feaac004810 2024-01-27T02:37:39.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.307+0000 7feac486e700 1 --2- 172.21.15.7:0/2493427844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaac004810 0x7feaac004c80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7feab000c1e0 tx=0x7feab0001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5350cbbe6dc5759 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:39.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.307+0000 7feab7fff700 1 -- 172.21.15.7:0/2493427844 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab000cb10 con 0x7feaac004810 2024-01-27T02:37:39.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.307+0000 7feab7fff700 1 -- 172.21.15.7:0/2493427844 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab000cc90 con 0x7feaac004810 2024-01-27T02:37:39.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.307+0000 7feab7fff700 1 -- 172.21.15.7:0/2493427844 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab0005b70 con 0x7feaac004810 2024-01-27T02:37:39.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.307+0000 7feac72d3700 1 -- 172.21.15.7:0/2493427844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaac004810 msgr2=0x7feaac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:39.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.307+0000 7feac72d3700 1 --2- 172.21.15.7:0/2493427844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaac004810 0x7feaac004c80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7feab000c1e0 tx=0x7feab0001d40 comp rx=0 tx=0).stop 2024-01-27T02:37:39.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.307+0000 7feac5870700 1 -- 172.21.15.7:0/2493427844 reap_dead start 2024-01-27T02:37:39.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.307+0000 7feac72d3700 1 -- 172.21.15.7:0/2493427844 shutdown_connections 2024-01-27T02:37:39.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.307+0000 7feac72d3700 1 -- 172.21.15.7:0/2493427844 >> 172.21.15.7:0/2493427844 conn(0x7feac0068430 msgr2=0x7feac006be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:39.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.307+0000 7feac72d3700 1 -- 172.21.15.7:0/2493427844 shutdown_connections 2024-01-27T02:37:39.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.307+0000 7feac72d3700 1 -- 172.21.15.7:0/2493427844 wait complete. 2024-01-27T02:37:39.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.309+0000 7feac72d3700 1 Processor -- start 2024-01-27T02:37:39.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.309+0000 7feac72d3700 1 -- start start 2024-01-27T02:37:39.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.310+0000 7feac72d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaac004810 0x7feac0169770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:39.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.310+0000 7feac72d3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feac0146760 0x7feac00ae490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:39.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.310+0000 7feac72d3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac0063580 0x7feac00639f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:39.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.310+0000 7feac72d3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac0063f30 con 0x7feaac004810 2024-01-27T02:37:39.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.310+0000 7feac72d3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac00ac450 con 0x7feac0146760 2024-01-27T02:37:39.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.310+0000 7feac72d3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac00ae9d0 con 0x7feac0063580 2024-01-27T02:37:39.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.310+0000 7feac506f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaac004810 0x7feac0169770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:39.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.310+0000 7feac506f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaac004810 0x7feac0169770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35550/0 (socket says 172.21.15.7:35550) 2024-01-27T02:37:39.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.310+0000 7feac506f700 1 -- 172.21.15.7:0/1451097478 learned_addr learned my addr 172.21.15.7:0/1451097478 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:39.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.310+0000 7feac486e700 1 --2- 172.21.15.7:0/1451097478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feac0146760 0x7feac00ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:39.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.310+0000 7feac5870700 1 --2- 172.21.15.7:0/1451097478 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac0063580 0x7feac00639f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.310+0000 7feac506f700 1 -- 172.21.15.7:0/1451097478 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac0063580 msgr2=0x7feac00639f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.310+0000 7feac506f700 1 --2- 172.21.15.7:0/1451097478 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac0063580 0x7feac00639f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.310+0000 7feac506f700 1 -- 172.21.15.7:0/1451097478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feac0146760 msgr2=0x7feac00ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.310+0000 7feac506f700 1 --2- 172.21.15.7:0/1451097478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feac0146760 0x7feac00ae490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.310+0000 7feac506f700 1 -- 172.21.15.7:0/1451097478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac00ab560 con 0x7feaac004810 2024-01-27T02:37:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.310+0000 7feac506f700 1 --2- 172.21.15.7:0/1451097478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaac004810 0x7feac0169770 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7feabc011d40 tx=0x7feabc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.310+0000 7feab57fa700 1 -- 172.21.15.7:0/1451097478 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc00fac0 con 0x7feaac004810 2024-01-27T02:37:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.310+0000 7feab57fa700 1 -- 172.21.15.7:0/1451097478 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feabc00fc20 con 0x7feaac004810 2024-01-27T02:37:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.310+0000 7feac72d3700 1 -- 172.21.15.7:0/1451097478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac00ab840 con 0x7feaac004810 2024-01-27T02:37:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.311+0000 7feab57fa700 1 -- 172.21.15.7:0/1451097478 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc014440 con 0x7feaac004810 2024-01-27T02:37:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.311+0000 7feac72d3700 1 -- 172.21.15.7:0/1451097478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac00abe00 con 0x7feaac004810 2024-01-27T02:37:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.311+0000 7feac72d3700 1 -- 172.21.15.7:0/1451097478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac004c710 con 0x7feaac004810 2024-01-27T02:37:39.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.312+0000 7feab57fa700 1 -- 172.21.15.7:0/1451097478 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7feabc014a80 con 0x7feaac004810 2024-01-27T02:37:39.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.312+0000 7feab57fa700 1 --2- 172.21.15.7:0/1451097478 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7feaa4066f30 0x7feaa40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:39.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.313+0000 7feab57fa700 1 -- 172.21.15.7:0/1451097478 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feabc018030 con 0x7feaac004810 2024-01-27T02:37:39.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.313+0000 7feac486e700 1 --2- 172.21.15.7:0/1451097478 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7feaa4066f30 0x7feaa40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:39.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.313+0000 7feac486e700 1 --2- 172.21.15.7:0/1451097478 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7feaa4066f30 0x7feaa40693e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7feac015b340 tx=0x7feab0015000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:39.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.317+0000 7feab57fa700 1 -- 172.21.15.7:0/1451097478 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feabc052410 con 0x7feaac004810 2024-01-27T02:37:39.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.571+0000 7feac72d3700 1 -- 172.21.15.7:0/1451097478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feac0158ab0 con 0x7feaac004810 2024-01-27T02:37:39.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.571+0000 7feab57fa700 1 -- 172.21.15.7:0/1451097478 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4584 (secure 0 0 0) 0x7feabc022090 con 0x7feaac004810 2024-01-27T02:37:39.573 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:39.573 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/450556982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":450556982},{"type":"v1","addr":"172.21.15.188:6836","nonce":450556982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1711422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1711422},{"type":"v1","addr":"172.21.15.188:6837","nonce":1711422}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6294,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1110306600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1110306600},{"type":"v1","addr":"172.21.15.7:6833","nonce":1110306600}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:37:03.404355+0000","modified":"2024-01-27T02:37:06.988798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6297":{"gid":6297,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/328992853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":328992853},{"type":"v1","addr":"172.21.15.7:6835","nonce":328992853}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-27T02:37:39.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.574+0000 7feac72d3700 1 -- 172.21.15.7:0/1451097478 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7feaa4066f30 msgr2=0x7feaa40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:39.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.574+0000 7feac72d3700 1 --2- 172.21.15.7:0/1451097478 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7feaa4066f30 0x7feaa40693e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7feac015b340 tx=0x7feab0015000 comp rx=0 tx=0).stop 2024-01-27T02:37:39.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.574+0000 7feac72d3700 1 -- 172.21.15.7:0/1451097478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaac004810 msgr2=0x7feac0169770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:39.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.574+0000 7feac72d3700 1 --2- 172.21.15.7:0/1451097478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaac004810 0x7feac0169770 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7feabc011d40 tx=0x7feabc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.574+0000 7feac72d3700 1 -- 172.21.15.7:0/1451097478 shutdown_connections 2024-01-27T02:37:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.574+0000 7feac72d3700 1 --2- 172.21.15.7:0/1451097478 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7feaa4066f30 0x7feaa40693e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.574+0000 7feac72d3700 1 --2- 172.21.15.7:0/1451097478 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac0063580 0x7feac00639f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.574+0000 7feac72d3700 1 --2- 172.21.15.7:0/1451097478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaac004810 0x7feac0169770 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.574+0000 7feac72d3700 1 --2- 172.21.15.7:0/1451097478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feac0146760 0x7feac00ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.574+0000 7feac72d3700 1 -- 172.21.15.7:0/1451097478 >> 172.21.15.7:0/1451097478 conn(0x7feac0068430 msgr2=0x7feac00afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.574+0000 7feac72d3700 1 -- 172.21.15.7:0/1451097478 shutdown_connections 2024-01-27T02:37:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.574+0000 7feac72d3700 1 -- 172.21.15.7:0/1451097478 wait complete. 2024-01-27T02:37:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 49 2024-01-27T02:37:39.589 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:37:39.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.718+0000 7f8f53c3a700 1 Processor -- start 2024-01-27T02:37:39.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.718+0000 7f8f53c3a700 1 -- start start 2024-01-27T02:37:39.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.719+0000 7f8f53c3a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8f4c0aafc0 0x7f8f4c063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:39.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.719+0000 7f8f53c3a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f4c0c30c0 con 0x7f8f4c159f40 2024-01-27T02:37:39.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.719+0000 7f8f53c3a700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f4c15e960 con 0x7f8f4c15a310 2024-01-27T02:37:39.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.719+0000 7f8f53c3a700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f4c0c4550 con 0x7f8f4c0aafc0 2024-01-27T02:37:39.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.719+0000 7f8f519d6700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8f4c159f40 0x7f8f4c0c3550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59572/0 (socket says 172.21.15.7:59572) 2024-01-27T02:37:39.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.719+0000 7f8f519d6700 1 -- 172.21.15.7:0/3586837536 learned_addr learned my addr 172.21.15.7:0/3586837536 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:39.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.719+0000 7f8f521d7700 1 --2- 172.21.15.7:0/3586837536 >> v2:172.21.15.188:3300/0 conn(0x7f8f4c0aafc0 0x7f8f4c063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:39.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.720+0000 7f8f509d4700 1 -- 172.21.15.7:0/3586837536 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2182916240 0 0) 0x7f8f4c0c30c0 con 0x7f8f4c159f40 2024-01-27T02:37:39.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.720+0000 7f8f509d4700 1 -- 172.21.15.7:0/3586837536 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f38003570 con 0x7f8f4c159f40 2024-01-27T02:37:39.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.720+0000 7f8f509d4700 1 -- 172.21.15.7:0/3586837536 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4272319035 0 0) 0x7f8f4c15e960 con 0x7f8f4c15a310 2024-01-27T02:37:39.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.720+0000 7f8f509d4700 1 -- 172.21.15.7:0/3586837536 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f4c0c30c0 con 0x7f8f4c15a310 2024-01-27T02:37:39.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.720+0000 7f8f509d4700 1 -- 172.21.15.7:0/3586837536 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1819040782 0 0) 0x7f8f38003570 con 0x7f8f4c159f40 2024-01-27T02:37:39.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.720+0000 7f8f521d7700 1 -- 172.21.15.7:0/3586837536 >> v1:172.21.15.188:6789/0 conn(0x7f8f4c15a310 legacy=0x7f8f4c0aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:39.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.720+0000 7f8f521d7700 1 -- 172.21.15.7:0/3586837536 >> v1:172.21.15.7:6789/0 conn(0x7f8f4c159f40 legacy=0x7f8f4c0c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:39.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.720+0000 7f8f521d7700 1 -- 172.21.15.7:0/3586837536 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c15ec40 con 0x7f8f4c0aafc0 2024-01-27T02:37:39.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.720+0000 7f8f521d7700 1 --2- 172.21.15.7:0/3586837536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f4c0aafc0 0x7f8f4c063580 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009e60 tx=0x7f8f44009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2f6dabc8d4d7b58 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:39.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.720+0000 7f8f509d4700 1 -- 172.21.15.7:0/3586837536 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4400e070 con 0x7f8f4c0aafc0 2024-01-27T02:37:39.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.720+0000 7f8f509d4700 1 -- 172.21.15.7:0/3586837536 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4400b7a0 con 0x7f8f4c0aafc0 2024-01-27T02:37:39.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.720+0000 7f8f509d4700 1 -- 172.21.15.7:0/3586837536 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48001470 con 0x7f8f4c0aafc0 2024-01-27T02:37:39.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.721+0000 7f8f53c3a700 1 -- 172.21.15.7:0/3586837536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f4c0aafc0 msgr2=0x7f8f4c063580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:39.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.721+0000 7f8f53c3a700 1 --2- 172.21.15.7:0/3586837536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f4c0aafc0 0x7f8f4c063580 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009e60 tx=0x7f8f44009780 comp rx=0 tx=0).stop 2024-01-27T02:37:39.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.721+0000 7f8f53c3a700 1 -- 172.21.15.7:0/3586837536 shutdown_connections 2024-01-27T02:37:39.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.721+0000 7f8f53c3a700 1 --2- 172.21.15.7:0/3586837536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f4c0aafc0 0x7f8f4c063580 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.721+0000 7f8f53c3a700 1 -- 172.21.15.7:0/3586837536 >> 172.21.15.7:0/3586837536 conn(0x7f8f4c0cdc30 msgr2=0x7f8f4c0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:39.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.721+0000 7f8f53c3a700 1 -- 172.21.15.7:0/3586837536 shutdown_connections 2024-01-27T02:37:39.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.721+0000 7f8f53c3a700 1 -- 172.21.15.7:0/3586837536 wait complete. 2024-01-27T02:37:39.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.722+0000 7f8f53c3a700 1 Processor -- start 2024-01-27T02:37:39.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.722+0000 7f8f53c3a700 1 -- start start 2024-01-27T02:37:39.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.723+0000 7f8f53c3a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f4c0aafc0 0x7f8f4c15a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:39.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.723+0000 7f8f53c3a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f4c159f40 0x7f8f4c144390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:39.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.723+0000 7f8f53c3a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f4c15a310 0x7f8f4c0b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:39.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.723+0000 7f8f53c3a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c1585c0 con 0x7f8f4c15a310 2024-01-27T02:37:39.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.723+0000 7f8f53c3a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c15acc0 con 0x7f8f4c159f40 2024-01-27T02:37:39.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.723+0000 7f8f53c3a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c15d990 con 0x7f8f4c0aafc0 2024-01-27T02:37:39.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.723+0000 7f8f521d7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f4c15a310 0x7f8f4c0b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:39.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.723+0000 7f8f521d7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f4c15a310 0x7f8f4c0b0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35564/0 (socket says 172.21.15.7:35564) 2024-01-27T02:37:39.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.723+0000 7f8f521d7700 1 -- 172.21.15.7:0/76144598 learned_addr learned my addr 172.21.15.7:0/76144598 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:39.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.723+0000 7f8f519d6700 1 --2- 172.21.15.7:0/76144598 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f4c0aafc0 0x7f8f4c15a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:39.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.723+0000 7f8f511d5700 1 --2- 172.21.15.7:0/76144598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f4c159f40 0x7f8f4c144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:39.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.724+0000 7f8f521d7700 1 -- 172.21.15.7:0/76144598 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f4c0aafc0 msgr2=0x7f8f4c15a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:39.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.724+0000 7f8f521d7700 1 --2- 172.21.15.7:0/76144598 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f4c0aafc0 0x7f8f4c15a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.724+0000 7f8f521d7700 1 -- 172.21.15.7:0/76144598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f4c159f40 msgr2=0x7f8f4c144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:39.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.724+0000 7f8f521d7700 1 --2- 172.21.15.7:0/76144598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f4c159f40 0x7f8f4c144390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.724+0000 7f8f521d7700 1 -- 172.21.15.7:0/76144598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f44009a20 con 0x7f8f4c15a310 2024-01-27T02:37:39.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.724+0000 7f8f521d7700 1 --2- 172.21.15.7:0/76144598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f4c15a310 0x7f8f4c0b0be0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c06dcc0 tx=0x7f8f44017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:39.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.724+0000 7f8f3dffb700 1 -- 172.21.15.7:0/76144598 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44012c00 con 0x7f8f4c15a310 2024-01-27T02:37:39.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.724+0000 7f8f3dffb700 1 -- 172.21.15.7:0/76144598 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f44012d60 con 0x7f8f4c15a310 2024-01-27T02:37:39.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.724+0000 7f8f53c3a700 1 -- 172.21.15.7:0/76144598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4c1448d0 con 0x7f8f4c15a310 2024-01-27T02:37:39.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.724+0000 7f8f3dffb700 1 -- 172.21.15.7:0/76144598 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4401e680 con 0x7f8f4c15a310 2024-01-27T02:37:39.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.724+0000 7f8f53c3a700 1 -- 172.21.15.7:0/76144598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4c15d060 con 0x7f8f4c15a310 2024-01-27T02:37:39.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.725+0000 7f8f53c3a700 1 -- 172.21.15.7:0/76144598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4c04c710 con 0x7f8f4c15a310 2024-01-27T02:37:39.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.726+0000 7f8f3dffb700 1 -- 172.21.15.7:0/76144598 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8f4401e9f0 con 0x7f8f4c15a310 2024-01-27T02:37:39.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.726+0000 7f8f3dffb700 1 --2- 172.21.15.7:0/76144598 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f30066f30 0x7f8f300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:39.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.726+0000 7f8f3dffb700 1 -- 172.21.15.7:0/76144598 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8f44086810 con 0x7f8f4c15a310 2024-01-27T02:37:39.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.726+0000 7f8f519d6700 1 --2- 172.21.15.7:0/76144598 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f30066f30 0x7f8f300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:39.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.727+0000 7f8f519d6700 1 --2- 172.21.15.7:0/76144598 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f30066f30 0x7f8f300693e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8f480015a0 tx=0x7f8f48000bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:39.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.731+0000 7f8f3dffb700 1 -- 172.21.15.7:0/76144598 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f440539e0 con 0x7f8f4c15a310 2024-01-27T02:37:39.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.983+0000 7f8f53c3a700 1 -- 172.21.15.7:0/76144598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f4c15dad0 con 0x7f8f4c15a310 2024-01-27T02:37:39.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.983+0000 7f8f3dffb700 1 -- 172.21.15.7:0/76144598 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4584 (secure 0 0 0) 0x7f8f44057000 con 0x7f8f4c15a310 2024-01-27T02:37:39.985 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:39.985 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/450556982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":450556982},{"type":"v1","addr":"172.21.15.188:6836","nonce":450556982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1711422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1711422},{"type":"v1","addr":"172.21.15.188:6837","nonce":1711422}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6294,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1110306600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1110306600},{"type":"v1","addr":"172.21.15.7:6833","nonce":1110306600}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:37:03.404355+0000","modified":"2024-01-27T02:37:06.988798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6297":{"gid":6297,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/328992853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":328992853},{"type":"v1","addr":"172.21.15.7:6835","nonce":328992853}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-27T02:37:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.985+0000 7f8f53c3a700 1 -- 172.21.15.7:0/76144598 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f30066f30 msgr2=0x7f8f300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.985+0000 7f8f53c3a700 1 --2- 172.21.15.7:0/76144598 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f30066f30 0x7f8f300693e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8f480015a0 tx=0x7f8f48000bc0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.985+0000 7f8f53c3a700 1 -- 172.21.15.7:0/76144598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f4c15a310 msgr2=0x7f8f4c0b0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.985+0000 7f8f53c3a700 1 --2- 172.21.15.7:0/76144598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f4c15a310 0x7f8f4c0b0be0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c06dcc0 tx=0x7f8f44017000 comp rx=0 tx=0).stop 2024-01-27T02:37:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.985+0000 7f8f53c3a700 1 -- 172.21.15.7:0/76144598 shutdown_connections 2024-01-27T02:37:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.985+0000 7f8f53c3a700 1 --2- 172.21.15.7:0/76144598 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f30066f30 0x7f8f300693e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.985+0000 7f8f53c3a700 1 --2- 172.21.15.7:0/76144598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f4c15a310 0x7f8f4c0b0be0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.985+0000 7f8f53c3a700 1 --2- 172.21.15.7:0/76144598 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f4c0aafc0 0x7f8f4c15a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.985+0000 7f8f53c3a700 1 --2- 172.21.15.7:0/76144598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f4c159f40 0x7f8f4c144390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.985+0000 7f8f53c3a700 1 -- 172.21.15.7:0/76144598 >> 172.21.15.7:0/76144598 conn(0x7f8f4c0cdc30 msgr2=0x7f8f4c06a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.985+0000 7f8f53c3a700 1 -- 172.21.15.7:0/76144598 shutdown_connections 2024-01-27T02:37:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:39.985+0000 7f8f53c3a700 1 -- 172.21.15.7:0/76144598 wait complete. 2024-01-27T02:37:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 49 2024-01-27T02:37:40.000 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:37:40.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.130+0000 7fac9d10c700 1 Processor -- start 2024-01-27T02:37:40.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.130+0000 7fac9d10c700 1 -- start start 2024-01-27T02:37:40.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.131+0000 7fac9d10c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fac980c3ae0 0x7fac980cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:40.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.131+0000 7fac9d10c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac980ca020 con 0x7fac98157ca0 2024-01-27T02:37:40.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.131+0000 7fac9d10c700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac980c5210 con 0x7fac98158070 2024-01-27T02:37:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.131+0000 7fac9d10c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac980c54f0 con 0x7fac980c3ae0 2024-01-27T02:37:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.131+0000 7fac9659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fac980c3ae0 0x7fac980cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.132+0000 7fac96d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fac98157ca0 0x7fac980c4a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59582/0 (socket says 172.21.15.7:59582) 2024-01-27T02:37:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.132+0000 7fac96d9d700 1 -- 172.21.15.7:0/2204991142 learned_addr learned my addr 172.21.15.7:0/2204991142 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.132+0000 7fac9659c700 1 -- 172.21.15.7:0/2204991142 >> v1:172.21.15.188:6789/0 conn(0x7fac98158070 legacy=0x7fac980ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.132+0000 7fac9659c700 1 -- 172.21.15.7:0/2204991142 >> v1:172.21.15.7:6789/0 conn(0x7fac98157ca0 legacy=0x7fac980c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.132+0000 7fac9659c700 1 -- 172.21.15.7:0/2204991142 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac980c57d0 con 0x7fac980c3ae0 2024-01-27T02:37:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.132+0000 7fac95d9b700 1 -- 172.21.15.7:0/2204991142 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957436549 0 0) 0x7fac980ca020 con 0x7fac98157ca0 2024-01-27T02:37:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.132+0000 7fac9659c700 1 --2- 172.21.15.7:0/2204991142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac980c3ae0 0x7fac980cd4e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fac8c0098e0 tx=0x7fac8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45275a5fa0a04ba3 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.132+0000 7fac95d9b700 1 -- 172.21.15.7:0/2204991142 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c009e80 con 0x7fac980c3ae0 2024-01-27T02:37:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.132+0000 7fac95d9b700 1 -- 172.21.15.7:0/2204991142 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8c00b860 con 0x7fac980c3ae0 2024-01-27T02:37:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.132+0000 7fac95d9b700 1 -- 172.21.15.7:0/2204991142 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c00bbe0 con 0x7fac980c3ae0 2024-01-27T02:37:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.133+0000 7fac9d10c700 1 -- 172.21.15.7:0/2204991142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac980c3ae0 msgr2=0x7fac980cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.133+0000 7fac9d10c700 1 --2- 172.21.15.7:0/2204991142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac980c3ae0 0x7fac980cd4e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fac8c0098e0 tx=0x7fac8c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.133+0000 7fac9d10c700 1 -- 172.21.15.7:0/2204991142 shutdown_connections 2024-01-27T02:37:40.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.133+0000 7fac9d10c700 1 --2- 172.21.15.7:0/2204991142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac980c3ae0 0x7fac980cd4e0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.133+0000 7fac9d10c700 1 -- 172.21.15.7:0/2204991142 >> 172.21.15.7:0/2204991142 conn(0x7fac98098020 msgr2=0x7fac980711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:40.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.133+0000 7fac9d10c700 1 -- 172.21.15.7:0/2204991142 shutdown_connections 2024-01-27T02:37:40.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.133+0000 7fac9d10c700 1 -- 172.21.15.7:0/2204991142 wait complete. 2024-01-27T02:37:40.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.134+0000 7fac9d10c700 1 Processor -- start 2024-01-27T02:37:40.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.134+0000 7fac9d10c700 1 -- start start 2024-01-27T02:37:40.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.135+0000 7fac9d10c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac980c3ae0 0x7fac980b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:40.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.135+0000 7fac9d10c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac98157ca0 0x7fac9815fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:40.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.135+0000 7fac9d10c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac98158070 0x7fac980c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:40.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.135+0000 7fac9d10c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac980c5f20 con 0x7fac98157ca0 2024-01-27T02:37:40.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.135+0000 7fac9d10c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9814dc70 con 0x7fac98158070 2024-01-27T02:37:40.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.135+0000 7fac9d10c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac981603e0 con 0x7fac980c3ae0 2024-01-27T02:37:40.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.135+0000 7fac9659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac98157ca0 0x7fac9815fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:40.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.135+0000 7fac9659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac98157ca0 0x7fac9815fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35588/0 (socket says 172.21.15.7:35588) 2024-01-27T02:37:40.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.135+0000 7fac9659c700 1 -- 172.21.15.7:0/700213335 learned_addr learned my addr 172.21.15.7:0/700213335 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:40.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.135+0000 7fac9759e700 1 --2- 172.21.15.7:0/700213335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac98158070 0x7fac980c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:40.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.135+0000 7fac96d9d700 1 --2- 172.21.15.7:0/700213335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac980c3ae0 0x7fac980b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:40.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.136+0000 7fac9659c700 1 -- 172.21.15.7:0/700213335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac980c3ae0 msgr2=0x7fac980b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:40.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.136+0000 7fac9659c700 1 --2- 172.21.15.7:0/700213335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac980c3ae0 0x7fac980b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.136+0000 7fac9659c700 1 -- 172.21.15.7:0/700213335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac98158070 msgr2=0x7fac980c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:40.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.136+0000 7fac9659c700 1 --2- 172.21.15.7:0/700213335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac98158070 0x7fac980c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.136+0000 7fac9659c700 1 -- 172.21.15.7:0/700213335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac8c009420 con 0x7fac98157ca0 2024-01-27T02:37:40.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.136+0000 7fac9659c700 1 --2- 172.21.15.7:0/700213335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac98157ca0 0x7fac9815fea0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fac9806ea80 tx=0x7fac8c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:40.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.136+0000 7fac7effd700 1 -- 172.21.15.7:0/700213335 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c01f070 con 0x7fac98157ca0 2024-01-27T02:37:40.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.136+0000 7fac7effd700 1 -- 172.21.15.7:0/700213335 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8c000c90 con 0x7fac98157ca0 2024-01-27T02:37:40.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.136+0000 7fac9d10c700 1 -- 172.21.15.7:0/700213335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac980b2ad0 con 0x7fac98157ca0 2024-01-27T02:37:40.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.136+0000 7fac7effd700 1 -- 172.21.15.7:0/700213335 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c025420 con 0x7fac98157ca0 2024-01-27T02:37:40.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.136+0000 7fac9d10c700 1 -- 172.21.15.7:0/700213335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac980b3090 con 0x7fac98157ca0 2024-01-27T02:37:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.137+0000 7fac9d10c700 1 -- 172.21.15.7:0/700213335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac980c6090 con 0x7fac98157ca0 2024-01-27T02:37:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.137+0000 7fac7effd700 1 -- 172.21.15.7:0/700213335 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fac8c023070 con 0x7fac98157ca0 2024-01-27T02:37:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.138+0000 7fac7effd700 1 --2- 172.21.15.7:0/700213335 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fac84066f30 0x7fac840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.138+0000 7fac7effd700 1 -- 172.21.15.7:0/700213335 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fac8c085a60 con 0x7fac98157ca0 2024-01-27T02:37:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.138+0000 7fac96d9d700 1 --2- 172.21.15.7:0/700213335 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fac84066f30 0x7fac840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.139+0000 7fac96d9d700 1 --2- 172.21.15.7:0/700213335 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fac84066f30 0x7fac840693e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fac88007fd0 tx=0x7fac88007d10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.143+0000 7fac7effd700 1 -- 172.21.15.7:0/700213335 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac8c052cd0 con 0x7fac98157ca0 2024-01-27T02:37:40.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.346+0000 7fac9d10c700 1 -- 172.21.15.7:0/700213335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac9804c710 con 0x7fac98157ca0 2024-01-27T02:37:40.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.347+0000 7fac7effd700 1 -- 172.21.15.7:0/700213335 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+15309 (secure 0 0 0) 0x7fac8c015090 con 0x7fac98157ca0 2024-01-27T02:37:40.349 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:40.349 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":106,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:37:32.458337+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:37:01.445734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-27T02:37:02.377151+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:37:40.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.349+0000 7fac9d10c700 1 -- 172.21.15.7:0/700213335 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fac84066f30 msgr2=0x7fac840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:40.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.349+0000 7fac9d10c700 1 --2- 172.21.15.7:0/700213335 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fac84066f30 0x7fac840693e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fac88007fd0 tx=0x7fac88007d10 comp rx=0 tx=0).stop 2024-01-27T02:37:40.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.349+0000 7fac9d10c700 1 -- 172.21.15.7:0/700213335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac98157ca0 msgr2=0x7fac9815fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:40.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.349+0000 7fac9d10c700 1 --2- 172.21.15.7:0/700213335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac98157ca0 0x7fac9815fea0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fac9806ea80 tx=0x7fac8c017000 comp rx=0 tx=0).stop 2024-01-27T02:37:40.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.350+0000 7fac9d10c700 1 -- 172.21.15.7:0/700213335 shutdown_connections 2024-01-27T02:37:40.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.350+0000 7fac9d10c700 1 --2- 172.21.15.7:0/700213335 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fac84066f30 0x7fac840693e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.350+0000 7fac9d10c700 1 --2- 172.21.15.7:0/700213335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac98158070 0x7fac980c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.350+0000 7fac9d10c700 1 --2- 172.21.15.7:0/700213335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac980c3ae0 0x7fac980b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.350+0000 7fac9d10c700 1 --2- 172.21.15.7:0/700213335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac98157ca0 0x7fac9815fea0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.350+0000 7fac9d10c700 1 -- 172.21.15.7:0/700213335 >> 172.21.15.7:0/700213335 conn(0x7fac98098020 msgr2=0x7fac9806bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:40.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.350+0000 7fac9d10c700 1 -- 172.21.15.7:0/700213335 shutdown_connections 2024-01-27T02:37:40.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.350+0000 7fac9d10c700 1 -- 172.21.15.7:0/700213335 wait complete. 2024-01-27T02:37:40.365 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-27T02:37:40.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.495+0000 7f0794eb2700 1 Processor -- start 2024-01-27T02:37:40.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.495+0000 7f0794eb2700 1 -- start start 2024-01-27T02:37:40.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.496+0000 7f0794eb2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07900b1c00 0x7f079006e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:40.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.496+0000 7f0794eb2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0790157610 con 0x7f07900b1fd0 2024-01-27T02:37:40.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.496+0000 7f0794eb2700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079006ee00 con 0x7f079015b5f0 2024-01-27T02:37:40.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.496+0000 7f0794eb2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079006f0e0 con 0x7f07900b1c00 2024-01-27T02:37:40.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.496+0000 7f078ed9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f07900b1fd0 0x7f0790156dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59594/0 (socket says 172.21.15.7:59594) 2024-01-27T02:37:40.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.496+0000 7f078ed9d700 1 -- 172.21.15.7:0/2923859208 learned_addr learned my addr 172.21.15.7:0/2923859208 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:40.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.497+0000 7f078e59c700 1 --2- 172.21.15.7:0/2923859208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07900b1c00 0x7f079006e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:40.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.497+0000 7f078dd9b700 1 -- 172.21.15.7:0/2923859208 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 207542317 0 0) 0x7f0790157610 con 0x7f07900b1fd0 2024-01-27T02:37:40.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.497+0000 7f078dd9b700 1 -- 172.21.15.7:0/2923859208 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f077c003570 con 0x7f07900b1fd0 2024-01-27T02:37:40.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.497+0000 7f078dd9b700 1 -- 172.21.15.7:0/2923859208 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 847030999 0 0) 0x7f077c003570 con 0x7f07900b1fd0 2024-01-27T02:37:40.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.497+0000 7f078dd9b700 1 -- 172.21.15.7:0/2923859208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07900b1c00 msgr2=0x7f079006e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:40.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.497+0000 7f078dd9b700 1 --2- 172.21.15.7:0/2923859208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07900b1c00 0x7f079006e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.497+0000 7f078dd9b700 1 -- 172.21.15.7:0/2923859208 >> v1:172.21.15.188:6789/0 conn(0x7f079015b5f0 legacy=0x7f07901566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:40.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.497+0000 7f078dd9b700 1 -- 172.21.15.7:0/2923859208 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079006f3c0 con 0x7f07900b1fd0 2024-01-27T02:37:40.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.497+0000 7f078dd9b700 1 -- 172.21.15.7:0/2923859208 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f07880017f0 con 0x7f07900b1fd0 2024-01-27T02:37:40.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.497+0000 7f078dd9b700 1 -- 172.21.15.7:0/2923859208 >> v1:172.21.15.7:6789/0 conn(0x7f07900b1fd0 legacy=0x7f0790156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:40.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.497+0000 7f078dd9b700 1 --2- 172.21.15.7:0/2923859208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07900b1c00 0x7f077c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:40.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.497+0000 7f078dd9b700 1 --2- 172.21.15.7:0/2923859208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f077c004060 0x7f077c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:40.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.497+0000 7f078dd9b700 1 --2- 172.21.15.7:0/2923859208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f077c004d50 0x7f077c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:40.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.497+0000 7f078dd9b700 1 -- 172.21.15.7:0/2923859208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079006f0e0 con 0x7f077c004d50 2024-01-27T02:37:40.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.497+0000 7f078dd9b700 1 -- 172.21.15.7:0/2923859208 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c00bc20 con 0x7f077c004060 2024-01-27T02:37:40.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.497+0000 7f078dd9b700 1 -- 172.21.15.7:0/2923859208 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077c00bde0 con 0x7f07900b1c00 2024-01-27T02:37:40.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.497+0000 7f078ed9d700 1 --2- 172.21.15.7:0/2923859208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f077c004d50 0x7f077c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:40.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.497+0000 7f078e59c700 1 --2- 172.21.15.7:0/2923859208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07900b1c00 0x7f077c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:40.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.497+0000 7f0785d9b700 1 --2- 172.21.15.7:0/2923859208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f077c004060 0x7f077c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:40.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.498+0000 7f078ed9d700 1 -- 172.21.15.7:0/2923859208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07900b1c00 msgr2=0x7f077c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:40.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.498+0000 7f078ed9d700 1 --2- 172.21.15.7:0/2923859208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07900b1c00 0x7f077c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.498+0000 7f078ed9d700 1 -- 172.21.15.7:0/2923859208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f077c004060 msgr2=0x7f077c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.498+0000 7f078ed9d700 1 --2- 172.21.15.7:0/2923859208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f077c004060 0x7f077c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.498+0000 7f078ed9d700 1 -- 172.21.15.7:0/2923859208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c003570 con 0x7f077c004d50 2024-01-27T02:37:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.498+0000 7f078e59c700 1 --2- 172.21.15.7:0/2923859208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07900b1c00 0x7f077c003a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:37:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.498+0000 7f078ed9d700 1 --2- 172.21.15.7:0/2923859208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f077c004d50 0x7f077c0051c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f078800a860 tx=0x7f0788002100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08562bb0817a16f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.498+0000 7f078dd9b700 1 -- 172.21.15.7:0/2923859208 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07880070f0 con 0x7f077c004d50 2024-01-27T02:37:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.498+0000 7f078dd9b700 1 -- 172.21.15.7:0/2923859208 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0788007270 con 0x7f077c004d50 2024-01-27T02:37:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.498+0000 7f078dd9b700 1 -- 172.21.15.7:0/2923859208 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788007630 con 0x7f077c004d50 2024-01-27T02:37:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.498+0000 7f0794eb2700 1 -- 172.21.15.7:0/2923859208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f077c004d50 msgr2=0x7f077c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.498+0000 7f0794eb2700 1 --2- 172.21.15.7:0/2923859208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f077c004d50 0x7f077c0051c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f078800a860 tx=0x7f0788002100 comp rx=0 tx=0).stop 2024-01-27T02:37:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.498+0000 7f078ed9d700 1 -- 172.21.15.7:0/2923859208 reap_dead start 2024-01-27T02:37:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.498+0000 7f0794eb2700 1 -- 172.21.15.7:0/2923859208 shutdown_connections 2024-01-27T02:37:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.498+0000 7f0794eb2700 1 -- 172.21.15.7:0/2923859208 >> 172.21.15.7:0/2923859208 conn(0x7f0790068430 msgr2=0x7f079006be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.499+0000 7f0794eb2700 1 -- 172.21.15.7:0/2923859208 shutdown_connections 2024-01-27T02:37:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.499+0000 7f0794eb2700 1 -- 172.21.15.7:0/2923859208 wait complete. 2024-01-27T02:37:40.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.500+0000 7f0794eb2700 1 Processor -- start 2024-01-27T02:37:40.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.500+0000 7f0794eb2700 1 -- start start 2024-01-27T02:37:40.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.501+0000 7f0794eb2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f077c004d50 0x7f079014b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:40.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.501+0000 7f0794eb2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f079016a3f0 0x7f0790063580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:40.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.501+0000 7f0794eb2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f079016a860 0x7f0790063ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:40.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.501+0000 7f0794eb2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07901505a0 con 0x7f077c004d50 2024-01-27T02:37:40.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.501+0000 7f0794eb2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079014d320 con 0x7f079016a860 2024-01-27T02:37:40.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.501+0000 7f0794eb2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079014fc70 con 0x7f079016a3f0 2024-01-27T02:37:40.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.502+0000 7f078e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f077c004d50 0x7f079014b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:40.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.502+0000 7f078e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f077c004d50 0x7f079014b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35600/0 (socket says 172.21.15.7:35600) 2024-01-27T02:37:40.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.502+0000 7f078e59c700 1 -- 172.21.15.7:0/1389290299 learned_addr learned my addr 172.21.15.7:0/1389290299 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:40.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.502+0000 7f0785d9b700 1 --2- 172.21.15.7:0/1389290299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f079016a3f0 0x7f0790063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:40.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.502+0000 7f078ed9d700 1 --2- 172.21.15.7:0/1389290299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f079016a860 0x7f0790063ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:40.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.502+0000 7f078e59c700 1 -- 172.21.15.7:0/1389290299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f079016a3f0 msgr2=0x7f0790063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:40.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.502+0000 7f078e59c700 1 --2- 172.21.15.7:0/1389290299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f079016a3f0 0x7f0790063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.502+0000 7f078e59c700 1 -- 172.21.15.7:0/1389290299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f079016a860 msgr2=0x7f0790063ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.502+0000 7f078e59c700 1 --2- 172.21.15.7:0/1389290299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f079016a860 0x7f0790063ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.502+0000 7f078e59c700 1 -- 172.21.15.7:0/1389290299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079014ff50 con 0x7f077c004d50 2024-01-27T02:37:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.502+0000 7f0785d9b700 1 --2- 172.21.15.7:0/1389290299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f079016a3f0 0x7f0790063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.502+0000 7f078e59c700 1 --2- 172.21.15.7:0/1389290299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f077c004d50 0x7f079014b860 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f0780011d70 tx=0x7f07800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.502+0000 7f0786ffd700 1 -- 172.21.15.7:0/1389290299 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078000fcd0 con 0x7f077c004d50 2024-01-27T02:37:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.502+0000 7f0786ffd700 1 -- 172.21.15.7:0/1389290299 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078000fe30 con 0x7f077c004d50 2024-01-27T02:37:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.502+0000 7f0786ffd700 1 -- 172.21.15.7:0/1389290299 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780014670 con 0x7f077c004d50 2024-01-27T02:37:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.502+0000 7f0794eb2700 1 -- 172.21.15.7:0/1389290299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0790150230 con 0x7f077c004d50 2024-01-27T02:37:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.502+0000 7f0794eb2700 1 -- 172.21.15.7:0/1389290299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079014c5b0 con 0x7f077c004d50 2024-01-27T02:37:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.504+0000 7f0794eb2700 1 -- 172.21.15.7:0/1389290299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f079004c710 con 0x7f077c004d50 2024-01-27T02:37:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.504+0000 7f0786ffd700 1 -- 172.21.15.7:0/1389290299 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0780014aa0 con 0x7f077c004d50 2024-01-27T02:37:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.504+0000 7f0786ffd700 1 --2- 172.21.15.7:0/1389290299 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0770066f30 0x7f07700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.504+0000 7f0786ffd700 1 -- 172.21.15.7:0/1389290299 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0780018070 con 0x7f077c004d50 2024-01-27T02:37:40.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.504+0000 7f0785d9b700 1 --2- 172.21.15.7:0/1389290299 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0770066f30 0x7f07700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:40.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.505+0000 7f0785d9b700 1 --2- 172.21.15.7:0/1389290299 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0770066f30 0x7f07700693e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0778007bc0 tx=0x7f0778009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:40.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.509+0000 7f0786ffd700 1 -- 172.21.15.7:0/1389290299 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0780052420 con 0x7f077c004d50 2024-01-27T02:37:40.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.757+0000 7f0794eb2700 1 -- 172.21.15.7:0/1389290299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0790158ab0 con 0x7f077c004d50 2024-01-27T02:37:40.760 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-27T02:37:40.758+0000 7fb5c503a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-27T02:37:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.763+0000 7f0786ffd700 1 -- 172.21.15.7:0/1389290299 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0780055a40 con 0x7f077c004d50 2024-01-27T02:37:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.766+0000 7f0794eb2700 1 -- 172.21.15.7:0/1389290299 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0770066f30 msgr2=0x7f07700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.766+0000 7f0794eb2700 1 --2- 172.21.15.7:0/1389290299 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0770066f30 0x7f07700693e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0778007bc0 tx=0x7f0778009040 comp rx=0 tx=0).stop 2024-01-27T02:37:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.766+0000 7f0794eb2700 1 -- 172.21.15.7:0/1389290299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f077c004d50 msgr2=0x7f079014b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.766+0000 7f0794eb2700 1 --2- 172.21.15.7:0/1389290299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f077c004d50 0x7f079014b860 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f0780011d70 tx=0x7f07800091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.766+0000 7f0794eb2700 1 -- 172.21.15.7:0/1389290299 shutdown_connections 2024-01-27T02:37:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.766+0000 7f0794eb2700 1 --2- 172.21.15.7:0/1389290299 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0770066f30 0x7f07700693e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.766+0000 7f0794eb2700 1 --2- 172.21.15.7:0/1389290299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f079016a860 0x7f0790063ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.766+0000 7f0794eb2700 1 --2- 172.21.15.7:0/1389290299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f077c004d50 0x7f079014b860 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.766+0000 7f0794eb2700 1 --2- 172.21.15.7:0/1389290299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f079016a3f0 0x7f0790063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.766+0000 7f0794eb2700 1 -- 172.21.15.7:0/1389290299 >> 172.21.15.7:0/1389290299 conn(0x7f0790068430 msgr2=0x7f079014d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.766+0000 7f0794eb2700 1 -- 172.21.15.7:0/1389290299 shutdown_connections 2024-01-27T02:37:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.766+0000 7f0794eb2700 1 -- 172.21.15.7:0/1389290299 wait complete. 2024-01-27T02:37:40.769 INFO:tasks.ceph.mds.c.smithi007.stderr: -9195> 2024-01-27T02:37:08.954+0000 7fdcc1d3a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-27T02:37:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-27T02:37:40.782 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-27T02:37:40.801 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-27T02:37:40.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.914+0000 7f5923cc4700 1 Processor -- start 2024-01-27T02:37:40.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.914+0000 7f5923cc4700 1 -- start start 2024-01-27T02:37:40.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.915+0000 7f5923cc4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f591c141a90 0x7f591c0cab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:40.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.915+0000 7f5923cc4700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f591c0cbdf0 0x7f591c0cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:40.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.915+0000 7f5923cc4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f591c15c7d0 con 0x7f591c1416c0 2024-01-27T02:37:40.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.915+0000 7f5923cc4700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591c0cd9c0 con 0x7f591c0cbdf0 2024-01-27T02:37:40.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.915+0000 7f5923cc4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591c0c9640 con 0x7f591c141a90 2024-01-27T02:37:40.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.915+0000 7f5921a60700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f591c1416c0 0x7f591c0cb6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59612/0 (socket says 172.21.15.7:59612) 2024-01-27T02:37:40.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.915+0000 7f5921a60700 1 -- 172.21.15.7:0/3342011565 learned_addr learned my addr 172.21.15.7:0/3342011565 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:40.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.916+0000 7f592125f700 1 --2- 172.21.15.7:0/3342011565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f591c141a90 0x7f591c0cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:40.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.916+0000 7f5922261700 1 --2- 172.21.15.7:0/3342011565 >> v2:172.21.15.188:3300/0 conn(0x7f591c0cbdf0 0x7f591c0cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:40.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.916+0000 7f5920a5e700 1 -- 172.21.15.7:0/3342011565 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3339607159 0 0) 0x7f591c15c7d0 con 0x7f591c1416c0 2024-01-27T02:37:40.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.916+0000 7f5920a5e700 1 -- 172.21.15.7:0/3342011565 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5908003570 con 0x7f591c1416c0 2024-01-27T02:37:40.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.916+0000 7f592125f700 1 -- 172.21.15.7:0/3342011565 >> v2:172.21.15.188:3300/0 conn(0x7f591c0cbdf0 msgr2=0x7f591c0cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:40.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.916+0000 7f592125f700 1 --2- 172.21.15.7:0/3342011565 >> v2:172.21.15.188:3300/0 conn(0x7f591c0cbdf0 0x7f591c0cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.916+0000 7f592125f700 1 -- 172.21.15.7:0/3342011565 >> v1:172.21.15.7:6789/0 conn(0x7f591c1416c0 legacy=0x7f591c0cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:40.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.916+0000 7f592125f700 1 -- 172.21.15.7:0/3342011565 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c15bbc0 con 0x7f591c141a90 2024-01-27T02:37:40.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.916+0000 7f5922261700 1 --2- 172.21.15.7:0/3342011565 >> v2:172.21.15.188:3300/0 conn(0x7f591c0cbdf0 0x7f591c0cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:37:40.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.916+0000 7f592125f700 1 --2- 172.21.15.7:0/3342011565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f591c141a90 0x7f591c0cab00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f59180098e0 tx=0x7f59180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a17da7567af65558 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:40.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.916+0000 7f5920a5e700 1 -- 172.21.15.7:0/3342011565 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5918011630 con 0x7f591c141a90 2024-01-27T02:37:40.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.916+0000 7f5920a5e700 1 -- 172.21.15.7:0/3342011565 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5918011790 con 0x7f591c141a90 2024-01-27T02:37:40.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.916+0000 7f5920a5e700 1 -- 172.21.15.7:0/3342011565 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5918011b30 con 0x7f591c141a90 2024-01-27T02:37:40.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.917+0000 7f5923cc4700 1 -- 172.21.15.7:0/3342011565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f591c141a90 msgr2=0x7f591c0cab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:40.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.917+0000 7f5923cc4700 1 --2- 172.21.15.7:0/3342011565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f591c141a90 0x7f591c0cab00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f59180098e0 tx=0x7f59180091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.917+0000 7f5923cc4700 1 -- 172.21.15.7:0/3342011565 shutdown_connections 2024-01-27T02:37:40.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.917+0000 7f5923cc4700 1 --2- 172.21.15.7:0/3342011565 >> v2:172.21.15.188:3300/0 conn(0x7f591c0cbdf0 0x7f591c0cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.917+0000 7f5923cc4700 1 --2- 172.21.15.7:0/3342011565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f591c141a90 0x7f591c0cab00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.917+0000 7f5923cc4700 1 -- 172.21.15.7:0/3342011565 >> 172.21.15.7:0/3342011565 conn(0x7f591c063580 msgr2=0x7f591c063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:40.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.917+0000 7f5923cc4700 1 -- 172.21.15.7:0/3342011565 shutdown_connections 2024-01-27T02:37:40.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.917+0000 7f5923cc4700 1 -- 172.21.15.7:0/3342011565 wait complete. 2024-01-27T02:37:40.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.918+0000 7f5923cc4700 1 Processor -- start 2024-01-27T02:37:40.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.918+0000 7f5923cc4700 1 -- start start 2024-01-27T02:37:40.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.919+0000 7f5923cc4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f591c0cbdf0 0x7f591c157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:40.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.919+0000 7f5923cc4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f591c1416c0 0x7f591c158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:40.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.919+0000 7f5923cc4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f591c141a90 0x7f591c13ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:40.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.919+0000 7f5923cc4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c168270 con 0x7f591c0cbdf0 2024-01-27T02:37:40.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.919+0000 7f5923cc4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c159860 con 0x7f591c1416c0 2024-01-27T02:37:40.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.919+0000 7f5923cc4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591c159b40 con 0x7f591c141a90 2024-01-27T02:37:40.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.919+0000 7f5921a60700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f591c0cbdf0 0x7f591c157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:40.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.919+0000 7f5921a60700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f591c0cbdf0 0x7f591c157800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35624/0 (socket says 172.21.15.7:35624) 2024-01-27T02:37:40.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.919+0000 7f5921a60700 1 -- 172.21.15.7:0/610075122 learned_addr learned my addr 172.21.15.7:0/610075122 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:40.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.919+0000 7f592125f700 1 --2- 172.21.15.7:0/610075122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f591c1416c0 0x7f591c158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:40.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.919+0000 7f5922261700 1 --2- 172.21.15.7:0/610075122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f591c141a90 0x7f591c13ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:40.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.920+0000 7f5921a60700 1 -- 172.21.15.7:0/610075122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f591c141a90 msgr2=0x7f591c13ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:40.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.920+0000 7f5921a60700 1 --2- 172.21.15.7:0/610075122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f591c141a90 0x7f591c13ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.920+0000 7f5921a60700 1 -- 172.21.15.7:0/610075122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f591c1416c0 msgr2=0x7f591c158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:40.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.920+0000 7f5921a60700 1 --2- 172.21.15.7:0/610075122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f591c1416c0 0x7f591c158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:40.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.920+0000 7f5921a60700 1 -- 172.21.15.7:0/610075122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5914009a20 con 0x7f591c0cbdf0 2024-01-27T02:37:40.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.920+0000 7f592125f700 1 --2- 172.21.15.7:0/610075122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f591c1416c0 0x7f591c158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:37:40.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.920+0000 7f5921a60700 1 --2- 172.21.15.7:0/610075122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f591c0cbdf0 0x7f591c157800 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f590c00c790 tx=0x7f590c007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:40.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.920+0000 7f5911ffb700 1 -- 172.21.15.7:0/610075122 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c007300 con 0x7f591c0cbdf0 2024-01-27T02:37:40.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.920+0000 7f5911ffb700 1 -- 172.21.15.7:0/610075122 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590c007460 con 0x7f591c0cbdf0 2024-01-27T02:37:40.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.920+0000 7f5923cc4700 1 -- 172.21.15.7:0/610075122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5918009420 con 0x7f591c0cbdf0 2024-01-27T02:37:40.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.920+0000 7f5911ffb700 1 -- 172.21.15.7:0/610075122 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c0057f0 con 0x7f591c0cbdf0 2024-01-27T02:37:40.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.920+0000 7f5923cc4700 1 -- 172.21.15.7:0/610075122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591c15a100 con 0x7f591c0cbdf0 2024-01-27T02:37:40.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.921+0000 7f5923cc4700 1 -- 172.21.15.7:0/610075122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591c072060 con 0x7f591c0cbdf0 2024-01-27T02:37:40.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.922+0000 7f5911ffb700 1 -- 172.21.15.7:0/610075122 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f590c005c40 con 0x7f591c0cbdf0 2024-01-27T02:37:40.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.922+0000 7f5911ffb700 1 --2- 172.21.15.7:0/610075122 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5900066f30 0x7f59000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:40.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.922+0000 7f5911ffb700 1 -- 172.21.15.7:0/610075122 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f590c084020 con 0x7f591c0cbdf0 2024-01-27T02:37:40.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.923+0000 7f592125f700 1 --2- 172.21.15.7:0/610075122 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5900066f30 0x7f59000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:40.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.923+0000 7f592125f700 1 --2- 172.21.15.7:0/610075122 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5900066f30 0x7f59000693e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f59180091e0 tx=0x7f5918017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:40.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:40.927+0000 7f5911ffb700 1 -- 172.21.15.7:0/610075122 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f590c0512c0 con 0x7f591c0cbdf0 2024-01-27T02:37:41.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.176+0000 7f5923cc4700 1 -- 172.21.15.7:0/610075122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f591c15bea0 con 0x7f591c0cbdf0 2024-01-27T02:37:41.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.763+0000 7f5911ffb700 1 -- 172.21.15.7:0/610075122 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7f590c0548e0 con 0x7f591c0cbdf0 2024-01-27T02:37:41.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.766+0000 7f5923cc4700 1 -- 172.21.15.7:0/610075122 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5900066f30 msgr2=0x7f59000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:41.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.766+0000 7f5923cc4700 1 --2- 172.21.15.7:0/610075122 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5900066f30 0x7f59000693e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f59180091e0 tx=0x7f5918017000 comp rx=0 tx=0).stop 2024-01-27T02:37:41.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.766+0000 7f5923cc4700 1 -- 172.21.15.7:0/610075122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f591c0cbdf0 msgr2=0x7f591c157800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:41.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.766+0000 7f5923cc4700 1 --2- 172.21.15.7:0/610075122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f591c0cbdf0 0x7f591c157800 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f590c00c790 tx=0x7f590c007c80 comp rx=0 tx=0).stop 2024-01-27T02:37:41.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.766+0000 7f5923cc4700 1 -- 172.21.15.7:0/610075122 shutdown_connections 2024-01-27T02:37:41.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.766+0000 7f5923cc4700 1 --2- 172.21.15.7:0/610075122 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5900066f30 0x7f59000693e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.766+0000 7f5923cc4700 1 --2- 172.21.15.7:0/610075122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f591c141a90 0x7f591c13ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.766+0000 7f5923cc4700 1 --2- 172.21.15.7:0/610075122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f591c0cbdf0 0x7f591c157800 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.766+0000 7f5923cc4700 1 --2- 172.21.15.7:0/610075122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f591c1416c0 0x7f591c158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.766+0000 7f5923cc4700 1 -- 172.21.15.7:0/610075122 >> 172.21.15.7:0/610075122 conn(0x7f591c063580 msgr2=0x7f591c13dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.766+0000 7f5923cc4700 1 -- 172.21.15.7:0/610075122 shutdown_connections 2024-01-27T02:37:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.766+0000 7f5923cc4700 1 -- 172.21.15.7:0/610075122 wait complete. 2024-01-27T02:37:41.784 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-27T02:37:41.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.968+0000 7f7eaf2f3700 1 Processor -- start 2024-01-27T02:37:41.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.969+0000 7f7eaf2f3700 1 -- start start 2024-01-27T02:37:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.969+0000 7f7eaf2f3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ea813ad50 0x7f7ea8159070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.969+0000 7f7eaf2f3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea8063c90 0x7f7ea815bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.969+0000 7f7eaf2f3700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea8158be0 con 0x7f7ea815bb40 2024-01-27T02:37:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.969+0000 7f7eaf2f3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea80c8f70 con 0x7f7ea813ad50 2024-01-27T02:37:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.969+0000 7f7eaf2f3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea815c600 con 0x7f7ea8063c90 2024-01-27T02:37:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.970+0000 7f7ead08f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ea813ad50 0x7f7ea8159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.970+0000 7f7ead08f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ea813ad50 0x7f7ea8159070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44472/0 (socket says 172.21.15.7:44472) 2024-01-27T02:37:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.970+0000 7f7ead08f700 1 -- 172.21.15.7:0/625298990 learned_addr learned my addr 172.21.15.7:0/625298990 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.970+0000 7f7ead890700 1 --2- 172.21.15.7:0/625298990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea8063c90 0x7f7ea815bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.970+0000 7f7ead08f700 1 -- 172.21.15.7:0/625298990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea8063c90 msgr2=0x7f7ea815bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.970+0000 7f7ead08f700 1 --2- 172.21.15.7:0/625298990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea8063c90 0x7f7ea815bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.970+0000 7f7ead08f700 1 -- 172.21.15.7:0/625298990 >> v1:172.21.15.188:6789/0 conn(0x7f7ea815bb40 legacy=0x7f7ea8063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.970+0000 7f7ead08f700 1 -- 172.21.15.7:0/625298990 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea815c8e0 con 0x7f7ea813ad50 2024-01-27T02:37:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.970+0000 7f7ead08f700 1 --2- 172.21.15.7:0/625298990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea813ad50 0x7f7ea8159070 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f7e980099b0 tx=0x7f7e98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f470b824592185 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.970+0000 7f7e9ffff700 1 -- 172.21.15.7:0/625298990 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9800b820 con 0x7f7ea813ad50 2024-01-27T02:37:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.970+0000 7f7e9ffff700 1 -- 172.21.15.7:0/625298990 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9800b980 con 0x7f7ea813ad50 2024-01-27T02:37:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.970+0000 7f7e9ffff700 1 -- 172.21.15.7:0/625298990 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9800bd20 con 0x7f7ea813ad50 2024-01-27T02:37:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.971+0000 7f7eaf2f3700 1 -- 172.21.15.7:0/625298990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea813ad50 msgr2=0x7f7ea8159070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:41.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.971+0000 7f7eaf2f3700 1 --2- 172.21.15.7:0/625298990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea813ad50 0x7f7ea8159070 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f7e980099b0 tx=0x7f7e98009250 comp rx=0 tx=0).stop 2024-01-27T02:37:41.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.971+0000 7f7eaf2f3700 1 -- 172.21.15.7:0/625298990 shutdown_connections 2024-01-27T02:37:41.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.971+0000 7f7eaf2f3700 1 --2- 172.21.15.7:0/625298990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea8063c90 0x7f7ea815bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:41.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.971+0000 7f7eaf2f3700 1 --2- 172.21.15.7:0/625298990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea813ad50 0x7f7ea8159070 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:41.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.971+0000 7f7eaf2f3700 1 -- 172.21.15.7:0/625298990 >> 172.21.15.7:0/625298990 conn(0x7f7ea80cdc30 msgr2=0x7f7ea80ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:41.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.971+0000 7f7eaf2f3700 1 -- 172.21.15.7:0/625298990 shutdown_connections 2024-01-27T02:37:41.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.971+0000 7f7eaf2f3700 1 -- 172.21.15.7:0/625298990 wait complete. 2024-01-27T02:37:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.972+0000 7f7eaf2f3700 1 Processor -- start 2024-01-27T02:37:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.973+0000 7f7eaf2f3700 1 -- start start 2024-01-27T02:37:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.973+0000 7f7eaf2f3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ea8063c90 0x7f7ea81536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.973+0000 7f7eaf2f3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea813ad50 0x7f7ea813a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.973+0000 7f7eaf2f3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea815bb40 0x7f7ea813a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.973+0000 7f7eaf2f3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea80b23c0 con 0x7f7ea813ad50 2024-01-27T02:37:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.974+0000 7f7eaf2f3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea81510f0 con 0x7f7ea8063c90 2024-01-27T02:37:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.974+0000 7f7eaf2f3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea80b20e0 con 0x7f7ea815bb40 2024-01-27T02:37:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.974+0000 7f7eac88e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea813ad50 0x7f7ea813a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.974+0000 7f7eac88e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea813ad50 0x7f7ea813a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44476/0 (socket says 172.21.15.7:44476) 2024-01-27T02:37:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.974+0000 7f7eac88e700 1 -- 172.21.15.7:0/2258236990 learned_addr learned my addr 172.21.15.7:0/2258236990 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.974+0000 7f7ead890700 1 --2- 172.21.15.7:0/2258236990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea815bb40 0x7f7ea813a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.974+0000 7f7ead08f700 1 --2- 172.21.15.7:0/2258236990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ea8063c90 0x7f7ea81536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.974+0000 7f7eac88e700 1 -- 172.21.15.7:0/2258236990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea815bb40 msgr2=0x7f7ea813a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.974+0000 7f7eac88e700 1 --2- 172.21.15.7:0/2258236990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea815bb40 0x7f7ea813a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.974+0000 7f7eac88e700 1 -- 172.21.15.7:0/2258236990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ea8063c90 msgr2=0x7f7ea81536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.974+0000 7f7eac88e700 1 --2- 172.21.15.7:0/2258236990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ea8063c90 0x7f7ea81536b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.974+0000 7f7eac88e700 1 -- 172.21.15.7:0/2258236990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e980094f0 con 0x7f7ea813ad50 2024-01-27T02:37:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.974+0000 7f7eac88e700 1 --2- 172.21.15.7:0/2258236990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea813ad50 0x7f7ea813a1a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400c5d0 tx=0x7f7ea4007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.974+0000 7f7e9d7fa700 1 -- 172.21.15.7:0/2258236990 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4011590 con 0x7f7ea813ad50 2024-01-27T02:37:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.974+0000 7f7e9d7fa700 1 -- 172.21.15.7:0/2258236990 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea4011710 con 0x7f7ea813ad50 2024-01-27T02:37:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.974+0000 7f7e9d7fa700 1 -- 172.21.15.7:0/2258236990 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4011ad0 con 0x7f7ea813ad50 2024-01-27T02:37:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.975+0000 7f7eaf2f3700 1 -- 172.21.15.7:0/2258236990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea81507c0 con 0x7f7ea813ad50 2024-01-27T02:37:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.975+0000 7f7eaf2f3700 1 -- 172.21.15.7:0/2258236990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea8150d80 con 0x7f7ea813ad50 2024-01-27T02:37:41.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.976+0000 7f7eaf2f3700 1 -- 172.21.15.7:0/2258236990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea8150f40 con 0x7f7ea813ad50 2024-01-27T02:37:41.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.976+0000 7f7e9d7fa700 1 -- 172.21.15.7:0/2258236990 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7ea401c460 con 0x7f7ea813ad50 2024-01-27T02:37:41.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.976+0000 7f7e9d7fa700 1 --2- 172.21.15.7:0/2258236990 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7e8c066f30 0x7f7e8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:41.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.976+0000 7f7e9d7fa700 1 -- 172.21.15.7:0/2258236990 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f7ea4084200 con 0x7f7ea813ad50 2024-01-27T02:37:41.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.977+0000 7f7ead08f700 1 --2- 172.21.15.7:0/2258236990 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7e8c066f30 0x7f7e8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:41.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.977+0000 7f7ead08f700 1 --2- 172.21.15.7:0/2258236990 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7e8c066f30 0x7f7e8c0693e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7e980099b0 tx=0x7f7e98016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:41.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:41.981+0000 7f7e9d7fa700 1 -- 172.21.15.7:0/2258236990 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ea4051420 con 0x7f7ea813ad50 2024-01-27T02:37:42.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.184+0000 7f7eaf2f3700 1 -- 172.21.15.7:0/2258236990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7ea804c710 con 0x7f7ea813ad50 2024-01-27T02:37:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.766+0000 7f7e9d7fa700 1 -- 172.21.15.7:0/2258236990 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7ea4054a40 con 0x7f7ea813ad50 2024-01-27T02:37:42.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.769+0000 7f7eaf2f3700 1 -- 172.21.15.7:0/2258236990 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7e8c066f30 msgr2=0x7f7e8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:42.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.769+0000 7f7eaf2f3700 1 --2- 172.21.15.7:0/2258236990 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7e8c066f30 0x7f7e8c0693e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7e980099b0 tx=0x7f7e98016000 comp rx=0 tx=0).stop 2024-01-27T02:37:42.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.769+0000 7f7eaf2f3700 1 -- 172.21.15.7:0/2258236990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea813ad50 msgr2=0x7f7ea813a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:42.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.769+0000 7f7eaf2f3700 1 --2- 172.21.15.7:0/2258236990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea813ad50 0x7f7ea813a1a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400c5d0 tx=0x7f7ea4007f60 comp rx=0 tx=0).stop 2024-01-27T02:37:42.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.769+0000 7f7eaf2f3700 1 -- 172.21.15.7:0/2258236990 shutdown_connections 2024-01-27T02:37:42.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.769+0000 7f7eaf2f3700 1 --2- 172.21.15.7:0/2258236990 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7e8c066f30 0x7f7e8c0693e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:42.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.769+0000 7f7eaf2f3700 1 --2- 172.21.15.7:0/2258236990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea815bb40 0x7f7ea813a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:42.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.769+0000 7f7eaf2f3700 1 --2- 172.21.15.7:0/2258236990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ea8063c90 0x7f7ea81536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:42.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.769+0000 7f7eaf2f3700 1 --2- 172.21.15.7:0/2258236990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea813ad50 0x7f7ea813a1a0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:42.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.769+0000 7f7eaf2f3700 1 -- 172.21.15.7:0/2258236990 >> 172.21.15.7:0/2258236990 conn(0x7f7ea80cdc30 msgr2=0x7f7ea8063580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:42.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.769+0000 7f7eaf2f3700 1 -- 172.21.15.7:0/2258236990 shutdown_connections 2024-01-27T02:37:42.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.769+0000 7f7eaf2f3700 1 -- 172.21.15.7:0/2258236990 wait complete. 2024-01-27T02:37:42.772 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-27T02:37:42.787 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-27T02:37:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.925+0000 7f29cf5e8700 1 Processor -- start 2024-01-27T02:37:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.925+0000 7f29cf5e8700 1 -- start start 2024-01-27T02:37:42.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.926+0000 7f29cf5e8700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f29c815a450 0x7f29c80b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:42.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.926+0000 7f29cf5e8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c80cdff0 0x7f29c815b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:42.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.926+0000 7f29cf5e8700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c80cb6e0 con 0x7f29c80cbf40 2024-01-27T02:37:42.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.926+0000 7f29cf5e8700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c815b980 con 0x7f29c815a450 2024-01-27T02:37:42.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.926+0000 7f29cf5e8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c80cc390 con 0x7f29c80cdff0 2024-01-27T02:37:42.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.927+0000 7f29cdb85700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c80cdff0 0x7f29c815b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:42.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.927+0000 7f29cd384700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f29c815a450 0x7f29c80b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:42.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.927+0000 7f29cdb85700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c80cdff0 0x7f29c815b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:41934/0 (socket says 172.21.15.7:41934) 2024-01-27T02:37:42.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.927+0000 7f29cdb85700 1 -- 172.21.15.7:0/683867622 learned_addr learned my addr 172.21.15.7:0/683867622 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:42.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.927+0000 7f29cdb85700 1 -- 172.21.15.7:0/683867622 >> v2:172.21.15.188:3300/0 conn(0x7f29c815a450 msgr2=0x7f29c80b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:42.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.927+0000 7f29cdb85700 1 --2- 172.21.15.7:0/683867622 >> v2:172.21.15.188:3300/0 conn(0x7f29c815a450 0x7f29c80b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:42.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.927+0000 7f29cdb85700 1 -- 172.21.15.7:0/683867622 >> v1:172.21.15.188:6789/0 conn(0x7f29c80cbf40 legacy=0x7f29c80cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:42.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.927+0000 7f29cdb85700 1 -- 172.21.15.7:0/683867622 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c815afb0 con 0x7f29c80cdff0 2024-01-27T02:37:42.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.927+0000 7f29cd384700 1 --2- 172.21.15.7:0/683867622 >> v2:172.21.15.188:3300/0 conn(0x7f29c815a450 0x7f29c80b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:37:42.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.927+0000 7f29cdb85700 1 --2- 172.21.15.7:0/683867622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c80cdff0 0x7f29c815b440 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f29c0009f20 tx=0x7f29c0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae44be440d3e8b9a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.927+0000 7f29bffff700 1 -- 172.21.15.7:0/683867622 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c000e070 con 0x7f29c80cdff0 2024-01-27T02:37:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.927+0000 7f29bffff700 1 -- 172.21.15.7:0/683867622 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c000b9d0 con 0x7f29c80cdff0 2024-01-27T02:37:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.927+0000 7f29bffff700 1 -- 172.21.15.7:0/683867622 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c00092e0 con 0x7f29c80cdff0 2024-01-27T02:37:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.928+0000 7f29cf5e8700 1 -- 172.21.15.7:0/683867622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c80cdff0 msgr2=0x7f29c815b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.928+0000 7f29cf5e8700 1 --2- 172.21.15.7:0/683867622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c80cdff0 0x7f29c815b440 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f29c0009f20 tx=0x7f29c0009840 comp rx=0 tx=0).stop 2024-01-27T02:37:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.928+0000 7f29cf5e8700 1 -- 172.21.15.7:0/683867622 shutdown_connections 2024-01-27T02:37:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.928+0000 7f29cf5e8700 1 --2- 172.21.15.7:0/683867622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c80cdff0 0x7f29c815b440 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.928+0000 7f29cf5e8700 1 --2- 172.21.15.7:0/683867622 >> v2:172.21.15.188:3300/0 conn(0x7f29c815a450 0x7f29c80b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.928+0000 7f29cf5e8700 1 -- 172.21.15.7:0/683867622 >> 172.21.15.7:0/683867622 conn(0x7f29c8068440 msgr2=0x7f29c8063580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.928+0000 7f29cf5e8700 1 -- 172.21.15.7:0/683867622 shutdown_connections 2024-01-27T02:37:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.928+0000 7f29cf5e8700 1 -- 172.21.15.7:0/683867622 wait complete. 2024-01-27T02:37:42.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.929+0000 7f29cf5e8700 1 Processor -- start 2024-01-27T02:37:42.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.929+0000 7f29cf5e8700 1 -- start start 2024-01-27T02:37:42.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.930+0000 7f29cf5e8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c80cbf40 0x7f29c8146d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:42.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.930+0000 7f29cf5e8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c80cdff0 0x7f29c8152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:42.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.930+0000 7f29cf5e8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c815a450 0x7f29c8152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:42.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.930+0000 7f29cf5e8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c816b310 con 0x7f29c80cdff0 2024-01-27T02:37:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.930+0000 7f29cf5e8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c80b64b0 con 0x7f29c815a450 2024-01-27T02:37:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.930+0000 7f29cf5e8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c80b5630 con 0x7f29c80cbf40 2024-01-27T02:37:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.930+0000 7f29cd384700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c80cbf40 0x7f29c8146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.930+0000 7f29ccb83700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c80cdff0 0x7f29c8152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.930+0000 7f29cd384700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c80cbf40 0x7f29c8146d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:41936/0 (socket says 172.21.15.7:41936) 2024-01-27T02:37:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.931+0000 7f29cd384700 1 -- 172.21.15.7:0/1225835087 learned_addr learned my addr 172.21.15.7:0/1225835087 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.931+0000 7f29cdb85700 1 --2- 172.21.15.7:0/1225835087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c815a450 0x7f29c8152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.931+0000 7f29ccb83700 1 -- 172.21.15.7:0/1225835087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c80cbf40 msgr2=0x7f29c8146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.931+0000 7f29ccb83700 1 --2- 172.21.15.7:0/1225835087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c80cbf40 0x7f29c8146d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.931+0000 7f29ccb83700 1 -- 172.21.15.7:0/1225835087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c815a450 msgr2=0x7f29c8152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.931+0000 7f29ccb83700 1 --2- 172.21.15.7:0/1225835087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c815a450 0x7f29c8152f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.931+0000 7f29ccb83700 1 -- 172.21.15.7:0/1225835087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b8009420 con 0x7f29c80cdff0 2024-01-27T02:37:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.931+0000 7f29ccb83700 1 --2- 172.21.15.7:0/1225835087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c80cdff0 0x7f29c8152a50 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f29c400c5f0 tx=0x7f29c4007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:42.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.931+0000 7f29bd7fa700 1 -- 172.21.15.7:0/1225835087 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c4011590 con 0x7f29c80cdff0 2024-01-27T02:37:42.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.931+0000 7f29bd7fa700 1 -- 172.21.15.7:0/1225835087 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c4011710 con 0x7f29c80cdff0 2024-01-27T02:37:42.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.931+0000 7f29cf5e8700 1 -- 172.21.15.7:0/1225835087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c0009ae0 con 0x7f29c80cdff0 2024-01-27T02:37:42.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.931+0000 7f29bd7fa700 1 -- 172.21.15.7:0/1225835087 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c4011ad0 con 0x7f29c80cdff0 2024-01-27T02:37:42.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.931+0000 7f29cf5e8700 1 -- 172.21.15.7:0/1225835087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c80b5b80 con 0x7f29c80cdff0 2024-01-27T02:37:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.932+0000 7f29cf5e8700 1 -- 172.21.15.7:0/1225835087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c806dfd0 con 0x7f29c80cdff0 2024-01-27T02:37:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.933+0000 7f29bd7fa700 1 -- 172.21.15.7:0/1225835087 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f29c401b3b0 con 0x7f29c80cdff0 2024-01-27T02:37:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.933+0000 7f29bd7fa700 1 --2- 172.21.15.7:0/1225835087 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f29ac066f30 0x7f29ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.933+0000 7f29bd7fa700 1 -- 172.21.15.7:0/1225835087 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f29c4083560 con 0x7f29c80cdff0 2024-01-27T02:37:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.933+0000 7f29cd384700 1 --2- 172.21.15.7:0/1225835087 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f29ac066f30 0x7f29ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.934+0000 7f29cd384700 1 --2- 172.21.15.7:0/1225835087 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f29ac066f30 0x7f29ac0693e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f29c80cbc60 tx=0x7f29b80091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:42.938+0000 7f29bd7fa700 1 -- 172.21.15.7:0/1225835087 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29c4050930 con 0x7f29c80cdff0 2024-01-27T02:37:43.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.144+0000 7f29cf5e8700 1 -- 172.21.15.7:0/1225835087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f29c80cb6e0 con 0x7f29c80cdff0 2024-01-27T02:37:43.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.771+0000 7f29bd7fa700 1 -- 172.21.15.7:0/1225835087 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v109) v1 ==== 169+0+0 (secure 0 0 0) 0x7f29c80cb6e0 con 0x7f29c80cdff0 2024-01-27T02:37:43.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.775+0000 7f29cf5e8700 1 -- 172.21.15.7:0/1225835087 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f29ac066f30 msgr2=0x7f29ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:43.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.775+0000 7f29cf5e8700 1 --2- 172.21.15.7:0/1225835087 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f29ac066f30 0x7f29ac0693e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f29c80cbc60 tx=0x7f29b80091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.775+0000 7f29cf5e8700 1 -- 172.21.15.7:0/1225835087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c80cdff0 msgr2=0x7f29c8152a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.775+0000 7f29cf5e8700 1 --2- 172.21.15.7:0/1225835087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c80cdff0 0x7f29c8152a50 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f29c400c5f0 tx=0x7f29c4007ec0 comp rx=0 tx=0).stop 2024-01-27T02:37:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.775+0000 7f29cf5e8700 1 -- 172.21.15.7:0/1225835087 shutdown_connections 2024-01-27T02:37:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.775+0000 7f29cf5e8700 1 --2- 172.21.15.7:0/1225835087 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f29ac066f30 0x7f29ac0693e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.775+0000 7f29cf5e8700 1 --2- 172.21.15.7:0/1225835087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c815a450 0x7f29c8152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.775+0000 7f29cf5e8700 1 --2- 172.21.15.7:0/1225835087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c80cbf40 0x7f29c8146d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.775+0000 7f29cf5e8700 1 --2- 172.21.15.7:0/1225835087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c80cdff0 0x7f29c8152a50 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.775+0000 7f29cf5e8700 1 -- 172.21.15.7:0/1225835087 >> 172.21.15.7:0/1225835087 conn(0x7f29c8068440 msgr2=0x7f29c8154cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.775+0000 7f29cf5e8700 1 -- 172.21.15.7:0/1225835087 shutdown_connections 2024-01-27T02:37:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.775+0000 7f29cf5e8700 1 -- 172.21.15.7:0/1225835087 wait complete. 2024-01-27T02:37:43.777 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-27T02:37:43.792 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-27T02:37:43.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.928+0000 7f36370bf700 1 Processor -- start 2024-01-27T02:37:43.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.928+0000 7f36370bf700 1 -- start start 2024-01-27T02:37:43.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.929+0000 7f36370bf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f36300cbe30 0x7f36300ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:43.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.929+0000 7f36370bf700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36300679b0 con 0x7f36300cba60 2024-01-27T02:37:43.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.929+0000 7f36370bf700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3630067c90 con 0x7f363006b4e0 2024-01-27T02:37:43.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.929+0000 7f36370bf700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36300cca50 con 0x7f36300cbe30 2024-01-27T02:37:43.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.929+0000 7f362ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f36300cbe30 0x7f36300ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:43.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.929+0000 7f3634e5b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f36300cba60 0x7f363015b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33480/0 (socket says 172.21.15.7:33480) 2024-01-27T02:37:43.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.929+0000 7f3634e5b700 1 -- 172.21.15.7:0/1822939796 learned_addr learned my addr 172.21.15.7:0/1822939796 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:43.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.929+0000 7f362ffff700 1 -- 172.21.15.7:0/1822939796 >> v1:172.21.15.188:6789/0 conn(0x7f363006b4e0 legacy=0x7f363006b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:43.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.929+0000 7f362ffff700 1 -- 172.21.15.7:0/1822939796 >> v1:172.21.15.7:6789/0 conn(0x7f36300cba60 legacy=0x7f363015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:43.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.929+0000 7f362ffff700 1 -- 172.21.15.7:0/1822939796 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36300ccd30 con 0x7f36300cbe30 2024-01-27T02:37:43.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.930+0000 7f362ffff700 1 --2- 172.21.15.7:0/1822939796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36300cbe30 0x7f36300ce410 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f361c0098e0 tx=0x7f361c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d448c8e19bd8dfec server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:43.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.930+0000 7f362f7fe700 1 -- 172.21.15.7:0/1822939796 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c009e80 con 0x7f36300cbe30 2024-01-27T02:37:43.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.930+0000 7f362f7fe700 1 -- 172.21.15.7:0/1822939796 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c00b760 con 0x7f36300cbe30 2024-01-27T02:37:43.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.930+0000 7f362f7fe700 1 -- 172.21.15.7:0/1822939796 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c00bae0 con 0x7f36300cbe30 2024-01-27T02:37:43.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.930+0000 7f36370bf700 1 -- 172.21.15.7:0/1822939796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36300cbe30 msgr2=0x7f36300ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:43.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.930+0000 7f36370bf700 1 --2- 172.21.15.7:0/1822939796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36300cbe30 0x7f36300ce410 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f361c0098e0 tx=0x7f361c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:43.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.930+0000 7f36370bf700 1 -- 172.21.15.7:0/1822939796 shutdown_connections 2024-01-27T02:37:43.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.930+0000 7f36370bf700 1 --2- 172.21.15.7:0/1822939796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36300cbe30 0x7f36300ce410 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:43.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.930+0000 7f36370bf700 1 -- 172.21.15.7:0/1822939796 >> 172.21.15.7:0/1822939796 conn(0x7f3630063580 msgr2=0x7f3630063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:43.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.931+0000 7f36370bf700 1 -- 172.21.15.7:0/1822939796 shutdown_connections 2024-01-27T02:37:43.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.931+0000 7f36370bf700 1 -- 172.21.15.7:0/1822939796 wait complete. 2024-01-27T02:37:43.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.932+0000 7f36370bf700 1 Processor -- start 2024-01-27T02:37:43.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.932+0000 7f36370bf700 1 -- start start 2024-01-27T02:37:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.933+0000 7f36370bf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f363006b4e0 0x7f36301583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.933+0000 7f36370bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36300cba60 0x7f36300b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.933+0000 7f36370bf700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36300cbe30 0x7f363015cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.933+0000 7f36370bf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36300b3e80 con 0x7f36300cba60 2024-01-27T02:37:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.933+0000 7f36370bf700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36300b39f0 con 0x7f363006b4e0 2024-01-27T02:37:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.933+0000 7f36370bf700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36300b2de0 con 0x7f36300cbe30 2024-01-27T02:37:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.933+0000 7f362ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36300cba60 0x7f36300b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.933+0000 7f362ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36300cba60 0x7f36300b2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44504/0 (socket says 172.21.15.7:44504) 2024-01-27T02:37:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.933+0000 7f362ffff700 1 -- 172.21.15.7:0/784038151 learned_addr learned my addr 172.21.15.7:0/784038151 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.933+0000 7f3634e5b700 1 --2- 172.21.15.7:0/784038151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f363006b4e0 0x7f36301583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.933+0000 7f363565c700 1 --2- 172.21.15.7:0/784038151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36300cbe30 0x7f363015cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.933+0000 7f362ffff700 1 -- 172.21.15.7:0/784038151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36300cbe30 msgr2=0x7f363015cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:43.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.933+0000 7f362ffff700 1 --2- 172.21.15.7:0/784038151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36300cbe30 0x7f363015cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:43.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.933+0000 7f362ffff700 1 -- 172.21.15.7:0/784038151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f363006b4e0 msgr2=0x7f36301583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:43.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.933+0000 7f362ffff700 1 --2- 172.21.15.7:0/784038151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f363006b4e0 0x7f36301583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:43.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.933+0000 7f362ffff700 1 -- 172.21.15.7:0/784038151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c009420 con 0x7f36300cba60 2024-01-27T02:37:43.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.933+0000 7f3634e5b700 1 --2- 172.21.15.7:0/784038151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f363006b4e0 0x7f36301583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:43.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.934+0000 7f362ffff700 1 --2- 172.21.15.7:0/784038151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36300cba60 0x7f36300b2460 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f361c0093f0 tx=0x7f361c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:43.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.934+0000 7f362cff9700 1 -- 172.21.15.7:0/784038151 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c00bee0 con 0x7f36300cba60 2024-01-27T02:37:43.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.934+0000 7f362cff9700 1 -- 172.21.15.7:0/784038151 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c00bae0 con 0x7f36300cba60 2024-01-27T02:37:43.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.934+0000 7f36370bf700 1 -- 172.21.15.7:0/784038151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36300b30c0 con 0x7f36300cba60 2024-01-27T02:37:43.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.934+0000 7f362cff9700 1 -- 172.21.15.7:0/784038151 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c01c6b0 con 0x7f36300cba60 2024-01-27T02:37:43.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.934+0000 7f36370bf700 1 -- 172.21.15.7:0/784038151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36300b3680 con 0x7f36300cba60 2024-01-27T02:37:43.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.935+0000 7f36370bf700 1 -- 172.21.15.7:0/784038151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36300b3b80 con 0x7f36300cba60 2024-01-27T02:37:43.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.935+0000 7f362cff9700 1 -- 172.21.15.7:0/784038151 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f361c01cae0 con 0x7f36300cba60 2024-01-27T02:37:43.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.936+0000 7f362cff9700 1 --2- 172.21.15.7:0/784038151 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3614066f30 0x7f36140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:43.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.936+0000 7f362cff9700 1 -- 172.21.15.7:0/784038151 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f361c086650 con 0x7f36300cba60 2024-01-27T02:37:43.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.936+0000 7f3634e5b700 1 --2- 172.21.15.7:0/784038151 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3614066f30 0x7f36140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:43.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.937+0000 7f3634e5b700 1 --2- 172.21.15.7:0/784038151 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3614066f30 0x7f36140693e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3624007fd0 tx=0x7f3624009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:43.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:43.940+0000 7f362cff9700 1 -- 172.21.15.7:0/784038151 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f361c053bc0 con 0x7f36300cba60 2024-01-27T02:37:44.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.151+0000 7f36370bf700 1 -- 172.21.15.7:0/784038151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f36300b3b80 con 0x7f36300cba60 2024-01-27T02:37:44.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.155+0000 7f362cff9700 1 -- 172.21.15.7:0/784038151 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f361c0571e0 con 0x7f36300cba60 2024-01-27T02:37:44.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.157+0000 7f36370bf700 1 -- 172.21.15.7:0/784038151 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3614066f30 msgr2=0x7f36140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:44.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.157+0000 7f36370bf700 1 --2- 172.21.15.7:0/784038151 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3614066f30 0x7f36140693e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3624007fd0 tx=0x7f3624009040 comp rx=0 tx=0).stop 2024-01-27T02:37:44.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.157+0000 7f36370bf700 1 -- 172.21.15.7:0/784038151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36300cba60 msgr2=0x7f36300b2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:44.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.157+0000 7f36370bf700 1 --2- 172.21.15.7:0/784038151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36300cba60 0x7f36300b2460 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f361c0093f0 tx=0x7f361c016000 comp rx=0 tx=0).stop 2024-01-27T02:37:44.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.157+0000 7f36370bf700 1 -- 172.21.15.7:0/784038151 shutdown_connections 2024-01-27T02:37:44.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.157+0000 7f36370bf700 1 --2- 172.21.15.7:0/784038151 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3614066f30 0x7f36140693e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:44.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.157+0000 7f36370bf700 1 --2- 172.21.15.7:0/784038151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36300cbe30 0x7f363015cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:44.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.157+0000 7f36370bf700 1 --2- 172.21.15.7:0/784038151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f363006b4e0 0x7f36301583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:44.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.157+0000 7f36370bf700 1 --2- 172.21.15.7:0/784038151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36300cba60 0x7f36300b2460 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:44.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.158+0000 7f36370bf700 1 -- 172.21.15.7:0/784038151 >> 172.21.15.7:0/784038151 conn(0x7f3630063580 msgr2=0x7f36300c4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:44.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.158+0000 7f36370bf700 1 -- 172.21.15.7:0/784038151 shutdown_connections 2024-01-27T02:37:44.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.158+0000 7f36370bf700 1 -- 172.21.15.7:0/784038151 wait complete. 2024-01-27T02:37:44.173 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2024-01-27T02:37:44.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.302+0000 7f9e7565b700 1 Processor -- start 2024-01-27T02:37:44.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.302+0000 7f9e7565b700 1 -- start start 2024-01-27T02:37:44.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.303+0000 7f9e7565b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9e70157c00 0x7f9e700cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:44.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.303+0000 7f9e7565b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e7015ba30 con 0x7f9e700cfd10 2024-01-27T02:37:44.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.303+0000 7f9e7565b700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e700d0390 con 0x7f9e700b3040 2024-01-27T02:37:44.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.303+0000 7f9e7565b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e700d0170 con 0x7f9e70157c00 2024-01-27T02:37:44.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.304+0000 7f9e6effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9e70157c00 0x7f9e700cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:44.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.304+0000 7f9e6e7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9e700cfd10 0x7f9e700b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33488/0 (socket says 172.21.15.7:33488) 2024-01-27T02:37:44.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.304+0000 7f9e6e7fc700 1 -- 172.21.15.7:0/75897751 learned_addr learned my addr 172.21.15.7:0/75897751 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:44.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.304+0000 7f9e6effd700 1 -- 172.21.15.7:0/75897751 >> v1:172.21.15.188:6789/0 conn(0x7f9e700b3040 legacy=0x7f9e7015bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:44.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.304+0000 7f9e6effd700 1 -- 172.21.15.7:0/75897751 >> v1:172.21.15.7:6789/0 conn(0x7f9e700cfd10 legacy=0x7f9e700b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:44.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.304+0000 7f9e6dffb700 1 -- 172.21.15.7:0/75897751 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3832743730 0 0) 0x7f9e7015ba30 con 0x7f9e700cfd10 2024-01-27T02:37:44.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.304+0000 7f9e6effd700 1 -- 172.21.15.7:0/75897751 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e700d0670 con 0x7f9e70157c00 2024-01-27T02:37:44.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.304+0000 7f9e6effd700 1 --2- 172.21.15.7:0/75897751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e70157c00 0x7f9e700cf7d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f9e600098e0 tx=0x7f9e600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ed804a094e42be server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:44.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.304+0000 7f9e6dffb700 1 -- 172.21.15.7:0/75897751 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60009e80 con 0x7f9e70157c00 2024-01-27T02:37:44.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.304+0000 7f9e6dffb700 1 -- 172.21.15.7:0/75897751 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6000b860 con 0x7f9e70157c00 2024-01-27T02:37:44.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.304+0000 7f9e6dffb700 1 -- 172.21.15.7:0/75897751 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6000bbe0 con 0x7f9e70157c00 2024-01-27T02:37:44.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.305+0000 7f9e7565b700 1 -- 172.21.15.7:0/75897751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e70157c00 msgr2=0x7f9e700cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:44.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.305+0000 7f9e7565b700 1 --2- 172.21.15.7:0/75897751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e70157c00 0x7f9e700cf7d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f9e600098e0 tx=0x7f9e600091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:44.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.305+0000 7f9e7565b700 1 -- 172.21.15.7:0/75897751 shutdown_connections 2024-01-27T02:37:44.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.305+0000 7f9e7565b700 1 --2- 172.21.15.7:0/75897751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e70157c00 0x7f9e700cf7d0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:44.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.305+0000 7f9e7565b700 1 -- 172.21.15.7:0/75897751 >> 172.21.15.7:0/75897751 conn(0x7f9e70068450 msgr2=0x7f9e700709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:44.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.305+0000 7f9e7565b700 1 -- 172.21.15.7:0/75897751 shutdown_connections 2024-01-27T02:37:44.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.305+0000 7f9e7565b700 1 -- 172.21.15.7:0/75897751 wait complete. 2024-01-27T02:37:44.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.306+0000 7f9e7565b700 1 Processor -- start 2024-01-27T02:37:44.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.307+0000 7f9e7565b700 1 -- start start 2024-01-27T02:37:44.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.307+0000 7f9e7565b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e700b3040 0x7f9e70141970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:44.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.307+0000 7f9e7565b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e700cfd10 0x7f9e7015c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:44.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.307+0000 7f9e7565b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e70157c00 0x7f9e70155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:44.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.307+0000 7f9e7565b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e70154800 con 0x7f9e700cfd10 2024-01-27T02:37:44.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.307+0000 7f9e7565b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e70141eb0 con 0x7f9e70157c00 2024-01-27T02:37:44.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.307+0000 7f9e7565b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e701549e0 con 0x7f9e700b3040 2024-01-27T02:37:44.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.307+0000 7f9e6e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e700cfd10 0x7f9e7015c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:44.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.307+0000 7f9e6e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e700cfd10 0x7f9e7015c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44522/0 (socket says 172.21.15.7:44522) 2024-01-27T02:37:44.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.307+0000 7f9e6e7fc700 1 -- 172.21.15.7:0/1809530038 learned_addr learned my addr 172.21.15.7:0/1809530038 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:44.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.308+0000 7f9e6effd700 1 --2- 172.21.15.7:0/1809530038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e700b3040 0x7f9e70141970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:44.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.308+0000 7f9e6f7fe700 1 --2- 172.21.15.7:0/1809530038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e70157c00 0x7f9e70155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:44.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.308+0000 7f9e6e7fc700 1 -- 172.21.15.7:0/1809530038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e700b3040 msgr2=0x7f9e70141970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:44.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.308+0000 7f9e6e7fc700 1 --2- 172.21.15.7:0/1809530038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e700b3040 0x7f9e70141970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:44.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.308+0000 7f9e6e7fc700 1 -- 172.21.15.7:0/1809530038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e70157c00 msgr2=0x7f9e70155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:44.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.308+0000 7f9e6e7fc700 1 --2- 172.21.15.7:0/1809530038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e70157c00 0x7f9e70155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:44.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.308+0000 7f9e6e7fc700 1 -- 172.21.15.7:0/1809530038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e60009420 con 0x7f9e700cfd10 2024-01-27T02:37:44.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.308+0000 7f9e6e7fc700 1 --2- 172.21.15.7:0/1809530038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e700cfd10 0x7f9e7015c5d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400c810 tx=0x7f9e64007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:44.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.308+0000 7f9e56ffd700 1 -- 172.21.15.7:0/1809530038 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e64005c00 con 0x7f9e700cfd10 2024-01-27T02:37:44.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.308+0000 7f9e56ffd700 1 -- 172.21.15.7:0/1809530038 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e64005d60 con 0x7f9e700cfd10 2024-01-27T02:37:44.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.308+0000 7f9e7565b700 1 -- 172.21.15.7:0/1809530038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e70154cc0 con 0x7f9e700cfd10 2024-01-27T02:37:44.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.308+0000 7f9e56ffd700 1 -- 172.21.15.7:0/1809530038 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e64007340 con 0x7f9e700cfd10 2024-01-27T02:37:44.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.308+0000 7f9e7565b700 1 -- 172.21.15.7:0/1809530038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e70155280 con 0x7f9e700cfd10 2024-01-27T02:37:44.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.309+0000 7f9e7565b700 1 -- 172.21.15.7:0/1809530038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e7004c710 con 0x7f9e700cfd10 2024-01-27T02:37:44.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.310+0000 7f9e56ffd700 1 -- 172.21.15.7:0/1809530038 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9e6401b400 con 0x7f9e700cfd10 2024-01-27T02:37:44.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.310+0000 7f9e56ffd700 1 --2- 172.21.15.7:0/1809530038 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9e5c066f30 0x7f9e5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:44.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.310+0000 7f9e56ffd700 1 -- 172.21.15.7:0/1809530038 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f9e64083d40 con 0x7f9e700cfd10 2024-01-27T02:37:44.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.310+0000 7f9e6effd700 1 --2- 172.21.15.7:0/1809530038 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9e5c066f30 0x7f9e5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:44.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.311+0000 7f9e6effd700 1 --2- 172.21.15.7:0/1809530038 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9e5c066f30 0x7f9e5c0693e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9e600093f0 tx=0x7f9e60017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.315+0000 7f9e56ffd700 1 -- 172.21.15.7:0/1809530038 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e64051330 con 0x7f9e700cfd10 2024-01-27T02:37:44.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.614+0000 7f9e7565b700 1 -- 172.21.15.7:0/1809530038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) v1 -- 0x7f9e700d0950 con 0x7f9e700cfd10 2024-01-27T02:37:44.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.772+0000 7f9e56ffd700 1 -- 172.21.15.7:0/1809530038 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v291) v1 ==== 137+0+0 (secure 0 0 0) 0x7f9e64054950 con 0x7f9e700cfd10 2024-01-27T02:37:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.775+0000 7f9e7565b700 1 -- 172.21.15.7:0/1809530038 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9e5c066f30 msgr2=0x7f9e5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.775+0000 7f9e7565b700 1 --2- 172.21.15.7:0/1809530038 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9e5c066f30 0x7f9e5c0693e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9e600093f0 tx=0x7f9e60017000 comp rx=0 tx=0).stop 2024-01-27T02:37:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.775+0000 7f9e7565b700 1 -- 172.21.15.7:0/1809530038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e700cfd10 msgr2=0x7f9e7015c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.775+0000 7f9e7565b700 1 --2- 172.21.15.7:0/1809530038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e700cfd10 0x7f9e7015c5d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400c810 tx=0x7f9e64007d90 comp rx=0 tx=0).stop 2024-01-27T02:37:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.775+0000 7f9e7565b700 1 -- 172.21.15.7:0/1809530038 shutdown_connections 2024-01-27T02:37:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.775+0000 7f9e7565b700 1 --2- 172.21.15.7:0/1809530038 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9e5c066f30 0x7f9e5c0693e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.775+0000 7f9e7565b700 1 --2- 172.21.15.7:0/1809530038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e70157c00 0x7f9e70155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.775+0000 7f9e7565b700 1 --2- 172.21.15.7:0/1809530038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e700b3040 0x7f9e70141970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.775+0000 7f9e7565b700 1 --2- 172.21.15.7:0/1809530038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e700cfd10 0x7f9e7015c5d0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.775+0000 7f9e7565b700 1 -- 172.21.15.7:0/1809530038 >> 172.21.15.7:0/1809530038 conn(0x7f9e70068450 msgr2=0x7f9e700709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.775+0000 7f9e7565b700 1 -- 172.21.15.7:0/1809530038 shutdown_connections 2024-01-27T02:37:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.775+0000 7f9e7565b700 1 -- 172.21.15.7:0/1809530038 wait complete. 2024-01-27T02:37:44.791 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2024-01-27T02:37:44.792 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestQuotaFull) That when an OSD epoch barrier is set on an MDS, subsequently 2024-01-27T02:37:44.792 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestQuotaFull) 2024-01-27T02:37:44.792 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2024-01-27T02:37:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.926+0000 7f05f2ac1700 1 Processor -- start 2024-01-27T02:37:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.926+0000 7f05f2ac1700 1 -- start start 2024-01-27T02:37:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.927+0000 7f05f2ac1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f05ec0b3350 0x7f05ec0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.927+0000 7f05f2ac1700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ec159390 con 0x7f05ec0ce810 2024-01-27T02:37:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.927+0000 7f05f2ac1700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ec158780 con 0x7f05ec0b2f80 2024-01-27T02:37:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.927+0000 7f05f2ac1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ec158a60 con 0x7f05ec0b3350 2024-01-27T02:37:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.928+0000 7f05ebfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f05ec0b3350 0x7f05ec0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.928+0000 7f05ebfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f05ec0b3350 0x7f05ec0ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44532/0 (socket says 172.21.15.7:44532) 2024-01-27T02:37:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.928+0000 7f05f105e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f05ec0ce810 0x7f05ec1553e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33492/0 (socket says 172.21.15.7:33492) 2024-01-27T02:37:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.928+0000 7f05ebfff700 1 -- 172.21.15.7:0/3177662818 learned_addr learned my addr 172.21.15.7:0/3177662818 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.928+0000 7f05ebfff700 1 -- 172.21.15.7:0/3177662818 >> v1:172.21.15.188:6789/0 conn(0x7f05ec0b2f80 legacy=0x7f05ec06a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.928+0000 7f05ebfff700 1 -- 172.21.15.7:0/3177662818 >> v1:172.21.15.7:6789/0 conn(0x7f05ec0ce810 legacy=0x7f05ec1553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.928+0000 7f05ebfff700 1 -- 172.21.15.7:0/3177662818 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec158d40 con 0x7f05ec0b3350 2024-01-27T02:37:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.928+0000 7f05eb7fe700 1 -- 172.21.15.7:0/3177662818 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2628454117 0 0) 0x7f05ec158780 con 0x7f05ec0b2f80 2024-01-27T02:37:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.928+0000 7f05ebfff700 1 --2- 172.21.15.7:0/3177662818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec0b3350 0x7f05ec0ce2d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f05e00099b0 tx=0x7f05e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d723181f28c442 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.928+0000 7f05eb7fe700 1 -- 172.21.15.7:0/3177662818 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e000b760 con 0x7f05ec0b3350 2024-01-27T02:37:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.928+0000 7f05eb7fe700 1 -- 172.21.15.7:0/3177662818 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e000b8c0 con 0x7f05ec0b3350 2024-01-27T02:37:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.928+0000 7f05eb7fe700 1 -- 172.21.15.7:0/3177662818 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e000bc60 con 0x7f05ec0b3350 2024-01-27T02:37:44.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.929+0000 7f05f2ac1700 1 -- 172.21.15.7:0/3177662818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec0b3350 msgr2=0x7f05ec0ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:44.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.929+0000 7f05f2ac1700 1 --2- 172.21.15.7:0/3177662818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec0b3350 0x7f05ec0ce2d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f05e00099b0 tx=0x7f05e0009250 comp rx=0 tx=0).stop 2024-01-27T02:37:44.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.929+0000 7f05f2ac1700 1 -- 172.21.15.7:0/3177662818 shutdown_connections 2024-01-27T02:37:44.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.929+0000 7f05f2ac1700 1 --2- 172.21.15.7:0/3177662818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec0b3350 0x7f05ec0ce2d0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:44.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.929+0000 7f05f2ac1700 1 -- 172.21.15.7:0/3177662818 >> 172.21.15.7:0/3177662818 conn(0x7f05ec068430 msgr2=0x7f05ec06be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:44.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.929+0000 7f05f2ac1700 1 -- 172.21.15.7:0/3177662818 shutdown_connections 2024-01-27T02:37:44.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.929+0000 7f05f2ac1700 1 -- 172.21.15.7:0/3177662818 wait complete. 2024-01-27T02:37:44.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.930+0000 7f05f2ac1700 1 Processor -- start 2024-01-27T02:37:44.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.930+0000 7f05f2ac1700 1 -- start start 2024-01-27T02:37:44.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.931+0000 7f05f2ac1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec0b2f80 0x7f05ec0c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:44.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.931+0000 7f05f2ac1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05ec0b3350 0x7f05ec142640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:44.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.931+0000 7f05f2ac1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05ec0ce810 0x7f05ec0c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:44.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.931+0000 7f05f2ac1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec0c9630 con 0x7f05ec0b2f80 2024-01-27T02:37:44.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.931+0000 7f05f2ac1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec0c8a20 con 0x7f05ec0b3350 2024-01-27T02:37:44.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.931+0000 7f05f2ac1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec0c8d00 con 0x7f05ec0ce810 2024-01-27T02:37:44.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.931+0000 7f05f085d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec0b2f80 0x7f05ec0c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:44.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.931+0000 7f05f085d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec0b2f80 0x7f05ec0c8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44542/0 (socket says 172.21.15.7:44542) 2024-01-27T02:37:44.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.931+0000 7f05f085d700 1 -- 172.21.15.7:0/2671164754 learned_addr learned my addr 172.21.15.7:0/2671164754 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:44.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.931+0000 7f05ebfff700 1 --2- 172.21.15.7:0/2671164754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05ec0b3350 0x7f05ec142640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.932+0000 7f05f105e700 1 --2- 172.21.15.7:0/2671164754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05ec0ce810 0x7f05ec0c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.932+0000 7f05f085d700 1 -- 172.21.15.7:0/2671164754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05ec0ce810 msgr2=0x7f05ec0c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.932+0000 7f05f085d700 1 --2- 172.21.15.7:0/2671164754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05ec0ce810 0x7f05ec0c55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.932+0000 7f05f085d700 1 -- 172.21.15.7:0/2671164754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05ec0b3350 msgr2=0x7f05ec142640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.932+0000 7f05f085d700 1 --2- 172.21.15.7:0/2671164754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05ec0b3350 0x7f05ec142640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.932+0000 7f05f085d700 1 -- 172.21.15.7:0/2671164754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e00094f0 con 0x7f05ec0b2f80 2024-01-27T02:37:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.932+0000 7f05f085d700 1 --2- 172.21.15.7:0/2671164754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec0b2f80 0x7f05ec0c8160 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00c7e0 tx=0x7f05dc007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.932+0000 7f05e8ff9700 1 -- 172.21.15.7:0/2671164754 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc011590 con 0x7f05ec0b2f80 2024-01-27T02:37:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.932+0000 7f05e8ff9700 1 -- 172.21.15.7:0/2671164754 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05dc011710 con 0x7f05ec0b2f80 2024-01-27T02:37:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.932+0000 7f05f2ac1700 1 -- 172.21.15.7:0/2671164754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ec0c8fe0 con 0x7f05ec0b2f80 2024-01-27T02:37:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.932+0000 7f05e8ff9700 1 -- 172.21.15.7:0/2671164754 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc011ad0 con 0x7f05ec0b2f80 2024-01-27T02:37:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.932+0000 7f05f2ac1700 1 -- 172.21.15.7:0/2671164754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ec0c86a0 con 0x7f05ec0b2f80 2024-01-27T02:37:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.933+0000 7f05f2ac1700 1 -- 172.21.15.7:0/2671164754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ec04c710 con 0x7f05ec0b2f80 2024-01-27T02:37:44.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.934+0000 7f05e8ff9700 1 -- 172.21.15.7:0/2671164754 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f05dc01b3b0 con 0x7f05ec0b2f80 2024-01-27T02:37:44.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.934+0000 7f05e8ff9700 1 --2- 172.21.15.7:0/2671164754 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f05d8066f30 0x7f05d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:44.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.934+0000 7f05e8ff9700 1 -- 172.21.15.7:0/2671164754 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f05dc083770 con 0x7f05ec0b2f80 2024-01-27T02:37:44.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.934+0000 7f05ebfff700 1 --2- 172.21.15.7:0/2671164754 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f05d8066f30 0x7f05d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:44.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.935+0000 7f05ebfff700 1 --2- 172.21.15.7:0/2671164754 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f05d8066f30 0x7f05d80693e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f05e00099b0 tx=0x7f05e0016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:44.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:44.939+0000 7f05e8ff9700 1 -- 172.21.15.7:0/2671164754 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05dc050ce0 con 0x7f05ec0b2f80 2024-01-27T02:37:45.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.227+0000 7f05f2ac1700 1 -- 172.21.15.7:0/2671164754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) v1 -- 0x7f05ec159020 con 0x7f05ec0b2f80 2024-01-27T02:37:45.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.774+0000 7f05e8ff9700 1 -- 172.21.15.7:0/2671164754 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v292) v1 ==== 131+0+0 (secure 0 0 0) 0x7f05dc017020 con 0x7f05ec0b2f80 2024-01-27T02:37:45.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.777+0000 7f05f2ac1700 1 -- 172.21.15.7:0/2671164754 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f05d8066f30 msgr2=0x7f05d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:45.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.777+0000 7f05f2ac1700 1 --2- 172.21.15.7:0/2671164754 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f05d8066f30 0x7f05d80693e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f05e00099b0 tx=0x7f05e0016000 comp rx=0 tx=0).stop 2024-01-27T02:37:45.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.777+0000 7f05f2ac1700 1 -- 172.21.15.7:0/2671164754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec0b2f80 msgr2=0x7f05ec0c8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:45.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.777+0000 7f05f2ac1700 1 --2- 172.21.15.7:0/2671164754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec0b2f80 0x7f05ec0c8160 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00c7e0 tx=0x7f05dc007df0 comp rx=0 tx=0).stop 2024-01-27T02:37:45.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.777+0000 7f05f2ac1700 1 -- 172.21.15.7:0/2671164754 shutdown_connections 2024-01-27T02:37:45.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.777+0000 7f05f2ac1700 1 --2- 172.21.15.7:0/2671164754 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f05d8066f30 0x7f05d80693e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:45.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.777+0000 7f05f2ac1700 1 --2- 172.21.15.7:0/2671164754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05ec0ce810 0x7f05ec0c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:45.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.777+0000 7f05f2ac1700 1 --2- 172.21.15.7:0/2671164754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ec0b2f80 0x7f05ec0c8160 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:45.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.777+0000 7f05f2ac1700 1 --2- 172.21.15.7:0/2671164754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05ec0b3350 0x7f05ec142640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:45.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.777+0000 7f05f2ac1700 1 -- 172.21.15.7:0/2671164754 >> 172.21.15.7:0/2671164754 conn(0x7f05ec068430 msgr2=0x7f05ec0cbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:45.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.777+0000 7f05f2ac1700 1 -- 172.21.15.7:0/2671164754 shutdown_connections 2024-01-27T02:37:45.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.777+0000 7f05f2ac1700 1 -- 172.21.15.7:0/2671164754 wait complete. 2024-01-27T02:37:45.794 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-27T02:37:45.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.933+0000 7ffa4e985700 1 Processor -- start 2024-01-27T02:37:45.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.933+0000 7ffa4e985700 1 -- start start 2024-01-27T02:37:45.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.934+0000 7ffa4e985700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffa48143670 0x7ffa48063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:45.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.934+0000 7ffa4e985700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffa480aa8b0 0x7ffa48063ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:45.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.934+0000 7ffa4e985700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa480cc290 con 0x7ffa48143a40 2024-01-27T02:37:45.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.934+0000 7ffa4e985700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa480cb960 con 0x7ffa480aa8b0 2024-01-27T02:37:45.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.934+0000 7ffa4e985700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa480cbc40 con 0x7ffa48143670 2024-01-27T02:37:45.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.934+0000 7ffa477fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffa480aa8b0 0x7ffa48063ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:45.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.934+0000 7ffa477fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffa480aa8b0 0x7ffa48063ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44548/0 (socket says 172.21.15.7:44548) 2024-01-27T02:37:45.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.934+0000 7ffa477fe700 1 -- 172.21.15.7:0/1827774553 learned_addr learned my addr 172.21.15.7:0/1827774553 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:45.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.934+0000 7ffa47fff700 1 --2- 172.21.15.7:0/1827774553 >> v2:172.21.15.188:3300/0 conn(0x7ffa48143670 0x7ffa48063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:45.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.934+0000 7ffa477fe700 1 -- 172.21.15.7:0/1827774553 >> v2:172.21.15.188:3300/0 conn(0x7ffa48143670 msgr2=0x7ffa48063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:45.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.934+0000 7ffa477fe700 1 --2- 172.21.15.7:0/1827774553 >> v2:172.21.15.188:3300/0 conn(0x7ffa48143670 0x7ffa48063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:45.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.934+0000 7ffa477fe700 1 -- 172.21.15.7:0/1827774553 >> v1:172.21.15.188:6789/0 conn(0x7ffa48143a40 legacy=0x7ffa480c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:45.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.934+0000 7ffa477fe700 1 -- 172.21.15.7:0/1827774553 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa480cbf20 con 0x7ffa480aa8b0 2024-01-27T02:37:45.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.934+0000 7ffa477fe700 1 --2- 172.21.15.7:0/1827774553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa480aa8b0 0x7ffa48063ac0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ffa480ca5c0 tx=0x7ffa3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a5abe7e6dbc3ff server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:45.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.935+0000 7ffa46ffd700 1 -- 172.21.15.7:0/1827774553 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c00b820 con 0x7ffa480aa8b0 2024-01-27T02:37:45.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.935+0000 7ffa46ffd700 1 -- 172.21.15.7:0/1827774553 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3c00b980 con 0x7ffa480aa8b0 2024-01-27T02:37:45.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.935+0000 7ffa46ffd700 1 -- 172.21.15.7:0/1827774553 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c00bd20 con 0x7ffa480aa8b0 2024-01-27T02:37:45.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.935+0000 7ffa4e985700 1 -- 172.21.15.7:0/1827774553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa480aa8b0 msgr2=0x7ffa48063ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:45.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.935+0000 7ffa4e985700 1 --2- 172.21.15.7:0/1827774553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa480aa8b0 0x7ffa48063ac0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ffa480ca5c0 tx=0x7ffa3c009250 comp rx=0 tx=0).stop 2024-01-27T02:37:45.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.935+0000 7ffa4e985700 1 -- 172.21.15.7:0/1827774553 shutdown_connections 2024-01-27T02:37:45.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.935+0000 7ffa4e985700 1 --2- 172.21.15.7:0/1827774553 >> v2:172.21.15.188:3300/0 conn(0x7ffa48143670 0x7ffa48063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.935+0000 7ffa4e985700 1 --2- 172.21.15.7:0/1827774553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa480aa8b0 0x7ffa48063ac0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.935+0000 7ffa4e985700 1 -- 172.21.15.7:0/1827774553 >> 172.21.15.7:0/1827774553 conn(0x7ffa480cdbf0 msgr2=0x7ffa480cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.935+0000 7ffa4e985700 1 -- 172.21.15.7:0/1827774553 shutdown_connections 2024-01-27T02:37:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.935+0000 7ffa4e985700 1 -- 172.21.15.7:0/1827774553 wait complete. 2024-01-27T02:37:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.936+0000 7ffa4e985700 1 Processor -- start 2024-01-27T02:37:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.937+0000 7ffa4e985700 1 -- start start 2024-01-27T02:37:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.937+0000 7ffa4e985700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa480aa8b0 0x7ffa4806b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.937+0000 7ffa4e985700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa48143670 0x7ffa480b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.937+0000 7ffa4e985700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa48143a40 0x7ffa480b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.937+0000 7ffa4e985700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa480c5b00 con 0x7ffa48143670 2024-01-27T02:37:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.937+0000 7ffa4e985700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa480c4ca0 con 0x7ffa480aa8b0 2024-01-27T02:37:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.937+0000 7ffa4e985700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa480c4ef0 con 0x7ffa48143a40 2024-01-27T02:37:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.938+0000 7ffa477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa48143670 0x7ffa480b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.938+0000 7ffa477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa48143670 0x7ffa480b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44554/0 (socket says 172.21.15.7:44554) 2024-01-27T02:37:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.938+0000 7ffa477fe700 1 -- 172.21.15.7:0/995132368 learned_addr learned my addr 172.21.15.7:0/995132368 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.938+0000 7ffa47fff700 1 --2- 172.21.15.7:0/995132368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa480aa8b0 0x7ffa4806b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.938+0000 7ffa477fe700 1 -- 172.21.15.7:0/995132368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa48143a40 msgr2=0x7ffa480b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.938+0000 7ffa4cf22700 1 --2- 172.21.15.7:0/995132368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa48143a40 0x7ffa480b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.938+0000 7ffa477fe700 1 --2- 172.21.15.7:0/995132368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa48143a40 0x7ffa480b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.938+0000 7ffa477fe700 1 -- 172.21.15.7:0/995132368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa480aa8b0 msgr2=0x7ffa4806b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.938+0000 7ffa477fe700 1 --2- 172.21.15.7:0/995132368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa480aa8b0 0x7ffa4806b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.938+0000 7ffa477fe700 1 -- 172.21.15.7:0/995132368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3c0094f0 con 0x7ffa48143670 2024-01-27T02:37:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.938+0000 7ffa47fff700 1 --2- 172.21.15.7:0/995132368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa480aa8b0 0x7ffa4806b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.938+0000 7ffa4cf22700 1 --2- 172.21.15.7:0/995132368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa48143a40 0x7ffa480b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.938+0000 7ffa477fe700 1 --2- 172.21.15.7:0/995132368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa48143670 0x7ffa480b1b50 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0030c0 tx=0x7ffa3c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.938+0000 7ffa2ffff700 1 -- 172.21.15.7:0/995132368 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c011e60 con 0x7ffa48143670 2024-01-27T02:37:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.938+0000 7ffa2ffff700 1 -- 172.21.15.7:0/995132368 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3c020070 con 0x7ffa48143670 2024-01-27T02:37:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.938+0000 7ffa4e985700 1 -- 172.21.15.7:0/995132368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa480c51d0 con 0x7ffa48143670 2024-01-27T02:37:45.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.938+0000 7ffa2ffff700 1 -- 172.21.15.7:0/995132368 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c01b6a0 con 0x7ffa48143670 2024-01-27T02:37:45.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.938+0000 7ffa4e985700 1 -- 172.21.15.7:0/995132368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa480c5790 con 0x7ffa48143670 2024-01-27T02:37:45.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.939+0000 7ffa4e985700 1 -- 172.21.15.7:0/995132368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa48072060 con 0x7ffa48143670 2024-01-27T02:37:45.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.940+0000 7ffa2ffff700 1 -- 172.21.15.7:0/995132368 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ffa3c022030 con 0x7ffa48143670 2024-01-27T02:37:45.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.940+0000 7ffa2ffff700 1 --2- 172.21.15.7:0/995132368 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffa34066f30 0x7ffa340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:45.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.940+0000 7ffa2ffff700 1 -- 172.21.15.7:0/995132368 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ffa3c086250 con 0x7ffa48143670 2024-01-27T02:37:45.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.941+0000 7ffa47fff700 1 --2- 172.21.15.7:0/995132368 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffa34066f30 0x7ffa340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:45.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.941+0000 7ffa47fff700 1 --2- 172.21.15.7:0/995132368 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffa34066f30 0x7ffa340693e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ffa38006fd0 tx=0x7ffa38008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:45.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:45.945+0000 7ffa2ffff700 1 -- 172.21.15.7:0/995132368 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa3c0537c0 con 0x7ffa48143670 2024-01-27T02:37:46.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.150+0000 7ffa4e985700 1 -- 172.21.15.7:0/995132368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ffa48072060 con 0x7ffa48143670 2024-01-27T02:37:46.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.154+0000 7ffa2ffff700 1 -- 172.21.15.7:0/995132368 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7ffa3c056de0 con 0x7ffa48143670 2024-01-27T02:37:46.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.156+0000 7ffa4e985700 1 -- 172.21.15.7:0/995132368 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffa34066f30 msgr2=0x7ffa340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:46.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.156+0000 7ffa4e985700 1 --2- 172.21.15.7:0/995132368 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffa34066f30 0x7ffa340693e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ffa38006fd0 tx=0x7ffa38008040 comp rx=0 tx=0).stop 2024-01-27T02:37:46.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.156+0000 7ffa4e985700 1 -- 172.21.15.7:0/995132368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa48143670 msgr2=0x7ffa480b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:46.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.156+0000 7ffa4e985700 1 --2- 172.21.15.7:0/995132368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa48143670 0x7ffa480b1b50 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0030c0 tx=0x7ffa3c016000 comp rx=0 tx=0).stop 2024-01-27T02:37:46.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.156+0000 7ffa4e985700 1 -- 172.21.15.7:0/995132368 shutdown_connections 2024-01-27T02:37:46.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.156+0000 7ffa4e985700 1 --2- 172.21.15.7:0/995132368 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffa34066f30 0x7ffa340693e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:46.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.156+0000 7ffa4e985700 1 --2- 172.21.15.7:0/995132368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa48143a40 0x7ffa480b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:46.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.156+0000 7ffa4e985700 1 --2- 172.21.15.7:0/995132368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa480aa8b0 0x7ffa4806b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:46.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.156+0000 7ffa4e985700 1 --2- 172.21.15.7:0/995132368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa48143670 0x7ffa480b1b50 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:46.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.156+0000 7ffa4e985700 1 -- 172.21.15.7:0/995132368 >> 172.21.15.7:0/995132368 conn(0x7ffa480cdbf0 msgr2=0x7ffa4815fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:46.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.156+0000 7ffa4e985700 1 -- 172.21.15.7:0/995132368 shutdown_connections 2024-01-27T02:37:46.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.156+0000 7ffa4e985700 1 -- 172.21.15.7:0/995132368 wait complete. 2024-01-27T02:37:46.171 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-27T02:37:46.171 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:37:46.222 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:37:46.267 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-27T02:37:46.268 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-27T02:37:46.268 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:37:46.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.405+0000 7fa93b82c700 1 Processor -- start 2024-01-27T02:37:46.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.405+0000 7fa93b82c700 1 -- start start 2024-01-27T02:37:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.406+0000 7fa93b82c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa934159f40 0x7fa9340c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.406+0000 7fa93b82c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9340c30c0 con 0x7fa9340c3a90 2024-01-27T02:37:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.406+0000 7fa93b82c700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa93415e960 con 0x7fa9340aafc0 2024-01-27T02:37:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.406+0000 7fa93b82c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9340c4550 con 0x7fa934159f40 2024-01-27T02:37:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.406+0000 7fa9395c8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa934159f40 0x7fa9340c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.406+0000 7fa9395c8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa934159f40 0x7fa9340c3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44556/0 (socket says 172.21.15.7:44556) 2024-01-27T02:37:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.406+0000 7fa938dc7700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa9340c3a90 0x7fa9340aa8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33496/0 (socket says 172.21.15.7:33496) 2024-01-27T02:37:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.406+0000 7fa9395c8700 1 -- 172.21.15.7:0/4057818088 learned_addr learned my addr 172.21.15.7:0/4057818088 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.406+0000 7fa9395c8700 1 -- 172.21.15.7:0/4057818088 >> v1:172.21.15.188:6789/0 conn(0x7fa9340aafc0 legacy=0x7fa934063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.406+0000 7fa9395c8700 1 -- 172.21.15.7:0/4057818088 >> v1:172.21.15.7:6789/0 conn(0x7fa9340c3a90 legacy=0x7fa9340aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.406+0000 7fa9395c8700 1 -- 172.21.15.7:0/4057818088 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93415ec40 con 0x7fa934159f40 2024-01-27T02:37:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.406+0000 7fa92bfff700 1 -- 172.21.15.7:0/4057818088 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2413988586 0 0) 0x7fa9340c30c0 con 0x7fa9340c3a90 2024-01-27T02:37:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.406+0000 7fa9395c8700 1 --2- 172.21.15.7:0/4057818088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa934159f40 0x7fa9340c3550 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fa9300099b0 tx=0x7fa930009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a9cc5110a32d5f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.407+0000 7fa92bfff700 1 -- 172.21.15.7:0/4057818088 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93000b860 con 0x7fa934159f40 2024-01-27T02:37:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.407+0000 7fa92bfff700 1 -- 172.21.15.7:0/4057818088 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93000b9c0 con 0x7fa934159f40 2024-01-27T02:37:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.407+0000 7fa92bfff700 1 -- 172.21.15.7:0/4057818088 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93000bd60 con 0x7fa934159f40 2024-01-27T02:37:46.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.407+0000 7fa93b82c700 1 -- 172.21.15.7:0/4057818088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa934159f40 msgr2=0x7fa9340c3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:46.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.407+0000 7fa93b82c700 1 --2- 172.21.15.7:0/4057818088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa934159f40 0x7fa9340c3550 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fa9300099b0 tx=0x7fa930009250 comp rx=0 tx=0).stop 2024-01-27T02:37:46.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.407+0000 7fa93b82c700 1 -- 172.21.15.7:0/4057818088 shutdown_connections 2024-01-27T02:37:46.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.407+0000 7fa93b82c700 1 --2- 172.21.15.7:0/4057818088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa934159f40 0x7fa9340c3550 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:46.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.407+0000 7fa93b82c700 1 -- 172.21.15.7:0/4057818088 >> 172.21.15.7:0/4057818088 conn(0x7fa9340cdc30 msgr2=0x7fa9340ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:46.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.407+0000 7fa93b82c700 1 -- 172.21.15.7:0/4057818088 shutdown_connections 2024-01-27T02:37:46.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.407+0000 7fa93b82c700 1 -- 172.21.15.7:0/4057818088 wait complete. 2024-01-27T02:37:46.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.408+0000 7fa93b82c700 1 Processor -- start 2024-01-27T02:37:46.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.409+0000 7fa93b82c700 1 -- start start 2024-01-27T02:37:46.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.410+0000 7fa93b82c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9340aafc0 0x7fa934150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:46.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.410+0000 7fa93b82c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9340c3a90 0x7fa93416afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:46.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.410+0000 7fa93b82c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa934159f40 0x7fa93415a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:46.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.410+0000 7fa93b82c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93416a5c0 con 0x7fa934159f40 2024-01-27T02:37:46.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.410+0000 7fa93b82c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa934150ca0 con 0x7fa9340c3a90 2024-01-27T02:37:46.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.410+0000 7fa93b82c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93415ab60 con 0x7fa9340aafc0 2024-01-27T02:37:46.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.410+0000 7fa939dc9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa934159f40 0x7fa93415a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:46.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.410+0000 7fa939dc9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa934159f40 0x7fa93415a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44568/0 (socket says 172.21.15.7:44568) 2024-01-27T02:37:46.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.410+0000 7fa939dc9700 1 -- 172.21.15.7:0/3227316686 learned_addr learned my addr 172.21.15.7:0/3227316686 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:46.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.410+0000 7fa938dc7700 1 --2- 172.21.15.7:0/3227316686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9340c3a90 0x7fa93416afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:46.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.410+0000 7fa9395c8700 1 --2- 172.21.15.7:0/3227316686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9340aafc0 0x7fa934150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:46.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.410+0000 7fa939dc9700 1 -- 172.21.15.7:0/3227316686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9340aafc0 msgr2=0x7fa934150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.410+0000 7fa939dc9700 1 --2- 172.21.15.7:0/3227316686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9340aafc0 0x7fa934150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.410+0000 7fa939dc9700 1 -- 172.21.15.7:0/3227316686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9340c3a90 msgr2=0x7fa93416afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.410+0000 7fa939dc9700 1 --2- 172.21.15.7:0/3227316686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9340c3a90 0x7fa93416afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.410+0000 7fa939dc9700 1 -- 172.21.15.7:0/3227316686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9300094f0 con 0x7fa934159f40 2024-01-27T02:37:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.410+0000 7fa939dc9700 1 --2- 172.21.15.7:0/3227316686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa934159f40 0x7fa93415a310 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00c840 tx=0x7fa92c001610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.410+0000 7fa9297fa700 1 -- 172.21.15.7:0/3227316686 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c012630 con 0x7fa934159f40 2024-01-27T02:37:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.410+0000 7fa9297fa700 1 -- 172.21.15.7:0/3227316686 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92c0127b0 con 0x7fa934159f40 2024-01-27T02:37:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.410+0000 7fa93b82c700 1 -- 172.21.15.7:0/3227316686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa93415ae40 con 0x7fa934159f40 2024-01-27T02:37:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.411+0000 7fa9297fa700 1 -- 172.21.15.7:0/3227316686 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c012b70 con 0x7fa934159f40 2024-01-27T02:37:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.411+0000 7fa93b82c700 1 -- 172.21.15.7:0/3227316686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa93416aab0 con 0x7fa934159f40 2024-01-27T02:37:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.411+0000 7fa93b82c700 1 -- 172.21.15.7:0/3227316686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa93415b080 con 0x7fa934159f40 2024-01-27T02:37:46.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.412+0000 7fa9297fa700 1 -- 172.21.15.7:0/3227316686 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa92c01a3a0 con 0x7fa934159f40 2024-01-27T02:37:46.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.412+0000 7fa9297fa700 1 --2- 172.21.15.7:0/3227316686 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa918066f30 0x7fa9180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:46.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.413+0000 7fa9297fa700 1 -- 172.21.15.7:0/3227316686 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa92c0842e0 con 0x7fa934159f40 2024-01-27T02:37:46.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.413+0000 7fa9395c8700 1 --2- 172.21.15.7:0/3227316686 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa918066f30 0x7fa9180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:46.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.413+0000 7fa9395c8700 1 --2- 172.21.15.7:0/3227316686 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa918066f30 0x7fa9180693e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa9300099b0 tx=0x7fa930017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:46.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.417+0000 7fa9297fa700 1 -- 172.21.15.7:0/3227316686 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa93415b080 con 0x7fa934159f40 2024-01-27T02:37:46.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.670+0000 7fa93b82c700 1 -- 172.21.15.7:0/3227316686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa93415b030 con 0x7fa934159f40 2024-01-27T02:37:46.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.670+0000 7fa9297fa700 1 -- 172.21.15.7:0/3227316686 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+3517 (secure 0 0 0) 0x7fa92c051850 con 0x7fa934159f40 2024-01-27T02:37:46.672 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:46.672 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":51,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/450556982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":450556982},{"type":"v1","addr":"172.21.15.188:6836","nonce":450556982}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1711422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1711422},{"type":"v1","addr":"172.21.15.188:6837","nonce":1711422}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6294,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1110306600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1110306600},{"type":"v1","addr":"172.21.15.7:6833","nonce":1110306600}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6576,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1955283762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1955283762},{"type":"v1","addr":"172.21.15.7:6835","nonce":1955283762}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[]} 2024-01-27T02:37:46.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.672+0000 7fa93b82c700 1 -- 172.21.15.7:0/3227316686 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa918066f30 msgr2=0x7fa9180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:46.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.672+0000 7fa93b82c700 1 --2- 172.21.15.7:0/3227316686 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa918066f30 0x7fa9180693e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa9300099b0 tx=0x7fa930017000 comp rx=0 tx=0).stop 2024-01-27T02:37:46.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.672+0000 7fa93b82c700 1 -- 172.21.15.7:0/3227316686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa934159f40 msgr2=0x7fa93415a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:46.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.672+0000 7fa93b82c700 1 --2- 172.21.15.7:0/3227316686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa934159f40 0x7fa93415a310 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00c840 tx=0x7fa92c001610 comp rx=0 tx=0).stop 2024-01-27T02:37:46.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.673+0000 7fa93b82c700 1 -- 172.21.15.7:0/3227316686 shutdown_connections 2024-01-27T02:37:46.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.673+0000 7fa93b82c700 1 --2- 172.21.15.7:0/3227316686 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa918066f30 0x7fa9180693e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:46.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.673+0000 7fa93b82c700 1 --2- 172.21.15.7:0/3227316686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa934159f40 0x7fa93415a310 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:46.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.673+0000 7fa93b82c700 1 --2- 172.21.15.7:0/3227316686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9340aafc0 0x7fa934150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:46.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.673+0000 7fa93b82c700 1 --2- 172.21.15.7:0/3227316686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9340c3a90 0x7fa93416afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:46.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.673+0000 7fa93b82c700 1 -- 172.21.15.7:0/3227316686 >> 172.21.15.7:0/3227316686 conn(0x7fa9340cdc30 msgr2=0x7fa9341583c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:46.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.673+0000 7fa93b82c700 1 -- 172.21.15.7:0/3227316686 shutdown_connections 2024-01-27T02:37:46.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.673+0000 7fa93b82c700 1 -- 172.21.15.7:0/3227316686 wait complete. 2024-01-27T02:37:46.675 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 51 2024-01-27T02:37:46.688 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-27T02:37:46.688 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-27T02:37:46.688 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-27T02:37:46.688 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:37:46.688 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-27T02:37:46.688 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-27T02:37:46.689 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-27T02:37:46.689 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:37:46.689 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-27T02:37:46.689 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-27T02:37:46.689 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-27T02:37:46.689 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:37:46.689 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-27T02:37:46.690 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-27T02:37:46.690 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-27T02:37:46.690 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:37:46.697 INFO:tasks.ceph.mds.a:Stopped 2024-01-27T02:37:46.697 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-27T02:37:46.699 INFO:tasks.ceph.mds.c:Stopped 2024-01-27T02:37:46.699 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-27T02:37:46.701 INFO:tasks.ceph.mds.b:Stopped 2024-01-27T02:37:46.701 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-27T02:37:46.701 INFO:tasks.ceph.mds.d:Stopped 2024-01-27T02:37:46.702 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-27T02:37:46.740 INFO:tasks.ceph.mds.a:Started 2024-01-27T02:37:46.740 DEBUG:teuthology.parallel:result is None 2024-01-27T02:37:46.741 INFO:tasks.ceph.mds.c:Started 2024-01-27T02:37:46.741 DEBUG:teuthology.parallel:result is None 2024-01-27T02:37:46.743 INFO:tasks.ceph.mds.b:Started 2024-01-27T02:37:46.744 DEBUG:teuthology.parallel:result is None 2024-01-27T02:37:46.744 INFO:tasks.ceph.mds.d:Started 2024-01-27T02:37:46.745 DEBUG:teuthology.parallel:result is None 2024-01-27T02:37:46.745 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-27T02:37:46.827 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-01-27T02:37:46.827 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-01-27T02:37:46.829 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-27T02:37:46.829 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-27T02:37:46.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.883+0000 7f91bdd11700 1 Processor -- start 2024-01-27T02:37:46.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.883+0000 7f91bdd11700 1 -- start start 2024-01-27T02:37:46.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.884+0000 7f91bdd11700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f91b80c4cc0 0x7f91b80cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:46.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.884+0000 7f91bdd11700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91b80ca540 0x7f91b8163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:46.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.884+0000 7f91bdd11700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b80ce5e0 con 0x7f91b815a100 2024-01-27T02:37:46.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.884+0000 7f91bdd11700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b80cd9d0 con 0x7f91b80c4cc0 2024-01-27T02:37:46.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.884+0000 7f91bdd11700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b80cdcb0 con 0x7f91b80ca540 2024-01-27T02:37:46.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.884+0000 7f91b77fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f91b815a100 0x7f91b80c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33502/0 (socket says 172.21.15.7:33502) 2024-01-27T02:37:46.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.884+0000 7f91b77fe700 1 -- 172.21.15.7:0/678742453 learned_addr learned my addr 172.21.15.7:0/678742453 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:46.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.884+0000 7f91b6ffd700 1 --2- 172.21.15.7:0/678742453 >> v2:172.21.15.188:3300/0 conn(0x7f91b80c4cc0 0x7f91b80cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:46.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.884+0000 7f91b7fff700 1 --2- 172.21.15.7:0/678742453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91b80ca540 0x7f91b8163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:46.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.884+0000 7f91b67fc700 1 -- 172.21.15.7:0/678742453 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3311522822 0 0) 0x7f91b80ce5e0 con 0x7f91b815a100 2024-01-27T02:37:46.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.884+0000 7f91b67fc700 1 -- 172.21.15.7:0/678742453 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91a0003570 con 0x7f91b815a100 2024-01-27T02:37:46.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.884+0000 7f91b7fff700 1 -- 172.21.15.7:0/678742453 >> v2:172.21.15.188:3300/0 conn(0x7f91b80c4cc0 msgr2=0x7f91b80cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:46.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.884+0000 7f91b7fff700 1 --2- 172.21.15.7:0/678742453 >> v2:172.21.15.188:3300/0 conn(0x7f91b80c4cc0 0x7f91b80cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:46.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.884+0000 7f91b7fff700 1 -- 172.21.15.7:0/678742453 >> v1:172.21.15.7:6789/0 conn(0x7f91b815a100 legacy=0x7f91b80c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:46.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.884+0000 7f91b7fff700 1 -- 172.21.15.7:0/678742453 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b80cdf90 con 0x7f91b80ca540 2024-01-27T02:37:46.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.884+0000 7f91b6ffd700 1 --2- 172.21.15.7:0/678742453 >> v2:172.21.15.188:3300/0 conn(0x7f91b80c4cc0 0x7f91b80cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:37:46.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.885+0000 7f91b7fff700 1 --2- 172.21.15.7:0/678742453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91b80ca540 0x7f91b8163d90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009e60 tx=0x7f91b0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18a1ff53847be38e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:46.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.885+0000 7f91b67fc700 1 -- 172.21.15.7:0/678742453 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b000e070 con 0x7f91b80ca540 2024-01-27T02:37:46.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.885+0000 7f91b67fc700 1 -- 172.21.15.7:0/678742453 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b000bab0 con 0x7f91b80ca540 2024-01-27T02:37:46.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.885+0000 7f91b67fc700 1 -- 172.21.15.7:0/678742453 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b000be70 con 0x7f91b80ca540 2024-01-27T02:37:46.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.885+0000 7f91bdd11700 1 -- 172.21.15.7:0/678742453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91b80ca540 msgr2=0x7f91b8163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:46.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.885+0000 7f91bdd11700 1 --2- 172.21.15.7:0/678742453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91b80ca540 0x7f91b8163d90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009e60 tx=0x7f91b0009780 comp rx=0 tx=0).stop 2024-01-27T02:37:46.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.885+0000 7f91bdd11700 1 -- 172.21.15.7:0/678742453 shutdown_connections 2024-01-27T02:37:46.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.885+0000 7f91bdd11700 1 --2- 172.21.15.7:0/678742453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91b80ca540 0x7f91b8163d90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:46.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.885+0000 7f91bdd11700 1 --2- 172.21.15.7:0/678742453 >> v2:172.21.15.188:3300/0 conn(0x7f91b80c4cc0 0x7f91b80cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:46.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.885+0000 7f91bdd11700 1 -- 172.21.15.7:0/678742453 >> 172.21.15.7:0/678742453 conn(0x7f91b8098020 msgr2=0x7f91b80711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:46.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.885+0000 7f91bdd11700 1 -- 172.21.15.7:0/678742453 shutdown_connections 2024-01-27T02:37:46.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.885+0000 7f91bdd11700 1 -- 172.21.15.7:0/678742453 wait complete. 2024-01-27T02:37:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.887+0000 7f91bdd11700 1 Processor -- start 2024-01-27T02:37:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.887+0000 7f91bdd11700 1 -- start start 2024-01-27T02:37:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.888+0000 7f91bdd11700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91b80c4cc0 0x7f91b80c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.888+0000 7f91bdd11700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91b80ca540 0x7f91b80c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.888+0000 7f91bdd11700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b815a100 0x7f91b80c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.888+0000 7f91bdd11700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b80c6330 con 0x7f91b815a100 2024-01-27T02:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.888+0000 7f91bdd11700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b815d2b0 con 0x7f91b80ca540 2024-01-27T02:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.888+0000 7f91bdd11700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b80c6560 con 0x7f91b80c4cc0 2024-01-27T02:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.888+0000 7f91b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b815a100 0x7f91b80c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.888+0000 7f91b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b815a100 0x7f91b80c6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44592/0 (socket says 172.21.15.7:44592) 2024-01-27T02:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.888+0000 7f91b7fff700 1 -- 172.21.15.7:0/2808989556 learned_addr learned my addr 172.21.15.7:0/2808989556 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.888+0000 7f91b6ffd700 1 --2- 172.21.15.7:0/2808989556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91b80ca540 0x7f91b80c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.888+0000 7f91b77fe700 1 --2- 172.21.15.7:0/2808989556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91b80c4cc0 0x7f91b80c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.888+0000 7f91b7fff700 1 -- 172.21.15.7:0/2808989556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91b80c4cc0 msgr2=0x7f91b80c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.888+0000 7f91b7fff700 1 --2- 172.21.15.7:0/2808989556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91b80c4cc0 0x7f91b80c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.888+0000 7f91b7fff700 1 -- 172.21.15.7:0/2808989556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91b80ca540 msgr2=0x7f91b80c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.888+0000 7f91b7fff700 1 --2- 172.21.15.7:0/2808989556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91b80ca540 0x7f91b80c5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.888+0000 7f91b7fff700 1 -- 172.21.15.7:0/2808989556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a8009420 con 0x7f91b815a100 2024-01-27T02:37:46.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.888+0000 7f91b6ffd700 1 --2- 172.21.15.7:0/2808989556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91b80ca540 0x7f91b80c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:37:46.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.888+0000 7f91b7fff700 1 --2- 172.21.15.7:0/2808989556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b815a100 0x7f91b80c6c50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f91b80cddf0 tx=0x7f91b0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:46.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.889+0000 7f919f7fe700 1 -- 172.21.15.7:0/2808989556 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0027470 con 0x7f91b815a100 2024-01-27T02:37:46.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.889+0000 7f919f7fe700 1 -- 172.21.15.7:0/2808989556 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b00275f0 con 0x7f91b815a100 2024-01-27T02:37:46.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.889+0000 7f91bdd11700 1 -- 172.21.15.7:0/2808989556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b0009a20 con 0x7f91b815a100 2024-01-27T02:37:46.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.889+0000 7f919f7fe700 1 -- 172.21.15.7:0/2808989556 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b00279b0 con 0x7f91b815a100 2024-01-27T02:37:46.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.889+0000 7f91bdd11700 1 -- 172.21.15.7:0/2808989556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b815c7d0 con 0x7f91b815a100 2024-01-27T02:37:46.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.890+0000 7f91bdd11700 1 -- 172.21.15.7:0/2808989556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b804c710 con 0x7f91b815a100 2024-01-27T02:37:46.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.890+0000 7f919f7fe700 1 -- 172.21.15.7:0/2808989556 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f91b0024070 con 0x7f91b815a100 2024-01-27T02:37:46.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.891+0000 7f919f7fe700 1 --2- 172.21.15.7:0/2808989556 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f91a4066f30 0x7f91a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:46.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.891+0000 7f919f7fe700 1 -- 172.21.15.7:0/2808989556 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f91b0086c60 con 0x7f91b815a100 2024-01-27T02:37:46.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.891+0000 7f91b77fe700 1 --2- 172.21.15.7:0/2808989556 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f91a4066f30 0x7f91a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:46.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.891+0000 7f91b77fe700 1 --2- 172.21.15.7:0/2808989556 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f91a4066f30 0x7f91a40693e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0058d0 tx=0x7f91ac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:46.895+0000 7f919f7fe700 1 -- 172.21.15.7:0/2808989556 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91b00541d0 con 0x7f91b815a100 2024-01-27T02:37:47.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.116+0000 7f91bdd11700 1 -- 172.21.15.7:0/2808989556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f91b80ce270 con 0x7f91b815a100 2024-01-27T02:37:47.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.784+0000 7f919f7fe700 1 -- 172.21.15.7:0/2808989556 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v111) v1 ==== 97+0+0 (secure 0 0 0) 0x7f91b00577f0 con 0x7f91b815a100 2024-01-27T02:37:47.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.787+0000 7f91bdd11700 1 -- 172.21.15.7:0/2808989556 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f91a4066f30 msgr2=0x7f91a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:47.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.787+0000 7f91bdd11700 1 --2- 172.21.15.7:0/2808989556 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f91a4066f30 0x7f91a40693e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0058d0 tx=0x7f91ac009040 comp rx=0 tx=0).stop 2024-01-27T02:37:47.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.787+0000 7f91bdd11700 1 -- 172.21.15.7:0/2808989556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b815a100 msgr2=0x7f91b80c6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:47.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.787+0000 7f91bdd11700 1 --2- 172.21.15.7:0/2808989556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b815a100 0x7f91b80c6c50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f91b80cddf0 tx=0x7f91b0019000 comp rx=0 tx=0).stop 2024-01-27T02:37:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.788+0000 7f91bdd11700 1 -- 172.21.15.7:0/2808989556 shutdown_connections 2024-01-27T02:37:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.788+0000 7f91bdd11700 1 --2- 172.21.15.7:0/2808989556 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f91a4066f30 0x7f91a40693e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.788+0000 7f91bdd11700 1 --2- 172.21.15.7:0/2808989556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b815a100 0x7f91b80c6c50 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.788+0000 7f91bdd11700 1 --2- 172.21.15.7:0/2808989556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91b80c4cc0 0x7f91b80c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.788+0000 7f91bdd11700 1 --2- 172.21.15.7:0/2808989556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91b80ca540 0x7f91b80c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.788+0000 7f91bdd11700 1 -- 172.21.15.7:0/2808989556 >> 172.21.15.7:0/2808989556 conn(0x7f91b8098020 msgr2=0x7f91b80c9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.789+0000 7f91bdd11700 1 -- 172.21.15.7:0/2808989556 shutdown_connections 2024-01-27T02:37:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.789+0000 7f91bdd11700 1 -- 172.21.15.7:0/2808989556 wait complete. 2024-01-27T02:37:47.791 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-01-27T02:37:47.807 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-27T02:37:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.941+0000 7f97a85e4700 1 Processor -- start 2024-01-27T02:37:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.941+0000 7f97a85e4700 1 -- start start 2024-01-27T02:37:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.942+0000 7f97a85e4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a015a100 0x7f97a00cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.942+0000 7f97a85e4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f97a00cb5b0 0x7f97a00c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.942+0000 7f97a85e4700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f97a00c4af0 0x7f97a00c9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.942+0000 7f97a85e4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a00c4f70 con 0x7f97a00cb5b0 2024-01-27T02:37:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.942+0000 7f97a85e4700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a00ce5e0 con 0x7f97a00c4af0 2024-01-27T02:37:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.942+0000 7f97a85e4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a00cd9d0 con 0x7f97a015a100 2024-01-27T02:37:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.942+0000 7f97a5b7f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f97a00cb5b0 0x7f97a00c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.942+0000 7f97a5b7f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f97a00cb5b0 0x7f97a00c45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44606/0 (socket says 172.21.15.7:44606) 2024-01-27T02:37:47.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.942+0000 7f97a5b7f700 1 -- 172.21.15.7:0/3249533666 learned_addr learned my addr 172.21.15.7:0/3249533666 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:47.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.943+0000 7f97a6b81700 1 --2- 172.21.15.7:0/3249533666 >> v2:172.21.15.188:3300/0 conn(0x7f97a00c4af0 0x7f97a00c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:47.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.943+0000 7f97a6380700 1 --2- 172.21.15.7:0/3249533666 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a015a100 0x7f97a00cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:47.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.943+0000 7f97a5b7f700 1 -- 172.21.15.7:0/3249533666 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a015a100 msgr2=0x7f97a00cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:47.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.943+0000 7f97a5b7f700 1 --2- 172.21.15.7:0/3249533666 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a015a100 0x7f97a00cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:47.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.943+0000 7f97a5b7f700 1 -- 172.21.15.7:0/3249533666 >> v2:172.21.15.188:3300/0 conn(0x7f97a00c4af0 msgr2=0x7f97a00c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:47.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.943+0000 7f97a5b7f700 1 --2- 172.21.15.7:0/3249533666 >> v2:172.21.15.188:3300/0 conn(0x7f97a00c4af0 0x7f97a00c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:47.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.943+0000 7f97a5b7f700 1 -- 172.21.15.7:0/3249533666 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a00cdcb0 con 0x7f97a00cb5b0 2024-01-27T02:37:47.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.943+0000 7f97a5b7f700 1 --2- 172.21.15.7:0/3249533666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97a00cb5b0 0x7f97a00c45b0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f97a006e7a0 tx=0x7f9790009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4f30028a75d7f2 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:47.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.943+0000 7f97a537e700 1 -- 172.21.15.7:0/3249533666 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979000b860 con 0x7f97a00cb5b0 2024-01-27T02:37:47.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.943+0000 7f97a537e700 1 -- 172.21.15.7:0/3249533666 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979000b9c0 con 0x7f97a00cb5b0 2024-01-27T02:37:47.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.943+0000 7f97a537e700 1 -- 172.21.15.7:0/3249533666 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979000bd60 con 0x7f97a00cb5b0 2024-01-27T02:37:47.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.944+0000 7f97a85e4700 1 -- 172.21.15.7:0/3249533666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97a00cb5b0 msgr2=0x7f97a00c45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:47.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.944+0000 7f97a85e4700 1 --2- 172.21.15.7:0/3249533666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97a00cb5b0 0x7f97a00c45b0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f97a006e7a0 tx=0x7f9790009250 comp rx=0 tx=0).stop 2024-01-27T02:37:47.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.944+0000 7f97a85e4700 1 -- 172.21.15.7:0/3249533666 shutdown_connections 2024-01-27T02:37:47.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.944+0000 7f97a85e4700 1 --2- 172.21.15.7:0/3249533666 >> v2:172.21.15.188:3300/0 conn(0x7f97a00c4af0 0x7f97a00c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:47.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.944+0000 7f97a85e4700 1 --2- 172.21.15.7:0/3249533666 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a015a100 0x7f97a00cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.944+0000 7f97a85e4700 1 --2- 172.21.15.7:0/3249533666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97a00cb5b0 0x7f97a00c45b0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.944+0000 7f97a85e4700 1 -- 172.21.15.7:0/3249533666 >> 172.21.15.7:0/3249533666 conn(0x7f97a0098020 msgr2=0x7f97a00711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.944+0000 7f97a85e4700 1 -- 172.21.15.7:0/3249533666 shutdown_connections 2024-01-27T02:37:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.944+0000 7f97a85e4700 1 -- 172.21.15.7:0/3249533666 wait complete. 2024-01-27T02:37:47.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.945+0000 7f97a85e4700 1 Processor -- start 2024-01-27T02:37:47.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.945+0000 7f97a85e4700 1 -- start start 2024-01-27T02:37:47.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.946+0000 7f97a85e4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97a00c4af0 0x7f97a00c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:47.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.946+0000 7f97a85e4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a00cb5b0 0x7f97a00c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:47.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.946+0000 7f97a85e4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a015a100 0x7f97a016add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:47.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.946+0000 7f97a85e4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a016a6d0 con 0x7f97a00c4af0 2024-01-27T02:37:47.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.946+0000 7f97a85e4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a016a940 con 0x7f97a015a100 2024-01-27T02:37:47.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.946+0000 7f97a85e4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a00c6cb0 con 0x7f97a00cb5b0 2024-01-27T02:37:47.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.946+0000 7f97a6380700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97a00c4af0 0x7f97a00c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:47.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.946+0000 7f97a6380700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97a00c4af0 0x7f97a00c8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44608/0 (socket says 172.21.15.7:44608) 2024-01-27T02:37:47.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.946+0000 7f97a6380700 1 -- 172.21.15.7:0/4114697172 learned_addr learned my addr 172.21.15.7:0/4114697172 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:47.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.946+0000 7f97a6b81700 1 --2- 172.21.15.7:0/4114697172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a015a100 0x7f97a016add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:47.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.946+0000 7f97a5b7f700 1 --2- 172.21.15.7:0/4114697172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a00cb5b0 0x7f97a00c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:47.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.946+0000 7f97a6380700 1 -- 172.21.15.7:0/4114697172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a00cb5b0 msgr2=0x7f97a00c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:47.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.946+0000 7f97a6380700 1 --2- 172.21.15.7:0/4114697172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a00cb5b0 0x7f97a00c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:47.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.946+0000 7f97a6380700 1 -- 172.21.15.7:0/4114697172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a015a100 msgr2=0x7f97a016add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:47.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.946+0000 7f97a6380700 1 --2- 172.21.15.7:0/4114697172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a015a100 0x7f97a016add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:47.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.946+0000 7f97a6380700 1 -- 172.21.15.7:0/4114697172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97900094f0 con 0x7f97a00c4af0 2024-01-27T02:37:47.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.946+0000 7f97a6b81700 1 --2- 172.21.15.7:0/4114697172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a015a100 0x7f97a016add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:37:47.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.946+0000 7f97a6380700 1 --2- 172.21.15.7:0/4114697172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97a00c4af0 0x7f97a00c8e50 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f979c00d890 tx=0x7f979c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:47.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.947+0000 7f97967fc700 1 -- 172.21.15.7:0/4114697172 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c00b600 con 0x7f97a00c4af0 2024-01-27T02:37:47.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.947+0000 7f97967fc700 1 -- 172.21.15.7:0/4114697172 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979c00b760 con 0x7f97a00c4af0 2024-01-27T02:37:47.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.947+0000 7f97a85e4700 1 -- 172.21.15.7:0/4114697172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a009c870 con 0x7f97a00c4af0 2024-01-27T02:37:47.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.947+0000 7f97967fc700 1 -- 172.21.15.7:0/4114697172 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c00bae0 con 0x7f97a00c4af0 2024-01-27T02:37:47.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.947+0000 7f97a85e4700 1 -- 172.21.15.7:0/4114697172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a009c590 con 0x7f97a00c4af0 2024-01-27T02:37:47.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.948+0000 7f97a85e4700 1 -- 172.21.15.7:0/4114697172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a004c710 con 0x7f97a00c4af0 2024-01-27T02:37:47.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.948+0000 7f97967fc700 1 -- 172.21.15.7:0/4114697172 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f979c01e460 con 0x7f97a00c4af0 2024-01-27T02:37:47.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.949+0000 7f97967fc700 1 --2- 172.21.15.7:0/4114697172 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9784066f30 0x7f97840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:47.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.949+0000 7f97967fc700 1 -- 172.21.15.7:0/4114697172 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f979c084800 con 0x7f97a00c4af0 2024-01-27T02:37:47.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.949+0000 7f97a5b7f700 1 --2- 172.21.15.7:0/4114697172 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9784066f30 0x7f97840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:47.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.949+0000 7f97a5b7f700 1 --2- 172.21.15.7:0/4114697172 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9784066f30 0x7f97840693e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f97a006e7a0 tx=0x7f9790017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:47.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:47.953+0000 7f97967fc700 1 -- 172.21.15.7:0/4114697172 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f979c051dc0 con 0x7f97a00c4af0 2024-01-27T02:37:48.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.241+0000 7f97a85e4700 1 -- 172.21.15.7:0/4114697172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f97a00cdf90 con 0x7f97a00c4af0 2024-01-27T02:37:48.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.242+0000 7f97967fc700 1 -- 172.21.15.7:0/4114697172 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f979c0553e0 con 0x7f97a00c4af0 2024-01-27T02:37:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:48.244 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:37:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-01-27T02:37:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-01-27T02:37:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlHdcPKBAAfhnh1I3ZSQEXWJXlXCs98Q==", 2024-01-27T02:37:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:37:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:37:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:37:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:37:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:37:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-01-27T02:37:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rl8eVYNRAAfP8rOqO50RpoNZ1+1j2DXQ==", 2024-01-27T02:37:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:37:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:37:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:37:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:37:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:37:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-01-27T02:37:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlJxRKMBAAS0j7/zZeqGglOxCK1pg0WA==", 2024-01-27T02:37:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:37:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:37:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:37:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:37:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:37:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-01-27T02:37:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlNwfUABAALRQAoHoGhniqCvTQOqbbug==", 2024-01-27T02:37:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:37:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:37:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:37:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:37:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:37:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-01-27T02:37:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7Rlh6VSJhAAttuCte1WhNEvUkv5x/IKQQ==", 2024-01-27T02:37:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:37:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:37:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:37:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:37:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-01-27T02:37:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7Rlu/RGLhAAKzaCNhzYKhsNhxkMvY++gw==", 2024-01-27T02:37:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:37:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:37:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:37:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:37:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-01-27T02:37:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7RliRMyNhAAr0tG3evmVV7ZlGxbxfWPMw==", 2024-01-27T02:37:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:37:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:37:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:37:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:37:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-01-27T02:37:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAPa7RlRuSNAhAA3e9atlb+ffvlnLhRYbWZPw==", 2024-01-27T02:37:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:37:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:37:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:37:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:37:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-01-27T02:37:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7RlfeA6DBAAXpklgbNHRO+5G+fdIERjRw==", 2024-01-27T02:37:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:37:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:37:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:37:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:37:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-01-27T02:37:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7Rl77ItExAABB7uABm63ubE/YUs2B5pGw==", 2024-01-27T02:37:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:37:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:37:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:37:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:37:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-01-27T02:37:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7RljDuwHBAAPPAlhoqS03qPpq58jmevLA==", 2024-01-27T02:37:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:37:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:37:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:37:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:37:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-01-27T02:37:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7Rl1YElJhAAqHP/QEdl1SEalGcbQwPwAg==", 2024-01-27T02:37:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:37:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:37:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:37:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:37:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-01-27T02:37:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlaEqaBRAATFiIeW8lT9BdNuN0m3x9dQ==", 2024-01-27T02:37:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:37:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:37:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-01-27T02:37:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-27T02:37:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-01-27T02:37:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlNADmEBAAJXwdJ35ylP5coy1dtHoD3g==", 2024-01-27T02:37:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:37:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:37:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:37:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:37:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:37:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-01-27T02:37:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7Rl5fRqCRAA/5swTGknnpP64TAh+2X41A==", 2024-01-27T02:37:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:37:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:37:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:37:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:37:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:37:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-01-27T02:37:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7Rl0SdwFhAAhCmw9ViiJbHVfS5a2r/D+w==", 2024-01-27T02:37:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:37:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:37:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:37:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:37:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:37:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-01-27T02:37:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlsNgPARAA0Qm0oJgA8GO1oUT3ndT8Tg==", 2024-01-27T02:37:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:37:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:37:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:37:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:37:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:37:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-01-27T02:37:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rlfb7qIxAAlsJYiSolJ7D8EykGWP62bA==", 2024-01-27T02:37:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:37:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:37:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-27T02:37:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:37:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:37:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-01-27T02:37:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rls7TaHxAALS3ghxKtfmgQfIELiD9+Bg==", 2024-01-27T02:37:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:37:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:37:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-27T02:37:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:37:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:37:48.259 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:37:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.244+0000 7f97a85e4700 1 -- 172.21.15.7:0/4114697172 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9784066f30 msgr2=0x7f97840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.244+0000 7f97a85e4700 1 --2- 172.21.15.7:0/4114697172 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9784066f30 0x7f97840693e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f97a006e7a0 tx=0x7f9790017000 comp rx=0 tx=0).stop 2024-01-27T02:37:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.244+0000 7f97a85e4700 1 -- 172.21.15.7:0/4114697172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97a00c4af0 msgr2=0x7f97a00c8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.244+0000 7f97a85e4700 1 --2- 172.21.15.7:0/4114697172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97a00c4af0 0x7f97a00c8e50 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f979c00d890 tx=0x7f979c008040 comp rx=0 tx=0).stop 2024-01-27T02:37:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.244+0000 7f97a85e4700 1 -- 172.21.15.7:0/4114697172 shutdown_connections 2024-01-27T02:37:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.244+0000 7f97a85e4700 1 --2- 172.21.15.7:0/4114697172 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9784066f30 0x7f97840693e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.244+0000 7f97a85e4700 1 --2- 172.21.15.7:0/4114697172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a015a100 0x7f97a016add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.244+0000 7f97a85e4700 1 --2- 172.21.15.7:0/4114697172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97a00c4af0 0x7f97a00c8e50 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.244+0000 7f97a85e4700 1 --2- 172.21.15.7:0/4114697172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a00cb5b0 0x7f97a00c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.244+0000 7f97a85e4700 1 -- 172.21.15.7:0/4114697172 >> 172.21.15.7:0/4114697172 conn(0x7f97a0098020 msgr2=0x7f97a0067cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:48.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.244+0000 7f97a85e4700 1 -- 172.21.15.7:0/4114697172 shutdown_connections 2024-01-27T02:37:48.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.244+0000 7f97a85e4700 1 -- 172.21.15.7:0/4114697172 wait complete. 2024-01-27T02:37:48.262 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:37:48.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.392+0000 7f68deb3d700 1 Processor -- start 2024-01-27T02:37:48.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.392+0000 7f68deb3d700 1 -- start start 2024-01-27T02:37:48.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.393+0000 7f68deb3d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68d80a7c10 0x7f68d80ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:48.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.393+0000 7f68deb3d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f68d80a7fe0 0x7f68d80b0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:48.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.393+0000 7f68deb3d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d815a880 con 0x7f68d80ae880 2024-01-27T02:37:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.393+0000 7f68deb3d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d80afde0 con 0x7f68d80a7fe0 2024-01-27T02:37:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.393+0000 7f68deb3d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d80af950 con 0x7f68d80a7c10 2024-01-27T02:37:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.393+0000 7f68dd0da700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f68d80a7fe0 0x7f68d80b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.393+0000 7f68d7fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f68d80ae880 0x7f68d806f360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33512/0 (socket says 172.21.15.7:33512) 2024-01-27T02:37:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.393+0000 7f68d7fff700 1 -- 172.21.15.7:0/2196299069 learned_addr learned my addr 172.21.15.7:0/2196299069 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.393+0000 7f68dc8d9700 1 --2- 172.21.15.7:0/2196299069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68d80a7c10 0x7f68d80ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.393+0000 7f68dd0da700 1 -- 172.21.15.7:0/2196299069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68d80a7c10 msgr2=0x7f68d80ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.393+0000 7f68dd0da700 1 --2- 172.21.15.7:0/2196299069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68d80a7c10 0x7f68d80ae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.393+0000 7f68dd0da700 1 -- 172.21.15.7:0/2196299069 >> v1:172.21.15.7:6789/0 conn(0x7f68d80ae880 legacy=0x7f68d806f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.393+0000 7f68dd0da700 1 -- 172.21.15.7:0/2196299069 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d80aed40 con 0x7f68d80a7fe0 2024-01-27T02:37:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.393+0000 7f68d77fe700 1 -- 172.21.15.7:0/2196299069 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2038815190 0 0) 0x7f68d815a880 con 0x7f68d80ae880 2024-01-27T02:37:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.394+0000 7f68dd0da700 1 --2- 172.21.15.7:0/2196299069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d80a7fe0 0x7f68d80b0180 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009e60 tx=0x7f68d0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372a52061a35839e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.394+0000 7f68d77fe700 1 -- 172.21.15.7:0/2196299069 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d000e070 con 0x7f68d80a7fe0 2024-01-27T02:37:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.394+0000 7f68d77fe700 1 -- 172.21.15.7:0/2196299069 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d000bc30 con 0x7f68d80a7fe0 2024-01-27T02:37:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.394+0000 7f68d77fe700 1 -- 172.21.15.7:0/2196299069 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d0012430 con 0x7f68d80a7fe0 2024-01-27T02:37:48.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.394+0000 7f68deb3d700 1 -- 172.21.15.7:0/2196299069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d80a7fe0 msgr2=0x7f68d80b0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:48.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.394+0000 7f68deb3d700 1 --2- 172.21.15.7:0/2196299069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d80a7fe0 0x7f68d80b0180 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009e60 tx=0x7f68d0009780 comp rx=0 tx=0).stop 2024-01-27T02:37:48.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.394+0000 7f68deb3d700 1 -- 172.21.15.7:0/2196299069 shutdown_connections 2024-01-27T02:37:48.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.394+0000 7f68deb3d700 1 --2- 172.21.15.7:0/2196299069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d80a7fe0 0x7f68d80b0180 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.394+0000 7f68deb3d700 1 --2- 172.21.15.7:0/2196299069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68d80a7c10 0x7f68d80ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.394+0000 7f68deb3d700 1 -- 172.21.15.7:0/2196299069 >> 172.21.15.7:0/2196299069 conn(0x7f68d80679b0 msgr2=0x7f68d8067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:48.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.395+0000 7f68deb3d700 1 -- 172.21.15.7:0/2196299069 shutdown_connections 2024-01-27T02:37:48.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.395+0000 7f68deb3d700 1 -- 172.21.15.7:0/2196299069 wait complete. 2024-01-27T02:37:48.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.396+0000 7f68deb3d700 1 Processor -- start 2024-01-27T02:37:48.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.396+0000 7f68deb3d700 1 -- start start 2024-01-27T02:37:48.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.397+0000 7f68deb3d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68d80a7c10 0x7f68d8145d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:48.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.397+0000 7f68deb3d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68d80a7fe0 0x7f68d816ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:48.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.397+0000 7f68deb3d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d80ae880 0x7f68d80c2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.397+0000 7f68deb3d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d8146260 con 0x7f68d80ae880 2024-01-27T02:37:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.397+0000 7f68deb3d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d816e7a0 con 0x7f68d80a7c10 2024-01-27T02:37:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.397+0000 7f68deb3d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d816ea80 con 0x7f68d80a7fe0 2024-01-27T02:37:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.397+0000 7f68dd0da700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d80ae880 0x7f68d80c2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.397+0000 7f68dd0da700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d80ae880 0x7f68d80c2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44622/0 (socket says 172.21.15.7:44622) 2024-01-27T02:37:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.397+0000 7f68dd0da700 1 -- 172.21.15.7:0/717939617 learned_addr learned my addr 172.21.15.7:0/717939617 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.397+0000 7f68d7fff700 1 --2- 172.21.15.7:0/717939617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68d80a7fe0 0x7f68d816ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.397+0000 7f68dc8d9700 1 --2- 172.21.15.7:0/717939617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68d80a7c10 0x7f68d8145d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.397+0000 7f68dd0da700 1 -- 172.21.15.7:0/717939617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68d80a7fe0 msgr2=0x7f68d816ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.397+0000 7f68dd0da700 1 --2- 172.21.15.7:0/717939617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68d80a7fe0 0x7f68d816ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.397+0000 7f68dd0da700 1 -- 172.21.15.7:0/717939617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68d80a7c10 msgr2=0x7f68d8145d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.397+0000 7f68dd0da700 1 --2- 172.21.15.7:0/717939617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68d80a7c10 0x7f68d8145d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.397+0000 7f68dd0da700 1 -- 172.21.15.7:0/717939617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d0009a20 con 0x7f68d80ae880 2024-01-27T02:37:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.397+0000 7f68dd0da700 1 --2- 172.21.15.7:0/717939617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d80ae880 0x7f68d80c2160 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f68d00097c0 tx=0x7f68d0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.398+0000 7f68d4ff9700 1 -- 172.21.15.7:0/717939617 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d0027470 con 0x7f68d80ae880 2024-01-27T02:37:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.398+0000 7f68d4ff9700 1 -- 172.21.15.7:0/717939617 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d00275d0 con 0x7f68d80ae880 2024-01-27T02:37:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.398+0000 7f68deb3d700 1 -- 172.21.15.7:0/717939617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d8099bd0 con 0x7f68d80ae880 2024-01-27T02:37:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.398+0000 7f68d4ff9700 1 -- 172.21.15.7:0/717939617 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d0027990 con 0x7f68d80ae880 2024-01-27T02:37:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.398+0000 7f68deb3d700 1 -- 172.21.15.7:0/717939617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d8158770 con 0x7f68d80ae880 2024-01-27T02:37:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.399+0000 7f68deb3d700 1 -- 172.21.15.7:0/717939617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d8158960 con 0x7f68d80ae880 2024-01-27T02:37:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.399+0000 7f68d4ff9700 1 -- 172.21.15.7:0/717939617 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f68d0024070 con 0x7f68d80ae880 2024-01-27T02:37:48.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.400+0000 7f68d4ff9700 1 --2- 172.21.15.7:0/717939617 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f68bc066f30 0x7f68bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:48.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.400+0000 7f68d4ff9700 1 -- 172.21.15.7:0/717939617 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f68d0086950 con 0x7f68d80ae880 2024-01-27T02:37:48.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.400+0000 7f68dc8d9700 1 --2- 172.21.15.7:0/717939617 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f68bc066f30 0x7f68bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:48.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.400+0000 7f68dc8d9700 1 --2- 172.21.15.7:0/717939617 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f68bc066f30 0x7f68bc0693e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f68cc006fd0 tx=0x7f68cc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:48.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.404+0000 7f68d4ff9700 1 -- 172.21.15.7:0/717939617 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68d8158960 con 0x7f68d80ae880 2024-01-27T02:37:48.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.610+0000 7f68deb3d700 1 -- 172.21.15.7:0/717939617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68d809a0f0 con 0x7f68d80ae880 2024-01-27T02:37:48.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.611+0000 7f68d4ff9700 1 -- 172.21.15.7:0/717939617 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+12089 (secure 0 0 0) 0x7f68d809a0f0 con 0x7f68d80ae880 2024-01-27T02:37:48.612 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:48.612 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":113,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:37:47.790919+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:37:48.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.613+0000 7f68deb3d700 1 -- 172.21.15.7:0/717939617 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f68bc066f30 msgr2=0x7f68bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:48.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.613+0000 7f68deb3d700 1 --2- 172.21.15.7:0/717939617 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f68bc066f30 0x7f68bc0693e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f68cc006fd0 tx=0x7f68cc008040 comp rx=0 tx=0).stop 2024-01-27T02:37:48.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.613+0000 7f68deb3d700 1 -- 172.21.15.7:0/717939617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d80ae880 msgr2=0x7f68d80c2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:48.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.613+0000 7f68deb3d700 1 --2- 172.21.15.7:0/717939617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d80ae880 0x7f68d80c2160 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f68d00097c0 tx=0x7f68d0019000 comp rx=0 tx=0).stop 2024-01-27T02:37:48.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.613+0000 7f68deb3d700 1 -- 172.21.15.7:0/717939617 shutdown_connections 2024-01-27T02:37:48.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.613+0000 7f68deb3d700 1 --2- 172.21.15.7:0/717939617 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f68bc066f30 0x7f68bc0693e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.613+0000 7f68deb3d700 1 --2- 172.21.15.7:0/717939617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d80ae880 0x7f68d80c2160 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.613+0000 7f68deb3d700 1 --2- 172.21.15.7:0/717939617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68d80a7c10 0x7f68d8145d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.613+0000 7f68deb3d700 1 --2- 172.21.15.7:0/717939617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68d80a7fe0 0x7f68d816ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.613+0000 7f68deb3d700 1 -- 172.21.15.7:0/717939617 >> 172.21.15.7:0/717939617 conn(0x7f68d80679b0 msgr2=0x7f68d8067d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:48.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.613+0000 7f68deb3d700 1 -- 172.21.15.7:0/717939617 shutdown_connections 2024-01-27T02:37:48.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.613+0000 7f68deb3d700 1 -- 172.21.15.7:0/717939617 wait complete. 2024-01-27T02:37:48.628 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-27T02:37:48.629 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:37:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.761+0000 7f40d78f7700 1 Processor -- start 2024-01-27T02:37:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.761+0000 7f40d78f7700 1 -- start start 2024-01-27T02:37:48.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.762+0000 7f40d78f7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40d00cba60 0x7f40d00ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:48.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.762+0000 7f40d78f7700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f40d00cbe30 0x7f40d015b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:48.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.762+0000 7f40d78f7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40d006b4e0 0x7f40d006b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:48.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.762+0000 7f40d78f7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d006bea0 con 0x7f40d006b4e0 2024-01-27T02:37:48.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.762+0000 7f40d78f7700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d00679b0 con 0x7f40d00cbe30 2024-01-27T02:37:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.762+0000 7f40d78f7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d0067c90 con 0x7f40d00cba60 2024-01-27T02:37:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.762+0000 7f40d5e94700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40d006b4e0 0x7f40d006b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.762+0000 7f40d5e94700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40d006b4e0 0x7f40d006b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44624/0 (socket says 172.21.15.7:44624) 2024-01-27T02:37:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.762+0000 7f40d5e94700 1 -- 172.21.15.7:0/496832079 learned_addr learned my addr 172.21.15.7:0/496832079 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.763+0000 7f40d4e92700 1 --2- 172.21.15.7:0/496832079 >> v2:172.21.15.188:3300/0 conn(0x7f40d00cbe30 0x7f40d015b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.763+0000 7f40d5693700 1 --2- 172.21.15.7:0/496832079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40d00cba60 0x7f40d00ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.763+0000 7f40d5e94700 1 -- 172.21.15.7:0/496832079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40d00cba60 msgr2=0x7f40d00ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:48.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.763+0000 7f40d5e94700 1 --2- 172.21.15.7:0/496832079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40d00cba60 0x7f40d00ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.763+0000 7f40d5e94700 1 -- 172.21.15.7:0/496832079 >> v2:172.21.15.188:3300/0 conn(0x7f40d00cbe30 msgr2=0x7f40d015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:48.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.763+0000 7f40d5e94700 1 --2- 172.21.15.7:0/496832079 >> v2:172.21.15.188:3300/0 conn(0x7f40d00cbe30 0x7f40d015b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.763+0000 7f40d5e94700 1 -- 172.21.15.7:0/496832079 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d015b7b0 con 0x7f40d006b4e0 2024-01-27T02:37:48.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.763+0000 7f40d5e94700 1 --2- 172.21.15.7:0/496832079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d006b4e0 0x7f40d006b960 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f40d00cfdb0 tx=0x7f40c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47729064c7027a6f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:48.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.763+0000 7f40c3fff700 1 -- 172.21.15.7:0/496832079 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c800e070 con 0x7f40d006b4e0 2024-01-27T02:37:48.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.763+0000 7f40c3fff700 1 -- 172.21.15.7:0/496832079 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c800b970 con 0x7f40d006b4e0 2024-01-27T02:37:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.763+0000 7f40c3fff700 1 -- 172.21.15.7:0/496832079 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c80092e0 con 0x7f40d006b4e0 2024-01-27T02:37:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.764+0000 7f40d78f7700 1 -- 172.21.15.7:0/496832079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d006b4e0 msgr2=0x7f40d006b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.764+0000 7f40d78f7700 1 --2- 172.21.15.7:0/496832079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d006b4e0 0x7f40d006b960 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f40d00cfdb0 tx=0x7f40c8009840 comp rx=0 tx=0).stop 2024-01-27T02:37:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.764+0000 7f40d78f7700 1 -- 172.21.15.7:0/496832079 shutdown_connections 2024-01-27T02:37:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.764+0000 7f40d78f7700 1 --2- 172.21.15.7:0/496832079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d006b4e0 0x7f40d006b960 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.764+0000 7f40d78f7700 1 --2- 172.21.15.7:0/496832079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40d00cba60 0x7f40d00ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.764+0000 7f40d78f7700 1 --2- 172.21.15.7:0/496832079 >> v2:172.21.15.188:3300/0 conn(0x7f40d00cbe30 0x7f40d015b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.764+0000 7f40d78f7700 1 -- 172.21.15.7:0/496832079 >> 172.21.15.7:0/496832079 conn(0x7f40d0063580 msgr2=0x7f40d0063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:48.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.764+0000 7f40d78f7700 1 -- 172.21.15.7:0/496832079 shutdown_connections 2024-01-27T02:37:48.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.764+0000 7f40d78f7700 1 -- 172.21.15.7:0/496832079 wait complete. 2024-01-27T02:37:48.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.765+0000 7f40d78f7700 1 Processor -- start 2024-01-27T02:37:48.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.765+0000 7f40d78f7700 1 -- start start 2024-01-27T02:37:48.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.766+0000 7f40d78f7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40d006b4e0 0x7f40d015f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:48.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.766+0000 7f40d78f7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d00cba60 0x7f40d0152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:48.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.766+0000 7f40d78f7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40d00cbe30 0x7f40d015ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:48.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.766+0000 7f40d78f7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d0159210 con 0x7f40d00cba60 2024-01-27T02:37:48.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.766+0000 7f40d78f7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d01594f0 con 0x7f40d006b4e0 2024-01-27T02:37:48.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.766+0000 7f40d78f7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d01597d0 con 0x7f40d00cbe30 2024-01-27T02:37:48.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.766+0000 7f40d4e92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d00cba60 0x7f40d0152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:48.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.766+0000 7f40d4e92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d00cba60 0x7f40d0152cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44630/0 (socket says 172.21.15.7:44630) 2024-01-27T02:37:48.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.766+0000 7f40d4e92700 1 -- 172.21.15.7:0/2950680941 learned_addr learned my addr 172.21.15.7:0/2950680941 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:48.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.766+0000 7f40d5693700 1 --2- 172.21.15.7:0/2950680941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40d006b4e0 0x7f40d015f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:48.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.766+0000 7f40d5e94700 1 --2- 172.21.15.7:0/2950680941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40d00cbe30 0x7f40d015ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:48.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.767+0000 7f40d4e92700 1 -- 172.21.15.7:0/2950680941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40d00cbe30 msgr2=0x7f40d015ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:48.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.767+0000 7f40d4e92700 1 --2- 172.21.15.7:0/2950680941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40d00cbe30 0x7f40d015ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.767+0000 7f40d4e92700 1 -- 172.21.15.7:0/2950680941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40d006b4e0 msgr2=0x7f40d015f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:48.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.767+0000 7f40d4e92700 1 --2- 172.21.15.7:0/2950680941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40d006b4e0 0x7f40d015f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.767+0000 7f40d4e92700 1 -- 172.21.15.7:0/2950680941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c8009ae0 con 0x7f40d00cba60 2024-01-27T02:37:48.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.767+0000 7f40d4e92700 1 --2- 172.21.15.7:0/2950680941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d00cba60 0x7f40d0152cd0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f40cc00ec50 tx=0x7f40cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:48.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.767+0000 7f40c17fa700 1 -- 172.21.15.7:0/2950680941 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc00c850 con 0x7f40d00cba60 2024-01-27T02:37:48.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.767+0000 7f40c17fa700 1 -- 172.21.15.7:0/2950680941 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40cc00c9b0 con 0x7f40d00cba60 2024-01-27T02:37:48.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.767+0000 7f40d78f7700 1 -- 172.21.15.7:0/2950680941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d0159ab0 con 0x7f40d00cba60 2024-01-27T02:37:48.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.767+0000 7f40c17fa700 1 -- 172.21.15.7:0/2950680941 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc00cd70 con 0x7f40d00cba60 2024-01-27T02:37:48.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.767+0000 7f40d78f7700 1 -- 172.21.15.7:0/2950680941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d015fc60 con 0x7f40d00cba60 2024-01-27T02:37:48.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.768+0000 7f40d78f7700 1 -- 172.21.15.7:0/2950680941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d0072060 con 0x7f40d00cba60 2024-01-27T02:37:48.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.769+0000 7f40c17fa700 1 -- 172.21.15.7:0/2950680941 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f40cc013680 con 0x7f40d00cba60 2024-01-27T02:37:48.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.769+0000 7f40c17fa700 1 --2- 172.21.15.7:0/2950680941 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f40b4066f30 0x7f40b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:48.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.769+0000 7f40c17fa700 1 -- 172.21.15.7:0/2950680941 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f40cc084470 con 0x7f40d00cba60 2024-01-27T02:37:48.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.769+0000 7f40d5693700 1 --2- 172.21.15.7:0/2950680941 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f40b4066f30 0x7f40b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:48.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.770+0000 7f40d5693700 1 --2- 172.21.15.7:0/2950680941 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f40b4066f30 0x7f40b40693e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009da0 tx=0x7f40c4009120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:48.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.773+0000 7f40c17fa700 1 -- 172.21.15.7:0/2950680941 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40cc051a30 con 0x7f40d00cba60 2024-01-27T02:37:48.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.979+0000 7f40d78f7700 1 -- 172.21.15.7:0/2950680941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40d00ce830 con 0x7f40d00cba60 2024-01-27T02:37:48.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.980+0000 7f40c17fa700 1 -- 172.21.15.7:0/2950680941 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f40cc017020 con 0x7f40d00cba60 2024-01-27T02:37:48.981 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:37:48.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.982+0000 7f40d78f7700 1 -- 172.21.15.7:0/2950680941 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f40b4066f30 msgr2=0x7f40b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:48.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.982+0000 7f40d78f7700 1 --2- 172.21.15.7:0/2950680941 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f40b4066f30 0x7f40b40693e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009da0 tx=0x7f40c4009120 comp rx=0 tx=0).stop 2024-01-27T02:37:48.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.982+0000 7f40d78f7700 1 -- 172.21.15.7:0/2950680941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d00cba60 msgr2=0x7f40d0152cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:48.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.982+0000 7f40d78f7700 1 --2- 172.21.15.7:0/2950680941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d00cba60 0x7f40d0152cd0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f40cc00ec50 tx=0x7f40cc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.982+0000 7f40d78f7700 1 -- 172.21.15.7:0/2950680941 shutdown_connections 2024-01-27T02:37:48.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.982+0000 7f40d78f7700 1 --2- 172.21.15.7:0/2950680941 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f40b4066f30 0x7f40b40693e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.982+0000 7f40d78f7700 1 --2- 172.21.15.7:0/2950680941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40d00cbe30 0x7f40d015ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.982+0000 7f40d78f7700 1 --2- 172.21.15.7:0/2950680941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40d006b4e0 0x7f40d015f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.982+0000 7f40d78f7700 1 --2- 172.21.15.7:0/2950680941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d00cba60 0x7f40d0152cd0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:48.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.982+0000 7f40d78f7700 1 -- 172.21.15.7:0/2950680941 >> 172.21.15.7:0/2950680941 conn(0x7f40d0063580 msgr2=0x7f40d015ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:48.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.982+0000 7f40d78f7700 1 -- 172.21.15.7:0/2950680941 shutdown_connections 2024-01-27T02:37:48.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:48.982+0000 7f40d78f7700 1 -- 172.21.15.7:0/2950680941 wait complete. 2024-01-27T02:37:48.997 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-27T02:37:49.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.127+0000 7fb47cab3700 1 Processor -- start 2024-01-27T02:37:49.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.127+0000 7fb47cab3700 1 -- start start 2024-01-27T02:37:49.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.128+0000 7fb47cab3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb4780c4cc0 0x7fb4780cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:49.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.129+0000 7fb47cab3700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb4780ca540 0x7fb478163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:49.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.129+0000 7fb47cab3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4780ce5e0 con 0x7fb47815a100 2024-01-27T02:37:49.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.129+0000 7fb47cab3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4780cd9d0 con 0x7fb4780c4cc0 2024-01-27T02:37:49.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.129+0000 7fb47cab3700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4780cdcb0 con 0x7fb4780ca540 2024-01-27T02:37:49.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.129+0000 7fb475d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb4780c4cc0 0x7fb4780cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:49.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.129+0000 7fb475d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb4780c4cc0 0x7fb4780cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44634/0 (socket says 172.21.15.7:44634) 2024-01-27T02:37:49.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.129+0000 7fb47659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb47815a100 0x7fb4780c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33522/0 (socket says 172.21.15.7:33522) 2024-01-27T02:37:49.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.129+0000 7fb475d9b700 1 -- 172.21.15.7:0/3706590648 learned_addr learned my addr 172.21.15.7:0/3706590648 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:49.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.129+0000 7fb476d9d700 1 --2- 172.21.15.7:0/3706590648 >> v2:172.21.15.188:3300/0 conn(0x7fb4780ca540 0x7fb478163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:49.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.129+0000 7fb475d9b700 1 -- 172.21.15.7:0/3706590648 >> v2:172.21.15.188:3300/0 conn(0x7fb4780ca540 msgr2=0x7fb478163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.129+0000 7fb475d9b700 1 --2- 172.21.15.7:0/3706590648 >> v2:172.21.15.188:3300/0 conn(0x7fb4780ca540 0x7fb478163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.129+0000 7fb475d9b700 1 -- 172.21.15.7:0/3706590648 >> v1:172.21.15.7:6789/0 conn(0x7fb47815a100 legacy=0x7fb4780c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.129+0000 7fb475d9b700 1 -- 172.21.15.7:0/3706590648 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4780cdf90 con 0x7fb4780c4cc0 2024-01-27T02:37:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.129+0000 7fb47559a700 1 -- 172.21.15.7:0/3706590648 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4173080557 0 0) 0x7fb4780ce5e0 con 0x7fb47815a100 2024-01-27T02:37:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.129+0000 7fb476d9d700 1 --2- 172.21.15.7:0/3706590648 >> v2:172.21.15.188:3300/0 conn(0x7fb4780ca540 0x7fb478163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.129+0000 7fb475d9b700 1 --2- 172.21.15.7:0/3706590648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4780c4cc0 0x7fb4780cd2f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb478069b70 tx=0x7fb46c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ed2b37a9afb1e9 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.129+0000 7fb47559a700 1 -- 172.21.15.7:0/3706590648 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c00b860 con 0x7fb4780c4cc0 2024-01-27T02:37:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.129+0000 7fb47559a700 1 -- 172.21.15.7:0/3706590648 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46c00b9c0 con 0x7fb4780c4cc0 2024-01-27T02:37:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.129+0000 7fb47559a700 1 -- 172.21.15.7:0/3706590648 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c00bd60 con 0x7fb4780c4cc0 2024-01-27T02:37:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.130+0000 7fb47cab3700 1 -- 172.21.15.7:0/3706590648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4780c4cc0 msgr2=0x7fb4780cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.130+0000 7fb47cab3700 1 --2- 172.21.15.7:0/3706590648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4780c4cc0 0x7fb4780cd2f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb478069b70 tx=0x7fb46c009250 comp rx=0 tx=0).stop 2024-01-27T02:37:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.130+0000 7fb47cab3700 1 -- 172.21.15.7:0/3706590648 shutdown_connections 2024-01-27T02:37:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.130+0000 7fb47cab3700 1 --2- 172.21.15.7:0/3706590648 >> v2:172.21.15.188:3300/0 conn(0x7fb4780ca540 0x7fb478163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.130+0000 7fb47cab3700 1 --2- 172.21.15.7:0/3706590648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4780c4cc0 0x7fb4780cd2f0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.130+0000 7fb47cab3700 1 -- 172.21.15.7:0/3706590648 >> 172.21.15.7:0/3706590648 conn(0x7fb478098020 msgr2=0x7fb4780711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:49.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.130+0000 7fb47cab3700 1 -- 172.21.15.7:0/3706590648 shutdown_connections 2024-01-27T02:37:49.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.130+0000 7fb47cab3700 1 -- 172.21.15.7:0/3706590648 wait complete. 2024-01-27T02:37:49.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.131+0000 7fb47cab3700 1 Processor -- start 2024-01-27T02:37:49.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.131+0000 7fb47cab3700 1 -- start start 2024-01-27T02:37:49.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.132+0000 7fb47cab3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4780c4cc0 0x7fb478153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:49.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.132+0000 7fb47cab3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4780ca540 0x7fb4780cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:49.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.132+0000 7fb47cab3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb47815a100 0x7fb478151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:49.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.132+0000 7fb47cab3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb478154420 con 0x7fb47815a100 2024-01-27T02:37:49.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.132+0000 7fb47cab3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780cbb90 con 0x7fb4780c4cc0 2024-01-27T02:37:49.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.132+0000 7fb47cab3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb47815adc0 con 0x7fb4780ca540 2024-01-27T02:37:49.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.132+0000 7fb476d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb47815a100 0x7fb478151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:49.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.132+0000 7fb476d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb47815a100 0x7fb478151bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44646/0 (socket says 172.21.15.7:44646) 2024-01-27T02:37:49.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.132+0000 7fb476d9d700 1 -- 172.21.15.7:0/3731149410 learned_addr learned my addr 172.21.15.7:0/3731149410 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:49.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.132+0000 7fb47659c700 1 --2- 172.21.15.7:0/3731149410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4780c4cc0 0x7fb478153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:49.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.132+0000 7fb475d9b700 1 --2- 172.21.15.7:0/3731149410 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4780ca540 0x7fb4780cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:49.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.133+0000 7fb476d9d700 1 -- 172.21.15.7:0/3731149410 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4780ca540 msgr2=0x7fb4780cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:49.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.133+0000 7fb476d9d700 1 --2- 172.21.15.7:0/3731149410 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4780ca540 0x7fb4780cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.133+0000 7fb476d9d700 1 -- 172.21.15.7:0/3731149410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4780c4cc0 msgr2=0x7fb478153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:49.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.133+0000 7fb476d9d700 1 --2- 172.21.15.7:0/3731149410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4780c4cc0 0x7fb478153ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.133+0000 7fb476d9d700 1 -- 172.21.15.7:0/3731149410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c0094f0 con 0x7fb47815a100 2024-01-27T02:37:49.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.133+0000 7fb476d9d700 1 --2- 172.21.15.7:0/3731149410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb47815a100 0x7fb478151bc0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb47000a380 tx=0x7fb470007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:49.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.133+0000 7fb4667fc700 1 -- 172.21.15.7:0/3731149410 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb470015440 con 0x7fb47815a100 2024-01-27T02:37:49.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.133+0000 7fb4667fc700 1 -- 172.21.15.7:0/3731149410 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4700155a0 con 0x7fb47815a100 2024-01-27T02:37:49.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.133+0000 7fb47cab3700 1 -- 172.21.15.7:0/3731149410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4780cbd90 con 0x7fb47815a100 2024-01-27T02:37:49.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.133+0000 7fb4667fc700 1 -- 172.21.15.7:0/3731149410 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb470015960 con 0x7fb47815a100 2024-01-27T02:37:49.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.133+0000 7fb47cab3700 1 -- 172.21.15.7:0/3731149410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4780cc350 con 0x7fb47815a100 2024-01-27T02:37:49.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.134+0000 7fb47cab3700 1 -- 172.21.15.7:0/3731149410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb47804c710 con 0x7fb47815a100 2024-01-27T02:37:49.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.135+0000 7fb4667fc700 1 -- 172.21.15.7:0/3731149410 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb47000d440 con 0x7fb47815a100 2024-01-27T02:37:49.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.135+0000 7fb4667fc700 1 --2- 172.21.15.7:0/3731149410 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb458066f30 0x7fb4580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:49.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.135+0000 7fb4667fc700 1 -- 172.21.15.7:0/3731149410 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb470084d80 con 0x7fb47815a100 2024-01-27T02:37:49.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.135+0000 7fb47659c700 1 --2- 172.21.15.7:0/3731149410 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb458066f30 0x7fb4580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:49.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.136+0000 7fb47659c700 1 --2- 172.21.15.7:0/3731149410 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb458066f30 0x7fb4580693e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb468007180 tx=0x7fb468007ef0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:49.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.140+0000 7fb4667fc700 1 -- 172.21.15.7:0/3731149410 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4700522a0 con 0x7fb47815a100 2024-01-27T02:37:49.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.346+0000 7fb47cab3700 1 -- 172.21.15.7:0/3731149410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb4780ce270 con 0x7fb47815a100 2024-01-27T02:37:49.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.346+0000 7fb4667fc700 1 -- 172.21.15.7:0/3731149410 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v113) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb47001a090 con 0x7fb47815a100 2024-01-27T02:37:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:49.348 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-27T02:37:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:37:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-27T02:37:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-27T02:37:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:37:49.348 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-27T02:37:49.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.348+0000 7fb47cab3700 1 -- 172.21.15.7:0/3731149410 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb458066f30 msgr2=0x7fb4580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:49.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.348+0000 7fb47cab3700 1 --2- 172.21.15.7:0/3731149410 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb458066f30 0x7fb4580693e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb468007180 tx=0x7fb468007ef0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.348+0000 7fb47cab3700 1 -- 172.21.15.7:0/3731149410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb47815a100 msgr2=0x7fb478151bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:49.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.348+0000 7fb47cab3700 1 --2- 172.21.15.7:0/3731149410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb47815a100 0x7fb478151bc0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb47000a380 tx=0x7fb470007900 comp rx=0 tx=0).stop 2024-01-27T02:37:49.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.348+0000 7fb47cab3700 1 -- 172.21.15.7:0/3731149410 shutdown_connections 2024-01-27T02:37:49.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.348+0000 7fb47cab3700 1 --2- 172.21.15.7:0/3731149410 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb458066f30 0x7fb4580693e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.348+0000 7fb47cab3700 1 --2- 172.21.15.7:0/3731149410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb47815a100 0x7fb478151bc0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.348+0000 7fb47cab3700 1 --2- 172.21.15.7:0/3731149410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4780c4cc0 0x7fb478153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.348+0000 7fb47cab3700 1 --2- 172.21.15.7:0/3731149410 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4780ca540 0x7fb4780cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.348+0000 7fb47cab3700 1 -- 172.21.15.7:0/3731149410 >> 172.21.15.7:0/3731149410 conn(0x7fb478098020 msgr2=0x7fb4780cebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:49.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.348+0000 7fb47cab3700 1 -- 172.21.15.7:0/3731149410 shutdown_connections 2024-01-27T02:37:49.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.349+0000 7fb47cab3700 1 -- 172.21.15.7:0/3731149410 wait complete. 2024-01-27T02:37:49.363 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-27T02:37:49.364 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-27T02:37:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.497+0000 7f1fdbb8e700 1 Processor -- start 2024-01-27T02:37:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.497+0000 7f1fdbb8e700 1 -- start start 2024-01-27T02:37:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.498+0000 7f1fdbb8e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fd40c4cc0 0x7f1fd40ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.498+0000 7f1fdbb8e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1fd40cd2f0 0x7f1fd40679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.498+0000 7f1fdbb8e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd4067ef0 con 0x7f1fd415a230 2024-01-27T02:37:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.498+0000 7f1fdbb8e700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd40cd790 con 0x7f1fd40cd2f0 2024-01-27T02:37:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.498+0000 7f1fdbb8e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd40c5320 con 0x7f1fd40c4cc0 2024-01-27T02:37:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.498+0000 7f1fd992a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1fd415a230 0x7f1fd40c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33530/0 (socket says 172.21.15.7:33530) 2024-01-27T02:37:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.498+0000 7f1fd992a700 1 -- 172.21.15.7:0/3150559731 learned_addr learned my addr 172.21.15.7:0/3150559731 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.499+0000 7f1fda12b700 1 --2- 172.21.15.7:0/3150559731 >> v2:172.21.15.188:3300/0 conn(0x7f1fd40cd2f0 0x7f1fd40679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.499+0000 7f1fd9129700 1 --2- 172.21.15.7:0/3150559731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fd40c4cc0 0x7f1fd40ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.499+0000 7f1fd8928700 1 -- 172.21.15.7:0/3150559731 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3561165717 0 0) 0x7f1fd4067ef0 con 0x7f1fd415a230 2024-01-27T02:37:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.499+0000 7f1fd8928700 1 -- 172.21.15.7:0/3150559731 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fc0003570 con 0x7f1fd415a230 2024-01-27T02:37:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.499+0000 7f1fd8928700 1 -- 172.21.15.7:0/3150559731 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2895748675 0 0) 0x7f1fc0003570 con 0x7f1fd415a230 2024-01-27T02:37:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.499+0000 7f1fda12b700 1 -- 172.21.15.7:0/3150559731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fd40c4cc0 msgr2=0x7f1fd40ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.499+0000 7f1fda12b700 1 --2- 172.21.15.7:0/3150559731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fd40c4cc0 0x7f1fd40ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.499+0000 7f1fda12b700 1 -- 172.21.15.7:0/3150559731 >> v1:172.21.15.7:6789/0 conn(0x7f1fd415a230 legacy=0x7f1fd40c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.499+0000 7f1fda12b700 1 -- 172.21.15.7:0/3150559731 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd40c9520 con 0x7f1fd40cd2f0 2024-01-27T02:37:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.499+0000 7f1fda12b700 1 --2- 172.21.15.7:0/3150559731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fd40cd2f0 0x7f1fd40679b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc009e60 tx=0x7f1fcc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e53178bc91191938 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.500+0000 7f1fd8928700 1 -- 172.21.15.7:0/3150559731 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc00e070 con 0x7f1fd40cd2f0 2024-01-27T02:37:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.500+0000 7f1fd8928700 1 -- 172.21.15.7:0/3150559731 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fcc00b7a0 con 0x7f1fd40cd2f0 2024-01-27T02:37:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.500+0000 7f1fd8928700 1 -- 172.21.15.7:0/3150559731 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd00015f0 con 0x7f1fd40cd2f0 2024-01-27T02:37:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.500+0000 7f1fdbb8e700 1 -- 172.21.15.7:0/3150559731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fd40cd2f0 msgr2=0x7f1fd40679b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.500+0000 7f1fdbb8e700 1 --2- 172.21.15.7:0/3150559731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fd40cd2f0 0x7f1fd40679b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc009e60 tx=0x7f1fcc009780 comp rx=0 tx=0).stop 2024-01-27T02:37:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.500+0000 7f1fdbb8e700 1 -- 172.21.15.7:0/3150559731 shutdown_connections 2024-01-27T02:37:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.500+0000 7f1fdbb8e700 1 --2- 172.21.15.7:0/3150559731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fd40cd2f0 0x7f1fd40679b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.500+0000 7f1fdbb8e700 1 --2- 172.21.15.7:0/3150559731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fd40c4cc0 0x7f1fd40ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.500+0000 7f1fdbb8e700 1 -- 172.21.15.7:0/3150559731 >> 172.21.15.7:0/3150559731 conn(0x7f1fd4098020 msgr2=0x7f1fd40711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.500+0000 7f1fdbb8e700 1 -- 172.21.15.7:0/3150559731 shutdown_connections 2024-01-27T02:37:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.500+0000 7f1fdbb8e700 1 -- 172.21.15.7:0/3150559731 wait complete. 2024-01-27T02:37:49.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.501+0000 7f1fdbb8e700 1 Processor -- start 2024-01-27T02:37:49.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.502+0000 7f1fdbb8e700 1 -- start start 2024-01-27T02:37:49.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.502+0000 7f1fdbb8e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fd40c4cc0 0x7f1fd40c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:49.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.502+0000 7f1fdbb8e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fd40cd2f0 0x7f1fd414de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:49.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.502+0000 7f1fdbb8e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fd415a230 0x7f1fd40b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:49.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.502+0000 7f1fdbb8e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd40b2ed0 con 0x7f1fd40c4cc0 2024-01-27T02:37:49.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.502+0000 7f1fdbb8e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd40b2040 con 0x7f1fd40cd2f0 2024-01-27T02:37:49.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.502+0000 7f1fdbb8e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd40b22c0 con 0x7f1fd415a230 2024-01-27T02:37:49.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.502+0000 7f1fd992a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fd40c4cc0 0x7f1fd40c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:49.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.502+0000 7f1fd992a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fd40c4cc0 0x7f1fd40c6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44654/0 (socket says 172.21.15.7:44654) 2024-01-27T02:37:49.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.502+0000 7f1fd992a700 1 -- 172.21.15.7:0/2942345197 learned_addr learned my addr 172.21.15.7:0/2942345197 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:49.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.502+0000 7f1fd9129700 1 --2- 172.21.15.7:0/2942345197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fd40cd2f0 0x7f1fd414de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:49.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.502+0000 7f1fda12b700 1 --2- 172.21.15.7:0/2942345197 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fd415a230 0x7f1fd40b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.503+0000 7f1fd992a700 1 -- 172.21.15.7:0/2942345197 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fd415a230 msgr2=0x7f1fd40b1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.503+0000 7f1fd992a700 1 --2- 172.21.15.7:0/2942345197 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fd415a230 0x7f1fd40b1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.503+0000 7f1fd992a700 1 -- 172.21.15.7:0/2942345197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fd40cd2f0 msgr2=0x7f1fd414de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.503+0000 7f1fd992a700 1 --2- 172.21.15.7:0/2942345197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fd40cd2f0 0x7f1fd414de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.503+0000 7f1fd992a700 1 -- 172.21.15.7:0/2942345197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc009a20 con 0x7f1fd40c4cc0 2024-01-27T02:37:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.503+0000 7f1fd992a700 1 --2- 172.21.15.7:0/2942345197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fd40c4cc0 0x7f1fd40c6ce0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00017e0 tx=0x7f1fd0002a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.503+0000 7f1fc5ffb700 1 -- 172.21.15.7:0/2942345197 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd00072e0 con 0x7f1fd40c4cc0 2024-01-27T02:37:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.503+0000 7f1fc5ffb700 1 -- 172.21.15.7:0/2942345197 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd0007440 con 0x7f1fd40c4cc0 2024-01-27T02:37:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.503+0000 7f1fdbb8e700 1 -- 172.21.15.7:0/2942345197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd40b25a0 con 0x7f1fd40c4cc0 2024-01-27T02:37:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.503+0000 7f1fc5ffb700 1 -- 172.21.15.7:0/2942345197 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd00057f0 con 0x7f1fd40c4cc0 2024-01-27T02:37:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.503+0000 7f1fdbb8e700 1 -- 172.21.15.7:0/2942345197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd40b2b60 con 0x7f1fd40c4cc0 2024-01-27T02:37:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.504+0000 7f1fdbb8e700 1 -- 172.21.15.7:0/2942345197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd404c710 con 0x7f1fd40c4cc0 2024-01-27T02:37:49.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.505+0000 7f1fc5ffb700 1 -- 172.21.15.7:0/2942345197 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1fd0005b80 con 0x7f1fd40c4cc0 2024-01-27T02:37:49.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.505+0000 7f1fc5ffb700 1 --2- 172.21.15.7:0/2942345197 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1fb8066f30 0x7f1fb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:49.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.505+0000 7f1fc5ffb700 1 -- 172.21.15.7:0/2942345197 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1fd0083870 con 0x7f1fd40c4cc0 2024-01-27T02:37:49.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.505+0000 7f1fd9129700 1 --2- 172.21.15.7:0/2942345197 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1fb8066f30 0x7f1fb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:49.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.506+0000 7f1fd9129700 1 --2- 172.21.15.7:0/2942345197 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1fb8066f30 0x7f1fb80693e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4159490 tx=0x7f1fc80091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:49.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.510+0000 7f1fc5ffb700 1 -- 172.21.15.7:0/2942345197 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1fd0050e30 con 0x7f1fd40c4cc0 2024-01-27T02:37:49.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.714+0000 7f1fdbb8e700 1 -- 172.21.15.7:0/2942345197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1fd40c9800 con 0x7f1fd40c4cc0 2024-01-27T02:37:49.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.790+0000 7f1fc5ffb700 1 -- 172.21.15.7:0/2942345197 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v114) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1fd0017070 con 0x7f1fd40c4cc0 2024-01-27T02:37:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.795+0000 7f1fdbb8e700 1 -- 172.21.15.7:0/2942345197 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1fb8066f30 msgr2=0x7f1fb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.795+0000 7f1fdbb8e700 1 --2- 172.21.15.7:0/2942345197 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1fb8066f30 0x7f1fb80693e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4159490 tx=0x7f1fc80091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.795+0000 7f1fdbb8e700 1 -- 172.21.15.7:0/2942345197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fd40c4cc0 msgr2=0x7f1fd40c6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.795+0000 7f1fdbb8e700 1 --2- 172.21.15.7:0/2942345197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fd40c4cc0 0x7f1fd40c6ce0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00017e0 tx=0x7f1fd0002a30 comp rx=0 tx=0).stop 2024-01-27T02:37:49.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.795+0000 7f1fdbb8e700 1 -- 172.21.15.7:0/2942345197 shutdown_connections 2024-01-27T02:37:49.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.795+0000 7f1fdbb8e700 1 --2- 172.21.15.7:0/2942345197 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1fb8066f30 0x7f1fb80693e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.795+0000 7f1fdbb8e700 1 --2- 172.21.15.7:0/2942345197 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fd415a230 0x7f1fd40b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.795+0000 7f1fdbb8e700 1 --2- 172.21.15.7:0/2942345197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fd40c4cc0 0x7f1fd40c6ce0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.795+0000 7f1fdbb8e700 1 --2- 172.21.15.7:0/2942345197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fd40cd2f0 0x7f1fd414de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.795+0000 7f1fdbb8e700 1 -- 172.21.15.7:0/2942345197 >> 172.21.15.7:0/2942345197 conn(0x7f1fd4098020 msgr2=0x7f1fd40680a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:49.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.795+0000 7f1fdbb8e700 1 -- 172.21.15.7:0/2942345197 shutdown_connections 2024-01-27T02:37:49.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.795+0000 7f1fdbb8e700 1 -- 172.21.15.7:0/2942345197 wait complete. 2024-01-27T02:37:49.799 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-27T02:37:49.811 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-27T02:37:49.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.949+0000 7f5e02f3a700 1 Processor -- start 2024-01-27T02:37:49.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.949+0000 7f5e02f3a700 1 -- start start 2024-01-27T02:37:49.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.950+0000 7f5e02f3a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5dfc1416c0 0x7f5dfc0cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:49.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.950+0000 7f5e02f3a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5dfc141a90 0x7f5dfc0cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:49.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.950+0000 7f5e02f3a700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dfc15c7d0 con 0x7f5dfc0cab00 2024-01-27T02:37:49.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.950+0000 7f5e02f3a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dfc0cd9c0 con 0x7f5dfc141a90 2024-01-27T02:37:49.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.950+0000 7f5e02f3a700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dfc0c9640 con 0x7f5dfc1416c0 2024-01-27T02:37:49.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.951+0000 7f5dfbfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5dfc141a90 0x7f5dfc0cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.951+0000 7f5dfbfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5dfc141a90 0x7f5dfc0cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44664/0 (socket says 172.21.15.7:44664) 2024-01-27T02:37:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.951+0000 7f5dfbfff700 1 -- 172.21.15.7:0/3512719179 learned_addr learned my addr 172.21.15.7:0/3512719179 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.951+0000 7f5e00cd6700 1 --2- 172.21.15.7:0/3512719179 >> v2:172.21.15.188:3300/0 conn(0x7f5dfc1416c0 0x7f5dfc0cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.951+0000 7f5dfbfff700 1 -- 172.21.15.7:0/3512719179 >> v2:172.21.15.188:3300/0 conn(0x7f5dfc1416c0 msgr2=0x7f5dfc0cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.951+0000 7f5dfbfff700 1 --2- 172.21.15.7:0/3512719179 >> v2:172.21.15.188:3300/0 conn(0x7f5dfc1416c0 0x7f5dfc0cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.951+0000 7f5dfbfff700 1 -- 172.21.15.7:0/3512719179 >> v1:172.21.15.188:6789/0 conn(0x7f5dfc0cab00 legacy=0x7f5dfc0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.951+0000 7f5dfbfff700 1 -- 172.21.15.7:0/3512719179 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dfc15bbc0 con 0x7f5dfc141a90 2024-01-27T02:37:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.951+0000 7f5dfbfff700 1 --2- 172.21.15.7:0/3512719179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dfc141a90 0x7f5dfc0cbc20 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0cb6b0 tx=0x7f5de8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7c56811a376303 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.951+0000 7f5dfb7fe700 1 -- 172.21.15.7:0/3512719179 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de800b860 con 0x7f5dfc141a90 2024-01-27T02:37:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.951+0000 7f5dfb7fe700 1 -- 172.21.15.7:0/3512719179 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de800b9c0 con 0x7f5dfc141a90 2024-01-27T02:37:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.951+0000 7f5dfb7fe700 1 -- 172.21.15.7:0/3512719179 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de800bd60 con 0x7f5dfc141a90 2024-01-27T02:37:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.952+0000 7f5e02f3a700 1 -- 172.21.15.7:0/3512719179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dfc141a90 msgr2=0x7f5dfc0cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.952+0000 7f5e02f3a700 1 --2- 172.21.15.7:0/3512719179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dfc141a90 0x7f5dfc0cbc20 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0cb6b0 tx=0x7f5de8009250 comp rx=0 tx=0).stop 2024-01-27T02:37:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.952+0000 7f5e02f3a700 1 -- 172.21.15.7:0/3512719179 shutdown_connections 2024-01-27T02:37:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.952+0000 7f5e02f3a700 1 --2- 172.21.15.7:0/3512719179 >> v2:172.21.15.188:3300/0 conn(0x7f5dfc1416c0 0x7f5dfc0cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.952+0000 7f5e02f3a700 1 --2- 172.21.15.7:0/3512719179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dfc141a90 0x7f5dfc0cbc20 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.952+0000 7f5e02f3a700 1 -- 172.21.15.7:0/3512719179 >> 172.21.15.7:0/3512719179 conn(0x7f5dfc063580 msgr2=0x7f5dfc063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.952+0000 7f5e02f3a700 1 -- 172.21.15.7:0/3512719179 shutdown_connections 2024-01-27T02:37:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.952+0000 7f5e02f3a700 1 -- 172.21.15.7:0/3512719179 wait complete. 2024-01-27T02:37:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.953+0000 7f5e02f3a700 1 Processor -- start 2024-01-27T02:37:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.953+0000 7f5e02f3a700 1 -- start start 2024-01-27T02:37:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.954+0000 7f5e02f3a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dfc0cab00 0x7f5dfc0679b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.954+0000 7f5e02f3a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dfc1416c0 0x7f5dfc15a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.954+0000 7f5e02f3a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dfc141a90 0x7f5dfc0b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.954+0000 7f5e02f3a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc13d500 con 0x7f5dfc141a90 2024-01-27T02:37:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.954+0000 7f5e02f3a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc15ac50 con 0x7f5dfc0cab00 2024-01-27T02:37:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.954+0000 7f5e02f3a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc069a30 con 0x7f5dfc1416c0 2024-01-27T02:37:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.954+0000 7f5e014d7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dfc141a90 0x7f5dfc0b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:49.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.954+0000 7f5e014d7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dfc141a90 0x7f5dfc0b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44680/0 (socket says 172.21.15.7:44680) 2024-01-27T02:37:49.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.954+0000 7f5e014d7700 1 -- 172.21.15.7:0/2113392393 learned_addr learned my addr 172.21.15.7:0/2113392393 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:49.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.954+0000 7f5e00cd6700 1 --2- 172.21.15.7:0/2113392393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dfc0cab00 0x7f5dfc0679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:49.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.955+0000 7f5dfbfff700 1 --2- 172.21.15.7:0/2113392393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dfc1416c0 0x7f5dfc15a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:49.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.955+0000 7f5e014d7700 1 -- 172.21.15.7:0/2113392393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dfc1416c0 msgr2=0x7f5dfc15a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:49.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.955+0000 7f5e014d7700 1 --2- 172.21.15.7:0/2113392393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dfc1416c0 0x7f5dfc15a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.955+0000 7f5e014d7700 1 -- 172.21.15.7:0/2113392393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dfc0cab00 msgr2=0x7f5dfc0679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:49.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.955+0000 7f5e014d7700 1 --2- 172.21.15.7:0/2113392393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dfc0cab00 0x7f5dfc0679b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:49.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.955+0000 7f5e014d7700 1 -- 172.21.15.7:0/2113392393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de80094f0 con 0x7f5dfc141a90 2024-01-27T02:37:49.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.955+0000 7f5e014d7700 1 --2- 172.21.15.7:0/2113392393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dfc141a90 0x7f5dfc0b0d00 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5df400c910 tx=0x7f5df40016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:49.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.955+0000 7f5df8ff9700 1 -- 172.21.15.7:0/2113392393 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df400a7e0 con 0x7f5dfc141a90 2024-01-27T02:37:49.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.955+0000 7f5df8ff9700 1 -- 172.21.15.7:0/2113392393 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df400a940 con 0x7f5dfc141a90 2024-01-27T02:37:49.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.955+0000 7f5df8ff9700 1 -- 172.21.15.7:0/2113392393 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df400ad00 con 0x7f5dfc141a90 2024-01-27T02:37:49.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.955+0000 7f5e02f3a700 1 -- 172.21.15.7:0/2113392393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dfc13c880 con 0x7f5dfc141a90 2024-01-27T02:37:49.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.955+0000 7f5e02f3a700 1 -- 172.21.15.7:0/2113392393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dfc13ce40 con 0x7f5dfc141a90 2024-01-27T02:37:49.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.956+0000 7f5e02f3a700 1 -- 172.21.15.7:0/2113392393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dfc13d640 con 0x7f5dfc141a90 2024-01-27T02:37:49.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.957+0000 7f5df8ff9700 1 -- 172.21.15.7:0/2113392393 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5df4012630 con 0x7f5dfc141a90 2024-01-27T02:37:49.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.957+0000 7f5df8ff9700 1 --2- 172.21.15.7:0/2113392393 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5de0066f30 0x7f5de00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:49.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.957+0000 7f5df8ff9700 1 -- 172.21.15.7:0/2113392393 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f5df40845b0 con 0x7f5dfc141a90 2024-01-27T02:37:49.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.958+0000 7f5e00cd6700 1 --2- 172.21.15.7:0/2113392393 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5de0066f30 0x7f5de00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:49.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.958+0000 7f5e00cd6700 1 --2- 172.21.15.7:0/2113392393 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5de0066f30 0x7f5de00693e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5df0006aa0 tx=0x7f5df0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:49.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:49.962+0000 7f5df8ff9700 1 -- 172.21.15.7:0/2113392393 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5dfc13d640 con 0x7f5dfc141a90 2024-01-27T02:37:50.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.166+0000 7f5e02f3a700 1 -- 172.21.15.7:0/2113392393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5dfc13d640 con 0x7f5dfc141a90 2024-01-27T02:37:50.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.795+0000 7f5df8ff9700 1 -- 172.21.15.7:0/2113392393 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v115) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5dfc13d640 con 0x7f5dfc141a90 2024-01-27T02:37:50.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.800+0000 7f5e02f3a700 1 -- 172.21.15.7:0/2113392393 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5de0066f30 msgr2=0x7f5de00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:50.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.800+0000 7f5e02f3a700 1 --2- 172.21.15.7:0/2113392393 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5de0066f30 0x7f5de00693e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5df0006aa0 tx=0x7f5df0008040 comp rx=0 tx=0).stop 2024-01-27T02:37:50.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.800+0000 7f5e02f3a700 1 -- 172.21.15.7:0/2113392393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dfc141a90 msgr2=0x7f5dfc0b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:50.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.800+0000 7f5e02f3a700 1 --2- 172.21.15.7:0/2113392393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dfc141a90 0x7f5dfc0b0d00 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5df400c910 tx=0x7f5df40016f0 comp rx=0 tx=0).stop 2024-01-27T02:37:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.801+0000 7f5e02f3a700 1 -- 172.21.15.7:0/2113392393 shutdown_connections 2024-01-27T02:37:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.801+0000 7f5e02f3a700 1 --2- 172.21.15.7:0/2113392393 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5de0066f30 0x7f5de00693e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.801+0000 7f5e02f3a700 1 --2- 172.21.15.7:0/2113392393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dfc141a90 0x7f5dfc0b0d00 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.801+0000 7f5e02f3a700 1 --2- 172.21.15.7:0/2113392393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dfc0cab00 0x7f5dfc0679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.801+0000 7f5e02f3a700 1 --2- 172.21.15.7:0/2113392393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dfc1416c0 0x7f5dfc15a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.801+0000 7f5e02f3a700 1 -- 172.21.15.7:0/2113392393 >> 172.21.15.7:0/2113392393 conn(0x7f5dfc063580 msgr2=0x7f5dfc0c4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:50.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.806+0000 7f5e02f3a700 1 -- 172.21.15.7:0/2113392393 shutdown_connections 2024-01-27T02:37:50.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.806+0000 7f5e02f3a700 1 -- 172.21.15.7:0/2113392393 wait complete. 2024-01-27T02:37:50.815 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-27T02:37:50.827 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-27T02:37:50.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.963+0000 7f0421c50700 1 Processor -- start 2024-01-27T02:37:50.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.963+0000 7f0421c50700 1 -- start start 2024-01-27T02:37:50.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.964+0000 7f0421c50700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f041c0ce510 0x7f041c158050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:50.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.964+0000 7f0421c50700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f041c063580 0x7f041c063a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:50.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.964+0000 7f0421c50700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f041c0b3510 con 0x7f041c157c80 2024-01-27T02:37:50.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.964+0000 7f0421c50700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041c063f40 con 0x7f041c063580 2024-01-27T02:37:50.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.964+0000 7f0421c50700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041c06f8b0 con 0x7f041c0ce510 2024-01-27T02:37:50.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.964+0000 7f041bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f041c063580 0x7f041c063a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:50.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.964+0000 7f041bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f041c063580 0x7f041c063a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44684/0 (socket says 172.21.15.7:44684) 2024-01-27T02:37:50.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.964+0000 7f041bfff700 1 -- 172.21.15.7:0/2085126943 learned_addr learned my addr 172.21.15.7:0/2085126943 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:50.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.965+0000 7f041affd700 1 --2- 172.21.15.7:0/2085126943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f041c0ce510 0x7f041c158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:50.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.965+0000 7f041bfff700 1 -- 172.21.15.7:0/2085126943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f041c0ce510 msgr2=0x7f041c158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:50.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.965+0000 7f041bfff700 1 --2- 172.21.15.7:0/2085126943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f041c0ce510 0x7f041c158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:50.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.965+0000 7f041bfff700 1 -- 172.21.15.7:0/2085126943 >> v1:172.21.15.188:6789/0 conn(0x7f041c157c80 legacy=0x7f041c0b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:50.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.965+0000 7f041bfff700 1 -- 172.21.15.7:0/2085126943 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c0b2900 con 0x7f041c063580 2024-01-27T02:37:50.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.965+0000 7f041bfff700 1 --2- 172.21.15.7:0/2085126943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c063580 0x7f041c063a00 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f0414009f20 tx=0x7f0414009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d25b6ba396aae26 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:50.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.965+0000 7f041a7fc700 1 -- 172.21.15.7:0/2085126943 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041400e070 con 0x7f041c063580 2024-01-27T02:37:50.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.965+0000 7f041a7fc700 1 -- 172.21.15.7:0/2085126943 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041400b9d0 con 0x7f041c063580 2024-01-27T02:37:50.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.965+0000 7f041a7fc700 1 -- 172.21.15.7:0/2085126943 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04140092e0 con 0x7f041c063580 2024-01-27T02:37:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.966+0000 7f0421c50700 1 -- 172.21.15.7:0/2085126943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c063580 msgr2=0x7f041c063a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.966+0000 7f0421c50700 1 --2- 172.21.15.7:0/2085126943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c063580 0x7f041c063a00 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f0414009f20 tx=0x7f0414009840 comp rx=0 tx=0).stop 2024-01-27T02:37:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.966+0000 7f0421c50700 1 -- 172.21.15.7:0/2085126943 shutdown_connections 2024-01-27T02:37:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.966+0000 7f0421c50700 1 --2- 172.21.15.7:0/2085126943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c063580 0x7f041c063a00 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.966+0000 7f0421c50700 1 --2- 172.21.15.7:0/2085126943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f041c0ce510 0x7f041c158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.966+0000 7f0421c50700 1 -- 172.21.15.7:0/2085126943 >> 172.21.15.7:0/2085126943 conn(0x7f041c068480 msgr2=0x7f041c0cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.966+0000 7f0421c50700 1 -- 172.21.15.7:0/2085126943 shutdown_connections 2024-01-27T02:37:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.966+0000 7f0421c50700 1 -- 172.21.15.7:0/2085126943 wait complete. 2024-01-27T02:37:50.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.968+0000 7f0421c50700 1 Processor -- start 2024-01-27T02:37:50.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.968+0000 7f0421c50700 1 -- start start 2024-01-27T02:37:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.969+0000 7f0421c50700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f041c063580 0x7f041c155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.969+0000 7f0421c50700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c0ce510 0x7f041c158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.969+0000 7f0421c50700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f041c157c80 0x7f041c156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.969+0000 7f0421c50700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c158aa0 con 0x7f041c0ce510 2024-01-27T02:37:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.969+0000 7f0421c50700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c158d80 con 0x7f041c063580 2024-01-27T02:37:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.969+0000 7f0421c50700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041c159060 con 0x7f041c157c80 2024-01-27T02:37:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.969+0000 7f041affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c0ce510 0x7f041c158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.969+0000 7f041affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c0ce510 0x7f041c158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44698/0 (socket says 172.21.15.7:44698) 2024-01-27T02:37:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.969+0000 7f041affd700 1 -- 172.21.15.7:0/534680158 learned_addr learned my addr 172.21.15.7:0/534680158 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.969+0000 7f041b7fe700 1 --2- 172.21.15.7:0/534680158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f041c063580 0x7f041c155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.969+0000 7f041bfff700 1 --2- 172.21.15.7:0/534680158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f041c157c80 0x7f041c156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.969+0000 7f041affd700 1 -- 172.21.15.7:0/534680158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f041c157c80 msgr2=0x7f041c156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.969+0000 7f041affd700 1 --2- 172.21.15.7:0/534680158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f041c157c80 0x7f041c156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.969+0000 7f041affd700 1 -- 172.21.15.7:0/534680158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f041c063580 msgr2=0x7f041c155980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.969+0000 7f041affd700 1 --2- 172.21.15.7:0/534680158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f041c063580 0x7f041c155980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.969+0000 7f041affd700 1 -- 172.21.15.7:0/534680158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0414009ae0 con 0x7f041c0ce510 2024-01-27T02:37:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.969+0000 7f041bfff700 1 --2- 172.21.15.7:0/534680158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f041c157c80 0x7f041c156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.969+0000 7f041affd700 1 --2- 172.21.15.7:0/534680158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c0ce510 0x7f041c158050 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f041000ec80 tx=0x7f04100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.970+0000 7f04037fe700 1 -- 172.21.15.7:0/534680158 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041000c9d0 con 0x7f041c0ce510 2024-01-27T02:37:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.970+0000 7f04037fe700 1 -- 172.21.15.7:0/534680158 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041000cb30 con 0x7f041c0ce510 2024-01-27T02:37:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.970+0000 7f0421c50700 1 -- 172.21.15.7:0/534680158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041c159340 con 0x7f041c0ce510 2024-01-27T02:37:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.970+0000 7f04037fe700 1 -- 172.21.15.7:0/534680158 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410013470 con 0x7f041c0ce510 2024-01-27T02:37:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.970+0000 7f0421c50700 1 -- 172.21.15.7:0/534680158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041c16a6d0 con 0x7f041c0ce510 2024-01-27T02:37:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.971+0000 7f0421c50700 1 -- 172.21.15.7:0/534680158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041c072060 con 0x7f041c0ce510 2024-01-27T02:37:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.971+0000 7f04037fe700 1 -- 172.21.15.7:0/534680158 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0410013820 con 0x7f041c0ce510 2024-01-27T02:37:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.972+0000 7f04037fe700 1 --2- 172.21.15.7:0/534680158 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0408066f30 0x7f04080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.972+0000 7f04037fe700 1 -- 172.21.15.7:0/534680158 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f0410084860 con 0x7f041c0ce510 2024-01-27T02:37:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.972+0000 7f041b7fe700 1 --2- 172.21.15.7:0/534680158 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0408066f30 0x7f04080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.972+0000 7f041b7fe700 1 --2- 172.21.15.7:0/534680158 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0408066f30 0x7f04080693e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f040c0037c0 tx=0x7f040c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:50.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:50.976+0000 7f04037fe700 1 -- 172.21.15.7:0/534680158 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0410051b30 con 0x7f041c0ce510 2024-01-27T02:37:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.229+0000 7f0421c50700 1 -- 172.21.15.7:0/534680158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f041c0b2be0 con 0x7f041c0ce510 2024-01-27T02:37:51.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.818+0000 7f04037fe700 1 -- 172.21.15.7:0/534680158 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v54) v1 ==== 174+0+0 (secure 0 0 0) 0x7f0410055150 con 0x7f041c0ce510 2024-01-27T02:37:51.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.825+0000 7f0421c50700 1 -- 172.21.15.7:0/534680158 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0408066f30 msgr2=0x7f04080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:51.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.825+0000 7f0421c50700 1 --2- 172.21.15.7:0/534680158 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0408066f30 0x7f04080693e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f040c0037c0 tx=0x7f040c009040 comp rx=0 tx=0).stop 2024-01-27T02:37:51.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.825+0000 7f0421c50700 1 -- 172.21.15.7:0/534680158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c0ce510 msgr2=0x7f041c158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:51.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.825+0000 7f0421c50700 1 --2- 172.21.15.7:0/534680158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c0ce510 0x7f041c158050 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f041000ec80 tx=0x7f04100091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:51.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.833+0000 7f0421c50700 1 -- 172.21.15.7:0/534680158 shutdown_connections 2024-01-27T02:37:51.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.833+0000 7f0421c50700 1 --2- 172.21.15.7:0/534680158 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0408066f30 0x7f04080693e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:51.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.833+0000 7f0421c50700 1 --2- 172.21.15.7:0/534680158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f041c157c80 0x7f041c156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:51.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.833+0000 7f0421c50700 1 --2- 172.21.15.7:0/534680158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f041c063580 0x7f041c155980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:51.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.833+0000 7f0421c50700 1 --2- 172.21.15.7:0/534680158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f041c0ce510 0x7f041c158050 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:51.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.833+0000 7f0421c50700 1 -- 172.21.15.7:0/534680158 >> 172.21.15.7:0/534680158 conn(0x7f041c068480 msgr2=0x7f041c0afef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:51.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.833+0000 7f0421c50700 1 -- 172.21.15.7:0/534680158 shutdown_connections 2024-01-27T02:37:51.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.833+0000 7f0421c50700 1 -- 172.21.15.7:0/534680158 wait complete. 2024-01-27T02:37:51.838 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 14 and data pool 15 2024-01-27T02:37:51.853 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:37:51.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.987+0000 7ffb5b9bb700 1 Processor -- start 2024-01-27T02:37:51.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.988+0000 7ffb5b9bb700 1 -- start start 2024-01-27T02:37:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.989+0000 7ffb5b9bb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb540c9f80 0x7ffb54157f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.989+0000 7ffb5b9bb700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb5406c880 con 0x7ffb540c9bb0 2024-01-27T02:37:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.989+0000 7ffb5b9bb700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb5406bc70 con 0x7ffb5406e330 2024-01-27T02:37:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.989+0000 7ffb5b9bb700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb5406cd10 con 0x7ffb540c9f80 2024-01-27T02:37:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.989+0000 7ffb58f56700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb540c9f80 0x7ffb54157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.989+0000 7ffb59757700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ffb540c9bb0 0x7ffb5406b400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50744/0 (socket says 172.21.15.7:50744) 2024-01-27T02:37:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.989+0000 7ffb58f56700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb540c9f80 0x7ffb54157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44030/0 (socket says 172.21.15.7:44030) 2024-01-27T02:37:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.989+0000 7ffb59757700 1 -- 172.21.15.7:0/4225853278 learned_addr learned my addr 172.21.15.7:0/4225853278 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.989+0000 7ffb58f56700 1 -- 172.21.15.7:0/4225853278 >> v1:172.21.15.188:6789/0 conn(0x7ffb5406e330 legacy=0x7ffb5406e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.989+0000 7ffb58f56700 1 -- 172.21.15.7:0/4225853278 >> v1:172.21.15.7:6789/0 conn(0x7ffb540c9bb0 legacy=0x7ffb5406b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.989+0000 7ffb58f56700 1 -- 172.21.15.7:0/4225853278 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5406bf50 con 0x7ffb540c9f80 2024-01-27T02:37:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.989+0000 7ffb58f56700 1 --2- 172.21.15.7:0/4225853278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb540c9f80 0x7ffb54157f70 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400bb90 tx=0x7ffb4400b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22051de973c9868b server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.989+0000 7ffb4bfff700 1 -- 172.21.15.7:0/4225853278 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4400dba0 con 0x7ffb540c9f80 2024-01-27T02:37:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.990+0000 7ffb4bfff700 1 -- 172.21.15.7:0/4225853278 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4400dd00 con 0x7ffb540c9f80 2024-01-27T02:37:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.990+0000 7ffb4bfff700 1 -- 172.21.15.7:0/4225853278 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb440196b0 con 0x7ffb540c9f80 2024-01-27T02:37:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.990+0000 7ffb5b9bb700 1 -- 172.21.15.7:0/4225853278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb540c9f80 msgr2=0x7ffb54157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.990+0000 7ffb5b9bb700 1 --2- 172.21.15.7:0/4225853278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb540c9f80 0x7ffb54157f70 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400bb90 tx=0x7ffb4400b430 comp rx=0 tx=0).stop 2024-01-27T02:37:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.990+0000 7ffb5b9bb700 1 -- 172.21.15.7:0/4225853278 shutdown_connections 2024-01-27T02:37:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.990+0000 7ffb5b9bb700 1 --2- 172.21.15.7:0/4225853278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb540c9f80 0x7ffb54157f70 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.990+0000 7ffb5b9bb700 1 -- 172.21.15.7:0/4225853278 >> 172.21.15.7:0/4225853278 conn(0x7ffb540cdc30 msgr2=0x7ffb540ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.990+0000 7ffb5b9bb700 1 -- 172.21.15.7:0/4225853278 shutdown_connections 2024-01-27T02:37:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.990+0000 7ffb5b9bb700 1 -- 172.21.15.7:0/4225853278 wait complete. 2024-01-27T02:37:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.991+0000 7ffb5b9bb700 1 Processor -- start 2024-01-27T02:37:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.991+0000 7ffb5b9bb700 1 -- start start 2024-01-27T02:37:51.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.992+0000 7ffb5b9bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb5406e330 0x7ffb54140d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:51.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.992+0000 7ffb5b9bb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb540c9bb0 0x7ffb5414e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:51.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.992+0000 7ffb5b9bb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb540c9f80 0x7ffb5415fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:51.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.992+0000 7ffb5b9bb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb540b3740 con 0x7ffb5406e330 2024-01-27T02:37:51.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.992+0000 7ffb5b9bb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5414fb50 con 0x7ffb540c9bb0 2024-01-27T02:37:51.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.992+0000 7ffb5b9bb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb54141290 con 0x7ffb540c9f80 2024-01-27T02:37:51.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.992+0000 7ffb59757700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb5406e330 0x7ffb54140d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:51.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.993+0000 7ffb59757700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb5406e330 0x7ffb54140d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44036/0 (socket says 172.21.15.7:44036) 2024-01-27T02:37:51.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.993+0000 7ffb59757700 1 -- 172.21.15.7:0/3217253901 learned_addr learned my addr 172.21.15.7:0/3217253901 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:51.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.993+0000 7ffb58f56700 1 --2- 172.21.15.7:0/3217253901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb540c9bb0 0x7ffb5414e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:51.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.993+0000 7ffb59f58700 1 --2- 172.21.15.7:0/3217253901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb540c9f80 0x7ffb5415fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:51.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.993+0000 7ffb59757700 1 -- 172.21.15.7:0/3217253901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb540c9f80 msgr2=0x7ffb5415fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:51.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.993+0000 7ffb59757700 1 --2- 172.21.15.7:0/3217253901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb540c9f80 0x7ffb5415fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:51.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.993+0000 7ffb59757700 1 -- 172.21.15.7:0/3217253901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb540c9bb0 msgr2=0x7ffb5414e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:51.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.993+0000 7ffb59757700 1 --2- 172.21.15.7:0/3217253901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb540c9bb0 0x7ffb5414e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:51.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.993+0000 7ffb59757700 1 -- 172.21.15.7:0/3217253901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4400b6d0 con 0x7ffb5406e330 2024-01-27T02:37:51.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.993+0000 7ffb58f56700 1 --2- 172.21.15.7:0/3217253901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb540c9bb0 0x7ffb5414e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:37:51.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.993+0000 7ffb59757700 1 --2- 172.21.15.7:0/3217253901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb5406e330 0x7ffb54140d50 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000c7f0 tx=0x7ffb50007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:51.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.993+0000 7ffb497fa700 1 -- 172.21.15.7:0/3217253901 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50005b30 con 0x7ffb5406e330 2024-01-27T02:37:51.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.993+0000 7ffb497fa700 1 -- 172.21.15.7:0/3217253901 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb50005cb0 con 0x7ffb5406e330 2024-01-27T02:37:51.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.993+0000 7ffb497fa700 1 -- 172.21.15.7:0/3217253901 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50004a80 con 0x7ffb5406e330 2024-01-27T02:37:51.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.993+0000 7ffb5b9bb700 1 -- 172.21.15.7:0/3217253901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb5414f2b0 con 0x7ffb5406e330 2024-01-27T02:37:51.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.993+0000 7ffb5b9bb700 1 -- 172.21.15.7:0/3217253901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5414f870 con 0x7ffb5406e330 2024-01-27T02:37:51.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.994+0000 7ffb5b9bb700 1 -- 172.21.15.7:0/3217253901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb5404c710 con 0x7ffb5406e330 2024-01-27T02:37:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.995+0000 7ffb497fa700 1 -- 172.21.15.7:0/3217253901 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ffb500070f0 con 0x7ffb5406e330 2024-01-27T02:37:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.995+0000 7ffb497fa700 1 --2- 172.21.15.7:0/3217253901 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb38066f30 0x7ffb380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.995+0000 7ffb497fa700 1 -- 172.21.15.7:0/3217253901 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffb50083970 con 0x7ffb5406e330 2024-01-27T02:37:51.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.995+0000 7ffb58f56700 1 --2- 172.21.15.7:0/3217253901 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb38066f30 0x7ffb380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:51.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:51.996+0000 7ffb58f56700 1 --2- 172.21.15.7:0/3217253901 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb38066f30 0x7ffb380693e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ffb540ca380 tx=0x7ffb4401b000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:52.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.000+0000 7ffb497fa700 1 -- 172.21.15.7:0/3217253901 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb50050c60 con 0x7ffb5406e330 2024-01-27T02:37:52.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.205+0000 7ffb5b9bb700 1 -- 172.21.15.7:0/3217253901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb5406f380 con 0x7ffb5406e330 2024-01-27T02:37:52.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.206+0000 7ffb497fa700 1 -- 172.21.15.7:0/3217253901 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+15330 (secure 0 0 0) 0x7ffb50018070 con 0x7ffb5406e330 2024-01-27T02:37:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:52.207 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":116,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:37:51.231585+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:37:49.747694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-27T02:37:50.199870+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:37:52.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.208+0000 7ffb5b9bb700 1 -- 172.21.15.7:0/3217253901 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb38066f30 msgr2=0x7ffb380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:52.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.208+0000 7ffb5b9bb700 1 --2- 172.21.15.7:0/3217253901 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb38066f30 0x7ffb380693e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ffb540ca380 tx=0x7ffb4401b000 comp rx=0 tx=0).stop 2024-01-27T02:37:52.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.208+0000 7ffb5b9bb700 1 -- 172.21.15.7:0/3217253901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb5406e330 msgr2=0x7ffb54140d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:52.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.208+0000 7ffb5b9bb700 1 --2- 172.21.15.7:0/3217253901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb5406e330 0x7ffb54140d50 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000c7f0 tx=0x7ffb50007f70 comp rx=0 tx=0).stop 2024-01-27T02:37:52.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.208+0000 7ffb5b9bb700 1 -- 172.21.15.7:0/3217253901 shutdown_connections 2024-01-27T02:37:52.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.208+0000 7ffb5b9bb700 1 --2- 172.21.15.7:0/3217253901 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb38066f30 0x7ffb380693e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:52.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.208+0000 7ffb5b9bb700 1 --2- 172.21.15.7:0/3217253901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb540c9f80 0x7ffb5415fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:52.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.208+0000 7ffb5b9bb700 1 --2- 172.21.15.7:0/3217253901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb5406e330 0x7ffb54140d50 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:52.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.208+0000 7ffb5b9bb700 1 --2- 172.21.15.7:0/3217253901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb540c9bb0 0x7ffb5414e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:52.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.208+0000 7ffb5b9bb700 1 -- 172.21.15.7:0/3217253901 >> 172.21.15.7:0/3217253901 conn(0x7ffb540cdc30 msgr2=0x7ffb541510d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:52.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.208+0000 7ffb5b9bb700 1 -- 172.21.15.7:0/3217253901 shutdown_connections 2024-01-27T02:37:52.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.208+0000 7ffb5b9bb700 1 -- 172.21.15.7:0/3217253901 wait complete. 2024-01-27T02:37:52.224 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:37:52.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.354+0000 7f58f6953700 1 Processor -- start 2024-01-27T02:37:52.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.354+0000 7f58f6953700 1 -- start start 2024-01-27T02:37:52.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.355+0000 7f58f6953700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58f00637d0 0x7f58f006ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:52.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.355+0000 7f58f6953700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f58f0063ba0 0x7f58f00c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:52.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.355+0000 7f58f6953700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f015b6c0 con 0x7f58f00c41e0 2024-01-27T02:37:52.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.355+0000 7f58f6953700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f015ae50 con 0x7f58f0063ba0 2024-01-27T02:37:52.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.355+0000 7f58f6953700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f015b100 con 0x7f58f00637d0 2024-01-27T02:37:52.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.355+0000 7f58effff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58f00637d0 0x7f58f006ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:52.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.355+0000 7f58f4ef0700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f58f0063ba0 0x7f58f00c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:52.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.355+0000 7f58f4ef0700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f58f0063ba0 0x7f58f00c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:56952/0 (socket says 172.21.15.7:56952) 2024-01-27T02:37:52.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.355+0000 7f58effff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58f00637d0 0x7f58f006ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:56704/0 (socket says 172.21.15.7:56704) 2024-01-27T02:37:52.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.355+0000 7f58f4ef0700 1 -- 172.21.15.7:0/4281020730 learned_addr learned my addr 172.21.15.7:0/4281020730 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:52.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.356+0000 7f58effff700 1 -- 172.21.15.7:0/4281020730 >> v2:172.21.15.188:3300/0 conn(0x7f58f0063ba0 msgr2=0x7f58f00c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.356+0000 7f58effff700 1 --2- 172.21.15.7:0/4281020730 >> v2:172.21.15.188:3300/0 conn(0x7f58f0063ba0 0x7f58f00c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.356+0000 7f58effff700 1 -- 172.21.15.7:0/4281020730 >> v1:172.21.15.188:6789/0 conn(0x7f58f00c41e0 legacy=0x7f58f015a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.356+0000 7f58effff700 1 -- 172.21.15.7:0/4281020730 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f015b3e0 con 0x7f58f00637d0 2024-01-27T02:37:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.356+0000 7f58f4ef0700 1 --2- 172.21.15.7:0/4281020730 >> v2:172.21.15.188:3300/0 conn(0x7f58f0063ba0 0x7f58f00c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:37:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.356+0000 7f58eeffd700 1 -- 172.21.15.7:0/4281020730 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81104539 0 0) 0x7f58f015b6c0 con 0x7f58f00c41e0 2024-01-27T02:37:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.356+0000 7f58effff700 1 --2- 172.21.15.7:0/4281020730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58f00637d0 0x7f58f006ebd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f58e00098e0 tx=0x7f58e00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59e5a2d87eb73ee4 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.356+0000 7f58eeffd700 1 -- 172.21.15.7:0/4281020730 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e000b640 con 0x7f58f00637d0 2024-01-27T02:37:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.356+0000 7f58eeffd700 1 -- 172.21.15.7:0/4281020730 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e000b7a0 con 0x7f58f00637d0 2024-01-27T02:37:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.356+0000 7f58eeffd700 1 -- 172.21.15.7:0/4281020730 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e000bb40 con 0x7f58f00637d0 2024-01-27T02:37:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.357+0000 7f58f6953700 1 -- 172.21.15.7:0/4281020730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58f00637d0 msgr2=0x7f58f006ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.357+0000 7f58f6953700 1 --2- 172.21.15.7:0/4281020730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58f00637d0 0x7f58f006ebd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f58e00098e0 tx=0x7f58e00091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.357+0000 7f58f6953700 1 -- 172.21.15.7:0/4281020730 shutdown_connections 2024-01-27T02:37:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.357+0000 7f58f6953700 1 --2- 172.21.15.7:0/4281020730 >> v2:172.21.15.188:3300/0 conn(0x7f58f0063ba0 0x7f58f00c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.357+0000 7f58f6953700 1 --2- 172.21.15.7:0/4281020730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58f00637d0 0x7f58f006ebd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.357+0000 7f58f6953700 1 -- 172.21.15.7:0/4281020730 >> 172.21.15.7:0/4281020730 conn(0x7f58f00cdc30 msgr2=0x7f58f00ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.357+0000 7f58f6953700 1 -- 172.21.15.7:0/4281020730 shutdown_connections 2024-01-27T02:37:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.357+0000 7f58f6953700 1 -- 172.21.15.7:0/4281020730 wait complete. 2024-01-27T02:37:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.358+0000 7f58f6953700 1 Processor -- start 2024-01-27T02:37:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.358+0000 7f58f6953700 1 -- start start 2024-01-27T02:37:52.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.359+0000 7f58f6953700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f00637d0 0x7f58f00afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:52.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.359+0000 7f58f6953700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58f0063ba0 0x7f58f00ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:52.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.359+0000 7f58f6953700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58f00c41e0 0x7f58f0158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:52.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.359+0000 7f58f6953700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f00b1680 con 0x7f58f00637d0 2024-01-27T02:37:52.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.359+0000 7f58f6953700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f00b0a70 con 0x7f58f00c41e0 2024-01-27T02:37:52.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.359+0000 7f58f6953700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f00b0d50 con 0x7f58f0063ba0 2024-01-27T02:37:52.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.359+0000 7f58effff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f00637d0 0x7f58f00afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:52.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.359+0000 7f58effff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f00637d0 0x7f58f00afa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44052/0 (socket says 172.21.15.7:44052) 2024-01-27T02:37:52.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.359+0000 7f58effff700 1 -- 172.21.15.7:0/2035819843 learned_addr learned my addr 172.21.15.7:0/2035819843 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:52.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.359+0000 7f58f4ef0700 1 --2- 172.21.15.7:0/2035819843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58f00c41e0 0x7f58f0158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:52.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.360+0000 7f58ef7fe700 1 --2- 172.21.15.7:0/2035819843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58f0063ba0 0x7f58f00ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:52.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.360+0000 7f58effff700 1 -- 172.21.15.7:0/2035819843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58f0063ba0 msgr2=0x7f58f00ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:52.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.360+0000 7f58effff700 1 --2- 172.21.15.7:0/2035819843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58f0063ba0 0x7f58f00ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:52.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.360+0000 7f58effff700 1 -- 172.21.15.7:0/2035819843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58f00c41e0 msgr2=0x7f58f0158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:52.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.360+0000 7f58effff700 1 --2- 172.21.15.7:0/2035819843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58f00c41e0 0x7f58f0158160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:52.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.360+0000 7f58effff700 1 -- 172.21.15.7:0/2035819843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e8009ae0 con 0x7f58f00637d0 2024-01-27T02:37:52.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.360+0000 7f58effff700 1 --2- 172.21.15.7:0/2035819843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f00637d0 0x7f58f00afa40 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f58f006e130 tx=0x7f58e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:52.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.360+0000 7f58d7fff700 1 -- 172.21.15.7:0/2035819843 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e000bb20 con 0x7f58f00637d0 2024-01-27T02:37:52.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.360+0000 7f58d7fff700 1 -- 172.21.15.7:0/2035819843 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e0011420 con 0x7f58f00637d0 2024-01-27T02:37:52.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.360+0000 7f58d7fff700 1 -- 172.21.15.7:0/2035819843 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e00117c0 con 0x7f58f00637d0 2024-01-27T02:37:52.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.360+0000 7f58f6953700 1 -- 172.21.15.7:0/2035819843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e0009420 con 0x7f58f00637d0 2024-01-27T02:37:52.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.360+0000 7f58f6953700 1 -- 172.21.15.7:0/2035819843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f00b1310 con 0x7f58f00637d0 2024-01-27T02:37:52.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.361+0000 7f58f6953700 1 -- 172.21.15.7:0/2035819843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f004c710 con 0x7f58f00637d0 2024-01-27T02:37:52.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.362+0000 7f58d7fff700 1 -- 172.21.15.7:0/2035819843 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f58e0011b70 con 0x7f58f00637d0 2024-01-27T02:37:52.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.362+0000 7f58d7fff700 1 --2- 172.21.15.7:0/2035819843 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f58dc066f30 0x7f58dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:52.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.362+0000 7f58d7fff700 1 -- 172.21.15.7:0/2035819843 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f58e0086060 con 0x7f58f00637d0 2024-01-27T02:37:52.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.363+0000 7f58ef7fe700 1 --2- 172.21.15.7:0/2035819843 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f58dc066f30 0x7f58dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:52.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.363+0000 7f58ef7fe700 1 --2- 172.21.15.7:0/2035819843 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f58dc066f30 0x7f58dc0693e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f58f015cd90 tx=0x7f58e4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:52.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.366+0000 7f58d7fff700 1 -- 172.21.15.7:0/2035819843 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f58e00532d0 con 0x7f58f00637d0 2024-01-27T02:37:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.570+0000 7f58f6953700 1 -- 172.21.15.7:0/2035819843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58f006f750 con 0x7f58f00637d0 2024-01-27T02:37:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.571+0000 7f58d7fff700 1 -- 172.21.15.7:0/2035819843 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+15330 (secure 0 0 0) 0x7f58e00568f0 con 0x7f58f00637d0 2024-01-27T02:37:52.572 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:52.572 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":116,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:37:51.231585+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:37:49.747694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-27T02:37:50.199870+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:37:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.573+0000 7f58f6953700 1 -- 172.21.15.7:0/2035819843 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f58dc066f30 msgr2=0x7f58dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.573+0000 7f58f6953700 1 --2- 172.21.15.7:0/2035819843 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f58dc066f30 0x7f58dc0693e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f58f015cd90 tx=0x7f58e4009040 comp rx=0 tx=0).stop 2024-01-27T02:37:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.573+0000 7f58f6953700 1 -- 172.21.15.7:0/2035819843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f00637d0 msgr2=0x7f58f00afa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.573+0000 7f58f6953700 1 --2- 172.21.15.7:0/2035819843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f00637d0 0x7f58f00afa40 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f58f006e130 tx=0x7f58e0017000 comp rx=0 tx=0).stop 2024-01-27T02:37:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.573+0000 7f58f6953700 1 -- 172.21.15.7:0/2035819843 shutdown_connections 2024-01-27T02:37:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.573+0000 7f58f6953700 1 --2- 172.21.15.7:0/2035819843 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f58dc066f30 0x7f58dc0693e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.573+0000 7f58f6953700 1 --2- 172.21.15.7:0/2035819843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58f00c41e0 0x7f58f0158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.573+0000 7f58f6953700 1 --2- 172.21.15.7:0/2035819843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f00637d0 0x7f58f00afa40 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:52.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.573+0000 7f58f6953700 1 --2- 172.21.15.7:0/2035819843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58f0063ba0 0x7f58f00ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:52.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.573+0000 7f58f6953700 1 -- 172.21.15.7:0/2035819843 >> 172.21.15.7:0/2035819843 conn(0x7f58f00cdc30 msgr2=0x7f58f00ab6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:52.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.573+0000 7f58f6953700 1 -- 172.21.15.7:0/2035819843 shutdown_connections 2024-01-27T02:37:52.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.573+0000 7f58f6953700 1 -- 172.21.15.7:0/2035819843 wait complete. 2024-01-27T02:37:52.589 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-27T02:37:52.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.722+0000 7fe022c82700 1 Processor -- start 2024-01-27T02:37:52.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.722+0000 7fe022c82700 1 -- start start 2024-01-27T02:37:52.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.724+0000 7fe022c82700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe01c156b60 0x7fe01c0ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:52.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.724+0000 7fe022c82700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe01c0ad9f0 0x7fe01c0aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:52.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.724+0000 7fe022c82700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe01c0af470 0x7fe01c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:52.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.724+0000 7fe022c82700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe01c0aeaa0 con 0x7fe01c0ad9f0 2024-01-27T02:37:52.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.724+0000 7fe022c82700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe01c0ade90 con 0x7fe01c0af470 2024-01-27T02:37:52.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.724+0000 7fe022c82700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c0ae170 con 0x7fe01c156b60 2024-01-27T02:37:52.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.724+0000 7fe01bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe01c0ad9f0 0x7fe01c0aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:52.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.724+0000 7fe01bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe01c0ad9f0 0x7fe01c0aef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44054/0 (socket says 172.21.15.7:44054) 2024-01-27T02:37:52.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.724+0000 7fe01bfff700 1 -- 172.21.15.7:0/3690695338 learned_addr learned my addr 172.21.15.7:0/3690695338 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:52.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.724+0000 7fe02121f700 1 --2- 172.21.15.7:0/3690695338 >> v2:172.21.15.188:3300/0 conn(0x7fe01c0af470 0x7fe01c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:52.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.724+0000 7fe020a1e700 1 --2- 172.21.15.7:0/3690695338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe01c156b60 0x7fe01c0ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:52.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.724+0000 7fe01bfff700 1 -- 172.21.15.7:0/3690695338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe01c156b60 msgr2=0x7fe01c0ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:52.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.724+0000 7fe01bfff700 1 --2- 172.21.15.7:0/3690695338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe01c156b60 0x7fe01c0ad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:52.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.724+0000 7fe01bfff700 1 -- 172.21.15.7:0/3690695338 >> v2:172.21.15.188:3300/0 conn(0x7fe01c0af470 msgr2=0x7fe01c15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:52.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.724+0000 7fe01bfff700 1 --2- 172.21.15.7:0/3690695338 >> v2:172.21.15.188:3300/0 conn(0x7fe01c0af470 0x7fe01c15a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:52.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.724+0000 7fe01bfff700 1 -- 172.21.15.7:0/3690695338 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c0ae450 con 0x7fe01c0ad9f0 2024-01-27T02:37:52.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.724+0000 7fe01bfff700 1 --2- 172.21.15.7:0/3690695338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe01c0ad9f0 0x7fe01c0aef30 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe01c06dab0 tx=0x7fe010009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa8eba8576fb3f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:52.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.725+0000 7fe01b7fe700 1 -- 172.21.15.7:0/3690695338 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01000b640 con 0x7fe01c0ad9f0 2024-01-27T02:37:52.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.725+0000 7fe01b7fe700 1 -- 172.21.15.7:0/3690695338 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01000b7a0 con 0x7fe01c0ad9f0 2024-01-27T02:37:52.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.725+0000 7fe01b7fe700 1 -- 172.21.15.7:0/3690695338 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01000bb60 con 0x7fe01c0ad9f0 2024-01-27T02:37:52.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.725+0000 7fe022c82700 1 -- 172.21.15.7:0/3690695338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe01c0ad9f0 msgr2=0x7fe01c0aef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:52.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.725+0000 7fe022c82700 1 --2- 172.21.15.7:0/3690695338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe01c0ad9f0 0x7fe01c0aef30 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe01c06dab0 tx=0x7fe010009250 comp rx=0 tx=0).stop 2024-01-27T02:37:52.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.725+0000 7fe022c82700 1 -- 172.21.15.7:0/3690695338 shutdown_connections 2024-01-27T02:37:52.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.725+0000 7fe022c82700 1 --2- 172.21.15.7:0/3690695338 >> v2:172.21.15.188:3300/0 conn(0x7fe01c0af470 0x7fe01c15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:52.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.725+0000 7fe022c82700 1 --2- 172.21.15.7:0/3690695338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe01c156b60 0x7fe01c0ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:52.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.725+0000 7fe022c82700 1 --2- 172.21.15.7:0/3690695338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe01c0ad9f0 0x7fe01c0aef30 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:52.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.725+0000 7fe022c82700 1 -- 172.21.15.7:0/3690695338 >> 172.21.15.7:0/3690695338 conn(0x7fe01c068480 msgr2=0x7fe01c0cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:52.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.725+0000 7fe022c82700 1 -- 172.21.15.7:0/3690695338 shutdown_connections 2024-01-27T02:37:52.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.725+0000 7fe022c82700 1 -- 172.21.15.7:0/3690695338 wait complete. 2024-01-27T02:37:52.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.726+0000 7fe022c82700 1 Processor -- start 2024-01-27T02:37:52.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.727+0000 7fe022c82700 1 -- start start 2024-01-27T02:37:52.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.727+0000 7fe022c82700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe01c0ad9f0 0x7fe01c0ac520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:52.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.727+0000 7fe022c82700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe01c0af470 0x7fe01c0a9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:52.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.727+0000 7fe022c82700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe01c156b60 0x7fe01c0cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.728+0000 7fe022c82700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c0aa5b0 con 0x7fe01c0ad9f0 2024-01-27T02:37:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.728+0000 7fe022c82700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c0aa890 con 0x7fe01c156b60 2024-01-27T02:37:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.728+0000 7fe022c82700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c0aab70 con 0x7fe01c0af470 2024-01-27T02:37:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.728+0000 7fe020a1e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe01c0ad9f0 0x7fe01c0ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.728+0000 7fe020a1e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe01c0ad9f0 0x7fe01c0ac520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44068/0 (socket says 172.21.15.7:44068) 2024-01-27T02:37:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.728+0000 7fe020a1e700 1 -- 172.21.15.7:0/2036575741 learned_addr learned my addr 172.21.15.7:0/2036575741 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.728+0000 7fe01bfff700 1 --2- 172.21.15.7:0/2036575741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe01c0af470 0x7fe01c0a9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.728+0000 7fe02121f700 1 --2- 172.21.15.7:0/2036575741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe01c156b60 0x7fe01c0cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.728+0000 7fe020a1e700 1 -- 172.21.15.7:0/2036575741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe01c0af470 msgr2=0x7fe01c0a9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.728+0000 7fe020a1e700 1 --2- 172.21.15.7:0/2036575741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe01c0af470 0x7fe01c0a9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.728+0000 7fe020a1e700 1 -- 172.21.15.7:0/2036575741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe01c156b60 msgr2=0x7fe01c0cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.728+0000 7fe020a1e700 1 --2- 172.21.15.7:0/2036575741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe01c156b60 0x7fe01c0cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.728+0000 7fe020a1e700 1 -- 172.21.15.7:0/2036575741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0100094f0 con 0x7fe01c0ad9f0 2024-01-27T02:37:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.728+0000 7fe01bfff700 1 --2- 172.21.15.7:0/2036575741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe01c0af470 0x7fe01c0a9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:37:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.728+0000 7fe02121f700 1 --2- 172.21.15.7:0/2036575741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe01c156b60 0x7fe01c0cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:37:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.728+0000 7fe020a1e700 1 --2- 172.21.15.7:0/2036575741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe01c0ad9f0 0x7fe01c0ac520 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00e930 tx=0x7fe00c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.728+0000 7fe018ff9700 1 -- 172.21.15.7:0/2036575741 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c00eed0 con 0x7fe01c0ad9f0 2024-01-27T02:37:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.728+0000 7fe018ff9700 1 -- 172.21.15.7:0/2036575741 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00c00c850 con 0x7fe01c0ad9f0 2024-01-27T02:37:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.728+0000 7fe022c82700 1 -- 172.21.15.7:0/2036575741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01c0aae50 con 0x7fe01c0ad9f0 2024-01-27T02:37:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.728+0000 7fe018ff9700 1 -- 172.21.15.7:0/2036575741 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c00cbf0 con 0x7fe01c0ad9f0 2024-01-27T02:37:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.728+0000 7fe022c82700 1 -- 172.21.15.7:0/2036575741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01c0aa2c0 con 0x7fe01c0ad9f0 2024-01-27T02:37:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.730+0000 7fe022c82700 1 -- 172.21.15.7:0/2036575741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe01c072060 con 0x7fe01c0ad9f0 2024-01-27T02:37:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.730+0000 7fe018ff9700 1 -- 172.21.15.7:0/2036575741 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe00c013470 con 0x7fe01c0ad9f0 2024-01-27T02:37:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.730+0000 7fe018ff9700 1 --2- 172.21.15.7:0/2036575741 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe008066f30 0x7fe0080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.731+0000 7fe018ff9700 1 -- 172.21.15.7:0/2036575741 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe00c084760 con 0x7fe01c0ad9f0 2024-01-27T02:37:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.731+0000 7fe01bfff700 1 --2- 172.21.15.7:0/2036575741 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe008066f30 0x7fe0080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:52.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.731+0000 7fe01bfff700 1 --2- 172.21.15.7:0/2036575741 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe008066f30 0x7fe0080693e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe01c06dab0 tx=0x7fe010017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:52.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.735+0000 7fe018ff9700 1 -- 172.21.15.7:0/2036575741 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe00c0519d0 con 0x7fe01c0ad9f0 2024-01-27T02:37:52.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:52.994+0000 7fe022c82700 1 -- 172.21.15.7:0/2036575741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe01c0ae730 con 0x7fe01c0ad9f0 2024-01-27T02:37:53.788 INFO:tasks.ceph.mds.a.smithi007.stderr:2024-01-27T02:37:53.787+0000 7f95be999700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-27T02:37:53.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:53.851+0000 7fe018ff9700 1 -- 172.21.15.7:0/2036575741 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v57) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe00c054ff0 con 0x7fe01c0ad9f0 2024-01-27T02:37:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:53.854+0000 7fe022c82700 1 -- 172.21.15.7:0/2036575741 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe008066f30 msgr2=0x7fe0080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:53.854+0000 7fe022c82700 1 --2- 172.21.15.7:0/2036575741 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe008066f30 0x7fe0080693e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe01c06dab0 tx=0x7fe010017000 comp rx=0 tx=0).stop 2024-01-27T02:37:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:53.854+0000 7fe022c82700 1 -- 172.21.15.7:0/2036575741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe01c0ad9f0 msgr2=0x7fe01c0ac520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:53.854+0000 7fe022c82700 1 --2- 172.21.15.7:0/2036575741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe01c0ad9f0 0x7fe01c0ac520 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00e930 tx=0x7fe00c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:53.854+0000 7fe022c82700 1 -- 172.21.15.7:0/2036575741 shutdown_connections 2024-01-27T02:37:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:53.854+0000 7fe022c82700 1 --2- 172.21.15.7:0/2036575741 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe008066f30 0x7fe0080693e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:53.854+0000 7fe022c82700 1 --2- 172.21.15.7:0/2036575741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe01c156b60 0x7fe01c0cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:53.854+0000 7fe022c82700 1 --2- 172.21.15.7:0/2036575741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe01c0ad9f0 0x7fe01c0ac520 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:53.854+0000 7fe022c82700 1 --2- 172.21.15.7:0/2036575741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe01c0af470 0x7fe01c0a9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:53.854+0000 7fe022c82700 1 -- 172.21.15.7:0/2036575741 >> 172.21.15.7:0/2036575741 conn(0x7fe01c068480 msgr2=0x7fe01c0a7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:53.854+0000 7fe022c82700 1 -- 172.21.15.7:0/2036575741 shutdown_connections 2024-01-27T02:37:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:53.854+0000 7fe022c82700 1 -- 172.21.15.7:0/2036575741 wait complete. 2024-01-27T02:37:53.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-27T02:37:54.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.009+0000 7f82a4044700 1 Processor -- start 2024-01-27T02:37:54.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.009+0000 7f82a4044700 1 -- start start 2024-01-27T02:37:54.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.010+0000 7f82a4044700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f829c063c90 0x7f829c0a8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:54.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.010+0000 7f82a4044700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f829c0a80c0 con 0x7f829c15e3d0 2024-01-27T02:37:54.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.010+0000 7f82a4044700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f829c0a7790 con 0x7f829c15e7a0 2024-01-27T02:37:54.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.010+0000 7f82a4044700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f829c0ce6b0 con 0x7f829c063c90 2024-01-27T02:37:54.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.010+0000 7f82a1de0700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f829c15e3d0 0x7f829c063580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50752/0 (socket says 172.21.15.7:50752) 2024-01-27T02:37:54.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.010+0000 7f82a1de0700 1 -- 172.21.15.7:0/1355350988 learned_addr learned my addr 172.21.15.7:0/1355350988 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:54.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.010+0000 7f82a25e1700 1 --2- 172.21.15.7:0/1355350988 >> v2:172.21.15.188:3300/0 conn(0x7f829c063c90 0x7f829c0a8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:54.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.011+0000 7f82a0dde700 1 -- 172.21.15.7:0/1355350988 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2164038306 0 0) 0x7f829c0a80c0 con 0x7f829c15e3d0 2024-01-27T02:37:54.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.011+0000 7f82a0dde700 1 -- 172.21.15.7:0/1355350988 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8288003570 con 0x7f829c15e3d0 2024-01-27T02:37:54.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.011+0000 7f82a25e1700 1 -- 172.21.15.7:0/1355350988 >> v1:172.21.15.188:6789/0 conn(0x7f829c15e7a0 legacy=0x7f829c0afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:54.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.011+0000 7f82a25e1700 1 -- 172.21.15.7:0/1355350988 >> v1:172.21.15.7:6789/0 conn(0x7f829c15e3d0 legacy=0x7f829c063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:54.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.011+0000 7f82a25e1700 1 -- 172.21.15.7:0/1355350988 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829c0a7a70 con 0x7f829c063c90 2024-01-27T02:37:54.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.011+0000 7f82a25e1700 1 --2- 172.21.15.7:0/1355350988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f829c063c90 0x7f829c0a8550 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8294009e60 tx=0x7f8294009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb394310cbe297af server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:54.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.011+0000 7f82a0dde700 1 -- 172.21.15.7:0/1355350988 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829400e070 con 0x7f829c063c90 2024-01-27T02:37:54.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.011+0000 7f82a0dde700 1 -- 172.21.15.7:0/1355350988 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829400bb60 con 0x7f829c063c90 2024-01-27T02:37:54.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.011+0000 7f82a0dde700 1 -- 172.21.15.7:0/1355350988 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8294012430 con 0x7f829c063c90 2024-01-27T02:37:54.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.012+0000 7f82a4044700 1 -- 172.21.15.7:0/1355350988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f829c063c90 msgr2=0x7f829c0a8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.012+0000 7f82a4044700 1 --2- 172.21.15.7:0/1355350988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f829c063c90 0x7f829c0a8550 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8294009e60 tx=0x7f8294009780 comp rx=0 tx=0).stop 2024-01-27T02:37:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.012+0000 7f82a4044700 1 -- 172.21.15.7:0/1355350988 shutdown_connections 2024-01-27T02:37:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.012+0000 7f82a4044700 1 --2- 172.21.15.7:0/1355350988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f829c063c90 0x7f829c0a8550 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.012+0000 7f82a4044700 1 -- 172.21.15.7:0/1355350988 >> 172.21.15.7:0/1355350988 conn(0x7f829c068480 msgr2=0x7f829c0cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.012+0000 7f82a4044700 1 -- 172.21.15.7:0/1355350988 shutdown_connections 2024-01-27T02:37:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.012+0000 7f82a4044700 1 -- 172.21.15.7:0/1355350988 wait complete. 2024-01-27T02:37:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.014+0000 7f82a4044700 1 Processor -- start 2024-01-27T02:37:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.014+0000 7f82a4044700 1 -- start start 2024-01-27T02:37:54.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.015+0000 7f82a4044700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f829c063c90 0x7f829c1500a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.015+0000 7f82a4044700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f829c15e3d0 0x7f829c160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.015+0000 7f82a4044700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f829c15e7a0 0x7f829c15cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.015+0000 7f82a4044700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c16e5a0 con 0x7f829c063c90 2024-01-27T02:37:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.015+0000 7f82a4044700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c16f7f0 con 0x7f829c15e3d0 2024-01-27T02:37:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.015+0000 7f82a4044700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829c16e790 con 0x7f829c15e7a0 2024-01-27T02:37:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.015+0000 7f82a1de0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f829c063c90 0x7f829c1500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.015+0000 7f82a1de0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f829c063c90 0x7f829c1500a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44076/0 (socket says 172.21.15.7:44076) 2024-01-27T02:37:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.015+0000 7f82a1de0700 1 -- 172.21.15.7:0/948038186 learned_addr learned my addr 172.21.15.7:0/948038186 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.015+0000 7f82a15df700 1 --2- 172.21.15.7:0/948038186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f829c15e3d0 0x7f829c160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.015+0000 7f82a25e1700 1 --2- 172.21.15.7:0/948038186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f829c15e7a0 0x7f829c15cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:54.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.015+0000 7f82a1de0700 1 -- 172.21.15.7:0/948038186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f829c15e7a0 msgr2=0x7f829c15cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:54.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.015+0000 7f82a1de0700 1 --2- 172.21.15.7:0/948038186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f829c15e7a0 0x7f829c15cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:54.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.015+0000 7f82a1de0700 1 -- 172.21.15.7:0/948038186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f829c15e3d0 msgr2=0x7f829c160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:54.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.015+0000 7f82a1de0700 1 --2- 172.21.15.7:0/948038186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f829c15e3d0 0x7f829c160cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:54.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.015+0000 7f82a1de0700 1 -- 172.21.15.7:0/948038186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8294009a20 con 0x7f829c063c90 2024-01-27T02:37:54.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.016+0000 7f82a1de0700 1 --2- 172.21.15.7:0/948038186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f829c063c90 0x7f829c1500a0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f829800c7c0 tx=0x7f8298007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:54.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.016+0000 7f8291ffb700 1 -- 172.21.15.7:0/948038186 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8298005cd0 con 0x7f829c063c90 2024-01-27T02:37:54.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.016+0000 7f8291ffb700 1 -- 172.21.15.7:0/948038186 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8298005e30 con 0x7f829c063c90 2024-01-27T02:37:54.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.016+0000 7f82a4044700 1 -- 172.21.15.7:0/948038186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829c16ea30 con 0x7f829c063c90 2024-01-27T02:37:54.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.016+0000 7f8291ffb700 1 -- 172.21.15.7:0/948038186 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8298007320 con 0x7f829c063c90 2024-01-27T02:37:54.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.016+0000 7f82a4044700 1 -- 172.21.15.7:0/948038186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829c16eff0 con 0x7f829c063c90 2024-01-27T02:37:54.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.017+0000 7f82a4044700 1 -- 172.21.15.7:0/948038186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f829c072060 con 0x7f829c063c90 2024-01-27T02:37:54.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.017+0000 7f8291ffb700 1 -- 172.21.15.7:0/948038186 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f829801b3b0 con 0x7f829c063c90 2024-01-27T02:37:54.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.018+0000 7f8291ffb700 1 --2- 172.21.15.7:0/948038186 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8280066f30 0x7f82800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:54.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.018+0000 7f8291ffb700 1 -- 172.21.15.7:0/948038186 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8298083bd0 con 0x7f829c063c90 2024-01-27T02:37:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.018+0000 7f82a15df700 1 --2- 172.21.15.7:0/948038186 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8280066f30 0x7f82800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.019+0000 7f82a15df700 1 --2- 172.21.15.7:0/948038186 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8280066f30 0x7f82800693e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f829c06dd60 tx=0x7f828c007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:54.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.022+0000 7f8291ffb700 1 -- 172.21.15.7:0/948038186 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8298050e40 con 0x7f829c063c90 2024-01-27T02:37:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.274+0000 7f82a4044700 1 -- 172.21.15.7:0/948038186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f829c15db90 con 0x7f829c063c90 2024-01-27T02:37:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.852+0000 7f8291ffb700 1 -- 172.21.15.7:0/948038186 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8298017020 con 0x7f829c063c90 2024-01-27T02:37:54.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.855+0000 7f82a4044700 1 -- 172.21.15.7:0/948038186 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8280066f30 msgr2=0x7f82800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:54.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.855+0000 7f82a4044700 1 --2- 172.21.15.7:0/948038186 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8280066f30 0x7f82800693e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f829c06dd60 tx=0x7f828c007ec0 comp rx=0 tx=0).stop 2024-01-27T02:37:54.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.855+0000 7f82a4044700 1 -- 172.21.15.7:0/948038186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f829c063c90 msgr2=0x7f829c1500a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:54.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.855+0000 7f82a4044700 1 --2- 172.21.15.7:0/948038186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f829c063c90 0x7f829c1500a0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f829800c7c0 tx=0x7f8298007e00 comp rx=0 tx=0).stop 2024-01-27T02:37:54.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.855+0000 7f82a4044700 1 -- 172.21.15.7:0/948038186 shutdown_connections 2024-01-27T02:37:54.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.855+0000 7f82a4044700 1 --2- 172.21.15.7:0/948038186 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8280066f30 0x7f82800693e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:54.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.855+0000 7f82a4044700 1 --2- 172.21.15.7:0/948038186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f829c15e7a0 0x7f829c15cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:54.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.855+0000 7f82a4044700 1 --2- 172.21.15.7:0/948038186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f829c063c90 0x7f829c1500a0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:54.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.855+0000 7f82a4044700 1 --2- 172.21.15.7:0/948038186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f829c15e3d0 0x7f829c160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:54.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.855+0000 7f82a4044700 1 -- 172.21.15.7:0/948038186 >> 172.21.15.7:0/948038186 conn(0x7f829c068480 msgr2=0x7f829c15aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:54.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.855+0000 7f82a4044700 1 -- 172.21.15.7:0/948038186 shutdown_connections 2024-01-27T02:37:54.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:54.855+0000 7f82a4044700 1 -- 172.21.15.7:0/948038186 wait complete. 2024-01-27T02:37:54.873 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:37:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.007+0000 7f54e6b35700 1 Processor -- start 2024-01-27T02:37:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.007+0000 7f54e6b35700 1 -- start start 2024-01-27T02:37:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.008+0000 7f54e6b35700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e0157ca0 0x7f54e00c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.008+0000 7f54e6b35700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54e00ca020 con 0x7f54e00c3ae0 2024-01-27T02:37:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.008+0000 7f54e6b35700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54e00c4f40 con 0x7f54e00cdbf0 2024-01-27T02:37:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.008+0000 7f54e6b35700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e00c5210 con 0x7f54e0157ca0 2024-01-27T02:37:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.008+0000 7f54dffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f54e00c3ae0 0x7f54e00cd4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50756/0 (socket says 172.21.15.7:50756) 2024-01-27T02:37:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.008+0000 7f54dffff700 1 -- 172.21.15.7:0/3995907901 learned_addr learned my addr 172.21.15.7:0/3995907901 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.008+0000 7f54e48d1700 1 --2- 172.21.15.7:0/3995907901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e0157ca0 0x7f54e00c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.009+0000 7f54df7fe700 1 -- 172.21.15.7:0/3995907901 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2797467253 0 0) 0x7f54e00ca020 con 0x7f54e00c3ae0 2024-01-27T02:37:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.009+0000 7f54df7fe700 1 -- 172.21.15.7:0/3995907901 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54cc003570 con 0x7f54e00c3ae0 2024-01-27T02:37:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.009+0000 7f54df7fe700 1 -- 172.21.15.7:0/3995907901 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819318007 0 0) 0x7f54e00c4f40 con 0x7f54e00cdbf0 2024-01-27T02:37:55.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.009+0000 7f54df7fe700 1 -- 172.21.15.7:0/3995907901 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54e00ca020 con 0x7f54e00cdbf0 2024-01-27T02:37:55.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.009+0000 7f54e48d1700 1 -- 172.21.15.7:0/3995907901 >> v1:172.21.15.188:6789/0 conn(0x7f54e00cdbf0 legacy=0x7f54e00ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:55.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.009+0000 7f54e48d1700 1 -- 172.21.15.7:0/3995907901 >> v1:172.21.15.7:6789/0 conn(0x7f54e00c3ae0 legacy=0x7f54e00cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:55.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.009+0000 7f54e48d1700 1 -- 172.21.15.7:0/3995907901 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e00c54f0 con 0x7f54e0157ca0 2024-01-27T02:37:55.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.009+0000 7f54e48d1700 1 --2- 172.21.15.7:0/3995907901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e0157ca0 0x7f54e00c4a00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f54d0009910 tx=0x7f54d00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=337fc717c5d18d10 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:55.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.009+0000 7f54df7fe700 1 -- 172.21.15.7:0/3995907901 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d0011630 con 0x7f54e0157ca0 2024-01-27T02:37:55.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.009+0000 7f54df7fe700 1 -- 172.21.15.7:0/3995907901 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d0011790 con 0x7f54e0157ca0 2024-01-27T02:37:55.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.009+0000 7f54df7fe700 1 -- 172.21.15.7:0/3995907901 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d0011b10 con 0x7f54e0157ca0 2024-01-27T02:37:55.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.010+0000 7f54e6b35700 1 -- 172.21.15.7:0/3995907901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e0157ca0 msgr2=0x7f54e00c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:55.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.010+0000 7f54e6b35700 1 --2- 172.21.15.7:0/3995907901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e0157ca0 0x7f54e00c4a00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f54d0009910 tx=0x7f54d00091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.010+0000 7f54e6b35700 1 -- 172.21.15.7:0/3995907901 shutdown_connections 2024-01-27T02:37:55.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.010+0000 7f54e6b35700 1 --2- 172.21.15.7:0/3995907901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e0157ca0 0x7f54e00c4a00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.010+0000 7f54e6b35700 1 -- 172.21.15.7:0/3995907901 >> 172.21.15.7:0/3995907901 conn(0x7f54e0098020 msgr2=0x7f54e00711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:55.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.010+0000 7f54e6b35700 1 -- 172.21.15.7:0/3995907901 shutdown_connections 2024-01-27T02:37:55.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.010+0000 7f54e6b35700 1 -- 172.21.15.7:0/3995907901 wait complete. 2024-01-27T02:37:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.012+0000 7f54e6b35700 1 Processor -- start 2024-01-27T02:37:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.012+0000 7f54e6b35700 1 -- start start 2024-01-27T02:37:55.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.013+0000 7f54e6b35700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54e00c3ae0 0x7f54e01687f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:55.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.013+0000 7f54e6b35700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54e00cdbf0 0x7f54e01567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:55.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.013+0000 7f54e6b35700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e0157ca0 0x7f54e00c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:55.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.013+0000 7f54e6b35700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e0156d10 con 0x7f54e00cdbf0 2024-01-27T02:37:55.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.013+0000 7f54e6b35700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e00b1da0 con 0x7f54e00c3ae0 2024-01-27T02:37:55.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.013+0000 7f54e6b35700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e00b2360 con 0x7f54e0157ca0 2024-01-27T02:37:55.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.013+0000 7f54dffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54e00cdbf0 0x7f54e01567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.013+0000 7f54dffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54e00cdbf0 0x7f54e01567d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44086/0 (socket says 172.21.15.7:44086) 2024-01-27T02:37:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.013+0000 7f54dffff700 1 -- 172.21.15.7:0/1519444934 learned_addr learned my addr 172.21.15.7:0/1519444934 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.013+0000 7f54e48d1700 1 --2- 172.21.15.7:0/1519444934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54e00c3ae0 0x7f54e01687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.013+0000 7f54e50d2700 1 --2- 172.21.15.7:0/1519444934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e0157ca0 0x7f54e00c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.013+0000 7f54dffff700 1 -- 172.21.15.7:0/1519444934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e0157ca0 msgr2=0x7f54e00c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.013+0000 7f54dffff700 1 --2- 172.21.15.7:0/1519444934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e0157ca0 0x7f54e00c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.013+0000 7f54dffff700 1 -- 172.21.15.7:0/1519444934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54e00c3ae0 msgr2=0x7f54e01687f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.013+0000 7f54dffff700 1 --2- 172.21.15.7:0/1519444934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54e00c3ae0 0x7f54e01687f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.013+0000 7f54dffff700 1 -- 172.21.15.7:0/1519444934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d0009420 con 0x7f54e00cdbf0 2024-01-27T02:37:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.013+0000 7f54dffff700 1 --2- 172.21.15.7:0/1519444934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54e00cdbf0 0x7f54e01567d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f54c800c790 tx=0x7f54c8007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.014+0000 7f54dcff9700 1 -- 172.21.15.7:0/1519444934 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c8005b30 con 0x7f54e00cdbf0 2024-01-27T02:37:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.014+0000 7f54dcff9700 1 -- 172.21.15.7:0/1519444934 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c8005c90 con 0x7f54e00cdbf0 2024-01-27T02:37:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.014+0000 7f54e6b35700 1 -- 172.21.15.7:0/1519444934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e00c90f0 con 0x7f54e00cdbf0 2024-01-27T02:37:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.014+0000 7f54dcff9700 1 -- 172.21.15.7:0/1519444934 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c8007300 con 0x7f54e00cdbf0 2024-01-27T02:37:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.014+0000 7f54e6b35700 1 -- 172.21.15.7:0/1519444934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e01571e0 con 0x7f54e00cdbf0 2024-01-27T02:37:55.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.015+0000 7f54e6b35700 1 -- 172.21.15.7:0/1519444934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e00b2510 con 0x7f54e00cdbf0 2024-01-27T02:37:55.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.015+0000 7f54dcff9700 1 -- 172.21.15.7:0/1519444934 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f54c801b400 con 0x7f54e00cdbf0 2024-01-27T02:37:55.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.016+0000 7f54dcff9700 1 --2- 172.21.15.7:0/1519444934 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f54c0066f30 0x7f54c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:55.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.016+0000 7f54dcff9700 1 -- 172.21.15.7:0/1519444934 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f54c8084030 con 0x7f54e00cdbf0 2024-01-27T02:37:55.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.016+0000 7f54e48d1700 1 --2- 172.21.15.7:0/1519444934 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f54c0066f30 0x7f54c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:55.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.017+0000 7f54e48d1700 1 --2- 172.21.15.7:0/1519444934 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f54c0066f30 0x7f54c00693e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f54d00091e0 tx=0x7f54d0017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:55.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.020+0000 7f54dcff9700 1 -- 172.21.15.7:0/1519444934 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f54e00b2510 con 0x7f54e00cdbf0 2024-01-27T02:37:55.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.271+0000 7f54e6b35700 1 -- 172.21.15.7:0/1519444934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54e00b24d0 con 0x7f54e00cdbf0 2024-01-27T02:37:55.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.271+0000 7f54dcff9700 1 -- 172.21.15.7:0/1519444934 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f54c8054e60 con 0x7f54e00cdbf0 2024-01-27T02:37:55.273 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:55.273 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2767759975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2767759975},{"type":"v1","addr":"172.21.15.188:6836","nonce":2767759975}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":5768,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/245473575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":245473575},{"type":"v1","addr":"172.21.15.7:6834","nonce":245473575}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1334164270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1334164270},{"type":"v1","addr":"172.21.15.188:6837","nonce":1334164270}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:37:51.231918+0000","modified":"2024-01-27T02:37:54.851112+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6657},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6657":{"gid":6657,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1251886476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1251886476},{"type":"v1","addr":"172.21.15.7:6835","nonce":1251886476}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2024-01-27T02:37:55.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.274+0000 7f54e6b35700 1 -- 172.21.15.7:0/1519444934 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f54c0066f30 msgr2=0x7f54c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:55.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.274+0000 7f54e6b35700 1 --2- 172.21.15.7:0/1519444934 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f54c0066f30 0x7f54c00693e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f54d00091e0 tx=0x7f54d0017000 comp rx=0 tx=0).stop 2024-01-27T02:37:55.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.274+0000 7f54e6b35700 1 -- 172.21.15.7:0/1519444934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54e00cdbf0 msgr2=0x7f54e01567d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:55.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.274+0000 7f54e6b35700 1 --2- 172.21.15.7:0/1519444934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54e00cdbf0 0x7f54e01567d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f54c800c790 tx=0x7f54c8007ea0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.274+0000 7f54e6b35700 1 -- 172.21.15.7:0/1519444934 shutdown_connections 2024-01-27T02:37:55.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.274+0000 7f54e6b35700 1 --2- 172.21.15.7:0/1519444934 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f54c0066f30 0x7f54c00693e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.274+0000 7f54e6b35700 1 --2- 172.21.15.7:0/1519444934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e0157ca0 0x7f54e00c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.274+0000 7f54e6b35700 1 --2- 172.21.15.7:0/1519444934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54e00c3ae0 0x7f54e01687f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.274+0000 7f54e6b35700 1 --2- 172.21.15.7:0/1519444934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54e00cdbf0 0x7f54e01567d0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.274+0000 7f54e6b35700 1 -- 172.21.15.7:0/1519444934 >> 172.21.15.7:0/1519444934 conn(0x7f54e0098020 msgr2=0x7f54e006bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:55.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.274+0000 7f54e6b35700 1 -- 172.21.15.7:0/1519444934 shutdown_connections 2024-01-27T02:37:55.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.274+0000 7f54e6b35700 1 -- 172.21.15.7:0/1519444934 wait complete. 2024-01-27T02:37:55.280 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 58 2024-01-27T02:37:55.289 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:37:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.419+0000 7f9a9426f700 1 Processor -- start 2024-01-27T02:37:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.419+0000 7f9a9426f700 1 -- start start 2024-01-27T02:37:55.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.420+0000 7f9a9426f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8c15a100 0x7f9a8c0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.420+0000 7f9a9426f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9a8c0cb5b0 0x7f9a8c0c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.420+0000 7f9a9426f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9a8c0c4af0 0x7f9a8c0c9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.420+0000 7f9a9426f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a8c0c4f70 con 0x7f9a8c0cb5b0 2024-01-27T02:37:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.420+0000 7f9a9426f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a8c0ce5e0 con 0x7f9a8c0c4af0 2024-01-27T02:37:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.420+0000 7f9a9426f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c0cd9d0 con 0x7f9a8c15a100 2024-01-27T02:37:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.420+0000 7f9a9180a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9a8c0cb5b0 0x7f9a8c0c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.420+0000 7f9a9180a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9a8c0cb5b0 0x7f9a8c0c45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44100/0 (socket says 172.21.15.7:44100) 2024-01-27T02:37:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.420+0000 7f9a9180a700 1 -- 172.21.15.7:0/1244542533 learned_addr learned my addr 172.21.15.7:0/1244542533 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.421+0000 7f9a9200b700 1 --2- 172.21.15.7:0/1244542533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8c15a100 0x7f9a8c0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.421+0000 7f9a9280c700 1 --2- 172.21.15.7:0/1244542533 >> v2:172.21.15.188:3300/0 conn(0x7f9a8c0c4af0 0x7f9a8c0c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.421+0000 7f9a9180a700 1 -- 172.21.15.7:0/1244542533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8c15a100 msgr2=0x7f9a8c0cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.421+0000 7f9a9180a700 1 --2- 172.21.15.7:0/1244542533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8c15a100 0x7f9a8c0cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.421+0000 7f9a9180a700 1 -- 172.21.15.7:0/1244542533 >> v2:172.21.15.188:3300/0 conn(0x7f9a8c0c4af0 msgr2=0x7f9a8c0c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.421+0000 7f9a9180a700 1 --2- 172.21.15.7:0/1244542533 >> v2:172.21.15.188:3300/0 conn(0x7f9a8c0c4af0 0x7f9a8c0c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.421+0000 7f9a9180a700 1 -- 172.21.15.7:0/1244542533 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8c0cdcb0 con 0x7f9a8c0cb5b0 2024-01-27T02:37:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.421+0000 7f9a9180a700 1 --2- 172.21.15.7:0/1244542533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c0cb5b0 0x7f9a8c0c45b0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c06e7a0 tx=0x7f9a7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bed45d1dd52ffc7 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.421+0000 7f9a91009700 1 -- 172.21.15.7:0/1244542533 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c00b860 con 0x7f9a8c0cb5b0 2024-01-27T02:37:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.421+0000 7f9a91009700 1 -- 172.21.15.7:0/1244542533 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7c00b9c0 con 0x7f9a8c0cb5b0 2024-01-27T02:37:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.421+0000 7f9a91009700 1 -- 172.21.15.7:0/1244542533 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c00bd60 con 0x7f9a8c0cb5b0 2024-01-27T02:37:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.422+0000 7f9a9426f700 1 -- 172.21.15.7:0/1244542533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c0cb5b0 msgr2=0x7f9a8c0c45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.422+0000 7f9a9426f700 1 --2- 172.21.15.7:0/1244542533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c0cb5b0 0x7f9a8c0c45b0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c06e7a0 tx=0x7f9a7c009250 comp rx=0 tx=0).stop 2024-01-27T02:37:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.422+0000 7f9a9426f700 1 -- 172.21.15.7:0/1244542533 shutdown_connections 2024-01-27T02:37:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.422+0000 7f9a9426f700 1 --2- 172.21.15.7:0/1244542533 >> v2:172.21.15.188:3300/0 conn(0x7f9a8c0c4af0 0x7f9a8c0c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.422+0000 7f9a9426f700 1 --2- 172.21.15.7:0/1244542533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8c15a100 0x7f9a8c0cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.422+0000 7f9a9426f700 1 --2- 172.21.15.7:0/1244542533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c0cb5b0 0x7f9a8c0c45b0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.422+0000 7f9a9426f700 1 -- 172.21.15.7:0/1244542533 >> 172.21.15.7:0/1244542533 conn(0x7f9a8c098020 msgr2=0x7f9a8c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.422+0000 7f9a9426f700 1 -- 172.21.15.7:0/1244542533 shutdown_connections 2024-01-27T02:37:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.422+0000 7f9a9426f700 1 -- 172.21.15.7:0/1244542533 wait complete. 2024-01-27T02:37:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.423+0000 7f9a9426f700 1 Processor -- start 2024-01-27T02:37:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.423+0000 7f9a9426f700 1 -- start start 2024-01-27T02:37:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.424+0000 7f9a9426f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a8c0c4af0 0x7f9a8c147b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.424+0000 7f9a9426f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c0cb5b0 0x7f9a8c0b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.424+0000 7f9a9426f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8c15a100 0x7f9a8c16d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.424+0000 7f9a9426f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c15d040 con 0x7f9a8c0cb5b0 2024-01-27T02:37:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.424+0000 7f9a9426f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c157200 con 0x7f9a8c0c4af0 2024-01-27T02:37:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.424+0000 7f9a9426f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c16e520 con 0x7f9a8c15a100 2024-01-27T02:37:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.424+0000 7f9a9180a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c0cb5b0 0x7f9a8c0b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.424+0000 7f9a9180a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c0cb5b0 0x7f9a8c0b1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44106/0 (socket says 172.21.15.7:44106) 2024-01-27T02:37:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.424+0000 7f9a9180a700 1 -- 172.21.15.7:0/2837318176 learned_addr learned my addr 172.21.15.7:0/2837318176 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.424+0000 7f9a9280c700 1 --2- 172.21.15.7:0/2837318176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8c15a100 0x7f9a8c16d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.424+0000 7f9a9200b700 1 --2- 172.21.15.7:0/2837318176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a8c0c4af0 0x7f9a8c147b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.424+0000 7f9a9180a700 1 -- 172.21.15.7:0/2837318176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8c15a100 msgr2=0x7f9a8c16d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.424+0000 7f9a9180a700 1 --2- 172.21.15.7:0/2837318176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8c15a100 0x7f9a8c16d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.424+0000 7f9a9180a700 1 -- 172.21.15.7:0/2837318176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a8c0c4af0 msgr2=0x7f9a8c147b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.424+0000 7f9a9180a700 1 --2- 172.21.15.7:0/2837318176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a8c0c4af0 0x7f9a8c147b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.424+0000 7f9a9180a700 1 -- 172.21.15.7:0/2837318176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7c0094f0 con 0x7f9a8c0cb5b0 2024-01-27T02:37:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.424+0000 7f9a9200b700 1 --2- 172.21.15.7:0/2837318176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a8c0c4af0 0x7f9a8c147b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.424+0000 7f9a9180a700 1 --2- 172.21.15.7:0/2837318176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c0cb5b0 0x7f9a8c0b1b00 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c06ea80 tx=0x7f9a7c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.425+0000 7f9a827fc700 1 -- 172.21.15.7:0/2837318176 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c024440 con 0x7f9a8c0cb5b0 2024-01-27T02:37:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.425+0000 7f9a827fc700 1 -- 172.21.15.7:0/2837318176 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7c0245a0 con 0x7f9a8c0cb5b0 2024-01-27T02:37:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.425+0000 7f9a9426f700 1 -- 172.21.15.7:0/2837318176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a8c15c9f0 con 0x7f9a8c0cb5b0 2024-01-27T02:37:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.425+0000 7f9a827fc700 1 -- 172.21.15.7:0/2837318176 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c024960 con 0x7f9a8c0cb5b0 2024-01-27T02:37:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.425+0000 7f9a9426f700 1 -- 172.21.15.7:0/2837318176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8c148060 con 0x7f9a8c0cb5b0 2024-01-27T02:37:55.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.426+0000 7f9a9426f700 1 -- 172.21.15.7:0/2837318176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a8c04c710 con 0x7f9a8c0cb5b0 2024-01-27T02:37:55.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.426+0000 7f9a827fc700 1 -- 172.21.15.7:0/2837318176 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9a7c01a420 con 0x7f9a8c0cb5b0 2024-01-27T02:37:55.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.427+0000 7f9a827fc700 1 --2- 172.21.15.7:0/2837318176 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9a70066f30 0x7f9a700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:55.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.427+0000 7f9a827fc700 1 -- 172.21.15.7:0/2837318176 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9a7c086290 con 0x7f9a8c0cb5b0 2024-01-27T02:37:55.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.427+0000 7f9a9200b700 1 --2- 172.21.15.7:0/2837318176 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9a70066f30 0x7f9a700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:55.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.427+0000 7f9a9200b700 1 --2- 172.21.15.7:0/2837318176 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9a70066f30 0x7f9a700693e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9a88006fd0 tx=0x7f9a88008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:55.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.432+0000 7f9a827fc700 1 -- 172.21.15.7:0/2837318176 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a7c053500 con 0x7f9a8c0cb5b0 2024-01-27T02:37:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.635+0000 7f9a9426f700 1 -- 172.21.15.7:0/2837318176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a8c0cdf90 con 0x7f9a8c0cb5b0 2024-01-27T02:37:55.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.636+0000 7f9a827fc700 1 -- 172.21.15.7:0/2837318176 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+15317 (secure 0 0 0) 0x7f9a7c056b20 con 0x7f9a8c0cb5b0 2024-01-27T02:37:55.637 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:55.637 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":117,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:37:52.811630+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:37:49.747694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-27T02:37:50.199870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:37:55.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.638+0000 7f9a9426f700 1 -- 172.21.15.7:0/2837318176 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9a70066f30 msgr2=0x7f9a700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:55.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.638+0000 7f9a9426f700 1 --2- 172.21.15.7:0/2837318176 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9a70066f30 0x7f9a700693e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9a88006fd0 tx=0x7f9a88008040 comp rx=0 tx=0).stop 2024-01-27T02:37:55.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.638+0000 7f9a9426f700 1 -- 172.21.15.7:0/2837318176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c0cb5b0 msgr2=0x7f9a8c0b1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:55.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.638+0000 7f9a9426f700 1 --2- 172.21.15.7:0/2837318176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c0cb5b0 0x7f9a8c0b1b00 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c06ea80 tx=0x7f9a7c017000 comp rx=0 tx=0).stop 2024-01-27T02:37:55.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.638+0000 7f9a9426f700 1 -- 172.21.15.7:0/2837318176 shutdown_connections 2024-01-27T02:37:55.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.638+0000 7f9a9426f700 1 --2- 172.21.15.7:0/2837318176 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9a70066f30 0x7f9a700693e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.638+0000 7f9a9426f700 1 --2- 172.21.15.7:0/2837318176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a8c15a100 0x7f9a8c16d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.638+0000 7f9a9426f700 1 --2- 172.21.15.7:0/2837318176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a8c0c4af0 0x7f9a8c147b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.638+0000 7f9a9426f700 1 --2- 172.21.15.7:0/2837318176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c0cb5b0 0x7f9a8c0b1b00 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.638+0000 7f9a9426f700 1 -- 172.21.15.7:0/2837318176 >> 172.21.15.7:0/2837318176 conn(0x7f9a8c098020 msgr2=0x7f9a8c158f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:55.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.638+0000 7f9a9426f700 1 -- 172.21.15.7:0/2837318176 shutdown_connections 2024-01-27T02:37:55.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.638+0000 7f9a9426f700 1 -- 172.21.15.7:0/2837318176 wait complete. 2024-01-27T02:37:55.653 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-27T02:37:55.654 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:37:55.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.785+0000 7fccf96f8700 1 Processor -- start 2024-01-27T02:37:55.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.785+0000 7fccf96f8700 1 -- start start 2024-01-27T02:37:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.785+0000 7fccf96f8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fccf4143860 0x7fccf40d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.785+0000 7fccf96f8700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fccf4143c30 0x7fccf40d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.785+0000 7fccf96f8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf40b3d90 con 0x7fccf415de00 2024-01-27T02:37:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.785+0000 7fccf96f8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccf40d0ea0 con 0x7fccf4143860 2024-01-27T02:37:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.785+0000 7fccf96f8700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccf40d1180 con 0x7fccf4143c30 2024-01-27T02:37:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.785+0000 7fccf2ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fccf4143860 0x7fccf40d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.785+0000 7fccf37fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fccf415de00 0x7fccf40cd190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50770/0 (socket says 172.21.15.7:50770) 2024-01-27T02:37:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.785+0000 7fccf37fe700 1 -- 172.21.15.7:0/3401722627 learned_addr learned my addr 172.21.15.7:0/3401722627 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.786+0000 7fccf27fc700 1 --2- 172.21.15.7:0/3401722627 >> v2:172.21.15.188:3300/0 conn(0x7fccf4143c30 0x7fccf40d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.786+0000 7fccf2ffd700 1 -- 172.21.15.7:0/3401722627 >> v2:172.21.15.188:3300/0 conn(0x7fccf4143c30 msgr2=0x7fccf40d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.786+0000 7fccf2ffd700 1 --2- 172.21.15.7:0/3401722627 >> v2:172.21.15.188:3300/0 conn(0x7fccf4143c30 0x7fccf40d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.786+0000 7fccf2ffd700 1 -- 172.21.15.7:0/3401722627 >> v1:172.21.15.7:6789/0 conn(0x7fccf415de00 legacy=0x7fccf40cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.786+0000 7fccf2ffd700 1 -- 172.21.15.7:0/3401722627 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf40d1460 con 0x7fccf4143860 2024-01-27T02:37:55.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.786+0000 7fccf1ffb700 1 -- 172.21.15.7:0/3401722627 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1041804047 0 0) 0x7fccf40b3d90 con 0x7fccf415de00 2024-01-27T02:37:55.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.786+0000 7fccf2ffd700 1 --2- 172.21.15.7:0/3401722627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccf4143860 0x7fccf40d02e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fccf40d1a50 tx=0x7fccdc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64658f2b7da0544d server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:55.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.786+0000 7fccf1ffb700 1 -- 172.21.15.7:0/3401722627 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc011630 con 0x7fccf4143860 2024-01-27T02:37:55.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.786+0000 7fccf1ffb700 1 -- 172.21.15.7:0/3401722627 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccdc011790 con 0x7fccf4143860 2024-01-27T02:37:55.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.786+0000 7fccf1ffb700 1 -- 172.21.15.7:0/3401722627 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc011b30 con 0x7fccf4143860 2024-01-27T02:37:55.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.787+0000 7fccf96f8700 1 -- 172.21.15.7:0/3401722627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccf4143860 msgr2=0x7fccf40d02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:55.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.787+0000 7fccf96f8700 1 --2- 172.21.15.7:0/3401722627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccf4143860 0x7fccf40d02e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fccf40d1a50 tx=0x7fccdc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.787+0000 7fccf96f8700 1 -- 172.21.15.7:0/3401722627 shutdown_connections 2024-01-27T02:37:55.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.787+0000 7fccf96f8700 1 --2- 172.21.15.7:0/3401722627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccf4143860 0x7fccf40d02e0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.787+0000 7fccf96f8700 1 --2- 172.21.15.7:0/3401722627 >> v2:172.21.15.188:3300/0 conn(0x7fccf4143c30 0x7fccf40d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.787+0000 7fccf96f8700 1 -- 172.21.15.7:0/3401722627 >> 172.21.15.7:0/3401722627 conn(0x7fccf4068440 msgr2=0x7fccf40709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:55.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.787+0000 7fccf96f8700 1 -- 172.21.15.7:0/3401722627 shutdown_connections 2024-01-27T02:37:55.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.787+0000 7fccf96f8700 1 -- 172.21.15.7:0/3401722627 wait complete. 2024-01-27T02:37:55.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.788+0000 7fccf96f8700 1 Processor -- start 2024-01-27T02:37:55.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.788+0000 7fccf96f8700 1 -- start start 2024-01-27T02:37:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.789+0000 7fccf96f8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccf4143860 0x7fccf416e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.789+0000 7fccf96f8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccf4143c30 0x7fccf40ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.789+0000 7fccf96f8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccf415de00 0x7fccf40af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.789+0000 7fccf96f8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf40acc10 con 0x7fccf4143c30 2024-01-27T02:37:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.789+0000 7fccf96f8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf40aa3e0 con 0x7fccf4143860 2024-01-27T02:37:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.789+0000 7fccf96f8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf40ace40 con 0x7fccf415de00 2024-01-27T02:37:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.789+0000 7fccf27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccf4143c30 0x7fccf40ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.789+0000 7fccf27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccf4143c30 0x7fccf40ac6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44126/0 (socket says 172.21.15.7:44126) 2024-01-27T02:37:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.789+0000 7fccf27fc700 1 -- 172.21.15.7:0/554888311 learned_addr learned my addr 172.21.15.7:0/554888311 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.789+0000 7fccf2ffd700 1 --2- 172.21.15.7:0/554888311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccf4143860 0x7fccf416e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.789+0000 7fccf37fe700 1 --2- 172.21.15.7:0/554888311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccf415de00 0x7fccf40af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.790+0000 7fccf27fc700 1 -- 172.21.15.7:0/554888311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccf415de00 msgr2=0x7fccf40af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.790+0000 7fccf27fc700 1 --2- 172.21.15.7:0/554888311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccf415de00 0x7fccf40af020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.790+0000 7fccf27fc700 1 -- 172.21.15.7:0/554888311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccf4143860 msgr2=0x7fccf416e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.790+0000 7fccf27fc700 1 --2- 172.21.15.7:0/554888311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccf4143860 0x7fccf416e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.790+0000 7fccf27fc700 1 -- 172.21.15.7:0/554888311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccdc009420 con 0x7fccf4143c30 2024-01-27T02:37:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.790+0000 7fccf37fe700 1 --2- 172.21.15.7:0/554888311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccf415de00 0x7fccf40af020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:37:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.790+0000 7fccf27fc700 1 --2- 172.21.15.7:0/554888311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccf4143c30 0x7fccf40ac6d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fcce400ec50 tx=0x7fcce40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.790+0000 7fcceaffd700 1 -- 172.21.15.7:0/554888311 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce400c850 con 0x7fccf4143c30 2024-01-27T02:37:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.790+0000 7fcceaffd700 1 -- 172.21.15.7:0/554888311 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce400c9b0 con 0x7fccf4143c30 2024-01-27T02:37:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.790+0000 7fccf96f8700 1 -- 172.21.15.7:0/554888311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf40ad120 con 0x7fccf4143c30 2024-01-27T02:37:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.790+0000 7fcceaffd700 1 -- 172.21.15.7:0/554888311 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce400cd70 con 0x7fccf4143c30 2024-01-27T02:37:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.790+0000 7fccf96f8700 1 -- 172.21.15.7:0/554888311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf40ad6e0 con 0x7fccf4143c30 2024-01-27T02:37:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.791+0000 7fccf96f8700 1 -- 172.21.15.7:0/554888311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf404c710 con 0x7fccf4143c30 2024-01-27T02:37:55.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.791+0000 7fcceaffd700 1 -- 172.21.15.7:0/554888311 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcce4013680 con 0x7fccf4143c30 2024-01-27T02:37:55.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.792+0000 7fcceaffd700 1 --2- 172.21.15.7:0/554888311 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fccd4066f60 0x7fccd4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:55.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.792+0000 7fcceaffd700 1 -- 172.21.15.7:0/554888311 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcce40847c0 con 0x7fccf4143c30 2024-01-27T02:37:55.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.792+0000 7fccf2ffd700 1 --2- 172.21.15.7:0/554888311 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fccd4066f60 0x7fccd4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:55.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.793+0000 7fccf2ffd700 1 --2- 172.21.15.7:0/554888311 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fccd4066f60 0x7fccd4069410 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0093f0 tx=0x7fccdc017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:55.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.796+0000 7fcceaffd700 1 -- 172.21.15.7:0/554888311 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcce4051a30 con 0x7fccf4143c30 2024-01-27T02:37:55.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:55.996+0000 7fccf96f8700 1 -- 172.21.15.7:0/554888311 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fccf40d1740 con 0x7fccd4066f60 2024-01-27T02:37:56.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.005+0000 7fcceaffd700 1 -- 172.21.15.7:0/554888311 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282556 (secure 0 0 0) 0x7fccf40d1740 con 0x7fccd4066f60 2024-01-27T02:37:56.010 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:56.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.008+0000 7fccf96f8700 1 -- 172.21.15.7:0/554888311 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fccd4066f60 msgr2=0x7fccd4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:56.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.008+0000 7fccf96f8700 1 --2- 172.21.15.7:0/554888311 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fccd4066f60 0x7fccd4069410 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0093f0 tx=0x7fccdc017000 comp rx=0 tx=0).stop 2024-01-27T02:37:56.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.008+0000 7fccf96f8700 1 -- 172.21.15.7:0/554888311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccf4143c30 msgr2=0x7fccf40ac6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:56.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.008+0000 7fccf96f8700 1 --2- 172.21.15.7:0/554888311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccf4143c30 0x7fccf40ac6d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fcce400ec50 tx=0x7fcce40091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.008+0000 7fccf96f8700 1 -- 172.21.15.7:0/554888311 shutdown_connections 2024-01-27T02:37:56.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.008+0000 7fccf96f8700 1 --2- 172.21.15.7:0/554888311 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fccd4066f60 0x7fccd4069410 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.008+0000 7fccf96f8700 1 --2- 172.21.15.7:0/554888311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccf415de00 0x7fccf40af020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.008+0000 7fccf96f8700 1 --2- 172.21.15.7:0/554888311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccf4143860 0x7fccf416e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.008+0000 7fccf96f8700 1 --2- 172.21.15.7:0/554888311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccf4143c30 0x7fccf40ac6d0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.008+0000 7fccf96f8700 1 -- 172.21.15.7:0/554888311 >> 172.21.15.7:0/554888311 conn(0x7fccf4068440 msgr2=0x7fccf415c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:56.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.008+0000 7fccf96f8700 1 -- 172.21.15.7:0/554888311 shutdown_connections 2024-01-27T02:37:56.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.008+0000 7fccf96f8700 1 -- 172.21.15.7:0/554888311 wait complete. 2024-01-27T02:37:56.013 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:37:56.023 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-01-27T02:37:55.658393+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6025,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556474,"statfs":{"total":1600000000,"available":1593829967,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.866207"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800705+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:29:06.789618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.811901+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:57.231078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800547+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:41:58.817551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.799051+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:17:03.370160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.808124+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:14:51.191272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801275+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:45:13.633104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800588+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:14:51.191272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.811924+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:04:59.885669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.805962+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:55.343516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.807734+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:55.343516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801974+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:28:34.028232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798839+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:26.529268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801629+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:55:18.331130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.809936+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:29:56.743131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798550+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:04:02.332379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801655+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:25:00.247752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.811458+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:46:44.234857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798522+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:14:51.191272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.809517+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:07:53.995321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.799143+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:12:54.826584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.806764+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:19:48.163385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.811035+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:33:08.282783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.811487+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:10:19.964021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800572+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:55.343516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.807163+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:33:14.691144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.807469+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:17:03.370160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800915+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:17:03.370160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.807224+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:24:13.096584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.808167+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:45:13.633104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.812972+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:50:16.532574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798553+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:55.343516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801905+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:23:06.453385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.809151+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:45:52.881907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.812131+0000","last_change":"2024-01-27T02:37:50.812131+0000","last_active":"2024-01-27T02:37:50.812131+0000","last_peered":"2024-01-27T02:37:50.812131+0000","last_clean":"2024-01-27T02:37:50.812131+0000","last_became_active":"2024-01-27T02:37:50.809514+0000","last_became_peered":"2024-01-27T02:37:50.809514+0000","last_unstale":"2024-01-27T02:37:50.812131+0000","last_undegraded":"2024-01-27T02:37:50.812131+0000","last_fullsized":"2024-01-27T02:37:50.812131+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:14:50.186256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.797941+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:24:13.096584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.808649+0000","last_change":"2024-01-27T02:37:50.808649+0000","last_active":"2024-01-27T02:37:50.808649+0000","last_peered":"2024-01-27T02:37:50.808649+0000","last_clean":"2024-01-27T02:37:50.808649+0000","last_became_active":"2024-01-27T02:37:50.808412+0000","last_became_peered":"2024-01-27T02:37:50.808412+0000","last_unstale":"2024-01-27T02:37:50.808649+0000","last_undegraded":"2024-01-27T02:37:50.808649+0000","last_fullsized":"2024-01-27T02:37:50.808649+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:24:12.091569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800223+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:10:27.309356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.806041+0000","last_change":"2024-01-27T02:37:50.806041+0000","last_active":"2024-01-27T02:37:50.806041+0000","last_peered":"2024-01-27T02:37:50.806041+0000","last_clean":"2024-01-27T02:37:50.806041+0000","last_became_active":"2024-01-27T02:37:50.805760+0000","last_became_peered":"2024-01-27T02:37:50.805760+0000","last_unstale":"2024-01-27T02:37:50.806041+0000","last_undegraded":"2024-01-27T02:37:50.806041+0000","last_fullsized":"2024-01-27T02:37:50.806041+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:31:21.182217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798399+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:57:57.067686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.808048+0000","last_change":"2024-01-27T02:37:50.808048+0000","last_active":"2024-01-27T02:37:50.808048+0000","last_peered":"2024-01-27T02:37:50.808048+0000","last_clean":"2024-01-27T02:37:50.808048+0000","last_became_active":"2024-01-27T02:37:50.807771+0000","last_became_peered":"2024-01-27T02:37:50.807771+0000","last_unstale":"2024-01-27T02:37:50.808048+0000","last_undegraded":"2024-01-27T02:37:50.808048+0000","last_fullsized":"2024-01-27T02:37:50.808048+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:24.808914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800346+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:26.529268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.806436+0000","last_change":"2024-01-27T02:37:50.806436+0000","last_active":"2024-01-27T02:37:50.806436+0000","last_peered":"2024-01-27T02:37:50.806436+0000","last_clean":"2024-01-27T02:37:50.806436+0000","last_became_active":"2024-01-27T02:37:50.806167+0000","last_became_peered":"2024-01-27T02:37:50.806167+0000","last_unstale":"2024-01-27T02:37:50.806436+0000","last_undegraded":"2024-01-27T02:37:50.806436+0000","last_fullsized":"2024-01-27T02:37:50.806436+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:58:12.533491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800174+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:25.813930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.812296+0000","last_change":"2024-01-27T02:37:50.812296+0000","last_active":"2024-01-27T02:37:50.812296+0000","last_peered":"2024-01-27T02:37:50.812296+0000","last_clean":"2024-01-27T02:37:50.812296+0000","last_became_active":"2024-01-27T02:37:50.809825+0000","last_became_peered":"2024-01-27T02:37:50.809825+0000","last_unstale":"2024-01-27T02:37:50.812296+0000","last_undegraded":"2024-01-27T02:37:50.812296+0000","last_fullsized":"2024-01-27T02:37:50.812296+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:54.338500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798046+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:33:14.691144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.804462+0000","last_change":"2024-01-27T02:37:50.804462+0000","last_active":"2024-01-27T02:37:50.804462+0000","last_peered":"2024-01-27T02:37:50.804462+0000","last_clean":"2024-01-27T02:37:50.804462+0000","last_became_active":"2024-01-27T02:37:50.804211+0000","last_became_peered":"2024-01-27T02:37:50.804211+0000","last_unstale":"2024-01-27T02:37:50.804462+0000","last_undegraded":"2024-01-27T02:37:50.804462+0000","last_fullsized":"2024-01-27T02:37:50.804462+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:16:34.794337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798801+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:16:35.799353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.804294+0000","last_change":"2024-01-27T02:37:50.804294+0000","last_active":"2024-01-27T02:37:50.804294+0000","last_peered":"2024-01-27T02:37:50.804294+0000","last_clean":"2024-01-27T02:37:50.804294+0000","last_became_active":"2024-01-27T02:37:50.803789+0000","last_became_peered":"2024-01-27T02:37:50.803789+0000","last_unstale":"2024-01-27T02:37:50.804294+0000","last_undegraded":"2024-01-27T02:37:50.804294+0000","last_fullsized":"2024-01-27T02:37:50.804294+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:37:47.399101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.808533+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:55:18.331130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.807055+0000","last_change":"2024-01-27T02:37:50.807055+0000","last_active":"2024-01-27T02:37:50.807055+0000","last_peered":"2024-01-27T02:37:50.807055+0000","last_clean":"2024-01-27T02:37:50.807055+0000","last_became_active":"2024-01-27T02:37:50.806246+0000","last_became_peered":"2024-01-27T02:37:50.806246+0000","last_unstale":"2024-01-27T02:37:50.807055+0000","last_undegraded":"2024-01-27T02:37:50.807055+0000","last_fullsized":"2024-01-27T02:37:50.807055+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:23.160028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798337+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:54:58.004832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.805856+0000","last_change":"2024-01-27T02:37:50.805856+0000","last_active":"2024-01-27T02:37:50.805856+0000","last_peered":"2024-01-27T02:37:50.805856+0000","last_clean":"2024-01-27T02:37:50.805856+0000","last_became_active":"2024-01-27T02:37:50.805568+0000","last_became_peered":"2024-01-27T02:37:50.805568+0000","last_unstale":"2024-01-27T02:37:50.805856+0000","last_undegraded":"2024-01-27T02:37:50.805856+0000","last_fullsized":"2024-01-27T02:37:50.805856+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:11:46.349861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.808509+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:45:13.633104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.809623+0000","last_change":"2024-01-27T02:37:50.809623+0000","last_active":"2024-01-27T02:37:50.809623+0000","last_peered":"2024-01-27T02:37:50.809623+0000","last_clean":"2024-01-27T02:37:50.809623+0000","last_became_active":"2024-01-27T02:37:50.809447+0000","last_became_peered":"2024-01-27T02:37:50.809447+0000","last_unstale":"2024-01-27T02:37:50.809623+0000","last_undegraded":"2024-01-27T02:37:50.809623+0000","last_fullsized":"2024-01-27T02:37:50.809623+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:50.938489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801315+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:55:18.331130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.814137+0000","last_change":"2024-01-27T02:37:50.814137+0000","last_active":"2024-01-27T02:37:50.814137+0000","last_peered":"2024-01-27T02:37:50.814137+0000","last_clean":"2024-01-27T02:37:50.814137+0000","last_became_active":"2024-01-27T02:37:50.813898+0000","last_became_peered":"2024-01-27T02:37:50.813898+0000","last_unstale":"2024-01-27T02:37:50.814137+0000","last_undegraded":"2024-01-27T02:37:50.814137+0000","last_fullsized":"2024-01-27T02:37:50.814137+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:56:37.175345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798050+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:25.813930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.813470+0000","last_change":"2024-01-27T02:37:50.813470+0000","last_active":"2024-01-27T02:37:50.813470+0000","last_peered":"2024-01-27T02:37:50.813470+0000","last_clean":"2024-01-27T02:37:50.813470+0000","last_became_active":"2024-01-27T02:37:50.812014+0000","last_became_peered":"2024-01-27T02:37:50.812014+0000","last_unstale":"2024-01-27T02:37:50.813470+0000","last_undegraded":"2024-01-27T02:37:50.813470+0000","last_fullsized":"2024-01-27T02:37:50.813470+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:56:29.266154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801084+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:53:55.703491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.807923+0000","last_change":"2024-01-27T02:37:50.807923+0000","last_active":"2024-01-27T02:37:50.807923+0000","last_peered":"2024-01-27T02:37:50.807923+0000","last_clean":"2024-01-27T02:37:50.807923+0000","last_became_active":"2024-01-27T02:37:50.807417+0000","last_became_peered":"2024-01-27T02:37:50.807417+0000","last_unstale":"2024-01-27T02:37:50.807923+0000","last_undegraded":"2024-01-27T02:37:50.807923+0000","last_fullsized":"2024-01-27T02:37:50.807923+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:26:35.727576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.797951+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:01:25.391647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.809421+0000","last_change":"2024-01-27T02:37:50.809421+0000","last_active":"2024-01-27T02:37:50.809421+0000","last_peered":"2024-01-27T02:37:50.809421+0000","last_clean":"2024-01-27T02:37:50.809421+0000","last_became_active":"2024-01-27T02:37:50.809169+0000","last_became_peered":"2024-01-27T02:37:50.809169+0000","last_unstale":"2024-01-27T02:37:50.809421+0000","last_undegraded":"2024-01-27T02:37:50.809421+0000","last_fullsized":"2024-01-27T02:37:50.809421+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:31:21.182217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.806904+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:46:21.029594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.810368+0000","last_change":"2024-01-27T02:37:50.810368+0000","last_active":"2024-01-27T02:37:50.810368+0000","last_peered":"2024-01-27T02:37:50.810368+0000","last_clean":"2024-01-27T02:37:50.810368+0000","last_became_active":"2024-01-27T02:37:50.810089+0000","last_became_peered":"2024-01-27T02:37:50.810089+0000","last_unstale":"2024-01-27T02:37:50.810368+0000","last_undegraded":"2024-01-27T02:37:50.810368+0000","last_fullsized":"2024-01-27T02:37:50.810368+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:50:15.527559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.807362+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:25.813930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.803536+0000","last_change":"2024-01-27T02:37:50.803536+0000","last_active":"2024-01-27T02:37:50.803536+0000","last_peered":"2024-01-27T02:37:50.803536+0000","last_clean":"2024-01-27T02:37:50.803536+0000","last_became_active":"2024-01-27T02:37:50.803332+0000","last_became_peered":"2024-01-27T02:37:50.803332+0000","last_unstale":"2024-01-27T02:37:50.803536+0000","last_undegraded":"2024-01-27T02:37:50.803536+0000","last_fullsized":"2024-01-27T02:37:50.803536+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:05:27.161024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.799402+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:57:57.067686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.802000+0000","last_change":"2024-01-27T02:37:50.802000+0000","last_active":"2024-01-27T02:37:50.802000+0000","last_peered":"2024-01-27T02:37:50.802000+0000","last_clean":"2024-01-27T02:37:50.802000+0000","last_became_active":"2024-01-27T02:37:50.801755+0000","last_became_peered":"2024-01-27T02:37:50.801755+0000","last_unstale":"2024-01-27T02:37:50.802000+0000","last_undegraded":"2024-01-27T02:37:50.802000+0000","last_fullsized":"2024-01-27T02:37:50.802000+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:54:56.999817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800829+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:04:02.332379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.812532+0000","last_change":"2024-01-27T02:37:50.812532+0000","last_active":"2024-01-27T02:37:50.812532+0000","last_peered":"2024-01-27T02:37:50.812532+0000","last_clean":"2024-01-27T02:37:50.812532+0000","last_became_active":"2024-01-27T02:37:50.812209+0000","last_became_peered":"2024-01-27T02:37:50.812209+0000","last_unstale":"2024-01-27T02:37:50.812532+0000","last_undegraded":"2024-01-27T02:37:50.812532+0000","last_fullsized":"2024-01-27T02:37:50.812532+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:11:46.349861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.808739+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:47:12.515208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.813813+0000","last_change":"2024-01-27T02:37:50.813813+0000","last_active":"2024-01-27T02:37:50.813813+0000","last_peered":"2024-01-27T02:37:50.813813+0000","last_clean":"2024-01-27T02:37:50.813813+0000","last_became_active":"2024-01-27T02:37:50.813235+0000","last_became_peered":"2024-01-27T02:37:50.813235+0000","last_unstale":"2024-01-27T02:37:50.813813+0000","last_undegraded":"2024-01-27T02:37:50.813813+0000","last_fullsized":"2024-01-27T02:37:50.813813+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:33:13.686128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.807809+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:14:52.244572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.804058+0000","last_change":"2024-01-27T02:37:50.804058+0000","last_active":"2024-01-27T02:37:50.804058+0000","last_peered":"2024-01-27T02:37:50.804058+0000","last_clean":"2024-01-27T02:37:50.804058+0000","last_became_active":"2024-01-27T02:37:50.803802+0000","last_became_peered":"2024-01-27T02:37:50.803802+0000","last_unstale":"2024-01-27T02:37:50.804058+0000","last_undegraded":"2024-01-27T02:37:50.804058+0000","last_fullsized":"2024-01-27T02:37:50.804058+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:01:24.386631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.806271+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:19:50.489258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.806415+0000","last_change":"2024-01-27T02:37:50.806415+0000","last_active":"2024-01-27T02:37:50.806415+0000","last_peered":"2024-01-27T02:37:50.806415+0000","last_clean":"2024-01-27T02:37:50.806415+0000","last_became_active":"2024-01-27T02:37:50.806149+0000","last_became_peered":"2024-01-27T02:37:50.806149+0000","last_unstale":"2024-01-27T02:37:50.806415+0000","last_undegraded":"2024-01-27T02:37:50.806415+0000","last_fullsized":"2024-01-27T02:37:50.806415+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:04:39.266345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.806325+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:28.200720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.809566+0000","last_change":"2024-01-27T02:37:50.809566+0000","last_active":"2024-01-27T02:37:50.809566+0000","last_peered":"2024-01-27T02:37:50.809566+0000","last_clean":"2024-01-27T02:37:50.809566+0000","last_became_active":"2024-01-27T02:37:50.809276+0000","last_became_peered":"2024-01-27T02:37:50.809276+0000","last_unstale":"2024-01-27T02:37:50.809566+0000","last_undegraded":"2024-01-27T02:37:50.809566+0000","last_fullsized":"2024-01-27T02:37:50.809566+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:24:12.091569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.811076+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:18:23.945261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.805793+0000","last_change":"2024-01-27T02:37:50.805793+0000","last_active":"2024-01-27T02:37:50.805793+0000","last_peered":"2024-01-27T02:37:50.805793+0000","last_clean":"2024-01-27T02:37:50.805793+0000","last_became_active":"2024-01-27T02:37:50.805524+0000","last_became_peered":"2024-01-27T02:37:50.805524+0000","last_unstale":"2024-01-27T02:37:50.805793+0000","last_undegraded":"2024-01-27T02:37:50.805793+0000","last_fullsized":"2024-01-27T02:37:50.805793+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:57:33.113268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801404+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:17:37.405535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.814412+0000","last_change":"2024-01-27T02:37:50.814412+0000","last_active":"2024-01-27T02:37:50.814412+0000","last_peered":"2024-01-27T02:37:50.814412+0000","last_clean":"2024-01-27T02:37:50.814412+0000","last_became_active":"2024-01-27T02:37:50.813560+0000","last_became_peered":"2024-01-27T02:37:50.813560+0000","last_unstale":"2024-01-27T02:37:50.814412+0000","last_undegraded":"2024-01-27T02:37:50.814412+0000","last_fullsized":"2024-01-27T02:37:50.814412+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:27.195705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800617+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:08:13.232740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.808731+0000","last_change":"2024-01-27T02:37:50.808731+0000","last_active":"2024-01-27T02:37:50.808731+0000","last_peered":"2024-01-27T02:37:50.808731+0000","last_clean":"2024-01-27T02:37:50.808731+0000","last_became_active":"2024-01-27T02:37:50.808482+0000","last_became_peered":"2024-01-27T02:37:50.808482+0000","last_unstale":"2024-01-27T02:37:50.808731+0000","last_undegraded":"2024-01-27T02:37:50.808731+0000","last_fullsized":"2024-01-27T02:37:50.808731+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:50.938489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.806197+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:11:47.354877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.808670+0000","last_change":"2024-01-27T02:37:50.808670+0000","last_active":"2024-01-27T02:37:50.808670+0000","last_peered":"2024-01-27T02:37:50.808670+0000","last_clean":"2024-01-27T02:37:50.808670+0000","last_became_active":"2024-01-27T02:37:50.808428+0000","last_became_peered":"2024-01-27T02:37:50.808428+0000","last_unstale":"2024-01-27T02:37:50.808670+0000","last_undegraded":"2024-01-27T02:37:50.808670+0000","last_fullsized":"2024-01-27T02:37:50.808670+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:57:56.062671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.806456+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:07:01.540761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.813250+0000","last_change":"2024-01-27T02:37:50.813250+0000","last_active":"2024-01-27T02:37:50.813250+0000","last_peered":"2024-01-27T02:37:50.813250+0000","last_clean":"2024-01-27T02:37:50.813250+0000","last_became_active":"2024-01-27T02:37:50.813038+0000","last_became_peered":"2024-01-27T02:37:50.813038+0000","last_unstale":"2024-01-27T02:37:50.813250+0000","last_undegraded":"2024-01-27T02:37:50.813250+0000","last_fullsized":"2024-01-27T02:37:50.813250+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:25.524253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.797951+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:57:34.118284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.803902+0000","last_change":"2024-01-27T02:37:50.803902+0000","last_active":"2024-01-27T02:37:50.803902+0000","last_peered":"2024-01-27T02:37:50.803902+0000","last_clean":"2024-01-27T02:37:50.803902+0000","last_became_active":"2024-01-27T02:37:50.803626+0000","last_became_peered":"2024-01-27T02:37:50.803626+0000","last_unstale":"2024-01-27T02:37:50.803902+0000","last_undegraded":"2024-01-27T02:37:50.803902+0000","last_fullsized":"2024-01-27T02:37:50.803902+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:25.524253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.808334+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:33:48.901396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.804116+0000","last_change":"2024-01-27T02:37:50.804116+0000","last_active":"2024-01-27T02:37:50.804116+0000","last_peered":"2024-01-27T02:37:50.804116+0000","last_clean":"2024-01-27T02:37:50.804116+0000","last_became_active":"2024-01-27T02:37:50.803880+0000","last_became_peered":"2024-01-27T02:37:50.803880+0000","last_unstale":"2024-01-27T02:37:50.804116+0000","last_undegraded":"2024-01-27T02:37:50.804116+0000","last_fullsized":"2024-01-27T02:37:50.804116+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:50.938489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.806238+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:51.943504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.807085+0000","last_change":"2024-01-27T02:37:50.807085+0000","last_active":"2024-01-27T02:37:50.807085+0000","last_peered":"2024-01-27T02:37:50.807085+0000","last_clean":"2024-01-27T02:37:50.807085+0000","last_became_active":"2024-01-27T02:37:50.804683+0000","last_became_peered":"2024-01-27T02:37:50.804683+0000","last_unstale":"2024-01-27T02:37:50.807085+0000","last_undegraded":"2024-01-27T02:37:50.807085+0000","last_fullsized":"2024-01-27T02:37:50.807085+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:49:46.671831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.812579+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:11:16.205143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.809913+0000","last_change":"2024-01-27T02:37:50.809913+0000","last_active":"2024-01-27T02:37:50.809913+0000","last_peered":"2024-01-27T02:37:50.809913+0000","last_clean":"2024-01-27T02:37:50.809913+0000","last_became_active":"2024-01-27T02:37:50.809679+0000","last_became_peered":"2024-01-27T02:37:50.809679+0000","last_unstale":"2024-01-27T02:37:50.809913+0000","last_undegraded":"2024-01-27T02:37:50.809913+0000","last_fullsized":"2024-01-27T02:37:50.809913+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:27.195705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":245,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-27T02:37:49.795519+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:37:49.795519+0000","last_peered":"2024-01-27T02:37:49.795519+0000","last_clean":"2024-01-27T02:37:49.795519+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:37:49.795519+0000","last_undegraded":"2024-01-27T02:37:49.795519+0000","last_fullsized":"2024-01-27T02:37:49.795519+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574921,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1186,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194126,"statfs":{"total":200000000,"available":198785187,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71299999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.59099999999999997,"15min":0.58999999999999997},"min":{"1min":0.49099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77400000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.64000000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59699999999999998,"15min":0.59799999999999998},"min":{"1min":0.46100000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.44400000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.753,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.53600000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58299999999999996},"min":{"1min":0.432,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.439},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77700000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.65500000000000003}]},{"osd":3,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60499999999999998,"15min":0.59299999999999997},"min":{"1min":0.495,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.60999999999999999,"15min":0.59499999999999997},"min":{"1min":0.5,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85099999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.71799999999999997}]},{"osd":4,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.47499999999999998,"15min":0.48699999999999999},"min":{"1min":0.32400000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63300000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.495,"15min":0.51300000000000001},"min":{"1min":0.35999999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68899999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.80000000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51800000000000002,"15min":0.53600000000000003},"min":{"1min":0.39000000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.56399999999999995},"min":{"1min":0.40100000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.75800000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.59499999999999997},"min":{"1min":0.40999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.751,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.57699999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.81599999999999995,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.73099999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":603,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194708,"statfs":{"total":200000000,"available":199381818,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58699999999999997,"15min":0.52100000000000002},"min":{"1min":0.45000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.52700000000000002},"min":{"1min":0.434,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71899999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.79900000000000004}]},{"osd":1,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.51800000000000002},"min":{"1min":0.41999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79700000000000004,"5min":0.874,"15min":0.874},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.60599999999999998,"15min":0.55500000000000005},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.54000000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.41199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.625,"15min":0.58299999999999996},"min":{"1min":0.45100000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79700000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59399999999999997}]},{"osd":3,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58199999999999996,"15min":0.55100000000000005},"min":{"1min":0.47599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.76800000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61499999999999999,"15min":0.57299999999999995},"min":{"1min":0.48499999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.78100000000000003,"5min":1.262,"15min":1.262},"last":0.57099999999999995}]},{"osd":4,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.34200000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52700000000000002,"15min":0.55300000000000005},"min":{"1min":0.41199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68300000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.72699999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53200000000000003,"15min":0.53500000000000003},"min":{"1min":0.38800000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70799999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.755},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.60599999999999998},"min":{"1min":0.40899999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70799999999999996,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.63300000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60599999999999998,"15min":0.61599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73699999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71099999999999997,"5min":1.038,"15min":1.038},"last":0.60999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":609,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194702,"statfs":{"total":200000000,"available":199375555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39700000000000002,"15min":0.377},"min":{"1min":0.27000000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.41999999999999998,"15min":0.40000000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.374}]},{"osd":2,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.40200000000000002,"15min":0.36599999999999999},"min":{"1min":0.28000000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45400000000000001,"15min":0.43099999999999999},"min":{"1min":0.30199999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.39500000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48999999999999999,"15min":0.46899999999999997},"min":{"1min":0.35799999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.434},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.47499999999999998,"15min":0.45800000000000002},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.314}]},{"osd":4,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.66200000000000003,"15min":0.63400000000000001},"min":{"1min":0.442,"5min":0.373,"15min":0.373},"max":{"1min":3.2890000000000001,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65800000000000003,"15min":0.63600000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.59299999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.68500000000000005,"15min":0.63700000000000001},"min":{"1min":0.50600000000000001,"5min":0.44,"15min":0.44},"max":{"1min":3.4500000000000002,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.68999999999999995,"15min":0.67600000000000005},"min":{"1min":0.48299999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.181,"5min":3.181,"15min":3.181},"last":0.748}]},{"osd":6,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70999999999999996,"15min":0.69599999999999995},"min":{"1min":0.52100000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96399999999999997,"5min":1.024,"15min":1.024},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.71299999999999997,"15min":0.67900000000000005},"min":{"1min":0.53400000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.79900000000000004}]},{"osd":7,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.72399999999999998,"15min":0.68600000000000005},"min":{"1min":0.56100000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.72499999999999998,"15min":0.69199999999999995},"min":{"1min":0.57799999999999996,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.70599999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574921,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1186,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194126,"statfs":{"total":200000000,"available":198785187,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.38900000000000001},"min":{"1min":0.28100000000000003,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.66600000000000004,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27800000000000002,"5min":0.223,"15min":0.223},"max":{"1min":0.65200000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.39400000000000002}]},{"osd":2,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.436,"15min":0.42199999999999999},"min":{"1min":0.32800000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56299999999999994,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.441},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.44900000000000001,"15min":0.435},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58999999999999997,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.628}]},{"osd":3,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42999999999999999,"15min":0.432},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.63600000000000001,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.35299999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.63200000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.36299999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.65500000000000003,"15min":0.64000000000000001},"min":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.501},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45900000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89700000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.59799999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67500000000000004,"15min":0.65600000000000003},"min":{"1min":0.48399999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.86499999999999999,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67700000000000005,"15min":0.68200000000000005},"min":{"1min":0.5,"5min":0.44,"15min":0.44},"max":{"1min":0.96599999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.53700000000000003}]},{"osd":6,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.56200000000000006,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91300000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69299999999999995,"15min":0.67900000000000005},"min":{"1min":0.56599999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94199999999999995,"5min":1.04,"15min":1.04},"last":0.55000000000000004}]},{"osd":7,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.55300000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.52100000000000002,"5min":0.498,"15min":0.498},"max":{"1min":0.95799999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":609,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194702,"statfs":{"total":200000000,"available":199375555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.41699999999999998,"15min":0.40999999999999998},"min":{"1min":0.28000000000000003,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56599999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.50900000000000001,"15min":0.63700000000000001},"min":{"1min":0.34499999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72799999999999998,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":0.48199999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.434,"15min":0.42399999999999999},"min":{"1min":0.32700000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.313},{"interface":"front","average":{"1min":0.434,"5min":0.44,"15min":0.44700000000000001},"min":{"1min":0.29799999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.65300000000000002,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.42699999999999999}]},{"osd":3,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.41499999999999998},"min":{"1min":0.30399999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.65000000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76500000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.44600000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68200000000000005,"15min":0.65600000000000003},"min":{"1min":0.53600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73399999999999999,"15min":0.79900000000000004},"min":{"1min":0.54900000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.995,"5min":3.581,"15min":3.581},"last":0.68999999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68000000000000005,"15min":0.74399999999999999},"min":{"1min":0.502,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90500000000000003,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67700000000000005,"15min":0.67100000000000004},"min":{"1min":0.501,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85899999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.54800000000000004}]},{"osd":6,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.53800000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93999999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0189999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.63800000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.53600000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70299999999999996,"15min":0.69099999999999995},"min":{"1min":0.625,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":609,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194702,"statfs":{"total":200000000,"available":199375555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.38800000000000001,"15min":0.35899999999999999},"min":{"1min":0.255,"5min":0.253,"15min":0.253},"max":{"1min":0.56200000000000006,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.432},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40600000000000003,"15min":0.39500000000000002},"min":{"1min":0.29199999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.626,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.30499999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.27400000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66300000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.45600000000000002,"15min":0.42399999999999999},"min":{"1min":0.30199999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.32400000000000001}]},{"osd":2,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46100000000000002,"15min":0.43099999999999999},"min":{"1min":0.33700000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.48399999999999999,"15min":0.45000000000000001},"min":{"1min":0.378,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.375}]},{"osd":4,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.66900000000000004,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.6660000000000004,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64200000000000002,"15min":0.60599999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93300000000000005,"5min":1.012,"15min":1.012},"last":0.68400000000000005}]},{"osd":5,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.61299999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90400000000000003,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.71599999999999997,"15min":0.68000000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":4.6849999999999996,"5min":4.6849999999999996,"15min":4.6849999999999996},"last":0.69699999999999995}]},{"osd":6,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90100000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.47099999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.71199999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.71699999999999997,"15min":0.65400000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":4.7130000000000001,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68500000000000005,"15min":0.66700000000000004},"min":{"1min":0.495,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.73399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574921,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":609,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194702,"statfs":{"total":200000000,"available":199375555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.36899999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.73399999999999999,"5min":1.099,"15min":1.099},"last":1.3839999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.47799999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71099999999999997,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":7.0670000000000002}]},{"osd":1,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.39500000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72499999999999998,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":5.1529999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59799999999999998,"15min":0.59499999999999997},"min":{"1min":0.40799999999999997,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77700000000000002,"5min":1.133,"15min":1.133},"last":3.0419999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76300000000000001,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":5.0019999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.44700000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.753,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":2.073}]},{"osd":3,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.42799999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68700000000000006,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":6.7220000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.50700000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.70799999999999996,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":6.5510000000000002}]},{"osd":5,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.47599999999999998,"15min":0.48399999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.63400000000000001,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.34599999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68899999999999995,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":6.9269999999999996}]},{"osd":6,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.5},"min":{"1min":0.40100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.66600000000000004,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":5.2229999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53800000000000003,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81200000000000006,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":1.46}]},{"osd":7,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53900000000000003,"15min":0.54900000000000004},"min":{"1min":0.38500000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.65400000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":7.5339999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.432,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68100000000000005,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":4.9550000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":609,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194702,"statfs":{"total":200000000,"available":199375555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56999999999999995,"15min":0.59399999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68700000000000006,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.60099999999999998,"15min":0.57699999999999996},"min":{"1min":0.34499999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67100000000000004,"5min":3.242,"15min":3.242},"last":0.48699999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.625,"15min":0.60799999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70599999999999996,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.626,"15min":0.64900000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69299999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.72099999999999997}]},{"osd":2,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61599999999999999,"15min":0.59599999999999997},"min":{"1min":0.39100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.71499999999999997,"5min":3.04,"15min":3.04},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59399999999999997,"15min":0.57699999999999996},"min":{"1min":0.40100000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72499999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.53500000000000003}]},{"osd":3,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.66500000000000004,"15min":0.67800000000000005},"min":{"1min":0.44400000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.65500000000000003,"5min":4.032,"15min":4.032},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.70399999999999996,"15min":0.67800000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.77000000000000002,"5min":4.282,"15min":4.282},"last":0.63800000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81399999999999995,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.51100000000000001,"15min":0.56499999999999995},"min":{"1min":0.36799999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.61499999999999999,"5min":1.101,"15min":1.101},"last":0.76300000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51300000000000001,"15min":0.55600000000000005},"min":{"1min":0.36499999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.629,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56499999999999995,"15min":0.58199999999999996},"min":{"1min":0.39100000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76800000000000002,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.74099999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.55400000000000005,"15min":0.56699999999999995},"min":{"1min":0.40200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.745,"5min":1.008,"15min":1.008},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.626,"15min":0.65800000000000003},"min":{"1min":0.38400000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84799999999999998,"5min":2,"15min":2},"last":0.70799999999999996}]}]}],"pool_statfs":[]}} 2024-01-27T02:37:56.028 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:37:56.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.196+0000 7f6df9e24700 1 Processor -- start 2024-01-27T02:37:56.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.196+0000 7f6df9e24700 1 -- start start 2024-01-27T02:37:56.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.198+0000 7f6df9e24700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6df40c9bb0 0x7f6df4157f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:56.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.198+0000 7f6df9e24700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6df40c9f80 0x7f6df406b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:56.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.198+0000 7f6df9e24700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6df406e330 0x7f6df406e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:56.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.198+0000 7f6df9e24700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df406cd10 con 0x7f6df40c9f80 2024-01-27T02:37:56.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.198+0000 7f6df9e24700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df406c880 con 0x7f6df406e330 2024-01-27T02:37:56.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.198+0000 7f6df9e24700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df406ecb0 con 0x7f6df40c9bb0 2024-01-27T02:37:56.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.198+0000 7f6df2ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6df40c9f80 0x7f6df406b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:56.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.198+0000 7f6df2ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6df40c9f80 0x7f6df406b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44130/0 (socket says 172.21.15.7:44130) 2024-01-27T02:37:56.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.198+0000 7f6df2ffd700 1 -- 172.21.15.7:0/2941036557 learned_addr learned my addr 172.21.15.7:0/2941036557 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:56.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.198+0000 7f6df37fe700 1 --2- 172.21.15.7:0/2941036557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6df40c9bb0 0x7f6df4157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:56.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.198+0000 7f6df3fff700 1 --2- 172.21.15.7:0/2941036557 >> v2:172.21.15.188:3300/0 conn(0x7f6df406e330 0x7f6df406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:56.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.198+0000 7f6df2ffd700 1 -- 172.21.15.7:0/2941036557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6df40c9bb0 msgr2=0x7f6df4157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:56.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.198+0000 7f6df2ffd700 1 --2- 172.21.15.7:0/2941036557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6df40c9bb0 0x7f6df4157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.198+0000 7f6df2ffd700 1 -- 172.21.15.7:0/2941036557 >> v2:172.21.15.188:3300/0 conn(0x7f6df406e330 msgr2=0x7f6df406e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:56.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.198+0000 7f6df2ffd700 1 --2- 172.21.15.7:0/2941036557 >> v2:172.21.15.188:3300/0 conn(0x7f6df406e330 0x7f6df406e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.198+0000 7f6df2ffd700 1 -- 172.21.15.7:0/2941036557 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df406bc70 con 0x7f6df40c9f80 2024-01-27T02:37:56.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.199+0000 7f6df2ffd700 1 --2- 172.21.15.7:0/2941036557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6df40c9f80 0x7f6df406b400 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6df40c8200 tx=0x7f6ddc00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f1b03aa70c3c16 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:56.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.199+0000 7f6df27fc700 1 -- 172.21.15.7:0/2941036557 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc00dc50 con 0x7f6df40c9f80 2024-01-27T02:37:56.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.199+0000 7f6df27fc700 1 -- 172.21.15.7:0/2941036557 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc00ddb0 con 0x7f6df40c9f80 2024-01-27T02:37:56.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.199+0000 7f6df27fc700 1 -- 172.21.15.7:0/2941036557 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc0196d0 con 0x7f6df40c9f80 2024-01-27T02:37:56.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.199+0000 7f6df9e24700 1 -- 172.21.15.7:0/2941036557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6df40c9f80 msgr2=0x7f6df406b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:56.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.199+0000 7f6df9e24700 1 --2- 172.21.15.7:0/2941036557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6df40c9f80 0x7f6df406b400 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6df40c8200 tx=0x7f6ddc00b4f0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.199+0000 7f6df9e24700 1 -- 172.21.15.7:0/2941036557 shutdown_connections 2024-01-27T02:37:56.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.199+0000 7f6df9e24700 1 --2- 172.21.15.7:0/2941036557 >> v2:172.21.15.188:3300/0 conn(0x7f6df406e330 0x7f6df406e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.199+0000 7f6df9e24700 1 --2- 172.21.15.7:0/2941036557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6df40c9bb0 0x7f6df4157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.199+0000 7f6df9e24700 1 --2- 172.21.15.7:0/2941036557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6df40c9f80 0x7f6df406b400 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.199+0000 7f6df9e24700 1 -- 172.21.15.7:0/2941036557 >> 172.21.15.7:0/2941036557 conn(0x7f6df40cdc30 msgr2=0x7f6df40ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:56.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.199+0000 7f6df9e24700 1 -- 172.21.15.7:0/2941036557 shutdown_connections 2024-01-27T02:37:56.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.199+0000 7f6df9e24700 1 -- 172.21.15.7:0/2941036557 wait complete. 2024-01-27T02:37:56.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.201+0000 7f6df9e24700 1 Processor -- start 2024-01-27T02:37:56.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.201+0000 7f6df9e24700 1 -- start start 2024-01-27T02:37:56.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.202+0000 7f6df9e24700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6df406e330 0x7f6df40b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:56.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.202+0000 7f6df9e24700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6df40c9bb0 0x7f6df416ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:56.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.202+0000 7f6df9e24700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6df40c9f80 0x7f6df40afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:56.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.202+0000 7f6df9e24700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df40b3ad0 con 0x7f6df406e330 2024-01-27T02:37:56.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.202+0000 7f6df9e24700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df416d840 con 0x7f6df40c9bb0 2024-01-27T02:37:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.202+0000 7f6df9e24700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df416db20 con 0x7f6df40c9f80 2024-01-27T02:37:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.202+0000 7f6df37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6df406e330 0x7f6df40b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.202+0000 7f6df37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6df406e330 0x7f6df40b3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44134/0 (socket says 172.21.15.7:44134) 2024-01-27T02:37:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.202+0000 7f6df37fe700 1 -- 172.21.15.7:0/1467381992 learned_addr learned my addr 172.21.15.7:0/1467381992 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.202+0000 7f6df37fe700 1 -- 172.21.15.7:0/1467381992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6df40c9f80 msgr2=0x7f6df40afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.202+0000 7f6df3fff700 1 --2- 172.21.15.7:0/1467381992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6df40c9f80 0x7f6df40afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.202+0000 7f6df2ffd700 1 --2- 172.21.15.7:0/1467381992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6df40c9bb0 0x7f6df416ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.202+0000 7f6df37fe700 1 --2- 172.21.15.7:0/1467381992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6df40c9f80 0x7f6df40afe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.202+0000 7f6df37fe700 1 -- 172.21.15.7:0/1467381992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6df40c9bb0 msgr2=0x7f6df416ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.202+0000 7f6df37fe700 1 --2- 172.21.15.7:0/1467381992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6df40c9bb0 0x7f6df416ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.202+0000 7f6df37fe700 1 -- 172.21.15.7:0/1467381992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ddc00b790 con 0x7f6df406e330 2024-01-27T02:37:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.202+0000 7f6df37fe700 1 --2- 172.21.15.7:0/1467381992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6df406e330 0x7f6df40b3460 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6de400d860 tx=0x7f6de4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.202+0000 7f6deb7fe700 1 -- 172.21.15.7:0/1467381992 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de4009610 con 0x7f6df406e330 2024-01-27T02:37:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.202+0000 7f6deb7fe700 1 -- 172.21.15.7:0/1467381992 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de4009770 con 0x7f6df406e330 2024-01-27T02:37:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.202+0000 7f6deb7fe700 1 -- 172.21.15.7:0/1467381992 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de4009b30 con 0x7f6df406e330 2024-01-27T02:37:56.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.203+0000 7f6df9e24700 1 -- 172.21.15.7:0/1467381992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df416de00 con 0x7f6df406e330 2024-01-27T02:37:56.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.203+0000 7f6df9e24700 1 -- 172.21.15.7:0/1467381992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df40b0360 con 0x7f6df406e330 2024-01-27T02:37:56.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.204+0000 7f6df9e24700 1 -- 172.21.15.7:0/1467381992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df416e320 con 0x7f6df406e330 2024-01-27T02:37:56.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.204+0000 7f6deb7fe700 1 -- 172.21.15.7:0/1467381992 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6de401e460 con 0x7f6df406e330 2024-01-27T02:37:56.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.204+0000 7f6deb7fe700 1 --2- 172.21.15.7:0/1467381992 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6dd4066f30 0x7f6dd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:56.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.205+0000 7f6deb7fe700 1 -- 172.21.15.7:0/1467381992 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6de4084ba0 con 0x7f6df406e330 2024-01-27T02:37:56.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.205+0000 7f6df2ffd700 1 --2- 172.21.15.7:0/1467381992 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6dd4066f30 0x7f6dd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:56.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.205+0000 7f6df2ffd700 1 --2- 172.21.15.7:0/1467381992 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6dd4066f30 0x7f6dd40693e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6df40c8200 tx=0x7f6ddc01b000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.210+0000 7f6deb7fe700 1 -- 172.21.15.7:0/1467381992 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6df416e320 con 0x7f6df406e330 2024-01-27T02:37:56.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.408+0000 7f6df9e24700 1 -- 172.21.15.7:0/1467381992 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6df416e320 con 0x7f6dd4066f30 2024-01-27T02:37:56.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.417+0000 7f6deb7fe700 1 -- 172.21.15.7:0/1467381992 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282556 (secure 0 0 0) 0x7f6df416e320 con 0x7f6dd4066f30 2024-01-27T02:37:56.422 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:56.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.421+0000 7f6df9e24700 1 -- 172.21.15.7:0/1467381992 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6dd4066f30 msgr2=0x7f6dd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:56.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.421+0000 7f6df9e24700 1 --2- 172.21.15.7:0/1467381992 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6dd4066f30 0x7f6dd40693e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6df40c8200 tx=0x7f6ddc01b000 comp rx=0 tx=0).stop 2024-01-27T02:37:56.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.421+0000 7f6df9e24700 1 -- 172.21.15.7:0/1467381992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6df406e330 msgr2=0x7f6df40b3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:56.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.421+0000 7f6df9e24700 1 --2- 172.21.15.7:0/1467381992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6df406e330 0x7f6df40b3460 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6de400d860 tx=0x7f6de4008040 comp rx=0 tx=0).stop 2024-01-27T02:37:56.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.421+0000 7f6df9e24700 1 -- 172.21.15.7:0/1467381992 shutdown_connections 2024-01-27T02:37:56.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.421+0000 7f6df9e24700 1 --2- 172.21.15.7:0/1467381992 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6dd4066f30 0x7f6dd40693e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.421+0000 7f6df9e24700 1 --2- 172.21.15.7:0/1467381992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6df40c9f80 0x7f6df40afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.421+0000 7f6df9e24700 1 --2- 172.21.15.7:0/1467381992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6df406e330 0x7f6df40b3460 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.421+0000 7f6df9e24700 1 --2- 172.21.15.7:0/1467381992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6df40c9bb0 0x7f6df416ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.421+0000 7f6df9e24700 1 -- 172.21.15.7:0/1467381992 >> 172.21.15.7:0/1467381992 conn(0x7f6df40cdc30 msgr2=0x7f6df40639c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:56.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.421+0000 7f6df9e24700 1 -- 172.21.15.7:0/1467381992 shutdown_connections 2024-01-27T02:37:56.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.421+0000 7f6df9e24700 1 -- 172.21.15.7:0/1467381992 wait complete. 2024-01-27T02:37:56.425 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:37:56.435 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-01-27T02:37:55.658393+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6025,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556474,"statfs":{"total":1600000000,"available":1593829967,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.866207"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800705+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:29:06.789618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.811901+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:57.231078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800547+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:41:58.817551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.799051+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:17:03.370160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.808124+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:14:51.191272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801275+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:45:13.633104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800588+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:14:51.191272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.811924+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:04:59.885669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.805962+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:55.343516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.807734+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:55.343516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801974+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:28:34.028232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798839+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:26.529268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801629+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:55:18.331130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.809936+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:29:56.743131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798550+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:04:02.332379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801655+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:25:00.247752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.811458+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:46:44.234857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798522+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:14:51.191272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.809517+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:07:53.995321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.799143+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:12:54.826584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.806764+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:19:48.163385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.811035+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:33:08.282783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.811487+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:10:19.964021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800572+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:55.343516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.807163+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:33:14.691144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.807469+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:17:03.370160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800915+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:17:03.370160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.807224+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:24:13.096584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.808167+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:45:13.633104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.812972+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:50:16.532574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798553+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:55.343516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801905+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:23:06.453385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.809151+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:45:52.881907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.812131+0000","last_change":"2024-01-27T02:37:50.812131+0000","last_active":"2024-01-27T02:37:50.812131+0000","last_peered":"2024-01-27T02:37:50.812131+0000","last_clean":"2024-01-27T02:37:50.812131+0000","last_became_active":"2024-01-27T02:37:50.809514+0000","last_became_peered":"2024-01-27T02:37:50.809514+0000","last_unstale":"2024-01-27T02:37:50.812131+0000","last_undegraded":"2024-01-27T02:37:50.812131+0000","last_fullsized":"2024-01-27T02:37:50.812131+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:14:50.186256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.797941+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:24:13.096584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.808649+0000","last_change":"2024-01-27T02:37:50.808649+0000","last_active":"2024-01-27T02:37:50.808649+0000","last_peered":"2024-01-27T02:37:50.808649+0000","last_clean":"2024-01-27T02:37:50.808649+0000","last_became_active":"2024-01-27T02:37:50.808412+0000","last_became_peered":"2024-01-27T02:37:50.808412+0000","last_unstale":"2024-01-27T02:37:50.808649+0000","last_undegraded":"2024-01-27T02:37:50.808649+0000","last_fullsized":"2024-01-27T02:37:50.808649+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:24:12.091569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800223+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:10:27.309356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.806041+0000","last_change":"2024-01-27T02:37:50.806041+0000","last_active":"2024-01-27T02:37:50.806041+0000","last_peered":"2024-01-27T02:37:50.806041+0000","last_clean":"2024-01-27T02:37:50.806041+0000","last_became_active":"2024-01-27T02:37:50.805760+0000","last_became_peered":"2024-01-27T02:37:50.805760+0000","last_unstale":"2024-01-27T02:37:50.806041+0000","last_undegraded":"2024-01-27T02:37:50.806041+0000","last_fullsized":"2024-01-27T02:37:50.806041+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:31:21.182217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798399+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:57:57.067686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.808048+0000","last_change":"2024-01-27T02:37:50.808048+0000","last_active":"2024-01-27T02:37:50.808048+0000","last_peered":"2024-01-27T02:37:50.808048+0000","last_clean":"2024-01-27T02:37:50.808048+0000","last_became_active":"2024-01-27T02:37:50.807771+0000","last_became_peered":"2024-01-27T02:37:50.807771+0000","last_unstale":"2024-01-27T02:37:50.808048+0000","last_undegraded":"2024-01-27T02:37:50.808048+0000","last_fullsized":"2024-01-27T02:37:50.808048+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:24.808914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800346+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:26.529268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.806436+0000","last_change":"2024-01-27T02:37:50.806436+0000","last_active":"2024-01-27T02:37:50.806436+0000","last_peered":"2024-01-27T02:37:50.806436+0000","last_clean":"2024-01-27T02:37:50.806436+0000","last_became_active":"2024-01-27T02:37:50.806167+0000","last_became_peered":"2024-01-27T02:37:50.806167+0000","last_unstale":"2024-01-27T02:37:50.806436+0000","last_undegraded":"2024-01-27T02:37:50.806436+0000","last_fullsized":"2024-01-27T02:37:50.806436+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:58:12.533491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800174+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:25.813930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.812296+0000","last_change":"2024-01-27T02:37:50.812296+0000","last_active":"2024-01-27T02:37:50.812296+0000","last_peered":"2024-01-27T02:37:50.812296+0000","last_clean":"2024-01-27T02:37:50.812296+0000","last_became_active":"2024-01-27T02:37:50.809825+0000","last_became_peered":"2024-01-27T02:37:50.809825+0000","last_unstale":"2024-01-27T02:37:50.812296+0000","last_undegraded":"2024-01-27T02:37:50.812296+0000","last_fullsized":"2024-01-27T02:37:50.812296+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:54.338500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798046+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:33:14.691144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.804462+0000","last_change":"2024-01-27T02:37:50.804462+0000","last_active":"2024-01-27T02:37:50.804462+0000","last_peered":"2024-01-27T02:37:50.804462+0000","last_clean":"2024-01-27T02:37:50.804462+0000","last_became_active":"2024-01-27T02:37:50.804211+0000","last_became_peered":"2024-01-27T02:37:50.804211+0000","last_unstale":"2024-01-27T02:37:50.804462+0000","last_undegraded":"2024-01-27T02:37:50.804462+0000","last_fullsized":"2024-01-27T02:37:50.804462+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:16:34.794337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798801+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:16:35.799353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.804294+0000","last_change":"2024-01-27T02:37:50.804294+0000","last_active":"2024-01-27T02:37:50.804294+0000","last_peered":"2024-01-27T02:37:50.804294+0000","last_clean":"2024-01-27T02:37:50.804294+0000","last_became_active":"2024-01-27T02:37:50.803789+0000","last_became_peered":"2024-01-27T02:37:50.803789+0000","last_unstale":"2024-01-27T02:37:50.804294+0000","last_undegraded":"2024-01-27T02:37:50.804294+0000","last_fullsized":"2024-01-27T02:37:50.804294+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:37:47.399101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.808533+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:55:18.331130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.807055+0000","last_change":"2024-01-27T02:37:50.807055+0000","last_active":"2024-01-27T02:37:50.807055+0000","last_peered":"2024-01-27T02:37:50.807055+0000","last_clean":"2024-01-27T02:37:50.807055+0000","last_became_active":"2024-01-27T02:37:50.806246+0000","last_became_peered":"2024-01-27T02:37:50.806246+0000","last_unstale":"2024-01-27T02:37:50.807055+0000","last_undegraded":"2024-01-27T02:37:50.807055+0000","last_fullsized":"2024-01-27T02:37:50.807055+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:23.160028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798337+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:54:58.004832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.805856+0000","last_change":"2024-01-27T02:37:50.805856+0000","last_active":"2024-01-27T02:37:50.805856+0000","last_peered":"2024-01-27T02:37:50.805856+0000","last_clean":"2024-01-27T02:37:50.805856+0000","last_became_active":"2024-01-27T02:37:50.805568+0000","last_became_peered":"2024-01-27T02:37:50.805568+0000","last_unstale":"2024-01-27T02:37:50.805856+0000","last_undegraded":"2024-01-27T02:37:50.805856+0000","last_fullsized":"2024-01-27T02:37:50.805856+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:11:46.349861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.808509+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:45:13.633104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.809623+0000","last_change":"2024-01-27T02:37:50.809623+0000","last_active":"2024-01-27T02:37:50.809623+0000","last_peered":"2024-01-27T02:37:50.809623+0000","last_clean":"2024-01-27T02:37:50.809623+0000","last_became_active":"2024-01-27T02:37:50.809447+0000","last_became_peered":"2024-01-27T02:37:50.809447+0000","last_unstale":"2024-01-27T02:37:50.809623+0000","last_undegraded":"2024-01-27T02:37:50.809623+0000","last_fullsized":"2024-01-27T02:37:50.809623+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:50.938489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801315+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:55:18.331130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.814137+0000","last_change":"2024-01-27T02:37:50.814137+0000","last_active":"2024-01-27T02:37:50.814137+0000","last_peered":"2024-01-27T02:37:50.814137+0000","last_clean":"2024-01-27T02:37:50.814137+0000","last_became_active":"2024-01-27T02:37:50.813898+0000","last_became_peered":"2024-01-27T02:37:50.813898+0000","last_unstale":"2024-01-27T02:37:50.814137+0000","last_undegraded":"2024-01-27T02:37:50.814137+0000","last_fullsized":"2024-01-27T02:37:50.814137+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:56:37.175345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798050+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:25.813930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.813470+0000","last_change":"2024-01-27T02:37:50.813470+0000","last_active":"2024-01-27T02:37:50.813470+0000","last_peered":"2024-01-27T02:37:50.813470+0000","last_clean":"2024-01-27T02:37:50.813470+0000","last_became_active":"2024-01-27T02:37:50.812014+0000","last_became_peered":"2024-01-27T02:37:50.812014+0000","last_unstale":"2024-01-27T02:37:50.813470+0000","last_undegraded":"2024-01-27T02:37:50.813470+0000","last_fullsized":"2024-01-27T02:37:50.813470+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:56:29.266154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801084+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:53:55.703491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.807923+0000","last_change":"2024-01-27T02:37:50.807923+0000","last_active":"2024-01-27T02:37:50.807923+0000","last_peered":"2024-01-27T02:37:50.807923+0000","last_clean":"2024-01-27T02:37:50.807923+0000","last_became_active":"2024-01-27T02:37:50.807417+0000","last_became_peered":"2024-01-27T02:37:50.807417+0000","last_unstale":"2024-01-27T02:37:50.807923+0000","last_undegraded":"2024-01-27T02:37:50.807923+0000","last_fullsized":"2024-01-27T02:37:50.807923+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:26:35.727576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.797951+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:01:25.391647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.809421+0000","last_change":"2024-01-27T02:37:50.809421+0000","last_active":"2024-01-27T02:37:50.809421+0000","last_peered":"2024-01-27T02:37:50.809421+0000","last_clean":"2024-01-27T02:37:50.809421+0000","last_became_active":"2024-01-27T02:37:50.809169+0000","last_became_peered":"2024-01-27T02:37:50.809169+0000","last_unstale":"2024-01-27T02:37:50.809421+0000","last_undegraded":"2024-01-27T02:37:50.809421+0000","last_fullsized":"2024-01-27T02:37:50.809421+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:31:21.182217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.806904+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:46:21.029594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.810368+0000","last_change":"2024-01-27T02:37:50.810368+0000","last_active":"2024-01-27T02:37:50.810368+0000","last_peered":"2024-01-27T02:37:50.810368+0000","last_clean":"2024-01-27T02:37:50.810368+0000","last_became_active":"2024-01-27T02:37:50.810089+0000","last_became_peered":"2024-01-27T02:37:50.810089+0000","last_unstale":"2024-01-27T02:37:50.810368+0000","last_undegraded":"2024-01-27T02:37:50.810368+0000","last_fullsized":"2024-01-27T02:37:50.810368+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:50:15.527559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.807362+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:25.813930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.803536+0000","last_change":"2024-01-27T02:37:50.803536+0000","last_active":"2024-01-27T02:37:50.803536+0000","last_peered":"2024-01-27T02:37:50.803536+0000","last_clean":"2024-01-27T02:37:50.803536+0000","last_became_active":"2024-01-27T02:37:50.803332+0000","last_became_peered":"2024-01-27T02:37:50.803332+0000","last_unstale":"2024-01-27T02:37:50.803536+0000","last_undegraded":"2024-01-27T02:37:50.803536+0000","last_fullsized":"2024-01-27T02:37:50.803536+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:05:27.161024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.799402+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:57:57.067686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.802000+0000","last_change":"2024-01-27T02:37:50.802000+0000","last_active":"2024-01-27T02:37:50.802000+0000","last_peered":"2024-01-27T02:37:50.802000+0000","last_clean":"2024-01-27T02:37:50.802000+0000","last_became_active":"2024-01-27T02:37:50.801755+0000","last_became_peered":"2024-01-27T02:37:50.801755+0000","last_unstale":"2024-01-27T02:37:50.802000+0000","last_undegraded":"2024-01-27T02:37:50.802000+0000","last_fullsized":"2024-01-27T02:37:50.802000+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:54:56.999817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800829+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:04:02.332379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.812532+0000","last_change":"2024-01-27T02:37:50.812532+0000","last_active":"2024-01-27T02:37:50.812532+0000","last_peered":"2024-01-27T02:37:50.812532+0000","last_clean":"2024-01-27T02:37:50.812532+0000","last_became_active":"2024-01-27T02:37:50.812209+0000","last_became_peered":"2024-01-27T02:37:50.812209+0000","last_unstale":"2024-01-27T02:37:50.812532+0000","last_undegraded":"2024-01-27T02:37:50.812532+0000","last_fullsized":"2024-01-27T02:37:50.812532+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:11:46.349861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.808739+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:47:12.515208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.813813+0000","last_change":"2024-01-27T02:37:50.813813+0000","last_active":"2024-01-27T02:37:50.813813+0000","last_peered":"2024-01-27T02:37:50.813813+0000","last_clean":"2024-01-27T02:37:50.813813+0000","last_became_active":"2024-01-27T02:37:50.813235+0000","last_became_peered":"2024-01-27T02:37:50.813235+0000","last_unstale":"2024-01-27T02:37:50.813813+0000","last_undegraded":"2024-01-27T02:37:50.813813+0000","last_fullsized":"2024-01-27T02:37:50.813813+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:33:13.686128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.807809+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:14:52.244572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.804058+0000","last_change":"2024-01-27T02:37:50.804058+0000","last_active":"2024-01-27T02:37:50.804058+0000","last_peered":"2024-01-27T02:37:50.804058+0000","last_clean":"2024-01-27T02:37:50.804058+0000","last_became_active":"2024-01-27T02:37:50.803802+0000","last_became_peered":"2024-01-27T02:37:50.803802+0000","last_unstale":"2024-01-27T02:37:50.804058+0000","last_undegraded":"2024-01-27T02:37:50.804058+0000","last_fullsized":"2024-01-27T02:37:50.804058+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:01:24.386631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.806271+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:19:50.489258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.806415+0000","last_change":"2024-01-27T02:37:50.806415+0000","last_active":"2024-01-27T02:37:50.806415+0000","last_peered":"2024-01-27T02:37:50.806415+0000","last_clean":"2024-01-27T02:37:50.806415+0000","last_became_active":"2024-01-27T02:37:50.806149+0000","last_became_peered":"2024-01-27T02:37:50.806149+0000","last_unstale":"2024-01-27T02:37:50.806415+0000","last_undegraded":"2024-01-27T02:37:50.806415+0000","last_fullsized":"2024-01-27T02:37:50.806415+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:04:39.266345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.806325+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:28.200720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.809566+0000","last_change":"2024-01-27T02:37:50.809566+0000","last_active":"2024-01-27T02:37:50.809566+0000","last_peered":"2024-01-27T02:37:50.809566+0000","last_clean":"2024-01-27T02:37:50.809566+0000","last_became_active":"2024-01-27T02:37:50.809276+0000","last_became_peered":"2024-01-27T02:37:50.809276+0000","last_unstale":"2024-01-27T02:37:50.809566+0000","last_undegraded":"2024-01-27T02:37:50.809566+0000","last_fullsized":"2024-01-27T02:37:50.809566+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:24:12.091569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.811076+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:18:23.945261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.805793+0000","last_change":"2024-01-27T02:37:50.805793+0000","last_active":"2024-01-27T02:37:50.805793+0000","last_peered":"2024-01-27T02:37:50.805793+0000","last_clean":"2024-01-27T02:37:50.805793+0000","last_became_active":"2024-01-27T02:37:50.805524+0000","last_became_peered":"2024-01-27T02:37:50.805524+0000","last_unstale":"2024-01-27T02:37:50.805793+0000","last_undegraded":"2024-01-27T02:37:50.805793+0000","last_fullsized":"2024-01-27T02:37:50.805793+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:57:33.113268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801404+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:17:37.405535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.814412+0000","last_change":"2024-01-27T02:37:50.814412+0000","last_active":"2024-01-27T02:37:50.814412+0000","last_peered":"2024-01-27T02:37:50.814412+0000","last_clean":"2024-01-27T02:37:50.814412+0000","last_became_active":"2024-01-27T02:37:50.813560+0000","last_became_peered":"2024-01-27T02:37:50.813560+0000","last_unstale":"2024-01-27T02:37:50.814412+0000","last_undegraded":"2024-01-27T02:37:50.814412+0000","last_fullsized":"2024-01-27T02:37:50.814412+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:27.195705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800617+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:08:13.232740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.808731+0000","last_change":"2024-01-27T02:37:50.808731+0000","last_active":"2024-01-27T02:37:50.808731+0000","last_peered":"2024-01-27T02:37:50.808731+0000","last_clean":"2024-01-27T02:37:50.808731+0000","last_became_active":"2024-01-27T02:37:50.808482+0000","last_became_peered":"2024-01-27T02:37:50.808482+0000","last_unstale":"2024-01-27T02:37:50.808731+0000","last_undegraded":"2024-01-27T02:37:50.808731+0000","last_fullsized":"2024-01-27T02:37:50.808731+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:50.938489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.806197+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:11:47.354877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.808670+0000","last_change":"2024-01-27T02:37:50.808670+0000","last_active":"2024-01-27T02:37:50.808670+0000","last_peered":"2024-01-27T02:37:50.808670+0000","last_clean":"2024-01-27T02:37:50.808670+0000","last_became_active":"2024-01-27T02:37:50.808428+0000","last_became_peered":"2024-01-27T02:37:50.808428+0000","last_unstale":"2024-01-27T02:37:50.808670+0000","last_undegraded":"2024-01-27T02:37:50.808670+0000","last_fullsized":"2024-01-27T02:37:50.808670+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:57:56.062671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.806456+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:07:01.540761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.813250+0000","last_change":"2024-01-27T02:37:50.813250+0000","last_active":"2024-01-27T02:37:50.813250+0000","last_peered":"2024-01-27T02:37:50.813250+0000","last_clean":"2024-01-27T02:37:50.813250+0000","last_became_active":"2024-01-27T02:37:50.813038+0000","last_became_peered":"2024-01-27T02:37:50.813038+0000","last_unstale":"2024-01-27T02:37:50.813250+0000","last_undegraded":"2024-01-27T02:37:50.813250+0000","last_fullsized":"2024-01-27T02:37:50.813250+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:25.524253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.797951+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:57:34.118284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.803902+0000","last_change":"2024-01-27T02:37:50.803902+0000","last_active":"2024-01-27T02:37:50.803902+0000","last_peered":"2024-01-27T02:37:50.803902+0000","last_clean":"2024-01-27T02:37:50.803902+0000","last_became_active":"2024-01-27T02:37:50.803626+0000","last_became_peered":"2024-01-27T02:37:50.803626+0000","last_unstale":"2024-01-27T02:37:50.803902+0000","last_undegraded":"2024-01-27T02:37:50.803902+0000","last_fullsized":"2024-01-27T02:37:50.803902+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:25.524253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.808334+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:33:48.901396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.804116+0000","last_change":"2024-01-27T02:37:50.804116+0000","last_active":"2024-01-27T02:37:50.804116+0000","last_peered":"2024-01-27T02:37:50.804116+0000","last_clean":"2024-01-27T02:37:50.804116+0000","last_became_active":"2024-01-27T02:37:50.803880+0000","last_became_peered":"2024-01-27T02:37:50.803880+0000","last_unstale":"2024-01-27T02:37:50.804116+0000","last_undegraded":"2024-01-27T02:37:50.804116+0000","last_fullsized":"2024-01-27T02:37:50.804116+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:50.938489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.806238+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:51.943504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.807085+0000","last_change":"2024-01-27T02:37:50.807085+0000","last_active":"2024-01-27T02:37:50.807085+0000","last_peered":"2024-01-27T02:37:50.807085+0000","last_clean":"2024-01-27T02:37:50.807085+0000","last_became_active":"2024-01-27T02:37:50.804683+0000","last_became_peered":"2024-01-27T02:37:50.804683+0000","last_unstale":"2024-01-27T02:37:50.807085+0000","last_undegraded":"2024-01-27T02:37:50.807085+0000","last_fullsized":"2024-01-27T02:37:50.807085+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:49:46.671831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.812579+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:11:16.205143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.809913+0000","last_change":"2024-01-27T02:37:50.809913+0000","last_active":"2024-01-27T02:37:50.809913+0000","last_peered":"2024-01-27T02:37:50.809913+0000","last_clean":"2024-01-27T02:37:50.809913+0000","last_became_active":"2024-01-27T02:37:50.809679+0000","last_became_peered":"2024-01-27T02:37:50.809679+0000","last_unstale":"2024-01-27T02:37:50.809913+0000","last_undegraded":"2024-01-27T02:37:50.809913+0000","last_fullsized":"2024-01-27T02:37:50.809913+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:27.195705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":245,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-27T02:37:49.795519+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:37:49.795519+0000","last_peered":"2024-01-27T02:37:49.795519+0000","last_clean":"2024-01-27T02:37:49.795519+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:37:49.795519+0000","last_undegraded":"2024-01-27T02:37:49.795519+0000","last_fullsized":"2024-01-27T02:37:49.795519+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574921,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1186,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194126,"statfs":{"total":200000000,"available":198785187,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71299999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.59099999999999997,"15min":0.58999999999999997},"min":{"1min":0.49099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77400000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.64000000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59699999999999998,"15min":0.59799999999999998},"min":{"1min":0.46100000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.44400000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.753,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.53600000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58299999999999996},"min":{"1min":0.432,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.439},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77700000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.65500000000000003}]},{"osd":3,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60499999999999998,"15min":0.59299999999999997},"min":{"1min":0.495,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.60999999999999999,"15min":0.59499999999999997},"min":{"1min":0.5,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85099999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.71799999999999997}]},{"osd":4,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.47499999999999998,"15min":0.48699999999999999},"min":{"1min":0.32400000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63300000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.495,"15min":0.51300000000000001},"min":{"1min":0.35999999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68899999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.80000000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51800000000000002,"15min":0.53600000000000003},"min":{"1min":0.39000000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.56399999999999995},"min":{"1min":0.40100000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.75800000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.59499999999999997},"min":{"1min":0.40999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.751,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.57699999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.81599999999999995,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.73099999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":603,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194708,"statfs":{"total":200000000,"available":199381818,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58699999999999997,"15min":0.52100000000000002},"min":{"1min":0.45000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.52700000000000002},"min":{"1min":0.434,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71899999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.79900000000000004}]},{"osd":1,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.51800000000000002},"min":{"1min":0.41999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79700000000000004,"5min":0.874,"15min":0.874},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.60599999999999998,"15min":0.55500000000000005},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.54000000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.41199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.625,"15min":0.58299999999999996},"min":{"1min":0.45100000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79700000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59399999999999997}]},{"osd":3,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58199999999999996,"15min":0.55100000000000005},"min":{"1min":0.47599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.76800000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61499999999999999,"15min":0.57299999999999995},"min":{"1min":0.48499999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.78100000000000003,"5min":1.262,"15min":1.262},"last":0.57099999999999995}]},{"osd":4,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.34200000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52700000000000002,"15min":0.55300000000000005},"min":{"1min":0.41199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68300000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.72699999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53200000000000003,"15min":0.53500000000000003},"min":{"1min":0.38800000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70799999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.755},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.60599999999999998},"min":{"1min":0.40899999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70799999999999996,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.63300000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60599999999999998,"15min":0.61599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73699999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71099999999999997,"5min":1.038,"15min":1.038},"last":0.60999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":609,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194702,"statfs":{"total":200000000,"available":199375555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39700000000000002,"15min":0.377},"min":{"1min":0.27000000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.41999999999999998,"15min":0.40000000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.374}]},{"osd":2,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.40200000000000002,"15min":0.36599999999999999},"min":{"1min":0.28000000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45400000000000001,"15min":0.43099999999999999},"min":{"1min":0.30199999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.39500000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48999999999999999,"15min":0.46899999999999997},"min":{"1min":0.35799999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.434},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.47499999999999998,"15min":0.45800000000000002},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.314}]},{"osd":4,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.66200000000000003,"15min":0.63400000000000001},"min":{"1min":0.442,"5min":0.373,"15min":0.373},"max":{"1min":3.2890000000000001,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65800000000000003,"15min":0.63600000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.59299999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.68500000000000005,"15min":0.63700000000000001},"min":{"1min":0.50600000000000001,"5min":0.44,"15min":0.44},"max":{"1min":3.4500000000000002,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.68999999999999995,"15min":0.67600000000000005},"min":{"1min":0.48299999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.181,"5min":3.181,"15min":3.181},"last":0.748}]},{"osd":6,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70999999999999996,"15min":0.69599999999999995},"min":{"1min":0.52100000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96399999999999997,"5min":1.024,"15min":1.024},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.71299999999999997,"15min":0.67900000000000005},"min":{"1min":0.53400000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.79900000000000004}]},{"osd":7,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.72399999999999998,"15min":0.68600000000000005},"min":{"1min":0.56100000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.72499999999999998,"15min":0.69199999999999995},"min":{"1min":0.57799999999999996,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.70599999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574921,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1186,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194126,"statfs":{"total":200000000,"available":198785187,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.38900000000000001},"min":{"1min":0.28100000000000003,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.66600000000000004,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27800000000000002,"5min":0.223,"15min":0.223},"max":{"1min":0.65200000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.39400000000000002}]},{"osd":2,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.436,"15min":0.42199999999999999},"min":{"1min":0.32800000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56299999999999994,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.441},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.44900000000000001,"15min":0.435},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58999999999999997,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.628}]},{"osd":3,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42999999999999999,"15min":0.432},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.63600000000000001,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.35299999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.63200000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.36299999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.65500000000000003,"15min":0.64000000000000001},"min":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.501},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45900000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89700000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.59799999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67500000000000004,"15min":0.65600000000000003},"min":{"1min":0.48399999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.86499999999999999,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67700000000000005,"15min":0.68200000000000005},"min":{"1min":0.5,"5min":0.44,"15min":0.44},"max":{"1min":0.96599999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.53700000000000003}]},{"osd":6,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.56200000000000006,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91300000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69299999999999995,"15min":0.67900000000000005},"min":{"1min":0.56599999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94199999999999995,"5min":1.04,"15min":1.04},"last":0.55000000000000004}]},{"osd":7,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.55300000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.52100000000000002,"5min":0.498,"15min":0.498},"max":{"1min":0.95799999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":609,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194702,"statfs":{"total":200000000,"available":199375555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.41699999999999998,"15min":0.40999999999999998},"min":{"1min":0.28000000000000003,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56599999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.50900000000000001,"15min":0.63700000000000001},"min":{"1min":0.34499999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72799999999999998,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":0.48199999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.434,"15min":0.42399999999999999},"min":{"1min":0.32700000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.313},{"interface":"front","average":{"1min":0.434,"5min":0.44,"15min":0.44700000000000001},"min":{"1min":0.29799999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.65300000000000002,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.42699999999999999}]},{"osd":3,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.41499999999999998},"min":{"1min":0.30399999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.65000000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76500000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.44600000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68200000000000005,"15min":0.65600000000000003},"min":{"1min":0.53600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73399999999999999,"15min":0.79900000000000004},"min":{"1min":0.54900000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.995,"5min":3.581,"15min":3.581},"last":0.68999999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68000000000000005,"15min":0.74399999999999999},"min":{"1min":0.502,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90500000000000003,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67700000000000005,"15min":0.67100000000000004},"min":{"1min":0.501,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85899999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.54800000000000004}]},{"osd":6,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.53800000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93999999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0189999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.63800000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.53600000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70299999999999996,"15min":0.69099999999999995},"min":{"1min":0.625,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":609,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194702,"statfs":{"total":200000000,"available":199375555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.38800000000000001,"15min":0.35899999999999999},"min":{"1min":0.255,"5min":0.253,"15min":0.253},"max":{"1min":0.56200000000000006,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.432},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40600000000000003,"15min":0.39500000000000002},"min":{"1min":0.29199999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.626,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.30499999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.27400000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66300000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.45600000000000002,"15min":0.42399999999999999},"min":{"1min":0.30199999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.32400000000000001}]},{"osd":2,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46100000000000002,"15min":0.43099999999999999},"min":{"1min":0.33700000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.48399999999999999,"15min":0.45000000000000001},"min":{"1min":0.378,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.375}]},{"osd":4,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.66900000000000004,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.6660000000000004,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64200000000000002,"15min":0.60599999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93300000000000005,"5min":1.012,"15min":1.012},"last":0.68400000000000005}]},{"osd":5,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.61299999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90400000000000003,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.71599999999999997,"15min":0.68000000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":4.6849999999999996,"5min":4.6849999999999996,"15min":4.6849999999999996},"last":0.69699999999999995}]},{"osd":6,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90100000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.47099999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.71199999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.71699999999999997,"15min":0.65400000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":4.7130000000000001,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68500000000000005,"15min":0.66700000000000004},"min":{"1min":0.495,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.73399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574921,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":609,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194702,"statfs":{"total":200000000,"available":199375555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.36899999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.73399999999999999,"5min":1.099,"15min":1.099},"last":1.3839999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.47799999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71099999999999997,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":7.0670000000000002}]},{"osd":1,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.39500000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72499999999999998,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":5.1529999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59799999999999998,"15min":0.59499999999999997},"min":{"1min":0.40799999999999997,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77700000000000002,"5min":1.133,"15min":1.133},"last":3.0419999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76300000000000001,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":5.0019999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.44700000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.753,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":2.073}]},{"osd":3,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.42799999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68700000000000006,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":6.7220000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.50700000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.70799999999999996,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":6.5510000000000002}]},{"osd":5,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.47599999999999998,"15min":0.48399999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.63400000000000001,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.34599999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68899999999999995,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":6.9269999999999996}]},{"osd":6,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.5},"min":{"1min":0.40100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.66600000000000004,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":5.2229999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53800000000000003,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81200000000000006,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":1.46}]},{"osd":7,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53900000000000003,"15min":0.54900000000000004},"min":{"1min":0.38500000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.65400000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":7.5339999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.432,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68100000000000005,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":4.9550000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":609,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194702,"statfs":{"total":200000000,"available":199375555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56999999999999995,"15min":0.59399999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68700000000000006,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.60099999999999998,"15min":0.57699999999999996},"min":{"1min":0.34499999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67100000000000004,"5min":3.242,"15min":3.242},"last":0.48699999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.625,"15min":0.60799999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70599999999999996,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.626,"15min":0.64900000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69299999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.72099999999999997}]},{"osd":2,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61599999999999999,"15min":0.59599999999999997},"min":{"1min":0.39100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.71499999999999997,"5min":3.04,"15min":3.04},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59399999999999997,"15min":0.57699999999999996},"min":{"1min":0.40100000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72499999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.53500000000000003}]},{"osd":3,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.66500000000000004,"15min":0.67800000000000005},"min":{"1min":0.44400000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.65500000000000003,"5min":4.032,"15min":4.032},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.70399999999999996,"15min":0.67800000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.77000000000000002,"5min":4.282,"15min":4.282},"last":0.63800000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81399999999999995,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.51100000000000001,"15min":0.56499999999999995},"min":{"1min":0.36799999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.61499999999999999,"5min":1.101,"15min":1.101},"last":0.76300000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51300000000000001,"15min":0.55600000000000005},"min":{"1min":0.36499999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.629,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56499999999999995,"15min":0.58199999999999996},"min":{"1min":0.39100000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76800000000000002,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.74099999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.55400000000000005,"15min":0.56699999999999995},"min":{"1min":0.40200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.745,"5min":1.008,"15min":1.008},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.626,"15min":0.65800000000000003},"min":{"1min":0.38400000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84799999999999998,"5min":2,"15min":2},"last":0.70799999999999996}]}]}],"pool_statfs":[]}} 2024-01-27T02:37:56.472 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:37:56.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.611+0000 7fc995869700 1 Processor -- start 2024-01-27T02:37:56.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.611+0000 7fc995869700 1 -- start start 2024-01-27T02:37:56.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.612+0000 7fc995869700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9900d1470 0x7fc9900b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:56.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.612+0000 7fc995869700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc990159b40 0x7fc990159f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:56.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.612+0000 7fc995869700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9900b4650 con 0x7fc9900d10a0 2024-01-27T02:37:56.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.612+0000 7fc995869700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9900b34d0 con 0x7fc990159b40 2024-01-27T02:37:56.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.612+0000 7fc995869700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9900b3d60 con 0x7fc9900d1470 2024-01-27T02:37:56.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.612+0000 7fc98effd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc9900d10a0 0x7fc9900b2dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50776/0 (socket says 172.21.15.7:50776) 2024-01-27T02:37:56.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.612+0000 7fc98effd700 1 -- 172.21.15.7:0/301931191 learned_addr learned my addr 172.21.15.7:0/301931191 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:56.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.612+0000 7fc98e7fc700 1 --2- 172.21.15.7:0/301931191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9900d1470 0x7fc9900b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:56.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.612+0000 7fc98f7fe700 1 --2- 172.21.15.7:0/301931191 >> v2:172.21.15.188:3300/0 conn(0x7fc990159b40 0x7fc990159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:56.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98dffb700 1 -- 172.21.15.7:0/301931191 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508393609 0 0) 0x7fc9900b4650 con 0x7fc9900d10a0 2024-01-27T02:37:56.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98dffb700 1 -- 172.21.15.7:0/301931191 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc97c003570 con 0x7fc9900d10a0 2024-01-27T02:37:56.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98dffb700 1 -- 172.21.15.7:0/301931191 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 330406079 0 0) 0x7fc97c003570 con 0x7fc9900d10a0 2024-01-27T02:37:56.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98dffb700 1 -- 172.21.15.7:0/301931191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9900d1470 msgr2=0x7fc9900b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:56.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98dffb700 1 --2- 172.21.15.7:0/301931191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9900d1470 0x7fc9900b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98dffb700 1 -- 172.21.15.7:0/301931191 >> v2:172.21.15.188:3300/0 conn(0x7fc990159b40 msgr2=0x7fc990159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:56.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98dffb700 1 --2- 172.21.15.7:0/301931191 >> v2:172.21.15.188:3300/0 conn(0x7fc990159b40 0x7fc990159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98dffb700 1 -- 172.21.15.7:0/301931191 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9900b4000 con 0x7fc9900d10a0 2024-01-27T02:37:56.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98dffb700 1 -- 172.21.15.7:0/301931191 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7fc980001470 con 0x7fc9900d10a0 2024-01-27T02:37:56.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98dffb700 1 -- 172.21.15.7:0/301931191 >> v1:172.21.15.7:6789/0 conn(0x7fc9900d10a0 legacy=0x7fc9900b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:56.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98dffb700 1 --2- 172.21.15.7:0/301931191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97c003a60 0x7fc97c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:56.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98dffb700 1 --2- 172.21.15.7:0/301931191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9900d1470 0x7fc97c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:56.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98dffb700 1 --2- 172.21.15.7:0/301931191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc97c004d50 0x7fc97c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:56.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98dffb700 1 -- 172.21.15.7:0/301931191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9900b34d0 con 0x7fc97c004d50 2024-01-27T02:37:56.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98dffb700 1 -- 172.21.15.7:0/301931191 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9900b3d60 con 0x7fc97c003a60 2024-01-27T02:37:56.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98dffb700 1 -- 172.21.15.7:0/301931191 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c00ba60 con 0x7fc9900d1470 2024-01-27T02:37:56.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98f7fe700 1 --2- 172.21.15.7:0/301931191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc97c004d50 0x7fc97c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:56.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98effd700 1 --2- 172.21.15.7:0/301931191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97c003a60 0x7fc97c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:56.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98e7fc700 1 --2- 172.21.15.7:0/301931191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9900d1470 0x7fc97c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:56.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98f7fe700 1 -- 172.21.15.7:0/301931191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9900d1470 msgr2=0x7fc97c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:56.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98f7fe700 1 --2- 172.21.15.7:0/301931191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9900d1470 0x7fc97c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98f7fe700 1 -- 172.21.15.7:0/301931191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97c003a60 msgr2=0x7fc97c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:56.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98f7fe700 1 --2- 172.21.15.7:0/301931191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97c003a60 0x7fc97c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.613+0000 7fc98f7fe700 1 -- 172.21.15.7:0/301931191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c003570 con 0x7fc97c004d50 2024-01-27T02:37:56.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.614+0000 7fc98f7fe700 1 --2- 172.21.15.7:0/301931191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc97c004d50 0x7fc97c0051c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc98800e210 tx=0x7fc98800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0f4d13fade45b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:56.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.614+0000 7fc98dffb700 1 -- 172.21.15.7:0/301931191 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98800ea60 con 0x7fc97c004d50 2024-01-27T02:37:56.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.614+0000 7fc98dffb700 1 -- 172.21.15.7:0/301931191 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98800f040 con 0x7fc97c004d50 2024-01-27T02:37:56.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.614+0000 7fc98dffb700 1 -- 172.21.15.7:0/301931191 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98800ee10 con 0x7fc97c004d50 2024-01-27T02:37:56.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.614+0000 7fc995869700 1 -- 172.21.15.7:0/301931191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc97c004d50 msgr2=0x7fc97c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:56.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.614+0000 7fc995869700 1 --2- 172.21.15.7:0/301931191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc97c004d50 0x7fc97c0051c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc98800e210 tx=0x7fc98800b040 comp rx=0 tx=0).stop 2024-01-27T02:37:56.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.614+0000 7fc98f7fe700 1 -- 172.21.15.7:0/301931191 reap_dead start 2024-01-27T02:37:56.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.614+0000 7fc995869700 1 -- 172.21.15.7:0/301931191 shutdown_connections 2024-01-27T02:37:56.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.614+0000 7fc995869700 1 -- 172.21.15.7:0/301931191 >> 172.21.15.7:0/301931191 conn(0x7fc9900679b0 msgr2=0x7fc990067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:56.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.614+0000 7fc995869700 1 -- 172.21.15.7:0/301931191 shutdown_connections 2024-01-27T02:37:56.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.615+0000 7fc995869700 1 -- 172.21.15.7:0/301931191 wait complete. 2024-01-27T02:37:56.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.616+0000 7fc995869700 1 Processor -- start 2024-01-27T02:37:56.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.616+0000 7fc995869700 1 -- start start 2024-01-27T02:37:56.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.617+0000 7fc995869700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97c004d50 0x7fc99015c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:56.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.617+0000 7fc995869700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc99016a200 0x7fc9900cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:56.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.617+0000 7fc995869700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99016a670 0x7fc9900cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:56.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.617+0000 7fc995869700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99015a9e0 con 0x7fc99016a200 2024-01-27T02:37:56.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.617+0000 7fc995869700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99015a0b0 con 0x7fc97c004d50 2024-01-27T02:37:56.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.617+0000 7fc995869700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99015a390 con 0x7fc99016a670 2024-01-27T02:37:56.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.617+0000 7fc98e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc99016a200 0x7fc9900cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:56.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.617+0000 7fc98e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc99016a200 0x7fc9900cc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44152/0 (socket says 172.21.15.7:44152) 2024-01-27T02:37:56.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.617+0000 7fc98e7fc700 1 -- 172.21.15.7:0/407943812 learned_addr learned my addr 172.21.15.7:0/407943812 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:56.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.617+0000 7fc98f7fe700 1 --2- 172.21.15.7:0/407943812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99016a670 0x7fc9900cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:56.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.617+0000 7fc98e7fc700 1 -- 172.21.15.7:0/407943812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99016a670 msgr2=0x7fc9900cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:56.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.617+0000 7fc98effd700 1 --2- 172.21.15.7:0/407943812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97c004d50 0x7fc99015c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:56.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.617+0000 7fc98e7fc700 1 --2- 172.21.15.7:0/407943812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99016a670 0x7fc9900cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.617+0000 7fc98e7fc700 1 -- 172.21.15.7:0/407943812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97c004d50 msgr2=0x7fc99015c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:56.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.617+0000 7fc98e7fc700 1 --2- 172.21.15.7:0/407943812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97c004d50 0x7fc99015c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.617+0000 7fc98e7fc700 1 -- 172.21.15.7:0/407943812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99015a670 con 0x7fc99016a200 2024-01-27T02:37:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.617+0000 7fc98f7fe700 1 --2- 172.21.15.7:0/407943812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99016a670 0x7fc9900cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.617+0000 7fc98effd700 1 --2- 172.21.15.7:0/407943812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97c004d50 0x7fc99015c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:37:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.617+0000 7fc98e7fc700 1 --2- 172.21.15.7:0/407943812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc99016a200 0x7fc9900cc970 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc978006ac0 tx=0x7fc9780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.617+0000 7fc986ffd700 1 -- 172.21.15.7:0/407943812 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97800fa10 con 0x7fc99016a200 2024-01-27T02:37:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.618+0000 7fc986ffd700 1 -- 172.21.15.7:0/407943812 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97800fb70 con 0x7fc99016a200 2024-01-27T02:37:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.618+0000 7fc995869700 1 -- 172.21.15.7:0/407943812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc99015cc00 con 0x7fc99016a200 2024-01-27T02:37:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.618+0000 7fc986ffd700 1 -- 172.21.15.7:0/407943812 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc978014440 con 0x7fc99016a200 2024-01-27T02:37:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.618+0000 7fc995869700 1 -- 172.21.15.7:0/407943812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99015b0b0 con 0x7fc99016a200 2024-01-27T02:37:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.618+0000 7fc995869700 1 -- 172.21.15.7:0/407943812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99004c710 con 0x7fc99016a200 2024-01-27T02:37:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.620+0000 7fc986ffd700 1 -- 172.21.15.7:0/407943812 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc978027480 con 0x7fc99016a200 2024-01-27T02:37:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.620+0000 7fc986ffd700 1 --2- 172.21.15.7:0/407943812 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc970066f30 0x7fc9700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.620+0000 7fc986ffd700 1 -- 172.21.15.7:0/407943812 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc978018030 con 0x7fc99016a200 2024-01-27T02:37:56.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.620+0000 7fc98effd700 1 --2- 172.21.15.7:0/407943812 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc970066f30 0x7fc9700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:56.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.621+0000 7fc98effd700 1 --2- 172.21.15.7:0/407943812 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc970066f30 0x7fc9700693e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc980001d80 tx=0x7fc980009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:56.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.624+0000 7fc986ffd700 1 -- 172.21.15.7:0/407943812 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc978052850 con 0x7fc99016a200 2024-01-27T02:37:56.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.825+0000 7fc995869700 1 -- 172.21.15.7:0/407943812 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc99015a220 con 0x7fc970066f30 2024-01-27T02:37:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.833+0000 7fc986ffd700 1 -- 172.21.15.7:0/407943812 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282556 (secure 0 0 0) 0x7fc99015a220 con 0x7fc970066f30 2024-01-27T02:37:56.837 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.836+0000 7fc995869700 1 -- 172.21.15.7:0/407943812 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc970066f30 msgr2=0x7fc9700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.836+0000 7fc995869700 1 --2- 172.21.15.7:0/407943812 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc970066f30 0x7fc9700693e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc980001d80 tx=0x7fc980009040 comp rx=0 tx=0).stop 2024-01-27T02:37:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.836+0000 7fc995869700 1 -- 172.21.15.7:0/407943812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc99016a200 msgr2=0x7fc9900cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.836+0000 7fc995869700 1 --2- 172.21.15.7:0/407943812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc99016a200 0x7fc9900cc970 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc978006ac0 tx=0x7fc9780091a0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.837+0000 7fc995869700 1 -- 172.21.15.7:0/407943812 shutdown_connections 2024-01-27T02:37:56.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.837+0000 7fc995869700 1 --2- 172.21.15.7:0/407943812 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc970066f30 0x7fc9700693e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.837+0000 7fc995869700 1 --2- 172.21.15.7:0/407943812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99016a670 0x7fc9900cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.837+0000 7fc995869700 1 --2- 172.21.15.7:0/407943812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97c004d50 0x7fc99015c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.837+0000 7fc995869700 1 --2- 172.21.15.7:0/407943812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc99016a200 0x7fc9900cc970 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:56.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.837+0000 7fc995869700 1 -- 172.21.15.7:0/407943812 >> 172.21.15.7:0/407943812 conn(0x7fc9900679b0 msgr2=0x7fc9901592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:56.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.837+0000 7fc995869700 1 -- 172.21.15.7:0/407943812 shutdown_connections 2024-01-27T02:37:56.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:56.837+0000 7fc995869700 1 -- 172.21.15.7:0/407943812 wait complete. 2024-01-27T02:37:56.840 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:37:56.851 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-01-27T02:37:55.658393+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6025,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556474,"statfs":{"total":1600000000,"available":1593829967,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.866207"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800705+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:29:06.789618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.811901+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:57.231078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800547+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:41:58.817551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.799051+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:17:03.370160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.808124+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:14:51.191272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801275+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:45:13.633104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800588+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:14:51.191272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.811924+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:04:59.885669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.805962+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:55.343516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.807734+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:55.343516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801974+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:28:34.028232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798839+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:26.529268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801629+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:55:18.331130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.809936+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:29:56.743131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798550+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:04:02.332379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801655+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:25:00.247752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.811458+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:46:44.234857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798522+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:14:51.191272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.809517+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:07:53.995321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.799143+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:12:54.826584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.806764+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:19:48.163385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.811035+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:33:08.282783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.811487+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:10:19.964021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800572+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:55.343516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.807163+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:33:14.691144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.807469+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:17:03.370160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800915+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:17:03.370160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.807224+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:24:13.096584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.808167+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:45:13.633104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.812972+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:50:16.532574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798553+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:55.343516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801905+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:23:06.453385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.809151+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:45:52.881907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.812131+0000","last_change":"2024-01-27T02:37:50.812131+0000","last_active":"2024-01-27T02:37:50.812131+0000","last_peered":"2024-01-27T02:37:50.812131+0000","last_clean":"2024-01-27T02:37:50.812131+0000","last_became_active":"2024-01-27T02:37:50.809514+0000","last_became_peered":"2024-01-27T02:37:50.809514+0000","last_unstale":"2024-01-27T02:37:50.812131+0000","last_undegraded":"2024-01-27T02:37:50.812131+0000","last_fullsized":"2024-01-27T02:37:50.812131+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:14:50.186256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.797941+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:24:13.096584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.808649+0000","last_change":"2024-01-27T02:37:50.808649+0000","last_active":"2024-01-27T02:37:50.808649+0000","last_peered":"2024-01-27T02:37:50.808649+0000","last_clean":"2024-01-27T02:37:50.808649+0000","last_became_active":"2024-01-27T02:37:50.808412+0000","last_became_peered":"2024-01-27T02:37:50.808412+0000","last_unstale":"2024-01-27T02:37:50.808649+0000","last_undegraded":"2024-01-27T02:37:50.808649+0000","last_fullsized":"2024-01-27T02:37:50.808649+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:24:12.091569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800223+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:10:27.309356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.806041+0000","last_change":"2024-01-27T02:37:50.806041+0000","last_active":"2024-01-27T02:37:50.806041+0000","last_peered":"2024-01-27T02:37:50.806041+0000","last_clean":"2024-01-27T02:37:50.806041+0000","last_became_active":"2024-01-27T02:37:50.805760+0000","last_became_peered":"2024-01-27T02:37:50.805760+0000","last_unstale":"2024-01-27T02:37:50.806041+0000","last_undegraded":"2024-01-27T02:37:50.806041+0000","last_fullsized":"2024-01-27T02:37:50.806041+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:31:21.182217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798399+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:57:57.067686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.808048+0000","last_change":"2024-01-27T02:37:50.808048+0000","last_active":"2024-01-27T02:37:50.808048+0000","last_peered":"2024-01-27T02:37:50.808048+0000","last_clean":"2024-01-27T02:37:50.808048+0000","last_became_active":"2024-01-27T02:37:50.807771+0000","last_became_peered":"2024-01-27T02:37:50.807771+0000","last_unstale":"2024-01-27T02:37:50.808048+0000","last_undegraded":"2024-01-27T02:37:50.808048+0000","last_fullsized":"2024-01-27T02:37:50.808048+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:24.808914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800346+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:26.529268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.806436+0000","last_change":"2024-01-27T02:37:50.806436+0000","last_active":"2024-01-27T02:37:50.806436+0000","last_peered":"2024-01-27T02:37:50.806436+0000","last_clean":"2024-01-27T02:37:50.806436+0000","last_became_active":"2024-01-27T02:37:50.806167+0000","last_became_peered":"2024-01-27T02:37:50.806167+0000","last_unstale":"2024-01-27T02:37:50.806436+0000","last_undegraded":"2024-01-27T02:37:50.806436+0000","last_fullsized":"2024-01-27T02:37:50.806436+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:58:12.533491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800174+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:25.813930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.812296+0000","last_change":"2024-01-27T02:37:50.812296+0000","last_active":"2024-01-27T02:37:50.812296+0000","last_peered":"2024-01-27T02:37:50.812296+0000","last_clean":"2024-01-27T02:37:50.812296+0000","last_became_active":"2024-01-27T02:37:50.809825+0000","last_became_peered":"2024-01-27T02:37:50.809825+0000","last_unstale":"2024-01-27T02:37:50.812296+0000","last_undegraded":"2024-01-27T02:37:50.812296+0000","last_fullsized":"2024-01-27T02:37:50.812296+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:54.338500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798046+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:33:14.691144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.804462+0000","last_change":"2024-01-27T02:37:50.804462+0000","last_active":"2024-01-27T02:37:50.804462+0000","last_peered":"2024-01-27T02:37:50.804462+0000","last_clean":"2024-01-27T02:37:50.804462+0000","last_became_active":"2024-01-27T02:37:50.804211+0000","last_became_peered":"2024-01-27T02:37:50.804211+0000","last_unstale":"2024-01-27T02:37:50.804462+0000","last_undegraded":"2024-01-27T02:37:50.804462+0000","last_fullsized":"2024-01-27T02:37:50.804462+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:16:34.794337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798801+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:16:35.799353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.804294+0000","last_change":"2024-01-27T02:37:50.804294+0000","last_active":"2024-01-27T02:37:50.804294+0000","last_peered":"2024-01-27T02:37:50.804294+0000","last_clean":"2024-01-27T02:37:50.804294+0000","last_became_active":"2024-01-27T02:37:50.803789+0000","last_became_peered":"2024-01-27T02:37:50.803789+0000","last_unstale":"2024-01-27T02:37:50.804294+0000","last_undegraded":"2024-01-27T02:37:50.804294+0000","last_fullsized":"2024-01-27T02:37:50.804294+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:37:47.399101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.808533+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:55:18.331130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.807055+0000","last_change":"2024-01-27T02:37:50.807055+0000","last_active":"2024-01-27T02:37:50.807055+0000","last_peered":"2024-01-27T02:37:50.807055+0000","last_clean":"2024-01-27T02:37:50.807055+0000","last_became_active":"2024-01-27T02:37:50.806246+0000","last_became_peered":"2024-01-27T02:37:50.806246+0000","last_unstale":"2024-01-27T02:37:50.807055+0000","last_undegraded":"2024-01-27T02:37:50.807055+0000","last_fullsized":"2024-01-27T02:37:50.807055+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:23.160028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798337+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:54:58.004832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.805856+0000","last_change":"2024-01-27T02:37:50.805856+0000","last_active":"2024-01-27T02:37:50.805856+0000","last_peered":"2024-01-27T02:37:50.805856+0000","last_clean":"2024-01-27T02:37:50.805856+0000","last_became_active":"2024-01-27T02:37:50.805568+0000","last_became_peered":"2024-01-27T02:37:50.805568+0000","last_unstale":"2024-01-27T02:37:50.805856+0000","last_undegraded":"2024-01-27T02:37:50.805856+0000","last_fullsized":"2024-01-27T02:37:50.805856+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:11:46.349861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.808509+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:45:13.633104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.809623+0000","last_change":"2024-01-27T02:37:50.809623+0000","last_active":"2024-01-27T02:37:50.809623+0000","last_peered":"2024-01-27T02:37:50.809623+0000","last_clean":"2024-01-27T02:37:50.809623+0000","last_became_active":"2024-01-27T02:37:50.809447+0000","last_became_peered":"2024-01-27T02:37:50.809447+0000","last_unstale":"2024-01-27T02:37:50.809623+0000","last_undegraded":"2024-01-27T02:37:50.809623+0000","last_fullsized":"2024-01-27T02:37:50.809623+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:50.938489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801315+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:55:18.331130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.814137+0000","last_change":"2024-01-27T02:37:50.814137+0000","last_active":"2024-01-27T02:37:50.814137+0000","last_peered":"2024-01-27T02:37:50.814137+0000","last_clean":"2024-01-27T02:37:50.814137+0000","last_became_active":"2024-01-27T02:37:50.813898+0000","last_became_peered":"2024-01-27T02:37:50.813898+0000","last_unstale":"2024-01-27T02:37:50.814137+0000","last_undegraded":"2024-01-27T02:37:50.814137+0000","last_fullsized":"2024-01-27T02:37:50.814137+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:56:37.175345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.798050+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:25.813930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.813470+0000","last_change":"2024-01-27T02:37:50.813470+0000","last_active":"2024-01-27T02:37:50.813470+0000","last_peered":"2024-01-27T02:37:50.813470+0000","last_clean":"2024-01-27T02:37:50.813470+0000","last_became_active":"2024-01-27T02:37:50.812014+0000","last_became_peered":"2024-01-27T02:37:50.812014+0000","last_unstale":"2024-01-27T02:37:50.813470+0000","last_undegraded":"2024-01-27T02:37:50.813470+0000","last_fullsized":"2024-01-27T02:37:50.813470+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:56:29.266154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801084+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:53:55.703491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.807923+0000","last_change":"2024-01-27T02:37:50.807923+0000","last_active":"2024-01-27T02:37:50.807923+0000","last_peered":"2024-01-27T02:37:50.807923+0000","last_clean":"2024-01-27T02:37:50.807923+0000","last_became_active":"2024-01-27T02:37:50.807417+0000","last_became_peered":"2024-01-27T02:37:50.807417+0000","last_unstale":"2024-01-27T02:37:50.807923+0000","last_undegraded":"2024-01-27T02:37:50.807923+0000","last_fullsized":"2024-01-27T02:37:50.807923+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:26:35.727576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.797951+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:01:25.391647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.809421+0000","last_change":"2024-01-27T02:37:50.809421+0000","last_active":"2024-01-27T02:37:50.809421+0000","last_peered":"2024-01-27T02:37:50.809421+0000","last_clean":"2024-01-27T02:37:50.809421+0000","last_became_active":"2024-01-27T02:37:50.809169+0000","last_became_peered":"2024-01-27T02:37:50.809169+0000","last_unstale":"2024-01-27T02:37:50.809421+0000","last_undegraded":"2024-01-27T02:37:50.809421+0000","last_fullsized":"2024-01-27T02:37:50.809421+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:31:21.182217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.806904+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:46:21.029594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.810368+0000","last_change":"2024-01-27T02:37:50.810368+0000","last_active":"2024-01-27T02:37:50.810368+0000","last_peered":"2024-01-27T02:37:50.810368+0000","last_clean":"2024-01-27T02:37:50.810368+0000","last_became_active":"2024-01-27T02:37:50.810089+0000","last_became_peered":"2024-01-27T02:37:50.810089+0000","last_unstale":"2024-01-27T02:37:50.810368+0000","last_undegraded":"2024-01-27T02:37:50.810368+0000","last_fullsized":"2024-01-27T02:37:50.810368+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:50:15.527559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.807362+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:25.813930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.803536+0000","last_change":"2024-01-27T02:37:50.803536+0000","last_active":"2024-01-27T02:37:50.803536+0000","last_peered":"2024-01-27T02:37:50.803536+0000","last_clean":"2024-01-27T02:37:50.803536+0000","last_became_active":"2024-01-27T02:37:50.803332+0000","last_became_peered":"2024-01-27T02:37:50.803332+0000","last_unstale":"2024-01-27T02:37:50.803536+0000","last_undegraded":"2024-01-27T02:37:50.803536+0000","last_fullsized":"2024-01-27T02:37:50.803536+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:05:27.161024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.799402+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:57:57.067686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.802000+0000","last_change":"2024-01-27T02:37:50.802000+0000","last_active":"2024-01-27T02:37:50.802000+0000","last_peered":"2024-01-27T02:37:50.802000+0000","last_clean":"2024-01-27T02:37:50.802000+0000","last_became_active":"2024-01-27T02:37:50.801755+0000","last_became_peered":"2024-01-27T02:37:50.801755+0000","last_unstale":"2024-01-27T02:37:50.802000+0000","last_undegraded":"2024-01-27T02:37:50.802000+0000","last_fullsized":"2024-01-27T02:37:50.802000+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:54:56.999817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800829+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:04:02.332379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.812532+0000","last_change":"2024-01-27T02:37:50.812532+0000","last_active":"2024-01-27T02:37:50.812532+0000","last_peered":"2024-01-27T02:37:50.812532+0000","last_clean":"2024-01-27T02:37:50.812532+0000","last_became_active":"2024-01-27T02:37:50.812209+0000","last_became_peered":"2024-01-27T02:37:50.812209+0000","last_unstale":"2024-01-27T02:37:50.812532+0000","last_undegraded":"2024-01-27T02:37:50.812532+0000","last_fullsized":"2024-01-27T02:37:50.812532+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:11:46.349861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.808739+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:47:12.515208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.813813+0000","last_change":"2024-01-27T02:37:50.813813+0000","last_active":"2024-01-27T02:37:50.813813+0000","last_peered":"2024-01-27T02:37:50.813813+0000","last_clean":"2024-01-27T02:37:50.813813+0000","last_became_active":"2024-01-27T02:37:50.813235+0000","last_became_peered":"2024-01-27T02:37:50.813235+0000","last_unstale":"2024-01-27T02:37:50.813813+0000","last_undegraded":"2024-01-27T02:37:50.813813+0000","last_fullsized":"2024-01-27T02:37:50.813813+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:33:13.686128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.807809+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:14:52.244572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.804058+0000","last_change":"2024-01-27T02:37:50.804058+0000","last_active":"2024-01-27T02:37:50.804058+0000","last_peered":"2024-01-27T02:37:50.804058+0000","last_clean":"2024-01-27T02:37:50.804058+0000","last_became_active":"2024-01-27T02:37:50.803802+0000","last_became_peered":"2024-01-27T02:37:50.803802+0000","last_unstale":"2024-01-27T02:37:50.804058+0000","last_undegraded":"2024-01-27T02:37:50.804058+0000","last_fullsized":"2024-01-27T02:37:50.804058+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:01:24.386631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.806271+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:19:50.489258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.806415+0000","last_change":"2024-01-27T02:37:50.806415+0000","last_active":"2024-01-27T02:37:50.806415+0000","last_peered":"2024-01-27T02:37:50.806415+0000","last_clean":"2024-01-27T02:37:50.806415+0000","last_became_active":"2024-01-27T02:37:50.806149+0000","last_became_peered":"2024-01-27T02:37:50.806149+0000","last_unstale":"2024-01-27T02:37:50.806415+0000","last_undegraded":"2024-01-27T02:37:50.806415+0000","last_fullsized":"2024-01-27T02:37:50.806415+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:04:39.266345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.806325+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:28.200720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.809566+0000","last_change":"2024-01-27T02:37:50.809566+0000","last_active":"2024-01-27T02:37:50.809566+0000","last_peered":"2024-01-27T02:37:50.809566+0000","last_clean":"2024-01-27T02:37:50.809566+0000","last_became_active":"2024-01-27T02:37:50.809276+0000","last_became_peered":"2024-01-27T02:37:50.809276+0000","last_unstale":"2024-01-27T02:37:50.809566+0000","last_undegraded":"2024-01-27T02:37:50.809566+0000","last_fullsized":"2024-01-27T02:37:50.809566+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:24:12.091569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.811076+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:18:23.945261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.805793+0000","last_change":"2024-01-27T02:37:50.805793+0000","last_active":"2024-01-27T02:37:50.805793+0000","last_peered":"2024-01-27T02:37:50.805793+0000","last_clean":"2024-01-27T02:37:50.805793+0000","last_became_active":"2024-01-27T02:37:50.805524+0000","last_became_peered":"2024-01-27T02:37:50.805524+0000","last_unstale":"2024-01-27T02:37:50.805793+0000","last_undegraded":"2024-01-27T02:37:50.805793+0000","last_fullsized":"2024-01-27T02:37:50.805793+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:57:33.113268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.801404+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:17:37.405535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.814412+0000","last_change":"2024-01-27T02:37:50.814412+0000","last_active":"2024-01-27T02:37:50.814412+0000","last_peered":"2024-01-27T02:37:50.814412+0000","last_clean":"2024-01-27T02:37:50.814412+0000","last_became_active":"2024-01-27T02:37:50.813560+0000","last_became_peered":"2024-01-27T02:37:50.813560+0000","last_unstale":"2024-01-27T02:37:50.814412+0000","last_undegraded":"2024-01-27T02:37:50.814412+0000","last_fullsized":"2024-01-27T02:37:50.814412+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:27.195705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.800617+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:08:13.232740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.808731+0000","last_change":"2024-01-27T02:37:50.808731+0000","last_active":"2024-01-27T02:37:50.808731+0000","last_peered":"2024-01-27T02:37:50.808731+0000","last_clean":"2024-01-27T02:37:50.808731+0000","last_became_active":"2024-01-27T02:37:50.808482+0000","last_became_peered":"2024-01-27T02:37:50.808482+0000","last_unstale":"2024-01-27T02:37:50.808731+0000","last_undegraded":"2024-01-27T02:37:50.808731+0000","last_fullsized":"2024-01-27T02:37:50.808731+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:50.938489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.806197+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:11:47.354877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.808670+0000","last_change":"2024-01-27T02:37:50.808670+0000","last_active":"2024-01-27T02:37:50.808670+0000","last_peered":"2024-01-27T02:37:50.808670+0000","last_clean":"2024-01-27T02:37:50.808670+0000","last_became_active":"2024-01-27T02:37:50.808428+0000","last_became_peered":"2024-01-27T02:37:50.808428+0000","last_unstale":"2024-01-27T02:37:50.808670+0000","last_undegraded":"2024-01-27T02:37:50.808670+0000","last_fullsized":"2024-01-27T02:37:50.808670+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:57:56.062671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.806456+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:07:01.540761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.813250+0000","last_change":"2024-01-27T02:37:50.813250+0000","last_active":"2024-01-27T02:37:50.813250+0000","last_peered":"2024-01-27T02:37:50.813250+0000","last_clean":"2024-01-27T02:37:50.813250+0000","last_became_active":"2024-01-27T02:37:50.813038+0000","last_became_peered":"2024-01-27T02:37:50.813038+0000","last_unstale":"2024-01-27T02:37:50.813250+0000","last_undegraded":"2024-01-27T02:37:50.813250+0000","last_fullsized":"2024-01-27T02:37:50.813250+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:25.524253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.797951+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:57:34.118284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.803902+0000","last_change":"2024-01-27T02:37:50.803902+0000","last_active":"2024-01-27T02:37:50.803902+0000","last_peered":"2024-01-27T02:37:50.803902+0000","last_clean":"2024-01-27T02:37:50.803902+0000","last_became_active":"2024-01-27T02:37:50.803626+0000","last_became_peered":"2024-01-27T02:37:50.803626+0000","last_unstale":"2024-01-27T02:37:50.803902+0000","last_undegraded":"2024-01-27T02:37:50.803902+0000","last_fullsized":"2024-01-27T02:37:50.803902+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:25.524253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.808334+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:33:48.901396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.804116+0000","last_change":"2024-01-27T02:37:50.804116+0000","last_active":"2024-01-27T02:37:50.804116+0000","last_peered":"2024-01-27T02:37:50.804116+0000","last_clean":"2024-01-27T02:37:50.804116+0000","last_became_active":"2024-01-27T02:37:50.803880+0000","last_became_peered":"2024-01-27T02:37:50.803880+0000","last_unstale":"2024-01-27T02:37:50.804116+0000","last_undegraded":"2024-01-27T02:37:50.804116+0000","last_fullsized":"2024-01-27T02:37:50.804116+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:50.938489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.806238+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:51.943504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.807085+0000","last_change":"2024-01-27T02:37:50.807085+0000","last_active":"2024-01-27T02:37:50.807085+0000","last_peered":"2024-01-27T02:37:50.807085+0000","last_clean":"2024-01-27T02:37:50.807085+0000","last_became_active":"2024-01-27T02:37:50.804683+0000","last_became_peered":"2024-01-27T02:37:50.804683+0000","last_unstale":"2024-01-27T02:37:50.807085+0000","last_undegraded":"2024-01-27T02:37:50.807085+0000","last_fullsized":"2024-01-27T02:37:50.807085+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:49:46.671831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:37:50.790887+0000","last_change":"2024-01-27T02:37:50.812579+0000","last_active":"2024-01-27T02:37:50.790887+0000","last_peered":"2024-01-27T02:37:50.790887+0000","last_clean":"2024-01-27T02:37:50.790887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:37:50.790887+0000","last_undegraded":"2024-01-27T02:37:50.790887+0000","last_fullsized":"2024-01-27T02:37:50.790887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:11:16.205143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":8,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-27T02:37:50.809913+0000","last_change":"2024-01-27T02:37:50.809913+0000","last_active":"2024-01-27T02:37:50.809913+0000","last_peered":"2024-01-27T02:37:50.809913+0000","last_clean":"2024-01-27T02:37:50.809913+0000","last_became_active":"2024-01-27T02:37:50.809679+0000","last_became_peered":"2024-01-27T02:37:50.809679+0000","last_unstale":"2024-01-27T02:37:50.809913+0000","last_undegraded":"2024-01-27T02:37:50.809913+0000","last_fullsized":"2024-01-27T02:37:50.809913+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:27.195705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":245,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-27T02:37:49.795519+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:37:49.795519+0000","last_peered":"2024-01-27T02:37:49.795519+0000","last_clean":"2024-01-27T02:37:49.795519+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:37:49.795519+0000","last_undegraded":"2024-01-27T02:37:49.795519+0000","last_fullsized":"2024-01-27T02:37:49.795519+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574921,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1186,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194126,"statfs":{"total":200000000,"available":198785187,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71299999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.59099999999999997,"15min":0.58999999999999997},"min":{"1min":0.49099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77400000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.64000000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59699999999999998,"15min":0.59799999999999998},"min":{"1min":0.46100000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.44400000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.753,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.53600000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58299999999999996},"min":{"1min":0.432,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.439},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77700000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.65500000000000003}]},{"osd":3,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60499999999999998,"15min":0.59299999999999997},"min":{"1min":0.495,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.60999999999999999,"15min":0.59499999999999997},"min":{"1min":0.5,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85099999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.71799999999999997}]},{"osd":4,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.47499999999999998,"15min":0.48699999999999999},"min":{"1min":0.32400000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63300000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.495,"15min":0.51300000000000001},"min":{"1min":0.35999999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68899999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.80000000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51800000000000002,"15min":0.53600000000000003},"min":{"1min":0.39000000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.56399999999999995},"min":{"1min":0.40100000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.75800000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.59499999999999997},"min":{"1min":0.40999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.751,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.57699999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.81599999999999995,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.73099999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":603,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194708,"statfs":{"total":200000000,"available":199381818,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58699999999999997,"15min":0.52100000000000002},"min":{"1min":0.45000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.52700000000000002},"min":{"1min":0.434,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71899999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.79900000000000004}]},{"osd":1,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.51800000000000002},"min":{"1min":0.41999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79700000000000004,"5min":0.874,"15min":0.874},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.60599999999999998,"15min":0.55500000000000005},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.54000000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.41199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.625,"15min":0.58299999999999996},"min":{"1min":0.45100000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79700000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59399999999999997}]},{"osd":3,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58199999999999996,"15min":0.55100000000000005},"min":{"1min":0.47599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.76800000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61499999999999999,"15min":0.57299999999999995},"min":{"1min":0.48499999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.78100000000000003,"5min":1.262,"15min":1.262},"last":0.57099999999999995}]},{"osd":4,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.34200000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52700000000000002,"15min":0.55300000000000005},"min":{"1min":0.41199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68300000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.72699999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53200000000000003,"15min":0.53500000000000003},"min":{"1min":0.38800000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70799999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.755},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.60599999999999998},"min":{"1min":0.40899999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70799999999999996,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.63300000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60599999999999998,"15min":0.61599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73699999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71099999999999997,"5min":1.038,"15min":1.038},"last":0.60999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":609,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194702,"statfs":{"total":200000000,"available":199375555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39700000000000002,"15min":0.377},"min":{"1min":0.27000000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.41999999999999998,"15min":0.40000000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.374}]},{"osd":2,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.40200000000000002,"15min":0.36599999999999999},"min":{"1min":0.28000000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45400000000000001,"15min":0.43099999999999999},"min":{"1min":0.30199999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.39500000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48999999999999999,"15min":0.46899999999999997},"min":{"1min":0.35799999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.434},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.47499999999999998,"15min":0.45800000000000002},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.314}]},{"osd":4,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.66200000000000003,"15min":0.63400000000000001},"min":{"1min":0.442,"5min":0.373,"15min":0.373},"max":{"1min":3.2890000000000001,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65800000000000003,"15min":0.63600000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.59299999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.68500000000000005,"15min":0.63700000000000001},"min":{"1min":0.50600000000000001,"5min":0.44,"15min":0.44},"max":{"1min":3.4500000000000002,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.68999999999999995,"15min":0.67600000000000005},"min":{"1min":0.48299999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.181,"5min":3.181,"15min":3.181},"last":0.748}]},{"osd":6,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70999999999999996,"15min":0.69599999999999995},"min":{"1min":0.52100000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96399999999999997,"5min":1.024,"15min":1.024},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.71299999999999997,"15min":0.67900000000000005},"min":{"1min":0.53400000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.79900000000000004}]},{"osd":7,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.72399999999999998,"15min":0.68600000000000005},"min":{"1min":0.56100000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.72499999999999998,"15min":0.69199999999999995},"min":{"1min":0.57799999999999996,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.70599999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574921,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1186,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194126,"statfs":{"total":200000000,"available":198785187,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.38900000000000001},"min":{"1min":0.28100000000000003,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.66600000000000004,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27800000000000002,"5min":0.223,"15min":0.223},"max":{"1min":0.65200000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.39400000000000002}]},{"osd":2,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.436,"15min":0.42199999999999999},"min":{"1min":0.32800000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56299999999999994,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.441},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.44900000000000001,"15min":0.435},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58999999999999997,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.628}]},{"osd":3,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42999999999999999,"15min":0.432},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.63600000000000001,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.35299999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.63200000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.36299999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.65500000000000003,"15min":0.64000000000000001},"min":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.501},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45900000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89700000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.59799999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67500000000000004,"15min":0.65600000000000003},"min":{"1min":0.48399999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.86499999999999999,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67700000000000005,"15min":0.68200000000000005},"min":{"1min":0.5,"5min":0.44,"15min":0.44},"max":{"1min":0.96599999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.53700000000000003}]},{"osd":6,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.56200000000000006,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91300000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69299999999999995,"15min":0.67900000000000005},"min":{"1min":0.56599999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94199999999999995,"5min":1.04,"15min":1.04},"last":0.55000000000000004}]},{"osd":7,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.55300000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.52100000000000002,"5min":0.498,"15min":0.498},"max":{"1min":0.95799999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":609,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194702,"statfs":{"total":200000000,"available":199375555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.41699999999999998,"15min":0.40999999999999998},"min":{"1min":0.28000000000000003,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56599999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.50900000000000001,"15min":0.63700000000000001},"min":{"1min":0.34499999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72799999999999998,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":0.48199999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.434,"15min":0.42399999999999999},"min":{"1min":0.32700000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.313},{"interface":"front","average":{"1min":0.434,"5min":0.44,"15min":0.44700000000000001},"min":{"1min":0.29799999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.65300000000000002,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.42699999999999999}]},{"osd":3,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.41499999999999998},"min":{"1min":0.30399999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.65000000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76500000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.44600000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68200000000000005,"15min":0.65600000000000003},"min":{"1min":0.53600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73399999999999999,"15min":0.79900000000000004},"min":{"1min":0.54900000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.995,"5min":3.581,"15min":3.581},"last":0.68999999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68000000000000005,"15min":0.74399999999999999},"min":{"1min":0.502,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90500000000000003,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67700000000000005,"15min":0.67100000000000004},"min":{"1min":0.501,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85899999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.54800000000000004}]},{"osd":6,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.53800000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93999999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0189999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.63800000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.53600000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70299999999999996,"15min":0.69099999999999995},"min":{"1min":0.625,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":609,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194702,"statfs":{"total":200000000,"available":199375555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.38800000000000001,"15min":0.35899999999999999},"min":{"1min":0.255,"5min":0.253,"15min":0.253},"max":{"1min":0.56200000000000006,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.432},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40600000000000003,"15min":0.39500000000000002},"min":{"1min":0.29199999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.626,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.30499999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.27400000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66300000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.45600000000000002,"15min":0.42399999999999999},"min":{"1min":0.30199999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.32400000000000001}]},{"osd":2,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46100000000000002,"15min":0.43099999999999999},"min":{"1min":0.33700000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.48399999999999999,"15min":0.45000000000000001},"min":{"1min":0.378,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.375}]},{"osd":4,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.66900000000000004,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.6660000000000004,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64200000000000002,"15min":0.60599999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93300000000000005,"5min":1.012,"15min":1.012},"last":0.68400000000000005}]},{"osd":5,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.61299999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90400000000000003,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.71599999999999997,"15min":0.68000000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":4.6849999999999996,"5min":4.6849999999999996,"15min":4.6849999999999996},"last":0.69699999999999995}]},{"osd":6,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90100000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.47099999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.71199999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.71699999999999997,"15min":0.65400000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":4.7130000000000001,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68500000000000005,"15min":0.66700000000000004},"min":{"1min":0.495,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.73399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574921,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":609,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194702,"statfs":{"total":200000000,"available":199375555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.36899999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.73399999999999999,"5min":1.099,"15min":1.099},"last":1.3839999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.47799999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71099999999999997,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":7.0670000000000002}]},{"osd":1,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.39500000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72499999999999998,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":5.1529999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59799999999999998,"15min":0.59499999999999997},"min":{"1min":0.40799999999999997,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77700000000000002,"5min":1.133,"15min":1.133},"last":3.0419999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76300000000000001,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":5.0019999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.44700000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.753,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":2.073}]},{"osd":3,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.42799999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68700000000000006,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":6.7220000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.50700000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.70799999999999996,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":6.5510000000000002}]},{"osd":5,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.47599999999999998,"15min":0.48399999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.63400000000000001,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.34599999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68899999999999995,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":6.9269999999999996}]},{"osd":6,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.5},"min":{"1min":0.40100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.66600000000000004,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":5.2229999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53800000000000003,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81200000000000006,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":1.46}]},{"osd":7,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53900000000000003,"15min":0.54900000000000004},"min":{"1min":0.38500000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.65400000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":7.5339999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.432,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68100000000000005,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":4.9550000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":609,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194702,"statfs":{"total":200000000,"available":199375555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56999999999999995,"15min":0.59399999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68700000000000006,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.60099999999999998,"15min":0.57699999999999996},"min":{"1min":0.34499999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67100000000000004,"5min":3.242,"15min":3.242},"last":0.48699999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.625,"15min":0.60799999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70599999999999996,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.626,"15min":0.64900000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69299999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.72099999999999997}]},{"osd":2,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61599999999999999,"15min":0.59599999999999997},"min":{"1min":0.39100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.71499999999999997,"5min":3.04,"15min":3.04},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59399999999999997,"15min":0.57699999999999996},"min":{"1min":0.40100000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72499999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.53500000000000003}]},{"osd":3,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.66500000000000004,"15min":0.67800000000000005},"min":{"1min":0.44400000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.65500000000000003,"5min":4.032,"15min":4.032},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.70399999999999996,"15min":0.67800000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.77000000000000002,"5min":4.282,"15min":4.282},"last":0.63800000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81399999999999995,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.51100000000000001,"15min":0.56499999999999995},"min":{"1min":0.36799999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.61499999999999999,"5min":1.101,"15min":1.101},"last":0.76300000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51300000000000001,"15min":0.55600000000000005},"min":{"1min":0.36499999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.629,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56499999999999995,"15min":0.58199999999999996},"min":{"1min":0.39100000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76800000000000002,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.74099999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.55400000000000005,"15min":0.56699999999999995},"min":{"1min":0.40200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.745,"5min":1.008,"15min":1.008},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.626,"15min":0.65800000000000003},"min":{"1min":0.38400000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84799999999999998,"5min":2,"15min":2},"last":0.70799999999999996}]}]}],"pool_statfs":[]}} 2024-01-27T02:37:56.855 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3f is not active+clean 2024-01-27T02:37:56.856 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.800705+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T06:29:06.789618+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:37:56.856 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3e is not active+clean 2024-01-27T02:37:56.856 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.811901+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:15:57.231078+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:37:56.856 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3d is not active+clean 2024-01-27T02:37:56.856 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.800547+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:41:58.817551+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:37:56.856 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3c is not active+clean 2024-01-27T02:37:56.856 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.799051+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:17:03.370160+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:37:56.856 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3b is not active+clean 2024-01-27T02:37:56.856 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.808124+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T12:14:51.191272+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:37:56.856 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3a is not active+clean 2024-01-27T02:37:56.857 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.801275+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T02:45:13.633104+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:37:56.857 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.39 is not active+clean 2024-01-27T02:37:56.857 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.800588+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T12:14:51.191272+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:37:56.857 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.38 is not active+clean 2024-01-27T02:37:56.857 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.811924+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T05:04:59.885669+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:37:56.857 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.37 is not active+clean 2024-01-27T02:37:56.857 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.805962+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T14:12:55.343516+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:37:56.857 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.36 is not active+clean 2024-01-27T02:37:56.857 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.807734+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T14:12:55.343516+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:37:56.857 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.35 is not active+clean 2024-01-27T02:37:56.857 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.801974+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:28:34.028232+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:37:56.858 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.34 is not active+clean 2024-01-27T02:37:56.858 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.798839+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:39:26.529268+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:37:56.858 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.33 is not active+clean 2024-01-27T02:37:56.858 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.801629+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T09:55:18.331130+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:37:56.858 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.32 is not active+clean 2024-01-27T02:37:56.858 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.809936+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T05:29:56.743131+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:37:56.858 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.31 is not active+clean 2024-01-27T02:37:56.858 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.798550+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T06:04:02.332379+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:37:56.858 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.30 is not active+clean 2024-01-27T02:37:56.858 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.801655+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T14:25:00.247752+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:37:56.859 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2f is not active+clean 2024-01-27T02:37:56.859 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.811458+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T13:46:44.234857+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:37:56.859 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2e is not active+clean 2024-01-27T02:37:56.859 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.798522+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T12:14:51.191272+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:37:56.859 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2d is not active+clean 2024-01-27T02:37:56.859 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.809517+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T14:07:53.995321+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:37:56.859 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2c is not active+clean 2024-01-27T02:37:56.859 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.799143+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T12:12:54.826584+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:37:56.859 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2b is not active+clean 2024-01-27T02:37:56.859 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.806764+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T14:19:48.163385+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:37:56.859 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2a is not active+clean 2024-01-27T02:37:56.860 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.811035+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T14:33:08.282783+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:37:56.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.29 is not active+clean 2024-01-27T02:37:56.860 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.811487+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T03:10:19.964021+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:37:56.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.28 is not active+clean 2024-01-27T02:37:56.860 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.800572+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T14:12:55.343516+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:37:56.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.27 is not active+clean 2024-01-27T02:37:56.860 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.807163+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:33:14.691144+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:37:56.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.26 is not active+clean 2024-01-27T02:37:56.860 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.807469+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:17:03.370160+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:37:56.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.25 is not active+clean 2024-01-27T02:37:56.860 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.800915+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:17:03.370160+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:37:56.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.24 is not active+clean 2024-01-27T02:37:56.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.807224+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:24:13.096584+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:37:56.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.23 is not active+clean 2024-01-27T02:37:56.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.808167+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T02:45:13.633104+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:37:56.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.22 is not active+clean 2024-01-27T02:37:56.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.812972+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:50:16.532574+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:37:56.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.21 is not active+clean 2024-01-27T02:37:56.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.798553+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T14:12:55.343516+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:37:56.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.20 is not active+clean 2024-01-27T02:37:56.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.801905+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:23:06.453385+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:37:56.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1e is not active+clean 2024-01-27T02:37:56.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.809151+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T13:45:52.881907+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:37:56.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1f is not active+clean 2024-01-27T02:37:56.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.797941+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:24:13.096584+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:37:56.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1c is not active+clean 2024-01-27T02:37:56.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.800223+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T14:10:27.309356+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:37:56.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1d is not active+clean 2024-01-27T02:37:56.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.798399+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T09:57:57.067686+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:37:56.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1a is not active+clean 2024-01-27T02:37:56.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.800346+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:39:26.529268+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:37:56.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1b is not active+clean 2024-01-27T02:37:56.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.800174+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T06:31:25.813930+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:37:56.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.18 is not active+clean 2024-01-27T02:37:56.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.798046+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:33:14.691144+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:37:56.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.19 is not active+clean 2024-01-27T02:37:56.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.798801+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:16:35.799353+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:37:56.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.16 is not active+clean 2024-01-27T02:37:56.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.808533+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T09:55:18.331130+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:37:56.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.17 is not active+clean 2024-01-27T02:37:56.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.798337+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T12:54:58.004832+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:37:56.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.14 is not active+clean 2024-01-27T02:37:56.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.808509+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T02:45:13.633104+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:37:56.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.15 is not active+clean 2024-01-27T02:37:56.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.801315+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T09:55:18.331130+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:37:56.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.12 is not active+clean 2024-01-27T02:37:56.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.798050+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T06:31:25.813930+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:37:56.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.13 is not active+clean 2024-01-27T02:37:56.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.801084+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T06:53:55.703491+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:37:56.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.10 is not active+clean 2024-01-27T02:37:56.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.797951+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T13:01:25.391647+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:37:56.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.11 is not active+clean 2024-01-27T02:37:56.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.806904+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T06:46:21.029594+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:37:56.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.f is not active+clean 2024-01-27T02:37:56.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.807362+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T06:31:25.813930+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:37:56.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.c is not active+clean 2024-01-27T02:37:56.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.799402+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T09:57:57.067686+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:37:56.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.d is not active+clean 2024-01-27T02:37:56.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.800829+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T06:04:02.332379+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:37:56.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.a is not active+clean 2024-01-27T02:37:56.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.808739+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T03:47:12.515208+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:37:56.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.b is not active+clean 2024-01-27T02:37:56.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.807809+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T14:14:52.244572+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:37:56.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.8 is not active+clean 2024-01-27T02:37:56.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.806271+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:19:50.489258+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:37:56.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.9 is not active+clean 2024-01-27T02:37:56.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.806325+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T06:31:28.200720+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:37:56.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.6 is not active+clean 2024-01-27T02:37:56.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.811076+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:18:23.945261+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:37:56.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.7 is not active+clean 2024-01-27T02:37:56.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.801404+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T14:17:37.405535+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:37:56.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.4 is not active+clean 2024-01-27T02:37:56.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.800617+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T05:08:13.232740+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:37:56.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.5 is not active+clean 2024-01-27T02:37:56.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.806197+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T03:11:47.354877+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:37:56.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2 is not active+clean 2024-01-27T02:37:56.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.806456+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T06:07:01.540761+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:37:56.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3 is not active+clean 2024-01-27T02:37:56.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.797951+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T09:57:34.118284+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:37:56.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.0 is not active+clean 2024-01-27T02:37:56.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.808334+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T12:33:48.901396+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:37:56.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1 is not active+clean 2024-01-27T02:37:56.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.806238+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T08:12:51.943504+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:37:56.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.e is not active+clean 2024-01-27T02:37:56.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:37:50.790887+0000', 'last_change': '2024-01-27T02:37:50.812579+0000', 'last_active': '2024-01-27T02:37:50.790887+0000', 'last_peered': '2024-01-27T02:37:50.790887+0000', 'last_clean': '2024-01-27T02:37:50.790887+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:37:50.790887+0000', 'last_undegraded': '2024-01-27T02:37:50.790887+0000', 'last_fullsized': '2024-01-27T02:37:50.790887+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'last_clean_scrub_stamp': '2024-01-27T02:37:50.790887+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:11:16.205143+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:37:56.868 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-01-27T02:37:57.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.025+0000 7f9471fa0700 1 Processor -- start 2024-01-27T02:37:57.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.025+0000 7f9471fa0700 1 -- start start 2024-01-27T02:37:57.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.026+0000 7f9471fa0700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f946c157f80 0x7f946c06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.026+0000 7f9471fa0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946c151840 con 0x7f946c139f50 2024-01-27T02:37:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.026+0000 7f9471fa0700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946c0c85a0 con 0x7f946c06e7e0 2024-01-27T02:37:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.026+0000 7f9471fa0700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946c06ec20 con 0x7f946c157f80 2024-01-27T02:37:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.026+0000 7f946b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f946c139f50 0x7f946c06bd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50778/0 (socket says 172.21.15.7:50778) 2024-01-27T02:37:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.026+0000 7f946b7fe700 1 -- 172.21.15.7:0/1102230626 learned_addr learned my addr 172.21.15.7:0/1102230626 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:37:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.026+0000 7f946affd700 1 --2- 172.21.15.7:0/1102230626 >> v2:172.21.15.188:3300/0 conn(0x7f946c157f80 0x7f946c06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.026+0000 7f946a7fc700 1 -- 172.21.15.7:0/1102230626 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300185003 0 0) 0x7f945c001290 con 0x7f946c139f50 2024-01-27T02:37:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.026+0000 7f946a7fc700 1 -- 172.21.15.7:0/1102230626 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9454003570 con 0x7f946c139f50 2024-01-27T02:37:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946a7fc700 1 -- 172.21.15.7:0/1102230626 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 48117605 0 0) 0x7f946c0c85a0 con 0x7f946c06e7e0 2024-01-27T02:37:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946a7fc700 1 -- 172.21.15.7:0/1102230626 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f945c001290 con 0x7f946c06e7e0 2024-01-27T02:37:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946a7fc700 1 -- 172.21.15.7:0/1102230626 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 753742886 0 0) 0x7f9454003570 con 0x7f946c139f50 2024-01-27T02:37:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946a7fc700 1 -- 172.21.15.7:0/1102230626 >> v2:172.21.15.188:3300/0 conn(0x7f946c157f80 msgr2=0x7f946c06e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946a7fc700 1 --2- 172.21.15.7:0/1102230626 >> v2:172.21.15.188:3300/0 conn(0x7f946c157f80 0x7f946c06e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946a7fc700 1 -- 172.21.15.7:0/1102230626 >> v1:172.21.15.188:6789/0 conn(0x7f946c06e7e0 legacy=0x7f946c0c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946a7fc700 1 -- 172.21.15.7:0/1102230626 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c06c580 con 0x7f946c139f50 2024-01-27T02:37:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946a7fc700 1 -- 172.21.15.7:0/1102230626 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f945c001780 con 0x7f946c139f50 2024-01-27T02:37:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946a7fc700 1 -- 172.21.15.7:0/1102230626 >> v1:172.21.15.7:6789/0 conn(0x7f946c139f50 legacy=0x7f946c06bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946a7fc700 1 --2- 172.21.15.7:0/1102230626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9454003c40 0x7f9454004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946a7fc700 1 --2- 172.21.15.7:0/1102230626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9454004810 0x7f9454004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946a7fc700 1 --2- 172.21.15.7:0/1102230626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9454005210 0x7f945400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946a7fc700 1 -- 172.21.15.7:0/1102230626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c06ec20 con 0x7f9454003c40 2024-01-27T02:37:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946a7fc700 1 -- 172.21.15.7:0/1102230626 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945400c1b0 con 0x7f9454005210 2024-01-27T02:37:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946a7fc700 1 -- 172.21.15.7:0/1102230626 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945400c370 con 0x7f9454004810 2024-01-27T02:37:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946b7fe700 1 --2- 172.21.15.7:0/1102230626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9454003c40 0x7f9454004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946bfff700 1 --2- 172.21.15.7:0/1102230626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9454005210 0x7f945400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946affd700 1 --2- 172.21.15.7:0/1102230626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9454004810 0x7f9454004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946b7fe700 1 -- 172.21.15.7:0/1102230626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9454004810 msgr2=0x7f9454004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946b7fe700 1 --2- 172.21.15.7:0/1102230626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9454004810 0x7f9454004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946b7fe700 1 -- 172.21.15.7:0/1102230626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9454005210 msgr2=0x7f945400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946b7fe700 1 --2- 172.21.15.7:0/1102230626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9454005210 0x7f945400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946bfff700 1 -- 172.21.15.7:0/1102230626 reap_dead start 2024-01-27T02:37:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946b7fe700 1 -- 172.21.15.7:0/1102230626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9454003570 con 0x7f9454003c40 2024-01-27T02:37:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.027+0000 7f946b7fe700 1 --2- 172.21.15.7:0/1102230626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9454003c40 0x7f9454004030 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f945c007c00 tx=0x7f945c007970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe110fce2c0c27c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.028+0000 7f946a7fc700 1 -- 172.21.15.7:0/1102230626 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c00cc50 con 0x7f9454003c40 2024-01-27T02:37:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.028+0000 7f946a7fc700 1 -- 172.21.15.7:0/1102230626 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945c00cdd0 con 0x7f9454003c40 2024-01-27T02:37:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.028+0000 7f946a7fc700 1 -- 172.21.15.7:0/1102230626 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c007320 con 0x7f9454003c40 2024-01-27T02:37:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.028+0000 7f9471fa0700 1 -- 172.21.15.7:0/1102230626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9454003c40 msgr2=0x7f9454004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.028+0000 7f9471fa0700 1 --2- 172.21.15.7:0/1102230626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9454003c40 0x7f9454004030 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f945c007c00 tx=0x7f945c007970 comp rx=0 tx=0).stop 2024-01-27T02:37:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.028+0000 7f9471fa0700 1 -- 172.21.15.7:0/1102230626 shutdown_connections 2024-01-27T02:37:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.028+0000 7f9471fa0700 1 --2- 172.21.15.7:0/1102230626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9454003c40 0x7f9454004030 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:57.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.028+0000 7f9471fa0700 1 -- 172.21.15.7:0/1102230626 >> 172.21.15.7:0/1102230626 conn(0x7f946c0cdbf0 msgr2=0x7f946c0cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:57.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.028+0000 7f9471fa0700 1 -- 172.21.15.7:0/1102230626 shutdown_connections 2024-01-27T02:37:57.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.028+0000 7f9471fa0700 1 -- 172.21.15.7:0/1102230626 wait complete. 2024-01-27T02:37:57.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.030+0000 7f9471fa0700 1 Processor -- start 2024-01-27T02:37:57.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.030+0000 7f9471fa0700 1 -- start start 2024-01-27T02:37:57.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.030+0000 7f9471fa0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9454003c40 0x7f946c14c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:57.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.030+0000 7f9471fa0700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f946c14ced0 0x7f946c1695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:57.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.030+0000 7f9471fa0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f946c169b00 0x7f946c169f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:57.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.030+0000 7f9471fa0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c14cd40 con 0x7f946c169b00 2024-01-27T02:37:57.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.030+0000 7f9471fa0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0b1a10 con 0x7f9454003c40 2024-01-27T02:37:57.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.030+0000 7f9471fa0700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0b3480 con 0x7f946c14ced0 2024-01-27T02:37:57.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.031+0000 7f946bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f946c169b00 0x7f946c169f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:57.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.031+0000 7f946bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f946c169b00 0x7f946c169f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44166/0 (socket says 172.21.15.7:44166) 2024-01-27T02:37:57.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.031+0000 7f946bfff700 1 -- 172.21.15.7:0/54003205 learned_addr learned my addr 172.21.15.7:0/54003205 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:57.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.031+0000 7f946b7fe700 1 --2- 172.21.15.7:0/54003205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9454003c40 0x7f946c14c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:57.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.031+0000 7f946affd700 1 --2- 172.21.15.7:0/54003205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f946c14ced0 0x7f946c1695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:57.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.031+0000 7f946bfff700 1 -- 172.21.15.7:0/54003205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f946c14ced0 msgr2=0x7f946c1695c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:57.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.031+0000 7f946bfff700 1 --2- 172.21.15.7:0/54003205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f946c14ced0 0x7f946c1695c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:57.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.031+0000 7f946bfff700 1 -- 172.21.15.7:0/54003205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9454003c40 msgr2=0x7f946c14c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:57.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.031+0000 7f946bfff700 1 --2- 172.21.15.7:0/54003205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9454003c40 0x7f946c14c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:57.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.031+0000 7f946bfff700 1 -- 172.21.15.7:0/54003205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c0b3760 con 0x7f946c169b00 2024-01-27T02:37:57.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.031+0000 7f946affd700 1 --2- 172.21.15.7:0/54003205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f946c14ced0 0x7f946c1695c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:37:57.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.031+0000 7f946bfff700 1 --2- 172.21.15.7:0/54003205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f946c169b00 0x7f946c169f70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f946c0cea50 tx=0x7f94640017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:57.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.031+0000 7f94537fe700 1 -- 172.21.15.7:0/54003205 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464006db0 con 0x7f946c169b00 2024-01-27T02:37:57.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.031+0000 7f94537fe700 1 -- 172.21.15.7:0/54003205 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946400c670 con 0x7f946c169b00 2024-01-27T02:37:57.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.031+0000 7f9471fa0700 1 -- 172.21.15.7:0/54003205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f946c0b16a0 con 0x7f946c169b00 2024-01-27T02:37:57.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.031+0000 7f94537fe700 1 -- 172.21.15.7:0/54003205 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946400ca30 con 0x7f946c169b00 2024-01-27T02:37:57.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.031+0000 7f9471fa0700 1 -- 172.21.15.7:0/54003205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946c0b3030 con 0x7f946c169b00 2024-01-27T02:37:57.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.032+0000 7f9471fa0700 1 -- 172.21.15.7:0/54003205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946c0b31f0 con 0x7f946c169b00 2024-01-27T02:37:57.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.033+0000 7f94537fe700 1 -- 172.21.15.7:0/54003205 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9464014430 con 0x7f946c169b00 2024-01-27T02:37:57.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.034+0000 7f94537fe700 1 --2- 172.21.15.7:0/54003205 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9458066f30 0x7f94580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:57.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.034+0000 7f94537fe700 1 -- 172.21.15.7:0/54003205 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9464084730 con 0x7f946c169b00 2024-01-27T02:37:57.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.034+0000 7f946b7fe700 1 --2- 172.21.15.7:0/54003205 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9458066f30 0x7f94580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:57.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.035+0000 7f946b7fe700 1 --2- 172.21.15.7:0/54003205 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9458066f30 0x7f94580693e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f945c016000 tx=0x7f945c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:57.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.037+0000 7f94537fe700 1 -- 172.21.15.7:0/54003205 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9464051980 con 0x7f946c169b00 2024-01-27T02:37:57.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.329+0000 7f9471fa0700 1 -- 172.21.15.7:0/54003205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f946c0b1bc0 con 0x7f946c169b00 2024-01-27T02:37:57.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.329+0000 7f94537fe700 1 -- 172.21.15.7:0/54003205 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1449 (secure 0 0 0) 0x7f94640225c0 con 0x7f946c169b00 2024-01-27T02:37:57.330 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:57.330 INFO:teuthology.orchestra.run.smithi007.stdout:{"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":367,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":117,"num_osds":8,"num_up_osds":8,"osd_up_since":1706322715,"num_in_osds":8,"osd_in_since":1706322712,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":6170033,"bytes_avail":1593829967,"bytes_total":1600000000,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":58,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":6657}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-01-27T02:37:41.655442+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-01-27T02:37:57.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.331+0000 7f9471fa0700 1 -- 172.21.15.7:0/54003205 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9458066f30 msgr2=0x7f94580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:57.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.331+0000 7f9471fa0700 1 --2- 172.21.15.7:0/54003205 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9458066f30 0x7f94580693e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f945c016000 tx=0x7f945c017000 comp rx=0 tx=0).stop 2024-01-27T02:37:57.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.331+0000 7f9471fa0700 1 -- 172.21.15.7:0/54003205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f946c169b00 msgr2=0x7f946c169f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:57.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.331+0000 7f9471fa0700 1 --2- 172.21.15.7:0/54003205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f946c169b00 0x7f946c169f70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f946c0cea50 tx=0x7f94640017a0 comp rx=0 tx=0).stop 2024-01-27T02:37:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.331+0000 7f9471fa0700 1 -- 172.21.15.7:0/54003205 shutdown_connections 2024-01-27T02:37:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.331+0000 7f9471fa0700 1 --2- 172.21.15.7:0/54003205 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9458066f30 0x7f94580693e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.331+0000 7f9471fa0700 1 --2- 172.21.15.7:0/54003205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f946c169b00 0x7f946c169f70 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.331+0000 7f9471fa0700 1 --2- 172.21.15.7:0/54003205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9454003c40 0x7f946c14c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.331+0000 7f9471fa0700 1 --2- 172.21.15.7:0/54003205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f946c14ced0 0x7f946c1695c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.331+0000 7f9471fa0700 1 -- 172.21.15.7:0/54003205 >> 172.21.15.7:0/54003205 conn(0x7f946c0cdbf0 msgr2=0x7f946c06b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.331+0000 7f9471fa0700 1 -- 172.21.15.7:0/54003205 shutdown_connections 2024-01-27T02:37:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.331+0000 7f9471fa0700 1 -- 172.21.15.7:0/54003205 wait complete. 2024-01-27T02:37:57.345 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-01-27T02:37:57.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:37:57.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.476+0000 7f21050f3700 1 Processor -- start 2024-01-27T02:37:57.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.477+0000 7f21050f3700 1 -- start start 2024-01-27T02:37:57.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.477+0000 7f21050f3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f210015a310 0x7f21000aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:57.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.477+0000 7f21050f3700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f21000aadf0 0x7f2100063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:57.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.477+0000 7f21050f3700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21000c30c0 con 0x7f2100159f40 2024-01-27T02:37:57.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.477+0000 7f21050f3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21000c4550 con 0x7f210015a310 2024-01-27T02:37:57.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.477+0000 7f21050f3700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21000ab270 con 0x7f21000aadf0 2024-01-27T02:37:57.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.478+0000 7f20fe59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f210015a310 0x7f21000aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:57.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.478+0000 7f20fe59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f210015a310 0x7f21000aa8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44182/0 (socket says 172.21.15.7:44182) 2024-01-27T02:37:57.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.478+0000 7f20fe59c700 1 -- 172.21.15.7:0/3068592693 learned_addr learned my addr 172.21.15.7:0/3068592693 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:57.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.478+0000 7f20ff59e700 1 --2- 172.21.15.7:0/3068592693 >> v2:172.21.15.188:3300/0 conn(0x7f21000aadf0 0x7f2100063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:57.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.478+0000 7f20fe59c700 1 -- 172.21.15.7:0/3068592693 >> v2:172.21.15.188:3300/0 conn(0x7f21000aadf0 msgr2=0x7f2100063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:57.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.478+0000 7f20fe59c700 1 --2- 172.21.15.7:0/3068592693 >> v2:172.21.15.188:3300/0 conn(0x7f21000aadf0 0x7f2100063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:57.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.478+0000 7f20fe59c700 1 -- 172.21.15.7:0/3068592693 >> v1:172.21.15.188:6789/0 conn(0x7f2100159f40 legacy=0x7f21000c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:37:57.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.478+0000 7f20fe59c700 1 -- 172.21.15.7:0/3068592693 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210015e960 con 0x7f210015a310 2024-01-27T02:37:57.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.478+0000 7f20fe59c700 1 --2- 172.21.15.7:0/3068592693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210015a310 0x7f21000aa8b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f21000ca640 tx=0x7f20f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4418c0c059f5525 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:57.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.478+0000 7f20fdd9b700 1 -- 172.21.15.7:0/3068592693 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f000b820 con 0x7f210015a310 2024-01-27T02:37:57.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.478+0000 7f20fdd9b700 1 -- 172.21.15.7:0/3068592693 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f000b980 con 0x7f210015a310 2024-01-27T02:37:57.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.478+0000 7f20fdd9b700 1 -- 172.21.15.7:0/3068592693 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f000bd20 con 0x7f210015a310 2024-01-27T02:37:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.479+0000 7f21050f3700 1 -- 172.21.15.7:0/3068592693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210015a310 msgr2=0x7f21000aa8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.479+0000 7f21050f3700 1 --2- 172.21.15.7:0/3068592693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210015a310 0x7f21000aa8b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f21000ca640 tx=0x7f20f0009250 comp rx=0 tx=0).stop 2024-01-27T02:37:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.479+0000 7f21050f3700 1 -- 172.21.15.7:0/3068592693 shutdown_connections 2024-01-27T02:37:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.479+0000 7f21050f3700 1 --2- 172.21.15.7:0/3068592693 >> v2:172.21.15.188:3300/0 conn(0x7f21000aadf0 0x7f2100063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.479+0000 7f21050f3700 1 --2- 172.21.15.7:0/3068592693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210015a310 0x7f21000aa8b0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.479+0000 7f21050f3700 1 -- 172.21.15.7:0/3068592693 >> 172.21.15.7:0/3068592693 conn(0x7f21000cdc30 msgr2=0x7f21000ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.479+0000 7f21050f3700 1 -- 172.21.15.7:0/3068592693 shutdown_connections 2024-01-27T02:37:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.479+0000 7f21050f3700 1 -- 172.21.15.7:0/3068592693 wait complete. 2024-01-27T02:37:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.480+0000 7f21050f3700 1 Processor -- start 2024-01-27T02:37:57.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.480+0000 7f21050f3700 1 -- start start 2024-01-27T02:37:57.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.481+0000 7f21050f3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21000aadf0 0x7f210014d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:57.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.481+0000 7f21050f3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2100159f40 0x7f210015dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:57.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.481+0000 7f21050f3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f210015a310 0x7f210015e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.481+0000 7f21050f3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210016e880 con 0x7f21000aadf0 2024-01-27T02:37:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.481+0000 7f21050f3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210016dc70 con 0x7f210015a310 2024-01-27T02:37:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.481+0000 7f21050f3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210016df50 con 0x7f2100159f40 2024-01-27T02:37:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.481+0000 7f20fed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21000aadf0 0x7f210014d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.481+0000 7f20fed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21000aadf0 0x7f210014d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44196/0 (socket says 172.21.15.7:44196) 2024-01-27T02:37:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.481+0000 7f20fed9d700 1 -- 172.21.15.7:0/1983279455 learned_addr learned my addr 172.21.15.7:0/1983279455 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:37:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.481+0000 7f20ff59e700 1 --2- 172.21.15.7:0/1983279455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f210015a310 0x7f210015e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.481+0000 7f20fe59c700 1 --2- 172.21.15.7:0/1983279455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2100159f40 0x7f210015dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.482+0000 7f20fed9d700 1 -- 172.21.15.7:0/1983279455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2100159f40 msgr2=0x7f210015dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.482+0000 7f20fed9d700 1 --2- 172.21.15.7:0/1983279455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2100159f40 0x7f210015dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.482+0000 7f20fed9d700 1 -- 172.21.15.7:0/1983279455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f210015a310 msgr2=0x7f210015e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.482+0000 7f20fed9d700 1 --2- 172.21.15.7:0/1983279455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f210015a310 0x7f210015e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.482+0000 7f20fed9d700 1 -- 172.21.15.7:0/1983279455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f00094f0 con 0x7f21000aadf0 2024-01-27T02:37:57.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.482+0000 7f20fed9d700 1 --2- 172.21.15.7:0/1983279455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21000aadf0 0x7f210014d5b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f20e800c5d0 tx=0x7f20e8007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:57.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.482+0000 7f20f6ffd700 1 -- 172.21.15.7:0/1983279455 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8011590 con 0x7f21000aadf0 2024-01-27T02:37:57.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.482+0000 7f20f6ffd700 1 -- 172.21.15.7:0/1983279455 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e8011710 con 0x7f21000aadf0 2024-01-27T02:37:57.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.482+0000 7f21050f3700 1 -- 172.21.15.7:0/1983279455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210016e230 con 0x7f21000aadf0 2024-01-27T02:37:57.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.482+0000 7f20f6ffd700 1 -- 172.21.15.7:0/1983279455 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8011ad0 con 0x7f21000aadf0 2024-01-27T02:37:57.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.482+0000 7f21050f3700 1 -- 172.21.15.7:0/1983279455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210014daf0 con 0x7f21000aadf0 2024-01-27T02:37:57.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.483+0000 7f21050f3700 1 -- 172.21.15.7:0/1983279455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210014dce0 con 0x7f21000aadf0 2024-01-27T02:37:57.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.484+0000 7f20f6ffd700 1 -- 172.21.15.7:0/1983279455 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f20e801c460 con 0x7f21000aadf0 2024-01-27T02:37:57.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.484+0000 7f20f6ffd700 1 --2- 172.21.15.7:0/1983279455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f20e0066f30 0x7f20e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:37:57.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.484+0000 7f20f6ffd700 1 -- 172.21.15.7:0/1983279455 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f20e8084190 con 0x7f21000aadf0 2024-01-27T02:37:57.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.484+0000 7f20fe59c700 1 --2- 172.21.15.7:0/1983279455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f20e0066f30 0x7f20e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:37:57.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.485+0000 7f20fe59c700 1 --2- 172.21.15.7:0/1983279455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f20e0066f30 0x7f20e00693e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f21000ca640 tx=0x7f20f0016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:37:57.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.489+0000 7f20f6ffd700 1 -- 172.21.15.7:0/1983279455 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f210014dce0 con 0x7f21000aadf0 2024-01-27T02:37:57.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.686+0000 7f21050f3700 1 -- 172.21.15.7:0/1983279455 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f210016e9c0 con 0x7f20e0066f30 2024-01-27T02:37:57.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.695+0000 7f20f6ffd700 1 -- 172.21.15.7:0/1983279455 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285635 (secure 0 0 0) 0x7f20f000baf0 con 0x7f20e0066f30 2024-01-27T02:37:57.699 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:37:57.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.698+0000 7f21050f3700 1 -- 172.21.15.7:0/1983279455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f20e0066f30 msgr2=0x7f20e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:57.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.698+0000 7f21050f3700 1 --2- 172.21.15.7:0/1983279455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f20e0066f30 0x7f20e00693e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f21000ca640 tx=0x7f20f0016000 comp rx=0 tx=0).stop 2024-01-27T02:37:57.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.698+0000 7f21050f3700 1 -- 172.21.15.7:0/1983279455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21000aadf0 msgr2=0x7f210014d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:37:57.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.698+0000 7f21050f3700 1 --2- 172.21.15.7:0/1983279455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21000aadf0 0x7f210014d5b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f20e800c5d0 tx=0x7f20e8007f60 comp rx=0 tx=0).stop 2024-01-27T02:37:57.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.698+0000 7f21050f3700 1 -- 172.21.15.7:0/1983279455 shutdown_connections 2024-01-27T02:37:57.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.698+0000 7f21050f3700 1 --2- 172.21.15.7:0/1983279455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f20e0066f30 0x7f20e00693e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:57.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.698+0000 7f21050f3700 1 --2- 172.21.15.7:0/1983279455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f210015a310 0x7f210015e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:57.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.698+0000 7f21050f3700 1 --2- 172.21.15.7:0/1983279455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21000aadf0 0x7f210014d5b0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:57.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.698+0000 7f21050f3700 1 --2- 172.21.15.7:0/1983279455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2100159f40 0x7f210015dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:37:57.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.698+0000 7f21050f3700 1 -- 172.21.15.7:0/1983279455 >> 172.21.15.7:0/1983279455 conn(0x7f21000cdc30 msgr2=0x7f210006c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:37:57.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.698+0000 7f21050f3700 1 -- 172.21.15.7:0/1983279455 shutdown_connections 2024-01-27T02:37:57.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:37:57.698+0000 7f21050f3700 1 -- 172.21.15.7:0/1983279455 wait complete. 2024-01-27T02:37:57.702 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:37:57.712 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-01-27T02:37:57.658907+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6138,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556361,"statfs":{"total":1600000000,"available":1593714684,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.857490"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.832733+0000","last_change":"2024-01-27T02:37:51.831826+0000","last_active":"2024-01-27T02:37:51.832733+0000","last_peered":"2024-01-27T02:37:51.832733+0000","last_clean":"2024-01-27T02:37:51.832733+0000","last_became_active":"2024-01-27T02:37:51.828656+0000","last_became_peered":"2024-01-27T02:37:51.828656+0000","last_unstale":"2024-01-27T02:37:51.832733+0000","last_undegraded":"2024-01-27T02:37:51.832733+0000","last_fullsized":"2024-01-27T02:37:51.832733+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:31:22.187233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.830345+0000","last_change":"2024-01-27T02:37:51.827439+0000","last_active":"2024-01-27T02:37:51.830345+0000","last_peered":"2024-01-27T02:37:51.830345+0000","last_clean":"2024-01-27T02:37:51.830345+0000","last_became_active":"2024-01-27T02:37:51.827041+0000","last_became_peered":"2024-01-27T02:37:51.827041+0000","last_unstale":"2024-01-27T02:37:51.830345+0000","last_undegraded":"2024-01-27T02:37:51.830345+0000","last_fullsized":"2024-01-27T02:37:51.830345+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:33:03.228144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.835215+0000","last_change":"2024-01-27T02:37:51.835020+0000","last_active":"2024-01-27T02:37:51.835215+0000","last_peered":"2024-01-27T02:37:51.835215+0000","last_clean":"2024-01-27T02:37:51.835215+0000","last_became_active":"2024-01-27T02:37:51.834822+0000","last_became_peered":"2024-01-27T02:37:51.834822+0000","last_unstale":"2024-01-27T02:37:51.835215+0000","last_undegraded":"2024-01-27T02:37:51.835215+0000","last_fullsized":"2024-01-27T02:37:51.835215+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:56:06.441719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.825809+0000","last_change":"2024-01-27T02:37:51.824465+0000","last_active":"2024-01-27T02:37:51.825809+0000","last_peered":"2024-01-27T02:37:51.825809+0000","last_clean":"2024-01-27T02:37:51.825809+0000","last_became_active":"2024-01-27T02:37:51.824096+0000","last_became_peered":"2024-01-27T02:37:51.824096+0000","last_unstale":"2024-01-27T02:37:51.825809+0000","last_undegraded":"2024-01-27T02:37:51.825809+0000","last_fullsized":"2024-01-27T02:37:51.825809+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:45:13.633104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.827665+0000","last_change":"2024-01-27T02:37:51.827575+0000","last_active":"2024-01-27T02:37:51.827665+0000","last_peered":"2024-01-27T02:37:51.827665+0000","last_clean":"2024-01-27T02:37:51.827665+0000","last_became_active":"2024-01-27T02:37:51.827302+0000","last_became_peered":"2024-01-27T02:37:51.827302+0000","last_unstale":"2024-01-27T02:37:51.827665+0000","last_undegraded":"2024-01-27T02:37:51.827665+0000","last_fullsized":"2024-01-27T02:37:51.827665+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:46:44.234857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.829784+0000","last_change":"2024-01-27T02:37:51.829687+0000","last_active":"2024-01-27T02:37:51.829784+0000","last_peered":"2024-01-27T02:37:51.829784+0000","last_clean":"2024-01-27T02:37:51.829784+0000","last_became_active":"2024-01-27T02:37:51.829229+0000","last_became_peered":"2024-01-27T02:37:51.829229+0000","last_unstale":"2024-01-27T02:37:51.829784+0000","last_undegraded":"2024-01-27T02:37:51.829784+0000","last_fullsized":"2024-01-27T02:37:51.829784+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:57.231078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.824072+0000","last_change":"2024-01-27T02:37:51.823916+0000","last_active":"2024-01-27T02:37:51.824072+0000","last_peered":"2024-01-27T02:37:51.824072+0000","last_clean":"2024-01-27T02:37:51.824072+0000","last_became_active":"2024-01-27T02:37:51.823677+0000","last_became_peered":"2024-01-27T02:37:51.823677+0000","last_unstale":"2024-01-27T02:37:51.824072+0000","last_undegraded":"2024-01-27T02:37:51.824072+0000","last_fullsized":"2024-01-27T02:37:51.824072+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:10:19.964021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.835156+0000","last_change":"2024-01-27T02:37:51.835076+0000","last_active":"2024-01-27T02:37:51.835156+0000","last_peered":"2024-01-27T02:37:51.835156+0000","last_clean":"2024-01-27T02:37:51.835156+0000","last_became_active":"2024-01-27T02:37:51.834270+0000","last_became_peered":"2024-01-27T02:37:51.834270+0000","last_unstale":"2024-01-27T02:37:51.835156+0000","last_undegraded":"2024-01-27T02:37:51.835156+0000","last_fullsized":"2024-01-27T02:37:51.835156+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:32:37.260822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.816665+0000","last_change":"2024-01-27T02:37:51.816582+0000","last_active":"2024-01-27T02:37:51.816665+0000","last_peered":"2024-01-27T02:37:51.816665+0000","last_clean":"2024-01-27T02:37:51.816665+0000","last_became_active":"2024-01-27T02:37:51.816341+0000","last_became_peered":"2024-01-27T02:37:51.816341+0000","last_unstale":"2024-01-27T02:37:51.816665+0000","last_undegraded":"2024-01-27T02:37:51.816665+0000","last_fullsized":"2024-01-27T02:37:51.816665+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:17:03.370160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.837046+0000","last_change":"2024-01-27T02:37:51.836999+0000","last_active":"2024-01-27T02:37:51.837046+0000","last_peered":"2024-01-27T02:37:51.837046+0000","last_clean":"2024-01-27T02:37:51.837046+0000","last_became_active":"2024-01-27T02:37:51.836568+0000","last_became_peered":"2024-01-27T02:37:51.836568+0000","last_unstale":"2024-01-27T02:37:51.837046+0000","last_undegraded":"2024-01-27T02:37:51.837046+0000","last_fullsized":"2024-01-27T02:37:51.837046+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:10:19.964021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.829603+0000","last_change":"2024-01-27T02:37:51.829508+0000","last_active":"2024-01-27T02:37:51.829603+0000","last_peered":"2024-01-27T02:37:51.829603+0000","last_clean":"2024-01-27T02:37:51.829603+0000","last_became_active":"2024-01-27T02:37:51.829047+0000","last_became_peered":"2024-01-27T02:37:51.829047+0000","last_unstale":"2024-01-27T02:37:51.829603+0000","last_undegraded":"2024-01-27T02:37:51.829603+0000","last_fullsized":"2024-01-27T02:37:51.829603+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:32:37.260822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.833263+0000","last_change":"2024-01-27T02:37:51.833172+0000","last_active":"2024-01-27T02:37:51.833263+0000","last_peered":"2024-01-27T02:37:51.833263+0000","last_clean":"2024-01-27T02:37:51.833263+0000","last_became_active":"2024-01-27T02:37:51.832923+0000","last_became_peered":"2024-01-27T02:37:51.832923+0000","last_unstale":"2024-01-27T02:37:51.833263+0000","last_undegraded":"2024-01-27T02:37:51.833263+0000","last_fullsized":"2024-01-27T02:37:51.833263+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:24:13.096584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.816999+0000","last_change":"2024-01-27T02:37:51.816884+0000","last_active":"2024-01-27T02:37:51.816999+0000","last_peered":"2024-01-27T02:37:51.816999+0000","last_clean":"2024-01-27T02:37:51.816999+0000","last_became_active":"2024-01-27T02:37:51.816521+0000","last_became_peered":"2024-01-27T02:37:51.816521+0000","last_unstale":"2024-01-27T02:37:51.816999+0000","last_undegraded":"2024-01-27T02:37:51.816999+0000","last_fullsized":"2024-01-27T02:37:51.816999+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:57.231078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.821816+0000","last_change":"2024-01-27T02:37:51.821616+0000","last_active":"2024-01-27T02:37:51.821816+0000","last_peered":"2024-01-27T02:37:51.821816+0000","last_clean":"2024-01-27T02:37:51.821816+0000","last_became_active":"2024-01-27T02:37:51.821168+0000","last_became_peered":"2024-01-27T02:37:51.821168+0000","last_unstale":"2024-01-27T02:37:51.821816+0000","last_undegraded":"2024-01-27T02:37:51.821816+0000","last_fullsized":"2024-01-27T02:37:51.821816+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:56:52.471976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.835379+0000","last_change":"2024-01-27T02:37:51.835113+0000","last_active":"2024-01-27T02:37:51.835379+0000","last_peered":"2024-01-27T02:37:51.835379+0000","last_clean":"2024-01-27T02:37:51.835379+0000","last_became_active":"2024-01-27T02:37:51.834542+0000","last_became_peered":"2024-01-27T02:37:51.834542+0000","last_unstale":"2024-01-27T02:37:51.835379+0000","last_undegraded":"2024-01-27T02:37:51.835379+0000","last_fullsized":"2024-01-27T02:37:51.835379+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:18:23.945261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.832883+0000","last_change":"2024-01-27T02:37:51.832781+0000","last_active":"2024-01-27T02:37:51.832883+0000","last_peered":"2024-01-27T02:37:51.832883+0000","last_clean":"2024-01-27T02:37:51.832883+0000","last_became_active":"2024-01-27T02:37:51.832535+0000","last_became_peered":"2024-01-27T02:37:51.832535+0000","last_unstale":"2024-01-27T02:37:51.832883+0000","last_undegraded":"2024-01-27T02:37:51.832883+0000","last_fullsized":"2024-01-27T02:37:51.832883+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:46:44.234857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.830493+0000","last_change":"2024-01-27T02:37:51.829796+0000","last_active":"2024-01-27T02:37:51.830493+0000","last_peered":"2024-01-27T02:37:51.830493+0000","last_clean":"2024-01-27T02:37:51.830493+0000","last_became_active":"2024-01-27T02:37:51.829526+0000","last_became_peered":"2024-01-27T02:37:51.829526+0000","last_unstale":"2024-01-27T02:37:51.830493+0000","last_undegraded":"2024-01-27T02:37:51.830493+0000","last_fullsized":"2024-01-27T02:37:51.830493+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:13:04.047913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.825862+0000","last_change":"2024-01-27T02:37:51.825613+0000","last_active":"2024-01-27T02:37:51.825862+0000","last_peered":"2024-01-27T02:37:51.825862+0000","last_clean":"2024-01-27T02:37:51.825862+0000","last_became_active":"2024-01-27T02:37:51.825184+0000","last_became_peered":"2024-01-27T02:37:51.825184+0000","last_unstale":"2024-01-27T02:37:51.825862+0000","last_undegraded":"2024-01-27T02:37:51.825862+0000","last_fullsized":"2024-01-27T02:37:51.825862+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:33:08.282783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.837170+0000","last_change":"2024-01-27T02:37:51.837018+0000","last_active":"2024-01-27T02:37:51.837170+0000","last_peered":"2024-01-27T02:37:51.837170+0000","last_clean":"2024-01-27T02:37:51.837170+0000","last_became_active":"2024-01-27T02:37:51.836744+0000","last_became_peered":"2024-01-27T02:37:51.836744+0000","last_unstale":"2024-01-27T02:37:51.837170+0000","last_undegraded":"2024-01-27T02:37:51.837170+0000","last_fullsized":"2024-01-27T02:37:51.837170+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:52:48.549784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.819849+0000","last_change":"2024-01-27T02:37:51.819774+0000","last_active":"2024-01-27T02:37:51.819849+0000","last_peered":"2024-01-27T02:37:51.819849+0000","last_clean":"2024-01-27T02:37:51.819849+0000","last_became_active":"2024-01-27T02:37:51.819537+0000","last_became_peered":"2024-01-27T02:37:51.819537+0000","last_unstale":"2024-01-27T02:37:51.819849+0000","last_undegraded":"2024-01-27T02:37:51.819849+0000","last_fullsized":"2024-01-27T02:37:51.819849+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:04:02.332379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.824588+0000","last_change":"2024-01-27T02:37:51.824517+0000","last_active":"2024-01-27T02:37:51.824588+0000","last_peered":"2024-01-27T02:37:51.824588+0000","last_clean":"2024-01-27T02:37:51.824588+0000","last_became_active":"2024-01-27T02:37:51.823909+0000","last_became_peered":"2024-01-27T02:37:51.823909+0000","last_unstale":"2024-01-27T02:37:51.824588+0000","last_undegraded":"2024-01-27T02:37:51.824588+0000","last_fullsized":"2024-01-27T02:37:51.824588+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:34:22.454613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.830328+0000","last_change":"2024-01-27T02:37:51.827287+0000","last_active":"2024-01-27T02:37:51.830328+0000","last_peered":"2024-01-27T02:37:51.830328+0000","last_clean":"2024-01-27T02:37:51.830328+0000","last_became_active":"2024-01-27T02:37:51.826700+0000","last_became_peered":"2024-01-27T02:37:51.826700+0000","last_unstale":"2024-01-27T02:37:51.830328+0000","last_undegraded":"2024-01-27T02:37:51.830328+0000","last_fullsized":"2024-01-27T02:37:51.830328+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:55:56.833988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.830309+0000","last_change":"2024-01-27T02:37:51.827254+0000","last_active":"2024-01-27T02:37:51.830309+0000","last_peered":"2024-01-27T02:37:51.830309+0000","last_clean":"2024-01-27T02:37:51.830309+0000","last_became_active":"2024-01-27T02:37:51.826830+0000","last_became_peered":"2024-01-27T02:37:51.826830+0000","last_unstale":"2024-01-27T02:37:51.830309+0000","last_undegraded":"2024-01-27T02:37:51.830309+0000","last_fullsized":"2024-01-27T02:37:51.830309+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:09:28.627404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.836533+0000","last_change":"2024-01-27T02:37:51.836459+0000","last_active":"2024-01-27T02:37:51.836533+0000","last_peered":"2024-01-27T02:37:51.836533+0000","last_clean":"2024-01-27T02:37:51.836533+0000","last_became_active":"2024-01-27T02:37:51.836270+0000","last_became_peered":"2024-01-27T02:37:51.836270+0000","last_unstale":"2024-01-27T02:37:51.836533+0000","last_undegraded":"2024-01-27T02:37:51.836533+0000","last_fullsized":"2024-01-27T02:37:51.836533+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:29:07.353494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.826720+0000","last_change":"2024-01-27T02:37:51.826523+0000","last_active":"2024-01-27T02:37:51.826720+0000","last_peered":"2024-01-27T02:37:51.826720+0000","last_clean":"2024-01-27T02:37:51.826720+0000","last_became_active":"2024-01-27T02:37:51.826229+0000","last_became_peered":"2024-01-27T02:37:51.826229+0000","last_unstale":"2024-01-27T02:37:51.826720+0000","last_undegraded":"2024-01-27T02:37:51.826720+0000","last_fullsized":"2024-01-27T02:37:51.826720+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:28:34.028232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.837037+0000","last_change":"2024-01-27T02:37:51.836964+0000","last_active":"2024-01-27T02:37:51.837037+0000","last_peered":"2024-01-27T02:37:51.837037+0000","last_clean":"2024-01-27T02:37:51.837037+0000","last_became_active":"2024-01-27T02:37:51.836726+0000","last_became_peered":"2024-01-27T02:37:51.836726+0000","last_unstale":"2024-01-27T02:37:51.837037+0000","last_undegraded":"2024-01-27T02:37:51.837037+0000","last_fullsized":"2024-01-27T02:37:51.837037+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:28:34.028232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.832864+0000","last_change":"2024-01-27T02:37:51.832531+0000","last_active":"2024-01-27T02:37:51.832864+0000","last_peered":"2024-01-27T02:37:51.832864+0000","last_clean":"2024-01-27T02:37:51.832864+0000","last_became_active":"2024-01-27T02:37:51.832307+0000","last_became_peered":"2024-01-27T02:37:51.832307+0000","last_unstale":"2024-01-27T02:37:51.832864+0000","last_undegraded":"2024-01-27T02:37:51.832864+0000","last_fullsized":"2024-01-27T02:37:51.832864+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:04:59.885669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.833411+0000","last_change":"2024-01-27T02:37:51.833286+0000","last_active":"2024-01-27T02:37:51.833411+0000","last_peered":"2024-01-27T02:37:51.833411+0000","last_clean":"2024-01-27T02:37:51.833411+0000","last_became_active":"2024-01-27T02:37:51.833006+0000","last_became_peered":"2024-01-27T02:37:51.833006+0000","last_unstale":"2024-01-27T02:37:51.833411+0000","last_undegraded":"2024-01-27T02:37:51.833411+0000","last_fullsized":"2024-01-27T02:37:51.833411+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:55:18.331130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.836543+0000","last_change":"2024-01-27T02:37:51.836473+0000","last_active":"2024-01-27T02:37:51.836543+0000","last_peered":"2024-01-27T02:37:51.836543+0000","last_clean":"2024-01-27T02:37:51.836543+0000","last_became_active":"2024-01-27T02:37:51.836257+0000","last_became_peered":"2024-01-27T02:37:51.836257+0000","last_unstale":"2024-01-27T02:37:51.836543+0000","last_undegraded":"2024-01-27T02:37:51.836543+0000","last_fullsized":"2024-01-27T02:37:51.836543+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:25:00.247752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.837819+0000","last_change":"2024-01-27T02:37:51.837717+0000","last_active":"2024-01-27T02:37:51.837819+0000","last_peered":"2024-01-27T02:37:51.837819+0000","last_clean":"2024-01-27T02:37:51.837819+0000","last_became_active":"2024-01-27T02:37:51.837280+0000","last_became_peered":"2024-01-27T02:37:51.837280+0000","last_unstale":"2024-01-27T02:37:51.837819+0000","last_undegraded":"2024-01-27T02:37:51.837819+0000","last_fullsized":"2024-01-27T02:37:51.837819+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:26:33.037819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.823183+0000","last_change":"2024-01-27T02:37:51.822736+0000","last_active":"2024-01-27T02:37:51.823183+0000","last_peered":"2024-01-27T02:37:51.823183+0000","last_clean":"2024-01-27T02:37:51.823183+0000","last_became_active":"2024-01-27T02:37:51.822445+0000","last_became_peered":"2024-01-27T02:37:51.822445+0000","last_unstale":"2024-01-27T02:37:51.823183+0000","last_undegraded":"2024-01-27T02:37:51.823183+0000","last_fullsized":"2024-01-27T02:37:51.823183+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:28:34.028232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.836314+0000","last_change":"2024-01-27T02:37:51.836032+0000","last_active":"2024-01-27T02:37:51.836314+0000","last_peered":"2024-01-27T02:37:51.836314+0000","last_clean":"2024-01-27T02:37:51.836314+0000","last_became_active":"2024-01-27T02:37:51.835774+0000","last_became_peered":"2024-01-27T02:37:51.835774+0000","last_unstale":"2024-01-27T02:37:51.836314+0000","last_undegraded":"2024-01-27T02:37:51.836314+0000","last_fullsized":"2024-01-27T02:37:51.836314+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:19:36.726696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.832209+0000","last_change":"2024-01-27T02:37:51.831833+0000","last_active":"2024-01-27T02:37:51.832209+0000","last_peered":"2024-01-27T02:37:51.832209+0000","last_clean":"2024-01-27T02:37:51.832209+0000","last_became_active":"2024-01-27T02:37:51.831415+0000","last_became_peered":"2024-01-27T02:37:51.831415+0000","last_unstale":"2024-01-27T02:37:51.832209+0000","last_undegraded":"2024-01-27T02:37:51.832209+0000","last_fullsized":"2024-01-27T02:37:51.832209+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:33:02.346431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"116'4","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.831677+0000","last_change":"2024-01-27T02:37:50.812131+0000","last_active":"2024-01-27T02:37:51.831677+0000","last_peered":"2024-01-27T02:37:51.831677+0000","last_clean":"2024-01-27T02:37:51.831677+0000","last_became_active":"2024-01-27T02:37:50.809514+0000","last_became_peered":"2024-01-27T02:37:50.809514+0000","last_unstale":"2024-01-27T02:37:51.831677+0000","last_undegraded":"2024-01-27T02:37:51.831677+0000","last_fullsized":"2024-01-27T02:37:51.831677+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:14:50.186256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.825845+0000","last_change":"2024-01-27T02:37:51.825465+0000","last_active":"2024-01-27T02:37:51.825845+0000","last_peered":"2024-01-27T02:37:51.825845+0000","last_clean":"2024-01-27T02:37:51.825845+0000","last_became_active":"2024-01-27T02:37:51.824864+0000","last_became_peered":"2024-01-27T02:37:51.824864+0000","last_unstale":"2024-01-27T02:37:51.825845+0000","last_undegraded":"2024-01-27T02:37:51.825845+0000","last_fullsized":"2024-01-27T02:37:51.825845+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:25:00.247752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.836024+0000","last_change":"2024-01-27T02:37:50.808649+0000","last_active":"2024-01-27T02:37:51.836024+0000","last_peered":"2024-01-27T02:37:51.836024+0000","last_clean":"2024-01-27T02:37:51.836024+0000","last_became_active":"2024-01-27T02:37:50.808412+0000","last_became_peered":"2024-01-27T02:37:50.808412+0000","last_unstale":"2024-01-27T02:37:51.836024+0000","last_undegraded":"2024-01-27T02:37:51.836024+0000","last_fullsized":"2024-01-27T02:37:51.836024+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:24:12.091569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.832717+0000","last_change":"2024-01-27T02:37:51.831656+0000","last_active":"2024-01-27T02:37:51.832717+0000","last_peered":"2024-01-27T02:37:51.832717+0000","last_clean":"2024-01-27T02:37:51.832717+0000","last_became_active":"2024-01-27T02:37:51.828372+0000","last_became_peered":"2024-01-27T02:37:51.828372+0000","last_unstale":"2024-01-27T02:37:51.832717+0000","last_undegraded":"2024-01-27T02:37:51.832717+0000","last_fullsized":"2024-01-27T02:37:51.832717+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:11:06.487989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.831171+0000","last_change":"2024-01-27T02:37:50.806041+0000","last_active":"2024-01-27T02:37:51.831171+0000","last_peered":"2024-01-27T02:37:51.831171+0000","last_clean":"2024-01-27T02:37:51.831171+0000","last_became_active":"2024-01-27T02:37:50.805760+0000","last_became_peered":"2024-01-27T02:37:50.805760+0000","last_unstale":"2024-01-27T02:37:51.831171+0000","last_undegraded":"2024-01-27T02:37:51.831171+0000","last_fullsized":"2024-01-27T02:37:51.831171+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:31:21.182217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.826351+0000","last_change":"2024-01-27T02:37:51.825416+0000","last_active":"2024-01-27T02:37:51.826351+0000","last_peered":"2024-01-27T02:37:51.826351+0000","last_clean":"2024-01-27T02:37:51.826351+0000","last_became_active":"2024-01-27T02:37:51.825167+0000","last_became_peered":"2024-01-27T02:37:51.825167+0000","last_unstale":"2024-01-27T02:37:51.826351+0000","last_undegraded":"2024-01-27T02:37:51.826351+0000","last_fullsized":"2024-01-27T02:37:51.826351+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:28.200720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.832255+0000","last_change":"2024-01-27T02:37:50.808048+0000","last_active":"2024-01-27T02:37:51.832255+0000","last_peered":"2024-01-27T02:37:51.832255+0000","last_clean":"2024-01-27T02:37:51.832255+0000","last_became_active":"2024-01-27T02:37:50.807771+0000","last_became_peered":"2024-01-27T02:37:50.807771+0000","last_unstale":"2024-01-27T02:37:51.832255+0000","last_undegraded":"2024-01-27T02:37:51.832255+0000","last_fullsized":"2024-01-27T02:37:51.832255+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:24.808914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.828125+0000","last_change":"2024-01-27T02:37:51.828060+0000","last_active":"2024-01-27T02:37:51.828125+0000","last_peered":"2024-01-27T02:37:51.828125+0000","last_clean":"2024-01-27T02:37:51.828125+0000","last_became_active":"2024-01-27T02:37:51.827829+0000","last_became_peered":"2024-01-27T02:37:51.827829+0000","last_unstale":"2024-01-27T02:37:51.828125+0000","last_undegraded":"2024-01-27T02:37:51.828125+0000","last_fullsized":"2024-01-27T02:37:51.828125+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:12:54.826584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.837161+0000","last_change":"2024-01-27T02:37:50.806436+0000","last_active":"2024-01-27T02:37:51.837161+0000","last_peered":"2024-01-27T02:37:51.837161+0000","last_clean":"2024-01-27T02:37:51.837161+0000","last_became_active":"2024-01-27T02:37:50.806167+0000","last_became_peered":"2024-01-27T02:37:50.806167+0000","last_unstale":"2024-01-27T02:37:51.837161+0000","last_undegraded":"2024-01-27T02:37:51.837161+0000","last_fullsized":"2024-01-27T02:37:51.837161+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:58:12.533491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.824051+0000","last_change":"2024-01-27T02:37:51.823428+0000","last_active":"2024-01-27T02:37:51.824051+0000","last_peered":"2024-01-27T02:37:51.824051+0000","last_clean":"2024-01-27T02:37:51.824051+0000","last_became_active":"2024-01-27T02:37:51.823162+0000","last_became_peered":"2024-01-27T02:37:51.823162+0000","last_unstale":"2024-01-27T02:37:51.824051+0000","last_undegraded":"2024-01-27T02:37:51.824051+0000","last_fullsized":"2024-01-27T02:37:51.824051+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:33:08.282783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.838812+0000","last_change":"2024-01-27T02:37:50.812296+0000","last_active":"2024-01-27T02:37:51.838812+0000","last_peered":"2024-01-27T02:37:51.838812+0000","last_clean":"2024-01-27T02:37:51.838812+0000","last_became_active":"2024-01-27T02:37:50.809825+0000","last_became_peered":"2024-01-27T02:37:50.809825+0000","last_unstale":"2024-01-27T02:37:51.838812+0000","last_undegraded":"2024-01-27T02:37:51.838812+0000","last_fullsized":"2024-01-27T02:37:51.838812+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:54.338500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.835400+0000","last_change":"2024-01-27T02:37:51.835270+0000","last_active":"2024-01-27T02:37:51.835400+0000","last_peered":"2024-01-27T02:37:51.835400+0000","last_clean":"2024-01-27T02:37:51.835400+0000","last_became_active":"2024-01-27T02:37:51.834877+0000","last_became_peered":"2024-01-27T02:37:51.834877+0000","last_unstale":"2024-01-27T02:37:51.835400+0000","last_undegraded":"2024-01-27T02:37:51.835400+0000","last_fullsized":"2024-01-27T02:37:51.835400+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:46:44.234857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.818753+0000","last_change":"2024-01-27T02:37:50.804462+0000","last_active":"2024-01-27T02:37:51.818753+0000","last_peered":"2024-01-27T02:37:51.818753+0000","last_clean":"2024-01-27T02:37:51.818753+0000","last_became_active":"2024-01-27T02:37:50.804211+0000","last_became_peered":"2024-01-27T02:37:50.804211+0000","last_unstale":"2024-01-27T02:37:51.818753+0000","last_undegraded":"2024-01-27T02:37:51.818753+0000","last_fullsized":"2024-01-27T02:37:51.818753+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:16:34.794337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.827327+0000","last_change":"2024-01-27T02:37:51.827257+0000","last_active":"2024-01-27T02:37:51.827327+0000","last_peered":"2024-01-27T02:37:51.827327+0000","last_clean":"2024-01-27T02:37:51.827327+0000","last_became_active":"2024-01-27T02:37:51.827022+0000","last_became_peered":"2024-01-27T02:37:51.827022+0000","last_unstale":"2024-01-27T02:37:51.827327+0000","last_undegraded":"2024-01-27T02:37:51.827327+0000","last_fullsized":"2024-01-27T02:37:51.827327+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:33:14.691144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.833948+0000","last_change":"2024-01-27T02:37:50.804294+0000","last_active":"2024-01-27T02:37:51.833948+0000","last_peered":"2024-01-27T02:37:51.833948+0000","last_clean":"2024-01-27T02:37:51.833948+0000","last_became_active":"2024-01-27T02:37:50.803789+0000","last_became_peered":"2024-01-27T02:37:50.803789+0000","last_unstale":"2024-01-27T02:37:51.833948+0000","last_undegraded":"2024-01-27T02:37:51.833948+0000","last_fullsized":"2024-01-27T02:37:51.833948+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:37:47.399101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.830461+0000","last_change":"2024-01-27T02:37:51.829268+0000","last_active":"2024-01-27T02:37:51.830461+0000","last_peered":"2024-01-27T02:37:51.830461+0000","last_clean":"2024-01-27T02:37:51.830461+0000","last_became_active":"2024-01-27T02:37:51.829000+0000","last_became_peered":"2024-01-27T02:37:51.829000+0000","last_unstale":"2024-01-27T02:37:51.830461+0000","last_undegraded":"2024-01-27T02:37:51.830461+0000","last_fullsized":"2024-01-27T02:37:51.830461+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:52:20.588218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.828195+0000","last_change":"2024-01-27T02:37:50.807055+0000","last_active":"2024-01-27T02:37:51.828195+0000","last_peered":"2024-01-27T02:37:51.828195+0000","last_clean":"2024-01-27T02:37:51.828195+0000","last_became_active":"2024-01-27T02:37:50.806246+0000","last_became_peered":"2024-01-27T02:37:50.806246+0000","last_unstale":"2024-01-27T02:37:51.828195+0000","last_undegraded":"2024-01-27T02:37:51.828195+0000","last_fullsized":"2024-01-27T02:37:51.828195+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:23.160028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.826753+0000","last_change":"2024-01-27T02:37:51.826505+0000","last_active":"2024-01-27T02:37:51.826753+0000","last_peered":"2024-01-27T02:37:51.826753+0000","last_clean":"2024-01-27T02:37:51.826753+0000","last_became_active":"2024-01-27T02:37:51.825678+0000","last_became_peered":"2024-01-27T02:37:51.825678+0000","last_unstale":"2024-01-27T02:37:51.826753+0000","last_undegraded":"2024-01-27T02:37:51.826753+0000","last_fullsized":"2024-01-27T02:37:51.826753+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:25.813930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.813125+0000","last_change":"2024-01-27T02:37:50.805856+0000","last_active":"2024-01-27T02:37:51.813125+0000","last_peered":"2024-01-27T02:37:51.813125+0000","last_clean":"2024-01-27T02:37:51.813125+0000","last_became_active":"2024-01-27T02:37:50.805568+0000","last_became_peered":"2024-01-27T02:37:50.805568+0000","last_unstale":"2024-01-27T02:37:51.813125+0000","last_undegraded":"2024-01-27T02:37:51.813125+0000","last_fullsized":"2024-01-27T02:37:51.813125+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:11:46.349861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.827322+0000","last_change":"2024-01-27T02:37:51.826716+0000","last_active":"2024-01-27T02:37:51.827322+0000","last_peered":"2024-01-27T02:37:51.827322+0000","last_clean":"2024-01-27T02:37:51.827322+0000","last_became_active":"2024-01-27T02:37:51.826267+0000","last_became_peered":"2024-01-27T02:37:51.826267+0000","last_unstale":"2024-01-27T02:37:51.827322+0000","last_undegraded":"2024-01-27T02:37:51.827322+0000","last_fullsized":"2024-01-27T02:37:51.827322+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:18:23.945261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"116'3","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.832239+0000","last_change":"2024-01-27T02:37:50.809623+0000","last_active":"2024-01-27T02:37:51.832239+0000","last_peered":"2024-01-27T02:37:51.832239+0000","last_clean":"2024-01-27T02:37:51.832239+0000","last_became_active":"2024-01-27T02:37:50.809447+0000","last_became_peered":"2024-01-27T02:37:50.809447+0000","last_unstale":"2024-01-27T02:37:51.832239+0000","last_undegraded":"2024-01-27T02:37:51.832239+0000","last_fullsized":"2024-01-27T02:37:51.832239+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:50.938489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.832898+0000","last_change":"2024-01-27T02:37:51.832799+0000","last_active":"2024-01-27T02:37:51.832898+0000","last_peered":"2024-01-27T02:37:51.832898+0000","last_clean":"2024-01-27T02:37:51.832898+0000","last_became_active":"2024-01-27T02:37:51.832558+0000","last_became_peered":"2024-01-27T02:37:51.832558+0000","last_unstale":"2024-01-27T02:37:51.832898+0000","last_undegraded":"2024-01-27T02:37:51.832898+0000","last_fullsized":"2024-01-27T02:37:51.832898+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:18:23.945261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"116'6","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.840033+0000","last_change":"2024-01-27T02:37:50.814137+0000","last_active":"2024-01-27T02:37:51.840033+0000","last_peered":"2024-01-27T02:37:51.840033+0000","last_clean":"2024-01-27T02:37:51.840033+0000","last_became_active":"2024-01-27T02:37:50.813898+0000","last_became_peered":"2024-01-27T02:37:50.813898+0000","last_unstale":"2024-01-27T02:37:51.840033+0000","last_undegraded":"2024-01-27T02:37:51.840033+0000","last_fullsized":"2024-01-27T02:37:51.840033+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:56:37.175345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.830778+0000","last_change":"2024-01-27T02:37:51.830650+0000","last_active":"2024-01-27T02:37:51.830778+0000","last_peered":"2024-01-27T02:37:51.830778+0000","last_clean":"2024-01-27T02:37:51.830778+0000","last_became_active":"2024-01-27T02:37:51.830400+0000","last_became_peered":"2024-01-27T02:37:51.830400+0000","last_unstale":"2024-01-27T02:37:51.830778+0000","last_undegraded":"2024-01-27T02:37:51.830778+0000","last_fullsized":"2024-01-27T02:37:51.830778+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:10:19.964021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.830254+0000","last_change":"2024-01-27T02:37:50.813470+0000","last_active":"2024-01-27T02:37:51.830254+0000","last_peered":"2024-01-27T02:37:51.830254+0000","last_clean":"2024-01-27T02:37:51.830254+0000","last_became_active":"2024-01-27T02:37:50.812014+0000","last_became_peered":"2024-01-27T02:37:50.812014+0000","last_unstale":"2024-01-27T02:37:51.830254+0000","last_undegraded":"2024-01-27T02:37:51.830254+0000","last_fullsized":"2024-01-27T02:37:51.830254+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:56:29.266154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.835233+0000","last_change":"2024-01-27T02:37:51.835063+0000","last_active":"2024-01-27T02:37:51.835233+0000","last_peered":"2024-01-27T02:37:51.835233+0000","last_clean":"2024-01-27T02:37:51.835233+0000","last_became_active":"2024-01-27T02:37:51.834540+0000","last_became_peered":"2024-01-27T02:37:51.834540+0000","last_unstale":"2024-01-27T02:37:51.835233+0000","last_undegraded":"2024-01-27T02:37:51.835233+0000","last_fullsized":"2024-01-27T02:37:51.835233+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:59:34.778013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.818741+0000","last_change":"2024-01-27T02:37:50.807923+0000","last_active":"2024-01-27T02:37:51.818741+0000","last_peered":"2024-01-27T02:37:51.818741+0000","last_clean":"2024-01-27T02:37:51.818741+0000","last_became_active":"2024-01-27T02:37:50.807417+0000","last_became_peered":"2024-01-27T02:37:50.807417+0000","last_unstale":"2024-01-27T02:37:51.818741+0000","last_undegraded":"2024-01-27T02:37:51.818741+0000","last_fullsized":"2024-01-27T02:37:51.818741+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:26:35.727576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.826428+0000","last_change":"2024-01-27T02:37:51.825934+0000","last_active":"2024-01-27T02:37:51.826428+0000","last_peered":"2024-01-27T02:37:51.826428+0000","last_clean":"2024-01-27T02:37:51.826428+0000","last_became_active":"2024-01-27T02:37:51.825688+0000","last_became_peered":"2024-01-27T02:37:51.825688+0000","last_unstale":"2024-01-27T02:37:51.826428+0000","last_undegraded":"2024-01-27T02:37:51.826428+0000","last_fullsized":"2024-01-27T02:37:51.826428+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:51.943504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.814416+0000","last_change":"2024-01-27T02:37:50.809421+0000","last_active":"2024-01-27T02:37:51.814416+0000","last_peered":"2024-01-27T02:37:51.814416+0000","last_clean":"2024-01-27T02:37:51.814416+0000","last_became_active":"2024-01-27T02:37:50.809169+0000","last_became_peered":"2024-01-27T02:37:50.809169+0000","last_unstale":"2024-01-27T02:37:51.814416+0000","last_undegraded":"2024-01-27T02:37:51.814416+0000","last_fullsized":"2024-01-27T02:37:51.814416+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:31:21.182217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.820200+0000","last_change":"2024-01-27T02:37:51.820121+0000","last_active":"2024-01-27T02:37:51.820200+0000","last_peered":"2024-01-27T02:37:51.820200+0000","last_clean":"2024-01-27T02:37:51.820200+0000","last_became_active":"2024-01-27T02:37:51.819763+0000","last_became_peered":"2024-01-27T02:37:51.819763+0000","last_unstale":"2024-01-27T02:37:51.820200+0000","last_undegraded":"2024-01-27T02:37:51.820200+0000","last_fullsized":"2024-01-27T02:37:51.820200+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:34:22.466946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"116'8","reported_seq":18,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.839283+0000","last_change":"2024-01-27T02:37:50.810368+0000","last_active":"2024-01-27T02:37:51.839283+0000","last_peered":"2024-01-27T02:37:51.839283+0000","last_clean":"2024-01-27T02:37:51.839283+0000","last_became_active":"2024-01-27T02:37:50.810089+0000","last_became_peered":"2024-01-27T02:37:50.810089+0000","last_unstale":"2024-01-27T02:37:51.839283+0000","last_undegraded":"2024-01-27T02:37:51.839283+0000","last_fullsized":"2024-01-27T02:37:51.839283+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:50:15.527559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.820861+0000","last_change":"2024-01-27T02:37:51.820762+0000","last_active":"2024-01-27T02:37:51.820861+0000","last_peered":"2024-01-27T02:37:51.820861+0000","last_clean":"2024-01-27T02:37:51.820861+0000","last_became_active":"2024-01-27T02:37:51.820515+0000","last_became_peered":"2024-01-27T02:37:51.820515+0000","last_unstale":"2024-01-27T02:37:51.820861+0000","last_undegraded":"2024-01-27T02:37:51.820861+0000","last_fullsized":"2024-01-27T02:37:51.820861+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:25:00.247752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.830967+0000","last_change":"2024-01-27T02:37:50.803536+0000","last_active":"2024-01-27T02:37:51.830967+0000","last_peered":"2024-01-27T02:37:51.830967+0000","last_clean":"2024-01-27T02:37:51.830967+0000","last_became_active":"2024-01-27T02:37:50.803332+0000","last_became_peered":"2024-01-27T02:37:50.803332+0000","last_unstale":"2024-01-27T02:37:51.830967+0000","last_undegraded":"2024-01-27T02:37:51.830967+0000","last_fullsized":"2024-01-27T02:37:51.830967+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:05:27.161024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.825834+0000","last_change":"2024-01-27T02:37:51.825726+0000","last_active":"2024-01-27T02:37:51.825834+0000","last_peered":"2024-01-27T02:37:51.825834+0000","last_clean":"2024-01-27T02:37:51.825834+0000","last_became_active":"2024-01-27T02:37:51.825411+0000","last_became_peered":"2024-01-27T02:37:51.825411+0000","last_unstale":"2024-01-27T02:37:51.825834+0000","last_undegraded":"2024-01-27T02:37:51.825834+0000","last_fullsized":"2024-01-27T02:37:51.825834+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:16:35.799353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.821259+0000","last_change":"2024-01-27T02:37:50.802000+0000","last_active":"2024-01-27T02:37:51.821259+0000","last_peered":"2024-01-27T02:37:51.821259+0000","last_clean":"2024-01-27T02:37:51.821259+0000","last_became_active":"2024-01-27T02:37:50.801755+0000","last_became_peered":"2024-01-27T02:37:50.801755+0000","last_unstale":"2024-01-27T02:37:51.821259+0000","last_undegraded":"2024-01-27T02:37:51.821259+0000","last_fullsized":"2024-01-27T02:37:51.821259+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:54:56.999817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.831050+0000","last_change":"2024-01-27T02:37:51.830287+0000","last_active":"2024-01-27T02:37:51.831050+0000","last_peered":"2024-01-27T02:37:51.831050+0000","last_clean":"2024-01-27T02:37:51.831050+0000","last_became_active":"2024-01-27T02:37:51.829998+0000","last_became_peered":"2024-01-27T02:37:51.829998+0000","last_unstale":"2024-01-27T02:37:51.831050+0000","last_undegraded":"2024-01-27T02:37:51.831050+0000","last_fullsized":"2024-01-27T02:37:51.831050+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:09:28.627404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.825846+0000","last_change":"2024-01-27T02:37:50.812532+0000","last_active":"2024-01-27T02:37:51.825846+0000","last_peered":"2024-01-27T02:37:51.825846+0000","last_clean":"2024-01-27T02:37:51.825846+0000","last_became_active":"2024-01-27T02:37:50.812209+0000","last_became_peered":"2024-01-27T02:37:50.812209+0000","last_unstale":"2024-01-27T02:37:51.825846+0000","last_undegraded":"2024-01-27T02:37:51.825846+0000","last_fullsized":"2024-01-27T02:37:51.825846+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:11:46.349861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.832190+0000","last_change":"2024-01-27T02:37:51.831671+0000","last_active":"2024-01-27T02:37:51.832190+0000","last_peered":"2024-01-27T02:37:51.832190+0000","last_clean":"2024-01-27T02:37:51.832190+0000","last_became_active":"2024-01-27T02:37:51.830906+0000","last_became_peered":"2024-01-27T02:37:51.830906+0000","last_unstale":"2024-01-27T02:37:51.832190+0000","last_undegraded":"2024-01-27T02:37:51.832190+0000","last_fullsized":"2024-01-27T02:37:51.832190+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:22:19.315343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.834659+0000","last_change":"2024-01-27T02:37:50.813813+0000","last_active":"2024-01-27T02:37:51.834659+0000","last_peered":"2024-01-27T02:37:51.834659+0000","last_clean":"2024-01-27T02:37:51.834659+0000","last_became_active":"2024-01-27T02:37:50.813235+0000","last_became_peered":"2024-01-27T02:37:50.813235+0000","last_unstale":"2024-01-27T02:37:51.834659+0000","last_undegraded":"2024-01-27T02:37:51.834659+0000","last_fullsized":"2024-01-27T02:37:51.834659+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:33:13.686128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.828983+0000","last_change":"2024-01-27T02:37:51.828132+0000","last_active":"2024-01-27T02:37:51.828983+0000","last_peered":"2024-01-27T02:37:51.828983+0000","last_clean":"2024-01-27T02:37:51.828983+0000","last_became_active":"2024-01-27T02:37:51.827879+0000","last_became_peered":"2024-01-27T02:37:51.827879+0000","last_unstale":"2024-01-27T02:37:51.828983+0000","last_undegraded":"2024-01-27T02:37:51.828983+0000","last_fullsized":"2024-01-27T02:37:51.828983+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:58:13.538507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.822138+0000","last_change":"2024-01-27T02:37:50.804058+0000","last_active":"2024-01-27T02:37:51.822138+0000","last_peered":"2024-01-27T02:37:51.822138+0000","last_clean":"2024-01-27T02:37:51.822138+0000","last_became_active":"2024-01-27T02:37:50.803802+0000","last_became_peered":"2024-01-27T02:37:50.803802+0000","last_unstale":"2024-01-27T02:37:51.822138+0000","last_undegraded":"2024-01-27T02:37:51.822138+0000","last_fullsized":"2024-01-27T02:37:51.822138+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:01:24.386631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.828920+0000","last_change":"2024-01-27T02:37:51.824693+0000","last_active":"2024-01-27T02:37:51.828920+0000","last_peered":"2024-01-27T02:37:51.828920+0000","last_clean":"2024-01-27T02:37:51.828920+0000","last_became_active":"2024-01-27T02:37:51.824263+0000","last_became_peered":"2024-01-27T02:37:51.824263+0000","last_unstale":"2024-01-27T02:37:51.828920+0000","last_undegraded":"2024-01-27T02:37:51.828920+0000","last_fullsized":"2024-01-27T02:37:51.828920+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:50:08.414227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.822884+0000","last_change":"2024-01-27T02:37:50.806415+0000","last_active":"2024-01-27T02:37:51.822884+0000","last_peered":"2024-01-27T02:37:51.822884+0000","last_clean":"2024-01-27T02:37:51.822884+0000","last_became_active":"2024-01-27T02:37:50.806149+0000","last_became_peered":"2024-01-27T02:37:50.806149+0000","last_unstale":"2024-01-27T02:37:51.822884+0000","last_undegraded":"2024-01-27T02:37:51.822884+0000","last_fullsized":"2024-01-27T02:37:51.822884+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:04:39.266345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.836957+0000","last_change":"2024-01-27T02:37:51.836898+0000","last_active":"2024-01-27T02:37:51.836957+0000","last_peered":"2024-01-27T02:37:51.836957+0000","last_clean":"2024-01-27T02:37:51.836957+0000","last_became_active":"2024-01-27T02:37:51.836232+0000","last_became_peered":"2024-01-27T02:37:51.836232+0000","last_unstale":"2024-01-27T02:37:51.836957+0000","last_undegraded":"2024-01-27T02:37:51.836957+0000","last_fullsized":"2024-01-27T02:37:51.836957+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:33:08.282783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.822894+0000","last_change":"2024-01-27T02:37:50.809566+0000","last_active":"2024-01-27T02:37:51.822894+0000","last_peered":"2024-01-27T02:37:51.822894+0000","last_clean":"2024-01-27T02:37:51.822894+0000","last_became_active":"2024-01-27T02:37:50.809276+0000","last_became_peered":"2024-01-27T02:37:50.809276+0000","last_unstale":"2024-01-27T02:37:51.822894+0000","last_undegraded":"2024-01-27T02:37:51.822894+0000","last_fullsized":"2024-01-27T02:37:51.822894+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:24:12.091569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.830289+0000","last_change":"2024-01-27T02:37:51.827097+0000","last_active":"2024-01-27T02:37:51.830289+0000","last_peered":"2024-01-27T02:37:51.830289+0000","last_clean":"2024-01-27T02:37:51.830289+0000","last_became_active":"2024-01-27T02:37:51.826483+0000","last_became_peered":"2024-01-27T02:37:51.826483+0000","last_unstale":"2024-01-27T02:37:51.830289+0000","last_undegraded":"2024-01-27T02:37:51.830289+0000","last_fullsized":"2024-01-27T02:37:51.830289+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:29:07.353494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.828305+0000","last_change":"2024-01-27T02:37:50.805793+0000","last_active":"2024-01-27T02:37:51.828305+0000","last_peered":"2024-01-27T02:37:51.828305+0000","last_clean":"2024-01-27T02:37:51.828305+0000","last_became_active":"2024-01-27T02:37:50.805524+0000","last_became_peered":"2024-01-27T02:37:50.805524+0000","last_unstale":"2024-01-27T02:37:51.828305+0000","last_undegraded":"2024-01-27T02:37:51.828305+0000","last_fullsized":"2024-01-27T02:37:51.828305+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:57:33.113268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.832749+0000","last_change":"2024-01-27T02:37:51.832632+0000","last_active":"2024-01-27T02:37:51.832749+0000","last_peered":"2024-01-27T02:37:51.832749+0000","last_clean":"2024-01-27T02:37:51.832749+0000","last_became_active":"2024-01-27T02:37:51.832311+0000","last_became_peered":"2024-01-27T02:37:51.832311+0000","last_unstale":"2024-01-27T02:37:51.832749+0000","last_undegraded":"2024-01-27T02:37:51.832749+0000","last_fullsized":"2024-01-27T02:37:51.832749+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:35:24.076704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.826105+0000","last_change":"2024-01-27T02:37:50.814412+0000","last_active":"2024-01-27T02:37:51.826105+0000","last_peered":"2024-01-27T02:37:51.826105+0000","last_clean":"2024-01-27T02:37:51.826105+0000","last_became_active":"2024-01-27T02:37:50.813560+0000","last_became_peered":"2024-01-27T02:37:50.813560+0000","last_unstale":"2024-01-27T02:37:51.826105+0000","last_undegraded":"2024-01-27T02:37:51.826105+0000","last_fullsized":"2024-01-27T02:37:51.826105+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:27.195705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.835195+0000","last_change":"2024-01-27T02:37:51.834798+0000","last_active":"2024-01-27T02:37:51.835195+0000","last_peered":"2024-01-27T02:37:51.835195+0000","last_clean":"2024-01-27T02:37:51.835195+0000","last_became_active":"2024-01-27T02:37:51.834048+0000","last_became_peered":"2024-01-27T02:37:51.834048+0000","last_unstale":"2024-01-27T02:37:51.835195+0000","last_undegraded":"2024-01-27T02:37:51.835195+0000","last_fullsized":"2024-01-27T02:37:51.835195+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:56:38.180360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.824580+0000","last_change":"2024-01-27T02:37:50.808731+0000","last_active":"2024-01-27T02:37:51.824580+0000","last_peered":"2024-01-27T02:37:51.824580+0000","last_clean":"2024-01-27T02:37:51.824580+0000","last_became_active":"2024-01-27T02:37:50.808482+0000","last_became_peered":"2024-01-27T02:37:50.808482+0000","last_unstale":"2024-01-27T02:37:51.824580+0000","last_undegraded":"2024-01-27T02:37:51.824580+0000","last_fullsized":"2024-01-27T02:37:51.824580+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:50.938489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.827341+0000","last_change":"2024-01-27T02:37:51.827238+0000","last_active":"2024-01-27T02:37:51.827341+0000","last_peered":"2024-01-27T02:37:51.827341+0000","last_clean":"2024-01-27T02:37:51.827341+0000","last_became_active":"2024-01-27T02:37:51.826966+0000","last_became_peered":"2024-01-27T02:37:51.826966+0000","last_unstale":"2024-01-27T02:37:51.827341+0000","last_undegraded":"2024-01-27T02:37:51.827341+0000","last_fullsized":"2024-01-27T02:37:51.827341+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:04:02.332379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.818574+0000","last_change":"2024-01-27T02:37:50.808670+0000","last_active":"2024-01-27T02:37:51.818574+0000","last_peered":"2024-01-27T02:37:51.818574+0000","last_clean":"2024-01-27T02:37:51.818574+0000","last_became_active":"2024-01-27T02:37:50.808428+0000","last_became_peered":"2024-01-27T02:37:50.808428+0000","last_unstale":"2024-01-27T02:37:51.818574+0000","last_undegraded":"2024-01-27T02:37:51.818574+0000","last_fullsized":"2024-01-27T02:37:51.818574+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:57:56.062671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.820070+0000","last_change":"2024-01-27T02:37:51.819988+0000","last_active":"2024-01-27T02:37:51.820070+0000","last_peered":"2024-01-27T02:37:51.820070+0000","last_clean":"2024-01-27T02:37:51.820070+0000","last_became_active":"2024-01-27T02:37:51.819476+0000","last_became_peered":"2024-01-27T02:37:51.819476+0000","last_unstale":"2024-01-27T02:37:51.820070+0000","last_undegraded":"2024-01-27T02:37:51.820070+0000","last_fullsized":"2024-01-27T02:37:51.820070+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:28:08.524359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.822992+0000","last_change":"2024-01-27T02:37:50.813250+0000","last_active":"2024-01-27T02:37:51.822992+0000","last_peered":"2024-01-27T02:37:51.822992+0000","last_clean":"2024-01-27T02:37:51.822992+0000","last_became_active":"2024-01-27T02:37:50.813038+0000","last_became_peered":"2024-01-27T02:37:50.813038+0000","last_unstale":"2024-01-27T02:37:51.822992+0000","last_undegraded":"2024-01-27T02:37:51.822992+0000","last_fullsized":"2024-01-27T02:37:51.822992+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:25.524253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.819116+0000","last_change":"2024-01-27T02:37:51.819051+0000","last_active":"2024-01-27T02:37:51.819116+0000","last_peered":"2024-01-27T02:37:51.819116+0000","last_clean":"2024-01-27T02:37:51.819116+0000","last_became_active":"2024-01-27T02:37:51.818770+0000","last_became_peered":"2024-01-27T02:37:51.818770+0000","last_unstale":"2024-01-27T02:37:51.819116+0000","last_undegraded":"2024-01-27T02:37:51.819116+0000","last_fullsized":"2024-01-27T02:37:51.819116+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:14:51.191272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.817859+0000","last_change":"2024-01-27T02:37:50.803902+0000","last_active":"2024-01-27T02:37:51.817859+0000","last_peered":"2024-01-27T02:37:51.817859+0000","last_clean":"2024-01-27T02:37:51.817859+0000","last_became_active":"2024-01-27T02:37:50.803626+0000","last_became_peered":"2024-01-27T02:37:50.803626+0000","last_unstale":"2024-01-27T02:37:51.817859+0000","last_undegraded":"2024-01-27T02:37:51.817859+0000","last_fullsized":"2024-01-27T02:37:51.817859+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:25.524253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.821790+0000","last_change":"2024-01-27T02:37:51.821433+0000","last_active":"2024-01-27T02:37:51.821790+0000","last_peered":"2024-01-27T02:37:51.821790+0000","last_clean":"2024-01-27T02:37:51.821790+0000","last_became_active":"2024-01-27T02:37:51.820831+0000","last_became_peered":"2024-01-27T02:37:51.820831+0000","last_unstale":"2024-01-27T02:37:51.821790+0000","last_undegraded":"2024-01-27T02:37:51.821790+0000","last_fullsized":"2024-01-27T02:37:51.821790+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:24.165044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.814170+0000","last_change":"2024-01-27T02:37:50.804116+0000","last_active":"2024-01-27T02:37:51.814170+0000","last_peered":"2024-01-27T02:37:51.814170+0000","last_clean":"2024-01-27T02:37:51.814170+0000","last_became_active":"2024-01-27T02:37:50.803880+0000","last_became_peered":"2024-01-27T02:37:50.803880+0000","last_unstale":"2024-01-27T02:37:51.814170+0000","last_undegraded":"2024-01-27T02:37:51.814170+0000","last_fullsized":"2024-01-27T02:37:51.814170+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:50.938489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.828702+0000","last_change":"2024-01-27T02:37:51.828506+0000","last_active":"2024-01-27T02:37:51.828702+0000","last_peered":"2024-01-27T02:37:51.828702+0000","last_clean":"2024-01-27T02:37:51.828702+0000","last_became_active":"2024-01-27T02:37:51.828120+0000","last_became_peered":"2024-01-27T02:37:51.828120+0000","last_unstale":"2024-01-27T02:37:51.828702+0000","last_undegraded":"2024-01-27T02:37:51.828702+0000","last_fullsized":"2024-01-27T02:37:51.828702+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:17:03.370160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.818190+0000","last_change":"2024-01-27T02:37:50.807085+0000","last_active":"2024-01-27T02:37:51.818190+0000","last_peered":"2024-01-27T02:37:51.818190+0000","last_clean":"2024-01-27T02:37:51.818190+0000","last_became_active":"2024-01-27T02:37:50.804683+0000","last_became_peered":"2024-01-27T02:37:50.804683+0000","last_unstale":"2024-01-27T02:37:51.818190+0000","last_undegraded":"2024-01-27T02:37:51.818190+0000","last_fullsized":"2024-01-27T02:37:51.818190+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:49:46.671831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.829974+0000","last_change":"2024-01-27T02:37:51.829674+0000","last_active":"2024-01-27T02:37:51.829974+0000","last_peered":"2024-01-27T02:37:51.829974+0000","last_clean":"2024-01-27T02:37:51.829974+0000","last_became_active":"2024-01-27T02:37:51.828989+0000","last_became_peered":"2024-01-27T02:37:51.828989+0000","last_unstale":"2024-01-27T02:37:51.829974+0000","last_undegraded":"2024-01-27T02:37:51.829974+0000","last_fullsized":"2024-01-27T02:37:51.829974+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:37:48.404117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.828244+0000","last_change":"2024-01-27T02:37:50.809913+0000","last_active":"2024-01-27T02:37:51.828244+0000","last_peered":"2024-01-27T02:37:51.828244+0000","last_clean":"2024-01-27T02:37:51.828244+0000","last_became_active":"2024-01-27T02:37:50.809679+0000","last_became_peered":"2024-01-27T02:37:50.809679+0000","last_unstale":"2024-01-27T02:37:51.828244+0000","last_undegraded":"2024-01-27T02:37:51.828244+0000","last_fullsized":"2024-01-27T02:37:51.828244+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:27.195705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":247,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.825230+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:37:51.825230+0000","last_peered":"2024-01-27T02:37:51.825230+0000","last_clean":"2024-01-27T02:37:51.825230+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:37:51.825230+0000","last_undegraded":"2024-01-27T02:37:51.825230+0000","last_fullsized":"2024-01-27T02:37:51.825230+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1199,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194113,"statfs":{"total":200000000,"available":198772024,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71299999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.59099999999999997,"15min":0.58999999999999997},"min":{"1min":0.49099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77400000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55400000000000005}]},{"osd":1,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59699999999999998,"15min":0.59799999999999998},"min":{"1min":0.46100000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.44400000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.753,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.51100000000000001}]},{"osd":2,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58299999999999996},"min":{"1min":0.432,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77700000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.56499999999999995}]},{"osd":3,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60499999999999998,"15min":0.59299999999999997},"min":{"1min":0.495,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.44},{"interface":"front","average":{"1min":0.628,"5min":0.60999999999999999,"15min":0.59499999999999997},"min":{"1min":0.5,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85099999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.48199999999999998}]},{"osd":4,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.47499999999999998,"15min":0.48699999999999999},"min":{"1min":0.32400000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63300000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.495,"15min":0.51300000000000001},"min":{"1min":0.35999999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68899999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.46600000000000003}]},{"osd":5,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51800000000000002,"15min":0.53600000000000003},"min":{"1min":0.39000000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.56399999999999995},"min":{"1min":0.40100000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.58699999999999997}]},{"osd":6,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.59499999999999997},"min":{"1min":0.40999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.751,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.57699999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.81599999999999995,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.53000000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574922,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":623,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194689,"statfs":{"total":200000000,"available":199361880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58699999999999997,"15min":0.52100000000000002},"min":{"1min":0.45000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.52700000000000002},"min":{"1min":0.434,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71899999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.622}]},{"osd":1,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.51800000000000002},"min":{"1min":0.41999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79700000000000004,"5min":0.874,"15min":0.874},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.60599999999999998,"15min":0.55500000000000005},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.72699999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.41199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.625,"15min":0.58299999999999996},"min":{"1min":0.45100000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79700000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.68000000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58199999999999996,"15min":0.55100000000000005},"min":{"1min":0.47599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.76800000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61499999999999999,"15min":0.57299999999999995},"min":{"1min":0.48499999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.78100000000000003,"5min":1.262,"15min":1.262},"last":0.54000000000000004}]},{"osd":4,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.34200000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52700000000000002,"15min":0.55300000000000005},"min":{"1min":0.41199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68300000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.51100000000000001}]},{"osd":5,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53200000000000003,"15min":0.53500000000000003},"min":{"1min":0.38800000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70799999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.60599999999999998},"min":{"1min":0.40899999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70799999999999996,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.58899999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60599999999999998,"15min":0.61599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73699999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71099999999999997,"5min":1.038,"15min":1.038},"last":0.60299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":622,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194689,"statfs":{"total":200000000,"available":199362504,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39700000000000002,"15min":0.377},"min":{"1min":0.27000000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.41999999999999998,"15min":0.40000000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.40999999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.40200000000000002,"15min":0.36599999999999999},"min":{"1min":0.28000000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.434},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45400000000000001,"15min":0.43099999999999999},"min":{"1min":0.30199999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.35499999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48999999999999999,"15min":0.46899999999999997},"min":{"1min":0.35799999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.47499999999999998,"15min":0.45800000000000002},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.36399999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.66200000000000003,"15min":0.63400000000000001},"min":{"1min":0.442,"5min":0.373,"15min":0.373},"max":{"1min":3.2890000000000001,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65800000000000003,"15min":0.63600000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.57299999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.68500000000000005,"15min":0.63700000000000001},"min":{"1min":0.50600000000000001,"5min":0.44,"15min":0.44},"max":{"1min":3.4500000000000002,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.68999999999999995,"15min":0.67600000000000005},"min":{"1min":0.48299999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.181,"5min":3.181,"15min":3.181},"last":0.63500000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70999999999999996,"15min":0.69599999999999995},"min":{"1min":0.52100000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96399999999999997,"5min":1.024,"15min":1.024},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.71299999999999997,"15min":0.67900000000000005},"min":{"1min":0.53400000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.60899999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.72399999999999998,"15min":0.68600000000000005},"min":{"1min":0.56100000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.72499999999999998,"15min":0.69199999999999995},"min":{"1min":0.57799999999999996,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.68500000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574922,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1199,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194112,"statfs":{"total":200000000,"available":198771582,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.38900000000000001},"min":{"1min":0.28100000000000003,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.66600000000000004,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27800000000000002,"5min":0.223,"15min":0.223},"max":{"1min":0.65200000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.36399999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.436,"15min":0.42199999999999999},"min":{"1min":0.32800000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56299999999999994,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.44900000000000001,"15min":0.435},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58999999999999997,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.434}]},{"osd":3,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42999999999999999,"15min":0.432},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.63600000000000001,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.35299999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.63200000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42199999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.65500000000000003,"15min":0.64000000000000001},"min":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45900000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89700000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.61299999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67500000000000004,"15min":0.65600000000000003},"min":{"1min":0.48399999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.86499999999999999,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67700000000000005,"15min":0.68200000000000005},"min":{"1min":0.5,"5min":0.44,"15min":0.44},"max":{"1min":0.96599999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.58999999999999997}]},{"osd":6,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.56200000000000006,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91300000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69299999999999995,"15min":0.67900000000000005},"min":{"1min":0.56599999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94199999999999995,"5min":1.04,"15min":1.04},"last":0.58099999999999996}]},{"osd":7,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.55300000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.52100000000000002,"5min":0.498,"15min":0.498},"max":{"1min":0.95799999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.56999999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":622,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194689,"statfs":{"total":200000000,"available":199362470,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.41699999999999998,"15min":0.40999999999999998},"min":{"1min":0.28000000000000003,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56599999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.50900000000000001,"15min":0.63700000000000001},"min":{"1min":0.34499999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72799999999999998,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":0.36299999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.434,"15min":0.42399999999999999},"min":{"1min":0.32700000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.434,"5min":0.44,"15min":0.44700000000000001},"min":{"1min":0.29799999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.65300000000000002,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.41899999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.41499999999999998},"min":{"1min":0.30399999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.65000000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76500000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.45200000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68200000000000005,"15min":0.65600000000000003},"min":{"1min":0.53600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73399999999999999,"15min":0.79900000000000004},"min":{"1min":0.54900000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.995,"5min":3.581,"15min":3.581},"last":0.627}]},{"osd":5,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68000000000000005,"15min":0.74399999999999999},"min":{"1min":0.502,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90500000000000003,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67700000000000005,"15min":0.67100000000000004},"min":{"1min":0.501,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85899999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.60699999999999998}]},{"osd":6,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.53800000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93999999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0189999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.58999999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.53600000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70299999999999996,"15min":0.69099999999999995},"min":{"1min":0.625,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.64100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":624,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194688,"statfs":{"total":200000000,"available":199360946,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.38800000000000001,"15min":0.35899999999999999},"min":{"1min":0.255,"5min":0.253,"15min":0.253},"max":{"1min":0.56200000000000006,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40600000000000003,"15min":0.39500000000000002},"min":{"1min":0.29199999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.626,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.41699999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.27400000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66300000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.45600000000000002,"15min":0.42399999999999999},"min":{"1min":0.30199999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45800000000000002}]},{"osd":2,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46100000000000002,"15min":0.43099999999999999},"min":{"1min":0.33700000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.48399999999999999,"15min":0.45000000000000001},"min":{"1min":0.378,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.47799999999999998}]},{"osd":4,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.66900000000000004,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.6660000000000004,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.751},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64200000000000002,"15min":0.60599999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93300000000000005,"5min":1.012,"15min":1.012},"last":0.60299999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.61299999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90400000000000003,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.71599999999999997,"15min":0.68000000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":4.6849999999999996,"5min":4.6849999999999996,"15min":4.6849999999999996},"last":0.85099999999999998}]},{"osd":6,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90100000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.47099999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.61599999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.71699999999999997,"15min":0.65400000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":4.7130000000000001,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68500000000000005,"15min":0.66700000000000004},"min":{"1min":0.495,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.82699999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":624,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194688,"statfs":{"total":200000000,"available":199360844,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.36899999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.73399999999999999,"5min":1.099,"15min":1.099},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.47799999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71099999999999997,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.57499999999999996}]},{"osd":1,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.39500000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72499999999999998,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59799999999999998,"15min":0.59499999999999997},"min":{"1min":0.40799999999999997,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77700000000000002,"5min":1.133,"15min":1.133},"last":0.58799999999999997}]},{"osd":2,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76300000000000001,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.44700000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.753,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.56599999999999995}]},{"osd":3,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.42799999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68700000000000006,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.50700000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.70799999999999996,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.59799999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.47599999999999998,"15min":0.48399999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.63400000000000001,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.34599999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68899999999999995,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.42099999999999999}]},{"osd":6,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.5},"min":{"1min":0.40100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.66600000000000004,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53800000000000003,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81200000000000006,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.45600000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53900000000000003,"15min":0.54900000000000004},"min":{"1min":0.38500000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.65400000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.432,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68100000000000005,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.44600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":622,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194689,"statfs":{"total":200000000,"available":199362434,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56999999999999995,"15min":0.59399999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68700000000000006,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.60099999999999998,"15min":0.57699999999999996},"min":{"1min":0.34499999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67100000000000004,"5min":3.242,"15min":3.242},"last":0.42899999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.625,"15min":0.60799999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70599999999999996,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.626,"15min":0.64900000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69299999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.58699999999999997}]},{"osd":2,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61599999999999999,"15min":0.59599999999999997},"min":{"1min":0.39100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.71499999999999997,"5min":3.04,"15min":3.04},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59399999999999997,"15min":0.57699999999999996},"min":{"1min":0.40100000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72499999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.55800000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.66500000000000004,"15min":0.67800000000000005},"min":{"1min":0.44400000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.65500000000000003,"5min":4.032,"15min":4.032},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.70399999999999996,"15min":0.67800000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.77000000000000002,"5min":4.282,"15min":4.282},"last":0.46000000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81399999999999995,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.51100000000000001,"15min":0.56499999999999995},"min":{"1min":0.36799999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.61499999999999999,"5min":1.101,"15min":1.101},"last":0.624}]},{"osd":6,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51300000000000001,"15min":0.55600000000000005},"min":{"1min":0.36499999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.629,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.505},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56499999999999995,"15min":0.58199999999999996},"min":{"1min":0.39100000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76800000000000002,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.61199999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.55400000000000005,"15min":0.56699999999999995},"min":{"1min":0.40200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.745,"5min":1.008,"15min":1.008},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.626,"15min":0.65800000000000003},"min":{"1min":0.38400000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84799999999999998,"5min":2,"15min":2},"last":0.64500000000000002}]}]}],"pool_statfs":[]}} 2024-01-27T02:38:00.718 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:38:00.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.859+0000 7f21c18b1700 1 Processor -- start 2024-01-27T02:38:00.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.859+0000 7f21c18b1700 1 -- start start 2024-01-27T02:38:00.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.860+0000 7f21c18b1700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f21bc15a390 0x7f21bc15e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:00.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.860+0000 7f21c18b1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21bc0d0da0 0x7f21bc0d1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:00.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.860+0000 7f21c18b1700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21bc0cc830 con 0x7f21bc159fc0 2024-01-27T02:38:00.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.860+0000 7f21c18b1700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21bc0d1780 con 0x7f21bc15a390 2024-01-27T02:38:00.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.860+0000 7f21c18b1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc0b35e0 con 0x7f21bc0d0da0 2024-01-27T02:38:00.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.860+0000 7f21baffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f21bc159fc0 0x7f21bc0b2ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50792/0 (socket says 172.21.15.7:50792) 2024-01-27T02:38:00.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.860+0000 7f21baffd700 1 -- 172.21.15.7:0/527628841 learned_addr learned my addr 172.21.15.7:0/527628841 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:00.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.860+0000 7f21ba7fc700 1 --2- 172.21.15.7:0/527628841 >> v2:172.21.15.188:3300/0 conn(0x7f21bc15a390 0x7f21bc15e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:00.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.860+0000 7f21b9ffb700 1 -- 172.21.15.7:0/527628841 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2635682689 0 0) 0x7f21bc0cc830 con 0x7f21bc159fc0 2024-01-27T02:38:00.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.860+0000 7f21b9ffb700 1 -- 172.21.15.7:0/527628841 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21a4003570 con 0x7f21bc159fc0 2024-01-27T02:38:00.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.860+0000 7f21bb7fe700 1 --2- 172.21.15.7:0/527628841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21bc0d0da0 0x7f21bc0d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:00.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21b9ffb700 1 -- 172.21.15.7:0/527628841 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3917744619 0 0) 0x7f21a4003570 con 0x7f21bc159fc0 2024-01-27T02:38:00.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21b9ffb700 1 -- 172.21.15.7:0/527628841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21bc0d0da0 msgr2=0x7f21bc0d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:00.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21b9ffb700 1 --2- 172.21.15.7:0/527628841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21bc0d0da0 0x7f21bc0d1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:00.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21b9ffb700 1 -- 172.21.15.7:0/527628841 >> v2:172.21.15.188:3300/0 conn(0x7f21bc15a390 msgr2=0x7f21bc15e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:00.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21b9ffb700 1 --2- 172.21.15.7:0/527628841 >> v2:172.21.15.188:3300/0 conn(0x7f21bc15a390 0x7f21bc15e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:00.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21b9ffb700 1 -- 172.21.15.7:0/527628841 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21bc0cbf00 con 0x7f21bc159fc0 2024-01-27T02:38:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21b9ffb700 1 -- 172.21.15.7:0/527628841 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f21ac001470 con 0x7f21bc159fc0 2024-01-27T02:38:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21b9ffb700 1 -- 172.21.15.7:0/527628841 >> v1:172.21.15.7:6789/0 conn(0x7f21bc159fc0 legacy=0x7f21bc0b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21b9ffb700 1 --2- 172.21.15.7:0/527628841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21a4003a60 0x7f21a4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21b9ffb700 1 --2- 172.21.15.7:0/527628841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21bc0d0da0 0x7f21a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21b9ffb700 1 --2- 172.21.15.7:0/527628841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21a4004d50 0x7f21a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21b9ffb700 1 -- 172.21.15.7:0/527628841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc0d1780 con 0x7f21a4003a60 2024-01-27T02:38:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21b9ffb700 1 -- 172.21.15.7:0/527628841 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc0b35e0 con 0x7f21a4004d50 2024-01-27T02:38:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21b9ffb700 1 -- 172.21.15.7:0/527628841 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a400ba60 con 0x7f21bc0d0da0 2024-01-27T02:38:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21ba7fc700 1 --2- 172.21.15.7:0/527628841 >> v2:172.21.15.188:3300/0 conn(0x7f21bc15a390 0x7f21bc15e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:38:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21baffd700 1 --2- 172.21.15.7:0/527628841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21a4003a60 0x7f21a4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21bb7fe700 1 --2- 172.21.15.7:0/527628841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21a4004d50 0x7f21a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21ba7fc700 1 --2- 172.21.15.7:0/527628841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21bc0d0da0 0x7f21a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21baffd700 1 -- 172.21.15.7:0/527628841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21bc0d0da0 msgr2=0x7f21a4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21baffd700 1 --2- 172.21.15.7:0/527628841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21bc0d0da0 0x7f21a4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21baffd700 1 -- 172.21.15.7:0/527628841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21a4004d50 msgr2=0x7f21a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21baffd700 1 --2- 172.21.15.7:0/527628841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21a4004d50 0x7f21a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21baffd700 1 -- 172.21.15.7:0/527628841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a4003570 con 0x7f21a4003a60 2024-01-27T02:38:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21bb7fe700 1 --2- 172.21.15.7:0/527628841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21a4004d50 0x7f21a40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.861+0000 7f21baffd700 1 --2- 172.21.15.7:0/527628841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21a4003a60 0x7f21a4003e70 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00c3c0 tx=0x7f21bc0cc7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0766f55fd38a797 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.862+0000 7f21b9ffb700 1 -- 172.21.15.7:0/527628841 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac00cc30 con 0x7f21a4003a60 2024-01-27T02:38:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.862+0000 7f21b9ffb700 1 -- 172.21.15.7:0/527628841 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21ac00cd90 con 0x7f21a4003a60 2024-01-27T02:38:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.862+0000 7f21b9ffb700 1 -- 172.21.15.7:0/527628841 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac005b70 con 0x7f21a4003a60 2024-01-27T02:38:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.862+0000 7f21c18b1700 1 -- 172.21.15.7:0/527628841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21a4003a60 msgr2=0x7f21a4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.862+0000 7f21c18b1700 1 --2- 172.21.15.7:0/527628841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21a4003a60 0x7f21a4003e70 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00c3c0 tx=0x7f21bc0cc7a0 comp rx=0 tx=0).stop 2024-01-27T02:38:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.862+0000 7f21bb7fe700 1 -- 172.21.15.7:0/527628841 reap_dead start 2024-01-27T02:38:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.862+0000 7f21c18b1700 1 -- 172.21.15.7:0/527628841 shutdown_connections 2024-01-27T02:38:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.862+0000 7f21c18b1700 1 -- 172.21.15.7:0/527628841 >> 172.21.15.7:0/527628841 conn(0x7f21bc0679b0 msgr2=0x7f21bc067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.862+0000 7f21c18b1700 1 -- 172.21.15.7:0/527628841 shutdown_connections 2024-01-27T02:38:00.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.862+0000 7f21c18b1700 1 -- 172.21.15.7:0/527628841 wait complete. 2024-01-27T02:38:00.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.864+0000 7f21c18b1700 1 Processor -- start 2024-01-27T02:38:00.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.864+0000 7f21c18b1700 1 -- start start 2024-01-27T02:38:00.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.865+0000 7f21c18b1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21a4003a60 0x7f21bc151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:00.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.865+0000 7f21c18b1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21bc0cfc40 0x7f21bc0d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:00.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.865+0000 7f21c18b1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21bc0c4850 0x7f21bc0c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:00.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.865+0000 7f21c18b1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc0d05f0 con 0x7f21bc0cfc40 2024-01-27T02:38:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.865+0000 7f21c18b1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc0c7710 con 0x7f21a4003a60 2024-01-27T02:38:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.865+0000 7f21c18b1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc0c6950 con 0x7f21bc0c4850 2024-01-27T02:38:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.865+0000 7f21ba7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21bc0cfc40 0x7f21bc0d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.865+0000 7f21ba7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21bc0cfc40 0x7f21bc0d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44206/0 (socket says 172.21.15.7:44206) 2024-01-27T02:38:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.865+0000 7f21ba7fc700 1 -- 172.21.15.7:0/3918561630 learned_addr learned my addr 172.21.15.7:0/3918561630 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.865+0000 7f21bb7fe700 1 --2- 172.21.15.7:0/3918561630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21bc0c4850 0x7f21bc0c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.865+0000 7f21baffd700 1 --2- 172.21.15.7:0/3918561630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21a4003a60 0x7f21bc151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.866+0000 7f21ba7fc700 1 -- 172.21.15.7:0/3918561630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21bc0c4850 msgr2=0x7f21bc0c4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.866+0000 7f21ba7fc700 1 --2- 172.21.15.7:0/3918561630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21bc0c4850 0x7f21bc0c4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.866+0000 7f21ba7fc700 1 -- 172.21.15.7:0/3918561630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21a4003a60 msgr2=0x7f21bc151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.866+0000 7f21ba7fc700 1 --2- 172.21.15.7:0/3918561630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21a4003a60 0x7f21bc151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.866+0000 7f21ba7fc700 1 -- 172.21.15.7:0/3918561630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21bc0c6c30 con 0x7f21bc0cfc40 2024-01-27T02:38:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.866+0000 7f21ba7fc700 1 --2- 172.21.15.7:0/3918561630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21bc0cfc40 0x7f21bc0d00b0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f21b00119f0 tx=0x7f21b0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.866+0000 7f21a2ffd700 1 -- 172.21.15.7:0/3918561630 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b000f860 con 0x7f21bc0cfc40 2024-01-27T02:38:00.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.866+0000 7f21a2ffd700 1 -- 172.21.15.7:0/3918561630 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b000f9c0 con 0x7f21bc0cfc40 2024-01-27T02:38:00.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.866+0000 7f21c18b1700 1 -- 172.21.15.7:0/3918561630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21bc0c6f10 con 0x7f21bc0cfc40 2024-01-27T02:38:00.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.866+0000 7f21a2ffd700 1 -- 172.21.15.7:0/3918561630 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b000fd40 con 0x7f21bc0cfc40 2024-01-27T02:38:00.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.866+0000 7f21c18b1700 1 -- 172.21.15.7:0/3918561630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21bc0c55c0 con 0x7f21bc0cfc40 2024-01-27T02:38:00.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.867+0000 7f21c18b1700 1 -- 172.21.15.7:0/3918561630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21bc0c73b0 con 0x7f21bc0cfc40 2024-01-27T02:38:00.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.868+0000 7f21a2ffd700 1 -- 172.21.15.7:0/3918561630 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f21b0015670 con 0x7f21bc0cfc40 2024-01-27T02:38:00.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.868+0000 7f21a2ffd700 1 --2- 172.21.15.7:0/3918561630 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f21a8066f30 0x7f21a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:00.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.868+0000 7f21a2ffd700 1 -- 172.21.15.7:0/3918561630 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f21b0084da0 con 0x7f21bc0cfc40 2024-01-27T02:38:00.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.868+0000 7f21baffd700 1 --2- 172.21.15.7:0/3918561630 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f21a8066f30 0x7f21a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:00.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.869+0000 7f21baffd700 1 --2- 172.21.15.7:0/3918561630 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f21a8066f30 0x7f21a80693e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00c1a0 tx=0x7f21ac015000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:00.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:00.873+0000 7f21a2ffd700 1 -- 172.21.15.7:0/3918561630 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f21b0052010 con 0x7f21bc0cfc40 2024-01-27T02:38:01.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.068+0000 7f21c18b1700 1 -- 172.21.15.7:0/3918561630 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f21bc0ce750 con 0x7f21a8066f30 2024-01-27T02:38:01.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.077+0000 7f21a2ffd700 1 -- 172.21.15.7:0/3918561630 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285635 (secure 0 0 0) 0x7f21bc0ce750 con 0x7f21a8066f30 2024-01-27T02:38:01.081 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:01.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.080+0000 7f21c18b1700 1 -- 172.21.15.7:0/3918561630 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f21a8066f30 msgr2=0x7f21a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:01.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.080+0000 7f21c18b1700 1 --2- 172.21.15.7:0/3918561630 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f21a8066f30 0x7f21a80693e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00c1a0 tx=0x7f21ac015000 comp rx=0 tx=0).stop 2024-01-27T02:38:01.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.080+0000 7f21c18b1700 1 -- 172.21.15.7:0/3918561630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21bc0cfc40 msgr2=0x7f21bc0d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.080+0000 7f21c18b1700 1 --2- 172.21.15.7:0/3918561630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21bc0cfc40 0x7f21bc0d00b0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f21b00119f0 tx=0x7f21b0008040 comp rx=0 tx=0).stop 2024-01-27T02:38:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.080+0000 7f21c18b1700 1 -- 172.21.15.7:0/3918561630 shutdown_connections 2024-01-27T02:38:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.080+0000 7f21c18b1700 1 --2- 172.21.15.7:0/3918561630 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f21a8066f30 0x7f21a80693e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.080+0000 7f21c18b1700 1 --2- 172.21.15.7:0/3918561630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21bc0c4850 0x7f21bc0c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.080+0000 7f21c18b1700 1 --2- 172.21.15.7:0/3918561630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21a4003a60 0x7f21bc151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.080+0000 7f21c18b1700 1 --2- 172.21.15.7:0/3918561630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21bc0cfc40 0x7f21bc0d00b0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.080+0000 7f21c18b1700 1 -- 172.21.15.7:0/3918561630 >> 172.21.15.7:0/3918561630 conn(0x7f21bc0679b0 msgr2=0x7f21bc0b6330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.080+0000 7f21c18b1700 1 -- 172.21.15.7:0/3918561630 shutdown_connections 2024-01-27T02:38:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.080+0000 7f21c18b1700 1 -- 172.21.15.7:0/3918561630 wait complete. 2024-01-27T02:38:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:38:01.094 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-01-27T02:37:59.659535+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6138,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556361,"statfs":{"total":1600000000,"available":1593714684,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002178"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.832733+0000","last_change":"2024-01-27T02:37:51.831826+0000","last_active":"2024-01-27T02:37:51.832733+0000","last_peered":"2024-01-27T02:37:51.832733+0000","last_clean":"2024-01-27T02:37:51.832733+0000","last_became_active":"2024-01-27T02:37:51.828656+0000","last_became_peered":"2024-01-27T02:37:51.828656+0000","last_unstale":"2024-01-27T02:37:51.832733+0000","last_undegraded":"2024-01-27T02:37:51.832733+0000","last_fullsized":"2024-01-27T02:37:51.832733+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:31:22.187233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.830345+0000","last_change":"2024-01-27T02:37:51.827439+0000","last_active":"2024-01-27T02:37:51.830345+0000","last_peered":"2024-01-27T02:37:51.830345+0000","last_clean":"2024-01-27T02:37:51.830345+0000","last_became_active":"2024-01-27T02:37:51.827041+0000","last_became_peered":"2024-01-27T02:37:51.827041+0000","last_unstale":"2024-01-27T02:37:51.830345+0000","last_undegraded":"2024-01-27T02:37:51.830345+0000","last_fullsized":"2024-01-27T02:37:51.830345+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:33:03.228144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.835215+0000","last_change":"2024-01-27T02:37:51.835020+0000","last_active":"2024-01-27T02:37:51.835215+0000","last_peered":"2024-01-27T02:37:51.835215+0000","last_clean":"2024-01-27T02:37:51.835215+0000","last_became_active":"2024-01-27T02:37:51.834822+0000","last_became_peered":"2024-01-27T02:37:51.834822+0000","last_unstale":"2024-01-27T02:37:51.835215+0000","last_undegraded":"2024-01-27T02:37:51.835215+0000","last_fullsized":"2024-01-27T02:37:51.835215+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:56:06.441719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.825809+0000","last_change":"2024-01-27T02:37:51.824465+0000","last_active":"2024-01-27T02:37:51.825809+0000","last_peered":"2024-01-27T02:37:51.825809+0000","last_clean":"2024-01-27T02:37:51.825809+0000","last_became_active":"2024-01-27T02:37:51.824096+0000","last_became_peered":"2024-01-27T02:37:51.824096+0000","last_unstale":"2024-01-27T02:37:51.825809+0000","last_undegraded":"2024-01-27T02:37:51.825809+0000","last_fullsized":"2024-01-27T02:37:51.825809+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:45:13.633104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.827665+0000","last_change":"2024-01-27T02:37:51.827575+0000","last_active":"2024-01-27T02:37:51.827665+0000","last_peered":"2024-01-27T02:37:51.827665+0000","last_clean":"2024-01-27T02:37:51.827665+0000","last_became_active":"2024-01-27T02:37:51.827302+0000","last_became_peered":"2024-01-27T02:37:51.827302+0000","last_unstale":"2024-01-27T02:37:51.827665+0000","last_undegraded":"2024-01-27T02:37:51.827665+0000","last_fullsized":"2024-01-27T02:37:51.827665+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:46:44.234857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.829784+0000","last_change":"2024-01-27T02:37:51.829687+0000","last_active":"2024-01-27T02:37:51.829784+0000","last_peered":"2024-01-27T02:37:51.829784+0000","last_clean":"2024-01-27T02:37:51.829784+0000","last_became_active":"2024-01-27T02:37:51.829229+0000","last_became_peered":"2024-01-27T02:37:51.829229+0000","last_unstale":"2024-01-27T02:37:51.829784+0000","last_undegraded":"2024-01-27T02:37:51.829784+0000","last_fullsized":"2024-01-27T02:37:51.829784+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:57.231078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.824072+0000","last_change":"2024-01-27T02:37:51.823916+0000","last_active":"2024-01-27T02:37:51.824072+0000","last_peered":"2024-01-27T02:37:51.824072+0000","last_clean":"2024-01-27T02:37:51.824072+0000","last_became_active":"2024-01-27T02:37:51.823677+0000","last_became_peered":"2024-01-27T02:37:51.823677+0000","last_unstale":"2024-01-27T02:37:51.824072+0000","last_undegraded":"2024-01-27T02:37:51.824072+0000","last_fullsized":"2024-01-27T02:37:51.824072+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:10:19.964021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.835156+0000","last_change":"2024-01-27T02:37:51.835076+0000","last_active":"2024-01-27T02:37:51.835156+0000","last_peered":"2024-01-27T02:37:51.835156+0000","last_clean":"2024-01-27T02:37:51.835156+0000","last_became_active":"2024-01-27T02:37:51.834270+0000","last_became_peered":"2024-01-27T02:37:51.834270+0000","last_unstale":"2024-01-27T02:37:51.835156+0000","last_undegraded":"2024-01-27T02:37:51.835156+0000","last_fullsized":"2024-01-27T02:37:51.835156+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:32:37.260822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.816665+0000","last_change":"2024-01-27T02:37:51.816582+0000","last_active":"2024-01-27T02:37:51.816665+0000","last_peered":"2024-01-27T02:37:51.816665+0000","last_clean":"2024-01-27T02:37:51.816665+0000","last_became_active":"2024-01-27T02:37:51.816341+0000","last_became_peered":"2024-01-27T02:37:51.816341+0000","last_unstale":"2024-01-27T02:37:51.816665+0000","last_undegraded":"2024-01-27T02:37:51.816665+0000","last_fullsized":"2024-01-27T02:37:51.816665+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:17:03.370160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.837046+0000","last_change":"2024-01-27T02:37:51.836999+0000","last_active":"2024-01-27T02:37:51.837046+0000","last_peered":"2024-01-27T02:37:51.837046+0000","last_clean":"2024-01-27T02:37:51.837046+0000","last_became_active":"2024-01-27T02:37:51.836568+0000","last_became_peered":"2024-01-27T02:37:51.836568+0000","last_unstale":"2024-01-27T02:37:51.837046+0000","last_undegraded":"2024-01-27T02:37:51.837046+0000","last_fullsized":"2024-01-27T02:37:51.837046+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:10:19.964021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.829603+0000","last_change":"2024-01-27T02:37:51.829508+0000","last_active":"2024-01-27T02:37:51.829603+0000","last_peered":"2024-01-27T02:37:51.829603+0000","last_clean":"2024-01-27T02:37:51.829603+0000","last_became_active":"2024-01-27T02:37:51.829047+0000","last_became_peered":"2024-01-27T02:37:51.829047+0000","last_unstale":"2024-01-27T02:37:51.829603+0000","last_undegraded":"2024-01-27T02:37:51.829603+0000","last_fullsized":"2024-01-27T02:37:51.829603+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:32:37.260822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.833263+0000","last_change":"2024-01-27T02:37:51.833172+0000","last_active":"2024-01-27T02:37:51.833263+0000","last_peered":"2024-01-27T02:37:51.833263+0000","last_clean":"2024-01-27T02:37:51.833263+0000","last_became_active":"2024-01-27T02:37:51.832923+0000","last_became_peered":"2024-01-27T02:37:51.832923+0000","last_unstale":"2024-01-27T02:37:51.833263+0000","last_undegraded":"2024-01-27T02:37:51.833263+0000","last_fullsized":"2024-01-27T02:37:51.833263+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:24:13.096584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.816999+0000","last_change":"2024-01-27T02:37:51.816884+0000","last_active":"2024-01-27T02:37:51.816999+0000","last_peered":"2024-01-27T02:37:51.816999+0000","last_clean":"2024-01-27T02:37:51.816999+0000","last_became_active":"2024-01-27T02:37:51.816521+0000","last_became_peered":"2024-01-27T02:37:51.816521+0000","last_unstale":"2024-01-27T02:37:51.816999+0000","last_undegraded":"2024-01-27T02:37:51.816999+0000","last_fullsized":"2024-01-27T02:37:51.816999+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:57.231078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.821816+0000","last_change":"2024-01-27T02:37:51.821616+0000","last_active":"2024-01-27T02:37:51.821816+0000","last_peered":"2024-01-27T02:37:51.821816+0000","last_clean":"2024-01-27T02:37:51.821816+0000","last_became_active":"2024-01-27T02:37:51.821168+0000","last_became_peered":"2024-01-27T02:37:51.821168+0000","last_unstale":"2024-01-27T02:37:51.821816+0000","last_undegraded":"2024-01-27T02:37:51.821816+0000","last_fullsized":"2024-01-27T02:37:51.821816+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:56:52.471976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.835379+0000","last_change":"2024-01-27T02:37:51.835113+0000","last_active":"2024-01-27T02:37:51.835379+0000","last_peered":"2024-01-27T02:37:51.835379+0000","last_clean":"2024-01-27T02:37:51.835379+0000","last_became_active":"2024-01-27T02:37:51.834542+0000","last_became_peered":"2024-01-27T02:37:51.834542+0000","last_unstale":"2024-01-27T02:37:51.835379+0000","last_undegraded":"2024-01-27T02:37:51.835379+0000","last_fullsized":"2024-01-27T02:37:51.835379+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:18:23.945261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.832883+0000","last_change":"2024-01-27T02:37:51.832781+0000","last_active":"2024-01-27T02:37:51.832883+0000","last_peered":"2024-01-27T02:37:51.832883+0000","last_clean":"2024-01-27T02:37:51.832883+0000","last_became_active":"2024-01-27T02:37:51.832535+0000","last_became_peered":"2024-01-27T02:37:51.832535+0000","last_unstale":"2024-01-27T02:37:51.832883+0000","last_undegraded":"2024-01-27T02:37:51.832883+0000","last_fullsized":"2024-01-27T02:37:51.832883+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:46:44.234857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.830493+0000","last_change":"2024-01-27T02:37:51.829796+0000","last_active":"2024-01-27T02:37:51.830493+0000","last_peered":"2024-01-27T02:37:51.830493+0000","last_clean":"2024-01-27T02:37:51.830493+0000","last_became_active":"2024-01-27T02:37:51.829526+0000","last_became_peered":"2024-01-27T02:37:51.829526+0000","last_unstale":"2024-01-27T02:37:51.830493+0000","last_undegraded":"2024-01-27T02:37:51.830493+0000","last_fullsized":"2024-01-27T02:37:51.830493+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:13:04.047913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.825862+0000","last_change":"2024-01-27T02:37:51.825613+0000","last_active":"2024-01-27T02:37:51.825862+0000","last_peered":"2024-01-27T02:37:51.825862+0000","last_clean":"2024-01-27T02:37:51.825862+0000","last_became_active":"2024-01-27T02:37:51.825184+0000","last_became_peered":"2024-01-27T02:37:51.825184+0000","last_unstale":"2024-01-27T02:37:51.825862+0000","last_undegraded":"2024-01-27T02:37:51.825862+0000","last_fullsized":"2024-01-27T02:37:51.825862+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:33:08.282783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.837170+0000","last_change":"2024-01-27T02:37:51.837018+0000","last_active":"2024-01-27T02:37:51.837170+0000","last_peered":"2024-01-27T02:37:51.837170+0000","last_clean":"2024-01-27T02:37:51.837170+0000","last_became_active":"2024-01-27T02:37:51.836744+0000","last_became_peered":"2024-01-27T02:37:51.836744+0000","last_unstale":"2024-01-27T02:37:51.837170+0000","last_undegraded":"2024-01-27T02:37:51.837170+0000","last_fullsized":"2024-01-27T02:37:51.837170+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:52:48.549784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.819849+0000","last_change":"2024-01-27T02:37:51.819774+0000","last_active":"2024-01-27T02:37:51.819849+0000","last_peered":"2024-01-27T02:37:51.819849+0000","last_clean":"2024-01-27T02:37:51.819849+0000","last_became_active":"2024-01-27T02:37:51.819537+0000","last_became_peered":"2024-01-27T02:37:51.819537+0000","last_unstale":"2024-01-27T02:37:51.819849+0000","last_undegraded":"2024-01-27T02:37:51.819849+0000","last_fullsized":"2024-01-27T02:37:51.819849+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:04:02.332379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.824588+0000","last_change":"2024-01-27T02:37:51.824517+0000","last_active":"2024-01-27T02:37:51.824588+0000","last_peered":"2024-01-27T02:37:51.824588+0000","last_clean":"2024-01-27T02:37:51.824588+0000","last_became_active":"2024-01-27T02:37:51.823909+0000","last_became_peered":"2024-01-27T02:37:51.823909+0000","last_unstale":"2024-01-27T02:37:51.824588+0000","last_undegraded":"2024-01-27T02:37:51.824588+0000","last_fullsized":"2024-01-27T02:37:51.824588+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:34:22.454613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.830328+0000","last_change":"2024-01-27T02:37:51.827287+0000","last_active":"2024-01-27T02:37:51.830328+0000","last_peered":"2024-01-27T02:37:51.830328+0000","last_clean":"2024-01-27T02:37:51.830328+0000","last_became_active":"2024-01-27T02:37:51.826700+0000","last_became_peered":"2024-01-27T02:37:51.826700+0000","last_unstale":"2024-01-27T02:37:51.830328+0000","last_undegraded":"2024-01-27T02:37:51.830328+0000","last_fullsized":"2024-01-27T02:37:51.830328+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:55:56.833988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.830309+0000","last_change":"2024-01-27T02:37:51.827254+0000","last_active":"2024-01-27T02:37:51.830309+0000","last_peered":"2024-01-27T02:37:51.830309+0000","last_clean":"2024-01-27T02:37:51.830309+0000","last_became_active":"2024-01-27T02:37:51.826830+0000","last_became_peered":"2024-01-27T02:37:51.826830+0000","last_unstale":"2024-01-27T02:37:51.830309+0000","last_undegraded":"2024-01-27T02:37:51.830309+0000","last_fullsized":"2024-01-27T02:37:51.830309+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:09:28.627404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.836533+0000","last_change":"2024-01-27T02:37:51.836459+0000","last_active":"2024-01-27T02:37:51.836533+0000","last_peered":"2024-01-27T02:37:51.836533+0000","last_clean":"2024-01-27T02:37:51.836533+0000","last_became_active":"2024-01-27T02:37:51.836270+0000","last_became_peered":"2024-01-27T02:37:51.836270+0000","last_unstale":"2024-01-27T02:37:51.836533+0000","last_undegraded":"2024-01-27T02:37:51.836533+0000","last_fullsized":"2024-01-27T02:37:51.836533+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:29:07.353494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.826720+0000","last_change":"2024-01-27T02:37:51.826523+0000","last_active":"2024-01-27T02:37:51.826720+0000","last_peered":"2024-01-27T02:37:51.826720+0000","last_clean":"2024-01-27T02:37:51.826720+0000","last_became_active":"2024-01-27T02:37:51.826229+0000","last_became_peered":"2024-01-27T02:37:51.826229+0000","last_unstale":"2024-01-27T02:37:51.826720+0000","last_undegraded":"2024-01-27T02:37:51.826720+0000","last_fullsized":"2024-01-27T02:37:51.826720+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:28:34.028232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.837037+0000","last_change":"2024-01-27T02:37:51.836964+0000","last_active":"2024-01-27T02:37:51.837037+0000","last_peered":"2024-01-27T02:37:51.837037+0000","last_clean":"2024-01-27T02:37:51.837037+0000","last_became_active":"2024-01-27T02:37:51.836726+0000","last_became_peered":"2024-01-27T02:37:51.836726+0000","last_unstale":"2024-01-27T02:37:51.837037+0000","last_undegraded":"2024-01-27T02:37:51.837037+0000","last_fullsized":"2024-01-27T02:37:51.837037+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:28:34.028232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.832864+0000","last_change":"2024-01-27T02:37:51.832531+0000","last_active":"2024-01-27T02:37:51.832864+0000","last_peered":"2024-01-27T02:37:51.832864+0000","last_clean":"2024-01-27T02:37:51.832864+0000","last_became_active":"2024-01-27T02:37:51.832307+0000","last_became_peered":"2024-01-27T02:37:51.832307+0000","last_unstale":"2024-01-27T02:37:51.832864+0000","last_undegraded":"2024-01-27T02:37:51.832864+0000","last_fullsized":"2024-01-27T02:37:51.832864+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:04:59.885669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.833411+0000","last_change":"2024-01-27T02:37:51.833286+0000","last_active":"2024-01-27T02:37:51.833411+0000","last_peered":"2024-01-27T02:37:51.833411+0000","last_clean":"2024-01-27T02:37:51.833411+0000","last_became_active":"2024-01-27T02:37:51.833006+0000","last_became_peered":"2024-01-27T02:37:51.833006+0000","last_unstale":"2024-01-27T02:37:51.833411+0000","last_undegraded":"2024-01-27T02:37:51.833411+0000","last_fullsized":"2024-01-27T02:37:51.833411+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:55:18.331130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.836543+0000","last_change":"2024-01-27T02:37:51.836473+0000","last_active":"2024-01-27T02:37:51.836543+0000","last_peered":"2024-01-27T02:37:51.836543+0000","last_clean":"2024-01-27T02:37:51.836543+0000","last_became_active":"2024-01-27T02:37:51.836257+0000","last_became_peered":"2024-01-27T02:37:51.836257+0000","last_unstale":"2024-01-27T02:37:51.836543+0000","last_undegraded":"2024-01-27T02:37:51.836543+0000","last_fullsized":"2024-01-27T02:37:51.836543+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:25:00.247752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.837819+0000","last_change":"2024-01-27T02:37:51.837717+0000","last_active":"2024-01-27T02:37:51.837819+0000","last_peered":"2024-01-27T02:37:51.837819+0000","last_clean":"2024-01-27T02:37:51.837819+0000","last_became_active":"2024-01-27T02:37:51.837280+0000","last_became_peered":"2024-01-27T02:37:51.837280+0000","last_unstale":"2024-01-27T02:37:51.837819+0000","last_undegraded":"2024-01-27T02:37:51.837819+0000","last_fullsized":"2024-01-27T02:37:51.837819+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:26:33.037819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.823183+0000","last_change":"2024-01-27T02:37:51.822736+0000","last_active":"2024-01-27T02:37:51.823183+0000","last_peered":"2024-01-27T02:37:51.823183+0000","last_clean":"2024-01-27T02:37:51.823183+0000","last_became_active":"2024-01-27T02:37:51.822445+0000","last_became_peered":"2024-01-27T02:37:51.822445+0000","last_unstale":"2024-01-27T02:37:51.823183+0000","last_undegraded":"2024-01-27T02:37:51.823183+0000","last_fullsized":"2024-01-27T02:37:51.823183+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:28:34.028232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.836314+0000","last_change":"2024-01-27T02:37:51.836032+0000","last_active":"2024-01-27T02:37:51.836314+0000","last_peered":"2024-01-27T02:37:51.836314+0000","last_clean":"2024-01-27T02:37:51.836314+0000","last_became_active":"2024-01-27T02:37:51.835774+0000","last_became_peered":"2024-01-27T02:37:51.835774+0000","last_unstale":"2024-01-27T02:37:51.836314+0000","last_undegraded":"2024-01-27T02:37:51.836314+0000","last_fullsized":"2024-01-27T02:37:51.836314+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:19:36.726696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.832209+0000","last_change":"2024-01-27T02:37:51.831833+0000","last_active":"2024-01-27T02:37:51.832209+0000","last_peered":"2024-01-27T02:37:51.832209+0000","last_clean":"2024-01-27T02:37:51.832209+0000","last_became_active":"2024-01-27T02:37:51.831415+0000","last_became_peered":"2024-01-27T02:37:51.831415+0000","last_unstale":"2024-01-27T02:37:51.832209+0000","last_undegraded":"2024-01-27T02:37:51.832209+0000","last_fullsized":"2024-01-27T02:37:51.832209+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:33:02.346431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"116'4","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.831677+0000","last_change":"2024-01-27T02:37:50.812131+0000","last_active":"2024-01-27T02:37:51.831677+0000","last_peered":"2024-01-27T02:37:51.831677+0000","last_clean":"2024-01-27T02:37:51.831677+0000","last_became_active":"2024-01-27T02:37:50.809514+0000","last_became_peered":"2024-01-27T02:37:50.809514+0000","last_unstale":"2024-01-27T02:37:51.831677+0000","last_undegraded":"2024-01-27T02:37:51.831677+0000","last_fullsized":"2024-01-27T02:37:51.831677+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:14:50.186256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.825845+0000","last_change":"2024-01-27T02:37:51.825465+0000","last_active":"2024-01-27T02:37:51.825845+0000","last_peered":"2024-01-27T02:37:51.825845+0000","last_clean":"2024-01-27T02:37:51.825845+0000","last_became_active":"2024-01-27T02:37:51.824864+0000","last_became_peered":"2024-01-27T02:37:51.824864+0000","last_unstale":"2024-01-27T02:37:51.825845+0000","last_undegraded":"2024-01-27T02:37:51.825845+0000","last_fullsized":"2024-01-27T02:37:51.825845+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:25:00.247752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.836024+0000","last_change":"2024-01-27T02:37:50.808649+0000","last_active":"2024-01-27T02:37:51.836024+0000","last_peered":"2024-01-27T02:37:51.836024+0000","last_clean":"2024-01-27T02:37:51.836024+0000","last_became_active":"2024-01-27T02:37:50.808412+0000","last_became_peered":"2024-01-27T02:37:50.808412+0000","last_unstale":"2024-01-27T02:37:51.836024+0000","last_undegraded":"2024-01-27T02:37:51.836024+0000","last_fullsized":"2024-01-27T02:37:51.836024+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:24:12.091569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.832717+0000","last_change":"2024-01-27T02:37:51.831656+0000","last_active":"2024-01-27T02:37:51.832717+0000","last_peered":"2024-01-27T02:37:51.832717+0000","last_clean":"2024-01-27T02:37:51.832717+0000","last_became_active":"2024-01-27T02:37:51.828372+0000","last_became_peered":"2024-01-27T02:37:51.828372+0000","last_unstale":"2024-01-27T02:37:51.832717+0000","last_undegraded":"2024-01-27T02:37:51.832717+0000","last_fullsized":"2024-01-27T02:37:51.832717+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:11:06.487989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.831171+0000","last_change":"2024-01-27T02:37:50.806041+0000","last_active":"2024-01-27T02:37:51.831171+0000","last_peered":"2024-01-27T02:37:51.831171+0000","last_clean":"2024-01-27T02:37:51.831171+0000","last_became_active":"2024-01-27T02:37:50.805760+0000","last_became_peered":"2024-01-27T02:37:50.805760+0000","last_unstale":"2024-01-27T02:37:51.831171+0000","last_undegraded":"2024-01-27T02:37:51.831171+0000","last_fullsized":"2024-01-27T02:37:51.831171+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:31:21.182217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.826351+0000","last_change":"2024-01-27T02:37:51.825416+0000","last_active":"2024-01-27T02:37:51.826351+0000","last_peered":"2024-01-27T02:37:51.826351+0000","last_clean":"2024-01-27T02:37:51.826351+0000","last_became_active":"2024-01-27T02:37:51.825167+0000","last_became_peered":"2024-01-27T02:37:51.825167+0000","last_unstale":"2024-01-27T02:37:51.826351+0000","last_undegraded":"2024-01-27T02:37:51.826351+0000","last_fullsized":"2024-01-27T02:37:51.826351+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:28.200720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.832255+0000","last_change":"2024-01-27T02:37:50.808048+0000","last_active":"2024-01-27T02:37:51.832255+0000","last_peered":"2024-01-27T02:37:51.832255+0000","last_clean":"2024-01-27T02:37:51.832255+0000","last_became_active":"2024-01-27T02:37:50.807771+0000","last_became_peered":"2024-01-27T02:37:50.807771+0000","last_unstale":"2024-01-27T02:37:51.832255+0000","last_undegraded":"2024-01-27T02:37:51.832255+0000","last_fullsized":"2024-01-27T02:37:51.832255+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:24.808914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.828125+0000","last_change":"2024-01-27T02:37:51.828060+0000","last_active":"2024-01-27T02:37:51.828125+0000","last_peered":"2024-01-27T02:37:51.828125+0000","last_clean":"2024-01-27T02:37:51.828125+0000","last_became_active":"2024-01-27T02:37:51.827829+0000","last_became_peered":"2024-01-27T02:37:51.827829+0000","last_unstale":"2024-01-27T02:37:51.828125+0000","last_undegraded":"2024-01-27T02:37:51.828125+0000","last_fullsized":"2024-01-27T02:37:51.828125+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:12:54.826584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.837161+0000","last_change":"2024-01-27T02:37:50.806436+0000","last_active":"2024-01-27T02:37:51.837161+0000","last_peered":"2024-01-27T02:37:51.837161+0000","last_clean":"2024-01-27T02:37:51.837161+0000","last_became_active":"2024-01-27T02:37:50.806167+0000","last_became_peered":"2024-01-27T02:37:50.806167+0000","last_unstale":"2024-01-27T02:37:51.837161+0000","last_undegraded":"2024-01-27T02:37:51.837161+0000","last_fullsized":"2024-01-27T02:37:51.837161+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:58:12.533491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.824051+0000","last_change":"2024-01-27T02:37:51.823428+0000","last_active":"2024-01-27T02:37:51.824051+0000","last_peered":"2024-01-27T02:37:51.824051+0000","last_clean":"2024-01-27T02:37:51.824051+0000","last_became_active":"2024-01-27T02:37:51.823162+0000","last_became_peered":"2024-01-27T02:37:51.823162+0000","last_unstale":"2024-01-27T02:37:51.824051+0000","last_undegraded":"2024-01-27T02:37:51.824051+0000","last_fullsized":"2024-01-27T02:37:51.824051+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:33:08.282783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.838812+0000","last_change":"2024-01-27T02:37:50.812296+0000","last_active":"2024-01-27T02:37:51.838812+0000","last_peered":"2024-01-27T02:37:51.838812+0000","last_clean":"2024-01-27T02:37:51.838812+0000","last_became_active":"2024-01-27T02:37:50.809825+0000","last_became_peered":"2024-01-27T02:37:50.809825+0000","last_unstale":"2024-01-27T02:37:51.838812+0000","last_undegraded":"2024-01-27T02:37:51.838812+0000","last_fullsized":"2024-01-27T02:37:51.838812+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:12:54.338500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.835400+0000","last_change":"2024-01-27T02:37:51.835270+0000","last_active":"2024-01-27T02:37:51.835400+0000","last_peered":"2024-01-27T02:37:51.835400+0000","last_clean":"2024-01-27T02:37:51.835400+0000","last_became_active":"2024-01-27T02:37:51.834877+0000","last_became_peered":"2024-01-27T02:37:51.834877+0000","last_unstale":"2024-01-27T02:37:51.835400+0000","last_undegraded":"2024-01-27T02:37:51.835400+0000","last_fullsized":"2024-01-27T02:37:51.835400+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:46:44.234857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.818753+0000","last_change":"2024-01-27T02:37:50.804462+0000","last_active":"2024-01-27T02:37:51.818753+0000","last_peered":"2024-01-27T02:37:51.818753+0000","last_clean":"2024-01-27T02:37:51.818753+0000","last_became_active":"2024-01-27T02:37:50.804211+0000","last_became_peered":"2024-01-27T02:37:50.804211+0000","last_unstale":"2024-01-27T02:37:51.818753+0000","last_undegraded":"2024-01-27T02:37:51.818753+0000","last_fullsized":"2024-01-27T02:37:51.818753+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:16:34.794337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.827327+0000","last_change":"2024-01-27T02:37:51.827257+0000","last_active":"2024-01-27T02:37:51.827327+0000","last_peered":"2024-01-27T02:37:51.827327+0000","last_clean":"2024-01-27T02:37:51.827327+0000","last_became_active":"2024-01-27T02:37:51.827022+0000","last_became_peered":"2024-01-27T02:37:51.827022+0000","last_unstale":"2024-01-27T02:37:51.827327+0000","last_undegraded":"2024-01-27T02:37:51.827327+0000","last_fullsized":"2024-01-27T02:37:51.827327+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:33:14.691144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.833948+0000","last_change":"2024-01-27T02:37:50.804294+0000","last_active":"2024-01-27T02:37:51.833948+0000","last_peered":"2024-01-27T02:37:51.833948+0000","last_clean":"2024-01-27T02:37:51.833948+0000","last_became_active":"2024-01-27T02:37:50.803789+0000","last_became_peered":"2024-01-27T02:37:50.803789+0000","last_unstale":"2024-01-27T02:37:51.833948+0000","last_undegraded":"2024-01-27T02:37:51.833948+0000","last_fullsized":"2024-01-27T02:37:51.833948+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:37:47.399101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.830461+0000","last_change":"2024-01-27T02:37:51.829268+0000","last_active":"2024-01-27T02:37:51.830461+0000","last_peered":"2024-01-27T02:37:51.830461+0000","last_clean":"2024-01-27T02:37:51.830461+0000","last_became_active":"2024-01-27T02:37:51.829000+0000","last_became_peered":"2024-01-27T02:37:51.829000+0000","last_unstale":"2024-01-27T02:37:51.830461+0000","last_undegraded":"2024-01-27T02:37:51.830461+0000","last_fullsized":"2024-01-27T02:37:51.830461+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:52:20.588218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.828195+0000","last_change":"2024-01-27T02:37:50.807055+0000","last_active":"2024-01-27T02:37:51.828195+0000","last_peered":"2024-01-27T02:37:51.828195+0000","last_clean":"2024-01-27T02:37:51.828195+0000","last_became_active":"2024-01-27T02:37:50.806246+0000","last_became_peered":"2024-01-27T02:37:50.806246+0000","last_unstale":"2024-01-27T02:37:51.828195+0000","last_undegraded":"2024-01-27T02:37:51.828195+0000","last_fullsized":"2024-01-27T02:37:51.828195+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:23.160028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.826753+0000","last_change":"2024-01-27T02:37:51.826505+0000","last_active":"2024-01-27T02:37:51.826753+0000","last_peered":"2024-01-27T02:37:51.826753+0000","last_clean":"2024-01-27T02:37:51.826753+0000","last_became_active":"2024-01-27T02:37:51.825678+0000","last_became_peered":"2024-01-27T02:37:51.825678+0000","last_unstale":"2024-01-27T02:37:51.826753+0000","last_undegraded":"2024-01-27T02:37:51.826753+0000","last_fullsized":"2024-01-27T02:37:51.826753+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:25.813930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.813125+0000","last_change":"2024-01-27T02:37:50.805856+0000","last_active":"2024-01-27T02:37:51.813125+0000","last_peered":"2024-01-27T02:37:51.813125+0000","last_clean":"2024-01-27T02:37:51.813125+0000","last_became_active":"2024-01-27T02:37:50.805568+0000","last_became_peered":"2024-01-27T02:37:50.805568+0000","last_unstale":"2024-01-27T02:37:51.813125+0000","last_undegraded":"2024-01-27T02:37:51.813125+0000","last_fullsized":"2024-01-27T02:37:51.813125+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:11:46.349861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.827322+0000","last_change":"2024-01-27T02:37:51.826716+0000","last_active":"2024-01-27T02:37:51.827322+0000","last_peered":"2024-01-27T02:37:51.827322+0000","last_clean":"2024-01-27T02:37:51.827322+0000","last_became_active":"2024-01-27T02:37:51.826267+0000","last_became_peered":"2024-01-27T02:37:51.826267+0000","last_unstale":"2024-01-27T02:37:51.827322+0000","last_undegraded":"2024-01-27T02:37:51.827322+0000","last_fullsized":"2024-01-27T02:37:51.827322+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:18:23.945261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"116'3","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.832239+0000","last_change":"2024-01-27T02:37:50.809623+0000","last_active":"2024-01-27T02:37:51.832239+0000","last_peered":"2024-01-27T02:37:51.832239+0000","last_clean":"2024-01-27T02:37:51.832239+0000","last_became_active":"2024-01-27T02:37:50.809447+0000","last_became_peered":"2024-01-27T02:37:50.809447+0000","last_unstale":"2024-01-27T02:37:51.832239+0000","last_undegraded":"2024-01-27T02:37:51.832239+0000","last_fullsized":"2024-01-27T02:37:51.832239+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:50.938489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.832898+0000","last_change":"2024-01-27T02:37:51.832799+0000","last_active":"2024-01-27T02:37:51.832898+0000","last_peered":"2024-01-27T02:37:51.832898+0000","last_clean":"2024-01-27T02:37:51.832898+0000","last_became_active":"2024-01-27T02:37:51.832558+0000","last_became_peered":"2024-01-27T02:37:51.832558+0000","last_unstale":"2024-01-27T02:37:51.832898+0000","last_undegraded":"2024-01-27T02:37:51.832898+0000","last_fullsized":"2024-01-27T02:37:51.832898+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:18:23.945261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"116'6","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.840033+0000","last_change":"2024-01-27T02:37:50.814137+0000","last_active":"2024-01-27T02:37:51.840033+0000","last_peered":"2024-01-27T02:37:51.840033+0000","last_clean":"2024-01-27T02:37:51.840033+0000","last_became_active":"2024-01-27T02:37:50.813898+0000","last_became_peered":"2024-01-27T02:37:50.813898+0000","last_unstale":"2024-01-27T02:37:51.840033+0000","last_undegraded":"2024-01-27T02:37:51.840033+0000","last_fullsized":"2024-01-27T02:37:51.840033+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:56:37.175345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.830778+0000","last_change":"2024-01-27T02:37:51.830650+0000","last_active":"2024-01-27T02:37:51.830778+0000","last_peered":"2024-01-27T02:37:51.830778+0000","last_clean":"2024-01-27T02:37:51.830778+0000","last_became_active":"2024-01-27T02:37:51.830400+0000","last_became_peered":"2024-01-27T02:37:51.830400+0000","last_unstale":"2024-01-27T02:37:51.830778+0000","last_undegraded":"2024-01-27T02:37:51.830778+0000","last_fullsized":"2024-01-27T02:37:51.830778+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:10:19.964021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.830254+0000","last_change":"2024-01-27T02:37:50.813470+0000","last_active":"2024-01-27T02:37:51.830254+0000","last_peered":"2024-01-27T02:37:51.830254+0000","last_clean":"2024-01-27T02:37:51.830254+0000","last_became_active":"2024-01-27T02:37:50.812014+0000","last_became_peered":"2024-01-27T02:37:50.812014+0000","last_unstale":"2024-01-27T02:37:51.830254+0000","last_undegraded":"2024-01-27T02:37:51.830254+0000","last_fullsized":"2024-01-27T02:37:51.830254+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:56:29.266154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.835233+0000","last_change":"2024-01-27T02:37:51.835063+0000","last_active":"2024-01-27T02:37:51.835233+0000","last_peered":"2024-01-27T02:37:51.835233+0000","last_clean":"2024-01-27T02:37:51.835233+0000","last_became_active":"2024-01-27T02:37:51.834540+0000","last_became_peered":"2024-01-27T02:37:51.834540+0000","last_unstale":"2024-01-27T02:37:51.835233+0000","last_undegraded":"2024-01-27T02:37:51.835233+0000","last_fullsized":"2024-01-27T02:37:51.835233+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:59:34.778013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.818741+0000","last_change":"2024-01-27T02:37:50.807923+0000","last_active":"2024-01-27T02:37:51.818741+0000","last_peered":"2024-01-27T02:37:51.818741+0000","last_clean":"2024-01-27T02:37:51.818741+0000","last_became_active":"2024-01-27T02:37:50.807417+0000","last_became_peered":"2024-01-27T02:37:50.807417+0000","last_unstale":"2024-01-27T02:37:51.818741+0000","last_undegraded":"2024-01-27T02:37:51.818741+0000","last_fullsized":"2024-01-27T02:37:51.818741+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:26:35.727576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.826428+0000","last_change":"2024-01-27T02:37:51.825934+0000","last_active":"2024-01-27T02:37:51.826428+0000","last_peered":"2024-01-27T02:37:51.826428+0000","last_clean":"2024-01-27T02:37:51.826428+0000","last_became_active":"2024-01-27T02:37:51.825688+0000","last_became_peered":"2024-01-27T02:37:51.825688+0000","last_unstale":"2024-01-27T02:37:51.826428+0000","last_undegraded":"2024-01-27T02:37:51.826428+0000","last_fullsized":"2024-01-27T02:37:51.826428+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:51.943504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.814416+0000","last_change":"2024-01-27T02:37:50.809421+0000","last_active":"2024-01-27T02:37:51.814416+0000","last_peered":"2024-01-27T02:37:51.814416+0000","last_clean":"2024-01-27T02:37:51.814416+0000","last_became_active":"2024-01-27T02:37:50.809169+0000","last_became_peered":"2024-01-27T02:37:50.809169+0000","last_unstale":"2024-01-27T02:37:51.814416+0000","last_undegraded":"2024-01-27T02:37:51.814416+0000","last_fullsized":"2024-01-27T02:37:51.814416+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:31:21.182217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.820200+0000","last_change":"2024-01-27T02:37:51.820121+0000","last_active":"2024-01-27T02:37:51.820200+0000","last_peered":"2024-01-27T02:37:51.820200+0000","last_clean":"2024-01-27T02:37:51.820200+0000","last_became_active":"2024-01-27T02:37:51.819763+0000","last_became_peered":"2024-01-27T02:37:51.819763+0000","last_unstale":"2024-01-27T02:37:51.820200+0000","last_undegraded":"2024-01-27T02:37:51.820200+0000","last_fullsized":"2024-01-27T02:37:51.820200+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:34:22.466946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"116'8","reported_seq":18,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.839283+0000","last_change":"2024-01-27T02:37:50.810368+0000","last_active":"2024-01-27T02:37:51.839283+0000","last_peered":"2024-01-27T02:37:51.839283+0000","last_clean":"2024-01-27T02:37:51.839283+0000","last_became_active":"2024-01-27T02:37:50.810089+0000","last_became_peered":"2024-01-27T02:37:50.810089+0000","last_unstale":"2024-01-27T02:37:51.839283+0000","last_undegraded":"2024-01-27T02:37:51.839283+0000","last_fullsized":"2024-01-27T02:37:51.839283+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:50:15.527559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.820861+0000","last_change":"2024-01-27T02:37:51.820762+0000","last_active":"2024-01-27T02:37:51.820861+0000","last_peered":"2024-01-27T02:37:51.820861+0000","last_clean":"2024-01-27T02:37:51.820861+0000","last_became_active":"2024-01-27T02:37:51.820515+0000","last_became_peered":"2024-01-27T02:37:51.820515+0000","last_unstale":"2024-01-27T02:37:51.820861+0000","last_undegraded":"2024-01-27T02:37:51.820861+0000","last_fullsized":"2024-01-27T02:37:51.820861+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:25:00.247752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.830967+0000","last_change":"2024-01-27T02:37:50.803536+0000","last_active":"2024-01-27T02:37:51.830967+0000","last_peered":"2024-01-27T02:37:51.830967+0000","last_clean":"2024-01-27T02:37:51.830967+0000","last_became_active":"2024-01-27T02:37:50.803332+0000","last_became_peered":"2024-01-27T02:37:50.803332+0000","last_unstale":"2024-01-27T02:37:51.830967+0000","last_undegraded":"2024-01-27T02:37:51.830967+0000","last_fullsized":"2024-01-27T02:37:51.830967+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:05:27.161024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.825834+0000","last_change":"2024-01-27T02:37:51.825726+0000","last_active":"2024-01-27T02:37:51.825834+0000","last_peered":"2024-01-27T02:37:51.825834+0000","last_clean":"2024-01-27T02:37:51.825834+0000","last_became_active":"2024-01-27T02:37:51.825411+0000","last_became_peered":"2024-01-27T02:37:51.825411+0000","last_unstale":"2024-01-27T02:37:51.825834+0000","last_undegraded":"2024-01-27T02:37:51.825834+0000","last_fullsized":"2024-01-27T02:37:51.825834+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:16:35.799353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.821259+0000","last_change":"2024-01-27T02:37:50.802000+0000","last_active":"2024-01-27T02:37:51.821259+0000","last_peered":"2024-01-27T02:37:51.821259+0000","last_clean":"2024-01-27T02:37:51.821259+0000","last_became_active":"2024-01-27T02:37:50.801755+0000","last_became_peered":"2024-01-27T02:37:50.801755+0000","last_unstale":"2024-01-27T02:37:51.821259+0000","last_undegraded":"2024-01-27T02:37:51.821259+0000","last_fullsized":"2024-01-27T02:37:51.821259+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:54:56.999817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.831050+0000","last_change":"2024-01-27T02:37:51.830287+0000","last_active":"2024-01-27T02:37:51.831050+0000","last_peered":"2024-01-27T02:37:51.831050+0000","last_clean":"2024-01-27T02:37:51.831050+0000","last_became_active":"2024-01-27T02:37:51.829998+0000","last_became_peered":"2024-01-27T02:37:51.829998+0000","last_unstale":"2024-01-27T02:37:51.831050+0000","last_undegraded":"2024-01-27T02:37:51.831050+0000","last_fullsized":"2024-01-27T02:37:51.831050+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:09:28.627404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.825846+0000","last_change":"2024-01-27T02:37:50.812532+0000","last_active":"2024-01-27T02:37:51.825846+0000","last_peered":"2024-01-27T02:37:51.825846+0000","last_clean":"2024-01-27T02:37:51.825846+0000","last_became_active":"2024-01-27T02:37:50.812209+0000","last_became_peered":"2024-01-27T02:37:50.812209+0000","last_unstale":"2024-01-27T02:37:51.825846+0000","last_undegraded":"2024-01-27T02:37:51.825846+0000","last_fullsized":"2024-01-27T02:37:51.825846+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:11:46.349861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.832190+0000","last_change":"2024-01-27T02:37:51.831671+0000","last_active":"2024-01-27T02:37:51.832190+0000","last_peered":"2024-01-27T02:37:51.832190+0000","last_clean":"2024-01-27T02:37:51.832190+0000","last_became_active":"2024-01-27T02:37:51.830906+0000","last_became_peered":"2024-01-27T02:37:51.830906+0000","last_unstale":"2024-01-27T02:37:51.832190+0000","last_undegraded":"2024-01-27T02:37:51.832190+0000","last_fullsized":"2024-01-27T02:37:51.832190+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:22:19.315343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.834659+0000","last_change":"2024-01-27T02:37:50.813813+0000","last_active":"2024-01-27T02:37:51.834659+0000","last_peered":"2024-01-27T02:37:51.834659+0000","last_clean":"2024-01-27T02:37:51.834659+0000","last_became_active":"2024-01-27T02:37:50.813235+0000","last_became_peered":"2024-01-27T02:37:50.813235+0000","last_unstale":"2024-01-27T02:37:51.834659+0000","last_undegraded":"2024-01-27T02:37:51.834659+0000","last_fullsized":"2024-01-27T02:37:51.834659+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:33:13.686128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.828983+0000","last_change":"2024-01-27T02:37:51.828132+0000","last_active":"2024-01-27T02:37:51.828983+0000","last_peered":"2024-01-27T02:37:51.828983+0000","last_clean":"2024-01-27T02:37:51.828983+0000","last_became_active":"2024-01-27T02:37:51.827879+0000","last_became_peered":"2024-01-27T02:37:51.827879+0000","last_unstale":"2024-01-27T02:37:51.828983+0000","last_undegraded":"2024-01-27T02:37:51.828983+0000","last_fullsized":"2024-01-27T02:37:51.828983+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:58:13.538507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.822138+0000","last_change":"2024-01-27T02:37:50.804058+0000","last_active":"2024-01-27T02:37:51.822138+0000","last_peered":"2024-01-27T02:37:51.822138+0000","last_clean":"2024-01-27T02:37:51.822138+0000","last_became_active":"2024-01-27T02:37:50.803802+0000","last_became_peered":"2024-01-27T02:37:50.803802+0000","last_unstale":"2024-01-27T02:37:51.822138+0000","last_undegraded":"2024-01-27T02:37:51.822138+0000","last_fullsized":"2024-01-27T02:37:51.822138+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:01:24.386631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.828920+0000","last_change":"2024-01-27T02:37:51.824693+0000","last_active":"2024-01-27T02:37:51.828920+0000","last_peered":"2024-01-27T02:37:51.828920+0000","last_clean":"2024-01-27T02:37:51.828920+0000","last_became_active":"2024-01-27T02:37:51.824263+0000","last_became_peered":"2024-01-27T02:37:51.824263+0000","last_unstale":"2024-01-27T02:37:51.828920+0000","last_undegraded":"2024-01-27T02:37:51.828920+0000","last_fullsized":"2024-01-27T02:37:51.828920+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:50:08.414227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.822884+0000","last_change":"2024-01-27T02:37:50.806415+0000","last_active":"2024-01-27T02:37:51.822884+0000","last_peered":"2024-01-27T02:37:51.822884+0000","last_clean":"2024-01-27T02:37:51.822884+0000","last_became_active":"2024-01-27T02:37:50.806149+0000","last_became_peered":"2024-01-27T02:37:50.806149+0000","last_unstale":"2024-01-27T02:37:51.822884+0000","last_undegraded":"2024-01-27T02:37:51.822884+0000","last_fullsized":"2024-01-27T02:37:51.822884+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:04:39.266345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.836957+0000","last_change":"2024-01-27T02:37:51.836898+0000","last_active":"2024-01-27T02:37:51.836957+0000","last_peered":"2024-01-27T02:37:51.836957+0000","last_clean":"2024-01-27T02:37:51.836957+0000","last_became_active":"2024-01-27T02:37:51.836232+0000","last_became_peered":"2024-01-27T02:37:51.836232+0000","last_unstale":"2024-01-27T02:37:51.836957+0000","last_undegraded":"2024-01-27T02:37:51.836957+0000","last_fullsized":"2024-01-27T02:37:51.836957+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:33:08.282783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.822894+0000","last_change":"2024-01-27T02:37:50.809566+0000","last_active":"2024-01-27T02:37:51.822894+0000","last_peered":"2024-01-27T02:37:51.822894+0000","last_clean":"2024-01-27T02:37:51.822894+0000","last_became_active":"2024-01-27T02:37:50.809276+0000","last_became_peered":"2024-01-27T02:37:50.809276+0000","last_unstale":"2024-01-27T02:37:51.822894+0000","last_undegraded":"2024-01-27T02:37:51.822894+0000","last_fullsized":"2024-01-27T02:37:51.822894+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:24:12.091569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.830289+0000","last_change":"2024-01-27T02:37:51.827097+0000","last_active":"2024-01-27T02:37:51.830289+0000","last_peered":"2024-01-27T02:37:51.830289+0000","last_clean":"2024-01-27T02:37:51.830289+0000","last_became_active":"2024-01-27T02:37:51.826483+0000","last_became_peered":"2024-01-27T02:37:51.826483+0000","last_unstale":"2024-01-27T02:37:51.830289+0000","last_undegraded":"2024-01-27T02:37:51.830289+0000","last_fullsized":"2024-01-27T02:37:51.830289+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:29:07.353494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.828305+0000","last_change":"2024-01-27T02:37:50.805793+0000","last_active":"2024-01-27T02:37:51.828305+0000","last_peered":"2024-01-27T02:37:51.828305+0000","last_clean":"2024-01-27T02:37:51.828305+0000","last_became_active":"2024-01-27T02:37:50.805524+0000","last_became_peered":"2024-01-27T02:37:50.805524+0000","last_unstale":"2024-01-27T02:37:51.828305+0000","last_undegraded":"2024-01-27T02:37:51.828305+0000","last_fullsized":"2024-01-27T02:37:51.828305+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:57:33.113268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.832749+0000","last_change":"2024-01-27T02:37:51.832632+0000","last_active":"2024-01-27T02:37:51.832749+0000","last_peered":"2024-01-27T02:37:51.832749+0000","last_clean":"2024-01-27T02:37:51.832749+0000","last_became_active":"2024-01-27T02:37:51.832311+0000","last_became_peered":"2024-01-27T02:37:51.832311+0000","last_unstale":"2024-01-27T02:37:51.832749+0000","last_undegraded":"2024-01-27T02:37:51.832749+0000","last_fullsized":"2024-01-27T02:37:51.832749+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:35:24.076704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.826105+0000","last_change":"2024-01-27T02:37:50.814412+0000","last_active":"2024-01-27T02:37:51.826105+0000","last_peered":"2024-01-27T02:37:51.826105+0000","last_clean":"2024-01-27T02:37:51.826105+0000","last_became_active":"2024-01-27T02:37:50.813560+0000","last_became_peered":"2024-01-27T02:37:50.813560+0000","last_unstale":"2024-01-27T02:37:51.826105+0000","last_undegraded":"2024-01-27T02:37:51.826105+0000","last_fullsized":"2024-01-27T02:37:51.826105+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:27.195705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.835195+0000","last_change":"2024-01-27T02:37:51.834798+0000","last_active":"2024-01-27T02:37:51.835195+0000","last_peered":"2024-01-27T02:37:51.835195+0000","last_clean":"2024-01-27T02:37:51.835195+0000","last_became_active":"2024-01-27T02:37:51.834048+0000","last_became_peered":"2024-01-27T02:37:51.834048+0000","last_unstale":"2024-01-27T02:37:51.835195+0000","last_undegraded":"2024-01-27T02:37:51.835195+0000","last_fullsized":"2024-01-27T02:37:51.835195+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:56:38.180360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.824580+0000","last_change":"2024-01-27T02:37:50.808731+0000","last_active":"2024-01-27T02:37:51.824580+0000","last_peered":"2024-01-27T02:37:51.824580+0000","last_clean":"2024-01-27T02:37:51.824580+0000","last_became_active":"2024-01-27T02:37:50.808482+0000","last_became_peered":"2024-01-27T02:37:50.808482+0000","last_unstale":"2024-01-27T02:37:51.824580+0000","last_undegraded":"2024-01-27T02:37:51.824580+0000","last_fullsized":"2024-01-27T02:37:51.824580+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:50.938489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.827341+0000","last_change":"2024-01-27T02:37:51.827238+0000","last_active":"2024-01-27T02:37:51.827341+0000","last_peered":"2024-01-27T02:37:51.827341+0000","last_clean":"2024-01-27T02:37:51.827341+0000","last_became_active":"2024-01-27T02:37:51.826966+0000","last_became_peered":"2024-01-27T02:37:51.826966+0000","last_unstale":"2024-01-27T02:37:51.827341+0000","last_undegraded":"2024-01-27T02:37:51.827341+0000","last_fullsized":"2024-01-27T02:37:51.827341+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:04:02.332379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.818574+0000","last_change":"2024-01-27T02:37:50.808670+0000","last_active":"2024-01-27T02:37:51.818574+0000","last_peered":"2024-01-27T02:37:51.818574+0000","last_clean":"2024-01-27T02:37:51.818574+0000","last_became_active":"2024-01-27T02:37:50.808428+0000","last_became_peered":"2024-01-27T02:37:50.808428+0000","last_unstale":"2024-01-27T02:37:51.818574+0000","last_undegraded":"2024-01-27T02:37:51.818574+0000","last_fullsized":"2024-01-27T02:37:51.818574+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:57:56.062671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.820070+0000","last_change":"2024-01-27T02:37:51.819988+0000","last_active":"2024-01-27T02:37:51.820070+0000","last_peered":"2024-01-27T02:37:51.820070+0000","last_clean":"2024-01-27T02:37:51.820070+0000","last_became_active":"2024-01-27T02:37:51.819476+0000","last_became_peered":"2024-01-27T02:37:51.819476+0000","last_unstale":"2024-01-27T02:37:51.820070+0000","last_undegraded":"2024-01-27T02:37:51.820070+0000","last_fullsized":"2024-01-27T02:37:51.820070+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:28:08.524359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.822992+0000","last_change":"2024-01-27T02:37:50.813250+0000","last_active":"2024-01-27T02:37:51.822992+0000","last_peered":"2024-01-27T02:37:51.822992+0000","last_clean":"2024-01-27T02:37:51.822992+0000","last_became_active":"2024-01-27T02:37:50.813038+0000","last_became_peered":"2024-01-27T02:37:50.813038+0000","last_unstale":"2024-01-27T02:37:51.822992+0000","last_undegraded":"2024-01-27T02:37:51.822992+0000","last_fullsized":"2024-01-27T02:37:51.822992+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:25.524253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.819116+0000","last_change":"2024-01-27T02:37:51.819051+0000","last_active":"2024-01-27T02:37:51.819116+0000","last_peered":"2024-01-27T02:37:51.819116+0000","last_clean":"2024-01-27T02:37:51.819116+0000","last_became_active":"2024-01-27T02:37:51.818770+0000","last_became_peered":"2024-01-27T02:37:51.818770+0000","last_unstale":"2024-01-27T02:37:51.819116+0000","last_undegraded":"2024-01-27T02:37:51.819116+0000","last_fullsized":"2024-01-27T02:37:51.819116+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:14:51.191272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.817859+0000","last_change":"2024-01-27T02:37:50.803902+0000","last_active":"2024-01-27T02:37:51.817859+0000","last_peered":"2024-01-27T02:37:51.817859+0000","last_clean":"2024-01-27T02:37:51.817859+0000","last_became_active":"2024-01-27T02:37:50.803626+0000","last_became_peered":"2024-01-27T02:37:50.803626+0000","last_unstale":"2024-01-27T02:37:51.817859+0000","last_undegraded":"2024-01-27T02:37:51.817859+0000","last_fullsized":"2024-01-27T02:37:51.817859+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:25.524253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.821790+0000","last_change":"2024-01-27T02:37:51.821433+0000","last_active":"2024-01-27T02:37:51.821790+0000","last_peered":"2024-01-27T02:37:51.821790+0000","last_clean":"2024-01-27T02:37:51.821790+0000","last_became_active":"2024-01-27T02:37:51.820831+0000","last_became_peered":"2024-01-27T02:37:51.820831+0000","last_unstale":"2024-01-27T02:37:51.821790+0000","last_undegraded":"2024-01-27T02:37:51.821790+0000","last_fullsized":"2024-01-27T02:37:51.821790+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:24.165044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.814170+0000","last_change":"2024-01-27T02:37:50.804116+0000","last_active":"2024-01-27T02:37:51.814170+0000","last_peered":"2024-01-27T02:37:51.814170+0000","last_clean":"2024-01-27T02:37:51.814170+0000","last_became_active":"2024-01-27T02:37:50.803880+0000","last_became_peered":"2024-01-27T02:37:50.803880+0000","last_unstale":"2024-01-27T02:37:51.814170+0000","last_undegraded":"2024-01-27T02:37:51.814170+0000","last_fullsized":"2024-01-27T02:37:51.814170+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:50.938489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.828702+0000","last_change":"2024-01-27T02:37:51.828506+0000","last_active":"2024-01-27T02:37:51.828702+0000","last_peered":"2024-01-27T02:37:51.828702+0000","last_clean":"2024-01-27T02:37:51.828702+0000","last_became_active":"2024-01-27T02:37:51.828120+0000","last_became_peered":"2024-01-27T02:37:51.828120+0000","last_unstale":"2024-01-27T02:37:51.828702+0000","last_undegraded":"2024-01-27T02:37:51.828702+0000","last_fullsized":"2024-01-27T02:37:51.828702+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:17:03.370160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.818190+0000","last_change":"2024-01-27T02:37:50.807085+0000","last_active":"2024-01-27T02:37:51.818190+0000","last_peered":"2024-01-27T02:37:51.818190+0000","last_clean":"2024-01-27T02:37:51.818190+0000","last_became_active":"2024-01-27T02:37:50.804683+0000","last_became_peered":"2024-01-27T02:37:50.804683+0000","last_unstale":"2024-01-27T02:37:51.818190+0000","last_undegraded":"2024-01-27T02:37:51.818190+0000","last_fullsized":"2024-01-27T02:37:51.818190+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:49:46.671831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.829974+0000","last_change":"2024-01-27T02:37:51.829674+0000","last_active":"2024-01-27T02:37:51.829974+0000","last_peered":"2024-01-27T02:37:51.829974+0000","last_clean":"2024-01-27T02:37:51.829974+0000","last_became_active":"2024-01-27T02:37:51.828989+0000","last_became_peered":"2024-01-27T02:37:51.828989+0000","last_unstale":"2024-01-27T02:37:51.829974+0000","last_undegraded":"2024-01-27T02:37:51.829974+0000","last_fullsized":"2024-01-27T02:37:51.829974+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:50.790887+0000","last_clean_scrub_stamp":"2024-01-27T02:37:50.790887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:37:48.404117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.828244+0000","last_change":"2024-01-27T02:37:50.809913+0000","last_active":"2024-01-27T02:37:51.828244+0000","last_peered":"2024-01-27T02:37:51.828244+0000","last_clean":"2024-01-27T02:37:51.828244+0000","last_became_active":"2024-01-27T02:37:50.809679+0000","last_became_peered":"2024-01-27T02:37:50.809679+0000","last_unstale":"2024-01-27T02:37:51.828244+0000","last_undegraded":"2024-01-27T02:37:51.828244+0000","last_fullsized":"2024-01-27T02:37:51.828244+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:37:49.785871+0000","last_clean_scrub_stamp":"2024-01-27T02:37:49.785871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:31:27.195705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":247,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-01-27T02:37:51.825230+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:37:51.825230+0000","last_peered":"2024-01-27T02:37:51.825230+0000","last_clean":"2024-01-27T02:37:51.825230+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:37:51.825230+0000","last_undegraded":"2024-01-27T02:37:51.825230+0000","last_fullsized":"2024-01-27T02:37:51.825230+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1199,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194113,"statfs":{"total":200000000,"available":198772024,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71299999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.59099999999999997,"15min":0.58999999999999997},"min":{"1min":0.49099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77400000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55400000000000005}]},{"osd":1,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59699999999999998,"15min":0.59799999999999998},"min":{"1min":0.46100000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.44400000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.753,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.51100000000000001}]},{"osd":2,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58299999999999996},"min":{"1min":0.432,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77700000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.56499999999999995}]},{"osd":3,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60499999999999998,"15min":0.59299999999999997},"min":{"1min":0.495,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.44},{"interface":"front","average":{"1min":0.628,"5min":0.60999999999999999,"15min":0.59499999999999997},"min":{"1min":0.5,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85099999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.48199999999999998}]},{"osd":4,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.47499999999999998,"15min":0.48699999999999999},"min":{"1min":0.32400000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63300000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.495,"15min":0.51300000000000001},"min":{"1min":0.35999999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68899999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.46600000000000003}]},{"osd":5,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51800000000000002,"15min":0.53600000000000003},"min":{"1min":0.39000000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.56399999999999995},"min":{"1min":0.40100000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.58699999999999997}]},{"osd":6,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.59499999999999997},"min":{"1min":0.40999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.751,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.57699999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.81599999999999995,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.53000000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574922,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":623,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194689,"statfs":{"total":200000000,"available":199361880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58699999999999997,"15min":0.52100000000000002},"min":{"1min":0.45000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57799999999999996,"15min":0.52700000000000002},"min":{"1min":0.434,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71899999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.622}]},{"osd":1,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.51800000000000002},"min":{"1min":0.41999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79700000000000004,"5min":0.874,"15min":0.874},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.60599999999999998,"15min":0.55500000000000005},"min":{"1min":0.379,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.72699999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.41199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.625,"15min":0.58299999999999996},"min":{"1min":0.45100000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79700000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.68000000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58199999999999996,"15min":0.55100000000000005},"min":{"1min":0.47599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.76800000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61499999999999999,"15min":0.57299999999999995},"min":{"1min":0.48499999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.78100000000000003,"5min":1.262,"15min":1.262},"last":0.54000000000000004}]},{"osd":4,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.34200000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52700000000000002,"15min":0.55300000000000005},"min":{"1min":0.41199999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68300000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.51100000000000001}]},{"osd":5,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53200000000000003,"15min":0.53500000000000003},"min":{"1min":0.38800000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70799999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.60599999999999998},"min":{"1min":0.40899999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70799999999999996,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.58899999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60599999999999998,"15min":0.61599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73699999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71099999999999997,"5min":1.038,"15min":1.038},"last":0.60299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":622,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194689,"statfs":{"total":200000000,"available":199362504,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.39700000000000002,"15min":0.377},"min":{"1min":0.27000000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.41999999999999998,"15min":0.40000000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.40999999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.40200000000000002,"15min":0.36599999999999999},"min":{"1min":0.28000000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.434},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.45400000000000001,"15min":0.43099999999999999},"min":{"1min":0.30199999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.35499999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48999999999999999,"15min":0.46899999999999997},"min":{"1min":0.35799999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.47499999999999998,"15min":0.45800000000000002},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.36399999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.66200000000000003,"15min":0.63400000000000001},"min":{"1min":0.442,"5min":0.373,"15min":0.373},"max":{"1min":3.2890000000000001,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65800000000000003,"15min":0.63600000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.57299999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.68500000000000005,"15min":0.63700000000000001},"min":{"1min":0.50600000000000001,"5min":0.44,"15min":0.44},"max":{"1min":3.4500000000000002,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.68999999999999995,"15min":0.67600000000000005},"min":{"1min":0.48299999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.181,"5min":3.181,"15min":3.181},"last":0.63500000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70999999999999996,"15min":0.69599999999999995},"min":{"1min":0.52100000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96399999999999997,"5min":1.024,"15min":1.024},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.71299999999999997,"15min":0.67900000000000005},"min":{"1min":0.53400000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.60899999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.72399999999999998,"15min":0.68600000000000005},"min":{"1min":0.56100000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.72499999999999998,"15min":0.69199999999999995},"min":{"1min":0.57799999999999996,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.68500000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574922,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1199,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194112,"statfs":{"total":200000000,"available":198771582,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.38900000000000001},"min":{"1min":0.28100000000000003,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.66600000000000004,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27800000000000002,"5min":0.223,"15min":0.223},"max":{"1min":0.65200000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.36399999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.436,"15min":0.42199999999999999},"min":{"1min":0.32800000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56299999999999994,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.44900000000000001,"15min":0.435},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58999999999999997,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.434}]},{"osd":3,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42999999999999999,"15min":0.432},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.63600000000000001,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.48099999999999998,"15min":0.47299999999999998},"min":{"1min":0.35299999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.63200000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42199999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.65500000000000003,"15min":0.64000000000000001},"min":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45900000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89700000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.61299999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67500000000000004,"15min":0.65600000000000003},"min":{"1min":0.48399999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.86499999999999999,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67700000000000005,"15min":0.68200000000000005},"min":{"1min":0.5,"5min":0.44,"15min":0.44},"max":{"1min":0.96599999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.58999999999999997}]},{"osd":6,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.56200000000000006,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91300000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69299999999999995,"15min":0.67900000000000005},"min":{"1min":0.56599999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94199999999999995,"5min":1.04,"15min":1.04},"last":0.58099999999999996}]},{"osd":7,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.55300000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.52100000000000002,"5min":0.498,"15min":0.498},"max":{"1min":0.95799999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.56999999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":622,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194689,"statfs":{"total":200000000,"available":199362470,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.41699999999999998,"15min":0.40999999999999998},"min":{"1min":0.28000000000000003,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.56599999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.50900000000000001,"15min":0.63700000000000001},"min":{"1min":0.34499999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72799999999999998,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":0.36299999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.434,"15min":0.42399999999999999},"min":{"1min":0.32700000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.434,"5min":0.44,"15min":0.44700000000000001},"min":{"1min":0.29799999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.65300000000000002,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.41899999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.41499999999999998},"min":{"1min":0.30399999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.65000000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.48899999999999999,"15min":0.48599999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76500000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.45200000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68200000000000005,"15min":0.65600000000000003},"min":{"1min":0.53600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73399999999999999,"15min":0.79900000000000004},"min":{"1min":0.54900000000000004,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.995,"5min":3.581,"15min":3.581},"last":0.627}]},{"osd":5,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68000000000000005,"15min":0.74399999999999999},"min":{"1min":0.502,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90500000000000003,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67700000000000005,"15min":0.67100000000000004},"min":{"1min":0.501,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85899999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.60699999999999998}]},{"osd":6,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.53800000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93999999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.48599999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0189999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.58999999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.53600000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70299999999999996,"15min":0.69099999999999995},"min":{"1min":0.625,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.64100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":624,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194688,"statfs":{"total":200000000,"available":199360946,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.38800000000000001,"15min":0.35899999999999999},"min":{"1min":0.255,"5min":0.253,"15min":0.253},"max":{"1min":0.56200000000000006,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.40600000000000003,"15min":0.39500000000000002},"min":{"1min":0.29199999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.626,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.41699999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.27400000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66300000000000003,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.45600000000000002,"15min":0.42399999999999999},"min":{"1min":0.30199999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45800000000000002}]},{"osd":2,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46100000000000002,"15min":0.43099999999999999},"min":{"1min":0.33700000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.48399999999999999,"15min":0.45000000000000001},"min":{"1min":0.378,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.47799999999999998}]},{"osd":4,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.66900000000000004,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.6660000000000004,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.751},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64200000000000002,"15min":0.60599999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93300000000000005,"5min":1.012,"15min":1.012},"last":0.60299999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.61299999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90400000000000003,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.71599999999999997,"15min":0.68000000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":4.6849999999999996,"5min":4.6849999999999996,"15min":4.6849999999999996},"last":0.85099999999999998}]},{"osd":6,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90100000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.47099999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.61599999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.71699999999999997,"15min":0.65400000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":4.7130000000000001,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68500000000000005,"15min":0.66700000000000004},"min":{"1min":0.495,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.82699999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":624,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194688,"statfs":{"total":200000000,"available":199360844,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.36899999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.73399999999999999,"5min":1.099,"15min":1.099},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.47799999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71099999999999997,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.57499999999999996}]},{"osd":1,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.39500000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72499999999999998,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59799999999999998,"15min":0.59499999999999997},"min":{"1min":0.40799999999999997,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77700000000000002,"5min":1.133,"15min":1.133},"last":0.58799999999999997}]},{"osd":2,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76300000000000001,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.44700000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.753,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.56599999999999995}]},{"osd":3,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.42799999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68700000000000006,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.50700000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.70799999999999996,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.59799999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.47599999999999998,"15min":0.48399999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.63400000000000001,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.34599999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68899999999999995,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.42099999999999999}]},{"osd":6,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.5},"min":{"1min":0.40100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.66600000000000004,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53800000000000003,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81200000000000006,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.45600000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:37:07 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53900000000000003,"15min":0.54900000000000004},"min":{"1min":0.38500000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.65400000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.432,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68100000000000005,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.44600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":622,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194689,"statfs":{"total":200000000,"available":199362434,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56999999999999995,"15min":0.59399999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68700000000000006,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.60099999999999998,"15min":0.57699999999999996},"min":{"1min":0.34499999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67100000000000004,"5min":3.242,"15min":3.242},"last":0.42899999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.625,"15min":0.60799999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70599999999999996,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.626,"15min":0.64900000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69299999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.58699999999999997}]},{"osd":2,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.61599999999999999,"15min":0.59599999999999997},"min":{"1min":0.39100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.71499999999999997,"5min":3.04,"15min":3.04},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59399999999999997,"15min":0.57699999999999996},"min":{"1min":0.40100000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72499999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.55800000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.66500000000000004,"15min":0.67800000000000005},"min":{"1min":0.44400000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.65500000000000003,"5min":4.032,"15min":4.032},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.70399999999999996,"15min":0.67800000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.77000000000000002,"5min":4.282,"15min":4.282},"last":0.46000000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.33900000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81399999999999995,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.51100000000000001,"15min":0.56499999999999995},"min":{"1min":0.36799999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.61499999999999999,"5min":1.101,"15min":1.101},"last":0.624}]},{"osd":6,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51300000000000001,"15min":0.55600000000000005},"min":{"1min":0.36499999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.629,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.505},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56499999999999995,"15min":0.58199999999999996},"min":{"1min":0.39100000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76800000000000002,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.61199999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.55400000000000005,"15min":0.56699999999999995},"min":{"1min":0.40200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.745,"5min":1.008,"15min":1.008},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.626,"15min":0.65800000000000003},"min":{"1min":0.38400000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84799999999999998,"5min":2,"15min":2},"last":0.64500000000000002}]}]}],"pool_statfs":[]}} 2024-01-27T02:38:01.099 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-27T02:38:01.099 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-27T02:38:01.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.277+0000 7f3994057700 1 Processor -- start 2024-01-27T02:38:01.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.277+0000 7f3994057700 1 -- start start 2024-01-27T02:38:01.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.278+0000 7f3994057700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c0cf280 0x7f398c0d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:01.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.278+0000 7f3994057700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f398c15d830 0x7f398c0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:01.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.278+0000 7f3994057700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f398c0b38d0 con 0x7f398c15d460 2024-01-27T02:38:01.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.278+0000 7f3994057700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f398c0d0390 con 0x7f398c15d830 2024-01-27T02:38:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.278+0000 7f3994057700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0cf780 con 0x7f398c0cf280 2024-01-27T02:38:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.279+0000 7f3991df3700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f398c15d460 0x7f398c15e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50796/0 (socket says 172.21.15.7:50796) 2024-01-27T02:38:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.279+0000 7f3991df3700 1 -- 172.21.15.7:0/1125269449 learned_addr learned my addr 172.21.15.7:0/1125269449 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.279+0000 7f39925f4700 1 --2- 172.21.15.7:0/1125269449 >> v2:172.21.15.188:3300/0 conn(0x7f398c15d830 0x7f398c0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.279+0000 7f39915f2700 1 --2- 172.21.15.7:0/1125269449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c0cf280 0x7f398c0d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.279+0000 7f3990df1700 1 -- 172.21.15.7:0/1125269449 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 874737565 0 0) 0x7f3988001110 con 0x7f398c15d460 2024-01-27T02:38:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.279+0000 7f3990df1700 1 -- 172.21.15.7:0/1125269449 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3978003570 con 0x7f398c15d460 2024-01-27T02:38:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.279+0000 7f39915f2700 1 -- 172.21.15.7:0/1125269449 >> v2:172.21.15.188:3300/0 conn(0x7f398c15d830 msgr2=0x7f398c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.279+0000 7f39915f2700 1 --2- 172.21.15.7:0/1125269449 >> v2:172.21.15.188:3300/0 conn(0x7f398c15d830 0x7f398c0cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.279+0000 7f39915f2700 1 -- 172.21.15.7:0/1125269449 >> v1:172.21.15.7:6789/0 conn(0x7f398c15d460 legacy=0x7f398c15e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.279+0000 7f39915f2700 1 -- 172.21.15.7:0/1125269449 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c0cfa60 con 0x7f398c0cf280 2024-01-27T02:38:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.279+0000 7f39925f4700 1 --2- 172.21.15.7:0/1125269449 >> v2:172.21.15.188:3300/0 conn(0x7f398c15d830 0x7f398c0cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:38:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.279+0000 7f39915f2700 1 --2- 172.21.15.7:0/1125269449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c0cf280 0x7f398c0d0820 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f397c0098e0 tx=0x7f397c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=738ecbdcf99ac3a9 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.280+0000 7f3990df1700 1 -- 172.21.15.7:0/1125269449 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c011630 con 0x7f398c0cf280 2024-01-27T02:38:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.280+0000 7f3990df1700 1 -- 172.21.15.7:0/1125269449 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397c011790 con 0x7f398c0cf280 2024-01-27T02:38:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.280+0000 7f3990df1700 1 -- 172.21.15.7:0/1125269449 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c011b30 con 0x7f398c0cf280 2024-01-27T02:38:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.280+0000 7f3994057700 1 -- 172.21.15.7:0/1125269449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c0cf280 msgr2=0x7f398c0d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.280+0000 7f3994057700 1 --2- 172.21.15.7:0/1125269449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c0cf280 0x7f398c0d0820 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f397c0098e0 tx=0x7f397c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.280+0000 7f3994057700 1 -- 172.21.15.7:0/1125269449 shutdown_connections 2024-01-27T02:38:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.280+0000 7f3994057700 1 --2- 172.21.15.7:0/1125269449 >> v2:172.21.15.188:3300/0 conn(0x7f398c15d830 0x7f398c0cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.280+0000 7f3994057700 1 --2- 172.21.15.7:0/1125269449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c0cf280 0x7f398c0d0820 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.280+0000 7f3994057700 1 -- 172.21.15.7:0/1125269449 >> 172.21.15.7:0/1125269449 conn(0x7f398c063580 msgr2=0x7f398c063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:01.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.280+0000 7f3994057700 1 -- 172.21.15.7:0/1125269449 shutdown_connections 2024-01-27T02:38:01.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.280+0000 7f3994057700 1 -- 172.21.15.7:0/1125269449 wait complete. 2024-01-27T02:38:01.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.282+0000 7f3994057700 1 Processor -- start 2024-01-27T02:38:01.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.282+0000 7f3994057700 1 -- start start 2024-01-27T02:38:01.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.282+0000 7f3994057700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f398c0cf280 0x7f398c0c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:01.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.282+0000 7f3994057700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c15d460 0x7f398c0ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.282+0000 7f3994057700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f398c15d830 0x7f398c150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.282+0000 7f3994057700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c16e4f0 con 0x7f398c15d830 2024-01-27T02:38:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.282+0000 7f3994057700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c150dc0 con 0x7f398c0cf280 2024-01-27T02:38:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.282+0000 7f3994057700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398c1508b0 con 0x7f398c15d460 2024-01-27T02:38:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.283+0000 7f39925f4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f398c15d830 0x7f398c150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.283+0000 7f39925f4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f398c15d830 0x7f398c150370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44218/0 (socket says 172.21.15.7:44218) 2024-01-27T02:38:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.283+0000 7f39925f4700 1 -- 172.21.15.7:0/2221014740 learned_addr learned my addr 172.21.15.7:0/2221014740 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.283+0000 7f39915f2700 1 --2- 172.21.15.7:0/2221014740 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c15d460 0x7f398c0ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.283+0000 7f3991df3700 1 --2- 172.21.15.7:0/2221014740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f398c0cf280 0x7f398c0c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.283+0000 7f39925f4700 1 -- 172.21.15.7:0/2221014740 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c15d460 msgr2=0x7f398c0ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.283+0000 7f39925f4700 1 --2- 172.21.15.7:0/2221014740 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c15d460 0x7f398c0ca200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.283+0000 7f39925f4700 1 -- 172.21.15.7:0/2221014740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f398c0cf280 msgr2=0x7f398c0c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.283+0000 7f39925f4700 1 --2- 172.21.15.7:0/2221014740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f398c0cf280 0x7f398c0c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.283+0000 7f39925f4700 1 -- 172.21.15.7:0/2221014740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3984009a20 con 0x7f398c15d830 2024-01-27T02:38:01.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.283+0000 7f3991df3700 1 --2- 172.21.15.7:0/2221014740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f398c0cf280 0x7f398c0c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:01.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.283+0000 7f39925f4700 1 --2- 172.21.15.7:0/2221014740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f398c15d830 0x7f398c150370 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3984009f50 tx=0x7f398400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:01.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.283+0000 7f3981ffb700 1 -- 172.21.15.7:0/2221014740 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398400cc00 con 0x7f398c15d830 2024-01-27T02:38:01.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.283+0000 7f3981ffb700 1 -- 172.21.15.7:0/2221014740 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398400f040 con 0x7f398c15d830 2024-01-27T02:38:01.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.283+0000 7f3994057700 1 -- 172.21.15.7:0/2221014740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397c009420 con 0x7f398c15d830 2024-01-27T02:38:01.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.283+0000 7f3981ffb700 1 -- 172.21.15.7:0/2221014740 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984014480 con 0x7f398c15d830 2024-01-27T02:38:01.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.284+0000 7f3994057700 1 -- 172.21.15.7:0/2221014740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398c16dbc0 con 0x7f398c15d830 2024-01-27T02:38:01.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.285+0000 7f3994057700 1 -- 172.21.15.7:0/2221014740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398c072060 con 0x7f398c15d830 2024-01-27T02:38:01.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.285+0000 7f3981ffb700 1 -- 172.21.15.7:0/2221014740 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3984003020 con 0x7f398c15d830 2024-01-27T02:38:01.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.285+0000 7f3981ffb700 1 --2- 172.21.15.7:0/2221014740 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3970066f30 0x7f39700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:01.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.285+0000 7f3981ffb700 1 -- 172.21.15.7:0/2221014740 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3984084ea0 con 0x7f398c15d830 2024-01-27T02:38:01.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.286+0000 7f3991df3700 1 --2- 172.21.15.7:0/2221014740 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3970066f30 0x7f39700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:01.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.286+0000 7f3991df3700 1 --2- 172.21.15.7:0/2221014740 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3970066f30 0x7f39700693e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3988001270 tx=0x7f3988009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:01.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.290+0000 7f3981ffb700 1 -- 172.21.15.7:0/2221014740 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3984052110 con 0x7f398c15d830 2024-01-27T02:38:01.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.578+0000 7f3994057700 1 -- 172.21.15.7:0/2221014740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f398c16e630 con 0x7f398c15d830 2024-01-27T02:38:01.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.582+0000 7f3981ffb700 1 -- 172.21.15.7:0/2221014740 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+0 (secure 0 0 0) 0x7f3984055730 con 0x7f398c15d830 2024-01-27T02:38:01.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.585+0000 7f3994057700 1 -- 172.21.15.7:0/2221014740 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3970066f30 msgr2=0x7f39700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:01.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.585+0000 7f3994057700 1 --2- 172.21.15.7:0/2221014740 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3970066f30 0x7f39700693e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3988001270 tx=0x7f3988009040 comp rx=0 tx=0).stop 2024-01-27T02:38:01.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.585+0000 7f3994057700 1 -- 172.21.15.7:0/2221014740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f398c15d830 msgr2=0x7f398c150370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:01.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.585+0000 7f3994057700 1 --2- 172.21.15.7:0/2221014740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f398c15d830 0x7f398c150370 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3984009f50 tx=0x7f398400b040 comp rx=0 tx=0).stop 2024-01-27T02:38:01.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.585+0000 7f3994057700 1 -- 172.21.15.7:0/2221014740 shutdown_connections 2024-01-27T02:38:01.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.585+0000 7f3994057700 1 --2- 172.21.15.7:0/2221014740 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3970066f30 0x7f39700693e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.585+0000 7f3994057700 1 --2- 172.21.15.7:0/2221014740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f398c15d830 0x7f398c150370 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.585+0000 7f3994057700 1 --2- 172.21.15.7:0/2221014740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f398c0cf280 0x7f398c0c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.585+0000 7f3994057700 1 --2- 172.21.15.7:0/2221014740 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c15d460 0x7f398c0ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.585+0000 7f3994057700 1 -- 172.21.15.7:0/2221014740 >> 172.21.15.7:0/2221014740 conn(0x7f398c063580 msgr2=0x7f398c151480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:01.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.585+0000 7f3994057700 1 -- 172.21.15.7:0/2221014740 shutdown_connections 2024-01-27T02:38:01.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.585+0000 7f3994057700 1 -- 172.21.15.7:0/2221014740 wait complete. 2024-01-27T02:38:01.587 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-01-27T02:38:01.599 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:38:01.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.730+0000 7f1a9da09700 1 Processor -- start 2024-01-27T02:38:01.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.730+0000 7f1a9da09700 1 -- start start 2024-01-27T02:38:01.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.731+0000 7f1a9da09700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a9806a9f0 0x7f1a98063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:01.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.731+0000 7f1a9da09700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1a9815c7e0 0x7f1a98158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:01.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.731+0000 7f1a9da09700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a98159570 con 0x7f1a9806adc0 2024-01-27T02:38:01.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.731+0000 7f1a9da09700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a98159290 con 0x7f1a9815c7e0 2024-01-27T02:38:01.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.731+0000 7f1a9da09700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9815bd00 con 0x7f1a9806a9f0 2024-01-27T02:38:01.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.731+0000 7f1a967fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1a9806adc0 0x7f1a98158590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44730/0 (socket says 172.21.15.7:44730) 2024-01-27T02:38:01.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.731+0000 7f1a967fc700 1 -- 172.21.15.7:0/2377822374 learned_addr learned my addr 172.21.15.7:0/2377822374 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.732+0000 7f1a96ffd700 1 --2- 172.21.15.7:0/2377822374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a9806a9f0 0x7f1a98063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.732+0000 7f1a977fe700 1 --2- 172.21.15.7:0/2377822374 >> v2:172.21.15.188:3300/0 conn(0x7f1a9815c7e0 0x7f1a98158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.732+0000 7f1a95ffb700 1 -- 172.21.15.7:0/2377822374 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2066809833 0 0) 0x7f1a88001110 con 0x7f1a9806adc0 2024-01-27T02:38:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.732+0000 7f1a95ffb700 1 -- 172.21.15.7:0/2377822374 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a84003570 con 0x7f1a9806adc0 2024-01-27T02:38:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.732+0000 7f1a95ffb700 1 -- 172.21.15.7:0/2377822374 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2608312944 0 0) 0x7f1a84003570 con 0x7f1a9806adc0 2024-01-27T02:38:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.732+0000 7f1a96ffd700 1 -- 172.21.15.7:0/2377822374 >> v2:172.21.15.188:3300/0 conn(0x7f1a9815c7e0 msgr2=0x7f1a98158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.732+0000 7f1a96ffd700 1 --2- 172.21.15.7:0/2377822374 >> v2:172.21.15.188:3300/0 conn(0x7f1a9815c7e0 0x7f1a98158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.732+0000 7f1a96ffd700 1 -- 172.21.15.7:0/2377822374 >> v1:172.21.15.7:6789/0 conn(0x7f1a9806adc0 legacy=0x7f1a98158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.732+0000 7f1a96ffd700 1 -- 172.21.15.7:0/2377822374 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9815bfe0 con 0x7f1a9806a9f0 2024-01-27T02:38:01.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.732+0000 7f1a977fe700 1 --2- 172.21.15.7:0/2377822374 >> v2:172.21.15.188:3300/0 conn(0x7f1a9815c7e0 0x7f1a98158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:38:01.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.732+0000 7f1a96ffd700 1 --2- 172.21.15.7:0/2377822374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a9806a9f0 0x7f1a98063580 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1a800098e0 tx=0x7f1a800091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=746a8737a9fe3c33 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:01.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.733+0000 7f1a95ffb700 1 -- 172.21.15.7:0/2377822374 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a88001600 con 0x7f1a9806a9f0 2024-01-27T02:38:01.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.733+0000 7f1a95ffb700 1 -- 172.21.15.7:0/2377822374 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a80011420 con 0x7f1a9806a9f0 2024-01-27T02:38:01.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.733+0000 7f1a95ffb700 1 -- 172.21.15.7:0/2377822374 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a800117c0 con 0x7f1a9806a9f0 2024-01-27T02:38:01.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.733+0000 7f1a9da09700 1 -- 172.21.15.7:0/2377822374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a9806a9f0 msgr2=0x7f1a98063580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:01.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.733+0000 7f1a9da09700 1 --2- 172.21.15.7:0/2377822374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a9806a9f0 0x7f1a98063580 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1a800098e0 tx=0x7f1a800091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.733+0000 7f1a9da09700 1 -- 172.21.15.7:0/2377822374 shutdown_connections 2024-01-27T02:38:01.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.733+0000 7f1a9da09700 1 --2- 172.21.15.7:0/2377822374 >> v2:172.21.15.188:3300/0 conn(0x7f1a9815c7e0 0x7f1a98158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.733+0000 7f1a9da09700 1 --2- 172.21.15.7:0/2377822374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a9806a9f0 0x7f1a98063580 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.733+0000 7f1a9da09700 1 -- 172.21.15.7:0/2377822374 >> 172.21.15.7:0/2377822374 conn(0x7f1a980cdc30 msgr2=0x7f1a980ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:01.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.733+0000 7f1a9da09700 1 -- 172.21.15.7:0/2377822374 shutdown_connections 2024-01-27T02:38:01.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.733+0000 7f1a9da09700 1 -- 172.21.15.7:0/2377822374 wait complete. 2024-01-27T02:38:01.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.735+0000 7f1a9da09700 1 Processor -- start 2024-01-27T02:38:01.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.735+0000 7f1a9da09700 1 -- start start 2024-01-27T02:38:01.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.735+0000 7f1a9da09700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a9806a9f0 0x7f1a9816b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:01.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.735+0000 7f1a9da09700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a9806adc0 0x7f1a980c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:01.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.735+0000 7f1a9da09700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a9815c7e0 0x7f1a9813d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:01.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.735+0000 7f1a9da09700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a981692c0 con 0x7f1a9815c7e0 2024-01-27T02:38:01.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.735+0000 7f1a9da09700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9816b8d0 con 0x7f1a9806adc0 2024-01-27T02:38:01.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.735+0000 7f1a9da09700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a981686b0 con 0x7f1a9806a9f0 2024-01-27T02:38:01.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.735+0000 7f1a977fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a9815c7e0 0x7f1a9813d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:01.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.736+0000 7f1a977fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a9815c7e0 0x7f1a9813d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40112/0 (socket says 172.21.15.7:40112) 2024-01-27T02:38:01.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.736+0000 7f1a977fe700 1 -- 172.21.15.7:0/120736703 learned_addr learned my addr 172.21.15.7:0/120736703 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:01.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.736+0000 7f1a96ffd700 1 --2- 172.21.15.7:0/120736703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a9806a9f0 0x7f1a9816b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:01.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.736+0000 7f1a967fc700 1 --2- 172.21.15.7:0/120736703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a9806adc0 0x7f1a980c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:01.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.736+0000 7f1a977fe700 1 -- 172.21.15.7:0/120736703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a9806a9f0 msgr2=0x7f1a9816b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:01.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.736+0000 7f1a977fe700 1 --2- 172.21.15.7:0/120736703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a9806a9f0 0x7f1a9816b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.736+0000 7f1a977fe700 1 -- 172.21.15.7:0/120736703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a9806adc0 msgr2=0x7f1a980c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:01.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.736+0000 7f1a977fe700 1 --2- 172.21.15.7:0/120736703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a9806adc0 0x7f1a980c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.736+0000 7f1a977fe700 1 -- 172.21.15.7:0/120736703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a90009a20 con 0x7f1a9815c7e0 2024-01-27T02:38:01.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.736+0000 7f1a977fe700 1 --2- 172.21.15.7:0/120736703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a9815c7e0 0x7f1a9813d590 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000efc0 tx=0x7f1a9000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:01.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.736+0000 7f1a8effd700 1 -- 172.21.15.7:0/120736703 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9000cd30 con 0x7f1a9815c7e0 2024-01-27T02:38:01.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.736+0000 7f1a8effd700 1 -- 172.21.15.7:0/120736703 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9000f040 con 0x7f1a9815c7e0 2024-01-27T02:38:01.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.736+0000 7f1a8effd700 1 -- 172.21.15.7:0/120736703 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a900146b0 con 0x7f1a9815c7e0 2024-01-27T02:38:01.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.736+0000 7f1a9da09700 1 -- 172.21.15.7:0/120736703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a80009420 con 0x7f1a9815c7e0 2024-01-27T02:38:01.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.736+0000 7f1a9da09700 1 -- 172.21.15.7:0/120736703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a98168c70 con 0x7f1a9815c7e0 2024-01-27T02:38:01.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.737+0000 7f1a9da09700 1 -- 172.21.15.7:0/120736703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a9804c710 con 0x7f1a9815c7e0 2024-01-27T02:38:01.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.738+0000 7f1a8effd700 1 -- 172.21.15.7:0/120736703 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1a90014aa0 con 0x7f1a9815c7e0 2024-01-27T02:38:01.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.738+0000 7f1a8effd700 1 --2- 172.21.15.7:0/120736703 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a78066f30 0x7f1a780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:01.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.738+0000 7f1a8effd700 1 -- 172.21.15.7:0/120736703 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1a90085390 con 0x7f1a9815c7e0 2024-01-27T02:38:01.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.738+0000 7f1a96ffd700 1 --2- 172.21.15.7:0/120736703 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a78066f30 0x7f1a780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:01.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.739+0000 7f1a96ffd700 1 --2- 172.21.15.7:0/120736703 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a78066f30 0x7f1a780693e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1a800093f0 tx=0x7f1a80016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:01.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.743+0000 7f1a8effd700 1 -- 172.21.15.7:0/120736703 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a90052600 con 0x7f1a9815c7e0 2024-01-27T02:38:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.989+0000 7f1a9da09700 1 -- 172.21.15.7:0/120736703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a98169400 con 0x7f1a9815c7e0 2024-01-27T02:38:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.990+0000 7f1a8effd700 1 -- 172.21.15.7:0/120736703 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f1a90055c20 con 0x7f1a9815c7e0 2024-01-27T02:38:01.991 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:01.991 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2767759975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2767759975},{"type":"v1","addr":"172.21.15.188:6836","nonce":2767759975}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":5768,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/245473575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":245473575},{"type":"v1","addr":"172.21.15.7:6834","nonce":245473575}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1334164270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1334164270},{"type":"v1","addr":"172.21.15.188:6837","nonce":1334164270}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:37:51.231918+0000","modified":"2024-01-27T02:37:54.851112+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6657},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6657":{"gid":6657,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1251886476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1251886476},{"type":"v1","addr":"172.21.15.7:6835","nonce":1251886476}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2024-01-27T02:38:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.992+0000 7f1a9da09700 1 -- 172.21.15.7:0/120736703 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a78066f30 msgr2=0x7f1a780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.992+0000 7f1a9da09700 1 --2- 172.21.15.7:0/120736703 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a78066f30 0x7f1a780693e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1a800093f0 tx=0x7f1a80016000 comp rx=0 tx=0).stop 2024-01-27T02:38:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.992+0000 7f1a9da09700 1 -- 172.21.15.7:0/120736703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a9815c7e0 msgr2=0x7f1a9813d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.992+0000 7f1a9da09700 1 --2- 172.21.15.7:0/120736703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a9815c7e0 0x7f1a9813d590 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000efc0 tx=0x7f1a9000b040 comp rx=0 tx=0).stop 2024-01-27T02:38:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.992+0000 7f1a9da09700 1 -- 172.21.15.7:0/120736703 shutdown_connections 2024-01-27T02:38:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.992+0000 7f1a9da09700 1 --2- 172.21.15.7:0/120736703 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a78066f30 0x7f1a780693e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.992+0000 7f1a9da09700 1 --2- 172.21.15.7:0/120736703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a9815c7e0 0x7f1a9813d590 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.992+0000 7f1a9da09700 1 --2- 172.21.15.7:0/120736703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a9806a9f0 0x7f1a9816b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.992+0000 7f1a9da09700 1 --2- 172.21.15.7:0/120736703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a9806adc0 0x7f1a980c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.992+0000 7f1a9da09700 1 -- 172.21.15.7:0/120736703 >> 172.21.15.7:0/120736703 conn(0x7f1a980cdc30 msgr2=0x7f1a98063580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.992+0000 7f1a9da09700 1 -- 172.21.15.7:0/120736703 shutdown_connections 2024-01-27T02:38:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:01.992+0000 7f1a9da09700 1 -- 172.21.15.7:0/120736703 wait complete. 2024-01-27T02:38:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 58 2024-01-27T02:38:02.410 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-27T02:37:51.231918+0000', 'modified': '2024-01-27T02:37:54.851112+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6657}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6657': {'gid': 6657, 'name': 'a', 'rank': 0, 'incarnation': 54, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6835/1251886476', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6833', 'nonce': 1251886476}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 1251886476}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-27T02:38:02.410 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-27T02:38:02.410 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-27T02:38:02.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.553+0000 7f7b0a352700 1 Processor -- start 2024-01-27T02:38:02.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.553+0000 7f7b0a352700 1 -- start start 2024-01-27T02:38:02.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.554+0000 7f7b0a352700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7b040d10a0 0x7f7b040b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.554+0000 7f7b0a352700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b04159b40 0x7f7b04159f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.554+0000 7f7b0a352700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b040b4650 con 0x7f7b040d1470 2024-01-27T02:38:02.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.554+0000 7f7b0a352700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b040b34d0 con 0x7f7b040d10a0 2024-01-27T02:38:02.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.554+0000 7f7b0a352700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b040b3d60 con 0x7f7b04159b40 2024-01-27T02:38:02.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.554+0000 7f7b03fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7b040d10a0 0x7f7b040b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.554+0000 7f7b037fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7b040d1470 0x7f7b040b2dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44734/0 (socket says 172.21.15.7:44734) 2024-01-27T02:38:02.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.554+0000 7f7b037fe700 1 -- 172.21.15.7:0/586954512 learned_addr learned my addr 172.21.15.7:0/586954512 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:02.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.554+0000 7f7b088ef700 1 --2- 172.21.15.7:0/586954512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b04159b40 0x7f7b04159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.554+0000 7f7b03fff700 1 -- 172.21.15.7:0/586954512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b04159b40 msgr2=0x7f7b04159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:02.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.554+0000 7f7b03fff700 1 --2- 172.21.15.7:0/586954512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b04159b40 0x7f7b04159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.554+0000 7f7b03fff700 1 -- 172.21.15.7:0/586954512 >> v1:172.21.15.7:6789/0 conn(0x7f7b040d1470 legacy=0x7f7b040b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:02.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.554+0000 7f7b03fff700 1 -- 172.21.15.7:0/586954512 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b040b4000 con 0x7f7b040d10a0 2024-01-27T02:38:02.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.554+0000 7f7b02ffd700 1 -- 172.21.15.7:0/586954512 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228396141 0 0) 0x7f7b040b4650 con 0x7f7b040d1470 2024-01-27T02:38:02.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.555+0000 7f7b03fff700 1 --2- 172.21.15.7:0/586954512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b040d10a0 0x7f7b040b1590 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f7af40098e0 tx=0x7f7af40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a11a94892bc6b71 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:02.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.555+0000 7f7b02ffd700 1 -- 172.21.15.7:0/586954512 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4009e80 con 0x7f7b040d10a0 2024-01-27T02:38:02.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.555+0000 7f7b02ffd700 1 -- 172.21.15.7:0/586954512 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af400b860 con 0x7f7b040d10a0 2024-01-27T02:38:02.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.555+0000 7f7b02ffd700 1 -- 172.21.15.7:0/586954512 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af400bbe0 con 0x7f7b040d10a0 2024-01-27T02:38:02.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.555+0000 7f7b0a352700 1 -- 172.21.15.7:0/586954512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b040d10a0 msgr2=0x7f7b040b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:02.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.555+0000 7f7b0a352700 1 --2- 172.21.15.7:0/586954512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b040d10a0 0x7f7b040b1590 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f7af40098e0 tx=0x7f7af40091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.555+0000 7f7b0a352700 1 -- 172.21.15.7:0/586954512 shutdown_connections 2024-01-27T02:38:02.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.555+0000 7f7b0a352700 1 --2- 172.21.15.7:0/586954512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b04159b40 0x7f7b04159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.555+0000 7f7b0a352700 1 --2- 172.21.15.7:0/586954512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b040d10a0 0x7f7b040b1590 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.555+0000 7f7b0a352700 1 -- 172.21.15.7:0/586954512 >> 172.21.15.7:0/586954512 conn(0x7f7b040679b0 msgr2=0x7f7b04067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:02.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.555+0000 7f7b0a352700 1 -- 172.21.15.7:0/586954512 shutdown_connections 2024-01-27T02:38:02.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.555+0000 7f7b0a352700 1 -- 172.21.15.7:0/586954512 wait complete. 2024-01-27T02:38:02.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.557+0000 7f7b0a352700 1 Processor -- start 2024-01-27T02:38:02.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.557+0000 7f7b0a352700 1 -- start start 2024-01-27T02:38:02.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.558+0000 7f7b0a352700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b040d10a0 0x7f7b0416af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.558+0000 7f7b0a352700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b040d1470 0x7f7b0415d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.558+0000 7f7b0a352700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b04159b40 0x7f7b0415b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.558+0000 7f7b0a352700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0415bac0 con 0x7f7b040d1470 2024-01-27T02:38:02.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.558+0000 7f7b0a352700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0416aac0 con 0x7f7b040d10a0 2024-01-27T02:38:02.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.558+0000 7f7b0a352700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0415bcd0 con 0x7f7b04159b40 2024-01-27T02:38:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.558+0000 7f7b037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b040d1470 0x7f7b0415d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.558+0000 7f7b037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b040d1470 0x7f7b0415d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40144/0 (socket says 172.21.15.7:40144) 2024-01-27T02:38:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.558+0000 7f7b037fe700 1 -- 172.21.15.7:0/1283197242 learned_addr learned my addr 172.21.15.7:0/1283197242 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.558+0000 7f7b037fe700 1 -- 172.21.15.7:0/1283197242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b04159b40 msgr2=0x7f7b0415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.558+0000 7f7b03fff700 1 --2- 172.21.15.7:0/1283197242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b040d10a0 0x7f7b0416af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.558+0000 7f7b037fe700 1 --2- 172.21.15.7:0/1283197242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b04159b40 0x7f7b0415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.558+0000 7f7b037fe700 1 -- 172.21.15.7:0/1283197242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b040d10a0 msgr2=0x7f7b0416af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.558+0000 7f7b037fe700 1 --2- 172.21.15.7:0/1283197242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b040d10a0 0x7f7b0416af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.558+0000 7f7b037fe700 1 -- 172.21.15.7:0/1283197242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af4009420 con 0x7f7b040d1470 2024-01-27T02:38:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.558+0000 7f7b037fe700 1 --2- 172.21.15.7:0/1283197242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b040d1470 0x7f7b0415d570 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00c7e0 tx=0x7f7aec007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.558+0000 7f7afbfff700 1 -- 172.21.15.7:0/1283197242 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec005b30 con 0x7f7b040d1470 2024-01-27T02:38:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.558+0000 7f7afbfff700 1 -- 172.21.15.7:0/1283197242 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec005c90 con 0x7f7b040d1470 2024-01-27T02:38:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.558+0000 7f7b0a352700 1 -- 172.21.15.7:0/1283197242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0415bfb0 con 0x7f7b040d1470 2024-01-27T02:38:02.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.558+0000 7f7afbfff700 1 -- 172.21.15.7:0/1283197242 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec007300 con 0x7f7b040d1470 2024-01-27T02:38:02.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.559+0000 7f7b0a352700 1 -- 172.21.15.7:0/1283197242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0416a750 con 0x7f7b040d1470 2024-01-27T02:38:02.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.560+0000 7f7afbfff700 1 -- 172.21.15.7:0/1283197242 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7aec01b400 con 0x7f7b040d1470 2024-01-27T02:38:02.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.560+0000 7f7afbfff700 1 --2- 172.21.15.7:0/1283197242 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ae4066f30 0x7f7ae40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.561+0000 7f7afbfff700 1 -- 172.21.15.7:0/1283197242 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7aec084020 con 0x7f7b040d1470 2024-01-27T02:38:02.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.561+0000 7f7b03fff700 1 --2- 172.21.15.7:0/1283197242 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ae4066f30 0x7f7ae40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.561+0000 7f7b03fff700 1 --2- 172.21.15.7:0/1283197242 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ae4066f30 0x7f7ae40693e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7b040b0d20 tx=0x7f7af4017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:02.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.562+0000 7f7b13655b80 1 Processor -- start 2024-01-27T02:38:02.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.562+0000 7f7b13655b80 1 -- start start 2024-01-27T02:38:02.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.563+0000 7f7b13655b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7858f010 0x556a7858f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.563+0000 7f7b13655b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a785d7360 0x556a785b9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.563+0000 7f7b13655b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a7858e5d0 0x556a785b9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.563+0000 7f7b13655b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a784929f0 con 0x556a785d7360 2024-01-27T02:38:02.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.563+0000 7f7b13655b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a784e4130 con 0x556a7858f010 2024-01-27T02:38:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.563+0000 7f7b13655b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a784e1930 con 0x556a7858e5d0 2024-01-27T02:38:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.563+0000 7f7b037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a785d7360 0x556a785b9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.563+0000 7f7b037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a785d7360 0x556a785b9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40152/0 (socket says 172.21.15.7:40152) 2024-01-27T02:38:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.563+0000 7f7b037fe700 1 -- 172.21.15.7:0/4264085888 learned_addr learned my addr 172.21.15.7:0/4264085888 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.564+0000 7f7b03fff700 1 --2- 172.21.15.7:0/4264085888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7858f010 0x556a7858f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.564+0000 7f7b088ef700 1 --2- 172.21.15.7:0/4264085888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a7858e5d0 0x556a785b9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.564+0000 7f7b037fe700 1 -- 172.21.15.7:0/4264085888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a7858e5d0 msgr2=0x556a785b9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.564+0000 7f7b037fe700 1 --2- 172.21.15.7:0/4264085888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a7858e5d0 0x556a785b9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.564+0000 7f7b037fe700 1 -- 172.21.15.7:0/4264085888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7858f010 msgr2=0x556a7858f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.564+0000 7f7b037fe700 1 --2- 172.21.15.7:0/4264085888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7858f010 0x556a7858f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.564+0000 7f7b037fe700 1 -- 172.21.15.7:0/4264085888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a7831cb10 con 0x556a785d7360 2024-01-27T02:38:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.564+0000 7f7b088ef700 1 --2- 172.21.15.7:0/4264085888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a7858e5d0 0x556a785b9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:38:02.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.564+0000 7f7b037fe700 1 --2- 172.21.15.7:0/4264085888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a785d7360 0x556a785b9840 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f7aec082570 tx=0x7f7aec0824a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc80b0ebd7bfbd5c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:02.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.564+0000 7f7af97fa700 1 -- 172.21.15.7:0/4264085888 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec054e50 con 0x556a785d7360 2024-01-27T02:38:02.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.564+0000 7f7af97fa700 1 -- 172.21.15.7:0/4264085888 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec05bf70 con 0x556a785d7360 2024-01-27T02:38:02.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.564+0000 7f7af97fa700 1 -- 172.21.15.7:0/4264085888 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec005e00 con 0x556a785d7360 2024-01-27T02:38:02.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.565+0000 7f7b13655b80 1 -- 172.21.15.7:0/4264085888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a785d7360 msgr2=0x556a785b9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:02.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.565+0000 7f7b13655b80 1 --2- 172.21.15.7:0/4264085888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a785d7360 0x556a785b9840 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f7aec082570 tx=0x7f7aec0824a0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.565+0000 7f7b13655b80 1 -- 172.21.15.7:0/4264085888 shutdown_connections 2024-01-27T02:38:02.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.565+0000 7f7b13655b80 1 --2- 172.21.15.7:0/4264085888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a7858e5d0 0x556a785b9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.565+0000 7f7b13655b80 1 --2- 172.21.15.7:0/4264085888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7858f010 0x556a7858f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.565+0000 7f7b13655b80 1 --2- 172.21.15.7:0/4264085888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a785d7360 0x556a785b9840 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.565+0000 7f7b13655b80 1 -- 172.21.15.7:0/4264085888 >> 172.21.15.7:0/4264085888 conn(0x556a7858ea70 msgr2=0x556a785d7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.565+0000 7f7b13655b80 1 -- 172.21.15.7:0/4264085888 shutdown_connections 2024-01-27T02:38:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.565+0000 7f7b13655b80 1 -- 172.21.15.7:0/4264085888 wait complete. 2024-01-27T02:38:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.566+0000 7f7b13655b80 1 -- start start 2024-01-27T02:38:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.566+0000 7f7b13655b80 1 Processor -- start 2024-01-27T02:38:02.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.567+0000 7f7b13655b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a7858e5d0 0x556a785b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.567+0000 7f7b13655b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7858f010 0x556a785d6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.567+0000 7f7b13655b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a785d7360 0x556a785e6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.567+0000 7f7b13655b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a784dd720 con 0x556a7858e5d0 2024-01-27T02:38:02.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.567+0000 7f7b13655b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a784891b0 con 0x556a7858f010 2024-01-27T02:38:02.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.567+0000 7f7b13655b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a78488e50 con 0x556a785d7360 2024-01-27T02:38:02.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.567+0000 7f7b13655b80 10 client.0 _ll_register_callbacks cb 0x556a782eb7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-27T02:38:02.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.567+0000 7f7b03fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a7858e5d0 0x556a785b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.567+0000 7f7b03fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a7858e5d0 0x556a785b4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40166/0 (socket says 172.21.15.7:40166) 2024-01-27T02:38:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.567+0000 7f7b03fff700 1 -- 172.21.15.7:0/2153321380 learned_addr learned my addr 172.21.15.7:0/2153321380 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.567+0000 7f7b088ef700 1 --2- 172.21.15.7:0/2153321380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a785d7360 0x556a785e6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.567+0000 7f7b037fe700 1 --2- 172.21.15.7:0/2153321380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7858f010 0x556a785d6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.568+0000 7f7b03fff700 1 -- 172.21.15.7:0/2153321380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a785d7360 msgr2=0x556a785e6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.568+0000 7f7b03fff700 1 --2- 172.21.15.7:0/2153321380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a785d7360 0x556a785e6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.568+0000 7f7b03fff700 1 -- 172.21.15.7:0/2153321380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7858f010 msgr2=0x556a785d6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.568+0000 7f7b03fff700 1 --2- 172.21.15.7:0/2153321380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7858f010 0x556a785d6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.568+0000 7f7b03fff700 1 -- 172.21.15.7:0/2153321380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0416a750 con 0x556a7858e5d0 2024-01-27T02:38:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.568+0000 7f7b03fff700 1 --2- 172.21.15.7:0/2153321380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a7858e5d0 0x556a785b4d70 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f7af4013000 tx=0x7f7af401e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0431ec4cc2875f0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.568+0000 7f7ae27fc700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-27T02:38:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.568+0000 7f7ae27fc700 1 -- 172.21.15.7:0/2153321380 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af402a640 con 0x556a7858e5d0 2024-01-27T02:38:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.568+0000 7f7ae27fc700 1 -- 172.21.15.7:0/2153321380 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7af402a7a0 con 0x556a7858e5d0 2024-01-27T02:38:02.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.568+0000 7f7b13655b80 1 -- 172.21.15.7:0/2153321380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a78582190 con 0x556a7858e5d0 2024-01-27T02:38:02.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.568+0000 7f7ae27fc700 1 -- 172.21.15.7:0/2153321380 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7af402ab70 con 0x556a7858e5d0 2024-01-27T02:38:02.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.568+0000 7f7b13655b80 10 client.6813 fetch_fsmap learned FSMap version 58 2024-01-27T02:38:02.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.568+0000 7f7b13655b80 1 -- 172.21.15.7:0/2153321380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x556a784ab290 con 0x556a7858e5d0 2024-01-27T02:38:02.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.568+0000 7f7ae27fc700 1 -- 172.21.15.7:0/2153321380 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556a78582190 con 0x556a7858e5d0 2024-01-27T02:38:02.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.568+0000 7f7ae27fc700 1 -- 172.21.15.7:0/2153321380 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 58) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f7af402acd0 con 0x556a7858e5d0 2024-01-27T02:38:02.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.568+0000 7f7b13655b80 10 client.6813 fetch_fsmap finished waiting for FSMap version 58 2024-01-27T02:38:02.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.568+0000 7f7b13655b80 10 client.6813 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-27T02:38:02.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.568+0000 7f7b13655b80 20 client.6813 populate_metadata read hostname 'smithi007' 2024-01-27T02:38:02.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.569+0000 7f7b13655b80 1 --2- 172.21.15.7:0/2153321380 >> [v2:172.21.15.7:6833/1251886476,v1:172.21.15.7:6835/1251886476] conn(0x556a785bad00 0x556a785bb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.569+0000 7f7b13655b80 4 client.6813 mds_command: new command op to 6657 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-27T02:38:02.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.569+0000 7f7b13655b80 1 -- 172.21.15.7:0/2153321380 --> [v2:172.21.15.7:6833/1251886476,v1:172.21.15.7:6835/1251886476] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a7847fee0 con 0x556a785bad00 2024-01-27T02:38:02.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.569+0000 7f7b037fe700 1 --2- 172.21.15.7:0/2153321380 >> [v2:172.21.15.7:6833/1251886476,v1:172.21.15.7:6835/1251886476] conn(0x556a785bad00 0x556a785bb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.569+0000 7f7ae27fc700 10 client.6813 ms_handle_connect on v2:172.21.15.7:6833/1251886476 2024-01-27T02:38:02.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.569+0000 7f7b037fe700 1 --2- 172.21.15.7:0/2153321380 >> [v2:172.21.15.7:6833/1251886476,v1:172.21.15.7:6835/1251886476] conn(0x556a785bad00 0x556a785bb0d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b659861c8583e06 server_cookie=284417db7be4e48d in_seq=0 out_seq=0 2024-01-27T02:38:02.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.570+0000 7f7ae27fc700 1 -- 172.21.15.7:0/2153321380 <== mds.0 v2:172.21.15.7:6833/1251886476 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17146 (crc 0 0 0) 0x556a7847fee0 con 0x556a785bad00 2024-01-27T02:38:02.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.570+0000 7f7ae27fc700 10 client.6813 handle_command_reply: tid=0 2024-01-27T02:38:02.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.570+0000 7f7b13655b80 1 client.6813 shutdown 2024-01-27T02:38:02.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.570+0000 7f7b13655b80 1 -- 172.21.15.7:0/2153321380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a7858e5d0 msgr2=0x556a785b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:02.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.570+0000 7f7b13655b80 1 --2- 172.21.15.7:0/2153321380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a7858e5d0 0x556a785b4d70 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f7af4013000 tx=0x7f7af401e040 comp rx=0 tx=0).stop 2024-01-27T02:38:02.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.570+0000 7f7b13655b80 1 -- 172.21.15.7:0/2153321380 shutdown_connections 2024-01-27T02:38:02.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.570+0000 7f7b13655b80 1 --2- 172.21.15.7:0/2153321380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a785d7360 0x556a785e6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.570+0000 7f7b13655b80 1 --2- 172.21.15.7:0/2153321380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a7858e5d0 0x556a785b4d70 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.570+0000 7f7b13655b80 1 --2- 172.21.15.7:0/2153321380 >> [v2:172.21.15.7:6833/1251886476,v1:172.21.15.7:6835/1251886476] conn(0x556a785bad00 0x556a785bb0d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.570+0000 7f7b13655b80 1 --2- 172.21.15.7:0/2153321380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7858f010 0x556a785d6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.570+0000 7f7ae27fc700 0 client.6813 ms_handle_reset on v2:172.21.15.7:6833/1251886476 2024-01-27T02:38:02.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.570+0000 7f7b13655b80 1 -- 172.21.15.7:0/2153321380 >> 172.21.15.7:0/2153321380 conn(0x556a7858ea70 msgr2=0x556a78588240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:02.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.570+0000 7f7b13655b80 1 -- 172.21.15.7:0/2153321380 shutdown_connections 2024-01-27T02:38:02.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.570+0000 7f7b13655b80 1 -- 172.21.15.7:0/2153321380 wait complete. 2024-01-27T02:38:02.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.570+0000 7f7b13655b80 20 client.6813 trim_cache size 0 max 16384 2024-01-27T02:38:02.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.588+0000 7f7b13655b80 1 Processor -- start 2024-01-27T02:38:02.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.588+0000 7f7b13655b80 1 -- start start 2024-01-27T02:38:02.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.589+0000 7f7b13655b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a7858e5d0 0x556a7858a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.589+0000 7f7b13655b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7858f010 0x556a7858b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.589+0000 7f7b13655b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a785d7360 0x556a785d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.589+0000 7f7b13655b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a783d5e40 con 0x556a7858e5d0 2024-01-27T02:38:02.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.589+0000 7f7b13655b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a7858b910 con 0x556a7858f010 2024-01-27T02:38:02.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.589+0000 7f7b13655b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a785e25a0 con 0x556a785d7360 2024-01-27T02:38:02.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.589+0000 7f7b03fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a7858e5d0 0x556a7858a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.589+0000 7f7b03fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a7858e5d0 0x556a7858a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40172/0 (socket says 172.21.15.7:40172) 2024-01-27T02:38:02.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.590+0000 7f7b03fff700 1 -- 172.21.15.7:0/3892732949 learned_addr learned my addr 172.21.15.7:0/3892732949 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:02.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.590+0000 7f7b088ef700 1 --2- 172.21.15.7:0/3892732949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a785d7360 0x556a785d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.590+0000 7f7b037fe700 1 --2- 172.21.15.7:0/3892732949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7858f010 0x556a7858b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.590+0000 7f7b03fff700 1 -- 172.21.15.7:0/3892732949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a785d7360 msgr2=0x556a785d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:02.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.590+0000 7f7b03fff700 1 --2- 172.21.15.7:0/3892732949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a785d7360 0x556a785d0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.590+0000 7f7b03fff700 1 -- 172.21.15.7:0/3892732949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7858f010 msgr2=0x556a7858b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:02.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.590+0000 7f7b03fff700 1 --2- 172.21.15.7:0/3892732949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7858f010 0x556a7858b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.590+0000 7f7b03fff700 1 -- 172.21.15.7:0/3892732949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a78374c80 con 0x556a7858e5d0 2024-01-27T02:38:02.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.590+0000 7f7b037fe700 1 --2- 172.21.15.7:0/3892732949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7858f010 0x556a7858b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:02.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.590+0000 7f7b03fff700 1 --2- 172.21.15.7:0/3892732949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a7858e5d0 0x556a7858a9d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7af4003010 tx=0x7f7af402c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f55629b8053007 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:02.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.590+0000 7f7af9ffb700 1 -- 172.21.15.7:0/3892732949 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af402a710 con 0x556a7858e5d0 2024-01-27T02:38:02.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.590+0000 7f7af9ffb700 1 -- 172.21.15.7:0/3892732949 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af400bbf0 con 0x556a7858e5d0 2024-01-27T02:38:02.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.590+0000 7f7af9ffb700 1 -- 172.21.15.7:0/3892732949 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af402ab70 con 0x556a7858e5d0 2024-01-27T02:38:02.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.591+0000 7f7b13655b80 1 -- 172.21.15.7:0/3892732949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a7858e5d0 msgr2=0x556a7858a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:02.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.591+0000 7f7b13655b80 1 --2- 172.21.15.7:0/3892732949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a7858e5d0 0x556a7858a9d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7af4003010 tx=0x7f7af402c070 comp rx=0 tx=0).stop 2024-01-27T02:38:02.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.591+0000 7f7b13655b80 1 -- 172.21.15.7:0/3892732949 shutdown_connections 2024-01-27T02:38:02.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.591+0000 7f7b13655b80 1 --2- 172.21.15.7:0/3892732949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a785d7360 0x556a785d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.591+0000 7f7b13655b80 1 --2- 172.21.15.7:0/3892732949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a7858e5d0 0x556a7858a9d0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.591+0000 7f7b13655b80 1 --2- 172.21.15.7:0/3892732949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7858f010 0x556a7858b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.591+0000 7f7b13655b80 1 -- 172.21.15.7:0/3892732949 >> 172.21.15.7:0/3892732949 conn(0x556a7858ea70 msgr2=0x556a785b95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:02.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.591+0000 7f7b13655b80 1 -- 172.21.15.7:0/3892732949 shutdown_connections 2024-01-27T02:38:02.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.591+0000 7f7b13655b80 1 -- 172.21.15.7:0/3892732949 wait complete. 2024-01-27T02:38:02.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.592+0000 7f7b13655b80 1 -- start start 2024-01-27T02:38:02.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.592+0000 7f7b13655b80 1 Processor -- start 2024-01-27T02:38:02.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.593+0000 7f7b13655b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7858e5d0 0x556a785ed490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.593+0000 7f7b13655b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a7858f010 0x556a786390b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.593+0000 7f7b13655b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a785d7360 0x556a7863bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.593+0000 7f7b13655b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a785e85a0 con 0x556a785d7360 2024-01-27T02:38:02.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.593+0000 7f7b13655b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a786386e0 con 0x556a7858e5d0 2024-01-27T02:38:02.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.593+0000 7f7b13655b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a785d0070 con 0x556a7858f010 2024-01-27T02:38:02.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.593+0000 7f7b13655b80 10 client.0 _ll_register_callbacks cb 0x556a782eb7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-27T02:38:02.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.593+0000 7f7b088ef700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a785d7360 0x556a7863bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.593+0000 7f7b088ef700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a785d7360 0x556a7863bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40184/0 (socket says 172.21.15.7:40184) 2024-01-27T02:38:02.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.593+0000 7f7b088ef700 1 -- 172.21.15.7:0/1844966757 learned_addr learned my addr 172.21.15.7:0/1844966757 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:02.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.593+0000 7f7b03fff700 1 --2- 172.21.15.7:0/1844966757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7858e5d0 0x556a785ed490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.594+0000 7f7b088ef700 1 -- 172.21.15.7:0/1844966757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a7858f010 msgr2=0x556a786390b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-27T02:38:02.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.594+0000 7f7b088ef700 1 --2- 172.21.15.7:0/1844966757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a7858f010 0x556a786390b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.594+0000 7f7b088ef700 1 -- 172.21.15.7:0/1844966757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7858e5d0 msgr2=0x556a785ed490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:02.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.594+0000 7f7b088ef700 1 --2- 172.21.15.7:0/1844966757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7858e5d0 0x556a785ed490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.594+0000 7f7b088ef700 1 -- 172.21.15.7:0/1844966757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a784ab290 con 0x556a785d7360 2024-01-27T02:38:02.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.594+0000 7f7b088ef700 1 --2- 172.21.15.7:0/1844966757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a785d7360 0x556a7863bce0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f7afc007df0 tx=0x7f7afc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a52f382ce972bfe server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:02.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.594+0000 7f7ae27fc700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-27T02:38:02.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.594+0000 7f7ae27fc700 1 -- 172.21.15.7:0/1844966757 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0137a0 con 0x556a785d7360 2024-01-27T02:38:02.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.594+0000 7f7ae27fc700 1 -- 172.21.15.7:0/1844966757 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7afc01a030 con 0x556a785d7360 2024-01-27T02:38:02.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.594+0000 7f7b13655b80 1 -- 172.21.15.7:0/1844966757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7aec05bf70 con 0x556a785d7360 2024-01-27T02:38:02.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.594+0000 7f7ae27fc700 1 -- 172.21.15.7:0/1844966757 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7afc013b10 con 0x556a785d7360 2024-01-27T02:38:02.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.594+0000 7f7b13655b80 10 client.6819 fetch_fsmap learned FSMap version 58 2024-01-27T02:38:02.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.594+0000 7f7b13655b80 1 -- 172.21.15.7:0/1844966757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x556a7831afe0 con 0x556a785d7360 2024-01-27T02:38:02.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.594+0000 7f7ae27fc700 1 -- 172.21.15.7:0/1844966757 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec05bf70 con 0x556a785d7360 2024-01-27T02:38:02.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.594+0000 7f7ae27fc700 1 -- 172.21.15.7:0/1844966757 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 58) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f7afc013c70 con 0x556a785d7360 2024-01-27T02:38:02.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.594+0000 7f7b13655b80 10 client.6819 fetch_fsmap finished waiting for FSMap version 58 2024-01-27T02:38:02.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.594+0000 7f7b13655b80 10 client.6819 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-27T02:38:02.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.594+0000 7f7b13655b80 20 client.6819 populate_metadata read hostname 'smithi007' 2024-01-27T02:38:02.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.594+0000 7f7b13655b80 1 --2- 172.21.15.7:0/1844966757 >> [v2:172.21.15.7:6833/1251886476,v1:172.21.15.7:6835/1251886476] conn(0x556a785d0d30 0x556a785e5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.595+0000 7f7b13655b80 4 client.6819 mds_command: new command op to 6657 tid=0[{"prefix": "status"}] 2024-01-27T02:38:02.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.595+0000 7f7b13655b80 1 -- 172.21.15.7:0/1844966757 --> [v2:172.21.15.7:6833/1251886476,v1:172.21.15.7:6835/1251886476] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556a7847fc00 con 0x556a785d0d30 2024-01-27T02:38:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.595+0000 7f7b03fff700 1 --2- 172.21.15.7:0/1844966757 >> [v2:172.21.15.7:6833/1251886476,v1:172.21.15.7:6835/1251886476] conn(0x556a785d0d30 0x556a785e5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.595+0000 7f7b03fff700 1 --2- 172.21.15.7:0/1844966757 >> [v2:172.21.15.7:6833/1251886476,v1:172.21.15.7:6835/1251886476] conn(0x556a785d0d30 0x556a785e5490 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=986a04b2cf0f5ca0 server_cookie=7eccb9846a1e381c in_seq=0 out_seq=0 2024-01-27T02:38:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.595+0000 7f7ae27fc700 10 client.6819 ms_handle_connect on v2:172.21.15.7:6833/1251886476 2024-01-27T02:38:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.595+0000 7f7ae27fc700 1 -- 172.21.15.7:0/1844966757 <== mds.0 v2:172.21.15.7:6833/1251886476 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x556a7847fc00 con 0x556a785d0d30 2024-01-27T02:38:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.595+0000 7f7ae27fc700 10 client.6819 handle_command_reply: tid=0 2024-01-27T02:38:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.595+0000 7f7b13655b80 1 client.6819 shutdown 2024-01-27T02:38:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.595+0000 7f7b13655b80 1 -- 172.21.15.7:0/1844966757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a785d7360 msgr2=0x556a7863bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.595+0000 7f7b13655b80 1 --2- 172.21.15.7:0/1844966757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a785d7360 0x556a7863bce0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f7afc007df0 tx=0x7f7afc00b040 comp rx=0 tx=0).stop 2024-01-27T02:38:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.595+0000 7f7b13655b80 1 -- 172.21.15.7:0/1844966757 shutdown_connections 2024-01-27T02:38:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.595+0000 7f7b13655b80 1 --2- 172.21.15.7:0/1844966757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a785d7360 0x556a7863bce0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.595+0000 7f7b13655b80 1 --2- 172.21.15.7:0/1844966757 >> [v2:172.21.15.7:6833/1251886476,v1:172.21.15.7:6835/1251886476] conn(0x556a785d0d30 0x556a785e5490 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.595+0000 7f7b13655b80 1 --2- 172.21.15.7:0/1844966757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a7858e5d0 0x556a785ed490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.595+0000 7f7b13655b80 1 --2- 172.21.15.7:0/1844966757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a7858f010 0x556a786390b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.595+0000 7f7ae27fc700 0 client.6819 ms_handle_reset on v2:172.21.15.7:6833/1251886476 2024-01-27T02:38:02.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.595+0000 7f7b13655b80 1 -- 172.21.15.7:0/1844966757 >> 172.21.15.7:0/1844966757 conn(0x556a7858ea70 msgr2=0x556a785cde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:02.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.595+0000 7f7b13655b80 1 -- 172.21.15.7:0/1844966757 shutdown_connections 2024-01-27T02:38:02.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.596+0000 7f7b13655b80 1 -- 172.21.15.7:0/1844966757 wait complete. 2024-01-27T02:38:02.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.596+0000 7f7b13655b80 20 client.6819 trim_cache size 0 max 16384 2024-01-27T02:38:02.604 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:38:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:38:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-27T02:38:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 6657, 2024-01-27T02:38:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-27T02:38:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-27T02:38:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-27T02:38:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 10.783971402000001, 2024-01-27T02:38:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 58, 2024-01-27T02:38:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 117, 2024-01-27T02:38:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 117, 2024-01-27T02:38:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 15.764253627 2024-01-27T02:38:02.605 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:38:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.596+0000 7f7b0a352700 1 -- 172.21.15.7:0/1283197242 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ae4066f30 msgr2=0x7f7ae40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.596+0000 7f7b0a352700 1 --2- 172.21.15.7:0/1283197242 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ae4066f30 0x7f7ae40693e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7b040b0d20 tx=0x7f7af4017000 comp rx=0 tx=0).stop 2024-01-27T02:38:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.596+0000 7f7b0a352700 1 -- 172.21.15.7:0/1283197242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b040d1470 msgr2=0x7f7b0415d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.596+0000 7f7b0a352700 1 --2- 172.21.15.7:0/1283197242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b040d1470 0x7f7b0415d570 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00c7e0 tx=0x7f7aec007df0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.596+0000 7f7b0a352700 1 -- 172.21.15.7:0/1283197242 shutdown_connections 2024-01-27T02:38:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.596+0000 7f7b0a352700 1 --2- 172.21.15.7:0/1283197242 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ae4066f30 0x7f7ae40693e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.596+0000 7f7b0a352700 1 --2- 172.21.15.7:0/1283197242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b04159b40 0x7f7b0415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.596+0000 7f7b0a352700 1 --2- 172.21.15.7:0/1283197242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b040d10a0 0x7f7b0416af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.596+0000 7f7b0a352700 1 --2- 172.21.15.7:0/1283197242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b040d1470 0x7f7b0415d570 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.596+0000 7f7b0a352700 1 -- 172.21.15.7:0/1283197242 >> 172.21.15.7:0/1283197242 conn(0x7f7b040679b0 msgr2=0x7f7b040cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:02.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.596+0000 7f7b0a352700 1 -- 172.21.15.7:0/1283197242 shutdown_connections 2024-01-27T02:38:02.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.596+0000 7f7b0a352700 1 -- 172.21.15.7:0/1283197242 wait complete. 2024-01-27T02:38:02.610 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:38:02.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.741+0000 7f8685b9f700 1 Processor -- start 2024-01-27T02:38:02.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.741+0000 7f8685b9f700 1 -- start start 2024-01-27T02:38:02.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.742+0000 7f8685b9f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f868015a100 0x7f86800cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.742+0000 7f8685b9f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86800ce5e0 con 0x7f86800cb5b0 2024-01-27T02:38:02.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.742+0000 7f8685b9f700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86800cd9d0 con 0x7f86800c4cc0 2024-01-27T02:38:02.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.742+0000 7f8685b9f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86800cdcb0 con 0x7f868015a100 2024-01-27T02:38:02.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.742+0000 7f867effd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f86800cb5b0 0x7f86800c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44738/0 (socket says 172.21.15.7:44738) 2024-01-27T02:38:02.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.743+0000 7f867effd700 1 -- 172.21.15.7:0/4248735720 learned_addr learned my addr 172.21.15.7:0/4248735720 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:02.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.743+0000 7f867f7fe700 1 --2- 172.21.15.7:0/4248735720 >> v2:172.21.15.188:3300/0 conn(0x7f868015a100 0x7f86800cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.743+0000 7f867e7fc700 1 -- 172.21.15.7:0/4248735720 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 133207744 0 0) 0x7f86800ce5e0 con 0x7f86800cb5b0 2024-01-27T02:38:02.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.743+0000 7f867e7fc700 1 -- 172.21.15.7:0/4248735720 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f866c003570 con 0x7f86800cb5b0 2024-01-27T02:38:02.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.743+0000 7f867e7fc700 1 -- 172.21.15.7:0/4248735720 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233435971 0 0) 0x7f86800cd9d0 con 0x7f86800c4cc0 2024-01-27T02:38:02.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.743+0000 7f867e7fc700 1 -- 172.21.15.7:0/4248735720 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86800ce5e0 con 0x7f86800c4cc0 2024-01-27T02:38:02.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.743+0000 7f867f7fe700 1 -- 172.21.15.7:0/4248735720 >> v1:172.21.15.188:6789/0 conn(0x7f86800c4cc0 legacy=0x7f8680163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:02.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.743+0000 7f867e7fc700 1 -- 172.21.15.7:0/4248735720 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 483634938 0 0) 0x7f866c003570 con 0x7f86800cb5b0 2024-01-27T02:38:02.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.743+0000 7f867f7fe700 1 -- 172.21.15.7:0/4248735720 >> v1:172.21.15.7:6789/0 conn(0x7f86800cb5b0 legacy=0x7f86800c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:02.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.743+0000 7f867f7fe700 1 -- 172.21.15.7:0/4248735720 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86800cdf90 con 0x7f868015a100 2024-01-27T02:38:02.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.743+0000 7f867f7fe700 1 --2- 172.21.15.7:0/4248735720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f868015a100 0x7f86800cd2f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f86700098e0 tx=0x7f86700091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaf1e83233c27795 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:02.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.743+0000 7f867e7fc700 1 -- 172.21.15.7:0/4248735720 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8668001470 con 0x7f868015a100 2024-01-27T02:38:02.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.744+0000 7f867e7fc700 1 -- 172.21.15.7:0/4248735720 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8670011420 con 0x7f868015a100 2024-01-27T02:38:02.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.744+0000 7f867e7fc700 1 -- 172.21.15.7:0/4248735720 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86700117c0 con 0x7f868015a100 2024-01-27T02:38:02.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.744+0000 7f8685b9f700 1 -- 172.21.15.7:0/4248735720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f868015a100 msgr2=0x7f86800cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:02.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.744+0000 7f8685b9f700 1 --2- 172.21.15.7:0/4248735720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f868015a100 0x7f86800cd2f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f86700098e0 tx=0x7f86700091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.744+0000 7f8685b9f700 1 -- 172.21.15.7:0/4248735720 shutdown_connections 2024-01-27T02:38:02.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.744+0000 7f8685b9f700 1 --2- 172.21.15.7:0/4248735720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f868015a100 0x7f86800cd2f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.744+0000 7f8685b9f700 1 -- 172.21.15.7:0/4248735720 >> 172.21.15.7:0/4248735720 conn(0x7f8680098020 msgr2=0x7f86800711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:02.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.744+0000 7f8685b9f700 1 -- 172.21.15.7:0/4248735720 shutdown_connections 2024-01-27T02:38:02.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.744+0000 7f8685b9f700 1 -- 172.21.15.7:0/4248735720 wait complete. 2024-01-27T02:38:02.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.746+0000 7f8685b9f700 1 Processor -- start 2024-01-27T02:38:02.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.746+0000 7f8685b9f700 1 -- start start 2024-01-27T02:38:02.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.746+0000 7f8685b9f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86800c4cc0 0x7f868015ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.746+0000 7f8685b9f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86800cb5b0 0x7f8680154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.746+0000 7f8685b9f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f868015a100 0x7f8680068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.746+0000 7f8685b9f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8680154b40 con 0x7f868015a100 2024-01-27T02:38:02.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.746+0000 7f8685b9f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86800b1f00 con 0x7f86800cb5b0 2024-01-27T02:38:02.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.746+0000 7f8685b9f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86800b21e0 con 0x7f86800c4cc0 2024-01-27T02:38:02.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.747+0000 7f867ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f868015a100 0x7f8680068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.747+0000 7f867ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f868015a100 0x7f8680068990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40186/0 (socket says 172.21.15.7:40186) 2024-01-27T02:38:02.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.747+0000 7f867ffff700 1 -- 172.21.15.7:0/2811986108 learned_addr learned my addr 172.21.15.7:0/2811986108 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:02.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.747+0000 7f867f7fe700 1 --2- 172.21.15.7:0/2811986108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86800c4cc0 0x7f868015ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.747+0000 7f867effd700 1 --2- 172.21.15.7:0/2811986108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86800cb5b0 0x7f8680154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.747+0000 7f867ffff700 1 -- 172.21.15.7:0/2811986108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86800c4cc0 msgr2=0x7f868015ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:02.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.747+0000 7f867ffff700 1 --2- 172.21.15.7:0/2811986108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86800c4cc0 0x7f868015ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.747+0000 7f867ffff700 1 -- 172.21.15.7:0/2811986108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86800cb5b0 msgr2=0x7f8680154600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:02.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.747+0000 7f867ffff700 1 --2- 172.21.15.7:0/2811986108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86800cb5b0 0x7f8680154600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:02.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.747+0000 7f867ffff700 1 -- 172.21.15.7:0/2811986108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8670009420 con 0x7f868015a100 2024-01-27T02:38:02.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.747+0000 7f867ffff700 1 --2- 172.21.15.7:0/2811986108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f868015a100 0x7f8680068990 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f867800c910 tx=0x7f8678002720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:02.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.747+0000 7f86777fe700 1 -- 172.21.15.7:0/2811986108 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867800ce50 con 0x7f868015a100 2024-01-27T02:38:02.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.747+0000 7f86777fe700 1 -- 172.21.15.7:0/2811986108 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867800a870 con 0x7f868015a100 2024-01-27T02:38:02.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.747+0000 7f8685b9f700 1 -- 172.21.15.7:0/2811986108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86800b24c0 con 0x7f868015a100 2024-01-27T02:38:02.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.747+0000 7f86777fe700 1 -- 172.21.15.7:0/2811986108 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867800ac30 con 0x7f868015a100 2024-01-27T02:38:02.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.747+0000 7f8685b9f700 1 -- 172.21.15.7:0/2811986108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86800c6590 con 0x7f868015a100 2024-01-27T02:38:02.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.748+0000 7f8685b9f700 1 -- 172.21.15.7:0/2811986108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f868006d9f0 con 0x7f868015a100 2024-01-27T02:38:02.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.749+0000 7f86777fe700 1 -- 172.21.15.7:0/2811986108 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8678012630 con 0x7f868015a100 2024-01-27T02:38:02.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.749+0000 7f86777fe700 1 --2- 172.21.15.7:0/2811986108 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8660066f30 0x7f86600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:02.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.750+0000 7f86777fe700 1 -- 172.21.15.7:0/2811986108 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8678084a00 con 0x7f868015a100 2024-01-27T02:38:02.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.750+0000 7f867f7fe700 1 --2- 172.21.15.7:0/2811986108 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8660066f30 0x7f86600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:02.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.750+0000 7f867f7fe700 1 --2- 172.21.15.7:0/2811986108 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8660066f30 0x7f86600693e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f86700093f0 tx=0x7f8670016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:02.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:02.754+0000 7f86777fe700 1 -- 172.21.15.7:0/2811986108 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8678051c70 con 0x7f868015a100 2024-01-27T02:38:03.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.008+0000 7f8685b9f700 1 -- 172.21.15.7:0/2811986108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f86800c67b0 con 0x7f868015a100 2024-01-27T02:38:03.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.008+0000 7f86777fe700 1 -- 172.21.15.7:0/2811986108 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v58) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8678055290 con 0x7f868015a100 2024-01-27T02:38:03.009 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:38:03.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.010+0000 7f8685b9f700 1 -- 172.21.15.7:0/2811986108 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8660066f30 msgr2=0x7f86600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:03.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.010+0000 7f8685b9f700 1 --2- 172.21.15.7:0/2811986108 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8660066f30 0x7f86600693e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f86700093f0 tx=0x7f8670016000 comp rx=0 tx=0).stop 2024-01-27T02:38:03.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.010+0000 7f8685b9f700 1 -- 172.21.15.7:0/2811986108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f868015a100 msgr2=0x7f8680068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:03.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.010+0000 7f8685b9f700 1 --2- 172.21.15.7:0/2811986108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f868015a100 0x7f8680068990 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f867800c910 tx=0x7f8678002720 comp rx=0 tx=0).stop 2024-01-27T02:38:03.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.010+0000 7f8685b9f700 1 -- 172.21.15.7:0/2811986108 shutdown_connections 2024-01-27T02:38:03.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.010+0000 7f8685b9f700 1 --2- 172.21.15.7:0/2811986108 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8660066f30 0x7f86600693e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:03.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.010+0000 7f8685b9f700 1 --2- 172.21.15.7:0/2811986108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f868015a100 0x7f8680068990 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:03.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.010+0000 7f8685b9f700 1 --2- 172.21.15.7:0/2811986108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86800c4cc0 0x7f868015ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:03.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.010+0000 7f8685b9f700 1 --2- 172.21.15.7:0/2811986108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86800cb5b0 0x7f8680154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:03.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.010+0000 7f8685b9f700 1 -- 172.21.15.7:0/2811986108 >> 172.21.15.7:0/2811986108 conn(0x7f8680098020 msgr2=0x7f8680154f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:03.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.010+0000 7f8685b9f700 1 -- 172.21.15.7:0/2811986108 shutdown_connections 2024-01-27T02:38:03.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.011+0000 7f8685b9f700 1 -- 172.21.15.7:0/2811986108 wait complete. 2024-01-27T02:38:03.024 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-27T02:38:03.025 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:38:03.025 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-27T02:38:03.025 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-27T02:38:03.025 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-27T02:38:03.025 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-27T02:38:03.025 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-27T02:38:03.025 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:38:03.025 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:38:03.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:03.025 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-27T02:38:03.045 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:38:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:38:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:38:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-27T02:38:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:03.045 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:38:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-27T02:38:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-27T02:38:03.045 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:38:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-27T02:38:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-27T02:38:03.046 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:38:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-27T02:38:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-27T02:38:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-27T02:38:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42028sec preferred_lft 42028sec 2024-01-27T02:38:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-27T02:38:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:03.046 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:38:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-27T02:38:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-27T02:38:03.046 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:38:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 4a:57:d9:3c:c5:a0 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:38:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-27T02:38:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::2477:9eff:fe39:433c/64 scope link 2024-01-27T02:38:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:03.047 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:38:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 4a:57:d9:3c:c5:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-27T02:38:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::4857:d9ff:fe3c:c5a0/64 scope link 2024-01-27T02:38:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:03.047 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:38:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ca:6e:b8:df:53:bf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-27T02:38:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c86e:b8ff:fedf:53bf/64 scope link 2024-01-27T02:38:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:03.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:03.047 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:38:03.116 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-27T02:38:03.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:03.118 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-27T02:38:03.182 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-27T02:38:03.182 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:38:03.182 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-27T02:38:03.182 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-27T02:38:03.183 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:38:03.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:03.183 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-27T02:38:03.183 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-27T02:38:03.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-27T02:38:03.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-27T02:38:03.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-27T02:38:03.183 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-27T02:38:03.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:03.307 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-27T02:38:03.328 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-27T02:38:03.328 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-27T02:38:03.328 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:38:03.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:03.329 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-27T02:38:03.404 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:38:03.404 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-27T02:38:03.467 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:38:03.467 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:38:03.467 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-27T02:38:03.529 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:38:03.529 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:38:03.529 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:38:03.624 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:38:03.624 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:38:03.624 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:38:03.634 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:38:03.634 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:38:03.634 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:38:03.634 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:38:03.634 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:38:03.652 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:38:03.652 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:38:03.652 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:38:03.652 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:38:03.652 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:38:03.652 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:38:03.653 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:38:03.653 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:38:03.655 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:38:03.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:03.655 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-27T02:38:03.688 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:38:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.866+0000 7f94eae28700 1 Processor -- start 2024-01-27T02:38:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.866+0000 7f94eae28700 1 -- start start 2024-01-27T02:38:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.867+0000 7f94eae28700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94e40c3ae0 0x7f94e40cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.867+0000 7f94eae28700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e40ca020 con 0x7f94e4158070 2024-01-27T02:38:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.867+0000 7f94eae28700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e40c5210 con 0x7f94e4157ca0 2024-01-27T02:38:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.867+0000 7f94eae28700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40c54f0 con 0x7f94e40c3ae0 2024-01-27T02:38:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.867+0000 7f94e93c5700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f94e4158070 0x7f94e40ca4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44748/0 (socket says 172.21.15.7:44748) 2024-01-27T02:38:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.867+0000 7f94e93c5700 1 -- 172.21.15.7:0/505161160 learned_addr learned my addr 172.21.15.7:0/505161160 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.868+0000 7f94e3fff700 1 --2- 172.21.15.7:0/505161160 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94e40c3ae0 0x7f94e40cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.868+0000 7f94e37fe700 1 -- 172.21.15.7:0/505161160 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2566140753 0 0) 0x7f94e40ca020 con 0x7f94e4158070 2024-01-27T02:38:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.868+0000 7f94e37fe700 1 -- 172.21.15.7:0/505161160 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94d0003570 con 0x7f94e4158070 2024-01-27T02:38:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.868+0000 7f94e37fe700 1 -- 172.21.15.7:0/505161160 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206204591 0 0) 0x7f94e40c5210 con 0x7f94e4157ca0 2024-01-27T02:38:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.868+0000 7f94e37fe700 1 -- 172.21.15.7:0/505161160 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94e40ca020 con 0x7f94e4157ca0 2024-01-27T02:38:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.868+0000 7f94e3fff700 1 -- 172.21.15.7:0/505161160 >> v1:172.21.15.188:6789/0 conn(0x7f94e4157ca0 legacy=0x7f94e40c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.868+0000 7f94e3fff700 1 -- 172.21.15.7:0/505161160 >> v1:172.21.15.7:6789/0 conn(0x7f94e4158070 legacy=0x7f94e40ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.868+0000 7f94e37fe700 1 -- 172.21.15.7:0/505161160 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2215616326 0 0) 0x7f94d0003570 con 0x7f94e4158070 2024-01-27T02:38:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.868+0000 7f94e3fff700 1 -- 172.21.15.7:0/505161160 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e40c57d0 con 0x7f94e40c3ae0 2024-01-27T02:38:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.868+0000 7f94e3fff700 1 --2- 172.21.15.7:0/505161160 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94e40c3ae0 0x7f94e40cd4e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009910 tx=0x7f94cc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71f54da177a3d252 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.868+0000 7f94e37fe700 1 -- 172.21.15.7:0/505161160 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc0017f0 con 0x7f94e40c3ae0 2024-01-27T02:38:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.868+0000 7f94e37fe700 1 -- 172.21.15.7:0/505161160 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94cc011420 con 0x7f94e40c3ae0 2024-01-27T02:38:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.868+0000 7f94e37fe700 1 -- 172.21.15.7:0/505161160 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0117a0 con 0x7f94e40c3ae0 2024-01-27T02:38:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.869+0000 7f94eae28700 1 -- 172.21.15.7:0/505161160 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94e40c3ae0 msgr2=0x7f94e40cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.869+0000 7f94eae28700 1 --2- 172.21.15.7:0/505161160 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94e40c3ae0 0x7f94e40cd4e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009910 tx=0x7f94cc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.869+0000 7f94eae28700 1 -- 172.21.15.7:0/505161160 shutdown_connections 2024-01-27T02:38:03.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.869+0000 7f94eae28700 1 --2- 172.21.15.7:0/505161160 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94e40c3ae0 0x7f94e40cd4e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:03.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.869+0000 7f94eae28700 1 -- 172.21.15.7:0/505161160 >> 172.21.15.7:0/505161160 conn(0x7f94e4098020 msgr2=0x7f94e40711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:03.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.869+0000 7f94eae28700 1 -- 172.21.15.7:0/505161160 shutdown_connections 2024-01-27T02:38:03.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.869+0000 7f94eae28700 1 -- 172.21.15.7:0/505161160 wait complete. 2024-01-27T02:38:03.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.870+0000 7f94eae28700 1 Processor -- start 2024-01-27T02:38:03.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.871+0000 7f94eae28700 1 -- start start 2024-01-27T02:38:03.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.871+0000 7f94eae28700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94e40c3ae0 0x7f94e4068890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:03.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.872+0000 7f94eae28700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94e4157ca0 0x7f94e40b2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:03.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.872+0000 7f94eae28700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94e4158070 0x7f94e414ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:03.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.872+0000 7f94eae28700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40b3870 con 0x7f94e4157ca0 2024-01-27T02:38:03.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.872+0000 7f94eae28700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40b29d0 con 0x7f94e4158070 2024-01-27T02:38:03.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.872+0000 7f94eae28700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40b2c60 con 0x7f94e40c3ae0 2024-01-27T02:38:03.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.872+0000 7f94e3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94e4157ca0 0x7f94e40b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.872+0000 7f94e3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94e4157ca0 0x7f94e40b2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40192/0 (socket says 172.21.15.7:40192) 2024-01-27T02:38:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.872+0000 7f94e3fff700 1 -- 172.21.15.7:0/2661478588 learned_addr learned my addr 172.21.15.7:0/2661478588 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.872+0000 7f94e8bc4700 1 --2- 172.21.15.7:0/2661478588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94e40c3ae0 0x7f94e4068890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.872+0000 7f94e93c5700 1 --2- 172.21.15.7:0/2661478588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94e4158070 0x7f94e414ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.872+0000 7f94e3fff700 1 -- 172.21.15.7:0/2661478588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94e40c3ae0 msgr2=0x7f94e4068890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.872+0000 7f94e3fff700 1 --2- 172.21.15.7:0/2661478588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94e40c3ae0 0x7f94e4068890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.872+0000 7f94e3fff700 1 -- 172.21.15.7:0/2661478588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94e4158070 msgr2=0x7f94e414ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.872+0000 7f94e3fff700 1 --2- 172.21.15.7:0/2661478588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94e4158070 0x7f94e414ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.872+0000 7f94e3fff700 1 -- 172.21.15.7:0/2661478588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94cc009420 con 0x7f94e4157ca0 2024-01-27T02:38:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.872+0000 7f94e3fff700 1 --2- 172.21.15.7:0/2661478588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94e4157ca0 0x7f94e40b2490 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009860 tx=0x7f94cc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.872+0000 7f94e0ff9700 1 -- 172.21.15.7:0/2661478588 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc01cc50 con 0x7f94e4157ca0 2024-01-27T02:38:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.872+0000 7f94e0ff9700 1 -- 172.21.15.7:0/2661478588 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94cc01cdd0 con 0x7f94e4157ca0 2024-01-27T02:38:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.872+0000 7f94eae28700 1 -- 172.21.15.7:0/2661478588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e40b2f40 con 0x7f94e4157ca0 2024-01-27T02:38:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.872+0000 7f94e0ff9700 1 -- 172.21.15.7:0/2661478588 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc01a6a0 con 0x7f94e4157ca0 2024-01-27T02:38:03.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.873+0000 7f94eae28700 1 -- 172.21.15.7:0/2661478588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e40b3500 con 0x7f94e4157ca0 2024-01-27T02:38:03.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.874+0000 7f94eae28700 1 -- 172.21.15.7:0/2661478588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e40b39e0 con 0x7f94e4157ca0 2024-01-27T02:38:03.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.874+0000 7f94e0ff9700 1 -- 172.21.15.7:0/2661478588 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f94cc01aad0 con 0x7f94e4157ca0 2024-01-27T02:38:03.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.874+0000 7f94e0ff9700 1 --2- 172.21.15.7:0/2661478588 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f94c4066f30 0x7f94c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:03.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.875+0000 7f94e0ff9700 1 -- 172.21.15.7:0/2661478588 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f94cc085e70 con 0x7f94e4157ca0 2024-01-27T02:38:03.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.875+0000 7f94e8bc4700 1 --2- 172.21.15.7:0/2661478588 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f94c4066f30 0x7f94c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:03.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.875+0000 7f94e8bc4700 1 --2- 172.21.15.7:0/2661478588 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f94c4066f30 0x7f94c40693e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f94d4003870 tx=0x7f94d4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:03.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:03.879+0000 7f94e0ff9700 1 -- 172.21.15.7:0/2661478588 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94cc0530e0 con 0x7f94e4157ca0 2024-01-27T02:38:04.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.133+0000 7f94eae28700 1 -- 172.21.15.7:0/2661478588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f94e404c710 con 0x7f94e4157ca0 2024-01-27T02:38:04.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.133+0000 7f94e0ff9700 1 -- 172.21.15.7:0/2661478588 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v58) v1 ==== 53+0+73 (secure 0 0 0) 0x7f94cc056700 con 0x7f94e4157ca0 2024-01-27T02:38:04.134 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:38:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.136+0000 7f94eae28700 1 -- 172.21.15.7:0/2661478588 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f94c4066f30 msgr2=0x7f94c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.136+0000 7f94eae28700 1 --2- 172.21.15.7:0/2661478588 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f94c4066f30 0x7f94c40693e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f94d4003870 tx=0x7f94d4009040 comp rx=0 tx=0).stop 2024-01-27T02:38:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.136+0000 7f94eae28700 1 -- 172.21.15.7:0/2661478588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94e4157ca0 msgr2=0x7f94e40b2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.136+0000 7f94eae28700 1 --2- 172.21.15.7:0/2661478588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94e4157ca0 0x7f94e40b2490 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009860 tx=0x7f94cc016000 comp rx=0 tx=0).stop 2024-01-27T02:38:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.136+0000 7f94eae28700 1 -- 172.21.15.7:0/2661478588 shutdown_connections 2024-01-27T02:38:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.136+0000 7f94eae28700 1 --2- 172.21.15.7:0/2661478588 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f94c4066f30 0x7f94c40693e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.136+0000 7f94eae28700 1 --2- 172.21.15.7:0/2661478588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94e4158070 0x7f94e414ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.136+0000 7f94eae28700 1 --2- 172.21.15.7:0/2661478588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94e40c3ae0 0x7f94e4068890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.136+0000 7f94eae28700 1 --2- 172.21.15.7:0/2661478588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94e4157ca0 0x7f94e40b2490 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.136+0000 7f94eae28700 1 -- 172.21.15.7:0/2661478588 >> 172.21.15.7:0/2661478588 conn(0x7f94e4098020 msgr2=0x7f94e406c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.136+0000 7f94eae28700 1 -- 172.21.15.7:0/2661478588 shutdown_connections 2024-01-27T02:38:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.136+0000 7f94eae28700 1 -- 172.21.15.7:0/2661478588 wait complete. 2024-01-27T02:38:04.150 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-27T02:38:04.150 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:38:04.150 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-01-27T02:38:04.150 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-27T02:38:04.150 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-27T02:38:04.150 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-27T02:38:04.150 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-27T02:38:04.150 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:38:04.150 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:38:04.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:04.151 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-01-27T02:38:04.172 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:38:04.172 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:38:04.172 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:38:04.172 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:04.172 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-01-27T02:38:04.172 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:04.172 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:38:04.172 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:38:04.172 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-01-27T02:38:04.172 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:38:04.172 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:38:04.172 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-01-27T02:38:04.173 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:38:04.173 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:38:04.173 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:38:04.173 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:38:04.173 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-27T02:38:04.173 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42034sec preferred_lft 42034sec 2024-01-27T02:38:04.173 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b1:d6f7:baf0:bc17/64 scope link noprefixroute 2024-01-27T02:38:04.173 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:04.173 INFO:teuthology.orchestra.run.smithi188.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:38:04.173 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b0:92:71:90:9d brd ff:ff:ff:ff:ff:ff 2024-01-27T02:38:04.173 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-27T02:38:04.173 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:04.173 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::10f8:32ff:feed:a752/64 scope link 2024-01-27T02:38:04.173 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:04.174 INFO:teuthology.orchestra.run.smithi188.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:38:04.174 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b0:92:71:90:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-27T02:38:04.174 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b8b0:92ff:fe71:909d/64 scope link 2024-01-27T02:38:04.174 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:04.174 INFO:teuthology.orchestra.run.smithi188.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:38:04.174 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ee:78:03:20:3a:a9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-27T02:38:04.174 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec78:3ff:fe20:3aa9/64 scope link 2024-01-27T02:38:04.174 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:04.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:04.174 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:38:04.242 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-01-27T02:38:04.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:04.244 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-01-27T02:38:04.306 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-01-27T02:38:04.306 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:38:04.306 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-27T02:38:04.306 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-27T02:38:04.306 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:38:04.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:04.307 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-27T02:38:04.307 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-27T02:38:04.307 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-27T02:38:04.307 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-27T02:38:04.307 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-27T02:38:04.307 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-27T02:38:04.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:04.417 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-01-27T02:38:04.434 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-27T02:38:04.434 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-27T02:38:04.434 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:38:04.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:04.434 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2024-01-27T02:38:04.501 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:38:04.502 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-27T02:38:04.559 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:38:04.559 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:38:04.559 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-27T02:38:04.615 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:38:04.615 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:38:04.615 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:38:04.697 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:38:04.697 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:38:04.697 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:38:04.705 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:38:04.705 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:38:04.705 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:38:04.706 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:38:04.706 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:38:04.723 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:38:04.723 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:38:04.723 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:38:04.723 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:38:04.723 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:38:04.723 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:38:04.723 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:38:04.723 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:38:04.725 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:38:04.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:04.726 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-27T02:38:04.793 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:38:04.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.934+0000 7fe1522a0700 1 Processor -- start 2024-01-27T02:38:04.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.934+0000 7fe1522a0700 1 -- start start 2024-01-27T02:38:04.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.935+0000 7fe1522a0700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe14c0c4cc0 0x7fe14c0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:04.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.935+0000 7fe1522a0700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c0ca540 0x7fe14c163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:04.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.935+0000 7fe1522a0700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe14c0ce5e0 con 0x7fe14c15a100 2024-01-27T02:38:04.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.935+0000 7fe1522a0700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe14c0cd9d0 con 0x7fe14c0c4cc0 2024-01-27T02:38:04.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.935+0000 7fe1522a0700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c0cdcb0 con 0x7fe14c0ca540 2024-01-27T02:38:04.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.935+0000 7fe15083d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c0ca540 0x7fe14c163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:04.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.935+0000 7fe14b7fe700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe14c0c4cc0 0x7fe14c0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:04.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.936+0000 7fe14bfff700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fe14c15a100 0x7fe14c0c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:34984/0 (socket says 172.21.15.7:34984) 2024-01-27T02:38:04.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.936+0000 7fe14bfff700 1 -- 172.21.15.7:0/2254272542 learned_addr learned my addr 172.21.15.7:0/2254272542 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:04.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.936+0000 7fe14affd700 1 -- 172.21.15.7:0/2254272542 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3707527091 0 0) 0x7fe14c0ce5e0 con 0x7fe14c15a100 2024-01-27T02:38:04.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.936+0000 7fe15083d700 1 -- 172.21.15.7:0/2254272542 >> v2:172.21.15.188:3300/0 conn(0x7fe14c0c4cc0 msgr2=0x7fe14c0cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:04.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.936+0000 7fe15083d700 1 --2- 172.21.15.7:0/2254272542 >> v2:172.21.15.188:3300/0 conn(0x7fe14c0c4cc0 0x7fe14c0cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:04.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.936+0000 7fe15083d700 1 -- 172.21.15.7:0/2254272542 >> v1:172.21.15.188:6789/0 conn(0x7fe14c15a100 legacy=0x7fe14c0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:04.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.936+0000 7fe15083d700 1 -- 172.21.15.7:0/2254272542 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14c0cdf90 con 0x7fe14c0ca540 2024-01-27T02:38:04.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.936+0000 7fe15083d700 1 --2- 172.21.15.7:0/2254272542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c0ca540 0x7fe14c163d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe144009e60 tx=0x7fe144009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d22681c873a13de server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:04.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.936+0000 7fe14affd700 1 -- 172.21.15.7:0/2254272542 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14400e070 con 0x7fe14c0ca540 2024-01-27T02:38:04.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.936+0000 7fe14affd700 1 -- 172.21.15.7:0/2254272542 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14400bb60 con 0x7fe14c0ca540 2024-01-27T02:38:04.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.936+0000 7fe14affd700 1 -- 172.21.15.7:0/2254272542 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe144012430 con 0x7fe14c0ca540 2024-01-27T02:38:04.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.937+0000 7fe1522a0700 1 -- 172.21.15.7:0/2254272542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c0ca540 msgr2=0x7fe14c163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:04.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.937+0000 7fe1522a0700 1 --2- 172.21.15.7:0/2254272542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c0ca540 0x7fe14c163d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe144009e60 tx=0x7fe144009780 comp rx=0 tx=0).stop 2024-01-27T02:38:04.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.937+0000 7fe1522a0700 1 -- 172.21.15.7:0/2254272542 shutdown_connections 2024-01-27T02:38:04.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.937+0000 7fe1522a0700 1 --2- 172.21.15.7:0/2254272542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c0ca540 0x7fe14c163d90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:04.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.937+0000 7fe1522a0700 1 --2- 172.21.15.7:0/2254272542 >> v2:172.21.15.188:3300/0 conn(0x7fe14c0c4cc0 0x7fe14c0cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:04.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.937+0000 7fe1522a0700 1 -- 172.21.15.7:0/2254272542 >> 172.21.15.7:0/2254272542 conn(0x7fe14c098020 msgr2=0x7fe14c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:04.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.937+0000 7fe1522a0700 1 -- 172.21.15.7:0/2254272542 shutdown_connections 2024-01-27T02:38:04.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.937+0000 7fe1522a0700 1 -- 172.21.15.7:0/2254272542 wait complete. 2024-01-27T02:38:04.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.938+0000 7fe1522a0700 1 Processor -- start 2024-01-27T02:38:04.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.938+0000 7fe1522a0700 1 -- start start 2024-01-27T02:38:04.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.939+0000 7fe1522a0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe14c0c4cc0 0x7fe14c0b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:04.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.939+0000 7fe1522a0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe14c0ca540 0x7fe14c0b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:04.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.939+0000 7fe1522a0700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c15a100 0x7fe14c0b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:04.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.939+0000 7fe1522a0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c0b3440 con 0x7fe14c0ca540 2024-01-27T02:38:04.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.939+0000 7fe1522a0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c15a4d0 con 0x7fe14c0c4cc0 2024-01-27T02:38:04.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.939+0000 7fe1522a0700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c15a7b0 con 0x7fe14c15a100 2024-01-27T02:38:04.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.939+0000 7fe14b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe14c0ca540 0x7fe14c0b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:04.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.939+0000 7fe14bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe14c0c4cc0 0x7fe14c0b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:04.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.939+0000 7fe14bfff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe14c0c4cc0 0x7fe14c0b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:40956/0 (socket says 172.21.15.7:40956) 2024-01-27T02:38:04.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.939+0000 7fe15083d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c15a100 0x7fe14c0b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:04.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.939+0000 7fe14bfff700 1 -- 172.21.15.7:0/2462112789 learned_addr learned my addr 172.21.15.7:0/2462112789 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:04.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.939+0000 7fe14b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe14c0ca540 0x7fe14c0b2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40204/0 (socket says 172.21.15.7:40204) 2024-01-27T02:38:04.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.940+0000 7fe14bfff700 1 -- 172.21.15.7:0/2462112789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c15a100 msgr2=0x7fe14c0b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.940+0000 7fe14bfff700 1 --2- 172.21.15.7:0/2462112789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c15a100 0x7fe14c0b3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.940+0000 7fe14bfff700 1 -- 172.21.15.7:0/2462112789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe14c0ca540 msgr2=0x7fe14c0b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.940+0000 7fe14bfff700 1 --2- 172.21.15.7:0/2462112789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe14c0ca540 0x7fe14c0b2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.940+0000 7fe14bfff700 1 -- 172.21.15.7:0/2462112789 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe144009a20 con 0x7fe14c0c4cc0 2024-01-27T02:38:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.940+0000 7fe15083d700 1 --2- 172.21.15.7:0/2462112789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c15a100 0x7fe14c0b3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:38:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.940+0000 7fe14bfff700 1 --2- 172.21.15.7:0/2462112789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe14c0c4cc0 0x7fe14c0b26d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00c7e0 tx=0x7fe13c007df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.940+0000 7fe133fff700 1 -- 172.21.15.7:0/2462112789 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c005b30 con 0x7fe14c0c4cc0 2024-01-27T02:38:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.940+0000 7fe133fff700 1 -- 172.21.15.7:0/2462112789 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c005c90 con 0x7fe14c0c4cc0 2024-01-27T02:38:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.940+0000 7fe1522a0700 1 -- 172.21.15.7:0/2462112789 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe14c15aa90 con 0x7fe14c0c4cc0 2024-01-27T02:38:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.940+0000 7fe133fff700 1 -- 172.21.15.7:0/2462112789 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c007300 con 0x7fe14c0c4cc0 2024-01-27T02:38:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.940+0000 7fe1522a0700 1 -- 172.21.15.7:0/2462112789 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe14c0b3150 con 0x7fe14c0c4cc0 2024-01-27T02:38:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.941+0000 7fe1522a0700 1 -- 172.21.15.7:0/2462112789 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe14c15af90 con 0x7fe14c0c4cc0 2024-01-27T02:38:04.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.942+0000 7fe133fff700 1 -- 172.21.15.7:0/2462112789 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe13c01b3b0 con 0x7fe14c0c4cc0 2024-01-27T02:38:04.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.942+0000 7fe133fff700 1 --2- 172.21.15.7:0/2462112789 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe138066f30 0x7fe1380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:04.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.942+0000 7fe133fff700 1 -- 172.21.15.7:0/2462112789 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe13c083af0 con 0x7fe14c0c4cc0 2024-01-27T02:38:04.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.942+0000 7fe14b7fe700 1 --2- 172.21.15.7:0/2462112789 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe138066f30 0x7fe1380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:04.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.943+0000 7fe14b7fe700 1 --2- 172.21.15.7:0/2462112789 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe138066f30 0x7fe1380693e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0b18a0 tx=0x7fe1400091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:04.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:04.947+0000 7fe133fff700 1 -- 172.21.15.7:0/2462112789 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe13c050cc0 con 0x7fe14c0c4cc0 2024-01-27T02:38:05.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.198+0000 7fe1522a0700 1 -- 172.21.15.7:0/2462112789 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe14c04c710 con 0x7fe14c0c4cc0 2024-01-27T02:38:05.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.199+0000 7fe133fff700 1 -- 172.21.15.7:0/2462112789 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fe13c017020 con 0x7fe14c0c4cc0 2024-01-27T02:38:05.199 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:05.200 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2767759975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2767759975},{"type":"v1","addr":"172.21.15.188:6836","nonce":2767759975}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":5768,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/245473575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":245473575},{"type":"v1","addr":"172.21.15.7:6834","nonce":245473575}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1334164270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1334164270},{"type":"v1","addr":"172.21.15.188:6837","nonce":1334164270}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:37:51.231918+0000","modified":"2024-01-27T02:37:54.851112+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6657},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6657":{"gid":6657,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1251886476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1251886476},{"type":"v1","addr":"172.21.15.7:6835","nonce":1251886476}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2024-01-27T02:38:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.201+0000 7fe1522a0700 1 -- 172.21.15.7:0/2462112789 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe138066f30 msgr2=0x7fe1380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.201+0000 7fe1522a0700 1 --2- 172.21.15.7:0/2462112789 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe138066f30 0x7fe1380693e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0b18a0 tx=0x7fe1400091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.201+0000 7fe1522a0700 1 -- 172.21.15.7:0/2462112789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe14c0c4cc0 msgr2=0x7fe14c0b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.201+0000 7fe1522a0700 1 --2- 172.21.15.7:0/2462112789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe14c0c4cc0 0x7fe14c0b26d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00c7e0 tx=0x7fe13c007df0 comp rx=0 tx=0).stop 2024-01-27T02:38:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.201+0000 7fe1522a0700 1 -- 172.21.15.7:0/2462112789 shutdown_connections 2024-01-27T02:38:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.201+0000 7fe1522a0700 1 --2- 172.21.15.7:0/2462112789 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe138066f30 0x7fe1380693e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.201+0000 7fe1522a0700 1 --2- 172.21.15.7:0/2462112789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c15a100 0x7fe14c0b3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.201+0000 7fe1522a0700 1 --2- 172.21.15.7:0/2462112789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe14c0c4cc0 0x7fe14c0b26d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:05.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.201+0000 7fe1522a0700 1 --2- 172.21.15.7:0/2462112789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe14c0ca540 0x7fe14c0b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:05.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.201+0000 7fe1522a0700 1 -- 172.21.15.7:0/2462112789 >> 172.21.15.7:0/2462112789 conn(0x7fe14c098020 msgr2=0x7fe14c0c9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:05.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.201+0000 7fe1522a0700 1 -- 172.21.15.7:0/2462112789 shutdown_connections 2024-01-27T02:38:05.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.201+0000 7fe1522a0700 1 -- 172.21.15.7:0/2462112789 wait complete. 2024-01-27T02:38:05.203 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 58 2024-01-27T02:38:05.216 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-01-27T02:38:05.367 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","whoami":0,"id":6657,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":13.544347892999999,"mdsmap_epoch":58,"osdmap_epoch":117,"osdmap_epoch_barrier":117,"uptime":18.524641995} 2024-01-27T02:38:05.368 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", "fs_name": "cephfs", "id": 6657, "mdsmap_epoch": 58, "osdmap_epoch": 117, "osdmap_epoch_barrier": 117, "rank_uptime": 13.544347893, "state": "up:active", "uptime": 18.524641995, "want_state": "up:active", "whoami": 0 } 2024-01-27T02:38:05.368 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-01-27T02:38:05.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.503+0000 7f9d5e494700 1 Processor -- start 2024-01-27T02:38:05.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.503+0000 7f9d5e494700 1 -- start start 2024-01-27T02:38:05.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.505+0000 7f9d5e494700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9d581406f0 0x7f9d580b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:05.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.505+0000 7f9d5e494700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d5806bef0 0x7f9d5806c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:05.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.505+0000 7f9d5e494700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9d5806e330 0x7f9d5806e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:05.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.505+0000 7f9d5e494700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d5806ecf0 con 0x7f9d581406f0 2024-01-27T02:38:05.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.505+0000 7f9d5e494700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d580c30c0 con 0x7f9d5806e330 2024-01-27T02:38:05.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.505+0000 7f9d5e494700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5806ccf0 con 0x7f9d5806bef0 2024-01-27T02:38:05.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.505+0000 7f9d57fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9d581406f0 0x7f9d580b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:05.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.505+0000 7f9d57fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9d581406f0 0x7f9d580b09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40214/0 (socket says 172.21.15.7:40214) 2024-01-27T02:38:05.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.505+0000 7f9d57fff700 1 -- 172.21.15.7:0/287661914 learned_addr learned my addr 172.21.15.7:0/287661914 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:05.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.505+0000 7f9d577fe700 1 --2- 172.21.15.7:0/287661914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d5806bef0 0x7f9d5806c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:05.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.505+0000 7f9d5ca31700 1 --2- 172.21.15.7:0/287661914 >> v2:172.21.15.188:3300/0 conn(0x7f9d5806e330 0x7f9d5806e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:05.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.505+0000 7f9d57fff700 1 -- 172.21.15.7:0/287661914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d5806bef0 msgr2=0x7f9d5806c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:05.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.505+0000 7f9d57fff700 1 --2- 172.21.15.7:0/287661914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d5806bef0 0x7f9d5806c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:05.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.505+0000 7f9d57fff700 1 -- 172.21.15.7:0/287661914 >> v2:172.21.15.188:3300/0 conn(0x7f9d5806e330 msgr2=0x7f9d5806e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:05.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.505+0000 7f9d57fff700 1 --2- 172.21.15.7:0/287661914 >> v2:172.21.15.188:3300/0 conn(0x7f9d5806e330 0x7f9d5806e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:05.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.505+0000 7f9d57fff700 1 -- 172.21.15.7:0/287661914 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5806ca10 con 0x7f9d581406f0 2024-01-27T02:38:05.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.505+0000 7f9d5ca31700 1 --2- 172.21.15.7:0/287661914 >> v2:172.21.15.188:3300/0 conn(0x7f9d5806e330 0x7f9d5806e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:05.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.505+0000 7f9d57fff700 1 --2- 172.21.15.7:0/287661914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d581406f0 0x7f9d580b09f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f9d5806e100 tx=0x7f9d4800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b38ec45ff8e7d5 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:05.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.505+0000 7f9d56ffd700 1 -- 172.21.15.7:0/287661914 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4800dba0 con 0x7f9d581406f0 2024-01-27T02:38:05.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.506+0000 7f9d56ffd700 1 -- 172.21.15.7:0/287661914 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4800dd00 con 0x7f9d581406f0 2024-01-27T02:38:05.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.506+0000 7f9d56ffd700 1 -- 172.21.15.7:0/287661914 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d480196d0 con 0x7f9d581406f0 2024-01-27T02:38:05.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.506+0000 7f9d5e494700 1 -- 172.21.15.7:0/287661914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d581406f0 msgr2=0x7f9d580b09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:05.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.506+0000 7f9d5e494700 1 --2- 172.21.15.7:0/287661914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d581406f0 0x7f9d580b09f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f9d5806e100 tx=0x7f9d4800b4f0 comp rx=0 tx=0).stop 2024-01-27T02:38:05.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.506+0000 7f9d5e494700 1 -- 172.21.15.7:0/287661914 shutdown_connections 2024-01-27T02:38:05.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.506+0000 7f9d5e494700 1 --2- 172.21.15.7:0/287661914 >> v2:172.21.15.188:3300/0 conn(0x7f9d5806e330 0x7f9d5806e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:05.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.506+0000 7f9d5e494700 1 --2- 172.21.15.7:0/287661914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d581406f0 0x7f9d580b09f0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:05.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.506+0000 7f9d5e494700 1 --2- 172.21.15.7:0/287661914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d5806bef0 0x7f9d5806c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:05.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.506+0000 7f9d5e494700 1 -- 172.21.15.7:0/287661914 >> 172.21.15.7:0/287661914 conn(0x7f9d580cdc30 msgr2=0x7f9d580ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:05.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.506+0000 7f9d5e494700 1 -- 172.21.15.7:0/287661914 shutdown_connections 2024-01-27T02:38:05.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.506+0000 7f9d5e494700 1 -- 172.21.15.7:0/287661914 wait complete. 2024-01-27T02:38:05.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.508+0000 7f9d5e494700 1 Processor -- start 2024-01-27T02:38:05.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.508+0000 7f9d5e494700 1 -- start start 2024-01-27T02:38:05.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.509+0000 7f9d5e494700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d5806bef0 0x7f9d580c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:05.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.509+0000 7f9d5e494700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d5806e330 0x7f9d58159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:05.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.509+0000 7f9d5e494700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d581406f0 0x7f9d58166850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:05.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.509+0000 7f9d5e494700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d581663c0 con 0x7f9d5806e330 2024-01-27T02:38:05.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.509+0000 7f9d5e494700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d58166090 con 0x7f9d5806bef0 2024-01-27T02:38:05.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.509+0000 7f9d5e494700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d58159a80 con 0x7f9d581406f0 2024-01-27T02:38:05.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.509+0000 7f9d577fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d5806e330 0x7f9d58159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:05.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.509+0000 7f9d577fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d5806e330 0x7f9d58159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40226/0 (socket says 172.21.15.7:40226) 2024-01-27T02:38:05.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.509+0000 7f9d577fe700 1 -- 172.21.15.7:0/1503237049 learned_addr learned my addr 172.21.15.7:0/1503237049 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:05.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.509+0000 7f9d57fff700 1 --2- 172.21.15.7:0/1503237049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d5806bef0 0x7f9d580c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:05.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.509+0000 7f9d5ca31700 1 --2- 172.21.15.7:0/1503237049 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d581406f0 0x7f9d58166850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:05.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.510+0000 7f9d577fe700 1 -- 172.21.15.7:0/1503237049 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d581406f0 msgr2=0x7f9d58166850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:05.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.510+0000 7f9d577fe700 1 --2- 172.21.15.7:0/1503237049 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d581406f0 0x7f9d58166850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:05.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.510+0000 7f9d577fe700 1 -- 172.21.15.7:0/1503237049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d5806bef0 msgr2=0x7f9d580c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:05.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.510+0000 7f9d577fe700 1 --2- 172.21.15.7:0/1503237049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d5806bef0 0x7f9d580c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:05.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.510+0000 7f9d577fe700 1 -- 172.21.15.7:0/1503237049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4800b790 con 0x7f9d5806e330 2024-01-27T02:38:05.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.510+0000 7f9d577fe700 1 --2- 172.21.15.7:0/1503237049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d5806e330 0x7f9d58159540 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c00e930 tx=0x7f9d4c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:05.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.510+0000 7f9d3ffff700 1 -- 172.21.15.7:0/1503237049 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c013680 con 0x7f9d5806e330 2024-01-27T02:38:05.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.510+0000 7f9d3ffff700 1 -- 172.21.15.7:0/1503237049 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4c0137e0 con 0x7f9d5806e330 2024-01-27T02:38:05.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.510+0000 7f9d5e494700 1 -- 172.21.15.7:0/1503237049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d58159d20 con 0x7f9d5806e330 2024-01-27T02:38:05.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.510+0000 7f9d3ffff700 1 -- 172.21.15.7:0/1503237049 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c013b80 con 0x7f9d5806e330 2024-01-27T02:38:05.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.510+0000 7f9d5e494700 1 -- 172.21.15.7:0/1503237049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d58166d90 con 0x7f9d5806e330 2024-01-27T02:38:05.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.511+0000 7f9d5e494700 1 -- 172.21.15.7:0/1503237049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d58166570 con 0x7f9d5806e330 2024-01-27T02:38:05.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.512+0000 7f9d3ffff700 1 -- 172.21.15.7:0/1503237049 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9d4c00c3e0 con 0x7f9d5806e330 2024-01-27T02:38:05.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.512+0000 7f9d3ffff700 1 --2- 172.21.15.7:0/1503237049 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9d44066f30 0x7f9d440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:05.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.512+0000 7f9d3ffff700 1 -- 172.21.15.7:0/1503237049 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9d4c0845e0 con 0x7f9d5806e330 2024-01-27T02:38:05.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.512+0000 7f9d57fff700 1 --2- 172.21.15.7:0/1503237049 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9d44066f30 0x7f9d440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:05.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.513+0000 7f9d57fff700 1 --2- 172.21.15.7:0/1503237049 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9d44066f30 0x7f9d440693e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9d5806e100 tx=0x7f9d4801b000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:05.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.517+0000 7f9d3ffff700 1 -- 172.21.15.7:0/1503237049 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d58166570 con 0x7f9d5806e330 2024-01-27T02:38:05.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.721+0000 7f9d5e494700 1 -- 172.21.15.7:0/1503237049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f9d5804c710 con 0x7f9d5806e330 2024-01-27T02:38:05.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.888+0000 7f9d3ffff700 1 -- 172.21.15.7:0/1503237049 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v118) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9d4c0552e0 con 0x7f9d5806e330 2024-01-27T02:38:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.891+0000 7f9d5e494700 1 -- 172.21.15.7:0/1503237049 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9d44066f30 msgr2=0x7f9d440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.891+0000 7f9d5e494700 1 --2- 172.21.15.7:0/1503237049 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9d44066f30 0x7f9d440693e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9d5806e100 tx=0x7f9d4801b000 comp rx=0 tx=0).stop 2024-01-27T02:38:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.891+0000 7f9d5e494700 1 -- 172.21.15.7:0/1503237049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d5806e330 msgr2=0x7f9d58159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.891+0000 7f9d5e494700 1 --2- 172.21.15.7:0/1503237049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d5806e330 0x7f9d58159540 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c00e930 tx=0x7f9d4c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.891+0000 7f9d5e494700 1 -- 172.21.15.7:0/1503237049 shutdown_connections 2024-01-27T02:38:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.891+0000 7f9d5e494700 1 --2- 172.21.15.7:0/1503237049 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9d44066f30 0x7f9d440693e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.891+0000 7f9d5e494700 1 --2- 172.21.15.7:0/1503237049 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d581406f0 0x7f9d58166850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.891+0000 7f9d5e494700 1 --2- 172.21.15.7:0/1503237049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d5806bef0 0x7f9d580c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.891+0000 7f9d5e494700 1 --2- 172.21.15.7:0/1503237049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d5806e330 0x7f9d58159540 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.891+0000 7f9d5e494700 1 -- 172.21.15.7:0/1503237049 >> 172.21.15.7:0/1503237049 conn(0x7f9d580cdc30 msgr2=0x7f9d580b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:05.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.891+0000 7f9d5e494700 1 -- 172.21.15.7:0/1503237049 shutdown_connections 2024-01-27T02:38:05.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:05.891+0000 7f9d5e494700 1 -- 172.21.15.7:0/1503237049 wait complete. 2024-01-27T02:38:05.893 INFO:teuthology.orchestra.run.smithi007.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-01-27T02:38:05.907 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-01-27T02:38:05.907 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-01-27T02:38:05.907 DEBUG:teuthology.orchestra.run.smithi007:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-01-27T02:38:05.907 DEBUG:teuthology.orchestra.run.smithi007:> os.fchmod(fd, 0O666) 2024-01-27T02:38:05.907 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(fd) 2024-01-27T02:38:05.907 DEBUG:teuthology.orchestra.run.smithi007:> os.close(fd) 2024-01-27T02:38:05.907 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-01-27T02:38:06.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:38:06.174 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-27T02:38:06.256 INFO:teuthology.orchestra.run.smithi007.stdout:client6828 2024-01-27T02:38:06.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:06.257 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/17e1bcca-e647-4a58-b0ac-7d39d8508814.client6828/osdmap') 2024-01-27T02:38:06.349 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 117 barrier 117 flags 0x588000 2024-01-27T02:38:06.350 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:38:06.350 INFO:teuthology.orchestra.run.smithi007.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:38:06.350 INFO:teuthology.orchestra.run.smithi007.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:38:06.350 INFO:teuthology.orchestra.run.smithi007.stdout:osd0 (1)172.21.15.7:6817 100% (exists, up) 100% -1 2024-01-27T02:38:06.350 INFO:teuthology.orchestra.run.smithi007.stdout:osd1 (1)172.21.15.7:6801 100% (exists, up) 100% -1 2024-01-27T02:38:06.350 INFO:teuthology.orchestra.run.smithi007.stdout:osd2 (1)172.21.15.7:6825 100% (exists, up) 100% -1 2024-01-27T02:38:06.351 INFO:teuthology.orchestra.run.smithi007.stdout:osd3 (1)172.21.15.7:6809 100% (exists, up) 100% -1 2024-01-27T02:38:06.351 INFO:teuthology.orchestra.run.smithi007.stdout:osd4 (1)172.21.15.188:6803 100% (exists, up) 100% -1 2024-01-27T02:38:06.351 INFO:teuthology.orchestra.run.smithi007.stdout:osd5 (1)172.21.15.188:6802 100% (exists, up) 100% -1 2024-01-27T02:38:06.351 INFO:teuthology.orchestra.run.smithi007.stdout:osd6 (1)172.21.15.188:6825 100% (exists, up) 100% -1 2024-01-27T02:38:06.351 INFO:teuthology.orchestra.run.smithi007.stdout:osd7 (1)172.21.15.188:6817 100% (exists, up) 100% -1 2024-01-27T02:38:06.352 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records in 2024-01-27T02:38:06.352 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records out 2024-01-27T02:38:06.352 INFO:teuthology.orchestra.run.smithi007.stderr:795 bytes copied, 4.3796e-05 s, 18.2 MB/s 2024-01-27T02:38:06.352 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2024-01-27T02:38:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.491+0000 7f350ea92700 1 Processor -- start 2024-01-27T02:38:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.491+0000 7f350ea92700 1 -- start start 2024-01-27T02:38:06.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.492+0000 7f350ea92700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f35081404e0 0x7f35080cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:06.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.492+0000 7f350ea92700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35080b09f0 0x7f35080cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:06.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.492+0000 7f350ea92700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35080cce30 con 0x7f35081408b0 2024-01-27T02:38:06.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.492+0000 7f350ea92700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f350806b3b0 con 0x7f35081404e0 2024-01-27T02:38:06.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.492+0000 7f350ea92700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35080cc3e0 con 0x7f35080b09f0 2024-01-27T02:38:06.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.492+0000 7f350c82e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f35081404e0 0x7f35080cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:06.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.492+0000 7f350c82e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f35081404e0 0x7f35080cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40240/0 (socket says 172.21.15.7:40240) 2024-01-27T02:38:06.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.492+0000 7f350c82e700 1 -- 172.21.15.7:0/3108302545 learned_addr learned my addr 172.21.15.7:0/3108302545 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:06.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.492+0000 7f34fffff700 1 --2- 172.21.15.7:0/3108302545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35080b09f0 0x7f35080cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:06.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.493+0000 7f350c82e700 1 -- 172.21.15.7:0/3108302545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35080b09f0 msgr2=0x7f35080cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:06.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.493+0000 7f350c82e700 1 --2- 172.21.15.7:0/3108302545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35080b09f0 0x7f35080cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:06.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.493+0000 7f350c82e700 1 -- 172.21.15.7:0/3108302545 >> v1:172.21.15.188:6789/0 conn(0x7f35081408b0 legacy=0x7f350806e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:06.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.493+0000 7f350c82e700 1 -- 172.21.15.7:0/3108302545 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35080cc6c0 con 0x7f35081404e0 2024-01-27T02:38:06.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.493+0000 7f34fffff700 1 --2- 172.21.15.7:0/3108302545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35080b09f0 0x7f35080cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:06.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.493+0000 7f350c82e700 1 --2- 172.21.15.7:0/3108302545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35081404e0 0x7f35080cb800 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f350400bc50 tx=0x7f350400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa28b7262581775e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:06.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.493+0000 7f34ff7fe700 1 -- 172.21.15.7:0/3108302545 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350400be60 con 0x7f35081404e0 2024-01-27T02:38:06.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.493+0000 7f34ff7fe700 1 -- 172.21.15.7:0/3108302545 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350400de00 con 0x7f35081404e0 2024-01-27T02:38:06.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.493+0000 7f34ff7fe700 1 -- 172.21.15.7:0/3108302545 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35040196b0 con 0x7f35081404e0 2024-01-27T02:38:06.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.494+0000 7f350ea92700 1 -- 172.21.15.7:0/3108302545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35081404e0 msgr2=0x7f35080cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:06.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.494+0000 7f350ea92700 1 --2- 172.21.15.7:0/3108302545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35081404e0 0x7f35080cb800 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f350400bc50 tx=0x7f350400b4f0 comp rx=0 tx=0).stop 2024-01-27T02:38:06.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.494+0000 7f350ea92700 1 -- 172.21.15.7:0/3108302545 shutdown_connections 2024-01-27T02:38:06.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.494+0000 7f350ea92700 1 --2- 172.21.15.7:0/3108302545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35081404e0 0x7f35080cb800 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:06.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.494+0000 7f350ea92700 1 --2- 172.21.15.7:0/3108302545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35080b09f0 0x7f35080cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:06.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.494+0000 7f350ea92700 1 -- 172.21.15.7:0/3108302545 >> 172.21.15.7:0/3108302545 conn(0x7f35080cdc30 msgr2=0x7f35080ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:06.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.494+0000 7f350ea92700 1 -- 172.21.15.7:0/3108302545 shutdown_connections 2024-01-27T02:38:06.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.494+0000 7f350ea92700 1 -- 172.21.15.7:0/3108302545 wait complete. 2024-01-27T02:38:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.495+0000 7f350ea92700 1 Processor -- start 2024-01-27T02:38:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.495+0000 7f350ea92700 1 -- start start 2024-01-27T02:38:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.496+0000 7f350ea92700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35080b09f0 0x7f3508166e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.496+0000 7f350ea92700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35081404e0 0x7f35080b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.496+0000 7f350ea92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35081408b0 0x7f35080b2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.496+0000 7f350ea92700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3508139290 con 0x7f35081408b0 2024-01-27T02:38:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.496+0000 7f350ea92700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3508138e00 con 0x7f35080b09f0 2024-01-27T02:38:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.496+0000 7f350ea92700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35081381f0 con 0x7f35081404e0 2024-01-27T02:38:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.496+0000 7f350d02f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35081408b0 0x7f35080b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.496+0000 7f350d02f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35081408b0 0x7f35080b2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40250/0 (socket says 172.21.15.7:40250) 2024-01-27T02:38:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.496+0000 7f350d02f700 1 -- 172.21.15.7:0/1562403455 learned_addr learned my addr 172.21.15.7:0/1562403455 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.496+0000 7f350c82e700 1 --2- 172.21.15.7:0/1562403455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35080b09f0 0x7f3508166e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:06.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.496+0000 7f34fffff700 1 --2- 172.21.15.7:0/1562403455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35081404e0 0x7f35080b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.496+0000 7f350d02f700 1 -- 172.21.15.7:0/1562403455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35081404e0 msgr2=0x7f35080b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.496+0000 7f350d02f700 1 --2- 172.21.15.7:0/1562403455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35081404e0 0x7f35080b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.496+0000 7f350d02f700 1 -- 172.21.15.7:0/1562403455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35080b09f0 msgr2=0x7f3508166e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.496+0000 7f350d02f700 1 --2- 172.21.15.7:0/1562403455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35080b09f0 0x7f3508166e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.496+0000 7f350d02f700 1 -- 172.21.15.7:0/1562403455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350400b790 con 0x7f35081408b0 2024-01-27T02:38:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.497+0000 7f350d02f700 1 --2- 172.21.15.7:0/1562403455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35081408b0 0x7f35080b2920 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f350000c840 tx=0x7f3500001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.497+0000 7f34fcff9700 1 -- 172.21.15.7:0/1562403455 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3500012630 con 0x7f35081408b0 2024-01-27T02:38:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.497+0000 7f34fcff9700 1 -- 172.21.15.7:0/1562403455 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35000127b0 con 0x7f35081408b0 2024-01-27T02:38:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.497+0000 7f350ea92700 1 -- 172.21.15.7:0/1562403455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35081384d0 con 0x7f35081408b0 2024-01-27T02:38:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.497+0000 7f34fcff9700 1 -- 172.21.15.7:0/1562403455 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3500012b70 con 0x7f35081408b0 2024-01-27T02:38:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.497+0000 7f350ea92700 1 -- 172.21.15.7:0/1562403455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3508138a90 con 0x7f35081408b0 2024-01-27T02:38:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.498+0000 7f350ea92700 1 -- 172.21.15.7:0/1562403455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3508138fb0 con 0x7f35081408b0 2024-01-27T02:38:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.498+0000 7f34fcff9700 1 -- 172.21.15.7:0/1562403455 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f350001a3a0 con 0x7f35081408b0 2024-01-27T02:38:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.499+0000 7f34fcff9700 1 --2- 172.21.15.7:0/1562403455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f34f4066f30 0x7f34f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.499+0000 7f34fcff9700 1 -- 172.21.15.7:0/1562403455 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3500084610 con 0x7f35081408b0 2024-01-27T02:38:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.499+0000 7f350c82e700 1 --2- 172.21.15.7:0/1562403455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f34f4066f30 0x7f34f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.500+0000 7f350c82e700 1 --2- 172.21.15.7:0/1562403455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f34f4066f30 0x7f34f40693e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f350400bc50 tx=0x7f350401b000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:06.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.503+0000 7f34fcff9700 1 -- 172.21.15.7:0/1562403455 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3508138fb0 con 0x7f35081408b0 2024-01-27T02:38:06.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.720+0000 7f350ea92700 1 -- 172.21.15.7:0/1562403455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) v1 -- 0x7f3508138f70 con 0x7f35081408b0 2024-01-27T02:38:06.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.897+0000 7f34fcff9700 1 -- 172.21.15.7:0/1562403455 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v119) v1 ==== 93+0+0 (secure 0 0 0) 0x7f3500051880 con 0x7f35081408b0 2024-01-27T02:38:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.900+0000 7f350ea92700 1 -- 172.21.15.7:0/1562403455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f34f4066f30 msgr2=0x7f34f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.900+0000 7f350ea92700 1 --2- 172.21.15.7:0/1562403455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f34f4066f30 0x7f34f40693e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f350400bc50 tx=0x7f350401b000 comp rx=0 tx=0).stop 2024-01-27T02:38:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.900+0000 7f350ea92700 1 -- 172.21.15.7:0/1562403455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35081408b0 msgr2=0x7f35080b2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.900+0000 7f350ea92700 1 --2- 172.21.15.7:0/1562403455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35081408b0 0x7f35080b2920 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f350000c840 tx=0x7f3500001670 comp rx=0 tx=0).stop 2024-01-27T02:38:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.900+0000 7f350ea92700 1 -- 172.21.15.7:0/1562403455 shutdown_connections 2024-01-27T02:38:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.900+0000 7f350ea92700 1 --2- 172.21.15.7:0/1562403455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f34f4066f30 0x7f34f40693e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.900+0000 7f350ea92700 1 --2- 172.21.15.7:0/1562403455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35081408b0 0x7f35080b2920 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.900+0000 7f350ea92700 1 --2- 172.21.15.7:0/1562403455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35080b09f0 0x7f3508166e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.900+0000 7f350ea92700 1 --2- 172.21.15.7:0/1562403455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35081404e0 0x7f35080b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.900+0000 7f350ea92700 1 -- 172.21.15.7:0/1562403455 >> 172.21.15.7:0/1562403455 conn(0x7f35080cdc30 msgr2=0x7f350813f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.900+0000 7f350ea92700 1 -- 172.21.15.7:0/1562403455 shutdown_connections 2024-01-27T02:38:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:06.900+0000 7f350ea92700 1 -- 172.21.15.7:0/1562403455 wait complete. 2024-01-27T02:38:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:pauserd,pausewr is set 2024-01-27T02:38:06.916 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2024-01-27T02:38:07.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.052+0000 7f90e71a0700 1 Processor -- start 2024-01-27T02:38:07.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.052+0000 7f90e71a0700 1 -- start start 2024-01-27T02:38:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.053+0000 7f90e71a0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f90e006d320 0x7f90e00cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.054+0000 7f90e71a0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e006b1c0 con 0x7f90e006d7c0 2024-01-27T02:38:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.054+0000 7f90e71a0700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e006b4a0 con 0x7f90e0157bf0 2024-01-27T02:38:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.054+0000 7f90e71a0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e00b00d0 con 0x7f90e006d320 2024-01-27T02:38:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.054+0000 7f90e599d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f90e006d320 0x7f90e00cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.054+0000 7f90e599d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f90e006d320 0x7f90e00cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40262/0 (socket says 172.21.15.7:40262) 2024-01-27T02:38:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.054+0000 7f90e599d700 1 -- 172.21.15.7:0/2342567110 learned_addr learned my addr 172.21.15.7:0/2342567110 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.054+0000 7f90e699f700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f90e006d7c0 0x7f90e015bb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44756/0 (socket says 172.21.15.7:44756) 2024-01-27T02:38:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.054+0000 7f90e599d700 1 -- 172.21.15.7:0/2342567110 >> v1:172.21.15.188:6789/0 conn(0x7f90e0157bf0 legacy=0x7f90e00af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:07.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.054+0000 7f90e599d700 1 -- 172.21.15.7:0/2342567110 >> v1:172.21.15.7:6789/0 conn(0x7f90e006d7c0 legacy=0x7f90e015bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:07.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.054+0000 7f90e599d700 1 -- 172.21.15.7:0/2342567110 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e006b780 con 0x7f90e006d320 2024-01-27T02:38:07.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.054+0000 7f90e519c700 1 -- 172.21.15.7:0/2342567110 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3704218559 0 0) 0x7f90e006b1c0 con 0x7f90e006d7c0 2024-01-27T02:38:07.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.054+0000 7f90e599d700 1 --2- 172.21.15.7:0/2342567110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90e006d320 0x7f90e00cb4f0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f90d00099b0 tx=0x7f90d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4539dbf69fb5ea server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:07.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.054+0000 7f90e519c700 1 -- 172.21.15.7:0/2342567110 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d000b860 con 0x7f90e006d320 2024-01-27T02:38:07.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.054+0000 7f90e519c700 1 -- 172.21.15.7:0/2342567110 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d000b9c0 con 0x7f90e006d320 2024-01-27T02:38:07.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.054+0000 7f90e519c700 1 -- 172.21.15.7:0/2342567110 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d000bd60 con 0x7f90e006d320 2024-01-27T02:38:07.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.055+0000 7f90e71a0700 1 -- 172.21.15.7:0/2342567110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90e006d320 msgr2=0x7f90e00cb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:07.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.055+0000 7f90e71a0700 1 --2- 172.21.15.7:0/2342567110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90e006d320 0x7f90e00cb4f0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f90d00099b0 tx=0x7f90d0009250 comp rx=0 tx=0).stop 2024-01-27T02:38:07.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.055+0000 7f90e71a0700 1 -- 172.21.15.7:0/2342567110 shutdown_connections 2024-01-27T02:38:07.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.055+0000 7f90e71a0700 1 --2- 172.21.15.7:0/2342567110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90e006d320 0x7f90e00cb4f0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:07.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.055+0000 7f90e71a0700 1 -- 172.21.15.7:0/2342567110 >> 172.21.15.7:0/2342567110 conn(0x7f90e006f160 msgr2=0x7f90e006f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:07.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.055+0000 7f90e71a0700 1 -- 172.21.15.7:0/2342567110 shutdown_connections 2024-01-27T02:38:07.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.055+0000 7f90e71a0700 1 -- 172.21.15.7:0/2342567110 wait complete. 2024-01-27T02:38:07.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.057+0000 7f90e71a0700 1 Processor -- start 2024-01-27T02:38:07.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.057+0000 7f90e71a0700 1 -- start start 2024-01-27T02:38:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.058+0000 7f90e71a0700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e006d320 0x7f90e016bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.058+0000 7f90e71a0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90e006d7c0 0x7f90e015ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.058+0000 7f90e71a0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90e0157bf0 0x7f90e015bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.058+0000 7f90e71a0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e01518b0 con 0x7f90e0157bf0 2024-01-27T02:38:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.058+0000 7f90e71a0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e00b18d0 con 0x7f90e006d7c0 2024-01-27T02:38:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.058+0000 7f90e71a0700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e00b1bb0 con 0x7f90e006d320 2024-01-27T02:38:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.058+0000 7f90e699f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90e0157bf0 0x7f90e015bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.058+0000 7f90e699f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90e0157bf0 0x7f90e015bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40264/0 (socket says 172.21.15.7:40264) 2024-01-27T02:38:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.058+0000 7f90e699f700 1 -- 172.21.15.7:0/3491380184 learned_addr learned my addr 172.21.15.7:0/3491380184 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.058+0000 7f90e599d700 1 --2- 172.21.15.7:0/3491380184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90e006d7c0 0x7f90e015ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.058+0000 7f90e619e700 1 --2- 172.21.15.7:0/3491380184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e006d320 0x7f90e016bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.058+0000 7f90e699f700 1 -- 172.21.15.7:0/3491380184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e006d320 msgr2=0x7f90e016bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.058+0000 7f90e699f700 1 --2- 172.21.15.7:0/3491380184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e006d320 0x7f90e016bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.058+0000 7f90e699f700 1 -- 172.21.15.7:0/3491380184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90e006d7c0 msgr2=0x7f90e015ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.058+0000 7f90e699f700 1 --2- 172.21.15.7:0/3491380184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90e006d7c0 0x7f90e015ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.058+0000 7f90e699f700 1 -- 172.21.15.7:0/3491380184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d00094f0 con 0x7f90e0157bf0 2024-01-27T02:38:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.058+0000 7f90e599d700 1 --2- 172.21.15.7:0/3491380184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90e006d7c0 0x7f90e015ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:38:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.059+0000 7f90e699f700 1 --2- 172.21.15.7:0/3491380184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90e0157bf0 0x7f90e015bfd0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f90d800cd50 tx=0x7f90d8001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.059+0000 7f90d67fc700 1 -- 172.21.15.7:0/3491380184 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d8005df0 con 0x7f90e0157bf0 2024-01-27T02:38:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.059+0000 7f90d67fc700 1 -- 172.21.15.7:0/3491380184 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d800a960 con 0x7f90e0157bf0 2024-01-27T02:38:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.059+0000 7f90e71a0700 1 -- 172.21.15.7:0/3491380184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e016c5f0 con 0x7f90e0157bf0 2024-01-27T02:38:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.059+0000 7f90d67fc700 1 -- 172.21.15.7:0/3491380184 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d800ad20 con 0x7f90e0157bf0 2024-01-27T02:38:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.059+0000 7f90e71a0700 1 -- 172.21.15.7:0/3491380184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e016cbb0 con 0x7f90e0157bf0 2024-01-27T02:38:07.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.060+0000 7f90e71a0700 1 -- 172.21.15.7:0/3491380184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e00b1d20 con 0x7f90e0157bf0 2024-01-27T02:38:07.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.060+0000 7f90d67fc700 1 -- 172.21.15.7:0/3491380184 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f90d8007300 con 0x7f90e0157bf0 2024-01-27T02:38:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.061+0000 7f90d67fc700 1 --2- 172.21.15.7:0/3491380184 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f90c4066f30 0x7f90c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.061+0000 7f90d67fc700 1 -- 172.21.15.7:0/3491380184 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f90d80847e0 con 0x7f90e0157bf0 2024-01-27T02:38:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.061+0000 7f90d67fc700 1 -- 172.21.15.7:0/3491380184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=120+}) v3 -- 0x7f90c406ea50 con 0x7f90e0157bf0 2024-01-27T02:38:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.061+0000 7f90e619e700 1 --2- 172.21.15.7:0/3491380184 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f90c4066f30 0x7f90c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.062+0000 7f90e619e700 1 --2- 172.21.15.7:0/3491380184 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f90c4066f30 0x7f90c40693e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f90e015ee00 tx=0x7f90dc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.065+0000 7f90d67fc700 1 -- 172.21.15.7:0/3491380184 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90d8051a30 con 0x7f90e0157bf0 2024-01-27T02:38:07.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.282+0000 7f90e71a0700 1 -- 172.21.15.7:0/3491380184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) v1 -- 0x7f90e004c710 con 0x7f90e0157bf0 2024-01-27T02:38:07.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.899+0000 7f90d67fc700 1 -- 172.21.15.7:0/3491380184 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(120..120 src has 1..120) v4 ==== 280+0+0 (secure 0 0 0) 0x7f90d8055df0 con 0x7f90e0157bf0 2024-01-27T02:38:07.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.899+0000 7f90d67fc700 1 -- 172.21.15.7:0/3491380184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=121}) v3 -- 0x7f90c406f340 con 0x7f90e0157bf0 2024-01-27T02:38:07.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.902+0000 7f90d67fc700 1 -- 172.21.15.7:0/3491380184 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v120) v1 ==== 97+0+0 (secure 0 0 0) 0x7f90d8055050 con 0x7f90e0157bf0 2024-01-27T02:38:07.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.905+0000 7f90e71a0700 1 -- 172.21.15.7:0/3491380184 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f90c4066f30 msgr2=0x7f90c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:07.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.905+0000 7f90e71a0700 1 --2- 172.21.15.7:0/3491380184 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f90c4066f30 0x7f90c40693e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f90e015ee00 tx=0x7f90dc009040 comp rx=0 tx=0).stop 2024-01-27T02:38:07.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.905+0000 7f90e71a0700 1 -- 172.21.15.7:0/3491380184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90e0157bf0 msgr2=0x7f90e015bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:07.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.905+0000 7f90e71a0700 1 --2- 172.21.15.7:0/3491380184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90e0157bf0 0x7f90e015bfd0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f90d800cd50 tx=0x7f90d8001b10 comp rx=0 tx=0).stop 2024-01-27T02:38:07.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.905+0000 7f90e71a0700 1 -- 172.21.15.7:0/3491380184 shutdown_connections 2024-01-27T02:38:07.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.905+0000 7f90e71a0700 1 --2- 172.21.15.7:0/3491380184 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f90c4066f30 0x7f90c40693e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:07.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.905+0000 7f90e71a0700 1 --2- 172.21.15.7:0/3491380184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90e0157bf0 0x7f90e015bfd0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:07.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.905+0000 7f90e71a0700 1 --2- 172.21.15.7:0/3491380184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90e006d320 0x7f90e016bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:07.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.905+0000 7f90e71a0700 1 --2- 172.21.15.7:0/3491380184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90e006d7c0 0x7f90e015ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:07.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.905+0000 7f90e71a0700 1 -- 172.21.15.7:0/3491380184 >> 172.21.15.7:0/3491380184 conn(0x7f90e006f160 msgr2=0x7f90e0156e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:07.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.905+0000 7f90e71a0700 1 -- 172.21.15.7:0/3491380184 shutdown_connections 2024-01-27T02:38:07.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:07.905+0000 7f90e71a0700 1 -- 172.21.15.7:0/3491380184 wait complete. 2024-01-27T02:38:07.917 INFO:teuthology.orchestra.run.smithi007.stderr:pauserd,pausewr is unset 2024-01-27T02:38:07.922 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:38:08.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.062+0000 7f78b05a8700 1 Processor -- start 2024-01-27T02:38:08.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.062+0000 7f78b05a8700 1 -- start start 2024-01-27T02:38:08.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.063+0000 7f78b05a8700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f78a80c3ae0 0x7f78a80cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:08.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.063+0000 7f78b05a8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a8158070 0x7f78a80cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:08.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.063+0000 7f78b05a8700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a80ca020 con 0x7f78a8157ca0 2024-01-27T02:38:08.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.063+0000 7f78b05a8700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a80c5210 con 0x7f78a80c3ae0 2024-01-27T02:38:08.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.063+0000 7f78b05a8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a80c54f0 con 0x7f78a8158070 2024-01-27T02:38:08.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.063+0000 7f78adb43700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f78a80c3ae0 0x7f78a80cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:08.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.063+0000 7f78aeb45700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a8158070 0x7f78a80cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:08.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.063+0000 7f78ae344700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f78a8157ca0 0x7f78a80c4a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:35020/0 (socket says 172.21.15.7:35020) 2024-01-27T02:38:08.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.063+0000 7f78ae344700 1 -- 172.21.15.7:0/846070641 learned_addr learned my addr 172.21.15.7:0/846070641 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:08.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.064+0000 7f78aeb45700 1 -- 172.21.15.7:0/846070641 >> v2:172.21.15.188:3300/0 conn(0x7f78a80c3ae0 msgr2=0x7f78a80cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:08.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.064+0000 7f78aeb45700 1 --2- 172.21.15.7:0/846070641 >> v2:172.21.15.188:3300/0 conn(0x7f78a80c3ae0 0x7f78a80cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:08.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.064+0000 7f78aeb45700 1 -- 172.21.15.7:0/846070641 >> v1:172.21.15.188:6789/0 conn(0x7f78a8157ca0 legacy=0x7f78a80c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:08.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.064+0000 7f78aeb45700 1 -- 172.21.15.7:0/846070641 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a80c57d0 con 0x7f78a8158070 2024-01-27T02:38:08.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.064+0000 7f78adb43700 1 --2- 172.21.15.7:0/846070641 >> v2:172.21.15.188:3300/0 conn(0x7f78a80c3ae0 0x7f78a80cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:38:08.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.064+0000 7f78aeb45700 1 --2- 172.21.15.7:0/846070641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a8158070 0x7f78a80cda20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009e60 tx=0x7f78a0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2615c126244d903 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:08.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.064+0000 7f78ad342700 1 -- 172.21.15.7:0/846070641 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a000e070 con 0x7f78a8158070 2024-01-27T02:38:08.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.064+0000 7f78ad342700 1 -- 172.21.15.7:0/846070641 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a000bb60 con 0x7f78a8158070 2024-01-27T02:38:08.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.064+0000 7f78ad342700 1 -- 172.21.15.7:0/846070641 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a0012430 con 0x7f78a8158070 2024-01-27T02:38:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.065+0000 7f78b05a8700 1 -- 172.21.15.7:0/846070641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a8158070 msgr2=0x7f78a80cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.065+0000 7f78b05a8700 1 --2- 172.21.15.7:0/846070641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a8158070 0x7f78a80cda20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009e60 tx=0x7f78a0009780 comp rx=0 tx=0).stop 2024-01-27T02:38:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.065+0000 7f78b05a8700 1 -- 172.21.15.7:0/846070641 shutdown_connections 2024-01-27T02:38:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.065+0000 7f78b05a8700 1 --2- 172.21.15.7:0/846070641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a8158070 0x7f78a80cda20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.065+0000 7f78b05a8700 1 --2- 172.21.15.7:0/846070641 >> v2:172.21.15.188:3300/0 conn(0x7f78a80c3ae0 0x7f78a80cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.065+0000 7f78b05a8700 1 -- 172.21.15.7:0/846070641 >> 172.21.15.7:0/846070641 conn(0x7f78a8098020 msgr2=0x7f78a80711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.065+0000 7f78b05a8700 1 -- 172.21.15.7:0/846070641 shutdown_connections 2024-01-27T02:38:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.065+0000 7f78b05a8700 1 -- 172.21.15.7:0/846070641 wait complete. 2024-01-27T02:38:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.066+0000 7f78b05a8700 1 Processor -- start 2024-01-27T02:38:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.066+0000 7f78b05a8700 1 -- start start 2024-01-27T02:38:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.067+0000 7f78b05a8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a80c3ae0 0x7f78a8155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.067+0000 7f78b05a8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a8157ca0 0x7f78a8153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.067+0000 7f78b05a8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78a8158070 0x7f78a8152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:08.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.067+0000 7f78b05a8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a81552d0 con 0x7f78a8157ca0 2024-01-27T02:38:08.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.067+0000 7f78b05a8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a8144cf0 con 0x7f78a8158070 2024-01-27T02:38:08.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.067+0000 7f78b05a8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a81546c0 con 0x7f78a80c3ae0 2024-01-27T02:38:08.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.067+0000 7f78adb43700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a8157ca0 0x7f78a8153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:08.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.067+0000 7f78adb43700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a8157ca0 0x7f78a8153e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40274/0 (socket says 172.21.15.7:40274) 2024-01-27T02:38:08.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.067+0000 7f78adb43700 1 -- 172.21.15.7:0/3729126981 learned_addr learned my addr 172.21.15.7:0/3729126981 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:08.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.067+0000 7f78aeb45700 1 --2- 172.21.15.7:0/3729126981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78a8158070 0x7f78a8152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.067+0000 7f78ae344700 1 --2- 172.21.15.7:0/3729126981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a80c3ae0 0x7f78a8155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.067+0000 7f78adb43700 1 -- 172.21.15.7:0/3729126981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a80c3ae0 msgr2=0x7f78a8155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.067+0000 7f78adb43700 1 --2- 172.21.15.7:0/3729126981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a80c3ae0 0x7f78a8155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.067+0000 7f78adb43700 1 -- 172.21.15.7:0/3729126981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78a8158070 msgr2=0x7f78a8152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.067+0000 7f78adb43700 1 --2- 172.21.15.7:0/3729126981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78a8158070 0x7f78a8152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.067+0000 7f78adb43700 1 -- 172.21.15.7:0/3729126981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7898009420 con 0x7f78a8157ca0 2024-01-27T02:38:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.067+0000 7f78aeb45700 1 --2- 172.21.15.7:0/3729126981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78a8158070 0x7f78a8152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.067+0000 7f78adb43700 1 --2- 172.21.15.7:0/3729126981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a8157ca0 0x7f78a8153e70 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f789800ec80 tx=0x7f78980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.067+0000 7f789e7fc700 1 -- 172.21.15.7:0/3729126981 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789800c9d0 con 0x7f78a8157ca0 2024-01-27T02:38:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.068+0000 7f789e7fc700 1 -- 172.21.15.7:0/3729126981 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789800cb30 con 0x7f78a8157ca0 2024-01-27T02:38:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.068+0000 7f789e7fc700 1 -- 172.21.15.7:0/3729126981 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789800ceb0 con 0x7f78a8157ca0 2024-01-27T02:38:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.068+0000 7f78b05a8700 1 -- 172.21.15.7:0/3729126981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a0009a20 con 0x7f78a8157ca0 2024-01-27T02:38:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.068+0000 7f78b05a8700 1 -- 172.21.15.7:0/3729126981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a8154c80 con 0x7f78a8157ca0 2024-01-27T02:38:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.069+0000 7f78b05a8700 1 -- 172.21.15.7:0/3729126981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a8155440 con 0x7f78a8157ca0 2024-01-27T02:38:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.069+0000 7f789e7fc700 1 -- 172.21.15.7:0/3729126981 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7898013680 con 0x7f78a8157ca0 2024-01-27T02:38:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.070+0000 7f789e7fc700 1 --2- 172.21.15.7:0/3729126981 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f788c066f30 0x7f788c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.070+0000 7f789e7fc700 1 -- 172.21.15.7:0/3729126981 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f78980848d0 con 0x7f78a8157ca0 2024-01-27T02:38:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.070+0000 7f78ae344700 1 --2- 172.21.15.7:0/3729126981 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f788c066f30 0x7f788c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.070+0000 7f78ae344700 1 --2- 172.21.15.7:0/3729126981 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f788c066f30 0x7f788c0693e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f78a4001820 tx=0x7f78a4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.074+0000 7f789e7fc700 1 -- 172.21.15.7:0/3729126981 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7898051b40 con 0x7f78a8157ca0 2024-01-27T02:38:08.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.277+0000 7f78b05a8700 1 -- 172.21.15.7:0/3729126981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78a804c710 con 0x7f78a8157ca0 2024-01-27T02:38:08.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.278+0000 7f789e7fc700 1 -- 172.21.15.7:0/3729126981 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+15324 (secure 0 0 0) 0x7f7898055160 con 0x7f78a8157ca0 2024-01-27T02:38:08.279 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:08.279 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":120,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:38:07.896480+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:37:49.747694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-27T02:37:50.199870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:38:08.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.280+0000 7f78b05a8700 1 -- 172.21.15.7:0/3729126981 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f788c066f30 msgr2=0x7f788c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:08.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.280+0000 7f78b05a8700 1 --2- 172.21.15.7:0/3729126981 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f788c066f30 0x7f788c0693e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f78a4001820 tx=0x7f78a4009040 comp rx=0 tx=0).stop 2024-01-27T02:38:08.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.280+0000 7f78b05a8700 1 -- 172.21.15.7:0/3729126981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a8157ca0 msgr2=0x7f78a8153e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.280+0000 7f78b05a8700 1 --2- 172.21.15.7:0/3729126981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a8157ca0 0x7f78a8153e70 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f789800ec80 tx=0x7f78980091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.280+0000 7f78b05a8700 1 -- 172.21.15.7:0/3729126981 shutdown_connections 2024-01-27T02:38:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.280+0000 7f78b05a8700 1 --2- 172.21.15.7:0/3729126981 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f788c066f30 0x7f788c0693e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.280+0000 7f78b05a8700 1 --2- 172.21.15.7:0/3729126981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78a8158070 0x7f78a8152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.280+0000 7f78b05a8700 1 --2- 172.21.15.7:0/3729126981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78a80c3ae0 0x7f78a8155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.280+0000 7f78b05a8700 1 --2- 172.21.15.7:0/3729126981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a8157ca0 0x7f78a8153e70 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.280+0000 7f78b05a8700 1 -- 172.21.15.7:0/3729126981 >> 172.21.15.7:0/3729126981 conn(0x7f78a8098020 msgr2=0x7f78a80cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.280+0000 7f78b05a8700 1 -- 172.21.15.7:0/3729126981 shutdown_connections 2024-01-27T02:38:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.281+0000 7f78b05a8700 1 -- 172.21.15.7:0/3729126981 wait complete. 2024-01-27T02:38:08.295 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-01-27T02:38:08.295 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-01-27T02:38:08.295 DEBUG:teuthology.orchestra.run.smithi007:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-01-27T02:38:08.295 DEBUG:teuthology.orchestra.run.smithi007:> os.fchmod(fd, 0O666) 2024-01-27T02:38:08.295 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(fd) 2024-01-27T02:38:08.295 DEBUG:teuthology.orchestra.run.smithi007:> os.close(fd) 2024-01-27T02:38:08.295 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-01-27T02:38:08.560 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:38:08.560 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-27T02:38:08.645 INFO:teuthology.orchestra.run.smithi007.stdout:client6828 2024-01-27T02:38:08.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:08.645 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/17e1bcca-e647-4a58-b0ac-7d39d8508814.client6828/osdmap') 2024-01-27T02:38:08.733 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:38:08.737 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:38:08.739 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 117 barrier 117 flags 0x588000 2024-01-27T02:38:08.739 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:38:08.739 INFO:teuthology.orchestra.run.smithi007.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:38:08.739 INFO:teuthology.orchestra.run.smithi007.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:38:08.740 INFO:teuthology.orchestra.run.smithi007.stdout:osd0 (1)172.21.15.7:6817 100% (exists, up) 100% -1 2024-01-27T02:38:08.740 INFO:teuthology.orchestra.run.smithi007.stdout:osd1 (1)172.21.15.7:6801 100% (exists, up) 100% -1 2024-01-27T02:38:08.740 INFO:teuthology.orchestra.run.smithi007.stdout:osd2 (1)172.21.15.7:6825 100% (exists, up) 100% -1 2024-01-27T02:38:08.740 INFO:teuthology.orchestra.run.smithi007.stdout:osd3 (1)172.21.15.7:6809 100% (exists, up) 100% -1 2024-01-27T02:38:08.740 INFO:teuthology.orchestra.run.smithi007.stdout:osd4 (1)172.21.15.188:6803 100% (exists, up) 100% -1 2024-01-27T02:38:08.740 INFO:teuthology.orchestra.run.smithi007.stdout:osd5 (1)172.21.15.188:6802 100% (exists, up) 100% -1 2024-01-27T02:38:08.740 INFO:teuthology.orchestra.run.smithi007.stdout:osd6 (1)172.21.15.188:6825 100% (exists, up) 100% -1 2024-01-27T02:38:08.740 INFO:teuthology.orchestra.run.smithi007.stdout:osd7 (1)172.21.15.188:6817 100% (exists, up) 100% -1 2024-01-27T02:38:08.740 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records in 2024-01-27T02:38:08.740 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records out 2024-01-27T02:38:08.740 INFO:teuthology.orchestra.run.smithi007.stderr:795 bytes copied, 4.7385e-05 s, 16.8 MB/s 2024-01-27T02:38:08.742 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:38:08.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.924+0000 7fccf2b1d700 1 Processor -- start 2024-01-27T02:38:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.924+0000 7fccf2b1d700 1 -- start start 2024-01-27T02:38:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.925+0000 7fccf2b1d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fccec15be10 0x7fccec0b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.925+0000 7fccf2b1d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccec15c1e0 0x7fccec0c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.925+0000 7fccf2b1d700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccec0b1470 con 0x7fccec0b0180 2024-01-27T02:38:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.925+0000 7fccf2b1d700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccec0b0600 con 0x7fccec15be10 2024-01-27T02:38:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.925+0000 7fccf2b1d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccec0b0860 con 0x7fccec15c1e0 2024-01-27T02:38:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.925+0000 7fccf10ba700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccec15c1e0 0x7fccec0c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.925+0000 7fccf10ba700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccec15c1e0 0x7fccec0c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:48194/0 (socket says 172.21.15.7:48194) 2024-01-27T02:38:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.925+0000 7fccf10ba700 1 -- 172.21.15.7:0/1945101939 learned_addr learned my addr 172.21.15.7:0/1945101939 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.925+0000 7fccf08b9700 1 --2- 172.21.15.7:0/1945101939 >> v2:172.21.15.188:3300/0 conn(0x7fccec15be10 0x7fccec0b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.926+0000 7fccf10ba700 1 -- 172.21.15.7:0/1945101939 >> v2:172.21.15.188:3300/0 conn(0x7fccec15be10 msgr2=0x7fccec0b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.926+0000 7fccf10ba700 1 --2- 172.21.15.7:0/1945101939 >> v2:172.21.15.188:3300/0 conn(0x7fccec15be10 0x7fccec0b1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.926+0000 7fccf10ba700 1 -- 172.21.15.7:0/1945101939 >> v1:172.21.15.188:6789/0 conn(0x7fccec0b0180 legacy=0x7fccec0aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:08.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.926+0000 7fccf10ba700 1 -- 172.21.15.7:0/1945101939 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec0b0b40 con 0x7fccec15c1e0 2024-01-27T02:38:08.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.926+0000 7fccf08b9700 1 --2- 172.21.15.7:0/1945101939 >> v2:172.21.15.188:3300/0 conn(0x7fccec15be10 0x7fccec0b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:38:08.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.926+0000 7fccf10ba700 1 --2- 172.21.15.7:0/1945101939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccec15c1e0 0x7fccec0c9500 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009f20 tx=0x7fcce4009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=356a1c29bd165c3b server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:08.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.926+0000 7fcceb7fe700 1 -- 172.21.15.7:0/1945101939 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce400e070 con 0x7fccec15c1e0 2024-01-27T02:38:08.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.926+0000 7fcceb7fe700 1 -- 172.21.15.7:0/1945101939 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce400b8d0 con 0x7fccec15c1e0 2024-01-27T02:38:08.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.926+0000 7fcceb7fe700 1 -- 172.21.15.7:0/1945101939 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce40092e0 con 0x7fccec15c1e0 2024-01-27T02:38:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.927+0000 7fccf2b1d700 1 -- 172.21.15.7:0/1945101939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccec15c1e0 msgr2=0x7fccec0c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.927+0000 7fccf2b1d700 1 --2- 172.21.15.7:0/1945101939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccec15c1e0 0x7fccec0c9500 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009f20 tx=0x7fcce4009840 comp rx=0 tx=0).stop 2024-01-27T02:38:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.927+0000 7fccf2b1d700 1 -- 172.21.15.7:0/1945101939 shutdown_connections 2024-01-27T02:38:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.927+0000 7fccf2b1d700 1 --2- 172.21.15.7:0/1945101939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccec15c1e0 0x7fccec0c9500 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.927+0000 7fccf2b1d700 1 --2- 172.21.15.7:0/1945101939 >> v2:172.21.15.188:3300/0 conn(0x7fccec15be10 0x7fccec0b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.927+0000 7fccf2b1d700 1 -- 172.21.15.7:0/1945101939 >> 172.21.15.7:0/1945101939 conn(0x7fccec0679b0 msgr2=0x7fccec067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.927+0000 7fccf2b1d700 1 -- 172.21.15.7:0/1945101939 shutdown_connections 2024-01-27T02:38:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.927+0000 7fccf2b1d700 1 -- 172.21.15.7:0/1945101939 wait complete. 2024-01-27T02:38:08.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.928+0000 7fccf2b1d700 1 Processor -- start 2024-01-27T02:38:08.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.929+0000 7fccf2b1d700 1 -- start start 2024-01-27T02:38:08.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.930+0000 7fccf2b1d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccec0b0180 0x7fccec14b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:08.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.930+0000 7fccf2b1d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccec15be10 0x7fccec1588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:08.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.930+0000 7fccf2b1d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccec15c1e0 0x7fccec099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:08.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.930+0000 7fccf2b1d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec14cd00 con 0x7fccec15be10 2024-01-27T02:38:08.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.930+0000 7fccf2b1d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec159210 con 0x7fccec0b0180 2024-01-27T02:38:08.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.930+0000 7fccf2b1d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccec1594f0 con 0x7fccec15c1e0 2024-01-27T02:38:08.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.930+0000 7fccebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccec15be10 0x7fccec1588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:08.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.930+0000 7fccebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccec15be10 0x7fccec1588c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40286/0 (socket says 172.21.15.7:40286) 2024-01-27T02:38:08.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.930+0000 7fccebfff700 1 -- 172.21.15.7:0/1689311439 learned_addr learned my addr 172.21.15.7:0/1689311439 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:08.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.930+0000 7fccf08b9700 1 --2- 172.21.15.7:0/1689311439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccec0b0180 0x7fccec14b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:08.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.930+0000 7fccf10ba700 1 --2- 172.21.15.7:0/1689311439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccec15c1e0 0x7fccec099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:08.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.930+0000 7fccebfff700 1 -- 172.21.15.7:0/1689311439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccec15c1e0 msgr2=0x7fccec099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:08.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.930+0000 7fccebfff700 1 --2- 172.21.15.7:0/1689311439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccec15c1e0 0x7fccec099a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:08.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.930+0000 7fccebfff700 1 -- 172.21.15.7:0/1689311439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccec0b0180 msgr2=0x7fccec14b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:08.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.930+0000 7fccebfff700 1 --2- 172.21.15.7:0/1689311439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccec0b0180 0x7fccec14b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:08.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.930+0000 7fccebfff700 1 -- 172.21.15.7:0/1689311439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccdc009420 con 0x7fccec15be10 2024-01-27T02:38:08.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.930+0000 7fccf08b9700 1 --2- 172.21.15.7:0/1689311439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccec0b0180 0x7fccec14b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:08.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.930+0000 7fccebfff700 1 --2- 172.21.15.7:0/1689311439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccec15be10 0x7fccec1588c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fccd400c760 tx=0x7fccd4007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:08.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.931+0000 7fcce8ff9700 1 -- 172.21.15.7:0/1689311439 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd4012630 con 0x7fccec15be10 2024-01-27T02:38:08.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.931+0000 7fcce8ff9700 1 -- 172.21.15.7:0/1689311439 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd4012790 con 0x7fccec15be10 2024-01-27T02:38:08.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.931+0000 7fcce8ff9700 1 -- 172.21.15.7:0/1689311439 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd4012b50 con 0x7fccec15be10 2024-01-27T02:38:08.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.931+0000 7fccf2b1d700 1 -- 172.21.15.7:0/1689311439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce4009ae0 con 0x7fccec15be10 2024-01-27T02:38:08.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.931+0000 7fccf2b1d700 1 -- 172.21.15.7:0/1689311439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccec14bc60 con 0x7fccec15be10 2024-01-27T02:38:08.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.932+0000 7fccf2b1d700 1 -- 172.21.15.7:0/1689311439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccec14ce70 con 0x7fccec15be10 2024-01-27T02:38:08.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.932+0000 7fcce8ff9700 1 -- 172.21.15.7:0/1689311439 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fccd401b400 con 0x7fccec15be10 2024-01-27T02:38:08.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.933+0000 7fcce8ff9700 1 --2- 172.21.15.7:0/1689311439 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcccc066f30 0x7fcccc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:08.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.933+0000 7fcce8ff9700 1 -- 172.21.15.7:0/1689311439 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fccd4084270 con 0x7fccec15be10 2024-01-27T02:38:08.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.933+0000 7fccf08b9700 1 --2- 172.21.15.7:0/1689311439 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcccc066f30 0x7fcccc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:08.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.933+0000 7fccf08b9700 1 --2- 172.21.15.7:0/1689311439 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcccc066f30 0x7fcccc0693e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0093f0 tx=0x7fccdc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:08.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:08.937+0000 7fcce8ff9700 1 -- 172.21.15.7:0/1689311439 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fccec14ce70 con 0x7fccec15be10 2024-01-27T02:38:09.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:09.187+0000 7fccf2b1d700 1 -- 172.21.15.7:0/1689311439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fccec09ad40 con 0x7fccec15be10 2024-01-27T02:38:09.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:09.188+0000 7fcce8ff9700 1 -- 172.21.15.7:0/1689311439 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fccd4051440 con 0x7fccec15be10 2024-01-27T02:38:09.192 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:09.192 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2767759975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2767759975},{"type":"v1","addr":"172.21.15.188:6836","nonce":2767759975}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":5768,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/245473575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":245473575},{"type":"v1","addr":"172.21.15.7:6834","nonce":245473575}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1334164270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1334164270},{"type":"v1","addr":"172.21.15.188:6837","nonce":1334164270}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:37:51.231918+0000","modified":"2024-01-27T02:37:54.851112+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6657},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6657":{"gid":6657,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1251886476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1251886476},{"type":"v1","addr":"172.21.15.7:6835","nonce":1251886476}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2024-01-27T02:38:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:09.190+0000 7fccf2b1d700 1 -- 172.21.15.7:0/1689311439 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcccc066f30 msgr2=0x7fcccc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:09.190+0000 7fccf2b1d700 1 --2- 172.21.15.7:0/1689311439 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcccc066f30 0x7fcccc0693e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0093f0 tx=0x7fccdc008040 comp rx=0 tx=0).stop 2024-01-27T02:38:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:09.190+0000 7fccf2b1d700 1 -- 172.21.15.7:0/1689311439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccec15be10 msgr2=0x7fccec1588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:09.190+0000 7fccf2b1d700 1 --2- 172.21.15.7:0/1689311439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccec15be10 0x7fccec1588c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fccd400c760 tx=0x7fccd4007f00 comp rx=0 tx=0).stop 2024-01-27T02:38:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:09.190+0000 7fccf2b1d700 1 -- 172.21.15.7:0/1689311439 shutdown_connections 2024-01-27T02:38:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:09.190+0000 7fccf2b1d700 1 --2- 172.21.15.7:0/1689311439 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcccc066f30 0x7fcccc0693e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:09.190+0000 7fccf2b1d700 1 --2- 172.21.15.7:0/1689311439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccec15c1e0 0x7fccec099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:09.190+0000 7fccf2b1d700 1 --2- 172.21.15.7:0/1689311439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccec0b0180 0x7fccec14b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:09.190+0000 7fccf2b1d700 1 --2- 172.21.15.7:0/1689311439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccec15be10 0x7fccec1588c0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:09.190+0000 7fccf2b1d700 1 -- 172.21.15.7:0/1689311439 >> 172.21.15.7:0/1689311439 conn(0x7fccec0679b0 msgr2=0x7fccec14c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:09.190+0000 7fccf2b1d700 1 -- 172.21.15.7:0/1689311439 shutdown_connections 2024-01-27T02:38:09.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:09.190+0000 7fccf2b1d700 1 -- 172.21.15.7:0/1689311439 wait complete. 2024-01-27T02:38:09.194 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 58 2024-01-27T02:38:09.205 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json osdmap barrier 120 2024-01-27T02:38:09.356 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2024-01-27T02:38:09.356 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-01-27T02:38:09.357 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-01-27T02:38:09.357 DEBUG:teuthology.orchestra.run.smithi007:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2024-01-27T02:38:09.357 DEBUG:teuthology.orchestra.run.smithi007:> os.fchmod(fd, 0O666) 2024-01-27T02:38:09.357 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(fd) 2024-01-27T02:38:09.357 DEBUG:teuthology.orchestra.run.smithi007:> os.close(fd) 2024-01-27T02:38:09.357 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-01-27T02:38:09.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:38:09.622 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-27T02:38:09.705 INFO:teuthology.orchestra.run.smithi007.stdout:client6828 2024-01-27T02:38:09.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:09.706 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/17e1bcca-e647-4a58-b0ac-7d39d8508814.client6828/osdmap') 2024-01-27T02:38:09.799 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 120 barrier 120 flags 0x588000 2024-01-27T02:38:09.799 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:38:09.799 INFO:teuthology.orchestra.run.smithi007.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:38:09.799 INFO:teuthology.orchestra.run.smithi007.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:38:09.799 INFO:teuthology.orchestra.run.smithi007.stdout:osd0 (1)172.21.15.7:6817 100% (exists, up) 100% -1 2024-01-27T02:38:09.800 INFO:teuthology.orchestra.run.smithi007.stdout:osd1 (1)172.21.15.7:6801 100% (exists, up) 100% -1 2024-01-27T02:38:09.800 INFO:teuthology.orchestra.run.smithi007.stdout:osd2 (1)172.21.15.7:6825 100% (exists, up) 100% -1 2024-01-27T02:38:09.800 INFO:teuthology.orchestra.run.smithi007.stdout:osd3 (1)172.21.15.7:6809 100% (exists, up) 100% -1 2024-01-27T02:38:09.800 INFO:teuthology.orchestra.run.smithi007.stdout:osd4 (1)172.21.15.188:6803 100% (exists, up) 100% -1 2024-01-27T02:38:09.800 INFO:teuthology.orchestra.run.smithi007.stdout:osd5 (1)172.21.15.188:6802 100% (exists, up) 100% -1 2024-01-27T02:38:09.800 INFO:teuthology.orchestra.run.smithi007.stdout:osd6 (1)172.21.15.188:6825 100% (exists, up) 100% -1 2024-01-27T02:38:09.800 INFO:teuthology.orchestra.run.smithi007.stdout:osd7 (1)172.21.15.188:6817 100% (exists, up) 100% -1 2024-01-27T02:38:09.804 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records in 2024-01-27T02:38:09.804 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records out 2024-01-27T02:38:09.804 INFO:teuthology.orchestra.run.smithi007.stderr:795 bytes copied, 5.1759e-05 s, 15.4 MB/s 2024-01-27T02:38:09.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:38:09.805 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-27T02:38:09.845 INFO:teuthology.orchestra.run.smithi007.stdout:client6828 2024-01-27T02:38:09.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:09.846 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/17e1bcca-e647-4a58-b0ac-7d39d8508814.client6828/osdmap') 2024-01-27T02:38:09.939 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 120 barrier 120 flags 0x588000 2024-01-27T02:38:09.940 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:38:09.940 INFO:teuthology.orchestra.run.smithi007.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:38:09.940 INFO:teuthology.orchestra.run.smithi007.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-27T02:38:09.940 INFO:teuthology.orchestra.run.smithi007.stdout:osd0 (1)172.21.15.7:6817 100% (exists, up) 100% -1 2024-01-27T02:38:09.940 INFO:teuthology.orchestra.run.smithi007.stdout:osd1 (1)172.21.15.7:6801 100% (exists, up) 100% -1 2024-01-27T02:38:09.940 INFO:teuthology.orchestra.run.smithi007.stdout:osd2 (1)172.21.15.7:6825 100% (exists, up) 100% -1 2024-01-27T02:38:09.940 INFO:teuthology.orchestra.run.smithi007.stdout:osd3 (1)172.21.15.7:6809 100% (exists, up) 100% -1 2024-01-27T02:38:09.940 INFO:teuthology.orchestra.run.smithi007.stdout:osd4 (1)172.21.15.188:6803 100% (exists, up) 100% -1 2024-01-27T02:38:09.940 INFO:teuthology.orchestra.run.smithi007.stdout:osd5 (1)172.21.15.188:6802 100% (exists, up) 100% -1 2024-01-27T02:38:09.940 INFO:teuthology.orchestra.run.smithi007.stdout:osd6 (1)172.21.15.188:6825 100% (exists, up) 100% -1 2024-01-27T02:38:09.940 INFO:teuthology.orchestra.run.smithi007.stdout:osd7 (1)172.21.15.188:6817 100% (exists, up) 100% -1 2024-01-27T02:38:09.941 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records in 2024-01-27T02:38:09.941 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records out 2024-01-27T02:38:09.941 INFO:teuthology.orchestra.run.smithi007.stderr:795 bytes copied, 4.4179e-05 s, 18.0 MB/s 2024-01-27T02:38:09.942 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-01-27T02:38:09.942 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-27T02:38:09.942 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:38:09.993 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:38:10.037 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-27T02:38:10.038 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-27T02:38:10.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:38:10.038 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-27T02:38:10.120 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:38:10.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:10.120 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-27T02:38:10.145 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-27T02:38:10.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:38:10.146 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-01-27T02:38:10.203 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:38:10.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:10.204 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-27T02:38:10.222 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:38:10.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.360+0000 7f4b94709700 1 Processor -- start 2024-01-27T02:38:10.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.360+0000 7f4b94709700 1 -- start start 2024-01-27T02:38:10.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.361+0000 7f4b94709700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4b8c0c41e0 0x7f4b8c15a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:10.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.361+0000 7f4b94709700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b8c15ac80 0x7f4b8c0c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:10.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.361+0000 7f4b94709700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b8c15b6c0 con 0x7f4b8c0637d0 2024-01-27T02:38:10.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.361+0000 7f4b94709700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b8c06f2e0 con 0x7f4b8c0c41e0 2024-01-27T02:38:10.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.361+0000 7f4b94709700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c15b100 con 0x7f4b8c15ac80 2024-01-27T02:38:10.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.361+0000 7f4b924a5700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4b8c0637d0 0x7f4b8c06ebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44760/0 (socket says 172.21.15.7:44760) 2024-01-27T02:38:10.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.361+0000 7f4b924a5700 1 -- 172.21.15.7:0/3741151728 learned_addr learned my addr 172.21.15.7:0/3741151728 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:10.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.361+0000 7f4b92ca6700 1 --2- 172.21.15.7:0/3741151728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b8c15ac80 0x7f4b8c0c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:10.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.361+0000 7f4b91ca4700 1 --2- 172.21.15.7:0/3741151728 >> v2:172.21.15.188:3300/0 conn(0x7f4b8c0c41e0 0x7f4b8c15a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:10.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.361+0000 7f4b914a3700 1 -- 172.21.15.7:0/3741151728 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4014444462 0 0) 0x7f4b8c15b6c0 con 0x7f4b8c0637d0 2024-01-27T02:38:10.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.361+0000 7f4b914a3700 1 -- 172.21.15.7:0/3741151728 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b78003570 con 0x7f4b8c0637d0 2024-01-27T02:38:10.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.362+0000 7f4b92ca6700 1 -- 172.21.15.7:0/3741151728 >> v2:172.21.15.188:3300/0 conn(0x7f4b8c0c41e0 msgr2=0x7f4b8c15a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:10.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.362+0000 7f4b92ca6700 1 --2- 172.21.15.7:0/3741151728 >> v2:172.21.15.188:3300/0 conn(0x7f4b8c0c41e0 0x7f4b8c15a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:10.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.362+0000 7f4b914a3700 1 -- 172.21.15.7:0/3741151728 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3778228213 0 0) 0x7f4b78003570 con 0x7f4b8c0637d0 2024-01-27T02:38:10.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.362+0000 7f4b92ca6700 1 -- 172.21.15.7:0/3741151728 >> v1:172.21.15.7:6789/0 conn(0x7f4b8c0637d0 legacy=0x7f4b8c06ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:10.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.362+0000 7f4b92ca6700 1 -- 172.21.15.7:0/3741151728 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c15b3e0 con 0x7f4b8c15ac80 2024-01-27T02:38:10.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.362+0000 7f4b91ca4700 1 --2- 172.21.15.7:0/3741151728 >> v2:172.21.15.188:3300/0 conn(0x7f4b8c0c41e0 0x7f4b8c15a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:38:10.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.362+0000 7f4b92ca6700 1 --2- 172.21.15.7:0/3741151728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b8c15ac80 0x7f4b8c0c86b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009e60 tx=0x7f4b84009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c375c4e5c3ded6 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:10.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.362+0000 7f4b914a3700 1 -- 172.21.15.7:0/3741151728 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8400e070 con 0x7f4b8c15ac80 2024-01-27T02:38:10.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.362+0000 7f4b914a3700 1 -- 172.21.15.7:0/3741151728 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8400b7a0 con 0x7f4b8c15ac80 2024-01-27T02:38:10.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.362+0000 7f4b914a3700 1 -- 172.21.15.7:0/3741151728 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b88001470 con 0x7f4b8c15ac80 2024-01-27T02:38:10.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.363+0000 7f4b94709700 1 -- 172.21.15.7:0/3741151728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b8c15ac80 msgr2=0x7f4b8c0c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:10.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.363+0000 7f4b94709700 1 --2- 172.21.15.7:0/3741151728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b8c15ac80 0x7f4b8c0c86b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009e60 tx=0x7f4b84009780 comp rx=0 tx=0).stop 2024-01-27T02:38:10.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.363+0000 7f4b94709700 1 -- 172.21.15.7:0/3741151728 shutdown_connections 2024-01-27T02:38:10.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.363+0000 7f4b94709700 1 --2- 172.21.15.7:0/3741151728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b8c15ac80 0x7f4b8c0c86b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:10.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.363+0000 7f4b94709700 1 --2- 172.21.15.7:0/3741151728 >> v2:172.21.15.188:3300/0 conn(0x7f4b8c0c41e0 0x7f4b8c15a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:10.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.363+0000 7f4b94709700 1 -- 172.21.15.7:0/3741151728 >> 172.21.15.7:0/3741151728 conn(0x7f4b8c0cdc30 msgr2=0x7f4b8c0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:10.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.363+0000 7f4b94709700 1 -- 172.21.15.7:0/3741151728 shutdown_connections 2024-01-27T02:38:10.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.363+0000 7f4b94709700 1 -- 172.21.15.7:0/3741151728 wait complete. 2024-01-27T02:38:10.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.364+0000 7f4b94709700 1 Processor -- start 2024-01-27T02:38:10.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.364+0000 7f4b94709700 1 -- start start 2024-01-27T02:38:10.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.365+0000 7f4b94709700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b8c0637d0 0x7f4b8c16b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:10.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.365+0000 7f4b94709700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b8c0c41e0 0x7f4b8c0cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:10.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.365+0000 7f4b94709700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b8c16dfe0 0x7f4b8c16e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:10.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.365+0000 7f4b94709700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c16afd0 con 0x7f4b8c16dfe0 2024-01-27T02:38:10.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.365+0000 7f4b94709700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c16a9e0 con 0x7f4b8c0637d0 2024-01-27T02:38:10.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.365+0000 7f4b94709700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c16b9a0 con 0x7f4b8c0c41e0 2024-01-27T02:38:10.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.366+0000 7f4b92ca6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b8c16dfe0 0x7f4b8c16e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:10.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.366+0000 7f4b92ca6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b8c16dfe0 0x7f4b8c16e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40302/0 (socket says 172.21.15.7:40302) 2024-01-27T02:38:10.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.366+0000 7f4b92ca6700 1 -- 172.21.15.7:0/606263377 learned_addr learned my addr 172.21.15.7:0/606263377 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:10.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.366+0000 7f4b91ca4700 1 --2- 172.21.15.7:0/606263377 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b8c0c41e0 0x7f4b8c0cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:10.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.366+0000 7f4b924a5700 1 --2- 172.21.15.7:0/606263377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b8c0637d0 0x7f4b8c16b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:10.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.366+0000 7f4b92ca6700 1 -- 172.21.15.7:0/606263377 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b8c0c41e0 msgr2=0x7f4b8c0cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:10.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.366+0000 7f4b92ca6700 1 --2- 172.21.15.7:0/606263377 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b8c0c41e0 0x7f4b8c0cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:10.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.366+0000 7f4b92ca6700 1 -- 172.21.15.7:0/606263377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b8c0637d0 msgr2=0x7f4b8c16b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:10.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.366+0000 7f4b92ca6700 1 --2- 172.21.15.7:0/606263377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b8c0637d0 0x7f4b8c16b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:10.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.366+0000 7f4b92ca6700 1 -- 172.21.15.7:0/606263377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7c009420 con 0x7f4b8c16dfe0 2024-01-27T02:38:10.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.366+0000 7f4b92ca6700 1 --2- 172.21.15.7:0/606263377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b8c16dfe0 0x7f4b8c16e430 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f4b84003000 tx=0x7f4b84018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:10.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.366+0000 7f4b827fc700 1 -- 172.21.15.7:0/606263377 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b84012bc0 con 0x7f4b8c16dfe0 2024-01-27T02:38:10.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.366+0000 7f4b827fc700 1 -- 172.21.15.7:0/606263377 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b84012d60 con 0x7f4b8c16dfe0 2024-01-27T02:38:10.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.366+0000 7f4b827fc700 1 -- 172.21.15.7:0/606263377 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8401d680 con 0x7f4b8c16dfe0 2024-01-27T02:38:10.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.366+0000 7f4b94709700 1 -- 172.21.15.7:0/606263377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b84009a20 con 0x7f4b8c16dfe0 2024-01-27T02:38:10.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.367+0000 7f4b94709700 1 -- 172.21.15.7:0/606263377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8c16ecf0 con 0x7f4b8c16dfe0 2024-01-27T02:38:10.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.367+0000 7f4b94709700 1 -- 172.21.15.7:0/606263377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8c0c8070 con 0x7f4b8c16dfe0 2024-01-27T02:38:10.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.368+0000 7f4b827fc700 1 -- 172.21.15.7:0/606263377 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4b84024070 con 0x7f4b8c16dfe0 2024-01-27T02:38:10.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.369+0000 7f4b827fc700 1 --2- 172.21.15.7:0/606263377 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4b70066f30 0x7f4b700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:10.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.369+0000 7f4b827fc700 1 -- 172.21.15.7:0/606263377 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4b840869d0 con 0x7f4b8c16dfe0 2024-01-27T02:38:10.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.369+0000 7f4b924a5700 1 --2- 172.21.15.7:0/606263377 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4b70066f30 0x7f4b700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:10.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.369+0000 7f4b924a5700 1 --2- 172.21.15.7:0/606263377 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4b70066f30 0x7f4b700693e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c1561c0 tx=0x7f4b88001a00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:10.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.373+0000 7f4b827fc700 1 -- 172.21.15.7:0/606263377 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b84053c20 con 0x7f4b8c16dfe0 2024-01-27T02:38:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.623+0000 7f4b94709700 1 -- 172.21.15.7:0/606263377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b8c04c710 con 0x7f4b8c16dfe0 2024-01-27T02:38:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.624+0000 7f4b827fc700 1 -- 172.21.15.7:0/606263377 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f4b84057240 con 0x7f4b8c16dfe0 2024-01-27T02:38:10.625 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:10.625 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2767759975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2767759975},{"type":"v1","addr":"172.21.15.188:6836","nonce":2767759975}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":5768,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/245473575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":245473575},{"type":"v1","addr":"172.21.15.7:6834","nonce":245473575}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1334164270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1334164270},{"type":"v1","addr":"172.21.15.188:6837","nonce":1334164270}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:37:51.231918+0000","modified":"2024-01-27T02:37:54.851112+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6657},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6657":{"gid":6657,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1251886476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1251886476},{"type":"v1","addr":"172.21.15.7:6835","nonce":1251886476}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2024-01-27T02:38:10.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.626+0000 7f4b94709700 1 -- 172.21.15.7:0/606263377 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4b70066f30 msgr2=0x7f4b700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:10.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.626+0000 7f4b94709700 1 --2- 172.21.15.7:0/606263377 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4b70066f30 0x7f4b700693e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c1561c0 tx=0x7f4b88001a00 comp rx=0 tx=0).stop 2024-01-27T02:38:10.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.626+0000 7f4b94709700 1 -- 172.21.15.7:0/606263377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b8c16dfe0 msgr2=0x7f4b8c16e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.626+0000 7f4b94709700 1 --2- 172.21.15.7:0/606263377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b8c16dfe0 0x7f4b8c16e430 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f4b84003000 tx=0x7f4b84018000 comp rx=0 tx=0).stop 2024-01-27T02:38:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.626+0000 7f4b94709700 1 -- 172.21.15.7:0/606263377 shutdown_connections 2024-01-27T02:38:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.626+0000 7f4b94709700 1 --2- 172.21.15.7:0/606263377 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4b70066f30 0x7f4b700693e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.626+0000 7f4b94709700 1 --2- 172.21.15.7:0/606263377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b8c16dfe0 0x7f4b8c16e430 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.626+0000 7f4b94709700 1 --2- 172.21.15.7:0/606263377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b8c0637d0 0x7f4b8c16b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.626+0000 7f4b94709700 1 --2- 172.21.15.7:0/606263377 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b8c0c41e0 0x7f4b8c0cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.626+0000 7f4b94709700 1 -- 172.21.15.7:0/606263377 >> 172.21.15.7:0/606263377 conn(0x7f4b8c0cdc30 msgr2=0x7f4b8c15a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.627+0000 7f4b94709700 1 -- 172.21.15.7:0/606263377 shutdown_connections 2024-01-27T02:38:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.627+0000 7f4b94709700 1 -- 172.21.15.7:0/606263377 wait complete. 2024-01-27T02:38:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 58 2024-01-27T02:38:10.641 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:38:10.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.771+0000 7f74c0733700 1 Processor -- start 2024-01-27T02:38:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.771+0000 7f74c0733700 1 -- start start 2024-01-27T02:38:10.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.772+0000 7f74c0733700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f74b8159f40 0x7f74b80c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.772+0000 7f74c0733700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b80c30c0 con 0x7f74b80c3a90 2024-01-27T02:38:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.772+0000 7f74c0733700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b815e960 con 0x7f74b80aafc0 2024-01-27T02:38:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.772+0000 7f74c0733700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b80c4550 con 0x7f74b8159f40 2024-01-27T02:38:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.772+0000 7f74be4cf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f74b8159f40 0x7f74b80c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.772+0000 7f74be4cf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f74b8159f40 0x7f74b80c3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40310/0 (socket says 172.21.15.7:40310) 2024-01-27T02:38:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.772+0000 7f74be4cf700 1 -- 172.21.15.7:0/1318506349 learned_addr learned my addr 172.21.15.7:0/1318506349 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.772+0000 7f74bdcce700 1 --1- 172.21.15.7:0/1318506349 >> v1:172.21.15.7:6789/0 conn(0x7f74b80c3a90 0x7f74b80aa8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44766/0 (socket says 172.21.15.7:44766) 2024-01-27T02:38:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.773+0000 7f74be4cf700 1 -- 172.21.15.7:0/1318506349 >> v1:172.21.15.188:6789/0 conn(0x7f74b80aafc0 legacy=0x7f74b8063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.773+0000 7f74be4cf700 1 -- 172.21.15.7:0/1318506349 >> v1:172.21.15.7:6789/0 conn(0x7f74b80c3a90 legacy=0x7f74b80aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.773+0000 7f74bd4cd700 1 -- 172.21.15.7:0/1318506349 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2023583310 0 0) 0x7f74b80c30c0 con 0x7f74b80c3a90 2024-01-27T02:38:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.773+0000 7f74be4cf700 1 -- 172.21.15.7:0/1318506349 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b815ec40 con 0x7f74b8159f40 2024-01-27T02:38:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.773+0000 7f74be4cf700 1 --2- 172.21.15.7:0/1318506349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b8159f40 0x7f74b80c3550 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f74b40099b0 tx=0x7f74b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4063a9a11c9db42f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:10.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.773+0000 7f74bd4cd700 1 -- 172.21.15.7:0/1318506349 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b400b860 con 0x7f74b8159f40 2024-01-27T02:38:10.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.773+0000 7f74bd4cd700 1 -- 172.21.15.7:0/1318506349 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b400b9c0 con 0x7f74b8159f40 2024-01-27T02:38:10.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.773+0000 7f74bd4cd700 1 -- 172.21.15.7:0/1318506349 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b400bd60 con 0x7f74b8159f40 2024-01-27T02:38:10.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.774+0000 7f74c0733700 1 -- 172.21.15.7:0/1318506349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b8159f40 msgr2=0x7f74b80c3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:10.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.774+0000 7f74c0733700 1 --2- 172.21.15.7:0/1318506349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b8159f40 0x7f74b80c3550 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f74b40099b0 tx=0x7f74b4009250 comp rx=0 tx=0).stop 2024-01-27T02:38:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.774+0000 7f74c0733700 1 -- 172.21.15.7:0/1318506349 shutdown_connections 2024-01-27T02:38:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.774+0000 7f74c0733700 1 --2- 172.21.15.7:0/1318506349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b8159f40 0x7f74b80c3550 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.774+0000 7f74c0733700 1 -- 172.21.15.7:0/1318506349 >> 172.21.15.7:0/1318506349 conn(0x7f74b80cdc30 msgr2=0x7f74b80ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.774+0000 7f74c0733700 1 -- 172.21.15.7:0/1318506349 shutdown_connections 2024-01-27T02:38:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.774+0000 7f74c0733700 1 -- 172.21.15.7:0/1318506349 wait complete. 2024-01-27T02:38:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.775+0000 7f74c0733700 1 Processor -- start 2024-01-27T02:38:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.775+0000 7f74c0733700 1 -- start start 2024-01-27T02:38:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.776+0000 7f74c0733700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74b80aafc0 0x7f74b8150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.776+0000 7f74c0733700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74b80c3a90 0x7f74b816afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.776+0000 7f74c0733700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b8159f40 0x7f74b815a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.776+0000 7f74c0733700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b816a5c0 con 0x7f74b8159f40 2024-01-27T02:38:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.776+0000 7f74c0733700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b8150ca0 con 0x7f74b80c3a90 2024-01-27T02:38:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.776+0000 7f74c0733700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b815ab60 con 0x7f74b80aafc0 2024-01-27T02:38:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.776+0000 7f74becd0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b8159f40 0x7f74b815a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.776+0000 7f74becd0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b8159f40 0x7f74b815a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40314/0 (socket says 172.21.15.7:40314) 2024-01-27T02:38:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.776+0000 7f74becd0700 1 -- 172.21.15.7:0/731612006 learned_addr learned my addr 172.21.15.7:0/731612006 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.776+0000 7f74be4cf700 1 --2- 172.21.15.7:0/731612006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74b80aafc0 0x7f74b8150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.776+0000 7f74bdcce700 1 --2- 172.21.15.7:0/731612006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74b80c3a90 0x7f74b816afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.776+0000 7f74becd0700 1 -- 172.21.15.7:0/731612006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74b80aafc0 msgr2=0x7f74b8150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.776+0000 7f74becd0700 1 --2- 172.21.15.7:0/731612006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74b80aafc0 0x7f74b8150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.776+0000 7f74becd0700 1 -- 172.21.15.7:0/731612006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74b80c3a90 msgr2=0x7f74b816afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.776+0000 7f74becd0700 1 --2- 172.21.15.7:0/731612006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74b80c3a90 0x7f74b816afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.776+0000 7f74becd0700 1 -- 172.21.15.7:0/731612006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b40094f0 con 0x7f74b8159f40 2024-01-27T02:38:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.776+0000 7f74becd0700 1 --2- 172.21.15.7:0/731612006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b8159f40 0x7f74b815a310 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f74b000c840 tx=0x7f74b0001610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.776+0000 7f74ae7fc700 1 -- 172.21.15.7:0/731612006 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0012630 con 0x7f74b8159f40 2024-01-27T02:38:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.777+0000 7f74ae7fc700 1 -- 172.21.15.7:0/731612006 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b00127b0 con 0x7f74b8159f40 2024-01-27T02:38:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.777+0000 7f74ae7fc700 1 -- 172.21.15.7:0/731612006 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0012b70 con 0x7f74b8159f40 2024-01-27T02:38:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.777+0000 7f74c0733700 1 -- 172.21.15.7:0/731612006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b815ae40 con 0x7f74b8159f40 2024-01-27T02:38:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.777+0000 7f74c0733700 1 -- 172.21.15.7:0/731612006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b816aab0 con 0x7f74b8159f40 2024-01-27T02:38:10.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.777+0000 7f74c0733700 1 -- 172.21.15.7:0/731612006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b815b080 con 0x7f74b8159f40 2024-01-27T02:38:10.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.778+0000 7f74ae7fc700 1 -- 172.21.15.7:0/731612006 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f74b0003030 con 0x7f74b8159f40 2024-01-27T02:38:10.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.778+0000 7f74ae7fc700 1 --2- 172.21.15.7:0/731612006 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f749c066f30 0x7f749c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:10.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.778+0000 7f74ae7fc700 1 -- 172.21.15.7:0/731612006 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f74b00843a0 con 0x7f74b8159f40 2024-01-27T02:38:10.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.779+0000 7f74be4cf700 1 --2- 172.21.15.7:0/731612006 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f749c066f30 0x7f749c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:10.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.779+0000 7f74be4cf700 1 --2- 172.21.15.7:0/731612006 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f749c066f30 0x7f749c0693e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f74b40099b0 tx=0x7f74b4017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:10.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.783+0000 7f74ae7fc700 1 -- 172.21.15.7:0/731612006 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f74b815b080 con 0x7f74b8159f40 2024-01-27T02:38:10.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.988+0000 7f74c0733700 1 -- 172.21.15.7:0/731612006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74b815b080 con 0x7f74b8159f40 2024-01-27T02:38:10.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.989+0000 7f74ae7fc700 1 -- 172.21.15.7:0/731612006 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+15324 (secure 0 0 0) 0x7f74b815b080 con 0x7f74b8159f40 2024-01-27T02:38:10.990 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:10.990 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":120,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:38:07.896480+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:37:49.747694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-27T02:37:50.199870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:38:10.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.991+0000 7f74c0733700 1 -- 172.21.15.7:0/731612006 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f749c066f30 msgr2=0x7f749c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:10.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.991+0000 7f74c0733700 1 --2- 172.21.15.7:0/731612006 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f749c066f30 0x7f749c0693e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f74b40099b0 tx=0x7f74b4017000 comp rx=0 tx=0).stop 2024-01-27T02:38:10.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.991+0000 7f74c0733700 1 -- 172.21.15.7:0/731612006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b8159f40 msgr2=0x7f74b815a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:10.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.991+0000 7f74c0733700 1 --2- 172.21.15.7:0/731612006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b8159f40 0x7f74b815a310 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f74b000c840 tx=0x7f74b0001610 comp rx=0 tx=0).stop 2024-01-27T02:38:10.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.991+0000 7f74c0733700 1 -- 172.21.15.7:0/731612006 shutdown_connections 2024-01-27T02:38:10.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.991+0000 7f74c0733700 1 --2- 172.21.15.7:0/731612006 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f749c066f30 0x7f749c0693e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:10.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.991+0000 7f74c0733700 1 --2- 172.21.15.7:0/731612006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b8159f40 0x7f74b815a310 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:10.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.991+0000 7f74c0733700 1 --2- 172.21.15.7:0/731612006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74b80aafc0 0x7f74b8150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:10.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.991+0000 7f74c0733700 1 --2- 172.21.15.7:0/731612006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74b80c3a90 0x7f74b816afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:10.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.991+0000 7f74c0733700 1 -- 172.21.15.7:0/731612006 >> 172.21.15.7:0/731612006 conn(0x7f74b80cdc30 msgr2=0x7f74b81583c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:10.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.991+0000 7f74c0733700 1 -- 172.21.15.7:0/731612006 shutdown_connections 2024-01-27T02:38:10.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:10.991+0000 7f74c0733700 1 -- 172.21.15.7:0/731612006 wait complete. 2024-01-27T02:38:11.005 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-27T02:37:49.747694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-01-27T02:37:50.199870+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-27T02:38:11.006 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:38:11.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.140+0000 7f7222887700 1 Processor -- start 2024-01-27T02:38:11.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.140+0000 7f7222887700 1 -- start start 2024-01-27T02:38:11.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.141+0000 7f7222887700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f721c15d460 0x7f721c0d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:11.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.141+0000 7f7222887700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f721c15d830 0x7f721c0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:11.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.141+0000 7f7222887700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f721c0b38d0 con 0x7f721c0cf280 2024-01-27T02:38:11.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.141+0000 7f7222887700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f721c0d0390 con 0x7f721c15d830 2024-01-27T02:38:11.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.141+0000 7f7222887700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0cf780 con 0x7f721c15d460 2024-01-27T02:38:11.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.141+0000 7f721b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f721c0cf280 0x7f721c15e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44782/0 (socket says 172.21.15.7:44782) 2024-01-27T02:38:11.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.141+0000 7f721b7fe700 1 -- 172.21.15.7:0/2365311892 learned_addr learned my addr 172.21.15.7:0/2365311892 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:11.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.141+0000 7f721bfff700 1 --2- 172.21.15.7:0/2365311892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f721c15d460 0x7f721c0d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:11.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.141+0000 7f7220e24700 1 --2- 172.21.15.7:0/2365311892 >> v2:172.21.15.188:3300/0 conn(0x7f721c15d830 0x7f721c0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.141+0000 7f721affd700 1 -- 172.21.15.7:0/2365311892 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1814164633 0 0) 0x7f7210001110 con 0x7f721c0cf280 2024-01-27T02:38:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.141+0000 7f721affd700 1 -- 172.21.15.7:0/2365311892 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7204003570 con 0x7f721c0cf280 2024-01-27T02:38:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.141+0000 7f721affd700 1 -- 172.21.15.7:0/2365311892 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2060307785 0 0) 0x7f7204003570 con 0x7f721c0cf280 2024-01-27T02:38:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.141+0000 7f721affd700 1 -- 172.21.15.7:0/2365311892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f721c15d460 msgr2=0x7f721c0d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.141+0000 7f721affd700 1 --2- 172.21.15.7:0/2365311892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f721c15d460 0x7f721c0d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.141+0000 7f721affd700 1 -- 172.21.15.7:0/2365311892 >> v2:172.21.15.188:3300/0 conn(0x7f721c15d830 msgr2=0x7f721c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.141+0000 7f721affd700 1 --2- 172.21.15.7:0/2365311892 >> v2:172.21.15.188:3300/0 conn(0x7f721c15d830 0x7f721c0cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.141+0000 7f721affd700 1 -- 172.21.15.7:0/2365311892 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c0cfa60 con 0x7f721c0cf280 2024-01-27T02:38:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.142+0000 7f721affd700 1 -- 172.21.15.7:0/2365311892 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f7210001600 con 0x7f721c0cf280 2024-01-27T02:38:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.142+0000 7f721affd700 1 -- 172.21.15.7:0/2365311892 >> v1:172.21.15.7:6789/0 conn(0x7f721c0cf280 legacy=0x7f721c15e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.142+0000 7f721affd700 1 --2- 172.21.15.7:0/2365311892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f721c15d460 0x7f7204003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.142+0000 7f721affd700 1 --2- 172.21.15.7:0/2365311892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7204003fe0 0x7f7204004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.142+0000 7f721affd700 1 --2- 172.21.15.7:0/2365311892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7204004d50 0x7f72040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.142+0000 7f721affd700 1 -- 172.21.15.7:0/2365311892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0d0390 con 0x7f7204004d50 2024-01-27T02:38:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.142+0000 7f721affd700 1 -- 172.21.15.7:0/2365311892 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0cf780 con 0x7f7204003fe0 2024-01-27T02:38:11.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.142+0000 7f721affd700 1 -- 172.21.15.7:0/2365311892 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720400ba90 con 0x7f721c15d460 2024-01-27T02:38:11.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.142+0000 7f7220e24700 1 --2- 172.21.15.7:0/2365311892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7204004d50 0x7f72040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:11.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.142+0000 7f721bfff700 1 --2- 172.21.15.7:0/2365311892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f721c15d460 0x7f7204003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:11.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.142+0000 7f721b7fe700 1 --2- 172.21.15.7:0/2365311892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7204003fe0 0x7f7204004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:11.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.142+0000 7f7220e24700 1 -- 172.21.15.7:0/2365311892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f721c15d460 msgr2=0x7f7204003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:11.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.142+0000 7f7220e24700 1 --2- 172.21.15.7:0/2365311892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f721c15d460 0x7f7204003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.142+0000 7f7220e24700 1 -- 172.21.15.7:0/2365311892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7204003fe0 msgr2=0x7f7204004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:11.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.142+0000 7f7220e24700 1 --2- 172.21.15.7:0/2365311892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7204003fe0 0x7f7204004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.142+0000 7f7220e24700 1 -- 172.21.15.7:0/2365311892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7204003570 con 0x7f7204004d50 2024-01-27T02:38:11.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.142+0000 7f7220e24700 1 --2- 172.21.15.7:0/2365311892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7204004d50 0x7f72040051c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f721400e210 tx=0x7f721400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870f703ff0e98895 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:11.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.142+0000 7f721affd700 1 -- 172.21.15.7:0/2365311892 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721400ea60 con 0x7f7204004d50 2024-01-27T02:38:11.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.142+0000 7f721affd700 1 -- 172.21.15.7:0/2365311892 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721400f040 con 0x7f7204004d50 2024-01-27T02:38:11.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.142+0000 7f721affd700 1 -- 172.21.15.7:0/2365311892 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721400ee10 con 0x7f7204004d50 2024-01-27T02:38:11.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.143+0000 7f7222887700 1 -- 172.21.15.7:0/2365311892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7204004d50 msgr2=0x7f72040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:11.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.143+0000 7f7222887700 1 --2- 172.21.15.7:0/2365311892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7204004d50 0x7f72040051c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f721400e210 tx=0x7f721400b040 comp rx=0 tx=0).stop 2024-01-27T02:38:11.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.143+0000 7f7220e24700 1 -- 172.21.15.7:0/2365311892 reap_dead start 2024-01-27T02:38:11.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.143+0000 7f7222887700 1 -- 172.21.15.7:0/2365311892 shutdown_connections 2024-01-27T02:38:11.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.143+0000 7f7222887700 1 -- 172.21.15.7:0/2365311892 >> 172.21.15.7:0/2365311892 conn(0x7f721c063580 msgr2=0x7f721c063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:11.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.143+0000 7f7222887700 1 -- 172.21.15.7:0/2365311892 shutdown_connections 2024-01-27T02:38:11.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.143+0000 7f7222887700 1 -- 172.21.15.7:0/2365311892 wait complete. 2024-01-27T02:38:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.144+0000 7f7222887700 1 Processor -- start 2024-01-27T02:38:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.145+0000 7f7222887700 1 -- start start 2024-01-27T02:38:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.145+0000 7f7222887700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7204004d50 0x7f721c0c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.145+0000 7f7222887700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f721c15f810 0x7f721c149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.145+0000 7f7222887700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f721c0b4ce0 0x7f721c0b5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.145+0000 7f7222887700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0c8d10 con 0x7f7204004d50 2024-01-27T02:38:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.145+0000 7f7222887700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0b5680 con 0x7f721c0b4ce0 2024-01-27T02:38:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.145+0000 7f7222887700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0c8fb0 con 0x7f721c15f810 2024-01-27T02:38:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.145+0000 7f721bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7204004d50 0x7f721c0c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.145+0000 7f721bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7204004d50 0x7f721c0c6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40324/0 (socket says 172.21.15.7:40324) 2024-01-27T02:38:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.145+0000 7f721bfff700 1 -- 172.21.15.7:0/4251132163 learned_addr learned my addr 172.21.15.7:0/4251132163 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.145+0000 7f721b7fe700 1 --2- 172.21.15.7:0/4251132163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f721c15f810 0x7f721c149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.145+0000 7f7220e24700 1 --2- 172.21.15.7:0/4251132163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f721c0b4ce0 0x7f721c0b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.146+0000 7f721bfff700 1 -- 172.21.15.7:0/4251132163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f721c15f810 msgr2=0x7f721c149930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.146+0000 7f721bfff700 1 --2- 172.21.15.7:0/4251132163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f721c15f810 0x7f721c149930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.146+0000 7f721bfff700 1 -- 172.21.15.7:0/4251132163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f721c0b4ce0 msgr2=0x7f721c0b5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.146+0000 7f721bfff700 1 --2- 172.21.15.7:0/4251132163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f721c0b4ce0 0x7f721c0b5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.146+0000 7f721bfff700 1 -- 172.21.15.7:0/4251132163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c160880 con 0x7f7204004d50 2024-01-27T02:38:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.146+0000 7f721bfff700 1 --2- 172.21.15.7:0/4251132163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7204004d50 0x7f721c0c6780 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f720c006ac0 tx=0x7f720c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.146+0000 7f7203fff700 1 -- 172.21.15.7:0/4251132163 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c00f8e0 con 0x7f7204004d50 2024-01-27T02:38:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.146+0000 7f7203fff700 1 -- 172.21.15.7:0/4251132163 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720c00fa40 con 0x7f7204004d50 2024-01-27T02:38:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.146+0000 7f7222887700 1 -- 172.21.15.7:0/4251132163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f721c15ffe0 con 0x7f7204004d50 2024-01-27T02:38:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.146+0000 7f7203fff700 1 -- 172.21.15.7:0/4251132163 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c00fdc0 con 0x7f7204004d50 2024-01-27T02:38:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.146+0000 7f7222887700 1 -- 172.21.15.7:0/4251132163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721c1605a0 con 0x7f7204004d50 2024-01-27T02:38:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.147+0000 7f7222887700 1 -- 172.21.15.7:0/4251132163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721c149ed0 con 0x7f7204004d50 2024-01-27T02:38:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.148+0000 7f7203fff700 1 -- 172.21.15.7:0/4251132163 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f720c014650 con 0x7f7204004d50 2024-01-27T02:38:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.148+0000 7f7203fff700 1 --2- 172.21.15.7:0/4251132163 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7208066f30 0x7f72080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.148+0000 7f7203fff700 1 -- 172.21.15.7:0/4251132163 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f720c018030 con 0x7f7204004d50 2024-01-27T02:38:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.148+0000 7f721b7fe700 1 --2- 172.21.15.7:0/4251132163 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7208066f30 0x7f72080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.149+0000 7f721b7fe700 1 --2- 172.21.15.7:0/4251132163 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7208066f30 0x7f72080693e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7210001270 tx=0x7f721c0d04d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:11.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.152+0000 7f7203fff700 1 -- 172.21.15.7:0/4251132163 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f720c052180 con 0x7f7204004d50 2024-01-27T02:38:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.353+0000 7f7222887700 1 -- 172.21.15.7:0/4251132163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f721c072060 con 0x7f7204004d50 2024-01-27T02:38:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.354+0000 7f7203fff700 1 -- 172.21.15.7:0/4251132163 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) v1 ==== 93+0+10 (secure 0 0 0) 0x7f720c0220a0 con 0x7f7204004d50 2024-01-27T02:38:11.354 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:38:11.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.356+0000 7f7222887700 1 -- 172.21.15.7:0/4251132163 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7208066f30 msgr2=0x7f72080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:11.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.356+0000 7f7222887700 1 --2- 172.21.15.7:0/4251132163 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7208066f30 0x7f72080693e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7210001270 tx=0x7f721c0d04d0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.356+0000 7f7222887700 1 -- 172.21.15.7:0/4251132163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7204004d50 msgr2=0x7f721c0c6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:11.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.356+0000 7f7222887700 1 --2- 172.21.15.7:0/4251132163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7204004d50 0x7f721c0c6780 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f720c006ac0 tx=0x7f720c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.356+0000 7f7222887700 1 -- 172.21.15.7:0/4251132163 shutdown_connections 2024-01-27T02:38:11.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.356+0000 7f7222887700 1 --2- 172.21.15.7:0/4251132163 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7208066f30 0x7f72080693e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.356+0000 7f7222887700 1 --2- 172.21.15.7:0/4251132163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f721c0b4ce0 0x7f721c0b5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.356+0000 7f7222887700 1 --2- 172.21.15.7:0/4251132163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7204004d50 0x7f721c0c6780 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.356+0000 7f7222887700 1 --2- 172.21.15.7:0/4251132163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f721c15f810 0x7f721c149930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.356+0000 7f7222887700 1 -- 172.21.15.7:0/4251132163 >> 172.21.15.7:0/4251132163 conn(0x7f721c063580 msgr2=0x7f721c15c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:11.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.356+0000 7f7222887700 1 -- 172.21.15.7:0/4251132163 shutdown_connections 2024-01-27T02:38:11.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.356+0000 7f7222887700 1 -- 172.21.15.7:0/4251132163 wait complete. 2024-01-27T02:38:11.370 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-27T02:38:11.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.504+0000 7f8e22413700 1 Processor -- start 2024-01-27T02:38:11.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.504+0000 7f8e22413700 1 -- start start 2024-01-27T02:38:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.505+0000 7f8e22413700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8e1c0cf280 0x7f8e1c0d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.505+0000 7f8e22413700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e1c0b38d0 con 0x7f8e1c15d830 2024-01-27T02:38:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.505+0000 7f8e22413700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e1c0d0390 con 0x7f8e1c15d460 2024-01-27T02:38:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.505+0000 7f8e22413700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e1c0cf780 con 0x7f8e1c0cf280 2024-01-27T02:38:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.505+0000 7f8e209b0700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8e1c15d830 0x7f8e1c0cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42164/0 (socket says 172.21.15.7:42164) 2024-01-27T02:38:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.505+0000 7f8e209b0700 1 -- 172.21.15.7:0/2340209993 learned_addr learned my addr 172.21.15.7:0/2340209993 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.505+0000 7f8e1b7fe700 1 --2- 172.21.15.7:0/2340209993 >> v2:172.21.15.188:3300/0 conn(0x7f8e1c0cf280 0x7f8e1c0d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.505+0000 7f8e1affd700 1 -- 172.21.15.7:0/2340209993 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1384462530 0 0) 0x7f8e14001490 con 0x7f8e1c15d830 2024-01-27T02:38:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.505+0000 7f8e1affd700 1 -- 172.21.15.7:0/2340209993 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e04003570 con 0x7f8e1c15d830 2024-01-27T02:38:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.505+0000 7f8e1affd700 1 -- 172.21.15.7:0/2340209993 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359030764 0 0) 0x7f8e1c0d0390 con 0x7f8e1c15d460 2024-01-27T02:38:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.505+0000 7f8e1affd700 1 -- 172.21.15.7:0/2340209993 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e14001490 con 0x7f8e1c15d460 2024-01-27T02:38:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.505+0000 7f8e1b7fe700 1 -- 172.21.15.7:0/2340209993 >> v1:172.21.15.188:6789/0 conn(0x7f8e1c15d460 legacy=0x7f8e1c15e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.505+0000 7f8e1b7fe700 1 -- 172.21.15.7:0/2340209993 >> v1:172.21.15.7:6789/0 conn(0x7f8e1c15d830 legacy=0x7f8e1c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.505+0000 7f8e1b7fe700 1 -- 172.21.15.7:0/2340209993 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c0cfa60 con 0x7f8e1c0cf280 2024-01-27T02:38:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.506+0000 7f8e1b7fe700 1 --2- 172.21.15.7:0/2340209993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e1c0cf280 0x7f8e1c0d0820 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8e100098e0 tx=0x7f8e100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44a763a5006189b7 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.506+0000 7f8e1affd700 1 -- 172.21.15.7:0/2340209993 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e10011630 con 0x7f8e1c0cf280 2024-01-27T02:38:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.506+0000 7f8e1affd700 1 -- 172.21.15.7:0/2340209993 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e10011790 con 0x7f8e1c0cf280 2024-01-27T02:38:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.506+0000 7f8e1affd700 1 -- 172.21.15.7:0/2340209993 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e10011b30 con 0x7f8e1c0cf280 2024-01-27T02:38:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.506+0000 7f8e22413700 1 -- 172.21.15.7:0/2340209993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e1c0cf280 msgr2=0x7f8e1c0d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.506+0000 7f8e22413700 1 --2- 172.21.15.7:0/2340209993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e1c0cf280 0x7f8e1c0d0820 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8e100098e0 tx=0x7f8e100091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.506+0000 7f8e22413700 1 -- 172.21.15.7:0/2340209993 shutdown_connections 2024-01-27T02:38:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.506+0000 7f8e22413700 1 --2- 172.21.15.7:0/2340209993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e1c0cf280 0x7f8e1c0d0820 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.506+0000 7f8e22413700 1 -- 172.21.15.7:0/2340209993 >> 172.21.15.7:0/2340209993 conn(0x7f8e1c063580 msgr2=0x7f8e1c063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.506+0000 7f8e22413700 1 -- 172.21.15.7:0/2340209993 shutdown_connections 2024-01-27T02:38:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.506+0000 7f8e22413700 1 -- 172.21.15.7:0/2340209993 wait complete. 2024-01-27T02:38:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.508+0000 7f8e22413700 1 Processor -- start 2024-01-27T02:38:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.508+0000 7f8e22413700 1 -- start start 2024-01-27T02:38:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.509+0000 7f8e22413700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e1c0cf280 0x7f8e1c149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.509+0000 7f8e22413700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e1c15d460 0x7f8e1c0ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.509+0000 7f8e22413700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e1c15d830 0x7f8e1c0b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.509+0000 7f8e22413700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c16ede0 con 0x7f8e1c15d830 2024-01-27T02:38:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.509+0000 7f8e22413700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c14a4c0 con 0x7f8e1c0cf280 2024-01-27T02:38:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.509+0000 7f8e22413700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c0cae10 con 0x7f8e1c15d460 2024-01-27T02:38:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.509+0000 7f8e209b0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e1c15d830 0x7f8e1c0b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.509+0000 7f8e209b0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e1c15d830 0x7f8e1c0b1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53878/0 (socket says 172.21.15.7:53878) 2024-01-27T02:38:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.509+0000 7f8e209b0700 1 -- 172.21.15.7:0/973117001 learned_addr learned my addr 172.21.15.7:0/973117001 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.509+0000 7f8e1bfff700 1 --2- 172.21.15.7:0/973117001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e1c0cf280 0x7f8e1c149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:11.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.509+0000 7f8e1b7fe700 1 --2- 172.21.15.7:0/973117001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e1c15d460 0x7f8e1c0ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:11.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.509+0000 7f8e209b0700 1 -- 172.21.15.7:0/973117001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e1c15d460 msgr2=0x7f8e1c0ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:11.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.509+0000 7f8e209b0700 1 --2- 172.21.15.7:0/973117001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e1c15d460 0x7f8e1c0ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.509+0000 7f8e209b0700 1 -- 172.21.15.7:0/973117001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e1c0cf280 msgr2=0x7f8e1c149f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:11.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.509+0000 7f8e209b0700 1 --2- 172.21.15.7:0/973117001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e1c0cf280 0x7f8e1c149f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.509+0000 7f8e209b0700 1 -- 172.21.15.7:0/973117001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e10009420 con 0x7f8e1c15d830 2024-01-27T02:38:11.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.509+0000 7f8e1b7fe700 1 --2- 172.21.15.7:0/973117001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e1c15d460 0x7f8e1c0ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:11.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.509+0000 7f8e209b0700 1 --2- 172.21.15.7:0/973117001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e1c15d830 0x7f8e1c0b1f30 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f8e140020a0 tx=0x7f8e14001c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:11.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.510+0000 7f8e03fff700 1 -- 172.21.15.7:0/973117001 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1400aad0 con 0x7f8e1c15d830 2024-01-27T02:38:11.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.510+0000 7f8e03fff700 1 -- 172.21.15.7:0/973117001 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1400ac50 con 0x7f8e1c15d830 2024-01-27T02:38:11.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.510+0000 7f8e03fff700 1 -- 172.21.15.7:0/973117001 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e140070f0 con 0x7f8e1c15d830 2024-01-27T02:38:11.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.510+0000 7f8e22413700 1 -- 172.21.15.7:0/973117001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e1c0c9c80 con 0x7f8e1c15d830 2024-01-27T02:38:11.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.510+0000 7f8e22413700 1 -- 172.21.15.7:0/973117001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1c0b4700 con 0x7f8e1c15d830 2024-01-27T02:38:11.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.511+0000 7f8e22413700 1 -- 172.21.15.7:0/973117001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e1c072060 con 0x7f8e1c15d830 2024-01-27T02:38:11.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.511+0000 7f8e03fff700 1 -- 172.21.15.7:0/973117001 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8e140074a0 con 0x7f8e1c15d830 2024-01-27T02:38:11.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.511+0000 7f8e03fff700 1 --2- 172.21.15.7:0/973117001 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8e08066f30 0x7f8e080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:11.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.512+0000 7f8e03fff700 1 -- 172.21.15.7:0/973117001 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8e14084990 con 0x7f8e1c15d830 2024-01-27T02:38:11.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.512+0000 7f8e1bfff700 1 --2- 172.21.15.7:0/973117001 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8e08066f30 0x7f8e080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:11.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.512+0000 7f8e1bfff700 1 --2- 172.21.15.7:0/973117001 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8e08066f30 0x7f8e080693e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c007fd0 tx=0x7f8e0c007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:11.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.516+0000 7f8e03fff700 1 -- 172.21.15.7:0/973117001 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8e14055690 con 0x7f8e1c15d830 2024-01-27T02:38:11.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.718+0000 7f8e22413700 1 -- 172.21.15.7:0/973117001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8e1c0b38d0 con 0x7f8e1c15d830 2024-01-27T02:38:11.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.718+0000 7f8e03fff700 1 -- 172.21.15.7:0/973117001 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v120) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8e1c0b38d0 con 0x7f8e1c15d830 2024-01-27T02:38:11.719 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-27T02:38:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.720+0000 7f8e22413700 1 -- 172.21.15.7:0/973117001 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8e08066f30 msgr2=0x7f8e080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.720+0000 7f8e22413700 1 --2- 172.21.15.7:0/973117001 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8e08066f30 0x7f8e080693e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c007fd0 tx=0x7f8e0c007d50 comp rx=0 tx=0).stop 2024-01-27T02:38:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.720+0000 7f8e22413700 1 -- 172.21.15.7:0/973117001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e1c15d830 msgr2=0x7f8e1c0b1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.720+0000 7f8e22413700 1 --2- 172.21.15.7:0/973117001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e1c15d830 0x7f8e1c0b1f30 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f8e140020a0 tx=0x7f8e14001c70 comp rx=0 tx=0).stop 2024-01-27T02:38:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.720+0000 7f8e22413700 1 -- 172.21.15.7:0/973117001 shutdown_connections 2024-01-27T02:38:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.720+0000 7f8e22413700 1 --2- 172.21.15.7:0/973117001 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8e08066f30 0x7f8e080693e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.720+0000 7f8e22413700 1 --2- 172.21.15.7:0/973117001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e1c15d830 0x7f8e1c0b1f30 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.720+0000 7f8e22413700 1 --2- 172.21.15.7:0/973117001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e1c0cf280 0x7f8e1c149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.720+0000 7f8e22413700 1 --2- 172.21.15.7:0/973117001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e1c15d460 0x7f8e1c0ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.720+0000 7f8e22413700 1 -- 172.21.15.7:0/973117001 >> 172.21.15.7:0/973117001 conn(0x7f8e1c063580 msgr2=0x7f8e1c14bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.720+0000 7f8e22413700 1 -- 172.21.15.7:0/973117001 shutdown_connections 2024-01-27T02:38:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.720+0000 7f8e22413700 1 -- 172.21.15.7:0/973117001 wait complete. 2024-01-27T02:38:11.734 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-27T02:38:11.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.869+0000 7f8b8e2d0700 1 Processor -- start 2024-01-27T02:38:11.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.869+0000 7f8b8e2d0700 1 -- start start 2024-01-27T02:38:11.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.870+0000 7f8b8e2d0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8b880cb950 0x7f8b880b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:11.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.870+0000 7f8b8e2d0700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b881412d0 0x7f8b88141700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:11.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.870+0000 7f8b8e2d0700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b880c9660 con 0x7f8b880cbd20 2024-01-27T02:38:11.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.870+0000 7f8b8e2d0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b8815c980 con 0x7f8b880cb950 2024-01-27T02:38:11.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.870+0000 7f8b8e2d0700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b88141c40 con 0x7f8b881412d0 2024-01-27T02:38:11.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.870+0000 7f8b87fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8b880cb950 0x7f8b880b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:11.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.870+0000 7f8b87fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8b880cb950 0x7f8b880b1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53880/0 (socket says 172.21.15.7:53880) 2024-01-27T02:38:11.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.870+0000 7f8b87fff700 1 -- 172.21.15.7:0/3823550724 learned_addr learned my addr 172.21.15.7:0/3823550724 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:11.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.870+0000 7f8b8c86d700 1 --2- 172.21.15.7:0/3823550724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b881412d0 0x7f8b88141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:11.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.870+0000 7f8b87fff700 1 -- 172.21.15.7:0/3823550724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b881412d0 msgr2=0x7f8b88141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:11.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.870+0000 7f8b87fff700 1 --2- 172.21.15.7:0/3823550724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b881412d0 0x7f8b88141700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.870+0000 7f8b87fff700 1 -- 172.21.15.7:0/3823550724 >> v1:172.21.15.188:6789/0 conn(0x7f8b880cbd20 legacy=0x7f8b880ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:11.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.870+0000 7f8b87fff700 1 -- 172.21.15.7:0/3823550724 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8815be00 con 0x7f8b880cb950 2024-01-27T02:38:11.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.870+0000 7f8b8c86d700 1 --2- 172.21.15.7:0/3823550724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b881412d0 0x7f8b88141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:11.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.870+0000 7f8b87fff700 1 --2- 172.21.15.7:0/3823550724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b880cb950 0x7f8b880b1ac0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f8b780099b0 tx=0x7f8b78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b3a4a55330bdda server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:11.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.870+0000 7f8b86ffd700 1 -- 172.21.15.7:0/3823550724 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7800b860 con 0x7f8b880cb950 2024-01-27T02:38:11.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.871+0000 7f8b86ffd700 1 -- 172.21.15.7:0/3823550724 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7800b9c0 con 0x7f8b880cb950 2024-01-27T02:38:11.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.871+0000 7f8b86ffd700 1 -- 172.21.15.7:0/3823550724 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7800bd60 con 0x7f8b880cb950 2024-01-27T02:38:11.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.871+0000 7f8b8e2d0700 1 -- 172.21.15.7:0/3823550724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b880cb950 msgr2=0x7f8b880b1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:11.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.871+0000 7f8b8e2d0700 1 --2- 172.21.15.7:0/3823550724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b880cb950 0x7f8b880b1ac0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f8b780099b0 tx=0x7f8b78009250 comp rx=0 tx=0).stop 2024-01-27T02:38:11.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.871+0000 7f8b8e2d0700 1 -- 172.21.15.7:0/3823550724 shutdown_connections 2024-01-27T02:38:11.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.871+0000 7f8b8e2d0700 1 --2- 172.21.15.7:0/3823550724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b881412d0 0x7f8b88141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.871+0000 7f8b8e2d0700 1 --2- 172.21.15.7:0/3823550724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b880cb950 0x7f8b880b1ac0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.871+0000 7f8b8e2d0700 1 -- 172.21.15.7:0/3823550724 >> 172.21.15.7:0/3823550724 conn(0x7f8b88063580 msgr2=0x7f8b88063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:11.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.871+0000 7f8b8e2d0700 1 -- 172.21.15.7:0/3823550724 shutdown_connections 2024-01-27T02:38:11.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.871+0000 7f8b8e2d0700 1 -- 172.21.15.7:0/3823550724 wait complete. 2024-01-27T02:38:11.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.872+0000 7f8b8e2d0700 1 Processor -- start 2024-01-27T02:38:11.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.873+0000 7f8b8e2d0700 1 -- start start 2024-01-27T02:38:11.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.873+0000 7f8b8e2d0700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b880cb950 0x7f8b88069560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:11.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.873+0000 7f8b8e2d0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b880cbd20 0x7f8b8816dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:11.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.873+0000 7f8b8e2d0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b881412d0 0x7f8b8813e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.873+0000 7f8b8e2d0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8816ea40 con 0x7f8b880cbd20 2024-01-27T02:38:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.873+0000 7f8b8e2d0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b880c7b70 con 0x7f8b881412d0 2024-01-27T02:38:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.873+0000 7f8b8e2d0700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8816e1a0 con 0x7f8b880cb950 2024-01-27T02:38:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.874+0000 7f8b877fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b880cbd20 0x7f8b8816dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.874+0000 7f8b877fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b880cbd20 0x7f8b8816dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53894/0 (socket says 172.21.15.7:53894) 2024-01-27T02:38:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.874+0000 7f8b877fe700 1 -- 172.21.15.7:0/3537737530 learned_addr learned my addr 172.21.15.7:0/3537737530 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.874+0000 7f8b87fff700 1 --2- 172.21.15.7:0/3537737530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b880cb950 0x7f8b88069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.874+0000 7f8b8c86d700 1 --2- 172.21.15.7:0/3537737530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b881412d0 0x7f8b8813e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.874+0000 7f8b877fe700 1 -- 172.21.15.7:0/3537737530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b880cb950 msgr2=0x7f8b88069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.874+0000 7f8b877fe700 1 --2- 172.21.15.7:0/3537737530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b880cb950 0x7f8b88069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.874+0000 7f8b877fe700 1 -- 172.21.15.7:0/3537737530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b881412d0 msgr2=0x7f8b8813e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.874+0000 7f8b877fe700 1 --2- 172.21.15.7:0/3537737530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b881412d0 0x7f8b8813e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.874+0000 7f8b877fe700 1 -- 172.21.15.7:0/3537737530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b780094f0 con 0x7f8b880cbd20 2024-01-27T02:38:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.874+0000 7f8b87fff700 1 --2- 172.21.15.7:0/3537737530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b880cb950 0x7f8b88069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:11.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.874+0000 7f8b877fe700 1 --2- 172.21.15.7:0/3537737530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b880cbd20 0x7f8b8816dae0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00c590 tx=0x7f8b7c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:11.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.874+0000 7f8b6ffff700 1 -- 172.21.15.7:0/3537737530 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c011590 con 0x7f8b880cbd20 2024-01-27T02:38:11.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.874+0000 7f8b6ffff700 1 -- 172.21.15.7:0/3537737530 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7c0116f0 con 0x7f8b880cbd20 2024-01-27T02:38:11.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.874+0000 7f8b8e2d0700 1 -- 172.21.15.7:0/3537737530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8816e480 con 0x7f8b880cbd20 2024-01-27T02:38:11.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.874+0000 7f8b6ffff700 1 -- 172.21.15.7:0/3537737530 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c011ab0 con 0x7f8b880cbd20 2024-01-27T02:38:11.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.874+0000 7f8b8e2d0700 1 -- 172.21.15.7:0/3537737530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b88067ae0 con 0x7f8b880cbd20 2024-01-27T02:38:11.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.875+0000 7f8b8e2d0700 1 -- 172.21.15.7:0/3537737530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b88067cd0 con 0x7f8b880cbd20 2024-01-27T02:38:11.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.876+0000 7f8b6ffff700 1 -- 172.21.15.7:0/3537737530 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8b7c01b3b0 con 0x7f8b880cbd20 2024-01-27T02:38:11.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.876+0000 7f8b6ffff700 1 --2- 172.21.15.7:0/3537737530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b74066f30 0x7f8b740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:11.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.877+0000 7f8b6ffff700 1 -- 172.21.15.7:0/3537737530 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8b7c083b80 con 0x7f8b880cbd20 2024-01-27T02:38:11.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.877+0000 7f8b87fff700 1 --2- 172.21.15.7:0/3537737530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b74066f30 0x7f8b740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:11.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.877+0000 7f8b87fff700 1 --2- 172.21.15.7:0/3537737530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b74066f30 0x7f8b740693e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8b780099b0 tx=0x7f8b78017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:11.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:11.881+0000 7f8b6ffff700 1 -- 172.21.15.7:0/3537737530 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b7c050d50 con 0x7f8b880cbd20 2024-01-27T02:38:12.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.081+0000 7f8b8e2d0700 1 -- 172.21.15.7:0/3537737530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8b88072060 con 0x7f8b880cbd20 2024-01-27T02:38:12.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.082+0000 7f8b6ffff700 1 -- 172.21.15.7:0/3537737530 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v120) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8b7c017070 con 0x7f8b880cbd20 2024-01-27T02:38:12.082 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-27T02:38:12.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.084+0000 7f8b8e2d0700 1 -- 172.21.15.7:0/3537737530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b74066f30 msgr2=0x7f8b740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:12.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.084+0000 7f8b8e2d0700 1 --2- 172.21.15.7:0/3537737530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b74066f30 0x7f8b740693e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8b780099b0 tx=0x7f8b78017000 comp rx=0 tx=0).stop 2024-01-27T02:38:12.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.084+0000 7f8b8e2d0700 1 -- 172.21.15.7:0/3537737530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b880cbd20 msgr2=0x7f8b8816dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:12.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.084+0000 7f8b8e2d0700 1 --2- 172.21.15.7:0/3537737530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b880cbd20 0x7f8b8816dae0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00c590 tx=0x7f8b7c00c950 comp rx=0 tx=0).stop 2024-01-27T02:38:12.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.084+0000 7f8b8e2d0700 1 -- 172.21.15.7:0/3537737530 shutdown_connections 2024-01-27T02:38:12.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.084+0000 7f8b8e2d0700 1 --2- 172.21.15.7:0/3537737530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b74066f30 0x7f8b740693e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:12.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.084+0000 7f8b8e2d0700 1 --2- 172.21.15.7:0/3537737530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b881412d0 0x7f8b8813e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:12.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.084+0000 7f8b8e2d0700 1 --2- 172.21.15.7:0/3537737530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b880cb950 0x7f8b88069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:12.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.084+0000 7f8b8e2d0700 1 --2- 172.21.15.7:0/3537737530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b880cbd20 0x7f8b8816dae0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:12.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.084+0000 7f8b8e2d0700 1 -- 172.21.15.7:0/3537737530 >> 172.21.15.7:0/3537737530 conn(0x7f8b88063580 msgr2=0x7f8b8806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:12.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.084+0000 7f8b8e2d0700 1 -- 172.21.15.7:0/3537737530 shutdown_connections 2024-01-27T02:38:12.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.084+0000 7f8b8e2d0700 1 -- 172.21.15.7:0/3537737530 wait complete. 2024-01-27T02:38:12.098 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:38:12.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.229+0000 7f4be414f700 1 Processor -- start 2024-01-27T02:38:12.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.229+0000 7f4be414f700 1 -- start start 2024-01-27T02:38:12.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.230+0000 7f4be414f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4bdc0c4cc0 0x7f4bdc0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:12.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.230+0000 7f4be414f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bdc0ce5e0 con 0x7f4bdc15a100 2024-01-27T02:38:12.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.230+0000 7f4be414f700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bdc0cd9d0 con 0x7f4bdc0ca540 2024-01-27T02:38:12.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.230+0000 7f4be414f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bdc0cdcb0 con 0x7f4bdc0c4cc0 2024-01-27T02:38:12.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.230+0000 7f4be16ea700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4bdc0c4cc0 0x7f4bdc0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:12.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.230+0000 7f4be1eeb700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4bdc15a100 0x7f4bdc0c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42180/0 (socket says 172.21.15.7:42180) 2024-01-27T02:38:12.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.230+0000 7f4be1eeb700 1 -- 172.21.15.7:0/1010772480 learned_addr learned my addr 172.21.15.7:0/1010772480 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:12.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.231+0000 7f4be16ea700 1 -- 172.21.15.7:0/1010772480 >> v1:172.21.15.188:6789/0 conn(0x7f4bdc0ca540 legacy=0x7f4bdc163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:12.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.231+0000 7f4be16ea700 1 -- 172.21.15.7:0/1010772480 >> v1:172.21.15.7:6789/0 conn(0x7f4bdc15a100 legacy=0x7f4bdc0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:12.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.231+0000 7f4be16ea700 1 -- 172.21.15.7:0/1010772480 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc0cdf90 con 0x7f4bdc0c4cc0 2024-01-27T02:38:12.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.231+0000 7f4be0ee9700 1 -- 172.21.15.7:0/1010772480 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3552565474 0 0) 0x7f4bdc0ce5e0 con 0x7f4bdc15a100 2024-01-27T02:38:12.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.231+0000 7f4be16ea700 1 --2- 172.21.15.7:0/1010772480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bdc0c4cc0 0x7f4bdc0cd2f0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0098e0 tx=0x7f4bcc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ed832a1ec9c6bb server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:12.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.231+0000 7f4be0ee9700 1 -- 172.21.15.7:0/1010772480 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc009e80 con 0x7f4bdc0c4cc0 2024-01-27T02:38:12.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.231+0000 7f4be0ee9700 1 -- 172.21.15.7:0/1010772480 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bcc00b860 con 0x7f4bdc0c4cc0 2024-01-27T02:38:12.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.231+0000 7f4be0ee9700 1 -- 172.21.15.7:0/1010772480 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc00bbe0 con 0x7f4bdc0c4cc0 2024-01-27T02:38:12.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.232+0000 7f4be414f700 1 -- 172.21.15.7:0/1010772480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bdc0c4cc0 msgr2=0x7f4bdc0cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:12.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.232+0000 7f4be414f700 1 --2- 172.21.15.7:0/1010772480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bdc0c4cc0 0x7f4bdc0cd2f0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0098e0 tx=0x7f4bcc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:12.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.232+0000 7f4be414f700 1 -- 172.21.15.7:0/1010772480 shutdown_connections 2024-01-27T02:38:12.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.232+0000 7f4be414f700 1 --2- 172.21.15.7:0/1010772480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bdc0c4cc0 0x7f4bdc0cd2f0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:12.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.232+0000 7f4be414f700 1 -- 172.21.15.7:0/1010772480 >> 172.21.15.7:0/1010772480 conn(0x7f4bdc098020 msgr2=0x7f4bdc0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:12.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.232+0000 7f4be414f700 1 -- 172.21.15.7:0/1010772480 shutdown_connections 2024-01-27T02:38:12.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.232+0000 7f4be414f700 1 -- 172.21.15.7:0/1010772480 wait complete. 2024-01-27T02:38:12.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.233+0000 7f4be414f700 1 Processor -- start 2024-01-27T02:38:12.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.233+0000 7f4be414f700 1 -- start start 2024-01-27T02:38:12.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.234+0000 7f4be414f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bdc0c4cc0 0x7f4bdc1525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:12.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.234+0000 7f4be414f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bdc0ca540 0x7f4bdc15caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:12.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.234+0000 7f4be414f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bdc15a100 0x7f4bdc0b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:12.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.234+0000 7f4be414f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc0b40f0 con 0x7f4bdc0c4cc0 2024-01-27T02:38:12.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.234+0000 7f4be414f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc0b4580 con 0x7f4bdc0ca540 2024-01-27T02:38:12.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.234+0000 7f4be414f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc0b34e0 con 0x7f4bdc15a100 2024-01-27T02:38:12.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.234+0000 7f4be1eeb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bdc0c4cc0 0x7f4bdc1525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:12.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.234+0000 7f4be1eeb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bdc0c4cc0 0x7f4bdc1525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53916/0 (socket says 172.21.15.7:53916) 2024-01-27T02:38:12.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.234+0000 7f4be1eeb700 1 -- 172.21.15.7:0/3994791085 learned_addr learned my addr 172.21.15.7:0/3994791085 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:12.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.235+0000 7f4be26ec700 1 --2- 172.21.15.7:0/3994791085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bdc15a100 0x7f4bdc0b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:12.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.235+0000 7f4be16ea700 1 --2- 172.21.15.7:0/3994791085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bdc0ca540 0x7f4bdc15caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:12.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.235+0000 7f4be1eeb700 1 -- 172.21.15.7:0/3994791085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bdc15a100 msgr2=0x7f4bdc0b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:12.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.235+0000 7f4be1eeb700 1 --2- 172.21.15.7:0/3994791085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bdc15a100 0x7f4bdc0b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:12.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.235+0000 7f4be1eeb700 1 -- 172.21.15.7:0/3994791085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bdc0ca540 msgr2=0x7f4bdc15caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:12.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.235+0000 7f4be1eeb700 1 --2- 172.21.15.7:0/3994791085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bdc0ca540 0x7f4bdc15caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:12.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.235+0000 7f4be1eeb700 1 -- 172.21.15.7:0/3994791085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bcc009420 con 0x7f4bdc0c4cc0 2024-01-27T02:38:12.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.235+0000 7f4be26ec700 1 --2- 172.21.15.7:0/3994791085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bdc15a100 0x7f4bdc0b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:12.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.235+0000 7f4be16ea700 1 --2- 172.21.15.7:0/3994791085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bdc0ca540 0x7f4bdc15caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:12.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.235+0000 7f4be1eeb700 1 --2- 172.21.15.7:0/3994791085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bdc0c4cc0 0x7f4bdc1525b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800c560 tx=0x7f4bd8000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:12.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.235+0000 7f4bd1ffb700 1 -- 172.21.15.7:0/3994791085 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8005b30 con 0x7f4bdc0c4cc0 2024-01-27T02:38:12.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.235+0000 7f4bd1ffb700 1 -- 172.21.15.7:0/3994791085 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd8005c90 con 0x7f4bdc0c4cc0 2024-01-27T02:38:12.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.235+0000 7f4be414f700 1 -- 172.21.15.7:0/3994791085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bdc0b37c0 con 0x7f4bdc0c4cc0 2024-01-27T02:38:12.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.235+0000 7f4bd1ffb700 1 -- 172.21.15.7:0/3994791085 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8007320 con 0x7f4bdc0c4cc0 2024-01-27T02:38:12.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.235+0000 7f4be414f700 1 -- 172.21.15.7:0/3994791085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bdc0b3d80 con 0x7f4bdc0c4cc0 2024-01-27T02:38:12.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.236+0000 7f4be414f700 1 -- 172.21.15.7:0/3994791085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bdc04c710 con 0x7f4bdc0c4cc0 2024-01-27T02:38:12.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.237+0000 7f4bd1ffb700 1 -- 172.21.15.7:0/3994791085 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4bd801b3b0 con 0x7f4bdc0c4cc0 2024-01-27T02:38:12.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.237+0000 7f4bd1ffb700 1 --2- 172.21.15.7:0/3994791085 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4bc0066f30 0x7f4bc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:12.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.237+0000 7f4bd1ffb700 1 -- 172.21.15.7:0/3994791085 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4bd8083b00 con 0x7f4bdc0c4cc0 2024-01-27T02:38:12.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.238+0000 7f4be16ea700 1 --2- 172.21.15.7:0/3994791085 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4bc0066f30 0x7f4bc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:12.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.238+0000 7f4be16ea700 1 --2- 172.21.15.7:0/3994791085 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4bc0066f30 0x7f4bc00693e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0093f0 tx=0x7f4bcc017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:12.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.242+0000 7f4bd1ffb700 1 -- 172.21.15.7:0/3994791085 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4bd8050d50 con 0x7f4bdc0c4cc0 2024-01-27T02:38:12.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.493+0000 7f4be414f700 1 -- 172.21.15.7:0/3994791085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4bdc0ce270 con 0x7f4bdc0c4cc0 2024-01-27T02:38:12.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.493+0000 7f4bd1ffb700 1 -- 172.21.15.7:0/3994791085 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f4bd8017020 con 0x7f4bdc0c4cc0 2024-01-27T02:38:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:12.494 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2767759975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2767759975},{"type":"v1","addr":"172.21.15.188:6836","nonce":2767759975}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":5768,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/245473575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":245473575},{"type":"v1","addr":"172.21.15.7:6834","nonce":245473575}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1334164270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1334164270},{"type":"v1","addr":"172.21.15.188:6837","nonce":1334164270}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:37:51.231918+0000","modified":"2024-01-27T02:37:54.851112+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6657},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6657":{"gid":6657,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1251886476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1251886476},{"type":"v1","addr":"172.21.15.7:6835","nonce":1251886476}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2024-01-27T02:38:12.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.496+0000 7f4be414f700 1 -- 172.21.15.7:0/3994791085 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4bc0066f30 msgr2=0x7f4bc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:12.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.496+0000 7f4be414f700 1 --2- 172.21.15.7:0/3994791085 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4bc0066f30 0x7f4bc00693e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0093f0 tx=0x7f4bcc017000 comp rx=0 tx=0).stop 2024-01-27T02:38:12.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.496+0000 7f4be414f700 1 -- 172.21.15.7:0/3994791085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bdc0c4cc0 msgr2=0x7f4bdc1525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:12.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.496+0000 7f4be414f700 1 --2- 172.21.15.7:0/3994791085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bdc0c4cc0 0x7f4bdc1525b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800c560 tx=0x7f4bd8000fe0 comp rx=0 tx=0).stop 2024-01-27T02:38:12.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.496+0000 7f4be414f700 1 -- 172.21.15.7:0/3994791085 shutdown_connections 2024-01-27T02:38:12.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.496+0000 7f4be414f700 1 --2- 172.21.15.7:0/3994791085 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4bc0066f30 0x7f4bc00693e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:12.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.496+0000 7f4be414f700 1 --2- 172.21.15.7:0/3994791085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bdc15a100 0x7f4bdc0b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:12.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.496+0000 7f4be414f700 1 --2- 172.21.15.7:0/3994791085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bdc0c4cc0 0x7f4bdc1525b0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:12.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.496+0000 7f4be414f700 1 --2- 172.21.15.7:0/3994791085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bdc0ca540 0x7f4bdc15caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:12.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.496+0000 7f4be414f700 1 -- 172.21.15.7:0/3994791085 >> 172.21.15.7:0/3994791085 conn(0x7f4bdc098020 msgr2=0x7f4bdc154450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:12.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.496+0000 7f4be414f700 1 -- 172.21.15.7:0/3994791085 shutdown_connections 2024-01-27T02:38:12.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.496+0000 7f4be414f700 1 -- 172.21.15.7:0/3994791085 wait complete. 2024-01-27T02:38:12.498 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 58 2024-01-27T02:38:12.510 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:38:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.641+0000 7f719704a700 1 Processor -- start 2024-01-27T02:38:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.641+0000 7f719704a700 1 -- start start 2024-01-27T02:38:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.642+0000 7f719704a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71900b2b80 0x7f71900b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.642+0000 7f719704a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7190143b40 0x7f71900d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.642+0000 7f719704a700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71900d0d70 con 0x7f71900b2f50 2024-01-27T02:38:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.642+0000 7f719704a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7190143f80 con 0x7f71900b2b80 2024-01-27T02:38:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.642+0000 7f719704a700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71900b43b0 con 0x7f7190143b40 2024-01-27T02:38:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.642+0000 7f7194de6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71900b2b80 0x7f71900b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.642+0000 7f7194de6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71900b2b80 0x7f71900b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53924/0 (socket says 172.21.15.7:53924) 2024-01-27T02:38:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.642+0000 7f7194de6700 1 -- 172.21.15.7:0/2286841845 learned_addr learned my addr 172.21.15.7:0/2286841845 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.642+0000 7f71955e7700 1 --2- 172.21.15.7:0/2286841845 >> v2:172.21.15.188:3300/0 conn(0x7f7190143b40 0x7f71900d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.642+0000 7f7194de6700 1 -- 172.21.15.7:0/2286841845 >> v2:172.21.15.188:3300/0 conn(0x7f7190143b40 msgr2=0x7f71900d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.642+0000 7f7194de6700 1 --2- 172.21.15.7:0/2286841845 >> v2:172.21.15.188:3300/0 conn(0x7f7190143b40 0x7f71900d0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.642+0000 7f7194de6700 1 -- 172.21.15.7:0/2286841845 >> v1:172.21.15.188:6789/0 conn(0x7f71900b2f50 legacy=0x7f7190143430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.642+0000 7f7194de6700 1 -- 172.21.15.7:0/2286841845 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71900b4690 con 0x7f71900b2b80 2024-01-27T02:38:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.642+0000 7f7194de6700 1 --2- 172.21.15.7:0/2286841845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71900b2b80 0x7f71900b3d20 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f71900ce550 tx=0x7f7180009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860ad6204ab69a49 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.643+0000 7f718f7fe700 1 -- 172.21.15.7:0/2286841845 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718000b860 con 0x7f71900b2b80 2024-01-27T02:38:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.643+0000 7f718f7fe700 1 -- 172.21.15.7:0/2286841845 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718000b9c0 con 0x7f71900b2b80 2024-01-27T02:38:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.643+0000 7f718f7fe700 1 -- 172.21.15.7:0/2286841845 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718000bd60 con 0x7f71900b2b80 2024-01-27T02:38:12.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.643+0000 7f719704a700 1 -- 172.21.15.7:0/2286841845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71900b2b80 msgr2=0x7f71900b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.643+0000 7f719704a700 1 --2- 172.21.15.7:0/2286841845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71900b2b80 0x7f71900b3d20 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f71900ce550 tx=0x7f7180009250 comp rx=0 tx=0).stop 2024-01-27T02:38:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.643+0000 7f719704a700 1 -- 172.21.15.7:0/2286841845 shutdown_connections 2024-01-27T02:38:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.643+0000 7f719704a700 1 --2- 172.21.15.7:0/2286841845 >> v2:172.21.15.188:3300/0 conn(0x7f7190143b40 0x7f71900d0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.643+0000 7f719704a700 1 --2- 172.21.15.7:0/2286841845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71900b2b80 0x7f71900b3d20 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.643+0000 7f719704a700 1 -- 172.21.15.7:0/2286841845 >> 172.21.15.7:0/2286841845 conn(0x7f7190068480 msgr2=0x7f71900709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.643+0000 7f719704a700 1 -- 172.21.15.7:0/2286841845 shutdown_connections 2024-01-27T02:38:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.643+0000 7f719704a700 1 -- 172.21.15.7:0/2286841845 wait complete. 2024-01-27T02:38:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.645+0000 7f719704a700 1 Processor -- start 2024-01-27T02:38:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.645+0000 7f719704a700 1 -- start start 2024-01-27T02:38:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.646+0000 7f719704a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71900b2b80 0x7f71900cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.646+0000 7f719704a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71900b2f50 0x7f7190140db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.646+0000 7f719704a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7190143b40 0x7f7190142520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.646+0000 7f719704a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71900cd340 con 0x7f7190143b40 2024-01-27T02:38:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.646+0000 7f719704a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71900cc4b0 con 0x7f71900b2b80 2024-01-27T02:38:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.646+0000 7f719704a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71900cc730 con 0x7f71900b2f50 2024-01-27T02:38:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.646+0000 7f71955e7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7190143b40 0x7f7190142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.646+0000 7f71955e7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7190143b40 0x7f7190142520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53926/0 (socket says 172.21.15.7:53926) 2024-01-27T02:38:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.646+0000 7f71955e7700 1 -- 172.21.15.7:0/1168780898 learned_addr learned my addr 172.21.15.7:0/1168780898 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.646+0000 7f71955e7700 1 -- 172.21.15.7:0/1168780898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71900b2f50 msgr2=0x7f7190140db0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-27T02:38:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.646+0000 7f71955e7700 1 --2- 172.21.15.7:0/1168780898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71900b2f50 0x7f7190140db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.646+0000 7f71955e7700 1 -- 172.21.15.7:0/1168780898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71900b2b80 msgr2=0x7f71900cd7d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-27T02:38:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.646+0000 7f71955e7700 1 --2- 172.21.15.7:0/1168780898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71900b2b80 0x7f71900cd7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.646+0000 7f71955e7700 1 -- 172.21.15.7:0/1168780898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71800094f0 con 0x7f7190143b40 2024-01-27T02:38:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.646+0000 7f71955e7700 1 --2- 172.21.15.7:0/1168780898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7190143b40 0x7f7190142520 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7188007f50 tx=0x7f7188009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.646+0000 7f718cff9700 1 -- 172.21.15.7:0/1168780898 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718800ecd0 con 0x7f7190143b40 2024-01-27T02:38:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.646+0000 7f718cff9700 1 -- 172.21.15.7:0/1168780898 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718800ee50 con 0x7f7190143b40 2024-01-27T02:38:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.646+0000 7f718cff9700 1 -- 172.21.15.7:0/1168780898 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188014670 con 0x7f7190143b40 2024-01-27T02:38:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.646+0000 7f719704a700 1 -- 172.21.15.7:0/1168780898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71900cca10 con 0x7f7190143b40 2024-01-27T02:38:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.647+0000 7f719704a700 1 -- 172.21.15.7:0/1168780898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71900ccfd0 con 0x7f7190143b40 2024-01-27T02:38:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.648+0000 7f719704a700 1 -- 172.21.15.7:0/1168780898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f719004c710 con 0x7f7190143b40 2024-01-27T02:38:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.648+0000 7f718cff9700 1 -- 172.21.15.7:0/1168780898 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f71880149e0 con 0x7f7190143b40 2024-01-27T02:38:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.648+0000 7f718cff9700 1 --2- 172.21.15.7:0/1168780898 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f717c066f30 0x7f717c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.648+0000 7f718cff9700 1 -- 172.21.15.7:0/1168780898 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7188084bf0 con 0x7f7190143b40 2024-01-27T02:38:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.649+0000 7f7194de6700 1 --2- 172.21.15.7:0/1168780898 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f717c066f30 0x7f717c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.650+0000 7f7194de6700 1 --2- 172.21.15.7:0/1168780898 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f717c066f30 0x7f717c0693e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f71900ce550 tx=0x7f7180017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.653+0000 7f718cff9700 1 -- 172.21.15.7:0/1168780898 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7188051e60 con 0x7f7190143b40 2024-01-27T02:38:12.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.857+0000 7f719704a700 1 -- 172.21.15.7:0/1168780898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71900b4970 con 0x7f7190143b40 2024-01-27T02:38:12.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.858+0000 7f718cff9700 1 -- 172.21.15.7:0/1168780898 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+15324 (secure 0 0 0) 0x7f7188055480 con 0x7f7190143b40 2024-01-27T02:38:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:12.859 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":120,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:38:07.896480+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:37:49.747694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-27T02:37:50.199870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:38:12.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.860+0000 7f719704a700 1 -- 172.21.15.7:0/1168780898 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f717c066f30 msgr2=0x7f717c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:12.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.860+0000 7f719704a700 1 --2- 172.21.15.7:0/1168780898 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f717c066f30 0x7f717c0693e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f71900ce550 tx=0x7f7180017000 comp rx=0 tx=0).stop 2024-01-27T02:38:12.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.860+0000 7f719704a700 1 -- 172.21.15.7:0/1168780898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7190143b40 msgr2=0x7f7190142520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:12.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.860+0000 7f719704a700 1 --2- 172.21.15.7:0/1168780898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7190143b40 0x7f7190142520 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7188007f50 tx=0x7f7188009860 comp rx=0 tx=0).stop 2024-01-27T02:38:12.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.860+0000 7f719704a700 1 -- 172.21.15.7:0/1168780898 shutdown_connections 2024-01-27T02:38:12.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.860+0000 7f719704a700 1 --2- 172.21.15.7:0/1168780898 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f717c066f30 0x7f717c0693e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:12.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.860+0000 7f719704a700 1 --2- 172.21.15.7:0/1168780898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7190143b40 0x7f7190142520 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:12.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.860+0000 7f719704a700 1 --2- 172.21.15.7:0/1168780898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71900b2b80 0x7f71900cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:12.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.860+0000 7f719704a700 1 --2- 172.21.15.7:0/1168780898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71900b2f50 0x7f7190140db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:12.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.860+0000 7f719704a700 1 -- 172.21.15.7:0/1168780898 >> 172.21.15.7:0/1168780898 conn(0x7f7190068480 msgr2=0x7f7190141c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:12.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.861+0000 7f719704a700 1 -- 172.21.15.7:0/1168780898 shutdown_connections 2024-01-27T02:38:12.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:12.861+0000 7f719704a700 1 -- 172.21.15.7:0/1168780898 wait complete. 2024-01-27T02:38:12.875 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-27T02:38:12.875 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:38:13.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.007+0000 7f690d744700 1 Processor -- start 2024-01-27T02:38:13.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.007+0000 7f690d744700 1 -- start start 2024-01-27T02:38:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.008+0000 7f690d744700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f69080b2ff0 0x7f69080b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.008+0000 7f690d744700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f690815e4e0 0x7f69080ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.008+0000 7f690d744700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69080b3900 con 0x7f690815b8e0 2024-01-27T02:38:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.008+0000 7f690d744700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69080ce7e0 con 0x7f69080b2ff0 2024-01-27T02:38:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.008+0000 7f690d744700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690815ebe0 con 0x7f690815e4e0 2024-01-27T02:38:13.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.008+0000 7f6906ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f69080b2ff0 0x7f69080b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.008+0000 7f69067fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f690815b8e0 0x7f690815ddd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42182/0 (socket says 172.21.15.7:42182) 2024-01-27T02:38:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.008+0000 7f69067fc700 1 -- 172.21.15.7:0/2629308916 learned_addr learned my addr 172.21.15.7:0/2629308916 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.008+0000 7f69077fe700 1 --2- 172.21.15.7:0/2629308916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f690815e4e0 0x7f69080ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.008+0000 7f6906ffd700 1 -- 172.21.15.7:0/2629308916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f690815e4e0 msgr2=0x7f69080ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.008+0000 7f6906ffd700 1 --2- 172.21.15.7:0/2629308916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f690815e4e0 0x7f69080ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.008+0000 7f6906ffd700 1 -- 172.21.15.7:0/2629308916 >> v1:172.21.15.7:6789/0 conn(0x7f690815b8e0 legacy=0x7f690815ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.008+0000 7f6906ffd700 1 -- 172.21.15.7:0/2629308916 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69081587b0 con 0x7f69080b2ff0 2024-01-27T02:38:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.008+0000 7f6906ffd700 1 --2- 172.21.15.7:0/2629308916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69080b2ff0 0x7f69080b33c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f68f80098d0 tx=0x7f68f8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c0d92ee0e4a050 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.009+0000 7f6905ffb700 1 -- 172.21.15.7:0/2629308916 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8009e70 con 0x7f69080b2ff0 2024-01-27T02:38:13.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.009+0000 7f6905ffb700 1 -- 172.21.15.7:0/2629308916 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f8005c10 con 0x7f69080b2ff0 2024-01-27T02:38:13.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.009+0000 7f6905ffb700 1 -- 172.21.15.7:0/2629308916 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f800b650 con 0x7f69080b2ff0 2024-01-27T02:38:13.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.009+0000 7f690d744700 1 -- 172.21.15.7:0/2629308916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69080b2ff0 msgr2=0x7f69080b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:13.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.009+0000 7f690d744700 1 --2- 172.21.15.7:0/2629308916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69080b2ff0 0x7f69080b33c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f68f80098d0 tx=0x7f68f8009120 comp rx=0 tx=0).stop 2024-01-27T02:38:13.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.009+0000 7f690d744700 1 -- 172.21.15.7:0/2629308916 shutdown_connections 2024-01-27T02:38:13.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.009+0000 7f690d744700 1 --2- 172.21.15.7:0/2629308916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f690815e4e0 0x7f69080ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:13.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.009+0000 7f690d744700 1 --2- 172.21.15.7:0/2629308916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69080b2ff0 0x7f69080b33c0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:13.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.009+0000 7f690d744700 1 -- 172.21.15.7:0/2629308916 >> 172.21.15.7:0/2629308916 conn(0x7f69080679b0 msgr2=0x7f6908067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:13.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.009+0000 7f690d744700 1 -- 172.21.15.7:0/2629308916 shutdown_connections 2024-01-27T02:38:13.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.009+0000 7f690d744700 1 -- 172.21.15.7:0/2629308916 wait complete. 2024-01-27T02:38:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.011+0000 7f690d744700 1 Processor -- start 2024-01-27T02:38:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.011+0000 7f690d744700 1 -- start start 2024-01-27T02:38:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.011+0000 7f690d744700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69080b2ff0 0x7f690815edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.011+0000 7f690d744700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f690815b8e0 0x7f690814efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.011+0000 7f690d744700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f690815e4e0 0x7f690814f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:13.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.011+0000 7f690d744700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690814fa60 con 0x7f690815e4e0 2024-01-27T02:38:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.011+0000 7f690d744700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690816bd10 con 0x7f690815b8e0 2024-01-27T02:38:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.011+0000 7f690d744700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690815ffc0 con 0x7f69080b2ff0 2024-01-27T02:38:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.011+0000 7f69077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f690815e4e0 0x7f690814f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.012+0000 7f69077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f690815e4e0 0x7f690814f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53956/0 (socket says 172.21.15.7:53956) 2024-01-27T02:38:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.012+0000 7f69077fe700 1 -- 172.21.15.7:0/2867355172 learned_addr learned my addr 172.21.15.7:0/2867355172 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.012+0000 7f69067fc700 1 --2- 172.21.15.7:0/2867355172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f690815b8e0 0x7f690814efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.012+0000 7f6906ffd700 1 --2- 172.21.15.7:0/2867355172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69080b2ff0 0x7f690815edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.012+0000 7f69077fe700 1 -- 172.21.15.7:0/2867355172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69080b2ff0 msgr2=0x7f690815edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.012+0000 7f69077fe700 1 --2- 172.21.15.7:0/2867355172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69080b2ff0 0x7f690815edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.012+0000 7f69077fe700 1 -- 172.21.15.7:0/2867355172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f690815b8e0 msgr2=0x7f690814efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.012+0000 7f69077fe700 1 --2- 172.21.15.7:0/2867355172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f690815b8e0 0x7f690814efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.012+0000 7f69077fe700 1 -- 172.21.15.7:0/2867355172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f80093c0 con 0x7f690815e4e0 2024-01-27T02:38:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.012+0000 7f6906ffd700 1 --2- 172.21.15.7:0/2867355172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69080b2ff0 0x7f690815edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:38:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.012+0000 7f69067fc700 1 --2- 172.21.15.7:0/2867355172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f690815b8e0 0x7f690814efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:38:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.012+0000 7f69077fe700 1 --2- 172.21.15.7:0/2867355172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f690815e4e0 0x7f690814f520 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6900009fa0 tx=0x7f6900009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.012+0000 7f68eeffd700 1 -- 172.21.15.7:0/2867355172 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690000ed20 con 0x7f690815e4e0 2024-01-27T02:38:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.012+0000 7f68eeffd700 1 -- 172.21.15.7:0/2867355172 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690000ee80 con 0x7f690815e4e0 2024-01-27T02:38:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.012+0000 7f690d744700 1 -- 172.21.15.7:0/2867355172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690815f720 con 0x7f690815e4e0 2024-01-27T02:38:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.012+0000 7f68eeffd700 1 -- 172.21.15.7:0/2867355172 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900014670 con 0x7f690815e4e0 2024-01-27T02:38:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.012+0000 7f690d744700 1 -- 172.21.15.7:0/2867355172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690815fce0 con 0x7f690815e4e0 2024-01-27T02:38:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.013+0000 7f690d744700 1 -- 172.21.15.7:0/2867355172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690816bae0 con 0x7f690815e4e0 2024-01-27T02:38:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.014+0000 7f68eeffd700 1 -- 172.21.15.7:0/2867355172 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6900014a20 con 0x7f690815e4e0 2024-01-27T02:38:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.014+0000 7f68eeffd700 1 --2- 172.21.15.7:0/2867355172 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f68f4066f30 0x7f68f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.014+0000 7f68eeffd700 1 -- 172.21.15.7:0/2867355172 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6900084c60 con 0x7f690815e4e0 2024-01-27T02:38:13.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.015+0000 7f6906ffd700 1 --2- 172.21.15.7:0/2867355172 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f68f4066f30 0x7f68f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:13.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.015+0000 7f6906ffd700 1 --2- 172.21.15.7:0/2867355172 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f68f4066f30 0x7f68f40693e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f69080b2e50 tx=0x7f68f8016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:13.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.019+0000 7f68eeffd700 1 -- 172.21.15.7:0/2867355172 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f690816bae0 con 0x7f690815e4e0 2024-01-27T02:38:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.269+0000 7f690d744700 1 -- 172.21.15.7:0/2867355172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f690816bae0 con 0x7f690815e4e0 2024-01-27T02:38:13.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.269+0000 7f68eeffd700 1 -- 172.21.15.7:0/2867355172 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f690816bae0 con 0x7f690815e4e0 2024-01-27T02:38:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:13.270 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2767759975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2767759975},{"type":"v1","addr":"172.21.15.188:6836","nonce":2767759975}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":5768,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/245473575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":245473575},{"type":"v1","addr":"172.21.15.7:6834","nonce":245473575}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1334164270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1334164270},{"type":"v1","addr":"172.21.15.188:6837","nonce":1334164270}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:37:51.231918+0000","modified":"2024-01-27T02:37:54.851112+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6657},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6657":{"gid":6657,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1251886476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1251886476},{"type":"v1","addr":"172.21.15.7:6835","nonce":1251886476}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2024-01-27T02:38:13.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.272+0000 7f690d744700 1 -- 172.21.15.7:0/2867355172 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f68f4066f30 msgr2=0x7f68f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:13.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.272+0000 7f690d744700 1 --2- 172.21.15.7:0/2867355172 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f68f4066f30 0x7f68f40693e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f69080b2e50 tx=0x7f68f8016000 comp rx=0 tx=0).stop 2024-01-27T02:38:13.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.272+0000 7f690d744700 1 -- 172.21.15.7:0/2867355172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f690815e4e0 msgr2=0x7f690814f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.272+0000 7f690d744700 1 --2- 172.21.15.7:0/2867355172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f690815e4e0 0x7f690814f520 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6900009fa0 tx=0x7f6900009900 comp rx=0 tx=0).stop 2024-01-27T02:38:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.272+0000 7f690d744700 1 -- 172.21.15.7:0/2867355172 shutdown_connections 2024-01-27T02:38:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.272+0000 7f690d744700 1 --2- 172.21.15.7:0/2867355172 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f68f4066f30 0x7f68f40693e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.272+0000 7f690d744700 1 --2- 172.21.15.7:0/2867355172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f690815e4e0 0x7f690814f520 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.272+0000 7f690d744700 1 --2- 172.21.15.7:0/2867355172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69080b2ff0 0x7f690815edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.272+0000 7f690d744700 1 --2- 172.21.15.7:0/2867355172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f690815b8e0 0x7f690814efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.272+0000 7f690d744700 1 -- 172.21.15.7:0/2867355172 >> 172.21.15.7:0/2867355172 conn(0x7f69080679b0 msgr2=0x7f69080b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.272+0000 7f690d744700 1 -- 172.21.15.7:0/2867355172 shutdown_connections 2024-01-27T02:38:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.272+0000 7f690d744700 1 -- 172.21.15.7:0/2867355172 wait complete. 2024-01-27T02:38:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 58 2024-01-27T02:38:13.286 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:38:13.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.417+0000 7fe2410f3700 1 Processor -- start 2024-01-27T02:38:13.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.417+0000 7fe2410f3700 1 -- start start 2024-01-27T02:38:13.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.418+0000 7fe2410f3700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe23c159f40 0x7fe23c0c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:13.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.418+0000 7fe2410f3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe23c0aafc0 0x7fe23c063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:13.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.418+0000 7fe2410f3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe23c0c30c0 con 0x7fe23c0c3a90 2024-01-27T02:38:13.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.418+0000 7fe2410f3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe23c0c4550 con 0x7fe23c0aafc0 2024-01-27T02:38:13.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.418+0000 7fe2410f3700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe23c15e960 con 0x7fe23c159f40 2024-01-27T02:38:13.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.418+0000 7fe23b59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe23c0aafc0 0x7fe23c063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:13.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.418+0000 7fe23b59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe23c0aafc0 0x7fe23c063580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53970/0 (socket says 172.21.15.7:53970) 2024-01-27T02:38:13.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.418+0000 7fe23b59e700 1 -- 172.21.15.7:0/3111557051 learned_addr learned my addr 172.21.15.7:0/3111557051 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:13.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.418+0000 7fe23a59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe23c0c3a90 0x7fe23c0aa8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42198/0 (socket says 172.21.15.7:42198) 2024-01-27T02:38:13.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.418+0000 7fe23ad9d700 1 --2- 172.21.15.7:0/3111557051 >> v2:172.21.15.188:3300/0 conn(0x7fe23c159f40 0x7fe23c0c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:13.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.418+0000 7fe23b59e700 1 -- 172.21.15.7:0/3111557051 >> v2:172.21.15.188:3300/0 conn(0x7fe23c159f40 msgr2=0x7fe23c0c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:13.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.418+0000 7fe23b59e700 1 --2- 172.21.15.7:0/3111557051 >> v2:172.21.15.188:3300/0 conn(0x7fe23c159f40 0x7fe23c0c3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:13.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.418+0000 7fe23b59e700 1 -- 172.21.15.7:0/3111557051 >> v1:172.21.15.7:6789/0 conn(0x7fe23c0c3a90 legacy=0x7fe23c0aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:13.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.418+0000 7fe23b59e700 1 -- 172.21.15.7:0/3111557051 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23c15ec40 con 0x7fe23c0aafc0 2024-01-27T02:38:13.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.418+0000 7fe239d9b700 1 -- 172.21.15.7:0/3111557051 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1643976298 0 0) 0x7fe23c0c30c0 con 0x7fe23c0c3a90 2024-01-27T02:38:13.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.418+0000 7fe23b59e700 1 --2- 172.21.15.7:0/3111557051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe23c0aafc0 0x7fe23c063580 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0cdb90 tx=0x7fe234009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12352fa1fabfff28 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:13.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.419+0000 7fe239d9b700 1 -- 172.21.15.7:0/3111557051 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23400e070 con 0x7fe23c0aafc0 2024-01-27T02:38:13.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.419+0000 7fe239d9b700 1 -- 172.21.15.7:0/3111557051 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23400ba50 con 0x7fe23c0aafc0 2024-01-27T02:38:13.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.419+0000 7fe239d9b700 1 -- 172.21.15.7:0/3111557051 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2340092e0 con 0x7fe23c0aafc0 2024-01-27T02:38:13.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.419+0000 7fe2410f3700 1 -- 172.21.15.7:0/3111557051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe23c0aafc0 msgr2=0x7fe23c063580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:13.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.419+0000 7fe2410f3700 1 --2- 172.21.15.7:0/3111557051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe23c0aafc0 0x7fe23c063580 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0cdb90 tx=0x7fe234009840 comp rx=0 tx=0).stop 2024-01-27T02:38:13.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.419+0000 7fe2410f3700 1 -- 172.21.15.7:0/3111557051 shutdown_connections 2024-01-27T02:38:13.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.419+0000 7fe2410f3700 1 --2- 172.21.15.7:0/3111557051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe23c0aafc0 0x7fe23c063580 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:13.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.419+0000 7fe2410f3700 1 --2- 172.21.15.7:0/3111557051 >> v2:172.21.15.188:3300/0 conn(0x7fe23c159f40 0x7fe23c0c3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:13.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.419+0000 7fe2410f3700 1 -- 172.21.15.7:0/3111557051 >> 172.21.15.7:0/3111557051 conn(0x7fe23c0cdc30 msgr2=0x7fe23c0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:13.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.419+0000 7fe2410f3700 1 -- 172.21.15.7:0/3111557051 shutdown_connections 2024-01-27T02:38:13.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.419+0000 7fe2410f3700 1 -- 172.21.15.7:0/3111557051 wait complete. 2024-01-27T02:38:13.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.421+0000 7fe2410f3700 1 Processor -- start 2024-01-27T02:38:13.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.421+0000 7fe2410f3700 1 -- start start 2024-01-27T02:38:13.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.422+0000 7fe2410f3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe23c0aafc0 0x7fe23c0b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:13.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.422+0000 7fe2410f3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe23c0c3a90 0x7fe23c14a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:13.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.422+0000 7fe2410f3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe23c159f40 0x7fe23c06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:13.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.422+0000 7fe2410f3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c0b3350 con 0x7fe23c159f40 2024-01-27T02:38:13.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.422+0000 7fe2410f3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c0b24a0 con 0x7fe23c0aafc0 2024-01-27T02:38:13.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.422+0000 7fe2410f3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c0b2740 con 0x7fe23c0c3a90 2024-01-27T02:38:13.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.422+0000 7fe23ad9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe23c0aafc0 0x7fe23c0b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:13.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.422+0000 7fe23a59c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe23c0c3a90 0x7fe23c14a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:13.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.422+0000 7fe23ad9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe23c0aafc0 0x7fe23c0b1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:45374/0 (socket says 172.21.15.7:45374) 2024-01-27T02:38:13.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.422+0000 7fe23a59c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe23c0c3a90 0x7fe23c14a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:52128/0 (socket says 172.21.15.7:52128) 2024-01-27T02:38:13.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.422+0000 7fe23ad9d700 1 -- 172.21.15.7:0/2686902602 learned_addr learned my addr 172.21.15.7:0/2686902602 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:13.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.423+0000 7fe23b59e700 1 --2- 172.21.15.7:0/2686902602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe23c159f40 0x7fe23c06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:13.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.423+0000 7fe23b59e700 1 -- 172.21.15.7:0/2686902602 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe23c0c3a90 msgr2=0x7fe23c14a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:13.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.423+0000 7fe23b59e700 1 --2- 172.21.15.7:0/2686902602 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe23c0c3a90 0x7fe23c14a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:13.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.423+0000 7fe23b59e700 1 -- 172.21.15.7:0/2686902602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe23c0aafc0 msgr2=0x7fe23c0b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:13.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.423+0000 7fe23b59e700 1 --2- 172.21.15.7:0/2686902602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe23c0aafc0 0x7fe23c0b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:13.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.423+0000 7fe23b59e700 1 -- 172.21.15.7:0/2686902602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe234009ae0 con 0x7fe23c159f40 2024-01-27T02:38:13.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.423+0000 7fe23b59e700 1 --2- 172.21.15.7:0/2686902602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe23c159f40 0x7fe23c06bce0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fe234015000 tx=0x7fe234018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:13.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.423+0000 7fe222ffd700 1 -- 172.21.15.7:0/2686902602 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23400be60 con 0x7fe23c159f40 2024-01-27T02:38:13.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.423+0000 7fe222ffd700 1 -- 172.21.15.7:0/2686902602 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe234008d60 con 0x7fe23c159f40 2024-01-27T02:38:13.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.423+0000 7fe222ffd700 1 -- 172.21.15.7:0/2686902602 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23401c680 con 0x7fe23c159f40 2024-01-27T02:38:13.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.423+0000 7fe2410f3700 1 -- 172.21.15.7:0/2686902602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23c0b2a20 con 0x7fe23c159f40 2024-01-27T02:38:13.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.423+0000 7fe2410f3700 1 -- 172.21.15.7:0/2686902602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe23c0b2fe0 con 0x7fe23c159f40 2024-01-27T02:38:13.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.424+0000 7fe2410f3700 1 -- 172.21.15.7:0/2686902602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe23c0b34c0 con 0x7fe23c159f40 2024-01-27T02:38:13.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.425+0000 7fe222ffd700 1 -- 172.21.15.7:0/2686902602 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe234003040 con 0x7fe23c159f40 2024-01-27T02:38:13.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.426+0000 7fe222ffd700 1 --2- 172.21.15.7:0/2686902602 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe228066f30 0x7fe2280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:13.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.426+0000 7fe222ffd700 1 -- 172.21.15.7:0/2686902602 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe2340863b0 con 0x7fe23c159f40 2024-01-27T02:38:13.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.426+0000 7fe23ad9d700 1 --2- 172.21.15.7:0/2686902602 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe228066f30 0x7fe2280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:13.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.427+0000 7fe23ad9d700 1 --2- 172.21.15.7:0/2686902602 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe228066f30 0x7fe2280693e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe22c006fd0 tx=0x7fe22c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:13.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.429+0000 7fe222ffd700 1 -- 172.21.15.7:0/2686902602 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe234053600 con 0x7fe23c159f40 2024-01-27T02:38:13.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.680+0000 7fe2410f3700 1 -- 172.21.15.7:0/2686902602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe23c04c710 con 0x7fe23c159f40 2024-01-27T02:38:13.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.681+0000 7fe222ffd700 1 -- 172.21.15.7:0/2686902602 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fe234056c20 con 0x7fe23c159f40 2024-01-27T02:38:13.681 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:13.682 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2767759975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2767759975},{"type":"v1","addr":"172.21.15.188:6836","nonce":2767759975}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":5768,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/245473575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":245473575},{"type":"v1","addr":"172.21.15.7:6834","nonce":245473575}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1334164270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1334164270},{"type":"v1","addr":"172.21.15.188:6837","nonce":1334164270}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:37:51.231918+0000","modified":"2024-01-27T02:37:54.851112+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6657},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6657":{"gid":6657,"name":"a","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1251886476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1251886476},{"type":"v1","addr":"172.21.15.7:6835","nonce":1251886476}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2024-01-27T02:38:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.683+0000 7fe2410f3700 1 -- 172.21.15.7:0/2686902602 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe228066f30 msgr2=0x7fe2280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.683+0000 7fe2410f3700 1 --2- 172.21.15.7:0/2686902602 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe228066f30 0x7fe2280693e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe22c006fd0 tx=0x7fe22c008040 comp rx=0 tx=0).stop 2024-01-27T02:38:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.683+0000 7fe2410f3700 1 -- 172.21.15.7:0/2686902602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe23c159f40 msgr2=0x7fe23c06bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.683+0000 7fe2410f3700 1 --2- 172.21.15.7:0/2686902602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe23c159f40 0x7fe23c06bce0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fe234015000 tx=0x7fe234018000 comp rx=0 tx=0).stop 2024-01-27T02:38:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.683+0000 7fe2410f3700 1 -- 172.21.15.7:0/2686902602 shutdown_connections 2024-01-27T02:38:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.683+0000 7fe2410f3700 1 --2- 172.21.15.7:0/2686902602 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe228066f30 0x7fe2280693e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.683+0000 7fe2410f3700 1 --2- 172.21.15.7:0/2686902602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe23c159f40 0x7fe23c06bce0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.683+0000 7fe2410f3700 1 --2- 172.21.15.7:0/2686902602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe23c0aafc0 0x7fe23c0b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.683+0000 7fe2410f3700 1 --2- 172.21.15.7:0/2686902602 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe23c0c3a90 0x7fe23c14a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.683+0000 7fe2410f3700 1 -- 172.21.15.7:0/2686902602 >> 172.21.15.7:0/2686902602 conn(0x7fe23c0cdc30 msgr2=0x7fe23c0ca6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.683+0000 7fe2410f3700 1 -- 172.21.15.7:0/2686902602 shutdown_connections 2024-01-27T02:38:13.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.683+0000 7fe2410f3700 1 -- 172.21.15.7:0/2686902602 wait complete. 2024-01-27T02:38:13.685 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 58 2024-01-27T02:38:13.697 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:38:13.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.829+0000 7fa13c8e6700 1 Processor -- start 2024-01-27T02:38:13.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.829+0000 7fa13c8e6700 1 -- start start 2024-01-27T02:38:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.830+0000 7fa13c8e6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa138157ca0 0x7fa1380c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.830+0000 7fa13c8e6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1380ca020 con 0x7fa1380c3ae0 2024-01-27T02:38:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.830+0000 7fa13c8e6700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1380c4f40 con 0x7fa1380cdbf0 2024-01-27T02:38:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.830+0000 7fa13c8e6700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1380c5210 con 0x7fa138157ca0 2024-01-27T02:38:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.830+0000 7fa135d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa1380c3ae0 0x7fa1380cd4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42206/0 (socket says 172.21.15.7:42206) 2024-01-27T02:38:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.830+0000 7fa135d9b700 1 -- 172.21.15.7:0/349182901 learned_addr learned my addr 172.21.15.7:0/349182901 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.830+0000 7fa13659c700 1 --2- 172.21.15.7:0/349182901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa138157ca0 0x7fa1380c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.831+0000 7fa13559a700 1 -- 172.21.15.7:0/349182901 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1066980487 0 0) 0x7fa1380ca020 con 0x7fa1380c3ae0 2024-01-27T02:38:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.831+0000 7fa13559a700 1 -- 172.21.15.7:0/349182901 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa124003570 con 0x7fa1380c3ae0 2024-01-27T02:38:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.831+0000 7fa13559a700 1 -- 172.21.15.7:0/349182901 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2304860096 0 0) 0x7fa1380c4f40 con 0x7fa1380cdbf0 2024-01-27T02:38:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.831+0000 7fa13559a700 1 -- 172.21.15.7:0/349182901 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1380ca020 con 0x7fa1380cdbf0 2024-01-27T02:38:13.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.831+0000 7fa13659c700 1 -- 172.21.15.7:0/349182901 >> v1:172.21.15.188:6789/0 conn(0x7fa1380cdbf0 legacy=0x7fa1380ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.831+0000 7fa13659c700 1 -- 172.21.15.7:0/349182901 >> v1:172.21.15.7:6789/0 conn(0x7fa1380c3ae0 legacy=0x7fa1380cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.831+0000 7fa13559a700 1 -- 172.21.15.7:0/349182901 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3169517060 0 0) 0x7fa124003570 con 0x7fa1380c3ae0 2024-01-27T02:38:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.831+0000 7fa13659c700 1 -- 172.21.15.7:0/349182901 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1380c54f0 con 0x7fa138157ca0 2024-01-27T02:38:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.831+0000 7fa13659c700 1 --2- 172.21.15.7:0/349182901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa138157ca0 0x7fa1380c4a00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa128009910 tx=0x7fa1280091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=963e9b71f7a6aa9c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.831+0000 7fa13559a700 1 -- 172.21.15.7:0/349182901 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa120001470 con 0x7fa138157ca0 2024-01-27T02:38:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.831+0000 7fa13559a700 1 -- 172.21.15.7:0/349182901 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa128011420 con 0x7fa138157ca0 2024-01-27T02:38:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.831+0000 7fa13559a700 1 -- 172.21.15.7:0/349182901 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1280117c0 con 0x7fa138157ca0 2024-01-27T02:38:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.832+0000 7fa13c8e6700 1 -- 172.21.15.7:0/349182901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa138157ca0 msgr2=0x7fa1380c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.832+0000 7fa13c8e6700 1 --2- 172.21.15.7:0/349182901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa138157ca0 0x7fa1380c4a00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa128009910 tx=0x7fa1280091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.832+0000 7fa13c8e6700 1 -- 172.21.15.7:0/349182901 shutdown_connections 2024-01-27T02:38:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.832+0000 7fa13c8e6700 1 --2- 172.21.15.7:0/349182901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa138157ca0 0x7fa1380c4a00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.832+0000 7fa13c8e6700 1 -- 172.21.15.7:0/349182901 >> 172.21.15.7:0/349182901 conn(0x7fa138098020 msgr2=0x7fa1380711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.832+0000 7fa13c8e6700 1 -- 172.21.15.7:0/349182901 shutdown_connections 2024-01-27T02:38:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.832+0000 7fa13c8e6700 1 -- 172.21.15.7:0/349182901 wait complete. 2024-01-27T02:38:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.833+0000 7fa13c8e6700 1 Processor -- start 2024-01-27T02:38:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.833+0000 7fa13c8e6700 1 -- start start 2024-01-27T02:38:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.834+0000 7fa13c8e6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1380c3ae0 0x7fa13814ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:13.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.834+0000 7fa13c8e6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1380cdbf0 0x7fa1380c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:13.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.834+0000 7fa13c8e6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa138157ca0 0x7fa13809bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:13.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.834+0000 7fa13c8e6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13814f150 con 0x7fa138157ca0 2024-01-27T02:38:13.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.834+0000 7fa13c8e6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1380c72f0 con 0x7fa1380c3ae0 2024-01-27T02:38:13.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.834+0000 7fa13c8e6700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1380c75d0 con 0x7fa1380cdbf0 2024-01-27T02:38:13.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.834+0000 7fa136d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa138157ca0 0x7fa13809bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:13.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.834+0000 7fa136d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa138157ca0 0x7fa13809bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53982/0 (socket says 172.21.15.7:53982) 2024-01-27T02:38:13.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.834+0000 7fa136d9d700 1 -- 172.21.15.7:0/3992275551 learned_addr learned my addr 172.21.15.7:0/3992275551 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:13.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.834+0000 7fa135d9b700 1 --2- 172.21.15.7:0/3992275551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1380cdbf0 0x7fa1380c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:13.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.834+0000 7fa13659c700 1 --2- 172.21.15.7:0/3992275551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1380c3ae0 0x7fa13814ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:13.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.835+0000 7fa136d9d700 1 -- 172.21.15.7:0/3992275551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1380cdbf0 msgr2=0x7fa1380c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:13.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.835+0000 7fa136d9d700 1 --2- 172.21.15.7:0/3992275551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1380cdbf0 0x7fa1380c6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:13.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.835+0000 7fa136d9d700 1 -- 172.21.15.7:0/3992275551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1380c3ae0 msgr2=0x7fa13814ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:13.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.835+0000 7fa136d9d700 1 --2- 172.21.15.7:0/3992275551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1380c3ae0 0x7fa13814ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:13.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.835+0000 7fa136d9d700 1 -- 172.21.15.7:0/3992275551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa128009420 con 0x7fa138157ca0 2024-01-27T02:38:13.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.835+0000 7fa136d9d700 1 --2- 172.21.15.7:0/3992275551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa138157ca0 0x7fa13809bf90 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fa13000c910 tx=0x7fa130002720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:13.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.835+0000 7fa12e7fc700 1 -- 172.21.15.7:0/3992275551 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13000ce50 con 0x7fa138157ca0 2024-01-27T02:38:13.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.835+0000 7fa12e7fc700 1 -- 172.21.15.7:0/3992275551 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13000a7e0 con 0x7fa138157ca0 2024-01-27T02:38:13.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.835+0000 7fa13c8e6700 1 -- 172.21.15.7:0/3992275551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1380c78b0 con 0x7fa138157ca0 2024-01-27T02:38:13.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.835+0000 7fa12e7fc700 1 -- 172.21.15.7:0/3992275551 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13000aba0 con 0x7fa138157ca0 2024-01-27T02:38:13.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.835+0000 7fa13c8e6700 1 -- 172.21.15.7:0/3992275551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1380c7e70 con 0x7fa138157ca0 2024-01-27T02:38:13.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.836+0000 7fa13c8e6700 1 -- 172.21.15.7:0/3992275551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1380c8090 con 0x7fa138157ca0 2024-01-27T02:38:13.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.837+0000 7fa12e7fc700 1 -- 172.21.15.7:0/3992275551 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa130012420 con 0x7fa138157ca0 2024-01-27T02:38:13.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.837+0000 7fa12e7fc700 1 --2- 172.21.15.7:0/3992275551 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa118066f30 0x7fa1180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:13.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.837+0000 7fa12e7fc700 1 -- 172.21.15.7:0/3992275551 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa130084960 con 0x7fa138157ca0 2024-01-27T02:38:13.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.837+0000 7fa13659c700 1 --2- 172.21.15.7:0/3992275551 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa118066f30 0x7fa1180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:13.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.838+0000 7fa13659c700 1 --2- 172.21.15.7:0/3992275551 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa118066f30 0x7fa1180693e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa1280030c0 tx=0x7fa128016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:13.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:13.841+0000 7fa12e7fc700 1 -- 172.21.15.7:0/3992275551 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa130051bd0 con 0x7fa138157ca0 2024-01-27T02:38:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.048+0000 7fa13c8e6700 1 -- 172.21.15.7:0/3992275551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa13804c710 con 0x7fa138157ca0 2024-01-27T02:38:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.049+0000 7fa12e7fc700 1 -- 172.21.15.7:0/3992275551 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+15324 (secure 0 0 0) 0x7fa1300551f0 con 0x7fa138157ca0 2024-01-27T02:38:14.050 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:14.050 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":120,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:38:07.896480+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:37:49.747694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-27T02:37:50.199870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:38:14.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.052+0000 7fa13c8e6700 1 -- 172.21.15.7:0/3992275551 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa118066f30 msgr2=0x7fa1180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.052+0000 7fa13c8e6700 1 --2- 172.21.15.7:0/3992275551 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa118066f30 0x7fa1180693e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa1280030c0 tx=0x7fa128016000 comp rx=0 tx=0).stop 2024-01-27T02:38:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.052+0000 7fa13c8e6700 1 -- 172.21.15.7:0/3992275551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa138157ca0 msgr2=0x7fa13809bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.052+0000 7fa13c8e6700 1 --2- 172.21.15.7:0/3992275551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa138157ca0 0x7fa13809bf90 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fa13000c910 tx=0x7fa130002720 comp rx=0 tx=0).stop 2024-01-27T02:38:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.052+0000 7fa13c8e6700 1 -- 172.21.15.7:0/3992275551 shutdown_connections 2024-01-27T02:38:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.052+0000 7fa13c8e6700 1 --2- 172.21.15.7:0/3992275551 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa118066f30 0x7fa1180693e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.052+0000 7fa13c8e6700 1 --2- 172.21.15.7:0/3992275551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa138157ca0 0x7fa13809bf90 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.052+0000 7fa13c8e6700 1 --2- 172.21.15.7:0/3992275551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1380c3ae0 0x7fa13814ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.052+0000 7fa13c8e6700 1 --2- 172.21.15.7:0/3992275551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1380cdbf0 0x7fa1380c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.052+0000 7fa13c8e6700 1 -- 172.21.15.7:0/3992275551 >> 172.21.15.7:0/3992275551 conn(0x7fa138098020 msgr2=0x7fa13806b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.052+0000 7fa13c8e6700 1 -- 172.21.15.7:0/3992275551 shutdown_connections 2024-01-27T02:38:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.052+0000 7fa13c8e6700 1 -- 172.21.15.7:0/3992275551 wait complete. 2024-01-27T02:38:14.067 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-27T02:38:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.200+0000 7f1abe7a4700 1 Processor -- start 2024-01-27T02:38:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.200+0000 7f1abe7a4700 1 -- start start 2024-01-27T02:38:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.201+0000 7f1abe7a4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1ab80c3ae0 0x7f1ab80cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.201+0000 7f1abe7a4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ab8158070 0x7f1ab80cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.201+0000 7f1abe7a4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ab80ca020 con 0x7f1ab8157ca0 2024-01-27T02:38:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.201+0000 7f1abe7a4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab80c5210 con 0x7f1ab80c3ae0 2024-01-27T02:38:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.201+0000 7f1abe7a4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab80c54f0 con 0x7f1ab8158070 2024-01-27T02:38:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.201+0000 7f1ab77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1ab80c3ae0 0x7f1ab80cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.201+0000 7f1ab7fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1ab8157ca0 0x7f1ab80c4a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42208/0 (socket says 172.21.15.7:42208) 2024-01-27T02:38:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.201+0000 7f1ab7fff700 1 -- 172.21.15.7:0/2508569778 learned_addr learned my addr 172.21.15.7:0/2508569778 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.201+0000 7f1abcd41700 1 --2- 172.21.15.7:0/2508569778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ab8158070 0x7f1ab80cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.201+0000 7f1ab77fe700 1 -- 172.21.15.7:0/2508569778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ab8158070 msgr2=0x7f1ab80cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.201+0000 7f1ab77fe700 1 --2- 172.21.15.7:0/2508569778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ab8158070 0x7f1ab80cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.201+0000 7f1ab77fe700 1 -- 172.21.15.7:0/2508569778 >> v1:172.21.15.7:6789/0 conn(0x7f1ab8157ca0 legacy=0x7f1ab80c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.201+0000 7f1ab77fe700 1 -- 172.21.15.7:0/2508569778 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab80c57d0 con 0x7f1ab80c3ae0 2024-01-27T02:38:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.201+0000 7f1ab6ffd700 1 -- 172.21.15.7:0/2508569778 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 745715581 0 0) 0x7f1ab80ca020 con 0x7f1ab8157ca0 2024-01-27T02:38:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.201+0000 7f1abcd41700 1 --2- 172.21.15.7:0/2508569778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ab8158070 0x7f1ab80cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:38:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.202+0000 7f1ab77fe700 1 --2- 172.21.15.7:0/2508569778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab80c3ae0 0x7f1ab80cd4e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00098e0 tx=0x7f1aa00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82165b530b5bef8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.202+0000 7f1ab6ffd700 1 -- 172.21.15.7:0/2508569778 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0009e80 con 0x7f1ab80c3ae0 2024-01-27T02:38:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.202+0000 7f1ab6ffd700 1 -- 172.21.15.7:0/2508569778 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa000b860 con 0x7f1ab80c3ae0 2024-01-27T02:38:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.202+0000 7f1ab6ffd700 1 -- 172.21.15.7:0/2508569778 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa000bbe0 con 0x7f1ab80c3ae0 2024-01-27T02:38:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.202+0000 7f1abe7a4700 1 -- 172.21.15.7:0/2508569778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab80c3ae0 msgr2=0x7f1ab80cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.202+0000 7f1abe7a4700 1 --2- 172.21.15.7:0/2508569778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab80c3ae0 0x7f1ab80cd4e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00098e0 tx=0x7f1aa00091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.202+0000 7f1abe7a4700 1 -- 172.21.15.7:0/2508569778 shutdown_connections 2024-01-27T02:38:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.202+0000 7f1abe7a4700 1 --2- 172.21.15.7:0/2508569778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ab8158070 0x7f1ab80cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.202+0000 7f1abe7a4700 1 --2- 172.21.15.7:0/2508569778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab80c3ae0 0x7f1ab80cd4e0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.202+0000 7f1abe7a4700 1 -- 172.21.15.7:0/2508569778 >> 172.21.15.7:0/2508569778 conn(0x7f1ab8098020 msgr2=0x7f1ab80711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.203+0000 7f1abe7a4700 1 -- 172.21.15.7:0/2508569778 shutdown_connections 2024-01-27T02:38:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.203+0000 7f1abe7a4700 1 -- 172.21.15.7:0/2508569778 wait complete. 2024-01-27T02:38:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.204+0000 7f1abe7a4700 1 Processor -- start 2024-01-27T02:38:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.204+0000 7f1abe7a4700 1 -- start start 2024-01-27T02:38:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.205+0000 7f1abe7a4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ab80c3ae0 0x7f1ab80c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.205+0000 7f1abe7a4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ab8157ca0 0x7f1ab80b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.205+0000 7f1abe7a4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab8158070 0x7f1ab8155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.205+0000 7f1abe7a4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab8154a90 con 0x7f1ab8158070 2024-01-27T02:38:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.205+0000 7f1abe7a4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab80c7fb0 con 0x7f1ab80c3ae0 2024-01-27T02:38:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.205+0000 7f1abe7a4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab8155290 con 0x7f1ab8157ca0 2024-01-27T02:38:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.205+0000 7f1abcd41700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab8158070 0x7f1ab8155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.205+0000 7f1abcd41700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab8158070 0x7f1ab8155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54002/0 (socket says 172.21.15.7:54002) 2024-01-27T02:38:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.205+0000 7f1abcd41700 1 -- 172.21.15.7:0/1666315179 learned_addr learned my addr 172.21.15.7:0/1666315179 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.205+0000 7f1ab77fe700 1 --2- 172.21.15.7:0/1666315179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ab8157ca0 0x7f1ab80b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.205+0000 7f1ab7fff700 1 --2- 172.21.15.7:0/1666315179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ab80c3ae0 0x7f1ab80c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.205+0000 7f1abcd41700 1 -- 172.21.15.7:0/1666315179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ab8157ca0 msgr2=0x7f1ab80b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.205+0000 7f1abcd41700 1 --2- 172.21.15.7:0/1666315179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ab8157ca0 0x7f1ab80b3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.205+0000 7f1abcd41700 1 -- 172.21.15.7:0/1666315179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ab80c3ae0 msgr2=0x7f1ab80c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.205+0000 7f1abcd41700 1 --2- 172.21.15.7:0/1666315179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ab80c3ae0 0x7f1ab80c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.205+0000 7f1abcd41700 1 -- 172.21.15.7:0/1666315179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa0009420 con 0x7f1ab8158070 2024-01-27T02:38:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.205+0000 7f1abcd41700 1 --2- 172.21.15.7:0/1666315179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab8158070 0x7f1ab8155720 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000cf20 tx=0x7f1ab0007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.206+0000 7f1aaffff700 1 -- 172.21.15.7:0/1666315179 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab000ae80 con 0x7f1ab8158070 2024-01-27T02:38:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.206+0000 7f1aaffff700 1 -- 172.21.15.7:0/1666315179 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab0016420 con 0x7f1ab8158070 2024-01-27T02:38:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.206+0000 7f1abe7a4700 1 -- 172.21.15.7:0/1666315179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab80c8d70 con 0x7f1ab8158070 2024-01-27T02:38:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.206+0000 7f1aaffff700 1 -- 172.21.15.7:0/1666315179 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab00167e0 con 0x7f1ab8158070 2024-01-27T02:38:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.206+0000 7f1abe7a4700 1 -- 172.21.15.7:0/1666315179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab8154f20 con 0x7f1ab8158070 2024-01-27T02:38:14.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.207+0000 7f1abe7a4700 1 -- 172.21.15.7:0/1666315179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab8154d30 con 0x7f1ab8158070 2024-01-27T02:38:14.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.207+0000 7f1aaffff700 1 -- 172.21.15.7:0/1666315179 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1ab0016bd0 con 0x7f1ab8158070 2024-01-27T02:38:14.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.208+0000 7f1aaffff700 1 --2- 172.21.15.7:0/1666315179 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a98066f30 0x7f1a980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.208+0000 7f1aaffff700 1 -- 172.21.15.7:0/1666315179 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1ab0012070 con 0x7f1ab8158070 2024-01-27T02:38:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.208+0000 7f1ab7fff700 1 --2- 172.21.15.7:0/1666315179 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a98066f30 0x7f1a980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.208+0000 7f1ab7fff700 1 --2- 172.21.15.7:0/1666315179 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a98066f30 0x7f1a980693e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80011e0 tx=0x7f1aa8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.212+0000 7f1aaffff700 1 -- 172.21.15.7:0/1666315179 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ab004e5a0 con 0x7f1ab8158070 2024-01-27T02:38:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.466+0000 7f1abe7a4700 1 -- 172.21.15.7:0/1666315179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1ab804c710 con 0x7f1ab8158070 2024-01-27T02:38:14.467 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-27T02:38:14.467+0000 7fb5c503a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-27T02:38:14.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.472+0000 7f1aaffff700 1 -- 172.21.15.7:0/1666315179 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1ab00521d0 con 0x7f1ab8158070 2024-01-27T02:38:14.475 INFO:tasks.ceph.mds.a.smithi007.stderr: -3595> 2024-01-27T02:37:53.787+0000 7f95be999700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-27T02:38:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.474+0000 7f1abe7a4700 1 -- 172.21.15.7:0/1666315179 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a98066f30 msgr2=0x7f1a980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.474+0000 7f1abe7a4700 1 --2- 172.21.15.7:0/1666315179 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a98066f30 0x7f1a980693e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80011e0 tx=0x7f1aa8009040 comp rx=0 tx=0).stop 2024-01-27T02:38:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.474+0000 7f1abe7a4700 1 -- 172.21.15.7:0/1666315179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab8158070 msgr2=0x7f1ab8155720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.474+0000 7f1abe7a4700 1 --2- 172.21.15.7:0/1666315179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab8158070 0x7f1ab8155720 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000cf20 tx=0x7f1ab0007860 comp rx=0 tx=0).stop 2024-01-27T02:38:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.474+0000 7f1abe7a4700 1 -- 172.21.15.7:0/1666315179 shutdown_connections 2024-01-27T02:38:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.474+0000 7f1abe7a4700 1 --2- 172.21.15.7:0/1666315179 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1a98066f30 0x7f1a980693e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.474+0000 7f1abe7a4700 1 --2- 172.21.15.7:0/1666315179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab8158070 0x7f1ab8155720 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.474+0000 7f1abe7a4700 1 --2- 172.21.15.7:0/1666315179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ab80c3ae0 0x7f1ab80c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.474+0000 7f1abe7a4700 1 --2- 172.21.15.7:0/1666315179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ab8157ca0 0x7f1ab80b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.474+0000 7f1abe7a4700 1 -- 172.21.15.7:0/1666315179 >> 172.21.15.7:0/1666315179 conn(0x7f1ab8098020 msgr2=0x7f1ab80ca4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.475+0000 7f1abe7a4700 1 -- 172.21.15.7:0/1666315179 shutdown_connections 2024-01-27T02:38:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.475+0000 7f1abe7a4700 1 -- 172.21.15.7:0/1666315179 wait complete. 2024-01-27T02:38:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-27T02:38:14.489 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-27T02:38:14.502 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-27T02:38:14.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.623+0000 7fc553d10700 1 Processor -- start 2024-01-27T02:38:14.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.623+0000 7fc553d10700 1 -- start start 2024-01-27T02:38:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.624+0000 7fc553d10700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc54c0cba60 0x7fc54c0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.624+0000 7fc553d10700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54c0679b0 con 0x7fc54c0cbe30 2024-01-27T02:38:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.624+0000 7fc553d10700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54c067c90 con 0x7fc54c06b4e0 2024-01-27T02:38:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.624+0000 7fc553d10700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc54c0cca50 con 0x7fc54c0cba60 2024-01-27T02:38:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.624+0000 7fc551aac700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc54c0cba60 0x7fc54c0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.624+0000 7fc5512ab700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc54c0cbe30 0x7fc54c15b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42230/0 (socket says 172.21.15.7:42230) 2024-01-27T02:38:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.624+0000 7fc5512ab700 1 -- 172.21.15.7:0/3559325974 learned_addr learned my addr 172.21.15.7:0/3559325974 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.625+0000 7fc551aac700 1 -- 172.21.15.7:0/3559325974 >> v1:172.21.15.188:6789/0 conn(0x7fc54c06b4e0 legacy=0x7fc54c06b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.625+0000 7fc551aac700 1 -- 172.21.15.7:0/3559325974 >> v1:172.21.15.7:6789/0 conn(0x7fc54c0cbe30 legacy=0x7fc54c15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.625+0000 7fc550aaa700 1 -- 172.21.15.7:0/3559325974 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3492772900 0 0) 0x7fc54c0679b0 con 0x7fc54c0cbe30 2024-01-27T02:38:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.625+0000 7fc551aac700 1 -- 172.21.15.7:0/3559325974 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c0ccd30 con 0x7fc54c0cba60 2024-01-27T02:38:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.625+0000 7fc551aac700 1 --2- 172.21.15.7:0/3559325974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc54c0cba60 0x7fc54c0ce410 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc5480098d0 tx=0x7fc548009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2a8350d17cc4e8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:14.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.625+0000 7fc550aaa700 1 -- 172.21.15.7:0/3559325974 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548009e70 con 0x7fc54c0cba60 2024-01-27T02:38:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.625+0000 7fc550aaa700 1 -- 172.21.15.7:0/3559325974 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc548005c10 con 0x7fc54c0cba60 2024-01-27T02:38:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.625+0000 7fc550aaa700 1 -- 172.21.15.7:0/3559325974 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54800b610 con 0x7fc54c0cba60 2024-01-27T02:38:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.626+0000 7fc553d10700 1 -- 172.21.15.7:0/3559325974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc54c0cba60 msgr2=0x7fc54c0ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.626+0000 7fc553d10700 1 --2- 172.21.15.7:0/3559325974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc54c0cba60 0x7fc54c0ce410 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc5480098d0 tx=0x7fc548009120 comp rx=0 tx=0).stop 2024-01-27T02:38:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.626+0000 7fc553d10700 1 -- 172.21.15.7:0/3559325974 shutdown_connections 2024-01-27T02:38:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.626+0000 7fc553d10700 1 --2- 172.21.15.7:0/3559325974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc54c0cba60 0x7fc54c0ce410 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.626+0000 7fc553d10700 1 -- 172.21.15.7:0/3559325974 >> 172.21.15.7:0/3559325974 conn(0x7fc54c063580 msgr2=0x7fc54c063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.626+0000 7fc553d10700 1 -- 172.21.15.7:0/3559325974 shutdown_connections 2024-01-27T02:38:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.626+0000 7fc553d10700 1 -- 172.21.15.7:0/3559325974 wait complete. 2024-01-27T02:38:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.627+0000 7fc553d10700 1 Processor -- start 2024-01-27T02:38:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.627+0000 7fc553d10700 1 -- start start 2024-01-27T02:38:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.628+0000 7fc553d10700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc54c06b4e0 0x7fc54c1467f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.628+0000 7fc553d10700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc54c0cba60 0x7fc54c0b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.628+0000 7fc553d10700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc54c0cbe30 0x7fc54c0b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.628+0000 7fc553d10700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c146d30 con 0x7fc54c06b4e0 2024-01-27T02:38:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.628+0000 7fc553d10700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c16b3d0 con 0x7fc54c0cba60 2024-01-27T02:38:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.628+0000 7fc553d10700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c0b4920 con 0x7fc54c0cbe30 2024-01-27T02:38:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.628+0000 7fc551aac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc54c06b4e0 0x7fc54c1467f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.628+0000 7fc551aac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc54c06b4e0 0x7fc54c1467f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54024/0 (socket says 172.21.15.7:54024) 2024-01-27T02:38:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.628+0000 7fc551aac700 1 -- 172.21.15.7:0/149812573 learned_addr learned my addr 172.21.15.7:0/149812573 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.628+0000 7fc5512ab700 1 --2- 172.21.15.7:0/149812573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc54c0cba60 0x7fc54c0b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.628+0000 7fc5522ad700 1 --2- 172.21.15.7:0/149812573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc54c0cbe30 0x7fc54c0b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.628+0000 7fc551aac700 1 -- 172.21.15.7:0/149812573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc54c0cbe30 msgr2=0x7fc54c0b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.628+0000 7fc551aac700 1 --2- 172.21.15.7:0/149812573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc54c0cbe30 0x7fc54c0b4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.628+0000 7fc551aac700 1 -- 172.21.15.7:0/149812573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc54c0cba60 msgr2=0x7fc54c0b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.628+0000 7fc551aac700 1 --2- 172.21.15.7:0/149812573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc54c0cba60 0x7fc54c0b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.628+0000 7fc551aac700 1 -- 172.21.15.7:0/149812573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5480093c0 con 0x7fc54c06b4e0 2024-01-27T02:38:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.628+0000 7fc5512ab700 1 --2- 172.21.15.7:0/149812573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc54c0cba60 0x7fc54c0b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:38:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.629+0000 7fc551aac700 1 --2- 172.21.15.7:0/149812573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc54c06b4e0 0x7fc54c1467f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc548009390 tx=0x7fc548015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.629+0000 7fc541ffb700 1 -- 172.21.15.7:0/149812573 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548006570 con 0x7fc54c06b4e0 2024-01-27T02:38:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.629+0000 7fc541ffb700 1 -- 172.21.15.7:0/149812573 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5480066d0 con 0x7fc54c06b4e0 2024-01-27T02:38:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.629+0000 7fc541ffb700 1 -- 172.21.15.7:0/149812573 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54801c430 con 0x7fc54c06b4e0 2024-01-27T02:38:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.629+0000 7fc553d10700 1 -- 172.21.15.7:0/149812573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54c16aaa0 con 0x7fc54c06b4e0 2024-01-27T02:38:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.629+0000 7fc553d10700 1 -- 172.21.15.7:0/149812573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54c16b060 con 0x7fc54c06b4e0 2024-01-27T02:38:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.630+0000 7fc553d10700 1 -- 172.21.15.7:0/149812573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54c072060 con 0x7fc54c06b4e0 2024-01-27T02:38:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.630+0000 7fc541ffb700 1 -- 172.21.15.7:0/149812573 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc548023070 con 0x7fc54c06b4e0 2024-01-27T02:38:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.631+0000 7fc541ffb700 1 --2- 172.21.15.7:0/149812573 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc530066f30 0x7fc5300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.631+0000 7fc541ffb700 1 -- 172.21.15.7:0/149812573 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fc5480865c0 con 0x7fc54c06b4e0 2024-01-27T02:38:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.631+0000 7fc5512ab700 1 --2- 172.21.15.7:0/149812573 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc530066f30 0x7fc5300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.632+0000 7fc5512ab700 1 --2- 172.21.15.7:0/149812573 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc530066f30 0x7fc5300693e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc54c0b3bb0 tx=0x7fc53c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.635+0000 7fc541ffb700 1 -- 172.21.15.7:0/149812573 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc5480537e0 con 0x7fc54c06b4e0 2024-01-27T02:38:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:14.890+0000 7fc553d10700 1 -- 172.21.15.7:0/149812573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc54c06d5c0 con 0x7fc54c06b4e0 2024-01-27T02:38:15.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.471+0000 7fc541ffb700 1 -- 172.21.15.7:0/149812573 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc548056e00 con 0x7fc54c06b4e0 2024-01-27T02:38:15.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.474+0000 7fc553d10700 1 -- 172.21.15.7:0/149812573 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc530066f30 msgr2=0x7fc5300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:15.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.474+0000 7fc553d10700 1 --2- 172.21.15.7:0/149812573 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc530066f30 0x7fc5300693e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc54c0b3bb0 tx=0x7fc53c009040 comp rx=0 tx=0).stop 2024-01-27T02:38:15.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.474+0000 7fc553d10700 1 -- 172.21.15.7:0/149812573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc54c06b4e0 msgr2=0x7fc54c1467f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:15.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.474+0000 7fc553d10700 1 --2- 172.21.15.7:0/149812573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc54c06b4e0 0x7fc54c1467f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc548009390 tx=0x7fc548015000 comp rx=0 tx=0).stop 2024-01-27T02:38:15.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.474+0000 7fc553d10700 1 -- 172.21.15.7:0/149812573 shutdown_connections 2024-01-27T02:38:15.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.474+0000 7fc553d10700 1 --2- 172.21.15.7:0/149812573 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc530066f30 0x7fc5300693e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:15.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.474+0000 7fc553d10700 1 --2- 172.21.15.7:0/149812573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc54c0cbe30 0x7fc54c0b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:15.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.474+0000 7fc553d10700 1 --2- 172.21.15.7:0/149812573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc54c06b4e0 0x7fc54c1467f0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:15.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.474+0000 7fc553d10700 1 --2- 172.21.15.7:0/149812573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc54c0cba60 0x7fc54c0b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:15.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.474+0000 7fc553d10700 1 -- 172.21.15.7:0/149812573 >> 172.21.15.7:0/149812573 conn(0x7fc54c063580 msgr2=0x7fc54c0cc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:15.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.474+0000 7fc553d10700 1 -- 172.21.15.7:0/149812573 shutdown_connections 2024-01-27T02:38:15.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.474+0000 7fc553d10700 1 -- 172.21.15.7:0/149812573 wait complete. 2024-01-27T02:38:15.490 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-27T02:38:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.673+0000 7f9db0776700 1 Processor -- start 2024-01-27T02:38:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.673+0000 7f9db0776700 1 -- start start 2024-01-27T02:38:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.674+0000 7f9db0776700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9da806e9f0 0x7f9da80cca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.674+0000 7f9db0776700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da8157f30 0x7f9da815b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.674+0000 7f9db0776700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da81583b0 con 0x7f9da8157b60 2024-01-27T02:38:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.674+0000 7f9db0776700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da80cd0c0 con 0x7f9da806e9f0 2024-01-27T02:38:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.674+0000 7f9db0776700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da80cd3a0 con 0x7f9da8157f30 2024-01-27T02:38:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.674+0000 7f9dae512700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9da8157b60 0x7f9da806e2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42234/0 (socket says 172.21.15.7:42234) 2024-01-27T02:38:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.674+0000 7f9dae512700 1 -- 172.21.15.7:0/81130451 learned_addr learned my addr 172.21.15.7:0/81130451 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.674+0000 7f9daed13700 1 --2- 172.21.15.7:0/81130451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da8157f30 0x7f9da815b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.674+0000 7f9dadd11700 1 --2- 172.21.15.7:0/81130451 >> v2:172.21.15.188:3300/0 conn(0x7f9da806e9f0 0x7f9da80cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.674+0000 7f9dad510700 1 -- 172.21.15.7:0/81130451 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344444550 0 0) 0x7f9da81583b0 con 0x7f9da8157b60 2024-01-27T02:38:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.675+0000 7f9dad510700 1 -- 172.21.15.7:0/81130451 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d94003570 con 0x7f9da8157b60 2024-01-27T02:38:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.675+0000 7f9dad510700 1 -- 172.21.15.7:0/81130451 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2923953328 0 0) 0x7f9d94003570 con 0x7f9da8157b60 2024-01-27T02:38:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.675+0000 7f9dadd11700 1 -- 172.21.15.7:0/81130451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da8157f30 msgr2=0x7f9da815b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.675+0000 7f9dadd11700 1 --2- 172.21.15.7:0/81130451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da8157f30 0x7f9da815b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.675+0000 7f9dadd11700 1 -- 172.21.15.7:0/81130451 >> v1:172.21.15.7:6789/0 conn(0x7f9da8157b60 legacy=0x7f9da806e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.675+0000 7f9dadd11700 1 -- 172.21.15.7:0/81130451 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da80cd680 con 0x7f9da806e9f0 2024-01-27T02:38:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.675+0000 7f9dadd11700 1 --2- 172.21.15.7:0/81130451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9da806e9f0 0x7f9da80cca30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009910 tx=0x7f9d980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=458a36e4d40c197a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:15.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.675+0000 7f9dad510700 1 -- 172.21.15.7:0/81130451 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da4001470 con 0x7f9da806e9f0 2024-01-27T02:38:15.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.675+0000 7f9dad510700 1 -- 172.21.15.7:0/81130451 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d98011420 con 0x7f9da806e9f0 2024-01-27T02:38:15.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.675+0000 7f9dad510700 1 -- 172.21.15.7:0/81130451 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d980117a0 con 0x7f9da806e9f0 2024-01-27T02:38:15.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.676+0000 7f9db0776700 1 -- 172.21.15.7:0/81130451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9da806e9f0 msgr2=0x7f9da80cca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:15.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.676+0000 7f9db0776700 1 --2- 172.21.15.7:0/81130451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9da806e9f0 0x7f9da80cca30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009910 tx=0x7f9d980091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:15.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.676+0000 7f9db0776700 1 -- 172.21.15.7:0/81130451 shutdown_connections 2024-01-27T02:38:15.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.676+0000 7f9db0776700 1 --2- 172.21.15.7:0/81130451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da8157f30 0x7f9da815b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:15.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.676+0000 7f9db0776700 1 --2- 172.21.15.7:0/81130451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9da806e9f0 0x7f9da80cca30 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:15.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.676+0000 7f9db0776700 1 -- 172.21.15.7:0/81130451 >> 172.21.15.7:0/81130451 conn(0x7f9da80cdc30 msgr2=0x7f9da80ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:15.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.676+0000 7f9db0776700 1 -- 172.21.15.7:0/81130451 shutdown_connections 2024-01-27T02:38:15.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.676+0000 7f9db0776700 1 -- 172.21.15.7:0/81130451 wait complete. 2024-01-27T02:38:15.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.677+0000 7f9db0776700 1 Processor -- start 2024-01-27T02:38:15.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.677+0000 7f9db0776700 1 -- start start 2024-01-27T02:38:15.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.678+0000 7f9db0776700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da806e9f0 0x7f9da80c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.678+0000 7f9db0776700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9da8157b60 0x7f9da8139610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.678+0000 7f9db0776700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9da8157f30 0x7f9da816e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.678+0000 7f9db0776700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da80c7be0 con 0x7f9da8157b60 2024-01-27T02:38:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.678+0000 7f9db0776700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da816dde0 con 0x7f9da8157f30 2024-01-27T02:38:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.678+0000 7f9db0776700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da80c73d0 con 0x7f9da806e9f0 2024-01-27T02:38:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.678+0000 7f9dadd11700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9da8157b60 0x7f9da8139610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.678+0000 7f9dadd11700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9da8157b60 0x7f9da8139610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54026/0 (socket says 172.21.15.7:54026) 2024-01-27T02:38:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.678+0000 7f9dadd11700 1 -- 172.21.15.7:0/24975532 learned_addr learned my addr 172.21.15.7:0/24975532 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.678+0000 7f9daed13700 1 --2- 172.21.15.7:0/24975532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9da8157f30 0x7f9da816e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.678+0000 7f9dae512700 1 --2- 172.21.15.7:0/24975532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da806e9f0 0x7f9da80c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.679+0000 7f9dadd11700 1 -- 172.21.15.7:0/24975532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da806e9f0 msgr2=0x7f9da80c6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.679+0000 7f9dadd11700 1 --2- 172.21.15.7:0/24975532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da806e9f0 0x7f9da80c6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.679+0000 7f9dadd11700 1 -- 172.21.15.7:0/24975532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9da8157f30 msgr2=0x7f9da816e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:15.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.679+0000 7f9dadd11700 1 --2- 172.21.15.7:0/24975532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9da8157f30 0x7f9da816e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:15.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.679+0000 7f9dadd11700 1 -- 172.21.15.7:0/24975532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d98009420 con 0x7f9da8157b60 2024-01-27T02:38:15.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.679+0000 7f9dadd11700 1 --2- 172.21.15.7:0/24975532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9da8157b60 0x7f9da8139610 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f9d98002000 tx=0x7f9d98016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:15.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.679+0000 7f9d9e7fc700 1 -- 172.21.15.7:0/24975532 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9801baa0 con 0x7f9da8157b60 2024-01-27T02:38:15.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.679+0000 7f9d9e7fc700 1 -- 172.21.15.7:0/24975532 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9801bc00 con 0x7f9da8157b60 2024-01-27T02:38:15.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.679+0000 7f9db0776700 1 -- 172.21.15.7:0/24975532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da8153bc0 con 0x7f9da8157b60 2024-01-27T02:38:15.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.679+0000 7f9d9e7fc700 1 -- 172.21.15.7:0/24975532 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9801c450 con 0x7f9da8157b60 2024-01-27T02:38:15.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.679+0000 7f9db0776700 1 -- 172.21.15.7:0/24975532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da80c7900 con 0x7f9da8157b60 2024-01-27T02:38:15.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.680+0000 7f9db0776700 1 -- 172.21.15.7:0/24975532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da816df90 con 0x7f9da8157b60 2024-01-27T02:38:15.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.681+0000 7f9d9e7fc700 1 -- 172.21.15.7:0/24975532 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9d9801c800 con 0x7f9da8157b60 2024-01-27T02:38:15.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.681+0000 7f9d9e7fc700 1 --2- 172.21.15.7:0/24975532 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9d8c066f30 0x7f9d8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:15.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.681+0000 7f9d9e7fc700 1 -- 172.21.15.7:0/24975532 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f9d98022030 con 0x7f9da8157b60 2024-01-27T02:38:15.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.681+0000 7f9dae512700 1 --2- 172.21.15.7:0/24975532 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9d8c066f30 0x7f9d8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:15.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.682+0000 7f9dae512700 1 --2- 172.21.15.7:0/24975532 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9d8c066f30 0x7f9d8c0693e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9da4004e40 tx=0x7f9da4001aa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:15.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.686+0000 7f9d9e7fc700 1 -- 172.21.15.7:0/24975532 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9da816df90 con 0x7f9da8157b60 2024-01-27T02:38:15.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:15.892+0000 7f9db0776700 1 -- 172.21.15.7:0/24975532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9da816df50 con 0x7f9da8157b60 2024-01-27T02:38:16.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.475+0000 7f9d9e7fc700 1 -- 172.21.15.7:0/24975532 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v122) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9d9802c020 con 0x7f9da8157b60 2024-01-27T02:38:16.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.478+0000 7f9db0776700 1 -- 172.21.15.7:0/24975532 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9d8c066f30 msgr2=0x7f9d8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:16.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.478+0000 7f9db0776700 1 --2- 172.21.15.7:0/24975532 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9d8c066f30 0x7f9d8c0693e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9da4004e40 tx=0x7f9da4001aa0 comp rx=0 tx=0).stop 2024-01-27T02:38:16.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.478+0000 7f9db0776700 1 -- 172.21.15.7:0/24975532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9da8157b60 msgr2=0x7f9da8139610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:16.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.478+0000 7f9db0776700 1 --2- 172.21.15.7:0/24975532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9da8157b60 0x7f9da8139610 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f9d98002000 tx=0x7f9d98016000 comp rx=0 tx=0).stop 2024-01-27T02:38:16.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.478+0000 7f9db0776700 1 -- 172.21.15.7:0/24975532 shutdown_connections 2024-01-27T02:38:16.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.478+0000 7f9db0776700 1 --2- 172.21.15.7:0/24975532 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9d8c066f30 0x7f9d8c0693e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:16.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.478+0000 7f9db0776700 1 --2- 172.21.15.7:0/24975532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9da8157f30 0x7f9da816e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:16.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.479+0000 7f9db0776700 1 --2- 172.21.15.7:0/24975532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da806e9f0 0x7f9da80c6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:16.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.479+0000 7f9db0776700 1 --2- 172.21.15.7:0/24975532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9da8157b60 0x7f9da8139610 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:16.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.479+0000 7f9db0776700 1 -- 172.21.15.7:0/24975532 >> 172.21.15.7:0/24975532 conn(0x7f9da80cdc30 msgr2=0x7f9da8065700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:16.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.479+0000 7f9db0776700 1 -- 172.21.15.7:0/24975532 shutdown_connections 2024-01-27T02:38:16.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.479+0000 7f9db0776700 1 -- 172.21.15.7:0/24975532 wait complete. 2024-01-27T02:38:16.480 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-27T02:38:16.494 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-27T02:38:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.639+0000 7fd29a4b2700 1 Processor -- start 2024-01-27T02:38:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.639+0000 7fd29a4b2700 1 -- start start 2024-01-27T02:38:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.640+0000 7fd29a4b2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2940b1b70 0x7fd29406e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.640+0000 7fd29a4b2700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd29415a960 0x7fd2941567b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.640+0000 7fd29a4b2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29406ea80 con 0x7fd2940b1f40 2024-01-27T02:38:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.640+0000 7fd29a4b2700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd29406ed20 con 0x7fd29415a960 2024-01-27T02:38:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.640+0000 7fd29a4b2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29406f000 con 0x7fd2940b1b70 2024-01-27T02:38:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.640+0000 7fd298a4f700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd2940b1f40 0x7fd29406a3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42236/0 (socket says 172.21.15.7:42236) 2024-01-27T02:38:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.640+0000 7fd298a4f700 1 -- 172.21.15.7:0/1652302143 learned_addr learned my addr 172.21.15.7:0/1652302143 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.640+0000 7fd293fff700 1 --2- 172.21.15.7:0/1652302143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2940b1b70 0x7fd29406e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.640+0000 7fd2937fe700 1 --2- 172.21.15.7:0/1652302143 >> v2:172.21.15.188:3300/0 conn(0x7fd29415a960 0x7fd2941567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.641+0000 7fd292ffd700 1 -- 172.21.15.7:0/1652302143 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668632779 0 0) 0x7fd29406ea80 con 0x7fd2940b1f40 2024-01-27T02:38:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.641+0000 7fd292ffd700 1 -- 172.21.15.7:0/1652302143 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd27c003570 con 0x7fd2940b1f40 2024-01-27T02:38:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.641+0000 7fd293fff700 1 -- 172.21.15.7:0/1652302143 >> v2:172.21.15.188:3300/0 conn(0x7fd29415a960 msgr2=0x7fd2941567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.641+0000 7fd293fff700 1 --2- 172.21.15.7:0/1652302143 >> v2:172.21.15.188:3300/0 conn(0x7fd29415a960 0x7fd2941567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.641+0000 7fd293fff700 1 -- 172.21.15.7:0/1652302143 >> v1:172.21.15.7:6789/0 conn(0x7fd2940b1f40 legacy=0x7fd29406a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.641+0000 7fd293fff700 1 -- 172.21.15.7:0/1652302143 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29406f2e0 con 0x7fd2940b1b70 2024-01-27T02:38:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.641+0000 7fd2937fe700 1 --2- 172.21.15.7:0/1652302143 >> v2:172.21.15.188:3300/0 conn(0x7fd29415a960 0x7fd2941567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:38:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.641+0000 7fd293fff700 1 --2- 172.21.15.7:0/1652302143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2940b1b70 0x7fd29406e540 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd2840098e0 tx=0x7fd2840091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=621c5e8060907de1 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.641+0000 7fd292ffd700 1 -- 172.21.15.7:0/1652302143 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd284011630 con 0x7fd2940b1b70 2024-01-27T02:38:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.641+0000 7fd292ffd700 1 -- 172.21.15.7:0/1652302143 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd284011790 con 0x7fd2940b1b70 2024-01-27T02:38:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.641+0000 7fd292ffd700 1 -- 172.21.15.7:0/1652302143 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd284011b30 con 0x7fd2940b1b70 2024-01-27T02:38:16.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.642+0000 7fd29a4b2700 1 -- 172.21.15.7:0/1652302143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2940b1b70 msgr2=0x7fd29406e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.642+0000 7fd29a4b2700 1 --2- 172.21.15.7:0/1652302143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2940b1b70 0x7fd29406e540 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd2840098e0 tx=0x7fd2840091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.642+0000 7fd29a4b2700 1 -- 172.21.15.7:0/1652302143 shutdown_connections 2024-01-27T02:38:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.642+0000 7fd29a4b2700 1 --2- 172.21.15.7:0/1652302143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2940b1b70 0x7fd29406e540 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.642+0000 7fd29a4b2700 1 --2- 172.21.15.7:0/1652302143 >> v2:172.21.15.188:3300/0 conn(0x7fd29415a960 0x7fd2941567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.642+0000 7fd29a4b2700 1 -- 172.21.15.7:0/1652302143 >> 172.21.15.7:0/1652302143 conn(0x7fd294068480 msgr2=0x7fd29406be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.642+0000 7fd29a4b2700 1 -- 172.21.15.7:0/1652302143 shutdown_connections 2024-01-27T02:38:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.642+0000 7fd29a4b2700 1 -- 172.21.15.7:0/1652302143 wait complete. 2024-01-27T02:38:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.643+0000 7fd29a4b2700 1 Processor -- start 2024-01-27T02:38:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.643+0000 7fd29a4b2700 1 -- start start 2024-01-27T02:38:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.644+0000 7fd29a4b2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2940b1b70 0x7fd2940aada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.644+0000 7fd29a4b2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2940b1f40 0x7fd2941519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.644+0000 7fd29a4b2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd29415a960 0x7fd2940af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.644+0000 7fd29a4b2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2940ad1a0 con 0x7fd29415a960 2024-01-27T02:38:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.644+0000 7fd29a4b2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2940ae000 con 0x7fd2940b1f40 2024-01-27T02:38:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.644+0000 7fd29a4b2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2940ab2e0 con 0x7fd2940b1b70 2024-01-27T02:38:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.644+0000 7fd298a4f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd29415a960 0x7fd2940af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.644+0000 7fd298a4f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd29415a960 0x7fd2940af620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54042/0 (socket says 172.21.15.7:54042) 2024-01-27T02:38:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.644+0000 7fd298a4f700 1 -- 172.21.15.7:0/3465712665 learned_addr learned my addr 172.21.15.7:0/3465712665 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.644+0000 7fd293fff700 1 --2- 172.21.15.7:0/3465712665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2940b1b70 0x7fd2940aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.645+0000 7fd2937fe700 1 --2- 172.21.15.7:0/3465712665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2940b1f40 0x7fd2941519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.645+0000 7fd298a4f700 1 -- 172.21.15.7:0/3465712665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2940b1b70 msgr2=0x7fd2940aada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.645+0000 7fd298a4f700 1 --2- 172.21.15.7:0/3465712665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2940b1b70 0x7fd2940aada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.645+0000 7fd298a4f700 1 -- 172.21.15.7:0/3465712665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2940b1f40 msgr2=0x7fd2941519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.645+0000 7fd298a4f700 1 --2- 172.21.15.7:0/3465712665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2940b1f40 0x7fd2941519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.645+0000 7fd298a4f700 1 -- 172.21.15.7:0/3465712665 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd288009420 con 0x7fd29415a960 2024-01-27T02:38:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.645+0000 7fd2937fe700 1 --2- 172.21.15.7:0/3465712665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2940b1f40 0x7fd2941519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:38:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.645+0000 7fd298a4f700 1 --2- 172.21.15.7:0/3465712665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd29415a960 0x7fd2940af620 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00cc40 tx=0x7fd28c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:16.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.645+0000 7fd27bfff700 1 -- 172.21.15.7:0/3465712665 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c00a940 con 0x7fd29415a960 2024-01-27T02:38:16.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.645+0000 7fd27bfff700 1 -- 172.21.15.7:0/3465712665 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c00aaa0 con 0x7fd29415a960 2024-01-27T02:38:16.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.645+0000 7fd27bfff700 1 -- 172.21.15.7:0/3465712665 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c00ae20 con 0x7fd29415a960 2024-01-27T02:38:16.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.645+0000 7fd29a4b2700 1 -- 172.21.15.7:0/3465712665 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd284009420 con 0x7fd29415a960 2024-01-27T02:38:16.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.645+0000 7fd29a4b2700 1 -- 172.21.15.7:0/3465712665 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2940ad9b0 con 0x7fd29415a960 2024-01-27T02:38:16.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.646+0000 7fd29a4b2700 1 -- 172.21.15.7:0/3465712665 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd29404c710 con 0x7fd29415a960 2024-01-27T02:38:16.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.646+0000 7fd27bfff700 1 -- 172.21.15.7:0/3465712665 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd28c007300 con 0x7fd29415a960 2024-01-27T02:38:16.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.647+0000 7fd27bfff700 1 --2- 172.21.15.7:0/3465712665 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd280066f30 0x7fd2800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:16.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.647+0000 7fd27bfff700 1 -- 172.21.15.7:0/3465712665 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5353+0+0 (secure 0 0 0) 0x7fd28c084850 con 0x7fd29415a960 2024-01-27T02:38:16.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.647+0000 7fd293fff700 1 --2- 172.21.15.7:0/3465712665 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd280066f30 0x7fd2800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:16.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.648+0000 7fd293fff700 1 --2- 172.21.15.7:0/3465712665 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd280066f30 0x7fd2800693e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd2840091e0 tx=0x7fd284017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:16.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.652+0000 7fd27bfff700 1 -- 172.21.15.7:0/3465712665 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd28c051c10 con 0x7fd29415a960 2024-01-27T02:38:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:16.858+0000 7fd29a4b2700 1 -- 172.21.15.7:0/3465712665 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd29406cb40 con 0x7fd29415a960 2024-01-27T02:38:17.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.478+0000 7fd27bfff700 1 -- 172.21.15.7:0/3465712665 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v123) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd28c055230 con 0x7fd29415a960 2024-01-27T02:38:17.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.481+0000 7fd29a4b2700 1 -- 172.21.15.7:0/3465712665 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd280066f30 msgr2=0x7fd2800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:17.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.482+0000 7fd29a4b2700 1 --2- 172.21.15.7:0/3465712665 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd280066f30 0x7fd2800693e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd2840091e0 tx=0x7fd284017000 comp rx=0 tx=0).stop 2024-01-27T02:38:17.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.482+0000 7fd29a4b2700 1 -- 172.21.15.7:0/3465712665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd29415a960 msgr2=0x7fd2940af620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:17.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.482+0000 7fd29a4b2700 1 --2- 172.21.15.7:0/3465712665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd29415a960 0x7fd2940af620 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00cc40 tx=0x7fd28c001a40 comp rx=0 tx=0).stop 2024-01-27T02:38:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.486+0000 7fd29a4b2700 1 -- 172.21.15.7:0/3465712665 shutdown_connections 2024-01-27T02:38:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.486+0000 7fd29a4b2700 1 --2- 172.21.15.7:0/3465712665 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd280066f30 0x7fd2800693e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.486+0000 7fd29a4b2700 1 --2- 172.21.15.7:0/3465712665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd29415a960 0x7fd2940af620 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.486+0000 7fd29a4b2700 1 --2- 172.21.15.7:0/3465712665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2940b1b70 0x7fd2940aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.486+0000 7fd29a4b2700 1 --2- 172.21.15.7:0/3465712665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2940b1f40 0x7fd2941519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.486+0000 7fd29a4b2700 1 -- 172.21.15.7:0/3465712665 >> 172.21.15.7:0/3465712665 conn(0x7fd294068480 msgr2=0x7fd2941598a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.486+0000 7fd29a4b2700 1 -- 172.21.15.7:0/3465712665 shutdown_connections 2024-01-27T02:38:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.486+0000 7fd29a4b2700 1 -- 172.21.15.7:0/3465712665 wait complete. 2024-01-27T02:38:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-27T02:38:17.502 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-27T02:38:17.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.638+0000 7f7ed6e25700 1 Processor -- start 2024-01-27T02:38:17.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.638+0000 7f7ed6e25700 1 -- start start 2024-01-27T02:38:17.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.639+0000 7f7ed6e25700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ed0063c90 0x7f7ed0164ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:17.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.639+0000 7f7ed6e25700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ed00cc290 con 0x7f7ed00aa8b0 2024-01-27T02:38:17.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.639+0000 7f7ed6e25700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ed00cb960 con 0x7f7ed0143670 2024-01-27T02:38:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.639+0000 7f7ed6e25700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ed00cbc40 con 0x7f7ed0063c90 2024-01-27T02:38:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.639+0000 7f7ed53c2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ed0063c90 0x7f7ed0164ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.639+0000 7f7ec7fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7ed00aa8b0 0x7f7ed00c8230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42248/0 (socket says 172.21.15.7:42248) 2024-01-27T02:38:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.639+0000 7f7ec7fff700 1 -- 172.21.15.7:0/220338715 learned_addr learned my addr 172.21.15.7:0/220338715 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.639+0000 7f7ed53c2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ed0063c90 0x7f7ed0164ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54052/0 (socket says 172.21.15.7:54052) 2024-01-27T02:38:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.640+0000 7f7ed53c2700 1 -- 172.21.15.7:0/220338715 >> v1:172.21.15.188:6789/0 conn(0x7f7ed0143670 legacy=0x7f7ed0063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.640+0000 7f7ed53c2700 1 -- 172.21.15.7:0/220338715 >> v1:172.21.15.7:6789/0 conn(0x7f7ed00aa8b0 legacy=0x7f7ed00c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.640+0000 7f7ed53c2700 1 -- 172.21.15.7:0/220338715 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed00cbf20 con 0x7f7ed0063c90 2024-01-27T02:38:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.640+0000 7f7ec77fe700 1 -- 172.21.15.7:0/220338715 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736182205 0 0) 0x7f7ed00cc290 con 0x7f7ed00aa8b0 2024-01-27T02:38:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.640+0000 7f7ed53c2700 1 --2- 172.21.15.7:0/220338715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ed0063c90 0x7f7ed0164ff0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009e60 tx=0x7f7ec8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad1889b197b8a36 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.640+0000 7f7ec77fe700 1 -- 172.21.15.7:0/220338715 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec800e070 con 0x7f7ed0063c90 2024-01-27T02:38:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.640+0000 7f7ec77fe700 1 -- 172.21.15.7:0/220338715 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec800bc30 con 0x7f7ed0063c90 2024-01-27T02:38:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.640+0000 7f7ec77fe700 1 -- 172.21.15.7:0/220338715 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8012430 con 0x7f7ed0063c90 2024-01-27T02:38:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.640+0000 7f7ed6e25700 1 -- 172.21.15.7:0/220338715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ed0063c90 msgr2=0x7f7ed0164ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.640+0000 7f7ed6e25700 1 --2- 172.21.15.7:0/220338715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ed0063c90 0x7f7ed0164ff0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009e60 tx=0x7f7ec8009780 comp rx=0 tx=0).stop 2024-01-27T02:38:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.641+0000 7f7ed6e25700 1 -- 172.21.15.7:0/220338715 shutdown_connections 2024-01-27T02:38:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.641+0000 7f7ed6e25700 1 --2- 172.21.15.7:0/220338715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ed0063c90 0x7f7ed0164ff0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.641+0000 7f7ed6e25700 1 -- 172.21.15.7:0/220338715 >> 172.21.15.7:0/220338715 conn(0x7f7ed00cdbf0 msgr2=0x7f7ed00cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.641+0000 7f7ed6e25700 1 -- 172.21.15.7:0/220338715 shutdown_connections 2024-01-27T02:38:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.641+0000 7f7ed6e25700 1 -- 172.21.15.7:0/220338715 wait complete. 2024-01-27T02:38:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.642+0000 7f7ed6e25700 1 Processor -- start 2024-01-27T02:38:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.642+0000 7f7ed6e25700 1 -- start start 2024-01-27T02:38:17.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.643+0000 7f7ed6e25700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ed0063c90 0x7f7ed00b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.643+0000 7f7ed6e25700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ed00aa8b0 0x7f7ed015eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.643+0000 7f7ed6e25700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ed0143670 0x7f7ed015b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.643+0000 7f7ed6e25700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed00b3a40 con 0x7f7ed00aa8b0 2024-01-27T02:38:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.643+0000 7f7ed6e25700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed00b3040 con 0x7f7ed0143670 2024-01-27T02:38:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.643+0000 7f7ed6e25700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed015f420 con 0x7f7ed0063c90 2024-01-27T02:38:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.643+0000 7f7ec7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ed00aa8b0 0x7f7ed015eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.643+0000 7f7ec7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ed00aa8b0 0x7f7ed015eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54066/0 (socket says 172.21.15.7:54066) 2024-01-27T02:38:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.643+0000 7f7ec7fff700 1 -- 172.21.15.7:0/1968695784 learned_addr learned my addr 172.21.15.7:0/1968695784 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.643+0000 7f7ed4bc1700 1 --2- 172.21.15.7:0/1968695784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ed0063c90 0x7f7ed00b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.643+0000 7f7ed53c2700 1 --2- 172.21.15.7:0/1968695784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ed0143670 0x7f7ed015b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.643+0000 7f7ec7fff700 1 -- 172.21.15.7:0/1968695784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ed0063c90 msgr2=0x7f7ed00b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.643+0000 7f7ec7fff700 1 --2- 172.21.15.7:0/1968695784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ed0063c90 0x7f7ed00b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.643+0000 7f7ec7fff700 1 -- 172.21.15.7:0/1968695784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ed0143670 msgr2=0x7f7ed015b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.643+0000 7f7ec7fff700 1 --2- 172.21.15.7:0/1968695784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ed0143670 0x7f7ed015b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.643+0000 7f7ec7fff700 1 -- 172.21.15.7:0/1968695784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec8009a20 con 0x7f7ed00aa8b0 2024-01-27T02:38:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.643+0000 7f7ed4bc1700 1 --2- 172.21.15.7:0/1968695784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ed0063c90 0x7f7ed00b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:38:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.643+0000 7f7ec7fff700 1 --2- 172.21.15.7:0/1968695784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ed00aa8b0 0x7f7ed015eee0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000c7e0 tx=0x7f7ec0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.643+0000 7f7ec4ff9700 1 -- 172.21.15.7:0/1968695784 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0005b30 con 0x7f7ed00aa8b0 2024-01-27T02:38:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.644+0000 7f7ec4ff9700 1 -- 172.21.15.7:0/1968695784 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec0005c90 con 0x7f7ed00aa8b0 2024-01-27T02:38:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.644+0000 7f7ed6e25700 1 -- 172.21.15.7:0/1968695784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed016ab50 con 0x7f7ed00aa8b0 2024-01-27T02:38:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.644+0000 7f7ec4ff9700 1 -- 172.21.15.7:0/1968695784 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0007300 con 0x7f7ed00aa8b0 2024-01-27T02:38:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.644+0000 7f7ed6e25700 1 -- 172.21.15.7:0/1968695784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed016b110 con 0x7f7ed00aa8b0 2024-01-27T02:38:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.645+0000 7f7ed6e25700 1 -- 172.21.15.7:0/1968695784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed0072060 con 0x7f7ed00aa8b0 2024-01-27T02:38:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.645+0000 7f7ec4ff9700 1 -- 172.21.15.7:0/1968695784 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7ec001b400 con 0x7f7ed00aa8b0 2024-01-27T02:38:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.646+0000 7f7ec4ff9700 1 --2- 172.21.15.7:0/1968695784 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ebc066f30 0x7f7ebc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.646+0000 7f7ec4ff9700 1 -- 172.21.15.7:0/1968695784 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7ec0083d80 con 0x7f7ed00aa8b0 2024-01-27T02:38:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.646+0000 7f7ed4bc1700 1 --2- 172.21.15.7:0/1968695784 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ebc066f30 0x7f7ebc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:17.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.647+0000 7f7ed4bc1700 1 --2- 172.21.15.7:0/1968695784 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ebc066f30 0x7f7ebc0693e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc006fd0 tx=0x7f7ecc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:17.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.650+0000 7f7ec4ff9700 1 -- 172.21.15.7:0/1968695784 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ec00512d0 con 0x7f7ed00aa8b0 2024-01-27T02:38:17.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.862+0000 7f7ed6e25700 1 -- 172.21.15.7:0/1968695784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7ed0143a40 con 0x7f7ed00aa8b0 2024-01-27T02:38:17.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.866+0000 7f7ec4ff9700 1 -- 172.21.15.7:0/1968695784 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7ec00548f0 con 0x7f7ed00aa8b0 2024-01-27T02:38:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.868+0000 7f7ed6e25700 1 -- 172.21.15.7:0/1968695784 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ebc066f30 msgr2=0x7f7ebc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.868+0000 7f7ed6e25700 1 --2- 172.21.15.7:0/1968695784 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ebc066f30 0x7f7ebc0693e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc006fd0 tx=0x7f7ecc008040 comp rx=0 tx=0).stop 2024-01-27T02:38:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.868+0000 7f7ed6e25700 1 -- 172.21.15.7:0/1968695784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ed00aa8b0 msgr2=0x7f7ed015eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.868+0000 7f7ed6e25700 1 --2- 172.21.15.7:0/1968695784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ed00aa8b0 0x7f7ed015eee0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000c7e0 tx=0x7f7ec0007df0 comp rx=0 tx=0).stop 2024-01-27T02:38:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.868+0000 7f7ed6e25700 1 -- 172.21.15.7:0/1968695784 shutdown_connections 2024-01-27T02:38:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.868+0000 7f7ed6e25700 1 --2- 172.21.15.7:0/1968695784 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ebc066f30 0x7f7ebc0693e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.868+0000 7f7ed6e25700 1 --2- 172.21.15.7:0/1968695784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ed0143670 0x7f7ed015b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.868+0000 7f7ed6e25700 1 --2- 172.21.15.7:0/1968695784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ed0063c90 0x7f7ed00b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:17.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.868+0000 7f7ed6e25700 1 --2- 172.21.15.7:0/1968695784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ed00aa8b0 0x7f7ed015eee0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:17.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.868+0000 7f7ed6e25700 1 -- 172.21.15.7:0/1968695784 >> 172.21.15.7:0/1968695784 conn(0x7f7ed00cdbf0 msgr2=0x7f7ed00c4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:17.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.868+0000 7f7ed6e25700 1 -- 172.21.15.7:0/1968695784 shutdown_connections 2024-01-27T02:38:17.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:17.868+0000 7f7ed6e25700 1 -- 172.21.15.7:0/1968695784 wait complete. 2024-01-27T02:38:17.882 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2024-01-27T02:38:18.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.014+0000 7f1176228700 1 Processor -- start 2024-01-27T02:38:18.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.014+0000 7f1176228700 1 -- start start 2024-01-27T02:38:18.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.015+0000 7f1176228700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f11700caf60 0x7f1170164fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.015+0000 7f1176228700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f117015c5b0 con 0x7f11700c97e0 2024-01-27T02:38:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.015+0000 7f1176228700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f117015b9a0 con 0x7f117015e920 2024-01-27T02:38:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.015+0000 7f1176228700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f117015bc80 con 0x7f11700caf60 2024-01-27T02:38:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.015+0000 7f116f7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f11700c97e0 0x7f117015afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42252/0 (socket says 172.21.15.7:42252) 2024-01-27T02:38:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.015+0000 7f116f7fe700 1 -- 172.21.15.7:0/2617478591 learned_addr learned my addr 172.21.15.7:0/2617478591 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.015+0000 7f116ffff700 1 --2- 172.21.15.7:0/2617478591 >> v2:172.21.15.188:3300/0 conn(0x7f11700caf60 0x7f1170164fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.015+0000 7f116e7fc700 1 -- 172.21.15.7:0/2617478591 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198245071 0 0) 0x7f117015c5b0 con 0x7f11700c97e0 2024-01-27T02:38:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.015+0000 7f116e7fc700 1 -- 172.21.15.7:0/2617478591 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f115c003570 con 0x7f11700c97e0 2024-01-27T02:38:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.016+0000 7f116ffff700 1 -- 172.21.15.7:0/2617478591 >> v1:172.21.15.188:6789/0 conn(0x7f117015e920 legacy=0x7f11700ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.016+0000 7f116ffff700 1 -- 172.21.15.7:0/2617478591 >> v1:172.21.15.7:6789/0 conn(0x7f11700c97e0 legacy=0x7f117015afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.016+0000 7f116e7fc700 1 -- 172.21.15.7:0/2617478591 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2917497247 0 0) 0x7f117015b9a0 con 0x7f117015e920 2024-01-27T02:38:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.016+0000 7f116ffff700 1 -- 172.21.15.7:0/2617478591 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117015bf60 con 0x7f11700caf60 2024-01-27T02:38:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.016+0000 7f116ffff700 1 --2- 172.21.15.7:0/2617478591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11700caf60 0x7f1170164fd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1168009e60 tx=0x7f1168009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e10a780d68820d7 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.016+0000 7f116e7fc700 1 -- 172.21.15.7:0/2617478591 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116800e070 con 0x7f11700caf60 2024-01-27T02:38:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.016+0000 7f116e7fc700 1 -- 172.21.15.7:0/2617478591 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116800b850 con 0x7f11700caf60 2024-01-27T02:38:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.016+0000 7f116e7fc700 1 -- 172.21.15.7:0/2617478591 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1160001470 con 0x7f11700caf60 2024-01-27T02:38:18.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.017+0000 7f1176228700 1 -- 172.21.15.7:0/2617478591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11700caf60 msgr2=0x7f1170164fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:18.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.017+0000 7f1176228700 1 --2- 172.21.15.7:0/2617478591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11700caf60 0x7f1170164fd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1168009e60 tx=0x7f1168009780 comp rx=0 tx=0).stop 2024-01-27T02:38:18.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.017+0000 7f1176228700 1 -- 172.21.15.7:0/2617478591 shutdown_connections 2024-01-27T02:38:18.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.017+0000 7f1176228700 1 --2- 172.21.15.7:0/2617478591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11700caf60 0x7f1170164fd0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:18.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.017+0000 7f1176228700 1 -- 172.21.15.7:0/2617478591 >> 172.21.15.7:0/2617478591 conn(0x7f1170063580 msgr2=0x7f1170063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:18.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.017+0000 7f1176228700 1 -- 172.21.15.7:0/2617478591 shutdown_connections 2024-01-27T02:38:18.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.017+0000 7f1176228700 1 -- 172.21.15.7:0/2617478591 wait complete. 2024-01-27T02:38:18.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.018+0000 7f1176228700 1 Processor -- start 2024-01-27T02:38:18.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.018+0000 7f1176228700 1 -- start start 2024-01-27T02:38:18.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.019+0000 7f1176228700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11700c97e0 0x7f11701556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:18.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.019+0000 7f1176228700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11700caf60 0x7f11700aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:18.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.019+0000 7f1176228700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f117015e920 0x7f11700aae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.019+0000 7f1176228700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11700c4550 con 0x7f117015e920 2024-01-27T02:38:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.019+0000 7f1176228700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117006f170 con 0x7f11700caf60 2024-01-27T02:38:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.019+0000 7f1176228700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117006e5f0 con 0x7f11700c97e0 2024-01-27T02:38:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.019+0000 7f116ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f117015e920 0x7f11700aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.019+0000 7f116ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f117015e920 0x7f11700aae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54076/0 (socket says 172.21.15.7:54076) 2024-01-27T02:38:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.019+0000 7f116ffff700 1 -- 172.21.15.7:0/2295091615 learned_addr learned my addr 172.21.15.7:0/2295091615 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.019+0000 7f116effd700 1 --2- 172.21.15.7:0/2295091615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11700caf60 0x7f11700aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.019+0000 7f116f7fe700 1 --2- 172.21.15.7:0/2295091615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11700c97e0 0x7f11701556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.020+0000 7f116ffff700 1 -- 172.21.15.7:0/2295091615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11700c97e0 msgr2=0x7f11701556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.020+0000 7f116ffff700 1 --2- 172.21.15.7:0/2295091615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11700c97e0 0x7f11701556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.020+0000 7f116ffff700 1 -- 172.21.15.7:0/2295091615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11700caf60 msgr2=0x7f11700aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.020+0000 7f116ffff700 1 --2- 172.21.15.7:0/2295091615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11700caf60 0x7f11700aa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.020+0000 7f116ffff700 1 -- 172.21.15.7:0/2295091615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1168009a20 con 0x7f117015e920 2024-01-27T02:38:18.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.020+0000 7f116ffff700 1 --2- 172.21.15.7:0/2295091615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f117015e920 0x7f11700aae00 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f11680097c0 tx=0x7f1168017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:18.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.020+0000 7f11677fe700 1 -- 172.21.15.7:0/2295091615 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168028070 con 0x7f117015e920 2024-01-27T02:38:18.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.020+0000 7f11677fe700 1 -- 172.21.15.7:0/2295091615 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116800be40 con 0x7f117015e920 2024-01-27T02:38:18.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.020+0000 7f1176228700 1 -- 172.21.15.7:0/2295091615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f117006e8d0 con 0x7f117015e920 2024-01-27T02:38:18.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.020+0000 7f11677fe700 1 -- 172.21.15.7:0/2295091615 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168012900 con 0x7f117015e920 2024-01-27T02:38:18.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.020+0000 7f1176228700 1 -- 172.21.15.7:0/2295091615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117006ee90 con 0x7f117015e920 2024-01-27T02:38:18.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.021+0000 7f1176228700 1 -- 172.21.15.7:0/2295091615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11700af940 con 0x7f117015e920 2024-01-27T02:38:18.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.021+0000 7f11677fe700 1 -- 172.21.15.7:0/2295091615 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1168012d30 con 0x7f117015e920 2024-01-27T02:38:18.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.022+0000 7f11677fe700 1 --2- 172.21.15.7:0/2295091615 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1150066f30 0x7f11500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:18.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.022+0000 7f11677fe700 1 -- 172.21.15.7:0/2295091615 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f116808de20 con 0x7f117015e920 2024-01-27T02:38:18.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.022+0000 7f116f7fe700 1 --2- 172.21.15.7:0/2295091615 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1150066f30 0x7f11500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:18.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.023+0000 7f116f7fe700 1 --2- 172.21.15.7:0/2295091615 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1150066f30 0x7f11500693e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f11600015a0 tx=0x7f1160000bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:18.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.026+0000 7f11677fe700 1 -- 172.21.15.7:0/2295091615 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f116805b440 con 0x7f117015e920 2024-01-27T02:38:18.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.314+0000 7f1176228700 1 -- 172.21.15.7:0/2295091615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) v1 -- 0x7f117004c710 con 0x7f117015e920 2024-01-27T02:38:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.481+0000 7f11677fe700 1 -- 172.21.15.7:0/2295091615 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v323) v1 ==== 128+0+0 (secure 0 0 0) 0x7f116805ea60 con 0x7f117015e920 2024-01-27T02:38:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.484+0000 7f1176228700 1 -- 172.21.15.7:0/2295091615 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1150066f30 msgr2=0x7f11500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.484+0000 7f1176228700 1 --2- 172.21.15.7:0/2295091615 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1150066f30 0x7f11500693e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f11600015a0 tx=0x7f1160000bc0 comp rx=0 tx=0).stop 2024-01-27T02:38:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.484+0000 7f1176228700 1 -- 172.21.15.7:0/2295091615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f117015e920 msgr2=0x7f11700aae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.484+0000 7f1176228700 1 --2- 172.21.15.7:0/2295091615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f117015e920 0x7f11700aae00 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f11680097c0 tx=0x7f1168017000 comp rx=0 tx=0).stop 2024-01-27T02:38:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.484+0000 7f1176228700 1 -- 172.21.15.7:0/2295091615 shutdown_connections 2024-01-27T02:38:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.484+0000 7f1176228700 1 --2- 172.21.15.7:0/2295091615 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1150066f30 0x7f11500693e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.484+0000 7f1176228700 1 --2- 172.21.15.7:0/2295091615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f117015e920 0x7f11700aae00 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.484+0000 7f1176228700 1 --2- 172.21.15.7:0/2295091615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11700c97e0 0x7f11701556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.484+0000 7f1176228700 1 --2- 172.21.15.7:0/2295091615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11700caf60 0x7f11700aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.484+0000 7f1176228700 1 -- 172.21.15.7:0/2295091615 >> 172.21.15.7:0/2295091615 conn(0x7f1170063580 msgr2=0x7f1170157560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.484+0000 7f1176228700 1 -- 172.21.15.7:0/2295091615 shutdown_connections 2024-01-27T02:38:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.484+0000 7f1176228700 1 -- 172.21.15.7:0/2295091615 wait complete. 2024-01-27T02:38:18.499 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2024-01-27T02:38:18.500 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) 2024-01-27T02:38:18.500 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2024-01-27T02:38:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.635+0000 7f7378e8a700 1 Processor -- start 2024-01-27T02:38:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.635+0000 7f7378e8a700 1 -- start start 2024-01-27T02:38:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.636+0000 7f7378e8a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f737406a9f0 0x7f737415c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.636+0000 7f7378e8a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f737406adc0 0x7f7374158520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.636+0000 7f7378e8a700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f737406c880 con 0x7f7374158a60 2024-01-27T02:38:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.636+0000 7f7378e8a700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7374158f00 con 0x7f737406a9f0 2024-01-27T02:38:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.636+0000 7f7378e8a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f737415c210 con 0x7f737406adc0 2024-01-27T02:38:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.637+0000 7f7371d9b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f737406adc0 0x7f7374158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.637+0000 7f737259c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f737406a9f0 0x7f737415c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.637+0000 7f737259c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f737406a9f0 0x7f737415c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:45504/0 (socket says 172.21.15.7:45504) 2024-01-27T02:38:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.637+0000 7f737259c700 1 -- 172.21.15.7:0/1916283198 learned_addr learned my addr 172.21.15.7:0/1916283198 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.637+0000 7f7372d9d700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f7374158a60 0x7f7374063580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:39392/0 (socket says 172.21.15.7:39392) 2024-01-27T02:38:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.637+0000 7f7371d9b700 1 -- 172.21.15.7:0/1916283198 >> v2:172.21.15.188:3300/0 conn(0x7f737406a9f0 msgr2=0x7f737415c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.637+0000 7f7371d9b700 1 --2- 172.21.15.7:0/1916283198 >> v2:172.21.15.188:3300/0 conn(0x7f737406a9f0 0x7f737415c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.637+0000 7f7371d9b700 1 -- 172.21.15.7:0/1916283198 >> v1:172.21.15.188:6789/0 conn(0x7f7374158a60 legacy=0x7f7374063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.637+0000 7f737159a700 1 -- 172.21.15.7:0/1916283198 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 223800741 0 0) 0x7f737406c880 con 0x7f7374158a60 2024-01-27T02:38:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.637+0000 7f7371d9b700 1 -- 172.21.15.7:0/1916283198 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73741593e0 con 0x7f737406adc0 2024-01-27T02:38:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.637+0000 7f737259c700 1 --2- 172.21.15.7:0/1916283198 >> v2:172.21.15.188:3300/0 conn(0x7f737406a9f0 0x7f737415c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:38:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.637+0000 7f7371d9b700 1 --2- 172.21.15.7:0/1916283198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f737406adc0 0x7f7374158520 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f73640098e0 tx=0x7f73640091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fed9ed5baf788825 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.638+0000 7f737159a700 1 -- 172.21.15.7:0/1916283198 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736400b640 con 0x7f737406adc0 2024-01-27T02:38:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.638+0000 7f737159a700 1 -- 172.21.15.7:0/1916283198 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736400b7a0 con 0x7f737406adc0 2024-01-27T02:38:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.638+0000 7f737159a700 1 -- 172.21.15.7:0/1916283198 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736400bb40 con 0x7f737406adc0 2024-01-27T02:38:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.638+0000 7f7378e8a700 1 -- 172.21.15.7:0/1916283198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f737406adc0 msgr2=0x7f7374158520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.638+0000 7f7378e8a700 1 --2- 172.21.15.7:0/1916283198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f737406adc0 0x7f7374158520 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f73640098e0 tx=0x7f73640091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.638+0000 7f7378e8a700 1 -- 172.21.15.7:0/1916283198 shutdown_connections 2024-01-27T02:38:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.638+0000 7f7378e8a700 1 --2- 172.21.15.7:0/1916283198 >> v2:172.21.15.188:3300/0 conn(0x7f737406a9f0 0x7f737415c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.638+0000 7f7378e8a700 1 --2- 172.21.15.7:0/1916283198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f737406adc0 0x7f7374158520 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.638+0000 7f7378e8a700 1 -- 172.21.15.7:0/1916283198 >> 172.21.15.7:0/1916283198 conn(0x7f73740cdc30 msgr2=0x7f73740ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.638+0000 7f7378e8a700 1 -- 172.21.15.7:0/1916283198 shutdown_connections 2024-01-27T02:38:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.638+0000 7f7378e8a700 1 -- 172.21.15.7:0/1916283198 wait complete. 2024-01-27T02:38:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.639+0000 7f7378e8a700 1 Processor -- start 2024-01-27T02:38:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.640+0000 7f7378e8a700 1 -- start start 2024-01-27T02:38:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.640+0000 7f7378e8a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f737406a9f0 0x7f73740c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.640+0000 7f7378e8a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f737406adc0 0x7f7374168800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.640+0000 7f7378e8a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7374158a60 0x7f7374168d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.640+0000 7f7378e8a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73740c78b0 con 0x7f7374158a60 2024-01-27T02:38:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.640+0000 7f7378e8a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73740c6ca0 con 0x7f737406a9f0 2024-01-27T02:38:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.640+0000 7f7378e8a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73740c6f80 con 0x7f737406adc0 2024-01-27T02:38:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.640+0000 7f7372d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7374158a60 0x7f7374168d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.641+0000 7f7372d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7374158a60 0x7f7374168d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54080/0 (socket says 172.21.15.7:54080) 2024-01-27T02:38:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.641+0000 7f7372d9d700 1 -- 172.21.15.7:0/4192921874 learned_addr learned my addr 172.21.15.7:0/4192921874 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.641+0000 7f737259c700 1 --2- 172.21.15.7:0/4192921874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f737406a9f0 0x7f73740c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.641+0000 7f7371d9b700 1 --2- 172.21.15.7:0/4192921874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f737406adc0 0x7f7374168800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.641+0000 7f7372d9d700 1 -- 172.21.15.7:0/4192921874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f737406adc0 msgr2=0x7f7374168800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.641+0000 7f7372d9d700 1 --2- 172.21.15.7:0/4192921874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f737406adc0 0x7f7374168800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.641+0000 7f7372d9d700 1 -- 172.21.15.7:0/4192921874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f737406a9f0 msgr2=0x7f73740c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.641+0000 7f7372d9d700 1 --2- 172.21.15.7:0/4192921874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f737406a9f0 0x7f73740c65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.641+0000 7f7372d9d700 1 -- 172.21.15.7:0/4192921874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735c0094f0 con 0x7f7374158a60 2024-01-27T02:38:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.641+0000 7f7372d9d700 1 --2- 172.21.15.7:0/4192921874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7374158a60 0x7f7374168d40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f736c00cd50 tx=0x7f736c001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.641+0000 7f736a7fc700 1 -- 172.21.15.7:0/4192921874 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c005df0 con 0x7f7374158a60 2024-01-27T02:38:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.641+0000 7f736a7fc700 1 -- 172.21.15.7:0/4192921874 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736c00a8b0 con 0x7f7374158a60 2024-01-27T02:38:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.641+0000 7f7378e8a700 1 -- 172.21.15.7:0/4192921874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7364009420 con 0x7f7374158a60 2024-01-27T02:38:18.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.641+0000 7f736a7fc700 1 -- 172.21.15.7:0/4192921874 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c00ac70 con 0x7f7374158a60 2024-01-27T02:38:18.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.641+0000 7f7378e8a700 1 -- 172.21.15.7:0/4192921874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73740c7540 con 0x7f7374158a60 2024-01-27T02:38:18.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.642+0000 7f7378e8a700 1 -- 172.21.15.7:0/4192921874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73740c8040 con 0x7f7374158a60 2024-01-27T02:38:18.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.643+0000 7f736a7fc700 1 -- 172.21.15.7:0/4192921874 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f736c007300 con 0x7f7374158a60 2024-01-27T02:38:18.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.643+0000 7f736a7fc700 1 --2- 172.21.15.7:0/4192921874 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7354066f30 0x7f73540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:18.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.643+0000 7f736a7fc700 1 -- 172.21.15.7:0/4192921874 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f736c084330 con 0x7f7374158a60 2024-01-27T02:38:18.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.643+0000 7f737259c700 1 --2- 172.21.15.7:0/4192921874 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7354066f30 0x7f73540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:18.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.644+0000 7f737259c700 1 --2- 172.21.15.7:0/4192921874 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7354066f30 0x7f73540693e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f735c006d20 tx=0x7f735c009250 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.648+0000 7f736a7fc700 1 -- 172.21.15.7:0/4192921874 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f736c051920 con 0x7f7374158a60 2024-01-27T02:38:18.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:18.936+0000 7f7378e8a700 1 -- 172.21.15.7:0/4192921874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) v1 -- 0x7f737404c710 con 0x7f7374158a60 2024-01-27T02:38:19.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.484+0000 7f736a7fc700 1 -- 172.21.15.7:0/4192921874 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v324) v1 ==== 143+0+0 (secure 0 0 0) 0x7f736c054f40 con 0x7f7374158a60 2024-01-27T02:38:19.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.487+0000 7f7378e8a700 1 -- 172.21.15.7:0/4192921874 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7354066f30 msgr2=0x7f73540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:19.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.487+0000 7f7378e8a700 1 --2- 172.21.15.7:0/4192921874 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7354066f30 0x7f73540693e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f735c006d20 tx=0x7f735c009250 comp rx=0 tx=0).stop 2024-01-27T02:38:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.487+0000 7f7378e8a700 1 -- 172.21.15.7:0/4192921874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7374158a60 msgr2=0x7f7374168d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.487+0000 7f7378e8a700 1 --2- 172.21.15.7:0/4192921874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7374158a60 0x7f7374168d40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f736c00cd50 tx=0x7f736c001b10 comp rx=0 tx=0).stop 2024-01-27T02:38:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.487+0000 7f7378e8a700 1 -- 172.21.15.7:0/4192921874 shutdown_connections 2024-01-27T02:38:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.487+0000 7f7378e8a700 1 --2- 172.21.15.7:0/4192921874 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7354066f30 0x7f73540693e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.487+0000 7f7378e8a700 1 --2- 172.21.15.7:0/4192921874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7374158a60 0x7f7374168d40 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.487+0000 7f7378e8a700 1 --2- 172.21.15.7:0/4192921874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f737406a9f0 0x7f73740c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.487+0000 7f7378e8a700 1 --2- 172.21.15.7:0/4192921874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f737406adc0 0x7f7374168800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.487+0000 7f7378e8a700 1 -- 172.21.15.7:0/4192921874 >> 172.21.15.7:0/4192921874 conn(0x7f73740cdc30 msgr2=0x7f73740cbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.487+0000 7f7378e8a700 1 -- 172.21.15.7:0/4192921874 shutdown_connections 2024-01-27T02:38:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.487+0000 7f7378e8a700 1 -- 172.21.15.7:0/4192921874 wait complete. 2024-01-27T02:38:19.503 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-27T02:38:19.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.642+0000 7f46fc860700 1 Processor -- start 2024-01-27T02:38:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.642+0000 7f46fc860700 1 -- start start 2024-01-27T02:38:19.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.643+0000 7f46fc860700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f46f80ce510 0x7f46f8158050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:19.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.643+0000 7f46fc860700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46f8063580 0x7f46f8063a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:19.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.643+0000 7f46fc860700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f80b3510 con 0x7f46f8157c80 2024-01-27T02:38:19.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.643+0000 7f46fc860700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f8063f40 con 0x7f46f80ce510 2024-01-27T02:38:19.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.643+0000 7f46fc860700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f806f8b0 con 0x7f46f8063580 2024-01-27T02:38:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.643+0000 7f46edd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f46f80ce510 0x7f46f8158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.643+0000 7f46edd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f46f80ce510 0x7f46f8158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54092/0 (socket says 172.21.15.7:54092) 2024-01-27T02:38:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.643+0000 7f46edd9b700 1 -- 172.21.15.7:0/3224027633 learned_addr learned my addr 172.21.15.7:0/3224027633 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.643+0000 7f46f6d9d700 1 --2- 172.21.15.7:0/3224027633 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46f8063580 0x7f46f8063a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.643+0000 7f46edd9b700 1 -- 172.21.15.7:0/3224027633 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46f8063580 msgr2=0x7f46f8063a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.643+0000 7f46edd9b700 1 --2- 172.21.15.7:0/3224027633 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46f8063580 0x7f46f8063a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.643+0000 7f46edd9b700 1 -- 172.21.15.7:0/3224027633 >> v1:172.21.15.188:6789/0 conn(0x7f46f8157c80 legacy=0x7f46f80b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.643+0000 7f46edd9b700 1 -- 172.21.15.7:0/3224027633 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f80b2900 con 0x7f46f80ce510 2024-01-27T02:38:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.644+0000 7f46edd9b700 1 --2- 172.21.15.7:0/3224027633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46f80ce510 0x7f46f8158050 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f46e00099b0 tx=0x7f46e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ab5ea532fa226d server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.644+0000 7f46f5d9b700 1 -- 172.21.15.7:0/3224027633 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e000b860 con 0x7f46f80ce510 2024-01-27T02:38:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.644+0000 7f46f5d9b700 1 -- 172.21.15.7:0/3224027633 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e000b9c0 con 0x7f46f80ce510 2024-01-27T02:38:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.644+0000 7f46f5d9b700 1 -- 172.21.15.7:0/3224027633 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e000bd60 con 0x7f46f80ce510 2024-01-27T02:38:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.644+0000 7f46fc860700 1 -- 172.21.15.7:0/3224027633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46f80ce510 msgr2=0x7f46f8158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.644+0000 7f46fc860700 1 --2- 172.21.15.7:0/3224027633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46f80ce510 0x7f46f8158050 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f46e00099b0 tx=0x7f46e0009250 comp rx=0 tx=0).stop 2024-01-27T02:38:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.644+0000 7f46fc860700 1 -- 172.21.15.7:0/3224027633 shutdown_connections 2024-01-27T02:38:19.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.644+0000 7f46fc860700 1 --2- 172.21.15.7:0/3224027633 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46f8063580 0x7f46f8063a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:19.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.644+0000 7f46fc860700 1 --2- 172.21.15.7:0/3224027633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46f80ce510 0x7f46f8158050 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:19.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.644+0000 7f46fc860700 1 -- 172.21.15.7:0/3224027633 >> 172.21.15.7:0/3224027633 conn(0x7f46f8068480 msgr2=0x7f46f80cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:19.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.644+0000 7f46fc860700 1 -- 172.21.15.7:0/3224027633 shutdown_connections 2024-01-27T02:38:19.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.645+0000 7f46fc860700 1 -- 172.21.15.7:0/3224027633 wait complete. 2024-01-27T02:38:19.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.646+0000 7f46fc860700 1 Processor -- start 2024-01-27T02:38:19.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.646+0000 7f46fc860700 1 -- start start 2024-01-27T02:38:19.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.647+0000 7f46fc860700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46f8063580 0x7f46f80aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:19.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.647+0000 7f46fc860700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46f80ce510 0x7f46f8169430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:19.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.647+0000 7f46fc860700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46f8157c80 0x7f46f81681f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:19.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.647+0000 7f46fc860700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f814bf50 con 0x7f46f80ce510 2024-01-27T02:38:19.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.647+0000 7f46fc860700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f80abf80 con 0x7f46f8063580 2024-01-27T02:38:19.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.647+0000 7f46fc860700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f8169970 con 0x7f46f8157c80 2024-01-27T02:38:19.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.647+0000 7f46edd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46f80ce510 0x7f46f8169430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:19.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.647+0000 7f46edd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46f80ce510 0x7f46f8169430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54098/0 (socket says 172.21.15.7:54098) 2024-01-27T02:38:19.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.647+0000 7f46edd9b700 1 -- 172.21.15.7:0/1665595331 learned_addr learned my addr 172.21.15.7:0/1665595331 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:19.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.647+0000 7f46f6d9d700 1 --2- 172.21.15.7:0/1665595331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46f8157c80 0x7f46f81681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:19.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.647+0000 7f46f659c700 1 --2- 172.21.15.7:0/1665595331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46f8063580 0x7f46f80aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:19.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.647+0000 7f46edd9b700 1 -- 172.21.15.7:0/1665595331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46f8157c80 msgr2=0x7f46f81681f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:19.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.647+0000 7f46edd9b700 1 --2- 172.21.15.7:0/1665595331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46f8157c80 0x7f46f81681f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:19.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.647+0000 7f46edd9b700 1 -- 172.21.15.7:0/1665595331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46f8063580 msgr2=0x7f46f80aacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:19.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.647+0000 7f46edd9b700 1 --2- 172.21.15.7:0/1665595331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46f8063580 0x7f46f80aacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:19.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.647+0000 7f46edd9b700 1 -- 172.21.15.7:0/1665595331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e00094f0 con 0x7f46f80ce510 2024-01-27T02:38:19.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.647+0000 7f46edd9b700 1 --2- 172.21.15.7:0/1665595331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46f80ce510 0x7f46f8169430 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f46f80700c0 tx=0x7f46e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:19.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.648+0000 7f46eeffd700 1 -- 172.21.15.7:0/1665595331 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e0011e40 con 0x7f46f80ce510 2024-01-27T02:38:19.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.648+0000 7f46eeffd700 1 -- 172.21.15.7:0/1665595331 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e0024440 con 0x7f46f80ce510 2024-01-27T02:38:19.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.648+0000 7f46fc860700 1 -- 172.21.15.7:0/1665595331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f80ab370 con 0x7f46f80ce510 2024-01-27T02:38:19.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.648+0000 7f46eeffd700 1 -- 172.21.15.7:0/1665595331 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e0024800 con 0x7f46f80ce510 2024-01-27T02:38:19.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.648+0000 7f46fc860700 1 -- 172.21.15.7:0/1665595331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f80ab930 con 0x7f46f80ce510 2024-01-27T02:38:19.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.649+0000 7f46fc860700 1 -- 172.21.15.7:0/1665595331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f80ac0f0 con 0x7f46f80ce510 2024-01-27T02:38:19.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.649+0000 7f46eeffd700 1 -- 172.21.15.7:0/1665595331 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f46e001a420 con 0x7f46f80ce510 2024-01-27T02:38:19.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.650+0000 7f46eeffd700 1 --2- 172.21.15.7:0/1665595331 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f46d8066f30 0x7f46d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:19.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.650+0000 7f46eeffd700 1 -- 172.21.15.7:0/1665595331 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f46e0086110 con 0x7f46f80ce510 2024-01-27T02:38:19.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.650+0000 7f46f659c700 1 --2- 172.21.15.7:0/1665595331 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f46d8066f30 0x7f46d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:19.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.650+0000 7f46f659c700 1 --2- 172.21.15.7:0/1665595331 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f46d8066f30 0x7f46d80693e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f46e80039e0 tx=0x7f46e8007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:19.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.654+0000 7f46eeffd700 1 -- 172.21.15.7:0/1665595331 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46e0053680 con 0x7f46f80ce510 2024-01-27T02:38:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.860+0000 7f46fc860700 1 -- 172.21.15.7:0/1665595331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f46f8072060 con 0x7f46f80ce510 2024-01-27T02:38:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.865+0000 7f46eeffd700 1 -- 172.21.15.7:0/1665595331 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f46e0056ca0 con 0x7f46f80ce510 2024-01-27T02:38:19.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.867+0000 7f46fc860700 1 -- 172.21.15.7:0/1665595331 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f46d8066f30 msgr2=0x7f46d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:19.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.867+0000 7f46fc860700 1 --2- 172.21.15.7:0/1665595331 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f46d8066f30 0x7f46d80693e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f46e80039e0 tx=0x7f46e8007ec0 comp rx=0 tx=0).stop 2024-01-27T02:38:19.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.867+0000 7f46fc860700 1 -- 172.21.15.7:0/1665595331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46f80ce510 msgr2=0x7f46f8169430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:19.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.867+0000 7f46fc860700 1 --2- 172.21.15.7:0/1665595331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46f80ce510 0x7f46f8169430 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f46f80700c0 tx=0x7f46e0017000 comp rx=0 tx=0).stop 2024-01-27T02:38:19.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.867+0000 7f46fc860700 1 -- 172.21.15.7:0/1665595331 shutdown_connections 2024-01-27T02:38:19.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.867+0000 7f46fc860700 1 --2- 172.21.15.7:0/1665595331 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f46d8066f30 0x7f46d80693e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:19.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.867+0000 7f46fc860700 1 --2- 172.21.15.7:0/1665595331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46f8157c80 0x7f46f81681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:19.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.867+0000 7f46fc860700 1 --2- 172.21.15.7:0/1665595331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46f8063580 0x7f46f80aacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.867+0000 7f46fc860700 1 --2- 172.21.15.7:0/1665595331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46f80ce510 0x7f46f8169430 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.867+0000 7f46fc860700 1 -- 172.21.15.7:0/1665595331 >> 172.21.15.7:0/1665595331 conn(0x7f46f8068480 msgr2=0x7f46f806ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.867+0000 7f46fc860700 1 -- 172.21.15.7:0/1665595331 shutdown_connections 2024-01-27T02:38:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:19.867+0000 7f46fc860700 1 -- 172.21.15.7:0/1665595331 wait complete. 2024-01-27T02:38:19.882 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-27T02:38:19.882 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:38:19.928 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:38:19.980 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-27T02:38:19.980 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-27T02:38:19.980 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:38:20.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.117+0000 7f0e48293700 1 Processor -- start 2024-01-27T02:38:20.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.117+0000 7f0e48293700 1 -- start start 2024-01-27T02:38:20.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.118+0000 7f0e48293700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0e400c8ab0 0x7f0e4015dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:20.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.118+0000 7f0e48293700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e4015a9b0 con 0x7f0e400c8e80 2024-01-27T02:38:20.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.118+0000 7f0e48293700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e4015ecb0 con 0x7f0e400cbbc0 2024-01-27T02:38:20.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.118+0000 7f0e48293700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e4015e410 con 0x7f0e400c8ab0 2024-01-27T02:38:20.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.118+0000 7f0e4582e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0e400c8e80 0x7f0e400cd4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42268/0 (socket says 172.21.15.7:42268) 2024-01-27T02:38:20.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.118+0000 7f0e4582e700 1 -- 172.21.15.7:0/1678488741 learned_addr learned my addr 172.21.15.7:0/1678488741 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:20.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.118+0000 7f0e4602f700 1 --2- 172.21.15.7:0/1678488741 >> v2:172.21.15.188:3300/0 conn(0x7f0e400c8ab0 0x7f0e4015dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:20.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.118+0000 7f0e4502d700 1 -- 172.21.15.7:0/1678488741 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3966210588 0 0) 0x7f0e4015a9b0 con 0x7f0e400c8e80 2024-01-27T02:38:20.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.118+0000 7f0e4502d700 1 -- 172.21.15.7:0/1678488741 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e2c003570 con 0x7f0e400c8e80 2024-01-27T02:38:20.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.119+0000 7f0e4502d700 1 -- 172.21.15.7:0/1678488741 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1154430426 0 0) 0x7f0e4015ecb0 con 0x7f0e400cbbc0 2024-01-27T02:38:20.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.119+0000 7f0e4502d700 1 -- 172.21.15.7:0/1678488741 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e4015a9b0 con 0x7f0e400cbbc0 2024-01-27T02:38:20.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.119+0000 7f0e4502d700 1 -- 172.21.15.7:0/1678488741 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3080406421 0 0) 0x7f0e2c003570 con 0x7f0e400c8e80 2024-01-27T02:38:20.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.119+0000 7f0e4602f700 1 -- 172.21.15.7:0/1678488741 >> v1:172.21.15.188:6789/0 conn(0x7f0e400cbbc0 legacy=0x7f0e4015ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:20.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.119+0000 7f0e4602f700 1 -- 172.21.15.7:0/1678488741 >> v1:172.21.15.7:6789/0 conn(0x7f0e400c8e80 legacy=0x7f0e400cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:20.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.119+0000 7f0e4602f700 1 -- 172.21.15.7:0/1678488741 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4015e6f0 con 0x7f0e400c8ab0 2024-01-27T02:38:20.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.119+0000 7f0e4602f700 1 --2- 172.21.15.7:0/1678488741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e400c8ab0 0x7f0e4015dd80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0098e0 tx=0x7f0e3c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b138a69d37cab41 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:20.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.119+0000 7f0e4502d700 1 -- 172.21.15.7:0/1678488741 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e30001470 con 0x7f0e400c8ab0 2024-01-27T02:38:20.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.119+0000 7f0e4502d700 1 -- 172.21.15.7:0/1678488741 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3c011420 con 0x7f0e400c8ab0 2024-01-27T02:38:20.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.119+0000 7f0e4502d700 1 -- 172.21.15.7:0/1678488741 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c0117c0 con 0x7f0e400c8ab0 2024-01-27T02:38:20.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.120+0000 7f0e48293700 1 -- 172.21.15.7:0/1678488741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e400c8ab0 msgr2=0x7f0e4015dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:20.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.120+0000 7f0e48293700 1 --2- 172.21.15.7:0/1678488741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e400c8ab0 0x7f0e4015dd80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0098e0 tx=0x7f0e3c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:20.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.120+0000 7f0e48293700 1 -- 172.21.15.7:0/1678488741 shutdown_connections 2024-01-27T02:38:20.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.120+0000 7f0e48293700 1 --2- 172.21.15.7:0/1678488741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e400c8ab0 0x7f0e4015dd80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:20.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.120+0000 7f0e48293700 1 -- 172.21.15.7:0/1678488741 >> 172.21.15.7:0/1678488741 conn(0x7f0e400cdc30 msgr2=0x7f0e400ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:20.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.120+0000 7f0e48293700 1 -- 172.21.15.7:0/1678488741 shutdown_connections 2024-01-27T02:38:20.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.120+0000 7f0e48293700 1 -- 172.21.15.7:0/1678488741 wait complete. 2024-01-27T02:38:20.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.121+0000 7f0e48293700 1 Processor -- start 2024-01-27T02:38:20.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.121+0000 7f0e48293700 1 -- start start 2024-01-27T02:38:20.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.122+0000 7f0e48293700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e400c8ab0 0x7f0e401575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:20.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.122+0000 7f0e48293700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e400c8e80 0x7f0e4015fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:20.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.122+0000 7f0e48293700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e400cbbc0 0x7f0e401603b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:20.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.122+0000 7f0e48293700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4016ad50 con 0x7f0e400c8e80 2024-01-27T02:38:20.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.122+0000 7f0e48293700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e400c7950 con 0x7f0e400c8ab0 2024-01-27T02:38:20.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.122+0000 7f0e48293700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4016dba0 con 0x7f0e400cbbc0 2024-01-27T02:38:20.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.122+0000 7f0e4582e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e400c8e80 0x7f0e4015fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:20.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.122+0000 7f0e4582e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e400c8e80 0x7f0e4015fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54102/0 (socket says 172.21.15.7:54102) 2024-01-27T02:38:20.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.122+0000 7f0e4582e700 1 -- 172.21.15.7:0/2932531016 learned_addr learned my addr 172.21.15.7:0/2932531016 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:20.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.122+0000 7f0e4602f700 1 --2- 172.21.15.7:0/2932531016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e400c8ab0 0x7f0e401575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:20.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.122+0000 7f0e46830700 1 --2- 172.21.15.7:0/2932531016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e400cbbc0 0x7f0e401603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:20.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.122+0000 7f0e4582e700 1 -- 172.21.15.7:0/2932531016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e400cbbc0 msgr2=0x7f0e401603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:20.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.122+0000 7f0e4582e700 1 --2- 172.21.15.7:0/2932531016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e400cbbc0 0x7f0e401603b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:20.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.122+0000 7f0e4582e700 1 -- 172.21.15.7:0/2932531016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e400c8ab0 msgr2=0x7f0e401575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:20.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.122+0000 7f0e4582e700 1 --2- 172.21.15.7:0/2932531016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e400c8ab0 0x7f0e401575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:20.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.122+0000 7f0e4582e700 1 -- 172.21.15.7:0/2932531016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3c009420 con 0x7f0e400c8e80 2024-01-27T02:38:20.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.122+0000 7f0e46830700 1 --2- 172.21.15.7:0/2932531016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e400cbbc0 0x7f0e401603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:38:20.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.122+0000 7f0e4602f700 1 --2- 172.21.15.7:0/2932531016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e400c8ab0 0x7f0e401575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:38:20.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.123+0000 7f0e4582e700 1 --2- 172.21.15.7:0/2932531016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e400c8e80 0x7f0e4015fe70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000cbc0 tx=0x7f0e30001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:20.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.123+0000 7f0e367fc700 1 -- 172.21.15.7:0/2932531016 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e30007360 con 0x7f0e400c8e80 2024-01-27T02:38:20.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.123+0000 7f0e367fc700 1 -- 172.21.15.7:0/2932531016 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e300074e0 con 0x7f0e400c8e80 2024-01-27T02:38:20.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.123+0000 7f0e48293700 1 -- 172.21.15.7:0/2932531016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4016aa70 con 0x7f0e400c8e80 2024-01-27T02:38:20.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.123+0000 7f0e367fc700 1 -- 172.21.15.7:0/2932531016 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e30004830 con 0x7f0e400c8e80 2024-01-27T02:38:20.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.123+0000 7f0e48293700 1 -- 172.21.15.7:0/2932531016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4016d830 con 0x7f0e400c8e80 2024-01-27T02:38:20.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.124+0000 7f0e48293700 1 -- 172.21.15.7:0/2932531016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e4016d3b0 con 0x7f0e400c8e80 2024-01-27T02:38:20.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.124+0000 7f0e367fc700 1 -- 172.21.15.7:0/2932531016 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0e30004bc0 con 0x7f0e400c8e80 2024-01-27T02:38:20.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.125+0000 7f0e367fc700 1 --2- 172.21.15.7:0/2932531016 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0e24066f30 0x7f0e240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:20.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.125+0000 7f0e367fc700 1 -- 172.21.15.7:0/2932531016 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0e300839f0 con 0x7f0e400c8e80 2024-01-27T02:38:20.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.125+0000 7f0e4602f700 1 --2- 172.21.15.7:0/2932531016 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0e24066f30 0x7f0e240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:20.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.126+0000 7f0e4602f700 1 --2- 172.21.15.7:0/2932531016 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0e24066f30 0x7f0e240693e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c003080 tx=0x7f0e3c016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:20.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.129+0000 7f0e367fc700 1 -- 172.21.15.7:0/2932531016 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e4016d3b0 con 0x7f0e400c8e80 2024-01-27T02:38:20.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.381+0000 7f0e48293700 1 -- 172.21.15.7:0/2932531016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e4016d3b0 con 0x7f0e400c8e80 2024-01-27T02:38:20.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.382+0000 7f0e367fc700 1 -- 172.21.15.7:0/2932531016 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f0e4016d3b0 con 0x7f0e400c8e80 2024-01-27T02:38:20.383 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:20.383 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":60,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2767759975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2767759975},{"type":"v1","addr":"172.21.15.188:6836","nonce":2767759975}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":5768,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/245473575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":245473575},{"type":"v1","addr":"172.21.15.7:6834","nonce":245473575}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":5966,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3408337775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3408337775},{"type":"v1","addr":"172.21.15.7:6835","nonce":3408337775}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1334164270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1334164270},{"type":"v1","addr":"172.21.15.188:6837","nonce":1334164270}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[]} 2024-01-27T02:38:20.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.384+0000 7f0e48293700 1 -- 172.21.15.7:0/2932531016 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0e24066f30 msgr2=0x7f0e240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:20.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.384+0000 7f0e48293700 1 --2- 172.21.15.7:0/2932531016 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0e24066f30 0x7f0e240693e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c003080 tx=0x7f0e3c016000 comp rx=0 tx=0).stop 2024-01-27T02:38:20.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.384+0000 7f0e48293700 1 -- 172.21.15.7:0/2932531016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e400c8e80 msgr2=0x7f0e4015fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:20.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.384+0000 7f0e48293700 1 --2- 172.21.15.7:0/2932531016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e400c8e80 0x7f0e4015fe70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000cbc0 tx=0x7f0e30001b50 comp rx=0 tx=0).stop 2024-01-27T02:38:20.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.384+0000 7f0e48293700 1 -- 172.21.15.7:0/2932531016 shutdown_connections 2024-01-27T02:38:20.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.384+0000 7f0e48293700 1 --2- 172.21.15.7:0/2932531016 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0e24066f30 0x7f0e240693e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:20.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.384+0000 7f0e48293700 1 --2- 172.21.15.7:0/2932531016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e400cbbc0 0x7f0e401603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:20.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.384+0000 7f0e48293700 1 --2- 172.21.15.7:0/2932531016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e400c8ab0 0x7f0e401575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:20.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.384+0000 7f0e48293700 1 --2- 172.21.15.7:0/2932531016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e400c8e80 0x7f0e4015fe70 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:20.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.384+0000 7f0e48293700 1 -- 172.21.15.7:0/2932531016 >> 172.21.15.7:0/2932531016 conn(0x7f0e400cdc30 msgr2=0x7f0e400ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:20.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.384+0000 7f0e48293700 1 -- 172.21.15.7:0/2932531016 shutdown_connections 2024-01-27T02:38:20.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:20.384+0000 7f0e48293700 1 -- 172.21.15.7:0/2932531016 wait complete. 2024-01-27T02:38:20.386 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 60 2024-01-27T02:38:20.398 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-27T02:38:20.398 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-27T02:38:20.398 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-27T02:38:20.399 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:38:20.399 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-27T02:38:20.399 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-27T02:38:20.399 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-27T02:38:20.399 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:38:20.399 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-27T02:38:20.399 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-27T02:38:20.400 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-27T02:38:20.400 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:38:20.400 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-27T02:38:20.400 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-27T02:38:20.400 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-27T02:38:20.400 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:38:20.411 INFO:tasks.ceph.mds.b:Stopped 2024-01-27T02:38:20.411 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-27T02:38:20.412 INFO:tasks.ceph.mds.d:Stopped 2024-01-27T02:38:20.412 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-27T02:38:20.413 INFO:tasks.ceph.mds.c:Stopped 2024-01-27T02:38:20.413 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-27T02:38:20.457 INFO:tasks.ceph.mds.b:Started 2024-01-27T02:38:20.457 DEBUG:teuthology.parallel:result is None 2024-01-27T02:38:20.458 INFO:tasks.ceph.mds.c:Started 2024-01-27T02:38:20.458 INFO:tasks.ceph.mds.d:Started 2024-01-27T02:38:20.458 DEBUG:teuthology.parallel:result is None 2024-01-27T02:38:20.458 DEBUG:teuthology.parallel:result is None 2024-01-27T02:38:20.543 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-27T02:38:20.544 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-01-27T02:38:20.544 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-01-27T02:38:26.413 INFO:tasks.ceph.mds.a:Stopped 2024-01-27T02:38:26.413 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-27T02:38:26.416 INFO:tasks.ceph.mds.a:Started 2024-01-27T02:38:26.417 DEBUG:teuthology.parallel:result is None 2024-01-27T02:38:26.417 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-27T02:38:26.508 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-27T02:38:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.559+0000 7fe8b3265700 1 Processor -- start 2024-01-27T02:38:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.559+0000 7fe8b3265700 1 -- start start 2024-01-27T02:38:26.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.560+0000 7fe8b3265700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe8ac0c4cc0 0x7fe8ac0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:26.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.560+0000 7fe8b3265700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe8ac0ca540 0x7fe8ac163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.560+0000 7fe8b3265700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ac0ce5e0 con 0x7fe8ac15a100 2024-01-27T02:38:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.560+0000 7fe8b3265700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ac0cd9d0 con 0x7fe8ac0ca540 2024-01-27T02:38:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.560+0000 7fe8b3265700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ac0cdcb0 con 0x7fe8ac0c4cc0 2024-01-27T02:38:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.560+0000 7fe8b1001700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe8ac15a100 0x7fe8ac0c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49282/0 (socket says 172.21.15.7:49282) 2024-01-27T02:38:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.560+0000 7fe8b1001700 1 -- 172.21.15.7:0/1580231073 learned_addr learned my addr 172.21.15.7:0/1580231073 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.560+0000 7fe8b1802700 1 --2- 172.21.15.7:0/1580231073 >> v2:172.21.15.7:3300/0 conn(0x7fe8ac0ca540 0x7fe8ac163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.560+0000 7fe8b0800700 1 --2- 172.21.15.7:0/1580231073 >> v2:172.21.15.188:3300/0 conn(0x7fe8ac0c4cc0 0x7fe8ac0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.561+0000 7fe8abfff700 1 -- 172.21.15.7:0/1580231073 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3006238179 0 0) 0x7fe8ac0ce5e0 con 0x7fe8ac15a100 2024-01-27T02:38:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.561+0000 7fe8abfff700 1 -- 172.21.15.7:0/1580231073 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe894003570 con 0x7fe8ac15a100 2024-01-27T02:38:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.561+0000 7fe8b1802700 1 -- 172.21.15.7:0/1580231073 >> v2:172.21.15.188:3300/0 conn(0x7fe8ac0c4cc0 msgr2=0x7fe8ac0cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.561+0000 7fe8b1802700 1 --2- 172.21.15.7:0/1580231073 >> v2:172.21.15.188:3300/0 conn(0x7fe8ac0c4cc0 0x7fe8ac0cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.561+0000 7fe8b1802700 1 -- 172.21.15.7:0/1580231073 >> v1:172.21.15.7:6789/0 conn(0x7fe8ac15a100 legacy=0x7fe8ac0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.561+0000 7fe8b1802700 1 -- 172.21.15.7:0/1580231073 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac0cdf90 con 0x7fe8ac0ca540 2024-01-27T02:38:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.561+0000 7fe8b0800700 1 --2- 172.21.15.7:0/1580231073 >> v2:172.21.15.188:3300/0 conn(0x7fe8ac0c4cc0 0x7fe8ac0cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:38:26.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.561+0000 7fe8b1802700 1 --2- 172.21.15.7:0/1580231073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8ac0ca540 0x7fe8ac163d90 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac06e170 tx=0x7fe8a4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924f6db3f01b33a7 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:26.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.561+0000 7fe8abfff700 1 -- 172.21.15.7:0/1580231073 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a400e070 con 0x7fe8ac0ca540 2024-01-27T02:38:26.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.561+0000 7fe8abfff700 1 -- 172.21.15.7:0/1580231073 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a400bab0 con 0x7fe8ac0ca540 2024-01-27T02:38:26.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.561+0000 7fe8abfff700 1 -- 172.21.15.7:0/1580231073 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a400be70 con 0x7fe8ac0ca540 2024-01-27T02:38:26.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.562+0000 7fe8b3265700 1 -- 172.21.15.7:0/1580231073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8ac0ca540 msgr2=0x7fe8ac163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:26.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.562+0000 7fe8b3265700 1 --2- 172.21.15.7:0/1580231073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8ac0ca540 0x7fe8ac163d90 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac06e170 tx=0x7fe8a4009780 comp rx=0 tx=0).stop 2024-01-27T02:38:26.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.562+0000 7fe8b3265700 1 -- 172.21.15.7:0/1580231073 shutdown_connections 2024-01-27T02:38:26.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.562+0000 7fe8b3265700 1 --2- 172.21.15.7:0/1580231073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8ac0ca540 0x7fe8ac163d90 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:26.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.562+0000 7fe8b3265700 1 --2- 172.21.15.7:0/1580231073 >> v2:172.21.15.188:3300/0 conn(0x7fe8ac0c4cc0 0x7fe8ac0cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:26.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.562+0000 7fe8b3265700 1 -- 172.21.15.7:0/1580231073 >> 172.21.15.7:0/1580231073 conn(0x7fe8ac098020 msgr2=0x7fe8ac0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:26.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.562+0000 7fe8b3265700 1 -- 172.21.15.7:0/1580231073 shutdown_connections 2024-01-27T02:38:26.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.562+0000 7fe8b3265700 1 -- 172.21.15.7:0/1580231073 wait complete. 2024-01-27T02:38:26.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.563+0000 7fe8b3265700 1 Processor -- start 2024-01-27T02:38:26.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.563+0000 7fe8b3265700 1 -- start start 2024-01-27T02:38:26.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.564+0000 7fe8b3265700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8ac0c4cc0 0x7fe8ac15c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:26.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.564+0000 7fe8b3265700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8ac0ca540 0x7fe8ac0b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:26.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.564+0000 7fe8b3265700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8ac15a100 0x7fe8ac14abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:26.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.564+0000 7fe8b3265700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0b2a30 con 0x7fe8ac0c4cc0 2024-01-27T02:38:26.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.564+0000 7fe8b3265700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0b3870 con 0x7fe8ac0ca540 2024-01-27T02:38:26.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.564+0000 7fe8b3265700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0b2c60 con 0x7fe8ac15a100 2024-01-27T02:38:26.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.564+0000 7fe8b1001700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8ac0c4cc0 0x7fe8ac15c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:26.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.564+0000 7fe8b1001700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8ac0c4cc0 0x7fe8ac15c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52680/0 (socket says 172.21.15.7:52680) 2024-01-27T02:38:26.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.564+0000 7fe8b1001700 1 -- 172.21.15.7:0/1171575665 learned_addr learned my addr 172.21.15.7:0/1171575665 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:26.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.565+0000 7fe8b0800700 1 --2- 172.21.15.7:0/1171575665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8ac0ca540 0x7fe8ac0b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:26.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.565+0000 7fe8b1802700 1 --2- 172.21.15.7:0/1171575665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8ac15a100 0x7fe8ac14abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:26.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.565+0000 7fe8b1001700 1 -- 172.21.15.7:0/1171575665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8ac15a100 msgr2=0x7fe8ac14abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:26.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.565+0000 7fe8b1001700 1 --2- 172.21.15.7:0/1171575665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8ac15a100 0x7fe8ac14abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:26.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.565+0000 7fe8b1001700 1 -- 172.21.15.7:0/1171575665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8ac0ca540 msgr2=0x7fe8ac0b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:26.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.565+0000 7fe8b1001700 1 --2- 172.21.15.7:0/1171575665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8ac0ca540 0x7fe8ac0b24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:26.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.565+0000 7fe8b1001700 1 -- 172.21.15.7:0/1171575665 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a0009420 con 0x7fe8ac0c4cc0 2024-01-27T02:38:26.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.565+0000 7fe8b1802700 1 --2- 172.21.15.7:0/1171575665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8ac15a100 0x7fe8ac14abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:38:26.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.565+0000 7fe8b1001700 1 --2- 172.21.15.7:0/1171575665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8ac0c4cc0 0x7fe8ac15c800 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00c590 tx=0x7fe89c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:26.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.565+0000 7fe8a97fa700 1 -- 172.21.15.7:0/1171575665 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c005b30 con 0x7fe8ac0c4cc0 2024-01-27T02:38:26.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.565+0000 7fe8a97fa700 1 -- 172.21.15.7:0/1171575665 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89c005cb0 con 0x7fe8ac0c4cc0 2024-01-27T02:38:26.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.565+0000 7fe8b3265700 1 -- 172.21.15.7:0/1171575665 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a4009a20 con 0x7fe8ac0c4cc0 2024-01-27T02:38:26.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.565+0000 7fe8a97fa700 1 -- 172.21.15.7:0/1171575665 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c007320 con 0x7fe8ac0c4cc0 2024-01-27T02:38:26.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.565+0000 7fe8b3265700 1 -- 172.21.15.7:0/1171575665 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ac0b3220 con 0x7fe8ac0c4cc0 2024-01-27T02:38:26.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.566+0000 7fe8b3265700 1 -- 172.21.15.7:0/1171575665 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ac06ef90 con 0x7fe8ac0c4cc0 2024-01-27T02:38:26.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.567+0000 7fe8a97fa700 1 -- 172.21.15.7:0/1171575665 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe89c01b3b0 con 0x7fe8ac0c4cc0 2024-01-27T02:38:26.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.567+0000 7fe8a97fa700 1 --2- 172.21.15.7:0/1171575665 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe898066f30 0x7fe8980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:26.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.567+0000 7fe8a97fa700 1 -- 172.21.15.7:0/1171575665 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fe89c0838f0 con 0x7fe8ac0c4cc0 2024-01-27T02:38:26.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.568+0000 7fe8b0800700 1 --2- 172.21.15.7:0/1171575665 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe898066f30 0x7fe8980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:26.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.568+0000 7fe8b0800700 1 --2- 172.21.15.7:0/1171575665 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe898066f30 0x7fe8980693e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe8a00093f0 tx=0x7fe8a0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:26.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.572+0000 7fe8a97fa700 1 -- 172.21.15.7:0/1171575665 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe89c050dc0 con 0x7fe8ac0c4cc0 2024-01-27T02:38:26.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:26.790+0000 7fe8b3265700 1 -- 172.21.15.7:0/1171575665 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe8ac04c710 con 0x7fe8ac0c4cc0 2024-01-27T02:38:27.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:27.871+0000 7fe8a97fa700 1 -- 172.21.15.7:0/1171575665 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v128) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe89c017020 con 0x7fe8ac0c4cc0 2024-01-27T02:38:27.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:27.874+0000 7fe8b3265700 1 -- 172.21.15.7:0/1171575665 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe898066f30 msgr2=0x7fe8980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:27.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:27.874+0000 7fe8b3265700 1 --2- 172.21.15.7:0/1171575665 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe898066f30 0x7fe8980693e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe8a00093f0 tx=0x7fe8a0008040 comp rx=0 tx=0).stop 2024-01-27T02:38:27.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:27.874+0000 7fe8b3265700 1 -- 172.21.15.7:0/1171575665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8ac0c4cc0 msgr2=0x7fe8ac15c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:27.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:27.874+0000 7fe8b3265700 1 --2- 172.21.15.7:0/1171575665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8ac0c4cc0 0x7fe8ac15c800 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00c590 tx=0x7fe89c007e00 comp rx=0 tx=0).stop 2024-01-27T02:38:27.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:27.874+0000 7fe8b3265700 1 -- 172.21.15.7:0/1171575665 shutdown_connections 2024-01-27T02:38:27.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:27.874+0000 7fe8b3265700 1 --2- 172.21.15.7:0/1171575665 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe898066f30 0x7fe8980693e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:27.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:27.874+0000 7fe8b3265700 1 --2- 172.21.15.7:0/1171575665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8ac15a100 0x7fe8ac14abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:27.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:27.874+0000 7fe8b3265700 1 --2- 172.21.15.7:0/1171575665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8ac0c4cc0 0x7fe8ac15c800 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:27.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:27.874+0000 7fe8b3265700 1 --2- 172.21.15.7:0/1171575665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8ac0ca540 0x7fe8ac0b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:27.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:27.874+0000 7fe8b3265700 1 -- 172.21.15.7:0/1171575665 >> 172.21.15.7:0/1171575665 conn(0x7fe8ac098020 msgr2=0x7fe8ac154f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:27.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:27.874+0000 7fe8b3265700 1 -- 172.21.15.7:0/1171575665 shutdown_connections 2024-01-27T02:38:27.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:27.874+0000 7fe8b3265700 1 -- 172.21.15.7:0/1171575665 wait complete. 2024-01-27T02:38:27.876 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-01-27T02:38:27.891 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-27T02:38:28.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.026+0000 7f4831d5f700 1 Processor -- start 2024-01-27T02:38:28.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.026+0000 7f4831d5f700 1 -- start start 2024-01-27T02:38:28.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.027+0000 7f4831d5f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f482c159f40 0x7f482c0c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:28.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.027+0000 7f4831d5f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f482c0c3a90 0x7f482c0aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:28.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.027+0000 7f4831d5f700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f482c0c30c0 con 0x7f482c15a310 2024-01-27T02:38:28.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.027+0000 7f4831d5f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f482c0c4550 con 0x7f482c159f40 2024-01-27T02:38:28.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.027+0000 7f4831d5f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482c15e960 con 0x7f482c0c3a90 2024-01-27T02:38:28.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.027+0000 7f482affd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f482c0c3a90 0x7f482c0aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:28.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.027+0000 7f482b7fe700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f482c159f40 0x7f482c0c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:28.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.027+0000 7f482bfff700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f482c15a310 0x7f482c063580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:38824/0 (socket says 172.21.15.7:38824) 2024-01-27T02:38:28.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.027+0000 7f482bfff700 1 -- 172.21.15.7:0/222156510 learned_addr learned my addr 172.21.15.7:0/222156510 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:28.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.028+0000 7f482affd700 1 -- 172.21.15.7:0/222156510 >> v2:172.21.15.188:3300/0 conn(0x7f482c159f40 msgr2=0x7f482c0c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:28.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.028+0000 7f482affd700 1 --2- 172.21.15.7:0/222156510 >> v2:172.21.15.188:3300/0 conn(0x7f482c159f40 0x7f482c0c3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.028+0000 7f482affd700 1 -- 172.21.15.7:0/222156510 >> v1:172.21.15.188:6789/0 conn(0x7f482c15a310 legacy=0x7f482c063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:28.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.028+0000 7f482affd700 1 -- 172.21.15.7:0/222156510 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c15ec40 con 0x7f482c0c3a90 2024-01-27T02:38:28.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.028+0000 7f482a7fc700 1 -- 172.21.15.7:0/222156510 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 89952502 0 0) 0x7f482c0c30c0 con 0x7f482c15a310 2024-01-27T02:38:28.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.028+0000 7f482b7fe700 1 --2- 172.21.15.7:0/222156510 >> v2:172.21.15.188:3300/0 conn(0x7f482c159f40 0x7f482c0c3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:38:28.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.028+0000 7f482affd700 1 --2- 172.21.15.7:0/222156510 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f482c0c3a90 0x7f482c0aa8b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f48140098e0 tx=0x7f48140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=800a4390c38cc826 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:28.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.028+0000 7f482a7fc700 1 -- 172.21.15.7:0/222156510 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481400b640 con 0x7f482c0c3a90 2024-01-27T02:38:28.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.028+0000 7f482a7fc700 1 -- 172.21.15.7:0/222156510 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481400b7a0 con 0x7f482c0c3a90 2024-01-27T02:38:28.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.028+0000 7f482a7fc700 1 -- 172.21.15.7:0/222156510 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481400bb40 con 0x7f482c0c3a90 2024-01-27T02:38:28.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.029+0000 7f4831d5f700 1 -- 172.21.15.7:0/222156510 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f482c0c3a90 msgr2=0x7f482c0aa8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:28.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.029+0000 7f4831d5f700 1 --2- 172.21.15.7:0/222156510 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f482c0c3a90 0x7f482c0aa8b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f48140098e0 tx=0x7f48140091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.029+0000 7f4831d5f700 1 -- 172.21.15.7:0/222156510 shutdown_connections 2024-01-27T02:38:28.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.029+0000 7f4831d5f700 1 --2- 172.21.15.7:0/222156510 >> v2:172.21.15.188:3300/0 conn(0x7f482c159f40 0x7f482c0c3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.029+0000 7f4831d5f700 1 --2- 172.21.15.7:0/222156510 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f482c0c3a90 0x7f482c0aa8b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.029+0000 7f4831d5f700 1 -- 172.21.15.7:0/222156510 >> 172.21.15.7:0/222156510 conn(0x7f482c0cdc30 msgr2=0x7f482c0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:28.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.029+0000 7f4831d5f700 1 -- 172.21.15.7:0/222156510 shutdown_connections 2024-01-27T02:38:28.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.029+0000 7f4831d5f700 1 -- 172.21.15.7:0/222156510 wait complete. 2024-01-27T02:38:28.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.030+0000 7f4831d5f700 1 Processor -- start 2024-01-27T02:38:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.030+0000 7f4831d5f700 1 -- start start 2024-01-27T02:38:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.031+0000 7f4831d5f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482c0c3a90 0x7f482c169f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.031+0000 7f4831d5f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f482c159f40 0x7f482c0c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.031+0000 7f4831d5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f482c15a310 0x7f482c159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.031+0000 7f4831d5f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0b2670 con 0x7f482c15a310 2024-01-27T02:38:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.031+0000 7f4831d5f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0b2950 con 0x7f482c0c3a90 2024-01-27T02:38:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.031+0000 7f4831d5f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482c16a730 con 0x7f482c159f40 2024-01-27T02:38:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.031+0000 7f482bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f482c15a310 0x7f482c159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.031+0000 7f482bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f482c15a310 0x7f482c159350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52682/0 (socket says 172.21.15.7:52682) 2024-01-27T02:38:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.031+0000 7f482bfff700 1 -- 172.21.15.7:0/1680711307 learned_addr learned my addr 172.21.15.7:0/1680711307 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:28.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.031+0000 7f482b7fe700 1 --2- 172.21.15.7:0/1680711307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482c0c3a90 0x7f482c169f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:28.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.032+0000 7f482affd700 1 --2- 172.21.15.7:0/1680711307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f482c159f40 0x7f482c0c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:28.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.032+0000 7f482bfff700 1 -- 172.21.15.7:0/1680711307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f482c159f40 msgr2=0x7f482c0c62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:28.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.032+0000 7f482bfff700 1 --2- 172.21.15.7:0/1680711307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f482c159f40 0x7f482c0c62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.032+0000 7f482bfff700 1 -- 172.21.15.7:0/1680711307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482c0c3a90 msgr2=0x7f482c169f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:28.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.032+0000 7f482bfff700 1 --2- 172.21.15.7:0/1680711307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482c0c3a90 0x7f482c169f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.032+0000 7f482bfff700 1 -- 172.21.15.7:0/1680711307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481c009420 con 0x7f482c15a310 2024-01-27T02:38:28.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.032+0000 7f482b7fe700 1 --2- 172.21.15.7:0/1680711307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482c0c3a90 0x7f482c169f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:28.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.032+0000 7f482bfff700 1 --2- 172.21.15.7:0/1680711307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f482c15a310 0x7f482c159350 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f482400cbf0 tx=0x7f4824001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:28.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.032+0000 7f48237fe700 1 -- 172.21.15.7:0/1680711307 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482400a980 con 0x7f482c15a310 2024-01-27T02:38:28.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.032+0000 7f48237fe700 1 -- 172.21.15.7:0/1680711307 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482400ab00 con 0x7f482c15a310 2024-01-27T02:38:28.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.032+0000 7f4831d5f700 1 -- 172.21.15.7:0/1680711307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4814009420 con 0x7f482c15a310 2024-01-27T02:38:28.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.032+0000 7f48237fe700 1 -- 172.21.15.7:0/1680711307 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48240070f0 con 0x7f482c15a310 2024-01-27T02:38:28.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.032+0000 7f4831d5f700 1 -- 172.21.15.7:0/1680711307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f482c16acf0 con 0x7f482c15a310 2024-01-27T02:38:28.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.033+0000 7f4831d5f700 1 -- 172.21.15.7:0/1680711307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f482c0cd300 con 0x7f482c15a310 2024-01-27T02:38:28.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.034+0000 7f48237fe700 1 -- 172.21.15.7:0/1680711307 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4824007520 con 0x7f482c15a310 2024-01-27T02:38:28.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.034+0000 7f48237fe700 1 --2- 172.21.15.7:0/1680711307 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f480c066f30 0x7f480c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:28.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.034+0000 7f48237fe700 1 -- 172.21.15.7:0/1680711307 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4824084500 con 0x7f482c15a310 2024-01-27T02:38:28.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.034+0000 7f482b7fe700 1 --2- 172.21.15.7:0/1680711307 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f480c066f30 0x7f480c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:28.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.035+0000 7f482b7fe700 1 --2- 172.21.15.7:0/1680711307 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f480c066f30 0x7f480c0693e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f481c0093f0 tx=0x7f481c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:28.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.038+0000 7f48237fe700 1 -- 172.21.15.7:0/1680711307 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4824051b40 con 0x7f482c15a310 2024-01-27T02:38:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.325+0000 7f4831d5f700 1 -- 172.21.15.7:0/1680711307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f482c04c710 con 0x7f482c15a310 2024-01-27T02:38:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.325+0000 7f48237fe700 1 -- 172.21.15.7:0/1680711307 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4824055160 con 0x7f482c15a310 2024-01-27T02:38:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:28.326 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:38:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-01-27T02:38:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-01-27T02:38:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlHdcPKBAAfhnh1I3ZSQEXWJXlXCs98Q==", 2024-01-27T02:38:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:38:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:38:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:38:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:38:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:38:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-01-27T02:38:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rl8eVYNRAAfP8rOqO50RpoNZ1+1j2DXQ==", 2024-01-27T02:38:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:38:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:38:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:38:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:38:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:38:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-01-27T02:38:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlJxRKMBAAS0j7/zZeqGglOxCK1pg0WA==", 2024-01-27T02:38:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:38:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:38:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:38:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:38:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:38:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-01-27T02:38:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlNwfUABAALRQAoHoGhniqCvTQOqbbug==", 2024-01-27T02:38:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:38:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:38:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:38:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:38:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:38:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-01-27T02:38:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7Rlh6VSJhAAttuCte1WhNEvUkv5x/IKQQ==", 2024-01-27T02:38:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:38:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:38:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:38:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:38:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-01-27T02:38:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7Rlu/RGLhAAKzaCNhzYKhsNhxkMvY++gw==", 2024-01-27T02:38:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:38:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:38:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:38:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:38:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-01-27T02:38:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7RliRMyNhAAr0tG3evmVV7ZlGxbxfWPMw==", 2024-01-27T02:38:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:38:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:38:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:38:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:38:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-01-27T02:38:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAPa7RlRuSNAhAA3e9atlb+ffvlnLhRYbWZPw==", 2024-01-27T02:38:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:38:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:38:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:38:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:38:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-01-27T02:38:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7RlfeA6DBAAXpklgbNHRO+5G+fdIERjRw==", 2024-01-27T02:38:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:38:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:38:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:38:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:38:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-01-27T02:38:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7Rl77ItExAABB7uABm63ubE/YUs2B5pGw==", 2024-01-27T02:38:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:38:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:38:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:38:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:38:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-01-27T02:38:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7RljDuwHBAAPPAlhoqS03qPpq58jmevLA==", 2024-01-27T02:38:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:38:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:38:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:38:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:38:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-01-27T02:38:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7Rl1YElJhAAqHP/QEdl1SEalGcbQwPwAg==", 2024-01-27T02:38:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:38:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:38:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:38:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:38:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-01-27T02:38:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlaEqaBRAATFiIeW8lT9BdNuN0m3x9dQ==", 2024-01-27T02:38:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:38:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:38:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-01-27T02:38:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-27T02:38:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-01-27T02:38:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlNADmEBAAJXwdJ35ylP5coy1dtHoD3g==", 2024-01-27T02:38:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:38:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:38:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:38:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:38:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:38:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-01-27T02:38:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7Rl5fRqCRAA/5swTGknnpP64TAh+2X41A==", 2024-01-27T02:38:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:38:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:38:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:38:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:38:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:38:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-01-27T02:38:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7Rl0SdwFhAAhCmw9ViiJbHVfS5a2r/D+w==", 2024-01-27T02:38:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:38:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:38:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:38:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:38:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:38:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-01-27T02:38:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlsNgPARAA0Qm0oJgA8GO1oUT3ndT8Tg==", 2024-01-27T02:38:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:38:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:38:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:38:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:38:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:38:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-01-27T02:38:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rlfb7qIxAAlsJYiSolJ7D8EykGWP62bA==", 2024-01-27T02:38:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:38:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:38:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-27T02:38:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:38:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-01-27T02:38:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rls7TaHxAALS3ghxKtfmgQfIELiD9+Bg==", 2024-01-27T02:38:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:38:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:38:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-27T02:38:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:38:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:28.339 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:38:28.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.328+0000 7f4831d5f700 1 -- 172.21.15.7:0/1680711307 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f480c066f30 msgr2=0x7f480c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:28.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.328+0000 7f4831d5f700 1 --2- 172.21.15.7:0/1680711307 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f480c066f30 0x7f480c0693e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f481c0093f0 tx=0x7f481c008040 comp rx=0 tx=0).stop 2024-01-27T02:38:28.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.328+0000 7f4831d5f700 1 -- 172.21.15.7:0/1680711307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f482c15a310 msgr2=0x7f482c159350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:28.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.328+0000 7f4831d5f700 1 --2- 172.21.15.7:0/1680711307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f482c15a310 0x7f482c159350 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f482400cbf0 tx=0x7f4824001980 comp rx=0 tx=0).stop 2024-01-27T02:38:28.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.328+0000 7f4831d5f700 1 -- 172.21.15.7:0/1680711307 shutdown_connections 2024-01-27T02:38:28.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.328+0000 7f4831d5f700 1 --2- 172.21.15.7:0/1680711307 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f480c066f30 0x7f480c0693e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.328+0000 7f4831d5f700 1 --2- 172.21.15.7:0/1680711307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f482c15a310 0x7f482c159350 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.328+0000 7f4831d5f700 1 --2- 172.21.15.7:0/1680711307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482c0c3a90 0x7f482c169f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.328+0000 7f4831d5f700 1 --2- 172.21.15.7:0/1680711307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f482c159f40 0x7f482c0c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.328+0000 7f4831d5f700 1 -- 172.21.15.7:0/1680711307 >> 172.21.15.7:0/1680711307 conn(0x7f482c0cdc30 msgr2=0x7f482c06a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:28.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.328+0000 7f4831d5f700 1 -- 172.21.15.7:0/1680711307 shutdown_connections 2024-01-27T02:38:28.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.328+0000 7f4831d5f700 1 -- 172.21.15.7:0/1680711307 wait complete. 2024-01-27T02:38:28.342 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:38:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.473+0000 7fa91a13c700 1 Processor -- start 2024-01-27T02:38:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.473+0000 7fa91a13c700 1 -- start start 2024-01-27T02:38:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.474+0000 7fa91a13c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa9140c3ae0 0x7fa9140cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.474+0000 7fa91a13c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa914158070 0x7fa9140cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.474+0000 7fa91a13c700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9140ca020 con 0x7fa914157ca0 2024-01-27T02:38:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.474+0000 7fa91a13c700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9140c5210 con 0x7fa9140c3ae0 2024-01-27T02:38:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.474+0000 7fa91a13c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9140c54f0 con 0x7fa914158070 2024-01-27T02:38:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.474+0000 7fa913fff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa914158070 0x7fa9140cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.474+0000 7fa912ffd700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa9140c3ae0 0x7fa9140cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.474+0000 7fa9137fe700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fa914157ca0 0x7fa9140c4a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:38832/0 (socket says 172.21.15.7:38832) 2024-01-27T02:38:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.475+0000 7fa9137fe700 1 -- 172.21.15.7:0/1124677047 learned_addr learned my addr 172.21.15.7:0/1124677047 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.475+0000 7fa912ffd700 1 -- 172.21.15.7:0/1124677047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa914158070 msgr2=0x7fa9140cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.475+0000 7fa912ffd700 1 --2- 172.21.15.7:0/1124677047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa914158070 0x7fa9140cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.475+0000 7fa912ffd700 1 -- 172.21.15.7:0/1124677047 >> v1:172.21.15.188:6789/0 conn(0x7fa914157ca0 legacy=0x7fa9140c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.475+0000 7fa912ffd700 1 -- 172.21.15.7:0/1124677047 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9140c57d0 con 0x7fa9140c3ae0 2024-01-27T02:38:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.475+0000 7fa9127fc700 1 -- 172.21.15.7:0/1124677047 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506809912 0 0) 0x7fa9140ca020 con 0x7fa914157ca0 2024-01-27T02:38:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.475+0000 7fa913fff700 1 --2- 172.21.15.7:0/1124677047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa914158070 0x7fa9140cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:38:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.475+0000 7fa912ffd700 1 --2- 172.21.15.7:0/1124677047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9140c3ae0 0x7fa9140cd4e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0098e0 tx=0x7fa8fc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ad718f933f6fcef server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.475+0000 7fa9127fc700 1 -- 172.21.15.7:0/1124677047 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc00b640 con 0x7fa9140c3ae0 2024-01-27T02:38:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.475+0000 7fa9127fc700 1 -- 172.21.15.7:0/1124677047 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8fc00b7a0 con 0x7fa9140c3ae0 2024-01-27T02:38:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.475+0000 7fa9127fc700 1 -- 172.21.15.7:0/1124677047 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc00bb40 con 0x7fa9140c3ae0 2024-01-27T02:38:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.476+0000 7fa91a13c700 1 -- 172.21.15.7:0/1124677047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9140c3ae0 msgr2=0x7fa9140cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.476+0000 7fa91a13c700 1 --2- 172.21.15.7:0/1124677047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9140c3ae0 0x7fa9140cd4e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0098e0 tx=0x7fa8fc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.476+0000 7fa91a13c700 1 -- 172.21.15.7:0/1124677047 shutdown_connections 2024-01-27T02:38:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.476+0000 7fa91a13c700 1 --2- 172.21.15.7:0/1124677047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa914158070 0x7fa9140cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.476+0000 7fa91a13c700 1 --2- 172.21.15.7:0/1124677047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9140c3ae0 0x7fa9140cd4e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.476+0000 7fa91a13c700 1 -- 172.21.15.7:0/1124677047 >> 172.21.15.7:0/1124677047 conn(0x7fa914098020 msgr2=0x7fa9140711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.476+0000 7fa91a13c700 1 -- 172.21.15.7:0/1124677047 shutdown_connections 2024-01-27T02:38:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.476+0000 7fa91a13c700 1 -- 172.21.15.7:0/1124677047 wait complete. 2024-01-27T02:38:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.477+0000 7fa91a13c700 1 Processor -- start 2024-01-27T02:38:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.477+0000 7fa91a13c700 1 -- start start 2024-01-27T02:38:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.478+0000 7fa91a13c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9140c3ae0 0x7fa9140c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.478+0000 7fa91a13c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa914157ca0 0x7fa91416abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.478+0000 7fa91a13c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa914158070 0x7fa914151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.478+0000 7fa91a13c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9140c8320 con 0x7fa9140c3ae0 2024-01-27T02:38:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.478+0000 7fa91a13c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa914152a60 con 0x7fa914157ca0 2024-01-27T02:38:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.478+0000 7fa91a13c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91416a740 con 0x7fa914158070 2024-01-27T02:38:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.478+0000 7fa9137fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9140c3ae0 0x7fa9140c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.478+0000 7fa9137fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9140c3ae0 0x7fa9140c7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52694/0 (socket says 172.21.15.7:52694) 2024-01-27T02:38:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.478+0000 7fa9137fe700 1 -- 172.21.15.7:0/423006094 learned_addr learned my addr 172.21.15.7:0/423006094 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.479+0000 7fa912ffd700 1 --2- 172.21.15.7:0/423006094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa914157ca0 0x7fa91416abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.479+0000 7fa913fff700 1 --2- 172.21.15.7:0/423006094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa914158070 0x7fa914151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.479+0000 7fa9137fe700 1 -- 172.21.15.7:0/423006094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa914158070 msgr2=0x7fa914151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.479+0000 7fa9137fe700 1 --2- 172.21.15.7:0/423006094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa914158070 0x7fa914151fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.479+0000 7fa9137fe700 1 -- 172.21.15.7:0/423006094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa914157ca0 msgr2=0x7fa91416abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.479+0000 7fa9137fe700 1 --2- 172.21.15.7:0/423006094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa914157ca0 0x7fa91416abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.479+0000 7fa9137fe700 1 -- 172.21.15.7:0/423006094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c009a20 con 0x7fa9140c3ae0 2024-01-27T02:38:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.479+0000 7fa9137fe700 1 --2- 172.21.15.7:0/423006094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9140c3ae0 0x7fa9140c7de0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fa90400c7e0 tx=0x7fa904007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.479+0000 7fa90b7fe700 1 -- 172.21.15.7:0/423006094 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa904005b30 con 0x7fa9140c3ae0 2024-01-27T02:38:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.479+0000 7fa90b7fe700 1 -- 172.21.15.7:0/423006094 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa904005c90 con 0x7fa9140c3ae0 2024-01-27T02:38:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.479+0000 7fa91a13c700 1 -- 172.21.15.7:0/423006094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8fc009420 con 0x7fa9140c3ae0 2024-01-27T02:38:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.479+0000 7fa90b7fe700 1 -- 172.21.15.7:0/423006094 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa904007300 con 0x7fa9140c3ae0 2024-01-27T02:38:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.479+0000 7fa91a13c700 1 -- 172.21.15.7:0/423006094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9140c8af0 con 0x7fa9140c3ae0 2024-01-27T02:38:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.480+0000 7fa91a13c700 1 -- 172.21.15.7:0/423006094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91404c710 con 0x7fa9140c3ae0 2024-01-27T02:38:28.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.481+0000 7fa90b7fe700 1 -- 172.21.15.7:0/423006094 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa9040076b0 con 0x7fa9140c3ae0 2024-01-27T02:38:28.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.481+0000 7fa90b7fe700 1 --2- 172.21.15.7:0/423006094 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa8f4066f30 0x7fa8f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:28.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.481+0000 7fa90b7fe700 1 -- 172.21.15.7:0/423006094 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fa904083690 con 0x7fa9140c3ae0 2024-01-27T02:38:28.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.481+0000 7fa912ffd700 1 --2- 172.21.15.7:0/423006094 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa8f4066f30 0x7fa8f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:28.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.482+0000 7fa912ffd700 1 --2- 172.21.15.7:0/423006094 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa8f4066f30 0x7fa8f40693e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0093f0 tx=0x7fa8fc017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:28.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.486+0000 7fa90b7fe700 1 -- 172.21.15.7:0/423006094 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa904050d10 con 0x7fa9140c3ae0 2024-01-27T02:38:28.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.690+0000 7fa91a13c700 1 -- 172.21.15.7:0/423006094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9140c9bb0 con 0x7fa9140c3ae0 2024-01-27T02:38:28.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.691+0000 7fa90b7fe700 1 -- 172.21.15.7:0/423006094 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+12097 (secure 0 0 0) 0x7fa904054330 con 0x7fa9140c3ae0 2024-01-27T02:38:28.691 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:28.692 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":128,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:38:27.868164+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:38:28.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.693+0000 7fa91a13c700 1 -- 172.21.15.7:0/423006094 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa8f4066f30 msgr2=0x7fa8f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:28.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.693+0000 7fa91a13c700 1 --2- 172.21.15.7:0/423006094 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa8f4066f30 0x7fa8f40693e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0093f0 tx=0x7fa8fc017000 comp rx=0 tx=0).stop 2024-01-27T02:38:28.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.693+0000 7fa91a13c700 1 -- 172.21.15.7:0/423006094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9140c3ae0 msgr2=0x7fa9140c7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:28.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.693+0000 7fa91a13c700 1 --2- 172.21.15.7:0/423006094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9140c3ae0 0x7fa9140c7de0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fa90400c7e0 tx=0x7fa904007df0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.693+0000 7fa91a13c700 1 -- 172.21.15.7:0/423006094 shutdown_connections 2024-01-27T02:38:28.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.693+0000 7fa91a13c700 1 --2- 172.21.15.7:0/423006094 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa8f4066f30 0x7fa8f40693e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.693+0000 7fa91a13c700 1 --2- 172.21.15.7:0/423006094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa914158070 0x7fa914151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.693+0000 7fa91a13c700 1 --2- 172.21.15.7:0/423006094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9140c3ae0 0x7fa9140c7de0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.693+0000 7fa91a13c700 1 --2- 172.21.15.7:0/423006094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa914157ca0 0x7fa91416abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.693+0000 7fa91a13c700 1 -- 172.21.15.7:0/423006094 >> 172.21.15.7:0/423006094 conn(0x7fa914098020 msgr2=0x7fa9140cc560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:28.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.693+0000 7fa91a13c700 1 -- 172.21.15.7:0/423006094 shutdown_connections 2024-01-27T02:38:28.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.693+0000 7fa91a13c700 1 -- 172.21.15.7:0/423006094 wait complete. 2024-01-27T02:38:28.707 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-27T02:38:28.708 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:38:28.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.842+0000 7f677137a700 1 Processor -- start 2024-01-27T02:38:28.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.842+0000 7f677137a700 1 -- start start 2024-01-27T02:38:28.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.843+0000 7f677137a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f676c0cc110 0x7f676c0cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:28.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.843+0000 7f677137a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f676c159c00 0x7f676c15a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:28.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.843+0000 7f677137a700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f676c0cfb40 con 0x7f676c15ea60 2024-01-27T02:38:28.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.843+0000 7f677137a700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f676c0cfe20 con 0x7f676c159c00 2024-01-27T02:38:28.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.843+0000 7f677137a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676c0d0100 con 0x7f676c0cc110 2024-01-27T02:38:28.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.843+0000 7f676b7fe700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f676c159c00 0x7f676c15a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:28.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.843+0000 7f676a7fc700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f676c15ea60 0x7f676c0cf2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:38848/0 (socket says 172.21.15.7:38848) 2024-01-27T02:38:28.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.843+0000 7f676a7fc700 1 -- 172.21.15.7:0/1915008350 learned_addr learned my addr 172.21.15.7:0/1915008350 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:28.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.843+0000 7f676affd700 1 --2- 172.21.15.7:0/1915008350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f676c0cc110 0x7f676c0cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:28.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.844+0000 7f6769ffb700 1 -- 172.21.15.7:0/1915008350 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084174352 0 0) 0x7f676c0cfb40 con 0x7f676c15ea60 2024-01-27T02:38:28.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.844+0000 7f6769ffb700 1 -- 172.21.15.7:0/1915008350 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6758003570 con 0x7f676c15ea60 2024-01-27T02:38:28.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.844+0000 7f676b7fe700 1 -- 172.21.15.7:0/1915008350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f676c0cc110 msgr2=0x7f676c0cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:28.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.844+0000 7f676b7fe700 1 --2- 172.21.15.7:0/1915008350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f676c0cc110 0x7f676c0cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.844+0000 7f676b7fe700 1 -- 172.21.15.7:0/1915008350 >> v1:172.21.15.188:6789/0 conn(0x7f676c15ea60 legacy=0x7f676c0cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:28.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.844+0000 7f676b7fe700 1 -- 172.21.15.7:0/1915008350 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676c0d03e0 con 0x7f676c159c00 2024-01-27T02:38:28.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.844+0000 7f676affd700 1 --2- 172.21.15.7:0/1915008350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f676c0cc110 0x7f676c0cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:38:28.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.844+0000 7f676b7fe700 1 --2- 172.21.15.7:0/1915008350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f676c159c00 0x7f676c15a080 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6764009e60 tx=0x7f6764009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f79abbb3c0eb3c4 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:28.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.844+0000 7f6769ffb700 1 -- 172.21.15.7:0/1915008350 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676400e070 con 0x7f676c159c00 2024-01-27T02:38:28.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.844+0000 7f6769ffb700 1 -- 172.21.15.7:0/1915008350 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676400bab0 con 0x7f676c159c00 2024-01-27T02:38:28.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.844+0000 7f6769ffb700 1 -- 172.21.15.7:0/1915008350 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676400be70 con 0x7f676c159c00 2024-01-27T02:38:28.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.845+0000 7f677137a700 1 -- 172.21.15.7:0/1915008350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f676c159c00 msgr2=0x7f676c15a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:28.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.845+0000 7f677137a700 1 --2- 172.21.15.7:0/1915008350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f676c159c00 0x7f676c15a080 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6764009e60 tx=0x7f6764009780 comp rx=0 tx=0).stop 2024-01-27T02:38:28.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.845+0000 7f677137a700 1 -- 172.21.15.7:0/1915008350 shutdown_connections 2024-01-27T02:38:28.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.845+0000 7f677137a700 1 --2- 172.21.15.7:0/1915008350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f676c159c00 0x7f676c15a080 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.845+0000 7f677137a700 1 --2- 172.21.15.7:0/1915008350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f676c0cc110 0x7f676c0cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.845+0000 7f677137a700 1 -- 172.21.15.7:0/1915008350 >> 172.21.15.7:0/1915008350 conn(0x7f676c063580 msgr2=0x7f676c063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:28.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.845+0000 7f677137a700 1 -- 172.21.15.7:0/1915008350 shutdown_connections 2024-01-27T02:38:28.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.845+0000 7f677137a700 1 -- 172.21.15.7:0/1915008350 wait complete. 2024-01-27T02:38:28.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.846+0000 7f677137a700 1 Processor -- start 2024-01-27T02:38:28.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.846+0000 7f677137a700 1 -- start start 2024-01-27T02:38:28.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.847+0000 7f677137a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f676c0cc110 0x7f676c16d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:28.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.847+0000 7f677137a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f676c159c00 0x7f676c157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:28.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.847+0000 7f677137a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f676c15ea60 0x7f676c157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:28.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.847+0000 7f677137a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c155d60 con 0x7f676c15ea60 2024-01-27T02:38:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.847+0000 7f677137a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c158080 con 0x7f676c0cc110 2024-01-27T02:38:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.847+0000 7f677137a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676c158360 con 0x7f676c159c00 2024-01-27T02:38:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.847+0000 7f676b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f676c15ea60 0x7f676c157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.847+0000 7f676b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f676c15ea60 0x7f676c157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52702/0 (socket says 172.21.15.7:52702) 2024-01-27T02:38:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.847+0000 7f676b7fe700 1 -- 172.21.15.7:0/2139474438 learned_addr learned my addr 172.21.15.7:0/2139474438 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.847+0000 7f676affd700 1 --2- 172.21.15.7:0/2139474438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f676c0cc110 0x7f676c16d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.847+0000 7f676a7fc700 1 --2- 172.21.15.7:0/2139474438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f676c159c00 0x7f676c157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.847+0000 7f676b7fe700 1 -- 172.21.15.7:0/2139474438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f676c159c00 msgr2=0x7f676c157450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.848+0000 7f676b7fe700 1 --2- 172.21.15.7:0/2139474438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f676c159c00 0x7f676c157450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.848+0000 7f676b7fe700 1 -- 172.21.15.7:0/2139474438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f676c0cc110 msgr2=0x7f676c16d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.848+0000 7f676b7fe700 1 --2- 172.21.15.7:0/2139474438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f676c0cc110 0x7f676c16d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.848+0000 7f676b7fe700 1 -- 172.21.15.7:0/2139474438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675c009420 con 0x7f676c15ea60 2024-01-27T02:38:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.848+0000 7f676a7fc700 1 --2- 172.21.15.7:0/2139474438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f676c159c00 0x7f676c157450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.848+0000 7f676b7fe700 1 --2- 172.21.15.7:0/2139474438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f676c15ea60 0x7f676c157990 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f676c08a040 tx=0x7f6764019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:28.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.848+0000 7f6762ffd700 1 -- 172.21.15.7:0/2139474438 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6764027470 con 0x7f676c15ea60 2024-01-27T02:38:28.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.848+0000 7f6762ffd700 1 -- 172.21.15.7:0/2139474438 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67640275f0 con 0x7f676c15ea60 2024-01-27T02:38:28.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.848+0000 7f677137a700 1 -- 172.21.15.7:0/2139474438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6764009a20 con 0x7f676c15ea60 2024-01-27T02:38:28.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.848+0000 7f6762ffd700 1 -- 172.21.15.7:0/2139474438 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67640279b0 con 0x7f676c15ea60 2024-01-27T02:38:28.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.848+0000 7f677137a700 1 -- 172.21.15.7:0/2139474438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676c158920 con 0x7f676c15ea60 2024-01-27T02:38:28.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.849+0000 7f677137a700 1 -- 172.21.15.7:0/2139474438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f676c072060 con 0x7f676c15ea60 2024-01-27T02:38:28.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.849+0000 7f6762ffd700 1 -- 172.21.15.7:0/2139474438 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6764024070 con 0x7f676c15ea60 2024-01-27T02:38:28.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.850+0000 7f6762ffd700 1 --2- 172.21.15.7:0/2139474438 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f674c066f30 0x7f674c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:28.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.850+0000 7f6762ffd700 1 -- 172.21.15.7:0/2139474438 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6764086a60 con 0x7f676c15ea60 2024-01-27T02:38:28.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.850+0000 7f676affd700 1 --2- 172.21.15.7:0/2139474438 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f674c066f30 0x7f674c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:28.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.851+0000 7f676affd700 1 --2- 172.21.15.7:0/2139474438 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f674c066f30 0x7f674c0693e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f675c009f40 tx=0x7f675c0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:28.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:28.854+0000 7f6762ffd700 1 -- 172.21.15.7:0/2139474438 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6764054020 con 0x7f676c15ea60 2024-01-27T02:38:29.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.059+0000 7f677137a700 1 -- 172.21.15.7:0/2139474438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f676c0d06c0 con 0x7f676c15ea60 2024-01-27T02:38:29.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.059+0000 7f6762ffd700 1 -- 172.21.15.7:0/2139474438 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6764057640 con 0x7f676c15ea60 2024-01-27T02:38:29.060 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:38:29.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.061+0000 7f677137a700 1 -- 172.21.15.7:0/2139474438 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f674c066f30 msgr2=0x7f674c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:29.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.061+0000 7f677137a700 1 --2- 172.21.15.7:0/2139474438 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f674c066f30 0x7f674c0693e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f675c009f40 tx=0x7f675c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.061+0000 7f677137a700 1 -- 172.21.15.7:0/2139474438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f676c15ea60 msgr2=0x7f676c157990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:29.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.061+0000 7f677137a700 1 --2- 172.21.15.7:0/2139474438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f676c15ea60 0x7f676c157990 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f676c08a040 tx=0x7f6764019000 comp rx=0 tx=0).stop 2024-01-27T02:38:29.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.062+0000 7f677137a700 1 -- 172.21.15.7:0/2139474438 shutdown_connections 2024-01-27T02:38:29.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.062+0000 7f677137a700 1 --2- 172.21.15.7:0/2139474438 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f674c066f30 0x7f674c0693e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.062+0000 7f677137a700 1 --2- 172.21.15.7:0/2139474438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f676c15ea60 0x7f676c157990 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.062+0000 7f677137a700 1 --2- 172.21.15.7:0/2139474438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f676c0cc110 0x7f676c16d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.062+0000 7f677137a700 1 --2- 172.21.15.7:0/2139474438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f676c159c00 0x7f676c157450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.062+0000 7f677137a700 1 -- 172.21.15.7:0/2139474438 >> 172.21.15.7:0/2139474438 conn(0x7f676c063580 msgr2=0x7f676c15ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:29.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.062+0000 7f677137a700 1 -- 172.21.15.7:0/2139474438 shutdown_connections 2024-01-27T02:38:29.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.062+0000 7f677137a700 1 -- 172.21.15.7:0/2139474438 wait complete. 2024-01-27T02:38:29.076 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-27T02:38:29.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.207+0000 7f84ca4b7700 1 Processor -- start 2024-01-27T02:38:29.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.207+0000 7f84ca4b7700 1 -- start start 2024-01-27T02:38:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.208+0000 7f84ca4b7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f84c406a8d0 0x7f84c406aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.208+0000 7f84ca4b7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84c4158ea0 0x7f84c4063a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.208+0000 7f84ca4b7700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c415bc90 con 0x7f84c4063580 2024-01-27T02:38:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.208+0000 7f84ca4b7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c41594c0 con 0x7f84c406a8d0 2024-01-27T02:38:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.208+0000 7f84ca4b7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c4063f60 con 0x7f84c4158ea0 2024-01-27T02:38:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.208+0000 7f84c3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f84c406a8d0 0x7f84c406aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.208+0000 7f84c3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f84c406a8d0 0x7f84c406aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52716/0 (socket says 172.21.15.7:52716) 2024-01-27T02:38:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.208+0000 7f84c3fff700 1 -- 172.21.15.7:0/2478237806 learned_addr learned my addr 172.21.15.7:0/2478237806 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.208+0000 7f84c8a54700 1 --2- 172.21.15.7:0/2478237806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84c4158ea0 0x7f84c4063a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.208+0000 7f84c3fff700 1 -- 172.21.15.7:0/2478237806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84c4158ea0 msgr2=0x7f84c4063a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.208+0000 7f84c3fff700 1 --2- 172.21.15.7:0/2478237806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84c4158ea0 0x7f84c4063a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.208+0000 7f84c3fff700 1 -- 172.21.15.7:0/2478237806 >> v1:172.21.15.188:6789/0 conn(0x7f84c4063580 legacy=0x7f84c4158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.208+0000 7f84c3fff700 1 -- 172.21.15.7:0/2478237806 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c415bf70 con 0x7f84c406a8d0 2024-01-27T02:38:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.208+0000 7f84c8a54700 1 --2- 172.21.15.7:0/2478237806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84c4158ea0 0x7f84c4063a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:38:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.208+0000 7f84c3fff700 1 --2- 172.21.15.7:0/2478237806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84c406a8d0 0x7f84c406aca0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f84b40099b0 tx=0x7f84b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21fef07bc8c15c5e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.209+0000 7f84c2ffd700 1 -- 172.21.15.7:0/2478237806 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b400b860 con 0x7f84c406a8d0 2024-01-27T02:38:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.209+0000 7f84c2ffd700 1 -- 172.21.15.7:0/2478237806 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b400b9c0 con 0x7f84c406a8d0 2024-01-27T02:38:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.209+0000 7f84c2ffd700 1 -- 172.21.15.7:0/2478237806 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b400bd60 con 0x7f84c406a8d0 2024-01-27T02:38:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.209+0000 7f84ca4b7700 1 -- 172.21.15.7:0/2478237806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84c406a8d0 msgr2=0x7f84c406aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.209+0000 7f84ca4b7700 1 --2- 172.21.15.7:0/2478237806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84c406a8d0 0x7f84c406aca0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f84b40099b0 tx=0x7f84b4009250 comp rx=0 tx=0).stop 2024-01-27T02:38:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.209+0000 7f84ca4b7700 1 -- 172.21.15.7:0/2478237806 shutdown_connections 2024-01-27T02:38:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.209+0000 7f84ca4b7700 1 --2- 172.21.15.7:0/2478237806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84c4158ea0 0x7f84c4063a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.209+0000 7f84ca4b7700 1 --2- 172.21.15.7:0/2478237806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84c406a8d0 0x7f84c406aca0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.209+0000 7f84ca4b7700 1 -- 172.21.15.7:0/2478237806 >> 172.21.15.7:0/2478237806 conn(0x7f84c40cdc30 msgr2=0x7f84c40ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:29.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.209+0000 7f84ca4b7700 1 -- 172.21.15.7:0/2478237806 shutdown_connections 2024-01-27T02:38:29.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.209+0000 7f84ca4b7700 1 -- 172.21.15.7:0/2478237806 wait complete. 2024-01-27T02:38:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.210+0000 7f84ca4b7700 1 Processor -- start 2024-01-27T02:38:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.211+0000 7f84ca4b7700 1 -- start start 2024-01-27T02:38:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.211+0000 7f84ca4b7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84c4063580 0x7f84c4152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.211+0000 7f84ca4b7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84c406a8d0 0x7f84c416e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.211+0000 7f84ca4b7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84c4158ea0 0x7f84c413a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:29.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.211+0000 7f84ca4b7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c4153430 con 0x7f84c4158ea0 2024-01-27T02:38:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.211+0000 7f84ca4b7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c416ec60 con 0x7f84c4063580 2024-01-27T02:38:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.211+0000 7f84ca4b7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c41527d0 con 0x7f84c406a8d0 2024-01-27T02:38:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.212+0000 7f84c8a54700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84c4158ea0 0x7f84c413a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.212+0000 7f84c8a54700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84c4158ea0 0x7f84c413a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52718/0 (socket says 172.21.15.7:52718) 2024-01-27T02:38:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.212+0000 7f84c8a54700 1 -- 172.21.15.7:0/692539914 learned_addr learned my addr 172.21.15.7:0/692539914 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.212+0000 7f84c37fe700 1 --2- 172.21.15.7:0/692539914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84c406a8d0 0x7f84c416e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.212+0000 7f84c3fff700 1 --2- 172.21.15.7:0/692539914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84c4063580 0x7f84c4152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.212+0000 7f84c8a54700 1 -- 172.21.15.7:0/692539914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84c406a8d0 msgr2=0x7f84c416e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.212+0000 7f84c8a54700 1 --2- 172.21.15.7:0/692539914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84c406a8d0 0x7f84c416e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.212+0000 7f84c8a54700 1 -- 172.21.15.7:0/692539914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84c4063580 msgr2=0x7f84c4152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.212+0000 7f84c8a54700 1 --2- 172.21.15.7:0/692539914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84c4063580 0x7f84c4152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.212+0000 7f84c8a54700 1 -- 172.21.15.7:0/692539914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b40094f0 con 0x7f84c4158ea0 2024-01-27T02:38:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.212+0000 7f84c8a54700 1 --2- 172.21.15.7:0/692539914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84c4158ea0 0x7f84c413a590 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00cef0 tx=0x7f84bc007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.212+0000 7f84abfff700 1 -- 172.21.15.7:0/692539914 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc00ae00 con 0x7f84c4158ea0 2024-01-27T02:38:29.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.212+0000 7f84abfff700 1 -- 172.21.15.7:0/692539914 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84bc016420 con 0x7f84c4158ea0 2024-01-27T02:38:29.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.212+0000 7f84ca4b7700 1 -- 172.21.15.7:0/692539914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c4152b00 con 0x7f84c4158ea0 2024-01-27T02:38:29.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.212+0000 7f84abfff700 1 -- 172.21.15.7:0/692539914 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc0167e0 con 0x7f84c4158ea0 2024-01-27T02:38:29.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.212+0000 7f84ca4b7700 1 -- 172.21.15.7:0/692539914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c41530c0 con 0x7f84c4158ea0 2024-01-27T02:38:29.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.213+0000 7f84ca4b7700 1 -- 172.21.15.7:0/692539914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c41535e0 con 0x7f84c4158ea0 2024-01-27T02:38:29.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.214+0000 7f84abfff700 1 -- 172.21.15.7:0/692539914 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f84bc016bd0 con 0x7f84c4158ea0 2024-01-27T02:38:29.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.214+0000 7f84abfff700 1 --2- 172.21.15.7:0/692539914 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f84b0066f30 0x7f84b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:29.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.215+0000 7f84abfff700 1 -- 172.21.15.7:0/692539914 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f84bc012070 con 0x7f84c4158ea0 2024-01-27T02:38:29.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.215+0000 7f84c3fff700 1 --2- 172.21.15.7:0/692539914 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f84b0066f30 0x7f84b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:29.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.215+0000 7f84c3fff700 1 --2- 172.21.15.7:0/692539914 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f84b0066f30 0x7f84b00693e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f84b40099b0 tx=0x7f84b4017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:29.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.219+0000 7f84abfff700 1 -- 172.21.15.7:0/692539914 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f84bc04e5a0 con 0x7f84c4158ea0 2024-01-27T02:38:29.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.423+0000 7f84ca4b7700 1 -- 172.21.15.7:0/692539914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f84c404c710 con 0x7f84c4158ea0 2024-01-27T02:38:29.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.424+0000 7f84abfff700 1 -- 172.21.15.7:0/692539914 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v128) v1 ==== 84+0+65 (secure 0 0 0) 0x7f84c41535e0 con 0x7f84c4158ea0 2024-01-27T02:38:29.424 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:29.424 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-27T02:38:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-27T02:38:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-27T02:38:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:29.425 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-27T02:38:29.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.426+0000 7f84ca4b7700 1 -- 172.21.15.7:0/692539914 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f84b0066f30 msgr2=0x7f84b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:29.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.426+0000 7f84ca4b7700 1 --2- 172.21.15.7:0/692539914 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f84b0066f30 0x7f84b00693e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f84b40099b0 tx=0x7f84b4017000 comp rx=0 tx=0).stop 2024-01-27T02:38:29.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.426+0000 7f84ca4b7700 1 -- 172.21.15.7:0/692539914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84c4158ea0 msgr2=0x7f84c413a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:29.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.426+0000 7f84ca4b7700 1 --2- 172.21.15.7:0/692539914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84c4158ea0 0x7f84c413a590 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00cef0 tx=0x7f84bc007860 comp rx=0 tx=0).stop 2024-01-27T02:38:29.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.426+0000 7f84ca4b7700 1 -- 172.21.15.7:0/692539914 shutdown_connections 2024-01-27T02:38:29.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.426+0000 7f84ca4b7700 1 --2- 172.21.15.7:0/692539914 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f84b0066f30 0x7f84b00693e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.426+0000 7f84ca4b7700 1 --2- 172.21.15.7:0/692539914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84c4158ea0 0x7f84c413a590 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.426+0000 7f84ca4b7700 1 --2- 172.21.15.7:0/692539914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84c4063580 0x7f84c4152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.426+0000 7f84ca4b7700 1 --2- 172.21.15.7:0/692539914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84c406a8d0 0x7f84c416e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.426+0000 7f84ca4b7700 1 -- 172.21.15.7:0/692539914 >> 172.21.15.7:0/692539914 conn(0x7f84c40cdc30 msgr2=0x7f84c4157d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:29.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.426+0000 7f84ca4b7700 1 -- 172.21.15.7:0/692539914 shutdown_connections 2024-01-27T02:38:29.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.427+0000 7f84ca4b7700 1 -- 172.21.15.7:0/692539914 wait complete. 2024-01-27T02:38:29.440 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-27T02:38:29.441 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-27T02:38:29.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.575+0000 7f5dbe447700 1 Processor -- start 2024-01-27T02:38:29.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.575+0000 7f5dbe447700 1 -- start start 2024-01-27T02:38:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.576+0000 7f5dbe447700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5db8141700 0x7f5db806ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.576+0000 7f5dbe447700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5db80caf20 0x7f5db80cb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.576+0000 7f5dbe447700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db80c9720 con 0x7f5db80ce330 2024-01-27T02:38:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.576+0000 7f5dbe447700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db80cd950 con 0x7f5db80caf20 2024-01-27T02:38:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.576+0000 7f5dbe447700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db806c2b0 con 0x7f5db8141700 2024-01-27T02:38:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.576+0000 7f5db77fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5db80ce330 0x7f5db806cfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49284/0 (socket says 172.21.15.7:49284) 2024-01-27T02:38:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.576+0000 7f5db77fe700 1 -- 172.21.15.7:0/303790212 learned_addr learned my addr 172.21.15.7:0/303790212 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.576+0000 7f5db7fff700 1 --2- 172.21.15.7:0/303790212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5db8141700 0x7f5db806ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.576+0000 7f5dbc9e4700 1 --2- 172.21.15.7:0/303790212 >> v2:172.21.15.188:3300/0 conn(0x7f5db80caf20 0x7f5db80cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.577+0000 7f5db6ffd700 1 -- 172.21.15.7:0/303790212 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3399706187 0 0) 0x7f5da0001110 con 0x7f5db80ce330 2024-01-27T02:38:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.577+0000 7f5db6ffd700 1 -- 172.21.15.7:0/303790212 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5da4003570 con 0x7f5db80ce330 2024-01-27T02:38:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.577+0000 7f5db7fff700 1 -- 172.21.15.7:0/303790212 >> v2:172.21.15.188:3300/0 conn(0x7f5db80caf20 msgr2=0x7f5db80cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:29.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.577+0000 7f5db7fff700 1 --2- 172.21.15.7:0/303790212 >> v2:172.21.15.188:3300/0 conn(0x7f5db80caf20 0x7f5db80cb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.577+0000 7f5db7fff700 1 -- 172.21.15.7:0/303790212 >> v1:172.21.15.7:6789/0 conn(0x7f5db80ce330 legacy=0x7f5db806cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:29.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.577+0000 7f5db7fff700 1 -- 172.21.15.7:0/303790212 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db80cd670 con 0x7f5db8141700 2024-01-27T02:38:29.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.577+0000 7f5db7fff700 1 --2- 172.21.15.7:0/303790212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5db8141700 0x7f5db806ca60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5da80098e0 tx=0x7f5da80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fa164509afe8fa4 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:29.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.577+0000 7f5db6ffd700 1 -- 172.21.15.7:0/303790212 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da8011630 con 0x7f5db8141700 2024-01-27T02:38:29.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.577+0000 7f5db6ffd700 1 -- 172.21.15.7:0/303790212 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da8011790 con 0x7f5db8141700 2024-01-27T02:38:29.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.577+0000 7f5db6ffd700 1 -- 172.21.15.7:0/303790212 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da8011b30 con 0x7f5db8141700 2024-01-27T02:38:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.578+0000 7f5dbe447700 1 -- 172.21.15.7:0/303790212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5db8141700 msgr2=0x7f5db806ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.578+0000 7f5dbe447700 1 --2- 172.21.15.7:0/303790212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5db8141700 0x7f5db806ca60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5da80098e0 tx=0x7f5da80091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.578+0000 7f5dbe447700 1 -- 172.21.15.7:0/303790212 shutdown_connections 2024-01-27T02:38:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.578+0000 7f5dbe447700 1 --2- 172.21.15.7:0/303790212 >> v2:172.21.15.188:3300/0 conn(0x7f5db80caf20 0x7f5db80cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.578+0000 7f5dbe447700 1 --2- 172.21.15.7:0/303790212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5db8141700 0x7f5db806ca60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.578+0000 7f5dbe447700 1 -- 172.21.15.7:0/303790212 >> 172.21.15.7:0/303790212 conn(0x7f5db80714b0 msgr2=0x7f5db806b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.578+0000 7f5dbe447700 1 -- 172.21.15.7:0/303790212 shutdown_connections 2024-01-27T02:38:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.578+0000 7f5dbe447700 1 -- 172.21.15.7:0/303790212 wait complete. 2024-01-27T02:38:29.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.579+0000 7f5dbe447700 1 Processor -- start 2024-01-27T02:38:29.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.580+0000 7f5dbe447700 1 -- start start 2024-01-27T02:38:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.580+0000 7f5dbe447700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5db80caf20 0x7f5db813e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.580+0000 7f5dbe447700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5db80ce330 0x7f5db80cf060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.580+0000 7f5dbe447700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5db8141700 0x7f5db80cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.580+0000 7f5dbe447700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db80b4ac0 con 0x7f5db80caf20 2024-01-27T02:38:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.580+0000 7f5dbe447700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db80b1bf0 con 0x7f5db80ce330 2024-01-27T02:38:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.580+0000 7f5dbe447700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db80b4500 con 0x7f5db8141700 2024-01-27T02:38:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.580+0000 7f5db7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5db80caf20 0x7f5db813e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:29.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.580+0000 7f5db7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5db80caf20 0x7f5db813e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52720/0 (socket says 172.21.15.7:52720) 2024-01-27T02:38:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.580+0000 7f5db7fff700 1 -- 172.21.15.7:0/515247553 learned_addr learned my addr 172.21.15.7:0/515247553 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.581+0000 7f5db77fe700 1 --2- 172.21.15.7:0/515247553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5db80ce330 0x7f5db80cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.581+0000 7f5dbc9e4700 1 --2- 172.21.15.7:0/515247553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5db8141700 0x7f5db80cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.581+0000 7f5db7fff700 1 -- 172.21.15.7:0/515247553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5db8141700 msgr2=0x7f5db80cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.581+0000 7f5db7fff700 1 --2- 172.21.15.7:0/515247553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5db8141700 0x7f5db80cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.581+0000 7f5db7fff700 1 -- 172.21.15.7:0/515247553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5db80ce330 msgr2=0x7f5db80cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.581+0000 7f5db7fff700 1 --2- 172.21.15.7:0/515247553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5db80ce330 0x7f5db80cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.581+0000 7f5db7fff700 1 -- 172.21.15.7:0/515247553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da8009420 con 0x7f5db80caf20 2024-01-27T02:38:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.581+0000 7f5db7fff700 1 --2- 172.21.15.7:0/515247553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5db80caf20 0x7f5db813e740 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5db806c3f0 tx=0x7f5da8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.581+0000 7f5daffff700 1 -- 172.21.15.7:0/515247553 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da8011900 con 0x7f5db80caf20 2024-01-27T02:38:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.581+0000 7f5daffff700 1 -- 172.21.15.7:0/515247553 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da8011a80 con 0x7f5db80caf20 2024-01-27T02:38:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.581+0000 7f5daffff700 1 -- 172.21.15.7:0/515247553 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da8025640 con 0x7f5db80caf20 2024-01-27T02:38:29.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.581+0000 7f5dbe447700 1 -- 172.21.15.7:0/515247553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db80b47e0 con 0x7f5db80caf20 2024-01-27T02:38:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.581+0000 7f5dbe447700 1 -- 172.21.15.7:0/515247553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db813ec80 con 0x7f5db80caf20 2024-01-27T02:38:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.582+0000 7f5dbe447700 1 -- 172.21.15.7:0/515247553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db804c710 con 0x7f5db80caf20 2024-01-27T02:38:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.583+0000 7f5daffff700 1 -- 172.21.15.7:0/515247553 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5da80259f0 con 0x7f5db80caf20 2024-01-27T02:38:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.583+0000 7f5daffff700 1 --2- 172.21.15.7:0/515247553 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5d98066f30 0x7f5d980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.583+0000 7f5daffff700 1 -- 172.21.15.7:0/515247553 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5da8085800 con 0x7f5db80caf20 2024-01-27T02:38:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.583+0000 7f5db77fe700 1 --2- 172.21.15.7:0/515247553 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5d98066f30 0x7f5d980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.584+0000 7f5db77fe700 1 --2- 172.21.15.7:0/515247553 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5d98066f30 0x7f5d980693e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5db816d9d0 tx=0x7f5da0007bf0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:29.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.588+0000 7f5daffff700 1 -- 172.21.15.7:0/515247553 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5da8052dc0 con 0x7f5db80caf20 2024-01-27T02:38:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.792+0000 7f5dbe447700 1 -- 172.21.15.7:0/515247553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5db806bc60 con 0x7f5db80caf20 2024-01-27T02:38:29.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.888+0000 7f5daffff700 1 -- 172.21.15.7:0/515247553 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v129) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5da80150b0 con 0x7f5db80caf20 2024-01-27T02:38:29.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.892+0000 7f5dbe447700 1 -- 172.21.15.7:0/515247553 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5d98066f30 msgr2=0x7f5d980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:29.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.892+0000 7f5dbe447700 1 --2- 172.21.15.7:0/515247553 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5d98066f30 0x7f5d980693e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5db816d9d0 tx=0x7f5da0007bf0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.892+0000 7f5dbe447700 1 -- 172.21.15.7:0/515247553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5db80caf20 msgr2=0x7f5db813e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:29.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.892+0000 7f5dbe447700 1 --2- 172.21.15.7:0/515247553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5db80caf20 0x7f5db813e740 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5db806c3f0 tx=0x7f5da8017000 comp rx=0 tx=0).stop 2024-01-27T02:38:29.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.892+0000 7f5dbe447700 1 -- 172.21.15.7:0/515247553 shutdown_connections 2024-01-27T02:38:29.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.892+0000 7f5dbe447700 1 --2- 172.21.15.7:0/515247553 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5d98066f30 0x7f5d980693e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.892+0000 7f5dbe447700 1 --2- 172.21.15.7:0/515247553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5db8141700 0x7f5db80cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.892+0000 7f5dbe447700 1 --2- 172.21.15.7:0/515247553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5db80caf20 0x7f5db813e740 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.892+0000 7f5dbe447700 1 --2- 172.21.15.7:0/515247553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5db80ce330 0x7f5db80cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:29.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.892+0000 7f5dbe447700 1 -- 172.21.15.7:0/515247553 >> 172.21.15.7:0/515247553 conn(0x7f5db80714b0 msgr2=0x7f5db80c5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:29.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.892+0000 7f5dbe447700 1 -- 172.21.15.7:0/515247553 shutdown_connections 2024-01-27T02:38:29.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:29.892+0000 7f5dbe447700 1 -- 172.21.15.7:0/515247553 wait complete. 2024-01-27T02:38:29.894 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-27T02:38:29.907 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-27T02:38:30.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.047+0000 7f8bbf867700 1 Processor -- start 2024-01-27T02:38:30.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.048+0000 7f8bbf867700 1 -- start start 2024-01-27T02:38:30.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.048+0000 7f8bbf867700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8bb8143670 0x7f8bb8063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:30.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.048+0000 7f8bbf867700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bb80aa8b0 0x7f8bb8063ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:30.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.049+0000 7f8bbf867700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb80cc290 con 0x7f8bb8143a40 2024-01-27T02:38:30.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.049+0000 7f8bbf867700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb80cb960 con 0x7f8bb8143670 2024-01-27T02:38:30.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.049+0000 7f8bbf867700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb80cbc40 con 0x7f8bb80aa8b0 2024-01-27T02:38:30.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.049+0000 7f8bbde04700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8bb8143a40 0x7f8bb80c8230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49296/0 (socket says 172.21.15.7:49296) 2024-01-27T02:38:30.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.049+0000 7f8bbde04700 1 -- 172.21.15.7:0/3089160073 learned_addr learned my addr 172.21.15.7:0/3089160073 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:30.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.049+0000 7f8bbce02700 1 --2- 172.21.15.7:0/3089160073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bb80aa8b0 0x7f8bb8063ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:30.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.049+0000 7f8bbd603700 1 --2- 172.21.15.7:0/3089160073 >> v2:172.21.15.188:3300/0 conn(0x7f8bb8143670 0x7f8bb8063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:30.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.049+0000 7f8baffff700 1 -- 172.21.15.7:0/3089160073 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284237617 0 0) 0x7f8bb80cc290 con 0x7f8bb8143a40 2024-01-27T02:38:30.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.049+0000 7f8baffff700 1 -- 172.21.15.7:0/3089160073 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ba4003570 con 0x7f8bb8143a40 2024-01-27T02:38:30.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.049+0000 7f8bbce02700 1 -- 172.21.15.7:0/3089160073 >> v2:172.21.15.188:3300/0 conn(0x7f8bb8143670 msgr2=0x7f8bb8063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:30.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.049+0000 7f8bbce02700 1 --2- 172.21.15.7:0/3089160073 >> v2:172.21.15.188:3300/0 conn(0x7f8bb8143670 0x7f8bb8063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:30.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.049+0000 7f8bbce02700 1 -- 172.21.15.7:0/3089160073 >> v1:172.21.15.7:6789/0 conn(0x7f8bb8143a40 legacy=0x7f8bb80c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:30.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.049+0000 7f8bbce02700 1 -- 172.21.15.7:0/3089160073 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb80cbf20 con 0x7f8bb80aa8b0 2024-01-27T02:38:30.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.050+0000 7f8bbce02700 1 --2- 172.21.15.7:0/3089160073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bb80aa8b0 0x7f8bb8063ac0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80098e0 tx=0x7f8ba80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e936b8faea7daae9 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:30.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.050+0000 7f8baffff700 1 -- 172.21.15.7:0/3089160073 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8011630 con 0x7f8bb80aa8b0 2024-01-27T02:38:30.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.050+0000 7f8baffff700 1 -- 172.21.15.7:0/3089160073 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba8011790 con 0x7f8bb80aa8b0 2024-01-27T02:38:30.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.050+0000 7f8baffff700 1 -- 172.21.15.7:0/3089160073 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8011b30 con 0x7f8bb80aa8b0 2024-01-27T02:38:30.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.051+0000 7f8bbf867700 1 -- 172.21.15.7:0/3089160073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bb80aa8b0 msgr2=0x7f8bb8063ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:30.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.051+0000 7f8bbf867700 1 --2- 172.21.15.7:0/3089160073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bb80aa8b0 0x7f8bb8063ac0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80098e0 tx=0x7f8ba80091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:30.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.051+0000 7f8bbf867700 1 -- 172.21.15.7:0/3089160073 shutdown_connections 2024-01-27T02:38:30.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.051+0000 7f8bbf867700 1 --2- 172.21.15.7:0/3089160073 >> v2:172.21.15.188:3300/0 conn(0x7f8bb8143670 0x7f8bb8063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:30.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.051+0000 7f8bbf867700 1 --2- 172.21.15.7:0/3089160073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bb80aa8b0 0x7f8bb8063ac0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:30.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.051+0000 7f8bbf867700 1 -- 172.21.15.7:0/3089160073 >> 172.21.15.7:0/3089160073 conn(0x7f8bb80cdbf0 msgr2=0x7f8bb80cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:30.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.051+0000 7f8bbf867700 1 -- 172.21.15.7:0/3089160073 shutdown_connections 2024-01-27T02:38:30.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.051+0000 7f8bbf867700 1 -- 172.21.15.7:0/3089160073 wait complete. 2024-01-27T02:38:30.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.052+0000 7f8bbf867700 1 Processor -- start 2024-01-27T02:38:30.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.052+0000 7f8bbf867700 1 -- start start 2024-01-27T02:38:30.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.053+0000 7f8bbf867700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bb80aa8b0 0x7f8bb80afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:30.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.053+0000 7f8bbf867700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bb8143670 0x7f8bb816b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:30.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.053+0000 7f8bbf867700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bb8143a40 0x7f8bb813f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:30.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.053+0000 7f8bbf867700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb80aeef0 con 0x7f8bb8143670 2024-01-27T02:38:30.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.053+0000 7f8bbf867700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb816ac60 con 0x7f8bb80aa8b0 2024-01-27T02:38:30.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.053+0000 7f8bbf867700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb816af40 con 0x7f8bb8143a40 2024-01-27T02:38:30.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.053+0000 7f8bbce02700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bb8143670 0x7f8bb816b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:30.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.053+0000 7f8bbce02700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bb8143670 0x7f8bb816b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52728/0 (socket says 172.21.15.7:52728) 2024-01-27T02:38:30.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.053+0000 7f8bbce02700 1 -- 172.21.15.7:0/2060593927 learned_addr learned my addr 172.21.15.7:0/2060593927 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:30.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.053+0000 7f8bbde04700 1 --2- 172.21.15.7:0/2060593927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bb8143a40 0x7f8bb813f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:30.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.053+0000 7f8bbd603700 1 --2- 172.21.15.7:0/2060593927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bb80aa8b0 0x7f8bb80afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:30.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.054+0000 7f8bbce02700 1 -- 172.21.15.7:0/2060593927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bb8143a40 msgr2=0x7f8bb813f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:30.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.054+0000 7f8bbce02700 1 --2- 172.21.15.7:0/2060593927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bb8143a40 0x7f8bb813f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:30.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.054+0000 7f8bbce02700 1 -- 172.21.15.7:0/2060593927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bb80aa8b0 msgr2=0x7f8bb80afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:30.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.054+0000 7f8bbce02700 1 --2- 172.21.15.7:0/2060593927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bb80aa8b0 0x7f8bb80afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:30.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.054+0000 7f8bbce02700 1 -- 172.21.15.7:0/2060593927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba8009420 con 0x7f8bb8143670 2024-01-27T02:38:30.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.054+0000 7f8bbde04700 1 --2- 172.21.15.7:0/2060593927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bb8143a40 0x7f8bb813f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:30.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.054+0000 7f8bbce02700 1 --2- 172.21.15.7:0/2060593927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bb8143670 0x7f8bb816b870 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80cbd80 tx=0x7f8ba8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:30.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.054+0000 7f8bad7fa700 1 -- 172.21.15.7:0/2060593927 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8011b10 con 0x7f8bb8143670 2024-01-27T02:38:30.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.054+0000 7f8bad7fa700 1 -- 172.21.15.7:0/2060593927 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba80253f0 con 0x7f8bb8143670 2024-01-27T02:38:30.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.054+0000 7f8bbf867700 1 -- 172.21.15.7:0/2060593927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb816b220 con 0x7f8bb8143670 2024-01-27T02:38:30.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.054+0000 7f8bad7fa700 1 -- 172.21.15.7:0/2060593927 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba80257b0 con 0x7f8bb8143670 2024-01-27T02:38:30.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.054+0000 7f8bbf867700 1 -- 172.21.15.7:0/2060593927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb80af6e0 con 0x7f8bb8143670 2024-01-27T02:38:30.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.055+0000 7f8bbf867700 1 -- 172.21.15.7:0/2060593927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb8072060 con 0x7f8bb8143670 2024-01-27T02:38:30.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.056+0000 7f8bad7fa700 1 -- 172.21.15.7:0/2060593927 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8ba8025ba0 con 0x7f8bb8143670 2024-01-27T02:38:30.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.056+0000 7f8bad7fa700 1 --2- 172.21.15.7:0/2060593927 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b9c066f30 0x7f8b9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:30.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.056+0000 7f8bad7fa700 1 -- 172.21.15.7:0/2060593927 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f8ba8085d10 con 0x7f8bb8143670 2024-01-27T02:38:30.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.057+0000 7f8bbd603700 1 --2- 172.21.15.7:0/2060593927 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b9c066f30 0x7f8b9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:30.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.057+0000 7f8bbd603700 1 --2- 172.21.15.7:0/2060593927 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b9c066f30 0x7f8b9c0693e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8bb815c450 tx=0x7f8bb400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:30.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.061+0000 7f8bad7fa700 1 -- 172.21.15.7:0/2060593927 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ba8053160 con 0x7f8bb8143670 2024-01-27T02:38:30.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.268+0000 7f8bbf867700 1 -- 172.21.15.7:0/2060593927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8bb80c8940 con 0x7f8bb8143670 2024-01-27T02:38:30.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.890+0000 7f8bad7fa700 1 -- 172.21.15.7:0/2060593927 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v130) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8ba8056780 con 0x7f8bb8143670 2024-01-27T02:38:30.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.895+0000 7f8bbf867700 1 -- 172.21.15.7:0/2060593927 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b9c066f30 msgr2=0x7f8b9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:30.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.895+0000 7f8bbf867700 1 --2- 172.21.15.7:0/2060593927 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b9c066f30 0x7f8b9c0693e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8bb815c450 tx=0x7f8bb400b040 comp rx=0 tx=0).stop 2024-01-27T02:38:30.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.895+0000 7f8bbf867700 1 -- 172.21.15.7:0/2060593927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bb8143670 msgr2=0x7f8bb816b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:30.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.895+0000 7f8bbf867700 1 --2- 172.21.15.7:0/2060593927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bb8143670 0x7f8bb816b870 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80cbd80 tx=0x7f8ba8017000 comp rx=0 tx=0).stop 2024-01-27T02:38:30.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.896+0000 7f8bbf867700 1 -- 172.21.15.7:0/2060593927 shutdown_connections 2024-01-27T02:38:30.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.896+0000 7f8bbf867700 1 --2- 172.21.15.7:0/2060593927 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b9c066f30 0x7f8b9c0693e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:30.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.896+0000 7f8bbf867700 1 --2- 172.21.15.7:0/2060593927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bb8143a40 0x7f8bb813f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:30.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.896+0000 7f8bbf867700 1 --2- 172.21.15.7:0/2060593927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bb80aa8b0 0x7f8bb80afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:30.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.896+0000 7f8bbf867700 1 --2- 172.21.15.7:0/2060593927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bb8143670 0x7f8bb816b870 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:30.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.896+0000 7f8bbf867700 1 -- 172.21.15.7:0/2060593927 >> 172.21.15.7:0/2060593927 conn(0x7f8bb80cdbf0 msgr2=0x7f8bb80ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:30.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.897+0000 7f8bbf867700 1 -- 172.21.15.7:0/2060593927 shutdown_connections 2024-01-27T02:38:30.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:30.900+0000 7f8bbf867700 1 -- 172.21.15.7:0/2060593927 wait complete. 2024-01-27T02:38:30.908 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-27T02:38:30.924 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-27T02:38:31.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.060+0000 7f79840b3700 1 Processor -- start 2024-01-27T02:38:31.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.061+0000 7f79840b3700 1 -- start start 2024-01-27T02:38:31.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.062+0000 7f79840b3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f797c0cba60 0x7f797c0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:31.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.062+0000 7f79840b3700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f797c06b4e0 0x7f797c06b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:31.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.062+0000 7f79840b3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f797c0679b0 con 0x7f797c0cbe30 2024-01-27T02:38:31.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.062+0000 7f79840b3700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f797c06bea0 con 0x7f797c06b4e0 2024-01-27T02:38:31.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.062+0000 7f79840b3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797c067c90 con 0x7f797c0cba60 2024-01-27T02:38:31.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.062+0000 7f798164e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f797c0cbe30 0x7f797c15b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49308/0 (socket says 172.21.15.7:49308) 2024-01-27T02:38:31.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.062+0000 7f798164e700 1 -- 172.21.15.7:0/115347368 learned_addr learned my addr 172.21.15.7:0/115347368 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:31.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.062+0000 7f7982650700 1 --2- 172.21.15.7:0/115347368 >> v2:172.21.15.188:3300/0 conn(0x7f797c06b4e0 0x7f797c06b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:31.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.062+0000 7f7981e4f700 1 --2- 172.21.15.7:0/115347368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f797c0cba60 0x7f797c0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:31.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.062+0000 7f7980e4d700 1 -- 172.21.15.7:0/115347368 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762153065 0 0) 0x7f797c0679b0 con 0x7f797c0cbe30 2024-01-27T02:38:31.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.062+0000 7f7980e4d700 1 -- 172.21.15.7:0/115347368 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7968003570 con 0x7f797c0cbe30 2024-01-27T02:38:31.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.062+0000 7f7980e4d700 1 -- 172.21.15.7:0/115347368 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2506756077 0 0) 0x7f7968003570 con 0x7f797c0cbe30 2024-01-27T02:38:31.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.062+0000 7f7980e4d700 1 -- 172.21.15.7:0/115347368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f797c0cba60 msgr2=0x7f797c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:31.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.062+0000 7f7980e4d700 1 --2- 172.21.15.7:0/115347368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f797c0cba60 0x7f797c0ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:31.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.062+0000 7f7980e4d700 1 -- 172.21.15.7:0/115347368 >> v2:172.21.15.188:3300/0 conn(0x7f797c06b4e0 msgr2=0x7f797c06b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:31.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.062+0000 7f7980e4d700 1 --2- 172.21.15.7:0/115347368 >> v2:172.21.15.188:3300/0 conn(0x7f797c06b4e0 0x7f797c06b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:31.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.063+0000 7f7980e4d700 1 -- 172.21.15.7:0/115347368 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797c0cca50 con 0x7f797c0cbe30 2024-01-27T02:38:31.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.063+0000 7f7980e4d700 1 -- 172.21.15.7:0/115347368 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f7970001470 con 0x7f797c0cbe30 2024-01-27T02:38:31.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.063+0000 7f7980e4d700 1 -- 172.21.15.7:0/115347368 >> v1:172.21.15.7:6789/0 conn(0x7f797c0cbe30 legacy=0x7f797c15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:31.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.063+0000 7f7980e4d700 1 --2- 172.21.15.7:0/115347368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7968003a60 0x7f7968003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:31.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.063+0000 7f7980e4d700 1 --2- 172.21.15.7:0/115347368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7968004810 0x7f7968004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:31.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.063+0000 7f7980e4d700 1 --2- 172.21.15.7:0/115347368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f797c0cba60 0x7f79680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:31.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.063+0000 7f7980e4d700 1 -- 172.21.15.7:0/115347368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c06bea0 con 0x7f7968004810 2024-01-27T02:38:31.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.063+0000 7f7980e4d700 1 -- 172.21.15.7:0/115347368 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c067c90 con 0x7f7968003a60 2024-01-27T02:38:31.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.063+0000 7f7980e4d700 1 -- 172.21.15.7:0/115347368 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796800ba60 con 0x7f797c0cba60 2024-01-27T02:38:31.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.063+0000 7f798164e700 1 --2- 172.21.15.7:0/115347368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7968004810 0x7f7968004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:31.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.063+0000 7f7981e4f700 1 --2- 172.21.15.7:0/115347368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7968003a60 0x7f7968003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:31.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.063+0000 7f7982650700 1 --2- 172.21.15.7:0/115347368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f797c0cba60 0x7f79680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:31.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.063+0000 7f798164e700 1 -- 172.21.15.7:0/115347368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f797c0cba60 msgr2=0x7f79680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:31.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.063+0000 7f798164e700 1 --2- 172.21.15.7:0/115347368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f797c0cba60 0x7f79680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:31.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.063+0000 7f798164e700 1 -- 172.21.15.7:0/115347368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7968003a60 msgr2=0x7f7968003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:31.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.063+0000 7f798164e700 1 --2- 172.21.15.7:0/115347368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7968003a60 0x7f7968003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:31.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.063+0000 7f798164e700 1 -- 172.21.15.7:0/115347368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7968003570 con 0x7f7968004810 2024-01-27T02:38:31.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.063+0000 7f798164e700 1 --2- 172.21.15.7:0/115347368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7968004810 0x7f7968004c80 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f797000c3c0 tx=0x7f7970001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b698f9bd1016966 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:31.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.064+0000 7f7980e4d700 1 -- 172.21.15.7:0/115347368 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797000cc50 con 0x7f7968004810 2024-01-27T02:38:31.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.064+0000 7f7980e4d700 1 -- 172.21.15.7:0/115347368 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797000cdd0 con 0x7f7968004810 2024-01-27T02:38:31.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.064+0000 7f7980e4d700 1 -- 172.21.15.7:0/115347368 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970005b70 con 0x7f7968004810 2024-01-27T02:38:31.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.064+0000 7f79840b3700 1 -- 172.21.15.7:0/115347368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7968004810 msgr2=0x7f7968004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:31.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.064+0000 7f79840b3700 1 --2- 172.21.15.7:0/115347368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7968004810 0x7f7968004c80 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f797000c3c0 tx=0x7f7970001e90 comp rx=0 tx=0).stop 2024-01-27T02:38:31.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.064+0000 7f7982650700 1 -- 172.21.15.7:0/115347368 reap_dead start 2024-01-27T02:38:31.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.064+0000 7f79840b3700 1 -- 172.21.15.7:0/115347368 shutdown_connections 2024-01-27T02:38:31.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.064+0000 7f79840b3700 1 -- 172.21.15.7:0/115347368 >> 172.21.15.7:0/115347368 conn(0x7f797c063580 msgr2=0x7f797c063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:31.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.064+0000 7f79840b3700 1 -- 172.21.15.7:0/115347368 shutdown_connections 2024-01-27T02:38:31.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.064+0000 7f79840b3700 1 -- 172.21.15.7:0/115347368 wait complete. 2024-01-27T02:38:31.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.065+0000 7f79840b3700 1 Processor -- start 2024-01-27T02:38:31.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.066+0000 7f79840b3700 1 -- start start 2024-01-27T02:38:31.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.066+0000 7f79840b3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7968004810 0x7f797c16dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:31.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.066+0000 7f79840b3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f797c16b040 0x7f797c15cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:31.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.066+0000 7f79840b3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f797c145360 0x7f797c145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:31.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.066+0000 7f79840b3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c145d40 con 0x7f797c145360 2024-01-27T02:38:31.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.066+0000 7f79840b3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c156400 con 0x7f797c16b040 2024-01-27T02:38:31.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.066+0000 7f79840b3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797c0b2f40 con 0x7f7968004810 2024-01-27T02:38:31.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.066+0000 7f7982650700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f797c145360 0x7f797c145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:31.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.066+0000 7f7982650700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f797c145360 0x7f797c145800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52742/0 (socket says 172.21.15.7:52742) 2024-01-27T02:38:31.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.066+0000 7f7982650700 1 -- 172.21.15.7:0/1253087977 learned_addr learned my addr 172.21.15.7:0/1253087977 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:31.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.067+0000 7f798164e700 1 --2- 172.21.15.7:0/1253087977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f797c16b040 0x7f797c15cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:31.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.067+0000 7f7981e4f700 1 --2- 172.21.15.7:0/1253087977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7968004810 0x7f797c16dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:31.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.067+0000 7f7982650700 1 -- 172.21.15.7:0/1253087977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7968004810 msgr2=0x7f797c16dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:31.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.067+0000 7f7982650700 1 --2- 172.21.15.7:0/1253087977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7968004810 0x7f797c16dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:31.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.067+0000 7f7982650700 1 -- 172.21.15.7:0/1253087977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f797c16b040 msgr2=0x7f797c15cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:31.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.067+0000 7f7982650700 1 --2- 172.21.15.7:0/1253087977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f797c16b040 0x7f797c15cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:31.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.067+0000 7f7982650700 1 -- 172.21.15.7:0/1253087977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797c16b4b0 con 0x7f797c145360 2024-01-27T02:38:31.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.067+0000 7f7982650700 1 --2- 172.21.15.7:0/1253087977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f797c145360 0x7f797c145800 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f797c0b1800 tx=0x7f797400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:31.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.067+0000 7f796dffb700 1 -- 172.21.15.7:0/1253087977 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797400e040 con 0x7f797c145360 2024-01-27T02:38:31.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.067+0000 7f796dffb700 1 -- 172.21.15.7:0/1253087977 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797400f040 con 0x7f797c145360 2024-01-27T02:38:31.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.067+0000 7f79840b3700 1 -- 172.21.15.7:0/1253087977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f797c16e1d0 con 0x7f797c145360 2024-01-27T02:38:31.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.067+0000 7f796dffb700 1 -- 172.21.15.7:0/1253087977 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797400fd10 con 0x7f797c145360 2024-01-27T02:38:31.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.068+0000 7f79840b3700 1 -- 172.21.15.7:0/1253087977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797c16e790 con 0x7f797c145360 2024-01-27T02:38:31.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.069+0000 7f79840b3700 1 -- 172.21.15.7:0/1253087977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f797c0b30b0 con 0x7f797c145360 2024-01-27T02:38:31.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.069+0000 7f796dffb700 1 -- 172.21.15.7:0/1253087977 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7974015640 con 0x7f797c145360 2024-01-27T02:38:31.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.070+0000 7f796dffb700 1 --2- 172.21.15.7:0/1253087977 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7960066f30 0x7f79600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:31.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.070+0000 7f796dffb700 1 -- 172.21.15.7:0/1253087977 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f7974084c30 con 0x7f797c145360 2024-01-27T02:38:31.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.070+0000 7f7981e4f700 1 --2- 172.21.15.7:0/1253087977 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7960066f30 0x7f79600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:31.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.071+0000 7f7981e4f700 1 --2- 172.21.15.7:0/1253087977 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7960066f30 0x7f79600693e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7978006ea0 tx=0x7f7978009120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:31.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.074+0000 7f796dffb700 1 -- 172.21.15.7:0/1253087977 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7974051f00 con 0x7f797c145360 2024-01-27T02:38:31.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.326+0000 7f79840b3700 1 -- 172.21.15.7:0/1253087977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f797c072060 con 0x7f797c145360 2024-01-27T02:38:31.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.925+0000 7f796dffb700 1 -- 172.21.15.7:0/1253087977 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v64) v1 ==== 174+0+0 (secure 0 0 0) 0x7f7974055520 con 0x7f797c145360 2024-01-27T02:38:31.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.931+0000 7f79840b3700 1 -- 172.21.15.7:0/1253087977 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7960066f30 msgr2=0x7f79600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:31.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.931+0000 7f79840b3700 1 --2- 172.21.15.7:0/1253087977 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7960066f30 0x7f79600693e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7978006ea0 tx=0x7f7978009120 comp rx=0 tx=0).stop 2024-01-27T02:38:31.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.931+0000 7f79840b3700 1 -- 172.21.15.7:0/1253087977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f797c145360 msgr2=0x7f797c145800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:31.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.931+0000 7f79840b3700 1 --2- 172.21.15.7:0/1253087977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f797c145360 0x7f797c145800 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f797c0b1800 tx=0x7f797400b040 comp rx=0 tx=0).stop 2024-01-27T02:38:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.932+0000 7f79840b3700 1 -- 172.21.15.7:0/1253087977 shutdown_connections 2024-01-27T02:38:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.932+0000 7f79840b3700 1 --2- 172.21.15.7:0/1253087977 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7960066f30 0x7f79600693e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.932+0000 7f79840b3700 1 --2- 172.21.15.7:0/1253087977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f797c145360 0x7f797c145800 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.932+0000 7f79840b3700 1 --2- 172.21.15.7:0/1253087977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7968004810 0x7f797c16dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.932+0000 7f79840b3700 1 --2- 172.21.15.7:0/1253087977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f797c16b040 0x7f797c15cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.932+0000 7f79840b3700 1 -- 172.21.15.7:0/1253087977 >> 172.21.15.7:0/1253087977 conn(0x7f797c063580 msgr2=0x7f797c068020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.934+0000 7f79840b3700 1 -- 172.21.15.7:0/1253087977 shutdown_connections 2024-01-27T02:38:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:31.934+0000 7f79840b3700 1 -- 172.21.15.7:0/1253087977 wait complete. 2024-01-27T02:38:31.938 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 16 and data pool 17 2024-01-27T02:38:31.953 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:38:32.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.087+0000 7f1232c07700 1 Processor -- start 2024-01-27T02:38:32.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.087+0000 7f1232c07700 1 -- start start 2024-01-27T02:38:32.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.088+0000 7f1232c07700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f122c0c89a0 0x7f122c06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:32.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.088+0000 7f1232c07700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f122c0c8d70 0x7f122c063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:32.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.088+0000 7f1232c07700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f122c063ac0 0x7f122c15a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:32.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.088+0000 7f1232c07700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f122c063f40 con 0x7f122c063ac0 2024-01-27T02:38:32.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.088+0000 7f1232c07700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f122c0c4450 con 0x7f122c0c8d70 2024-01-27T02:38:32.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.088+0000 7f1232c07700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122c06e9e0 con 0x7f122c0c89a0 2024-01-27T02:38:32.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.088+0000 7f12311a4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f122c063ac0 0x7f122c15a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:32.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.088+0000 7f12311a4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f122c063ac0 0x7f122c15a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58738/0 (socket says 172.21.15.7:58738) 2024-01-27T02:38:32.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.088+0000 7f12311a4700 1 -- 172.21.15.7:0/760006673 learned_addr learned my addr 172.21.15.7:0/760006673 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:32.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.088+0000 7f12309a3700 1 --2- 172.21.15.7:0/760006673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f122c0c89a0 0x7f122c06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:32.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.088+0000 7f122bfff700 1 --2- 172.21.15.7:0/760006673 >> v2:172.21.15.188:3300/0 conn(0x7f122c0c8d70 0x7f122c063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:32.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.089+0000 7f12311a4700 1 -- 172.21.15.7:0/760006673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f122c0c89a0 msgr2=0x7f122c06ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:32.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.089+0000 7f12311a4700 1 --2- 172.21.15.7:0/760006673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f122c0c89a0 0x7f122c06ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.089+0000 7f12311a4700 1 -- 172.21.15.7:0/760006673 >> v2:172.21.15.188:3300/0 conn(0x7f122c0c8d70 msgr2=0x7f122c063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:32.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.089+0000 7f12311a4700 1 --2- 172.21.15.7:0/760006673 >> v2:172.21.15.188:3300/0 conn(0x7f122c0c8d70 0x7f122c063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.089+0000 7f12311a4700 1 -- 172.21.15.7:0/760006673 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122c06e3d0 con 0x7f122c063ac0 2024-01-27T02:38:32.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.089+0000 7f12311a4700 1 --2- 172.21.15.7:0/760006673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f122c063ac0 0x7f122c15a710 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f122c0ced40 tx=0x7f1224009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3407d1a08a887290 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:32.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.089+0000 7f122b7fe700 1 -- 172.21.15.7:0/760006673 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122400e070 con 0x7f122c063ac0 2024-01-27T02:38:32.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.089+0000 7f122b7fe700 1 -- 172.21.15.7:0/760006673 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122400ba40 con 0x7f122c063ac0 2024-01-27T02:38:32.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.089+0000 7f122b7fe700 1 -- 172.21.15.7:0/760006673 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12240092e0 con 0x7f122c063ac0 2024-01-27T02:38:32.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.089+0000 7f1232c07700 1 -- 172.21.15.7:0/760006673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f122c063ac0 msgr2=0x7f122c15a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:32.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.089+0000 7f1232c07700 1 --2- 172.21.15.7:0/760006673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f122c063ac0 0x7f122c15a710 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f122c0ced40 tx=0x7f1224009840 comp rx=0 tx=0).stop 2024-01-27T02:38:32.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.090+0000 7f1232c07700 1 -- 172.21.15.7:0/760006673 shutdown_connections 2024-01-27T02:38:32.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.090+0000 7f1232c07700 1 --2- 172.21.15.7:0/760006673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f122c063ac0 0x7f122c15a710 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.090+0000 7f1232c07700 1 --2- 172.21.15.7:0/760006673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f122c0c89a0 0x7f122c06ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.090+0000 7f1232c07700 1 --2- 172.21.15.7:0/760006673 >> v2:172.21.15.188:3300/0 conn(0x7f122c0c8d70 0x7f122c063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.090+0000 7f1232c07700 1 -- 172.21.15.7:0/760006673 >> 172.21.15.7:0/760006673 conn(0x7f122c0cdc30 msgr2=0x7f122c0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:32.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.090+0000 7f1232c07700 1 -- 172.21.15.7:0/760006673 shutdown_connections 2024-01-27T02:38:32.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.090+0000 7f1232c07700 1 -- 172.21.15.7:0/760006673 wait complete. 2024-01-27T02:38:32.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.091+0000 7f1232c07700 1 Processor -- start 2024-01-27T02:38:32.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.091+0000 7f1232c07700 1 -- start start 2024-01-27T02:38:32.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.092+0000 7f1232c07700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f122c063ac0 0x7f122c16b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:32.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.092+0000 7f1232c07700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f122c0c89a0 0x7f122c157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:32.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.092+0000 7f1232c07700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f122c0c8d70 0x7f122c06a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:32.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.092+0000 7f1232c07700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c0adf60 con 0x7f122c0c89a0 2024-01-27T02:38:32.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.092+0000 7f1232c07700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c06a840 con 0x7f122c063ac0 2024-01-27T02:38:32.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.092+0000 7f1232c07700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122c06b360 con 0x7f122c0c8d70 2024-01-27T02:38:32.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.092+0000 7f122bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f122c0c89a0 0x7f122c157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:32.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.092+0000 7f122bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f122c0c89a0 0x7f122c157760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58752/0 (socket says 172.21.15.7:58752) 2024-01-27T02:38:32.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.092+0000 7f122bfff700 1 -- 172.21.15.7:0/2576274015 learned_addr learned my addr 172.21.15.7:0/2576274015 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:32.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.092+0000 7f12311a4700 1 --2- 172.21.15.7:0/2576274015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f122c0c8d70 0x7f122c06a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:32.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.092+0000 7f12309a3700 1 --2- 172.21.15.7:0/2576274015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f122c063ac0 0x7f122c16b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:32.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.092+0000 7f122bfff700 1 -- 172.21.15.7:0/2576274015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f122c0c8d70 msgr2=0x7f122c06a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:32.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.092+0000 7f122bfff700 1 --2- 172.21.15.7:0/2576274015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f122c0c8d70 0x7f122c06a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.092+0000 7f122bfff700 1 -- 172.21.15.7:0/2576274015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f122c063ac0 msgr2=0x7f122c16b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:32.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.092+0000 7f122bfff700 1 --2- 172.21.15.7:0/2576274015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f122c063ac0 0x7f122c16b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.092+0000 7f122bfff700 1 -- 172.21.15.7:0/2576274015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1224009ae0 con 0x7f122c0c89a0 2024-01-27T02:38:32.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.093+0000 7f122bfff700 1 --2- 172.21.15.7:0/2576274015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f122c0c89a0 0x7f122c157760 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f121800d890 tx=0x7f1218008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:32.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.093+0000 7f1228ff9700 1 -- 172.21.15.7:0/2576274015 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218018070 con 0x7f122c0c89a0 2024-01-27T02:38:32.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.093+0000 7f1228ff9700 1 -- 172.21.15.7:0/2576274015 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12180125f0 con 0x7f122c0c89a0 2024-01-27T02:38:32.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.093+0000 7f1232c07700 1 -- 172.21.15.7:0/2576274015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122c06aac0 con 0x7f122c0c89a0 2024-01-27T02:38:32.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.093+0000 7f1228ff9700 1 -- 172.21.15.7:0/2576274015 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218012990 con 0x7f122c0c89a0 2024-01-27T02:38:32.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.093+0000 7f1232c07700 1 -- 172.21.15.7:0/2576274015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122c06b080 con 0x7f122c0c89a0 2024-01-27T02:38:32.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.094+0000 7f1232c07700 1 -- 172.21.15.7:0/2576274015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f122c0add30 con 0x7f122c0c89a0 2024-01-27T02:38:32.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.094+0000 7f1228ff9700 1 -- 172.21.15.7:0/2576274015 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1218012d40 con 0x7f122c0c89a0 2024-01-27T02:38:32.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.095+0000 7f1228ff9700 1 --2- 172.21.15.7:0/2576274015 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1210066f30 0x7f12100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:32.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.095+0000 7f1228ff9700 1 -- 172.21.15.7:0/2576274015 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f12180846f0 con 0x7f122c0c89a0 2024-01-27T02:38:32.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.095+0000 7f12309a3700 1 --2- 172.21.15.7:0/2576274015 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1210066f30 0x7f12100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:32.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.096+0000 7f12309a3700 1 --2- 172.21.15.7:0/2576274015 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1210066f30 0x7f12100693e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f122c0cda00 tx=0x7f1220008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:32.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.099+0000 7f1228ff9700 1 -- 172.21.15.7:0/2576274015 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f122c0add30 con 0x7f122c0c89a0 2024-01-27T02:38:32.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.301+0000 7f1232c07700 1 -- 172.21.15.7:0/2576274015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f122c0add30 con 0x7f122c0c89a0 2024-01-27T02:38:32.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.302+0000 7f1228ff9700 1 -- 172.21.15.7:0/2576274015 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+15330 (secure 0 0 0) 0x7f122c0add30 con 0x7f122c0c89a0 2024-01-27T02:38:32.303 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:32.303 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":131,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:38:31.327949+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:38:29.824892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-27T02:38:30.301072+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:38:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.305+0000 7f1232c07700 1 -- 172.21.15.7:0/2576274015 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1210066f30 msgr2=0x7f12100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.305+0000 7f1232c07700 1 --2- 172.21.15.7:0/2576274015 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1210066f30 0x7f12100693e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f122c0cda00 tx=0x7f1220008040 comp rx=0 tx=0).stop 2024-01-27T02:38:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.305+0000 7f1232c07700 1 -- 172.21.15.7:0/2576274015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f122c0c89a0 msgr2=0x7f122c157760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.305+0000 7f1232c07700 1 --2- 172.21.15.7:0/2576274015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f122c0c89a0 0x7f122c157760 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f121800d890 tx=0x7f1218008040 comp rx=0 tx=0).stop 2024-01-27T02:38:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.305+0000 7f1232c07700 1 -- 172.21.15.7:0/2576274015 shutdown_connections 2024-01-27T02:38:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.305+0000 7f1232c07700 1 --2- 172.21.15.7:0/2576274015 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1210066f30 0x7f12100693e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.305+0000 7f1232c07700 1 --2- 172.21.15.7:0/2576274015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f122c0c8d70 0x7f122c06a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.305+0000 7f1232c07700 1 --2- 172.21.15.7:0/2576274015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f122c063ac0 0x7f122c16b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.305+0000 7f1232c07700 1 --2- 172.21.15.7:0/2576274015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f122c0c89a0 0x7f122c157760 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.305+0000 7f1232c07700 1 -- 172.21.15.7:0/2576274015 >> 172.21.15.7:0/2576274015 conn(0x7f122c0cdc30 msgr2=0x7f122c15b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.305+0000 7f1232c07700 1 -- 172.21.15.7:0/2576274015 shutdown_connections 2024-01-27T02:38:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.305+0000 7f1232c07700 1 -- 172.21.15.7:0/2576274015 wait complete. 2024-01-27T02:38:32.319 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:38:32.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.451+0000 7f324a8d6700 1 Processor -- start 2024-01-27T02:38:32.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.451+0000 7f324a8d6700 1 -- start start 2024-01-27T02:38:32.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.452+0000 7f324a8d6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3244157ca0 0x7f32440c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:32.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.452+0000 7f324a8d6700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f32440c3ae0 0x7f32440cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:32.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.452+0000 7f324a8d6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32440ca020 con 0x7f3244158070 2024-01-27T02:38:32.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.452+0000 7f324a8d6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32440c4f40 con 0x7f3244157ca0 2024-01-27T02:38:32.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.452+0000 7f324a8d6700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32440c5210 con 0x7f32440c3ae0 2024-01-27T02:38:32.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.452+0000 7f3243fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3244157ca0 0x7f32440c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:32.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.452+0000 7f3248e73700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3244158070 0x7f324415bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50590/0 (socket says 172.21.15.7:50590) 2024-01-27T02:38:32.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.452+0000 7f3248e73700 1 -- 172.21.15.7:0/1279472296 learned_addr learned my addr 172.21.15.7:0/1279472296 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:32.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.453+0000 7f32437fe700 1 --2- 172.21.15.7:0/1279472296 >> v2:172.21.15.188:3300/0 conn(0x7f32440c3ae0 0x7f32440cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:32.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.453+0000 7f3243fff700 1 -- 172.21.15.7:0/1279472296 >> v2:172.21.15.188:3300/0 conn(0x7f32440c3ae0 msgr2=0x7f32440cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:32.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.453+0000 7f3243fff700 1 --2- 172.21.15.7:0/1279472296 >> v2:172.21.15.188:3300/0 conn(0x7f32440c3ae0 0x7f32440cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.453+0000 7f3243fff700 1 -- 172.21.15.7:0/1279472296 >> v1:172.21.15.7:6789/0 conn(0x7f3244158070 legacy=0x7f324415bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:32.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.453+0000 7f3243fff700 1 -- 172.21.15.7:0/1279472296 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32440c54f0 con 0x7f3244157ca0 2024-01-27T02:38:32.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.453+0000 7f3242ffd700 1 -- 172.21.15.7:0/1279472296 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1608600233 0 0) 0x7f32440ca020 con 0x7f3244158070 2024-01-27T02:38:32.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.453+0000 7f3243fff700 1 --2- 172.21.15.7:0/1279472296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3244157ca0 0x7f32440c4a00 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f324406e170 tx=0x7f32340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec52c57aaafe898 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:32.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.453+0000 7f3242ffd700 1 -- 172.21.15.7:0/1279472296 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323400b6b0 con 0x7f3244157ca0 2024-01-27T02:38:32.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.453+0000 7f3242ffd700 1 -- 172.21.15.7:0/1279472296 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323400b810 con 0x7f3244157ca0 2024-01-27T02:38:32.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.453+0000 7f3242ffd700 1 -- 172.21.15.7:0/1279472296 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323400bbb0 con 0x7f3244157ca0 2024-01-27T02:38:32.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.454+0000 7f324a8d6700 1 -- 172.21.15.7:0/1279472296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3244157ca0 msgr2=0x7f32440c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:32.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.454+0000 7f324a8d6700 1 --2- 172.21.15.7:0/1279472296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3244157ca0 0x7f32440c4a00 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f324406e170 tx=0x7f32340091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.454+0000 7f324a8d6700 1 -- 172.21.15.7:0/1279472296 shutdown_connections 2024-01-27T02:38:32.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.454+0000 7f324a8d6700 1 --2- 172.21.15.7:0/1279472296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3244157ca0 0x7f32440c4a00 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.454+0000 7f324a8d6700 1 --2- 172.21.15.7:0/1279472296 >> v2:172.21.15.188:3300/0 conn(0x7f32440c3ae0 0x7f32440cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.454+0000 7f324a8d6700 1 -- 172.21.15.7:0/1279472296 >> 172.21.15.7:0/1279472296 conn(0x7f3244098020 msgr2=0x7f32440711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:32.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.454+0000 7f324a8d6700 1 -- 172.21.15.7:0/1279472296 shutdown_connections 2024-01-27T02:38:32.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.454+0000 7f324a8d6700 1 -- 172.21.15.7:0/1279472296 wait complete. 2024-01-27T02:38:32.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.455+0000 7f324a8d6700 1 Processor -- start 2024-01-27T02:38:32.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.455+0000 7f324a8d6700 1 -- start start 2024-01-27T02:38:32.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.456+0000 7f324a8d6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32440c3ae0 0x7f32441457c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:32.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.456+0000 7f324a8d6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3244157ca0 0x7f3244155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:32.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.456+0000 7f324a8d6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3244158070 0x7f32441560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:32.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.456+0000 7f324a8d6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32440c77b0 con 0x7f32440c3ae0 2024-01-27T02:38:32.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.456+0000 7f324a8d6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32440c6ba0 con 0x7f3244158070 2024-01-27T02:38:32.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.456+0000 7f324a8d6700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32440c6e80 con 0x7f3244157ca0 2024-01-27T02:38:32.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.456+0000 7f3243fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32440c3ae0 0x7f32441457c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:32.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.456+0000 7f3243fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32440c3ae0 0x7f32441457c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58762/0 (socket says 172.21.15.7:58762) 2024-01-27T02:38:32.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.456+0000 7f3243fff700 1 -- 172.21.15.7:0/412065517 learned_addr learned my addr 172.21.15.7:0/412065517 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:32.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.456+0000 7f32437fe700 1 --2- 172.21.15.7:0/412065517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3244157ca0 0x7f3244155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:32.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.456+0000 7f3248e73700 1 --2- 172.21.15.7:0/412065517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3244158070 0x7f32441560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:32.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.456+0000 7f3243fff700 1 -- 172.21.15.7:0/412065517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3244157ca0 msgr2=0x7f3244155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:32.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.456+0000 7f3243fff700 1 --2- 172.21.15.7:0/412065517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3244157ca0 0x7f3244155b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.456+0000 7f3243fff700 1 -- 172.21.15.7:0/412065517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3244158070 msgr2=0x7f32441560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:32.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.456+0000 7f3243fff700 1 --2- 172.21.15.7:0/412065517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3244158070 0x7f32441560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.456+0000 7f3243fff700 1 -- 172.21.15.7:0/412065517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3234009420 con 0x7f32440c3ae0 2024-01-27T02:38:32.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.456+0000 7f32437fe700 1 --2- 172.21.15.7:0/412065517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3244157ca0 0x7f3244155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:38:32.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.456+0000 7f3248e73700 1 --2- 172.21.15.7:0/412065517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3244158070 0x7f32441560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:38:32.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.457+0000 7f3243fff700 1 --2- 172.21.15.7:0/412065517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32440c3ae0 0x7f32441457c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f324406ea80 tx=0x7f3234017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:32.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.457+0000 7f322bfff700 1 -- 172.21.15.7:0/412065517 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3234011420 con 0x7f32440c3ae0 2024-01-27T02:38:32.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.457+0000 7f322bfff700 1 -- 172.21.15.7:0/412065517 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3234011580 con 0x7f32440c3ae0 2024-01-27T02:38:32.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.457+0000 7f324a8d6700 1 -- 172.21.15.7:0/412065517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32440c7160 con 0x7f32440c3ae0 2024-01-27T02:38:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.457+0000 7f322bfff700 1 -- 172.21.15.7:0/412065517 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3234011940 con 0x7f32440c3ae0 2024-01-27T02:38:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.457+0000 7f324a8d6700 1 -- 172.21.15.7:0/412065517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3244145d00 con 0x7f32440c3ae0 2024-01-27T02:38:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.458+0000 7f324a8d6700 1 -- 172.21.15.7:0/412065517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f324404c710 con 0x7f32440c3ae0 2024-01-27T02:38:32.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.458+0000 7f322bfff700 1 -- 172.21.15.7:0/412065517 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3234011d30 con 0x7f32440c3ae0 2024-01-27T02:38:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.459+0000 7f322bfff700 1 --2- 172.21.15.7:0/412065517 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3230066f30 0x7f32300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.459+0000 7f322bfff700 1 -- 172.21.15.7:0/412065517 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3234085a60 con 0x7f32440c3ae0 2024-01-27T02:38:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.459+0000 7f32437fe700 1 --2- 172.21.15.7:0/412065517 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3230066f30 0x7f32300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.460+0000 7f32437fe700 1 --2- 172.21.15.7:0/412065517 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3230066f30 0x7f32300693e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3244099160 tx=0x7f3238008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.463+0000 7f322bfff700 1 -- 172.21.15.7:0/412065517 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3234052cd0 con 0x7f32440c3ae0 2024-01-27T02:38:32.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.667+0000 7f324a8d6700 1 -- 172.21.15.7:0/412065517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32440c57d0 con 0x7f32440c3ae0 2024-01-27T02:38:32.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.668+0000 7f322bfff700 1 -- 172.21.15.7:0/412065517 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+15330 (secure 0 0 0) 0x7f3234015020 con 0x7f32440c3ae0 2024-01-27T02:38:32.669 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:32.669 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":131,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:38:31.327949+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:38:29.824892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-27T02:38:30.301072+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:38:32.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.670+0000 7f324a8d6700 1 -- 172.21.15.7:0/412065517 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3230066f30 msgr2=0x7f32300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:32.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.670+0000 7f324a8d6700 1 --2- 172.21.15.7:0/412065517 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3230066f30 0x7f32300693e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3244099160 tx=0x7f3238008040 comp rx=0 tx=0).stop 2024-01-27T02:38:32.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.670+0000 7f324a8d6700 1 -- 172.21.15.7:0/412065517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32440c3ae0 msgr2=0x7f32441457c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:32.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.670+0000 7f324a8d6700 1 --2- 172.21.15.7:0/412065517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32440c3ae0 0x7f32441457c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f324406ea80 tx=0x7f3234017000 comp rx=0 tx=0).stop 2024-01-27T02:38:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.670+0000 7f324a8d6700 1 -- 172.21.15.7:0/412065517 shutdown_connections 2024-01-27T02:38:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.670+0000 7f324a8d6700 1 --2- 172.21.15.7:0/412065517 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3230066f30 0x7f32300693e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.670+0000 7f324a8d6700 1 --2- 172.21.15.7:0/412065517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3244158070 0x7f32441560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.670+0000 7f324a8d6700 1 --2- 172.21.15.7:0/412065517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32440c3ae0 0x7f32441457c0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.670+0000 7f324a8d6700 1 --2- 172.21.15.7:0/412065517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3244157ca0 0x7f3244155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.671+0000 7f324a8d6700 1 -- 172.21.15.7:0/412065517 >> 172.21.15.7:0/412065517 conn(0x7f3244098020 msgr2=0x7f32440ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.671+0000 7f324a8d6700 1 -- 172.21.15.7:0/412065517 shutdown_connections 2024-01-27T02:38:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.671+0000 7f324a8d6700 1 -- 172.21.15.7:0/412065517 wait complete. 2024-01-27T02:38:32.685 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-27T02:38:32.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.820+0000 7fc185a44700 1 Processor -- start 2024-01-27T02:38:32.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.820+0000 7fc185a44700 1 -- start start 2024-01-27T02:38:32.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.821+0000 7fc185a44700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc18015ea60 0x7fc1800cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:32.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.821+0000 7fc185a44700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc180159c00 0x7fc18015a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:32.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.821+0000 7fc185a44700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1800cfb40 con 0x7fc1800cc110 2024-01-27T02:38:32.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.821+0000 7fc185a44700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1800cfe20 con 0x7fc180159c00 2024-01-27T02:38:32.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.821+0000 7fc185a44700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1800d0100 con 0x7fc18015ea60 2024-01-27T02:38:32.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.821+0000 7fc17f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc180159c00 0x7fc18015a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:32.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.821+0000 7fc17f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc180159c00 0x7fc18015a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58772/0 (socket says 172.21.15.7:58772) 2024-01-27T02:38:32.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.821+0000 7fc17f7fe700 1 -- 172.21.15.7:0/890252178 learned_addr learned my addr 172.21.15.7:0/890252178 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:32.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.821+0000 7fc17e7fc700 1 --2- 172.21.15.7:0/890252178 >> v2:172.21.15.188:3300/0 conn(0x7fc18015ea60 0x7fc1800cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:32.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.821+0000 7fc17f7fe700 1 -- 172.21.15.7:0/890252178 >> v2:172.21.15.188:3300/0 conn(0x7fc18015ea60 msgr2=0x7fc1800cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:32.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.821+0000 7fc17f7fe700 1 --2- 172.21.15.7:0/890252178 >> v2:172.21.15.188:3300/0 conn(0x7fc18015ea60 0x7fc1800cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.821+0000 7fc17f7fe700 1 -- 172.21.15.7:0/890252178 >> v1:172.21.15.188:6789/0 conn(0x7fc1800cc110 legacy=0x7fc1800cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:32.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.821+0000 7fc17f7fe700 1 -- 172.21.15.7:0/890252178 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1800d03e0 con 0x7fc180159c00 2024-01-27T02:38:32.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.821+0000 7fc17e7fc700 1 --2- 172.21.15.7:0/890252178 >> v2:172.21.15.188:3300/0 conn(0x7fc18015ea60 0x7fc1800cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:32.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.822+0000 7fc17f7fe700 1 --2- 172.21.15.7:0/890252178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc180159c00 0x7fc18015a0a0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc1800cca20 tx=0x7fc178009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cac6ef626a1acd3 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:32.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.822+0000 7fc17dffb700 1 -- 172.21.15.7:0/890252178 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17800e070 con 0x7fc180159c00 2024-01-27T02:38:32.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.822+0000 7fc17dffb700 1 -- 172.21.15.7:0/890252178 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17800ba50 con 0x7fc180159c00 2024-01-27T02:38:32.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.822+0000 7fc17dffb700 1 -- 172.21.15.7:0/890252178 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1780092e0 con 0x7fc180159c00 2024-01-27T02:38:32.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.822+0000 7fc185a44700 1 -- 172.21.15.7:0/890252178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc180159c00 msgr2=0x7fc18015a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:32.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.822+0000 7fc185a44700 1 --2- 172.21.15.7:0/890252178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc180159c00 0x7fc18015a0a0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc1800cca20 tx=0x7fc178009840 comp rx=0 tx=0).stop 2024-01-27T02:38:32.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.822+0000 7fc185a44700 1 -- 172.21.15.7:0/890252178 shutdown_connections 2024-01-27T02:38:32.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.822+0000 7fc185a44700 1 --2- 172.21.15.7:0/890252178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc180159c00 0x7fc18015a0a0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.822+0000 7fc185a44700 1 --2- 172.21.15.7:0/890252178 >> v2:172.21.15.188:3300/0 conn(0x7fc18015ea60 0x7fc1800cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.822+0000 7fc185a44700 1 -- 172.21.15.7:0/890252178 >> 172.21.15.7:0/890252178 conn(0x7fc180063580 msgr2=0x7fc180063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:32.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.822+0000 7fc185a44700 1 -- 172.21.15.7:0/890252178 shutdown_connections 2024-01-27T02:38:32.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.822+0000 7fc185a44700 1 -- 172.21.15.7:0/890252178 wait complete. 2024-01-27T02:38:32.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.824+0000 7fc185a44700 1 Processor -- start 2024-01-27T02:38:32.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.824+0000 7fc185a44700 1 -- start start 2024-01-27T02:38:32.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.825+0000 7fc185a44700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1800cc110 0x7fc1801583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:32.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.825+0000 7fc185a44700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc180159c00 0x7fc180155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:32.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.825+0000 7fc185a44700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc18015ea60 0x7fc180156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:32.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.825+0000 7fc185a44700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1800c9c50 con 0x7fc18015ea60 2024-01-27T02:38:32.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.825+0000 7fc185a44700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1800c9040 con 0x7fc180159c00 2024-01-27T02:38:32.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.825+0000 7fc185a44700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1800c9320 con 0x7fc1800cc110 2024-01-27T02:38:32.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.825+0000 7fc17f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc18015ea60 0x7fc180156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:32.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.825+0000 7fc17f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc18015ea60 0x7fc180156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58784/0 (socket says 172.21.15.7:58784) 2024-01-27T02:38:32.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.825+0000 7fc17f7fe700 1 -- 172.21.15.7:0/2751591056 learned_addr learned my addr 172.21.15.7:0/2751591056 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:32.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.825+0000 7fc17e7fc700 1 --2- 172.21.15.7:0/2751591056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc180159c00 0x7fc180155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:32.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.825+0000 7fc17effd700 1 --2- 172.21.15.7:0/2751591056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1800cc110 0x7fc1801583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:32.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.825+0000 7fc17f7fe700 1 -- 172.21.15.7:0/2751591056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1800cc110 msgr2=0x7fc1801583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:32.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.825+0000 7fc17f7fe700 1 --2- 172.21.15.7:0/2751591056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1800cc110 0x7fc1801583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.825+0000 7fc17f7fe700 1 -- 172.21.15.7:0/2751591056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc180159c00 msgr2=0x7fc180155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:32.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.825+0000 7fc17f7fe700 1 --2- 172.21.15.7:0/2751591056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc180159c00 0x7fc180155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:32.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.825+0000 7fc17f7fe700 1 -- 172.21.15.7:0/2751591056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc178009ae0 con 0x7fc18015ea60 2024-01-27T02:38:32.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.826+0000 7fc17f7fe700 1 --2- 172.21.15.7:0/2751591056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc18015ea60 0x7fc180156280 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc18008a040 tx=0x7fc178018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:32.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.826+0000 7fc176ffd700 1 -- 172.21.15.7:0/2751591056 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17800be60 con 0x7fc18015ea60 2024-01-27T02:38:32.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.826+0000 7fc176ffd700 1 -- 172.21.15.7:0/2751591056 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc178008ce0 con 0x7fc18015ea60 2024-01-27T02:38:32.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.826+0000 7fc176ffd700 1 -- 172.21.15.7:0/2751591056 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1780266b0 con 0x7fc18015ea60 2024-01-27T02:38:32.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.826+0000 7fc185a44700 1 -- 172.21.15.7:0/2751591056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1800c9600 con 0x7fc18015ea60 2024-01-27T02:38:32.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.826+0000 7fc185a44700 1 -- 172.21.15.7:0/2751591056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc180159490 con 0x7fc18015ea60 2024-01-27T02:38:32.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.827+0000 7fc185a44700 1 -- 172.21.15.7:0/2751591056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1800c9e00 con 0x7fc18015ea60 2024-01-27T02:38:32.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.827+0000 7fc176ffd700 1 -- 172.21.15.7:0/2751591056 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc178003040 con 0x7fc18015ea60 2024-01-27T02:38:32.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.828+0000 7fc176ffd700 1 --2- 172.21.15.7:0/2751591056 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc160066f30 0x7fc1600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:32.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.828+0000 7fc176ffd700 1 -- 172.21.15.7:0/2751591056 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc178085f10 con 0x7fc18015ea60 2024-01-27T02:38:32.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.828+0000 7fc17effd700 1 --2- 172.21.15.7:0/2751591056 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc160066f30 0x7fc1600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:32.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.829+0000 7fc17effd700 1 --2- 172.21.15.7:0/2751591056 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc160066f30 0x7fc1600693e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc1801581b0 tx=0x7fc170009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:32.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:32.832+0000 7fc176ffd700 1 -- 172.21.15.7:0/2751591056 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc178053230 con 0x7fc18015ea60 2024-01-27T02:38:33.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:33.086+0000 7fc185a44700 1 -- 172.21.15.7:0/2751591056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc180072060 con 0x7fc18015ea60 2024-01-27T02:38:33.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:33.941+0000 7fc176ffd700 1 -- 172.21.15.7:0/2751591056 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v67) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc1780210b0 con 0x7fc18015ea60 2024-01-27T02:38:33.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:33.944+0000 7fc185a44700 1 -- 172.21.15.7:0/2751591056 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc160066f30 msgr2=0x7fc1600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:33.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:33.944+0000 7fc185a44700 1 --2- 172.21.15.7:0/2751591056 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc160066f30 0x7fc1600693e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc1801581b0 tx=0x7fc170009040 comp rx=0 tx=0).stop 2024-01-27T02:38:33.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:33.944+0000 7fc185a44700 1 -- 172.21.15.7:0/2751591056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc18015ea60 msgr2=0x7fc180156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:33.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:33.944+0000 7fc185a44700 1 --2- 172.21.15.7:0/2751591056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc18015ea60 0x7fc180156280 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc18008a040 tx=0x7fc178018000 comp rx=0 tx=0).stop 2024-01-27T02:38:33.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:33.944+0000 7fc185a44700 1 -- 172.21.15.7:0/2751591056 shutdown_connections 2024-01-27T02:38:33.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:33.944+0000 7fc185a44700 1 --2- 172.21.15.7:0/2751591056 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc160066f30 0x7fc1600693e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:33.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:33.944+0000 7fc185a44700 1 --2- 172.21.15.7:0/2751591056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc18015ea60 0x7fc180156280 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:33.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:33.944+0000 7fc185a44700 1 --2- 172.21.15.7:0/2751591056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1800cc110 0x7fc1801583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:33.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:33.945+0000 7fc185a44700 1 --2- 172.21.15.7:0/2751591056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc180159c00 0x7fc180155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:33.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:33.945+0000 7fc185a44700 1 -- 172.21.15.7:0/2751591056 >> 172.21.15.7:0/2751591056 conn(0x7fc180063580 msgr2=0x7fc1800c71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:33.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:33.945+0000 7fc185a44700 1 -- 172.21.15.7:0/2751591056 shutdown_connections 2024-01-27T02:38:33.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:33.945+0000 7fc185a44700 1 -- 172.21.15.7:0/2751591056 wait complete. 2024-01-27T02:38:33.960 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-27T02:38:34.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.099+0000 7f5a96e26700 1 Processor -- start 2024-01-27T02:38:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.099+0000 7f5a96e26700 1 -- start start 2024-01-27T02:38:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.100+0000 7f5a96e26700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5a900cf280 0x7f5a900d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.100+0000 7f5a96e26700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5a9015d830 0x7f5a900cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.100+0000 7f5a96e26700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a900b38d0 con 0x7f5a9015d460 2024-01-27T02:38:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.100+0000 7f5a96e26700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a900d0390 con 0x7f5a9015d830 2024-01-27T02:38:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.100+0000 7f5a96e26700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a900cf780 con 0x7f5a900cf280 2024-01-27T02:38:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.100+0000 7f5a953c3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5a9015d830 0x7f5a900cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:34.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.100+0000 7f5a94bc2700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5a9015d460 0x7f5a9015e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50600/0 (socket says 172.21.15.7:50600) 2024-01-27T02:38:34.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.100+0000 7f5a94bc2700 1 -- 172.21.15.7:0/1705653496 learned_addr learned my addr 172.21.15.7:0/1705653496 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:34.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.100+0000 7f5a8ffff700 1 --2- 172.21.15.7:0/1705653496 >> v2:172.21.15.188:3300/0 conn(0x7f5a900cf280 0x7f5a900d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:34.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.100+0000 7f5a953c3700 1 -- 172.21.15.7:0/1705653496 >> v2:172.21.15.188:3300/0 conn(0x7f5a900cf280 msgr2=0x7f5a900d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:34.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.100+0000 7f5a953c3700 1 --2- 172.21.15.7:0/1705653496 >> v2:172.21.15.188:3300/0 conn(0x7f5a900cf280 0x7f5a900d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:34.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.100+0000 7f5a953c3700 1 -- 172.21.15.7:0/1705653496 >> v1:172.21.15.7:6789/0 conn(0x7f5a9015d460 legacy=0x7f5a9015e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:34.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.100+0000 7f5a953c3700 1 -- 172.21.15.7:0/1705653496 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a900cfa60 con 0x7f5a9015d830 2024-01-27T02:38:34.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.100+0000 7f5a8ffff700 1 --2- 172.21.15.7:0/1705653496 >> v2:172.21.15.188:3300/0 conn(0x7f5a900cf280 0x7f5a900d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:38:34.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.101+0000 7f5a953c3700 1 --2- 172.21.15.7:0/1705653496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a9015d830 0x7f5a900cd700 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f5a9008a1e0 tx=0x7f5a88009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0d9d08094c42c7 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:34.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.101+0000 7f5a8f7fe700 1 -- 172.21.15.7:0/1705653496 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8800e070 con 0x7f5a9015d830 2024-01-27T02:38:34.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.101+0000 7f5a8f7fe700 1 -- 172.21.15.7:0/1705653496 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8800bc30 con 0x7f5a9015d830 2024-01-27T02:38:34.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.101+0000 7f5a8f7fe700 1 -- 172.21.15.7:0/1705653496 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a88012430 con 0x7f5a9015d830 2024-01-27T02:38:34.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.101+0000 7f5a96e26700 1 -- 172.21.15.7:0/1705653496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a9015d830 msgr2=0x7f5a900cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:34.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.101+0000 7f5a96e26700 1 --2- 172.21.15.7:0/1705653496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a9015d830 0x7f5a900cd700 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f5a9008a1e0 tx=0x7f5a88009780 comp rx=0 tx=0).stop 2024-01-27T02:38:34.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.101+0000 7f5a96e26700 1 -- 172.21.15.7:0/1705653496 shutdown_connections 2024-01-27T02:38:34.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.101+0000 7f5a96e26700 1 --2- 172.21.15.7:0/1705653496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a9015d830 0x7f5a900cd700 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.101+0000 7f5a96e26700 1 --2- 172.21.15.7:0/1705653496 >> v2:172.21.15.188:3300/0 conn(0x7f5a900cf280 0x7f5a900d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.102+0000 7f5a96e26700 1 -- 172.21.15.7:0/1705653496 >> 172.21.15.7:0/1705653496 conn(0x7f5a90063580 msgr2=0x7f5a90063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.102+0000 7f5a96e26700 1 -- 172.21.15.7:0/1705653496 shutdown_connections 2024-01-27T02:38:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.102+0000 7f5a96e26700 1 -- 172.21.15.7:0/1705653496 wait complete. 2024-01-27T02:38:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.103+0000 7f5a96e26700 1 Processor -- start 2024-01-27T02:38:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.103+0000 7f5a96e26700 1 -- start start 2024-01-27T02:38:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.104+0000 7f5a96e26700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a900cf280 0x7f5a9015c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.104+0000 7f5a96e26700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a9015d460 0x7f5a9014f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.104+0000 7f5a96e26700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a9015d830 0x7f5a9014f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.104+0000 7f5a96e26700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a900c80c0 con 0x7f5a9015d830 2024-01-27T02:38:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.104+0000 7f5a96e26700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9015bb80 con 0x7f5a9015d460 2024-01-27T02:38:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.104+0000 7f5a96e26700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9015b530 con 0x7f5a900cf280 2024-01-27T02:38:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.104+0000 7f5a953c3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a9015d830 0x7f5a9014f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.104+0000 7f5a953c3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a9015d830 0x7f5a9014f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58794/0 (socket says 172.21.15.7:58794) 2024-01-27T02:38:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.104+0000 7f5a953c3700 1 -- 172.21.15.7:0/2381403821 learned_addr learned my addr 172.21.15.7:0/2381403821 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.104+0000 7f5a8ffff700 1 --2- 172.21.15.7:0/2381403821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a9015d460 0x7f5a9014f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.104+0000 7f5a94bc2700 1 --2- 172.21.15.7:0/2381403821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a900cf280 0x7f5a9015c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.104+0000 7f5a953c3700 1 -- 172.21.15.7:0/2381403821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a900cf280 msgr2=0x7f5a9015c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.104+0000 7f5a953c3700 1 --2- 172.21.15.7:0/2381403821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a900cf280 0x7f5a9015c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.104+0000 7f5a953c3700 1 -- 172.21.15.7:0/2381403821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a9015d460 msgr2=0x7f5a9014f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.104+0000 7f5a953c3700 1 --2- 172.21.15.7:0/2381403821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a9015d460 0x7f5a9014f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.104+0000 7f5a953c3700 1 -- 172.21.15.7:0/2381403821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a88009a20 con 0x7f5a9015d830 2024-01-27T02:38:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.104+0000 7f5a8ffff700 1 --2- 172.21.15.7:0/2381403821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a9015d460 0x7f5a9014f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.105+0000 7f5a953c3700 1 --2- 172.21.15.7:0/2381403821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a9015d830 0x7f5a9014f060 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f5a88013000 tx=0x7f5a88019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.105+0000 7f5a8cff9700 1 -- 172.21.15.7:0/2381403821 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8801e630 con 0x7f5a9015d830 2024-01-27T02:38:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.105+0000 7f5a8cff9700 1 -- 172.21.15.7:0/2381403821 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8801e790 con 0x7f5a9015d830 2024-01-27T02:38:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.105+0000 7f5a96e26700 1 -- 172.21.15.7:0/2381403821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9015b810 con 0x7f5a9015d830 2024-01-27T02:38:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.105+0000 7f5a8cff9700 1 -- 172.21.15.7:0/2381403821 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8801eb50 con 0x7f5a9015d830 2024-01-27T02:38:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.105+0000 7f5a96e26700 1 -- 172.21.15.7:0/2381403821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9014fe50 con 0x7f5a9015d830 2024-01-27T02:38:34.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.106+0000 7f5a96e26700 1 -- 172.21.15.7:0/2381403821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a9015b9a0 con 0x7f5a9015d830 2024-01-27T02:38:34.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.106+0000 7f5a8cff9700 1 -- 172.21.15.7:0/2381403821 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5a88024070 con 0x7f5a9015d830 2024-01-27T02:38:34.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.107+0000 7f5a8cff9700 1 --2- 172.21.15.7:0/2381403821 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5a74066f30 0x7f5a740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:34.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.107+0000 7f5a8cff9700 1 -- 172.21.15.7:0/2381403821 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5a88086f80 con 0x7f5a9015d830 2024-01-27T02:38:34.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.107+0000 7f5a94bc2700 1 --2- 172.21.15.7:0/2381403821 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5a74066f30 0x7f5a740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:34.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.107+0000 7f5a94bc2700 1 --2- 172.21.15.7:0/2381403821 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5a74066f30 0x7f5a740693e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5a900ca950 tx=0x7f5a84009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.111+0000 7f5a8cff9700 1 -- 172.21.15.7:0/2381403821 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a880b85a0 con 0x7f5a9015d830 2024-01-27T02:38:34.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.359+0000 7f5a96e26700 1 -- 172.21.15.7:0/2381403821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5a90072060 con 0x7f5a9015d830 2024-01-27T02:38:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.943+0000 7f5a8cff9700 1 -- 172.21.15.7:0/2381403821 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v68) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5a880541f0 con 0x7f5a9015d830 2024-01-27T02:38:34.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.946+0000 7f5a96e26700 1 -- 172.21.15.7:0/2381403821 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5a74066f30 msgr2=0x7f5a740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:34.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.946+0000 7f5a96e26700 1 --2- 172.21.15.7:0/2381403821 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5a74066f30 0x7f5a740693e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5a900ca950 tx=0x7f5a84009040 comp rx=0 tx=0).stop 2024-01-27T02:38:34.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.946+0000 7f5a96e26700 1 -- 172.21.15.7:0/2381403821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a9015d830 msgr2=0x7f5a9014f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:34.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.946+0000 7f5a96e26700 1 --2- 172.21.15.7:0/2381403821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a9015d830 0x7f5a9014f060 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f5a88013000 tx=0x7f5a88019000 comp rx=0 tx=0).stop 2024-01-27T02:38:34.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.946+0000 7f5a96e26700 1 -- 172.21.15.7:0/2381403821 shutdown_connections 2024-01-27T02:38:34.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.946+0000 7f5a96e26700 1 --2- 172.21.15.7:0/2381403821 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5a74066f30 0x7f5a740693e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:34.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.946+0000 7f5a96e26700 1 --2- 172.21.15.7:0/2381403821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a9015d830 0x7f5a9014f060 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:34.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.946+0000 7f5a96e26700 1 --2- 172.21.15.7:0/2381403821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a900cf280 0x7f5a9015c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:34.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.946+0000 7f5a96e26700 1 --2- 172.21.15.7:0/2381403821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a9015d460 0x7f5a9014f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:34.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.946+0000 7f5a96e26700 1 -- 172.21.15.7:0/2381403821 >> 172.21.15.7:0/2381403821 conn(0x7f5a90063580 msgr2=0x7f5a900c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:34.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.946+0000 7f5a96e26700 1 -- 172.21.15.7:0/2381403821 shutdown_connections 2024-01-27T02:38:34.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:34.946+0000 7f5a96e26700 1 -- 172.21.15.7:0/2381403821 wait complete. 2024-01-27T02:38:34.962 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:38:35.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.097+0000 7f7db244b700 1 Processor -- start 2024-01-27T02:38:35.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.097+0000 7f7db244b700 1 -- start start 2024-01-27T02:38:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.098+0000 7f7db244b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7dac15a100 0x7f7dac0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.099+0000 7f7dabfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7dac15a100 0x7f7dac0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.099+0000 7f7db244b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dac0ce5e0 con 0x7f7dac0c4cc0 2024-01-27T02:38:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.099+0000 7f7db244b700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dac0cd9d0 con 0x7f7dac0cb5b0 2024-01-27T02:38:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.099+0000 7f7db244b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dac0cdcb0 con 0x7f7dac15a100 2024-01-27T02:38:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.099+0000 7f7dabfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7dac15a100 0x7f7dac0cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58804/0 (socket says 172.21.15.7:58804) 2024-01-27T02:38:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.099+0000 7f7dabfff700 1 -- 172.21.15.7:0/1574300721 learned_addr learned my addr 172.21.15.7:0/1574300721 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.099+0000 7f7dabfff700 1 -- 172.21.15.7:0/1574300721 >> v1:172.21.15.188:6789/0 conn(0x7f7dac0cb5b0 legacy=0x7f7dac0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.099+0000 7f7dabfff700 1 -- 172.21.15.7:0/1574300721 >> v1:172.21.15.7:6789/0 conn(0x7f7dac0c4cc0 legacy=0x7f7dac163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.099+0000 7f7dabfff700 1 -- 172.21.15.7:0/1574300721 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dac0cdf90 con 0x7f7dac15a100 2024-01-27T02:38:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.099+0000 7f7daaffd700 1 -- 172.21.15.7:0/1574300721 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 450073777 0 0) 0x7f7dac0ce5e0 con 0x7f7dac0c4cc0 2024-01-27T02:38:35.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.099+0000 7f7dabfff700 1 --2- 172.21.15.7:0/1574300721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dac15a100 0x7f7dac0cd2f0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c0099b0 tx=0x7f7d9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec5f8c5d38b024b server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:35.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.099+0000 7f7daaffd700 1 -- 172.21.15.7:0/1574300721 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c00b820 con 0x7f7dac15a100 2024-01-27T02:38:35.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.099+0000 7f7daaffd700 1 -- 172.21.15.7:0/1574300721 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9c00b980 con 0x7f7dac15a100 2024-01-27T02:38:35.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.099+0000 7f7daaffd700 1 -- 172.21.15.7:0/1574300721 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c00bd20 con 0x7f7dac15a100 2024-01-27T02:38:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.100+0000 7f7db244b700 1 -- 172.21.15.7:0/1574300721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dac15a100 msgr2=0x7f7dac0cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.100+0000 7f7db244b700 1 --2- 172.21.15.7:0/1574300721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dac15a100 0x7f7dac0cd2f0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c0099b0 tx=0x7f7d9c009250 comp rx=0 tx=0).stop 2024-01-27T02:38:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.100+0000 7f7db244b700 1 -- 172.21.15.7:0/1574300721 shutdown_connections 2024-01-27T02:38:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.100+0000 7f7db244b700 1 --2- 172.21.15.7:0/1574300721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dac15a100 0x7f7dac0cd2f0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.100+0000 7f7db244b700 1 -- 172.21.15.7:0/1574300721 >> 172.21.15.7:0/1574300721 conn(0x7f7dac098020 msgr2=0x7f7dac0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.100+0000 7f7db244b700 1 -- 172.21.15.7:0/1574300721 shutdown_connections 2024-01-27T02:38:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.100+0000 7f7db244b700 1 -- 172.21.15.7:0/1574300721 wait complete. 2024-01-27T02:38:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.101+0000 7f7db244b700 1 Processor -- start 2024-01-27T02:38:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.102+0000 7f7db244b700 1 -- start start 2024-01-27T02:38:35.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.102+0000 7f7db244b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dac0c4cc0 0x7f7dac15ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:35.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.102+0000 7f7db244b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dac0cb5b0 0x7f7dac15b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.102+0000 7f7db244b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dac15a100 0x7f7dac0b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.102+0000 7f7db244b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac15d350 con 0x7f7dac0cb5b0 2024-01-27T02:38:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.102+0000 7f7db244b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac15b5b0 con 0x7f7dac15a100 2024-01-27T02:38:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.102+0000 7f7db244b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac0c61d0 con 0x7f7dac0c4cc0 2024-01-27T02:38:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.102+0000 7f7dab7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dac0cb5b0 0x7f7dac15b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.102+0000 7f7dab7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dac0cb5b0 0x7f7dac15b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58812/0 (socket says 172.21.15.7:58812) 2024-01-27T02:38:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.102+0000 7f7dab7fe700 1 -- 172.21.15.7:0/4224153793 learned_addr learned my addr 172.21.15.7:0/4224153793 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.103+0000 7f7db09e8700 1 --2- 172.21.15.7:0/4224153793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dac15a100 0x7f7dac0b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.103+0000 7f7dabfff700 1 --2- 172.21.15.7:0/4224153793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dac0c4cc0 0x7f7dac15ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.103+0000 7f7dab7fe700 1 -- 172.21.15.7:0/4224153793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dac0c4cc0 msgr2=0x7f7dac15ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.103+0000 7f7dab7fe700 1 --2- 172.21.15.7:0/4224153793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dac0c4cc0 0x7f7dac15ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.103+0000 7f7dab7fe700 1 -- 172.21.15.7:0/4224153793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dac15a100 msgr2=0x7f7dac0b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.103+0000 7f7dab7fe700 1 --2- 172.21.15.7:0/4224153793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dac15a100 0x7f7dac0b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.103+0000 7f7dab7fe700 1 -- 172.21.15.7:0/4224153793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9c0094f0 con 0x7f7dac0cb5b0 2024-01-27T02:38:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.103+0000 7f7dab7fe700 1 --2- 172.21.15.7:0/4224153793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dac0cb5b0 0x7f7dac15b070 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f7da000c630 tx=0x7f7da0007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.103+0000 7f7d93fff700 1 -- 172.21.15.7:0/4224153793 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da0011590 con 0x7f7dac0cb5b0 2024-01-27T02:38:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.103+0000 7f7d93fff700 1 -- 172.21.15.7:0/4224153793 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da0011710 con 0x7f7dac0cb5b0 2024-01-27T02:38:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.103+0000 7f7db244b700 1 -- 172.21.15.7:0/4224153793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dac15dd90 con 0x7f7dac0cb5b0 2024-01-27T02:38:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.103+0000 7f7d93fff700 1 -- 172.21.15.7:0/4224153793 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da0011ad0 con 0x7f7dac0cb5b0 2024-01-27T02:38:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.103+0000 7f7db244b700 1 -- 172.21.15.7:0/4224153793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dac15e350 con 0x7f7dac0cb5b0 2024-01-27T02:38:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.104+0000 7f7db244b700 1 -- 172.21.15.7:0/4224153793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dac04c710 con 0x7f7dac0cb5b0 2024-01-27T02:38:35.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.105+0000 7f7d93fff700 1 -- 172.21.15.7:0/4224153793 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7da001c460 con 0x7f7dac0cb5b0 2024-01-27T02:38:35.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.105+0000 7f7d93fff700 1 --2- 172.21.15.7:0/4224153793 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7d98066f30 0x7f7d980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:35.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.105+0000 7f7d93fff700 1 -- 172.21.15.7:0/4224153793 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7da0084250 con 0x7f7dac0cb5b0 2024-01-27T02:38:35.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.105+0000 7f7dabfff700 1 --2- 172.21.15.7:0/4224153793 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7d98066f30 0x7f7d980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:35.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.106+0000 7f7dabfff700 1 --2- 172.21.15.7:0/4224153793 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7d98066f30 0x7f7d980693e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c0099b0 tx=0x7f7d9c016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.109+0000 7f7d93fff700 1 -- 172.21.15.7:0/4224153793 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7da00514c0 con 0x7f7dac0cb5b0 2024-01-27T02:38:35.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.359+0000 7f7db244b700 1 -- 172.21.15.7:0/4224153793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7dac0ce270 con 0x7f7dac0cb5b0 2024-01-27T02:38:35.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.360+0000 7f7d93fff700 1 -- 172.21.15.7:0/4224153793 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f7da0054ae0 con 0x7f7dac0cb5b0 2024-01-27T02:38:35.361 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:35.361 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3997828193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3997828193},{"type":"v1","addr":"172.21.15.188:6837","nonce":3997828193}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1132338302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1132338302},{"type":"v1","addr":"172.21.15.188:6836","nonce":1132338302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/740290645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":740290645},{"type":"v1","addr":"172.21.15.7:6833","nonce":740290645}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:38:31.328277+0000","modified":"2024-01-27T02:38:34.940811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7029":{"gid":7029,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1258751919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1258751919},{"type":"v1","addr":"172.21.15.7:6835","nonce":1258751919}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-27T02:38:35.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.362+0000 7f7db244b700 1 -- 172.21.15.7:0/4224153793 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7d98066f30 msgr2=0x7f7d980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:35.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.362+0000 7f7db244b700 1 --2- 172.21.15.7:0/4224153793 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7d98066f30 0x7f7d980693e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c0099b0 tx=0x7f7d9c016000 comp rx=0 tx=0).stop 2024-01-27T02:38:35.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.362+0000 7f7db244b700 1 -- 172.21.15.7:0/4224153793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dac0cb5b0 msgr2=0x7f7dac15b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:35.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.362+0000 7f7db244b700 1 --2- 172.21.15.7:0/4224153793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dac0cb5b0 0x7f7dac15b070 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f7da000c630 tx=0x7f7da0007f60 comp rx=0 tx=0).stop 2024-01-27T02:38:35.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.362+0000 7f7db244b700 1 -- 172.21.15.7:0/4224153793 shutdown_connections 2024-01-27T02:38:35.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.362+0000 7f7db244b700 1 --2- 172.21.15.7:0/4224153793 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7d98066f30 0x7f7d980693e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:35.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.362+0000 7f7db244b700 1 --2- 172.21.15.7:0/4224153793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7dac15a100 0x7f7dac0b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:35.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.362+0000 7f7db244b700 1 --2- 172.21.15.7:0/4224153793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7dac0c4cc0 0x7f7dac15ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:35.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.362+0000 7f7db244b700 1 --2- 172.21.15.7:0/4224153793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dac0cb5b0 0x7f7dac15b070 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:35.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.362+0000 7f7db244b700 1 -- 172.21.15.7:0/4224153793 >> 172.21.15.7:0/4224153793 conn(0x7f7dac098020 msgr2=0x7f7dac067bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:35.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.362+0000 7f7db244b700 1 -- 172.21.15.7:0/4224153793 shutdown_connections 2024-01-27T02:38:35.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.362+0000 7f7db244b700 1 -- 172.21.15.7:0/4224153793 wait complete. 2024-01-27T02:38:35.364 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 68 2024-01-27T02:38:35.376 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:38:35.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.507+0000 7f15db590700 1 Processor -- start 2024-01-27T02:38:35.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.507+0000 7f15db590700 1 -- start start 2024-01-27T02:38:35.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.508+0000 7f15db590700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f15d415a100 0x7f15d40cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:35.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.508+0000 7f15db590700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d40c4cc0 0x7f15d4163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:35.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.508+0000 7f15db590700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d40ce5e0 con 0x7f15d40cb5b0 2024-01-27T02:38:35.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.508+0000 7f15db590700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d40cd9d0 con 0x7f15d415a100 2024-01-27T02:38:35.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.508+0000 7f15db590700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40cdcb0 con 0x7f15d40c4cc0 2024-01-27T02:38:35.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.509+0000 7f15d9b2d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d40c4cc0 0x7f15d4163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:35.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.509+0000 7f15d932c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f15d415a100 0x7f15d40cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:35.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.509+0000 7f15d9b2d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d40c4cc0 0x7f15d4163d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:55044/0 (socket says 172.21.15.7:55044) 2024-01-27T02:38:35.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.509+0000 7f15d9b2d700 1 -- 172.21.15.7:0/108239210 learned_addr learned my addr 172.21.15.7:0/108239210 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:35.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.509+0000 7f15d9b2d700 1 -- 172.21.15.7:0/108239210 >> v2:172.21.15.188:3300/0 conn(0x7f15d415a100 msgr2=0x7f15d40cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:35.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.509+0000 7f15d9b2d700 1 --2- 172.21.15.7:0/108239210 >> v2:172.21.15.188:3300/0 conn(0x7f15d415a100 0x7f15d40cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:35.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.509+0000 7f15d9b2d700 1 -- 172.21.15.7:0/108239210 >> v1:172.21.15.188:6789/0 conn(0x7f15d40cb5b0 legacy=0x7f15d40c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:35.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.509+0000 7f15d9b2d700 1 -- 172.21.15.7:0/108239210 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d40cdf90 con 0x7f15d40c4cc0 2024-01-27T02:38:35.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.509+0000 7f15d932c700 1 --2- 172.21.15.7:0/108239210 >> v2:172.21.15.188:3300/0 conn(0x7f15d415a100 0x7f15d40cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:38:35.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.509+0000 7f15cbfff700 1 -- 172.21.15.7:0/108239210 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1767165402 0 0) 0x7f15d40ce5e0 con 0x7f15d40cb5b0 2024-01-27T02:38:35.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.509+0000 7f15d9b2d700 1 --2- 172.21.15.7:0/108239210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d40c4cc0 0x7f15d4163d90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f15cc009f20 tx=0x7f15cc009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44322dfae409c675 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:35.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.509+0000 7f15cbfff700 1 -- 172.21.15.7:0/108239210 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc00e070 con 0x7f15d40c4cc0 2024-01-27T02:38:35.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.509+0000 7f15cbfff700 1 -- 172.21.15.7:0/108239210 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15cc00b9d0 con 0x7f15d40c4cc0 2024-01-27T02:38:35.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.509+0000 7f15cbfff700 1 -- 172.21.15.7:0/108239210 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc0092e0 con 0x7f15d40c4cc0 2024-01-27T02:38:35.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.510+0000 7f15db590700 1 -- 172.21.15.7:0/108239210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d40c4cc0 msgr2=0x7f15d4163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:35.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.510+0000 7f15db590700 1 --2- 172.21.15.7:0/108239210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d40c4cc0 0x7f15d4163d90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f15cc009f20 tx=0x7f15cc009840 comp rx=0 tx=0).stop 2024-01-27T02:38:35.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.510+0000 7f15db590700 1 -- 172.21.15.7:0/108239210 shutdown_connections 2024-01-27T02:38:35.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.510+0000 7f15db590700 1 --2- 172.21.15.7:0/108239210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d40c4cc0 0x7f15d4163d90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:35.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.510+0000 7f15db590700 1 --2- 172.21.15.7:0/108239210 >> v2:172.21.15.188:3300/0 conn(0x7f15d415a100 0x7f15d40cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:35.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.510+0000 7f15db590700 1 -- 172.21.15.7:0/108239210 >> 172.21.15.7:0/108239210 conn(0x7f15d4098020 msgr2=0x7f15d40711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:35.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.510+0000 7f15db590700 1 -- 172.21.15.7:0/108239210 shutdown_connections 2024-01-27T02:38:35.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.510+0000 7f15db590700 1 -- 172.21.15.7:0/108239210 wait complete. 2024-01-27T02:38:35.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.511+0000 7f15db590700 1 Processor -- start 2024-01-27T02:38:35.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.511+0000 7f15db590700 1 -- start start 2024-01-27T02:38:35.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.512+0000 7f15db590700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d40c4cc0 0x7f15d416ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:35.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.512+0000 7f15db590700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15d40cb5b0 0x7f15d40b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:35.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.512+0000 7f15db590700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d415a100 0x7f15d40b4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:35.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.512+0000 7f15db590700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40b49b0 con 0x7f15d40c4cc0 2024-01-27T02:38:35.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.512+0000 7f15db590700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40b1ca0 con 0x7f15d40cb5b0 2024-01-27T02:38:35.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.512+0000 7f15db590700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40b1f40 con 0x7f15d415a100 2024-01-27T02:38:35.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.512+0000 7f15d932c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d40c4cc0 0x7f15d416ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:35.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.513+0000 7f15d932c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d40c4cc0 0x7f15d416ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58824/0 (socket says 172.21.15.7:58824) 2024-01-27T02:38:35.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.513+0000 7f15d932c700 1 -- 172.21.15.7:0/3462933196 learned_addr learned my addr 172.21.15.7:0/3462933196 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:35.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.513+0000 7f15d8b2b700 1 --2- 172.21.15.7:0/3462933196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15d40cb5b0 0x7f15d40b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:35.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.513+0000 7f15d9b2d700 1 --2- 172.21.15.7:0/3462933196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d415a100 0x7f15d40b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:35.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.513+0000 7f15d932c700 1 -- 172.21.15.7:0/3462933196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d415a100 msgr2=0x7f15d40b4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:35.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.513+0000 7f15d932c700 1 --2- 172.21.15.7:0/3462933196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d415a100 0x7f15d40b4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:35.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.513+0000 7f15d932c700 1 -- 172.21.15.7:0/3462933196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15d40cb5b0 msgr2=0x7f15d40b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:35.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.513+0000 7f15d932c700 1 --2- 172.21.15.7:0/3462933196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15d40cb5b0 0x7f15d40b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:35.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.513+0000 7f15d932c700 1 -- 172.21.15.7:0/3462933196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d0009420 con 0x7f15d40c4cc0 2024-01-27T02:38:35.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.513+0000 7f15d932c700 1 --2- 172.21.15.7:0/3462933196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d40c4cc0 0x7f15d416ae70 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f15d000ec80 tx=0x7f15d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:35.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.513+0000 7f15c97fa700 1 -- 172.21.15.7:0/3462933196 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d000c920 con 0x7f15d40c4cc0 2024-01-27T02:38:35.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.513+0000 7f15c97fa700 1 -- 172.21.15.7:0/3462933196 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d000ca80 con 0x7f15d40c4cc0 2024-01-27T02:38:35.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.513+0000 7f15db590700 1 -- 172.21.15.7:0/3462933196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15cc009ae0 con 0x7f15d40c4cc0 2024-01-27T02:38:35.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.513+0000 7f15c97fa700 1 -- 172.21.15.7:0/3462933196 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d000ce00 con 0x7f15d40c4cc0 2024-01-27T02:38:35.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.513+0000 7f15db590700 1 -- 172.21.15.7:0/3462933196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d40b2500 con 0x7f15d40c4cc0 2024-01-27T02:38:35.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.515+0000 7f15db590700 1 -- 172.21.15.7:0/3462933196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d40b2870 con 0x7f15d40c4cc0 2024-01-27T02:38:35.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.515+0000 7f15c97fa700 1 -- 172.21.15.7:0/3462933196 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f15d0013680 con 0x7f15d40c4cc0 2024-01-27T02:38:35.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.515+0000 7f15c97fa700 1 --2- 172.21.15.7:0/3462933196 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f15b8066f30 0x7f15b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:35.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.516+0000 7f15c97fa700 1 -- 172.21.15.7:0/3462933196 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f15d00849b0 con 0x7f15d40c4cc0 2024-01-27T02:38:35.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.516+0000 7f15d8b2b700 1 --2- 172.21.15.7:0/3462933196 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f15b8066f30 0x7f15b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:35.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.516+0000 7f15d8b2b700 1 --2- 172.21.15.7:0/3462933196 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f15b8066f30 0x7f15b80693e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f15d41599e0 tx=0x7f15c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:35.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.520+0000 7f15c97fa700 1 -- 172.21.15.7:0/3462933196 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f15d0051b80 con 0x7f15d40c4cc0 2024-01-27T02:38:35.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.725+0000 7f15db590700 1 -- 172.21.15.7:0/3462933196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15d416b470 con 0x7f15d40c4cc0 2024-01-27T02:38:35.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.725+0000 7f15c97fa700 1 -- 172.21.15.7:0/3462933196 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+15317 (secure 0 0 0) 0x7f15d006be60 con 0x7f15d40c4cc0 2024-01-27T02:38:35.726 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:35.726 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":132,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:38:32.906200+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:38:29.824892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-27T02:38:30.301072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:38:35.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.728+0000 7f15db590700 1 -- 172.21.15.7:0/3462933196 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f15b8066f30 msgr2=0x7f15b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:35.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.728+0000 7f15db590700 1 --2- 172.21.15.7:0/3462933196 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f15b8066f30 0x7f15b80693e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f15d41599e0 tx=0x7f15c4009040 comp rx=0 tx=0).stop 2024-01-27T02:38:35.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.728+0000 7f15db590700 1 -- 172.21.15.7:0/3462933196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d40c4cc0 msgr2=0x7f15d416ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:35.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.728+0000 7f15db590700 1 --2- 172.21.15.7:0/3462933196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d40c4cc0 0x7f15d416ae70 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f15d000ec80 tx=0x7f15d00091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:35.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.728+0000 7f15db590700 1 -- 172.21.15.7:0/3462933196 shutdown_connections 2024-01-27T02:38:35.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.728+0000 7f15db590700 1 --2- 172.21.15.7:0/3462933196 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f15b8066f30 0x7f15b80693e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:35.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.728+0000 7f15db590700 1 --2- 172.21.15.7:0/3462933196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d415a100 0x7f15d40b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:35.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.728+0000 7f15db590700 1 --2- 172.21.15.7:0/3462933196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d40c4cc0 0x7f15d416ae70 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:35.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.728+0000 7f15db590700 1 --2- 172.21.15.7:0/3462933196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15d40cb5b0 0x7f15d40b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:35.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.728+0000 7f15db590700 1 -- 172.21.15.7:0/3462933196 >> 172.21.15.7:0/3462933196 conn(0x7f15d4098020 msgr2=0x7f15d40ca5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:35.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.728+0000 7f15db590700 1 -- 172.21.15.7:0/3462933196 shutdown_connections 2024-01-27T02:38:35.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.728+0000 7f15db590700 1 -- 172.21.15.7:0/3462933196 wait complete. 2024-01-27T02:38:35.742 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-27T02:38:35.742 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:38:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.874+0000 7f293dca8700 1 Processor -- start 2024-01-27T02:38:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.874+0000 7f293dca8700 1 -- start start 2024-01-27T02:38:35.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.874+0000 7f293dca8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29380c4cc0 0x7f29380cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:35.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.874+0000 7f293dca8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f29380ca540 0x7f2938163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:35.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.875+0000 7f293dca8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29380ce5e0 con 0x7f293815a100 2024-01-27T02:38:35.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.875+0000 7f293dca8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29380cd9d0 con 0x7f29380ca540 2024-01-27T02:38:35.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.875+0000 7f293dca8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29380cdcb0 con 0x7f29380c4cc0 2024-01-27T02:38:35.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.875+0000 7f29377fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f293815a100 0x7f29380c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50622/0 (socket says 172.21.15.7:50622) 2024-01-27T02:38:35.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.875+0000 7f29377fe700 1 -- 172.21.15.7:0/1582287123 learned_addr learned my addr 172.21.15.7:0/1582287123 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:35.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.875+0000 7f2937fff700 1 --2- 172.21.15.7:0/1582287123 >> v2:172.21.15.7:3300/0 conn(0x7f29380ca540 0x7f2938163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:35.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.875+0000 7f2936ffd700 1 --2- 172.21.15.7:0/1582287123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29380c4cc0 0x7f29380cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.875+0000 7f2937fff700 1 -- 172.21.15.7:0/1582287123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29380c4cc0 msgr2=0x7f29380cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.875+0000 7f2937fff700 1 --2- 172.21.15.7:0/1582287123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29380c4cc0 0x7f29380cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.875+0000 7f2937fff700 1 -- 172.21.15.7:0/1582287123 >> v1:172.21.15.7:6789/0 conn(0x7f293815a100 legacy=0x7f29380c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.875+0000 7f2937fff700 1 -- 172.21.15.7:0/1582287123 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29380cdf90 con 0x7f29380ca540 2024-01-27T02:38:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.875+0000 7f2937fff700 1 --2- 172.21.15.7:0/1582287123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29380ca540 0x7f2938163d90 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2930009e60 tx=0x7f2930009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf40ab7d6099c4a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.875+0000 7f29367fc700 1 -- 172.21.15.7:0/1582287123 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293000e070 con 0x7f29380ca540 2024-01-27T02:38:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.875+0000 7f29367fc700 1 -- 172.21.15.7:0/1582287123 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293000bc30 con 0x7f29380ca540 2024-01-27T02:38:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.875+0000 7f29367fc700 1 -- 172.21.15.7:0/1582287123 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2930012430 con 0x7f29380ca540 2024-01-27T02:38:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.876+0000 7f293dca8700 1 -- 172.21.15.7:0/1582287123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29380ca540 msgr2=0x7f2938163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.876+0000 7f293dca8700 1 --2- 172.21.15.7:0/1582287123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29380ca540 0x7f2938163d90 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2930009e60 tx=0x7f2930009780 comp rx=0 tx=0).stop 2024-01-27T02:38:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.876+0000 7f293dca8700 1 -- 172.21.15.7:0/1582287123 shutdown_connections 2024-01-27T02:38:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.876+0000 7f293dca8700 1 --2- 172.21.15.7:0/1582287123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29380ca540 0x7f2938163d90 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.876+0000 7f293dca8700 1 --2- 172.21.15.7:0/1582287123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29380c4cc0 0x7f29380cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.876+0000 7f293dca8700 1 -- 172.21.15.7:0/1582287123 >> 172.21.15.7:0/1582287123 conn(0x7f2938098020 msgr2=0x7f29380711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.876+0000 7f293dca8700 1 -- 172.21.15.7:0/1582287123 shutdown_connections 2024-01-27T02:38:35.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.876+0000 7f293dca8700 1 -- 172.21.15.7:0/1582287123 wait complete. 2024-01-27T02:38:35.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.878+0000 7f293dca8700 1 Processor -- start 2024-01-27T02:38:35.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.878+0000 7f293dca8700 1 -- start start 2024-01-27T02:38:35.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.878+0000 7f293dca8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29380c4cc0 0x7f2938169e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:35.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.878+0000 7f293dca8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29380ca540 0x7f2938154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:35.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.878+0000 7f293dca8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f293815a100 0x7f293815b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:35.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.878+0000 7f293dca8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293816a360 con 0x7f29380ca540 2024-01-27T02:38:35.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.878+0000 7f293dca8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2938154550 con 0x7f293815a100 2024-01-27T02:38:35.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.878+0000 7f293dca8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293815a9d0 con 0x7f29380c4cc0 2024-01-27T02:38:35.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.879+0000 7f2936ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29380ca540 0x7f2938154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:35.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.879+0000 7f2936ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29380ca540 0x7f2938154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58832/0 (socket says 172.21.15.7:58832) 2024-01-27T02:38:35.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.879+0000 7f2936ffd700 1 -- 172.21.15.7:0/2341250923 learned_addr learned my addr 172.21.15.7:0/2341250923 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:35.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.879+0000 7f29377fe700 1 --2- 172.21.15.7:0/2341250923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29380c4cc0 0x7f2938169e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:35.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.879+0000 7f2937fff700 1 --2- 172.21.15.7:0/2341250923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f293815a100 0x7f293815b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:35.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.879+0000 7f2936ffd700 1 -- 172.21.15.7:0/2341250923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29380c4cc0 msgr2=0x7f2938169e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:35.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.879+0000 7f2936ffd700 1 --2- 172.21.15.7:0/2341250923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29380c4cc0 0x7f2938169e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:35.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.879+0000 7f2936ffd700 1 -- 172.21.15.7:0/2341250923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f293815a100 msgr2=0x7f293815b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:35.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.879+0000 7f2936ffd700 1 --2- 172.21.15.7:0/2341250923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f293815a100 0x7f293815b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:35.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.879+0000 7f2936ffd700 1 -- 172.21.15.7:0/2341250923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2930009a20 con 0x7f29380ca540 2024-01-27T02:38:35.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.879+0000 7f2936ffd700 1 --2- 172.21.15.7:0/2341250923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29380ca540 0x7f2938154010 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f292c00d860 tx=0x7f292c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:35.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.879+0000 7f291f7fe700 1 -- 172.21.15.7:0/2341250923 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c0125f0 con 0x7f29380ca540 2024-01-27T02:38:35.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.879+0000 7f291f7fe700 1 -- 172.21.15.7:0/2341250923 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292c012750 con 0x7f29380ca540 2024-01-27T02:38:35.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.879+0000 7f293dca8700 1 -- 172.21.15.7:0/2341250923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f293815acb0 con 0x7f29380ca540 2024-01-27T02:38:35.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.879+0000 7f291f7fe700 1 -- 172.21.15.7:0/2341250923 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c012b10 con 0x7f29380ca540 2024-01-27T02:38:35.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.879+0000 7f293dca8700 1 -- 172.21.15.7:0/2341250923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f293815b270 con 0x7f29380ca540 2024-01-27T02:38:35.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.880+0000 7f293dca8700 1 -- 172.21.15.7:0/2341250923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f293804c710 con 0x7f29380ca540 2024-01-27T02:38:35.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.881+0000 7f291f7fe700 1 -- 172.21.15.7:0/2341250923 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f292c01e460 con 0x7f29380ca540 2024-01-27T02:38:35.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.881+0000 7f291f7fe700 1 --2- 172.21.15.7:0/2341250923 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2924066f30 0x7f29240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:35.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.881+0000 7f291f7fe700 1 -- 172.21.15.7:0/2341250923 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f292c0849c0 con 0x7f29380ca540 2024-01-27T02:38:35.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.882+0000 7f29377fe700 1 --2- 172.21.15.7:0/2341250923 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2924066f30 0x7f29240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:35.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.882+0000 7f29377fe700 1 --2- 172.21.15.7:0/2341250923 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2924066f30 0x7f29240693e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2928007130 tx=0x7f2928007eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:35.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:35.886+0000 7f291f7fe700 1 -- 172.21.15.7:0/2341250923 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f292c0556c0 con 0x7f29380ca540 2024-01-27T02:38:36.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.088+0000 7f293dca8700 1 -- 172.21.15.7:0/2341250923 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f29380ce270 con 0x7f2924066f30 2024-01-27T02:38:36.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.096+0000 7f291f7fe700 1 -- 172.21.15.7:0/2341250923 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282419 (secure 0 0 0) 0x7f29380ce270 con 0x7f2924066f30 2024-01-27T02:38:36.101 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:36.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.100+0000 7f293dca8700 1 -- 172.21.15.7:0/2341250923 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2924066f30 msgr2=0x7f29240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:36.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.100+0000 7f293dca8700 1 --2- 172.21.15.7:0/2341250923 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2924066f30 0x7f29240693e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2928007130 tx=0x7f2928007eb0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.100+0000 7f293dca8700 1 -- 172.21.15.7:0/2341250923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29380ca540 msgr2=0x7f2938154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:36.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.100+0000 7f293dca8700 1 --2- 172.21.15.7:0/2341250923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29380ca540 0x7f2938154010 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f292c00d860 tx=0x7f292c008040 comp rx=0 tx=0).stop 2024-01-27T02:38:36.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.100+0000 7f293dca8700 1 -- 172.21.15.7:0/2341250923 shutdown_connections 2024-01-27T02:38:36.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.100+0000 7f293dca8700 1 --2- 172.21.15.7:0/2341250923 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2924066f30 0x7f29240693e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.100+0000 7f293dca8700 1 --2- 172.21.15.7:0/2341250923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f293815a100 0x7f293815b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.100+0000 7f293dca8700 1 --2- 172.21.15.7:0/2341250923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29380c4cc0 0x7f2938169e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.100+0000 7f293dca8700 1 --2- 172.21.15.7:0/2341250923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29380ca540 0x7f2938154010 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.100+0000 7f293dca8700 1 -- 172.21.15.7:0/2341250923 >> 172.21.15.7:0/2341250923 conn(0x7f2938098020 msgr2=0x7f29380c9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:36.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.100+0000 7f293dca8700 1 -- 172.21.15.7:0/2341250923 shutdown_connections 2024-01-27T02:38:36.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.100+0000 7f293dca8700 1 -- 172.21.15.7:0/2341250923 wait complete. 2024-01-27T02:38:36.105 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:38:36.115 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-01-27T02:38:35.667278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6694,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555805,"statfs":{"total":1600000000,"available":1593144606,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.780828"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897968+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:32:52.668655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895827+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:08.702500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896528+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:11:34.261074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896262+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:22:55.022123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.899959+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:30:08.176207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896813+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:02:36.439070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897666+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:14:25.858808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896102+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:51.023684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895424+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:57:59.292913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897605+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:55:31.456845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.894631+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:05:45.293447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896477+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:40:57.016595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895636+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:51:24.348901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895665+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:19:55.078411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.902105+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:52:50.516673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897186+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:15:59.844843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895485+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:11:34.261074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895678+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:20:34.579927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897488+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:23:46.870039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893388+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:16:13.624836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893364+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:39:26.847232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.894037+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:23:46.870039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893002+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:53:05.325595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893907+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:05:23.541565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893041+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:18.354062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896299+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:36:18.147300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897512+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:07:33.422861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893752+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:30:10.336341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893836+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:18:05.101790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896393+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:36:36.290157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895868+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:54:13.383710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.901471+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:02:27.766940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896480+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:47:09.052904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.900146+0000","last_change":"2024-01-27T02:38:30.900146+0000","last_active":"2024-01-27T02:38:30.900146+0000","last_peered":"2024-01-27T02:38:30.900146+0000","last_clean":"2024-01-27T02:38:30.900146+0000","last_became_active":"2024-01-27T02:38:30.899920+0000","last_became_peered":"2024-01-27T02:38:30.899920+0000","last_unstale":"2024-01-27T02:38:30.900146+0000","last_undegraded":"2024-01-27T02:38:30.900146+0000","last_fullsized":"2024-01-27T02:38:30.900146+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:02:26.763414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895285+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:01:42.972197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.900419+0000","last_change":"2024-01-27T02:38:30.900419+0000","last_active":"2024-01-27T02:38:30.900419+0000","last_peered":"2024-01-27T02:38:30.900419+0000","last_clean":"2024-01-27T02:38:30.900419+0000","last_became_active":"2024-01-27T02:38:30.900188+0000","last_became_peered":"2024-01-27T02:38:30.900188+0000","last_unstale":"2024-01-27T02:38:30.900419+0000","last_undegraded":"2024-01-27T02:38:30.900419+0000","last_fullsized":"2024-01-27T02:38:30.900419+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:16:12.621310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896381+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:32:20.165836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.907104+0000","last_change":"2024-01-27T02:38:30.907104+0000","last_active":"2024-01-27T02:38:30.907104+0000","last_peered":"2024-01-27T02:38:30.907104+0000","last_clean":"2024-01-27T02:38:30.907104+0000","last_became_active":"2024-01-27T02:38:30.906807+0000","last_became_peered":"2024-01-27T02:38:30.906807+0000","last_unstale":"2024-01-27T02:38:30.907104+0000","last_undegraded":"2024-01-27T02:38:30.907104+0000","last_fullsized":"2024-01-27T02:38:30.907104+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:16:05.064427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895895+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:05:45.293447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.906090+0000","last_change":"2024-01-27T02:38:30.906090+0000","last_active":"2024-01-27T02:38:30.906090+0000","last_peered":"2024-01-27T02:38:30.906090+0000","last_clean":"2024-01-27T02:38:30.906090+0000","last_became_active":"2024-01-27T02:38:30.905815+0000","last_became_peered":"2024-01-27T02:38:30.905815+0000","last_unstale":"2024-01-27T02:38:30.906090+0000","last_undegraded":"2024-01-27T02:38:30.906090+0000","last_fullsized":"2024-01-27T02:38:30.906090+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:42:16.605919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895621+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:00:23.173837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.908846+0000","last_change":"2024-01-27T02:38:30.908846+0000","last_active":"2024-01-27T02:38:30.908846+0000","last_peered":"2024-01-27T02:38:30.908846+0000","last_clean":"2024-01-27T02:38:30.908846+0000","last_became_active":"2024-01-27T02:38:30.908634+0000","last_became_peered":"2024-01-27T02:38:30.908634+0000","last_unstale":"2024-01-27T02:38:30.908846+0000","last_undegraded":"2024-01-27T02:38:30.908846+0000","last_fullsized":"2024-01-27T02:38:30.908846+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:44:21.414722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895115+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:01:42.972197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.897433+0000","last_change":"2024-01-27T02:38:30.897433+0000","last_active":"2024-01-27T02:38:30.897433+0000","last_peered":"2024-01-27T02:38:30.897433+0000","last_clean":"2024-01-27T02:38:30.897433+0000","last_became_active":"2024-01-27T02:38:30.897134+0000","last_became_peered":"2024-01-27T02:38:30.897134+0000","last_unstale":"2024-01-27T02:38:30.897433+0000","last_undegraded":"2024-01-27T02:38:30.897433+0000","last_fullsized":"2024-01-27T02:38:30.897433+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:18:04.098263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897157+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:32:05.199937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.903018+0000","last_change":"2024-01-27T02:38:30.903018+0000","last_active":"2024-01-27T02:38:30.903018+0000","last_peered":"2024-01-27T02:38:30.903018+0000","last_clean":"2024-01-27T02:38:30.903018+0000","last_became_active":"2024-01-27T02:38:30.902748+0000","last_became_peered":"2024-01-27T02:38:30.902748+0000","last_unstale":"2024-01-27T02:38:30.903018+0000","last_undegraded":"2024-01-27T02:38:30.903018+0000","last_fullsized":"2024-01-27T02:38:30.903018+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:14:24.855281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896050+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:56:12.566293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.903329+0000","last_change":"2024-01-27T02:38:30.903329+0000","last_active":"2024-01-27T02:38:30.903329+0000","last_peered":"2024-01-27T02:38:30.903329+0000","last_clean":"2024-01-27T02:38:30.903329+0000","last_became_active":"2024-01-27T02:38:30.903084+0000","last_became_peered":"2024-01-27T02:38:30.903084+0000","last_unstale":"2024-01-27T02:38:30.903329+0000","last_undegraded":"2024-01-27T02:38:30.903329+0000","last_fullsized":"2024-01-27T02:38:30.903329+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:11:33.257548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896315+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:36:50.676219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.904630+0000","last_change":"2024-01-27T02:38:30.904630+0000","last_active":"2024-01-27T02:38:30.904630+0000","last_peered":"2024-01-27T02:38:30.904630+0000","last_clean":"2024-01-27T02:38:30.904630+0000","last_became_active":"2024-01-27T02:38:30.904406+0000","last_became_peered":"2024-01-27T02:38:30.904406+0000","last_unstale":"2024-01-27T02:38:30.904630+0000","last_undegraded":"2024-01-27T02:38:30.904630+0000","last_fullsized":"2024-01-27T02:38:30.904630+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:22:44.330464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893390+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:05:45.293447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.893083+0000","last_change":"2024-01-27T02:38:30.893083+0000","last_active":"2024-01-27T02:38:30.893083+0000","last_peered":"2024-01-27T02:38:30.893083+0000","last_clean":"2024-01-27T02:38:30.893083+0000","last_became_active":"2024-01-27T02:38:30.892801+0000","last_became_peered":"2024-01-27T02:38:30.892801+0000","last_unstale":"2024-01-27T02:38:30.893083+0000","last_undegraded":"2024-01-27T02:38:30.893083+0000","last_fullsized":"2024-01-27T02:38:30.893083+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:51:23.345374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892794+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:08.702500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.905086+0000","last_change":"2024-01-27T02:38:30.905086+0000","last_active":"2024-01-27T02:38:30.905086+0000","last_peered":"2024-01-27T02:38:30.905086+0000","last_clean":"2024-01-27T02:38:30.905086+0000","last_became_active":"2024-01-27T02:38:30.904853+0000","last_became_peered":"2024-01-27T02:38:30.904853+0000","last_unstale":"2024-01-27T02:38:30.905086+0000","last_undegraded":"2024-01-27T02:38:30.905086+0000","last_fullsized":"2024-01-27T02:38:30.905086+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:54:14.886540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896562+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:03:26.614534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.905975+0000","last_change":"2024-01-27T02:38:30.905975+0000","last_active":"2024-01-27T02:38:30.905975+0000","last_peered":"2024-01-27T02:38:30.905975+0000","last_clean":"2024-01-27T02:38:30.905975+0000","last_became_active":"2024-01-27T02:38:30.904239+0000","last_became_peered":"2024-01-27T02:38:30.904239+0000","last_unstale":"2024-01-27T02:38:30.905975+0000","last_undegraded":"2024-01-27T02:38:30.905975+0000","last_fullsized":"2024-01-27T02:38:30.905975+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:05:22.538039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892200+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:05:45.293447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.904960+0000","last_change":"2024-01-27T02:38:30.904960+0000","last_active":"2024-01-27T02:38:30.904960+0000","last_peered":"2024-01-27T02:38:30.904960+0000","last_clean":"2024-01-27T02:38:30.904960+0000","last_became_active":"2024-01-27T02:38:30.904479+0000","last_became_peered":"2024-01-27T02:38:30.904479+0000","last_unstale":"2024-01-27T02:38:30.904960+0000","last_undegraded":"2024-01-27T02:38:30.904960+0000","last_fullsized":"2024-01-27T02:38:30.904960+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:20:17.775604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892827+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:11:34.261074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.901125+0000","last_change":"2024-01-27T02:38:30.901125+0000","last_active":"2024-01-27T02:38:30.901125+0000","last_peered":"2024-01-27T02:38:30.901125+0000","last_clean":"2024-01-27T02:38:30.901125+0000","last_became_active":"2024-01-27T02:38:30.900811+0000","last_became_peered":"2024-01-27T02:38:30.900811+0000","last_unstale":"2024-01-27T02:38:30.901125+0000","last_undegraded":"2024-01-27T02:38:30.901125+0000","last_fullsized":"2024-01-27T02:38:30.901125+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:18:04.098263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892350+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:20:34.579927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.904062+0000","last_change":"2024-01-27T02:38:30.904062+0000","last_active":"2024-01-27T02:38:30.904062+0000","last_peered":"2024-01-27T02:38:30.904062+0000","last_clean":"2024-01-27T02:38:30.904062+0000","last_became_active":"2024-01-27T02:38:30.903827+0000","last_became_peered":"2024-01-27T02:38:30.903827+0000","last_unstale":"2024-01-27T02:38:30.904062+0000","last_undegraded":"2024-01-27T02:38:30.904062+0000","last_fullsized":"2024-01-27T02:38:30.904062+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:22:30.041364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.900402+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:59:45.149668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.901317+0000","last_change":"2024-01-27T02:38:30.901317+0000","last_active":"2024-01-27T02:38:30.901317+0000","last_peered":"2024-01-27T02:38:30.901317+0000","last_clean":"2024-01-27T02:38:30.901317+0000","last_became_active":"2024-01-27T02:38:30.900905+0000","last_became_peered":"2024-01-27T02:38:30.900905+0000","last_unstale":"2024-01-27T02:38:30.901317+0000","last_undegraded":"2024-01-27T02:38:30.901317+0000","last_fullsized":"2024-01-27T02:38:30.901317+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:23:45.866513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895981+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:48:06.641759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.901405+0000","last_change":"2024-01-27T02:38:30.901405+0000","last_active":"2024-01-27T02:38:30.901405+0000","last_peered":"2024-01-27T02:38:30.901405+0000","last_clean":"2024-01-27T02:38:30.901405+0000","last_became_active":"2024-01-27T02:38:30.901168+0000","last_became_peered":"2024-01-27T02:38:30.901168+0000","last_unstale":"2024-01-27T02:38:30.901405+0000","last_undegraded":"2024-01-27T02:38:30.901405+0000","last_fullsized":"2024-01-27T02:38:30.901405+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:52:03.557302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896480+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:20:34.579927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.894794+0000","last_change":"2024-01-27T02:38:30.894794+0000","last_active":"2024-01-27T02:38:30.894794+0000","last_peered":"2024-01-27T02:38:30.894794+0000","last_clean":"2024-01-27T02:38:30.894794+0000","last_became_active":"2024-01-27T02:38:30.894519+0000","last_became_peered":"2024-01-27T02:38:30.894519+0000","last_unstale":"2024-01-27T02:38:30.894794+0000","last_undegraded":"2024-01-27T02:38:30.894794+0000","last_fullsized":"2024-01-27T02:38:30.894794+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:39:25.843705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896689+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:30:10.336341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.901197+0000","last_change":"2024-01-27T02:38:30.901197+0000","last_active":"2024-01-27T02:38:30.901197+0000","last_peered":"2024-01-27T02:38:30.901197+0000","last_clean":"2024-01-27T02:38:30.901197+0000","last_became_active":"2024-01-27T02:38:30.900235+0000","last_became_peered":"2024-01-27T02:38:30.900235+0000","last_unstale":"2024-01-27T02:38:30.901197+0000","last_undegraded":"2024-01-27T02:38:30.901197+0000","last_fullsized":"2024-01-27T02:38:30.901197+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:07:32.419335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897132+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:05:23.541565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.904351+0000","last_change":"2024-01-27T02:38:30.904351+0000","last_active":"2024-01-27T02:38:30.904351+0000","last_peered":"2024-01-27T02:38:30.904351+0000","last_clean":"2024-01-27T02:38:30.904351+0000","last_became_active":"2024-01-27T02:38:30.904118+0000","last_became_peered":"2024-01-27T02:38:30.904118+0000","last_unstale":"2024-01-27T02:38:30.904351+0000","last_undegraded":"2024-01-27T02:38:30.904351+0000","last_fullsized":"2024-01-27T02:38:30.904351+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:53:19.168298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895194+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:07.978917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.904812+0000","last_change":"2024-01-27T02:38:30.904812+0000","last_active":"2024-01-27T02:38:30.904812+0000","last_peered":"2024-01-27T02:38:30.904812+0000","last_clean":"2024-01-27T02:38:30.904812+0000","last_became_active":"2024-01-27T02:38:30.904483+0000","last_became_peered":"2024-01-27T02:38:30.904483+0000","last_unstale":"2024-01-27T02:38:30.904812+0000","last_undegraded":"2024-01-27T02:38:30.904812+0000","last_fullsized":"2024-01-27T02:38:30.904812+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:45:43.930079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896313+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:01:42.972197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.904780+0000","last_change":"2024-01-27T02:38:30.904780+0000","last_active":"2024-01-27T02:38:30.904780+0000","last_peered":"2024-01-27T02:38:30.904780+0000","last_clean":"2024-01-27T02:38:30.904780+0000","last_became_active":"2024-01-27T02:38:30.904179+0000","last_became_peered":"2024-01-27T02:38:30.904179+0000","last_unstale":"2024-01-27T02:38:30.904780+0000","last_undegraded":"2024-01-27T02:38:30.904780+0000","last_fullsized":"2024-01-27T02:38:30.904780+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:14:13.119229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892253+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:51.023684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.899295+0000","last_change":"2024-01-27T02:38:30.899295+0000","last_active":"2024-01-27T02:38:30.899295+0000","last_peered":"2024-01-27T02:38:30.899295+0000","last_clean":"2024-01-27T02:38:30.899295+0000","last_became_active":"2024-01-27T02:38:30.899015+0000","last_became_peered":"2024-01-27T02:38:30.899015+0000","last_unstale":"2024-01-27T02:38:30.899295+0000","last_undegraded":"2024-01-27T02:38:30.899295+0000","last_fullsized":"2024-01-27T02:38:30.899295+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:34:25.075555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893779+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:51.023684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.902450+0000","last_change":"2024-01-27T02:38:30.902450+0000","last_active":"2024-01-27T02:38:30.902450+0000","last_peered":"2024-01-27T02:38:30.902450+0000","last_clean":"2024-01-27T02:38:30.902450+0000","last_became_active":"2024-01-27T02:38:30.902130+0000","last_became_peered":"2024-01-27T02:38:30.902130+0000","last_unstale":"2024-01-27T02:38:30.902450+0000","last_undegraded":"2024-01-27T02:38:30.902450+0000","last_fullsized":"2024-01-27T02:38:30.902450+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:39:25.843705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.894773+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:40:17.611985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.901304+0000","last_change":"2024-01-27T02:38:30.901304+0000","last_active":"2024-01-27T02:38:30.901304+0000","last_peered":"2024-01-27T02:38:30.901304+0000","last_clean":"2024-01-27T02:38:30.901304+0000","last_became_active":"2024-01-27T02:38:30.901065+0000","last_became_peered":"2024-01-27T02:38:30.901065+0000","last_unstale":"2024-01-27T02:38:30.901304+0000","last_undegraded":"2024-01-27T02:38:30.901304+0000","last_fullsized":"2024-01-27T02:38:30.901304+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:16:12.621310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.894797+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:02:03.382943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.901161+0000","last_change":"2024-01-27T02:38:30.901161+0000","last_active":"2024-01-27T02:38:30.901161+0000","last_peered":"2024-01-27T02:38:30.901161+0000","last_clean":"2024-01-27T02:38:30.901161+0000","last_became_active":"2024-01-27T02:38:30.899610+0000","last_became_peered":"2024-01-27T02:38:30.899610+0000","last_unstale":"2024-01-27T02:38:30.901161+0000","last_undegraded":"2024-01-27T02:38:30.901161+0000","last_fullsized":"2024-01-27T02:38:30.901161+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:30:09.332815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892858+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:19:55.078411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.904612+0000","last_change":"2024-01-27T02:38:30.904612+0000","last_active":"2024-01-27T02:38:30.904612+0000","last_peered":"2024-01-27T02:38:30.904612+0000","last_clean":"2024-01-27T02:38:30.904612+0000","last_became_active":"2024-01-27T02:38:30.904299+0000","last_became_peered":"2024-01-27T02:38:30.904299+0000","last_unstale":"2024-01-27T02:38:30.904612+0000","last_undegraded":"2024-01-27T02:38:30.904612+0000","last_fullsized":"2024-01-27T02:38:30.904612+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:16:12.621310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.894726+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:19:55.078411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.899705+0000","last_change":"2024-01-27T02:38:30.899705+0000","last_active":"2024-01-27T02:38:30.899705+0000","last_peered":"2024-01-27T02:38:30.899705+0000","last_clean":"2024-01-27T02:38:30.899705+0000","last_became_active":"2024-01-27T02:38:30.899453+0000","last_became_peered":"2024-01-27T02:38:30.899453+0000","last_unstale":"2024-01-27T02:38:30.899705+0000","last_undegraded":"2024-01-27T02:38:30.899705+0000","last_fullsized":"2024-01-27T02:38:30.899705+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:02:35.435543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892871+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:18:12.582908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.902907+0000","last_change":"2024-01-27T02:38:30.902907+0000","last_active":"2024-01-27T02:38:30.902907+0000","last_peered":"2024-01-27T02:38:30.902907+0000","last_clean":"2024-01-27T02:38:30.902907+0000","last_became_active":"2024-01-27T02:38:30.902691+0000","last_became_peered":"2024-01-27T02:38:30.902691+0000","last_unstale":"2024-01-27T02:38:30.902907+0000","last_undegraded":"2024-01-27T02:38:30.902907+0000","last_fullsized":"2024-01-27T02:38:30.902907+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:25:13.514723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896780+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:08.702500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.906486+0000","last_change":"2024-01-27T02:38:30.906486+0000","last_active":"2024-01-27T02:38:30.906486+0000","last_peered":"2024-01-27T02:38:30.906486+0000","last_clean":"2024-01-27T02:38:30.906486+0000","last_became_active":"2024-01-27T02:38:30.906233+0000","last_became_peered":"2024-01-27T02:38:30.906233+0000","last_unstale":"2024-01-27T02:38:30.906486+0000","last_undegraded":"2024-01-27T02:38:30.906486+0000","last_fullsized":"2024-01-27T02:38:30.906486+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:27:08.109911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.894650+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:11:36.249589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.897972+0000","last_change":"2024-01-27T02:38:30.897972+0000","last_active":"2024-01-27T02:38:30.897972+0000","last_peered":"2024-01-27T02:38:30.897972+0000","last_clean":"2024-01-27T02:38:30.897972+0000","last_became_active":"2024-01-27T02:38:30.897716+0000","last_became_peered":"2024-01-27T02:38:30.897716+0000","last_unstale":"2024-01-27T02:38:30.897972+0000","last_undegraded":"2024-01-27T02:38:30.897972+0000","last_fullsized":"2024-01-27T02:38:30.897972+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:07.698973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892275+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:01:42.972197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.901349+0000","last_change":"2024-01-27T02:38:30.901349+0000","last_active":"2024-01-27T02:38:30.901349+0000","last_peered":"2024-01-27T02:38:30.901349+0000","last_clean":"2024-01-27T02:38:30.901349+0000","last_became_active":"2024-01-27T02:38:30.900973+0000","last_became_peered":"2024-01-27T02:38:30.900973+0000","last_unstale":"2024-01-27T02:38:30.901349+0000","last_undegraded":"2024-01-27T02:38:30.901349+0000","last_fullsized":"2024-01-27T02:38:30.901349+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:55:50.241360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":257,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-01-27T02:38:29.888544+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:38:29.888544+0000","last_peered":"2024-01-27T02:38:29.888544+0000","last_clean":"2024-01-27T02:38:29.888544+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:38:29.888544+0000","last_undegraded":"2024-01-27T02:38:29.888544+0000","last_fullsized":"2024-01-27T02:38:29.888544+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574929,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1270,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194041,"statfs":{"total":200000000,"available":198698734,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77400000000000002,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70299999999999996,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.60999999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.71999999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.42799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69999999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.65600000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.68200000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71399999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.63100000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.73099999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.48199999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.71799999999999997,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.71499999999999997}]},{"osd":4,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.47599999999999998,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.51100000000000001},"min":{"1min":0.33700000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.50600000000000001}]},{"osd":5,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51300000000000001,"15min":0.53400000000000003},"min":{"1min":0.35999999999999999,"5min":0.32100000000000001,"15min":0.307},"max":{"1min":0.68500000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.56299999999999994},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75800000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.73099999999999998}]},{"osd":6,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57199999999999995,"15min":0.59399999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.78800000000000003,"5min":0.80300000000000005,"15min":0.81299999999999994},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56100000000000005,"15min":0.57499999999999996},"min":{"1min":0.42499999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.73099999999999998,"5min":0.81599999999999995,"15min":0.90800000000000003},"last":0.70299999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574929,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":688,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194624,"statfs":{"total":200000000,"available":199295365,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.64900000000000002,"15min":0.54200000000000004},"min":{"1min":0.442,"5min":0.36499999999999999,"15min":0.29999999999999999},"max":{"1min":4.125,"5min":4.125,"15min":4.125},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60599999999999998,"15min":0.53600000000000003},"min":{"1min":0.45100000000000001,"5min":0.35699999999999998,"15min":0.32700000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.61099999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.68400000000000005,"15min":0.55200000000000005},"min":{"1min":0.433,"5min":0.41999999999999998,"15min":0.34100000000000003},"max":{"1min":8.2699999999999996,"5min":8.2699999999999996,"15min":8.2699999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":2.0409999999999999,"5min":0.90800000000000003,"15min":0.65600000000000003},"min":{"1min":0.46999999999999997,"5min":0.379,"15min":0.33800000000000002},"max":{"1min":26.411999999999999,"5min":26.411999999999999,"15min":26.411999999999999},"last":0.53500000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":0.77700000000000002,"15min":0.60899999999999999},"min":{"1min":0.45100000000000001,"5min":0.41199999999999998,"15min":0.39600000000000002},"max":{"1min":17.143999999999998,"5min":17.143999999999998,"15min":17.143999999999998},"last":0.435},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.64900000000000002,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":0.55900000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.63300000000000001,"15min":0.56799999999999995},"min":{"1min":0.45800000000000002,"5min":0.435,"15min":0.376},"max":{"1min":4.3689999999999998,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64100000000000001,"15min":0.58099999999999996},"min":{"1min":0.433,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.58499999999999996}]},{"osd":4,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.33200000000000002,"5min":0.32000000000000001,"15min":0.30199999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51600000000000001,"15min":0.54900000000000004},"min":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74299999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.38900000000000001}]},{"osd":5,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.52100000000000002,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76100000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56000000000000005,"15min":0.59999999999999998},"min":{"1min":0.41099999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77000000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.46100000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.58999999999999997,"15min":0.61099999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79400000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55800000000000005,"15min":0.58499999999999996},"min":{"1min":0.40000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71499999999999997,"5min":1.038,"15min":1.038},"last":0.5}]}]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":688,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194624,"statfs":{"total":200000000,"available":199295365,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.39700000000000002,"15min":0.377},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25800000000000001},"max":{"1min":0.50600000000000001,"5min":0.873,"15min":0.873},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.41899999999999998,"15min":0.39900000000000002},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.53100000000000003,"5min":1.121,"15min":1.121},"last":0.31900000000000001}]},{"osd":2,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.40999999999999998,"15min":0.36899999999999999},"min":{"1min":0.27200000000000002,"5min":0.26600000000000001,"15min":0.23999999999999999},"max":{"1min":0.48599999999999999,"5min":1.25,"15min":1.25},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.45600000000000002,"15min":0.432},"min":{"1min":0.32000000000000001,"5min":0.30199999999999999,"15min":0.29899999999999999},"max":{"1min":0.58599999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.44600000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.49399999999999999,"15min":0.46999999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.32100000000000001},"max":{"1min":0.66800000000000004,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.46999999999999997,"15min":0.45600000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.58199999999999996,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.38400000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.66300000000000003,"15min":0.63500000000000001},"min":{"1min":0.501,"5min":0.373,"15min":0.373},"max":{"1min":0.81599999999999995,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66200000000000003,"15min":0.63800000000000001},"min":{"1min":0.497,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.82599999999999996,"5min":1.026,"15min":1.026},"last":0.84999999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.68899999999999995,"15min":0.63900000000000001},"min":{"1min":0.51000000000000001,"5min":0.46200000000000002,"15min":0.44},"max":{"1min":0.80900000000000005,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.752},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69099999999999995,"15min":0.67600000000000005},"min":{"1min":0.53000000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83799999999999997,"5min":3.181,"15min":3.181},"last":0.89600000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71199999999999997,"15min":0.69699999999999995},"min":{"1min":0.51400000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.875,"5min":1.024,"15min":1.024},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71499999999999997,"15min":0.68000000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.85699999999999998,"5min":1.869,"15min":1.869},"last":0.86599999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.72699999999999998,"15min":0.68700000000000006},"min":{"1min":0.55800000000000005,"5min":0.495,"15min":0.47299999999999998},"max":{"1min":0.82599999999999996,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.73099999999999998,"15min":0.69399999999999995},"min":{"1min":0.56000000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.86799999999999999,"5min":1.708,"15min":1.708},"last":0.79500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1270,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194041,"statfs":{"total":200000000,"available":198698734,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39000000000000001},"min":{"1min":0.253,"5min":0.24399999999999999,"15min":0.23799999999999999},"max":{"1min":0.52100000000000002,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.41999999999999998,"15min":0.41599999999999998},"min":{"1min":0.29599999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.64200000000000002,"5min":0.65200000000000002,"15min":0.68400000000000005},"last":0.60399999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.437,"15min":0.42199999999999999},"min":{"1min":0.30499999999999999,"5min":0.28699999999999998,"15min":0.28199999999999997},"max":{"1min":0.60299999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.435},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.44900000000000001,"15min":0.435},"min":{"1min":0.32400000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.628,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.47999999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42799999999999999,"15min":0.43099999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58199999999999996,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.629},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48499999999999999,"15min":0.47499999999999998},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69399999999999995,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.55900000000000005}]},{"osd":4,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65900000000000003,"15min":0.64100000000000001},"min":{"1min":0.49399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91700000000000004,"5min":0.997,"15min":0.997},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.44700000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92100000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.67300000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67700000000000005,"15min":0.65700000000000003},"min":{"1min":0.496,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88600000000000001,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.68100000000000005},"min":{"1min":0.53100000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.82099999999999995,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.748}]},{"osd":6,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.57299999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.80300000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69599999999999995,"15min":0.68000000000000005},"min":{"1min":0.51400000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94299999999999995,"5min":1.04,"15min":1.04},"last":0.65800000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68899999999999995,"15min":0.69199999999999995},"min":{"1min":0.45900000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90800000000000003,"5min":1.014,"15min":1.014},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68700000000000006,"15min":0.69299999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.83899999999999997,"5min":0.95799999999999996,"15min":0.97299999999999998},"last":0.72999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":694,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194618,"statfs":{"total":200000000,"available":199289102,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.41399999999999998,"15min":0.40899999999999997},"min":{"1min":0.23599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.58099999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.308},{"interface":"front","average":{"1min":0.443,"5min":0.45700000000000002,"15min":0.62},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.30499999999999999},"max":{"1min":0.64700000000000002,"5min":0.72799999999999998,"15min":6.2729999999999997},"last":0.38200000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.42999999999999999,"15min":0.42299999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.71899999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.433,"15min":0.44500000000000001},"min":{"1min":0.32600000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.57999999999999996,"5min":0.65300000000000002,"15min":0.70099999999999996},"last":0.42599999999999999}]},{"osd":3,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42399999999999999,"15min":0.41599999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.27800000000000002},"max":{"1min":0.64700000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.372},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.48699999999999999,"15min":0.48499999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.627,"5min":0.76500000000000001,"15min":0.80800000000000005},"last":0.40999999999999998}]},{"osd":4,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68799999999999994,"15min":0.65800000000000003},"min":{"1min":0.46800000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93999999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70599999999999996,"15min":0.79000000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.006,"5min":1.006,"15min":3.581},"last":0.64400000000000002}]},{"osd":5,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65300000000000002,"15min":0.73499999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":3.5369999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92200000000000004,"5min":0.96599999999999997,"15min":1.0529999999999999},"last":0.52200000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.45000000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.51300000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97499999999999998,"5min":1.0189999999999999,"15min":1.0209999999999999},"last":0.503}]},{"osd":7,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.68300000000000005,"15min":0.67800000000000005},"min":{"1min":0.50800000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95599999999999996,"5min":1.03,"15min":1.03},"last":0.62},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.69799999999999995,"15min":0.68999999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.42699999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.55200000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":694,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194618,"statfs":{"total":200000000,"available":199289102,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.45600000000000002,"15min":0.38200000000000001},"min":{"1min":0.28999999999999998,"5min":0.253,"15min":0.253},"max":{"1min":4.7380000000000004,"5min":4.7380000000000004,"15min":4.7380000000000004},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.41599999999999998,"15min":0.39800000000000002},"min":{"1min":0.27600000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.35399999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.47199999999999998,"15min":0.41199999999999998},"min":{"1min":0.27900000000000003,"5min":0.27400000000000002,"15min":0.22900000000000001},"max":{"1min":4.5739999999999998,"5min":4.5739999999999998,"15min":4.5739999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.52400000000000002,"15min":0.44700000000000001},"min":{"1min":0.311,"5min":0.30199999999999999,"15min":0.27900000000000003},"max":{"1min":5.0529999999999999,"5min":5.0529999999999999,"15min":5.0529999999999999},"last":0.39700000000000002}]},{"osd":2,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.48499999999999999,"15min":0.439},"min":{"1min":0.34799999999999998,"5min":0.33600000000000002,"15min":0.28100000000000003},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.55500000000000005,"15min":0.47399999999999998},"min":{"1min":0.375,"5min":0.35299999999999998,"15min":0.34599999999999997},"max":{"1min":5.1879999999999997,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":0.432}]},{"osd":4,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.69399999999999995,"15min":0.63600000000000001},"min":{"1min":0.42799999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0629999999999999,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66500000000000004,"15min":0.61399999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.79300000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.67800000000000005,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.006,"5min":0.78500000000000003,"15min":0.70299999999999996},"min":{"1min":0.56200000000000006,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":4.7960000000000003,"5min":4.7960000000000003,"15min":4.7960000000000003},"last":0.68400000000000005}]},{"osd":6,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.73499999999999999,"15min":0.67800000000000005},"min":{"1min":0.58899999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":5.0069999999999997,"5min":5.0069999999999997,"15min":5.0069999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.68100000000000005,"15min":0.65400000000000003},"min":{"1min":0.504,"5min":0.435,"15min":0.435},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.74399999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.74299999999999999,"15min":0.66300000000000003},"min":{"1min":0.57699999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.139,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.71299999999999997,"15min":0.67600000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.81299999999999994}]}]},{"osd":4,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":694,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194618,"statfs":{"total":200000000,"available":199289102,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.34599999999999997,"5min":0.33400000000000002,"15min":0.314},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.64600000000000002,"15min":0.60999999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":7.0670000000000002,"5min":7.0670000000000002,"15min":7.0670000000000002},"last":0.76000000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.64700000000000002,"15min":0.60899999999999999},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":5.1529999999999996,"5min":5.1529999999999996,"15min":5.1529999999999996},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.626,"15min":0.60399999999999998},"min":{"1min":0.43099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.0419999999999998,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.751}]},{"osd":2,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.64900000000000002,"15min":0.60999999999999999},"min":{"1min":0.46899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":5.0019999999999998,"5min":5.0019999999999998,"15min":5.0019999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.67600000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.64700000000000002,"15min":0.60099999999999998},"min":{"1min":0.42099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.7220000000000004,"5min":6.7220000000000004,"15min":6.7220000000000004},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.51000000000000001,"5min":0.44900000000000001,"15min":0.41399999999999998},"max":{"1min":6.5510000000000002,"5min":6.5510000000000002,"15min":6.5510000000000002},"last":0.73199999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.503},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.56200000000000006,"15min":0.51400000000000001},"min":{"1min":0.38500000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":6.9269999999999996,"5min":6.9269999999999996,"15min":6.9269999999999996},"last":0.57099999999999995}]},{"osd":6,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.55400000000000005,"15min":0.51700000000000002},"min":{"1min":0.40100000000000002,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":5.2229999999999999,"5min":5.2229999999999999,"15min":5.2229999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.71099999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.60799999999999998,"15min":0.57199999999999995},"min":{"1min":0.34799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":7.5339999999999998,"5min":7.5339999999999998,"15min":7.5339999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.63200000000000001,"15min":0.58999999999999997},"min":{"1min":0.44600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.9550000000000001,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":0.65900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574929,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":694,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194618,"statfs":{"total":200000000,"available":199289102,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56100000000000005,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.84199999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60599999999999998,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.34499999999999997,"15min":0.27600000000000002},"max":{"1min":0.78900000000000003,"5min":3.242,"15min":3.242},"last":0.69499999999999995}]},{"osd":1,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63,"15min":0.60999999999999999},"min":{"1min":0.47299999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80100000000000005,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.62,"15min":0.64700000000000002},"min":{"1min":0.46899999999999997,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77600000000000002,"5min":0.88500000000000001,"15min":0.95499999999999996},"last":0.63700000000000001}]},{"osd":2,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61699999999999999,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.372},"max":{"1min":0.80900000000000005,"5min":3.04,"15min":3.04},"last":0.62},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59599999999999997,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74299999999999999,"5min":0.85899999999999999,"15min":0.89400000000000002},"last":0.59999999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65000000000000002,"15min":0.67300000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82099999999999995,"5min":4.032,"15min":4.032},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.70599999999999996,"15min":0.67900000000000005},"min":{"1min":0.46000000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.78800000000000003,"5min":4.282,"15min":4.282},"last":0.70899999999999996}]},{"osd":4,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.371,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.747,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.499,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76300000000000001,"5min":0.79800000000000004,"15min":1.101},"last":0.51100000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.502,"15min":0.55200000000000005},"min":{"1min":0.38600000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69199999999999995,"5min":0.73199999999999998,"15min":1.7969999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.58099999999999996},"min":{"1min":0.433,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83399999999999996,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.53100000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55100000000000005,"15min":0.56599999999999995},"min":{"1min":0.45000000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68899999999999995,"5min":0.91100000000000003,"15min":1.008},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.61499999999999999,"15min":0.65400000000000003},"min":{"1min":0.40699999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78400000000000003,"5min":2,"15min":2},"last":0.67000000000000004}]}]}],"pool_statfs":[]}} 2024-01-27T02:38:36.120 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:38:36.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.289+0000 7f5437571700 1 Processor -- start 2024-01-27T02:38:36.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.290+0000 7f5437571700 1 -- start start 2024-01-27T02:38:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.290+0000 7f5437571700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54300b1900 0x7f54300b0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.290+0000 7f5437571700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f543015c1e0 0x7f54300c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f5437571700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54300b1470 con 0x7f543015be10 2024-01-27T02:38:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f5437571700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54300b1d50 con 0x7f543015c1e0 2024-01-27T02:38:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f5437571700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54300b0860 con 0x7f54300b1900 2024-01-27T02:38:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f543530d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f543015be10 0x7f54300aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50624/0 (socket says 172.21.15.7:50624) 2024-01-27T02:38:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f543530d700 1 -- 172.21.15.7:0/436609067 learned_addr learned my addr 172.21.15.7:0/436609067 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f5435b0e700 1 --2- 172.21.15.7:0/436609067 >> v2:172.21.15.188:3300/0 conn(0x7f543015c1e0 0x7f54300c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f5434b0c700 1 --2- 172.21.15.7:0/436609067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54300b1900 0x7f54300b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f5427fff700 1 -- 172.21.15.7:0/436609067 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609540599 0 0) 0x7f54300b1470 con 0x7f543015be10 2024-01-27T02:38:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f5427fff700 1 -- 172.21.15.7:0/436609067 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f541c003570 con 0x7f543015be10 2024-01-27T02:38:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f5427fff700 1 -- 172.21.15.7:0/436609067 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2472109104 0 0) 0x7f541c003570 con 0x7f543015be10 2024-01-27T02:38:36.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f5427fff700 1 -- 172.21.15.7:0/436609067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54300b1900 msgr2=0x7f54300b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f5427fff700 1 --2- 172.21.15.7:0/436609067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54300b1900 0x7f54300b0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f5427fff700 1 -- 172.21.15.7:0/436609067 >> v2:172.21.15.188:3300/0 conn(0x7f543015c1e0 msgr2=0x7f54300c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f5427fff700 1 --2- 172.21.15.7:0/436609067 >> v2:172.21.15.188:3300/0 conn(0x7f543015c1e0 0x7f54300c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f5427fff700 1 -- 172.21.15.7:0/436609067 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54300b0b40 con 0x7f543015be10 2024-01-27T02:38:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f5427fff700 1 -- 172.21.15.7:0/436609067 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f542c001470 con 0x7f543015be10 2024-01-27T02:38:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f5427fff700 1 -- 172.21.15.7:0/436609067 >> v1:172.21.15.7:6789/0 conn(0x7f543015be10 legacy=0x7f54300aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f5427fff700 1 --2- 172.21.15.7:0/436609067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f541c003a60 0x7f541c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f5427fff700 1 --2- 172.21.15.7:0/436609067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f541c004810 0x7f541c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f5427fff700 1 --2- 172.21.15.7:0/436609067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54300b1900 0x7f541c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f5427fff700 1 -- 172.21.15.7:0/436609067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54300b1d50 con 0x7f541c004810 2024-01-27T02:38:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f5427fff700 1 -- 172.21.15.7:0/436609067 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54300b0860 con 0x7f541c003a60 2024-01-27T02:38:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.291+0000 7f5427fff700 1 -- 172.21.15.7:0/436609067 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541c00ba60 con 0x7f54300b1900 2024-01-27T02:38:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.292+0000 7f5434b0c700 1 --2- 172.21.15.7:0/436609067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f541c004810 0x7f541c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.292+0000 7f543530d700 1 --2- 172.21.15.7:0/436609067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f541c003a60 0x7f541c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.292+0000 7f5435b0e700 1 --2- 172.21.15.7:0/436609067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54300b1900 0x7f541c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.292+0000 7f5434b0c700 1 -- 172.21.15.7:0/436609067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54300b1900 msgr2=0x7f541c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.292+0000 7f5434b0c700 1 --2- 172.21.15.7:0/436609067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54300b1900 0x7f541c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.292+0000 7f5434b0c700 1 -- 172.21.15.7:0/436609067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f541c003a60 msgr2=0x7f541c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.292+0000 7f5434b0c700 1 --2- 172.21.15.7:0/436609067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f541c003a60 0x7f541c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.292+0000 7f5434b0c700 1 -- 172.21.15.7:0/436609067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c003570 con 0x7f541c004810 2024-01-27T02:38:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.292+0000 7f5435b0e700 1 --2- 172.21.15.7:0/436609067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54300b1900 0x7f541c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.292+0000 7f5434b0c700 1 --2- 172.21.15.7:0/436609067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f541c004810 0x7f541c004c80 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f5420009c30 tx=0x7f54200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ed94b551ec4f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.292+0000 7f5427fff700 1 -- 172.21.15.7:0/436609067 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542000e710 con 0x7f541c004810 2024-01-27T02:38:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.292+0000 7f5427fff700 1 -- 172.21.15.7:0/436609067 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542000e870 con 0x7f541c004810 2024-01-27T02:38:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.292+0000 7f5427fff700 1 -- 172.21.15.7:0/436609067 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542000ec30 con 0x7f541c004810 2024-01-27T02:38:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.293+0000 7f5437571700 1 -- 172.21.15.7:0/436609067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f541c004810 msgr2=0x7f541c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.293+0000 7f5437571700 1 --2- 172.21.15.7:0/436609067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f541c004810 0x7f541c004c80 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f5420009c30 tx=0x7f54200091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.293+0000 7f5435b0e700 1 -- 172.21.15.7:0/436609067 reap_dead start 2024-01-27T02:38:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.293+0000 7f5437571700 1 -- 172.21.15.7:0/436609067 shutdown_connections 2024-01-27T02:38:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.293+0000 7f5437571700 1 -- 172.21.15.7:0/436609067 >> 172.21.15.7:0/436609067 conn(0x7f54300679b0 msgr2=0x7f5430067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.293+0000 7f5437571700 1 -- 172.21.15.7:0/436609067 shutdown_connections 2024-01-27T02:38:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.293+0000 7f5437571700 1 -- 172.21.15.7:0/436609067 wait complete. 2024-01-27T02:38:36.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.294+0000 7f5437571700 1 Processor -- start 2024-01-27T02:38:36.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.294+0000 7f5437571700 1 -- start start 2024-01-27T02:38:36.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.295+0000 7f5437571700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f541c004810 0x7f543015a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:36.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.295+0000 7f5437571700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54300c3860 0x7f5430155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:36.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.295+0000 7f5437571700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5430155960 0x7f5430099870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:36.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.295+0000 7f5437571700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543015b760 con 0x7f5430155960 2024-01-27T02:38:36.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.295+0000 7f5437571700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543015a8b0 con 0x7f54300c3860 2024-01-27T02:38:36.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.295+0000 7f5437571700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543015ab50 con 0x7f541c004810 2024-01-27T02:38:36.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.295+0000 7f5435b0e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5430155960 0x7f5430099870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:36.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.295+0000 7f5435b0e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5430155960 0x7f5430099870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58858/0 (socket says 172.21.15.7:58858) 2024-01-27T02:38:36.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.295+0000 7f5435b0e700 1 -- 172.21.15.7:0/3075517219 learned_addr learned my addr 172.21.15.7:0/3075517219 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:36.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.295+0000 7f543530d700 1 --2- 172.21.15.7:0/3075517219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f541c004810 0x7f543015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:36.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.295+0000 7f5434b0c700 1 --2- 172.21.15.7:0/3075517219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54300c3860 0x7f5430155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:36.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.296+0000 7f5435b0e700 1 -- 172.21.15.7:0/3075517219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f541c004810 msgr2=0x7f543015a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:36.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.296+0000 7f5435b0e700 1 --2- 172.21.15.7:0/3075517219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f541c004810 0x7f543015a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.296+0000 7f5435b0e700 1 -- 172.21.15.7:0/3075517219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54300c3860 msgr2=0x7f5430155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:36.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.296+0000 7f5435b0e700 1 --2- 172.21.15.7:0/3075517219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54300c3860 0x7f5430155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.296+0000 7f5435b0e700 1 -- 172.21.15.7:0/3075517219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543015ae30 con 0x7f5430155960 2024-01-27T02:38:36.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.296+0000 7f543530d700 1 --2- 172.21.15.7:0/3075517219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f541c004810 0x7f543015a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:36.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.296+0000 7f5435b0e700 1 --2- 172.21.15.7:0/3075517219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5430155960 0x7f5430099870 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f54300b21a0 tx=0x7f542800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:36.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.296+0000 7f54257fa700 1 -- 172.21.15.7:0/3075517219 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542800e050 con 0x7f5430155960 2024-01-27T02:38:36.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.296+0000 7f54257fa700 1 -- 172.21.15.7:0/3075517219 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542800f040 con 0x7f5430155960 2024-01-27T02:38:36.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.296+0000 7f54257fa700 1 -- 172.21.15.7:0/3075517219 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428010df0 con 0x7f5430155960 2024-01-27T02:38:36.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.296+0000 7f5437571700 1 -- 172.21.15.7:0/3075517219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f543015b110 con 0x7f5430155960 2024-01-27T02:38:36.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.296+0000 7f5437571700 1 -- 172.21.15.7:0/3075517219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5430099db0 con 0x7f5430155960 2024-01-27T02:38:36.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.297+0000 7f5437571700 1 -- 172.21.15.7:0/3075517219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f543004c710 con 0x7f5430155960 2024-01-27T02:38:36.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.298+0000 7f54257fa700 1 -- 172.21.15.7:0/3075517219 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5428003020 con 0x7f5430155960 2024-01-27T02:38:36.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.298+0000 7f54257fa700 1 --2- 172.21.15.7:0/3075517219 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5414066f30 0x7f54140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:36.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.298+0000 7f54257fa700 1 -- 172.21.15.7:0/3075517219 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f54280848b0 con 0x7f5430155960 2024-01-27T02:38:36.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.298+0000 7f543530d700 1 --2- 172.21.15.7:0/3075517219 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5414066f30 0x7f54140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:36.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.299+0000 7f543530d700 1 --2- 172.21.15.7:0/3075517219 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5414066f30 0x7f54140693e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f542c0016e0 tx=0x7f542c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:36.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.302+0000 7f54257fa700 1 -- 172.21.15.7:0/3075517219 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5428051b20 con 0x7f5430155960 2024-01-27T02:38:36.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.500+0000 7f5437571700 1 -- 172.21.15.7:0/3075517219 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f543015b5b0 con 0x7f5414066f30 2024-01-27T02:38:36.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.508+0000 7f54257fa700 1 -- 172.21.15.7:0/3075517219 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282419 (secure 0 0 0) 0x7f543015b5b0 con 0x7f5414066f30 2024-01-27T02:38:36.512 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:36.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.511+0000 7f5437571700 1 -- 172.21.15.7:0/3075517219 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5414066f30 msgr2=0x7f54140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:36.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.511+0000 7f5437571700 1 --2- 172.21.15.7:0/3075517219 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5414066f30 0x7f54140693e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f542c0016e0 tx=0x7f542c009040 comp rx=0 tx=0).stop 2024-01-27T02:38:36.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.511+0000 7f5437571700 1 -- 172.21.15.7:0/3075517219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5430155960 msgr2=0x7f5430099870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:36.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.511+0000 7f5437571700 1 --2- 172.21.15.7:0/3075517219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5430155960 0x7f5430099870 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f54300b21a0 tx=0x7f542800b040 comp rx=0 tx=0).stop 2024-01-27T02:38:36.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.511+0000 7f5437571700 1 -- 172.21.15.7:0/3075517219 shutdown_connections 2024-01-27T02:38:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.511+0000 7f5437571700 1 --2- 172.21.15.7:0/3075517219 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5414066f30 0x7f54140693e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.512+0000 7f5437571700 1 --2- 172.21.15.7:0/3075517219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5430155960 0x7f5430099870 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.512+0000 7f5437571700 1 --2- 172.21.15.7:0/3075517219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f541c004810 0x7f543015a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.512+0000 7f5437571700 1 --2- 172.21.15.7:0/3075517219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54300c3860 0x7f5430155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.512+0000 7f5437571700 1 -- 172.21.15.7:0/3075517219 >> 172.21.15.7:0/3075517219 conn(0x7f54300679b0 msgr2=0x7f5430156430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.512+0000 7f5437571700 1 -- 172.21.15.7:0/3075517219 shutdown_connections 2024-01-27T02:38:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.512+0000 7f5437571700 1 -- 172.21.15.7:0/3075517219 wait complete. 2024-01-27T02:38:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:38:36.526 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-01-27T02:38:35.667278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6694,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555805,"statfs":{"total":1600000000,"available":1593144606,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.780828"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897968+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:32:52.668655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895827+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:08.702500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896528+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:11:34.261074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896262+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:22:55.022123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.899959+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:30:08.176207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896813+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:02:36.439070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897666+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:14:25.858808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896102+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:51.023684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895424+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:57:59.292913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897605+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:55:31.456845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.894631+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:05:45.293447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896477+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:40:57.016595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895636+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:51:24.348901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895665+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:19:55.078411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.902105+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:52:50.516673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897186+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:15:59.844843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895485+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:11:34.261074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895678+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:20:34.579927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897488+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:23:46.870039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893388+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:16:13.624836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893364+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:39:26.847232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.894037+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:23:46.870039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893002+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:53:05.325595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893907+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:05:23.541565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893041+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:18.354062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896299+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:36:18.147300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897512+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:07:33.422861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893752+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:30:10.336341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893836+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:18:05.101790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896393+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:36:36.290157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895868+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:54:13.383710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.901471+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:02:27.766940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896480+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:47:09.052904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.900146+0000","last_change":"2024-01-27T02:38:30.900146+0000","last_active":"2024-01-27T02:38:30.900146+0000","last_peered":"2024-01-27T02:38:30.900146+0000","last_clean":"2024-01-27T02:38:30.900146+0000","last_became_active":"2024-01-27T02:38:30.899920+0000","last_became_peered":"2024-01-27T02:38:30.899920+0000","last_unstale":"2024-01-27T02:38:30.900146+0000","last_undegraded":"2024-01-27T02:38:30.900146+0000","last_fullsized":"2024-01-27T02:38:30.900146+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:02:26.763414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895285+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:01:42.972197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.900419+0000","last_change":"2024-01-27T02:38:30.900419+0000","last_active":"2024-01-27T02:38:30.900419+0000","last_peered":"2024-01-27T02:38:30.900419+0000","last_clean":"2024-01-27T02:38:30.900419+0000","last_became_active":"2024-01-27T02:38:30.900188+0000","last_became_peered":"2024-01-27T02:38:30.900188+0000","last_unstale":"2024-01-27T02:38:30.900419+0000","last_undegraded":"2024-01-27T02:38:30.900419+0000","last_fullsized":"2024-01-27T02:38:30.900419+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:16:12.621310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896381+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:32:20.165836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.907104+0000","last_change":"2024-01-27T02:38:30.907104+0000","last_active":"2024-01-27T02:38:30.907104+0000","last_peered":"2024-01-27T02:38:30.907104+0000","last_clean":"2024-01-27T02:38:30.907104+0000","last_became_active":"2024-01-27T02:38:30.906807+0000","last_became_peered":"2024-01-27T02:38:30.906807+0000","last_unstale":"2024-01-27T02:38:30.907104+0000","last_undegraded":"2024-01-27T02:38:30.907104+0000","last_fullsized":"2024-01-27T02:38:30.907104+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:16:05.064427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895895+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:05:45.293447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.906090+0000","last_change":"2024-01-27T02:38:30.906090+0000","last_active":"2024-01-27T02:38:30.906090+0000","last_peered":"2024-01-27T02:38:30.906090+0000","last_clean":"2024-01-27T02:38:30.906090+0000","last_became_active":"2024-01-27T02:38:30.905815+0000","last_became_peered":"2024-01-27T02:38:30.905815+0000","last_unstale":"2024-01-27T02:38:30.906090+0000","last_undegraded":"2024-01-27T02:38:30.906090+0000","last_fullsized":"2024-01-27T02:38:30.906090+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:42:16.605919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895621+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:00:23.173837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.908846+0000","last_change":"2024-01-27T02:38:30.908846+0000","last_active":"2024-01-27T02:38:30.908846+0000","last_peered":"2024-01-27T02:38:30.908846+0000","last_clean":"2024-01-27T02:38:30.908846+0000","last_became_active":"2024-01-27T02:38:30.908634+0000","last_became_peered":"2024-01-27T02:38:30.908634+0000","last_unstale":"2024-01-27T02:38:30.908846+0000","last_undegraded":"2024-01-27T02:38:30.908846+0000","last_fullsized":"2024-01-27T02:38:30.908846+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:44:21.414722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895115+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:01:42.972197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.897433+0000","last_change":"2024-01-27T02:38:30.897433+0000","last_active":"2024-01-27T02:38:30.897433+0000","last_peered":"2024-01-27T02:38:30.897433+0000","last_clean":"2024-01-27T02:38:30.897433+0000","last_became_active":"2024-01-27T02:38:30.897134+0000","last_became_peered":"2024-01-27T02:38:30.897134+0000","last_unstale":"2024-01-27T02:38:30.897433+0000","last_undegraded":"2024-01-27T02:38:30.897433+0000","last_fullsized":"2024-01-27T02:38:30.897433+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:18:04.098263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897157+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:32:05.199937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.903018+0000","last_change":"2024-01-27T02:38:30.903018+0000","last_active":"2024-01-27T02:38:30.903018+0000","last_peered":"2024-01-27T02:38:30.903018+0000","last_clean":"2024-01-27T02:38:30.903018+0000","last_became_active":"2024-01-27T02:38:30.902748+0000","last_became_peered":"2024-01-27T02:38:30.902748+0000","last_unstale":"2024-01-27T02:38:30.903018+0000","last_undegraded":"2024-01-27T02:38:30.903018+0000","last_fullsized":"2024-01-27T02:38:30.903018+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:14:24.855281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896050+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:56:12.566293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.903329+0000","last_change":"2024-01-27T02:38:30.903329+0000","last_active":"2024-01-27T02:38:30.903329+0000","last_peered":"2024-01-27T02:38:30.903329+0000","last_clean":"2024-01-27T02:38:30.903329+0000","last_became_active":"2024-01-27T02:38:30.903084+0000","last_became_peered":"2024-01-27T02:38:30.903084+0000","last_unstale":"2024-01-27T02:38:30.903329+0000","last_undegraded":"2024-01-27T02:38:30.903329+0000","last_fullsized":"2024-01-27T02:38:30.903329+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:11:33.257548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896315+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:36:50.676219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.904630+0000","last_change":"2024-01-27T02:38:30.904630+0000","last_active":"2024-01-27T02:38:30.904630+0000","last_peered":"2024-01-27T02:38:30.904630+0000","last_clean":"2024-01-27T02:38:30.904630+0000","last_became_active":"2024-01-27T02:38:30.904406+0000","last_became_peered":"2024-01-27T02:38:30.904406+0000","last_unstale":"2024-01-27T02:38:30.904630+0000","last_undegraded":"2024-01-27T02:38:30.904630+0000","last_fullsized":"2024-01-27T02:38:30.904630+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:22:44.330464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893390+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:05:45.293447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.893083+0000","last_change":"2024-01-27T02:38:30.893083+0000","last_active":"2024-01-27T02:38:30.893083+0000","last_peered":"2024-01-27T02:38:30.893083+0000","last_clean":"2024-01-27T02:38:30.893083+0000","last_became_active":"2024-01-27T02:38:30.892801+0000","last_became_peered":"2024-01-27T02:38:30.892801+0000","last_unstale":"2024-01-27T02:38:30.893083+0000","last_undegraded":"2024-01-27T02:38:30.893083+0000","last_fullsized":"2024-01-27T02:38:30.893083+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:51:23.345374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892794+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:08.702500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.905086+0000","last_change":"2024-01-27T02:38:30.905086+0000","last_active":"2024-01-27T02:38:30.905086+0000","last_peered":"2024-01-27T02:38:30.905086+0000","last_clean":"2024-01-27T02:38:30.905086+0000","last_became_active":"2024-01-27T02:38:30.904853+0000","last_became_peered":"2024-01-27T02:38:30.904853+0000","last_unstale":"2024-01-27T02:38:30.905086+0000","last_undegraded":"2024-01-27T02:38:30.905086+0000","last_fullsized":"2024-01-27T02:38:30.905086+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:54:14.886540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896562+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:03:26.614534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.905975+0000","last_change":"2024-01-27T02:38:30.905975+0000","last_active":"2024-01-27T02:38:30.905975+0000","last_peered":"2024-01-27T02:38:30.905975+0000","last_clean":"2024-01-27T02:38:30.905975+0000","last_became_active":"2024-01-27T02:38:30.904239+0000","last_became_peered":"2024-01-27T02:38:30.904239+0000","last_unstale":"2024-01-27T02:38:30.905975+0000","last_undegraded":"2024-01-27T02:38:30.905975+0000","last_fullsized":"2024-01-27T02:38:30.905975+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:05:22.538039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892200+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:05:45.293447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.904960+0000","last_change":"2024-01-27T02:38:30.904960+0000","last_active":"2024-01-27T02:38:30.904960+0000","last_peered":"2024-01-27T02:38:30.904960+0000","last_clean":"2024-01-27T02:38:30.904960+0000","last_became_active":"2024-01-27T02:38:30.904479+0000","last_became_peered":"2024-01-27T02:38:30.904479+0000","last_unstale":"2024-01-27T02:38:30.904960+0000","last_undegraded":"2024-01-27T02:38:30.904960+0000","last_fullsized":"2024-01-27T02:38:30.904960+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:20:17.775604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892827+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:11:34.261074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.901125+0000","last_change":"2024-01-27T02:38:30.901125+0000","last_active":"2024-01-27T02:38:30.901125+0000","last_peered":"2024-01-27T02:38:30.901125+0000","last_clean":"2024-01-27T02:38:30.901125+0000","last_became_active":"2024-01-27T02:38:30.900811+0000","last_became_peered":"2024-01-27T02:38:30.900811+0000","last_unstale":"2024-01-27T02:38:30.901125+0000","last_undegraded":"2024-01-27T02:38:30.901125+0000","last_fullsized":"2024-01-27T02:38:30.901125+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:18:04.098263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892350+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:20:34.579927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.904062+0000","last_change":"2024-01-27T02:38:30.904062+0000","last_active":"2024-01-27T02:38:30.904062+0000","last_peered":"2024-01-27T02:38:30.904062+0000","last_clean":"2024-01-27T02:38:30.904062+0000","last_became_active":"2024-01-27T02:38:30.903827+0000","last_became_peered":"2024-01-27T02:38:30.903827+0000","last_unstale":"2024-01-27T02:38:30.904062+0000","last_undegraded":"2024-01-27T02:38:30.904062+0000","last_fullsized":"2024-01-27T02:38:30.904062+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:22:30.041364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.900402+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:59:45.149668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.901317+0000","last_change":"2024-01-27T02:38:30.901317+0000","last_active":"2024-01-27T02:38:30.901317+0000","last_peered":"2024-01-27T02:38:30.901317+0000","last_clean":"2024-01-27T02:38:30.901317+0000","last_became_active":"2024-01-27T02:38:30.900905+0000","last_became_peered":"2024-01-27T02:38:30.900905+0000","last_unstale":"2024-01-27T02:38:30.901317+0000","last_undegraded":"2024-01-27T02:38:30.901317+0000","last_fullsized":"2024-01-27T02:38:30.901317+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:23:45.866513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895981+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:48:06.641759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.901405+0000","last_change":"2024-01-27T02:38:30.901405+0000","last_active":"2024-01-27T02:38:30.901405+0000","last_peered":"2024-01-27T02:38:30.901405+0000","last_clean":"2024-01-27T02:38:30.901405+0000","last_became_active":"2024-01-27T02:38:30.901168+0000","last_became_peered":"2024-01-27T02:38:30.901168+0000","last_unstale":"2024-01-27T02:38:30.901405+0000","last_undegraded":"2024-01-27T02:38:30.901405+0000","last_fullsized":"2024-01-27T02:38:30.901405+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:52:03.557302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896480+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:20:34.579927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.894794+0000","last_change":"2024-01-27T02:38:30.894794+0000","last_active":"2024-01-27T02:38:30.894794+0000","last_peered":"2024-01-27T02:38:30.894794+0000","last_clean":"2024-01-27T02:38:30.894794+0000","last_became_active":"2024-01-27T02:38:30.894519+0000","last_became_peered":"2024-01-27T02:38:30.894519+0000","last_unstale":"2024-01-27T02:38:30.894794+0000","last_undegraded":"2024-01-27T02:38:30.894794+0000","last_fullsized":"2024-01-27T02:38:30.894794+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:39:25.843705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896689+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:30:10.336341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.901197+0000","last_change":"2024-01-27T02:38:30.901197+0000","last_active":"2024-01-27T02:38:30.901197+0000","last_peered":"2024-01-27T02:38:30.901197+0000","last_clean":"2024-01-27T02:38:30.901197+0000","last_became_active":"2024-01-27T02:38:30.900235+0000","last_became_peered":"2024-01-27T02:38:30.900235+0000","last_unstale":"2024-01-27T02:38:30.901197+0000","last_undegraded":"2024-01-27T02:38:30.901197+0000","last_fullsized":"2024-01-27T02:38:30.901197+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:07:32.419335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897132+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:05:23.541565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.904351+0000","last_change":"2024-01-27T02:38:30.904351+0000","last_active":"2024-01-27T02:38:30.904351+0000","last_peered":"2024-01-27T02:38:30.904351+0000","last_clean":"2024-01-27T02:38:30.904351+0000","last_became_active":"2024-01-27T02:38:30.904118+0000","last_became_peered":"2024-01-27T02:38:30.904118+0000","last_unstale":"2024-01-27T02:38:30.904351+0000","last_undegraded":"2024-01-27T02:38:30.904351+0000","last_fullsized":"2024-01-27T02:38:30.904351+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:53:19.168298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895194+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:07.978917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.904812+0000","last_change":"2024-01-27T02:38:30.904812+0000","last_active":"2024-01-27T02:38:30.904812+0000","last_peered":"2024-01-27T02:38:30.904812+0000","last_clean":"2024-01-27T02:38:30.904812+0000","last_became_active":"2024-01-27T02:38:30.904483+0000","last_became_peered":"2024-01-27T02:38:30.904483+0000","last_unstale":"2024-01-27T02:38:30.904812+0000","last_undegraded":"2024-01-27T02:38:30.904812+0000","last_fullsized":"2024-01-27T02:38:30.904812+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:45:43.930079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896313+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:01:42.972197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.904780+0000","last_change":"2024-01-27T02:38:30.904780+0000","last_active":"2024-01-27T02:38:30.904780+0000","last_peered":"2024-01-27T02:38:30.904780+0000","last_clean":"2024-01-27T02:38:30.904780+0000","last_became_active":"2024-01-27T02:38:30.904179+0000","last_became_peered":"2024-01-27T02:38:30.904179+0000","last_unstale":"2024-01-27T02:38:30.904780+0000","last_undegraded":"2024-01-27T02:38:30.904780+0000","last_fullsized":"2024-01-27T02:38:30.904780+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:14:13.119229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892253+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:51.023684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.899295+0000","last_change":"2024-01-27T02:38:30.899295+0000","last_active":"2024-01-27T02:38:30.899295+0000","last_peered":"2024-01-27T02:38:30.899295+0000","last_clean":"2024-01-27T02:38:30.899295+0000","last_became_active":"2024-01-27T02:38:30.899015+0000","last_became_peered":"2024-01-27T02:38:30.899015+0000","last_unstale":"2024-01-27T02:38:30.899295+0000","last_undegraded":"2024-01-27T02:38:30.899295+0000","last_fullsized":"2024-01-27T02:38:30.899295+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:34:25.075555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893779+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:51.023684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.902450+0000","last_change":"2024-01-27T02:38:30.902450+0000","last_active":"2024-01-27T02:38:30.902450+0000","last_peered":"2024-01-27T02:38:30.902450+0000","last_clean":"2024-01-27T02:38:30.902450+0000","last_became_active":"2024-01-27T02:38:30.902130+0000","last_became_peered":"2024-01-27T02:38:30.902130+0000","last_unstale":"2024-01-27T02:38:30.902450+0000","last_undegraded":"2024-01-27T02:38:30.902450+0000","last_fullsized":"2024-01-27T02:38:30.902450+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:39:25.843705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.894773+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:40:17.611985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.901304+0000","last_change":"2024-01-27T02:38:30.901304+0000","last_active":"2024-01-27T02:38:30.901304+0000","last_peered":"2024-01-27T02:38:30.901304+0000","last_clean":"2024-01-27T02:38:30.901304+0000","last_became_active":"2024-01-27T02:38:30.901065+0000","last_became_peered":"2024-01-27T02:38:30.901065+0000","last_unstale":"2024-01-27T02:38:30.901304+0000","last_undegraded":"2024-01-27T02:38:30.901304+0000","last_fullsized":"2024-01-27T02:38:30.901304+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:16:12.621310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.894797+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:02:03.382943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.901161+0000","last_change":"2024-01-27T02:38:30.901161+0000","last_active":"2024-01-27T02:38:30.901161+0000","last_peered":"2024-01-27T02:38:30.901161+0000","last_clean":"2024-01-27T02:38:30.901161+0000","last_became_active":"2024-01-27T02:38:30.899610+0000","last_became_peered":"2024-01-27T02:38:30.899610+0000","last_unstale":"2024-01-27T02:38:30.901161+0000","last_undegraded":"2024-01-27T02:38:30.901161+0000","last_fullsized":"2024-01-27T02:38:30.901161+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:30:09.332815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892858+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:19:55.078411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.904612+0000","last_change":"2024-01-27T02:38:30.904612+0000","last_active":"2024-01-27T02:38:30.904612+0000","last_peered":"2024-01-27T02:38:30.904612+0000","last_clean":"2024-01-27T02:38:30.904612+0000","last_became_active":"2024-01-27T02:38:30.904299+0000","last_became_peered":"2024-01-27T02:38:30.904299+0000","last_unstale":"2024-01-27T02:38:30.904612+0000","last_undegraded":"2024-01-27T02:38:30.904612+0000","last_fullsized":"2024-01-27T02:38:30.904612+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:16:12.621310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.894726+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:19:55.078411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.899705+0000","last_change":"2024-01-27T02:38:30.899705+0000","last_active":"2024-01-27T02:38:30.899705+0000","last_peered":"2024-01-27T02:38:30.899705+0000","last_clean":"2024-01-27T02:38:30.899705+0000","last_became_active":"2024-01-27T02:38:30.899453+0000","last_became_peered":"2024-01-27T02:38:30.899453+0000","last_unstale":"2024-01-27T02:38:30.899705+0000","last_undegraded":"2024-01-27T02:38:30.899705+0000","last_fullsized":"2024-01-27T02:38:30.899705+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:02:35.435543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892871+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:18:12.582908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.902907+0000","last_change":"2024-01-27T02:38:30.902907+0000","last_active":"2024-01-27T02:38:30.902907+0000","last_peered":"2024-01-27T02:38:30.902907+0000","last_clean":"2024-01-27T02:38:30.902907+0000","last_became_active":"2024-01-27T02:38:30.902691+0000","last_became_peered":"2024-01-27T02:38:30.902691+0000","last_unstale":"2024-01-27T02:38:30.902907+0000","last_undegraded":"2024-01-27T02:38:30.902907+0000","last_fullsized":"2024-01-27T02:38:30.902907+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:25:13.514723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896780+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:08.702500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.906486+0000","last_change":"2024-01-27T02:38:30.906486+0000","last_active":"2024-01-27T02:38:30.906486+0000","last_peered":"2024-01-27T02:38:30.906486+0000","last_clean":"2024-01-27T02:38:30.906486+0000","last_became_active":"2024-01-27T02:38:30.906233+0000","last_became_peered":"2024-01-27T02:38:30.906233+0000","last_unstale":"2024-01-27T02:38:30.906486+0000","last_undegraded":"2024-01-27T02:38:30.906486+0000","last_fullsized":"2024-01-27T02:38:30.906486+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:27:08.109911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.894650+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:11:36.249589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.897972+0000","last_change":"2024-01-27T02:38:30.897972+0000","last_active":"2024-01-27T02:38:30.897972+0000","last_peered":"2024-01-27T02:38:30.897972+0000","last_clean":"2024-01-27T02:38:30.897972+0000","last_became_active":"2024-01-27T02:38:30.897716+0000","last_became_peered":"2024-01-27T02:38:30.897716+0000","last_unstale":"2024-01-27T02:38:30.897972+0000","last_undegraded":"2024-01-27T02:38:30.897972+0000","last_fullsized":"2024-01-27T02:38:30.897972+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:07.698973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892275+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:01:42.972197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.901349+0000","last_change":"2024-01-27T02:38:30.901349+0000","last_active":"2024-01-27T02:38:30.901349+0000","last_peered":"2024-01-27T02:38:30.901349+0000","last_clean":"2024-01-27T02:38:30.901349+0000","last_became_active":"2024-01-27T02:38:30.900973+0000","last_became_peered":"2024-01-27T02:38:30.900973+0000","last_unstale":"2024-01-27T02:38:30.901349+0000","last_undegraded":"2024-01-27T02:38:30.901349+0000","last_fullsized":"2024-01-27T02:38:30.901349+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:55:50.241360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":257,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-01-27T02:38:29.888544+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:38:29.888544+0000","last_peered":"2024-01-27T02:38:29.888544+0000","last_clean":"2024-01-27T02:38:29.888544+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:38:29.888544+0000","last_undegraded":"2024-01-27T02:38:29.888544+0000","last_fullsized":"2024-01-27T02:38:29.888544+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574929,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1270,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194041,"statfs":{"total":200000000,"available":198698734,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77400000000000002,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70299999999999996,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.60999999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.71999999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.42799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69999999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.65600000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.68200000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71399999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.63100000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.73099999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.48199999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.71799999999999997,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.71499999999999997}]},{"osd":4,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.47599999999999998,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.51100000000000001},"min":{"1min":0.33700000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.50600000000000001}]},{"osd":5,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51300000000000001,"15min":0.53400000000000003},"min":{"1min":0.35999999999999999,"5min":0.32100000000000001,"15min":0.307},"max":{"1min":0.68500000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.56299999999999994},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75800000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.73099999999999998}]},{"osd":6,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57199999999999995,"15min":0.59399999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.78800000000000003,"5min":0.80300000000000005,"15min":0.81299999999999994},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56100000000000005,"15min":0.57499999999999996},"min":{"1min":0.42499999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.73099999999999998,"5min":0.81599999999999995,"15min":0.90800000000000003},"last":0.70299999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574929,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":688,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194624,"statfs":{"total":200000000,"available":199295365,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.64900000000000002,"15min":0.54200000000000004},"min":{"1min":0.442,"5min":0.36499999999999999,"15min":0.29999999999999999},"max":{"1min":4.125,"5min":4.125,"15min":4.125},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60599999999999998,"15min":0.53600000000000003},"min":{"1min":0.45100000000000001,"5min":0.35699999999999998,"15min":0.32700000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.61099999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.68400000000000005,"15min":0.55200000000000005},"min":{"1min":0.433,"5min":0.41999999999999998,"15min":0.34100000000000003},"max":{"1min":8.2699999999999996,"5min":8.2699999999999996,"15min":8.2699999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":2.0409999999999999,"5min":0.90800000000000003,"15min":0.65600000000000003},"min":{"1min":0.46999999999999997,"5min":0.379,"15min":0.33800000000000002},"max":{"1min":26.411999999999999,"5min":26.411999999999999,"15min":26.411999999999999},"last":0.53500000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":0.77700000000000002,"15min":0.60899999999999999},"min":{"1min":0.45100000000000001,"5min":0.41199999999999998,"15min":0.39600000000000002},"max":{"1min":17.143999999999998,"5min":17.143999999999998,"15min":17.143999999999998},"last":0.435},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.64900000000000002,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":0.55900000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.63300000000000001,"15min":0.56799999999999995},"min":{"1min":0.45800000000000002,"5min":0.435,"15min":0.376},"max":{"1min":4.3689999999999998,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64100000000000001,"15min":0.58099999999999996},"min":{"1min":0.433,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.58499999999999996}]},{"osd":4,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.33200000000000002,"5min":0.32000000000000001,"15min":0.30199999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51600000000000001,"15min":0.54900000000000004},"min":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74299999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.38900000000000001}]},{"osd":5,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.52100000000000002,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76100000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56000000000000005,"15min":0.59999999999999998},"min":{"1min":0.41099999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77000000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.46100000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.58999999999999997,"15min":0.61099999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79400000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55800000000000005,"15min":0.58499999999999996},"min":{"1min":0.40000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71499999999999997,"5min":1.038,"15min":1.038},"last":0.5}]}]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":688,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194624,"statfs":{"total":200000000,"available":199295365,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.39700000000000002,"15min":0.377},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25800000000000001},"max":{"1min":0.50600000000000001,"5min":0.873,"15min":0.873},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.41899999999999998,"15min":0.39900000000000002},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.53100000000000003,"5min":1.121,"15min":1.121},"last":0.31900000000000001}]},{"osd":2,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.40999999999999998,"15min":0.36899999999999999},"min":{"1min":0.27200000000000002,"5min":0.26600000000000001,"15min":0.23999999999999999},"max":{"1min":0.48599999999999999,"5min":1.25,"15min":1.25},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.45600000000000002,"15min":0.432},"min":{"1min":0.32000000000000001,"5min":0.30199999999999999,"15min":0.29899999999999999},"max":{"1min":0.58599999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.44600000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.49399999999999999,"15min":0.46999999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.32100000000000001},"max":{"1min":0.66800000000000004,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.46999999999999997,"15min":0.45600000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.58199999999999996,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.38400000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.66300000000000003,"15min":0.63500000000000001},"min":{"1min":0.501,"5min":0.373,"15min":0.373},"max":{"1min":0.81599999999999995,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66200000000000003,"15min":0.63800000000000001},"min":{"1min":0.497,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.82599999999999996,"5min":1.026,"15min":1.026},"last":0.84999999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.68899999999999995,"15min":0.63900000000000001},"min":{"1min":0.51000000000000001,"5min":0.46200000000000002,"15min":0.44},"max":{"1min":0.80900000000000005,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.752},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69099999999999995,"15min":0.67600000000000005},"min":{"1min":0.53000000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83799999999999997,"5min":3.181,"15min":3.181},"last":0.89600000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71199999999999997,"15min":0.69699999999999995},"min":{"1min":0.51400000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.875,"5min":1.024,"15min":1.024},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71499999999999997,"15min":0.68000000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.85699999999999998,"5min":1.869,"15min":1.869},"last":0.86599999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.72699999999999998,"15min":0.68700000000000006},"min":{"1min":0.55800000000000005,"5min":0.495,"15min":0.47299999999999998},"max":{"1min":0.82599999999999996,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.73099999999999998,"15min":0.69399999999999995},"min":{"1min":0.56000000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.86799999999999999,"5min":1.708,"15min":1.708},"last":0.79500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1270,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194041,"statfs":{"total":200000000,"available":198698734,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39000000000000001},"min":{"1min":0.253,"5min":0.24399999999999999,"15min":0.23799999999999999},"max":{"1min":0.52100000000000002,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.41999999999999998,"15min":0.41599999999999998},"min":{"1min":0.29599999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.64200000000000002,"5min":0.65200000000000002,"15min":0.68400000000000005},"last":0.60399999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.437,"15min":0.42199999999999999},"min":{"1min":0.30499999999999999,"5min":0.28699999999999998,"15min":0.28199999999999997},"max":{"1min":0.60299999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.435},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.44900000000000001,"15min":0.435},"min":{"1min":0.32400000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.628,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.47999999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42799999999999999,"15min":0.43099999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58199999999999996,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.629},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48499999999999999,"15min":0.47499999999999998},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69399999999999995,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.55900000000000005}]},{"osd":4,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65900000000000003,"15min":0.64100000000000001},"min":{"1min":0.49399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91700000000000004,"5min":0.997,"15min":0.997},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.44700000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92100000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.67300000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67700000000000005,"15min":0.65700000000000003},"min":{"1min":0.496,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88600000000000001,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.68100000000000005},"min":{"1min":0.53100000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.82099999999999995,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.748}]},{"osd":6,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.57299999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.80300000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69599999999999995,"15min":0.68000000000000005},"min":{"1min":0.51400000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94299999999999995,"5min":1.04,"15min":1.04},"last":0.65800000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68899999999999995,"15min":0.69199999999999995},"min":{"1min":0.45900000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90800000000000003,"5min":1.014,"15min":1.014},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68700000000000006,"15min":0.69299999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.83899999999999997,"5min":0.95799999999999996,"15min":0.97299999999999998},"last":0.72999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":694,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194618,"statfs":{"total":200000000,"available":199289102,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.41399999999999998,"15min":0.40899999999999997},"min":{"1min":0.23599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.58099999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.308},{"interface":"front","average":{"1min":0.443,"5min":0.45700000000000002,"15min":0.62},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.30499999999999999},"max":{"1min":0.64700000000000002,"5min":0.72799999999999998,"15min":6.2729999999999997},"last":0.38200000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.42999999999999999,"15min":0.42299999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.71899999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.433,"15min":0.44500000000000001},"min":{"1min":0.32600000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.57999999999999996,"5min":0.65300000000000002,"15min":0.70099999999999996},"last":0.42599999999999999}]},{"osd":3,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42399999999999999,"15min":0.41599999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.27800000000000002},"max":{"1min":0.64700000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.372},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.48699999999999999,"15min":0.48499999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.627,"5min":0.76500000000000001,"15min":0.80800000000000005},"last":0.40999999999999998}]},{"osd":4,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68799999999999994,"15min":0.65800000000000003},"min":{"1min":0.46800000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93999999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70599999999999996,"15min":0.79000000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.006,"5min":1.006,"15min":3.581},"last":0.64400000000000002}]},{"osd":5,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65300000000000002,"15min":0.73499999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":3.5369999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92200000000000004,"5min":0.96599999999999997,"15min":1.0529999999999999},"last":0.52200000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.45000000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.51300000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97499999999999998,"5min":1.0189999999999999,"15min":1.0209999999999999},"last":0.503}]},{"osd":7,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.68300000000000005,"15min":0.67800000000000005},"min":{"1min":0.50800000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95599999999999996,"5min":1.03,"15min":1.03},"last":0.62},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.69799999999999995,"15min":0.68999999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.42699999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.55200000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":694,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194618,"statfs":{"total":200000000,"available":199289102,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.45600000000000002,"15min":0.38200000000000001},"min":{"1min":0.28999999999999998,"5min":0.253,"15min":0.253},"max":{"1min":4.7380000000000004,"5min":4.7380000000000004,"15min":4.7380000000000004},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.41599999999999998,"15min":0.39800000000000002},"min":{"1min":0.27600000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.35399999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.47199999999999998,"15min":0.41199999999999998},"min":{"1min":0.27900000000000003,"5min":0.27400000000000002,"15min":0.22900000000000001},"max":{"1min":4.5739999999999998,"5min":4.5739999999999998,"15min":4.5739999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.52400000000000002,"15min":0.44700000000000001},"min":{"1min":0.311,"5min":0.30199999999999999,"15min":0.27900000000000003},"max":{"1min":5.0529999999999999,"5min":5.0529999999999999,"15min":5.0529999999999999},"last":0.39700000000000002}]},{"osd":2,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.48499999999999999,"15min":0.439},"min":{"1min":0.34799999999999998,"5min":0.33600000000000002,"15min":0.28100000000000003},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.55500000000000005,"15min":0.47399999999999998},"min":{"1min":0.375,"5min":0.35299999999999998,"15min":0.34599999999999997},"max":{"1min":5.1879999999999997,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":0.432}]},{"osd":4,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.69399999999999995,"15min":0.63600000000000001},"min":{"1min":0.42799999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0629999999999999,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66500000000000004,"15min":0.61399999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.79300000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.67800000000000005,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.006,"5min":0.78500000000000003,"15min":0.70299999999999996},"min":{"1min":0.56200000000000006,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":4.7960000000000003,"5min":4.7960000000000003,"15min":4.7960000000000003},"last":0.68400000000000005}]},{"osd":6,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.73499999999999999,"15min":0.67800000000000005},"min":{"1min":0.58899999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":5.0069999999999997,"5min":5.0069999999999997,"15min":5.0069999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.68100000000000005,"15min":0.65400000000000003},"min":{"1min":0.504,"5min":0.435,"15min":0.435},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.74399999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.74299999999999999,"15min":0.66300000000000003},"min":{"1min":0.57699999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.139,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.71299999999999997,"15min":0.67600000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.81299999999999994}]}]},{"osd":4,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":694,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194618,"statfs":{"total":200000000,"available":199289102,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.34599999999999997,"5min":0.33400000000000002,"15min":0.314},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.64600000000000002,"15min":0.60999999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":7.0670000000000002,"5min":7.0670000000000002,"15min":7.0670000000000002},"last":0.76000000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.64700000000000002,"15min":0.60899999999999999},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":5.1529999999999996,"5min":5.1529999999999996,"15min":5.1529999999999996},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.626,"15min":0.60399999999999998},"min":{"1min":0.43099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.0419999999999998,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.751}]},{"osd":2,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.64900000000000002,"15min":0.60999999999999999},"min":{"1min":0.46899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":5.0019999999999998,"5min":5.0019999999999998,"15min":5.0019999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.67600000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.64700000000000002,"15min":0.60099999999999998},"min":{"1min":0.42099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.7220000000000004,"5min":6.7220000000000004,"15min":6.7220000000000004},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.51000000000000001,"5min":0.44900000000000001,"15min":0.41399999999999998},"max":{"1min":6.5510000000000002,"5min":6.5510000000000002,"15min":6.5510000000000002},"last":0.73199999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.503},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.56200000000000006,"15min":0.51400000000000001},"min":{"1min":0.38500000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":6.9269999999999996,"5min":6.9269999999999996,"15min":6.9269999999999996},"last":0.57099999999999995}]},{"osd":6,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.55400000000000005,"15min":0.51700000000000002},"min":{"1min":0.40100000000000002,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":5.2229999999999999,"5min":5.2229999999999999,"15min":5.2229999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.71099999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.60799999999999998,"15min":0.57199999999999995},"min":{"1min":0.34799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":7.5339999999999998,"5min":7.5339999999999998,"15min":7.5339999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.63200000000000001,"15min":0.58999999999999997},"min":{"1min":0.44600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.9550000000000001,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":0.65900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574929,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":694,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194618,"statfs":{"total":200000000,"available":199289102,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56100000000000005,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.84199999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60599999999999998,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.34499999999999997,"15min":0.27600000000000002},"max":{"1min":0.78900000000000003,"5min":3.242,"15min":3.242},"last":0.69499999999999995}]},{"osd":1,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63,"15min":0.60999999999999999},"min":{"1min":0.47299999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80100000000000005,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.62,"15min":0.64700000000000002},"min":{"1min":0.46899999999999997,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77600000000000002,"5min":0.88500000000000001,"15min":0.95499999999999996},"last":0.63700000000000001}]},{"osd":2,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61699999999999999,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.372},"max":{"1min":0.80900000000000005,"5min":3.04,"15min":3.04},"last":0.62},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59599999999999997,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74299999999999999,"5min":0.85899999999999999,"15min":0.89400000000000002},"last":0.59999999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65000000000000002,"15min":0.67300000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82099999999999995,"5min":4.032,"15min":4.032},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.70599999999999996,"15min":0.67900000000000005},"min":{"1min":0.46000000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.78800000000000003,"5min":4.282,"15min":4.282},"last":0.70899999999999996}]},{"osd":4,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.371,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.747,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.499,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76300000000000001,"5min":0.79800000000000004,"15min":1.101},"last":0.51100000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.502,"15min":0.55200000000000005},"min":{"1min":0.38600000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69199999999999995,"5min":0.73199999999999998,"15min":1.7969999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.58099999999999996},"min":{"1min":0.433,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83399999999999996,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.53100000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55100000000000005,"15min":0.56599999999999995},"min":{"1min":0.45000000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68899999999999995,"5min":0.91100000000000003,"15min":1.008},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.61499999999999999,"15min":0.65400000000000003},"min":{"1min":0.40699999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78400000000000003,"5min":2,"15min":2},"last":0.67000000000000004}]}]}],"pool_statfs":[]}} 2024-01-27T02:38:36.531 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:38:36.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.701+0000 7ffbbdb83700 1 Processor -- start 2024-01-27T02:38:36.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.701+0000 7ffbbdb83700 1 -- start start 2024-01-27T02:38:36.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.702+0000 7ffbbdb83700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffbb815be10 0x7ffbb80b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:36.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.702+0000 7ffbbdb83700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffbb80b0180 0x7ffbb80aee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:36.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.702+0000 7ffbbdb83700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbb815c1e0 0x7ffbb80c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:36.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.702+0000 7ffbbdb83700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb80b1470 con 0x7ffbb815be10 2024-01-27T02:38:36.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.702+0000 7ffbbdb83700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb80b0600 con 0x7ffbb80b0180 2024-01-27T02:38:36.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.702+0000 7ffbbdb83700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80af370 con 0x7ffbb815c1e0 2024-01-27T02:38:36.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.702+0000 7ffbb77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffbb815be10 0x7ffbb80b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:36.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.702+0000 7ffbb77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffbb815be10 0x7ffbb80b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58864/0 (socket says 172.21.15.7:58864) 2024-01-27T02:38:36.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.702+0000 7ffbb77fe700 1 -- 172.21.15.7:0/1255002092 learned_addr learned my addr 172.21.15.7:0/1255002092 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:36.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.702+0000 7ffbb6ffd700 1 --2- 172.21.15.7:0/1255002092 >> v2:172.21.15.188:3300/0 conn(0x7ffbb80b0180 0x7ffbb80aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.702+0000 7ffbb7fff700 1 --2- 172.21.15.7:0/1255002092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbb815c1e0 0x7ffbb80c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.702+0000 7ffbb77fe700 1 -- 172.21.15.7:0/1255002092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbb815c1e0 msgr2=0x7ffbb80c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.702+0000 7ffbb77fe700 1 --2- 172.21.15.7:0/1255002092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbb815c1e0 0x7ffbb80c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.702+0000 7ffbb77fe700 1 -- 172.21.15.7:0/1255002092 >> v2:172.21.15.188:3300/0 conn(0x7ffbb80b0180 msgr2=0x7ffbb80aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.702+0000 7ffbb77fe700 1 --2- 172.21.15.7:0/1255002092 >> v2:172.21.15.188:3300/0 conn(0x7ffbb80b0180 0x7ffbb80aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.702+0000 7ffbb77fe700 1 -- 172.21.15.7:0/1255002092 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb80b0860 con 0x7ffbb815be10 2024-01-27T02:38:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.702+0000 7ffbb77fe700 1 --2- 172.21.15.7:0/1255002092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbb815be10 0x7ffbb80b1900 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80afe80 tx=0x7ffba0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c71fcf64d670aa6 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.702+0000 7ffbb67fc700 1 -- 172.21.15.7:0/1255002092 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba000b860 con 0x7ffbb815be10 2024-01-27T02:38:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.703+0000 7ffbb67fc700 1 -- 172.21.15.7:0/1255002092 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba000b9c0 con 0x7ffbb815be10 2024-01-27T02:38:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.703+0000 7ffbb67fc700 1 -- 172.21.15.7:0/1255002092 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba000bd60 con 0x7ffbb815be10 2024-01-27T02:38:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.703+0000 7ffbbdb83700 1 -- 172.21.15.7:0/1255002092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbb815be10 msgr2=0x7ffbb80b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.703+0000 7ffbbdb83700 1 --2- 172.21.15.7:0/1255002092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbb815be10 0x7ffbb80b1900 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80afe80 tx=0x7ffba0009250 comp rx=0 tx=0).stop 2024-01-27T02:38:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.703+0000 7ffbbdb83700 1 -- 172.21.15.7:0/1255002092 shutdown_connections 2024-01-27T02:38:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.703+0000 7ffbbdb83700 1 --2- 172.21.15.7:0/1255002092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbb815c1e0 0x7ffbb80c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.703+0000 7ffbbdb83700 1 --2- 172.21.15.7:0/1255002092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbb815be10 0x7ffbb80b1900 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.703+0000 7ffbbdb83700 1 --2- 172.21.15.7:0/1255002092 >> v2:172.21.15.188:3300/0 conn(0x7ffbb80b0180 0x7ffbb80aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.703+0000 7ffbbdb83700 1 -- 172.21.15.7:0/1255002092 >> 172.21.15.7:0/1255002092 conn(0x7ffbb80679b0 msgr2=0x7ffbb8067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:36.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.703+0000 7ffbbdb83700 1 -- 172.21.15.7:0/1255002092 shutdown_connections 2024-01-27T02:38:36.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.703+0000 7ffbbdb83700 1 -- 172.21.15.7:0/1255002092 wait complete. 2024-01-27T02:38:36.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.704+0000 7ffbbdb83700 1 Processor -- start 2024-01-27T02:38:36.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.705+0000 7ffbbdb83700 1 -- start start 2024-01-27T02:38:36.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.705+0000 7ffbbdb83700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbb80b0180 0x7ffbb8155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:36.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.705+0000 7ffbbdb83700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbb815be10 0x7ffbb80997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.705+0000 7ffbbdb83700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbb815c1e0 0x7ffbb8099d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.705+0000 7ffbbdb83700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb806def0 con 0x7ffbb815c1e0 2024-01-27T02:38:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.705+0000 7ffbbdb83700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb8158900 con 0x7ffbb80b0180 2024-01-27T02:38:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.705+0000 7ffbbdb83700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb809a370 con 0x7ffbb815be10 2024-01-27T02:38:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.705+0000 7ffbb7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbb815c1e0 0x7ffbb8099d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.705+0000 7ffbb6ffd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbb815be10 0x7ffbb80997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.705+0000 7ffbb77fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbb80b0180 0x7ffbb8155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.705+0000 7ffbb7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbb815c1e0 0x7ffbb8099d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58870/0 (socket says 172.21.15.7:58870) 2024-01-27T02:38:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.706+0000 7ffbb7fff700 1 -- 172.21.15.7:0/1652364530 learned_addr learned my addr 172.21.15.7:0/1652364530 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.706+0000 7ffbb7fff700 1 -- 172.21.15.7:0/1652364530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbb815be10 msgr2=0x7ffbb80997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.706+0000 7ffbb7fff700 1 --2- 172.21.15.7:0/1652364530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbb815be10 0x7ffbb80997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.706+0000 7ffbb7fff700 1 -- 172.21.15.7:0/1652364530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbb80b0180 msgr2=0x7ffbb8155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.706+0000 7ffbb7fff700 1 --2- 172.21.15.7:0/1652364530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbb80b0180 0x7ffbb8155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.706+0000 7ffbb7fff700 1 -- 172.21.15.7:0/1652364530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba00094f0 con 0x7ffbb815c1e0 2024-01-27T02:38:36.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.706+0000 7ffbb7fff700 1 --2- 172.21.15.7:0/1652364530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbb815c1e0 0x7ffbb8099d10 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000cdb0 tx=0x7ffbb00077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:36.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.706+0000 7ffbaf7fe700 1 -- 172.21.15.7:0/1652364530 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb000a9d0 con 0x7ffbb815c1e0 2024-01-27T02:38:36.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.706+0000 7ffbaf7fe700 1 -- 172.21.15.7:0/1652364530 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb000ab30 con 0x7ffbb815c1e0 2024-01-27T02:38:36.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.706+0000 7ffbbdb83700 1 -- 172.21.15.7:0/1652364530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb809a650 con 0x7ffbb815c1e0 2024-01-27T02:38:36.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.706+0000 7ffbaf7fe700 1 -- 172.21.15.7:0/1652364530 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0017430 con 0x7ffbb815c1e0 2024-01-27T02:38:36.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.706+0000 7ffbbdb83700 1 -- 172.21.15.7:0/1652364530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb806db80 con 0x7ffbb815c1e0 2024-01-27T02:38:36.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.707+0000 7ffbbdb83700 1 -- 172.21.15.7:0/1652364530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb80b18e0 con 0x7ffbb815c1e0 2024-01-27T02:38:36.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.708+0000 7ffbaf7fe700 1 -- 172.21.15.7:0/1652364530 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ffbb0003020 con 0x7ffbb815c1e0 2024-01-27T02:38:36.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.708+0000 7ffbaf7fe700 1 --2- 172.21.15.7:0/1652364530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb98066f30 0x7ffb980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:36.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.708+0000 7ffbaf7fe700 1 -- 172.21.15.7:0/1652364530 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffbb0013070 con 0x7ffbb815c1e0 2024-01-27T02:38:36.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.708+0000 7ffbb77fe700 1 --2- 172.21.15.7:0/1652364530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb98066f30 0x7ffb980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:36.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.709+0000 7ffbb77fe700 1 --2- 172.21.15.7:0/1652364530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb98066f30 0x7ffb980693e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80afe80 tx=0x7ffba0017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.713+0000 7ffbaf7fe700 1 -- 172.21.15.7:0/1652364530 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffbb0051910 con 0x7ffbb815c1e0 2024-01-27T02:38:36.874 INFO:tasks.ceph.mds.a.smithi007.stderr:2024-01-27T02:38:36.873+0000 7fa3b6e5c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-27T02:38:36.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.912+0000 7ffbbdb83700 1 -- 172.21.15.7:0/1652364530 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffbb804c710 con 0x7ffb98066f30 2024-01-27T02:38:36.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.921+0000 7ffbaf7fe700 1 -- 172.21.15.7:0/1652364530 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282419 (secure 0 0 0) 0x7ffbb804c710 con 0x7ffb98066f30 2024-01-27T02:38:36.925 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:36.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.924+0000 7ffbbdb83700 1 -- 172.21.15.7:0/1652364530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb98066f30 msgr2=0x7ffb980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:36.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.924+0000 7ffbbdb83700 1 --2- 172.21.15.7:0/1652364530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb98066f30 0x7ffb980693e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80afe80 tx=0x7ffba0017000 comp rx=0 tx=0).stop 2024-01-27T02:38:36.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.924+0000 7ffbbdb83700 1 -- 172.21.15.7:0/1652364530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbb815c1e0 msgr2=0x7ffbb8099d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:36.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.924+0000 7ffbbdb83700 1 --2- 172.21.15.7:0/1652364530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbb815c1e0 0x7ffbb8099d10 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000cdb0 tx=0x7ffbb00077d0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.924+0000 7ffbbdb83700 1 -- 172.21.15.7:0/1652364530 shutdown_connections 2024-01-27T02:38:36.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.924+0000 7ffbbdb83700 1 --2- 172.21.15.7:0/1652364530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb98066f30 0x7ffb980693e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.924+0000 7ffbbdb83700 1 --2- 172.21.15.7:0/1652364530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbb815c1e0 0x7ffbb8099d10 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.924+0000 7ffbbdb83700 1 --2- 172.21.15.7:0/1652364530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbb80b0180 0x7ffbb8155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.924+0000 7ffbbdb83700 1 --2- 172.21.15.7:0/1652364530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbb815be10 0x7ffbb80997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:36.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.924+0000 7ffbbdb83700 1 -- 172.21.15.7:0/1652364530 >> 172.21.15.7:0/1652364530 conn(0x7ffbb80679b0 msgr2=0x7ffbb80acc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:36.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.924+0000 7ffbbdb83700 1 -- 172.21.15.7:0/1652364530 shutdown_connections 2024-01-27T02:38:36.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:36.924+0000 7ffbbdb83700 1 -- 172.21.15.7:0/1652364530 wait complete. 2024-01-27T02:38:36.928 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:38:36.938 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-01-27T02:38:35.667278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6694,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555805,"statfs":{"total":1600000000,"available":1593144606,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.780828"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897968+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:32:52.668655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895827+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:08.702500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896528+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:11:34.261074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896262+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:22:55.022123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.899959+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:30:08.176207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896813+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:02:36.439070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897666+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:14:25.858808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896102+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:51.023684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895424+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:57:59.292913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897605+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:55:31.456845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.894631+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:05:45.293447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896477+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:40:57.016595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895636+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:51:24.348901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895665+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:19:55.078411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.902105+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:52:50.516673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897186+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:15:59.844843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895485+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:11:34.261074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895678+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:20:34.579927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897488+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:23:46.870039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893388+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:16:13.624836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893364+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:39:26.847232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.894037+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:23:46.870039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893002+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:53:05.325595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893907+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:05:23.541565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893041+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:18.354062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896299+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:36:18.147300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897512+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:07:33.422861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893752+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:30:10.336341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893836+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:18:05.101790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896393+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:36:36.290157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895868+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:54:13.383710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.901471+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:02:27.766940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896480+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:47:09.052904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.900146+0000","last_change":"2024-01-27T02:38:30.900146+0000","last_active":"2024-01-27T02:38:30.900146+0000","last_peered":"2024-01-27T02:38:30.900146+0000","last_clean":"2024-01-27T02:38:30.900146+0000","last_became_active":"2024-01-27T02:38:30.899920+0000","last_became_peered":"2024-01-27T02:38:30.899920+0000","last_unstale":"2024-01-27T02:38:30.900146+0000","last_undegraded":"2024-01-27T02:38:30.900146+0000","last_fullsized":"2024-01-27T02:38:30.900146+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:02:26.763414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895285+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:01:42.972197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.900419+0000","last_change":"2024-01-27T02:38:30.900419+0000","last_active":"2024-01-27T02:38:30.900419+0000","last_peered":"2024-01-27T02:38:30.900419+0000","last_clean":"2024-01-27T02:38:30.900419+0000","last_became_active":"2024-01-27T02:38:30.900188+0000","last_became_peered":"2024-01-27T02:38:30.900188+0000","last_unstale":"2024-01-27T02:38:30.900419+0000","last_undegraded":"2024-01-27T02:38:30.900419+0000","last_fullsized":"2024-01-27T02:38:30.900419+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:16:12.621310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896381+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:32:20.165836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.907104+0000","last_change":"2024-01-27T02:38:30.907104+0000","last_active":"2024-01-27T02:38:30.907104+0000","last_peered":"2024-01-27T02:38:30.907104+0000","last_clean":"2024-01-27T02:38:30.907104+0000","last_became_active":"2024-01-27T02:38:30.906807+0000","last_became_peered":"2024-01-27T02:38:30.906807+0000","last_unstale":"2024-01-27T02:38:30.907104+0000","last_undegraded":"2024-01-27T02:38:30.907104+0000","last_fullsized":"2024-01-27T02:38:30.907104+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:16:05.064427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895895+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:05:45.293447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.906090+0000","last_change":"2024-01-27T02:38:30.906090+0000","last_active":"2024-01-27T02:38:30.906090+0000","last_peered":"2024-01-27T02:38:30.906090+0000","last_clean":"2024-01-27T02:38:30.906090+0000","last_became_active":"2024-01-27T02:38:30.905815+0000","last_became_peered":"2024-01-27T02:38:30.905815+0000","last_unstale":"2024-01-27T02:38:30.906090+0000","last_undegraded":"2024-01-27T02:38:30.906090+0000","last_fullsized":"2024-01-27T02:38:30.906090+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:42:16.605919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895621+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:00:23.173837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.908846+0000","last_change":"2024-01-27T02:38:30.908846+0000","last_active":"2024-01-27T02:38:30.908846+0000","last_peered":"2024-01-27T02:38:30.908846+0000","last_clean":"2024-01-27T02:38:30.908846+0000","last_became_active":"2024-01-27T02:38:30.908634+0000","last_became_peered":"2024-01-27T02:38:30.908634+0000","last_unstale":"2024-01-27T02:38:30.908846+0000","last_undegraded":"2024-01-27T02:38:30.908846+0000","last_fullsized":"2024-01-27T02:38:30.908846+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:44:21.414722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895115+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:01:42.972197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.897433+0000","last_change":"2024-01-27T02:38:30.897433+0000","last_active":"2024-01-27T02:38:30.897433+0000","last_peered":"2024-01-27T02:38:30.897433+0000","last_clean":"2024-01-27T02:38:30.897433+0000","last_became_active":"2024-01-27T02:38:30.897134+0000","last_became_peered":"2024-01-27T02:38:30.897134+0000","last_unstale":"2024-01-27T02:38:30.897433+0000","last_undegraded":"2024-01-27T02:38:30.897433+0000","last_fullsized":"2024-01-27T02:38:30.897433+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:18:04.098263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897157+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:32:05.199937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.903018+0000","last_change":"2024-01-27T02:38:30.903018+0000","last_active":"2024-01-27T02:38:30.903018+0000","last_peered":"2024-01-27T02:38:30.903018+0000","last_clean":"2024-01-27T02:38:30.903018+0000","last_became_active":"2024-01-27T02:38:30.902748+0000","last_became_peered":"2024-01-27T02:38:30.902748+0000","last_unstale":"2024-01-27T02:38:30.903018+0000","last_undegraded":"2024-01-27T02:38:30.903018+0000","last_fullsized":"2024-01-27T02:38:30.903018+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:14:24.855281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896050+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:56:12.566293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.903329+0000","last_change":"2024-01-27T02:38:30.903329+0000","last_active":"2024-01-27T02:38:30.903329+0000","last_peered":"2024-01-27T02:38:30.903329+0000","last_clean":"2024-01-27T02:38:30.903329+0000","last_became_active":"2024-01-27T02:38:30.903084+0000","last_became_peered":"2024-01-27T02:38:30.903084+0000","last_unstale":"2024-01-27T02:38:30.903329+0000","last_undegraded":"2024-01-27T02:38:30.903329+0000","last_fullsized":"2024-01-27T02:38:30.903329+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:11:33.257548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896315+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:36:50.676219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.904630+0000","last_change":"2024-01-27T02:38:30.904630+0000","last_active":"2024-01-27T02:38:30.904630+0000","last_peered":"2024-01-27T02:38:30.904630+0000","last_clean":"2024-01-27T02:38:30.904630+0000","last_became_active":"2024-01-27T02:38:30.904406+0000","last_became_peered":"2024-01-27T02:38:30.904406+0000","last_unstale":"2024-01-27T02:38:30.904630+0000","last_undegraded":"2024-01-27T02:38:30.904630+0000","last_fullsized":"2024-01-27T02:38:30.904630+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:22:44.330464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893390+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:05:45.293447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.893083+0000","last_change":"2024-01-27T02:38:30.893083+0000","last_active":"2024-01-27T02:38:30.893083+0000","last_peered":"2024-01-27T02:38:30.893083+0000","last_clean":"2024-01-27T02:38:30.893083+0000","last_became_active":"2024-01-27T02:38:30.892801+0000","last_became_peered":"2024-01-27T02:38:30.892801+0000","last_unstale":"2024-01-27T02:38:30.893083+0000","last_undegraded":"2024-01-27T02:38:30.893083+0000","last_fullsized":"2024-01-27T02:38:30.893083+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:51:23.345374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892794+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:08.702500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.905086+0000","last_change":"2024-01-27T02:38:30.905086+0000","last_active":"2024-01-27T02:38:30.905086+0000","last_peered":"2024-01-27T02:38:30.905086+0000","last_clean":"2024-01-27T02:38:30.905086+0000","last_became_active":"2024-01-27T02:38:30.904853+0000","last_became_peered":"2024-01-27T02:38:30.904853+0000","last_unstale":"2024-01-27T02:38:30.905086+0000","last_undegraded":"2024-01-27T02:38:30.905086+0000","last_fullsized":"2024-01-27T02:38:30.905086+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:54:14.886540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896562+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:03:26.614534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.905975+0000","last_change":"2024-01-27T02:38:30.905975+0000","last_active":"2024-01-27T02:38:30.905975+0000","last_peered":"2024-01-27T02:38:30.905975+0000","last_clean":"2024-01-27T02:38:30.905975+0000","last_became_active":"2024-01-27T02:38:30.904239+0000","last_became_peered":"2024-01-27T02:38:30.904239+0000","last_unstale":"2024-01-27T02:38:30.905975+0000","last_undegraded":"2024-01-27T02:38:30.905975+0000","last_fullsized":"2024-01-27T02:38:30.905975+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:05:22.538039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892200+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:05:45.293447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.904960+0000","last_change":"2024-01-27T02:38:30.904960+0000","last_active":"2024-01-27T02:38:30.904960+0000","last_peered":"2024-01-27T02:38:30.904960+0000","last_clean":"2024-01-27T02:38:30.904960+0000","last_became_active":"2024-01-27T02:38:30.904479+0000","last_became_peered":"2024-01-27T02:38:30.904479+0000","last_unstale":"2024-01-27T02:38:30.904960+0000","last_undegraded":"2024-01-27T02:38:30.904960+0000","last_fullsized":"2024-01-27T02:38:30.904960+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:20:17.775604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892827+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:11:34.261074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.901125+0000","last_change":"2024-01-27T02:38:30.901125+0000","last_active":"2024-01-27T02:38:30.901125+0000","last_peered":"2024-01-27T02:38:30.901125+0000","last_clean":"2024-01-27T02:38:30.901125+0000","last_became_active":"2024-01-27T02:38:30.900811+0000","last_became_peered":"2024-01-27T02:38:30.900811+0000","last_unstale":"2024-01-27T02:38:30.901125+0000","last_undegraded":"2024-01-27T02:38:30.901125+0000","last_fullsized":"2024-01-27T02:38:30.901125+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:18:04.098263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892350+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:20:34.579927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.904062+0000","last_change":"2024-01-27T02:38:30.904062+0000","last_active":"2024-01-27T02:38:30.904062+0000","last_peered":"2024-01-27T02:38:30.904062+0000","last_clean":"2024-01-27T02:38:30.904062+0000","last_became_active":"2024-01-27T02:38:30.903827+0000","last_became_peered":"2024-01-27T02:38:30.903827+0000","last_unstale":"2024-01-27T02:38:30.904062+0000","last_undegraded":"2024-01-27T02:38:30.904062+0000","last_fullsized":"2024-01-27T02:38:30.904062+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:22:30.041364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.900402+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:59:45.149668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.901317+0000","last_change":"2024-01-27T02:38:30.901317+0000","last_active":"2024-01-27T02:38:30.901317+0000","last_peered":"2024-01-27T02:38:30.901317+0000","last_clean":"2024-01-27T02:38:30.901317+0000","last_became_active":"2024-01-27T02:38:30.900905+0000","last_became_peered":"2024-01-27T02:38:30.900905+0000","last_unstale":"2024-01-27T02:38:30.901317+0000","last_undegraded":"2024-01-27T02:38:30.901317+0000","last_fullsized":"2024-01-27T02:38:30.901317+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:23:45.866513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895981+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:48:06.641759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.901405+0000","last_change":"2024-01-27T02:38:30.901405+0000","last_active":"2024-01-27T02:38:30.901405+0000","last_peered":"2024-01-27T02:38:30.901405+0000","last_clean":"2024-01-27T02:38:30.901405+0000","last_became_active":"2024-01-27T02:38:30.901168+0000","last_became_peered":"2024-01-27T02:38:30.901168+0000","last_unstale":"2024-01-27T02:38:30.901405+0000","last_undegraded":"2024-01-27T02:38:30.901405+0000","last_fullsized":"2024-01-27T02:38:30.901405+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:52:03.557302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896480+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:20:34.579927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.894794+0000","last_change":"2024-01-27T02:38:30.894794+0000","last_active":"2024-01-27T02:38:30.894794+0000","last_peered":"2024-01-27T02:38:30.894794+0000","last_clean":"2024-01-27T02:38:30.894794+0000","last_became_active":"2024-01-27T02:38:30.894519+0000","last_became_peered":"2024-01-27T02:38:30.894519+0000","last_unstale":"2024-01-27T02:38:30.894794+0000","last_undegraded":"2024-01-27T02:38:30.894794+0000","last_fullsized":"2024-01-27T02:38:30.894794+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:39:25.843705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896689+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:30:10.336341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.901197+0000","last_change":"2024-01-27T02:38:30.901197+0000","last_active":"2024-01-27T02:38:30.901197+0000","last_peered":"2024-01-27T02:38:30.901197+0000","last_clean":"2024-01-27T02:38:30.901197+0000","last_became_active":"2024-01-27T02:38:30.900235+0000","last_became_peered":"2024-01-27T02:38:30.900235+0000","last_unstale":"2024-01-27T02:38:30.901197+0000","last_undegraded":"2024-01-27T02:38:30.901197+0000","last_fullsized":"2024-01-27T02:38:30.901197+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:07:32.419335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.897132+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:05:23.541565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.904351+0000","last_change":"2024-01-27T02:38:30.904351+0000","last_active":"2024-01-27T02:38:30.904351+0000","last_peered":"2024-01-27T02:38:30.904351+0000","last_clean":"2024-01-27T02:38:30.904351+0000","last_became_active":"2024-01-27T02:38:30.904118+0000","last_became_peered":"2024-01-27T02:38:30.904118+0000","last_unstale":"2024-01-27T02:38:30.904351+0000","last_undegraded":"2024-01-27T02:38:30.904351+0000","last_fullsized":"2024-01-27T02:38:30.904351+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:53:19.168298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.895194+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:07.978917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.904812+0000","last_change":"2024-01-27T02:38:30.904812+0000","last_active":"2024-01-27T02:38:30.904812+0000","last_peered":"2024-01-27T02:38:30.904812+0000","last_clean":"2024-01-27T02:38:30.904812+0000","last_became_active":"2024-01-27T02:38:30.904483+0000","last_became_peered":"2024-01-27T02:38:30.904483+0000","last_unstale":"2024-01-27T02:38:30.904812+0000","last_undegraded":"2024-01-27T02:38:30.904812+0000","last_fullsized":"2024-01-27T02:38:30.904812+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:45:43.930079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896313+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:01:42.972197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.904780+0000","last_change":"2024-01-27T02:38:30.904780+0000","last_active":"2024-01-27T02:38:30.904780+0000","last_peered":"2024-01-27T02:38:30.904780+0000","last_clean":"2024-01-27T02:38:30.904780+0000","last_became_active":"2024-01-27T02:38:30.904179+0000","last_became_peered":"2024-01-27T02:38:30.904179+0000","last_unstale":"2024-01-27T02:38:30.904780+0000","last_undegraded":"2024-01-27T02:38:30.904780+0000","last_fullsized":"2024-01-27T02:38:30.904780+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:14:13.119229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892253+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:51.023684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.899295+0000","last_change":"2024-01-27T02:38:30.899295+0000","last_active":"2024-01-27T02:38:30.899295+0000","last_peered":"2024-01-27T02:38:30.899295+0000","last_clean":"2024-01-27T02:38:30.899295+0000","last_became_active":"2024-01-27T02:38:30.899015+0000","last_became_peered":"2024-01-27T02:38:30.899015+0000","last_unstale":"2024-01-27T02:38:30.899295+0000","last_undegraded":"2024-01-27T02:38:30.899295+0000","last_fullsized":"2024-01-27T02:38:30.899295+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:34:25.075555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.893779+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:51.023684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.902450+0000","last_change":"2024-01-27T02:38:30.902450+0000","last_active":"2024-01-27T02:38:30.902450+0000","last_peered":"2024-01-27T02:38:30.902450+0000","last_clean":"2024-01-27T02:38:30.902450+0000","last_became_active":"2024-01-27T02:38:30.902130+0000","last_became_peered":"2024-01-27T02:38:30.902130+0000","last_unstale":"2024-01-27T02:38:30.902450+0000","last_undegraded":"2024-01-27T02:38:30.902450+0000","last_fullsized":"2024-01-27T02:38:30.902450+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:39:25.843705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.894773+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:40:17.611985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.901304+0000","last_change":"2024-01-27T02:38:30.901304+0000","last_active":"2024-01-27T02:38:30.901304+0000","last_peered":"2024-01-27T02:38:30.901304+0000","last_clean":"2024-01-27T02:38:30.901304+0000","last_became_active":"2024-01-27T02:38:30.901065+0000","last_became_peered":"2024-01-27T02:38:30.901065+0000","last_unstale":"2024-01-27T02:38:30.901304+0000","last_undegraded":"2024-01-27T02:38:30.901304+0000","last_fullsized":"2024-01-27T02:38:30.901304+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:16:12.621310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.894797+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:02:03.382943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.901161+0000","last_change":"2024-01-27T02:38:30.901161+0000","last_active":"2024-01-27T02:38:30.901161+0000","last_peered":"2024-01-27T02:38:30.901161+0000","last_clean":"2024-01-27T02:38:30.901161+0000","last_became_active":"2024-01-27T02:38:30.899610+0000","last_became_peered":"2024-01-27T02:38:30.899610+0000","last_unstale":"2024-01-27T02:38:30.901161+0000","last_undegraded":"2024-01-27T02:38:30.901161+0000","last_fullsized":"2024-01-27T02:38:30.901161+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:30:09.332815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892858+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:19:55.078411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.904612+0000","last_change":"2024-01-27T02:38:30.904612+0000","last_active":"2024-01-27T02:38:30.904612+0000","last_peered":"2024-01-27T02:38:30.904612+0000","last_clean":"2024-01-27T02:38:30.904612+0000","last_became_active":"2024-01-27T02:38:30.904299+0000","last_became_peered":"2024-01-27T02:38:30.904299+0000","last_unstale":"2024-01-27T02:38:30.904612+0000","last_undegraded":"2024-01-27T02:38:30.904612+0000","last_fullsized":"2024-01-27T02:38:30.904612+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:16:12.621310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.894726+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:19:55.078411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.899705+0000","last_change":"2024-01-27T02:38:30.899705+0000","last_active":"2024-01-27T02:38:30.899705+0000","last_peered":"2024-01-27T02:38:30.899705+0000","last_clean":"2024-01-27T02:38:30.899705+0000","last_became_active":"2024-01-27T02:38:30.899453+0000","last_became_peered":"2024-01-27T02:38:30.899453+0000","last_unstale":"2024-01-27T02:38:30.899705+0000","last_undegraded":"2024-01-27T02:38:30.899705+0000","last_fullsized":"2024-01-27T02:38:30.899705+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:02:35.435543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892871+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:18:12.582908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.902907+0000","last_change":"2024-01-27T02:38:30.902907+0000","last_active":"2024-01-27T02:38:30.902907+0000","last_peered":"2024-01-27T02:38:30.902907+0000","last_clean":"2024-01-27T02:38:30.902907+0000","last_became_active":"2024-01-27T02:38:30.902691+0000","last_became_peered":"2024-01-27T02:38:30.902691+0000","last_unstale":"2024-01-27T02:38:30.902907+0000","last_undegraded":"2024-01-27T02:38:30.902907+0000","last_fullsized":"2024-01-27T02:38:30.902907+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:25:13.514723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.896780+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:08.702500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.906486+0000","last_change":"2024-01-27T02:38:30.906486+0000","last_active":"2024-01-27T02:38:30.906486+0000","last_peered":"2024-01-27T02:38:30.906486+0000","last_clean":"2024-01-27T02:38:30.906486+0000","last_became_active":"2024-01-27T02:38:30.906233+0000","last_became_peered":"2024-01-27T02:38:30.906233+0000","last_unstale":"2024-01-27T02:38:30.906486+0000","last_undegraded":"2024-01-27T02:38:30.906486+0000","last_fullsized":"2024-01-27T02:38:30.906486+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:27:08.109911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.894650+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:11:36.249589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.897972+0000","last_change":"2024-01-27T02:38:30.897972+0000","last_active":"2024-01-27T02:38:30.897972+0000","last_peered":"2024-01-27T02:38:30.897972+0000","last_clean":"2024-01-27T02:38:30.897972+0000","last_became_active":"2024-01-27T02:38:30.897716+0000","last_became_peered":"2024-01-27T02:38:30.897716+0000","last_unstale":"2024-01-27T02:38:30.897972+0000","last_undegraded":"2024-01-27T02:38:30.897972+0000","last_fullsized":"2024-01-27T02:38:30.897972+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:07.698973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:38:30.885345+0000","last_change":"2024-01-27T02:38:30.892275+0000","last_active":"2024-01-27T02:38:30.885345+0000","last_peered":"2024-01-27T02:38:30.885345+0000","last_clean":"2024-01-27T02:38:30.885345+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:38:30.885345+0000","last_undegraded":"2024-01-27T02:38:30.885345+0000","last_fullsized":"2024-01-27T02:38:30.885345+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:01:42.972197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":8,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-01-27T02:38:30.901349+0000","last_change":"2024-01-27T02:38:30.901349+0000","last_active":"2024-01-27T02:38:30.901349+0000","last_peered":"2024-01-27T02:38:30.901349+0000","last_clean":"2024-01-27T02:38:30.901349+0000","last_became_active":"2024-01-27T02:38:30.900973+0000","last_became_peered":"2024-01-27T02:38:30.900973+0000","last_unstale":"2024-01-27T02:38:30.901349+0000","last_undegraded":"2024-01-27T02:38:30.901349+0000","last_fullsized":"2024-01-27T02:38:30.901349+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:55:50.241360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":257,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-01-27T02:38:29.888544+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:38:29.888544+0000","last_peered":"2024-01-27T02:38:29.888544+0000","last_clean":"2024-01-27T02:38:29.888544+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:38:29.888544+0000","last_undegraded":"2024-01-27T02:38:29.888544+0000","last_fullsized":"2024-01-27T02:38:29.888544+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574929,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1270,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194041,"statfs":{"total":200000000,"available":198698734,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77400000000000002,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70299999999999996,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.60999999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.71999999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.42799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69999999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.65600000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.68200000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71399999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.63100000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.73099999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.48199999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.71799999999999997,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.71499999999999997}]},{"osd":4,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.47599999999999998,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.51100000000000001},"min":{"1min":0.33700000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.50600000000000001}]},{"osd":5,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51300000000000001,"15min":0.53400000000000003},"min":{"1min":0.35999999999999999,"5min":0.32100000000000001,"15min":0.307},"max":{"1min":0.68500000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.56299999999999994},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75800000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.73099999999999998}]},{"osd":6,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57199999999999995,"15min":0.59399999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.78800000000000003,"5min":0.80300000000000005,"15min":0.81299999999999994},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56100000000000005,"15min":0.57499999999999996},"min":{"1min":0.42499999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.73099999999999998,"5min":0.81599999999999995,"15min":0.90800000000000003},"last":0.70299999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574929,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":688,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194624,"statfs":{"total":200000000,"available":199295365,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.64900000000000002,"15min":0.54200000000000004},"min":{"1min":0.442,"5min":0.36499999999999999,"15min":0.29999999999999999},"max":{"1min":4.125,"5min":4.125,"15min":4.125},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60599999999999998,"15min":0.53600000000000003},"min":{"1min":0.45100000000000001,"5min":0.35699999999999998,"15min":0.32700000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.61099999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.68400000000000005,"15min":0.55200000000000005},"min":{"1min":0.433,"5min":0.41999999999999998,"15min":0.34100000000000003},"max":{"1min":8.2699999999999996,"5min":8.2699999999999996,"15min":8.2699999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":2.0409999999999999,"5min":0.90800000000000003,"15min":0.65600000000000003},"min":{"1min":0.46999999999999997,"5min":0.379,"15min":0.33800000000000002},"max":{"1min":26.411999999999999,"5min":26.411999999999999,"15min":26.411999999999999},"last":0.53500000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":0.77700000000000002,"15min":0.60899999999999999},"min":{"1min":0.45100000000000001,"5min":0.41199999999999998,"15min":0.39600000000000002},"max":{"1min":17.143999999999998,"5min":17.143999999999998,"15min":17.143999999999998},"last":0.435},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.64900000000000002,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":0.55900000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.63300000000000001,"15min":0.56799999999999995},"min":{"1min":0.45800000000000002,"5min":0.435,"15min":0.376},"max":{"1min":4.3689999999999998,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64100000000000001,"15min":0.58099999999999996},"min":{"1min":0.433,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.58499999999999996}]},{"osd":4,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.33200000000000002,"5min":0.32000000000000001,"15min":0.30199999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51600000000000001,"15min":0.54900000000000004},"min":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74299999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.38900000000000001}]},{"osd":5,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.52100000000000002,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76100000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56000000000000005,"15min":0.59999999999999998},"min":{"1min":0.41099999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77000000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.46100000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.58999999999999997,"15min":0.61099999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79400000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55800000000000005,"15min":0.58499999999999996},"min":{"1min":0.40000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71499999999999997,"5min":1.038,"15min":1.038},"last":0.5}]}]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":688,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194624,"statfs":{"total":200000000,"available":199295365,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.39700000000000002,"15min":0.377},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25800000000000001},"max":{"1min":0.50600000000000001,"5min":0.873,"15min":0.873},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.41899999999999998,"15min":0.39900000000000002},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.53100000000000003,"5min":1.121,"15min":1.121},"last":0.31900000000000001}]},{"osd":2,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.40999999999999998,"15min":0.36899999999999999},"min":{"1min":0.27200000000000002,"5min":0.26600000000000001,"15min":0.23999999999999999},"max":{"1min":0.48599999999999999,"5min":1.25,"15min":1.25},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.45600000000000002,"15min":0.432},"min":{"1min":0.32000000000000001,"5min":0.30199999999999999,"15min":0.29899999999999999},"max":{"1min":0.58599999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.44600000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.49399999999999999,"15min":0.46999999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.32100000000000001},"max":{"1min":0.66800000000000004,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.46999999999999997,"15min":0.45600000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.58199999999999996,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.38400000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.66300000000000003,"15min":0.63500000000000001},"min":{"1min":0.501,"5min":0.373,"15min":0.373},"max":{"1min":0.81599999999999995,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66200000000000003,"15min":0.63800000000000001},"min":{"1min":0.497,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.82599999999999996,"5min":1.026,"15min":1.026},"last":0.84999999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.68899999999999995,"15min":0.63900000000000001},"min":{"1min":0.51000000000000001,"5min":0.46200000000000002,"15min":0.44},"max":{"1min":0.80900000000000005,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.752},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69099999999999995,"15min":0.67600000000000005},"min":{"1min":0.53000000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83799999999999997,"5min":3.181,"15min":3.181},"last":0.89600000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71199999999999997,"15min":0.69699999999999995},"min":{"1min":0.51400000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.875,"5min":1.024,"15min":1.024},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71499999999999997,"15min":0.68000000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.85699999999999998,"5min":1.869,"15min":1.869},"last":0.86599999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.72699999999999998,"15min":0.68700000000000006},"min":{"1min":0.55800000000000005,"5min":0.495,"15min":0.47299999999999998},"max":{"1min":0.82599999999999996,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.73099999999999998,"15min":0.69399999999999995},"min":{"1min":0.56000000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.86799999999999999,"5min":1.708,"15min":1.708},"last":0.79500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1270,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194041,"statfs":{"total":200000000,"available":198698734,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39000000000000001},"min":{"1min":0.253,"5min":0.24399999999999999,"15min":0.23799999999999999},"max":{"1min":0.52100000000000002,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.41999999999999998,"15min":0.41599999999999998},"min":{"1min":0.29599999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.64200000000000002,"5min":0.65200000000000002,"15min":0.68400000000000005},"last":0.60399999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.437,"15min":0.42199999999999999},"min":{"1min":0.30499999999999999,"5min":0.28699999999999998,"15min":0.28199999999999997},"max":{"1min":0.60299999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.435},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.44900000000000001,"15min":0.435},"min":{"1min":0.32400000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.628,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.47999999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42799999999999999,"15min":0.43099999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58199999999999996,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.629},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48499999999999999,"15min":0.47499999999999998},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69399999999999995,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.55900000000000005}]},{"osd":4,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65900000000000003,"15min":0.64100000000000001},"min":{"1min":0.49399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91700000000000004,"5min":0.997,"15min":0.997},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.44700000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92100000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.67300000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67700000000000005,"15min":0.65700000000000003},"min":{"1min":0.496,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88600000000000001,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.68100000000000005},"min":{"1min":0.53100000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.82099999999999995,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.748}]},{"osd":6,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.57299999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.80300000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69599999999999995,"15min":0.68000000000000005},"min":{"1min":0.51400000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94299999999999995,"5min":1.04,"15min":1.04},"last":0.65800000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68899999999999995,"15min":0.69199999999999995},"min":{"1min":0.45900000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90800000000000003,"5min":1.014,"15min":1.014},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68700000000000006,"15min":0.69299999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.83899999999999997,"5min":0.95799999999999996,"15min":0.97299999999999998},"last":0.72999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":694,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194618,"statfs":{"total":200000000,"available":199289102,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.41399999999999998,"15min":0.40899999999999997},"min":{"1min":0.23599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.58099999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.308},{"interface":"front","average":{"1min":0.443,"5min":0.45700000000000002,"15min":0.62},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.30499999999999999},"max":{"1min":0.64700000000000002,"5min":0.72799999999999998,"15min":6.2729999999999997},"last":0.38200000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.42999999999999999,"15min":0.42299999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.71899999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.433,"15min":0.44500000000000001},"min":{"1min":0.32600000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.57999999999999996,"5min":0.65300000000000002,"15min":0.70099999999999996},"last":0.42599999999999999}]},{"osd":3,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42399999999999999,"15min":0.41599999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.27800000000000002},"max":{"1min":0.64700000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.372},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.48699999999999999,"15min":0.48499999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.627,"5min":0.76500000000000001,"15min":0.80800000000000005},"last":0.40999999999999998}]},{"osd":4,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68799999999999994,"15min":0.65800000000000003},"min":{"1min":0.46800000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93999999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70599999999999996,"15min":0.79000000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.006,"5min":1.006,"15min":3.581},"last":0.64400000000000002}]},{"osd":5,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65300000000000002,"15min":0.73499999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":3.5369999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92200000000000004,"5min":0.96599999999999997,"15min":1.0529999999999999},"last":0.52200000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.45000000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.51300000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97499999999999998,"5min":1.0189999999999999,"15min":1.0209999999999999},"last":0.503}]},{"osd":7,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.68300000000000005,"15min":0.67800000000000005},"min":{"1min":0.50800000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95599999999999996,"5min":1.03,"15min":1.03},"last":0.62},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.69799999999999995,"15min":0.68999999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.42699999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.55200000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":694,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194618,"statfs":{"total":200000000,"available":199289102,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.45600000000000002,"15min":0.38200000000000001},"min":{"1min":0.28999999999999998,"5min":0.253,"15min":0.253},"max":{"1min":4.7380000000000004,"5min":4.7380000000000004,"15min":4.7380000000000004},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.41599999999999998,"15min":0.39800000000000002},"min":{"1min":0.27600000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.35399999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.47199999999999998,"15min":0.41199999999999998},"min":{"1min":0.27900000000000003,"5min":0.27400000000000002,"15min":0.22900000000000001},"max":{"1min":4.5739999999999998,"5min":4.5739999999999998,"15min":4.5739999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.52400000000000002,"15min":0.44700000000000001},"min":{"1min":0.311,"5min":0.30199999999999999,"15min":0.27900000000000003},"max":{"1min":5.0529999999999999,"5min":5.0529999999999999,"15min":5.0529999999999999},"last":0.39700000000000002}]},{"osd":2,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.48499999999999999,"15min":0.439},"min":{"1min":0.34799999999999998,"5min":0.33600000000000002,"15min":0.28100000000000003},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.55500000000000005,"15min":0.47399999999999998},"min":{"1min":0.375,"5min":0.35299999999999998,"15min":0.34599999999999997},"max":{"1min":5.1879999999999997,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":0.432}]},{"osd":4,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.69399999999999995,"15min":0.63600000000000001},"min":{"1min":0.42799999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0629999999999999,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66500000000000004,"15min":0.61399999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.79300000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.67800000000000005,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.006,"5min":0.78500000000000003,"15min":0.70299999999999996},"min":{"1min":0.56200000000000006,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":4.7960000000000003,"5min":4.7960000000000003,"15min":4.7960000000000003},"last":0.68400000000000005}]},{"osd":6,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.73499999999999999,"15min":0.67800000000000005},"min":{"1min":0.58899999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":5.0069999999999997,"5min":5.0069999999999997,"15min":5.0069999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.68100000000000005,"15min":0.65400000000000003},"min":{"1min":0.504,"5min":0.435,"15min":0.435},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.74399999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.74299999999999999,"15min":0.66300000000000003},"min":{"1min":0.57699999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.139,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.71299999999999997,"15min":0.67600000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.81299999999999994}]}]},{"osd":4,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":694,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194618,"statfs":{"total":200000000,"available":199289102,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.34599999999999997,"5min":0.33400000000000002,"15min":0.314},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.64600000000000002,"15min":0.60999999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":7.0670000000000002,"5min":7.0670000000000002,"15min":7.0670000000000002},"last":0.76000000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.64700000000000002,"15min":0.60899999999999999},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":5.1529999999999996,"5min":5.1529999999999996,"15min":5.1529999999999996},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.626,"15min":0.60399999999999998},"min":{"1min":0.43099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.0419999999999998,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.751}]},{"osd":2,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.64900000000000002,"15min":0.60999999999999999},"min":{"1min":0.46899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":5.0019999999999998,"5min":5.0019999999999998,"15min":5.0019999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.67600000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.64700000000000002,"15min":0.60099999999999998},"min":{"1min":0.42099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.7220000000000004,"5min":6.7220000000000004,"15min":6.7220000000000004},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.51000000000000001,"5min":0.44900000000000001,"15min":0.41399999999999998},"max":{"1min":6.5510000000000002,"5min":6.5510000000000002,"15min":6.5510000000000002},"last":0.73199999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.503},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.56200000000000006,"15min":0.51400000000000001},"min":{"1min":0.38500000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":6.9269999999999996,"5min":6.9269999999999996,"15min":6.9269999999999996},"last":0.57099999999999995}]},{"osd":6,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.55400000000000005,"15min":0.51700000000000002},"min":{"1min":0.40100000000000002,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":5.2229999999999999,"5min":5.2229999999999999,"15min":5.2229999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.71099999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.60799999999999998,"15min":0.57199999999999995},"min":{"1min":0.34799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":7.5339999999999998,"5min":7.5339999999999998,"15min":7.5339999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.63200000000000001,"15min":0.58999999999999997},"min":{"1min":0.44600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.9550000000000001,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":0.65900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574929,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":694,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194618,"statfs":{"total":200000000,"available":199289102,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56100000000000005,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.84199999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60599999999999998,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.34499999999999997,"15min":0.27600000000000002},"max":{"1min":0.78900000000000003,"5min":3.242,"15min":3.242},"last":0.69499999999999995}]},{"osd":1,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63,"15min":0.60999999999999999},"min":{"1min":0.47299999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80100000000000005,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.62,"15min":0.64700000000000002},"min":{"1min":0.46899999999999997,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77600000000000002,"5min":0.88500000000000001,"15min":0.95499999999999996},"last":0.63700000000000001}]},{"osd":2,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61699999999999999,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.372},"max":{"1min":0.80900000000000005,"5min":3.04,"15min":3.04},"last":0.62},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59599999999999997,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74299999999999999,"5min":0.85899999999999999,"15min":0.89400000000000002},"last":0.59999999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65000000000000002,"15min":0.67300000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82099999999999995,"5min":4.032,"15min":4.032},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.70599999999999996,"15min":0.67900000000000005},"min":{"1min":0.46000000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.78800000000000003,"5min":4.282,"15min":4.282},"last":0.70899999999999996}]},{"osd":4,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.371,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.747,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.499,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76300000000000001,"5min":0.79800000000000004,"15min":1.101},"last":0.51100000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.502,"15min":0.55200000000000005},"min":{"1min":0.38600000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69199999999999995,"5min":0.73199999999999998,"15min":1.7969999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.58099999999999996},"min":{"1min":0.433,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83399999999999996,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.53100000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55100000000000005,"15min":0.56599999999999995},"min":{"1min":0.45000000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68899999999999995,"5min":0.91100000000000003,"15min":1.008},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.61499999999999999,"15min":0.65400000000000003},"min":{"1min":0.40699999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78400000000000003,"5min":2,"15min":2},"last":0.67000000000000004}]}]}],"pool_statfs":[]}} 2024-01-27T02:38:36.943 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3f is not active+clean 2024-01-27T02:38:36.943 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.897968+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T12:32:52.668655+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:38:36.943 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3e is not active+clean 2024-01-27T02:38:36.943 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.895827+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:15:08.702500+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:38:36.943 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3d is not active+clean 2024-01-27T02:38:36.943 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.896528+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T06:11:34.261074+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:38:36.944 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3c is not active+clean 2024-01-27T02:38:36.944 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.896262+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T09:22:55.022123+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:38:36.944 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3b is not active+clean 2024-01-27T02:38:36.944 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.899959+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T05:30:08.176207+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:38:36.944 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3a is not active+clean 2024-01-27T02:38:36.944 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.896813+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T04:02:36.439070+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:38:36.944 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.39 is not active+clean 2024-01-27T02:38:36.944 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.897666+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T05:14:25.858808+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:38:36.944 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.38 is not active+clean 2024-01-27T02:38:36.944 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.896102+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:35:51.023684+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:38:36.945 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.37 is not active+clean 2024-01-27T02:38:36.945 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.895424+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T05:57:59.292913+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:38:36.945 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.36 is not active+clean 2024-01-27T02:38:36.945 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.897605+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T03:55:31.456845+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:38:36.945 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.35 is not active+clean 2024-01-27T02:38:36.945 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.894631+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T04:05:45.293447+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:38:36.945 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.34 is not active+clean 2024-01-27T02:38:36.945 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.896477+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T04:40:57.016595+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:38:36.945 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.33 is not active+clean 2024-01-27T02:38:36.945 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.895636+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T09:51:24.348901+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:38:36.945 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.32 is not active+clean 2024-01-27T02:38:36.945 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.895665+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T08:19:55.078411+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:38:36.946 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.31 is not active+clean 2024-01-27T02:38:36.946 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.902105+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:52:50.516673+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:38:36.946 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.30 is not active+clean 2024-01-27T02:38:36.946 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.897186+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T03:15:59.844843+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:38:36.946 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2f is not active+clean 2024-01-27T02:38:36.946 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.895485+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T06:11:34.261074+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:38:36.946 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2e is not active+clean 2024-01-27T02:38:36.946 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.895678+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T08:20:34.579927+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:38:36.946 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2d is not active+clean 2024-01-27T02:38:36.946 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.897488+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T04:23:46.870039+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:38:36.946 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2c is not active+clean 2024-01-27T02:38:36.947 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.893388+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T13:16:13.624836+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:38:36.947 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2b is not active+clean 2024-01-27T02:38:36.947 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.893364+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T08:39:26.847232+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:38:36.947 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2a is not active+clean 2024-01-27T02:38:36.947 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.894037+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T04:23:46.870039+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:38:36.947 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.29 is not active+clean 2024-01-27T02:38:36.947 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.893002+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T13:53:05.325595+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:38:36.947 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.28 is not active+clean 2024-01-27T02:38:36.947 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.893907+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T09:05:23.541565+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:38:36.947 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.27 is not active+clean 2024-01-27T02:38:36.947 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.893041+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:39:18.354062+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:38:36.948 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.26 is not active+clean 2024-01-27T02:38:36.948 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.896299+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T14:36:18.147300+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:38:36.948 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.25 is not active+clean 2024-01-27T02:38:36.948 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.897512+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T12:07:33.422861+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:38:36.948 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.24 is not active+clean 2024-01-27T02:38:36.948 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.893752+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:30:10.336341+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:38:36.948 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.23 is not active+clean 2024-01-27T02:38:36.948 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.893836+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T14:18:05.101790+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:38:36.948 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.22 is not active+clean 2024-01-27T02:38:36.948 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.896393+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:36:36.290157+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:38:36.948 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.21 is not active+clean 2024-01-27T02:38:36.949 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.895868+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T04:54:13.383710+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:38:36.949 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.20 is not active+clean 2024-01-27T02:38:36.949 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.901471+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T09:02:27.766940+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:38:36.949 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1e is not active+clean 2024-01-27T02:38:36.949 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.896480+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:47:09.052904+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:38:36.949 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1f is not active+clean 2024-01-27T02:38:36.949 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.895285+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T12:01:42.972197+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:38:36.949 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1c is not active+clean 2024-01-27T02:38:36.949 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.896381+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T05:32:20.165836+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:38:36.949 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1d is not active+clean 2024-01-27T02:38:36.949 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.895895+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T04:05:45.293447+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:38:36.950 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1a is not active+clean 2024-01-27T02:38:36.950 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.895621+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T12:00:23.173837+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:38:36.950 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1b is not active+clean 2024-01-27T02:38:36.950 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.895115+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T12:01:42.972197+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:38:36.950 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.18 is not active+clean 2024-01-27T02:38:36.950 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.897157+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T03:32:05.199937+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:38:36.950 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.19 is not active+clean 2024-01-27T02:38:36.950 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.896050+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T12:56:12.566293+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:38:36.950 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.16 is not active+clean 2024-01-27T02:38:36.950 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.896315+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T10:36:50.676219+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:38:36.950 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.17 is not active+clean 2024-01-27T02:38:36.951 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.893390+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T04:05:45.293447+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:38:36.951 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.14 is not active+clean 2024-01-27T02:38:36.951 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.892794+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:15:08.702500+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:38:36.951 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.15 is not active+clean 2024-01-27T02:38:36.951 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.896562+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T06:03:26.614534+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-27T02:38:36.951 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.12 is not active+clean 2024-01-27T02:38:36.951 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.892200+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T04:05:45.293447+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:38:36.951 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.13 is not active+clean 2024-01-27T02:38:36.951 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.892827+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T06:11:34.261074+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:38:36.951 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.11 is not active+clean 2024-01-27T02:38:36.951 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.892350+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T08:20:34.579927+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:38:36.952 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.e is not active+clean 2024-01-27T02:38:36.952 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.900402+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:59:45.149668+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-27T02:38:36.952 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.f is not active+clean 2024-01-27T02:38:36.952 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.895981+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T13:48:06.641759+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-27T02:38:36.952 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.c is not active+clean 2024-01-27T02:38:36.952 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.896480+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T08:20:34.579927+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:38:36.952 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.d is not active+clean 2024-01-27T02:38:36.952 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.896689+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:30:10.336341+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:38:36.952 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.a is not active+clean 2024-01-27T02:38:36.952 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.897132+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T09:05:23.541565+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:38:36.952 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.b is not active+clean 2024-01-27T02:38:36.953 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.895194+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T08:56:07.978917+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:38:36.953 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.8 is not active+clean 2024-01-27T02:38:36.953 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.896313+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T12:01:42.972197+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:38:36.953 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.9 is not active+clean 2024-01-27T02:38:36.953 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.892253+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:35:51.023684+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:38:36.953 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.6 is not active+clean 2024-01-27T02:38:36.953 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.893779+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:35:51.023684+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:38:36.953 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.7 is not active+clean 2024-01-27T02:38:36.953 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.894773+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T07:40:17.611985+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:38:36.953 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.4 is not active+clean 2024-01-27T02:38:36.953 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.894797+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T04:02:03.382943+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:38:36.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.5 is not active+clean 2024-01-27T02:38:36.954 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.892858+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T08:19:55.078411+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-27T02:38:36.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2 is not active+clean 2024-01-27T02:38:36.954 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.894726+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T08:19:55.078411+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:38:36.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3 is not active+clean 2024-01-27T02:38:36.954 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.892871+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T03:18:12.582908+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-27T02:38:36.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.0 is not active+clean 2024-01-27T02:38:36.954 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.896780+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T11:15:08.702500+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-27T02:38:36.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1 is not active+clean 2024-01-27T02:38:36.954 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.894650+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T05:11:36.249589+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-27T02:38:36.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.10 is not active+clean 2024-01-27T02:38:36.955 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-27T02:38:30.885345+0000', 'last_change': '2024-01-27T02:38:30.892275+0000', 'last_active': '2024-01-27T02:38:30.885345+0000', 'last_peered': '2024-01-27T02:38:30.885345+0000', 'last_clean': '2024-01-27T02:38:30.885345+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-27T02:38:30.885345+0000', 'last_undegraded': '2024-01-27T02:38:30.885345+0000', 'last_fullsized': '2024-01-27T02:38:30.885345+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'last_clean_scrub_stamp': '2024-01-27T02:38:30.885345+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-28T12:01:42.972197+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-27T02:38:36.955 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-01-27T02:38:37.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.111+0000 7f6c7c8b9700 1 Processor -- start 2024-01-27T02:38:37.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.111+0000 7f6c7c8b9700 1 -- start start 2024-01-27T02:38:37.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.112+0000 7f6c7c8b9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6c781581c0 0x7f6c78158590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:37.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.112+0000 7f6c7c8b9700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6c780cd390 0x7f6c780c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:37.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.112+0000 7f6c7c8b9700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c780cfbf0 con 0x7f6c780cc830 2024-01-27T02:38:37.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.112+0000 7f6c7c8b9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c780ca7f0 con 0x7f6c781581c0 2024-01-27T02:38:37.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.112+0000 7f6c7c8b9700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c780b24a0 con 0x7f6c780cd390 2024-01-27T02:38:37.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.112+0000 7f6c7659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6c781581c0 0x7f6c78158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:37.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.112+0000 7f6c7659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6c781581c0 0x7f6c78158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58876/0 (socket says 172.21.15.7:58876) 2024-01-27T02:38:37.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.112+0000 7f6c7659c700 1 -- 172.21.15.7:0/3405415270 learned_addr learned my addr 172.21.15.7:0/3405415270 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:37.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.112+0000 7f6c76d9d700 1 --2- 172.21.15.7:0/3405415270 >> v2:172.21.15.188:3300/0 conn(0x7f6c780cd390 0x7f6c780c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:37.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.113+0000 7f6c7659c700 1 -- 172.21.15.7:0/3405415270 >> v2:172.21.15.188:3300/0 conn(0x7f6c780cd390 msgr2=0x7f6c780c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.113+0000 7f6c7659c700 1 --2- 172.21.15.7:0/3405415270 >> v2:172.21.15.188:3300/0 conn(0x7f6c780cd390 0x7f6c780c92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.113+0000 7f6c7659c700 1 -- 172.21.15.7:0/3405415270 >> v1:172.21.15.188:6789/0 conn(0x7f6c780cc830 legacy=0x7f6c780ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.113+0000 7f6c7659c700 1 -- 172.21.15.7:0/3405415270 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c780b1b70 con 0x7f6c781581c0 2024-01-27T02:38:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.113+0000 7f6c7659c700 1 --2- 172.21.15.7:0/3405415270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c781581c0 0x7f6c78158590 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6c7806b160 tx=0x7f6c68009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0120206bc842e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.113+0000 7f6c7559a700 1 -- 172.21.15.7:0/3405415270 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c68005c10 con 0x7f6c781581c0 2024-01-27T02:38:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.113+0000 7f6c7559a700 1 -- 172.21.15.7:0/3405415270 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c68005d70 con 0x7f6c781581c0 2024-01-27T02:38:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.113+0000 7f6c7559a700 1 -- 172.21.15.7:0/3405415270 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c68006260 con 0x7f6c781581c0 2024-01-27T02:38:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.113+0000 7f6c7c8b9700 1 -- 172.21.15.7:0/3405415270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c781581c0 msgr2=0x7f6c78158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.113+0000 7f6c7c8b9700 1 --2- 172.21.15.7:0/3405415270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c781581c0 0x7f6c78158590 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6c7806b160 tx=0x7f6c68009220 comp rx=0 tx=0).stop 2024-01-27T02:38:37.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.114+0000 7f6c7c8b9700 1 -- 172.21.15.7:0/3405415270 shutdown_connections 2024-01-27T02:38:37.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.114+0000 7f6c7c8b9700 1 --2- 172.21.15.7:0/3405415270 >> v2:172.21.15.188:3300/0 conn(0x7f6c780cd390 0x7f6c780c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:37.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.114+0000 7f6c7c8b9700 1 --2- 172.21.15.7:0/3405415270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c781581c0 0x7f6c78158590 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:37.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.114+0000 7f6c7c8b9700 1 -- 172.21.15.7:0/3405415270 >> 172.21.15.7:0/3405415270 conn(0x7f6c780711c0 msgr2=0x7f6c780ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:37.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.114+0000 7f6c7c8b9700 1 -- 172.21.15.7:0/3405415270 shutdown_connections 2024-01-27T02:38:37.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.114+0000 7f6c7c8b9700 1 -- 172.21.15.7:0/3405415270 wait complete. 2024-01-27T02:38:37.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.115+0000 7f6c7c8b9700 1 Processor -- start 2024-01-27T02:38:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.115+0000 7f6c7c8b9700 1 -- start start 2024-01-27T02:38:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.116+0000 7f6c7c8b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c780cc830 0x7f6c78155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.116+0000 7f6c7c8b9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c780cd390 0x7f6c7815c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.116+0000 7f6c7c8b9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c781581c0 0x7f6c78098960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.116+0000 7f6c7c8b9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c78159300 con 0x7f6c780cc830 2024-01-27T02:38:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.116+0000 7f6c7c8b9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7815d330 con 0x7f6c781581c0 2024-01-27T02:38:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.116+0000 7f6c7c8b9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7815d610 con 0x7f6c780cd390 2024-01-27T02:38:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.116+0000 7f6c7659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c780cc830 0x7f6c78155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.116+0000 7f6c7659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c780cc830 0x7f6c78155200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58890/0 (socket says 172.21.15.7:58890) 2024-01-27T02:38:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.116+0000 7f6c7659c700 1 -- 172.21.15.7:0/4261847791 learned_addr learned my addr 172.21.15.7:0/4261847791 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.116+0000 7f6c76d9d700 1 --2- 172.21.15.7:0/4261847791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c781581c0 0x7f6c78098960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.116+0000 7f6c75d9b700 1 --2- 172.21.15.7:0/4261847791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c780cd390 0x7f6c7815c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.116+0000 7f6c7659c700 1 -- 172.21.15.7:0/4261847791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c780cd390 msgr2=0x7f6c7815c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.116+0000 7f6c7659c700 1 --2- 172.21.15.7:0/4261847791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c780cd390 0x7f6c7815c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.116+0000 7f6c7659c700 1 -- 172.21.15.7:0/4261847791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c781581c0 msgr2=0x7f6c78098960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.116+0000 7f6c7659c700 1 --2- 172.21.15.7:0/4261847791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c781581c0 0x7f6c78098960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.116+0000 7f6c7659c700 1 -- 172.21.15.7:0/4261847791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c68009480 con 0x7f6c780cc830 2024-01-27T02:38:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.116+0000 7f6c7659c700 1 --2- 172.21.15.7:0/4261847791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c780cc830 0x7f6c78155200 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800b0e0 tx=0x7f6c68016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.116+0000 7f6c667fc700 1 -- 172.21.15.7:0/4261847791 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6800b850 con 0x7f6c780cc830 2024-01-27T02:38:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.117+0000 7f6c667fc700 1 -- 172.21.15.7:0/4261847791 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c68020070 con 0x7f6c780cc830 2024-01-27T02:38:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.117+0000 7f6c7c8b9700 1 -- 172.21.15.7:0/4261847791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c78158f90 con 0x7f6c780cc830 2024-01-27T02:38:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.117+0000 7f6c667fc700 1 -- 172.21.15.7:0/4261847791 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6800bbe0 con 0x7f6c780cc830 2024-01-27T02:38:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.117+0000 7f6c7c8b9700 1 -- 172.21.15.7:0/4261847791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7809bd40 con 0x7f6c780cc830 2024-01-27T02:38:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.117+0000 7f6c7c8b9700 1 -- 172.21.15.7:0/4261847791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c78158df0 con 0x7f6c780cc830 2024-01-27T02:38:37.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.118+0000 7f6c667fc700 1 -- 172.21.15.7:0/4261847791 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6c68022030 con 0x7f6c780cc830 2024-01-27T02:38:37.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.118+0000 7f6c667fc700 1 --2- 172.21.15.7:0/4261847791 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6c58066f30 0x7f6c580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:37.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.118+0000 7f6c667fc700 1 -- 172.21.15.7:0/4261847791 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6c68086100 con 0x7f6c780cc830 2024-01-27T02:38:37.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.119+0000 7f6c75d9b700 1 --2- 172.21.15.7:0/4261847791 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6c58066f30 0x7f6c580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:37.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.119+0000 7f6c75d9b700 1 --2- 172.21.15.7:0/4261847791 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6c58066f30 0x7f6c580693e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6c7809c600 tx=0x7f6c6c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:37.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.123+0000 7f6c667fc700 1 -- 172.21.15.7:0/4261847791 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c78158df0 con 0x7f6c780cc830 2024-01-27T02:38:37.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.412+0000 7f6c7c8b9700 1 -- 172.21.15.7:0/4261847791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6c78158df0 con 0x7f6c780cc830 2024-01-27T02:38:37.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.413+0000 7f6c667fc700 1 -- 172.21.15.7:0/4261847791 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1449 (secure 0 0 0) 0x7f6c78158df0 con 0x7f6c780cc830 2024-01-27T02:38:37.414 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:37.414 INFO:teuthology.orchestra.run.smithi007.stdout:{"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":407,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":132,"num_osds":8,"num_up_osds":8,"osd_up_since":1706322715,"num_in_osds":8,"osd_in_since":1706322712,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":6855394,"bytes_avail":1593144606,"bytes_total":1600000000,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":68,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":7029}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-01-27T02:37:41.655442+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-01-27T02:38:37.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.415+0000 7f6c7c8b9700 1 -- 172.21.15.7:0/4261847791 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6c58066f30 msgr2=0x7f6c580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:37.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.415+0000 7f6c7c8b9700 1 --2- 172.21.15.7:0/4261847791 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6c58066f30 0x7f6c580693e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6c7809c600 tx=0x7f6c6c009040 comp rx=0 tx=0).stop 2024-01-27T02:38:37.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.415+0000 7f6c7c8b9700 1 -- 172.21.15.7:0/4261847791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c780cc830 msgr2=0x7f6c78155200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:37.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.415+0000 7f6c7c8b9700 1 --2- 172.21.15.7:0/4261847791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c780cc830 0x7f6c78155200 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800b0e0 tx=0x7f6c68016000 comp rx=0 tx=0).stop 2024-01-27T02:38:37.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.415+0000 7f6c7c8b9700 1 -- 172.21.15.7:0/4261847791 shutdown_connections 2024-01-27T02:38:37.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.415+0000 7f6c7c8b9700 1 --2- 172.21.15.7:0/4261847791 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6c58066f30 0x7f6c580693e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:37.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.415+0000 7f6c7c8b9700 1 --2- 172.21.15.7:0/4261847791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c781581c0 0x7f6c78098960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:37.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.415+0000 7f6c7c8b9700 1 --2- 172.21.15.7:0/4261847791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c780cc830 0x7f6c78155200 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:37.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.415+0000 7f6c7c8b9700 1 --2- 172.21.15.7:0/4261847791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c780cd390 0x7f6c7815c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:37.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.415+0000 7f6c7c8b9700 1 -- 172.21.15.7:0/4261847791 >> 172.21.15.7:0/4261847791 conn(0x7f6c780711c0 msgr2=0x7f6c780b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:37.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.415+0000 7f6c7c8b9700 1 -- 172.21.15.7:0/4261847791 shutdown_connections 2024-01-27T02:38:37.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.415+0000 7f6c7c8b9700 1 -- 172.21.15.7:0/4261847791 wait complete. 2024-01-27T02:38:37.429 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-01-27T02:38:37.429 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:38:37.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.560+0000 7f3ea28b9700 1 Processor -- start 2024-01-27T02:38:37.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.560+0000 7f3ea28b9700 1 -- start start 2024-01-27T02:38:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.560+0000 7f3ea28b9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e9c159f40 0x7f3e9c0c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.560+0000 7f3ea28b9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3e9c0c3a90 0x7f3e9c0aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.560+0000 7f3ea28b9700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3e9c15a310 0x7f3e9c0aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.560+0000 7f3ea28b9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e9c0c4550 con 0x7f3e9c0c3a90 2024-01-27T02:38:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.560+0000 7f3ea28b9700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e9c0c30c0 con 0x7f3e9c15a310 2024-01-27T02:38:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.560+0000 7f3ea28b9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c15e960 con 0x7f3e9c159f40 2024-01-27T02:38:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.561+0000 7f3e9b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3e9c0c3a90 0x7f3e9c0aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.561+0000 7f3e9b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3e9c0c3a90 0x7f3e9c0aa8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58906/0 (socket says 172.21.15.7:58906) 2024-01-27T02:38:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.561+0000 7f3e9b7fe700 1 -- 172.21.15.7:0/338132837 learned_addr learned my addr 172.21.15.7:0/338132837 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.561+0000 7f3e9bfff700 1 --2- 172.21.15.7:0/338132837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e9c159f40 0x7f3e9c0c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.561+0000 7f3ea0e56700 1 --2- 172.21.15.7:0/338132837 >> v2:172.21.15.188:3300/0 conn(0x7f3e9c15a310 0x7f3e9c0aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.561+0000 7f3e9b7fe700 1 -- 172.21.15.7:0/338132837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e9c159f40 msgr2=0x7f3e9c0c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.561+0000 7f3e9b7fe700 1 --2- 172.21.15.7:0/338132837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e9c159f40 0x7f3e9c0c3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.561+0000 7f3e9b7fe700 1 -- 172.21.15.7:0/338132837 >> v2:172.21.15.188:3300/0 conn(0x7f3e9c15a310 msgr2=0x7f3e9c0aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.561+0000 7f3e9b7fe700 1 --2- 172.21.15.7:0/338132837 >> v2:172.21.15.188:3300/0 conn(0x7f3e9c15a310 0x7f3e9c0aadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.561+0000 7f3e9b7fe700 1 -- 172.21.15.7:0/338132837 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c15ec40 con 0x7f3e9c0c3a90 2024-01-27T02:38:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.561+0000 7f3e9b7fe700 1 --2- 172.21.15.7:0/338132837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e9c0c3a90 0x7f3e9c0aa8b0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c06f740 tx=0x7f3e90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752389ef0bdcec7c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.561+0000 7f3e9affd700 1 -- 172.21.15.7:0/338132837 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9000b860 con 0x7f3e9c0c3a90 2024-01-27T02:38:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.561+0000 7f3e9affd700 1 -- 172.21.15.7:0/338132837 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9000b9c0 con 0x7f3e9c0c3a90 2024-01-27T02:38:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.561+0000 7f3e9affd700 1 -- 172.21.15.7:0/338132837 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9000bd60 con 0x7f3e9c0c3a90 2024-01-27T02:38:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.562+0000 7f3ea28b9700 1 -- 172.21.15.7:0/338132837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e9c0c3a90 msgr2=0x7f3e9c0aa8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.562+0000 7f3ea28b9700 1 --2- 172.21.15.7:0/338132837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e9c0c3a90 0x7f3e9c0aa8b0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c06f740 tx=0x7f3e90009250 comp rx=0 tx=0).stop 2024-01-27T02:38:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.562+0000 7f3ea28b9700 1 -- 172.21.15.7:0/338132837 shutdown_connections 2024-01-27T02:38:37.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.562+0000 7f3ea28b9700 1 --2- 172.21.15.7:0/338132837 >> v2:172.21.15.188:3300/0 conn(0x7f3e9c15a310 0x7f3e9c0aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:37.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.562+0000 7f3ea28b9700 1 --2- 172.21.15.7:0/338132837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e9c159f40 0x7f3e9c0c3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:37.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.562+0000 7f3ea28b9700 1 --2- 172.21.15.7:0/338132837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e9c0c3a90 0x7f3e9c0aa8b0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:37.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.562+0000 7f3ea28b9700 1 -- 172.21.15.7:0/338132837 >> 172.21.15.7:0/338132837 conn(0x7f3e9c0cdc30 msgr2=0x7f3e9c0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:37.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.562+0000 7f3ea28b9700 1 -- 172.21.15.7:0/338132837 shutdown_connections 2024-01-27T02:38:37.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.562+0000 7f3ea28b9700 1 -- 172.21.15.7:0/338132837 wait complete. 2024-01-27T02:38:37.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.564+0000 7f3ea28b9700 1 Processor -- start 2024-01-27T02:38:37.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.564+0000 7f3ea28b9700 1 -- start start 2024-01-27T02:38:37.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.565+0000 7f3ea28b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e9c0c3a90 0x7f3e9c16d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:37.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.565+0000 7f3ea28b9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e9c159f40 0x7f3e9c16a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.565+0000 7f3ea28b9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e9c15a310 0x7f3e9c167b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.565+0000 7f3ea28b9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c16da70 con 0x7f3e9c0c3a90 2024-01-27T02:38:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.565+0000 7f3ea28b9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c16dca0 con 0x7f3e9c159f40 2024-01-27T02:38:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.565+0000 7f3ea28b9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c16df80 con 0x7f3e9c15a310 2024-01-27T02:38:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.565+0000 7f3e9bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e9c0c3a90 0x7f3e9c16d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.565+0000 7f3e9bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e9c0c3a90 0x7f3e9c16d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58916/0 (socket says 172.21.15.7:58916) 2024-01-27T02:38:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.565+0000 7f3e9bfff700 1 -- 172.21.15.7:0/2230341641 learned_addr learned my addr 172.21.15.7:0/2230341641 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.565+0000 7f3ea0e56700 1 --2- 172.21.15.7:0/2230341641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e9c15a310 0x7f3e9c167b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.565+0000 7f3e9b7fe700 1 --2- 172.21.15.7:0/2230341641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e9c159f40 0x7f3e9c16a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.565+0000 7f3e9bfff700 1 -- 172.21.15.7:0/2230341641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e9c15a310 msgr2=0x7f3e9c167b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.565+0000 7f3e9bfff700 1 --2- 172.21.15.7:0/2230341641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e9c15a310 0x7f3e9c167b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.565+0000 7f3e9bfff700 1 -- 172.21.15.7:0/2230341641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e9c159f40 msgr2=0x7f3e9c16a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:37.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.565+0000 7f3e9bfff700 1 --2- 172.21.15.7:0/2230341641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e9c159f40 0x7f3e9c16a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:37.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.565+0000 7f3e9bfff700 1 -- 172.21.15.7:0/2230341641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e900094f0 con 0x7f3e9c0c3a90 2024-01-27T02:38:37.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.565+0000 7f3ea0e56700 1 --2- 172.21.15.7:0/2230341641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e9c15a310 0x7f3e9c167b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:37.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.566+0000 7f3e9bfff700 1 --2- 172.21.15.7:0/2230341641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e9c0c3a90 0x7f3e9c16d530 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00d890 tx=0x7f3e8c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:37.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.566+0000 7f3e83fff700 1 -- 172.21.15.7:0/2230341641 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c0125f0 con 0x7f3e9c0c3a90 2024-01-27T02:38:37.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.566+0000 7f3e83fff700 1 -- 172.21.15.7:0/2230341641 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8c012770 con 0x7f3e9c0c3a90 2024-01-27T02:38:37.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.566+0000 7f3ea28b9700 1 -- 172.21.15.7:0/2230341641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9c16e260 con 0x7f3e9c0c3a90 2024-01-27T02:38:37.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.566+0000 7f3e83fff700 1 -- 172.21.15.7:0/2230341641 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c012b30 con 0x7f3e9c0c3a90 2024-01-27T02:38:37.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.566+0000 7f3ea28b9700 1 -- 172.21.15.7:0/2230341641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9c16ada0 con 0x7f3e9c0c3a90 2024-01-27T02:38:37.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.567+0000 7f3ea28b9700 1 -- 172.21.15.7:0/2230341641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9c04c710 con 0x7f3e9c0c3a90 2024-01-27T02:38:37.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.567+0000 7f3e83fff700 1 -- 172.21.15.7:0/2230341641 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3e8c01e460 con 0x7f3e9c0c3a90 2024-01-27T02:38:37.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.568+0000 7f3e83fff700 1 --2- 172.21.15.7:0/2230341641 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3e88066f30 0x7f3e880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:37.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.568+0000 7f3e83fff700 1 -- 172.21.15.7:0/2230341641 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3e8c084c30 con 0x7f3e9c0c3a90 2024-01-27T02:38:37.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.568+0000 7f3e9b7fe700 1 --2- 172.21.15.7:0/2230341641 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3e88066f30 0x7f3e880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:37.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.568+0000 7f3e9b7fe700 1 --2- 172.21.15.7:0/2230341641 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3e88066f30 0x7f3e880693e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c06f740 tx=0x7f3e90017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:37.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.572+0000 7f3e83fff700 1 -- 172.21.15.7:0/2230341641 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e8c051ea0 con 0x7f3e9c0c3a90 2024-01-27T02:38:37.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.771+0000 7f3ea28b9700 1 -- 172.21.15.7:0/2230341641 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e9c06f770 con 0x7f3e88066f30 2024-01-27T02:38:37.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.779+0000 7f3e83fff700 1 -- 172.21.15.7:0/2230341641 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285448 (secure 0 0 0) 0x7f3e9c06f770 con 0x7f3e88066f30 2024-01-27T02:38:37.783 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.782+0000 7f3ea28b9700 1 -- 172.21.15.7:0/2230341641 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3e88066f30 msgr2=0x7f3e880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.782+0000 7f3ea28b9700 1 --2- 172.21.15.7:0/2230341641 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3e88066f30 0x7f3e880693e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c06f740 tx=0x7f3e90017000 comp rx=0 tx=0).stop 2024-01-27T02:38:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.782+0000 7f3ea28b9700 1 -- 172.21.15.7:0/2230341641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e9c0c3a90 msgr2=0x7f3e9c16d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.782+0000 7f3ea28b9700 1 --2- 172.21.15.7:0/2230341641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e9c0c3a90 0x7f3e9c16d530 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00d890 tx=0x7f3e8c008040 comp rx=0 tx=0).stop 2024-01-27T02:38:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.782+0000 7f3ea28b9700 1 -- 172.21.15.7:0/2230341641 shutdown_connections 2024-01-27T02:38:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.783+0000 7f3ea28b9700 1 --2- 172.21.15.7:0/2230341641 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3e88066f30 0x7f3e880693e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.783+0000 7f3ea28b9700 1 --2- 172.21.15.7:0/2230341641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e9c15a310 0x7f3e9c167b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.783+0000 7f3ea28b9700 1 --2- 172.21.15.7:0/2230341641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e9c0c3a90 0x7f3e9c16d530 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.783+0000 7f3ea28b9700 1 --2- 172.21.15.7:0/2230341641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e9c159f40 0x7f3e9c16a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.783+0000 7f3ea28b9700 1 -- 172.21.15.7:0/2230341641 >> 172.21.15.7:0/2230341641 conn(0x7f3e9c0cdc30 msgr2=0x7f3e9c0af700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.783+0000 7f3ea28b9700 1 -- 172.21.15.7:0/2230341641 shutdown_connections 2024-01-27T02:38:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:37.783+0000 7f3ea28b9700 1 -- 172.21.15.7:0/2230341641 wait complete. 2024-01-27T02:38:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:38:37.797 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-01-27T02:38:37.667642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6813,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555686,"statfs":{"total":1600000000,"available":1593023060,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.776715"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.925873+0000","last_change":"2024-01-27T02:38:31.925629+0000","last_active":"2024-01-27T02:38:31.925873+0000","last_peered":"2024-01-27T02:38:31.925873+0000","last_clean":"2024-01-27T02:38:31.925873+0000","last_became_active":"2024-01-27T02:38:31.924701+0000","last_became_peered":"2024-01-27T02:38:31.924701+0000","last_unstale":"2024-01-27T02:38:31.925873+0000","last_undegraded":"2024-01-27T02:38:31.925873+0000","last_fullsized":"2024-01-27T02:38:31.925873+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:14:38.389382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.926896+0000","last_change":"2024-01-27T02:38:31.926735+0000","last_active":"2024-01-27T02:38:31.926896+0000","last_peered":"2024-01-27T02:38:31.926896+0000","last_clean":"2024-01-27T02:38:31.926896+0000","last_became_active":"2024-01-27T02:38:31.926181+0000","last_became_peered":"2024-01-27T02:38:31.926181+0000","last_unstale":"2024-01-27T02:38:31.926896+0000","last_undegraded":"2024-01-27T02:38:31.926896+0000","last_fullsized":"2024-01-27T02:38:31.926896+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:40:17.611985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.924582+0000","last_change":"2024-01-27T02:38:31.924341+0000","last_active":"2024-01-27T02:38:31.924582+0000","last_peered":"2024-01-27T02:38:31.924582+0000","last_clean":"2024-01-27T02:38:31.924582+0000","last_became_active":"2024-01-27T02:38:31.923266+0000","last_became_peered":"2024-01-27T02:38:31.923266+0000","last_unstale":"2024-01-27T02:38:31.924582+0000","last_undegraded":"2024-01-27T02:38:31.924582+0000","last_fullsized":"2024-01-27T02:38:31.924582+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:00:23.173837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.927328+0000","last_change":"2024-01-27T02:38:31.926614+0000","last_active":"2024-01-27T02:38:31.927328+0000","last_peered":"2024-01-27T02:38:31.927328+0000","last_clean":"2024-01-27T02:38:31.927328+0000","last_became_active":"2024-01-27T02:38:31.926200+0000","last_became_peered":"2024-01-27T02:38:31.926200+0000","last_unstale":"2024-01-27T02:38:31.927328+0000","last_undegraded":"2024-01-27T02:38:31.927328+0000","last_fullsized":"2024-01-27T02:38:31.927328+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:29:49.270287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.916155+0000","last_change":"2024-01-27T02:38:31.915978+0000","last_active":"2024-01-27T02:38:31.916155+0000","last_peered":"2024-01-27T02:38:31.916155+0000","last_clean":"2024-01-27T02:38:31.916155+0000","last_became_active":"2024-01-27T02:38:31.915733+0000","last_became_peered":"2024-01-27T02:38:31.915733+0000","last_unstale":"2024-01-27T02:38:31.916155+0000","last_undegraded":"2024-01-27T02:38:31.916155+0000","last_fullsized":"2024-01-27T02:38:31.916155+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:01:39.184586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.921125+0000","last_change":"2024-01-27T02:38:31.921025+0000","last_active":"2024-01-27T02:38:31.921125+0000","last_peered":"2024-01-27T02:38:31.921125+0000","last_clean":"2024-01-27T02:38:31.921125+0000","last_became_active":"2024-01-27T02:38:31.920667+0000","last_became_peered":"2024-01-27T02:38:31.920667+0000","last_unstale":"2024-01-27T02:38:31.921125+0000","last_undegraded":"2024-01-27T02:38:31.921125+0000","last_fullsized":"2024-01-27T02:38:31.921125+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:44:22.418248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.921472+0000","last_change":"2024-01-27T02:38:31.921408+0000","last_active":"2024-01-27T02:38:31.921472+0000","last_peered":"2024-01-27T02:38:31.921472+0000","last_clean":"2024-01-27T02:38:31.921472+0000","last_became_active":"2024-01-27T02:38:31.921189+0000","last_became_peered":"2024-01-27T02:38:31.921189+0000","last_unstale":"2024-01-27T02:38:31.921472+0000","last_undegraded":"2024-01-27T02:38:31.921472+0000","last_fullsized":"2024-01-27T02:38:31.921472+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:18:12.582908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.924598+0000","last_change":"2024-01-27T02:38:31.924520+0000","last_active":"2024-01-27T02:38:31.924598+0000","last_peered":"2024-01-27T02:38:31.924598+0000","last_clean":"2024-01-27T02:38:31.924598+0000","last_became_active":"2024-01-27T02:38:31.923881+0000","last_became_peered":"2024-01-27T02:38:31.923881+0000","last_unstale":"2024-01-27T02:38:31.924598+0000","last_undegraded":"2024-01-27T02:38:31.924598+0000","last_fullsized":"2024-01-27T02:38:31.924598+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:22:55.022123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.927311+0000","last_change":"2024-01-27T02:38:31.926457+0000","last_active":"2024-01-27T02:38:31.927311+0000","last_peered":"2024-01-27T02:38:31.927311+0000","last_clean":"2024-01-27T02:38:31.927311+0000","last_became_active":"2024-01-27T02:38:31.925649+0000","last_became_peered":"2024-01-27T02:38:31.925649+0000","last_unstale":"2024-01-27T02:38:31.927311+0000","last_undegraded":"2024-01-27T02:38:31.927311+0000","last_fullsized":"2024-01-27T02:38:31.927311+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:00:12.784356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.924416+0000","last_change":"2024-01-27T02:38:31.923623+0000","last_active":"2024-01-27T02:38:31.924416+0000","last_peered":"2024-01-27T02:38:31.924416+0000","last_clean":"2024-01-27T02:38:31.924416+0000","last_became_active":"2024-01-27T02:38:31.922781+0000","last_became_peered":"2024-01-27T02:38:31.922781+0000","last_unstale":"2024-01-27T02:38:31.924416+0000","last_undegraded":"2024-01-27T02:38:31.924416+0000","last_fullsized":"2024-01-27T02:38:31.924416+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:37:50.476235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.925225+0000","last_change":"2024-01-27T02:38:31.925132+0000","last_active":"2024-01-27T02:38:31.925225+0000","last_peered":"2024-01-27T02:38:31.925225+0000","last_clean":"2024-01-27T02:38:31.925225+0000","last_became_active":"2024-01-27T02:38:31.924831+0000","last_became_peered":"2024-01-27T02:38:31.924831+0000","last_unstale":"2024-01-27T02:38:31.925225+0000","last_undegraded":"2024-01-27T02:38:31.925225+0000","last_fullsized":"2024-01-27T02:38:31.925225+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:20:34.579927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.928284+0000","last_change":"2024-01-27T02:38:31.928214+0000","last_active":"2024-01-27T02:38:31.928284+0000","last_peered":"2024-01-27T02:38:31.928284+0000","last_clean":"2024-01-27T02:38:31.928284+0000","last_became_active":"2024-01-27T02:38:31.926113+0000","last_became_peered":"2024-01-27T02:38:31.926113+0000","last_unstale":"2024-01-27T02:38:31.928284+0000","last_undegraded":"2024-01-27T02:38:31.928284+0000","last_fullsized":"2024-01-27T02:38:31.928284+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:59:44.278065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.927538+0000","last_change":"2024-01-27T02:38:31.927475+0000","last_active":"2024-01-27T02:38:31.927538+0000","last_peered":"2024-01-27T02:38:31.927538+0000","last_clean":"2024-01-27T02:38:31.927538+0000","last_became_active":"2024-01-27T02:38:31.926843+0000","last_became_peered":"2024-01-27T02:38:31.926843+0000","last_unstale":"2024-01-27T02:38:31.927538+0000","last_undegraded":"2024-01-27T02:38:31.927538+0000","last_fullsized":"2024-01-27T02:38:31.927538+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:54:13.383710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.923085+0000","last_change":"2024-01-27T02:38:31.922687+0000","last_active":"2024-01-27T02:38:31.923085+0000","last_peered":"2024-01-27T02:38:31.923085+0000","last_clean":"2024-01-27T02:38:31.923085+0000","last_became_active":"2024-01-27T02:38:31.922433+0000","last_became_peered":"2024-01-27T02:38:31.922433+0000","last_unstale":"2024-01-27T02:38:31.923085+0000","last_undegraded":"2024-01-27T02:38:31.923085+0000","last_fullsized":"2024-01-27T02:38:31.923085+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:54:15.890067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.922490+0000","last_change":"2024-01-27T02:38:31.922397+0000","last_active":"2024-01-27T02:38:31.922490+0000","last_peered":"2024-01-27T02:38:31.922490+0000","last_clean":"2024-01-27T02:38:31.922490+0000","last_became_active":"2024-01-27T02:38:31.922139+0000","last_became_peered":"2024-01-27T02:38:31.922139+0000","last_unstale":"2024-01-27T02:38:31.922490+0000","last_undegraded":"2024-01-27T02:38:31.922490+0000","last_fullsized":"2024-01-27T02:38:31.922490+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:36:36.290157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.925575+0000","last_change":"2024-01-27T02:38:31.925434+0000","last_active":"2024-01-27T02:38:31.925575+0000","last_peered":"2024-01-27T02:38:31.925575+0000","last_clean":"2024-01-27T02:38:31.925575+0000","last_became_active":"2024-01-27T02:38:31.924412+0000","last_became_peered":"2024-01-27T02:38:31.924412+0000","last_unstale":"2024-01-27T02:38:31.925575+0000","last_undegraded":"2024-01-27T02:38:31.925575+0000","last_fullsized":"2024-01-27T02:38:31.925575+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:39:41.293202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.926463+0000","last_change":"2024-01-27T02:38:31.926401+0000","last_active":"2024-01-27T02:38:31.926463+0000","last_peered":"2024-01-27T02:38:31.926463+0000","last_clean":"2024-01-27T02:38:31.926463+0000","last_became_active":"2024-01-27T02:38:31.925453+0000","last_became_peered":"2024-01-27T02:38:31.925453+0000","last_unstale":"2024-01-27T02:38:31.926463+0000","last_undegraded":"2024-01-27T02:38:31.926463+0000","last_fullsized":"2024-01-27T02:38:31.926463+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:30:10.336341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.928420+0000","last_change":"2024-01-27T02:38:31.927136+0000","last_active":"2024-01-27T02:38:31.928420+0000","last_peered":"2024-01-27T02:38:31.928420+0000","last_clean":"2024-01-27T02:38:31.928420+0000","last_became_active":"2024-01-27T02:38:31.926906+0000","last_became_peered":"2024-01-27T02:38:31.926906+0000","last_unstale":"2024-01-27T02:38:31.928420+0000","last_undegraded":"2024-01-27T02:38:31.928420+0000","last_fullsized":"2024-01-27T02:38:31.928420+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:08.702500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.921615+0000","last_change":"2024-01-27T02:38:31.921550+0000","last_active":"2024-01-27T02:38:31.921615+0000","last_peered":"2024-01-27T02:38:31.921615+0000","last_clean":"2024-01-27T02:38:31.921615+0000","last_became_active":"2024-01-27T02:38:31.921163+0000","last_became_peered":"2024-01-27T02:38:31.921163+0000","last_unstale":"2024-01-27T02:38:31.921615+0000","last_undegraded":"2024-01-27T02:38:31.921615+0000","last_fullsized":"2024-01-27T02:38:31.921615+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:11:36.249589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.918985+0000","last_change":"2024-01-27T02:38:31.918643+0000","last_active":"2024-01-27T02:38:31.918985+0000","last_peered":"2024-01-27T02:38:31.918985+0000","last_clean":"2024-01-27T02:38:31.918985+0000","last_became_active":"2024-01-27T02:38:31.918187+0000","last_became_peered":"2024-01-27T02:38:31.918187+0000","last_unstale":"2024-01-27T02:38:31.918985+0000","last_undegraded":"2024-01-27T02:38:31.918985+0000","last_fullsized":"2024-01-27T02:38:31.918985+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:34:26.079082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.912132+0000","last_change":"2024-01-27T02:38:31.912046+0000","last_active":"2024-01-27T02:38:31.912132+0000","last_peered":"2024-01-27T02:38:31.912132+0000","last_clean":"2024-01-27T02:38:31.912132+0000","last_became_active":"2024-01-27T02:38:31.911789+0000","last_became_peered":"2024-01-27T02:38:31.911789+0000","last_unstale":"2024-01-27T02:38:31.912132+0000","last_undegraded":"2024-01-27T02:38:31.912132+0000","last_fullsized":"2024-01-27T02:38:31.912132+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:51.023684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.924055+0000","last_change":"2024-01-27T02:38:31.923490+0000","last_active":"2024-01-27T02:38:31.924055+0000","last_peered":"2024-01-27T02:38:31.924055+0000","last_clean":"2024-01-27T02:38:31.924055+0000","last_became_active":"2024-01-27T02:38:31.923210+0000","last_became_peered":"2024-01-27T02:38:31.923210+0000","last_unstale":"2024-01-27T02:38:31.924055+0000","last_undegraded":"2024-01-27T02:38:31.924055+0000","last_fullsized":"2024-01-27T02:38:31.924055+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:54:15.890067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.918931+0000","last_change":"2024-01-27T02:38:31.917222+0000","last_active":"2024-01-27T02:38:31.918931+0000","last_peered":"2024-01-27T02:38:31.918931+0000","last_clean":"2024-01-27T02:38:31.918931+0000","last_became_active":"2024-01-27T02:38:31.917004+0000","last_became_peered":"2024-01-27T02:38:31.917004+0000","last_unstale":"2024-01-27T02:38:31.918931+0000","last_undegraded":"2024-01-27T02:38:31.918931+0000","last_fullsized":"2024-01-27T02:38:31.918931+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:23:46.870039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.919670+0000","last_change":"2024-01-27T02:38:31.919512+0000","last_active":"2024-01-27T02:38:31.919670+0000","last_peered":"2024-01-27T02:38:31.919670+0000","last_clean":"2024-01-27T02:38:31.919670+0000","last_became_active":"2024-01-27T02:38:31.918679+0000","last_became_peered":"2024-01-27T02:38:31.918679+0000","last_unstale":"2024-01-27T02:38:31.919670+0000","last_undegraded":"2024-01-27T02:38:31.919670+0000","last_fullsized":"2024-01-27T02:38:31.919670+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:07.978917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.925909+0000","last_change":"2024-01-27T02:38:31.925271+0000","last_active":"2024-01-27T02:38:31.925909+0000","last_peered":"2024-01-27T02:38:31.925909+0000","last_clean":"2024-01-27T02:38:31.925909+0000","last_became_active":"2024-01-27T02:38:31.925019+0000","last_became_peered":"2024-01-27T02:38:31.925019+0000","last_unstale":"2024-01-27T02:38:31.925909+0000","last_undegraded":"2024-01-27T02:38:31.925909+0000","last_fullsized":"2024-01-27T02:38:31.925909+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:51:24.348901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.925243+0000","last_change":"2024-01-27T02:38:31.925134+0000","last_active":"2024-01-27T02:38:31.925243+0000","last_peered":"2024-01-27T02:38:31.925243+0000","last_clean":"2024-01-27T02:38:31.925243+0000","last_became_active":"2024-01-27T02:38:31.924109+0000","last_became_peered":"2024-01-27T02:38:31.924109+0000","last_unstale":"2024-01-27T02:38:31.925243+0000","last_undegraded":"2024-01-27T02:38:31.925243+0000","last_fullsized":"2024-01-27T02:38:31.925243+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:21:44.987764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.929805+0000","last_change":"2024-01-27T02:38:31.929737+0000","last_active":"2024-01-27T02:38:31.929805+0000","last_peered":"2024-01-27T02:38:31.929805+0000","last_clean":"2024-01-27T02:38:31.929805+0000","last_became_active":"2024-01-27T02:38:31.929489+0000","last_became_peered":"2024-01-27T02:38:31.929489+0000","last_unstale":"2024-01-27T02:38:31.929805+0000","last_undegraded":"2024-01-27T02:38:31.929805+0000","last_fullsized":"2024-01-27T02:38:31.929805+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:03:26.614534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.927319+0000","last_change":"2024-01-27T02:38:31.927229+0000","last_active":"2024-01-27T02:38:31.927319+0000","last_peered":"2024-01-27T02:38:31.927319+0000","last_clean":"2024-01-27T02:38:31.927319+0000","last_became_active":"2024-01-27T02:38:31.926973+0000","last_became_peered":"2024-01-27T02:38:31.926973+0000","last_unstale":"2024-01-27T02:38:31.927319+0000","last_undegraded":"2024-01-27T02:38:31.927319+0000","last_fullsized":"2024-01-27T02:38:31.927319+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:40:17.611985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.918966+0000","last_change":"2024-01-27T02:38:31.918415+0000","last_active":"2024-01-27T02:38:31.918966+0000","last_peered":"2024-01-27T02:38:31.918966+0000","last_clean":"2024-01-27T02:38:31.918966+0000","last_became_active":"2024-01-27T02:38:31.918138+0000","last_became_peered":"2024-01-27T02:38:31.918138+0000","last_unstale":"2024-01-27T02:38:31.918966+0000","last_undegraded":"2024-01-27T02:38:31.918966+0000","last_fullsized":"2024-01-27T02:38:31.918966+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:54:15.890067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.915353+0000","last_change":"2024-01-27T02:38:31.915280+0000","last_active":"2024-01-27T02:38:31.915353+0000","last_peered":"2024-01-27T02:38:31.915353+0000","last_clean":"2024-01-27T02:38:31.915353+0000","last_became_active":"2024-01-27T02:38:31.915025+0000","last_became_peered":"2024-01-27T02:38:31.915025+0000","last_unstale":"2024-01-27T02:38:31.915353+0000","last_undegraded":"2024-01-27T02:38:31.915353+0000","last_fullsized":"2024-01-27T02:38:31.915353+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:25:14.518250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.925052+0000","last_change":"2024-01-27T02:38:31.924959+0000","last_active":"2024-01-27T02:38:31.925052+0000","last_peered":"2024-01-27T02:38:31.925052+0000","last_clean":"2024-01-27T02:38:31.925052+0000","last_became_active":"2024-01-27T02:38:31.923077+0000","last_became_peered":"2024-01-27T02:38:31.923077+0000","last_unstale":"2024-01-27T02:38:31.925052+0000","last_undegraded":"2024-01-27T02:38:31.925052+0000","last_fullsized":"2024-01-27T02:38:31.925052+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:27:09.408626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.921090+0000","last_change":"2024-01-27T02:38:31.920385+0000","last_active":"2024-01-27T02:38:31.921090+0000","last_peered":"2024-01-27T02:38:31.921090+0000","last_clean":"2024-01-27T02:38:31.921090+0000","last_became_active":"2024-01-27T02:38:31.920104+0000","last_became_peered":"2024-01-27T02:38:31.920104+0000","last_unstale":"2024-01-27T02:38:31.921090+0000","last_undegraded":"2024-01-27T02:38:31.921090+0000","last_fullsized":"2024-01-27T02:38:31.921090+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:36:50.676219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.919758+0000","last_change":"2024-01-27T02:38:31.919669+0000","last_active":"2024-01-27T02:38:31.919758+0000","last_peered":"2024-01-27T02:38:31.919758+0000","last_clean":"2024-01-27T02:38:31.919758+0000","last_became_active":"2024-01-27T02:38:31.919354+0000","last_became_peered":"2024-01-27T02:38:31.919354+0000","last_unstale":"2024-01-27T02:38:31.919758+0000","last_undegraded":"2024-01-27T02:38:31.919758+0000","last_fullsized":"2024-01-27T02:38:31.919758+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:23:02.614674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"131'4","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.934542+0000","last_change":"2024-01-27T02:38:30.900146+0000","last_active":"2024-01-27T02:38:31.934542+0000","last_peered":"2024-01-27T02:38:31.934542+0000","last_clean":"2024-01-27T02:38:31.934542+0000","last_became_active":"2024-01-27T02:38:30.899920+0000","last_became_peered":"2024-01-27T02:38:30.899920+0000","last_unstale":"2024-01-27T02:38:31.934542+0000","last_undegraded":"2024-01-27T02:38:31.934542+0000","last_fullsized":"2024-01-27T02:38:31.934542+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:02:26.763414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.928408+0000","last_change":"2024-01-27T02:38:31.924700+0000","last_active":"2024-01-27T02:38:31.928408+0000","last_peered":"2024-01-27T02:38:31.928408+0000","last_clean":"2024-01-27T02:38:31.928408+0000","last_became_active":"2024-01-27T02:38:31.924467+0000","last_became_peered":"2024-01-27T02:38:31.924467+0000","last_unstale":"2024-01-27T02:38:31.928408+0000","last_undegraded":"2024-01-27T02:38:31.928408+0000","last_fullsized":"2024-01-27T02:38:31.928408+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:05:23.541565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.934021+0000","last_change":"2024-01-27T02:38:30.900419+0000","last_active":"2024-01-27T02:38:31.934021+0000","last_peered":"2024-01-27T02:38:31.934021+0000","last_clean":"2024-01-27T02:38:31.934021+0000","last_became_active":"2024-01-27T02:38:30.900188+0000","last_became_peered":"2024-01-27T02:38:30.900188+0000","last_unstale":"2024-01-27T02:38:31.934021+0000","last_undegraded":"2024-01-27T02:38:31.934021+0000","last_fullsized":"2024-01-27T02:38:31.934021+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:16:12.621310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.921107+0000","last_change":"2024-01-27T02:38:31.920894+0000","last_active":"2024-01-27T02:38:31.921107+0000","last_peered":"2024-01-27T02:38:31.921107+0000","last_clean":"2024-01-27T02:38:31.921107+0000","last_became_active":"2024-01-27T02:38:31.920365+0000","last_became_peered":"2024-01-27T02:38:31.920365+0000","last_unstale":"2024-01-27T02:38:31.921107+0000","last_undegraded":"2024-01-27T02:38:31.921107+0000","last_fullsized":"2024-01-27T02:38:31.921107+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:55:51.244887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.937266+0000","last_change":"2024-01-27T02:38:30.907104+0000","last_active":"2024-01-27T02:38:31.937266+0000","last_peered":"2024-01-27T02:38:31.937266+0000","last_clean":"2024-01-27T02:38:31.937266+0000","last_became_active":"2024-01-27T02:38:30.906807+0000","last_became_peered":"2024-01-27T02:38:30.906807+0000","last_unstale":"2024-01-27T02:38:31.937266+0000","last_undegraded":"2024-01-27T02:38:31.937266+0000","last_fullsized":"2024-01-27T02:38:31.937266+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:16:05.064427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.925359+0000","last_change":"2024-01-27T02:38:31.925298+0000","last_active":"2024-01-27T02:38:31.925359+0000","last_peered":"2024-01-27T02:38:31.925359+0000","last_clean":"2024-01-27T02:38:31.925359+0000","last_became_active":"2024-01-27T02:38:31.925075+0000","last_became_peered":"2024-01-27T02:38:31.925075+0000","last_unstale":"2024-01-27T02:38:31.925359+0000","last_undegraded":"2024-01-27T02:38:31.925359+0000","last_fullsized":"2024-01-27T02:38:31.925359+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:07.978917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.936708+0000","last_change":"2024-01-27T02:38:30.906090+0000","last_active":"2024-01-27T02:38:31.936708+0000","last_peered":"2024-01-27T02:38:31.936708+0000","last_clean":"2024-01-27T02:38:31.936708+0000","last_became_active":"2024-01-27T02:38:30.905815+0000","last_became_peered":"2024-01-27T02:38:30.905815+0000","last_unstale":"2024-01-27T02:38:31.936708+0000","last_undegraded":"2024-01-27T02:38:31.936708+0000","last_fullsized":"2024-01-27T02:38:31.936708+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:42:16.605919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.927368+0000","last_change":"2024-01-27T02:38:31.927281+0000","last_active":"2024-01-27T02:38:31.927368+0000","last_peered":"2024-01-27T02:38:31.927368+0000","last_clean":"2024-01-27T02:38:31.927368+0000","last_became_active":"2024-01-27T02:38:31.926844+0000","last_became_peered":"2024-01-27T02:38:31.926844+0000","last_unstale":"2024-01-27T02:38:31.927368+0000","last_undegraded":"2024-01-27T02:38:31.927368+0000","last_fullsized":"2024-01-27T02:38:31.927368+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:13:24.438761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.933763+0000","last_change":"2024-01-27T02:38:30.908846+0000","last_active":"2024-01-27T02:38:31.933763+0000","last_peered":"2024-01-27T02:38:31.933763+0000","last_clean":"2024-01-27T02:38:31.933763+0000","last_became_active":"2024-01-27T02:38:30.908634+0000","last_became_peered":"2024-01-27T02:38:30.908634+0000","last_unstale":"2024-01-27T02:38:31.933763+0000","last_undegraded":"2024-01-27T02:38:31.933763+0000","last_fullsized":"2024-01-27T02:38:31.933763+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:44:21.414722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.926877+0000","last_change":"2024-01-27T02:38:31.926584+0000","last_active":"2024-01-27T02:38:31.926877+0000","last_peered":"2024-01-27T02:38:31.926877+0000","last_clean":"2024-01-27T02:38:31.926877+0000","last_became_active":"2024-01-27T02:38:31.925803+0000","last_became_peered":"2024-01-27T02:38:31.925803+0000","last_unstale":"2024-01-27T02:38:31.926877+0000","last_undegraded":"2024-01-27T02:38:31.926877+0000","last_fullsized":"2024-01-27T02:38:31.926877+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:11:36.249589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.934988+0000","last_change":"2024-01-27T02:38:30.897433+0000","last_active":"2024-01-27T02:38:31.934988+0000","last_peered":"2024-01-27T02:38:31.934988+0000","last_clean":"2024-01-27T02:38:31.934988+0000","last_became_active":"2024-01-27T02:38:30.897134+0000","last_became_peered":"2024-01-27T02:38:30.897134+0000","last_unstale":"2024-01-27T02:38:31.934988+0000","last_undegraded":"2024-01-27T02:38:31.934988+0000","last_fullsized":"2024-01-27T02:38:31.934988+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:18:04.098263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.927292+0000","last_change":"2024-01-27T02:38:31.926378+0000","last_active":"2024-01-27T02:38:31.927292+0000","last_peered":"2024-01-27T02:38:31.927292+0000","last_clean":"2024-01-27T02:38:31.927292+0000","last_became_active":"2024-01-27T02:38:31.925226+0000","last_became_peered":"2024-01-27T02:38:31.925226+0000","last_unstale":"2024-01-27T02:38:31.927292+0000","last_undegraded":"2024-01-27T02:38:31.927292+0000","last_fullsized":"2024-01-27T02:38:31.927292+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:27:05.440576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.911415+0000","last_change":"2024-01-27T02:38:30.903018+0000","last_active":"2024-01-27T02:38:31.911415+0000","last_peered":"2024-01-27T02:38:31.911415+0000","last_clean":"2024-01-27T02:38:31.911415+0000","last_became_active":"2024-01-27T02:38:30.902748+0000","last_became_peered":"2024-01-27T02:38:30.902748+0000","last_unstale":"2024-01-27T02:38:31.911415+0000","last_undegraded":"2024-01-27T02:38:31.911415+0000","last_fullsized":"2024-01-27T02:38:31.911415+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:14:24.855281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.921982+0000","last_change":"2024-01-27T02:38:31.921936+0000","last_active":"2024-01-27T02:38:31.921982+0000","last_peered":"2024-01-27T02:38:31.921982+0000","last_clean":"2024-01-27T02:38:31.921982+0000","last_became_active":"2024-01-27T02:38:31.921776+0000","last_became_peered":"2024-01-27T02:38:31.921776+0000","last_unstale":"2024-01-27T02:38:31.921982+0000","last_undegraded":"2024-01-27T02:38:31.921982+0000","last_fullsized":"2024-01-27T02:38:31.921982+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:47:41.063180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.921233+0000","last_change":"2024-01-27T02:38:30.903329+0000","last_active":"2024-01-27T02:38:31.921233+0000","last_peered":"2024-01-27T02:38:31.921233+0000","last_clean":"2024-01-27T02:38:31.921233+0000","last_became_active":"2024-01-27T02:38:30.903084+0000","last_became_peered":"2024-01-27T02:38:30.903084+0000","last_unstale":"2024-01-27T02:38:31.921233+0000","last_undegraded":"2024-01-27T02:38:31.921233+0000","last_fullsized":"2024-01-27T02:38:31.921233+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:11:33.257548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.919275+0000","last_change":"2024-01-27T02:38:31.919209+0000","last_active":"2024-01-27T02:38:31.919275+0000","last_peered":"2024-01-27T02:38:31.919275+0000","last_clean":"2024-01-27T02:38:31.919275+0000","last_became_active":"2024-01-27T02:38:31.918761+0000","last_became_peered":"2024-01-27T02:38:31.918761+0000","last_unstale":"2024-01-27T02:38:31.919275+0000","last_undegraded":"2024-01-27T02:38:31.919275+0000","last_fullsized":"2024-01-27T02:38:31.919275+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:18.354062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.913366+0000","last_change":"2024-01-27T02:38:30.904630+0000","last_active":"2024-01-27T02:38:31.913366+0000","last_peered":"2024-01-27T02:38:31.913366+0000","last_clean":"2024-01-27T02:38:31.913366+0000","last_became_active":"2024-01-27T02:38:30.904406+0000","last_became_peered":"2024-01-27T02:38:30.904406+0000","last_unstale":"2024-01-27T02:38:31.913366+0000","last_undegraded":"2024-01-27T02:38:31.913366+0000","last_fullsized":"2024-01-27T02:38:31.913366+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:22:44.330464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.929159+0000","last_change":"2024-01-27T02:38:31.929087+0000","last_active":"2024-01-27T02:38:31.929159+0000","last_peered":"2024-01-27T02:38:31.929159+0000","last_clean":"2024-01-27T02:38:31.929159+0000","last_became_active":"2024-01-27T02:38:31.928717+0000","last_became_peered":"2024-01-27T02:38:31.928717+0000","last_unstale":"2024-01-27T02:38:31.929159+0000","last_undegraded":"2024-01-27T02:38:31.929159+0000","last_fullsized":"2024-01-27T02:38:31.929159+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:23:46.870039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.908261+0000","last_change":"2024-01-27T02:38:30.893083+0000","last_active":"2024-01-27T02:38:31.908261+0000","last_peered":"2024-01-27T02:38:31.908261+0000","last_clean":"2024-01-27T02:38:31.908261+0000","last_became_active":"2024-01-27T02:38:30.892801+0000","last_became_peered":"2024-01-27T02:38:30.892801+0000","last_unstale":"2024-01-27T02:38:31.908261+0000","last_undegraded":"2024-01-27T02:38:31.908261+0000","last_fullsized":"2024-01-27T02:38:31.908261+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:51:23.345374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.912173+0000","last_change":"2024-01-27T02:38:31.912087+0000","last_active":"2024-01-27T02:38:31.912173+0000","last_peered":"2024-01-27T02:38:31.912173+0000","last_clean":"2024-01-27T02:38:31.912173+0000","last_became_active":"2024-01-27T02:38:31.911806+0000","last_became_peered":"2024-01-27T02:38:31.911806+0000","last_unstale":"2024-01-27T02:38:31.912173+0000","last_undegraded":"2024-01-27T02:38:31.912173+0000","last_fullsized":"2024-01-27T02:38:31.912173+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:34:26.079082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"131'3","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.941757+0000","last_change":"2024-01-27T02:38:30.905086+0000","last_active":"2024-01-27T02:38:31.941757+0000","last_peered":"2024-01-27T02:38:31.941757+0000","last_clean":"2024-01-27T02:38:31.941757+0000","last_became_active":"2024-01-27T02:38:30.904853+0000","last_became_peered":"2024-01-27T02:38:30.904853+0000","last_unstale":"2024-01-27T02:38:31.941757+0000","last_undegraded":"2024-01-27T02:38:31.941757+0000","last_fullsized":"2024-01-27T02:38:31.941757+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:54:14.886540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.917312+0000","last_change":"2024-01-27T02:38:31.917239+0000","last_active":"2024-01-27T02:38:31.917312+0000","last_peered":"2024-01-27T02:38:31.917312+0000","last_clean":"2024-01-27T02:38:31.917312+0000","last_became_active":"2024-01-27T02:38:31.916971+0000","last_became_peered":"2024-01-27T02:38:31.916971+0000","last_unstale":"2024-01-27T02:38:31.917312+0000","last_undegraded":"2024-01-27T02:38:31.917312+0000","last_fullsized":"2024-01-27T02:38:31.917312+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:53:05.325595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"131'6","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.940272+0000","last_change":"2024-01-27T02:38:30.905975+0000","last_active":"2024-01-27T02:38:31.940272+0000","last_peered":"2024-01-27T02:38:31.940272+0000","last_clean":"2024-01-27T02:38:31.940272+0000","last_became_active":"2024-01-27T02:38:30.904239+0000","last_became_peered":"2024-01-27T02:38:30.904239+0000","last_unstale":"2024-01-27T02:38:31.940272+0000","last_undegraded":"2024-01-27T02:38:31.940272+0000","last_fullsized":"2024-01-27T02:38:31.940272+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:05:22.538039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.917006+0000","last_change":"2024-01-27T02:38:31.916918+0000","last_active":"2024-01-27T02:38:31.917006+0000","last_peered":"2024-01-27T02:38:31.917006+0000","last_clean":"2024-01-27T02:38:31.917006+0000","last_became_active":"2024-01-27T02:38:31.915711+0000","last_became_peered":"2024-01-27T02:38:31.915711+0000","last_unstale":"2024-01-27T02:38:31.917006+0000","last_undegraded":"2024-01-27T02:38:31.917006+0000","last_fullsized":"2024-01-27T02:38:31.917006+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:47:43.751635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.938505+0000","last_change":"2024-01-27T02:38:30.904960+0000","last_active":"2024-01-27T02:38:31.938505+0000","last_peered":"2024-01-27T02:38:31.938505+0000","last_clean":"2024-01-27T02:38:31.938505+0000","last_became_active":"2024-01-27T02:38:30.904479+0000","last_became_peered":"2024-01-27T02:38:30.904479+0000","last_unstale":"2024-01-27T02:38:31.938505+0000","last_undegraded":"2024-01-27T02:38:31.938505+0000","last_fullsized":"2024-01-27T02:38:31.938505+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:20:17.775604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.919480+0000","last_change":"2024-01-27T02:38:31.919224+0000","last_active":"2024-01-27T02:38:31.919480+0000","last_peered":"2024-01-27T02:38:31.919480+0000","last_clean":"2024-01-27T02:38:31.919480+0000","last_became_active":"2024-01-27T02:38:31.918967+0000","last_became_peered":"2024-01-27T02:38:31.918967+0000","last_unstale":"2024-01-27T02:38:31.919480+0000","last_undegraded":"2024-01-27T02:38:31.919480+0000","last_fullsized":"2024-01-27T02:38:31.919480+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:00:23.173837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.918372+0000","last_change":"2024-01-27T02:38:30.901125+0000","last_active":"2024-01-27T02:38:31.918372+0000","last_peered":"2024-01-27T02:38:31.918372+0000","last_clean":"2024-01-27T02:38:31.918372+0000","last_became_active":"2024-01-27T02:38:30.900811+0000","last_became_peered":"2024-01-27T02:38:30.900811+0000","last_unstale":"2024-01-27T02:38:31.918372+0000","last_undegraded":"2024-01-27T02:38:31.918372+0000","last_fullsized":"2024-01-27T02:38:31.918372+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:18:04.098263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.913229+0000","last_change":"2024-01-27T02:38:31.913151+0000","last_active":"2024-01-27T02:38:31.913229+0000","last_peered":"2024-01-27T02:38:31.913229+0000","last_clean":"2024-01-27T02:38:31.913229+0000","last_became_active":"2024-01-27T02:38:31.912872+0000","last_became_peered":"2024-01-27T02:38:31.912872+0000","last_unstale":"2024-01-27T02:38:31.913229+0000","last_undegraded":"2024-01-27T02:38:31.913229+0000","last_fullsized":"2024-01-27T02:38:31.913229+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:57:59.292913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"131'8","reported_seq":18,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.940973+0000","last_change":"2024-01-27T02:38:30.904062+0000","last_active":"2024-01-27T02:38:31.940973+0000","last_peered":"2024-01-27T02:38:31.940973+0000","last_clean":"2024-01-27T02:38:31.940973+0000","last_became_active":"2024-01-27T02:38:30.903827+0000","last_became_peered":"2024-01-27T02:38:30.903827+0000","last_unstale":"2024-01-27T02:38:31.940973+0000","last_undegraded":"2024-01-27T02:38:31.940973+0000","last_fullsized":"2024-01-27T02:38:31.940973+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:22:30.041364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.927351+0000","last_change":"2024-01-27T02:38:31.927269+0000","last_active":"2024-01-27T02:38:31.927351+0000","last_peered":"2024-01-27T02:38:31.927351+0000","last_clean":"2024-01-27T02:38:31.927351+0000","last_became_active":"2024-01-27T02:38:31.926727+0000","last_became_peered":"2024-01-27T02:38:31.926727+0000","last_unstale":"2024-01-27T02:38:31.927351+0000","last_undegraded":"2024-01-27T02:38:31.927351+0000","last_fullsized":"2024-01-27T02:38:31.927351+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:17:17.633072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.938198+0000","last_change":"2024-01-27T02:38:30.901317+0000","last_active":"2024-01-27T02:38:31.938198+0000","last_peered":"2024-01-27T02:38:31.938198+0000","last_clean":"2024-01-27T02:38:31.938198+0000","last_became_active":"2024-01-27T02:38:30.900905+0000","last_became_peered":"2024-01-27T02:38:30.900905+0000","last_unstale":"2024-01-27T02:38:31.938198+0000","last_undegraded":"2024-01-27T02:38:31.938198+0000","last_fullsized":"2024-01-27T02:38:31.938198+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:23:45.866513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.930083+0000","last_change":"2024-01-27T02:38:31.930009+0000","last_active":"2024-01-27T02:38:31.930083+0000","last_peered":"2024-01-27T02:38:31.930083+0000","last_clean":"2024-01-27T02:38:31.930083+0000","last_became_active":"2024-01-27T02:38:31.929718+0000","last_became_peered":"2024-01-27T02:38:31.929718+0000","last_unstale":"2024-01-27T02:38:31.930083+0000","last_undegraded":"2024-01-27T02:38:31.930083+0000","last_fullsized":"2024-01-27T02:38:31.930083+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:56:12.566293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.937933+0000","last_change":"2024-01-27T02:38:30.901405+0000","last_active":"2024-01-27T02:38:31.937933+0000","last_peered":"2024-01-27T02:38:31.937933+0000","last_clean":"2024-01-27T02:38:31.937933+0000","last_became_active":"2024-01-27T02:38:30.901168+0000","last_became_peered":"2024-01-27T02:38:30.901168+0000","last_unstale":"2024-01-27T02:38:31.937933+0000","last_undegraded":"2024-01-27T02:38:31.937933+0000","last_fullsized":"2024-01-27T02:38:31.937933+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:52:03.557302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.928377+0000","last_change":"2024-01-27T02:38:31.928314+0000","last_active":"2024-01-27T02:38:31.928377+0000","last_peered":"2024-01-27T02:38:31.928377+0000","last_clean":"2024-01-27T02:38:31.928377+0000","last_became_active":"2024-01-27T02:38:31.928109+0000","last_became_peered":"2024-01-27T02:38:31.928109+0000","last_unstale":"2024-01-27T02:38:31.928377+0000","last_undegraded":"2024-01-27T02:38:31.928377+0000","last_fullsized":"2024-01-27T02:38:31.928377+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:40:17.611985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.935106+0000","last_change":"2024-01-27T02:38:30.894794+0000","last_active":"2024-01-27T02:38:31.935106+0000","last_peered":"2024-01-27T02:38:31.935106+0000","last_clean":"2024-01-27T02:38:31.935106+0000","last_became_active":"2024-01-27T02:38:30.894519+0000","last_became_peered":"2024-01-27T02:38:30.894519+0000","last_unstale":"2024-01-27T02:38:31.935106+0000","last_undegraded":"2024-01-27T02:38:31.935106+0000","last_fullsized":"2024-01-27T02:38:31.935106+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:39:25.843705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.921065+0000","last_change":"2024-01-27T02:38:31.920963+0000","last_active":"2024-01-27T02:38:31.921065+0000","last_peered":"2024-01-27T02:38:31.921065+0000","last_clean":"2024-01-27T02:38:31.921065+0000","last_became_active":"2024-01-27T02:38:31.920276+0000","last_became_peered":"2024-01-27T02:38:31.920276+0000","last_unstale":"2024-01-27T02:38:31.921065+0000","last_undegraded":"2024-01-27T02:38:31.921065+0000","last_fullsized":"2024-01-27T02:38:31.921065+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:47:43.751635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.915796+0000","last_change":"2024-01-27T02:38:30.901197+0000","last_active":"2024-01-27T02:38:31.915796+0000","last_peered":"2024-01-27T02:38:31.915796+0000","last_clean":"2024-01-27T02:38:31.915796+0000","last_became_active":"2024-01-27T02:38:30.900235+0000","last_became_peered":"2024-01-27T02:38:30.900235+0000","last_unstale":"2024-01-27T02:38:31.915796+0000","last_undegraded":"2024-01-27T02:38:31.915796+0000","last_fullsized":"2024-01-27T02:38:31.915796+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:07:32.419335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.926876+0000","last_change":"2024-01-27T02:38:31.926758+0000","last_active":"2024-01-27T02:38:31.926876+0000","last_peered":"2024-01-27T02:38:31.926876+0000","last_clean":"2024-01-27T02:38:31.926876+0000","last_became_active":"2024-01-27T02:38:31.926434+0000","last_became_peered":"2024-01-27T02:38:31.926434+0000","last_unstale":"2024-01-27T02:38:31.926876+0000","last_undegraded":"2024-01-27T02:38:31.926876+0000","last_fullsized":"2024-01-27T02:38:31.926876+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:02:03.382943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.937501+0000","last_change":"2024-01-27T02:38:30.904351+0000","last_active":"2024-01-27T02:38:31.937501+0000","last_peered":"2024-01-27T02:38:31.937501+0000","last_clean":"2024-01-27T02:38:31.937501+0000","last_became_active":"2024-01-27T02:38:30.904118+0000","last_became_peered":"2024-01-27T02:38:30.904118+0000","last_unstale":"2024-01-27T02:38:31.937501+0000","last_undegraded":"2024-01-27T02:38:31.937501+0000","last_fullsized":"2024-01-27T02:38:31.937501+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:53:19.168298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.927349+0000","last_change":"2024-01-27T02:38:31.927097+0000","last_active":"2024-01-27T02:38:31.927349+0000","last_peered":"2024-01-27T02:38:31.927349+0000","last_clean":"2024-01-27T02:38:31.927349+0000","last_became_active":"2024-01-27T02:38:31.926471+0000","last_became_peered":"2024-01-27T02:38:31.926471+0000","last_unstale":"2024-01-27T02:38:31.927349+0000","last_undegraded":"2024-01-27T02:38:31.927349+0000","last_fullsized":"2024-01-27T02:38:31.927349+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:33:14.196336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.913953+0000","last_change":"2024-01-27T02:38:30.904812+0000","last_active":"2024-01-27T02:38:31.913953+0000","last_peered":"2024-01-27T02:38:31.913953+0000","last_clean":"2024-01-27T02:38:31.913953+0000","last_became_active":"2024-01-27T02:38:30.904483+0000","last_became_peered":"2024-01-27T02:38:30.904483+0000","last_unstale":"2024-01-27T02:38:31.913953+0000","last_undegraded":"2024-01-27T02:38:31.913953+0000","last_fullsized":"2024-01-27T02:38:31.913953+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:45:43.930079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.926114+0000","last_change":"2024-01-27T02:38:31.926049+0000","last_active":"2024-01-27T02:38:31.926114+0000","last_peered":"2024-01-27T02:38:31.926114+0000","last_clean":"2024-01-27T02:38:31.926114+0000","last_became_active":"2024-01-27T02:38:31.925808+0000","last_became_peered":"2024-01-27T02:38:31.925808+0000","last_unstale":"2024-01-27T02:38:31.926114+0000","last_undegraded":"2024-01-27T02:38:31.926114+0000","last_fullsized":"2024-01-27T02:38:31.926114+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:57:59.292913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.915740+0000","last_change":"2024-01-27T02:38:30.904780+0000","last_active":"2024-01-27T02:38:31.915740+0000","last_peered":"2024-01-27T02:38:31.915740+0000","last_clean":"2024-01-27T02:38:31.915740+0000","last_became_active":"2024-01-27T02:38:30.904179+0000","last_became_peered":"2024-01-27T02:38:30.904179+0000","last_unstale":"2024-01-27T02:38:31.915740+0000","last_undegraded":"2024-01-27T02:38:31.915740+0000","last_fullsized":"2024-01-27T02:38:31.915740+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:14:13.119229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.925284+0000","last_change":"2024-01-27T02:38:31.925213+0000","last_active":"2024-01-27T02:38:31.925284+0000","last_peered":"2024-01-27T02:38:31.925284+0000","last_clean":"2024-01-27T02:38:31.925284+0000","last_became_active":"2024-01-27T02:38:31.924951+0000","last_became_peered":"2024-01-27T02:38:31.924951+0000","last_unstale":"2024-01-27T02:38:31.925284+0000","last_undegraded":"2024-01-27T02:38:31.925284+0000","last_fullsized":"2024-01-27T02:38:31.925284+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:11:36.249589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.918380+0000","last_change":"2024-01-27T02:38:30.899295+0000","last_active":"2024-01-27T02:38:31.918380+0000","last_peered":"2024-01-27T02:38:31.918380+0000","last_clean":"2024-01-27T02:38:31.918380+0000","last_became_active":"2024-01-27T02:38:30.899015+0000","last_became_peered":"2024-01-27T02:38:30.899015+0000","last_unstale":"2024-01-27T02:38:31.918380+0000","last_undegraded":"2024-01-27T02:38:31.918380+0000","last_fullsized":"2024-01-27T02:38:31.918380+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:34:25.075555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.929138+0000","last_change":"2024-01-27T02:38:31.929089+0000","last_active":"2024-01-27T02:38:31.929138+0000","last_peered":"2024-01-27T02:38:31.929138+0000","last_clean":"2024-01-27T02:38:31.929138+0000","last_became_active":"2024-01-27T02:38:31.928801+0000","last_became_peered":"2024-01-27T02:38:31.928801+0000","last_unstale":"2024-01-27T02:38:31.929138+0000","last_undegraded":"2024-01-27T02:38:31.929138+0000","last_fullsized":"2024-01-27T02:38:31.929138+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:30:10.336341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.938107+0000","last_change":"2024-01-27T02:38:30.902450+0000","last_active":"2024-01-27T02:38:31.938107+0000","last_peered":"2024-01-27T02:38:31.938107+0000","last_clean":"2024-01-27T02:38:31.938107+0000","last_became_active":"2024-01-27T02:38:30.902130+0000","last_became_peered":"2024-01-27T02:38:30.902130+0000","last_unstale":"2024-01-27T02:38:31.938107+0000","last_undegraded":"2024-01-27T02:38:31.938107+0000","last_fullsized":"2024-01-27T02:38:31.938107+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:39:25.843705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.928531+0000","last_change":"2024-01-27T02:38:31.928476+0000","last_active":"2024-01-27T02:38:31.928531+0000","last_peered":"2024-01-27T02:38:31.928531+0000","last_clean":"2024-01-27T02:38:31.928531+0000","last_became_active":"2024-01-27T02:38:31.927989+0000","last_became_peered":"2024-01-27T02:38:31.927989+0000","last_unstale":"2024-01-27T02:38:31.928531+0000","last_undegraded":"2024-01-27T02:38:31.928531+0000","last_fullsized":"2024-01-27T02:38:31.928531+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:12:26.885080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"132'1","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-01-27T02:38:32.909749+0000","last_change":"2024-01-27T02:38:30.901304+0000","last_active":"2024-01-27T02:38:32.909749+0000","last_peered":"2024-01-27T02:38:32.909749+0000","last_clean":"2024-01-27T02:38:32.909749+0000","last_became_active":"2024-01-27T02:38:30.901065+0000","last_became_peered":"2024-01-27T02:38:30.901065+0000","last_unstale":"2024-01-27T02:38:32.909749+0000","last_undegraded":"2024-01-27T02:38:32.909749+0000","last_fullsized":"2024-01-27T02:38:32.909749+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:16:12.621310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.925601+0000","last_change":"2024-01-27T02:38:31.925498+0000","last_active":"2024-01-27T02:38:31.925601+0000","last_peered":"2024-01-27T02:38:31.925601+0000","last_clean":"2024-01-27T02:38:31.925601+0000","last_became_active":"2024-01-27T02:38:31.924867+0000","last_became_peered":"2024-01-27T02:38:31.924867+0000","last_unstale":"2024-01-27T02:38:31.925601+0000","last_undegraded":"2024-01-27T02:38:31.925601+0000","last_fullsized":"2024-01-27T02:38:31.925601+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:14:28.293390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.919029+0000","last_change":"2024-01-27T02:38:30.901161+0000","last_active":"2024-01-27T02:38:31.919029+0000","last_peered":"2024-01-27T02:38:31.919029+0000","last_clean":"2024-01-27T02:38:31.919029+0000","last_became_active":"2024-01-27T02:38:30.899610+0000","last_became_peered":"2024-01-27T02:38:30.899610+0000","last_unstale":"2024-01-27T02:38:31.919029+0000","last_undegraded":"2024-01-27T02:38:31.919029+0000","last_fullsized":"2024-01-27T02:38:31.919029+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:30:09.332815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.929042+0000","last_change":"2024-01-27T02:38:31.928975+0000","last_active":"2024-01-27T02:38:31.929042+0000","last_peered":"2024-01-27T02:38:31.929042+0000","last_clean":"2024-01-27T02:38:31.929042+0000","last_became_active":"2024-01-27T02:38:31.928745+0000","last_became_peered":"2024-01-27T02:38:31.928745+0000","last_unstale":"2024-01-27T02:38:31.929042+0000","last_undegraded":"2024-01-27T02:38:31.929042+0000","last_fullsized":"2024-01-27T02:38:31.929042+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:11:34.261074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.908273+0000","last_change":"2024-01-27T02:38:30.904612+0000","last_active":"2024-01-27T02:38:31.908273+0000","last_peered":"2024-01-27T02:38:31.908273+0000","last_clean":"2024-01-27T02:38:31.908273+0000","last_became_active":"2024-01-27T02:38:30.904299+0000","last_became_peered":"2024-01-27T02:38:30.904299+0000","last_unstale":"2024-01-27T02:38:31.908273+0000","last_undegraded":"2024-01-27T02:38:31.908273+0000","last_fullsized":"2024-01-27T02:38:31.908273+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:16:12.621310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.916639+0000","last_change":"2024-01-27T02:38:31.916561+0000","last_active":"2024-01-27T02:38:31.916639+0000","last_peered":"2024-01-27T02:38:31.916639+0000","last_clean":"2024-01-27T02:38:31.916639+0000","last_became_active":"2024-01-27T02:38:31.916319+0000","last_became_peered":"2024-01-27T02:38:31.916319+0000","last_unstale":"2024-01-27T02:38:31.916639+0000","last_undegraded":"2024-01-27T02:38:31.916639+0000","last_fullsized":"2024-01-27T02:38:31.916639+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:05:23.541565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.913921+0000","last_change":"2024-01-27T02:38:30.899705+0000","last_active":"2024-01-27T02:38:31.913921+0000","last_peered":"2024-01-27T02:38:31.913921+0000","last_clean":"2024-01-27T02:38:31.913921+0000","last_became_active":"2024-01-27T02:38:30.899453+0000","last_became_peered":"2024-01-27T02:38:30.899453+0000","last_unstale":"2024-01-27T02:38:31.913921+0000","last_undegraded":"2024-01-27T02:38:31.913921+0000","last_fullsized":"2024-01-27T02:38:31.913921+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:02:35.435543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.916730+0000","last_change":"2024-01-27T02:38:31.916672+0000","last_active":"2024-01-27T02:38:31.916730+0000","last_peered":"2024-01-27T02:38:31.916730+0000","last_clean":"2024-01-27T02:38:31.916730+0000","last_became_active":"2024-01-27T02:38:31.916440+0000","last_became_peered":"2024-01-27T02:38:31.916440+0000","last_unstale":"2024-01-27T02:38:31.916730+0000","last_undegraded":"2024-01-27T02:38:31.916730+0000","last_fullsized":"2024-01-27T02:38:31.916730+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:47:43.751635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.912323+0000","last_change":"2024-01-27T02:38:30.902907+0000","last_active":"2024-01-27T02:38:31.912323+0000","last_peered":"2024-01-27T02:38:31.912323+0000","last_clean":"2024-01-27T02:38:31.912323+0000","last_became_active":"2024-01-27T02:38:30.902691+0000","last_became_peered":"2024-01-27T02:38:30.902691+0000","last_unstale":"2024-01-27T02:38:31.912323+0000","last_undegraded":"2024-01-27T02:38:31.912323+0000","last_fullsized":"2024-01-27T02:38:31.912323+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:25:13.514723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.921474+0000","last_change":"2024-01-27T02:38:31.921408+0000","last_active":"2024-01-27T02:38:31.921474+0000","last_peered":"2024-01-27T02:38:31.921474+0000","last_clean":"2024-01-27T02:38:31.921474+0000","last_became_active":"2024-01-27T02:38:31.920845+0000","last_became_peered":"2024-01-27T02:38:31.920845+0000","last_unstale":"2024-01-27T02:38:31.921474+0000","last_undegraded":"2024-01-27T02:38:31.921474+0000","last_fullsized":"2024-01-27T02:38:31.921474+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:34:26.079082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.914181+0000","last_change":"2024-01-27T02:38:30.906486+0000","last_active":"2024-01-27T02:38:31.914181+0000","last_peered":"2024-01-27T02:38:31.914181+0000","last_clean":"2024-01-27T02:38:31.914181+0000","last_became_active":"2024-01-27T02:38:30.906233+0000","last_became_peered":"2024-01-27T02:38:30.906233+0000","last_unstale":"2024-01-27T02:38:31.914181+0000","last_undegraded":"2024-01-27T02:38:31.914181+0000","last_fullsized":"2024-01-27T02:38:31.914181+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:27:08.109911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.927271+0000","last_change":"2024-01-27T02:38:31.926086+0000","last_active":"2024-01-27T02:38:31.927271+0000","last_peered":"2024-01-27T02:38:31.927271+0000","last_clean":"2024-01-27T02:38:31.927271+0000","last_became_active":"2024-01-27T02:38:31.924140+0000","last_became_peered":"2024-01-27T02:38:31.924140+0000","last_unstale":"2024-01-27T02:38:31.927271+0000","last_undegraded":"2024-01-27T02:38:31.927271+0000","last_fullsized":"2024-01-27T02:38:31.927271+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:16:55.806750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.919136+0000","last_change":"2024-01-27T02:38:30.897972+0000","last_active":"2024-01-27T02:38:31.919136+0000","last_peered":"2024-01-27T02:38:31.919136+0000","last_clean":"2024-01-27T02:38:31.919136+0000","last_became_active":"2024-01-27T02:38:30.897716+0000","last_became_peered":"2024-01-27T02:38:30.897716+0000","last_unstale":"2024-01-27T02:38:31.919136+0000","last_undegraded":"2024-01-27T02:38:31.919136+0000","last_fullsized":"2024-01-27T02:38:31.919136+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:07.698973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.924090+0000","last_change":"2024-01-27T02:38:31.923808+0000","last_active":"2024-01-27T02:38:31.924090+0000","last_peered":"2024-01-27T02:38:31.924090+0000","last_clean":"2024-01-27T02:38:31.924090+0000","last_became_active":"2024-01-27T02:38:31.922834+0000","last_became_peered":"2024-01-27T02:38:31.922834+0000","last_unstale":"2024-01-27T02:38:31.924090+0000","last_undegraded":"2024-01-27T02:38:31.924090+0000","last_fullsized":"2024-01-27T02:38:31.924090+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:02:03.382943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.915586+0000","last_change":"2024-01-27T02:38:30.901349+0000","last_active":"2024-01-27T02:38:31.915586+0000","last_peered":"2024-01-27T02:38:31.915586+0000","last_clean":"2024-01-27T02:38:31.915586+0000","last_became_active":"2024-01-27T02:38:30.900973+0000","last_became_peered":"2024-01-27T02:38:30.900973+0000","last_unstale":"2024-01-27T02:38:31.915586+0000","last_undegraded":"2024-01-27T02:38:31.915586+0000","last_fullsized":"2024-01-27T02:38:31.915586+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:55:50.241360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":259,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.916927+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:38:31.916927+0000","last_peered":"2024-01-27T02:38:31.916927+0000","last_clean":"2024-01-27T02:38:31.916927+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:38:31.916927+0000","last_undegraded":"2024-01-27T02:38:31.916927+0000","last_fullsized":"2024-01-27T02:38:31.916927+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574930,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1284,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194028,"statfs":{"total":200000000,"available":198685095,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77400000000000002,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70299999999999996,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.65500000000000003}]},{"osd":1,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.71999999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.501},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.42799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69999999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.54700000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.68200000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71399999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.76200000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.73099999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.48199999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.71799999999999997,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.58799999999999997}]},{"osd":4,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.47599999999999998,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.51100000000000001},"min":{"1min":0.33700000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.51900000000000002}]},{"osd":5,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51300000000000001,"15min":0.53400000000000003},"min":{"1min":0.35999999999999999,"5min":0.32100000000000001,"15min":0.307},"max":{"1min":0.68500000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.56299999999999994},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75800000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.78400000000000003}]},{"osd":6,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57199999999999995,"15min":0.59399999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.78800000000000003,"5min":0.80300000000000005,"15min":0.81299999999999994},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56100000000000005,"15min":0.57499999999999996},"min":{"1min":0.42499999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.73099999999999998,"5min":0.81599999999999995,"15min":0.90800000000000003},"last":0.67700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574930,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":707,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194605,"statfs":{"total":200000000,"available":199275981,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.64900000000000002,"15min":0.54200000000000004},"min":{"1min":0.442,"5min":0.36499999999999999,"15min":0.29999999999999999},"max":{"1min":4.125,"5min":4.125,"15min":4.125},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60599999999999998,"15min":0.53600000000000003},"min":{"1min":0.45100000000000001,"5min":0.35699999999999998,"15min":0.32700000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.628}]},{"osd":1,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.68400000000000005,"15min":0.55200000000000005},"min":{"1min":0.433,"5min":0.41999999999999998,"15min":0.34100000000000003},"max":{"1min":8.2699999999999996,"5min":8.2699999999999996,"15min":8.2699999999999996},"last":0.59899999999999998},{"interface":"front","average":{"1min":2.0409999999999999,"5min":0.90800000000000003,"15min":0.65600000000000003},"min":{"1min":0.46999999999999997,"5min":0.379,"15min":0.33800000000000002},"max":{"1min":26.411999999999999,"5min":26.411999999999999,"15min":26.411999999999999},"last":0.57299999999999995}]},{"osd":2,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":0.77700000000000002,"15min":0.60899999999999999},"min":{"1min":0.45100000000000001,"5min":0.41199999999999998,"15min":0.39600000000000002},"max":{"1min":17.143999999999998,"5min":17.143999999999998,"15min":17.143999999999998},"last":0.504},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.64900000000000002,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":0.69099999999999995}]},{"osd":3,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.63300000000000001,"15min":0.56799999999999995},"min":{"1min":0.45800000000000002,"5min":0.435,"15min":0.376},"max":{"1min":4.3689999999999998,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64100000000000001,"15min":0.58099999999999996},"min":{"1min":0.433,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.56200000000000006}]},{"osd":4,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.33200000000000002,"5min":0.32000000000000001,"15min":0.30199999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51600000000000001,"15min":0.54900000000000004},"min":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74299999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.60899999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.52100000000000002,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76100000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56000000000000005,"15min":0.59999999999999998},"min":{"1min":0.41099999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77000000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.65600000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.58999999999999997,"15min":0.61099999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79400000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55800000000000005,"15min":0.58499999999999996},"min":{"1min":0.40000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71499999999999997,"5min":1.038,"15min":1.038},"last":0.53900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":707,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194605,"statfs":{"total":200000000,"available":199275927,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.39700000000000002,"15min":0.377},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25800000000000001},"max":{"1min":0.50600000000000001,"5min":0.873,"15min":0.873},"last":0.496},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.41899999999999998,"15min":0.39900000000000002},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.53100000000000003,"5min":1.121,"15min":1.121},"last":0.38200000000000001}]},{"osd":2,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.40999999999999998,"15min":0.36899999999999999},"min":{"1min":0.27200000000000002,"5min":0.26600000000000001,"15min":0.23999999999999999},"max":{"1min":0.48599999999999999,"5min":1.25,"15min":1.25},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.45600000000000002,"15min":0.432},"min":{"1min":0.32000000000000001,"5min":0.30199999999999999,"15min":0.29899999999999999},"max":{"1min":0.58599999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.40600000000000003}]},{"osd":3,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.49399999999999999,"15min":0.46999999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.32100000000000001},"max":{"1min":0.66800000000000004,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.46999999999999997,"15min":0.45600000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.58199999999999996,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.44700000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.66300000000000003,"15min":0.63500000000000001},"min":{"1min":0.501,"5min":0.373,"15min":0.373},"max":{"1min":0.81599999999999995,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66200000000000003,"15min":0.63800000000000001},"min":{"1min":0.497,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.82599999999999996,"5min":1.026,"15min":1.026},"last":0.65200000000000002}]},{"osd":5,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.68899999999999995,"15min":0.63900000000000001},"min":{"1min":0.51000000000000001,"5min":0.46200000000000002,"15min":0.44},"max":{"1min":0.80900000000000005,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69099999999999995,"15min":0.67600000000000005},"min":{"1min":0.53000000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83799999999999997,"5min":3.181,"15min":3.181},"last":0.76900000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71199999999999997,"15min":0.69699999999999995},"min":{"1min":0.51400000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.875,"5min":1.024,"15min":1.024},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71499999999999997,"15min":0.68000000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.85699999999999998,"5min":1.869,"15min":1.869},"last":0.747}]},{"osd":7,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.72699999999999998,"15min":0.68700000000000006},"min":{"1min":0.55800000000000005,"5min":0.495,"15min":0.47299999999999998},"max":{"1min":0.82599999999999996,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.73099999999999998,"15min":0.69399999999999995},"min":{"1min":0.56000000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.86799999999999999,"5min":1.708,"15min":1.708},"last":0.81000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574930,"num_pgs":36,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1285,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194027,"statfs":{"total":200000000,"available":198684035,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39000000000000001},"min":{"1min":0.253,"5min":0.24399999999999999,"15min":0.23799999999999999},"max":{"1min":0.52100000000000002,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.41999999999999998,"15min":0.41599999999999998},"min":{"1min":0.29599999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.64200000000000002,"5min":0.65200000000000002,"15min":0.68400000000000005},"last":0.34599999999999997}]},{"osd":2,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.437,"15min":0.42199999999999999},"min":{"1min":0.30499999999999999,"5min":0.28699999999999998,"15min":0.28199999999999997},"max":{"1min":0.60299999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.374},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.44900000000000001,"15min":0.435},"min":{"1min":0.32400000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.628,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.38900000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42799999999999999,"15min":0.43099999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58199999999999996,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48499999999999999,"15min":0.47499999999999998},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69399999999999995,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.45100000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65900000000000003,"15min":0.64100000000000001},"min":{"1min":0.49399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91700000000000004,"5min":0.997,"15min":0.997},"last":0.505},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.44700000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92100000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.61799999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67700000000000005,"15min":0.65700000000000003},"min":{"1min":0.496,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88600000000000001,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.68100000000000005},"min":{"1min":0.53100000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.82099999999999995,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.57099999999999995}]},{"osd":6,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.57299999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.80300000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69599999999999995,"15min":0.68000000000000005},"min":{"1min":0.51400000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94299999999999995,"5min":1.04,"15min":1.04},"last":0.51600000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68899999999999995,"15min":0.69199999999999995},"min":{"1min":0.45900000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90800000000000003,"5min":1.014,"15min":1.014},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68700000000000006,"15min":0.69299999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.83899999999999997,"5min":0.95799999999999996,"15min":0.97299999999999998},"last":0.54500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194605,"statfs":{"total":200000000,"available":199276051,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.41399999999999998,"15min":0.40899999999999997},"min":{"1min":0.23599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.58099999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.45700000000000002,"15min":0.62},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.30499999999999999},"max":{"1min":0.64700000000000002,"5min":0.72799999999999998,"15min":6.2729999999999997},"last":0.44400000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.42999999999999999,"15min":0.42299999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.71899999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.433,"15min":0.44500000000000001},"min":{"1min":0.32600000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.57999999999999996,"5min":0.65300000000000002,"15min":0.70099999999999996},"last":0.48799999999999999}]},{"osd":3,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42399999999999999,"15min":0.41599999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.27800000000000002},"max":{"1min":0.64700000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.48699999999999999,"15min":0.48499999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.627,"5min":0.76500000000000001,"15min":0.80800000000000005},"last":0.51000000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68799999999999994,"15min":0.65800000000000003},"min":{"1min":0.46800000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93999999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70599999999999996,"15min":0.79000000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.006,"5min":1.006,"15min":3.581},"last":0.63400000000000001}]},{"osd":5,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65300000000000002,"15min":0.73499999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":3.5369999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92200000000000004,"5min":0.96599999999999997,"15min":1.0529999999999999},"last":0.64900000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.45000000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.51300000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97499999999999998,"5min":1.0189999999999999,"15min":1.0209999999999999},"last":0.59899999999999998}]},{"osd":7,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.68300000000000005,"15min":0.67800000000000005},"min":{"1min":0.50800000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95599999999999996,"5min":1.03,"15min":1.03},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.69799999999999995,"15min":0.68999999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.42699999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.72499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574930,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":708,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194603,"statfs":{"total":200000000,"available":199274445,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.45600000000000002,"15min":0.38200000000000001},"min":{"1min":0.28999999999999998,"5min":0.253,"15min":0.253},"max":{"1min":4.7380000000000004,"5min":4.7380000000000004,"15min":4.7380000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.41599999999999998,"15min":0.39800000000000002},"min":{"1min":0.27600000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.29399999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.47199999999999998,"15min":0.41199999999999998},"min":{"1min":0.27900000000000003,"5min":0.27400000000000002,"15min":0.22900000000000001},"max":{"1min":4.5739999999999998,"5min":4.5739999999999998,"15min":4.5739999999999998},"last":0.371},{"interface":"front","average":{"1min":0.752,"5min":0.52400000000000002,"15min":0.44700000000000001},"min":{"1min":0.311,"5min":0.30199999999999999,"15min":0.27900000000000003},"max":{"1min":5.0529999999999999,"5min":5.0529999999999999,"15min":5.0529999999999999},"last":0.34499999999999997}]},{"osd":2,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.48499999999999999,"15min":0.439},"min":{"1min":0.34799999999999998,"5min":0.33600000000000002,"15min":0.28100000000000003},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.55500000000000005,"15min":0.47399999999999998},"min":{"1min":0.375,"5min":0.35299999999999998,"15min":0.34599999999999997},"max":{"1min":5.1879999999999997,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":0.42799999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.69399999999999995,"15min":0.63600000000000001},"min":{"1min":0.42799999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0629999999999999,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66500000000000004,"15min":0.61399999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.68000000000000005}]},{"osd":5,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.67800000000000005,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.56699999999999995},{"interface":"front","average":{"1min":1.006,"5min":0.78500000000000003,"15min":0.70299999999999996},"min":{"1min":0.56200000000000006,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":4.7960000000000003,"5min":4.7960000000000003,"15min":4.7960000000000003},"last":0.69299999999999995}]},{"osd":6,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.73499999999999999,"15min":0.67800000000000005},"min":{"1min":0.58899999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":5.0069999999999997,"5min":5.0069999999999997,"15min":5.0069999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.68100000000000005,"15min":0.65400000000000003},"min":{"1min":0.504,"5min":0.435,"15min":0.435},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.70299999999999996}]},{"osd":7,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.74299999999999999,"15min":0.66300000000000003},"min":{"1min":0.57699999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.139,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.71299999999999997,"15min":0.67600000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.73899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194605,"statfs":{"total":200000000,"available":199276051,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.34599999999999997,"5min":0.33400000000000002,"15min":0.314},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.64600000000000002,"15min":0.60999999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":7.0670000000000002,"5min":7.0670000000000002,"15min":7.0670000000000002},"last":0.54900000000000004}]},{"osd":1,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.64700000000000002,"15min":0.60899999999999999},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":5.1529999999999996,"5min":5.1529999999999996,"15min":5.1529999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.626,"15min":0.60399999999999998},"min":{"1min":0.43099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.0419999999999998,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.46400000000000002}]},{"osd":2,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.64900000000000002,"15min":0.60999999999999999},"min":{"1min":0.46899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":5.0019999999999998,"5min":5.0019999999999998,"15min":5.0019999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.5}]},{"osd":3,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.64700000000000002,"15min":0.60099999999999998},"min":{"1min":0.42099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.7220000000000004,"5min":6.7220000000000004,"15min":6.7220000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.51000000000000001,"5min":0.44900000000000001,"15min":0.41399999999999998},"max":{"1min":6.5510000000000002,"5min":6.5510000000000002,"15min":6.5510000000000002},"last":0.53400000000000003}]},{"osd":5,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.56200000000000006,"15min":0.51400000000000001},"min":{"1min":0.38500000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":6.9269999999999996,"5min":6.9269999999999996,"15min":6.9269999999999996},"last":0.44400000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.55400000000000005,"15min":0.51700000000000002},"min":{"1min":0.40100000000000002,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":5.2229999999999999,"5min":5.2229999999999999,"15min":5.2229999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.52600000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.60799999999999998,"15min":0.57199999999999995},"min":{"1min":0.34799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":7.5339999999999998,"5min":7.5339999999999998,"15min":7.5339999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.63200000000000001,"15min":0.58999999999999997},"min":{"1min":0.44600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.9550000000000001,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":0.59599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":707,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194604,"statfs":{"total":200000000,"available":199275475,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56100000000000005,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.84199999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.443},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60599999999999998,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.34499999999999997,"15min":0.27600000000000002},"max":{"1min":0.78900000000000003,"5min":3.242,"15min":3.242},"last":0.58599999999999997}]},{"osd":1,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63,"15min":0.60999999999999999},"min":{"1min":0.47299999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80100000000000005,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.62,"15min":0.64700000000000002},"min":{"1min":0.46899999999999997,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77600000000000002,"5min":0.88500000000000001,"15min":0.95499999999999996},"last":0.55100000000000005}]},{"osd":2,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61699999999999999,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.372},"max":{"1min":0.80900000000000005,"5min":3.04,"15min":3.04},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59599999999999997,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74299999999999999,"5min":0.85899999999999999,"15min":0.89400000000000002},"last":0.53900000000000003}]},{"osd":3,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65000000000000002,"15min":0.67300000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82099999999999995,"5min":4.032,"15min":4.032},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.70599999999999996,"15min":0.67900000000000005},"min":{"1min":0.46000000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.78800000000000003,"5min":4.282,"15min":4.282},"last":0.56200000000000006}]},{"osd":4,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.371,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.747,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.376},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.499,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76300000000000001,"5min":0.79800000000000004,"15min":1.101},"last":0.49399999999999999}]},{"osd":6,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.502,"15min":0.55200000000000005},"min":{"1min":0.38600000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69199999999999995,"5min":0.73199999999999998,"15min":1.7969999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.58099999999999996},"min":{"1min":0.433,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83399999999999996,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.63600000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55100000000000005,"15min":0.56599999999999995},"min":{"1min":0.45000000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68899999999999995,"5min":0.91100000000000003,"15min":1.008},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.61499999999999999,"15min":0.65400000000000003},"min":{"1min":0.40699999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78400000000000003,"5min":2,"15min":2},"last":0.51000000000000001}]}]}],"pool_statfs":[]}} 2024-01-27T02:38:38.801 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:38:38.804 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:38:40.802 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:38:40.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.942+0000 7f2f0bf7f700 1 Processor -- start 2024-01-27T02:38:40.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.942+0000 7f2f0bf7f700 1 -- start start 2024-01-27T02:38:40.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.943+0000 7f2f0bf7f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2f040b1c00 0x7f2f0406e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:40.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.943+0000 7f2f0bf7f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2f0415b5f0 0x7f2f041566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:40.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.943+0000 7f2f0bf7f700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f04157610 con 0x7f2f040b1fd0 2024-01-27T02:38:40.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.943+0000 7f2f0bf7f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f04157310 con 0x7f2f0415b5f0 2024-01-27T02:38:40.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.943+0000 7f2f0bf7f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f0406ee00 con 0x7f2f040b1c00 2024-01-27T02:38:40.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.943+0000 7f2f0951a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2f0415b5f0 0x7f2f041566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:40.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.943+0000 7f2f0951a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2f0415b5f0 0x7f2f041566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58918/0 (socket says 172.21.15.7:58918) 2024-01-27T02:38:40.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.944+0000 7f2f0951a700 1 -- 172.21.15.7:0/1066170256 learned_addr learned my addr 172.21.15.7:0/1066170256 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:40.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.944+0000 7f2f09d1b700 1 --2- 172.21.15.7:0/1066170256 >> v2:172.21.15.188:3300/0 conn(0x7f2f040b1c00 0x7f2f0406e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:40.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.944+0000 7f2f0951a700 1 -- 172.21.15.7:0/1066170256 >> v2:172.21.15.188:3300/0 conn(0x7f2f040b1c00 msgr2=0x7f2f0406e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:40.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.944+0000 7f2f0951a700 1 --2- 172.21.15.7:0/1066170256 >> v2:172.21.15.188:3300/0 conn(0x7f2f040b1c00 0x7f2f0406e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:40.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.944+0000 7f2f0951a700 1 -- 172.21.15.7:0/1066170256 >> v1:172.21.15.188:6789/0 conn(0x7f2f040b1fd0 legacy=0x7f2f04156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:40.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.944+0000 7f2f0951a700 1 -- 172.21.15.7:0/1066170256 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0406f0e0 con 0x7f2f0415b5f0 2024-01-27T02:38:40.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.944+0000 7f2f0951a700 1 --2- 172.21.15.7:0/1066170256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f0415b5f0 0x7f2f041566c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2f0406e130 tx=0x7f2f0000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480c59eba596d113 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:40.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.944+0000 7f2f08d19700 1 -- 172.21.15.7:0/1066170256 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0000dba0 con 0x7f2f0415b5f0 2024-01-27T02:38:40.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.944+0000 7f2f08d19700 1 -- 172.21.15.7:0/1066170256 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0000dd00 con 0x7f2f0415b5f0 2024-01-27T02:38:40.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.944+0000 7f2f08d19700 1 -- 172.21.15.7:0/1066170256 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f000196d0 con 0x7f2f0415b5f0 2024-01-27T02:38:40.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.945+0000 7f2f0bf7f700 1 -- 172.21.15.7:0/1066170256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f0415b5f0 msgr2=0x7f2f041566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:40.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.945+0000 7f2f0bf7f700 1 --2- 172.21.15.7:0/1066170256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f0415b5f0 0x7f2f041566c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2f0406e130 tx=0x7f2f0000b4f0 comp rx=0 tx=0).stop 2024-01-27T02:38:40.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.945+0000 7f2f0bf7f700 1 -- 172.21.15.7:0/1066170256 shutdown_connections 2024-01-27T02:38:40.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.945+0000 7f2f0bf7f700 1 --2- 172.21.15.7:0/1066170256 >> v2:172.21.15.188:3300/0 conn(0x7f2f040b1c00 0x7f2f0406e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:40.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.945+0000 7f2f0bf7f700 1 --2- 172.21.15.7:0/1066170256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f0415b5f0 0x7f2f041566c0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:40.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.945+0000 7f2f0bf7f700 1 -- 172.21.15.7:0/1066170256 >> 172.21.15.7:0/1066170256 conn(0x7f2f04068430 msgr2=0x7f2f0406be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:40.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.945+0000 7f2f0bf7f700 1 -- 172.21.15.7:0/1066170256 shutdown_connections 2024-01-27T02:38:40.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.945+0000 7f2f0bf7f700 1 -- 172.21.15.7:0/1066170256 wait complete. 2024-01-27T02:38:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.946+0000 7f2f0bf7f700 1 Processor -- start 2024-01-27T02:38:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.946+0000 7f2f0bf7f700 1 -- start start 2024-01-27T02:38:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.947+0000 7f2f0bf7f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f040b1c00 0x7f2f0413dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.947+0000 7f2f0bf7f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f040b1fd0 0x7f2f04155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.947+0000 7f2f0bf7f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f0415b5f0 0x7f2f04155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.947+0000 7f2f0bf7f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f040aec70 con 0x7f2f040b1c00 2024-01-27T02:38:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.947+0000 7f2f0bf7f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f040aef50 con 0x7f2f0415b5f0 2024-01-27T02:38:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.947+0000 7f2f0bf7f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f040af230 con 0x7f2f040b1fd0 2024-01-27T02:38:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.947+0000 7f2f09d1b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f040b1c00 0x7f2f0413dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.947+0000 7f2f09d1b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f040b1c00 0x7f2f0413dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58924/0 (socket says 172.21.15.7:58924) 2024-01-27T02:38:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.947+0000 7f2f09d1b700 1 -- 172.21.15.7:0/474951557 learned_addr learned my addr 172.21.15.7:0/474951557 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.947+0000 7f2f0951a700 1 --2- 172.21.15.7:0/474951557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f040b1fd0 0x7f2f04155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.947+0000 7f2f09d1b700 1 -- 172.21.15.7:0/474951557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f040b1fd0 msgr2=0x7f2f04155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.947+0000 7f2f0a51c700 1 --2- 172.21.15.7:0/474951557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f0415b5f0 0x7f2f04155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.947+0000 7f2f09d1b700 1 --2- 172.21.15.7:0/474951557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f040b1fd0 0x7f2f04155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.947+0000 7f2f09d1b700 1 -- 172.21.15.7:0/474951557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f0415b5f0 msgr2=0x7f2f04155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.947+0000 7f2f09d1b700 1 --2- 172.21.15.7:0/474951557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f0415b5f0 0x7f2f04155b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:40.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.947+0000 7f2f09d1b700 1 -- 172.21.15.7:0/474951557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0000b790 con 0x7f2f040b1c00 2024-01-27T02:38:40.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.948+0000 7f2f09d1b700 1 --2- 172.21.15.7:0/474951557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f040b1c00 0x7f2f0413dfe0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800ec80 tx=0x7f2ef80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:40.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.948+0000 7f2ef5ffb700 1 -- 172.21.15.7:0/474951557 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef800c9d0 con 0x7f2f040b1c00 2024-01-27T02:38:40.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.948+0000 7f2ef5ffb700 1 -- 172.21.15.7:0/474951557 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef800cb30 con 0x7f2f040b1c00 2024-01-27T02:38:40.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.948+0000 7f2f0bf7f700 1 -- 172.21.15.7:0/474951557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0413ea60 con 0x7f2f040b1c00 2024-01-27T02:38:40.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.948+0000 7f2ef5ffb700 1 -- 172.21.15.7:0/474951557 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8013470 con 0x7f2f040b1c00 2024-01-27T02:38:40.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.948+0000 7f2f0bf7f700 1 -- 172.21.15.7:0/474951557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f040adf00 con 0x7f2f040b1c00 2024-01-27T02:38:40.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.949+0000 7f2f0bf7f700 1 -- 172.21.15.7:0/474951557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f0413edd0 con 0x7f2f040b1c00 2024-01-27T02:38:40.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.950+0000 7f2ef5ffb700 1 -- 172.21.15.7:0/474951557 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2ef8013820 con 0x7f2f040b1c00 2024-01-27T02:38:40.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.950+0000 7f2ef5ffb700 1 --2- 172.21.15.7:0/474951557 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2ee8066f30 0x7f2ee80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:40.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.950+0000 7f2ef5ffb700 1 -- 172.21.15.7:0/474951557 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2ef80848c0 con 0x7f2f040b1c00 2024-01-27T02:38:40.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.950+0000 7f2f0951a700 1 --2- 172.21.15.7:0/474951557 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2ee8066f30 0x7f2ee80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:40.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.951+0000 7f2f0951a700 1 --2- 172.21.15.7:0/474951557 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2ee8066f30 0x7f2ee80693e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2f0406e130 tx=0x7f2f0001b000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:40.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:40.955+0000 7f2ef5ffb700 1 -- 172.21.15.7:0/474951557 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ef8051b30 con 0x7f2f040b1c00 2024-01-27T02:38:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.153+0000 7f2f0bf7f700 1 -- 172.21.15.7:0/474951557 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2f0413e8a0 con 0x7f2ee8066f30 2024-01-27T02:38:41.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.161+0000 7f2ef5ffb700 1 -- 172.21.15.7:0/474951557 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285448 (secure 0 0 0) 0x7f2f0413e8a0 con 0x7f2ee8066f30 2024-01-27T02:38:41.166 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:41.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.165+0000 7f2f0bf7f700 1 -- 172.21.15.7:0/474951557 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2ee8066f30 msgr2=0x7f2ee80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:41.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.165+0000 7f2f0bf7f700 1 --2- 172.21.15.7:0/474951557 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2ee8066f30 0x7f2ee80693e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2f0406e130 tx=0x7f2f0001b000 comp rx=0 tx=0).stop 2024-01-27T02:38:41.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.165+0000 7f2f0bf7f700 1 -- 172.21.15.7:0/474951557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f040b1c00 msgr2=0x7f2f0413dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:41.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.165+0000 7f2f0bf7f700 1 --2- 172.21.15.7:0/474951557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f040b1c00 0x7f2f0413dfe0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800ec80 tx=0x7f2ef80091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:41.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.165+0000 7f2f0bf7f700 1 -- 172.21.15.7:0/474951557 shutdown_connections 2024-01-27T02:38:41.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.165+0000 7f2f0bf7f700 1 --2- 172.21.15.7:0/474951557 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2ee8066f30 0x7f2ee80693e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:41.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.165+0000 7f2f0bf7f700 1 --2- 172.21.15.7:0/474951557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f0415b5f0 0x7f2f04155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:41.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.165+0000 7f2f0bf7f700 1 --2- 172.21.15.7:0/474951557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f040b1c00 0x7f2f0413dfe0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:41.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.165+0000 7f2f0bf7f700 1 --2- 172.21.15.7:0/474951557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f040b1fd0 0x7f2f04155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:41.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.165+0000 7f2f0bf7f700 1 -- 172.21.15.7:0/474951557 >> 172.21.15.7:0/474951557 conn(0x7f2f04068430 msgr2=0x7f2f04152600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:41.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.165+0000 7f2f0bf7f700 1 -- 172.21.15.7:0/474951557 shutdown_connections 2024-01-27T02:38:41.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.165+0000 7f2f0bf7f700 1 -- 172.21.15.7:0/474951557 wait complete. 2024-01-27T02:38:41.169 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:38:41.180 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-01-27T02:38:39.668315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6813,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555686,"statfs":{"total":1600000000,"available":1593023060,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001986"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.925873+0000","last_change":"2024-01-27T02:38:31.925629+0000","last_active":"2024-01-27T02:38:31.925873+0000","last_peered":"2024-01-27T02:38:31.925873+0000","last_clean":"2024-01-27T02:38:31.925873+0000","last_became_active":"2024-01-27T02:38:31.924701+0000","last_became_peered":"2024-01-27T02:38:31.924701+0000","last_unstale":"2024-01-27T02:38:31.925873+0000","last_undegraded":"2024-01-27T02:38:31.925873+0000","last_fullsized":"2024-01-27T02:38:31.925873+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:14:38.389382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.926896+0000","last_change":"2024-01-27T02:38:31.926735+0000","last_active":"2024-01-27T02:38:31.926896+0000","last_peered":"2024-01-27T02:38:31.926896+0000","last_clean":"2024-01-27T02:38:31.926896+0000","last_became_active":"2024-01-27T02:38:31.926181+0000","last_became_peered":"2024-01-27T02:38:31.926181+0000","last_unstale":"2024-01-27T02:38:31.926896+0000","last_undegraded":"2024-01-27T02:38:31.926896+0000","last_fullsized":"2024-01-27T02:38:31.926896+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:40:17.611985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.924582+0000","last_change":"2024-01-27T02:38:31.924341+0000","last_active":"2024-01-27T02:38:31.924582+0000","last_peered":"2024-01-27T02:38:31.924582+0000","last_clean":"2024-01-27T02:38:31.924582+0000","last_became_active":"2024-01-27T02:38:31.923266+0000","last_became_peered":"2024-01-27T02:38:31.923266+0000","last_unstale":"2024-01-27T02:38:31.924582+0000","last_undegraded":"2024-01-27T02:38:31.924582+0000","last_fullsized":"2024-01-27T02:38:31.924582+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:00:23.173837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.927328+0000","last_change":"2024-01-27T02:38:31.926614+0000","last_active":"2024-01-27T02:38:31.927328+0000","last_peered":"2024-01-27T02:38:31.927328+0000","last_clean":"2024-01-27T02:38:31.927328+0000","last_became_active":"2024-01-27T02:38:31.926200+0000","last_became_peered":"2024-01-27T02:38:31.926200+0000","last_unstale":"2024-01-27T02:38:31.927328+0000","last_undegraded":"2024-01-27T02:38:31.927328+0000","last_fullsized":"2024-01-27T02:38:31.927328+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:29:49.270287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.916155+0000","last_change":"2024-01-27T02:38:31.915978+0000","last_active":"2024-01-27T02:38:31.916155+0000","last_peered":"2024-01-27T02:38:31.916155+0000","last_clean":"2024-01-27T02:38:31.916155+0000","last_became_active":"2024-01-27T02:38:31.915733+0000","last_became_peered":"2024-01-27T02:38:31.915733+0000","last_unstale":"2024-01-27T02:38:31.916155+0000","last_undegraded":"2024-01-27T02:38:31.916155+0000","last_fullsized":"2024-01-27T02:38:31.916155+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:01:39.184586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.921125+0000","last_change":"2024-01-27T02:38:31.921025+0000","last_active":"2024-01-27T02:38:31.921125+0000","last_peered":"2024-01-27T02:38:31.921125+0000","last_clean":"2024-01-27T02:38:31.921125+0000","last_became_active":"2024-01-27T02:38:31.920667+0000","last_became_peered":"2024-01-27T02:38:31.920667+0000","last_unstale":"2024-01-27T02:38:31.921125+0000","last_undegraded":"2024-01-27T02:38:31.921125+0000","last_fullsized":"2024-01-27T02:38:31.921125+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:44:22.418248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.921472+0000","last_change":"2024-01-27T02:38:31.921408+0000","last_active":"2024-01-27T02:38:31.921472+0000","last_peered":"2024-01-27T02:38:31.921472+0000","last_clean":"2024-01-27T02:38:31.921472+0000","last_became_active":"2024-01-27T02:38:31.921189+0000","last_became_peered":"2024-01-27T02:38:31.921189+0000","last_unstale":"2024-01-27T02:38:31.921472+0000","last_undegraded":"2024-01-27T02:38:31.921472+0000","last_fullsized":"2024-01-27T02:38:31.921472+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:18:12.582908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.924598+0000","last_change":"2024-01-27T02:38:31.924520+0000","last_active":"2024-01-27T02:38:31.924598+0000","last_peered":"2024-01-27T02:38:31.924598+0000","last_clean":"2024-01-27T02:38:31.924598+0000","last_became_active":"2024-01-27T02:38:31.923881+0000","last_became_peered":"2024-01-27T02:38:31.923881+0000","last_unstale":"2024-01-27T02:38:31.924598+0000","last_undegraded":"2024-01-27T02:38:31.924598+0000","last_fullsized":"2024-01-27T02:38:31.924598+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:22:55.022123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.927311+0000","last_change":"2024-01-27T02:38:31.926457+0000","last_active":"2024-01-27T02:38:31.927311+0000","last_peered":"2024-01-27T02:38:31.927311+0000","last_clean":"2024-01-27T02:38:31.927311+0000","last_became_active":"2024-01-27T02:38:31.925649+0000","last_became_peered":"2024-01-27T02:38:31.925649+0000","last_unstale":"2024-01-27T02:38:31.927311+0000","last_undegraded":"2024-01-27T02:38:31.927311+0000","last_fullsized":"2024-01-27T02:38:31.927311+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:00:12.784356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.924416+0000","last_change":"2024-01-27T02:38:31.923623+0000","last_active":"2024-01-27T02:38:31.924416+0000","last_peered":"2024-01-27T02:38:31.924416+0000","last_clean":"2024-01-27T02:38:31.924416+0000","last_became_active":"2024-01-27T02:38:31.922781+0000","last_became_peered":"2024-01-27T02:38:31.922781+0000","last_unstale":"2024-01-27T02:38:31.924416+0000","last_undegraded":"2024-01-27T02:38:31.924416+0000","last_fullsized":"2024-01-27T02:38:31.924416+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:37:50.476235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.925225+0000","last_change":"2024-01-27T02:38:31.925132+0000","last_active":"2024-01-27T02:38:31.925225+0000","last_peered":"2024-01-27T02:38:31.925225+0000","last_clean":"2024-01-27T02:38:31.925225+0000","last_became_active":"2024-01-27T02:38:31.924831+0000","last_became_peered":"2024-01-27T02:38:31.924831+0000","last_unstale":"2024-01-27T02:38:31.925225+0000","last_undegraded":"2024-01-27T02:38:31.925225+0000","last_fullsized":"2024-01-27T02:38:31.925225+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:20:34.579927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.928284+0000","last_change":"2024-01-27T02:38:31.928214+0000","last_active":"2024-01-27T02:38:31.928284+0000","last_peered":"2024-01-27T02:38:31.928284+0000","last_clean":"2024-01-27T02:38:31.928284+0000","last_became_active":"2024-01-27T02:38:31.926113+0000","last_became_peered":"2024-01-27T02:38:31.926113+0000","last_unstale":"2024-01-27T02:38:31.928284+0000","last_undegraded":"2024-01-27T02:38:31.928284+0000","last_fullsized":"2024-01-27T02:38:31.928284+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:59:44.278065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.927538+0000","last_change":"2024-01-27T02:38:31.927475+0000","last_active":"2024-01-27T02:38:31.927538+0000","last_peered":"2024-01-27T02:38:31.927538+0000","last_clean":"2024-01-27T02:38:31.927538+0000","last_became_active":"2024-01-27T02:38:31.926843+0000","last_became_peered":"2024-01-27T02:38:31.926843+0000","last_unstale":"2024-01-27T02:38:31.927538+0000","last_undegraded":"2024-01-27T02:38:31.927538+0000","last_fullsized":"2024-01-27T02:38:31.927538+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:54:13.383710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.923085+0000","last_change":"2024-01-27T02:38:31.922687+0000","last_active":"2024-01-27T02:38:31.923085+0000","last_peered":"2024-01-27T02:38:31.923085+0000","last_clean":"2024-01-27T02:38:31.923085+0000","last_became_active":"2024-01-27T02:38:31.922433+0000","last_became_peered":"2024-01-27T02:38:31.922433+0000","last_unstale":"2024-01-27T02:38:31.923085+0000","last_undegraded":"2024-01-27T02:38:31.923085+0000","last_fullsized":"2024-01-27T02:38:31.923085+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:54:15.890067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.922490+0000","last_change":"2024-01-27T02:38:31.922397+0000","last_active":"2024-01-27T02:38:31.922490+0000","last_peered":"2024-01-27T02:38:31.922490+0000","last_clean":"2024-01-27T02:38:31.922490+0000","last_became_active":"2024-01-27T02:38:31.922139+0000","last_became_peered":"2024-01-27T02:38:31.922139+0000","last_unstale":"2024-01-27T02:38:31.922490+0000","last_undegraded":"2024-01-27T02:38:31.922490+0000","last_fullsized":"2024-01-27T02:38:31.922490+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:36:36.290157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.925575+0000","last_change":"2024-01-27T02:38:31.925434+0000","last_active":"2024-01-27T02:38:31.925575+0000","last_peered":"2024-01-27T02:38:31.925575+0000","last_clean":"2024-01-27T02:38:31.925575+0000","last_became_active":"2024-01-27T02:38:31.924412+0000","last_became_peered":"2024-01-27T02:38:31.924412+0000","last_unstale":"2024-01-27T02:38:31.925575+0000","last_undegraded":"2024-01-27T02:38:31.925575+0000","last_fullsized":"2024-01-27T02:38:31.925575+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:39:41.293202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.926463+0000","last_change":"2024-01-27T02:38:31.926401+0000","last_active":"2024-01-27T02:38:31.926463+0000","last_peered":"2024-01-27T02:38:31.926463+0000","last_clean":"2024-01-27T02:38:31.926463+0000","last_became_active":"2024-01-27T02:38:31.925453+0000","last_became_peered":"2024-01-27T02:38:31.925453+0000","last_unstale":"2024-01-27T02:38:31.926463+0000","last_undegraded":"2024-01-27T02:38:31.926463+0000","last_fullsized":"2024-01-27T02:38:31.926463+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:30:10.336341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.928420+0000","last_change":"2024-01-27T02:38:31.927136+0000","last_active":"2024-01-27T02:38:31.928420+0000","last_peered":"2024-01-27T02:38:31.928420+0000","last_clean":"2024-01-27T02:38:31.928420+0000","last_became_active":"2024-01-27T02:38:31.926906+0000","last_became_peered":"2024-01-27T02:38:31.926906+0000","last_unstale":"2024-01-27T02:38:31.928420+0000","last_undegraded":"2024-01-27T02:38:31.928420+0000","last_fullsized":"2024-01-27T02:38:31.928420+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:08.702500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.921615+0000","last_change":"2024-01-27T02:38:31.921550+0000","last_active":"2024-01-27T02:38:31.921615+0000","last_peered":"2024-01-27T02:38:31.921615+0000","last_clean":"2024-01-27T02:38:31.921615+0000","last_became_active":"2024-01-27T02:38:31.921163+0000","last_became_peered":"2024-01-27T02:38:31.921163+0000","last_unstale":"2024-01-27T02:38:31.921615+0000","last_undegraded":"2024-01-27T02:38:31.921615+0000","last_fullsized":"2024-01-27T02:38:31.921615+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:11:36.249589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.918985+0000","last_change":"2024-01-27T02:38:31.918643+0000","last_active":"2024-01-27T02:38:31.918985+0000","last_peered":"2024-01-27T02:38:31.918985+0000","last_clean":"2024-01-27T02:38:31.918985+0000","last_became_active":"2024-01-27T02:38:31.918187+0000","last_became_peered":"2024-01-27T02:38:31.918187+0000","last_unstale":"2024-01-27T02:38:31.918985+0000","last_undegraded":"2024-01-27T02:38:31.918985+0000","last_fullsized":"2024-01-27T02:38:31.918985+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:34:26.079082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.912132+0000","last_change":"2024-01-27T02:38:31.912046+0000","last_active":"2024-01-27T02:38:31.912132+0000","last_peered":"2024-01-27T02:38:31.912132+0000","last_clean":"2024-01-27T02:38:31.912132+0000","last_became_active":"2024-01-27T02:38:31.911789+0000","last_became_peered":"2024-01-27T02:38:31.911789+0000","last_unstale":"2024-01-27T02:38:31.912132+0000","last_undegraded":"2024-01-27T02:38:31.912132+0000","last_fullsized":"2024-01-27T02:38:31.912132+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:51.023684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.924055+0000","last_change":"2024-01-27T02:38:31.923490+0000","last_active":"2024-01-27T02:38:31.924055+0000","last_peered":"2024-01-27T02:38:31.924055+0000","last_clean":"2024-01-27T02:38:31.924055+0000","last_became_active":"2024-01-27T02:38:31.923210+0000","last_became_peered":"2024-01-27T02:38:31.923210+0000","last_unstale":"2024-01-27T02:38:31.924055+0000","last_undegraded":"2024-01-27T02:38:31.924055+0000","last_fullsized":"2024-01-27T02:38:31.924055+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:54:15.890067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.918931+0000","last_change":"2024-01-27T02:38:31.917222+0000","last_active":"2024-01-27T02:38:31.918931+0000","last_peered":"2024-01-27T02:38:31.918931+0000","last_clean":"2024-01-27T02:38:31.918931+0000","last_became_active":"2024-01-27T02:38:31.917004+0000","last_became_peered":"2024-01-27T02:38:31.917004+0000","last_unstale":"2024-01-27T02:38:31.918931+0000","last_undegraded":"2024-01-27T02:38:31.918931+0000","last_fullsized":"2024-01-27T02:38:31.918931+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:23:46.870039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.919670+0000","last_change":"2024-01-27T02:38:31.919512+0000","last_active":"2024-01-27T02:38:31.919670+0000","last_peered":"2024-01-27T02:38:31.919670+0000","last_clean":"2024-01-27T02:38:31.919670+0000","last_became_active":"2024-01-27T02:38:31.918679+0000","last_became_peered":"2024-01-27T02:38:31.918679+0000","last_unstale":"2024-01-27T02:38:31.919670+0000","last_undegraded":"2024-01-27T02:38:31.919670+0000","last_fullsized":"2024-01-27T02:38:31.919670+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:07.978917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.925909+0000","last_change":"2024-01-27T02:38:31.925271+0000","last_active":"2024-01-27T02:38:31.925909+0000","last_peered":"2024-01-27T02:38:31.925909+0000","last_clean":"2024-01-27T02:38:31.925909+0000","last_became_active":"2024-01-27T02:38:31.925019+0000","last_became_peered":"2024-01-27T02:38:31.925019+0000","last_unstale":"2024-01-27T02:38:31.925909+0000","last_undegraded":"2024-01-27T02:38:31.925909+0000","last_fullsized":"2024-01-27T02:38:31.925909+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:51:24.348901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.925243+0000","last_change":"2024-01-27T02:38:31.925134+0000","last_active":"2024-01-27T02:38:31.925243+0000","last_peered":"2024-01-27T02:38:31.925243+0000","last_clean":"2024-01-27T02:38:31.925243+0000","last_became_active":"2024-01-27T02:38:31.924109+0000","last_became_peered":"2024-01-27T02:38:31.924109+0000","last_unstale":"2024-01-27T02:38:31.925243+0000","last_undegraded":"2024-01-27T02:38:31.925243+0000","last_fullsized":"2024-01-27T02:38:31.925243+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:21:44.987764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.929805+0000","last_change":"2024-01-27T02:38:31.929737+0000","last_active":"2024-01-27T02:38:31.929805+0000","last_peered":"2024-01-27T02:38:31.929805+0000","last_clean":"2024-01-27T02:38:31.929805+0000","last_became_active":"2024-01-27T02:38:31.929489+0000","last_became_peered":"2024-01-27T02:38:31.929489+0000","last_unstale":"2024-01-27T02:38:31.929805+0000","last_undegraded":"2024-01-27T02:38:31.929805+0000","last_fullsized":"2024-01-27T02:38:31.929805+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:03:26.614534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.927319+0000","last_change":"2024-01-27T02:38:31.927229+0000","last_active":"2024-01-27T02:38:31.927319+0000","last_peered":"2024-01-27T02:38:31.927319+0000","last_clean":"2024-01-27T02:38:31.927319+0000","last_became_active":"2024-01-27T02:38:31.926973+0000","last_became_peered":"2024-01-27T02:38:31.926973+0000","last_unstale":"2024-01-27T02:38:31.927319+0000","last_undegraded":"2024-01-27T02:38:31.927319+0000","last_fullsized":"2024-01-27T02:38:31.927319+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:40:17.611985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.918966+0000","last_change":"2024-01-27T02:38:31.918415+0000","last_active":"2024-01-27T02:38:31.918966+0000","last_peered":"2024-01-27T02:38:31.918966+0000","last_clean":"2024-01-27T02:38:31.918966+0000","last_became_active":"2024-01-27T02:38:31.918138+0000","last_became_peered":"2024-01-27T02:38:31.918138+0000","last_unstale":"2024-01-27T02:38:31.918966+0000","last_undegraded":"2024-01-27T02:38:31.918966+0000","last_fullsized":"2024-01-27T02:38:31.918966+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:54:15.890067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.915353+0000","last_change":"2024-01-27T02:38:31.915280+0000","last_active":"2024-01-27T02:38:31.915353+0000","last_peered":"2024-01-27T02:38:31.915353+0000","last_clean":"2024-01-27T02:38:31.915353+0000","last_became_active":"2024-01-27T02:38:31.915025+0000","last_became_peered":"2024-01-27T02:38:31.915025+0000","last_unstale":"2024-01-27T02:38:31.915353+0000","last_undegraded":"2024-01-27T02:38:31.915353+0000","last_fullsized":"2024-01-27T02:38:31.915353+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:25:14.518250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.925052+0000","last_change":"2024-01-27T02:38:31.924959+0000","last_active":"2024-01-27T02:38:31.925052+0000","last_peered":"2024-01-27T02:38:31.925052+0000","last_clean":"2024-01-27T02:38:31.925052+0000","last_became_active":"2024-01-27T02:38:31.923077+0000","last_became_peered":"2024-01-27T02:38:31.923077+0000","last_unstale":"2024-01-27T02:38:31.925052+0000","last_undegraded":"2024-01-27T02:38:31.925052+0000","last_fullsized":"2024-01-27T02:38:31.925052+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:27:09.408626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.921090+0000","last_change":"2024-01-27T02:38:31.920385+0000","last_active":"2024-01-27T02:38:31.921090+0000","last_peered":"2024-01-27T02:38:31.921090+0000","last_clean":"2024-01-27T02:38:31.921090+0000","last_became_active":"2024-01-27T02:38:31.920104+0000","last_became_peered":"2024-01-27T02:38:31.920104+0000","last_unstale":"2024-01-27T02:38:31.921090+0000","last_undegraded":"2024-01-27T02:38:31.921090+0000","last_fullsized":"2024-01-27T02:38:31.921090+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:36:50.676219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.919758+0000","last_change":"2024-01-27T02:38:31.919669+0000","last_active":"2024-01-27T02:38:31.919758+0000","last_peered":"2024-01-27T02:38:31.919758+0000","last_clean":"2024-01-27T02:38:31.919758+0000","last_became_active":"2024-01-27T02:38:31.919354+0000","last_became_peered":"2024-01-27T02:38:31.919354+0000","last_unstale":"2024-01-27T02:38:31.919758+0000","last_undegraded":"2024-01-27T02:38:31.919758+0000","last_fullsized":"2024-01-27T02:38:31.919758+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:23:02.614674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"131'4","reported_seq":14,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.934542+0000","last_change":"2024-01-27T02:38:30.900146+0000","last_active":"2024-01-27T02:38:31.934542+0000","last_peered":"2024-01-27T02:38:31.934542+0000","last_clean":"2024-01-27T02:38:31.934542+0000","last_became_active":"2024-01-27T02:38:30.899920+0000","last_became_peered":"2024-01-27T02:38:30.899920+0000","last_unstale":"2024-01-27T02:38:31.934542+0000","last_undegraded":"2024-01-27T02:38:31.934542+0000","last_fullsized":"2024-01-27T02:38:31.934542+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:02:26.763414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.928408+0000","last_change":"2024-01-27T02:38:31.924700+0000","last_active":"2024-01-27T02:38:31.928408+0000","last_peered":"2024-01-27T02:38:31.928408+0000","last_clean":"2024-01-27T02:38:31.928408+0000","last_became_active":"2024-01-27T02:38:31.924467+0000","last_became_peered":"2024-01-27T02:38:31.924467+0000","last_unstale":"2024-01-27T02:38:31.928408+0000","last_undegraded":"2024-01-27T02:38:31.928408+0000","last_fullsized":"2024-01-27T02:38:31.928408+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:05:23.541565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.934021+0000","last_change":"2024-01-27T02:38:30.900419+0000","last_active":"2024-01-27T02:38:31.934021+0000","last_peered":"2024-01-27T02:38:31.934021+0000","last_clean":"2024-01-27T02:38:31.934021+0000","last_became_active":"2024-01-27T02:38:30.900188+0000","last_became_peered":"2024-01-27T02:38:30.900188+0000","last_unstale":"2024-01-27T02:38:31.934021+0000","last_undegraded":"2024-01-27T02:38:31.934021+0000","last_fullsized":"2024-01-27T02:38:31.934021+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:16:12.621310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.921107+0000","last_change":"2024-01-27T02:38:31.920894+0000","last_active":"2024-01-27T02:38:31.921107+0000","last_peered":"2024-01-27T02:38:31.921107+0000","last_clean":"2024-01-27T02:38:31.921107+0000","last_became_active":"2024-01-27T02:38:31.920365+0000","last_became_peered":"2024-01-27T02:38:31.920365+0000","last_unstale":"2024-01-27T02:38:31.921107+0000","last_undegraded":"2024-01-27T02:38:31.921107+0000","last_fullsized":"2024-01-27T02:38:31.921107+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:55:51.244887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.937266+0000","last_change":"2024-01-27T02:38:30.907104+0000","last_active":"2024-01-27T02:38:31.937266+0000","last_peered":"2024-01-27T02:38:31.937266+0000","last_clean":"2024-01-27T02:38:31.937266+0000","last_became_active":"2024-01-27T02:38:30.906807+0000","last_became_peered":"2024-01-27T02:38:30.906807+0000","last_unstale":"2024-01-27T02:38:31.937266+0000","last_undegraded":"2024-01-27T02:38:31.937266+0000","last_fullsized":"2024-01-27T02:38:31.937266+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:16:05.064427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.925359+0000","last_change":"2024-01-27T02:38:31.925298+0000","last_active":"2024-01-27T02:38:31.925359+0000","last_peered":"2024-01-27T02:38:31.925359+0000","last_clean":"2024-01-27T02:38:31.925359+0000","last_became_active":"2024-01-27T02:38:31.925075+0000","last_became_peered":"2024-01-27T02:38:31.925075+0000","last_unstale":"2024-01-27T02:38:31.925359+0000","last_undegraded":"2024-01-27T02:38:31.925359+0000","last_fullsized":"2024-01-27T02:38:31.925359+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:56:07.978917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.936708+0000","last_change":"2024-01-27T02:38:30.906090+0000","last_active":"2024-01-27T02:38:31.936708+0000","last_peered":"2024-01-27T02:38:31.936708+0000","last_clean":"2024-01-27T02:38:31.936708+0000","last_became_active":"2024-01-27T02:38:30.905815+0000","last_became_peered":"2024-01-27T02:38:30.905815+0000","last_unstale":"2024-01-27T02:38:31.936708+0000","last_undegraded":"2024-01-27T02:38:31.936708+0000","last_fullsized":"2024-01-27T02:38:31.936708+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:42:16.605919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.927368+0000","last_change":"2024-01-27T02:38:31.927281+0000","last_active":"2024-01-27T02:38:31.927368+0000","last_peered":"2024-01-27T02:38:31.927368+0000","last_clean":"2024-01-27T02:38:31.927368+0000","last_became_active":"2024-01-27T02:38:31.926844+0000","last_became_peered":"2024-01-27T02:38:31.926844+0000","last_unstale":"2024-01-27T02:38:31.927368+0000","last_undegraded":"2024-01-27T02:38:31.927368+0000","last_fullsized":"2024-01-27T02:38:31.927368+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:13:24.438761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.933763+0000","last_change":"2024-01-27T02:38:30.908846+0000","last_active":"2024-01-27T02:38:31.933763+0000","last_peered":"2024-01-27T02:38:31.933763+0000","last_clean":"2024-01-27T02:38:31.933763+0000","last_became_active":"2024-01-27T02:38:30.908634+0000","last_became_peered":"2024-01-27T02:38:30.908634+0000","last_unstale":"2024-01-27T02:38:31.933763+0000","last_undegraded":"2024-01-27T02:38:31.933763+0000","last_fullsized":"2024-01-27T02:38:31.933763+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:44:21.414722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.926877+0000","last_change":"2024-01-27T02:38:31.926584+0000","last_active":"2024-01-27T02:38:31.926877+0000","last_peered":"2024-01-27T02:38:31.926877+0000","last_clean":"2024-01-27T02:38:31.926877+0000","last_became_active":"2024-01-27T02:38:31.925803+0000","last_became_peered":"2024-01-27T02:38:31.925803+0000","last_unstale":"2024-01-27T02:38:31.926877+0000","last_undegraded":"2024-01-27T02:38:31.926877+0000","last_fullsized":"2024-01-27T02:38:31.926877+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:11:36.249589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.934988+0000","last_change":"2024-01-27T02:38:30.897433+0000","last_active":"2024-01-27T02:38:31.934988+0000","last_peered":"2024-01-27T02:38:31.934988+0000","last_clean":"2024-01-27T02:38:31.934988+0000","last_became_active":"2024-01-27T02:38:30.897134+0000","last_became_peered":"2024-01-27T02:38:30.897134+0000","last_unstale":"2024-01-27T02:38:31.934988+0000","last_undegraded":"2024-01-27T02:38:31.934988+0000","last_fullsized":"2024-01-27T02:38:31.934988+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:18:04.098263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.927292+0000","last_change":"2024-01-27T02:38:31.926378+0000","last_active":"2024-01-27T02:38:31.927292+0000","last_peered":"2024-01-27T02:38:31.927292+0000","last_clean":"2024-01-27T02:38:31.927292+0000","last_became_active":"2024-01-27T02:38:31.925226+0000","last_became_peered":"2024-01-27T02:38:31.925226+0000","last_unstale":"2024-01-27T02:38:31.927292+0000","last_undegraded":"2024-01-27T02:38:31.927292+0000","last_fullsized":"2024-01-27T02:38:31.927292+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:27:05.440576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.911415+0000","last_change":"2024-01-27T02:38:30.903018+0000","last_active":"2024-01-27T02:38:31.911415+0000","last_peered":"2024-01-27T02:38:31.911415+0000","last_clean":"2024-01-27T02:38:31.911415+0000","last_became_active":"2024-01-27T02:38:30.902748+0000","last_became_peered":"2024-01-27T02:38:30.902748+0000","last_unstale":"2024-01-27T02:38:31.911415+0000","last_undegraded":"2024-01-27T02:38:31.911415+0000","last_fullsized":"2024-01-27T02:38:31.911415+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:14:24.855281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.921982+0000","last_change":"2024-01-27T02:38:31.921936+0000","last_active":"2024-01-27T02:38:31.921982+0000","last_peered":"2024-01-27T02:38:31.921982+0000","last_clean":"2024-01-27T02:38:31.921982+0000","last_became_active":"2024-01-27T02:38:31.921776+0000","last_became_peered":"2024-01-27T02:38:31.921776+0000","last_unstale":"2024-01-27T02:38:31.921982+0000","last_undegraded":"2024-01-27T02:38:31.921982+0000","last_fullsized":"2024-01-27T02:38:31.921982+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:47:41.063180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.921233+0000","last_change":"2024-01-27T02:38:30.903329+0000","last_active":"2024-01-27T02:38:31.921233+0000","last_peered":"2024-01-27T02:38:31.921233+0000","last_clean":"2024-01-27T02:38:31.921233+0000","last_became_active":"2024-01-27T02:38:30.903084+0000","last_became_peered":"2024-01-27T02:38:30.903084+0000","last_unstale":"2024-01-27T02:38:31.921233+0000","last_undegraded":"2024-01-27T02:38:31.921233+0000","last_fullsized":"2024-01-27T02:38:31.921233+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:11:33.257548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.919275+0000","last_change":"2024-01-27T02:38:31.919209+0000","last_active":"2024-01-27T02:38:31.919275+0000","last_peered":"2024-01-27T02:38:31.919275+0000","last_clean":"2024-01-27T02:38:31.919275+0000","last_became_active":"2024-01-27T02:38:31.918761+0000","last_became_peered":"2024-01-27T02:38:31.918761+0000","last_unstale":"2024-01-27T02:38:31.919275+0000","last_undegraded":"2024-01-27T02:38:31.919275+0000","last_fullsized":"2024-01-27T02:38:31.919275+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:18.354062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.913366+0000","last_change":"2024-01-27T02:38:30.904630+0000","last_active":"2024-01-27T02:38:31.913366+0000","last_peered":"2024-01-27T02:38:31.913366+0000","last_clean":"2024-01-27T02:38:31.913366+0000","last_became_active":"2024-01-27T02:38:30.904406+0000","last_became_peered":"2024-01-27T02:38:30.904406+0000","last_unstale":"2024-01-27T02:38:31.913366+0000","last_undegraded":"2024-01-27T02:38:31.913366+0000","last_fullsized":"2024-01-27T02:38:31.913366+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:22:44.330464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.929159+0000","last_change":"2024-01-27T02:38:31.929087+0000","last_active":"2024-01-27T02:38:31.929159+0000","last_peered":"2024-01-27T02:38:31.929159+0000","last_clean":"2024-01-27T02:38:31.929159+0000","last_became_active":"2024-01-27T02:38:31.928717+0000","last_became_peered":"2024-01-27T02:38:31.928717+0000","last_unstale":"2024-01-27T02:38:31.929159+0000","last_undegraded":"2024-01-27T02:38:31.929159+0000","last_fullsized":"2024-01-27T02:38:31.929159+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:23:46.870039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.908261+0000","last_change":"2024-01-27T02:38:30.893083+0000","last_active":"2024-01-27T02:38:31.908261+0000","last_peered":"2024-01-27T02:38:31.908261+0000","last_clean":"2024-01-27T02:38:31.908261+0000","last_became_active":"2024-01-27T02:38:30.892801+0000","last_became_peered":"2024-01-27T02:38:30.892801+0000","last_unstale":"2024-01-27T02:38:31.908261+0000","last_undegraded":"2024-01-27T02:38:31.908261+0000","last_fullsized":"2024-01-27T02:38:31.908261+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:51:23.345374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.912173+0000","last_change":"2024-01-27T02:38:31.912087+0000","last_active":"2024-01-27T02:38:31.912173+0000","last_peered":"2024-01-27T02:38:31.912173+0000","last_clean":"2024-01-27T02:38:31.912173+0000","last_became_active":"2024-01-27T02:38:31.911806+0000","last_became_peered":"2024-01-27T02:38:31.911806+0000","last_unstale":"2024-01-27T02:38:31.912173+0000","last_undegraded":"2024-01-27T02:38:31.912173+0000","last_fullsized":"2024-01-27T02:38:31.912173+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:34:26.079082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"131'3","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.941757+0000","last_change":"2024-01-27T02:38:30.905086+0000","last_active":"2024-01-27T02:38:31.941757+0000","last_peered":"2024-01-27T02:38:31.941757+0000","last_clean":"2024-01-27T02:38:31.941757+0000","last_became_active":"2024-01-27T02:38:30.904853+0000","last_became_peered":"2024-01-27T02:38:30.904853+0000","last_unstale":"2024-01-27T02:38:31.941757+0000","last_undegraded":"2024-01-27T02:38:31.941757+0000","last_fullsized":"2024-01-27T02:38:31.941757+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:54:14.886540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.917312+0000","last_change":"2024-01-27T02:38:31.917239+0000","last_active":"2024-01-27T02:38:31.917312+0000","last_peered":"2024-01-27T02:38:31.917312+0000","last_clean":"2024-01-27T02:38:31.917312+0000","last_became_active":"2024-01-27T02:38:31.916971+0000","last_became_peered":"2024-01-27T02:38:31.916971+0000","last_unstale":"2024-01-27T02:38:31.917312+0000","last_undegraded":"2024-01-27T02:38:31.917312+0000","last_fullsized":"2024-01-27T02:38:31.917312+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:53:05.325595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"131'6","reported_seq":16,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.940272+0000","last_change":"2024-01-27T02:38:30.905975+0000","last_active":"2024-01-27T02:38:31.940272+0000","last_peered":"2024-01-27T02:38:31.940272+0000","last_clean":"2024-01-27T02:38:31.940272+0000","last_became_active":"2024-01-27T02:38:30.904239+0000","last_became_peered":"2024-01-27T02:38:30.904239+0000","last_unstale":"2024-01-27T02:38:31.940272+0000","last_undegraded":"2024-01-27T02:38:31.940272+0000","last_fullsized":"2024-01-27T02:38:31.940272+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:05:22.538039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.917006+0000","last_change":"2024-01-27T02:38:31.916918+0000","last_active":"2024-01-27T02:38:31.917006+0000","last_peered":"2024-01-27T02:38:31.917006+0000","last_clean":"2024-01-27T02:38:31.917006+0000","last_became_active":"2024-01-27T02:38:31.915711+0000","last_became_peered":"2024-01-27T02:38:31.915711+0000","last_unstale":"2024-01-27T02:38:31.917006+0000","last_undegraded":"2024-01-27T02:38:31.917006+0000","last_fullsized":"2024-01-27T02:38:31.917006+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:47:43.751635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.938505+0000","last_change":"2024-01-27T02:38:30.904960+0000","last_active":"2024-01-27T02:38:31.938505+0000","last_peered":"2024-01-27T02:38:31.938505+0000","last_clean":"2024-01-27T02:38:31.938505+0000","last_became_active":"2024-01-27T02:38:30.904479+0000","last_became_peered":"2024-01-27T02:38:30.904479+0000","last_unstale":"2024-01-27T02:38:31.938505+0000","last_undegraded":"2024-01-27T02:38:31.938505+0000","last_fullsized":"2024-01-27T02:38:31.938505+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:20:17.775604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.919480+0000","last_change":"2024-01-27T02:38:31.919224+0000","last_active":"2024-01-27T02:38:31.919480+0000","last_peered":"2024-01-27T02:38:31.919480+0000","last_clean":"2024-01-27T02:38:31.919480+0000","last_became_active":"2024-01-27T02:38:31.918967+0000","last_became_peered":"2024-01-27T02:38:31.918967+0000","last_unstale":"2024-01-27T02:38:31.919480+0000","last_undegraded":"2024-01-27T02:38:31.919480+0000","last_fullsized":"2024-01-27T02:38:31.919480+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:00:23.173837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.918372+0000","last_change":"2024-01-27T02:38:30.901125+0000","last_active":"2024-01-27T02:38:31.918372+0000","last_peered":"2024-01-27T02:38:31.918372+0000","last_clean":"2024-01-27T02:38:31.918372+0000","last_became_active":"2024-01-27T02:38:30.900811+0000","last_became_peered":"2024-01-27T02:38:30.900811+0000","last_unstale":"2024-01-27T02:38:31.918372+0000","last_undegraded":"2024-01-27T02:38:31.918372+0000","last_fullsized":"2024-01-27T02:38:31.918372+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:18:04.098263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.913229+0000","last_change":"2024-01-27T02:38:31.913151+0000","last_active":"2024-01-27T02:38:31.913229+0000","last_peered":"2024-01-27T02:38:31.913229+0000","last_clean":"2024-01-27T02:38:31.913229+0000","last_became_active":"2024-01-27T02:38:31.912872+0000","last_became_peered":"2024-01-27T02:38:31.912872+0000","last_unstale":"2024-01-27T02:38:31.913229+0000","last_undegraded":"2024-01-27T02:38:31.913229+0000","last_fullsized":"2024-01-27T02:38:31.913229+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:57:59.292913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"131'8","reported_seq":18,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.940973+0000","last_change":"2024-01-27T02:38:30.904062+0000","last_active":"2024-01-27T02:38:31.940973+0000","last_peered":"2024-01-27T02:38:31.940973+0000","last_clean":"2024-01-27T02:38:31.940973+0000","last_became_active":"2024-01-27T02:38:30.903827+0000","last_became_peered":"2024-01-27T02:38:30.903827+0000","last_unstale":"2024-01-27T02:38:31.940973+0000","last_undegraded":"2024-01-27T02:38:31.940973+0000","last_fullsized":"2024-01-27T02:38:31.940973+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:22:30.041364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.927351+0000","last_change":"2024-01-27T02:38:31.927269+0000","last_active":"2024-01-27T02:38:31.927351+0000","last_peered":"2024-01-27T02:38:31.927351+0000","last_clean":"2024-01-27T02:38:31.927351+0000","last_became_active":"2024-01-27T02:38:31.926727+0000","last_became_peered":"2024-01-27T02:38:31.926727+0000","last_unstale":"2024-01-27T02:38:31.927351+0000","last_undegraded":"2024-01-27T02:38:31.927351+0000","last_fullsized":"2024-01-27T02:38:31.927351+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:17:17.633072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.938198+0000","last_change":"2024-01-27T02:38:30.901317+0000","last_active":"2024-01-27T02:38:31.938198+0000","last_peered":"2024-01-27T02:38:31.938198+0000","last_clean":"2024-01-27T02:38:31.938198+0000","last_became_active":"2024-01-27T02:38:30.900905+0000","last_became_peered":"2024-01-27T02:38:30.900905+0000","last_unstale":"2024-01-27T02:38:31.938198+0000","last_undegraded":"2024-01-27T02:38:31.938198+0000","last_fullsized":"2024-01-27T02:38:31.938198+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:23:45.866513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.930083+0000","last_change":"2024-01-27T02:38:31.930009+0000","last_active":"2024-01-27T02:38:31.930083+0000","last_peered":"2024-01-27T02:38:31.930083+0000","last_clean":"2024-01-27T02:38:31.930083+0000","last_became_active":"2024-01-27T02:38:31.929718+0000","last_became_peered":"2024-01-27T02:38:31.929718+0000","last_unstale":"2024-01-27T02:38:31.930083+0000","last_undegraded":"2024-01-27T02:38:31.930083+0000","last_fullsized":"2024-01-27T02:38:31.930083+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:56:12.566293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.937933+0000","last_change":"2024-01-27T02:38:30.901405+0000","last_active":"2024-01-27T02:38:31.937933+0000","last_peered":"2024-01-27T02:38:31.937933+0000","last_clean":"2024-01-27T02:38:31.937933+0000","last_became_active":"2024-01-27T02:38:30.901168+0000","last_became_peered":"2024-01-27T02:38:30.901168+0000","last_unstale":"2024-01-27T02:38:31.937933+0000","last_undegraded":"2024-01-27T02:38:31.937933+0000","last_fullsized":"2024-01-27T02:38:31.937933+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:52:03.557302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.928377+0000","last_change":"2024-01-27T02:38:31.928314+0000","last_active":"2024-01-27T02:38:31.928377+0000","last_peered":"2024-01-27T02:38:31.928377+0000","last_clean":"2024-01-27T02:38:31.928377+0000","last_became_active":"2024-01-27T02:38:31.928109+0000","last_became_peered":"2024-01-27T02:38:31.928109+0000","last_unstale":"2024-01-27T02:38:31.928377+0000","last_undegraded":"2024-01-27T02:38:31.928377+0000","last_fullsized":"2024-01-27T02:38:31.928377+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:40:17.611985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"131'1","reported_seq":11,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.935106+0000","last_change":"2024-01-27T02:38:30.894794+0000","last_active":"2024-01-27T02:38:31.935106+0000","last_peered":"2024-01-27T02:38:31.935106+0000","last_clean":"2024-01-27T02:38:31.935106+0000","last_became_active":"2024-01-27T02:38:30.894519+0000","last_became_peered":"2024-01-27T02:38:30.894519+0000","last_unstale":"2024-01-27T02:38:31.935106+0000","last_undegraded":"2024-01-27T02:38:31.935106+0000","last_fullsized":"2024-01-27T02:38:31.935106+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:39:25.843705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.921065+0000","last_change":"2024-01-27T02:38:31.920963+0000","last_active":"2024-01-27T02:38:31.921065+0000","last_peered":"2024-01-27T02:38:31.921065+0000","last_clean":"2024-01-27T02:38:31.921065+0000","last_became_active":"2024-01-27T02:38:31.920276+0000","last_became_peered":"2024-01-27T02:38:31.920276+0000","last_unstale":"2024-01-27T02:38:31.921065+0000","last_undegraded":"2024-01-27T02:38:31.921065+0000","last_fullsized":"2024-01-27T02:38:31.921065+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:47:43.751635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.915796+0000","last_change":"2024-01-27T02:38:30.901197+0000","last_active":"2024-01-27T02:38:31.915796+0000","last_peered":"2024-01-27T02:38:31.915796+0000","last_clean":"2024-01-27T02:38:31.915796+0000","last_became_active":"2024-01-27T02:38:30.900235+0000","last_became_peered":"2024-01-27T02:38:30.900235+0000","last_unstale":"2024-01-27T02:38:31.915796+0000","last_undegraded":"2024-01-27T02:38:31.915796+0000","last_fullsized":"2024-01-27T02:38:31.915796+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:07:32.419335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.926876+0000","last_change":"2024-01-27T02:38:31.926758+0000","last_active":"2024-01-27T02:38:31.926876+0000","last_peered":"2024-01-27T02:38:31.926876+0000","last_clean":"2024-01-27T02:38:31.926876+0000","last_became_active":"2024-01-27T02:38:31.926434+0000","last_became_peered":"2024-01-27T02:38:31.926434+0000","last_unstale":"2024-01-27T02:38:31.926876+0000","last_undegraded":"2024-01-27T02:38:31.926876+0000","last_fullsized":"2024-01-27T02:38:31.926876+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:02:03.382943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.937501+0000","last_change":"2024-01-27T02:38:30.904351+0000","last_active":"2024-01-27T02:38:31.937501+0000","last_peered":"2024-01-27T02:38:31.937501+0000","last_clean":"2024-01-27T02:38:31.937501+0000","last_became_active":"2024-01-27T02:38:30.904118+0000","last_became_peered":"2024-01-27T02:38:30.904118+0000","last_unstale":"2024-01-27T02:38:31.937501+0000","last_undegraded":"2024-01-27T02:38:31.937501+0000","last_fullsized":"2024-01-27T02:38:31.937501+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:53:19.168298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.927349+0000","last_change":"2024-01-27T02:38:31.927097+0000","last_active":"2024-01-27T02:38:31.927349+0000","last_peered":"2024-01-27T02:38:31.927349+0000","last_clean":"2024-01-27T02:38:31.927349+0000","last_became_active":"2024-01-27T02:38:31.926471+0000","last_became_peered":"2024-01-27T02:38:31.926471+0000","last_unstale":"2024-01-27T02:38:31.927349+0000","last_undegraded":"2024-01-27T02:38:31.927349+0000","last_fullsized":"2024-01-27T02:38:31.927349+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:33:14.196336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.913953+0000","last_change":"2024-01-27T02:38:30.904812+0000","last_active":"2024-01-27T02:38:31.913953+0000","last_peered":"2024-01-27T02:38:31.913953+0000","last_clean":"2024-01-27T02:38:31.913953+0000","last_became_active":"2024-01-27T02:38:30.904483+0000","last_became_peered":"2024-01-27T02:38:30.904483+0000","last_unstale":"2024-01-27T02:38:31.913953+0000","last_undegraded":"2024-01-27T02:38:31.913953+0000","last_fullsized":"2024-01-27T02:38:31.913953+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:45:43.930079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.926114+0000","last_change":"2024-01-27T02:38:31.926049+0000","last_active":"2024-01-27T02:38:31.926114+0000","last_peered":"2024-01-27T02:38:31.926114+0000","last_clean":"2024-01-27T02:38:31.926114+0000","last_became_active":"2024-01-27T02:38:31.925808+0000","last_became_peered":"2024-01-27T02:38:31.925808+0000","last_unstale":"2024-01-27T02:38:31.926114+0000","last_undegraded":"2024-01-27T02:38:31.926114+0000","last_fullsized":"2024-01-27T02:38:31.926114+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:57:59.292913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.915740+0000","last_change":"2024-01-27T02:38:30.904780+0000","last_active":"2024-01-27T02:38:31.915740+0000","last_peered":"2024-01-27T02:38:31.915740+0000","last_clean":"2024-01-27T02:38:31.915740+0000","last_became_active":"2024-01-27T02:38:30.904179+0000","last_became_peered":"2024-01-27T02:38:30.904179+0000","last_unstale":"2024-01-27T02:38:31.915740+0000","last_undegraded":"2024-01-27T02:38:31.915740+0000","last_fullsized":"2024-01-27T02:38:31.915740+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:14:13.119229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.925284+0000","last_change":"2024-01-27T02:38:31.925213+0000","last_active":"2024-01-27T02:38:31.925284+0000","last_peered":"2024-01-27T02:38:31.925284+0000","last_clean":"2024-01-27T02:38:31.925284+0000","last_became_active":"2024-01-27T02:38:31.924951+0000","last_became_peered":"2024-01-27T02:38:31.924951+0000","last_unstale":"2024-01-27T02:38:31.925284+0000","last_undegraded":"2024-01-27T02:38:31.925284+0000","last_fullsized":"2024-01-27T02:38:31.925284+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:11:36.249589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.918380+0000","last_change":"2024-01-27T02:38:30.899295+0000","last_active":"2024-01-27T02:38:31.918380+0000","last_peered":"2024-01-27T02:38:31.918380+0000","last_clean":"2024-01-27T02:38:31.918380+0000","last_became_active":"2024-01-27T02:38:30.899015+0000","last_became_peered":"2024-01-27T02:38:30.899015+0000","last_unstale":"2024-01-27T02:38:31.918380+0000","last_undegraded":"2024-01-27T02:38:31.918380+0000","last_fullsized":"2024-01-27T02:38:31.918380+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:34:25.075555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.929138+0000","last_change":"2024-01-27T02:38:31.929089+0000","last_active":"2024-01-27T02:38:31.929138+0000","last_peered":"2024-01-27T02:38:31.929138+0000","last_clean":"2024-01-27T02:38:31.929138+0000","last_became_active":"2024-01-27T02:38:31.928801+0000","last_became_peered":"2024-01-27T02:38:31.928801+0000","last_unstale":"2024-01-27T02:38:31.929138+0000","last_undegraded":"2024-01-27T02:38:31.929138+0000","last_fullsized":"2024-01-27T02:38:31.929138+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:30:10.336341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"131'2","reported_seq":12,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.938107+0000","last_change":"2024-01-27T02:38:30.902450+0000","last_active":"2024-01-27T02:38:31.938107+0000","last_peered":"2024-01-27T02:38:31.938107+0000","last_clean":"2024-01-27T02:38:31.938107+0000","last_became_active":"2024-01-27T02:38:30.902130+0000","last_became_peered":"2024-01-27T02:38:30.902130+0000","last_unstale":"2024-01-27T02:38:31.938107+0000","last_undegraded":"2024-01-27T02:38:31.938107+0000","last_fullsized":"2024-01-27T02:38:31.938107+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:39:25.843705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.928531+0000","last_change":"2024-01-27T02:38:31.928476+0000","last_active":"2024-01-27T02:38:31.928531+0000","last_peered":"2024-01-27T02:38:31.928531+0000","last_clean":"2024-01-27T02:38:31.928531+0000","last_became_active":"2024-01-27T02:38:31.927989+0000","last_became_peered":"2024-01-27T02:38:31.927989+0000","last_unstale":"2024-01-27T02:38:31.928531+0000","last_undegraded":"2024-01-27T02:38:31.928531+0000","last_fullsized":"2024-01-27T02:38:31.928531+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:12:26.885080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"132'1","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-01-27T02:38:32.909749+0000","last_change":"2024-01-27T02:38:30.901304+0000","last_active":"2024-01-27T02:38:32.909749+0000","last_peered":"2024-01-27T02:38:32.909749+0000","last_clean":"2024-01-27T02:38:32.909749+0000","last_became_active":"2024-01-27T02:38:30.901065+0000","last_became_peered":"2024-01-27T02:38:30.901065+0000","last_unstale":"2024-01-27T02:38:32.909749+0000","last_undegraded":"2024-01-27T02:38:32.909749+0000","last_fullsized":"2024-01-27T02:38:32.909749+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:16:12.621310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.925601+0000","last_change":"2024-01-27T02:38:31.925498+0000","last_active":"2024-01-27T02:38:31.925601+0000","last_peered":"2024-01-27T02:38:31.925601+0000","last_clean":"2024-01-27T02:38:31.925601+0000","last_became_active":"2024-01-27T02:38:31.924867+0000","last_became_peered":"2024-01-27T02:38:31.924867+0000","last_unstale":"2024-01-27T02:38:31.925601+0000","last_undegraded":"2024-01-27T02:38:31.925601+0000","last_fullsized":"2024-01-27T02:38:31.925601+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:14:28.293390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.919029+0000","last_change":"2024-01-27T02:38:30.901161+0000","last_active":"2024-01-27T02:38:31.919029+0000","last_peered":"2024-01-27T02:38:31.919029+0000","last_clean":"2024-01-27T02:38:31.919029+0000","last_became_active":"2024-01-27T02:38:30.899610+0000","last_became_peered":"2024-01-27T02:38:30.899610+0000","last_unstale":"2024-01-27T02:38:31.919029+0000","last_undegraded":"2024-01-27T02:38:31.919029+0000","last_fullsized":"2024-01-27T02:38:31.919029+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:30:09.332815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.929042+0000","last_change":"2024-01-27T02:38:31.928975+0000","last_active":"2024-01-27T02:38:31.929042+0000","last_peered":"2024-01-27T02:38:31.929042+0000","last_clean":"2024-01-27T02:38:31.929042+0000","last_became_active":"2024-01-27T02:38:31.928745+0000","last_became_peered":"2024-01-27T02:38:31.928745+0000","last_unstale":"2024-01-27T02:38:31.929042+0000","last_undegraded":"2024-01-27T02:38:31.929042+0000","last_fullsized":"2024-01-27T02:38:31.929042+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:11:34.261074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.908273+0000","last_change":"2024-01-27T02:38:30.904612+0000","last_active":"2024-01-27T02:38:31.908273+0000","last_peered":"2024-01-27T02:38:31.908273+0000","last_clean":"2024-01-27T02:38:31.908273+0000","last_became_active":"2024-01-27T02:38:30.904299+0000","last_became_peered":"2024-01-27T02:38:30.904299+0000","last_unstale":"2024-01-27T02:38:31.908273+0000","last_undegraded":"2024-01-27T02:38:31.908273+0000","last_fullsized":"2024-01-27T02:38:31.908273+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:16:12.621310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.916639+0000","last_change":"2024-01-27T02:38:31.916561+0000","last_active":"2024-01-27T02:38:31.916639+0000","last_peered":"2024-01-27T02:38:31.916639+0000","last_clean":"2024-01-27T02:38:31.916639+0000","last_became_active":"2024-01-27T02:38:31.916319+0000","last_became_peered":"2024-01-27T02:38:31.916319+0000","last_unstale":"2024-01-27T02:38:31.916639+0000","last_undegraded":"2024-01-27T02:38:31.916639+0000","last_fullsized":"2024-01-27T02:38:31.916639+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:05:23.541565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.913921+0000","last_change":"2024-01-27T02:38:30.899705+0000","last_active":"2024-01-27T02:38:31.913921+0000","last_peered":"2024-01-27T02:38:31.913921+0000","last_clean":"2024-01-27T02:38:31.913921+0000","last_became_active":"2024-01-27T02:38:30.899453+0000","last_became_peered":"2024-01-27T02:38:30.899453+0000","last_unstale":"2024-01-27T02:38:31.913921+0000","last_undegraded":"2024-01-27T02:38:31.913921+0000","last_fullsized":"2024-01-27T02:38:31.913921+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:02:35.435543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.916730+0000","last_change":"2024-01-27T02:38:31.916672+0000","last_active":"2024-01-27T02:38:31.916730+0000","last_peered":"2024-01-27T02:38:31.916730+0000","last_clean":"2024-01-27T02:38:31.916730+0000","last_became_active":"2024-01-27T02:38:31.916440+0000","last_became_peered":"2024-01-27T02:38:31.916440+0000","last_unstale":"2024-01-27T02:38:31.916730+0000","last_undegraded":"2024-01-27T02:38:31.916730+0000","last_fullsized":"2024-01-27T02:38:31.916730+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:47:43.751635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.912323+0000","last_change":"2024-01-27T02:38:30.902907+0000","last_active":"2024-01-27T02:38:31.912323+0000","last_peered":"2024-01-27T02:38:31.912323+0000","last_clean":"2024-01-27T02:38:31.912323+0000","last_became_active":"2024-01-27T02:38:30.902691+0000","last_became_peered":"2024-01-27T02:38:30.902691+0000","last_unstale":"2024-01-27T02:38:31.912323+0000","last_undegraded":"2024-01-27T02:38:31.912323+0000","last_fullsized":"2024-01-27T02:38:31.912323+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:25:13.514723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.921474+0000","last_change":"2024-01-27T02:38:31.921408+0000","last_active":"2024-01-27T02:38:31.921474+0000","last_peered":"2024-01-27T02:38:31.921474+0000","last_clean":"2024-01-27T02:38:31.921474+0000","last_became_active":"2024-01-27T02:38:31.920845+0000","last_became_peered":"2024-01-27T02:38:31.920845+0000","last_unstale":"2024-01-27T02:38:31.921474+0000","last_undegraded":"2024-01-27T02:38:31.921474+0000","last_fullsized":"2024-01-27T02:38:31.921474+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:34:26.079082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.914181+0000","last_change":"2024-01-27T02:38:30.906486+0000","last_active":"2024-01-27T02:38:31.914181+0000","last_peered":"2024-01-27T02:38:31.914181+0000","last_clean":"2024-01-27T02:38:31.914181+0000","last_became_active":"2024-01-27T02:38:30.906233+0000","last_became_peered":"2024-01-27T02:38:30.906233+0000","last_unstale":"2024-01-27T02:38:31.914181+0000","last_undegraded":"2024-01-27T02:38:31.914181+0000","last_fullsized":"2024-01-27T02:38:31.914181+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:27:08.109911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.927271+0000","last_change":"2024-01-27T02:38:31.926086+0000","last_active":"2024-01-27T02:38:31.927271+0000","last_peered":"2024-01-27T02:38:31.927271+0000","last_clean":"2024-01-27T02:38:31.927271+0000","last_became_active":"2024-01-27T02:38:31.924140+0000","last_became_peered":"2024-01-27T02:38:31.924140+0000","last_unstale":"2024-01-27T02:38:31.927271+0000","last_undegraded":"2024-01-27T02:38:31.927271+0000","last_fullsized":"2024-01-27T02:38:31.927271+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:16:55.806750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.919136+0000","last_change":"2024-01-27T02:38:30.897972+0000","last_active":"2024-01-27T02:38:31.919136+0000","last_peered":"2024-01-27T02:38:31.919136+0000","last_clean":"2024-01-27T02:38:31.919136+0000","last_became_active":"2024-01-27T02:38:30.897716+0000","last_became_peered":"2024-01-27T02:38:30.897716+0000","last_unstale":"2024-01-27T02:38:31.919136+0000","last_undegraded":"2024-01-27T02:38:31.919136+0000","last_fullsized":"2024-01-27T02:38:31.919136+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:15:07.698973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.924090+0000","last_change":"2024-01-27T02:38:31.923808+0000","last_active":"2024-01-27T02:38:31.924090+0000","last_peered":"2024-01-27T02:38:31.924090+0000","last_clean":"2024-01-27T02:38:31.924090+0000","last_became_active":"2024-01-27T02:38:31.922834+0000","last_became_peered":"2024-01-27T02:38:31.922834+0000","last_unstale":"2024-01-27T02:38:31.924090+0000","last_undegraded":"2024-01-27T02:38:31.924090+0000","last_fullsized":"2024-01-27T02:38:31.924090+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:30.885345+0000","last_clean_scrub_stamp":"2024-01-27T02:38:30.885345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:02:03.382943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.915586+0000","last_change":"2024-01-27T02:38:30.901349+0000","last_active":"2024-01-27T02:38:31.915586+0000","last_peered":"2024-01-27T02:38:31.915586+0000","last_clean":"2024-01-27T02:38:31.915586+0000","last_became_active":"2024-01-27T02:38:30.900973+0000","last_became_peered":"2024-01-27T02:38:30.900973+0000","last_unstale":"2024-01-27T02:38:31.915586+0000","last_undegraded":"2024-01-27T02:38:31.915586+0000","last_fullsized":"2024-01-27T02:38:31.915586+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:38:29.881818+0000","last_clean_scrub_stamp":"2024-01-27T02:38:29.881818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:55:50.241360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":259,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-01-27T02:38:31.916927+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:38:31.916927+0000","last_peered":"2024-01-27T02:38:31.916927+0000","last_clean":"2024-01-27T02:38:31.916927+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:38:31.916927+0000","last_undegraded":"2024-01-27T02:38:31.916927+0000","last_fullsized":"2024-01-27T02:38:31.916927+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574930,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1284,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194028,"statfs":{"total":200000000,"available":198685095,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77400000000000002,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.376,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70299999999999996,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.65500000000000003}]},{"osd":1,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.40899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.71999999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.501},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.42799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69999999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.54700000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.68200000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71399999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.76200000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.73099999999999998,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.48199999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.71799999999999997,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.58799999999999997}]},{"osd":4,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.47599999999999998,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.51100000000000001},"min":{"1min":0.33700000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.51900000000000002}]},{"osd":5,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51300000000000001,"15min":0.53400000000000003},"min":{"1min":0.35999999999999999,"5min":0.32100000000000001,"15min":0.307},"max":{"1min":0.68500000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54800000000000004,"15min":0.56299999999999994},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75800000000000001,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.78400000000000003}]},{"osd":6,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57199999999999995,"15min":0.59399999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.78800000000000003,"5min":0.80300000000000005,"15min":0.81299999999999994},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56100000000000005,"15min":0.57499999999999996},"min":{"1min":0.42499999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.73099999999999998,"5min":0.81599999999999995,"15min":0.90800000000000003},"last":0.67700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574930,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":707,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194605,"statfs":{"total":200000000,"available":199275981,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.64900000000000002,"15min":0.54200000000000004},"min":{"1min":0.442,"5min":0.36499999999999999,"15min":0.29999999999999999},"max":{"1min":4.125,"5min":4.125,"15min":4.125},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60599999999999998,"15min":0.53600000000000003},"min":{"1min":0.45100000000000001,"5min":0.35699999999999998,"15min":0.32700000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.628}]},{"osd":1,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.68400000000000005,"15min":0.55200000000000005},"min":{"1min":0.433,"5min":0.41999999999999998,"15min":0.34100000000000003},"max":{"1min":8.2699999999999996,"5min":8.2699999999999996,"15min":8.2699999999999996},"last":0.59899999999999998},{"interface":"front","average":{"1min":2.0409999999999999,"5min":0.90800000000000003,"15min":0.65600000000000003},"min":{"1min":0.46999999999999997,"5min":0.379,"15min":0.33800000000000002},"max":{"1min":26.411999999999999,"5min":26.411999999999999,"15min":26.411999999999999},"last":0.57299999999999995}]},{"osd":2,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":0.77700000000000002,"15min":0.60899999999999999},"min":{"1min":0.45100000000000001,"5min":0.41199999999999998,"15min":0.39600000000000002},"max":{"1min":17.143999999999998,"5min":17.143999999999998,"15min":17.143999999999998},"last":0.504},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.64900000000000002,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":0.69099999999999995}]},{"osd":3,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.63300000000000001,"15min":0.56799999999999995},"min":{"1min":0.45800000000000002,"5min":0.435,"15min":0.376},"max":{"1min":4.3689999999999998,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64100000000000001,"15min":0.58099999999999996},"min":{"1min":0.433,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.56200000000000006}]},{"osd":4,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.48699999999999999},"min":{"1min":0.33200000000000002,"5min":0.32000000000000001,"15min":0.30199999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51600000000000001,"15min":0.54900000000000004},"min":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.74299999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.60899999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.52100000000000002,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76100000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56000000000000005,"15min":0.59999999999999998},"min":{"1min":0.41099999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77000000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.65600000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:38:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.58999999999999997,"15min":0.61099999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79400000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55800000000000005,"15min":0.58499999999999996},"min":{"1min":0.40000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71499999999999997,"5min":1.038,"15min":1.038},"last":0.53900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":707,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194605,"statfs":{"total":200000000,"available":199275927,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.39700000000000002,"15min":0.377},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25800000000000001},"max":{"1min":0.50600000000000001,"5min":0.873,"15min":0.873},"last":0.496},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.41899999999999998,"15min":0.39900000000000002},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.53100000000000003,"5min":1.121,"15min":1.121},"last":0.38200000000000001}]},{"osd":2,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.40999999999999998,"15min":0.36899999999999999},"min":{"1min":0.27200000000000002,"5min":0.26600000000000001,"15min":0.23999999999999999},"max":{"1min":0.48599999999999999,"5min":1.25,"15min":1.25},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.45600000000000002,"15min":0.432},"min":{"1min":0.32000000000000001,"5min":0.30199999999999999,"15min":0.29899999999999999},"max":{"1min":0.58599999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.40600000000000003}]},{"osd":3,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.49399999999999999,"15min":0.46999999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.32100000000000001},"max":{"1min":0.66800000000000004,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.46999999999999997,"15min":0.45600000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.58199999999999996,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.44700000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.66300000000000003,"15min":0.63500000000000001},"min":{"1min":0.501,"5min":0.373,"15min":0.373},"max":{"1min":0.81599999999999995,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66200000000000003,"15min":0.63800000000000001},"min":{"1min":0.497,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.82599999999999996,"5min":1.026,"15min":1.026},"last":0.65200000000000002}]},{"osd":5,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.68899999999999995,"15min":0.63900000000000001},"min":{"1min":0.51000000000000001,"5min":0.46200000000000002,"15min":0.44},"max":{"1min":0.80900000000000005,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69099999999999995,"15min":0.67600000000000005},"min":{"1min":0.53000000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83799999999999997,"5min":3.181,"15min":3.181},"last":0.76900000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71199999999999997,"15min":0.69699999999999995},"min":{"1min":0.51400000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.875,"5min":1.024,"15min":1.024},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71499999999999997,"15min":0.68000000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.85699999999999998,"5min":1.869,"15min":1.869},"last":0.747}]},{"osd":7,"last update":"Sat Jan 27 02:38:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.72699999999999998,"15min":0.68700000000000006},"min":{"1min":0.55800000000000005,"5min":0.495,"15min":0.47299999999999998},"max":{"1min":0.82599999999999996,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.73099999999999998,"15min":0.69399999999999995},"min":{"1min":0.56000000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.86799999999999999,"5min":1.708,"15min":1.708},"last":0.81000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574930,"num_pgs":36,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1285,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194027,"statfs":{"total":200000000,"available":198684035,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.39000000000000001},"min":{"1min":0.253,"5min":0.24399999999999999,"15min":0.23799999999999999},"max":{"1min":0.52100000000000002,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.41999999999999998,"15min":0.41599999999999998},"min":{"1min":0.29599999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.64200000000000002,"5min":0.65200000000000002,"15min":0.68400000000000005},"last":0.34599999999999997}]},{"osd":2,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.437,"15min":0.42199999999999999},"min":{"1min":0.30499999999999999,"5min":0.28699999999999998,"15min":0.28199999999999997},"max":{"1min":0.60299999999999998,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.374},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.44900000000000001,"15min":0.435},"min":{"1min":0.32400000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.628,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.38900000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42799999999999999,"15min":0.43099999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58199999999999996,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48499999999999999,"15min":0.47499999999999998},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69399999999999995,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.45100000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65900000000000003,"15min":0.64100000000000001},"min":{"1min":0.49399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91700000000000004,"5min":0.997,"15min":0.997},"last":0.505},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.44700000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92100000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.61799999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67700000000000005,"15min":0.65700000000000003},"min":{"1min":0.496,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88600000000000001,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.68100000000000005},"min":{"1min":0.53100000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.82099999999999995,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.57099999999999995}]},{"osd":6,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.57299999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.80300000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69599999999999995,"15min":0.68000000000000005},"min":{"1min":0.51400000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.94299999999999995,"5min":1.04,"15min":1.04},"last":0.51600000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68899999999999995,"15min":0.69199999999999995},"min":{"1min":0.45900000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90800000000000003,"5min":1.014,"15min":1.014},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68700000000000006,"15min":0.69299999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.83899999999999997,"5min":0.95799999999999996,"15min":0.97299999999999998},"last":0.54500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194605,"statfs":{"total":200000000,"available":199276051,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.41399999999999998,"15min":0.40899999999999997},"min":{"1min":0.23599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.58099999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.45700000000000002,"15min":0.62},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.30499999999999999},"max":{"1min":0.64700000000000002,"5min":0.72799999999999998,"15min":6.2729999999999997},"last":0.44400000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.42999999999999999,"15min":0.42299999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.71899999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.433,"15min":0.44500000000000001},"min":{"1min":0.32600000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.57999999999999996,"5min":0.65300000000000002,"15min":0.70099999999999996},"last":0.48799999999999999}]},{"osd":3,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42399999999999999,"15min":0.41599999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.27800000000000002},"max":{"1min":0.64700000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.48699999999999999,"15min":0.48499999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.627,"5min":0.76500000000000001,"15min":0.80800000000000005},"last":0.51000000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68799999999999994,"15min":0.65800000000000003},"min":{"1min":0.46800000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93999999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70599999999999996,"15min":0.79000000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.006,"5min":1.006,"15min":3.581},"last":0.63400000000000001}]},{"osd":5,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65300000000000002,"15min":0.73499999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":3.5369999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92200000000000004,"5min":0.96599999999999997,"15min":1.0529999999999999},"last":0.64900000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.45000000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.51300000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97499999999999998,"5min":1.0189999999999999,"15min":1.0209999999999999},"last":0.59899999999999998}]},{"osd":7,"last update":"Sat Jan 27 02:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.68300000000000005,"15min":0.67800000000000005},"min":{"1min":0.50800000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.95599999999999996,"5min":1.03,"15min":1.03},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.69799999999999995,"15min":0.68999999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.42699999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.72499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574930,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":708,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194603,"statfs":{"total":200000000,"available":199274445,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.45600000000000002,"15min":0.38200000000000001},"min":{"1min":0.28999999999999998,"5min":0.253,"15min":0.253},"max":{"1min":4.7380000000000004,"5min":4.7380000000000004,"15min":4.7380000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.41599999999999998,"15min":0.39800000000000002},"min":{"1min":0.27600000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.29399999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.47199999999999998,"15min":0.41199999999999998},"min":{"1min":0.27900000000000003,"5min":0.27400000000000002,"15min":0.22900000000000001},"max":{"1min":4.5739999999999998,"5min":4.5739999999999998,"15min":4.5739999999999998},"last":0.371},{"interface":"front","average":{"1min":0.752,"5min":0.52400000000000002,"15min":0.44700000000000001},"min":{"1min":0.311,"5min":0.30199999999999999,"15min":0.27900000000000003},"max":{"1min":5.0529999999999999,"5min":5.0529999999999999,"15min":5.0529999999999999},"last":0.34499999999999997}]},{"osd":2,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.48499999999999999,"15min":0.439},"min":{"1min":0.34799999999999998,"5min":0.33600000000000002,"15min":0.28100000000000003},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.55500000000000005,"15min":0.47399999999999998},"min":{"1min":0.375,"5min":0.35299999999999998,"15min":0.34599999999999997},"max":{"1min":5.1879999999999997,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":0.42799999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.69399999999999995,"15min":0.63600000000000001},"min":{"1min":0.42799999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0629999999999999,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66500000000000004,"15min":0.61399999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.68000000000000005}]},{"osd":5,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.67800000000000005,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.56699999999999995},{"interface":"front","average":{"1min":1.006,"5min":0.78500000000000003,"15min":0.70299999999999996},"min":{"1min":0.56200000000000006,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":4.7960000000000003,"5min":4.7960000000000003,"15min":4.7960000000000003},"last":0.69299999999999995}]},{"osd":6,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.73499999999999999,"15min":0.67800000000000005},"min":{"1min":0.58899999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":5.0069999999999997,"5min":5.0069999999999997,"15min":5.0069999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.68100000000000005,"15min":0.65400000000000003},"min":{"1min":0.504,"5min":0.435,"15min":0.435},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.70299999999999996}]},{"osd":7,"last update":"Sat Jan 27 02:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.74299999999999999,"15min":0.66300000000000003},"min":{"1min":0.57699999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.139,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.71299999999999997,"15min":0.67600000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.73899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194605,"statfs":{"total":200000000,"available":199276051,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.34599999999999997,"5min":0.33400000000000002,"15min":0.314},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.64600000000000002,"15min":0.60999999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":7.0670000000000002,"5min":7.0670000000000002,"15min":7.0670000000000002},"last":0.54900000000000004}]},{"osd":1,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.64700000000000002,"15min":0.60899999999999999},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":5.1529999999999996,"5min":5.1529999999999996,"15min":5.1529999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.626,"15min":0.60399999999999998},"min":{"1min":0.43099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.0419999999999998,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.46400000000000002}]},{"osd":2,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.64900000000000002,"15min":0.60999999999999999},"min":{"1min":0.46899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":5.0019999999999998,"5min":5.0019999999999998,"15min":5.0019999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.5}]},{"osd":3,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.64700000000000002,"15min":0.60099999999999998},"min":{"1min":0.42099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.7220000000000004,"5min":6.7220000000000004,"15min":6.7220000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.51000000000000001,"5min":0.44900000000000001,"15min":0.41399999999999998},"max":{"1min":6.5510000000000002,"5min":6.5510000000000002,"15min":6.5510000000000002},"last":0.53400000000000003}]},{"osd":5,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.56200000000000006,"15min":0.51400000000000001},"min":{"1min":0.38500000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":6.9269999999999996,"5min":6.9269999999999996,"15min":6.9269999999999996},"last":0.44400000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.55400000000000005,"15min":0.51700000000000002},"min":{"1min":0.40100000000000002,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":5.2229999999999999,"5min":5.2229999999999999,"15min":5.2229999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.52600000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:38:09 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.60799999999999998,"15min":0.57199999999999995},"min":{"1min":0.34799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":7.5339999999999998,"5min":7.5339999999999998,"15min":7.5339999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.63200000000000001,"15min":0.58999999999999997},"min":{"1min":0.44600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.9550000000000001,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":0.59599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":707,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194604,"statfs":{"total":200000000,"available":199275475,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56100000000000005,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.84199999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.443},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60599999999999998,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.34499999999999997,"15min":0.27600000000000002},"max":{"1min":0.78900000000000003,"5min":3.242,"15min":3.242},"last":0.58599999999999997}]},{"osd":1,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63,"15min":0.60999999999999999},"min":{"1min":0.47299999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80100000000000005,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.62,"15min":0.64700000000000002},"min":{"1min":0.46899999999999997,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77600000000000002,"5min":0.88500000000000001,"15min":0.95499999999999996},"last":0.55100000000000005}]},{"osd":2,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61699999999999999,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.372},"max":{"1min":0.80900000000000005,"5min":3.04,"15min":3.04},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59599999999999997,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74299999999999999,"5min":0.85899999999999999,"15min":0.89400000000000002},"last":0.53900000000000003}]},{"osd":3,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65000000000000002,"15min":0.67300000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82099999999999995,"5min":4.032,"15min":4.032},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.70599999999999996,"15min":0.67900000000000005},"min":{"1min":0.46000000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.78800000000000003,"5min":4.282,"15min":4.282},"last":0.56200000000000006}]},{"osd":4,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.371,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.747,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.376},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.499,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76300000000000001,"5min":0.79800000000000004,"15min":1.101},"last":0.49399999999999999}]},{"osd":6,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.502,"15min":0.55200000000000005},"min":{"1min":0.38600000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69199999999999995,"5min":0.73199999999999998,"15min":1.7969999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.58099999999999996},"min":{"1min":0.433,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83399999999999996,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.63600000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55100000000000005,"15min":0.56599999999999995},"min":{"1min":0.45000000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68899999999999995,"5min":0.91100000000000003,"15min":1.008},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.61499999999999999,"15min":0.65400000000000003},"min":{"1min":0.40699999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78400000000000003,"5min":2,"15min":2},"last":0.51000000000000001}]}]}],"pool_statfs":[]}} 2024-01-27T02:38:41.185 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-27T02:38:41.185 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-27T02:38:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.360+0000 7fc83e6be700 1 Processor -- start 2024-01-27T02:38:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.360+0000 7fc83e6be700 1 -- start start 2024-01-27T02:38:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.361+0000 7fc83e6be700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc838156b60 0x7fc8380ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.361+0000 7fc83e6be700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8380aeaa0 con 0x7fc838156f30 2024-01-27T02:38:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.361+0000 7fc83e6be700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8380ade90 con 0x7fc8380ad9f0 2024-01-27T02:38:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.361+0000 7fc83e6be700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8380ae170 con 0x7fc838156b60 2024-01-27T02:38:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.361+0000 7fc83cc5b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc838156f30 0x7fc83815a9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50634/0 (socket says 172.21.15.7:50634) 2024-01-27T02:38:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.361+0000 7fc83cc5b700 1 -- 172.21.15.7:0/3769148056 learned_addr learned my addr 172.21.15.7:0/3769148056 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.361+0000 7fc837fff700 1 --2- 172.21.15.7:0/3769148056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc838156b60 0x7fc8380ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.362+0000 7fc836ffd700 1 -- 172.21.15.7:0/3769148056 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 848881403 0 0) 0x7fc8380aeaa0 con 0x7fc838156f30 2024-01-27T02:38:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.362+0000 7fc836ffd700 1 -- 172.21.15.7:0/3769148056 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc820003570 con 0x7fc838156f30 2024-01-27T02:38:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.362+0000 7fc836ffd700 1 -- 172.21.15.7:0/3769148056 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443144705 0 0) 0x7fc8380ade90 con 0x7fc8380ad9f0 2024-01-27T02:38:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.362+0000 7fc836ffd700 1 -- 172.21.15.7:0/3769148056 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8380aeaa0 con 0x7fc8380ad9f0 2024-01-27T02:38:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.362+0000 7fc837fff700 1 -- 172.21.15.7:0/3769148056 >> v1:172.21.15.188:6789/0 conn(0x7fc8380ad9f0 legacy=0x7fc8380aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.362+0000 7fc837fff700 1 -- 172.21.15.7:0/3769148056 >> v1:172.21.15.7:6789/0 conn(0x7fc838156f30 legacy=0x7fc83815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.362+0000 7fc837fff700 1 -- 172.21.15.7:0/3769148056 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8380ae450 con 0x7fc838156b60 2024-01-27T02:38:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.362+0000 7fc837fff700 1 --2- 172.21.15.7:0/3769148056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc838156b60 0x7fc8380ad4b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc828009910 tx=0x7fc8280091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e4bea59fadf22ed server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.362+0000 7fc836ffd700 1 -- 172.21.15.7:0/3769148056 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc828011630 con 0x7fc838156b60 2024-01-27T02:38:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.362+0000 7fc836ffd700 1 -- 172.21.15.7:0/3769148056 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc828011790 con 0x7fc838156b60 2024-01-27T02:38:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.362+0000 7fc836ffd700 1 -- 172.21.15.7:0/3769148056 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc828011b30 con 0x7fc838156b60 2024-01-27T02:38:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.363+0000 7fc83e6be700 1 -- 172.21.15.7:0/3769148056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc838156b60 msgr2=0x7fc8380ad4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.363+0000 7fc83e6be700 1 --2- 172.21.15.7:0/3769148056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc838156b60 0x7fc8380ad4b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc828009910 tx=0x7fc8280091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.363+0000 7fc83e6be700 1 -- 172.21.15.7:0/3769148056 shutdown_connections 2024-01-27T02:38:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.363+0000 7fc83e6be700 1 --2- 172.21.15.7:0/3769148056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc838156b60 0x7fc8380ad4b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.363+0000 7fc83e6be700 1 -- 172.21.15.7:0/3769148056 >> 172.21.15.7:0/3769148056 conn(0x7fc838068480 msgr2=0x7fc8380cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.363+0000 7fc83e6be700 1 -- 172.21.15.7:0/3769148056 shutdown_connections 2024-01-27T02:38:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.363+0000 7fc83e6be700 1 -- 172.21.15.7:0/3769148056 wait complete. 2024-01-27T02:38:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.364+0000 7fc83e6be700 1 Processor -- start 2024-01-27T02:38:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.364+0000 7fc83e6be700 1 -- start start 2024-01-27T02:38:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.365+0000 7fc83e6be700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8380ad9f0 0x7fc8380a8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.365+0000 7fc83e6be700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc838156b60 0x7fc8381488f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.365+0000 7fc83e6be700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc838156f30 0x7fc83814aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.365+0000 7fc83e6be700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83814fb90 con 0x7fc8380ad9f0 2024-01-27T02:38:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.365+0000 7fc83e6be700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8380a9050 con 0x7fc838156b60 2024-01-27T02:38:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.365+0000 7fc83e6be700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc83814c460 con 0x7fc838156f30 2024-01-27T02:38:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.365+0000 7fc837fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8380ad9f0 0x7fc8380a8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.365+0000 7fc837fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8380ad9f0 0x7fc8380a8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58926/0 (socket says 172.21.15.7:58926) 2024-01-27T02:38:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.365+0000 7fc837fff700 1 -- 172.21.15.7:0/2549920539 learned_addr learned my addr 172.21.15.7:0/2549920539 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.365+0000 7fc83cc5b700 1 --2- 172.21.15.7:0/2549920539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc838156f30 0x7fc83814aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.365+0000 7fc8377fe700 1 --2- 172.21.15.7:0/2549920539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc838156b60 0x7fc8381488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.366+0000 7fc837fff700 1 -- 172.21.15.7:0/2549920539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc838156f30 msgr2=0x7fc83814aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.366+0000 7fc837fff700 1 --2- 172.21.15.7:0/2549920539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc838156f30 0x7fc83814aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.366+0000 7fc837fff700 1 -- 172.21.15.7:0/2549920539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc838156b60 msgr2=0x7fc8381488f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.366+0000 7fc837fff700 1 --2- 172.21.15.7:0/2549920539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc838156b60 0x7fc8381488f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.366+0000 7fc837fff700 1 -- 172.21.15.7:0/2549920539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc828009420 con 0x7fc8380ad9f0 2024-01-27T02:38:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.366+0000 7fc837fff700 1 --2- 172.21.15.7:0/2549920539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8380ad9f0 0x7fc8380a8b10 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc828009f00 tx=0x7fc828017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.366+0000 7fc81ffff700 1 -- 172.21.15.7:0/2549920539 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82801f070 con 0x7fc8380ad9f0 2024-01-27T02:38:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.366+0000 7fc81ffff700 1 -- 172.21.15.7:0/2549920539 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc828011900 con 0x7fc8380ad9f0 2024-01-27T02:38:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.366+0000 7fc81ffff700 1 -- 172.21.15.7:0/2549920539 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc828023410 con 0x7fc8380ad9f0 2024-01-27T02:38:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.366+0000 7fc83e6be700 1 -- 172.21.15.7:0/2549920539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8380a9280 con 0x7fc8380ad9f0 2024-01-27T02:38:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.366+0000 7fc83e6be700 1 -- 172.21.15.7:0/2549920539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc83814be90 con 0x7fc8380ad9f0 2024-01-27T02:38:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.367+0000 7fc83e6be700 1 -- 172.21.15.7:0/2549920539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc838072060 con 0x7fc8380ad9f0 2024-01-27T02:38:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.368+0000 7fc81ffff700 1 -- 172.21.15.7:0/2549920539 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc828023840 con 0x7fc8380ad9f0 2024-01-27T02:38:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.368+0000 7fc81ffff700 1 --2- 172.21.15.7:0/2549920539 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc824066f30 0x7fc8240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.368+0000 7fc81ffff700 1 -- 172.21.15.7:0/2549920539 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc828085740 con 0x7fc8380ad9f0 2024-01-27T02:38:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.368+0000 7fc8377fe700 1 --2- 172.21.15.7:0/2549920539 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc824066f30 0x7fc8240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.369+0000 7fc8377fe700 1 --2- 172.21.15.7:0/2549920539 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc824066f30 0x7fc8240693e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc8380a8a60 tx=0x7fc82c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.373+0000 7fc81ffff700 1 -- 172.21.15.7:0/2549920539 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc8280529b0 con 0x7fc8380ad9f0 2024-01-27T02:38:41.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.660+0000 7fc83e6be700 1 -- 172.21.15.7:0/2549920539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc8380ae730 con 0x7fc8380ad9f0 2024-01-27T02:38:41.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.663+0000 7fc81ffff700 1 -- 172.21.15.7:0/2549920539 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+0 (secure 0 0 0) 0x7fc8280150e0 con 0x7fc8380ad9f0 2024-01-27T02:38:41.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.666+0000 7fc83e6be700 1 -- 172.21.15.7:0/2549920539 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc824066f30 msgr2=0x7fc8240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:41.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.666+0000 7fc83e6be700 1 --2- 172.21.15.7:0/2549920539 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc824066f30 0x7fc8240693e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc8380a8a60 tx=0x7fc82c009040 comp rx=0 tx=0).stop 2024-01-27T02:38:41.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.666+0000 7fc83e6be700 1 -- 172.21.15.7:0/2549920539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8380ad9f0 msgr2=0x7fc8380a8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:41.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.666+0000 7fc83e6be700 1 --2- 172.21.15.7:0/2549920539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8380ad9f0 0x7fc8380a8b10 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc828009f00 tx=0x7fc828017000 comp rx=0 tx=0).stop 2024-01-27T02:38:41.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.666+0000 7fc83e6be700 1 -- 172.21.15.7:0/2549920539 shutdown_connections 2024-01-27T02:38:41.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.666+0000 7fc83e6be700 1 --2- 172.21.15.7:0/2549920539 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc824066f30 0x7fc8240693e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:41.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.666+0000 7fc83e6be700 1 --2- 172.21.15.7:0/2549920539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc838156f30 0x7fc83814aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:41.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.666+0000 7fc83e6be700 1 --2- 172.21.15.7:0/2549920539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8380ad9f0 0x7fc8380a8b10 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:41.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.666+0000 7fc83e6be700 1 --2- 172.21.15.7:0/2549920539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc838156b60 0x7fc8381488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:41.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.666+0000 7fc83e6be700 1 -- 172.21.15.7:0/2549920539 >> 172.21.15.7:0/2549920539 conn(0x7fc838068480 msgr2=0x7fc8381527e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:41.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.666+0000 7fc83e6be700 1 -- 172.21.15.7:0/2549920539 shutdown_connections 2024-01-27T02:38:41.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.666+0000 7fc83e6be700 1 -- 172.21.15.7:0/2549920539 wait complete. 2024-01-27T02:38:41.669 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-01-27T02:38:41.681 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:38:41.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.811+0000 7f763839a700 1 Processor -- start 2024-01-27T02:38:41.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.811+0000 7f763839a700 1 -- start start 2024-01-27T02:38:41.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.812+0000 7f763839a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7630157ca0 0x7f76300c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:41.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.812+0000 7f763839a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f76300c3ae0 0x7f76300cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:41.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.812+0000 7f763839a700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76300ca020 con 0x7f7630158070 2024-01-27T02:38:41.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.812+0000 7f763839a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76300c4f40 con 0x7f76300c3ae0 2024-01-27T02:38:41.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.812+0000 7f763839a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76300c5210 con 0x7f7630157ca0 2024-01-27T02:38:41.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.812+0000 7f7635935700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f76300c3ae0 0x7f76300cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:41.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.812+0000 7f7635935700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f76300c3ae0 0x7f76300cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37402/0 (socket says 172.21.15.7:37402) 2024-01-27T02:38:41.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.812+0000 7f7635935700 1 -- 172.21.15.7:0/464268760 learned_addr learned my addr 172.21.15.7:0/464268760 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:41.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.812+0000 7f7635935700 1 -- 172.21.15.7:0/464268760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7630157ca0 msgr2=0x7f76300c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:41.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.812+0000 7f7636136700 1 --2- 172.21.15.7:0/464268760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7630157ca0 0x7f76300c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:41.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.812+0000 7f7635935700 1 --2- 172.21.15.7:0/464268760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7630157ca0 0x7f76300c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:41.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.812+0000 7f7635935700 1 -- 172.21.15.7:0/464268760 >> v1:172.21.15.188:6789/0 conn(0x7f7630158070 legacy=0x7f763015bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:41.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.812+0000 7f7635935700 1 -- 172.21.15.7:0/464268760 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76300c54f0 con 0x7f76300c3ae0 2024-01-27T02:38:41.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.812+0000 7f7636136700 1 --2- 172.21.15.7:0/464268760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7630157ca0 0x7f76300c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:38:41.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.813+0000 7f7635935700 1 --2- 172.21.15.7:0/464268760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76300c3ae0 0x7f76300cd4e0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f76200099b0 tx=0x7f7620009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749f71ae4d44e7d0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:41.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.813+0000 7f7635134700 1 -- 172.21.15.7:0/464268760 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762000b820 con 0x7f76300c3ae0 2024-01-27T02:38:41.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.813+0000 7f7635134700 1 -- 172.21.15.7:0/464268760 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762000b980 con 0x7f76300c3ae0 2024-01-27T02:38:41.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.813+0000 7f7635134700 1 -- 172.21.15.7:0/464268760 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762000bd20 con 0x7f76300c3ae0 2024-01-27T02:38:41.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.813+0000 7f763839a700 1 -- 172.21.15.7:0/464268760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76300c3ae0 msgr2=0x7f76300cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:41.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.813+0000 7f763839a700 1 --2- 172.21.15.7:0/464268760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76300c3ae0 0x7f76300cd4e0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f76200099b0 tx=0x7f7620009250 comp rx=0 tx=0).stop 2024-01-27T02:38:41.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.813+0000 7f763839a700 1 -- 172.21.15.7:0/464268760 shutdown_connections 2024-01-27T02:38:41.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.813+0000 7f763839a700 1 --2- 172.21.15.7:0/464268760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7630157ca0 0x7f76300c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:41.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.813+0000 7f763839a700 1 --2- 172.21.15.7:0/464268760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76300c3ae0 0x7f76300cd4e0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:41.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.813+0000 7f763839a700 1 -- 172.21.15.7:0/464268760 >> 172.21.15.7:0/464268760 conn(0x7f7630098020 msgr2=0x7f76300711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:41.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.814+0000 7f763839a700 1 -- 172.21.15.7:0/464268760 shutdown_connections 2024-01-27T02:38:41.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.814+0000 7f763839a700 1 -- 172.21.15.7:0/464268760 wait complete. 2024-01-27T02:38:41.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.815+0000 7f763839a700 1 Processor -- start 2024-01-27T02:38:41.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.815+0000 7f763839a700 1 -- start start 2024-01-27T02:38:41.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.816+0000 7f763839a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76300c3ae0 0x7f7630168600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:41.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.816+0000 7f763839a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7630157ca0 0x7f76300c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:41.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.816+0000 7f763839a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7630158070 0x7f763016af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:41.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.816+0000 7f763839a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76300b36e0 con 0x7f7630157ca0 2024-01-27T02:38:41.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.816+0000 7f763839a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76300b2ad0 con 0x7f7630158070 2024-01-27T02:38:41.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.816+0000 7f763839a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76300b2db0 con 0x7f76300c3ae0 2024-01-27T02:38:41.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.816+0000 7f7635935700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7630157ca0 0x7f76300c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:41.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.816+0000 7f7635935700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7630157ca0 0x7f76300c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37406/0 (socket says 172.21.15.7:37406) 2024-01-27T02:38:41.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.816+0000 7f7635935700 1 -- 172.21.15.7:0/2543566943 learned_addr learned my addr 172.21.15.7:0/2543566943 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:41.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.816+0000 7f7636136700 1 --2- 172.21.15.7:0/2543566943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76300c3ae0 0x7f7630168600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:41.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.816+0000 7f7636937700 1 --2- 172.21.15.7:0/2543566943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7630158070 0x7f763016af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:41.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.816+0000 7f7635935700 1 -- 172.21.15.7:0/2543566943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76300c3ae0 msgr2=0x7f7630168600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:41.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.816+0000 7f7635935700 1 --2- 172.21.15.7:0/2543566943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76300c3ae0 0x7f7630168600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:41.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.816+0000 7f7635935700 1 -- 172.21.15.7:0/2543566943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7630158070 msgr2=0x7f763016af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:41.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.816+0000 7f7635935700 1 --2- 172.21.15.7:0/2543566943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7630158070 0x7f763016af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:41.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.816+0000 7f7635935700 1 -- 172.21.15.7:0/2543566943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76200094f0 con 0x7f7630157ca0 2024-01-27T02:38:41.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.816+0000 7f7635935700 1 --2- 172.21.15.7:0/2543566943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7630157ca0 0x7f76300c85d0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7620009290 tx=0x7f7620016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:41.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.816+0000 7f76267fc700 1 -- 172.21.15.7:0/2543566943 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7620011c10 con 0x7f7630157ca0 2024-01-27T02:38:41.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.816+0000 7f76267fc700 1 -- 172.21.15.7:0/2543566943 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7620020070 con 0x7f7630157ca0 2024-01-27T02:38:41.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.816+0000 7f76267fc700 1 -- 172.21.15.7:0/2543566943 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762001b450 con 0x7f7630157ca0 2024-01-27T02:38:41.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.816+0000 7f763839a700 1 -- 172.21.15.7:0/2543566943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76300b3090 con 0x7f7630157ca0 2024-01-27T02:38:41.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.817+0000 7f763839a700 1 -- 172.21.15.7:0/2543566943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76300c5e60 con 0x7f7630157ca0 2024-01-27T02:38:41.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.818+0000 7f763839a700 1 -- 172.21.15.7:0/2543566943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f763004c710 con 0x7f7630157ca0 2024-01-27T02:38:41.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.818+0000 7f76267fc700 1 -- 172.21.15.7:0/2543566943 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7620022030 con 0x7f7630157ca0 2024-01-27T02:38:41.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.818+0000 7f76267fc700 1 --2- 172.21.15.7:0/2543566943 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7614066f30 0x7f76140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:41.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.818+0000 7f76267fc700 1 -- 172.21.15.7:0/2543566943 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7620086460 con 0x7f7630157ca0 2024-01-27T02:38:41.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.819+0000 7f7636136700 1 --2- 172.21.15.7:0/2543566943 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7614066f30 0x7f76140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:41.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.819+0000 7f7636136700 1 --2- 172.21.15.7:0/2543566943 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7614066f30 0x7f76140693e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f762c00adc0 tx=0x7f762c006f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:41.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:41.823+0000 7f76267fc700 1 -- 172.21.15.7:0/2543566943 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7620053630 con 0x7f7630157ca0 2024-01-27T02:38:42.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.075+0000 7f763839a700 1 -- 172.21.15.7:0/2543566943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f763004c710 con 0x7f7630157ca0 2024-01-27T02:38:42.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.076+0000 7f76267fc700 1 -- 172.21.15.7:0/2543566943 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f7620056c50 con 0x7f7630157ca0 2024-01-27T02:38:42.077 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:42.078 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3997828193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3997828193},{"type":"v1","addr":"172.21.15.188:6837","nonce":3997828193}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1132338302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1132338302},{"type":"v1","addr":"172.21.15.188:6836","nonce":1132338302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/740290645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":740290645},{"type":"v1","addr":"172.21.15.7:6833","nonce":740290645}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:38:31.328277+0000","modified":"2024-01-27T02:38:34.940811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7029":{"gid":7029,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1258751919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1258751919},{"type":"v1","addr":"172.21.15.7:6835","nonce":1258751919}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-27T02:38:42.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.078+0000 7f763839a700 1 -- 172.21.15.7:0/2543566943 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7614066f30 msgr2=0x7f76140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:42.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.078+0000 7f763839a700 1 --2- 172.21.15.7:0/2543566943 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7614066f30 0x7f76140693e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f762c00adc0 tx=0x7f762c006f90 comp rx=0 tx=0).stop 2024-01-27T02:38:42.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.078+0000 7f763839a700 1 -- 172.21.15.7:0/2543566943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7630157ca0 msgr2=0x7f76300c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:42.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.078+0000 7f763839a700 1 --2- 172.21.15.7:0/2543566943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7630157ca0 0x7f76300c85d0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7620009290 tx=0x7f7620016000 comp rx=0 tx=0).stop 2024-01-27T02:38:42.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.078+0000 7f763839a700 1 -- 172.21.15.7:0/2543566943 shutdown_connections 2024-01-27T02:38:42.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.078+0000 7f763839a700 1 --2- 172.21.15.7:0/2543566943 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7614066f30 0x7f76140693e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.078+0000 7f763839a700 1 --2- 172.21.15.7:0/2543566943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7630158070 0x7f763016af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.078+0000 7f763839a700 1 --2- 172.21.15.7:0/2543566943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76300c3ae0 0x7f7630168600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.078+0000 7f763839a700 1 --2- 172.21.15.7:0/2543566943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7630157ca0 0x7f76300c85d0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.078+0000 7f763839a700 1 -- 172.21.15.7:0/2543566943 >> 172.21.15.7:0/2543566943 conn(0x7f7630098020 msgr2=0x7f76300cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:42.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.078+0000 7f763839a700 1 -- 172.21.15.7:0/2543566943 shutdown_connections 2024-01-27T02:38:42.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.078+0000 7f763839a700 1 -- 172.21.15.7:0/2543566943 wait complete. 2024-01-27T02:38:42.082 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 68 2024-01-27T02:38:42.496 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-27T02:38:31.328277+0000', 'modified': '2024-01-27T02:38:34.940811+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7029}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7029': {'gid': 7029, 'name': 'a', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6835/1258751919', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 1258751919}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 1258751919}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-27T02:38:42.496 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-27T02:38:42.496 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-27T02:38:42.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.638+0000 7f39155a7700 1 Processor -- start 2024-01-27T02:38:42.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.638+0000 7f39155a7700 1 -- start start 2024-01-27T02:38:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.638+0000 7f39155a7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3910159fc0 0x7f391015e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.639+0000 7f39155a7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39100cc830 con 0x7f391015a390 2024-01-27T02:38:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.639+0000 7f39155a7700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39100b35e0 con 0x7f39100d0da0 2024-01-27T02:38:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.639+0000 7f39155a7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39100cbf00 con 0x7f3910159fc0 2024-01-27T02:38:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.639+0000 7f390e7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f391015a390 0x7f39100b2ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46588/0 (socket says 172.21.15.7:46588) 2024-01-27T02:38:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.639+0000 7f390e7fc700 1 -- 172.21.15.7:0/2464633903 learned_addr learned my addr 172.21.15.7:0/2464633903 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.639+0000 7f390effd700 1 --2- 172.21.15.7:0/2464633903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3910159fc0 0x7f391015e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.639+0000 7f390dffb700 1 -- 172.21.15.7:0/2464633903 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3874047937 0 0) 0x7f39100cc830 con 0x7f391015a390 2024-01-27T02:38:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.639+0000 7f390dffb700 1 -- 172.21.15.7:0/2464633903 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38f8003570 con 0x7f391015a390 2024-01-27T02:38:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.639+0000 7f390dffb700 1 -- 172.21.15.7:0/2464633903 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1132371842 0 0) 0x7f39100b35e0 con 0x7f39100d0da0 2024-01-27T02:38:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.639+0000 7f390dffb700 1 -- 172.21.15.7:0/2464633903 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39100cc830 con 0x7f39100d0da0 2024-01-27T02:38:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.639+0000 7f390effd700 1 -- 172.21.15.7:0/2464633903 >> v1:172.21.15.188:6789/0 conn(0x7f39100d0da0 legacy=0x7f39100af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.639+0000 7f390effd700 1 -- 172.21.15.7:0/2464633903 >> v1:172.21.15.7:6789/0 conn(0x7f391015a390 legacy=0x7f39100b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.639+0000 7f390effd700 1 -- 172.21.15.7:0/2464633903 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39100cc1e0 con 0x7f3910159fc0 2024-01-27T02:38:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.640+0000 7f390effd700 1 --2- 172.21.15.7:0/2464633903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3910159fc0 0x7f391015e8d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3900009910 tx=0x7f39000091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53496ed0a6f015cc server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.640+0000 7f390dffb700 1 -- 172.21.15.7:0/2464633903 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3900011630 con 0x7f3910159fc0 2024-01-27T02:38:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.640+0000 7f390dffb700 1 -- 172.21.15.7:0/2464633903 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3900011790 con 0x7f3910159fc0 2024-01-27T02:38:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.640+0000 7f390dffb700 1 -- 172.21.15.7:0/2464633903 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3900011b30 con 0x7f3910159fc0 2024-01-27T02:38:42.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.640+0000 7f39155a7700 1 -- 172.21.15.7:0/2464633903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3910159fc0 msgr2=0x7f391015e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.640+0000 7f39155a7700 1 --2- 172.21.15.7:0/2464633903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3910159fc0 0x7f391015e8d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3900009910 tx=0x7f39000091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.640+0000 7f39155a7700 1 -- 172.21.15.7:0/2464633903 shutdown_connections 2024-01-27T02:38:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.640+0000 7f39155a7700 1 --2- 172.21.15.7:0/2464633903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3910159fc0 0x7f391015e8d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.640+0000 7f39155a7700 1 -- 172.21.15.7:0/2464633903 >> 172.21.15.7:0/2464633903 conn(0x7f39100679b0 msgr2=0x7f3910067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.641+0000 7f39155a7700 1 -- 172.21.15.7:0/2464633903 shutdown_connections 2024-01-27T02:38:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.641+0000 7f39155a7700 1 -- 172.21.15.7:0/2464633903 wait complete. 2024-01-27T02:38:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.642+0000 7f39155a7700 1 Processor -- start 2024-01-27T02:38:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.642+0000 7f39155a7700 1 -- start start 2024-01-27T02:38:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.643+0000 7f39155a7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39100d0da0 0x7f391016b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.643+0000 7f39155a7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3910159fc0 0x7f39100ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.643+0000 7f39155a7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f391015a390 0x7f391015a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.643+0000 7f39155a7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39100c7f00 con 0x7f391015a390 2024-01-27T02:38:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.643+0000 7f39155a7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39100c8a80 con 0x7f3910159fc0 2024-01-27T02:38:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.643+0000 7f39155a7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39100cacd0 con 0x7f39100d0da0 2024-01-27T02:38:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.643+0000 7f390f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f391015a390 0x7f391015a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.643+0000 7f390f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f391015a390 0x7f391015a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37410/0 (socket says 172.21.15.7:37410) 2024-01-27T02:38:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.643+0000 7f390f7fe700 1 -- 172.21.15.7:0/1746041753 learned_addr learned my addr 172.21.15.7:0/1746041753 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.643+0000 7f390effd700 1 --2- 172.21.15.7:0/1746041753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39100d0da0 0x7f391016b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.643+0000 7f390e7fc700 1 --2- 172.21.15.7:0/1746041753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3910159fc0 0x7f39100ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.643+0000 7f390f7fe700 1 -- 172.21.15.7:0/1746041753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39100d0da0 msgr2=0x7f391016b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.643+0000 7f390f7fe700 1 --2- 172.21.15.7:0/1746041753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39100d0da0 0x7f391016b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.643+0000 7f390f7fe700 1 -- 172.21.15.7:0/1746041753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3910159fc0 msgr2=0x7f39100ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.643+0000 7f390f7fe700 1 --2- 172.21.15.7:0/1746041753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3910159fc0 0x7f39100ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.643+0000 7f390f7fe700 1 -- 172.21.15.7:0/1746041753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3900009420 con 0x7f391015a390 2024-01-27T02:38:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.643+0000 7f390f7fe700 1 --2- 172.21.15.7:0/1746041753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f391015a390 0x7f391015a7a0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f390800c940 tx=0x7f39080027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.644+0000 7f38f6ffd700 1 -- 172.21.15.7:0/1746041753 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908012630 con 0x7f391015a390 2024-01-27T02:38:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.644+0000 7f38f6ffd700 1 -- 172.21.15.7:0/1746041753 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39080127b0 con 0x7f391015a390 2024-01-27T02:38:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.644+0000 7f39155a7700 1 -- 172.21.15.7:0/1746041753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39100c8150 con 0x7f391015a390 2024-01-27T02:38:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.644+0000 7f38f6ffd700 1 -- 172.21.15.7:0/1746041753 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908012b70 con 0x7f391015a390 2024-01-27T02:38:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.644+0000 7f39155a7700 1 -- 172.21.15.7:0/1746041753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39100c8710 con 0x7f391015a390 2024-01-27T02:38:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.645+0000 7f38f6ffd700 1 -- 172.21.15.7:0/1746041753 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f390801b400 con 0x7f391015a390 2024-01-27T02:38:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.645+0000 7f38f6ffd700 1 --2- 172.21.15.7:0/1746041753 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f38fc066f30 0x7f38fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.646+0000 7f38f6ffd700 1 -- 172.21.15.7:0/1746041753 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f39080847c0 con 0x7f391015a390 2024-01-27T02:38:42.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.646+0000 7f390effd700 1 --2- 172.21.15.7:0/1746041753 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f38fc066f30 0x7f38fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.646+0000 7f390effd700 1 --2- 172.21.15.7:0/1746041753 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f38fc066f30 0x7f38fc0693e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3900009860 tx=0x7f3900017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:42.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.648+0000 7f391e8aab80 1 Processor -- start 2024-01-27T02:38:42.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.648+0000 7f391e8aab80 1 -- start start 2024-01-27T02:38:42.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.649+0000 7f391e8aab80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f384377010 0x55f3843773e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.649+0000 7f391e8aab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3843bf360 0x55f3843a1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.649+0000 7f391e8aab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3843765d0 0x55f3843a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.649+0000 7f391e8aab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f38427a9f0 con 0x55f3843765d0 2024-01-27T02:38:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.649+0000 7f391e8aab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3842cc130 con 0x55f3843bf360 2024-01-27T02:38:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.649+0000 7f391e8aab80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3842c9930 con 0x55f384377010 2024-01-27T02:38:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.649+0000 7f390f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3843765d0 0x55f3843a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.649+0000 7f390effd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f384377010 0x55f3843773e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.649+0000 7f390effd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f384377010 0x55f3843773e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:56332/0 (socket says 172.21.15.7:56332) 2024-01-27T02:38:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.649+0000 7f390f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3843765d0 0x55f3843a1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37420/0 (socket says 172.21.15.7:37420) 2024-01-27T02:38:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.649+0000 7f390effd700 1 -- 172.21.15.7:0/4071260320 learned_addr learned my addr 172.21.15.7:0/4071260320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.649+0000 7f390e7fc700 1 --2- 172.21.15.7:0/4071260320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3843bf360 0x55f3843a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.650+0000 7f390effd700 1 -- 172.21.15.7:0/4071260320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3843bf360 msgr2=0x55f3843a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.650+0000 7f390effd700 1 --2- 172.21.15.7:0/4071260320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3843bf360 0x55f3843a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.650+0000 7f390effd700 1 -- 172.21.15.7:0/4071260320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3843765d0 msgr2=0x55f3843a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.650+0000 7f390effd700 1 --2- 172.21.15.7:0/4071260320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3843765d0 0x55f3843a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.650+0000 7f390effd700 1 -- 172.21.15.7:0/4071260320 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f384104b10 con 0x55f384377010 2024-01-27T02:38:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.650+0000 7f390f7fe700 1 --2- 172.21.15.7:0/4071260320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3843765d0 0x55f3843a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:38:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.650+0000 7f390effd700 1 --2- 172.21.15.7:0/4071260320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f384377010 0x55f3843773e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3900000c70 tx=0x7f3900022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28f3b9350e3a55a0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.650+0000 7f38effff700 1 -- 172.21.15.7:0/4071260320 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390002c070 con 0x55f384377010 2024-01-27T02:38:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.650+0000 7f38effff700 1 -- 172.21.15.7:0/4071260320 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3900020040 con 0x55f384377010 2024-01-27T02:38:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.650+0000 7f38effff700 1 -- 172.21.15.7:0/4071260320 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390001ab00 con 0x55f384377010 2024-01-27T02:38:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.651+0000 7f391e8aab80 1 -- 172.21.15.7:0/4071260320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f384377010 msgr2=0x55f3843773e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.651+0000 7f391e8aab80 1 --2- 172.21.15.7:0/4071260320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f384377010 0x55f3843773e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3900000c70 tx=0x7f3900022000 comp rx=0 tx=0).stop 2024-01-27T02:38:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.651+0000 7f391e8aab80 1 -- 172.21.15.7:0/4071260320 shutdown_connections 2024-01-27T02:38:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.651+0000 7f391e8aab80 1 --2- 172.21.15.7:0/4071260320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3843765d0 0x55f3843a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.651+0000 7f391e8aab80 1 --2- 172.21.15.7:0/4071260320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f384377010 0x55f3843773e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.651+0000 7f391e8aab80 1 --2- 172.21.15.7:0/4071260320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3843bf360 0x55f3843a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.651+0000 7f391e8aab80 1 -- 172.21.15.7:0/4071260320 >> 172.21.15.7:0/4071260320 conn(0x55f384376a70 msgr2=0x55f3843bf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.651+0000 7f391e8aab80 1 -- 172.21.15.7:0/4071260320 shutdown_connections 2024-01-27T02:38:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.651+0000 7f391e8aab80 1 -- 172.21.15.7:0/4071260320 wait complete. 2024-01-27T02:38:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.652+0000 7f391e8aab80 1 -- start start 2024-01-27T02:38:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.652+0000 7f391e8aab80 1 Processor -- start 2024-01-27T02:38:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.653+0000 7f391e8aab80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f3843765d0 0x55f3843cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.653+0000 7f391e8aab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f384377010 0x55f3843a5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.653+0000 7f391e8aab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3843bf360 0x55f3843b90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.653+0000 7f391e8aab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3842c5720 con 0x55f3843bf360 2024-01-27T02:38:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.653+0000 7f391e8aab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3842711b0 con 0x55f384377010 2024-01-27T02:38:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.653+0000 7f391e8aab80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f384270e50 con 0x55f3843765d0 2024-01-27T02:38:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.653+0000 7f391e8aab80 10 client.0 _ll_register_callbacks cb 0x55f3840d37b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-27T02:38:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.653+0000 7f390f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3843bf360 0x55f3843b90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.653+0000 7f390f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3843bf360 0x55f3843b90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37428/0 (socket says 172.21.15.7:37428) 2024-01-27T02:38:42.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.653+0000 7f390f7fe700 1 -- 172.21.15.7:0/1645380958 learned_addr learned my addr 172.21.15.7:0/1645380958 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:42.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.653+0000 7f390effd700 1 --2- 172.21.15.7:0/1645380958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f3843765d0 0x55f3843cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.653+0000 7f390e7fc700 1 --2- 172.21.15.7:0/1645380958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f384377010 0x55f3843a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.653+0000 7f390f7fe700 1 -- 172.21.15.7:0/1645380958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f3843765d0 msgr2=0x55f3843cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:42.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.653+0000 7f390f7fe700 1 --2- 172.21.15.7:0/1645380958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f3843765d0 0x55f3843cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.653+0000 7f390f7fe700 1 -- 172.21.15.7:0/1645380958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f384377010 msgr2=0x55f3843a5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:42.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.653+0000 7f390f7fe700 1 --2- 172.21.15.7:0/1645380958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f384377010 0x55f3843a5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.653+0000 7f390f7fe700 1 -- 172.21.15.7:0/1645380958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39100c8710 con 0x55f3843bf360 2024-01-27T02:38:42.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.653+0000 7f390f7fe700 1 --2- 172.21.15.7:0/1645380958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3843bf360 0x55f3843b90e0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f39080553f0 tx=0x7f3908082c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b3aff476fe5889 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:42.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.653+0000 7f38ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-27T02:38:42.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.653+0000 7f38ed7fa700 1 -- 172.21.15.7:0/1645380958 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908053b60 con 0x55f3843bf360 2024-01-27T02:38:42.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.653+0000 7f38ed7fa700 1 -- 172.21.15.7:0/1645380958 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3908054df0 con 0x55f3843bf360 2024-01-27T02:38:42.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.653+0000 7f391e8aab80 1 -- 172.21.15.7:0/1645380958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f38436a190 con 0x55f3843bf360 2024-01-27T02:38:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.654+0000 7f38ed7fa700 1 -- 172.21.15.7:0/1645380958 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f390805c200 con 0x55f3843bf360 2024-01-27T02:38:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.654+0000 7f391e8aab80 10 client.7176 fetch_fsmap learned FSMap version 68 2024-01-27T02:38:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.654+0000 7f391e8aab80 1 -- 172.21.15.7:0/1645380958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x7f39100cc1e0 con 0x55f3843bf360 2024-01-27T02:38:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.654+0000 7f38ed7fa700 1 -- 172.21.15.7:0/1645380958 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3908060020 con 0x55f3843bf360 2024-01-27T02:38:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.654+0000 7f38ed7fa700 1 -- 172.21.15.7:0/1645380958 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 68) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f39080514e0 con 0x55f3843bf360 2024-01-27T02:38:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.654+0000 7f391e8aab80 10 client.7176 fetch_fsmap finished waiting for FSMap version 68 2024-01-27T02:38:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.654+0000 7f391e8aab80 10 client.7176 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-27T02:38:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.654+0000 7f391e8aab80 20 client.7176 populate_metadata read hostname 'smithi007' 2024-01-27T02:38:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.654+0000 7f391e8aab80 1 --2- 172.21.15.7:0/1645380958 >> [v2:172.21.15.7:6834/1258751919,v1:172.21.15.7:6835/1258751919] conn(0x55f384370730 0x55f3843b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.654+0000 7f391e8aab80 4 client.7176 mds_command: new command op to 7029 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-27T02:38:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.654+0000 7f391e8aab80 1 -- 172.21.15.7:0/1645380958 --> [v2:172.21.15.7:6834/1258751919,v1:172.21.15.7:6835/1258751919] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f384267ee0 con 0x55f384370730 2024-01-27T02:38:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.654+0000 7f390effd700 1 --2- 172.21.15.7:0/1645380958 >> [v2:172.21.15.7:6834/1258751919,v1:172.21.15.7:6835/1258751919] conn(0x55f384370730 0x55f3843b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.655+0000 7f38ed7fa700 10 client.7176 ms_handle_connect on v2:172.21.15.7:6834/1258751919 2024-01-27T02:38:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.655+0000 7f390effd700 1 --2- 172.21.15.7:0/1645380958 >> [v2:172.21.15.7:6834/1258751919,v1:172.21.15.7:6835/1258751919] conn(0x55f384370730 0x55f3843b6ec0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a06fb13b07fd1dd0 server_cookie=a42420d9ef4380ed in_seq=0 out_seq=0 2024-01-27T02:38:42.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.655+0000 7f38ed7fa700 1 -- 172.21.15.7:0/1645380958 <== mds.0 v2:172.21.15.7:6834/1258751919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17146 (crc 0 0 0) 0x55f384267ee0 con 0x55f384370730 2024-01-27T02:38:42.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.655+0000 7f38ed7fa700 10 client.7176 handle_command_reply: tid=0 2024-01-27T02:38:42.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.655+0000 7f391e8aab80 1 client.7176 shutdown 2024-01-27T02:38:42.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.655+0000 7f391e8aab80 1 -- 172.21.15.7:0/1645380958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3843bf360 msgr2=0x55f3843b90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:42.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.655+0000 7f391e8aab80 1 --2- 172.21.15.7:0/1645380958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3843bf360 0x55f3843b90e0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f39080553f0 tx=0x7f3908082c40 comp rx=0 tx=0).stop 2024-01-27T02:38:42.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.655+0000 7f391e8aab80 1 -- 172.21.15.7:0/1645380958 shutdown_connections 2024-01-27T02:38:42.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.655+0000 7f391e8aab80 1 --2- 172.21.15.7:0/1645380958 >> [v2:172.21.15.7:6834/1258751919,v1:172.21.15.7:6835/1258751919] conn(0x55f384370730 0x55f3843b6ec0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.655+0000 7f391e8aab80 1 --2- 172.21.15.7:0/1645380958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3843bf360 0x55f3843b90e0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.655+0000 7f391e8aab80 1 --2- 172.21.15.7:0/1645380958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f3843765d0 0x55f3843cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.655+0000 7f38ed7fa700 0 client.7176 ms_handle_reset on v2:172.21.15.7:6834/1258751919 2024-01-27T02:38:42.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.655+0000 7f391e8aab80 1 --2- 172.21.15.7:0/1645380958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f384377010 0x55f3843a5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.656+0000 7f391e8aab80 1 -- 172.21.15.7:0/1645380958 >> 172.21.15.7:0/1645380958 conn(0x55f384376a70 msgr2=0x55f3843b9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:42.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.656+0000 7f391e8aab80 1 -- 172.21.15.7:0/1645380958 shutdown_connections 2024-01-27T02:38:42.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.656+0000 7f391e8aab80 1 -- 172.21.15.7:0/1645380958 wait complete. 2024-01-27T02:38:42.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.656+0000 7f391e8aab80 20 client.7176 trim_cache size 0 max 16384 2024-01-27T02:38:42.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.674+0000 7f391e8aab80 1 Processor -- start 2024-01-27T02:38:42.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.674+0000 7f391e8aab80 1 -- start start 2024-01-27T02:38:42.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.675+0000 7f391e8aab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3843765d0 0x55f3843a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.675+0000 7f391e8aab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f384377010 0x55f3843a5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.675+0000 7f391e8aab80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f3843bf360 0x55f3843b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.675+0000 7f391e8aab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3841bde40 con 0x55f384377010 2024-01-27T02:38:42.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.675+0000 7f391e8aab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3843cc880 con 0x55f3843765d0 2024-01-27T02:38:42.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.675+0000 7f391e8aab80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f38436fbc0 con 0x55f3843bf360 2024-01-27T02:38:42.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.675+0000 7f390e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f384377010 0x55f3843a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.675+0000 7f390e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f384377010 0x55f3843a5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37430/0 (socket says 172.21.15.7:37430) 2024-01-27T02:38:42.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.675+0000 7f390e7fc700 1 -- 172.21.15.7:0/2941044365 learned_addr learned my addr 172.21.15.7:0/2941044365 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:42.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.675+0000 7f390f7fe700 1 --2- 172.21.15.7:0/2941044365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f3843bf360 0x55f3843b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.675+0000 7f390effd700 1 --2- 172.21.15.7:0/2941044365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3843765d0 0x55f3843a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.676+0000 7f390e7fc700 1 -- 172.21.15.7:0/2941044365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f3843bf360 msgr2=0x55f3843b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:42.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.676+0000 7f390e7fc700 1 --2- 172.21.15.7:0/2941044365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f3843bf360 0x55f3843b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.676+0000 7f390e7fc700 1 -- 172.21.15.7:0/2941044365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3843765d0 msgr2=0x55f3843a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:42.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.676+0000 7f390e7fc700 1 --2- 172.21.15.7:0/2941044365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3843765d0 0x55f3843a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.676+0000 7f390e7fc700 1 -- 172.21.15.7:0/2941044365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f384293290 con 0x55f384377010 2024-01-27T02:38:42.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.676+0000 7f390e7fc700 1 --2- 172.21.15.7:0/2941044365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f384377010 0x55f3843a5090 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f391015c4d0 tx=0x7f3904009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ebfa60bd55f7ad8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:42.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.676+0000 7f38f4ff9700 1 -- 172.21.15.7:0/2941044365 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904001d20 con 0x55f384377010 2024-01-27T02:38:42.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.676+0000 7f38f4ff9700 1 -- 172.21.15.7:0/2941044365 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3904005b10 con 0x55f384377010 2024-01-27T02:38:42.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.676+0000 7f38f4ff9700 1 -- 172.21.15.7:0/2941044365 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904005cc0 con 0x55f384377010 2024-01-27T02:38:42.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.677+0000 7f391e8aab80 1 -- 172.21.15.7:0/2941044365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f384377010 msgr2=0x55f3843a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:42.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.677+0000 7f391e8aab80 1 --2- 172.21.15.7:0/2941044365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f384377010 0x55f3843a5090 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f391015c4d0 tx=0x7f3904009040 comp rx=0 tx=0).stop 2024-01-27T02:38:42.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.677+0000 7f391e8aab80 1 -- 172.21.15.7:0/2941044365 shutdown_connections 2024-01-27T02:38:42.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.677+0000 7f391e8aab80 1 --2- 172.21.15.7:0/2941044365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f3843bf360 0x55f3843b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.677+0000 7f391e8aab80 1 --2- 172.21.15.7:0/2941044365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f3843765d0 0x55f3843a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.677+0000 7f391e8aab80 1 --2- 172.21.15.7:0/2941044365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f384377010 0x55f3843a5090 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.677+0000 7f391e8aab80 1 -- 172.21.15.7:0/2941044365 >> 172.21.15.7:0/2941044365 conn(0x55f384376a70 msgr2=0x55f3843b9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:42.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.677+0000 7f391e8aab80 1 -- 172.21.15.7:0/2941044365 shutdown_connections 2024-01-27T02:38:42.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.677+0000 7f391e8aab80 1 -- 172.21.15.7:0/2941044365 wait complete. 2024-01-27T02:38:42.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.677+0000 7f391e8aab80 1 -- start start 2024-01-27T02:38:42.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.677+0000 7f391e8aab80 1 Processor -- start 2024-01-27T02:38:42.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.679+0000 7f391e8aab80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f3843765d0 0x55f384403440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.679+0000 7f391e8aab80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f384377010 0x55f3843d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.679+0000 7f391e8aab80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3843bf360 0x55f384403b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.679+0000 7f391e8aab80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f384404900 con 0x55f3843bf360 2024-01-27T02:38:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.679+0000 7f391e8aab80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3843d5220 con 0x55f384377010 2024-01-27T02:38:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.679+0000 7f391e8aab80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3843b5be0 con 0x55f3843765d0 2024-01-27T02:38:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.679+0000 7f391e8aab80 10 client.0 _ll_register_callbacks cb 0x55f3840d37b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-27T02:38:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.679+0000 7f390f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3843bf360 0x55f384403b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.679+0000 7f390f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3843bf360 0x55f384403b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37440/0 (socket says 172.21.15.7:37440) 2024-01-27T02:38:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.679+0000 7f390f7fe700 1 -- 172.21.15.7:0/3113118328 learned_addr learned my addr 172.21.15.7:0/3113118328 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.679+0000 7f390effd700 1 --2- 172.21.15.7:0/3113118328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f3843765d0 0x55f384403440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.679+0000 7f390e7fc700 1 --2- 172.21.15.7:0/3113118328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f384377010 0x55f3843d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.679+0000 7f390f7fe700 1 -- 172.21.15.7:0/3113118328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f3843765d0 msgr2=0x55f384403440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:42.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.679+0000 7f390f7fe700 1 --2- 172.21.15.7:0/3113118328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f3843765d0 0x55f384403440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.679+0000 7f390f7fe700 1 -- 172.21.15.7:0/3113118328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f384377010 msgr2=0x55f3843d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:42.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.679+0000 7f390f7fe700 1 --2- 172.21.15.7:0/3113118328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f384377010 0x55f3843d4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.680+0000 7f390f7fe700 1 -- 172.21.15.7:0/3113118328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f8003570 con 0x55f3843bf360 2024-01-27T02:38:42.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.680+0000 7f390e7fc700 1 --2- 172.21.15.7:0/3113118328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f384377010 0x55f3843d4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:38:42.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.680+0000 7f390f7fe700 1 --2- 172.21.15.7:0/3113118328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3843bf360 0x55f384403b00 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f390800c910 tx=0x7f390806d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e7ba69f361e2da server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:42.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.680+0000 7f38ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-27T02:38:42.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.680+0000 7f38ed7fa700 1 -- 172.21.15.7:0/3113118328 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908055230 con 0x55f3843bf360 2024-01-27T02:38:42.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.680+0000 7f38ed7fa700 1 -- 172.21.15.7:0/3113118328 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3908061080 con 0x55f3843bf360 2024-01-27T02:38:42.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.680+0000 7f391e8aab80 1 -- 172.21.15.7:0/3113118328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3900020040 con 0x55f3843bf360 2024-01-27T02:38:42.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.680+0000 7f38ed7fa700 1 -- 172.21.15.7:0/3113118328 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3908053a90 con 0x55f3843bf360 2024-01-27T02:38:42.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.680+0000 7f391e8aab80 10 client.7182 fetch_fsmap learned FSMap version 68 2024-01-27T02:38:42.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.680+0000 7f391e8aab80 1 -- 172.21.15.7:0/3113118328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x55f38415cc80 con 0x55f3843bf360 2024-01-27T02:38:42.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.680+0000 7f38ed7fa700 1 -- 172.21.15.7:0/3113118328 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3900020040 con 0x55f3843bf360 2024-01-27T02:38:42.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.680+0000 7f38ed7fa700 1 -- 172.21.15.7:0/3113118328 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 68) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f3908078050 con 0x55f3843bf360 2024-01-27T02:38:42.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.680+0000 7f391e8aab80 10 client.7182 fetch_fsmap finished waiting for FSMap version 68 2024-01-27T02:38:42.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.680+0000 7f391e8aab80 10 client.7182 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-27T02:38:42.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.680+0000 7f391e8aab80 20 client.7182 populate_metadata read hostname 'smithi007' 2024-01-27T02:38:42.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.680+0000 7f391e8aab80 1 --2- 172.21.15.7:0/3113118328 >> [v2:172.21.15.7:6834/1258751919,v1:172.21.15.7:6835/1258751919] conn(0x55f3843a4b50 0x55f3843a4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.680+0000 7f391e8aab80 4 client.7182 mds_command: new command op to 7029 tid=0[{"prefix": "status"}] 2024-01-27T02:38:42.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.680+0000 7f391e8aab80 1 -- 172.21.15.7:0/3113118328 --> [v2:172.21.15.7:6834/1258751919,v1:172.21.15.7:6835/1258751919] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f384267c00 con 0x55f3843a4b50 2024-01-27T02:38:42.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.681+0000 7f390effd700 1 --2- 172.21.15.7:0/3113118328 >> [v2:172.21.15.7:6834/1258751919,v1:172.21.15.7:6835/1258751919] conn(0x55f3843a4b50 0x55f3843a4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.681+0000 7f38ed7fa700 10 client.7182 ms_handle_connect on v2:172.21.15.7:6834/1258751919 2024-01-27T02:38:42.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.681+0000 7f390effd700 1 --2- 172.21.15.7:0/3113118328 >> [v2:172.21.15.7:6834/1258751919,v1:172.21.15.7:6835/1258751919] conn(0x55f3843a4b50 0x55f3843a4f20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17068fac036ae4fd server_cookie=b311cfb0e0706109 in_seq=0 out_seq=0 2024-01-27T02:38:42.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.681+0000 7f38ed7fa700 1 -- 172.21.15.7:0/3113118328 <== mds.0 v2:172.21.15.7:6834/1258751919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55f384267c00 con 0x55f3843a4b50 2024-01-27T02:38:42.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.681+0000 7f38ed7fa700 10 client.7182 handle_command_reply: tid=0 2024-01-27T02:38:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.681+0000 7f391e8aab80 1 client.7182 shutdown 2024-01-27T02:38:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.681+0000 7f391e8aab80 1 -- 172.21.15.7:0/3113118328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3843bf360 msgr2=0x55f384403b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.681+0000 7f391e8aab80 1 --2- 172.21.15.7:0/3113118328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3843bf360 0x55f384403b00 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f390800c910 tx=0x7f390806d000 comp rx=0 tx=0).stop 2024-01-27T02:38:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.681+0000 7f391e8aab80 1 -- 172.21.15.7:0/3113118328 shutdown_connections 2024-01-27T02:38:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.681+0000 7f391e8aab80 1 --2- 172.21.15.7:0/3113118328 >> [v2:172.21.15.7:6834/1258751919,v1:172.21.15.7:6835/1258751919] conn(0x55f3843a4b50 0x55f3843a4f20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.681+0000 7f391e8aab80 1 --2- 172.21.15.7:0/3113118328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f3843bf360 0x55f384403b00 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.681+0000 7f391e8aab80 1 --2- 172.21.15.7:0/3113118328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f3843765d0 0x55f384403440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.681+0000 7f38ed7fa700 0 client.7182 ms_handle_reset on v2:172.21.15.7:6834/1258751919 2024-01-27T02:38:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.681+0000 7f391e8aab80 1 --2- 172.21.15.7:0/3113118328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f384377010 0x55f3843d4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.681+0000 7f391e8aab80 1 -- 172.21.15.7:0/3113118328 >> 172.21.15.7:0/3113118328 conn(0x55f384376a70 msgr2=0x55f3843b70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.682+0000 7f391e8aab80 1 -- 172.21.15.7:0/3113118328 shutdown_connections 2024-01-27T02:38:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.682+0000 7f391e8aab80 1 -- 172.21.15.7:0/3113118328 wait complete. 2024-01-27T02:38:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.682+0000 7f391e8aab80 20 client.7182 trim_cache size 0 max 16384 2024-01-27T02:38:42.689 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:38:42.689 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:38:42.689 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-27T02:38:42.689 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 7029, 2024-01-27T02:38:42.689 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-27T02:38:42.689 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-27T02:38:42.689 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-27T02:38:42.689 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 10.764343522000001, 2024-01-27T02:38:42.689 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 68, 2024-01-27T02:38:42.689 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 132, 2024-01-27T02:38:42.689 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 132, 2024-01-27T02:38:42.690 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 16.172712112999999 2024-01-27T02:38:42.690 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:38:42.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.682+0000 7f39155a7700 1 -- 172.21.15.7:0/1746041753 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f38fc066f30 msgr2=0x7f38fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:42.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.682+0000 7f39155a7700 1 --2- 172.21.15.7:0/1746041753 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f38fc066f30 0x7f38fc0693e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3900009860 tx=0x7f3900017000 comp rx=0 tx=0).stop 2024-01-27T02:38:42.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.682+0000 7f39155a7700 1 -- 172.21.15.7:0/1746041753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f391015a390 msgr2=0x7f391015a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:42.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.682+0000 7f39155a7700 1 --2- 172.21.15.7:0/1746041753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f391015a390 0x7f391015a7a0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f390800c940 tx=0x7f39080027a0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.682+0000 7f39155a7700 1 -- 172.21.15.7:0/1746041753 shutdown_connections 2024-01-27T02:38:42.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.682+0000 7f39155a7700 1 --2- 172.21.15.7:0/1746041753 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f38fc066f30 0x7f38fc0693e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.682+0000 7f39155a7700 1 --2- 172.21.15.7:0/1746041753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f391015a390 0x7f391015a7a0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.682+0000 7f39155a7700 1 --2- 172.21.15.7:0/1746041753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39100d0da0 0x7f391016b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.682+0000 7f39155a7700 1 --2- 172.21.15.7:0/1746041753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3910159fc0 0x7f39100ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.682+0000 7f39155a7700 1 -- 172.21.15.7:0/1746041753 >> 172.21.15.7:0/1746041753 conn(0x7f39100679b0 msgr2=0x7f39101593d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:42.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.682+0000 7f39155a7700 1 -- 172.21.15.7:0/1746041753 shutdown_connections 2024-01-27T02:38:42.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.682+0000 7f39155a7700 1 -- 172.21.15.7:0/1746041753 wait complete. 2024-01-27T02:38:42.697 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:38:42.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.826+0000 7f1bb4870700 1 Processor -- start 2024-01-27T02:38:42.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.827+0000 7f1bb4870700 1 -- start start 2024-01-27T02:38:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.828+0000 7f1bb4870700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1bb0159f40 0x7f1bb00c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.828+0000 7f1bb4870700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb00c30c0 con 0x7f1bb00c3a90 2024-01-27T02:38:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.828+0000 7f1bb4870700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb015e960 con 0x7f1bb00aafc0 2024-01-27T02:38:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.828+0000 7f1bb4870700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb00c4550 con 0x7f1bb0159f40 2024-01-27T02:38:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.828+0000 7f1bae59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1bb0159f40 0x7f1bb00c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.828+0000 7f1bae59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1bb0159f40 0x7f1bb00c3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37444/0 (socket says 172.21.15.7:37444) 2024-01-27T02:38:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.828+0000 7f1badd9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1bb00c3a90 0x7f1bb00aa8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46590/0 (socket says 172.21.15.7:46590) 2024-01-27T02:38:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.828+0000 7f1bae59c700 1 -- 172.21.15.7:0/689971387 learned_addr learned my addr 172.21.15.7:0/689971387 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.828+0000 7f1bae59c700 1 -- 172.21.15.7:0/689971387 >> v1:172.21.15.188:6789/0 conn(0x7f1bb00aafc0 legacy=0x7f1bb0063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.828+0000 7f1bae59c700 1 -- 172.21.15.7:0/689971387 >> v1:172.21.15.7:6789/0 conn(0x7f1bb00c3a90 legacy=0x7f1bb00aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.828+0000 7f1bae59c700 1 -- 172.21.15.7:0/689971387 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb015ec40 con 0x7f1bb0159f40 2024-01-27T02:38:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.828+0000 7f1bad59a700 1 -- 172.21.15.7:0/689971387 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 800891846 0 0) 0x7f1bb00c30c0 con 0x7f1bb00c3a90 2024-01-27T02:38:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.828+0000 7f1bae59c700 1 --2- 172.21.15.7:0/689971387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb0159f40 0x7f1bb00c3550 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00099b0 tx=0x7f1ba0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b8c664a2ef7bd3 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.828+0000 7f1bad59a700 1 -- 172.21.15.7:0/689971387 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba000b860 con 0x7f1bb0159f40 2024-01-27T02:38:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.829+0000 7f1bad59a700 1 -- 172.21.15.7:0/689971387 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba000b9c0 con 0x7f1bb0159f40 2024-01-27T02:38:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.829+0000 7f1bad59a700 1 -- 172.21.15.7:0/689971387 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba000bd60 con 0x7f1bb0159f40 2024-01-27T02:38:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.829+0000 7f1bb4870700 1 -- 172.21.15.7:0/689971387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb0159f40 msgr2=0x7f1bb00c3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.829+0000 7f1bb4870700 1 --2- 172.21.15.7:0/689971387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb0159f40 0x7f1bb00c3550 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00099b0 tx=0x7f1ba0009250 comp rx=0 tx=0).stop 2024-01-27T02:38:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.829+0000 7f1bb4870700 1 -- 172.21.15.7:0/689971387 shutdown_connections 2024-01-27T02:38:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.829+0000 7f1bb4870700 1 --2- 172.21.15.7:0/689971387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb0159f40 0x7f1bb00c3550 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.829+0000 7f1bb4870700 1 -- 172.21.15.7:0/689971387 >> 172.21.15.7:0/689971387 conn(0x7f1bb00cdc30 msgr2=0x7f1bb00ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.829+0000 7f1bb4870700 1 -- 172.21.15.7:0/689971387 shutdown_connections 2024-01-27T02:38:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.829+0000 7f1bb4870700 1 -- 172.21.15.7:0/689971387 wait complete. 2024-01-27T02:38:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.830+0000 7f1bb4870700 1 Processor -- start 2024-01-27T02:38:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.831+0000 7f1bb4870700 1 -- start start 2024-01-27T02:38:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.831+0000 7f1bb4870700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bb00aafc0 0x7f1bb0150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.831+0000 7f1bb4870700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bb00c3a90 0x7f1bb016afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.831+0000 7f1bb4870700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb0159f40 0x7f1bb015a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.831+0000 7f1bb4870700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb016a5c0 con 0x7f1bb0159f40 2024-01-27T02:38:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.831+0000 7f1bb4870700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb0150ca0 con 0x7f1bb00c3a90 2024-01-27T02:38:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.831+0000 7f1bb4870700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb015ab60 con 0x7f1bb00aafc0 2024-01-27T02:38:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.831+0000 7f1baed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb0159f40 0x7f1bb015a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.832+0000 7f1baed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb0159f40 0x7f1bb015a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37454/0 (socket says 172.21.15.7:37454) 2024-01-27T02:38:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.832+0000 7f1baed9d700 1 -- 172.21.15.7:0/3962985931 learned_addr learned my addr 172.21.15.7:0/3962985931 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.832+0000 7f1badd9b700 1 --2- 172.21.15.7:0/3962985931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bb00c3a90 0x7f1bb016afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.832+0000 7f1bae59c700 1 --2- 172.21.15.7:0/3962985931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bb00aafc0 0x7f1bb0150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.832+0000 7f1baed9d700 1 -- 172.21.15.7:0/3962985931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bb00aafc0 msgr2=0x7f1bb0150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.832+0000 7f1baed9d700 1 --2- 172.21.15.7:0/3962985931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bb00aafc0 0x7f1bb0150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.832+0000 7f1baed9d700 1 -- 172.21.15.7:0/3962985931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bb00c3a90 msgr2=0x7f1bb016afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.832+0000 7f1baed9d700 1 --2- 172.21.15.7:0/3962985931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bb00c3a90 0x7f1bb016afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.832+0000 7f1baed9d700 1 -- 172.21.15.7:0/3962985931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba00094f0 con 0x7f1bb0159f40 2024-01-27T02:38:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.832+0000 7f1baed9d700 1 --2- 172.21.15.7:0/3962985931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb0159f40 0x7f1bb015a310 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800c840 tx=0x7f1ba8001610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.832+0000 7f1b9e7fc700 1 -- 172.21.15.7:0/3962985931 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8012630 con 0x7f1bb0159f40 2024-01-27T02:38:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.832+0000 7f1b9e7fc700 1 -- 172.21.15.7:0/3962985931 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba80127b0 con 0x7f1bb0159f40 2024-01-27T02:38:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.832+0000 7f1b9e7fc700 1 -- 172.21.15.7:0/3962985931 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8012b70 con 0x7f1bb0159f40 2024-01-27T02:38:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.832+0000 7f1bb4870700 1 -- 172.21.15.7:0/3962985931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb015ae40 con 0x7f1bb0159f40 2024-01-27T02:38:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.832+0000 7f1bb4870700 1 -- 172.21.15.7:0/3962985931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb016aab0 con 0x7f1bb0159f40 2024-01-27T02:38:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.833+0000 7f1bb4870700 1 -- 172.21.15.7:0/3962985931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb015b030 con 0x7f1bb0159f40 2024-01-27T02:38:42.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.834+0000 7f1b9e7fc700 1 -- 172.21.15.7:0/3962985931 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1ba801a3a0 con 0x7f1bb0159f40 2024-01-27T02:38:42.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.834+0000 7f1b9e7fc700 1 --2- 172.21.15.7:0/3962985931 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1b90066f30 0x7f1b900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:42.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.834+0000 7f1b9e7fc700 1 -- 172.21.15.7:0/3962985931 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1ba8084690 con 0x7f1bb0159f40 2024-01-27T02:38:42.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.834+0000 7f1bae59c700 1 --2- 172.21.15.7:0/3962985931 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1b90066f30 0x7f1b900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:42.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.835+0000 7f1bae59c700 1 --2- 172.21.15.7:0/3962985931 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1b90066f30 0x7f1b900693e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00099b0 tx=0x7f1ba0017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:42.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:42.839+0000 7f1b9e7fc700 1 -- 172.21.15.7:0/3962985931 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ba8051860 con 0x7f1bb0159f40 2024-01-27T02:38:43.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.098+0000 7f1bb4870700 1 -- 172.21.15.7:0/3962985931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1bb004c710 con 0x7f1bb0159f40 2024-01-27T02:38:43.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.098+0000 7f1b9e7fc700 1 -- 172.21.15.7:0/3962985931 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v68) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1ba8054e80 con 0x7f1bb0159f40 2024-01-27T02:38:43.100 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:38:43.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.100+0000 7f1bb4870700 1 -- 172.21.15.7:0/3962985931 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1b90066f30 msgr2=0x7f1b900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:43.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.100+0000 7f1bb4870700 1 --2- 172.21.15.7:0/3962985931 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1b90066f30 0x7f1b900693e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00099b0 tx=0x7f1ba0017000 comp rx=0 tx=0).stop 2024-01-27T02:38:43.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.100+0000 7f1bb4870700 1 -- 172.21.15.7:0/3962985931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb0159f40 msgr2=0x7f1bb015a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:43.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.100+0000 7f1bb4870700 1 --2- 172.21.15.7:0/3962985931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb0159f40 0x7f1bb015a310 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800c840 tx=0x7f1ba8001610 comp rx=0 tx=0).stop 2024-01-27T02:38:43.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.100+0000 7f1bb4870700 1 -- 172.21.15.7:0/3962985931 shutdown_connections 2024-01-27T02:38:43.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.100+0000 7f1bb4870700 1 --2- 172.21.15.7:0/3962985931 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1b90066f30 0x7f1b900693e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:43.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.100+0000 7f1bb4870700 1 --2- 172.21.15.7:0/3962985931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb0159f40 0x7f1bb015a310 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:43.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.100+0000 7f1bb4870700 1 --2- 172.21.15.7:0/3962985931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bb00aafc0 0x7f1bb0150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:43.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.100+0000 7f1bb4870700 1 --2- 172.21.15.7:0/3962985931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bb00c3a90 0x7f1bb016afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:43.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.100+0000 7f1bb4870700 1 -- 172.21.15.7:0/3962985931 >> 172.21.15.7:0/3962985931 conn(0x7f1bb00cdc30 msgr2=0x7f1bb01583c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:43.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.100+0000 7f1bb4870700 1 -- 172.21.15.7:0/3962985931 shutdown_connections 2024-01-27T02:38:43.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.100+0000 7f1bb4870700 1 -- 172.21.15.7:0/3962985931 wait complete. 2024-01-27T02:38:43.115 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-27T02:38:43.115 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:38:43.116 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-27T02:38:43.116 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-27T02:38:43.116 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-27T02:38:43.116 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-27T02:38:43.116 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-27T02:38:43.116 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:38:43.116 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:38:43.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:43.117 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-27T02:38:43.137 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:38:43.137 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:38:43.137 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:38:43.137 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:43.137 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-27T02:38:43.137 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:43.138 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:38:43.138 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-27T02:38:43.138 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-27T02:38:43.138 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:38:43.138 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-27T02:38:43.138 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-27T02:38:43.138 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:38:43.138 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-27T02:38:43.138 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-27T02:38:43.138 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-27T02:38:43.138 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 41988sec preferred_lft 41988sec 2024-01-27T02:38:43.138 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-27T02:38:43.138 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:43.138 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:38:43.139 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-27T02:38:43.139 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-27T02:38:43.139 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:38:43.139 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 4a:57:d9:3c:c5:a0 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:38:43.139 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-27T02:38:43.139 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:43.139 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::2477:9eff:fe39:433c/64 scope link 2024-01-27T02:38:43.139 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:43.139 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:38:43.139 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 4a:57:d9:3c:c5:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-27T02:38:43.139 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::4857:d9ff:fe3c:c5a0/64 scope link 2024-01-27T02:38:43.139 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:43.139 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:38:43.139 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ca:6e:b8:df:53:bf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-27T02:38:43.140 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c86e:b8ff:fedf:53bf/64 scope link 2024-01-27T02:38:43.140 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:43.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:43.140 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:38:43.208 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-27T02:38:43.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:43.210 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-27T02:38:43.275 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-27T02:38:43.275 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:38:43.275 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-27T02:38:43.275 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-27T02:38:43.275 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:38:43.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:43.276 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-27T02:38:43.276 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-27T02:38:43.276 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-27T02:38:43.276 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-27T02:38:43.276 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-27T02:38:43.276 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-27T02:38:43.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:43.400 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-27T02:38:43.421 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-27T02:38:43.421 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-27T02:38:43.421 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:38:43.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:43.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-27T02:38:43.497 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:38:43.497 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-27T02:38:43.559 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:38:43.559 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:38:43.559 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-27T02:38:43.620 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:38:43.620 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:38:43.621 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:38:43.715 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:38:43.716 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:38:43.716 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:38:43.725 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:38:43.725 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:38:43.726 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:38:43.726 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:38:43.726 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:38:43.743 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:38:43.743 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:38:43.743 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:38:43.743 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:38:43.743 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:38:43.743 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:38:43.743 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:38:43.743 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:38:43.745 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:38:43.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:43.745 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-27T02:38:43.777 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:38:43.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.956+0000 7f98d9e20700 1 Processor -- start 2024-01-27T02:38:43.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.956+0000 7f98d9e20700 1 -- start start 2024-01-27T02:38:43.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.957+0000 7f98d9e20700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f98d40a7c10 0x7f98d40ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:43.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.957+0000 7f98d9e20700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f98d40a7fe0 0x7f98d40b0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:43.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.957+0000 7f98d9e20700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d415a880 con 0x7f98d40ae880 2024-01-27T02:38:43.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.957+0000 7f98d9e20700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98d40afde0 con 0x7f98d40a7fe0 2024-01-27T02:38:43.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.957+0000 7f98d9e20700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98d40af950 con 0x7f98d40a7c10 2024-01-27T02:38:43.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.957+0000 7f98d3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f98d40a7fe0 0x7f98d40b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:43.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.957+0000 7f98d3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f98d40a7fe0 0x7f98d40b0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37456/0 (socket says 172.21.15.7:37456) 2024-01-27T02:38:43.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.958+0000 7f98d3fff700 1 -- 172.21.15.7:0/309652934 learned_addr learned my addr 172.21.15.7:0/309652934 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:43.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.958+0000 7f98d37fe700 1 --2- 172.21.15.7:0/309652934 >> v2:172.21.15.188:3300/0 conn(0x7f98d40a7c10 0x7f98d40ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:43.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.958+0000 7f98d3fff700 1 -- 172.21.15.7:0/309652934 >> v2:172.21.15.188:3300/0 conn(0x7f98d40a7c10 msgr2=0x7f98d40ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:43.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.958+0000 7f98d3fff700 1 --2- 172.21.15.7:0/309652934 >> v2:172.21.15.188:3300/0 conn(0x7f98d40a7c10 0x7f98d40ae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:43.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.958+0000 7f98d3fff700 1 -- 172.21.15.7:0/309652934 >> v1:172.21.15.188:6789/0 conn(0x7f98d40ae880 legacy=0x7f98d406f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:43.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.958+0000 7f98d3fff700 1 -- 172.21.15.7:0/309652934 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d40aed40 con 0x7f98d40a7fe0 2024-01-27T02:38:43.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.958+0000 7f98d3fff700 1 --2- 172.21.15.7:0/309652934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d40a7fe0 0x7f98d40b0180 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f98d40ac950 tx=0x7f98cc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ee36f870fec02f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:43.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.958+0000 7f98d27fc700 1 -- 172.21.15.7:0/309652934 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc00e070 con 0x7f98d40a7fe0 2024-01-27T02:38:43.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.958+0000 7f98d27fc700 1 -- 172.21.15.7:0/309652934 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98cc00ba50 con 0x7f98d40a7fe0 2024-01-27T02:38:43.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.958+0000 7f98d27fc700 1 -- 172.21.15.7:0/309652934 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0092e0 con 0x7f98d40a7fe0 2024-01-27T02:38:43.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.959+0000 7f98d9e20700 1 -- 172.21.15.7:0/309652934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d40a7fe0 msgr2=0x7f98d40b0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:43.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.959+0000 7f98d9e20700 1 --2- 172.21.15.7:0/309652934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d40a7fe0 0x7f98d40b0180 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f98d40ac950 tx=0x7f98cc009840 comp rx=0 tx=0).stop 2024-01-27T02:38:43.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.959+0000 7f98d9e20700 1 -- 172.21.15.7:0/309652934 shutdown_connections 2024-01-27T02:38:43.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.959+0000 7f98d9e20700 1 --2- 172.21.15.7:0/309652934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d40a7fe0 0x7f98d40b0180 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:43.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.959+0000 7f98d9e20700 1 --2- 172.21.15.7:0/309652934 >> v2:172.21.15.188:3300/0 conn(0x7f98d40a7c10 0x7f98d40ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:43.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.959+0000 7f98d9e20700 1 -- 172.21.15.7:0/309652934 >> 172.21.15.7:0/309652934 conn(0x7f98d40679b0 msgr2=0x7f98d4067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:43.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.959+0000 7f98d9e20700 1 -- 172.21.15.7:0/309652934 shutdown_connections 2024-01-27T02:38:43.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.959+0000 7f98d9e20700 1 -- 172.21.15.7:0/309652934 wait complete. 2024-01-27T02:38:43.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.961+0000 7f98d9e20700 1 Processor -- start 2024-01-27T02:38:43.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.961+0000 7f98d9e20700 1 -- start start 2024-01-27T02:38:43.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.962+0000 7f98d9e20700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98d40a7c10 0x7f98d414eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:43.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.962+0000 7f98d9e20700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d40a7fe0 0x7f98d4152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:43.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.962+0000 7f98d9e20700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d40ae880 0x7f98d40985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:43.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.962+0000 7f98d9e20700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d4098130 con 0x7f98d40a7fe0 2024-01-27T02:38:43.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.962+0000 7f98d9e20700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d414f1e0 con 0x7f98d40a7c10 2024-01-27T02:38:43.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.962+0000 7f98d9e20700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d4153360 con 0x7f98d40ae880 2024-01-27T02:38:43.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.962+0000 7f98d2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d40a7fe0 0x7f98d4152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:43.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.962+0000 7f98d2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d40a7fe0 0x7f98d4152830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37466/0 (socket says 172.21.15.7:37466) 2024-01-27T02:38:43.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.962+0000 7f98d2ffd700 1 -- 172.21.15.7:0/3306945686 learned_addr learned my addr 172.21.15.7:0/3306945686 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:43.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.962+0000 7f98d3fff700 1 --2- 172.21.15.7:0/3306945686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d40ae880 0x7f98d40985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:43.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.962+0000 7f98d37fe700 1 --2- 172.21.15.7:0/3306945686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98d40a7c10 0x7f98d414eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:43.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.963+0000 7f98d2ffd700 1 -- 172.21.15.7:0/3306945686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d40ae880 msgr2=0x7f98d40985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:43.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.963+0000 7f98d2ffd700 1 --2- 172.21.15.7:0/3306945686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d40ae880 0x7f98d40985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:43.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.963+0000 7f98d2ffd700 1 -- 172.21.15.7:0/3306945686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98d40a7c10 msgr2=0x7f98d414eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:43.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.963+0000 7f98d2ffd700 1 --2- 172.21.15.7:0/3306945686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98d40a7c10 0x7f98d414eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:43.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.963+0000 7f98d2ffd700 1 -- 172.21.15.7:0/3306945686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc009ae0 con 0x7f98d40a7fe0 2024-01-27T02:38:43.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.963+0000 7f98d2ffd700 1 --2- 172.21.15.7:0/3306945686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d40a7fe0 0x7f98d4152830 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f98c800c590 tx=0x7f98c8007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:43.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.965+0000 7f98bb7fe700 1 -- 172.21.15.7:0/3306945686 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c8011590 con 0x7f98d40a7fe0 2024-01-27T02:38:43.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.965+0000 7f98bb7fe700 1 -- 172.21.15.7:0/3306945686 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c8011710 con 0x7f98d40a7fe0 2024-01-27T02:38:43.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.965+0000 7f98bb7fe700 1 -- 172.21.15.7:0/3306945686 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c8011ad0 con 0x7f98d40a7fe0 2024-01-27T02:38:43.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.965+0000 7f98d9e20700 1 -- 172.21.15.7:0/3306945686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d4153080 con 0x7f98d40a7fe0 2024-01-27T02:38:43.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.965+0000 7f98d9e20700 1 -- 172.21.15.7:0/3306945686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d4097dc0 con 0x7f98d40a7fe0 2024-01-27T02:38:43.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.966+0000 7f98d9e20700 1 -- 172.21.15.7:0/3306945686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d404c710 con 0x7f98d40a7fe0 2024-01-27T02:38:43.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.967+0000 7f98bb7fe700 1 -- 172.21.15.7:0/3306945686 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f98c801a360 con 0x7f98d40a7fe0 2024-01-27T02:38:43.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.967+0000 7f98bb7fe700 1 --2- 172.21.15.7:0/3306945686 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f98c0066f30 0x7f98c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:43.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.967+0000 7f98bb7fe700 1 -- 172.21.15.7:0/3306945686 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f98c8083360 con 0x7f98d40a7fe0 2024-01-27T02:38:43.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.968+0000 7f98d37fe700 1 --2- 172.21.15.7:0/3306945686 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f98c0066f30 0x7f98c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:43.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.968+0000 7f98d37fe700 1 --2- 172.21.15.7:0/3306945686 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f98c0066f30 0x7f98c00693e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f98c4006fd0 tx=0x7f98c4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:43.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:43.972+0000 7f98bb7fe700 1 -- 172.21.15.7:0/3306945686 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98c80505d0 con 0x7f98d40a7fe0 2024-01-27T02:38:44.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:44.220+0000 7f98d9e20700 1 -- 172.21.15.7:0/3306945686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f98d40af020 con 0x7f98d40a7fe0 2024-01-27T02:38:44.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:44.220+0000 7f98bb7fe700 1 -- 172.21.15.7:0/3306945686 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v68) v1 ==== 53+0+73 (secure 0 0 0) 0x7f98c8053bf0 con 0x7f98d40a7fe0 2024-01-27T02:38:44.222 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:38:44.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:44.222+0000 7f98d9e20700 1 -- 172.21.15.7:0/3306945686 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f98c0066f30 msgr2=0x7f98c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:44.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:44.222+0000 7f98d9e20700 1 --2- 172.21.15.7:0/3306945686 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f98c0066f30 0x7f98c00693e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f98c4006fd0 tx=0x7f98c4008040 comp rx=0 tx=0).stop 2024-01-27T02:38:44.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:44.222+0000 7f98d9e20700 1 -- 172.21.15.7:0/3306945686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d40a7fe0 msgr2=0x7f98d4152830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:44.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:44.222+0000 7f98d9e20700 1 --2- 172.21.15.7:0/3306945686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d40a7fe0 0x7f98d4152830 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f98c800c590 tx=0x7f98c8007f50 comp rx=0 tx=0).stop 2024-01-27T02:38:44.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:44.223+0000 7f98d9e20700 1 -- 172.21.15.7:0/3306945686 shutdown_connections 2024-01-27T02:38:44.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:44.223+0000 7f98d9e20700 1 --2- 172.21.15.7:0/3306945686 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f98c0066f30 0x7f98c00693e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:44.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:44.223+0000 7f98d9e20700 1 --2- 172.21.15.7:0/3306945686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d40ae880 0x7f98d40985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:44.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:44.223+0000 7f98d9e20700 1 --2- 172.21.15.7:0/3306945686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98d40a7c10 0x7f98d414eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:44.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:44.223+0000 7f98d9e20700 1 --2- 172.21.15.7:0/3306945686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d40a7fe0 0x7f98d4152830 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:44.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:44.223+0000 7f98d9e20700 1 -- 172.21.15.7:0/3306945686 >> 172.21.15.7:0/3306945686 conn(0x7f98d40679b0 msgr2=0x7f98d406b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:44.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:44.223+0000 7f98d9e20700 1 -- 172.21.15.7:0/3306945686 shutdown_connections 2024-01-27T02:38:44.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:44.223+0000 7f98d9e20700 1 -- 172.21.15.7:0/3306945686 wait complete. 2024-01-27T02:38:44.238 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-27T02:38:44.238 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:38:44.238 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-01-27T02:38:44.238 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-27T02:38:44.238 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-27T02:38:44.238 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-27T02:38:44.239 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-27T02:38:44.239 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:38:44.239 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:38:44.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:44.239 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-01-27T02:38:44.261 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:38:44.261 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:38:44.261 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:38:44.261 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:44.261 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-01-27T02:38:44.261 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:44.261 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:38:44.262 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:38:44.262 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-01-27T02:38:44.262 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:38:44.262 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:38:44.262 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-01-27T02:38:44.262 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:38:44.262 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:38:44.262 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:38:44.262 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:38:44.262 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-27T02:38:44.262 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 41995sec preferred_lft 41995sec 2024-01-27T02:38:44.262 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b1:d6f7:baf0:bc17/64 scope link noprefixroute 2024-01-27T02:38:44.262 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:44.262 INFO:teuthology.orchestra.run.smithi188.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:38:44.263 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b0:92:71:90:9d brd ff:ff:ff:ff:ff:ff 2024-01-27T02:38:44.263 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-27T02:38:44.263 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:44.263 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::10f8:32ff:feed:a752/64 scope link 2024-01-27T02:38:44.263 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:44.263 INFO:teuthology.orchestra.run.smithi188.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:38:44.263 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b0:92:71:90:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-27T02:38:44.263 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b8b0:92ff:fe71:909d/64 scope link 2024-01-27T02:38:44.263 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:44.263 INFO:teuthology.orchestra.run.smithi188.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:38:44.263 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ee:78:03:20:3a:a9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-27T02:38:44.263 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec78:3ff:fe20:3aa9/64 scope link 2024-01-27T02:38:44.263 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:38:44.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:44.264 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:38:44.331 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-01-27T02:38:44.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:44.333 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-01-27T02:38:44.394 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-01-27T02:38:44.394 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:38:44.394 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-27T02:38:44.394 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-27T02:38:44.394 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:38:44.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:44.395 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-27T02:38:44.395 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-27T02:38:44.395 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-27T02:38:44.395 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-27T02:38:44.395 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-27T02:38:44.395 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-27T02:38:44.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:44.506 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-01-27T02:38:44.523 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-27T02:38:44.524 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-27T02:38:44.524 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:38:44.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:44.524 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2024-01-27T02:38:44.592 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:38:44.592 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-27T02:38:44.649 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:38:44.650 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:38:44.650 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-27T02:38:44.707 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:38:44.707 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:38:44.707 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:38:44.789 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:38:44.789 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:38:44.789 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:38:44.797 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:38:44.797 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:38:44.797 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:38:44.797 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:38:44.797 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:38:44.815 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:38:44.815 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:38:44.816 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:38:44.816 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:38:44.816 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:38:44.816 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:38:44.816 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:38:44.816 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:38:44.817 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:38:44.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:38:44.818 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-27T02:38:44.885 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:38:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.024+0000 7fb6237fb700 1 Processor -- start 2024-01-27T02:38:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.025+0000 7fb6237fb700 1 -- start start 2024-01-27T02:38:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.026+0000 7fb6237fb700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb61c0a7fe0 0x7fb61c06f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.026+0000 7fb6237fb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb61c15dfb0 0x7fb61c06f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.026+0000 7fb6237fb700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb61c15a880 con 0x7fb61c0a7c10 2024-01-27T02:38:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.026+0000 7fb6237fb700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb61c0afde0 con 0x7fb61c15dfb0 2024-01-27T02:38:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.026+0000 7fb6237fb700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb61c0af950 con 0x7fb61c0a7fe0 2024-01-27T02:38:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.026+0000 7fb621d98700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb61c15dfb0 0x7fb61c06f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.026+0000 7fb621597700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb61c0a7c10 0x7fb61c0ae340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46600/0 (socket says 172.21.15.7:46600) 2024-01-27T02:38:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.026+0000 7fb621597700 1 -- 172.21.15.7:0/3397725429 learned_addr learned my addr 172.21.15.7:0/3397725429 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.026+0000 7fb620d96700 1 --2- 172.21.15.7:0/3397725429 >> v2:172.21.15.188:3300/0 conn(0x7fb61c0a7fe0 0x7fb61c06f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.026+0000 7fb613fff700 1 -- 172.21.15.7:0/3397725429 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4063125301 0 0) 0x7fb61c15a880 con 0x7fb61c0a7c10 2024-01-27T02:38:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.026+0000 7fb613fff700 1 -- 172.21.15.7:0/3397725429 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb608003570 con 0x7fb61c0a7c10 2024-01-27T02:38:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.026+0000 7fb621d98700 1 -- 172.21.15.7:0/3397725429 >> v2:172.21.15.188:3300/0 conn(0x7fb61c0a7fe0 msgr2=0x7fb61c06f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.026+0000 7fb621d98700 1 --2- 172.21.15.7:0/3397725429 >> v2:172.21.15.188:3300/0 conn(0x7fb61c0a7fe0 0x7fb61c06f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.026+0000 7fb621d98700 1 -- 172.21.15.7:0/3397725429 >> v1:172.21.15.7:6789/0 conn(0x7fb61c0a7c10 legacy=0x7fb61c0ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.026+0000 7fb621d98700 1 -- 172.21.15.7:0/3397725429 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c0aed40 con 0x7fb61c15dfb0 2024-01-27T02:38:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.027+0000 7fb621d98700 1 --2- 172.21.15.7:0/3397725429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb61c15dfb0 0x7fb61c06f8a0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb61c0ce460 tx=0x7fb614009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b23689d96d1931 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.027+0000 7fb613fff700 1 -- 172.21.15.7:0/3397725429 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61400e070 con 0x7fb61c15dfb0 2024-01-27T02:38:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.027+0000 7fb613fff700 1 -- 172.21.15.7:0/3397725429 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61400bab0 con 0x7fb61c15dfb0 2024-01-27T02:38:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.027+0000 7fb613fff700 1 -- 172.21.15.7:0/3397725429 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61400be70 con 0x7fb61c15dfb0 2024-01-27T02:38:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.027+0000 7fb6237fb700 1 -- 172.21.15.7:0/3397725429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb61c15dfb0 msgr2=0x7fb61c06f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.027+0000 7fb6237fb700 1 --2- 172.21.15.7:0/3397725429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb61c15dfb0 0x7fb61c06f8a0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb61c0ce460 tx=0x7fb614009780 comp rx=0 tx=0).stop 2024-01-27T02:38:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.028+0000 7fb6237fb700 1 -- 172.21.15.7:0/3397725429 shutdown_connections 2024-01-27T02:38:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.028+0000 7fb6237fb700 1 --2- 172.21.15.7:0/3397725429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb61c15dfb0 0x7fb61c06f8a0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.028+0000 7fb6237fb700 1 --2- 172.21.15.7:0/3397725429 >> v2:172.21.15.188:3300/0 conn(0x7fb61c0a7fe0 0x7fb61c06f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.028+0000 7fb6237fb700 1 -- 172.21.15.7:0/3397725429 >> 172.21.15.7:0/3397725429 conn(0x7fb61c0679b0 msgr2=0x7fb61c067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.028+0000 7fb6237fb700 1 -- 172.21.15.7:0/3397725429 shutdown_connections 2024-01-27T02:38:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.028+0000 7fb6237fb700 1 -- 172.21.15.7:0/3397725429 wait complete. 2024-01-27T02:38:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.029+0000 7fb6237fb700 1 Processor -- start 2024-01-27T02:38:45.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.029+0000 7fb6237fb700 1 -- start start 2024-01-27T02:38:45.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.030+0000 7fb6237fb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb61c0a7c10 0x7fb61c097a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:45.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.030+0000 7fb6237fb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb61c0a7fe0 0x7fb61c157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:45.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.030+0000 7fb6237fb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb61c15dfb0 0x7fb61c0cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:45.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.030+0000 7fb6237fb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c16a380 con 0x7fb61c0a7c10 2024-01-27T02:38:45.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.030+0000 7fb6237fb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c16a660 con 0x7fb61c15dfb0 2024-01-27T02:38:45.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.030+0000 7fb6237fb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c16a940 con 0x7fb61c0a7fe0 2024-01-27T02:38:45.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.030+0000 7fb621597700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb61c0a7c10 0x7fb61c097a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:45.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.030+0000 7fb621597700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb61c0a7c10 0x7fb61c097a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37482/0 (socket says 172.21.15.7:37482) 2024-01-27T02:38:45.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.030+0000 7fb621597700 1 -- 172.21.15.7:0/4243793973 learned_addr learned my addr 172.21.15.7:0/4243793973 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:45.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.030+0000 7fb621d98700 1 --2- 172.21.15.7:0/4243793973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb61c15dfb0 0x7fb61c0cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:45.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.030+0000 7fb620d96700 1 --2- 172.21.15.7:0/4243793973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb61c0a7fe0 0x7fb61c157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:45.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.030+0000 7fb621597700 1 -- 172.21.15.7:0/4243793973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb61c0a7fe0 msgr2=0x7fb61c157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:45.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.030+0000 7fb621597700 1 --2- 172.21.15.7:0/4243793973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb61c0a7fe0 0x7fb61c157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:45.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.030+0000 7fb621597700 1 -- 172.21.15.7:0/4243793973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb61c15dfb0 msgr2=0x7fb61c0cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:45.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.030+0000 7fb621597700 1 --2- 172.21.15.7:0/4243793973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb61c15dfb0 0x7fb61c0cc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:45.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.030+0000 7fb621597700 1 -- 172.21.15.7:0/4243793973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb614009a20 con 0x7fb61c0a7c10 2024-01-27T02:38:45.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.030+0000 7fb621d98700 1 --2- 172.21.15.7:0/4243793973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb61c15dfb0 0x7fb61c0cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:38:45.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.030+0000 7fb620d96700 1 --2- 172.21.15.7:0/4243793973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb61c0a7fe0 0x7fb61c157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:38:45.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.031+0000 7fb621597700 1 --2- 172.21.15.7:0/4243793973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb61c0a7c10 0x7fb61c097a40 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb61800c7c0 tx=0x7fb618007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:45.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.031+0000 7fb6117fa700 1 -- 172.21.15.7:0/4243793973 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb618005b30 con 0x7fb61c0a7c10 2024-01-27T02:38:45.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.031+0000 7fb6117fa700 1 -- 172.21.15.7:0/4243793973 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb618005c90 con 0x7fb61c0a7c10 2024-01-27T02:38:45.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.031+0000 7fb6237fb700 1 -- 172.21.15.7:0/4243793973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61c16ac20 con 0x7fb61c0a7c10 2024-01-27T02:38:45.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.031+0000 7fb6117fa700 1 -- 172.21.15.7:0/4243793973 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb618007340 con 0x7fb61c0a7c10 2024-01-27T02:38:45.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.031+0000 7fb6237fb700 1 -- 172.21.15.7:0/4243793973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb61c169f30 con 0x7fb61c0a7c10 2024-01-27T02:38:45.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.032+0000 7fb6237fb700 1 -- 172.21.15.7:0/4243793973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb61c16d760 con 0x7fb61c0a7c10 2024-01-27T02:38:45.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.033+0000 7fb6117fa700 1 -- 172.21.15.7:0/4243793973 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb618025430 con 0x7fb61c0a7c10 2024-01-27T02:38:45.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.033+0000 7fb6117fa700 1 --2- 172.21.15.7:0/4243793973 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb600066f30 0x7fb6000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:45.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.034+0000 7fb6117fa700 1 -- 172.21.15.7:0/4243793973 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb618084af0 con 0x7fb61c0a7c10 2024-01-27T02:38:45.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.034+0000 7fb620d96700 1 --2- 172.21.15.7:0/4243793973 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb600066f30 0x7fb6000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:45.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.034+0000 7fb620d96700 1 --2- 172.21.15.7:0/4243793973 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb600066f30 0x7fb6000693e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb60c006f00 tx=0x7fb60c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:45.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.038+0000 7fb6117fa700 1 -- 172.21.15.7:0/4243793973 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb618051ef0 con 0x7fb61c0a7c10 2024-01-27T02:38:45.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.287+0000 7fb6237fb700 1 -- 172.21.15.7:0/4243793973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb61c04c710 con 0x7fb61c0a7c10 2024-01-27T02:38:45.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.288+0000 7fb6117fa700 1 -- 172.21.15.7:0/4243793973 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fb618055510 con 0x7fb61c0a7c10 2024-01-27T02:38:45.289 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:45.289 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3997828193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3997828193},{"type":"v1","addr":"172.21.15.188:6837","nonce":3997828193}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1132338302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1132338302},{"type":"v1","addr":"172.21.15.188:6836","nonce":1132338302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/740290645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":740290645},{"type":"v1","addr":"172.21.15.7:6833","nonce":740290645}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:38:31.328277+0000","modified":"2024-01-27T02:38:34.940811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7029":{"gid":7029,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1258751919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1258751919},{"type":"v1","addr":"172.21.15.7:6835","nonce":1258751919}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-27T02:38:45.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.290+0000 7fb6237fb700 1 -- 172.21.15.7:0/4243793973 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb600066f30 msgr2=0x7fb6000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:45.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.290+0000 7fb6237fb700 1 --2- 172.21.15.7:0/4243793973 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb600066f30 0x7fb6000693e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb60c006f00 tx=0x7fb60c008040 comp rx=0 tx=0).stop 2024-01-27T02:38:45.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.290+0000 7fb6237fb700 1 -- 172.21.15.7:0/4243793973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb61c0a7c10 msgr2=0x7fb61c097a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:45.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.290+0000 7fb6237fb700 1 --2- 172.21.15.7:0/4243793973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb61c0a7c10 0x7fb61c097a40 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb61800c7c0 tx=0x7fb618007e00 comp rx=0 tx=0).stop 2024-01-27T02:38:45.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.290+0000 7fb6237fb700 1 -- 172.21.15.7:0/4243793973 shutdown_connections 2024-01-27T02:38:45.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.290+0000 7fb6237fb700 1 --2- 172.21.15.7:0/4243793973 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb600066f30 0x7fb6000693e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:45.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.290+0000 7fb6237fb700 1 --2- 172.21.15.7:0/4243793973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb61c15dfb0 0x7fb61c0cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:45.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.290+0000 7fb6237fb700 1 --2- 172.21.15.7:0/4243793973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb61c0a7c10 0x7fb61c097a40 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:45.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.290+0000 7fb6237fb700 1 --2- 172.21.15.7:0/4243793973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb61c0a7fe0 0x7fb61c157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:45.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.290+0000 7fb6237fb700 1 -- 172.21.15.7:0/4243793973 >> 172.21.15.7:0/4243793973 conn(0x7fb61c0679b0 msgr2=0x7fb61c06af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:45.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.290+0000 7fb6237fb700 1 -- 172.21.15.7:0/4243793973 shutdown_connections 2024-01-27T02:38:45.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.290+0000 7fb6237fb700 1 -- 172.21.15.7:0/4243793973 wait complete. 2024-01-27T02:38:45.293 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 68 2024-01-27T02:38:45.305 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-01-27T02:38:45.459 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","whoami":0,"id":7029,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":13.527504985,"mdsmap_epoch":68,"osdmap_epoch":132,"osdmap_epoch_barrier":132,"uptime":18.935874754} 2024-01-27T02:38:45.460 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", "fs_name": "cephfs", "id": 7029, "mdsmap_epoch": 68, "osdmap_epoch": 132, "osdmap_epoch_barrier": 132, "rank_uptime": 13.527504985, "state": "up:active", "uptime": 18.935874754, "want_state": "up:active", "whoami": 0 } 2024-01-27T02:38:45.460 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-01-27T02:38:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.595+0000 7f60906ab700 1 Processor -- start 2024-01-27T02:38:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.595+0000 7f60906ab700 1 -- start start 2024-01-27T02:38:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.596+0000 7f60906ab700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f60880cc1d0 0x7f60880afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.596+0000 7f60906ab700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60880cf1f0 con 0x7f60880b0010 2024-01-27T02:38:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.596+0000 7f60906ab700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60880b0500 con 0x7f60880cc5a0 2024-01-27T02:38:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.596+0000 7f60906ab700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60880b07e0 con 0x7f60880cc1d0 2024-01-27T02:38:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.597+0000 7f608dc46700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f60880b0010 0x7f6088159cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46614/0 (socket says 172.21.15.7:46614) 2024-01-27T02:38:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.597+0000 7f608dc46700 1 -- 172.21.15.7:0/2356487038 learned_addr learned my addr 172.21.15.7:0/2356487038 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.597+0000 7f608e447700 1 --2- 172.21.15.7:0/2356487038 >> v2:172.21.15.188:3300/0 conn(0x7f60880cc1d0 0x7f60880afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.597+0000 7f608d445700 1 -- 172.21.15.7:0/2356487038 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4159401451 0 0) 0x7f60880cf1f0 con 0x7f60880b0010 2024-01-27T02:38:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.597+0000 7f608d445700 1 -- 172.21.15.7:0/2356487038 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6074003570 con 0x7f60880b0010 2024-01-27T02:38:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.597+0000 7f608e447700 1 -- 172.21.15.7:0/2356487038 >> v1:172.21.15.188:6789/0 conn(0x7f60880cc5a0 legacy=0x7f60880cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.597+0000 7f608e447700 1 -- 172.21.15.7:0/2356487038 >> v1:172.21.15.7:6789/0 conn(0x7f60880b0010 legacy=0x7f6088159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.597+0000 7f608e447700 1 -- 172.21.15.7:0/2356487038 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60880ceba0 con 0x7f60880cc1d0 2024-01-27T02:38:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.597+0000 7f608e447700 1 --2- 172.21.15.7:0/2356487038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60880cc1d0 0x7f60880afad0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f607c009910 tx=0x7f607c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee2b04a4dbff68 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:45.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.597+0000 7f608d445700 1 -- 172.21.15.7:0/2356487038 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c00b640 con 0x7f60880cc1d0 2024-01-27T02:38:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.598+0000 7f608d445700 1 -- 172.21.15.7:0/2356487038 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c00b7a0 con 0x7f60880cc1d0 2024-01-27T02:38:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.598+0000 7f608d445700 1 -- 172.21.15.7:0/2356487038 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c00bb60 con 0x7f60880cc1d0 2024-01-27T02:38:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.598+0000 7f60906ab700 1 -- 172.21.15.7:0/2356487038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60880cc1d0 msgr2=0x7f60880afad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.598+0000 7f60906ab700 1 --2- 172.21.15.7:0/2356487038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60880cc1d0 0x7f60880afad0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f607c009910 tx=0x7f607c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.598+0000 7f60906ab700 1 -- 172.21.15.7:0/2356487038 shutdown_connections 2024-01-27T02:38:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.598+0000 7f60906ab700 1 --2- 172.21.15.7:0/2356487038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60880cc1d0 0x7f60880afad0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.598+0000 7f60906ab700 1 -- 172.21.15.7:0/2356487038 >> 172.21.15.7:0/2356487038 conn(0x7f6088068480 msgr2=0x7f60880709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.598+0000 7f60906ab700 1 -- 172.21.15.7:0/2356487038 shutdown_connections 2024-01-27T02:38:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.598+0000 7f60906ab700 1 -- 172.21.15.7:0/2356487038 wait complete. 2024-01-27T02:38:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.600+0000 7f60906ab700 1 Processor -- start 2024-01-27T02:38:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.600+0000 7f60906ab700 1 -- start start 2024-01-27T02:38:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.601+0000 7f60906ab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60880b0010 0x7f608814dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.601+0000 7f60906ab700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60880cc1d0 0x7f60880c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.601+0000 7f60906ab700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60880cc5a0 0x7f60880c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.601+0000 7f60906ab700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60880c8890 con 0x7f60880b0010 2024-01-27T02:38:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.601+0000 7f60906ab700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6088159240 con 0x7f60880cc1d0 2024-01-27T02:38:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.601+0000 7f60906ab700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60881589a0 con 0x7f60880cc5a0 2024-01-27T02:38:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.601+0000 7f608e447700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60880b0010 0x7f608814dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.601+0000 7f608e447700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60880b0010 0x7f608814dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37498/0 (socket says 172.21.15.7:37498) 2024-01-27T02:38:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.601+0000 7f608e447700 1 -- 172.21.15.7:0/2710832336 learned_addr learned my addr 172.21.15.7:0/2710832336 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.601+0000 7f608dc46700 1 --2- 172.21.15.7:0/2710832336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60880cc1d0 0x7f60880c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.601+0000 7f608e447700 1 -- 172.21.15.7:0/2710832336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60880cc5a0 msgr2=0x7f60880c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.601+0000 7f608e447700 1 --2- 172.21.15.7:0/2710832336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60880cc5a0 0x7f60880c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.601+0000 7f608e447700 1 -- 172.21.15.7:0/2710832336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60880cc1d0 msgr2=0x7f60880c6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.601+0000 7f608e447700 1 --2- 172.21.15.7:0/2710832336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60880cc1d0 0x7f60880c6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.601+0000 7f608e447700 1 -- 172.21.15.7:0/2710832336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c009420 con 0x7f60880b0010 2024-01-27T02:38:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.602+0000 7f608e447700 1 --2- 172.21.15.7:0/2710832336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60880b0010 0x7f608814dcc0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f607c0093f0 tx=0x7f607c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.602+0000 7f607a7fc700 1 -- 172.21.15.7:0/2710832336 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c01f070 con 0x7f60880b0010 2024-01-27T02:38:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.602+0000 7f607a7fc700 1 -- 172.21.15.7:0/2710832336 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c00bb20 con 0x7f60880b0010 2024-01-27T02:38:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.602+0000 7f60906ab700 1 -- 172.21.15.7:0/2710832336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6088158c80 con 0x7f60880b0010 2024-01-27T02:38:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.602+0000 7f607a7fc700 1 -- 172.21.15.7:0/2710832336 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c011670 con 0x7f60880b0010 2024-01-27T02:38:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.602+0000 7f60906ab700 1 -- 172.21.15.7:0/2710832336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608814e200 con 0x7f60880b0010 2024-01-27T02:38:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.603+0000 7f60906ab700 1 -- 172.21.15.7:0/2710832336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6088089070 con 0x7f60880b0010 2024-01-27T02:38:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.603+0000 7f607a7fc700 1 -- 172.21.15.7:0/2710832336 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f607c0119e0 con 0x7f60880b0010 2024-01-27T02:38:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.604+0000 7f607a7fc700 1 --2- 172.21.15.7:0/2710832336 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f606c066f30 0x7f606c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.604+0000 7f607a7fc700 1 -- 172.21.15.7:0/2710832336 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f607c085870 con 0x7f60880b0010 2024-01-27T02:38:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.604+0000 7f608dc46700 1 --2- 172.21.15.7:0/2710832336 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f606c066f30 0x7f606c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.604+0000 7f608dc46700 1 --2- 172.21.15.7:0/2710832336 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f606c066f30 0x7f606c0693e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f60840058d0 tx=0x7f6084009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:45.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.608+0000 7f607a7fc700 1 -- 172.21.15.7:0/2710832336 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f607c052b20 con 0x7f60880b0010 2024-01-27T02:38:45.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.813+0000 7f60906ab700 1 -- 172.21.15.7:0/2710832336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f608804c710 con 0x7f60880b0010 2024-01-27T02:38:45.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.973+0000 7f607a7fc700 1 -- 172.21.15.7:0/2710832336 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v133) v1 ==== 181+0+0 (secure 0 0 0) 0x7f607c015020 con 0x7f60880b0010 2024-01-27T02:38:45.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.975+0000 7f60906ab700 1 -- 172.21.15.7:0/2710832336 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f606c066f30 msgr2=0x7f606c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:45.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.975+0000 7f60906ab700 1 --2- 172.21.15.7:0/2710832336 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f606c066f30 0x7f606c0693e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f60840058d0 tx=0x7f6084009040 comp rx=0 tx=0).stop 2024-01-27T02:38:45.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.975+0000 7f60906ab700 1 -- 172.21.15.7:0/2710832336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60880b0010 msgr2=0x7f608814dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:45.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.975+0000 7f60906ab700 1 --2- 172.21.15.7:0/2710832336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60880b0010 0x7f608814dcc0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f607c0093f0 tx=0x7f607c017000 comp rx=0 tx=0).stop 2024-01-27T02:38:45.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.975+0000 7f60906ab700 1 -- 172.21.15.7:0/2710832336 shutdown_connections 2024-01-27T02:38:45.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.975+0000 7f60906ab700 1 --2- 172.21.15.7:0/2710832336 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f606c066f30 0x7f606c0693e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:45.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.975+0000 7f60906ab700 1 --2- 172.21.15.7:0/2710832336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60880cc5a0 0x7f60880c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:45.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.975+0000 7f60906ab700 1 --2- 172.21.15.7:0/2710832336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60880b0010 0x7f608814dcc0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:45.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.975+0000 7f60906ab700 1 --2- 172.21.15.7:0/2710832336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60880cc1d0 0x7f60880c6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:45.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.975+0000 7f60906ab700 1 -- 172.21.15.7:0/2710832336 >> 172.21.15.7:0/2710832336 conn(0x7f6088068480 msgr2=0x7f60880b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:45.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.975+0000 7f60906ab700 1 -- 172.21.15.7:0/2710832336 shutdown_connections 2024-01-27T02:38:45.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:45.975+0000 7f60906ab700 1 -- 172.21.15.7:0/2710832336 wait complete. 2024-01-27T02:38:45.978 INFO:teuthology.orchestra.run.smithi007.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-01-27T02:38:45.992 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-01-27T02:38:46.143 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_mon_report_interval":"5"} 2024-01-27T02:38:46.143 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-01-27T02:38:46.143 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2024-01-27T02:38:46.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:38:46.143 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2024-01-27T02:38:46.266 INFO:teuthology.orchestra.run.smithi007.stderr:16+0 records in 2024-01-27T02:38:46.266 INFO:teuthology.orchestra.run.smithi007.stderr:16+0 records out 2024-01-27T02:38:46.266 INFO:teuthology.orchestra.run.smithi007.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.100009 s, 168 MB/s 2024-01-27T02:38:46.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:38:46.267 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2024-01-27T02:38:46.405 INFO:teuthology.orchestra.run.smithi007.stderr:17+0 records in 2024-01-27T02:38:46.405 INFO:teuthology.orchestra.run.smithi007.stderr:17+0 records out 2024-01-27T02:38:46.405 INFO:teuthology.orchestra.run.smithi007.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.115975 s, 154 MB/s 2024-01-27T02:38:46.406 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-01-27T02:38:46.406 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:38:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.537+0000 7f7eeac64700 1 Processor -- start 2024-01-27T02:38:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.537+0000 7f7eeac64700 1 -- start start 2024-01-27T02:38:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.538+0000 7f7eeac64700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ee40b1c00 0x7f7ee406e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.538+0000 7f7eeac64700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee4157610 con 0x7f7ee40b1fd0 2024-01-27T02:38:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.538+0000 7f7eeac64700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee406ee00 con 0x7f7ee415b5f0 2024-01-27T02:38:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.538+0000 7f7eeac64700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee406f0e0 con 0x7f7ee40b1c00 2024-01-27T02:38:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.538+0000 7f7ee8a00700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ee40b1c00 0x7f7ee406e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.538+0000 7f7ee8a00700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ee40b1c00 0x7f7ee406e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37510/0 (socket says 172.21.15.7:37510) 2024-01-27T02:38:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.538+0000 7f7ee8a00700 1 -- 172.21.15.7:0/1250671394 learned_addr learned my addr 172.21.15.7:0/1250671394 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.539+0000 7f7ee8a00700 1 -- 172.21.15.7:0/1250671394 >> v1:172.21.15.188:6789/0 conn(0x7f7ee415b5f0 legacy=0x7f7ee41566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.539+0000 7f7ee8a00700 1 -- 172.21.15.7:0/1250671394 >> v1:172.21.15.7:6789/0 conn(0x7f7ee40b1fd0 legacy=0x7f7ee4156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.539+0000 7f7ee8a00700 1 -- 172.21.15.7:0/1250671394 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee406f3c0 con 0x7f7ee40b1c00 2024-01-27T02:38:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.539+0000 7f7ee8a00700 1 --2- 172.21.15.7:0/1250671394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee40b1c00 0x7f7ee406e540 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80099b0 tx=0x7f7ed8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f381c2d3cd6542 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.539+0000 7f7ee37fe700 1 -- 172.21.15.7:0/1250671394 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed800b860 con 0x7f7ee40b1c00 2024-01-27T02:38:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.539+0000 7f7ee37fe700 1 -- 172.21.15.7:0/1250671394 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed800b9c0 con 0x7f7ee40b1c00 2024-01-27T02:38:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.539+0000 7f7ee37fe700 1 -- 172.21.15.7:0/1250671394 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed800bd60 con 0x7f7ee40b1c00 2024-01-27T02:38:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.540+0000 7f7eeac64700 1 -- 172.21.15.7:0/1250671394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee40b1c00 msgr2=0x7f7ee406e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.540+0000 7f7eeac64700 1 --2- 172.21.15.7:0/1250671394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee40b1c00 0x7f7ee406e540 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80099b0 tx=0x7f7ed8009250 comp rx=0 tx=0).stop 2024-01-27T02:38:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.540+0000 7f7eeac64700 1 -- 172.21.15.7:0/1250671394 shutdown_connections 2024-01-27T02:38:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.540+0000 7f7eeac64700 1 --2- 172.21.15.7:0/1250671394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee40b1c00 0x7f7ee406e540 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.540+0000 7f7eeac64700 1 -- 172.21.15.7:0/1250671394 >> 172.21.15.7:0/1250671394 conn(0x7f7ee4068430 msgr2=0x7f7ee406be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.540+0000 7f7eeac64700 1 -- 172.21.15.7:0/1250671394 shutdown_connections 2024-01-27T02:38:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.540+0000 7f7eeac64700 1 -- 172.21.15.7:0/1250671394 wait complete. 2024-01-27T02:38:46.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.541+0000 7f7eeac64700 1 Processor -- start 2024-01-27T02:38:46.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.541+0000 7f7eeac64700 1 -- start start 2024-01-27T02:38:46.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.542+0000 7f7eeac64700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ee40b1c00 0x7f7ee40637d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:46.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.542+0000 7f7eeac64700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ee40b1fd0 0x7f7ee4152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:46.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.542+0000 7f7eeac64700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee415b5f0 0x7f7ee4152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:46.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.542+0000 7f7eeac64700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee40aa860 con 0x7f7ee415b5f0 2024-01-27T02:38:46.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.542+0000 7f7eeac64700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee40aa3d0 con 0x7f7ee40b1c00 2024-01-27T02:38:46.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.542+0000 7f7eeac64700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee40a97c0 con 0x7f7ee40b1fd0 2024-01-27T02:38:46.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.542+0000 7f7ee9201700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee415b5f0 0x7f7ee4152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:46.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.542+0000 7f7ee9201700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee415b5f0 0x7f7ee4152ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37520/0 (socket says 172.21.15.7:37520) 2024-01-27T02:38:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.542+0000 7f7ee9201700 1 -- 172.21.15.7:0/2768968670 learned_addr learned my addr 172.21.15.7:0/2768968670 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.542+0000 7f7ee8a00700 1 --2- 172.21.15.7:0/2768968670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ee40b1c00 0x7f7ee40637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.542+0000 7f7ee3fff700 1 --2- 172.21.15.7:0/2768968670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ee40b1fd0 0x7f7ee4152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.542+0000 7f7ee9201700 1 -- 172.21.15.7:0/2768968670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ee40b1fd0 msgr2=0x7f7ee4152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.543+0000 7f7ee9201700 1 --2- 172.21.15.7:0/2768968670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ee40b1fd0 0x7f7ee4152660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.543+0000 7f7ee9201700 1 -- 172.21.15.7:0/2768968670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ee40b1c00 msgr2=0x7f7ee40637d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.543+0000 7f7ee9201700 1 --2- 172.21.15.7:0/2768968670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ee40b1c00 0x7f7ee40637d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.543+0000 7f7ee9201700 1 -- 172.21.15.7:0/2768968670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed80094f0 con 0x7f7ee415b5f0 2024-01-27T02:38:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.543+0000 7f7ee9201700 1 --2- 172.21.15.7:0/2768968670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee415b5f0 0x7f7ee4152ba0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00c8e0 tx=0x7f7edc001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.543+0000 7f7ee0ff9700 1 -- 172.21.15.7:0/2768968670 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc012630 con 0x7f7ee415b5f0 2024-01-27T02:38:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.543+0000 7f7ee0ff9700 1 -- 172.21.15.7:0/2768968670 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7edc012790 con 0x7f7ee415b5f0 2024-01-27T02:38:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.543+0000 7f7eeac64700 1 -- 172.21.15.7:0/2768968670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee40a9aa0 con 0x7f7ee415b5f0 2024-01-27T02:38:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.543+0000 7f7ee0ff9700 1 -- 172.21.15.7:0/2768968670 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc012b50 con 0x7f7ee415b5f0 2024-01-27T02:38:46.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.543+0000 7f7eeac64700 1 -- 172.21.15.7:0/2768968670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee40aa060 con 0x7f7ee415b5f0 2024-01-27T02:38:46.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.544+0000 7f7eeac64700 1 -- 172.21.15.7:0/2768968670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee404c710 con 0x7f7ee415b5f0 2024-01-27T02:38:46.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.544+0000 7f7ee0ff9700 1 -- 172.21.15.7:0/2768968670 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7edc01a3a0 con 0x7f7ee415b5f0 2024-01-27T02:38:46.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.545+0000 7f7ee0ff9700 1 --2- 172.21.15.7:0/2768968670 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ec8066f30 0x7f7ec80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:46.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.545+0000 7f7ee0ff9700 1 -- 172.21.15.7:0/2768968670 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7edc084740 con 0x7f7ee415b5f0 2024-01-27T02:38:46.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.545+0000 7f7ee8a00700 1 --2- 172.21.15.7:0/2768968670 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ec8066f30 0x7f7ec80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:46.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.546+0000 7f7ee8a00700 1 --2- 172.21.15.7:0/2768968670 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ec8066f30 0x7f7ec80693e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80099b0 tx=0x7f7ed8017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:46.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.550+0000 7f7ee0ff9700 1 -- 172.21.15.7:0/2768968670 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7edc0519b0 con 0x7f7ee415b5f0 2024-01-27T02:38:46.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.755+0000 7f7eeac64700 1 -- 172.21.15.7:0/2768968670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f7ee406ea80 con 0x7f7ee415b5f0 2024-01-27T02:38:46.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.756+0000 7f7ee0ff9700 1 -- 172.21.15.7:0/2768968670 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v133) v1 ==== 81+0+32234 (secure 0 0 0) 0x7f7edc054fd0 con 0x7f7ee415b5f0 2024-01-27T02:38:46.758 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:46.758 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:38:46.759 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 133, 2024-01-27T02:38:46.759 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:38:46.759 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:38:46.759 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:38:45.966836+0000", 2024-01-27T02:38:46.759 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:38:46.759 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:38:46.759 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:38:46.759 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:38:46.760 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:38:46.760 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:38:46.760 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:38:46.760 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:38:46.760 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:38:46.760 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:38:46.760 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:38:46.760 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:38:46.760 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:38:46.760 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:38:46.761 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:38:46.761 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 17, 2024-01-27T02:38:46.761 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:38:46.761 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:38:46.761 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:38:46.761 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:38:46.761 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:38:46.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.761 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:38:46.761 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:38:46.761 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:38:46.761 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:38:46.761 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:38:46.761 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:38:46.762 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:38:46.762 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:38:46.762 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:38:46.762 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:38:46.762 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:38:46.762 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:38:46.762 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:38:46.762 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:38:46.762 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:38:46.762 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:38:46.762 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:38:46.762 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:38:46.762 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:38:46.762 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:38:46.762 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:38:46.763 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:38:46.763 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:38:46.763 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:38:46.763 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:38:46.763 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:38:46.763 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:38:46.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.763 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "106", 2024-01-27T02:38:46.763 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:38:46.763 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:38:46.763 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:38:46.763 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:38:46.763 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:38:46.763 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:38:46.764 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:38:46.764 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:38:46.764 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:38:46.764 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:38:46.764 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:38:46.764 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:38:46.764 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:38:46.764 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:38:46.764 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:38:46.764 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:38:46.764 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:38:46.764 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:38:46.764 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:38:46.764 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:38:46.765 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:38:46.765 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:38:46.765 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:38:46.765 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:38:46.765 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:38:46.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:38:46.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.765 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:38:46.765 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:38:46.765 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:38:46.765 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:38:46.765 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:38:46.765 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:38:46.765 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:38:46.765 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:38:46.766 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:38:46.766 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:38:46.766 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:38:46.766 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:38:46.766 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:38:46.766 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:38:46.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.766 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:38:46.766 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:38:46.766 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.766 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 16, 2024-01-27T02:38:46.766 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:38:46.766 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:38:29.824892+0000", 2024-01-27T02:38:46.766 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:38:46.767 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:38:46.767 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:38:46.767 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:38:46.767 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:38:46.767 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:38:46.767 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:38:46.767 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:38:46.767 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:38:46.767 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:38:46.767 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:38:46.767 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:38:46.767 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:38:46.767 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:38:46.767 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:38:46.767 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:38:46.768 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:38:46.768 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:38:46.768 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:38:46.768 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:38:46.768 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:38:46.768 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:38:46.768 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:38:46.768 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:38:46.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.768 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "131", 2024-01-27T02:38:46.768 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:38:46.768 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:38:46.768 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:38:46.769 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:38:46.769 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:38:46.769 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:38:46.769 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:38:46.769 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:38:46.769 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:38:46.769 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:38:46.769 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:38:46.769 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:38:46.769 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:38:46.769 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:38:46.769 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:38:46.769 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:38:46.770 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:38:46.770 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:38:46.770 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:38:46.770 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:38:46.770 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:38:46.770 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:38:46.770 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:38:46.770 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:38:46.770 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:38:46.770 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:38:46.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.770 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:38:46.770 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:38:46.770 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:38:46.770 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:38:46.771 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:38:46.771 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:38:46.771 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:38:46.771 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:38:46.771 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:38:46.771 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:38:46.771 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:38:46.771 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:38:46.771 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:38:46.771 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:38:46.771 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:38:46.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.771 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:38:46.772 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:38:46.772 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:38:46.772 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.772 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.772 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 17, 2024-01-27T02:38:46.772 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:38:46.772 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:38:30.301072+0000", 2024-01-27T02:38:46.772 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:38:46.772 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:38:46.772 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:38:46.772 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:38:46.772 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:38:46.773 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:38:46.773 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:38:46.773 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:38:46.773 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:38:46.773 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:38:46.773 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:38:46.773 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:38:46.773 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:38:46.773 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:38:46.773 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:38:46.773 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:38:46.773 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:38:46.773 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:38:46.773 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:38:46.774 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:38:46.774 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:38:46.774 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:38:46.774 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:38:46.774 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:38:46.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.774 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "133", 2024-01-27T02:38:46.774 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:38:46.774 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:38:46.774 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:38:46.774 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:38:46.774 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:38:46.774 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:38:46.774 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:38:46.775 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:38:46.775 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:38:46.775 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-27T02:38:46.775 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:38:46.775 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:38:46.775 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:38:46.775 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:38:46.775 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:38:46.775 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:38:46.775 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:38:46.775 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:38:46.775 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:38:46.775 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:38:46.775 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:38:46.775 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:38:46.776 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:38:46.776 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:38:46.776 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:38:46.776 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:38:46.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.776 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:38:46.776 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:38:46.776 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:38:46.776 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:38:46.776 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:38:46.776 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:38:46.776 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:38:46.776 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:38:46.777 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:38:46.777 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:38:46.777 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:38:46.777 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:38:46.777 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:38:46.777 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:38:46.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.777 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:38:46.777 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:38:46.777 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:38:46.777 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.777 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.777 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.777 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:38:46.778 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:38:46.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.778 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:38:46.778 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:46.778 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:46.778 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:46.778 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:46.778 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:46.778 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:46.778 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:46.778 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:46.778 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:46.778 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:46.778 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:46.779 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:46.779 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.779 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.779 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:38:46.779 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:46.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.779 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.779 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:38:46.779 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:46.779 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.779 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.779 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:46.780 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.780 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.780 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:38:46.780 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:46.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.780 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.780 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:38:46.780 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:46.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.780 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.781 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:46.781 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.781 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.781 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:38:46.781 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:46.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.781 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.781 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:38:46.781 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:46.781 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.781 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.781 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:46.782 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.782 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.782 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:38:46.782 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:46.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.782 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.782 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:38:46.782 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:46.782 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.782 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.782 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:38:46.783 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:38:46.783 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:38:46.783 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:38:46.783 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:46.783 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:46.783 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:46.783 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.783 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:38:46.783 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:46.783 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:46.783 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:46.783 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:46.784 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:46.784 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:46.784 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:46.784 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:46.784 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:46.784 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:46.784 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:46.784 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:46.784 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.784 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.784 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:38:46.784 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:46.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.785 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.785 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:38:46.785 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:46.785 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.785 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.785 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:46.785 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.785 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.785 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:38:46.785 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:46.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.786 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.786 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:38:46.786 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:46.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.786 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.786 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:46.786 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.786 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.786 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:38:46.786 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:46.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.786 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.787 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:38:46.787 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:46.787 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.787 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.787 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:46.787 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.787 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:38:46.787 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:46.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.787 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:38:46.788 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:46.788 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.788 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.788 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:38:46.788 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:38:46.788 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:38:46.788 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:38:46.788 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:46.788 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:46.788 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:46.788 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.789 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:38:46.789 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:46.789 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:46.789 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:46.789 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:46.789 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:46.789 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:46.789 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:46.789 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:46.789 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:46.789 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:46.789 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:46.789 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:46.789 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.789 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.790 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:38:46.790 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:46.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.790 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.790 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:38:46.790 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:46.790 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.790 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.790 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:46.790 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.790 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.790 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:38:46.791 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:46.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.791 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:38:46.791 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:46.791 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.791 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.791 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:46.791 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.791 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:38:46.792 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:46.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.792 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.792 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:38:46.792 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:46.792 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.792 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.792 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:46.792 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.792 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.792 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:38:46.793 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:46.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.793 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.793 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:38:46.793 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:46.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.793 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.793 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:38:46.793 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:38:46.793 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:38:46.793 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:38:46.793 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:46.794 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:46.794 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:46.794 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.794 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:38:46.794 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:46.794 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:46.794 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:46.794 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:46.794 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:46.794 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:46.794 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:46.794 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:46.795 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:46.795 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:46.795 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:46.795 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:46.795 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.795 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.795 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:38:46.795 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:46.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.795 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.795 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:38:46.795 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:46.795 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.796 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.796 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:46.796 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.796 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.796 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:38:46.796 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:46.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.796 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.796 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:38:46.796 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:46.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.797 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.797 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:46.797 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.797 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:38:46.797 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:46.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.797 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:38:46.797 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:46.797 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.797 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.798 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:46.798 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.798 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.798 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:38:46.798 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:46.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.798 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.798 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:38:46.798 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:46.798 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.798 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.799 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:38:46.799 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:38:46.799 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:38:46.799 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:38:46.799 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:46.799 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:46.799 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:46.799 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.799 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:38:46.799 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:46.799 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:46.799 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:46.800 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:46.800 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:46.800 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:46.800 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:46.800 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:46.800 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:46.800 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:46.800 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:46.800 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:46.800 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.800 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:38:46.800 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:46.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.801 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:38:46.801 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:46.801 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.801 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.801 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:46.801 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.801 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:38:46.801 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:46.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.802 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:38:46.802 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:46.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.802 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.802 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:46.802 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.802 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:38:46.802 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:46.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.803 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:38:46.803 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:46.803 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.803 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.803 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:46.803 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.803 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:38:46.803 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:46.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.804 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:38:46.804 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:46.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.804 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.804 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:38:46.804 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:38:46.804 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:38:46.804 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:38:46.804 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:46.804 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:46.804 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:46.804 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.805 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:38:46.805 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:46.805 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:46.805 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:46.805 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:46.805 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:46.805 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:46.805 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:46.805 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:46.805 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:46.805 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:46.806 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:46.806 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:46.806 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.806 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:38:46.806 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:46.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.806 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:38:46.806 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:46.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.806 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.807 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:46.807 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.807 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.807 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:38:46.807 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:46.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.807 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.807 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:38:46.807 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:46.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.807 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.808 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:46.808 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.808 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:38:46.808 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:46.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.808 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:38:46.808 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:46.808 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.808 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.809 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:46.809 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.809 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:38:46.809 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:46.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.809 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:38:46.809 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:46.809 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.809 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.809 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:38:46.810 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:38:46.810 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:38:46.810 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:38:46.810 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:46.810 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:46.810 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:46.810 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.810 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:38:46.810 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:46.810 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:46.810 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:46.810 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:46.810 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:46.811 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:46.811 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:46.811 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:46.811 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:46.811 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:46.811 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:46.811 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:46.811 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.811 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:38:46.811 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:46.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.812 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:38:46.812 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:46.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.812 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.812 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:46.812 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.812 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:38:46.812 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:46.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.813 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.813 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:38:46.813 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:46.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.813 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.813 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:46.813 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.813 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.813 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:38:46.813 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:46.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.814 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:38:46.814 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:46.814 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.814 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.814 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:46.814 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.814 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:38:46.814 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:46.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:38:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:38:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:38:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:38:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:38:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.816 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:38:46.816 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:46.816 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:46.816 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:46.816 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:46.816 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:46.816 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:46.816 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:46.816 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:46.816 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:46.816 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:46.816 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:46.816 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:46.816 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.817 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:38:46.817 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:46.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.817 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:38:46.817 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:46.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.817 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.817 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:46.817 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.818 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.818 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:38:46.818 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:46.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.818 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.818 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:38:46.818 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:46.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.818 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.818 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:46.818 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.819 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.819 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:38:46.819 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:46.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.819 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.819 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:38:46.819 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:46.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.819 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.819 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:46.819 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:46.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.819 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:46.820 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:38:46.820 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:46.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:46.820 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:38:46.820 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:46.820 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.820 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.820 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:38:46.820 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:38:46.820 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:38:46.820 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:38:46.820 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:38:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:38:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:38:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:38:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:46.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.822 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:38:46.822 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:46.822 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:46.822 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:46.822 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:46.822 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:46.822 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:38:46.822 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:46.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.822 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:38:46.822 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:46.823 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:46.823 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:46.823 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:46.823 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:46.823 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:38:46.823 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:46.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.823 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:38:46.823 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:46.823 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:46.823 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:46.823 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:46.823 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:46.823 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:38:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:38:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:38:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:38:46.824 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:38:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:38:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:46.825 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:46.826 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:38:46.826 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:46.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:46.826 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:38:46.826 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:46.826 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:46.826 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:46.826 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:46.826 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:46.826 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:38:46.826 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:46.826 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.826 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:38:46.826 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:38:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:38:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:38:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:38:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:38:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:38:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:38:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:38:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:38:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:38:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:38:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:38:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:38:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:38:46.827 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:38:46.828 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:38:46.828 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:38:46.828 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:38:46.828 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:38:46.828 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:38:46.828 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:38:46.828 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:38:46.828 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:38:46.828 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:38:46.828 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:46.828 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:38:46.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.758+0000 7f7eeac64700 1 -- 172.21.15.7:0/2768968670 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ec8066f30 msgr2=0x7f7ec80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:46.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.758+0000 7f7eeac64700 1 --2- 172.21.15.7:0/2768968670 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ec8066f30 0x7f7ec80693e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80099b0 tx=0x7f7ed8017000 comp rx=0 tx=0).stop 2024-01-27T02:38:46.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.758+0000 7f7eeac64700 1 -- 172.21.15.7:0/2768968670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee415b5f0 msgr2=0x7f7ee4152ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:46.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.758+0000 7f7eeac64700 1 --2- 172.21.15.7:0/2768968670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee415b5f0 0x7f7ee4152ba0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00c8e0 tx=0x7f7edc001710 comp rx=0 tx=0).stop 2024-01-27T02:38:46.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.758+0000 7f7eeac64700 1 -- 172.21.15.7:0/2768968670 shutdown_connections 2024-01-27T02:38:46.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.758+0000 7f7eeac64700 1 --2- 172.21.15.7:0/2768968670 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7ec8066f30 0x7f7ec80693e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:46.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.758+0000 7f7eeac64700 1 --2- 172.21.15.7:0/2768968670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee415b5f0 0x7f7ee4152ba0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:46.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.758+0000 7f7eeac64700 1 --2- 172.21.15.7:0/2768968670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ee40b1c00 0x7f7ee40637d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:46.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.759+0000 7f7eeac64700 1 --2- 172.21.15.7:0/2768968670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ee40b1fd0 0x7f7ee4152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:46.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.759+0000 7f7eeac64700 1 -- 172.21.15.7:0/2768968670 >> 172.21.15.7:0/2768968670 conn(0x7f7ee4068430 msgr2=0x7f7ee41538a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:46.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.759+0000 7f7eeac64700 1 -- 172.21.15.7:0/2768968670 shutdown_connections 2024-01-27T02:38:46.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:46.759+0000 7f7eeac64700 1 -- 172.21.15.7:0/2768968670 wait complete. 2024-01-27T02:38:46.831 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-27T02:38:51.832 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:38:51.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.970+0000 7f0d26aa4700 1 Processor -- start 2024-01-27T02:38:51.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.970+0000 7f0d26aa4700 1 -- start start 2024-01-27T02:38:51.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.971+0000 7f0d26aa4700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0d2015d770 0x7f0d200cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:51.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.971+0000 7f0d26aa4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d2006e5d0 con 0x7f0d2006ea60 2024-01-27T02:38:51.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.971+0000 7f0d26aa4700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d200ce860 con 0x7f0d2015d3a0 2024-01-27T02:38:51.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.971+0000 7f0d26aa4700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d200cbe10 con 0x7f0d2015d770 2024-01-27T02:38:51.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.971+0000 7f0d25041700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0d2006ea60 0x7f0d2015dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57042/0 (socket says 172.21.15.7:57042) 2024-01-27T02:38:51.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.971+0000 7f0d25041700 1 -- 172.21.15.7:0/1735911823 learned_addr learned my addr 172.21.15.7:0/1735911823 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:51.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.971+0000 7f0d17fff700 1 --2- 172.21.15.7:0/1735911823 >> v2:172.21.15.188:3300/0 conn(0x7f0d2015d770 0x7f0d200cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:51.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.972+0000 7f0d177fe700 1 -- 172.21.15.7:0/1735911823 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1736883561 0 0) 0x7f0d2006e5d0 con 0x7f0d2006ea60 2024-01-27T02:38:51.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.972+0000 7f0d177fe700 1 -- 172.21.15.7:0/1735911823 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d08003570 con 0x7f0d2006ea60 2024-01-27T02:38:51.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.972+0000 7f0d177fe700 1 -- 172.21.15.7:0/1735911823 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4017015590 0 0) 0x7f0d200ce860 con 0x7f0d2015d3a0 2024-01-27T02:38:51.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.972+0000 7f0d177fe700 1 -- 172.21.15.7:0/1735911823 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d2006e5d0 con 0x7f0d2015d3a0 2024-01-27T02:38:51.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.972+0000 7f0d17fff700 1 -- 172.21.15.7:0/1735911823 >> v1:172.21.15.188:6789/0 conn(0x7f0d2015d3a0 legacy=0x7f0d200c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:51.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.972+0000 7f0d177fe700 1 -- 172.21.15.7:0/1735911823 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3333432696 0 0) 0x7f0d08003570 con 0x7f0d2006ea60 2024-01-27T02:38:51.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.972+0000 7f0d17fff700 1 -- 172.21.15.7:0/1735911823 >> v1:172.21.15.7:6789/0 conn(0x7f0d2006ea60 legacy=0x7f0d2015dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:51.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.972+0000 7f0d17fff700 1 -- 172.21.15.7:0/1735911823 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d200cd8f0 con 0x7f0d2015d770 2024-01-27T02:38:51.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.972+0000 7f0d17fff700 1 --2- 172.21.15.7:0/1735911823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d2015d770 0x7f0d200cd310 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0d200cdae0 tx=0x7f0d100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0d2e14946412270 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:51.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.972+0000 7f0d177fe700 1 -- 172.21.15.7:0/1735911823 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d180017f0 con 0x7f0d2015d770 2024-01-27T02:38:51.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.972+0000 7f0d177fe700 1 -- 172.21.15.7:0/1735911823 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d10011420 con 0x7f0d2015d770 2024-01-27T02:38:51.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.972+0000 7f0d177fe700 1 -- 172.21.15.7:0/1735911823 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d100117c0 con 0x7f0d2015d770 2024-01-27T02:38:51.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.973+0000 7f0d26aa4700 1 -- 172.21.15.7:0/1735911823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d2015d770 msgr2=0x7f0d200cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:51.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.973+0000 7f0d26aa4700 1 --2- 172.21.15.7:0/1735911823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d2015d770 0x7f0d200cd310 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0d200cdae0 tx=0x7f0d100091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:51.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.973+0000 7f0d26aa4700 1 -- 172.21.15.7:0/1735911823 shutdown_connections 2024-01-27T02:38:51.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.973+0000 7f0d26aa4700 1 --2- 172.21.15.7:0/1735911823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d2015d770 0x7f0d200cd310 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:51.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.973+0000 7f0d26aa4700 1 -- 172.21.15.7:0/1735911823 >> 172.21.15.7:0/1735911823 conn(0x7f0d20068480 msgr2=0x7f0d200cdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:51.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.973+0000 7f0d26aa4700 1 -- 172.21.15.7:0/1735911823 shutdown_connections 2024-01-27T02:38:51.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.973+0000 7f0d26aa4700 1 -- 172.21.15.7:0/1735911823 wait complete. 2024-01-27T02:38:51.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.974+0000 7f0d26aa4700 1 Processor -- start 2024-01-27T02:38:51.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.974+0000 7f0d26aa4700 1 -- start start 2024-01-27T02:38:51.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.975+0000 7f0d26aa4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2006ea60 0x7f0d2016de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:51.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.975+0000 7f0d26aa4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d2015d3a0 0x7f0d200af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:51.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.975+0000 7f0d26aa4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d2015d770 0x7f0d200b0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:51.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.975+0000 7f0d26aa4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d200ad4b0 con 0x7f0d2015d3a0 2024-01-27T02:38:51.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.975+0000 7f0d26aa4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2016e410 con 0x7f0d2015d770 2024-01-27T02:38:51.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.975+0000 7f0d26aa4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2016e6f0 con 0x7f0d2006ea60 2024-01-27T02:38:51.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.975+0000 7f0d17fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d2015d3a0 0x7f0d200af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:51.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.975+0000 7f0d17fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d2015d3a0 0x7f0d200af390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42234/0 (socket says 172.21.15.7:42234) 2024-01-27T02:38:51.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.975+0000 7f0d17fff700 1 -- 172.21.15.7:0/1640740368 learned_addr learned my addr 172.21.15.7:0/1640740368 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:51.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.975+0000 7f0d25041700 1 --2- 172.21.15.7:0/1640740368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d2015d770 0x7f0d200b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:51.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.975+0000 7f0d24840700 1 --2- 172.21.15.7:0/1640740368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2006ea60 0x7f0d2016de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:51.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.976+0000 7f0d17fff700 1 -- 172.21.15.7:0/1640740368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2006ea60 msgr2=0x7f0d2016de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:51.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.976+0000 7f0d17fff700 1 --2- 172.21.15.7:0/1640740368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2006ea60 0x7f0d2016de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:51.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.976+0000 7f0d17fff700 1 -- 172.21.15.7:0/1640740368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d2015d770 msgr2=0x7f0d200b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:51.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.976+0000 7f0d17fff700 1 --2- 172.21.15.7:0/1640740368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d2015d770 0x7f0d200b0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:51.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.976+0000 7f0d17fff700 1 -- 172.21.15.7:0/1640740368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d10009420 con 0x7f0d2015d3a0 2024-01-27T02:38:51.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.976+0000 7f0d25041700 1 --2- 172.21.15.7:0/1640740368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d2015d770 0x7f0d200b0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.976+0000 7f0d17fff700 1 --2- 172.21.15.7:0/1640740368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d2015d3a0 0x7f0d200af390 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f0d100030c0 tx=0x7f0d10016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.976+0000 7f0d14ff9700 1 -- 172.21.15.7:0/1640740368 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1001cbe0 con 0x7f0d2015d3a0 2024-01-27T02:38:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.976+0000 7f0d14ff9700 1 -- 172.21.15.7:0/1640740368 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1001cd80 con 0x7f0d2015d3a0 2024-01-27T02:38:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.976+0000 7f0d14ff9700 1 -- 172.21.15.7:0/1640740368 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1001a6a0 con 0x7f0d2015d3a0 2024-01-27T02:38:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.976+0000 7f0d26aa4700 1 -- 172.21.15.7:0/1640740368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d2016e9d0 con 0x7f0d2015d3a0 2024-01-27T02:38:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.976+0000 7f0d26aa4700 1 -- 172.21.15.7:0/1640740368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d200b0ec0 con 0x7f0d2015d3a0 2024-01-27T02:38:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.977+0000 7f0d26aa4700 1 -- 172.21.15.7:0/1640740368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d200ad620 con 0x7f0d2015d3a0 2024-01-27T02:38:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.977+0000 7f0d14ff9700 1 -- 172.21.15.7:0/1640740368 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0d1001aa90 con 0x7f0d2015d3a0 2024-01-27T02:38:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.978+0000 7f0d14ff9700 1 --2- 172.21.15.7:0/1640740368 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0d0c066f30 0x7f0d0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.978+0000 7f0d14ff9700 1 -- 172.21.15.7:0/1640740368 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0d10085a20 con 0x7f0d2015d3a0 2024-01-27T02:38:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.978+0000 7f0d14ff9700 1 -- 172.21.15.7:0/1640740368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=135}) v3 -- 0x7f0d0c06ea50 con 0x7f0d2015d3a0 2024-01-27T02:38:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.978+0000 7f0d24840700 1 --2- 172.21.15.7:0/1640740368 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0d0c066f30 0x7f0d0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.979+0000 7f0d24840700 1 --2- 172.21.15.7:0/1640740368 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0d0c066f30 0x7f0d0c0693e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c007fd0 tx=0x7f0d1c007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:51.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:51.982+0000 7f0d14ff9700 1 -- 172.21.15.7:0/1640740368 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d10029060 con 0x7f0d2015d3a0 2024-01-27T02:38:52.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.185+0000 7f0d26aa4700 1 -- 172.21.15.7:0/1640740368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f0d2004c710 con 0x7f0d2015d3a0 2024-01-27T02:38:52.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.186+0000 7f0d14ff9700 1 -- 172.21.15.7:0/1640740368 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v134) v1 ==== 81+0+32253 (secure 0 0 0) 0x7f0d10052d40 con 0x7f0d2015d3a0 2024-01-27T02:38:52.188 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:52.188 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:38:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 134, 2024-01-27T02:38:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:38:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:38:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:38:49.869384+0000", 2024-01-27T02:38:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:38:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:38:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:38:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:38:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:38:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:38:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:38:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:38:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:38:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:38:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:38:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:38:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:38:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:38:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:38:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 17, 2024-01-27T02:38:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:38:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:38:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:38:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:38:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:38:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:38:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:38:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:38:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:38:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:38:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:38:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:38:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:38:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:38:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:38:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:38:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:38:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:38:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:38:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:38:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:38:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:38:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:38:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:38:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:38:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:38:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:38:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:38:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:38:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:38:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:38:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:38:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "106", 2024-01-27T02:38:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:38:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:38:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:38:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:38:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:38:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:38:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:38:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:38:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:38:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:38:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:38:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:38:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:38:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:38:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:38:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:38:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:38:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:38:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:38:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:38:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:38:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:38:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:38:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:38:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:38:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:38:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:38:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:38:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:38:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:38:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:38:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:38:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:38:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:38:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:38:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:38:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:38:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:38:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:38:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:38:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:38:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:38:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 16, 2024-01-27T02:38:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:38:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:38:29.824892+0000", 2024-01-27T02:38:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:38:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:38:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:38:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:38:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:38:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:38:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:38:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:38:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:38:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:38:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:38:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:38:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:38:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:38:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:38:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:38:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:38:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:38:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:38:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:38:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:38:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:38:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:38:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:38:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "131", 2024-01-27T02:38:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:38:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:38:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:38:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:38:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:38:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:38:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:38:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:38:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:38:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:38:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:38:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:38:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:38:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:38:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:38:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:38:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:38:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:38:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:38:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:38:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:38:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:38:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:38:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:38:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:38:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:38:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:38:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:38:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:38:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:38:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:38:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:38:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:38:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:38:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:38:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:38:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:38:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:38:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:38:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:38:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:38:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:38:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:38:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:38:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 17, 2024-01-27T02:38:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:38:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:38:30.301072+0000", 2024-01-27T02:38:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1027, 2024-01-27T02:38:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full,full_quota", 2024-01-27T02:38:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:38:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:38:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:38:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:38:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:38:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:38:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:38:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:38:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:38:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:38:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:38:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:38:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:38:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:38:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:38:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:38:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:38:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:38:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:38:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:38:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:38:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:38:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "134", 2024-01-27T02:38:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:38:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:38:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:38:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:38:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:38:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:38:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:38:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:38:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:38:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-27T02:38:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:38:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:38:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:38:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:38:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:38:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:38:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:38:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:38:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:38:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:38:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:38:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:38:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:38:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:38:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:38:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:38:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:38:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:38:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:38:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:38:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:38:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:38:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:38:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:38:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:38:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:38:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:38:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:38:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:38:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:38:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:38:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:38:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:38:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:38:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:38:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:38:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:38:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:38:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:38:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:38:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.210 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.210 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:52.210 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.211 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.211 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:38:52.211 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:52.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:38:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:38:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:38:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:38:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:38:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:38:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:38:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:38:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:38:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:38:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:38:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:38:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:38:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:38:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:38:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:38:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:38:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:38:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:38:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:38:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:38:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:38:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:38:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:38:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:38:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:38:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:38:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:38:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:38:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:38:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:38:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:38:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:38:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:38:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:38:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:38:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:38:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:38:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:38:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:38:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:38:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:38:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:38:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:38:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:38:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:38:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:38:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:38:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:38:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:38:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:38:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:38:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:38:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:38:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:38:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:38:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:38:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:38:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:38:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:38:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:38:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:38:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:38:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:38:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:38:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:38:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:38:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:38:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:38:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:38:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:38:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:38:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:38:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:38:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:38:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:38:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:38:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:38:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:38:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:38:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:38:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:38:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:38:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:38:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:38:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:38:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:38:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:38:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:38:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:38:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:38:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:38:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:38:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:38:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:38:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:38:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:38:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:38:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:38:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:38:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:38:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:38:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:38:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:38:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:38:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:38:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:38:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:38:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:38:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:38:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:38:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:38:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:38:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:38:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:38:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:38:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:38:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:38:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:38:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:38:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:38:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:38:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:38:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:38:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:38:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:38:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:38:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:38:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:38:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:38:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:38:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:38:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:38:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:38:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:38:52.260 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:38:52.260 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:38:52.260 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:38:52.260 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:38:52.260 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:38:52.260 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:38:52.260 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.260 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:38:52.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.188+0000 7f0d26aa4700 1 -- 172.21.15.7:0/1640740368 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0d0c066f30 msgr2=0x7f0d0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:52.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.189+0000 7f0d26aa4700 1 --2- 172.21.15.7:0/1640740368 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0d0c066f30 0x7f0d0c0693e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c007fd0 tx=0x7f0d1c007d50 comp rx=0 tx=0).stop 2024-01-27T02:38:52.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.189+0000 7f0d26aa4700 1 -- 172.21.15.7:0/1640740368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d2015d3a0 msgr2=0x7f0d200af390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:52.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.189+0000 7f0d26aa4700 1 --2- 172.21.15.7:0/1640740368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d2015d3a0 0x7f0d200af390 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f0d100030c0 tx=0x7f0d10016000 comp rx=0 tx=0).stop 2024-01-27T02:38:52.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.189+0000 7f0d26aa4700 1 -- 172.21.15.7:0/1640740368 shutdown_connections 2024-01-27T02:38:52.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.189+0000 7f0d26aa4700 1 --2- 172.21.15.7:0/1640740368 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0d0c066f30 0x7f0d0c0693e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:52.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.189+0000 7f0d26aa4700 1 --2- 172.21.15.7:0/1640740368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d2015d770 0x7f0d200b0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:52.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.189+0000 7f0d26aa4700 1 --2- 172.21.15.7:0/1640740368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2006ea60 0x7f0d2016de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:52.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.189+0000 7f0d26aa4700 1 --2- 172.21.15.7:0/1640740368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d2015d3a0 0x7f0d200af390 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:52.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.189+0000 7f0d26aa4700 1 -- 172.21.15.7:0/1640740368 >> 172.21.15.7:0/1640740368 conn(0x7f0d20068480 msgr2=0x7f0d2015a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:52.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.189+0000 7f0d26aa4700 1 -- 172.21.15.7:0/1640740368 shutdown_connections 2024-01-27T02:38:52.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.189+0000 7f0d26aa4700 1 -- 172.21.15.7:0/1640740368 wait complete. 2024-01-27T02:38:52.263 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-01-27T02:38:52.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:38:52.264 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2024-01-27T02:38:52.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-27T02:38:52.549 INFO:teuthology.orchestra.run.smithi007.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2024-01-27T02:38:52.549 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records in 2024-01-27T02:38:52.549 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records out 2024-01-27T02:38:52.549 INFO:teuthology.orchestra.run.smithi007.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.261611 s, 200 MB/s 2024-01-27T02:38:52.549 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:38:52.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.683+0000 7f6026932700 1 Processor -- start 2024-01-27T02:38:52.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.683+0000 7f6026932700 1 -- start start 2024-01-27T02:38:52.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.684+0000 7f6026932700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f602006ea40 0x7f60200b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:52.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.684+0000 7f6026932700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60200c30c0 con 0x7f60201409e0 2024-01-27T02:38:52.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.684+0000 7f6026932700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602006ccf0 con 0x7f6020140610 2024-01-27T02:38:52.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.684+0000 7f6026932700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f602006ca10 con 0x7f602006ea40 2024-01-27T02:38:52.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.684+0000 7f601f7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f60201409e0 0x7f602006e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57044/0 (socket says 172.21.15.7:57044) 2024-01-27T02:38:52.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.684+0000 7f6024ecf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f602006ea40 0x7f60200b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:52.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.685+0000 7f601f7fe700 1 -- 172.21.15.7:0/3133202212 learned_addr learned my addr 172.21.15.7:0/3133202212 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:52.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.685+0000 7f6024ecf700 1 -- 172.21.15.7:0/3133202212 >> v1:172.21.15.188:6789/0 conn(0x7f6020140610 legacy=0x7f602006bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.685+0000 7f6024ecf700 1 -- 172.21.15.7:0/3133202212 >> v1:172.21.15.7:6789/0 conn(0x7f60201409e0 legacy=0x7f602006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.685+0000 7f6024ecf700 1 -- 172.21.15.7:0/3133202212 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60200c2a70 con 0x7f602006ea40 2024-01-27T02:38:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.685+0000 7f601effd700 1 -- 172.21.15.7:0/3133202212 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552064645 0 0) 0x7f60200c30c0 con 0x7f60201409e0 2024-01-27T02:38:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.685+0000 7f6024ecf700 1 --2- 172.21.15.7:0/3133202212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f602006ea40 0x7f60200b09f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f6018009e60 tx=0x7f6018009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc944bea65d115f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.685+0000 7f601effd700 1 -- 172.21.15.7:0/3133202212 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601800e070 con 0x7f602006ea40 2024-01-27T02:38:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.685+0000 7f601effd700 1 -- 172.21.15.7:0/3133202212 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601800bbf0 con 0x7f602006ea40 2024-01-27T02:38:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.685+0000 7f601effd700 1 -- 172.21.15.7:0/3133202212 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6018012430 con 0x7f602006ea40 2024-01-27T02:38:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.686+0000 7f6026932700 1 -- 172.21.15.7:0/3133202212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f602006ea40 msgr2=0x7f60200b09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.686+0000 7f6026932700 1 --2- 172.21.15.7:0/3133202212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f602006ea40 0x7f60200b09f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f6018009e60 tx=0x7f6018009780 comp rx=0 tx=0).stop 2024-01-27T02:38:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.686+0000 7f6026932700 1 -- 172.21.15.7:0/3133202212 shutdown_connections 2024-01-27T02:38:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.686+0000 7f6026932700 1 --2- 172.21.15.7:0/3133202212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f602006ea40 0x7f60200b09f0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.686+0000 7f6026932700 1 -- 172.21.15.7:0/3133202212 >> 172.21.15.7:0/3133202212 conn(0x7f60200cdc30 msgr2=0x7f60200ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.686+0000 7f6026932700 1 -- 172.21.15.7:0/3133202212 shutdown_connections 2024-01-27T02:38:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.686+0000 7f6026932700 1 -- 172.21.15.7:0/3133202212 wait complete. 2024-01-27T02:38:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.687+0000 7f6026932700 1 Processor -- start 2024-01-27T02:38:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.687+0000 7f6026932700 1 -- start start 2024-01-27T02:38:52.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.688+0000 7f6026932700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f602006ea40 0x7f602013cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:52.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.688+0000 7f6026932700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6020140610 0x7f6020157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:52.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.688+0000 7f6026932700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60201409e0 0x7f60200b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:52.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.688+0000 7f6026932700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602013d020 con 0x7f60201409e0 2024-01-27T02:38:52.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.688+0000 7f6026932700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602013c650 con 0x7f6020140610 2024-01-27T02:38:52.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.688+0000 7f6026932700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602013ba40 con 0x7f602006ea40 2024-01-27T02:38:52.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.688+0000 7f6024ecf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60201409e0 0x7f60200b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:52.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.688+0000 7f6024ecf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60201409e0 0x7f60200b20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42252/0 (socket says 172.21.15.7:42252) 2024-01-27T02:38:52.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.688+0000 7f6024ecf700 1 -- 172.21.15.7:0/691438836 learned_addr learned my addr 172.21.15.7:0/691438836 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:52.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.688+0000 7f601f7fe700 1 --2- 172.21.15.7:0/691438836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6020140610 0x7f6020157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:52.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.688+0000 7f601ffff700 1 --2- 172.21.15.7:0/691438836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f602006ea40 0x7f602013cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:52.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.689+0000 7f6024ecf700 1 -- 172.21.15.7:0/691438836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f602006ea40 msgr2=0x7f602013cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:52.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.689+0000 7f6024ecf700 1 --2- 172.21.15.7:0/691438836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f602006ea40 0x7f602013cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:52.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.689+0000 7f6024ecf700 1 -- 172.21.15.7:0/691438836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6020140610 msgr2=0x7f6020157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:52.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.689+0000 7f6024ecf700 1 --2- 172.21.15.7:0/691438836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6020140610 0x7f6020157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:52.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.689+0000 7f6024ecf700 1 -- 172.21.15.7:0/691438836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6018009a20 con 0x7f60201409e0 2024-01-27T02:38:52.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.689+0000 7f601f7fe700 1 --2- 172.21.15.7:0/691438836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6020140610 0x7f6020157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:52.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.689+0000 7f601ffff700 1 --2- 172.21.15.7:0/691438836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f602006ea40 0x7f602013cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:52.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.689+0000 7f6024ecf700 1 --2- 172.21.15.7:0/691438836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60201409e0 0x7f60200b20a0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f6018003000 tx=0x7f6018018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:52.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.689+0000 7f6007fff700 1 -- 172.21.15.7:0/691438836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601801e710 con 0x7f60201409e0 2024-01-27T02:38:52.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.689+0000 7f6007fff700 1 -- 172.21.15.7:0/691438836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601801e870 con 0x7f60201409e0 2024-01-27T02:38:52.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.689+0000 7f6026932700 1 -- 172.21.15.7:0/691438836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602013bd20 con 0x7f60201409e0 2024-01-27T02:38:52.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.689+0000 7f6007fff700 1 -- 172.21.15.7:0/691438836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601801ec30 con 0x7f60201409e0 2024-01-27T02:38:52.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.689+0000 7f6026932700 1 -- 172.21.15.7:0/691438836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602013c2e0 con 0x7f60201409e0 2024-01-27T02:38:52.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.690+0000 7f6026932700 1 -- 172.21.15.7:0/691438836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f602004c710 con 0x7f60201409e0 2024-01-27T02:38:52.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.691+0000 7f6007fff700 1 -- 172.21.15.7:0/691438836 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6018024070 con 0x7f60201409e0 2024-01-27T02:38:52.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.691+0000 7f6007fff700 1 --2- 172.21.15.7:0/691438836 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f600c066f30 0x7f600c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:52.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.691+0000 7f6007fff700 1 -- 172.21.15.7:0/691438836 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6018087120 con 0x7f60201409e0 2024-01-27T02:38:52.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.691+0000 7f6007fff700 1 -- 172.21.15.7:0/691438836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=135}) v3 -- 0x7f600c06ea50 con 0x7f60201409e0 2024-01-27T02:38:52.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.691+0000 7f601ffff700 1 --2- 172.21.15.7:0/691438836 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f600c066f30 0x7f600c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:52.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.692+0000 7f601ffff700 1 --2- 172.21.15.7:0/691438836 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f600c066f30 0x7f600c0693e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6010007fd0 tx=0x7f6010009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:52.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.695+0000 7f6007fff700 1 -- 172.21.15.7:0/691438836 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6018054390 con 0x7f60201409e0 2024-01-27T02:38:52.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.905+0000 7f6026932700 1 -- 172.21.15.7:0/691438836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f602013c790 con 0x7f60201409e0 2024-01-27T02:38:52.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.906+0000 7f6007fff700 1 -- 172.21.15.7:0/691438836 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v134) v1 ==== 81+0+32253 (secure 0 0 0) 0x7f60180579b0 con 0x7f60201409e0 2024-01-27T02:38:52.908 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:52.908 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:38:52.908 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 134, 2024-01-27T02:38:52.908 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:38:52.908 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:38:52.909 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:38:49.869384+0000", 2024-01-27T02:38:52.909 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:38:52.909 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:38:52.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:38:52.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:38:52.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:38:52.909 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:38:52.909 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:38:52.909 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:38:52.909 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:38:52.909 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:38:52.909 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:38:52.909 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:38:52.909 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:38:52.909 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:38:52.910 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:38:52.910 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 17, 2024-01-27T02:38:52.910 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:38:52.910 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:38:52.910 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:38:52.910 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:38:52.910 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:38:52.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.910 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:38:52.910 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:38:52.910 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:38:52.910 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:38:52.910 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:38:52.910 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:38:52.910 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:38:52.911 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:38:52.911 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:38:52.911 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:38:52.911 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:38:52.911 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:38:52.911 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:38:52.911 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:38:52.911 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:38:52.911 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:38:52.911 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:38:52.911 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:38:52.911 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:38:52.911 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:38:52.911 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:38:52.912 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:38:52.912 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:38:52.912 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:38:52.912 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:38:52.912 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:38:52.913 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:38:52.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.913 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "106", 2024-01-27T02:38:52.913 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:38:52.913 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:38:52.913 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:38:52.913 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:38:52.913 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:38:52.913 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:38:52.913 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:38:52.913 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:38:52.913 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:38:52.913 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:38:52.913 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:38:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:38:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:38:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:38:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:38:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:38:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:38:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:38:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:38:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:38:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:38:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:38:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:38:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:38:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:38:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:38:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:38:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:38:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:38:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:38:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:38:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:38:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:38:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:38:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:38:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:38:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:38:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:38:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:38:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:38:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:38:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:38:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 16, 2024-01-27T02:38:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:38:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:38:29.824892+0000", 2024-01-27T02:38:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:38:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:38:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:38:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:38:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:38:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:38:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:38:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:38:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:38:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:38:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:38:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:38:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:38:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:38:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:38:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:38:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:38:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:38:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:38:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:38:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:38:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:38:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:38:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:38:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "131", 2024-01-27T02:38:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:38:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:38:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:38:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:38:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:38:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:38:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:38:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:38:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:38:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:38:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:38:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:38:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:38:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:38:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:38:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:38:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:38:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:38:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:38:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:38:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:38:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:38:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:38:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:38:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:38:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:38:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:38:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:38:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:38:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:38:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:38:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:38:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:38:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:38:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:38:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:38:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:38:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:38:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:38:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:38:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:38:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:38:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:38:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:38:52.922 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.922 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.922 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 17, 2024-01-27T02:38:52.922 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:38:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:38:30.301072+0000", 2024-01-27T02:38:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1027, 2024-01-27T02:38:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full,full_quota", 2024-01-27T02:38:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:38:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:38:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:38:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:38:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:38:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:38:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:38:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:38:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:38:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:38:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:38:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:38:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:38:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:38:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:38:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:38:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:38:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:38:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:38:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:38:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:38:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:38:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "134", 2024-01-27T02:38:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:38:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:38:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:38:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:38:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:38:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:38:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:38:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:38:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:38:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-27T02:38:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:38:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:38:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:38:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:38:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:38:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:38:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:38:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:38:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:38:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:38:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:38:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:38:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:38:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:38:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:38:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:38:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:38:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:38:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:38:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:38:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:38:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:38:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:38:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:38:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:38:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:38:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:38:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:38:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:38:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:38:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:38:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:38:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:38:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:38:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:38:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:38:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:38:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:38:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:38:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:38:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:38:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:38:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:38:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:38:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:38:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:38:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:38:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:38:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:52.934 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:52.934 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.934 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:38:52.934 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:52.934 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:52.934 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:52.934 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:52.934 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:38:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.936 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:38:52.936 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:52.936 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.936 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.936 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:52.936 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:38:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:38:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:38:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:38:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:38:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:38:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:38:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:38:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:38:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:38:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:38:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:38:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:38:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:38:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:38:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:38:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:38:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:38:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:38:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:38:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:38:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:38:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:38:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:38:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:38:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:38:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:38:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:38:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:38:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:38:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:38:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:38:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:38:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:38:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:38:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:38:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:38:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:52.953 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:52.953 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:52.953 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:52.953 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:52.953 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:52.953 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.953 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:38:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:38:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:38:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:38:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:38:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:38:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:38:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:38:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:38:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:38:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:38:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:38:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:38:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:38:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:38:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:38:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:38:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:38:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:38:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:38:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:52.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:38:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:38:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:38:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:38:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:38:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:38:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:38:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:38:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:38:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:38:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:38:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:38:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:38:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:38:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:38:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:38:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:38:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:38:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:38:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:38:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:38:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:38:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:38:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:38:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:38:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:38:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:38:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:38:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:38:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:38:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:38:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:38:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:38:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:38:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:38:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:38:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:38:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:38:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:38:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:38:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:38:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:38:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:38:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:38:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:38:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:38:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:38:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:38:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:38:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:38:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:38:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:38:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:38:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:38:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:38:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:38:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:38:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:38:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:38:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:38:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:38:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:38:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:38:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:38:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:38:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:38:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:38:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:38:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:38:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:38:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:38:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:38:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:38:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:38:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:52.984 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:38:52.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.909+0000 7f6026932700 1 -- 172.21.15.7:0/691438836 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f600c066f30 msgr2=0x7f600c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:52.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.909+0000 7f6026932700 1 --2- 172.21.15.7:0/691438836 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f600c066f30 0x7f600c0693e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6010007fd0 tx=0x7f6010009040 comp rx=0 tx=0).stop 2024-01-27T02:38:52.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.909+0000 7f6026932700 1 -- 172.21.15.7:0/691438836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60201409e0 msgr2=0x7f60200b20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:52.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.909+0000 7f6026932700 1 --2- 172.21.15.7:0/691438836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60201409e0 0x7f60200b20a0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f6018003000 tx=0x7f6018018000 comp rx=0 tx=0).stop 2024-01-27T02:38:52.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.909+0000 7f6026932700 1 -- 172.21.15.7:0/691438836 shutdown_connections 2024-01-27T02:38:52.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.909+0000 7f6026932700 1 --2- 172.21.15.7:0/691438836 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f600c066f30 0x7f600c0693e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:52.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.909+0000 7f6026932700 1 --2- 172.21.15.7:0/691438836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60201409e0 0x7f60200b20a0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:52.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.909+0000 7f6026932700 1 --2- 172.21.15.7:0/691438836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f602006ea40 0x7f602013cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:52.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.909+0000 7f6026932700 1 --2- 172.21.15.7:0/691438836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6020140610 0x7f6020157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:52.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.909+0000 7f6026932700 1 -- 172.21.15.7:0/691438836 >> 172.21.15.7:0/691438836 conn(0x7f60200cdc30 msgr2=0x7f60200cbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:52.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.909+0000 7f6026932700 1 -- 172.21.15.7:0/691438836 shutdown_connections 2024-01-27T02:38:52.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:52.909+0000 7f6026932700 1 -- 172.21.15.7:0/691438836 wait complete. 2024-01-27T02:38:52.987 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:38:53.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.119+0000 7fae3b246700 1 Processor -- start 2024-01-27T02:38:53.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.119+0000 7fae3b246700 1 -- start start 2024-01-27T02:38:53.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.120+0000 7fae3b246700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae3415b740 0x7fae340c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:53.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.120+0000 7fae3b246700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fae340cc8e0 0x7fae340ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:53.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.120+0000 7fae3b246700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fae340c4ec0 0x7fae340c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:53.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.120+0000 7fae3b246700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae340cd290 con 0x7fae340c4ec0 2024-01-27T02:38:53.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.120+0000 7fae3b246700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae340c4a30 con 0x7fae340cc8e0 2024-01-27T02:38:53.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.120+0000 7fae3b246700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3415bb10 con 0x7fae3415b740 2024-01-27T02:38:53.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.120+0000 7fae397e3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fae340c4ec0 0x7fae340c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:53.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.120+0000 7fae397e3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fae340c4ec0 0x7fae340c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42254/0 (socket says 172.21.15.7:42254) 2024-01-27T02:38:53.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.120+0000 7fae397e3700 1 -- 172.21.15.7:0/1874316167 learned_addr learned my addr 172.21.15.7:0/1874316167 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:53.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.120+0000 7fae33fff700 1 --2- 172.21.15.7:0/1874316167 >> v2:172.21.15.188:3300/0 conn(0x7fae340cc8e0 0x7fae340ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:53.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.120+0000 7fae38fe2700 1 --2- 172.21.15.7:0/1874316167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae3415b740 0x7fae340c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:53.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.120+0000 7fae397e3700 1 -- 172.21.15.7:0/1874316167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae3415b740 msgr2=0x7fae340c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:53.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.120+0000 7fae397e3700 1 --2- 172.21.15.7:0/1874316167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae3415b740 0x7fae340c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:53.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.120+0000 7fae397e3700 1 -- 172.21.15.7:0/1874316167 >> v2:172.21.15.188:3300/0 conn(0x7fae340cc8e0 msgr2=0x7fae340ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:53.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.120+0000 7fae397e3700 1 --2- 172.21.15.7:0/1874316167 >> v2:172.21.15.188:3300/0 conn(0x7fae340cc8e0 0x7fae340ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:53.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.120+0000 7fae397e3700 1 -- 172.21.15.7:0/1874316167 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae340c5880 con 0x7fae340c4ec0 2024-01-27T02:38:53.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.120+0000 7fae38fe2700 1 --2- 172.21.15.7:0/1874316167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae3415b740 0x7fae340c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:38:53.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.120+0000 7fae33fff700 1 --2- 172.21.15.7:0/1874316167 >> v2:172.21.15.188:3300/0 conn(0x7fae340cc8e0 0x7fae340ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:38:53.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.121+0000 7fae397e3700 1 --2- 172.21.15.7:0/1874316167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae340c4ec0 0x7fae340c5340 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fae3406e540 tx=0x7fae2c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e29d1661e7b641 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.121+0000 7fae337fe700 1 -- 172.21.15.7:0/1874316167 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c00e070 con 0x7fae340c4ec0 2024-01-27T02:38:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.121+0000 7fae337fe700 1 -- 172.21.15.7:0/1874316167 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2c00ba50 con 0x7fae340c4ec0 2024-01-27T02:38:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.121+0000 7fae337fe700 1 -- 172.21.15.7:0/1874316167 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c0092e0 con 0x7fae340c4ec0 2024-01-27T02:38:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.121+0000 7fae3b246700 1 -- 172.21.15.7:0/1874316167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae340c4ec0 msgr2=0x7fae340c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.121+0000 7fae3b246700 1 --2- 172.21.15.7:0/1874316167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae340c4ec0 0x7fae340c5340 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fae3406e540 tx=0x7fae2c009840 comp rx=0 tx=0).stop 2024-01-27T02:38:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.121+0000 7fae3b246700 1 -- 172.21.15.7:0/1874316167 shutdown_connections 2024-01-27T02:38:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.121+0000 7fae3b246700 1 --2- 172.21.15.7:0/1874316167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae340c4ec0 0x7fae340c5340 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.121+0000 7fae3b246700 1 --2- 172.21.15.7:0/1874316167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae3415b740 0x7fae340c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.121+0000 7fae3b246700 1 --2- 172.21.15.7:0/1874316167 >> v2:172.21.15.188:3300/0 conn(0x7fae340cc8e0 0x7fae340ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.121+0000 7fae3b246700 1 -- 172.21.15.7:0/1874316167 >> 172.21.15.7:0/1874316167 conn(0x7fae34063d60 msgr2=0x7fae340ced10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.121+0000 7fae3b246700 1 -- 172.21.15.7:0/1874316167 shutdown_connections 2024-01-27T02:38:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.121+0000 7fae3b246700 1 -- 172.21.15.7:0/1874316167 wait complete. 2024-01-27T02:38:53.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.123+0000 7fae3b246700 1 Processor -- start 2024-01-27T02:38:53.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.123+0000 7fae3b246700 1 -- start start 2024-01-27T02:38:53.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.124+0000 7fae3b246700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae340c4ec0 0x7fae340b1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:53.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.124+0000 7fae3b246700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae340cc8e0 0x7fae3415d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:53.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.124+0000 7fae3b246700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae3415b740 0x7fae341541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:53.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.124+0000 7fae3b246700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae340b4740 con 0x7fae340cc8e0 2024-01-27T02:38:53.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.124+0000 7fae3b246700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae340b3e50 con 0x7fae340c4ec0 2024-01-27T02:38:53.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.124+0000 7fae3b246700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae340b40f0 con 0x7fae3415b740 2024-01-27T02:38:53.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.124+0000 7fae33fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae340cc8e0 0x7fae3415d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:53.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.124+0000 7fae33fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae340cc8e0 0x7fae3415d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42264/0 (socket says 172.21.15.7:42264) 2024-01-27T02:38:53.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.124+0000 7fae33fff700 1 -- 172.21.15.7:0/1148154027 learned_addr learned my addr 172.21.15.7:0/1148154027 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:53.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.124+0000 7fae397e3700 1 --2- 172.21.15.7:0/1148154027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae3415b740 0x7fae341541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:53.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.124+0000 7fae38fe2700 1 --2- 172.21.15.7:0/1148154027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae340c4ec0 0x7fae340b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:53.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.125+0000 7fae33fff700 1 -- 172.21.15.7:0/1148154027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae3415b740 msgr2=0x7fae341541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:53.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.125+0000 7fae33fff700 1 --2- 172.21.15.7:0/1148154027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae3415b740 0x7fae341541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:53.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.125+0000 7fae33fff700 1 -- 172.21.15.7:0/1148154027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae340c4ec0 msgr2=0x7fae340b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:53.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.125+0000 7fae33fff700 1 --2- 172.21.15.7:0/1148154027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae340c4ec0 0x7fae340b1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:53.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.125+0000 7fae33fff700 1 -- 172.21.15.7:0/1148154027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c009ae0 con 0x7fae340cc8e0 2024-01-27T02:38:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.125+0000 7fae33fff700 1 --2- 172.21.15.7:0/1148154027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae340cc8e0 0x7fae3415d1e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fae2000c880 tx=0x7fae20008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.125+0000 7fae30ff9700 1 -- 172.21.15.7:0/1148154027 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae20011600 con 0x7fae340cc8e0 2024-01-27T02:38:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.125+0000 7fae30ff9700 1 -- 172.21.15.7:0/1148154027 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae20011760 con 0x7fae340cc8e0 2024-01-27T02:38:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.125+0000 7fae3b246700 1 -- 172.21.15.7:0/1148154027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae340b43d0 con 0x7fae340cc8e0 2024-01-27T02:38:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.125+0000 7fae30ff9700 1 -- 172.21.15.7:0/1148154027 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae20011ae0 con 0x7fae340cc8e0 2024-01-27T02:38:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.125+0000 7fae3b246700 1 -- 172.21.15.7:0/1148154027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae34155050 con 0x7fae340cc8e0 2024-01-27T02:38:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.126+0000 7fae3b246700 1 -- 172.21.15.7:0/1148154027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae340b48b0 con 0x7fae340cc8e0 2024-01-27T02:38:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.127+0000 7fae30ff9700 1 -- 172.21.15.7:0/1148154027 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fae20018350 con 0x7fae340cc8e0 2024-01-27T02:38:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.127+0000 7fae30ff9700 1 --2- 172.21.15.7:0/1148154027 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fae18066f30 0x7fae180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.127+0000 7fae30ff9700 1 -- 172.21.15.7:0/1148154027 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fae20084420 con 0x7fae340cc8e0 2024-01-27T02:38:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.127+0000 7fae30ff9700 1 -- 172.21.15.7:0/1148154027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=135}) v3 -- 0x7fae1806ea50 con 0x7fae340cc8e0 2024-01-27T02:38:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.127+0000 7fae38fe2700 1 --2- 172.21.15.7:0/1148154027 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fae18066f30 0x7fae180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.128+0000 7fae38fe2700 1 --2- 172.21.15.7:0/1148154027 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fae18066f30 0x7fae180693e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fae28006fd0 tx=0x7fae28008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:53.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.132+0000 7fae30ff9700 1 -- 172.21.15.7:0/1148154027 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae20055080 con 0x7fae340cc8e0 2024-01-27T02:38:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.384+0000 7fae3b246700 1 -- 172.21.15.7:0/1148154027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae3404c710 con 0x7fae340cc8e0 2024-01-27T02:38:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.385+0000 7fae30ff9700 1 -- 172.21.15.7:0/1148154027 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fae2001e020 con 0x7fae340cc8e0 2024-01-27T02:38:53.387 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:53.387 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3997828193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3997828193},{"type":"v1","addr":"172.21.15.188:6837","nonce":3997828193}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1132338302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1132338302},{"type":"v1","addr":"172.21.15.188:6836","nonce":1132338302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/740290645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":740290645},{"type":"v1","addr":"172.21.15.7:6833","nonce":740290645}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:38:31.328277+0000","modified":"2024-01-27T02:38:34.940811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7029":{"gid":7029,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1258751919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1258751919},{"type":"v1","addr":"172.21.15.7:6835","nonce":1258751919}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-27T02:38:53.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.387+0000 7fae3b246700 1 -- 172.21.15.7:0/1148154027 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fae18066f30 msgr2=0x7fae180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:53.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.387+0000 7fae3b246700 1 --2- 172.21.15.7:0/1148154027 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fae18066f30 0x7fae180693e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fae28006fd0 tx=0x7fae28008040 comp rx=0 tx=0).stop 2024-01-27T02:38:53.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.387+0000 7fae3b246700 1 -- 172.21.15.7:0/1148154027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae340cc8e0 msgr2=0x7fae3415d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:53.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.387+0000 7fae3b246700 1 --2- 172.21.15.7:0/1148154027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae340cc8e0 0x7fae3415d1e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fae2000c880 tx=0x7fae20008040 comp rx=0 tx=0).stop 2024-01-27T02:38:53.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.387+0000 7fae3b246700 1 -- 172.21.15.7:0/1148154027 shutdown_connections 2024-01-27T02:38:53.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.387+0000 7fae3b246700 1 --2- 172.21.15.7:0/1148154027 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fae18066f30 0x7fae180693e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:53.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.387+0000 7fae3b246700 1 --2- 172.21.15.7:0/1148154027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae3415b740 0x7fae341541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:53.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.387+0000 7fae3b246700 1 --2- 172.21.15.7:0/1148154027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae340c4ec0 0x7fae340b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:53.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.387+0000 7fae3b246700 1 --2- 172.21.15.7:0/1148154027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae340cc8e0 0x7fae3415d1e0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:53.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.387+0000 7fae3b246700 1 -- 172.21.15.7:0/1148154027 >> 172.21.15.7:0/1148154027 conn(0x7fae34063d60 msgr2=0x7fae34156cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:53.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.387+0000 7fae3b246700 1 -- 172.21.15.7:0/1148154027 shutdown_connections 2024-01-27T02:38:53.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.388+0000 7fae3b246700 1 -- 172.21.15.7:0/1148154027 wait complete. 2024-01-27T02:38:53.390 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 68 2024-01-27T02:38:53.402 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-01-27T02:38:53.555 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","whoami":0,"id":7029,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":21.625472819999999,"mdsmap_epoch":68,"osdmap_epoch":134,"osdmap_epoch_barrier":134,"uptime":27.033844813999998} 2024-01-27T02:38:53.555 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", "fs_name": "cephfs", "id": 7029, "mdsmap_epoch": 68, "osdmap_epoch": 134, "osdmap_epoch_barrier": 134, "rank_uptime": 21.62547282, "state": "up:active", "uptime": 27.033844814, "want_state": "up:active", "whoami": 0 } 2024-01-27T02:38:53.556 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-01-27T02:38:53.556 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:38:53.556 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-01-27T02:38:53.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:38:53.577 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-01-27T02:38:53.642 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:38:53.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.820+0000 7f4ebd478700 1 Processor -- start 2024-01-27T02:38:53.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.820+0000 7f4ebd478700 1 -- start start 2024-01-27T02:38:53.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.821+0000 7f4ebd478700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4eb815a210 0x7f4eb80b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:53.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.821+0000 7f4ebd478700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4eb80cb500 0x7f4eb80b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:53.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.821+0000 7f4ebd478700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb80b37d0 con 0x7f4eb8159e40 2024-01-27T02:38:53.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.821+0000 7f4ebd478700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eb80cc470 con 0x7f4eb815a210 2024-01-27T02:38:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.821+0000 7f4ebd478700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb80cc750 con 0x7f4eb80cb500 2024-01-27T02:38:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.821+0000 7f4eb77fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4eb80cb500 0x7f4eb80b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.821+0000 7f4eb6ffd700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f4eb8159e40 0x7f4eb80cbc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:59314/0 (socket says 172.21.15.7:59314) 2024-01-27T02:38:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.821+0000 7f4eb6ffd700 1 -- 172.21.15.7:0/200108011 learned_addr learned my addr 172.21.15.7:0/200108011 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.821+0000 7f4eb67fc700 1 --2- 172.21.15.7:0/200108011 >> v2:172.21.15.188:3300/0 conn(0x7f4eb815a210 0x7f4eb80b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.821+0000 7f4eb5ffb700 1 -- 172.21.15.7:0/200108011 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 623441157 0 0) 0x7f4eb80b37d0 con 0x7f4eb8159e40 2024-01-27T02:38:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.821+0000 7f4eb5ffb700 1 -- 172.21.15.7:0/200108011 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ea0003570 con 0x7f4eb8159e40 2024-01-27T02:38:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.821+0000 7f4eb77fe700 1 -- 172.21.15.7:0/200108011 >> v2:172.21.15.188:3300/0 conn(0x7f4eb815a210 msgr2=0x7f4eb80b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.821+0000 7f4eb77fe700 1 --2- 172.21.15.7:0/200108011 >> v2:172.21.15.188:3300/0 conn(0x7f4eb815a210 0x7f4eb80b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.821+0000 7f4eb77fe700 1 -- 172.21.15.7:0/200108011 >> v1:172.21.15.188:6789/0 conn(0x7f4eb8159e40 legacy=0x7f4eb80cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.821+0000 7f4eb77fe700 1 -- 172.21.15.7:0/200108011 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb80cca30 con 0x7f4eb80cb500 2024-01-27T02:38:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.821+0000 7f4eb67fc700 1 --2- 172.21.15.7:0/200108011 >> v2:172.21.15.188:3300/0 conn(0x7f4eb815a210 0x7f4eb80b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:38:53.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.822+0000 7f4eb77fe700 1 --2- 172.21.15.7:0/200108011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4eb80cb500 0x7f4eb80b3290 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009e60 tx=0x7f4eb0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea3eaccfc88fe4e4 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:53.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.822+0000 7f4eb5ffb700 1 -- 172.21.15.7:0/200108011 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb000e070 con 0x7f4eb80cb500 2024-01-27T02:38:53.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.822+0000 7f4eb5ffb700 1 -- 172.21.15.7:0/200108011 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb000bab0 con 0x7f4eb80cb500 2024-01-27T02:38:53.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.822+0000 7f4eb5ffb700 1 -- 172.21.15.7:0/200108011 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb000be70 con 0x7f4eb80cb500 2024-01-27T02:38:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.822+0000 7f4ebd478700 1 -- 172.21.15.7:0/200108011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4eb80cb500 msgr2=0x7f4eb80b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.822+0000 7f4ebd478700 1 --2- 172.21.15.7:0/200108011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4eb80cb500 0x7f4eb80b3290 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009e60 tx=0x7f4eb0009780 comp rx=0 tx=0).stop 2024-01-27T02:38:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.822+0000 7f4ebd478700 1 -- 172.21.15.7:0/200108011 shutdown_connections 2024-01-27T02:38:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.822+0000 7f4ebd478700 1 --2- 172.21.15.7:0/200108011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4eb80cb500 0x7f4eb80b3290 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.822+0000 7f4ebd478700 1 --2- 172.21.15.7:0/200108011 >> v2:172.21.15.188:3300/0 conn(0x7f4eb815a210 0x7f4eb80b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.822+0000 7f4ebd478700 1 -- 172.21.15.7:0/200108011 >> 172.21.15.7:0/200108011 conn(0x7f4eb80679b0 msgr2=0x7f4eb8067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.822+0000 7f4ebd478700 1 -- 172.21.15.7:0/200108011 shutdown_connections 2024-01-27T02:38:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.823+0000 7f4ebd478700 1 -- 172.21.15.7:0/200108011 wait complete. 2024-01-27T02:38:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.824+0000 7f4ebd478700 1 Processor -- start 2024-01-27T02:38:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.824+0000 7f4ebd478700 1 -- start start 2024-01-27T02:38:53.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.825+0000 7f4ebd478700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4eb80cb500 0x7f4eb8155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:53.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.825+0000 7f4ebd478700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4eb8159e40 0x7f4eb815b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:53.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.825+0000 7f4ebd478700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4eb815a210 0x7f4eb80b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:53.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.825+0000 7f4ebd478700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb815c7d0 con 0x7f4eb8159e40 2024-01-27T02:38:53.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.825+0000 7f4ebd478700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb815cc60 con 0x7f4eb815a210 2024-01-27T02:38:53.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.825+0000 7f4ebd478700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb815bbc0 con 0x7f4eb80cb500 2024-01-27T02:38:53.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.825+0000 7f4eb67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4eb8159e40 0x7f4eb815b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:53.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.825+0000 7f4eb67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4eb8159e40 0x7f4eb815b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42268/0 (socket says 172.21.15.7:42268) 2024-01-27T02:38:53.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.825+0000 7f4eb67fc700 1 -- 172.21.15.7:0/2445939847 learned_addr learned my addr 172.21.15.7:0/2445939847 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:53.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.825+0000 7f4eb77fe700 1 --2- 172.21.15.7:0/2445939847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4eb815a210 0x7f4eb80b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:53.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.825+0000 7f4eb6ffd700 1 --2- 172.21.15.7:0/2445939847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4eb80cb500 0x7f4eb8155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:53.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.825+0000 7f4eb67fc700 1 -- 172.21.15.7:0/2445939847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4eb80cb500 msgr2=0x7f4eb8155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:53.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.825+0000 7f4eb67fc700 1 --2- 172.21.15.7:0/2445939847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4eb80cb500 0x7f4eb8155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:53.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.825+0000 7f4eb67fc700 1 -- 172.21.15.7:0/2445939847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4eb815a210 msgr2=0x7f4eb80b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:53.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.825+0000 7f4eb67fc700 1 --2- 172.21.15.7:0/2445939847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4eb815a210 0x7f4eb80b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:53.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.825+0000 7f4eb67fc700 1 -- 172.21.15.7:0/2445939847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eac009420 con 0x7f4eb8159e40 2024-01-27T02:38:53.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.825+0000 7f4eb77fe700 1 --2- 172.21.15.7:0/2445939847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4eb815a210 0x7f4eb80b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:38:53.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.825+0000 7f4eb67fc700 1 --2- 172.21.15.7:0/2445939847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4eb8159e40 0x7f4eb815b0d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00ec80 tx=0x7f4eac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:53.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.826+0000 7f4e9effd700 1 -- 172.21.15.7:0/2445939847 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac00c9d0 con 0x7f4eb8159e40 2024-01-27T02:38:53.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.826+0000 7f4ebd478700 1 -- 172.21.15.7:0/2445939847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb0009a20 con 0x7f4eb8159e40 2024-01-27T02:38:53.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.826+0000 7f4e9effd700 1 -- 172.21.15.7:0/2445939847 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eac00cb30 con 0x7f4eb8159e40 2024-01-27T02:38:53.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.826+0000 7f4e9effd700 1 -- 172.21.15.7:0/2445939847 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac00ceb0 con 0x7f4eb8159e40 2024-01-27T02:38:53.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.826+0000 7f4ebd478700 1 -- 172.21.15.7:0/2445939847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb815c180 con 0x7f4eb8159e40 2024-01-27T02:38:53.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.827+0000 7f4ebd478700 1 -- 172.21.15.7:0/2445939847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb815c940 con 0x7f4eb8159e40 2024-01-27T02:38:53.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.827+0000 7f4e9effd700 1 -- 172.21.15.7:0/2445939847 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4eac013680 con 0x7f4eb8159e40 2024-01-27T02:38:53.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.828+0000 7f4e9effd700 1 --2- 172.21.15.7:0/2445939847 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4ea4066f30 0x7f4ea40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:53.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.828+0000 7f4e9effd700 1 -- 172.21.15.7:0/2445939847 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4eac084920 con 0x7f4eb8159e40 2024-01-27T02:38:53.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.828+0000 7f4e9effd700 1 -- 172.21.15.7:0/2445939847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=135}) v3 -- 0x7f4ea406ea50 con 0x7f4eb8159e40 2024-01-27T02:38:53.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.828+0000 7f4eb6ffd700 1 --2- 172.21.15.7:0/2445939847 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4ea4066f30 0x7f4ea40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:53.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.828+0000 7f4eb6ffd700 1 --2- 172.21.15.7:0/2445939847 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4ea4066f30 0x7f4ea40693e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80058d0 tx=0x7f4ea8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:53.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:53.832+0000 7f4e9effd700 1 -- 172.21.15.7:0/2445939847 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4eac017060 con 0x7f4eb8159e40 2024-01-27T02:38:54.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:54.037+0000 7f4ebd478700 1 -- 172.21.15.7:0/2445939847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f4eb804c710 con 0x7f4eb8159e40 2024-01-27T02:38:54.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:54.038+0000 7f4e9effd700 1 -- 172.21.15.7:0/2445939847 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v134) v1 ==== 81+0+32253 (secure 0 0 0) 0x7f4eac051b90 con 0x7f4eb8159e40 2024-01-27T02:38:54.040 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:54.040 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:38:54.040 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 134, 2024-01-27T02:38:54.040 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:38:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:38:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:38:49.869384+0000", 2024-01-27T02:38:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:38:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:38:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:38:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:38:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:38:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:38:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:38:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:38:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:38:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:38:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:38:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:38:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:38:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:38:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:38:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 17, 2024-01-27T02:38:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:38:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:38:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:38:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:38:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:38:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:38:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:38:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:38:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:38:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:38:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:38:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:38:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:38:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:38:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:38:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:38:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:38:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:38:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:38:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:38:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:38:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:38:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:38:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:38:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:38:54.044 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:38:54.044 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:38:54.044 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:38:54.044 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:38:54.044 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:38:54.044 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:38:54.044 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:38:54.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.044 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "106", 2024-01-27T02:38:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:38:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:38:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:38:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:38:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:38:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:38:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:38:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:38:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:38:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:38:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:38:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:38:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:38:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:38:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:38:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:38:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:38:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:38:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:38:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:38:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:38:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:38:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:38:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:38:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:38:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:38:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:38:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:38:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:38:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:38:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:38:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:38:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:38:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:38:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:38:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:38:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:38:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:38:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:38:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:38:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:38:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:38:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 16, 2024-01-27T02:38:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:38:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:38:29.824892+0000", 2024-01-27T02:38:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:38:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:38:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:38:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:38:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:38:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:38:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:38:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:38:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:38:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:38:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:38:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:38:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:38:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:38:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:38:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:38:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:38:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:38:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:38:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:38:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:38:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:38:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:38:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:38:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "131", 2024-01-27T02:38:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:38:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:38:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:38:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:38:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:38:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:38:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:38:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:38:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:38:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:38:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:38:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:38:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:38:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:38:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:38:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:38:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:38:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:38:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:38:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:38:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:38:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:38:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:38:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:38:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:38:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:38:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:38:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:38:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:38:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:38:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:38:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:38:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:38:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:38:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:38:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:38:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:38:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:38:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:38:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:38:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:38:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:38:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:38:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:38:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 17, 2024-01-27T02:38:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:38:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:38:30.301072+0000", 2024-01-27T02:38:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1027, 2024-01-27T02:38:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full,full_quota", 2024-01-27T02:38:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:38:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:38:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:38:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:38:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:38:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:38:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:38:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:38:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:38:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:38:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:38:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:38:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:38:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:38:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:38:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:38:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:38:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:38:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:38:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:38:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:38:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:38:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "134", 2024-01-27T02:38:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:38:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:38:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:38:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:38:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:38:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:38:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:38:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:38:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:38:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-27T02:38:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:38:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:38:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:38:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:38:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:38:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:38:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:38:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:38:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:38:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:38:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:38:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:38:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:38:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:38:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:38:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:38:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:38:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:38:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:38:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:38:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:38:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:38:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:38:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:38:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:38:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:38:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:38:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:38:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:38:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:38:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:38:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:38:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:38:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:38:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:38:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:38:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:38:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:38:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:38:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:38:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:38:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:38:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:38:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:38:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:38:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:38:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:38:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:38:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:38:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:38:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:38:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:38:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:38:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:38:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:38:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:38:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:38:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:38:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:38:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:38:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:38:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:38:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:38:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:38:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:38:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:38:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:38:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:38:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:38:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:38:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:38:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:38:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:38:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:38:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:38:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:38:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:38:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:38:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:38:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:38:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:38:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:38:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:38:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:38:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:38:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:38:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:38:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:38:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:38:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:38:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:38:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:38:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:38:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:38:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:38:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:38:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:38:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:38:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:38:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:38:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:38:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:38:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:38:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:38:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:38:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:38:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:38:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:38:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:38:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:38:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:38:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:38:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:38:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:38:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:38:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:38:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:38:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:38:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:38:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:38:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:38:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:38:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:38:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:38:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:38:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:38:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:38:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:38:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:38:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:38:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:38:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:38:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:38:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:38:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:38:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:38:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:38:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:38:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:38:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:38:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:38:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:38:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:38:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:38:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:38:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:38:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:38:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:38:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:38:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:38:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:38:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:38:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:38:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:38:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:38:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:38:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:38:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:38:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:38:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:38:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:38:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:38:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:38:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:38:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:38:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:38:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:38:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:38:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:38:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:38:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:38:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:38:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:38:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:38:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:38:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:38:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:38:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:38:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:38:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:38:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:38:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:38:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:54.132 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:38:54.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:54.041+0000 7f4ebd478700 1 -- 172.21.15.7:0/2445939847 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4ea4066f30 msgr2=0x7f4ea40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:54.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:54.041+0000 7f4ebd478700 1 --2- 172.21.15.7:0/2445939847 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4ea4066f30 0x7f4ea40693e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80058d0 tx=0x7f4ea8009040 comp rx=0 tx=0).stop 2024-01-27T02:38:54.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:54.041+0000 7f4ebd478700 1 -- 172.21.15.7:0/2445939847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4eb8159e40 msgr2=0x7f4eb815b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:54.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:54.041+0000 7f4ebd478700 1 --2- 172.21.15.7:0/2445939847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4eb8159e40 0x7f4eb815b0d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00ec80 tx=0x7f4eac0091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:54.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:54.041+0000 7f4ebd478700 1 -- 172.21.15.7:0/2445939847 shutdown_connections 2024-01-27T02:38:54.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:54.041+0000 7f4ebd478700 1 --2- 172.21.15.7:0/2445939847 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4ea4066f30 0x7f4ea40693e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:54.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:54.041+0000 7f4ebd478700 1 --2- 172.21.15.7:0/2445939847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4eb815a210 0x7f4eb80b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:54.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:54.041+0000 7f4ebd478700 1 --2- 172.21.15.7:0/2445939847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4eb80cb500 0x7f4eb8155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:54.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:54.041+0000 7f4ebd478700 1 --2- 172.21.15.7:0/2445939847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4eb8159e40 0x7f4eb815b0d0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:54.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:54.041+0000 7f4ebd478700 1 -- 172.21.15.7:0/2445939847 >> 172.21.15.7:0/2445939847 conn(0x7f4eb80679b0 msgr2=0x7f4eb81576f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:54.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:54.041+0000 7f4ebd478700 1 -- 172.21.15.7:0/2445939847 shutdown_connections 2024-01-27T02:38:54.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:54.041+0000 7f4ebd478700 1 -- 172.21.15.7:0/2445939847 wait complete. 2024-01-27T02:38:54.136 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-27T02:38:59.137 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:38:59.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.277+0000 7f1e5c1bc700 1 Processor -- start 2024-01-27T02:38:59.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.277+0000 7f1e5c1bc700 1 -- start start 2024-01-27T02:38:59.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.278+0000 7f1e5c1bc700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1e540c41e0 0x7f1e5415a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:59.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.278+0000 7f1e5c1bc700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5415b6c0 con 0x7f1e540637d0 2024-01-27T02:38:59.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.278+0000 7f1e5c1bc700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5406f2e0 con 0x7f1e5415ac80 2024-01-27T02:38:59.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.278+0000 7f1e5c1bc700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e5415b100 con 0x7f1e540c41e0 2024-01-27T02:38:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.278+0000 7f1e59f58700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1e540637d0 0x7f1e5406ebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57056/0 (socket says 172.21.15.7:57056) 2024-01-27T02:38:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.278+0000 7f1e59f58700 1 -- 172.21.15.7:0/62935507 learned_addr learned my addr 172.21.15.7:0/62935507 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:38:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.278+0000 7f1e58f56700 1 -- 172.21.15.7:0/62935507 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2480777731 0 0) 0x7f1e5415b6c0 con 0x7f1e540637d0 2024-01-27T02:38:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.278+0000 7f1e59757700 1 --2- 172.21.15.7:0/62935507 >> v2:172.21.15.188:3300/0 conn(0x7f1e540c41e0 0x7f1e5415a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.278+0000 7f1e58f56700 1 -- 172.21.15.7:0/62935507 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e40003570 con 0x7f1e540637d0 2024-01-27T02:38:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e58f56700 1 -- 172.21.15.7:0/62935507 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2620185216 0 0) 0x7f1e40003570 con 0x7f1e540637d0 2024-01-27T02:38:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e58f56700 1 -- 172.21.15.7:0/62935507 >> v2:172.21.15.188:3300/0 conn(0x7f1e540c41e0 msgr2=0x7f1e5415a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e58f56700 1 --2- 172.21.15.7:0/62935507 >> v2:172.21.15.188:3300/0 conn(0x7f1e540c41e0 0x7f1e5415a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e58f56700 1 -- 172.21.15.7:0/62935507 >> v1:172.21.15.188:6789/0 conn(0x7f1e5415ac80 legacy=0x7f1e540c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e58f56700 1 -- 172.21.15.7:0/62935507 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5415b3e0 con 0x7f1e540637d0 2024-01-27T02:38:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e58f56700 1 -- 172.21.15.7:0/62935507 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f1e50001470 con 0x7f1e540637d0 2024-01-27T02:38:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e58f56700 1 -- 172.21.15.7:0/62935507 >> v1:172.21.15.7:6789/0 conn(0x7f1e540637d0 legacy=0x7f1e5406ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e58f56700 1 --2- 172.21.15.7:0/62935507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e40003a60 0x7f1e40003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e58f56700 1 --2- 172.21.15.7:0/62935507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e40004810 0x7f1e40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e58f56700 1 --2- 172.21.15.7:0/62935507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e400051c0 0x7f1e4000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e58f56700 1 -- 172.21.15.7:0/62935507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5415b100 con 0x7f1e400051c0 2024-01-27T02:38:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e58f56700 1 -- 172.21.15.7:0/62935507 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4000c160 con 0x7f1e40003a60 2024-01-27T02:38:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e58f56700 1 -- 172.21.15.7:0/62935507 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4000c320 con 0x7f1e40004810 2024-01-27T02:38:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e5a759700 1 --2- 172.21.15.7:0/62935507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e400051c0 0x7f1e4000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e59f58700 1 --2- 172.21.15.7:0/62935507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e40003a60 0x7f1e40003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e59757700 1 --2- 172.21.15.7:0/62935507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e40004810 0x7f1e40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e5a759700 1 -- 172.21.15.7:0/62935507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e40004810 msgr2=0x7f1e40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e5a759700 1 --2- 172.21.15.7:0/62935507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e40004810 0x7f1e40004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e5a759700 1 -- 172.21.15.7:0/62935507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e40003a60 msgr2=0x7f1e40003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:38:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e5a759700 1 --2- 172.21.15.7:0/62935507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e40003a60 0x7f1e40003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e5a759700 1 -- 172.21.15.7:0/62935507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e40003570 con 0x7f1e400051c0 2024-01-27T02:38:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e5a759700 1 -- 172.21.15.7:0/62935507 reap_dead start 2024-01-27T02:38:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.279+0000 7f1e5a759700 1 --2- 172.21.15.7:0/62935507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e400051c0 0x7f1e4000ba60 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c001fc0 tx=0x7f1e4c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2459119a175d65 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.280+0000 7f1e58f56700 1 -- 172.21.15.7:0/62935507 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c001950 con 0x7f1e400051c0 2024-01-27T02:38:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.280+0000 7f1e58f56700 1 -- 172.21.15.7:0/62935507 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4c00c8b0 con 0x7f1e400051c0 2024-01-27T02:38:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.280+0000 7f1e58f56700 1 -- 172.21.15.7:0/62935507 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c00cc50 con 0x7f1e400051c0 2024-01-27T02:38:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.280+0000 7f1e5c1bc700 1 -- 172.21.15.7:0/62935507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e400051c0 msgr2=0x7f1e4000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.280+0000 7f1e5c1bc700 1 --2- 172.21.15.7:0/62935507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e400051c0 0x7f1e4000ba60 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c001fc0 tx=0x7f1e4c001640 comp rx=0 tx=0).stop 2024-01-27T02:38:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.280+0000 7f1e5c1bc700 1 -- 172.21.15.7:0/62935507 shutdown_connections 2024-01-27T02:38:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.280+0000 7f1e5c1bc700 1 --2- 172.21.15.7:0/62935507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e400051c0 0x7f1e4000ba60 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.280+0000 7f1e5c1bc700 1 -- 172.21.15.7:0/62935507 >> 172.21.15.7:0/62935507 conn(0x7f1e540cdc30 msgr2=0x7f1e540ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.280+0000 7f1e5c1bc700 1 -- 172.21.15.7:0/62935507 shutdown_connections 2024-01-27T02:38:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.280+0000 7f1e5c1bc700 1 -- 172.21.15.7:0/62935507 wait complete. 2024-01-27T02:38:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.281+0000 7f1e5c1bc700 1 Processor -- start 2024-01-27T02:38:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.281+0000 7f1e5c1bc700 1 -- start start 2024-01-27T02:38:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.282+0000 7f1e5c1bc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e400051c0 0x7f1e540b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.282+0000 7f1e5c1bc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e540cd5b0 0x7f1e541574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.282+0000 7f1e5c1bc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e540cc010 0x7f1e540cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:59.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.282+0000 7f1e5c1bc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5415c970 con 0x7f1e540cd5b0 2024-01-27T02:38:59.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.282+0000 7f1e5c1bc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540aeb40 con 0x7f1e540cc010 2024-01-27T02:38:59.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.282+0000 7f1e5c1bc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540cc9a0 con 0x7f1e400051c0 2024-01-27T02:38:59.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.282+0000 7f1e59757700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e540cd5b0 0x7f1e541574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:59.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.282+0000 7f1e59757700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e540cd5b0 0x7f1e541574b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42290/0 (socket says 172.21.15.7:42290) 2024-01-27T02:38:59.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.282+0000 7f1e59757700 1 -- 172.21.15.7:0/1669511355 learned_addr learned my addr 172.21.15.7:0/1669511355 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:38:59.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.282+0000 7f1e59f58700 1 --2- 172.21.15.7:0/1669511355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e400051c0 0x7f1e540b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:59.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.282+0000 7f1e5a759700 1 --2- 172.21.15.7:0/1669511355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e540cc010 0x7f1e540cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:59.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.283+0000 7f1e59757700 1 -- 172.21.15.7:0/1669511355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e400051c0 msgr2=0x7f1e540b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:59.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.283+0000 7f1e59757700 1 --2- 172.21.15.7:0/1669511355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e400051c0 0x7f1e540b09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:59.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.283+0000 7f1e59757700 1 -- 172.21.15.7:0/1669511355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e540cc010 msgr2=0x7f1e540cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:59.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.283+0000 7f1e59757700 1 --2- 172.21.15.7:0/1669511355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e540cc010 0x7f1e540cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:59.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.283+0000 7f1e59757700 1 -- 172.21.15.7:0/1669511355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e540b1320 con 0x7f1e540cd5b0 2024-01-27T02:38:59.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.283+0000 7f1e59757700 1 --2- 172.21.15.7:0/1669511355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e540cd5b0 0x7f1e541574b0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f1e44011aa0 tx=0x7f1e440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:59.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.283+0000 7f1e49ffb700 1 -- 172.21.15.7:0/1669511355 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4400f940 con 0x7f1e540cd5b0 2024-01-27T02:38:59.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.283+0000 7f1e49ffb700 1 -- 172.21.15.7:0/1669511355 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4400faa0 con 0x7f1e540cd5b0 2024-01-27T02:38:59.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.283+0000 7f1e5c1bc700 1 -- 172.21.15.7:0/1669511355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e540b1600 con 0x7f1e540cd5b0 2024-01-27T02:38:59.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.283+0000 7f1e49ffb700 1 -- 172.21.15.7:0/1669511355 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4400fe20 con 0x7f1e540cd5b0 2024-01-27T02:38:59.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.283+0000 7f1e5c1bc700 1 -- 172.21.15.7:0/1669511355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e540b0f10 con 0x7f1e540cd5b0 2024-01-27T02:38:59.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.284+0000 7f1e5c1bc700 1 -- 172.21.15.7:0/1669511355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e540b0150 con 0x7f1e540cd5b0 2024-01-27T02:38:59.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.285+0000 7f1e49ffb700 1 -- 172.21.15.7:0/1669511355 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1e44014650 con 0x7f1e540cd5b0 2024-01-27T02:38:59.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.285+0000 7f1e49ffb700 1 --2- 172.21.15.7:0/1669511355 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1e38066f30 0x7f1e380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:38:59.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.285+0000 7f1e49ffb700 1 -- 172.21.15.7:0/1669511355 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f1e44018030 con 0x7f1e540cd5b0 2024-01-27T02:38:59.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.286+0000 7f1e49ffb700 1 -- 172.21.15.7:0/1669511355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=137}) v3 -- 0x7f1e3806eb10 con 0x7f1e540cd5b0 2024-01-27T02:38:59.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.286+0000 7f1e59f58700 1 --2- 172.21.15.7:0/1669511355 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1e38066f30 0x7f1e380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:38:59.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.286+0000 7f1e59f58700 1 --2- 172.21.15.7:0/1669511355 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1e38066f30 0x7f1e380693e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1e50001ed0 tx=0x7f1e540c9480 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:38:59.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.290+0000 7f1e49ffb700 1 -- 172.21.15.7:0/1669511355 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e4408b080 con 0x7f1e540cd5b0 2024-01-27T02:38:59.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.493+0000 7f1e5c1bc700 1 -- 172.21.15.7:0/1669511355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f1e5404c710 con 0x7f1e540cd5b0 2024-01-27T02:38:59.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.494+0000 7f1e49ffb700 1 -- 172.21.15.7:0/1669511355 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v136) v1 ==== 81+0+32629 (secure 0 0 0) 0x7f1e44022070 con 0x7f1e540cd5b0 2024-01-27T02:38:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:38:59.733 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:38:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 136, 2024-01-27T02:38:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:38:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:38:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:38:56.999418+0000", 2024-01-27T02:38:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:38:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:38:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:38:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:38:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:38:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:38:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:38:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:38:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:38:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:38:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:38:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:38:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:38:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:38:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:38:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 17, 2024-01-27T02:38:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:38:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:38:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-01-27T02:38:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:38:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:38:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:38:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:38:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:38:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:38:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:38:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:38:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:38:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:38:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:38:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:38:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:38:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:38:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:38:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:38:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:38:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:38:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:38:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:38:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:38:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:38:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:38:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:38:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:38:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:38:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:38:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:38:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:38:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "106", 2024-01-27T02:38:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:38:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:38:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:38:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:38:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:38:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:38:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:38:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:38:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:38:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:38:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:38:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:38:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:38:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:38:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:38:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:38:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:38:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:38:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:38:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:38:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:38:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:38:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:38:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:38:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:38:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:38:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:38:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:38:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:38:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:38:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:38:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:38:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:38:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:38:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:38:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:38:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:38:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:38:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:38:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:38:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:38:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:38:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 16, 2024-01-27T02:38:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:38:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:38:29.824892+0000", 2024-01-27T02:38:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:38:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:38:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:38:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:38:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:38:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:38:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:38:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:38:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:38:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:38:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:38:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:38:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:38:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:38:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:38:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:38:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:38:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:38:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:38:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:38:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:38:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:38:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:38:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:38:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "131", 2024-01-27T02:38:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:38:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:38:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:38:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:38:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:38:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:38:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:38:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:38:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:38:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:38:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:38:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:38:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:38:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:38:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:38:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:38:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:38:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:38:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:38:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:38:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:38:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:38:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:38:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:38:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:38:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:38:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:38:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:38:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:38:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:38:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:38:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:38:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:38:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:38:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:38:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:38:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:38:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:38:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:38:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:38:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:38:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:38:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:38:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:38:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 17, 2024-01-27T02:38:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:38:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:38:30.301072+0000", 2024-01-27T02:38:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1027, 2024-01-27T02:38:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full,full_quota", 2024-01-27T02:38:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:38:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:38:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:38:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:38:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:38:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:38:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:38:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:38:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:38:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:38:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:38:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:38:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:38:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:38:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:38:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:38:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:38:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:38:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:38:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:38:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:38:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:38:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "134", 2024-01-27T02:38:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:38:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:38:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:38:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:38:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:38:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:38:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:38:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:38:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:38:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-27T02:38:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:38:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:38:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:38:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:38:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:38:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:38:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:38:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:38:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:38:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:38:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:38:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:38:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:38:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:38:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:38:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:38:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:38:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:38:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:38:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:38:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:38:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:38:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:38:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:38:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:38:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:38:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:38:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:38:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:38:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:38:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:38:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:38:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:38:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:38:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:38:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:38:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:38:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:38:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:38:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:38:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:38:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:38:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:38:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:38:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:38:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:38:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:38:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:38:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:38:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:38:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:38:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:38:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:38:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:38:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:38:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:38:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:38:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:38:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:38:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:38:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:38:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:38:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:38:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:38:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 135, 2024-01-27T02:38:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:38:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:38:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:38:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:38:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:38:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:38:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:38:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:38:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:38:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:38:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:38:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:38:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:38:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:38:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 135, 2024-01-27T02:38:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:38:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:38:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:38:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:38:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:38:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:38:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:38:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:38:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:38:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:38:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:38:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:38:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:38:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:38:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:38:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:38:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:38:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:38:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:38:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:38:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.779 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:38:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:38:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:38:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:38:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:38:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:38:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:38:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:38:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:38:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:38:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:38:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:38:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:38:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:38:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:38:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:38:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:38:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:38:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:38:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:38:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:38:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:38:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:38:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:38:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:38:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:38:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:38:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:38:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:38:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:38:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:38:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:38:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:38:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:38:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:38:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:38:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:38:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:38:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:38:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:38:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:38:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:38:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:38:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:38:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:38:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:38:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:38:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:38:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:38:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:38:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:38:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:38:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:38:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:38:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:38:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:38:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:38:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:38:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:38:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:38:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:38:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:38:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:38:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:38:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:38:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:38:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:38:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:38:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:38:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:38:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:38:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:38:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:38:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:38:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:38:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:38:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:38:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:38:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:38:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:38:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:38:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:38:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:38:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:38:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:38:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:38:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:38:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:38:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:38:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:38:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:38:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:38:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:38:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:38:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:38:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:38:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [ 2024-01-27T02:38:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "17.e", 2024-01-27T02:38:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-01-27T02:38:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 0, 2024-01-27T02:38:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 5 2024-01-27T02:38:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "17.24", 2024-01-27T02:38:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-01-27T02:38:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:38:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 0, 2024-01-27T02:38:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 2 2024-01-27T02:38:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:38:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:38:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:38:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:38:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:38:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:38:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:38:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:38:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:38:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:38:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:38:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:38:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:38:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:38:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:38:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:38:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:38:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:38:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:38:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:38:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:38:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:38:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:38:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:38:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:38:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:38:59.806 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:38:59.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.497+0000 7f1e5c1bc700 1 -- 172.21.15.7:0/1669511355 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1e38066f30 msgr2=0x7f1e380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:59.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.497+0000 7f1e5c1bc700 1 --2- 172.21.15.7:0/1669511355 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1e38066f30 0x7f1e380693e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1e50001ed0 tx=0x7f1e540c9480 comp rx=0 tx=0).stop 2024-01-27T02:38:59.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.497+0000 7f1e5c1bc700 1 -- 172.21.15.7:0/1669511355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e540cd5b0 msgr2=0x7f1e541574b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:38:59.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.497+0000 7f1e5c1bc700 1 --2- 172.21.15.7:0/1669511355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e540cd5b0 0x7f1e541574b0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f1e44011aa0 tx=0x7f1e440091a0 comp rx=0 tx=0).stop 2024-01-27T02:38:59.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.497+0000 7f1e5c1bc700 1 -- 172.21.15.7:0/1669511355 shutdown_connections 2024-01-27T02:38:59.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.497+0000 7f1e5c1bc700 1 --2- 172.21.15.7:0/1669511355 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1e38066f30 0x7f1e380693e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:59.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.497+0000 7f1e5c1bc700 1 --2- 172.21.15.7:0/1669511355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e540cc010 0x7f1e540cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:59.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.497+0000 7f1e5c1bc700 1 --2- 172.21.15.7:0/1669511355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e400051c0 0x7f1e540b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:59.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.497+0000 7f1e5c1bc700 1 --2- 172.21.15.7:0/1669511355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e540cd5b0 0x7f1e541574b0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:38:59.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.497+0000 7f1e5c1bc700 1 -- 172.21.15.7:0/1669511355 >> 172.21.15.7:0/1669511355 conn(0x7f1e540cdc30 msgr2=0x7f1e5415a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:38:59.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.497+0000 7f1e5c1bc700 1 -- 172.21.15.7:0/1669511355 shutdown_connections 2024-01-27T02:38:59.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:38:59.497+0000 7f1e5c1bc700 1 -- 172.21.15.7:0/1669511355 wait complete. 2024-01-27T02:38:59.808 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-27T02:39:04.809 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:39:04.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.950+0000 7fc87191b700 1 Processor -- start 2024-01-27T02:39:04.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.950+0000 7fc87191b700 1 -- start start 2024-01-27T02:39:04.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.951+0000 7fc87191b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc86c15a450 0x7fc86c0b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:04.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.951+0000 7fc87191b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc86c0cc980 con 0x7fc86c0b2030 2024-01-27T02:39:04.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.951+0000 7fc87191b700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc86c0ccc20 con 0x7fc86c0ce0f0 2024-01-27T02:39:04.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.951+0000 7fc87191b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c0ccf00 con 0x7fc86c15a450 2024-01-27T02:39:04.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.951+0000 7fc86a7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc86c0b2030 0x7fc86c0cd9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59772/0 (socket says 172.21.15.7:59772) 2024-01-27T02:39:04.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.951+0000 7fc86a7fc700 1 -- 172.21.15.7:0/2011675904 learned_addr learned my addr 172.21.15.7:0/2011675904 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:04.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.951+0000 7fc86affd700 1 --2- 172.21.15.7:0/2011675904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc86c15a450 0x7fc86c0b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:04.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.952+0000 7fc869ffb700 1 -- 172.21.15.7:0/2011675904 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1983141270 0 0) 0x7fc86c0cc980 con 0x7fc86c0b2030 2024-01-27T02:39:04.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.952+0000 7fc869ffb700 1 -- 172.21.15.7:0/2011675904 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc854003570 con 0x7fc86c0b2030 2024-01-27T02:39:04.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.952+0000 7fc869ffb700 1 -- 172.21.15.7:0/2011675904 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 913109373 0 0) 0x7fc86c0ccc20 con 0x7fc86c0ce0f0 2024-01-27T02:39:04.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.952+0000 7fc869ffb700 1 -- 172.21.15.7:0/2011675904 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc86c0cc980 con 0x7fc86c0ce0f0 2024-01-27T02:39:04.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.952+0000 7fc86affd700 1 -- 172.21.15.7:0/2011675904 >> v1:172.21.15.188:6789/0 conn(0x7fc86c0ce0f0 legacy=0x7fc86c0c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:04.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.952+0000 7fc86affd700 1 -- 172.21.15.7:0/2011675904 >> v1:172.21.15.7:6789/0 conn(0x7fc86c0b2030 legacy=0x7fc86c0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:04.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.952+0000 7fc86affd700 1 -- 172.21.15.7:0/2011675904 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c0cd1e0 con 0x7fc86c15a450 2024-01-27T02:39:04.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.952+0000 7fc86affd700 1 --2- 172.21.15.7:0/2011675904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc86c15a450 0x7fc86c0b1af0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009900 tx=0x7fc85c009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92f97ec5a80017e4 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:04.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.952+0000 7fc869ffb700 1 -- 172.21.15.7:0/2011675904 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c005c10 con 0x7fc86c15a450 2024-01-27T02:39:04.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.952+0000 7fc869ffb700 1 -- 172.21.15.7:0/2011675904 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85c005d70 con 0x7fc86c15a450 2024-01-27T02:39:04.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.952+0000 7fc869ffb700 1 -- 172.21.15.7:0/2011675904 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c006260 con 0x7fc86c15a450 2024-01-27T02:39:04.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.953+0000 7fc87191b700 1 -- 172.21.15.7:0/2011675904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc86c15a450 msgr2=0x7fc86c0b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:04.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.953+0000 7fc87191b700 1 --2- 172.21.15.7:0/2011675904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc86c15a450 0x7fc86c0b1af0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009900 tx=0x7fc85c009120 comp rx=0 tx=0).stop 2024-01-27T02:39:04.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.953+0000 7fc87191b700 1 -- 172.21.15.7:0/2011675904 shutdown_connections 2024-01-27T02:39:04.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.953+0000 7fc87191b700 1 --2- 172.21.15.7:0/2011675904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc86c15a450 0x7fc86c0b1af0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:04.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.953+0000 7fc87191b700 1 -- 172.21.15.7:0/2011675904 >> 172.21.15.7:0/2011675904 conn(0x7fc86c0711c0 msgr2=0x7fc86c0ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:04.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.953+0000 7fc87191b700 1 -- 172.21.15.7:0/2011675904 shutdown_connections 2024-01-27T02:39:04.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.953+0000 7fc87191b700 1 -- 172.21.15.7:0/2011675904 wait complete. 2024-01-27T02:39:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.954+0000 7fc87191b700 1 Processor -- start 2024-01-27T02:39:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.954+0000 7fc87191b700 1 -- start start 2024-01-27T02:39:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.955+0000 7fc87191b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86c0b2030 0x7fc86c16a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.955+0000 7fc87191b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc86c0ce0f0 0x7fc86c16d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.955+0000 7fc87191b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc86c15a450 0x7fc86c0c4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.955+0000 7fc87191b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c0c56c0 con 0x7fc86c0ce0f0 2024-01-27T02:39:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.955+0000 7fc87191b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c16ad60 con 0x7fc86c0b2030 2024-01-27T02:39:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.955+0000 7fc87191b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c0c6cd0 con 0x7fc86c15a450 2024-01-27T02:39:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.955+0000 7fc86a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc86c0ce0f0 0x7fc86c16d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.955+0000 7fc86a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc86c0ce0f0 0x7fc86c16d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49734/0 (socket says 172.21.15.7:49734) 2024-01-27T02:39:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.955+0000 7fc86a7fc700 1 -- 172.21.15.7:0/129434851 learned_addr learned my addr 172.21.15.7:0/129434851 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.955+0000 7fc86affd700 1 --2- 172.21.15.7:0/129434851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86c0b2030 0x7fc86c16a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.956+0000 7fc86b7fe700 1 --2- 172.21.15.7:0/129434851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc86c15a450 0x7fc86c0c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.956+0000 7fc86a7fc700 1 -- 172.21.15.7:0/129434851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc86c15a450 msgr2=0x7fc86c0c4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.956+0000 7fc86a7fc700 1 --2- 172.21.15.7:0/129434851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc86c15a450 0x7fc86c0c4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.956+0000 7fc86a7fc700 1 -- 172.21.15.7:0/129434851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86c0b2030 msgr2=0x7fc86c16a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.956+0000 7fc86a7fc700 1 --2- 172.21.15.7:0/129434851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86c0b2030 0x7fc86c16a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.956+0000 7fc86a7fc700 1 -- 172.21.15.7:0/129434851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c0093c0 con 0x7fc86c0ce0f0 2024-01-27T02:39:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.956+0000 7fc86affd700 1 --2- 172.21.15.7:0/129434851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86c0b2030 0x7fc86c16a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:39:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.956+0000 7fc86b7fe700 1 --2- 172.21.15.7:0/129434851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc86c15a450 0x7fc86c0c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:39:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.956+0000 7fc86a7fc700 1 --2- 172.21.15.7:0/129434851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc86c0ce0f0 0x7fc86c16d6a0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fc86000c790 tx=0x7fc860007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.956+0000 7fc852ffd700 1 -- 172.21.15.7:0/129434851 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc860005b30 con 0x7fc86c0ce0f0 2024-01-27T02:39:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.956+0000 7fc852ffd700 1 -- 172.21.15.7:0/129434851 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc860005c90 con 0x7fc86c0ce0f0 2024-01-27T02:39:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.956+0000 7fc87191b700 1 -- 172.21.15.7:0/129434851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86c16dbe0 con 0x7fc86c0ce0f0 2024-01-27T02:39:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.956+0000 7fc852ffd700 1 -- 172.21.15.7:0/129434851 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8600072e0 con 0x7fc86c0ce0f0 2024-01-27T02:39:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.956+0000 7fc87191b700 1 -- 172.21.15.7:0/129434851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86c0c6680 con 0x7fc86c0ce0f0 2024-01-27T02:39:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.957+0000 7fc87191b700 1 -- 172.21.15.7:0/129434851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86c071830 con 0x7fc86c0ce0f0 2024-01-27T02:39:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.957+0000 7fc852ffd700 1 -- 172.21.15.7:0/129434851 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc860007690 con 0x7fc86c0ce0f0 2024-01-27T02:39:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.958+0000 7fc852ffd700 1 --2- 172.21.15.7:0/129434851 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc858066f30 0x7fc8580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.958+0000 7fc852ffd700 1 -- 172.21.15.7:0/129434851 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fc8600839a0 con 0x7fc86c0ce0f0 2024-01-27T02:39:04.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.958+0000 7fc86affd700 1 --2- 172.21.15.7:0/129434851 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc858066f30 0x7fc8580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:04.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.959+0000 7fc86affd700 1 --2- 172.21.15.7:0/129434851 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc858066f30 0x7fc8580693e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009850 tx=0x7fc85c016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:04.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:04.962+0000 7fc852ffd700 1 -- 172.21.15.7:0/129434851 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc86004b5a0 con 0x7fc86c0ce0f0 2024-01-27T02:39:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.168+0000 7fc87191b700 1 -- 172.21.15.7:0/129434851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fc86c04c710 con 0x7fc86c0ce0f0 2024-01-27T02:39:05.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.169+0000 7fc852ffd700 1 -- 172.21.15.7:0/129434851 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v137) v1 ==== 81+0+32610 (secure 0 0 0) 0x7fc860017020 con 0x7fc86c0ce0f0 2024-01-27T02:39:05.170 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:05.170 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:39:05.170 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 137, 2024-01-27T02:39:05.170 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:39:05.170 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:39:05.171 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:38:59.871665+0000", 2024-01-27T02:39:05.171 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:39:05.171 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:39:05.171 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:39:05.171 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:39:05.171 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:39:05.171 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:39:05.171 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:39:05.171 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:39:05.171 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:39:05.171 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:39:05.171 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:39:05.171 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:39:05.171 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:39:05.172 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:39:05.172 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:39:05.172 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 17, 2024-01-27T02:39:05.172 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:39:05.172 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:39:05.172 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-01-27T02:39:05.172 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:39:05.172 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:39:05.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.172 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:39:05.172 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:39:05.172 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:39:05.172 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:39:05.172 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:39:05.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:39:05.173 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:39:05.173 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:39:05.173 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:39:05.173 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:39:05.173 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:39:05.173 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:39:05.173 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:39:05.173 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:39:05.173 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:39:05.173 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:39:05.173 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:39:05.173 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:39:05.173 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:39:05.174 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:39:05.174 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:39:05.174 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:39:05.174 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:39:05.174 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:39:05.174 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:39:05.174 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:39:05.174 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:39:05.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.174 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "106", 2024-01-27T02:39:05.174 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:39:05.174 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:39:05.174 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:39:05.174 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:39:05.174 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:39:05.175 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:39:05.175 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:39:05.175 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:39:05.175 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:39:05.175 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:39:05.175 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:39:05.175 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:39:05.175 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:39:05.175 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:39:05.175 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:39:05.175 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:39:05.175 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:39:05.175 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:39:05.175 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:39:05.176 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:39:05.176 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:39:05.176 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:39:05.176 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:39:05.176 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:39:05.176 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:39:05.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:39:05.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.176 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:39:05.176 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:39:05.176 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:39:05.176 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:39:05.176 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:39:05.176 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:39:05.177 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:39:05.177 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:39:05.177 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:39:05.177 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:39:05.177 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:39:05.177 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:39:05.177 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:39:05.177 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:39:05.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.177 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:39:05.177 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:39:05.177 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.178 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 16, 2024-01-27T02:39:05.178 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:39:05.178 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:38:29.824892+0000", 2024-01-27T02:39:05.178 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:39:05.178 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:39:05.178 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:39:05.178 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:39:05.178 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:39:05.178 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:39:05.178 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:39:05.178 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:39:05.178 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:39:05.178 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:39:05.178 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:39:05.179 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:39:05.179 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:39:05.179 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:39:05.179 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:39:05.179 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:39:05.179 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:39:05.179 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:39:05.179 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:39:05.179 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:39:05.179 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:39:05.179 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:39:05.179 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:39:05.179 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:39:05.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.180 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "131", 2024-01-27T02:39:05.180 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:39:05.180 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:39:05.180 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:39:05.180 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:39:05.180 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:39:05.180 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:39:05.180 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:39:05.180 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:39:05.180 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:39:05.180 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:39:05.180 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:39:05.180 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:39:05.180 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:39:05.180 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:39:05.181 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:39:05.181 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:39:05.181 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:39:05.181 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:39:05.181 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:39:05.181 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:39:05.181 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:39:05.181 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:39:05.181 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:39:05.181 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:39:05.181 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:39:05.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:39:05.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.181 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:39:05.182 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:39:05.182 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:39:05.182 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:39:05.182 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:39:05.182 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:39:05.182 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:39:05.182 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:39:05.182 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:39:05.182 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:39:05.182 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:39:05.182 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:39:05.182 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:39:05.182 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:39:05.182 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:39:05.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.183 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:39:05.183 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:39:05.183 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:39:05.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.183 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 17, 2024-01-27T02:39:05.183 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:39:05.183 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:38:30.301072+0000", 2024-01-27T02:39:05.183 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:39:05.183 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:39:05.183 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:39:05.183 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:39:05.184 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:39:05.184 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:39:05.184 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:39:05.184 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:39:05.184 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:39:05.184 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:39:05.184 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:39:05.184 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:39:05.184 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:39:05.184 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:39:05.184 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:39:05.184 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:39:05.184 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:39:05.184 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:39:05.185 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:39:05.185 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:39:05.185 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:39:05.185 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:39:05.185 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:39:05.185 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:39:05.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.185 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "137", 2024-01-27T02:39:05.185 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:39:05.185 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:39:05.185 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:39:05.185 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:39:05.185 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:39:05.185 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:39:05.186 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:39:05.186 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:39:05.186 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:39:05.186 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-27T02:39:05.186 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:39:05.186 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:39:05.186 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:39:05.186 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:39:05.186 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:39:05.186 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:39:05.186 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:39:05.186 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:39:05.186 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:39:05.187 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:39:05.187 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:39:05.187 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:39:05.187 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:39:05.187 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:39:05.187 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:39:05.187 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:39:05.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.187 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:39:05.187 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:39:05.187 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:39:05.187 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:39:05.187 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:39:05.187 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:39:05.188 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:39:05.188 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:39:05.188 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:39:05.188 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:39:05.188 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:39:05.188 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:39:05.188 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:39:05.188 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:39:05.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.188 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:39:05.188 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:39:05.188 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:39:05.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.189 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.189 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.189 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:39:05.189 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:39:05.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.189 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:39:05.189 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:39:05.189 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:39:05.189 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:39:05.189 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:39:05.189 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:39:05.189 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:39:05.190 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:39:05.190 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:39:05.190 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:39:05.190 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:39:05.190 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:39:05.190 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:39:05.190 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.190 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:39:05.190 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:39:05.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.190 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:39:05.191 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:39:05.191 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.191 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.191 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:39:05.191 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.191 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:39:05.191 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:39:05.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.191 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:39:05.191 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:39:05.192 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.192 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.192 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:39:05.192 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.192 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.192 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:39:05.192 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:39:05.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.192 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.192 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:39:05.193 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:39:05.193 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.193 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.193 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:39:05.193 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.193 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.193 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:39:05.193 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:39:05.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.193 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.193 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:39:05.194 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:39:05.194 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.194 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.194 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:39:05.194 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:39:05.194 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:39:05.194 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:39:05.194 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:39:05.194 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:39:05.194 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:39:05.194 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.195 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:39:05.195 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:39:05.195 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:39:05.195 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:39:05.195 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:39:05.195 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:39:05.195 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:39:05.195 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:39:05.195 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:39:05.195 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:39:05.195 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:39:05.195 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:39:05.196 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:39:05.196 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.196 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.196 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:39:05.196 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:39:05.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.196 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.196 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:39:05.196 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:39:05.196 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.196 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.196 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:39:05.196 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.197 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.197 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:39:05.197 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:39:05.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.197 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.197 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:39:05.197 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:39:05.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.197 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.197 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:39:05.197 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.198 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:39:05.198 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:39:05.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.198 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:39:05.198 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:39:05.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.198 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.198 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:39:05.198 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.199 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:39:05.199 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:39:05.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.199 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:39:05.199 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:39:05.199 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.199 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.199 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:39:05.199 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:39:05.199 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:39:05.200 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:39:05.200 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:39:05.200 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:39:05.200 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:39:05.200 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.200 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:39:05.200 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:39:05.200 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:39:05.200 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:39:05.200 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:39:05.200 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:39:05.200 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:39:05.201 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:39:05.201 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:39:05.201 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 135, 2024-01-27T02:39:05.201 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:39:05.201 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:39:05.201 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:39:05.201 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.201 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.201 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:39:05.201 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:39:05.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.201 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.202 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:39:05.202 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:39:05.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.202 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.202 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:39:05.202 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.202 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:39:05.202 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:39:05.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.203 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:39:05.203 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:39:05.203 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.203 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.203 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:39:05.203 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.203 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.203 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:39:05.203 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:39:05.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.204 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.204 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:39:05.204 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:39:05.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.204 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.204 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:39:05.204 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.204 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.204 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:39:05.204 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:39:05.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.205 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.205 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:39:05.205 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:39:05.205 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.205 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.205 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:39:05.205 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:39:05.205 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:39:05.205 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:39:05.205 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:39:05.205 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:39:05.205 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:39:05.206 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.206 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:39:05.206 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:39:05.206 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:39:05.206 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:39:05.206 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:39:05.206 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:39:05.206 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:39:05.206 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:39:05.206 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:39:05.206 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 135, 2024-01-27T02:39:05.206 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:39:05.206 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:39:05.207 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:39:05.207 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.207 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.207 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:39:05.207 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:39:05.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.207 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.207 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:39:05.207 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:39:05.207 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.207 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.208 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:39:05.208 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.208 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:39:05.208 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:39:05.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.208 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:39:05.208 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:39:05.208 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.208 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.209 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:39:05.209 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.209 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:39:05.209 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:39:05.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.209 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:39:05.209 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:39:05.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.209 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.210 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:39:05.210 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.210 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:39:05.210 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:39:05.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.210 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:39:05.210 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:39:05.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.210 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.211 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:39:05.211 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:39:05.211 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:39:05.211 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:39:05.211 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:39:05.211 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:39:05.211 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:39:05.211 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.211 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:39:05.211 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:39:05.211 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:39:05.211 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:39:05.212 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:39:05.212 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:39:05.212 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:39:05.212 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:39:05.212 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:39:05.212 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:39:05.212 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:39:05.212 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:39:05.212 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:39:05.212 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.212 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.212 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:39:05.213 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:39:05.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.213 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.213 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:39:05.213 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:39:05.213 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.213 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.213 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:39:05.213 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.213 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.213 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:39:05.214 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:39:05.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.214 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.214 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:39:05.214 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:39:05.214 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.214 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.214 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:39:05.214 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.214 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.215 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:39:05.215 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:39:05.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.215 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:39:05.215 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:39:05.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.215 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.215 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:39:05.215 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.216 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:39:05.216 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:39:05.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.216 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.216 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:39:05.216 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:39:05.216 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.216 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.216 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:39:05.216 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:39:05.217 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:39:05.217 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:39:05.217 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:39:05.217 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:39:05.217 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:39:05.217 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.217 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:39:05.217 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:39:05.217 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:39:05.217 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:39:05.217 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:39:05.217 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:39:05.217 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:39:05.218 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:39:05.218 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:39:05.218 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:39:05.218 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:39:05.218 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:39:05.218 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:39:05.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:39:05.218 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:39:05.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:39:05.218 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:39:05.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.219 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.219 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:39:05.219 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.219 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:39:05.219 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:39:05.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.219 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:39:05.219 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:39:05.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.220 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.220 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:39:05.220 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.220 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:39:05.220 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:39:05.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.220 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:39:05.220 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:39:05.221 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.221 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.221 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:39:05.221 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.221 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:39:05.221 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:39:05.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.221 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:39:05.221 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:39:05.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.222 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.222 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:39:05.222 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:39:05.222 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:39:05.222 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:39:05.222 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:39:05.222 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:39:05.222 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:39:05.222 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.222 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:39:05.222 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:39:05.223 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:39:05.223 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:39:05.223 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:39:05.223 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:39:05.223 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:39:05.223 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:39:05.223 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:39:05.223 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:39:05.223 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:39:05.223 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:39:05.223 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:39:05.223 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:39:05.224 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:39:05.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:39:05.224 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:39:05.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.224 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.225 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:39:05.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:39:05.225 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:39:05.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:39:05.225 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:39:05.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.226 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.226 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:39:05.226 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.226 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:39:05.226 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:39:05.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.226 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:39:05.226 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:39:05.227 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.227 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.227 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:39:05.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:39:05.227 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:39:05.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:39:05.228 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:39:05.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.228 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.228 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:39:05.228 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:39:05.228 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:39:05.228 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:39:05.228 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:39:05.228 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:39:05.228 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:39:05.228 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.229 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:39:05.229 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:39:05.229 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:39:05.229 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:39:05.229 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:39:05.229 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:39:05.229 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:39:05.229 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:39:05.229 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:39:05.229 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:39:05.229 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:39:05.229 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:39:05.229 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:39:05.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:39:05.230 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:39:05.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:39:05.230 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:39:05.230 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.230 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.231 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:39:05.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:39:05.231 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:39:05.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:39:05.231 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:39:05.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.231 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.232 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:39:05.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:39:05.232 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:39:05.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:39:05.232 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:39:05.232 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.232 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.233 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:39:05.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:39:05.233 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:39:05.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:39:05.233 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:39:05.234 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.234 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.234 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:39:05.234 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:39:05.234 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:39:05.234 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:39:05.234 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:39:05.234 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:39:05.234 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:39:05.234 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.234 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.234 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:39:05.235 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:39:05.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.235 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:39:05.235 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:39:05.235 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:39:05.235 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:39:05.235 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:39:05.235 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:39:05.235 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:39:05.235 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:39:05.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.235 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:39:05.235 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:39:05.235 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:39:05.236 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:39:05.236 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:39:05.236 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:39:05.236 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:39:05.236 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:39:05.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.236 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:39:05.236 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:39:05.236 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:39:05.236 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:39:05.236 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:39:05.236 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:39:05.236 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:39:05.237 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:39:05.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.237 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:39:05.237 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:39:05.237 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:39:05.237 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:39:05.237 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:39:05.237 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:39:05.237 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:39:05.237 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:39:05.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.237 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:39:05.238 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:39:05.238 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:39:05.238 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:39:05.238 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:39:05.238 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:39:05.238 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:39:05.238 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:39:05.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.238 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:39:05.238 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:39:05.238 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:39:05.238 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:39:05.238 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:39:05.238 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:39:05.239 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:39:05.239 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:39:05.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.239 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:39:05.239 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:39:05.239 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:39:05.239 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:39:05.239 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:39:05.239 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:39:05.239 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:39:05.239 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:39:05.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.239 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:39:05.240 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:39:05.240 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:39:05.240 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:39:05.240 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:39:05.240 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:39:05.240 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:39:05.240 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:39:05.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.240 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:39:05.240 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:39:05.240 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [ 2024-01-27T02:39:05.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.240 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "17.e", 2024-01-27T02:39:05.241 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-01-27T02:39:05.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.241 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 0, 2024-01-27T02:39:05.241 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 5 2024-01-27T02:39:05.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.241 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "17.24", 2024-01-27T02:39:05.241 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-01-27T02:39:05.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.241 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 0, 2024-01-27T02:39:05.241 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 2 2024-01-27T02:39:05.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.242 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.242 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.242 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:39:05.242 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:39:05.242 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:39:05.242 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:39:05.242 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:39:05.242 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:39:05.242 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:39:05.242 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:39:05.242 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:39:05.242 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:39:05.242 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:39:05.242 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:39:05.242 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.243 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:39:05.243 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:39:05.243 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:39:05.243 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:39:05.243 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:39:05.243 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:39:05.243 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:39:05.243 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:39:05.243 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:39:05.243 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:39:05.243 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:39:05.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.243 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:39:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.172+0000 7fc87191b700 1 -- 172.21.15.7:0/129434851 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc858066f30 msgr2=0x7fc8580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.172+0000 7fc87191b700 1 --2- 172.21.15.7:0/129434851 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc858066f30 0x7fc8580693e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009850 tx=0x7fc85c016000 comp rx=0 tx=0).stop 2024-01-27T02:39:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.172+0000 7fc87191b700 1 -- 172.21.15.7:0/129434851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc86c0ce0f0 msgr2=0x7fc86c16d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.172+0000 7fc87191b700 1 --2- 172.21.15.7:0/129434851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc86c0ce0f0 0x7fc86c16d6a0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fc86000c790 tx=0x7fc860007ea0 comp rx=0 tx=0).stop 2024-01-27T02:39:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.172+0000 7fc87191b700 1 -- 172.21.15.7:0/129434851 shutdown_connections 2024-01-27T02:39:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.172+0000 7fc87191b700 1 --2- 172.21.15.7:0/129434851 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc858066f30 0x7fc8580693e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.172+0000 7fc87191b700 1 --2- 172.21.15.7:0/129434851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc86c15a450 0x7fc86c0c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.172+0000 7fc87191b700 1 --2- 172.21.15.7:0/129434851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86c0b2030 0x7fc86c16a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:05.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.172+0000 7fc87191b700 1 --2- 172.21.15.7:0/129434851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc86c0ce0f0 0x7fc86c16d6a0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:05.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.172+0000 7fc87191b700 1 -- 172.21.15.7:0/129434851 >> 172.21.15.7:0/129434851 conn(0x7fc86c0711c0 msgr2=0x7fc86c159670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:05.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.172+0000 7fc87191b700 1 -- 172.21.15.7:0/129434851 shutdown_connections 2024-01-27T02:39:05.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.172+0000 7fc87191b700 1 -- 172.21.15.7:0/129434851 wait complete. 2024-01-27T02:39:05.246 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-01-27T02:39:05.247 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:39:05.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.380+0000 7fbb796d3700 1 Processor -- start 2024-01-27T02:39:05.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.380+0000 7fbb796d3700 1 -- start start 2024-01-27T02:39:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.380+0000 7fbb796d3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb740d10a0 0x7fbb740b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.380+0000 7fbb796d3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb740b4650 con 0x7fbb740d1470 2024-01-27T02:39:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.380+0000 7fbb796d3700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb740b3d60 con 0x7fbb74159b40 2024-01-27T02:39:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.380+0000 7fbb796d3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb740b34d0 con 0x7fbb740d10a0 2024-01-27T02:39:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.381+0000 7fbb727fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbb740d1470 0x7fbb740b2dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59786/0 (socket says 172.21.15.7:59786) 2024-01-27T02:39:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.381+0000 7fbb727fc700 1 -- 172.21.15.7:0/713565210 learned_addr learned my addr 172.21.15.7:0/713565210 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.381+0000 7fbb72ffd700 1 --2- 172.21.15.7:0/713565210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb740d10a0 0x7fbb740b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.381+0000 7fbb71ffb700 1 -- 172.21.15.7:0/713565210 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2681173644 0 0) 0x7fbb740b4650 con 0x7fbb740d1470 2024-01-27T02:39:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.381+0000 7fbb71ffb700 1 -- 172.21.15.7:0/713565210 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb5c003570 con 0x7fbb740d1470 2024-01-27T02:39:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.381+0000 7fbb71ffb700 1 -- 172.21.15.7:0/713565210 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655967563 0 0) 0x7fbb740b3d60 con 0x7fbb74159b40 2024-01-27T02:39:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.381+0000 7fbb71ffb700 1 -- 172.21.15.7:0/713565210 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb740b4650 con 0x7fbb74159b40 2024-01-27T02:39:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.381+0000 7fbb71ffb700 1 -- 172.21.15.7:0/713565210 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 52775762 0 0) 0x7fbb5c003570 con 0x7fbb740d1470 2024-01-27T02:39:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.381+0000 7fbb71ffb700 1 -- 172.21.15.7:0/713565210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb740d10a0 msgr2=0x7fbb740b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:05.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.381+0000 7fbb71ffb700 1 --2- 172.21.15.7:0/713565210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb740d10a0 0x7fbb740b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:05.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.381+0000 7fbb71ffb700 1 -- 172.21.15.7:0/713565210 >> v1:172.21.15.188:6789/0 conn(0x7fbb74159b40 legacy=0x7fbb74159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:05.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.381+0000 7fbb71ffb700 1 -- 172.21.15.7:0/713565210 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb740b4000 con 0x7fbb740d1470 2024-01-27T02:39:05.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.381+0000 7fbb71ffb700 1 -- 172.21.15.7:0/713565210 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7fbb68001470 con 0x7fbb740d1470 2024-01-27T02:39:05.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.381+0000 7fbb71ffb700 1 -- 172.21.15.7:0/713565210 >> v1:172.21.15.7:6789/0 conn(0x7fbb740d1470 legacy=0x7fbb740b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:05.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.381+0000 7fbb71ffb700 1 --2- 172.21.15.7:0/713565210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb5c003c20 0x7fbb5c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:05.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.381+0000 7fbb71ffb700 1 --2- 172.21.15.7:0/713565210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5c004810 0x7fbb5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:05.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.381+0000 7fbb71ffb700 1 --2- 172.21.15.7:0/713565210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb740d10a0 0x7fbb5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:05.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.381+0000 7fbb71ffb700 1 -- 172.21.15.7:0/713565210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb740b34d0 con 0x7fbb5c004810 2024-01-27T02:39:05.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.381+0000 7fbb71ffb700 1 -- 172.21.15.7:0/713565210 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c00bb40 con 0x7fbb5c003c20 2024-01-27T02:39:05.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.381+0000 7fbb71ffb700 1 -- 172.21.15.7:0/713565210 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c00bd00 con 0x7fbb740d10a0 2024-01-27T02:39:05.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.382+0000 7fbb727fc700 1 --2- 172.21.15.7:0/713565210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5c004810 0x7fbb5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:05.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.382+0000 7fbb72ffd700 1 --2- 172.21.15.7:0/713565210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb5c003c20 0x7fbb5c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.382+0000 7fbb737fe700 1 --2- 172.21.15.7:0/713565210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb740d10a0 0x7fbb5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.382+0000 7fbb727fc700 1 -- 172.21.15.7:0/713565210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb740d10a0 msgr2=0x7fbb5c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.382+0000 7fbb727fc700 1 --2- 172.21.15.7:0/713565210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb740d10a0 0x7fbb5c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.382+0000 7fbb727fc700 1 -- 172.21.15.7:0/713565210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb5c003c20 msgr2=0x7fbb5c004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.382+0000 7fbb727fc700 1 --2- 172.21.15.7:0/713565210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb5c003c20 0x7fbb5c004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.382+0000 7fbb727fc700 1 -- 172.21.15.7:0/713565210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c003570 con 0x7fbb5c004810 2024-01-27T02:39:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.382+0000 7fbb737fe700 1 --2- 172.21.15.7:0/713565210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb740d10a0 0x7fbb5c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.382+0000 7fbb727fc700 1 --2- 172.21.15.7:0/713565210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5c004810 0x7fbb5c004c80 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800c2d0 tx=0x7fbb740b3ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d34e8f1a4b3ec5 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.382+0000 7fbb71ffb700 1 -- 172.21.15.7:0/713565210 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6800cc00 con 0x7fbb5c004810 2024-01-27T02:39:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.382+0000 7fbb71ffb700 1 -- 172.21.15.7:0/713565210 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6800cda0 con 0x7fbb5c004810 2024-01-27T02:39:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.382+0000 7fbb71ffb700 1 -- 172.21.15.7:0/713565210 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb68005b70 con 0x7fbb5c004810 2024-01-27T02:39:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.383+0000 7fbb796d3700 1 -- 172.21.15.7:0/713565210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5c004810 msgr2=0x7fbb5c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.383+0000 7fbb796d3700 1 --2- 172.21.15.7:0/713565210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5c004810 0x7fbb5c004c80 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800c2d0 tx=0x7fbb740b3ef0 comp rx=0 tx=0).stop 2024-01-27T02:39:05.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.383+0000 7fbb737fe700 1 -- 172.21.15.7:0/713565210 reap_dead start 2024-01-27T02:39:05.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.383+0000 7fbb796d3700 1 -- 172.21.15.7:0/713565210 shutdown_connections 2024-01-27T02:39:05.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.383+0000 7fbb796d3700 1 -- 172.21.15.7:0/713565210 >> 172.21.15.7:0/713565210 conn(0x7fbb740679b0 msgr2=0x7fbb74067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:05.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.383+0000 7fbb796d3700 1 -- 172.21.15.7:0/713565210 shutdown_connections 2024-01-27T02:39:05.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.383+0000 7fbb796d3700 1 -- 172.21.15.7:0/713565210 wait complete. 2024-01-27T02:39:05.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.384+0000 7fbb796d3700 1 Processor -- start 2024-01-27T02:39:05.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.384+0000 7fbb796d3700 1 -- start start 2024-01-27T02:39:05.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.385+0000 7fbb796d3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb5c004810 0x7fbb7415c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:05.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.385+0000 7fbb796d3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb740caca0 0x7fbb7414c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:05.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.385+0000 7fbb796d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb7415dc20 0x7fbb7415e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:05.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.385+0000 7fbb796d3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7415e5e0 con 0x7fbb7415dc20 2024-01-27T02:39:05.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.385+0000 7fbb796d3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb740ca810 con 0x7fbb5c004810 2024-01-27T02:39:05.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.385+0000 7fbb796d3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb740ca200 con 0x7fbb740caca0 2024-01-27T02:39:05.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.385+0000 7fbb737fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb7415dc20 0x7fbb7415e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:05.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.385+0000 7fbb737fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb7415dc20 0x7fbb7415e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49750/0 (socket says 172.21.15.7:49750) 2024-01-27T02:39:05.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.385+0000 7fbb737fe700 1 -- 172.21.15.7:0/3551103072 learned_addr learned my addr 172.21.15.7:0/3551103072 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:05.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.385+0000 7fbb72ffd700 1 --2- 172.21.15.7:0/3551103072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb5c004810 0x7fbb7415c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:05.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.385+0000 7fbb737fe700 1 -- 172.21.15.7:0/3551103072 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb740caca0 msgr2=0x7fbb7414c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:05.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.385+0000 7fbb727fc700 1 --2- 172.21.15.7:0/3551103072 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb740caca0 0x7fbb7414c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:05.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.385+0000 7fbb737fe700 1 --2- 172.21.15.7:0/3551103072 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb740caca0 0x7fbb7414c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:05.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.385+0000 7fbb737fe700 1 -- 172.21.15.7:0/3551103072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb5c004810 msgr2=0x7fbb7415c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:05.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.385+0000 7fbb737fe700 1 --2- 172.21.15.7:0/3551103072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb5c004810 0x7fbb7415c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:05.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.385+0000 7fbb737fe700 1 -- 172.21.15.7:0/3551103072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7414c870 con 0x7fbb7415dc20 2024-01-27T02:39:05.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.385+0000 7fbb72ffd700 1 --2- 172.21.15.7:0/3551103072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb5c004810 0x7fbb7415c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:39:05.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.385+0000 7fbb727fc700 1 --2- 172.21.15.7:0/3551103072 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb740caca0 0x7fbb7414c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:39:05.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.386+0000 7fbb737fe700 1 --2- 172.21.15.7:0/3551103072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb7415dc20 0x7fbb7415e0a0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fbb740b45c0 tx=0x7fbb6c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:05.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.386+0000 7fbb5affd700 1 -- 172.21.15.7:0/3551103072 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c00ee70 con 0x7fbb7415dc20 2024-01-27T02:39:05.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.386+0000 7fbb5affd700 1 -- 172.21.15.7:0/3551103072 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c006ba0 con 0x7fbb7415dc20 2024-01-27T02:39:05.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.386+0000 7fbb796d3700 1 -- 172.21.15.7:0/3551103072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb740c7560 con 0x7fbb7415dc20 2024-01-27T02:39:05.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.386+0000 7fbb5affd700 1 -- 172.21.15.7:0/3551103072 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c0032e0 con 0x7fbb7415dc20 2024-01-27T02:39:05.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.386+0000 7fbb796d3700 1 -- 172.21.15.7:0/3551103072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb740c7840 con 0x7fbb7415dc20 2024-01-27T02:39:05.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.387+0000 7fbb796d3700 1 -- 172.21.15.7:0/3551103072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb740ca9a0 con 0x7fbb7415dc20 2024-01-27T02:39:05.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.387+0000 7fbb5affd700 1 -- 172.21.15.7:0/3551103072 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbb6c001720 con 0x7fbb7415dc20 2024-01-27T02:39:05.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.388+0000 7fbb5affd700 1 --2- 172.21.15.7:0/3551103072 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb60066f30 0x7fbb600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:05.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.388+0000 7fbb5affd700 1 -- 172.21.15.7:0/3551103072 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fbb6c083b70 con 0x7fbb7415dc20 2024-01-27T02:39:05.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.388+0000 7fbb72ffd700 1 --2- 172.21.15.7:0/3551103072 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb60066f30 0x7fbb600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:05.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.389+0000 7fbb72ffd700 1 --2- 172.21.15.7:0/3551103072 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb60066f30 0x7fbb600693e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbb64006dc0 tx=0x7fbb64008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:05.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.392+0000 7fbb5affd700 1 -- 172.21.15.7:0/3551103072 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb6c04d5a0 con 0x7fbb7415dc20 2024-01-27T02:39:05.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.595+0000 7fbb796d3700 1 -- 172.21.15.7:0/3551103072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fbb7404c710 con 0x7fbb7415dc20 2024-01-27T02:39:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.596+0000 7fbb5affd700 1 -- 172.21.15.7:0/3551103072 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v137) v1 ==== 81+0+32610 (secure 0 0 0) 0x7fbb6c019090 con 0x7fbb7415dc20 2024-01-27T02:39:05.597 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:05.597 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:39:05.597 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 137, 2024-01-27T02:39:05.597 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:39:05.597 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:39:05.597 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:38:59.871665+0000", 2024-01-27T02:39:05.597 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:39:05.597 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:39:05.598 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:39:05.598 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:39:05.598 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:39:05.598 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:39:05.598 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:39:05.598 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:39:05.598 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:39:05.598 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:39:05.598 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:39:05.598 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:39:05.599 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:39:05.599 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:39:05.599 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:39:05.599 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 17, 2024-01-27T02:39:05.599 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:39:05.599 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:39:05.599 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-01-27T02:39:05.599 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:39:05.599 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:39:05.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.599 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:39:05.599 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:39:05.599 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:39:05.600 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:39:05.600 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:39:05.600 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:39:05.600 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:39:05.600 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:39:05.600 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:39:05.600 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:39:05.600 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:39:05.600 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:39:05.600 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:39:05.600 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:39:05.601 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:39:05.601 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:39:05.601 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:39:05.601 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:39:05.601 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:39:05.601 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:39:05.601 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:39:05.601 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:39:05.601 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:39:05.601 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:39:05.601 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:39:05.601 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:39:05.602 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:39:05.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.602 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "106", 2024-01-27T02:39:05.602 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:39:05.602 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:39:05.602 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:39:05.602 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:39:05.602 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:39:05.602 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:39:05.602 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:39:05.602 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:39:05.603 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:39:05.603 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:39:05.603 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:39:05.603 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:39:05.603 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:39:05.603 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:39:05.603 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:39:05.603 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:39:05.603 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:39:05.603 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:39:05.603 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:39:05.603 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:39:05.604 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:39:05.604 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:39:05.604 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:39:05.604 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:39:05.604 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:39:05.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:39:05.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.604 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:39:05.604 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:39:05.605 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:39:05.605 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:39:05.605 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:39:05.605 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:39:05.605 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:39:05.605 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:39:05.605 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:39:05.605 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:39:05.605 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:39:05.605 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:39:05.605 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:39:05.605 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:39:05.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.606 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:39:05.606 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:39:05.606 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.606 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 16, 2024-01-27T02:39:05.606 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:39:05.606 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:38:29.824892+0000", 2024-01-27T02:39:05.606 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:39:05.606 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:39:05.606 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:39:05.607 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:39:05.607 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:39:05.607 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:39:05.607 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:39:05.607 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:39:05.607 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:39:05.607 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:39:05.607 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:39:05.607 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:39:05.607 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:39:05.607 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:39:05.608 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:39:05.608 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:39:05.608 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:39:05.608 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:39:05.608 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:39:05.608 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:39:05.608 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:39:05.608 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:39:05.608 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:39:05.608 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:39:05.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.608 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "131", 2024-01-27T02:39:05.609 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:39:05.609 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:39:05.609 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:39:05.609 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:39:05.609 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:39:05.609 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:39:05.609 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:39:05.609 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:39:05.609 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:39:05.609 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:39:05.609 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:39:05.610 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:39:05.610 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:39:05.610 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:39:05.610 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:39:05.610 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:39:05.610 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:39:05.610 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:39:05.610 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:39:05.610 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:39:05.610 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:39:05.610 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:39:05.611 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:39:05.611 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:39:05.611 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:39:05.611 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:39:05.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.611 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:39:05.611 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:39:05.611 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:39:05.611 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:39:05.611 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:39:05.611 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:39:05.612 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:39:05.612 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:39:05.612 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:39:05.612 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:39:05.612 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:39:05.612 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:39:05.612 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:39:05.612 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:39:05.612 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:39:05.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.613 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:39:05.613 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:39:05.613 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:39:05.613 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.613 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.613 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 17, 2024-01-27T02:39:05.613 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:39:05.613 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:38:30.301072+0000", 2024-01-27T02:39:05.613 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:39:05.613 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:39:05.614 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:39:05.614 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:39:05.614 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:39:05.614 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:39:05.614 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:39:05.614 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:39:05.614 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:39:05.614 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:39:05.614 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:39:05.614 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:39:05.614 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:39:05.614 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:39:05.615 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:39:05.615 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:39:05.615 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:39:05.615 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:39:05.615 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:39:05.615 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:39:05.615 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:39:05.615 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:39:05.615 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:39:05.615 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:39:05.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.615 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "137", 2024-01-27T02:39:05.616 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:39:05.616 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:39:05.616 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:39:05.616 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:39:05.616 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:39:05.616 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:39:05.616 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:39:05.616 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:39:05.616 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:39:05.617 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-27T02:39:05.617 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:39:05.617 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:39:05.617 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:39:05.617 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:39:05.617 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:39:05.617 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:39:05.617 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:39:05.617 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:39:05.617 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:39:05.617 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:39:05.617 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:39:05.617 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:39:05.618 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:39:05.618 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:39:05.618 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:39:05.618 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:39:05.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.618 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:39:05.618 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:39:05.618 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:39:05.618 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:39:05.618 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:39:05.618 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:39:05.619 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:39:05.619 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:39:05.619 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:39:05.619 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:39:05.619 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:39:05.619 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:39:05.619 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:39:05.619 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:39:05.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.619 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:39:05.619 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:39:05.619 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:39:05.620 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.620 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.620 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.620 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:39:05.620 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:39:05.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.620 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:39:05.620 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:39:05.620 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:39:05.620 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:39:05.620 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:39:05.620 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:39:05.621 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:39:05.621 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:39:05.621 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:39:05.621 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:39:05.621 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:39:05.621 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:39:05.621 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:39:05.621 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.621 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.621 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:39:05.622 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:39:05.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.622 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.622 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:39:05.622 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:39:05.622 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.622 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.622 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:39:05.622 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.622 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.623 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:39:05.623 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:39:05.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.623 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.623 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:39:05.623 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:39:05.623 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.623 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.623 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:39:05.624 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.624 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.624 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:39:05.624 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:39:05.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.624 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.624 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:39:05.624 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:39:05.624 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.625 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.625 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:39:05.625 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.625 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.625 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:39:05.625 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:39:05.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.625 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.625 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:39:05.626 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:39:05.626 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.626 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.626 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:39:05.626 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:39:05.626 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:39:05.626 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:39:05.626 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:39:05.626 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:39:05.626 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:39:05.627 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.627 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:39:05.627 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:39:05.627 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:39:05.627 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:39:05.627 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:39:05.627 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:39:05.627 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:39:05.627 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:39:05.627 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:39:05.628 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:39:05.628 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:39:05.628 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:39:05.628 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:39:05.628 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.628 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.628 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:39:05.628 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:39:05.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.629 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.629 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:39:05.629 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:39:05.629 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.629 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.629 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:39:05.629 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.629 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.629 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:39:05.629 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:39:05.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.630 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.630 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:39:05.630 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:39:05.630 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.630 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.630 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:39:05.630 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.630 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.631 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:39:05.631 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:39:05.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.631 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.631 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:39:05.631 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:39:05.631 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.631 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.632 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:39:05.632 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.632 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.632 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:39:05.632 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:39:05.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.632 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.632 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:39:05.632 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:39:05.632 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.633 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.633 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:39:05.633 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:39:05.633 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:39:05.633 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:39:05.633 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:39:05.633 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:39:05.633 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:39:05.633 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.634 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:39:05.634 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:39:05.634 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:39:05.634 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:39:05.634 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:39:05.634 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:39:05.634 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:39:05.634 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:39:05.634 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:39:05.634 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 135, 2024-01-27T02:39:05.634 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:39:05.634 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:39:05.635 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:39:05.635 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.635 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.635 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:39:05.635 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:39:05.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.635 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.635 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:39:05.635 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:39:05.635 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.636 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.636 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:39:05.636 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.636 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.636 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:39:05.636 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:39:05.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.636 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.637 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:39:05.637 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:39:05.637 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.637 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.637 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:39:05.637 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.637 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.637 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:39:05.637 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:39:05.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.638 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.638 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:39:05.638 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:39:05.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.638 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.638 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:39:05.638 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.638 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.638 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:39:05.638 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:39:05.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.639 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:39:05.639 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:39:05.639 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.639 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.639 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:39:05.639 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:39:05.639 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:39:05.639 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:39:05.639 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:39:05.640 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:39:05.640 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:39:05.640 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.640 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:39:05.640 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:39:05.640 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:39:05.640 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:39:05.640 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:39:05.640 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:39:05.640 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:39:05.641 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:39:05.641 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:39:05.641 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 135, 2024-01-27T02:39:05.641 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:39:05.641 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:39:05.641 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:39:05.641 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.641 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.641 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:39:05.642 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:39:05.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.642 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.642 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:39:05.642 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:39:05.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.642 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.642 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:39:05.642 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.642 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.643 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:39:05.643 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:39:05.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.643 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.643 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:39:05.643 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:39:05.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.643 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.643 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:39:05.644 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.644 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.644 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:39:05.644 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:39:05.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.644 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.644 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:39:05.644 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:39:05.644 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.645 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.645 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:39:05.645 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.645 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.645 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:39:05.645 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:39:05.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.645 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.645 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:39:05.646 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:39:05.646 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.646 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.646 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:39:05.646 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:39:05.646 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:39:05.646 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:39:05.646 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:39:05.646 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:39:05.646 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:39:05.647 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.647 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:39:05.647 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:39:05.647 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:39:05.647 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:39:05.647 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:39:05.647 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:39:05.647 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:39:05.647 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:39:05.647 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:39:05.648 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:39:05.648 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:39:05.648 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:39:05.648 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:39:05.648 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.648 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.648 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:39:05.648 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:39:05.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.649 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.649 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:39:05.649 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:39:05.649 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.649 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.649 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:39:05.649 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.650 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.650 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:39:05.650 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:39:05.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.650 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.650 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:39:05.650 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:39:05.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.650 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.650 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:39:05.651 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.651 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.651 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:39:05.651 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:39:05.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.651 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.651 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:39:05.651 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:39:05.651 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.651 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.652 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:39:05.652 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.652 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.652 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:39:05.652 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:39:05.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.652 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.653 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:39:05.653 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:39:05.653 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.653 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.653 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:39:05.653 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:39:05.653 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:39:05.653 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:39:05.653 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:39:05.653 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:39:05.654 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:39:05.654 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.654 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:39:05.654 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:39:05.654 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:39:05.654 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:39:05.654 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:39:05.654 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:39:05.654 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:39:05.654 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:39:05.655 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:39:05.655 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:39:05.655 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:39:05.655 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:39:05.655 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:39:05.655 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.655 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.655 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:39:05.655 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:39:05.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.656 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.656 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:39:05.656 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:39:05.656 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.656 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.656 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:39:05.656 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.656 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.656 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:39:05.656 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:39:05.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.657 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:39:05.657 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:39:05.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.657 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.657 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:39:05.657 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.658 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:39:05.658 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:39:05.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.658 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.658 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:39:05.658 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:39:05.658 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.658 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.658 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:39:05.658 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.659 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.659 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:39:05.659 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:39:05.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.659 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.659 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:39:05.659 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:39:05.659 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.659 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.659 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:39:05.660 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:39:05.660 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:39:05.660 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:39:05.660 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:39:05.660 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:39:05.660 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:39:05.660 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.660 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:39:05.660 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:39:05.661 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:39:05.661 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:39:05.661 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:39:05.661 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:39:05.661 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:39:05.661 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:39:05.661 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:39:05.661 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:39:05.661 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:39:05.661 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:39:05.661 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:39:05.661 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.661 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.662 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:39:05.662 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:39:05.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.662 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.662 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:39:05.662 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:39:05.662 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.662 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:39:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:39:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:39:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:39:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:39:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.664 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.664 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:39:05.664 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.664 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:39:05.664 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:39:05.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.665 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:39:05.665 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:39:05.665 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.665 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.665 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:39:05.665 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.665 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.665 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:39:05.666 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:39:05.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.666 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.666 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:39:05.666 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:39:05.666 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.666 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.666 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:39:05.666 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:39:05.666 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:39:05.667 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:39:05.667 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:39:05.667 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:39:05.667 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:39:05.667 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.667 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:39:05.667 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:39:05.667 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:39:05.667 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:39:05.667 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:39:05.668 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:39:05.668 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:39:05.668 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:39:05.668 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:39:05.668 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-01-27T02:39:05.668 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:39:05.668 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:39:05.668 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:39:05.668 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.668 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.668 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:39:05.669 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:39:05.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.669 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.669 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:39:05.669 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:39:05.669 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.669 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.669 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:39:05.669 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.670 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.670 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:39:05.670 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:39:05.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.670 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.670 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:39:05.670 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:39:05.670 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.670 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.670 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:39:05.671 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.671 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.671 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:39:05.671 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:39:05.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.671 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.671 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:39:05.671 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:39:05.671 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.672 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.672 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:39:05.672 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:39:05.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:39:05.672 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:39:05.672 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:39:05.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:39:05.672 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:39:05.673 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:39:05.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.673 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.673 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:39:05.673 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:39:05.673 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:39:05.673 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:39:05.673 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:39:05.673 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:39:05.673 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:39:05.673 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.674 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.674 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:39:05.674 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:39:05.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.674 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:39:05.674 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:39:05.674 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:39:05.674 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:39:05.674 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:39:05.674 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:39:05.674 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:39:05.674 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:39:05.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.675 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:39:05.675 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:39:05.675 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:39:05.675 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:39:05.675 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:39:05.675 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:39:05.675 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:39:05.675 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:39:05.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.676 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:39:05.676 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:39:05.676 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:39:05.676 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:39:05.676 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:39:05.676 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:39:05.676 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:39:05.676 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:39:05.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.677 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:39:05.677 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:39:05.677 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:39:05.677 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:39:05.677 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:39:05.677 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:39:05.677 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:39:05.677 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:39:05.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.677 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:39:05.677 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:39:05.678 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:39:05.678 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:39:05.678 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:39:05.678 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:39:05.678 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:39:05.678 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:39:05.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.678 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:39:05.678 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:39:05.678 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:39:05.679 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:39:05.679 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:39:05.679 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:39:05.679 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:39:05.679 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:39:05.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.679 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:39:05.679 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:39:05.679 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:39:05.679 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:39:05.679 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:39:05.680 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:39:05.680 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:39:05.680 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:39:05.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.680 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:39:05.680 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:39:05.680 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:39:05.680 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:39:05.680 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:39:05.680 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:39:05.680 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:39:05.680 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:39:05.681 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.681 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:39:05.681 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:39:05.681 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [ 2024-01-27T02:39:05.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.681 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "17.e", 2024-01-27T02:39:05.681 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-01-27T02:39:05.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.681 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 0, 2024-01-27T02:39:05.681 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 5 2024-01-27T02:39:05.681 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.681 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.682 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "17.24", 2024-01-27T02:39:05.682 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-01-27T02:39:05.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:05.682 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 0, 2024-01-27T02:39:05.682 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 2 2024-01-27T02:39:05.682 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.682 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:05.682 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.682 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:39:05.682 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:39:05.682 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:39:05.682 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:39:05.683 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:39:05.683 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:39:05.683 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:39:05.683 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:39:05.683 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:39:05.683 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:39:05.683 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:39:05.683 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:39:05.683 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:05.683 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:39:05.683 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:39:05.684 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:39:05.684 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:39:05.684 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:39:05.684 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:39:05.684 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:39:05.684 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:39:05.684 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:39:05.684 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:39:05.684 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:39:05.684 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:05.684 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:39:05.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.598+0000 7fbb796d3700 1 -- 172.21.15.7:0/3551103072 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb60066f30 msgr2=0x7fbb600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:05.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.598+0000 7fbb796d3700 1 --2- 172.21.15.7:0/3551103072 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb60066f30 0x7fbb600693e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbb64006dc0 tx=0x7fbb64008040 comp rx=0 tx=0).stop 2024-01-27T02:39:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.598+0000 7fbb796d3700 1 -- 172.21.15.7:0/3551103072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb7415dc20 msgr2=0x7fbb7415e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.598+0000 7fbb796d3700 1 --2- 172.21.15.7:0/3551103072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb7415dc20 0x7fbb7415e0a0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fbb740b45c0 tx=0x7fbb6c009040 comp rx=0 tx=0).stop 2024-01-27T02:39:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.598+0000 7fbb796d3700 1 -- 172.21.15.7:0/3551103072 shutdown_connections 2024-01-27T02:39:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.598+0000 7fbb796d3700 1 --2- 172.21.15.7:0/3551103072 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb60066f30 0x7fbb600693e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.598+0000 7fbb796d3700 1 --2- 172.21.15.7:0/3551103072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb7415dc20 0x7fbb7415e0a0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.598+0000 7fbb796d3700 1 --2- 172.21.15.7:0/3551103072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb5c004810 0x7fbb7415c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.598+0000 7fbb796d3700 1 --2- 172.21.15.7:0/3551103072 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb740caca0 0x7fbb7414c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.598+0000 7fbb796d3700 1 -- 172.21.15.7:0/3551103072 >> 172.21.15.7:0/3551103072 conn(0x7fbb740679b0 msgr2=0x7fbb740cd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.598+0000 7fbb796d3700 1 -- 172.21.15.7:0/3551103072 shutdown_connections 2024-01-27T02:39:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.598+0000 7fbb796d3700 1 -- 172.21.15.7:0/3551103072 wait complete. 2024-01-27T02:39:05.688 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:39:05.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.822+0000 7fea96e0c700 1 Processor -- start 2024-01-27T02:39:05.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.823+0000 7fea96e0c700 1 -- start start 2024-01-27T02:39:05.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.823+0000 7fea96e0c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea900c4cc0 0x7fea900cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:05.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.823+0000 7fea96e0c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fea900ca540 0x7fea90163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:05.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.823+0000 7fea96e0c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea900ce5e0 con 0x7fea9015a100 2024-01-27T02:39:05.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.823+0000 7fea96e0c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea900cd9d0 con 0x7fea900ca540 2024-01-27T02:39:05.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.823+0000 7fea96e0c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea900cdcb0 con 0x7fea900c4cc0 2024-01-27T02:39:05.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.824+0000 7fea953a9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fea900ca540 0x7fea90163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:05.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.824+0000 7fea94ba8700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fea9015a100 0x7fea900c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59800/0 (socket says 172.21.15.7:59800) 2024-01-27T02:39:05.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.824+0000 7fea94ba8700 1 -- 172.21.15.7:0/253826481 learned_addr learned my addr 172.21.15.7:0/253826481 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:05.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.824+0000 7fea8ffff700 1 --2- 172.21.15.7:0/253826481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea900c4cc0 0x7fea900cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:05.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.824+0000 7fea953a9700 1 -- 172.21.15.7:0/253826481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea900c4cc0 msgr2=0x7fea900cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:05.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.824+0000 7fea953a9700 1 --2- 172.21.15.7:0/253826481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea900c4cc0 0x7fea900cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:05.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.824+0000 7fea953a9700 1 -- 172.21.15.7:0/253826481 >> v1:172.21.15.7:6789/0 conn(0x7fea9015a100 legacy=0x7fea900c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:05.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.824+0000 7fea953a9700 1 -- 172.21.15.7:0/253826481 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea900cdf90 con 0x7fea900ca540 2024-01-27T02:39:05.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.824+0000 7fea8f7fe700 1 -- 172.21.15.7:0/253826481 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992372510 0 0) 0x7fea900ce5e0 con 0x7fea9015a100 2024-01-27T02:39:05.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.824+0000 7fea953a9700 1 --2- 172.21.15.7:0/253826481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea900ca540 0x7fea90163d90 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fea88009e60 tx=0x7fea88009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c77222a70ed67b3 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:05.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.824+0000 7fea8f7fe700 1 -- 172.21.15.7:0/253826481 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8800e070 con 0x7fea900ca540 2024-01-27T02:39:05.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.824+0000 7fea8f7fe700 1 -- 172.21.15.7:0/253826481 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8800bc30 con 0x7fea900ca540 2024-01-27T02:39:05.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.824+0000 7fea8f7fe700 1 -- 172.21.15.7:0/253826481 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea88012430 con 0x7fea900ca540 2024-01-27T02:39:05.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.825+0000 7fea96e0c700 1 -- 172.21.15.7:0/253826481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea900ca540 msgr2=0x7fea90163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:05.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.825+0000 7fea96e0c700 1 --2- 172.21.15.7:0/253826481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea900ca540 0x7fea90163d90 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fea88009e60 tx=0x7fea88009780 comp rx=0 tx=0).stop 2024-01-27T02:39:05.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.825+0000 7fea96e0c700 1 -- 172.21.15.7:0/253826481 shutdown_connections 2024-01-27T02:39:05.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.825+0000 7fea96e0c700 1 --2- 172.21.15.7:0/253826481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea900ca540 0x7fea90163d90 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:05.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.825+0000 7fea96e0c700 1 --2- 172.21.15.7:0/253826481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea900c4cc0 0x7fea900cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:05.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.825+0000 7fea96e0c700 1 -- 172.21.15.7:0/253826481 >> 172.21.15.7:0/253826481 conn(0x7fea90098020 msgr2=0x7fea900711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:05.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.825+0000 7fea96e0c700 1 -- 172.21.15.7:0/253826481 shutdown_connections 2024-01-27T02:39:05.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.825+0000 7fea96e0c700 1 -- 172.21.15.7:0/253826481 wait complete. 2024-01-27T02:39:05.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.827+0000 7fea96e0c700 1 Processor -- start 2024-01-27T02:39:05.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.827+0000 7fea96e0c700 1 -- start start 2024-01-27T02:39:05.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.828+0000 7fea96e0c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea900c4cc0 0x7fea90153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:05.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.828+0000 7fea96e0c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea900ca540 0x7fea900b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:05.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.828+0000 7fea96e0c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea9015a100 0x7fea9015bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:05.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.828+0000 7fea96e0c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea900b3dd0 con 0x7fea900ca540 2024-01-27T02:39:05.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.828+0000 7fea96e0c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea900b40b0 con 0x7fea9015a100 2024-01-27T02:39:05.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.828+0000 7fea96e0c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea900b4390 con 0x7fea900c4cc0 2024-01-27T02:39:05.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.828+0000 7fea8ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea900ca540 0x7fea900b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:05.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.828+0000 7fea8ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea900ca540 0x7fea900b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49768/0 (socket says 172.21.15.7:49768) 2024-01-27T02:39:05.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.828+0000 7fea8ffff700 1 -- 172.21.15.7:0/3106175839 learned_addr learned my addr 172.21.15.7:0/3106175839 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:05.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.828+0000 7fea94ba8700 1 --2- 172.21.15.7:0/3106175839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea900c4cc0 0x7fea90153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:05.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.828+0000 7fea953a9700 1 --2- 172.21.15.7:0/3106175839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea9015a100 0x7fea9015bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:05.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.828+0000 7fea8ffff700 1 -- 172.21.15.7:0/3106175839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea900c4cc0 msgr2=0x7fea90153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:05.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.828+0000 7fea8ffff700 1 --2- 172.21.15.7:0/3106175839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea900c4cc0 0x7fea90153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:05.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.828+0000 7fea8ffff700 1 -- 172.21.15.7:0/3106175839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea9015a100 msgr2=0x7fea9015bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:05.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.828+0000 7fea8ffff700 1 --2- 172.21.15.7:0/3106175839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea9015a100 0x7fea9015bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:05.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.828+0000 7fea8ffff700 1 -- 172.21.15.7:0/3106175839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea88009a20 con 0x7fea900ca540 2024-01-27T02:39:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.829+0000 7fea8ffff700 1 --2- 172.21.15.7:0/3106175839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea900ca540 0x7fea900b3480 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fea8400d860 tx=0x7fea84008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.829+0000 7fea8cff9700 1 -- 172.21.15.7:0/3106175839 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84009610 con 0x7fea900ca540 2024-01-27T02:39:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.829+0000 7fea8cff9700 1 -- 172.21.15.7:0/3106175839 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea84009770 con 0x7fea900ca540 2024-01-27T02:39:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.829+0000 7fea96e0c700 1 -- 172.21.15.7:0/3106175839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea900b4670 con 0x7fea900ca540 2024-01-27T02:39:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.829+0000 7fea8cff9700 1 -- 172.21.15.7:0/3106175839 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84009b30 con 0x7fea900ca540 2024-01-27T02:39:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.829+0000 7fea96e0c700 1 -- 172.21.15.7:0/3106175839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea900b3b80 con 0x7fea900ca540 2024-01-27T02:39:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.830+0000 7fea96e0c700 1 -- 172.21.15.7:0/3106175839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea9004c710 con 0x7fea900ca540 2024-01-27T02:39:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.830+0000 7fea8cff9700 1 -- 172.21.15.7:0/3106175839 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fea8401e460 con 0x7fea900ca540 2024-01-27T02:39:05.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.831+0000 7fea8cff9700 1 --2- 172.21.15.7:0/3106175839 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fea7c066f30 0x7fea7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:05.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.831+0000 7fea8cff9700 1 -- 172.21.15.7:0/3106175839 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fea84084be0 con 0x7fea900ca540 2024-01-27T02:39:05.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.831+0000 7fea94ba8700 1 --2- 172.21.15.7:0/3106175839 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fea7c066f30 0x7fea7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:05.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.831+0000 7fea94ba8700 1 --2- 172.21.15.7:0/3106175839 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fea7c066f30 0x7fea7c0693e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fea800011e0 tx=0x7fea80009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:05.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:05.835+0000 7fea8cff9700 1 -- 172.21.15.7:0/3106175839 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea84051e10 con 0x7fea900ca540 2024-01-27T02:39:06.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:06.088+0000 7fea96e0c700 1 -- 172.21.15.7:0/3106175839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea900ce270 con 0x7fea900ca540 2024-01-27T02:39:06.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:06.088+0000 7fea8cff9700 1 -- 172.21.15.7:0/3106175839 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fea84055430 con 0x7fea900ca540 2024-01-27T02:39:06.089 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:06.089 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3997828193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3997828193},{"type":"v1","addr":"172.21.15.188:6837","nonce":3997828193}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1132338302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1132338302},{"type":"v1","addr":"172.21.15.188:6836","nonce":1132338302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/740290645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":740290645},{"type":"v1","addr":"172.21.15.7:6833","nonce":740290645}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:38:31.328277+0000","modified":"2024-01-27T02:38:34.940811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7029":{"gid":7029,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1258751919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1258751919},{"type":"v1","addr":"172.21.15.7:6835","nonce":1258751919}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-27T02:39:06.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:06.091+0000 7fea96e0c700 1 -- 172.21.15.7:0/3106175839 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fea7c066f30 msgr2=0x7fea7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:06.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:06.091+0000 7fea96e0c700 1 --2- 172.21.15.7:0/3106175839 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fea7c066f30 0x7fea7c0693e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fea800011e0 tx=0x7fea80009040 comp rx=0 tx=0).stop 2024-01-27T02:39:06.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:06.091+0000 7fea96e0c700 1 -- 172.21.15.7:0/3106175839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea900ca540 msgr2=0x7fea900b3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:06.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:06.091+0000 7fea96e0c700 1 --2- 172.21.15.7:0/3106175839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea900ca540 0x7fea900b3480 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fea8400d860 tx=0x7fea84008040 comp rx=0 tx=0).stop 2024-01-27T02:39:06.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:06.091+0000 7fea96e0c700 1 -- 172.21.15.7:0/3106175839 shutdown_connections 2024-01-27T02:39:06.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:06.091+0000 7fea96e0c700 1 --2- 172.21.15.7:0/3106175839 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fea7c066f30 0x7fea7c0693e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:06.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:06.091+0000 7fea96e0c700 1 --2- 172.21.15.7:0/3106175839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea9015a100 0x7fea9015bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:06.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:06.091+0000 7fea96e0c700 1 --2- 172.21.15.7:0/3106175839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea900c4cc0 0x7fea90153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:06.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:06.091+0000 7fea96e0c700 1 --2- 172.21.15.7:0/3106175839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea900ca540 0x7fea900b3480 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:06.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:06.091+0000 7fea96e0c700 1 -- 172.21.15.7:0/3106175839 >> 172.21.15.7:0/3106175839 conn(0x7fea90098020 msgr2=0x7fea900c9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:06.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:06.091+0000 7fea96e0c700 1 -- 172.21.15.7:0/3106175839 shutdown_connections 2024-01-27T02:39:06.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:06.091+0000 7fea96e0c700 1 -- 172.21.15.7:0/3106175839 wait complete. 2024-01-27T02:39:06.093 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 68 2024-01-27T02:39:06.105 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-01-27T02:39:06.257 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","whoami":0,"id":7029,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":34.327277029999998,"mdsmap_epoch":68,"osdmap_epoch":137,"osdmap_epoch_barrier":134,"uptime":39.735648855000001} 2024-01-27T02:39:06.258 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", "fs_name": "cephfs", "id": 7029, "mdsmap_epoch": 68, "osdmap_epoch": 137, "osdmap_epoch_barrier": 134, "rank_uptime": 34.32727703, "state": "up:active", "uptime": 39.735648855, "want_state": "up:active", "whoami": 0 } 2024-01-27T02:39:06.258 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-01-27T02:39:06.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:39:06.259 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-01-27T02:39:06.575 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records in 2024-01-27T02:39:06.576 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records out 2024-01-27T02:39:06.576 INFO:teuthology.orchestra.run.smithi007.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.296453 s, 177 MB/s 2024-01-27T02:39:06.577 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-27T02:39:06.577 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:39:06.627 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:39:06.668 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-27T02:39:06.668 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-27T02:39:06.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:39:06.668 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-27T02:39:06.748 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:39:06.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:39:06.749 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-27T02:39:06.773 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-27T02:39:06.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:39:06.773 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-01-27T02:39:06.817 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:39:06.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:39:06.817 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-27T02:39:06.874 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:39:07.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.014+0000 7f662003d700 1 Processor -- start 2024-01-27T02:39:07.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.015+0000 7f662003d700 1 -- start start 2024-01-27T02:39:07.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.015+0000 7f662003d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f66180a7fe0 0x7f661806f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:07.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.015+0000 7f662003d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f661815dfb0 0x7f661806f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:07.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.015+0000 7f662003d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661815a880 con 0x7f66180a7c10 2024-01-27T02:39:07.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.016+0000 7f662003d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66180afde0 con 0x7f66180a7fe0 2024-01-27T02:39:07.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.016+0000 7f662003d700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66180af950 con 0x7f661815dfb0 2024-01-27T02:39:07.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.016+0000 7f661d5d8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f66180a7fe0 0x7f661806f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:07.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.016+0000 7f661ddd9700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f66180a7c10 0x7f66180ae340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59830/0 (socket says 172.21.15.7:59830) 2024-01-27T02:39:07.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.016+0000 7f661ddd9700 1 -- 172.21.15.7:0/3670368921 learned_addr learned my addr 172.21.15.7:0/3670368921 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:07.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.016+0000 7f661e5da700 1 --2- 172.21.15.7:0/3670368921 >> v2:172.21.15.188:3300/0 conn(0x7f661815dfb0 0x7f661806f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:07.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.016+0000 7f661d5d8700 1 -- 172.21.15.7:0/3670368921 >> v2:172.21.15.188:3300/0 conn(0x7f661815dfb0 msgr2=0x7f661806f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:07.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.016+0000 7f661d5d8700 1 --2- 172.21.15.7:0/3670368921 >> v2:172.21.15.188:3300/0 conn(0x7f661815dfb0 0x7f661806f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.016+0000 7f661d5d8700 1 -- 172.21.15.7:0/3670368921 >> v1:172.21.15.7:6789/0 conn(0x7f66180a7c10 legacy=0x7f66180ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:07.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.016+0000 7f661d5d8700 1 -- 172.21.15.7:0/3670368921 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66180aed40 con 0x7f66180a7fe0 2024-01-27T02:39:07.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.016+0000 7f661cdd7700 1 -- 172.21.15.7:0/3670368921 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1485163326 0 0) 0x7f661815a880 con 0x7f66180a7c10 2024-01-27T02:39:07.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.016+0000 7f661e5da700 1 --2- 172.21.15.7:0/3670368921 >> v2:172.21.15.188:3300/0 conn(0x7f661815dfb0 0x7f661806f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:07.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.016+0000 7f661d5d8700 1 --2- 172.21.15.7:0/3670368921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66180a7fe0 0x7f661806f360 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f6618068a40 tx=0x7f66080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92c2bd7a136ff92 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:07.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.016+0000 7f661cdd7700 1 -- 172.21.15.7:0/3670368921 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660800b6b0 con 0x7f66180a7fe0 2024-01-27T02:39:07.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.016+0000 7f661cdd7700 1 -- 172.21.15.7:0/3670368921 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660800b810 con 0x7f66180a7fe0 2024-01-27T02:39:07.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.016+0000 7f661cdd7700 1 -- 172.21.15.7:0/3670368921 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660800bbb0 con 0x7f66180a7fe0 2024-01-27T02:39:07.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.017+0000 7f662003d700 1 -- 172.21.15.7:0/3670368921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66180a7fe0 msgr2=0x7f661806f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:07.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.017+0000 7f662003d700 1 --2- 172.21.15.7:0/3670368921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66180a7fe0 0x7f661806f360 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f6618068a40 tx=0x7f66080091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.017+0000 7f662003d700 1 -- 172.21.15.7:0/3670368921 shutdown_connections 2024-01-27T02:39:07.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.017+0000 7f662003d700 1 --2- 172.21.15.7:0/3670368921 >> v2:172.21.15.188:3300/0 conn(0x7f661815dfb0 0x7f661806f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.017+0000 7f662003d700 1 --2- 172.21.15.7:0/3670368921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66180a7fe0 0x7f661806f360 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.017+0000 7f662003d700 1 -- 172.21.15.7:0/3670368921 >> 172.21.15.7:0/3670368921 conn(0x7f66180679b0 msgr2=0x7f6618067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:07.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.017+0000 7f662003d700 1 -- 172.21.15.7:0/3670368921 shutdown_connections 2024-01-27T02:39:07.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.017+0000 7f662003d700 1 -- 172.21.15.7:0/3670368921 wait complete. 2024-01-27T02:39:07.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.019+0000 7f662003d700 1 Processor -- start 2024-01-27T02:39:07.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.019+0000 7f662003d700 1 -- start start 2024-01-27T02:39:07.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.019+0000 7f662003d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66180a7c10 0x7f6618152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:07.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.019+0000 7f662003d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66180a7fe0 0x7f66180a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:07.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.019+0000 7f662003d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f661815dfb0 0x7f66181513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:07.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.019+0000 7f662003d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66180a74d0 con 0x7f66180a7c10 2024-01-27T02:39:07.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.019+0000 7f662003d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6618152a90 con 0x7f661815dfb0 2024-01-27T02:39:07.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.019+0000 7f662003d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6618151e80 con 0x7f66180a7fe0 2024-01-27T02:39:07.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.019+0000 7f661ddd9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66180a7c10 0x7f6618152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:07.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.020+0000 7f661ddd9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66180a7c10 0x7f6618152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49788/0 (socket says 172.21.15.7:49788) 2024-01-27T02:39:07.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.020+0000 7f661ddd9700 1 -- 172.21.15.7:0/224240611 learned_addr learned my addr 172.21.15.7:0/224240611 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:07.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.020+0000 7f661e5da700 1 --2- 172.21.15.7:0/224240611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f661815dfb0 0x7f66181513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:07.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.020+0000 7f661d5d8700 1 --2- 172.21.15.7:0/224240611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66180a7fe0 0x7f66180a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:07.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.020+0000 7f661ddd9700 1 -- 172.21.15.7:0/224240611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66180a7fe0 msgr2=0x7f66180a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:07.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.020+0000 7f661ddd9700 1 --2- 172.21.15.7:0/224240611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66180a7fe0 0x7f66180a6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.020+0000 7f661ddd9700 1 -- 172.21.15.7:0/224240611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f661815dfb0 msgr2=0x7f66181513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:07.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.020+0000 7f661ddd9700 1 --2- 172.21.15.7:0/224240611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f661815dfb0 0x7f66181513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.020+0000 7f661ddd9700 1 -- 172.21.15.7:0/224240611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6608009420 con 0x7f66180a7c10 2024-01-27T02:39:07.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.020+0000 7f661ddd9700 1 --2- 172.21.15.7:0/224240611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66180a7c10 0x7f6618152f20 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f661400c7e0 tx=0x7f6614007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:07.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.020+0000 7f660dffb700 1 -- 172.21.15.7:0/224240611 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614005b30 con 0x7f66180a7c10 2024-01-27T02:39:07.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.020+0000 7f660dffb700 1 -- 172.21.15.7:0/224240611 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6614005c90 con 0x7f66180a7c10 2024-01-27T02:39:07.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.020+0000 7f662003d700 1 -- 172.21.15.7:0/224240611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6618152160 con 0x7f66180a7c10 2024-01-27T02:39:07.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.020+0000 7f660dffb700 1 -- 172.21.15.7:0/224240611 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614007300 con 0x7f66180a7c10 2024-01-27T02:39:07.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.021+0000 7f662003d700 1 -- 172.21.15.7:0/224240611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6618152720 con 0x7f66180a7c10 2024-01-27T02:39:07.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.021+0000 7f662003d700 1 -- 172.21.15.7:0/224240611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f661804c710 con 0x7f66180a7c10 2024-01-27T02:39:07.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.022+0000 7f660dffb700 1 -- 172.21.15.7:0/224240611 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f661401b3b0 con 0x7f66180a7c10 2024-01-27T02:39:07.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.022+0000 7f660dffb700 1 --2- 172.21.15.7:0/224240611 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f65fc066f30 0x7f65fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:07.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.022+0000 7f660dffb700 1 -- 172.21.15.7:0/224240611 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f6614083a90 con 0x7f66180a7c10 2024-01-27T02:39:07.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.023+0000 7f661d5d8700 1 --2- 172.21.15.7:0/224240611 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f65fc066f30 0x7f65fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:07.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.023+0000 7f661d5d8700 1 --2- 172.21.15.7:0/224240611 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f65fc066f30 0x7f65fc0693e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f66180a83d0 tx=0x7f6608017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:07.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.027+0000 7f660dffb700 1 -- 172.21.15.7:0/224240611 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6614050cc0 con 0x7f66180a7c10 2024-01-27T02:39:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.277+0000 7f662003d700 1 -- 172.21.15.7:0/224240611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66180af020 con 0x7f66180a7c10 2024-01-27T02:39:07.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.277+0000 7f660dffb700 1 -- 172.21.15.7:0/224240611 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f6614017020 con 0x7f66180a7c10 2024-01-27T02:39:07.278 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:07.278 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3997828193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3997828193},{"type":"v1","addr":"172.21.15.188:6837","nonce":3997828193}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1132338302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1132338302},{"type":"v1","addr":"172.21.15.188:6836","nonce":1132338302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/740290645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":740290645},{"type":"v1","addr":"172.21.15.7:6833","nonce":740290645}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:38:31.328277+0000","modified":"2024-01-27T02:38:34.940811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7029":{"gid":7029,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1258751919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1258751919},{"type":"v1","addr":"172.21.15.7:6835","nonce":1258751919}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-27T02:39:07.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.280+0000 7f662003d700 1 -- 172.21.15.7:0/224240611 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f65fc066f30 msgr2=0x7f65fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:07.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.280+0000 7f662003d700 1 --2- 172.21.15.7:0/224240611 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f65fc066f30 0x7f65fc0693e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f66180a83d0 tx=0x7f6608017000 comp rx=0 tx=0).stop 2024-01-27T02:39:07.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.280+0000 7f662003d700 1 -- 172.21.15.7:0/224240611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66180a7c10 msgr2=0x7f6618152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:07.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.280+0000 7f662003d700 1 --2- 172.21.15.7:0/224240611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66180a7c10 0x7f6618152f20 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f661400c7e0 tx=0x7f6614007df0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.280+0000 7f662003d700 1 -- 172.21.15.7:0/224240611 shutdown_connections 2024-01-27T02:39:07.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.280+0000 7f662003d700 1 --2- 172.21.15.7:0/224240611 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f65fc066f30 0x7f65fc0693e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.280+0000 7f662003d700 1 --2- 172.21.15.7:0/224240611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f661815dfb0 0x7f66181513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.280+0000 7f662003d700 1 --2- 172.21.15.7:0/224240611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66180a7c10 0x7f6618152f20 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.280+0000 7f662003d700 1 --2- 172.21.15.7:0/224240611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66180a7fe0 0x7f66180a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.280+0000 7f662003d700 1 -- 172.21.15.7:0/224240611 >> 172.21.15.7:0/224240611 conn(0x7f66180679b0 msgr2=0x7f6618157450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:07.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.280+0000 7f662003d700 1 -- 172.21.15.7:0/224240611 shutdown_connections 2024-01-27T02:39:07.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.280+0000 7f662003d700 1 -- 172.21.15.7:0/224240611 wait complete. 2024-01-27T02:39:07.281 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 68 2024-01-27T02:39:07.294 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:39:07.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.426+0000 7f688d2e2700 1 Processor -- start 2024-01-27T02:39:07.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.426+0000 7f688d2e2700 1 -- start start 2024-01-27T02:39:07.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.427+0000 7f688d2e2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f68880b2ff0 0x7f68880b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:07.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.427+0000 7f688d2e2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f688815e4e0 0x7f68880ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:07.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.427+0000 7f688d2e2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68880b3900 con 0x7f688815b8e0 2024-01-27T02:39:07.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.427+0000 7f688d2e2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68880ce7e0 con 0x7f68880b2ff0 2024-01-27T02:39:07.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.427+0000 7f688d2e2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688815ebe0 con 0x7f688815e4e0 2024-01-27T02:39:07.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.427+0000 7f6886ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f68880b2ff0 0x7f68880b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:07.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.427+0000 7f68867fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f688815b8e0 0x7f688815ddd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59846/0 (socket says 172.21.15.7:59846) 2024-01-27T02:39:07.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.427+0000 7f6886ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f68880b2ff0 0x7f68880b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49804/0 (socket says 172.21.15.7:49804) 2024-01-27T02:39:07.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.427+0000 7f68867fc700 1 -- 172.21.15.7:0/384656114 learned_addr learned my addr 172.21.15.7:0/384656114 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:07.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.427+0000 7f68877fe700 1 --2- 172.21.15.7:0/384656114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f688815e4e0 0x7f68880ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:07.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.428+0000 7f6886ffd700 1 -- 172.21.15.7:0/384656114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f688815e4e0 msgr2=0x7f68880ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:07.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.428+0000 7f6886ffd700 1 --2- 172.21.15.7:0/384656114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f688815e4e0 0x7f68880ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.428+0000 7f6886ffd700 1 -- 172.21.15.7:0/384656114 >> v1:172.21.15.7:6789/0 conn(0x7f688815b8e0 legacy=0x7f688815ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:07.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.428+0000 7f6886ffd700 1 -- 172.21.15.7:0/384656114 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68881587b0 con 0x7f68880b2ff0 2024-01-27T02:39:07.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.428+0000 7f6885ffb700 1 -- 172.21.15.7:0/384656114 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390674917 0 0) 0x7f68880b3900 con 0x7f688815b8e0 2024-01-27T02:39:07.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.428+0000 7f6886ffd700 1 --2- 172.21.15.7:0/384656114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68880b2ff0 0x7f68880b33c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f68780098d0 tx=0x7f6878009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05877061e8080e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:07.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.428+0000 7f6885ffb700 1 -- 172.21.15.7:0/384656114 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6878009e70 con 0x7f68880b2ff0 2024-01-27T02:39:07.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.428+0000 7f6885ffb700 1 -- 172.21.15.7:0/384656114 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6878005c10 con 0x7f68880b2ff0 2024-01-27T02:39:07.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.428+0000 7f6885ffb700 1 -- 172.21.15.7:0/384656114 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687800b650 con 0x7f68880b2ff0 2024-01-27T02:39:07.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.428+0000 7f688d2e2700 1 -- 172.21.15.7:0/384656114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68880b2ff0 msgr2=0x7f68880b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.428+0000 7f688d2e2700 1 --2- 172.21.15.7:0/384656114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68880b2ff0 0x7f68880b33c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f68780098d0 tx=0x7f6878009120 comp rx=0 tx=0).stop 2024-01-27T02:39:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.429+0000 7f688d2e2700 1 -- 172.21.15.7:0/384656114 shutdown_connections 2024-01-27T02:39:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.429+0000 7f688d2e2700 1 --2- 172.21.15.7:0/384656114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f688815e4e0 0x7f68880ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.429+0000 7f688d2e2700 1 --2- 172.21.15.7:0/384656114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68880b2ff0 0x7f68880b33c0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.429+0000 7f688d2e2700 1 -- 172.21.15.7:0/384656114 >> 172.21.15.7:0/384656114 conn(0x7f68880679b0 msgr2=0x7f6888067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.429+0000 7f688d2e2700 1 -- 172.21.15.7:0/384656114 shutdown_connections 2024-01-27T02:39:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.429+0000 7f688d2e2700 1 -- 172.21.15.7:0/384656114 wait complete. 2024-01-27T02:39:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.430+0000 7f688d2e2700 1 Processor -- start 2024-01-27T02:39:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.430+0000 7f688d2e2700 1 -- start start 2024-01-27T02:39:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.431+0000 7f688d2e2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68880b2ff0 0x7f688814f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.431+0000 7f688d2e2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f688815b8e0 0x7f6888152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.431+0000 7f688d2e2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f688815e4e0 0x7f6888153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.431+0000 7f688d2e2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6888151810 con 0x7f688815e4e0 2024-01-27T02:39:07.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.431+0000 7f688d2e2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6888151a20 con 0x7f688815b8e0 2024-01-27T02:39:07.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.431+0000 7f688d2e2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6888151d00 con 0x7f68880b2ff0 2024-01-27T02:39:07.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.431+0000 7f68877fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f688815e4e0 0x7f6888153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:07.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.431+0000 7f68877fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f688815e4e0 0x7f6888153000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49814/0 (socket says 172.21.15.7:49814) 2024-01-27T02:39:07.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.431+0000 7f68877fe700 1 -- 172.21.15.7:0/309121584 learned_addr learned my addr 172.21.15.7:0/309121584 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:07.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.431+0000 7f6886ffd700 1 --2- 172.21.15.7:0/309121584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68880b2ff0 0x7f688814f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:07.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.431+0000 7f68867fc700 1 --2- 172.21.15.7:0/309121584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f688815b8e0 0x7f6888152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:07.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.431+0000 7f68877fe700 1 -- 172.21.15.7:0/309121584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68880b2ff0 msgr2=0x7f688814f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:07.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.431+0000 7f68877fe700 1 --2- 172.21.15.7:0/309121584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68880b2ff0 0x7f688814f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.431+0000 7f68877fe700 1 -- 172.21.15.7:0/309121584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f688815b8e0 msgr2=0x7f6888152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:07.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.431+0000 7f68877fe700 1 --2- 172.21.15.7:0/309121584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f688815b8e0 0x7f6888152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.431+0000 7f68877fe700 1 -- 172.21.15.7:0/309121584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68780093c0 con 0x7f688815e4e0 2024-01-27T02:39:07.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.431+0000 7f6886ffd700 1 --2- 172.21.15.7:0/309121584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68880b2ff0 0x7f688814f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:39:07.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.431+0000 7f68867fc700 1 --2- 172.21.15.7:0/309121584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f688815b8e0 0x7f6888152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:39:07.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.431+0000 7f68877fe700 1 --2- 172.21.15.7:0/309121584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f688815e4e0 0x7f6888153000 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f6880009fa0 tx=0x7f6880009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:07.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.432+0000 7f686effd700 1 -- 172.21.15.7:0/309121584 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688000edb0 con 0x7f688815e4e0 2024-01-27T02:39:07.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.432+0000 7f686effd700 1 -- 172.21.15.7:0/309121584 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6880014420 con 0x7f688815e4e0 2024-01-27T02:39:07.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.432+0000 7f688d2e2700 1 -- 172.21.15.7:0/309121584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6888151fe0 con 0x7f688815e4e0 2024-01-27T02:39:07.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.432+0000 7f686effd700 1 -- 172.21.15.7:0/309121584 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68800147e0 con 0x7f688815e4e0 2024-01-27T02:39:07.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.432+0000 7f688d2e2700 1 -- 172.21.15.7:0/309121584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68881525a0 con 0x7f688815e4e0 2024-01-27T02:39:07.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.433+0000 7f688d2e2700 1 -- 172.21.15.7:0/309121584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68881527c0 con 0x7f688815e4e0 2024-01-27T02:39:07.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.433+0000 7f686effd700 1 -- 172.21.15.7:0/309121584 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6880014b50 con 0x7f688815e4e0 2024-01-27T02:39:07.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.434+0000 7f686effd700 1 --2- 172.21.15.7:0/309121584 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6874066f30 0x7f68740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:07.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.434+0000 7f686effd700 1 -- 172.21.15.7:0/309121584 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f6880084df0 con 0x7f688815e4e0 2024-01-27T02:39:07.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.434+0000 7f6886ffd700 1 --2- 172.21.15.7:0/309121584 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6874066f30 0x7f68740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:07.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.435+0000 7f6886ffd700 1 --2- 172.21.15.7:0/309121584 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6874066f30 0x7f68740693e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6878009160 tx=0x7f6878016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:07.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.438+0000 7f686effd700 1 -- 172.21.15.7:0/309121584 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6880052020 con 0x7f688815e4e0 2024-01-27T02:39:07.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.643+0000 7f688d2e2700 1 -- 172.21.15.7:0/309121584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f688815be30 con 0x7f688815e4e0 2024-01-27T02:39:07.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.644+0000 7f686effd700 1 -- 172.21.15.7:0/309121584 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+15421 (secure 0 0 0) 0x7f6880055640 con 0x7f688815e4e0 2024-01-27T02:39:07.644 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:07.644 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":137,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:38:59.871665+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:38:29.824892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-27T02:38:30.301072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.e","mappings":[{"from":0,"to":5}]},{"pgid":"17.24","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:39:07.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.646+0000 7f688d2e2700 1 -- 172.21.15.7:0/309121584 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6874066f30 msgr2=0x7f68740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:07.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.646+0000 7f688d2e2700 1 --2- 172.21.15.7:0/309121584 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6874066f30 0x7f68740693e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6878009160 tx=0x7f6878016000 comp rx=0 tx=0).stop 2024-01-27T02:39:07.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.646+0000 7f688d2e2700 1 -- 172.21.15.7:0/309121584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f688815e4e0 msgr2=0x7f6888153000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:07.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.646+0000 7f688d2e2700 1 --2- 172.21.15.7:0/309121584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f688815e4e0 0x7f6888153000 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f6880009fa0 tx=0x7f6880009900 comp rx=0 tx=0).stop 2024-01-27T02:39:07.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.646+0000 7f688d2e2700 1 -- 172.21.15.7:0/309121584 shutdown_connections 2024-01-27T02:39:07.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.646+0000 7f688d2e2700 1 --2- 172.21.15.7:0/309121584 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6874066f30 0x7f68740693e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.646+0000 7f688d2e2700 1 --2- 172.21.15.7:0/309121584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f688815e4e0 0x7f6888153000 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.646+0000 7f688d2e2700 1 --2- 172.21.15.7:0/309121584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68880b2ff0 0x7f688814f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.646+0000 7f688d2e2700 1 --2- 172.21.15.7:0/309121584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f688815b8e0 0x7f6888152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.646+0000 7f688d2e2700 1 -- 172.21.15.7:0/309121584 >> 172.21.15.7:0/309121584 conn(0x7f68880679b0 msgr2=0x7f6888150980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:07.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.646+0000 7f688d2e2700 1 -- 172.21.15.7:0/309121584 shutdown_connections 2024-01-27T02:39:07.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.646+0000 7f688d2e2700 1 -- 172.21.15.7:0/309121584 wait complete. 2024-01-27T02:39:07.660 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-27T02:38:29.824892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-01-27T02:38:30.301072+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-27T02:39:07.661 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:39:07.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.794+0000 7fa0cb115700 1 Processor -- start 2024-01-27T02:39:07.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.794+0000 7fa0cb115700 1 -- start start 2024-01-27T02:39:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.795+0000 7fa0cb115700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa0c41416c0 0x7fa0c40cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.795+0000 7fa0cb115700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c4141a90 0x7fa0c40cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.795+0000 7fa0cb115700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa0c40cab00 0x7fa0c40caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.795+0000 7fa0cb115700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c415c7d0 con 0x7fa0c41416c0 2024-01-27T02:39:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.795+0000 7fa0cb115700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c40cb480 con 0x7fa0c40cab00 2024-01-27T02:39:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.795+0000 7fa0cb115700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40c9640 con 0x7fa0c4141a90 2024-01-27T02:39:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.796+0000 7fa0c8eb1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa0c41416c0 0x7fa0c40cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.796+0000 7fa0c8eb1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa0c41416c0 0x7fa0c40cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49818/0 (socket says 172.21.15.7:49818) 2024-01-27T02:39:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.796+0000 7fa0c8eb1700 1 -- 172.21.15.7:0/3193169917 learned_addr learned my addr 172.21.15.7:0/3193169917 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.796+0000 7fa0c3fff700 1 --2- 172.21.15.7:0/3193169917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c4141a90 0x7fa0c40cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.796+0000 7fa0c96b2700 1 --2- 172.21.15.7:0/3193169917 >> v2:172.21.15.188:3300/0 conn(0x7fa0c40cab00 0x7fa0c40caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.796+0000 7fa0c8eb1700 1 -- 172.21.15.7:0/3193169917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c4141a90 msgr2=0x7fa0c40cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.796+0000 7fa0c8eb1700 1 --2- 172.21.15.7:0/3193169917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c4141a90 0x7fa0c40cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.796+0000 7fa0c8eb1700 1 -- 172.21.15.7:0/3193169917 >> v2:172.21.15.188:3300/0 conn(0x7fa0c40cab00 msgr2=0x7fa0c40caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.796+0000 7fa0c8eb1700 1 --2- 172.21.15.7:0/3193169917 >> v2:172.21.15.188:3300/0 conn(0x7fa0c40cab00 0x7fa0c40caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.796+0000 7fa0c8eb1700 1 -- 172.21.15.7:0/3193169917 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c415bbc0 con 0x7fa0c41416c0 2024-01-27T02:39:07.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.796+0000 7fa0c96b2700 1 --2- 172.21.15.7:0/3193169917 >> v2:172.21.15.188:3300/0 conn(0x7fa0c40cab00 0x7fa0c40caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:39:07.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.796+0000 7fa0c8eb1700 1 --2- 172.21.15.7:0/3193169917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c41416c0 0x7fa0c40cb6e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40c92b0 tx=0x7fa0b8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a3db9998bdb3db server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:07.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.796+0000 7fa0c37fe700 1 -- 172.21.15.7:0/3193169917 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8005c10 con 0x7fa0c41416c0 2024-01-27T02:39:07.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.796+0000 7fa0c37fe700 1 -- 172.21.15.7:0/3193169917 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b8005d70 con 0x7fa0c41416c0 2024-01-27T02:39:07.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.796+0000 7fa0c37fe700 1 -- 172.21.15.7:0/3193169917 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b800b880 con 0x7fa0c41416c0 2024-01-27T02:39:07.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.797+0000 7fa0cb115700 1 -- 172.21.15.7:0/3193169917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c41416c0 msgr2=0x7fa0c40cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:07.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.797+0000 7fa0cb115700 1 --2- 172.21.15.7:0/3193169917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c41416c0 0x7fa0c40cb6e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40c92b0 tx=0x7fa0b8009220 comp rx=0 tx=0).stop 2024-01-27T02:39:07.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.797+0000 7fa0cb115700 1 -- 172.21.15.7:0/3193169917 shutdown_connections 2024-01-27T02:39:07.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.797+0000 7fa0cb115700 1 --2- 172.21.15.7:0/3193169917 >> v2:172.21.15.188:3300/0 conn(0x7fa0c40cab00 0x7fa0c40caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.797+0000 7fa0cb115700 1 --2- 172.21.15.7:0/3193169917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c41416c0 0x7fa0c40cb6e0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.797+0000 7fa0cb115700 1 --2- 172.21.15.7:0/3193169917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c4141a90 0x7fa0c40cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.797+0000 7fa0cb115700 1 -- 172.21.15.7:0/3193169917 >> 172.21.15.7:0/3193169917 conn(0x7fa0c4063580 msgr2=0x7fa0c4063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:07.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.797+0000 7fa0cb115700 1 -- 172.21.15.7:0/3193169917 shutdown_connections 2024-01-27T02:39:07.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.797+0000 7fa0cb115700 1 -- 172.21.15.7:0/3193169917 wait complete. 2024-01-27T02:39:07.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.798+0000 7fa0cb115700 1 Processor -- start 2024-01-27T02:39:07.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.798+0000 7fa0cb115700 1 -- start start 2024-01-27T02:39:07.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.799+0000 7fa0cb115700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c40cab00 0x7fa0c40c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:07.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.799+0000 7fa0cb115700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c41416c0 0x7fa0c41466c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:07.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.799+0000 7fa0cb115700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c4141a90 0x7fa0c4157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:07.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.799+0000 7fa0cb115700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c4147b00 con 0x7fa0c41416c0 2024-01-27T02:39:07.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.799+0000 7fa0cb115700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c4146ef0 con 0x7fa0c4141a90 2024-01-27T02:39:07.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.799+0000 7fa0cb115700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c41471d0 con 0x7fa0c40cab00 2024-01-27T02:39:07.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.799+0000 7fa0c3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c41416c0 0x7fa0c41466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:07.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.799+0000 7fa0c3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c41416c0 0x7fa0c41466c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49830/0 (socket says 172.21.15.7:49830) 2024-01-27T02:39:07.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.799+0000 7fa0c3fff700 1 -- 172.21.15.7:0/3059392348 learned_addr learned my addr 172.21.15.7:0/3059392348 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:07.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.799+0000 7fa0c96b2700 1 --2- 172.21.15.7:0/3059392348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c4141a90 0x7fa0c4157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:07.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.799+0000 7fa0c8eb1700 1 --2- 172.21.15.7:0/3059392348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c40cab00 0x7fa0c40c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:07.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.800+0000 7fa0c3fff700 1 -- 172.21.15.7:0/3059392348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c40cab00 msgr2=0x7fa0c40c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:07.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.800+0000 7fa0c3fff700 1 --2- 172.21.15.7:0/3059392348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c40cab00 0x7fa0c40c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.800+0000 7fa0c3fff700 1 -- 172.21.15.7:0/3059392348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c4141a90 msgr2=0x7fa0c4157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:07.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.800+0000 7fa0c3fff700 1 --2- 172.21.15.7:0/3059392348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c4141a90 0x7fa0c4157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:07.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.800+0000 7fa0c3fff700 1 -- 172.21.15.7:0/3059392348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b8009480 con 0x7fa0c41416c0 2024-01-27T02:39:07.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.800+0000 7fa0c8eb1700 1 --2- 172.21.15.7:0/3059392348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c40cab00 0x7fa0c40c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:39:07.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.800+0000 7fa0c3fff700 1 --2- 172.21.15.7:0/3059392348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c41416c0 0x7fa0c41466c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000c880 tx=0x7fa0b0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:07.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.800+0000 7fa0c0ff9700 1 -- 172.21.15.7:0/3059392348 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0011600 con 0x7fa0c41416c0 2024-01-27T02:39:07.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.800+0000 7fa0c0ff9700 1 -- 172.21.15.7:0/3059392348 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b0011780 con 0x7fa0c41416c0 2024-01-27T02:39:07.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.800+0000 7fa0cb115700 1 -- 172.21.15.7:0/3059392348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c41474b0 con 0x7fa0c41416c0 2024-01-27T02:39:07.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.800+0000 7fa0c0ff9700 1 -- 172.21.15.7:0/3059392348 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0011b40 con 0x7fa0c41416c0 2024-01-27T02:39:07.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.800+0000 7fa0cb115700 1 -- 172.21.15.7:0/3059392348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c4146c00 con 0x7fa0c41416c0 2024-01-27T02:39:07.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.801+0000 7fa0cb115700 1 -- 172.21.15.7:0/3059392348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c4147cb0 con 0x7fa0c41416c0 2024-01-27T02:39:07.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.802+0000 7fa0c0ff9700 1 -- 172.21.15.7:0/3059392348 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa0b0018360 con 0x7fa0c41416c0 2024-01-27T02:39:07.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.802+0000 7fa0c0ff9700 1 --2- 172.21.15.7:0/3059392348 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa0a8066f30 0x7fa0a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:07.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.802+0000 7fa0c0ff9700 1 -- 172.21.15.7:0/3059392348 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fa0b0084280 con 0x7fa0c41416c0 2024-01-27T02:39:07.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.802+0000 7fa0c8eb1700 1 --2- 172.21.15.7:0/3059392348 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa0a8066f30 0x7fa0a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:07.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.803+0000 7fa0c8eb1700 1 --2- 172.21.15.7:0/3059392348 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa0a8066f30 0x7fa0a80693e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009260 tx=0x7fa0b8016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:07.806+0000 7fa0c0ff9700 1 -- 172.21.15.7:0/3059392348 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0c4147cb0 con 0x7fa0c41416c0 2024-01-27T02:39:08.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.010+0000 7fa0cb115700 1 -- 172.21.15.7:0/3059392348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa0c4147cb0 con 0x7fa0c41416c0 2024-01-27T02:39:08.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.010+0000 7fa0c0ff9700 1 -- 172.21.15.7:0/3059392348 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa0c4147cb0 con 0x7fa0c41416c0 2024-01-27T02:39:08.011 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:39:08.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.012+0000 7fa0cb115700 1 -- 172.21.15.7:0/3059392348 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa0a8066f30 msgr2=0x7fa0a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:08.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.012+0000 7fa0cb115700 1 --2- 172.21.15.7:0/3059392348 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa0a8066f30 0x7fa0a80693e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009260 tx=0x7fa0b8016000 comp rx=0 tx=0).stop 2024-01-27T02:39:08.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.012+0000 7fa0cb115700 1 -- 172.21.15.7:0/3059392348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c41416c0 msgr2=0x7fa0c41466c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:08.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.012+0000 7fa0cb115700 1 --2- 172.21.15.7:0/3059392348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c41416c0 0x7fa0c41466c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000c880 tx=0x7fa0b0008040 comp rx=0 tx=0).stop 2024-01-27T02:39:08.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.012+0000 7fa0cb115700 1 -- 172.21.15.7:0/3059392348 shutdown_connections 2024-01-27T02:39:08.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.012+0000 7fa0cb115700 1 --2- 172.21.15.7:0/3059392348 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa0a8066f30 0x7fa0a80693e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.012+0000 7fa0cb115700 1 --2- 172.21.15.7:0/3059392348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c4141a90 0x7fa0c4157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.012+0000 7fa0cb115700 1 --2- 172.21.15.7:0/3059392348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c40cab00 0x7fa0c40c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.012+0000 7fa0cb115700 1 --2- 172.21.15.7:0/3059392348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c41416c0 0x7fa0c41466c0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.012+0000 7fa0cb115700 1 -- 172.21.15.7:0/3059392348 >> 172.21.15.7:0/3059392348 conn(0x7fa0c4063580 msgr2=0x7fa0c40cc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:08.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.012+0000 7fa0cb115700 1 -- 172.21.15.7:0/3059392348 shutdown_connections 2024-01-27T02:39:08.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.013+0000 7fa0cb115700 1 -- 172.21.15.7:0/3059392348 wait complete. 2024-01-27T02:39:08.026 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-27T02:39:08.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.161+0000 7faffee1c700 1 Processor -- start 2024-01-27T02:39:08.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.161+0000 7faffee1c700 1 -- start start 2024-01-27T02:39:08.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.162+0000 7faffee1c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faff80cba60 0x7faff80ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:08.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.162+0000 7faffee1c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faff806b4e0 0x7faff806b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:08.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.162+0000 7faffee1c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faff80679b0 con 0x7faff80cbe30 2024-01-27T02:39:08.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.162+0000 7faffee1c700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff806bea0 con 0x7faff806b4e0 2024-01-27T02:39:08.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.162+0000 7faffee1c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff8067c90 con 0x7faff80cba60 2024-01-27T02:39:08.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.163+0000 7faff7fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7faff80cbe30 0x7faff815b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59862/0 (socket says 172.21.15.7:59862) 2024-01-27T02:39:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.163+0000 7faff7fff700 1 -- 172.21.15.7:0/876407796 learned_addr learned my addr 172.21.15.7:0/876407796 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.163+0000 7faffd3b9700 1 --2- 172.21.15.7:0/876407796 >> v2:172.21.15.188:3300/0 conn(0x7faff806b4e0 0x7faff806b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.163+0000 7faffcbb8700 1 --2- 172.21.15.7:0/876407796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faff80cba60 0x7faff80ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.163+0000 7faff77fe700 1 -- 172.21.15.7:0/876407796 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771371698 0 0) 0x7faff80679b0 con 0x7faff80cbe30 2024-01-27T02:39:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.163+0000 7faff77fe700 1 -- 172.21.15.7:0/876407796 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafe8003570 con 0x7faff80cbe30 2024-01-27T02:39:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.163+0000 7faff77fe700 1 -- 172.21.15.7:0/876407796 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3875648115 0 0) 0x7fafe8003570 con 0x7faff80cbe30 2024-01-27T02:39:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.163+0000 7faff77fe700 1 -- 172.21.15.7:0/876407796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faff80cba60 msgr2=0x7faff80ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.163+0000 7faff77fe700 1 --2- 172.21.15.7:0/876407796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faff80cba60 0x7faff80ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.163+0000 7faff77fe700 1 -- 172.21.15.7:0/876407796 >> v2:172.21.15.188:3300/0 conn(0x7faff806b4e0 msgr2=0x7faff806b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:08.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.163+0000 7faff77fe700 1 --2- 172.21.15.7:0/876407796 >> v2:172.21.15.188:3300/0 conn(0x7faff806b4e0 0x7faff806b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.163+0000 7faff77fe700 1 -- 172.21.15.7:0/876407796 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff80cca50 con 0x7faff80cbe30 2024-01-27T02:39:08.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.163+0000 7faff77fe700 1 -- 172.21.15.7:0/876407796 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7fafe4001470 con 0x7faff80cbe30 2024-01-27T02:39:08.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.163+0000 7faff77fe700 1 -- 172.21.15.7:0/876407796 >> v1:172.21.15.7:6789/0 conn(0x7faff80cbe30 legacy=0x7faff815b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:08.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.163+0000 7faff77fe700 1 --2- 172.21.15.7:0/876407796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafe8003a60 0x7fafe8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.163+0000 7faff77fe700 1 --2- 172.21.15.7:0/876407796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafe8004810 0x7fafe8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.163+0000 7faff77fe700 1 --2- 172.21.15.7:0/876407796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faff80cba60 0x7fafe80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.163+0000 7faff77fe700 1 -- 172.21.15.7:0/876407796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff806bea0 con 0x7fafe8004810 2024-01-27T02:39:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.163+0000 7faff77fe700 1 -- 172.21.15.7:0/876407796 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff8067c90 con 0x7fafe8003a60 2024-01-27T02:39:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.163+0000 7faff77fe700 1 -- 172.21.15.7:0/876407796 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe800ba60 con 0x7faff80cba60 2024-01-27T02:39:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.163+0000 7faff7fff700 1 --2- 172.21.15.7:0/876407796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafe8004810 0x7fafe8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.163+0000 7faffcbb8700 1 --2- 172.21.15.7:0/876407796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafe8003a60 0x7fafe8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.164+0000 7faffd3b9700 1 --2- 172.21.15.7:0/876407796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faff80cba60 0x7fafe80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.164+0000 7faff7fff700 1 -- 172.21.15.7:0/876407796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faff80cba60 msgr2=0x7fafe80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.164+0000 7faff7fff700 1 --2- 172.21.15.7:0/876407796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faff80cba60 0x7fafe80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.164+0000 7faff7fff700 1 -- 172.21.15.7:0/876407796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafe8003a60 msgr2=0x7fafe8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.164+0000 7faff7fff700 1 --2- 172.21.15.7:0/876407796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafe8003a60 0x7fafe8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.164+0000 7faff7fff700 1 -- 172.21.15.7:0/876407796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe8003570 con 0x7fafe8004810 2024-01-27T02:39:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.164+0000 7faff7fff700 1 --2- 172.21.15.7:0/876407796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafe8004810 0x7fafe8004c80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fafe400c3c0 tx=0x7fafe4001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffbd2489b668690 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.164+0000 7faff77fe700 1 -- 172.21.15.7:0/876407796 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe400cc50 con 0x7fafe8004810 2024-01-27T02:39:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.164+0000 7faff77fe700 1 -- 172.21.15.7:0/876407796 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe400cdf0 con 0x7fafe8004810 2024-01-27T02:39:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.164+0000 7faff77fe700 1 -- 172.21.15.7:0/876407796 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe4005b70 con 0x7fafe8004810 2024-01-27T02:39:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.165+0000 7faffee1c700 1 -- 172.21.15.7:0/876407796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafe8004810 msgr2=0x7fafe8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.165+0000 7faffee1c700 1 --2- 172.21.15.7:0/876407796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafe8004810 0x7fafe8004c80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fafe400c3c0 tx=0x7fafe4001e90 comp rx=0 tx=0).stop 2024-01-27T02:39:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.165+0000 7faffd3b9700 1 -- 172.21.15.7:0/876407796 reap_dead start 2024-01-27T02:39:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.165+0000 7faffee1c700 1 -- 172.21.15.7:0/876407796 shutdown_connections 2024-01-27T02:39:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.165+0000 7faffee1c700 1 -- 172.21.15.7:0/876407796 >> 172.21.15.7:0/876407796 conn(0x7faff8063580 msgr2=0x7faff8063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.165+0000 7faffee1c700 1 -- 172.21.15.7:0/876407796 shutdown_connections 2024-01-27T02:39:08.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.165+0000 7faffee1c700 1 -- 172.21.15.7:0/876407796 wait complete. 2024-01-27T02:39:08.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.166+0000 7faffee1c700 1 Processor -- start 2024-01-27T02:39:08.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.166+0000 7faffee1c700 1 -- start start 2024-01-27T02:39:08.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.167+0000 7faffee1c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafe8004810 0x7faff816dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:08.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.167+0000 7faffee1c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faff816b040 0x7faff815cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:08.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.167+0000 7faffee1c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faff8145360 0x7faff8145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:08.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.167+0000 7faffee1c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff8145d40 con 0x7fafe8004810 2024-01-27T02:39:08.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.167+0000 7faffee1c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff8156400 con 0x7faff8145360 2024-01-27T02:39:08.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.167+0000 7faffee1c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff80b2f40 con 0x7faff816b040 2024-01-27T02:39:08.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.167+0000 7faffcbb8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafe8004810 0x7faff816dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:08.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.167+0000 7faffcbb8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafe8004810 0x7faff816dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49842/0 (socket says 172.21.15.7:49842) 2024-01-27T02:39:08.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.167+0000 7faffcbb8700 1 -- 172.21.15.7:0/1678433924 learned_addr learned my addr 172.21.15.7:0/1678433924 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:08.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.167+0000 7faff7fff700 1 --2- 172.21.15.7:0/1678433924 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faff816b040 0x7faff815cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:08.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.167+0000 7faffd3b9700 1 --2- 172.21.15.7:0/1678433924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faff8145360 0x7faff8145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:08.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.167+0000 7faffcbb8700 1 -- 172.21.15.7:0/1678433924 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faff816b040 msgr2=0x7faff815cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:08.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.167+0000 7faffcbb8700 1 --2- 172.21.15.7:0/1678433924 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faff816b040 0x7faff815cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.167+0000 7faffcbb8700 1 -- 172.21.15.7:0/1678433924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faff8145360 msgr2=0x7faff8145800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:08.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.167+0000 7faffcbb8700 1 --2- 172.21.15.7:0/1678433924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faff8145360 0x7faff8145800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.167+0000 7faffcbb8700 1 -- 172.21.15.7:0/1678433924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff816b4b0 con 0x7fafe8004810 2024-01-27T02:39:08.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.167+0000 7faffd3b9700 1 --2- 172.21.15.7:0/1678433924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faff8145360 0x7faff8145800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:08.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.168+0000 7faffcbb8700 1 --2- 172.21.15.7:0/1678433924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafe8004810 0x7faff816dbf0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fafec011b60 tx=0x7fafec009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:08.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.168+0000 7faff4ff9700 1 -- 172.21.15.7:0/1678433924 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec005c10 con 0x7fafe8004810 2024-01-27T02:39:08.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.168+0000 7faff4ff9700 1 -- 172.21.15.7:0/1678433924 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafec005d70 con 0x7fafe8004810 2024-01-27T02:39:08.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.168+0000 7faffee1c700 1 -- 172.21.15.7:0/1678433924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff816e1d0 con 0x7fafe8004810 2024-01-27T02:39:08.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.168+0000 7faff4ff9700 1 -- 172.21.15.7:0/1678433924 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec00f9f0 con 0x7fafe8004810 2024-01-27T02:39:08.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.168+0000 7faffee1c700 1 -- 172.21.15.7:0/1678433924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff816e790 con 0x7fafe8004810 2024-01-27T02:39:08.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.169+0000 7faffee1c700 1 -- 172.21.15.7:0/1678433924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff8072060 con 0x7fafe8004810 2024-01-27T02:39:08.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.170+0000 7faff4ff9700 1 -- 172.21.15.7:0/1678433924 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fafec006050 con 0x7fafe8004810 2024-01-27T02:39:08.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.170+0000 7faff4ff9700 1 --2- 172.21.15.7:0/1678433924 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fafdc066f30 0x7fafdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:08.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.170+0000 7faff4ff9700 1 -- 172.21.15.7:0/1678433924 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fafec084c60 con 0x7fafe8004810 2024-01-27T02:39:08.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.170+0000 7faff7fff700 1 --2- 172.21.15.7:0/1678433924 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fafdc066f30 0x7fafdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:08.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.171+0000 7faff7fff700 1 --2- 172.21.15.7:0/1678433924 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fafdc066f30 0x7fafdc0693e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fafe400c3c0 tx=0x7fafe4015000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:08.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.174+0000 7faff4ff9700 1 -- 172.21.15.7:0/1678433924 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fafec051e90 con 0x7fafe8004810 2024-01-27T02:39:08.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.380+0000 7faffee1c700 1 -- 172.21.15.7:0/1678433924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faff806d5c0 con 0x7fafe8004810 2024-01-27T02:39:08.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.380+0000 7faff4ff9700 1 -- 172.21.15.7:0/1678433924 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v137) v1 ==== 104+0+11 (secure 0 0 0) 0x7fafec0210a0 con 0x7fafe8004810 2024-01-27T02:39:08.381 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-27T02:39:08.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.382+0000 7faffee1c700 1 -- 172.21.15.7:0/1678433924 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fafdc066f30 msgr2=0x7fafdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:08.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.382+0000 7faffee1c700 1 --2- 172.21.15.7:0/1678433924 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fafdc066f30 0x7fafdc0693e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fafe400c3c0 tx=0x7fafe4015000 comp rx=0 tx=0).stop 2024-01-27T02:39:08.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.382+0000 7faffee1c700 1 -- 172.21.15.7:0/1678433924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafe8004810 msgr2=0x7faff816dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:08.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.382+0000 7faffee1c700 1 --2- 172.21.15.7:0/1678433924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafe8004810 0x7faff816dbf0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fafec011b60 tx=0x7fafec009120 comp rx=0 tx=0).stop 2024-01-27T02:39:08.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.382+0000 7faffee1c700 1 -- 172.21.15.7:0/1678433924 shutdown_connections 2024-01-27T02:39:08.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.382+0000 7faffee1c700 1 --2- 172.21.15.7:0/1678433924 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fafdc066f30 0x7fafdc0693e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.382+0000 7faffee1c700 1 --2- 172.21.15.7:0/1678433924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faff8145360 0x7faff8145800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.382+0000 7faffee1c700 1 --2- 172.21.15.7:0/1678433924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafe8004810 0x7faff816dbf0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.382+0000 7faffee1c700 1 --2- 172.21.15.7:0/1678433924 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faff816b040 0x7faff815cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.382+0000 7faffee1c700 1 -- 172.21.15.7:0/1678433924 >> 172.21.15.7:0/1678433924 conn(0x7faff8063580 msgr2=0x7faff8068020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:08.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.383+0000 7faffee1c700 1 -- 172.21.15.7:0/1678433924 shutdown_connections 2024-01-27T02:39:08.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.383+0000 7faffee1c700 1 -- 172.21.15.7:0/1678433924 wait complete. 2024-01-27T02:39:08.396 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-27T02:39:08.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.532+0000 7f130aa18700 1 Processor -- start 2024-01-27T02:39:08.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.532+0000 7f130aa18700 1 -- start start 2024-01-27T02:39:08.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.533+0000 7f130aa18700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f13040cb950 0x7f13040b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:08.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.533+0000 7f130aa18700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f13041412d0 0x7f1304141730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:08.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.533+0000 7f130aa18700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13040c9660 con 0x7f13040cbd20 2024-01-27T02:39:08.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.533+0000 7f130aa18700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f130415c980 con 0x7f13040cb950 2024-01-27T02:39:08.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.533+0000 7f130aa18700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1304141c70 con 0x7f13041412d0 2024-01-27T02:39:08.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.534+0000 7f1303fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f13040cb950 0x7f13040b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:08.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.534+0000 7f13037fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f13040cbd20 0x7f13040ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59874/0 (socket says 172.21.15.7:59874) 2024-01-27T02:39:08.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.534+0000 7f13037fe700 1 -- 172.21.15.7:0/1646631098 learned_addr learned my addr 172.21.15.7:0/1646631098 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:08.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.534+0000 7f1308fb5700 1 --2- 172.21.15.7:0/1646631098 >> v2:172.21.15.188:3300/0 conn(0x7f13041412d0 0x7f1304141730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:08.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.534+0000 7f1303fff700 1 -- 172.21.15.7:0/1646631098 >> v2:172.21.15.188:3300/0 conn(0x7f13041412d0 msgr2=0x7f1304141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:08.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.534+0000 7f1303fff700 1 --2- 172.21.15.7:0/1646631098 >> v2:172.21.15.188:3300/0 conn(0x7f13041412d0 0x7f1304141730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.534+0000 7f1303fff700 1 -- 172.21.15.7:0/1646631098 >> v1:172.21.15.7:6789/0 conn(0x7f13040cbd20 legacy=0x7f13040ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:08.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.534+0000 7f1303fff700 1 -- 172.21.15.7:0/1646631098 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130415be00 con 0x7f13040cb950 2024-01-27T02:39:08.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.534+0000 7f1303fff700 1 --2- 172.21.15.7:0/1646631098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13040cb950 0x7f13040b1ac0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f13041584a0 tx=0x7f12f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129e243d4e424707 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:08.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.534+0000 7f1302ffd700 1 -- 172.21.15.7:0/1646631098 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f400b6b0 con 0x7f13040cb950 2024-01-27T02:39:08.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.534+0000 7f1302ffd700 1 -- 172.21.15.7:0/1646631098 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f400b810 con 0x7f13040cb950 2024-01-27T02:39:08.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.534+0000 7f1302ffd700 1 -- 172.21.15.7:0/1646631098 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f400bbb0 con 0x7f13040cb950 2024-01-27T02:39:08.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.535+0000 7f130aa18700 1 -- 172.21.15.7:0/1646631098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13040cb950 msgr2=0x7f13040b1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:08.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.535+0000 7f130aa18700 1 --2- 172.21.15.7:0/1646631098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13040cb950 0x7f13040b1ac0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f13041584a0 tx=0x7f12f40091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.535+0000 7f130aa18700 1 -- 172.21.15.7:0/1646631098 shutdown_connections 2024-01-27T02:39:08.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.535+0000 7f130aa18700 1 --2- 172.21.15.7:0/1646631098 >> v2:172.21.15.188:3300/0 conn(0x7f13041412d0 0x7f1304141730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.535+0000 7f130aa18700 1 --2- 172.21.15.7:0/1646631098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13040cb950 0x7f13040b1ac0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.535+0000 7f130aa18700 1 -- 172.21.15.7:0/1646631098 >> 172.21.15.7:0/1646631098 conn(0x7f1304063580 msgr2=0x7f1304063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:08.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.535+0000 7f130aa18700 1 -- 172.21.15.7:0/1646631098 shutdown_connections 2024-01-27T02:39:08.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.535+0000 7f130aa18700 1 -- 172.21.15.7:0/1646631098 wait complete. 2024-01-27T02:39:08.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.537+0000 7f130aa18700 1 Processor -- start 2024-01-27T02:39:08.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.537+0000 7f130aa18700 1 -- start start 2024-01-27T02:39:08.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.538+0000 7f130aa18700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13040cb950 0x7f1304168a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:08.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.538+0000 7f130aa18700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13040cbd20 0x7f130413e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:08.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.538+0000 7f130aa18700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13041412d0 0x7f13040c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:08.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.538+0000 7f130aa18700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130413ed60 con 0x7f13040cbd20 2024-01-27T02:39:08.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.538+0000 7f130aa18700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1304168590 con 0x7f13040cb950 2024-01-27T02:39:08.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.538+0000 7f130aa18700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13040c9040 con 0x7f13041412d0 2024-01-27T02:39:08.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.538+0000 7f13037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13040cbd20 0x7f130413e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:08.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.538+0000 7f13037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13040cbd20 0x7f130413e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49868/0 (socket says 172.21.15.7:49868) 2024-01-27T02:39:08.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.538+0000 7f13037fe700 1 -- 172.21.15.7:0/1781210320 learned_addr learned my addr 172.21.15.7:0/1781210320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:08.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.538+0000 7f1303fff700 1 --2- 172.21.15.7:0/1781210320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13040cb950 0x7f1304168a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:08.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.538+0000 7f1308fb5700 1 --2- 172.21.15.7:0/1781210320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13041412d0 0x7f13040c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:08.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.538+0000 7f13037fe700 1 -- 172.21.15.7:0/1781210320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13041412d0 msgr2=0x7f13040c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:08.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.538+0000 7f13037fe700 1 --2- 172.21.15.7:0/1781210320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13041412d0 0x7f13040c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.538+0000 7f13037fe700 1 -- 172.21.15.7:0/1781210320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13040cb950 msgr2=0x7f1304168a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:08.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.538+0000 7f13037fe700 1 --2- 172.21.15.7:0/1781210320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13040cb950 0x7f1304168a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.538+0000 7f13037fe700 1 -- 172.21.15.7:0/1781210320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f4009420 con 0x7f13040cbd20 2024-01-27T02:39:08.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.539+0000 7f13037fe700 1 --2- 172.21.15.7:0/1781210320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13040cbd20 0x7f130413e820 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f12f800c7c0 tx=0x7f12f8007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:08.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.539+0000 7f12ebfff700 1 -- 172.21.15.7:0/1781210320 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8005b30 con 0x7f13040cbd20 2024-01-27T02:39:08.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.539+0000 7f12ebfff700 1 -- 172.21.15.7:0/1781210320 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f8005cb0 con 0x7f13040cbd20 2024-01-27T02:39:08.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.539+0000 7f130aa18700 1 -- 172.21.15.7:0/1781210320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13040c9320 con 0x7f13040cbd20 2024-01-27T02:39:08.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.539+0000 7f12ebfff700 1 -- 172.21.15.7:0/1781210320 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8004a80 con 0x7f13040cbd20 2024-01-27T02:39:08.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.539+0000 7f130aa18700 1 -- 172.21.15.7:0/1781210320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1304167e70 con 0x7f13040cbd20 2024-01-27T02:39:08.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.540+0000 7f130aa18700 1 -- 172.21.15.7:0/1781210320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1304072060 con 0x7f13040cbd20 2024-01-27T02:39:08.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.540+0000 7f12ebfff700 1 -- 172.21.15.7:0/1781210320 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f12f80070f0 con 0x7f13040cbd20 2024-01-27T02:39:08.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.541+0000 7f12ebfff700 1 --2- 172.21.15.7:0/1781210320 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f12f0066f30 0x7f12f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:08.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.541+0000 7f12ebfff700 1 -- 172.21.15.7:0/1781210320 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f12f80839f0 con 0x7f13040cbd20 2024-01-27T02:39:08.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.541+0000 7f1303fff700 1 --2- 172.21.15.7:0/1781210320 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f12f0066f30 0x7f12f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:08.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.542+0000 7f1303fff700 1 --2- 172.21.15.7:0/1781210320 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f12f0066f30 0x7f12f00693e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f12f40093f0 tx=0x7f12f4017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:08.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.545+0000 7f12ebfff700 1 -- 172.21.15.7:0/1781210320 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12f8050ca0 con 0x7f13040cbd20 2024-01-27T02:39:08.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.748+0000 7f130aa18700 1 -- 172.21.15.7:0/1781210320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f13041686d0 con 0x7f13040cbd20 2024-01-27T02:39:08.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.749+0000 7f12ebfff700 1 -- 172.21.15.7:0/1781210320 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v137) v1 ==== 100+0+11 (secure 0 0 0) 0x7f12f8018030 con 0x7f13040cbd20 2024-01-27T02:39:08.749 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-27T02:39:08.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.751+0000 7f130aa18700 1 -- 172.21.15.7:0/1781210320 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f12f0066f30 msgr2=0x7f12f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.751+0000 7f130aa18700 1 --2- 172.21.15.7:0/1781210320 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f12f0066f30 0x7f12f00693e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f12f40093f0 tx=0x7f12f4017000 comp rx=0 tx=0).stop 2024-01-27T02:39:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.751+0000 7f130aa18700 1 -- 172.21.15.7:0/1781210320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13040cbd20 msgr2=0x7f130413e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.751+0000 7f130aa18700 1 --2- 172.21.15.7:0/1781210320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13040cbd20 0x7f130413e820 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f12f800c7c0 tx=0x7f12f8007f70 comp rx=0 tx=0).stop 2024-01-27T02:39:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.751+0000 7f130aa18700 1 -- 172.21.15.7:0/1781210320 shutdown_connections 2024-01-27T02:39:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.751+0000 7f130aa18700 1 --2- 172.21.15.7:0/1781210320 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f12f0066f30 0x7f12f00693e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.751+0000 7f130aa18700 1 --2- 172.21.15.7:0/1781210320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13041412d0 0x7f13040c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.751+0000 7f130aa18700 1 --2- 172.21.15.7:0/1781210320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13040cb950 0x7f1304168a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.751+0000 7f130aa18700 1 --2- 172.21.15.7:0/1781210320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13040cbd20 0x7f130413e820 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.751+0000 7f130aa18700 1 -- 172.21.15.7:0/1781210320 >> 172.21.15.7:0/1781210320 conn(0x7f1304063580 msgr2=0x7f1304063950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.751+0000 7f130aa18700 1 -- 172.21.15.7:0/1781210320 shutdown_connections 2024-01-27T02:39:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.751+0000 7f130aa18700 1 -- 172.21.15.7:0/1781210320 wait complete. 2024-01-27T02:39:08.765 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:39:08.874 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:39:08.877 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:39:08.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.897+0000 7fadf268f700 1 Processor -- start 2024-01-27T02:39:08.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.897+0000 7fadf268f700 1 -- start start 2024-01-27T02:39:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.898+0000 7fadf268f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fadec1409e0 0x7fadec0b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.898+0000 7fadf268f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fadec06e330 0x7fadec06e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.898+0000 7fadf268f700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadec0c30c0 con 0x7fadec140610 2024-01-27T02:39:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.898+0000 7fadf268f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadec06ecb0 con 0x7fadec06e330 2024-01-27T02:39:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.898+0000 7fadf268f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadec06ccf0 con 0x7fadec1409e0 2024-01-27T02:39:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.898+0000 7fadeb7fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fadec1409e0 0x7fadec0b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.898+0000 7fadf0c2c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fadec06e330 0x7fadec06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.898+0000 7fadebfff700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fadec140610 0x7fadec06bef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:52848/0 (socket says 172.21.15.7:52848) 2024-01-27T02:39:08.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.898+0000 7fadebfff700 1 -- 172.21.15.7:0/1634158627 learned_addr learned my addr 172.21.15.7:0/1634158627 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:08.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.899+0000 7fadeb7fe700 1 -- 172.21.15.7:0/1634158627 >> v2:172.21.15.188:3300/0 conn(0x7fadec06e330 msgr2=0x7fadec06e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:08.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.899+0000 7fadeb7fe700 1 --2- 172.21.15.7:0/1634158627 >> v2:172.21.15.188:3300/0 conn(0x7fadec06e330 0x7fadec06e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.899+0000 7fadeb7fe700 1 -- 172.21.15.7:0/1634158627 >> v1:172.21.15.188:6789/0 conn(0x7fadec140610 legacy=0x7fadec06bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:08.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.899+0000 7fadeb7fe700 1 -- 172.21.15.7:0/1634158627 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec06ca10 con 0x7fadec1409e0 2024-01-27T02:39:08.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.899+0000 7fadf0c2c700 1 --2- 172.21.15.7:0/1634158627 >> v2:172.21.15.188:3300/0 conn(0x7fadec06e330 0x7fadec06e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:39:08.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.899+0000 7fadeb7fe700 1 --2- 172.21.15.7:0/1634158627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fadec1409e0 0x7fadec0b09f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fade00098e0 tx=0x7fade00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d595759138720aa server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:08.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.899+0000 7fadeaffd700 1 -- 172.21.15.7:0/1634158627 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade000b640 con 0x7fadec1409e0 2024-01-27T02:39:08.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.899+0000 7fadeaffd700 1 -- 172.21.15.7:0/1634158627 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade000b7a0 con 0x7fadec1409e0 2024-01-27T02:39:08.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.899+0000 7fadeaffd700 1 -- 172.21.15.7:0/1634158627 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade000bb40 con 0x7fadec1409e0 2024-01-27T02:39:08.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.900+0000 7fadf268f700 1 -- 172.21.15.7:0/1634158627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fadec1409e0 msgr2=0x7fadec0b09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:08.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.900+0000 7fadf268f700 1 --2- 172.21.15.7:0/1634158627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fadec1409e0 0x7fadec0b09f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fade00098e0 tx=0x7fade00091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.900+0000 7fadf268f700 1 -- 172.21.15.7:0/1634158627 shutdown_connections 2024-01-27T02:39:08.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.900+0000 7fadf268f700 1 --2- 172.21.15.7:0/1634158627 >> v2:172.21.15.188:3300/0 conn(0x7fadec06e330 0x7fadec06e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.900+0000 7fadf268f700 1 --2- 172.21.15.7:0/1634158627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fadec1409e0 0x7fadec0b09f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.900+0000 7fadf268f700 1 -- 172.21.15.7:0/1634158627 >> 172.21.15.7:0/1634158627 conn(0x7fadec0cdc30 msgr2=0x7fadec0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:08.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.900+0000 7fadf268f700 1 -- 172.21.15.7:0/1634158627 shutdown_connections 2024-01-27T02:39:08.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.900+0000 7fadf268f700 1 -- 172.21.15.7:0/1634158627 wait complete. 2024-01-27T02:39:08.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.901+0000 7fadf268f700 1 Processor -- start 2024-01-27T02:39:08.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.902+0000 7fadf268f700 1 -- start start 2024-01-27T02:39:08.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.902+0000 7fadf268f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadec06e330 0x7fadec0afe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:08.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.902+0000 7fadf268f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fadec140610 0x7fadec15c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:08.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.903+0000 7fadf268f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec1409e0 0x7fadec06b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:08.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.903+0000 7fadf268f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec1580b0 con 0x7fadec1409e0 2024-01-27T02:39:08.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.903+0000 7fadf268f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec1574e0 con 0x7fadec06e330 2024-01-27T02:39:08.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.903+0000 7fadf268f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadec0b03a0 con 0x7fadec140610 2024-01-27T02:39:08.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.903+0000 7fadf0c2c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec1409e0 0x7fadec06b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:08.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.903+0000 7fadf0c2c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec1409e0 0x7fadec06b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49876/0 (socket says 172.21.15.7:49876) 2024-01-27T02:39:08.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.903+0000 7fadf0c2c700 1 -- 172.21.15.7:0/4020907823 learned_addr learned my addr 172.21.15.7:0/4020907823 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:08.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.903+0000 7fadeb7fe700 1 --2- 172.21.15.7:0/4020907823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fadec140610 0x7fadec15c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:08.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.903+0000 7fadebfff700 1 --2- 172.21.15.7:0/4020907823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadec06e330 0x7fadec0afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:08.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.903+0000 7fadf0c2c700 1 -- 172.21.15.7:0/4020907823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fadec140610 msgr2=0x7fadec15c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:08.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.903+0000 7fadf0c2c700 1 --2- 172.21.15.7:0/4020907823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fadec140610 0x7fadec15c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.903+0000 7fadf0c2c700 1 -- 172.21.15.7:0/4020907823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadec06e330 msgr2=0x7fadec0afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:08.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.903+0000 7fadf0c2c700 1 --2- 172.21.15.7:0/4020907823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadec06e330 0x7fadec0afe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:08.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.903+0000 7fadf0c2c700 1 -- 172.21.15.7:0/4020907823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade4009a20 con 0x7fadec1409e0 2024-01-27T02:39:08.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.903+0000 7fadebfff700 1 --2- 172.21.15.7:0/4020907823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadec06e330 0x7fadec0afe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:39:08.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.903+0000 7fadf0c2c700 1 --2- 172.21.15.7:0/4020907823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec1409e0 0x7fadec06b6e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fade4009f20 tx=0x7fade400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:08.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.904+0000 7fadd3fff700 1 -- 172.21.15.7:0/4020907823 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade400cb70 con 0x7fadec1409e0 2024-01-27T02:39:08.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.904+0000 7fadd3fff700 1 -- 172.21.15.7:0/4020907823 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade400f040 con 0x7fadec1409e0 2024-01-27T02:39:08.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.904+0000 7fadf268f700 1 -- 172.21.15.7:0/4020907823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade0009420 con 0x7fadec1409e0 2024-01-27T02:39:08.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.904+0000 7fadd3fff700 1 -- 172.21.15.7:0/4020907823 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4014480 con 0x7fadec1409e0 2024-01-27T02:39:08.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.904+0000 7fadf268f700 1 -- 172.21.15.7:0/4020907823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadec157a60 con 0x7fadec1409e0 2024-01-27T02:39:08.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.905+0000 7fadd3fff700 1 -- 172.21.15.7:0/4020907823 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fade4003020 con 0x7fadec1409e0 2024-01-27T02:39:08.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.905+0000 7fadf268f700 1 -- 172.21.15.7:0/4020907823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadec158220 con 0x7fadec1409e0 2024-01-27T02:39:08.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.906+0000 7fadd3fff700 1 --2- 172.21.15.7:0/4020907823 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fadd8066f30 0x7fadd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:08.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.906+0000 7fadd3fff700 1 -- 172.21.15.7:0/4020907823 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fade4084dd0 con 0x7fadec1409e0 2024-01-27T02:39:08.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.906+0000 7fadebfff700 1 --2- 172.21.15.7:0/4020907823 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fadd8066f30 0x7fadd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:08.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.907+0000 7fadebfff700 1 --2- 172.21.15.7:0/4020907823 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fadd8066f30 0x7fadd80693e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7faddc007110 tx=0x7faddc007f10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:08.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:08.910+0000 7fadd3fff700 1 -- 172.21.15.7:0/4020907823 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fade4052000 con 0x7fadec1409e0 2024-01-27T02:39:09.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.166+0000 7fadf268f700 1 -- 172.21.15.7:0/4020907823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadec04c710 con 0x7fadec1409e0 2024-01-27T02:39:09.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.167+0000 7fadd3fff700 1 -- 172.21.15.7:0/4020907823 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fade4055620 con 0x7fadec1409e0 2024-01-27T02:39:09.167 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:09.167 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3997828193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3997828193},{"type":"v1","addr":"172.21.15.188:6837","nonce":3997828193}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1132338302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1132338302},{"type":"v1","addr":"172.21.15.188:6836","nonce":1132338302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/740290645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":740290645},{"type":"v1","addr":"172.21.15.7:6833","nonce":740290645}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:38:31.328277+0000","modified":"2024-01-27T02:38:34.940811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7029":{"gid":7029,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1258751919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1258751919},{"type":"v1","addr":"172.21.15.7:6835","nonce":1258751919}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-27T02:39:09.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.169+0000 7fadf268f700 1 -- 172.21.15.7:0/4020907823 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fadd8066f30 msgr2=0x7fadd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:09.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.169+0000 7fadf268f700 1 --2- 172.21.15.7:0/4020907823 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fadd8066f30 0x7fadd80693e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7faddc007110 tx=0x7faddc007f10 comp rx=0 tx=0).stop 2024-01-27T02:39:09.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.169+0000 7fadf268f700 1 -- 172.21.15.7:0/4020907823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec1409e0 msgr2=0x7fadec06b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:09.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.169+0000 7fadf268f700 1 --2- 172.21.15.7:0/4020907823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec1409e0 0x7fadec06b6e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fade4009f20 tx=0x7fade400b040 comp rx=0 tx=0).stop 2024-01-27T02:39:09.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.169+0000 7fadf268f700 1 -- 172.21.15.7:0/4020907823 shutdown_connections 2024-01-27T02:39:09.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.169+0000 7fadf268f700 1 --2- 172.21.15.7:0/4020907823 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fadd8066f30 0x7fadd80693e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.169+0000 7fadf268f700 1 --2- 172.21.15.7:0/4020907823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec1409e0 0x7fadec06b6e0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.169+0000 7fadf268f700 1 --2- 172.21.15.7:0/4020907823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadec06e330 0x7fadec0afe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.169+0000 7fadf268f700 1 --2- 172.21.15.7:0/4020907823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fadec140610 0x7fadec15c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.169+0000 7fadf268f700 1 -- 172.21.15.7:0/4020907823 >> 172.21.15.7:0/4020907823 conn(0x7fadec0cdc30 msgr2=0x7fadec0c3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:09.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.169+0000 7fadf268f700 1 -- 172.21.15.7:0/4020907823 shutdown_connections 2024-01-27T02:39:09.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.169+0000 7fadf268f700 1 -- 172.21.15.7:0/4020907823 wait complete. 2024-01-27T02:39:09.171 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 68 2024-01-27T02:39:09.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:39:09.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.313+0000 7fd6c42bb700 1 Processor -- start 2024-01-27T02:39:09.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.313+0000 7fd6c42bb700 1 -- start start 2024-01-27T02:39:09.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.315+0000 7fd6c42bb700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd6bc0ca540 0x7fd6bc0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:09.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.315+0000 7fd6c42bb700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6bc0ce5e0 con 0x7fd6bc0c4cc0 2024-01-27T02:39:09.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.315+0000 7fd6c42bb700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6bc0cd9d0 con 0x7fd6bc15a100 2024-01-27T02:39:09.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.315+0000 7fd6c42bb700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6bc0cdcb0 con 0x7fd6bc0ca540 2024-01-27T02:39:09.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.315+0000 7fd6c1856700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd6bc0c4cc0 0x7fd6bc161d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59878/0 (socket says 172.21.15.7:59878) 2024-01-27T02:39:09.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.315+0000 7fd6c1856700 1 -- 172.21.15.7:0/531863348 learned_addr learned my addr 172.21.15.7:0/531863348 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:09.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.315+0000 7fd6c2858700 1 --2- 172.21.15.7:0/531863348 >> v2:172.21.15.188:3300/0 conn(0x7fd6bc0ca540 0x7fd6bc0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:09.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.315+0000 7fd6c1055700 1 -- 172.21.15.7:0/531863348 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3384215956 0 0) 0x7fd6bc0ce5e0 con 0x7fd6bc0c4cc0 2024-01-27T02:39:09.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.315+0000 7fd6c1055700 1 -- 172.21.15.7:0/531863348 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6a8003570 con 0x7fd6bc0c4cc0 2024-01-27T02:39:09.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.315+0000 7fd6c1055700 1 -- 172.21.15.7:0/531863348 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 894022641 0 0) 0x7fd6bc0cd9d0 con 0x7fd6bc15a100 2024-01-27T02:39:09.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.315+0000 7fd6c1055700 1 -- 172.21.15.7:0/531863348 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6bc0ce5e0 con 0x7fd6bc15a100 2024-01-27T02:39:09.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.315+0000 7fd6c1055700 1 -- 172.21.15.7:0/531863348 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3933185472 0 0) 0x7fd6a8003570 con 0x7fd6bc0c4cc0 2024-01-27T02:39:09.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.315+0000 7fd6c1055700 1 -- 172.21.15.7:0/531863348 >> v2:172.21.15.188:3300/0 conn(0x7fd6bc0ca540 msgr2=0x7fd6bc0cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:09.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.315+0000 7fd6c1055700 1 --2- 172.21.15.7:0/531863348 >> v2:172.21.15.188:3300/0 conn(0x7fd6bc0ca540 0x7fd6bc0cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.315+0000 7fd6c1055700 1 -- 172.21.15.7:0/531863348 >> v1:172.21.15.188:6789/0 conn(0x7fd6bc15a100 legacy=0x7fd6bc0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:09.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.315+0000 7fd6c1055700 1 -- 172.21.15.7:0/531863348 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6bc0cdf90 con 0x7fd6bc0c4cc0 2024-01-27T02:39:09.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.315+0000 7fd6c1055700 1 -- 172.21.15.7:0/531863348 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7fd6ac001470 con 0x7fd6bc0c4cc0 2024-01-27T02:39:09.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.315+0000 7fd6c1055700 1 -- 172.21.15.7:0/531863348 >> v1:172.21.15.7:6789/0 conn(0x7fd6bc0c4cc0 legacy=0x7fd6bc161d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:09.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.316+0000 7fd6c1055700 1 --2- 172.21.15.7:0/531863348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6a8003c40 0x7fd6a8004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:09.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.316+0000 7fd6c1055700 1 --2- 172.21.15.7:0/531863348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6a8004810 0x7fd6a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:09.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.316+0000 7fd6c1055700 1 --2- 172.21.15.7:0/531863348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a8005210 0x7fd6a800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:09.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.316+0000 7fd6c1055700 1 -- 172.21.15.7:0/531863348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc0cdcb0 con 0x7fd6a8005210 2024-01-27T02:39:09.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.316+0000 7fd6c1055700 1 -- 172.21.15.7:0/531863348 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a800c130 con 0x7fd6a8003c40 2024-01-27T02:39:09.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.316+0000 7fd6c1055700 1 -- 172.21.15.7:0/531863348 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a800c2c0 con 0x7fd6a8004810 2024-01-27T02:39:09.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.316+0000 7fd6c2858700 1 --2- 172.21.15.7:0/531863348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a8005210 0x7fd6a800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:09.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.316+0000 7fd6c2057700 1 --2- 172.21.15.7:0/531863348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6a8003c40 0x7fd6a8004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:09.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.316+0000 7fd6c1856700 1 --2- 172.21.15.7:0/531863348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6a8004810 0x7fd6a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:09.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.316+0000 7fd6c2858700 1 -- 172.21.15.7:0/531863348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6a8004810 msgr2=0x7fd6a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:09.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.316+0000 7fd6c2858700 1 --2- 172.21.15.7:0/531863348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6a8004810 0x7fd6a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.316+0000 7fd6c2858700 1 -- 172.21.15.7:0/531863348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6a8003c40 msgr2=0x7fd6a8004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:09.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.316+0000 7fd6c2858700 1 --2- 172.21.15.7:0/531863348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6a8003c40 0x7fd6a8004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.316+0000 7fd6c2858700 1 -- 172.21.15.7:0/531863348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a8003570 con 0x7fd6a8005210 2024-01-27T02:39:09.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.316+0000 7fd6c1856700 1 --2- 172.21.15.7:0/531863348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6a8004810 0x7fd6a8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:09.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.316+0000 7fd6c2858700 1 -- 172.21.15.7:0/531863348 reap_dead start 2024-01-27T02:39:09.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.316+0000 7fd6c2858700 1 --2- 172.21.15.7:0/531863348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a8005210 0x7fd6a800ba60 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400e240 tx=0x7fd6b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e48a1becbc4d51c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:09.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.316+0000 7fd6c1055700 1 -- 172.21.15.7:0/531863348 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4009ac0 con 0x7fd6a8005210 2024-01-27T02:39:09.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.316+0000 7fd6c1055700 1 -- 172.21.15.7:0/531863348 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b400f040 con 0x7fd6a8005210 2024-01-27T02:39:09.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.316+0000 7fd6c1055700 1 -- 172.21.15.7:0/531863348 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b400ed40 con 0x7fd6a8005210 2024-01-27T02:39:09.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.317+0000 7fd6c42bb700 1 -- 172.21.15.7:0/531863348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a8005210 msgr2=0x7fd6a800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:09.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.317+0000 7fd6c42bb700 1 --2- 172.21.15.7:0/531863348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a8005210 0x7fd6a800ba60 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400e240 tx=0x7fd6b400b040 comp rx=0 tx=0).stop 2024-01-27T02:39:09.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.317+0000 7fd6c42bb700 1 -- 172.21.15.7:0/531863348 shutdown_connections 2024-01-27T02:39:09.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.317+0000 7fd6c42bb700 1 --2- 172.21.15.7:0/531863348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a8005210 0x7fd6a800ba60 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.317+0000 7fd6c42bb700 1 -- 172.21.15.7:0/531863348 >> 172.21.15.7:0/531863348 conn(0x7fd6bc098020 msgr2=0x7fd6bc0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:09.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.317+0000 7fd6c42bb700 1 -- 172.21.15.7:0/531863348 shutdown_connections 2024-01-27T02:39:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.317+0000 7fd6c42bb700 1 -- 172.21.15.7:0/531863348 wait complete. 2024-01-27T02:39:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.319+0000 7fd6c42bb700 1 Processor -- start 2024-01-27T02:39:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.319+0000 7fd6c42bb700 1 -- start start 2024-01-27T02:39:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.320+0000 7fd6c42bb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6a8005210 0x7fd6bc154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.320+0000 7fd6c42bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6bc0c8400 0x7fd6bc068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.320+0000 7fd6c42bb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6bc0c8850 0x7fd6bc0685d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.320+0000 7fd6c42bb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc0b4730 con 0x7fd6bc0c8400 2024-01-27T02:39:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.320+0000 7fd6c42bb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc154ab0 con 0x7fd6bc0c8850 2024-01-27T02:39:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.320+0000 7fd6c42bb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc15b480 con 0x7fd6a8005210 2024-01-27T02:39:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.320+0000 7fd6c1856700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6bc0c8400 0x7fd6bc068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.320+0000 7fd6c1856700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6bc0c8400 0x7fd6bc068090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49894/0 (socket says 172.21.15.7:49894) 2024-01-27T02:39:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.320+0000 7fd6c1856700 1 -- 172.21.15.7:0/1827067442 learned_addr learned my addr 172.21.15.7:0/1827067442 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.320+0000 7fd6c2858700 1 --2- 172.21.15.7:0/1827067442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6bc0c8850 0x7fd6bc0685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.320+0000 7fd6c2057700 1 --2- 172.21.15.7:0/1827067442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6a8005210 0x7fd6bc154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.320+0000 7fd6c1856700 1 -- 172.21.15.7:0/1827067442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6a8005210 msgr2=0x7fd6bc154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.320+0000 7fd6c1856700 1 --2- 172.21.15.7:0/1827067442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6a8005210 0x7fd6bc154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.320+0000 7fd6c1856700 1 -- 172.21.15.7:0/1827067442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6bc0c8850 msgr2=0x7fd6bc0685d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.320+0000 7fd6c1856700 1 --2- 172.21.15.7:0/1827067442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6bc0c8850 0x7fd6bc0685d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.320+0000 7fd6c1856700 1 -- 172.21.15.7:0/1827067442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6bc15a870 con 0x7fd6bc0c8400 2024-01-27T02:39:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.321+0000 7fd6c1856700 1 --2- 172.21.15.7:0/1827067442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6bc0c8400 0x7fd6bc068090 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00e7b0 tx=0x7fd6ac000c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.321+0000 7fd6b27fc700 1 -- 172.21.15.7:0/1827067442 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac001aa0 con 0x7fd6bc0c8400 2024-01-27T02:39:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.321+0000 7fd6b27fc700 1 -- 172.21.15.7:0/1827067442 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6ac010040 con 0x7fd6bc0c8400 2024-01-27T02:39:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.321+0000 7fd6c42bb700 1 -- 172.21.15.7:0/1827067442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6bc15ab50 con 0x7fd6bc0c8400 2024-01-27T02:39:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.321+0000 7fd6b27fc700 1 -- 172.21.15.7:0/1827067442 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac011670 con 0x7fd6bc0c8400 2024-01-27T02:39:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.321+0000 7fd6c42bb700 1 -- 172.21.15.7:0/1827067442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6bc15b110 con 0x7fd6bc0c8400 2024-01-27T02:39:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.322+0000 7fd6c42bb700 1 -- 172.21.15.7:0/1827067442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6bc04c710 con 0x7fd6bc0c8400 2024-01-27T02:39:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.322+0000 7fd6b27fc700 1 -- 172.21.15.7:0/1827067442 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd6ac011a20 con 0x7fd6bc0c8400 2024-01-27T02:39:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.323+0000 7fd6b27fc700 1 --2- 172.21.15.7:0/1827067442 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd6a0066f30 0x7fd6a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.323+0000 7fd6b27fc700 1 -- 172.21.15.7:0/1827067442 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fd6ac0835a0 con 0x7fd6bc0c8400 2024-01-27T02:39:09.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.323+0000 7fd6c2057700 1 --2- 172.21.15.7:0/1827067442 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd6a0066f30 0x7fd6a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:09.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.323+0000 7fd6c2057700 1 --2- 172.21.15.7:0/1827067442 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd6a0066f30 0x7fd6a00693e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8000c00 tx=0x7fd6b8007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:09.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.327+0000 7fd6b27fc700 1 -- 172.21.15.7:0/1827067442 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd6ac04d5a0 con 0x7fd6bc0c8400 2024-01-27T02:39:09.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.527+0000 7fd6c42bb700 1 -- 172.21.15.7:0/1827067442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6bc0ce270 con 0x7fd6bc0c8400 2024-01-27T02:39:09.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.528+0000 7fd6b27fc700 1 -- 172.21.15.7:0/1827067442 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+15421 (secure 0 0 0) 0x7fd6ac019090 con 0x7fd6bc0c8400 2024-01-27T02:39:09.529 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:09.529 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":137,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:38:59.871665+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:38:29.824892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-27T02:38:30.301072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.e","mappings":[{"from":0,"to":5}]},{"pgid":"17.24","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:39:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.530+0000 7fd6c42bb700 1 -- 172.21.15.7:0/1827067442 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd6a0066f30 msgr2=0x7fd6a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.530+0000 7fd6c42bb700 1 --2- 172.21.15.7:0/1827067442 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd6a0066f30 0x7fd6a00693e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8000c00 tx=0x7fd6b8007d30 comp rx=0 tx=0).stop 2024-01-27T02:39:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.530+0000 7fd6c42bb700 1 -- 172.21.15.7:0/1827067442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6bc0c8400 msgr2=0x7fd6bc068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.530+0000 7fd6c42bb700 1 --2- 172.21.15.7:0/1827067442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6bc0c8400 0x7fd6bc068090 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00e7b0 tx=0x7fd6ac000c50 comp rx=0 tx=0).stop 2024-01-27T02:39:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.531+0000 7fd6c42bb700 1 -- 172.21.15.7:0/1827067442 shutdown_connections 2024-01-27T02:39:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.531+0000 7fd6c42bb700 1 --2- 172.21.15.7:0/1827067442 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd6a0066f30 0x7fd6a00693e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.531+0000 7fd6c42bb700 1 --2- 172.21.15.7:0/1827067442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6bc0c8850 0x7fd6bc0685d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.531+0000 7fd6c42bb700 1 --2- 172.21.15.7:0/1827067442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6a8005210 0x7fd6bc154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.531+0000 7fd6c42bb700 1 --2- 172.21.15.7:0/1827067442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6bc0c8400 0x7fd6bc068090 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.531+0000 7fd6c42bb700 1 -- 172.21.15.7:0/1827067442 >> 172.21.15.7:0/1827067442 conn(0x7fd6bc098020 msgr2=0x7fd6bc06c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.531+0000 7fd6c42bb700 1 -- 172.21.15.7:0/1827067442 shutdown_connections 2024-01-27T02:39:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.531+0000 7fd6c42bb700 1 -- 172.21.15.7:0/1827067442 wait complete. 2024-01-27T02:39:09.545 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-27T02:39:09.545 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:39:09.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.676+0000 7f4a530fb700 1 Processor -- start 2024-01-27T02:39:09.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.676+0000 7f4a530fb700 1 -- start start 2024-01-27T02:39:09.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.677+0000 7f4a530fb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4a4c0a7c10 0x7f4a4c0ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:09.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.677+0000 7f4a530fb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c0ae880 0x7f4a4c06f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:09.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.677+0000 7f4a530fb700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4a4c0a7fe0 0x7f4a4c06f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:09.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.677+0000 7f4a530fb700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a4c0afde0 con 0x7f4a4c0a7c10 2024-01-27T02:39:09.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.677+0000 7f4a530fb700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a4c15a880 con 0x7f4a4c0a7fe0 2024-01-27T02:39:09.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.677+0000 7f4a530fb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c0af950 con 0x7f4a4c0ae880 2024-01-27T02:39:09.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.677+0000 7f4a50e97700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4a4c0a7c10 0x7f4a4c0ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:09.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.678+0000 7f4a50e97700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4a4c0a7c10 0x7f4a4c0ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49908/0 (socket says 172.21.15.7:49908) 2024-01-27T02:39:09.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.678+0000 7f4a50e97700 1 -- 172.21.15.7:0/3583240100 learned_addr learned my addr 172.21.15.7:0/3583240100 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:09.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.678+0000 7f4a51698700 1 --2- 172.21.15.7:0/3583240100 >> v2:172.21.15.188:3300/0 conn(0x7f4a4c0a7fe0 0x7f4a4c06f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:09.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.678+0000 7f4a4bfff700 1 --2- 172.21.15.7:0/3583240100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c0ae880 0x7f4a4c06f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:09.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.678+0000 7f4a50e97700 1 -- 172.21.15.7:0/3583240100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c0ae880 msgr2=0x7f4a4c06f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:09.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.678+0000 7f4a50e97700 1 --2- 172.21.15.7:0/3583240100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c0ae880 0x7f4a4c06f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.678+0000 7f4a50e97700 1 -- 172.21.15.7:0/3583240100 >> v2:172.21.15.188:3300/0 conn(0x7f4a4c0a7fe0 msgr2=0x7f4a4c06f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:09.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.678+0000 7f4a50e97700 1 --2- 172.21.15.7:0/3583240100 >> v2:172.21.15.188:3300/0 conn(0x7f4a4c0a7fe0 0x7f4a4c06f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.678+0000 7f4a50e97700 1 -- 172.21.15.7:0/3583240100 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c0aed40 con 0x7f4a4c0a7c10 2024-01-27T02:39:09.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.678+0000 7f4a4bfff700 1 --2- 172.21.15.7:0/3583240100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c0ae880 0x7f4a4c06f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:39:09.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.678+0000 7f4a51698700 1 --2- 172.21.15.7:0/3583240100 >> v2:172.21.15.188:3300/0 conn(0x7f4a4c0a7fe0 0x7f4a4c06f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:09.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.678+0000 7f4a50e97700 1 --2- 172.21.15.7:0/3583240100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c0a7c10 0x7f4a4c0ae340 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c06da70 tx=0x7f4a40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3876c7f59aa92aae server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:09.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.678+0000 7f4a4b7fe700 1 -- 172.21.15.7:0/3583240100 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4000b860 con 0x7f4a4c0a7c10 2024-01-27T02:39:09.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.678+0000 7f4a4b7fe700 1 -- 172.21.15.7:0/3583240100 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4000b9c0 con 0x7f4a4c0a7c10 2024-01-27T02:39:09.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.678+0000 7f4a4b7fe700 1 -- 172.21.15.7:0/3583240100 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4000bd60 con 0x7f4a4c0a7c10 2024-01-27T02:39:09.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.679+0000 7f4a530fb700 1 -- 172.21.15.7:0/3583240100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c0a7c10 msgr2=0x7f4a4c0ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:09.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.679+0000 7f4a530fb700 1 --2- 172.21.15.7:0/3583240100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c0a7c10 0x7f4a4c0ae340 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c06da70 tx=0x7f4a40009250 comp rx=0 tx=0).stop 2024-01-27T02:39:09.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.679+0000 7f4a530fb700 1 -- 172.21.15.7:0/3583240100 shutdown_connections 2024-01-27T02:39:09.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.679+0000 7f4a530fb700 1 --2- 172.21.15.7:0/3583240100 >> v2:172.21.15.188:3300/0 conn(0x7f4a4c0a7fe0 0x7f4a4c06f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.679+0000 7f4a530fb700 1 --2- 172.21.15.7:0/3583240100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c0a7c10 0x7f4a4c0ae340 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.679+0000 7f4a530fb700 1 --2- 172.21.15.7:0/3583240100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c0ae880 0x7f4a4c06f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.679+0000 7f4a530fb700 1 -- 172.21.15.7:0/3583240100 >> 172.21.15.7:0/3583240100 conn(0x7f4a4c0679b0 msgr2=0x7f4a4c067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:09.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.679+0000 7f4a530fb700 1 -- 172.21.15.7:0/3583240100 shutdown_connections 2024-01-27T02:39:09.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.679+0000 7f4a530fb700 1 -- 172.21.15.7:0/3583240100 wait complete. 2024-01-27T02:39:09.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.680+0000 7f4a530fb700 1 Processor -- start 2024-01-27T02:39:09.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.680+0000 7f4a530fb700 1 -- start start 2024-01-27T02:39:09.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.681+0000 7f4a530fb700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a4c0a7c10 0x7f4a4c098d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:09.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.681+0000 7f4a530fb700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c0a7fe0 0x7f4a4c150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:09.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.681+0000 7f4a530fb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c0ae880 0x7f4a4c0b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:09.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.681+0000 7f4a530fb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c151730 con 0x7f4a4c0ae880 2024-01-27T02:39:09.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.681+0000 7f4a530fb700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c150880 con 0x7f4a4c0a7c10 2024-01-27T02:39:09.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.681+0000 7f4a530fb700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c150b20 con 0x7f4a4c0a7fe0 2024-01-27T02:39:09.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.681+0000 7f4a51698700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c0ae880 0x7f4a4c0b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:09.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.681+0000 7f4a51698700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c0ae880 0x7f4a4c0b1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49912/0 (socket says 172.21.15.7:49912) 2024-01-27T02:39:09.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.681+0000 7f4a51698700 1 -- 172.21.15.7:0/2138565394 learned_addr learned my addr 172.21.15.7:0/2138565394 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:09.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.681+0000 7f4a50e97700 1 --2- 172.21.15.7:0/2138565394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a4c0a7c10 0x7f4a4c098d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:09.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.681+0000 7f4a4bfff700 1 --2- 172.21.15.7:0/2138565394 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c0a7fe0 0x7f4a4c150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:09.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.682+0000 7f4a51698700 1 -- 172.21.15.7:0/2138565394 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c0a7fe0 msgr2=0x7f4a4c150340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:09.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.682+0000 7f4a51698700 1 --2- 172.21.15.7:0/2138565394 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c0a7fe0 0x7f4a4c150340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.682+0000 7f4a51698700 1 -- 172.21.15.7:0/2138565394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a4c0a7c10 msgr2=0x7f4a4c098d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:09.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.682+0000 7f4a51698700 1 --2- 172.21.15.7:0/2138565394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a4c0a7c10 0x7f4a4c098d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.682+0000 7f4a51698700 1 -- 172.21.15.7:0/2138565394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a400094f0 con 0x7f4a4c0ae880 2024-01-27T02:39:09.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.682+0000 7f4a51698700 1 --2- 172.21.15.7:0/2138565394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c0ae880 0x7f4a4c0b1240 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400d150 tx=0x7f4a44009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:09.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.682+0000 7f4a48ff9700 1 -- 172.21.15.7:0/2138565394 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4400dc30 con 0x7f4a4c0ae880 2024-01-27T02:39:09.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.682+0000 7f4a48ff9700 1 -- 172.21.15.7:0/2138565394 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4400f040 con 0x7f4a4c0ae880 2024-01-27T02:39:09.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.682+0000 7f4a530fb700 1 -- 172.21.15.7:0/2138565394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4c150e00 con 0x7f4a4c0ae880 2024-01-27T02:39:09.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.682+0000 7f4a48ff9700 1 -- 172.21.15.7:0/2138565394 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a44015400 con 0x7f4a4c0ae880 2024-01-27T02:39:09.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.682+0000 7f4a530fb700 1 -- 172.21.15.7:0/2138565394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4c1513c0 con 0x7f4a4c0ae880 2024-01-27T02:39:09.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.683+0000 7f4a530fb700 1 -- 172.21.15.7:0/2138565394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a4c1518a0 con 0x7f4a4c0ae880 2024-01-27T02:39:09.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.683+0000 7f4a48ff9700 1 -- 172.21.15.7:0/2138565394 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4a440157f0 con 0x7f4a4c0ae880 2024-01-27T02:39:09.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.684+0000 7f4a48ff9700 1 --2- 172.21.15.7:0/2138565394 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a30066f30 0x7f4a300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:09.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.684+0000 7f4a48ff9700 1 -- 172.21.15.7:0/2138565394 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f4a44084ce0 con 0x7f4a4c0ae880 2024-01-27T02:39:09.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.684+0000 7f4a50e97700 1 --2- 172.21.15.7:0/2138565394 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a30066f30 0x7f4a300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:09.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.685+0000 7f4a50e97700 1 --2- 172.21.15.7:0/2138565394 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a30066f30 0x7f4a300693e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c06da70 tx=0x7f4a40017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:09.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.688+0000 7f4a48ff9700 1 -- 172.21.15.7:0/2138565394 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a44051f10 con 0x7f4a4c0ae880 2024-01-27T02:39:09.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.940+0000 7f4a530fb700 1 -- 172.21.15.7:0/2138565394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a4c04c710 con 0x7f4a4c0ae880 2024-01-27T02:39:09.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.941+0000 7f4a48ff9700 1 -- 172.21.15.7:0/2138565394 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f4a44055530 con 0x7f4a4c0ae880 2024-01-27T02:39:09.942 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:09.942 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3997828193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3997828193},{"type":"v1","addr":"172.21.15.188:6837","nonce":3997828193}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1132338302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1132338302},{"type":"v1","addr":"172.21.15.188:6836","nonce":1132338302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/740290645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":740290645},{"type":"v1","addr":"172.21.15.7:6833","nonce":740290645}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:38:31.328277+0000","modified":"2024-01-27T02:38:34.940811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7029":{"gid":7029,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1258751919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1258751919},{"type":"v1","addr":"172.21.15.7:6835","nonce":1258751919}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-27T02:39:09.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.943+0000 7f4a530fb700 1 -- 172.21.15.7:0/2138565394 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a30066f30 msgr2=0x7f4a300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:09.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.943+0000 7f4a530fb700 1 --2- 172.21.15.7:0/2138565394 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a30066f30 0x7f4a300693e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c06da70 tx=0x7f4a40017000 comp rx=0 tx=0).stop 2024-01-27T02:39:09.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.943+0000 7f4a530fb700 1 -- 172.21.15.7:0/2138565394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c0ae880 msgr2=0x7f4a4c0b1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:09.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.943+0000 7f4a530fb700 1 --2- 172.21.15.7:0/2138565394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c0ae880 0x7f4a4c0b1240 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400d150 tx=0x7f4a44009a50 comp rx=0 tx=0).stop 2024-01-27T02:39:09.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.943+0000 7f4a530fb700 1 -- 172.21.15.7:0/2138565394 shutdown_connections 2024-01-27T02:39:09.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.943+0000 7f4a530fb700 1 --2- 172.21.15.7:0/2138565394 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4a30066f30 0x7f4a300693e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.943+0000 7f4a530fb700 1 --2- 172.21.15.7:0/2138565394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c0ae880 0x7f4a4c0b1240 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.943+0000 7f4a530fb700 1 --2- 172.21.15.7:0/2138565394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a4c0a7c10 0x7f4a4c098d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.943+0000 7f4a530fb700 1 --2- 172.21.15.7:0/2138565394 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c0a7fe0 0x7f4a4c150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:09.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.943+0000 7f4a530fb700 1 -- 172.21.15.7:0/2138565394 >> 172.21.15.7:0/2138565394 conn(0x7f4a4c0679b0 msgr2=0x7f4a4c159530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:09.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.943+0000 7f4a530fb700 1 -- 172.21.15.7:0/2138565394 shutdown_connections 2024-01-27T02:39:09.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:09.943+0000 7f4a530fb700 1 -- 172.21.15.7:0/2138565394 wait complete. 2024-01-27T02:39:09.945 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 68 2024-01-27T02:39:09.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:39:10.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.089+0000 7fe51c261700 1 Processor -- start 2024-01-27T02:39:10.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.089+0000 7fe51c261700 1 -- start start 2024-01-27T02:39:10.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.090+0000 7fe51c261700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe51415b2a0 0x7fe5140aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:10.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.090+0000 7fe51c261700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe5140b04a0 0x7fe514163d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:10.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.090+0000 7fe51c261700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5140b0920 con 0x7fe51415a070 2024-01-27T02:39:10.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.090+0000 7fe51c261700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe51415b710 con 0x7fe5140b04a0 2024-01-27T02:39:10.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.090+0000 7fe51c261700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5140689c0 con 0x7fe51415b2a0 2024-01-27T02:39:10.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.090+0000 7fe5197fc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe51415b2a0 0x7fe5140aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:10.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.090+0000 7fe51a7fe700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe5140b04a0 0x7fe514163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:10.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.091+0000 7fe51a7fe700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe5140b04a0 0x7fe514163d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:52660/0 (socket says 172.21.15.7:52660) 2024-01-27T02:39:10.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.091+0000 7fe5197fc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe51415b2a0 0x7fe5140aff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:41334/0 (socket says 172.21.15.7:41334) 2024-01-27T02:39:10.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.091+0000 7fe51a7fe700 1 -- 172.21.15.7:0/1808445572 learned_addr learned my addr 172.21.15.7:0/1808445572 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:10.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.091+0000 7fe519ffd700 1 --1- 172.21.15.7:0/1808445572 >> v1:172.21.15.188:6789/0 conn(0x7fe51415a070 0x7fe5140caf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:52874/0 (socket says 172.21.15.7:52874) 2024-01-27T02:39:10.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.091+0000 7fe5197fc700 1 -- 172.21.15.7:0/1808445572 >> v2:172.21.15.188:3300/0 conn(0x7fe5140b04a0 msgr2=0x7fe514163d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:10.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.091+0000 7fe5197fc700 1 --2- 172.21.15.7:0/1808445572 >> v2:172.21.15.188:3300/0 conn(0x7fe5140b04a0 0x7fe514163d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.091+0000 7fe5197fc700 1 -- 172.21.15.7:0/1808445572 >> v1:172.21.15.188:6789/0 conn(0x7fe51415a070 legacy=0x7fe5140caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:10.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.091+0000 7fe5197fc700 1 -- 172.21.15.7:0/1808445572 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5140ae5f0 con 0x7fe51415b2a0 2024-01-27T02:39:10.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.091+0000 7fe5197fc700 1 --2- 172.21.15.7:0/1808445572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe51415b2a0 0x7fe5140aff60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe5040098e0 tx=0x7fe5040091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c380c284e6c560d3 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:10.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.091+0000 7fe518ffb700 1 -- 172.21.15.7:0/1808445572 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50400b640 con 0x7fe51415b2a0 2024-01-27T02:39:10.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.091+0000 7fe518ffb700 1 -- 172.21.15.7:0/1808445572 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50400b7a0 con 0x7fe51415b2a0 2024-01-27T02:39:10.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.091+0000 7fe518ffb700 1 -- 172.21.15.7:0/1808445572 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50400bb40 con 0x7fe51415b2a0 2024-01-27T02:39:10.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.092+0000 7fe51c261700 1 -- 172.21.15.7:0/1808445572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe51415b2a0 msgr2=0x7fe5140aff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.092+0000 7fe51c261700 1 --2- 172.21.15.7:0/1808445572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe51415b2a0 0x7fe5140aff60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe5040098e0 tx=0x7fe5040091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.092+0000 7fe51c261700 1 -- 172.21.15.7:0/1808445572 shutdown_connections 2024-01-27T02:39:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.092+0000 7fe51c261700 1 --2- 172.21.15.7:0/1808445572 >> v2:172.21.15.188:3300/0 conn(0x7fe5140b04a0 0x7fe514163d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.092+0000 7fe51c261700 1 --2- 172.21.15.7:0/1808445572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe51415b2a0 0x7fe5140aff60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.092+0000 7fe51c261700 1 -- 172.21.15.7:0/1808445572 >> 172.21.15.7:0/1808445572 conn(0x7fe514098020 msgr2=0x7fe5140711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.092+0000 7fe51c261700 1 -- 172.21.15.7:0/1808445572 shutdown_connections 2024-01-27T02:39:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.092+0000 7fe51c261700 1 -- 172.21.15.7:0/1808445572 wait complete. 2024-01-27T02:39:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.094+0000 7fe51c261700 1 Processor -- start 2024-01-27T02:39:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.094+0000 7fe51c261700 1 -- start start 2024-01-27T02:39:10.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.095+0000 7fe51c261700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5140b04a0 0x7fe51415ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:10.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.095+0000 7fe51c261700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe51415a070 0x7fe514155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.095+0000 7fe51c261700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe51415b2a0 0x7fe51415fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.095+0000 7fe51c261700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51415f050 con 0x7fe5140b04a0 2024-01-27T02:39:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.095+0000 7fe51c261700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51415f4e0 con 0x7fe51415b2a0 2024-01-27T02:39:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.095+0000 7fe51c261700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe51415e440 con 0x7fe51415a070 2024-01-27T02:39:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.095+0000 7fe519ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5140b04a0 0x7fe51415ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.095+0000 7fe519ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5140b04a0 0x7fe51415ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49928/0 (socket says 172.21.15.7:49928) 2024-01-27T02:39:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.095+0000 7fe519ffd700 1 -- 172.21.15.7:0/3895543728 learned_addr learned my addr 172.21.15.7:0/3895543728 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.095+0000 7fe51a7fe700 1 --2- 172.21.15.7:0/3895543728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe51415b2a0 0x7fe51415fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.095+0000 7fe5197fc700 1 --2- 172.21.15.7:0/3895543728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe51415a070 0x7fe514155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.095+0000 7fe519ffd700 1 -- 172.21.15.7:0/3895543728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe51415a070 msgr2=0x7fe514155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.095+0000 7fe519ffd700 1 --2- 172.21.15.7:0/3895543728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe51415a070 0x7fe514155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.095+0000 7fe519ffd700 1 -- 172.21.15.7:0/3895543728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe51415b2a0 msgr2=0x7fe51415fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:10.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.095+0000 7fe519ffd700 1 --2- 172.21.15.7:0/3895543728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe51415b2a0 0x7fe51415fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.095+0000 7fe519ffd700 1 -- 172.21.15.7:0/3895543728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe504009420 con 0x7fe5140b04a0 2024-01-27T02:39:10.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.096+0000 7fe519ffd700 1 --2- 172.21.15.7:0/3895543728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5140b04a0 0x7fe51415ddd0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe51000c7c0 tx=0x7fe510007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:10.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.096+0000 7fe509ffb700 1 -- 172.21.15.7:0/3895543728 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510005a70 con 0x7fe5140b04a0 2024-01-27T02:39:10.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.096+0000 7fe509ffb700 1 -- 172.21.15.7:0/3895543728 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe510005bf0 con 0x7fe5140b04a0 2024-01-27T02:39:10.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.096+0000 7fe51c261700 1 -- 172.21.15.7:0/3895543728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51415e720 con 0x7fe5140b04a0 2024-01-27T02:39:10.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.096+0000 7fe509ffb700 1 -- 172.21.15.7:0/3895543728 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5100070f0 con 0x7fe5140b04a0 2024-01-27T02:39:10.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.096+0000 7fe51c261700 1 -- 172.21.15.7:0/3895543728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe51415ece0 con 0x7fe5140b04a0 2024-01-27T02:39:10.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.097+0000 7fe51c261700 1 -- 172.21.15.7:0/3895543728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe51404c710 con 0x7fe5140b04a0 2024-01-27T02:39:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.098+0000 7fe509ffb700 1 -- 172.21.15.7:0/3895543728 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe510007460 con 0x7fe5140b04a0 2024-01-27T02:39:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.098+0000 7fe509ffb700 1 --2- 172.21.15.7:0/3895543728 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe4f8066f30 0x7fe4f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.098+0000 7fe509ffb700 1 -- 172.21.15.7:0/3895543728 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fe510083e80 con 0x7fe5140b04a0 2024-01-27T02:39:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.098+0000 7fe5197fc700 1 --2- 172.21.15.7:0/3895543728 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe4f8066f30 0x7fe4f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:10.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.099+0000 7fe509ffb700 1 -- 172.21.15.7:0/3895543728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=139}) v3 -- 0x7fe4f806eb10 con 0x7fe5140b04a0 2024-01-27T02:39:10.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.099+0000 7fe5197fc700 1 --2- 172.21.15.7:0/3895543728 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe4f8066f30 0x7fe4f80693e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe5040093f0 tx=0x7fe504017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:10.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.103+0000 7fe509ffb700 1 -- 172.21.15.7:0/3895543728 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe510051130 con 0x7fe5140b04a0 2024-01-27T02:39:10.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.352+0000 7fe51c261700 1 -- 172.21.15.7:0/3895543728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5140b0920 con 0x7fe5140b04a0 2024-01-27T02:39:10.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.352+0000 7fe509ffb700 1 -- 172.21.15.7:0/3895543728 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fe5140b0920 con 0x7fe5140b04a0 2024-01-27T02:39:10.353 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:10.353 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3997828193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3997828193},{"type":"v1","addr":"172.21.15.188:6837","nonce":3997828193}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1132338302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1132338302},{"type":"v1","addr":"172.21.15.188:6836","nonce":1132338302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/740290645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":740290645},{"type":"v1","addr":"172.21.15.7:6833","nonce":740290645}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:38:31.328277+0000","modified":"2024-01-27T02:38:34.940811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7029":{"gid":7029,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1258751919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1258751919},{"type":"v1","addr":"172.21.15.7:6835","nonce":1258751919}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-27T02:39:10.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.354+0000 7fe51c261700 1 -- 172.21.15.7:0/3895543728 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe4f8066f30 msgr2=0x7fe4f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:10.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.354+0000 7fe51c261700 1 --2- 172.21.15.7:0/3895543728 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe4f8066f30 0x7fe4f80693e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe5040093f0 tx=0x7fe504017000 comp rx=0 tx=0).stop 2024-01-27T02:39:10.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.354+0000 7fe51c261700 1 -- 172.21.15.7:0/3895543728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5140b04a0 msgr2=0x7fe51415ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:10.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.354+0000 7fe51c261700 1 --2- 172.21.15.7:0/3895543728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5140b04a0 0x7fe51415ddd0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe51000c7c0 tx=0x7fe510007f70 comp rx=0 tx=0).stop 2024-01-27T02:39:10.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.354+0000 7fe51c261700 1 -- 172.21.15.7:0/3895543728 shutdown_connections 2024-01-27T02:39:10.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.354+0000 7fe51c261700 1 --2- 172.21.15.7:0/3895543728 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe4f8066f30 0x7fe4f80693e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.354+0000 7fe51c261700 1 --2- 172.21.15.7:0/3895543728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe51415b2a0 0x7fe51415fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.354+0000 7fe51c261700 1 --2- 172.21.15.7:0/3895543728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5140b04a0 0x7fe51415ddd0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.354+0000 7fe51c261700 1 --2- 172.21.15.7:0/3895543728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe51415a070 0x7fe514155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.354+0000 7fe51c261700 1 -- 172.21.15.7:0/3895543728 >> 172.21.15.7:0/3895543728 conn(0x7fe514098020 msgr2=0x7fe5140b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:10.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.355+0000 7fe51c261700 1 -- 172.21.15.7:0/3895543728 shutdown_connections 2024-01-27T02:39:10.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.355+0000 7fe51c261700 1 -- 172.21.15.7:0/3895543728 wait complete. 2024-01-27T02:39:10.356 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 68 2024-01-27T02:39:10.368 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:39:10.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.499+0000 7fbf78da6700 1 Processor -- start 2024-01-27T02:39:10.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.499+0000 7fbf78da6700 1 -- start start 2024-01-27T02:39:10.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.500+0000 7fbf78da6700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbf7415b740 0x7fbf740c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:10.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.501+0000 7fbf78da6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbf740c55d0 0x7fbf740ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:10.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.501+0000 7fbf78da6700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf740c4a30 con 0x7fbf740cc8e0 2024-01-27T02:39:10.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.501+0000 7fbf78da6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf740cd2c0 con 0x7fbf740c55d0 2024-01-27T02:39:10.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.501+0000 7fbf78da6700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf7415bb10 con 0x7fbf7415b740 2024-01-27T02:39:10.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.501+0000 7fbf72d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbf740c55d0 0x7fbf740ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:10.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.501+0000 7fbf72d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbf740c55d0 0x7fbf740ccd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49940/0 (socket says 172.21.15.7:49940) 2024-01-27T02:39:10.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.501+0000 7fbf72d9d700 1 -- 172.21.15.7:0/439720874 learned_addr learned my addr 172.21.15.7:0/439720874 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:10.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.501+0000 7fbf7259c700 1 --2- 172.21.15.7:0/439720874 >> v2:172.21.15.188:3300/0 conn(0x7fbf7415b740 0x7fbf740c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:10.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.501+0000 7fbf72d9d700 1 -- 172.21.15.7:0/439720874 >> v2:172.21.15.188:3300/0 conn(0x7fbf7415b740 msgr2=0x7fbf740c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:10.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.501+0000 7fbf72d9d700 1 --2- 172.21.15.7:0/439720874 >> v2:172.21.15.188:3300/0 conn(0x7fbf7415b740 0x7fbf740c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.501+0000 7fbf72d9d700 1 -- 172.21.15.7:0/439720874 >> v1:172.21.15.188:6789/0 conn(0x7fbf740cc8e0 legacy=0x7fbf740c4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:10.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.501+0000 7fbf72d9d700 1 -- 172.21.15.7:0/439720874 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf740c3e20 con 0x7fbf740c55d0 2024-01-27T02:39:10.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.501+0000 7fbf7159a700 1 -- 172.21.15.7:0/439720874 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3988599520 0 0) 0x7fbf740c4a30 con 0x7fbf740cc8e0 2024-01-27T02:39:10.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.501+0000 7fbf72d9d700 1 --2- 172.21.15.7:0/439720874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf740c55d0 0x7fbf740ccd80 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c009f20 tx=0x7fbf6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86baaca77851f4a9 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:10.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.501+0000 7fbf7159a700 1 -- 172.21.15.7:0/439720874 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c00e070 con 0x7fbf740c55d0 2024-01-27T02:39:10.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.502+0000 7fbf7159a700 1 -- 172.21.15.7:0/439720874 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6c00b990 con 0x7fbf740c55d0 2024-01-27T02:39:10.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.502+0000 7fbf7159a700 1 -- 172.21.15.7:0/439720874 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c0092e0 con 0x7fbf740c55d0 2024-01-27T02:39:10.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.502+0000 7fbf78da6700 1 -- 172.21.15.7:0/439720874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf740c55d0 msgr2=0x7fbf740ccd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:10.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.502+0000 7fbf78da6700 1 --2- 172.21.15.7:0/439720874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf740c55d0 0x7fbf740ccd80 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c009f20 tx=0x7fbf6c009840 comp rx=0 tx=0).stop 2024-01-27T02:39:10.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.502+0000 7fbf78da6700 1 -- 172.21.15.7:0/439720874 shutdown_connections 2024-01-27T02:39:10.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.502+0000 7fbf78da6700 1 --2- 172.21.15.7:0/439720874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf740c55d0 0x7fbf740ccd80 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.502+0000 7fbf78da6700 1 --2- 172.21.15.7:0/439720874 >> v2:172.21.15.188:3300/0 conn(0x7fbf7415b740 0x7fbf740c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.502+0000 7fbf78da6700 1 -- 172.21.15.7:0/439720874 >> 172.21.15.7:0/439720874 conn(0x7fbf74063d60 msgr2=0x7fbf740ced10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:10.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.502+0000 7fbf78da6700 1 -- 172.21.15.7:0/439720874 shutdown_connections 2024-01-27T02:39:10.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.502+0000 7fbf78da6700 1 -- 172.21.15.7:0/439720874 wait complete. 2024-01-27T02:39:10.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.503+0000 7fbf78da6700 1 Processor -- start 2024-01-27T02:39:10.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.503+0000 7fbf78da6700 1 -- start start 2024-01-27T02:39:10.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.504+0000 7fbf78da6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf740c55d0 0x7fbf74156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:10.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.504+0000 7fbf78da6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf740cc8e0 0x7fbf740683c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:10.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.504+0000 7fbf78da6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf7415b740 0x7fbf74068900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:10.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.504+0000 7fbf78da6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf740b24c0 con 0x7fbf740cc8e0 2024-01-27T02:39:10.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.504+0000 7fbf78da6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7416c100 con 0x7fbf7415b740 2024-01-27T02:39:10.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.504+0000 7fbf78da6700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7416baf0 con 0x7fbf740c55d0 2024-01-27T02:39:10.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.504+0000 7fbf71d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf740cc8e0 0x7fbf740683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:10.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.504+0000 7fbf71d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf740cc8e0 0x7fbf740683c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49946/0 (socket says 172.21.15.7:49946) 2024-01-27T02:39:10.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.504+0000 7fbf71d9b700 1 -- 172.21.15.7:0/2450321110 learned_addr learned my addr 172.21.15.7:0/2450321110 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:10.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.504+0000 7fbf7259c700 1 --2- 172.21.15.7:0/2450321110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf740c55d0 0x7fbf74156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:10.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.505+0000 7fbf72d9d700 1 --2- 172.21.15.7:0/2450321110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf7415b740 0x7fbf74068900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:10.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.505+0000 7fbf71d9b700 1 -- 172.21.15.7:0/2450321110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf740c55d0 msgr2=0x7fbf74156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:10.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.505+0000 7fbf71d9b700 1 --2- 172.21.15.7:0/2450321110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf740c55d0 0x7fbf74156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.505+0000 7fbf71d9b700 1 -- 172.21.15.7:0/2450321110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf7415b740 msgr2=0x7fbf74068900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:10.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.505+0000 7fbf71d9b700 1 --2- 172.21.15.7:0/2450321110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf7415b740 0x7fbf74068900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.505+0000 7fbf71d9b700 1 -- 172.21.15.7:0/2450321110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6c009ae0 con 0x7fbf740cc8e0 2024-01-27T02:39:10.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.505+0000 7fbf71d9b700 1 --2- 172.21.15.7:0/2450321110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf740cc8e0 0x7fbf740683c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800c810 tx=0x7fbf68007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:10.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.505+0000 7fbf627fc700 1 -- 172.21.15.7:0/2450321110 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68011590 con 0x7fbf740cc8e0 2024-01-27T02:39:10.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.505+0000 7fbf627fc700 1 -- 172.21.15.7:0/2450321110 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf68011710 con 0x7fbf740cc8e0 2024-01-27T02:39:10.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.505+0000 7fbf78da6700 1 -- 172.21.15.7:0/2450321110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf74156750 con 0x7fbf740cc8e0 2024-01-27T02:39:10.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.505+0000 7fbf627fc700 1 -- 172.21.15.7:0/2450321110 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68011ad0 con 0x7fbf740cc8e0 2024-01-27T02:39:10.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.505+0000 7fbf78da6700 1 -- 172.21.15.7:0/2450321110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7416bd90 con 0x7fbf740cc8e0 2024-01-27T02:39:10.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.506+0000 7fbf78da6700 1 -- 172.21.15.7:0/2450321110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf7416c290 con 0x7fbf740cc8e0 2024-01-27T02:39:10.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.507+0000 7fbf627fc700 1 -- 172.21.15.7:0/2450321110 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbf6801c460 con 0x7fbf740cc8e0 2024-01-27T02:39:10.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.507+0000 7fbf627fc700 1 --2- 172.21.15.7:0/2450321110 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbf54066f30 0x7fbf540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:10.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.507+0000 7fbf627fc700 1 -- 172.21.15.7:0/2450321110 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fbf68084240 con 0x7fbf740cc8e0 2024-01-27T02:39:10.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.507+0000 7fbf627fc700 1 -- 172.21.15.7:0/2450321110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=139}) v3 -- 0x7fbf5406eb10 con 0x7fbf740cc8e0 2024-01-27T02:39:10.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.507+0000 7fbf7259c700 1 --2- 172.21.15.7:0/2450321110 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbf54066f30 0x7fbf540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:10.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.508+0000 7fbf7259c700 1 --2- 172.21.15.7:0/2450321110 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbf54066f30 0x7fbf540693e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009d10 tx=0x7fbf640091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.511+0000 7fbf627fc700 1 -- 172.21.15.7:0/2450321110 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf68051470 con 0x7fbf740cc8e0 2024-01-27T02:39:10.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.714+0000 7fbf78da6700 1 -- 172.21.15.7:0/2450321110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf7404c710 con 0x7fbf740cc8e0 2024-01-27T02:39:10.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.715+0000 7fbf627fc700 1 -- 172.21.15.7:0/2450321110 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+15440 (secure 0 0 0) 0x7fbf68054a90 con 0x7fbf740cc8e0 2024-01-27T02:39:10.716 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:10.716 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":138,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:39:09.874085+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:38:29.824892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-27T02:38:30.301072+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.e","mappings":[{"from":0,"to":5}]},{"pgid":"17.24","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:39:10.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.717+0000 7fbf78da6700 1 -- 172.21.15.7:0/2450321110 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbf54066f30 msgr2=0x7fbf540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:10.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.717+0000 7fbf78da6700 1 --2- 172.21.15.7:0/2450321110 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbf54066f30 0x7fbf540693e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009d10 tx=0x7fbf640091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.717+0000 7fbf78da6700 1 -- 172.21.15.7:0/2450321110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf740cc8e0 msgr2=0x7fbf740683c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:10.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.717+0000 7fbf78da6700 1 --2- 172.21.15.7:0/2450321110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf740cc8e0 0x7fbf740683c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800c810 tx=0x7fbf68007df0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.717+0000 7fbf78da6700 1 -- 172.21.15.7:0/2450321110 shutdown_connections 2024-01-27T02:39:10.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.717+0000 7fbf78da6700 1 --2- 172.21.15.7:0/2450321110 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbf54066f30 0x7fbf540693e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.717+0000 7fbf78da6700 1 --2- 172.21.15.7:0/2450321110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf7415b740 0x7fbf74068900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.717+0000 7fbf78da6700 1 --2- 172.21.15.7:0/2450321110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf740c55d0 0x7fbf74156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.717+0000 7fbf78da6700 1 --2- 172.21.15.7:0/2450321110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf740cc8e0 0x7fbf740683c0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.717+0000 7fbf78da6700 1 -- 172.21.15.7:0/2450321110 >> 172.21.15.7:0/2450321110 conn(0x7fbf74063d60 msgr2=0x7fbf74158010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:10.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.718+0000 7fbf78da6700 1 -- 172.21.15.7:0/2450321110 shutdown_connections 2024-01-27T02:39:10.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.718+0000 7fbf78da6700 1 -- 172.21.15.7:0/2450321110 wait complete. 2024-01-27T02:39:10.732 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-27T02:39:10.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.865+0000 7f8f38f28700 1 Processor -- start 2024-01-27T02:39:10.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.865+0000 7f8f38f28700 1 -- start start 2024-01-27T02:39:10.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.866+0000 7f8f38f28700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f34157ca0 0x7f8f340c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:10.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.866+0000 7f8f38f28700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8f340c3ae0 0x7f8f340cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:10.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.866+0000 7f8f38f28700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f340ca020 con 0x7f8f34158070 2024-01-27T02:39:10.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.866+0000 7f8f38f28700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f340c4f40 con 0x7f8f340c3ae0 2024-01-27T02:39:10.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.866+0000 7f8f38f28700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f340c5210 con 0x7f8f34157ca0 2024-01-27T02:39:10.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.866+0000 7f8f32d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8f34158070 0x7f8f3415bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59886/0 (socket says 172.21.15.7:59886) 2024-01-27T02:39:10.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.866+0000 7f8f32d9d700 1 -- 172.21.15.7:0/1699356068 learned_addr learned my addr 172.21.15.7:0/1699356068 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:10.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.866+0000 7f8f31d9b700 1 --2- 172.21.15.7:0/1699356068 >> v2:172.21.15.188:3300/0 conn(0x7f8f340c3ae0 0x7f8f340cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:10.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.866+0000 7f8f3259c700 1 --2- 172.21.15.7:0/1699356068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f34157ca0 0x7f8f340c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:10.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.866+0000 7f8f3159a700 1 -- 172.21.15.7:0/1699356068 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3689792839 0 0) 0x7f8f340ca020 con 0x7f8f34158070 2024-01-27T02:39:10.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.866+0000 7f8f3159a700 1 -- 172.21.15.7:0/1699356068 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f1c003570 con 0x7f8f34158070 2024-01-27T02:39:10.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.866+0000 7f8f3159a700 1 -- 172.21.15.7:0/1699356068 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 978027678 0 0) 0x7f8f1c003570 con 0x7f8f34158070 2024-01-27T02:39:10.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.866+0000 7f8f3159a700 1 -- 172.21.15.7:0/1699356068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f34157ca0 msgr2=0x7f8f340c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:10.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.866+0000 7f8f3159a700 1 --2- 172.21.15.7:0/1699356068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f34157ca0 0x7f8f340c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.866+0000 7f8f3159a700 1 -- 172.21.15.7:0/1699356068 >> v2:172.21.15.188:3300/0 conn(0x7f8f340c3ae0 msgr2=0x7f8f340cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:10.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.866+0000 7f8f3159a700 1 --2- 172.21.15.7:0/1699356068 >> v2:172.21.15.188:3300/0 conn(0x7f8f340c3ae0 0x7f8f340cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.866+0000 7f8f3159a700 1 -- 172.21.15.7:0/1699356068 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f340c54f0 con 0x7f8f34158070 2024-01-27T02:39:10.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.866+0000 7f8f3259c700 1 --2- 172.21.15.7:0/1699356068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f34157ca0 0x7f8f340c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:39:10.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.866+0000 7f8f3159a700 1 -- 172.21.15.7:0/1699356068 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f8f2c0017f0 con 0x7f8f34158070 2024-01-27T02:39:10.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.866+0000 7f8f31d9b700 1 --2- 172.21.15.7:0/1699356068 >> v2:172.21.15.188:3300/0 conn(0x7f8f340c3ae0 0x7f8f340cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:39:10.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.867+0000 7f8f3159a700 1 -- 172.21.15.7:0/1699356068 >> v1:172.21.15.7:6789/0 conn(0x7f8f34158070 legacy=0x7f8f3415bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:10.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.867+0000 7f8f3159a700 1 --2- 172.21.15.7:0/1699356068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f1c003a60 0x7f8f1c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:10.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.867+0000 7f8f3159a700 1 --2- 172.21.15.7:0/1699356068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f34157ca0 0x7f8f1c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:10.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.867+0000 7f8f3159a700 1 --2- 172.21.15.7:0/1699356068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f1c004d50 0x7f8f1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.867+0000 7f8f3159a700 1 -- 172.21.15.7:0/1699356068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f340c4f40 con 0x7f8f1c004d50 2024-01-27T02:39:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.867+0000 7f8f3159a700 1 -- 172.21.15.7:0/1699356068 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f340c5210 con 0x7f8f1c003a60 2024-01-27T02:39:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.867+0000 7f8f3159a700 1 -- 172.21.15.7:0/1699356068 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c00ba60 con 0x7f8f34157ca0 2024-01-27T02:39:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.867+0000 7f8f32d9d700 1 --2- 172.21.15.7:0/1699356068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f1c004d50 0x7f8f1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.867+0000 7f8f3259c700 1 --2- 172.21.15.7:0/1699356068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f1c003a60 0x7f8f1c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.867+0000 7f8f31d9b700 1 --2- 172.21.15.7:0/1699356068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f34157ca0 0x7f8f1c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.867+0000 7f8f32d9d700 1 -- 172.21.15.7:0/1699356068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f34157ca0 msgr2=0x7f8f1c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.867+0000 7f8f32d9d700 1 --2- 172.21.15.7:0/1699356068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f34157ca0 0x7f8f1c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.867+0000 7f8f32d9d700 1 -- 172.21.15.7:0/1699356068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f1c003a60 msgr2=0x7f8f1c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.867+0000 7f8f32d9d700 1 --2- 172.21.15.7:0/1699356068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f1c003a60 0x7f8f1c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.867+0000 7f8f32d9d700 1 -- 172.21.15.7:0/1699356068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f24009420 con 0x7f8f1c004d50 2024-01-27T02:39:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.867+0000 7f8f32d9d700 1 --2- 172.21.15.7:0/1699356068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f1c004d50 0x7f8f1c0051c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00a970 tx=0x7f8f2c002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd933825624cdf1c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.867+0000 7f8f3159a700 1 -- 172.21.15.7:0/1699356068 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c0070d0 con 0x7f8f1c004d50 2024-01-27T02:39:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.867+0000 7f8f3159a700 1 -- 172.21.15.7:0/1699356068 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2c007270 con 0x7f8f1c004d50 2024-01-27T02:39:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.867+0000 7f8f3159a700 1 -- 172.21.15.7:0/1699356068 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c007630 con 0x7f8f1c004d50 2024-01-27T02:39:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.868+0000 7f8f38f28700 1 -- 172.21.15.7:0/1699356068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f1c004d50 msgr2=0x7f8f1c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.868+0000 7f8f38f28700 1 --2- 172.21.15.7:0/1699356068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f1c004d50 0x7f8f1c0051c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00a970 tx=0x7f8f2c002210 comp rx=0 tx=0).stop 2024-01-27T02:39:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.868+0000 7f8f32d9d700 1 -- 172.21.15.7:0/1699356068 reap_dead start 2024-01-27T02:39:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.868+0000 7f8f38f28700 1 -- 172.21.15.7:0/1699356068 shutdown_connections 2024-01-27T02:39:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.868+0000 7f8f38f28700 1 -- 172.21.15.7:0/1699356068 >> 172.21.15.7:0/1699356068 conn(0x7f8f34098020 msgr2=0x7f8f340711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.868+0000 7f8f38f28700 1 -- 172.21.15.7:0/1699356068 shutdown_connections 2024-01-27T02:39:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.868+0000 7f8f38f28700 1 -- 172.21.15.7:0/1699356068 wait complete. 2024-01-27T02:39:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.869+0000 7f8f38f28700 1 Processor -- start 2024-01-27T02:39:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.869+0000 7f8f38f28700 1 -- start start 2024-01-27T02:39:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.870+0000 7f8f38f28700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f1c004d50 0x7f8f34153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.870+0000 7f8f38f28700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f3414d8d0 0x7f8f3415cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.870+0000 7f8f38f28700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f340cb380 0x7f8f340cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.870+0000 7f8f38f28700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f340cbd60 con 0x7f8f1c004d50 2024-01-27T02:39:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.870+0000 7f8f38f28700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f340c7c60 con 0x7f8f340cb380 2024-01-27T02:39:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.870+0000 7f8f38f28700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3415d540 con 0x7f8f3414d8d0 2024-01-27T02:39:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.870+0000 7f8f3259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f1c004d50 0x7f8f34153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.870+0000 7f8f3259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f1c004d50 0x7f8f34153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49974/0 (socket says 172.21.15.7:49974) 2024-01-27T02:39:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.870+0000 7f8f3259c700 1 -- 172.21.15.7:0/278452772 learned_addr learned my addr 172.21.15.7:0/278452772 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.870+0000 7f8f31d9b700 1 --2- 172.21.15.7:0/278452772 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f3414d8d0 0x7f8f3415cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:10.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.870+0000 7f8f32d9d700 1 --2- 172.21.15.7:0/278452772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f340cb380 0x7f8f340cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:10.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.871+0000 7f8f3259c700 1 -- 172.21.15.7:0/278452772 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f3414d8d0 msgr2=0x7f8f3415cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:10.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.871+0000 7f8f3259c700 1 --2- 172.21.15.7:0/278452772 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f3414d8d0 0x7f8f3415cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.871+0000 7f8f3259c700 1 -- 172.21.15.7:0/278452772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f340cb380 msgr2=0x7f8f340cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:10.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.871+0000 7f8f3259c700 1 --2- 172.21.15.7:0/278452772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f340cb380 0x7f8f340cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:10.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.871+0000 7f8f3259c700 1 -- 172.21.15.7:0/278452772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3415d820 con 0x7f8f1c004d50 2024-01-27T02:39:10.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.871+0000 7f8f3259c700 1 --2- 172.21.15.7:0/278452772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f1c004d50 0x7f8f34153db0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f8f24011d70 tx=0x7f8f240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:10.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.871+0000 7f8f227fc700 1 -- 172.21.15.7:0/278452772 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f24011e70 con 0x7f8f1c004d50 2024-01-27T02:39:10.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.871+0000 7f8f227fc700 1 -- 172.21.15.7:0/278452772 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2400fc20 con 0x7f8f1c004d50 2024-01-27T02:39:10.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.871+0000 7f8f38f28700 1 -- 172.21.15.7:0/278452772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f340c78f0 con 0x7f8f1c004d50 2024-01-27T02:39:10.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.871+0000 7f8f227fc700 1 -- 172.21.15.7:0/278452772 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2401d480 con 0x7f8f1c004d50 2024-01-27T02:39:10.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.871+0000 7f8f38f28700 1 -- 172.21.15.7:0/278452772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3414dd40 con 0x7f8f1c004d50 2024-01-27T02:39:10.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.872+0000 7f8f38f28700 1 -- 172.21.15.7:0/278452772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f340c7750 con 0x7f8f1c004d50 2024-01-27T02:39:10.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.873+0000 7f8f227fc700 1 -- 172.21.15.7:0/278452772 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8f2401d830 con 0x7f8f1c004d50 2024-01-27T02:39:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.873+0000 7f8f227fc700 1 --2- 172.21.15.7:0/278452772 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f14066f30 0x7f8f140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.873+0000 7f8f227fc700 1 -- 172.21.15.7:0/278452772 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f8f24018070 con 0x7f8f1c004d50 2024-01-27T02:39:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.873+0000 7f8f31d9b700 1 --2- 172.21.15.7:0/278452772 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f14066f30 0x7f8f140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.873+0000 7f8f227fc700 1 -- 172.21.15.7:0/278452772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=139}) v3 -- 0x7f8f1406eb10 con 0x7f8f1c004d50 2024-01-27T02:39:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.874+0000 7f8f31d9b700 1 --2- 172.21.15.7:0/278452772 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f14066f30 0x7f8f140693e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8f3416b1a0 tx=0x7f8f280091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:10.878+0000 7f8f227fc700 1 -- 172.21.15.7:0/278452772 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f340c7750 con 0x7f8f1c004d50 2024-01-27T02:39:11.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.129+0000 7f8f38f28700 1 -- 172.21.15.7:0/278452772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8f340c7f40 con 0x7f8f1c004d50 2024-01-27T02:39:11.130 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-27T02:39:11.129+0000 7fb5c503a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-27T02:39:11.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.131+0000 7f8f227fc700 1 -- 172.21.15.7:0/278452772 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(139..139 src has 1..139) v4 ==== 366+0+0 (secure 0 0 0) 0x7f8f24056240 con 0x7f8f1c004d50 2024-01-27T02:39:11.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.131+0000 7f8f227fc700 1 -- 172.21.15.7:0/278452772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=140}) v3 -- 0x7f8f1406f400 con 0x7f8f1c004d50 2024-01-27T02:39:11.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.133+0000 7f8f227fc700 1 -- 172.21.15.7:0/278452772 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8f24052580 con 0x7f8f1c004d50 2024-01-27T02:39:11.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.137+0000 7f8f38f28700 1 -- 172.21.15.7:0/278452772 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f14066f30 msgr2=0x7f8f140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:11.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.137+0000 7f8f38f28700 1 --2- 172.21.15.7:0/278452772 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f14066f30 0x7f8f140693e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8f3416b1a0 tx=0x7f8f280091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:11.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.137+0000 7f8f38f28700 1 -- 172.21.15.7:0/278452772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f1c004d50 msgr2=0x7f8f34153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:11.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.137+0000 7f8f38f28700 1 --2- 172.21.15.7:0/278452772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f1c004d50 0x7f8f34153db0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f8f24011d70 tx=0x7f8f240091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:11.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.137+0000 7f8f38f28700 1 -- 172.21.15.7:0/278452772 shutdown_connections 2024-01-27T02:39:11.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.137+0000 7f8f38f28700 1 --2- 172.21.15.7:0/278452772 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f14066f30 0x7f8f140693e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:11.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.137+0000 7f8f38f28700 1 --2- 172.21.15.7:0/278452772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f340cb380 0x7f8f340cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:11.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.137+0000 7f8f38f28700 1 --2- 172.21.15.7:0/278452772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f1c004d50 0x7f8f34153db0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:11.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.137+0000 7f8f38f28700 1 --2- 172.21.15.7:0/278452772 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f3414d8d0 0x7f8f3415cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:11.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.137+0000 7f8f38f28700 1 -- 172.21.15.7:0/278452772 >> 172.21.15.7:0/278452772 conn(0x7f8f34098020 msgr2=0x7f8f3406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:11.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.137+0000 7f8f38f28700 1 -- 172.21.15.7:0/278452772 shutdown_connections 2024-01-27T02:39:11.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.137+0000 7f8f38f28700 1 -- 172.21.15.7:0/278452772 wait complete. 2024-01-27T02:39:11.139 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-27T02:39:11.139 INFO:tasks.ceph.mds.a.smithi007.stderr: -7708> 2024-01-27T02:38:36.873+0000 7fa3b6e5c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-27T02:39:11.152 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-27T02:39:11.171 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-27T02:39:11.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.286+0000 7fd244480700 1 Processor -- start 2024-01-27T02:39:11.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.286+0000 7fd244480700 1 -- start start 2024-01-27T02:39:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.287+0000 7fd244480700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd23c1436f0 0x7fd23c063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.287+0000 7fd244480700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd23c0aa8b0 0x7fd23c063ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.287+0000 7fd244480700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd23c0c8230 0x7fd23c0c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.287+0000 7fd244480700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd23c0cb7c0 con 0x7fd23c0aa8b0 2024-01-27T02:39:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.287+0000 7fd244480700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd23c0cc340 con 0x7fd23c1436f0 2024-01-27T02:39:11.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.287+0000 7fd244480700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c0cba10 con 0x7fd23c0c8230 2024-01-27T02:39:11.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.287+0000 7fd241a1b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd23c0aa8b0 0x7fd23c063ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:11.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.287+0000 7fd241a1b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd23c0aa8b0 0x7fd23c063ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50010/0 (socket says 172.21.15.7:50010) 2024-01-27T02:39:11.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.287+0000 7fd241a1b700 1 -- 172.21.15.7:0/2165096025 learned_addr learned my addr 172.21.15.7:0/2165096025 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:11.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.287+0000 7fd24221c700 1 --2- 172.21.15.7:0/2165096025 >> v2:172.21.15.188:3300/0 conn(0x7fd23c1436f0 0x7fd23c063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:11.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.287+0000 7fd242a1d700 1 --2- 172.21.15.7:0/2165096025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd23c0c8230 0x7fd23c0c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:11.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.287+0000 7fd241a1b700 1 -- 172.21.15.7:0/2165096025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd23c0c8230 msgr2=0x7fd23c0c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:11.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.287+0000 7fd241a1b700 1 --2- 172.21.15.7:0/2165096025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd23c0c8230 0x7fd23c0c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:11.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.287+0000 7fd241a1b700 1 -- 172.21.15.7:0/2165096025 >> v2:172.21.15.188:3300/0 conn(0x7fd23c1436f0 msgr2=0x7fd23c063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:11.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.287+0000 7fd241a1b700 1 --2- 172.21.15.7:0/2165096025 >> v2:172.21.15.188:3300/0 conn(0x7fd23c1436f0 0x7fd23c063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:11.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.288+0000 7fd241a1b700 1 -- 172.21.15.7:0/2165096025 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23c0cbcf0 con 0x7fd23c0aa8b0 2024-01-27T02:39:11.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.288+0000 7fd24221c700 1 --2- 172.21.15.7:0/2165096025 >> v2:172.21.15.188:3300/0 conn(0x7fd23c1436f0 0x7fd23c063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:11.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.288+0000 7fd241a1b700 1 --2- 172.21.15.7:0/2165096025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23c0aa8b0 0x7fd23c063ac0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fd23c064600 tx=0x7fd22c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41369844c9b45aca server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:11.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.288+0000 7fd24121a700 1 -- 172.21.15.7:0/2165096025 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c00b860 con 0x7fd23c0aa8b0 2024-01-27T02:39:11.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.288+0000 7fd24121a700 1 -- 172.21.15.7:0/2165096025 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22c00b9c0 con 0x7fd23c0aa8b0 2024-01-27T02:39:11.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.288+0000 7fd24121a700 1 -- 172.21.15.7:0/2165096025 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c00bd60 con 0x7fd23c0aa8b0 2024-01-27T02:39:11.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.288+0000 7fd244480700 1 -- 172.21.15.7:0/2165096025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23c0aa8b0 msgr2=0x7fd23c063ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:11.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.288+0000 7fd244480700 1 --2- 172.21.15.7:0/2165096025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23c0aa8b0 0x7fd23c063ac0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fd23c064600 tx=0x7fd22c009250 comp rx=0 tx=0).stop 2024-01-27T02:39:11.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.289+0000 7fd244480700 1 -- 172.21.15.7:0/2165096025 shutdown_connections 2024-01-27T02:39:11.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.289+0000 7fd244480700 1 --2- 172.21.15.7:0/2165096025 >> v2:172.21.15.188:3300/0 conn(0x7fd23c1436f0 0x7fd23c063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:11.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.289+0000 7fd244480700 1 --2- 172.21.15.7:0/2165096025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd23c0c8230 0x7fd23c0c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:11.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.289+0000 7fd244480700 1 --2- 172.21.15.7:0/2165096025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23c0aa8b0 0x7fd23c063ac0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:11.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.289+0000 7fd244480700 1 -- 172.21.15.7:0/2165096025 >> 172.21.15.7:0/2165096025 conn(0x7fd23c0cdbf0 msgr2=0x7fd23c0cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:11.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.289+0000 7fd244480700 1 -- 172.21.15.7:0/2165096025 shutdown_connections 2024-01-27T02:39:11.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.289+0000 7fd244480700 1 -- 172.21.15.7:0/2165096025 wait complete. 2024-01-27T02:39:11.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.290+0000 7fd244480700 1 Processor -- start 2024-01-27T02:39:11.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.290+0000 7fd244480700 1 -- start start 2024-01-27T02:39:11.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd244480700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd23c0aa8b0 0x7fd23c0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:11.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd244480700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23c0c8230 0x7fd23c142540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:11.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd244480700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd23c1436f0 0x7fd23c06bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:11.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd244480700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c16f9d0 con 0x7fd23c0c8230 2024-01-27T02:39:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd244480700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c0b1130 con 0x7fd23c1436f0 2024-01-27T02:39:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd244480700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c16f6f0 con 0x7fd23c0aa8b0 2024-01-27T02:39:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd241a1b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23c0c8230 0x7fd23c142540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd241a1b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23c0c8230 0x7fd23c142540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50026/0 (socket says 172.21.15.7:50026) 2024-01-27T02:39:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd241a1b700 1 -- 172.21.15.7:0/4289077813 learned_addr learned my addr 172.21.15.7:0/4289077813 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd24221c700 1 --2- 172.21.15.7:0/4289077813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd23c0aa8b0 0x7fd23c0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd242a1d700 1 --2- 172.21.15.7:0/4289077813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd23c1436f0 0x7fd23c06bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd241a1b700 1 -- 172.21.15.7:0/4289077813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd23c0aa8b0 msgr2=0x7fd23c0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd241a1b700 1 --2- 172.21.15.7:0/4289077813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd23c0aa8b0 0x7fd23c0b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd241a1b700 1 -- 172.21.15.7:0/4289077813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd23c1436f0 msgr2=0x7fd23c06bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd241a1b700 1 --2- 172.21.15.7:0/4289077813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd23c1436f0 0x7fd23c06bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd241a1b700 1 -- 172.21.15.7:0/4289077813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22c0094f0 con 0x7fd23c0c8230 2024-01-27T02:39:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd24221c700 1 --2- 172.21.15.7:0/4289077813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd23c0aa8b0 0x7fd23c0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:39:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd242a1d700 1 --2- 172.21.15.7:0/4289077813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd23c1436f0 0x7fd23c06bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:39:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd241a1b700 1 --2- 172.21.15.7:0/4289077813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23c0c8230 0x7fd23c142540 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009290 tx=0x7fd22c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd2327fc700 1 -- 172.21.15.7:0/4289077813 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c01a450 con 0x7fd23c0c8230 2024-01-27T02:39:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd2327fc700 1 -- 172.21.15.7:0/4289077813 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22c01a5b0 con 0x7fd23c0c8230 2024-01-27T02:39:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd244480700 1 -- 172.21.15.7:0/4289077813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd23c0b0ae0 con 0x7fd23c0c8230 2024-01-27T02:39:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd2327fc700 1 -- 172.21.15.7:0/4289077813 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c01a970 con 0x7fd23c0c8230 2024-01-27T02:39:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.291+0000 7fd244480700 1 -- 172.21.15.7:0/4289077813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd23c142a80 con 0x7fd23c0c8230 2024-01-27T02:39:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.292+0000 7fd244480700 1 -- 172.21.15.7:0/4289077813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd23c0b12e0 con 0x7fd23c0c8230 2024-01-27T02:39:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.293+0000 7fd2327fc700 1 -- 172.21.15.7:0/4289077813 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd22c01ad20 con 0x7fd23c0c8230 2024-01-27T02:39:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.293+0000 7fd2327fc700 1 --2- 172.21.15.7:0/4289077813 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd220066f60 0x7fd220069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.293+0000 7fd2327fc700 1 -- 172.21.15.7:0/4289077813 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5849+0+0 (secure 0 0 0) 0x7fd22c085c60 con 0x7fd23c0c8230 2024-01-27T02:39:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.293+0000 7fd24221c700 1 --2- 172.21.15.7:0/4289077813 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd220066f60 0x7fd220069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.293+0000 7fd2327fc700 1 -- 172.21.15.7:0/4289077813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=140}) v3 -- 0x7fd22006ebe0 con 0x7fd23c0c8230 2024-01-27T02:39:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.294+0000 7fd24221c700 1 --2- 172.21.15.7:0/4289077813 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd220066f60 0x7fd220069410 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd238006f00 tx=0x7fd238008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:11.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.298+0000 7fd2327fc700 1 -- 172.21.15.7:0/4289077813 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd22c02a060 con 0x7fd23c0c8230 2024-01-27T02:39:11.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:11.551+0000 7fd244480700 1 -- 172.21.15.7:0/4289077813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd23c072060 con 0x7fd23c0c8230 2024-01-27T02:39:12.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.139+0000 7fd2327fc700 1 -- 172.21.15.7:0/4289077813 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v70) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd22c020020 con 0x7fd23c0c8230 2024-01-27T02:39:12.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.142+0000 7fd244480700 1 -- 172.21.15.7:0/4289077813 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd220066f60 msgr2=0x7fd220069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:12.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.142+0000 7fd244480700 1 --2- 172.21.15.7:0/4289077813 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd220066f60 0x7fd220069410 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd238006f00 tx=0x7fd238008040 comp rx=0 tx=0).stop 2024-01-27T02:39:12.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.142+0000 7fd244480700 1 -- 172.21.15.7:0/4289077813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23c0c8230 msgr2=0x7fd23c142540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:12.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.142+0000 7fd244480700 1 --2- 172.21.15.7:0/4289077813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23c0c8230 0x7fd23c142540 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009290 tx=0x7fd22c017000 comp rx=0 tx=0).stop 2024-01-27T02:39:12.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.142+0000 7fd244480700 1 -- 172.21.15.7:0/4289077813 shutdown_connections 2024-01-27T02:39:12.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.142+0000 7fd244480700 1 --2- 172.21.15.7:0/4289077813 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd220066f60 0x7fd220069410 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:12.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.142+0000 7fd244480700 1 --2- 172.21.15.7:0/4289077813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd23c1436f0 0x7fd23c06bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:12.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.142+0000 7fd244480700 1 --2- 172.21.15.7:0/4289077813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd23c0aa8b0 0x7fd23c0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:12.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.142+0000 7fd244480700 1 --2- 172.21.15.7:0/4289077813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23c0c8230 0x7fd23c142540 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:12.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.142+0000 7fd244480700 1 -- 172.21.15.7:0/4289077813 >> 172.21.15.7:0/4289077813 conn(0x7fd23c0cdbf0 msgr2=0x7fd23c0c3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:12.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.142+0000 7fd244480700 1 -- 172.21.15.7:0/4289077813 shutdown_connections 2024-01-27T02:39:12.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.142+0000 7fd244480700 1 -- 172.21.15.7:0/4289077813 wait complete. 2024-01-27T02:39:12.158 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-27T02:39:12.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.341+0000 7f96a399b700 1 Processor -- start 2024-01-27T02:39:12.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.341+0000 7f96a399b700 1 -- start start 2024-01-27T02:39:12.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.342+0000 7f96a399b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f969c0ac450 0x7f969c06c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:12.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.342+0000 7f96a399b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f969c063580 0x7f969c063a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:12.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.342+0000 7f96a399b700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969c0c89c0 con 0x7f969c0ac080 2024-01-27T02:39:12.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.342+0000 7f96a399b700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f969c15ee00 con 0x7f969c063580 2024-01-27T02:39:12.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.342+0000 7f96a399b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c063f40 con 0x7f969c0ac450 2024-01-27T02:39:12.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.342+0000 7f96a1f38700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f969c063580 0x7f969c063a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:12.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.343+0000 7f96a0f36700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f969c0ac450 0x7f969c06c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:12.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.342+0000 7f96a1737700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f969c0ac080 0x7f969c0c81e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:36340/0 (socket says 172.21.15.7:36340) 2024-01-27T02:39:12.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.343+0000 7f96a1737700 1 -- 172.21.15.7:0/3220505421 learned_addr learned my addr 172.21.15.7:0/3220505421 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:12.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.343+0000 7f96a1f38700 1 --2- 172.21.15.7:0/3220505421 >> v2:172.21.15.188:3300/0 conn(0x7f969c063580 0x7f969c063a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:55742/0 (socket says 172.21.15.7:55742) 2024-01-27T02:39:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.343+0000 7f96a1f38700 1 -- 172.21.15.7:0/3220505421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f969c0ac450 msgr2=0x7f969c06c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.343+0000 7f96a1f38700 1 --2- 172.21.15.7:0/3220505421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f969c0ac450 0x7f969c06c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.343+0000 7f96a1f38700 1 -- 172.21.15.7:0/3220505421 >> v1:172.21.15.188:6789/0 conn(0x7f969c0ac080 legacy=0x7f969c0c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.343+0000 7f96a1f38700 1 -- 172.21.15.7:0/3220505421 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c15e040 con 0x7f969c063580 2024-01-27T02:39:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.343+0000 7f9693fff700 1 -- 172.21.15.7:0/3220505421 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1958190647 0 0) 0x7f9698001110 con 0x7f969c0ac080 2024-01-27T02:39:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.343+0000 7f96a1f38700 1 --2- 172.21.15.7:0/3220505421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c063580 0x7f969c063a00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9694009e90 tx=0x7f9694009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b06f7abe0c0b7717 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.343+0000 7f9693fff700 1 -- 172.21.15.7:0/3220505421 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969400e070 con 0x7f969c063580 2024-01-27T02:39:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.343+0000 7f9693fff700 1 -- 172.21.15.7:0/3220505421 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969400bbf0 con 0x7f969c063580 2024-01-27T02:39:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.343+0000 7f9693fff700 1 -- 172.21.15.7:0/3220505421 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9694012430 con 0x7f969c063580 2024-01-27T02:39:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.344+0000 7f96a399b700 1 -- 172.21.15.7:0/3220505421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c063580 msgr2=0x7f969c063a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.344+0000 7f96a399b700 1 --2- 172.21.15.7:0/3220505421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c063580 0x7f969c063a00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9694009e90 tx=0x7f9694009780 comp rx=0 tx=0).stop 2024-01-27T02:39:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.344+0000 7f96a399b700 1 -- 172.21.15.7:0/3220505421 shutdown_connections 2024-01-27T02:39:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.344+0000 7f96a399b700 1 --2- 172.21.15.7:0/3220505421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c063580 0x7f969c063a00 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.344+0000 7f96a399b700 1 --2- 172.21.15.7:0/3220505421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f969c0ac450 0x7f969c06c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.344+0000 7f96a399b700 1 -- 172.21.15.7:0/3220505421 >> 172.21.15.7:0/3220505421 conn(0x7f969c068480 msgr2=0x7f969c0cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.344+0000 7f96a399b700 1 -- 172.21.15.7:0/3220505421 shutdown_connections 2024-01-27T02:39:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.344+0000 7f96a399b700 1 -- 172.21.15.7:0/3220505421 wait complete. 2024-01-27T02:39:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.345+0000 7f96a399b700 1 Processor -- start 2024-01-27T02:39:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.345+0000 7f96a399b700 1 -- start start 2024-01-27T02:39:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.346+0000 7f96a399b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c063580 0x7f969c15c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.346+0000 7f96a399b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f969c0ac080 0x7f969c0cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.346+0000 7f96a399b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f969c0ac450 0x7f969c0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.346+0000 7f96a399b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c15c540 con 0x7f969c0ac450 2024-01-27T02:39:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.346+0000 7f96a399b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c0c2e90 con 0x7f969c063580 2024-01-27T02:39:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.346+0000 7f96a399b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c0c3140 con 0x7f969c0ac080 2024-01-27T02:39:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.346+0000 7f96a1f38700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f969c0ac450 0x7f969c0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.347+0000 7f96a1f38700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f969c0ac450 0x7f969c0aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47996/0 (socket says 172.21.15.7:47996) 2024-01-27T02:39:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.347+0000 7f96a1f38700 1 -- 172.21.15.7:0/325452259 learned_addr learned my addr 172.21.15.7:0/325452259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.347+0000 7f96a1737700 1 --2- 172.21.15.7:0/325452259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c063580 0x7f969c15c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.347+0000 7f96a0f36700 1 --2- 172.21.15.7:0/325452259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f969c0ac080 0x7f969c0cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.347+0000 7f96a1f38700 1 -- 172.21.15.7:0/325452259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f969c0ac080 msgr2=0x7f969c0cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.347+0000 7f96a1f38700 1 --2- 172.21.15.7:0/325452259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f969c0ac080 0x7f969c0cdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.347+0000 7f96a1f38700 1 -- 172.21.15.7:0/325452259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c063580 msgr2=0x7f969c15c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.347+0000 7f96a1f38700 1 --2- 172.21.15.7:0/325452259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c063580 0x7f969c15c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.347+0000 7f96a1f38700 1 -- 172.21.15.7:0/325452259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9694009a20 con 0x7f969c0ac450 2024-01-27T02:39:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.347+0000 7f96a0f36700 1 --2- 172.21.15.7:0/325452259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f969c0ac080 0x7f969c0cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.347+0000 7f96a1f38700 1 --2- 172.21.15.7:0/325452259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f969c0ac450 0x7f969c0aedb0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f969400b6c0 tx=0x7f9694019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.347+0000 7f96917fa700 1 -- 172.21.15.7:0/325452259 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9694012e40 con 0x7f969c0ac450 2024-01-27T02:39:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.347+0000 7f96a399b700 1 -- 172.21.15.7:0/325452259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969c0c3420 con 0x7f969c0ac450 2024-01-27T02:39:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.347+0000 7f96917fa700 1 -- 172.21.15.7:0/325452259 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969401d630 con 0x7f969c0ac450 2024-01-27T02:39:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.347+0000 7f96917fa700 1 -- 172.21.15.7:0/325452259 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969401d9f0 con 0x7f969c0ac450 2024-01-27T02:39:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.347+0000 7f96a399b700 1 -- 172.21.15.7:0/325452259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969c0ad000 con 0x7f969c0ac450 2024-01-27T02:39:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.348+0000 7f96a399b700 1 -- 172.21.15.7:0/325452259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f969c04c710 con 0x7f969c0ac450 2024-01-27T02:39:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.349+0000 7f96917fa700 1 -- 172.21.15.7:0/325452259 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9694024070 con 0x7f969c0ac450 2024-01-27T02:39:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.350+0000 7f96917fa700 1 --2- 172.21.15.7:0/325452259 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9680066f30 0x7f96800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:12.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.350+0000 7f96917fa700 1 -- 172.21.15.7:0/325452259 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f96940871b0 con 0x7f969c0ac450 2024-01-27T02:39:12.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.350+0000 7f96917fa700 1 -- 172.21.15.7:0/325452259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=140}) v3 -- 0x7f968006ebb0 con 0x7f969c0ac450 2024-01-27T02:39:12.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.350+0000 7f96a1737700 1 --2- 172.21.15.7:0/325452259 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9680066f30 0x7f96800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:12.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.351+0000 7f96a1737700 1 --2- 172.21.15.7:0/325452259 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9680066f30 0x7f96800693e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9698007160 tx=0x7f9698007f50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:12.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.354+0000 7f96917fa700 1 -- 172.21.15.7:0/325452259 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9694054390 con 0x7f969c0ac450 2024-01-27T02:39:12.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:12.558+0000 7f96a399b700 1 -- 172.21.15.7:0/325452259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f969c0ceac0 con 0x7f969c0ac450 2024-01-27T02:39:13.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.139+0000 7f96917fa700 1 -- 172.21.15.7:0/325452259 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(140..140 src has 1..140) v4 ==== 288+0+0 (secure 0 0 0) 0x7f9694058750 con 0x7f969c0ac450 2024-01-27T02:39:13.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.139+0000 7f96917fa700 1 -- 172.21.15.7:0/325452259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=141}) v3 -- 0x7f968006cfe0 con 0x7f969c0ac450 2024-01-27T02:39:13.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.144+0000 7f96917fa700 1 -- 172.21.15.7:0/325452259 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v140) v1 ==== 181+0+0 (secure 0 0 0) 0x7f969402c030 con 0x7f969c0ac450 2024-01-27T02:39:13.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.150+0000 7f96a399b700 1 -- 172.21.15.7:0/325452259 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9680066f30 msgr2=0x7f96800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:13.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.150+0000 7f96a399b700 1 --2- 172.21.15.7:0/325452259 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9680066f30 0x7f96800693e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9698007160 tx=0x7f9698007f50 comp rx=0 tx=0).stop 2024-01-27T02:39:13.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.150+0000 7f96a399b700 1 -- 172.21.15.7:0/325452259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f969c0ac450 msgr2=0x7f969c0aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:13.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.150+0000 7f96a399b700 1 --2- 172.21.15.7:0/325452259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f969c0ac450 0x7f969c0aedb0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f969400b6c0 tx=0x7f9694019000 comp rx=0 tx=0).stop 2024-01-27T02:39:13.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.150+0000 7f96a399b700 1 -- 172.21.15.7:0/325452259 shutdown_connections 2024-01-27T02:39:13.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.150+0000 7f96a399b700 1 --2- 172.21.15.7:0/325452259 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9680066f30 0x7f96800693e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:13.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.150+0000 7f96a399b700 1 --2- 172.21.15.7:0/325452259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f969c0ac450 0x7f969c0aedb0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:13.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.150+0000 7f96a399b700 1 --2- 172.21.15.7:0/325452259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c063580 0x7f969c15c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:13.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.150+0000 7f96a399b700 1 --2- 172.21.15.7:0/325452259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f969c0ac080 0x7f969c0cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:13.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.150+0000 7f96a399b700 1 -- 172.21.15.7:0/325452259 >> 172.21.15.7:0/325452259 conn(0x7f969c068480 msgr2=0x7f969c0ca650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:13.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.151+0000 7f96a399b700 1 -- 172.21.15.7:0/325452259 shutdown_connections 2024-01-27T02:39:13.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.151+0000 7f96a399b700 1 -- 172.21.15.7:0/325452259 wait complete. 2024-01-27T02:39:13.152 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-27T02:39:13.167 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-27T02:39:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.308+0000 7f643667c700 1 Processor -- start 2024-01-27T02:39:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.308+0000 7f643667c700 1 -- start start 2024-01-27T02:39:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.308+0000 7f643667c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f643015a020 0x7f643015e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.308+0000 7f643667c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f64300680c0 0x7f64300c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.308+0000 7f643667c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64300ce6a0 con 0x7f643006bb30 2024-01-27T02:39:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.308+0000 7f643667c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f643015ec30 con 0x7f64300680c0 2024-01-27T02:39:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.308+0000 7f643667c700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64300ceb30 con 0x7f643015a020 2024-01-27T02:39:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.309+0000 7f6434c19700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f64300680c0 0x7f64300c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.309+0000 7f642f7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f643006bb30 0x7f64300679b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50830/0 (socket says 172.21.15.7:50830) 2024-01-27T02:39:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.309+0000 7f642f7fe700 1 -- 172.21.15.7:0/3143496544 learned_addr learned my addr 172.21.15.7:0/3143496544 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:13.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.309+0000 7f6434c19700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f64300680c0 0x7f64300c5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48008/0 (socket says 172.21.15.7:48008) 2024-01-27T02:39:13.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.309+0000 7f642ffff700 1 --2- 172.21.15.7:0/3143496544 >> v2:172.21.15.188:3300/0 conn(0x7f643015a020 0x7f643015e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:13.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.309+0000 7f6434c19700 1 -- 172.21.15.7:0/3143496544 >> v2:172.21.15.188:3300/0 conn(0x7f643015a020 msgr2=0x7f643015e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:13.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.309+0000 7f6434c19700 1 --2- 172.21.15.7:0/3143496544 >> v2:172.21.15.188:3300/0 conn(0x7f643015a020 0x7f643015e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:13.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.309+0000 7f6434c19700 1 -- 172.21.15.7:0/3143496544 >> v1:172.21.15.7:6789/0 conn(0x7f643006bb30 legacy=0x7f64300679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:13.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.309+0000 7f6434c19700 1 -- 172.21.15.7:0/3143496544 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6430069570 con 0x7f64300680c0 2024-01-27T02:39:13.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.309+0000 7f642effd700 1 -- 172.21.15.7:0/3143496544 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1140548009 0 0) 0x7f64300ce6a0 con 0x7f643006bb30 2024-01-27T02:39:13.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.309+0000 7f6434c19700 1 --2- 172.21.15.7:0/3143496544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64300680c0 0x7f64300c5dd0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f64300690b0 tx=0x7f6428009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3c65fd5da993b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:13.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.309+0000 7f642effd700 1 -- 172.21.15.7:0/3143496544 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642800e070 con 0x7f64300680c0 2024-01-27T02:39:13.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.309+0000 7f642effd700 1 -- 172.21.15.7:0/3143496544 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642800bb60 con 0x7f64300680c0 2024-01-27T02:39:13.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.309+0000 7f642effd700 1 -- 172.21.15.7:0/3143496544 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6428012430 con 0x7f64300680c0 2024-01-27T02:39:13.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.310+0000 7f643667c700 1 -- 172.21.15.7:0/3143496544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64300680c0 msgr2=0x7f64300c5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:13.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.310+0000 7f643667c700 1 --2- 172.21.15.7:0/3143496544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64300680c0 0x7f64300c5dd0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f64300690b0 tx=0x7f6428009780 comp rx=0 tx=0).stop 2024-01-27T02:39:13.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.310+0000 7f643667c700 1 -- 172.21.15.7:0/3143496544 shutdown_connections 2024-01-27T02:39:13.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.310+0000 7f643667c700 1 --2- 172.21.15.7:0/3143496544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64300680c0 0x7f64300c5dd0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:13.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.310+0000 7f643667c700 1 --2- 172.21.15.7:0/3143496544 >> v2:172.21.15.188:3300/0 conn(0x7f643015a020 0x7f643015e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:13.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.310+0000 7f643667c700 1 -- 172.21.15.7:0/3143496544 >> 172.21.15.7:0/3143496544 conn(0x7f64300714b0 msgr2=0x7f643006b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:13.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.310+0000 7f643667c700 1 -- 172.21.15.7:0/3143496544 shutdown_connections 2024-01-27T02:39:13.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.310+0000 7f643667c700 1 -- 172.21.15.7:0/3143496544 wait complete. 2024-01-27T02:39:13.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.312+0000 7f643667c700 1 Processor -- start 2024-01-27T02:39:13.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.312+0000 7f643667c700 1 -- start start 2024-01-27T02:39:13.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.313+0000 7f643667c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64300680c0 0x7f6430155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:13.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.313+0000 7f643667c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f643006bb30 0x7f64300cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:13.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.313+0000 7f643667c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f643015a020 0x7f6430158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:13.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.313+0000 7f643667c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64300b2a00 con 0x7f643015a020 2024-01-27T02:39:13.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.313+0000 7f643667c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64300cbb50 con 0x7f643006bb30 2024-01-27T02:39:13.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.313+0000 7f643667c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64300b1df0 con 0x7f64300680c0 2024-01-27T02:39:13.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.313+0000 7f6434c19700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f643015a020 0x7f6430158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:13.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.313+0000 7f6434c19700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f643015a020 0x7f6430158dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48018/0 (socket says 172.21.15.7:48018) 2024-01-27T02:39:13.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.313+0000 7f6434c19700 1 -- 172.21.15.7:0/3514217201 learned_addr learned my addr 172.21.15.7:0/3514217201 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:13.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.313+0000 7f642f7fe700 1 --2- 172.21.15.7:0/3514217201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f643006bb30 0x7f64300cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:13.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.313+0000 7f642ffff700 1 --2- 172.21.15.7:0/3514217201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64300680c0 0x7f6430155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:13.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.314+0000 7f6434c19700 1 -- 172.21.15.7:0/3514217201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64300680c0 msgr2=0x7f6430155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:13.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.314+0000 7f6434c19700 1 --2- 172.21.15.7:0/3514217201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64300680c0 0x7f6430155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:13.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.314+0000 7f6434c19700 1 -- 172.21.15.7:0/3514217201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f643006bb30 msgr2=0x7f64300cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:13.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.314+0000 7f6434c19700 1 --2- 172.21.15.7:0/3514217201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f643006bb30 0x7f64300cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:13.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.314+0000 7f6434c19700 1 -- 172.21.15.7:0/3514217201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6428009a20 con 0x7f643015a020 2024-01-27T02:39:13.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.314+0000 7f642f7fe700 1 --2- 172.21.15.7:0/3514217201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f643006bb30 0x7f64300cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:13.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.314+0000 7f6434c19700 1 --2- 172.21.15.7:0/3514217201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f643015a020 0x7f6430158dd0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f6428013000 tx=0x7f6428019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:13.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.314+0000 7f6417fff700 1 -- 172.21.15.7:0/3514217201 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6428012e80 con 0x7f643015a020 2024-01-27T02:39:13.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.314+0000 7f6417fff700 1 -- 172.21.15.7:0/3514217201 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642801d630 con 0x7f643015a020 2024-01-27T02:39:13.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.314+0000 7f643667c700 1 -- 172.21.15.7:0/3514217201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64300b20d0 con 0x7f643015a020 2024-01-27T02:39:13.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.314+0000 7f6417fff700 1 -- 172.21.15.7:0/3514217201 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642801d9d0 con 0x7f643015a020 2024-01-27T02:39:13.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.314+0000 7f643667c700 1 -- 172.21.15.7:0/3514217201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64300b2690 con 0x7f643015a020 2024-01-27T02:39:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.315+0000 7f643667c700 1 -- 172.21.15.7:0/3514217201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64300b2bb0 con 0x7f643015a020 2024-01-27T02:39:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.316+0000 7f6417fff700 1 -- 172.21.15.7:0/3514217201 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6428024070 con 0x7f643015a020 2024-01-27T02:39:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.316+0000 7f6417fff700 1 --2- 172.21.15.7:0/3514217201 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f641c066f30 0x7f641c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.316+0000 7f6417fff700 1 -- 172.21.15.7:0/3514217201 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5411+0+0 (secure 0 0 0) 0x7f6428086c00 con 0x7f643015a020 2024-01-27T02:39:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.316+0000 7f6417fff700 1 -- 172.21.15.7:0/3514217201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=141}) v3 -- 0x7f641c06ddb0 con 0x7f643015a020 2024-01-27T02:39:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.316+0000 7f642ffff700 1 --2- 172.21.15.7:0/3514217201 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f641c066f30 0x7f641c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:13.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.317+0000 7f642ffff700 1 --2- 172.21.15.7:0/3514217201 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f641c066f30 0x7f641c0693e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6420009d40 tx=0x7f6420008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:13.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.321+0000 7f6417fff700 1 -- 172.21.15.7:0/3514217201 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64300b2bb0 con 0x7f643015a020 2024-01-27T02:39:13.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:13.524+0000 7f643667c700 1 -- 172.21.15.7:0/3514217201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f643004c710 con 0x7f643015a020 2024-01-27T02:39:14.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.144+0000 7f6417fff700 1 -- 172.21.15.7:0/3514217201 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(141..141 src has 1..141) v4 ==== 322+0+0 (secure 0 0 0) 0x7f6428057c50 con 0x7f643015a020 2024-01-27T02:39:14.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.144+0000 7f6417fff700 1 -- 172.21.15.7:0/3514217201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=142}) v3 -- 0x7f641c06cb40 con 0x7f643015a020 2024-01-27T02:39:14.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.156+0000 7f6417fff700 1 -- 172.21.15.7:0/3514217201 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v141) v1 ==== 169+0+0 (secure 0 0 0) 0x7f642802c020 con 0x7f643015a020 2024-01-27T02:39:14.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.161+0000 7f643667c700 1 -- 172.21.15.7:0/3514217201 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f641c066f30 msgr2=0x7f641c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:14.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.161+0000 7f643667c700 1 --2- 172.21.15.7:0/3514217201 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f641c066f30 0x7f641c0693e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6420009d40 tx=0x7f6420008040 comp rx=0 tx=0).stop 2024-01-27T02:39:14.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.161+0000 7f643667c700 1 -- 172.21.15.7:0/3514217201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f643015a020 msgr2=0x7f6430158dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:14.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.161+0000 7f643667c700 1 --2- 172.21.15.7:0/3514217201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f643015a020 0x7f6430158dd0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f6428013000 tx=0x7f6428019000 comp rx=0 tx=0).stop 2024-01-27T02:39:14.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.161+0000 7f643667c700 1 -- 172.21.15.7:0/3514217201 shutdown_connections 2024-01-27T02:39:14.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.161+0000 7f643667c700 1 --2- 172.21.15.7:0/3514217201 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f641c066f30 0x7f641c0693e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:14.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.161+0000 7f643667c700 1 --2- 172.21.15.7:0/3514217201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f643015a020 0x7f6430158dd0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:14.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.161+0000 7f643667c700 1 --2- 172.21.15.7:0/3514217201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64300680c0 0x7f6430155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:14.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.161+0000 7f643667c700 1 --2- 172.21.15.7:0/3514217201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f643006bb30 0x7f64300cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:14.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.161+0000 7f643667c700 1 -- 172.21.15.7:0/3514217201 >> 172.21.15.7:0/3514217201 conn(0x7f64300714b0 msgr2=0x7f64300c67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:14.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.161+0000 7f643667c700 1 -- 172.21.15.7:0/3514217201 shutdown_connections 2024-01-27T02:39:14.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.161+0000 7f643667c700 1 -- 172.21.15.7:0/3514217201 wait complete. 2024-01-27T02:39:14.163 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-27T02:39:14.177 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-27T02:39:14.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.315+0000 7f0aa0846700 1 Processor -- start 2024-01-27T02:39:14.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.315+0000 7f0aa0846700 1 -- start start 2024-01-27T02:39:14.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.316+0000 7f0aa0846700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0a9c157c80 0x7f0a9c158050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:14.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.316+0000 7f0aa0846700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a9c0b3510 con 0x7f0a9c0ce510 2024-01-27T02:39:14.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.316+0000 7f0aa0846700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a9c06f8b0 con 0x7f0a9c063580 2024-01-27T02:39:14.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.316+0000 7f0aa0846700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9c0b2900 con 0x7f0a9c157c80 2024-01-27T02:39:14.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.316+0000 7f0a9a59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0a9c157c80 0x7f0a9c158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:14.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.316+0000 7f0a99d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0a9c0ce510 0x7f0a9c0b2090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50846/0 (socket says 172.21.15.7:50846) 2024-01-27T02:39:14.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.316+0000 7f0a99d9b700 1 -- 172.21.15.7:0/1763919745 learned_addr learned my addr 172.21.15.7:0/1763919745 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:14.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.316+0000 7f0a9a59c700 1 -- 172.21.15.7:0/1763919745 >> v1:172.21.15.188:6789/0 conn(0x7f0a9c063580 legacy=0x7f0a9c0ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:14.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.316+0000 7f0a9959a700 1 -- 172.21.15.7:0/1763919745 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688125610 0 0) 0x7f0a9c0b3510 con 0x7f0a9c0ce510 2024-01-27T02:39:14.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.316+0000 7f0a9a59c700 1 -- 172.21.15.7:0/1763919745 >> v1:172.21.15.7:6789/0 conn(0x7f0a9c0ce510 legacy=0x7f0a9c0b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:14.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.316+0000 7f0a9a59c700 1 -- 172.21.15.7:0/1763919745 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c0b2be0 con 0x7f0a9c157c80 2024-01-27T02:39:14.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.317+0000 7f0a9a59c700 1 --2- 172.21.15.7:0/1763919745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c157c80 0x7f0a9c158050 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0098e0 tx=0x7f0a8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b91ae8de4863f5 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:14.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.317+0000 7f0a9959a700 1 -- 172.21.15.7:0/1763919745 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c009e80 con 0x7f0a9c157c80 2024-01-27T02:39:14.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.317+0000 7f0a9959a700 1 -- 172.21.15.7:0/1763919745 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8c00b820 con 0x7f0a9c157c80 2024-01-27T02:39:14.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.317+0000 7f0a9959a700 1 -- 172.21.15.7:0/1763919745 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c00bba0 con 0x7f0a9c157c80 2024-01-27T02:39:14.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.317+0000 7f0aa0846700 1 -- 172.21.15.7:0/1763919745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c157c80 msgr2=0x7f0a9c158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:14.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.317+0000 7f0aa0846700 1 --2- 172.21.15.7:0/1763919745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c157c80 0x7f0a9c158050 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0098e0 tx=0x7f0a8c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:14.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.317+0000 7f0aa0846700 1 -- 172.21.15.7:0/1763919745 shutdown_connections 2024-01-27T02:39:14.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.317+0000 7f0aa0846700 1 --2- 172.21.15.7:0/1763919745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c157c80 0x7f0a9c158050 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:14.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.317+0000 7f0aa0846700 1 -- 172.21.15.7:0/1763919745 >> 172.21.15.7:0/1763919745 conn(0x7f0a9c068480 msgr2=0x7f0a9c0cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:14.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.318+0000 7f0aa0846700 1 -- 172.21.15.7:0/1763919745 shutdown_connections 2024-01-27T02:39:14.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.318+0000 7f0aa0846700 1 -- 172.21.15.7:0/1763919745 wait complete. 2024-01-27T02:39:14.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.319+0000 7f0aa0846700 1 Processor -- start 2024-01-27T02:39:14.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.319+0000 7f0aa0846700 1 -- start start 2024-01-27T02:39:14.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.319+0000 7f0aa0846700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a9c063580 0x7f0a9c16b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:14.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.319+0000 7f0aa0846700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a9c0ce510 0x7f0a9c14d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:14.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.319+0000 7f0aa0846700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c157c80 0x7f0a9c150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:14.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.320+0000 7f0aa0846700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c16b6b0 con 0x7f0a9c157c80 2024-01-27T02:39:14.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.320+0000 7f0aa0846700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c151580 con 0x7f0a9c063580 2024-01-27T02:39:14.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.320+0000 7f0aa0846700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c14dde0 con 0x7f0a9c0ce510 2024-01-27T02:39:14.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.320+0000 7f0a9ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c157c80 0x7f0a9c150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:14.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.320+0000 7f0a9ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c157c80 0x7f0a9c150170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48028/0 (socket says 172.21.15.7:48028) 2024-01-27T02:39:14.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.320+0000 7f0a9ad9d700 1 -- 172.21.15.7:0/1531744079 learned_addr learned my addr 172.21.15.7:0/1531744079 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:14.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.320+0000 7f0a9a59c700 1 --2- 172.21.15.7:0/1531744079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a9c063580 0x7f0a9c16b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:14.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.320+0000 7f0a99d9b700 1 --2- 172.21.15.7:0/1531744079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a9c0ce510 0x7f0a9c14d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:14.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.320+0000 7f0a9ad9d700 1 -- 172.21.15.7:0/1531744079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a9c0ce510 msgr2=0x7f0a9c14d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:14.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.320+0000 7f0a9ad9d700 1 --2- 172.21.15.7:0/1531744079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a9c0ce510 0x7f0a9c14d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:14.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.320+0000 7f0a9ad9d700 1 -- 172.21.15.7:0/1531744079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a9c063580 msgr2=0x7f0a9c16b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:14.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.320+0000 7f0a9ad9d700 1 --2- 172.21.15.7:0/1531744079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a9c063580 0x7f0a9c16b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:14.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.320+0000 7f0a9ad9d700 1 -- 172.21.15.7:0/1531744079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8c009420 con 0x7f0a9c157c80 2024-01-27T02:39:14.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.320+0000 7f0a9ad9d700 1 --2- 172.21.15.7:0/1531744079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c157c80 0x7f0a9c150170 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400c930 tx=0x7f0a940026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:14.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.320+0000 7f0a8a7fc700 1 -- 172.21.15.7:0/1531744079 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a94012630 con 0x7f0a9c157c80 2024-01-27T02:39:14.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.320+0000 7f0a8a7fc700 1 -- 172.21.15.7:0/1531744079 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a940127d0 con 0x7f0a9c157c80 2024-01-27T02:39:14.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.320+0000 7f0aa0846700 1 -- 172.21.15.7:0/1531744079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9c1506b0 con 0x7f0a9c157c80 2024-01-27T02:39:14.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.320+0000 7f0a8a7fc700 1 -- 172.21.15.7:0/1531744079 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a94012b90 con 0x7f0a9c157c80 2024-01-27T02:39:14.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.321+0000 7f0aa0846700 1 -- 172.21.15.7:0/1531744079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9c150c50 con 0x7f0a9c157c80 2024-01-27T02:39:14.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.321+0000 7f0aa0846700 1 -- 172.21.15.7:0/1531744079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a9c072060 con 0x7f0a9c157c80 2024-01-27T02:39:14.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.322+0000 7f0a8a7fc700 1 -- 172.21.15.7:0/1531744079 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0a9400a3b0 con 0x7f0a9c157c80 2024-01-27T02:39:14.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.322+0000 7f0a8a7fc700 1 --2- 172.21.15.7:0/1531744079 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0a7c066f30 0x7f0a7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:14.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.322+0000 7f0a8a7fc700 1 -- 172.21.15.7:0/1531744079 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0a94083fe0 con 0x7f0a9c157c80 2024-01-27T02:39:14.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.323+0000 7f0a9a59c700 1 --2- 172.21.15.7:0/1531744079 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0a7c066f30 0x7f0a7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:14.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.323+0000 7f0a9a59c700 1 --2- 172.21.15.7:0/1531744079 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0a7c066f30 0x7f0a7c0693e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0030c0 tx=0x7f0a8c016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:14.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.327+0000 7f0a8a7fc700 1 -- 172.21.15.7:0/1531744079 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a940514b0 con 0x7f0a9c157c80 2024-01-27T02:39:14.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.529+0000 7f0aa0846700 1 -- 172.21.15.7:0/1531744079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0a9c0b2ec0 con 0x7f0a9c157c80 2024-01-27T02:39:14.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.533+0000 7f0a8a7fc700 1 -- 172.21.15.7:0/1531744079 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0a94054ad0 con 0x7f0a9c157c80 2024-01-27T02:39:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.535+0000 7f0aa0846700 1 -- 172.21.15.7:0/1531744079 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0a7c066f30 msgr2=0x7f0a7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.535+0000 7f0aa0846700 1 --2- 172.21.15.7:0/1531744079 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0a7c066f30 0x7f0a7c0693e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0030c0 tx=0x7f0a8c016000 comp rx=0 tx=0).stop 2024-01-27T02:39:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.535+0000 7f0aa0846700 1 -- 172.21.15.7:0/1531744079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c157c80 msgr2=0x7f0a9c150170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.535+0000 7f0aa0846700 1 --2- 172.21.15.7:0/1531744079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c157c80 0x7f0a9c150170 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400c930 tx=0x7f0a940026e0 comp rx=0 tx=0).stop 2024-01-27T02:39:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.535+0000 7f0aa0846700 1 -- 172.21.15.7:0/1531744079 shutdown_connections 2024-01-27T02:39:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.535+0000 7f0aa0846700 1 --2- 172.21.15.7:0/1531744079 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0a7c066f30 0x7f0a7c0693e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:14.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.535+0000 7f0aa0846700 1 --2- 172.21.15.7:0/1531744079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c157c80 0x7f0a9c150170 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:14.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.535+0000 7f0aa0846700 1 --2- 172.21.15.7:0/1531744079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a9c063580 0x7f0a9c16b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:14.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.535+0000 7f0aa0846700 1 --2- 172.21.15.7:0/1531744079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a9c0ce510 0x7f0a9c14d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:14.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.535+0000 7f0aa0846700 1 -- 172.21.15.7:0/1531744079 >> 172.21.15.7:0/1531744079 conn(0x7f0a9c068480 msgr2=0x7f0a9c0afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:14.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.535+0000 7f0aa0846700 1 -- 172.21.15.7:0/1531744079 shutdown_connections 2024-01-27T02:39:14.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.535+0000 7f0aa0846700 1 -- 172.21.15.7:0/1531744079 wait complete. 2024-01-27T02:39:14.549 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2024-01-27T02:39:14.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.680+0000 7fe4050f5700 1 Processor -- start 2024-01-27T02:39:14.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.680+0000 7fe4050f5700 1 -- start start 2024-01-27T02:39:14.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.681+0000 7fe4050f5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe4000c89a0 0x7fe40006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:14.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.681+0000 7fe4050f5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4000c4450 con 0x7fe400063c90 2024-01-27T02:39:14.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.681+0000 7fe4050f5700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40006e9e0 con 0x7fe4000c8d70 2024-01-27T02:39:14.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.681+0000 7fe4050f5700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe40006e3d0 con 0x7fe4000c89a0 2024-01-27T02:39:14.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.681+0000 7fe3fed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe4000c89a0 0x7fe40006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:14.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.681+0000 7fe3ff59e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe400063c90 0x7fe40015a710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50850/0 (socket says 172.21.15.7:50850) 2024-01-27T02:39:14.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.681+0000 7fe3ff59e700 1 -- 172.21.15.7:0/2688729278 learned_addr learned my addr 172.21.15.7:0/2688729278 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:14.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.682+0000 7fe3fed9d700 1 -- 172.21.15.7:0/2688729278 >> v1:172.21.15.188:6789/0 conn(0x7fe4000c8d70 legacy=0x7fe400063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:14.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.682+0000 7fe3fed9d700 1 -- 172.21.15.7:0/2688729278 >> v1:172.21.15.7:6789/0 conn(0x7fe400063c90 legacy=0x7fe40015a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:14.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.682+0000 7fe3fdd9b700 1 -- 172.21.15.7:0/2688729278 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432728617 0 0) 0x7fe4000c4450 con 0x7fe400063c90 2024-01-27T02:39:14.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.682+0000 7fe3fed9d700 1 -- 172.21.15.7:0/2688729278 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40015b360 con 0x7fe4000c89a0 2024-01-27T02:39:14.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.682+0000 7fe3fed9d700 1 --2- 172.21.15.7:0/2688729278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4000c89a0 0x7fe40006ee70 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80098e0 tx=0x7fe3e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13257a003643daf0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:14.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.682+0000 7fe3fdd9b700 1 -- 172.21.15.7:0/2688729278 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8009e80 con 0x7fe4000c89a0 2024-01-27T02:39:14.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.682+0000 7fe3fdd9b700 1 -- 172.21.15.7:0/2688729278 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e800b860 con 0x7fe4000c89a0 2024-01-27T02:39:14.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.682+0000 7fe3fdd9b700 1 -- 172.21.15.7:0/2688729278 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e800bbe0 con 0x7fe4000c89a0 2024-01-27T02:39:14.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.683+0000 7fe4050f5700 1 -- 172.21.15.7:0/2688729278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4000c89a0 msgr2=0x7fe40006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:14.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.683+0000 7fe4050f5700 1 --2- 172.21.15.7:0/2688729278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4000c89a0 0x7fe40006ee70 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80098e0 tx=0x7fe3e80091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:14.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.683+0000 7fe4050f5700 1 -- 172.21.15.7:0/2688729278 shutdown_connections 2024-01-27T02:39:14.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.683+0000 7fe4050f5700 1 --2- 172.21.15.7:0/2688729278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4000c89a0 0x7fe40006ee70 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:14.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.683+0000 7fe4050f5700 1 -- 172.21.15.7:0/2688729278 >> 172.21.15.7:0/2688729278 conn(0x7fe4000cdc30 msgr2=0x7fe4000ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:14.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.683+0000 7fe4050f5700 1 -- 172.21.15.7:0/2688729278 shutdown_connections 2024-01-27T02:39:14.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.683+0000 7fe4050f5700 1 -- 172.21.15.7:0/2688729278 wait complete. 2024-01-27T02:39:14.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.684+0000 7fe4050f5700 1 Processor -- start 2024-01-27T02:39:14.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.684+0000 7fe4050f5700 1 -- start start 2024-01-27T02:39:14.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.685+0000 7fe4050f5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe400063c90 0x7fe400156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:14.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.685+0000 7fe4050f5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4000c89a0 0x7fe40006bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:14.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.685+0000 7fe4050f5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4000c8d70 0x7fe40015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:14.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.685+0000 7fe4050f5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4000ae0d0 con 0x7fe400063c90 2024-01-27T02:39:14.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.685+0000 7fe4050f5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4001570c0 con 0x7fe4000c8d70 2024-01-27T02:39:14.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.685+0000 7fe4050f5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4000ad4c0 con 0x7fe4000c89a0 2024-01-27T02:39:14.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.685+0000 7fe3fed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe400063c90 0x7fe400156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:14.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.685+0000 7fe3fed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe400063c90 0x7fe400156b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48048/0 (socket says 172.21.15.7:48048) 2024-01-27T02:39:14.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.685+0000 7fe3fed9d700 1 -- 172.21.15.7:0/4004578397 learned_addr learned my addr 172.21.15.7:0/4004578397 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:14.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.685+0000 7fe3ff59e700 1 --2- 172.21.15.7:0/4004578397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4000c8d70 0x7fe40015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:14.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.685+0000 7fe3fe59c700 1 --2- 172.21.15.7:0/4004578397 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4000c89a0 0x7fe40006bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:14.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.685+0000 7fe3fed9d700 1 -- 172.21.15.7:0/4004578397 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4000c89a0 msgr2=0x7fe40006bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:14.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.685+0000 7fe3fed9d700 1 --2- 172.21.15.7:0/4004578397 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4000c89a0 0x7fe40006bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:14.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.685+0000 7fe3fed9d700 1 -- 172.21.15.7:0/4004578397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4000c8d70 msgr2=0x7fe40015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:14.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.685+0000 7fe3fed9d700 1 --2- 172.21.15.7:0/4004578397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4000c8d70 0x7fe40015dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:14.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.686+0000 7fe3fed9d700 1 -- 172.21.15.7:0/4004578397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e8009420 con 0x7fe400063c90 2024-01-27T02:39:14.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.686+0000 7fe3fed9d700 1 --2- 172.21.15.7:0/4004578397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe400063c90 0x7fe400156b80 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fe40006dcc0 tx=0x7fe3e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:14.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.686+0000 7fe3f6ffd700 1 -- 172.21.15.7:0/4004578397 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e801f070 con 0x7fe400063c90 2024-01-27T02:39:14.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.686+0000 7fe3f6ffd700 1 -- 172.21.15.7:0/4004578397 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e800b9d0 con 0x7fe400063c90 2024-01-27T02:39:14.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.686+0000 7fe3f6ffd700 1 -- 172.21.15.7:0/4004578397 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8025670 con 0x7fe400063c90 2024-01-27T02:39:14.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.686+0000 7fe4050f5700 1 -- 172.21.15.7:0/4004578397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4000ad7a0 con 0x7fe400063c90 2024-01-27T02:39:14.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.686+0000 7fe4050f5700 1 -- 172.21.15.7:0/4004578397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4000add60 con 0x7fe400063c90 2024-01-27T02:39:14.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.687+0000 7fe4050f5700 1 -- 172.21.15.7:0/4004578397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40004c710 con 0x7fe400063c90 2024-01-27T02:39:14.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.687+0000 7fe3f6ffd700 1 -- 172.21.15.7:0/4004578397 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe3e8023070 con 0x7fe400063c90 2024-01-27T02:39:14.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.688+0000 7fe3f6ffd700 1 --2- 172.21.15.7:0/4004578397 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe3e0066f30 0x7fe3e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:14.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.688+0000 7fe3f6ffd700 1 -- 172.21.15.7:0/4004578397 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fe3e8085880 con 0x7fe400063c90 2024-01-27T02:39:14.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.688+0000 7fe3fe59c700 1 --2- 172.21.15.7:0/4004578397 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe3e0066f30 0x7fe3e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:14.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.688+0000 7fe3fe59c700 1 --2- 172.21.15.7:0/4004578397 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe3e0066f30 0x7fe3e00693e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe4000b31b0 tx=0x7fe3f0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:14.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.692+0000 7fe3f6ffd700 1 -- 172.21.15.7:0/4004578397 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3e8052df0 con 0x7fe400063c90 2024-01-27T02:39:14.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:14.983+0000 7fe4050f5700 1 -- 172.21.15.7:0/4004578397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) v1 -- 0x7fe40006f710 con 0x7fe400063c90 2024-01-27T02:39:15.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.148+0000 7fe3f6ffd700 1 -- 172.21.15.7:0/4004578397 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v373) v1 ==== 140+0+0 (secure 0 0 0) 0x7fe3e80150d0 con 0x7fe400063c90 2024-01-27T02:39:15.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.150+0000 7fe4050f5700 1 -- 172.21.15.7:0/4004578397 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe3e0066f30 msgr2=0x7fe3e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:15.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.150+0000 7fe4050f5700 1 --2- 172.21.15.7:0/4004578397 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe3e0066f30 0x7fe3e00693e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe4000b31b0 tx=0x7fe3f0009040 comp rx=0 tx=0).stop 2024-01-27T02:39:15.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.150+0000 7fe4050f5700 1 -- 172.21.15.7:0/4004578397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe400063c90 msgr2=0x7fe400156b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:15.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.150+0000 7fe4050f5700 1 --2- 172.21.15.7:0/4004578397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe400063c90 0x7fe400156b80 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fe40006dcc0 tx=0x7fe3e8017000 comp rx=0 tx=0).stop 2024-01-27T02:39:15.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.150+0000 7fe4050f5700 1 -- 172.21.15.7:0/4004578397 shutdown_connections 2024-01-27T02:39:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.150+0000 7fe4050f5700 1 --2- 172.21.15.7:0/4004578397 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe3e0066f30 0x7fe3e00693e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.150+0000 7fe4050f5700 1 --2- 172.21.15.7:0/4004578397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4000c8d70 0x7fe40015dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.150+0000 7fe4050f5700 1 --2- 172.21.15.7:0/4004578397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe400063c90 0x7fe400156b80 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.150+0000 7fe4050f5700 1 --2- 172.21.15.7:0/4004578397 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4000c89a0 0x7fe40006bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.150+0000 7fe4050f5700 1 -- 172.21.15.7:0/4004578397 >> 172.21.15.7:0/4004578397 conn(0x7fe4000cdc30 msgr2=0x7fe400063580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.151+0000 7fe4050f5700 1 -- 172.21.15.7:0/4004578397 shutdown_connections 2024-01-27T02:39:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.151+0000 7fe4050f5700 1 -- 172.21.15.7:0/4004578397 wait complete. 2024-01-27T02:39:15.166 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2024-01-27T02:39:15.167 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) 2024-01-27T02:39:15.167 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2024-01-27T02:39:15.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.303+0000 7fc9d07ff700 1 Processor -- start 2024-01-27T02:39:15.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.303+0000 7fc9d07ff700 1 -- start start 2024-01-27T02:39:15.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.304+0000 7fc9d07ff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9c80b1c00 0x7fc9c806e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:15.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.304+0000 7fc9d07ff700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc9c80b1fd0 0x7fc9c8156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:15.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.304+0000 7fc9d07ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c8157610 con 0x7fc9c815b5f0 2024-01-27T02:39:15.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.304+0000 7fc9d07ff700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9c8157310 con 0x7fc9c80b1fd0 2024-01-27T02:39:15.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.304+0000 7fc9d07ff700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c806ee00 con 0x7fc9c80b1c00 2024-01-27T02:39:15.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.304+0000 7fc9cdd9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc9c815b5f0 0x7fc9c81566c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50862/0 (socket says 172.21.15.7:50862) 2024-01-27T02:39:15.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.304+0000 7fc9cdd9a700 1 -- 172.21.15.7:0/2397112878 learned_addr learned my addr 172.21.15.7:0/2397112878 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:15.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.305+0000 7fc9ce59b700 1 --2- 172.21.15.7:0/2397112878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9c80b1c00 0x7fc9c806e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:15.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.305+0000 7fc9ced9c700 1 --2- 172.21.15.7:0/2397112878 >> v2:172.21.15.188:3300/0 conn(0x7fc9c80b1fd0 0x7fc9c8156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.305+0000 7fc9cd599700 1 -- 172.21.15.7:0/2397112878 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278186841 0 0) 0x7fc9c8157610 con 0x7fc9c815b5f0 2024-01-27T02:39:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.305+0000 7fc9cd599700 1 -- 172.21.15.7:0/2397112878 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9b4003570 con 0x7fc9c815b5f0 2024-01-27T02:39:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.305+0000 7fc9ce59b700 1 -- 172.21.15.7:0/2397112878 >> v2:172.21.15.188:3300/0 conn(0x7fc9c80b1fd0 msgr2=0x7fc9c8156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.305+0000 7fc9ce59b700 1 --2- 172.21.15.7:0/2397112878 >> v2:172.21.15.188:3300/0 conn(0x7fc9c80b1fd0 0x7fc9c8156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.305+0000 7fc9ce59b700 1 -- 172.21.15.7:0/2397112878 >> v1:172.21.15.7:6789/0 conn(0x7fc9c815b5f0 legacy=0x7fc9c81566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.305+0000 7fc9cd599700 1 -- 172.21.15.7:0/2397112878 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1056341231 0 0) 0x7fc9b4003570 con 0x7fc9c815b5f0 2024-01-27T02:39:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.305+0000 7fc9ce59b700 1 -- 172.21.15.7:0/2397112878 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c806f0e0 con 0x7fc9c80b1c00 2024-01-27T02:39:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.305+0000 7fc9ced9c700 1 --2- 172.21.15.7:0/2397112878 >> v2:172.21.15.188:3300/0 conn(0x7fc9c80b1fd0 0x7fc9c8156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:39:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.305+0000 7fc9ce59b700 1 --2- 172.21.15.7:0/2397112878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9c80b1c00 0x7fc9c806e540 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40098e0 tx=0x7fc9c40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8737de40ace5a3a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.305+0000 7fc9cd599700 1 -- 172.21.15.7:0/2397112878 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8001470 con 0x7fc9c80b1c00 2024-01-27T02:39:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.305+0000 7fc9cd599700 1 -- 172.21.15.7:0/2397112878 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c4011420 con 0x7fc9c80b1c00 2024-01-27T02:39:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.305+0000 7fc9cd599700 1 -- 172.21.15.7:0/2397112878 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c40117c0 con 0x7fc9c80b1c00 2024-01-27T02:39:15.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.306+0000 7fc9d07ff700 1 -- 172.21.15.7:0/2397112878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9c80b1c00 msgr2=0x7fc9c806e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:15.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.306+0000 7fc9d07ff700 1 --2- 172.21.15.7:0/2397112878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9c80b1c00 0x7fc9c806e540 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40098e0 tx=0x7fc9c40091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:15.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.306+0000 7fc9d07ff700 1 -- 172.21.15.7:0/2397112878 shutdown_connections 2024-01-27T02:39:15.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.306+0000 7fc9d07ff700 1 --2- 172.21.15.7:0/2397112878 >> v2:172.21.15.188:3300/0 conn(0x7fc9c80b1fd0 0x7fc9c8156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:15.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.306+0000 7fc9d07ff700 1 --2- 172.21.15.7:0/2397112878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9c80b1c00 0x7fc9c806e540 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:15.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.306+0000 7fc9d07ff700 1 -- 172.21.15.7:0/2397112878 >> 172.21.15.7:0/2397112878 conn(0x7fc9c8068430 msgr2=0x7fc9c806be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:15.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.306+0000 7fc9d07ff700 1 -- 172.21.15.7:0/2397112878 shutdown_connections 2024-01-27T02:39:15.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.306+0000 7fc9d07ff700 1 -- 172.21.15.7:0/2397112878 wait complete. 2024-01-27T02:39:15.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.307+0000 7fc9d07ff700 1 Processor -- start 2024-01-27T02:39:15.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.307+0000 7fc9d07ff700 1 -- start start 2024-01-27T02:39:15.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.308+0000 7fc9d07ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c80b1c00 0x7fc9c80ab700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:15.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.308+0000 7fc9d07ff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9c80b1fd0 0x7fc9c81532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:15.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.308+0000 7fc9d07ff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9c815b5f0 0x7fc9c80aae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:15.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.308+0000 7fc9d07ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c813ff70 con 0x7fc9c80b1c00 2024-01-27T02:39:15.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.308+0000 7fc9d07ff700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c80ac8f0 con 0x7fc9c80b1fd0 2024-01-27T02:39:15.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.308+0000 7fc9d07ff700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c80abce0 con 0x7fc9c815b5f0 2024-01-27T02:39:15.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.308+0000 7fc9ce59b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c80b1c00 0x7fc9c80ab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:15.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.308+0000 7fc9ce59b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c80b1c00 0x7fc9c80ab700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48058/0 (socket says 172.21.15.7:48058) 2024-01-27T02:39:15.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.308+0000 7fc9ce59b700 1 -- 172.21.15.7:0/2641010024 learned_addr learned my addr 172.21.15.7:0/2641010024 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:15.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.308+0000 7fc9cdd9a700 1 --2- 172.21.15.7:0/2641010024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9c80b1fd0 0x7fc9c81532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:15.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.308+0000 7fc9ced9c700 1 --2- 172.21.15.7:0/2641010024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9c815b5f0 0x7fc9c80aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:15.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.309+0000 7fc9ce59b700 1 -- 172.21.15.7:0/2641010024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9c815b5f0 msgr2=0x7fc9c80aae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:15.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.309+0000 7fc9ce59b700 1 --2- 172.21.15.7:0/2641010024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9c815b5f0 0x7fc9c80aae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:15.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.309+0000 7fc9ce59b700 1 -- 172.21.15.7:0/2641010024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9c80b1fd0 msgr2=0x7fc9c81532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:15.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.309+0000 7fc9ce59b700 1 --2- 172.21.15.7:0/2641010024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9c80b1fd0 0x7fc9c81532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:15.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.309+0000 7fc9ce59b700 1 -- 172.21.15.7:0/2641010024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c0009a20 con 0x7fc9c80b1c00 2024-01-27T02:39:15.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.309+0000 7fc9ce59b700 1 --2- 172.21.15.7:0/2641010024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c80b1c00 0x7fc9c80ab700 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40093f0 tx=0x7fc9c4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:15.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.309+0000 7fc9be7fc700 1 -- 172.21.15.7:0/2641010024 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c401cc90 con 0x7fc9c80b1c00 2024-01-27T02:39:15.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.309+0000 7fc9be7fc700 1 -- 172.21.15.7:0/2641010024 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c401cdf0 con 0x7fc9c80b1c00 2024-01-27T02:39:15.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.309+0000 7fc9d07ff700 1 -- 172.21.15.7:0/2641010024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c4009420 con 0x7fc9c80b1c00 2024-01-27T02:39:15.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.309+0000 7fc9be7fc700 1 -- 172.21.15.7:0/2641010024 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c401a660 con 0x7fc9c80b1c00 2024-01-27T02:39:15.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.309+0000 7fc9d07ff700 1 -- 172.21.15.7:0/2641010024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c80ac2a0 con 0x7fc9c80b1c00 2024-01-27T02:39:15.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.310+0000 7fc9d07ff700 1 -- 172.21.15.7:0/2641010024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c804c710 con 0x7fc9c80b1c00 2024-01-27T02:39:15.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.310+0000 7fc9be7fc700 1 -- 172.21.15.7:0/2641010024 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc9c401aa50 con 0x7fc9c80b1c00 2024-01-27T02:39:15.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.311+0000 7fc9be7fc700 1 --2- 172.21.15.7:0/2641010024 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc9ac066f30 0x7fc9ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:15.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.311+0000 7fc9be7fc700 1 -- 172.21.15.7:0/2641010024 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fc9c4085bb0 con 0x7fc9c80b1c00 2024-01-27T02:39:15.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.311+0000 7fc9cdd9a700 1 --2- 172.21.15.7:0/2641010024 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc9ac066f30 0x7fc9ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:15.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.312+0000 7fc9cdd9a700 1 --2- 172.21.15.7:0/2641010024 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc9ac066f30 0x7fc9ac0693e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8005b80 tx=0x7fc9b8001120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:15.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.316+0000 7fc9be7fc700 1 -- 172.21.15.7:0/2641010024 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc9c40531d0 con 0x7fc9c80b1c00 2024-01-27T02:39:15.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:15.598+0000 7fc9d07ff700 1 -- 172.21.15.7:0/2641010024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) v1 -- 0x7fc9c806f3c0 con 0x7fc9c80b1c00 2024-01-27T02:39:16.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.153+0000 7fc9be7fc700 1 -- 172.21.15.7:0/2641010024 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v374) v1 ==== 135+0+0 (secure 0 0 0) 0x7fc9c40567f0 con 0x7fc9c80b1c00 2024-01-27T02:39:16.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.156+0000 7fc9d07ff700 1 -- 172.21.15.7:0/2641010024 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc9ac066f30 msgr2=0x7fc9ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:16.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.156+0000 7fc9d07ff700 1 --2- 172.21.15.7:0/2641010024 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc9ac066f30 0x7fc9ac0693e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8005b80 tx=0x7fc9b8001120 comp rx=0 tx=0).stop 2024-01-27T02:39:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.156+0000 7fc9d07ff700 1 -- 172.21.15.7:0/2641010024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c80b1c00 msgr2=0x7fc9c80ab700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.156+0000 7fc9d07ff700 1 --2- 172.21.15.7:0/2641010024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c80b1c00 0x7fc9c80ab700 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40093f0 tx=0x7fc9c4016000 comp rx=0 tx=0).stop 2024-01-27T02:39:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.156+0000 7fc9d07ff700 1 -- 172.21.15.7:0/2641010024 shutdown_connections 2024-01-27T02:39:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.156+0000 7fc9d07ff700 1 --2- 172.21.15.7:0/2641010024 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc9ac066f30 0x7fc9ac0693e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.156+0000 7fc9d07ff700 1 --2- 172.21.15.7:0/2641010024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9c815b5f0 0x7fc9c80aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.156+0000 7fc9d07ff700 1 --2- 172.21.15.7:0/2641010024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c80b1c00 0x7fc9c80ab700 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.156+0000 7fc9d07ff700 1 --2- 172.21.15.7:0/2641010024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9c80b1fd0 0x7fc9c81532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.156+0000 7fc9d07ff700 1 -- 172.21.15.7:0/2641010024 >> 172.21.15.7:0/2641010024 conn(0x7fc9c8068430 msgr2=0x7fc9c815a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.156+0000 7fc9d07ff700 1 -- 172.21.15.7:0/2641010024 shutdown_connections 2024-01-27T02:39:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.156+0000 7fc9d07ff700 1 -- 172.21.15.7:0/2641010024 wait complete. 2024-01-27T02:39:16.171 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-27T02:39:16.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.309+0000 7fcd78ddf700 1 Processor -- start 2024-01-27T02:39:16.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.309+0000 7fcd78ddf700 1 -- start start 2024-01-27T02:39:16.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.310+0000 7fcd78ddf700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd74157c80 0x7fcd74158050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:16.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.310+0000 7fcd78ddf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcd740ce510 0x7fcd740b2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:16.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.310+0000 7fcd78ddf700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd740b3510 con 0x7fcd74063580 2024-01-27T02:39:16.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.310+0000 7fcd78ddf700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7406f8b0 con 0x7fcd740ce510 2024-01-27T02:39:16.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.310+0000 7fcd78ddf700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd740b2900 con 0x7fcd74157c80 2024-01-27T02:39:16.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.310+0000 7fcd71d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcd740ce510 0x7fcd740b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:16.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.311+0000 7fcd71d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcd740ce510 0x7fcd740b2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48068/0 (socket says 172.21.15.7:48068) 2024-01-27T02:39:16.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.311+0000 7fcd71d9b700 1 -- 172.21.15.7:0/3020461747 learned_addr learned my addr 172.21.15.7:0/3020461747 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:16.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.311+0000 7fcd7259c700 1 --2- 172.21.15.7:0/3020461747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd74157c80 0x7fcd74158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:16.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.311+0000 7fcd71d9b700 1 -- 172.21.15.7:0/3020461747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd74157c80 msgr2=0x7fcd74158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:16.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.311+0000 7fcd71d9b700 1 --2- 172.21.15.7:0/3020461747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd74157c80 0x7fcd74158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:16.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.311+0000 7fcd71d9b700 1 -- 172.21.15.7:0/3020461747 >> v1:172.21.15.188:6789/0 conn(0x7fcd74063580 legacy=0x7fcd7415bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.311+0000 7fcd71d9b700 1 -- 172.21.15.7:0/3020461747 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd740b2be0 con 0x7fcd740ce510 2024-01-27T02:39:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.311+0000 7fcd71d9b700 1 --2- 172.21.15.7:0/3020461747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd740ce510 0x7fcd740b2090 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0099b0 tx=0x7fcd5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40adc2379cc8238 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.311+0000 7fcd7159a700 1 -- 172.21.15.7:0/3020461747 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c009ea0 con 0x7fcd740ce510 2024-01-27T02:39:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.311+0000 7fcd7159a700 1 -- 172.21.15.7:0/3020461747 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5c00b860 con 0x7fcd740ce510 2024-01-27T02:39:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.311+0000 7fcd7159a700 1 -- 172.21.15.7:0/3020461747 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c00bc00 con 0x7fcd740ce510 2024-01-27T02:39:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.312+0000 7fcd78ddf700 1 -- 172.21.15.7:0/3020461747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd740ce510 msgr2=0x7fcd740b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.312+0000 7fcd78ddf700 1 --2- 172.21.15.7:0/3020461747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd740ce510 0x7fcd740b2090 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0099b0 tx=0x7fcd5c009250 comp rx=0 tx=0).stop 2024-01-27T02:39:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.312+0000 7fcd78ddf700 1 -- 172.21.15.7:0/3020461747 shutdown_connections 2024-01-27T02:39:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.312+0000 7fcd78ddf700 1 --2- 172.21.15.7:0/3020461747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd74157c80 0x7fcd74158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.312+0000 7fcd78ddf700 1 --2- 172.21.15.7:0/3020461747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd740ce510 0x7fcd740b2090 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.312+0000 7fcd78ddf700 1 -- 172.21.15.7:0/3020461747 >> 172.21.15.7:0/3020461747 conn(0x7fcd74068480 msgr2=0x7fcd740cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.312+0000 7fcd78ddf700 1 -- 172.21.15.7:0/3020461747 shutdown_connections 2024-01-27T02:39:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.312+0000 7fcd78ddf700 1 -- 172.21.15.7:0/3020461747 wait complete. 2024-01-27T02:39:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.314+0000 7fcd78ddf700 1 Processor -- start 2024-01-27T02:39:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.314+0000 7fcd78ddf700 1 -- start start 2024-01-27T02:39:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.315+0000 7fcd78ddf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd74063580 0x7fcd7414d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.315+0000 7fcd78ddf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd740ce510 0x7fcd74148640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.315+0000 7fcd78ddf700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd74157c80 0x7fcd740af610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.315+0000 7fcd78ddf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7415a3b0 con 0x7fcd740ce510 2024-01-27T02:39:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.315+0000 7fcd78ddf700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7414da90 con 0x7fcd74063580 2024-01-27T02:39:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.315+0000 7fcd78ddf700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd741592c0 con 0x7fcd74157c80 2024-01-27T02:39:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.315+0000 7fcd71d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd740ce510 0x7fcd74148640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.315+0000 7fcd71d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd740ce510 0x7fcd74148640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48070/0 (socket says 172.21.15.7:48070) 2024-01-27T02:39:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.315+0000 7fcd71d9b700 1 -- 172.21.15.7:0/1431976118 learned_addr learned my addr 172.21.15.7:0/1431976118 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.315+0000 7fcd7259c700 1 --2- 172.21.15.7:0/1431976118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd74063580 0x7fcd7414d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.315+0000 7fcd72d9d700 1 --2- 172.21.15.7:0/1431976118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd74157c80 0x7fcd740af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.315+0000 7fcd71d9b700 1 -- 172.21.15.7:0/1431976118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd74157c80 msgr2=0x7fcd740af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.315+0000 7fcd71d9b700 1 --2- 172.21.15.7:0/1431976118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd74157c80 0x7fcd740af610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.315+0000 7fcd71d9b700 1 -- 172.21.15.7:0/1431976118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd74063580 msgr2=0x7fcd7414d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.315+0000 7fcd71d9b700 1 --2- 172.21.15.7:0/1431976118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd74063580 0x7fcd7414d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.315+0000 7fcd71d9b700 1 -- 172.21.15.7:0/1431976118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5c0094f0 con 0x7fcd740ce510 2024-01-27T02:39:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.315+0000 7fcd71d9b700 1 --2- 172.21.15.7:0/1431976118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd740ce510 0x7fcd74148640 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fcd740700c0 tx=0x7fcd5c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.316+0000 7fcd6a7fc700 1 -- 172.21.15.7:0/1431976118 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c00bbe0 con 0x7fcd740ce510 2024-01-27T02:39:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.316+0000 7fcd6a7fc700 1 -- 172.21.15.7:0/1431976118 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5c011420 con 0x7fcd740ce510 2024-01-27T02:39:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.316+0000 7fcd78ddf700 1 -- 172.21.15.7:0/1431976118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd741595a0 con 0x7fcd740ce510 2024-01-27T02:39:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.316+0000 7fcd6a7fc700 1 -- 172.21.15.7:0/1431976118 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c0117c0 con 0x7fcd740ce510 2024-01-27T02:39:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.316+0000 7fcd78ddf700 1 -- 172.21.15.7:0/1431976118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7415a040 con 0x7fcd740ce510 2024-01-27T02:39:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.317+0000 7fcd78ddf700 1 -- 172.21.15.7:0/1431976118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7415a1d0 con 0x7fcd740ce510 2024-01-27T02:39:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.317+0000 7fcd6a7fc700 1 -- 172.21.15.7:0/1431976118 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcd5c011bf0 con 0x7fcd740ce510 2024-01-27T02:39:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.317+0000 7fcd6a7fc700 1 --2- 172.21.15.7:0/1431976118 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcd54066f30 0x7fcd540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.318+0000 7fcd6a7fc700 1 -- 172.21.15.7:0/1431976118 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fcd5c085c60 con 0x7fcd740ce510 2024-01-27T02:39:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.318+0000 7fcd7259c700 1 --2- 172.21.15.7:0/1431976118 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcd54066f30 0x7fcd540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.318+0000 7fcd7259c700 1 --2- 172.21.15.7:0/1431976118 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcd54066f30 0x7fcd540693e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009ca0 tx=0x7fcd64008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.322+0000 7fcd6a7fc700 1 -- 172.21.15.7:0/1431976118 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd5c053130 con 0x7fcd740ce510 2024-01-27T02:39:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.526+0000 7fcd78ddf700 1 -- 172.21.15.7:0/1431976118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcd740b1700 con 0x7fcd740ce510 2024-01-27T02:39:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.529+0000 7fcd6a7fc700 1 -- 172.21.15.7:0/1431976118 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcd5c056750 con 0x7fcd740ce510 2024-01-27T02:39:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.532+0000 7fcd78ddf700 1 -- 172.21.15.7:0/1431976118 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcd54066f30 msgr2=0x7fcd540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.532+0000 7fcd78ddf700 1 --2- 172.21.15.7:0/1431976118 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcd54066f30 0x7fcd540693e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009ca0 tx=0x7fcd64008040 comp rx=0 tx=0).stop 2024-01-27T02:39:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.532+0000 7fcd78ddf700 1 -- 172.21.15.7:0/1431976118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd740ce510 msgr2=0x7fcd74148640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.532+0000 7fcd78ddf700 1 --2- 172.21.15.7:0/1431976118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd740ce510 0x7fcd74148640 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fcd740700c0 tx=0x7fcd5c017000 comp rx=0 tx=0).stop 2024-01-27T02:39:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.532+0000 7fcd78ddf700 1 -- 172.21.15.7:0/1431976118 shutdown_connections 2024-01-27T02:39:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.532+0000 7fcd78ddf700 1 --2- 172.21.15.7:0/1431976118 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcd54066f30 0x7fcd540693e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:16.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.532+0000 7fcd78ddf700 1 --2- 172.21.15.7:0/1431976118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd74157c80 0x7fcd740af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:16.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.532+0000 7fcd78ddf700 1 --2- 172.21.15.7:0/1431976118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd74063580 0x7fcd7414d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:16.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.532+0000 7fcd78ddf700 1 --2- 172.21.15.7:0/1431976118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd740ce510 0x7fcd74148640 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:16.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.532+0000 7fcd78ddf700 1 -- 172.21.15.7:0/1431976118 >> 172.21.15.7:0/1431976118 conn(0x7fcd74068480 msgr2=0x7fcd7414b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:16.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.532+0000 7fcd78ddf700 1 -- 172.21.15.7:0/1431976118 shutdown_connections 2024-01-27T02:39:16.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.532+0000 7fcd78ddf700 1 -- 172.21.15.7:0/1431976118 wait complete. 2024-01-27T02:39:16.546 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-27T02:39:16.546 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:39:16.590 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:39:16.638 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-27T02:39:16.638 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-27T02:39:16.638 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:39:16.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.772+0000 7f1b37a20700 1 Processor -- start 2024-01-27T02:39:16.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.773+0000 7f1b37a20700 1 -- start start 2024-01-27T02:39:16.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.774+0000 7f1b37a20700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1b30143520 0x7f1b30063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:16.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.774+0000 7f1b37a20700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b301438f0 0x7f1b30063ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:16.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.774+0000 7f1b37a20700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1b300aa8b0 0x7f1b300c8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:16.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.774+0000 7f1b37a20700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b300cb7a0 con 0x7f1b300aa8b0 2024-01-27T02:39:16.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.774+0000 7f1b37a20700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b300cb9f0 con 0x7f1b30143520 2024-01-27T02:39:16.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.774+0000 7f1b37a20700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b300cbcd0 con 0x7f1b301438f0 2024-01-27T02:39:16.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.774+0000 7f1b35fbd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1b300aa8b0 0x7f1b300c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:16.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.774+0000 7f1b35fbd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1b300aa8b0 0x7f1b300c8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48086/0 (socket says 172.21.15.7:48086) 2024-01-27T02:39:16.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.774+0000 7f1b35fbd700 1 -- 172.21.15.7:0/4226470773 learned_addr learned my addr 172.21.15.7:0/4226470773 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:16.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.774+0000 7f1b34fbb700 1 --2- 172.21.15.7:0/4226470773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b301438f0 0x7f1b30063ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:16.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.774+0000 7f1b357bc700 1 --2- 172.21.15.7:0/4226470773 >> v2:172.21.15.188:3300/0 conn(0x7f1b30143520 0x7f1b30063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:16.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.774+0000 7f1b35fbd700 1 -- 172.21.15.7:0/4226470773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b301438f0 msgr2=0x7f1b30063ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:16.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.774+0000 7f1b35fbd700 1 --2- 172.21.15.7:0/4226470773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b301438f0 0x7f1b30063ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:16.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.774+0000 7f1b35fbd700 1 -- 172.21.15.7:0/4226470773 >> v2:172.21.15.188:3300/0 conn(0x7f1b30143520 msgr2=0x7f1b30063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:16.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.774+0000 7f1b35fbd700 1 --2- 172.21.15.7:0/4226470773 >> v2:172.21.15.188:3300/0 conn(0x7f1b30143520 0x7f1b30063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:16.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.774+0000 7f1b35fbd700 1 -- 172.21.15.7:0/4226470773 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b300cbfb0 con 0x7f1b300aa8b0 2024-01-27T02:39:16.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.774+0000 7f1b35fbd700 1 --2- 172.21.15.7:0/4226470773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b300aa8b0 0x7f1b300c8210 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f1b300cece0 tx=0x7f1b28009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83da5a1cd93cf487 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:16.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.775+0000 7f1b27fff700 1 -- 172.21.15.7:0/4226470773 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2800e070 con 0x7f1b300aa8b0 2024-01-27T02:39:16.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.775+0000 7f1b27fff700 1 -- 172.21.15.7:0/4226470773 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2800ba50 con 0x7f1b300aa8b0 2024-01-27T02:39:16.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.775+0000 7f1b27fff700 1 -- 172.21.15.7:0/4226470773 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b280092e0 con 0x7f1b300aa8b0 2024-01-27T02:39:16.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.775+0000 7f1b37a20700 1 -- 172.21.15.7:0/4226470773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b300aa8b0 msgr2=0x7f1b300c8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:16.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.775+0000 7f1b37a20700 1 --2- 172.21.15.7:0/4226470773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b300aa8b0 0x7f1b300c8210 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f1b300cece0 tx=0x7f1b28009840 comp rx=0 tx=0).stop 2024-01-27T02:39:16.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.775+0000 7f1b37a20700 1 -- 172.21.15.7:0/4226470773 shutdown_connections 2024-01-27T02:39:16.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.775+0000 7f1b37a20700 1 --2- 172.21.15.7:0/4226470773 >> v2:172.21.15.188:3300/0 conn(0x7f1b30143520 0x7f1b30063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:16.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.775+0000 7f1b37a20700 1 --2- 172.21.15.7:0/4226470773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b300aa8b0 0x7f1b300c8210 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:16.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.775+0000 7f1b37a20700 1 --2- 172.21.15.7:0/4226470773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b301438f0 0x7f1b30063ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:16.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.775+0000 7f1b37a20700 1 -- 172.21.15.7:0/4226470773 >> 172.21.15.7:0/4226470773 conn(0x7f1b300cdbd0 msgr2=0x7f1b300cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:16.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.775+0000 7f1b37a20700 1 -- 172.21.15.7:0/4226470773 shutdown_connections 2024-01-27T02:39:16.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.775+0000 7f1b37a20700 1 -- 172.21.15.7:0/4226470773 wait complete. 2024-01-27T02:39:16.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.777+0000 7f1b37a20700 1 Processor -- start 2024-01-27T02:39:16.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.777+0000 7f1b37a20700 1 -- start start 2024-01-27T02:39:16.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.778+0000 7f1b37a20700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b300aa8b0 0x7f1b301598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:16.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.778+0000 7f1b37a20700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b30143520 0x7f1b300b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:16.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.778+0000 7f1b37a20700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b301438f0 0x7f1b300ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:16.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.778+0000 7f1b37a20700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b300b33a0 con 0x7f1b300aa8b0 2024-01-27T02:39:16.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.778+0000 7f1b37a20700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b300b2790 con 0x7f1b30143520 2024-01-27T02:39:16.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.778+0000 7f1b37a20700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b300b2a70 con 0x7f1b301438f0 2024-01-27T02:39:16.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.778+0000 7f1b357bc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b300aa8b0 0x7f1b301598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:16.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.778+0000 7f1b357bc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b300aa8b0 0x7f1b301598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48088/0 (socket says 172.21.15.7:48088) 2024-01-27T02:39:16.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.778+0000 7f1b357bc700 1 -- 172.21.15.7:0/2390776968 learned_addr learned my addr 172.21.15.7:0/2390776968 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:16.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.778+0000 7f1b35fbd700 1 --2- 172.21.15.7:0/2390776968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b301438f0 0x7f1b300ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:16.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.778+0000 7f1b34fbb700 1 --2- 172.21.15.7:0/2390776968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b30143520 0x7f1b300b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:16.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.778+0000 7f1b357bc700 1 -- 172.21.15.7:0/2390776968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b301438f0 msgr2=0x7f1b300ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:16.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.778+0000 7f1b357bc700 1 --2- 172.21.15.7:0/2390776968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b301438f0 0x7f1b300ca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:16.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.778+0000 7f1b357bc700 1 -- 172.21.15.7:0/2390776968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b30143520 msgr2=0x7f1b300b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:16.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.778+0000 7f1b357bc700 1 --2- 172.21.15.7:0/2390776968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b30143520 0x7f1b300b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:16.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.778+0000 7f1b357bc700 1 -- 172.21.15.7:0/2390776968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b28009ae0 con 0x7f1b300aa8b0 2024-01-27T02:39:16.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.778+0000 7f1b35fbd700 1 --2- 172.21.15.7:0/2390776968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b301438f0 0x7f1b300ca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:16.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.778+0000 7f1b34fbb700 1 --2- 172.21.15.7:0/2390776968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b30143520 0x7f1b300b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:16.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.778+0000 7f1b357bc700 1 --2- 172.21.15.7:0/2390776968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b300aa8b0 0x7f1b301598d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00d810 tx=0x7f1b2c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:16.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.778+0000 7f1b257fa700 1 -- 172.21.15.7:0/2390776968 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c0125f0 con 0x7f1b300aa8b0 2024-01-27T02:39:16.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.779+0000 7f1b257fa700 1 -- 172.21.15.7:0/2390776968 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2c012750 con 0x7f1b300aa8b0 2024-01-27T02:39:16.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.779+0000 7f1b37a20700 1 -- 172.21.15.7:0/2390776968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b300b2d50 con 0x7f1b300aa8b0 2024-01-27T02:39:16.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.779+0000 7f1b257fa700 1 -- 172.21.15.7:0/2390776968 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c012b10 con 0x7f1b300aa8b0 2024-01-27T02:39:16.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.779+0000 7f1b37a20700 1 -- 172.21.15.7:0/2390776968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b300b23f0 con 0x7f1b300aa8b0 2024-01-27T02:39:16.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.780+0000 7f1b37a20700 1 -- 172.21.15.7:0/2390776968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b30159e60 con 0x7f1b300aa8b0 2024-01-27T02:39:16.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.780+0000 7f1b257fa700 1 -- 172.21.15.7:0/2390776968 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1b2c01e460 con 0x7f1b300aa8b0 2024-01-27T02:39:16.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.781+0000 7f1b257fa700 1 --2- 172.21.15.7:0/2390776968 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1b14066f30 0x7f1b140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:16.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.781+0000 7f1b257fa700 1 -- 172.21.15.7:0/2390776968 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1b2c0848c0 con 0x7f1b300aa8b0 2024-01-27T02:39:16.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.781+0000 7f1b34fbb700 1 --2- 172.21.15.7:0/2390776968 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1b14066f30 0x7f1b140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:16.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.781+0000 7f1b34fbb700 1 --2- 172.21.15.7:0/2390776968 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1b14066f30 0x7f1b140693e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1b3015fb50 tx=0x7f1b20008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:16.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:16.785+0000 7f1b257fa700 1 -- 172.21.15.7:0/2390776968 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b2c051e30 con 0x7f1b300aa8b0 2024-01-27T02:39:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.031+0000 7f1b37a20700 1 -- 172.21.15.7:0/2390776968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b3004c710 con 0x7f1b300aa8b0 2024-01-27T02:39:17.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.031+0000 7f1b257fa700 1 -- 172.21.15.7:0/2390776968 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+3523 (secure 0 0 0) 0x7f1b2c055450 con 0x7f1b300aa8b0 2024-01-27T02:39:17.032 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:17.032 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":70,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3997828193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3997828193},{"type":"v1","addr":"172.21.15.188:6837","nonce":3997828193}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1132338302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1132338302},{"type":"v1","addr":"172.21.15.188:6836","nonce":1132338302}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":7020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/740290645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":740290645},{"type":"v1","addr":"172.21.15.7:6833","nonce":740290645}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":7353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/903619853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":903619853},{"type":"v1","addr":"172.21.15.7:6835","nonce":903619853}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70}],"filesystems":[]} 2024-01-27T02:39:17.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.033+0000 7f1b37a20700 1 -- 172.21.15.7:0/2390776968 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1b14066f30 msgr2=0x7f1b140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:17.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.033+0000 7f1b37a20700 1 --2- 172.21.15.7:0/2390776968 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1b14066f30 0x7f1b140693e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1b3015fb50 tx=0x7f1b20008040 comp rx=0 tx=0).stop 2024-01-27T02:39:17.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.033+0000 7f1b37a20700 1 -- 172.21.15.7:0/2390776968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b300aa8b0 msgr2=0x7f1b301598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:17.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.033+0000 7f1b37a20700 1 --2- 172.21.15.7:0/2390776968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b300aa8b0 0x7f1b301598d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00d810 tx=0x7f1b2c008040 comp rx=0 tx=0).stop 2024-01-27T02:39:17.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.033+0000 7f1b37a20700 1 -- 172.21.15.7:0/2390776968 shutdown_connections 2024-01-27T02:39:17.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.033+0000 7f1b37a20700 1 --2- 172.21.15.7:0/2390776968 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1b14066f30 0x7f1b140693e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.033+0000 7f1b37a20700 1 --2- 172.21.15.7:0/2390776968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b301438f0 0x7f1b300ca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.033+0000 7f1b37a20700 1 --2- 172.21.15.7:0/2390776968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b300aa8b0 0x7f1b301598d0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.033+0000 7f1b37a20700 1 --2- 172.21.15.7:0/2390776968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b30143520 0x7f1b300b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.033+0000 7f1b37a20700 1 -- 172.21.15.7:0/2390776968 >> 172.21.15.7:0/2390776968 conn(0x7f1b300cdbd0 msgr2=0x7f1b300c2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.033+0000 7f1b37a20700 1 -- 172.21.15.7:0/2390776968 shutdown_connections 2024-01-27T02:39:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.033+0000 7f1b37a20700 1 -- 172.21.15.7:0/2390776968 wait complete. 2024-01-27T02:39:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 70 2024-01-27T02:39:17.047 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-27T02:39:17.048 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-27T02:39:17.048 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-27T02:39:17.048 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:39:17.048 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-27T02:39:17.048 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-27T02:39:17.048 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-27T02:39:17.048 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:39:17.049 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-27T02:39:17.049 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-27T02:39:17.049 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-27T02:39:17.049 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:39:17.049 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-27T02:39:17.049 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-27T02:39:17.049 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-27T02:39:17.050 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:39:17.057 INFO:tasks.ceph.mds.a:Stopped 2024-01-27T02:39:17.057 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-27T02:39:17.058 INFO:tasks.ceph.mds.c:Stopped 2024-01-27T02:39:17.058 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-27T02:39:17.060 INFO:tasks.ceph.mds.b:Stopped 2024-01-27T02:39:17.060 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-27T02:39:17.061 INFO:tasks.ceph.mds.d:Stopped 2024-01-27T02:39:17.061 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-27T02:39:17.100 INFO:tasks.ceph.mds.a:Started 2024-01-27T02:39:17.100 DEBUG:teuthology.parallel:result is None 2024-01-27T02:39:17.100 INFO:tasks.ceph.mds.c:Started 2024-01-27T02:39:17.101 DEBUG:teuthology.parallel:result is None 2024-01-27T02:39:17.103 INFO:tasks.ceph.mds.b:Started 2024-01-27T02:39:17.103 DEBUG:teuthology.parallel:result is None 2024-01-27T02:39:17.104 INFO:tasks.ceph.mds.d:Started 2024-01-27T02:39:17.104 DEBUG:teuthology.parallel:result is None 2024-01-27T02:39:17.104 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-27T02:39:17.187 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-27T02:39:17.187 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-27T02:39:17.187 INFO:tasks.ceph.mds.a.smithi007.stderr:2024-01-27T02:39:17.187+0000 7f3196993700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-01-27T02:39:17.192 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-01-27T02:39:17.192 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-01-27T02:39:17.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.244+0000 7f8f2ded9700 1 Processor -- start 2024-01-27T02:39:17.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.245+0000 7f8f2ded9700 1 -- start start 2024-01-27T02:39:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.245+0000 7f8f2ded9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8f280637d0 0x7f8f2806ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.245+0000 7f8f2ded9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f280c41e0 0x7f8f2815a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.245+0000 7f8f2ded9700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8f2815ac80 0x7f8f280c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.245+0000 7f8f2ded9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f2815b6c0 con 0x7f8f280637d0 2024-01-27T02:39:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.245+0000 7f8f2ded9700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f2815b100 con 0x7f8f2815ac80 2024-01-27T02:39:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.245+0000 7f8f2ded9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2815b3e0 con 0x7f8f280c41e0 2024-01-27T02:39:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.246+0000 7f8f277fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8f280637d0 0x7f8f2806ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.246+0000 7f8f277fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8f280637d0 0x7f8f2806ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48108/0 (socket says 172.21.15.7:48108) 2024-01-27T02:39:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.246+0000 7f8f277fe700 1 -- 172.21.15.7:0/2702772418 learned_addr learned my addr 172.21.15.7:0/2702772418 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.246+0000 7f8f26ffd700 1 --2- 172.21.15.7:0/2702772418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f280c41e0 0x7f8f2815a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.246+0000 7f8f27fff700 1 --2- 172.21.15.7:0/2702772418 >> v2:172.21.15.188:3300/0 conn(0x7f8f2815ac80 0x7f8f280c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.246+0000 7f8f277fe700 1 -- 172.21.15.7:0/2702772418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f280c41e0 msgr2=0x7f8f2815a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.246+0000 7f8f277fe700 1 --2- 172.21.15.7:0/2702772418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f280c41e0 0x7f8f2815a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.246+0000 7f8f277fe700 1 -- 172.21.15.7:0/2702772418 >> v2:172.21.15.188:3300/0 conn(0x7f8f2815ac80 msgr2=0x7f8f280c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.246+0000 7f8f277fe700 1 --2- 172.21.15.7:0/2702772418 >> v2:172.21.15.188:3300/0 conn(0x7f8f2815ac80 0x7f8f280c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.246+0000 7f8f277fe700 1 -- 172.21.15.7:0/2702772418 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2806e3d0 con 0x7f8f280637d0 2024-01-27T02:39:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.246+0000 7f8f277fe700 1 --2- 172.21.15.7:0/2702772418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f280637d0 0x7f8f2806ebd0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f8f280c8200 tx=0x7f8f18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0f39c7d2fa7562 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.246+0000 7f8f267fc700 1 -- 172.21.15.7:0/2702772418 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1800b640 con 0x7f8f280637d0 2024-01-27T02:39:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.246+0000 7f8f267fc700 1 -- 172.21.15.7:0/2702772418 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1800b7a0 con 0x7f8f280637d0 2024-01-27T02:39:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.246+0000 7f8f267fc700 1 -- 172.21.15.7:0/2702772418 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1800bb60 con 0x7f8f280637d0 2024-01-27T02:39:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.247+0000 7f8f2ded9700 1 -- 172.21.15.7:0/2702772418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f280637d0 msgr2=0x7f8f2806ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.247+0000 7f8f2ded9700 1 --2- 172.21.15.7:0/2702772418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f280637d0 0x7f8f2806ebd0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f8f280c8200 tx=0x7f8f18009250 comp rx=0 tx=0).stop 2024-01-27T02:39:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.247+0000 7f8f2ded9700 1 -- 172.21.15.7:0/2702772418 shutdown_connections 2024-01-27T02:39:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.247+0000 7f8f2ded9700 1 --2- 172.21.15.7:0/2702772418 >> v2:172.21.15.188:3300/0 conn(0x7f8f2815ac80 0x7f8f280c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.247+0000 7f8f2ded9700 1 --2- 172.21.15.7:0/2702772418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f280637d0 0x7f8f2806ebd0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.247+0000 7f8f2ded9700 1 --2- 172.21.15.7:0/2702772418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f280c41e0 0x7f8f2815a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.247+0000 7f8f2ded9700 1 -- 172.21.15.7:0/2702772418 >> 172.21.15.7:0/2702772418 conn(0x7f8f280cdc30 msgr2=0x7f8f280ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.247+0000 7f8f2ded9700 1 -- 172.21.15.7:0/2702772418 shutdown_connections 2024-01-27T02:39:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.247+0000 7f8f2ded9700 1 -- 172.21.15.7:0/2702772418 wait complete. 2024-01-27T02:39:17.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.248+0000 7f8f2ded9700 1 Processor -- start 2024-01-27T02:39:17.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.249+0000 7f8f2ded9700 1 -- start start 2024-01-27T02:39:17.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.249+0000 7f8f2ded9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f280637d0 0x7f8f281511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:17.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.249+0000 7f8f2ded9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f280c41e0 0x7f8f280b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:17.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.249+0000 7f8f2ded9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f2816dbc0 0x7f8f2816e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:17.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.249+0000 7f8f2ded9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f28152da0 con 0x7f8f2816dbc0 2024-01-27T02:39:17.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.249+0000 7f8f2ded9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f28153360 con 0x7f8f280c41e0 2024-01-27T02:39:17.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.249+0000 7f8f2ded9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f281516f0 con 0x7f8f280637d0 2024-01-27T02:39:17.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.249+0000 7f8f27fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f2816dbc0 0x7f8f2816e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:17.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.249+0000 7f8f27fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f2816dbc0 0x7f8f2816e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48120/0 (socket says 172.21.15.7:48120) 2024-01-27T02:39:17.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.249+0000 7f8f27fff700 1 -- 172.21.15.7:0/930937970 learned_addr learned my addr 172.21.15.7:0/930937970 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:17.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.250+0000 7f8f26ffd700 1 --2- 172.21.15.7:0/930937970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f280c41e0 0x7f8f280b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:17.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.250+0000 7f8f277fe700 1 --2- 172.21.15.7:0/930937970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f280637d0 0x7f8f281511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:17.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.250+0000 7f8f27fff700 1 -- 172.21.15.7:0/930937970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f280637d0 msgr2=0x7f8f281511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:17.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.250+0000 7f8f27fff700 1 --2- 172.21.15.7:0/930937970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f280637d0 0x7f8f281511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:17.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.250+0000 7f8f27fff700 1 -- 172.21.15.7:0/930937970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f280c41e0 msgr2=0x7f8f280b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:17.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.250+0000 7f8f27fff700 1 --2- 172.21.15.7:0/930937970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f280c41e0 0x7f8f280b09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:17.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.250+0000 7f8f27fff700 1 -- 172.21.15.7:0/930937970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f180094f0 con 0x7f8f2816dbc0 2024-01-27T02:39:17.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.250+0000 7f8f27fff700 1 --2- 172.21.15.7:0/930937970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f2816dbc0 0x7f8f2816e010 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009e50 tx=0x7f8f2000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:17.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.250+0000 7f8f1f7fe700 1 -- 172.21.15.7:0/930937970 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2000caa0 con 0x7f8f2816dbc0 2024-01-27T02:39:17.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.250+0000 7f8f1f7fe700 1 -- 172.21.15.7:0/930937970 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2000f040 con 0x7f8f2816dbc0 2024-01-27T02:39:17.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.250+0000 7f8f2ded9700 1 -- 172.21.15.7:0/930937970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f2816e640 con 0x7f8f2816dbc0 2024-01-27T02:39:17.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.250+0000 7f8f1f7fe700 1 -- 172.21.15.7:0/930937970 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2000ce70 con 0x7f8f2816dbc0 2024-01-27T02:39:17.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.250+0000 7f8f2ded9700 1 -- 172.21.15.7:0/930937970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f28152ff0 con 0x7f8f2816dbc0 2024-01-27T02:39:17.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.251+0000 7f8f2ded9700 1 -- 172.21.15.7:0/930937970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f2804c710 con 0x7f8f2816dbc0 2024-01-27T02:39:17.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.252+0000 7f8f1f7fe700 1 -- 172.21.15.7:0/930937970 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8f20014690 con 0x7f8f2816dbc0 2024-01-27T02:39:17.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.252+0000 7f8f1f7fe700 1 --2- 172.21.15.7:0/930937970 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f08066f30 0x7f8f080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:17.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.252+0000 7f8f1f7fe700 1 -- 172.21.15.7:0/930937970 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8f20084a60 con 0x7f8f2816dbc0 2024-01-27T02:39:17.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.252+0000 7f8f277fe700 1 --2- 172.21.15.7:0/930937970 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f08066f30 0x7f8f080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:17.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.253+0000 7f8f277fe700 1 --2- 172.21.15.7:0/930937970 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f08066f30 0x7f8f080693e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8f280c8200 tx=0x7f8f18017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:17.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.257+0000 7f8f1f7fe700 1 -- 172.21.15.7:0/930937970 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f20055a60 con 0x7f8f2816dbc0 2024-01-27T02:39:17.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:17.479+0000 7f8f2ded9700 1 -- 172.21.15.7:0/930937970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8f281534a0 con 0x7f8f2816dbc0 2024-01-27T02:39:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.164+0000 7f8f1f7fe700 1 -- 172.21.15.7:0/930937970 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v143) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8f200555f0 con 0x7f8f2816dbc0 2024-01-27T02:39:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.167+0000 7f8f2ded9700 1 -- 172.21.15.7:0/930937970 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f08066f30 msgr2=0x7f8f080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.167+0000 7f8f2ded9700 1 --2- 172.21.15.7:0/930937970 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f08066f30 0x7f8f080693e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8f280c8200 tx=0x7f8f18017000 comp rx=0 tx=0).stop 2024-01-27T02:39:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.167+0000 7f8f2ded9700 1 -- 172.21.15.7:0/930937970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f2816dbc0 msgr2=0x7f8f2816e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.167+0000 7f8f2ded9700 1 --2- 172.21.15.7:0/930937970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f2816dbc0 0x7f8f2816e010 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009e50 tx=0x7f8f2000b040 comp rx=0 tx=0).stop 2024-01-27T02:39:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.168+0000 7f8f2ded9700 1 -- 172.21.15.7:0/930937970 shutdown_connections 2024-01-27T02:39:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.168+0000 7f8f2ded9700 1 --2- 172.21.15.7:0/930937970 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f08066f30 0x7f8f080693e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.168+0000 7f8f2ded9700 1 --2- 172.21.15.7:0/930937970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f2816dbc0 0x7f8f2816e010 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.168+0000 7f8f2ded9700 1 --2- 172.21.15.7:0/930937970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f280637d0 0x7f8f281511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.168+0000 7f8f2ded9700 1 --2- 172.21.15.7:0/930937970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f280c41e0 0x7f8f280b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.168+0000 7f8f2ded9700 1 -- 172.21.15.7:0/930937970 >> 172.21.15.7:0/930937970 conn(0x7f8f280cdc30 msgr2=0x7f8f280cbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.168+0000 7f8f2ded9700 1 -- 172.21.15.7:0/930937970 shutdown_connections 2024-01-27T02:39:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.168+0000 7f8f2ded9700 1 -- 172.21.15.7:0/930937970 wait complete. 2024-01-27T02:39:18.169 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-01-27T02:39:18.185 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-27T02:39:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.320+0000 7f5a73886700 1 Processor -- start 2024-01-27T02:39:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.320+0000 7f5a73886700 1 -- start start 2024-01-27T02:39:18.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.321+0000 7f5a73886700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a6c0b0d40 0x7f5a6c0cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:18.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.321+0000 7f5a73886700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5a6c0b69b0 0x7f5a6c0d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:18.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.321+0000 7f5a73886700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a6c162560 con 0x7f5a6c0b1110 2024-01-27T02:39:18.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.321+0000 7f5a73886700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a6c163100 con 0x7f5a6c0b69b0 2024-01-27T02:39:18.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.321+0000 7f5a73886700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c0cde50 con 0x7f5a6c0b0d40 2024-01-27T02:39:18.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.321+0000 7f5a71e23700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5a6c0b69b0 0x7f5a6c0d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:18.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.321+0000 7f5a70e21700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5a6c0b1110 0x7f5a6c1629f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50868/0 (socket says 172.21.15.7:50868) 2024-01-27T02:39:18.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.322+0000 7f5a71e23700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5a6c0b69b0 0x7f5a6c0d0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48122/0 (socket says 172.21.15.7:48122) 2024-01-27T02:39:18.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.322+0000 7f5a70e21700 1 -- 172.21.15.7:0/3254195986 learned_addr learned my addr 172.21.15.7:0/3254195986 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:18.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.322+0000 7f5a71622700 1 --2- 172.21.15.7:0/3254195986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a6c0b0d40 0x7f5a6c0cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:18.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.322+0000 7f5a71e23700 1 -- 172.21.15.7:0/3254195986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a6c0b0d40 msgr2=0x7f5a6c0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:18.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.322+0000 7f5a71e23700 1 --2- 172.21.15.7:0/3254195986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a6c0b0d40 0x7f5a6c0cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:18.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.322+0000 7f5a71e23700 1 -- 172.21.15.7:0/3254195986 >> v1:172.21.15.7:6789/0 conn(0x7f5a6c0b1110 legacy=0x7f5a6c1629f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:18.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.322+0000 7f5a63fff700 1 -- 172.21.15.7:0/3254195986 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3161099431 0 0) 0x7f5a6c162560 con 0x7f5a6c0b1110 2024-01-27T02:39:18.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.322+0000 7f5a71e23700 1 -- 172.21.15.7:0/3254195986 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6c0ce0f0 con 0x7f5a6c0b69b0 2024-01-27T02:39:18.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.322+0000 7f5a71622700 1 --2- 172.21.15.7:0/3254195986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a6c0b0d40 0x7f5a6c0cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:18.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.322+0000 7f5a71e23700 1 --2- 172.21.15.7:0/3254195986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a6c0b69b0 0x7f5a6c0d0e40 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009e60 tx=0x7f5a64009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e0086f1f13daed server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:18.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.322+0000 7f5a63fff700 1 -- 172.21.15.7:0/3254195986 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6400e070 con 0x7f5a6c0b69b0 2024-01-27T02:39:18.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.322+0000 7f5a63fff700 1 -- 172.21.15.7:0/3254195986 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6400bc30 con 0x7f5a6c0b69b0 2024-01-27T02:39:18.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.322+0000 7f5a63fff700 1 -- 172.21.15.7:0/3254195986 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a64012430 con 0x7f5a6c0b69b0 2024-01-27T02:39:18.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.323+0000 7f5a73886700 1 -- 172.21.15.7:0/3254195986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a6c0b69b0 msgr2=0x7f5a6c0d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:18.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.323+0000 7f5a73886700 1 --2- 172.21.15.7:0/3254195986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a6c0b69b0 0x7f5a6c0d0e40 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009e60 tx=0x7f5a64009780 comp rx=0 tx=0).stop 2024-01-27T02:39:18.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.323+0000 7f5a73886700 1 -- 172.21.15.7:0/3254195986 shutdown_connections 2024-01-27T02:39:18.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.323+0000 7f5a73886700 1 --2- 172.21.15.7:0/3254195986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a6c0b69b0 0x7f5a6c0d0e40 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:18.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.323+0000 7f5a73886700 1 --2- 172.21.15.7:0/3254195986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a6c0b0d40 0x7f5a6c0cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.323+0000 7f5a73886700 1 -- 172.21.15.7:0/3254195986 >> 172.21.15.7:0/3254195986 conn(0x7f5a6c0679b0 msgr2=0x7f5a6c067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.323+0000 7f5a73886700 1 -- 172.21.15.7:0/3254195986 shutdown_connections 2024-01-27T02:39:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.323+0000 7f5a73886700 1 -- 172.21.15.7:0/3254195986 wait complete. 2024-01-27T02:39:18.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.324+0000 7f5a73886700 1 Processor -- start 2024-01-27T02:39:18.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.324+0000 7f5a73886700 1 -- start start 2024-01-27T02:39:18.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.325+0000 7f5a73886700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a6c0b0d40 0x7f5a6c0c4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:18.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.325+0000 7f5a73886700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a6c0b1110 0x7f5a6c0c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:18.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.325+0000 7f5a73886700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a6c0b69b0 0x7f5a6c0c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.325+0000 7f5a73886700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c0c62a0 con 0x7f5a6c0b69b0 2024-01-27T02:39:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.325+0000 7f5a73886700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c0c98f0 con 0x7f5a6c0b0d40 2024-01-27T02:39:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.325+0000 7f5a73886700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c155620 con 0x7f5a6c0b1110 2024-01-27T02:39:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.325+0000 7f5a71e23700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a6c0b69b0 0x7f5a6c0c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.325+0000 7f5a71e23700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a6c0b69b0 0x7f5a6c0c8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48136/0 (socket says 172.21.15.7:48136) 2024-01-27T02:39:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.325+0000 7f5a71e23700 1 -- 172.21.15.7:0/998349155 learned_addr learned my addr 172.21.15.7:0/998349155 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.326+0000 7f5a70e21700 1 --2- 172.21.15.7:0/998349155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a6c0b1110 0x7f5a6c0c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.326+0000 7f5a71622700 1 --2- 172.21.15.7:0/998349155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a6c0b0d40 0x7f5a6c0c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.326+0000 7f5a71e23700 1 -- 172.21.15.7:0/998349155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a6c0b1110 msgr2=0x7f5a6c0c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.326+0000 7f5a71e23700 1 --2- 172.21.15.7:0/998349155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a6c0b1110 0x7f5a6c0c8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.326+0000 7f5a71e23700 1 -- 172.21.15.7:0/998349155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a6c0b0d40 msgr2=0x7f5a6c0c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.326+0000 7f5a71e23700 1 --2- 172.21.15.7:0/998349155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a6c0b0d40 0x7f5a6c0c4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.326+0000 7f5a71e23700 1 -- 172.21.15.7:0/998349155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a64009a20 con 0x7f5a6c0b69b0 2024-01-27T02:39:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.326+0000 7f5a71e23700 1 --2- 172.21.15.7:0/998349155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a6c0b69b0 0x7f5a6c0c8650 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0d0aa0 tx=0x7f5a64019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.326+0000 7f5a617fa700 1 -- 172.21.15.7:0/998349155 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a64012e50 con 0x7f5a6c0b69b0 2024-01-27T02:39:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.326+0000 7f5a617fa700 1 -- 172.21.15.7:0/998349155 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6401d630 con 0x7f5a6c0b69b0 2024-01-27T02:39:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.326+0000 7f5a73886700 1 -- 172.21.15.7:0/998349155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a6c0c9330 con 0x7f5a6c0b69b0 2024-01-27T02:39:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.326+0000 7f5a617fa700 1 -- 172.21.15.7:0/998349155 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6401d9f0 con 0x7f5a6c0b69b0 2024-01-27T02:39:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.326+0000 7f5a73886700 1 -- 172.21.15.7:0/998349155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a6c0c96c0 con 0x7f5a6c0b69b0 2024-01-27T02:39:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.327+0000 7f5a73886700 1 -- 172.21.15.7:0/998349155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a6c04c710 con 0x7f5a6c0b69b0 2024-01-27T02:39:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.328+0000 7f5a617fa700 1 -- 172.21.15.7:0/998349155 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5a64024070 con 0x7f5a6c0b69b0 2024-01-27T02:39:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.328+0000 7f5a617fa700 1 --2- 172.21.15.7:0/998349155 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5a50066f30 0x7f5a500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.328+0000 7f5a617fa700 1 -- 172.21.15.7:0/998349155 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5a64086990 con 0x7f5a6c0b69b0 2024-01-27T02:39:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.328+0000 7f5a71622700 1 --2- 172.21.15.7:0/998349155 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5a50066f30 0x7f5a500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.329+0000 7f5a71622700 1 --2- 172.21.15.7:0/998349155 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5a50066f30 0x7f5a500693e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5a68006f00 tx=0x7f5a68008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:18.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.333+0000 7f5a617fa700 1 -- 172.21.15.7:0/998349155 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a64053f50 con 0x7f5a6c0b69b0 2024-01-27T02:39:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.615+0000 7f5a73886700 1 -- 172.21.15.7:0/998349155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5a6c0b0490 con 0x7f5a6c0b69b0 2024-01-27T02:39:18.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.616+0000 7f5a617fa700 1 -- 172.21.15.7:0/998349155 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5a6402c090 con 0x7f5a6c0b69b0 2024-01-27T02:39:18.616 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:18.616 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:39:18.616 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-01-27T02:39:18.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:18.617 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-01-27T02:39:18.617 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlHdcPKBAAfhnh1I3ZSQEXWJXlXCs98Q==", 2024-01-27T02:39:18.617 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:39:18.617 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:39:18.617 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:39:18.617 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:39:18.617 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:39:18.617 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:18.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:18.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:18.617 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-01-27T02:39:18.618 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rl8eVYNRAAfP8rOqO50RpoNZ1+1j2DXQ==", 2024-01-27T02:39:18.618 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:39:18.618 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:39:18.618 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:39:18.618 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:39:18.618 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:39:18.618 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:18.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:18.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:18.618 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-01-27T02:39:18.618 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlJxRKMBAAS0j7/zZeqGglOxCK1pg0WA==", 2024-01-27T02:39:18.618 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:39:18.619 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:39:18.619 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:39:18.619 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:39:18.619 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:39:18.619 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:18.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:18.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:18.619 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-01-27T02:39:18.619 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlNwfUABAALRQAoHoGhniqCvTQOqbbug==", 2024-01-27T02:39:18.619 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:39:18.619 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:39:18.619 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:39:18.619 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:39:18.619 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:39:18.620 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:18.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:18.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:18.620 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-01-27T02:39:18.620 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7Rlh6VSJhAAttuCte1WhNEvUkv5x/IKQQ==", 2024-01-27T02:39:18.620 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:39:18.620 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:39:18.620 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:39:18.620 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:39:18.620 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:18.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:18.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:18.620 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-01-27T02:39:18.620 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7Rlu/RGLhAAKzaCNhzYKhsNhxkMvY++gw==", 2024-01-27T02:39:18.621 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:39:18.621 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:39:18.621 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:39:18.621 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:39:18.621 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:18.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:18.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:18.621 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-01-27T02:39:18.621 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7RliRMyNhAAr0tG3evmVV7ZlGxbxfWPMw==", 2024-01-27T02:39:18.621 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:39:18.621 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:39:18.621 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:39:18.621 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:39:18.621 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:18.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:18.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:18.622 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-01-27T02:39:18.622 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAPa7RlRuSNAhAA3e9atlb+ffvlnLhRYbWZPw==", 2024-01-27T02:39:18.622 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:39:18.622 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:39:18.622 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:39:18.622 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:39:18.622 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:18.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:18.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:18.622 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-01-27T02:39:18.622 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7RlfeA6DBAAXpklgbNHRO+5G+fdIERjRw==", 2024-01-27T02:39:18.622 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:39:18.622 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:39:18.623 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:39:18.623 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:39:18.623 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:18.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:18.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:18.623 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-01-27T02:39:18.623 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7Rl77ItExAABB7uABm63ubE/YUs2B5pGw==", 2024-01-27T02:39:18.623 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:39:18.623 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:39:18.623 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:39:18.623 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:39:18.623 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:18.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:18.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:18.624 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-01-27T02:39:18.624 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7RljDuwHBAAPPAlhoqS03qPpq58jmevLA==", 2024-01-27T02:39:18.624 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:39:18.624 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:39:18.624 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:39:18.625 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:39:18.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:18.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:18.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:18.625 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-01-27T02:39:18.625 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7Rl1YElJhAAqHP/QEdl1SEalGcbQwPwAg==", 2024-01-27T02:39:18.625 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:39:18.625 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:39:18.625 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:39:18.625 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:39:18.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:18.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:18.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:18.626 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-01-27T02:39:18.626 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlaEqaBRAATFiIeW8lT9BdNuN0m3x9dQ==", 2024-01-27T02:39:18.626 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:39:18.626 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:39:18.626 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-01-27T02:39:18.626 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-27T02:39:18.626 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:18.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:18.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:18.626 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-01-27T02:39:18.626 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlNADmEBAAJXwdJ35ylP5coy1dtHoD3g==", 2024-01-27T02:39:18.626 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:39:18.626 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:39:18.626 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:39:18.626 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:39:18.627 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:39:18.627 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:18.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:18.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:18.627 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-01-27T02:39:18.627 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7Rl5fRqCRAA/5swTGknnpP64TAh+2X41A==", 2024-01-27T02:39:18.627 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:39:18.627 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:39:18.627 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:39:18.627 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:39:18.627 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:39:18.627 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:18.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:18.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:18.627 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-01-27T02:39:18.628 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7Rl0SdwFhAAhCmw9ViiJbHVfS5a2r/D+w==", 2024-01-27T02:39:18.628 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:39:18.628 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:39:18.628 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:39:18.628 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:39:18.628 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:39:18.628 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:18.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:18.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:18.628 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-01-27T02:39:18.628 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlsNgPARAA0Qm0oJgA8GO1oUT3ndT8Tg==", 2024-01-27T02:39:18.628 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:39:18.628 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:39:18.629 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:39:18.629 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:39:18.629 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:39:18.629 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:18.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:18.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:18.629 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-01-27T02:39:18.629 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rlfb7qIxAAlsJYiSolJ7D8EykGWP62bA==", 2024-01-27T02:39:18.629 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:39:18.629 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:39:18.629 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-27T02:39:18.629 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:39:18.629 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:18.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:39:18.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:18.630 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-01-27T02:39:18.630 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rls7TaHxAALS3ghxKtfmgQfIELiD9+Bg==", 2024-01-27T02:39:18.630 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:39:18.630 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:39:18.630 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-27T02:39:18.630 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:39:18.630 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:18.630 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:18.630 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:39:18.630 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:39:18.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.618+0000 7f5a73886700 1 -- 172.21.15.7:0/998349155 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5a50066f30 msgr2=0x7f5a500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:18.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.618+0000 7f5a73886700 1 --2- 172.21.15.7:0/998349155 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5a50066f30 0x7f5a500693e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5a68006f00 tx=0x7f5a68008040 comp rx=0 tx=0).stop 2024-01-27T02:39:18.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.618+0000 7f5a73886700 1 -- 172.21.15.7:0/998349155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a6c0b69b0 msgr2=0x7f5a6c0c8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:18.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.618+0000 7f5a73886700 1 --2- 172.21.15.7:0/998349155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a6c0b69b0 0x7f5a6c0c8650 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0d0aa0 tx=0x7f5a64019000 comp rx=0 tx=0).stop 2024-01-27T02:39:18.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.618+0000 7f5a73886700 1 -- 172.21.15.7:0/998349155 shutdown_connections 2024-01-27T02:39:18.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.618+0000 7f5a73886700 1 --2- 172.21.15.7:0/998349155 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5a50066f30 0x7f5a500693e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.618+0000 7f5a73886700 1 --2- 172.21.15.7:0/998349155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a6c0b69b0 0x7f5a6c0c8650 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.618+0000 7f5a73886700 1 --2- 172.21.15.7:0/998349155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a6c0b0d40 0x7f5a6c0c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.618+0000 7f5a73886700 1 --2- 172.21.15.7:0/998349155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a6c0b1110 0x7f5a6c0c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.618+0000 7f5a73886700 1 -- 172.21.15.7:0/998349155 >> 172.21.15.7:0/998349155 conn(0x7f5a6c0679b0 msgr2=0x7f5a6c160f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.618+0000 7f5a73886700 1 -- 172.21.15.7:0/998349155 shutdown_connections 2024-01-27T02:39:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.618+0000 7f5a73886700 1 -- 172.21.15.7:0/998349155 wait complete. 2024-01-27T02:39:18.633 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:39:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.765+0000 7f2531898700 1 Processor -- start 2024-01-27T02:39:18.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.765+0000 7f2531898700 1 -- start start 2024-01-27T02:39:18.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.766+0000 7f2531898700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f252c0cd310 0x7f252c15dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.766+0000 7f2531898700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f252c06e5d0 con 0x7f252c15d3a0 2024-01-27T02:39:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.766+0000 7f2531898700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f252c0ce860 con 0x7f252c15d770 2024-01-27T02:39:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.766+0000 7f2531898700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f252c0cbe10 con 0x7f252c0cd310 2024-01-27T02:39:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.766+0000 7f252b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f252c0cd310 0x7f252c15dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.766+0000 7f252affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f252c15d3a0 0x7f252c0c4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50870/0 (socket says 172.21.15.7:50870) 2024-01-27T02:39:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.766+0000 7f252affd700 1 -- 172.21.15.7:0/1821225119 learned_addr learned my addr 172.21.15.7:0/1821225119 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.766+0000 7f252b7fe700 1 -- 172.21.15.7:0/1821225119 >> v1:172.21.15.188:6789/0 conn(0x7f252c15d770 legacy=0x7f252c06ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.766+0000 7f252b7fe700 1 -- 172.21.15.7:0/1821225119 >> v1:172.21.15.7:6789/0 conn(0x7f252c15d3a0 legacy=0x7f252c0c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.766+0000 7f252b7fe700 1 -- 172.21.15.7:0/1821225119 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252c0cd8f0 con 0x7f252c0cd310 2024-01-27T02:39:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.766+0000 7f2529ffb700 1 -- 172.21.15.7:0/1821225119 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 445349733 0 0) 0x7f252c06e5d0 con 0x7f252c15d3a0 2024-01-27T02:39:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.767+0000 7f252b7fe700 1 --2- 172.21.15.7:0/1821225119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c0cd310 0x7f252c15dc60 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f252c0cdae0 tx=0x7f2524009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1b1be81322c75 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.767+0000 7f2529ffb700 1 -- 172.21.15.7:0/1821225119 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252400e070 con 0x7f252c0cd310 2024-01-27T02:39:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.767+0000 7f2529ffb700 1 -- 172.21.15.7:0/1821225119 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252400bbf0 con 0x7f252c0cd310 2024-01-27T02:39:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.767+0000 7f2529ffb700 1 -- 172.21.15.7:0/1821225119 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2524012430 con 0x7f252c0cd310 2024-01-27T02:39:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.767+0000 7f2531898700 1 -- 172.21.15.7:0/1821225119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c0cd310 msgr2=0x7f252c15dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.767+0000 7f2531898700 1 --2- 172.21.15.7:0/1821225119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c0cd310 0x7f252c15dc60 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f252c0cdae0 tx=0x7f2524009780 comp rx=0 tx=0).stop 2024-01-27T02:39:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.767+0000 7f2531898700 1 -- 172.21.15.7:0/1821225119 shutdown_connections 2024-01-27T02:39:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.767+0000 7f2531898700 1 --2- 172.21.15.7:0/1821225119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c0cd310 0x7f252c15dc60 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.767+0000 7f2531898700 1 -- 172.21.15.7:0/1821225119 >> 172.21.15.7:0/1821225119 conn(0x7f252c068480 msgr2=0x7f252c0cdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.767+0000 7f2531898700 1 -- 172.21.15.7:0/1821225119 shutdown_connections 2024-01-27T02:39:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.767+0000 7f2531898700 1 -- 172.21.15.7:0/1821225119 wait complete. 2024-01-27T02:39:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.769+0000 7f2531898700 1 Processor -- start 2024-01-27T02:39:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.769+0000 7f2531898700 1 -- start start 2024-01-27T02:39:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.770+0000 7f2531898700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252c0cd310 0x7f252c150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.770+0000 7f2531898700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f252c15d3a0 0x7f252c0aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.770+0000 7f2531898700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c15d770 0x7f252c0af470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.770+0000 7f2531898700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252c0b0980 con 0x7f252c15d770 2024-01-27T02:39:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.770+0000 7f2531898700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252c0b17e0 con 0x7f252c15d3a0 2024-01-27T02:39:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.770+0000 7f2531898700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252c0b0bd0 con 0x7f252c0cd310 2024-01-27T02:39:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.770+0000 7f252b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c15d770 0x7f252c0af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.770+0000 7f252b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c15d770 0x7f252c0af470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48160/0 (socket says 172.21.15.7:48160) 2024-01-27T02:39:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.770+0000 7f252b7fe700 1 -- 172.21.15.7:0/1213042148 learned_addr learned my addr 172.21.15.7:0/1213042148 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.770+0000 7f252affd700 1 --2- 172.21.15.7:0/1213042148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252c0cd310 0x7f252c150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.770+0000 7f252a7fc700 1 --2- 172.21.15.7:0/1213042148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f252c15d3a0 0x7f252c0aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.770+0000 7f252b7fe700 1 -- 172.21.15.7:0/1213042148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252c0cd310 msgr2=0x7f252c150600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.770+0000 7f252b7fe700 1 --2- 172.21.15.7:0/1213042148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252c0cd310 0x7f252c150600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.770+0000 7f252b7fe700 1 -- 172.21.15.7:0/1213042148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f252c15d3a0 msgr2=0x7f252c0aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.770+0000 7f252b7fe700 1 --2- 172.21.15.7:0/1213042148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f252c15d3a0 0x7f252c0aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.770+0000 7f252b7fe700 1 -- 172.21.15.7:0/1213042148 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2524009a20 con 0x7f252c15d770 2024-01-27T02:39:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.770+0000 7f252affd700 1 --2- 172.21.15.7:0/1213042148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252c0cd310 0x7f252c150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.770+0000 7f252a7fc700 1 --2- 172.21.15.7:0/1213042148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f252c15d3a0 0x7f252c0aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.770+0000 7f252b7fe700 1 --2- 172.21.15.7:0/1213042148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c15d770 0x7f252c0af470 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f2524009fd0 tx=0x7f2524019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:18.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.771+0000 7f2522ffd700 1 -- 172.21.15.7:0/1213042148 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252401d630 con 0x7f252c15d770 2024-01-27T02:39:18.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.771+0000 7f2522ffd700 1 -- 172.21.15.7:0/1213042148 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252401d790 con 0x7f252c15d770 2024-01-27T02:39:18.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.771+0000 7f2531898700 1 -- 172.21.15.7:0/1213042148 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f252c0b0eb0 con 0x7f252c15d770 2024-01-27T02:39:18.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.771+0000 7f2522ffd700 1 -- 172.21.15.7:0/1213042148 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252401db50 con 0x7f252c15d770 2024-01-27T02:39:18.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.771+0000 7f2531898700 1 -- 172.21.15.7:0/1213042148 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f252c0b1470 con 0x7f252c15d770 2024-01-27T02:39:18.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.772+0000 7f2531898700 1 -- 172.21.15.7:0/1213042148 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f252c0b1950 con 0x7f252c15d770 2024-01-27T02:39:18.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.772+0000 7f2522ffd700 1 -- 172.21.15.7:0/1213042148 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2524024070 con 0x7f252c15d770 2024-01-27T02:39:18.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.773+0000 7f2522ffd700 1 --2- 172.21.15.7:0/1213042148 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f250c066f30 0x7f250c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:18.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.773+0000 7f2522ffd700 1 -- 172.21.15.7:0/1213042148 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2524086b50 con 0x7f252c15d770 2024-01-27T02:39:18.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.773+0000 7f252affd700 1 --2- 172.21.15.7:0/1213042148 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f250c066f30 0x7f250c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:18.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.773+0000 7f252affd700 1 --2- 172.21.15.7:0/1213042148 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f250c066f30 0x7f250c0693e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f251c0011e0 tx=0x7f251c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:18.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.777+0000 7f2522ffd700 1 -- 172.21.15.7:0/1213042148 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2524054110 con 0x7f252c15d770 2024-01-27T02:39:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.983+0000 7f2531898700 1 -- 172.21.15.7:0/1213042148 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f252c04c710 con 0x7f252c15d770 2024-01-27T02:39:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.984+0000 7f2522ffd700 1 -- 172.21.15.7:0/1213042148 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+12097 (secure 0 0 0) 0x7f2524057730 con 0x7f252c15d770 2024-01-27T02:39:18.985 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:18.985 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":145,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:39:18.169618+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:39:18.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.986+0000 7f2531898700 1 -- 172.21.15.7:0/1213042148 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f250c066f30 msgr2=0x7f250c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:18.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.986+0000 7f2531898700 1 --2- 172.21.15.7:0/1213042148 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f250c066f30 0x7f250c0693e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f251c0011e0 tx=0x7f251c009040 comp rx=0 tx=0).stop 2024-01-27T02:39:18.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.986+0000 7f2531898700 1 -- 172.21.15.7:0/1213042148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c15d770 msgr2=0x7f252c0af470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:18.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.986+0000 7f2531898700 1 --2- 172.21.15.7:0/1213042148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c15d770 0x7f252c0af470 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f2524009fd0 tx=0x7f2524019000 comp rx=0 tx=0).stop 2024-01-27T02:39:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.986+0000 7f2531898700 1 -- 172.21.15.7:0/1213042148 shutdown_connections 2024-01-27T02:39:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.986+0000 7f2531898700 1 --2- 172.21.15.7:0/1213042148 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f250c066f30 0x7f250c0693e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.986+0000 7f2531898700 1 --2- 172.21.15.7:0/1213042148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c15d770 0x7f252c0af470 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.986+0000 7f2531898700 1 --2- 172.21.15.7:0/1213042148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252c0cd310 0x7f252c150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.986+0000 7f2531898700 1 --2- 172.21.15.7:0/1213042148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f252c15d3a0 0x7f252c0aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.986+0000 7f2531898700 1 -- 172.21.15.7:0/1213042148 >> 172.21.15.7:0/1213042148 conn(0x7f252c068480 msgr2=0x7f252c06ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.987+0000 7f2531898700 1 -- 172.21.15.7:0/1213042148 shutdown_connections 2024-01-27T02:39:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:18.987+0000 7f2531898700 1 -- 172.21.15.7:0/1213042148 wait complete. 2024-01-27T02:39:19.001 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-27T02:39:19.001 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:39:19.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.135+0000 7fe6ce043700 1 Processor -- start 2024-01-27T02:39:19.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.135+0000 7fe6ce043700 1 -- start start 2024-01-27T02:39:19.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.136+0000 7fe6ce043700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe6c80cba60 0x7fe6c80ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:19.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.136+0000 7fe6ce043700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe6c806b4e0 0x7fe6c806b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:19.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.136+0000 7fe6ce043700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6c80679b0 con 0x7fe6c80cbe30 2024-01-27T02:39:19.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.136+0000 7fe6ce043700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c8067c90 con 0x7fe6c806b4e0 2024-01-27T02:39:19.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.136+0000 7fe6ce043700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c806be20 con 0x7fe6c80cba60 2024-01-27T02:39:19.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.136+0000 7fe6c7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe6c806b4e0 0x7fe6c806b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:19.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.136+0000 7fe6c6ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe6c80cbe30 0x7fe6c815b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50884/0 (socket says 172.21.15.7:50884) 2024-01-27T02:39:19.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.136+0000 7fe6c6ffd700 1 -- 172.21.15.7:0/421270111 learned_addr learned my addr 172.21.15.7:0/421270111 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:19.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.136+0000 7fe6c77fe700 1 --2- 172.21.15.7:0/421270111 >> v2:172.21.15.188:3300/0 conn(0x7fe6c80cba60 0x7fe6c80ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:19.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.137+0000 7fe6c7fff700 1 -- 172.21.15.7:0/421270111 >> v2:172.21.15.188:3300/0 conn(0x7fe6c80cba60 msgr2=0x7fe6c80ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:19.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.137+0000 7fe6c7fff700 1 --2- 172.21.15.7:0/421270111 >> v2:172.21.15.188:3300/0 conn(0x7fe6c80cba60 0x7fe6c80ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.137+0000 7fe6c7fff700 1 -- 172.21.15.7:0/421270111 >> v1:172.21.15.7:6789/0 conn(0x7fe6c80cbe30 legacy=0x7fe6c815b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:19.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.137+0000 7fe6c7fff700 1 -- 172.21.15.7:0/421270111 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c80cca50 con 0x7fe6c806b4e0 2024-01-27T02:39:19.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.137+0000 7fe6c67fc700 1 -- 172.21.15.7:0/421270111 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2721259292 0 0) 0x7fe6c80679b0 con 0x7fe6c80cbe30 2024-01-27T02:39:19.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.137+0000 7fe6c77fe700 1 --2- 172.21.15.7:0/421270111 >> v2:172.21.15.188:3300/0 conn(0x7fe6c80cba60 0x7fe6c80ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:39:19.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.137+0000 7fe6c7fff700 1 --2- 172.21.15.7:0/421270111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c806b4e0 0x7fe6c806b8e0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80cfd50 tx=0x7fe6c0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1695460569fb66e8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:19.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.137+0000 7fe6c67fc700 1 -- 172.21.15.7:0/421270111 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c000e070 con 0x7fe6c806b4e0 2024-01-27T02:39:19.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.137+0000 7fe6c67fc700 1 -- 172.21.15.7:0/421270111 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c000bc30 con 0x7fe6c806b4e0 2024-01-27T02:39:19.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.137+0000 7fe6c67fc700 1 -- 172.21.15.7:0/421270111 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0012430 con 0x7fe6c806b4e0 2024-01-27T02:39:19.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.137+0000 7fe6ce043700 1 -- 172.21.15.7:0/421270111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c806b4e0 msgr2=0x7fe6c806b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:19.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.137+0000 7fe6ce043700 1 --2- 172.21.15.7:0/421270111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c806b4e0 0x7fe6c806b8e0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80cfd50 tx=0x7fe6c0009780 comp rx=0 tx=0).stop 2024-01-27T02:39:19.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.138+0000 7fe6ce043700 1 -- 172.21.15.7:0/421270111 shutdown_connections 2024-01-27T02:39:19.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.138+0000 7fe6ce043700 1 --2- 172.21.15.7:0/421270111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c806b4e0 0x7fe6c806b8e0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.138+0000 7fe6ce043700 1 --2- 172.21.15.7:0/421270111 >> v2:172.21.15.188:3300/0 conn(0x7fe6c80cba60 0x7fe6c80ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.138+0000 7fe6ce043700 1 -- 172.21.15.7:0/421270111 >> 172.21.15.7:0/421270111 conn(0x7fe6c8063580 msgr2=0x7fe6c8063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:19.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.138+0000 7fe6ce043700 1 -- 172.21.15.7:0/421270111 shutdown_connections 2024-01-27T02:39:19.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.138+0000 7fe6ce043700 1 -- 172.21.15.7:0/421270111 wait complete. 2024-01-27T02:39:19.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.139+0000 7fe6ce043700 1 Processor -- start 2024-01-27T02:39:19.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.139+0000 7fe6ce043700 1 -- start start 2024-01-27T02:39:19.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.140+0000 7fe6ce043700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6c806b4e0 0x7fe6c80aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:19.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.140+0000 7fe6ce043700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c80cba60 0x7fe6c8157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:19.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.140+0000 7fe6ce043700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6c80cbe30 0x7fe6c80b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:19.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.140+0000 7fe6ce043700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c80ab380 con 0x7fe6c80cba60 2024-01-27T02:39:19.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.140+0000 7fe6ce043700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c80af1f0 con 0x7fe6c80cbe30 2024-01-27T02:39:19.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.140+0000 7fe6ce043700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c80aaa50 con 0x7fe6c806b4e0 2024-01-27T02:39:19.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.140+0000 7fe6c6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c80cba60 0x7fe6c8157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:19.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.140+0000 7fe6c6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c80cba60 0x7fe6c8157970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48176/0 (socket says 172.21.15.7:48176) 2024-01-27T02:39:19.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.140+0000 7fe6c6ffd700 1 -- 172.21.15.7:0/901214015 learned_addr learned my addr 172.21.15.7:0/901214015 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:19.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.140+0000 7fe6c77fe700 1 --2- 172.21.15.7:0/901214015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6c806b4e0 0x7fe6c80aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:19.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.140+0000 7fe6c7fff700 1 --2- 172.21.15.7:0/901214015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6c80cbe30 0x7fe6c80b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:19.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.140+0000 7fe6c6ffd700 1 -- 172.21.15.7:0/901214015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6c806b4e0 msgr2=0x7fe6c80aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:19.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.140+0000 7fe6c6ffd700 1 --2- 172.21.15.7:0/901214015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6c806b4e0 0x7fe6c80aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.140+0000 7fe6c6ffd700 1 -- 172.21.15.7:0/901214015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6c80cbe30 msgr2=0x7fe6c80b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:19.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.140+0000 7fe6c6ffd700 1 --2- 172.21.15.7:0/901214015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6c80cbe30 0x7fe6c80b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.140+0000 7fe6c6ffd700 1 -- 172.21.15.7:0/901214015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c0009a20 con 0x7fe6c80cba60 2024-01-27T02:39:19.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.140+0000 7fe6c7fff700 1 --2- 172.21.15.7:0/901214015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6c80cbe30 0x7fe6c80b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.141+0000 7fe6c6ffd700 1 --2- 172.21.15.7:0/901214015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c80cba60 0x7fe6c8157970 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00c5e0 tx=0x7fe6bc000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.141+0000 7fe6af7fe700 1 -- 172.21.15.7:0/901214015 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc005b30 con 0x7fe6c80cba60 2024-01-27T02:39:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.141+0000 7fe6af7fe700 1 -- 172.21.15.7:0/901214015 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6bc005c90 con 0x7fe6c80cba60 2024-01-27T02:39:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.141+0000 7fe6ce043700 1 -- 172.21.15.7:0/901214015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c80aad30 con 0x7fe6c80cba60 2024-01-27T02:39:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.141+0000 7fe6af7fe700 1 -- 172.21.15.7:0/901214015 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc0072e0 con 0x7fe6c80cba60 2024-01-27T02:39:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.141+0000 7fe6ce043700 1 -- 172.21.15.7:0/901214015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c8157eb0 con 0x7fe6c80cba60 2024-01-27T02:39:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.142+0000 7fe6ce043700 1 -- 172.21.15.7:0/901214015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c80ab530 con 0x7fe6c80cba60 2024-01-27T02:39:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.142+0000 7fe6af7fe700 1 -- 172.21.15.7:0/901214015 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe6bc01b3b0 con 0x7fe6c80cba60 2024-01-27T02:39:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.143+0000 7fe6af7fe700 1 --2- 172.21.15.7:0/901214015 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe6b4066f30 0x7fe6b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.143+0000 7fe6af7fe700 1 -- 172.21.15.7:0/901214015 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe6bc083510 con 0x7fe6c80cba60 2024-01-27T02:39:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.143+0000 7fe6c77fe700 1 --2- 172.21.15.7:0/901214015 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe6b4066f30 0x7fe6b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.144+0000 7fe6c77fe700 1 --2- 172.21.15.7:0/901214015 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe6b4066f30 0x7fe6b40693e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800ad40 tx=0x7fe6b8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.147+0000 7fe6af7fe700 1 -- 172.21.15.7:0/901214015 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe6bc04b5a0 con 0x7fe6c80cba60 2024-01-27T02:39:19.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.352+0000 7fe6ce043700 1 -- 172.21.15.7:0/901214015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe6c8072060 con 0x7fe6c80cba60 2024-01-27T02:39:19.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.353+0000 7fe6af7fe700 1 -- 172.21.15.7:0/901214015 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe6bc017020 con 0x7fe6c80cba60 2024-01-27T02:39:19.354 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:39:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.355+0000 7fe6ce043700 1 -- 172.21.15.7:0/901214015 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe6b4066f30 msgr2=0x7fe6b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.355+0000 7fe6ce043700 1 --2- 172.21.15.7:0/901214015 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe6b4066f30 0x7fe6b40693e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800ad40 tx=0x7fe6b8008040 comp rx=0 tx=0).stop 2024-01-27T02:39:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.355+0000 7fe6ce043700 1 -- 172.21.15.7:0/901214015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c80cba60 msgr2=0x7fe6c8157970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.355+0000 7fe6ce043700 1 --2- 172.21.15.7:0/901214015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c80cba60 0x7fe6c8157970 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00c5e0 tx=0x7fe6bc000fe0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.355+0000 7fe6ce043700 1 -- 172.21.15.7:0/901214015 shutdown_connections 2024-01-27T02:39:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.355+0000 7fe6ce043700 1 --2- 172.21.15.7:0/901214015 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe6b4066f30 0x7fe6b40693e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.355+0000 7fe6ce043700 1 --2- 172.21.15.7:0/901214015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6c80cbe30 0x7fe6c80b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.355+0000 7fe6ce043700 1 --2- 172.21.15.7:0/901214015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6c806b4e0 0x7fe6c80aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.355+0000 7fe6ce043700 1 --2- 172.21.15.7:0/901214015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c80cba60 0x7fe6c8157970 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.355+0000 7fe6ce043700 1 -- 172.21.15.7:0/901214015 >> 172.21.15.7:0/901214015 conn(0x7fe6c8063580 msgr2=0x7fe6c81597e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.355+0000 7fe6ce043700 1 -- 172.21.15.7:0/901214015 shutdown_connections 2024-01-27T02:39:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.355+0000 7fe6ce043700 1 -- 172.21.15.7:0/901214015 wait complete. 2024-01-27T02:39:19.370 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-27T02:39:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.501+0000 7f7d440b6700 1 Processor -- start 2024-01-27T02:39:19.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.501+0000 7f7d440b6700 1 -- start start 2024-01-27T02:39:19.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.502+0000 7f7d440b6700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7d3c159f40 0x7f7d3c0c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:19.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.502+0000 7f7d440b6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d3c0c30c0 con 0x7f7d3c0c3a90 2024-01-27T02:39:19.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.502+0000 7f7d440b6700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d3c15e960 con 0x7f7d3c0aafc0 2024-01-27T02:39:19.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.502+0000 7f7d440b6700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d3c0c4550 con 0x7f7d3c159f40 2024-01-27T02:39:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.502+0000 7f7d41651700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7d3c0c3a90 0x7f7d3c0aa8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50896/0 (socket says 172.21.15.7:50896) 2024-01-27T02:39:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.502+0000 7f7d41651700 1 -- 172.21.15.7:0/2912323156 learned_addr learned my addr 172.21.15.7:0/2912323156 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.502+0000 7f7d41e52700 1 --2- 172.21.15.7:0/2912323156 >> v2:172.21.15.188:3300/0 conn(0x7f7d3c159f40 0x7f7d3c0c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.502+0000 7f7d40e50700 1 -- 172.21.15.7:0/2912323156 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3642140244 0 0) 0x7f7d3c0c30c0 con 0x7f7d3c0c3a90 2024-01-27T02:39:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.502+0000 7f7d40e50700 1 -- 172.21.15.7:0/2912323156 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d28003570 con 0x7f7d3c0c3a90 2024-01-27T02:39:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d40e50700 1 -- 172.21.15.7:0/2912323156 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2758976129 0 0) 0x7f7d28003570 con 0x7f7d3c0c3a90 2024-01-27T02:39:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d40e50700 1 -- 172.21.15.7:0/2912323156 >> v2:172.21.15.188:3300/0 conn(0x7f7d3c159f40 msgr2=0x7f7d3c0c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d40e50700 1 --2- 172.21.15.7:0/2912323156 >> v2:172.21.15.188:3300/0 conn(0x7f7d3c159f40 0x7f7d3c0c3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d40e50700 1 -- 172.21.15.7:0/2912323156 >> v1:172.21.15.188:6789/0 conn(0x7f7d3c0aafc0 legacy=0x7f7d3c063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d40e50700 1 -- 172.21.15.7:0/2912323156 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3c15ec40 con 0x7f7d3c0c3a90 2024-01-27T02:39:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d40e50700 1 -- 172.21.15.7:0/2912323156 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f7d2c001470 con 0x7f7d3c0c3a90 2024-01-27T02:39:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d40e50700 1 -- 172.21.15.7:0/2912323156 >> v1:172.21.15.7:6789/0 conn(0x7f7d3c0c3a90 legacy=0x7f7d3c0aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d40e50700 1 --2- 172.21.15.7:0/2912323156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d28003a60 0x7f7d28003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d40e50700 1 --2- 172.21.15.7:0/2912323156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d28004810 0x7f7d28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d40e50700 1 --2- 172.21.15.7:0/2912323156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d280051c0 0x7f7d2800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d40e50700 1 -- 172.21.15.7:0/2912323156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c0c4550 con 0x7f7d28003a60 2024-01-27T02:39:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d40e50700 1 -- 172.21.15.7:0/2912323156 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2800c0d0 con 0x7f7d28004810 2024-01-27T02:39:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d40e50700 1 -- 172.21.15.7:0/2912323156 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2800c290 con 0x7f7d280051c0 2024-01-27T02:39:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d41e52700 1 --2- 172.21.15.7:0/2912323156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d28003a60 0x7f7d28003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d41651700 1 --2- 172.21.15.7:0/2912323156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d28004810 0x7f7d28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d42653700 1 --2- 172.21.15.7:0/2912323156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d280051c0 0x7f7d2800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d41e52700 1 -- 172.21.15.7:0/2912323156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d280051c0 msgr2=0x7f7d2800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d41e52700 1 --2- 172.21.15.7:0/2912323156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d280051c0 0x7f7d2800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d41e52700 1 -- 172.21.15.7:0/2912323156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d28004810 msgr2=0x7f7d28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d41e52700 1 --2- 172.21.15.7:0/2912323156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d28004810 0x7f7d28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d41e52700 1 -- 172.21.15.7:0/2912323156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d28003570 con 0x7f7d28003a60 2024-01-27T02:39:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d42653700 1 -- 172.21.15.7:0/2912323156 reap_dead start 2024-01-27T02:39:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.503+0000 7f7d41e52700 1 --2- 172.21.15.7:0/2912323156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d28003a60 0x7f7d28003e70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009c00 tx=0x7f7d380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59965112d0f18afc server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.504+0000 7f7d40e50700 1 -- 172.21.15.7:0/2912323156 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3800e710 con 0x7f7d28003a60 2024-01-27T02:39:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.504+0000 7f7d40e50700 1 -- 172.21.15.7:0/2912323156 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3800e870 con 0x7f7d28003a60 2024-01-27T02:39:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.504+0000 7f7d40e50700 1 -- 172.21.15.7:0/2912323156 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3800ec30 con 0x7f7d28003a60 2024-01-27T02:39:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.504+0000 7f7d440b6700 1 -- 172.21.15.7:0/2912323156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d28003a60 msgr2=0x7f7d28003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.504+0000 7f7d440b6700 1 --2- 172.21.15.7:0/2912323156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d28003a60 0x7f7d28003e70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009c00 tx=0x7f7d380091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.504+0000 7f7d440b6700 1 -- 172.21.15.7:0/2912323156 shutdown_connections 2024-01-27T02:39:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.504+0000 7f7d440b6700 1 --2- 172.21.15.7:0/2912323156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d28003a60 0x7f7d28003e70 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.504+0000 7f7d440b6700 1 -- 172.21.15.7:0/2912323156 >> 172.21.15.7:0/2912323156 conn(0x7f7d3c0cdc30 msgr2=0x7f7d3c0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.504+0000 7f7d440b6700 1 -- 172.21.15.7:0/2912323156 shutdown_connections 2024-01-27T02:39:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.504+0000 7f7d440b6700 1 -- 172.21.15.7:0/2912323156 wait complete. 2024-01-27T02:39:19.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.506+0000 7f7d440b6700 1 Processor -- start 2024-01-27T02:39:19.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.506+0000 7f7d440b6700 1 -- start start 2024-01-27T02:39:19.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.506+0000 7f7d440b6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d28003a60 0x7f7d3c156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:19.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.506+0000 7f7d440b6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d3c0b0d20 0x7f7d3c0af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:19.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.506+0000 7f7d440b6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d3c0af8d0 0x7f7d3c0c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:19.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.506+0000 7f7d440b6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c0ada90 con 0x7f7d3c0af8d0 2024-01-27T02:39:19.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.506+0000 7f7d440b6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c0ace80 con 0x7f7d28003a60 2024-01-27T02:39:19.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.506+0000 7f7d440b6700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c0ad160 con 0x7f7d3c0b0d20 2024-01-27T02:39:19.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.507+0000 7f7d42653700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d3c0af8d0 0x7f7d3c0c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:19.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.507+0000 7f7d42653700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d3c0af8d0 0x7f7d3c0c5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48194/0 (socket says 172.21.15.7:48194) 2024-01-27T02:39:19.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.507+0000 7f7d42653700 1 -- 172.21.15.7:0/2312853885 learned_addr learned my addr 172.21.15.7:0/2312853885 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:19.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.507+0000 7f7d41e52700 1 --2- 172.21.15.7:0/2312853885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d28003a60 0x7f7d3c156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:19.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.507+0000 7f7d41651700 1 --2- 172.21.15.7:0/2312853885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d3c0b0d20 0x7f7d3c0af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:19.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.507+0000 7f7d42653700 1 -- 172.21.15.7:0/2312853885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d3c0b0d20 msgr2=0x7f7d3c0af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:19.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.507+0000 7f7d42653700 1 --2- 172.21.15.7:0/2312853885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d3c0b0d20 0x7f7d3c0af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.507+0000 7f7d42653700 1 -- 172.21.15.7:0/2312853885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d28003a60 msgr2=0x7f7d3c156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:19.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.507+0000 7f7d42653700 1 --2- 172.21.15.7:0/2312853885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d28003a60 0x7f7d3c156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.507+0000 7f7d42653700 1 -- 172.21.15.7:0/2312853885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3c0ad440 con 0x7f7d3c0af8d0 2024-01-27T02:39:19.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.507+0000 7f7d42653700 1 --2- 172.21.15.7:0/2312853885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d3c0af8d0 0x7f7d3c0c5fb0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c0c8220 tx=0x7f7d34001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:19.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.507+0000 7f7d31ffb700 1 -- 172.21.15.7:0/2312853885 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d34006db0 con 0x7f7d3c0af8d0 2024-01-27T02:39:19.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.507+0000 7f7d31ffb700 1 -- 172.21.15.7:0/2312853885 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3400c5a0 con 0x7f7d3c0af8d0 2024-01-27T02:39:19.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.507+0000 7f7d440b6700 1 -- 172.21.15.7:0/2312853885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d3c0ad720 con 0x7f7d3c0af8d0 2024-01-27T02:39:19.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.507+0000 7f7d31ffb700 1 -- 172.21.15.7:0/2312853885 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3400c960 con 0x7f7d3c0af8d0 2024-01-27T02:39:19.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.508+0000 7f7d440b6700 1 -- 172.21.15.7:0/2312853885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3c0c5d10 con 0x7f7d3c0af8d0 2024-01-27T02:39:19.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.509+0000 7f7d440b6700 1 -- 172.21.15.7:0/2312853885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d3c04c710 con 0x7f7d3c0af8d0 2024-01-27T02:39:19.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.509+0000 7f7d31ffb700 1 -- 172.21.15.7:0/2312853885 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7d3400cd50 con 0x7f7d3c0af8d0 2024-01-27T02:39:19.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.510+0000 7f7d31ffb700 1 --2- 172.21.15.7:0/2312853885 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7d20066f30 0x7f7d200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:19.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.510+0000 7f7d31ffb700 1 -- 172.21.15.7:0/2312853885 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7d34083e90 con 0x7f7d3c0af8d0 2024-01-27T02:39:19.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.510+0000 7f7d41e52700 1 --2- 172.21.15.7:0/2312853885 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7d20066f30 0x7f7d200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:19.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.510+0000 7f7d41e52700 1 --2- 172.21.15.7:0/2312853885 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7d20066f30 0x7f7d200693e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7d380091e0 tx=0x7f7d3801a000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:19.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.514+0000 7f7d31ffb700 1 -- 172.21.15.7:0/2312853885 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d34051450 con 0x7f7d3c0af8d0 2024-01-27T02:39:19.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.720+0000 7f7d440b6700 1 -- 172.21.15.7:0/2312853885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7d3c06f770 con 0x7f7d3c0af8d0 2024-01-27T02:39:19.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.721+0000 7f7d31ffb700 1 -- 172.21.15.7:0/2312853885 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v145) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7d34054a70 con 0x7f7d3c0af8d0 2024-01-27T02:39:19.721 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:19.722 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-27T02:39:19.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:39:19.722 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-27T02:39:19.722 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-27T02:39:19.722 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:39:19.722 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-27T02:39:19.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.723+0000 7f7d440b6700 1 -- 172.21.15.7:0/2312853885 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7d20066f30 msgr2=0x7f7d200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:19.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.723+0000 7f7d440b6700 1 --2- 172.21.15.7:0/2312853885 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7d20066f30 0x7f7d200693e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7d380091e0 tx=0x7f7d3801a000 comp rx=0 tx=0).stop 2024-01-27T02:39:19.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.723+0000 7f7d440b6700 1 -- 172.21.15.7:0/2312853885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d3c0af8d0 msgr2=0x7f7d3c0c5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:19.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.723+0000 7f7d440b6700 1 --2- 172.21.15.7:0/2312853885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d3c0af8d0 0x7f7d3c0c5fb0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c0c8220 tx=0x7f7d34001640 comp rx=0 tx=0).stop 2024-01-27T02:39:19.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.723+0000 7f7d440b6700 1 -- 172.21.15.7:0/2312853885 shutdown_connections 2024-01-27T02:39:19.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.723+0000 7f7d440b6700 1 --2- 172.21.15.7:0/2312853885 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7d20066f30 0x7f7d200693e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.723+0000 7f7d440b6700 1 --2- 172.21.15.7:0/2312853885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d3c0af8d0 0x7f7d3c0c5fb0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.723+0000 7f7d440b6700 1 --2- 172.21.15.7:0/2312853885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d28003a60 0x7f7d3c156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.723+0000 7f7d440b6700 1 --2- 172.21.15.7:0/2312853885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d3c0b0d20 0x7f7d3c0af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.723+0000 7f7d440b6700 1 -- 172.21.15.7:0/2312853885 >> 172.21.15.7:0/2312853885 conn(0x7f7d3c0cdc30 msgr2=0x7f7d3c0ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:19.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.723+0000 7f7d440b6700 1 -- 172.21.15.7:0/2312853885 shutdown_connections 2024-01-27T02:39:19.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.723+0000 7f7d440b6700 1 -- 172.21.15.7:0/2312853885 wait complete. 2024-01-27T02:39:19.738 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-27T02:39:19.738 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-27T02:39:19.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.873+0000 7fca5e8d7700 1 Processor -- start 2024-01-27T02:39:19.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.873+0000 7fca5e8d7700 1 -- start start 2024-01-27T02:39:19.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.874+0000 7fca5e8d7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca580c4cc0 0x7fca580ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:19.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.874+0000 7fca5e8d7700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fca580cd2f0 0x7fca580679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:19.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.874+0000 7fca5e8d7700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca58067ef0 con 0x7fca5815a230 2024-01-27T02:39:19.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.874+0000 7fca5e8d7700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca580cd790 con 0x7fca580cd2f0 2024-01-27T02:39:19.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.874+0000 7fca5e8d7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca580c5320 con 0x7fca580c4cc0 2024-01-27T02:39:19.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.874+0000 7fca5ce74700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fca580cd2f0 0x7fca580679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:19.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.874+0000 7fca577fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca580c4cc0 0x7fca580ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:19.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.874+0000 7fca57fff700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fca5815a230 0x7fca580c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:36416/0 (socket says 172.21.15.7:36416) 2024-01-27T02:39:19.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.874+0000 7fca57fff700 1 -- 172.21.15.7:0/3769522005 learned_addr learned my addr 172.21.15.7:0/3769522005 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:19.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.875+0000 7fca577fe700 1 -- 172.21.15.7:0/3769522005 >> v2:172.21.15.188:3300/0 conn(0x7fca580cd2f0 msgr2=0x7fca580679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:19.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.875+0000 7fca577fe700 1 --2- 172.21.15.7:0/3769522005 >> v2:172.21.15.188:3300/0 conn(0x7fca580cd2f0 0x7fca580679b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.875+0000 7fca577fe700 1 -- 172.21.15.7:0/3769522005 >> v1:172.21.15.188:6789/0 conn(0x7fca5815a230 legacy=0x7fca580c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:19.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.875+0000 7fca577fe700 1 -- 172.21.15.7:0/3769522005 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca580c9520 con 0x7fca580c4cc0 2024-01-27T02:39:19.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.875+0000 7fca5ce74700 1 --2- 172.21.15.7:0/3769522005 >> v2:172.21.15.188:3300/0 conn(0x7fca580cd2f0 0x7fca580679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:39:19.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.875+0000 7fca577fe700 1 --2- 172.21.15.7:0/3769522005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca580c4cc0 0x7fca580ce310 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fca400098e0 tx=0x7fca400091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=568cd76c16f3e898 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:19.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.875+0000 7fca56ffd700 1 -- 172.21.15.7:0/3769522005 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4000b640 con 0x7fca580c4cc0 2024-01-27T02:39:19.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.875+0000 7fca56ffd700 1 -- 172.21.15.7:0/3769522005 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4000b7a0 con 0x7fca580c4cc0 2024-01-27T02:39:19.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.875+0000 7fca56ffd700 1 -- 172.21.15.7:0/3769522005 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4000bb40 con 0x7fca580c4cc0 2024-01-27T02:39:19.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.876+0000 7fca5e8d7700 1 -- 172.21.15.7:0/3769522005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca580c4cc0 msgr2=0x7fca580ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:19.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.876+0000 7fca5e8d7700 1 --2- 172.21.15.7:0/3769522005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca580c4cc0 0x7fca580ce310 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fca400098e0 tx=0x7fca400091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.876+0000 7fca5e8d7700 1 -- 172.21.15.7:0/3769522005 shutdown_connections 2024-01-27T02:39:19.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.876+0000 7fca5e8d7700 1 --2- 172.21.15.7:0/3769522005 >> v2:172.21.15.188:3300/0 conn(0x7fca580cd2f0 0x7fca580679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.876+0000 7fca5e8d7700 1 --2- 172.21.15.7:0/3769522005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca580c4cc0 0x7fca580ce310 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.876+0000 7fca5e8d7700 1 -- 172.21.15.7:0/3769522005 >> 172.21.15.7:0/3769522005 conn(0x7fca58098020 msgr2=0x7fca580711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:19.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.876+0000 7fca5e8d7700 1 -- 172.21.15.7:0/3769522005 shutdown_connections 2024-01-27T02:39:19.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.876+0000 7fca5e8d7700 1 -- 172.21.15.7:0/3769522005 wait complete. 2024-01-27T02:39:19.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.877+0000 7fca5e8d7700 1 Processor -- start 2024-01-27T02:39:19.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.878+0000 7fca5e8d7700 1 -- start start 2024-01-27T02:39:19.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.879+0000 7fca5e8d7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca580c4cc0 0x7fca58158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:19.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.879+0000 7fca5e8d7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca580cd2f0 0x7fca580c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:19.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.879+0000 7fca5e8d7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca5815a230 0x7fca5816dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:19.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.879+0000 7fca5e8d7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca58158590 con 0x7fca5815a230 2024-01-27T02:39:19.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.879+0000 7fca5e8d7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca580c65d0 con 0x7fca580cd2f0 2024-01-27T02:39:19.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.879+0000 7fca5e8d7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca580b1d10 con 0x7fca580c4cc0 2024-01-27T02:39:19.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.879+0000 7fca5ce74700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca5815a230 0x7fca5816dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:19.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.879+0000 7fca5ce74700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca5815a230 0x7fca5816dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48208/0 (socket says 172.21.15.7:48208) 2024-01-27T02:39:19.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.879+0000 7fca5ce74700 1 -- 172.21.15.7:0/3908539211 learned_addr learned my addr 172.21.15.7:0/3908539211 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:19.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.879+0000 7fca57fff700 1 --2- 172.21.15.7:0/3908539211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca580c4cc0 0x7fca58158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:19.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.879+0000 7fca577fe700 1 --2- 172.21.15.7:0/3908539211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca580cd2f0 0x7fca580c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:19.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.879+0000 7fca5ce74700 1 -- 172.21.15.7:0/3908539211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca580c4cc0 msgr2=0x7fca58158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:19.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.879+0000 7fca5ce74700 1 --2- 172.21.15.7:0/3908539211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca580c4cc0 0x7fca58158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.879+0000 7fca5ce74700 1 -- 172.21.15.7:0/3908539211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca580cd2f0 msgr2=0x7fca580c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:19.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.879+0000 7fca5ce74700 1 --2- 172.21.15.7:0/3908539211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca580cd2f0 0x7fca580c6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:19.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.879+0000 7fca5ce74700 1 -- 172.21.15.7:0/3908539211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca50009a20 con 0x7fca5815a230 2024-01-27T02:39:19.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.879+0000 7fca57fff700 1 --2- 172.21.15.7:0/3908539211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca580c4cc0 0x7fca58158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:39:19.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.879+0000 7fca5ce74700 1 --2- 172.21.15.7:0/3908539211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca5815a230 0x7fca5816dbe0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fca5000c0f0 tx=0x7fca5000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:19.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.880+0000 7fca4ffff700 1 -- 172.21.15.7:0/3908539211 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5000cbd0 con 0x7fca5815a230 2024-01-27T02:39:19.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.880+0000 7fca4ffff700 1 -- 172.21.15.7:0/3908539211 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5000f040 con 0x7fca5815a230 2024-01-27T02:39:19.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.880+0000 7fca5e8d7700 1 -- 172.21.15.7:0/3908539211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca40009420 con 0x7fca5815a230 2024-01-27T02:39:19.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.880+0000 7fca4ffff700 1 -- 172.21.15.7:0/3908539211 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca50014480 con 0x7fca5815a230 2024-01-27T02:39:19.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.880+0000 7fca5e8d7700 1 -- 172.21.15.7:0/3908539211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca58158880 con 0x7fca5815a230 2024-01-27T02:39:19.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.881+0000 7fca5e8d7700 1 -- 172.21.15.7:0/3908539211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca5804c710 con 0x7fca5815a230 2024-01-27T02:39:19.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.881+0000 7fca4ffff700 1 -- 172.21.15.7:0/3908539211 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fca50003020 con 0x7fca5815a230 2024-01-27T02:39:19.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.882+0000 7fca4ffff700 1 --2- 172.21.15.7:0/3908539211 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fca38066f30 0x7fca380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:19.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.882+0000 7fca4ffff700 1 -- 172.21.15.7:0/3908539211 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fca50084b10 con 0x7fca5815a230 2024-01-27T02:39:19.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.882+0000 7fca57fff700 1 --2- 172.21.15.7:0/3908539211 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fca38066f30 0x7fca380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:19.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.882+0000 7fca57fff700 1 --2- 172.21.15.7:0/3908539211 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fca38066f30 0x7fca380693e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fca48007fd0 tx=0x7fca48007f00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:19.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:19.886+0000 7fca4ffff700 1 -- 172.21.15.7:0/3908539211 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca500520d0 con 0x7fca5815a230 2024-01-27T02:39:20.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:20.086+0000 7fca5e8d7700 1 -- 172.21.15.7:0/3908539211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fca580b1e50 con 0x7fca5815a230 2024-01-27T02:39:20.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:20.881+0000 7fca4ffff700 1 -- 172.21.15.7:0/3908539211 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v146) v1 ==== 138+0+0 (secure 0 0 0) 0x7fca500556f0 con 0x7fca5815a230 2024-01-27T02:39:20.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:20.885+0000 7fca5e8d7700 1 -- 172.21.15.7:0/3908539211 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fca38066f30 msgr2=0x7fca380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:20.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:20.885+0000 7fca5e8d7700 1 --2- 172.21.15.7:0/3908539211 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fca38066f30 0x7fca380693e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fca48007fd0 tx=0x7fca48007f00 comp rx=0 tx=0).stop 2024-01-27T02:39:20.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:20.885+0000 7fca5e8d7700 1 -- 172.21.15.7:0/3908539211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca5815a230 msgr2=0x7fca5816dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:20.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:20.885+0000 7fca5e8d7700 1 --2- 172.21.15.7:0/3908539211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca5815a230 0x7fca5816dbe0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fca5000c0f0 tx=0x7fca5000b040 comp rx=0 tx=0).stop 2024-01-27T02:39:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:20.886+0000 7fca5e8d7700 1 -- 172.21.15.7:0/3908539211 shutdown_connections 2024-01-27T02:39:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:20.886+0000 7fca5e8d7700 1 --2- 172.21.15.7:0/3908539211 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fca38066f30 0x7fca380693e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:20.886+0000 7fca5e8d7700 1 --2- 172.21.15.7:0/3908539211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca5815a230 0x7fca5816dbe0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:20.886+0000 7fca5e8d7700 1 --2- 172.21.15.7:0/3908539211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca580c4cc0 0x7fca58158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:20.886+0000 7fca5e8d7700 1 --2- 172.21.15.7:0/3908539211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca580cd2f0 0x7fca580c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:20.886+0000 7fca5e8d7700 1 -- 172.21.15.7:0/3908539211 >> 172.21.15.7:0/3908539211 conn(0x7fca58098020 msgr2=0x7fca580cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:20.886+0000 7fca5e8d7700 1 -- 172.21.15.7:0/3908539211 shutdown_connections 2024-01-27T02:39:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:20.886+0000 7fca5e8d7700 1 -- 172.21.15.7:0/3908539211 wait complete. 2024-01-27T02:39:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-27T02:39:20.902 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-27T02:39:21.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.044+0000 7fcdc4ba4700 1 Processor -- start 2024-01-27T02:39:21.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.044+0000 7fcdc4ba4700 1 -- start start 2024-01-27T02:39:21.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.045+0000 7fcdc4ba4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdc0154550 0x7fcdc00ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:21.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.046+0000 7fcdc4ba4700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcdc00ad7c0 0x7fcdc0063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:21.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.046+0000 7fcdc4ba4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc00adec0 con 0x7fcdc0154920 2024-01-27T02:39:21.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.046+0000 7fcdc4ba4700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc00ae1a0 con 0x7fcdc00ad7c0 2024-01-27T02:39:21.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.046+0000 7fcdc4ba4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc015a850 con 0x7fcdc0154550 2024-01-27T02:39:21.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.046+0000 7fcdbdd9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcdc0154920 0x7fcdc015b050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50904/0 (socket says 172.21.15.7:50904) 2024-01-27T02:39:21.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.046+0000 7fcdbdd9b700 1 -- 172.21.15.7:0/3753182763 learned_addr learned my addr 172.21.15.7:0/3753182763 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:21.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.046+0000 7fcdbe59c700 1 --2- 172.21.15.7:0/3753182763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdc0154550 0x7fcdc00ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:21.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.046+0000 7fcdbed9d700 1 --2- 172.21.15.7:0/3753182763 >> v2:172.21.15.188:3300/0 conn(0x7fcdc00ad7c0 0x7fcdc0063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:21.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.046+0000 7fcdbd59a700 1 -- 172.21.15.7:0/3753182763 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3317103131 0 0) 0x7fcdc00adec0 con 0x7fcdc0154920 2024-01-27T02:39:21.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.046+0000 7fcdbd59a700 1 -- 172.21.15.7:0/3753182763 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdac003570 con 0x7fcdc0154920 2024-01-27T02:39:21.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.046+0000 7fcdbe59c700 1 -- 172.21.15.7:0/3753182763 >> v2:172.21.15.188:3300/0 conn(0x7fcdc00ad7c0 msgr2=0x7fcdc0063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:21.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.046+0000 7fcdbe59c700 1 --2- 172.21.15.7:0/3753182763 >> v2:172.21.15.188:3300/0 conn(0x7fcdc00ad7c0 0x7fcdc0063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:21.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.046+0000 7fcdbe59c700 1 -- 172.21.15.7:0/3753182763 >> v1:172.21.15.7:6789/0 conn(0x7fcdc0154920 legacy=0x7fcdc015b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:21.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.046+0000 7fcdbe59c700 1 -- 172.21.15.7:0/3753182763 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc015ab30 con 0x7fcdc0154550 2024-01-27T02:39:21.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.046+0000 7fcdbed9d700 1 --2- 172.21.15.7:0/3753182763 >> v2:172.21.15.188:3300/0 conn(0x7fcdc00ad7c0 0x7fcdc0063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:39:21.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.046+0000 7fcdbe59c700 1 --2- 172.21.15.7:0/3753182763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdc0154550 0x7fcdc00ce6b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00098e0 tx=0x7fcdb00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80db951c1cc8bdd0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:21.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.047+0000 7fcdbd59a700 1 -- 172.21.15.7:0/3753182763 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0011630 con 0x7fcdc0154550 2024-01-27T02:39:21.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.047+0000 7fcdbd59a700 1 -- 172.21.15.7:0/3753182763 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb0011790 con 0x7fcdc0154550 2024-01-27T02:39:21.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.047+0000 7fcdbd59a700 1 -- 172.21.15.7:0/3753182763 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0011b30 con 0x7fcdc0154550 2024-01-27T02:39:21.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.047+0000 7fcdc4ba4700 1 -- 172.21.15.7:0/3753182763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdc0154550 msgr2=0x7fcdc00ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:21.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.047+0000 7fcdc4ba4700 1 --2- 172.21.15.7:0/3753182763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdc0154550 0x7fcdc00ce6b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00098e0 tx=0x7fcdb00091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:21.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.047+0000 7fcdc4ba4700 1 -- 172.21.15.7:0/3753182763 shutdown_connections 2024-01-27T02:39:21.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.047+0000 7fcdc4ba4700 1 --2- 172.21.15.7:0/3753182763 >> v2:172.21.15.188:3300/0 conn(0x7fcdc00ad7c0 0x7fcdc0063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:21.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.047+0000 7fcdc4ba4700 1 --2- 172.21.15.7:0/3753182763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdc0154550 0x7fcdc00ce6b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:21.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.047+0000 7fcdc4ba4700 1 -- 172.21.15.7:0/3753182763 >> 172.21.15.7:0/3753182763 conn(0x7fcdc0068480 msgr2=0x7fcdc00cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:21.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.047+0000 7fcdc4ba4700 1 -- 172.21.15.7:0/3753182763 shutdown_connections 2024-01-27T02:39:21.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.047+0000 7fcdc4ba4700 1 -- 172.21.15.7:0/3753182763 wait complete. 2024-01-27T02:39:21.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.049+0000 7fcdc4ba4700 1 Processor -- start 2024-01-27T02:39:21.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.049+0000 7fcdc4ba4700 1 -- start start 2024-01-27T02:39:21.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.050+0000 7fcdc4ba4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc00ad7c0 0x7fcdc014b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:21.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.050+0000 7fcdc4ba4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdc0154550 0x7fcdc00b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:21.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.050+0000 7fcdc4ba4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdc0154920 0x7fcdc006a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:21.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.050+0000 7fcdc4ba4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc014ffa0 con 0x7fcdc0154920 2024-01-27T02:39:21.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.050+0000 7fcdc4ba4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc014bd30 con 0x7fcdc00ad7c0 2024-01-27T02:39:21.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.050+0000 7fcdc4ba4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc00b2ff0 con 0x7fcdc0154550 2024-01-27T02:39:21.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.050+0000 7fcdbed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdc0154920 0x7fcdc006a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:21.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.050+0000 7fcdbed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdc0154920 0x7fcdc006a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48222/0 (socket says 172.21.15.7:48222) 2024-01-27T02:39:21.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.050+0000 7fcdbed9d700 1 -- 172.21.15.7:0/1870541464 learned_addr learned my addr 172.21.15.7:0/1870541464 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:21.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.050+0000 7fcdbdd9b700 1 --2- 172.21.15.7:0/1870541464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdc0154550 0x7fcdc00b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:21.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.050+0000 7fcdbe59c700 1 --2- 172.21.15.7:0/1870541464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc00ad7c0 0x7fcdc014b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:21.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.051+0000 7fcdbed9d700 1 -- 172.21.15.7:0/1870541464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdc0154550 msgr2=0x7fcdc00b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:21.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.051+0000 7fcdbed9d700 1 --2- 172.21.15.7:0/1870541464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdc0154550 0x7fcdc00b26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:21.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.051+0000 7fcdbed9d700 1 -- 172.21.15.7:0/1870541464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc00ad7c0 msgr2=0x7fcdc014b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:21.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.051+0000 7fcdbed9d700 1 --2- 172.21.15.7:0/1870541464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc00ad7c0 0x7fcdc014b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:21.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.051+0000 7fcdbed9d700 1 -- 172.21.15.7:0/1870541464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb8009a20 con 0x7fcdc0154920 2024-01-27T02:39:21.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.051+0000 7fcdbed9d700 1 --2- 172.21.15.7:0/1870541464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdc0154920 0x7fcdc006a3e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009fc0 tx=0x7fcdb800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:21.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.051+0000 7fcdb67fc700 1 -- 172.21.15.7:0/1870541464 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb800cbd0 con 0x7fcdc0154920 2024-01-27T02:39:21.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.051+0000 7fcdb67fc700 1 -- 172.21.15.7:0/1870541464 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb800f040 con 0x7fcdc0154920 2024-01-27T02:39:21.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.051+0000 7fcdb67fc700 1 -- 172.21.15.7:0/1870541464 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8014480 con 0x7fcdc0154920 2024-01-27T02:39:21.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.051+0000 7fcdc4ba4700 1 -- 172.21.15.7:0/1870541464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb0009420 con 0x7fcdc0154920 2024-01-27T02:39:21.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.051+0000 7fcdc4ba4700 1 -- 172.21.15.7:0/1870541464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc014f950 con 0x7fcdc0154920 2024-01-27T02:39:21.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.052+0000 7fcdc4ba4700 1 -- 172.21.15.7:0/1870541464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc0150110 con 0x7fcdc0154920 2024-01-27T02:39:21.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.053+0000 7fcdb67fc700 1 -- 172.21.15.7:0/1870541464 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcdb8003020 con 0x7fcdc0154920 2024-01-27T02:39:21.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.053+0000 7fcdb67fc700 1 --2- 172.21.15.7:0/1870541464 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcda0066f30 0x7fcda00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:21.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.053+0000 7fcdb67fc700 1 -- 172.21.15.7:0/1870541464 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fcdb8084c70 con 0x7fcdc0154920 2024-01-27T02:39:21.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.053+0000 7fcdbe59c700 1 --2- 172.21.15.7:0/1870541464 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcda0066f30 0x7fcda00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:21.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.054+0000 7fcdbe59c700 1 --2- 172.21.15.7:0/1870541464 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcda0066f30 0x7fcda00693e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00091e0 tx=0x7fcdb0017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:21.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.058+0000 7fcdb67fc700 1 -- 172.21.15.7:0/1870541464 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcdb80520c0 con 0x7fcdc0154920 2024-01-27T02:39:21.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.261+0000 7fcdc4ba4700 1 -- 172.21.15.7:0/1870541464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcdc0072060 con 0x7fcdc0154920 2024-01-27T02:39:21.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.891+0000 7fcdb67fc700 1 -- 172.21.15.7:0/1870541464 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v147) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcdb80556e0 con 0x7fcdc0154920 2024-01-27T02:39:21.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.902+0000 7fcdc4ba4700 1 -- 172.21.15.7:0/1870541464 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcda0066f30 msgr2=0x7fcda00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:21.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.902+0000 7fcdc4ba4700 1 --2- 172.21.15.7:0/1870541464 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcda0066f30 0x7fcda00693e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00091e0 tx=0x7fcdb0017000 comp rx=0 tx=0).stop 2024-01-27T02:39:21.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.902+0000 7fcdc4ba4700 1 -- 172.21.15.7:0/1870541464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdc0154920 msgr2=0x7fcdc006a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:21.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.902+0000 7fcdc4ba4700 1 --2- 172.21.15.7:0/1870541464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdc0154920 0x7fcdc006a3e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009fc0 tx=0x7fcdb800b040 comp rx=0 tx=0).stop 2024-01-27T02:39:21.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.902+0000 7fcdc4ba4700 1 -- 172.21.15.7:0/1870541464 shutdown_connections 2024-01-27T02:39:21.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.902+0000 7fcdc4ba4700 1 --2- 172.21.15.7:0/1870541464 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcda0066f30 0x7fcda00693e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:21.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.902+0000 7fcdc4ba4700 1 --2- 172.21.15.7:0/1870541464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdc0154920 0x7fcdc006a3e0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:21.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.902+0000 7fcdc4ba4700 1 --2- 172.21.15.7:0/1870541464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc00ad7c0 0x7fcdc014b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:21.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.902+0000 7fcdc4ba4700 1 --2- 172.21.15.7:0/1870541464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdc0154550 0x7fcdc00b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:21.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.902+0000 7fcdc4ba4700 1 -- 172.21.15.7:0/1870541464 >> 172.21.15.7:0/1870541464 conn(0x7fcdc0068480 msgr2=0x7fcdc00a64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:21.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.904+0000 7fcdc4ba4700 1 -- 172.21.15.7:0/1870541464 shutdown_connections 2024-01-27T02:39:21.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:21.904+0000 7fcdc4ba4700 1 -- 172.21.15.7:0/1870541464 wait complete. 2024-01-27T02:39:21.905 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-27T02:39:21.920 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-27T02:39:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.059+0000 7fa9575ee700 1 Processor -- start 2024-01-27T02:39:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.059+0000 7fa9575ee700 1 -- start start 2024-01-27T02:39:22.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.060+0000 7fa9575ee700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa95015a180 0x7fa9500c3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:22.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.060+0000 7fa9575ee700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9500ca3b0 0x7fa9500aef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:22.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.060+0000 7fa9575ee700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9500b1450 con 0x7fa95015a550 2024-01-27T02:39:22.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.060+0000 7fa9575ee700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9500b18e0 con 0x7fa95015a180 2024-01-27T02:39:22.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.060+0000 7fa9575ee700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9500b0840 con 0x7fa9500ca3b0 2024-01-27T02:39:22.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.061+0000 7fa955b8b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9500ca3b0 0x7fa9500aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:22.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.061+0000 7fa954b89700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fa95015a550 0x7fa9500c9ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:48340/0 (socket says 172.21.15.7:48340) 2024-01-27T02:39:22.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.061+0000 7fa95538a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa95015a180 0x7fa9500c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:22.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.061+0000 7fa954b89700 1 -- 172.21.15.7:0/2315564280 learned_addr learned my addr 172.21.15.7:0/2315564280 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:22.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.061+0000 7fa947fff700 1 -- 172.21.15.7:0/2315564280 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 599187866 0 0) 0x7fa9500b1450 con 0x7fa95015a550 2024-01-27T02:39:22.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.061+0000 7fa947fff700 1 -- 172.21.15.7:0/2315564280 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa93c003570 con 0x7fa95015a550 2024-01-27T02:39:22.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.061+0000 7fa95538a700 1 -- 172.21.15.7:0/2315564280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9500ca3b0 msgr2=0x7fa9500aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:22.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.061+0000 7fa95538a700 1 --2- 172.21.15.7:0/2315564280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9500ca3b0 0x7fa9500aef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:22.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.061+0000 7fa95538a700 1 -- 172.21.15.7:0/2315564280 >> v1:172.21.15.188:6789/0 conn(0x7fa95015a550 legacy=0x7fa9500c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:22.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.061+0000 7fa95538a700 1 -- 172.21.15.7:0/2315564280 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9500b0b20 con 0x7fa95015a180 2024-01-27T02:39:22.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.061+0000 7fa955b8b700 1 --2- 172.21.15.7:0/2315564280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9500ca3b0 0x7fa9500aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:39:22.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.061+0000 7fa95538a700 1 --2- 172.21.15.7:0/2315564280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa95015a180 0x7fa9500c3750 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0098d0 tx=0x7fa94c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8694924cedd1c54 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:22.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.061+0000 7fa947fff700 1 -- 172.21.15.7:0/2315564280 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c005c10 con 0x7fa95015a180 2024-01-27T02:39:22.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.061+0000 7fa947fff700 1 -- 172.21.15.7:0/2315564280 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94c005d70 con 0x7fa95015a180 2024-01-27T02:39:22.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.061+0000 7fa947fff700 1 -- 172.21.15.7:0/2315564280 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c006260 con 0x7fa95015a180 2024-01-27T02:39:22.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.062+0000 7fa9575ee700 1 -- 172.21.15.7:0/2315564280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa95015a180 msgr2=0x7fa9500c3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:22.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.062+0000 7fa9575ee700 1 --2- 172.21.15.7:0/2315564280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa95015a180 0x7fa9500c3750 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0098d0 tx=0x7fa94c009120 comp rx=0 tx=0).stop 2024-01-27T02:39:22.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.062+0000 7fa9575ee700 1 -- 172.21.15.7:0/2315564280 shutdown_connections 2024-01-27T02:39:22.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.062+0000 7fa9575ee700 1 --2- 172.21.15.7:0/2315564280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9500ca3b0 0x7fa9500aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:22.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.062+0000 7fa9575ee700 1 --2- 172.21.15.7:0/2315564280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa95015a180 0x7fa9500c3750 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:22.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.062+0000 7fa9575ee700 1 -- 172.21.15.7:0/2315564280 >> 172.21.15.7:0/2315564280 conn(0x7fa9500679b0 msgr2=0x7fa950067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:22.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.062+0000 7fa9575ee700 1 -- 172.21.15.7:0/2315564280 shutdown_connections 2024-01-27T02:39:22.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.062+0000 7fa9575ee700 1 -- 172.21.15.7:0/2315564280 wait complete. 2024-01-27T02:39:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.063+0000 7fa9575ee700 1 Processor -- start 2024-01-27T02:39:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.063+0000 7fa9575ee700 1 -- start start 2024-01-27T02:39:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.064+0000 7fa9575ee700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9500ca3b0 0x7fa95015e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.064+0000 7fa9575ee700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa95015a180 0x7fa95016b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.064+0000 7fa9575ee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa95015a550 0x7fa95016e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.064+0000 7fa9575ee700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95009a9a0 con 0x7fa95015a550 2024-01-27T02:39:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.064+0000 7fa9575ee700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95016e340 con 0x7fa95015a180 2024-01-27T02:39:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.064+0000 7fa9575ee700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95016b8a0 con 0x7fa9500ca3b0 2024-01-27T02:39:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.064+0000 7fa955b8b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa95015a550 0x7fa95016e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.064+0000 7fa95538a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9500ca3b0 0x7fa95015e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.064+0000 7fa955b8b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa95015a550 0x7fa95016e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37634/0 (socket says 172.21.15.7:37634) 2024-01-27T02:39:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.064+0000 7fa954b89700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa95015a180 0x7fa95016b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.064+0000 7fa955b8b700 1 -- 172.21.15.7:0/637065874 learned_addr learned my addr 172.21.15.7:0/637065874 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.065+0000 7fa95538a700 1 -- 172.21.15.7:0/637065874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa95015a180 msgr2=0x7fa95016b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.065+0000 7fa95538a700 1 --2- 172.21.15.7:0/637065874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa95015a180 0x7fa95016b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.065+0000 7fa95538a700 1 -- 172.21.15.7:0/637065874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa95015a550 msgr2=0x7fa95016e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.065+0000 7fa95538a700 1 --2- 172.21.15.7:0/637065874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa95015a550 0x7fa95016e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.065+0000 7fa95538a700 1 -- 172.21.15.7:0/637065874 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa948009a20 con 0x7fa9500ca3b0 2024-01-27T02:39:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.065+0000 7fa955b8b700 1 --2- 172.21.15.7:0/637065874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa95015a550 0x7fa95016e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:39:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.065+0000 7fa95538a700 1 --2- 172.21.15.7:0/637065874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9500ca3b0 0x7fa95015e6d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa94c015000 tx=0x7fa94c016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.065+0000 7fa9457fa700 1 -- 172.21.15.7:0/637065874 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c01d6b0 con 0x7fa9500ca3b0 2024-01-27T02:39:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.065+0000 7fa9457fa700 1 -- 172.21.15.7:0/637065874 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94c01d810 con 0x7fa9500ca3b0 2024-01-27T02:39:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.065+0000 7fa9575ee700 1 -- 172.21.15.7:0/637065874 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94c0093c0 con 0x7fa9500ca3b0 2024-01-27T02:39:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.065+0000 7fa9457fa700 1 -- 172.21.15.7:0/637065874 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c01dbd0 con 0x7fa9500ca3b0 2024-01-27T02:39:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.065+0000 7fa9575ee700 1 -- 172.21.15.7:0/637065874 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95009a6c0 con 0x7fa9500ca3b0 2024-01-27T02:39:22.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.066+0000 7fa9575ee700 1 -- 172.21.15.7:0/637065874 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa950072060 con 0x7fa9500ca3b0 2024-01-27T02:39:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.067+0000 7fa9457fa700 1 -- 172.21.15.7:0/637065874 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa94c045bd0 con 0x7fa9500ca3b0 2024-01-27T02:39:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.067+0000 7fa9457fa700 1 --2- 172.21.15.7:0/637065874 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa934066f30 0x7fa9340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.067+0000 7fa9457fa700 1 -- 172.21.15.7:0/637065874 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fa94c086520 con 0x7fa9500ca3b0 2024-01-27T02:39:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.069+0000 7fa954b89700 1 --2- 172.21.15.7:0/637065874 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa934066f30 0x7fa9340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:22.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.071+0000 7fa954b89700 1 --2- 172.21.15.7:0/637065874 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa934066f30 0x7fa9340693e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa95006fe70 tx=0x7fa940009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:22.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.072+0000 7fa9457fa700 1 -- 172.21.15.7:0/637065874 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa94c053750 con 0x7fa9500ca3b0 2024-01-27T02:39:22.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.326+0000 7fa9575ee700 1 -- 172.21.15.7:0/637065874 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa95016e620 con 0x7fa9500ca3b0 2024-01-27T02:39:22.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.892+0000 7fa9457fa700 1 -- 172.21.15.7:0/637065874 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v73) v1 ==== 174+0+0 (secure 0 0 0) 0x7fa94c056d70 con 0x7fa9500ca3b0 2024-01-27T02:39:22.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.903+0000 7fa9575ee700 1 -- 172.21.15.7:0/637065874 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa934066f30 msgr2=0x7fa9340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:22.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.903+0000 7fa9575ee700 1 --2- 172.21.15.7:0/637065874 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa934066f30 0x7fa9340693e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa95006fe70 tx=0x7fa940009040 comp rx=0 tx=0).stop 2024-01-27T02:39:22.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.903+0000 7fa9575ee700 1 -- 172.21.15.7:0/637065874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9500ca3b0 msgr2=0x7fa95015e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:22.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.903+0000 7fa9575ee700 1 --2- 172.21.15.7:0/637065874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9500ca3b0 0x7fa95015e6d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa94c015000 tx=0x7fa94c016000 comp rx=0 tx=0).stop 2024-01-27T02:39:22.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.909+0000 7fa9575ee700 1 -- 172.21.15.7:0/637065874 shutdown_connections 2024-01-27T02:39:22.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.909+0000 7fa9575ee700 1 --2- 172.21.15.7:0/637065874 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa934066f30 0x7fa9340693e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:22.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.909+0000 7fa9575ee700 1 --2- 172.21.15.7:0/637065874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa95015a550 0x7fa95016e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:22.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.909+0000 7fa9575ee700 1 --2- 172.21.15.7:0/637065874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9500ca3b0 0x7fa95015e6d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:22.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.909+0000 7fa9575ee700 1 --2- 172.21.15.7:0/637065874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa95015a180 0x7fa95016b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:22.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.909+0000 7fa9575ee700 1 -- 172.21.15.7:0/637065874 >> 172.21.15.7:0/637065874 conn(0x7fa9500679b0 msgr2=0x7fa9500cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:22.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.909+0000 7fa9575ee700 1 -- 172.21.15.7:0/637065874 shutdown_connections 2024-01-27T02:39:22.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:22.909+0000 7fa9575ee700 1 -- 172.21.15.7:0/637065874 wait complete. 2024-01-27T02:39:22.915 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 18 and data pool 19 2024-01-27T02:39:22.927 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:39:23.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.061+0000 7f41425d4700 1 Processor -- start 2024-01-27T02:39:23.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.061+0000 7f41425d4700 1 -- start start 2024-01-27T02:39:23.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.062+0000 7f41425d4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f413c0c4cc0 0x7f413c0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:23.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.062+0000 7f41425d4700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f413c0ca540 0x7f413c163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:23.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.062+0000 7f41425d4700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f413c0ce5e0 con 0x7f413c15a100 2024-01-27T02:39:23.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.062+0000 7f41425d4700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f413c0cd9d0 con 0x7f413c0ca540 2024-01-27T02:39:23.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.062+0000 7f41425d4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413c0cdcb0 con 0x7f413c0c4cc0 2024-01-27T02:39:23.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.063+0000 7f4140b71700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f413c0ca540 0x7f413c163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:23.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.063+0000 7f413b7fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f413c0c4cc0 0x7f413c0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:23.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.063+0000 7f4140b71700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f413c0ca540 0x7f413c163d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:38978/0 (socket says 172.21.15.7:38978) 2024-01-27T02:39:23.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.063+0000 7f413b7fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f413c0c4cc0 0x7f413c0cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:54532/0 (socket says 172.21.15.7:54532) 2024-01-27T02:39:23.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.063+0000 7f413bfff700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f413c15a100 0x7f413c0c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:48346/0 (socket says 172.21.15.7:48346) 2024-01-27T02:39:23.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.063+0000 7f4140b71700 1 -- 172.21.15.7:0/69649911 learned_addr learned my addr 172.21.15.7:0/69649911 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:23.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.063+0000 7f413b7fe700 1 -- 172.21.15.7:0/69649911 >> v2:172.21.15.188:3300/0 conn(0x7f413c0ca540 msgr2=0x7f413c163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:23.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.063+0000 7f413b7fe700 1 --2- 172.21.15.7:0/69649911 >> v2:172.21.15.188:3300/0 conn(0x7f413c0ca540 0x7f413c163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.063+0000 7f413b7fe700 1 -- 172.21.15.7:0/69649911 >> v1:172.21.15.188:6789/0 conn(0x7f413c15a100 legacy=0x7f413c0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:23.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.063+0000 7f413b7fe700 1 -- 172.21.15.7:0/69649911 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413c0cdf90 con 0x7f413c0c4cc0 2024-01-27T02:39:23.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.063+0000 7f4140b71700 1 --2- 172.21.15.7:0/69649911 >> v2:172.21.15.188:3300/0 conn(0x7f413c0ca540 0x7f413c163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:39:23.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.063+0000 7f413b7fe700 1 --2- 172.21.15.7:0/69649911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f413c0c4cc0 0x7f413c0cd2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f41300098e0 tx=0x7f41300091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17c89d6f088da922 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:23.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.063+0000 7f413affd700 1 -- 172.21.15.7:0/69649911 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413000b640 con 0x7f413c0c4cc0 2024-01-27T02:39:23.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.063+0000 7f413affd700 1 -- 172.21.15.7:0/69649911 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413000b7a0 con 0x7f413c0c4cc0 2024-01-27T02:39:23.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.063+0000 7f413affd700 1 -- 172.21.15.7:0/69649911 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413000bb40 con 0x7f413c0c4cc0 2024-01-27T02:39:23.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.064+0000 7f41425d4700 1 -- 172.21.15.7:0/69649911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f413c0c4cc0 msgr2=0x7f413c0cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:23.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.064+0000 7f41425d4700 1 --2- 172.21.15.7:0/69649911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f413c0c4cc0 0x7f413c0cd2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f41300098e0 tx=0x7f41300091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.064+0000 7f41425d4700 1 -- 172.21.15.7:0/69649911 shutdown_connections 2024-01-27T02:39:23.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.064+0000 7f41425d4700 1 --2- 172.21.15.7:0/69649911 >> v2:172.21.15.188:3300/0 conn(0x7f413c0ca540 0x7f413c163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.064+0000 7f41425d4700 1 --2- 172.21.15.7:0/69649911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f413c0c4cc0 0x7f413c0cd2f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.064+0000 7f41425d4700 1 -- 172.21.15.7:0/69649911 >> 172.21.15.7:0/69649911 conn(0x7f413c098020 msgr2=0x7f413c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:23.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.064+0000 7f41425d4700 1 -- 172.21.15.7:0/69649911 shutdown_connections 2024-01-27T02:39:23.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.064+0000 7f41425d4700 1 -- 172.21.15.7:0/69649911 wait complete. 2024-01-27T02:39:23.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.065+0000 7f41425d4700 1 Processor -- start 2024-01-27T02:39:23.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.066+0000 7f41425d4700 1 -- start start 2024-01-27T02:39:23.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.066+0000 7f41425d4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f413c0c4cc0 0x7f413c15fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:23.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.066+0000 7f41425d4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f413c0ca540 0x7f413c06c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:23.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.066+0000 7f41425d4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f413c15a100 0x7f413c0b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:23.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.066+0000 7f41425d4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413c0b4700 con 0x7f413c15a100 2024-01-27T02:39:23.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.066+0000 7f41425d4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413c15ed30 con 0x7f413c0ca540 2024-01-27T02:39:23.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.066+0000 7f41425d4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413c15efd0 con 0x7f413c0c4cc0 2024-01-27T02:39:23.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.066+0000 7f4140b71700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f413c15a100 0x7f413c0b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:23.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.067+0000 7f4140b71700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f413c15a100 0x7f413c0b3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37636/0 (socket says 172.21.15.7:37636) 2024-01-27T02:39:23.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.067+0000 7f4140b71700 1 -- 172.21.15.7:0/4001912828 learned_addr learned my addr 172.21.15.7:0/4001912828 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:23.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.067+0000 7f413bfff700 1 --2- 172.21.15.7:0/4001912828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f413c0c4cc0 0x7f413c15fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:23.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.067+0000 7f413b7fe700 1 --2- 172.21.15.7:0/4001912828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f413c0ca540 0x7f413c06c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:23.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.067+0000 7f4140b71700 1 -- 172.21.15.7:0/4001912828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f413c0c4cc0 msgr2=0x7f413c15fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:23.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.067+0000 7f4140b71700 1 --2- 172.21.15.7:0/4001912828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f413c0c4cc0 0x7f413c15fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.067+0000 7f4140b71700 1 -- 172.21.15.7:0/4001912828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f413c0ca540 msgr2=0x7f413c06c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:23.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.067+0000 7f4140b71700 1 --2- 172.21.15.7:0/4001912828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f413c0ca540 0x7f413c06c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.067+0000 7f4140b71700 1 -- 172.21.15.7:0/4001912828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4134009ae0 con 0x7f413c15a100 2024-01-27T02:39:23.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.067+0000 7f413bfff700 1 --2- 172.21.15.7:0/4001912828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f413c0c4cc0 0x7f413c15fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:39:23.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.067+0000 7f4140b71700 1 --2- 172.21.15.7:0/4001912828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f413c15a100 0x7f413c0b3c10 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f4134009fd0 tx=0x7f413400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:23.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.067+0000 7f4123fff700 1 -- 172.21.15.7:0/4001912828 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4134018070 con 0x7f413c15a100 2024-01-27T02:39:23.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.067+0000 7f4123fff700 1 -- 172.21.15.7:0/4001912828 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413400f040 con 0x7f413c15a100 2024-01-27T02:39:23.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.067+0000 7f41425d4700 1 -- 172.21.15.7:0/4001912828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4130009420 con 0x7f413c15a100 2024-01-27T02:39:23.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.067+0000 7f4123fff700 1 -- 172.21.15.7:0/4001912828 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413400cbd0 con 0x7f413c15a100 2024-01-27T02:39:23.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.067+0000 7f41425d4700 1 -- 172.21.15.7:0/4001912828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f413c15f870 con 0x7f413c15a100 2024-01-27T02:39:23.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.068+0000 7f41425d4700 1 -- 172.21.15.7:0/4001912828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f413c15f3c0 con 0x7f413c15a100 2024-01-27T02:39:23.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.069+0000 7f4123fff700 1 -- 172.21.15.7:0/4001912828 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4134003020 con 0x7f413c15a100 2024-01-27T02:39:23.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.069+0000 7f4123fff700 1 --2- 172.21.15.7:0/4001912828 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4128066f30 0x7f41280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:23.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.070+0000 7f4123fff700 1 -- 172.21.15.7:0/4001912828 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4134084a20 con 0x7f413c15a100 2024-01-27T02:39:23.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.070+0000 7f413bfff700 1 --2- 172.21.15.7:0/4001912828 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4128066f30 0x7f41280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:23.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.070+0000 7f413bfff700 1 --2- 172.21.15.7:0/4001912828 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4128066f30 0x7f41280693e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f412c004f60 tx=0x7f412c007eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:23.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.074+0000 7f4123fff700 1 -- 172.21.15.7:0/4001912828 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4134051bf0 con 0x7f413c15a100 2024-01-27T02:39:23.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.282+0000 7f41425d4700 1 -- 172.21.15.7:0/4001912828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f413c15fa60 con 0x7f413c15a100 2024-01-27T02:39:23.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.282+0000 7f4123fff700 1 -- 172.21.15.7:0/4001912828 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+15330 (secure 0 0 0) 0x7f4134055210 con 0x7f413c15a100 2024-01-27T02:39:23.283 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:23.283 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":148,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:39:22.327612+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:39:20.119244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-01-27T02:39:21.293772+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:39:23.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.285+0000 7f41425d4700 1 -- 172.21.15.7:0/4001912828 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4128066f30 msgr2=0x7f41280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:23.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.285+0000 7f41425d4700 1 --2- 172.21.15.7:0/4001912828 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4128066f30 0x7f41280693e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f412c004f60 tx=0x7f412c007eb0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.285+0000 7f41425d4700 1 -- 172.21.15.7:0/4001912828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f413c15a100 msgr2=0x7f413c0b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:23.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.285+0000 7f41425d4700 1 --2- 172.21.15.7:0/4001912828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f413c15a100 0x7f413c0b3c10 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f4134009fd0 tx=0x7f413400b040 comp rx=0 tx=0).stop 2024-01-27T02:39:23.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.285+0000 7f41425d4700 1 -- 172.21.15.7:0/4001912828 shutdown_connections 2024-01-27T02:39:23.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.285+0000 7f41425d4700 1 --2- 172.21.15.7:0/4001912828 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4128066f30 0x7f41280693e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.285+0000 7f41425d4700 1 --2- 172.21.15.7:0/4001912828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f413c15a100 0x7f413c0b3c10 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.285+0000 7f41425d4700 1 --2- 172.21.15.7:0/4001912828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f413c0c4cc0 0x7f413c15fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.285+0000 7f41425d4700 1 --2- 172.21.15.7:0/4001912828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f413c0ca540 0x7f413c06c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.285+0000 7f41425d4700 1 -- 172.21.15.7:0/4001912828 >> 172.21.15.7:0/4001912828 conn(0x7f413c098020 msgr2=0x7f413c151fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:23.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.285+0000 7f41425d4700 1 -- 172.21.15.7:0/4001912828 shutdown_connections 2024-01-27T02:39:23.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.285+0000 7f41425d4700 1 -- 172.21.15.7:0/4001912828 wait complete. 2024-01-27T02:39:23.300 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:39:23.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.431+0000 7fb4d87c0700 1 Processor -- start 2024-01-27T02:39:23.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.431+0000 7fb4d87c0700 1 -- start start 2024-01-27T02:39:23.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.432+0000 7fb4d87c0700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb4d00a7fe0 0x7fb4d006f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:23.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.432+0000 7fb4d87c0700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4d015dfb0 0x7fb4d006f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:23.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.432+0000 7fb4d87c0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d015a880 con 0x7fb4d00a7c10 2024-01-27T02:39:23.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.432+0000 7fb4d87c0700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d00afde0 con 0x7fb4d00a7fe0 2024-01-27T02:39:23.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.432+0000 7fb4d87c0700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d00af950 con 0x7fb4d015dfb0 2024-01-27T02:39:23.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.432+0000 7fb4d655c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb4d00a7c10 0x7fb4d00ae340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41278/0 (socket says 172.21.15.7:41278) 2024-01-27T02:39:23.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.432+0000 7fb4d655c700 1 -- 172.21.15.7:0/2694616982 learned_addr learned my addr 172.21.15.7:0/2694616982 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:23.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.432+0000 7fb4d6d5d700 1 --2- 172.21.15.7:0/2694616982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4d015dfb0 0x7fb4d006f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:23.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.433+0000 7fb4d5d5b700 1 --2- 172.21.15.7:0/2694616982 >> v2:172.21.15.188:3300/0 conn(0x7fb4d00a7fe0 0x7fb4d006f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:23.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.433+0000 7fb4d555a700 1 -- 172.21.15.7:0/2694616982 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474004540 0 0) 0x7fb4d015a880 con 0x7fb4d00a7c10 2024-01-27T02:39:23.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.433+0000 7fb4d555a700 1 -- 172.21.15.7:0/2694616982 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4bc003570 con 0x7fb4d00a7c10 2024-01-27T02:39:23.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.433+0000 7fb4d6d5d700 1 -- 172.21.15.7:0/2694616982 >> v2:172.21.15.188:3300/0 conn(0x7fb4d00a7fe0 msgr2=0x7fb4d006f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:23.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.433+0000 7fb4d6d5d700 1 --2- 172.21.15.7:0/2694616982 >> v2:172.21.15.188:3300/0 conn(0x7fb4d00a7fe0 0x7fb4d006f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.433+0000 7fb4d6d5d700 1 -- 172.21.15.7:0/2694616982 >> v1:172.21.15.7:6789/0 conn(0x7fb4d00a7c10 legacy=0x7fb4d00ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:23.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.433+0000 7fb4d555a700 1 -- 172.21.15.7:0/2694616982 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 665410399 0 0) 0x7fb4bc003570 con 0x7fb4d00a7c10 2024-01-27T02:39:23.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.433+0000 7fb4d6d5d700 1 -- 172.21.15.7:0/2694616982 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d00aed40 con 0x7fb4d015dfb0 2024-01-27T02:39:23.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.433+0000 7fb4d6d5d700 1 --2- 172.21.15.7:0/2694616982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4d015dfb0 0x7fb4d006f8a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009e60 tx=0x7fb4c8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=263be2a5b24adedb server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:23.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.433+0000 7fb4d555a700 1 -- 172.21.15.7:0/2694616982 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c800e070 con 0x7fb4d015dfb0 2024-01-27T02:39:23.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.433+0000 7fb4d555a700 1 -- 172.21.15.7:0/2694616982 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c800b7a0 con 0x7fb4d015dfb0 2024-01-27T02:39:23.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.433+0000 7fb4d555a700 1 -- 172.21.15.7:0/2694616982 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc001470 con 0x7fb4d015dfb0 2024-01-27T02:39:23.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.434+0000 7fb4d87c0700 1 -- 172.21.15.7:0/2694616982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4d015dfb0 msgr2=0x7fb4d006f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:23.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.434+0000 7fb4d87c0700 1 --2- 172.21.15.7:0/2694616982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4d015dfb0 0x7fb4d006f8a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009e60 tx=0x7fb4c8009780 comp rx=0 tx=0).stop 2024-01-27T02:39:23.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.434+0000 7fb4d87c0700 1 -- 172.21.15.7:0/2694616982 shutdown_connections 2024-01-27T02:39:23.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.434+0000 7fb4d87c0700 1 --2- 172.21.15.7:0/2694616982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4d015dfb0 0x7fb4d006f8a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.434+0000 7fb4d87c0700 1 --2- 172.21.15.7:0/2694616982 >> v2:172.21.15.188:3300/0 conn(0x7fb4d00a7fe0 0x7fb4d006f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.434+0000 7fb4d87c0700 1 -- 172.21.15.7:0/2694616982 >> 172.21.15.7:0/2694616982 conn(0x7fb4d00679b0 msgr2=0x7fb4d0067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:23.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.434+0000 7fb4d87c0700 1 -- 172.21.15.7:0/2694616982 shutdown_connections 2024-01-27T02:39:23.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.434+0000 7fb4d87c0700 1 -- 172.21.15.7:0/2694616982 wait complete. 2024-01-27T02:39:23.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.435+0000 7fb4d87c0700 1 Processor -- start 2024-01-27T02:39:23.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.435+0000 7fb4d87c0700 1 -- start start 2024-01-27T02:39:23.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.436+0000 7fb4d87c0700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4d00a7c10 0x7fb4d01524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:23.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.436+0000 7fb4d87c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4d00a7fe0 0x7fb4d014eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:23.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.436+0000 7fb4d87c0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4d015dfb0 0x7fb4d009a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:23.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.436+0000 7fb4d87c0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d0152030 con 0x7fb4d00a7fe0 2024-01-27T02:39:23.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.436+0000 7fb4d87c0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d014f050 con 0x7fb4d015dfb0 2024-01-27T02:39:23.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.436+0000 7fb4d87c0700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d0151420 con 0x7fb4d00a7c10 2024-01-27T02:39:23.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.436+0000 7fb4d5d5b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4d00a7fe0 0x7fb4d014eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:23.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.436+0000 7fb4d5d5b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4d00a7fe0 0x7fb4d014eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37640/0 (socket says 172.21.15.7:37640) 2024-01-27T02:39:23.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.436+0000 7fb4d5d5b700 1 -- 172.21.15.7:0/1308895772 learned_addr learned my addr 172.21.15.7:0/1308895772 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:23.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.436+0000 7fb4d655c700 1 --2- 172.21.15.7:0/1308895772 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4d00a7c10 0x7fb4d01524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:23.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.436+0000 7fb4d6d5d700 1 --2- 172.21.15.7:0/1308895772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4d015dfb0 0x7fb4d009a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:23.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.437+0000 7fb4d5d5b700 1 -- 172.21.15.7:0/1308895772 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4d00a7c10 msgr2=0x7fb4d01524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:23.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.437+0000 7fb4d5d5b700 1 --2- 172.21.15.7:0/1308895772 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4d00a7c10 0x7fb4d01524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.437+0000 7fb4d5d5b700 1 -- 172.21.15.7:0/1308895772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4d015dfb0 msgr2=0x7fb4d009a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:23.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.437+0000 7fb4d5d5b700 1 --2- 172.21.15.7:0/1308895772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4d015dfb0 0x7fb4d009a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.437+0000 7fb4d5d5b700 1 -- 172.21.15.7:0/1308895772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c8009a20 con 0x7fb4d00a7fe0 2024-01-27T02:39:23.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.437+0000 7fb4d6d5d700 1 --2- 172.21.15.7:0/1308895772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4d015dfb0 0x7fb4d009a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:23.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.437+0000 7fb4d5d5b700 1 --2- 172.21.15.7:0/1308895772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4d00a7fe0 0x7fb4d014eb10 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000ec80 tx=0x7fb4c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:23.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.437+0000 7fb4c67fc700 1 -- 172.21.15.7:0/1308895772 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c000c850 con 0x7fb4d00a7fe0 2024-01-27T02:39:23.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.437+0000 7fb4c67fc700 1 -- 172.21.15.7:0/1308895772 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c000c9b0 con 0x7fb4d00a7fe0 2024-01-27T02:39:23.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.437+0000 7fb4d87c0700 1 -- 172.21.15.7:0/1308895772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d0151700 con 0x7fb4d00a7fe0 2024-01-27T02:39:23.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.437+0000 7fb4c67fc700 1 -- 172.21.15.7:0/1308895772 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c000cd70 con 0x7fb4d00a7fe0 2024-01-27T02:39:23.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.437+0000 7fb4d87c0700 1 -- 172.21.15.7:0/1308895772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d0151cc0 con 0x7fb4d00a7fe0 2024-01-27T02:39:23.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.438+0000 7fb4d87c0700 1 -- 172.21.15.7:0/1308895772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d0151e50 con 0x7fb4d00a7fe0 2024-01-27T02:39:23.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.439+0000 7fb4c67fc700 1 -- 172.21.15.7:0/1308895772 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb4c0013680 con 0x7fb4d00a7fe0 2024-01-27T02:39:23.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.439+0000 7fb4c67fc700 1 --2- 172.21.15.7:0/1308895772 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb4b4066f30 0x7fb4b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:23.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.439+0000 7fb4c67fc700 1 -- 172.21.15.7:0/1308895772 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb4c00847d0 con 0x7fb4d00a7fe0 2024-01-27T02:39:23.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.439+0000 7fb4d655c700 1 --2- 172.21.15.7:0/1308895772 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb4b4066f30 0x7fb4b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:23.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.440+0000 7fb4d655c700 1 --2- 172.21.15.7:0/1308895772 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb4b4066f30 0x7fb4b40693e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc004e60 tx=0x7fb4cc001a10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:23.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.444+0000 7fb4c67fc700 1 -- 172.21.15.7:0/1308895772 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4c0051a40 con 0x7fb4d00a7fe0 2024-01-27T02:39:23.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.650+0000 7fb4d87c0700 1 -- 172.21.15.7:0/1308895772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4d01521e0 con 0x7fb4d00a7fe0 2024-01-27T02:39:23.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.651+0000 7fb4c67fc700 1 -- 172.21.15.7:0/1308895772 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+15330 (secure 0 0 0) 0x7fb4d01521e0 con 0x7fb4d00a7fe0 2024-01-27T02:39:23.651 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:23.651 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":148,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:39:22.327612+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:39:20.119244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-01-27T02:39:21.293772+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:39:23.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.653+0000 7fb4d87c0700 1 -- 172.21.15.7:0/1308895772 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb4b4066f30 msgr2=0x7fb4b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:23.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.653+0000 7fb4d87c0700 1 --2- 172.21.15.7:0/1308895772 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb4b4066f30 0x7fb4b40693e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc004e60 tx=0x7fb4cc001a10 comp rx=0 tx=0).stop 2024-01-27T02:39:23.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.653+0000 7fb4d87c0700 1 -- 172.21.15.7:0/1308895772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4d00a7fe0 msgr2=0x7fb4d014eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:23.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.653+0000 7fb4d87c0700 1 --2- 172.21.15.7:0/1308895772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4d00a7fe0 0x7fb4d014eb10 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000ec80 tx=0x7fb4c00091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.653+0000 7fb4d87c0700 1 -- 172.21.15.7:0/1308895772 shutdown_connections 2024-01-27T02:39:23.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.653+0000 7fb4d87c0700 1 --2- 172.21.15.7:0/1308895772 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fb4b4066f30 0x7fb4b40693e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.653+0000 7fb4d87c0700 1 --2- 172.21.15.7:0/1308895772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4d015dfb0 0x7fb4d009a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.653+0000 7fb4d87c0700 1 --2- 172.21.15.7:0/1308895772 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4d00a7c10 0x7fb4d01524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.653+0000 7fb4d87c0700 1 --2- 172.21.15.7:0/1308895772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4d00a7fe0 0x7fb4d014eb10 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.653+0000 7fb4d87c0700 1 -- 172.21.15.7:0/1308895772 >> 172.21.15.7:0/1308895772 conn(0x7fb4d00679b0 msgr2=0x7fb4d006dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:23.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.653+0000 7fb4d87c0700 1 -- 172.21.15.7:0/1308895772 shutdown_connections 2024-01-27T02:39:23.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.653+0000 7fb4d87c0700 1 -- 172.21.15.7:0/1308895772 wait complete. 2024-01-27T02:39:23.667 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-27T02:39:23.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.803+0000 7f92950c8700 1 Processor -- start 2024-01-27T02:39:23.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.803+0000 7f92950c8700 1 -- start start 2024-01-27T02:39:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.803+0000 7f92950c8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f92900680c0 0x7f92900693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:23.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.803+0000 7f92950c8700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f92900cc8d0 0x7f92900c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:23.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.804+0000 7f92950c8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9290069930 con 0x7f92900b0f70 2024-01-27T02:39:23.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.804+0000 7f92950c8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9290068840 con 0x7f92900680c0 2024-01-27T02:39:23.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.804+0000 7f92950c8700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f929015e820 con 0x7f92900cc8d0 2024-01-27T02:39:23.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.804+0000 7f928e59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f92900680c0 0x7f92900693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:23.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.804+0000 7f928ed9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f92900b0f70 0x7f92900679b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41284/0 (socket says 172.21.15.7:41284) 2024-01-27T02:39:23.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.804+0000 7f928ed9d700 1 -- 172.21.15.7:0/175686081 learned_addr learned my addr 172.21.15.7:0/175686081 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:23.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.804+0000 7f928f59e700 1 --2- 172.21.15.7:0/175686081 >> v2:172.21.15.188:3300/0 conn(0x7f92900cc8d0 0x7f92900c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:23.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.804+0000 7f928dd9b700 1 -- 172.21.15.7:0/175686081 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747090180 0 0) 0x7f9290069930 con 0x7f92900b0f70 2024-01-27T02:39:23.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.804+0000 7f928dd9b700 1 -- 172.21.15.7:0/175686081 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9278003570 con 0x7f92900b0f70 2024-01-27T02:39:23.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.804+0000 7f928e59c700 1 -- 172.21.15.7:0/175686081 >> v2:172.21.15.188:3300/0 conn(0x7f92900cc8d0 msgr2=0x7f92900c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:23.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.804+0000 7f928e59c700 1 --2- 172.21.15.7:0/175686081 >> v2:172.21.15.188:3300/0 conn(0x7f92900cc8d0 0x7f92900c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.804+0000 7f928e59c700 1 -- 172.21.15.7:0/175686081 >> v1:172.21.15.7:6789/0 conn(0x7f92900b0f70 legacy=0x7f92900679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:23.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.804+0000 7f928e59c700 1 -- 172.21.15.7:0/175686081 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929015dc10 con 0x7f92900680c0 2024-01-27T02:39:23.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.804+0000 7f928e59c700 1 --2- 172.21.15.7:0/175686081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92900680c0 0x7f92900693f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f92900cfda0 tx=0x7f92840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16edacc8405ae66 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:23.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.805+0000 7f928dd9b700 1 -- 172.21.15.7:0/175686081 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9284011630 con 0x7f92900680c0 2024-01-27T02:39:23.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.805+0000 7f928dd9b700 1 -- 172.21.15.7:0/175686081 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9284011790 con 0x7f92900680c0 2024-01-27T02:39:23.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.805+0000 7f928dd9b700 1 -- 172.21.15.7:0/175686081 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9284011b30 con 0x7f92900680c0 2024-01-27T02:39:23.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.805+0000 7f92950c8700 1 -- 172.21.15.7:0/175686081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92900680c0 msgr2=0x7f92900693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:23.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.805+0000 7f92950c8700 1 --2- 172.21.15.7:0/175686081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92900680c0 0x7f92900693f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f92900cfda0 tx=0x7f92840091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.805+0000 7f92950c8700 1 -- 172.21.15.7:0/175686081 shutdown_connections 2024-01-27T02:39:23.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.805+0000 7f92950c8700 1 --2- 172.21.15.7:0/175686081 >> v2:172.21.15.188:3300/0 conn(0x7f92900cc8d0 0x7f92900c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.805+0000 7f92950c8700 1 --2- 172.21.15.7:0/175686081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92900680c0 0x7f92900693f0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.805+0000 7f92950c8700 1 -- 172.21.15.7:0/175686081 >> 172.21.15.7:0/175686081 conn(0x7f9290063580 msgr2=0x7f9290063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:23.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.805+0000 7f92950c8700 1 -- 172.21.15.7:0/175686081 shutdown_connections 2024-01-27T02:39:23.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.805+0000 7f92950c8700 1 -- 172.21.15.7:0/175686081 wait complete. 2024-01-27T02:39:23.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.806+0000 7f92950c8700 1 Processor -- start 2024-01-27T02:39:23.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.807+0000 7f92950c8700 1 -- start start 2024-01-27T02:39:23.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.807+0000 7f92950c8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92900680c0 0x7f92900cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:23.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.807+0000 7f92950c8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92900b0f70 0x7f929014b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:23.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.807+0000 7f92950c8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92900cc8d0 0x7f929015caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:23.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.807+0000 7f92950c8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92900c9320 con 0x7f92900b0f70 2024-01-27T02:39:23.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.807+0000 7f92950c8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92900c7570 con 0x7f92900680c0 2024-01-27T02:39:23.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.807+0000 7f92950c8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929015b880 con 0x7f92900cc8d0 2024-01-27T02:39:23.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.807+0000 7f928e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92900b0f70 0x7f929014b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:23.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.807+0000 7f928e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92900b0f70 0x7f929014b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37648/0 (socket says 172.21.15.7:37648) 2024-01-27T02:39:23.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.807+0000 7f928e59c700 1 -- 172.21.15.7:0/35043749 learned_addr learned my addr 172.21.15.7:0/35043749 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:23.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.808+0000 7f928ed9d700 1 --2- 172.21.15.7:0/35043749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92900680c0 0x7f92900cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:23.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.808+0000 7f928f59e700 1 --2- 172.21.15.7:0/35043749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92900cc8d0 0x7f929015caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:23.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.808+0000 7f928e59c700 1 -- 172.21.15.7:0/35043749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92900cc8d0 msgr2=0x7f929015caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:23.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.808+0000 7f928e59c700 1 --2- 172.21.15.7:0/35043749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92900cc8d0 0x7f929015caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.808+0000 7f928e59c700 1 -- 172.21.15.7:0/35043749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92900680c0 msgr2=0x7f92900cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:23.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.808+0000 7f928e59c700 1 --2- 172.21.15.7:0/35043749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92900680c0 0x7f92900cbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:23.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.808+0000 7f928e59c700 1 -- 172.21.15.7:0/35043749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9284009420 con 0x7f92900b0f70 2024-01-27T02:39:23.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.808+0000 7f928f59e700 1 --2- 172.21.15.7:0/35043749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92900cc8d0 0x7f929015caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:23.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.808+0000 7f928e59c700 1 --2- 172.21.15.7:0/35043749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92900b0f70 0x7f929014b6d0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f9284013000 tx=0x7f9284017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:23.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.808+0000 7f9276ffd700 1 -- 172.21.15.7:0/35043749 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9284011b10 con 0x7f92900b0f70 2024-01-27T02:39:23.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.808+0000 7f9276ffd700 1 -- 172.21.15.7:0/35043749 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92840253f0 con 0x7f92900b0f70 2024-01-27T02:39:23.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.808+0000 7f92950c8700 1 -- 172.21.15.7:0/35043749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929015bb60 con 0x7f92900b0f70 2024-01-27T02:39:23.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.808+0000 7f9276ffd700 1 -- 172.21.15.7:0/35043749 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9284025790 con 0x7f92900b0f70 2024-01-27T02:39:23.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.808+0000 7f92950c8700 1 -- 172.21.15.7:0/35043749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92900c71f0 con 0x7f92900b0f70 2024-01-27T02:39:23.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.809+0000 7f92950c8700 1 -- 172.21.15.7:0/35043749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92900c76e0 con 0x7f92900b0f70 2024-01-27T02:39:23.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.810+0000 7f9276ffd700 1 -- 172.21.15.7:0/35043749 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9284025b80 con 0x7f92900b0f70 2024-01-27T02:39:23.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.810+0000 7f9276ffd700 1 --2- 172.21.15.7:0/35043749 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f927c066f30 0x7f927c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:23.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.810+0000 7f9276ffd700 1 -- 172.21.15.7:0/35043749 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9284085d00 con 0x7f92900b0f70 2024-01-27T02:39:23.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.810+0000 7f928ed9d700 1 --2- 172.21.15.7:0/35043749 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f927c066f30 0x7f927c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:23.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.811+0000 7f928ed9d700 1 --2- 172.21.15.7:0/35043749 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f927c066f30 0x7f927c0693e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9280007fd0 tx=0x7f9280009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:23.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:23.815+0000 7f9276ffd700 1 -- 172.21.15.7:0/35043749 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9284052f70 con 0x7f92900b0f70 2024-01-27T02:39:24.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:24.068+0000 7f92950c8700 1 -- 172.21.15.7:0/35043749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9290072060 con 0x7f92900b0f70 2024-01-27T02:39:24.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:24.911+0000 7f9276ffd700 1 -- 172.21.15.7:0/35043749 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v76) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9284056590 con 0x7f92900b0f70 2024-01-27T02:39:24.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:24.913+0000 7f92950c8700 1 -- 172.21.15.7:0/35043749 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f927c066f30 msgr2=0x7f927c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:24.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:24.913+0000 7f92950c8700 1 --2- 172.21.15.7:0/35043749 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f927c066f30 0x7f927c0693e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9280007fd0 tx=0x7f9280009040 comp rx=0 tx=0).stop 2024-01-27T02:39:24.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:24.913+0000 7f92950c8700 1 -- 172.21.15.7:0/35043749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92900b0f70 msgr2=0x7f929014b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:24.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:24.913+0000 7f92950c8700 1 --2- 172.21.15.7:0/35043749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92900b0f70 0x7f929014b6d0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f9284013000 tx=0x7f9284017000 comp rx=0 tx=0).stop 2024-01-27T02:39:24.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:24.913+0000 7f92950c8700 1 -- 172.21.15.7:0/35043749 shutdown_connections 2024-01-27T02:39:24.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:24.913+0000 7f92950c8700 1 --2- 172.21.15.7:0/35043749 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f927c066f30 0x7f927c0693e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:24.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:24.913+0000 7f92950c8700 1 --2- 172.21.15.7:0/35043749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92900cc8d0 0x7f929015caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:24.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:24.913+0000 7f92950c8700 1 --2- 172.21.15.7:0/35043749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92900680c0 0x7f92900cbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:24.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:24.913+0000 7f92950c8700 1 --2- 172.21.15.7:0/35043749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92900b0f70 0x7f929014b6d0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:24.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:24.913+0000 7f92950c8700 1 -- 172.21.15.7:0/35043749 >> 172.21.15.7:0/35043749 conn(0x7f9290063580 msgr2=0x7f92900cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:24.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:24.914+0000 7f92950c8700 1 -- 172.21.15.7:0/35043749 shutdown_connections 2024-01-27T02:39:24.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:24.914+0000 7f92950c8700 1 -- 172.21.15.7:0/35043749 wait complete. 2024-01-27T02:39:24.929 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-27T02:39:25.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.068+0000 7f09df754700 1 Processor -- start 2024-01-27T02:39:25.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.068+0000 7f09df754700 1 -- start start 2024-01-27T02:39:25.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.069+0000 7f09df754700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f09d81416c0 0x7f09d80cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.069+0000 7f09df754700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f09d80cb210 0x7f09d80cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.069+0000 7f09df754700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d815c7d0 con 0x7f09d8141a90 2024-01-27T02:39:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.069+0000 7f09df754700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d80cd9c0 con 0x7f09d81416c0 2024-01-27T02:39:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.069+0000 7f09df754700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d80c9640 con 0x7f09d80cb210 2024-01-27T02:39:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.069+0000 7f09dd4f0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f09d81416c0 0x7f09d80cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.069+0000 7f09dd4f0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f09d81416c0 0x7f09d80cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37656/0 (socket says 172.21.15.7:37656) 2024-01-27T02:39:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.069+0000 7f09dd4f0700 1 -- 172.21.15.7:0/2920390290 learned_addr learned my addr 172.21.15.7:0/2920390290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.070+0000 7f09dd4f0700 1 -- 172.21.15.7:0/2920390290 >> v2:172.21.15.188:3300/0 conn(0x7f09d80cb210 msgr2=0x7f09d80cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.070+0000 7f09dd4f0700 1 --2- 172.21.15.7:0/2920390290 >> v2:172.21.15.188:3300/0 conn(0x7f09d80cb210 0x7f09d80cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.070+0000 7f09dd4f0700 1 -- 172.21.15.7:0/2920390290 >> v1:172.21.15.188:6789/0 conn(0x7f09d8141a90 legacy=0x7f09d80cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.070+0000 7f09dd4f0700 1 -- 172.21.15.7:0/2920390290 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d815bbc0 con 0x7f09d81416c0 2024-01-27T02:39:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.070+0000 7f09dd4f0700 1 --2- 172.21.15.7:0/2920390290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d81416c0 0x7f09d80cb6e0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f09d806f1e0 tx=0x7f09d4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f69185a7bebcc9 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.070+0000 7f09cbfff700 1 -- 172.21.15.7:0/2920390290 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d4005c10 con 0x7f09d81416c0 2024-01-27T02:39:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.070+0000 7f09cbfff700 1 -- 172.21.15.7:0/2920390290 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d4005d70 con 0x7f09d81416c0 2024-01-27T02:39:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.070+0000 7f09cbfff700 1 -- 172.21.15.7:0/2920390290 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d400b880 con 0x7f09d81416c0 2024-01-27T02:39:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.071+0000 7f09df754700 1 -- 172.21.15.7:0/2920390290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d81416c0 msgr2=0x7f09d80cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.071+0000 7f09df754700 1 --2- 172.21.15.7:0/2920390290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d81416c0 0x7f09d80cb6e0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f09d806f1e0 tx=0x7f09d4009220 comp rx=0 tx=0).stop 2024-01-27T02:39:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.071+0000 7f09df754700 1 -- 172.21.15.7:0/2920390290 shutdown_connections 2024-01-27T02:39:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.071+0000 7f09df754700 1 --2- 172.21.15.7:0/2920390290 >> v2:172.21.15.188:3300/0 conn(0x7f09d80cb210 0x7f09d80cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.071+0000 7f09df754700 1 --2- 172.21.15.7:0/2920390290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d81416c0 0x7f09d80cb6e0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.071+0000 7f09df754700 1 -- 172.21.15.7:0/2920390290 >> 172.21.15.7:0/2920390290 conn(0x7f09d8063580 msgr2=0x7f09d8063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.071+0000 7f09df754700 1 -- 172.21.15.7:0/2920390290 shutdown_connections 2024-01-27T02:39:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.071+0000 7f09df754700 1 -- 172.21.15.7:0/2920390290 wait complete. 2024-01-27T02:39:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.072+0000 7f09df754700 1 Processor -- start 2024-01-27T02:39:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.072+0000 7f09df754700 1 -- start start 2024-01-27T02:39:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.073+0000 7f09df754700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d80cb210 0x7f09d80b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.073+0000 7f09df754700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09d81416c0 0x7f09d8157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.073+0000 7f09df754700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09d8141a90 0x7f09d816da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.073+0000 7f09df754700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d80679b0 con 0x7f09d80cb210 2024-01-27T02:39:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.073+0000 7f09df754700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d80b1430 con 0x7f09d81416c0 2024-01-27T02:39:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.073+0000 7f09df754700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d80c71d0 con 0x7f09d8141a90 2024-01-27T02:39:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.073+0000 7f09dd4f0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d80cb210 0x7f09d80b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.073+0000 7f09dd4f0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d80cb210 0x7f09d80b0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37664/0 (socket says 172.21.15.7:37664) 2024-01-27T02:39:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.073+0000 7f09dd4f0700 1 -- 172.21.15.7:0/3744954992 learned_addr learned my addr 172.21.15.7:0/3744954992 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.073+0000 7f09dd4f0700 1 -- 172.21.15.7:0/3744954992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09d8141a90 msgr2=0x7f09d816da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.073+0000 7f09dccef700 1 --2- 172.21.15.7:0/3744954992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09d81416c0 0x7f09d8157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:25.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.073+0000 7f09ddcf1700 1 --2- 172.21.15.7:0/3744954992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09d8141a90 0x7f09d816da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:25.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.073+0000 7f09dd4f0700 1 --2- 172.21.15.7:0/3744954992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09d8141a90 0x7f09d816da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:25.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.074+0000 7f09dd4f0700 1 -- 172.21.15.7:0/3744954992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09d81416c0 msgr2=0x7f09d8157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:25.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.074+0000 7f09dd4f0700 1 --2- 172.21.15.7:0/3744954992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09d81416c0 0x7f09d8157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:25.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.074+0000 7f09dd4f0700 1 -- 172.21.15.7:0/3744954992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d4009480 con 0x7f09d80cb210 2024-01-27T02:39:25.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.074+0000 7f09dccef700 1 --2- 172.21.15.7:0/3744954992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09d81416c0 0x7f09d8157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:39:25.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.074+0000 7f09ddcf1700 1 --2- 172.21.15.7:0/3744954992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09d8141a90 0x7f09d816da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:39:25.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.074+0000 7f09dd4f0700 1 --2- 172.21.15.7:0/3744954992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d80cb210 0x7f09d80b0ef0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f09d806e580 tx=0x7f09d4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:25.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.074+0000 7f09c97fa700 1 -- 172.21.15.7:0/3744954992 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d4011890 con 0x7f09d80cb210 2024-01-27T02:39:25.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.074+0000 7f09c97fa700 1 -- 172.21.15.7:0/3744954992 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d4020070 con 0x7f09d80cb210 2024-01-27T02:39:25.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.074+0000 7f09df754700 1 -- 172.21.15.7:0/3744954992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d80c74b0 con 0x7f09d80cb210 2024-01-27T02:39:25.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.074+0000 7f09c97fa700 1 -- 172.21.15.7:0/3744954992 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d4011c60 con 0x7f09d80cb210 2024-01-27T02:39:25.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.074+0000 7f09df754700 1 -- 172.21.15.7:0/3744954992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d8067ee0 con 0x7f09d80cb210 2024-01-27T02:39:25.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.075+0000 7f09df754700 1 -- 172.21.15.7:0/3744954992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d8072060 con 0x7f09d80cb210 2024-01-27T02:39:25.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.075+0000 7f09c97fa700 1 -- 172.21.15.7:0/3744954992 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f09d4022030 con 0x7f09d80cb210 2024-01-27T02:39:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.076+0000 7f09c97fa700 1 --2- 172.21.15.7:0/3744954992 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f09bc066f30 0x7f09bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.076+0000 7f09c97fa700 1 -- 172.21.15.7:0/3744954992 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f09d4086100 con 0x7f09d80cb210 2024-01-27T02:39:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.076+0000 7f09dccef700 1 --2- 172.21.15.7:0/3744954992 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f09bc066f30 0x7f09bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.077+0000 7f09dccef700 1 --2- 172.21.15.7:0/3744954992 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f09bc066f30 0x7f09bc0693e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f09d816b5a0 tx=0x7f09cc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:25.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.081+0000 7f09c97fa700 1 -- 172.21.15.7:0/3744954992 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09d4053370 con 0x7f09d80cb210 2024-01-27T02:39:25.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.337+0000 7f09df754700 1 -- 172.21.15.7:0/3744954992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f09d815bea0 con 0x7f09d80cb210 2024-01-27T02:39:25.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.915+0000 7f09c97fa700 1 -- 172.21.15.7:0/3744954992 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v77) v1 ==== 122+0+0 (secure 0 0 0) 0x7f09d4056990 con 0x7f09d80cb210 2024-01-27T02:39:25.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.918+0000 7f09df754700 1 -- 172.21.15.7:0/3744954992 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f09bc066f30 msgr2=0x7f09bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:25.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.918+0000 7f09df754700 1 --2- 172.21.15.7:0/3744954992 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f09bc066f30 0x7f09bc0693e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f09d816b5a0 tx=0x7f09cc009040 comp rx=0 tx=0).stop 2024-01-27T02:39:25.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.918+0000 7f09df754700 1 -- 172.21.15.7:0/3744954992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d80cb210 msgr2=0x7f09d80b0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:25.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.918+0000 7f09df754700 1 --2- 172.21.15.7:0/3744954992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d80cb210 0x7f09d80b0ef0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f09d806e580 tx=0x7f09d4016000 comp rx=0 tx=0).stop 2024-01-27T02:39:25.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.918+0000 7f09df754700 1 -- 172.21.15.7:0/3744954992 shutdown_connections 2024-01-27T02:39:25.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.918+0000 7f09df754700 1 --2- 172.21.15.7:0/3744954992 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f09bc066f30 0x7f09bc0693e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:25.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.918+0000 7f09df754700 1 --2- 172.21.15.7:0/3744954992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09d8141a90 0x7f09d816da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:25.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.918+0000 7f09df754700 1 --2- 172.21.15.7:0/3744954992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d80cb210 0x7f09d80b0ef0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:25.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.918+0000 7f09df754700 1 --2- 172.21.15.7:0/3744954992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09d81416c0 0x7f09d8157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:25.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.918+0000 7f09df754700 1 -- 172.21.15.7:0/3744954992 >> 172.21.15.7:0/3744954992 conn(0x7f09d8063580 msgr2=0x7f09d806bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:25.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.919+0000 7f09df754700 1 -- 172.21.15.7:0/3744954992 shutdown_connections 2024-01-27T02:39:25.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:25.919+0000 7f09df754700 1 -- 172.21.15.7:0/3744954992 wait complete. 2024-01-27T02:39:25.934 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:39:26.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.070+0000 7facd7710700 1 Processor -- start 2024-01-27T02:39:26.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.070+0000 7facd7710700 1 -- start start 2024-01-27T02:39:26.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.071+0000 7facd7710700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7facd01404e0 0x7facd00cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:26.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.071+0000 7facd7710700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facd00b09f0 0x7facd00cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:26.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.071+0000 7facd7710700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facd00cce30 con 0x7facd01408b0 2024-01-27T02:39:26.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.071+0000 7facd7710700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facd006b3b0 con 0x7facd01404e0 2024-01-27T02:39:26.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.071+0000 7facd7710700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd00cc3e0 con 0x7facd00b09f0 2024-01-27T02:39:26.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.071+0000 7facd5cad700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7facd01408b0 0x7facd006e2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41296/0 (socket says 172.21.15.7:41296) 2024-01-27T02:39:26.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.071+0000 7facd5cad700 1 -- 172.21.15.7:0/2570979516 learned_addr learned my addr 172.21.15.7:0/2570979516 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:26.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.071+0000 7facd4cab700 1 --2- 172.21.15.7:0/2570979516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facd00b09f0 0x7facd00cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:26.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.071+0000 7facd54ac700 1 --2- 172.21.15.7:0/2570979516 >> v2:172.21.15.188:3300/0 conn(0x7facd01404e0 0x7facd00cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:26.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facc7fff700 1 -- 172.21.15.7:0/2570979516 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3415978830 0 0) 0x7facd00cce30 con 0x7facd01408b0 2024-01-27T02:39:26.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facc7fff700 1 -- 172.21.15.7:0/2570979516 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facbc003570 con 0x7facd01408b0 2024-01-27T02:39:26.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facc7fff700 1 -- 172.21.15.7:0/2570979516 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4146466475 0 0) 0x7facbc003570 con 0x7facd01408b0 2024-01-27T02:39:26.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facc7fff700 1 -- 172.21.15.7:0/2570979516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facd00b09f0 msgr2=0x7facd00cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:26.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facc7fff700 1 --2- 172.21.15.7:0/2570979516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facd00b09f0 0x7facd00cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facc7fff700 1 -- 172.21.15.7:0/2570979516 >> v2:172.21.15.188:3300/0 conn(0x7facd01404e0 msgr2=0x7facd00cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:26.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facc7fff700 1 --2- 172.21.15.7:0/2570979516 >> v2:172.21.15.188:3300/0 conn(0x7facd01404e0 0x7facd00cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facc7fff700 1 -- 172.21.15.7:0/2570979516 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd00cc6c0 con 0x7facd01408b0 2024-01-27T02:39:26.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facc7fff700 1 -- 172.21.15.7:0/2570979516 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7facc80017f0 con 0x7facd01408b0 2024-01-27T02:39:26.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facd4cab700 1 --2- 172.21.15.7:0/2570979516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facd00b09f0 0x7facd00cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:39:26.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facd54ac700 1 --2- 172.21.15.7:0/2570979516 >> v2:172.21.15.188:3300/0 conn(0x7facd01404e0 0x7facd00cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:39:26.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facc7fff700 1 -- 172.21.15.7:0/2570979516 >> v1:172.21.15.7:6789/0 conn(0x7facd01408b0 legacy=0x7facd006e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:26.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facc7fff700 1 --2- 172.21.15.7:0/2570979516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facbc003a60 0x7facbc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:26.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facc7fff700 1 --2- 172.21.15.7:0/2570979516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facd00b09f0 0x7facbc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:26.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facc7fff700 1 --2- 172.21.15.7:0/2570979516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facbc004d50 0x7facbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:26.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facc7fff700 1 -- 172.21.15.7:0/2570979516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd006b3b0 con 0x7facbc003a60 2024-01-27T02:39:26.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facc7fff700 1 -- 172.21.15.7:0/2570979516 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd00cc3e0 con 0x7facbc004d50 2024-01-27T02:39:26.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facc7fff700 1 -- 172.21.15.7:0/2570979516 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facbc00ba60 con 0x7facd00b09f0 2024-01-27T02:39:26.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facd54ac700 1 --2- 172.21.15.7:0/2570979516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facbc003a60 0x7facbc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:26.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facd4cab700 1 --2- 172.21.15.7:0/2570979516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facd00b09f0 0x7facbc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:26.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facd5cad700 1 --2- 172.21.15.7:0/2570979516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facbc004d50 0x7facbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:26.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facd54ac700 1 -- 172.21.15.7:0/2570979516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facd00b09f0 msgr2=0x7facbc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:26.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facd54ac700 1 --2- 172.21.15.7:0/2570979516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facd00b09f0 0x7facbc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facd54ac700 1 -- 172.21.15.7:0/2570979516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facbc004d50 msgr2=0x7facbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:26.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facd54ac700 1 --2- 172.21.15.7:0/2570979516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facbc004d50 0x7facbc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.072+0000 7facd54ac700 1 -- 172.21.15.7:0/2570979516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc0009420 con 0x7facbc003a60 2024-01-27T02:39:26.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.073+0000 7facd54ac700 1 --2- 172.21.15.7:0/2570979516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facbc003a60 0x7facbc003e70 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7faccc009e20 tx=0x7faccc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d79b4a2ecdd25f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:26.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.073+0000 7facc7fff700 1 -- 172.21.15.7:0/2570979516 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc00e8c0 con 0x7facbc003a60 2024-01-27T02:39:26.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.073+0000 7facc7fff700 1 -- 172.21.15.7:0/2570979516 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faccc00ea20 con 0x7facbc003a60 2024-01-27T02:39:26.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.073+0000 7facc7fff700 1 -- 172.21.15.7:0/2570979516 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc00edc0 con 0x7facbc003a60 2024-01-27T02:39:26.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.073+0000 7facd7710700 1 -- 172.21.15.7:0/2570979516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facbc003a60 msgr2=0x7facbc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:26.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.073+0000 7facd7710700 1 --2- 172.21.15.7:0/2570979516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facbc003a60 0x7facbc003e70 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7faccc009e20 tx=0x7faccc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.073+0000 7facd5cad700 1 -- 172.21.15.7:0/2570979516 reap_dead start 2024-01-27T02:39:26.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.073+0000 7facd7710700 1 -- 172.21.15.7:0/2570979516 shutdown_connections 2024-01-27T02:39:26.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.073+0000 7facd7710700 1 -- 172.21.15.7:0/2570979516 >> 172.21.15.7:0/2570979516 conn(0x7facd00cdc30 msgr2=0x7facd00ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:26.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.074+0000 7facd7710700 1 -- 172.21.15.7:0/2570979516 shutdown_connections 2024-01-27T02:39:26.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.074+0000 7facd7710700 1 -- 172.21.15.7:0/2570979516 wait complete. 2024-01-27T02:39:26.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.075+0000 7facd7710700 1 Processor -- start 2024-01-27T02:39:26.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.075+0000 7facd7710700 1 -- start start 2024-01-27T02:39:26.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.076+0000 7facd7710700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facbc003a60 0x7facd00b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:26.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.076+0000 7facd7710700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd0156ca0 0x7facd015a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:26.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.076+0000 7facd7710700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facd00b2510 0x7facd015a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:26.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.076+0000 7facd7710700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd015ae70 con 0x7facd0156ca0 2024-01-27T02:39:26.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.076+0000 7facd7710700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd00b0f30 con 0x7facbc003a60 2024-01-27T02:39:26.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.076+0000 7facd7710700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd00b1210 con 0x7facd00b2510 2024-01-27T02:39:26.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.076+0000 7facd4cab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd0156ca0 0x7facd015a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:26.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.076+0000 7facd4cab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd0156ca0 0x7facd015a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37674/0 (socket says 172.21.15.7:37674) 2024-01-27T02:39:26.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.076+0000 7facd4cab700 1 -- 172.21.15.7:0/4220956578 learned_addr learned my addr 172.21.15.7:0/4220956578 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:26.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.076+0000 7facd5cad700 1 --2- 172.21.15.7:0/4220956578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facd00b2510 0x7facd015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:26.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.076+0000 7facd54ac700 1 --2- 172.21.15.7:0/4220956578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facbc003a60 0x7facd00b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:26.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.076+0000 7facd4cab700 1 -- 172.21.15.7:0/4220956578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facd00b2510 msgr2=0x7facd015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:26.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.076+0000 7facd4cab700 1 --2- 172.21.15.7:0/4220956578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facd00b2510 0x7facd015a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.076+0000 7facd4cab700 1 -- 172.21.15.7:0/4220956578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facbc003a60 msgr2=0x7facd00b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:26.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.076+0000 7facd4cab700 1 --2- 172.21.15.7:0/4220956578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facbc003a60 0x7facd00b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.076+0000 7facd4cab700 1 -- 172.21.15.7:0/4220956578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd00b14f0 con 0x7facd0156ca0 2024-01-27T02:39:26.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.076+0000 7facd54ac700 1 --2- 172.21.15.7:0/4220956578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facbc003a60 0x7facd00b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:26.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.076+0000 7facd4cab700 1 --2- 172.21.15.7:0/4220956578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd0156ca0 0x7facd015a3f0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7facc0011cc0 tx=0x7facc00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:26.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.077+0000 7facc57fa700 1 -- 172.21.15.7:0/4220956578 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc0011ed0 con 0x7facd0156ca0 2024-01-27T02:39:26.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.077+0000 7facc57fa700 1 -- 172.21.15.7:0/4220956578 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc000fcd0 con 0x7facd0156ca0 2024-01-27T02:39:26.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.077+0000 7facd7710700 1 -- 172.21.15.7:0/4220956578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd00b17d0 con 0x7facd0156ca0 2024-01-27T02:39:26.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.077+0000 7facc57fa700 1 -- 172.21.15.7:0/4220956578 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc001d6b0 con 0x7facd0156ca0 2024-01-27T02:39:26.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.077+0000 7facd7710700 1 -- 172.21.15.7:0/4220956578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd013c3e0 con 0x7facd0156ca0 2024-01-27T02:39:26.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.078+0000 7facd7710700 1 -- 172.21.15.7:0/4220956578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd004c710 con 0x7facd0156ca0 2024-01-27T02:39:26.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.078+0000 7facc57fa700 1 -- 172.21.15.7:0/4220956578 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7facc001dae0 con 0x7facd0156ca0 2024-01-27T02:39:26.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.078+0000 7facc57fa700 1 --2- 172.21.15.7:0/4220956578 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7facb4066f30 0x7facb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:26.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.079+0000 7facc57fa700 1 -- 172.21.15.7:0/4220956578 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5705+0+0 (secure 0 0 0) 0x7facc0018070 con 0x7facd0156ca0 2024-01-27T02:39:26.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.079+0000 7facd54ac700 1 --2- 172.21.15.7:0/4220956578 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7facb4066f30 0x7facb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:26.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.079+0000 7facd54ac700 1 --2- 172.21.15.7:0/4220956578 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7facb4066f30 0x7facb40693e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7faccc0091e0 tx=0x7faccc01a000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:26.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.083+0000 7facc57fa700 1 -- 172.21.15.7:0/4220956578 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7facc00527a0 con 0x7facd0156ca0 2024-01-27T02:39:26.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.339+0000 7facd7710700 1 -- 172.21.15.7:0/4220956578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7facd00ccd10 con 0x7facd0156ca0 2024-01-27T02:39:26.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.340+0000 7facc57fa700 1 -- 172.21.15.7:0/4220956578 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4599 (secure 0 0 0) 0x7facc0055dc0 con 0x7facd0156ca0 2024-01-27T02:39:26.340 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:26.341 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6260,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3240780787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3240780787},{"type":"v1","addr":"172.21.15.188:6837","nonce":3240780787}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7156,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/4051709811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4051709811},{"type":"v1","addr":"172.21.15.188:6836","nonce":4051709811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7416,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/4000944335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":4000944335},{"type":"v1","addr":"172.21.15.7:6834","nonce":4000944335}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:39:22.327946+0000","modified":"2024-01-27T02:39:25.910241+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7419":{"gid":7419,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2386550039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2386550039},{"type":"v1","addr":"172.21.15.7:6835","nonce":2386550039}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2024-01-27T02:39:26.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.342+0000 7facd7710700 1 -- 172.21.15.7:0/4220956578 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7facb4066f30 msgr2=0x7facb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:26.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.342+0000 7facd7710700 1 --2- 172.21.15.7:0/4220956578 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7facb4066f30 0x7facb40693e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7faccc0091e0 tx=0x7faccc01a000 comp rx=0 tx=0).stop 2024-01-27T02:39:26.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.342+0000 7facd7710700 1 -- 172.21.15.7:0/4220956578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd0156ca0 msgr2=0x7facd015a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:26.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.342+0000 7facd7710700 1 --2- 172.21.15.7:0/4220956578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd0156ca0 0x7facd015a3f0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7facc0011cc0 tx=0x7facc00091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.342+0000 7facd7710700 1 -- 172.21.15.7:0/4220956578 shutdown_connections 2024-01-27T02:39:26.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.342+0000 7facd7710700 1 --2- 172.21.15.7:0/4220956578 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7facb4066f30 0x7facb40693e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.342+0000 7facd7710700 1 --2- 172.21.15.7:0/4220956578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facd00b2510 0x7facd015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.342+0000 7facd7710700 1 --2- 172.21.15.7:0/4220956578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facbc003a60 0x7facd00b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.342+0000 7facd7710700 1 --2- 172.21.15.7:0/4220956578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd0156ca0 0x7facd015a3f0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.342+0000 7facd7710700 1 -- 172.21.15.7:0/4220956578 >> 172.21.15.7:0/4220956578 conn(0x7facd00cdc30 msgr2=0x7facd00c29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.342+0000 7facd7710700 1 -- 172.21.15.7:0/4220956578 shutdown_connections 2024-01-27T02:39:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.343+0000 7facd7710700 1 -- 172.21.15.7:0/4220956578 wait complete. 2024-01-27T02:39:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 77 2024-01-27T02:39:26.357 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:39:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.489+0000 7f7693d53700 1 Processor -- start 2024-01-27T02:39:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.490+0000 7f7693d53700 1 -- start start 2024-01-27T02:39:26.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.491+0000 7f7693d53700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f768c0c41e0 0x7f768c15a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:26.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.491+0000 7f7693d53700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c15ac80 0x7f768c0c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:26.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.491+0000 7f7693d53700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f768c15b6c0 con 0x7f768c0637d0 2024-01-27T02:39:26.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.491+0000 7f7693d53700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f768c06f2e0 con 0x7f768c0c41e0 2024-01-27T02:39:26.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.491+0000 7f7693d53700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f768c15b100 con 0x7f768c15ac80 2024-01-27T02:39:26.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.491+0000 7f76912ee700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f768c0c41e0 0x7f768c15a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:26.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.491+0000 7f7691aef700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f768c0637d0 0x7f768c06ebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41300/0 (socket says 172.21.15.7:41300) 2024-01-27T02:39:26.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.491+0000 7f7691aef700 1 -- 172.21.15.7:0/666470819 learned_addr learned my addr 172.21.15.7:0/666470819 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:26.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.491+0000 7f76922f0700 1 --2- 172.21.15.7:0/666470819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c15ac80 0x7f768c0c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:26.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.491+0000 7f76912ee700 1 -- 172.21.15.7:0/666470819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c15ac80 msgr2=0x7f768c0c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:26.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.491+0000 7f76912ee700 1 --2- 172.21.15.7:0/666470819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c15ac80 0x7f768c0c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.491+0000 7f76912ee700 1 -- 172.21.15.7:0/666470819 >> v1:172.21.15.7:6789/0 conn(0x7f768c0637d0 legacy=0x7f768c06ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:26.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.491+0000 7f76912ee700 1 -- 172.21.15.7:0/666470819 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768c15b3e0 con 0x7f768c0c41e0 2024-01-27T02:39:26.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.491+0000 7f7690aed700 1 -- 172.21.15.7:0/666470819 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704411784 0 0) 0x7f768c15b6c0 con 0x7f768c0637d0 2024-01-27T02:39:26.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.491+0000 7f76912ee700 1 --2- 172.21.15.7:0/666470819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f768c0c41e0 0x7f768c15a740 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f767c0098e0 tx=0x7f767c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb31320828d651b2 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:26.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.491+0000 7f7690aed700 1 -- 172.21.15.7:0/666470819 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c009e80 con 0x7f768c0c41e0 2024-01-27T02:39:26.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.492+0000 7f7690aed700 1 -- 172.21.15.7:0/666470819 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767c00b860 con 0x7f768c0c41e0 2024-01-27T02:39:26.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.492+0000 7f7690aed700 1 -- 172.21.15.7:0/666470819 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c00bbe0 con 0x7f768c0c41e0 2024-01-27T02:39:26.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.492+0000 7f7693d53700 1 -- 172.21.15.7:0/666470819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f768c0c41e0 msgr2=0x7f768c15a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:26.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.492+0000 7f7693d53700 1 --2- 172.21.15.7:0/666470819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f768c0c41e0 0x7f768c15a740 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f767c0098e0 tx=0x7f767c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.492+0000 7f7693d53700 1 -- 172.21.15.7:0/666470819 shutdown_connections 2024-01-27T02:39:26.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.492+0000 7f7693d53700 1 --2- 172.21.15.7:0/666470819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c15ac80 0x7f768c0c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.492+0000 7f7693d53700 1 --2- 172.21.15.7:0/666470819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f768c0c41e0 0x7f768c15a740 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.492+0000 7f7693d53700 1 -- 172.21.15.7:0/666470819 >> 172.21.15.7:0/666470819 conn(0x7f768c0cdc30 msgr2=0x7f768c0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.492+0000 7f7693d53700 1 -- 172.21.15.7:0/666470819 shutdown_connections 2024-01-27T02:39:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.492+0000 7f7693d53700 1 -- 172.21.15.7:0/666470819 wait complete. 2024-01-27T02:39:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.493+0000 7f7693d53700 1 Processor -- start 2024-01-27T02:39:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.493+0000 7f7693d53700 1 -- start start 2024-01-27T02:39:26.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.494+0000 7f7693d53700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f768c0637d0 0x7f768c16a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:26.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.494+0000 7f7693d53700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f768c0c41e0 0x7f768c0b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:26.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.494+0000 7f7693d53700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c0b3060 0x7f768c1564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:26.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.494+0000 7f7693d53700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c0b3500 con 0x7f768c0c41e0 2024-01-27T02:39:26.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.494+0000 7f7693d53700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c157200 con 0x7f768c0637d0 2024-01-27T02:39:26.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.494+0000 7f7693d53700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f768c1574e0 con 0x7f768c0b3060 2024-01-27T02:39:26.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.494+0000 7f76912ee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f768c0c41e0 0x7f768c0b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:26.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.494+0000 7f76912ee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f768c0c41e0 0x7f768c0b2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37682/0 (socket says 172.21.15.7:37682) 2024-01-27T02:39:26.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.494+0000 7f76912ee700 1 -- 172.21.15.7:0/3034119578 learned_addr learned my addr 172.21.15.7:0/3034119578 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:26.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.495+0000 7f76922f0700 1 --2- 172.21.15.7:0/3034119578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c0b3060 0x7f768c1564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:26.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.495+0000 7f7691aef700 1 --2- 172.21.15.7:0/3034119578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f768c0637d0 0x7f768c16a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:26.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.495+0000 7f76912ee700 1 -- 172.21.15.7:0/3034119578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c0b3060 msgr2=0x7f768c1564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.495+0000 7f76912ee700 1 --2- 172.21.15.7:0/3034119578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c0b3060 0x7f768c1564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.495+0000 7f76912ee700 1 -- 172.21.15.7:0/3034119578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f768c0637d0 msgr2=0x7f768c16a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.495+0000 7f76912ee700 1 --2- 172.21.15.7:0/3034119578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f768c0637d0 0x7f768c16a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.495+0000 7f76912ee700 1 -- 172.21.15.7:0/3034119578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c009420 con 0x7f768c0c41e0 2024-01-27T02:39:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.495+0000 7f76912ee700 1 --2- 172.21.15.7:0/3034119578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f768c0c41e0 0x7f768c0b2b20 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f767c013000 tx=0x7f767c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.495+0000 7f7681ffb700 1 -- 172.21.15.7:0/3034119578 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c00bbe0 con 0x7f768c0c41e0 2024-01-27T02:39:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.495+0000 7f7681ffb700 1 -- 172.21.15.7:0/3034119578 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767c011420 con 0x7f768c0c41e0 2024-01-27T02:39:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.495+0000 7f7693d53700 1 -- 172.21.15.7:0/3034119578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f768c1577c0 con 0x7f768c0c41e0 2024-01-27T02:39:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.495+0000 7f7681ffb700 1 -- 172.21.15.7:0/3034119578 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c0117c0 con 0x7f768c0c41e0 2024-01-27T02:39:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.495+0000 7f7693d53700 1 -- 172.21.15.7:0/3034119578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f768c157d80 con 0x7f768c0c41e0 2024-01-27T02:39:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.496+0000 7f7693d53700 1 -- 172.21.15.7:0/3034119578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f768c157f70 con 0x7f768c0c41e0 2024-01-27T02:39:26.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.497+0000 7f7681ffb700 1 -- 172.21.15.7:0/3034119578 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f767c011bb0 con 0x7f768c0c41e0 2024-01-27T02:39:26.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.497+0000 7f7681ffb700 1 --2- 172.21.15.7:0/3034119578 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7670066f30 0x7f76700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:26.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.497+0000 7f7681ffb700 1 -- 172.21.15.7:0/3034119578 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f767c085ef0 con 0x7f768c0c41e0 2024-01-27T02:39:26.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.497+0000 7f7691aef700 1 --2- 172.21.15.7:0/3034119578 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7670066f30 0x7f76700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:26.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.498+0000 7f7691aef700 1 --2- 172.21.15.7:0/3034119578 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7670066f30 0x7f76700693e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f768800c8f0 tx=0x7f7688009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:26.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.502+0000 7f7681ffb700 1 -- 172.21.15.7:0/3034119578 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f767c053160 con 0x7f768c0c41e0 2024-01-27T02:39:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.708+0000 7f7693d53700 1 -- 172.21.15.7:0/3034119578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f768c04c710 con 0x7f768c0c41e0 2024-01-27T02:39:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.709+0000 7f7681ffb700 1 -- 172.21.15.7:0/3034119578 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+15317 (secure 0 0 0) 0x7f767c011e60 con 0x7f768c0c41e0 2024-01-27T02:39:26.710 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:26.710 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":149,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:39:23.888060+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:39:20.119244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-01-27T02:39:21.293772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:39:26.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.711+0000 7f7693d53700 1 -- 172.21.15.7:0/3034119578 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7670066f30 msgr2=0x7f76700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:26.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.711+0000 7f7693d53700 1 --2- 172.21.15.7:0/3034119578 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7670066f30 0x7f76700693e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f768800c8f0 tx=0x7f7688009040 comp rx=0 tx=0).stop 2024-01-27T02:39:26.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.711+0000 7f7693d53700 1 -- 172.21.15.7:0/3034119578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f768c0c41e0 msgr2=0x7f768c0b2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:26.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.711+0000 7f7693d53700 1 --2- 172.21.15.7:0/3034119578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f768c0c41e0 0x7f768c0b2b20 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f767c013000 tx=0x7f767c017000 comp rx=0 tx=0).stop 2024-01-27T02:39:26.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.712+0000 7f7693d53700 1 -- 172.21.15.7:0/3034119578 shutdown_connections 2024-01-27T02:39:26.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.712+0000 7f7693d53700 1 --2- 172.21.15.7:0/3034119578 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7670066f30 0x7f76700693e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.712+0000 7f7693d53700 1 --2- 172.21.15.7:0/3034119578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768c0b3060 0x7f768c1564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.712+0000 7f7693d53700 1 --2- 172.21.15.7:0/3034119578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f768c0637d0 0x7f768c16a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.712+0000 7f7693d53700 1 --2- 172.21.15.7:0/3034119578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f768c0c41e0 0x7f768c0b2b20 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.712+0000 7f7693d53700 1 -- 172.21.15.7:0/3034119578 >> 172.21.15.7:0/3034119578 conn(0x7f768c0cdc30 msgr2=0x7f768c0aa8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:26.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.712+0000 7f7693d53700 1 -- 172.21.15.7:0/3034119578 shutdown_connections 2024-01-27T02:39:26.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.712+0000 7f7693d53700 1 -- 172.21.15.7:0/3034119578 wait complete. 2024-01-27T02:39:26.726 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-27T02:39:26.726 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:39:26.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.857+0000 7f4dd1913700 1 Processor -- start 2024-01-27T02:39:26.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.857+0000 7f4dd1913700 1 -- start start 2024-01-27T02:39:26.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.858+0000 7f4dd1913700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4dcc15a070 0x7f4dcc15b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:26.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.858+0000 7f4dd1913700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4dcc0b0670 0x7f4dcc163d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:26.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.858+0000 7f4dd1913700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dcc0689c0 con 0x7f4dcc0caf10 2024-01-27T02:39:26.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.858+0000 7f4dd1913700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dcc0ae5f0 con 0x7f4dcc15a070 2024-01-27T02:39:26.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.858+0000 7f4dd1913700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dcc0ae8b0 con 0x7f4dcc0b0670 2024-01-27T02:39:26.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.858+0000 7f4dcaffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4dcc15a070 0x7f4dcc15b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:26.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.858+0000 7f4dcaffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4dcc15a070 0x7f4dcc15b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37688/0 (socket says 172.21.15.7:37688) 2024-01-27T02:39:26.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.858+0000 7f4dca7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4dcc0caf10 0x7f4dcc0aff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41310/0 (socket says 172.21.15.7:41310) 2024-01-27T02:39:26.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.858+0000 7f4dcaffd700 1 -- 172.21.15.7:0/3319467873 learned_addr learned my addr 172.21.15.7:0/3319467873 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:26.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.858+0000 7f4dcb7fe700 1 --2- 172.21.15.7:0/3319467873 >> v2:172.21.15.188:3300/0 conn(0x7f4dcc0b0670 0x7f4dcc163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:26.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.858+0000 7f4dcaffd700 1 -- 172.21.15.7:0/3319467873 >> v2:172.21.15.188:3300/0 conn(0x7f4dcc0b0670 msgr2=0x7f4dcc163d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:26.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.858+0000 7f4dcaffd700 1 --2- 172.21.15.7:0/3319467873 >> v2:172.21.15.188:3300/0 conn(0x7f4dcc0b0670 0x7f4dcc163d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.858+0000 7f4dcaffd700 1 -- 172.21.15.7:0/3319467873 >> v1:172.21.15.7:6789/0 conn(0x7f4dcc0caf10 legacy=0x7f4dcc0aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.858+0000 7f4dc9ffb700 1 -- 172.21.15.7:0/3319467873 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474503675 0 0) 0x7f4dcc0689c0 con 0x7f4dcc0caf10 2024-01-27T02:39:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.858+0000 7f4dcaffd700 1 -- 172.21.15.7:0/3319467873 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dcc0aeb90 con 0x7f4dcc15a070 2024-01-27T02:39:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.858+0000 7f4dcaffd700 1 --2- 172.21.15.7:0/3319467873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dcc15a070 0x7f4dcc15b2a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc06e540 tx=0x7f4dbc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750b6127a8030c38 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.859+0000 7f4dc9ffb700 1 -- 172.21.15.7:0/3319467873 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc00b860 con 0x7f4dcc15a070 2024-01-27T02:39:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.859+0000 7f4dc9ffb700 1 -- 172.21.15.7:0/3319467873 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dbc00b9c0 con 0x7f4dcc15a070 2024-01-27T02:39:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.859+0000 7f4dc9ffb700 1 -- 172.21.15.7:0/3319467873 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc00bd60 con 0x7f4dcc15a070 2024-01-27T02:39:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.859+0000 7f4dd1913700 1 -- 172.21.15.7:0/3319467873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dcc15a070 msgr2=0x7f4dcc15b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.859+0000 7f4dd1913700 1 --2- 172.21.15.7:0/3319467873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dcc15a070 0x7f4dcc15b2a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc06e540 tx=0x7f4dbc009250 comp rx=0 tx=0).stop 2024-01-27T02:39:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.859+0000 7f4dd1913700 1 -- 172.21.15.7:0/3319467873 shutdown_connections 2024-01-27T02:39:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.859+0000 7f4dd1913700 1 --2- 172.21.15.7:0/3319467873 >> v2:172.21.15.188:3300/0 conn(0x7f4dcc0b0670 0x7f4dcc163d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.859+0000 7f4dd1913700 1 --2- 172.21.15.7:0/3319467873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dcc15a070 0x7f4dcc15b2a0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.859+0000 7f4dd1913700 1 -- 172.21.15.7:0/3319467873 >> 172.21.15.7:0/3319467873 conn(0x7f4dcc098020 msgr2=0x7f4dcc0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.859+0000 7f4dd1913700 1 -- 172.21.15.7:0/3319467873 shutdown_connections 2024-01-27T02:39:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.859+0000 7f4dd1913700 1 -- 172.21.15.7:0/3319467873 wait complete. 2024-01-27T02:39:26.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.861+0000 7f4dd1913700 1 Processor -- start 2024-01-27T02:39:26.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.861+0000 7f4dd1913700 1 -- start start 2024-01-27T02:39:26.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.861+0000 7f4dd1913700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dcc0b0670 0x7f4dcc0cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:26.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.862+0000 7f4dd1913700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dcc0caf10 0x7f4dcc15d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:26.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.862+0000 7f4dd1913700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4dcc15a070 0x7f4dcc15c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:26.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.862+0000 7f4dd1913700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc15d920 con 0x7f4dcc0b0670 2024-01-27T02:39:26.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.862+0000 7f4dd1913700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc0cf630 con 0x7f4dcc0caf10 2024-01-27T02:39:26.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.862+0000 7f4dd1913700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc0cf910 con 0x7f4dcc15a070 2024-01-27T02:39:26.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.862+0000 7f4dcaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dcc0b0670 0x7f4dcc0cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:26.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.862+0000 7f4dcaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dcc0b0670 0x7f4dcc0cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37700/0 (socket says 172.21.15.7:37700) 2024-01-27T02:39:26.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.862+0000 7f4dcaffd700 1 -- 172.21.15.7:0/773409387 learned_addr learned my addr 172.21.15.7:0/773409387 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:26.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.862+0000 7f4dcb7fe700 1 --2- 172.21.15.7:0/773409387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4dcc15a070 0x7f4dcc15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:26.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.862+0000 7f4dca7fc700 1 --2- 172.21.15.7:0/773409387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dcc0caf10 0x7f4dcc15d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:26.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.862+0000 7f4dcaffd700 1 -- 172.21.15.7:0/773409387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4dcc15a070 msgr2=0x7f4dcc15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:26.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.862+0000 7f4dcaffd700 1 --2- 172.21.15.7:0/773409387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4dcc15a070 0x7f4dcc15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.862+0000 7f4dcaffd700 1 -- 172.21.15.7:0/773409387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dcc0caf10 msgr2=0x7f4dcc15d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:26.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.862+0000 7f4dcaffd700 1 --2- 172.21.15.7:0/773409387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dcc0caf10 0x7f4dcc15d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:26.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.862+0000 7f4dcaffd700 1 -- 172.21.15.7:0/773409387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dbc0094f0 con 0x7f4dcc0b0670 2024-01-27T02:39:26.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.862+0000 7f4dcaffd700 1 --2- 172.21.15.7:0/773409387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dcc0b0670 0x7f4dcc0cc3c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc009290 tx=0x7f4dbc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:26.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.862+0000 7f4db2ffd700 1 -- 172.21.15.7:0/773409387 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc026450 con 0x7f4dcc0b0670 2024-01-27T02:39:26.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.862+0000 7f4db2ffd700 1 -- 172.21.15.7:0/773409387 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dbc0265b0 con 0x7f4dcc0b0670 2024-01-27T02:39:26.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.862+0000 7f4dd1913700 1 -- 172.21.15.7:0/773409387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dcc0cfbf0 con 0x7f4dcc0b0670 2024-01-27T02:39:26.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.862+0000 7f4db2ffd700 1 -- 172.21.15.7:0/773409387 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc026970 con 0x7f4dcc0b0670 2024-01-27T02:39:26.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.863+0000 7f4dd1913700 1 -- 172.21.15.7:0/773409387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dcc0ce070 con 0x7f4dcc0b0670 2024-01-27T02:39:26.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.863+0000 7f4dd1913700 1 -- 172.21.15.7:0/773409387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dcc04c710 con 0x7f4dcc0b0670 2024-01-27T02:39:26.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.864+0000 7f4db2ffd700 1 -- 172.21.15.7:0/773409387 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4dbc01a420 con 0x7f4dcc0b0670 2024-01-27T02:39:26.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.864+0000 7f4db2ffd700 1 --2- 172.21.15.7:0/773409387 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4db8066f30 0x7f4db80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:26.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.864+0000 7f4db2ffd700 1 -- 172.21.15.7:0/773409387 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4dbc086290 con 0x7f4dcc0b0670 2024-01-27T02:39:26.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.865+0000 7f4dca7fc700 1 --2- 172.21.15.7:0/773409387 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4db8066f30 0x7f4db80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:26.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.865+0000 7f4dca7fc700 1 --2- 172.21.15.7:0/773409387 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4db8066f30 0x7f4db80693e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0687b0 tx=0x7f4dc0007bf0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:26.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:26.869+0000 7f4db2ffd700 1 -- 172.21.15.7:0/773409387 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4dbc053630 con 0x7f4dcc0b0670 2024-01-27T02:39:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.066+0000 7f4dd1913700 1 -- 172.21.15.7:0/773409387 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4dcc0aee70 con 0x7f4db8066f30 2024-01-27T02:39:27.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.075+0000 7f4db2ffd700 1 -- 172.21.15.7:0/773409387 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276311 (secure 0 0 0) 0x7f4dcc0aee70 con 0x7f4db8066f30 2024-01-27T02:39:27.080 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:27.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.078+0000 7f4dd1913700 1 -- 172.21.15.7:0/773409387 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4db8066f30 msgr2=0x7f4db80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:27.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.078+0000 7f4dd1913700 1 --2- 172.21.15.7:0/773409387 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4db8066f30 0x7f4db80693e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0687b0 tx=0x7f4dc0007bf0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.078+0000 7f4dd1913700 1 -- 172.21.15.7:0/773409387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dcc0b0670 msgr2=0x7f4dcc0cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:27.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.078+0000 7f4dd1913700 1 --2- 172.21.15.7:0/773409387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dcc0b0670 0x7f4dcc0cc3c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc009290 tx=0x7f4dbc017000 comp rx=0 tx=0).stop 2024-01-27T02:39:27.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.078+0000 7f4dd1913700 1 -- 172.21.15.7:0/773409387 shutdown_connections 2024-01-27T02:39:27.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.078+0000 7f4dd1913700 1 --2- 172.21.15.7:0/773409387 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4db8066f30 0x7f4db80693e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.078+0000 7f4dd1913700 1 --2- 172.21.15.7:0/773409387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4dcc15a070 0x7f4dcc15c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.079+0000 7f4dd1913700 1 --2- 172.21.15.7:0/773409387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dcc0b0670 0x7f4dcc0cc3c0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.079+0000 7f4dd1913700 1 --2- 172.21.15.7:0/773409387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dcc0caf10 0x7f4dcc15d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.079+0000 7f4dd1913700 1 -- 172.21.15.7:0/773409387 >> 172.21.15.7:0/773409387 conn(0x7f4dcc098020 msgr2=0x7f4dcc06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:27.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.079+0000 7f4dd1913700 1 -- 172.21.15.7:0/773409387 shutdown_connections 2024-01-27T02:39:27.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.079+0000 7f4dd1913700 1 -- 172.21.15.7:0/773409387 wait complete. 2024-01-27T02:39:27.084 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:39:27.093 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-01-27T02:39:25.679505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7420,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555079,"statfs":{"total":1600000000,"available":1592401644,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.799011"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.884300+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:30:40.460262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.883346+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:35:46.505332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.885036+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:50:06.070356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.886377+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:00:11.450382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.885720+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:50:34.405562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.883530+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:09:13.278758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.881548+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:25:46.555556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.884760+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:55:51.094591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.882435+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:00:11.450382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.882953+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:04:42.115329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.883124+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:26:01.500470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.884186+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:35:37.535621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.881260+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:14:14.430917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.885318+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:08:53.065440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.884206+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:06:23.555202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.882920+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:49.973643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.884627+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:15:14.538837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.881587+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:17:06.668947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.882767+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:52:32.402932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.882686+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:26:41.909600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.882944+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:08:05.792443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.882900+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:47.650928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.884008+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:11.861136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.881284+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:15:18.285546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.883832+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:37:46.123420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.884817+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:17:25.995339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.884190+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:53:49.412236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.886042+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:51:21.995075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.882722+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:19:26.932201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.882871+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:18:01.492341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.884605+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:33:45.692091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.883302+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:47:35.007943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":271,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-01-27T02:39:19.239201+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:39:19.239201+0000","last_peered":"2024-01-27T02:39:19.239201+0000","last_clean":"2024-01-27T02:39:19.239201+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:39:19.239201+0000","last_undegraded":"2024-01-27T02:39:19.239201+0000","last_fullsized":"2024-01-27T02:39:19.239201+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574939,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1362,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193949,"statfs":{"total":200000000,"available":198604527,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.39400000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76200000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66900000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.58199999999999996}]},{"osd":1,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76900000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.56000000000000005},"min":{"1min":0.41799999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.67600000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.52700000000000002}]},{"osd":2,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.67900000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.49199999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76200000000000001,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.63300000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.48399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.66800000000000004,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.45400000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.74099999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.76300000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47699999999999998,"15min":0.48899999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70699999999999996,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.437},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.48899999999999999,"15min":0.51200000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69299999999999995,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.59399999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.51800000000000002,"15min":0.53700000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.307},"max":{"1min":0.73199999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55200000000000005,"15min":0.56499999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33500000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.61699999999999999}]},{"osd":6,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57799999999999996,"15min":0.59599999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.90800000000000003},"last":0.64500000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574939,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":780,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194531,"statfs":{"total":200000000,"available":199200700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.63,"15min":0.54400000000000004},"min":{"1min":0.375,"5min":0.36499999999999999,"15min":0.29999999999999999},"max":{"1min":0.83099999999999996,"5min":4.125,"15min":4.125},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59699999999999998,"15min":0.54000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32700000000000001},"max":{"1min":0.75600000000000001,"5min":1.216,"15min":1.216},"last":0.68200000000000005}]},{"osd":1,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.67200000000000004,"15min":0.55600000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.34100000000000003},"max":{"1min":0.84199999999999997,"5min":8.2699999999999996,"15min":8.2699999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.88900000000000001,"15min":0.65500000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80600000000000005,"5min":26.411999999999999,"15min":26.411999999999999},"last":0.54100000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.75700000000000001,"15min":0.60799999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77000000000000002,"5min":17.143999999999998,"15min":17.143999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.64400000000000002,"15min":0.59299999999999997},"min":{"1min":0.45800000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85799999999999998,"5min":1.76,"15min":1.76},"last":0.622}]},{"osd":3,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.624,"15min":0.56899999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.376},"max":{"1min":0.78000000000000003,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.624,"15min":0.58199999999999996},"min":{"1min":0.44500000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79300000000000004,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.59099999999999997}]},{"osd":4,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.30199999999999999},"max":{"1min":0.73599999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.50900000000000001,"15min":0.54300000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84899999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.73999999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.80800000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56299999999999994,"15min":0.59599999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.71099999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59299999999999997,"15min":0.60999999999999999},"min":{"1min":0.40999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.87,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55700000000000005,"15min":0.58199999999999996},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.36499999999999999},"max":{"1min":0.82899999999999996,"5min":1.038,"15min":1.038},"last":0.69599999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":780,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194531,"statfs":{"total":200000000,"available":199200700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39800000000000002,"15min":0.378},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41799999999999998,"15min":0.40000000000000002},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.50600000000000001,"5min":1.121,"15min":1.121},"last":0.66100000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40799999999999997,"15min":0.373},"min":{"1min":0.249,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":1.0940000000000001,"5min":1.25,"15min":1.25},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.44900000000000001,"15min":0.43099999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29899999999999999},"max":{"1min":0.498,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.54200000000000004}]},{"osd":3,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.48499999999999999,"15min":0.46899999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.32100000000000001},"max":{"1min":0.56699999999999995,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.46600000000000003,"15min":0.45500000000000002},"min":{"1min":0.34899999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.56100000000000005,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.55600000000000005}]},{"osd":4,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.66600000000000004,"15min":0.63600000000000001},"min":{"1min":0.48299999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.85699999999999998,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66600000000000004,"15min":0.64000000000000001},"min":{"1min":0.45000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91600000000000004,"5min":1.026,"15min":1.026},"last":0.628}]},{"osd":5,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69099999999999995,"15min":0.64200000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":0.93400000000000005,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69699999999999995,"15min":0.67600000000000005},"min":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96199999999999997,"5min":3.181,"15min":3.181},"last":0.64600000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.70599999999999996,"15min":0.69499999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88500000000000001,"5min":0.96399999999999997,"15min":1.024},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72099999999999997,"15min":0.68300000000000005},"min":{"1min":0.54000000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93600000000000005,"5min":1.869,"15min":1.869},"last":0.81100000000000005}]},{"osd":7,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73199999999999998,"15min":0.69099999999999995},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.47299999999999998},"max":{"1min":0.91400000000000003,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73099999999999998,"15min":0.69499999999999995},"min":{"1min":0.53000000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92000000000000004,"5min":1.708,"15min":1.708},"last":0.68100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574939,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1362,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193949,"statfs":{"total":200000000,"available":198604527,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.40600000000000003,"15min":0.39400000000000002},"min":{"1min":0.252,"5min":0.24399999999999999,"15min":0.23799999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.437,"5min":0.42899999999999999,"15min":0.41799999999999998},"min":{"1min":0.28899999999999998,"5min":0.27800000000000002,"15min":0.223},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.372}]},{"osd":2,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.433,"15min":0.42199999999999999},"min":{"1min":0.31,"5min":0.28699999999999998,"15min":0.28199999999999997},"max":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.45700000000000002,"15min":0.438},"min":{"1min":0.312,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.443}]},{"osd":3,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.441,"15min":0.432},"min":{"1min":0.315,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.66100000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.495,"15min":0.47599999999999998},"min":{"1min":0.38800000000000001,"5min":0.33600000000000002,"15min":0.29099999999999998},"max":{"1min":0.67600000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.39300000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66800000000000004,"15min":0.64400000000000002},"min":{"1min":0.505,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80900000000000005,"5min":0.997,"15min":0.997},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.45800000000000002,"5min":0.44,"15min":0.41999999999999998},"max":{"1min":0.83399999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.81899999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68899999999999995,"15min":0.65800000000000003},"min":{"1min":0.51900000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.78800000000000003,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.54900000000000004,"5min":0.47799999999999998,"15min":0.44},"max":{"1min":0.84999999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.83699999999999997}]},{"osd":6,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68300000000000005,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.42499999999999999},"max":{"1min":0.79800000000000004,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69899999999999995,"15min":0.68000000000000005},"min":{"1min":0.51600000000000001,"5min":0.45700000000000002,"15min":0.45000000000000001},"max":{"1min":0.90300000000000002,"5min":1.04,"15min":1.04},"last":0.77200000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.50800000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90200000000000002,"5min":1.014,"15min":1.014},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69699999999999995,"15min":0.69199999999999995},"min":{"1min":0.54500000000000004,"5min":0.498,"15min":0.498},"max":{"1min":0.81999999999999995,"5min":0.95799999999999996,"15min":0.97299999999999998},"last":0.90300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":786,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194526,"statfs":{"total":200000000,"available":199194895,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.41199999999999998},"min":{"1min":0.27300000000000002,"5min":0.23599999999999999,"15min":0.23000000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.46700000000000003,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.30499999999999999},"max":{"1min":0.68899999999999995,"5min":0.72799999999999998,"15min":6.2729999999999997},"last":0.34300000000000003}]},{"osd":1,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.433,"15min":0.42299999999999999},"min":{"1min":0.33800000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.621,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.44},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.439,"15min":0.44400000000000001},"min":{"1min":0.33900000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.60399999999999998,"5min":0.65300000000000002,"15min":0.70099999999999996},"last":0.29399999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.434,"15min":0.41799999999999998},"min":{"1min":0.32200000000000001,"5min":0.28899999999999998,"15min":0.27800000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49099999999999999,"15min":0.48599999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.38100000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68799999999999994,"15min":0.65900000000000003},"min":{"1min":0.499,"5min":0.46800000000000003,"15min":0.39300000000000002},"max":{"1min":0.90500000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.70599999999999996,"15min":0.78100000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95799999999999996,"5min":1.006,"15min":3.581},"last":0.65600000000000003}]},{"osd":5,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65800000000000003,"15min":0.72699999999999998},"min":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92200000000000004,"5min":0.95899999999999996,"15min":3.5369999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.52200000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89100000000000001,"5min":0.92200000000000004,"15min":1.0529999999999999},"last":0.68000000000000005}]},{"osd":6,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67200000000000004,"15min":0.66700000000000004},"min":{"1min":0.57499999999999996,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96499999999999997,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69599999999999995,"15min":0.68000000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.97099999999999997,"5min":1.0189999999999999,"15min":1.0209999999999999},"last":0.69599999999999995}]},{"osd":7,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69199999999999995,"15min":0.67900000000000005},"min":{"1min":0.58799999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97699999999999998,"5min":1.03,"15min":1.03},"last":0.62},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70899999999999996,"15min":0.69299999999999995},"min":{"1min":0.55200000000000005,"5min":0.441,"15min":0.42699999999999999},"max":{"1min":0.99099999999999999,"5min":1.036,"15min":1.036},"last":0.72699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":780,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194531,"statfs":{"total":200000000,"available":199200700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.45500000000000002,"15min":0.38600000000000001},"min":{"1min":0.29699999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.67900000000000005,"5min":4.7380000000000004,"15min":4.7380000000000004},"last":0.37},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41899999999999998,"15min":0.40000000000000002},"min":{"1min":0.29399999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55500000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.501}]},{"osd":1,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.47499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28000000000000003,"5min":0.27400000000000002,"15min":0.22900000000000001},"max":{"1min":0.61599999999999999,"5min":4.5739999999999998,"15min":4.5739999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.53300000000000003,"15min":0.45200000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.98199999999999998,"5min":5.0529999999999999,"15min":5.0529999999999999},"last":0.45900000000000002}]},{"osd":2,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.49199999999999999,"15min":0.443},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.28100000000000003},"max":{"1min":0.76700000000000002,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.55400000000000005,"15min":0.47699999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71799999999999997,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":0.52100000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69999999999999996,"15min":0.63800000000000001},"min":{"1min":0.45300000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91600000000000004,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.62},"min":{"1min":0.48199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86099999999999999,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.749}]},{"osd":5,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67600000000000005,"15min":0.626},"min":{"1min":0.438,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82399999999999995,"5min":1.409,"15min":1.409},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.78700000000000003,"15min":0.70399999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.89600000000000002,"5min":4.7960000000000003,"15min":4.7960000000000003},"last":0.70199999999999996}]},{"osd":6,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.73899999999999999,"15min":0.67900000000000005},"min":{"1min":0.42799999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92700000000000005,"5min":5.0069999999999997,"15min":5.0069999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.69399999999999995,"15min":0.65900000000000003},"min":{"1min":0.47499999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.94199999999999995,"5min":1.103,"15min":1.103},"last":0.68100000000000005}]},{"osd":7,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.75700000000000001,"15min":0.66900000000000004},"min":{"1min":0.57599999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72699999999999998,"15min":0.68000000000000005},"min":{"1min":0.52800000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.88900000000000001,"5min":1.361,"15min":1.361},"last":0.83399999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":786,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194526,"statfs":{"total":200000000,"available":199194895,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.35599999999999998,"5min":0.34599999999999997,"15min":0.314},"max":{"1min":1.137,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66200000000000003,"15min":0.61399999999999999},"min":{"1min":0.496,"5min":0.372,"15min":0.372},"max":{"1min":1.016,"5min":7.0670000000000002,"15min":7.0670000000000002},"last":0.623}]},{"osd":1,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.64200000000000002,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.35399999999999998},"max":{"1min":0.81000000000000005,"5min":5.1529999999999996,"15min":5.1529999999999996},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64400000000000002,"15min":0.60899999999999999},"min":{"1min":0.46400000000000002,"5min":0.40799999999999997,"15min":0.39000000000000001},"max":{"1min":1.054,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.60699999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.64200000000000002,"15min":0.61099999999999999},"min":{"1min":0.48399999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76600000000000001,"5min":5.0019999999999998,"15min":5.0019999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.63300000000000001,"15min":0.62},"min":{"1min":0.432,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2549999999999999,"5min":2.073,"15min":2.073},"last":0.57199999999999995}]},{"osd":3,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.60499999999999998},"min":{"1min":0.45200000000000001,"5min":0.42099999999999999,"15min":0.36199999999999999},"max":{"1min":1.1899999999999999,"5min":6.7220000000000004,"15min":6.7220000000000004},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68999999999999995,"15min":0.66400000000000003},"min":{"1min":0.46100000000000002,"5min":0.44900000000000001,"15min":0.41399999999999998},"max":{"1min":0.88,"5min":6.5510000000000002,"15min":6.5510000000000002},"last":0.51600000000000001}]},{"osd":5,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.34899999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.77500000000000002,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.503},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56599999999999995,"15min":0.51700000000000002},"min":{"1min":0.372,"5min":0.312,"15min":0.29999999999999999},"max":{"1min":0.68799999999999994,"5min":6.9269999999999996,"15min":6.9269999999999996},"last":0.52700000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56100000000000005,"15min":0.52200000000000002},"min":{"1min":0.40400000000000003,"5min":0.34999999999999998,"15min":0.32900000000000001},"max":{"1min":0.78600000000000003,"5min":5.2229999999999999,"15min":5.2229999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56200000000000006,"15min":0.54900000000000004},"min":{"1min":0.41799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82999999999999996,"5min":1.46,"15min":1.46},"last":0.44700000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61699999999999999,"15min":0.57499999999999996},"min":{"1min":0.434,"5min":0.34799999999999998,"15min":0.33600000000000002},"max":{"1min":0.78300000000000003,"5min":7.5339999999999998,"15min":7.5339999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.629,"15min":0.59399999999999997},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85299999999999998,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":0.42699999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574939,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":780,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194531,"statfs":{"total":200000000,"available":199200700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56399999999999995,"15min":0.58999999999999997},"min":{"1min":0.42099999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56699999999999995,"15min":0.57899999999999996},"min":{"1min":0.442,"5min":0.34499999999999997,"15min":0.27600000000000002},"max":{"1min":0.77100000000000002,"5min":0.78900000000000003,"15min":3.242},"last":0.65100000000000002}]},{"osd":1,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.622,"15min":0.61299999999999999},"min":{"1min":0.38500000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.3380000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61099999999999999,"15min":0.64300000000000002},"min":{"1min":0.35399999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78400000000000003,"5min":0.84399999999999997,"15min":0.95499999999999996},"last":0.66500000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58599999999999997,"15min":0.59699999999999998},"min":{"1min":0.40400000000000003,"5min":0.39100000000000001,"15min":0.372},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":3.04},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.48499999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80400000000000005,"5min":0.85899999999999999,"15min":0.89400000000000002},"last":0.55600000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60499999999999998,"15min":0.66800000000000004},"min":{"1min":0.40300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":4.032},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65500000000000003,"15min":0.67700000000000005},"min":{"1min":0.48999999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.76100000000000001,"5min":0.86199999999999999,"15min":4.282},"last":0.63600000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50700000000000001,"15min":0.52200000000000002},"min":{"1min":0.35799999999999998,"5min":0.30599999999999999,"15min":0.26500000000000001},"max":{"1min":0.63600000000000001,"5min":0.81399999999999995,"15min":1.2869999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.495,"15min":0.55300000000000005},"min":{"1min":0.32800000000000001,"5min":0.318,"15min":0.30399999999999999},"max":{"1min":0.66900000000000004,"5min":0.79800000000000004,"15min":1.101},"last":0.70499999999999996}]},{"osd":6,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.497,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65200000000000002,"5min":0.73199999999999998,"15min":1.7969999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55300000000000005,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80100000000000005,"5min":0.85799999999999998,"15min":1.6739999999999999},"last":0.75}]},{"osd":7,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.38200000000000001,"5min":0.34799999999999998,"15min":0.32700000000000001},"max":{"1min":0.66800000000000004,"5min":0.77100000000000002,"15min":1.008},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59699999999999998,"15min":0.64900000000000002},"min":{"1min":0.436,"5min":0.35899999999999999,"15min":0.33700000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":2},"last":0.77000000000000002}]}]}],"pool_statfs":[]}} 2024-01-27T02:39:27.099 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:39:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.267+0000 7fa1c0a0d700 1 Processor -- start 2024-01-27T02:39:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.267+0000 7fa1c0a0d700 1 -- start start 2024-01-27T02:39:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.268+0000 7fa1c0a0d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa1bc0b09b0 0x7fa1bc15ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.268+0000 7fa1c0a0d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa1bc141850 0x7fa1bc06c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.268+0000 7fa1c0a0d700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1bc0c8860 con 0x7fa1bc0637d0 2024-01-27T02:39:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.268+0000 7fa1c0a0d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1bc15c9d0 con 0x7fa1bc141850 2024-01-27T02:39:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.268+0000 7fa1c0a0d700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1bc15c130 con 0x7fa1bc0b09b0 2024-01-27T02:39:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.268+0000 7fa1bad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa1bc141850 0x7fa1bc06c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.268+0000 7fa1bad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa1bc141850 0x7fa1bc06c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37704/0 (socket says 172.21.15.7:37704) 2024-01-27T02:39:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.268+0000 7fa1bad9d700 1 -- 172.21.15.7:0/2618848619 learned_addr learned my addr 172.21.15.7:0/2618848619 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.268+0000 7fa1b9d9b700 1 --2- 172.21.15.7:0/2618848619 >> v2:172.21.15.188:3300/0 conn(0x7fa1bc0b09b0 0x7fa1bc15ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.269+0000 7fa1bad9d700 1 -- 172.21.15.7:0/2618848619 >> v2:172.21.15.188:3300/0 conn(0x7fa1bc0b09b0 msgr2=0x7fa1bc15ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.269+0000 7fa1bad9d700 1 --2- 172.21.15.7:0/2618848619 >> v2:172.21.15.188:3300/0 conn(0x7fa1bc0b09b0 0x7fa1bc15ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.269+0000 7fa1bad9d700 1 -- 172.21.15.7:0/2618848619 >> v1:172.21.15.188:6789/0 conn(0x7fa1bc0637d0 legacy=0x7fa1bc141140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.269+0000 7fa1bad9d700 1 -- 172.21.15.7:0/2618848619 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1bc15c410 con 0x7fa1bc141850 2024-01-27T02:39:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.269+0000 7fa1b9d9b700 1 --2- 172.21.15.7:0/2618848619 >> v2:172.21.15.188:3300/0 conn(0x7fa1bc0b09b0 0x7fa1bc15ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:39:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.269+0000 7fa1bad9d700 1 --2- 172.21.15.7:0/2618848619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1bc141850 0x7fa1bc06c800 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc0ca450 tx=0x7fa1b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d3771dbb183378 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.269+0000 7fa1b959a700 1 -- 172.21.15.7:0/2618848619 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b400e070 con 0x7fa1bc141850 2024-01-27T02:39:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.269+0000 7fa1b959a700 1 -- 172.21.15.7:0/2618848619 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b400ba10 con 0x7fa1bc141850 2024-01-27T02:39:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.269+0000 7fa1b959a700 1 -- 172.21.15.7:0/2618848619 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b40092e0 con 0x7fa1bc141850 2024-01-27T02:39:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.269+0000 7fa1c0a0d700 1 -- 172.21.15.7:0/2618848619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1bc141850 msgr2=0x7fa1bc06c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.269+0000 7fa1c0a0d700 1 --2- 172.21.15.7:0/2618848619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1bc141850 0x7fa1bc06c800 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc0ca450 tx=0x7fa1b4009840 comp rx=0 tx=0).stop 2024-01-27T02:39:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.270+0000 7fa1c0a0d700 1 -- 172.21.15.7:0/2618848619 shutdown_connections 2024-01-27T02:39:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.270+0000 7fa1c0a0d700 1 --2- 172.21.15.7:0/2618848619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1bc141850 0x7fa1bc06c800 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.270+0000 7fa1c0a0d700 1 --2- 172.21.15.7:0/2618848619 >> v2:172.21.15.188:3300/0 conn(0x7fa1bc0b09b0 0x7fa1bc15ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.270+0000 7fa1c0a0d700 1 -- 172.21.15.7:0/2618848619 >> 172.21.15.7:0/2618848619 conn(0x7fa1bc0cdbd0 msgr2=0x7fa1bc0cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.270+0000 7fa1c0a0d700 1 -- 172.21.15.7:0/2618848619 shutdown_connections 2024-01-27T02:39:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.270+0000 7fa1c0a0d700 1 -- 172.21.15.7:0/2618848619 wait complete. 2024-01-27T02:39:27.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.271+0000 7fa1c0a0d700 1 Processor -- start 2024-01-27T02:39:27.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.271+0000 7fa1c0a0d700 1 -- start start 2024-01-27T02:39:27.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.272+0000 7fa1c0a0d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1bc0637d0 0x7fa1bc0c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.272+0000 7fa1c0a0d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1bc0b09b0 0x7fa1bc156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.272+0000 7fa1c0a0d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1bc141850 0x7fa1bc15d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.272+0000 7fa1c0a0d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1bc15d4c0 con 0x7fa1bc0637d0 2024-01-27T02:39:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.272+0000 7fa1c0a0d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1bc0cd190 con 0x7fa1bc141850 2024-01-27T02:39:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.272+0000 7fa1c0a0d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1bc15cb90 con 0x7fa1bc0b09b0 2024-01-27T02:39:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.272+0000 7fa1ba59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1bc0637d0 0x7fa1bc0c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.272+0000 7fa1ba59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1bc0637d0 0x7fa1bc0c55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37706/0 (socket says 172.21.15.7:37706) 2024-01-27T02:39:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.272+0000 7fa1ba59c700 1 -- 172.21.15.7:0/2160112757 learned_addr learned my addr 172.21.15.7:0/2160112757 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.272+0000 7fa1bad9d700 1 --2- 172.21.15.7:0/2160112757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1bc141850 0x7fa1bc15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.272+0000 7fa1b9d9b700 1 --2- 172.21.15.7:0/2160112757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1bc0b09b0 0x7fa1bc156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.272+0000 7fa1ba59c700 1 -- 172.21.15.7:0/2160112757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1bc0b09b0 msgr2=0x7fa1bc156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.272+0000 7fa1ba59c700 1 --2- 172.21.15.7:0/2160112757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1bc0b09b0 0x7fa1bc156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.272+0000 7fa1ba59c700 1 -- 172.21.15.7:0/2160112757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1bc141850 msgr2=0x7fa1bc15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:27.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.272+0000 7fa1ba59c700 1 --2- 172.21.15.7:0/2160112757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1bc141850 0x7fa1bc15d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.272+0000 7fa1ba59c700 1 -- 172.21.15.7:0/2160112757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b4009ae0 con 0x7fa1bc0637d0 2024-01-27T02:39:27.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.272+0000 7fa1ba59c700 1 --2- 172.21.15.7:0/2160112757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1bc0637d0 0x7fa1bc0c55d0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00c5b0 tx=0x7fa1ac0020e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:27.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.272+0000 7fa1aa7fc700 1 -- 172.21.15.7:0/2160112757 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac011590 con 0x7fa1bc0637d0 2024-01-27T02:39:27.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.273+0000 7fa1aa7fc700 1 -- 172.21.15.7:0/2160112757 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ac0116f0 con 0x7fa1bc0637d0 2024-01-27T02:39:27.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.273+0000 7fa1aa7fc700 1 -- 172.21.15.7:0/2160112757 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac011a90 con 0x7fa1bc0637d0 2024-01-27T02:39:27.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.273+0000 7fa1c0a0d700 1 -- 172.21.15.7:0/2160112757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1bc15ce70 con 0x7fa1bc0637d0 2024-01-27T02:39:27.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.273+0000 7fa1c0a0d700 1 -- 172.21.15.7:0/2160112757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1bc0c5b10 con 0x7fa1bc0637d0 2024-01-27T02:39:27.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.274+0000 7fa1c0a0d700 1 -- 172.21.15.7:0/2160112757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1bc04c710 con 0x7fa1bc0637d0 2024-01-27T02:39:27.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.274+0000 7fa1aa7fc700 1 -- 172.21.15.7:0/2160112757 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa1ac012420 con 0x7fa1bc0637d0 2024-01-27T02:39:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.275+0000 7fa1aa7fc700 1 --2- 172.21.15.7:0/2160112757 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa19c066f30 0x7fa19c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.275+0000 7fa1aa7fc700 1 -- 172.21.15.7:0/2160112757 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa1ac084040 con 0x7fa1bc0637d0 2024-01-27T02:39:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.275+0000 7fa1b9d9b700 1 --2- 172.21.15.7:0/2160112757 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa19c066f30 0x7fa19c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.275+0000 7fa1b9d9b700 1 --2- 172.21.15.7:0/2160112757 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa19c066f30 0x7fa19c0693e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc1403f0 tx=0x7fa1b0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.279+0000 7fa1aa7fc700 1 -- 172.21.15.7:0/2160112757 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1ac0512b0 con 0x7fa1bc0637d0 2024-01-27T02:39:27.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.479+0000 7fa1c0a0d700 1 -- 172.21.15.7:0/2160112757 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa1bc15d600 con 0x7fa19c066f30 2024-01-27T02:39:27.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.487+0000 7fa1aa7fc700 1 -- 172.21.15.7:0/2160112757 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276311 (secure 0 0 0) 0x7fa1b000c050 con 0x7fa19c066f30 2024-01-27T02:39:27.491 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:27.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.491+0000 7fa1c0a0d700 1 -- 172.21.15.7:0/2160112757 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa19c066f30 msgr2=0x7fa19c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:27.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.491+0000 7fa1c0a0d700 1 --2- 172.21.15.7:0/2160112757 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa19c066f30 0x7fa19c0693e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc1403f0 tx=0x7fa1b0008040 comp rx=0 tx=0).stop 2024-01-27T02:39:27.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.491+0000 7fa1c0a0d700 1 -- 172.21.15.7:0/2160112757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1bc0637d0 msgr2=0x7fa1bc0c55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:27.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.491+0000 7fa1c0a0d700 1 --2- 172.21.15.7:0/2160112757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1bc0637d0 0x7fa1bc0c55d0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00c5b0 tx=0x7fa1ac0020e0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.491+0000 7fa1c0a0d700 1 -- 172.21.15.7:0/2160112757 shutdown_connections 2024-01-27T02:39:27.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.491+0000 7fa1c0a0d700 1 --2- 172.21.15.7:0/2160112757 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa19c066f30 0x7fa19c0693e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.491+0000 7fa1c0a0d700 1 --2- 172.21.15.7:0/2160112757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1bc141850 0x7fa1bc15d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.491+0000 7fa1c0a0d700 1 --2- 172.21.15.7:0/2160112757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1bc0637d0 0x7fa1bc0c55d0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.491+0000 7fa1c0a0d700 1 --2- 172.21.15.7:0/2160112757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1bc0b09b0 0x7fa1bc156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.491+0000 7fa1c0a0d700 1 -- 172.21.15.7:0/2160112757 >> 172.21.15.7:0/2160112757 conn(0x7fa1bc0cdbd0 msgr2=0x7fa1bc0c2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:27.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.491+0000 7fa1c0a0d700 1 -- 172.21.15.7:0/2160112757 shutdown_connections 2024-01-27T02:39:27.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.491+0000 7fa1c0a0d700 1 -- 172.21.15.7:0/2160112757 wait complete. 2024-01-27T02:39:27.494 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:39:27.505 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-01-27T02:39:25.679505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7420,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555079,"statfs":{"total":1600000000,"available":1592401644,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.799011"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.884300+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:30:40.460262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.883346+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:35:46.505332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.885036+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:50:06.070356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.886377+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:00:11.450382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.885720+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:50:34.405562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.883530+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:09:13.278758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.881548+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:25:46.555556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.884760+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:55:51.094591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.882435+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:00:11.450382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.882953+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:04:42.115329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.883124+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:26:01.500470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.884186+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:35:37.535621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.881260+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:14:14.430917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.885318+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:08:53.065440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.884206+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:06:23.555202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.882920+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:49.973643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.884627+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:15:14.538837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.881587+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:17:06.668947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.882767+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:52:32.402932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.882686+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:26:41.909600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.882944+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:08:05.792443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.882900+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:47.650928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.884008+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:11.861136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.881284+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:15:18.285546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.883832+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:37:46.123420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.884817+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:17:25.995339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.884190+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:53:49.412236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.886042+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:51:21.995075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.882722+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:19:26.932201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.882871+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:18:01.492341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.884605+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:33:45.692091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-27T02:39:21.879721+0000","last_change":"2024-01-27T02:39:21.879721+0000","last_active":"2024-01-27T02:39:21.879721+0000","last_peered":"2024-01-27T02:39:21.879721+0000","last_clean":"2024-01-27T02:39:21.879721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:21.879721+0000","last_undegraded":"2024-01-27T02:39:21.879721+0000","last_fullsized":"2024-01-27T02:39:21.879721+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-27T02:39:20.877501+0000","last_change":"2024-01-27T02:39:20.883302+0000","last_active":"2024-01-27T02:39:20.877501+0000","last_peered":"2024-01-27T02:39:20.877501+0000","last_clean":"2024-01-27T02:39:20.877501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-27T02:39:20.877501+0000","last_undegraded":"2024-01-27T02:39:20.877501+0000","last_fullsized":"2024-01-27T02:39:20.877501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:47:35.007943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":271,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-01-27T02:39:19.239201+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:39:19.239201+0000","last_peered":"2024-01-27T02:39:19.239201+0000","last_clean":"2024-01-27T02:39:19.239201+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:39:19.239201+0000","last_undegraded":"2024-01-27T02:39:19.239201+0000","last_fullsized":"2024-01-27T02:39:19.239201+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574939,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1362,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193949,"statfs":{"total":200000000,"available":198604527,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.39400000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76200000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66900000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.58199999999999996}]},{"osd":1,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76900000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.56000000000000005},"min":{"1min":0.41799999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.67600000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.52700000000000002}]},{"osd":2,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.67900000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.49199999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76200000000000001,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.63300000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.48399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.66800000000000004,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.45400000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.74099999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.76300000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47699999999999998,"15min":0.48899999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70699999999999996,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.437},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.48899999999999999,"15min":0.51200000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69299999999999995,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.59399999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.51800000000000002,"15min":0.53700000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.307},"max":{"1min":0.73199999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55200000000000005,"15min":0.56499999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33500000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.61699999999999999}]},{"osd":6,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57799999999999996,"15min":0.59599999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.90800000000000003},"last":0.64500000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574939,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":780,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194531,"statfs":{"total":200000000,"available":199200700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.63,"15min":0.54400000000000004},"min":{"1min":0.375,"5min":0.36499999999999999,"15min":0.29999999999999999},"max":{"1min":0.83099999999999996,"5min":4.125,"15min":4.125},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59699999999999998,"15min":0.54000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32700000000000001},"max":{"1min":0.75600000000000001,"5min":1.216,"15min":1.216},"last":0.68200000000000005}]},{"osd":1,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.67200000000000004,"15min":0.55600000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.34100000000000003},"max":{"1min":0.84199999999999997,"5min":8.2699999999999996,"15min":8.2699999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.88900000000000001,"15min":0.65500000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80600000000000005,"5min":26.411999999999999,"15min":26.411999999999999},"last":0.54100000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.75700000000000001,"15min":0.60799999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77000000000000002,"5min":17.143999999999998,"15min":17.143999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.64400000000000002,"15min":0.59299999999999997},"min":{"1min":0.45800000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85799999999999998,"5min":1.76,"15min":1.76},"last":0.622}]},{"osd":3,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.624,"15min":0.56899999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.376},"max":{"1min":0.78000000000000003,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.624,"15min":0.58199999999999996},"min":{"1min":0.44500000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79300000000000004,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.59099999999999997}]},{"osd":4,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.30199999999999999},"max":{"1min":0.73599999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.50900000000000001,"15min":0.54300000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84899999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.73999999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.80800000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56299999999999994,"15min":0.59599999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.71099999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59299999999999997,"15min":0.60999999999999999},"min":{"1min":0.40999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.87,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55700000000000005,"15min":0.58199999999999996},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.36499999999999999},"max":{"1min":0.82899999999999996,"5min":1.038,"15min":1.038},"last":0.69599999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":780,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194531,"statfs":{"total":200000000,"available":199200700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39800000000000002,"15min":0.378},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41799999999999998,"15min":0.40000000000000002},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.50600000000000001,"5min":1.121,"15min":1.121},"last":0.66100000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40799999999999997,"15min":0.373},"min":{"1min":0.249,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":1.0940000000000001,"5min":1.25,"15min":1.25},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.44900000000000001,"15min":0.43099999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29899999999999999},"max":{"1min":0.498,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.54200000000000004}]},{"osd":3,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.48499999999999999,"15min":0.46899999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.32100000000000001},"max":{"1min":0.56699999999999995,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.46600000000000003,"15min":0.45500000000000002},"min":{"1min":0.34899999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.56100000000000005,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.55600000000000005}]},{"osd":4,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.66600000000000004,"15min":0.63600000000000001},"min":{"1min":0.48299999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.85699999999999998,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66600000000000004,"15min":0.64000000000000001},"min":{"1min":0.45000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91600000000000004,"5min":1.026,"15min":1.026},"last":0.628}]},{"osd":5,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69099999999999995,"15min":0.64200000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":0.93400000000000005,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69699999999999995,"15min":0.67600000000000005},"min":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96199999999999997,"5min":3.181,"15min":3.181},"last":0.64600000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.70599999999999996,"15min":0.69499999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88500000000000001,"5min":0.96399999999999997,"15min":1.024},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72099999999999997,"15min":0.68300000000000005},"min":{"1min":0.54000000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93600000000000005,"5min":1.869,"15min":1.869},"last":0.81100000000000005}]},{"osd":7,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73199999999999998,"15min":0.69099999999999995},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.47299999999999998},"max":{"1min":0.91400000000000003,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73099999999999998,"15min":0.69499999999999995},"min":{"1min":0.53000000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92000000000000004,"5min":1.708,"15min":1.708},"last":0.68100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574939,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1362,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193949,"statfs":{"total":200000000,"available":198604527,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.40600000000000003,"15min":0.39400000000000002},"min":{"1min":0.252,"5min":0.24399999999999999,"15min":0.23799999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.437,"5min":0.42899999999999999,"15min":0.41799999999999998},"min":{"1min":0.28899999999999998,"5min":0.27800000000000002,"15min":0.223},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.372}]},{"osd":2,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.433,"15min":0.42199999999999999},"min":{"1min":0.31,"5min":0.28699999999999998,"15min":0.28199999999999997},"max":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.45700000000000002,"15min":0.438},"min":{"1min":0.312,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.443}]},{"osd":3,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.441,"15min":0.432},"min":{"1min":0.315,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.66100000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.495,"15min":0.47599999999999998},"min":{"1min":0.38800000000000001,"5min":0.33600000000000002,"15min":0.29099999999999998},"max":{"1min":0.67600000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.39300000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66800000000000004,"15min":0.64400000000000002},"min":{"1min":0.505,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80900000000000005,"5min":0.997,"15min":0.997},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.45800000000000002,"5min":0.44,"15min":0.41999999999999998},"max":{"1min":0.83399999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.81899999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68899999999999995,"15min":0.65800000000000003},"min":{"1min":0.51900000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.78800000000000003,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.54900000000000004,"5min":0.47799999999999998,"15min":0.44},"max":{"1min":0.84999999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.83699999999999997}]},{"osd":6,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68300000000000005,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.42499999999999999},"max":{"1min":0.79800000000000004,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69899999999999995,"15min":0.68000000000000005},"min":{"1min":0.51600000000000001,"5min":0.45700000000000002,"15min":0.45000000000000001},"max":{"1min":0.90300000000000002,"5min":1.04,"15min":1.04},"last":0.77200000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.50800000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90200000000000002,"5min":1.014,"15min":1.014},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69699999999999995,"15min":0.69199999999999995},"min":{"1min":0.54500000000000004,"5min":0.498,"15min":0.498},"max":{"1min":0.81999999999999995,"5min":0.95799999999999996,"15min":0.97299999999999998},"last":0.90300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":786,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194526,"statfs":{"total":200000000,"available":199194895,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.41199999999999998},"min":{"1min":0.27300000000000002,"5min":0.23599999999999999,"15min":0.23000000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.46700000000000003,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.30499999999999999},"max":{"1min":0.68899999999999995,"5min":0.72799999999999998,"15min":6.2729999999999997},"last":0.34300000000000003}]},{"osd":1,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.433,"15min":0.42299999999999999},"min":{"1min":0.33800000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.621,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.44},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.439,"15min":0.44400000000000001},"min":{"1min":0.33900000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.60399999999999998,"5min":0.65300000000000002,"15min":0.70099999999999996},"last":0.29399999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.434,"15min":0.41799999999999998},"min":{"1min":0.32200000000000001,"5min":0.28899999999999998,"15min":0.27800000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49099999999999999,"15min":0.48599999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.38100000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68799999999999994,"15min":0.65900000000000003},"min":{"1min":0.499,"5min":0.46800000000000003,"15min":0.39300000000000002},"max":{"1min":0.90500000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.70599999999999996,"15min":0.78100000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95799999999999996,"5min":1.006,"15min":3.581},"last":0.65600000000000003}]},{"osd":5,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65800000000000003,"15min":0.72699999999999998},"min":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92200000000000004,"5min":0.95899999999999996,"15min":3.5369999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.52200000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89100000000000001,"5min":0.92200000000000004,"15min":1.0529999999999999},"last":0.68000000000000005}]},{"osd":6,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67200000000000004,"15min":0.66700000000000004},"min":{"1min":0.57499999999999996,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96499999999999997,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69599999999999995,"15min":0.68000000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.97099999999999997,"5min":1.0189999999999999,"15min":1.0209999999999999},"last":0.69599999999999995}]},{"osd":7,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69199999999999995,"15min":0.67900000000000005},"min":{"1min":0.58799999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97699999999999998,"5min":1.03,"15min":1.03},"last":0.62},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70899999999999996,"15min":0.69299999999999995},"min":{"1min":0.55200000000000005,"5min":0.441,"15min":0.42699999999999999},"max":{"1min":0.99099999999999999,"5min":1.036,"15min":1.036},"last":0.72699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":780,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194531,"statfs":{"total":200000000,"available":199200700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.45500000000000002,"15min":0.38600000000000001},"min":{"1min":0.29699999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.67900000000000005,"5min":4.7380000000000004,"15min":4.7380000000000004},"last":0.37},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41899999999999998,"15min":0.40000000000000002},"min":{"1min":0.29399999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55500000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.501}]},{"osd":1,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.47499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28000000000000003,"5min":0.27400000000000002,"15min":0.22900000000000001},"max":{"1min":0.61599999999999999,"5min":4.5739999999999998,"15min":4.5739999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.53300000000000003,"15min":0.45200000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.98199999999999998,"5min":5.0529999999999999,"15min":5.0529999999999999},"last":0.45900000000000002}]},{"osd":2,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.49199999999999999,"15min":0.443},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.28100000000000003},"max":{"1min":0.76700000000000002,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.55400000000000005,"15min":0.47699999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71799999999999997,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":0.52100000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69999999999999996,"15min":0.63800000000000001},"min":{"1min":0.45300000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91600000000000004,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.62},"min":{"1min":0.48199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86099999999999999,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.749}]},{"osd":5,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67600000000000005,"15min":0.626},"min":{"1min":0.438,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82399999999999995,"5min":1.409,"15min":1.409},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.78700000000000003,"15min":0.70399999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.89600000000000002,"5min":4.7960000000000003,"15min":4.7960000000000003},"last":0.70199999999999996}]},{"osd":6,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.73899999999999999,"15min":0.67900000000000005},"min":{"1min":0.42799999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92700000000000005,"5min":5.0069999999999997,"15min":5.0069999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.69399999999999995,"15min":0.65900000000000003},"min":{"1min":0.47499999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.94199999999999995,"5min":1.103,"15min":1.103},"last":0.68100000000000005}]},{"osd":7,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.75700000000000001,"15min":0.66900000000000004},"min":{"1min":0.57599999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72699999999999998,"15min":0.68000000000000005},"min":{"1min":0.52800000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.88900000000000001,"5min":1.361,"15min":1.361},"last":0.83399999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":786,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194526,"statfs":{"total":200000000,"available":199194895,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.35599999999999998,"5min":0.34599999999999997,"15min":0.314},"max":{"1min":1.137,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66200000000000003,"15min":0.61399999999999999},"min":{"1min":0.496,"5min":0.372,"15min":0.372},"max":{"1min":1.016,"5min":7.0670000000000002,"15min":7.0670000000000002},"last":0.623}]},{"osd":1,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.64200000000000002,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.35399999999999998},"max":{"1min":0.81000000000000005,"5min":5.1529999999999996,"15min":5.1529999999999996},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64400000000000002,"15min":0.60899999999999999},"min":{"1min":0.46400000000000002,"5min":0.40799999999999997,"15min":0.39000000000000001},"max":{"1min":1.054,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.60699999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.64200000000000002,"15min":0.61099999999999999},"min":{"1min":0.48399999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76600000000000001,"5min":5.0019999999999998,"15min":5.0019999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.63300000000000001,"15min":0.62},"min":{"1min":0.432,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2549999999999999,"5min":2.073,"15min":2.073},"last":0.57199999999999995}]},{"osd":3,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.60499999999999998},"min":{"1min":0.45200000000000001,"5min":0.42099999999999999,"15min":0.36199999999999999},"max":{"1min":1.1899999999999999,"5min":6.7220000000000004,"15min":6.7220000000000004},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68999999999999995,"15min":0.66400000000000003},"min":{"1min":0.46100000000000002,"5min":0.44900000000000001,"15min":0.41399999999999998},"max":{"1min":0.88,"5min":6.5510000000000002,"15min":6.5510000000000002},"last":0.51600000000000001}]},{"osd":5,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.34899999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.77500000000000002,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.503},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56599999999999995,"15min":0.51700000000000002},"min":{"1min":0.372,"5min":0.312,"15min":0.29999999999999999},"max":{"1min":0.68799999999999994,"5min":6.9269999999999996,"15min":6.9269999999999996},"last":0.52700000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56100000000000005,"15min":0.52200000000000002},"min":{"1min":0.40400000000000003,"5min":0.34999999999999998,"15min":0.32900000000000001},"max":{"1min":0.78600000000000003,"5min":5.2229999999999999,"15min":5.2229999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56200000000000006,"15min":0.54900000000000004},"min":{"1min":0.41799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82999999999999996,"5min":1.46,"15min":1.46},"last":0.44700000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61699999999999999,"15min":0.57499999999999996},"min":{"1min":0.434,"5min":0.34799999999999998,"15min":0.33600000000000002},"max":{"1min":0.78300000000000003,"5min":7.5339999999999998,"15min":7.5339999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.629,"15min":0.59399999999999997},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85299999999999998,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":0.42699999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574939,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":780,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194531,"statfs":{"total":200000000,"available":199200700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56399999999999995,"15min":0.58999999999999997},"min":{"1min":0.42099999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56699999999999995,"15min":0.57899999999999996},"min":{"1min":0.442,"5min":0.34499999999999997,"15min":0.27600000000000002},"max":{"1min":0.77100000000000002,"5min":0.78900000000000003,"15min":3.242},"last":0.65100000000000002}]},{"osd":1,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.622,"15min":0.61299999999999999},"min":{"1min":0.38500000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.3380000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61099999999999999,"15min":0.64300000000000002},"min":{"1min":0.35399999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78400000000000003,"5min":0.84399999999999997,"15min":0.95499999999999996},"last":0.66500000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58599999999999997,"15min":0.59699999999999998},"min":{"1min":0.40400000000000003,"5min":0.39100000000000001,"15min":0.372},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":3.04},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.48499999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80400000000000005,"5min":0.85899999999999999,"15min":0.89400000000000002},"last":0.55600000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60499999999999998,"15min":0.66800000000000004},"min":{"1min":0.40300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":4.032},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65500000000000003,"15min":0.67700000000000005},"min":{"1min":0.48999999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.76100000000000001,"5min":0.86199999999999999,"15min":4.282},"last":0.63600000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50700000000000001,"15min":0.52200000000000002},"min":{"1min":0.35799999999999998,"5min":0.30599999999999999,"15min":0.26500000000000001},"max":{"1min":0.63600000000000001,"5min":0.81399999999999995,"15min":1.2869999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.495,"15min":0.55300000000000005},"min":{"1min":0.32800000000000001,"5min":0.318,"15min":0.30399999999999999},"max":{"1min":0.66900000000000004,"5min":0.79800000000000004,"15min":1.101},"last":0.70499999999999996}]},{"osd":6,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.497,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65200000000000002,"5min":0.73199999999999998,"15min":1.7969999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55300000000000005,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80100000000000005,"5min":0.85799999999999998,"15min":1.6739999999999999},"last":0.75}]},{"osd":7,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.38200000000000001,"5min":0.34799999999999998,"15min":0.32700000000000001},"max":{"1min":0.66800000000000004,"5min":0.77100000000000002,"15min":1.008},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59699999999999998,"15min":0.64900000000000002},"min":{"1min":0.436,"5min":0.35899999999999999,"15min":0.33700000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":2},"last":0.77000000000000002}]}]}],"pool_statfs":[]}} 2024-01-27T02:39:27.509 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:39:27.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.679+0000 7f22c22e0700 1 Processor -- start 2024-01-27T02:39:27.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.679+0000 7f22c22e0700 1 -- start start 2024-01-27T02:39:27.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.680+0000 7f22c22e0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22bc0c4cc0 0x7f22bc0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:27.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.680+0000 7f22c22e0700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22bc0ca540 0x7f22bc163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:27.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.680+0000 7f22c22e0700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22bc0ce5e0 con 0x7f22bc15a100 2024-01-27T02:39:27.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.680+0000 7f22c22e0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22bc0cd9d0 con 0x7f22bc0c4cc0 2024-01-27T02:39:27.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.680+0000 7f22c22e0700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0cdcb0 con 0x7f22bc0ca540 2024-01-27T02:39:27.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.680+0000 7f22bb7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22bc0c4cc0 0x7f22bc0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:27.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.680+0000 7f22bb7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22bc0c4cc0 0x7f22bc0cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37720/0 (socket says 172.21.15.7:37720) 2024-01-27T02:39:27.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.680+0000 7f22bb7fe700 1 -- 172.21.15.7:0/2829651099 learned_addr learned my addr 172.21.15.7:0/2829651099 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:27.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.680+0000 7f22c087d700 1 --2- 172.21.15.7:0/2829651099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22bc0ca540 0x7f22bc163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:27.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.680+0000 7f22bb7fe700 1 -- 172.21.15.7:0/2829651099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22bc0ca540 msgr2=0x7f22bc163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:27.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.680+0000 7f22bb7fe700 1 --2- 172.21.15.7:0/2829651099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22bc0ca540 0x7f22bc163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.680+0000 7f22bb7fe700 1 -- 172.21.15.7:0/2829651099 >> v1:172.21.15.188:6789/0 conn(0x7f22bc15a100 legacy=0x7f22bc0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:27.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.680+0000 7f22bb7fe700 1 -- 172.21.15.7:0/2829651099 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22bc0cdf90 con 0x7f22bc0c4cc0 2024-01-27T02:39:27.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.681+0000 7f22bb7fe700 1 --2- 172.21.15.7:0/2829651099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc0c4cc0 0x7f22bc0cd2f0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f22b00099b0 tx=0x7f22b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9abdb4a7cfed5f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:27.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.681+0000 7f22baffd700 1 -- 172.21.15.7:0/2829651099 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b0009ea0 con 0x7f22bc0c4cc0 2024-01-27T02:39:27.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.681+0000 7f22baffd700 1 -- 172.21.15.7:0/2829651099 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b000b860 con 0x7f22bc0c4cc0 2024-01-27T02:39:27.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.681+0000 7f22baffd700 1 -- 172.21.15.7:0/2829651099 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b000bc00 con 0x7f22bc0c4cc0 2024-01-27T02:39:27.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.681+0000 7f22c22e0700 1 -- 172.21.15.7:0/2829651099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc0c4cc0 msgr2=0x7f22bc0cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:27.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.681+0000 7f22c22e0700 1 --2- 172.21.15.7:0/2829651099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc0c4cc0 0x7f22bc0cd2f0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f22b00099b0 tx=0x7f22b0009250 comp rx=0 tx=0).stop 2024-01-27T02:39:27.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.681+0000 7f22c22e0700 1 -- 172.21.15.7:0/2829651099 shutdown_connections 2024-01-27T02:39:27.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.681+0000 7f22c22e0700 1 --2- 172.21.15.7:0/2829651099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22bc0ca540 0x7f22bc163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.681+0000 7f22c22e0700 1 --2- 172.21.15.7:0/2829651099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc0c4cc0 0x7f22bc0cd2f0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.682+0000 7f22c22e0700 1 -- 172.21.15.7:0/2829651099 >> 172.21.15.7:0/2829651099 conn(0x7f22bc098020 msgr2=0x7f22bc0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:27.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.682+0000 7f22c22e0700 1 -- 172.21.15.7:0/2829651099 shutdown_connections 2024-01-27T02:39:27.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.682+0000 7f22c22e0700 1 -- 172.21.15.7:0/2829651099 wait complete. 2024-01-27T02:39:27.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.683+0000 7f22c22e0700 1 Processor -- start 2024-01-27T02:39:27.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.683+0000 7f22c22e0700 1 -- start start 2024-01-27T02:39:27.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.684+0000 7f22c22e0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc0c4cc0 0x7f22bc152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:27.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.684+0000 7f22c22e0700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22bc0ca540 0x7f22bc0b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:27.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.684+0000 7f22c22e0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22bc15a100 0x7f22bc0b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:27.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.684+0000 7f22c22e0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0c6900 con 0x7f22bc0c4cc0 2024-01-27T02:39:27.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.684+0000 7f22c22e0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0b2a30 con 0x7f22bc15a100 2024-01-27T02:39:27.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.684+0000 7f22c22e0700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0b2d10 con 0x7f22bc0ca540 2024-01-27T02:39:27.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.685+0000 7f22bbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc0c4cc0 0x7f22bc152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:27.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.685+0000 7f22bbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc0c4cc0 0x7f22bc152710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37734/0 (socket says 172.21.15.7:37734) 2024-01-27T02:39:27.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.685+0000 7f22bbfff700 1 -- 172.21.15.7:0/1725227465 learned_addr learned my addr 172.21.15.7:0/1725227465 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:27.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.685+0000 7f22bb7fe700 1 --2- 172.21.15.7:0/1725227465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22bc0ca540 0x7f22bc0b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:27.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.685+0000 7f22c087d700 1 --2- 172.21.15.7:0/1725227465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22bc15a100 0x7f22bc0b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:27.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.685+0000 7f22bbfff700 1 -- 172.21.15.7:0/1725227465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22bc0ca540 msgr2=0x7f22bc0b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:27.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.685+0000 7f22bbfff700 1 --2- 172.21.15.7:0/1725227465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22bc0ca540 0x7f22bc0b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.685+0000 7f22bbfff700 1 -- 172.21.15.7:0/1725227465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22bc15a100 msgr2=0x7f22bc0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:27.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.685+0000 7f22bbfff700 1 --2- 172.21.15.7:0/1725227465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22bc15a100 0x7f22bc0b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.685+0000 7f22bbfff700 1 -- 172.21.15.7:0/1725227465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b00094f0 con 0x7f22bc0c4cc0 2024-01-27T02:39:27.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.685+0000 7f22bbfff700 1 --2- 172.21.15.7:0/1725227465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc0c4cc0 0x7f22bc152710 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00c590 tx=0x7f22ac00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:27.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.685+0000 7f22a3fff700 1 -- 172.21.15.7:0/1725227465 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac011590 con 0x7f22bc0c4cc0 2024-01-27T02:39:27.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.685+0000 7f22a3fff700 1 -- 172.21.15.7:0/1725227465 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ac011710 con 0x7f22bc0c4cc0 2024-01-27T02:39:27.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.685+0000 7f22c22e0700 1 -- 172.21.15.7:0/1725227465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22bc0b2ff0 con 0x7f22bc0c4cc0 2024-01-27T02:39:27.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.685+0000 7f22a3fff700 1 -- 172.21.15.7:0/1725227465 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac011ad0 con 0x7f22bc0c4cc0 2024-01-27T02:39:27.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.685+0000 7f22c22e0700 1 -- 172.21.15.7:0/1725227465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22bc152c50 con 0x7f22bc0c4cc0 2024-01-27T02:39:27.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.687+0000 7f22c22e0700 1 -- 172.21.15.7:0/1725227465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22bc0b3510 con 0x7f22bc0c4cc0 2024-01-27T02:39:27.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.687+0000 7f22a3fff700 1 -- 172.21.15.7:0/1725227465 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f22ac01b3b0 con 0x7f22bc0c4cc0 2024-01-27T02:39:27.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.688+0000 7f22a3fff700 1 --2- 172.21.15.7:0/1725227465 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f22a8066f30 0x7f22a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:27.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.688+0000 7f22a3fff700 1 -- 172.21.15.7:0/1725227465 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f22ac0839a0 con 0x7f22bc0c4cc0 2024-01-27T02:39:27.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.688+0000 7f22bb7fe700 1 --2- 172.21.15.7:0/1725227465 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f22a8066f30 0x7f22a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:27.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.688+0000 7f22bb7fe700 1 --2- 172.21.15.7:0/1725227465 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f22a8066f30 0x7f22a80693e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f22b00099b0 tx=0x7f22b0017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.692+0000 7f22a3fff700 1 -- 172.21.15.7:0/1725227465 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22bc0b3510 con 0x7f22bc0c4cc0 2024-01-27T02:39:27.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.890+0000 7f22c22e0700 1 -- 172.21.15.7:0/1725227465 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22bc152e40 con 0x7f22a8066f30 2024-01-27T02:39:27.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.899+0000 7f22a3fff700 1 -- 172.21.15.7:0/1725227465 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285521 (secure 0 0 0) 0x7f22bc152e40 con 0x7f22a8066f30 2024-01-27T02:39:27.903 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:27.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.902+0000 7f22c22e0700 1 -- 172.21.15.7:0/1725227465 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f22a8066f30 msgr2=0x7f22a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:27.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.902+0000 7f22c22e0700 1 --2- 172.21.15.7:0/1725227465 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f22a8066f30 0x7f22a80693e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f22b00099b0 tx=0x7f22b0017000 comp rx=0 tx=0).stop 2024-01-27T02:39:27.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.902+0000 7f22c22e0700 1 -- 172.21.15.7:0/1725227465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc0c4cc0 msgr2=0x7f22bc152710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:27.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.902+0000 7f22c22e0700 1 --2- 172.21.15.7:0/1725227465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc0c4cc0 0x7f22bc152710 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00c590 tx=0x7f22ac00c8a0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.902+0000 7f22c22e0700 1 -- 172.21.15.7:0/1725227465 shutdown_connections 2024-01-27T02:39:27.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.902+0000 7f22c22e0700 1 --2- 172.21.15.7:0/1725227465 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f22a8066f30 0x7f22a80693e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.902+0000 7f22c22e0700 1 --2- 172.21.15.7:0/1725227465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22bc15a100 0x7f22bc0b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.902+0000 7f22c22e0700 1 --2- 172.21.15.7:0/1725227465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc0c4cc0 0x7f22bc152710 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.902+0000 7f22c22e0700 1 --2- 172.21.15.7:0/1725227465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22bc0ca540 0x7f22bc0b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:27.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.902+0000 7f22c22e0700 1 -- 172.21.15.7:0/1725227465 >> 172.21.15.7:0/1725227465 conn(0x7f22bc098020 msgr2=0x7f22bc154650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:27.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.902+0000 7f22c22e0700 1 -- 172.21.15.7:0/1725227465 shutdown_connections 2024-01-27T02:39:27.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:27.902+0000 7f22c22e0700 1 -- 172.21.15.7:0/1725227465 wait complete. 2024-01-27T02:39:27.906 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:39:27.916 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-01-27T02:39:27.679897+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7598,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554901,"statfs":{"total":1600000000,"available":1592219300,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001385"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.898603+0000","last_change":"2024-01-27T02:39:22.898429+0000","last_active":"2024-01-27T02:39:22.898603+0000","last_peered":"2024-01-27T02:39:22.898603+0000","last_clean":"2024-01-27T02:39:22.898603+0000","last_became_active":"2024-01-27T02:39:22.897112+0000","last_became_peered":"2024-01-27T02:39:22.897112+0000","last_unstale":"2024-01-27T02:39:22.898603+0000","last_undegraded":"2024-01-27T02:39:22.898603+0000","last_fullsized":"2024-01-27T02:39:22.898603+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:00:12.452602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902737+0000","last_change":"2024-01-27T02:39:22.902650+0000","last_active":"2024-01-27T02:39:22.902737+0000","last_peered":"2024-01-27T02:39:22.902737+0000","last_clean":"2024-01-27T02:39:22.902737+0000","last_became_active":"2024-01-27T02:39:22.902344+0000","last_became_peered":"2024-01-27T02:39:22.902344+0000","last_unstale":"2024-01-27T02:39:22.902737+0000","last_undegraded":"2024-01-27T02:39:22.902737+0000","last_fullsized":"2024-01-27T02:39:22.902737+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:02:57.717352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.899868+0000","last_change":"2024-01-27T02:39:22.899799+0000","last_active":"2024-01-27T02:39:22.899868+0000","last_peered":"2024-01-27T02:39:22.899868+0000","last_clean":"2024-01-27T02:39:22.899868+0000","last_became_active":"2024-01-27T02:39:22.899535+0000","last_became_peered":"2024-01-27T02:39:22.899535+0000","last_unstale":"2024-01-27T02:39:22.899868+0000","last_undegraded":"2024-01-27T02:39:22.899868+0000","last_fullsized":"2024-01-27T02:39:22.899868+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:27:21.148523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.905689+0000","last_change":"2024-01-27T02:39:22.904667+0000","last_active":"2024-01-27T02:39:22.905689+0000","last_peered":"2024-01-27T02:39:22.905689+0000","last_clean":"2024-01-27T02:39:22.905689+0000","last_became_active":"2024-01-27T02:39:22.900570+0000","last_became_peered":"2024-01-27T02:39:22.900570+0000","last_unstale":"2024-01-27T02:39:22.905689+0000","last_undegraded":"2024-01-27T02:39:22.905689+0000","last_fullsized":"2024-01-27T02:39:22.905689+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:17:26.997560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.897810+0000","last_change":"2024-01-27T02:39:22.897296+0000","last_active":"2024-01-27T02:39:22.897810+0000","last_peered":"2024-01-27T02:39:22.897810+0000","last_clean":"2024-01-27T02:39:22.897810+0000","last_became_active":"2024-01-27T02:39:22.897051+0000","last_became_peered":"2024-01-27T02:39:22.897051+0000","last_unstale":"2024-01-27T02:39:22.897810+0000","last_undegraded":"2024-01-27T02:39:22.897810+0000","last_fullsized":"2024-01-27T02:39:22.897810+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:33:46.694312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.900968+0000","last_change":"2024-01-27T02:39:22.900656+0000","last_active":"2024-01-27T02:39:22.900968+0000","last_peered":"2024-01-27T02:39:22.900968+0000","last_clean":"2024-01-27T02:39:22.900968+0000","last_became_active":"2024-01-27T02:39:22.899965+0000","last_became_peered":"2024-01-27T02:39:22.899965+0000","last_unstale":"2024-01-27T02:39:22.900968+0000","last_undegraded":"2024-01-27T02:39:22.900968+0000","last_fullsized":"2024-01-27T02:39:22.900968+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:09:14.280979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.897983+0000","last_change":"2024-01-27T02:39:22.897809+0000","last_active":"2024-01-27T02:39:22.897983+0000","last_peered":"2024-01-27T02:39:22.897983+0000","last_clean":"2024-01-27T02:39:22.897983+0000","last_became_active":"2024-01-27T02:39:22.897437+0000","last_became_peered":"2024-01-27T02:39:22.897437+0000","last_unstale":"2024-01-27T02:39:22.897983+0000","last_undegraded":"2024-01-27T02:39:22.897983+0000","last_fullsized":"2024-01-27T02:39:22.897983+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:02:31.201590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.904534+0000","last_change":"2024-01-27T02:39:22.903395+0000","last_active":"2024-01-27T02:39:22.904534+0000","last_peered":"2024-01-27T02:39:22.904534+0000","last_clean":"2024-01-27T02:39:22.904534+0000","last_became_active":"2024-01-27T02:39:22.903023+0000","last_became_peered":"2024-01-27T02:39:22.903023+0000","last_unstale":"2024-01-27T02:39:22.904534+0000","last_undegraded":"2024-01-27T02:39:22.904534+0000","last_fullsized":"2024-01-27T02:39:22.904534+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:08:06.794664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910976+0000","last_change":"2024-01-27T02:39:22.906721+0000","last_active":"2024-01-27T02:39:22.910976+0000","last_peered":"2024-01-27T02:39:22.910976+0000","last_clean":"2024-01-27T02:39:22.910976+0000","last_became_active":"2024-01-27T02:39:22.906469+0000","last_became_peered":"2024-01-27T02:39:22.906469+0000","last_unstale":"2024-01-27T02:39:22.910976+0000","last_undegraded":"2024-01-27T02:39:22.910976+0000","last_fullsized":"2024-01-27T02:39:22.910976+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:04:43.117550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.897833+0000","last_change":"2024-01-27T02:39:22.897761+0000","last_active":"2024-01-27T02:39:22.897833+0000","last_peered":"2024-01-27T02:39:22.897833+0000","last_clean":"2024-01-27T02:39:22.897833+0000","last_became_active":"2024-01-27T02:39:22.897518+0000","last_became_peered":"2024-01-27T02:39:22.897518+0000","last_unstale":"2024-01-27T02:39:22.897833+0000","last_undegraded":"2024-01-27T02:39:22.897833+0000","last_fullsized":"2024-01-27T02:39:22.897833+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:47:36.010163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.900063+0000","last_change":"2024-01-27T02:39:22.899989+0000","last_active":"2024-01-27T02:39:22.900063+0000","last_peered":"2024-01-27T02:39:22.900063+0000","last_clean":"2024-01-27T02:39:22.900063+0000","last_became_active":"2024-01-27T02:39:22.899715+0000","last_became_peered":"2024-01-27T02:39:22.899715+0000","last_unstale":"2024-01-27T02:39:22.900063+0000","last_undegraded":"2024-01-27T02:39:22.900063+0000","last_fullsized":"2024-01-27T02:39:22.900063+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:22:14.092868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.898551+0000","last_change":"2024-01-27T02:39:22.898312+0000","last_active":"2024-01-27T02:39:22.898551+0000","last_peered":"2024-01-27T02:39:22.898551+0000","last_clean":"2024-01-27T02:39:22.898551+0000","last_became_active":"2024-01-27T02:39:22.898071+0000","last_became_peered":"2024-01-27T02:39:22.898071+0000","last_unstale":"2024-01-27T02:39:22.898551+0000","last_undegraded":"2024-01-27T02:39:22.898551+0000","last_fullsized":"2024-01-27T02:39:22.898551+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:17.789817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.904646+0000","last_change":"2024-01-27T02:39:22.901995+0000","last_active":"2024-01-27T02:39:22.904646+0000","last_peered":"2024-01-27T02:39:22.904646+0000","last_clean":"2024-01-27T02:39:22.904646+0000","last_became_active":"2024-01-27T02:39:22.901761+0000","last_became_peered":"2024-01-27T02:39:22.901761+0000","last_unstale":"2024-01-27T02:39:22.904646+0000","last_undegraded":"2024-01-27T02:39:22.904646+0000","last_fullsized":"2024-01-27T02:39:22.904646+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:50:35.407782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902718+0000","last_change":"2024-01-27T02:39:22.902243+0000","last_active":"2024-01-27T02:39:22.902718+0000","last_peered":"2024-01-27T02:39:22.902718+0000","last_clean":"2024-01-27T02:39:22.902718+0000","last_became_active":"2024-01-27T02:39:22.901971+0000","last_became_peered":"2024-01-27T02:39:22.901971+0000","last_unstale":"2024-01-27T02:39:22.902718+0000","last_undegraded":"2024-01-27T02:39:22.902718+0000","last_fullsized":"2024-01-27T02:39:22.902718+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:35:38.537841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.907395+0000","last_change":"2024-01-27T02:39:22.906123+0000","last_active":"2024-01-27T02:39:22.907395+0000","last_peered":"2024-01-27T02:39:22.907395+0000","last_clean":"2024-01-27T02:39:22.907395+0000","last_became_active":"2024-01-27T02:39:22.905311+0000","last_became_peered":"2024-01-27T02:39:22.905311+0000","last_unstale":"2024-01-27T02:39:22.907395+0000","last_undegraded":"2024-01-27T02:39:22.907395+0000","last_fullsized":"2024-01-27T02:39:22.907395+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:26:42.911820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.907825+0000","last_change":"2024-01-27T02:39:22.907096+0000","last_active":"2024-01-27T02:39:22.907825+0000","last_peered":"2024-01-27T02:39:22.907825+0000","last_clean":"2024-01-27T02:39:22.907825+0000","last_became_active":"2024-01-27T02:39:22.906214+0000","last_became_peered":"2024-01-27T02:39:22.906214+0000","last_unstale":"2024-01-27T02:39:22.907825+0000","last_undegraded":"2024-01-27T02:39:22.907825+0000","last_fullsized":"2024-01-27T02:39:22.907825+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:35:39.970169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.904556+0000","last_change":"2024-01-27T02:39:22.904462+0000","last_active":"2024-01-27T02:39:22.904556+0000","last_peered":"2024-01-27T02:39:22.904556+0000","last_clean":"2024-01-27T02:39:22.904556+0000","last_became_active":"2024-01-27T02:39:22.904033+0000","last_became_peered":"2024-01-27T02:39:22.904033+0000","last_unstale":"2024-01-27T02:39:22.904556+0000","last_undegraded":"2024-01-27T02:39:22.904556+0000","last_fullsized":"2024-01-27T02:39:22.904556+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:59:03.340181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902308+0000","last_change":"2024-01-27T02:39:22.901938+0000","last_active":"2024-01-27T02:39:22.902308+0000","last_peered":"2024-01-27T02:39:22.902308+0000","last_clean":"2024-01-27T02:39:22.902308+0000","last_became_active":"2024-01-27T02:39:22.897160+0000","last_became_peered":"2024-01-27T02:39:22.897160+0000","last_unstale":"2024-01-27T02:39:22.902308+0000","last_undegraded":"2024-01-27T02:39:22.902308+0000","last_fullsized":"2024-01-27T02:39:22.902308+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:04:33.028169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.898697+0000","last_change":"2024-01-27T02:39:22.897541+0000","last_active":"2024-01-27T02:39:22.898697+0000","last_peered":"2024-01-27T02:39:22.898697+0000","last_clean":"2024-01-27T02:39:22.898697+0000","last_became_active":"2024-01-27T02:39:22.897251+0000","last_became_peered":"2024-01-27T02:39:22.897251+0000","last_unstale":"2024-01-27T02:39:22.898697+0000","last_undegraded":"2024-01-27T02:39:22.898697+0000","last_fullsized":"2024-01-27T02:39:22.898697+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:34:23.208150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902274+0000","last_change":"2024-01-27T02:39:22.899543+0000","last_active":"2024-01-27T02:39:22.902274+0000","last_peered":"2024-01-27T02:39:22.902274+0000","last_clean":"2024-01-27T02:39:22.902274+0000","last_became_active":"2024-01-27T02:39:22.899256+0000","last_became_peered":"2024-01-27T02:39:22.899256+0000","last_unstale":"2024-01-27T02:39:22.902274+0000","last_undegraded":"2024-01-27T02:39:22.902274+0000","last_fullsized":"2024-01-27T02:39:22.902274+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:14:55.893637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.898760+0000","last_change":"2024-01-27T02:39:22.898466+0000","last_active":"2024-01-27T02:39:22.898760+0000","last_peered":"2024-01-27T02:39:22.898760+0000","last_clean":"2024-01-27T02:39:22.898760+0000","last_became_active":"2024-01-27T02:39:22.897999+0000","last_became_peered":"2024-01-27T02:39:22.897999+0000","last_unstale":"2024-01-27T02:39:22.898760+0000","last_undegraded":"2024-01-27T02:39:22.898760+0000","last_fullsized":"2024-01-27T02:39:22.898760+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:39:21.733098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.904084+0000","last_change":"2024-01-27T02:39:22.904004+0000","last_active":"2024-01-27T02:39:22.904084+0000","last_peered":"2024-01-27T02:39:22.904084+0000","last_clean":"2024-01-27T02:39:22.904084+0000","last_became_active":"2024-01-27T02:39:22.903705+0000","last_became_peered":"2024-01-27T02:39:22.903705+0000","last_unstale":"2024-01-27T02:39:22.904084+0000","last_undegraded":"2024-01-27T02:39:22.904084+0000","last_fullsized":"2024-01-27T02:39:22.904084+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:13:03.822990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902780+0000","last_change":"2024-01-27T02:39:22.902663+0000","last_active":"2024-01-27T02:39:22.902780+0000","last_peered":"2024-01-27T02:39:22.902780+0000","last_clean":"2024-01-27T02:39:22.902780+0000","last_became_active":"2024-01-27T02:39:22.902445+0000","last_became_peered":"2024-01-27T02:39:22.902445+0000","last_unstale":"2024-01-27T02:39:22.902780+0000","last_undegraded":"2024-01-27T02:39:22.902780+0000","last_fullsized":"2024-01-27T02:39:22.902780+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:09:14.280979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.905378+0000","last_change":"2024-01-27T02:39:22.905287+0000","last_active":"2024-01-27T02:39:22.905378+0000","last_peered":"2024-01-27T02:39:22.905378+0000","last_clean":"2024-01-27T02:39:22.905378+0000","last_became_active":"2024-01-27T02:39:22.905095+0000","last_became_peered":"2024-01-27T02:39:22.905095+0000","last_unstale":"2024-01-27T02:39:22.905378+0000","last_undegraded":"2024-01-27T02:39:22.905378+0000","last_fullsized":"2024-01-27T02:39:22.905378+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:35:11.141652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.908684+0000","last_change":"2024-01-27T02:39:22.907502+0000","last_active":"2024-01-27T02:39:22.908684+0000","last_peered":"2024-01-27T02:39:22.908684+0000","last_clean":"2024-01-27T02:39:22.908684+0000","last_became_active":"2024-01-27T02:39:22.907247+0000","last_became_peered":"2024-01-27T02:39:22.907247+0000","last_unstale":"2024-01-27T02:39:22.908684+0000","last_undegraded":"2024-01-27T02:39:22.908684+0000","last_fullsized":"2024-01-27T02:39:22.908684+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:08:14.937701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.907847+0000","last_change":"2024-01-27T02:39:22.907272+0000","last_active":"2024-01-27T02:39:22.907847+0000","last_peered":"2024-01-27T02:39:22.907847+0000","last_clean":"2024-01-27T02:39:22.907847+0000","last_became_active":"2024-01-27T02:39:22.906623+0000","last_became_peered":"2024-01-27T02:39:22.906623+0000","last_unstale":"2024-01-27T02:39:22.907847+0000","last_undegraded":"2024-01-27T02:39:22.907847+0000","last_fullsized":"2024-01-27T02:39:22.907847+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:39:21.733098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.908642+0000","last_change":"2024-01-27T02:39:22.908551+0000","last_active":"2024-01-27T02:39:22.908642+0000","last_peered":"2024-01-27T02:39:22.908642+0000","last_clean":"2024-01-27T02:39:22.908642+0000","last_became_active":"2024-01-27T02:39:22.907980+0000","last_became_peered":"2024-01-27T02:39:22.907980+0000","last_unstale":"2024-01-27T02:39:22.908642+0000","last_undegraded":"2024-01-27T02:39:22.908642+0000","last_fullsized":"2024-01-27T02:39:22.908642+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:50.975864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.903922+0000","last_change":"2024-01-27T02:39:22.903237+0000","last_active":"2024-01-27T02:39:22.903922+0000","last_peered":"2024-01-27T02:39:22.903922+0000","last_clean":"2024-01-27T02:39:22.903922+0000","last_became_active":"2024-01-27T02:39:22.902973+0000","last_became_peered":"2024-01-27T02:39:22.902973+0000","last_unstale":"2024-01-27T02:39:22.903922+0000","last_undegraded":"2024-01-27T02:39:22.903922+0000","last_fullsized":"2024-01-27T02:39:22.903922+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:35:38.537841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.900393+0000","last_change":"2024-01-27T02:39:22.900300+0000","last_active":"2024-01-27T02:39:22.900393+0000","last_peered":"2024-01-27T02:39:22.900393+0000","last_clean":"2024-01-27T02:39:22.900393+0000","last_became_active":"2024-01-27T02:39:22.899932+0000","last_became_peered":"2024-01-27T02:39:22.899932+0000","last_unstale":"2024-01-27T02:39:22.900393+0000","last_undegraded":"2024-01-27T02:39:22.900393+0000","last_fullsized":"2024-01-27T02:39:22.900393+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:51:22.997295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.907415+0000","last_change":"2024-01-27T02:39:22.906323+0000","last_active":"2024-01-27T02:39:22.907415+0000","last_peered":"2024-01-27T02:39:22.907415+0000","last_clean":"2024-01-27T02:39:22.907415+0000","last_became_active":"2024-01-27T02:39:22.905609+0000","last_became_peered":"2024-01-27T02:39:22.905609+0000","last_unstale":"2024-01-27T02:39:22.907415+0000","last_undegraded":"2024-01-27T02:39:22.907415+0000","last_fullsized":"2024-01-27T02:39:22.907415+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:52:33.405152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902602+0000","last_change":"2024-01-27T02:39:22.902281+0000","last_active":"2024-01-27T02:39:22.902602+0000","last_peered":"2024-01-27T02:39:22.902602+0000","last_clean":"2024-01-27T02:39:22.902602+0000","last_became_active":"2024-01-27T02:39:22.901653+0000","last_became_peered":"2024-01-27T02:39:22.901653+0000","last_unstale":"2024-01-27T02:39:22.902602+0000","last_undegraded":"2024-01-27T02:39:22.902602+0000","last_fullsized":"2024-01-27T02:39:22.902602+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:57:54.178723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.897739+0000","last_change":"2024-01-27T02:39:22.897658+0000","last_active":"2024-01-27T02:39:22.897739+0000","last_peered":"2024-01-27T02:39:22.897739+0000","last_clean":"2024-01-27T02:39:22.897739+0000","last_became_active":"2024-01-27T02:39:22.897115+0000","last_became_peered":"2024-01-27T02:39:22.897115+0000","last_unstale":"2024-01-27T02:39:22.897739+0000","last_undegraded":"2024-01-27T02:39:22.897739+0000","last_fullsized":"2024-01-27T02:39:22.897739+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:11:12.572257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.904065+0000","last_change":"2024-01-27T02:39:22.903978+0000","last_active":"2024-01-27T02:39:22.904065+0000","last_peered":"2024-01-27T02:39:22.904065+0000","last_clean":"2024-01-27T02:39:22.904065+0000","last_became_active":"2024-01-27T02:39:22.903705+0000","last_became_peered":"2024-01-27T02:39:22.903705+0000","last_unstale":"2024-01-27T02:39:22.904065+0000","last_undegraded":"2024-01-27T02:39:22.904065+0000","last_fullsized":"2024-01-27T02:39:22.904065+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:36:32.750631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"148'4","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.913408+0000","last_change":"2024-01-27T02:39:21.897428+0000","last_active":"2024-01-27T02:39:22.913408+0000","last_peered":"2024-01-27T02:39:22.913408+0000","last_clean":"2024-01-27T02:39:22.913408+0000","last_became_active":"2024-01-27T02:39:21.897179+0000","last_became_peered":"2024-01-27T02:39:21.897179+0000","last_unstale":"2024-01-27T02:39:22.913408+0000","last_undegraded":"2024-01-27T02:39:22.913408+0000","last_fullsized":"2024-01-27T02:39:22.913408+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:47.650928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.911163+0000","last_change":"2024-01-27T02:39:22.907668+0000","last_active":"2024-01-27T02:39:22.911163+0000","last_peered":"2024-01-27T02:39:22.911163+0000","last_clean":"2024-01-27T02:39:22.911163+0000","last_became_active":"2024-01-27T02:39:22.907423+0000","last_became_peered":"2024-01-27T02:39:22.907423+0000","last_unstale":"2024-01-27T02:39:22.911163+0000","last_undegraded":"2024-01-27T02:39:22.911163+0000","last_fullsized":"2024-01-27T02:39:22.911163+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:48.653148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"148'1","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.909191+0000","last_change":"2024-01-27T02:39:21.895025+0000","last_active":"2024-01-27T02:39:22.909191+0000","last_peered":"2024-01-27T02:39:22.909191+0000","last_clean":"2024-01-27T02:39:22.909191+0000","last_became_active":"2024-01-27T02:39:21.894753+0000","last_became_peered":"2024-01-27T02:39:21.894753+0000","last_unstale":"2024-01-27T02:39:22.909191+0000","last_undegraded":"2024-01-27T02:39:22.909191+0000","last_fullsized":"2024-01-27T02:39:22.909191+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:23:12.301215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.898741+0000","last_change":"2024-01-27T02:39:22.898310+0000","last_active":"2024-01-27T02:39:22.898741+0000","last_peered":"2024-01-27T02:39:22.898741+0000","last_clean":"2024-01-27T02:39:22.898741+0000","last_became_active":"2024-01-27T02:39:22.897673+0000","last_became_peered":"2024-01-27T02:39:22.897673+0000","last_unstale":"2024-01-27T02:39:22.898741+0000","last_undegraded":"2024-01-27T02:39:22.898741+0000","last_fullsized":"2024-01-27T02:39:22.898741+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:15:15.541057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910286+0000","last_change":"2024-01-27T02:39:21.894906+0000","last_active":"2024-01-27T02:39:22.910286+0000","last_peered":"2024-01-27T02:39:22.910286+0000","last_clean":"2024-01-27T02:39:22.910286+0000","last_became_active":"2024-01-27T02:39:21.894564+0000","last_became_peered":"2024-01-27T02:39:21.894564+0000","last_unstale":"2024-01-27T02:39:22.910286+0000","last_undegraded":"2024-01-27T02:39:22.910286+0000","last_fullsized":"2024-01-27T02:39:22.910286+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:36:31.748411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910835+0000","last_change":"2024-01-27T02:39:22.909673+0000","last_active":"2024-01-27T02:39:22.910835+0000","last_peered":"2024-01-27T02:39:22.910835+0000","last_clean":"2024-01-27T02:39:22.910835+0000","last_became_active":"2024-01-27T02:39:22.908815+0000","last_became_peered":"2024-01-27T02:39:22.908815+0000","last_unstale":"2024-01-27T02:39:22.910835+0000","last_undegraded":"2024-01-27T02:39:22.910835+0000","last_fullsized":"2024-01-27T02:39:22.910835+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:02:57.570709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.912880+0000","last_change":"2024-01-27T02:39:21.900974+0000","last_active":"2024-01-27T02:39:22.912880+0000","last_peered":"2024-01-27T02:39:22.912880+0000","last_clean":"2024-01-27T02:39:22.912880+0000","last_became_active":"2024-01-27T02:39:21.900576+0000","last_became_peered":"2024-01-27T02:39:21.900576+0000","last_unstale":"2024-01-27T02:39:22.912880+0000","last_undegraded":"2024-01-27T02:39:22.912880+0000","last_fullsized":"2024-01-27T02:39:22.912880+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:10:31.872654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902332+0000","last_change":"2024-01-27T02:39:22.902089+0000","last_active":"2024-01-27T02:39:22.902332+0000","last_peered":"2024-01-27T02:39:22.902332+0000","last_clean":"2024-01-27T02:39:22.902332+0000","last_became_active":"2024-01-27T02:39:22.897524+0000","last_became_peered":"2024-01-27T02:39:22.897524+0000","last_unstale":"2024-01-27T02:39:22.902332+0000","last_undegraded":"2024-01-27T02:39:22.902332+0000","last_fullsized":"2024-01-27T02:39:22.902332+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:06:41.763215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"148'1","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.912090+0000","last_change":"2024-01-27T02:39:21.898367+0000","last_active":"2024-01-27T02:39:22.912090+0000","last_peered":"2024-01-27T02:39:22.912090+0000","last_clean":"2024-01-27T02:39:22.912090+0000","last_became_active":"2024-01-27T02:39:21.898114+0000","last_became_peered":"2024-01-27T02:39:21.898114+0000","last_unstale":"2024-01-27T02:39:22.912090+0000","last_undegraded":"2024-01-27T02:39:22.912090+0000","last_fullsized":"2024-01-27T02:39:22.912090+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:35:46.505332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.904385+0000","last_change":"2024-01-27T02:39:22.904273+0000","last_active":"2024-01-27T02:39:22.904385+0000","last_peered":"2024-01-27T02:39:22.904385+0000","last_clean":"2024-01-27T02:39:22.904385+0000","last_became_active":"2024-01-27T02:39:22.903665+0000","last_became_peered":"2024-01-27T02:39:22.903665+0000","last_unstale":"2024-01-27T02:39:22.904385+0000","last_undegraded":"2024-01-27T02:39:22.904385+0000","last_fullsized":"2024-01-27T02:39:22.904385+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:40:33.462225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"148'1","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.909959+0000","last_change":"2024-01-27T02:39:21.899252+0000","last_active":"2024-01-27T02:39:22.909959+0000","last_peered":"2024-01-27T02:39:22.909959+0000","last_clean":"2024-01-27T02:39:22.909959+0000","last_became_active":"2024-01-27T02:39:21.898982+0000","last_became_peered":"2024-01-27T02:39:21.898982+0000","last_unstale":"2024-01-27T02:39:22.909959+0000","last_undegraded":"2024-01-27T02:39:22.909959+0000","last_fullsized":"2024-01-27T02:39:22.909959+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:11:11.570037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910749+0000","last_change":"2024-01-27T02:39:22.906183+0000","last_active":"2024-01-27T02:39:22.910749+0000","last_peered":"2024-01-27T02:39:22.910749+0000","last_clean":"2024-01-27T02:39:22.910749+0000","last_became_active":"2024-01-27T02:39:22.904977+0000","last_became_peered":"2024-01-27T02:39:22.904977+0000","last_unstale":"2024-01-27T02:39:22.910749+0000","last_undegraded":"2024-01-27T02:39:22.910749+0000","last_fullsized":"2024-01-27T02:39:22.910749+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:18:02.494561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.891115+0000","last_change":"2024-01-27T02:39:21.898838+0000","last_active":"2024-01-27T02:39:22.891115+0000","last_peered":"2024-01-27T02:39:22.891115+0000","last_clean":"2024-01-27T02:39:22.891115+0000","last_became_active":"2024-01-27T02:39:21.898603+0000","last_became_peered":"2024-01-27T02:39:21.898603+0000","last_unstale":"2024-01-27T02:39:22.891115+0000","last_undegraded":"2024-01-27T02:39:22.891115+0000","last_fullsized":"2024-01-27T02:39:22.891115+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:09:29.890085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910984+0000","last_change":"2024-01-27T02:39:22.910891+0000","last_active":"2024-01-27T02:39:22.910984+0000","last_peered":"2024-01-27T02:39:22.910984+0000","last_clean":"2024-01-27T02:39:22.910984+0000","last_became_active":"2024-01-27T02:39:22.910593+0000","last_became_peered":"2024-01-27T02:39:22.910593+0000","last_unstale":"2024-01-27T02:39:22.910984+0000","last_undegraded":"2024-01-27T02:39:22.910984+0000","last_fullsized":"2024-01-27T02:39:22.910984+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:03:28.071762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.897217+0000","last_change":"2024-01-27T02:39:21.903093+0000","last_active":"2024-01-27T02:39:22.897217+0000","last_peered":"2024-01-27T02:39:22.897217+0000","last_clean":"2024-01-27T02:39:22.897217+0000","last_became_active":"2024-01-27T02:39:21.902782+0000","last_became_peered":"2024-01-27T02:39:21.902782+0000","last_unstale":"2024-01-27T02:39:22.897217+0000","last_undegraded":"2024-01-27T02:39:22.897217+0000","last_fullsized":"2024-01-27T02:39:22.897217+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:26:01.500470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.908830+0000","last_change":"2024-01-27T02:39:22.908736+0000","last_active":"2024-01-27T02:39:22.908830+0000","last_peered":"2024-01-27T02:39:22.908830+0000","last_clean":"2024-01-27T02:39:22.908830+0000","last_became_active":"2024-01-27T02:39:22.908321+0000","last_became_peered":"2024-01-27T02:39:22.908321+0000","last_unstale":"2024-01-27T02:39:22.908830+0000","last_undegraded":"2024-01-27T02:39:22.908830+0000","last_fullsized":"2024-01-27T02:39:22.908830+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:23:05.214270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.894643+0000","last_change":"2024-01-27T02:39:21.898565+0000","last_active":"2024-01-27T02:39:22.894643+0000","last_peered":"2024-01-27T02:39:22.894643+0000","last_clean":"2024-01-27T02:39:22.894643+0000","last_became_active":"2024-01-27T02:39:21.898337+0000","last_became_peered":"2024-01-27T02:39:21.898337+0000","last_unstale":"2024-01-27T02:39:22.894643+0000","last_undegraded":"2024-01-27T02:39:22.894643+0000","last_fullsized":"2024-01-27T02:39:22.894643+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:10:31.872654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.903942+0000","last_change":"2024-01-27T02:39:22.903289+0000","last_active":"2024-01-27T02:39:22.903942+0000","last_peered":"2024-01-27T02:39:22.903942+0000","last_clean":"2024-01-27T02:39:22.903942+0000","last_became_active":"2024-01-27T02:39:22.903021+0000","last_became_peered":"2024-01-27T02:39:22.903021+0000","last_unstale":"2024-01-27T02:39:22.903942+0000","last_undegraded":"2024-01-27T02:39:22.903942+0000","last_fullsized":"2024-01-27T02:39:22.903942+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:03:29.364153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.889969+0000","last_change":"2024-01-27T02:39:21.901433+0000","last_active":"2024-01-27T02:39:22.889969+0000","last_peered":"2024-01-27T02:39:22.889969+0000","last_clean":"2024-01-27T02:39:22.889969+0000","last_became_active":"2024-01-27T02:39:21.901145+0000","last_became_peered":"2024-01-27T02:39:21.901145+0000","last_unstale":"2024-01-27T02:39:22.889969+0000","last_undegraded":"2024-01-27T02:39:22.889969+0000","last_fullsized":"2024-01-27T02:39:22.889969+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:27:20.146303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.905809+0000","last_change":"2024-01-27T02:39:22.905679+0000","last_active":"2024-01-27T02:39:22.905809+0000","last_peered":"2024-01-27T02:39:22.905809+0000","last_clean":"2024-01-27T02:39:22.905809+0000","last_became_active":"2024-01-27T02:39:22.905262+0000","last_became_peered":"2024-01-27T02:39:22.905262+0000","last_unstale":"2024-01-27T02:39:22.905809+0000","last_undegraded":"2024-01-27T02:39:22.905809+0000","last_fullsized":"2024-01-27T02:39:22.905809+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:53:50.414456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"148'3","reported_seq":13,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910510+0000","last_change":"2024-01-27T02:39:21.894472+0000","last_active":"2024-01-27T02:39:22.910510+0000","last_peered":"2024-01-27T02:39:22.910510+0000","last_clean":"2024-01-27T02:39:22.910510+0000","last_became_active":"2024-01-27T02:39:21.893990+0000","last_became_peered":"2024-01-27T02:39:21.893990+0000","last_unstale":"2024-01-27T02:39:22.910510+0000","last_undegraded":"2024-01-27T02:39:22.910510+0000","last_fullsized":"2024-01-27T02:39:22.910510+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:01:18.616886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.897842+0000","last_change":"2024-01-27T02:39:22.897409+0000","last_active":"2024-01-27T02:39:22.897842+0000","last_peered":"2024-01-27T02:39:22.897842+0000","last_clean":"2024-01-27T02:39:22.897842+0000","last_became_active":"2024-01-27T02:39:22.897150+0000","last_became_peered":"2024-01-27T02:39:22.897150+0000","last_unstale":"2024-01-27T02:39:22.897842+0000","last_undegraded":"2024-01-27T02:39:22.897842+0000","last_fullsized":"2024-01-27T02:39:22.897842+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:35:39.970169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"148'6","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.914572+0000","last_change":"2024-01-27T02:39:21.901023+0000","last_active":"2024-01-27T02:39:22.914572+0000","last_peered":"2024-01-27T02:39:22.914572+0000","last_clean":"2024-01-27T02:39:22.914572+0000","last_became_active":"2024-01-27T02:39:21.899485+0000","last_became_peered":"2024-01-27T02:39:21.899485+0000","last_unstale":"2024-01-27T02:39:22.914572+0000","last_undegraded":"2024-01-27T02:39:22.914572+0000","last_fullsized":"2024-01-27T02:39:22.914572+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:03:28.361932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.907434+0000","last_change":"2024-01-27T02:39:22.906514+0000","last_active":"2024-01-27T02:39:22.907434+0000","last_peered":"2024-01-27T02:39:22.907434+0000","last_clean":"2024-01-27T02:39:22.907434+0000","last_became_active":"2024-01-27T02:39:22.905927+0000","last_became_peered":"2024-01-27T02:39:22.905927+0000","last_unstale":"2024-01-27T02:39:22.907434+0000","last_undegraded":"2024-01-27T02:39:22.907434+0000","last_fullsized":"2024-01-27T02:39:22.907434+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:30:29.146275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.892289+0000","last_change":"2024-01-27T02:39:21.901950+0000","last_active":"2024-01-27T02:39:22.892289+0000","last_peered":"2024-01-27T02:39:22.892289+0000","last_clean":"2024-01-27T02:39:22.892289+0000","last_became_active":"2024-01-27T02:39:21.901709+0000","last_became_peered":"2024-01-27T02:39:21.901709+0000","last_unstale":"2024-01-27T02:39:22.892289+0000","last_undegraded":"2024-01-27T02:39:22.892289+0000","last_fullsized":"2024-01-27T02:39:22.892289+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:27:55.432732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.900762+0000","last_change":"2024-01-27T02:39:22.900687+0000","last_active":"2024-01-27T02:39:22.900762+0000","last_peered":"2024-01-27T02:39:22.900762+0000","last_clean":"2024-01-27T02:39:22.900762+0000","last_became_active":"2024-01-27T02:39:22.900372+0000","last_became_peered":"2024-01-27T02:39:22.900372+0000","last_unstale":"2024-01-27T02:39:22.900762+0000","last_undegraded":"2024-01-27T02:39:22.900762+0000","last_fullsized":"2024-01-27T02:39:22.900762+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:19:27.934421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.894074+0000","last_change":"2024-01-27T02:39:21.901620+0000","last_active":"2024-01-27T02:39:22.894074+0000","last_peered":"2024-01-27T02:39:22.894074+0000","last_clean":"2024-01-27T02:39:22.894074+0000","last_became_active":"2024-01-27T02:39:21.901102+0000","last_became_peered":"2024-01-27T02:39:21.901102+0000","last_unstale":"2024-01-27T02:39:22.894074+0000","last_undegraded":"2024-01-27T02:39:22.894074+0000","last_fullsized":"2024-01-27T02:39:22.894074+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:52:32.402932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910954+0000","last_change":"2024-01-27T02:39:22.906855+0000","last_active":"2024-01-27T02:39:22.910954+0000","last_peered":"2024-01-27T02:39:22.910954+0000","last_clean":"2024-01-27T02:39:22.910954+0000","last_became_active":"2024-01-27T02:39:22.906480+0000","last_became_peered":"2024-01-27T02:39:22.906480+0000","last_unstale":"2024-01-27T02:39:22.910954+0000","last_undegraded":"2024-01-27T02:39:22.910954+0000","last_fullsized":"2024-01-27T02:39:22.910954+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:40:33.462225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"148'8","reported_seq":18,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.911228+0000","last_change":"2024-01-27T02:39:21.900650+0000","last_active":"2024-01-27T02:39:22.911228+0000","last_peered":"2024-01-27T02:39:22.911228+0000","last_clean":"2024-01-27T02:39:22.911228+0000","last_became_active":"2024-01-27T02:39:21.899844+0000","last_became_peered":"2024-01-27T02:39:21.899844+0000","last_unstale":"2024-01-27T02:39:22.911228+0000","last_undegraded":"2024-01-27T02:39:22.911228+0000","last_fullsized":"2024-01-27T02:39:22.911228+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:03:27.069542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.911277+0000","last_change":"2024-01-27T02:39:22.908170+0000","last_active":"2024-01-27T02:39:22.911277+0000","last_peered":"2024-01-27T02:39:22.911277+0000","last_clean":"2024-01-27T02:39:22.911277+0000","last_became_active":"2024-01-27T02:39:22.907713+0000","last_became_peered":"2024-01-27T02:39:22.907713+0000","last_unstale":"2024-01-27T02:39:22.911277+0000","last_undegraded":"2024-01-27T02:39:22.911277+0000","last_fullsized":"2024-01-27T02:39:22.911277+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:59:03.340181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"148'1","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.907916+0000","last_change":"2024-01-27T02:39:21.898336+0000","last_active":"2024-01-27T02:39:22.907916+0000","last_peered":"2024-01-27T02:39:22.907916+0000","last_clean":"2024-01-27T02:39:22.907916+0000","last_became_active":"2024-01-27T02:39:21.898073+0000","last_became_peered":"2024-01-27T02:39:21.898073+0000","last_unstale":"2024-01-27T02:39:22.907916+0000","last_undegraded":"2024-01-27T02:39:22.907916+0000","last_fullsized":"2024-01-27T02:39:22.907916+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:59:06.427886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.907601+0000","last_change":"2024-01-27T02:39:22.907034+0000","last_active":"2024-01-27T02:39:22.907601+0000","last_peered":"2024-01-27T02:39:22.907601+0000","last_clean":"2024-01-27T02:39:22.907601+0000","last_became_active":"2024-01-27T02:39:22.906810+0000","last_became_peered":"2024-01-27T02:39:22.906810+0000","last_unstale":"2024-01-27T02:39:22.907601+0000","last_undegraded":"2024-01-27T02:39:22.907601+0000","last_fullsized":"2024-01-27T02:39:22.907601+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:03:28.071762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.911846+0000","last_change":"2024-01-27T02:39:21.902022+0000","last_active":"2024-01-27T02:39:22.911846+0000","last_peered":"2024-01-27T02:39:22.911846+0000","last_clean":"2024-01-27T02:39:22.911846+0000","last_became_active":"2024-01-27T02:39:21.901762+0000","last_became_peered":"2024-01-27T02:39:21.901762+0000","last_unstale":"2024-01-27T02:39:22.911846+0000","last_undegraded":"2024-01-27T02:39:22.911846+0000","last_fullsized":"2024-01-27T02:39:22.911846+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:35:38.967949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.903467+0000","last_change":"2024-01-27T02:39:22.902929+0000","last_active":"2024-01-27T02:39:22.903467+0000","last_peered":"2024-01-27T02:39:22.903467+0000","last_clean":"2024-01-27T02:39:22.903467+0000","last_became_active":"2024-01-27T02:39:22.902654+0000","last_became_peered":"2024-01-27T02:39:22.902654+0000","last_unstale":"2024-01-27T02:39:22.903467+0000","last_undegraded":"2024-01-27T02:39:22.903467+0000","last_fullsized":"2024-01-27T02:39:22.903467+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:02:57.570709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"148'1","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910217+0000","last_change":"2024-01-27T02:39:21.895442+0000","last_active":"2024-01-27T02:39:22.910217+0000","last_peered":"2024-01-27T02:39:22.910217+0000","last_clean":"2024-01-27T02:39:22.910217+0000","last_became_active":"2024-01-27T02:39:21.894130+0000","last_became_peered":"2024-01-27T02:39:21.894130+0000","last_unstale":"2024-01-27T02:39:22.910217+0000","last_undegraded":"2024-01-27T02:39:22.910217+0000","last_fullsized":"2024-01-27T02:39:22.910217+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:02:56.715132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.908720+0000","last_change":"2024-01-27T02:39:22.908370+0000","last_active":"2024-01-27T02:39:22.908720+0000","last_peered":"2024-01-27T02:39:22.908720+0000","last_clean":"2024-01-27T02:39:22.908720+0000","last_became_active":"2024-01-27T02:39:22.907779+0000","last_became_peered":"2024-01-27T02:39:22.907779+0000","last_unstale":"2024-01-27T02:39:22.908720+0000","last_undegraded":"2024-01-27T02:39:22.908720+0000","last_fullsized":"2024-01-27T02:39:22.908720+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:17:07.671167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.894420+0000","last_change":"2024-01-27T02:39:21.895019+0000","last_active":"2024-01-27T02:39:22.894420+0000","last_peered":"2024-01-27T02:39:22.894420+0000","last_clean":"2024-01-27T02:39:22.894420+0000","last_became_active":"2024-01-27T02:39:21.894813+0000","last_became_peered":"2024-01-27T02:39:21.894813+0000","last_unstale":"2024-01-27T02:39:22.894420+0000","last_undegraded":"2024-01-27T02:39:22.894420+0000","last_fullsized":"2024-01-27T02:39:22.894420+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:01:58.908495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.900222+0000","last_change":"2024-01-27T02:39:22.900150+0000","last_active":"2024-01-27T02:39:22.900222+0000","last_peered":"2024-01-27T02:39:22.900222+0000","last_clean":"2024-01-27T02:39:22.900222+0000","last_became_active":"2024-01-27T02:39:22.899453+0000","last_became_peered":"2024-01-27T02:39:22.899453+0000","last_unstale":"2024-01-27T02:39:22.900222+0000","last_undegraded":"2024-01-27T02:39:22.900222+0000","last_fullsized":"2024-01-27T02:39:22.900222+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:08:54.067660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910284+0000","last_change":"2024-01-27T02:39:21.896656+0000","last_active":"2024-01-27T02:39:22.910284+0000","last_peered":"2024-01-27T02:39:22.910284+0000","last_clean":"2024-01-27T02:39:22.910284+0000","last_became_active":"2024-01-27T02:39:21.896422+0000","last_became_peered":"2024-01-27T02:39:21.896422+0000","last_unstale":"2024-01-27T02:39:22.910284+0000","last_undegraded":"2024-01-27T02:39:22.910284+0000","last_fullsized":"2024-01-27T02:39:22.910284+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:10:40.058748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.900541+0000","last_change":"2024-01-27T02:39:22.900467+0000","last_active":"2024-01-27T02:39:22.900541+0000","last_peered":"2024-01-27T02:39:22.900541+0000","last_clean":"2024-01-27T02:39:22.900541+0000","last_became_active":"2024-01-27T02:39:22.899866+0000","last_became_peered":"2024-01-27T02:39:22.899866+0000","last_unstale":"2024-01-27T02:39:22.900541+0000","last_undegraded":"2024-01-27T02:39:22.900541+0000","last_fullsized":"2024-01-27T02:39:22.900541+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:27:21.148523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.893201+0000","last_change":"2024-01-27T02:39:21.900412+0000","last_active":"2024-01-27T02:39:22.893201+0000","last_peered":"2024-01-27T02:39:22.893201+0000","last_clean":"2024-01-27T02:39:22.893201+0000","last_became_active":"2024-01-27T02:39:21.900133+0000","last_became_peered":"2024-01-27T02:39:21.900133+0000","last_unstale":"2024-01-27T02:39:22.893201+0000","last_undegraded":"2024-01-27T02:39:22.893201+0000","last_fullsized":"2024-01-27T02:39:22.893201+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:40:32.460005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.908737+0000","last_change":"2024-01-27T02:39:22.908527+0000","last_active":"2024-01-27T02:39:22.908737+0000","last_peered":"2024-01-27T02:39:22.908737+0000","last_clean":"2024-01-27T02:39:22.908737+0000","last_became_active":"2024-01-27T02:39:22.908130+0000","last_became_peered":"2024-01-27T02:39:22.908130+0000","last_unstale":"2024-01-27T02:39:22.908737+0000","last_undegraded":"2024-01-27T02:39:22.908737+0000","last_fullsized":"2024-01-27T02:39:22.908737+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:25:03.541488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.890111+0000","last_change":"2024-01-27T02:39:21.889901+0000","last_active":"2024-01-27T02:39:22.890111+0000","last_peered":"2024-01-27T02:39:22.890111+0000","last_clean":"2024-01-27T02:39:22.890111+0000","last_became_active":"2024-01-27T02:39:21.889524+0000","last_became_peered":"2024-01-27T02:39:21.889524+0000","last_unstale":"2024-01-27T02:39:22.890111+0000","last_undegraded":"2024-01-27T02:39:22.890111+0000","last_fullsized":"2024-01-27T02:39:22.890111+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:16.787597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910855+0000","last_change":"2024-01-27T02:39:22.909847+0000","last_active":"2024-01-27T02:39:22.910855+0000","last_peered":"2024-01-27T02:39:22.910855+0000","last_clean":"2024-01-27T02:39:22.910855+0000","last_became_active":"2024-01-27T02:39:22.909357+0000","last_became_peered":"2024-01-27T02:39:22.909357+0000","last_unstale":"2024-01-27T02:39:22.910855+0000","last_undegraded":"2024-01-27T02:39:22.910855+0000","last_fullsized":"2024-01-27T02:39:22.910855+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:12.863356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.894295+0000","last_change":"2024-01-27T02:39:21.901504+0000","last_active":"2024-01-27T02:39:22.894295+0000","last_peered":"2024-01-27T02:39:22.894295+0000","last_clean":"2024-01-27T02:39:22.894295+0000","last_became_active":"2024-01-27T02:39:21.901245+0000","last_became_peered":"2024-01-27T02:39:21.901245+0000","last_unstale":"2024-01-27T02:39:22.894295+0000","last_undegraded":"2024-01-27T02:39:22.894295+0000","last_fullsized":"2024-01-27T02:39:22.894295+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:57:53.176503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.900280+0000","last_change":"2024-01-27T02:39:22.900231+0000","last_active":"2024-01-27T02:39:22.900280+0000","last_peered":"2024-01-27T02:39:22.900280+0000","last_clean":"2024-01-27T02:39:22.900280+0000","last_became_active":"2024-01-27T02:39:22.899499+0000","last_became_peered":"2024-01-27T02:39:22.899499+0000","last_unstale":"2024-01-27T02:39:22.900280+0000","last_undegraded":"2024-01-27T02:39:22.900280+0000","last_fullsized":"2024-01-27T02:39:22.900280+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:04:43.117550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.912471+0000","last_change":"2024-01-27T02:39:21.896736+0000","last_active":"2024-01-27T02:39:22.912471+0000","last_peered":"2024-01-27T02:39:22.912471+0000","last_clean":"2024-01-27T02:39:22.912471+0000","last_became_active":"2024-01-27T02:39:21.896197+0000","last_became_peered":"2024-01-27T02:39:21.896197+0000","last_unstale":"2024-01-27T02:39:22.912471+0000","last_undegraded":"2024-01-27T02:39:22.912471+0000","last_fullsized":"2024-01-27T02:39:22.912471+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:00:34.270221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.898823+0000","last_change":"2024-01-27T02:39:22.898657+0000","last_active":"2024-01-27T02:39:22.898823+0000","last_peered":"2024-01-27T02:39:22.898823+0000","last_clean":"2024-01-27T02:39:22.898823+0000","last_became_active":"2024-01-27T02:39:22.898442+0000","last_became_peered":"2024-01-27T02:39:22.898442+0000","last_unstale":"2024-01-27T02:39:22.898823+0000","last_undegraded":"2024-01-27T02:39:22.898823+0000","last_fullsized":"2024-01-27T02:39:22.898823+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:55:52.096811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.895235+0000","last_change":"2024-01-27T02:39:21.898780+0000","last_active":"2024-01-27T02:39:22.895235+0000","last_peered":"2024-01-27T02:39:22.895235+0000","last_clean":"2024-01-27T02:39:22.895235+0000","last_became_active":"2024-01-27T02:39:21.898579+0000","last_became_peered":"2024-01-27T02:39:21.898579+0000","last_unstale":"2024-01-27T02:39:22.895235+0000","last_undegraded":"2024-01-27T02:39:22.895235+0000","last_fullsized":"2024-01-27T02:39:22.895235+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:59:36.125517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.906436+0000","last_change":"2024-01-27T02:39:22.906355+0000","last_active":"2024-01-27T02:39:22.906436+0000","last_peered":"2024-01-27T02:39:22.906436+0000","last_clean":"2024-01-27T02:39:22.906436+0000","last_became_active":"2024-01-27T02:39:22.902011+0000","last_became_peered":"2024-01-27T02:39:22.902011+0000","last_unstale":"2024-01-27T02:39:22.906436+0000","last_undegraded":"2024-01-27T02:39:22.906436+0000","last_fullsized":"2024-01-27T02:39:22.906436+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:09:30.892305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.894225+0000","last_change":"2024-01-27T02:39:21.900466+0000","last_active":"2024-01-27T02:39:22.894225+0000","last_peered":"2024-01-27T02:39:22.894225+0000","last_clean":"2024-01-27T02:39:22.894225+0000","last_became_active":"2024-01-27T02:39:21.900211+0000","last_became_peered":"2024-01-27T02:39:21.900211+0000","last_unstale":"2024-01-27T02:39:22.894225+0000","last_undegraded":"2024-01-27T02:39:22.894225+0000","last_fullsized":"2024-01-27T02:39:22.894225+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:26:41.909600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.901978+0000","last_change":"2024-01-27T02:39:22.901912+0000","last_active":"2024-01-27T02:39:22.901978+0000","last_peered":"2024-01-27T02:39:22.901978+0000","last_clean":"2024-01-27T02:39:22.901978+0000","last_became_active":"2024-01-27T02:39:22.901571+0000","last_became_peered":"2024-01-27T02:39:22.901571+0000","last_unstale":"2024-01-27T02:39:22.901978+0000","last_undegraded":"2024-01-27T02:39:22.901978+0000","last_fullsized":"2024-01-27T02:39:22.901978+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:17.789817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.897627+0000","last_change":"2024-01-27T02:39:21.903774+0000","last_active":"2024-01-27T02:39:22.897627+0000","last_peered":"2024-01-27T02:39:22.897627+0000","last_clean":"2024-01-27T02:39:22.897627+0000","last_became_active":"2024-01-27T02:39:21.903535+0000","last_became_peered":"2024-01-27T02:39:21.903535+0000","last_unstale":"2024-01-27T02:39:22.897627+0000","last_undegraded":"2024-01-27T02:39:22.897627+0000","last_fullsized":"2024-01-27T02:39:22.897627+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:02:56.568488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.898120+0000","last_change":"2024-01-27T02:39:22.898044+0000","last_active":"2024-01-27T02:39:22.898120+0000","last_peered":"2024-01-27T02:39:22.898120+0000","last_clean":"2024-01-27T02:39:22.898120+0000","last_became_active":"2024-01-27T02:39:22.897789+0000","last_became_peered":"2024-01-27T02:39:22.897789+0000","last_unstale":"2024-01-27T02:39:22.898120+0000","last_undegraded":"2024-01-27T02:39:22.898120+0000","last_fullsized":"2024-01-27T02:39:22.898120+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:36:32.750631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.895852+0000","last_change":"2024-01-27T02:39:21.900816+0000","last_active":"2024-01-27T02:39:22.895852+0000","last_peered":"2024-01-27T02:39:22.895852+0000","last_clean":"2024-01-27T02:39:22.895852+0000","last_became_active":"2024-01-27T02:39:21.900263+0000","last_became_peered":"2024-01-27T02:39:21.900263+0000","last_unstale":"2024-01-27T02:39:22.895852+0000","last_undegraded":"2024-01-27T02:39:22.895852+0000","last_fullsized":"2024-01-27T02:39:22.895852+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:49.973643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.901979+0000","last_change":"2024-01-27T02:39:22.901592+0000","last_active":"2024-01-27T02:39:22.901979+0000","last_peered":"2024-01-27T02:39:22.901979+0000","last_clean":"2024-01-27T02:39:22.901979+0000","last_became_active":"2024-01-27T02:39:22.901146+0000","last_became_peered":"2024-01-27T02:39:22.901146+0000","last_unstale":"2024-01-27T02:39:22.901979+0000","last_undegraded":"2024-01-27T02:39:22.901979+0000","last_fullsized":"2024-01-27T02:39:22.901979+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:25:47.557777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.894599+0000","last_change":"2024-01-27T02:39:21.899411+0000","last_active":"2024-01-27T02:39:22.894599+0000","last_peered":"2024-01-27T02:39:22.894599+0000","last_clean":"2024-01-27T02:39:22.894599+0000","last_became_active":"2024-01-27T02:39:21.899153+0000","last_became_peered":"2024-01-27T02:39:21.899153+0000","last_unstale":"2024-01-27T02:39:22.894599+0000","last_undegraded":"2024-01-27T02:39:22.894599+0000","last_fullsized":"2024-01-27T02:39:22.894599+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:18:01.492341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902253+0000","last_change":"2024-01-27T02:39:22.899280+0000","last_active":"2024-01-27T02:39:22.902253+0000","last_peered":"2024-01-27T02:39:22.902253+0000","last_clean":"2024-01-27T02:39:22.902253+0000","last_became_active":"2024-01-27T02:39:22.899002+0000","last_became_peered":"2024-01-27T02:39:22.899002+0000","last_unstale":"2024-01-27T02:39:22.902253+0000","last_undegraded":"2024-01-27T02:39:22.902253+0000","last_fullsized":"2024-01-27T02:39:22.902253+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:43:03.625409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.890445+0000","last_change":"2024-01-27T02:39:21.901048+0000","last_active":"2024-01-27T02:39:22.890445+0000","last_peered":"2024-01-27T02:39:22.890445+0000","last_clean":"2024-01-27T02:39:22.890445+0000","last_became_active":"2024-01-27T02:39:21.900699+0000","last_became_peered":"2024-01-27T02:39:21.900699+0000","last_unstale":"2024-01-27T02:39:22.890445+0000","last_undegraded":"2024-01-27T02:39:22.890445+0000","last_fullsized":"2024-01-27T02:39:22.890445+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:59:02.337961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.908703+0000","last_change":"2024-01-27T02:39:22.908250+0000","last_active":"2024-01-27T02:39:22.908703+0000","last_peered":"2024-01-27T02:39:22.908703+0000","last_clean":"2024-01-27T02:39:22.908703+0000","last_became_active":"2024-01-27T02:39:22.908011+0000","last_became_peered":"2024-01-27T02:39:22.908011+0000","last_unstale":"2024-01-27T02:39:22.908703+0000","last_undegraded":"2024-01-27T02:39:22.908703+0000","last_fullsized":"2024-01-27T02:39:22.908703+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:25:47.557777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.895255+0000","last_change":"2024-01-27T02:39:21.899014+0000","last_active":"2024-01-27T02:39:22.895255+0000","last_peered":"2024-01-27T02:39:22.895255+0000","last_clean":"2024-01-27T02:39:22.895255+0000","last_became_active":"2024-01-27T02:39:21.898758+0000","last_became_peered":"2024-01-27T02:39:21.898758+0000","last_unstale":"2024-01-27T02:39:22.895255+0000","last_undegraded":"2024-01-27T02:39:22.895255+0000","last_fullsized":"2024-01-27T02:39:22.895255+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:39:20.730878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902623+0000","last_change":"2024-01-27T02:39:22.902514+0000","last_active":"2024-01-27T02:39:22.902623+0000","last_peered":"2024-01-27T02:39:22.902623+0000","last_clean":"2024-01-27T02:39:22.902623+0000","last_became_active":"2024-01-27T02:39:22.902016+0000","last_became_peered":"2024-01-27T02:39:22.902016+0000","last_unstale":"2024-01-27T02:39:22.902623+0000","last_undegraded":"2024-01-27T02:39:22.902623+0000","last_fullsized":"2024-01-27T02:39:22.902623+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:50:07.072576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.912561+0000","last_change":"2024-01-27T02:39:21.901188+0000","last_active":"2024-01-27T02:39:22.912561+0000","last_peered":"2024-01-27T02:39:22.912561+0000","last_clean":"2024-01-27T02:39:22.912561+0000","last_became_active":"2024-01-27T02:39:21.900932+0000","last_became_peered":"2024-01-27T02:39:21.900932+0000","last_unstale":"2024-01-27T02:39:22.912561+0000","last_undegraded":"2024-01-27T02:39:22.912561+0000","last_fullsized":"2024-01-27T02:39:22.912561+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:58:36.696301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":274,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.894459+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:39:22.894459+0000","last_peered":"2024-01-27T02:39:22.894459+0000","last_clean":"2024-01-27T02:39:22.894459+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:39:22.894459+0000","last_undegraded":"2024-01-27T02:39:22.894459+0000","last_fullsized":"2024-01-27T02:39:22.894459+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1383,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193928,"statfs":{"total":200000000,"available":198583067,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.39400000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76200000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66900000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.58799999999999997}]},{"osd":1,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76900000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.56000000000000005},"min":{"1min":0.41799999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.67600000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.56499999999999995}]},{"osd":2,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.67900000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.49199999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76200000000000001,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.55400000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.48399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.66800000000000004,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.45400000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.74099999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.495}]},{"osd":4,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47699999999999998,"15min":0.48899999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70699999999999996,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.48899999999999999,"15min":0.51200000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69299999999999995,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.47099999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.51800000000000002,"15min":0.53700000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.307},"max":{"1min":0.73199999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55200000000000005,"15min":0.56499999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33500000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.51300000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57799999999999996,"15min":0.59599999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.90800000000000003},"last":0.57499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574940,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":806,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194505,"statfs":{"total":200000000,"available":199174045,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.63,"15min":0.54400000000000004},"min":{"1min":0.375,"5min":0.36499999999999999,"15min":0.29999999999999999},"max":{"1min":0.83099999999999996,"5min":4.125,"15min":4.125},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59699999999999998,"15min":0.54000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32700000000000001},"max":{"1min":0.75600000000000001,"5min":1.216,"15min":1.216},"last":0.96599999999999997}]},{"osd":1,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.67200000000000004,"15min":0.55600000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.34100000000000003},"max":{"1min":0.84199999999999997,"5min":8.2699999999999996,"15min":8.2699999999999996},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.88900000000000001,"15min":0.65500000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80600000000000005,"5min":26.411999999999999,"15min":26.411999999999999},"last":1.0129999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.75700000000000001,"15min":0.60799999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77000000000000002,"5min":17.143999999999998,"15min":17.143999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.64400000000000002,"15min":0.59299999999999997},"min":{"1min":0.45800000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85799999999999998,"5min":1.76,"15min":1.76},"last":0.57399999999999995}]},{"osd":3,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.624,"15min":0.56899999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.376},"max":{"1min":0.78000000000000003,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.624,"15min":0.58199999999999996},"min":{"1min":0.44500000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79300000000000004,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.53100000000000003}]},{"osd":4,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.30199999999999999},"max":{"1min":0.73599999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.50900000000000001,"15min":0.54300000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84899999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.61499999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.80800000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56299999999999994,"15min":0.59599999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.48599999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59299999999999997,"15min":0.60999999999999999},"min":{"1min":0.40999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.87,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55700000000000005,"15min":0.58199999999999996},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.36499999999999999},"max":{"1min":0.82899999999999996,"5min":1.038,"15min":1.038},"last":0.58899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":805,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194507,"statfs":{"total":200000000,"available":199175581,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39800000000000002,"15min":0.378},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41799999999999998,"15min":0.40000000000000002},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.50600000000000001,"5min":1.121,"15min":1.121},"last":0.42099999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40799999999999997,"15min":0.373},"min":{"1min":0.249,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":1.0940000000000001,"5min":1.25,"15min":1.25},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.44900000000000001,"15min":0.43099999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29899999999999999},"max":{"1min":0.498,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.39000000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.48499999999999999,"15min":0.46899999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.32100000000000001},"max":{"1min":0.56699999999999995,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.433},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.46600000000000003,"15min":0.45500000000000002},"min":{"1min":0.34899999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.56100000000000005,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.35699999999999998}]},{"osd":4,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.66600000000000004,"15min":0.63600000000000001},"min":{"1min":0.48299999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.85699999999999998,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66600000000000004,"15min":0.64000000000000001},"min":{"1min":0.45000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91600000000000004,"5min":1.026,"15min":1.026},"last":0.54300000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69099999999999995,"15min":0.64200000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":0.93400000000000005,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69699999999999995,"15min":0.67600000000000005},"min":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96199999999999997,"5min":3.181,"15min":3.181},"last":0.66900000000000004}]},{"osd":6,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.70599999999999996,"15min":0.69499999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88500000000000001,"5min":0.96399999999999997,"15min":1.024},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72099999999999997,"15min":0.68300000000000005},"min":{"1min":0.54000000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93600000000000005,"5min":1.869,"15min":1.869},"last":0.69299999999999995}]},{"osd":7,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73199999999999998,"15min":0.69099999999999995},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.47299999999999998},"max":{"1min":0.91400000000000003,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73099999999999998,"15min":0.69499999999999995},"min":{"1min":0.53000000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92000000000000004,"5min":1.708,"15min":1.708},"last":0.56100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1381,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193930,"statfs":{"total":200000000,"available":198585213,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.40600000000000003,"15min":0.39400000000000002},"min":{"1min":0.252,"5min":0.24399999999999999,"15min":0.23799999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.42899999999999999,"15min":0.41799999999999998},"min":{"1min":0.28899999999999998,"5min":0.27800000000000002,"15min":0.223},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.27800000000000002}]},{"osd":2,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.433,"15min":0.42199999999999999},"min":{"1min":0.31,"5min":0.28699999999999998,"15min":0.28199999999999997},"max":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.45700000000000002,"15min":0.438},"min":{"1min":0.312,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.32000000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.441,"15min":0.432},"min":{"1min":0.315,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.66100000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.495,"15min":0.47599999999999998},"min":{"1min":0.38800000000000001,"5min":0.33600000000000002,"15min":0.29099999999999998},"max":{"1min":0.67600000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.40899999999999997}]},{"osd":4,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66800000000000004,"15min":0.64400000000000002},"min":{"1min":0.505,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80900000000000005,"5min":0.997,"15min":0.997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.45800000000000002,"5min":0.44,"15min":0.41999999999999998},"max":{"1min":0.83399999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64900000000000002}]},{"osd":5,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68899999999999995,"15min":0.65800000000000003},"min":{"1min":0.51900000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.78800000000000003,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.54900000000000004,"5min":0.47799999999999998,"15min":0.44},"max":{"1min":0.84999999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.622}]},{"osd":6,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68300000000000005,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.42499999999999999},"max":{"1min":0.79800000000000004,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69899999999999995,"15min":0.68000000000000005},"min":{"1min":0.51600000000000001,"5min":0.45700000000000002,"15min":0.45000000000000001},"max":{"1min":0.90300000000000002,"5min":1.04,"15min":1.04},"last":0.59499999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.50800000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90200000000000002,"5min":1.014,"15min":1.014},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69699999999999995,"15min":0.69199999999999995},"min":{"1min":0.54500000000000004,"5min":0.498,"15min":0.498},"max":{"1min":0.81999999999999995,"5min":0.95799999999999996,"15min":0.97299999999999998},"last":0.60799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":805,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194506,"statfs":{"total":200000000,"available":199174993,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.41199999999999998},"min":{"1min":0.27300000000000002,"5min":0.23599999999999999,"15min":0.23000000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.46700000000000003,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.30499999999999999},"max":{"1min":0.68899999999999995,"5min":0.72799999999999998,"15min":6.2729999999999997},"last":0.45200000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.433,"15min":0.42299999999999999},"min":{"1min":0.33800000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.621,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.439,"15min":0.44400000000000001},"min":{"1min":0.33900000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.60399999999999998,"5min":0.65300000000000002,"15min":0.70099999999999996},"last":0.36199999999999999}]},{"osd":3,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.434,"15min":0.41799999999999998},"min":{"1min":0.32200000000000001,"5min":0.28899999999999998,"15min":0.27800000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49099999999999999,"15min":0.48599999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.505}]},{"osd":4,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68799999999999994,"15min":0.65900000000000003},"min":{"1min":0.499,"5min":0.46800000000000003,"15min":0.39300000000000002},"max":{"1min":0.90500000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.70599999999999996,"15min":0.78100000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95799999999999996,"5min":1.006,"15min":3.581},"last":0.79400000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65800000000000003,"15min":0.72699999999999998},"min":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92200000000000004,"5min":0.95899999999999996,"15min":3.5369999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.52200000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89100000000000001,"5min":0.92200000000000004,"15min":1.0529999999999999},"last":0.77700000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67200000000000004,"15min":0.66700000000000004},"min":{"1min":0.57499999999999996,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96499999999999997,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69599999999999995,"15min":0.68000000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.97099999999999997,"5min":1.0189999999999999,"15min":1.0209999999999999},"last":0.69599999999999995}]},{"osd":7,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69199999999999995,"15min":0.67900000000000005},"min":{"1min":0.58799999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97699999999999998,"5min":1.03,"15min":1.03},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70899999999999996,"15min":0.69299999999999995},"min":{"1min":0.55200000000000005,"5min":0.441,"15min":0.42699999999999999},"max":{"1min":0.99099999999999999,"5min":1.036,"15min":1.036},"last":0.67300000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":805,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194507,"statfs":{"total":200000000,"available":199175421,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.45500000000000002,"15min":0.38600000000000001},"min":{"1min":0.29699999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.67900000000000005,"5min":4.7380000000000004,"15min":4.7380000000000004},"last":0.309},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41899999999999998,"15min":0.40000000000000002},"min":{"1min":0.29399999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55500000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.378}]},{"osd":1,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.47499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28000000000000003,"5min":0.27400000000000002,"15min":0.22900000000000001},"max":{"1min":0.61599999999999999,"5min":4.5739999999999998,"15min":4.5739999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.53300000000000003,"15min":0.45200000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.98199999999999998,"5min":5.0529999999999999,"15min":5.0529999999999999},"last":0.35999999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.49199999999999999,"15min":0.443},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.28100000000000003},"max":{"1min":0.76700000000000002,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.55400000000000005,"15min":0.47699999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71799999999999997,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":0.49099999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69999999999999996,"15min":0.63800000000000001},"min":{"1min":0.45300000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91600000000000004,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.62},"min":{"1min":0.48199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86099999999999999,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.60899999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67600000000000005,"15min":0.626},"min":{"1min":0.438,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82399999999999995,"5min":1.409,"15min":1.409},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.78700000000000003,"15min":0.70399999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.89600000000000002,"5min":4.7960000000000003,"15min":4.7960000000000003},"last":0.625}]},{"osd":6,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.73899999999999999,"15min":0.67900000000000005},"min":{"1min":0.42799999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92700000000000005,"5min":5.0069999999999997,"15min":5.0069999999999997},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.69399999999999995,"15min":0.65900000000000003},"min":{"1min":0.47499999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.94199999999999995,"5min":1.103,"15min":1.103},"last":0.68899999999999995}]},{"osd":7,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.75700000000000001,"15min":0.66900000000000004},"min":{"1min":0.57599999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72699999999999998,"15min":0.68000000000000005},"min":{"1min":0.52800000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.88900000000000001,"5min":1.361,"15min":1.361},"last":0.67400000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":805,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194507,"statfs":{"total":200000000,"available":199175511,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.35599999999999998,"5min":0.34599999999999997,"15min":0.314},"max":{"1min":1.137,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66200000000000003,"15min":0.61399999999999999},"min":{"1min":0.496,"5min":0.372,"15min":0.372},"max":{"1min":1.016,"5min":7.0670000000000002,"15min":7.0670000000000002},"last":0.68999999999999995}]},{"osd":1,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.64200000000000002,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.35399999999999998},"max":{"1min":0.81000000000000005,"5min":5.1529999999999996,"15min":5.1529999999999996},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64400000000000002,"15min":0.60899999999999999},"min":{"1min":0.46400000000000002,"5min":0.40799999999999997,"15min":0.39000000000000001},"max":{"1min":1.054,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.60199999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.64200000000000002,"15min":0.61099999999999999},"min":{"1min":0.48399999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76600000000000001,"5min":5.0019999999999998,"15min":5.0019999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.63300000000000001,"15min":0.62},"min":{"1min":0.432,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2549999999999999,"5min":2.073,"15min":2.073},"last":0.55700000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.60499999999999998},"min":{"1min":0.45200000000000001,"5min":0.42099999999999999,"15min":0.36199999999999999},"max":{"1min":1.1899999999999999,"5min":6.7220000000000004,"15min":6.7220000000000004},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68999999999999995,"15min":0.66400000000000003},"min":{"1min":0.46100000000000002,"5min":0.44900000000000001,"15min":0.41399999999999998},"max":{"1min":0.88,"5min":6.5510000000000002,"15min":6.5510000000000002},"last":0.54000000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.34899999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.77500000000000002,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56599999999999995,"15min":0.51700000000000002},"min":{"1min":0.372,"5min":0.312,"15min":0.29999999999999999},"max":{"1min":0.68799999999999994,"5min":6.9269999999999996,"15min":6.9269999999999996},"last":0.443}]},{"osd":6,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56100000000000005,"15min":0.52200000000000002},"min":{"1min":0.40400000000000003,"5min":0.34999999999999998,"15min":0.32900000000000001},"max":{"1min":0.78600000000000003,"5min":5.2229999999999999,"15min":5.2229999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56200000000000006,"15min":0.54900000000000004},"min":{"1min":0.41799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82999999999999996,"5min":1.46,"15min":1.46},"last":0.375}]},{"osd":7,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61699999999999999,"15min":0.57499999999999996},"min":{"1min":0.434,"5min":0.34799999999999998,"15min":0.33600000000000002},"max":{"1min":0.78300000000000003,"5min":7.5339999999999998,"15min":7.5339999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.629,"15min":0.59399999999999997},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85299999999999998,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":0.42499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574940,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":805,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194507,"statfs":{"total":200000000,"available":199175469,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56399999999999995,"15min":0.58999999999999997},"min":{"1min":0.42099999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56699999999999995,"15min":0.57899999999999996},"min":{"1min":0.442,"5min":0.34499999999999997,"15min":0.27600000000000002},"max":{"1min":0.77100000000000002,"5min":0.78900000000000003,"15min":3.242},"last":0.40600000000000003}]},{"osd":1,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.622,"15min":0.61299999999999999},"min":{"1min":0.38500000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.3380000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61099999999999999,"15min":0.64300000000000002},"min":{"1min":0.35399999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78400000000000003,"5min":0.84399999999999997,"15min":0.95499999999999996},"last":0.61499999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58599999999999997,"15min":0.59699999999999998},"min":{"1min":0.40400000000000003,"5min":0.39100000000000001,"15min":0.372},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":3.04},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.48499999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80400000000000005,"5min":0.85899999999999999,"15min":0.89400000000000002},"last":0.44900000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60499999999999998,"15min":0.66800000000000004},"min":{"1min":0.40300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":4.032},"last":0.505},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65500000000000003,"15min":0.67700000000000005},"min":{"1min":0.48999999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.76100000000000001,"5min":0.86199999999999999,"15min":4.282},"last":0.52700000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50700000000000001,"15min":0.52200000000000002},"min":{"1min":0.35799999999999998,"5min":0.30599999999999999,"15min":0.26500000000000001},"max":{"1min":0.63600000000000001,"5min":0.81399999999999995,"15min":1.2869999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.495,"15min":0.55300000000000005},"min":{"1min":0.32800000000000001,"5min":0.318,"15min":0.30399999999999999},"max":{"1min":0.66900000000000004,"5min":0.79800000000000004,"15min":1.101},"last":0.56000000000000005}]},{"osd":6,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.497,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65200000000000002,"5min":0.73199999999999998,"15min":1.7969999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55300000000000005,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80100000000000005,"5min":0.85799999999999998,"15min":1.6739999999999999},"last":0.48399999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.38200000000000001,"5min":0.34799999999999998,"15min":0.32700000000000001},"max":{"1min":0.66800000000000004,"5min":0.77100000000000002,"15min":1.008},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59699999999999998,"15min":0.64900000000000002},"min":{"1min":0.436,"5min":0.35899999999999999,"15min":0.33700000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":2},"last":0.57599999999999996}]}]}],"pool_statfs":[]}} 2024-01-27T02:39:27.922 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-01-27T02:39:28.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.090+0000 7fc2ae583700 1 Processor -- start 2024-01-27T02:39:28.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.090+0000 7fc2ae583700 1 -- start start 2024-01-27T02:39:28.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.091+0000 7fc2ae583700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc2a815d720 0x7fc2a80693d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:28.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.091+0000 7fc2ae583700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a80699b0 con 0x7fc2a8067fd0 2024-01-27T02:39:28.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.091+0000 7fc2ae583700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a806bc60 con 0x7fc2a8067c00 2024-01-27T02:39:28.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.091+0000 7fc2ae583700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a80cc210 con 0x7fc2a815d720 2024-01-27T02:39:28.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.091+0000 7fc2a77fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc2a8067fd0 0x7fc2a815d010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41320/0 (socket says 172.21.15.7:41320) 2024-01-27T02:39:28.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.091+0000 7fc2a77fe700 1 -- 172.21.15.7:0/619441327 learned_addr learned my addr 172.21.15.7:0/619441327 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:28.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.091+0000 7fc2acb20700 1 --2- 172.21.15.7:0/619441327 >> v2:172.21.15.7:3300/0 conn(0x7fc2a815d720 0x7fc2a80693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:28.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.092+0000 7fc2a6ffd700 1 -- 172.21.15.7:0/619441327 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3994255409 0 0) 0x7fc2a80699b0 con 0x7fc2a8067fd0 2024-01-27T02:39:28.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.092+0000 7fc2a6ffd700 1 -- 172.21.15.7:0/619441327 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc290003570 con 0x7fc2a8067fd0 2024-01-27T02:39:28.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.092+0000 7fc2acb20700 1 -- 172.21.15.7:0/619441327 >> v1:172.21.15.188:6789/0 conn(0x7fc2a8067c00 legacy=0x7fc2a806d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:28.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.092+0000 7fc2acb20700 1 -- 172.21.15.7:0/619441327 >> v1:172.21.15.7:6789/0 conn(0x7fc2a8067fd0 legacy=0x7fc2a815d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:28.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.092+0000 7fc2acb20700 1 -- 172.21.15.7:0/619441327 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a806bf40 con 0x7fc2a815d720 2024-01-27T02:39:28.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.092+0000 7fc2acb20700 1 --2- 172.21.15.7:0/619441327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a815d720 0x7fc2a80693d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009e90 tx=0x7fc2a0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863c5e0931941ed6 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:28.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.092+0000 7fc2a6ffd700 1 -- 172.21.15.7:0/619441327 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a000e070 con 0x7fc2a815d720 2024-01-27T02:39:28.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.092+0000 7fc2a6ffd700 1 -- 172.21.15.7:0/619441327 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a000bc40 con 0x7fc2a815d720 2024-01-27T02:39:28.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.092+0000 7fc2a6ffd700 1 -- 172.21.15.7:0/619441327 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0012430 con 0x7fc2a815d720 2024-01-27T02:39:28.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.093+0000 7fc2ae583700 1 -- 172.21.15.7:0/619441327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a815d720 msgr2=0x7fc2a80693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:28.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.093+0000 7fc2ae583700 1 --2- 172.21.15.7:0/619441327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a815d720 0x7fc2a80693d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009e90 tx=0x7fc2a0009780 comp rx=0 tx=0).stop 2024-01-27T02:39:28.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.093+0000 7fc2ae583700 1 -- 172.21.15.7:0/619441327 shutdown_connections 2024-01-27T02:39:28.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.093+0000 7fc2ae583700 1 --2- 172.21.15.7:0/619441327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a815d720 0x7fc2a80693d0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:28.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.093+0000 7fc2ae583700 1 -- 172.21.15.7:0/619441327 >> 172.21.15.7:0/619441327 conn(0x7fc2a8071480 msgr2=0x7fc2a806b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:28.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.093+0000 7fc2ae583700 1 -- 172.21.15.7:0/619441327 shutdown_connections 2024-01-27T02:39:28.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.093+0000 7fc2ae583700 1 -- 172.21.15.7:0/619441327 wait complete. 2024-01-27T02:39:28.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.094+0000 7fc2ae583700 1 Processor -- start 2024-01-27T02:39:28.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.094+0000 7fc2ae583700 1 -- start start 2024-01-27T02:39:28.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.095+0000 7fc2ae583700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a8067c00 0x7fc2a8160730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:28.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.095+0000 7fc2ae583700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2a8067fd0 0x7fc2a80c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:28.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.095+0000 7fc2ae583700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2a815d720 0x7fc2a80c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:28.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.095+0000 7fc2ae583700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a81466d0 con 0x7fc2a8067c00 2024-01-27T02:39:28.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.095+0000 7fc2ae583700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a81600a0 con 0x7fc2a815d720 2024-01-27T02:39:28.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.095+0000 7fc2ae583700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a80c78d0 con 0x7fc2a8067fd0 2024-01-27T02:39:28.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.095+0000 7fc2a7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a8067c00 0x7fc2a8160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:28.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.095+0000 7fc2a7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a8067c00 0x7fc2a8160730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37752/0 (socket says 172.21.15.7:37752) 2024-01-27T02:39:28.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.095+0000 7fc2a7fff700 1 -- 172.21.15.7:0/1483325647 learned_addr learned my addr 172.21.15.7:0/1483325647 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:28.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.095+0000 7fc2acb20700 1 --2- 172.21.15.7:0/1483325647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2a815d720 0x7fc2a80c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:28.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.095+0000 7fc2a77fe700 1 --2- 172.21.15.7:0/1483325647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2a8067fd0 0x7fc2a80c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:28.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.095+0000 7fc2a7fff700 1 -- 172.21.15.7:0/1483325647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2a8067fd0 msgr2=0x7fc2a80c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:28.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.095+0000 7fc2a7fff700 1 --2- 172.21.15.7:0/1483325647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2a8067fd0 0x7fc2a80c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:28.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.095+0000 7fc2a7fff700 1 -- 172.21.15.7:0/1483325647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2a815d720 msgr2=0x7fc2a80c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:28.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.095+0000 7fc2a7fff700 1 --2- 172.21.15.7:0/1483325647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2a815d720 0x7fc2a80c72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:28.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.096+0000 7fc2a7fff700 1 -- 172.21.15.7:0/1483325647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a0009a20 con 0x7fc2a8067c00 2024-01-27T02:39:28.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.096+0000 7fc2a77fe700 1 --2- 172.21.15.7:0/1483325647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2a8067fd0 0x7fc2a80c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:39:28.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.096+0000 7fc2a7fff700 1 --2- 172.21.15.7:0/1483325647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a8067c00 0x7fc2a8160730 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fc29800c4e0 tx=0x7fc29800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:28.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.096+0000 7fc28ffff700 1 -- 172.21.15.7:0/1483325647 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298011590 con 0x7fc2a8067c00 2024-01-27T02:39:28.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.096+0000 7fc28ffff700 1 -- 172.21.15.7:0/1483325647 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc298011710 con 0x7fc2a8067c00 2024-01-27T02:39:28.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.096+0000 7fc2ae583700 1 -- 172.21.15.7:0/1483325647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a80c7bb0 con 0x7fc2a8067c00 2024-01-27T02:39:28.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.096+0000 7fc28ffff700 1 -- 172.21.15.7:0/1483325647 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298011ad0 con 0x7fc2a8067c00 2024-01-27T02:39:28.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.096+0000 7fc2ae583700 1 -- 172.21.15.7:0/1483325647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a80c8170 con 0x7fc2a8067c00 2024-01-27T02:39:28.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.097+0000 7fc2ae583700 1 -- 172.21.15.7:0/1483325647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a804c710 con 0x7fc2a8067c00 2024-01-27T02:39:28.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.097+0000 7fc28ffff700 1 -- 172.21.15.7:0/1483325647 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc29801b3b0 con 0x7fc2a8067c00 2024-01-27T02:39:28.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.098+0000 7fc28ffff700 1 --2- 172.21.15.7:0/1483325647 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc294066f30 0x7fc2940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:28.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.098+0000 7fc28ffff700 1 -- 172.21.15.7:0/1483325647 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc298083a70 con 0x7fc2a8067c00 2024-01-27T02:39:28.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.098+0000 7fc2a77fe700 1 --2- 172.21.15.7:0/1483325647 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc294066f30 0x7fc2940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:28.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.099+0000 7fc2a77fe700 1 --2- 172.21.15.7:0/1483325647 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc294066f30 0x7fc2940693e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc29c007fd0 tx=0x7fc29c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:28.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.102+0000 7fc28ffff700 1 -- 172.21.15.7:0/1483325647 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc298050ce0 con 0x7fc2a8067c00 2024-01-27T02:39:28.165 INFO:tasks.ceph.mds.a.smithi007.stderr:2024-01-27T02:39:28.165+0000 7f3190987700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-27T02:39:28.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.398+0000 7fc2ae583700 1 -- 172.21.15.7:0/1483325647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc2a806c220 con 0x7fc2a8067c00 2024-01-27T02:39:28.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.399+0000 7fc28ffff700 1 -- 172.21.15.7:0/1483325647 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1428 (secure 0 0 0) 0x7fc298017020 con 0x7fc2a8067c00 2024-01-27T02:39:28.399 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:28.399 INFO:teuthology.orchestra.run.smithi007.stdout:{"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":458,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":149,"num_osds":8,"num_up_osds":8,"osd_up_since":1706322715,"num_in_osds":8,"osd_in_since":1706322712,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592674,"bytes_used":7780700,"bytes_avail":1592219300,"bytes_total":1600000000,"write_bytes_sec":2218,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":77,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":7419}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-01-27T02:39:11.676743+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-01-27T02:39:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.401+0000 7fc2ae583700 1 -- 172.21.15.7:0/1483325647 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc294066f30 msgr2=0x7fc2940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.401+0000 7fc2ae583700 1 --2- 172.21.15.7:0/1483325647 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc294066f30 0x7fc2940693e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc29c007fd0 tx=0x7fc29c009040 comp rx=0 tx=0).stop 2024-01-27T02:39:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.401+0000 7fc2ae583700 1 -- 172.21.15.7:0/1483325647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a8067c00 msgr2=0x7fc2a8160730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.401+0000 7fc2ae583700 1 --2- 172.21.15.7:0/1483325647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a8067c00 0x7fc2a8160730 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fc29800c4e0 tx=0x7fc29800c8a0 comp rx=0 tx=0).stop 2024-01-27T02:39:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.401+0000 7fc2ae583700 1 -- 172.21.15.7:0/1483325647 shutdown_connections 2024-01-27T02:39:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.401+0000 7fc2ae583700 1 --2- 172.21.15.7:0/1483325647 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc294066f30 0x7fc2940693e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.401+0000 7fc2ae583700 1 --2- 172.21.15.7:0/1483325647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2a815d720 0x7fc2a80c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.401+0000 7fc2ae583700 1 --2- 172.21.15.7:0/1483325647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a8067c00 0x7fc2a8160730 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.401+0000 7fc2ae583700 1 --2- 172.21.15.7:0/1483325647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2a8067fd0 0x7fc2a80c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.401+0000 7fc2ae583700 1 -- 172.21.15.7:0/1483325647 >> 172.21.15.7:0/1483325647 conn(0x7fc2a8071480 msgr2=0x7fc2a80691a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.401+0000 7fc2ae583700 1 -- 172.21.15.7:0/1483325647 shutdown_connections 2024-01-27T02:39:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.401+0000 7fc2ae583700 1 -- 172.21.15.7:0/1483325647 wait complete. 2024-01-27T02:39:28.415 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-01-27T02:39:28.415 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:39:28.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.547+0000 7fd158f53700 1 Processor -- start 2024-01-27T02:39:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.547+0000 7fd158f53700 1 -- start start 2024-01-27T02:39:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.548+0000 7fd158f53700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd1540a7bd0 0x7fd1540af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.548+0000 7fd158f53700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1540a7fa0 0x7fd15406e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.548+0000 7fd158f53700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd15406ea30 con 0x7fd1540b1390 2024-01-27T02:39:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.548+0000 7fd158f53700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd15406fe40 con 0x7fd1540a7bd0 2024-01-27T02:39:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.548+0000 7fd158f53700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15406f870 con 0x7fd1540a7fa0 2024-01-27T02:39:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.548+0000 7fd151d9b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1540a7fa0 0x7fd15406e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.548+0000 7fd15259c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd1540a7bd0 0x7fd1540af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.548+0000 7fd151d9b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1540a7fa0 0x7fd15406e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:54654/0 (socket says 172.21.15.7:54654) 2024-01-27T02:39:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.548+0000 7fd151d9b700 1 -- 172.21.15.7:0/1013906035 learned_addr learned my addr 172.21.15.7:0/1013906035 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.549+0000 7fd15159a700 1 -- 172.21.15.7:0/1013906035 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045188836 0 0) 0x7fd15406ea30 con 0x7fd1540b1390 2024-01-27T02:39:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.549+0000 7fd15159a700 1 -- 172.21.15.7:0/1013906035 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd140003570 con 0x7fd1540b1390 2024-01-27T02:39:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.549+0000 7fd15259c700 1 -- 172.21.15.7:0/1013906035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1540a7fa0 msgr2=0x7fd15406e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.549+0000 7fd15259c700 1 --2- 172.21.15.7:0/1013906035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1540a7fa0 0x7fd15406e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.549+0000 7fd15259c700 1 -- 172.21.15.7:0/1013906035 >> v1:172.21.15.188:6789/0 conn(0x7fd1540b1390 legacy=0x7fd1540ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.549+0000 7fd15259c700 1 -- 172.21.15.7:0/1013906035 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd15406ec60 con 0x7fd1540a7bd0 2024-01-27T02:39:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.549+0000 7fd151d9b700 1 --2- 172.21.15.7:0/1013906035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1540a7fa0 0x7fd15406e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:39:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.549+0000 7fd15259c700 1 --2- 172.21.15.7:0/1013906035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1540a7bd0 0x7fd1540af6d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd1440098e0 tx=0x7fd1440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1dc185e0a1aef65 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.549+0000 7fd15159a700 1 -- 172.21.15.7:0/1013906035 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd144011630 con 0x7fd1540a7bd0 2024-01-27T02:39:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.549+0000 7fd15159a700 1 -- 172.21.15.7:0/1013906035 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd144011790 con 0x7fd1540a7bd0 2024-01-27T02:39:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.549+0000 7fd15159a700 1 -- 172.21.15.7:0/1013906035 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd144011b30 con 0x7fd1540a7bd0 2024-01-27T02:39:28.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.550+0000 7fd158f53700 1 -- 172.21.15.7:0/1013906035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1540a7bd0 msgr2=0x7fd1540af6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:28.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.550+0000 7fd158f53700 1 --2- 172.21.15.7:0/1013906035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1540a7bd0 0x7fd1540af6d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd1440098e0 tx=0x7fd1440091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:28.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.550+0000 7fd158f53700 1 -- 172.21.15.7:0/1013906035 shutdown_connections 2024-01-27T02:39:28.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.550+0000 7fd158f53700 1 --2- 172.21.15.7:0/1013906035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1540a7bd0 0x7fd1540af6d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:28.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.550+0000 7fd158f53700 1 --2- 172.21.15.7:0/1013906035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1540a7fa0 0x7fd15406e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:28.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.550+0000 7fd158f53700 1 -- 172.21.15.7:0/1013906035 >> 172.21.15.7:0/1013906035 conn(0x7fd1540679b0 msgr2=0x7fd154067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:28.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.550+0000 7fd158f53700 1 -- 172.21.15.7:0/1013906035 shutdown_connections 2024-01-27T02:39:28.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.550+0000 7fd158f53700 1 -- 172.21.15.7:0/1013906035 wait complete. 2024-01-27T02:39:28.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.551+0000 7fd158f53700 1 Processor -- start 2024-01-27T02:39:28.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.551+0000 7fd158f53700 1 -- start start 2024-01-27T02:39:28.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.552+0000 7fd158f53700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1540a7bd0 0x7fd154099700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:28.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.552+0000 7fd158f53700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1540a7fa0 0x7fd15414f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:28.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.552+0000 7fd158f53700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1540b1390 0x7fd1540cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:28.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.552+0000 7fd158f53700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd154099c40 con 0x7fd1540a7bd0 2024-01-27T02:39:28.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.552+0000 7fd158f53700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15416aeb0 con 0x7fd1540a7fa0 2024-01-27T02:39:28.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.552+0000 7fd158f53700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15416a330 con 0x7fd1540b1390 2024-01-27T02:39:28.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.552+0000 7fd15259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1540a7bd0 0x7fd154099700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:28.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.552+0000 7fd15259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1540a7bd0 0x7fd154099700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37758/0 (socket says 172.21.15.7:37758) 2024-01-27T02:39:28.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.552+0000 7fd15259c700 1 -- 172.21.15.7:0/2066794929 learned_addr learned my addr 172.21.15.7:0/2066794929 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:28.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.552+0000 7fd151d9b700 1 --2- 172.21.15.7:0/2066794929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1540a7fa0 0x7fd15414f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:28.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.552+0000 7fd152d9d700 1 --2- 172.21.15.7:0/2066794929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1540b1390 0x7fd1540cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:28.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.553+0000 7fd15259c700 1 -- 172.21.15.7:0/2066794929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1540b1390 msgr2=0x7fd1540cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:28.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.553+0000 7fd15259c700 1 --2- 172.21.15.7:0/2066794929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1540b1390 0x7fd1540cdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:28.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.553+0000 7fd15259c700 1 -- 172.21.15.7:0/2066794929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1540a7fa0 msgr2=0x7fd15414f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:28.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.553+0000 7fd15259c700 1 --2- 172.21.15.7:0/2066794929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1540a7fa0 0x7fd15414f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:28.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.553+0000 7fd15259c700 1 -- 172.21.15.7:0/2066794929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13c0094f0 con 0x7fd1540a7bd0 2024-01-27T02:39:28.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.553+0000 7fd15259c700 1 --2- 172.21.15.7:0/2066794929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1540a7bd0 0x7fd154099700 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fd1540aff00 tx=0x7fd144017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:28.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.553+0000 7fd14a7fc700 1 -- 172.21.15.7:0/2066794929 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd144011900 con 0x7fd1540a7bd0 2024-01-27T02:39:28.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.553+0000 7fd14a7fc700 1 -- 172.21.15.7:0/2066794929 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd144011a60 con 0x7fd1540a7bd0 2024-01-27T02:39:28.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.553+0000 7fd158f53700 1 -- 172.21.15.7:0/2066794929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd144009420 con 0x7fd1540a7bd0 2024-01-27T02:39:28.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.553+0000 7fd14a7fc700 1 -- 172.21.15.7:0/2066794929 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd144025640 con 0x7fd1540a7bd0 2024-01-27T02:39:28.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.553+0000 7fd158f53700 1 -- 172.21.15.7:0/2066794929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd15416a8f0 con 0x7fd1540a7bd0 2024-01-27T02:39:28.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.554+0000 7fd158f53700 1 -- 172.21.15.7:0/2066794929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd15404c710 con 0x7fd1540a7bd0 2024-01-27T02:39:28.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.554+0000 7fd14a7fc700 1 -- 172.21.15.7:0/2066794929 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd1440259f0 con 0x7fd1540a7bd0 2024-01-27T02:39:28.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.555+0000 7fd14a7fc700 1 --2- 172.21.15.7:0/2066794929 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd134066f30 0x7fd1340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:28.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.555+0000 7fd14a7fc700 1 -- 172.21.15.7:0/2066794929 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd144085cc0 con 0x7fd1540a7bd0 2024-01-27T02:39:28.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.555+0000 7fd151d9b700 1 --2- 172.21.15.7:0/2066794929 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd134066f30 0x7fd1340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:28.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.556+0000 7fd151d9b700 1 --2- 172.21.15.7:0/2066794929 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd134066f30 0x7fd1340693e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd15416d480 tx=0x7fd13c009250 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:28.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.559+0000 7fd14a7fc700 1 -- 172.21.15.7:0/2066794929 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd144052f30 con 0x7fd1540a7bd0 2024-01-27T02:39:28.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.757+0000 7fd158f53700 1 -- 172.21.15.7:0/2066794929 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd154069410 con 0x7fd134066f30 2024-01-27T02:39:28.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.766+0000 7fd14a7fc700 1 -- 172.21.15.7:0/2066794929 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285521 (secure 0 0 0) 0x7fd154069410 con 0x7fd134066f30 2024-01-27T02:39:28.770 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:28.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.769+0000 7fd158f53700 1 -- 172.21.15.7:0/2066794929 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd134066f30 msgr2=0x7fd1340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:28.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.769+0000 7fd158f53700 1 --2- 172.21.15.7:0/2066794929 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd134066f30 0x7fd1340693e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd15416d480 tx=0x7fd13c009250 comp rx=0 tx=0).stop 2024-01-27T02:39:28.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.769+0000 7fd158f53700 1 -- 172.21.15.7:0/2066794929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1540a7bd0 msgr2=0x7fd154099700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:28.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.769+0000 7fd158f53700 1 --2- 172.21.15.7:0/2066794929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1540a7bd0 0x7fd154099700 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fd1540aff00 tx=0x7fd144017000 comp rx=0 tx=0).stop 2024-01-27T02:39:28.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.769+0000 7fd158f53700 1 -- 172.21.15.7:0/2066794929 shutdown_connections 2024-01-27T02:39:28.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.769+0000 7fd158f53700 1 --2- 172.21.15.7:0/2066794929 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd134066f30 0x7fd1340693e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:28.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.769+0000 7fd158f53700 1 --2- 172.21.15.7:0/2066794929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1540b1390 0x7fd1540cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:28.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.769+0000 7fd158f53700 1 --2- 172.21.15.7:0/2066794929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1540a7bd0 0x7fd154099700 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:28.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.769+0000 7fd158f53700 1 --2- 172.21.15.7:0/2066794929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1540a7fa0 0x7fd15414f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:28.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.769+0000 7fd158f53700 1 -- 172.21.15.7:0/2066794929 >> 172.21.15.7:0/2066794929 conn(0x7fd1540679b0 msgr2=0x7fd15414d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:28.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.769+0000 7fd158f53700 1 -- 172.21.15.7:0/2066794929 shutdown_connections 2024-01-27T02:39:28.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:28.769+0000 7fd158f53700 1 -- 172.21.15.7:0/2066794929 wait complete. 2024-01-27T02:39:28.773 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:39:28.783 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-01-27T02:39:27.679897+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7598,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554901,"statfs":{"total":1600000000,"available":1592219300,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001385"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.898603+0000","last_change":"2024-01-27T02:39:22.898429+0000","last_active":"2024-01-27T02:39:22.898603+0000","last_peered":"2024-01-27T02:39:22.898603+0000","last_clean":"2024-01-27T02:39:22.898603+0000","last_became_active":"2024-01-27T02:39:22.897112+0000","last_became_peered":"2024-01-27T02:39:22.897112+0000","last_unstale":"2024-01-27T02:39:22.898603+0000","last_undegraded":"2024-01-27T02:39:22.898603+0000","last_fullsized":"2024-01-27T02:39:22.898603+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:00:12.452602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902737+0000","last_change":"2024-01-27T02:39:22.902650+0000","last_active":"2024-01-27T02:39:22.902737+0000","last_peered":"2024-01-27T02:39:22.902737+0000","last_clean":"2024-01-27T02:39:22.902737+0000","last_became_active":"2024-01-27T02:39:22.902344+0000","last_became_peered":"2024-01-27T02:39:22.902344+0000","last_unstale":"2024-01-27T02:39:22.902737+0000","last_undegraded":"2024-01-27T02:39:22.902737+0000","last_fullsized":"2024-01-27T02:39:22.902737+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:02:57.717352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.899868+0000","last_change":"2024-01-27T02:39:22.899799+0000","last_active":"2024-01-27T02:39:22.899868+0000","last_peered":"2024-01-27T02:39:22.899868+0000","last_clean":"2024-01-27T02:39:22.899868+0000","last_became_active":"2024-01-27T02:39:22.899535+0000","last_became_peered":"2024-01-27T02:39:22.899535+0000","last_unstale":"2024-01-27T02:39:22.899868+0000","last_undegraded":"2024-01-27T02:39:22.899868+0000","last_fullsized":"2024-01-27T02:39:22.899868+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:27:21.148523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.905689+0000","last_change":"2024-01-27T02:39:22.904667+0000","last_active":"2024-01-27T02:39:22.905689+0000","last_peered":"2024-01-27T02:39:22.905689+0000","last_clean":"2024-01-27T02:39:22.905689+0000","last_became_active":"2024-01-27T02:39:22.900570+0000","last_became_peered":"2024-01-27T02:39:22.900570+0000","last_unstale":"2024-01-27T02:39:22.905689+0000","last_undegraded":"2024-01-27T02:39:22.905689+0000","last_fullsized":"2024-01-27T02:39:22.905689+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:17:26.997560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.897810+0000","last_change":"2024-01-27T02:39:22.897296+0000","last_active":"2024-01-27T02:39:22.897810+0000","last_peered":"2024-01-27T02:39:22.897810+0000","last_clean":"2024-01-27T02:39:22.897810+0000","last_became_active":"2024-01-27T02:39:22.897051+0000","last_became_peered":"2024-01-27T02:39:22.897051+0000","last_unstale":"2024-01-27T02:39:22.897810+0000","last_undegraded":"2024-01-27T02:39:22.897810+0000","last_fullsized":"2024-01-27T02:39:22.897810+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:33:46.694312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.900968+0000","last_change":"2024-01-27T02:39:22.900656+0000","last_active":"2024-01-27T02:39:22.900968+0000","last_peered":"2024-01-27T02:39:22.900968+0000","last_clean":"2024-01-27T02:39:22.900968+0000","last_became_active":"2024-01-27T02:39:22.899965+0000","last_became_peered":"2024-01-27T02:39:22.899965+0000","last_unstale":"2024-01-27T02:39:22.900968+0000","last_undegraded":"2024-01-27T02:39:22.900968+0000","last_fullsized":"2024-01-27T02:39:22.900968+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:09:14.280979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.897983+0000","last_change":"2024-01-27T02:39:22.897809+0000","last_active":"2024-01-27T02:39:22.897983+0000","last_peered":"2024-01-27T02:39:22.897983+0000","last_clean":"2024-01-27T02:39:22.897983+0000","last_became_active":"2024-01-27T02:39:22.897437+0000","last_became_peered":"2024-01-27T02:39:22.897437+0000","last_unstale":"2024-01-27T02:39:22.897983+0000","last_undegraded":"2024-01-27T02:39:22.897983+0000","last_fullsized":"2024-01-27T02:39:22.897983+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:02:31.201590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.904534+0000","last_change":"2024-01-27T02:39:22.903395+0000","last_active":"2024-01-27T02:39:22.904534+0000","last_peered":"2024-01-27T02:39:22.904534+0000","last_clean":"2024-01-27T02:39:22.904534+0000","last_became_active":"2024-01-27T02:39:22.903023+0000","last_became_peered":"2024-01-27T02:39:22.903023+0000","last_unstale":"2024-01-27T02:39:22.904534+0000","last_undegraded":"2024-01-27T02:39:22.904534+0000","last_fullsized":"2024-01-27T02:39:22.904534+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:08:06.794664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910976+0000","last_change":"2024-01-27T02:39:22.906721+0000","last_active":"2024-01-27T02:39:22.910976+0000","last_peered":"2024-01-27T02:39:22.910976+0000","last_clean":"2024-01-27T02:39:22.910976+0000","last_became_active":"2024-01-27T02:39:22.906469+0000","last_became_peered":"2024-01-27T02:39:22.906469+0000","last_unstale":"2024-01-27T02:39:22.910976+0000","last_undegraded":"2024-01-27T02:39:22.910976+0000","last_fullsized":"2024-01-27T02:39:22.910976+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:04:43.117550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.897833+0000","last_change":"2024-01-27T02:39:22.897761+0000","last_active":"2024-01-27T02:39:22.897833+0000","last_peered":"2024-01-27T02:39:22.897833+0000","last_clean":"2024-01-27T02:39:22.897833+0000","last_became_active":"2024-01-27T02:39:22.897518+0000","last_became_peered":"2024-01-27T02:39:22.897518+0000","last_unstale":"2024-01-27T02:39:22.897833+0000","last_undegraded":"2024-01-27T02:39:22.897833+0000","last_fullsized":"2024-01-27T02:39:22.897833+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:47:36.010163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.900063+0000","last_change":"2024-01-27T02:39:22.899989+0000","last_active":"2024-01-27T02:39:22.900063+0000","last_peered":"2024-01-27T02:39:22.900063+0000","last_clean":"2024-01-27T02:39:22.900063+0000","last_became_active":"2024-01-27T02:39:22.899715+0000","last_became_peered":"2024-01-27T02:39:22.899715+0000","last_unstale":"2024-01-27T02:39:22.900063+0000","last_undegraded":"2024-01-27T02:39:22.900063+0000","last_fullsized":"2024-01-27T02:39:22.900063+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:22:14.092868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.898551+0000","last_change":"2024-01-27T02:39:22.898312+0000","last_active":"2024-01-27T02:39:22.898551+0000","last_peered":"2024-01-27T02:39:22.898551+0000","last_clean":"2024-01-27T02:39:22.898551+0000","last_became_active":"2024-01-27T02:39:22.898071+0000","last_became_peered":"2024-01-27T02:39:22.898071+0000","last_unstale":"2024-01-27T02:39:22.898551+0000","last_undegraded":"2024-01-27T02:39:22.898551+0000","last_fullsized":"2024-01-27T02:39:22.898551+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:17.789817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.904646+0000","last_change":"2024-01-27T02:39:22.901995+0000","last_active":"2024-01-27T02:39:22.904646+0000","last_peered":"2024-01-27T02:39:22.904646+0000","last_clean":"2024-01-27T02:39:22.904646+0000","last_became_active":"2024-01-27T02:39:22.901761+0000","last_became_peered":"2024-01-27T02:39:22.901761+0000","last_unstale":"2024-01-27T02:39:22.904646+0000","last_undegraded":"2024-01-27T02:39:22.904646+0000","last_fullsized":"2024-01-27T02:39:22.904646+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:50:35.407782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902718+0000","last_change":"2024-01-27T02:39:22.902243+0000","last_active":"2024-01-27T02:39:22.902718+0000","last_peered":"2024-01-27T02:39:22.902718+0000","last_clean":"2024-01-27T02:39:22.902718+0000","last_became_active":"2024-01-27T02:39:22.901971+0000","last_became_peered":"2024-01-27T02:39:22.901971+0000","last_unstale":"2024-01-27T02:39:22.902718+0000","last_undegraded":"2024-01-27T02:39:22.902718+0000","last_fullsized":"2024-01-27T02:39:22.902718+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:35:38.537841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.907395+0000","last_change":"2024-01-27T02:39:22.906123+0000","last_active":"2024-01-27T02:39:22.907395+0000","last_peered":"2024-01-27T02:39:22.907395+0000","last_clean":"2024-01-27T02:39:22.907395+0000","last_became_active":"2024-01-27T02:39:22.905311+0000","last_became_peered":"2024-01-27T02:39:22.905311+0000","last_unstale":"2024-01-27T02:39:22.907395+0000","last_undegraded":"2024-01-27T02:39:22.907395+0000","last_fullsized":"2024-01-27T02:39:22.907395+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:26:42.911820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.907825+0000","last_change":"2024-01-27T02:39:22.907096+0000","last_active":"2024-01-27T02:39:22.907825+0000","last_peered":"2024-01-27T02:39:22.907825+0000","last_clean":"2024-01-27T02:39:22.907825+0000","last_became_active":"2024-01-27T02:39:22.906214+0000","last_became_peered":"2024-01-27T02:39:22.906214+0000","last_unstale":"2024-01-27T02:39:22.907825+0000","last_undegraded":"2024-01-27T02:39:22.907825+0000","last_fullsized":"2024-01-27T02:39:22.907825+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:35:39.970169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.904556+0000","last_change":"2024-01-27T02:39:22.904462+0000","last_active":"2024-01-27T02:39:22.904556+0000","last_peered":"2024-01-27T02:39:22.904556+0000","last_clean":"2024-01-27T02:39:22.904556+0000","last_became_active":"2024-01-27T02:39:22.904033+0000","last_became_peered":"2024-01-27T02:39:22.904033+0000","last_unstale":"2024-01-27T02:39:22.904556+0000","last_undegraded":"2024-01-27T02:39:22.904556+0000","last_fullsized":"2024-01-27T02:39:22.904556+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:59:03.340181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902308+0000","last_change":"2024-01-27T02:39:22.901938+0000","last_active":"2024-01-27T02:39:22.902308+0000","last_peered":"2024-01-27T02:39:22.902308+0000","last_clean":"2024-01-27T02:39:22.902308+0000","last_became_active":"2024-01-27T02:39:22.897160+0000","last_became_peered":"2024-01-27T02:39:22.897160+0000","last_unstale":"2024-01-27T02:39:22.902308+0000","last_undegraded":"2024-01-27T02:39:22.902308+0000","last_fullsized":"2024-01-27T02:39:22.902308+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:04:33.028169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.898697+0000","last_change":"2024-01-27T02:39:22.897541+0000","last_active":"2024-01-27T02:39:22.898697+0000","last_peered":"2024-01-27T02:39:22.898697+0000","last_clean":"2024-01-27T02:39:22.898697+0000","last_became_active":"2024-01-27T02:39:22.897251+0000","last_became_peered":"2024-01-27T02:39:22.897251+0000","last_unstale":"2024-01-27T02:39:22.898697+0000","last_undegraded":"2024-01-27T02:39:22.898697+0000","last_fullsized":"2024-01-27T02:39:22.898697+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:34:23.208150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902274+0000","last_change":"2024-01-27T02:39:22.899543+0000","last_active":"2024-01-27T02:39:22.902274+0000","last_peered":"2024-01-27T02:39:22.902274+0000","last_clean":"2024-01-27T02:39:22.902274+0000","last_became_active":"2024-01-27T02:39:22.899256+0000","last_became_peered":"2024-01-27T02:39:22.899256+0000","last_unstale":"2024-01-27T02:39:22.902274+0000","last_undegraded":"2024-01-27T02:39:22.902274+0000","last_fullsized":"2024-01-27T02:39:22.902274+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:14:55.893637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.898760+0000","last_change":"2024-01-27T02:39:22.898466+0000","last_active":"2024-01-27T02:39:22.898760+0000","last_peered":"2024-01-27T02:39:22.898760+0000","last_clean":"2024-01-27T02:39:22.898760+0000","last_became_active":"2024-01-27T02:39:22.897999+0000","last_became_peered":"2024-01-27T02:39:22.897999+0000","last_unstale":"2024-01-27T02:39:22.898760+0000","last_undegraded":"2024-01-27T02:39:22.898760+0000","last_fullsized":"2024-01-27T02:39:22.898760+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:39:21.733098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.904084+0000","last_change":"2024-01-27T02:39:22.904004+0000","last_active":"2024-01-27T02:39:22.904084+0000","last_peered":"2024-01-27T02:39:22.904084+0000","last_clean":"2024-01-27T02:39:22.904084+0000","last_became_active":"2024-01-27T02:39:22.903705+0000","last_became_peered":"2024-01-27T02:39:22.903705+0000","last_unstale":"2024-01-27T02:39:22.904084+0000","last_undegraded":"2024-01-27T02:39:22.904084+0000","last_fullsized":"2024-01-27T02:39:22.904084+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:13:03.822990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902780+0000","last_change":"2024-01-27T02:39:22.902663+0000","last_active":"2024-01-27T02:39:22.902780+0000","last_peered":"2024-01-27T02:39:22.902780+0000","last_clean":"2024-01-27T02:39:22.902780+0000","last_became_active":"2024-01-27T02:39:22.902445+0000","last_became_peered":"2024-01-27T02:39:22.902445+0000","last_unstale":"2024-01-27T02:39:22.902780+0000","last_undegraded":"2024-01-27T02:39:22.902780+0000","last_fullsized":"2024-01-27T02:39:22.902780+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:09:14.280979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.905378+0000","last_change":"2024-01-27T02:39:22.905287+0000","last_active":"2024-01-27T02:39:22.905378+0000","last_peered":"2024-01-27T02:39:22.905378+0000","last_clean":"2024-01-27T02:39:22.905378+0000","last_became_active":"2024-01-27T02:39:22.905095+0000","last_became_peered":"2024-01-27T02:39:22.905095+0000","last_unstale":"2024-01-27T02:39:22.905378+0000","last_undegraded":"2024-01-27T02:39:22.905378+0000","last_fullsized":"2024-01-27T02:39:22.905378+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:35:11.141652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.908684+0000","last_change":"2024-01-27T02:39:22.907502+0000","last_active":"2024-01-27T02:39:22.908684+0000","last_peered":"2024-01-27T02:39:22.908684+0000","last_clean":"2024-01-27T02:39:22.908684+0000","last_became_active":"2024-01-27T02:39:22.907247+0000","last_became_peered":"2024-01-27T02:39:22.907247+0000","last_unstale":"2024-01-27T02:39:22.908684+0000","last_undegraded":"2024-01-27T02:39:22.908684+0000","last_fullsized":"2024-01-27T02:39:22.908684+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:08:14.937701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.907847+0000","last_change":"2024-01-27T02:39:22.907272+0000","last_active":"2024-01-27T02:39:22.907847+0000","last_peered":"2024-01-27T02:39:22.907847+0000","last_clean":"2024-01-27T02:39:22.907847+0000","last_became_active":"2024-01-27T02:39:22.906623+0000","last_became_peered":"2024-01-27T02:39:22.906623+0000","last_unstale":"2024-01-27T02:39:22.907847+0000","last_undegraded":"2024-01-27T02:39:22.907847+0000","last_fullsized":"2024-01-27T02:39:22.907847+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:39:21.733098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.908642+0000","last_change":"2024-01-27T02:39:22.908551+0000","last_active":"2024-01-27T02:39:22.908642+0000","last_peered":"2024-01-27T02:39:22.908642+0000","last_clean":"2024-01-27T02:39:22.908642+0000","last_became_active":"2024-01-27T02:39:22.907980+0000","last_became_peered":"2024-01-27T02:39:22.907980+0000","last_unstale":"2024-01-27T02:39:22.908642+0000","last_undegraded":"2024-01-27T02:39:22.908642+0000","last_fullsized":"2024-01-27T02:39:22.908642+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:50.975864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.903922+0000","last_change":"2024-01-27T02:39:22.903237+0000","last_active":"2024-01-27T02:39:22.903922+0000","last_peered":"2024-01-27T02:39:22.903922+0000","last_clean":"2024-01-27T02:39:22.903922+0000","last_became_active":"2024-01-27T02:39:22.902973+0000","last_became_peered":"2024-01-27T02:39:22.902973+0000","last_unstale":"2024-01-27T02:39:22.903922+0000","last_undegraded":"2024-01-27T02:39:22.903922+0000","last_fullsized":"2024-01-27T02:39:22.903922+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:35:38.537841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.900393+0000","last_change":"2024-01-27T02:39:22.900300+0000","last_active":"2024-01-27T02:39:22.900393+0000","last_peered":"2024-01-27T02:39:22.900393+0000","last_clean":"2024-01-27T02:39:22.900393+0000","last_became_active":"2024-01-27T02:39:22.899932+0000","last_became_peered":"2024-01-27T02:39:22.899932+0000","last_unstale":"2024-01-27T02:39:22.900393+0000","last_undegraded":"2024-01-27T02:39:22.900393+0000","last_fullsized":"2024-01-27T02:39:22.900393+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:51:22.997295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.907415+0000","last_change":"2024-01-27T02:39:22.906323+0000","last_active":"2024-01-27T02:39:22.907415+0000","last_peered":"2024-01-27T02:39:22.907415+0000","last_clean":"2024-01-27T02:39:22.907415+0000","last_became_active":"2024-01-27T02:39:22.905609+0000","last_became_peered":"2024-01-27T02:39:22.905609+0000","last_unstale":"2024-01-27T02:39:22.907415+0000","last_undegraded":"2024-01-27T02:39:22.907415+0000","last_fullsized":"2024-01-27T02:39:22.907415+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:52:33.405152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902602+0000","last_change":"2024-01-27T02:39:22.902281+0000","last_active":"2024-01-27T02:39:22.902602+0000","last_peered":"2024-01-27T02:39:22.902602+0000","last_clean":"2024-01-27T02:39:22.902602+0000","last_became_active":"2024-01-27T02:39:22.901653+0000","last_became_peered":"2024-01-27T02:39:22.901653+0000","last_unstale":"2024-01-27T02:39:22.902602+0000","last_undegraded":"2024-01-27T02:39:22.902602+0000","last_fullsized":"2024-01-27T02:39:22.902602+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:57:54.178723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.897739+0000","last_change":"2024-01-27T02:39:22.897658+0000","last_active":"2024-01-27T02:39:22.897739+0000","last_peered":"2024-01-27T02:39:22.897739+0000","last_clean":"2024-01-27T02:39:22.897739+0000","last_became_active":"2024-01-27T02:39:22.897115+0000","last_became_peered":"2024-01-27T02:39:22.897115+0000","last_unstale":"2024-01-27T02:39:22.897739+0000","last_undegraded":"2024-01-27T02:39:22.897739+0000","last_fullsized":"2024-01-27T02:39:22.897739+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:11:12.572257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.904065+0000","last_change":"2024-01-27T02:39:22.903978+0000","last_active":"2024-01-27T02:39:22.904065+0000","last_peered":"2024-01-27T02:39:22.904065+0000","last_clean":"2024-01-27T02:39:22.904065+0000","last_became_active":"2024-01-27T02:39:22.903705+0000","last_became_peered":"2024-01-27T02:39:22.903705+0000","last_unstale":"2024-01-27T02:39:22.904065+0000","last_undegraded":"2024-01-27T02:39:22.904065+0000","last_fullsized":"2024-01-27T02:39:22.904065+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:36:32.750631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"148'4","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.913408+0000","last_change":"2024-01-27T02:39:21.897428+0000","last_active":"2024-01-27T02:39:22.913408+0000","last_peered":"2024-01-27T02:39:22.913408+0000","last_clean":"2024-01-27T02:39:22.913408+0000","last_became_active":"2024-01-27T02:39:21.897179+0000","last_became_peered":"2024-01-27T02:39:21.897179+0000","last_unstale":"2024-01-27T02:39:22.913408+0000","last_undegraded":"2024-01-27T02:39:22.913408+0000","last_fullsized":"2024-01-27T02:39:22.913408+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:47.650928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.911163+0000","last_change":"2024-01-27T02:39:22.907668+0000","last_active":"2024-01-27T02:39:22.911163+0000","last_peered":"2024-01-27T02:39:22.911163+0000","last_clean":"2024-01-27T02:39:22.911163+0000","last_became_active":"2024-01-27T02:39:22.907423+0000","last_became_peered":"2024-01-27T02:39:22.907423+0000","last_unstale":"2024-01-27T02:39:22.911163+0000","last_undegraded":"2024-01-27T02:39:22.911163+0000","last_fullsized":"2024-01-27T02:39:22.911163+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:48.653148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"148'1","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.909191+0000","last_change":"2024-01-27T02:39:21.895025+0000","last_active":"2024-01-27T02:39:22.909191+0000","last_peered":"2024-01-27T02:39:22.909191+0000","last_clean":"2024-01-27T02:39:22.909191+0000","last_became_active":"2024-01-27T02:39:21.894753+0000","last_became_peered":"2024-01-27T02:39:21.894753+0000","last_unstale":"2024-01-27T02:39:22.909191+0000","last_undegraded":"2024-01-27T02:39:22.909191+0000","last_fullsized":"2024-01-27T02:39:22.909191+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:23:12.301215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.898741+0000","last_change":"2024-01-27T02:39:22.898310+0000","last_active":"2024-01-27T02:39:22.898741+0000","last_peered":"2024-01-27T02:39:22.898741+0000","last_clean":"2024-01-27T02:39:22.898741+0000","last_became_active":"2024-01-27T02:39:22.897673+0000","last_became_peered":"2024-01-27T02:39:22.897673+0000","last_unstale":"2024-01-27T02:39:22.898741+0000","last_undegraded":"2024-01-27T02:39:22.898741+0000","last_fullsized":"2024-01-27T02:39:22.898741+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:15:15.541057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910286+0000","last_change":"2024-01-27T02:39:21.894906+0000","last_active":"2024-01-27T02:39:22.910286+0000","last_peered":"2024-01-27T02:39:22.910286+0000","last_clean":"2024-01-27T02:39:22.910286+0000","last_became_active":"2024-01-27T02:39:21.894564+0000","last_became_peered":"2024-01-27T02:39:21.894564+0000","last_unstale":"2024-01-27T02:39:22.910286+0000","last_undegraded":"2024-01-27T02:39:22.910286+0000","last_fullsized":"2024-01-27T02:39:22.910286+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:36:31.748411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910835+0000","last_change":"2024-01-27T02:39:22.909673+0000","last_active":"2024-01-27T02:39:22.910835+0000","last_peered":"2024-01-27T02:39:22.910835+0000","last_clean":"2024-01-27T02:39:22.910835+0000","last_became_active":"2024-01-27T02:39:22.908815+0000","last_became_peered":"2024-01-27T02:39:22.908815+0000","last_unstale":"2024-01-27T02:39:22.910835+0000","last_undegraded":"2024-01-27T02:39:22.910835+0000","last_fullsized":"2024-01-27T02:39:22.910835+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:02:57.570709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.912880+0000","last_change":"2024-01-27T02:39:21.900974+0000","last_active":"2024-01-27T02:39:22.912880+0000","last_peered":"2024-01-27T02:39:22.912880+0000","last_clean":"2024-01-27T02:39:22.912880+0000","last_became_active":"2024-01-27T02:39:21.900576+0000","last_became_peered":"2024-01-27T02:39:21.900576+0000","last_unstale":"2024-01-27T02:39:22.912880+0000","last_undegraded":"2024-01-27T02:39:22.912880+0000","last_fullsized":"2024-01-27T02:39:22.912880+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:10:31.872654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902332+0000","last_change":"2024-01-27T02:39:22.902089+0000","last_active":"2024-01-27T02:39:22.902332+0000","last_peered":"2024-01-27T02:39:22.902332+0000","last_clean":"2024-01-27T02:39:22.902332+0000","last_became_active":"2024-01-27T02:39:22.897524+0000","last_became_peered":"2024-01-27T02:39:22.897524+0000","last_unstale":"2024-01-27T02:39:22.902332+0000","last_undegraded":"2024-01-27T02:39:22.902332+0000","last_fullsized":"2024-01-27T02:39:22.902332+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:06:41.763215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"148'1","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.912090+0000","last_change":"2024-01-27T02:39:21.898367+0000","last_active":"2024-01-27T02:39:22.912090+0000","last_peered":"2024-01-27T02:39:22.912090+0000","last_clean":"2024-01-27T02:39:22.912090+0000","last_became_active":"2024-01-27T02:39:21.898114+0000","last_became_peered":"2024-01-27T02:39:21.898114+0000","last_unstale":"2024-01-27T02:39:22.912090+0000","last_undegraded":"2024-01-27T02:39:22.912090+0000","last_fullsized":"2024-01-27T02:39:22.912090+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:35:46.505332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.904385+0000","last_change":"2024-01-27T02:39:22.904273+0000","last_active":"2024-01-27T02:39:22.904385+0000","last_peered":"2024-01-27T02:39:22.904385+0000","last_clean":"2024-01-27T02:39:22.904385+0000","last_became_active":"2024-01-27T02:39:22.903665+0000","last_became_peered":"2024-01-27T02:39:22.903665+0000","last_unstale":"2024-01-27T02:39:22.904385+0000","last_undegraded":"2024-01-27T02:39:22.904385+0000","last_fullsized":"2024-01-27T02:39:22.904385+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:40:33.462225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"148'1","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.909959+0000","last_change":"2024-01-27T02:39:21.899252+0000","last_active":"2024-01-27T02:39:22.909959+0000","last_peered":"2024-01-27T02:39:22.909959+0000","last_clean":"2024-01-27T02:39:22.909959+0000","last_became_active":"2024-01-27T02:39:21.898982+0000","last_became_peered":"2024-01-27T02:39:21.898982+0000","last_unstale":"2024-01-27T02:39:22.909959+0000","last_undegraded":"2024-01-27T02:39:22.909959+0000","last_fullsized":"2024-01-27T02:39:22.909959+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:11:11.570037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910749+0000","last_change":"2024-01-27T02:39:22.906183+0000","last_active":"2024-01-27T02:39:22.910749+0000","last_peered":"2024-01-27T02:39:22.910749+0000","last_clean":"2024-01-27T02:39:22.910749+0000","last_became_active":"2024-01-27T02:39:22.904977+0000","last_became_peered":"2024-01-27T02:39:22.904977+0000","last_unstale":"2024-01-27T02:39:22.910749+0000","last_undegraded":"2024-01-27T02:39:22.910749+0000","last_fullsized":"2024-01-27T02:39:22.910749+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:18:02.494561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.891115+0000","last_change":"2024-01-27T02:39:21.898838+0000","last_active":"2024-01-27T02:39:22.891115+0000","last_peered":"2024-01-27T02:39:22.891115+0000","last_clean":"2024-01-27T02:39:22.891115+0000","last_became_active":"2024-01-27T02:39:21.898603+0000","last_became_peered":"2024-01-27T02:39:21.898603+0000","last_unstale":"2024-01-27T02:39:22.891115+0000","last_undegraded":"2024-01-27T02:39:22.891115+0000","last_fullsized":"2024-01-27T02:39:22.891115+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:09:29.890085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910984+0000","last_change":"2024-01-27T02:39:22.910891+0000","last_active":"2024-01-27T02:39:22.910984+0000","last_peered":"2024-01-27T02:39:22.910984+0000","last_clean":"2024-01-27T02:39:22.910984+0000","last_became_active":"2024-01-27T02:39:22.910593+0000","last_became_peered":"2024-01-27T02:39:22.910593+0000","last_unstale":"2024-01-27T02:39:22.910984+0000","last_undegraded":"2024-01-27T02:39:22.910984+0000","last_fullsized":"2024-01-27T02:39:22.910984+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:03:28.071762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.897217+0000","last_change":"2024-01-27T02:39:21.903093+0000","last_active":"2024-01-27T02:39:22.897217+0000","last_peered":"2024-01-27T02:39:22.897217+0000","last_clean":"2024-01-27T02:39:22.897217+0000","last_became_active":"2024-01-27T02:39:21.902782+0000","last_became_peered":"2024-01-27T02:39:21.902782+0000","last_unstale":"2024-01-27T02:39:22.897217+0000","last_undegraded":"2024-01-27T02:39:22.897217+0000","last_fullsized":"2024-01-27T02:39:22.897217+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:26:01.500470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.908830+0000","last_change":"2024-01-27T02:39:22.908736+0000","last_active":"2024-01-27T02:39:22.908830+0000","last_peered":"2024-01-27T02:39:22.908830+0000","last_clean":"2024-01-27T02:39:22.908830+0000","last_became_active":"2024-01-27T02:39:22.908321+0000","last_became_peered":"2024-01-27T02:39:22.908321+0000","last_unstale":"2024-01-27T02:39:22.908830+0000","last_undegraded":"2024-01-27T02:39:22.908830+0000","last_fullsized":"2024-01-27T02:39:22.908830+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:23:05.214270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.894643+0000","last_change":"2024-01-27T02:39:21.898565+0000","last_active":"2024-01-27T02:39:22.894643+0000","last_peered":"2024-01-27T02:39:22.894643+0000","last_clean":"2024-01-27T02:39:22.894643+0000","last_became_active":"2024-01-27T02:39:21.898337+0000","last_became_peered":"2024-01-27T02:39:21.898337+0000","last_unstale":"2024-01-27T02:39:22.894643+0000","last_undegraded":"2024-01-27T02:39:22.894643+0000","last_fullsized":"2024-01-27T02:39:22.894643+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:10:31.872654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.903942+0000","last_change":"2024-01-27T02:39:22.903289+0000","last_active":"2024-01-27T02:39:22.903942+0000","last_peered":"2024-01-27T02:39:22.903942+0000","last_clean":"2024-01-27T02:39:22.903942+0000","last_became_active":"2024-01-27T02:39:22.903021+0000","last_became_peered":"2024-01-27T02:39:22.903021+0000","last_unstale":"2024-01-27T02:39:22.903942+0000","last_undegraded":"2024-01-27T02:39:22.903942+0000","last_fullsized":"2024-01-27T02:39:22.903942+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:03:29.364153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.889969+0000","last_change":"2024-01-27T02:39:21.901433+0000","last_active":"2024-01-27T02:39:22.889969+0000","last_peered":"2024-01-27T02:39:22.889969+0000","last_clean":"2024-01-27T02:39:22.889969+0000","last_became_active":"2024-01-27T02:39:21.901145+0000","last_became_peered":"2024-01-27T02:39:21.901145+0000","last_unstale":"2024-01-27T02:39:22.889969+0000","last_undegraded":"2024-01-27T02:39:22.889969+0000","last_fullsized":"2024-01-27T02:39:22.889969+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:27:20.146303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.905809+0000","last_change":"2024-01-27T02:39:22.905679+0000","last_active":"2024-01-27T02:39:22.905809+0000","last_peered":"2024-01-27T02:39:22.905809+0000","last_clean":"2024-01-27T02:39:22.905809+0000","last_became_active":"2024-01-27T02:39:22.905262+0000","last_became_peered":"2024-01-27T02:39:22.905262+0000","last_unstale":"2024-01-27T02:39:22.905809+0000","last_undegraded":"2024-01-27T02:39:22.905809+0000","last_fullsized":"2024-01-27T02:39:22.905809+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:53:50.414456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"148'3","reported_seq":13,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910510+0000","last_change":"2024-01-27T02:39:21.894472+0000","last_active":"2024-01-27T02:39:22.910510+0000","last_peered":"2024-01-27T02:39:22.910510+0000","last_clean":"2024-01-27T02:39:22.910510+0000","last_became_active":"2024-01-27T02:39:21.893990+0000","last_became_peered":"2024-01-27T02:39:21.893990+0000","last_unstale":"2024-01-27T02:39:22.910510+0000","last_undegraded":"2024-01-27T02:39:22.910510+0000","last_fullsized":"2024-01-27T02:39:22.910510+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:01:18.616886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.897842+0000","last_change":"2024-01-27T02:39:22.897409+0000","last_active":"2024-01-27T02:39:22.897842+0000","last_peered":"2024-01-27T02:39:22.897842+0000","last_clean":"2024-01-27T02:39:22.897842+0000","last_became_active":"2024-01-27T02:39:22.897150+0000","last_became_peered":"2024-01-27T02:39:22.897150+0000","last_unstale":"2024-01-27T02:39:22.897842+0000","last_undegraded":"2024-01-27T02:39:22.897842+0000","last_fullsized":"2024-01-27T02:39:22.897842+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:35:39.970169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"148'6","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.914572+0000","last_change":"2024-01-27T02:39:21.901023+0000","last_active":"2024-01-27T02:39:22.914572+0000","last_peered":"2024-01-27T02:39:22.914572+0000","last_clean":"2024-01-27T02:39:22.914572+0000","last_became_active":"2024-01-27T02:39:21.899485+0000","last_became_peered":"2024-01-27T02:39:21.899485+0000","last_unstale":"2024-01-27T02:39:22.914572+0000","last_undegraded":"2024-01-27T02:39:22.914572+0000","last_fullsized":"2024-01-27T02:39:22.914572+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:03:28.361932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.907434+0000","last_change":"2024-01-27T02:39:22.906514+0000","last_active":"2024-01-27T02:39:22.907434+0000","last_peered":"2024-01-27T02:39:22.907434+0000","last_clean":"2024-01-27T02:39:22.907434+0000","last_became_active":"2024-01-27T02:39:22.905927+0000","last_became_peered":"2024-01-27T02:39:22.905927+0000","last_unstale":"2024-01-27T02:39:22.907434+0000","last_undegraded":"2024-01-27T02:39:22.907434+0000","last_fullsized":"2024-01-27T02:39:22.907434+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:30:29.146275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.892289+0000","last_change":"2024-01-27T02:39:21.901950+0000","last_active":"2024-01-27T02:39:22.892289+0000","last_peered":"2024-01-27T02:39:22.892289+0000","last_clean":"2024-01-27T02:39:22.892289+0000","last_became_active":"2024-01-27T02:39:21.901709+0000","last_became_peered":"2024-01-27T02:39:21.901709+0000","last_unstale":"2024-01-27T02:39:22.892289+0000","last_undegraded":"2024-01-27T02:39:22.892289+0000","last_fullsized":"2024-01-27T02:39:22.892289+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:27:55.432732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.900762+0000","last_change":"2024-01-27T02:39:22.900687+0000","last_active":"2024-01-27T02:39:22.900762+0000","last_peered":"2024-01-27T02:39:22.900762+0000","last_clean":"2024-01-27T02:39:22.900762+0000","last_became_active":"2024-01-27T02:39:22.900372+0000","last_became_peered":"2024-01-27T02:39:22.900372+0000","last_unstale":"2024-01-27T02:39:22.900762+0000","last_undegraded":"2024-01-27T02:39:22.900762+0000","last_fullsized":"2024-01-27T02:39:22.900762+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:19:27.934421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.894074+0000","last_change":"2024-01-27T02:39:21.901620+0000","last_active":"2024-01-27T02:39:22.894074+0000","last_peered":"2024-01-27T02:39:22.894074+0000","last_clean":"2024-01-27T02:39:22.894074+0000","last_became_active":"2024-01-27T02:39:21.901102+0000","last_became_peered":"2024-01-27T02:39:21.901102+0000","last_unstale":"2024-01-27T02:39:22.894074+0000","last_undegraded":"2024-01-27T02:39:22.894074+0000","last_fullsized":"2024-01-27T02:39:22.894074+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:52:32.402932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910954+0000","last_change":"2024-01-27T02:39:22.906855+0000","last_active":"2024-01-27T02:39:22.910954+0000","last_peered":"2024-01-27T02:39:22.910954+0000","last_clean":"2024-01-27T02:39:22.910954+0000","last_became_active":"2024-01-27T02:39:22.906480+0000","last_became_peered":"2024-01-27T02:39:22.906480+0000","last_unstale":"2024-01-27T02:39:22.910954+0000","last_undegraded":"2024-01-27T02:39:22.910954+0000","last_fullsized":"2024-01-27T02:39:22.910954+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:40:33.462225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"148'8","reported_seq":18,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.911228+0000","last_change":"2024-01-27T02:39:21.900650+0000","last_active":"2024-01-27T02:39:22.911228+0000","last_peered":"2024-01-27T02:39:22.911228+0000","last_clean":"2024-01-27T02:39:22.911228+0000","last_became_active":"2024-01-27T02:39:21.899844+0000","last_became_peered":"2024-01-27T02:39:21.899844+0000","last_unstale":"2024-01-27T02:39:22.911228+0000","last_undegraded":"2024-01-27T02:39:22.911228+0000","last_fullsized":"2024-01-27T02:39:22.911228+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:03:27.069542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.911277+0000","last_change":"2024-01-27T02:39:22.908170+0000","last_active":"2024-01-27T02:39:22.911277+0000","last_peered":"2024-01-27T02:39:22.911277+0000","last_clean":"2024-01-27T02:39:22.911277+0000","last_became_active":"2024-01-27T02:39:22.907713+0000","last_became_peered":"2024-01-27T02:39:22.907713+0000","last_unstale":"2024-01-27T02:39:22.911277+0000","last_undegraded":"2024-01-27T02:39:22.911277+0000","last_fullsized":"2024-01-27T02:39:22.911277+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:59:03.340181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"148'1","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.907916+0000","last_change":"2024-01-27T02:39:21.898336+0000","last_active":"2024-01-27T02:39:22.907916+0000","last_peered":"2024-01-27T02:39:22.907916+0000","last_clean":"2024-01-27T02:39:22.907916+0000","last_became_active":"2024-01-27T02:39:21.898073+0000","last_became_peered":"2024-01-27T02:39:21.898073+0000","last_unstale":"2024-01-27T02:39:22.907916+0000","last_undegraded":"2024-01-27T02:39:22.907916+0000","last_fullsized":"2024-01-27T02:39:22.907916+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:59:06.427886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.907601+0000","last_change":"2024-01-27T02:39:22.907034+0000","last_active":"2024-01-27T02:39:22.907601+0000","last_peered":"2024-01-27T02:39:22.907601+0000","last_clean":"2024-01-27T02:39:22.907601+0000","last_became_active":"2024-01-27T02:39:22.906810+0000","last_became_peered":"2024-01-27T02:39:22.906810+0000","last_unstale":"2024-01-27T02:39:22.907601+0000","last_undegraded":"2024-01-27T02:39:22.907601+0000","last_fullsized":"2024-01-27T02:39:22.907601+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:03:28.071762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.911846+0000","last_change":"2024-01-27T02:39:21.902022+0000","last_active":"2024-01-27T02:39:22.911846+0000","last_peered":"2024-01-27T02:39:22.911846+0000","last_clean":"2024-01-27T02:39:22.911846+0000","last_became_active":"2024-01-27T02:39:21.901762+0000","last_became_peered":"2024-01-27T02:39:21.901762+0000","last_unstale":"2024-01-27T02:39:22.911846+0000","last_undegraded":"2024-01-27T02:39:22.911846+0000","last_fullsized":"2024-01-27T02:39:22.911846+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:35:38.967949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.903467+0000","last_change":"2024-01-27T02:39:22.902929+0000","last_active":"2024-01-27T02:39:22.903467+0000","last_peered":"2024-01-27T02:39:22.903467+0000","last_clean":"2024-01-27T02:39:22.903467+0000","last_became_active":"2024-01-27T02:39:22.902654+0000","last_became_peered":"2024-01-27T02:39:22.902654+0000","last_unstale":"2024-01-27T02:39:22.903467+0000","last_undegraded":"2024-01-27T02:39:22.903467+0000","last_fullsized":"2024-01-27T02:39:22.903467+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:02:57.570709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"148'1","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910217+0000","last_change":"2024-01-27T02:39:21.895442+0000","last_active":"2024-01-27T02:39:22.910217+0000","last_peered":"2024-01-27T02:39:22.910217+0000","last_clean":"2024-01-27T02:39:22.910217+0000","last_became_active":"2024-01-27T02:39:21.894130+0000","last_became_peered":"2024-01-27T02:39:21.894130+0000","last_unstale":"2024-01-27T02:39:22.910217+0000","last_undegraded":"2024-01-27T02:39:22.910217+0000","last_fullsized":"2024-01-27T02:39:22.910217+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:02:56.715132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.908720+0000","last_change":"2024-01-27T02:39:22.908370+0000","last_active":"2024-01-27T02:39:22.908720+0000","last_peered":"2024-01-27T02:39:22.908720+0000","last_clean":"2024-01-27T02:39:22.908720+0000","last_became_active":"2024-01-27T02:39:22.907779+0000","last_became_peered":"2024-01-27T02:39:22.907779+0000","last_unstale":"2024-01-27T02:39:22.908720+0000","last_undegraded":"2024-01-27T02:39:22.908720+0000","last_fullsized":"2024-01-27T02:39:22.908720+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:17:07.671167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.894420+0000","last_change":"2024-01-27T02:39:21.895019+0000","last_active":"2024-01-27T02:39:22.894420+0000","last_peered":"2024-01-27T02:39:22.894420+0000","last_clean":"2024-01-27T02:39:22.894420+0000","last_became_active":"2024-01-27T02:39:21.894813+0000","last_became_peered":"2024-01-27T02:39:21.894813+0000","last_unstale":"2024-01-27T02:39:22.894420+0000","last_undegraded":"2024-01-27T02:39:22.894420+0000","last_fullsized":"2024-01-27T02:39:22.894420+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:01:58.908495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.900222+0000","last_change":"2024-01-27T02:39:22.900150+0000","last_active":"2024-01-27T02:39:22.900222+0000","last_peered":"2024-01-27T02:39:22.900222+0000","last_clean":"2024-01-27T02:39:22.900222+0000","last_became_active":"2024-01-27T02:39:22.899453+0000","last_became_peered":"2024-01-27T02:39:22.899453+0000","last_unstale":"2024-01-27T02:39:22.900222+0000","last_undegraded":"2024-01-27T02:39:22.900222+0000","last_fullsized":"2024-01-27T02:39:22.900222+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:08:54.067660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910284+0000","last_change":"2024-01-27T02:39:21.896656+0000","last_active":"2024-01-27T02:39:22.910284+0000","last_peered":"2024-01-27T02:39:22.910284+0000","last_clean":"2024-01-27T02:39:22.910284+0000","last_became_active":"2024-01-27T02:39:21.896422+0000","last_became_peered":"2024-01-27T02:39:21.896422+0000","last_unstale":"2024-01-27T02:39:22.910284+0000","last_undegraded":"2024-01-27T02:39:22.910284+0000","last_fullsized":"2024-01-27T02:39:22.910284+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:10:40.058748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.900541+0000","last_change":"2024-01-27T02:39:22.900467+0000","last_active":"2024-01-27T02:39:22.900541+0000","last_peered":"2024-01-27T02:39:22.900541+0000","last_clean":"2024-01-27T02:39:22.900541+0000","last_became_active":"2024-01-27T02:39:22.899866+0000","last_became_peered":"2024-01-27T02:39:22.899866+0000","last_unstale":"2024-01-27T02:39:22.900541+0000","last_undegraded":"2024-01-27T02:39:22.900541+0000","last_fullsized":"2024-01-27T02:39:22.900541+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:27:21.148523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.893201+0000","last_change":"2024-01-27T02:39:21.900412+0000","last_active":"2024-01-27T02:39:22.893201+0000","last_peered":"2024-01-27T02:39:22.893201+0000","last_clean":"2024-01-27T02:39:22.893201+0000","last_became_active":"2024-01-27T02:39:21.900133+0000","last_became_peered":"2024-01-27T02:39:21.900133+0000","last_unstale":"2024-01-27T02:39:22.893201+0000","last_undegraded":"2024-01-27T02:39:22.893201+0000","last_fullsized":"2024-01-27T02:39:22.893201+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:40:32.460005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.908737+0000","last_change":"2024-01-27T02:39:22.908527+0000","last_active":"2024-01-27T02:39:22.908737+0000","last_peered":"2024-01-27T02:39:22.908737+0000","last_clean":"2024-01-27T02:39:22.908737+0000","last_became_active":"2024-01-27T02:39:22.908130+0000","last_became_peered":"2024-01-27T02:39:22.908130+0000","last_unstale":"2024-01-27T02:39:22.908737+0000","last_undegraded":"2024-01-27T02:39:22.908737+0000","last_fullsized":"2024-01-27T02:39:22.908737+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:25:03.541488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.890111+0000","last_change":"2024-01-27T02:39:21.889901+0000","last_active":"2024-01-27T02:39:22.890111+0000","last_peered":"2024-01-27T02:39:22.890111+0000","last_clean":"2024-01-27T02:39:22.890111+0000","last_became_active":"2024-01-27T02:39:21.889524+0000","last_became_peered":"2024-01-27T02:39:21.889524+0000","last_unstale":"2024-01-27T02:39:22.890111+0000","last_undegraded":"2024-01-27T02:39:22.890111+0000","last_fullsized":"2024-01-27T02:39:22.890111+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:16.787597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910855+0000","last_change":"2024-01-27T02:39:22.909847+0000","last_active":"2024-01-27T02:39:22.910855+0000","last_peered":"2024-01-27T02:39:22.910855+0000","last_clean":"2024-01-27T02:39:22.910855+0000","last_became_active":"2024-01-27T02:39:22.909357+0000","last_became_peered":"2024-01-27T02:39:22.909357+0000","last_unstale":"2024-01-27T02:39:22.910855+0000","last_undegraded":"2024-01-27T02:39:22.910855+0000","last_fullsized":"2024-01-27T02:39:22.910855+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:12.863356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.894295+0000","last_change":"2024-01-27T02:39:21.901504+0000","last_active":"2024-01-27T02:39:22.894295+0000","last_peered":"2024-01-27T02:39:22.894295+0000","last_clean":"2024-01-27T02:39:22.894295+0000","last_became_active":"2024-01-27T02:39:21.901245+0000","last_became_peered":"2024-01-27T02:39:21.901245+0000","last_unstale":"2024-01-27T02:39:22.894295+0000","last_undegraded":"2024-01-27T02:39:22.894295+0000","last_fullsized":"2024-01-27T02:39:22.894295+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:57:53.176503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.900280+0000","last_change":"2024-01-27T02:39:22.900231+0000","last_active":"2024-01-27T02:39:22.900280+0000","last_peered":"2024-01-27T02:39:22.900280+0000","last_clean":"2024-01-27T02:39:22.900280+0000","last_became_active":"2024-01-27T02:39:22.899499+0000","last_became_peered":"2024-01-27T02:39:22.899499+0000","last_unstale":"2024-01-27T02:39:22.900280+0000","last_undegraded":"2024-01-27T02:39:22.900280+0000","last_fullsized":"2024-01-27T02:39:22.900280+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:04:43.117550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.912471+0000","last_change":"2024-01-27T02:39:21.896736+0000","last_active":"2024-01-27T02:39:22.912471+0000","last_peered":"2024-01-27T02:39:22.912471+0000","last_clean":"2024-01-27T02:39:22.912471+0000","last_became_active":"2024-01-27T02:39:21.896197+0000","last_became_peered":"2024-01-27T02:39:21.896197+0000","last_unstale":"2024-01-27T02:39:22.912471+0000","last_undegraded":"2024-01-27T02:39:22.912471+0000","last_fullsized":"2024-01-27T02:39:22.912471+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:00:34.270221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.898823+0000","last_change":"2024-01-27T02:39:22.898657+0000","last_active":"2024-01-27T02:39:22.898823+0000","last_peered":"2024-01-27T02:39:22.898823+0000","last_clean":"2024-01-27T02:39:22.898823+0000","last_became_active":"2024-01-27T02:39:22.898442+0000","last_became_peered":"2024-01-27T02:39:22.898442+0000","last_unstale":"2024-01-27T02:39:22.898823+0000","last_undegraded":"2024-01-27T02:39:22.898823+0000","last_fullsized":"2024-01-27T02:39:22.898823+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:55:52.096811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.895235+0000","last_change":"2024-01-27T02:39:21.898780+0000","last_active":"2024-01-27T02:39:22.895235+0000","last_peered":"2024-01-27T02:39:22.895235+0000","last_clean":"2024-01-27T02:39:22.895235+0000","last_became_active":"2024-01-27T02:39:21.898579+0000","last_became_peered":"2024-01-27T02:39:21.898579+0000","last_unstale":"2024-01-27T02:39:22.895235+0000","last_undegraded":"2024-01-27T02:39:22.895235+0000","last_fullsized":"2024-01-27T02:39:22.895235+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:59:36.125517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.906436+0000","last_change":"2024-01-27T02:39:22.906355+0000","last_active":"2024-01-27T02:39:22.906436+0000","last_peered":"2024-01-27T02:39:22.906436+0000","last_clean":"2024-01-27T02:39:22.906436+0000","last_became_active":"2024-01-27T02:39:22.902011+0000","last_became_peered":"2024-01-27T02:39:22.902011+0000","last_unstale":"2024-01-27T02:39:22.906436+0000","last_undegraded":"2024-01-27T02:39:22.906436+0000","last_fullsized":"2024-01-27T02:39:22.906436+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:09:30.892305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.894225+0000","last_change":"2024-01-27T02:39:21.900466+0000","last_active":"2024-01-27T02:39:22.894225+0000","last_peered":"2024-01-27T02:39:22.894225+0000","last_clean":"2024-01-27T02:39:22.894225+0000","last_became_active":"2024-01-27T02:39:21.900211+0000","last_became_peered":"2024-01-27T02:39:21.900211+0000","last_unstale":"2024-01-27T02:39:22.894225+0000","last_undegraded":"2024-01-27T02:39:22.894225+0000","last_fullsized":"2024-01-27T02:39:22.894225+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:26:41.909600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.901978+0000","last_change":"2024-01-27T02:39:22.901912+0000","last_active":"2024-01-27T02:39:22.901978+0000","last_peered":"2024-01-27T02:39:22.901978+0000","last_clean":"2024-01-27T02:39:22.901978+0000","last_became_active":"2024-01-27T02:39:22.901571+0000","last_became_peered":"2024-01-27T02:39:22.901571+0000","last_unstale":"2024-01-27T02:39:22.901978+0000","last_undegraded":"2024-01-27T02:39:22.901978+0000","last_fullsized":"2024-01-27T02:39:22.901978+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:17.789817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.897627+0000","last_change":"2024-01-27T02:39:21.903774+0000","last_active":"2024-01-27T02:39:22.897627+0000","last_peered":"2024-01-27T02:39:22.897627+0000","last_clean":"2024-01-27T02:39:22.897627+0000","last_became_active":"2024-01-27T02:39:21.903535+0000","last_became_peered":"2024-01-27T02:39:21.903535+0000","last_unstale":"2024-01-27T02:39:22.897627+0000","last_undegraded":"2024-01-27T02:39:22.897627+0000","last_fullsized":"2024-01-27T02:39:22.897627+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:02:56.568488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.898120+0000","last_change":"2024-01-27T02:39:22.898044+0000","last_active":"2024-01-27T02:39:22.898120+0000","last_peered":"2024-01-27T02:39:22.898120+0000","last_clean":"2024-01-27T02:39:22.898120+0000","last_became_active":"2024-01-27T02:39:22.897789+0000","last_became_peered":"2024-01-27T02:39:22.897789+0000","last_unstale":"2024-01-27T02:39:22.898120+0000","last_undegraded":"2024-01-27T02:39:22.898120+0000","last_fullsized":"2024-01-27T02:39:22.898120+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:36:32.750631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.895852+0000","last_change":"2024-01-27T02:39:21.900816+0000","last_active":"2024-01-27T02:39:22.895852+0000","last_peered":"2024-01-27T02:39:22.895852+0000","last_clean":"2024-01-27T02:39:22.895852+0000","last_became_active":"2024-01-27T02:39:21.900263+0000","last_became_peered":"2024-01-27T02:39:21.900263+0000","last_unstale":"2024-01-27T02:39:22.895852+0000","last_undegraded":"2024-01-27T02:39:22.895852+0000","last_fullsized":"2024-01-27T02:39:22.895852+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:49.973643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.901979+0000","last_change":"2024-01-27T02:39:22.901592+0000","last_active":"2024-01-27T02:39:22.901979+0000","last_peered":"2024-01-27T02:39:22.901979+0000","last_clean":"2024-01-27T02:39:22.901979+0000","last_became_active":"2024-01-27T02:39:22.901146+0000","last_became_peered":"2024-01-27T02:39:22.901146+0000","last_unstale":"2024-01-27T02:39:22.901979+0000","last_undegraded":"2024-01-27T02:39:22.901979+0000","last_fullsized":"2024-01-27T02:39:22.901979+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:25:47.557777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.894599+0000","last_change":"2024-01-27T02:39:21.899411+0000","last_active":"2024-01-27T02:39:22.894599+0000","last_peered":"2024-01-27T02:39:22.894599+0000","last_clean":"2024-01-27T02:39:22.894599+0000","last_became_active":"2024-01-27T02:39:21.899153+0000","last_became_peered":"2024-01-27T02:39:21.899153+0000","last_unstale":"2024-01-27T02:39:22.894599+0000","last_undegraded":"2024-01-27T02:39:22.894599+0000","last_fullsized":"2024-01-27T02:39:22.894599+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:18:01.492341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902253+0000","last_change":"2024-01-27T02:39:22.899280+0000","last_active":"2024-01-27T02:39:22.902253+0000","last_peered":"2024-01-27T02:39:22.902253+0000","last_clean":"2024-01-27T02:39:22.902253+0000","last_became_active":"2024-01-27T02:39:22.899002+0000","last_became_peered":"2024-01-27T02:39:22.899002+0000","last_unstale":"2024-01-27T02:39:22.902253+0000","last_undegraded":"2024-01-27T02:39:22.902253+0000","last_fullsized":"2024-01-27T02:39:22.902253+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:43:03.625409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.890445+0000","last_change":"2024-01-27T02:39:21.901048+0000","last_active":"2024-01-27T02:39:22.890445+0000","last_peered":"2024-01-27T02:39:22.890445+0000","last_clean":"2024-01-27T02:39:22.890445+0000","last_became_active":"2024-01-27T02:39:21.900699+0000","last_became_peered":"2024-01-27T02:39:21.900699+0000","last_unstale":"2024-01-27T02:39:22.890445+0000","last_undegraded":"2024-01-27T02:39:22.890445+0000","last_fullsized":"2024-01-27T02:39:22.890445+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:59:02.337961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.908703+0000","last_change":"2024-01-27T02:39:22.908250+0000","last_active":"2024-01-27T02:39:22.908703+0000","last_peered":"2024-01-27T02:39:22.908703+0000","last_clean":"2024-01-27T02:39:22.908703+0000","last_became_active":"2024-01-27T02:39:22.908011+0000","last_became_peered":"2024-01-27T02:39:22.908011+0000","last_unstale":"2024-01-27T02:39:22.908703+0000","last_undegraded":"2024-01-27T02:39:22.908703+0000","last_fullsized":"2024-01-27T02:39:22.908703+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:25:47.557777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.895255+0000","last_change":"2024-01-27T02:39:21.899014+0000","last_active":"2024-01-27T02:39:22.895255+0000","last_peered":"2024-01-27T02:39:22.895255+0000","last_clean":"2024-01-27T02:39:22.895255+0000","last_became_active":"2024-01-27T02:39:21.898758+0000","last_became_peered":"2024-01-27T02:39:21.898758+0000","last_unstale":"2024-01-27T02:39:22.895255+0000","last_undegraded":"2024-01-27T02:39:22.895255+0000","last_fullsized":"2024-01-27T02:39:22.895255+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:39:20.730878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902623+0000","last_change":"2024-01-27T02:39:22.902514+0000","last_active":"2024-01-27T02:39:22.902623+0000","last_peered":"2024-01-27T02:39:22.902623+0000","last_clean":"2024-01-27T02:39:22.902623+0000","last_became_active":"2024-01-27T02:39:22.902016+0000","last_became_peered":"2024-01-27T02:39:22.902016+0000","last_unstale":"2024-01-27T02:39:22.902623+0000","last_undegraded":"2024-01-27T02:39:22.902623+0000","last_fullsized":"2024-01-27T02:39:22.902623+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:50:07.072576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.912561+0000","last_change":"2024-01-27T02:39:21.901188+0000","last_active":"2024-01-27T02:39:22.912561+0000","last_peered":"2024-01-27T02:39:22.912561+0000","last_clean":"2024-01-27T02:39:22.912561+0000","last_became_active":"2024-01-27T02:39:21.900932+0000","last_became_peered":"2024-01-27T02:39:21.900932+0000","last_unstale":"2024-01-27T02:39:22.912561+0000","last_undegraded":"2024-01-27T02:39:22.912561+0000","last_fullsized":"2024-01-27T02:39:22.912561+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:58:36.696301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":274,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.894459+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:39:22.894459+0000","last_peered":"2024-01-27T02:39:22.894459+0000","last_clean":"2024-01-27T02:39:22.894459+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:39:22.894459+0000","last_undegraded":"2024-01-27T02:39:22.894459+0000","last_fullsized":"2024-01-27T02:39:22.894459+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1383,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193928,"statfs":{"total":200000000,"available":198583067,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.39400000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76200000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66900000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.58799999999999997}]},{"osd":1,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76900000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.56000000000000005},"min":{"1min":0.41799999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.67600000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.56499999999999995}]},{"osd":2,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.67900000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.49199999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76200000000000001,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.55400000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.48399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.66800000000000004,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.45400000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.74099999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.495}]},{"osd":4,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47699999999999998,"15min":0.48899999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70699999999999996,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.48899999999999999,"15min":0.51200000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69299999999999995,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.47099999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.51800000000000002,"15min":0.53700000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.307},"max":{"1min":0.73199999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55200000000000005,"15min":0.56499999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33500000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.51300000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57799999999999996,"15min":0.59599999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.90800000000000003},"last":0.57499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574940,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":806,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194505,"statfs":{"total":200000000,"available":199174045,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.63,"15min":0.54400000000000004},"min":{"1min":0.375,"5min":0.36499999999999999,"15min":0.29999999999999999},"max":{"1min":0.83099999999999996,"5min":4.125,"15min":4.125},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59699999999999998,"15min":0.54000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32700000000000001},"max":{"1min":0.75600000000000001,"5min":1.216,"15min":1.216},"last":0.96599999999999997}]},{"osd":1,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.67200000000000004,"15min":0.55600000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.34100000000000003},"max":{"1min":0.84199999999999997,"5min":8.2699999999999996,"15min":8.2699999999999996},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.88900000000000001,"15min":0.65500000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80600000000000005,"5min":26.411999999999999,"15min":26.411999999999999},"last":1.0129999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.75700000000000001,"15min":0.60799999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77000000000000002,"5min":17.143999999999998,"15min":17.143999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.64400000000000002,"15min":0.59299999999999997},"min":{"1min":0.45800000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85799999999999998,"5min":1.76,"15min":1.76},"last":0.57399999999999995}]},{"osd":3,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.624,"15min":0.56899999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.376},"max":{"1min":0.78000000000000003,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.624,"15min":0.58199999999999996},"min":{"1min":0.44500000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79300000000000004,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.53100000000000003}]},{"osd":4,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.30199999999999999},"max":{"1min":0.73599999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.50900000000000001,"15min":0.54300000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84899999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.61499999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.80800000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56299999999999994,"15min":0.59599999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.48599999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59299999999999997,"15min":0.60999999999999999},"min":{"1min":0.40999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.87,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55700000000000005,"15min":0.58199999999999996},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.36499999999999999},"max":{"1min":0.82899999999999996,"5min":1.038,"15min":1.038},"last":0.58899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":805,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194507,"statfs":{"total":200000000,"available":199175581,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39800000000000002,"15min":0.378},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41799999999999998,"15min":0.40000000000000002},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.50600000000000001,"5min":1.121,"15min":1.121},"last":0.42099999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40799999999999997,"15min":0.373},"min":{"1min":0.249,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":1.0940000000000001,"5min":1.25,"15min":1.25},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.44900000000000001,"15min":0.43099999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29899999999999999},"max":{"1min":0.498,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.39000000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.48499999999999999,"15min":0.46899999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.32100000000000001},"max":{"1min":0.56699999999999995,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.433},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.46600000000000003,"15min":0.45500000000000002},"min":{"1min":0.34899999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.56100000000000005,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.35699999999999998}]},{"osd":4,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.66600000000000004,"15min":0.63600000000000001},"min":{"1min":0.48299999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.85699999999999998,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66600000000000004,"15min":0.64000000000000001},"min":{"1min":0.45000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91600000000000004,"5min":1.026,"15min":1.026},"last":0.54300000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69099999999999995,"15min":0.64200000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":0.93400000000000005,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69699999999999995,"15min":0.67600000000000005},"min":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96199999999999997,"5min":3.181,"15min":3.181},"last":0.66900000000000004}]},{"osd":6,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.70599999999999996,"15min":0.69499999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88500000000000001,"5min":0.96399999999999997,"15min":1.024},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72099999999999997,"15min":0.68300000000000005},"min":{"1min":0.54000000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93600000000000005,"5min":1.869,"15min":1.869},"last":0.69299999999999995}]},{"osd":7,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73199999999999998,"15min":0.69099999999999995},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.47299999999999998},"max":{"1min":0.91400000000000003,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73099999999999998,"15min":0.69499999999999995},"min":{"1min":0.53000000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92000000000000004,"5min":1.708,"15min":1.708},"last":0.56100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1381,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193930,"statfs":{"total":200000000,"available":198585213,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.40600000000000003,"15min":0.39400000000000002},"min":{"1min":0.252,"5min":0.24399999999999999,"15min":0.23799999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.42899999999999999,"15min":0.41799999999999998},"min":{"1min":0.28899999999999998,"5min":0.27800000000000002,"15min":0.223},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.27800000000000002}]},{"osd":2,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.433,"15min":0.42199999999999999},"min":{"1min":0.31,"5min":0.28699999999999998,"15min":0.28199999999999997},"max":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.45700000000000002,"15min":0.438},"min":{"1min":0.312,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.32000000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.441,"15min":0.432},"min":{"1min":0.315,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.66100000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.495,"15min":0.47599999999999998},"min":{"1min":0.38800000000000001,"5min":0.33600000000000002,"15min":0.29099999999999998},"max":{"1min":0.67600000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.40899999999999997}]},{"osd":4,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66800000000000004,"15min":0.64400000000000002},"min":{"1min":0.505,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80900000000000005,"5min":0.997,"15min":0.997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.45800000000000002,"5min":0.44,"15min":0.41999999999999998},"max":{"1min":0.83399999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64900000000000002}]},{"osd":5,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68899999999999995,"15min":0.65800000000000003},"min":{"1min":0.51900000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.78800000000000003,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.54900000000000004,"5min":0.47799999999999998,"15min":0.44},"max":{"1min":0.84999999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.622}]},{"osd":6,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68300000000000005,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.42499999999999999},"max":{"1min":0.79800000000000004,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69899999999999995,"15min":0.68000000000000005},"min":{"1min":0.51600000000000001,"5min":0.45700000000000002,"15min":0.45000000000000001},"max":{"1min":0.90300000000000002,"5min":1.04,"15min":1.04},"last":0.59499999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.50800000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90200000000000002,"5min":1.014,"15min":1.014},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69699999999999995,"15min":0.69199999999999995},"min":{"1min":0.54500000000000004,"5min":0.498,"15min":0.498},"max":{"1min":0.81999999999999995,"5min":0.95799999999999996,"15min":0.97299999999999998},"last":0.60799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":805,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194506,"statfs":{"total":200000000,"available":199174993,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.41199999999999998},"min":{"1min":0.27300000000000002,"5min":0.23599999999999999,"15min":0.23000000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.46700000000000003,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.30499999999999999},"max":{"1min":0.68899999999999995,"5min":0.72799999999999998,"15min":6.2729999999999997},"last":0.45200000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.433,"15min":0.42299999999999999},"min":{"1min":0.33800000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.621,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.439,"15min":0.44400000000000001},"min":{"1min":0.33900000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.60399999999999998,"5min":0.65300000000000002,"15min":0.70099999999999996},"last":0.36199999999999999}]},{"osd":3,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.434,"15min":0.41799999999999998},"min":{"1min":0.32200000000000001,"5min":0.28899999999999998,"15min":0.27800000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49099999999999999,"15min":0.48599999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.505}]},{"osd":4,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68799999999999994,"15min":0.65900000000000003},"min":{"1min":0.499,"5min":0.46800000000000003,"15min":0.39300000000000002},"max":{"1min":0.90500000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.70599999999999996,"15min":0.78100000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95799999999999996,"5min":1.006,"15min":3.581},"last":0.79400000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65800000000000003,"15min":0.72699999999999998},"min":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92200000000000004,"5min":0.95899999999999996,"15min":3.5369999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.52200000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89100000000000001,"5min":0.92200000000000004,"15min":1.0529999999999999},"last":0.77700000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67200000000000004,"15min":0.66700000000000004},"min":{"1min":0.57499999999999996,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96499999999999997,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69599999999999995,"15min":0.68000000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.97099999999999997,"5min":1.0189999999999999,"15min":1.0209999999999999},"last":0.69599999999999995}]},{"osd":7,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69199999999999995,"15min":0.67900000000000005},"min":{"1min":0.58799999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97699999999999998,"5min":1.03,"15min":1.03},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70899999999999996,"15min":0.69299999999999995},"min":{"1min":0.55200000000000005,"5min":0.441,"15min":0.42699999999999999},"max":{"1min":0.99099999999999999,"5min":1.036,"15min":1.036},"last":0.67300000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":805,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194507,"statfs":{"total":200000000,"available":199175421,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.45500000000000002,"15min":0.38600000000000001},"min":{"1min":0.29699999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.67900000000000005,"5min":4.7380000000000004,"15min":4.7380000000000004},"last":0.309},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41899999999999998,"15min":0.40000000000000002},"min":{"1min":0.29399999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55500000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.378}]},{"osd":1,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.47499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28000000000000003,"5min":0.27400000000000002,"15min":0.22900000000000001},"max":{"1min":0.61599999999999999,"5min":4.5739999999999998,"15min":4.5739999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.53300000000000003,"15min":0.45200000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.98199999999999998,"5min":5.0529999999999999,"15min":5.0529999999999999},"last":0.35999999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.49199999999999999,"15min":0.443},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.28100000000000003},"max":{"1min":0.76700000000000002,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.55400000000000005,"15min":0.47699999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71799999999999997,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":0.49099999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69999999999999996,"15min":0.63800000000000001},"min":{"1min":0.45300000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91600000000000004,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.62},"min":{"1min":0.48199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86099999999999999,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.60899999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67600000000000005,"15min":0.626},"min":{"1min":0.438,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82399999999999995,"5min":1.409,"15min":1.409},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.78700000000000003,"15min":0.70399999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.89600000000000002,"5min":4.7960000000000003,"15min":4.7960000000000003},"last":0.625}]},{"osd":6,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.73899999999999999,"15min":0.67900000000000005},"min":{"1min":0.42799999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92700000000000005,"5min":5.0069999999999997,"15min":5.0069999999999997},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.69399999999999995,"15min":0.65900000000000003},"min":{"1min":0.47499999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.94199999999999995,"5min":1.103,"15min":1.103},"last":0.68899999999999995}]},{"osd":7,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.75700000000000001,"15min":0.66900000000000004},"min":{"1min":0.57599999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72699999999999998,"15min":0.68000000000000005},"min":{"1min":0.52800000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.88900000000000001,"5min":1.361,"15min":1.361},"last":0.67400000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":805,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194507,"statfs":{"total":200000000,"available":199175511,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.35599999999999998,"5min":0.34599999999999997,"15min":0.314},"max":{"1min":1.137,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66200000000000003,"15min":0.61399999999999999},"min":{"1min":0.496,"5min":0.372,"15min":0.372},"max":{"1min":1.016,"5min":7.0670000000000002,"15min":7.0670000000000002},"last":0.68999999999999995}]},{"osd":1,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.64200000000000002,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.35399999999999998},"max":{"1min":0.81000000000000005,"5min":5.1529999999999996,"15min":5.1529999999999996},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64400000000000002,"15min":0.60899999999999999},"min":{"1min":0.46400000000000002,"5min":0.40799999999999997,"15min":0.39000000000000001},"max":{"1min":1.054,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.60199999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.64200000000000002,"15min":0.61099999999999999},"min":{"1min":0.48399999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76600000000000001,"5min":5.0019999999999998,"15min":5.0019999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.63300000000000001,"15min":0.62},"min":{"1min":0.432,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2549999999999999,"5min":2.073,"15min":2.073},"last":0.55700000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.60499999999999998},"min":{"1min":0.45200000000000001,"5min":0.42099999999999999,"15min":0.36199999999999999},"max":{"1min":1.1899999999999999,"5min":6.7220000000000004,"15min":6.7220000000000004},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68999999999999995,"15min":0.66400000000000003},"min":{"1min":0.46100000000000002,"5min":0.44900000000000001,"15min":0.41399999999999998},"max":{"1min":0.88,"5min":6.5510000000000002,"15min":6.5510000000000002},"last":0.54000000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.34899999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.77500000000000002,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56599999999999995,"15min":0.51700000000000002},"min":{"1min":0.372,"5min":0.312,"15min":0.29999999999999999},"max":{"1min":0.68799999999999994,"5min":6.9269999999999996,"15min":6.9269999999999996},"last":0.443}]},{"osd":6,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56100000000000005,"15min":0.52200000000000002},"min":{"1min":0.40400000000000003,"5min":0.34999999999999998,"15min":0.32900000000000001},"max":{"1min":0.78600000000000003,"5min":5.2229999999999999,"15min":5.2229999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56200000000000006,"15min":0.54900000000000004},"min":{"1min":0.41799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82999999999999996,"5min":1.46,"15min":1.46},"last":0.375}]},{"osd":7,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61699999999999999,"15min":0.57499999999999996},"min":{"1min":0.434,"5min":0.34799999999999998,"15min":0.33600000000000002},"max":{"1min":0.78300000000000003,"5min":7.5339999999999998,"15min":7.5339999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.629,"15min":0.59399999999999997},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85299999999999998,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":0.42499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574940,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":805,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194507,"statfs":{"total":200000000,"available":199175469,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56399999999999995,"15min":0.58999999999999997},"min":{"1min":0.42099999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56699999999999995,"15min":0.57899999999999996},"min":{"1min":0.442,"5min":0.34499999999999997,"15min":0.27600000000000002},"max":{"1min":0.77100000000000002,"5min":0.78900000000000003,"15min":3.242},"last":0.40600000000000003}]},{"osd":1,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.622,"15min":0.61299999999999999},"min":{"1min":0.38500000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.3380000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61099999999999999,"15min":0.64300000000000002},"min":{"1min":0.35399999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78400000000000003,"5min":0.84399999999999997,"15min":0.95499999999999996},"last":0.61499999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58599999999999997,"15min":0.59699999999999998},"min":{"1min":0.40400000000000003,"5min":0.39100000000000001,"15min":0.372},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":3.04},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.48499999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80400000000000005,"5min":0.85899999999999999,"15min":0.89400000000000002},"last":0.44900000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60499999999999998,"15min":0.66800000000000004},"min":{"1min":0.40300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":4.032},"last":0.505},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65500000000000003,"15min":0.67700000000000005},"min":{"1min":0.48999999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.76100000000000001,"5min":0.86199999999999999,"15min":4.282},"last":0.52700000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50700000000000001,"15min":0.52200000000000002},"min":{"1min":0.35799999999999998,"5min":0.30599999999999999,"15min":0.26500000000000001},"max":{"1min":0.63600000000000001,"5min":0.81399999999999995,"15min":1.2869999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.495,"15min":0.55300000000000005},"min":{"1min":0.32800000000000001,"5min":0.318,"15min":0.30399999999999999},"max":{"1min":0.66900000000000004,"5min":0.79800000000000004,"15min":1.101},"last":0.56000000000000005}]},{"osd":6,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.497,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65200000000000002,"5min":0.73199999999999998,"15min":1.7969999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55300000000000005,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80100000000000005,"5min":0.85799999999999998,"15min":1.6739999999999999},"last":0.48399999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.38200000000000001,"5min":0.34799999999999998,"15min":0.32700000000000001},"max":{"1min":0.66800000000000004,"5min":0.77100000000000002,"15min":1.008},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59699999999999998,"15min":0.64900000000000002},"min":{"1min":0.436,"5min":0.35899999999999999,"15min":0.33700000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":2},"last":0.57599999999999996}]}]}],"pool_statfs":[]}} 2024-01-27T02:39:31.789 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:39:31.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.930+0000 7f75710f8700 1 Processor -- start 2024-01-27T02:39:31.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.930+0000 7f75710f8700 1 -- start start 2024-01-27T02:39:31.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.931+0000 7f75710f8700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f756c0b3040 0x7f756c164e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:31.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.931+0000 7f75710f8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f756c15ba30 con 0x7f756c157c00 2024-01-27T02:39:31.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.931+0000 7f75710f8700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f756c0d0390 con 0x7f756c0cfee0 2024-01-27T02:39:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.931+0000 7f75710f8700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f756c0d0670 con 0x7f756c0b3040 2024-01-27T02:39:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.931+0000 7f756ad9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f756c157c00 0x7f756c0cf7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48954/0 (socket says 172.21.15.7:48954) 2024-01-27T02:39:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.931+0000 7f756ad9d700 1 -- 172.21.15.7:0/4113357225 learned_addr learned my addr 172.21.15.7:0/4113357225 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.931+0000 7f756b59e700 1 --2- 172.21.15.7:0/4113357225 >> v2:172.21.15.188:3300/0 conn(0x7f756c0b3040 0x7f756c164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.931+0000 7f7569d9b700 1 -- 172.21.15.7:0/4113357225 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1560077922 0 0) 0x7f756c15ba30 con 0x7f756c157c00 2024-01-27T02:39:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.931+0000 7f7569d9b700 1 -- 172.21.15.7:0/4113357225 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7558003570 con 0x7f756c157c00 2024-01-27T02:39:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.931+0000 7f756b59e700 1 -- 172.21.15.7:0/4113357225 >> v1:172.21.15.188:6789/0 conn(0x7f756c0cfee0 legacy=0x7f756c0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.932+0000 7f756b59e700 1 -- 172.21.15.7:0/4113357225 >> v1:172.21.15.7:6789/0 conn(0x7f756c157c00 legacy=0x7f756c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.932+0000 7f756b59e700 1 -- 172.21.15.7:0/4113357225 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c0d0950 con 0x7f756c0b3040 2024-01-27T02:39:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.932+0000 7f756b59e700 1 --2- 172.21.15.7:0/4113357225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f756c0b3040 0x7f756c164e80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7564009e60 tx=0x7f7564009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b22d0a378d421e7f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.932+0000 7f7569d9b700 1 -- 172.21.15.7:0/4113357225 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756400e070 con 0x7f756c0b3040 2024-01-27T02:39:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.932+0000 7f7569d9b700 1 -- 172.21.15.7:0/4113357225 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756400bb60 con 0x7f756c0b3040 2024-01-27T02:39:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.932+0000 7f7569d9b700 1 -- 172.21.15.7:0/4113357225 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564012430 con 0x7f756c0b3040 2024-01-27T02:39:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.932+0000 7f75710f8700 1 -- 172.21.15.7:0/4113357225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f756c0b3040 msgr2=0x7f756c164e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:31.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.932+0000 7f75710f8700 1 --2- 172.21.15.7:0/4113357225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f756c0b3040 0x7f756c164e80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7564009e60 tx=0x7f7564009780 comp rx=0 tx=0).stop 2024-01-27T02:39:31.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.932+0000 7f75710f8700 1 -- 172.21.15.7:0/4113357225 shutdown_connections 2024-01-27T02:39:31.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.932+0000 7f75710f8700 1 --2- 172.21.15.7:0/4113357225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f756c0b3040 0x7f756c164e80 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:31.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.932+0000 7f75710f8700 1 -- 172.21.15.7:0/4113357225 >> 172.21.15.7:0/4113357225 conn(0x7f756c068450 msgr2=0x7f756c0709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:31.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.933+0000 7f75710f8700 1 -- 172.21.15.7:0/4113357225 shutdown_connections 2024-01-27T02:39:31.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.933+0000 7f75710f8700 1 -- 172.21.15.7:0/4113357225 wait complete. 2024-01-27T02:39:31.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.934+0000 7f75710f8700 1 Processor -- start 2024-01-27T02:39:31.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.934+0000 7f75710f8700 1 -- start start 2024-01-27T02:39:31.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.935+0000 7f75710f8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f756c0b3040 0x7f756c147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:31.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.935+0000 7f75710f8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f756c0cfee0 0x7f756c159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:31.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.935+0000 7f75710f8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f756c157c00 0x7f756c15a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:31.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.935+0000 7f75710f8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c0c7660 con 0x7f756c157c00 2024-01-27T02:39:31.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.935+0000 7f75710f8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c0c7940 con 0x7f756c0b3040 2024-01-27T02:39:31.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.935+0000 7f75710f8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756c0cb670 con 0x7f756c0cfee0 2024-01-27T02:39:31.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.935+0000 7f756b59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f756c157c00 0x7f756c15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:31.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.935+0000 7f756b59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f756c157c00 0x7f756c15a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34136/0 (socket says 172.21.15.7:34136) 2024-01-27T02:39:31.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.935+0000 7f756b59e700 1 -- 172.21.15.7:0/4220477440 learned_addr learned my addr 172.21.15.7:0/4220477440 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:31.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.935+0000 7f756ad9d700 1 --2- 172.21.15.7:0/4220477440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f756c0b3040 0x7f756c147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:31.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.935+0000 7f756a59c700 1 --2- 172.21.15.7:0/4220477440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f756c0cfee0 0x7f756c159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:31.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.935+0000 7f756b59e700 1 -- 172.21.15.7:0/4220477440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f756c0cfee0 msgr2=0x7f756c159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:31.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.935+0000 7f756b59e700 1 --2- 172.21.15.7:0/4220477440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f756c0cfee0 0x7f756c159af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:31.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.935+0000 7f756b59e700 1 -- 172.21.15.7:0/4220477440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f756c0b3040 msgr2=0x7f756c147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:31.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.935+0000 7f756b59e700 1 --2- 172.21.15.7:0/4220477440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f756c0b3040 0x7f756c147b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.935+0000 7f756b59e700 1 -- 172.21.15.7:0/4220477440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7564009a20 con 0x7f756c157c00 2024-01-27T02:39:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.935+0000 7f756b59e700 1 --2- 172.21.15.7:0/4220477440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f756c157c00 0x7f756c15a030 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f7564013000 tx=0x7f7564019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.935+0000 7f7562ffd700 1 -- 172.21.15.7:0/4220477440 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564027470 con 0x7f756c157c00 2024-01-27T02:39:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.936+0000 7f7562ffd700 1 -- 172.21.15.7:0/4220477440 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75640275f0 con 0x7f756c157c00 2024-01-27T02:39:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.936+0000 7f75710f8700 1 -- 172.21.15.7:0/4220477440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756c0c6bc0 con 0x7f756c157c00 2024-01-27T02:39:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.936+0000 7f7562ffd700 1 -- 172.21.15.7:0/4220477440 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75640279b0 con 0x7f756c157c00 2024-01-27T02:39:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.936+0000 7f75710f8700 1 -- 172.21.15.7:0/4220477440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756c0c7180 con 0x7f756c157c00 2024-01-27T02:39:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.937+0000 7f75710f8700 1 -- 172.21.15.7:0/4220477440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f756c0c7ab0 con 0x7f756c157c00 2024-01-27T02:39:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.937+0000 7f7562ffd700 1 -- 172.21.15.7:0/4220477440 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7564024070 con 0x7f756c157c00 2024-01-27T02:39:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.937+0000 7f7562ffd700 1 --2- 172.21.15.7:0/4220477440 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f754c066f30 0x7f754c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.938+0000 7f7562ffd700 1 -- 172.21.15.7:0/4220477440 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7564086f70 con 0x7f756c157c00 2024-01-27T02:39:31.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.938+0000 7f756ad9d700 1 --2- 172.21.15.7:0/4220477440 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f754c066f30 0x7f754c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:31.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.938+0000 7f756ad9d700 1 --2- 172.21.15.7:0/4220477440 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f754c066f30 0x7f754c0693e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f75540058d0 tx=0x7f7554009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:31.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:31.942+0000 7f7562ffd700 1 -- 172.21.15.7:0/4220477440 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75640b85a0 con 0x7f756c157c00 2024-01-27T02:39:32.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.144+0000 7f75710f8700 1 -- 172.21.15.7:0/4220477440 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f756c04c710 con 0x7f754c066f30 2024-01-27T02:39:32.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.152+0000 7f7562ffd700 1 -- 172.21.15.7:0/4220477440 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285538 (secure 0 0 0) 0x7f756c04c710 con 0x7f754c066f30 2024-01-27T02:39:32.157 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:32.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.156+0000 7f75710f8700 1 -- 172.21.15.7:0/4220477440 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f754c066f30 msgr2=0x7f754c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:32.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.156+0000 7f75710f8700 1 --2- 172.21.15.7:0/4220477440 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f754c066f30 0x7f754c0693e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f75540058d0 tx=0x7f7554009040 comp rx=0 tx=0).stop 2024-01-27T02:39:32.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.156+0000 7f75710f8700 1 -- 172.21.15.7:0/4220477440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f756c157c00 msgr2=0x7f756c15a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:32.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.156+0000 7f75710f8700 1 --2- 172.21.15.7:0/4220477440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f756c157c00 0x7f756c15a030 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f7564013000 tx=0x7f7564019000 comp rx=0 tx=0).stop 2024-01-27T02:39:32.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.156+0000 7f75710f8700 1 -- 172.21.15.7:0/4220477440 shutdown_connections 2024-01-27T02:39:32.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.156+0000 7f75710f8700 1 --2- 172.21.15.7:0/4220477440 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f754c066f30 0x7f754c0693e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:32.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.156+0000 7f75710f8700 1 --2- 172.21.15.7:0/4220477440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f756c157c00 0x7f756c15a030 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:32.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.156+0000 7f75710f8700 1 --2- 172.21.15.7:0/4220477440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f756c0b3040 0x7f756c147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:32.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.156+0000 7f75710f8700 1 --2- 172.21.15.7:0/4220477440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f756c0cfee0 0x7f756c159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:32.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.156+0000 7f75710f8700 1 -- 172.21.15.7:0/4220477440 >> 172.21.15.7:0/4220477440 conn(0x7f756c068450 msgr2=0x7f756c156cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:32.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.156+0000 7f75710f8700 1 -- 172.21.15.7:0/4220477440 shutdown_connections 2024-01-27T02:39:32.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.156+0000 7f75710f8700 1 -- 172.21.15.7:0/4220477440 wait complete. 2024-01-27T02:39:32.160 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:39:32.170 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-01-27T02:39:31.680812+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7598,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554901,"statfs":{"total":1600000000,"available":1592219300,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.791135"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.898603+0000","last_change":"2024-01-27T02:39:22.898429+0000","last_active":"2024-01-27T02:39:22.898603+0000","last_peered":"2024-01-27T02:39:22.898603+0000","last_clean":"2024-01-27T02:39:22.898603+0000","last_became_active":"2024-01-27T02:39:22.897112+0000","last_became_peered":"2024-01-27T02:39:22.897112+0000","last_unstale":"2024-01-27T02:39:22.898603+0000","last_undegraded":"2024-01-27T02:39:22.898603+0000","last_fullsized":"2024-01-27T02:39:22.898603+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:00:12.452602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902737+0000","last_change":"2024-01-27T02:39:22.902650+0000","last_active":"2024-01-27T02:39:22.902737+0000","last_peered":"2024-01-27T02:39:22.902737+0000","last_clean":"2024-01-27T02:39:22.902737+0000","last_became_active":"2024-01-27T02:39:22.902344+0000","last_became_peered":"2024-01-27T02:39:22.902344+0000","last_unstale":"2024-01-27T02:39:22.902737+0000","last_undegraded":"2024-01-27T02:39:22.902737+0000","last_fullsized":"2024-01-27T02:39:22.902737+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:02:57.717352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.899868+0000","last_change":"2024-01-27T02:39:22.899799+0000","last_active":"2024-01-27T02:39:22.899868+0000","last_peered":"2024-01-27T02:39:22.899868+0000","last_clean":"2024-01-27T02:39:22.899868+0000","last_became_active":"2024-01-27T02:39:22.899535+0000","last_became_peered":"2024-01-27T02:39:22.899535+0000","last_unstale":"2024-01-27T02:39:22.899868+0000","last_undegraded":"2024-01-27T02:39:22.899868+0000","last_fullsized":"2024-01-27T02:39:22.899868+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:27:21.148523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.905689+0000","last_change":"2024-01-27T02:39:22.904667+0000","last_active":"2024-01-27T02:39:22.905689+0000","last_peered":"2024-01-27T02:39:22.905689+0000","last_clean":"2024-01-27T02:39:22.905689+0000","last_became_active":"2024-01-27T02:39:22.900570+0000","last_became_peered":"2024-01-27T02:39:22.900570+0000","last_unstale":"2024-01-27T02:39:22.905689+0000","last_undegraded":"2024-01-27T02:39:22.905689+0000","last_fullsized":"2024-01-27T02:39:22.905689+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:17:26.997560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.897810+0000","last_change":"2024-01-27T02:39:22.897296+0000","last_active":"2024-01-27T02:39:22.897810+0000","last_peered":"2024-01-27T02:39:22.897810+0000","last_clean":"2024-01-27T02:39:22.897810+0000","last_became_active":"2024-01-27T02:39:22.897051+0000","last_became_peered":"2024-01-27T02:39:22.897051+0000","last_unstale":"2024-01-27T02:39:22.897810+0000","last_undegraded":"2024-01-27T02:39:22.897810+0000","last_fullsized":"2024-01-27T02:39:22.897810+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:33:46.694312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.900968+0000","last_change":"2024-01-27T02:39:22.900656+0000","last_active":"2024-01-27T02:39:22.900968+0000","last_peered":"2024-01-27T02:39:22.900968+0000","last_clean":"2024-01-27T02:39:22.900968+0000","last_became_active":"2024-01-27T02:39:22.899965+0000","last_became_peered":"2024-01-27T02:39:22.899965+0000","last_unstale":"2024-01-27T02:39:22.900968+0000","last_undegraded":"2024-01-27T02:39:22.900968+0000","last_fullsized":"2024-01-27T02:39:22.900968+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:09:14.280979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.897983+0000","last_change":"2024-01-27T02:39:22.897809+0000","last_active":"2024-01-27T02:39:22.897983+0000","last_peered":"2024-01-27T02:39:22.897983+0000","last_clean":"2024-01-27T02:39:22.897983+0000","last_became_active":"2024-01-27T02:39:22.897437+0000","last_became_peered":"2024-01-27T02:39:22.897437+0000","last_unstale":"2024-01-27T02:39:22.897983+0000","last_undegraded":"2024-01-27T02:39:22.897983+0000","last_fullsized":"2024-01-27T02:39:22.897983+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:02:31.201590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.904534+0000","last_change":"2024-01-27T02:39:22.903395+0000","last_active":"2024-01-27T02:39:22.904534+0000","last_peered":"2024-01-27T02:39:22.904534+0000","last_clean":"2024-01-27T02:39:22.904534+0000","last_became_active":"2024-01-27T02:39:22.903023+0000","last_became_peered":"2024-01-27T02:39:22.903023+0000","last_unstale":"2024-01-27T02:39:22.904534+0000","last_undegraded":"2024-01-27T02:39:22.904534+0000","last_fullsized":"2024-01-27T02:39:22.904534+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:08:06.794664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910976+0000","last_change":"2024-01-27T02:39:22.906721+0000","last_active":"2024-01-27T02:39:22.910976+0000","last_peered":"2024-01-27T02:39:22.910976+0000","last_clean":"2024-01-27T02:39:22.910976+0000","last_became_active":"2024-01-27T02:39:22.906469+0000","last_became_peered":"2024-01-27T02:39:22.906469+0000","last_unstale":"2024-01-27T02:39:22.910976+0000","last_undegraded":"2024-01-27T02:39:22.910976+0000","last_fullsized":"2024-01-27T02:39:22.910976+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:04:43.117550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.897833+0000","last_change":"2024-01-27T02:39:22.897761+0000","last_active":"2024-01-27T02:39:22.897833+0000","last_peered":"2024-01-27T02:39:22.897833+0000","last_clean":"2024-01-27T02:39:22.897833+0000","last_became_active":"2024-01-27T02:39:22.897518+0000","last_became_peered":"2024-01-27T02:39:22.897518+0000","last_unstale":"2024-01-27T02:39:22.897833+0000","last_undegraded":"2024-01-27T02:39:22.897833+0000","last_fullsized":"2024-01-27T02:39:22.897833+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:47:36.010163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.900063+0000","last_change":"2024-01-27T02:39:22.899989+0000","last_active":"2024-01-27T02:39:22.900063+0000","last_peered":"2024-01-27T02:39:22.900063+0000","last_clean":"2024-01-27T02:39:22.900063+0000","last_became_active":"2024-01-27T02:39:22.899715+0000","last_became_peered":"2024-01-27T02:39:22.899715+0000","last_unstale":"2024-01-27T02:39:22.900063+0000","last_undegraded":"2024-01-27T02:39:22.900063+0000","last_fullsized":"2024-01-27T02:39:22.900063+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:22:14.092868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.898551+0000","last_change":"2024-01-27T02:39:22.898312+0000","last_active":"2024-01-27T02:39:22.898551+0000","last_peered":"2024-01-27T02:39:22.898551+0000","last_clean":"2024-01-27T02:39:22.898551+0000","last_became_active":"2024-01-27T02:39:22.898071+0000","last_became_peered":"2024-01-27T02:39:22.898071+0000","last_unstale":"2024-01-27T02:39:22.898551+0000","last_undegraded":"2024-01-27T02:39:22.898551+0000","last_fullsized":"2024-01-27T02:39:22.898551+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:17.789817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.904646+0000","last_change":"2024-01-27T02:39:22.901995+0000","last_active":"2024-01-27T02:39:22.904646+0000","last_peered":"2024-01-27T02:39:22.904646+0000","last_clean":"2024-01-27T02:39:22.904646+0000","last_became_active":"2024-01-27T02:39:22.901761+0000","last_became_peered":"2024-01-27T02:39:22.901761+0000","last_unstale":"2024-01-27T02:39:22.904646+0000","last_undegraded":"2024-01-27T02:39:22.904646+0000","last_fullsized":"2024-01-27T02:39:22.904646+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:50:35.407782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902718+0000","last_change":"2024-01-27T02:39:22.902243+0000","last_active":"2024-01-27T02:39:22.902718+0000","last_peered":"2024-01-27T02:39:22.902718+0000","last_clean":"2024-01-27T02:39:22.902718+0000","last_became_active":"2024-01-27T02:39:22.901971+0000","last_became_peered":"2024-01-27T02:39:22.901971+0000","last_unstale":"2024-01-27T02:39:22.902718+0000","last_undegraded":"2024-01-27T02:39:22.902718+0000","last_fullsized":"2024-01-27T02:39:22.902718+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:35:38.537841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.907395+0000","last_change":"2024-01-27T02:39:22.906123+0000","last_active":"2024-01-27T02:39:22.907395+0000","last_peered":"2024-01-27T02:39:22.907395+0000","last_clean":"2024-01-27T02:39:22.907395+0000","last_became_active":"2024-01-27T02:39:22.905311+0000","last_became_peered":"2024-01-27T02:39:22.905311+0000","last_unstale":"2024-01-27T02:39:22.907395+0000","last_undegraded":"2024-01-27T02:39:22.907395+0000","last_fullsized":"2024-01-27T02:39:22.907395+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:26:42.911820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.907825+0000","last_change":"2024-01-27T02:39:22.907096+0000","last_active":"2024-01-27T02:39:22.907825+0000","last_peered":"2024-01-27T02:39:22.907825+0000","last_clean":"2024-01-27T02:39:22.907825+0000","last_became_active":"2024-01-27T02:39:22.906214+0000","last_became_peered":"2024-01-27T02:39:22.906214+0000","last_unstale":"2024-01-27T02:39:22.907825+0000","last_undegraded":"2024-01-27T02:39:22.907825+0000","last_fullsized":"2024-01-27T02:39:22.907825+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:35:39.970169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.904556+0000","last_change":"2024-01-27T02:39:22.904462+0000","last_active":"2024-01-27T02:39:22.904556+0000","last_peered":"2024-01-27T02:39:22.904556+0000","last_clean":"2024-01-27T02:39:22.904556+0000","last_became_active":"2024-01-27T02:39:22.904033+0000","last_became_peered":"2024-01-27T02:39:22.904033+0000","last_unstale":"2024-01-27T02:39:22.904556+0000","last_undegraded":"2024-01-27T02:39:22.904556+0000","last_fullsized":"2024-01-27T02:39:22.904556+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:59:03.340181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902308+0000","last_change":"2024-01-27T02:39:22.901938+0000","last_active":"2024-01-27T02:39:22.902308+0000","last_peered":"2024-01-27T02:39:22.902308+0000","last_clean":"2024-01-27T02:39:22.902308+0000","last_became_active":"2024-01-27T02:39:22.897160+0000","last_became_peered":"2024-01-27T02:39:22.897160+0000","last_unstale":"2024-01-27T02:39:22.902308+0000","last_undegraded":"2024-01-27T02:39:22.902308+0000","last_fullsized":"2024-01-27T02:39:22.902308+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:04:33.028169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.898697+0000","last_change":"2024-01-27T02:39:22.897541+0000","last_active":"2024-01-27T02:39:22.898697+0000","last_peered":"2024-01-27T02:39:22.898697+0000","last_clean":"2024-01-27T02:39:22.898697+0000","last_became_active":"2024-01-27T02:39:22.897251+0000","last_became_peered":"2024-01-27T02:39:22.897251+0000","last_unstale":"2024-01-27T02:39:22.898697+0000","last_undegraded":"2024-01-27T02:39:22.898697+0000","last_fullsized":"2024-01-27T02:39:22.898697+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:34:23.208150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902274+0000","last_change":"2024-01-27T02:39:22.899543+0000","last_active":"2024-01-27T02:39:22.902274+0000","last_peered":"2024-01-27T02:39:22.902274+0000","last_clean":"2024-01-27T02:39:22.902274+0000","last_became_active":"2024-01-27T02:39:22.899256+0000","last_became_peered":"2024-01-27T02:39:22.899256+0000","last_unstale":"2024-01-27T02:39:22.902274+0000","last_undegraded":"2024-01-27T02:39:22.902274+0000","last_fullsized":"2024-01-27T02:39:22.902274+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:14:55.893637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.898760+0000","last_change":"2024-01-27T02:39:22.898466+0000","last_active":"2024-01-27T02:39:22.898760+0000","last_peered":"2024-01-27T02:39:22.898760+0000","last_clean":"2024-01-27T02:39:22.898760+0000","last_became_active":"2024-01-27T02:39:22.897999+0000","last_became_peered":"2024-01-27T02:39:22.897999+0000","last_unstale":"2024-01-27T02:39:22.898760+0000","last_undegraded":"2024-01-27T02:39:22.898760+0000","last_fullsized":"2024-01-27T02:39:22.898760+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:39:21.733098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.904084+0000","last_change":"2024-01-27T02:39:22.904004+0000","last_active":"2024-01-27T02:39:22.904084+0000","last_peered":"2024-01-27T02:39:22.904084+0000","last_clean":"2024-01-27T02:39:22.904084+0000","last_became_active":"2024-01-27T02:39:22.903705+0000","last_became_peered":"2024-01-27T02:39:22.903705+0000","last_unstale":"2024-01-27T02:39:22.904084+0000","last_undegraded":"2024-01-27T02:39:22.904084+0000","last_fullsized":"2024-01-27T02:39:22.904084+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:13:03.822990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902780+0000","last_change":"2024-01-27T02:39:22.902663+0000","last_active":"2024-01-27T02:39:22.902780+0000","last_peered":"2024-01-27T02:39:22.902780+0000","last_clean":"2024-01-27T02:39:22.902780+0000","last_became_active":"2024-01-27T02:39:22.902445+0000","last_became_peered":"2024-01-27T02:39:22.902445+0000","last_unstale":"2024-01-27T02:39:22.902780+0000","last_undegraded":"2024-01-27T02:39:22.902780+0000","last_fullsized":"2024-01-27T02:39:22.902780+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:09:14.280979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.905378+0000","last_change":"2024-01-27T02:39:22.905287+0000","last_active":"2024-01-27T02:39:22.905378+0000","last_peered":"2024-01-27T02:39:22.905378+0000","last_clean":"2024-01-27T02:39:22.905378+0000","last_became_active":"2024-01-27T02:39:22.905095+0000","last_became_peered":"2024-01-27T02:39:22.905095+0000","last_unstale":"2024-01-27T02:39:22.905378+0000","last_undegraded":"2024-01-27T02:39:22.905378+0000","last_fullsized":"2024-01-27T02:39:22.905378+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:35:11.141652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.908684+0000","last_change":"2024-01-27T02:39:22.907502+0000","last_active":"2024-01-27T02:39:22.908684+0000","last_peered":"2024-01-27T02:39:22.908684+0000","last_clean":"2024-01-27T02:39:22.908684+0000","last_became_active":"2024-01-27T02:39:22.907247+0000","last_became_peered":"2024-01-27T02:39:22.907247+0000","last_unstale":"2024-01-27T02:39:22.908684+0000","last_undegraded":"2024-01-27T02:39:22.908684+0000","last_fullsized":"2024-01-27T02:39:22.908684+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:08:14.937701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.907847+0000","last_change":"2024-01-27T02:39:22.907272+0000","last_active":"2024-01-27T02:39:22.907847+0000","last_peered":"2024-01-27T02:39:22.907847+0000","last_clean":"2024-01-27T02:39:22.907847+0000","last_became_active":"2024-01-27T02:39:22.906623+0000","last_became_peered":"2024-01-27T02:39:22.906623+0000","last_unstale":"2024-01-27T02:39:22.907847+0000","last_undegraded":"2024-01-27T02:39:22.907847+0000","last_fullsized":"2024-01-27T02:39:22.907847+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:39:21.733098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.908642+0000","last_change":"2024-01-27T02:39:22.908551+0000","last_active":"2024-01-27T02:39:22.908642+0000","last_peered":"2024-01-27T02:39:22.908642+0000","last_clean":"2024-01-27T02:39:22.908642+0000","last_became_active":"2024-01-27T02:39:22.907980+0000","last_became_peered":"2024-01-27T02:39:22.907980+0000","last_unstale":"2024-01-27T02:39:22.908642+0000","last_undegraded":"2024-01-27T02:39:22.908642+0000","last_fullsized":"2024-01-27T02:39:22.908642+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:50.975864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.903922+0000","last_change":"2024-01-27T02:39:22.903237+0000","last_active":"2024-01-27T02:39:22.903922+0000","last_peered":"2024-01-27T02:39:22.903922+0000","last_clean":"2024-01-27T02:39:22.903922+0000","last_became_active":"2024-01-27T02:39:22.902973+0000","last_became_peered":"2024-01-27T02:39:22.902973+0000","last_unstale":"2024-01-27T02:39:22.903922+0000","last_undegraded":"2024-01-27T02:39:22.903922+0000","last_fullsized":"2024-01-27T02:39:22.903922+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:35:38.537841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.900393+0000","last_change":"2024-01-27T02:39:22.900300+0000","last_active":"2024-01-27T02:39:22.900393+0000","last_peered":"2024-01-27T02:39:22.900393+0000","last_clean":"2024-01-27T02:39:22.900393+0000","last_became_active":"2024-01-27T02:39:22.899932+0000","last_became_peered":"2024-01-27T02:39:22.899932+0000","last_unstale":"2024-01-27T02:39:22.900393+0000","last_undegraded":"2024-01-27T02:39:22.900393+0000","last_fullsized":"2024-01-27T02:39:22.900393+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:51:22.997295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.907415+0000","last_change":"2024-01-27T02:39:22.906323+0000","last_active":"2024-01-27T02:39:22.907415+0000","last_peered":"2024-01-27T02:39:22.907415+0000","last_clean":"2024-01-27T02:39:22.907415+0000","last_became_active":"2024-01-27T02:39:22.905609+0000","last_became_peered":"2024-01-27T02:39:22.905609+0000","last_unstale":"2024-01-27T02:39:22.907415+0000","last_undegraded":"2024-01-27T02:39:22.907415+0000","last_fullsized":"2024-01-27T02:39:22.907415+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:52:33.405152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902602+0000","last_change":"2024-01-27T02:39:22.902281+0000","last_active":"2024-01-27T02:39:22.902602+0000","last_peered":"2024-01-27T02:39:22.902602+0000","last_clean":"2024-01-27T02:39:22.902602+0000","last_became_active":"2024-01-27T02:39:22.901653+0000","last_became_peered":"2024-01-27T02:39:22.901653+0000","last_unstale":"2024-01-27T02:39:22.902602+0000","last_undegraded":"2024-01-27T02:39:22.902602+0000","last_fullsized":"2024-01-27T02:39:22.902602+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:57:54.178723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.897739+0000","last_change":"2024-01-27T02:39:22.897658+0000","last_active":"2024-01-27T02:39:22.897739+0000","last_peered":"2024-01-27T02:39:22.897739+0000","last_clean":"2024-01-27T02:39:22.897739+0000","last_became_active":"2024-01-27T02:39:22.897115+0000","last_became_peered":"2024-01-27T02:39:22.897115+0000","last_unstale":"2024-01-27T02:39:22.897739+0000","last_undegraded":"2024-01-27T02:39:22.897739+0000","last_fullsized":"2024-01-27T02:39:22.897739+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:11:12.572257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.904065+0000","last_change":"2024-01-27T02:39:22.903978+0000","last_active":"2024-01-27T02:39:22.904065+0000","last_peered":"2024-01-27T02:39:22.904065+0000","last_clean":"2024-01-27T02:39:22.904065+0000","last_became_active":"2024-01-27T02:39:22.903705+0000","last_became_peered":"2024-01-27T02:39:22.903705+0000","last_unstale":"2024-01-27T02:39:22.904065+0000","last_undegraded":"2024-01-27T02:39:22.904065+0000","last_fullsized":"2024-01-27T02:39:22.904065+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:36:32.750631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"148'4","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.913408+0000","last_change":"2024-01-27T02:39:21.897428+0000","last_active":"2024-01-27T02:39:22.913408+0000","last_peered":"2024-01-27T02:39:22.913408+0000","last_clean":"2024-01-27T02:39:22.913408+0000","last_became_active":"2024-01-27T02:39:21.897179+0000","last_became_peered":"2024-01-27T02:39:21.897179+0000","last_unstale":"2024-01-27T02:39:22.913408+0000","last_undegraded":"2024-01-27T02:39:22.913408+0000","last_fullsized":"2024-01-27T02:39:22.913408+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:47.650928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.911163+0000","last_change":"2024-01-27T02:39:22.907668+0000","last_active":"2024-01-27T02:39:22.911163+0000","last_peered":"2024-01-27T02:39:22.911163+0000","last_clean":"2024-01-27T02:39:22.911163+0000","last_became_active":"2024-01-27T02:39:22.907423+0000","last_became_peered":"2024-01-27T02:39:22.907423+0000","last_unstale":"2024-01-27T02:39:22.911163+0000","last_undegraded":"2024-01-27T02:39:22.911163+0000","last_fullsized":"2024-01-27T02:39:22.911163+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:48.653148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"148'1","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.909191+0000","last_change":"2024-01-27T02:39:21.895025+0000","last_active":"2024-01-27T02:39:22.909191+0000","last_peered":"2024-01-27T02:39:22.909191+0000","last_clean":"2024-01-27T02:39:22.909191+0000","last_became_active":"2024-01-27T02:39:21.894753+0000","last_became_peered":"2024-01-27T02:39:21.894753+0000","last_unstale":"2024-01-27T02:39:22.909191+0000","last_undegraded":"2024-01-27T02:39:22.909191+0000","last_fullsized":"2024-01-27T02:39:22.909191+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:23:12.301215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.898741+0000","last_change":"2024-01-27T02:39:22.898310+0000","last_active":"2024-01-27T02:39:22.898741+0000","last_peered":"2024-01-27T02:39:22.898741+0000","last_clean":"2024-01-27T02:39:22.898741+0000","last_became_active":"2024-01-27T02:39:22.897673+0000","last_became_peered":"2024-01-27T02:39:22.897673+0000","last_unstale":"2024-01-27T02:39:22.898741+0000","last_undegraded":"2024-01-27T02:39:22.898741+0000","last_fullsized":"2024-01-27T02:39:22.898741+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:15:15.541057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910286+0000","last_change":"2024-01-27T02:39:21.894906+0000","last_active":"2024-01-27T02:39:22.910286+0000","last_peered":"2024-01-27T02:39:22.910286+0000","last_clean":"2024-01-27T02:39:22.910286+0000","last_became_active":"2024-01-27T02:39:21.894564+0000","last_became_peered":"2024-01-27T02:39:21.894564+0000","last_unstale":"2024-01-27T02:39:22.910286+0000","last_undegraded":"2024-01-27T02:39:22.910286+0000","last_fullsized":"2024-01-27T02:39:22.910286+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:36:31.748411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910835+0000","last_change":"2024-01-27T02:39:22.909673+0000","last_active":"2024-01-27T02:39:22.910835+0000","last_peered":"2024-01-27T02:39:22.910835+0000","last_clean":"2024-01-27T02:39:22.910835+0000","last_became_active":"2024-01-27T02:39:22.908815+0000","last_became_peered":"2024-01-27T02:39:22.908815+0000","last_unstale":"2024-01-27T02:39:22.910835+0000","last_undegraded":"2024-01-27T02:39:22.910835+0000","last_fullsized":"2024-01-27T02:39:22.910835+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:02:57.570709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.912880+0000","last_change":"2024-01-27T02:39:21.900974+0000","last_active":"2024-01-27T02:39:22.912880+0000","last_peered":"2024-01-27T02:39:22.912880+0000","last_clean":"2024-01-27T02:39:22.912880+0000","last_became_active":"2024-01-27T02:39:21.900576+0000","last_became_peered":"2024-01-27T02:39:21.900576+0000","last_unstale":"2024-01-27T02:39:22.912880+0000","last_undegraded":"2024-01-27T02:39:22.912880+0000","last_fullsized":"2024-01-27T02:39:22.912880+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:10:31.872654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902332+0000","last_change":"2024-01-27T02:39:22.902089+0000","last_active":"2024-01-27T02:39:22.902332+0000","last_peered":"2024-01-27T02:39:22.902332+0000","last_clean":"2024-01-27T02:39:22.902332+0000","last_became_active":"2024-01-27T02:39:22.897524+0000","last_became_peered":"2024-01-27T02:39:22.897524+0000","last_unstale":"2024-01-27T02:39:22.902332+0000","last_undegraded":"2024-01-27T02:39:22.902332+0000","last_fullsized":"2024-01-27T02:39:22.902332+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:06:41.763215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"148'1","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.912090+0000","last_change":"2024-01-27T02:39:21.898367+0000","last_active":"2024-01-27T02:39:22.912090+0000","last_peered":"2024-01-27T02:39:22.912090+0000","last_clean":"2024-01-27T02:39:22.912090+0000","last_became_active":"2024-01-27T02:39:21.898114+0000","last_became_peered":"2024-01-27T02:39:21.898114+0000","last_unstale":"2024-01-27T02:39:22.912090+0000","last_undegraded":"2024-01-27T02:39:22.912090+0000","last_fullsized":"2024-01-27T02:39:22.912090+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:35:46.505332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.904385+0000","last_change":"2024-01-27T02:39:22.904273+0000","last_active":"2024-01-27T02:39:22.904385+0000","last_peered":"2024-01-27T02:39:22.904385+0000","last_clean":"2024-01-27T02:39:22.904385+0000","last_became_active":"2024-01-27T02:39:22.903665+0000","last_became_peered":"2024-01-27T02:39:22.903665+0000","last_unstale":"2024-01-27T02:39:22.904385+0000","last_undegraded":"2024-01-27T02:39:22.904385+0000","last_fullsized":"2024-01-27T02:39:22.904385+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:40:33.462225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"148'1","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.909959+0000","last_change":"2024-01-27T02:39:21.899252+0000","last_active":"2024-01-27T02:39:22.909959+0000","last_peered":"2024-01-27T02:39:22.909959+0000","last_clean":"2024-01-27T02:39:22.909959+0000","last_became_active":"2024-01-27T02:39:21.898982+0000","last_became_peered":"2024-01-27T02:39:21.898982+0000","last_unstale":"2024-01-27T02:39:22.909959+0000","last_undegraded":"2024-01-27T02:39:22.909959+0000","last_fullsized":"2024-01-27T02:39:22.909959+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:11:11.570037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910749+0000","last_change":"2024-01-27T02:39:22.906183+0000","last_active":"2024-01-27T02:39:22.910749+0000","last_peered":"2024-01-27T02:39:22.910749+0000","last_clean":"2024-01-27T02:39:22.910749+0000","last_became_active":"2024-01-27T02:39:22.904977+0000","last_became_peered":"2024-01-27T02:39:22.904977+0000","last_unstale":"2024-01-27T02:39:22.910749+0000","last_undegraded":"2024-01-27T02:39:22.910749+0000","last_fullsized":"2024-01-27T02:39:22.910749+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:18:02.494561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.891115+0000","last_change":"2024-01-27T02:39:21.898838+0000","last_active":"2024-01-27T02:39:22.891115+0000","last_peered":"2024-01-27T02:39:22.891115+0000","last_clean":"2024-01-27T02:39:22.891115+0000","last_became_active":"2024-01-27T02:39:21.898603+0000","last_became_peered":"2024-01-27T02:39:21.898603+0000","last_unstale":"2024-01-27T02:39:22.891115+0000","last_undegraded":"2024-01-27T02:39:22.891115+0000","last_fullsized":"2024-01-27T02:39:22.891115+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:09:29.890085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910984+0000","last_change":"2024-01-27T02:39:22.910891+0000","last_active":"2024-01-27T02:39:22.910984+0000","last_peered":"2024-01-27T02:39:22.910984+0000","last_clean":"2024-01-27T02:39:22.910984+0000","last_became_active":"2024-01-27T02:39:22.910593+0000","last_became_peered":"2024-01-27T02:39:22.910593+0000","last_unstale":"2024-01-27T02:39:22.910984+0000","last_undegraded":"2024-01-27T02:39:22.910984+0000","last_fullsized":"2024-01-27T02:39:22.910984+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:03:28.071762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.897217+0000","last_change":"2024-01-27T02:39:21.903093+0000","last_active":"2024-01-27T02:39:22.897217+0000","last_peered":"2024-01-27T02:39:22.897217+0000","last_clean":"2024-01-27T02:39:22.897217+0000","last_became_active":"2024-01-27T02:39:21.902782+0000","last_became_peered":"2024-01-27T02:39:21.902782+0000","last_unstale":"2024-01-27T02:39:22.897217+0000","last_undegraded":"2024-01-27T02:39:22.897217+0000","last_fullsized":"2024-01-27T02:39:22.897217+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:26:01.500470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.908830+0000","last_change":"2024-01-27T02:39:22.908736+0000","last_active":"2024-01-27T02:39:22.908830+0000","last_peered":"2024-01-27T02:39:22.908830+0000","last_clean":"2024-01-27T02:39:22.908830+0000","last_became_active":"2024-01-27T02:39:22.908321+0000","last_became_peered":"2024-01-27T02:39:22.908321+0000","last_unstale":"2024-01-27T02:39:22.908830+0000","last_undegraded":"2024-01-27T02:39:22.908830+0000","last_fullsized":"2024-01-27T02:39:22.908830+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:23:05.214270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.894643+0000","last_change":"2024-01-27T02:39:21.898565+0000","last_active":"2024-01-27T02:39:22.894643+0000","last_peered":"2024-01-27T02:39:22.894643+0000","last_clean":"2024-01-27T02:39:22.894643+0000","last_became_active":"2024-01-27T02:39:21.898337+0000","last_became_peered":"2024-01-27T02:39:21.898337+0000","last_unstale":"2024-01-27T02:39:22.894643+0000","last_undegraded":"2024-01-27T02:39:22.894643+0000","last_fullsized":"2024-01-27T02:39:22.894643+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:10:31.872654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.903942+0000","last_change":"2024-01-27T02:39:22.903289+0000","last_active":"2024-01-27T02:39:22.903942+0000","last_peered":"2024-01-27T02:39:22.903942+0000","last_clean":"2024-01-27T02:39:22.903942+0000","last_became_active":"2024-01-27T02:39:22.903021+0000","last_became_peered":"2024-01-27T02:39:22.903021+0000","last_unstale":"2024-01-27T02:39:22.903942+0000","last_undegraded":"2024-01-27T02:39:22.903942+0000","last_fullsized":"2024-01-27T02:39:22.903942+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:03:29.364153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.889969+0000","last_change":"2024-01-27T02:39:21.901433+0000","last_active":"2024-01-27T02:39:22.889969+0000","last_peered":"2024-01-27T02:39:22.889969+0000","last_clean":"2024-01-27T02:39:22.889969+0000","last_became_active":"2024-01-27T02:39:21.901145+0000","last_became_peered":"2024-01-27T02:39:21.901145+0000","last_unstale":"2024-01-27T02:39:22.889969+0000","last_undegraded":"2024-01-27T02:39:22.889969+0000","last_fullsized":"2024-01-27T02:39:22.889969+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:27:20.146303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.905809+0000","last_change":"2024-01-27T02:39:22.905679+0000","last_active":"2024-01-27T02:39:22.905809+0000","last_peered":"2024-01-27T02:39:22.905809+0000","last_clean":"2024-01-27T02:39:22.905809+0000","last_became_active":"2024-01-27T02:39:22.905262+0000","last_became_peered":"2024-01-27T02:39:22.905262+0000","last_unstale":"2024-01-27T02:39:22.905809+0000","last_undegraded":"2024-01-27T02:39:22.905809+0000","last_fullsized":"2024-01-27T02:39:22.905809+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:53:50.414456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"148'3","reported_seq":13,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910510+0000","last_change":"2024-01-27T02:39:21.894472+0000","last_active":"2024-01-27T02:39:22.910510+0000","last_peered":"2024-01-27T02:39:22.910510+0000","last_clean":"2024-01-27T02:39:22.910510+0000","last_became_active":"2024-01-27T02:39:21.893990+0000","last_became_peered":"2024-01-27T02:39:21.893990+0000","last_unstale":"2024-01-27T02:39:22.910510+0000","last_undegraded":"2024-01-27T02:39:22.910510+0000","last_fullsized":"2024-01-27T02:39:22.910510+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:01:18.616886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.897842+0000","last_change":"2024-01-27T02:39:22.897409+0000","last_active":"2024-01-27T02:39:22.897842+0000","last_peered":"2024-01-27T02:39:22.897842+0000","last_clean":"2024-01-27T02:39:22.897842+0000","last_became_active":"2024-01-27T02:39:22.897150+0000","last_became_peered":"2024-01-27T02:39:22.897150+0000","last_unstale":"2024-01-27T02:39:22.897842+0000","last_undegraded":"2024-01-27T02:39:22.897842+0000","last_fullsized":"2024-01-27T02:39:22.897842+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:35:39.970169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"148'6","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.914572+0000","last_change":"2024-01-27T02:39:21.901023+0000","last_active":"2024-01-27T02:39:22.914572+0000","last_peered":"2024-01-27T02:39:22.914572+0000","last_clean":"2024-01-27T02:39:22.914572+0000","last_became_active":"2024-01-27T02:39:21.899485+0000","last_became_peered":"2024-01-27T02:39:21.899485+0000","last_unstale":"2024-01-27T02:39:22.914572+0000","last_undegraded":"2024-01-27T02:39:22.914572+0000","last_fullsized":"2024-01-27T02:39:22.914572+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:03:28.361932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.907434+0000","last_change":"2024-01-27T02:39:22.906514+0000","last_active":"2024-01-27T02:39:22.907434+0000","last_peered":"2024-01-27T02:39:22.907434+0000","last_clean":"2024-01-27T02:39:22.907434+0000","last_became_active":"2024-01-27T02:39:22.905927+0000","last_became_peered":"2024-01-27T02:39:22.905927+0000","last_unstale":"2024-01-27T02:39:22.907434+0000","last_undegraded":"2024-01-27T02:39:22.907434+0000","last_fullsized":"2024-01-27T02:39:22.907434+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:30:29.146275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.892289+0000","last_change":"2024-01-27T02:39:21.901950+0000","last_active":"2024-01-27T02:39:22.892289+0000","last_peered":"2024-01-27T02:39:22.892289+0000","last_clean":"2024-01-27T02:39:22.892289+0000","last_became_active":"2024-01-27T02:39:21.901709+0000","last_became_peered":"2024-01-27T02:39:21.901709+0000","last_unstale":"2024-01-27T02:39:22.892289+0000","last_undegraded":"2024-01-27T02:39:22.892289+0000","last_fullsized":"2024-01-27T02:39:22.892289+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:27:55.432732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.900762+0000","last_change":"2024-01-27T02:39:22.900687+0000","last_active":"2024-01-27T02:39:22.900762+0000","last_peered":"2024-01-27T02:39:22.900762+0000","last_clean":"2024-01-27T02:39:22.900762+0000","last_became_active":"2024-01-27T02:39:22.900372+0000","last_became_peered":"2024-01-27T02:39:22.900372+0000","last_unstale":"2024-01-27T02:39:22.900762+0000","last_undegraded":"2024-01-27T02:39:22.900762+0000","last_fullsized":"2024-01-27T02:39:22.900762+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:19:27.934421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.894074+0000","last_change":"2024-01-27T02:39:21.901620+0000","last_active":"2024-01-27T02:39:22.894074+0000","last_peered":"2024-01-27T02:39:22.894074+0000","last_clean":"2024-01-27T02:39:22.894074+0000","last_became_active":"2024-01-27T02:39:21.901102+0000","last_became_peered":"2024-01-27T02:39:21.901102+0000","last_unstale":"2024-01-27T02:39:22.894074+0000","last_undegraded":"2024-01-27T02:39:22.894074+0000","last_fullsized":"2024-01-27T02:39:22.894074+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:52:32.402932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910954+0000","last_change":"2024-01-27T02:39:22.906855+0000","last_active":"2024-01-27T02:39:22.910954+0000","last_peered":"2024-01-27T02:39:22.910954+0000","last_clean":"2024-01-27T02:39:22.910954+0000","last_became_active":"2024-01-27T02:39:22.906480+0000","last_became_peered":"2024-01-27T02:39:22.906480+0000","last_unstale":"2024-01-27T02:39:22.910954+0000","last_undegraded":"2024-01-27T02:39:22.910954+0000","last_fullsized":"2024-01-27T02:39:22.910954+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:40:33.462225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"148'8","reported_seq":18,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.911228+0000","last_change":"2024-01-27T02:39:21.900650+0000","last_active":"2024-01-27T02:39:22.911228+0000","last_peered":"2024-01-27T02:39:22.911228+0000","last_clean":"2024-01-27T02:39:22.911228+0000","last_became_active":"2024-01-27T02:39:21.899844+0000","last_became_peered":"2024-01-27T02:39:21.899844+0000","last_unstale":"2024-01-27T02:39:22.911228+0000","last_undegraded":"2024-01-27T02:39:22.911228+0000","last_fullsized":"2024-01-27T02:39:22.911228+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:03:27.069542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.911277+0000","last_change":"2024-01-27T02:39:22.908170+0000","last_active":"2024-01-27T02:39:22.911277+0000","last_peered":"2024-01-27T02:39:22.911277+0000","last_clean":"2024-01-27T02:39:22.911277+0000","last_became_active":"2024-01-27T02:39:22.907713+0000","last_became_peered":"2024-01-27T02:39:22.907713+0000","last_unstale":"2024-01-27T02:39:22.911277+0000","last_undegraded":"2024-01-27T02:39:22.911277+0000","last_fullsized":"2024-01-27T02:39:22.911277+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:59:03.340181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"148'1","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.907916+0000","last_change":"2024-01-27T02:39:21.898336+0000","last_active":"2024-01-27T02:39:22.907916+0000","last_peered":"2024-01-27T02:39:22.907916+0000","last_clean":"2024-01-27T02:39:22.907916+0000","last_became_active":"2024-01-27T02:39:21.898073+0000","last_became_peered":"2024-01-27T02:39:21.898073+0000","last_unstale":"2024-01-27T02:39:22.907916+0000","last_undegraded":"2024-01-27T02:39:22.907916+0000","last_fullsized":"2024-01-27T02:39:22.907916+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:59:06.427886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.907601+0000","last_change":"2024-01-27T02:39:22.907034+0000","last_active":"2024-01-27T02:39:22.907601+0000","last_peered":"2024-01-27T02:39:22.907601+0000","last_clean":"2024-01-27T02:39:22.907601+0000","last_became_active":"2024-01-27T02:39:22.906810+0000","last_became_peered":"2024-01-27T02:39:22.906810+0000","last_unstale":"2024-01-27T02:39:22.907601+0000","last_undegraded":"2024-01-27T02:39:22.907601+0000","last_fullsized":"2024-01-27T02:39:22.907601+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:03:28.071762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.911846+0000","last_change":"2024-01-27T02:39:21.902022+0000","last_active":"2024-01-27T02:39:22.911846+0000","last_peered":"2024-01-27T02:39:22.911846+0000","last_clean":"2024-01-27T02:39:22.911846+0000","last_became_active":"2024-01-27T02:39:21.901762+0000","last_became_peered":"2024-01-27T02:39:21.901762+0000","last_unstale":"2024-01-27T02:39:22.911846+0000","last_undegraded":"2024-01-27T02:39:22.911846+0000","last_fullsized":"2024-01-27T02:39:22.911846+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:35:38.967949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.903467+0000","last_change":"2024-01-27T02:39:22.902929+0000","last_active":"2024-01-27T02:39:22.903467+0000","last_peered":"2024-01-27T02:39:22.903467+0000","last_clean":"2024-01-27T02:39:22.903467+0000","last_became_active":"2024-01-27T02:39:22.902654+0000","last_became_peered":"2024-01-27T02:39:22.902654+0000","last_unstale":"2024-01-27T02:39:22.903467+0000","last_undegraded":"2024-01-27T02:39:22.903467+0000","last_fullsized":"2024-01-27T02:39:22.903467+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:02:57.570709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"148'1","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910217+0000","last_change":"2024-01-27T02:39:21.895442+0000","last_active":"2024-01-27T02:39:22.910217+0000","last_peered":"2024-01-27T02:39:22.910217+0000","last_clean":"2024-01-27T02:39:22.910217+0000","last_became_active":"2024-01-27T02:39:21.894130+0000","last_became_peered":"2024-01-27T02:39:21.894130+0000","last_unstale":"2024-01-27T02:39:22.910217+0000","last_undegraded":"2024-01-27T02:39:22.910217+0000","last_fullsized":"2024-01-27T02:39:22.910217+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:02:56.715132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.908720+0000","last_change":"2024-01-27T02:39:22.908370+0000","last_active":"2024-01-27T02:39:22.908720+0000","last_peered":"2024-01-27T02:39:22.908720+0000","last_clean":"2024-01-27T02:39:22.908720+0000","last_became_active":"2024-01-27T02:39:22.907779+0000","last_became_peered":"2024-01-27T02:39:22.907779+0000","last_unstale":"2024-01-27T02:39:22.908720+0000","last_undegraded":"2024-01-27T02:39:22.908720+0000","last_fullsized":"2024-01-27T02:39:22.908720+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:17:07.671167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.894420+0000","last_change":"2024-01-27T02:39:21.895019+0000","last_active":"2024-01-27T02:39:22.894420+0000","last_peered":"2024-01-27T02:39:22.894420+0000","last_clean":"2024-01-27T02:39:22.894420+0000","last_became_active":"2024-01-27T02:39:21.894813+0000","last_became_peered":"2024-01-27T02:39:21.894813+0000","last_unstale":"2024-01-27T02:39:22.894420+0000","last_undegraded":"2024-01-27T02:39:22.894420+0000","last_fullsized":"2024-01-27T02:39:22.894420+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:01:58.908495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.900222+0000","last_change":"2024-01-27T02:39:22.900150+0000","last_active":"2024-01-27T02:39:22.900222+0000","last_peered":"2024-01-27T02:39:22.900222+0000","last_clean":"2024-01-27T02:39:22.900222+0000","last_became_active":"2024-01-27T02:39:22.899453+0000","last_became_peered":"2024-01-27T02:39:22.899453+0000","last_unstale":"2024-01-27T02:39:22.900222+0000","last_undegraded":"2024-01-27T02:39:22.900222+0000","last_fullsized":"2024-01-27T02:39:22.900222+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:08:54.067660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910284+0000","last_change":"2024-01-27T02:39:21.896656+0000","last_active":"2024-01-27T02:39:22.910284+0000","last_peered":"2024-01-27T02:39:22.910284+0000","last_clean":"2024-01-27T02:39:22.910284+0000","last_became_active":"2024-01-27T02:39:21.896422+0000","last_became_peered":"2024-01-27T02:39:21.896422+0000","last_unstale":"2024-01-27T02:39:22.910284+0000","last_undegraded":"2024-01-27T02:39:22.910284+0000","last_fullsized":"2024-01-27T02:39:22.910284+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:10:40.058748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.900541+0000","last_change":"2024-01-27T02:39:22.900467+0000","last_active":"2024-01-27T02:39:22.900541+0000","last_peered":"2024-01-27T02:39:22.900541+0000","last_clean":"2024-01-27T02:39:22.900541+0000","last_became_active":"2024-01-27T02:39:22.899866+0000","last_became_peered":"2024-01-27T02:39:22.899866+0000","last_unstale":"2024-01-27T02:39:22.900541+0000","last_undegraded":"2024-01-27T02:39:22.900541+0000","last_fullsized":"2024-01-27T02:39:22.900541+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:27:21.148523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.893201+0000","last_change":"2024-01-27T02:39:21.900412+0000","last_active":"2024-01-27T02:39:22.893201+0000","last_peered":"2024-01-27T02:39:22.893201+0000","last_clean":"2024-01-27T02:39:22.893201+0000","last_became_active":"2024-01-27T02:39:21.900133+0000","last_became_peered":"2024-01-27T02:39:21.900133+0000","last_unstale":"2024-01-27T02:39:22.893201+0000","last_undegraded":"2024-01-27T02:39:22.893201+0000","last_fullsized":"2024-01-27T02:39:22.893201+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:40:32.460005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.908737+0000","last_change":"2024-01-27T02:39:22.908527+0000","last_active":"2024-01-27T02:39:22.908737+0000","last_peered":"2024-01-27T02:39:22.908737+0000","last_clean":"2024-01-27T02:39:22.908737+0000","last_became_active":"2024-01-27T02:39:22.908130+0000","last_became_peered":"2024-01-27T02:39:22.908130+0000","last_unstale":"2024-01-27T02:39:22.908737+0000","last_undegraded":"2024-01-27T02:39:22.908737+0000","last_fullsized":"2024-01-27T02:39:22.908737+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:25:03.541488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.890111+0000","last_change":"2024-01-27T02:39:21.889901+0000","last_active":"2024-01-27T02:39:22.890111+0000","last_peered":"2024-01-27T02:39:22.890111+0000","last_clean":"2024-01-27T02:39:22.890111+0000","last_became_active":"2024-01-27T02:39:21.889524+0000","last_became_peered":"2024-01-27T02:39:21.889524+0000","last_unstale":"2024-01-27T02:39:22.890111+0000","last_undegraded":"2024-01-27T02:39:22.890111+0000","last_fullsized":"2024-01-27T02:39:22.890111+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:16.787597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.910855+0000","last_change":"2024-01-27T02:39:22.909847+0000","last_active":"2024-01-27T02:39:22.910855+0000","last_peered":"2024-01-27T02:39:22.910855+0000","last_clean":"2024-01-27T02:39:22.910855+0000","last_became_active":"2024-01-27T02:39:22.909357+0000","last_became_peered":"2024-01-27T02:39:22.909357+0000","last_unstale":"2024-01-27T02:39:22.910855+0000","last_undegraded":"2024-01-27T02:39:22.910855+0000","last_fullsized":"2024-01-27T02:39:22.910855+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:12.863356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.894295+0000","last_change":"2024-01-27T02:39:21.901504+0000","last_active":"2024-01-27T02:39:22.894295+0000","last_peered":"2024-01-27T02:39:22.894295+0000","last_clean":"2024-01-27T02:39:22.894295+0000","last_became_active":"2024-01-27T02:39:21.901245+0000","last_became_peered":"2024-01-27T02:39:21.901245+0000","last_unstale":"2024-01-27T02:39:22.894295+0000","last_undegraded":"2024-01-27T02:39:22.894295+0000","last_fullsized":"2024-01-27T02:39:22.894295+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:57:53.176503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.900280+0000","last_change":"2024-01-27T02:39:22.900231+0000","last_active":"2024-01-27T02:39:22.900280+0000","last_peered":"2024-01-27T02:39:22.900280+0000","last_clean":"2024-01-27T02:39:22.900280+0000","last_became_active":"2024-01-27T02:39:22.899499+0000","last_became_peered":"2024-01-27T02:39:22.899499+0000","last_unstale":"2024-01-27T02:39:22.900280+0000","last_undegraded":"2024-01-27T02:39:22.900280+0000","last_fullsized":"2024-01-27T02:39:22.900280+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:04:43.117550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.912471+0000","last_change":"2024-01-27T02:39:21.896736+0000","last_active":"2024-01-27T02:39:22.912471+0000","last_peered":"2024-01-27T02:39:22.912471+0000","last_clean":"2024-01-27T02:39:22.912471+0000","last_became_active":"2024-01-27T02:39:21.896197+0000","last_became_peered":"2024-01-27T02:39:21.896197+0000","last_unstale":"2024-01-27T02:39:22.912471+0000","last_undegraded":"2024-01-27T02:39:22.912471+0000","last_fullsized":"2024-01-27T02:39:22.912471+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:00:34.270221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.898823+0000","last_change":"2024-01-27T02:39:22.898657+0000","last_active":"2024-01-27T02:39:22.898823+0000","last_peered":"2024-01-27T02:39:22.898823+0000","last_clean":"2024-01-27T02:39:22.898823+0000","last_became_active":"2024-01-27T02:39:22.898442+0000","last_became_peered":"2024-01-27T02:39:22.898442+0000","last_unstale":"2024-01-27T02:39:22.898823+0000","last_undegraded":"2024-01-27T02:39:22.898823+0000","last_fullsized":"2024-01-27T02:39:22.898823+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:55:52.096811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"149'1","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-01-27T02:39:25.059245+0000","last_change":"2024-01-27T02:39:21.898780+0000","last_active":"2024-01-27T02:39:25.059245+0000","last_peered":"2024-01-27T02:39:25.059245+0000","last_clean":"2024-01-27T02:39:25.059245+0000","last_became_active":"2024-01-27T02:39:21.898579+0000","last_became_peered":"2024-01-27T02:39:21.898579+0000","last_unstale":"2024-01-27T02:39:25.059245+0000","last_undegraded":"2024-01-27T02:39:25.059245+0000","last_fullsized":"2024-01-27T02:39:25.059245+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:59:36.125517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.906436+0000","last_change":"2024-01-27T02:39:22.906355+0000","last_active":"2024-01-27T02:39:22.906436+0000","last_peered":"2024-01-27T02:39:22.906436+0000","last_clean":"2024-01-27T02:39:22.906436+0000","last_became_active":"2024-01-27T02:39:22.902011+0000","last_became_peered":"2024-01-27T02:39:22.902011+0000","last_unstale":"2024-01-27T02:39:22.906436+0000","last_undegraded":"2024-01-27T02:39:22.906436+0000","last_fullsized":"2024-01-27T02:39:22.906436+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:09:30.892305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.894225+0000","last_change":"2024-01-27T02:39:21.900466+0000","last_active":"2024-01-27T02:39:22.894225+0000","last_peered":"2024-01-27T02:39:22.894225+0000","last_clean":"2024-01-27T02:39:22.894225+0000","last_became_active":"2024-01-27T02:39:21.900211+0000","last_became_peered":"2024-01-27T02:39:21.900211+0000","last_unstale":"2024-01-27T02:39:22.894225+0000","last_undegraded":"2024-01-27T02:39:22.894225+0000","last_fullsized":"2024-01-27T02:39:22.894225+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:26:41.909600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.901978+0000","last_change":"2024-01-27T02:39:22.901912+0000","last_active":"2024-01-27T02:39:22.901978+0000","last_peered":"2024-01-27T02:39:22.901978+0000","last_clean":"2024-01-27T02:39:22.901978+0000","last_became_active":"2024-01-27T02:39:22.901571+0000","last_became_peered":"2024-01-27T02:39:22.901571+0000","last_unstale":"2024-01-27T02:39:22.901978+0000","last_undegraded":"2024-01-27T02:39:22.901978+0000","last_fullsized":"2024-01-27T02:39:22.901978+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:39:17.789817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.897627+0000","last_change":"2024-01-27T02:39:21.903774+0000","last_active":"2024-01-27T02:39:22.897627+0000","last_peered":"2024-01-27T02:39:22.897627+0000","last_clean":"2024-01-27T02:39:22.897627+0000","last_became_active":"2024-01-27T02:39:21.903535+0000","last_became_peered":"2024-01-27T02:39:21.903535+0000","last_unstale":"2024-01-27T02:39:22.897627+0000","last_undegraded":"2024-01-27T02:39:22.897627+0000","last_fullsized":"2024-01-27T02:39:22.897627+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:02:56.568488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.898120+0000","last_change":"2024-01-27T02:39:22.898044+0000","last_active":"2024-01-27T02:39:22.898120+0000","last_peered":"2024-01-27T02:39:22.898120+0000","last_clean":"2024-01-27T02:39:22.898120+0000","last_became_active":"2024-01-27T02:39:22.897789+0000","last_became_peered":"2024-01-27T02:39:22.897789+0000","last_unstale":"2024-01-27T02:39:22.898120+0000","last_undegraded":"2024-01-27T02:39:22.898120+0000","last_fullsized":"2024-01-27T02:39:22.898120+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:36:32.750631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.895852+0000","last_change":"2024-01-27T02:39:21.900816+0000","last_active":"2024-01-27T02:39:22.895852+0000","last_peered":"2024-01-27T02:39:22.895852+0000","last_clean":"2024-01-27T02:39:22.895852+0000","last_became_active":"2024-01-27T02:39:21.900263+0000","last_became_peered":"2024-01-27T02:39:21.900263+0000","last_unstale":"2024-01-27T02:39:22.895852+0000","last_undegraded":"2024-01-27T02:39:22.895852+0000","last_fullsized":"2024-01-27T02:39:22.895852+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:48:49.973643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.901979+0000","last_change":"2024-01-27T02:39:22.901592+0000","last_active":"2024-01-27T02:39:22.901979+0000","last_peered":"2024-01-27T02:39:22.901979+0000","last_clean":"2024-01-27T02:39:22.901979+0000","last_became_active":"2024-01-27T02:39:22.901146+0000","last_became_peered":"2024-01-27T02:39:22.901146+0000","last_unstale":"2024-01-27T02:39:22.901979+0000","last_undegraded":"2024-01-27T02:39:22.901979+0000","last_fullsized":"2024-01-27T02:39:22.901979+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:25:47.557777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.894599+0000","last_change":"2024-01-27T02:39:21.899411+0000","last_active":"2024-01-27T02:39:22.894599+0000","last_peered":"2024-01-27T02:39:22.894599+0000","last_clean":"2024-01-27T02:39:22.894599+0000","last_became_active":"2024-01-27T02:39:21.899153+0000","last_became_peered":"2024-01-27T02:39:21.899153+0000","last_unstale":"2024-01-27T02:39:22.894599+0000","last_undegraded":"2024-01-27T02:39:22.894599+0000","last_fullsized":"2024-01-27T02:39:22.894599+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:18:01.492341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902253+0000","last_change":"2024-01-27T02:39:22.899280+0000","last_active":"2024-01-27T02:39:22.902253+0000","last_peered":"2024-01-27T02:39:22.902253+0000","last_clean":"2024-01-27T02:39:22.902253+0000","last_became_active":"2024-01-27T02:39:22.899002+0000","last_became_peered":"2024-01-27T02:39:22.899002+0000","last_unstale":"2024-01-27T02:39:22.902253+0000","last_undegraded":"2024-01-27T02:39:22.902253+0000","last_fullsized":"2024-01-27T02:39:22.902253+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:43:03.625409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.890445+0000","last_change":"2024-01-27T02:39:21.901048+0000","last_active":"2024-01-27T02:39:22.890445+0000","last_peered":"2024-01-27T02:39:22.890445+0000","last_clean":"2024-01-27T02:39:22.890445+0000","last_became_active":"2024-01-27T02:39:21.900699+0000","last_became_peered":"2024-01-27T02:39:21.900699+0000","last_unstale":"2024-01-27T02:39:22.890445+0000","last_undegraded":"2024-01-27T02:39:22.890445+0000","last_fullsized":"2024-01-27T02:39:22.890445+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:59:02.337961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.908703+0000","last_change":"2024-01-27T02:39:22.908250+0000","last_active":"2024-01-27T02:39:22.908703+0000","last_peered":"2024-01-27T02:39:22.908703+0000","last_clean":"2024-01-27T02:39:22.908703+0000","last_became_active":"2024-01-27T02:39:22.908011+0000","last_became_peered":"2024-01-27T02:39:22.908011+0000","last_unstale":"2024-01-27T02:39:22.908703+0000","last_undegraded":"2024-01-27T02:39:22.908703+0000","last_fullsized":"2024-01-27T02:39:22.908703+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:25:47.557777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.895255+0000","last_change":"2024-01-27T02:39:21.899014+0000","last_active":"2024-01-27T02:39:22.895255+0000","last_peered":"2024-01-27T02:39:22.895255+0000","last_clean":"2024-01-27T02:39:22.895255+0000","last_became_active":"2024-01-27T02:39:21.898758+0000","last_became_peered":"2024-01-27T02:39:21.898758+0000","last_unstale":"2024-01-27T02:39:22.895255+0000","last_undegraded":"2024-01-27T02:39:22.895255+0000","last_fullsized":"2024-01-27T02:39:22.895255+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:39:20.730878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.902623+0000","last_change":"2024-01-27T02:39:22.902514+0000","last_active":"2024-01-27T02:39:22.902623+0000","last_peered":"2024-01-27T02:39:22.902623+0000","last_clean":"2024-01-27T02:39:22.902623+0000","last_became_active":"2024-01-27T02:39:22.902016+0000","last_became_peered":"2024-01-27T02:39:22.902016+0000","last_unstale":"2024-01-27T02:39:22.902623+0000","last_undegraded":"2024-01-27T02:39:22.902623+0000","last_fullsized":"2024-01-27T02:39:22.902623+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:21.879721+0000","last_clean_scrub_stamp":"2024-01-27T02:39:21.879721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:50:07.072576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"148'2","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.912561+0000","last_change":"2024-01-27T02:39:21.901188+0000","last_active":"2024-01-27T02:39:22.912561+0000","last_peered":"2024-01-27T02:39:22.912561+0000","last_clean":"2024-01-27T02:39:22.912561+0000","last_became_active":"2024-01-27T02:39:21.900932+0000","last_became_peered":"2024-01-27T02:39:21.900932+0000","last_unstale":"2024-01-27T02:39:22.912561+0000","last_undegraded":"2024-01-27T02:39:22.912561+0000","last_fullsized":"2024-01-27T02:39:22.912561+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:39:20.877501+0000","last_clean_scrub_stamp":"2024-01-27T02:39:20.877501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:58:36.696301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":274,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-27T02:39:22.894459+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:39:22.894459+0000","last_peered":"2024-01-27T02:39:22.894459+0000","last_clean":"2024-01-27T02:39:22.894459+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:39:22.894459+0000","last_undegraded":"2024-01-27T02:39:22.894459+0000","last_fullsized":"2024-01-27T02:39:22.894459+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574941,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1383,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193928,"statfs":{"total":200000000,"available":198583067,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.39400000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76200000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66900000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.59499999999999997}]},{"osd":1,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76900000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.628},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.56000000000000005},"min":{"1min":0.41799999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.67600000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.54100000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.67900000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.49199999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76200000000000001,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.68200000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.48399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.66800000000000004,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.45400000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.74099999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.78800000000000003}]},{"osd":4,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47699999999999998,"15min":0.48899999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70699999999999996,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.48899999999999999,"15min":0.51200000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69299999999999995,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.61199999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.51800000000000002,"15min":0.53700000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.307},"max":{"1min":0.73199999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55200000000000005,"15min":0.56499999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33500000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.70699999999999996}]},{"osd":6,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57799999999999996,"15min":0.59599999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.90800000000000003},"last":0.69499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574941,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":806,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194505,"statfs":{"total":200000000,"available":199174045,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.63,"15min":0.54400000000000004},"min":{"1min":0.375,"5min":0.36499999999999999,"15min":0.29999999999999999},"max":{"1min":0.83099999999999996,"5min":4.125,"15min":4.125},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59699999999999998,"15min":0.54000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32700000000000001},"max":{"1min":0.75600000000000001,"5min":1.216,"15min":1.216},"last":0.61799999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.67200000000000004,"15min":0.55600000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.34100000000000003},"max":{"1min":0.84199999999999997,"5min":8.2699999999999996,"15min":8.2699999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.88900000000000001,"15min":0.65500000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80600000000000005,"5min":26.411999999999999,"15min":26.411999999999999},"last":0.63600000000000001}]},{"osd":2,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.75700000000000001,"15min":0.60799999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77000000000000002,"5min":17.143999999999998,"15min":17.143999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.64400000000000002,"15min":0.59299999999999997},"min":{"1min":0.45800000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85799999999999998,"5min":1.76,"15min":1.76},"last":0.67100000000000004}]},{"osd":3,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.624,"15min":0.56899999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.376},"max":{"1min":0.78000000000000003,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.624,"15min":0.58199999999999996},"min":{"1min":0.44500000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79300000000000004,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.64600000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.30199999999999999},"max":{"1min":0.73599999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.50900000000000001,"15min":0.54300000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84899999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.47299999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.80800000000000005,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56299999999999994,"15min":0.59599999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.56399999999999995}]},{"osd":7,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59299999999999997,"15min":0.60999999999999999},"min":{"1min":0.40999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.87,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55700000000000005,"15min":0.58199999999999996},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.36499999999999999},"max":{"1min":0.82899999999999996,"5min":1.038,"15min":1.038},"last":0.50600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":805,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194507,"statfs":{"total":200000000,"available":199175581,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39800000000000002,"15min":0.378},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.248},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41799999999999998,"15min":0.40000000000000002},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.50600000000000001,"5min":1.121,"15min":1.121},"last":0.372}]},{"osd":2,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40799999999999997,"15min":0.373},"min":{"1min":0.249,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":1.0940000000000001,"5min":1.25,"15min":1.25},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.44900000000000001,"15min":0.43099999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29899999999999999},"max":{"1min":0.498,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.38800000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.48499999999999999,"15min":0.46899999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.32100000000000001},"max":{"1min":0.56699999999999995,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.46600000000000003,"15min":0.45500000000000002},"min":{"1min":0.34899999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.56100000000000005,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.30499999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.66600000000000004,"15min":0.63600000000000001},"min":{"1min":0.48299999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.85699999999999998,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66600000000000004,"15min":0.64000000000000001},"min":{"1min":0.45000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91600000000000004,"5min":1.026,"15min":1.026},"last":0.58199999999999996}]},{"osd":5,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69099999999999995,"15min":0.64200000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":0.93400000000000005,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69699999999999995,"15min":0.67600000000000005},"min":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96199999999999997,"5min":3.181,"15min":3.181},"last":0.55300000000000005}]},{"osd":6,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.70599999999999996,"15min":0.69499999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88500000000000001,"5min":0.96399999999999997,"15min":1.024},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72099999999999997,"15min":0.68300000000000005},"min":{"1min":0.54000000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93600000000000005,"5min":1.869,"15min":1.869},"last":0.60099999999999998}]},{"osd":7,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73199999999999998,"15min":0.69099999999999995},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.47299999999999998},"max":{"1min":0.91400000000000003,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73099999999999998,"15min":0.69499999999999995},"min":{"1min":0.53000000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92000000000000004,"5min":1.708,"15min":1.708},"last":0.51900000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574941,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1381,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193930,"statfs":{"total":200000000,"available":198585213,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.40600000000000003,"15min":0.39400000000000002},"min":{"1min":0.252,"5min":0.24399999999999999,"15min":0.23799999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.434},{"interface":"front","average":{"1min":0.437,"5min":0.42899999999999999,"15min":0.41799999999999998},"min":{"1min":0.28899999999999998,"5min":0.27800000000000002,"15min":0.223},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.46800000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.433,"15min":0.42199999999999999},"min":{"1min":0.31,"5min":0.28699999999999998,"15min":0.28199999999999997},"max":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.45700000000000002,"15min":0.438},"min":{"1min":0.312,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.41499999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.441,"15min":0.432},"min":{"1min":0.315,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.66100000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.495,"15min":0.47599999999999998},"min":{"1min":0.38800000000000001,"5min":0.33600000000000002,"15min":0.29099999999999998},"max":{"1min":0.67600000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.52100000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66800000000000004,"15min":0.64400000000000002},"min":{"1min":0.505,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80900000000000005,"5min":0.997,"15min":0.997},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.45800000000000002,"5min":0.44,"15min":0.41999999999999998},"max":{"1min":0.83399999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.87}]},{"osd":5,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68899999999999995,"15min":0.65800000000000003},"min":{"1min":0.51900000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.78800000000000003,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.54900000000000004,"5min":0.47799999999999998,"15min":0.44},"max":{"1min":0.84999999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.73799999999999999}]},{"osd":6,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68300000000000005,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.42499999999999999},"max":{"1min":0.79800000000000004,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69899999999999995,"15min":0.68000000000000005},"min":{"1min":0.51600000000000001,"5min":0.45700000000000002,"15min":0.45000000000000001},"max":{"1min":0.90300000000000002,"5min":1.04,"15min":1.04},"last":0.85599999999999998}]},{"osd":7,"last update":"Sat Jan 27 02:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.50800000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90200000000000002,"5min":1.014,"15min":1.014},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69699999999999995,"15min":0.69199999999999995},"min":{"1min":0.54500000000000004,"5min":0.498,"15min":0.498},"max":{"1min":0.81999999999999995,"5min":0.95799999999999996,"15min":0.97299999999999998},"last":0.90400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574941,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":805,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194506,"statfs":{"total":200000000,"available":199174993,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.41199999999999998},"min":{"1min":0.27300000000000002,"5min":0.23599999999999999,"15min":0.23000000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.46700000000000003,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.30499999999999999},"max":{"1min":0.68899999999999995,"5min":0.72799999999999998,"15min":6.2729999999999997},"last":0.45800000000000002}]},{"osd":1,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.433,"15min":0.42299999999999999},"min":{"1min":0.33800000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.621,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.439,"15min":0.44400000000000001},"min":{"1min":0.33900000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.60399999999999998,"5min":0.65300000000000002,"15min":0.70099999999999996},"last":0.39300000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.434,"15min":0.41799999999999998},"min":{"1min":0.32200000000000001,"5min":0.28899999999999998,"15min":0.27800000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.314},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.49099999999999999,"15min":0.48599999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.43099999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68799999999999994,"15min":0.65900000000000003},"min":{"1min":0.499,"5min":0.46800000000000003,"15min":0.39300000000000002},"max":{"1min":0.90500000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.70599999999999996,"15min":0.78100000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95799999999999996,"5min":1.006,"15min":3.581},"last":0.61199999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65800000000000003,"15min":0.72699999999999998},"min":{"1min":0.47099999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92200000000000004,"5min":0.95899999999999996,"15min":3.5369999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.52200000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89100000000000001,"5min":0.92200000000000004,"15min":1.0529999999999999},"last":0.55400000000000005}]},{"osd":6,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67200000000000004,"15min":0.66700000000000004},"min":{"1min":0.57499999999999996,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96499999999999997,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69599999999999995,"15min":0.68000000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.97099999999999997,"5min":1.0189999999999999,"15min":1.0209999999999999},"last":0.93500000000000005}]},{"osd":7,"last update":"Sat Jan 27 02:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69199999999999995,"15min":0.67900000000000005},"min":{"1min":0.58799999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97699999999999998,"5min":1.03,"15min":1.03},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70899999999999996,"15min":0.69299999999999995},"min":{"1min":0.55200000000000005,"5min":0.441,"15min":0.42699999999999999},"max":{"1min":0.99099999999999999,"5min":1.036,"15min":1.036},"last":1.004}]}]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":805,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194507,"statfs":{"total":200000000,"available":199175421,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.45500000000000002,"15min":0.38600000000000001},"min":{"1min":0.29699999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.67900000000000005,"5min":4.7380000000000004,"15min":4.7380000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41899999999999998,"15min":0.40000000000000002},"min":{"1min":0.29399999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55500000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.38400000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.47499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28000000000000003,"5min":0.27400000000000002,"15min":0.22900000000000001},"max":{"1min":0.61599999999999999,"5min":4.5739999999999998,"15min":4.5739999999999998},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.53300000000000003,"15min":0.45200000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.98199999999999998,"5min":5.0529999999999999,"15min":5.0529999999999999},"last":0.625}]},{"osd":2,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.49199999999999999,"15min":0.443},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.28100000000000003},"max":{"1min":0.76700000000000002,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.55400000000000005,"15min":0.47699999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71799999999999997,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":0.49399999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69999999999999996,"15min":0.63800000000000001},"min":{"1min":0.45300000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91600000000000004,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.62},"min":{"1min":0.48199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86099999999999999,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.60199999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67600000000000005,"15min":0.626},"min":{"1min":0.438,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82399999999999995,"5min":1.409,"15min":1.409},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.78700000000000003,"15min":0.70399999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.89600000000000002,"5min":4.7960000000000003,"15min":4.7960000000000003},"last":0.72599999999999998}]},{"osd":6,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.73899999999999999,"15min":0.67900000000000005},"min":{"1min":0.42799999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92700000000000005,"5min":5.0069999999999997,"15min":5.0069999999999997},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.69399999999999995,"15min":0.65900000000000003},"min":{"1min":0.47499999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.94199999999999995,"5min":1.103,"15min":1.103},"last":0.67400000000000004}]},{"osd":7,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.75700000000000001,"15min":0.66900000000000004},"min":{"1min":0.57599999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72699999999999998,"15min":0.68000000000000005},"min":{"1min":0.52800000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.88900000000000001,"5min":1.361,"15min":1.361},"last":0.69699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":805,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194507,"statfs":{"total":200000000,"available":199175511,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.35599999999999998,"5min":0.34599999999999997,"15min":0.314},"max":{"1min":1.137,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.66200000000000003,"15min":0.61399999999999999},"min":{"1min":0.496,"5min":0.372,"15min":0.372},"max":{"1min":1.016,"5min":7.0670000000000002,"15min":7.0670000000000002},"last":0.59099999999999997}]},{"osd":1,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.64200000000000002,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.39500000000000002,"15min":0.35399999999999998},"max":{"1min":0.81000000000000005,"5min":5.1529999999999996,"15min":5.1529999999999996},"last":0.501},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64400000000000002,"15min":0.60899999999999999},"min":{"1min":0.46400000000000002,"5min":0.40799999999999997,"15min":0.39000000000000001},"max":{"1min":1.054,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.63700000000000001}]},{"osd":2,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.64200000000000002,"15min":0.61099999999999999},"min":{"1min":0.48399999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76600000000000001,"5min":5.0019999999999998,"15min":5.0019999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.63300000000000001,"15min":0.62},"min":{"1min":0.432,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2549999999999999,"5min":2.073,"15min":2.073},"last":0.61399999999999999}]},{"osd":3,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.60499999999999998},"min":{"1min":0.45200000000000001,"5min":0.42099999999999999,"15min":0.36199999999999999},"max":{"1min":1.1899999999999999,"5min":6.7220000000000004,"15min":6.7220000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68999999999999995,"15min":0.66400000000000003},"min":{"1min":0.46100000000000002,"5min":0.44900000000000001,"15min":0.41399999999999998},"max":{"1min":0.88,"5min":6.5510000000000002,"15min":6.5510000000000002},"last":0.56000000000000005}]},{"osd":5,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.34899999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.77500000000000002,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56599999999999995,"15min":0.51700000000000002},"min":{"1min":0.372,"5min":0.312,"15min":0.29999999999999999},"max":{"1min":0.68799999999999994,"5min":6.9269999999999996,"15min":6.9269999999999996},"last":0.40400000000000003}]},{"osd":6,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56100000000000005,"15min":0.52200000000000002},"min":{"1min":0.40400000000000003,"5min":0.34999999999999998,"15min":0.32900000000000001},"max":{"1min":0.78600000000000003,"5min":5.2229999999999999,"15min":5.2229999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56200000000000006,"15min":0.54900000000000004},"min":{"1min":0.41799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82999999999999996,"5min":1.46,"15min":1.46},"last":0.51600000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:39:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61699999999999999,"15min":0.57499999999999996},"min":{"1min":0.434,"5min":0.34799999999999998,"15min":0.33600000000000002},"max":{"1min":0.78300000000000003,"5min":7.5339999999999998,"15min":7.5339999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.629,"15min":0.59399999999999997},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85299999999999998,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":0.44600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":805,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194507,"statfs":{"total":200000000,"available":199175469,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56399999999999995,"15min":0.58999999999999997},"min":{"1min":0.42099999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56699999999999995,"15min":0.57899999999999996},"min":{"1min":0.442,"5min":0.34499999999999997,"15min":0.27600000000000002},"max":{"1min":0.77100000000000002,"5min":0.78900000000000003,"15min":3.242},"last":0.51200000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.622,"15min":0.61299999999999999},"min":{"1min":0.38500000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.3380000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61099999999999999,"15min":0.64300000000000002},"min":{"1min":0.35399999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78400000000000003,"5min":0.84399999999999997,"15min":0.95499999999999996},"last":0.67500000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58599999999999997,"15min":0.59699999999999998},"min":{"1min":0.40400000000000003,"5min":0.39100000000000001,"15min":0.372},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":3.04},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.48499999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80400000000000005,"5min":0.85899999999999999,"15min":0.89400000000000002},"last":0.61899999999999999}]},{"osd":3,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60499999999999998,"15min":0.66800000000000004},"min":{"1min":0.40300000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":4.032},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65500000000000003,"15min":0.67700000000000005},"min":{"1min":0.48999999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.76100000000000001,"5min":0.86199999999999999,"15min":4.282},"last":0.71299999999999997}]},{"osd":4,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50700000000000001,"15min":0.52200000000000002},"min":{"1min":0.35799999999999998,"5min":0.30599999999999999,"15min":0.26500000000000001},"max":{"1min":0.63600000000000001,"5min":0.81399999999999995,"15min":1.2869999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.495,"15min":0.55300000000000005},"min":{"1min":0.32800000000000001,"5min":0.318,"15min":0.30399999999999999},"max":{"1min":0.66900000000000004,"5min":0.79800000000000004,"15min":1.101},"last":0.69599999999999995}]},{"osd":6,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.497,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.65200000000000002,"5min":0.73199999999999998,"15min":1.7969999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55300000000000005,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80100000000000005,"5min":0.85799999999999998,"15min":1.6739999999999999},"last":0.73399999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.38200000000000001,"5min":0.34799999999999998,"15min":0.32700000000000001},"max":{"1min":0.66800000000000004,"5min":0.77100000000000002,"15min":1.008},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59699999999999998,"15min":0.64900000000000002},"min":{"1min":0.436,"5min":0.35899999999999999,"15min":0.33700000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":2},"last":0.65500000000000003}]}]}],"pool_statfs":[]}} 2024-01-27T02:39:32.175 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-27T02:39:32.175 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-27T02:39:32.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.352+0000 7f9792cb7700 1 Processor -- start 2024-01-27T02:39:32.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.352+0000 7f9792cb7700 1 -- start start 2024-01-27T02:39:32.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.353+0000 7f9792cb7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f978c0cb950 0x7f978c0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:32.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.353+0000 7f9792cb7700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f978c0cbd20 0x7f978c0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:32.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.353+0000 7f9792cb7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f978c0c9660 con 0x7f978c1412d0 2024-01-27T02:39:32.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.353+0000 7f9792cb7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f978c15c980 con 0x7f978c0cb950 2024-01-27T02:39:32.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.353+0000 7f9792cb7700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f978c15be00 con 0x7f978c0cbd20 2024-01-27T02:39:32.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.353+0000 7f9790a53700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f978c0cb950 0x7f978c0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:32.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.353+0000 7f9791254700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f978c1412d0 0x7f978c141730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48970/0 (socket says 172.21.15.7:48970) 2024-01-27T02:39:32.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.353+0000 7f9790a53700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f978c0cb950 0x7f978c0b1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34142/0 (socket says 172.21.15.7:34142) 2024-01-27T02:39:32.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.353+0000 7f9791254700 1 -- 172.21.15.7:0/496015528 learned_addr learned my addr 172.21.15.7:0/496015528 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:32.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.353+0000 7f978bfff700 1 --2- 172.21.15.7:0/496015528 >> v2:172.21.15.188:3300/0 conn(0x7f978c0cbd20 0x7f978c0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:32.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.353+0000 7f9790a53700 1 -- 172.21.15.7:0/496015528 >> v2:172.21.15.188:3300/0 conn(0x7f978c0cbd20 msgr2=0x7f978c0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:32.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.353+0000 7f9790a53700 1 --2- 172.21.15.7:0/496015528 >> v2:172.21.15.188:3300/0 conn(0x7f978c0cbd20 0x7f978c0ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.353+0000 7f9790a53700 1 -- 172.21.15.7:0/496015528 >> v1:172.21.15.7:6789/0 conn(0x7f978c1412d0 legacy=0x7f978c141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.353+0000 7f9790a53700 1 -- 172.21.15.7:0/496015528 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978c15c0e0 con 0x7f978c0cb950 2024-01-27T02:39:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.353+0000 7f978b7fe700 1 -- 172.21.15.7:0/496015528 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276031496 0 0) 0x7f978c0c9660 con 0x7f978c1412d0 2024-01-27T02:39:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.354+0000 7f9790a53700 1 --2- 172.21.15.7:0/496015528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f978c0cb950 0x7f978c0b1ac0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f978c0cb6a0 tx=0x7f97800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d9964a00d0b73 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.354+0000 7f978b7fe700 1 -- 172.21.15.7:0/496015528 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978000b6b0 con 0x7f978c0cb950 2024-01-27T02:39:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.354+0000 7f978b7fe700 1 -- 172.21.15.7:0/496015528 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978000b810 con 0x7f978c0cb950 2024-01-27T02:39:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.354+0000 7f978b7fe700 1 -- 172.21.15.7:0/496015528 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978000bbb0 con 0x7f978c0cb950 2024-01-27T02:39:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.354+0000 7f9792cb7700 1 -- 172.21.15.7:0/496015528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f978c0cb950 msgr2=0x7f978c0b1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.354+0000 7f9792cb7700 1 --2- 172.21.15.7:0/496015528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f978c0cb950 0x7f978c0b1ac0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f978c0cb6a0 tx=0x7f97800091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.354+0000 7f9792cb7700 1 -- 172.21.15.7:0/496015528 shutdown_connections 2024-01-27T02:39:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.354+0000 7f9792cb7700 1 --2- 172.21.15.7:0/496015528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f978c0cb950 0x7f978c0b1ac0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.354+0000 7f9792cb7700 1 --2- 172.21.15.7:0/496015528 >> v2:172.21.15.188:3300/0 conn(0x7f978c0cbd20 0x7f978c0ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.354+0000 7f9792cb7700 1 -- 172.21.15.7:0/496015528 >> 172.21.15.7:0/496015528 conn(0x7f978c063580 msgr2=0x7f978c063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.354+0000 7f9792cb7700 1 -- 172.21.15.7:0/496015528 shutdown_connections 2024-01-27T02:39:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.354+0000 7f9792cb7700 1 -- 172.21.15.7:0/496015528 wait complete. 2024-01-27T02:39:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.356+0000 7f9792cb7700 1 Processor -- start 2024-01-27T02:39:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.356+0000 7f9792cb7700 1 -- start start 2024-01-27T02:39:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.357+0000 7f9792cb7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f978c0cb950 0x7f978c15b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.357+0000 7f9792cb7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f978c0cbd20 0x7f978c1584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.357+0000 7f9792cb7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f978c1412d0 0x7f978c13f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.357+0000 7f9792cb7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c159930 con 0x7f978c1412d0 2024-01-27T02:39:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.357+0000 7f9792cb7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c158d20 con 0x7f978c0cbd20 2024-01-27T02:39:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.357+0000 7f9792cb7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978c159000 con 0x7f978c0cb950 2024-01-27T02:39:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.357+0000 7f9791254700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f978c1412d0 0x7f978c13f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.358+0000 7f9791254700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f978c1412d0 0x7f978c13f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34150/0 (socket says 172.21.15.7:34150) 2024-01-27T02:39:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.358+0000 7f9791254700 1 -- 172.21.15.7:0/150652246 learned_addr learned my addr 172.21.15.7:0/150652246 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.358+0000 7f978bfff700 1 --2- 172.21.15.7:0/150652246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f978c0cbd20 0x7f978c1584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.358+0000 7f9790a53700 1 --2- 172.21.15.7:0/150652246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f978c0cb950 0x7f978c15b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.358+0000 7f9791254700 1 -- 172.21.15.7:0/150652246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f978c0cb950 msgr2=0x7f978c15b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.358+0000 7f9791254700 1 --2- 172.21.15.7:0/150652246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f978c0cb950 0x7f978c15b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.358+0000 7f9791254700 1 -- 172.21.15.7:0/150652246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f978c0cbd20 msgr2=0x7f978c1584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.358+0000 7f9791254700 1 --2- 172.21.15.7:0/150652246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f978c0cbd20 0x7f978c1584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.358+0000 7f9791254700 1 -- 172.21.15.7:0/150652246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9780009420 con 0x7f978c1412d0 2024-01-27T02:39:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.358+0000 7f978bfff700 1 --2- 172.21.15.7:0/150652246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f978c0cbd20 0x7f978c1584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.358+0000 7f9791254700 1 --2- 172.21.15.7:0/150652246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f978c1412d0 0x7f978c13f740 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f978400cc90 tx=0x7f9784001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.358+0000 7f9788ff9700 1 -- 172.21.15.7:0/150652246 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978400cea0 con 0x7f978c1412d0 2024-01-27T02:39:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.358+0000 7f9788ff9700 1 -- 172.21.15.7:0/150652246 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978400ab70 con 0x7f978c1412d0 2024-01-27T02:39:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.358+0000 7f9792cb7700 1 -- 172.21.15.7:0/150652246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f978c1592e0 con 0x7f978c1412d0 2024-01-27T02:39:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.358+0000 7f9788ff9700 1 -- 172.21.15.7:0/150652246 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97840070f0 con 0x7f978c1412d0 2024-01-27T02:39:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.358+0000 7f9792cb7700 1 -- 172.21.15.7:0/150652246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f978c158a10 con 0x7f978c1412d0 2024-01-27T02:39:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.359+0000 7f9792cb7700 1 -- 172.21.15.7:0/150652246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f978c159ae0 con 0x7f978c1412d0 2024-01-27T02:39:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.360+0000 7f9788ff9700 1 -- 172.21.15.7:0/150652246 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f97840074a0 con 0x7f978c1412d0 2024-01-27T02:39:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.361+0000 7f9788ff9700 1 --2- 172.21.15.7:0/150652246 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9770066f30 0x7f97700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.361+0000 7f9788ff9700 1 -- 172.21.15.7:0/150652246 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9784085230 con 0x7f978c1412d0 2024-01-27T02:39:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.361+0000 7f9790a53700 1 --2- 172.21.15.7:0/150652246 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9770066f30 0x7f97700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.362+0000 7f9790a53700 1 --2- 172.21.15.7:0/150652246 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9770066f30 0x7f97700693e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f97800093f0 tx=0x7f9780017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.365+0000 7f9788ff9700 1 -- 172.21.15.7:0/150652246 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9784052600 con 0x7f978c1412d0 2024-01-27T02:39:32.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.656+0000 7f9792cb7700 1 -- 172.21.15.7:0/150652246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f978c06d960 con 0x7f978c1412d0 2024-01-27T02:39:32.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.660+0000 7f9788ff9700 1 -- 172.21.15.7:0/150652246 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+0 (secure 0 0 0) 0x7f9784055c20 con 0x7f978c1412d0 2024-01-27T02:39:32.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.663+0000 7f9792cb7700 1 -- 172.21.15.7:0/150652246 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9770066f30 msgr2=0x7f97700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:32.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.663+0000 7f9792cb7700 1 --2- 172.21.15.7:0/150652246 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9770066f30 0x7f97700693e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f97800093f0 tx=0x7f9780017000 comp rx=0 tx=0).stop 2024-01-27T02:39:32.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.663+0000 7f9792cb7700 1 -- 172.21.15.7:0/150652246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f978c1412d0 msgr2=0x7f978c13f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:32.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.663+0000 7f9792cb7700 1 --2- 172.21.15.7:0/150652246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f978c1412d0 0x7f978c13f740 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f978400cc90 tx=0x7f9784001a80 comp rx=0 tx=0).stop 2024-01-27T02:39:32.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.663+0000 7f9792cb7700 1 -- 172.21.15.7:0/150652246 shutdown_connections 2024-01-27T02:39:32.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.663+0000 7f9792cb7700 1 --2- 172.21.15.7:0/150652246 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9770066f30 0x7f97700693e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:32.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.663+0000 7f9792cb7700 1 --2- 172.21.15.7:0/150652246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f978c1412d0 0x7f978c13f740 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:32.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.663+0000 7f9792cb7700 1 --2- 172.21.15.7:0/150652246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f978c0cb950 0x7f978c15b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:32.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.663+0000 7f9792cb7700 1 --2- 172.21.15.7:0/150652246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f978c0cbd20 0x7f978c1584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:32.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.663+0000 7f9792cb7700 1 -- 172.21.15.7:0/150652246 >> 172.21.15.7:0/150652246 conn(0x7f978c063580 msgr2=0x7f978c0c42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:32.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.663+0000 7f9792cb7700 1 -- 172.21.15.7:0/150652246 shutdown_connections 2024-01-27T02:39:32.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.663+0000 7f9792cb7700 1 -- 172.21.15.7:0/150652246 wait complete. 2024-01-27T02:39:32.665 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-01-27T02:39:32.677 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:39:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.808+0000 7faf54912700 1 Processor -- start 2024-01-27T02:39:32.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.808+0000 7faf54912700 1 -- start start 2024-01-27T02:39:32.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.809+0000 7faf54912700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf50157ca0 0x7faf500c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:32.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.809+0000 7faf54912700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faf500c3ae0 0x7faf500cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:32.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.809+0000 7faf54912700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faf50158070 0x7faf500cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:32.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.809+0000 7faf54912700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf500ca020 con 0x7faf50158070 2024-01-27T02:39:32.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.809+0000 7faf54912700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf500c4f40 con 0x7faf500c3ae0 2024-01-27T02:39:32.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.809+0000 7faf54912700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf500c5210 con 0x7faf50157ca0 2024-01-27T02:39:32.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.809+0000 7faf4ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faf50158070 0x7faf500cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:32.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.809+0000 7faf4ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faf50158070 0x7faf500cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34156/0 (socket says 172.21.15.7:34156) 2024-01-27T02:39:32.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.809+0000 7faf4ed9d700 1 -- 172.21.15.7:0/2987144714 learned_addr learned my addr 172.21.15.7:0/2987144714 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:32.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.809+0000 7faf4dd9b700 1 --2- 172.21.15.7:0/2987144714 >> v2:172.21.15.188:3300/0 conn(0x7faf500c3ae0 0x7faf500cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:32.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.809+0000 7faf4e59c700 1 --2- 172.21.15.7:0/2987144714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf50157ca0 0x7faf500c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:32.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.810+0000 7faf4ed9d700 1 -- 172.21.15.7:0/2987144714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf50157ca0 msgr2=0x7faf500c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:32.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.810+0000 7faf4ed9d700 1 --2- 172.21.15.7:0/2987144714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf50157ca0 0x7faf500c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:32.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.810+0000 7faf4ed9d700 1 -- 172.21.15.7:0/2987144714 >> v2:172.21.15.188:3300/0 conn(0x7faf500c3ae0 msgr2=0x7faf500cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:32.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.810+0000 7faf4ed9d700 1 --2- 172.21.15.7:0/2987144714 >> v2:172.21.15.188:3300/0 conn(0x7faf500c3ae0 0x7faf500cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:32.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.810+0000 7faf4ed9d700 1 -- 172.21.15.7:0/2987144714 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf500c54f0 con 0x7faf50158070 2024-01-27T02:39:32.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.810+0000 7faf4e59c700 1 --2- 172.21.15.7:0/2987144714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf50157ca0 0x7faf500c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:32.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.810+0000 7faf4ed9d700 1 --2- 172.21.15.7:0/2987144714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf50158070 0x7faf500cda20 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7faf5006ee30 tx=0x7faf48009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc1790570dc83e4 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:32.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.810+0000 7faf4d59a700 1 -- 172.21.15.7:0/2987144714 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4800e070 con 0x7faf50158070 2024-01-27T02:39:32.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.810+0000 7faf4d59a700 1 -- 172.21.15.7:0/2987144714 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4800ba20 con 0x7faf50158070 2024-01-27T02:39:32.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.810+0000 7faf4d59a700 1 -- 172.21.15.7:0/2987144714 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf480092e0 con 0x7faf50158070 2024-01-27T02:39:32.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.811+0000 7faf54912700 1 -- 172.21.15.7:0/2987144714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf50158070 msgr2=0x7faf500cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:32.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.811+0000 7faf54912700 1 --2- 172.21.15.7:0/2987144714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf50158070 0x7faf500cda20 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7faf5006ee30 tx=0x7faf48009840 comp rx=0 tx=0).stop 2024-01-27T02:39:32.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.811+0000 7faf54912700 1 -- 172.21.15.7:0/2987144714 shutdown_connections 2024-01-27T02:39:32.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.811+0000 7faf54912700 1 --2- 172.21.15.7:0/2987144714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf50158070 0x7faf500cda20 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:32.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.811+0000 7faf54912700 1 --2- 172.21.15.7:0/2987144714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf50157ca0 0x7faf500c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:32.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.811+0000 7faf54912700 1 --2- 172.21.15.7:0/2987144714 >> v2:172.21.15.188:3300/0 conn(0x7faf500c3ae0 0x7faf500cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:32.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.811+0000 7faf54912700 1 -- 172.21.15.7:0/2987144714 >> 172.21.15.7:0/2987144714 conn(0x7faf50098020 msgr2=0x7faf500711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:32.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.811+0000 7faf54912700 1 -- 172.21.15.7:0/2987144714 shutdown_connections 2024-01-27T02:39:32.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.811+0000 7faf54912700 1 -- 172.21.15.7:0/2987144714 wait complete. 2024-01-27T02:39:32.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.812+0000 7faf54912700 1 Processor -- start 2024-01-27T02:39:32.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.812+0000 7faf54912700 1 -- start start 2024-01-27T02:39:32.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.813+0000 7faf54912700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf500c3ae0 0x7faf50155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:32.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.813+0000 7faf54912700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf50157ca0 0x7faf500b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:32.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.813+0000 7faf54912700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf50158070 0x7faf50150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:32.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.813+0000 7faf54912700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf500c8280 con 0x7faf50157ca0 2024-01-27T02:39:32.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.813+0000 7faf54912700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf501540c0 con 0x7faf50158070 2024-01-27T02:39:32.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.813+0000 7faf54912700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf501543a0 con 0x7faf500c3ae0 2024-01-27T02:39:32.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.813+0000 7faf4dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf50157ca0 0x7faf500b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:32.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.813+0000 7faf4dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf50157ca0 0x7faf500b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34164/0 (socket says 172.21.15.7:34164) 2024-01-27T02:39:32.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.813+0000 7faf4dd9b700 1 -- 172.21.15.7:0/4242300755 learned_addr learned my addr 172.21.15.7:0/4242300755 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:32.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.813+0000 7faf4ed9d700 1 --2- 172.21.15.7:0/4242300755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf50158070 0x7faf50150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:32.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.813+0000 7faf4dd9b700 1 -- 172.21.15.7:0/4242300755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf500c3ae0 msgr2=0x7faf50155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:32.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.813+0000 7faf4e59c700 1 --2- 172.21.15.7:0/4242300755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf500c3ae0 0x7faf50155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:32.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.813+0000 7faf4dd9b700 1 --2- 172.21.15.7:0/4242300755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf500c3ae0 0x7faf50155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:32.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.813+0000 7faf4dd9b700 1 -- 172.21.15.7:0/4242300755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf50158070 msgr2=0x7faf50150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:32.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.813+0000 7faf4dd9b700 1 --2- 172.21.15.7:0/4242300755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf50158070 0x7faf50150620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:32.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.813+0000 7faf4dd9b700 1 -- 172.21.15.7:0/4242300755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf48009ae0 con 0x7faf50157ca0 2024-01-27T02:39:32.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.813+0000 7faf4ed9d700 1 --2- 172.21.15.7:0/4242300755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf50158070 0x7faf50150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:39:32.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.814+0000 7faf4dd9b700 1 --2- 172.21.15.7:0/4242300755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf50157ca0 0x7faf500b3d20 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7faf4400ec80 tx=0x7faf440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:32.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.814+0000 7faf3e7fc700 1 -- 172.21.15.7:0/4242300755 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4400c9d0 con 0x7faf50157ca0 2024-01-27T02:39:32.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.814+0000 7faf3e7fc700 1 -- 172.21.15.7:0/4242300755 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4400cb30 con 0x7faf50157ca0 2024-01-27T02:39:32.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.814+0000 7faf54912700 1 -- 172.21.15.7:0/4242300755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf50154680 con 0x7faf50157ca0 2024-01-27T02:39:32.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.814+0000 7faf3e7fc700 1 -- 172.21.15.7:0/4242300755 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44013470 con 0x7faf50157ca0 2024-01-27T02:39:32.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.814+0000 7faf54912700 1 -- 172.21.15.7:0/4242300755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf50167d80 con 0x7faf50157ca0 2024-01-27T02:39:32.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.815+0000 7faf54912700 1 -- 172.21.15.7:0/4242300755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5004c710 con 0x7faf50157ca0 2024-01-27T02:39:32.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.815+0000 7faf3e7fc700 1 -- 172.21.15.7:0/4242300755 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7faf44013820 con 0x7faf50157ca0 2024-01-27T02:39:32.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.816+0000 7faf3e7fc700 1 --2- 172.21.15.7:0/4242300755 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faf30066f30 0x7faf300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:32.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.816+0000 7faf3e7fc700 1 -- 172.21.15.7:0/4242300755 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faf44084920 con 0x7faf50157ca0 2024-01-27T02:39:32.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.816+0000 7faf4e59c700 1 --2- 172.21.15.7:0/4242300755 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faf30066f30 0x7faf300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:32.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.816+0000 7faf4e59c700 1 --2- 172.21.15.7:0/4242300755 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faf30066f30 0x7faf300693e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7faf40006f00 tx=0x7faf40008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:32.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:32.820+0000 7faf3e7fc700 1 -- 172.21.15.7:0/4242300755 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf44051b90 con 0x7faf50157ca0 2024-01-27T02:39:33.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.068+0000 7faf54912700 1 -- 172.21.15.7:0/4242300755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf500c57d0 con 0x7faf50157ca0 2024-01-27T02:39:33.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.068+0000 7faf3e7fc700 1 -- 172.21.15.7:0/4242300755 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4599 (secure 0 0 0) 0x7faf440551b0 con 0x7faf50157ca0 2024-01-27T02:39:33.069 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:33.069 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6260,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3240780787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3240780787},{"type":"v1","addr":"172.21.15.188:6837","nonce":3240780787}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7156,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/4051709811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4051709811},{"type":"v1","addr":"172.21.15.188:6836","nonce":4051709811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7416,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/4000944335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":4000944335},{"type":"v1","addr":"172.21.15.7:6834","nonce":4000944335}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:39:22.327946+0000","modified":"2024-01-27T02:39:25.910241+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7419":{"gid":7419,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2386550039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2386550039},{"type":"v1","addr":"172.21.15.7:6835","nonce":2386550039}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2024-01-27T02:39:33.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.070+0000 7faf54912700 1 -- 172.21.15.7:0/4242300755 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faf30066f30 msgr2=0x7faf300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:33.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.070+0000 7faf54912700 1 --2- 172.21.15.7:0/4242300755 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faf30066f30 0x7faf300693e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7faf40006f00 tx=0x7faf40008040 comp rx=0 tx=0).stop 2024-01-27T02:39:33.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.070+0000 7faf54912700 1 -- 172.21.15.7:0/4242300755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf50157ca0 msgr2=0x7faf500b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:33.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.070+0000 7faf54912700 1 --2- 172.21.15.7:0/4242300755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf50157ca0 0x7faf500b3d20 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7faf4400ec80 tx=0x7faf440091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.070+0000 7faf54912700 1 -- 172.21.15.7:0/4242300755 shutdown_connections 2024-01-27T02:39:33.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.070+0000 7faf54912700 1 --2- 172.21.15.7:0/4242300755 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faf30066f30 0x7faf300693e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.070+0000 7faf54912700 1 --2- 172.21.15.7:0/4242300755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf50158070 0x7faf50150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.070+0000 7faf54912700 1 --2- 172.21.15.7:0/4242300755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf500c3ae0 0x7faf50155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.070+0000 7faf54912700 1 --2- 172.21.15.7:0/4242300755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf50157ca0 0x7faf500b3d20 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.070+0000 7faf54912700 1 -- 172.21.15.7:0/4242300755 >> 172.21.15.7:0/4242300755 conn(0x7faf50098020 msgr2=0x7faf5006bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:33.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.071+0000 7faf54912700 1 -- 172.21.15.7:0/4242300755 shutdown_connections 2024-01-27T02:39:33.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.071+0000 7faf54912700 1 -- 172.21.15.7:0/4242300755 wait complete. 2024-01-27T02:39:33.073 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 77 2024-01-27T02:39:33.488 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-27T02:39:22.327946+0000', 'modified': '2024-01-27T02:39:25.910241+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7419}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7419': {'gid': 7419, 'name': 'a', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6835/2386550039', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6833', 'nonce': 2386550039}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 2386550039}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-27T02:39:33.488 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-27T02:39:33.488 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-27T02:39:33.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.630+0000 7f5167d36700 1 Processor -- start 2024-01-27T02:39:33.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.630+0000 7f5167d36700 1 -- start start 2024-01-27T02:39:33.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.631+0000 7f5167d36700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5160156c00 0x7f5160156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.631+0000 7f5167d36700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f51600c9a00 0x7f51600c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.631+0000 7f5167d36700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f516006dbb0 con 0x7f516006e040 2024-01-27T02:39:33.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.631+0000 7f5167d36700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51600c8bc0 con 0x7f5160156c00 2024-01-27T02:39:33.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.631+0000 7f5167d36700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51600c88e0 con 0x7f51600c9a00 2024-01-27T02:39:33.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.631+0000 7f5165ad2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5160156c00 0x7f5160156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.631+0000 7f51652d1700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f516006e040 0x7f51600c92f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48976/0 (socket says 172.21.15.7:48976) 2024-01-27T02:39:33.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.631+0000 7f5165ad2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5160156c00 0x7f5160156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34172/0 (socket says 172.21.15.7:34172) 2024-01-27T02:39:33.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.631+0000 7f51652d1700 1 -- 172.21.15.7:0/886288594 learned_addr learned my addr 172.21.15.7:0/886288594 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:33.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.631+0000 7f51662d3700 1 --2- 172.21.15.7:0/886288594 >> v2:172.21.15.188:3300/0 conn(0x7f51600c9a00 0x7f51600c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.632+0000 7f5165ad2700 1 -- 172.21.15.7:0/886288594 >> v2:172.21.15.188:3300/0 conn(0x7f51600c9a00 msgr2=0x7f51600c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:33.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.632+0000 7f5165ad2700 1 --2- 172.21.15.7:0/886288594 >> v2:172.21.15.188:3300/0 conn(0x7f51600c9a00 0x7f51600c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.632+0000 7f5165ad2700 1 -- 172.21.15.7:0/886288594 >> v1:172.21.15.7:6789/0 conn(0x7f516006e040 legacy=0x7f51600c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:33.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.632+0000 7f5165ad2700 1 -- 172.21.15.7:0/886288594 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516006d560 con 0x7f5160156c00 2024-01-27T02:39:33.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.632+0000 7f5164ad0700 1 -- 172.21.15.7:0/886288594 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 473038730 0 0) 0x7f516006dbb0 con 0x7f516006e040 2024-01-27T02:39:33.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.632+0000 7f51662d3700 1 --2- 172.21.15.7:0/886288594 >> v2:172.21.15.188:3300/0 conn(0x7f51600c9a00 0x7f51600c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:33.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.632+0000 7f5165ad2700 1 --2- 172.21.15.7:0/886288594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5160156c00 0x7f5160156fd0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f51600ca170 tx=0x7f515c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed975f11236af3c2 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:33.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.632+0000 7f5164ad0700 1 -- 172.21.15.7:0/886288594 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c00b6b0 con 0x7f5160156c00 2024-01-27T02:39:33.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.632+0000 7f5164ad0700 1 -- 172.21.15.7:0/886288594 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c00b810 con 0x7f5160156c00 2024-01-27T02:39:33.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.632+0000 7f5164ad0700 1 -- 172.21.15.7:0/886288594 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c00bbb0 con 0x7f5160156c00 2024-01-27T02:39:33.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.633+0000 7f5167d36700 1 -- 172.21.15.7:0/886288594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5160156c00 msgr2=0x7f5160156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:33.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.633+0000 7f5167d36700 1 --2- 172.21.15.7:0/886288594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5160156c00 0x7f5160156fd0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f51600ca170 tx=0x7f515c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.633+0000 7f5167d36700 1 -- 172.21.15.7:0/886288594 shutdown_connections 2024-01-27T02:39:33.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.633+0000 7f5167d36700 1 --2- 172.21.15.7:0/886288594 >> v2:172.21.15.188:3300/0 conn(0x7f51600c9a00 0x7f51600c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.633+0000 7f5167d36700 1 --2- 172.21.15.7:0/886288594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5160156c00 0x7f5160156fd0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.633+0000 7f5167d36700 1 -- 172.21.15.7:0/886288594 >> 172.21.15.7:0/886288594 conn(0x7f5160063580 msgr2=0x7f5160063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:33.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.633+0000 7f5167d36700 1 -- 172.21.15.7:0/886288594 shutdown_connections 2024-01-27T02:39:33.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.633+0000 7f5167d36700 1 -- 172.21.15.7:0/886288594 wait complete. 2024-01-27T02:39:33.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.634+0000 7f5167d36700 1 Processor -- start 2024-01-27T02:39:33.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.634+0000 7f5167d36700 1 -- start start 2024-01-27T02:39:33.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.635+0000 7f5167d36700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f516006e040 0x7f516016b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.635+0000 7f5167d36700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51600c9a00 0x7f516016a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.635+0000 7f5167d36700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5160156c00 0x7f516006abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.635+0000 7f5167d36700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51600aa8c0 con 0x7f5160156c00 2024-01-27T02:39:33.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.635+0000 7f5167d36700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516016ba10 con 0x7f516006e040 2024-01-27T02:39:33.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.635+0000 7f5167d36700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516016aec0 con 0x7f51600c9a00 2024-01-27T02:39:33.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.635+0000 7f51662d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5160156c00 0x7f516006abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.635+0000 7f51662d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5160156c00 0x7f516006abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34176/0 (socket says 172.21.15.7:34176) 2024-01-27T02:39:33.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.635+0000 7f51662d3700 1 -- 172.21.15.7:0/2925744530 learned_addr learned my addr 172.21.15.7:0/2925744530 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:33.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.635+0000 7f5165ad2700 1 --2- 172.21.15.7:0/2925744530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f516006e040 0x7f516016b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.635+0000 7f51652d1700 1 --2- 172.21.15.7:0/2925744530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51600c9a00 0x7f516016a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.635+0000 7f51662d3700 1 -- 172.21.15.7:0/2925744530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51600c9a00 msgr2=0x7f516016a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:33.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.635+0000 7f51662d3700 1 --2- 172.21.15.7:0/2925744530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51600c9a00 0x7f516016a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.635+0000 7f51662d3700 1 -- 172.21.15.7:0/2925744530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f516006e040 msgr2=0x7f516016b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:33.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.635+0000 7f51662d3700 1 --2- 172.21.15.7:0/2925744530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f516006e040 0x7f516016b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.635+0000 7f51662d3700 1 -- 172.21.15.7:0/2925744530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515c009420 con 0x7f5160156c00 2024-01-27T02:39:33.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.635+0000 7f5165ad2700 1 --2- 172.21.15.7:0/2925744530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f516006e040 0x7f516016b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:39:33.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.635+0000 7f51652d1700 1 --2- 172.21.15.7:0/2925744530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51600c9a00 0x7f516016a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:39:33.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.635+0000 7f51662d3700 1 --2- 172.21.15.7:0/2925744530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5160156c00 0x7f516006abe0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f515800d180 tx=0x7f5158009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:33.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.636+0000 7f5155ffb700 1 -- 172.21.15.7:0/2925744530 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515800de20 con 0x7f5160156c00 2024-01-27T02:39:33.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.636+0000 7f5155ffb700 1 -- 172.21.15.7:0/2925744530 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515800f040 con 0x7f5160156c00 2024-01-27T02:39:33.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.636+0000 7f5167d36700 1 -- 172.21.15.7:0/2925744530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516006b120 con 0x7f5160156c00 2024-01-27T02:39:33.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.636+0000 7f5155ffb700 1 -- 172.21.15.7:0/2925744530 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5158015690 con 0x7f5160156c00 2024-01-27T02:39:33.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.636+0000 7f5167d36700 1 -- 172.21.15.7:0/2925744530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51600aadb0 con 0x7f5160156c00 2024-01-27T02:39:33.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.637+0000 7f5155ffb700 1 -- 172.21.15.7:0/2925744530 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f515801e3b0 con 0x7f5160156c00 2024-01-27T02:39:33.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.638+0000 7f5155ffb700 1 --2- 172.21.15.7:0/2925744530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5144066f30 0x7f51440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.638+0000 7f5155ffb700 1 -- 172.21.15.7:0/2925744530 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5158084b30 con 0x7f5160156c00 2024-01-27T02:39:33.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.638+0000 7f5165ad2700 1 --2- 172.21.15.7:0/2925744530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5144066f30 0x7f51440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.639+0000 7f5165ad2700 1 --2- 172.21.15.7:0/2925744530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5144066f30 0x7f51440693e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f51600c34b0 tx=0x7f515c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:33.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.640+0000 7f5171039b80 1 Processor -- start 2024-01-27T02:39:33.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.640+0000 7f5171039b80 1 -- start start 2024-01-27T02:39:33.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.641+0000 7f5171039b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ed010 0x5633314ed3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.641+0000 7f5171039b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563331535360 0x563331517840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.641+0000 7f5171039b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5633314ec5d0 0x563331517d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.641+0000 7f5171039b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633313f09f0 con 0x5633314ed010 2024-01-27T02:39:33.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.641+0000 7f5171039b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x563331442130 con 0x563331535360 2024-01-27T02:39:33.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.641+0000 7f5171039b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56333143f930 con 0x5633314ec5d0 2024-01-27T02:39:33.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.641+0000 7f5165ad2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ed010 0x5633314ed3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.641+0000 7f5165ad2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ed010 0x5633314ed3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34180/0 (socket says 172.21.15.7:34180) 2024-01-27T02:39:33.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.641+0000 7f5165ad2700 1 -- 172.21.15.7:0/2943517796 learned_addr learned my addr 172.21.15.7:0/2943517796 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:33.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.641+0000 7f51652d1700 1 --2- 172.21.15.7:0/2943517796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563331535360 0x563331517840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.641+0000 7f51662d3700 1 --2- 172.21.15.7:0/2943517796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5633314ec5d0 0x563331517d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.641+0000 7f5165ad2700 1 -- 172.21.15.7:0/2943517796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5633314ec5d0 msgr2=0x563331517d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:33.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.641+0000 7f5165ad2700 1 --2- 172.21.15.7:0/2943517796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5633314ec5d0 0x563331517d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.641+0000 7f5165ad2700 1 -- 172.21.15.7:0/2943517796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563331535360 msgr2=0x563331517840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:33.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.641+0000 7f5165ad2700 1 --2- 172.21.15.7:0/2943517796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563331535360 0x563331517840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.641+0000 7f5165ad2700 1 -- 172.21.15.7:0/2943517796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56333127ab10 con 0x5633314ed010 2024-01-27T02:39:33.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.641+0000 7f51662d3700 1 --2- 172.21.15.7:0/2943517796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5633314ec5d0 0x563331517d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:33.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.641+0000 7f5165ad2700 1 --2- 172.21.15.7:0/2943517796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ed010 0x5633314ed3e0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f51600cc9a0 tx=0x7f515c01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df37a0f1fd9527c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:33.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.641+0000 7f514affd700 1 -- 172.21.15.7:0/2943517796 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c01cc30 con 0x5633314ed010 2024-01-27T02:39:33.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.642+0000 7f514affd700 1 -- 172.21.15.7:0/2943517796 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c01cd90 con 0x5633314ed010 2024-01-27T02:39:33.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.642+0000 7f514affd700 1 -- 172.21.15.7:0/2943517796 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c01a680 con 0x5633314ed010 2024-01-27T02:39:33.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.642+0000 7f5171039b80 1 -- 172.21.15.7:0/2943517796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ed010 msgr2=0x5633314ed3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:33.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.642+0000 7f5171039b80 1 --2- 172.21.15.7:0/2943517796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ed010 0x5633314ed3e0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f51600cc9a0 tx=0x7f515c01e040 comp rx=0 tx=0).stop 2024-01-27T02:39:33.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.642+0000 7f5171039b80 1 -- 172.21.15.7:0/2943517796 shutdown_connections 2024-01-27T02:39:33.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.642+0000 7f5171039b80 1 --2- 172.21.15.7:0/2943517796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5633314ec5d0 0x563331517d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.642+0000 7f5171039b80 1 --2- 172.21.15.7:0/2943517796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ed010 0x5633314ed3e0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.642+0000 7f5171039b80 1 --2- 172.21.15.7:0/2943517796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563331535360 0x563331517840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.642+0000 7f5171039b80 1 -- 172.21.15.7:0/2943517796 >> 172.21.15.7:0/2943517796 conn(0x5633314eca70 msgr2=0x563331535820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:33.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.642+0000 7f5171039b80 1 -- 172.21.15.7:0/2943517796 shutdown_connections 2024-01-27T02:39:33.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.642+0000 7f5171039b80 1 -- 172.21.15.7:0/2943517796 wait complete. 2024-01-27T02:39:33.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.643+0000 7f5171039b80 1 -- start start 2024-01-27T02:39:33.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.643+0000 7f5171039b80 1 Processor -- start 2024-01-27T02:39:33.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.644+0000 7f5171039b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5633314ec5d0 0x563331541f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.644+0000 7f5171039b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ed010 0x56333152f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.644+0000 7f5171039b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563331535360 0x56333152c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.644+0000 7f5171039b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56333143b720 con 0x5633314ed010 2024-01-27T02:39:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.644+0000 7f5171039b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633313e71b0 con 0x5633314ec5d0 2024-01-27T02:39:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.644+0000 7f5171039b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633313e6e50 con 0x563331535360 2024-01-27T02:39:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.644+0000 7f5171039b80 10 client.0 _ll_register_callbacks cb 0x5633312497b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-27T02:39:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.644+0000 7f51652d1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ed010 0x56333152f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.644+0000 7f51652d1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ed010 0x56333152f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34194/0 (socket says 172.21.15.7:34194) 2024-01-27T02:39:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.644+0000 7f51652d1700 1 -- 172.21.15.7:0/2886554830 learned_addr learned my addr 172.21.15.7:0/2886554830 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.644+0000 7f51662d3700 1 --2- 172.21.15.7:0/2886554830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563331535360 0x56333152c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.644+0000 7f5165ad2700 1 --2- 172.21.15.7:0/2886554830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5633314ec5d0 0x563331541f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.644+0000 7f51652d1700 1 -- 172.21.15.7:0/2886554830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563331535360 msgr2=0x56333152c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.644+0000 7f51652d1700 1 --2- 172.21.15.7:0/2886554830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563331535360 0x56333152c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.644+0000 7f51652d1700 1 -- 172.21.15.7:0/2886554830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5633314ec5d0 msgr2=0x563331541f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.644+0000 7f51652d1700 1 --2- 172.21.15.7:0/2886554830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5633314ec5d0 0x563331541f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.644+0000 7f51652d1700 1 -- 172.21.15.7:0/2886554830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f515c022070 con 0x5633314ed010 2024-01-27T02:39:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.644+0000 7f5165ad2700 1 --2- 172.21.15.7:0/2886554830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5633314ec5d0 0x563331541f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:39:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.645+0000 7f51652d1700 1 --2- 172.21.15.7:0/2886554830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ed010 0x56333152f310 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f5160155810 tx=0x7f5150009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59551dd0e0ddf3a5 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.645+0000 7f513ffff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-27T02:39:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.645+0000 7f513ffff700 1 -- 172.21.15.7:0/2886554830 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5150014460 con 0x5633314ed010 2024-01-27T02:39:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.645+0000 7f513ffff700 1 -- 172.21.15.7:0/2886554830 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(149..149 src has 1..149) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5150005ef0 con 0x5633314ed010 2024-01-27T02:39:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.645+0000 7f5171039b80 1 -- 172.21.15.7:0/2886554830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633314e0190 con 0x5633314ed010 2024-01-27T02:39:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.645+0000 7f513ffff700 1 -- 172.21.15.7:0/2886554830 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51500062c0 con 0x5633314ed010 2024-01-27T02:39:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.645+0000 7f5171039b80 10 client.7593 fetch_fsmap learned FSMap version 77 2024-01-27T02:39:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.645+0000 7f5171039b80 1 -- 172.21.15.7:0/2886554830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x563331409290 con 0x5633314ed010 2024-01-27T02:39:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.645+0000 7f513ffff700 1 -- 172.21.15.7:0/2886554830 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5150024020 con 0x5633314ed010 2024-01-27T02:39:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.645+0000 7f513ffff700 1 -- 172.21.15.7:0/2886554830 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 77) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5150012040 con 0x5633314ed010 2024-01-27T02:39:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.645+0000 7f5171039b80 10 client.7593 fetch_fsmap finished waiting for FSMap version 77 2024-01-27T02:39:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.645+0000 7f5171039b80 10 client.7593 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-27T02:39:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.645+0000 7f5171039b80 20 client.7593 populate_metadata read hostname 'smithi007' 2024-01-27T02:39:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.645+0000 7f5171039b80 1 --2- 172.21.15.7:0/2886554830 >> [v2:172.21.15.7:6833/2386550039,v1:172.21.15.7:6835/2386550039] conn(0x563331545030 0x563331513d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.645+0000 7f5171039b80 4 client.7593 mds_command: new command op to 7419 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-27T02:39:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.645+0000 7f5171039b80 1 -- 172.21.15.7:0/2886554830 --> [v2:172.21.15.7:6833/2386550039,v1:172.21.15.7:6835/2386550039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633313ddee0 con 0x563331545030 2024-01-27T02:39:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.645+0000 7f5165ad2700 1 --2- 172.21.15.7:0/2886554830 >> [v2:172.21.15.7:6833/2386550039,v1:172.21.15.7:6835/2386550039] conn(0x563331545030 0x563331513d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.646+0000 7f513ffff700 10 client.7593 ms_handle_connect on v2:172.21.15.7:6833/2386550039 2024-01-27T02:39:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.646+0000 7f5165ad2700 1 --2- 172.21.15.7:0/2886554830 >> [v2:172.21.15.7:6833/2386550039,v1:172.21.15.7:6835/2386550039] conn(0x563331545030 0x563331513d50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47e2b5034a2a2c33 server_cookie=ad054c2bb98505bd in_seq=0 out_seq=0 2024-01-27T02:39:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.647+0000 7f513ffff700 1 -- 172.21.15.7:0/2886554830 <== mds.0 v2:172.21.15.7:6833/2386550039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17146 (crc 0 0 0) 0x5633313ddee0 con 0x563331545030 2024-01-27T02:39:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.647+0000 7f513ffff700 10 client.7593 handle_command_reply: tid=0 2024-01-27T02:39:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.647+0000 7f5171039b80 1 client.7593 shutdown 2024-01-27T02:39:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.647+0000 7f5171039b80 1 -- 172.21.15.7:0/2886554830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ed010 msgr2=0x56333152f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.647+0000 7f5171039b80 1 --2- 172.21.15.7:0/2886554830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ed010 0x56333152f310 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f5160155810 tx=0x7f5150009040 comp rx=0 tx=0).stop 2024-01-27T02:39:33.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.647+0000 7f5171039b80 1 -- 172.21.15.7:0/2886554830 shutdown_connections 2024-01-27T02:39:33.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.647+0000 7f5171039b80 1 --2- 172.21.15.7:0/2886554830 >> [v2:172.21.15.7:6833/2386550039,v1:172.21.15.7:6835/2386550039] conn(0x563331545030 0x563331513d50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.647+0000 7f5171039b80 1 --2- 172.21.15.7:0/2886554830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563331535360 0x56333152c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.647+0000 7f5171039b80 1 --2- 172.21.15.7:0/2886554830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5633314ec5d0 0x563331541f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.647+0000 7f513ffff700 0 client.7593 ms_handle_reset on v2:172.21.15.7:6833/2386550039 2024-01-27T02:39:33.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.647+0000 7f5171039b80 1 --2- 172.21.15.7:0/2886554830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ed010 0x56333152f310 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.647+0000 7f5171039b80 1 -- 172.21.15.7:0/2886554830 >> 172.21.15.7:0/2886554830 conn(0x5633314eca70 msgr2=0x5633315174e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:33.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.647+0000 7f5171039b80 1 -- 172.21.15.7:0/2886554830 shutdown_connections 2024-01-27T02:39:33.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.647+0000 7f5171039b80 1 -- 172.21.15.7:0/2886554830 wait complete. 2024-01-27T02:39:33.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.647+0000 7f5171039b80 20 client.7593 trim_cache size 0 max 16384 2024-01-27T02:39:33.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.665+0000 7f5171039b80 1 Processor -- start 2024-01-27T02:39:33.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.665+0000 7f5171039b80 1 -- start start 2024-01-27T02:39:33.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.666+0000 7f5171039b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5633314ec5d0 0x563331512690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.666+0000 7f5171039b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ed010 0x5633314e91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.666+0000 7f5171039b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563331535360 0x5633315341c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.666+0000 7f5171039b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563331333e40 con 0x5633314ed010 2024-01-27T02:39:33.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.666+0000 7f5171039b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633315317d0 con 0x563331535360 2024-01-27T02:39:33.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.666+0000 7f5171039b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633315180c0 con 0x5633314ec5d0 2024-01-27T02:39:33.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.666+0000 7f51652d1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ed010 0x5633314e91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.666+0000 7f51652d1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ed010 0x5633314e91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34202/0 (socket says 172.21.15.7:34202) 2024-01-27T02:39:33.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.666+0000 7f51652d1700 1 -- 172.21.15.7:0/46599389 learned_addr learned my addr 172.21.15.7:0/46599389 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:33.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.666+0000 7f5165ad2700 1 --2- 172.21.15.7:0/46599389 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5633314ec5d0 0x563331512690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.666+0000 7f51662d3700 1 --2- 172.21.15.7:0/46599389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563331535360 0x5633315341c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.666+0000 7f51652d1700 1 -- 172.21.15.7:0/46599389 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5633314ec5d0 msgr2=0x563331512690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:33.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.666+0000 7f51652d1700 1 --2- 172.21.15.7:0/46599389 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5633314ec5d0 0x563331512690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.666+0000 7f51652d1700 1 -- 172.21.15.7:0/46599389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563331535360 msgr2=0x5633315341c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:33.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.666+0000 7f51652d1700 1 --2- 172.21.15.7:0/46599389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563331535360 0x5633315341c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.666+0000 7f51652d1700 1 -- 172.21.15.7:0/46599389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633314ec3d0 con 0x5633314ed010 2024-01-27T02:39:33.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.666+0000 7f51662d3700 1 --2- 172.21.15.7:0/46599389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563331535360 0x5633315341c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:33.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.666+0000 7f5165ad2700 1 --2- 172.21.15.7:0/46599389 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5633314ec5d0 0x563331512690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:39:33.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.667+0000 7f51652d1700 1 --2- 172.21.15.7:0/46599389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ed010 0x5633314e91e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f5150006ab0 tx=0x7f5150020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e066620541c49cc2 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:33.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.667+0000 7f514b7fe700 1 -- 172.21.15.7:0/46599389 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5150000d30 con 0x5633314ed010 2024-01-27T02:39:33.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.667+0000 7f514b7fe700 1 -- 172.21.15.7:0/46599389 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5150024070 con 0x5633314ed010 2024-01-27T02:39:33.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.667+0000 7f514b7fe700 1 -- 172.21.15.7:0/46599389 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5150006630 con 0x5633314ed010 2024-01-27T02:39:33.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.667+0000 7f5171039b80 1 -- 172.21.15.7:0/46599389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ed010 msgr2=0x5633314e91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:33.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.667+0000 7f5171039b80 1 --2- 172.21.15.7:0/46599389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ed010 0x5633314e91e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f5150006ab0 tx=0x7f5150020000 comp rx=0 tx=0).stop 2024-01-27T02:39:33.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.667+0000 7f5171039b80 1 -- 172.21.15.7:0/46599389 shutdown_connections 2024-01-27T02:39:33.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.667+0000 7f5171039b80 1 --2- 172.21.15.7:0/46599389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563331535360 0x5633315341c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.667+0000 7f5171039b80 1 --2- 172.21.15.7:0/46599389 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5633314ec5d0 0x563331512690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.667+0000 7f5171039b80 1 --2- 172.21.15.7:0/46599389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ed010 0x5633314e91e0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.667+0000 7f5171039b80 1 -- 172.21.15.7:0/46599389 >> 172.21.15.7:0/46599389 conn(0x5633314eca70 msgr2=0x563331535730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:33.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.668+0000 7f5171039b80 1 -- 172.21.15.7:0/46599389 shutdown_connections 2024-01-27T02:39:33.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.668+0000 7f5171039b80 1 -- 172.21.15.7:0/46599389 wait complete. 2024-01-27T02:39:33.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.668+0000 7f5171039b80 1 -- start start 2024-01-27T02:39:33.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.668+0000 7f5171039b80 1 Processor -- start 2024-01-27T02:39:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.669+0000 7f5171039b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ec5d0 0x563331580060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.669+0000 7f5171039b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5633314ed010 0x563331596ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.669+0000 7f5171039b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563331535360 0x563331534930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.669+0000 7f5171039b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563331517e30 con 0x5633314ec5d0 2024-01-27T02:39:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.669+0000 7f5171039b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x563331597940 con 0x5633314ed010 2024-01-27T02:39:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.669+0000 7f5171039b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633315341c0 con 0x563331535360 2024-01-27T02:39:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.669+0000 7f5171039b80 10 client.0 _ll_register_callbacks cb 0x5633312497b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-27T02:39:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.669+0000 7f5165ad2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ec5d0 0x563331580060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.669+0000 7f5165ad2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ec5d0 0x563331580060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34208/0 (socket says 172.21.15.7:34208) 2024-01-27T02:39:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.669+0000 7f5165ad2700 1 -- 172.21.15.7:0/2843890971 learned_addr learned my addr 172.21.15.7:0/2843890971 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.669+0000 7f51662d3700 1 --2- 172.21.15.7:0/2843890971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563331535360 0x563331534930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.669+0000 7f51652d1700 1 --2- 172.21.15.7:0/2843890971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5633314ed010 0x563331596ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.670+0000 7f5165ad2700 1 -- 172.21.15.7:0/2843890971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563331535360 msgr2=0x563331534930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.670+0000 7f5165ad2700 1 --2- 172.21.15.7:0/2843890971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563331535360 0x563331534930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.670+0000 7f5165ad2700 1 -- 172.21.15.7:0/2843890971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5633314ed010 msgr2=0x563331596ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.670+0000 7f5165ad2700 1 --2- 172.21.15.7:0/2843890971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5633314ed010 0x563331596ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.670+0000 7f5165ad2700 1 -- 172.21.15.7:0/2843890971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563331409290 con 0x5633314ec5d0 2024-01-27T02:39:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.670+0000 7f5165ad2700 1 --2- 172.21.15.7:0/2843890971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ec5d0 0x563331580060 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f515c02d000 tx=0x7f515c033040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5792bac765ff7498 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.670+0000 7f513ffff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-27T02:39:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.670+0000 7f513ffff700 1 -- 172.21.15.7:0/2843890971 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c040070 con 0x5633314ec5d0 2024-01-27T02:39:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.670+0000 7f513ffff700 1 -- 172.21.15.7:0/2843890971 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(149..149 src has 1..149) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f515c01a430 con 0x5633314ec5d0 2024-01-27T02:39:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.670+0000 7f5171039b80 1 -- 172.21.15.7:0/2843890971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633315303c0 con 0x5633314ec5d0 2024-01-27T02:39:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.670+0000 7f513ffff700 1 -- 172.21.15.7:0/2843890971 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f515c036040 con 0x5633314ec5d0 2024-01-27T02:39:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.670+0000 7f5171039b80 10 client.7599 fetch_fsmap learned FSMap version 77 2024-01-27T02:39:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.670+0000 7f5171039b80 1 -- 172.21.15.7:0/2843890971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x5633312d2c80 con 0x5633314ec5d0 2024-01-27T02:39:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.670+0000 7f513ffff700 1 -- 172.21.15.7:0/2843890971 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5633315303c0 con 0x5633314ec5d0 2024-01-27T02:39:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.670+0000 7f513ffff700 1 -- 172.21.15.7:0/2843890971 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 77) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f515c02f020 con 0x5633314ec5d0 2024-01-27T02:39:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.670+0000 7f5171039b80 10 client.7599 fetch_fsmap finished waiting for FSMap version 77 2024-01-27T02:39:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.670+0000 7f5171039b80 10 client.7599 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-27T02:39:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.670+0000 7f5171039b80 20 client.7599 populate_metadata read hostname 'smithi007' 2024-01-27T02:39:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.670+0000 7f5171039b80 1 --2- 172.21.15.7:0/2843890971 >> [v2:172.21.15.7:6833/2386550039,v1:172.21.15.7:6835/2386550039] conn(0x563331517040 0x563331517410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.671+0000 7f5171039b80 4 client.7599 mds_command: new command op to 7419 tid=0[{"prefix": "status"}] 2024-01-27T02:39:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.671+0000 7f5171039b80 1 -- 172.21.15.7:0/2843890971 --> [v2:172.21.15.7:6833/2386550039,v1:172.21.15.7:6835/2386550039] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f515c01cd90 con 0x563331517040 2024-01-27T02:39:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.671+0000 7f51652d1700 1 --2- 172.21.15.7:0/2843890971 >> [v2:172.21.15.7:6833/2386550039,v1:172.21.15.7:6835/2386550039] conn(0x563331517040 0x563331517410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.671+0000 7f513ffff700 10 client.7599 ms_handle_connect on v2:172.21.15.7:6833/2386550039 2024-01-27T02:39:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.671+0000 7f51652d1700 1 --2- 172.21.15.7:0/2843890971 >> [v2:172.21.15.7:6833/2386550039,v1:172.21.15.7:6835/2386550039] conn(0x563331517040 0x563331517410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2daf2f91da361cf server_cookie=415e3e9d1f49ba08 in_seq=0 out_seq=0 2024-01-27T02:39:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.671+0000 7f513ffff700 1 -- 172.21.15.7:0/2843890971 <== mds.0 v2:172.21.15.7:6833/2386550039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x7f515c01cd90 con 0x563331517040 2024-01-27T02:39:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.671+0000 7f513ffff700 10 client.7599 handle_command_reply: tid=0 2024-01-27T02:39:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.671+0000 7f5171039b80 1 client.7599 shutdown 2024-01-27T02:39:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.671+0000 7f5171039b80 1 -- 172.21.15.7:0/2843890971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ec5d0 msgr2=0x563331580060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.671+0000 7f5171039b80 1 --2- 172.21.15.7:0/2843890971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ec5d0 0x563331580060 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f515c02d000 tx=0x7f515c033040 comp rx=0 tx=0).stop 2024-01-27T02:39:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.671+0000 7f5171039b80 1 -- 172.21.15.7:0/2843890971 shutdown_connections 2024-01-27T02:39:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.671+0000 7f5171039b80 1 --2- 172.21.15.7:0/2843890971 >> [v2:172.21.15.7:6833/2386550039,v1:172.21.15.7:6835/2386550039] conn(0x563331517040 0x563331517410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.671+0000 7f5171039b80 1 --2- 172.21.15.7:0/2843890971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563331535360 0x563331534930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.671+0000 7f5171039b80 1 --2- 172.21.15.7:0/2843890971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633314ec5d0 0x563331580060 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.671+0000 7f513ffff700 0 client.7599 ms_handle_reset on v2:172.21.15.7:6833/2386550039 2024-01-27T02:39:33.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.671+0000 7f5171039b80 1 --2- 172.21.15.7:0/2843890971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5633314ed010 0x563331596ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.671+0000 7f5171039b80 1 -- 172.21.15.7:0/2843890971 >> 172.21.15.7:0/2843890971 conn(0x5633314eca70 msgr2=0x56333152e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:33.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.672+0000 7f5171039b80 1 -- 172.21.15.7:0/2843890971 shutdown_connections 2024-01-27T02:39:33.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.672+0000 7f5171039b80 1 -- 172.21.15.7:0/2843890971 wait complete. 2024-01-27T02:39:33.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.672+0000 7f5171039b80 20 client.7599 trim_cache size 0 max 16384 2024-01-27T02:39:33.678 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:39:33.678 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:39:33.678 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-27T02:39:33.678 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 7419, 2024-01-27T02:39:33.679 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-27T02:39:33.679 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-27T02:39:33.679 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-27T02:39:33.679 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 10.783527686999999, 2024-01-27T02:39:33.679 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 77, 2024-01-27T02:39:33.679 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 149, 2024-01-27T02:39:33.679 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 149, 2024-01-27T02:39:33.679 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 16.482867766999998 2024-01-27T02:39:33.679 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:39:33.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.672+0000 7f5167d36700 1 -- 172.21.15.7:0/2925744530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5144066f30 msgr2=0x7f51440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:33.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.672+0000 7f5167d36700 1 --2- 172.21.15.7:0/2925744530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5144066f30 0x7f51440693e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f51600c34b0 tx=0x7f515c017000 comp rx=0 tx=0).stop 2024-01-27T02:39:33.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.672+0000 7f5167d36700 1 -- 172.21.15.7:0/2925744530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5160156c00 msgr2=0x7f516006abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:33.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.672+0000 7f5167d36700 1 --2- 172.21.15.7:0/2925744530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5160156c00 0x7f516006abe0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f515800d180 tx=0x7f5158009a50 comp rx=0 tx=0).stop 2024-01-27T02:39:33.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.672+0000 7f5167d36700 1 -- 172.21.15.7:0/2925744530 shutdown_connections 2024-01-27T02:39:33.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.672+0000 7f5167d36700 1 --2- 172.21.15.7:0/2925744530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5144066f30 0x7f51440693e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.672+0000 7f5167d36700 1 --2- 172.21.15.7:0/2925744530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5160156c00 0x7f516006abe0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.672+0000 7f5167d36700 1 --2- 172.21.15.7:0/2925744530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f516006e040 0x7f516016b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.672+0000 7f5167d36700 1 --2- 172.21.15.7:0/2925744530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51600c9a00 0x7f516016a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.672+0000 7f5167d36700 1 -- 172.21.15.7:0/2925744530 >> 172.21.15.7:0/2925744530 conn(0x7f5160063580 msgr2=0x7f51601548d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:33.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.672+0000 7f5167d36700 1 -- 172.21.15.7:0/2925744530 shutdown_connections 2024-01-27T02:39:33.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.672+0000 7f5167d36700 1 -- 172.21.15.7:0/2925744530 wait complete. 2024-01-27T02:39:33.687 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:39:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.818+0000 7f026fd5d700 1 Processor -- start 2024-01-27T02:39:33.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.818+0000 7f026fd5d700 1 -- start start 2024-01-27T02:39:33.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.819+0000 7f026fd5d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0268147d60 0x7f02680c3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.819+0000 7f026fd5d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0268148130 0x7f02680c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.819+0000 7f026fd5d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02680c8b10 0x7f0268063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.819+0000 7f026fd5d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02680c8fb0 con 0x7f02680c8b10 2024-01-27T02:39:33.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.819+0000 7f026fd5d700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02680c3780 con 0x7f0268148130 2024-01-27T02:39:33.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.819+0000 7f026fd5d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02680c2b70 con 0x7f0268147d60 2024-01-27T02:39:33.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.819+0000 7f026e2fa700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02680c8b10 0x7f0268063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.819+0000 7f026e2fa700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02680c8b10 0x7f0268063580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34222/0 (socket says 172.21.15.7:34222) 2024-01-27T02:39:33.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.819+0000 7f026e2fa700 1 -- 172.21.15.7:0/1895920120 learned_addr learned my addr 172.21.15.7:0/1895920120 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:33.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.819+0000 7f026d2f8700 1 --2- 172.21.15.7:0/1895920120 >> v2:172.21.15.188:3300/0 conn(0x7f0268148130 0x7f02680c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.819+0000 7f026daf9700 1 --2- 172.21.15.7:0/1895920120 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0268147d60 0x7f02680c3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.819+0000 7f026e2fa700 1 -- 172.21.15.7:0/1895920120 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0268147d60 msgr2=0x7f02680c3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:33.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.819+0000 7f026e2fa700 1 --2- 172.21.15.7:0/1895920120 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0268147d60 0x7f02680c3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.819+0000 7f026e2fa700 1 -- 172.21.15.7:0/1895920120 >> v2:172.21.15.188:3300/0 conn(0x7f0268148130 msgr2=0x7f02680c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:33.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.819+0000 7f026e2fa700 1 --2- 172.21.15.7:0/1895920120 >> v2:172.21.15.188:3300/0 conn(0x7f0268148130 0x7f02680c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.819+0000 7f026e2fa700 1 -- 172.21.15.7:0/1895920120 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02680c2e50 con 0x7f02680c8b10 2024-01-27T02:39:33.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.820+0000 7f026e2fa700 1 --2- 172.21.15.7:0/1895920120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02680c8b10 0x7f0268063580 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f02680cb740 tx=0x7f0260009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ea61c3a5c9a1eb server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:33.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.820+0000 7f026caf7700 1 -- 172.21.15.7:0/1895920120 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026000e070 con 0x7f02680c8b10 2024-01-27T02:39:33.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.820+0000 7f026caf7700 1 -- 172.21.15.7:0/1895920120 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026000ba50 con 0x7f02680c8b10 2024-01-27T02:39:33.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.820+0000 7f026caf7700 1 -- 172.21.15.7:0/1895920120 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02600092e0 con 0x7f02680c8b10 2024-01-27T02:39:33.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.820+0000 7f026fd5d700 1 -- 172.21.15.7:0/1895920120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02680c8b10 msgr2=0x7f0268063580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:33.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.820+0000 7f026fd5d700 1 --2- 172.21.15.7:0/1895920120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02680c8b10 0x7f0268063580 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f02680cb740 tx=0x7f0260009840 comp rx=0 tx=0).stop 2024-01-27T02:39:33.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.820+0000 7f026fd5d700 1 -- 172.21.15.7:0/1895920120 shutdown_connections 2024-01-27T02:39:33.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.820+0000 7f026fd5d700 1 --2- 172.21.15.7:0/1895920120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02680c8b10 0x7f0268063580 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.820+0000 7f026fd5d700 1 --2- 172.21.15.7:0/1895920120 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0268147d60 0x7f02680c3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.820+0000 7f026fd5d700 1 --2- 172.21.15.7:0/1895920120 >> v2:172.21.15.188:3300/0 conn(0x7f0268148130 0x7f02680c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.820+0000 7f026fd5d700 1 -- 172.21.15.7:0/1895920120 >> 172.21.15.7:0/1895920120 conn(0x7f02680cdc30 msgr2=0x7f02680ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:33.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.821+0000 7f026fd5d700 1 -- 172.21.15.7:0/1895920120 shutdown_connections 2024-01-27T02:39:33.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.821+0000 7f026fd5d700 1 -- 172.21.15.7:0/1895920120 wait complete. 2024-01-27T02:39:33.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.822+0000 7f026fd5d700 1 Processor -- start 2024-01-27T02:39:33.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.822+0000 7f026fd5d700 1 -- start start 2024-01-27T02:39:33.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.823+0000 7f026fd5d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02680c8b10 0x7f02680c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.823+0000 7f026fd5d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0268147d60 0x7f0268161010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.823+0000 7f026fd5d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0268148130 0x7f026815de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.823+0000 7f026fd5d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02680c6920 con 0x7f0268147d60 2024-01-27T02:39:33.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.823+0000 7f026fd5d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02680c5d80 con 0x7f0268148130 2024-01-27T02:39:33.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.823+0000 7f026fd5d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02680c5ff0 con 0x7f02680c8b10 2024-01-27T02:39:33.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.823+0000 7f026d2f8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0268147d60 0x7f0268161010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.823+0000 7f026d2f8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0268147d60 0x7f0268161010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34236/0 (socket says 172.21.15.7:34236) 2024-01-27T02:39:33.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.823+0000 7f026d2f8700 1 -- 172.21.15.7:0/1229633761 learned_addr learned my addr 172.21.15.7:0/1229633761 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:33.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.823+0000 7f026daf9700 1 --2- 172.21.15.7:0/1229633761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02680c8b10 0x7f02680c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.823+0000 7f026e2fa700 1 --2- 172.21.15.7:0/1229633761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0268148130 0x7f026815de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.824+0000 7f026d2f8700 1 -- 172.21.15.7:0/1229633761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02680c8b10 msgr2=0x7f02680c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:33.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.824+0000 7f026d2f8700 1 --2- 172.21.15.7:0/1229633761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02680c8b10 0x7f02680c6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.824+0000 7f026d2f8700 1 -- 172.21.15.7:0/1229633761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0268148130 msgr2=0x7f026815de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:33.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.824+0000 7f026d2f8700 1 --2- 172.21.15.7:0/1229633761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0268148130 0x7f026815de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:33.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.824+0000 7f026d2f8700 1 -- 172.21.15.7:0/1229633761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0260009ae0 con 0x7f0268147d60 2024-01-27T02:39:33.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.824+0000 7f026d2f8700 1 --2- 172.21.15.7:0/1229633761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0268147d60 0x7f0268161010 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f025800d890 tx=0x7f0258008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:33.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.824+0000 7f025dffb700 1 -- 172.21.15.7:0/1229633761 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02580125f0 con 0x7f0268147d60 2024-01-27T02:39:33.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.824+0000 7f025dffb700 1 -- 172.21.15.7:0/1229633761 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0258012750 con 0x7f0268147d60 2024-01-27T02:39:33.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.824+0000 7f025dffb700 1 -- 172.21.15.7:0/1229633761 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0258012af0 con 0x7f0268147d60 2024-01-27T02:39:33.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.824+0000 7f026fd5d700 1 -- 172.21.15.7:0/1229633761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02680c62d0 con 0x7f0268147d60 2024-01-27T02:39:33.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.824+0000 7f026fd5d700 1 -- 172.21.15.7:0/1229633761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0268161550 con 0x7f0268147d60 2024-01-27T02:39:33.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.825+0000 7f026fd5d700 1 -- 172.21.15.7:0/1229633761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02680c6a90 con 0x7f0268147d60 2024-01-27T02:39:33.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.826+0000 7f025dffb700 1 -- 172.21.15.7:0/1229633761 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f025801e460 con 0x7f0268147d60 2024-01-27T02:39:33.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.826+0000 7f025dffb700 1 --2- 172.21.15.7:0/1229633761 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f024c066f30 0x7f024c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:33.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.826+0000 7f025dffb700 1 -- 172.21.15.7:0/1229633761 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0258084a80 con 0x7f0268147d60 2024-01-27T02:39:33.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.826+0000 7f026daf9700 1 --2- 172.21.15.7:0/1229633761 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f024c066f30 0x7f024c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:33.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.827+0000 7f026daf9700 1 --2- 172.21.15.7:0/1229633761 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f024c066f30 0x7f024c0693e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0264006fd0 tx=0x7f0264008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:33.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:33.831+0000 7f025dffb700 1 -- 172.21.15.7:0/1229633761 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0258055780 con 0x7f0268147d60 2024-01-27T02:39:34.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.080+0000 7f026fd5d700 1 -- 172.21.15.7:0/1229633761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f02680c6770 con 0x7f0268147d60 2024-01-27T02:39:34.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.080+0000 7f025dffb700 1 -- 172.21.15.7:0/1229633761 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v77) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0258055310 con 0x7f0268147d60 2024-01-27T02:39:34.081 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:39:34.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.082+0000 7f026fd5d700 1 -- 172.21.15.7:0/1229633761 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f024c066f30 msgr2=0x7f024c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:34.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.082+0000 7f026fd5d700 1 --2- 172.21.15.7:0/1229633761 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f024c066f30 0x7f024c0693e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0264006fd0 tx=0x7f0264008040 comp rx=0 tx=0).stop 2024-01-27T02:39:34.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.082+0000 7f026fd5d700 1 -- 172.21.15.7:0/1229633761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0268147d60 msgr2=0x7f0268161010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:34.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.082+0000 7f026fd5d700 1 --2- 172.21.15.7:0/1229633761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0268147d60 0x7f0268161010 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f025800d890 tx=0x7f0258008040 comp rx=0 tx=0).stop 2024-01-27T02:39:34.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.082+0000 7f026fd5d700 1 -- 172.21.15.7:0/1229633761 shutdown_connections 2024-01-27T02:39:34.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.082+0000 7f026fd5d700 1 --2- 172.21.15.7:0/1229633761 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f024c066f30 0x7f024c0693e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:34.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.082+0000 7f026fd5d700 1 --2- 172.21.15.7:0/1229633761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0268148130 0x7f026815de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:34.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.082+0000 7f026fd5d700 1 --2- 172.21.15.7:0/1229633761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02680c8b10 0x7f02680c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:34.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.082+0000 7f026fd5d700 1 --2- 172.21.15.7:0/1229633761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0268147d60 0x7f0268161010 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:34.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.082+0000 7f026fd5d700 1 -- 172.21.15.7:0/1229633761 >> 172.21.15.7:0/1229633761 conn(0x7f02680cdc30 msgr2=0x7f02680c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:34.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.082+0000 7f026fd5d700 1 -- 172.21.15.7:0/1229633761 shutdown_connections 2024-01-27T02:39:34.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.082+0000 7f026fd5d700 1 -- 172.21.15.7:0/1229633761 wait complete. 2024-01-27T02:39:34.096 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-27T02:39:34.097 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:39:34.097 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-27T02:39:34.097 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-27T02:39:34.097 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-27T02:39:34.097 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-27T02:39:34.097 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-27T02:39:34.097 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:39:34.097 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:39:34.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:39:34.097 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-27T02:39:34.117 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:39:34.117 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:39:34.117 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:39:34.117 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:39:34.117 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-27T02:39:34.117 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:39:34.118 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:39:34.118 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-27T02:39:34.118 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-27T02:39:34.118 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:39:34.118 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-27T02:39:34.118 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-27T02:39:34.118 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:39:34.118 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-27T02:39:34.118 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-27T02:39:34.118 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-27T02:39:34.118 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 41937sec preferred_lft 41937sec 2024-01-27T02:39:34.118 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-27T02:39:34.119 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:39:34.119 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:39:34.119 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-27T02:39:34.119 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-27T02:39:34.119 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:39:34.119 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 4a:57:d9:3c:c5:a0 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:39:34.119 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-27T02:39:34.119 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:39:34.119 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::2477:9eff:fe39:433c/64 scope link 2024-01-27T02:39:34.119 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:39:34.119 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:39:34.119 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 4a:57:d9:3c:c5:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-27T02:39:34.120 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::4857:d9ff:fe3c:c5a0/64 scope link 2024-01-27T02:39:34.120 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:39:34.120 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:39:34.120 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ca:6e:b8:df:53:bf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-27T02:39:34.120 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c86e:b8ff:fedf:53bf/64 scope link 2024-01-27T02:39:34.120 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:39:34.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:39:34.121 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:39:34.188 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-27T02:39:34.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:39:34.190 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-27T02:39:34.253 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-27T02:39:34.254 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:39:34.254 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-27T02:39:34.254 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-27T02:39:34.254 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:39:34.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:39:34.254 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-27T02:39:34.254 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-27T02:39:34.254 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-27T02:39:34.255 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-27T02:39:34.255 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-27T02:39:34.255 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-27T02:39:34.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:39:34.379 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-27T02:39:34.399 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-27T02:39:34.399 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-27T02:39:34.399 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:39:34.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:39:34.399 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-27T02:39:34.474 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:39:34.475 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-27T02:39:34.537 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:39:34.537 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:39:34.538 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-27T02:39:34.599 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:39:34.599 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:39:34.599 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:39:34.693 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:39:34.694 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:39:34.694 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:39:34.703 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:39:34.704 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:39:34.704 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:39:34.704 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:39:34.704 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:39:34.723 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:39:34.723 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:39:34.723 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:39:34.723 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:39:34.723 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:39:34.723 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:39:34.723 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:39:34.724 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:39:34.726 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:39:34.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:39:34.726 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-27T02:39:34.759 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:39:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.938+0000 7f5448595700 1 Processor -- start 2024-01-27T02:39:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.938+0000 7f5448595700 1 -- start start 2024-01-27T02:39:34.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.940+0000 7f5448595700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54400b1900 0x7f54400b0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:34.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.940+0000 7f5448595700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f544015c1e0 0x7f54400c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:34.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.940+0000 7f5448595700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54400b1470 con 0x7f544015be10 2024-01-27T02:39:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.940+0000 7f5448595700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54400b1d50 con 0x7f544015c1e0 2024-01-27T02:39:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.940+0000 7f5448595700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54400b0860 con 0x7f54400b1900 2024-01-27T02:39:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.940+0000 7f5446b32700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f544015c1e0 0x7f54400c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.940+0000 7f5446b32700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f544015c1e0 0x7f54400c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34244/0 (socket says 172.21.15.7:34244) 2024-01-27T02:39:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.940+0000 7f5446b32700 1 -- 172.21.15.7:0/2111600139 learned_addr learned my addr 172.21.15.7:0/2111600139 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.940+0000 7f5445b30700 1 --2- 172.21.15.7:0/2111600139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54400b1900 0x7f54400b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.940+0000 7f5446b32700 1 -- 172.21.15.7:0/2111600139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54400b1900 msgr2=0x7f54400b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.940+0000 7f5446b32700 1 --2- 172.21.15.7:0/2111600139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54400b1900 0x7f54400b0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.940+0000 7f5446b32700 1 -- 172.21.15.7:0/2111600139 >> v1:172.21.15.188:6789/0 conn(0x7f544015be10 legacy=0x7f54400aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.940+0000 7f5446b32700 1 -- 172.21.15.7:0/2111600139 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54400b0b40 con 0x7f544015c1e0 2024-01-27T02:39:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.940+0000 7f5446b32700 1 --2- 172.21.15.7:0/2111600139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f544015c1e0 0x7f54400c9500 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f5438009f20 tx=0x7f5438009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca9a74f99a8ac2e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.941+0000 7f544532f700 1 -- 172.21.15.7:0/2111600139 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543800e070 con 0x7f544015c1e0 2024-01-27T02:39:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.941+0000 7f544532f700 1 -- 172.21.15.7:0/2111600139 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543800b9d0 con 0x7f544015c1e0 2024-01-27T02:39:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.941+0000 7f544532f700 1 -- 172.21.15.7:0/2111600139 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54380092e0 con 0x7f544015c1e0 2024-01-27T02:39:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.941+0000 7f5448595700 1 -- 172.21.15.7:0/2111600139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f544015c1e0 msgr2=0x7f54400c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.941+0000 7f5448595700 1 --2- 172.21.15.7:0/2111600139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f544015c1e0 0x7f54400c9500 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f5438009f20 tx=0x7f5438009840 comp rx=0 tx=0).stop 2024-01-27T02:39:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.941+0000 7f5448595700 1 -- 172.21.15.7:0/2111600139 shutdown_connections 2024-01-27T02:39:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.941+0000 7f5448595700 1 --2- 172.21.15.7:0/2111600139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f544015c1e0 0x7f54400c9500 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.941+0000 7f5448595700 1 --2- 172.21.15.7:0/2111600139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54400b1900 0x7f54400b0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.941+0000 7f5448595700 1 -- 172.21.15.7:0/2111600139 >> 172.21.15.7:0/2111600139 conn(0x7f54400679b0 msgr2=0x7f5440067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.941+0000 7f5448595700 1 -- 172.21.15.7:0/2111600139 shutdown_connections 2024-01-27T02:39:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.942+0000 7f5448595700 1 -- 172.21.15.7:0/2111600139 wait complete. 2024-01-27T02:39:34.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.943+0000 7f5448595700 1 Processor -- start 2024-01-27T02:39:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.943+0000 7f5448595700 1 -- start start 2024-01-27T02:39:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.943+0000 7f5448595700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54400b1900 0x7f5440153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.943+0000 7f5448595700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f544015be10 0x7f544016b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.943+0000 7f5448595700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f544015c1e0 0x7f54400cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.943+0000 7f5448595700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54400cc210 con 0x7f544015be10 2024-01-27T02:39:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.943+0000 7f5448595700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544016b700 con 0x7f54400b1900 2024-01-27T02:39:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.943+0000 7f5448595700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54400cb600 con 0x7f544015c1e0 2024-01-27T02:39:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.944+0000 7f5445b30700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f544015be10 0x7f544016b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.944+0000 7f5445b30700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f544015be10 0x7f544016b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34252/0 (socket says 172.21.15.7:34252) 2024-01-27T02:39:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.944+0000 7f5445b30700 1 -- 172.21.15.7:0/1809340905 learned_addr learned my addr 172.21.15.7:0/1809340905 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:34.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.944+0000 7f5446b32700 1 --2- 172.21.15.7:0/1809340905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f544015c1e0 0x7f54400cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:34.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.944+0000 7f5446331700 1 --2- 172.21.15.7:0/1809340905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54400b1900 0x7f5440153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:34.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.944+0000 7f5445b30700 1 -- 172.21.15.7:0/1809340905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f544015c1e0 msgr2=0x7f54400cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:34.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.944+0000 7f5445b30700 1 --2- 172.21.15.7:0/1809340905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f544015c1e0 0x7f54400cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:34.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.944+0000 7f5445b30700 1 -- 172.21.15.7:0/1809340905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54400b1900 msgr2=0x7f5440153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:34.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.944+0000 7f5445b30700 1 --2- 172.21.15.7:0/1809340905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54400b1900 0x7f5440153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:34.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.944+0000 7f5445b30700 1 -- 172.21.15.7:0/1809340905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5438009ae0 con 0x7f544015be10 2024-01-27T02:39:34.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.944+0000 7f5445b30700 1 --2- 172.21.15.7:0/1809340905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f544015be10 0x7f544016b1c0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f543000ec80 tx=0x7f54300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:34.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.944+0000 7f54367fc700 1 -- 172.21.15.7:0/1809340905 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543000c8e0 con 0x7f544015be10 2024-01-27T02:39:34.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.944+0000 7f54367fc700 1 -- 172.21.15.7:0/1809340905 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543000ca40 con 0x7f544015be10 2024-01-27T02:39:34.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.944+0000 7f5448595700 1 -- 172.21.15.7:0/1809340905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54400cb8e0 con 0x7f544015be10 2024-01-27T02:39:34.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.944+0000 7f54367fc700 1 -- 172.21.15.7:0/1809340905 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543000cde0 con 0x7f544015be10 2024-01-27T02:39:34.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.944+0000 7f5448595700 1 -- 172.21.15.7:0/1809340905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54400cbea0 con 0x7f544015be10 2024-01-27T02:39:34.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.945+0000 7f5448595700 1 -- 172.21.15.7:0/1809340905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54400cc380 con 0x7f544015be10 2024-01-27T02:39:34.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.946+0000 7f54367fc700 1 -- 172.21.15.7:0/1809340905 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5430013680 con 0x7f544015be10 2024-01-27T02:39:34.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.946+0000 7f54367fc700 1 --2- 172.21.15.7:0/1809340905 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5424066f30 0x7f54240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:34.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.946+0000 7f54367fc700 1 -- 172.21.15.7:0/1809340905 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5430084830 con 0x7f544015be10 2024-01-27T02:39:34.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.947+0000 7f5446331700 1 --2- 172.21.15.7:0/1809340905 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5424066f30 0x7f54240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:34.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.947+0000 7f5446331700 1 --2- 172.21.15.7:0/1809340905 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5424066f30 0x7f54240693e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f544006abe0 tx=0x7f543c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:34.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:34.951+0000 7f54367fc700 1 -- 172.21.15.7:0/1809340905 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5430051aa0 con 0x7f544015be10 2024-01-27T02:39:35.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:35.203+0000 7f5448595700 1 -- 172.21.15.7:0/1809340905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f544004c710 con 0x7f544015be10 2024-01-27T02:39:35.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:35.204+0000 7f54367fc700 1 -- 172.21.15.7:0/1809340905 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v77) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5430084070 con 0x7f544015be10 2024-01-27T02:39:35.205 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:39:35.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:35.206+0000 7f5448595700 1 -- 172.21.15.7:0/1809340905 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5424066f30 msgr2=0x7f54240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:35.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:35.206+0000 7f5448595700 1 --2- 172.21.15.7:0/1809340905 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5424066f30 0x7f54240693e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f544006abe0 tx=0x7f543c009040 comp rx=0 tx=0).stop 2024-01-27T02:39:35.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:35.206+0000 7f5448595700 1 -- 172.21.15.7:0/1809340905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f544015be10 msgr2=0x7f544016b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:35.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:35.206+0000 7f5448595700 1 --2- 172.21.15.7:0/1809340905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f544015be10 0x7f544016b1c0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f543000ec80 tx=0x7f54300091a0 comp rx=0 tx=0).stop 2024-01-27T02:39:35.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:35.206+0000 7f5448595700 1 -- 172.21.15.7:0/1809340905 shutdown_connections 2024-01-27T02:39:35.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:35.206+0000 7f5448595700 1 --2- 172.21.15.7:0/1809340905 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5424066f30 0x7f54240693e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:35.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:35.206+0000 7f5448595700 1 --2- 172.21.15.7:0/1809340905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f544015c1e0 0x7f54400cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:35.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:35.206+0000 7f5448595700 1 --2- 172.21.15.7:0/1809340905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54400b1900 0x7f5440153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:35.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:35.206+0000 7f5448595700 1 --2- 172.21.15.7:0/1809340905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f544015be10 0x7f544016b1c0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:35.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:35.206+0000 7f5448595700 1 -- 172.21.15.7:0/1809340905 >> 172.21.15.7:0/1809340905 conn(0x7f54400679b0 msgr2=0x7f5440069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:35.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:35.206+0000 7f5448595700 1 -- 172.21.15.7:0/1809340905 shutdown_connections 2024-01-27T02:39:35.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:35.206+0000 7f5448595700 1 -- 172.21.15.7:0/1809340905 wait complete. 2024-01-27T02:39:35.220 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-27T02:39:35.220 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:39:35.220 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-01-27T02:39:35.220 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-27T02:39:35.220 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-27T02:39:35.220 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-27T02:39:35.220 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-27T02:39:35.220 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:39:35.220 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:39:35.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:39:35.221 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-01-27T02:39:35.243 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:39:35.243 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:39:35.243 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:39:35.243 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:39:35.243 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-01-27T02:39:35.243 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:39:35.243 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:39:35.243 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:39:35.243 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-01-27T02:39:35.243 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:39:35.244 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:39:35.244 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-01-27T02:39:35.244 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:39:35.244 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:39:35.244 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:39:35.244 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:39:35.244 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-27T02:39:35.244 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 41944sec preferred_lft 41944sec 2024-01-27T02:39:35.244 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b1:d6f7:baf0:bc17/64 scope link noprefixroute 2024-01-27T02:39:35.244 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:39:35.244 INFO:teuthology.orchestra.run.smithi188.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:39:35.244 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b0:92:71:90:9d brd ff:ff:ff:ff:ff:ff 2024-01-27T02:39:35.244 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-27T02:39:35.244 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:39:35.245 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::10f8:32ff:feed:a752/64 scope link 2024-01-27T02:39:35.245 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:39:35.245 INFO:teuthology.orchestra.run.smithi188.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:39:35.245 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b0:92:71:90:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-27T02:39:35.245 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b8b0:92ff:fe71:909d/64 scope link 2024-01-27T02:39:35.245 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:39:35.245 INFO:teuthology.orchestra.run.smithi188.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:39:35.245 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ee:78:03:20:3a:a9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-27T02:39:35.245 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec78:3ff:fe20:3aa9/64 scope link 2024-01-27T02:39:35.245 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:39:35.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:39:35.245 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:39:35.313 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-01-27T02:39:35.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:39:35.315 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-01-27T02:39:35.376 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-01-27T02:39:35.377 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:39:35.377 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-27T02:39:35.377 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-27T02:39:35.377 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:39:35.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:39:35.377 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-27T02:39:35.377 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-27T02:39:35.377 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-27T02:39:35.377 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-27T02:39:35.377 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-27T02:39:35.377 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-27T02:39:35.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:39:35.487 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-01-27T02:39:35.504 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-27T02:39:35.504 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-27T02:39:35.505 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:39:35.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:39:35.505 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2024-01-27T02:39:35.572 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:39:35.573 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-27T02:39:35.630 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:39:35.630 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:39:35.630 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-27T02:39:35.686 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:39:35.686 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:39:35.687 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:39:35.769 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:39:35.769 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:39:35.769 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:39:35.777 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:39:35.778 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:39:35.778 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:39:35.778 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:39:35.778 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:39:35.796 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:39:35.796 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:39:35.796 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:39:35.797 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:39:35.797 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:39:35.797 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:39:35.797 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:39:35.797 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:39:35.798 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:39:35.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:39:35.798 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-27T02:39:35.865 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:39:36.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.008+0000 7f6fc961b700 1 Processor -- start 2024-01-27T02:39:36.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.008+0000 7f6fc961b700 1 -- start start 2024-01-27T02:39:36.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.009+0000 7f6fc961b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6fc415a450 0x7f6fc40b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:36.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.009+0000 7f6fc961b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc40cc980 con 0x7f6fc40ce0f0 2024-01-27T02:39:36.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.009+0000 7f6fc961b700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc40ccc20 con 0x7f6fc40b2030 2024-01-27T02:39:36.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.009+0000 7f6fc961b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc40ccf00 con 0x7f6fc415a450 2024-01-27T02:39:36.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.009+0000 7f6fc2ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6fc415a450 0x7f6fc40b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:36.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.009+0000 7f6fc37fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6fc40ce0f0 0x7f6fc40c9e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48992/0 (socket says 172.21.15.7:48992) 2024-01-27T02:39:36.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.009+0000 7f6fc37fe700 1 -- 172.21.15.7:0/3918783915 learned_addr learned my addr 172.21.15.7:0/3918783915 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:39:36.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.009+0000 7f6fc2ffd700 1 -- 172.21.15.7:0/3918783915 >> v1:172.21.15.188:6789/0 conn(0x7f6fc40b2030 legacy=0x7f6fc40cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:36.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.009+0000 7f6fc2ffd700 1 -- 172.21.15.7:0/3918783915 >> v1:172.21.15.7:6789/0 conn(0x7f6fc40ce0f0 legacy=0x7f6fc40c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:36.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.009+0000 7f6fc2ffd700 1 -- 172.21.15.7:0/3918783915 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc40cd1e0 con 0x7f6fc415a450 2024-01-27T02:39:36.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.009+0000 7f6fc2ffd700 1 --2- 172.21.15.7:0/3918783915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc415a450 0x7f6fc40b1af0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f6fb40098d0 tx=0x7f6fb4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c770239908a4f24a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:36.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.009+0000 7f6fc1ffb700 1 -- 172.21.15.7:0/3918783915 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4009e70 con 0x7f6fc415a450 2024-01-27T02:39:36.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.009+0000 7f6fc1ffb700 1 -- 172.21.15.7:0/3918783915 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb4005c10 con 0x7f6fc415a450 2024-01-27T02:39:36.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.010+0000 7f6fc1ffb700 1 -- 172.21.15.7:0/3918783915 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb400b650 con 0x7f6fc415a450 2024-01-27T02:39:36.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.010+0000 7f6fc961b700 1 -- 172.21.15.7:0/3918783915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc415a450 msgr2=0x7f6fc40b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:36.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.010+0000 7f6fc961b700 1 --2- 172.21.15.7:0/3918783915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc415a450 0x7f6fc40b1af0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f6fb40098d0 tx=0x7f6fb4009120 comp rx=0 tx=0).stop 2024-01-27T02:39:36.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.010+0000 7f6fc961b700 1 -- 172.21.15.7:0/3918783915 shutdown_connections 2024-01-27T02:39:36.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.010+0000 7f6fc961b700 1 --2- 172.21.15.7:0/3918783915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc415a450 0x7f6fc40b1af0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:36.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.010+0000 7f6fc961b700 1 -- 172.21.15.7:0/3918783915 >> 172.21.15.7:0/3918783915 conn(0x7f6fc40711c0 msgr2=0x7f6fc40ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:36.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.010+0000 7f6fc961b700 1 -- 172.21.15.7:0/3918783915 shutdown_connections 2024-01-27T02:39:36.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.010+0000 7f6fc961b700 1 -- 172.21.15.7:0/3918783915 wait complete. 2024-01-27T02:39:36.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.012+0000 7f6fc961b700 1 Processor -- start 2024-01-27T02:39:36.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.012+0000 7f6fc961b700 1 -- start start 2024-01-27T02:39:36.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.012+0000 7f6fc961b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc40b2030 0x7f6fc41550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:36.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.012+0000 7f6fc961b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fc40ce0f0 0x7f6fc416a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:36.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.012+0000 7f6fc961b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fc415a450 0x7f6fc40b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:36.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.012+0000 7f6fc961b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc416deb0 con 0x7f6fc40b2030 2024-01-27T02:39:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.012+0000 7f6fc961b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40c7550 con 0x7f6fc415a450 2024-01-27T02:39:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.012+0000 7f6fc961b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40c7830 con 0x7f6fc40ce0f0 2024-01-27T02:39:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.012+0000 7f6fc2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc40b2030 0x7f6fc41550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.012+0000 7f6fc2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc40b2030 0x7f6fc41550f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34262/0 (socket says 172.21.15.7:34262) 2024-01-27T02:39:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.012+0000 7f6fc2ffd700 1 -- 172.21.15.7:0/895611175 learned_addr learned my addr 172.21.15.7:0/895611175 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.013+0000 7f6fc37fe700 1 --2- 172.21.15.7:0/895611175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fc415a450 0x7f6fc40b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.013+0000 7f6fc2ffd700 1 -- 172.21.15.7:0/895611175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fc40ce0f0 msgr2=0x7f6fc416a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.013+0000 7f6fc2ffd700 1 --2- 172.21.15.7:0/895611175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fc40ce0f0 0x7f6fc416a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.013+0000 7f6fc2ffd700 1 -- 172.21.15.7:0/895611175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fc415a450 msgr2=0x7f6fc40b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.013+0000 7f6fc2ffd700 1 --2- 172.21.15.7:0/895611175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fc415a450 0x7f6fc40b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.013+0000 7f6fc2ffd700 1 -- 172.21.15.7:0/895611175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb40093c0 con 0x7f6fc40b2030 2024-01-27T02:39:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.013+0000 7f6fc2ffd700 1 --2- 172.21.15.7:0/895611175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc40b2030 0x7f6fc41550f0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f6fc406e650 tx=0x7f6fb4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.013+0000 7f6faaffd700 1 -- 172.21.15.7:0/895611175 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4006290 con 0x7f6fc40b2030 2024-01-27T02:39:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.013+0000 7f6faaffd700 1 -- 172.21.15.7:0/895611175 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb40063f0 con 0x7f6fc40b2030 2024-01-27T02:39:36.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.013+0000 7f6faaffd700 1 -- 172.21.15.7:0/895611175 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb40067b0 con 0x7f6fc40b2030 2024-01-27T02:39:36.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.013+0000 7f6fc961b700 1 -- 172.21.15.7:0/895611175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc40c7b10 con 0x7f6fc40b2030 2024-01-27T02:39:36.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.013+0000 7f6fc961b700 1 -- 172.21.15.7:0/895611175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc40c80d0 con 0x7f6fc40b2030 2024-01-27T02:39:36.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.014+0000 7f6fc961b700 1 -- 172.21.15.7:0/895611175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc404c710 con 0x7f6fc40b2030 2024-01-27T02:39:36.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.015+0000 7f6faaffd700 1 -- 172.21.15.7:0/895611175 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6fb401b690 con 0x7f6fc40b2030 2024-01-27T02:39:36.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.015+0000 7f6faaffd700 1 --2- 172.21.15.7:0/895611175 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6fb0066f30 0x7f6fb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:36.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.015+0000 7f6faaffd700 1 -- 172.21.15.7:0/895611175 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6fb4086800 con 0x7f6fc40b2030 2024-01-27T02:39:36.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.016+0000 7f6fc27fc700 1 --2- 172.21.15.7:0/895611175 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6fb0066f30 0x7f6fb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:36.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.016+0000 7f6fc27fc700 1 --2- 172.21.15.7:0/895611175 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6fb0066f30 0x7f6fb00693e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4089110 tx=0x7f6fb8007a70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:36.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.020+0000 7f6faaffd700 1 -- 172.21.15.7:0/895611175 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6fb40539d0 con 0x7f6fc40b2030 2024-01-27T02:39:36.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.273+0000 7f6fc961b700 1 -- 172.21.15.7:0/895611175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fc40cd4c0 con 0x7f6fc40b2030 2024-01-27T02:39:36.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.274+0000 7f6faaffd700 1 -- 172.21.15.7:0/895611175 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f6fb4056ff0 con 0x7f6fc40b2030 2024-01-27T02:39:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:39:36.275 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6260,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3240780787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3240780787},{"type":"v1","addr":"172.21.15.188:6837","nonce":3240780787}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7156,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/4051709811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4051709811},{"type":"v1","addr":"172.21.15.188:6836","nonce":4051709811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7416,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/4000944335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":4000944335},{"type":"v1","addr":"172.21.15.7:6834","nonce":4000944335}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:39:22.327946+0000","modified":"2024-01-27T02:39:25.910241+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7419":{"gid":7419,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2386550039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2386550039},{"type":"v1","addr":"172.21.15.7:6835","nonce":2386550039}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2024-01-27T02:39:36.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.276+0000 7f6fc961b700 1 -- 172.21.15.7:0/895611175 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6fb0066f30 msgr2=0x7f6fb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:36.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.276+0000 7f6fc961b700 1 --2- 172.21.15.7:0/895611175 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6fb0066f30 0x7f6fb00693e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4089110 tx=0x7f6fb8007a70 comp rx=0 tx=0).stop 2024-01-27T02:39:36.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.276+0000 7f6fc961b700 1 -- 172.21.15.7:0/895611175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc40b2030 msgr2=0x7f6fc41550f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:36.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.276+0000 7f6fc961b700 1 --2- 172.21.15.7:0/895611175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc40b2030 0x7f6fc41550f0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f6fc406e650 tx=0x7f6fb4016000 comp rx=0 tx=0).stop 2024-01-27T02:39:36.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.276+0000 7f6fc961b700 1 -- 172.21.15.7:0/895611175 shutdown_connections 2024-01-27T02:39:36.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.276+0000 7f6fc961b700 1 --2- 172.21.15.7:0/895611175 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6fb0066f30 0x7f6fb00693e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:36.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.276+0000 7f6fc961b700 1 --2- 172.21.15.7:0/895611175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fc415a450 0x7f6fc40b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:36.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.276+0000 7f6fc961b700 1 --2- 172.21.15.7:0/895611175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc40b2030 0x7f6fc41550f0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:36.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.276+0000 7f6fc961b700 1 --2- 172.21.15.7:0/895611175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fc40ce0f0 0x7f6fc416a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:36.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.276+0000 7f6fc961b700 1 -- 172.21.15.7:0/895611175 >> 172.21.15.7:0/895611175 conn(0x7f6fc40711c0 msgr2=0x7f6fc41570f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:36.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.277+0000 7f6fc961b700 1 -- 172.21.15.7:0/895611175 shutdown_connections 2024-01-27T02:39:36.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.277+0000 7f6fc961b700 1 -- 172.21.15.7:0/895611175 wait complete. 2024-01-27T02:39:36.279 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 77 2024-01-27T02:39:36.291 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-01-27T02:39:36.444 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","whoami":0,"id":7419,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":13.544472782,"mdsmap_epoch":77,"osdmap_epoch":149,"osdmap_epoch_barrier":149,"uptime":19.243812757000001} 2024-01-27T02:39:36.444 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", "fs_name": "cephfs", "id": 7419, "mdsmap_epoch": 77, "osdmap_epoch": 149, "osdmap_epoch_barrier": 149, "rank_uptime": 13.544472782, "state": "up:active", "uptime": 19.243812757, "want_state": "up:active", "whoami": 0 } 2024-01-27T02:39:36.445 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-01-27T02:39:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.580+0000 7f0925263700 1 Processor -- start 2024-01-27T02:39:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.581+0000 7f0925263700 1 -- start start 2024-01-27T02:39:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.581+0000 7f0925263700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f092015a230 0x7f09200ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.581+0000 7f0925263700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f09200cd2f0 0x7f09200679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.581+0000 7f0925263700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0920067ef0 con 0x7f09200c45b0 2024-01-27T02:39:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.581+0000 7f0925263700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09200cd760 con 0x7f09200cd2f0 2024-01-27T02:39:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.581+0000 7f0925263700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09200c5320 con 0x7f092015a230 2024-01-27T02:39:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.582+0000 7f091e7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f09200cd2f0 0x7f09200679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.582+0000 7f091e7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f09200cd2f0 0x7f09200679b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34276/0 (socket says 172.21.15.7:34276) 2024-01-27T02:39:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.582+0000 7f091e7fc700 1 -- 172.21.15.7:0/1886701182 learned_addr learned my addr 172.21.15.7:0/1886701182 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.582+0000 7f091effd700 1 --2- 172.21.15.7:0/1886701182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f092015a230 0x7f09200ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.582+0000 7f091e7fc700 1 -- 172.21.15.7:0/1886701182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f092015a230 msgr2=0x7f09200ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:39:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.582+0000 7f091e7fc700 1 --2- 172.21.15.7:0/1886701182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f092015a230 0x7f09200ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:36.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.582+0000 7f091e7fc700 1 -- 172.21.15.7:0/1886701182 >> v1:172.21.15.188:6789/0 conn(0x7f09200c45b0 legacy=0x7f09200ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:36.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.582+0000 7f091e7fc700 1 -- 172.21.15.7:0/1886701182 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09200c9520 con 0x7f09200cd2f0 2024-01-27T02:39:36.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.582+0000 7f091e7fc700 1 --2- 172.21.15.7:0/1886701182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09200cd2f0 0x7f09200679b0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f09140099b0 tx=0x7f0914009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd13397a1e280cf server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:36.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.582+0000 7f091dffb700 1 -- 172.21.15.7:0/1886701182 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091400b860 con 0x7f09200cd2f0 2024-01-27T02:39:36.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.582+0000 7f091dffb700 1 -- 172.21.15.7:0/1886701182 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091400b9c0 con 0x7f09200cd2f0 2024-01-27T02:39:36.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.582+0000 7f091dffb700 1 -- 172.21.15.7:0/1886701182 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091400bd60 con 0x7f09200cd2f0 2024-01-27T02:39:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.583+0000 7f0925263700 1 -- 172.21.15.7:0/1886701182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09200cd2f0 msgr2=0x7f09200679b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.583+0000 7f0925263700 1 --2- 172.21.15.7:0/1886701182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09200cd2f0 0x7f09200679b0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f09140099b0 tx=0x7f0914009250 comp rx=0 tx=0).stop 2024-01-27T02:39:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.583+0000 7f0925263700 1 -- 172.21.15.7:0/1886701182 shutdown_connections 2024-01-27T02:39:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.583+0000 7f0925263700 1 --2- 172.21.15.7:0/1886701182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f092015a230 0x7f09200ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.583+0000 7f0925263700 1 --2- 172.21.15.7:0/1886701182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09200cd2f0 0x7f09200679b0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.583+0000 7f0925263700 1 -- 172.21.15.7:0/1886701182 >> 172.21.15.7:0/1886701182 conn(0x7f0920098020 msgr2=0x7f09200711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.583+0000 7f0925263700 1 -- 172.21.15.7:0/1886701182 shutdown_connections 2024-01-27T02:39:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.583+0000 7f0925263700 1 -- 172.21.15.7:0/1886701182 wait complete. 2024-01-27T02:39:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.584+0000 7f0925263700 1 Processor -- start 2024-01-27T02:39:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.584+0000 7f0925263700 1 -- start start 2024-01-27T02:39:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.585+0000 7f0925263700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09200c45b0 0x7f09200cece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.585+0000 7f0925263700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09200cd2f0 0x7f09200cf220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.585+0000 7f0925263700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f092015a230 0x7f09200ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.585+0000 7f0925263700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09200c8c60 con 0x7f09200cd2f0 2024-01-27T02:39:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.585+0000 7f0925263700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09200b1c30 con 0x7f092015a230 2024-01-27T02:39:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.585+0000 7f0925263700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09201550d0 con 0x7f09200c45b0 2024-01-27T02:39:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.585+0000 7f091e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09200cd2f0 0x7f09200cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.585+0000 7f091e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09200cd2f0 0x7f09200cf220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34284/0 (socket says 172.21.15.7:34284) 2024-01-27T02:39:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.585+0000 7f091e7fc700 1 -- 172.21.15.7:0/2146730711 learned_addr learned my addr 172.21.15.7:0/2146730711 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:39:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.585+0000 7f091effd700 1 --2- 172.21.15.7:0/2146730711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09200c45b0 0x7f09200cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:36.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.585+0000 7f091f7fe700 1 --2- 172.21.15.7:0/2146730711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f092015a230 0x7f09200ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:36.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.586+0000 7f091e7fc700 1 -- 172.21.15.7:0/2146730711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09200c45b0 msgr2=0x7f09200cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:36.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.586+0000 7f091e7fc700 1 --2- 172.21.15.7:0/2146730711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09200c45b0 0x7f09200cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:36.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.586+0000 7f091e7fc700 1 -- 172.21.15.7:0/2146730711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f092015a230 msgr2=0x7f09200ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:36.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.586+0000 7f091e7fc700 1 --2- 172.21.15.7:0/2146730711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f092015a230 0x7f09200ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:36.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.586+0000 7f091e7fc700 1 -- 172.21.15.7:0/2146730711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09140094f0 con 0x7f09200cd2f0 2024-01-27T02:39:36.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.586+0000 7f091effd700 1 --2- 172.21.15.7:0/2146730711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09200c45b0 0x7f09200cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:39:36.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.586+0000 7f091e7fc700 1 --2- 172.21.15.7:0/2146730711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09200cd2f0 0x7f09200cf220 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f092006ea80 tx=0x7f0914017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:36.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.586+0000 7f0906ffd700 1 -- 172.21.15.7:0/2146730711 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914024440 con 0x7f09200cd2f0 2024-01-27T02:39:36.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.586+0000 7f0906ffd700 1 -- 172.21.15.7:0/2146730711 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09140245c0 con 0x7f09200cd2f0 2024-01-27T02:39:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.586+0000 7f0925263700 1 -- 172.21.15.7:0/2146730711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09200c83c0 con 0x7f09200cd2f0 2024-01-27T02:39:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.586+0000 7f0906ffd700 1 -- 172.21.15.7:0/2146730711 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914024980 con 0x7f09200cd2f0 2024-01-27T02:39:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.586+0000 7f0925263700 1 -- 172.21.15.7:0/2146730711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09200c8980 con 0x7f09200cd2f0 2024-01-27T02:39:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.587+0000 7f0925263700 1 -- 172.21.15.7:0/2146730711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f092004c710 con 0x7f09200cd2f0 2024-01-27T02:39:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.588+0000 7f0906ffd700 1 -- 172.21.15.7:0/2146730711 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f091401a420 con 0x7f09200cd2f0 2024-01-27T02:39:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.588+0000 7f0906ffd700 1 --2- 172.21.15.7:0/2146730711 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f090c066f30 0x7f090c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:39:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.588+0000 7f0906ffd700 1 -- 172.21.15.7:0/2146730711 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f09140863b0 con 0x7f09200cd2f0 2024-01-27T02:39:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.588+0000 7f091effd700 1 --2- 172.21.15.7:0/2146730711 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f090c066f30 0x7f090c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:39:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.589+0000 7f091effd700 1 --2- 172.21.15.7:0/2146730711 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f090c066f30 0x7f090c0693e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0910006aa0 tx=0x7f0910008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:39:36.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.592+0000 7f0906ffd700 1 -- 172.21.15.7:0/2146730711 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0914053620 con 0x7f09200cd2f0 2024-01-27T02:39:36.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.794+0000 7f0925263700 1 -- 172.21.15.7:0/2146730711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f09200c9800 con 0x7f09200cd2f0 2024-01-27T02:39:36.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.946+0000 7f0906ffd700 1 -- 172.21.15.7:0/2146730711 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v150) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0914056c40 con 0x7f09200cd2f0 2024-01-27T02:39:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.949+0000 7f0925263700 1 -- 172.21.15.7:0/2146730711 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f090c066f30 msgr2=0x7f090c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.949+0000 7f0925263700 1 --2- 172.21.15.7:0/2146730711 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f090c066f30 0x7f090c0693e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0910006aa0 tx=0x7f0910008040 comp rx=0 tx=0).stop 2024-01-27T02:39:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.949+0000 7f0925263700 1 -- 172.21.15.7:0/2146730711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09200cd2f0 msgr2=0x7f09200cf220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:39:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.949+0000 7f0925263700 1 --2- 172.21.15.7:0/2146730711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09200cd2f0 0x7f09200cf220 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f092006ea80 tx=0x7f0914017000 comp rx=0 tx=0).stop 2024-01-27T02:39:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.949+0000 7f0925263700 1 -- 172.21.15.7:0/2146730711 shutdown_connections 2024-01-27T02:39:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.949+0000 7f0925263700 1 --2- 172.21.15.7:0/2146730711 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f090c066f30 0x7f090c0693e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.949+0000 7f0925263700 1 --2- 172.21.15.7:0/2146730711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f092015a230 0x7f09200ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.949+0000 7f0925263700 1 --2- 172.21.15.7:0/2146730711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09200c45b0 0x7f09200cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.949+0000 7f0925263700 1 --2- 172.21.15.7:0/2146730711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09200cd2f0 0x7f09200cf220 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:39:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.949+0000 7f0925263700 1 -- 172.21.15.7:0/2146730711 >> 172.21.15.7:0/2146730711 conn(0x7f0920098020 msgr2=0x7f0920068180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:39:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.949+0000 7f0925263700 1 -- 172.21.15.7:0/2146730711 shutdown_connections 2024-01-27T02:39:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:39:36.949+0000 7f0925263700 1 -- 172.21.15.7:0/2146730711 wait complete. 2024-01-27T02:39:36.951 INFO:teuthology.orchestra.run.smithi007.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-01-27T02:39:36.965 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-01-27T02:39:37.118 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_mon_report_interval":"5"} 2024-01-27T02:39:37.118 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-01-27T02:39:37.118 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-01-27T02:39:37.261 INFO:teuthology.orchestra.run.smithi007.stdout:{"mon_tick_interval":"5"} 2024-01-27T02:39:37.261 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-01-27T02:39:37.261 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2024-01-27T02:39:37.261 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-01-27T02:39:37.262 DEBUG:teuthology.orchestra.run.smithi007:> import time 2024-01-27T02:39:37.262 DEBUG:teuthology.orchestra.run.smithi007:> import datetime 2024-01-27T02:39:37.262 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2024-01-27T02:39:37.262 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-01-27T02:39:37.262 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:39:37.262 DEBUG:teuthology.orchestra.run.smithi007:> # Write some buffered data through before going full, all should be well 2024-01-27T02:39:37.262 DEBUG:teuthology.orchestra.run.smithi007:> print("writing some data through which we expect to succeed") 2024-01-27T02:39:37.262 DEBUG:teuthology.orchestra.run.smithi007:> bytes = 0 2024-01-27T02:39:37.262 DEBUG:teuthology.orchestra.run.smithi007:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-01-27T02:39:37.262 DEBUG:teuthology.orchestra.run.smithi007:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-01-27T02:39:37.262 DEBUG:teuthology.orchestra.run.smithi007:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2024-01-27T02:39:37.262 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f) 2024-01-27T02:39:37.262 DEBUG:teuthology.orchestra.run.smithi007:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-01-27T02:39:37.262 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:39:37.262 DEBUG:teuthology.orchestra.run.smithi007:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-01-27T02:39:37.262 DEBUG:teuthology.orchestra.run.smithi007:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-01-27T02:39:37.263 DEBUG:teuthology.orchestra.run.smithi007:> # buffered IO, the error should always only appear from fsync and not 2024-01-27T02:39:37.263 DEBUG:teuthology.orchestra.run.smithi007:> # from write 2024-01-27T02:39:37.263 DEBUG:teuthology.orchestra.run.smithi007:> full = False 2024-01-27T02:39:37.263 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:39:37.263 DEBUG:teuthology.orchestra.run.smithi007:> for n in range(0, int(32 * 0.9)): 2024-01-27T02:39:37.263 DEBUG:teuthology.orchestra.run.smithi007:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-01-27T02:39:37.263 DEBUG:teuthology.orchestra.run.smithi007:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2024-01-27T02:39:37.263 DEBUG:teuthology.orchestra.run.smithi007:> print("done writing {0} bytes".format(bytes)) 2024-01-27T02:39:37.263 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:39:37.263 DEBUG:teuthology.orchestra.run.smithi007:> # OK, now we should sneak in under the full condition 2024-01-27T02:39:37.263 DEBUG:teuthology.orchestra.run.smithi007:> # due to the time it takes the OSDs to report to the 2024-01-27T02:39:37.263 DEBUG:teuthology.orchestra.run.smithi007:> # mons, and get a successful fsync on our full-making data 2024-01-27T02:39:37.263 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f) 2024-01-27T02:39:37.263 DEBUG:teuthology.orchestra.run.smithi007:> print("successfully fsync'"'"'ed prior to getting full state reported") 2024-01-27T02:39:37.263 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:39:37.263 DEBUG:teuthology.orchestra.run.smithi007:> # buffered write, add more dirty data to the buffer 2024-01-27T02:39:37.264 DEBUG:teuthology.orchestra.run.smithi007:> print("starting buffered write") 2024-01-27T02:39:37.264 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-01-27T02:39:37.264 DEBUG:teuthology.orchestra.run.smithi007:> for n in range(0, int(32 * 0.2)): 2024-01-27T02:39:37.264 DEBUG:teuthology.orchestra.run.smithi007:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-01-27T02:39:37.264 DEBUG:teuthology.orchestra.run.smithi007:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-01-27T02:39:37.264 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(15.0) 2024-01-27T02:39:37.264 DEBUG:teuthology.orchestra.run.smithi007:> except OSError: 2024-01-27T02:39:37.264 DEBUG:teuthology.orchestra.run.smithi007:> pass; 2024-01-27T02:39:37.264 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:39:37.264 DEBUG:teuthology.orchestra.run.smithi007:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2024-01-27T02:39:37.264 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:39:37.264 DEBUG:teuthology.orchestra.run.smithi007:> # Wait long enough for a background flush that should fail 2024-01-27T02:39:37.264 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(30) 2024-01-27T02:39:37.264 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:39:37.265 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-01-27T02:39:37.265 DEBUG:teuthology.orchestra.run.smithi007:> # ...and check that the failed background flush is reflected in fclose 2024-01-27T02:39:37.265 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-01-27T02:39:37.265 DEBUG:teuthology.orchestra.run.smithi007:> os.close(f) 2024-01-27T02:39:37.265 DEBUG:teuthology.orchestra.run.smithi007:> except OSError: 2024-01-27T02:39:37.265 DEBUG:teuthology.orchestra.run.smithi007:> print("close() returned an error as expected") 2024-01-27T02:39:37.265 DEBUG:teuthology.orchestra.run.smithi007:> else: 2024-01-27T02:39:37.265 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("close() failed to raise error") 2024-01-27T02:39:37.265 DEBUG:teuthology.orchestra.run.smithi007:> else: 2024-01-27T02:39:37.265 DEBUG:teuthology.orchestra.run.smithi007:> # The kernel cephfs client does not raise errors on fclose 2024-01-27T02:39:37.265 DEBUG:teuthology.orchestra.run.smithi007:> os.close(f) 2024-01-27T02:39:37.266 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:39:37.266 DEBUG:teuthology.orchestra.run.smithi007:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-01-27T02:39:37.266 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-01-27T02:39:38.942 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:39:38.945 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:40:09.016 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:40:09.019 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:40:39.086 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:40:39.089 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:41:09.180 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:41:09.182 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:41:39.267 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:41:39.271 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:41:42.370 INFO:teuthology.orchestra.run.smithi007.stdout:writing some data through which we expect to succeed 2024-01-27T02:41:42.370 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-01-27T02:41:42.370 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 1572864 bytes via buffered write, may repeat 2024-01-27T02:41:42.370 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 2621440 bytes via buffered write, may repeat 2024-01-27T02:41:42.370 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 3670016 bytes via buffered write, may repeat 2024-01-27T02:41:42.370 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 4718592 bytes via buffered write, may repeat 2024-01-27T02:41:42.370 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 5767168 bytes via buffered write, may repeat 2024-01-27T02:41:42.370 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 6815744 bytes via buffered write, may repeat 2024-01-27T02:41:42.370 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 7864320 bytes via buffered write, may repeat 2024-01-27T02:41:42.371 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 8912896 bytes via buffered write, may repeat 2024-01-27T02:41:42.371 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 9961472 bytes via buffered write, may repeat 2024-01-27T02:41:42.371 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 11010048 bytes via buffered write, may repeat 2024-01-27T02:41:42.371 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 12058624 bytes via buffered write, may repeat 2024-01-27T02:41:42.371 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 13107200 bytes via buffered write, may repeat 2024-01-27T02:41:42.371 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 14155776 bytes via buffered write, may repeat 2024-01-27T02:41:42.371 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 15204352 bytes via buffered write, may repeat 2024-01-27T02:41:42.371 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 16252928 bytes via buffered write, may repeat 2024-01-27T02:41:42.371 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 17301504 bytes via buffered write, may repeat 2024-01-27T02:41:42.371 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 18350080 bytes via buffered write, may repeat 2024-01-27T02:41:42.371 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 19398656 bytes via buffered write, may repeat 2024-01-27T02:41:42.371 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 20447232 bytes via buffered write, may repeat 2024-01-27T02:41:42.371 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 21495808 bytes via buffered write, may repeat 2024-01-27T02:41:42.371 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 22544384 bytes via buffered write, may repeat 2024-01-27T02:41:42.371 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 23592960 bytes via buffered write, may repeat 2024-01-27T02:41:42.371 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 24641536 bytes via buffered write, may repeat 2024-01-27T02:41:42.372 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 25690112 bytes via buffered write, may repeat 2024-01-27T02:41:42.372 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 26738688 bytes via buffered write, may repeat 2024-01-27T02:41:42.372 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 27787264 bytes via buffered write, may repeat 2024-01-27T02:41:42.372 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 28835840 bytes via buffered write, may repeat 2024-01-27T02:41:42.372 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 29884416 bytes via buffered write, may repeat 2024-01-27T02:41:42.372 INFO:teuthology.orchestra.run.smithi007.stdout:done writing 29884416 bytes 2024-01-27T02:41:42.372 INFO:teuthology.orchestra.run.smithi007.stdout:successfully fsync'ed prior to getting full state reported 2024-01-27T02:41:42.372 INFO:teuthology.orchestra.run.smithi007.stdout:starting buffered write 2024-01-27T02:41:42.372 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-27T02:41:42.372 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-27T02:41:42.372 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-27T02:41:42.372 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-27T02:41:42.373 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-27T02:41:42.373 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-27T02:41:42.373 INFO:teuthology.orchestra.run.smithi007.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2024-01-27T02:41:42.513 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-27T02:41:42.513 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:41:42.563 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:41:42.616 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-27T02:41:42.616 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-27T02:41:42.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:41:42.616 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-27T02:41:42.672 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:41:42.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:41:42.672 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-27T02:41:42.693 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-27T02:41:42.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:41:42.693 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-01-27T02:41:42.737 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:41:42.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:41:42.738 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-27T02:41:42.797 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:41:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.939+0000 7fa160e71700 1 Processor -- start 2024-01-27T02:41:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.939+0000 7fa160e71700 1 -- start start 2024-01-27T02:41:42.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.940+0000 7fa160e71700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa15c0c9bb0 0x7fa15c157f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:42.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.940+0000 7fa160e71700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa15c0c9f80 0x7fa15c06b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:42.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.940+0000 7fa160e71700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa15c06c880 con 0x7fa15c06e330 2024-01-27T02:41:42.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.940+0000 7fa160e71700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa15c06cd10 con 0x7fa15c0c9f80 2024-01-27T02:41:42.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.940+0000 7fa160e71700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c06bc70 con 0x7fa15c0c9bb0 2024-01-27T02:41:42.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.940+0000 7fa15ad9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa15c06e330 0x7fa15c0c8270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52426/0 (socket says 172.21.15.7:52426) 2024-01-27T02:41:42.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.940+0000 7fa15ad9d700 1 -- 172.21.15.7:0/29998068 learned_addr learned my addr 172.21.15.7:0/29998068 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:41:42.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.940+0000 7fa15a59c700 1 --2- 172.21.15.7:0/29998068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa15c0c9bb0 0x7fa15c157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:42.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.940+0000 7fa159d9b700 1 --2- 172.21.15.7:0/29998068 >> v2:172.21.15.188:3300/0 conn(0x7fa15c0c9f80 0x7fa15c06b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:42.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.940+0000 7fa15959a700 1 -- 172.21.15.7:0/29998068 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2990058386 0 0) 0x7fa15c06c880 con 0x7fa15c06e330 2024-01-27T02:41:42.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.940+0000 7fa15959a700 1 -- 172.21.15.7:0/29998068 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa144003570 con 0x7fa15c06e330 2024-01-27T02:41:42.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa15959a700 1 -- 172.21.15.7:0/29998068 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3332701197 0 0) 0x7fa144003570 con 0x7fa15c06e330 2024-01-27T02:41:42.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa15959a700 1 -- 172.21.15.7:0/29998068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa15c0c9bb0 msgr2=0x7fa15c157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:42.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa15959a700 1 --2- 172.21.15.7:0/29998068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa15c0c9bb0 0x7fa15c157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:42.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa15959a700 1 -- 172.21.15.7:0/29998068 >> v2:172.21.15.188:3300/0 conn(0x7fa15c0c9f80 msgr2=0x7fa15c06b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:42.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa15959a700 1 --2- 172.21.15.7:0/29998068 >> v2:172.21.15.188:3300/0 conn(0x7fa15c0c9f80 0x7fa15c06b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:42.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa15959a700 1 -- 172.21.15.7:0/29998068 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c06bf50 con 0x7fa15c06e330 2024-01-27T02:41:42.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa15959a700 1 -- 172.21.15.7:0/29998068 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7fa1540017f0 con 0x7fa15c06e330 2024-01-27T02:41:42.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa15959a700 1 -- 172.21.15.7:0/29998068 >> v1:172.21.15.7:6789/0 conn(0x7fa15c06e330 legacy=0x7fa15c0c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:42.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa15959a700 1 --2- 172.21.15.7:0/29998068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa15c0c9bb0 0x7fa144003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:42.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa15959a700 1 --2- 172.21.15.7:0/29998068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa144003fe0 0x7fa144004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:42.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa15959a700 1 --2- 172.21.15.7:0/29998068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa144004d50 0x7fa1440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:42.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa15959a700 1 -- 172.21.15.7:0/29998068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c06cd10 con 0x7fa144004d50 2024-01-27T02:41:42.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa15959a700 1 -- 172.21.15.7:0/29998068 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c06bc70 con 0x7fa144003fe0 2024-01-27T02:41:42.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa15959a700 1 -- 172.21.15.7:0/29998068 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14400ba90 con 0x7fa15c0c9bb0 2024-01-27T02:41:42.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa15ad9d700 1 --2- 172.21.15.7:0/29998068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa144004d50 0x7fa1440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:42.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa15a59c700 1 --2- 172.21.15.7:0/29998068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa15c0c9bb0 0x7fa144003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:42.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa159d9b700 1 --2- 172.21.15.7:0/29998068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa144003fe0 0x7fa144004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:42.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa15ad9d700 1 -- 172.21.15.7:0/29998068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa15c0c9bb0 msgr2=0x7fa144003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:42.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa15ad9d700 1 --2- 172.21.15.7:0/29998068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa15c0c9bb0 0x7fa144003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:42.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa15ad9d700 1 -- 172.21.15.7:0/29998068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa144003fe0 msgr2=0x7fa144004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:42.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa15ad9d700 1 --2- 172.21.15.7:0/29998068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa144003fe0 0x7fa144004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:42.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa15ad9d700 1 -- 172.21.15.7:0/29998068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa144003570 con 0x7fa144004d50 2024-01-27T02:41:42.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.941+0000 7fa15ad9d700 1 --2- 172.21.15.7:0/29998068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa144004d50 0x7fa1440051c0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fa15400a420 tx=0x7fa15c157440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d09ab4af773962d server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:42.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.942+0000 7fa15959a700 1 -- 172.21.15.7:0/29998068 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1540104a0 con 0x7fa144004d50 2024-01-27T02:41:42.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.942+0000 7fa15959a700 1 -- 172.21.15.7:0/29998068 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa154010600 con 0x7fa144004d50 2024-01-27T02:41:42.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.942+0000 7fa15959a700 1 -- 172.21.15.7:0/29998068 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1540109c0 con 0x7fa144004d50 2024-01-27T02:41:42.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.942+0000 7fa160e71700 1 -- 172.21.15.7:0/29998068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa144004d50 msgr2=0x7fa1440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:42.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.942+0000 7fa160e71700 1 --2- 172.21.15.7:0/29998068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa144004d50 0x7fa1440051c0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fa15400a420 tx=0x7fa15c157440 comp rx=0 tx=0).stop 2024-01-27T02:41:42.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.942+0000 7fa15ad9d700 1 -- 172.21.15.7:0/29998068 reap_dead start 2024-01-27T02:41:42.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.942+0000 7fa160e71700 1 -- 172.21.15.7:0/29998068 shutdown_connections 2024-01-27T02:41:42.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.942+0000 7fa160e71700 1 -- 172.21.15.7:0/29998068 >> 172.21.15.7:0/29998068 conn(0x7fa15c0cdc30 msgr2=0x7fa15c0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:42.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.942+0000 7fa160e71700 1 -- 172.21.15.7:0/29998068 shutdown_connections 2024-01-27T02:41:42.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.942+0000 7fa160e71700 1 -- 172.21.15.7:0/29998068 wait complete. 2024-01-27T02:41:42.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.943+0000 7fa160e71700 1 Processor -- start 2024-01-27T02:41:42.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.944+0000 7fa160e71700 1 -- start start 2024-01-27T02:41:42.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.944+0000 7fa160e71700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa144004d50 0x7fa15c147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:42.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.944+0000 7fa160e71700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa15c169e60 0x7fa15c159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:42.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.944+0000 7fa160e71700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa15c145fa0 0x7fa15c146440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:42.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.944+0000 7fa160e71700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c146980 con 0x7fa144004d50 2024-01-27T02:41:42.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.944+0000 7fa160e71700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c15a530 con 0x7fa15c169e60 2024-01-27T02:41:42.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.944+0000 7fa160e71700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c149150 con 0x7fa15c145fa0 2024-01-27T02:41:42.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.944+0000 7fa15a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa144004d50 0x7fa15c147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:42.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.944+0000 7fa15a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa144004d50 0x7fa15c147f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40682/0 (socket says 172.21.15.7:40682) 2024-01-27T02:41:42.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.944+0000 7fa15a59c700 1 -- 172.21.15.7:0/601972512 learned_addr learned my addr 172.21.15.7:0/601972512 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.944+0000 7fa15ad9d700 1 --2- 172.21.15.7:0/601972512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa15c145fa0 0x7fa15c146440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.945+0000 7fa159d9b700 1 --2- 172.21.15.7:0/601972512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa15c169e60 0x7fa15c159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.945+0000 7fa15a59c700 1 -- 172.21.15.7:0/601972512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa15c145fa0 msgr2=0x7fa15c146440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.945+0000 7fa15a59c700 1 --2- 172.21.15.7:0/601972512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa15c145fa0 0x7fa15c146440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.945+0000 7fa15a59c700 1 -- 172.21.15.7:0/601972512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa15c169e60 msgr2=0x7fa15c159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.945+0000 7fa15a59c700 1 --2- 172.21.15.7:0/601972512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa15c169e60 0x7fa15c159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.945+0000 7fa15a59c700 1 -- 172.21.15.7:0/601972512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c139870 con 0x7fa144004d50 2024-01-27T02:41:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.945+0000 7fa15a59c700 1 --2- 172.21.15.7:0/601972512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa144004d50 0x7fa15c147f50 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fa14c006ac0 tx=0x7fa14c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.945+0000 7fa14a7fc700 1 -- 172.21.15.7:0/601972512 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c00f8e0 con 0x7fa144004d50 2024-01-27T02:41:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.945+0000 7fa14a7fc700 1 -- 172.21.15.7:0/601972512 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14c00fa40 con 0x7fa144004d50 2024-01-27T02:41:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.945+0000 7fa160e71700 1 -- 172.21.15.7:0/601972512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa15c148540 con 0x7fa144004d50 2024-01-27T02:41:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.945+0000 7fa14a7fc700 1 -- 172.21.15.7:0/601972512 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c00fdc0 con 0x7fa144004d50 2024-01-27T02:41:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.945+0000 7fa160e71700 1 -- 172.21.15.7:0/601972512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa15c148b00 con 0x7fa144004d50 2024-01-27T02:41:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.946+0000 7fa160e71700 1 -- 172.21.15.7:0/601972512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa15c04c710 con 0x7fa144004d50 2024-01-27T02:41:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.947+0000 7fa14a7fc700 1 -- 172.21.15.7:0/601972512 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa14c014650 con 0x7fa144004d50 2024-01-27T02:41:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.947+0000 7fa14a7fc700 1 --2- 172.21.15.7:0/601972512 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa13c066f30 0x7fa13c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.947+0000 7fa14a7fc700 1 -- 172.21.15.7:0/601972512 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fa14c018030 con 0x7fa144004d50 2024-01-27T02:41:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.947+0000 7fa14a7fc700 1 -- 172.21.15.7:0/601972512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=154}) v3 -- 0x7fa13c06eb10 con 0x7fa144004d50 2024-01-27T02:41:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.947+0000 7fa159d9b700 1 --2- 172.21.15.7:0/601972512 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa13c066f30 0x7fa13c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.948+0000 7fa159d9b700 1 --2- 172.21.15.7:0/601972512 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa13c066f30 0x7fa13c0693e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0b2820 tx=0x7fa1500091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:42.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:42.952+0000 7fa14a7fc700 1 -- 172.21.15.7:0/601972512 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa14c022060 con 0x7fa144004d50 2024-01-27T02:41:43.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.210+0000 7fa160e71700 1 -- 172.21.15.7:0/601972512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa15c145d60 con 0x7fa144004d50 2024-01-27T02:41:43.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.210+0000 7fa14a7fc700 1 -- 172.21.15.7:0/601972512 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fa14c0520d0 con 0x7fa144004d50 2024-01-27T02:41:43.211 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:41:43.211 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6260,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3240780787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3240780787},{"type":"v1","addr":"172.21.15.188:6837","nonce":3240780787}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7156,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/4051709811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4051709811},{"type":"v1","addr":"172.21.15.188:6836","nonce":4051709811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7416,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/4000944335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":4000944335},{"type":"v1","addr":"172.21.15.7:6834","nonce":4000944335}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:39:22.327946+0000","modified":"2024-01-27T02:39:25.910241+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7419":{"gid":7419,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2386550039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2386550039},{"type":"v1","addr":"172.21.15.7:6835","nonce":2386550039}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2024-01-27T02:41:43.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.213+0000 7fa160e71700 1 -- 172.21.15.7:0/601972512 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa13c066f30 msgr2=0x7fa13c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:43.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.213+0000 7fa160e71700 1 --2- 172.21.15.7:0/601972512 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa13c066f30 0x7fa13c0693e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0b2820 tx=0x7fa1500091a0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.213+0000 7fa160e71700 1 -- 172.21.15.7:0/601972512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa144004d50 msgr2=0x7fa15c147f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:43.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.213+0000 7fa160e71700 1 --2- 172.21.15.7:0/601972512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa144004d50 0x7fa15c147f50 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fa14c006ac0 tx=0x7fa14c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.213+0000 7fa160e71700 1 -- 172.21.15.7:0/601972512 shutdown_connections 2024-01-27T02:41:43.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.213+0000 7fa160e71700 1 --2- 172.21.15.7:0/601972512 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa13c066f30 0x7fa13c0693e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.213+0000 7fa160e71700 1 --2- 172.21.15.7:0/601972512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa15c145fa0 0x7fa15c146440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.213+0000 7fa160e71700 1 --2- 172.21.15.7:0/601972512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa144004d50 0x7fa15c147f50 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.213+0000 7fa160e71700 1 --2- 172.21.15.7:0/601972512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa15c169e60 0x7fa15c159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.213+0000 7fa160e71700 1 -- 172.21.15.7:0/601972512 >> 172.21.15.7:0/601972512 conn(0x7fa15c0cdc30 msgr2=0x7fa15c0c2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:43.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.213+0000 7fa160e71700 1 -- 172.21.15.7:0/601972512 shutdown_connections 2024-01-27T02:41:43.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.213+0000 7fa160e71700 1 -- 172.21.15.7:0/601972512 wait complete. 2024-01-27T02:41:43.214 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 77 2024-01-27T02:41:43.227 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:41:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.359+0000 7f0a2116b700 1 Processor -- start 2024-01-27T02:41:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.359+0000 7f0a2116b700 1 -- start start 2024-01-27T02:41:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.359+0000 7f0a2116b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0a1c0c4cc0 0x7f0a1c0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.359+0000 7f0a2116b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a1c0ce5e0 con 0x7f0a1c15a100 2024-01-27T02:41:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.360+0000 7f0a2116b700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a1c0cd9d0 con 0x7f0a1c0ca540 2024-01-27T02:41:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.360+0000 7f0a2116b700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a1c0cdcb0 con 0x7f0a1c0c4cc0 2024-01-27T02:41:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.360+0000 7f0a1ad9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0a1c15a100 0x7f0a1c0c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52434/0 (socket says 172.21.15.7:52434) 2024-01-27T02:41:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.360+0000 7f0a1ad9d700 1 -- 172.21.15.7:0/356000507 learned_addr learned my addr 172.21.15.7:0/356000507 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:41:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.360+0000 7f0a1a59c700 1 --2- 172.21.15.7:0/356000507 >> v2:172.21.15.188:3300/0 conn(0x7f0a1c0c4cc0 0x7f0a1c0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.360+0000 7f0a19d9b700 1 -- 172.21.15.7:0/356000507 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3677323125 0 0) 0x7f0a1c0ce5e0 con 0x7f0a1c15a100 2024-01-27T02:41:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.360+0000 7f0a19d9b700 1 -- 172.21.15.7:0/356000507 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a04003570 con 0x7f0a1c15a100 2024-01-27T02:41:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.360+0000 7f0a19d9b700 1 -- 172.21.15.7:0/356000507 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638956075 0 0) 0x7f0a1c0cd9d0 con 0x7f0a1c0ca540 2024-01-27T02:41:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.360+0000 7f0a19d9b700 1 -- 172.21.15.7:0/356000507 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a1c0ce5e0 con 0x7f0a1c0ca540 2024-01-27T02:41:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.360+0000 7f0a1a59c700 1 -- 172.21.15.7:0/356000507 >> v1:172.21.15.188:6789/0 conn(0x7f0a1c0ca540 legacy=0x7f0a1c163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.360+0000 7f0a1a59c700 1 -- 172.21.15.7:0/356000507 >> v1:172.21.15.7:6789/0 conn(0x7f0a1c15a100 legacy=0x7f0a1c0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.360+0000 7f0a1a59c700 1 -- 172.21.15.7:0/356000507 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1c0cdf90 con 0x7f0a1c0c4cc0 2024-01-27T02:41:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.361+0000 7f0a1a59c700 1 --2- 172.21.15.7:0/356000507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a1c0c4cc0 0x7f0a1c0cd2f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0a100098e0 tx=0x7f0a100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2aa35025e56b0be server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.361+0000 7f0a19d9b700 1 -- 172.21.15.7:0/356000507 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10011630 con 0x7f0a1c0c4cc0 2024-01-27T02:41:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.361+0000 7f0a19d9b700 1 -- 172.21.15.7:0/356000507 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a10011790 con 0x7f0a1c0c4cc0 2024-01-27T02:41:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.361+0000 7f0a19d9b700 1 -- 172.21.15.7:0/356000507 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10011b30 con 0x7f0a1c0c4cc0 2024-01-27T02:41:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.361+0000 7f0a2116b700 1 -- 172.21.15.7:0/356000507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a1c0c4cc0 msgr2=0x7f0a1c0cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:43.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.361+0000 7f0a2116b700 1 --2- 172.21.15.7:0/356000507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a1c0c4cc0 0x7f0a1c0cd2f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0a100098e0 tx=0x7f0a100091a0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.361+0000 7f0a2116b700 1 -- 172.21.15.7:0/356000507 shutdown_connections 2024-01-27T02:41:43.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.361+0000 7f0a2116b700 1 --2- 172.21.15.7:0/356000507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a1c0c4cc0 0x7f0a1c0cd2f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.361+0000 7f0a2116b700 1 -- 172.21.15.7:0/356000507 >> 172.21.15.7:0/356000507 conn(0x7f0a1c098020 msgr2=0x7f0a1c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:43.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.361+0000 7f0a2116b700 1 -- 172.21.15.7:0/356000507 shutdown_connections 2024-01-27T02:41:43.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.361+0000 7f0a2116b700 1 -- 172.21.15.7:0/356000507 wait complete. 2024-01-27T02:41:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.363+0000 7f0a2116b700 1 Processor -- start 2024-01-27T02:41:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.363+0000 7f0a2116b700 1 -- start start 2024-01-27T02:41:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.363+0000 7f0a2116b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a1c0c4cc0 0x7f0a1c0b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.363+0000 7f0a2116b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a1c0ca540 0x7f0a1c16dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.363+0000 7f0a2116b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a1c15a100 0x7f0a1c149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.364+0000 7f0a2116b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c16e270 con 0x7f0a1c15a100 2024-01-27T02:41:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.364+0000 7f0a2116b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c0c71b0 con 0x7f0a1c0ca540 2024-01-27T02:41:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.364+0000 7f0a2116b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c16e4a0 con 0x7f0a1c0c4cc0 2024-01-27T02:41:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.364+0000 7f0a1b59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a1c15a100 0x7f0a1c149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.364+0000 7f0a1b59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a1c15a100 0x7f0a1c149aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40692/0 (socket says 172.21.15.7:40692) 2024-01-27T02:41:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.364+0000 7f0a1b59e700 1 -- 172.21.15.7:0/3363361266 learned_addr learned my addr 172.21.15.7:0/3363361266 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:43.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.364+0000 7f0a1a59c700 1 --2- 172.21.15.7:0/3363361266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a1c0ca540 0x7f0a1c16dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:43.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.364+0000 7f0a1ad9d700 1 --2- 172.21.15.7:0/3363361266 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a1c0c4cc0 0x7f0a1c0b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:43.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.364+0000 7f0a1b59e700 1 -- 172.21.15.7:0/3363361266 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a1c0c4cc0 msgr2=0x7f0a1c0b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:43.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.364+0000 7f0a1b59e700 1 --2- 172.21.15.7:0/3363361266 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a1c0c4cc0 0x7f0a1c0b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.364+0000 7f0a1b59e700 1 -- 172.21.15.7:0/3363361266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a1c0ca540 msgr2=0x7f0a1c16dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:43.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.364+0000 7f0a1b59e700 1 --2- 172.21.15.7:0/3363361266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a1c0ca540 0x7f0a1c16dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.364+0000 7f0a1b59e700 1 -- 172.21.15.7:0/3363361266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a10009420 con 0x7f0a1c15a100 2024-01-27T02:41:43.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.364+0000 7f0a1b59e700 1 --2- 172.21.15.7:0/3363361266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a1c15a100 0x7f0a1c149aa0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400c910 tx=0x7f0a140027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:43.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.364+0000 7f0a02ffd700 1 -- 172.21.15.7:0/3363361266 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14012630 con 0x7f0a1c15a100 2024-01-27T02:41:43.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.364+0000 7f0a02ffd700 1 -- 172.21.15.7:0/3363361266 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a140127b0 con 0x7f0a1c15a100 2024-01-27T02:41:43.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.364+0000 7f0a2116b700 1 -- 172.21.15.7:0/3363361266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1c16e780 con 0x7f0a1c15a100 2024-01-27T02:41:43.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.364+0000 7f0a02ffd700 1 -- 172.21.15.7:0/3363361266 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14012b70 con 0x7f0a1c15a100 2024-01-27T02:41:43.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.365+0000 7f0a2116b700 1 -- 172.21.15.7:0/3363361266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1c149fe0 con 0x7f0a1c15a100 2024-01-27T02:41:43.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.365+0000 7f0a2116b700 1 -- 172.21.15.7:0/3363361266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1c16eca0 con 0x7f0a1c15a100 2024-01-27T02:41:43.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.366+0000 7f0a02ffd700 1 -- 172.21.15.7:0/3363361266 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0a1401b400 con 0x7f0a1c15a100 2024-01-27T02:41:43.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.366+0000 7f0a02ffd700 1 --2- 172.21.15.7:0/3363361266 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0a08066f30 0x7f0a080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:43.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.366+0000 7f0a02ffd700 1 -- 172.21.15.7:0/3363361266 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f0a14084880 con 0x7f0a1c15a100 2024-01-27T02:41:43.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.367+0000 7f0a02ffd700 1 -- 172.21.15.7:0/3363361266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=154}) v3 -- 0x7f0a0806eb10 con 0x7f0a1c15a100 2024-01-27T02:41:43.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.367+0000 7f0a1ad9d700 1 --2- 172.21.15.7:0/3363361266 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0a08066f30 0x7f0a080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:43.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.367+0000 7f0a1ad9d700 1 --2- 172.21.15.7:0/3363361266 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0a08066f30 0x7f0a080693e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c007fd0 tx=0x7f0a0c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:43.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.371+0000 7f0a02ffd700 1 -- 172.21.15.7:0/3363361266 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a14016020 con 0x7f0a1c15a100 2024-01-27T02:41:43.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.576+0000 7f0a2116b700 1 -- 172.21.15.7:0/3363361266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a1c0c6f80 con 0x7f0a1c15a100 2024-01-27T02:41:43.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.577+0000 7f0a02ffd700 1 -- 172.21.15.7:0/3363361266 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+15440 (secure 0 0 0) 0x7f0a1c0c6f80 con 0x7f0a1c15a100 2024-01-27T02:41:43.577 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:41:43.577 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":153,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:41:04.900114+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:39:20.119244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-01-27T02:39:21.293772+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.7","mappings":[{"from":2,"to":0}]},{"pgid":"19.3f","mappings":[{"from":2,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:41:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.579+0000 7f0a2116b700 1 -- 172.21.15.7:0/3363361266 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0a08066f30 msgr2=0x7f0a080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.579+0000 7f0a2116b700 1 --2- 172.21.15.7:0/3363361266 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0a08066f30 0x7f0a080693e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c007fd0 tx=0x7f0a0c009040 comp rx=0 tx=0).stop 2024-01-27T02:41:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.579+0000 7f0a2116b700 1 -- 172.21.15.7:0/3363361266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a1c15a100 msgr2=0x7f0a1c149aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.579+0000 7f0a2116b700 1 --2- 172.21.15.7:0/3363361266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a1c15a100 0x7f0a1c149aa0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400c910 tx=0x7f0a140027a0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.579+0000 7f0a2116b700 1 -- 172.21.15.7:0/3363361266 shutdown_connections 2024-01-27T02:41:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.579+0000 7f0a2116b700 1 --2- 172.21.15.7:0/3363361266 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0a08066f30 0x7f0a080693e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.579+0000 7f0a2116b700 1 --2- 172.21.15.7:0/3363361266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a1c15a100 0x7f0a1c149aa0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.579+0000 7f0a2116b700 1 --2- 172.21.15.7:0/3363361266 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a1c0c4cc0 0x7f0a1c0b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.579+0000 7f0a2116b700 1 --2- 172.21.15.7:0/3363361266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a1c0ca540 0x7f0a1c16dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.579+0000 7f0a2116b700 1 -- 172.21.15.7:0/3363361266 >> 172.21.15.7:0/3363361266 conn(0x7f0a1c098020 msgr2=0x7f0a1c154650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.579+0000 7f0a2116b700 1 -- 172.21.15.7:0/3363361266 shutdown_connections 2024-01-27T02:41:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.579+0000 7f0a2116b700 1 -- 172.21.15.7:0/3363361266 wait complete. 2024-01-27T02:41:43.593 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-27T02:39:20.119244+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-01-27T02:39:21.293772+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '153', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-27T02:41:43.594 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:41:43.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.729+0000 7fc6ee597700 1 Processor -- start 2024-01-27T02:41:43.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.730+0000 7fc6ee597700 1 -- start start 2024-01-27T02:41:43.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.730+0000 7fc6ee597700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6e815a180 0x7fc6e80c3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:43.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.730+0000 7fc6ee597700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc6e80ca3b0 0x7fc6e80aef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:43.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.730+0000 7fc6ee597700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6e80b1450 con 0x7fc6e815a550 2024-01-27T02:41:43.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.730+0000 7fc6ee597700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e80b18e0 con 0x7fc6e80ca3b0 2024-01-27T02:41:43.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.730+0000 7fc6ee597700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e80b0840 con 0x7fc6e815a180 2024-01-27T02:41:43.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.731+0000 7fc6e77fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc6e815a550 0x7fc6e80c9ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52446/0 (socket says 172.21.15.7:52446) 2024-01-27T02:41:43.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.731+0000 7fc6e77fe700 1 -- 172.21.15.7:0/814956876 learned_addr learned my addr 172.21.15.7:0/814956876 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:41:43.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.731+0000 7fc6e7fff700 1 --2- 172.21.15.7:0/814956876 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6e815a180 0x7fc6e80c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:43.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.731+0000 7fc6ecb34700 1 --2- 172.21.15.7:0/814956876 >> v2:172.21.15.188:3300/0 conn(0x7fc6e80ca3b0 0x7fc6e80aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:43.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.731+0000 7fc6e6ffd700 1 -- 172.21.15.7:0/814956876 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 138470243 0 0) 0x7fc6e80b1450 con 0x7fc6e815a550 2024-01-27T02:41:43.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.731+0000 7fc6e6ffd700 1 -- 172.21.15.7:0/814956876 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6d0003570 con 0x7fc6e815a550 2024-01-27T02:41:43.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.731+0000 7fc6e6ffd700 1 -- 172.21.15.7:0/814956876 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2403414976 0 0) 0x7fc6d0003570 con 0x7fc6e815a550 2024-01-27T02:41:43.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.731+0000 7fc6e6ffd700 1 -- 172.21.15.7:0/814956876 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6e815a180 msgr2=0x7fc6e80c3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:43.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.731+0000 7fc6e6ffd700 1 --2- 172.21.15.7:0/814956876 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6e815a180 0x7fc6e80c3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.731+0000 7fc6e6ffd700 1 -- 172.21.15.7:0/814956876 >> v2:172.21.15.188:3300/0 conn(0x7fc6e80ca3b0 msgr2=0x7fc6e80aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.731+0000 7fc6e6ffd700 1 --2- 172.21.15.7:0/814956876 >> v2:172.21.15.188:3300/0 conn(0x7fc6e80ca3b0 0x7fc6e80aef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.731+0000 7fc6e6ffd700 1 -- 172.21.15.7:0/814956876 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e80b0b20 con 0x7fc6e815a550 2024-01-27T02:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.731+0000 7fc6e6ffd700 1 -- 172.21.15.7:0/814956876 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7fc6dc001470 con 0x7fc6e815a550 2024-01-27T02:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.731+0000 7fc6e6ffd700 1 -- 172.21.15.7:0/814956876 >> v1:172.21.15.7:6789/0 conn(0x7fc6e815a550 legacy=0x7fc6e80c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.731+0000 7fc6e6ffd700 1 --2- 172.21.15.7:0/814956876 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6e815a180 0x7fc6d0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.731+0000 7fc6e6ffd700 1 --2- 172.21.15.7:0/814956876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d0003fe0 0x7fc6d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.731+0000 7fc6e6ffd700 1 --2- 172.21.15.7:0/814956876 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6d0004d50 0x7fc6d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.731+0000 7fc6e6ffd700 1 -- 172.21.15.7:0/814956876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e80b18e0 con 0x7fc6d0003fe0 2024-01-27T02:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.731+0000 7fc6e6ffd700 1 -- 172.21.15.7:0/814956876 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e80b0840 con 0x7fc6d0004d50 2024-01-27T02:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.731+0000 7fc6e6ffd700 1 -- 172.21.15.7:0/814956876 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d000ba90 con 0x7fc6e815a180 2024-01-27T02:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.731+0000 7fc6e77fe700 1 --2- 172.21.15.7:0/814956876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d0003fe0 0x7fc6d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.732+0000 7fc6e7fff700 1 --2- 172.21.15.7:0/814956876 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6e815a180 0x7fc6d0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.732+0000 7fc6ecb34700 1 --2- 172.21.15.7:0/814956876 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6d0004d50 0x7fc6d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.732+0000 7fc6e77fe700 1 -- 172.21.15.7:0/814956876 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6e815a180 msgr2=0x7fc6d0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.732+0000 7fc6e77fe700 1 --2- 172.21.15.7:0/814956876 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6e815a180 0x7fc6d0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.732+0000 7fc6e77fe700 1 -- 172.21.15.7:0/814956876 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6d0004d50 msgr2=0x7fc6d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.732+0000 7fc6e77fe700 1 --2- 172.21.15.7:0/814956876 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6d0004d50 0x7fc6d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.732+0000 7fc6e77fe700 1 -- 172.21.15.7:0/814956876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d0003570 con 0x7fc6d0003fe0 2024-01-27T02:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.732+0000 7fc6e77fe700 1 --2- 172.21.15.7:0/814956876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d0003fe0 0x7fc6d0004810 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00c3c0 tx=0x7fc6dc001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa45a91f7db1d475 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.732+0000 7fc6e6ffd700 1 -- 172.21.15.7:0/814956876 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc00cc30 con 0x7fc6d0003fe0 2024-01-27T02:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.732+0000 7fc6e6ffd700 1 -- 172.21.15.7:0/814956876 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6dc00cdb0 con 0x7fc6d0003fe0 2024-01-27T02:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.732+0000 7fc6e6ffd700 1 -- 172.21.15.7:0/814956876 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc005b70 con 0x7fc6d0003fe0 2024-01-27T02:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.733+0000 7fc6ee597700 1 -- 172.21.15.7:0/814956876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d0003fe0 msgr2=0x7fc6d0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.733+0000 7fc6ee597700 1 --2- 172.21.15.7:0/814956876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d0003fe0 0x7fc6d0004810 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00c3c0 tx=0x7fc6dc001e90 comp rx=0 tx=0).stop 2024-01-27T02:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.733+0000 7fc6ecb34700 1 -- 172.21.15.7:0/814956876 reap_dead start 2024-01-27T02:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.733+0000 7fc6ee597700 1 -- 172.21.15.7:0/814956876 shutdown_connections 2024-01-27T02:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.733+0000 7fc6ee597700 1 -- 172.21.15.7:0/814956876 >> 172.21.15.7:0/814956876 conn(0x7fc6e80679b0 msgr2=0x7fc6e8067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.733+0000 7fc6ee597700 1 -- 172.21.15.7:0/814956876 shutdown_connections 2024-01-27T02:41:43.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.733+0000 7fc6ee597700 1 -- 172.21.15.7:0/814956876 wait complete. 2024-01-27T02:41:43.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.734+0000 7fc6ee597700 1 Processor -- start 2024-01-27T02:41:43.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.734+0000 7fc6ee597700 1 -- start start 2024-01-27T02:41:43.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.735+0000 7fc6ee597700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6d0003fe0 0x7fc6e8157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:43.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.735+0000 7fc6ee597700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6e8149940 0x7fc6e806f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:43.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.735+0000 7fc6ee597700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6e8149db0 0x7fc6e806f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:43.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.735+0000 7fc6ee597700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e81494b0 con 0x7fc6e8149940 2024-01-27T02:41:43.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.735+0000 7fc6ee597700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e81584f0 con 0x7fc6d0003fe0 2024-01-27T02:41:43.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.735+0000 7fc6ee597700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e81488a0 con 0x7fc6e8149db0 2024-01-27T02:41:43.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.735+0000 7fc6e77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6e8149940 0x7fc6e806f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:43.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.735+0000 7fc6e77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6e8149940 0x7fc6e806f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40720/0 (socket says 172.21.15.7:40720) 2024-01-27T02:41:43.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.735+0000 7fc6e77fe700 1 -- 172.21.15.7:0/186854994 learned_addr learned my addr 172.21.15.7:0/186854994 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:43.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.735+0000 7fc6ecb34700 1 --2- 172.21.15.7:0/186854994 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6e8149db0 0x7fc6e806f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:43.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.735+0000 7fc6e7fff700 1 --2- 172.21.15.7:0/186854994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6d0003fe0 0x7fc6e8157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:43.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.735+0000 7fc6e77fe700 1 -- 172.21.15.7:0/186854994 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6e8149db0 msgr2=0x7fc6e806f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:43.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.735+0000 7fc6e77fe700 1 --2- 172.21.15.7:0/186854994 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6e8149db0 0x7fc6e806f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.735+0000 7fc6e77fe700 1 -- 172.21.15.7:0/186854994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6d0003fe0 msgr2=0x7fc6e8157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:43.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.735+0000 7fc6e77fe700 1 --2- 172.21.15.7:0/186854994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6d0003fe0 0x7fc6e8157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.735+0000 7fc6e77fe700 1 -- 172.21.15.7:0/186854994 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e8148b80 con 0x7fc6e8149940 2024-01-27T02:41:43.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.735+0000 7fc6e7fff700 1 --2- 172.21.15.7:0/186854994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6d0003fe0 0x7fc6e8157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:41:43.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.736+0000 7fc6e77fe700 1 --2- 172.21.15.7:0/186854994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6e8149940 0x7fc6e806f180 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc001ed0 tx=0x7fc6dc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:43.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.736+0000 7fc6cffff700 1 -- 172.21.15.7:0/186854994 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc001430 con 0x7fc6e8149940 2024-01-27T02:41:43.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.736+0000 7fc6cffff700 1 -- 172.21.15.7:0/186854994 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6dc0072e0 con 0x7fc6e8149940 2024-01-27T02:41:43.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.736+0000 7fc6ee597700 1 -- 172.21.15.7:0/186854994 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e8148e60 con 0x7fc6e8149940 2024-01-27T02:41:43.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.736+0000 7fc6cffff700 1 -- 172.21.15.7:0/186854994 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc0076a0 con 0x7fc6e8149940 2024-01-27T02:41:43.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.736+0000 7fc6ee597700 1 -- 172.21.15.7:0/186854994 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e814a5b0 con 0x7fc6e8149940 2024-01-27T02:41:43.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.737+0000 7fc6ee597700 1 -- 172.21.15.7:0/186854994 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6e8149620 con 0x7fc6e8149940 2024-01-27T02:41:43.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.737+0000 7fc6cffff700 1 -- 172.21.15.7:0/186854994 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc6dc019030 con 0x7fc6e8149940 2024-01-27T02:41:43.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.738+0000 7fc6cffff700 1 --2- 172.21.15.7:0/186854994 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc6d4066f30 0x7fc6d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:43.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.738+0000 7fc6cffff700 1 -- 172.21.15.7:0/186854994 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fc6dc085c80 con 0x7fc6e8149940 2024-01-27T02:41:43.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.738+0000 7fc6cffff700 1 -- 172.21.15.7:0/186854994 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=154}) v3 -- 0x7fc6d406eb10 con 0x7fc6e8149940 2024-01-27T02:41:43.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.738+0000 7fc6e7fff700 1 --2- 172.21.15.7:0/186854994 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc6d4066f30 0x7fc6d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:43.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.739+0000 7fc6e7fff700 1 --2- 172.21.15.7:0/186854994 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc6d4066f30 0x7fc6d40693e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009390 tx=0x7fc6d8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:43.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.742+0000 7fc6cffff700 1 -- 172.21.15.7:0/186854994 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc6dc017060 con 0x7fc6e8149940 2024-01-27T02:41:43.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.944+0000 7fc6ee597700 1 -- 172.21.15.7:0/186854994 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc6e8072060 con 0x7fc6e8149940 2024-01-27T02:41:43.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.944+0000 7fc6cffff700 1 -- 172.21.15.7:0/186854994 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v153) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc6dc052f60 con 0x7fc6e8149940 2024-01-27T02:41:43.945 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:41:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.946+0000 7fc6ee597700 1 -- 172.21.15.7:0/186854994 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc6d4066f30 msgr2=0x7fc6d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.946+0000 7fc6ee597700 1 --2- 172.21.15.7:0/186854994 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc6d4066f30 0x7fc6d40693e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009390 tx=0x7fc6d8008040 comp rx=0 tx=0).stop 2024-01-27T02:41:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.946+0000 7fc6ee597700 1 -- 172.21.15.7:0/186854994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6e8149940 msgr2=0x7fc6e806f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.946+0000 7fc6ee597700 1 --2- 172.21.15.7:0/186854994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6e8149940 0x7fc6e806f180 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc001ed0 tx=0x7fc6dc015000 comp rx=0 tx=0).stop 2024-01-27T02:41:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.946+0000 7fc6ee597700 1 -- 172.21.15.7:0/186854994 shutdown_connections 2024-01-27T02:41:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.946+0000 7fc6ee597700 1 --2- 172.21.15.7:0/186854994 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc6d4066f30 0x7fc6d40693e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.946+0000 7fc6ee597700 1 --2- 172.21.15.7:0/186854994 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6e8149db0 0x7fc6e806f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.946+0000 7fc6ee597700 1 --2- 172.21.15.7:0/186854994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6d0003fe0 0x7fc6e8157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.946+0000 7fc6ee597700 1 --2- 172.21.15.7:0/186854994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6e8149940 0x7fc6e806f180 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:43.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.946+0000 7fc6ee597700 1 -- 172.21.15.7:0/186854994 >> 172.21.15.7:0/186854994 conn(0x7fc6e80679b0 msgr2=0x7fc6e806adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:43.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.946+0000 7fc6ee597700 1 -- 172.21.15.7:0/186854994 shutdown_connections 2024-01-27T02:41:43.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:43.946+0000 7fc6ee597700 1 -- 172.21.15.7:0/186854994 wait complete. 2024-01-27T02:41:43.961 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-27T02:41:44.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.094+0000 7f7a7ecd9700 1 Processor -- start 2024-01-27T02:41:44.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.094+0000 7f7a7ecd9700 1 -- start start 2024-01-27T02:41:44.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.095+0000 7f7a7ecd9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7a78156b60 0x7f7a780ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:44.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.095+0000 7f7a7ecd9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a780aeaa0 con 0x7f7a78156f30 2024-01-27T02:41:44.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.095+0000 7f7a7ecd9700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a780ade90 con 0x7f7a780ad9f0 2024-01-27T02:41:44.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.095+0000 7f7a7ecd9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a780ae170 con 0x7f7a78156b60 2024-01-27T02:41:44.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.096+0000 7f7a7ca75700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7a78156b60 0x7f7a780ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:44.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.096+0000 7f7a7ca75700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7a78156b60 0x7f7a780ad4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40722/0 (socket says 172.21.15.7:40722) 2024-01-27T02:41:44.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.096+0000 7f7a7d276700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7a78156f30 0x7f7a7815a9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52462/0 (socket says 172.21.15.7:52462) 2024-01-27T02:41:44.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.096+0000 7f7a7ca75700 1 -- 172.21.15.7:0/3614754301 learned_addr learned my addr 172.21.15.7:0/3614754301 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:44.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.096+0000 7f7a7ca75700 1 -- 172.21.15.7:0/3614754301 >> v1:172.21.15.188:6789/0 conn(0x7f7a780ad9f0 legacy=0x7f7a780aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:44.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.096+0000 7f7a7ca75700 1 -- 172.21.15.7:0/3614754301 >> v1:172.21.15.7:6789/0 conn(0x7f7a78156f30 legacy=0x7f7a7815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:44.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.096+0000 7f7a777fe700 1 -- 172.21.15.7:0/3614754301 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 60477922 0 0) 0x7f7a780aeaa0 con 0x7f7a78156f30 2024-01-27T02:41:44.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.096+0000 7f7a7ca75700 1 -- 172.21.15.7:0/3614754301 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a780ae450 con 0x7f7a78156b60 2024-01-27T02:41:44.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.096+0000 7f7a7ca75700 1 --2- 172.21.15.7:0/3614754301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a78156b60 0x7f7a780ad4b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c0099b0 tx=0x7f7a6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9341fefd5691d05f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:44.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.096+0000 7f7a777fe700 1 -- 172.21.15.7:0/3614754301 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c00b820 con 0x7f7a78156b60 2024-01-27T02:41:44.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.096+0000 7f7a777fe700 1 -- 172.21.15.7:0/3614754301 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6c00b980 con 0x7f7a78156b60 2024-01-27T02:41:44.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.096+0000 7f7a777fe700 1 -- 172.21.15.7:0/3614754301 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c00bd20 con 0x7f7a78156b60 2024-01-27T02:41:44.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.097+0000 7f7a7ecd9700 1 -- 172.21.15.7:0/3614754301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a78156b60 msgr2=0x7f7a780ad4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:44.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.097+0000 7f7a7ecd9700 1 --2- 172.21.15.7:0/3614754301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a78156b60 0x7f7a780ad4b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c0099b0 tx=0x7f7a6c009250 comp rx=0 tx=0).stop 2024-01-27T02:41:44.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.097+0000 7f7a7ecd9700 1 -- 172.21.15.7:0/3614754301 shutdown_connections 2024-01-27T02:41:44.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.097+0000 7f7a7ecd9700 1 --2- 172.21.15.7:0/3614754301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a78156b60 0x7f7a780ad4b0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:44.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.097+0000 7f7a7ecd9700 1 -- 172.21.15.7:0/3614754301 >> 172.21.15.7:0/3614754301 conn(0x7f7a78068480 msgr2=0x7f7a780cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:44.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.097+0000 7f7a7ecd9700 1 -- 172.21.15.7:0/3614754301 shutdown_connections 2024-01-27T02:41:44.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.097+0000 7f7a7ecd9700 1 -- 172.21.15.7:0/3614754301 wait complete. 2024-01-27T02:41:44.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.098+0000 7f7a7ecd9700 1 Processor -- start 2024-01-27T02:41:44.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.098+0000 7f7a7ecd9700 1 -- start start 2024-01-27T02:41:44.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.099+0000 7f7a7ecd9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a780ad9f0 0x7f7a78145620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:44.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.099+0000 7f7a7ecd9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a78156b60 0x7f7a780ac610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:44.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.099+0000 7f7a7ecd9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a78156f30 0x7f7a780acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:44.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.099+0000 7f7a7ecd9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a78159d20 con 0x7f7a78156b60 2024-01-27T02:41:44.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.099+0000 7f7a7ecd9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7815a000 con 0x7f7a78156f30 2024-01-27T02:41:44.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.099+0000 7f7a7ecd9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7815a2e0 con 0x7f7a780ad9f0 2024-01-27T02:41:44.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.099+0000 7f7a77fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a78156b60 0x7f7a780ac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:44.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.099+0000 7f7a77fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a78156b60 0x7f7a780ac610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40736/0 (socket says 172.21.15.7:40736) 2024-01-27T02:41:44.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.099+0000 7f7a77fff700 1 -- 172.21.15.7:0/2567796142 learned_addr learned my addr 172.21.15.7:0/2567796142 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:44.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.099+0000 7f7a7ca75700 1 --2- 172.21.15.7:0/2567796142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a780ad9f0 0x7f7a78145620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:44.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.099+0000 7f7a7d276700 1 --2- 172.21.15.7:0/2567796142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a78156f30 0x7f7a780acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:44.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.100+0000 7f7a77fff700 1 -- 172.21.15.7:0/2567796142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a780ad9f0 msgr2=0x7f7a78145620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:44.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.100+0000 7f7a77fff700 1 --2- 172.21.15.7:0/2567796142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a780ad9f0 0x7f7a78145620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:44.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.100+0000 7f7a77fff700 1 -- 172.21.15.7:0/2567796142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a78156f30 msgr2=0x7f7a780acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:44.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.100+0000 7f7a77fff700 1 --2- 172.21.15.7:0/2567796142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a78156f30 0x7f7a780acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:44.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.100+0000 7f7a77fff700 1 -- 172.21.15.7:0/2567796142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c0094f0 con 0x7f7a78156b60 2024-01-27T02:41:44.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.100+0000 7f7a77fff700 1 --2- 172.21.15.7:0/2567796142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a78156b60 0x7f7a780ac610 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400c5d0 tx=0x7f7a64007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:44.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.100+0000 7f7a74ff9700 1 -- 172.21.15.7:0/2567796142 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a64011590 con 0x7f7a78156b60 2024-01-27T02:41:44.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.100+0000 7f7a74ff9700 1 -- 172.21.15.7:0/2567796142 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a64011710 con 0x7f7a78156b60 2024-01-27T02:41:44.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.100+0000 7f7a7ecd9700 1 -- 172.21.15.7:0/2567796142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a7815a5c0 con 0x7f7a78156b60 2024-01-27T02:41:44.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.100+0000 7f7a74ff9700 1 -- 172.21.15.7:0/2567796142 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a64011ad0 con 0x7f7a78156b60 2024-01-27T02:41:44.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.100+0000 7f7a7ecd9700 1 -- 172.21.15.7:0/2567796142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a78159080 con 0x7f7a78156b60 2024-01-27T02:41:44.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.101+0000 7f7a7ecd9700 1 -- 172.21.15.7:0/2567796142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a78145ec0 con 0x7f7a78156b60 2024-01-27T02:41:44.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.101+0000 7f7a74ff9700 1 -- 172.21.15.7:0/2567796142 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7a6401c460 con 0x7f7a78156b60 2024-01-27T02:41:44.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.102+0000 7f7a74ff9700 1 --2- 172.21.15.7:0/2567796142 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7a5c066f30 0x7f7a5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:44.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.102+0000 7f7a74ff9700 1 -- 172.21.15.7:0/2567796142 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f7a64084230 con 0x7f7a78156b60 2024-01-27T02:41:44.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.102+0000 7f7a74ff9700 1 -- 172.21.15.7:0/2567796142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=154}) v3 -- 0x7f7a5c06eb10 con 0x7f7a78156b60 2024-01-27T02:41:44.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.102+0000 7f7a7ca75700 1 --2- 172.21.15.7:0/2567796142 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7a5c066f30 0x7f7a5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:44.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.103+0000 7f7a7ca75700 1 --2- 172.21.15.7:0/2567796142 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7a5c066f30 0x7f7a5c0693e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c0099b0 tx=0x7f7a6c016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:44.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.106+0000 7f7a74ff9700 1 -- 172.21.15.7:0/2567796142 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a78145ec0 con 0x7f7a78156b60 2024-01-27T02:41:44.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.312+0000 7f7a7ecd9700 1 -- 172.21.15.7:0/2567796142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7a7815a750 con 0x7f7a78156b60 2024-01-27T02:41:44.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.313+0000 7f7a74ff9700 1 -- 172.21.15.7:0/2567796142 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v153) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7a64051460 con 0x7f7a78156b60 2024-01-27T02:41:44.313 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-27T02:41:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.315+0000 7f7a7ecd9700 1 -- 172.21.15.7:0/2567796142 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7a5c066f30 msgr2=0x7f7a5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.315+0000 7f7a7ecd9700 1 --2- 172.21.15.7:0/2567796142 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7a5c066f30 0x7f7a5c0693e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c0099b0 tx=0x7f7a6c016000 comp rx=0 tx=0).stop 2024-01-27T02:41:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.315+0000 7f7a7ecd9700 1 -- 172.21.15.7:0/2567796142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a78156b60 msgr2=0x7f7a780ac610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.315+0000 7f7a7ecd9700 1 --2- 172.21.15.7:0/2567796142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a78156b60 0x7f7a780ac610 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400c5d0 tx=0x7f7a64007f60 comp rx=0 tx=0).stop 2024-01-27T02:41:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.315+0000 7f7a7ecd9700 1 -- 172.21.15.7:0/2567796142 shutdown_connections 2024-01-27T02:41:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.315+0000 7f7a7ecd9700 1 --2- 172.21.15.7:0/2567796142 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7a5c066f30 0x7f7a5c0693e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.315+0000 7f7a7ecd9700 1 --2- 172.21.15.7:0/2567796142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a78156f30 0x7f7a780acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.315+0000 7f7a7ecd9700 1 --2- 172.21.15.7:0/2567796142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a780ad9f0 0x7f7a78145620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.315+0000 7f7a7ecd9700 1 --2- 172.21.15.7:0/2567796142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a78156b60 0x7f7a780ac610 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.315+0000 7f7a7ecd9700 1 -- 172.21.15.7:0/2567796142 >> 172.21.15.7:0/2567796142 conn(0x7f7a78068480 msgr2=0x7f7a7806ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:44.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.315+0000 7f7a7ecd9700 1 -- 172.21.15.7:0/2567796142 shutdown_connections 2024-01-27T02:41:44.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.315+0000 7f7a7ecd9700 1 -- 172.21.15.7:0/2567796142 wait complete. 2024-01-27T02:41:44.329 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-27T02:41:44.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.463+0000 7fc494021700 1 Processor -- start 2024-01-27T02:41:44.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.463+0000 7fc494021700 1 -- start start 2024-01-27T02:41:44.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.464+0000 7fc494021700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc48c0cb950 0x7fc48c0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:44.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.464+0000 7fc494021700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc48c1412d0 0x7fc48c141730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:44.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.464+0000 7fc494021700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc48c0c9660 con 0x7fc48c0cbd20 2024-01-27T02:41:44.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.464+0000 7fc494021700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc48c15c980 con 0x7fc48c1412d0 2024-01-27T02:41:44.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.464+0000 7fc494021700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c141c70 con 0x7fc48c0cb950 2024-01-27T02:41:44.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.464+0000 7fc491dbd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc48c0cb950 0x7fc48c0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:44.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.464+0000 7fc4925be700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc48c1412d0 0x7fc48c141730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:44.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.464+0000 7fc4925be700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc48c1412d0 0x7fc48c141730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:44028/0 (socket says 172.21.15.7:44028) 2024-01-27T02:41:44.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.464+0000 7fc4925be700 1 -- 172.21.15.7:0/1882463082 learned_addr learned my addr 172.21.15.7:0/1882463082 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:44.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.464+0000 7fc491dbd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc48c0cb950 0x7fc48c0b1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:51770/0 (socket says 172.21.15.7:51770) 2024-01-27T02:41:44.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.465+0000 7fc4925be700 1 -- 172.21.15.7:0/1882463082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc48c0cb950 msgr2=0x7fc48c0b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:44.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.465+0000 7fc4925be700 1 --2- 172.21.15.7:0/1882463082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc48c0cb950 0x7fc48c0b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:44.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.465+0000 7fc4925be700 1 -- 172.21.15.7:0/1882463082 >> v1:172.21.15.188:6789/0 conn(0x7fc48c0cbd20 legacy=0x7fc48c0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:44.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.465+0000 7fc4925be700 1 -- 172.21.15.7:0/1882463082 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48c15be00 con 0x7fc48c1412d0 2024-01-27T02:41:44.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.465+0000 7fc490dbb700 1 -- 172.21.15.7:0/1882463082 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142607323 0 0) 0x7fc48c0c9660 con 0x7fc48c0cbd20 2024-01-27T02:41:44.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.465+0000 7fc491dbd700 1 --2- 172.21.15.7:0/1882463082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc48c0cb950 0x7fc48c0b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:41:44.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.465+0000 7fc4925be700 1 --2- 172.21.15.7:0/1882463082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc48c1412d0 0x7fc48c141730 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc484009f20 tx=0x7fc484009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdaa6abe87e5264e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:44.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.465+0000 7fc490dbb700 1 -- 172.21.15.7:0/1882463082 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48400e070 con 0x7fc48c1412d0 2024-01-27T02:41:44.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.465+0000 7fc490dbb700 1 -- 172.21.15.7:0/1882463082 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48400b9d0 con 0x7fc48c1412d0 2024-01-27T02:41:44.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.465+0000 7fc490dbb700 1 -- 172.21.15.7:0/1882463082 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4840092e0 con 0x7fc48c1412d0 2024-01-27T02:41:44.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.466+0000 7fc494021700 1 -- 172.21.15.7:0/1882463082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc48c1412d0 msgr2=0x7fc48c141730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:44.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.466+0000 7fc494021700 1 --2- 172.21.15.7:0/1882463082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc48c1412d0 0x7fc48c141730 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc484009f20 tx=0x7fc484009840 comp rx=0 tx=0).stop 2024-01-27T02:41:44.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.466+0000 7fc494021700 1 -- 172.21.15.7:0/1882463082 shutdown_connections 2024-01-27T02:41:44.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.466+0000 7fc494021700 1 --2- 172.21.15.7:0/1882463082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc48c1412d0 0x7fc48c141730 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:44.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.466+0000 7fc494021700 1 --2- 172.21.15.7:0/1882463082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc48c0cb950 0x7fc48c0b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:44.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.466+0000 7fc494021700 1 -- 172.21.15.7:0/1882463082 >> 172.21.15.7:0/1882463082 conn(0x7fc48c063580 msgr2=0x7fc48c063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:44.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.466+0000 7fc494021700 1 -- 172.21.15.7:0/1882463082 shutdown_connections 2024-01-27T02:41:44.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.466+0000 7fc494021700 1 -- 172.21.15.7:0/1882463082 wait complete. 2024-01-27T02:41:44.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.467+0000 7fc494021700 1 Processor -- start 2024-01-27T02:41:44.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.467+0000 7fc494021700 1 -- start start 2024-01-27T02:41:44.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.468+0000 7fc494021700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc48c0cb950 0x7fc48c16a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:44.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.468+0000 7fc494021700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc48c0cbd20 0x7fc48c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:44.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.468+0000 7fc494021700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc48c1412d0 0x7fc48c0b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:44.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.468+0000 7fc494021700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c15d5a0 con 0x7fc48c0cb950 2024-01-27T02:41:44.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.468+0000 7fc494021700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c16ad50 con 0x7fc48c1412d0 2024-01-27T02:41:44.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.468+0000 7fc494021700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c16b030 con 0x7fc48c0cbd20 2024-01-27T02:41:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.468+0000 7fc491dbd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc48c0cb950 0x7fc48c16a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.468+0000 7fc491dbd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc48c0cb950 0x7fc48c16a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40752/0 (socket says 172.21.15.7:40752) 2024-01-27T02:41:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.468+0000 7fc491dbd700 1 -- 172.21.15.7:0/2707516081 learned_addr learned my addr 172.21.15.7:0/2707516081 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.468+0000 7fc4925be700 1 --2- 172.21.15.7:0/2707516081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc48c1412d0 0x7fc48c0b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.468+0000 7fc4915bc700 1 --2- 172.21.15.7:0/2707516081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc48c0cbd20 0x7fc48c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.469+0000 7fc491dbd700 1 -- 172.21.15.7:0/2707516081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc48c0cbd20 msgr2=0x7fc48c0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.469+0000 7fc491dbd700 1 --2- 172.21.15.7:0/2707516081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc48c0cbd20 0x7fc48c0cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.469+0000 7fc491dbd700 1 -- 172.21.15.7:0/2707516081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc48c1412d0 msgr2=0x7fc48c0b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.469+0000 7fc491dbd700 1 --2- 172.21.15.7:0/2707516081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc48c1412d0 0x7fc48c0b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.469+0000 7fc491dbd700 1 -- 172.21.15.7:0/2707516081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc488009420 con 0x7fc48c0cb950 2024-01-27T02:41:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.469+0000 7fc491dbd700 1 --2- 172.21.15.7:0/2707516081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc48c0cb950 0x7fc48c16a450 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fc48800ec80 tx=0x7fc4880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.469+0000 7fc481ffb700 1 -- 172.21.15.7:0/2707516081 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48800c850 con 0x7fc48c0cb950 2024-01-27T02:41:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.469+0000 7fc481ffb700 1 -- 172.21.15.7:0/2707516081 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48800c9b0 con 0x7fc48c0cb950 2024-01-27T02:41:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.469+0000 7fc494021700 1 -- 172.21.15.7:0/2707516081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc484009ae0 con 0x7fc48c0cb950 2024-01-27T02:41:44.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.469+0000 7fc481ffb700 1 -- 172.21.15.7:0/2707516081 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48800cd30 con 0x7fc48c0cb950 2024-01-27T02:41:44.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.469+0000 7fc494021700 1 -- 172.21.15.7:0/2707516081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48c15d7f0 con 0x7fc48c0cb950 2024-01-27T02:41:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.470+0000 7fc494021700 1 -- 172.21.15.7:0/2707516081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc48c072060 con 0x7fc48c0cb950 2024-01-27T02:41:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.471+0000 7fc481ffb700 1 -- 172.21.15.7:0/2707516081 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc488013680 con 0x7fc48c0cb950 2024-01-27T02:41:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.471+0000 7fc481ffb700 1 --2- 172.21.15.7:0/2707516081 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc470066f30 0x7fc4700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.471+0000 7fc481ffb700 1 -- 172.21.15.7:0/2707516081 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fc4880847b0 con 0x7fc48c0cb950 2024-01-27T02:41:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.471+0000 7fc481ffb700 1 -- 172.21.15.7:0/2707516081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=154}) v3 -- 0x7fc47006eb10 con 0x7fc48c0cb950 2024-01-27T02:41:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.471+0000 7fc4915bc700 1 --2- 172.21.15.7:0/2707516081 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc470066f30 0x7fc4700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.472+0000 7fc4915bc700 1 --2- 172.21.15.7:0/2707516081 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc470066f30 0x7fc4700693e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0cb310 tx=0x7fc47c007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:44.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.476+0000 7fc481ffb700 1 -- 172.21.15.7:0/2707516081 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc488055470 con 0x7fc48c0cb950 2024-01-27T02:41:44.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.677+0000 7fc494021700 1 -- 172.21.15.7:0/2707516081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc48c15db60 con 0x7fc48c0cb950 2024-01-27T02:41:44.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.677+0000 7fc481ffb700 1 -- 172.21.15.7:0/2707516081 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v153) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc488055000 con 0x7fc48c0cb950 2024-01-27T02:41:44.678 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-27T02:41:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.679+0000 7fc494021700 1 -- 172.21.15.7:0/2707516081 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc470066f30 msgr2=0x7fc4700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.679+0000 7fc494021700 1 --2- 172.21.15.7:0/2707516081 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc470066f30 0x7fc4700693e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0cb310 tx=0x7fc47c007d30 comp rx=0 tx=0).stop 2024-01-27T02:41:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.679+0000 7fc494021700 1 -- 172.21.15.7:0/2707516081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc48c0cb950 msgr2=0x7fc48c16a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.679+0000 7fc494021700 1 --2- 172.21.15.7:0/2707516081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc48c0cb950 0x7fc48c16a450 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fc48800ec80 tx=0x7fc4880091a0 comp rx=0 tx=0).stop 2024-01-27T02:41:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.679+0000 7fc494021700 1 -- 172.21.15.7:0/2707516081 shutdown_connections 2024-01-27T02:41:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.679+0000 7fc494021700 1 --2- 172.21.15.7:0/2707516081 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc470066f30 0x7fc4700693e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.679+0000 7fc494021700 1 --2- 172.21.15.7:0/2707516081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc48c1412d0 0x7fc48c0b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.679+0000 7fc494021700 1 --2- 172.21.15.7:0/2707516081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc48c0cb950 0x7fc48c16a450 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.679+0000 7fc494021700 1 --2- 172.21.15.7:0/2707516081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc48c0cbd20 0x7fc48c0cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.679+0000 7fc494021700 1 -- 172.21.15.7:0/2707516081 >> 172.21.15.7:0/2707516081 conn(0x7fc48c063580 msgr2=0x7fc48c0c4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:44.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.679+0000 7fc494021700 1 -- 172.21.15.7:0/2707516081 shutdown_connections 2024-01-27T02:41:44.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.679+0000 7fc494021700 1 -- 172.21.15.7:0/2707516081 wait complete. 2024-01-27T02:41:44.693 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:41:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.824+0000 7fe667c5c700 1 Processor -- start 2024-01-27T02:41:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.825+0000 7fe667c5c700 1 -- start start 2024-01-27T02:41:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.825+0000 7fe667c5c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6600cfee0 0x7fe6600b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.825+0000 7fe667c5c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe66015ba30 con 0x7fe6600b2e70 2024-01-27T02:41:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.825+0000 7fe667c5c700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6600d0390 con 0x7fe660157c00 2024-01-27T02:41:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.825+0000 7fe667c5c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6600b3310 con 0x7fe6600cfee0 2024-01-27T02:41:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.826+0000 7fe6661f9700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe6600b2e70 0x7fe66015bec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52476/0 (socket says 172.21.15.7:52476) 2024-01-27T02:41:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.826+0000 7fe6661f9700 1 -- 172.21.15.7:0/2407130048 learned_addr learned my addr 172.21.15.7:0/2407130048 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:41:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.826+0000 7fe6651f7700 1 --2- 172.21.15.7:0/2407130048 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6600cfee0 0x7fe6600b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.826+0000 7fe6649f6700 1 -- 172.21.15.7:0/2407130048 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788509831 0 0) 0x7fe66015ba30 con 0x7fe6600b2e70 2024-01-27T02:41:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.826+0000 7fe6649f6700 1 -- 172.21.15.7:0/2407130048 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe64c003570 con 0x7fe6600b2e70 2024-01-27T02:41:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.826+0000 7fe6649f6700 1 -- 172.21.15.7:0/2407130048 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3672177557 0 0) 0x7fe6600d0390 con 0x7fe660157c00 2024-01-27T02:41:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.826+0000 7fe6649f6700 1 -- 172.21.15.7:0/2407130048 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe66015ba30 con 0x7fe660157c00 2024-01-27T02:41:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.826+0000 7fe6649f6700 1 -- 172.21.15.7:0/2407130048 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1853741021 0 0) 0x7fe64c003570 con 0x7fe6600b2e70 2024-01-27T02:41:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.826+0000 7fe6651f7700 1 -- 172.21.15.7:0/2407130048 >> v1:172.21.15.188:6789/0 conn(0x7fe660157c00 legacy=0x7fe6600cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.826+0000 7fe6651f7700 1 -- 172.21.15.7:0/2407130048 >> v1:172.21.15.7:6789/0 conn(0x7fe6600b2e70 legacy=0x7fe66015bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.826+0000 7fe6651f7700 1 -- 172.21.15.7:0/2407130048 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6600d0670 con 0x7fe6600cfee0 2024-01-27T02:41:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.826+0000 7fe6651f7700 1 --2- 172.21.15.7:0/2407130048 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6600cfee0 0x7fe6600b2930 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe650009910 tx=0x7fe6500091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a41b0a5c4620c81d server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.827+0000 7fe6649f6700 1 -- 172.21.15.7:0/2407130048 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6580017f0 con 0x7fe6600cfee0 2024-01-27T02:41:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.827+0000 7fe6649f6700 1 -- 172.21.15.7:0/2407130048 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe650011420 con 0x7fe6600cfee0 2024-01-27T02:41:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.827+0000 7fe6649f6700 1 -- 172.21.15.7:0/2407130048 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6500117c0 con 0x7fe6600cfee0 2024-01-27T02:41:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.827+0000 7fe667c5c700 1 -- 172.21.15.7:0/2407130048 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6600cfee0 msgr2=0x7fe6600b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.827+0000 7fe667c5c700 1 --2- 172.21.15.7:0/2407130048 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6600cfee0 0x7fe6600b2930 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe650009910 tx=0x7fe6500091a0 comp rx=0 tx=0).stop 2024-01-27T02:41:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.827+0000 7fe667c5c700 1 -- 172.21.15.7:0/2407130048 shutdown_connections 2024-01-27T02:41:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.827+0000 7fe667c5c700 1 --2- 172.21.15.7:0/2407130048 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6600cfee0 0x7fe6600b2930 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.827+0000 7fe667c5c700 1 -- 172.21.15.7:0/2407130048 >> 172.21.15.7:0/2407130048 conn(0x7fe660068450 msgr2=0x7fe6600709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.827+0000 7fe667c5c700 1 -- 172.21.15.7:0/2407130048 shutdown_connections 2024-01-27T02:41:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.827+0000 7fe667c5c700 1 -- 172.21.15.7:0/2407130048 wait complete. 2024-01-27T02:41:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.829+0000 7fe667c5c700 1 Processor -- start 2024-01-27T02:41:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.829+0000 7fe667c5c700 1 -- start start 2024-01-27T02:41:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.829+0000 7fe667c5c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6600b2e70 0x7fe6601538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.829+0000 7fe667c5c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6600cfee0 0x7fe6600b5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.829+0000 7fe667c5c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe660157c00 0x7fe6600b5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.829+0000 7fe667c5c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6600b6cc0 con 0x7fe6600b2e70 2024-01-27T02:41:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.829+0000 7fe667c5c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6600b5e90 con 0x7fe660157c00 2024-01-27T02:41:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.829+0000 7fe667c5c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6600b6140 con 0x7fe6600cfee0 2024-01-27T02:41:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.829+0000 7fe6659f8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6600b2e70 0x7fe6601538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.830+0000 7fe6659f8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6600b2e70 0x7fe6601538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40766/0 (socket says 172.21.15.7:40766) 2024-01-27T02:41:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.830+0000 7fe6659f8700 1 -- 172.21.15.7:0/4237361382 learned_addr learned my addr 172.21.15.7:0/4237361382 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.830+0000 7fe6651f7700 1 --2- 172.21.15.7:0/4237361382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6600cfee0 0x7fe6600b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.830+0000 7fe6661f9700 1 --2- 172.21.15.7:0/4237361382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe660157c00 0x7fe6600b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.830+0000 7fe6659f8700 1 -- 172.21.15.7:0/4237361382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6600cfee0 msgr2=0x7fe6600b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.830+0000 7fe6659f8700 1 --2- 172.21.15.7:0/4237361382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6600cfee0 0x7fe6600b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.830+0000 7fe6659f8700 1 -- 172.21.15.7:0/4237361382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe660157c00 msgr2=0x7fe6600b5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.830+0000 7fe6659f8700 1 --2- 172.21.15.7:0/4237361382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe660157c00 0x7fe6600b5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.830+0000 7fe6659f8700 1 -- 172.21.15.7:0/4237361382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe650009420 con 0x7fe6600b2e70 2024-01-27T02:41:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.830+0000 7fe6651f7700 1 --2- 172.21.15.7:0/4237361382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6600cfee0 0x7fe6600b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:41:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.830+0000 7fe6659f8700 1 --2- 172.21.15.7:0/4237361382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6600b2e70 0x7fe6601538f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00c580 tx=0x7fe65c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.830+0000 7fe655ffb700 1 -- 172.21.15.7:0/4237361382 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c010510 con 0x7fe6600b2e70 2024-01-27T02:41:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.830+0000 7fe655ffb700 1 -- 172.21.15.7:0/4237361382 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65c010670 con 0x7fe6600b2e70 2024-01-27T02:41:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.830+0000 7fe667c5c700 1 -- 172.21.15.7:0/4237361382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6600b6420 con 0x7fe6600b2e70 2024-01-27T02:41:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.830+0000 7fe655ffb700 1 -- 172.21.15.7:0/4237361382 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c010a30 con 0x7fe6600b2e70 2024-01-27T02:41:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.830+0000 7fe667c5c700 1 -- 172.21.15.7:0/4237361382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6600b69e0 con 0x7fe6600b2e70 2024-01-27T02:41:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.831+0000 7fe667c5c700 1 -- 172.21.15.7:0/4237361382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe66004c710 con 0x7fe6600b2e70 2024-01-27T02:41:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.832+0000 7fe655ffb700 1 -- 172.21.15.7:0/4237361382 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe65c01b400 con 0x7fe6600b2e70 2024-01-27T02:41:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.832+0000 7fe655ffb700 1 --2- 172.21.15.7:0/4237361382 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe644066f30 0x7fe6440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.832+0000 7fe655ffb700 1 -- 172.21.15.7:0/4237361382 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fe65c084200 con 0x7fe6600b2e70 2024-01-27T02:41:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.833+0000 7fe655ffb700 1 -- 172.21.15.7:0/4237361382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=154}) v3 -- 0x7fe64406eb10 con 0x7fe6600b2e70 2024-01-27T02:41:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.833+0000 7fe6651f7700 1 --2- 172.21.15.7:0/4237361382 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe644066f30 0x7fe6440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.833+0000 7fe6651f7700 1 --2- 172.21.15.7:0/4237361382 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe644066f30 0x7fe6440693e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe6500030c0 tx=0x7fe650016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:44.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:44.837+0000 7fe655ffb700 1 -- 172.21.15.7:0/4237361382 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe65c051430 con 0x7fe6600b2e70 2024-01-27T02:41:45.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.087+0000 7fe667c5c700 1 -- 172.21.15.7:0/4237361382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6600d0950 con 0x7fe6600b2e70 2024-01-27T02:41:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.087+0000 7fe655ffb700 1 -- 172.21.15.7:0/4237361382 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fe65c054a50 con 0x7fe6600b2e70 2024-01-27T02:41:45.088 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:41:45.088 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6260,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3240780787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3240780787},{"type":"v1","addr":"172.21.15.188:6837","nonce":3240780787}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7156,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/4051709811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4051709811},{"type":"v1","addr":"172.21.15.188:6836","nonce":4051709811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7416,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/4000944335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":4000944335},{"type":"v1","addr":"172.21.15.7:6834","nonce":4000944335}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:39:22.327946+0000","modified":"2024-01-27T02:39:25.910241+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7419":{"gid":7419,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2386550039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2386550039},{"type":"v1","addr":"172.21.15.7:6835","nonce":2386550039}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2024-01-27T02:41:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.089+0000 7fe667c5c700 1 -- 172.21.15.7:0/4237361382 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe644066f30 msgr2=0x7fe6440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.089+0000 7fe667c5c700 1 --2- 172.21.15.7:0/4237361382 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe644066f30 0x7fe6440693e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe6500030c0 tx=0x7fe650016000 comp rx=0 tx=0).stop 2024-01-27T02:41:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.089+0000 7fe667c5c700 1 -- 172.21.15.7:0/4237361382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6600b2e70 msgr2=0x7fe6601538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.089+0000 7fe667c5c700 1 --2- 172.21.15.7:0/4237361382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6600b2e70 0x7fe6601538f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00c580 tx=0x7fe65c007e10 comp rx=0 tx=0).stop 2024-01-27T02:41:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.089+0000 7fe667c5c700 1 -- 172.21.15.7:0/4237361382 shutdown_connections 2024-01-27T02:41:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.089+0000 7fe667c5c700 1 --2- 172.21.15.7:0/4237361382 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe644066f30 0x7fe6440693e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.089+0000 7fe667c5c700 1 --2- 172.21.15.7:0/4237361382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe660157c00 0x7fe6600b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.089+0000 7fe667c5c700 1 --2- 172.21.15.7:0/4237361382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6600b2e70 0x7fe6601538f0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.089+0000 7fe667c5c700 1 --2- 172.21.15.7:0/4237361382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6600cfee0 0x7fe6600b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.089+0000 7fe667c5c700 1 -- 172.21.15.7:0/4237361382 >> 172.21.15.7:0/4237361382 conn(0x7fe660068450 msgr2=0x7fe660159fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.089+0000 7fe667c5c700 1 -- 172.21.15.7:0/4237361382 shutdown_connections 2024-01-27T02:41:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.089+0000 7fe667c5c700 1 -- 172.21.15.7:0/4237361382 wait complete. 2024-01-27T02:41:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 77 2024-01-27T02:41:45.103 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:41:45.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.236+0000 7f35ee1a9700 1 Processor -- start 2024-01-27T02:41:45.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.236+0000 7f35ee1a9700 1 -- start start 2024-01-27T02:41:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.237+0000 7f35ee1a9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e80ca540 0x7f35e80cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.237+0000 7f35ee1a9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e80ce5e0 con 0x7f35e815a100 2024-01-27T02:41:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.237+0000 7f35ee1a9700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e80cd9d0 con 0x7f35e80c4cc0 2024-01-27T02:41:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.237+0000 7f35ee1a9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e80cdcb0 con 0x7f35e80ca540 2024-01-27T02:41:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.237+0000 7f35e77fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f35e815a100 0x7f35e80c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52486/0 (socket says 172.21.15.7:52486) 2024-01-27T02:41:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.237+0000 7f35e77fe700 1 -- 172.21.15.7:0/1141887863 learned_addr learned my addr 172.21.15.7:0/1141887863 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:41:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e7fff700 1 --2- 172.21.15.7:0/1141887863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e80ca540 0x7f35e80cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e67fc700 1 -- 172.21.15.7:0/1141887863 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1403609905 0 0) 0x7f35e80ce5e0 con 0x7f35e815a100 2024-01-27T02:41:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e67fc700 1 -- 172.21.15.7:0/1141887863 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35d4003570 con 0x7f35e815a100 2024-01-27T02:41:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e67fc700 1 -- 172.21.15.7:0/1141887863 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163812939 0 0) 0x7f35e80cd9d0 con 0x7f35e80c4cc0 2024-01-27T02:41:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e67fc700 1 -- 172.21.15.7:0/1141887863 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35e80ce5e0 con 0x7f35e80c4cc0 2024-01-27T02:41:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e67fc700 1 -- 172.21.15.7:0/1141887863 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2263868296 0 0) 0x7f35d4003570 con 0x7f35e815a100 2024-01-27T02:41:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e67fc700 1 -- 172.21.15.7:0/1141887863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e80ca540 msgr2=0x7f35e80cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e67fc700 1 --2- 172.21.15.7:0/1141887863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e80ca540 0x7f35e80cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e67fc700 1 -- 172.21.15.7:0/1141887863 >> v1:172.21.15.188:6789/0 conn(0x7f35e80c4cc0 legacy=0x7f35e8161d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e67fc700 1 -- 172.21.15.7:0/1141887863 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e80cdf90 con 0x7f35e815a100 2024-01-27T02:41:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e67fc700 1 -- 172.21.15.7:0/1141887863 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f35d8001470 con 0x7f35e815a100 2024-01-27T02:41:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e67fc700 1 -- 172.21.15.7:0/1141887863 >> v1:172.21.15.7:6789/0 conn(0x7f35e815a100 legacy=0x7f35e80c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e67fc700 1 --2- 172.21.15.7:0/1141887863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e80ca540 0x7f35d4003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e67fc700 1 --2- 172.21.15.7:0/1141887863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35d40041d0 0x7f35d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e67fc700 1 --2- 172.21.15.7:0/1141887863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35d4004d50 0x7f35d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e67fc700 1 -- 172.21.15.7:0/1141887863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e80cdcb0 con 0x7f35d4004d50 2024-01-27T02:41:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e67fc700 1 -- 172.21.15.7:0/1141887863 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d400bbf0 con 0x7f35d40041d0 2024-01-27T02:41:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e67fc700 1 -- 172.21.15.7:0/1141887863 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d400bdb0 con 0x7f35e80ca540 2024-01-27T02:41:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e7fff700 1 --2- 172.21.15.7:0/1141887863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35d4004d50 0x7f35d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e77fe700 1 --2- 172.21.15.7:0/1141887863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e80ca540 0x7f35d4003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e6ffd700 1 --2- 172.21.15.7:0/1141887863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35d40041d0 0x7f35d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e7fff700 1 -- 172.21.15.7:0/1141887863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e80ca540 msgr2=0x7f35d4003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e7fff700 1 --2- 172.21.15.7:0/1141887863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e80ca540 0x7f35d4003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e7fff700 1 -- 172.21.15.7:0/1141887863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35d40041d0 msgr2=0x7f35d4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.238+0000 7f35e7fff700 1 --2- 172.21.15.7:0/1141887863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35d40041d0 0x7f35d4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.239+0000 7f35e7fff700 1 -- 172.21.15.7:0/1141887863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d4003570 con 0x7f35d4004d50 2024-01-27T02:41:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.239+0000 7f35e7fff700 1 --2- 172.21.15.7:0/1141887863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35d4004d50 0x7f35d40051c0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f35e000e210 tx=0x7f35e000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45e14fa47f20cb5 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.239+0000 7f35e67fc700 1 -- 172.21.15.7:0/1141887863 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e000eac0 con 0x7f35d4004d50 2024-01-27T02:41:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.239+0000 7f35e67fc700 1 -- 172.21.15.7:0/1141887863 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e000f040 con 0x7f35d4004d50 2024-01-27T02:41:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.239+0000 7f35e67fc700 1 -- 172.21.15.7:0/1141887863 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e000ee70 con 0x7f35d4004d50 2024-01-27T02:41:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.239+0000 7f35ee1a9700 1 -- 172.21.15.7:0/1141887863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35d4004d50 msgr2=0x7f35d40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.239+0000 7f35ee1a9700 1 --2- 172.21.15.7:0/1141887863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35d4004d50 0x7f35d40051c0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f35e000e210 tx=0x7f35e000b040 comp rx=0 tx=0).stop 2024-01-27T02:41:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.239+0000 7f35e7fff700 1 -- 172.21.15.7:0/1141887863 reap_dead start 2024-01-27T02:41:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.239+0000 7f35ee1a9700 1 -- 172.21.15.7:0/1141887863 shutdown_connections 2024-01-27T02:41:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.239+0000 7f35ee1a9700 1 -- 172.21.15.7:0/1141887863 >> 172.21.15.7:0/1141887863 conn(0x7f35e8098020 msgr2=0x7f35e80711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.239+0000 7f35ee1a9700 1 -- 172.21.15.7:0/1141887863 shutdown_connections 2024-01-27T02:41:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.240+0000 7f35ee1a9700 1 -- 172.21.15.7:0/1141887863 wait complete. 2024-01-27T02:41:45.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.241+0000 7f35ee1a9700 1 Processor -- start 2024-01-27T02:41:45.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.241+0000 7f35ee1a9700 1 -- start start 2024-01-27T02:41:45.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.242+0000 7f35ee1a9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e814ddb0 0x7f35e80b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:45.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.242+0000 7f35ee1a9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e815a4d0 0x7f35e815bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:45.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.242+0000 7f35ee1a9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e806c490 0x7f35e806c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:45.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.242+0000 7f35ee1a9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e806ce40 con 0x7f35e814ddb0 2024-01-27T02:41:45.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.242+0000 7f35ee1a9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e815b8c0 con 0x7f35e815a4d0 2024-01-27T02:41:45.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.242+0000 7f35ee1a9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e815c290 con 0x7f35e806c490 2024-01-27T02:41:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.242+0000 7f35e77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e814ddb0 0x7f35e80b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.242+0000 7f35e77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e814ddb0 0x7f35e80b45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40788/0 (socket says 172.21.15.7:40788) 2024-01-27T02:41:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.242+0000 7f35e77fe700 1 -- 172.21.15.7:0/3309614558 learned_addr learned my addr 172.21.15.7:0/3309614558 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.242+0000 7f35e7fff700 1 --2- 172.21.15.7:0/3309614558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e806c490 0x7f35e806c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.242+0000 7f35e6ffd700 1 --2- 172.21.15.7:0/3309614558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e815a4d0 0x7f35e815bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.242+0000 7f35e77fe700 1 -- 172.21.15.7:0/3309614558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e806c490 msgr2=0x7f35e806c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.242+0000 7f35e77fe700 1 --2- 172.21.15.7:0/3309614558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e806c490 0x7f35e806c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.242+0000 7f35e77fe700 1 -- 172.21.15.7:0/3309614558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e815a4d0 msgr2=0x7f35e815bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.242+0000 7f35e77fe700 1 --2- 172.21.15.7:0/3309614558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e815a4d0 0x7f35e815bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.242+0000 7f35e77fe700 1 -- 172.21.15.7:0/3309614558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e815acb0 con 0x7f35e814ddb0 2024-01-27T02:41:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.242+0000 7f35e6ffd700 1 --2- 172.21.15.7:0/3309614558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e815a4d0 0x7f35e815bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:41:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.242+0000 7f35e77fe700 1 --2- 172.21.15.7:0/3309614558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e814ddb0 0x7f35e80b45b0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f35d8007390 tx=0x7f35d8001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.242+0000 7f35df7fe700 1 -- 172.21.15.7:0/3309614558 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d8000f50 con 0x7f35e814ddb0 2024-01-27T02:41:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.243+0000 7f35df7fe700 1 -- 172.21.15.7:0/3309614558 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d800f8d0 con 0x7f35e814ddb0 2024-01-27T02:41:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.243+0000 7f35ee1a9700 1 -- 172.21.15.7:0/3309614558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e815af90 con 0x7f35e814ddb0 2024-01-27T02:41:45.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.243+0000 7f35df7fe700 1 -- 172.21.15.7:0/3309614558 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d800fc90 con 0x7f35e814ddb0 2024-01-27T02:41:45.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.243+0000 7f35ee1a9700 1 -- 172.21.15.7:0/3309614558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e815b550 con 0x7f35e814ddb0 2024-01-27T02:41:45.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.243+0000 7f35ee1a9700 1 -- 172.21.15.7:0/3309614558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e804c710 con 0x7f35e814ddb0 2024-01-27T02:41:45.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.244+0000 7f35df7fe700 1 -- 172.21.15.7:0/3309614558 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f35d801d460 con 0x7f35e814ddb0 2024-01-27T02:41:45.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.245+0000 7f35df7fe700 1 --2- 172.21.15.7:0/3309614558 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f35c8066f30 0x7f35c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:45.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.245+0000 7f35df7fe700 1 -- 172.21.15.7:0/3309614558 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f35d80846f0 con 0x7f35e814ddb0 2024-01-27T02:41:45.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.245+0000 7f35df7fe700 1 -- 172.21.15.7:0/3309614558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=154}) v3 -- 0x7f35c806eb10 con 0x7f35e814ddb0 2024-01-27T02:41:45.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.245+0000 7f35e6ffd700 1 --2- 172.21.15.7:0/3309614558 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f35c8066f30 0x7f35c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:45.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.245+0000 7f35e6ffd700 1 --2- 172.21.15.7:0/3309614558 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f35c8066f30 0x7f35c80693e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f35d0000f60 tx=0x7f35d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:45.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.249+0000 7f35df7fe700 1 -- 172.21.15.7:0/3309614558 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35d8051920 con 0x7f35e814ddb0 2024-01-27T02:41:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.451+0000 7f35ee1a9700 1 -- 172.21.15.7:0/3309614558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35e80ce270 con 0x7f35e814ddb0 2024-01-27T02:41:45.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.452+0000 7f35df7fe700 1 -- 172.21.15.7:0/3309614558 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+15440 (secure 0 0 0) 0x7f35d8054f40 con 0x7f35e814ddb0 2024-01-27T02:41:45.452 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:41:45.452 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":153,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:41:04.900114+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:39:20.119244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-01-27T02:39:21.293772+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.7","mappings":[{"from":2,"to":0}]},{"pgid":"19.3f","mappings":[{"from":2,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:41:45.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.454+0000 7f35ee1a9700 1 -- 172.21.15.7:0/3309614558 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f35c8066f30 msgr2=0x7f35c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:45.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.454+0000 7f35ee1a9700 1 --2- 172.21.15.7:0/3309614558 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f35c8066f30 0x7f35c80693e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f35d0000f60 tx=0x7f35d0009040 comp rx=0 tx=0).stop 2024-01-27T02:41:45.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.454+0000 7f35ee1a9700 1 -- 172.21.15.7:0/3309614558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e814ddb0 msgr2=0x7f35e80b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:45.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.454+0000 7f35ee1a9700 1 --2- 172.21.15.7:0/3309614558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e814ddb0 0x7f35e80b45b0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f35d8007390 tx=0x7f35d8001d40 comp rx=0 tx=0).stop 2024-01-27T02:41:45.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.454+0000 7f35ee1a9700 1 -- 172.21.15.7:0/3309614558 shutdown_connections 2024-01-27T02:41:45.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.454+0000 7f35ee1a9700 1 --2- 172.21.15.7:0/3309614558 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f35c8066f30 0x7f35c80693e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.454+0000 7f35ee1a9700 1 --2- 172.21.15.7:0/3309614558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e806c490 0x7f35e806c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.454+0000 7f35ee1a9700 1 --2- 172.21.15.7:0/3309614558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e814ddb0 0x7f35e80b45b0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.454+0000 7f35ee1a9700 1 --2- 172.21.15.7:0/3309614558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e815a4d0 0x7f35e815bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.454+0000 7f35ee1a9700 1 -- 172.21.15.7:0/3309614558 >> 172.21.15.7:0/3309614558 conn(0x7f35e8098020 msgr2=0x7f35e814ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:45.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.454+0000 7f35ee1a9700 1 -- 172.21.15.7:0/3309614558 shutdown_connections 2024-01-27T02:41:45.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.454+0000 7f35ee1a9700 1 -- 172.21.15.7:0/3309614558 wait complete. 2024-01-27T02:41:45.469 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-27T02:41:45.469 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:41:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.600+0000 7ff0bdc10700 1 Processor -- start 2024-01-27T02:41:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.600+0000 7ff0bdc10700 1 -- start start 2024-01-27T02:41:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.601+0000 7ff0bdc10700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b80c8ab0 0x7ff0b815dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.601+0000 7ff0bdc10700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff0b80cbbc0 0x7ff0b80cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.601+0000 7ff0bdc10700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b815a9b0 con 0x7ff0b80c8e80 2024-01-27T02:41:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.601+0000 7ff0bdc10700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b815ecb0 con 0x7ff0b80cbbc0 2024-01-27T02:41:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.601+0000 7ff0bdc10700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b815e410 con 0x7ff0b80c8ab0 2024-01-27T02:41:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.601+0000 7ff0b7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff0b80cbbc0 0x7ff0b80cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.602+0000 7ff0b6ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff0b80c8e80 0x7ff0b80cd4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52492/0 (socket says 172.21.15.7:52492) 2024-01-27T02:41:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.602+0000 7ff0b7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff0b80cbbc0 0x7ff0b80cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40790/0 (socket says 172.21.15.7:40790) 2024-01-27T02:41:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.602+0000 7ff0b6ffd700 1 -- 172.21.15.7:0/289658386 learned_addr learned my addr 172.21.15.7:0/289658386 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:41:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.602+0000 7ff0b77fe700 1 --2- 172.21.15.7:0/289658386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b80c8ab0 0x7ff0b815dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.602+0000 7ff0b7fff700 1 -- 172.21.15.7:0/289658386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b80c8ab0 msgr2=0x7ff0b815dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.602+0000 7ff0b7fff700 1 --2- 172.21.15.7:0/289658386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b80c8ab0 0x7ff0b815dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.602+0000 7ff0b7fff700 1 -- 172.21.15.7:0/289658386 >> v1:172.21.15.7:6789/0 conn(0x7ff0b80c8e80 legacy=0x7ff0b80cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.602+0000 7ff0b7fff700 1 -- 172.21.15.7:0/289658386 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b815e6f0 con 0x7ff0b80cbbc0 2024-01-27T02:41:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.602+0000 7ff0b67fc700 1 -- 172.21.15.7:0/289658386 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147872285 0 0) 0x7ff0b815a9b0 con 0x7ff0b80c8e80 2024-01-27T02:41:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.602+0000 7ff0b77fe700 1 --2- 172.21.15.7:0/289658386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b80c8ab0 0x7ff0b815dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:41:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.602+0000 7ff0b7fff700 1 --2- 172.21.15.7:0/289658386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0b80cbbc0 0x7ff0b80cc040 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009e60 tx=0x7ff0b0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b078e2e55f0fc90c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.602+0000 7ff0b67fc700 1 -- 172.21.15.7:0/289658386 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b000e070 con 0x7ff0b80cbbc0 2024-01-27T02:41:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.602+0000 7ff0b67fc700 1 -- 172.21.15.7:0/289658386 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b000bc30 con 0x7ff0b80cbbc0 2024-01-27T02:41:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.602+0000 7ff0b67fc700 1 -- 172.21.15.7:0/289658386 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0012430 con 0x7ff0b80cbbc0 2024-01-27T02:41:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.603+0000 7ff0bdc10700 1 -- 172.21.15.7:0/289658386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0b80cbbc0 msgr2=0x7ff0b80cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.603+0000 7ff0bdc10700 1 --2- 172.21.15.7:0/289658386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0b80cbbc0 0x7ff0b80cc040 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009e60 tx=0x7ff0b0009780 comp rx=0 tx=0).stop 2024-01-27T02:41:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.603+0000 7ff0bdc10700 1 -- 172.21.15.7:0/289658386 shutdown_connections 2024-01-27T02:41:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.603+0000 7ff0bdc10700 1 --2- 172.21.15.7:0/289658386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0b80cbbc0 0x7ff0b80cc040 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.603+0000 7ff0bdc10700 1 --2- 172.21.15.7:0/289658386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b80c8ab0 0x7ff0b815dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.603+0000 7ff0bdc10700 1 -- 172.21.15.7:0/289658386 >> 172.21.15.7:0/289658386 conn(0x7ff0b80cdc30 msgr2=0x7ff0b80ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.603+0000 7ff0bdc10700 1 -- 172.21.15.7:0/289658386 shutdown_connections 2024-01-27T02:41:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.603+0000 7ff0bdc10700 1 -- 172.21.15.7:0/289658386 wait complete. 2024-01-27T02:41:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.604+0000 7ff0bdc10700 1 Processor -- start 2024-01-27T02:41:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.604+0000 7ff0bdc10700 1 -- start start 2024-01-27T02:41:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.605+0000 7ff0bdc10700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0b80c8ab0 0x7ff0b816d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.605+0000 7ff0bdc10700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0b80c8e80 0x7ff0b816d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.605+0000 7ff0bdc10700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b80cbbc0 0x7ff0b80b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.605+0000 7ff0bdc10700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b8159550 con 0x7ff0b80c8e80 2024-01-27T02:41:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.605+0000 7ff0bdc10700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b8158940 con 0x7ff0b80c8ab0 2024-01-27T02:41:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.605+0000 7ff0bdc10700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b8158c20 con 0x7ff0b80cbbc0 2024-01-27T02:41:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.605+0000 7ff0b6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0b80c8e80 0x7ff0b816d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.605+0000 7ff0b6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0b80c8e80 0x7ff0b816d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40806/0 (socket says 172.21.15.7:40806) 2024-01-27T02:41:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.605+0000 7ff0b6ffd700 1 -- 172.21.15.7:0/552234255 learned_addr learned my addr 172.21.15.7:0/552234255 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.606+0000 7ff0b77fe700 1 --2- 172.21.15.7:0/552234255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0b80c8ab0 0x7ff0b816d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.606+0000 7ff0b7fff700 1 --2- 172.21.15.7:0/552234255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b80cbbc0 0x7ff0b80b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.606+0000 7ff0b6ffd700 1 -- 172.21.15.7:0/552234255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b80cbbc0 msgr2=0x7ff0b80b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.606+0000 7ff0b6ffd700 1 --2- 172.21.15.7:0/552234255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b80cbbc0 0x7ff0b80b2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.606+0000 7ff0b6ffd700 1 -- 172.21.15.7:0/552234255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0b80c8ab0 msgr2=0x7ff0b816d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:45.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.606+0000 7ff0b6ffd700 1 --2- 172.21.15.7:0/552234255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0b80c8ab0 0x7ff0b816d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.606+0000 7ff0b6ffd700 1 -- 172.21.15.7:0/552234255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b0009a20 con 0x7ff0b80c8e80 2024-01-27T02:41:45.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.606+0000 7ff0b6ffd700 1 --2- 172.21.15.7:0/552234255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0b80c8e80 0x7ff0b816d640 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00c7e0 tx=0x7ff0ac007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:45.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.606+0000 7ff09f7fe700 1 -- 172.21.15.7:0/552234255 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac005b30 con 0x7ff0b80c8e80 2024-01-27T02:41:45.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.606+0000 7ff09f7fe700 1 -- 172.21.15.7:0/552234255 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ac005c90 con 0x7ff0b80c8e80 2024-01-27T02:41:45.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.606+0000 7ff0bdc10700 1 -- 172.21.15.7:0/552234255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b8158f00 con 0x7ff0b80c8e80 2024-01-27T02:41:45.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.606+0000 7ff09f7fe700 1 -- 172.21.15.7:0/552234255 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac007300 con 0x7ff0b80c8e80 2024-01-27T02:41:45.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.606+0000 7ff0bdc10700 1 -- 172.21.15.7:0/552234255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b81599e0 con 0x7ff0b80c8e80 2024-01-27T02:41:45.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.607+0000 7ff0bdc10700 1 -- 172.21.15.7:0/552234255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b8159c90 con 0x7ff0b80c8e80 2024-01-27T02:41:45.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.608+0000 7ff09f7fe700 1 -- 172.21.15.7:0/552234255 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff0ac01b400 con 0x7ff0b80c8e80 2024-01-27T02:41:45.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.608+0000 7ff09f7fe700 1 --2- 172.21.15.7:0/552234255 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff0a4066f30 0x7ff0a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:45.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.608+0000 7ff09f7fe700 1 -- 172.21.15.7:0/552234255 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5763+0+0 (secure 0 0 0) 0x7ff0ac0840f0 con 0x7ff0b80c8e80 2024-01-27T02:41:45.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.608+0000 7ff09f7fe700 1 -- 172.21.15.7:0/552234255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=154}) v3 -- 0x7ff0a406eb10 con 0x7ff0b80c8e80 2024-01-27T02:41:45.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.608+0000 7ff0b77fe700 1 --2- 172.21.15.7:0/552234255 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff0a4066f30 0x7ff0a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:45.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.609+0000 7ff0b77fe700 1 --2- 172.21.15.7:0/552234255 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff0a4066f30 0x7ff0a40693e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8006f00 tx=0x7ff0a8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.613+0000 7ff09f7fe700 1 -- 172.21.15.7:0/552234255 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff0ac0513a0 con 0x7ff0b80c8e80 2024-01-27T02:41:45.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.870+0000 7ff0bdc10700 1 -- 172.21.15.7:0/552234255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0b81593a0 con 0x7ff0b80c8e80 2024-01-27T02:41:45.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.870+0000 7ff09f7fe700 1 -- 172.21.15.7:0/552234255 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4599 (secure 0 0 0) 0x7ff0ac0549c0 con 0x7ff0b80c8e80 2024-01-27T02:41:45.871 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:41:45.871 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6260,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3240780787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3240780787},{"type":"v1","addr":"172.21.15.188:6837","nonce":3240780787}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7156,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/4051709811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4051709811},{"type":"v1","addr":"172.21.15.188:6836","nonce":4051709811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7416,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/4000944335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":4000944335},{"type":"v1","addr":"172.21.15.7:6834","nonce":4000944335}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:39:22.327946+0000","modified":"2024-01-27T02:39:25.910241+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7419":{"gid":7419,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2386550039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2386550039},{"type":"v1","addr":"172.21.15.7:6835","nonce":2386550039}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2024-01-27T02:41:45.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.873+0000 7ff0bdc10700 1 -- 172.21.15.7:0/552234255 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff0a4066f30 msgr2=0x7ff0a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.873+0000 7ff0bdc10700 1 --2- 172.21.15.7:0/552234255 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff0a4066f30 0x7ff0a40693e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8006f00 tx=0x7ff0a8008040 comp rx=0 tx=0).stop 2024-01-27T02:41:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.873+0000 7ff0bdc10700 1 -- 172.21.15.7:0/552234255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0b80c8e80 msgr2=0x7ff0b816d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.873+0000 7ff0bdc10700 1 --2- 172.21.15.7:0/552234255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0b80c8e80 0x7ff0b816d640 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00c7e0 tx=0x7ff0ac007df0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.873+0000 7ff0bdc10700 1 -- 172.21.15.7:0/552234255 shutdown_connections 2024-01-27T02:41:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.873+0000 7ff0bdc10700 1 --2- 172.21.15.7:0/552234255 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff0a4066f30 0x7ff0a40693e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.873+0000 7ff0bdc10700 1 --2- 172.21.15.7:0/552234255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b80cbbc0 0x7ff0b80b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.873+0000 7ff0bdc10700 1 --2- 172.21.15.7:0/552234255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0b80c8ab0 0x7ff0b816d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.873+0000 7ff0bdc10700 1 --2- 172.21.15.7:0/552234255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0b80c8e80 0x7ff0b816d640 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.873+0000 7ff0bdc10700 1 -- 172.21.15.7:0/552234255 >> 172.21.15.7:0/552234255 conn(0x7ff0b80cdc30 msgr2=0x7ff0b80ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.873+0000 7ff0bdc10700 1 -- 172.21.15.7:0/552234255 shutdown_connections 2024-01-27T02:41:45.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:45.873+0000 7ff0bdc10700 1 -- 172.21.15.7:0/552234255 wait complete. 2024-01-27T02:41:45.875 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 77 2024-01-27T02:41:45.887 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:41:46.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.018+0000 7fe751335700 1 Processor -- start 2024-01-27T02:41:46.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.019+0000 7fe751335700 1 -- start start 2024-01-27T02:41:46.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.019+0000 7fe751335700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe74c15b8e0 0x7fe74c0b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:46.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.019+0000 7fe751335700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe74c0b3900 con 0x7fe74c0b2ff0 2024-01-27T02:41:46.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.019+0000 7fe751335700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe74c15ebe0 con 0x7fe74c15e4e0 2024-01-27T02:41:46.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.019+0000 7fe751335700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe74c1587b0 con 0x7fe74c15b8e0 2024-01-27T02:41:46.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.020+0000 7fe74b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe74c15b8e0 0x7fe74c0b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:46.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.020+0000 7fe74affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe74c0b2ff0 0x7fe74c15ddd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52500/0 (socket says 172.21.15.7:52500) 2024-01-27T02:41:46.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.020+0000 7fe74affd700 1 -- 172.21.15.7:0/1153670487 learned_addr learned my addr 172.21.15.7:0/1153670487 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:41:46.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.020+0000 7fe74b7fe700 1 -- 172.21.15.7:0/1153670487 >> v1:172.21.15.188:6789/0 conn(0x7fe74c15e4e0 legacy=0x7fe74c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:46.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.020+0000 7fe74b7fe700 1 -- 172.21.15.7:0/1153670487 >> v1:172.21.15.7:6789/0 conn(0x7fe74c0b2ff0 legacy=0x7fe74c15ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:46.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.020+0000 7fe74b7fe700 1 -- 172.21.15.7:0/1153670487 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74c15be30 con 0x7fe74c15b8e0 2024-01-27T02:41:46.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.020+0000 7fe74b7fe700 1 --2- 172.21.15.7:0/1153670487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe74c15b8e0 0x7fe74c0b33c0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fe744009e60 tx=0x7fe744009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a0f3bf2b428b83 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:46.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.020+0000 7fe749ffb700 1 -- 172.21.15.7:0/1153670487 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74400e070 con 0x7fe74c15b8e0 2024-01-27T02:41:46.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.020+0000 7fe749ffb700 1 -- 172.21.15.7:0/1153670487 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74400bc30 con 0x7fe74c15b8e0 2024-01-27T02:41:46.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.020+0000 7fe749ffb700 1 -- 172.21.15.7:0/1153670487 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744012430 con 0x7fe74c15b8e0 2024-01-27T02:41:46.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.021+0000 7fe751335700 1 -- 172.21.15.7:0/1153670487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe74c15b8e0 msgr2=0x7fe74c0b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:46.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.021+0000 7fe751335700 1 --2- 172.21.15.7:0/1153670487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe74c15b8e0 0x7fe74c0b33c0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fe744009e60 tx=0x7fe744009780 comp rx=0 tx=0).stop 2024-01-27T02:41:46.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.021+0000 7fe751335700 1 -- 172.21.15.7:0/1153670487 shutdown_connections 2024-01-27T02:41:46.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.021+0000 7fe751335700 1 --2- 172.21.15.7:0/1153670487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe74c15b8e0 0x7fe74c0b33c0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:46.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.021+0000 7fe751335700 1 -- 172.21.15.7:0/1153670487 >> 172.21.15.7:0/1153670487 conn(0x7fe74c0679b0 msgr2=0x7fe74c067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:46.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.021+0000 7fe751335700 1 -- 172.21.15.7:0/1153670487 shutdown_connections 2024-01-27T02:41:46.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.021+0000 7fe751335700 1 -- 172.21.15.7:0/1153670487 wait complete. 2024-01-27T02:41:46.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.022+0000 7fe751335700 1 Processor -- start 2024-01-27T02:41:46.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.023+0000 7fe751335700 1 -- start start 2024-01-27T02:41:46.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.023+0000 7fe751335700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe74c0b2ff0 0x7fe74c0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:46.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.023+0000 7fe751335700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe74c15b8e0 0x7fe74c153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:46.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.023+0000 7fe751335700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe74c15e4e0 0x7fe74c15afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.023+0000 7fe751335700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c157cc0 con 0x7fe74c15b8e0 2024-01-27T02:41:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.023+0000 7fe751335700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c157420 con 0x7fe74c0b2ff0 2024-01-27T02:41:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.023+0000 7fe751335700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c157700 con 0x7fe74c15e4e0 2024-01-27T02:41:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.023+0000 7fe74a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe74c15b8e0 0x7fe74c153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.024+0000 7fe74a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe74c15b8e0 0x7fe74c153e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40822/0 (socket says 172.21.15.7:40822) 2024-01-27T02:41:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.024+0000 7fe74a7fc700 1 -- 172.21.15.7:0/603293818 learned_addr learned my addr 172.21.15.7:0/603293818 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.024+0000 7fe74b7fe700 1 --2- 172.21.15.7:0/603293818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe74c15e4e0 0x7fe74c15afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.024+0000 7fe74affd700 1 --2- 172.21.15.7:0/603293818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe74c0b2ff0 0x7fe74c0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.024+0000 7fe74a7fc700 1 -- 172.21.15.7:0/603293818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe74c15e4e0 msgr2=0x7fe74c15afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.024+0000 7fe74a7fc700 1 --2- 172.21.15.7:0/603293818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe74c15e4e0 0x7fe74c15afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.024+0000 7fe74a7fc700 1 -- 172.21.15.7:0/603293818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe74c0b2ff0 msgr2=0x7fe74c0b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.024+0000 7fe74a7fc700 1 --2- 172.21.15.7:0/603293818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe74c0b2ff0 0x7fe74c0b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:46.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.024+0000 7fe74a7fc700 1 -- 172.21.15.7:0/603293818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe744009a20 con 0x7fe74c15b8e0 2024-01-27T02:41:46.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.024+0000 7fe74a7fc700 1 --2- 172.21.15.7:0/603293818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe74c15b8e0 0x7fe74c153e70 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fe74000c590 tx=0x7fe740007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:46.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.024+0000 7fe732ffd700 1 -- 172.21.15.7:0/603293818 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe740011590 con 0x7fe74c15b8e0 2024-01-27T02:41:46.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.024+0000 7fe732ffd700 1 -- 172.21.15.7:0/603293818 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7400116f0 con 0x7fe74c15b8e0 2024-01-27T02:41:46.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.024+0000 7fe732ffd700 1 -- 172.21.15.7:0/603293818 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe740011ab0 con 0x7fe74c15b8e0 2024-01-27T02:41:46.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.024+0000 7fe751335700 1 -- 172.21.15.7:0/603293818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe74c1579e0 con 0x7fe74c15b8e0 2024-01-27T02:41:46.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.024+0000 7fe751335700 1 -- 172.21.15.7:0/603293818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe74c1543b0 con 0x7fe74c15b8e0 2024-01-27T02:41:46.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.025+0000 7fe751335700 1 -- 172.21.15.7:0/603293818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74c152600 con 0x7fe74c15b8e0 2024-01-27T02:41:46.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.026+0000 7fe732ffd700 1 -- 172.21.15.7:0/603293818 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe74001b3b0 con 0x7fe74c15b8e0 2024-01-27T02:41:46.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.026+0000 7fe732ffd700 1 --2- 172.21.15.7:0/603293818 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe738066f30 0x7fe7380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:46.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.026+0000 7fe732ffd700 1 -- 172.21.15.7:0/603293818 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fe7400838e0 con 0x7fe74c15b8e0 2024-01-27T02:41:46.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.026+0000 7fe74affd700 1 --2- 172.21.15.7:0/603293818 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe738066f30 0x7fe7380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:46.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.026+0000 7fe732ffd700 1 -- 172.21.15.7:0/603293818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=154}) v3 -- 0x7fe73806eb10 con 0x7fe74c15b8e0 2024-01-27T02:41:46.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.027+0000 7fe74affd700 1 --2- 172.21.15.7:0/603293818 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe738066f30 0x7fe7380693e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe73c002950 tx=0x7fe73c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:46.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.030+0000 7fe732ffd700 1 -- 172.21.15.7:0/603293818 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe740017070 con 0x7fe74c15b8e0 2024-01-27T02:41:46.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.284+0000 7fe751335700 1 -- 172.21.15.7:0/603293818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe74c04c710 con 0x7fe74c15b8e0 2024-01-27T02:41:46.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.285+0000 7fe732ffd700 1 -- 172.21.15.7:0/603293818 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fe74c152600 con 0x7fe74c15b8e0 2024-01-27T02:41:46.286 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:41:46.286 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6260,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3240780787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3240780787},{"type":"v1","addr":"172.21.15.188:6837","nonce":3240780787}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7156,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/4051709811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4051709811},{"type":"v1","addr":"172.21.15.188:6836","nonce":4051709811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7416,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/4000944335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":4000944335},{"type":"v1","addr":"172.21.15.7:6834","nonce":4000944335}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:39:22.327946+0000","modified":"2024-01-27T02:39:25.910241+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7419":{"gid":7419,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2386550039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2386550039},{"type":"v1","addr":"172.21.15.7:6835","nonce":2386550039}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2024-01-27T02:41:46.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.287+0000 7fe751335700 1 -- 172.21.15.7:0/603293818 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe738066f30 msgr2=0x7fe7380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:46.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.287+0000 7fe751335700 1 --2- 172.21.15.7:0/603293818 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe738066f30 0x7fe7380693e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe73c002950 tx=0x7fe73c009040 comp rx=0 tx=0).stop 2024-01-27T02:41:46.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.287+0000 7fe751335700 1 -- 172.21.15.7:0/603293818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe74c15b8e0 msgr2=0x7fe74c153e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:46.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.287+0000 7fe751335700 1 --2- 172.21.15.7:0/603293818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe74c15b8e0 0x7fe74c153e70 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fe74000c590 tx=0x7fe740007ee0 comp rx=0 tx=0).stop 2024-01-27T02:41:46.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.287+0000 7fe751335700 1 -- 172.21.15.7:0/603293818 shutdown_connections 2024-01-27T02:41:46.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.287+0000 7fe751335700 1 --2- 172.21.15.7:0/603293818 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe738066f30 0x7fe7380693e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:46.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.287+0000 7fe751335700 1 --2- 172.21.15.7:0/603293818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe74c15e4e0 0x7fe74c15afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:46.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.287+0000 7fe751335700 1 --2- 172.21.15.7:0/603293818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe74c0b2ff0 0x7fe74c0b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:46.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.287+0000 7fe751335700 1 --2- 172.21.15.7:0/603293818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe74c15b8e0 0x7fe74c153e70 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:46.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.287+0000 7fe751335700 1 -- 172.21.15.7:0/603293818 >> 172.21.15.7:0/603293818 conn(0x7fe74c0679b0 msgr2=0x7fe74c0cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:46.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.287+0000 7fe751335700 1 -- 172.21.15.7:0/603293818 shutdown_connections 2024-01-27T02:41:46.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.287+0000 7fe751335700 1 -- 172.21.15.7:0/603293818 wait complete. 2024-01-27T02:41:46.291 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 77 2024-01-27T02:41:46.302 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:41:46.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.434+0000 7f81f5a6a700 1 Processor -- start 2024-01-27T02:41:46.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.434+0000 7f81f5a6a700 1 -- start start 2024-01-27T02:41:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.435+0000 7f81f5a6a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f81f00c3ae0 0x7f81f00cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.435+0000 7f81f5a6a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f00ca020 con 0x7f81f0157ca0 2024-01-27T02:41:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.435+0000 7f81f5a6a700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f00c5210 con 0x7f81f0158070 2024-01-27T02:41:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.435+0000 7f81f5a6a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f00c54f0 con 0x7f81f00c3ae0 2024-01-27T02:41:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.435+0000 7f81eeffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f81f00c3ae0 0x7f81f00cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.435+0000 7f81ef7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f81f0157ca0 0x7f81f00c4a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52506/0 (socket says 172.21.15.7:52506) 2024-01-27T02:41:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.435+0000 7f81ef7fe700 1 -- 172.21.15.7:0/1335832259 learned_addr learned my addr 172.21.15.7:0/1335832259 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:41:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.435+0000 7f81eeffd700 1 -- 172.21.15.7:0/1335832259 >> v1:172.21.15.188:6789/0 conn(0x7f81f0158070 legacy=0x7f81f00ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.435+0000 7f81eeffd700 1 -- 172.21.15.7:0/1335832259 >> v1:172.21.15.7:6789/0 conn(0x7f81f0157ca0 legacy=0x7f81f00c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.435+0000 7f81eeffd700 1 -- 172.21.15.7:0/1335832259 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f00c57d0 con 0x7f81f00c3ae0 2024-01-27T02:41:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.435+0000 7f81ee7fc700 1 -- 172.21.15.7:0/1335832259 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2541694785 0 0) 0x7f81f00ca020 con 0x7f81f0157ca0 2024-01-27T02:41:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.436+0000 7f81eeffd700 1 --2- 172.21.15.7:0/1335832259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81f00c3ae0 0x7f81f00cd4e0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f81e40098e0 tx=0x7f81e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0db18931ceb05ae server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.436+0000 7f81ee7fc700 1 -- 172.21.15.7:0/1335832259 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e4009e80 con 0x7f81f00c3ae0 2024-01-27T02:41:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.436+0000 7f81ee7fc700 1 -- 172.21.15.7:0/1335832259 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e400b860 con 0x7f81f00c3ae0 2024-01-27T02:41:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.436+0000 7f81ee7fc700 1 -- 172.21.15.7:0/1335832259 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e400bbe0 con 0x7f81f00c3ae0 2024-01-27T02:41:46.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.436+0000 7f81f5a6a700 1 -- 172.21.15.7:0/1335832259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81f00c3ae0 msgr2=0x7f81f00cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:46.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.436+0000 7f81f5a6a700 1 --2- 172.21.15.7:0/1335832259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81f00c3ae0 0x7f81f00cd4e0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f81e40098e0 tx=0x7f81e40091a0 comp rx=0 tx=0).stop 2024-01-27T02:41:46.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.436+0000 7f81f5a6a700 1 -- 172.21.15.7:0/1335832259 shutdown_connections 2024-01-27T02:41:46.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.436+0000 7f81f5a6a700 1 --2- 172.21.15.7:0/1335832259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81f00c3ae0 0x7f81f00cd4e0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:46.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.436+0000 7f81f5a6a700 1 -- 172.21.15.7:0/1335832259 >> 172.21.15.7:0/1335832259 conn(0x7f81f0098020 msgr2=0x7f81f00711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:46.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.436+0000 7f81f5a6a700 1 -- 172.21.15.7:0/1335832259 shutdown_connections 2024-01-27T02:41:46.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.436+0000 7f81f5a6a700 1 -- 172.21.15.7:0/1335832259 wait complete. 2024-01-27T02:41:46.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.438+0000 7f81f5a6a700 1 Processor -- start 2024-01-27T02:41:46.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.438+0000 7f81f5a6a700 1 -- start start 2024-01-27T02:41:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81f5a6a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81f00c3ae0 0x7f81f00b4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81f5a6a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81f0157ca0 0x7f81f00c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81f5a6a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81f0158070 0x7f81f00c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81f5a6a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f00b14f0 con 0x7f81f00c3ae0 2024-01-27T02:41:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81f5a6a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f0068310 con 0x7f81f0158070 2024-01-27T02:41:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81f5a6a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f00b4970 con 0x7f81f0157ca0 2024-01-27T02:41:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81f00c3ae0 0x7f81f00b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81f00c3ae0 0x7f81f00b4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40852/0 (socket says 172.21.15.7:40852) 2024-01-27T02:41:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81ef7fe700 1 -- 172.21.15.7:0/2037969357 learned_addr learned my addr 172.21.15.7:0/2037969357 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81effff700 1 --2- 172.21.15.7:0/2037969357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81f0158070 0x7f81f00c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:46.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81eeffd700 1 --2- 172.21.15.7:0/2037969357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81f0157ca0 0x7f81f00c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:46.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81ef7fe700 1 -- 172.21.15.7:0/2037969357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81f0157ca0 msgr2=0x7f81f00c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:46.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81ef7fe700 1 --2- 172.21.15.7:0/2037969357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81f0157ca0 0x7f81f00c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:46.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81ef7fe700 1 -- 172.21.15.7:0/2037969357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81f0158070 msgr2=0x7f81f00c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:46.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81ef7fe700 1 --2- 172.21.15.7:0/2037969357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81f0158070 0x7f81f00c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:46.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81ef7fe700 1 -- 172.21.15.7:0/2037969357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e4009420 con 0x7f81f00c3ae0 2024-01-27T02:41:46.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81effff700 1 --2- 172.21.15.7:0/2037969357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81f0158070 0x7f81f00c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:41:46.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81eeffd700 1 --2- 172.21.15.7:0/2037969357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81f0157ca0 0x7f81f00c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:41:46.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81ef7fe700 1 --2- 172.21.15.7:0/2037969357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81f00c3ae0 0x7f81f00b4430 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f81e000c810 tx=0x7f81e0007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:46.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81d77fe700 1 -- 172.21.15.7:0/2037969357 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e0005c00 con 0x7f81f00c3ae0 2024-01-27T02:41:46.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81d77fe700 1 -- 172.21.15.7:0/2037969357 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e0005d60 con 0x7f81f00c3ae0 2024-01-27T02:41:46.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81d77fe700 1 -- 172.21.15.7:0/2037969357 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e0007340 con 0x7f81f00c3ae0 2024-01-27T02:41:46.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.439+0000 7f81f5a6a700 1 -- 172.21.15.7:0/2037969357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f0067a50 con 0x7f81f00c3ae0 2024-01-27T02:41:46.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.440+0000 7f81f5a6a700 1 -- 172.21.15.7:0/2037969357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f0067fa0 con 0x7f81f00c3ae0 2024-01-27T02:41:46.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.441+0000 7f81f5a6a700 1 -- 172.21.15.7:0/2037969357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f004c710 con 0x7f81f00c3ae0 2024-01-27T02:41:46.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.441+0000 7f81d77fe700 1 -- 172.21.15.7:0/2037969357 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f81e001b400 con 0x7f81f00c3ae0 2024-01-27T02:41:46.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.441+0000 7f81d77fe700 1 --2- 172.21.15.7:0/2037969357 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f81dc066f30 0x7f81dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.441+0000 7f81d77fe700 1 -- 172.21.15.7:0/2037969357 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f81e0084020 con 0x7f81f00c3ae0 2024-01-27T02:41:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.442+0000 7f81d77fe700 1 -- 172.21.15.7:0/2037969357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=154}) v3 -- 0x7f81dc06eb10 con 0x7f81f00c3ae0 2024-01-27T02:41:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.442+0000 7f81eeffd700 1 --2- 172.21.15.7:0/2037969357 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f81dc066f30 0x7f81dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.442+0000 7f81eeffd700 1 --2- 172.21.15.7:0/2037969357 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f81dc066f30 0x7f81dc0693e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f81e40093f0 tx=0x7f81e4017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:46.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.446+0000 7f81d77fe700 1 -- 172.21.15.7:0/2037969357 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81e00512d0 con 0x7f81f00c3ae0 2024-01-27T02:41:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.654+0000 7f81f5a6a700 1 -- 172.21.15.7:0/2037969357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81f00c9bb0 con 0x7f81f00c3ae0 2024-01-27T02:41:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.655+0000 7f81d77fe700 1 -- 172.21.15.7:0/2037969357 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+15440 (secure 0 0 0) 0x7f81e00548f0 con 0x7f81f00c3ae0 2024-01-27T02:41:46.655 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:41:46.655 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":153,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:41:04.900114+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:39:20.119244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-01-27T02:39:21.293772+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.7","mappings":[{"from":2,"to":0}]},{"pgid":"19.3f","mappings":[{"from":2,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:41:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.657+0000 7f81f5a6a700 1 -- 172.21.15.7:0/2037969357 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f81dc066f30 msgr2=0x7f81dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.657+0000 7f81f5a6a700 1 --2- 172.21.15.7:0/2037969357 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f81dc066f30 0x7f81dc0693e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f81e40093f0 tx=0x7f81e4017000 comp rx=0 tx=0).stop 2024-01-27T02:41:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.657+0000 7f81f5a6a700 1 -- 172.21.15.7:0/2037969357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81f00c3ae0 msgr2=0x7f81f00b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.657+0000 7f81f5a6a700 1 --2- 172.21.15.7:0/2037969357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81f00c3ae0 0x7f81f00b4430 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f81e000c810 tx=0x7f81e0007d90 comp rx=0 tx=0).stop 2024-01-27T02:41:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.657+0000 7f81f5a6a700 1 -- 172.21.15.7:0/2037969357 shutdown_connections 2024-01-27T02:41:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.657+0000 7f81f5a6a700 1 --2- 172.21.15.7:0/2037969357 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f81dc066f30 0x7f81dc0693e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:46.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.657+0000 7f81f5a6a700 1 --2- 172.21.15.7:0/2037969357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81f0158070 0x7f81f00c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:46.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.657+0000 7f81f5a6a700 1 --2- 172.21.15.7:0/2037969357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81f00c3ae0 0x7f81f00b4430 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:46.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.657+0000 7f81f5a6a700 1 --2- 172.21.15.7:0/2037969357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81f0157ca0 0x7f81f00c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:46.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.657+0000 7f81f5a6a700 1 -- 172.21.15.7:0/2037969357 >> 172.21.15.7:0/2037969357 conn(0x7f81f0098020 msgr2=0x7f81f006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:46.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.657+0000 7f81f5a6a700 1 -- 172.21.15.7:0/2037969357 shutdown_connections 2024-01-27T02:41:46.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.657+0000 7f81f5a6a700 1 -- 172.21.15.7:0/2037969357 wait complete. 2024-01-27T02:41:46.672 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-27T02:41:46.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.805+0000 7faf7c69f700 1 Processor -- start 2024-01-27T02:41:46.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.805+0000 7faf7c69f700 1 -- start start 2024-01-27T02:41:46.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.806+0000 7faf7c69f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faf74157ca0 0x7faf740c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:46.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.806+0000 7faf7c69f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faf740c3ae0 0x7faf740cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:46.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.806+0000 7faf7c69f700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf740ca020 con 0x7faf74158070 2024-01-27T02:41:46.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.806+0000 7faf7c69f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf740c4f40 con 0x7faf74157ca0 2024-01-27T02:41:46.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.806+0000 7faf7c69f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf740c5210 con 0x7faf740c3ae0 2024-01-27T02:41:46.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.806+0000 7faf7a43b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faf74157ca0 0x7faf740c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:46.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.806+0000 7faf7a43b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faf74157ca0 0x7faf740c4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40866/0 (socket says 172.21.15.7:40866) 2024-01-27T02:41:46.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.806+0000 7faf7a43b700 1 -- 172.21.15.7:0/4003533996 learned_addr learned my addr 172.21.15.7:0/4003533996 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:46.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.806+0000 7faf79c3a700 1 --2- 172.21.15.7:0/4003533996 >> v2:172.21.15.188:3300/0 conn(0x7faf740c3ae0 0x7faf740cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:46.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.806+0000 7faf7a43b700 1 -- 172.21.15.7:0/4003533996 >> v2:172.21.15.188:3300/0 conn(0x7faf740c3ae0 msgr2=0x7faf740cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:46.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.806+0000 7faf7a43b700 1 --2- 172.21.15.7:0/4003533996 >> v2:172.21.15.188:3300/0 conn(0x7faf740c3ae0 0x7faf740cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:46.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.806+0000 7faf7a43b700 1 -- 172.21.15.7:0/4003533996 >> v1:172.21.15.188:6789/0 conn(0x7faf74158070 legacy=0x7faf7415bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:46.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.806+0000 7faf7a43b700 1 -- 172.21.15.7:0/4003533996 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf740c54f0 con 0x7faf74157ca0 2024-01-27T02:41:46.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.806+0000 7faf79c3a700 1 --2- 172.21.15.7:0/4003533996 >> v2:172.21.15.188:3300/0 conn(0x7faf740c3ae0 0x7faf740cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:41:46.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.806+0000 7faf7a43b700 1 --2- 172.21.15.7:0/4003533996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf74157ca0 0x7faf740c4a00 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7faf7406e170 tx=0x7faf70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5627718b16bb402 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:46.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.807+0000 7faf79439700 1 -- 172.21.15.7:0/4003533996 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7000b820 con 0x7faf74157ca0 2024-01-27T02:41:46.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.807+0000 7faf79439700 1 -- 172.21.15.7:0/4003533996 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7000b980 con 0x7faf74157ca0 2024-01-27T02:41:46.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.807+0000 7faf79439700 1 -- 172.21.15.7:0/4003533996 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7000bd20 con 0x7faf74157ca0 2024-01-27T02:41:46.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.807+0000 7faf7c69f700 1 -- 172.21.15.7:0/4003533996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf74157ca0 msgr2=0x7faf740c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:46.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.807+0000 7faf7c69f700 1 --2- 172.21.15.7:0/4003533996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf74157ca0 0x7faf740c4a00 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7faf7406e170 tx=0x7faf70009250 comp rx=0 tx=0).stop 2024-01-27T02:41:46.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.807+0000 7faf7c69f700 1 -- 172.21.15.7:0/4003533996 shutdown_connections 2024-01-27T02:41:46.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.807+0000 7faf7c69f700 1 --2- 172.21.15.7:0/4003533996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf74157ca0 0x7faf740c4a00 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:46.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.807+0000 7faf7c69f700 1 --2- 172.21.15.7:0/4003533996 >> v2:172.21.15.188:3300/0 conn(0x7faf740c3ae0 0x7faf740cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:46.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.807+0000 7faf7c69f700 1 -- 172.21.15.7:0/4003533996 >> 172.21.15.7:0/4003533996 conn(0x7faf74098020 msgr2=0x7faf740711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:46.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.807+0000 7faf7c69f700 1 -- 172.21.15.7:0/4003533996 shutdown_connections 2024-01-27T02:41:46.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.807+0000 7faf7c69f700 1 -- 172.21.15.7:0/4003533996 wait complete. 2024-01-27T02:41:46.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.809+0000 7faf7c69f700 1 Processor -- start 2024-01-27T02:41:46.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.809+0000 7faf7c69f700 1 -- start start 2024-01-27T02:41:46.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.809+0000 7faf7c69f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf740c3ae0 0x7faf7414d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:46.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.809+0000 7faf7c69f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf74157ca0 0x7faf740c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:46.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.809+0000 7faf7c69f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf74158070 0x7faf740c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:46.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.809+0000 7faf7c69f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7414d780 con 0x7faf740c3ae0 2024-01-27T02:41:46.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.809+0000 7faf7c69f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7414dc50 con 0x7faf74158070 2024-01-27T02:41:46.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.809+0000 7faf7c69f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf740b2250 con 0x7faf74157ca0 2024-01-27T02:41:46.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.809+0000 7faf7a43b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf740c3ae0 0x7faf7414d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:46.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.810+0000 7faf7a43b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf740c3ae0 0x7faf7414d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40872/0 (socket says 172.21.15.7:40872) 2024-01-27T02:41:46.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.810+0000 7faf7a43b700 1 -- 172.21.15.7:0/3505271065 learned_addr learned my addr 172.21.15.7:0/3505271065 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:46.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.810+0000 7faf79c3a700 1 --2- 172.21.15.7:0/3505271065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf74157ca0 0x7faf740c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:46.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.810+0000 7faf7ac3c700 1 --2- 172.21.15.7:0/3505271065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf74158070 0x7faf740c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:46.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.810+0000 7faf7a43b700 1 -- 172.21.15.7:0/3505271065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf74157ca0 msgr2=0x7faf740c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:46.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.810+0000 7faf7a43b700 1 --2- 172.21.15.7:0/3505271065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf74157ca0 0x7faf740c5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:46.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.810+0000 7faf7a43b700 1 -- 172.21.15.7:0/3505271065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf74158070 msgr2=0x7faf740c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:46.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.810+0000 7faf7a43b700 1 --2- 172.21.15.7:0/3505271065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf74158070 0x7faf740c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:46.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.810+0000 7faf7a43b700 1 -- 172.21.15.7:0/3505271065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf700094f0 con 0x7faf740c3ae0 2024-01-27T02:41:46.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.810+0000 7faf7a43b700 1 --2- 172.21.15.7:0/3505271065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf740c3ae0 0x7faf7414d240 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7faf700030c0 tx=0x7faf70016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:46.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.810+0000 7faf6a7fc700 1 -- 172.21.15.7:0/3505271065 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70011e60 con 0x7faf740c3ae0 2024-01-27T02:41:46.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.810+0000 7faf6a7fc700 1 -- 172.21.15.7:0/3505271065 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf70020070 con 0x7faf740c3ae0 2024-01-27T02:41:46.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.810+0000 7faf7c69f700 1 -- 172.21.15.7:0/3505271065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7415e1c0 con 0x7faf740c3ae0 2024-01-27T02:41:46.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.810+0000 7faf6a7fc700 1 -- 172.21.15.7:0/3505271065 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7001b6a0 con 0x7faf740c3ae0 2024-01-27T02:41:46.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.810+0000 7faf7c69f700 1 -- 172.21.15.7:0/3505271065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf740b27d0 con 0x7faf740c3ae0 2024-01-27T02:41:46.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.811+0000 7faf7c69f700 1 -- 172.21.15.7:0/3505271065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf740b29c0 con 0x7faf740c3ae0 2024-01-27T02:41:46.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.812+0000 7faf6a7fc700 1 -- 172.21.15.7:0/3505271065 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7faf70022030 con 0x7faf740c3ae0 2024-01-27T02:41:46.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.812+0000 7faf6a7fc700 1 --2- 172.21.15.7:0/3505271065 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faf58066f30 0x7faf580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:46.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.812+0000 7faf6a7fc700 1 -- 172.21.15.7:0/3505271065 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5763+0+0 (secure 0 0 0) 0x7faf70086590 con 0x7faf740c3ae0 2024-01-27T02:41:46.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.812+0000 7faf6a7fc700 1 -- 172.21.15.7:0/3505271065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=154}) v3 -- 0x7faf5806eb10 con 0x7faf740c3ae0 2024-01-27T02:41:46.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.812+0000 7faf79c3a700 1 --2- 172.21.15.7:0/3505271065 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faf58066f30 0x7faf580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:46.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.813+0000 7faf79c3a700 1 --2- 172.21.15.7:0/3505271065 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faf58066f30 0x7faf580693e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7faf6400abe0 tx=0x7faf64006f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:46.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:46.817+0000 7faf6a7fc700 1 -- 172.21.15.7:0/3505271065 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf700537c0 con 0x7faf740c3ae0 2024-01-27T02:41:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.070+0000 7faf7c69f700 1 -- 172.21.15.7:0/3505271065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7faf740b29c0 con 0x7faf740c3ae0 2024-01-27T02:41:47.071 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-27T02:41:47.071+0000 7fb5c503a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-27T02:41:47.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.073+0000 7faf6a7fc700 1 -- 172.21.15.7:0/3505271065 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(154..154 src has 1..154) v4 ==== 366+0+0 (secure 0 0 0) 0x7faf70057480 con 0x7faf740c3ae0 2024-01-27T02:41:47.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.073+0000 7faf6a7fc700 1 -- 172.21.15.7:0/3505271065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=155}) v3 -- 0x7faf5806f2c0 con 0x7faf740c3ae0 2024-01-27T02:41:47.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.075+0000 7faf6a7fc700 1 -- 172.21.15.7:0/3505271065 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v78) v1 ==== 161+0+0 (secure 0 0 0) 0x7faf70056de0 con 0x7faf740c3ae0 2024-01-27T02:41:47.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.078+0000 7faf7c69f700 1 -- 172.21.15.7:0/3505271065 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faf58066f30 msgr2=0x7faf580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:47.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.078+0000 7faf7c69f700 1 --2- 172.21.15.7:0/3505271065 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faf58066f30 0x7faf580693e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7faf6400abe0 tx=0x7faf64006f90 comp rx=0 tx=0).stop 2024-01-27T02:41:47.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.078+0000 7faf7c69f700 1 -- 172.21.15.7:0/3505271065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf740c3ae0 msgr2=0x7faf7414d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:47.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.078+0000 7faf7c69f700 1 --2- 172.21.15.7:0/3505271065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf740c3ae0 0x7faf7414d240 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7faf700030c0 tx=0x7faf70016000 comp rx=0 tx=0).stop 2024-01-27T02:41:47.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.078+0000 7faf7c69f700 1 -- 172.21.15.7:0/3505271065 shutdown_connections 2024-01-27T02:41:47.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.078+0000 7faf7c69f700 1 --2- 172.21.15.7:0/3505271065 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faf58066f30 0x7faf580693e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:47.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.078+0000 7faf7c69f700 1 --2- 172.21.15.7:0/3505271065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf74158070 0x7faf740c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:47.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.078+0000 7faf7c69f700 1 --2- 172.21.15.7:0/3505271065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf740c3ae0 0x7faf7414d240 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:47.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.078+0000 7faf7c69f700 1 --2- 172.21.15.7:0/3505271065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf74157ca0 0x7faf740c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:47.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.078+0000 7faf7c69f700 1 -- 172.21.15.7:0/3505271065 >> 172.21.15.7:0/3505271065 conn(0x7faf74098020 msgr2=0x7faf740ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:47.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.078+0000 7faf7c69f700 1 -- 172.21.15.7:0/3505271065 shutdown_connections 2024-01-27T02:41:47.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.078+0000 7faf7c69f700 1 -- 172.21.15.7:0/3505271065 wait complete. 2024-01-27T02:41:47.080 INFO:tasks.ceph.mds.a.smithi007.stderr: -9470> 2024-01-27T02:39:28.165+0000 7f3190987700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-27T02:41:47.080 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-27T02:41:47.093 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-27T02:41:47.114 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-27T02:41:47.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.228+0000 7f17f5c99700 1 Processor -- start 2024-01-27T02:41:47.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.228+0000 7f17f5c99700 1 -- start start 2024-01-27T02:41:47.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.229+0000 7f17f5c99700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f17f0156b60 0x7f17f00ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:47.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.229+0000 7f17f5c99700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f00ad9f0 0x7f17f00aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:47.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.229+0000 7f17f5c99700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17f00aeaa0 con 0x7f17f00af470 2024-01-27T02:41:47.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.229+0000 7f17f5c99700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17f00ade90 con 0x7f17f0156b60 2024-01-27T02:41:47.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.229+0000 7f17f5c99700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f00ae170 con 0x7f17f00ad9f0 2024-01-27T02:41:47.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.229+0000 7f17ef7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f17f0156b60 0x7f17f00ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:47.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.229+0000 7f17effff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f17f00af470 0x7f17f015a9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52518/0 (socket says 172.21.15.7:52518) 2024-01-27T02:41:47.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.229+0000 7f17effff700 1 -- 172.21.15.7:0/440643140 learned_addr learned my addr 172.21.15.7:0/440643140 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:41:47.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.229+0000 7f17eeffd700 1 --2- 172.21.15.7:0/440643140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f00ad9f0 0x7f17f00aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:47.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.229+0000 7f17ef7fe700 1 -- 172.21.15.7:0/440643140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f00ad9f0 msgr2=0x7f17f00aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:47.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.229+0000 7f17ef7fe700 1 --2- 172.21.15.7:0/440643140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f00ad9f0 0x7f17f00aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:47.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.229+0000 7f17ef7fe700 1 -- 172.21.15.7:0/440643140 >> v1:172.21.15.7:6789/0 conn(0x7f17f00af470 legacy=0x7f17f015a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:47.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.229+0000 7f17ee7fc700 1 -- 172.21.15.7:0/440643140 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445981968 0 0) 0x7f17f00aeaa0 con 0x7f17f00af470 2024-01-27T02:41:47.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.229+0000 7f17ef7fe700 1 -- 172.21.15.7:0/440643140 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f00ae450 con 0x7f17f0156b60 2024-01-27T02:41:47.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.229+0000 7f17eeffd700 1 --2- 172.21.15.7:0/440643140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f00ad9f0 0x7f17f00aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:41:47.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.230+0000 7f17ef7fe700 1 --2- 172.21.15.7:0/440643140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f0156b60 0x7f17f00ad4b0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f17e00098e0 tx=0x7f17e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47cb6e3bda6f45b server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:47.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.230+0000 7f17ee7fc700 1 -- 172.21.15.7:0/440643140 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e0009e80 con 0x7f17f0156b60 2024-01-27T02:41:47.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.230+0000 7f17ee7fc700 1 -- 172.21.15.7:0/440643140 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e000b860 con 0x7f17f0156b60 2024-01-27T02:41:47.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.230+0000 7f17ee7fc700 1 -- 172.21.15.7:0/440643140 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e000bbe0 con 0x7f17f0156b60 2024-01-27T02:41:47.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.230+0000 7f17f5c99700 1 -- 172.21.15.7:0/440643140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f0156b60 msgr2=0x7f17f00ad4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:47.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.230+0000 7f17f5c99700 1 --2- 172.21.15.7:0/440643140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f0156b60 0x7f17f00ad4b0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f17e00098e0 tx=0x7f17e00091a0 comp rx=0 tx=0).stop 2024-01-27T02:41:47.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.230+0000 7f17f5c99700 1 -- 172.21.15.7:0/440643140 shutdown_connections 2024-01-27T02:41:47.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.230+0000 7f17f5c99700 1 --2- 172.21.15.7:0/440643140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f0156b60 0x7f17f00ad4b0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:47.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.230+0000 7f17f5c99700 1 --2- 172.21.15.7:0/440643140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f00ad9f0 0x7f17f00aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:47.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.230+0000 7f17f5c99700 1 -- 172.21.15.7:0/440643140 >> 172.21.15.7:0/440643140 conn(0x7f17f0068480 msgr2=0x7f17f00cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:47.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.230+0000 7f17f5c99700 1 -- 172.21.15.7:0/440643140 shutdown_connections 2024-01-27T02:41:47.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.230+0000 7f17f5c99700 1 -- 172.21.15.7:0/440643140 wait complete. 2024-01-27T02:41:47.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.232+0000 7f17f5c99700 1 Processor -- start 2024-01-27T02:41:47.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.232+0000 7f17f5c99700 1 -- start start 2024-01-27T02:41:47.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.233+0000 7f17f5c99700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f00ad9f0 0x7f17f00ab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:47.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.233+0000 7f17f5c99700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17f00af470 0x7f17f00a8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:47.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.233+0000 7f17f5c99700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f0156b60 0x7f17f01680a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:47.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.233+0000 7f17f5c99700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f00abb20 con 0x7f17f0156b60 2024-01-27T02:41:47.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.233+0000 7f17f5c99700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f00acf00 con 0x7f17f00af470 2024-01-27T02:41:47.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.233+0000 7f17f5c99700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f00a9320 con 0x7f17f00ad9f0 2024-01-27T02:41:47.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.234+0000 7f17effff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f0156b60 0x7f17f01680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:47.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.234+0000 7f17effff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f0156b60 0x7f17f01680a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40908/0 (socket says 172.21.15.7:40908) 2024-01-27T02:41:47.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.234+0000 7f17effff700 1 -- 172.21.15.7:0/3185778859 learned_addr learned my addr 172.21.15.7:0/3185778859 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:47.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.234+0000 7f17ef7fe700 1 --2- 172.21.15.7:0/3185778859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f00ad9f0 0x7f17f00ab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:47.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.234+0000 7f17eeffd700 1 --2- 172.21.15.7:0/3185778859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17f00af470 0x7f17f00a8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:47.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.234+0000 7f17effff700 1 -- 172.21.15.7:0/3185778859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f00ad9f0 msgr2=0x7f17f00ab5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:47.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.234+0000 7f17effff700 1 --2- 172.21.15.7:0/3185778859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f00ad9f0 0x7f17f00ab5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:47.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.234+0000 7f17effff700 1 -- 172.21.15.7:0/3185778859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17f00af470 msgr2=0x7f17f00a8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:47.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.234+0000 7f17effff700 1 --2- 172.21.15.7:0/3185778859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17f00af470 0x7f17f00a8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:47.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.234+0000 7f17effff700 1 -- 172.21.15.7:0/3185778859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e0009420 con 0x7f17f0156b60 2024-01-27T02:41:47.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.234+0000 7f17effff700 1 --2- 172.21.15.7:0/3185778859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f0156b60 0x7f17f01680a0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f17e800cc90 tx=0x7f17e8001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:47.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.234+0000 7f17d77fe700 1 -- 172.21.15.7:0/3185778859 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e800a8b0 con 0x7f17f0156b60 2024-01-27T02:41:47.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.234+0000 7f17d77fe700 1 -- 172.21.15.7:0/3185778859 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e800aa10 con 0x7f17f0156b60 2024-01-27T02:41:47.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.234+0000 7f17f5c99700 1 -- 172.21.15.7:0/3185778859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f00a9550 con 0x7f17f0156b60 2024-01-27T02:41:47.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.234+0000 7f17d77fe700 1 -- 172.21.15.7:0/3185778859 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e800add0 con 0x7f17f0156b60 2024-01-27T02:41:47.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.234+0000 7f17f5c99700 1 -- 172.21.15.7:0/3185778859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f00a98e0 con 0x7f17f0156b60 2024-01-27T02:41:47.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.235+0000 7f17f5c99700 1 -- 172.21.15.7:0/3185778859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f00a9f10 con 0x7f17f0156b60 2024-01-27T02:41:47.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.236+0000 7f17d77fe700 1 -- 172.21.15.7:0/3185778859 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f17e8007300 con 0x7f17f0156b60 2024-01-27T02:41:47.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.236+0000 7f17d77fe700 1 --2- 172.21.15.7:0/3185778859 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f17dc066fb0 0x7f17dc069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:47.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.236+0000 7f17d77fe700 1 -- 172.21.15.7:0/3185778859 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f17e8084980 con 0x7f17f0156b60 2024-01-27T02:41:47.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.237+0000 7f17d77fe700 1 -- 172.21.15.7:0/3185778859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=155}) v3 -- 0x7f17dc06ec30 con 0x7f17f0156b60 2024-01-27T02:41:47.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.237+0000 7f17ef7fe700 1 --2- 172.21.15.7:0/3185778859 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f17dc066fb0 0x7f17dc069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:47.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.237+0000 7f17ef7fe700 1 --2- 172.21.15.7:0/3185778859 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f17dc066fb0 0x7f17dc069460 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f17e00091e0 tx=0x7f17e0017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:47.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.241+0000 7f17d77fe700 1 -- 172.21.15.7:0/3185778859 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17e8051b40 con 0x7f17f0156b60 2024-01-27T02:41:47.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:47.487+0000 7f17f5c99700 1 -- 172.21.15.7:0/3185778859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f17f0072060 con 0x7f17f0156b60 2024-01-27T02:41:48.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.076+0000 7f17d77fe700 1 -- 172.21.15.7:0/3185778859 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v79) v1 ==== 104+0+0 (secure 0 0 0) 0x7f17e8055160 con 0x7f17f0156b60 2024-01-27T02:41:48.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.079+0000 7f17f5c99700 1 -- 172.21.15.7:0/3185778859 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f17dc066fb0 msgr2=0x7f17dc069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:48.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.079+0000 7f17f5c99700 1 --2- 172.21.15.7:0/3185778859 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f17dc066fb0 0x7f17dc069460 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f17e00091e0 tx=0x7f17e0017000 comp rx=0 tx=0).stop 2024-01-27T02:41:48.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.079+0000 7f17f5c99700 1 -- 172.21.15.7:0/3185778859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f0156b60 msgr2=0x7f17f01680a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:48.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.079+0000 7f17f5c99700 1 --2- 172.21.15.7:0/3185778859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f0156b60 0x7f17f01680a0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f17e800cc90 tx=0x7f17e8001a80 comp rx=0 tx=0).stop 2024-01-27T02:41:48.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.079+0000 7f17f5c99700 1 -- 172.21.15.7:0/3185778859 shutdown_connections 2024-01-27T02:41:48.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.079+0000 7f17f5c99700 1 --2- 172.21.15.7:0/3185778859 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f17dc066fb0 0x7f17dc069460 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:48.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.079+0000 7f17f5c99700 1 --2- 172.21.15.7:0/3185778859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17f0156b60 0x7f17f01680a0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:48.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.080+0000 7f17f5c99700 1 --2- 172.21.15.7:0/3185778859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17f00ad9f0 0x7f17f00ab5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:48.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.080+0000 7f17f5c99700 1 --2- 172.21.15.7:0/3185778859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17f00af470 0x7f17f00a8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:48.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.080+0000 7f17f5c99700 1 -- 172.21.15.7:0/3185778859 >> 172.21.15.7:0/3185778859 conn(0x7f17f0068480 msgr2=0x7f17f00b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:48.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.081+0000 7f17f5c99700 1 -- 172.21.15.7:0/3185778859 shutdown_connections 2024-01-27T02:41:48.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.081+0000 7f17f5c99700 1 -- 172.21.15.7:0/3185778859 wait complete. 2024-01-27T02:41:48.097 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-27T02:41:48.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.284+0000 7fded6fce700 1 Processor -- start 2024-01-27T02:41:48.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.284+0000 7fded6fce700 1 -- start start 2024-01-27T02:41:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.285+0000 7fded6fce700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fded015a0b0 0x7fded00ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.285+0000 7fded6fce700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fded00ac7c0 0x7fded00aaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.285+0000 7fded6fce700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded00acca0 con 0x7fded00ab000 2024-01-27T02:41:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.285+0000 7fded6fce700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded00acf80 con 0x7fded015a0b0 2024-01-27T02:41:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.285+0000 7fded6fce700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded00cdc70 con 0x7fded00ac7c0 2024-01-27T02:41:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.285+0000 7fded556b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fded00ab000 0x7fded0063580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52520/0 (socket says 172.21.15.7:52520) 2024-01-27T02:41:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.285+0000 7fded556b700 1 -- 172.21.15.7:0/3393189244 learned_addr learned my addr 172.21.15.7:0/3393189244 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:41:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.285+0000 7fded4d6a700 1 --2- 172.21.15.7:0/3393189244 >> v2:172.21.15.188:3300/0 conn(0x7fded015a0b0 0x7fded00ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.285+0000 7fdecffff700 1 --2- 172.21.15.7:0/3393189244 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fded00ac7c0 0x7fded00aaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.285+0000 7fdecf7fe700 1 -- 172.21.15.7:0/3393189244 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2275884345 0 0) 0x7fded00acca0 con 0x7fded00ab000 2024-01-27T02:41:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.285+0000 7fdecf7fe700 1 -- 172.21.15.7:0/3393189244 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdec0003570 con 0x7fded00ab000 2024-01-27T02:41:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.285+0000 7fdecf7fe700 1 -- 172.21.15.7:0/3393189244 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1023137802 0 0) 0x7fdec0003570 con 0x7fded00ab000 2024-01-27T02:41:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.285+0000 7fdecf7fe700 1 -- 172.21.15.7:0/3393189244 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fded00ac7c0 msgr2=0x7fded00aaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.285+0000 7fdecf7fe700 1 --2- 172.21.15.7:0/3393189244 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fded00ac7c0 0x7fded00aaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.286+0000 7fdecf7fe700 1 -- 172.21.15.7:0/3393189244 >> v2:172.21.15.188:3300/0 conn(0x7fded015a0b0 msgr2=0x7fded00ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.286+0000 7fdecf7fe700 1 --2- 172.21.15.7:0/3393189244 >> v2:172.21.15.188:3300/0 conn(0x7fded015a0b0 0x7fded00ce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.286+0000 7fdecf7fe700 1 -- 172.21.15.7:0/3393189244 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded00cdf50 con 0x7fded00ab000 2024-01-27T02:41:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.286+0000 7fdecf7fe700 1 -- 172.21.15.7:0/3393189244 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7fdec80017f0 con 0x7fded00ab000 2024-01-27T02:41:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.286+0000 7fdecf7fe700 1 -- 172.21.15.7:0/3393189244 >> v1:172.21.15.7:6789/0 conn(0x7fded00ab000 legacy=0x7fded0063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.286+0000 7fdecf7fe700 1 --2- 172.21.15.7:0/3393189244 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fded00ac7c0 0x7fdec0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.286+0000 7fdecf7fe700 1 --2- 172.21.15.7:0/3393189244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdec0003fe0 0x7fdec0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.286+0000 7fdecf7fe700 1 --2- 172.21.15.7:0/3393189244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdec0004d50 0x7fdec00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.286+0000 7fdecf7fe700 1 -- 172.21.15.7:0/3393189244 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded00acf80 con 0x7fdec0003fe0 2024-01-27T02:41:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.286+0000 7fdecf7fe700 1 -- 172.21.15.7:0/3393189244 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded00cdc70 con 0x7fdec0004d50 2024-01-27T02:41:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.286+0000 7fdecf7fe700 1 -- 172.21.15.7:0/3393189244 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec000ba90 con 0x7fded00ac7c0 2024-01-27T02:41:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.286+0000 7fded4d6a700 1 --2- 172.21.15.7:0/3393189244 >> v2:172.21.15.188:3300/0 conn(0x7fded015a0b0 0x7fded00ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:41:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.286+0000 7fdecffff700 1 --2- 172.21.15.7:0/3393189244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdec0003fe0 0x7fdec0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.286+0000 7fded556b700 1 --2- 172.21.15.7:0/3393189244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdec0004d50 0x7fdec00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.286+0000 7fded4d6a700 1 --2- 172.21.15.7:0/3393189244 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fded00ac7c0 0x7fdec0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.286+0000 7fdecffff700 1 -- 172.21.15.7:0/3393189244 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fded00ac7c0 msgr2=0x7fdec0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.286+0000 7fdecffff700 1 --2- 172.21.15.7:0/3393189244 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fded00ac7c0 0x7fdec0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.286+0000 7fdecffff700 1 -- 172.21.15.7:0/3393189244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdec0004d50 msgr2=0x7fdec00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:48.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.286+0000 7fdecffff700 1 --2- 172.21.15.7:0/3393189244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdec0004d50 0x7fdec00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:48.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.286+0000 7fdecffff700 1 -- 172.21.15.7:0/3393189244 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec0003570 con 0x7fdec0003fe0 2024-01-27T02:41:48.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.286+0000 7fdecffff700 1 --2- 172.21.15.7:0/3393189244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdec0003fe0 0x7fdec0004810 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fded00acbd0 tx=0x7fdebc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef8a62190e481c9 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:48.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.286+0000 7fdecf7fe700 1 -- 172.21.15.7:0/3393189244 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc00e6b0 con 0x7fdec0003fe0 2024-01-27T02:41:48.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.287+0000 7fdecf7fe700 1 -- 172.21.15.7:0/3393189244 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdebc00e810 con 0x7fdec0003fe0 2024-01-27T02:41:48.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.287+0000 7fdecf7fe700 1 -- 172.21.15.7:0/3393189244 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc00ebd0 con 0x7fdec0003fe0 2024-01-27T02:41:48.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.287+0000 7fded6fce700 1 -- 172.21.15.7:0/3393189244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdec0003fe0 msgr2=0x7fdec0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:48.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.287+0000 7fded6fce700 1 --2- 172.21.15.7:0/3393189244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdec0003fe0 0x7fdec0004810 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fded00acbd0 tx=0x7fdebc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:41:48.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.287+0000 7fded556b700 1 -- 172.21.15.7:0/3393189244 reap_dead start 2024-01-27T02:41:48.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.287+0000 7fded6fce700 1 -- 172.21.15.7:0/3393189244 shutdown_connections 2024-01-27T02:41:48.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.287+0000 7fded6fce700 1 -- 172.21.15.7:0/3393189244 >> 172.21.15.7:0/3393189244 conn(0x7fded0068480 msgr2=0x7fded006be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:48.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.287+0000 7fded6fce700 1 -- 172.21.15.7:0/3393189244 shutdown_connections 2024-01-27T02:41:48.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.287+0000 7fded6fce700 1 -- 172.21.15.7:0/3393189244 wait complete. 2024-01-27T02:41:48.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.288+0000 7fded6fce700 1 Processor -- start 2024-01-27T02:41:48.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.289+0000 7fded6fce700 1 -- start start 2024-01-27T02:41:48.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.289+0000 7fded6fce700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdec0003fe0 0x7fded01445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:48.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.289+0000 7fded6fce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fded00cbfd0 0x7fded00cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:48.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.289+0000 7fded6fce700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fded015c3a0 0x7fded015c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:48.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.289+0000 7fded6fce700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded00cc980 con 0x7fded00cbfd0 2024-01-27T02:41:48.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.289+0000 7fded6fce700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded016b300 con 0x7fded015c3a0 2024-01-27T02:41:48.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.289+0000 7fded6fce700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded016a9d0 con 0x7fdec0003fe0 2024-01-27T02:41:48.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.290+0000 7fdecffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fded00cbfd0 0x7fded00cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:48.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.290+0000 7fdecffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fded00cbfd0 0x7fded00cc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40928/0 (socket says 172.21.15.7:40928) 2024-01-27T02:41:48.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.290+0000 7fdecffff700 1 -- 172.21.15.7:0/2027794258 learned_addr learned my addr 172.21.15.7:0/2027794258 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:48.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.290+0000 7fded4d6a700 1 --2- 172.21.15.7:0/2027794258 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdec0003fe0 0x7fded01445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:48.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.290+0000 7fded556b700 1 --2- 172.21.15.7:0/2027794258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fded015c3a0 0x7fded015c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:48.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.290+0000 7fdecffff700 1 -- 172.21.15.7:0/2027794258 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdec0003fe0 msgr2=0x7fded01445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:48.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.290+0000 7fdecffff700 1 --2- 172.21.15.7:0/2027794258 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdec0003fe0 0x7fded01445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:48.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.290+0000 7fdecffff700 1 -- 172.21.15.7:0/2027794258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fded015c3a0 msgr2=0x7fded015c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:48.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.290+0000 7fdecffff700 1 --2- 172.21.15.7:0/2027794258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fded015c3a0 0x7fded015c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:48.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.290+0000 7fdecffff700 1 -- 172.21.15.7:0/2027794258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded016acb0 con 0x7fded00cbfd0 2024-01-27T02:41:48.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.290+0000 7fdecffff700 1 --2- 172.21.15.7:0/2027794258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fded00cbfd0 0x7fded00cc440 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fdec0004730 tx=0x7fdebc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:48.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.290+0000 7fdeccff9700 1 -- 172.21.15.7:0/2027794258 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc022070 con 0x7fded00cbfd0 2024-01-27T02:41:48.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.290+0000 7fdeccff9700 1 -- 172.21.15.7:0/2027794258 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdebc00e980 con 0x7fded00cbfd0 2024-01-27T02:41:48.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.290+0000 7fded6fce700 1 -- 172.21.15.7:0/2027794258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded016af90 con 0x7fded00cbfd0 2024-01-27T02:41:48.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.290+0000 7fdeccff9700 1 -- 172.21.15.7:0/2027794258 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc00e6b0 con 0x7fded00cbfd0 2024-01-27T02:41:48.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.290+0000 7fded6fce700 1 -- 172.21.15.7:0/2027794258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded016a670 con 0x7fded00cbfd0 2024-01-27T02:41:48.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.292+0000 7fded6fce700 1 -- 172.21.15.7:0/2027794258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded00cc980 con 0x7fded00cbfd0 2024-01-27T02:41:48.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.292+0000 7fdeccff9700 1 -- 172.21.15.7:0/2027794258 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fdebc027680 con 0x7fded00cbfd0 2024-01-27T02:41:48.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.292+0000 7fdeccff9700 1 --2- 172.21.15.7:0/2027794258 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdeb4066f30 0x7fdeb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:48.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.292+0000 7fdeccff9700 1 -- 172.21.15.7:0/2027794258 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5849+0+0 (secure 0 0 0) 0x7fdebc085f50 con 0x7fded00cbfd0 2024-01-27T02:41:48.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.292+0000 7fded4d6a700 1 --2- 172.21.15.7:0/2027794258 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdeb4066f30 0x7fdeb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:48.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.292+0000 7fdeccff9700 1 -- 172.21.15.7:0/2027794258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=155}) v3 -- 0x7fdeb406ebb0 con 0x7fded00cbfd0 2024-01-27T02:41:48.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.293+0000 7fded4d6a700 1 --2- 172.21.15.7:0/2027794258 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdeb4066f30 0x7fdeb40693e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdec4006f00 tx=0x7fdec40091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:48.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.297+0000 7fdeccff9700 1 -- 172.21.15.7:0/2027794258 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdebc0531e0 con 0x7fded00cbfd0 2024-01-27T02:41:48.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:48.501+0000 7fded6fce700 1 -- 172.21.15.7:0/2027794258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fded004c710 con 0x7fded00cbfd0 2024-01-27T02:41:49.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.176+0000 7fdeccff9700 1 -- 172.21.15.7:0/2027794258 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(155..155 src has 1..155) v4 ==== 288+0+0 (secure 0 0 0) 0x7fdebc056ea0 con 0x7fded00cbfd0 2024-01-27T02:41:49.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.176+0000 7fdeccff9700 1 -- 172.21.15.7:0/2027794258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7fdeb406cfe0 con 0x7fded00cbfd0 2024-01-27T02:41:49.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.178+0000 7fdeccff9700 1 -- 172.21.15.7:0/2027794258 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v155) v1 ==== 181+0+0 (secure 0 0 0) 0x7fdebc02c030 con 0x7fded00cbfd0 2024-01-27T02:41:49.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.188+0000 7fded6fce700 1 -- 172.21.15.7:0/2027794258 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdeb4066f30 msgr2=0x7fdeb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:49.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.188+0000 7fded6fce700 1 --2- 172.21.15.7:0/2027794258 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdeb4066f30 0x7fdeb40693e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdec4006f00 tx=0x7fdec40091a0 comp rx=0 tx=0).stop 2024-01-27T02:41:49.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.188+0000 7fded6fce700 1 -- 172.21.15.7:0/2027794258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fded00cbfd0 msgr2=0x7fded00cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:49.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.188+0000 7fded6fce700 1 --2- 172.21.15.7:0/2027794258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fded00cbfd0 0x7fded00cc440 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fdec0004730 tx=0x7fdebc01a000 comp rx=0 tx=0).stop 2024-01-27T02:41:49.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.188+0000 7fded6fce700 1 -- 172.21.15.7:0/2027794258 shutdown_connections 2024-01-27T02:41:49.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.188+0000 7fded6fce700 1 --2- 172.21.15.7:0/2027794258 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdeb4066f30 0x7fdeb40693e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:49.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.188+0000 7fded6fce700 1 --2- 172.21.15.7:0/2027794258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fded015c3a0 0x7fded015c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:49.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.188+0000 7fded6fce700 1 --2- 172.21.15.7:0/2027794258 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdec0003fe0 0x7fded01445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:49.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.188+0000 7fded6fce700 1 --2- 172.21.15.7:0/2027794258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fded00cbfd0 0x7fded00cc440 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:49.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.188+0000 7fded6fce700 1 -- 172.21.15.7:0/2027794258 >> 172.21.15.7:0/2027794258 conn(0x7fded0068480 msgr2=0x7fded00ad580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:49.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.189+0000 7fded6fce700 1 -- 172.21.15.7:0/2027794258 shutdown_connections 2024-01-27T02:41:49.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.189+0000 7fded6fce700 1 -- 172.21.15.7:0/2027794258 wait complete. 2024-01-27T02:41:49.191 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-27T02:41:49.205 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-27T02:41:49.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.345+0000 7f483bcbf700 1 Processor -- start 2024-01-27T02:41:49.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.345+0000 7f483bcbf700 1 -- start start 2024-01-27T02:41:49.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.346+0000 7f483bcbf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f483406f190 0x7f4834063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:49.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.346+0000 7f483bcbf700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4834063f20 con 0x7f4834063ac0 2024-01-27T02:41:49.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.346+0000 7f483bcbf700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48340b14e0 con 0x7f4834147f30 2024-01-27T02:41:49.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.346+0000 7f483bcbf700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48340b2260 con 0x7f483406f190 2024-01-27T02:41:49.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.346+0000 7f483925a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f483406f190 0x7f4834063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:49.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.346+0000 7f483a25c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4834063ac0 0x7f483415eae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52526/0 (socket says 172.21.15.7:52526) 2024-01-27T02:41:49.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.346+0000 7f483a25c700 1 -- 172.21.15.7:0/2266853457 learned_addr learned my addr 172.21.15.7:0/2266853457 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:41:49.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.347+0000 7f483925a700 1 -- 172.21.15.7:0/2266853457 >> v1:172.21.15.188:6789/0 conn(0x7f4834147f30 legacy=0x7f48340b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:49.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.347+0000 7f483925a700 1 -- 172.21.15.7:0/2266853457 >> v1:172.21.15.7:6789/0 conn(0x7f4834063ac0 legacy=0x7f483415eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:49.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.347+0000 7f483925a700 1 -- 172.21.15.7:0/2266853457 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483406ed00 con 0x7f483406f190 2024-01-27T02:41:49.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.347+0000 7f4838a59700 1 -- 172.21.15.7:0/2266853457 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4280217221 0 0) 0x7f482c001490 con 0x7f4834063ac0 2024-01-27T02:41:49.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.347+0000 7f483925a700 1 --2- 172.21.15.7:0/2266853457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f483406f190 0x7f4834063580 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f48240098e0 tx=0x7f48240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb2edee5b7a3188 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:49.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.347+0000 7f4838a59700 1 -- 172.21.15.7:0/2266853457 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824009e80 con 0x7f483406f190 2024-01-27T02:41:49.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.347+0000 7f4838a59700 1 -- 172.21.15.7:0/2266853457 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482400b820 con 0x7f483406f190 2024-01-27T02:41:49.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.347+0000 7f4838a59700 1 -- 172.21.15.7:0/2266853457 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482400bba0 con 0x7f483406f190 2024-01-27T02:41:49.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.348+0000 7f483bcbf700 1 -- 172.21.15.7:0/2266853457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f483406f190 msgr2=0x7f4834063580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:49.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.348+0000 7f483bcbf700 1 --2- 172.21.15.7:0/2266853457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f483406f190 0x7f4834063580 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f48240098e0 tx=0x7f48240091a0 comp rx=0 tx=0).stop 2024-01-27T02:41:49.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.348+0000 7f483bcbf700 1 -- 172.21.15.7:0/2266853457 shutdown_connections 2024-01-27T02:41:49.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.348+0000 7f483bcbf700 1 --2- 172.21.15.7:0/2266853457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f483406f190 0x7f4834063580 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:49.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.348+0000 7f483bcbf700 1 -- 172.21.15.7:0/2266853457 >> 172.21.15.7:0/2266853457 conn(0x7f4834068480 msgr2=0x7f483406be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:49.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.348+0000 7f483bcbf700 1 -- 172.21.15.7:0/2266853457 shutdown_connections 2024-01-27T02:41:49.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.348+0000 7f483bcbf700 1 -- 172.21.15.7:0/2266853457 wait complete. 2024-01-27T02:41:49.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.349+0000 7f483bcbf700 1 Processor -- start 2024-01-27T02:41:49.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.349+0000 7f483bcbf700 1 -- start start 2024-01-27T02:41:49.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.350+0000 7f483bcbf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4834063ac0 0x7f483416c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:49.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.350+0000 7f483bcbf700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f483406f190 0x7f483416f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:49.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.350+0000 7f483bcbf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4834147f30 0x7f483416df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:49.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.350+0000 7f483bcbf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48340a6780 con 0x7f4834063ac0 2024-01-27T02:41:49.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.350+0000 7f483bcbf700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483416fe40 con 0x7f4834147f30 2024-01-27T02:41:49.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.350+0000 7f483bcbf700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4834170120 con 0x7f483406f190 2024-01-27T02:41:49.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.350+0000 7f4839a5b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4834063ac0 0x7f483416c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:49.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.350+0000 7f4839a5b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4834063ac0 0x7f483416c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40934/0 (socket says 172.21.15.7:40934) 2024-01-27T02:41:49.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.350+0000 7f4839a5b700 1 -- 172.21.15.7:0/3078957600 learned_addr learned my addr 172.21.15.7:0/3078957600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:49.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.350+0000 7f483925a700 1 --2- 172.21.15.7:0/3078957600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f483406f190 0x7f483416f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.350+0000 7f483a25c700 1 --2- 172.21.15.7:0/3078957600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4834147f30 0x7f483416df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.350+0000 7f4839a5b700 1 -- 172.21.15.7:0/3078957600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f483406f190 msgr2=0x7f483416f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.350+0000 7f4839a5b700 1 --2- 172.21.15.7:0/3078957600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f483406f190 0x7f483416f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.350+0000 7f4839a5b700 1 -- 172.21.15.7:0/3078957600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4834147f30 msgr2=0x7f483416df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.350+0000 7f4839a5b700 1 --2- 172.21.15.7:0/3078957600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4834147f30 0x7f483416df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.350+0000 7f4839a5b700 1 -- 172.21.15.7:0/3078957600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4824009420 con 0x7f4834063ac0 2024-01-27T02:41:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.350+0000 7f483925a700 1 --2- 172.21.15.7:0/3078957600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f483406f190 0x7f483416f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:41:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.351+0000 7f4839a5b700 1 --2- 172.21.15.7:0/3078957600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4834063ac0 0x7f483416c420 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f483000c5d0 tx=0x7f4830007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.351+0000 7f4829ffb700 1 -- 172.21.15.7:0/3078957600 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830011590 con 0x7f4834063ac0 2024-01-27T02:41:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.351+0000 7f4829ffb700 1 -- 172.21.15.7:0/3078957600 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4830011710 con 0x7f4834063ac0 2024-01-27T02:41:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.351+0000 7f483bcbf700 1 -- 172.21.15.7:0/3078957600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4834170400 con 0x7f4834063ac0 2024-01-27T02:41:49.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.351+0000 7f4829ffb700 1 -- 172.21.15.7:0/3078957600 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830011ad0 con 0x7f4834063ac0 2024-01-27T02:41:49.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.351+0000 7f483bcbf700 1 -- 172.21.15.7:0/3078957600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48341709c0 con 0x7f4834063ac0 2024-01-27T02:41:49.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.352+0000 7f483bcbf700 1 -- 172.21.15.7:0/3078957600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f483404c710 con 0x7f4834063ac0 2024-01-27T02:41:49.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.353+0000 7f4829ffb700 1 -- 172.21.15.7:0/3078957600 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f483001c460 con 0x7f4834063ac0 2024-01-27T02:41:49.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.353+0000 7f4829ffb700 1 --2- 172.21.15.7:0/3078957600 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4818066f30 0x7f48180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:49.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.353+0000 7f4829ffb700 1 -- 172.21.15.7:0/3078957600 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5411+0+0 (secure 0 0 0) 0x7f4830084210 con 0x7f4834063ac0 2024-01-27T02:41:49.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.353+0000 7f4829ffb700 1 -- 172.21.15.7:0/3078957600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7f481806ddb0 con 0x7f4834063ac0 2024-01-27T02:41:49.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.353+0000 7f483925a700 1 --2- 172.21.15.7:0/3078957600 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4818066f30 0x7f48180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:49.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.354+0000 7f483925a700 1 --2- 172.21.15.7:0/3078957600 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4818066f30 0x7f48180693e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f48240030c0 tx=0x7f4824016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:49.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.357+0000 7f4829ffb700 1 -- 172.21.15.7:0/3078957600 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48300515a0 con 0x7f4834063ac0 2024-01-27T02:41:49.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.562+0000 7f483bcbf700 1 -- 172.21.15.7:0/3078957600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f483416f750 con 0x7f4834063ac0 2024-01-27T02:41:49.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.910+0000 7f4829ffb700 1 -- 172.21.15.7:0/3078957600 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(156..156 src has 1..156) v4 ==== 713+0+0 (secure 0 0 0) 0x7f4830055960 con 0x7f4834063ac0 2024-01-27T02:41:49.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.910+0000 7f4829ffb700 1 -- 172.21.15.7:0/3078957600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7f481806cb40 con 0x7f4834063ac0 2024-01-27T02:41:49.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.913+0000 7f4829ffb700 1 -- 172.21.15.7:0/3078957600 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v156) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4830054bc0 con 0x7f4834063ac0 2024-01-27T02:41:49.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.921+0000 7f483bcbf700 1 -- 172.21.15.7:0/3078957600 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4818066f30 msgr2=0x7f48180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:49.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.921+0000 7f483bcbf700 1 --2- 172.21.15.7:0/3078957600 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4818066f30 0x7f48180693e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f48240030c0 tx=0x7f4824016000 comp rx=0 tx=0).stop 2024-01-27T02:41:49.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.921+0000 7f483bcbf700 1 -- 172.21.15.7:0/3078957600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4834063ac0 msgr2=0x7f483416c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:49.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.921+0000 7f483bcbf700 1 --2- 172.21.15.7:0/3078957600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4834063ac0 0x7f483416c420 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f483000c5d0 tx=0x7f4830007f60 comp rx=0 tx=0).stop 2024-01-27T02:41:49.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.922+0000 7f483bcbf700 1 -- 172.21.15.7:0/3078957600 shutdown_connections 2024-01-27T02:41:49.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.922+0000 7f483bcbf700 1 --2- 172.21.15.7:0/3078957600 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4818066f30 0x7f48180693e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:49.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.922+0000 7f483bcbf700 1 --2- 172.21.15.7:0/3078957600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4834147f30 0x7f483416df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:49.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.922+0000 7f483bcbf700 1 --2- 172.21.15.7:0/3078957600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4834063ac0 0x7f483416c420 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:49.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.922+0000 7f483bcbf700 1 --2- 172.21.15.7:0/3078957600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f483406f190 0x7f483416f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:49.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.922+0000 7f483bcbf700 1 -- 172.21.15.7:0/3078957600 >> 172.21.15.7:0/3078957600 conn(0x7f4834068480 msgr2=0x7f48341648f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.923+0000 7f483bcbf700 1 -- 172.21.15.7:0/3078957600 shutdown_connections 2024-01-27T02:41:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:49.923+0000 7f483bcbf700 1 -- 172.21.15.7:0/3078957600 wait complete. 2024-01-27T02:41:49.927 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-27T02:41:49.941 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-27T02:41:50.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.082+0000 7fe8f6a33700 1 Processor -- start 2024-01-27T02:41:50.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.082+0000 7fe8f6a33700 1 -- start start 2024-01-27T02:41:50.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.084+0000 7fe8f6a33700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8f0156b60 0x7fe8f00ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:50.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.084+0000 7fe8f6a33700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe8f00ad9f0 0x7fe8f00aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:50.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.084+0000 7fe8f6a33700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f00aeaa0 con 0x7fe8f00af470 2024-01-27T02:41:50.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.084+0000 7fe8f6a33700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8f00ade90 con 0x7fe8f00ad9f0 2024-01-27T02:41:50.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.084+0000 7fe8f6a33700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f00ae170 con 0x7fe8f0156b60 2024-01-27T02:41:50.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.084+0000 7fe8ef7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe8f00ad9f0 0x7fe8f00aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:50.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.084+0000 7fe8ef7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe8f00ad9f0 0x7fe8f00aef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40944/0 (socket says 172.21.15.7:40944) 2024-01-27T02:41:50.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.084+0000 7fe8ef7fe700 1 -- 172.21.15.7:0/2649897371 learned_addr learned my addr 172.21.15.7:0/2649897371 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:50.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.084+0000 7fe8effff700 1 --2- 172.21.15.7:0/2649897371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8f0156b60 0x7fe8f00ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:50.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.084+0000 7fe8ef7fe700 1 -- 172.21.15.7:0/2649897371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8f0156b60 msgr2=0x7fe8f00ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:50.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.084+0000 7fe8ef7fe700 1 --2- 172.21.15.7:0/2649897371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8f0156b60 0x7fe8f00ad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:50.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.084+0000 7fe8ef7fe700 1 -- 172.21.15.7:0/2649897371 >> v1:172.21.15.188:6789/0 conn(0x7fe8f00af470 legacy=0x7fe8f015a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.084+0000 7fe8ef7fe700 1 -- 172.21.15.7:0/2649897371 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f00ae450 con 0x7fe8f00ad9f0 2024-01-27T02:41:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.084+0000 7fe8ef7fe700 1 --2- 172.21.15.7:0/2649897371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8f00ad9f0 0x7fe8f00aef30 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fe8e40099b0 tx=0x7fe8e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ddfb07419fbaa6 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.085+0000 7fe8eeffd700 1 -- 172.21.15.7:0/2649897371 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4009ea0 con 0x7fe8f00ad9f0 2024-01-27T02:41:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.085+0000 7fe8eeffd700 1 -- 172.21.15.7:0/2649897371 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e400b860 con 0x7fe8f00ad9f0 2024-01-27T02:41:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.085+0000 7fe8eeffd700 1 -- 172.21.15.7:0/2649897371 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e400bc00 con 0x7fe8f00ad9f0 2024-01-27T02:41:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.085+0000 7fe8f6a33700 1 -- 172.21.15.7:0/2649897371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8f00ad9f0 msgr2=0x7fe8f00aef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.085+0000 7fe8f6a33700 1 --2- 172.21.15.7:0/2649897371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8f00ad9f0 0x7fe8f00aef30 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fe8e40099b0 tx=0x7fe8e4009250 comp rx=0 tx=0).stop 2024-01-27T02:41:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.085+0000 7fe8f6a33700 1 -- 172.21.15.7:0/2649897371 shutdown_connections 2024-01-27T02:41:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.085+0000 7fe8f6a33700 1 --2- 172.21.15.7:0/2649897371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8f0156b60 0x7fe8f00ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.085+0000 7fe8f6a33700 1 --2- 172.21.15.7:0/2649897371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8f00ad9f0 0x7fe8f00aef30 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.085+0000 7fe8f6a33700 1 -- 172.21.15.7:0/2649897371 >> 172.21.15.7:0/2649897371 conn(0x7fe8f0068480 msgr2=0x7fe8f00cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.085+0000 7fe8f6a33700 1 -- 172.21.15.7:0/2649897371 shutdown_connections 2024-01-27T02:41:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.085+0000 7fe8f6a33700 1 -- 172.21.15.7:0/2649897371 wait complete. 2024-01-27T02:41:50.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.087+0000 7fe8f6a33700 1 Processor -- start 2024-01-27T02:41:50.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.087+0000 7fe8f6a33700 1 -- start start 2024-01-27T02:41:50.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.088+0000 7fe8f6a33700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8f00ad9f0 0x7fe8f014fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:50.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.088+0000 7fe8f6a33700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8f00af470 0x7fe8f0168470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:50.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.088+0000 7fe8f6a33700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8f0156b60 0x7fe8f0159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:50.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.088+0000 7fe8f6a33700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f0157f00 con 0x7fe8f00af470 2024-01-27T02:41:50.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.088+0000 7fe8f6a33700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f01581e0 con 0x7fe8f00ad9f0 2024-01-27T02:41:50.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.088+0000 7fe8f6a33700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f01584c0 con 0x7fe8f0156b60 2024-01-27T02:41:50.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.088+0000 7fe8ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8f00af470 0x7fe8f0168470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:50.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.088+0000 7fe8ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8f00af470 0x7fe8f0168470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40956/0 (socket says 172.21.15.7:40956) 2024-01-27T02:41:50.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.088+0000 7fe8ef7fe700 1 -- 172.21.15.7:0/1044896734 learned_addr learned my addr 172.21.15.7:0/1044896734 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:50.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.089+0000 7fe8effff700 1 --2- 172.21.15.7:0/1044896734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8f00ad9f0 0x7fe8f014fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:50.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.089+0000 7fe8f4fd0700 1 --2- 172.21.15.7:0/1044896734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8f0156b60 0x7fe8f0159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:50.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.089+0000 7fe8ef7fe700 1 -- 172.21.15.7:0/1044896734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8f0156b60 msgr2=0x7fe8f0159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:50.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.089+0000 7fe8ef7fe700 1 --2- 172.21.15.7:0/1044896734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8f0156b60 0x7fe8f0159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:50.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.089+0000 7fe8ef7fe700 1 -- 172.21.15.7:0/1044896734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8f00ad9f0 msgr2=0x7fe8f014fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:50.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.089+0000 7fe8ef7fe700 1 --2- 172.21.15.7:0/1044896734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8f00ad9f0 0x7fe8f014fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:50.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.089+0000 7fe8ef7fe700 1 -- 172.21.15.7:0/1044896734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e40094f0 con 0x7fe8f00af470 2024-01-27T02:41:50.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.089+0000 7fe8ef7fe700 1 --2- 172.21.15.7:0/1044896734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8f00af470 0x7fe8f0168470 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00700c0 tx=0x7fe8e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:50.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.089+0000 7fe8d7fff700 1 -- 172.21.15.7:0/1044896734 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e400bbe0 con 0x7fe8f00af470 2024-01-27T02:41:50.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.089+0000 7fe8d7fff700 1 -- 172.21.15.7:0/1044896734 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e4011420 con 0x7fe8f00af470 2024-01-27T02:41:50.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.089+0000 7fe8f6a33700 1 -- 172.21.15.7:0/1044896734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f01587a0 con 0x7fe8f00af470 2024-01-27T02:41:50.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.089+0000 7fe8d7fff700 1 -- 172.21.15.7:0/1044896734 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e40117c0 con 0x7fe8f00af470 2024-01-27T02:41:50.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.089+0000 7fe8f6a33700 1 -- 172.21.15.7:0/1044896734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f01689b0 con 0x7fe8f00af470 2024-01-27T02:41:50.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.090+0000 7fe8f6a33700 1 -- 172.21.15.7:0/1044896734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f0158cc0 con 0x7fe8f00af470 2024-01-27T02:41:50.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.091+0000 7fe8d7fff700 1 -- 172.21.15.7:0/1044896734 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe8e4011bf0 con 0x7fe8f00af470 2024-01-27T02:41:50.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.091+0000 7fe8d7fff700 1 --2- 172.21.15.7:0/1044896734 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe8dc066f30 0x7fe8dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:50.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.091+0000 7fe8d7fff700 1 -- 172.21.15.7:0/1044896734 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fe8e4085b30 con 0x7fe8f00af470 2024-01-27T02:41:50.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.091+0000 7fe8effff700 1 --2- 172.21.15.7:0/1044896734 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe8dc066f30 0x7fe8dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:50.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.092+0000 7fe8effff700 1 --2- 172.21.15.7:0/1044896734 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe8dc066f30 0x7fe8dc0693e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009ca0 tx=0x7fe8e0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:50.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.096+0000 7fe8d7fff700 1 -- 172.21.15.7:0/1044896734 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8f0158cc0 con 0x7fe8f00af470 2024-01-27T02:41:50.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.301+0000 7fe8f6a33700 1 -- 172.21.15.7:0/1044896734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe8f0158cc0 con 0x7fe8f00af470 2024-01-27T02:41:50.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.304+0000 7fe8d7fff700 1 -- 172.21.15.7:0/1044896734 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe8f0158cc0 con 0x7fe8f00af470 2024-01-27T02:41:50.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.306+0000 7fe8f6a33700 1 -- 172.21.15.7:0/1044896734 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe8dc066f30 msgr2=0x7fe8dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:50.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.306+0000 7fe8f6a33700 1 --2- 172.21.15.7:0/1044896734 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe8dc066f30 0x7fe8dc0693e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009ca0 tx=0x7fe8e0008040 comp rx=0 tx=0).stop 2024-01-27T02:41:50.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.306+0000 7fe8f6a33700 1 -- 172.21.15.7:0/1044896734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8f00af470 msgr2=0x7fe8f0168470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:50.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.306+0000 7fe8f6a33700 1 --2- 172.21.15.7:0/1044896734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8f00af470 0x7fe8f0168470 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00700c0 tx=0x7fe8e4017000 comp rx=0 tx=0).stop 2024-01-27T02:41:50.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.306+0000 7fe8f6a33700 1 -- 172.21.15.7:0/1044896734 shutdown_connections 2024-01-27T02:41:50.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.306+0000 7fe8f6a33700 1 --2- 172.21.15.7:0/1044896734 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe8dc066f30 0x7fe8dc0693e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:50.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.306+0000 7fe8f6a33700 1 --2- 172.21.15.7:0/1044896734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8f0156b60 0x7fe8f0159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:50.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.306+0000 7fe8f6a33700 1 --2- 172.21.15.7:0/1044896734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8f00ad9f0 0x7fe8f014fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:50.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.306+0000 7fe8f6a33700 1 --2- 172.21.15.7:0/1044896734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8f00af470 0x7fe8f0168470 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:50.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.306+0000 7fe8f6a33700 1 -- 172.21.15.7:0/1044896734 >> 172.21.15.7:0/1044896734 conn(0x7fe8f0068480 msgr2=0x7fe8f00a7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:50.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.306+0000 7fe8f6a33700 1 -- 172.21.15.7:0/1044896734 shutdown_connections 2024-01-27T02:41:50.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.306+0000 7fe8f6a33700 1 -- 172.21.15.7:0/1044896734 wait complete. 2024-01-27T02:41:50.321 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2024-01-27T02:41:50.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.453+0000 7f40f1bd1700 1 Processor -- start 2024-01-27T02:41:50.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.453+0000 7f40f1bd1700 1 -- start start 2024-01-27T02:41:50.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.454+0000 7f40f1bd1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40ec063c90 0x7f40ec158590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:50.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.454+0000 7f40f1bd1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40ec06adc0 0x7f40ec158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:50.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.454+0000 7f40f1bd1700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40ec159570 con 0x7f40ec06a9f0 2024-01-27T02:41:50.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.454+0000 7f40f1bd1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ec159010 con 0x7f40ec063c90 2024-01-27T02:41:50.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.454+0000 7f40f1bd1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec159290 con 0x7f40ec06adc0 2024-01-27T02:41:50.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.454+0000 7f40eaffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40ec063c90 0x7f40ec158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:50.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.454+0000 7f40eaffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40ec063c90 0x7f40ec158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40972/0 (socket says 172.21.15.7:40972) 2024-01-27T02:41:50.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.454+0000 7f40eaffd700 1 -- 172.21.15.7:0/1190858570 learned_addr learned my addr 172.21.15.7:0/1190858570 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:50.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.454+0000 7f40eb7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f40ec06a9f0 0x7f40ec063580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52540/0 (socket says 172.21.15.7:52540) 2024-01-27T02:41:50.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.454+0000 7f40ebfff700 1 --2- 172.21.15.7:0/1190858570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40ec06adc0 0x7f40ec158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:50.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.454+0000 7f40eaffd700 1 -- 172.21.15.7:0/1190858570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40ec06adc0 msgr2=0x7f40ec158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:50.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.454+0000 7f40eaffd700 1 --2- 172.21.15.7:0/1190858570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40ec06adc0 0x7f40ec158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:50.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.454+0000 7f40eaffd700 1 -- 172.21.15.7:0/1190858570 >> v1:172.21.15.7:6789/0 conn(0x7f40ec06a9f0 legacy=0x7f40ec063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:50.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.454+0000 7f40eaffd700 1 -- 172.21.15.7:0/1190858570 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec15bd00 con 0x7f40ec063c90 2024-01-27T02:41:50.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.454+0000 7f40ea7fc700 1 -- 172.21.15.7:0/1190858570 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4092607761 0 0) 0x7f40dc001110 con 0x7f40ec06a9f0 2024-01-27T02:41:50.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.454+0000 7f40ebfff700 1 --2- 172.21.15.7:0/1190858570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40ec06adc0 0x7f40ec158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:41:50.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.455+0000 7f40eaffd700 1 --2- 172.21.15.7:0/1190858570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40ec063c90 0x7f40ec158590 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f40d40099b0 tx=0x7f40d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8439332b7de1de2 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:50.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.455+0000 7f40ea7fc700 1 -- 172.21.15.7:0/1190858570 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d400b860 con 0x7f40ec063c90 2024-01-27T02:41:50.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.455+0000 7f40ea7fc700 1 -- 172.21.15.7:0/1190858570 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d400b9c0 con 0x7f40ec063c90 2024-01-27T02:41:50.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.455+0000 7f40ea7fc700 1 -- 172.21.15.7:0/1190858570 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d400bd60 con 0x7f40ec063c90 2024-01-27T02:41:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.455+0000 7f40f1bd1700 1 -- 172.21.15.7:0/1190858570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40ec063c90 msgr2=0x7f40ec158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.455+0000 7f40f1bd1700 1 --2- 172.21.15.7:0/1190858570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40ec063c90 0x7f40ec158590 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f40d40099b0 tx=0x7f40d4009250 comp rx=0 tx=0).stop 2024-01-27T02:41:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.455+0000 7f40f1bd1700 1 -- 172.21.15.7:0/1190858570 shutdown_connections 2024-01-27T02:41:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.455+0000 7f40f1bd1700 1 --2- 172.21.15.7:0/1190858570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40ec06adc0 0x7f40ec158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.455+0000 7f40f1bd1700 1 --2- 172.21.15.7:0/1190858570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40ec063c90 0x7f40ec158590 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.455+0000 7f40f1bd1700 1 -- 172.21.15.7:0/1190858570 >> 172.21.15.7:0/1190858570 conn(0x7f40ec0cdc30 msgr2=0x7f40ec0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.455+0000 7f40f1bd1700 1 -- 172.21.15.7:0/1190858570 shutdown_connections 2024-01-27T02:41:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.455+0000 7f40f1bd1700 1 -- 172.21.15.7:0/1190858570 wait complete. 2024-01-27T02:41:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.457+0000 7f40f1bd1700 1 Processor -- start 2024-01-27T02:41:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.457+0000 7f40f1bd1700 1 -- start start 2024-01-27T02:41:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.458+0000 7f40f1bd1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40ec063c90 0x7f40ec16a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.458+0000 7f40f1bd1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40ec06a9f0 0x7f40ec0ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:50.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.458+0000 7f40f1bd1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40ec06adc0 0x7f40ec0cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:50.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.458+0000 7f40f1bd1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec0b3870 con 0x7f40ec06adc0 2024-01-27T02:41:50.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.458+0000 7f40f1bd1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec157530 con 0x7f40ec063c90 2024-01-27T02:41:50.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.458+0000 7f40f1bd1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec16a5d0 con 0x7f40ec06a9f0 2024-01-27T02:41:50.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.458+0000 7f40ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40ec06adc0 0x7f40ec0cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:50.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.458+0000 7f40ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40ec06adc0 0x7f40ec0cab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40976/0 (socket says 172.21.15.7:40976) 2024-01-27T02:41:50.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.458+0000 7f40ebfff700 1 -- 172.21.15.7:0/4214127185 learned_addr learned my addr 172.21.15.7:0/4214127185 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:50.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.458+0000 7f40eb7fe700 1 --2- 172.21.15.7:0/4214127185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40ec063c90 0x7f40ec16a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:50.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.458+0000 7f40eaffd700 1 --2- 172.21.15.7:0/4214127185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40ec06a9f0 0x7f40ec0ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:50.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.458+0000 7f40ebfff700 1 -- 172.21.15.7:0/4214127185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40ec06a9f0 msgr2=0x7f40ec0ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:50.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.458+0000 7f40ebfff700 1 --2- 172.21.15.7:0/4214127185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40ec06a9f0 0x7f40ec0ca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:50.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.458+0000 7f40ebfff700 1 -- 172.21.15.7:0/4214127185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40ec063c90 msgr2=0x7f40ec16a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:50.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.458+0000 7f40ebfff700 1 --2- 172.21.15.7:0/4214127185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40ec063c90 0x7f40ec16a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:50.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.458+0000 7f40ebfff700 1 -- 172.21.15.7:0/4214127185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d40094f0 con 0x7f40ec06adc0 2024-01-27T02:41:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.458+0000 7f40ebfff700 1 --2- 172.21.15.7:0/4214127185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40ec06adc0 0x7f40ec0cab00 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f40e400a380 tx=0x7f40e4007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.458+0000 7f40e37fe700 1 -- 172.21.15.7:0/4214127185 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4015440 con 0x7f40ec06adc0 2024-01-27T02:41:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.458+0000 7f40e37fe700 1 -- 172.21.15.7:0/4214127185 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e40155a0 con 0x7f40ec06adc0 2024-01-27T02:41:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.458+0000 7f40e37fe700 1 -- 172.21.15.7:0/4214127185 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4015960 con 0x7f40ec06adc0 2024-01-27T02:41:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.458+0000 7f40f1bd1700 1 -- 172.21.15.7:0/4214127185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ec0b32b0 con 0x7f40ec06adc0 2024-01-27T02:41:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.459+0000 7f40f1bd1700 1 -- 172.21.15.7:0/4214127185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ec1571c0 con 0x7f40ec06adc0 2024-01-27T02:41:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.459+0000 7f40f1bd1700 1 -- 172.21.15.7:0/4214127185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40ec157020 con 0x7f40ec06adc0 2024-01-27T02:41:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.460+0000 7f40e37fe700 1 -- 172.21.15.7:0/4214127185 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f40e400d440 con 0x7f40ec06adc0 2024-01-27T02:41:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.460+0000 7f40e37fe700 1 --2- 172.21.15.7:0/4214127185 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f40cc066f30 0x7f40cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.460+0000 7f40e37fe700 1 -- 172.21.15.7:0/4214127185 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f40e4084dc0 con 0x7f40ec06adc0 2024-01-27T02:41:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.461+0000 7f40eb7fe700 1 --2- 172.21.15.7:0/4214127185 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f40cc066f30 0x7f40cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.461+0000 7f40eb7fe700 1 --2- 172.21.15.7:0/4214127185 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f40cc066f30 0x7f40cc0693e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f40dc007fd0 tx=0x7f40dc007b90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:50.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.465+0000 7f40e37fe700 1 -- 172.21.15.7:0/4214127185 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40ec157020 con 0x7f40ec06adc0 2024-01-27T02:41:50.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:50.748+0000 7f40f1bd1700 1 -- 172.21.15.7:0/4214127185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) v1 -- 0x7f40ec157020 con 0x7f40ec06adc0 2024-01-27T02:41:51.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.189+0000 7f40e37fe700 1 -- 172.21.15.7:0/4214127185 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v466) v1 ==== 132+0+0 (secure 0 0 0) 0x7f40ec157020 con 0x7f40ec06adc0 2024-01-27T02:41:51.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.192+0000 7f40f1bd1700 1 -- 172.21.15.7:0/4214127185 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f40cc066f30 msgr2=0x7f40cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:51.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.192+0000 7f40f1bd1700 1 --2- 172.21.15.7:0/4214127185 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f40cc066f30 0x7f40cc0693e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f40dc007fd0 tx=0x7f40dc007b90 comp rx=0 tx=0).stop 2024-01-27T02:41:51.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.192+0000 7f40f1bd1700 1 -- 172.21.15.7:0/4214127185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40ec06adc0 msgr2=0x7f40ec0cab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:51.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.192+0000 7f40f1bd1700 1 --2- 172.21.15.7:0/4214127185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40ec06adc0 0x7f40ec0cab00 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f40e400a380 tx=0x7f40e4007900 comp rx=0 tx=0).stop 2024-01-27T02:41:51.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.192+0000 7f40f1bd1700 1 -- 172.21.15.7:0/4214127185 shutdown_connections 2024-01-27T02:41:51.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.192+0000 7f40f1bd1700 1 --2- 172.21.15.7:0/4214127185 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f40cc066f30 0x7f40cc0693e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:51.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.192+0000 7f40f1bd1700 1 --2- 172.21.15.7:0/4214127185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40ec06adc0 0x7f40ec0cab00 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:51.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.192+0000 7f40f1bd1700 1 --2- 172.21.15.7:0/4214127185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40ec063c90 0x7f40ec16a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:51.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.192+0000 7f40f1bd1700 1 --2- 172.21.15.7:0/4214127185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40ec06a9f0 0x7f40ec0ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:51.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.192+0000 7f40f1bd1700 1 -- 172.21.15.7:0/4214127185 >> 172.21.15.7:0/4214127185 conn(0x7f40ec0cdc30 msgr2=0x7f40ec0ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:51.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.192+0000 7f40f1bd1700 1 -- 172.21.15.7:0/4214127185 shutdown_connections 2024-01-27T02:41:51.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.192+0000 7f40f1bd1700 1 -- 172.21.15.7:0/4214127185 wait complete. 2024-01-27T02:41:51.208 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) ... ok 2024-01-27T02:41:51.209 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) That when the full flag is encountered during asynchronous 2024-01-27T02:41:51.209 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) 2024-01-27T02:41:51.209 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2024-01-27T02:41:51.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.344+0000 7f8a9501a700 1 Processor -- start 2024-01-27T02:41:51.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.344+0000 7f8a9501a700 1 -- start start 2024-01-27T02:41:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.345+0000 7f8a9501a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8a90159f40 0x7f8a900c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.345+0000 7f8a9501a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8a900c3a90 0x7f8a900aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.345+0000 7f8a9501a700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a900c30c0 con 0x7f8a9015a310 2024-01-27T02:41:51.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.345+0000 7f8a9501a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a900c4550 con 0x7f8a90159f40 2024-01-27T02:41:51.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.345+0000 7f8a9501a700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a9015e960 con 0x7f8a900c3a90 2024-01-27T02:41:51.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.345+0000 7f8a8ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8a90159f40 0x7f8a900c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:51.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.345+0000 7f8a8ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8a90159f40 0x7f8a900c3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40978/0 (socket says 172.21.15.7:40978) 2024-01-27T02:41:51.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.345+0000 7f8a8ed9d700 1 -- 172.21.15.7:0/1242258191 learned_addr learned my addr 172.21.15.7:0/1242258191 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:51.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.345+0000 7f8a8e59c700 1 --2- 172.21.15.7:0/1242258191 >> v2:172.21.15.188:3300/0 conn(0x7f8a900c3a90 0x7f8a900aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:51.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.346+0000 7f8a8ed9d700 1 -- 172.21.15.7:0/1242258191 >> v2:172.21.15.188:3300/0 conn(0x7f8a900c3a90 msgr2=0x7f8a900aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:51.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.346+0000 7f8a8ed9d700 1 --2- 172.21.15.7:0/1242258191 >> v2:172.21.15.188:3300/0 conn(0x7f8a900c3a90 0x7f8a900aa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:51.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.346+0000 7f8a8ed9d700 1 -- 172.21.15.7:0/1242258191 >> v1:172.21.15.188:6789/0 conn(0x7f8a9015a310 legacy=0x7f8a90063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:51.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.346+0000 7f8a8ed9d700 1 -- 172.21.15.7:0/1242258191 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9015ec40 con 0x7f8a90159f40 2024-01-27T02:41:51.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.346+0000 7f8a8ed9d700 1 --2- 172.21.15.7:0/1242258191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a90159f40 0x7f8a900c3550 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f8a90064660 tx=0x7f8a84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b2afe497ab8a0a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:51.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.346+0000 7f8a8dd9b700 1 -- 172.21.15.7:0/1242258191 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8400b860 con 0x7f8a90159f40 2024-01-27T02:41:51.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.346+0000 7f8a8dd9b700 1 -- 172.21.15.7:0/1242258191 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8400b9c0 con 0x7f8a90159f40 2024-01-27T02:41:51.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.346+0000 7f8a8dd9b700 1 -- 172.21.15.7:0/1242258191 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8400bd60 con 0x7f8a90159f40 2024-01-27T02:41:51.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.347+0000 7f8a9501a700 1 -- 172.21.15.7:0/1242258191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a90159f40 msgr2=0x7f8a900c3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:51.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.347+0000 7f8a9501a700 1 --2- 172.21.15.7:0/1242258191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a90159f40 0x7f8a900c3550 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f8a90064660 tx=0x7f8a84009250 comp rx=0 tx=0).stop 2024-01-27T02:41:51.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.347+0000 7f8a9501a700 1 -- 172.21.15.7:0/1242258191 shutdown_connections 2024-01-27T02:41:51.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.347+0000 7f8a9501a700 1 --2- 172.21.15.7:0/1242258191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a90159f40 0x7f8a900c3550 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:51.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.347+0000 7f8a9501a700 1 --2- 172.21.15.7:0/1242258191 >> v2:172.21.15.188:3300/0 conn(0x7f8a900c3a90 0x7f8a900aa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:51.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.347+0000 7f8a9501a700 1 -- 172.21.15.7:0/1242258191 >> 172.21.15.7:0/1242258191 conn(0x7f8a900cdc30 msgr2=0x7f8a900ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:51.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.347+0000 7f8a9501a700 1 -- 172.21.15.7:0/1242258191 shutdown_connections 2024-01-27T02:41:51.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.347+0000 7f8a9501a700 1 -- 172.21.15.7:0/1242258191 wait complete. 2024-01-27T02:41:51.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.348+0000 7f8a9501a700 1 Processor -- start 2024-01-27T02:41:51.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.348+0000 7f8a9501a700 1 -- start start 2024-01-27T02:41:51.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.349+0000 7f8a9501a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a900c3a90 0x7f8a90145040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:51.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.349+0000 7f8a9501a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a90159f40 0x7f8a900c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:51.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.349+0000 7f8a9501a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a9015a310 0x7f8a9016a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:51.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.349+0000 7f8a9501a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a900b39a0 con 0x7f8a900c3a90 2024-01-27T02:41:51.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.349+0000 7f8a9501a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a900cc180 con 0x7f8a9015a310 2024-01-27T02:41:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.349+0000 7f8a9501a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a90145580 con 0x7f8a90159f40 2024-01-27T02:41:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.349+0000 7f8a8ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a900c3a90 0x7f8a90145040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.349+0000 7f8a8ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a900c3a90 0x7f8a90145040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40992/0 (socket says 172.21.15.7:40992) 2024-01-27T02:41:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.349+0000 7f8a8ed9d700 1 -- 172.21.15.7:0/1386704868 learned_addr learned my addr 172.21.15.7:0/1386704868 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.349+0000 7f8a8f59e700 1 --2- 172.21.15.7:0/1386704868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a9015a310 0x7f8a9016a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.349+0000 7f8a8e59c700 1 --2- 172.21.15.7:0/1386704868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a90159f40 0x7f8a900c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.349+0000 7f8a8ed9d700 1 -- 172.21.15.7:0/1386704868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a90159f40 msgr2=0x7f8a900c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.349+0000 7f8a8ed9d700 1 --2- 172.21.15.7:0/1386704868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a90159f40 0x7f8a900c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.349+0000 7f8a8ed9d700 1 -- 172.21.15.7:0/1386704868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a9015a310 msgr2=0x7f8a9016a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.349+0000 7f8a8ed9d700 1 --2- 172.21.15.7:0/1386704868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a9015a310 0x7f8a9016a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.349+0000 7f8a8ed9d700 1 -- 172.21.15.7:0/1386704868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a840094f0 con 0x7f8a900c3a90 2024-01-27T02:41:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.350+0000 7f8a8ed9d700 1 --2- 172.21.15.7:0/1386704868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a900c3a90 0x7f8a90145040 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f8a9006dcc0 tx=0x7f8a84017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.350+0000 7f8a76ffd700 1 -- 172.21.15.7:0/1386704868 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a84011e60 con 0x7f8a900c3a90 2024-01-27T02:41:51.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.350+0000 7f8a76ffd700 1 -- 172.21.15.7:0/1386704868 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8401a630 con 0x7f8a900c3a90 2024-01-27T02:41:51.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.350+0000 7f8a9501a700 1 -- 172.21.15.7:0/1386704868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a900c7c30 con 0x7f8a900c3a90 2024-01-27T02:41:51.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.350+0000 7f8a76ffd700 1 -- 172.21.15.7:0/1386704868 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8401a9f0 con 0x7f8a900c3a90 2024-01-27T02:41:51.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.350+0000 7f8a9501a700 1 -- 172.21.15.7:0/1386704868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a900c7fc0 con 0x7f8a900c3a90 2024-01-27T02:41:51.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.351+0000 7f8a9501a700 1 -- 172.21.15.7:0/1386704868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a9004c710 con 0x7f8a900c3a90 2024-01-27T02:41:51.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.351+0000 7f8a76ffd700 1 -- 172.21.15.7:0/1386704868 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8a8402d440 con 0x7f8a900c3a90 2024-01-27T02:41:51.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.352+0000 7f8a76ffd700 1 --2- 172.21.15.7:0/1386704868 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8a7c066f30 0x7f8a7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:51.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.352+0000 7f8a76ffd700 1 -- 172.21.15.7:0/1386704868 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8a84085d60 con 0x7f8a900c3a90 2024-01-27T02:41:51.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.352+0000 7f8a8e59c700 1 --2- 172.21.15.7:0/1386704868 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8a7c066f30 0x7f8a7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:51.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.354+0000 7f8a8e59c700 1 --2- 172.21.15.7:0/1386704868 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8a7c066f30 0x7f8a7c0693e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8a9006a8d0 tx=0x7f8a80008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.356+0000 7f8a76ffd700 1 -- 172.21.15.7:0/1386704868 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a840532d0 con 0x7f8a900c3a90 2024-01-27T02:41:51.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:51.647+0000 7f8a9501a700 1 -- 172.21.15.7:0/1386704868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) v1 -- 0x7f8a9006f770 con 0x7f8a900c3a90 2024-01-27T02:41:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.193+0000 7f8a76ffd700 1 -- 172.21.15.7:0/1386704868 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v467) v1 ==== 134+0+0 (secure 0 0 0) 0x7f8a840568f0 con 0x7f8a900c3a90 2024-01-27T02:41:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.196+0000 7f8a9501a700 1 -- 172.21.15.7:0/1386704868 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8a7c066f30 msgr2=0x7f8a7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.196+0000 7f8a9501a700 1 --2- 172.21.15.7:0/1386704868 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8a7c066f30 0x7f8a7c0693e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8a9006a8d0 tx=0x7f8a80008040 comp rx=0 tx=0).stop 2024-01-27T02:41:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.196+0000 7f8a9501a700 1 -- 172.21.15.7:0/1386704868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a900c3a90 msgr2=0x7f8a90145040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.196+0000 7f8a9501a700 1 --2- 172.21.15.7:0/1386704868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a900c3a90 0x7f8a90145040 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f8a9006dcc0 tx=0x7f8a84017000 comp rx=0 tx=0).stop 2024-01-27T02:41:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.196+0000 7f8a9501a700 1 -- 172.21.15.7:0/1386704868 shutdown_connections 2024-01-27T02:41:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.196+0000 7f8a9501a700 1 --2- 172.21.15.7:0/1386704868 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8a7c066f30 0x7f8a7c0693e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.196+0000 7f8a9501a700 1 --2- 172.21.15.7:0/1386704868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a9015a310 0x7f8a9016a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.196+0000 7f8a9501a700 1 --2- 172.21.15.7:0/1386704868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a900c3a90 0x7f8a90145040 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.196+0000 7f8a9501a700 1 --2- 172.21.15.7:0/1386704868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a90159f40 0x7f8a900c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.196+0000 7f8a9501a700 1 -- 172.21.15.7:0/1386704868 >> 172.21.15.7:0/1386704868 conn(0x7f8a900cdc30 msgr2=0x7f8a901572e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.196+0000 7f8a9501a700 1 -- 172.21.15.7:0/1386704868 shutdown_connections 2024-01-27T02:41:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.196+0000 7f8a9501a700 1 -- 172.21.15.7:0/1386704868 wait complete. 2024-01-27T02:41:52.212 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-27T02:41:52.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.351+0000 7fab158e9700 1 Processor -- start 2024-01-27T02:41:52.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.351+0000 7fab158e9700 1 -- start start 2024-01-27T02:41:52.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.352+0000 7fab158e9700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fab1015d460 0x7fab100d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:52.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.352+0000 7fab158e9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab100cf280 0x7fab1015e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:52.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.352+0000 7fab158e9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab100b38d0 con 0x7fab1015d830 2024-01-27T02:41:52.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.352+0000 7fab158e9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab100d0390 con 0x7fab100cf280 2024-01-27T02:41:52.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.352+0000 7fab158e9700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab100cf780 con 0x7fab1015d460 2024-01-27T02:41:52.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.352+0000 7fab0e7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab100cf280 0x7fab1015e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:52.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.352+0000 7fab0f7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fab1015d830 0x7fab100cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39624/0 (socket says 172.21.15.7:39624) 2024-01-27T02:41:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.352+0000 7fab0f7fe700 1 -- 172.21.15.7:0/4097056971 learned_addr learned my addr 172.21.15.7:0/4097056971 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:41:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.352+0000 7fab0effd700 1 --2- 172.21.15.7:0/4097056971 >> v2:172.21.15.188:3300/0 conn(0x7fab1015d460 0x7fab100d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.352+0000 7fab0e7fc700 1 -- 172.21.15.7:0/4097056971 >> v2:172.21.15.188:3300/0 conn(0x7fab1015d460 msgr2=0x7fab100d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.352+0000 7fab0e7fc700 1 --2- 172.21.15.7:0/4097056971 >> v2:172.21.15.188:3300/0 conn(0x7fab1015d460 0x7fab100d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.352+0000 7fab0e7fc700 1 -- 172.21.15.7:0/4097056971 >> v1:172.21.15.7:6789/0 conn(0x7fab1015d830 legacy=0x7fab100cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.352+0000 7fab0dffb700 1 -- 172.21.15.7:0/4097056971 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857701528 0 0) 0x7fab08001490 con 0x7fab1015d830 2024-01-27T02:41:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.352+0000 7fab0e7fc700 1 -- 172.21.15.7:0/4097056971 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab100cfa60 con 0x7fab100cf280 2024-01-27T02:41:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.352+0000 7fab0effd700 1 --2- 172.21.15.7:0/4097056971 >> v2:172.21.15.188:3300/0 conn(0x7fab1015d460 0x7fab100d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:41:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.353+0000 7fab0e7fc700 1 --2- 172.21.15.7:0/4097056971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab100cf280 0x7fab1015e6b0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fab100ce640 tx=0x7fab040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa6404126f0225d server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.353+0000 7fab0dffb700 1 -- 172.21.15.7:0/4097056971 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0400b6b0 con 0x7fab100cf280 2024-01-27T02:41:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.353+0000 7fab0dffb700 1 -- 172.21.15.7:0/4097056971 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0400b810 con 0x7fab100cf280 2024-01-27T02:41:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.353+0000 7fab0dffb700 1 -- 172.21.15.7:0/4097056971 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0400bbb0 con 0x7fab100cf280 2024-01-27T02:41:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.353+0000 7fab158e9700 1 -- 172.21.15.7:0/4097056971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab100cf280 msgr2=0x7fab1015e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.353+0000 7fab158e9700 1 --2- 172.21.15.7:0/4097056971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab100cf280 0x7fab1015e6b0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fab100ce640 tx=0x7fab040091a0 comp rx=0 tx=0).stop 2024-01-27T02:41:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.353+0000 7fab158e9700 1 -- 172.21.15.7:0/4097056971 shutdown_connections 2024-01-27T02:41:52.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.353+0000 7fab158e9700 1 --2- 172.21.15.7:0/4097056971 >> v2:172.21.15.188:3300/0 conn(0x7fab1015d460 0x7fab100d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:52.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.353+0000 7fab158e9700 1 --2- 172.21.15.7:0/4097056971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab100cf280 0x7fab1015e6b0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:52.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.353+0000 7fab158e9700 1 -- 172.21.15.7:0/4097056971 >> 172.21.15.7:0/4097056971 conn(0x7fab10063580 msgr2=0x7fab10063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:52.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.353+0000 7fab158e9700 1 -- 172.21.15.7:0/4097056971 shutdown_connections 2024-01-27T02:41:52.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.354+0000 7fab158e9700 1 -- 172.21.15.7:0/4097056971 wait complete. 2024-01-27T02:41:52.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.355+0000 7fab158e9700 1 Processor -- start 2024-01-27T02:41:52.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.355+0000 7fab158e9700 1 -- start start 2024-01-27T02:41:52.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.356+0000 7fab158e9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab100cf280 0x7fab10151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:52.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.356+0000 7fab158e9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab1015d460 0x7fab100caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:52.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.356+0000 7fab158e9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab1015d830 0x7fab1015fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:52.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.356+0000 7fab158e9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1015ae30 con 0x7fab1015d830 2024-01-27T02:41:52.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.356+0000 7fab158e9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1015a220 con 0x7fab1015d460 2024-01-27T02:41:52.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.356+0000 7fab158e9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1015a500 con 0x7fab100cf280 2024-01-27T02:41:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.356+0000 7fab0f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab1015d830 0x7fab1015fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.356+0000 7fab0f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab1015d830 0x7fab1015fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42948/0 (socket says 172.21.15.7:42948) 2024-01-27T02:41:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.356+0000 7fab0f7fe700 1 -- 172.21.15.7:0/622302271 learned_addr learned my addr 172.21.15.7:0/622302271 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.356+0000 7fab0e7fc700 1 --2- 172.21.15.7:0/622302271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab1015d460 0x7fab100caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.356+0000 7fab0effd700 1 --2- 172.21.15.7:0/622302271 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab100cf280 0x7fab10151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.356+0000 7fab0f7fe700 1 -- 172.21.15.7:0/622302271 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab100cf280 msgr2=0x7fab10151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.356+0000 7fab0f7fe700 1 --2- 172.21.15.7:0/622302271 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab100cf280 0x7fab10151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.356+0000 7fab0f7fe700 1 -- 172.21.15.7:0/622302271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab1015d460 msgr2=0x7fab100caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.356+0000 7fab0f7fe700 1 --2- 172.21.15.7:0/622302271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab1015d460 0x7fab100caa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.356+0000 7fab0f7fe700 1 -- 172.21.15.7:0/622302271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab04009420 con 0x7fab1015d830 2024-01-27T02:41:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.356+0000 7fab0f7fe700 1 --2- 172.21.15.7:0/622302271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab1015d830 0x7fab1015fd20 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fab0800cbf0 tx=0x7fab080019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.357+0000 7faaf6ffd700 1 -- 172.21.15.7:0/622302271 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0800a8b0 con 0x7fab1015d830 2024-01-27T02:41:52.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.357+0000 7faaf6ffd700 1 -- 172.21.15.7:0/622302271 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0800aa10 con 0x7fab1015d830 2024-01-27T02:41:52.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.357+0000 7fab158e9700 1 -- 172.21.15.7:0/622302271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1015a7e0 con 0x7fab1015d830 2024-01-27T02:41:52.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.357+0000 7faaf6ffd700 1 -- 172.21.15.7:0/622302271 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0800add0 con 0x7fab1015d830 2024-01-27T02:41:52.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.357+0000 7fab158e9700 1 -- 172.21.15.7:0/622302271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab10152080 con 0x7fab1015d830 2024-01-27T02:41:52.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.358+0000 7fab158e9700 1 -- 172.21.15.7:0/622302271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab10072060 con 0x7fab1015d830 2024-01-27T02:41:52.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.358+0000 7faaf6ffd700 1 -- 172.21.15.7:0/622302271 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fab08007300 con 0x7fab1015d830 2024-01-27T02:41:52.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.358+0000 7faaf6ffd700 1 --2- 172.21.15.7:0/622302271 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faafc066f30 0x7faafc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:52.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.359+0000 7faaf6ffd700 1 -- 172.21.15.7:0/622302271 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fab08084380 con 0x7fab1015d830 2024-01-27T02:41:52.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.359+0000 7fab0effd700 1 --2- 172.21.15.7:0/622302271 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faafc066f30 0x7faafc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:52.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.359+0000 7fab0effd700 1 --2- 172.21.15.7:0/622302271 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faafc066f30 0x7faafc0693e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fab0000adc0 tx=0x7fab00006f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:52.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.363+0000 7faaf6ffd700 1 -- 172.21.15.7:0/622302271 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab08051970 con 0x7fab1015d830 2024-01-27T02:41:52.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.566+0000 7fab158e9700 1 -- 172.21.15.7:0/622302271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fab100b38d0 con 0x7fab1015d830 2024-01-27T02:41:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.570+0000 7faaf6ffd700 1 -- 172.21.15.7:0/622302271 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fab100b38d0 con 0x7fab1015d830 2024-01-27T02:41:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.572+0000 7fab158e9700 1 -- 172.21.15.7:0/622302271 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faafc066f30 msgr2=0x7faafc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.572+0000 7fab158e9700 1 --2- 172.21.15.7:0/622302271 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faafc066f30 0x7faafc0693e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fab0000adc0 tx=0x7fab00006f90 comp rx=0 tx=0).stop 2024-01-27T02:41:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.572+0000 7fab158e9700 1 -- 172.21.15.7:0/622302271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab1015d830 msgr2=0x7fab1015fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.572+0000 7fab158e9700 1 --2- 172.21.15.7:0/622302271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab1015d830 0x7fab1015fd20 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fab0800cbf0 tx=0x7fab080019b0 comp rx=0 tx=0).stop 2024-01-27T02:41:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.572+0000 7fab158e9700 1 -- 172.21.15.7:0/622302271 shutdown_connections 2024-01-27T02:41:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.572+0000 7fab158e9700 1 --2- 172.21.15.7:0/622302271 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faafc066f30 0x7faafc0693e0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.572+0000 7fab158e9700 1 --2- 172.21.15.7:0/622302271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab1015d830 0x7fab1015fd20 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.572+0000 7fab158e9700 1 --2- 172.21.15.7:0/622302271 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab100cf280 0x7fab10151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.572+0000 7fab158e9700 1 --2- 172.21.15.7:0/622302271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab1015d460 0x7fab100caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.572+0000 7fab158e9700 1 -- 172.21.15.7:0/622302271 >> 172.21.15.7:0/622302271 conn(0x7fab10063580 msgr2=0x7fab10152400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.572+0000 7fab158e9700 1 -- 172.21.15.7:0/622302271 shutdown_connections 2024-01-27T02:41:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.572+0000 7fab158e9700 1 -- 172.21.15.7:0/622302271 wait complete. 2024-01-27T02:41:52.586 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-27T02:41:52.587 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:41:52.628 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:41:52.673 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-27T02:41:52.673 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-27T02:41:52.673 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:41:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.808+0000 7f54122cc700 1 Processor -- start 2024-01-27T02:41:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.808+0000 7f54122cc700 1 -- start start 2024-01-27T02:41:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.809+0000 7f54122cc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f540c15a100 0x7f540c0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.809+0000 7f54122cc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f540c0cb5b0 0x7f540c0c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.809+0000 7f54122cc700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f540c0ce5e0 con 0x7f540c0c4af0 2024-01-27T02:41:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.809+0000 7f54122cc700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f540c0c4f70 con 0x7f540c0cb5b0 2024-01-27T02:41:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.809+0000 7f54122cc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540c0cd9d0 con 0x7f540c15a100 2024-01-27T02:41:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.809+0000 7f540b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f540c0cb5b0 0x7f540c0c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.809+0000 7f540b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f540c0cb5b0 0x7f540c0c45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42950/0 (socket says 172.21.15.7:42950) 2024-01-27T02:41:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.810+0000 7f540b7fe700 1 -- 172.21.15.7:0/1778975489 learned_addr learned my addr 172.21.15.7:0/1778975489 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.810+0000 7f540bfff700 1 --2- 172.21.15.7:0/1778975489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f540c15a100 0x7f540c0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.810+0000 7f540b7fe700 1 -- 172.21.15.7:0/1778975489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f540c15a100 msgr2=0x7f540c0cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.810+0000 7f540b7fe700 1 --2- 172.21.15.7:0/1778975489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f540c15a100 0x7f540c0cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.810+0000 7f540b7fe700 1 -- 172.21.15.7:0/1778975489 >> v1:172.21.15.188:6789/0 conn(0x7f540c0c4af0 legacy=0x7f540c0c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.810+0000 7f540b7fe700 1 -- 172.21.15.7:0/1778975489 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540c0cdcb0 con 0x7f540c0cb5b0 2024-01-27T02:41:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.810+0000 7f540b7fe700 1 --2- 172.21.15.7:0/1778975489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f540c0cb5b0 0x7f540c0c45b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f53f40099b0 tx=0x7f53f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9d7897583c1d07 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.810+0000 7f540affd700 1 -- 172.21.15.7:0/1778975489 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4009ea0 con 0x7f540c0cb5b0 2024-01-27T02:41:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.810+0000 7f540affd700 1 -- 172.21.15.7:0/1778975489 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f400b860 con 0x7f540c0cb5b0 2024-01-27T02:41:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.810+0000 7f540affd700 1 -- 172.21.15.7:0/1778975489 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f400bc00 con 0x7f540c0cb5b0 2024-01-27T02:41:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.811+0000 7f54122cc700 1 -- 172.21.15.7:0/1778975489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f540c0cb5b0 msgr2=0x7f540c0c45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.811+0000 7f54122cc700 1 --2- 172.21.15.7:0/1778975489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f540c0cb5b0 0x7f540c0c45b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f53f40099b0 tx=0x7f53f4009250 comp rx=0 tx=0).stop 2024-01-27T02:41:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.811+0000 7f54122cc700 1 -- 172.21.15.7:0/1778975489 shutdown_connections 2024-01-27T02:41:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.811+0000 7f54122cc700 1 --2- 172.21.15.7:0/1778975489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f540c15a100 0x7f540c0cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.811+0000 7f54122cc700 1 --2- 172.21.15.7:0/1778975489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f540c0cb5b0 0x7f540c0c45b0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.811+0000 7f54122cc700 1 -- 172.21.15.7:0/1778975489 >> 172.21.15.7:0/1778975489 conn(0x7f540c098020 msgr2=0x7f540c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.811+0000 7f54122cc700 1 -- 172.21.15.7:0/1778975489 shutdown_connections 2024-01-27T02:41:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.811+0000 7f54122cc700 1 -- 172.21.15.7:0/1778975489 wait complete. 2024-01-27T02:41:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.813+0000 7f54122cc700 1 Processor -- start 2024-01-27T02:41:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.813+0000 7f54122cc700 1 -- start start 2024-01-27T02:41:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.814+0000 7f54122cc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f540c0c4af0 0x7f540c156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.814+0000 7f54122cc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f540c0cb5b0 0x7f540c157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.814+0000 7f54122cc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f540c15a100 0x7f540c06c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.814+0000 7f54122cc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c1577d0 con 0x7f540c0cb5b0 2024-01-27T02:41:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.814+0000 7f54122cc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c156bc0 con 0x7f540c15a100 2024-01-27T02:41:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.814+0000 7f54122cc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540c156ea0 con 0x7f540c0c4af0 2024-01-27T02:41:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.814+0000 7f540b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f540c0cb5b0 0x7f540c157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.814+0000 7f540b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f540c0cb5b0 0x7f540c157c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42960/0 (socket says 172.21.15.7:42960) 2024-01-27T02:41:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.814+0000 7f540b7fe700 1 -- 172.21.15.7:0/2595946864 learned_addr learned my addr 172.21.15.7:0/2595946864 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.814+0000 7f540bfff700 1 --2- 172.21.15.7:0/2595946864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f540c0c4af0 0x7f540c156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.814+0000 7f5410869700 1 --2- 172.21.15.7:0/2595946864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f540c15a100 0x7f540c06c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.814+0000 7f540b7fe700 1 -- 172.21.15.7:0/2595946864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f540c0c4af0 msgr2=0x7f540c156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.814+0000 7f540b7fe700 1 --2- 172.21.15.7:0/2595946864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f540c0c4af0 0x7f540c156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.814+0000 7f540b7fe700 1 -- 172.21.15.7:0/2595946864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f540c15a100 msgr2=0x7f540c06c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.814+0000 7f540b7fe700 1 --2- 172.21.15.7:0/2595946864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f540c15a100 0x7f540c06c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.814+0000 7f540b7fe700 1 -- 172.21.15.7:0/2595946864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f40094f0 con 0x7f540c0cb5b0 2024-01-27T02:41:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.814+0000 7f540bfff700 1 --2- 172.21.15.7:0/2595946864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f540c0c4af0 0x7f540c156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:41:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.814+0000 7f5410869700 1 --2- 172.21.15.7:0/2595946864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f540c15a100 0x7f540c06c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:41:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.815+0000 7f540b7fe700 1 --2- 172.21.15.7:0/2595946864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f540c0cb5b0 0x7f540c157c60 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f540c06ea80 tx=0x7f53f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.815+0000 7f5403fff700 1 -- 172.21.15.7:0/2595946864 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f400b9d0 con 0x7f540c0cb5b0 2024-01-27T02:41:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.815+0000 7f5403fff700 1 -- 172.21.15.7:0/2595946864 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f400bb50 con 0x7f540c0cb5b0 2024-01-27T02:41:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.815+0000 7f5403fff700 1 -- 172.21.15.7:0/2595946864 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4025670 con 0x7f540c0cb5b0 2024-01-27T02:41:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.815+0000 7f54122cc700 1 -- 172.21.15.7:0/2595946864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f540c157180 con 0x7f540c0cb5b0 2024-01-27T02:41:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.815+0000 7f54122cc700 1 -- 172.21.15.7:0/2595946864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f540c1581a0 con 0x7f540c0cb5b0 2024-01-27T02:41:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.816+0000 7f54122cc700 1 -- 172.21.15.7:0/2595946864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f540c04c710 con 0x7f540c0cb5b0 2024-01-27T02:41:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.817+0000 7f5403fff700 1 -- 172.21.15.7:0/2595946864 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f53f4023070 con 0x7f540c0cb5b0 2024-01-27T02:41:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.817+0000 7f5403fff700 1 --2- 172.21.15.7:0/2595946864 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f53ec066f30 0x7f53ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.818+0000 7f5403fff700 1 -- 172.21.15.7:0/2595946864 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f53f4085bf0 con 0x7f540c0cb5b0 2024-01-27T02:41:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.818+0000 7f540bfff700 1 --2- 172.21.15.7:0/2595946864 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f53ec066f30 0x7f53ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.818+0000 7f540bfff700 1 --2- 172.21.15.7:0/2595946864 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f53ec066f30 0x7f53ec0693e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f53fc006f00 tx=0x7f53fc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:52.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:52.821+0000 7f5403fff700 1 -- 172.21.15.7:0/2595946864 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53f40531e0 con 0x7f540c0cb5b0 2024-01-27T02:41:53.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.071+0000 7f54122cc700 1 -- 172.21.15.7:0/2595946864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f540c157910 con 0x7f540c0cb5b0 2024-01-27T02:41:53.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.072+0000 7f5403fff700 1 -- 172.21.15.7:0/2595946864 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f53f402ade0 con 0x7f540c0cb5b0 2024-01-27T02:41:53.073 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:41:53.073 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":79,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6260,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3240780787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3240780787},{"type":"v1","addr":"172.21.15.188:6837","nonce":3240780787}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6437,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3802870138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3802870138},{"type":"v1","addr":"172.21.15.7:6835","nonce":3802870138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":7156,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/4051709811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4051709811},{"type":"v1","addr":"172.21.15.188:6836","nonce":4051709811}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7416,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/4000944335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":4000944335},{"type":"v1","addr":"172.21.15.7:6834","nonce":4000944335}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[]} 2024-01-27T02:41:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.074+0000 7f54122cc700 1 -- 172.21.15.7:0/2595946864 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f53ec066f30 msgr2=0x7f53ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.074+0000 7f54122cc700 1 --2- 172.21.15.7:0/2595946864 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f53ec066f30 0x7f53ec0693e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f53fc006f00 tx=0x7f53fc008040 comp rx=0 tx=0).stop 2024-01-27T02:41:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.074+0000 7f54122cc700 1 -- 172.21.15.7:0/2595946864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f540c0cb5b0 msgr2=0x7f540c157c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.074+0000 7f54122cc700 1 --2- 172.21.15.7:0/2595946864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f540c0cb5b0 0x7f540c157c60 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f540c06ea80 tx=0x7f53f4017000 comp rx=0 tx=0).stop 2024-01-27T02:41:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.074+0000 7f54122cc700 1 -- 172.21.15.7:0/2595946864 shutdown_connections 2024-01-27T02:41:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.074+0000 7f54122cc700 1 --2- 172.21.15.7:0/2595946864 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f53ec066f30 0x7f53ec0693e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.074+0000 7f54122cc700 1 --2- 172.21.15.7:0/2595946864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f540c15a100 0x7f540c06c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.074+0000 7f54122cc700 1 --2- 172.21.15.7:0/2595946864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f540c0c4af0 0x7f540c156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.074+0000 7f54122cc700 1 --2- 172.21.15.7:0/2595946864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f540c0cb5b0 0x7f540c157c60 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.074+0000 7f54122cc700 1 -- 172.21.15.7:0/2595946864 >> 172.21.15.7:0/2595946864 conn(0x7f540c098020 msgr2=0x7f540c0cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.074+0000 7f54122cc700 1 -- 172.21.15.7:0/2595946864 shutdown_connections 2024-01-27T02:41:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.074+0000 7f54122cc700 1 -- 172.21.15.7:0/2595946864 wait complete. 2024-01-27T02:41:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 79 2024-01-27T02:41:53.089 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-27T02:41:53.089 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-27T02:41:53.089 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-27T02:41:53.089 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:41:53.089 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-27T02:41:53.089 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-27T02:41:53.089 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-27T02:41:53.090 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:41:53.090 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-27T02:41:53.090 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-27T02:41:53.090 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-27T02:41:53.090 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:41:53.090 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-27T02:41:53.090 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-27T02:41:53.091 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-27T02:41:53.091 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:41:53.099 INFO:tasks.ceph.mds.c:Stopped 2024-01-27T02:41:53.099 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-27T02:41:53.100 INFO:tasks.ceph.mds.a:Stopped 2024-01-27T02:41:53.100 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-27T02:41:53.101 INFO:tasks.ceph.mds.b:Stopped 2024-01-27T02:41:53.101 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-27T02:41:53.102 INFO:tasks.ceph.mds.d:Stopped 2024-01-27T02:41:53.102 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-27T02:41:53.142 INFO:tasks.ceph.mds.c:Started 2024-01-27T02:41:53.142 DEBUG:teuthology.parallel:result is None 2024-01-27T02:41:53.143 INFO:tasks.ceph.mds.a:Started 2024-01-27T02:41:53.143 DEBUG:teuthology.parallel:result is None 2024-01-27T02:41:53.144 INFO:tasks.ceph.mds.b:Started 2024-01-27T02:41:53.144 DEBUG:teuthology.parallel:result is None 2024-01-27T02:41:53.145 INFO:tasks.ceph.mds.d:Started 2024-01-27T02:41:53.145 DEBUG:teuthology.parallel:result is None 2024-01-27T02:41:53.145 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-27T02:41:53.227 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-01-27T02:41:53.228 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-01-27T02:41:53.230 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-27T02:41:53.240 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-27T02:41:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.295+0000 7fbe0219b700 1 Processor -- start 2024-01-27T02:41:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.295+0000 7fbe0219b700 1 -- start start 2024-01-27T02:41:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.296+0000 7fbe0219b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdfc15d3a0 0x7fbdfc0cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.296+0000 7fbe0219b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbdfc06ea60 0x7fbdfc06eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.296+0000 7fbe0219b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdfc06e5d0 con 0x7fbdfc15d770 2024-01-27T02:41:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.296+0000 7fbe0219b700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdfc0cbe10 con 0x7fbdfc06ea60 2024-01-27T02:41:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.296+0000 7fbe0219b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc0ce860 con 0x7fbdfc15d3a0 2024-01-27T02:41:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.296+0000 7fbdfaffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbdfc15d770 0x7fbdfc0c4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39636/0 (socket says 172.21.15.7:39636) 2024-01-27T02:41:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.296+0000 7fbdfaffd700 1 -- 172.21.15.7:0/991836021 learned_addr learned my addr 172.21.15.7:0/991836021 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:41:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.296+0000 7fbdfb7fe700 1 --2- 172.21.15.7:0/991836021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdfc15d3a0 0x7fbdfc0cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.296+0000 7fbdfbfff700 1 --2- 172.21.15.7:0/991836021 >> v2:172.21.15.188:3300/0 conn(0x7fbdfc06ea60 0x7fbdfc06eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.296+0000 7fbdfa7fc700 1 -- 172.21.15.7:0/991836021 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918575059 0 0) 0x7fbdfc06e5d0 con 0x7fbdfc15d770 2024-01-27T02:41:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.297+0000 7fbdfa7fc700 1 -- 172.21.15.7:0/991836021 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbde8003570 con 0x7fbdfc15d770 2024-01-27T02:41:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.297+0000 7fbdfbfff700 1 -- 172.21.15.7:0/991836021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdfc15d3a0 msgr2=0x7fbdfc0cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.297+0000 7fbdfbfff700 1 --2- 172.21.15.7:0/991836021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdfc15d3a0 0x7fbdfc0cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.297+0000 7fbdfbfff700 1 -- 172.21.15.7:0/991836021 >> v1:172.21.15.7:6789/0 conn(0x7fbdfc15d770 legacy=0x7fbdfc0c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.297+0000 7fbdfbfff700 1 -- 172.21.15.7:0/991836021 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdfc0cd8f0 con 0x7fbdfc06ea60 2024-01-27T02:41:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.297+0000 7fbdfbfff700 1 --2- 172.21.15.7:0/991836021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdfc06ea60 0x7fbdfc06eee0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4009e60 tx=0x7fbdf4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cc3d004040c8c50 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.297+0000 7fbdfa7fc700 1 -- 172.21.15.7:0/991836021 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf400e070 con 0x7fbdfc06ea60 2024-01-27T02:41:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.297+0000 7fbdfa7fc700 1 -- 172.21.15.7:0/991836021 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf400bab0 con 0x7fbdfc06ea60 2024-01-27T02:41:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.297+0000 7fbdfa7fc700 1 -- 172.21.15.7:0/991836021 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf400be70 con 0x7fbdfc06ea60 2024-01-27T02:41:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.298+0000 7fbe0219b700 1 -- 172.21.15.7:0/991836021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdfc06ea60 msgr2=0x7fbdfc06eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.298+0000 7fbe0219b700 1 --2- 172.21.15.7:0/991836021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdfc06ea60 0x7fbdfc06eee0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4009e60 tx=0x7fbdf4009780 comp rx=0 tx=0).stop 2024-01-27T02:41:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.298+0000 7fbe0219b700 1 -- 172.21.15.7:0/991836021 shutdown_connections 2024-01-27T02:41:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.298+0000 7fbe0219b700 1 --2- 172.21.15.7:0/991836021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdfc06ea60 0x7fbdfc06eee0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.298+0000 7fbe0219b700 1 --2- 172.21.15.7:0/991836021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdfc15d3a0 0x7fbdfc0cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.298+0000 7fbe0219b700 1 -- 172.21.15.7:0/991836021 >> 172.21.15.7:0/991836021 conn(0x7fbdfc068480 msgr2=0x7fbdfc0cdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.298+0000 7fbe0219b700 1 -- 172.21.15.7:0/991836021 shutdown_connections 2024-01-27T02:41:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.298+0000 7fbe0219b700 1 -- 172.21.15.7:0/991836021 wait complete. 2024-01-27T02:41:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.299+0000 7fbe0219b700 1 Processor -- start 2024-01-27T02:41:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.299+0000 7fbe0219b700 1 -- start start 2024-01-27T02:41:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.300+0000 7fbe0219b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdfc06ea60 0x7fbdfc1588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.300+0000 7fbe0219b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdfc15d3a0 0x7fbdfc0ad380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.300+0000 7fbe0219b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdfc15d770 0x7fbdfc15bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.300+0000 7fbe0219b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc0ae670 con 0x7fbdfc06ea60 2024-01-27T02:41:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.300+0000 7fbe0219b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc0b1cd0 con 0x7fbdfc15d770 2024-01-27T02:41:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.300+0000 7fbe0219b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc0addd0 con 0x7fbdfc15d3a0 2024-01-27T02:41:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.300+0000 7fbdfb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdfc06ea60 0x7fbdfc1588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.300+0000 7fbdfb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdfc06ea60 0x7fbdfc1588d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42992/0 (socket says 172.21.15.7:42992) 2024-01-27T02:41:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.300+0000 7fbdfb7fe700 1 -- 172.21.15.7:0/3508709606 learned_addr learned my addr 172.21.15.7:0/3508709606 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.300+0000 7fbdfaffd700 1 --2- 172.21.15.7:0/3508709606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdfc15d3a0 0x7fbdfc0ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.300+0000 7fbdfbfff700 1 --2- 172.21.15.7:0/3508709606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdfc15d770 0x7fbdfc15bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.300+0000 7fbdfb7fe700 1 -- 172.21.15.7:0/3508709606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdfc15d3a0 msgr2=0x7fbdfc0ad380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.300+0000 7fbdfb7fe700 1 --2- 172.21.15.7:0/3508709606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdfc15d3a0 0x7fbdfc0ad380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.300+0000 7fbdfb7fe700 1 -- 172.21.15.7:0/3508709606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdfc15d770 msgr2=0x7fbdfc15bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.300+0000 7fbdfb7fe700 1 --2- 172.21.15.7:0/3508709606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdfc15d770 0x7fbdfc15bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.300+0000 7fbdfb7fe700 1 -- 172.21.15.7:0/3508709606 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf4009a20 con 0x7fbdfc06ea60 2024-01-27T02:41:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.301+0000 7fbdfb7fe700 1 --2- 172.21.15.7:0/3508709606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdfc06ea60 0x7fbdfc1588d0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00ecb0 tx=0x7fbdec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.301+0000 7fbdf37fe700 1 -- 172.21.15.7:0/3508709606 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec00c9d0 con 0x7fbdfc06ea60 2024-01-27T02:41:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.301+0000 7fbdf37fe700 1 -- 172.21.15.7:0/3508709606 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdec00cb30 con 0x7fbdfc06ea60 2024-01-27T02:41:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.301+0000 7fbe0219b700 1 -- 172.21.15.7:0/3508709606 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdfc0ae0b0 con 0x7fbdfc06ea60 2024-01-27T02:41:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.301+0000 7fbdf37fe700 1 -- 172.21.15.7:0/3508709606 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec00ced0 con 0x7fbdfc06ea60 2024-01-27T02:41:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.301+0000 7fbe0219b700 1 -- 172.21.15.7:0/3508709606 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdfc0ac8d0 con 0x7fbdfc06ea60 2024-01-27T02:41:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.302+0000 7fbe0219b700 1 -- 172.21.15.7:0/3508709606 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdfc0b1aa0 con 0x7fbdfc06ea60 2024-01-27T02:41:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.302+0000 7fbdf37fe700 1 -- 172.21.15.7:0/3508709606 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbdec013680 con 0x7fbdfc06ea60 2024-01-27T02:41:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.303+0000 7fbdf37fe700 1 --2- 172.21.15.7:0/3508709606 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbddc066f30 0x7fbddc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:53.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.303+0000 7fbdf37fe700 1 -- 172.21.15.7:0/3508709606 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fbdec084590 con 0x7fbdfc06ea60 2024-01-27T02:41:53.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.303+0000 7fbdfaffd700 1 --2- 172.21.15.7:0/3508709606 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbddc066f30 0x7fbddc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:53.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.304+0000 7fbdfaffd700 1 --2- 172.21.15.7:0/3508709606 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbddc066f30 0x7fbddc0693e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0abf00 tx=0x7fbde4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:53.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.307+0000 7fbdf37fe700 1 -- 172.21.15.7:0/3508709606 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbdec051b00 con 0x7fbdfc06ea60 2024-01-27T02:41:53.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:53.527+0000 7fbe0219b700 1 -- 172.21.15.7:0/3508709606 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbdfc04c710 con 0x7fbdfc06ea60 2024-01-27T02:41:54.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.210+0000 7fbdf37fe700 1 -- 172.21.15.7:0/3508709606 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v158) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbdec055120 con 0x7fbdfc06ea60 2024-01-27T02:41:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.212+0000 7fbe0219b700 1 -- 172.21.15.7:0/3508709606 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbddc066f30 msgr2=0x7fbddc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.212+0000 7fbe0219b700 1 --2- 172.21.15.7:0/3508709606 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbddc066f30 0x7fbddc0693e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0abf00 tx=0x7fbde4009040 comp rx=0 tx=0).stop 2024-01-27T02:41:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.212+0000 7fbe0219b700 1 -- 172.21.15.7:0/3508709606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdfc06ea60 msgr2=0x7fbdfc1588d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.212+0000 7fbe0219b700 1 --2- 172.21.15.7:0/3508709606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdfc06ea60 0x7fbdfc1588d0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00ecb0 tx=0x7fbdec0091a0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.212+0000 7fbe0219b700 1 -- 172.21.15.7:0/3508709606 shutdown_connections 2024-01-27T02:41:54.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.212+0000 7fbe0219b700 1 --2- 172.21.15.7:0/3508709606 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbddc066f30 0x7fbddc0693e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.212+0000 7fbe0219b700 1 --2- 172.21.15.7:0/3508709606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdfc15d770 0x7fbdfc15bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.212+0000 7fbe0219b700 1 --2- 172.21.15.7:0/3508709606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdfc06ea60 0x7fbdfc1588d0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.212+0000 7fbe0219b700 1 --2- 172.21.15.7:0/3508709606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdfc15d3a0 0x7fbdfc0ad380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.213+0000 7fbe0219b700 1 -- 172.21.15.7:0/3508709606 >> 172.21.15.7:0/3508709606 conn(0x7fbdfc068480 msgr2=0x7fbdfc0c2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:54.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.213+0000 7fbe0219b700 1 -- 172.21.15.7:0/3508709606 shutdown_connections 2024-01-27T02:41:54.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.213+0000 7fbe0219b700 1 -- 172.21.15.7:0/3508709606 wait complete. 2024-01-27T02:41:54.214 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-01-27T02:41:54.230 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-27T02:41:54.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.366+0000 7ff57194b700 1 Processor -- start 2024-01-27T02:41:54.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.366+0000 7ff57194b700 1 -- start start 2024-01-27T02:41:54.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.366+0000 7ff57194b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff56c15a070 0x7ff56c15b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:54.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.366+0000 7ff57194b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff56c0caf10 0x7ff56c0aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:54.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.366+0000 7ff57194b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff56c0b04a0 0x7ff56c0679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:54.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.366+0000 7ff57194b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff56c0b0920 con 0x7ff56c0caf10 2024-01-27T02:41:54.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.366+0000 7ff57194b700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff56c0ae5f0 con 0x7ff56c0b04a0 2024-01-27T02:41:54.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.366+0000 7ff57194b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c0ae8b0 con 0x7ff56c15a070 2024-01-27T02:41:54.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.367+0000 7ff56a7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff56c0caf10 0x7ff56c0aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:54.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.367+0000 7ff56a7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff56c0caf10 0x7ff56c0aff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42994/0 (socket says 172.21.15.7:42994) 2024-01-27T02:41:54.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.367+0000 7ff56a7fc700 1 -- 172.21.15.7:0/3193206180 learned_addr learned my addr 172.21.15.7:0/3193206180 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:54.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.367+0000 7ff56b7fe700 1 --2- 172.21.15.7:0/3193206180 >> v2:172.21.15.188:3300/0 conn(0x7ff56c0b04a0 0x7ff56c0679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:54.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.367+0000 7ff56affd700 1 --2- 172.21.15.7:0/3193206180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff56c15a070 0x7ff56c15b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:54.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.367+0000 7ff56a7fc700 1 -- 172.21.15.7:0/3193206180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff56c15a070 msgr2=0x7ff56c15b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:54.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.367+0000 7ff56a7fc700 1 --2- 172.21.15.7:0/3193206180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff56c15a070 0x7ff56c15b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.367+0000 7ff56a7fc700 1 -- 172.21.15.7:0/3193206180 >> v2:172.21.15.188:3300/0 conn(0x7ff56c0b04a0 msgr2=0x7ff56c0679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:54.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.367+0000 7ff56a7fc700 1 --2- 172.21.15.7:0/3193206180 >> v2:172.21.15.188:3300/0 conn(0x7ff56c0b04a0 0x7ff56c0679b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.367+0000 7ff56a7fc700 1 -- 172.21.15.7:0/3193206180 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56c0aeb90 con 0x7ff56c0caf10 2024-01-27T02:41:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.367+0000 7ff56a7fc700 1 --2- 172.21.15.7:0/3193206180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff56c0caf10 0x7ff56c0aff60 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7ff56c06a780 tx=0x7ff560009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce01b4306be9d6b server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.367+0000 7ff569ffb700 1 -- 172.21.15.7:0/3193206180 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56000e070 con 0x7ff56c0caf10 2024-01-27T02:41:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.367+0000 7ff569ffb700 1 -- 172.21.15.7:0/3193206180 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56000b970 con 0x7ff56c0caf10 2024-01-27T02:41:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.367+0000 7ff569ffb700 1 -- 172.21.15.7:0/3193206180 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5600092e0 con 0x7ff56c0caf10 2024-01-27T02:41:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.368+0000 7ff57194b700 1 -- 172.21.15.7:0/3193206180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff56c0caf10 msgr2=0x7ff56c0aff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.368+0000 7ff57194b700 1 --2- 172.21.15.7:0/3193206180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff56c0caf10 0x7ff56c0aff60 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7ff56c06a780 tx=0x7ff560009840 comp rx=0 tx=0).stop 2024-01-27T02:41:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.368+0000 7ff57194b700 1 -- 172.21.15.7:0/3193206180 shutdown_connections 2024-01-27T02:41:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.368+0000 7ff57194b700 1 --2- 172.21.15.7:0/3193206180 >> v2:172.21.15.188:3300/0 conn(0x7ff56c0b04a0 0x7ff56c0679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.368+0000 7ff57194b700 1 --2- 172.21.15.7:0/3193206180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff56c15a070 0x7ff56c15b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.368+0000 7ff57194b700 1 --2- 172.21.15.7:0/3193206180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff56c0caf10 0x7ff56c0aff60 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.368+0000 7ff57194b700 1 -- 172.21.15.7:0/3193206180 >> 172.21.15.7:0/3193206180 conn(0x7ff56c098020 msgr2=0x7ff56c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.368+0000 7ff57194b700 1 -- 172.21.15.7:0/3193206180 shutdown_connections 2024-01-27T02:41:54.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.368+0000 7ff57194b700 1 -- 172.21.15.7:0/3193206180 wait complete. 2024-01-27T02:41:54.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.369+0000 7ff57194b700 1 Processor -- start 2024-01-27T02:41:54.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.370+0000 7ff57194b700 1 -- start start 2024-01-27T02:41:54.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.370+0000 7ff57194b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff56c0b04a0 0x7ff56c155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:54.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.370+0000 7ff57194b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff56c0caf10 0x7ff56c06c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:54.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.370+0000 7ff57194b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff56c15a070 0x7ff56c06c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:54.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.370+0000 7ff57194b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c15c810 con 0x7ff56c0b04a0 2024-01-27T02:41:54.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.370+0000 7ff57194b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c15cd90 con 0x7ff56c0caf10 2024-01-27T02:41:54.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.370+0000 7ff57194b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c09c490 con 0x7ff56c15a070 2024-01-27T02:41:54.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.370+0000 7ff56affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff56c0b04a0 0x7ff56c155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:54.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.370+0000 7ff56affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff56c0b04a0 0x7ff56c155da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43006/0 (socket says 172.21.15.7:43006) 2024-01-27T02:41:54.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.370+0000 7ff56affd700 1 -- 172.21.15.7:0/825462894 learned_addr learned my addr 172.21.15.7:0/825462894 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:54.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.371+0000 7ff56b7fe700 1 --2- 172.21.15.7:0/825462894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff56c15a070 0x7ff56c06c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:54.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.371+0000 7ff56a7fc700 1 --2- 172.21.15.7:0/825462894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff56c0caf10 0x7ff56c06c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:54.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.371+0000 7ff56affd700 1 -- 172.21.15.7:0/825462894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff56c15a070 msgr2=0x7ff56c06c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:54.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.371+0000 7ff56affd700 1 --2- 172.21.15.7:0/825462894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff56c15a070 0x7ff56c06c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.371+0000 7ff56affd700 1 -- 172.21.15.7:0/825462894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff56c0caf10 msgr2=0x7ff56c06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:54.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.371+0000 7ff56affd700 1 --2- 172.21.15.7:0/825462894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff56c0caf10 0x7ff56c06c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.371+0000 7ff56affd700 1 -- 172.21.15.7:0/825462894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff560009ae0 con 0x7ff56c0b04a0 2024-01-27T02:41:54.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.371+0000 7ff56a7fc700 1 --2- 172.21.15.7:0/825462894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff56c0caf10 0x7ff56c06c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:41:54.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.371+0000 7ff56affd700 1 --2- 172.21.15.7:0/825462894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff56c0b04a0 0x7ff56c155da0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00e930 tx=0x7ff55c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:54.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.371+0000 7ff552ffd700 1 -- 172.21.15.7:0/825462894 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c013680 con 0x7ff56c0b04a0 2024-01-27T02:41:54.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.371+0000 7ff552ffd700 1 -- 172.21.15.7:0/825462894 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55c0137e0 con 0x7ff56c0b04a0 2024-01-27T02:41:54.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.371+0000 7ff552ffd700 1 -- 172.21.15.7:0/825462894 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c013b80 con 0x7ff56c0b04a0 2024-01-27T02:41:54.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.371+0000 7ff57194b700 1 -- 172.21.15.7:0/825462894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff56c09c770 con 0x7ff56c0b04a0 2024-01-27T02:41:54.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.371+0000 7ff57194b700 1 -- 172.21.15.7:0/825462894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff56c1562e0 con 0x7ff56c0b04a0 2024-01-27T02:41:54.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.372+0000 7ff57194b700 1 -- 172.21.15.7:0/825462894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff56c15d070 con 0x7ff56c0b04a0 2024-01-27T02:41:54.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.373+0000 7ff552ffd700 1 -- 172.21.15.7:0/825462894 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff55c00c460 con 0x7ff56c0b04a0 2024-01-27T02:41:54.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.373+0000 7ff552ffd700 1 --2- 172.21.15.7:0/825462894 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff558066f30 0x7ff5580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:54.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.373+0000 7ff552ffd700 1 -- 172.21.15.7:0/825462894 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff55c084350 con 0x7ff56c0b04a0 2024-01-27T02:41:54.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.373+0000 7ff56a7fc700 1 --2- 172.21.15.7:0/825462894 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff558066f30 0x7ff5580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:54.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.374+0000 7ff56a7fc700 1 --2- 172.21.15.7:0/825462894 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff558066f30 0x7ff5580693e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff560009880 tx=0x7ff560018000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:54.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.378+0000 7ff552ffd700 1 -- 172.21.15.7:0/825462894 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff55c051910 con 0x7ff56c0b04a0 2024-01-27T02:41:54.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.662+0000 7ff57194b700 1 -- 172.21.15.7:0/825462894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff56c09c960 con 0x7ff56c0b04a0 2024-01-27T02:41:54.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.663+0000 7ff552ffd700 1 -- 172.21.15.7:0/825462894 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff55c054f30 con 0x7ff56c0b04a0 2024-01-27T02:41:54.663 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:41:54.663 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:41:54.663 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-01-27T02:41:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:41:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-01-27T02:41:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlHdcPKBAAfhnh1I3ZSQEXWJXlXCs98Q==", 2024-01-27T02:41:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:41:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:41:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:41:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:41:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:41:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:41:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:41:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:41:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-01-27T02:41:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rl8eVYNRAAfP8rOqO50RpoNZ1+1j2DXQ==", 2024-01-27T02:41:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:41:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:41:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:41:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:41:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:41:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:41:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:41:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:41:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-01-27T02:41:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlJxRKMBAAS0j7/zZeqGglOxCK1pg0WA==", 2024-01-27T02:41:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:41:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:41:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:41:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:41:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:41:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:41:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:41:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:41:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-01-27T02:41:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlNwfUABAALRQAoHoGhniqCvTQOqbbug==", 2024-01-27T02:41:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:41:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:41:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:41:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:41:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:41:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:41:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:41:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:41:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-01-27T02:41:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7Rlh6VSJhAAttuCte1WhNEvUkv5x/IKQQ==", 2024-01-27T02:41:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:41:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:41:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:41:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:41:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:41:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:41:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:41:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-01-27T02:41:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7Rlu/RGLhAAKzaCNhzYKhsNhxkMvY++gw==", 2024-01-27T02:41:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:41:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:41:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:41:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:41:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:41:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:41:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:41:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-01-27T02:41:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7RliRMyNhAAr0tG3evmVV7ZlGxbxfWPMw==", 2024-01-27T02:41:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:41:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:41:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:41:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:41:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:41:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:41:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:41:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-01-27T02:41:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAPa7RlRuSNAhAA3e9atlb+ffvlnLhRYbWZPw==", 2024-01-27T02:41:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:41:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:41:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:41:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:41:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:41:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:41:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:41:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-01-27T02:41:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7RlfeA6DBAAXpklgbNHRO+5G+fdIERjRw==", 2024-01-27T02:41:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:41:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:41:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:41:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:41:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:41:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:41:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:41:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-01-27T02:41:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7Rl77ItExAABB7uABm63ubE/YUs2B5pGw==", 2024-01-27T02:41:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:41:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:41:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:41:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:41:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:41:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:41:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:41:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-01-27T02:41:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7RljDuwHBAAPPAlhoqS03qPpq58jmevLA==", 2024-01-27T02:41:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:41:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:41:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:41:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:41:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:41:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:41:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:41:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-01-27T02:41:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7Rl1YElJhAAqHP/QEdl1SEalGcbQwPwAg==", 2024-01-27T02:41:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:41:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:41:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:41:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:41:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:41:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:41:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:41:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-01-27T02:41:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlaEqaBRAATFiIeW8lT9BdNuN0m3x9dQ==", 2024-01-27T02:41:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:41:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:41:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-01-27T02:41:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-27T02:41:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:41:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:41:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:41:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-01-27T02:41:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlNADmEBAAJXwdJ35ylP5coy1dtHoD3g==", 2024-01-27T02:41:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:41:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:41:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:41:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:41:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:41:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:41:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:41:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:41:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-01-27T02:41:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7Rl5fRqCRAA/5swTGknnpP64TAh+2X41A==", 2024-01-27T02:41:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:41:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:41:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:41:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:41:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:41:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:41:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:41:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:41:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-01-27T02:41:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7Rl0SdwFhAAhCmw9ViiJbHVfS5a2r/D+w==", 2024-01-27T02:41:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:41:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:41:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:41:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:41:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:41:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:41:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:41:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:41:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-01-27T02:41:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlsNgPARAA0Qm0oJgA8GO1oUT3ndT8Tg==", 2024-01-27T02:41:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:41:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:41:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:41:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:41:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:41:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:41:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:41:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:41:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-01-27T02:41:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rlfb7qIxAAlsJYiSolJ7D8EykGWP62bA==", 2024-01-27T02:41:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:41:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:41:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-27T02:41:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:41:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:41:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:41:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:41:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-01-27T02:41:54.676 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rls7TaHxAALS3ghxKtfmgQfIELiD9+Bg==", 2024-01-27T02:41:54.676 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:41:54.676 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:41:54.676 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-27T02:41:54.676 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:41:54.676 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:41:54.676 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:41:54.676 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:41:54.676 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:41:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.665+0000 7ff57194b700 1 -- 172.21.15.7:0/825462894 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff558066f30 msgr2=0x7ff5580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.665+0000 7ff57194b700 1 --2- 172.21.15.7:0/825462894 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff558066f30 0x7ff5580693e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff560009880 tx=0x7ff560018000 comp rx=0 tx=0).stop 2024-01-27T02:41:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.665+0000 7ff57194b700 1 -- 172.21.15.7:0/825462894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff56c0b04a0 msgr2=0x7ff56c155da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.665+0000 7ff57194b700 1 --2- 172.21.15.7:0/825462894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff56c0b04a0 0x7ff56c155da0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00e930 tx=0x7ff55c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.665+0000 7ff57194b700 1 -- 172.21.15.7:0/825462894 shutdown_connections 2024-01-27T02:41:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.665+0000 7ff57194b700 1 --2- 172.21.15.7:0/825462894 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff558066f30 0x7ff5580693e0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.665+0000 7ff57194b700 1 --2- 172.21.15.7:0/825462894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff56c15a070 0x7ff56c06c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.665+0000 7ff57194b700 1 --2- 172.21.15.7:0/825462894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff56c0b04a0 0x7ff56c155da0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.665+0000 7ff57194b700 1 --2- 172.21.15.7:0/825462894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff56c0caf10 0x7ff56c06c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.665+0000 7ff57194b700 1 -- 172.21.15.7:0/825462894 >> 172.21.15.7:0/825462894 conn(0x7ff56c098020 msgr2=0x7ff56c159250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:54.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.665+0000 7ff57194b700 1 -- 172.21.15.7:0/825462894 shutdown_connections 2024-01-27T02:41:54.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.665+0000 7ff57194b700 1 -- 172.21.15.7:0/825462894 wait complete. 2024-01-27T02:41:54.679 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:41:54.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.811+0000 7fca4961b700 1 Processor -- start 2024-01-27T02:41:54.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.811+0000 7fca4961b700 1 -- start start 2024-01-27T02:41:54.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.811+0000 7fca4961b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fca440b2ff0 0x7fca440b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:54.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.811+0000 7fca4961b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca4415b8e0 0x7fca440ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:54.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.811+0000 7fca4961b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fca4415ddd0 0x7fca4415e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:54.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.812+0000 7fca4961b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca440b3900 con 0x7fca440b2ff0 2024-01-27T02:41:54.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.812+0000 7fca4961b700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca440ce7e0 con 0x7fca4415ddd0 2024-01-27T02:41:54.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.812+0000 7fca4961b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4415ebe0 con 0x7fca4415b8e0 2024-01-27T02:41:54.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.812+0000 7fca42ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fca440b2ff0 0x7fca440b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:54.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.812+0000 7fca42ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fca440b2ff0 0x7fca440b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43010/0 (socket says 172.21.15.7:43010) 2024-01-27T02:41:54.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.812+0000 7fca42ffd700 1 -- 172.21.15.7:0/3659763789 learned_addr learned my addr 172.21.15.7:0/3659763789 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:54.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.812+0000 7fca437fe700 1 --2- 172.21.15.7:0/3659763789 >> v2:172.21.15.188:3300/0 conn(0x7fca4415ddd0 0x7fca4415e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:54.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.812+0000 7fca427fc700 1 --2- 172.21.15.7:0/3659763789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca4415b8e0 0x7fca440ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:54.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.812+0000 7fca42ffd700 1 -- 172.21.15.7:0/3659763789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca4415b8e0 msgr2=0x7fca440ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:54.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.812+0000 7fca42ffd700 1 --2- 172.21.15.7:0/3659763789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca4415b8e0 0x7fca440ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.812+0000 7fca42ffd700 1 -- 172.21.15.7:0/3659763789 >> v2:172.21.15.188:3300/0 conn(0x7fca4415ddd0 msgr2=0x7fca4415e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:54.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.812+0000 7fca42ffd700 1 --2- 172.21.15.7:0/3659763789 >> v2:172.21.15.188:3300/0 conn(0x7fca4415ddd0 0x7fca4415e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.812+0000 7fca42ffd700 1 -- 172.21.15.7:0/3659763789 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4415e720 con 0x7fca440b2ff0 2024-01-27T02:41:54.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.812+0000 7fca437fe700 1 --2- 172.21.15.7:0/3659763789 >> v2:172.21.15.188:3300/0 conn(0x7fca4415ddd0 0x7fca4415e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:41:54.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.812+0000 7fca42ffd700 1 --2- 172.21.15.7:0/3659763789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca440b2ff0 0x7fca440b33c0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fca44154d80 tx=0x7fca34009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb06a8f5186fd88 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.812+0000 7fca41ffb700 1 -- 172.21.15.7:0/3659763789 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3400e070 con 0x7fca440b2ff0 2024-01-27T02:41:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.812+0000 7fca41ffb700 1 -- 172.21.15.7:0/3659763789 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3400bad0 con 0x7fca440b2ff0 2024-01-27T02:41:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.812+0000 7fca41ffb700 1 -- 172.21.15.7:0/3659763789 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca340092e0 con 0x7fca440b2ff0 2024-01-27T02:41:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.813+0000 7fca4961b700 1 -- 172.21.15.7:0/3659763789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca440b2ff0 msgr2=0x7fca440b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.813+0000 7fca4961b700 1 --2- 172.21.15.7:0/3659763789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca440b2ff0 0x7fca440b33c0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fca44154d80 tx=0x7fca34009840 comp rx=0 tx=0).stop 2024-01-27T02:41:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.813+0000 7fca4961b700 1 -- 172.21.15.7:0/3659763789 shutdown_connections 2024-01-27T02:41:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.813+0000 7fca4961b700 1 --2- 172.21.15.7:0/3659763789 >> v2:172.21.15.188:3300/0 conn(0x7fca4415ddd0 0x7fca4415e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.813+0000 7fca4961b700 1 --2- 172.21.15.7:0/3659763789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca440b2ff0 0x7fca440b33c0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.813+0000 7fca4961b700 1 --2- 172.21.15.7:0/3659763789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca4415b8e0 0x7fca440ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.813+0000 7fca4961b700 1 -- 172.21.15.7:0/3659763789 >> 172.21.15.7:0/3659763789 conn(0x7fca440679b0 msgr2=0x7fca44067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.813+0000 7fca4961b700 1 -- 172.21.15.7:0/3659763789 shutdown_connections 2024-01-27T02:41:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.813+0000 7fca4961b700 1 -- 172.21.15.7:0/3659763789 wait complete. 2024-01-27T02:41:54.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.815+0000 7fca4961b700 1 Processor -- start 2024-01-27T02:41:54.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.815+0000 7fca4961b700 1 -- start start 2024-01-27T02:41:54.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.816+0000 7fca4961b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca440b2ff0 0x7fca4413e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:54.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.816+0000 7fca4961b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca4415b8e0 0x7fca440b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:54.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.816+0000 7fca4961b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca4415ddd0 0x7fca440b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:54.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.816+0000 7fca4961b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4413e580 con 0x7fca4415b8e0 2024-01-27T02:41:54.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.816+0000 7fca4961b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca440b6c00 con 0x7fca440b2ff0 2024-01-27T02:41:54.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.816+0000 7fca4961b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca44155800 con 0x7fca4415ddd0 2024-01-27T02:41:54.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.816+0000 7fca427fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca4415b8e0 0x7fca440b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:54.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.816+0000 7fca427fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca4415b8e0 0x7fca440b66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43012/0 (socket says 172.21.15.7:43012) 2024-01-27T02:41:54.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.816+0000 7fca427fc700 1 -- 172.21.15.7:0/3507704216 learned_addr learned my addr 172.21.15.7:0/3507704216 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:54.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.816+0000 7fca437fe700 1 --2- 172.21.15.7:0/3507704216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca4415ddd0 0x7fca440b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:54.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.816+0000 7fca42ffd700 1 --2- 172.21.15.7:0/3507704216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca440b2ff0 0x7fca4413e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:54.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.817+0000 7fca427fc700 1 -- 172.21.15.7:0/3507704216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca4415ddd0 msgr2=0x7fca440b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:54.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.817+0000 7fca427fc700 1 --2- 172.21.15.7:0/3507704216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca4415ddd0 0x7fca440b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.817+0000 7fca427fc700 1 -- 172.21.15.7:0/3507704216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca440b2ff0 msgr2=0x7fca4413e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:54.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.817+0000 7fca427fc700 1 --2- 172.21.15.7:0/3507704216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca440b2ff0 0x7fca4413e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:54.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.817+0000 7fca427fc700 1 -- 172.21.15.7:0/3507704216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca34009ae0 con 0x7fca4415b8e0 2024-01-27T02:41:54.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.817+0000 7fca427fc700 1 --2- 172.21.15.7:0/3507704216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca4415b8e0 0x7fca440b66c0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fca3800d890 tx=0x7fca38008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:54.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.817+0000 7fca2affd700 1 -- 172.21.15.7:0/3507704216 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca380125f0 con 0x7fca4415b8e0 2024-01-27T02:41:54.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.817+0000 7fca2affd700 1 -- 172.21.15.7:0/3507704216 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca38012750 con 0x7fca4415b8e0 2024-01-27T02:41:54.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.817+0000 7fca4961b700 1 -- 172.21.15.7:0/3507704216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca44155ae0 con 0x7fca4415b8e0 2024-01-27T02:41:54.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.817+0000 7fca2affd700 1 -- 172.21.15.7:0/3507704216 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca38012af0 con 0x7fca4415b8e0 2024-01-27T02:41:54.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.817+0000 7fca4961b700 1 -- 172.21.15.7:0/3507704216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca441560a0 con 0x7fca4415b8e0 2024-01-27T02:41:54.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.818+0000 7fca4961b700 1 -- 172.21.15.7:0/3507704216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4404c710 con 0x7fca4415b8e0 2024-01-27T02:41:54.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.819+0000 7fca2affd700 1 -- 172.21.15.7:0/3507704216 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fca3801e460 con 0x7fca4415b8e0 2024-01-27T02:41:54.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.819+0000 7fca2affd700 1 --2- 172.21.15.7:0/3507704216 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fca30066f30 0x7fca300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:54.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.819+0000 7fca2affd700 1 -- 172.21.15.7:0/3507704216 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fca38084730 con 0x7fca4415b8e0 2024-01-27T02:41:54.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.819+0000 7fca42ffd700 1 --2- 172.21.15.7:0/3507704216 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fca30066f30 0x7fca300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:54.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.820+0000 7fca42ffd700 1 --2- 172.21.15.7:0/3507704216 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fca30066f30 0x7fca300693e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fca34009fd0 tx=0x7fca34018000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:54.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:54.823+0000 7fca2affd700 1 -- 172.21.15.7:0/3507704216 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca38055780 con 0x7fca4415b8e0 2024-01-27T02:41:55.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.030+0000 7fca4961b700 1 -- 172.21.15.7:0/3507704216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca441587b0 con 0x7fca4415b8e0 2024-01-27T02:41:55.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.030+0000 7fca2affd700 1 -- 172.21.15.7:0/3507704216 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+12097 (secure 0 0 0) 0x7fca38055310 con 0x7fca4415b8e0 2024-01-27T02:41:55.031 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:41:55.031 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":160,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:41:54.217393+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:41:55.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.033+0000 7fca4961b700 1 -- 172.21.15.7:0/3507704216 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fca30066f30 msgr2=0x7fca300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:55.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.033+0000 7fca4961b700 1 --2- 172.21.15.7:0/3507704216 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fca30066f30 0x7fca300693e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fca34009fd0 tx=0x7fca34018000 comp rx=0 tx=0).stop 2024-01-27T02:41:55.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.033+0000 7fca4961b700 1 -- 172.21.15.7:0/3507704216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca4415b8e0 msgr2=0x7fca440b66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:55.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.033+0000 7fca4961b700 1 --2- 172.21.15.7:0/3507704216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca4415b8e0 0x7fca440b66c0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fca3800d890 tx=0x7fca38008040 comp rx=0 tx=0).stop 2024-01-27T02:41:55.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.033+0000 7fca4961b700 1 -- 172.21.15.7:0/3507704216 shutdown_connections 2024-01-27T02:41:55.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.033+0000 7fca4961b700 1 --2- 172.21.15.7:0/3507704216 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fca30066f30 0x7fca300693e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.033+0000 7fca4961b700 1 --2- 172.21.15.7:0/3507704216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca4415ddd0 0x7fca440b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.033+0000 7fca4961b700 1 --2- 172.21.15.7:0/3507704216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca440b2ff0 0x7fca4413e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.033+0000 7fca4961b700 1 --2- 172.21.15.7:0/3507704216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca4415b8e0 0x7fca440b66c0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.033+0000 7fca4961b700 1 -- 172.21.15.7:0/3507704216 >> 172.21.15.7:0/3507704216 conn(0x7fca440679b0 msgr2=0x7fca440b22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:55.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.033+0000 7fca4961b700 1 -- 172.21.15.7:0/3507704216 shutdown_connections 2024-01-27T02:41:55.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.033+0000 7fca4961b700 1 -- 172.21.15.7:0/3507704216 wait complete. 2024-01-27T02:41:55.047 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-27T02:41:55.047 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:41:55.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.183+0000 7f3fb9ad9700 1 Processor -- start 2024-01-27T02:41:55.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.183+0000 7f3fb9ad9700 1 -- start start 2024-01-27T02:41:55.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.184+0000 7f3fb9ad9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3fb415a180 0x7f3fb40c3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:55.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.184+0000 7f3fb9ad9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb40ca3b0 0x7f3fb40aef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:55.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.184+0000 7f3fb9ad9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fb40b1450 con 0x7f3fb415a550 2024-01-27T02:41:55.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.184+0000 7f3fb9ad9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb40b18e0 con 0x7f3fb415a180 2024-01-27T02:41:55.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.184+0000 7f3fb9ad9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb40b0840 con 0x7f3fb40ca3b0 2024-01-27T02:41:55.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.184+0000 7f3fb37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3fb415a180 0x7f3fb40c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.184+0000 7f3fb2ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3fb415a550 0x7f3fb40c9ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39650/0 (socket says 172.21.15.7:39650) 2024-01-27T02:41:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.184+0000 7f3fb2ffd700 1 -- 172.21.15.7:0/951698695 learned_addr learned my addr 172.21.15.7:0/951698695 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:41:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.184+0000 7f3fb3fff700 1 --2- 172.21.15.7:0/951698695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb40ca3b0 0x7f3fb40aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.184+0000 7f3fb37fe700 1 -- 172.21.15.7:0/951698695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb40ca3b0 msgr2=0x7f3fb40aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.184+0000 7f3fb37fe700 1 --2- 172.21.15.7:0/951698695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb40ca3b0 0x7f3fb40aef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.184+0000 7f3fb37fe700 1 -- 172.21.15.7:0/951698695 >> v1:172.21.15.7:6789/0 conn(0x7f3fb415a550 legacy=0x7f3fb40c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.184+0000 7f3fb37fe700 1 -- 172.21.15.7:0/951698695 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb40b0b20 con 0x7f3fb415a180 2024-01-27T02:41:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.184+0000 7f3fb27fc700 1 -- 172.21.15.7:0/951698695 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1217306562 0 0) 0x7f3fb40b1450 con 0x7f3fb415a550 2024-01-27T02:41:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.185+0000 7f3fb37fe700 1 --2- 172.21.15.7:0/951698695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fb415a180 0x7f3fb40c3750 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40098d0 tx=0x7f3fa4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed311efeb32673d server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.185+0000 7f3fb27fc700 1 -- 172.21.15.7:0/951698695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4009e70 con 0x7f3fb415a180 2024-01-27T02:41:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.185+0000 7f3fb27fc700 1 -- 172.21.15.7:0/951698695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa4005c10 con 0x7f3fb415a180 2024-01-27T02:41:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.185+0000 7f3fb27fc700 1 -- 172.21.15.7:0/951698695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa400b650 con 0x7f3fb415a180 2024-01-27T02:41:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.185+0000 7f3fb9ad9700 1 -- 172.21.15.7:0/951698695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fb415a180 msgr2=0x7f3fb40c3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.185+0000 7f3fb9ad9700 1 --2- 172.21.15.7:0/951698695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fb415a180 0x7f3fb40c3750 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40098d0 tx=0x7f3fa4009120 comp rx=0 tx=0).stop 2024-01-27T02:41:55.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.185+0000 7f3fb9ad9700 1 -- 172.21.15.7:0/951698695 shutdown_connections 2024-01-27T02:41:55.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.185+0000 7f3fb9ad9700 1 --2- 172.21.15.7:0/951698695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb40ca3b0 0x7f3fb40aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.185+0000 7f3fb9ad9700 1 --2- 172.21.15.7:0/951698695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fb415a180 0x7f3fb40c3750 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.185+0000 7f3fb9ad9700 1 -- 172.21.15.7:0/951698695 >> 172.21.15.7:0/951698695 conn(0x7f3fb40679b0 msgr2=0x7f3fb4067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:55.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.186+0000 7f3fb9ad9700 1 -- 172.21.15.7:0/951698695 shutdown_connections 2024-01-27T02:41:55.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.186+0000 7f3fb9ad9700 1 -- 172.21.15.7:0/951698695 wait complete. 2024-01-27T02:41:55.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.187+0000 7f3fb9ad9700 1 Processor -- start 2024-01-27T02:41:55.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.187+0000 7f3fb9ad9700 1 -- start start 2024-01-27T02:41:55.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.188+0000 7f3fb9ad9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fb40ca3b0 0x7f3fb4126790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:55.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.188+0000 7f3fb9ad9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb415a180 0x7f3fb4126cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:55.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.188+0000 7f3fb9ad9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb415a550 0x7f3fb406e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:55.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.188+0000 7f3fb9ad9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb415c350 con 0x7f3fb40ca3b0 2024-01-27T02:41:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.188+0000 7f3fb9ad9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb416fff0 con 0x7f3fb415a550 2024-01-27T02:41:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.188+0000 7f3fb9ad9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb416fd10 con 0x7f3fb415a180 2024-01-27T02:41:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.188+0000 7f3fb37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fb40ca3b0 0x7f3fb4126790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.188+0000 7f3fb37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fb40ca3b0 0x7f3fb4126790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43032/0 (socket says 172.21.15.7:43032) 2024-01-27T02:41:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.188+0000 7f3fb37fe700 1 -- 172.21.15.7:0/1195841332 learned_addr learned my addr 172.21.15.7:0/1195841332 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.188+0000 7f3fb2ffd700 1 --2- 172.21.15.7:0/1195841332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb415a180 0x7f3fb4126cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.188+0000 7f3fb3fff700 1 --2- 172.21.15.7:0/1195841332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb415a550 0x7f3fb406e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.188+0000 7f3fb37fe700 1 -- 172.21.15.7:0/1195841332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb415a180 msgr2=0x7f3fb4126cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.188+0000 7f3fb37fe700 1 --2- 172.21.15.7:0/1195841332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb415a180 0x7f3fb4126cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.188+0000 7f3fb37fe700 1 -- 172.21.15.7:0/1195841332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb415a550 msgr2=0x7f3fb406e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.188+0000 7f3fb37fe700 1 --2- 172.21.15.7:0/1195841332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb415a550 0x7f3fb406e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.188+0000 7f3fb37fe700 1 -- 172.21.15.7:0/1195841332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa40093c0 con 0x7f3fb40ca3b0 2024-01-27T02:41:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.188+0000 7f3fb37fe700 1 --2- 172.21.15.7:0/1195841332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fb40ca3b0 0x7f3fb4126790 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4000e50 tx=0x7f3fa4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:55.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.189+0000 7f3fab7fe700 1 -- 172.21.15.7:0/1195841332 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa40063e0 con 0x7f3fb40ca3b0 2024-01-27T02:41:55.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.189+0000 7f3fab7fe700 1 -- 172.21.15.7:0/1195841332 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa4006540 con 0x7f3fb40ca3b0 2024-01-27T02:41:55.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.189+0000 7f3fb9ad9700 1 -- 172.21.15.7:0/1195841332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb415ab50 con 0x7f3fb40ca3b0 2024-01-27T02:41:55.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.189+0000 7f3fab7fe700 1 -- 172.21.15.7:0/1195841332 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa40068c0 con 0x7f3fb40ca3b0 2024-01-27T02:41:55.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.189+0000 7f3fb9ad9700 1 -- 172.21.15.7:0/1195841332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb415b630 con 0x7f3fb40ca3b0 2024-01-27T02:41:55.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.190+0000 7f3fb9ad9700 1 -- 172.21.15.7:0/1195841332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb4072060 con 0x7f3fb40ca3b0 2024-01-27T02:41:55.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.190+0000 7f3fab7fe700 1 -- 172.21.15.7:0/1195841332 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3fa401a630 con 0x7f3fb40ca3b0 2024-01-27T02:41:55.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.191+0000 7f3fab7fe700 1 --2- 172.21.15.7:0/1195841332 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3f94066f30 0x7f3f940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:55.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.191+0000 7f3fab7fe700 1 -- 172.21.15.7:0/1195841332 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3fa40858a0 con 0x7f3fb40ca3b0 2024-01-27T02:41:55.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.191+0000 7f3fb2ffd700 1 --2- 172.21.15.7:0/1195841332 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3f94066f30 0x7f3f940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:55.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.191+0000 7f3fb2ffd700 1 --2- 172.21.15.7:0/1195841332 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3f94066f30 0x7f3f940693e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40b12c0 tx=0x7f3f9c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:55.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.195+0000 7f3fab7fe700 1 -- 172.21.15.7:0/1195841332 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3fa4052e60 con 0x7f3fb40ca3b0 2024-01-27T02:41:55.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.399+0000 7f3fb9ad9700 1 -- 172.21.15.7:0/1195841332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3fb40cacb0 con 0x7f3fb40ca3b0 2024-01-27T02:41:55.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.400+0000 7f3fab7fe700 1 -- 172.21.15.7:0/1195841332 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v160) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3fa402b0a0 con 0x7f3fb40ca3b0 2024-01-27T02:41:55.400 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:41:55.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.402+0000 7f3fb9ad9700 1 -- 172.21.15.7:0/1195841332 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3f94066f30 msgr2=0x7f3f940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:55.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.402+0000 7f3fb9ad9700 1 --2- 172.21.15.7:0/1195841332 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3f94066f30 0x7f3f940693e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40b12c0 tx=0x7f3f9c009040 comp rx=0 tx=0).stop 2024-01-27T02:41:55.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.402+0000 7f3fb9ad9700 1 -- 172.21.15.7:0/1195841332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fb40ca3b0 msgr2=0x7f3fb4126790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:55.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.402+0000 7f3fb9ad9700 1 --2- 172.21.15.7:0/1195841332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fb40ca3b0 0x7f3fb4126790 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4000e50 tx=0x7f3fa4016000 comp rx=0 tx=0).stop 2024-01-27T02:41:55.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.402+0000 7f3fb9ad9700 1 -- 172.21.15.7:0/1195841332 shutdown_connections 2024-01-27T02:41:55.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.402+0000 7f3fb9ad9700 1 --2- 172.21.15.7:0/1195841332 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3f94066f30 0x7f3f940693e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.402+0000 7f3fb9ad9700 1 --2- 172.21.15.7:0/1195841332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb415a550 0x7f3fb406e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.402+0000 7f3fb9ad9700 1 --2- 172.21.15.7:0/1195841332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fb40ca3b0 0x7f3fb4126790 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.402+0000 7f3fb9ad9700 1 --2- 172.21.15.7:0/1195841332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb415a180 0x7f3fb4126cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.402+0000 7f3fb9ad9700 1 -- 172.21.15.7:0/1195841332 >> 172.21.15.7:0/1195841332 conn(0x7f3fb40679b0 msgr2=0x7f3fb40cc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:55.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.402+0000 7f3fb9ad9700 1 -- 172.21.15.7:0/1195841332 shutdown_connections 2024-01-27T02:41:55.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.402+0000 7f3fb9ad9700 1 -- 172.21.15.7:0/1195841332 wait complete. 2024-01-27T02:41:55.416 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-27T02:41:55.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.549+0000 7f1f380f4700 1 Processor -- start 2024-01-27T02:41:55.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.549+0000 7f1f380f4700 1 -- start start 2024-01-27T02:41:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.550+0000 7f1f380f4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f300c4cc0 0x7f1f300cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.550+0000 7f1f380f4700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1f300ca540 0x7f1f30163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.550+0000 7f1f380f4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f300ce5e0 con 0x7f1f3015a100 2024-01-27T02:41:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.550+0000 7f1f380f4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f300cd9d0 con 0x7f1f300c4cc0 2024-01-27T02:41:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.550+0000 7f1f380f4700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f300cdcb0 con 0x7f1f300ca540 2024-01-27T02:41:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.550+0000 7f1f3568f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f300c4cc0 0x7f1f300cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.550+0000 7f1f3568f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f300c4cc0 0x7f1f300cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43048/0 (socket says 172.21.15.7:43048) 2024-01-27T02:41:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.550+0000 7f1f35e90700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1f3015a100 0x7f1f300c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39652/0 (socket says 172.21.15.7:39652) 2024-01-27T02:41:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.550+0000 7f1f3568f700 1 -- 172.21.15.7:0/3720593107 learned_addr learned my addr 172.21.15.7:0/3720593107 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.550+0000 7f1f36691700 1 --2- 172.21.15.7:0/3720593107 >> v2:172.21.15.188:3300/0 conn(0x7f1f300ca540 0x7f1f30163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.551+0000 7f1f3568f700 1 -- 172.21.15.7:0/3720593107 >> v2:172.21.15.188:3300/0 conn(0x7f1f300ca540 msgr2=0x7f1f30163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.551+0000 7f1f3568f700 1 --2- 172.21.15.7:0/3720593107 >> v2:172.21.15.188:3300/0 conn(0x7f1f300ca540 0x7f1f30163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.551+0000 7f1f3568f700 1 -- 172.21.15.7:0/3720593107 >> v1:172.21.15.7:6789/0 conn(0x7f1f3015a100 legacy=0x7f1f300c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.551+0000 7f1f3568f700 1 -- 172.21.15.7:0/3720593107 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f300cdf90 con 0x7f1f300c4cc0 2024-01-27T02:41:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.551+0000 7f1f34e8e700 1 -- 172.21.15.7:0/3720593107 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389192335 0 0) 0x7f1f300ce5e0 con 0x7f1f3015a100 2024-01-27T02:41:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.551+0000 7f1f3568f700 1 --2- 172.21.15.7:0/3720593107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f300c4cc0 0x7f1f300cd2f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f1f30069b70 tx=0x7f1f20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54889036516da98 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.551+0000 7f1f34e8e700 1 -- 172.21.15.7:0/3720593107 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2000b860 con 0x7f1f300c4cc0 2024-01-27T02:41:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.551+0000 7f1f34e8e700 1 -- 172.21.15.7:0/3720593107 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2000b9c0 con 0x7f1f300c4cc0 2024-01-27T02:41:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.551+0000 7f1f34e8e700 1 -- 172.21.15.7:0/3720593107 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2000bd60 con 0x7f1f300c4cc0 2024-01-27T02:41:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.552+0000 7f1f380f4700 1 -- 172.21.15.7:0/3720593107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f300c4cc0 msgr2=0x7f1f300cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.552+0000 7f1f380f4700 1 --2- 172.21.15.7:0/3720593107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f300c4cc0 0x7f1f300cd2f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f1f30069b70 tx=0x7f1f20009250 comp rx=0 tx=0).stop 2024-01-27T02:41:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.552+0000 7f1f380f4700 1 -- 172.21.15.7:0/3720593107 shutdown_connections 2024-01-27T02:41:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.552+0000 7f1f380f4700 1 --2- 172.21.15.7:0/3720593107 >> v2:172.21.15.188:3300/0 conn(0x7f1f300ca540 0x7f1f30163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.552+0000 7f1f380f4700 1 --2- 172.21.15.7:0/3720593107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f300c4cc0 0x7f1f300cd2f0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.552+0000 7f1f380f4700 1 -- 172.21.15.7:0/3720593107 >> 172.21.15.7:0/3720593107 conn(0x7f1f30098020 msgr2=0x7f1f300711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.552+0000 7f1f380f4700 1 -- 172.21.15.7:0/3720593107 shutdown_connections 2024-01-27T02:41:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.552+0000 7f1f380f4700 1 -- 172.21.15.7:0/3720593107 wait complete. 2024-01-27T02:41:55.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.553+0000 7f1f380f4700 1 Processor -- start 2024-01-27T02:41:55.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.553+0000 7f1f380f4700 1 -- start start 2024-01-27T02:41:55.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.554+0000 7f1f380f4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f300c4cc0 0x7f1f300b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:55.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.554+0000 7f1f380f4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f300ca540 0x7f1f3015c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:55.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.554+0000 7f1f380f4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f3015a100 0x7f1f3015ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:55.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.554+0000 7f1f380f4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3016b220 con 0x7f1f300ca540 2024-01-27T02:41:55.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.554+0000 7f1f380f4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f30151d60 con 0x7f1f300c4cc0 2024-01-27T02:41:55.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.554+0000 7f1f380f4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3015d130 con 0x7f1f3015a100 2024-01-27T02:41:55.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.555+0000 7f1f3568f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f300ca540 0x7f1f3015c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:55.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.555+0000 7f1f3568f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f300ca540 0x7f1f3015c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43062/0 (socket says 172.21.15.7:43062) 2024-01-27T02:41:55.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.555+0000 7f1f3568f700 1 -- 172.21.15.7:0/3994629029 learned_addr learned my addr 172.21.15.7:0/3994629029 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:55.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.555+0000 7f1f35e90700 1 --2- 172.21.15.7:0/3994629029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f300c4cc0 0x7f1f300b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:55.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.555+0000 7f1f36691700 1 --2- 172.21.15.7:0/3994629029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f3015a100 0x7f1f3015ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:55.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.555+0000 7f1f3568f700 1 -- 172.21.15.7:0/3994629029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f3015a100 msgr2=0x7f1f3015ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:55.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.555+0000 7f1f3568f700 1 --2- 172.21.15.7:0/3994629029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f3015a100 0x7f1f3015ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.555+0000 7f1f3568f700 1 -- 172.21.15.7:0/3994629029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f300c4cc0 msgr2=0x7f1f300b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:55.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.555+0000 7f1f3568f700 1 --2- 172.21.15.7:0/3994629029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f300c4cc0 0x7f1f300b25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.555+0000 7f1f3568f700 1 -- 172.21.15.7:0/3994629029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f200094f0 con 0x7f1f300ca540 2024-01-27T02:41:55.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.555+0000 7f1f35e90700 1 --2- 172.21.15.7:0/3994629029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f300c4cc0 0x7f1f300b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:41:55.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.555+0000 7f1f3568f700 1 --2- 172.21.15.7:0/3994629029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f300ca540 0x7f1f3015c550 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f1f20006f50 tx=0x7f1f20017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:55.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.555+0000 7f1f25ffb700 1 -- 172.21.15.7:0/3994629029 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f20024440 con 0x7f1f300ca540 2024-01-27T02:41:55.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.555+0000 7f1f25ffb700 1 -- 172.21.15.7:0/3994629029 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f200245e0 con 0x7f1f300ca540 2024-01-27T02:41:55.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.555+0000 7f1f380f4700 1 -- 172.21.15.7:0/3994629029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3016a4a0 con 0x7f1f300ca540 2024-01-27T02:41:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.555+0000 7f1f25ffb700 1 -- 172.21.15.7:0/3994629029 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f200249a0 con 0x7f1f300ca540 2024-01-27T02:41:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.556+0000 7f1f380f4700 1 -- 172.21.15.7:0/3994629029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3016a740 con 0x7f1f300ca540 2024-01-27T02:41:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.556+0000 7f1f380f4700 1 -- 172.21.15.7:0/3994629029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3004c710 con 0x7f1f300ca540 2024-01-27T02:41:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.557+0000 7f1f25ffb700 1 -- 172.21.15.7:0/3994629029 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1f2001a420 con 0x7f1f300ca540 2024-01-27T02:41:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.557+0000 7f1f25ffb700 1 --2- 172.21.15.7:0/3994629029 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1f14066f30 0x7f1f140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.557+0000 7f1f25ffb700 1 -- 172.21.15.7:0/3994629029 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1f20086120 con 0x7f1f300ca540 2024-01-27T02:41:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.558+0000 7f1f35e90700 1 --2- 172.21.15.7:0/3994629029 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1f14066f30 0x7f1f140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.558+0000 7f1f35e90700 1 --2- 172.21.15.7:0/3994629029 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1f14066f30 0x7f1f140693e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c001810 tx=0x7f1f2c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:55.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.562+0000 7f1f25ffb700 1 -- 172.21.15.7:0/3994629029 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f200536e0 con 0x7f1f300ca540 2024-01-27T02:41:55.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.775+0000 7f1f380f4700 1 -- 172.21.15.7:0/3994629029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1f300ce270 con 0x7f1f300ca540 2024-01-27T02:41:55.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.775+0000 7f1f25ffb700 1 -- 172.21.15.7:0/3994629029 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v160) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1f20056d00 con 0x7f1f300ca540 2024-01-27T02:41:55.776 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:41:55.776 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-27T02:41:55.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:41:55.776 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-27T02:41:55.777 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-27T02:41:55.777 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:41:55.777 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-27T02:41:55.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.778+0000 7f1f380f4700 1 -- 172.21.15.7:0/3994629029 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1f14066f30 msgr2=0x7f1f140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:55.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.778+0000 7f1f380f4700 1 --2- 172.21.15.7:0/3994629029 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1f14066f30 0x7f1f140693e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c001810 tx=0x7f1f2c009040 comp rx=0 tx=0).stop 2024-01-27T02:41:55.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.778+0000 7f1f380f4700 1 -- 172.21.15.7:0/3994629029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f300ca540 msgr2=0x7f1f3015c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:55.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.778+0000 7f1f380f4700 1 --2- 172.21.15.7:0/3994629029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f300ca540 0x7f1f3015c550 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f1f20006f50 tx=0x7f1f20017000 comp rx=0 tx=0).stop 2024-01-27T02:41:55.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.778+0000 7f1f380f4700 1 -- 172.21.15.7:0/3994629029 shutdown_connections 2024-01-27T02:41:55.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.778+0000 7f1f380f4700 1 --2- 172.21.15.7:0/3994629029 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1f14066f30 0x7f1f140693e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.778+0000 7f1f380f4700 1 --2- 172.21.15.7:0/3994629029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f3015a100 0x7f1f3015ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.778+0000 7f1f380f4700 1 --2- 172.21.15.7:0/3994629029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f300c4cc0 0x7f1f300b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.778+0000 7f1f380f4700 1 --2- 172.21.15.7:0/3994629029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f300ca540 0x7f1f3015c550 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.778+0000 7f1f380f4700 1 -- 172.21.15.7:0/3994629029 >> 172.21.15.7:0/3994629029 conn(0x7f1f30098020 msgr2=0x7f1f30154d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:55.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.778+0000 7f1f380f4700 1 -- 172.21.15.7:0/3994629029 shutdown_connections 2024-01-27T02:41:55.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.778+0000 7f1f380f4700 1 -- 172.21.15.7:0/3994629029 wait complete. 2024-01-27T02:41:55.792 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-27T02:41:55.792 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-27T02:41:55.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.928+0000 7f91ce2b8700 1 Processor -- start 2024-01-27T02:41:55.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.928+0000 7f91ce2b8700 1 -- start start 2024-01-27T02:41:55.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.929+0000 7f91ce2b8700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f91c815d3f0 0x7f91c80cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.929+0000 7f91ce2b8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c80cc1e0 con 0x7f91c815d7c0 2024-01-27T02:41:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.929+0000 7f91ce2b8700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c80cc4c0 con 0x7f91c80cbd40 2024-01-27T02:41:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.929+0000 7f91ce2b8700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91c80cc7a0 con 0x7f91c815d3f0 2024-01-27T02:41:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.929+0000 7f91cc855700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f91c815d7c0 0x7f91c80aacb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39664/0 (socket says 172.21.15.7:39664) 2024-01-27T02:41:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.929+0000 7f91cc855700 1 -- 172.21.15.7:0/4145540712 learned_addr learned my addr 172.21.15.7:0/4145540712 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:41:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.929+0000 7f91c7fff700 1 --2- 172.21.15.7:0/4145540712 >> v2:172.21.15.188:3300/0 conn(0x7f91c815d3f0 0x7f91c80cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c6ffd700 1 -- 172.21.15.7:0/4145540712 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2615494553 0 0) 0x7f91c80cc1e0 con 0x7f91c815d7c0 2024-01-27T02:41:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c6ffd700 1 -- 172.21.15.7:0/4145540712 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91b0003570 con 0x7f91c815d7c0 2024-01-27T02:41:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c6ffd700 1 -- 172.21.15.7:0/4145540712 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175754177 0 0) 0x7f91c80cc4c0 con 0x7f91c80cbd40 2024-01-27T02:41:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c6ffd700 1 -- 172.21.15.7:0/4145540712 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91c80cc1e0 con 0x7f91c80cbd40 2024-01-27T02:41:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c6ffd700 1 -- 172.21.15.7:0/4145540712 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4050886357 0 0) 0x7f91b0003570 con 0x7f91c815d7c0 2024-01-27T02:41:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c6ffd700 1 -- 172.21.15.7:0/4145540712 >> v2:172.21.15.188:3300/0 conn(0x7f91c815d3f0 msgr2=0x7f91c80cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c6ffd700 1 --2- 172.21.15.7:0/4145540712 >> v2:172.21.15.188:3300/0 conn(0x7f91c815d3f0 0x7f91c80cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c6ffd700 1 -- 172.21.15.7:0/4145540712 >> v1:172.21.15.188:6789/0 conn(0x7f91c80cbd40 legacy=0x7f91c8063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c6ffd700 1 -- 172.21.15.7:0/4145540712 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c80cca80 con 0x7f91c815d7c0 2024-01-27T02:41:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c6ffd700 1 -- 172.21.15.7:0/4145540712 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f91c00017f0 con 0x7f91c815d7c0 2024-01-27T02:41:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c6ffd700 1 -- 172.21.15.7:0/4145540712 >> v1:172.21.15.7:6789/0 conn(0x7f91c815d7c0 legacy=0x7f91c80aacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c6ffd700 1 --2- 172.21.15.7:0/4145540712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b0003bf0 0x7f91b0003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c6ffd700 1 --2- 172.21.15.7:0/4145540712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91b0004810 0x7f91b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c6ffd700 1 --2- 172.21.15.7:0/4145540712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91b0005230 0x7f91b000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c6ffd700 1 -- 172.21.15.7:0/4145540712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c80cc7a0 con 0x7f91b0003bf0 2024-01-27T02:41:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c6ffd700 1 -- 172.21.15.7:0/4145540712 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b000c200 con 0x7f91b0004810 2024-01-27T02:41:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c6ffd700 1 -- 172.21.15.7:0/4145540712 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b000c3c0 con 0x7f91b0005230 2024-01-27T02:41:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c7fff700 1 --2- 172.21.15.7:0/4145540712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b0003bf0 0x7f91b0003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c77fe700 1 --2- 172.21.15.7:0/4145540712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91b0004810 0x7f91b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91cc855700 1 --2- 172.21.15.7:0/4145540712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91b0005230 0x7f91b000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c7fff700 1 -- 172.21.15.7:0/4145540712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91b0005230 msgr2=0x7f91b000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c7fff700 1 --2- 172.21.15.7:0/4145540712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91b0005230 0x7f91b000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c7fff700 1 -- 172.21.15.7:0/4145540712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91b0004810 msgr2=0x7f91b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c7fff700 1 --2- 172.21.15.7:0/4145540712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91b0004810 0x7f91b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91c7fff700 1 -- 172.21.15.7:0/4145540712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b0003570 con 0x7f91b0003bf0 2024-01-27T02:41:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.930+0000 7f91cc855700 1 --2- 172.21.15.7:0/4145540712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91b0005230 0x7f91b000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:41:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.931+0000 7f91cc855700 1 -- 172.21.15.7:0/4145540712 reap_dead start 2024-01-27T02:41:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.931+0000 7f91c7fff700 1 --2- 172.21.15.7:0/4145540712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b0003bf0 0x7f91b0003fe0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009c00 tx=0x7f91b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4476f081a77516c2 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.931+0000 7f91c6ffd700 1 -- 172.21.15.7:0/4145540712 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b800e710 con 0x7f91b0003bf0 2024-01-27T02:41:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.931+0000 7f91c6ffd700 1 -- 172.21.15.7:0/4145540712 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b800e870 con 0x7f91b0003bf0 2024-01-27T02:41:55.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.931+0000 7f91c6ffd700 1 -- 172.21.15.7:0/4145540712 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b800ec30 con 0x7f91b0003bf0 2024-01-27T02:41:55.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.931+0000 7f91ce2b8700 1 -- 172.21.15.7:0/4145540712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b0003bf0 msgr2=0x7f91b0003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:55.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.931+0000 7f91ce2b8700 1 --2- 172.21.15.7:0/4145540712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b0003bf0 0x7f91b0003fe0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009c00 tx=0x7f91b80091a0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.931+0000 7f91ce2b8700 1 -- 172.21.15.7:0/4145540712 shutdown_connections 2024-01-27T02:41:55.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.931+0000 7f91ce2b8700 1 --2- 172.21.15.7:0/4145540712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b0003bf0 0x7f91b0003fe0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.931+0000 7f91ce2b8700 1 -- 172.21.15.7:0/4145540712 >> 172.21.15.7:0/4145540712 conn(0x7f91c80cdc30 msgr2=0x7f91c80ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:55.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.932+0000 7f91ce2b8700 1 -- 172.21.15.7:0/4145540712 shutdown_connections 2024-01-27T02:41:55.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.932+0000 7f91ce2b8700 1 -- 172.21.15.7:0/4145540712 wait complete. 2024-01-27T02:41:55.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.933+0000 7f91ce2b8700 1 Processor -- start 2024-01-27T02:41:55.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.933+0000 7f91ce2b8700 1 -- start start 2024-01-27T02:41:55.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.934+0000 7f91ce2b8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91b0003bf0 0x7f91c8159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:55.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.934+0000 7f91ce2b8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c8152450 0x7f91c81528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:55.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.934+0000 7f91ce2b8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91c80acdb0 0x7f91c80ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.934+0000 7f91ce2b8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c806b1e0 con 0x7f91c8152450 2024-01-27T02:41:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.934+0000 7f91ce2b8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c8152e00 con 0x7f91b0003bf0 2024-01-27T02:41:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.934+0000 7f91ce2b8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c80adaa0 con 0x7f91c80acdb0 2024-01-27T02:41:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.934+0000 7f91c77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c8152450 0x7f91c81528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.934+0000 7f91c77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c8152450 0x7f91c81528c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43086/0 (socket says 172.21.15.7:43086) 2024-01-27T02:41:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.934+0000 7f91c77fe700 1 -- 172.21.15.7:0/2354869337 learned_addr learned my addr 172.21.15.7:0/2354869337 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.934+0000 7f91cc855700 1 --2- 172.21.15.7:0/2354869337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91c80acdb0 0x7f91c80ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.934+0000 7f91c7fff700 1 --2- 172.21.15.7:0/2354869337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91b0003bf0 0x7f91c8159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.934+0000 7f91c77fe700 1 -- 172.21.15.7:0/2354869337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91c80acdb0 msgr2=0x7f91c80ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.934+0000 7f91c77fe700 1 --2- 172.21.15.7:0/2354869337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91c80acdb0 0x7f91c80ad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.934+0000 7f91c77fe700 1 -- 172.21.15.7:0/2354869337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91b0003bf0 msgr2=0x7f91c8159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.934+0000 7f91c77fe700 1 --2- 172.21.15.7:0/2354869337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91b0003bf0 0x7f91c8159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.934+0000 7f91c77fe700 1 -- 172.21.15.7:0/2354869337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c80add80 con 0x7f91c8152450 2024-01-27T02:41:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.934+0000 7f91c7fff700 1 --2- 172.21.15.7:0/2354869337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91b0003bf0 0x7f91c8159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:41:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.934+0000 7f91cc855700 1 --2- 172.21.15.7:0/2354869337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91c80acdb0 0x7f91c80ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:41:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.934+0000 7f91c77fe700 1 --2- 172.21.15.7:0/2354869337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c8152450 0x7f91c81528c0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f91bc002ab0 tx=0x7f91bc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:55.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.935+0000 7f91affff700 1 -- 172.21.15.7:0/2354869337 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc00f9a0 con 0x7f91c8152450 2024-01-27T02:41:55.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.935+0000 7f91affff700 1 -- 172.21.15.7:0/2354869337 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91bc00fb00 con 0x7f91c8152450 2024-01-27T02:41:55.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.935+0000 7f91ce2b8700 1 -- 172.21.15.7:0/2354869337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c80ae060 con 0x7f91c8152450 2024-01-27T02:41:55.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.935+0000 7f91affff700 1 -- 172.21.15.7:0/2354869337 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc00fec0 con 0x7f91c8152450 2024-01-27T02:41:55.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.935+0000 7f91ce2b8700 1 -- 172.21.15.7:0/2354869337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c80ae340 con 0x7f91c8152450 2024-01-27T02:41:55.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.936+0000 7f91ce2b8700 1 -- 172.21.15.7:0/2354869337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c806b390 con 0x7f91c8152450 2024-01-27T02:41:55.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.936+0000 7f91affff700 1 -- 172.21.15.7:0/2354869337 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f91bc013600 con 0x7f91c8152450 2024-01-27T02:41:55.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.936+0000 7f91affff700 1 --2- 172.21.15.7:0/2354869337 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f91b4066f30 0x7f91b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:55.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.937+0000 7f91affff700 1 -- 172.21.15.7:0/2354869337 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f91bc084850 con 0x7f91c8152450 2024-01-27T02:41:55.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.937+0000 7f91c7fff700 1 --2- 172.21.15.7:0/2354869337 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f91b4066f30 0x7f91b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:55.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.937+0000 7f91c7fff700 1 --2- 172.21.15.7:0/2354869337 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f91b4066f30 0x7f91b40693e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f91b80091e0 tx=0x7f91b801a000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:55.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:55.941+0000 7f91affff700 1 -- 172.21.15.7:0/2354869337 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91c806b390 con 0x7f91c8152450 2024-01-27T02:41:56.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.143+0000 7f91ce2b8700 1 -- 172.21.15.7:0/2354869337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f91c806b390 con 0x7f91c8152450 2024-01-27T02:41:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.210+0000 7f91affff700 1 -- 172.21.15.7:0/2354869337 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v161) v1 ==== 138+0+0 (secure 0 0 0) 0x7f91c806b390 con 0x7f91c8152450 2024-01-27T02:41:56.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.214+0000 7f91ce2b8700 1 -- 172.21.15.7:0/2354869337 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f91b4066f30 msgr2=0x7f91b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.214+0000 7f91ce2b8700 1 --2- 172.21.15.7:0/2354869337 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f91b4066f30 0x7f91b40693e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f91b80091e0 tx=0x7f91b801a000 comp rx=0 tx=0).stop 2024-01-27T02:41:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.214+0000 7f91ce2b8700 1 -- 172.21.15.7:0/2354869337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c8152450 msgr2=0x7f91c81528c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.214+0000 7f91ce2b8700 1 --2- 172.21.15.7:0/2354869337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c8152450 0x7f91c81528c0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f91bc002ab0 tx=0x7f91bc007d30 comp rx=0 tx=0).stop 2024-01-27T02:41:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.215+0000 7f91ce2b8700 1 -- 172.21.15.7:0/2354869337 shutdown_connections 2024-01-27T02:41:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.215+0000 7f91ce2b8700 1 --2- 172.21.15.7:0/2354869337 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f91b4066f30 0x7f91b40693e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.215+0000 7f91ce2b8700 1 --2- 172.21.15.7:0/2354869337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91c80acdb0 0x7f91c80ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.215+0000 7f91ce2b8700 1 --2- 172.21.15.7:0/2354869337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91b0003bf0 0x7f91c8159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.215+0000 7f91ce2b8700 1 --2- 172.21.15.7:0/2354869337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91c8152450 0x7f91c81528c0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.215+0000 7f91ce2b8700 1 -- 172.21.15.7:0/2354869337 >> 172.21.15.7:0/2354869337 conn(0x7f91c80cdc30 msgr2=0x7f91c815b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.215+0000 7f91ce2b8700 1 -- 172.21.15.7:0/2354869337 shutdown_connections 2024-01-27T02:41:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.215+0000 7f91ce2b8700 1 -- 172.21.15.7:0/2354869337 wait complete. 2024-01-27T02:41:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-27T02:41:56.229 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-27T02:41:56.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.368+0000 7f5fdaab7700 1 Processor -- start 2024-01-27T02:41:56.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.369+0000 7f5fdaab7700 1 -- start start 2024-01-27T02:41:56.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.369+0000 7f5fdaab7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5fd415d460 0x7f5fd40d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:56.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.369+0000 7f5fdaab7700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5fd40cf280 0x7f5fd415e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:56.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.369+0000 7f5fdaab7700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd40b38d0 con 0x7f5fd415d830 2024-01-27T02:41:56.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.369+0000 7f5fdaab7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd40d0390 con 0x7f5fd415d460 2024-01-27T02:41:56.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.369+0000 7f5fdaab7700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd40cf780 con 0x7f5fd40cf280 2024-01-27T02:41:56.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.370+0000 7f5fd8853700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5fd415d460 0x7f5fd40d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:56.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.370+0000 7f5fd8853700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5fd415d460 0x7f5fd40d0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43098/0 (socket says 172.21.15.7:43098) 2024-01-27T02:41:56.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.370+0000 7f5fd8853700 1 -- 172.21.15.7:0/2477354698 learned_addr learned my addr 172.21.15.7:0/2477354698 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:56.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.370+0000 7f5fd3fff700 1 --2- 172.21.15.7:0/2477354698 >> v2:172.21.15.188:3300/0 conn(0x7f5fd40cf280 0x7f5fd415e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:56.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.370+0000 7f5fd8853700 1 -- 172.21.15.7:0/2477354698 >> v2:172.21.15.188:3300/0 conn(0x7f5fd40cf280 msgr2=0x7f5fd415e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:56.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.370+0000 7f5fd8853700 1 --2- 172.21.15.7:0/2477354698 >> v2:172.21.15.188:3300/0 conn(0x7f5fd40cf280 0x7f5fd415e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:56.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.370+0000 7f5fd8853700 1 -- 172.21.15.7:0/2477354698 >> v1:172.21.15.188:6789/0 conn(0x7f5fd415d830 legacy=0x7f5fd40cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:56.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.370+0000 7f5fd8853700 1 -- 172.21.15.7:0/2477354698 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd40cfa60 con 0x7f5fd415d460 2024-01-27T02:41:56.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.370+0000 7f5fd3fff700 1 --2- 172.21.15.7:0/2477354698 >> v2:172.21.15.188:3300/0 conn(0x7f5fd40cf280 0x7f5fd415e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:41:56.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.370+0000 7f5fd8853700 1 --2- 172.21.15.7:0/2477354698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd415d460 0x7f5fd40d0820 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f5fd408a1e0 tx=0x7f5fc8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60da25e12072c1a1 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:56.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.370+0000 7f5fd37fe700 1 -- 172.21.15.7:0/2477354698 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc800b820 con 0x7f5fd415d460 2024-01-27T02:41:56.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.370+0000 7f5fd37fe700 1 -- 172.21.15.7:0/2477354698 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc800b980 con 0x7f5fd415d460 2024-01-27T02:41:56.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.370+0000 7f5fd37fe700 1 -- 172.21.15.7:0/2477354698 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc800bd20 con 0x7f5fd415d460 2024-01-27T02:41:56.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.371+0000 7f5fdaab7700 1 -- 172.21.15.7:0/2477354698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd415d460 msgr2=0x7f5fd40d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:56.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.371+0000 7f5fdaab7700 1 --2- 172.21.15.7:0/2477354698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd415d460 0x7f5fd40d0820 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f5fd408a1e0 tx=0x7f5fc8009250 comp rx=0 tx=0).stop 2024-01-27T02:41:56.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.371+0000 7f5fdaab7700 1 -- 172.21.15.7:0/2477354698 shutdown_connections 2024-01-27T02:41:56.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.371+0000 7f5fdaab7700 1 --2- 172.21.15.7:0/2477354698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd415d460 0x7f5fd40d0820 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:56.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.371+0000 7f5fdaab7700 1 --2- 172.21.15.7:0/2477354698 >> v2:172.21.15.188:3300/0 conn(0x7f5fd40cf280 0x7f5fd415e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:56.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.371+0000 7f5fdaab7700 1 -- 172.21.15.7:0/2477354698 >> 172.21.15.7:0/2477354698 conn(0x7f5fd4063580 msgr2=0x7f5fd4063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:56.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.371+0000 7f5fdaab7700 1 -- 172.21.15.7:0/2477354698 shutdown_connections 2024-01-27T02:41:56.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.371+0000 7f5fdaab7700 1 -- 172.21.15.7:0/2477354698 wait complete. 2024-01-27T02:41:56.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.373+0000 7f5fdaab7700 1 Processor -- start 2024-01-27T02:41:56.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.373+0000 7f5fdaab7700 1 -- start start 2024-01-27T02:41:56.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.374+0000 7f5fdaab7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd40cf280 0x7f5fd40c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:56.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.374+0000 7f5fdaab7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd415d460 0x7f5fd41525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:56.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.374+0000 7f5fdaab7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd415d830 0x7f5fd40cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:56.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.374+0000 7f5fdaab7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd4152b30 con 0x7f5fd415d830 2024-01-27T02:41:56.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.374+0000 7f5fdaab7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd40b5060 con 0x7f5fd40cf280 2024-01-27T02:41:56.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.374+0000 7f5fdaab7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd40cb1f0 con 0x7f5fd415d460 2024-01-27T02:41:56.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.374+0000 7f5fd9054700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd415d830 0x7f5fd40cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:56.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.374+0000 7f5fd9054700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd415d830 0x7f5fd40cacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43112/0 (socket says 172.21.15.7:43112) 2024-01-27T02:41:56.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.374+0000 7f5fd9054700 1 -- 172.21.15.7:0/2566661134 learned_addr learned my addr 172.21.15.7:0/2566661134 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:56.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.374+0000 7f5fd3fff700 1 --2- 172.21.15.7:0/2566661134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd415d460 0x7f5fd41525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:56.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.374+0000 7f5fd8853700 1 --2- 172.21.15.7:0/2566661134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd40cf280 0x7f5fd40c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.375+0000 7f5fd9054700 1 -- 172.21.15.7:0/2566661134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd415d460 msgr2=0x7f5fd41525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.375+0000 7f5fd9054700 1 --2- 172.21.15.7:0/2566661134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd415d460 0x7f5fd41525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.375+0000 7f5fd9054700 1 -- 172.21.15.7:0/2566661134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd40cf280 msgr2=0x7f5fd40c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.375+0000 7f5fd9054700 1 --2- 172.21.15.7:0/2566661134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd40cf280 0x7f5fd40c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.375+0000 7f5fd9054700 1 -- 172.21.15.7:0/2566661134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc80094f0 con 0x7f5fd415d830 2024-01-27T02:41:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.375+0000 7f5fd8853700 1 --2- 172.21.15.7:0/2566661134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd40cf280 0x7f5fd40c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:41:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.375+0000 7f5fd9054700 1 --2- 172.21.15.7:0/2566661134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd415d830 0x7f5fd40cacb0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00bc60 tx=0x7f5fcc009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.375+0000 7f5fd0ff9700 1 -- 172.21.15.7:0/2566661134 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc00c9c0 con 0x7f5fd415d830 2024-01-27T02:41:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.375+0000 7f5fd0ff9700 1 -- 172.21.15.7:0/2566661134 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fcc00cb60 con 0x7f5fd415d830 2024-01-27T02:41:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.375+0000 7f5fdaab7700 1 -- 172.21.15.7:0/2566661134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd40b4750 con 0x7f5fd415d830 2024-01-27T02:41:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.375+0000 7f5fd0ff9700 1 -- 172.21.15.7:0/2566661134 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc014410 con 0x7f5fd415d830 2024-01-27T02:41:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.375+0000 7f5fdaab7700 1 -- 172.21.15.7:0/2566661134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd40b4a10 con 0x7f5fd415d830 2024-01-27T02:41:56.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.376+0000 7f5fdaab7700 1 -- 172.21.15.7:0/2566661134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd40b51f0 con 0x7f5fd415d830 2024-01-27T02:41:56.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.377+0000 7f5fd0ff9700 1 -- 172.21.15.7:0/2566661134 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5fcc014800 con 0x7f5fd415d830 2024-01-27T02:41:56.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.377+0000 7f5fd0ff9700 1 --2- 172.21.15.7:0/2566661134 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5fb8066f30 0x7f5fb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:56.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.377+0000 7f5fd0ff9700 1 -- 172.21.15.7:0/2566661134 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f5fcc0843d0 con 0x7f5fd415d830 2024-01-27T02:41:56.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.377+0000 7f5fd8853700 1 --2- 172.21.15.7:0/2566661134 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5fb8066f30 0x7f5fb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:56.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.378+0000 7f5fd8853700 1 --2- 172.21.15.7:0/2566661134 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5fb8066f30 0x7f5fb80693e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5fd408a1e0 tx=0x7f5fc8016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:56.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.382+0000 7f5fd0ff9700 1 -- 172.21.15.7:0/2566661134 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5fcc04c5a0 con 0x7f5fd415d830 2024-01-27T02:41:56.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:56.588+0000 7f5fdaab7700 1 -- 172.21.15.7:0/2566661134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5fd4072060 con 0x7f5fd415d830 2024-01-27T02:41:57.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.216+0000 7f5fd0ff9700 1 -- 172.21.15.7:0/2566661134 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v162) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5fcc0552b0 con 0x7f5fd415d830 2024-01-27T02:41:57.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.224+0000 7f5fdaab7700 1 -- 172.21.15.7:0/2566661134 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5fb8066f30 msgr2=0x7f5fb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:57.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.224+0000 7f5fdaab7700 1 --2- 172.21.15.7:0/2566661134 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5fb8066f30 0x7f5fb80693e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5fd408a1e0 tx=0x7f5fc8016000 comp rx=0 tx=0).stop 2024-01-27T02:41:57.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.224+0000 7f5fdaab7700 1 -- 172.21.15.7:0/2566661134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd415d830 msgr2=0x7f5fd40cacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:57.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.224+0000 7f5fdaab7700 1 --2- 172.21.15.7:0/2566661134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd415d830 0x7f5fd40cacb0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00bc60 tx=0x7f5fcc009820 comp rx=0 tx=0).stop 2024-01-27T02:41:57.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.227+0000 7f5fdaab7700 1 -- 172.21.15.7:0/2566661134 shutdown_connections 2024-01-27T02:41:57.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.227+0000 7f5fdaab7700 1 --2- 172.21.15.7:0/2566661134 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5fb8066f30 0x7f5fb80693e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:57.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.227+0000 7f5fdaab7700 1 --2- 172.21.15.7:0/2566661134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd415d830 0x7f5fd40cacb0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:57.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.227+0000 7f5fdaab7700 1 --2- 172.21.15.7:0/2566661134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd40cf280 0x7f5fd40c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:57.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.227+0000 7f5fdaab7700 1 --2- 172.21.15.7:0/2566661134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd415d460 0x7f5fd41525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:57.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.227+0000 7f5fdaab7700 1 -- 172.21.15.7:0/2566661134 >> 172.21.15.7:0/2566661134 conn(0x7f5fd4063580 msgr2=0x7f5fd415c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:57.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.231+0000 7f5fdaab7700 1 -- 172.21.15.7:0/2566661134 shutdown_connections 2024-01-27T02:41:57.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.231+0000 7f5fdaab7700 1 -- 172.21.15.7:0/2566661134 wait complete. 2024-01-27T02:41:57.233 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-27T02:41:57.247 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-27T02:41:57.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.384+0000 7f34a2e44700 1 Processor -- start 2024-01-27T02:41:57.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.385+0000 7f34a2e44700 1 -- start start 2024-01-27T02:41:57.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.385+0000 7f34a2e44700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f349c1416c0 0x7f349c0cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:57.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.385+0000 7f34a2e44700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349c141a90 0x7f349c0cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:57.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.385+0000 7f34a2e44700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f349c0cab00 0x7f349c0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:57.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.385+0000 7f34a2e44700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f349c15c7d0 con 0x7f349c1416c0 2024-01-27T02:41:57.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.385+0000 7f34a2e44700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f349c0cb480 con 0x7f349c0cab00 2024-01-27T02:41:57.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.385+0000 7f34a2e44700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349c0c9640 con 0x7f349c141a90 2024-01-27T02:41:57.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.386+0000 7f34a0be0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f349c1416c0 0x7f349c0cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:57.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.386+0000 7f34a0be0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f349c1416c0 0x7f349c0cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43126/0 (socket says 172.21.15.7:43126) 2024-01-27T02:41:57.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.386+0000 7f34a0be0700 1 -- 172.21.15.7:0/3829259093 learned_addr learned my addr 172.21.15.7:0/3829259093 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:57.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.386+0000 7f349bfff700 1 --2- 172.21.15.7:0/3829259093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349c141a90 0x7f349c0cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:57.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.386+0000 7f34a13e1700 1 --2- 172.21.15.7:0/3829259093 >> v2:172.21.15.188:3300/0 conn(0x7f349c0cab00 0x7f349c0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:57.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.386+0000 7f34a0be0700 1 -- 172.21.15.7:0/3829259093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349c141a90 msgr2=0x7f349c0cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:57.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.386+0000 7f34a0be0700 1 --2- 172.21.15.7:0/3829259093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349c141a90 0x7f349c0cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:57.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.386+0000 7f34a0be0700 1 -- 172.21.15.7:0/3829259093 >> v2:172.21.15.188:3300/0 conn(0x7f349c0cab00 msgr2=0x7f349c0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:57.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.386+0000 7f34a0be0700 1 --2- 172.21.15.7:0/3829259093 >> v2:172.21.15.188:3300/0 conn(0x7f349c0cab00 0x7f349c0caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:57.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.386+0000 7f34a0be0700 1 -- 172.21.15.7:0/3829259093 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349c15bbc0 con 0x7f349c1416c0 2024-01-27T02:41:57.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.386+0000 7f34a0be0700 1 --2- 172.21.15.7:0/3829259093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f349c1416c0 0x7f349c0cb6e0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f349c0c92b0 tx=0x7f3490009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c20b1c77c889b2 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:57.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.386+0000 7f349b7fe700 1 -- 172.21.15.7:0/3829259093 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3490014ca0 con 0x7f349c1416c0 2024-01-27T02:41:57.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.386+0000 7f349b7fe700 1 -- 172.21.15.7:0/3829259093 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3490014e00 con 0x7f349c1416c0 2024-01-27T02:41:57.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.387+0000 7f349b7fe700 1 -- 172.21.15.7:0/3829259093 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3490005c30 con 0x7f349c1416c0 2024-01-27T02:41:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.387+0000 7f34a2e44700 1 -- 172.21.15.7:0/3829259093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f349c1416c0 msgr2=0x7f349c0cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.387+0000 7f34a2e44700 1 --2- 172.21.15.7:0/3829259093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f349c1416c0 0x7f349c0cb6e0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f349c0c92b0 tx=0x7f3490009220 comp rx=0 tx=0).stop 2024-01-27T02:41:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.387+0000 7f34a2e44700 1 -- 172.21.15.7:0/3829259093 shutdown_connections 2024-01-27T02:41:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.387+0000 7f34a2e44700 1 --2- 172.21.15.7:0/3829259093 >> v2:172.21.15.188:3300/0 conn(0x7f349c0cab00 0x7f349c0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.387+0000 7f34a2e44700 1 --2- 172.21.15.7:0/3829259093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f349c1416c0 0x7f349c0cb6e0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.387+0000 7f34a2e44700 1 --2- 172.21.15.7:0/3829259093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349c141a90 0x7f349c0cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.387+0000 7f34a2e44700 1 -- 172.21.15.7:0/3829259093 >> 172.21.15.7:0/3829259093 conn(0x7f349c063580 msgr2=0x7f349c063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.387+0000 7f34a2e44700 1 -- 172.21.15.7:0/3829259093 shutdown_connections 2024-01-27T02:41:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.387+0000 7f34a2e44700 1 -- 172.21.15.7:0/3829259093 wait complete. 2024-01-27T02:41:57.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.388+0000 7f34a2e44700 1 Processor -- start 2024-01-27T02:41:57.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.389+0000 7f34a2e44700 1 -- start start 2024-01-27T02:41:57.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.389+0000 7f34a2e44700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f349c0cab00 0x7f349c0cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:57.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.389+0000 7f34a2e44700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f349c1416c0 0x7f349c0c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:57.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.389+0000 7f34a2e44700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349c141a90 0x7f349c13f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:57.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.389+0000 7f34a2e44700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349c15b210 con 0x7f349c0cab00 2024-01-27T02:41:57.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.389+0000 7f34a2e44700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349c15b4f0 con 0x7f349c1416c0 2024-01-27T02:41:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.389+0000 7f34a2e44700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349c0c7ef0 con 0x7f349c141a90 2024-01-27T02:41:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.390+0000 7f34a0be0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f349c0cab00 0x7f349c0cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.390+0000 7f34a0be0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f349c0cab00 0x7f349c0cd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43142/0 (socket says 172.21.15.7:43142) 2024-01-27T02:41:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.390+0000 7f34a0be0700 1 -- 172.21.15.7:0/109308804 learned_addr learned my addr 172.21.15.7:0/109308804 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.390+0000 7f34a13e1700 1 --2- 172.21.15.7:0/109308804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349c141a90 0x7f349c13f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.390+0000 7f349bfff700 1 --2- 172.21.15.7:0/109308804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f349c1416c0 0x7f349c0c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.390+0000 7f34a0be0700 1 -- 172.21.15.7:0/109308804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349c141a90 msgr2=0x7f349c13f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.390+0000 7f34a0be0700 1 --2- 172.21.15.7:0/109308804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349c141a90 0x7f349c13f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.390+0000 7f34a0be0700 1 -- 172.21.15.7:0/109308804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f349c1416c0 msgr2=0x7f349c0c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.390+0000 7f34a0be0700 1 --2- 172.21.15.7:0/109308804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f349c1416c0 0x7f349c0c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.390+0000 7f34a0be0700 1 -- 172.21.15.7:0/109308804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3490009480 con 0x7f349c0cab00 2024-01-27T02:41:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.390+0000 7f34a0be0700 1 --2- 172.21.15.7:0/109308804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f349c0cab00 0x7f349c0cd8d0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f3490007fd0 tx=0x7f349001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.390+0000 7f3498ff9700 1 -- 172.21.15.7:0/109308804 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3490005c10 con 0x7f349c0cab00 2024-01-27T02:41:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.390+0000 7f3498ff9700 1 -- 172.21.15.7:0/109308804 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3490026420 con 0x7f349c0cab00 2024-01-27T02:41:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.390+0000 7f34a2e44700 1 -- 172.21.15.7:0/109308804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349c0c81d0 con 0x7f349c0cab00 2024-01-27T02:41:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.390+0000 7f3498ff9700 1 -- 172.21.15.7:0/109308804 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34900267c0 con 0x7f349c0cab00 2024-01-27T02:41:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.390+0000 7f34a2e44700 1 -- 172.21.15.7:0/109308804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349c0cde10 con 0x7f349c0cab00 2024-01-27T02:41:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.391+0000 7f34a2e44700 1 -- 172.21.15.7:0/109308804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f349c15b710 con 0x7f349c0cab00 2024-01-27T02:41:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.392+0000 7f3498ff9700 1 -- 172.21.15.7:0/109308804 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3490026b70 con 0x7f349c0cab00 2024-01-27T02:41:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.392+0000 7f3498ff9700 1 --2- 172.21.15.7:0/109308804 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3480066f30 0x7f34800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.392+0000 7f3498ff9700 1 -- 172.21.15.7:0/109308804 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f349008e0e0 con 0x7f349c0cab00 2024-01-27T02:41:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.393+0000 7f349bfff700 1 --2- 172.21.15.7:0/109308804 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3480066f30 0x7f34800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:57.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.393+0000 7f349bfff700 1 --2- 172.21.15.7:0/109308804 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3480066f30 0x7f34800693e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f349c0cce00 tx=0x7f34880091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:57.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.397+0000 7f3498ff9700 1 -- 172.21.15.7:0/109308804 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f349005b3b0 con 0x7f349c0cab00 2024-01-27T02:41:57.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:57.644+0000 7f34a2e44700 1 -- 172.21.15.7:0/109308804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f349c072060 con 0x7f349c0cab00 2024-01-27T02:41:58.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.221+0000 7f3498ff9700 1 -- 172.21.15.7:0/109308804 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v82) v1 ==== 174+0+0 (secure 0 0 0) 0x7f349005e9d0 con 0x7f349c0cab00 2024-01-27T02:41:58.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.230+0000 7f34a2e44700 1 -- 172.21.15.7:0/109308804 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3480066f30 msgr2=0x7f34800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:58.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.230+0000 7f34a2e44700 1 --2- 172.21.15.7:0/109308804 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3480066f30 0x7f34800693e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f349c0cce00 tx=0x7f34880091a0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.230+0000 7f34a2e44700 1 -- 172.21.15.7:0/109308804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f349c0cab00 msgr2=0x7f349c0cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:58.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.230+0000 7f34a2e44700 1 --2- 172.21.15.7:0/109308804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f349c0cab00 0x7f349c0cd8d0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f3490007fd0 tx=0x7f349001b000 comp rx=0 tx=0).stop 2024-01-27T02:41:58.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.232+0000 7f34a2e44700 1 -- 172.21.15.7:0/109308804 shutdown_connections 2024-01-27T02:41:58.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.232+0000 7f34a2e44700 1 --2- 172.21.15.7:0/109308804 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3480066f30 0x7f34800693e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.232+0000 7f34a2e44700 1 --2- 172.21.15.7:0/109308804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349c141a90 0x7f349c13f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.232+0000 7f34a2e44700 1 --2- 172.21.15.7:0/109308804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f349c0cab00 0x7f349c0cd8d0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.232+0000 7f34a2e44700 1 --2- 172.21.15.7:0/109308804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f349c1416c0 0x7f349c0c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.232+0000 7f34a2e44700 1 -- 172.21.15.7:0/109308804 >> 172.21.15.7:0/109308804 conn(0x7f349c063580 msgr2=0x7f349c13ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:58.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.234+0000 7f34a2e44700 1 -- 172.21.15.7:0/109308804 shutdown_connections 2024-01-27T02:41:58.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.236+0000 7f34a2e44700 1 -- 172.21.15.7:0/109308804 wait complete. 2024-01-27T02:41:58.239 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 20 and data pool 21 2024-01-27T02:41:58.255 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:41:58.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.389+0000 7fbb87e0b700 1 Processor -- start 2024-01-27T02:41:58.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.389+0000 7fbb87e0b700 1 -- start start 2024-01-27T02:41:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.390+0000 7fbb87e0b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb8015a100 0x7fbb800cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.390+0000 7fbb87e0b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb800c4cc0 0x7fbb80163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.390+0000 7fbb87e0b700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb800ce5e0 con 0x7fbb800cb5b0 2024-01-27T02:41:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.390+0000 7fbb87e0b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb800cd9d0 con 0x7fbb800c4cc0 2024-01-27T02:41:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.390+0000 7fbb87e0b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb800cdcb0 con 0x7fbb8015a100 2024-01-27T02:41:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.390+0000 7fbb863a8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb800c4cc0 0x7fbb80163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.391+0000 7fbb863a8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb800c4cc0 0x7fbb80163d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43144/0 (socket says 172.21.15.7:43144) 2024-01-27T02:41:58.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.391+0000 7fbb863a8700 1 -- 172.21.15.7:0/267466220 learned_addr learned my addr 172.21.15.7:0/267466220 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:58.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.391+0000 7fbb85ba7700 1 --2- 172.21.15.7:0/267466220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb8015a100 0x7fbb800cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:58.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.391+0000 7fbb863a8700 1 -- 172.21.15.7:0/267466220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb8015a100 msgr2=0x7fbb800cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:58.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.391+0000 7fbb863a8700 1 --2- 172.21.15.7:0/267466220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb8015a100 0x7fbb800cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.391+0000 7fbb863a8700 1 -- 172.21.15.7:0/267466220 >> v1:172.21.15.188:6789/0 conn(0x7fbb800cb5b0 legacy=0x7fbb800c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:58.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.391+0000 7fbb863a8700 1 -- 172.21.15.7:0/267466220 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb800cdf90 con 0x7fbb800c4cc0 2024-01-27T02:41:58.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.391+0000 7fbb863a8700 1 --2- 172.21.15.7:0/267466220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb800c4cc0 0x7fbb80163d90 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fbb78009f20 tx=0x7fbb78009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a9362779a29509 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:58.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.391+0000 7fbb84ba5700 1 -- 172.21.15.7:0/267466220 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7800e070 con 0x7fbb800c4cc0 2024-01-27T02:41:58.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.391+0000 7fbb84ba5700 1 -- 172.21.15.7:0/267466220 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7800ba80 con 0x7fbb800c4cc0 2024-01-27T02:41:58.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.391+0000 7fbb84ba5700 1 -- 172.21.15.7:0/267466220 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb780092e0 con 0x7fbb800c4cc0 2024-01-27T02:41:58.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.392+0000 7fbb87e0b700 1 -- 172.21.15.7:0/267466220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb800c4cc0 msgr2=0x7fbb80163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:58.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.392+0000 7fbb87e0b700 1 --2- 172.21.15.7:0/267466220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb800c4cc0 0x7fbb80163d90 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fbb78009f20 tx=0x7fbb78009840 comp rx=0 tx=0).stop 2024-01-27T02:41:58.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.392+0000 7fbb87e0b700 1 -- 172.21.15.7:0/267466220 shutdown_connections 2024-01-27T02:41:58.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.392+0000 7fbb87e0b700 1 --2- 172.21.15.7:0/267466220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb800c4cc0 0x7fbb80163d90 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.392+0000 7fbb87e0b700 1 --2- 172.21.15.7:0/267466220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb8015a100 0x7fbb800cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.392+0000 7fbb87e0b700 1 -- 172.21.15.7:0/267466220 >> 172.21.15.7:0/267466220 conn(0x7fbb80098020 msgr2=0x7fbb800711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:58.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.392+0000 7fbb87e0b700 1 -- 172.21.15.7:0/267466220 shutdown_connections 2024-01-27T02:41:58.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.392+0000 7fbb87e0b700 1 -- 172.21.15.7:0/267466220 wait complete. 2024-01-27T02:41:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.394+0000 7fbb87e0b700 1 Processor -- start 2024-01-27T02:41:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.394+0000 7fbb87e0b700 1 -- start start 2024-01-27T02:41:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.395+0000 7fbb87e0b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800c4cc0 0x7fbb800c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.395+0000 7fbb87e0b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb800cb5b0 0x7fbb80153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.395+0000 7fbb87e0b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb8015a100 0x7fbb80169ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.395+0000 7fbb87e0b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb801577d0 con 0x7fbb8015a100 2024-01-27T02:41:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.395+0000 7fbb87e0b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb800c7180 con 0x7fbb800cb5b0 2024-01-27T02:41:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.395+0000 7fbb87e0b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb80157a20 con 0x7fbb800c4cc0 2024-01-27T02:41:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.395+0000 7fbb863a8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb8015a100 0x7fbb80169ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.395+0000 7fbb863a8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb8015a100 0x7fbb80169ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43156/0 (socket says 172.21.15.7:43156) 2024-01-27T02:41:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.395+0000 7fbb863a8700 1 -- 172.21.15.7:0/3304847940 learned_addr learned my addr 172.21.15.7:0/3304847940 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.395+0000 7fbb85ba7700 1 --2- 172.21.15.7:0/3304847940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800c4cc0 0x7fbb800c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.395+0000 7fbb853a6700 1 --2- 172.21.15.7:0/3304847940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb800cb5b0 0x7fbb80153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.395+0000 7fbb863a8700 1 -- 172.21.15.7:0/3304847940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800c4cc0 msgr2=0x7fbb800c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.395+0000 7fbb863a8700 1 --2- 172.21.15.7:0/3304847940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800c4cc0 0x7fbb800c6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.395+0000 7fbb863a8700 1 -- 172.21.15.7:0/3304847940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb800cb5b0 msgr2=0x7fbb80153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.395+0000 7fbb863a8700 1 --2- 172.21.15.7:0/3304847940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb800cb5b0 0x7fbb80153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.395+0000 7fbb863a8700 1 -- 172.21.15.7:0/3304847940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb78009ae0 con 0x7fbb8015a100 2024-01-27T02:41:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.395+0000 7fbb85ba7700 1 --2- 172.21.15.7:0/3304847940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800c4cc0 0x7fbb800c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:41:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.395+0000 7fbb863a8700 1 --2- 172.21.15.7:0/3304847940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb8015a100 0x7fbb80169ae0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fbb8006ea80 tx=0x7fbb78018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.396+0000 7fbb75ffb700 1 -- 172.21.15.7:0/3304847940 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb780094e0 con 0x7fbb8015a100 2024-01-27T02:41:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.396+0000 7fbb75ffb700 1 -- 172.21.15.7:0/3304847940 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb78008e80 con 0x7fbb8015a100 2024-01-27T02:41:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.396+0000 7fbb87e0b700 1 -- 172.21.15.7:0/3304847940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb80157d00 con 0x7fbb8015a100 2024-01-27T02:41:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.396+0000 7fbb75ffb700 1 -- 172.21.15.7:0/3304847940 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb78026690 con 0x7fbb8015a100 2024-01-27T02:41:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.396+0000 7fbb87e0b700 1 -- 172.21.15.7:0/3304847940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb801582c0 con 0x7fbb8015a100 2024-01-27T02:41:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.397+0000 7fbb87e0b700 1 -- 172.21.15.7:0/3304847940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8004c710 con 0x7fbb8015a100 2024-01-27T02:41:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.397+0000 7fbb75ffb700 1 -- 172.21.15.7:0/3304847940 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbb78003040 con 0x7fbb8015a100 2024-01-27T02:41:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.398+0000 7fbb75ffb700 1 --2- 172.21.15.7:0/3304847940 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb64066f30 0x7fbb640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.398+0000 7fbb75ffb700 1 -- 172.21.15.7:0/3304847940 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbb78086560 con 0x7fbb8015a100 2024-01-27T02:41:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.398+0000 7fbb85ba7700 1 --2- 172.21.15.7:0/3304847940 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb64066f30 0x7fbb640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.399+0000 7fbb85ba7700 1 --2- 172.21.15.7:0/3304847940 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb64066f30 0x7fbb640693e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c006fd0 tx=0x7fbb7c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:58.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.402+0000 7fbb75ffb700 1 -- 172.21.15.7:0/3304847940 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb78053880 con 0x7fbb8015a100 2024-01-27T02:41:58.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.609+0000 7fbb87e0b700 1 -- 172.21.15.7:0/3304847940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb800ce270 con 0x7fbb8015a100 2024-01-27T02:41:58.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.610+0000 7fbb75ffb700 1 -- 172.21.15.7:0/3304847940 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+15330 (secure 0 0 0) 0x7fbb78056ea0 con 0x7fbb8015a100 2024-01-27T02:41:58.610 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:41:58.610 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":163,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:41:57.645686+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:41:56.176076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-01-27T02:41:56.620523+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:41:58.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.612+0000 7fbb87e0b700 1 -- 172.21.15.7:0/3304847940 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb64066f30 msgr2=0x7fbb640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:58.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.612+0000 7fbb87e0b700 1 --2- 172.21.15.7:0/3304847940 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb64066f30 0x7fbb640693e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c006fd0 tx=0x7fbb7c008040 comp rx=0 tx=0).stop 2024-01-27T02:41:58.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.612+0000 7fbb87e0b700 1 -- 172.21.15.7:0/3304847940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb8015a100 msgr2=0x7fbb80169ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:58.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.612+0000 7fbb87e0b700 1 --2- 172.21.15.7:0/3304847940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb8015a100 0x7fbb80169ae0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fbb8006ea80 tx=0x7fbb78018000 comp rx=0 tx=0).stop 2024-01-27T02:41:58.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.612+0000 7fbb87e0b700 1 -- 172.21.15.7:0/3304847940 shutdown_connections 2024-01-27T02:41:58.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.612+0000 7fbb87e0b700 1 --2- 172.21.15.7:0/3304847940 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbb64066f30 0x7fbb640693e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.612+0000 7fbb87e0b700 1 --2- 172.21.15.7:0/3304847940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb8015a100 0x7fbb80169ae0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.612+0000 7fbb87e0b700 1 --2- 172.21.15.7:0/3304847940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800c4cc0 0x7fbb800c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.612+0000 7fbb87e0b700 1 --2- 172.21.15.7:0/3304847940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb800cb5b0 0x7fbb80153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.612+0000 7fbb87e0b700 1 -- 172.21.15.7:0/3304847940 >> 172.21.15.7:0/3304847940 conn(0x7fbb80098020 msgr2=0x7fbb800ca480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:58.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.612+0000 7fbb87e0b700 1 -- 172.21.15.7:0/3304847940 shutdown_connections 2024-01-27T02:41:58.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.612+0000 7fbb87e0b700 1 -- 172.21.15.7:0/3304847940 wait complete. 2024-01-27T02:41:58.627 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:41:58.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.759+0000 7f9e45108700 1 Processor -- start 2024-01-27T02:41:58.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.760+0000 7f9e45108700 1 -- start start 2024-01-27T02:41:58.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.760+0000 7f9e45108700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e40157ca0 0x7f9e400c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:58.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.760+0000 7f9e45108700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9e400c3ae0 0x7f9e400cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:58.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.760+0000 7f9e45108700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9e40158070 0x7f9e400cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:58.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.760+0000 7f9e45108700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e400ca020 con 0x7f9e400c3ae0 2024-01-27T02:41:58.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.760+0000 7f9e45108700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e400c4f40 con 0x7f9e40158070 2024-01-27T02:41:58.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.760+0000 7f9e45108700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e400c5210 con 0x7f9e40157ca0 2024-01-27T02:41:58.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.761+0000 7f9e3e59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9e400c3ae0 0x7f9e400cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:58.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.761+0000 7f9e3e59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9e400c3ae0 0x7f9e400cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43158/0 (socket says 172.21.15.7:43158) 2024-01-27T02:41:58.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.761+0000 7f9e3e59c700 1 -- 172.21.15.7:0/3948285580 learned_addr learned my addr 172.21.15.7:0/3948285580 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:58.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.761+0000 7f9e3ed9d700 1 --2- 172.21.15.7:0/3948285580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e40157ca0 0x7f9e400c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:58.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.761+0000 7f9e3f59e700 1 --2- 172.21.15.7:0/3948285580 >> v2:172.21.15.188:3300/0 conn(0x7f9e40158070 0x7f9e400cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:58.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.761+0000 7f9e3e59c700 1 -- 172.21.15.7:0/3948285580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e40157ca0 msgr2=0x7f9e400c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:58.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.761+0000 7f9e3e59c700 1 --2- 172.21.15.7:0/3948285580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e40157ca0 0x7f9e400c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.761+0000 7f9e3e59c700 1 -- 172.21.15.7:0/3948285580 >> v2:172.21.15.188:3300/0 conn(0x7f9e40158070 msgr2=0x7f9e400cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:41:58.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.761+0000 7f9e3e59c700 1 --2- 172.21.15.7:0/3948285580 >> v2:172.21.15.188:3300/0 conn(0x7f9e40158070 0x7f9e400cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.761+0000 7f9e3e59c700 1 -- 172.21.15.7:0/3948285580 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e400c54f0 con 0x7f9e400c3ae0 2024-01-27T02:41:58.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.761+0000 7f9e3f59e700 1 --2- 172.21.15.7:0/3948285580 >> v2:172.21.15.188:3300/0 conn(0x7f9e40158070 0x7f9e400cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:41:58.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.761+0000 7f9e3e59c700 1 --2- 172.21.15.7:0/3948285580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e400c3ae0 0x7f9e400cd4e0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f9e4006f3b0 tx=0x7f9e34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe1b8bdf8c965bf server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:58.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.761+0000 7f9e3dd9b700 1 -- 172.21.15.7:0/3948285580 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3400b860 con 0x7f9e400c3ae0 2024-01-27T02:41:58.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.761+0000 7f9e3dd9b700 1 -- 172.21.15.7:0/3948285580 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3400b9c0 con 0x7f9e400c3ae0 2024-01-27T02:41:58.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.761+0000 7f9e3dd9b700 1 -- 172.21.15.7:0/3948285580 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3400bd60 con 0x7f9e400c3ae0 2024-01-27T02:41:58.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.762+0000 7f9e45108700 1 -- 172.21.15.7:0/3948285580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e400c3ae0 msgr2=0x7f9e400cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:58.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.762+0000 7f9e45108700 1 --2- 172.21.15.7:0/3948285580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e400c3ae0 0x7f9e400cd4e0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f9e4006f3b0 tx=0x7f9e34009250 comp rx=0 tx=0).stop 2024-01-27T02:41:58.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.762+0000 7f9e45108700 1 -- 172.21.15.7:0/3948285580 shutdown_connections 2024-01-27T02:41:58.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.762+0000 7f9e45108700 1 --2- 172.21.15.7:0/3948285580 >> v2:172.21.15.188:3300/0 conn(0x7f9e40158070 0x7f9e400cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.762+0000 7f9e45108700 1 --2- 172.21.15.7:0/3948285580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e40157ca0 0x7f9e400c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.762+0000 7f9e45108700 1 --2- 172.21.15.7:0/3948285580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e400c3ae0 0x7f9e400cd4e0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.762+0000 7f9e45108700 1 -- 172.21.15.7:0/3948285580 >> 172.21.15.7:0/3948285580 conn(0x7f9e40098020 msgr2=0x7f9e400711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:58.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.762+0000 7f9e45108700 1 -- 172.21.15.7:0/3948285580 shutdown_connections 2024-01-27T02:41:58.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.762+0000 7f9e45108700 1 -- 172.21.15.7:0/3948285580 wait complete. 2024-01-27T02:41:58.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.763+0000 7f9e45108700 1 Processor -- start 2024-01-27T02:41:58.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.763+0000 7f9e45108700 1 -- start start 2024-01-27T02:41:58.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.764+0000 7f9e45108700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e400c3ae0 0x7f9e4014d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:58.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.764+0000 7f9e45108700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e40157ca0 0x7f9e400b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:58.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.764+0000 7f9e45108700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e40158070 0x7f9e4014cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:58.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.764+0000 7f9e45108700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e400b3e50 con 0x7f9e400c3ae0 2024-01-27T02:41:58.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.764+0000 7f9e45108700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e40068ca0 con 0x7f9e40158070 2024-01-27T02:41:58.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.764+0000 7f9e45108700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4014d4a0 con 0x7f9e40157ca0 2024-01-27T02:41:58.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.764+0000 7f9e3ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e400c3ae0 0x7f9e4014d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:58.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.764+0000 7f9e3ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e400c3ae0 0x7f9e4014d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43166/0 (socket says 172.21.15.7:43166) 2024-01-27T02:41:58.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.764+0000 7f9e3ed9d700 1 -- 172.21.15.7:0/1035613460 learned_addr learned my addr 172.21.15.7:0/1035613460 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:58.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.764+0000 7f9e3e59c700 1 --2- 172.21.15.7:0/1035613460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e40157ca0 0x7f9e400b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:58.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.765+0000 7f9e3f59e700 1 --2- 172.21.15.7:0/1035613460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e40158070 0x7f9e4014cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:58.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.765+0000 7f9e3ed9d700 1 -- 172.21.15.7:0/1035613460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e40157ca0 msgr2=0x7f9e400b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:58.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.765+0000 7f9e3ed9d700 1 --2- 172.21.15.7:0/1035613460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e40157ca0 0x7f9e400b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.765+0000 7f9e3ed9d700 1 -- 172.21.15.7:0/1035613460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e40158070 msgr2=0x7f9e4014cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:58.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.765+0000 7f9e3ed9d700 1 --2- 172.21.15.7:0/1035613460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e40158070 0x7f9e4014cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.765+0000 7f9e3ed9d700 1 -- 172.21.15.7:0/1035613460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e340094f0 con 0x7f9e400c3ae0 2024-01-27T02:41:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.765+0000 7f9e3f59e700 1 --2- 172.21.15.7:0/1035613460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e40158070 0x7f9e4014cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:41:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.765+0000 7f9e3ed9d700 1 --2- 172.21.15.7:0/1035613460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e400c3ae0 0x7f9e4014d6f0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000d890 tx=0x7f9e30008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.765+0000 7f9e26ffd700 1 -- 172.21.15.7:0/1035613460 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e300125f0 con 0x7f9e400c3ae0 2024-01-27T02:41:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.765+0000 7f9e26ffd700 1 -- 172.21.15.7:0/1035613460 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e30012770 con 0x7f9e400c3ae0 2024-01-27T02:41:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.765+0000 7f9e45108700 1 -- 172.21.15.7:0/1035613460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e400b3b70 con 0x7f9e400c3ae0 2024-01-27T02:41:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.765+0000 7f9e26ffd700 1 -- 172.21.15.7:0/1035613460 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e30012b30 con 0x7f9e400c3ae0 2024-01-27T02:41:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.765+0000 7f9e45108700 1 -- 172.21.15.7:0/1035613460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e40068930 con 0x7f9e400c3ae0 2024-01-27T02:41:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.766+0000 7f9e45108700 1 -- 172.21.15.7:0/1035613460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e4004c710 con 0x7f9e400c3ae0 2024-01-27T02:41:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.767+0000 7f9e26ffd700 1 -- 172.21.15.7:0/1035613460 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9e3001e460 con 0x7f9e400c3ae0 2024-01-27T02:41:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.767+0000 7f9e26ffd700 1 --2- 172.21.15.7:0/1035613460 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9e2c066f30 0x7f9e2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.767+0000 7f9e26ffd700 1 -- 172.21.15.7:0/1035613460 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9e30084c30 con 0x7f9e400c3ae0 2024-01-27T02:41:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.767+0000 7f9e3e59c700 1 --2- 172.21.15.7:0/1035613460 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9e2c066f30 0x7f9e2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.768+0000 7f9e3e59c700 1 --2- 172.21.15.7:0/1035613460 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9e2c066f30 0x7f9e2c0693e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9e4006f3b0 tx=0x7f9e34017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:58.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.771+0000 7f9e26ffd700 1 -- 172.21.15.7:0/1035613460 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e30051ea0 con 0x7f9e400c3ae0 2024-01-27T02:41:58.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.974+0000 7f9e45108700 1 -- 172.21.15.7:0/1035613460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e400c57d0 con 0x7f9e400c3ae0 2024-01-27T02:41:58.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.975+0000 7f9e26ffd700 1 -- 172.21.15.7:0/1035613460 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+15330 (secure 0 0 0) 0x7f9e300554c0 con 0x7f9e400c3ae0 2024-01-27T02:41:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:41:58.976 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":163,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:41:57.645686+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:41:56.176076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-01-27T02:41:56.620523+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:41:58.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.977+0000 7f9e45108700 1 -- 172.21.15.7:0/1035613460 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9e2c066f30 msgr2=0x7f9e2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:58.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.977+0000 7f9e45108700 1 --2- 172.21.15.7:0/1035613460 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9e2c066f30 0x7f9e2c0693e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9e4006f3b0 tx=0x7f9e34017000 comp rx=0 tx=0).stop 2024-01-27T02:41:58.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.977+0000 7f9e45108700 1 -- 172.21.15.7:0/1035613460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e400c3ae0 msgr2=0x7f9e4014d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.977+0000 7f9e45108700 1 --2- 172.21.15.7:0/1035613460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e400c3ae0 0x7f9e4014d6f0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000d890 tx=0x7f9e30008040 comp rx=0 tx=0).stop 2024-01-27T02:41:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.978+0000 7f9e45108700 1 -- 172.21.15.7:0/1035613460 shutdown_connections 2024-01-27T02:41:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.978+0000 7f9e45108700 1 --2- 172.21.15.7:0/1035613460 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9e2c066f30 0x7f9e2c0693e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.978+0000 7f9e45108700 1 --2- 172.21.15.7:0/1035613460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e40158070 0x7f9e4014cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.978+0000 7f9e45108700 1 --2- 172.21.15.7:0/1035613460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e400c3ae0 0x7f9e4014d6f0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.978+0000 7f9e45108700 1 --2- 172.21.15.7:0/1035613460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e40157ca0 0x7f9e400b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.978+0000 7f9e45108700 1 -- 172.21.15.7:0/1035613460 >> 172.21.15.7:0/1035613460 conn(0x7f9e40098020 msgr2=0x7f9e4006c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.978+0000 7f9e45108700 1 -- 172.21.15.7:0/1035613460 shutdown_connections 2024-01-27T02:41:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:58.978+0000 7f9e45108700 1 -- 172.21.15.7:0/1035613460 wait complete. 2024-01-27T02:41:58.992 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-27T02:41:59.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.125+0000 7f79f65ef700 1 Processor -- start 2024-01-27T02:41:59.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.125+0000 7f79f65ef700 1 -- start start 2024-01-27T02:41:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.126+0000 7f79f65ef700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f79f00b0f70 0x7f79f00693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.126+0000 7f79f65ef700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f0069930 con 0x7f79f00cc8d0 2024-01-27T02:41:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.126+0000 7f79f65ef700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f015e820 con 0x7f79f00680c0 2024-01-27T02:41:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.126+0000 7f79f65ef700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f0068840 con 0x7f79f00b0f70 2024-01-27T02:41:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.126+0000 7f79effff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f79f00b0f70 0x7f79f00693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.126+0000 7f79ef7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f79f00cc8d0 0x7f79f00679b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39668/0 (socket says 172.21.15.7:39668) 2024-01-27T02:41:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.126+0000 7f79ef7fe700 1 -- 172.21.15.7:0/3014198250 learned_addr learned my addr 172.21.15.7:0/3014198250 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:41:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.126+0000 7f79effff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f79f00b0f70 0x7f79f00693f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43178/0 (socket says 172.21.15.7:43178) 2024-01-27T02:41:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.126+0000 7f79effff700 1 -- 172.21.15.7:0/3014198250 >> v1:172.21.15.188:6789/0 conn(0x7f79f00680c0 legacy=0x7f79f00c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.127+0000 7f79effff700 1 -- 172.21.15.7:0/3014198250 >> v1:172.21.15.7:6789/0 conn(0x7f79f00cc8d0 legacy=0x7f79f00679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.127+0000 7f79effff700 1 -- 172.21.15.7:0/3014198250 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f015dc10 con 0x7f79f00b0f70 2024-01-27T02:41:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.127+0000 7f79eeffd700 1 -- 172.21.15.7:0/3014198250 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 489691317 0 0) 0x7f79f0069930 con 0x7f79f00cc8d0 2024-01-27T02:41:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.127+0000 7f79effff700 1 --2- 172.21.15.7:0/3014198250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79f00b0f70 0x7f79f00693f0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f79e00098e0 tx=0x7f79e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9608fe01f898205 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.127+0000 7f79eeffd700 1 -- 172.21.15.7:0/3014198250 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0009e80 con 0x7f79f00b0f70 2024-01-27T02:41:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.127+0000 7f79eeffd700 1 -- 172.21.15.7:0/3014198250 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e000b860 con 0x7f79f00b0f70 2024-01-27T02:41:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.127+0000 7f79eeffd700 1 -- 172.21.15.7:0/3014198250 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e000bbe0 con 0x7f79f00b0f70 2024-01-27T02:41:59.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.127+0000 7f79f65ef700 1 -- 172.21.15.7:0/3014198250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79f00b0f70 msgr2=0x7f79f00693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:59.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.127+0000 7f79f65ef700 1 --2- 172.21.15.7:0/3014198250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79f00b0f70 0x7f79f00693f0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f79e00098e0 tx=0x7f79e00091a0 comp rx=0 tx=0).stop 2024-01-27T02:41:59.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.128+0000 7f79f65ef700 1 -- 172.21.15.7:0/3014198250 shutdown_connections 2024-01-27T02:41:59.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.128+0000 7f79f65ef700 1 --2- 172.21.15.7:0/3014198250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79f00b0f70 0x7f79f00693f0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:59.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.128+0000 7f79f65ef700 1 -- 172.21.15.7:0/3014198250 >> 172.21.15.7:0/3014198250 conn(0x7f79f0063580 msgr2=0x7f79f0063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:41:59.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.128+0000 7f79f65ef700 1 -- 172.21.15.7:0/3014198250 shutdown_connections 2024-01-27T02:41:59.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.128+0000 7f79f65ef700 1 -- 172.21.15.7:0/3014198250 wait complete. 2024-01-27T02:41:59.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.129+0000 7f79f65ef700 1 Processor -- start 2024-01-27T02:41:59.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.129+0000 7f79f65ef700 1 -- start start 2024-01-27T02:41:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.130+0000 7f79f65ef700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79f00680c0 0x7f79f014cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.130+0000 7f79f65ef700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79f00b0f70 0x7f79f00c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.130+0000 7f79f65ef700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79f00cc8d0 0x7f79f015c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.130+0000 7f79f65ef700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f014a230 con 0x7f79f00cc8d0 2024-01-27T02:41:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.130+0000 7f79f65ef700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00c76b0 con 0x7f79f00b0f70 2024-01-27T02:41:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.130+0000 7f79f65ef700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f015a7c0 con 0x7f79f00680c0 2024-01-27T02:41:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.130+0000 7f79f4b8c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79f00cc8d0 0x7f79f015c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.130+0000 7f79f4b8c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79f00cc8d0 0x7f79f015c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43192/0 (socket says 172.21.15.7:43192) 2024-01-27T02:41:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.130+0000 7f79f4b8c700 1 -- 172.21.15.7:0/2828930429 learned_addr learned my addr 172.21.15.7:0/2828930429 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:41:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.130+0000 7f79effff700 1 --2- 172.21.15.7:0/2828930429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79f00680c0 0x7f79f014cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.130+0000 7f79ef7fe700 1 --2- 172.21.15.7:0/2828930429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79f00b0f70 0x7f79f00c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:59.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.130+0000 7f79f4b8c700 1 -- 172.21.15.7:0/2828930429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79f00680c0 msgr2=0x7f79f014cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:59.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.130+0000 7f79f4b8c700 1 --2- 172.21.15.7:0/2828930429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79f00680c0 0x7f79f014cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:59.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.130+0000 7f79f4b8c700 1 -- 172.21.15.7:0/2828930429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79f00b0f70 msgr2=0x7f79f00c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:41:59.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.130+0000 7f79f4b8c700 1 --2- 172.21.15.7:0/2828930429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79f00b0f70 0x7f79f00c6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:41:59.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.130+0000 7f79f4b8c700 1 -- 172.21.15.7:0/2828930429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e0009420 con 0x7f79f00cc8d0 2024-01-27T02:41:59.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.130+0000 7f79f4b8c700 1 --2- 172.21.15.7:0/2828930429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79f00cc8d0 0x7f79f015c570 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f79e800c840 tx=0x7f79e8001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:59.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.131+0000 7f79e7fff700 1 -- 172.21.15.7:0/2828930429 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8012630 con 0x7f79f00cc8d0 2024-01-27T02:41:59.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.131+0000 7f79e7fff700 1 -- 172.21.15.7:0/2828930429 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e80127b0 con 0x7f79f00cc8d0 2024-01-27T02:41:59.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.131+0000 7f79f65ef700 1 -- 172.21.15.7:0/2828930429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f015aaa0 con 0x7f79f00cc8d0 2024-01-27T02:41:59.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.131+0000 7f79e7fff700 1 -- 172.21.15.7:0/2828930429 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8012b70 con 0x7f79f00cc8d0 2024-01-27T02:41:59.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.131+0000 7f79f65ef700 1 -- 172.21.15.7:0/2828930429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f014d160 con 0x7f79f00cc8d0 2024-01-27T02:41:59.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.132+0000 7f79f65ef700 1 -- 172.21.15.7:0/2828930429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f015a620 con 0x7f79f00cc8d0 2024-01-27T02:41:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.132+0000 7f79e7fff700 1 -- 172.21.15.7:0/2828930429 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f79e8003030 con 0x7f79f00cc8d0 2024-01-27T02:41:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.133+0000 7f79e7fff700 1 --2- 172.21.15.7:0/2828930429 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f79d0066f30 0x7f79d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:41:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.133+0000 7f79e7fff700 1 -- 172.21.15.7:0/2828930429 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f79e80844a0 con 0x7f79f00cc8d0 2024-01-27T02:41:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.133+0000 7f79effff700 1 --2- 172.21.15.7:0/2828930429 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f79d0066f30 0x7f79d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:41:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.133+0000 7f79effff700 1 --2- 172.21.15.7:0/2828930429 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f79d0066f30 0x7f79d00693e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f79e00093f0 tx=0x7f79e0017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:41:59.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.137+0000 7f79e7fff700 1 -- 172.21.15.7:0/2828930429 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f79f015a620 con 0x7f79f00cc8d0 2024-01-27T02:41:59.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:41:59.392+0000 7f79f65ef700 1 -- 172.21.15.7:0/2828930429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f79f015afc0 con 0x7f79f00cc8d0 2024-01-27T02:42:00.212 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-01-27T02:42:00.210+0000 7fb83d9a1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-27T02:42:00.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.251+0000 7f79e7fff700 1 -- 172.21.15.7:0/2828930429 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v85) v1 ==== 118+0+0 (secure 0 0 0) 0x7f79f015afc0 con 0x7f79f00cc8d0 2024-01-27T02:42:00.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.254+0000 7f79f65ef700 1 -- 172.21.15.7:0/2828930429 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f79d0066f30 msgr2=0x7f79d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:00.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.254+0000 7f79f65ef700 1 --2- 172.21.15.7:0/2828930429 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f79d0066f30 0x7f79d00693e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f79e00093f0 tx=0x7f79e0017000 comp rx=0 tx=0).stop 2024-01-27T02:42:00.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.254+0000 7f79f65ef700 1 -- 172.21.15.7:0/2828930429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79f00cc8d0 msgr2=0x7f79f015c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:00.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.254+0000 7f79f65ef700 1 --2- 172.21.15.7:0/2828930429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79f00cc8d0 0x7f79f015c570 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f79e800c840 tx=0x7f79e8001670 comp rx=0 tx=0).stop 2024-01-27T02:42:00.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.254+0000 7f79f65ef700 1 -- 172.21.15.7:0/2828930429 shutdown_connections 2024-01-27T02:42:00.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.254+0000 7f79f65ef700 1 --2- 172.21.15.7:0/2828930429 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f79d0066f30 0x7f79d00693e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:00.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.254+0000 7f79f65ef700 1 --2- 172.21.15.7:0/2828930429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79f00cc8d0 0x7f79f015c570 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:00.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.254+0000 7f79f65ef700 1 --2- 172.21.15.7:0/2828930429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79f00680c0 0x7f79f014cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:00.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.254+0000 7f79f65ef700 1 --2- 172.21.15.7:0/2828930429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79f00b0f70 0x7f79f00c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:00.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.254+0000 7f79f65ef700 1 -- 172.21.15.7:0/2828930429 >> 172.21.15.7:0/2828930429 conn(0x7f79f0063580 msgr2=0x7f79f0158a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:00.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.254+0000 7f79f65ef700 1 -- 172.21.15.7:0/2828930429 shutdown_connections 2024-01-27T02:42:00.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.254+0000 7f79f65ef700 1 -- 172.21.15.7:0/2828930429 wait complete. 2024-01-27T02:42:00.274 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-27T02:42:00.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.412+0000 7fade145a700 1 Processor -- start 2024-01-27T02:42:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.412+0000 7fade145a700 1 -- start start 2024-01-27T02:42:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.413+0000 7fade145a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faddc15d460 0x7faddc0d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.413+0000 7fade145a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faddc0cf280 0x7faddc15e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.413+0000 7fade145a700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faddc0b38d0 con 0x7faddc15d830 2024-01-27T02:42:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.413+0000 7fade145a700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faddc0d0390 con 0x7faddc0cf280 2024-01-27T02:42:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.413+0000 7fade145a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faddc0cf780 con 0x7faddc15d460 2024-01-27T02:42:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.414+0000 7faddaffd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faddc15d460 0x7faddc0d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.414+0000 7fadda7fc700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faddc0cf280 0x7faddc15e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.414+0000 7faddaffd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faddc15d460 0x7faddc0d0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:44448/0 (socket says 172.21.15.7:44448) 2024-01-27T02:42:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.414+0000 7faddaffd700 1 -- 172.21.15.7:0/3529561658 learned_addr learned my addr 172.21.15.7:0/3529561658 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:42:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.414+0000 7fadda7fc700 1 -- 172.21.15.7:0/3529561658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faddc15d460 msgr2=0x7faddc0d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.414+0000 7fadda7fc700 1 --2- 172.21.15.7:0/3529561658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faddc15d460 0x7faddc0d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.414+0000 7fadda7fc700 1 -- 172.21.15.7:0/3529561658 >> v1:172.21.15.188:6789/0 conn(0x7faddc15d830 legacy=0x7faddc0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.414+0000 7fadda7fc700 1 -- 172.21.15.7:0/3529561658 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faddc0cfa60 con 0x7faddc0cf280 2024-01-27T02:42:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.414+0000 7fadda7fc700 1 --2- 172.21.15.7:0/3529561658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faddc0cf280 0x7faddc15e6b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fadd00098e0 tx=0x7fadd00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf082c0906e8ba74 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.415+0000 7fadd9ffb700 1 -- 172.21.15.7:0/3529561658 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd000b640 con 0x7faddc0cf280 2024-01-27T02:42:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.415+0000 7fadd9ffb700 1 -- 172.21.15.7:0/3529561658 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd000b7a0 con 0x7faddc0cf280 2024-01-27T02:42:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.415+0000 7fadd9ffb700 1 -- 172.21.15.7:0/3529561658 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd000bb40 con 0x7faddc0cf280 2024-01-27T02:42:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.415+0000 7fade145a700 1 -- 172.21.15.7:0/3529561658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faddc0cf280 msgr2=0x7faddc15e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.415+0000 7fade145a700 1 --2- 172.21.15.7:0/3529561658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faddc0cf280 0x7faddc15e6b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fadd00098e0 tx=0x7fadd00091a0 comp rx=0 tx=0).stop 2024-01-27T02:42:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.415+0000 7fade145a700 1 -- 172.21.15.7:0/3529561658 shutdown_connections 2024-01-27T02:42:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.415+0000 7fade145a700 1 --2- 172.21.15.7:0/3529561658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faddc15d460 0x7faddc0d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.415+0000 7fade145a700 1 --2- 172.21.15.7:0/3529561658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faddc0cf280 0x7faddc15e6b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.415+0000 7fade145a700 1 -- 172.21.15.7:0/3529561658 >> 172.21.15.7:0/3529561658 conn(0x7faddc063580 msgr2=0x7faddc063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.415+0000 7fade145a700 1 -- 172.21.15.7:0/3529561658 shutdown_connections 2024-01-27T02:42:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.415+0000 7fade145a700 1 -- 172.21.15.7:0/3529561658 wait complete. 2024-01-27T02:42:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.417+0000 7fade145a700 1 Processor -- start 2024-01-27T02:42:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.417+0000 7fade145a700 1 -- start start 2024-01-27T02:42:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.418+0000 7fade145a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faddc0cf280 0x7faddc15dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.418+0000 7fade145a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faddc15d460 0x7faddc0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.418+0000 7fade145a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faddc15d830 0x7faddc1528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.418+0000 7fade145a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faddc15eba0 con 0x7faddc15d460 2024-01-27T02:42:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.418+0000 7fade145a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faddc156ab0 con 0x7faddc15d830 2024-01-27T02:42:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.418+0000 7fade145a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faddc152de0 con 0x7faddc0cf280 2024-01-27T02:42:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.418+0000 7fadda7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faddc15d460 0x7faddc0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.418+0000 7fadda7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faddc15d460 0x7faddc0b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43206/0 (socket says 172.21.15.7:43206) 2024-01-27T02:42:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.418+0000 7fadda7fc700 1 -- 172.21.15.7:0/3196888326 learned_addr learned my addr 172.21.15.7:0/3196888326 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:42:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.418+0000 7faddaffd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faddc0cf280 0x7faddc15dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.418+0000 7faddb7fe700 1 --2- 172.21.15.7:0/3196888326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faddc15d830 0x7faddc1528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.419+0000 7fadda7fc700 1 -- 172.21.15.7:0/3196888326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faddc0cf280 msgr2=0x7faddc15dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.419+0000 7fadda7fc700 1 --2- 172.21.15.7:0/3196888326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faddc0cf280 0x7faddc15dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.419+0000 7fadda7fc700 1 -- 172.21.15.7:0/3196888326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faddc15d830 msgr2=0x7faddc1528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.419+0000 7fadda7fc700 1 --2- 172.21.15.7:0/3196888326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faddc15d830 0x7faddc1528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.419+0000 7fadda7fc700 1 -- 172.21.15.7:0/3196888326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd0009420 con 0x7faddc15d460 2024-01-27T02:42:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.419+0000 7faddaffd700 1 --2- 172.21.15.7:0/3196888326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faddc0cf280 0x7faddc15dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:42:00.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.419+0000 7faddb7fe700 1 --2- 172.21.15.7:0/3196888326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faddc15d830 0x7faddc1528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:42:00.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.419+0000 7fadda7fc700 1 --2- 172.21.15.7:0/3196888326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faddc15d460 0x7faddc0b5430 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fadd0013000 tx=0x7fadd0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:00.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.419+0000 7fadc2ffd700 1 -- 172.21.15.7:0/3196888326 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd001bc40 con 0x7faddc15d460 2024-01-27T02:42:00.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.419+0000 7fadc2ffd700 1 -- 172.21.15.7:0/3196888326 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd001bde0 con 0x7faddc15d460 2024-01-27T02:42:00.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.419+0000 7fadc2ffd700 1 -- 172.21.15.7:0/3196888326 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0011670 con 0x7faddc15d460 2024-01-27T02:42:00.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.419+0000 7fade145a700 1 -- 172.21.15.7:0/3196888326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faddc15e300 con 0x7faddc15d460 2024-01-27T02:42:00.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.419+0000 7fade145a700 1 -- 172.21.15.7:0/3196888326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faddc15e8c0 con 0x7faddc15d460 2024-01-27T02:42:00.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.420+0000 7fade145a700 1 -- 172.21.15.7:0/3196888326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faddc0b5970 con 0x7faddc15d460 2024-01-27T02:42:00.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.421+0000 7fadc2ffd700 1 -- 172.21.15.7:0/3196888326 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fadd00119e0 con 0x7faddc15d460 2024-01-27T02:42:00.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.421+0000 7fadc2ffd700 1 --2- 172.21.15.7:0/3196888326 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fadc8066f30 0x7fadc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:00.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.421+0000 7fadc2ffd700 1 -- 172.21.15.7:0/3196888326 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fadd0085ab0 con 0x7faddc15d460 2024-01-27T02:42:00.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.421+0000 7faddaffd700 1 --2- 172.21.15.7:0/3196888326 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fadc8066f30 0x7fadc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:00.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.422+0000 7faddaffd700 1 --2- 172.21.15.7:0/3196888326 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fadc8066f30 0x7fadc80693e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009fd0 tx=0x7fadcc009250 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:00.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.426+0000 7fadc2ffd700 1 -- 172.21.15.7:0/3196888326 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fadd0052d20 con 0x7faddc15d460 2024-01-27T02:42:00.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:00.683+0000 7fade145a700 1 -- 172.21.15.7:0/3196888326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7faddc072060 con 0x7faddc15d460 2024-01-27T02:42:01.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.252+0000 7fadc2ffd700 1 -- 172.21.15.7:0/3196888326 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v86) v1 ==== 122+0+0 (secure 0 0 0) 0x7fadd0056340 con 0x7faddc15d460 2024-01-27T02:42:01.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.255+0000 7fade145a700 1 -- 172.21.15.7:0/3196888326 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fadc8066f30 msgr2=0x7fadc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:01.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.255+0000 7fade145a700 1 --2- 172.21.15.7:0/3196888326 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fadc8066f30 0x7fadc80693e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009fd0 tx=0x7fadcc009250 comp rx=0 tx=0).stop 2024-01-27T02:42:01.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.255+0000 7fade145a700 1 -- 172.21.15.7:0/3196888326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faddc15d460 msgr2=0x7faddc0b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:01.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.255+0000 7fade145a700 1 --2- 172.21.15.7:0/3196888326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faddc15d460 0x7faddc0b5430 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fadd0013000 tx=0x7fadd0017000 comp rx=0 tx=0).stop 2024-01-27T02:42:01.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.255+0000 7fade145a700 1 -- 172.21.15.7:0/3196888326 shutdown_connections 2024-01-27T02:42:01.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.255+0000 7fade145a700 1 --2- 172.21.15.7:0/3196888326 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fadc8066f30 0x7fadc80693e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:01.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.255+0000 7fade145a700 1 --2- 172.21.15.7:0/3196888326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faddc15d830 0x7faddc1528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:01.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.255+0000 7fade145a700 1 --2- 172.21.15.7:0/3196888326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faddc0cf280 0x7faddc15dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:01.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.255+0000 7fade145a700 1 --2- 172.21.15.7:0/3196888326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faddc15d460 0x7faddc0b5430 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:01.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.255+0000 7fade145a700 1 -- 172.21.15.7:0/3196888326 >> 172.21.15.7:0/3196888326 conn(0x7faddc063580 msgr2=0x7faddc0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:01.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.256+0000 7fade145a700 1 -- 172.21.15.7:0/3196888326 shutdown_connections 2024-01-27T02:42:01.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.256+0000 7fade145a700 1 -- 172.21.15.7:0/3196888326 wait complete. 2024-01-27T02:42:01.273 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:42:01.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.406+0000 7f0d18097700 1 Processor -- start 2024-01-27T02:42:01.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.406+0000 7f0d18097700 1 -- start start 2024-01-27T02:42:01.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.407+0000 7f0d18097700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d100d1470 0x7f0d100b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:01.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.407+0000 7f0d18097700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d100b4650 con 0x7f0d100d10a0 2024-01-27T02:42:01.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.407+0000 7f0d18097700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d100b3d60 con 0x7f0d10159b40 2024-01-27T02:42:01.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.407+0000 7f0d18097700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d100b34d0 con 0x7f0d100d1470 2024-01-27T02:42:01.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.407+0000 7f0d15e33700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0d100d10a0 0x7f0d100b2dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34904/0 (socket says 172.21.15.7:34904) 2024-01-27T02:42:01.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.407+0000 7f0d15e33700 1 -- 172.21.15.7:0/1632696315 learned_addr learned my addr 172.21.15.7:0/1632696315 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:42:01.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.408+0000 7f0d15632700 1 --2- 172.21.15.7:0/1632696315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d100d1470 0x7f0d100b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:01.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.408+0000 7f0d14e31700 1 -- 172.21.15.7:0/1632696315 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3016057515 0 0) 0x7f0d100b4650 con 0x7f0d100d10a0 2024-01-27T02:42:01.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.408+0000 7f0d14e31700 1 -- 172.21.15.7:0/1632696315 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cfc003570 con 0x7f0d100d10a0 2024-01-27T02:42:01.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.408+0000 7f0d14e31700 1 -- 172.21.15.7:0/1632696315 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3844743855 0 0) 0x7f0d100b3d60 con 0x7f0d10159b40 2024-01-27T02:42:01.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.408+0000 7f0d14e31700 1 -- 172.21.15.7:0/1632696315 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d100b4650 con 0x7f0d10159b40 2024-01-27T02:42:01.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.408+0000 7f0d14e31700 1 -- 172.21.15.7:0/1632696315 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3058326662 0 0) 0x7f0cfc003570 con 0x7f0d100d10a0 2024-01-27T02:42:01.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.408+0000 7f0d14e31700 1 -- 172.21.15.7:0/1632696315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d100d1470 msgr2=0x7f0d100b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:01.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.408+0000 7f0d14e31700 1 --2- 172.21.15.7:0/1632696315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d100d1470 0x7f0d100b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:01.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.408+0000 7f0d14e31700 1 -- 172.21.15.7:0/1632696315 >> v1:172.21.15.188:6789/0 conn(0x7f0d10159b40 legacy=0x7f0d10159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:01.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.408+0000 7f0d14e31700 1 -- 172.21.15.7:0/1632696315 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d100b4000 con 0x7f0d100d10a0 2024-01-27T02:42:01.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.408+0000 7f0d14e31700 1 -- 172.21.15.7:0/1632696315 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f0d0c001470 con 0x7f0d100d10a0 2024-01-27T02:42:01.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.408+0000 7f0d14e31700 1 -- 172.21.15.7:0/1632696315 >> v1:172.21.15.7:6789/0 conn(0x7f0d100d10a0 legacy=0x7f0d100b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:01.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.408+0000 7f0d14e31700 1 --2- 172.21.15.7:0/1632696315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d100d1470 0x7f0cfc003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:01.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.408+0000 7f0d14e31700 1 --2- 172.21.15.7:0/1632696315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc004180 0x7f0cfc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:01.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.408+0000 7f0d14e31700 1 --2- 172.21.15.7:0/1632696315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cfc004d50 0x7f0cfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:01.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.408+0000 7f0d14e31700 1 -- 172.21.15.7:0/1632696315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d100b34d0 con 0x7f0cfc004180 2024-01-27T02:42:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.408+0000 7f0d14e31700 1 -- 172.21.15.7:0/1632696315 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc00bba0 con 0x7f0cfc004d50 2024-01-27T02:42:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.408+0000 7f0d14e31700 1 -- 172.21.15.7:0/1632696315 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc00bd60 con 0x7f0d100d1470 2024-01-27T02:42:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.408+0000 7f0d15632700 1 --2- 172.21.15.7:0/1632696315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc004180 0x7f0cfc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.408+0000 7f0d15e33700 1 --2- 172.21.15.7:0/1632696315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d100d1470 0x7f0cfc003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.408+0000 7f0d16634700 1 --2- 172.21.15.7:0/1632696315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cfc004d50 0x7f0cfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.409+0000 7f0d15632700 1 -- 172.21.15.7:0/1632696315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d100d1470 msgr2=0x7f0cfc003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.409+0000 7f0d15632700 1 --2- 172.21.15.7:0/1632696315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d100d1470 0x7f0cfc003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.409+0000 7f0d15632700 1 -- 172.21.15.7:0/1632696315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cfc004d50 msgr2=0x7f0cfc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.409+0000 7f0d15632700 1 --2- 172.21.15.7:0/1632696315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cfc004d50 0x7f0cfc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.409+0000 7f0d15632700 1 -- 172.21.15.7:0/1632696315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc003570 con 0x7f0cfc004180 2024-01-27T02:42:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.409+0000 7f0d15632700 1 --2- 172.21.15.7:0/1632696315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc004180 0x7f0cfc004810 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009c00 tx=0x7f0d000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db7285470969eb6 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.409+0000 7f0d14e31700 1 -- 172.21.15.7:0/1632696315 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0000e710 con 0x7f0cfc004180 2024-01-27T02:42:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.409+0000 7f0d14e31700 1 -- 172.21.15.7:0/1632696315 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0000e870 con 0x7f0cfc004180 2024-01-27T02:42:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.409+0000 7f0d14e31700 1 -- 172.21.15.7:0/1632696315 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0000ec30 con 0x7f0cfc004180 2024-01-27T02:42:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.410+0000 7f0d18097700 1 -- 172.21.15.7:0/1632696315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc004180 msgr2=0x7f0cfc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.410+0000 7f0d18097700 1 --2- 172.21.15.7:0/1632696315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc004180 0x7f0cfc004810 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009c00 tx=0x7f0d000091a0 comp rx=0 tx=0).stop 2024-01-27T02:42:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.410+0000 7f0d16634700 1 -- 172.21.15.7:0/1632696315 reap_dead start 2024-01-27T02:42:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.410+0000 7f0d18097700 1 -- 172.21.15.7:0/1632696315 shutdown_connections 2024-01-27T02:42:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.410+0000 7f0d18097700 1 -- 172.21.15.7:0/1632696315 >> 172.21.15.7:0/1632696315 conn(0x7f0d100679b0 msgr2=0x7f0d10067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.410+0000 7f0d18097700 1 -- 172.21.15.7:0/1632696315 shutdown_connections 2024-01-27T02:42:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.410+0000 7f0d18097700 1 -- 172.21.15.7:0/1632696315 wait complete. 2024-01-27T02:42:01.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.411+0000 7f0d18097700 1 Processor -- start 2024-01-27T02:42:01.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.411+0000 7f0d18097700 1 -- start start 2024-01-27T02:42:01.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.412+0000 7f0d18097700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cfc004180 0x7f0d101494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:01.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.412+0000 7f0d18097700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d100c8ed0 0x7f0d100c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:01.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.412+0000 7f0d18097700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d1015cbf0 0x7f0d1015d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:01.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.412+0000 7f0d18097700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d100ca580 con 0x7f0d100c8ed0 2024-01-27T02:42:01.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.412+0000 7f0d18097700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d100c7710 con 0x7f0cfc004180 2024-01-27T02:42:01.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.412+0000 7f0d18097700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d100c9970 con 0x7f0d1015cbf0 2024-01-27T02:42:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.412+0000 7f0d15632700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d100c8ed0 0x7f0d100c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.412+0000 7f0d15632700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d100c8ed0 0x7f0d100c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50400/0 (socket says 172.21.15.7:50400) 2024-01-27T02:42:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.412+0000 7f0d15632700 1 -- 172.21.15.7:0/944659837 learned_addr learned my addr 172.21.15.7:0/944659837 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:42:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.412+0000 7f0d16634700 1 --2- 172.21.15.7:0/944659837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d1015cbf0 0x7f0d1015d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.412+0000 7f0d15e33700 1 --2- 172.21.15.7:0/944659837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cfc004180 0x7f0d101494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.412+0000 7f0d15632700 1 -- 172.21.15.7:0/944659837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d1015cbf0 msgr2=0x7f0d1015d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.412+0000 7f0d15632700 1 --2- 172.21.15.7:0/944659837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d1015cbf0 0x7f0d1015d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.412+0000 7f0d15632700 1 -- 172.21.15.7:0/944659837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cfc004180 msgr2=0x7f0d101494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.412+0000 7f0d15632700 1 --2- 172.21.15.7:0/944659837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cfc004180 0x7f0d101494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.412+0000 7f0d15632700 1 -- 172.21.15.7:0/944659837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d100c9c50 con 0x7f0d100c8ed0 2024-01-27T02:42:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.412+0000 7f0d16634700 1 --2- 172.21.15.7:0/944659837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d1015cbf0 0x7f0d1015d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:42:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.412+0000 7f0d15e33700 1 --2- 172.21.15.7:0/944659837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cfc004180 0x7f0d101494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:42:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.412+0000 7f0d15632700 1 --2- 172.21.15.7:0/944659837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d100c8ed0 0x7f0d100c9340 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00bb70 tx=0x7f0d0001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.413+0000 7f0d05ffb700 1 -- 172.21.15.7:0/944659837 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00022070 con 0x7f0d100c8ed0 2024-01-27T02:42:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.413+0000 7f0d05ffb700 1 -- 172.21.15.7:0/944659837 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0000ebf0 con 0x7f0d100c8ed0 2024-01-27T02:42:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.413+0000 7f0d05ffb700 1 -- 172.21.15.7:0/944659837 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0001c6a0 con 0x7f0d100c8ed0 2024-01-27T02:42:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.413+0000 7f0d18097700 1 -- 172.21.15.7:0/944659837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d100ca580 con 0x7f0d100c8ed0 2024-01-27T02:42:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.413+0000 7f0d18097700 1 -- 172.21.15.7:0/944659837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d100ca210 con 0x7f0d100c8ed0 2024-01-27T02:42:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.414+0000 7f0d18097700 1 -- 172.21.15.7:0/944659837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d1004c710 con 0x7f0d100c8ed0 2024-01-27T02:42:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.414+0000 7f0d05ffb700 1 -- 172.21.15.7:0/944659837 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0d0001ca50 con 0x7f0d100c8ed0 2024-01-27T02:42:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.415+0000 7f0d05ffb700 1 --2- 172.21.15.7:0/944659837 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0cf4066f30 0x7f0cf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.415+0000 7f0d05ffb700 1 -- 172.21.15.7:0/944659837 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0d00085d70 con 0x7f0d100c8ed0 2024-01-27T02:42:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.415+0000 7f0d15e33700 1 --2- 172.21.15.7:0/944659837 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0cf4066f30 0x7f0cf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.416+0000 7f0d15e33700 1 --2- 172.21.15.7:0/944659837 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0cf4066f30 0x7f0cf40693e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c007d60 tx=0x7f0d0c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.419+0000 7f0d05ffb700 1 -- 172.21.15.7:0/944659837 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d00052fe0 con 0x7f0d100c8ed0 2024-01-27T02:42:01.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.669+0000 7f0d18097700 1 -- 172.21.15.7:0/944659837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d100cb640 con 0x7f0d100c8ed0 2024-01-27T02:42:01.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.670+0000 7f0d05ffb700 1 -- 172.21.15.7:0/944659837 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f0d00006cb0 con 0x7f0d100c8ed0 2024-01-27T02:42:01.672 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:42:01.672 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6473,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/3449593528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3449593528},{"type":"v1","addr":"172.21.15.188:6836","nonce":3449593528}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3473642033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3473642033},{"type":"v1","addr":"172.21.15.188:6837","nonce":3473642033}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6479,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2550009140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2550009140},{"type":"v1","addr":"172.21.15.7:6833","nonce":2550009140}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:41:57.646012+0000","modified":"2024-01-27T02:42:01.251419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6485":{"gid":6485,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3213683076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3213683076},{"type":"v1","addr":"172.21.15.7:6835","nonce":3213683076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2024-01-27T02:42:01.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.672+0000 7f0d18097700 1 -- 172.21.15.7:0/944659837 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0cf4066f30 msgr2=0x7f0cf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:01.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.672+0000 7f0d18097700 1 --2- 172.21.15.7:0/944659837 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0cf4066f30 0x7f0cf40693e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c007d60 tx=0x7f0d0c009040 comp rx=0 tx=0).stop 2024-01-27T02:42:01.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.672+0000 7f0d18097700 1 -- 172.21.15.7:0/944659837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d100c8ed0 msgr2=0x7f0d100c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:01.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.672+0000 7f0d18097700 1 --2- 172.21.15.7:0/944659837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d100c8ed0 0x7f0d100c9340 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00bb70 tx=0x7f0d0001a000 comp rx=0 tx=0).stop 2024-01-27T02:42:01.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.672+0000 7f0d18097700 1 -- 172.21.15.7:0/944659837 shutdown_connections 2024-01-27T02:42:01.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.672+0000 7f0d18097700 1 --2- 172.21.15.7:0/944659837 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0cf4066f30 0x7f0cf40693e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:01.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.672+0000 7f0d18097700 1 --2- 172.21.15.7:0/944659837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d1015cbf0 0x7f0d1015d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:01.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.672+0000 7f0d18097700 1 --2- 172.21.15.7:0/944659837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cfc004180 0x7f0d101494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:01.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.672+0000 7f0d18097700 1 --2- 172.21.15.7:0/944659837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d100c8ed0 0x7f0d100c9340 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:01.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.672+0000 7f0d18097700 1 -- 172.21.15.7:0/944659837 >> 172.21.15.7:0/944659837 conn(0x7f0d100679b0 msgr2=0x7f0d101585e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:01.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.673+0000 7f0d18097700 1 -- 172.21.15.7:0/944659837 shutdown_connections 2024-01-27T02:42:01.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.673+0000 7f0d18097700 1 -- 172.21.15.7:0/944659837 wait complete. 2024-01-27T02:42:01.675 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 86 2024-01-27T02:42:01.688 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:42:01.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.818+0000 7fa775c63700 1 Processor -- start 2024-01-27T02:42:01.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.819+0000 7fa775c63700 1 -- start start 2024-01-27T02:42:01.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.819+0000 7fa775c63700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa7700caf60 0x7fa770164fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:01.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.819+0000 7fa775c63700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77015c5b0 con 0x7fa77015e920 2024-01-27T02:42:01.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.819+0000 7fa775c63700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77015b9a0 con 0x7fa7700c97e0 2024-01-27T02:42:01.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.819+0000 7fa775c63700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa77015bc80 con 0x7fa7700caf60 2024-01-27T02:42:01.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.820+0000 7fa76effd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa77015e920 0x7fa7700ca850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34910/0 (socket says 172.21.15.7:34910) 2024-01-27T02:42:01.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.820+0000 7fa76effd700 1 -- 172.21.15.7:0/5833566 learned_addr learned my addr 172.21.15.7:0/5833566 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:42:01.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.820+0000 7fa76ffff700 1 --2- 172.21.15.7:0/5833566 >> v2:172.21.15.188:3300/0 conn(0x7fa7700caf60 0x7fa770164fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:01.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.820+0000 7fa76e7fc700 1 -- 172.21.15.7:0/5833566 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2962308021 0 0) 0x7fa77015c5b0 con 0x7fa77015e920 2024-01-27T02:42:01.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.820+0000 7fa76e7fc700 1 -- 172.21.15.7:0/5833566 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa75c003570 con 0x7fa77015e920 2024-01-27T02:42:01.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.820+0000 7fa76ffff700 1 -- 172.21.15.7:0/5833566 >> v1:172.21.15.188:6789/0 conn(0x7fa7700c97e0 legacy=0x7fa77015afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:01.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.820+0000 7fa76e7fc700 1 -- 172.21.15.7:0/5833566 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3581321741 0 0) 0x7fa77015b9a0 con 0x7fa7700c97e0 2024-01-27T02:42:01.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.820+0000 7fa76ffff700 1 -- 172.21.15.7:0/5833566 >> v1:172.21.15.7:6789/0 conn(0x7fa77015e920 legacy=0x7fa7700ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:01.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.820+0000 7fa76ffff700 1 -- 172.21.15.7:0/5833566 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77015bf60 con 0x7fa7700caf60 2024-01-27T02:42:01.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.820+0000 7fa76ffff700 1 --2- 172.21.15.7:0/5833566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7700caf60 0x7fa770164fd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa768009e60 tx=0x7fa768009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22fa8fb853fcf18 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:01.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.821+0000 7fa76e7fc700 1 -- 172.21.15.7:0/5833566 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76800e070 con 0x7fa7700caf60 2024-01-27T02:42:01.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.821+0000 7fa76e7fc700 1 -- 172.21.15.7:0/5833566 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76800bb60 con 0x7fa7700caf60 2024-01-27T02:42:01.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.821+0000 7fa76e7fc700 1 -- 172.21.15.7:0/5833566 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768012430 con 0x7fa7700caf60 2024-01-27T02:42:01.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.821+0000 7fa775c63700 1 -- 172.21.15.7:0/5833566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7700caf60 msgr2=0x7fa770164fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:01.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.821+0000 7fa775c63700 1 --2- 172.21.15.7:0/5833566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7700caf60 0x7fa770164fd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa768009e60 tx=0x7fa768009780 comp rx=0 tx=0).stop 2024-01-27T02:42:01.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.821+0000 7fa775c63700 1 -- 172.21.15.7:0/5833566 shutdown_connections 2024-01-27T02:42:01.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.821+0000 7fa775c63700 1 --2- 172.21.15.7:0/5833566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7700caf60 0x7fa770164fd0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:01.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.821+0000 7fa775c63700 1 -- 172.21.15.7:0/5833566 >> 172.21.15.7:0/5833566 conn(0x7fa770063580 msgr2=0x7fa770063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:01.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.821+0000 7fa775c63700 1 -- 172.21.15.7:0/5833566 shutdown_connections 2024-01-27T02:42:01.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.821+0000 7fa775c63700 1 -- 172.21.15.7:0/5833566 wait complete. 2024-01-27T02:42:01.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.823+0000 7fa775c63700 1 Processor -- start 2024-01-27T02:42:01.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.823+0000 7fa775c63700 1 -- start start 2024-01-27T02:42:01.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.824+0000 7fa775c63700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7700c97e0 0x7fa7700c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:01.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.824+0000 7fa775c63700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7700caf60 0x7fa77006d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:01.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.824+0000 7fa775c63700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa77015e920 0x7fa7700aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:01.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.824+0000 7fa775c63700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7700c84e0 con 0x7fa77015e920 2024-01-27T02:42:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.824+0000 7fa775c63700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7700afbf0 con 0x7fa7700c97e0 2024-01-27T02:42:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.824+0000 7fa775c63700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7700c74a0 con 0x7fa7700caf60 2024-01-27T02:42:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.824+0000 7fa76ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa77015e920 0x7fa7700aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.824+0000 7fa76ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa77015e920 0x7fa7700aa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50406/0 (socket says 172.21.15.7:50406) 2024-01-27T02:42:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.824+0000 7fa76ffff700 1 -- 172.21.15.7:0/4162900236 learned_addr learned my addr 172.21.15.7:0/4162900236 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:42:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.824+0000 7fa76effd700 1 --2- 172.21.15.7:0/4162900236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7700caf60 0x7fa77006d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.824+0000 7fa76f7fe700 1 --2- 172.21.15.7:0/4162900236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7700c97e0 0x7fa7700c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.824+0000 7fa76ffff700 1 -- 172.21.15.7:0/4162900236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7700caf60 msgr2=0x7fa77006d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.824+0000 7fa76ffff700 1 --2- 172.21.15.7:0/4162900236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7700caf60 0x7fa77006d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.824+0000 7fa76ffff700 1 -- 172.21.15.7:0/4162900236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7700c97e0 msgr2=0x7fa7700c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.824+0000 7fa76ffff700 1 --2- 172.21.15.7:0/4162900236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7700c97e0 0x7fa7700c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.824+0000 7fa76ffff700 1 -- 172.21.15.7:0/4162900236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa768009a20 con 0x7fa77015e920 2024-01-27T02:42:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.824+0000 7fa76ffff700 1 --2- 172.21.15.7:0/4162900236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa77015e920 0x7fa7700aa8c0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fa768013000 tx=0x7fa768019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.825+0000 7fa7677fe700 1 -- 172.21.15.7:0/4162900236 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768026480 con 0x7fa77015e920 2024-01-27T02:42:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.825+0000 7fa7677fe700 1 -- 172.21.15.7:0/4162900236 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa768026600 con 0x7fa77015e920 2024-01-27T02:42:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.825+0000 7fa775c63700 1 -- 172.21.15.7:0/4162900236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa77006e270 con 0x7fa77015e920 2024-01-27T02:42:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.825+0000 7fa7677fe700 1 -- 172.21.15.7:0/4162900236 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7680269c0 con 0x7fa77015e920 2024-01-27T02:42:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.825+0000 7fa775c63700 1 -- 172.21.15.7:0/4162900236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa77006e830 con 0x7fa77015e920 2024-01-27T02:42:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.826+0000 7fa775c63700 1 -- 172.21.15.7:0/4162900236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7700cbe40 con 0x7fa77015e920 2024-01-27T02:42:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.826+0000 7fa7677fe700 1 -- 172.21.15.7:0/4162900236 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa76801c400 con 0x7fa77015e920 2024-01-27T02:42:01.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.826+0000 7fa7677fe700 1 --2- 172.21.15.7:0/4162900236 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa750066f30 0x7fa7500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:01.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.827+0000 7fa7677fe700 1 -- 172.21.15.7:0/4162900236 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa768086750 con 0x7fa77015e920 2024-01-27T02:42:01.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.827+0000 7fa76f7fe700 1 --2- 172.21.15.7:0/4162900236 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa750066f30 0x7fa7500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:01.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.827+0000 7fa76f7fe700 1 --2- 172.21.15.7:0/4162900236 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa750066f30 0x7fa7500693e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa77016ddc0 tx=0x7fa760009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:01.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:01.831+0000 7fa7677fe700 1 -- 172.21.15.7:0/4162900236 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7680539c0 con 0x7fa77015e920 2024-01-27T02:42:02.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.037+0000 7fa775c63700 1 -- 172.21.15.7:0/4162900236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa77004c710 con 0x7fa77015e920 2024-01-27T02:42:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.038+0000 7fa7677fe700 1 -- 172.21.15.7:0/4162900236 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+15317 (secure 0 0 0) 0x7fa768056fe0 con 0x7fa77015e920 2024-01-27T02:42:02.039 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:42:02.039 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":164,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:41:59.216585+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:41:56.176076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-01-27T02:41:56.620523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:42:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.040+0000 7fa775c63700 1 -- 172.21.15.7:0/4162900236 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa750066f30 msgr2=0x7fa7500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.040+0000 7fa775c63700 1 --2- 172.21.15.7:0/4162900236 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa750066f30 0x7fa7500693e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa77016ddc0 tx=0x7fa760009040 comp rx=0 tx=0).stop 2024-01-27T02:42:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.040+0000 7fa775c63700 1 -- 172.21.15.7:0/4162900236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa77015e920 msgr2=0x7fa7700aa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.040+0000 7fa775c63700 1 --2- 172.21.15.7:0/4162900236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa77015e920 0x7fa7700aa8c0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fa768013000 tx=0x7fa768019000 comp rx=0 tx=0).stop 2024-01-27T02:42:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.040+0000 7fa775c63700 1 -- 172.21.15.7:0/4162900236 shutdown_connections 2024-01-27T02:42:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.040+0000 7fa775c63700 1 --2- 172.21.15.7:0/4162900236 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa750066f30 0x7fa7500693e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.040+0000 7fa775c63700 1 --2- 172.21.15.7:0/4162900236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa77015e920 0x7fa7700aa8c0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.040+0000 7fa775c63700 1 --2- 172.21.15.7:0/4162900236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7700c97e0 0x7fa7700c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.040+0000 7fa775c63700 1 --2- 172.21.15.7:0/4162900236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7700caf60 0x7fa77006d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.040+0000 7fa775c63700 1 -- 172.21.15.7:0/4162900236 >> 172.21.15.7:0/4162900236 conn(0x7fa770063580 msgr2=0x7fa77006a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.040+0000 7fa775c63700 1 -- 172.21.15.7:0/4162900236 shutdown_connections 2024-01-27T02:42:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.040+0000 7fa775c63700 1 -- 172.21.15.7:0/4162900236 wait complete. 2024-01-27T02:42:02.056 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-27T02:42:02.056 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:42:02.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.187+0000 7ff7e9418700 1 Processor -- start 2024-01-27T02:42:02.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.188+0000 7ff7e9418700 1 -- start start 2024-01-27T02:42:02.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.188+0000 7ff7e9418700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7e40a7c10 0x7ff7e40ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:02.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.188+0000 7ff7e9418700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e415a880 con 0x7ff7e40ae880 2024-01-27T02:42:02.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.188+0000 7ff7e9418700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e40af950 con 0x7ff7e40a7fe0 2024-01-27T02:42:02.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.188+0000 7ff7e9418700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e40afde0 con 0x7ff7e40a7c10 2024-01-27T02:42:02.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.189+0000 7ff7e27fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff7e40ae880 0x7ff7e406f360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34924/0 (socket says 172.21.15.7:34924) 2024-01-27T02:42:02.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.189+0000 7ff7e27fc700 1 -- 172.21.15.7:0/3046588575 learned_addr learned my addr 172.21.15.7:0/3046588575 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:42:02.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.189+0000 7ff7e2ffd700 1 --2- 172.21.15.7:0/3046588575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7e40a7c10 0x7ff7e40ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:02.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.189+0000 7ff7e1ffb700 1 -- 172.21.15.7:0/3046588575 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939828426 0 0) 0x7ff7e415a880 con 0x7ff7e40ae880 2024-01-27T02:42:02.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.189+0000 7ff7e1ffb700 1 -- 172.21.15.7:0/3046588575 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7cc003570 con 0x7ff7e40ae880 2024-01-27T02:42:02.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.189+0000 7ff7e1ffb700 1 -- 172.21.15.7:0/3046588575 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1488772031 0 0) 0x7ff7cc003570 con 0x7ff7e40ae880 2024-01-27T02:42:02.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.189+0000 7ff7e1ffb700 1 -- 172.21.15.7:0/3046588575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7e40a7c10 msgr2=0x7ff7e40ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:02.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.189+0000 7ff7e1ffb700 1 --2- 172.21.15.7:0/3046588575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7e40a7c10 0x7ff7e40ae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.189+0000 7ff7e1ffb700 1 -- 172.21.15.7:0/3046588575 >> v1:172.21.15.188:6789/0 conn(0x7ff7e40a7fe0 legacy=0x7ff7e40b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:02.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.189+0000 7ff7e1ffb700 1 -- 172.21.15.7:0/3046588575 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e40aed40 con 0x7ff7e40ae880 2024-01-27T02:42:02.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.189+0000 7ff7e1ffb700 1 -- 172.21.15.7:0/3046588575 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7ff7d8001470 con 0x7ff7e40ae880 2024-01-27T02:42:02.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.189+0000 7ff7e1ffb700 1 -- 172.21.15.7:0/3046588575 >> v1:172.21.15.7:6789/0 conn(0x7ff7e40ae880 legacy=0x7ff7e406f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:02.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.189+0000 7ff7e1ffb700 1 --2- 172.21.15.7:0/3046588575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7cc003a90 0x7ff7cc003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:02.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.189+0000 7ff7e1ffb700 1 --2- 172.21.15.7:0/3046588575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7e40a7c10 0x7ff7cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:02.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.189+0000 7ff7e1ffb700 1 --2- 172.21.15.7:0/3046588575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7cc004d50 0x7ff7cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:02.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.189+0000 7ff7e1ffb700 1 -- 172.21.15.7:0/3046588575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc00bb10 con 0x7ff7cc003a90 2024-01-27T02:42:02.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.189+0000 7ff7e1ffb700 1 -- 172.21.15.7:0/3046588575 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc00bc50 con 0x7ff7cc004d50 2024-01-27T02:42:02.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.189+0000 7ff7e1ffb700 1 -- 172.21.15.7:0/3046588575 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc00be10 con 0x7ff7e40a7c10 2024-01-27T02:42:02.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.189+0000 7ff7e2ffd700 1 --2- 172.21.15.7:0/3046588575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7cc003a90 0x7ff7cc003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:02.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.190+0000 7ff7e27fc700 1 --2- 172.21.15.7:0/3046588575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7e40a7c10 0x7ff7cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:02.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.190+0000 7ff7e37fe700 1 --2- 172.21.15.7:0/3046588575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7cc004d50 0x7ff7cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:02.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.190+0000 7ff7e2ffd700 1 -- 172.21.15.7:0/3046588575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7e40a7c10 msgr2=0x7ff7cc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:02.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.190+0000 7ff7e2ffd700 1 --2- 172.21.15.7:0/3046588575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7e40a7c10 0x7ff7cc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.190+0000 7ff7e2ffd700 1 -- 172.21.15.7:0/3046588575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7cc004d50 msgr2=0x7ff7cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.190+0000 7ff7e2ffd700 1 --2- 172.21.15.7:0/3046588575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7cc004d50 0x7ff7cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.190+0000 7ff7e2ffd700 1 -- 172.21.15.7:0/3046588575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc003570 con 0x7ff7cc003a90 2024-01-27T02:42:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.190+0000 7ff7e2ffd700 1 --2- 172.21.15.7:0/3046588575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7cc003a90 0x7ff7cc003e60 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7ff7d400e210 tx=0x7ff7d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833e115a32d4642e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.190+0000 7ff7e1ffb700 1 -- 172.21.15.7:0/3046588575 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d400eac0 con 0x7ff7cc003a90 2024-01-27T02:42:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.190+0000 7ff7e1ffb700 1 -- 172.21.15.7:0/3046588575 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d400f040 con 0x7ff7cc003a90 2024-01-27T02:42:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.190+0000 7ff7e1ffb700 1 -- 172.21.15.7:0/3046588575 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d400ee70 con 0x7ff7cc003a90 2024-01-27T02:42:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.191+0000 7ff7e9418700 1 -- 172.21.15.7:0/3046588575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7cc003a90 msgr2=0x7ff7cc003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.191+0000 7ff7e9418700 1 --2- 172.21.15.7:0/3046588575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7cc003a90 0x7ff7cc003e60 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7ff7d400e210 tx=0x7ff7d400b040 comp rx=0 tx=0).stop 2024-01-27T02:42:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.191+0000 7ff7e37fe700 1 -- 172.21.15.7:0/3046588575 reap_dead start 2024-01-27T02:42:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.191+0000 7ff7e9418700 1 -- 172.21.15.7:0/3046588575 shutdown_connections 2024-01-27T02:42:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.191+0000 7ff7e9418700 1 -- 172.21.15.7:0/3046588575 >> 172.21.15.7:0/3046588575 conn(0x7ff7e40679b0 msgr2=0x7ff7e4067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.191+0000 7ff7e9418700 1 -- 172.21.15.7:0/3046588575 shutdown_connections 2024-01-27T02:42:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.191+0000 7ff7e9418700 1 -- 172.21.15.7:0/3046588575 wait complete. 2024-01-27T02:42:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.192+0000 7ff7e9418700 1 Processor -- start 2024-01-27T02:42:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.192+0000 7ff7e9418700 1 -- start start 2024-01-27T02:42:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.193+0000 7ff7e9418700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7cc003a90 0x7ff7e415be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.193+0000 7ff7e9418700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7e41565a0 0x7ff7e416b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.193+0000 7ff7e9418700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7e40992f0 0x7ff7e415ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:02.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.193+0000 7ff7e9418700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e416bb70 con 0x7ff7e40992f0 2024-01-27T02:42:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.193+0000 7ff7e9418700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e415b970 con 0x7ff7e41565a0 2024-01-27T02:42:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.193+0000 7ff7e9418700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e415ad60 con 0x7ff7cc003a90 2024-01-27T02:42:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.193+0000 7ff7e37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7e40992f0 0x7ff7e415ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.193+0000 7ff7e37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7e40992f0 0x7ff7e415ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50418/0 (socket says 172.21.15.7:50418) 2024-01-27T02:42:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.193+0000 7ff7e37fe700 1 -- 172.21.15.7:0/542301749 learned_addr learned my addr 172.21.15.7:0/542301749 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:42:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.193+0000 7ff7e2ffd700 1 --2- 172.21.15.7:0/542301749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7cc003a90 0x7ff7e415be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.194+0000 7ff7e27fc700 1 --2- 172.21.15.7:0/542301749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7e41565a0 0x7ff7e416b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.194+0000 7ff7e37fe700 1 -- 172.21.15.7:0/542301749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7cc003a90 msgr2=0x7ff7e415be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.194+0000 7ff7e37fe700 1 --2- 172.21.15.7:0/542301749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7cc003a90 0x7ff7e415be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.194+0000 7ff7e37fe700 1 -- 172.21.15.7:0/542301749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7e41565a0 msgr2=0x7ff7e416b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.194+0000 7ff7e37fe700 1 --2- 172.21.15.7:0/542301749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7e41565a0 0x7ff7e416b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.194+0000 7ff7e37fe700 1 -- 172.21.15.7:0/542301749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e415b040 con 0x7ff7e40992f0 2024-01-27T02:42:02.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.194+0000 7ff7e37fe700 1 --2- 172.21.15.7:0/542301749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7e40992f0 0x7ff7e415ce60 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40a83d0 tx=0x7ff7dc007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.194+0000 7ff7caffd700 1 -- 172.21.15.7:0/542301749 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc018070 con 0x7ff7e40992f0 2024-01-27T02:42:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.194+0000 7ff7caffd700 1 -- 172.21.15.7:0/542301749 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7dc006820 con 0x7ff7e40992f0 2024-01-27T02:42:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.194+0000 7ff7caffd700 1 -- 172.21.15.7:0/542301749 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc006be0 con 0x7ff7e40992f0 2024-01-27T02:42:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.194+0000 7ff7e9418700 1 -- 172.21.15.7:0/542301749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e415b320 con 0x7ff7e40992f0 2024-01-27T02:42:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.194+0000 7ff7e9418700 1 -- 172.21.15.7:0/542301749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e415aa30 con 0x7ff7e40992f0 2024-01-27T02:42:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.195+0000 7ff7e9418700 1 -- 172.21.15.7:0/542301749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e415bb00 con 0x7ff7e40992f0 2024-01-27T02:42:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.196+0000 7ff7caffd700 1 -- 172.21.15.7:0/542301749 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff7dc007230 con 0x7ff7e40992f0 2024-01-27T02:42:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.196+0000 7ff7caffd700 1 --2- 172.21.15.7:0/542301749 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff7d0066f30 0x7ff7d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.196+0000 7ff7caffd700 1 -- 172.21.15.7:0/542301749 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff7dc084310 con 0x7ff7e40992f0 2024-01-27T02:42:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.196+0000 7ff7e2ffd700 1 --2- 172.21.15.7:0/542301749 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff7d0066f30 0x7ff7d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:02.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.197+0000 7ff7e2ffd700 1 --2- 172.21.15.7:0/542301749 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff7d0066f30 0x7ff7d00693e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff7d400afd0 tx=0x7ff7d401a000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:02.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.200+0000 7ff7caffd700 1 -- 172.21.15.7:0/542301749 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7dc051580 con 0x7ff7e40992f0 2024-01-27T02:42:02.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.398+0000 7ff7e9418700 1 -- 172.21.15.7:0/542301749 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff7e404c710 con 0x7ff7d0066f30 2024-01-27T02:42:02.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.407+0000 7ff7caffd700 1 -- 172.21.15.7:0/542301749 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285480 (secure 0 0 0) 0x7ff7e404c710 con 0x7ff7d0066f30 2024-01-27T02:42:02.413 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:42:02.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.411+0000 7ff7e9418700 1 -- 172.21.15.7:0/542301749 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff7d0066f30 msgr2=0x7ff7d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:02.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.411+0000 7ff7e9418700 1 --2- 172.21.15.7:0/542301749 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff7d0066f30 0x7ff7d00693e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff7d400afd0 tx=0x7ff7d401a000 comp rx=0 tx=0).stop 2024-01-27T02:42:02.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.411+0000 7ff7e9418700 1 -- 172.21.15.7:0/542301749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7e40992f0 msgr2=0x7ff7e415ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:02.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.411+0000 7ff7e9418700 1 --2- 172.21.15.7:0/542301749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7e40992f0 0x7ff7e415ce60 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40a83d0 tx=0x7ff7dc007d10 comp rx=0 tx=0).stop 2024-01-27T02:42:02.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.411+0000 7ff7e9418700 1 -- 172.21.15.7:0/542301749 shutdown_connections 2024-01-27T02:42:02.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.411+0000 7ff7e9418700 1 --2- 172.21.15.7:0/542301749 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff7d0066f30 0x7ff7d00693e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.411+0000 7ff7e9418700 1 --2- 172.21.15.7:0/542301749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7e40992f0 0x7ff7e415ce60 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.411+0000 7ff7e9418700 1 --2- 172.21.15.7:0/542301749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7cc003a90 0x7ff7e415be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.411+0000 7ff7e9418700 1 --2- 172.21.15.7:0/542301749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7e41565a0 0x7ff7e416b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.411+0000 7ff7e9418700 1 -- 172.21.15.7:0/542301749 >> 172.21.15.7:0/542301749 conn(0x7ff7e40679b0 msgr2=0x7ff7e406dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.411+0000 7ff7e9418700 1 -- 172.21.15.7:0/542301749 shutdown_connections 2024-01-27T02:42:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.411+0000 7ff7e9418700 1 -- 172.21.15.7:0/542301749 wait complete. 2024-01-27T02:42:02.416 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:42:02.426 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-01-27T02:42:01.718733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8287,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554212,"statfs":{"total":1600000000,"available":1591513548,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.508947"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.237102+0000","last_change":"2024-01-27T02:41:58.236903+0000","last_active":"2024-01-27T02:41:58.237102+0000","last_peered":"2024-01-27T02:41:58.237102+0000","last_clean":"2024-01-27T02:41:58.237102+0000","last_became_active":"2024-01-27T02:41:58.236163+0000","last_became_peered":"2024-01-27T02:41:58.236163+0000","last_unstale":"2024-01-27T02:41:58.237102+0000","last_undegraded":"2024-01-27T02:41:58.237102+0000","last_fullsized":"2024-01-27T02:41:58.237102+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:00:01.448473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.229252+0000","last_change":"2024-01-27T02:41:58.229159+0000","last_active":"2024-01-27T02:41:58.229252+0000","last_peered":"2024-01-27T02:41:58.229252+0000","last_clean":"2024-01-27T02:41:58.229252+0000","last_became_active":"2024-01-27T02:41:58.228868+0000","last_became_peered":"2024-01-27T02:41:58.228868+0000","last_unstale":"2024-01-27T02:41:58.229252+0000","last_undegraded":"2024-01-27T02:41:58.229252+0000","last_fullsized":"2024-01-27T02:41:58.229252+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:52:43.126709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.229119+0000","last_change":"2024-01-27T02:41:58.229059+0000","last_active":"2024-01-27T02:41:58.229119+0000","last_peered":"2024-01-27T02:41:58.229119+0000","last_clean":"2024-01-27T02:41:58.229119+0000","last_became_active":"2024-01-27T02:41:58.228650+0000","last_became_peered":"2024-01-27T02:41:58.228650+0000","last_unstale":"2024-01-27T02:41:58.229119+0000","last_undegraded":"2024-01-27T02:41:58.229119+0000","last_fullsized":"2024-01-27T02:41:58.229119+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:24:56.326639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.229468+0000","last_change":"2024-01-27T02:41:58.226840+0000","last_active":"2024-01-27T02:41:58.229468+0000","last_peered":"2024-01-27T02:41:58.229468+0000","last_clean":"2024-01-27T02:41:58.229468+0000","last_became_active":"2024-01-27T02:41:58.226615+0000","last_became_peered":"2024-01-27T02:41:58.226615+0000","last_unstale":"2024-01-27T02:41:58.229468+0000","last_undegraded":"2024-01-27T02:41:58.229468+0000","last_fullsized":"2024-01-27T02:41:58.229468+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:12:17.783382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.228237+0000","last_change":"2024-01-27T02:41:58.227831+0000","last_active":"2024-01-27T02:41:58.228237+0000","last_peered":"2024-01-27T02:41:58.228237+0000","last_clean":"2024-01-27T02:41:58.228237+0000","last_became_active":"2024-01-27T02:41:58.227589+0000","last_became_peered":"2024-01-27T02:41:58.227589+0000","last_unstale":"2024-01-27T02:41:58.228237+0000","last_undegraded":"2024-01-27T02:41:58.228237+0000","last_fullsized":"2024-01-27T02:41:58.228237+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:18:42.076141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.237059+0000","last_change":"2024-01-27T02:41:58.236574+0000","last_active":"2024-01-27T02:41:58.237059+0000","last_peered":"2024-01-27T02:41:58.237059+0000","last_clean":"2024-01-27T02:41:58.237059+0000","last_became_active":"2024-01-27T02:41:58.235034+0000","last_became_peered":"2024-01-27T02:41:58.235034+0000","last_unstale":"2024-01-27T02:41:58.237059+0000","last_undegraded":"2024-01-27T02:41:58.237059+0000","last_fullsized":"2024-01-27T02:41:58.237059+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:43:12.603658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.236500+0000","last_change":"2024-01-27T02:41:58.236424+0000","last_active":"2024-01-27T02:41:58.236500+0000","last_peered":"2024-01-27T02:41:58.236500+0000","last_clean":"2024-01-27T02:41:58.236500+0000","last_became_active":"2024-01-27T02:41:58.236188+0000","last_became_peered":"2024-01-27T02:41:58.236188+0000","last_unstale":"2024-01-27T02:41:58.236500+0000","last_undegraded":"2024-01-27T02:41:58.236500+0000","last_fullsized":"2024-01-27T02:41:58.236500+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:17:07.770031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.228375+0000","last_change":"2024-01-27T02:41:58.228306+0000","last_active":"2024-01-27T02:41:58.228375+0000","last_peered":"2024-01-27T02:41:58.228375+0000","last_clean":"2024-01-27T02:41:58.228375+0000","last_became_active":"2024-01-27T02:41:58.228074+0000","last_became_peered":"2024-01-27T02:41:58.228074+0000","last_unstale":"2024-01-27T02:41:58.228375+0000","last_undegraded":"2024-01-27T02:41:58.228375+0000","last_fullsized":"2024-01-27T02:41:58.228375+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:53:41.533979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.231792+0000","last_change":"2024-01-27T02:41:58.229988+0000","last_active":"2024-01-27T02:41:58.231792+0000","last_peered":"2024-01-27T02:41:58.231792+0000","last_clean":"2024-01-27T02:41:58.231792+0000","last_became_active":"2024-01-27T02:41:58.229734+0000","last_became_peered":"2024-01-27T02:41:58.229734+0000","last_unstale":"2024-01-27T02:41:58.231792+0000","last_undegraded":"2024-01-27T02:41:58.231792+0000","last_fullsized":"2024-01-27T02:41:58.231792+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:15:53.724042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.239634+0000","last_change":"2024-01-27T02:41:58.239527+0000","last_active":"2024-01-27T02:41:58.239634+0000","last_peered":"2024-01-27T02:41:58.239634+0000","last_clean":"2024-01-27T02:41:58.239634+0000","last_became_active":"2024-01-27T02:41:58.238894+0000","last_became_peered":"2024-01-27T02:41:58.238894+0000","last_unstale":"2024-01-27T02:41:58.239634+0000","last_undegraded":"2024-01-27T02:41:58.239634+0000","last_fullsized":"2024-01-27T02:41:58.239634+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:12.009262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.237022+0000","last_change":"2024-01-27T02:41:58.235875+0000","last_active":"2024-01-27T02:41:58.237022+0000","last_peered":"2024-01-27T02:41:58.237022+0000","last_clean":"2024-01-27T02:41:58.237022+0000","last_became_active":"2024-01-27T02:41:58.233509+0000","last_became_peered":"2024-01-27T02:41:58.233509+0000","last_unstale":"2024-01-27T02:41:58.237022+0000","last_undegraded":"2024-01-27T02:41:58.237022+0000","last_fullsized":"2024-01-27T02:41:58.237022+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:12:17.783382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.241279+0000","last_change":"2024-01-27T02:41:58.241228+0000","last_active":"2024-01-27T02:41:58.241279+0000","last_peered":"2024-01-27T02:41:58.241279+0000","last_clean":"2024-01-27T02:41:58.241279+0000","last_became_active":"2024-01-27T02:41:58.240602+0000","last_became_peered":"2024-01-27T02:41:58.240602+0000","last_unstale":"2024-01-27T02:41:58.241279+0000","last_undegraded":"2024-01-27T02:41:58.241279+0000","last_fullsized":"2024-01-27T02:41:58.241279+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:46:14.298161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.237323+0000","last_change":"2024-01-27T02:41:58.237218+0000","last_active":"2024-01-27T02:41:58.237323+0000","last_peered":"2024-01-27T02:41:58.237323+0000","last_clean":"2024-01-27T02:41:58.237323+0000","last_became_active":"2024-01-27T02:41:58.236703+0000","last_became_peered":"2024-01-27T02:41:58.236703+0000","last_unstale":"2024-01-27T02:41:58.237323+0000","last_undegraded":"2024-01-27T02:41:58.237323+0000","last_fullsized":"2024-01-27T02:41:58.237323+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:48:11.363286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.235565+0000","last_change":"2024-01-27T02:41:58.235283+0000","last_active":"2024-01-27T02:41:58.235565+0000","last_peered":"2024-01-27T02:41:58.235565+0000","last_clean":"2024-01-27T02:41:58.235565+0000","last_became_active":"2024-01-27T02:41:58.235040+0000","last_became_peered":"2024-01-27T02:41:58.235040+0000","last_unstale":"2024-01-27T02:41:58.235565+0000","last_undegraded":"2024-01-27T02:41:58.235565+0000","last_fullsized":"2024-01-27T02:41:58.235565+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:02:28.112417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.239316+0000","last_change":"2024-01-27T02:41:58.239145+0000","last_active":"2024-01-27T02:41:58.239316+0000","last_peered":"2024-01-27T02:41:58.239316+0000","last_clean":"2024-01-27T02:41:58.239316+0000","last_became_active":"2024-01-27T02:41:58.235924+0000","last_became_peered":"2024-01-27T02:41:58.235924+0000","last_unstale":"2024-01-27T02:41:58.239316+0000","last_undegraded":"2024-01-27T02:41:58.239316+0000","last_fullsized":"2024-01-27T02:41:58.239316+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:36:27.403658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.230941+0000","last_change":"2024-01-27T02:41:58.230747+0000","last_active":"2024-01-27T02:41:58.230941+0000","last_peered":"2024-01-27T02:41:58.230941+0000","last_clean":"2024-01-27T02:41:58.230941+0000","last_became_active":"2024-01-27T02:41:58.230487+0000","last_became_peered":"2024-01-27T02:41:58.230487+0000","last_unstale":"2024-01-27T02:41:58.230941+0000","last_undegraded":"2024-01-27T02:41:58.230941+0000","last_fullsized":"2024-01-27T02:41:58.230941+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:54:28.400528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.241420+0000","last_change":"2024-01-27T02:41:58.241349+0000","last_active":"2024-01-27T02:41:58.241420+0000","last_peered":"2024-01-27T02:41:58.241420+0000","last_clean":"2024-01-27T02:41:58.241420+0000","last_became_active":"2024-01-27T02:41:58.240923+0000","last_became_peered":"2024-01-27T02:41:58.240923+0000","last_unstale":"2024-01-27T02:41:58.241420+0000","last_undegraded":"2024-01-27T02:41:58.241420+0000","last_fullsized":"2024-01-27T02:41:58.241420+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:03:29.567386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.228959+0000","last_change":"2024-01-27T02:41:58.228876+0000","last_active":"2024-01-27T02:41:58.228959+0000","last_peered":"2024-01-27T02:41:58.228959+0000","last_clean":"2024-01-27T02:41:58.228959+0000","last_became_active":"2024-01-27T02:41:58.228311+0000","last_became_peered":"2024-01-27T02:41:58.228311+0000","last_unstale":"2024-01-27T02:41:58.228959+0000","last_undegraded":"2024-01-27T02:41:58.228959+0000","last_fullsized":"2024-01-27T02:41:58.228959+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:29:41.199197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.234868+0000","last_change":"2024-01-27T02:41:58.234796+0000","last_active":"2024-01-27T02:41:58.234868+0000","last_peered":"2024-01-27T02:41:58.234868+0000","last_clean":"2024-01-27T02:41:58.234868+0000","last_became_active":"2024-01-27T02:41:58.233945+0000","last_became_peered":"2024-01-27T02:41:58.233945+0000","last_unstale":"2024-01-27T02:41:58.234868+0000","last_undegraded":"2024-01-27T02:41:58.234868+0000","last_fullsized":"2024-01-27T02:41:58.234868+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:13:51.409141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.229919+0000","last_change":"2024-01-27T02:41:58.229633+0000","last_active":"2024-01-27T02:41:58.229919+0000","last_peered":"2024-01-27T02:41:58.229919+0000","last_clean":"2024-01-27T02:41:58.229919+0000","last_became_active":"2024-01-27T02:41:58.229112+0000","last_became_peered":"2024-01-27T02:41:58.229112+0000","last_unstale":"2024-01-27T02:41:58.229919+0000","last_undegraded":"2024-01-27T02:41:58.229919+0000","last_fullsized":"2024-01-27T02:41:58.229919+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:50:40.993114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.236637+0000","last_change":"2024-01-27T02:41:58.236532+0000","last_active":"2024-01-27T02:41:58.236637+0000","last_peered":"2024-01-27T02:41:58.236637+0000","last_clean":"2024-01-27T02:41:58.236637+0000","last_became_active":"2024-01-27T02:41:58.235714+0000","last_became_peered":"2024-01-27T02:41:58.235714+0000","last_unstale":"2024-01-27T02:41:58.236637+0000","last_undegraded":"2024-01-27T02:41:58.236637+0000","last_fullsized":"2024-01-27T02:41:58.236637+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:17:07.770031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.231786+0000","last_change":"2024-01-27T02:41:58.231521+0000","last_active":"2024-01-27T02:41:58.231786+0000","last_peered":"2024-01-27T02:41:58.231786+0000","last_clean":"2024-01-27T02:41:58.231786+0000","last_became_active":"2024-01-27T02:41:58.229690+0000","last_became_peered":"2024-01-27T02:41:58.229690+0000","last_unstale":"2024-01-27T02:41:58.231786+0000","last_undegraded":"2024-01-27T02:41:58.231786+0000","last_fullsized":"2024-01-27T02:41:58.231786+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:49:56.921151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.234707+0000","last_change":"2024-01-27T02:41:58.234111+0000","last_active":"2024-01-27T02:41:58.234707+0000","last_peered":"2024-01-27T02:41:58.234707+0000","last_clean":"2024-01-27T02:41:58.234707+0000","last_became_active":"2024-01-27T02:41:58.233623+0000","last_became_peered":"2024-01-27T02:41:58.233623+0000","last_unstale":"2024-01-27T02:41:58.234707+0000","last_undegraded":"2024-01-27T02:41:58.234707+0000","last_fullsized":"2024-01-27T02:41:58.234707+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:29:41.199197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.235823+0000","last_change":"2024-01-27T02:41:58.235526+0000","last_active":"2024-01-27T02:41:58.235823+0000","last_peered":"2024-01-27T02:41:58.235823+0000","last_clean":"2024-01-27T02:41:58.235823+0000","last_became_active":"2024-01-27T02:41:58.235031+0000","last_became_peered":"2024-01-27T02:41:58.235031+0000","last_unstale":"2024-01-27T02:41:58.235823+0000","last_undegraded":"2024-01-27T02:41:58.235823+0000","last_fullsized":"2024-01-27T02:41:58.235823+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:08:12.315061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.229944+0000","last_change":"2024-01-27T02:41:58.229794+0000","last_active":"2024-01-27T02:41:58.229944+0000","last_peered":"2024-01-27T02:41:58.229944+0000","last_clean":"2024-01-27T02:41:58.229944+0000","last_became_active":"2024-01-27T02:41:58.229405+0000","last_became_peered":"2024-01-27T02:41:58.229405+0000","last_unstale":"2024-01-27T02:41:58.229944+0000","last_undegraded":"2024-01-27T02:41:58.229944+0000","last_fullsized":"2024-01-27T02:41:58.229944+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:16:11.939669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.237082+0000","last_change":"2024-01-27T02:41:58.236728+0000","last_active":"2024-01-27T02:41:58.237082+0000","last_peered":"2024-01-27T02:41:58.237082+0000","last_clean":"2024-01-27T02:41:58.237082+0000","last_became_active":"2024-01-27T02:41:58.235399+0000","last_became_peered":"2024-01-27T02:41:58.235399+0000","last_unstale":"2024-01-27T02:41:58.237082+0000","last_undegraded":"2024-01-27T02:41:58.237082+0000","last_fullsized":"2024-01-27T02:41:58.237082+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:43.649919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.232304+0000","last_change":"2024-01-27T02:41:58.231845+0000","last_active":"2024-01-27T02:41:58.232304+0000","last_peered":"2024-01-27T02:41:58.232304+0000","last_clean":"2024-01-27T02:41:58.232304+0000","last_became_active":"2024-01-27T02:41:58.230892+0000","last_became_peered":"2024-01-27T02:41:58.230892+0000","last_unstale":"2024-01-27T02:41:58.232304+0000","last_undegraded":"2024-01-27T02:41:58.232304+0000","last_fullsized":"2024-01-27T02:41:58.232304+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:41:09.872021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.237167+0000","last_change":"2024-01-27T02:41:58.237095+0000","last_active":"2024-01-27T02:41:58.237167+0000","last_peered":"2024-01-27T02:41:58.237167+0000","last_clean":"2024-01-27T02:41:58.237167+0000","last_became_active":"2024-01-27T02:41:58.236780+0000","last_became_peered":"2024-01-27T02:41:58.236780+0000","last_unstale":"2024-01-27T02:41:58.237167+0000","last_undegraded":"2024-01-27T02:41:58.237167+0000","last_fullsized":"2024-01-27T02:41:58.237167+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:12.009262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.234503+0000","last_change":"2024-01-27T02:41:58.234424+0000","last_active":"2024-01-27T02:41:58.234503+0000","last_peered":"2024-01-27T02:41:58.234503+0000","last_clean":"2024-01-27T02:41:58.234503+0000","last_became_active":"2024-01-27T02:41:58.232879+0000","last_became_peered":"2024-01-27T02:41:58.232879+0000","last_unstale":"2024-01-27T02:41:58.234503+0000","last_undegraded":"2024-01-27T02:41:58.234503+0000","last_fullsized":"2024-01-27T02:41:58.234503+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:36:46.237152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.233995+0000","last_change":"2024-01-27T02:41:58.233907+0000","last_active":"2024-01-27T02:41:58.233995+0000","last_peered":"2024-01-27T02:41:58.233995+0000","last_clean":"2024-01-27T02:41:58.233995+0000","last_became_active":"2024-01-27T02:41:58.233433+0000","last_became_peered":"2024-01-27T02:41:58.233433+0000","last_unstale":"2024-01-27T02:41:58.233995+0000","last_undegraded":"2024-01-27T02:41:58.233995+0000","last_fullsized":"2024-01-27T02:41:58.233995+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:11:30.446141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.225354+0000","last_change":"2024-01-27T02:41:58.225158+0000","last_active":"2024-01-27T02:41:58.225354+0000","last_peered":"2024-01-27T02:41:58.225354+0000","last_clean":"2024-01-27T02:41:58.225354+0000","last_became_active":"2024-01-27T02:41:58.224827+0000","last_became_peered":"2024-01-27T02:41:58.224827+0000","last_unstale":"2024-01-27T02:41:58.225354+0000","last_undegraded":"2024-01-27T02:41:58.225354+0000","last_fullsized":"2024-01-27T02:41:58.225354+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:36:27.403658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.232322+0000","last_change":"2024-01-27T02:41:58.232011+0000","last_active":"2024-01-27T02:41:58.232322+0000","last_peered":"2024-01-27T02:41:58.232322+0000","last_clean":"2024-01-27T02:41:58.232322+0000","last_became_active":"2024-01-27T02:41:58.231232+0000","last_became_peered":"2024-01-27T02:41:58.231232+0000","last_unstale":"2024-01-27T02:41:58.232322+0000","last_undegraded":"2024-01-27T02:41:58.232322+0000","last_fullsized":"2024-01-27T02:41:58.232322+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:19:42.020949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.235755+0000","last_change":"2024-01-27T02:41:58.234784+0000","last_active":"2024-01-27T02:41:58.235755+0000","last_peered":"2024-01-27T02:41:58.235755+0000","last_clean":"2024-01-27T02:41:58.235755+0000","last_became_active":"2024-01-27T02:41:58.234030+0000","last_became_peered":"2024-01-27T02:41:58.234030+0000","last_unstale":"2024-01-27T02:41:58.235755+0000","last_undegraded":"2024-01-27T02:41:58.235755+0000","last_fullsized":"2024-01-27T02:41:58.235755+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:15:53.724042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"163'4","reported_seq":14,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.242446+0000","last_change":"2024-01-27T02:41:57.229996+0000","last_active":"2024-01-27T02:41:58.242446+0000","last_peered":"2024-01-27T02:41:58.242446+0000","last_clean":"2024-01-27T02:41:58.242446+0000","last_became_active":"2024-01-27T02:41:57.229757+0000","last_became_peered":"2024-01-27T02:41:57.229757+0000","last_unstale":"2024-01-27T02:41:58.242446+0000","last_undegraded":"2024-01-27T02:41:58.242446+0000","last_fullsized":"2024-01-27T02:41:58.242446+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:12:16.778462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.233014+0000","last_change":"2024-01-27T02:41:58.232947+0000","last_active":"2024-01-27T02:41:58.233014+0000","last_peered":"2024-01-27T02:41:58.233014+0000","last_clean":"2024-01-27T02:41:58.233014+0000","last_became_active":"2024-01-27T02:41:58.232614+0000","last_became_peered":"2024-01-27T02:41:58.232614+0000","last_unstale":"2024-01-27T02:41:58.233014+0000","last_undegraded":"2024-01-27T02:41:58.233014+0000","last_fullsized":"2024-01-27T02:41:58.233014+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:08:12.315061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"163'1","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.238518+0000","last_change":"2024-01-27T02:41:57.229201+0000","last_active":"2024-01-27T02:41:58.238518+0000","last_peered":"2024-01-27T02:41:58.238518+0000","last_clean":"2024-01-27T02:41:58.238518+0000","last_became_active":"2024-01-27T02:41:57.228775+0000","last_became_peered":"2024-01-27T02:41:57.228775+0000","last_unstale":"2024-01-27T02:41:58.238518+0000","last_undegraded":"2024-01-27T02:41:58.238518+0000","last_fullsized":"2024-01-27T02:41:58.238518+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:18:38.936550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.237042+0000","last_change":"2024-01-27T02:41:58.236423+0000","last_active":"2024-01-27T02:41:58.237042+0000","last_peered":"2024-01-27T02:41:58.237042+0000","last_clean":"2024-01-27T02:41:58.237042+0000","last_became_active":"2024-01-27T02:41:58.234458+0000","last_became_peered":"2024-01-27T02:41:58.234458+0000","last_unstale":"2024-01-27T02:41:58.237042+0000","last_undegraded":"2024-01-27T02:41:58.237042+0000","last_fullsized":"2024-01-27T02:41:58.237042+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:49:38.618223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"163'2","reported_seq":12,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.238500+0000","last_change":"2024-01-27T02:41:57.226508+0000","last_active":"2024-01-27T02:41:58.238500+0000","last_peered":"2024-01-27T02:41:58.238500+0000","last_clean":"2024-01-27T02:41:58.238500+0000","last_became_active":"2024-01-27T02:41:57.226240+0000","last_became_peered":"2024-01-27T02:41:57.226240+0000","last_unstale":"2024-01-27T02:41:58.238500+0000","last_undegraded":"2024-01-27T02:41:58.238500+0000","last_fullsized":"2024-01-27T02:41:58.238500+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:48:10.358366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.239451+0000","last_change":"2024-01-27T02:41:58.239364+0000","last_active":"2024-01-27T02:41:58.239451+0000","last_peered":"2024-01-27T02:41:58.239451+0000","last_clean":"2024-01-27T02:41:58.239451+0000","last_became_active":"2024-01-27T02:41:58.238586+0000","last_became_peered":"2024-01-27T02:41:58.238586+0000","last_unstale":"2024-01-27T02:41:58.239451+0000","last_undegraded":"2024-01-27T02:41:58.239451+0000","last_fullsized":"2024-01-27T02:41:58.239451+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:38:12.599262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"163'2","reported_seq":12,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.236396+0000","last_change":"2024-01-27T02:41:57.231250+0000","last_active":"2024-01-27T02:41:58.236396+0000","last_peered":"2024-01-27T02:41:58.236396+0000","last_clean":"2024-01-27T02:41:58.236396+0000","last_became_active":"2024-01-27T02:41:57.230763+0000","last_became_peered":"2024-01-27T02:41:57.230763+0000","last_unstale":"2024-01-27T02:41:58.236396+0000","last_undegraded":"2024-01-27T02:41:58.236396+0000","last_fullsized":"2024-01-27T02:41:58.236396+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:10:51.840479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.227064+0000","last_change":"2024-01-27T02:41:58.226654+0000","last_active":"2024-01-27T02:41:58.227064+0000","last_peered":"2024-01-27T02:41:58.227064+0000","last_clean":"2024-01-27T02:41:58.227064+0000","last_became_active":"2024-01-27T02:41:58.226406+0000","last_became_peered":"2024-01-27T02:41:58.226406+0000","last_unstale":"2024-01-27T02:41:58.227064+0000","last_undegraded":"2024-01-27T02:41:58.227064+0000","last_fullsized":"2024-01-27T02:41:58.227064+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:46:14.298161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"163'1","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.238982+0000","last_change":"2024-01-27T02:41:57.231266+0000","last_active":"2024-01-27T02:41:58.238982+0000","last_peered":"2024-01-27T02:41:58.238982+0000","last_clean":"2024-01-27T02:41:58.238982+0000","last_became_active":"2024-01-27T02:41:57.230744+0000","last_became_peered":"2024-01-27T02:41:57.230744+0000","last_unstale":"2024-01-27T02:41:58.238982+0000","last_undegraded":"2024-01-27T02:41:58.238982+0000","last_fullsized":"2024-01-27T02:41:58.238982+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:11:25.491467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.232727+0000","last_change":"2024-01-27T02:41:58.232556+0000","last_active":"2024-01-27T02:41:58.232727+0000","last_peered":"2024-01-27T02:41:58.232727+0000","last_clean":"2024-01-27T02:41:58.232727+0000","last_became_active":"2024-01-27T02:41:58.232319+0000","last_became_peered":"2024-01-27T02:41:58.232319+0000","last_unstale":"2024-01-27T02:41:58.232727+0000","last_undegraded":"2024-01-27T02:41:58.232727+0000","last_fullsized":"2024-01-27T02:41:58.232727+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:43:12.603658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"163'1","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.238383+0000","last_change":"2024-01-27T02:41:57.229425+0000","last_active":"2024-01-27T02:41:58.238383+0000","last_peered":"2024-01-27T02:41:58.238383+0000","last_clean":"2024-01-27T02:41:58.238383+0000","last_became_active":"2024-01-27T02:41:57.228786+0000","last_became_peered":"2024-01-27T02:41:57.228786+0000","last_unstale":"2024-01-27T02:41:58.238383+0000","last_undegraded":"2024-01-27T02:41:58.238383+0000","last_fullsized":"2024-01-27T02:41:58.238383+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:19:41.016029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.223536+0000","last_change":"2024-01-27T02:41:58.223426+0000","last_active":"2024-01-27T02:41:58.223536+0000","last_peered":"2024-01-27T02:41:58.223536+0000","last_clean":"2024-01-27T02:41:58.223536+0000","last_became_active":"2024-01-27T02:41:58.222684+0000","last_became_peered":"2024-01-27T02:41:58.222684+0000","last_unstale":"2024-01-27T02:41:58.223536+0000","last_undegraded":"2024-01-27T02:41:58.223536+0000","last_fullsized":"2024-01-27T02:41:58.223536+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:38:12.599262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.221284+0000","last_change":"2024-01-27T02:41:57.228906+0000","last_active":"2024-01-27T02:41:58.221284+0000","last_peered":"2024-01-27T02:41:58.221284+0000","last_clean":"2024-01-27T02:41:58.221284+0000","last_became_active":"2024-01-27T02:41:57.225731+0000","last_became_peered":"2024-01-27T02:41:57.225731+0000","last_unstale":"2024-01-27T02:41:58.221284+0000","last_undegraded":"2024-01-27T02:41:58.221284+0000","last_fullsized":"2024-01-27T02:41:58.221284+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:56:47.517031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.228858+0000","last_change":"2024-01-27T02:41:58.228795+0000","last_active":"2024-01-27T02:41:58.228858+0000","last_peered":"2024-01-27T02:41:58.228858+0000","last_clean":"2024-01-27T02:41:58.228858+0000","last_became_active":"2024-01-27T02:41:58.228570+0000","last_became_peered":"2024-01-27T02:41:58.228570+0000","last_unstale":"2024-01-27T02:41:58.228858+0000","last_undegraded":"2024-01-27T02:41:58.228858+0000","last_fullsized":"2024-01-27T02:41:58.228858+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:36:00.504815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.227225+0000","last_change":"2024-01-27T02:41:57.228643+0000","last_active":"2024-01-27T02:41:58.227225+0000","last_peered":"2024-01-27T02:41:58.227225+0000","last_clean":"2024-01-27T02:41:58.227225+0000","last_became_active":"2024-01-27T02:41:57.228341+0000","last_became_peered":"2024-01-27T02:41:57.228341+0000","last_unstale":"2024-01-27T02:41:58.227225+0000","last_undegraded":"2024-01-27T02:41:58.227225+0000","last_fullsized":"2024-01-27T02:41:58.227225+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:10:18.418960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.232734+0000","last_change":"2024-01-27T02:41:58.231409+0000","last_active":"2024-01-27T02:41:58.232734+0000","last_peered":"2024-01-27T02:41:58.232734+0000","last_clean":"2024-01-27T02:41:58.232734+0000","last_became_active":"2024-01-27T02:41:58.231158+0000","last_became_peered":"2024-01-27T02:41:58.231158+0000","last_unstale":"2024-01-27T02:41:58.232734+0000","last_undegraded":"2024-01-27T02:41:58.232734+0000","last_fullsized":"2024-01-27T02:41:58.232734+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:10:52.845400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.219196+0000","last_change":"2024-01-27T02:41:57.230321+0000","last_active":"2024-01-27T02:41:58.219196+0000","last_peered":"2024-01-27T02:41:58.219196+0000","last_clean":"2024-01-27T02:41:58.219196+0000","last_became_active":"2024-01-27T02:41:57.230006+0000","last_became_peered":"2024-01-27T02:41:57.230006+0000","last_unstale":"2024-01-27T02:41:58.219196+0000","last_undegraded":"2024-01-27T02:41:58.219196+0000","last_fullsized":"2024-01-27T02:41:58.219196+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:21:15.480614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.231225+0000","last_change":"2024-01-27T02:41:58.230653+0000","last_active":"2024-01-27T02:41:58.231225+0000","last_peered":"2024-01-27T02:41:58.231225+0000","last_clean":"2024-01-27T02:41:58.231225+0000","last_became_active":"2024-01-27T02:41:58.230378+0000","last_became_peered":"2024-01-27T02:41:58.230378+0000","last_unstale":"2024-01-27T02:41:58.231225+0000","last_undegraded":"2024-01-27T02:41:58.231225+0000","last_fullsized":"2024-01-27T02:41:58.231225+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:45:13.570151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.235761+0000","last_change":"2024-01-27T02:41:57.233559+0000","last_active":"2024-01-27T02:41:58.235761+0000","last_peered":"2024-01-27T02:41:58.235761+0000","last_clean":"2024-01-27T02:41:58.235761+0000","last_became_active":"2024-01-27T02:41:57.232866+0000","last_became_peered":"2024-01-27T02:41:57.232866+0000","last_unstale":"2024-01-27T02:41:58.235761+0000","last_undegraded":"2024-01-27T02:41:58.235761+0000","last_fullsized":"2024-01-27T02:41:58.235761+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:02:27.107496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.234618+0000","last_change":"2024-01-27T02:41:58.234566+0000","last_active":"2024-01-27T02:41:58.234618+0000","last_peered":"2024-01-27T02:41:58.234618+0000","last_clean":"2024-01-27T02:41:58.234618+0000","last_became_active":"2024-01-27T02:41:58.234317+0000","last_became_peered":"2024-01-27T02:41:58.234317+0000","last_unstale":"2024-01-27T02:41:58.234618+0000","last_undegraded":"2024-01-27T02:41:58.234618+0000","last_fullsized":"2024-01-27T02:41:58.234618+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:37:54.293485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"163'6","reported_seq":16,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.240271+0000","last_change":"2024-01-27T02:41:57.225500+0000","last_active":"2024-01-27T02:41:58.240271+0000","last_peered":"2024-01-27T02:41:58.240271+0000","last_clean":"2024-01-27T02:41:58.240271+0000","last_became_active":"2024-01-27T02:41:57.225243+0000","last_became_peered":"2024-01-27T02:41:57.225243+0000","last_unstale":"2024-01-27T02:41:58.240271+0000","last_undegraded":"2024-01-27T02:41:58.240271+0000","last_fullsized":"2024-01-27T02:41:58.240271+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:55:23.803469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.234975+0000","last_change":"2024-01-27T02:41:58.233414+0000","last_active":"2024-01-27T02:41:58.234975+0000","last_peered":"2024-01-27T02:41:58.234975+0000","last_clean":"2024-01-27T02:41:58.234975+0000","last_became_active":"2024-01-27T02:41:58.233182+0000","last_became_peered":"2024-01-27T02:41:58.233182+0000","last_unstale":"2024-01-27T02:41:58.234975+0000","last_undegraded":"2024-01-27T02:41:58.234975+0000","last_fullsized":"2024-01-27T02:41:58.234975+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:20:57.415345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"163'2","reported_seq":12,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.233497+0000","last_change":"2024-01-27T02:41:57.222999+0000","last_active":"2024-01-27T02:41:58.233497+0000","last_peered":"2024-01-27T02:41:58.233497+0000","last_clean":"2024-01-27T02:41:58.233497+0000","last_became_active":"2024-01-27T02:41:57.222807+0000","last_became_peered":"2024-01-27T02:41:57.222807+0000","last_unstale":"2024-01-27T02:41:58.233497+0000","last_undegraded":"2024-01-27T02:41:58.233497+0000","last_fullsized":"2024-01-27T02:41:58.233497+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:18:27.914551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.228808+0000","last_change":"2024-01-27T02:41:58.228731+0000","last_active":"2024-01-27T02:41:58.228808+0000","last_peered":"2024-01-27T02:41:58.228808+0000","last_clean":"2024-01-27T02:41:58.228808+0000","last_became_active":"2024-01-27T02:41:58.228475+0000","last_became_peered":"2024-01-27T02:41:58.228475+0000","last_unstale":"2024-01-27T02:41:58.228808+0000","last_undegraded":"2024-01-27T02:41:58.228808+0000","last_fullsized":"2024-01-27T02:41:58.228808+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:12.009262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.223674+0000","last_change":"2024-01-27T02:41:57.225422+0000","last_active":"2024-01-27T02:41:58.223674+0000","last_peered":"2024-01-27T02:41:58.223674+0000","last_clean":"2024-01-27T02:41:58.223674+0000","last_became_active":"2024-01-27T02:41:57.224949+0000","last_became_peered":"2024-01-27T02:41:57.224949+0000","last_unstale":"2024-01-27T02:41:58.223674+0000","last_undegraded":"2024-01-27T02:41:58.223674+0000","last_fullsized":"2024-01-27T02:41:58.223674+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:13:50.404221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.237308+0000","last_change":"2024-01-27T02:41:58.237050+0000","last_active":"2024-01-27T02:41:58.237308+0000","last_peered":"2024-01-27T02:41:58.237308+0000","last_clean":"2024-01-27T02:41:58.237308+0000","last_became_active":"2024-01-27T02:41:58.236388+0000","last_became_peered":"2024-01-27T02:41:58.236388+0000","last_unstale":"2024-01-27T02:41:58.237308+0000","last_undegraded":"2024-01-27T02:41:58.237308+0000","last_fullsized":"2024-01-27T02:41:58.237308+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:10:19.423880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.222725+0000","last_change":"2024-01-27T02:41:57.223933+0000","last_active":"2024-01-27T02:41:58.222725+0000","last_peered":"2024-01-27T02:41:58.222725+0000","last_clean":"2024-01-27T02:41:58.222725+0000","last_became_active":"2024-01-27T02:41:57.223693+0000","last_became_peered":"2024-01-27T02:41:57.223693+0000","last_unstale":"2024-01-27T02:41:58.222725+0000","last_undegraded":"2024-01-27T02:41:58.222725+0000","last_fullsized":"2024-01-27T02:41:58.222725+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:34:18.049031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.232161+0000","last_change":"2024-01-27T02:41:58.231557+0000","last_active":"2024-01-27T02:41:58.232161+0000","last_peered":"2024-01-27T02:41:58.232161+0000","last_clean":"2024-01-27T02:41:58.232161+0000","last_became_active":"2024-01-27T02:41:58.231312+0000","last_became_peered":"2024-01-27T02:41:58.231312+0000","last_unstale":"2024-01-27T02:41:58.232161+0000","last_undegraded":"2024-01-27T02:41:58.232161+0000","last_fullsized":"2024-01-27T02:41:58.232161+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:06:04.234073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"163'8","reported_seq":18,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.239917+0000","last_change":"2024-01-27T02:41:57.227028+0000","last_active":"2024-01-27T02:41:58.239917+0000","last_peered":"2024-01-27T02:41:58.239917+0000","last_clean":"2024-01-27T02:41:58.239917+0000","last_became_active":"2024-01-27T02:41:57.226710+0000","last_became_peered":"2024-01-27T02:41:57.226710+0000","last_unstale":"2024-01-27T02:41:58.239917+0000","last_undegraded":"2024-01-27T02:41:58.239917+0000","last_fullsized":"2024-01-27T02:41:58.239917+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:25:17.160949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.231834+0000","last_change":"2024-01-27T02:41:58.231707+0000","last_active":"2024-01-27T02:41:58.231834+0000","last_peered":"2024-01-27T02:41:58.231834+0000","last_clean":"2024-01-27T02:41:58.231834+0000","last_became_active":"2024-01-27T02:41:58.231476+0000","last_became_peered":"2024-01-27T02:41:58.231476+0000","last_unstale":"2024-01-27T02:41:58.231834+0000","last_undegraded":"2024-01-27T02:41:58.231834+0000","last_fullsized":"2024-01-27T02:41:58.231834+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:56:27.990324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"163'1","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.235061+0000","last_change":"2024-01-27T02:41:57.225058+0000","last_active":"2024-01-27T02:41:58.235061+0000","last_peered":"2024-01-27T02:41:58.235061+0000","last_clean":"2024-01-27T02:41:58.235061+0000","last_became_active":"2024-01-27T02:41:57.224513+0000","last_became_peered":"2024-01-27T02:41:57.224513+0000","last_unstale":"2024-01-27T02:41:58.235061+0000","last_undegraded":"2024-01-27T02:41:58.235061+0000","last_fullsized":"2024-01-27T02:41:58.235061+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:41:08.867101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.231668+0000","last_change":"2024-01-27T02:41:58.231587+0000","last_active":"2024-01-27T02:41:58.231668+0000","last_peered":"2024-01-27T02:41:58.231668+0000","last_clean":"2024-01-27T02:41:58.231668+0000","last_became_active":"2024-01-27T02:41:58.231283+0000","last_became_peered":"2024-01-27T02:41:58.231283+0000","last_unstale":"2024-01-27T02:41:58.231668+0000","last_undegraded":"2024-01-27T02:41:58.231668+0000","last_fullsized":"2024-01-27T02:41:58.231668+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:28:07.248554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"163'2","reported_seq":12,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.234920+0000","last_change":"2024-01-27T02:41:57.222690+0000","last_active":"2024-01-27T02:41:58.234920+0000","last_peered":"2024-01-27T02:41:58.234920+0000","last_clean":"2024-01-27T02:41:58.234920+0000","last_became_active":"2024-01-27T02:41:57.222033+0000","last_became_peered":"2024-01-27T02:41:57.222033+0000","last_unstale":"2024-01-27T02:41:58.234920+0000","last_undegraded":"2024-01-27T02:41:58.234920+0000","last_fullsized":"2024-01-27T02:41:58.234920+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:16:05.334560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.234867+0000","last_change":"2024-01-27T02:41:58.232682+0000","last_active":"2024-01-27T02:41:58.234867+0000","last_peered":"2024-01-27T02:41:58.234867+0000","last_clean":"2024-01-27T02:41:58.234867+0000","last_became_active":"2024-01-27T02:41:58.232126+0000","last_became_peered":"2024-01-27T02:41:58.232126+0000","last_unstale":"2024-01-27T02:41:58.234867+0000","last_undegraded":"2024-01-27T02:41:58.234867+0000","last_fullsized":"2024-01-27T02:41:58.234867+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:02:28.112417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"163'1","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.237621+0000","last_change":"2024-01-27T02:41:57.230038+0000","last_active":"2024-01-27T02:41:58.237621+0000","last_peered":"2024-01-27T02:41:58.237621+0000","last_clean":"2024-01-27T02:41:58.237621+0000","last_became_active":"2024-01-27T02:41:57.229763+0000","last_became_peered":"2024-01-27T02:41:57.229763+0000","last_unstale":"2024-01-27T02:41:58.237621+0000","last_undegraded":"2024-01-27T02:41:58.237621+0000","last_fullsized":"2024-01-27T02:41:58.237621+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:40:29.355463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.228056+0000","last_change":"2024-01-27T02:41:58.227519+0000","last_active":"2024-01-27T02:41:58.228056+0000","last_peered":"2024-01-27T02:41:58.228056+0000","last_clean":"2024-01-27T02:41:58.228056+0000","last_became_active":"2024-01-27T02:41:58.227286+0000","last_became_peered":"2024-01-27T02:41:58.227286+0000","last_unstale":"2024-01-27T02:41:58.228056+0000","last_undegraded":"2024-01-27T02:41:58.228056+0000","last_fullsized":"2024-01-27T02:41:58.228056+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:16:40.241138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.218431+0000","last_change":"2024-01-27T02:41:57.231163+0000","last_active":"2024-01-27T02:41:58.218431+0000","last_peered":"2024-01-27T02:41:58.218431+0000","last_clean":"2024-01-27T02:41:58.218431+0000","last_became_active":"2024-01-27T02:41:57.230980+0000","last_became_peered":"2024-01-27T02:41:57.230980+0000","last_unstale":"2024-01-27T02:41:58.218431+0000","last_undegraded":"2024-01-27T02:41:58.218431+0000","last_fullsized":"2024-01-27T02:41:58.218431+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:40:27.220792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.232404+0000","last_change":"2024-01-27T02:41:58.232049+0000","last_active":"2024-01-27T02:41:58.232404+0000","last_peered":"2024-01-27T02:41:58.232404+0000","last_clean":"2024-01-27T02:41:58.232404+0000","last_became_active":"2024-01-27T02:41:58.231836+0000","last_became_peered":"2024-01-27T02:41:58.231836+0000","last_unstale":"2024-01-27T02:41:58.232404+0000","last_undegraded":"2024-01-27T02:41:58.232404+0000","last_fullsized":"2024-01-27T02:41:58.232404+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:20:57.415345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"163'2","reported_seq":12,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.238377+0000","last_change":"2024-01-27T02:41:57.224112+0000","last_active":"2024-01-27T02:41:58.238377+0000","last_peered":"2024-01-27T02:41:58.238377+0000","last_clean":"2024-01-27T02:41:58.238377+0000","last_became_active":"2024-01-27T02:41:57.223817+0000","last_became_peered":"2024-01-27T02:41:57.223817+0000","last_unstale":"2024-01-27T02:41:58.238377+0000","last_undegraded":"2024-01-27T02:41:58.238377+0000","last_fullsized":"2024-01-27T02:41:58.238377+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:41:51.998569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.238841+0000","last_change":"2024-01-27T02:41:58.238780+0000","last_active":"2024-01-27T02:41:58.238841+0000","last_peered":"2024-01-27T02:41:58.238841+0000","last_clean":"2024-01-27T02:41:58.238841+0000","last_became_active":"2024-01-27T02:41:58.238075+0000","last_became_peered":"2024-01-27T02:41:58.238075+0000","last_unstale":"2024-01-27T02:41:58.238841+0000","last_undegraded":"2024-01-27T02:41:58.238841+0000","last_fullsized":"2024-01-27T02:41:58.238841+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:03:29.567386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.225949+0000","last_change":"2024-01-27T02:41:57.230721+0000","last_active":"2024-01-27T02:41:58.225949+0000","last_peered":"2024-01-27T02:41:58.225949+0000","last_clean":"2024-01-27T02:41:58.225949+0000","last_became_active":"2024-01-27T02:41:57.230457+0000","last_became_peered":"2024-01-27T02:41:57.230457+0000","last_unstale":"2024-01-27T02:41:58.225949+0000","last_undegraded":"2024-01-27T02:41:58.225949+0000","last_fullsized":"2024-01-27T02:41:58.225949+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:22:02.038987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.231762+0000","last_change":"2024-01-27T02:41:58.230958+0000","last_active":"2024-01-27T02:41:58.231762+0000","last_peered":"2024-01-27T02:41:58.231762+0000","last_clean":"2024-01-27T02:41:58.231762+0000","last_became_active":"2024-01-27T02:41:58.230634+0000","last_became_peered":"2024-01-27T02:41:58.230634+0000","last_unstale":"2024-01-27T02:41:58.231762+0000","last_undegraded":"2024-01-27T02:41:58.231762+0000","last_fullsized":"2024-01-27T02:41:58.231762+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:24:56.326639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.225646+0000","last_change":"2024-01-27T02:41:57.231323+0000","last_active":"2024-01-27T02:41:58.225646+0000","last_peered":"2024-01-27T02:41:58.225646+0000","last_clean":"2024-01-27T02:41:58.225646+0000","last_became_active":"2024-01-27T02:41:57.231103+0000","last_became_peered":"2024-01-27T02:41:57.231103+0000","last_unstale":"2024-01-27T02:41:58.225646+0000","last_undegraded":"2024-01-27T02:41:58.225646+0000","last_fullsized":"2024-01-27T02:41:58.225646+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:06:03.229153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.233362+0000","last_change":"2024-01-27T02:41:58.233254+0000","last_active":"2024-01-27T02:41:58.233362+0000","last_peered":"2024-01-27T02:41:58.233362+0000","last_clean":"2024-01-27T02:41:58.233362+0000","last_became_active":"2024-01-27T02:41:58.232847+0000","last_became_peered":"2024-01-27T02:41:58.232847+0000","last_unstale":"2024-01-27T02:41:58.233362+0000","last_undegraded":"2024-01-27T02:41:58.233362+0000","last_fullsized":"2024-01-27T02:41:58.233362+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:15:53.724042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.223776+0000","last_change":"2024-01-27T02:41:57.225212+0000","last_active":"2024-01-27T02:41:58.223776+0000","last_peered":"2024-01-27T02:41:58.223776+0000","last_clean":"2024-01-27T02:41:58.223776+0000","last_became_active":"2024-01-27T02:41:57.224598+0000","last_became_peered":"2024-01-27T02:41:57.224598+0000","last_unstale":"2024-01-27T02:41:58.223776+0000","last_undegraded":"2024-01-27T02:41:58.223776+0000","last_fullsized":"2024-01-27T02:41:58.223776+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:16:54.556346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.237291+0000","last_change":"2024-01-27T02:41:58.236931+0000","last_active":"2024-01-27T02:41:58.237291+0000","last_peered":"2024-01-27T02:41:58.237291+0000","last_clean":"2024-01-27T02:41:58.237291+0000","last_became_active":"2024-01-27T02:41:58.236065+0000","last_became_peered":"2024-01-27T02:41:58.236065+0000","last_unstale":"2024-01-27T02:41:58.237291+0000","last_undegraded":"2024-01-27T02:41:58.237291+0000","last_fullsized":"2024-01-27T02:41:58.237291+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:49:12.407976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"163'2","reported_seq":12,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.242124+0000","last_change":"2024-01-27T02:41:57.233141+0000","last_active":"2024-01-27T02:41:58.242124+0000","last_peered":"2024-01-27T02:41:58.242124+0000","last_clean":"2024-01-27T02:41:58.242124+0000","last_became_active":"2024-01-27T02:41:57.231882+0000","last_became_peered":"2024-01-27T02:41:57.231882+0000","last_unstale":"2024-01-27T02:41:58.242124+0000","last_undegraded":"2024-01-27T02:41:58.242124+0000","last_fullsized":"2024-01-27T02:41:58.242124+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:24:55.321719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.234933+0000","last_change":"2024-01-27T02:41:58.232825+0000","last_active":"2024-01-27T02:41:58.234933+0000","last_peered":"2024-01-27T02:41:58.234933+0000","last_clean":"2024-01-27T02:41:58.234933+0000","last_became_active":"2024-01-27T02:41:58.232450+0000","last_became_peered":"2024-01-27T02:41:58.232450+0000","last_unstale":"2024-01-27T02:41:58.234933+0000","last_undegraded":"2024-01-27T02:41:58.234933+0000","last_fullsized":"2024-01-27T02:41:58.234933+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:11:30.446141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.224456+0000","last_change":"2024-01-27T02:41:57.228792+0000","last_active":"2024-01-27T02:41:58.224456+0000","last_peered":"2024-01-27T02:41:58.224456+0000","last_clean":"2024-01-27T02:41:58.224456+0000","last_became_active":"2024-01-27T02:41:57.228498+0000","last_became_peered":"2024-01-27T02:41:57.228498+0000","last_unstale":"2024-01-27T02:41:58.224456+0000","last_undegraded":"2024-01-27T02:41:58.224456+0000","last_fullsized":"2024-01-27T02:41:58.224456+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:45:32.325959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.232568+0000","last_change":"2024-01-27T02:41:58.232400+0000","last_active":"2024-01-27T02:41:58.232568+0000","last_peered":"2024-01-27T02:41:58.232568+0000","last_clean":"2024-01-27T02:41:58.232568+0000","last_became_active":"2024-01-27T02:41:58.232159+0000","last_became_peered":"2024-01-27T02:41:58.232159+0000","last_unstale":"2024-01-27T02:41:58.232568+0000","last_undegraded":"2024-01-27T02:41:58.232568+0000","last_fullsized":"2024-01-27T02:41:58.232568+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:03:29.567386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.227175+0000","last_change":"2024-01-27T02:41:57.226223+0000","last_active":"2024-01-27T02:41:58.227175+0000","last_peered":"2024-01-27T02:41:58.227175+0000","last_clean":"2024-01-27T02:41:58.227175+0000","last_became_active":"2024-01-27T02:41:57.225947+0000","last_became_peered":"2024-01-27T02:41:57.225947+0000","last_unstale":"2024-01-27T02:41:58.227175+0000","last_undegraded":"2024-01-27T02:41:58.227175+0000","last_fullsized":"2024-01-27T02:41:58.227175+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:38:16.363217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.234700+0000","last_change":"2024-01-27T02:41:58.234599+0000","last_active":"2024-01-27T02:41:58.234700+0000","last_peered":"2024-01-27T02:41:58.234700+0000","last_clean":"2024-01-27T02:41:58.234700+0000","last_became_active":"2024-01-27T02:41:58.233379+0000","last_became_peered":"2024-01-27T02:41:58.233379+0000","last_unstale":"2024-01-27T02:41:58.234700+0000","last_undegraded":"2024-01-27T02:41:58.234700+0000","last_fullsized":"2024-01-27T02:41:58.234700+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:18:28.919472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.225062+0000","last_change":"2024-01-27T02:41:57.227151+0000","last_active":"2024-01-27T02:41:58.225062+0000","last_peered":"2024-01-27T02:41:58.225062+0000","last_clean":"2024-01-27T02:41:58.225062+0000","last_became_active":"2024-01-27T02:41:57.226892+0000","last_became_peered":"2024-01-27T02:41:57.226892+0000","last_unstale":"2024-01-27T02:41:58.225062+0000","last_undegraded":"2024-01-27T02:41:58.225062+0000","last_fullsized":"2024-01-27T02:41:58.225062+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:16:20.195961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.232338+0000","last_change":"2024-01-27T02:41:58.232150+0000","last_active":"2024-01-27T02:41:58.232338+0000","last_peered":"2024-01-27T02:41:58.232338+0000","last_clean":"2024-01-27T02:41:58.232338+0000","last_became_active":"2024-01-27T02:41:58.231574+0000","last_became_peered":"2024-01-27T02:41:58.231574+0000","last_unstale":"2024-01-27T02:41:58.232338+0000","last_undegraded":"2024-01-27T02:41:58.232338+0000","last_fullsized":"2024-01-27T02:41:58.232338+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:15:13.889275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.227183+0000","last_change":"2024-01-27T02:41:57.230166+0000","last_active":"2024-01-27T02:41:58.227183+0000","last_peered":"2024-01-27T02:41:58.227183+0000","last_clean":"2024-01-27T02:41:58.227183+0000","last_became_active":"2024-01-27T02:41:57.229940+0000","last_became_peered":"2024-01-27T02:41:57.229940+0000","last_unstale":"2024-01-27T02:41:58.227183+0000","last_undegraded":"2024-01-27T02:41:58.227183+0000","last_fullsized":"2024-01-27T02:41:58.227183+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:47:14.775401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.225312+0000","last_change":"2024-01-27T02:41:58.225237+0000","last_active":"2024-01-27T02:41:58.225312+0000","last_peered":"2024-01-27T02:41:58.225312+0000","last_clean":"2024-01-27T02:41:58.225312+0000","last_became_active":"2024-01-27T02:41:58.224897+0000","last_became_peered":"2024-01-27T02:41:58.224897+0000","last_unstale":"2024-01-27T02:41:58.225312+0000","last_undegraded":"2024-01-27T02:41:58.225312+0000","last_fullsized":"2024-01-27T02:41:58.225312+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:05:51.451369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.235982+0000","last_change":"2024-01-27T02:41:57.233354+0000","last_active":"2024-01-27T02:41:58.235982+0000","last_peered":"2024-01-27T02:41:58.235982+0000","last_clean":"2024-01-27T02:41:58.235982+0000","last_became_active":"2024-01-27T02:41:57.232311+0000","last_became_peered":"2024-01-27T02:41:57.232311+0000","last_unstale":"2024-01-27T02:41:58.235982+0000","last_undegraded":"2024-01-27T02:41:58.235982+0000","last_fullsized":"2024-01-27T02:41:58.235982+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:11:29.441221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.235803+0000","last_change":"2024-01-27T02:41:58.235364+0000","last_active":"2024-01-27T02:41:58.235803+0000","last_peered":"2024-01-27T02:41:58.235803+0000","last_clean":"2024-01-27T02:41:58.235803+0000","last_became_active":"2024-01-27T02:41:58.234487+0000","last_became_peered":"2024-01-27T02:41:58.234487+0000","last_unstale":"2024-01-27T02:41:58.235803+0000","last_undegraded":"2024-01-27T02:41:58.235803+0000","last_fullsized":"2024-01-27T02:41:58.235803+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:52:43.126709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.225047+0000","last_change":"2024-01-27T02:41:57.224148+0000","last_active":"2024-01-27T02:41:58.225047+0000","last_peered":"2024-01-27T02:41:58.225047+0000","last_clean":"2024-01-27T02:41:58.225047+0000","last_became_active":"2024-01-27T02:41:57.223447+0000","last_became_peered":"2024-01-27T02:41:57.223447+0000","last_unstale":"2024-01-27T02:41:58.225047+0000","last_undegraded":"2024-01-27T02:41:58.225047+0000","last_fullsized":"2024-01-27T02:41:58.225047+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:36:45.232232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.232752+0000","last_change":"2024-01-27T02:41:58.231765+0000","last_active":"2024-01-27T02:41:58.232752+0000","last_peered":"2024-01-27T02:41:58.232752+0000","last_clean":"2024-01-27T02:41:58.232752+0000","last_became_active":"2024-01-27T02:41:58.231548+0000","last_became_peered":"2024-01-27T02:41:58.231548+0000","last_unstale":"2024-01-27T02:41:58.232752+0000","last_undegraded":"2024-01-27T02:41:58.232752+0000","last_fullsized":"2024-01-27T02:41:58.232752+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:55:28.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.219588+0000","last_change":"2024-01-27T02:41:57.229029+0000","last_active":"2024-01-27T02:41:58.219588+0000","last_peered":"2024-01-27T02:41:58.219588+0000","last_clean":"2024-01-27T02:41:58.219588+0000","last_became_active":"2024-01-27T02:41:57.228428+0000","last_became_peered":"2024-01-27T02:41:57.228428+0000","last_unstale":"2024-01-27T02:41:58.219588+0000","last_undegraded":"2024-01-27T02:41:58.219588+0000","last_fullsized":"2024-01-27T02:41:58.219588+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:28:21.096641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.238864+0000","last_change":"2024-01-27T02:41:58.238781+0000","last_active":"2024-01-27T02:41:58.238864+0000","last_peered":"2024-01-27T02:41:58.238864+0000","last_clean":"2024-01-27T02:41:58.238864+0000","last_became_active":"2024-01-27T02:41:58.237662+0000","last_became_peered":"2024-01-27T02:41:58.237662+0000","last_unstale":"2024-01-27T02:41:58.238864+0000","last_undegraded":"2024-01-27T02:41:58.238864+0000","last_fullsized":"2024-01-27T02:41:58.238864+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:08:12.315061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"163'3","reported_seq":13,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.236554+0000","last_change":"2024-01-27T02:41:57.229465+0000","last_active":"2024-01-27T02:41:58.236554+0000","last_peered":"2024-01-27T02:41:58.236554+0000","last_clean":"2024-01-27T02:41:58.236554+0000","last_became_active":"2024-01-27T02:41:57.229195+0000","last_became_peered":"2024-01-27T02:41:57.229195+0000","last_unstale":"2024-01-27T02:41:58.236554+0000","last_undegraded":"2024-01-27T02:41:58.236554+0000","last_fullsized":"2024-01-27T02:41:58.236554+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:53:40.529059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":288,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.226039+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:41:58.226039+0000","last_peered":"2024-01-27T02:41:58.226039+0000","last_clean":"2024-01-27T02:41:58.226039+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:41:58.226039+0000","last_undegraded":"2024-01-27T02:41:58.226039+0000","last_fullsized":"2024-01-27T02:41:58.226039+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574971,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1468,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193844,"statfs":{"total":200000000,"available":198496370,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:41:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.44500000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.85499999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.48499999999999999,"5min":0.376,"15min":0.30099999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.66700000000000004}]},{"osd":1,"last update":"Sat Jan 27 02:41:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.52000000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76500000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.59599999999999997,"15min":0.56999999999999995},"min":{"1min":0.432,"5min":0.41799999999999998,"15min":0.34999999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.59799999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:41:14 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.441,"5min":0.376,"15min":0.376},"max":{"1min":0.77600000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.60899999999999999,"15min":0.59399999999999997},"min":{"1min":0.51300000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.83899999999999997},"last":0.65700000000000003}]},{"osd":3,"last update":"Sat Jan 27 02:41:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.54800000000000004,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78900000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63,"15min":0.60699999999999998},"min":{"1min":0.50600000000000001,"5min":0.41799999999999998,"15min":0.40999999999999998},"max":{"1min":0.81599999999999995,"5min":0.85099999999999998,"15min":0.88600000000000001},"last":0.69299999999999995}]},{"osd":4,"last update":"Sat Jan 27 02:41:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.48099999999999998,"15min":0.48599999999999999},"min":{"1min":0.29799999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68400000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.37,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70399999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.52300000000000002}]},{"osd":5,"last update":"Sat Jan 27 02:41:14 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.53800000000000003,"15min":0.53500000000000003},"min":{"1min":0.39400000000000002,"5min":0.32400000000000001,"15min":0.307},"max":{"1min":0.75900000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.36599999999999999,"5min":0.33900000000000002,"15min":0.33500000000000002},"max":{"1min":0.72699999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.63100000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:41:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.41399999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.79000000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.42699999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81799999999999995,"5min":0.83499999999999996,"15min":0.90800000000000003},"last":0.68000000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574971,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":891,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194421,"statfs":{"total":200000000,"available":199087216,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63800000000000001,"15min":0.56200000000000006},"min":{"1min":0.46300000000000002,"5min":0.375,"15min":0.29999999999999999},"max":{"1min":0.81499999999999995,"5min":4.125,"15min":4.125},"last":0.623},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60599999999999998,"15min":0.55500000000000005},"min":{"1min":0.439,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73499999999999999,"5min":1.216,"15min":1.216},"last":0.64700000000000002}]},{"osd":1,"last update":"Sat Jan 27 02:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.67400000000000004,"15min":0.57299999999999995},"min":{"1min":0.46400000000000002,"5min":0.39900000000000002,"15min":0.34100000000000003},"max":{"1min":0.81200000000000006,"5min":8.2699999999999996,"15min":8.2699999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.88900000000000001,"15min":0.66800000000000004},"min":{"1min":0.46100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79100000000000004,"5min":26.411999999999999,"15min":26.411999999999999},"last":0.66100000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.76000000000000001,"15min":0.61699999999999999},"min":{"1min":0.39500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78300000000000003,"5min":17.143999999999998,"15min":17.143999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63500000000000001,"15min":0.60199999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83699999999999997,"5min":1.76,"15min":1.76},"last":0.69999999999999996}]},{"osd":3,"last update":"Sat Jan 27 02:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.622,"15min":0.57499999999999996},"min":{"1min":0.47999999999999998,"5min":0.40699999999999997,"15min":0.376},"max":{"1min":0.79800000000000004,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.625,"15min":0.59199999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41199999999999998},"max":{"1min":0.81000000000000005,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.67200000000000004}]},{"osd":4,"last update":"Sat Jan 27 02:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.47199999999999998,"15min":0.48299999999999998},"min":{"1min":0.34699999999999998,"5min":0.311,"15min":0.30199999999999999},"max":{"1min":0.67500000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.50700000000000001,"15min":0.53600000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76500000000000001,"5min":0.84899999999999998,"15min":0.89600000000000002},"last":0.56599999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52300000000000002,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.65800000000000003,"5min":0.80800000000000005,"15min":0.81299999999999994},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.55300000000000005,"15min":0.58699999999999997},"min":{"1min":0.36799999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77000000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.42099999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.57999999999999996,"15min":0.60299999999999998},"min":{"1min":0.40400000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.70499999999999996,"5min":0.88700000000000001,"15min":1.0009999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54800000000000004,"15min":0.57499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78000000000000003,"5min":0.82899999999999996,"15min":1.038},"last":0.51700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":891,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194420,"statfs":{"total":200000000,"available":199086808,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.40799999999999997,"15min":0.38700000000000001},"min":{"1min":0.29299999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.64900000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.42999999999999999,"15min":0.40899999999999997},"min":{"1min":0.34799999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56399999999999995,"5min":1.121,"15min":1.121},"last":0.47099999999999997}]},{"osd":2,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.41999999999999998,"15min":0.38400000000000001},"min":{"1min":0.318,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":0.70199999999999996,"5min":1.25,"15min":1.25},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.45400000000000001,"15min":0.439},"min":{"1min":0.317,"5min":0.29999999999999999,"15min":0.29899999999999999},"max":{"1min":0.66700000000000004,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.40200000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.49399999999999999,"15min":0.47799999999999998},"min":{"1min":0.38400000000000001,"5min":0.33100000000000002,"15min":0.32100000000000001},"max":{"1min":0.64700000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.47599999999999998,"15min":0.46300000000000002},"min":{"1min":0.35099999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.63200000000000001,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.38100000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.68799999999999994,"15min":0.64700000000000002},"min":{"1min":0.54100000000000004,"5min":0.442,"15min":0.373},"max":{"1min":1.075,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.623},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.67800000000000005,"15min":0.65000000000000002},"min":{"1min":0.59999999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97199999999999998,"5min":1.026,"15min":1.026},"last":0.72399999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71399999999999997,"15min":0.65500000000000003},"min":{"1min":0.58299999999999996,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":0.94599999999999995,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71499999999999997,"15min":0.68200000000000005},"min":{"1min":0.52800000000000002,"5min":0.44,"15min":0.42399999999999999},"max":{"1min":1.004,"5min":3.181,"15min":3.181},"last":0.68500000000000005}]},{"osd":6,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.72199999999999998,"15min":0.70499999999999996},"min":{"1min":0.68200000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.75600000000000001,"15min":0.70099999999999996},"min":{"1min":0.65000000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.109,"5min":1.869,"15min":1.869},"last":0.78300000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.754,"15min":0.70599999999999996},"min":{"1min":0.64500000000000002,"5min":0.51800000000000002,"15min":0.47299999999999998},"max":{"1min":1.056,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.73999999999999999,"15min":0.70599999999999996},"min":{"1min":0.66600000000000004,"5min":0.51900000000000002,"15min":0.45300000000000001},"max":{"1min":1.0429999999999999,"5min":1.708,"15min":1.708},"last":0.64800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574971,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1469,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193843,"statfs":{"total":200000000,"available":198495424,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.40000000000000002,"15min":0.39300000000000002},"min":{"1min":0.25700000000000001,"5min":0.252,"15min":0.23799999999999999},"max":{"1min":0.58999999999999997,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.442,"5min":0.433,"15min":0.41999999999999998},"min":{"1min":0.34000000000000002,"5min":0.26000000000000001,"15min":0.223},"max":{"1min":0.65300000000000002,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.41499999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42599999999999999,"15min":0.42299999999999999},"min":{"1min":0.30499999999999999,"5min":0.29899999999999999,"15min":0.28199999999999997},"max":{"1min":0.625,"5min":0.625,"15min":0.64000000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44900000000000001,"15min":0.439},"min":{"1min":0.32300000000000001,"5min":0.308,"15min":0.28799999999999998},"max":{"1min":0.59099999999999997,"5min":0.745,"15min":0.745},"last":0.47599999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.443,"15min":0.435},"min":{"1min":0.315,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.61499999999999999,"5min":0.66100000000000003,"15min":0.67300000000000004},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.495,"15min":0.47999999999999998},"min":{"1min":0.35999999999999999,"5min":0.34300000000000003,"15min":0.29099999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71699999999999997},"last":0.436}]},{"osd":4,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66900000000000004,"15min":0.64900000000000002},"min":{"1min":0.502,"5min":0.432,"15min":0.42099999999999999},"max":{"1min":0.89500000000000002,"5min":0.997,"15min":0.997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65200000000000002,"15min":0.65600000000000003},"min":{"1min":0.48899999999999999,"5min":0.42199999999999999,"15min":0.41999999999999998},"max":{"1min":0.81699999999999995,"5min":0.92100000000000004,"15min":0.96699999999999997},"last":0.57199999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67200000000000004,"15min":0.66100000000000003},"min":{"1min":0.505,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":2.4860000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.68799999999999994,"15min":0.68300000000000005},"min":{"1min":0.51800000000000002,"5min":0.5,"15min":0.44},"max":{"1min":0.86199999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.54300000000000004}]},{"osd":6,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.46899999999999997,"5min":0.45600000000000002,"15min":0.42499999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70199999999999996,"15min":0.68600000000000005},"min":{"1min":0.50700000000000001,"5min":0.46500000000000002,"15min":0.45000000000000001},"max":{"1min":0.96499999999999997,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.58399999999999996}]},{"osd":7,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70399999999999996,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.45900000000000002,"15min":0.42599999999999999},"max":{"1min":0.94999999999999996,"5min":1.014,"15min":1.014},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69499999999999995,"15min":0.69199999999999995},"min":{"1min":0.499,"5min":0.498,"15min":0.498},"max":{"1min":0.89300000000000002,"5min":0.95799999999999996,"15min":0.97299999999999998},"last":0.499}]}]},{"osd":2,"up_from":13,"seq":55834574971,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":892,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194419,"statfs":{"total":200000000,"available":199085756,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.247,"5min":0.23599999999999999,"15min":0.23000000000000001},"max":{"1min":0.59299999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.57199999999999995},"min":{"1min":0.378,"5min":0.32100000000000001,"15min":0.30499999999999999},"max":{"1min":0.67700000000000005,"5min":0.72799999999999998,"15min":6.2729999999999997},"last":0.42499999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.41999999999999998,"15min":0.42099999999999999},"min":{"1min":0.312,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.64100000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.439,"15min":0.44400000000000001},"min":{"1min":0.32200000000000001,"5min":0.29399999999999998,"15min":0.28899999999999998},"max":{"1min":0.58699999999999997,"5min":0.65300000000000002,"15min":0.70099999999999996},"last":0.47899999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.41899999999999998,"15min":0.41699999999999998},"min":{"1min":0.30399999999999999,"5min":0.28499999999999998,"15min":0.27800000000000002},"max":{"1min":0.60699999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.48299999999999998,"15min":0.48499999999999999},"min":{"1min":0.379,"5min":0.314,"15min":0.314},"max":{"1min":0.70699999999999996,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.45200000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67800000000000005,"15min":0.66300000000000003},"min":{"1min":0.501,"5min":0.46800000000000003,"15min":0.39300000000000002},"max":{"1min":0.89200000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.70199999999999996,"15min":0.76400000000000001},"min":{"1min":0.55600000000000005,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92600000000000005,"5min":1.006,"15min":3.581},"last":0.80400000000000005}]},{"osd":5,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.65700000000000003,"15min":0.71099999999999997},"min":{"1min":0.51100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85399999999999998,"5min":0.95899999999999996,"15min":3.5369999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65700000000000003,"15min":0.66700000000000004},"min":{"1min":0.44800000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89000000000000001,"5min":0.92200000000000004,"15min":1.0529999999999999},"last":0.77700000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.67900000000000005,"15min":0.67000000000000004},"min":{"1min":0.48699999999999999,"5min":0.45000000000000001,"15min":0.42299999999999999},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69599999999999995,"15min":0.68500000000000005},"min":{"1min":0.54800000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.72299999999999998}]},{"osd":7,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69899999999999995,"15min":0.68300000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.71899999999999997,"15min":0.69899999999999995},"min":{"1min":0.53200000000000003,"5min":0.441,"15min":0.42699999999999999},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.68400000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574971,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":891,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194421,"statfs":{"total":200000000,"available":199087272,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.46100000000000002,"15min":0.39400000000000002},"min":{"1min":0.27500000000000002,"5min":0.255,"15min":0.253},"max":{"1min":0.54400000000000004,"5min":4.7380000000000004,"15min":4.7380000000000004},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.42799999999999999,"15min":0.40600000000000003},"min":{"1min":0.29899999999999999,"5min":0.27600000000000002,"15min":0.26500000000000001},"max":{"1min":0.58499999999999996,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.57999999999999996}]},{"osd":1,"last update":"Sat Jan 27 02:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.46400000000000002,"15min":0.41799999999999998},"min":{"1min":0.28299999999999997,"5min":0.24399999999999999,"15min":0.22900000000000001},"max":{"1min":0.52200000000000002,"5min":4.5739999999999998,"15min":4.5739999999999998},"last":0.436},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.53300000000000003,"15min":0.45900000000000002},"min":{"1min":0.314,"5min":0.254,"15min":0.254},"max":{"1min":0.65100000000000002,"5min":5.0529999999999999,"15min":5.0529999999999999},"last":0.46100000000000002}]},{"osd":2,"last update":"Sat Jan 27 02:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.49199999999999999,"15min":0.45100000000000001},"min":{"1min":0.40899999999999997,"5min":0.33700000000000002,"15min":0.28100000000000003},"max":{"1min":0.61399999999999999,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.505},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55600000000000005,"15min":0.48599999999999999},"min":{"1min":0.33000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73499999999999999,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":0.52500000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70499999999999996,"15min":0.64400000000000002},"min":{"1min":0.42999999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91100000000000003,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.624},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.629},"min":{"1min":0.49299999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94799999999999995,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.64700000000000002}]},{"osd":5,"last update":"Sat Jan 27 02:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67400000000000004,"15min":0.63500000000000001},"min":{"1min":0.52600000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89000000000000001,"5min":1.409,"15min":1.409},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.79300000000000004,"15min":0.70699999999999996},"min":{"1min":0.48799999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.96599999999999997,"5min":4.7960000000000003,"15min":4.7960000000000003},"last":0.67300000000000004}]},{"osd":6,"last update":"Sat Jan 27 02:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.74299999999999999,"15min":0.68400000000000005},"min":{"1min":0.56599999999999995,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93500000000000005,"5min":5.0069999999999997,"15min":5.0069999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71099999999999997,"15min":0.66600000000000004},"min":{"1min":0.54300000000000004,"5min":0.47099999999999997,"15min":0.435},"max":{"1min":0.85599999999999998,"5min":1.103,"15min":1.103},"last":0.69599999999999995}]},{"osd":7,"last update":"Sat Jan 27 02:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.754,"15min":0.67800000000000005},"min":{"1min":0.495,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92100000000000004,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72699999999999998,"15min":0.68600000000000005},"min":{"1min":0.58699999999999997,"5min":0.435,"15min":0.435},"max":{"1min":0.85999999999999999,"5min":1.361,"15min":1.361},"last":0.59499999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574971,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":891,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194421,"statfs":{"total":200000000,"available":199087362,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.47799999999999998,"5min":0.34599999999999997,"15min":0.314},"max":{"1min":0.77800000000000002,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66300000000000003,"15min":0.61599999999999999},"min":{"1min":0.47399999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.78500000000000003,"5min":7.0670000000000002,"15min":7.0670000000000002},"last":0.51900000000000002}]},{"osd":1,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63800000000000001,"15min":0.61199999999999999},"min":{"1min":0.54900000000000004,"5min":0.39500000000000002,"15min":0.35399999999999998},"max":{"1min":0.74399999999999999,"5min":5.1529999999999996,"15min":5.1529999999999996},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64600000000000002,"15min":0.61199999999999999},"min":{"1min":0.51800000000000002,"5min":0.40799999999999997,"15min":0.39000000000000001},"max":{"1min":0.76900000000000002,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.45200000000000001}]},{"osd":2,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63600000000000001,"15min":0.61199999999999999},"min":{"1min":0.44400000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73999999999999999,"5min":5.0019999999999998,"15min":5.0019999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.628,"15min":0.61799999999999999},"min":{"1min":0.48099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.75900000000000001,"5min":2.073,"15min":2.073},"last":0.60199999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.66400000000000003,"15min":0.60899999999999999},"min":{"1min":0.53000000000000003,"5min":0.42099999999999999,"15min":0.36199999999999999},"max":{"1min":0.76400000000000001,"5min":6.7220000000000004,"15min":6.7220000000000004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.68500000000000005,"15min":0.66000000000000003},"min":{"1min":0.53100000000000003,"5min":0.45400000000000001,"15min":0.41399999999999998},"max":{"1min":0.82399999999999995,"5min":6.5510000000000002,"15min":6.5510000000000002},"last":0.57399999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.496,"15min":0.48799999999999999},"min":{"1min":0.375,"5min":0.312,"15min":0.312},"max":{"1min":0.71299999999999997,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.56200000000000006,"15min":0.51400000000000001},"min":{"1min":0.35799999999999998,"5min":0.32500000000000001,"15min":0.29999999999999999},"max":{"1min":0.69199999999999995,"5min":6.9269999999999996,"15min":6.9269999999999996},"last":0.42399999999999999}]},{"osd":6,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.55700000000000005,"15min":0.51900000000000002},"min":{"1min":0.373,"5min":0.35899999999999999,"15min":0.32900000000000001},"max":{"1min":0.58299999999999996,"5min":5.2229999999999999,"15min":5.2229999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56000000000000005,"15min":0.54700000000000004},"min":{"1min":0.41599999999999998,"5min":0.35099999999999998,"15min":0.33200000000000002},"max":{"1min":0.70299999999999996,"5min":1.46,"15min":1.46},"last":0.65900000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.61099999999999999,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.34799999999999998,"15min":0.33600000000000002},"max":{"1min":0.72199999999999998,"5min":7.5339999999999998,"15min":7.5339999999999998},"last":0.623},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.622,"15min":0.59099999999999997},"min":{"1min":0.442,"5min":0.34699999999999998,"15min":0.34599999999999997},"max":{"1min":0.73999999999999999,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":0.72699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574971,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":891,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194421,"statfs":{"total":200000000,"available":199087340,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.33900000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78300000000000003,"5min":1.071,"15min":1.071},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.58199999999999996},"min":{"1min":0.38500000000000001,"5min":0.34499999999999997,"15min":0.27600000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":3.242},"last":0.59199999999999997}]},{"osd":1,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63100000000000001,"15min":0.62},"min":{"1min":0.496,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78500000000000003,"5min":1.018,"15min":1.3380000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.61499999999999999,"15min":0.64000000000000001},"min":{"1min":0.47999999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.95499999999999996},"last":0.54600000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.59899999999999998},"min":{"1min":0.48999999999999999,"5min":0.39100000000000001,"15min":0.372},"max":{"1min":0.81999999999999995,"5min":0.96699999999999997,"15min":3.04},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.47899999999999998,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":0.79700000000000004,"5min":0.81000000000000005,"15min":0.89400000000000002},"last":0.64000000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60899999999999999,"15min":0.66100000000000003},"min":{"1min":0.48999999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79000000000000004,"5min":1.0409999999999999,"15min":4.032},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65800000000000003,"15min":0.67800000000000005},"min":{"1min":0.48799999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":4.282},"last":0.65400000000000003}]},{"osd":4,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.501,"15min":0.51700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.26500000000000001},"max":{"1min":0.73099999999999998,"5min":0.81399999999999995,"15min":1.2869999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.498,"15min":0.54100000000000004},"min":{"1min":0.376,"5min":0.32800000000000001,"15min":0.30399999999999999},"max":{"1min":0.748,"5min":0.76300000000000001,"15min":1.101},"last":0.70499999999999996}]},{"osd":6,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.496,"15min":0.53600000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.29099999999999998},"max":{"1min":0.75800000000000001,"5min":0.76100000000000001,"15min":1.7969999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55400000000000005,"15min":0.57399999999999995},"min":{"1min":0.40400000000000003,"5min":0.39100000000000001,"15min":0.33600000000000002},"max":{"1min":0.77500000000000002,"5min":0.83399999999999996,"15min":1.6739999999999999},"last":0.627}]},{"osd":7,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.56000000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.32700000000000001},"max":{"1min":0.69799999999999995,"5min":0.745,"15min":1.008},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59399999999999997,"15min":0.63800000000000001},"min":{"1min":0.40600000000000003,"5min":0.38400000000000001,"15min":0.33700000000000002},"max":{"1min":0.76800000000000002,"5min":0.91300000000000003,"15min":2},"last":0.69199999999999995}]}]}],"pool_statfs":[]}} 2024-01-27T02:42:02.431 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:42:02.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.599+0000 7fc508a3f700 1 Processor -- start 2024-01-27T02:42:02.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.599+0000 7fc508a3f700 1 -- start start 2024-01-27T02:42:02.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.600+0000 7fc508a3f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc50406a9f0 0x7fc50415c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:02.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.600+0000 7fc508a3f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc50406adc0 0x7fc504158520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:02.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.600+0000 7fc508a3f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc504158a60 0x7fc504063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:02.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.600+0000 7fc508a3f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc504158f00 con 0x7fc50406a9f0 2024-01-27T02:42:02.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.600+0000 7fc508a3f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc50406c880 con 0x7fc50406adc0 2024-01-27T02:42:02.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.600+0000 7fc508a3f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50415c210 con 0x7fc504158a60 2024-01-27T02:42:02.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.601+0000 7fc50259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc50406a9f0 0x7fc50415c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:02.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.601+0000 7fc50259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc50406a9f0 0x7fc50415c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50420/0 (socket says 172.21.15.7:50420) 2024-01-27T02:42:02.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.601+0000 7fc50259c700 1 -- 172.21.15.7:0/1501140358 learned_addr learned my addr 172.21.15.7:0/1501140358 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:42:02.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.601+0000 7fc501d9b700 1 --2- 172.21.15.7:0/1501140358 >> v2:172.21.15.188:3300/0 conn(0x7fc50406adc0 0x7fc504158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:02.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.601+0000 7fc502d9d700 1 --2- 172.21.15.7:0/1501140358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc504158a60 0x7fc504063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:02.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.601+0000 7fc50259c700 1 -- 172.21.15.7:0/1501140358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc504158a60 msgr2=0x7fc504063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:02.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.601+0000 7fc50259c700 1 --2- 172.21.15.7:0/1501140358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc504158a60 0x7fc504063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.601+0000 7fc50259c700 1 -- 172.21.15.7:0/1501140358 >> v2:172.21.15.188:3300/0 conn(0x7fc50406adc0 msgr2=0x7fc504158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:02.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.601+0000 7fc50259c700 1 --2- 172.21.15.7:0/1501140358 >> v2:172.21.15.188:3300/0 conn(0x7fc50406adc0 0x7fc504158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.601+0000 7fc50259c700 1 -- 172.21.15.7:0/1501140358 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5041593e0 con 0x7fc50406a9f0 2024-01-27T02:42:02.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.601+0000 7fc50259c700 1 --2- 172.21.15.7:0/1501140358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50406a9f0 0x7fc50415c6a0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fc50406dde0 tx=0x7fc4f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd9ddc85d787833 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:02.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.601+0000 7fc50159a700 1 -- 172.21.15.7:0/1501140358 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f400b640 con 0x7fc50406a9f0 2024-01-27T02:42:02.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.601+0000 7fc50159a700 1 -- 172.21.15.7:0/1501140358 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f400b7a0 con 0x7fc50406a9f0 2024-01-27T02:42:02.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.601+0000 7fc50159a700 1 -- 172.21.15.7:0/1501140358 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f400bb60 con 0x7fc50406a9f0 2024-01-27T02:42:02.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.602+0000 7fc508a3f700 1 -- 172.21.15.7:0/1501140358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50406a9f0 msgr2=0x7fc50415c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:02.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.602+0000 7fc508a3f700 1 --2- 172.21.15.7:0/1501140358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50406a9f0 0x7fc50415c6a0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fc50406dde0 tx=0x7fc4f4009250 comp rx=0 tx=0).stop 2024-01-27T02:42:02.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.602+0000 7fc508a3f700 1 -- 172.21.15.7:0/1501140358 shutdown_connections 2024-01-27T02:42:02.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.602+0000 7fc508a3f700 1 --2- 172.21.15.7:0/1501140358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc504158a60 0x7fc504063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.602+0000 7fc508a3f700 1 --2- 172.21.15.7:0/1501140358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50406a9f0 0x7fc50415c6a0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.602+0000 7fc508a3f700 1 --2- 172.21.15.7:0/1501140358 >> v2:172.21.15.188:3300/0 conn(0x7fc50406adc0 0x7fc504158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.602+0000 7fc508a3f700 1 -- 172.21.15.7:0/1501140358 >> 172.21.15.7:0/1501140358 conn(0x7fc5040cdc30 msgr2=0x7fc5040ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.602+0000 7fc508a3f700 1 -- 172.21.15.7:0/1501140358 shutdown_connections 2024-01-27T02:42:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.602+0000 7fc508a3f700 1 -- 172.21.15.7:0/1501140358 wait complete. 2024-01-27T02:42:02.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.604+0000 7fc508a3f700 1 Processor -- start 2024-01-27T02:42:02.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.604+0000 7fc508a3f700 1 -- start start 2024-01-27T02:42:02.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.605+0000 7fc508a3f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc50406a9f0 0x7fc50412f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:02.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.605+0000 7fc508a3f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50406adc0 0x7fc5041684b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:02.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.605+0000 7fc508a3f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc504158a60 0x7fc5040c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:02.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.605+0000 7fc508a3f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5040c7d40 con 0x7fc50406adc0 2024-01-27T02:42:02.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.605+0000 7fc508a3f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5040c7f90 con 0x7fc50406a9f0 2024-01-27T02:42:02.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.605+0000 7fc508a3f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5040c8270 con 0x7fc504158a60 2024-01-27T02:42:02.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.605+0000 7fc501d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50406adc0 0x7fc5041684b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:02.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.605+0000 7fc501d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50406adc0 0x7fc5041684b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50428/0 (socket says 172.21.15.7:50428) 2024-01-27T02:42:02.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.605+0000 7fc501d9b700 1 -- 172.21.15.7:0/666802923 learned_addr learned my addr 172.21.15.7:0/666802923 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:42:02.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.605+0000 7fc50259c700 1 --2- 172.21.15.7:0/666802923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc50406a9f0 0x7fc50412f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:02.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.605+0000 7fc502d9d700 1 --2- 172.21.15.7:0/666802923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc504158a60 0x7fc5040c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:02.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.605+0000 7fc501d9b700 1 -- 172.21.15.7:0/666802923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc504158a60 msgr2=0x7fc5040c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:02.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.605+0000 7fc501d9b700 1 --2- 172.21.15.7:0/666802923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc504158a60 0x7fc5040c6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.605+0000 7fc501d9b700 1 -- 172.21.15.7:0/666802923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc50406a9f0 msgr2=0x7fc50412f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:02.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.605+0000 7fc501d9b700 1 --2- 172.21.15.7:0/666802923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc50406a9f0 0x7fc50412f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.605+0000 7fc501d9b700 1 -- 172.21.15.7:0/666802923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f40094f0 con 0x7fc50406adc0 2024-01-27T02:42:02.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.606+0000 7fc501d9b700 1 --2- 172.21.15.7:0/666802923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50406adc0 0x7fc5041684b0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800e900 tx=0x7fc4f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:02.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.606+0000 7fc4f27fc700 1 -- 172.21.15.7:0/666802923 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f800eea0 con 0x7fc50406adc0 2024-01-27T02:42:02.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.606+0000 7fc4f27fc700 1 -- 172.21.15.7:0/666802923 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f800c850 con 0x7fc50406adc0 2024-01-27T02:42:02.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.606+0000 7fc4f27fc700 1 -- 172.21.15.7:0/666802923 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f800cc10 con 0x7fc50406adc0 2024-01-27T02:42:02.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.606+0000 7fc508a3f700 1 -- 172.21.15.7:0/666802923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5040c8550 con 0x7fc50406adc0 2024-01-27T02:42:02.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.606+0000 7fc508a3f700 1 -- 172.21.15.7:0/666802923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5040c8b10 con 0x7fc50406adc0 2024-01-27T02:42:02.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.607+0000 7fc508a3f700 1 -- 172.21.15.7:0/666802923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5040c8cd0 con 0x7fc50406adc0 2024-01-27T02:42:02.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.607+0000 7fc4f27fc700 1 -- 172.21.15.7:0/666802923 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc4f8013470 con 0x7fc50406adc0 2024-01-27T02:42:02.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.608+0000 7fc4f27fc700 1 --2- 172.21.15.7:0/666802923 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc4e4066f30 0x7fc4e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:02.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.608+0000 7fc4f27fc700 1 -- 172.21.15.7:0/666802923 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc4f80846a0 con 0x7fc50406adc0 2024-01-27T02:42:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.608+0000 7fc50259c700 1 --2- 172.21.15.7:0/666802923 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc4e4066f30 0x7fc4e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.608+0000 7fc50259c700 1 --2- 172.21.15.7:0/666802923 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc4e4066f30 0x7fc4e40693e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc50406dde0 tx=0x7fc4f4017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:02.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.612+0000 7fc4f27fc700 1 -- 172.21.15.7:0/666802923 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc4f8051910 con 0x7fc50406adc0 2024-01-27T02:42:02.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.810+0000 7fc508a3f700 1 -- 172.21.15.7:0/666802923 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc5040c8e80 con 0x7fc4e4066f30 2024-01-27T02:42:02.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.818+0000 7fc4f27fc700 1 -- 172.21.15.7:0/666802923 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285480 (secure 0 0 0) 0x7fc5040c8e80 con 0x7fc4e4066f30 2024-01-27T02:42:02.823 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:42:02.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.822+0000 7fc508a3f700 1 -- 172.21.15.7:0/666802923 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc4e4066f30 msgr2=0x7fc4e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:02.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.822+0000 7fc508a3f700 1 --2- 172.21.15.7:0/666802923 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc4e4066f30 0x7fc4e40693e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc50406dde0 tx=0x7fc4f4017000 comp rx=0 tx=0).stop 2024-01-27T02:42:02.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.822+0000 7fc508a3f700 1 -- 172.21.15.7:0/666802923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50406adc0 msgr2=0x7fc5041684b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:02.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.822+0000 7fc508a3f700 1 --2- 172.21.15.7:0/666802923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50406adc0 0x7fc5041684b0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800e900 tx=0x7fc4f80091a0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.822+0000 7fc508a3f700 1 -- 172.21.15.7:0/666802923 shutdown_connections 2024-01-27T02:42:02.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.822+0000 7fc508a3f700 1 --2- 172.21.15.7:0/666802923 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc4e4066f30 0x7fc4e40693e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.822+0000 7fc508a3f700 1 --2- 172.21.15.7:0/666802923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc504158a60 0x7fc5040c6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.822+0000 7fc508a3f700 1 --2- 172.21.15.7:0/666802923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc50406a9f0 0x7fc50412f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.822+0000 7fc508a3f700 1 --2- 172.21.15.7:0/666802923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50406adc0 0x7fc5041684b0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:02.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.822+0000 7fc508a3f700 1 -- 172.21.15.7:0/666802923 >> 172.21.15.7:0/666802923 conn(0x7fc5040cdc30 msgr2=0x7fc5040b29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:02.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.822+0000 7fc508a3f700 1 -- 172.21.15.7:0/666802923 shutdown_connections 2024-01-27T02:42:02.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:02.822+0000 7fc508a3f700 1 -- 172.21.15.7:0/666802923 wait complete. 2024-01-27T02:42:02.827 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:42:02.837 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-01-27T02:42:01.718733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8287,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554212,"statfs":{"total":1600000000,"available":1591513548,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.508947"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.237102+0000","last_change":"2024-01-27T02:41:58.236903+0000","last_active":"2024-01-27T02:41:58.237102+0000","last_peered":"2024-01-27T02:41:58.237102+0000","last_clean":"2024-01-27T02:41:58.237102+0000","last_became_active":"2024-01-27T02:41:58.236163+0000","last_became_peered":"2024-01-27T02:41:58.236163+0000","last_unstale":"2024-01-27T02:41:58.237102+0000","last_undegraded":"2024-01-27T02:41:58.237102+0000","last_fullsized":"2024-01-27T02:41:58.237102+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:00:01.448473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.229252+0000","last_change":"2024-01-27T02:41:58.229159+0000","last_active":"2024-01-27T02:41:58.229252+0000","last_peered":"2024-01-27T02:41:58.229252+0000","last_clean":"2024-01-27T02:41:58.229252+0000","last_became_active":"2024-01-27T02:41:58.228868+0000","last_became_peered":"2024-01-27T02:41:58.228868+0000","last_unstale":"2024-01-27T02:41:58.229252+0000","last_undegraded":"2024-01-27T02:41:58.229252+0000","last_fullsized":"2024-01-27T02:41:58.229252+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:52:43.126709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.229119+0000","last_change":"2024-01-27T02:41:58.229059+0000","last_active":"2024-01-27T02:41:58.229119+0000","last_peered":"2024-01-27T02:41:58.229119+0000","last_clean":"2024-01-27T02:41:58.229119+0000","last_became_active":"2024-01-27T02:41:58.228650+0000","last_became_peered":"2024-01-27T02:41:58.228650+0000","last_unstale":"2024-01-27T02:41:58.229119+0000","last_undegraded":"2024-01-27T02:41:58.229119+0000","last_fullsized":"2024-01-27T02:41:58.229119+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:24:56.326639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.229468+0000","last_change":"2024-01-27T02:41:58.226840+0000","last_active":"2024-01-27T02:41:58.229468+0000","last_peered":"2024-01-27T02:41:58.229468+0000","last_clean":"2024-01-27T02:41:58.229468+0000","last_became_active":"2024-01-27T02:41:58.226615+0000","last_became_peered":"2024-01-27T02:41:58.226615+0000","last_unstale":"2024-01-27T02:41:58.229468+0000","last_undegraded":"2024-01-27T02:41:58.229468+0000","last_fullsized":"2024-01-27T02:41:58.229468+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:12:17.783382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.228237+0000","last_change":"2024-01-27T02:41:58.227831+0000","last_active":"2024-01-27T02:41:58.228237+0000","last_peered":"2024-01-27T02:41:58.228237+0000","last_clean":"2024-01-27T02:41:58.228237+0000","last_became_active":"2024-01-27T02:41:58.227589+0000","last_became_peered":"2024-01-27T02:41:58.227589+0000","last_unstale":"2024-01-27T02:41:58.228237+0000","last_undegraded":"2024-01-27T02:41:58.228237+0000","last_fullsized":"2024-01-27T02:41:58.228237+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:18:42.076141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.237059+0000","last_change":"2024-01-27T02:41:58.236574+0000","last_active":"2024-01-27T02:41:58.237059+0000","last_peered":"2024-01-27T02:41:58.237059+0000","last_clean":"2024-01-27T02:41:58.237059+0000","last_became_active":"2024-01-27T02:41:58.235034+0000","last_became_peered":"2024-01-27T02:41:58.235034+0000","last_unstale":"2024-01-27T02:41:58.237059+0000","last_undegraded":"2024-01-27T02:41:58.237059+0000","last_fullsized":"2024-01-27T02:41:58.237059+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:43:12.603658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.236500+0000","last_change":"2024-01-27T02:41:58.236424+0000","last_active":"2024-01-27T02:41:58.236500+0000","last_peered":"2024-01-27T02:41:58.236500+0000","last_clean":"2024-01-27T02:41:58.236500+0000","last_became_active":"2024-01-27T02:41:58.236188+0000","last_became_peered":"2024-01-27T02:41:58.236188+0000","last_unstale":"2024-01-27T02:41:58.236500+0000","last_undegraded":"2024-01-27T02:41:58.236500+0000","last_fullsized":"2024-01-27T02:41:58.236500+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:17:07.770031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.228375+0000","last_change":"2024-01-27T02:41:58.228306+0000","last_active":"2024-01-27T02:41:58.228375+0000","last_peered":"2024-01-27T02:41:58.228375+0000","last_clean":"2024-01-27T02:41:58.228375+0000","last_became_active":"2024-01-27T02:41:58.228074+0000","last_became_peered":"2024-01-27T02:41:58.228074+0000","last_unstale":"2024-01-27T02:41:58.228375+0000","last_undegraded":"2024-01-27T02:41:58.228375+0000","last_fullsized":"2024-01-27T02:41:58.228375+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:53:41.533979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.231792+0000","last_change":"2024-01-27T02:41:58.229988+0000","last_active":"2024-01-27T02:41:58.231792+0000","last_peered":"2024-01-27T02:41:58.231792+0000","last_clean":"2024-01-27T02:41:58.231792+0000","last_became_active":"2024-01-27T02:41:58.229734+0000","last_became_peered":"2024-01-27T02:41:58.229734+0000","last_unstale":"2024-01-27T02:41:58.231792+0000","last_undegraded":"2024-01-27T02:41:58.231792+0000","last_fullsized":"2024-01-27T02:41:58.231792+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:15:53.724042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.239634+0000","last_change":"2024-01-27T02:41:58.239527+0000","last_active":"2024-01-27T02:41:58.239634+0000","last_peered":"2024-01-27T02:41:58.239634+0000","last_clean":"2024-01-27T02:41:58.239634+0000","last_became_active":"2024-01-27T02:41:58.238894+0000","last_became_peered":"2024-01-27T02:41:58.238894+0000","last_unstale":"2024-01-27T02:41:58.239634+0000","last_undegraded":"2024-01-27T02:41:58.239634+0000","last_fullsized":"2024-01-27T02:41:58.239634+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:12.009262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.237022+0000","last_change":"2024-01-27T02:41:58.235875+0000","last_active":"2024-01-27T02:41:58.237022+0000","last_peered":"2024-01-27T02:41:58.237022+0000","last_clean":"2024-01-27T02:41:58.237022+0000","last_became_active":"2024-01-27T02:41:58.233509+0000","last_became_peered":"2024-01-27T02:41:58.233509+0000","last_unstale":"2024-01-27T02:41:58.237022+0000","last_undegraded":"2024-01-27T02:41:58.237022+0000","last_fullsized":"2024-01-27T02:41:58.237022+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:12:17.783382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.241279+0000","last_change":"2024-01-27T02:41:58.241228+0000","last_active":"2024-01-27T02:41:58.241279+0000","last_peered":"2024-01-27T02:41:58.241279+0000","last_clean":"2024-01-27T02:41:58.241279+0000","last_became_active":"2024-01-27T02:41:58.240602+0000","last_became_peered":"2024-01-27T02:41:58.240602+0000","last_unstale":"2024-01-27T02:41:58.241279+0000","last_undegraded":"2024-01-27T02:41:58.241279+0000","last_fullsized":"2024-01-27T02:41:58.241279+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:46:14.298161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.237323+0000","last_change":"2024-01-27T02:41:58.237218+0000","last_active":"2024-01-27T02:41:58.237323+0000","last_peered":"2024-01-27T02:41:58.237323+0000","last_clean":"2024-01-27T02:41:58.237323+0000","last_became_active":"2024-01-27T02:41:58.236703+0000","last_became_peered":"2024-01-27T02:41:58.236703+0000","last_unstale":"2024-01-27T02:41:58.237323+0000","last_undegraded":"2024-01-27T02:41:58.237323+0000","last_fullsized":"2024-01-27T02:41:58.237323+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:48:11.363286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.235565+0000","last_change":"2024-01-27T02:41:58.235283+0000","last_active":"2024-01-27T02:41:58.235565+0000","last_peered":"2024-01-27T02:41:58.235565+0000","last_clean":"2024-01-27T02:41:58.235565+0000","last_became_active":"2024-01-27T02:41:58.235040+0000","last_became_peered":"2024-01-27T02:41:58.235040+0000","last_unstale":"2024-01-27T02:41:58.235565+0000","last_undegraded":"2024-01-27T02:41:58.235565+0000","last_fullsized":"2024-01-27T02:41:58.235565+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:02:28.112417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.239316+0000","last_change":"2024-01-27T02:41:58.239145+0000","last_active":"2024-01-27T02:41:58.239316+0000","last_peered":"2024-01-27T02:41:58.239316+0000","last_clean":"2024-01-27T02:41:58.239316+0000","last_became_active":"2024-01-27T02:41:58.235924+0000","last_became_peered":"2024-01-27T02:41:58.235924+0000","last_unstale":"2024-01-27T02:41:58.239316+0000","last_undegraded":"2024-01-27T02:41:58.239316+0000","last_fullsized":"2024-01-27T02:41:58.239316+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:36:27.403658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.230941+0000","last_change":"2024-01-27T02:41:58.230747+0000","last_active":"2024-01-27T02:41:58.230941+0000","last_peered":"2024-01-27T02:41:58.230941+0000","last_clean":"2024-01-27T02:41:58.230941+0000","last_became_active":"2024-01-27T02:41:58.230487+0000","last_became_peered":"2024-01-27T02:41:58.230487+0000","last_unstale":"2024-01-27T02:41:58.230941+0000","last_undegraded":"2024-01-27T02:41:58.230941+0000","last_fullsized":"2024-01-27T02:41:58.230941+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:54:28.400528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.241420+0000","last_change":"2024-01-27T02:41:58.241349+0000","last_active":"2024-01-27T02:41:58.241420+0000","last_peered":"2024-01-27T02:41:58.241420+0000","last_clean":"2024-01-27T02:41:58.241420+0000","last_became_active":"2024-01-27T02:41:58.240923+0000","last_became_peered":"2024-01-27T02:41:58.240923+0000","last_unstale":"2024-01-27T02:41:58.241420+0000","last_undegraded":"2024-01-27T02:41:58.241420+0000","last_fullsized":"2024-01-27T02:41:58.241420+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:03:29.567386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.228959+0000","last_change":"2024-01-27T02:41:58.228876+0000","last_active":"2024-01-27T02:41:58.228959+0000","last_peered":"2024-01-27T02:41:58.228959+0000","last_clean":"2024-01-27T02:41:58.228959+0000","last_became_active":"2024-01-27T02:41:58.228311+0000","last_became_peered":"2024-01-27T02:41:58.228311+0000","last_unstale":"2024-01-27T02:41:58.228959+0000","last_undegraded":"2024-01-27T02:41:58.228959+0000","last_fullsized":"2024-01-27T02:41:58.228959+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:29:41.199197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.234868+0000","last_change":"2024-01-27T02:41:58.234796+0000","last_active":"2024-01-27T02:41:58.234868+0000","last_peered":"2024-01-27T02:41:58.234868+0000","last_clean":"2024-01-27T02:41:58.234868+0000","last_became_active":"2024-01-27T02:41:58.233945+0000","last_became_peered":"2024-01-27T02:41:58.233945+0000","last_unstale":"2024-01-27T02:41:58.234868+0000","last_undegraded":"2024-01-27T02:41:58.234868+0000","last_fullsized":"2024-01-27T02:41:58.234868+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:13:51.409141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.229919+0000","last_change":"2024-01-27T02:41:58.229633+0000","last_active":"2024-01-27T02:41:58.229919+0000","last_peered":"2024-01-27T02:41:58.229919+0000","last_clean":"2024-01-27T02:41:58.229919+0000","last_became_active":"2024-01-27T02:41:58.229112+0000","last_became_peered":"2024-01-27T02:41:58.229112+0000","last_unstale":"2024-01-27T02:41:58.229919+0000","last_undegraded":"2024-01-27T02:41:58.229919+0000","last_fullsized":"2024-01-27T02:41:58.229919+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:50:40.993114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.236637+0000","last_change":"2024-01-27T02:41:58.236532+0000","last_active":"2024-01-27T02:41:58.236637+0000","last_peered":"2024-01-27T02:41:58.236637+0000","last_clean":"2024-01-27T02:41:58.236637+0000","last_became_active":"2024-01-27T02:41:58.235714+0000","last_became_peered":"2024-01-27T02:41:58.235714+0000","last_unstale":"2024-01-27T02:41:58.236637+0000","last_undegraded":"2024-01-27T02:41:58.236637+0000","last_fullsized":"2024-01-27T02:41:58.236637+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:17:07.770031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.231786+0000","last_change":"2024-01-27T02:41:58.231521+0000","last_active":"2024-01-27T02:41:58.231786+0000","last_peered":"2024-01-27T02:41:58.231786+0000","last_clean":"2024-01-27T02:41:58.231786+0000","last_became_active":"2024-01-27T02:41:58.229690+0000","last_became_peered":"2024-01-27T02:41:58.229690+0000","last_unstale":"2024-01-27T02:41:58.231786+0000","last_undegraded":"2024-01-27T02:41:58.231786+0000","last_fullsized":"2024-01-27T02:41:58.231786+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:49:56.921151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.234707+0000","last_change":"2024-01-27T02:41:58.234111+0000","last_active":"2024-01-27T02:41:58.234707+0000","last_peered":"2024-01-27T02:41:58.234707+0000","last_clean":"2024-01-27T02:41:58.234707+0000","last_became_active":"2024-01-27T02:41:58.233623+0000","last_became_peered":"2024-01-27T02:41:58.233623+0000","last_unstale":"2024-01-27T02:41:58.234707+0000","last_undegraded":"2024-01-27T02:41:58.234707+0000","last_fullsized":"2024-01-27T02:41:58.234707+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:29:41.199197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.235823+0000","last_change":"2024-01-27T02:41:58.235526+0000","last_active":"2024-01-27T02:41:58.235823+0000","last_peered":"2024-01-27T02:41:58.235823+0000","last_clean":"2024-01-27T02:41:58.235823+0000","last_became_active":"2024-01-27T02:41:58.235031+0000","last_became_peered":"2024-01-27T02:41:58.235031+0000","last_unstale":"2024-01-27T02:41:58.235823+0000","last_undegraded":"2024-01-27T02:41:58.235823+0000","last_fullsized":"2024-01-27T02:41:58.235823+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:08:12.315061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.229944+0000","last_change":"2024-01-27T02:41:58.229794+0000","last_active":"2024-01-27T02:41:58.229944+0000","last_peered":"2024-01-27T02:41:58.229944+0000","last_clean":"2024-01-27T02:41:58.229944+0000","last_became_active":"2024-01-27T02:41:58.229405+0000","last_became_peered":"2024-01-27T02:41:58.229405+0000","last_unstale":"2024-01-27T02:41:58.229944+0000","last_undegraded":"2024-01-27T02:41:58.229944+0000","last_fullsized":"2024-01-27T02:41:58.229944+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:16:11.939669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.237082+0000","last_change":"2024-01-27T02:41:58.236728+0000","last_active":"2024-01-27T02:41:58.237082+0000","last_peered":"2024-01-27T02:41:58.237082+0000","last_clean":"2024-01-27T02:41:58.237082+0000","last_became_active":"2024-01-27T02:41:58.235399+0000","last_became_peered":"2024-01-27T02:41:58.235399+0000","last_unstale":"2024-01-27T02:41:58.237082+0000","last_undegraded":"2024-01-27T02:41:58.237082+0000","last_fullsized":"2024-01-27T02:41:58.237082+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:52:43.649919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.232304+0000","last_change":"2024-01-27T02:41:58.231845+0000","last_active":"2024-01-27T02:41:58.232304+0000","last_peered":"2024-01-27T02:41:58.232304+0000","last_clean":"2024-01-27T02:41:58.232304+0000","last_became_active":"2024-01-27T02:41:58.230892+0000","last_became_peered":"2024-01-27T02:41:58.230892+0000","last_unstale":"2024-01-27T02:41:58.232304+0000","last_undegraded":"2024-01-27T02:41:58.232304+0000","last_fullsized":"2024-01-27T02:41:58.232304+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:41:09.872021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.237167+0000","last_change":"2024-01-27T02:41:58.237095+0000","last_active":"2024-01-27T02:41:58.237167+0000","last_peered":"2024-01-27T02:41:58.237167+0000","last_clean":"2024-01-27T02:41:58.237167+0000","last_became_active":"2024-01-27T02:41:58.236780+0000","last_became_peered":"2024-01-27T02:41:58.236780+0000","last_unstale":"2024-01-27T02:41:58.237167+0000","last_undegraded":"2024-01-27T02:41:58.237167+0000","last_fullsized":"2024-01-27T02:41:58.237167+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:12.009262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.234503+0000","last_change":"2024-01-27T02:41:58.234424+0000","last_active":"2024-01-27T02:41:58.234503+0000","last_peered":"2024-01-27T02:41:58.234503+0000","last_clean":"2024-01-27T02:41:58.234503+0000","last_became_active":"2024-01-27T02:41:58.232879+0000","last_became_peered":"2024-01-27T02:41:58.232879+0000","last_unstale":"2024-01-27T02:41:58.234503+0000","last_undegraded":"2024-01-27T02:41:58.234503+0000","last_fullsized":"2024-01-27T02:41:58.234503+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:36:46.237152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.233995+0000","last_change":"2024-01-27T02:41:58.233907+0000","last_active":"2024-01-27T02:41:58.233995+0000","last_peered":"2024-01-27T02:41:58.233995+0000","last_clean":"2024-01-27T02:41:58.233995+0000","last_became_active":"2024-01-27T02:41:58.233433+0000","last_became_peered":"2024-01-27T02:41:58.233433+0000","last_unstale":"2024-01-27T02:41:58.233995+0000","last_undegraded":"2024-01-27T02:41:58.233995+0000","last_fullsized":"2024-01-27T02:41:58.233995+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:11:30.446141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.225354+0000","last_change":"2024-01-27T02:41:58.225158+0000","last_active":"2024-01-27T02:41:58.225354+0000","last_peered":"2024-01-27T02:41:58.225354+0000","last_clean":"2024-01-27T02:41:58.225354+0000","last_became_active":"2024-01-27T02:41:58.224827+0000","last_became_peered":"2024-01-27T02:41:58.224827+0000","last_unstale":"2024-01-27T02:41:58.225354+0000","last_undegraded":"2024-01-27T02:41:58.225354+0000","last_fullsized":"2024-01-27T02:41:58.225354+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:36:27.403658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.232322+0000","last_change":"2024-01-27T02:41:58.232011+0000","last_active":"2024-01-27T02:41:58.232322+0000","last_peered":"2024-01-27T02:41:58.232322+0000","last_clean":"2024-01-27T02:41:58.232322+0000","last_became_active":"2024-01-27T02:41:58.231232+0000","last_became_peered":"2024-01-27T02:41:58.231232+0000","last_unstale":"2024-01-27T02:41:58.232322+0000","last_undegraded":"2024-01-27T02:41:58.232322+0000","last_fullsized":"2024-01-27T02:41:58.232322+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:19:42.020949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.235755+0000","last_change":"2024-01-27T02:41:58.234784+0000","last_active":"2024-01-27T02:41:58.235755+0000","last_peered":"2024-01-27T02:41:58.235755+0000","last_clean":"2024-01-27T02:41:58.235755+0000","last_became_active":"2024-01-27T02:41:58.234030+0000","last_became_peered":"2024-01-27T02:41:58.234030+0000","last_unstale":"2024-01-27T02:41:58.235755+0000","last_undegraded":"2024-01-27T02:41:58.235755+0000","last_fullsized":"2024-01-27T02:41:58.235755+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:15:53.724042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"163'4","reported_seq":14,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.242446+0000","last_change":"2024-01-27T02:41:57.229996+0000","last_active":"2024-01-27T02:41:58.242446+0000","last_peered":"2024-01-27T02:41:58.242446+0000","last_clean":"2024-01-27T02:41:58.242446+0000","last_became_active":"2024-01-27T02:41:57.229757+0000","last_became_peered":"2024-01-27T02:41:57.229757+0000","last_unstale":"2024-01-27T02:41:58.242446+0000","last_undegraded":"2024-01-27T02:41:58.242446+0000","last_fullsized":"2024-01-27T02:41:58.242446+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:12:16.778462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.233014+0000","last_change":"2024-01-27T02:41:58.232947+0000","last_active":"2024-01-27T02:41:58.233014+0000","last_peered":"2024-01-27T02:41:58.233014+0000","last_clean":"2024-01-27T02:41:58.233014+0000","last_became_active":"2024-01-27T02:41:58.232614+0000","last_became_peered":"2024-01-27T02:41:58.232614+0000","last_unstale":"2024-01-27T02:41:58.233014+0000","last_undegraded":"2024-01-27T02:41:58.233014+0000","last_fullsized":"2024-01-27T02:41:58.233014+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:08:12.315061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"163'1","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.238518+0000","last_change":"2024-01-27T02:41:57.229201+0000","last_active":"2024-01-27T02:41:58.238518+0000","last_peered":"2024-01-27T02:41:58.238518+0000","last_clean":"2024-01-27T02:41:58.238518+0000","last_became_active":"2024-01-27T02:41:57.228775+0000","last_became_peered":"2024-01-27T02:41:57.228775+0000","last_unstale":"2024-01-27T02:41:58.238518+0000","last_undegraded":"2024-01-27T02:41:58.238518+0000","last_fullsized":"2024-01-27T02:41:58.238518+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:18:38.936550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.237042+0000","last_change":"2024-01-27T02:41:58.236423+0000","last_active":"2024-01-27T02:41:58.237042+0000","last_peered":"2024-01-27T02:41:58.237042+0000","last_clean":"2024-01-27T02:41:58.237042+0000","last_became_active":"2024-01-27T02:41:58.234458+0000","last_became_peered":"2024-01-27T02:41:58.234458+0000","last_unstale":"2024-01-27T02:41:58.237042+0000","last_undegraded":"2024-01-27T02:41:58.237042+0000","last_fullsized":"2024-01-27T02:41:58.237042+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:49:38.618223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"163'2","reported_seq":12,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.238500+0000","last_change":"2024-01-27T02:41:57.226508+0000","last_active":"2024-01-27T02:41:58.238500+0000","last_peered":"2024-01-27T02:41:58.238500+0000","last_clean":"2024-01-27T02:41:58.238500+0000","last_became_active":"2024-01-27T02:41:57.226240+0000","last_became_peered":"2024-01-27T02:41:57.226240+0000","last_unstale":"2024-01-27T02:41:58.238500+0000","last_undegraded":"2024-01-27T02:41:58.238500+0000","last_fullsized":"2024-01-27T02:41:58.238500+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:48:10.358366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.239451+0000","last_change":"2024-01-27T02:41:58.239364+0000","last_active":"2024-01-27T02:41:58.239451+0000","last_peered":"2024-01-27T02:41:58.239451+0000","last_clean":"2024-01-27T02:41:58.239451+0000","last_became_active":"2024-01-27T02:41:58.238586+0000","last_became_peered":"2024-01-27T02:41:58.238586+0000","last_unstale":"2024-01-27T02:41:58.239451+0000","last_undegraded":"2024-01-27T02:41:58.239451+0000","last_fullsized":"2024-01-27T02:41:58.239451+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:38:12.599262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"163'2","reported_seq":12,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.236396+0000","last_change":"2024-01-27T02:41:57.231250+0000","last_active":"2024-01-27T02:41:58.236396+0000","last_peered":"2024-01-27T02:41:58.236396+0000","last_clean":"2024-01-27T02:41:58.236396+0000","last_became_active":"2024-01-27T02:41:57.230763+0000","last_became_peered":"2024-01-27T02:41:57.230763+0000","last_unstale":"2024-01-27T02:41:58.236396+0000","last_undegraded":"2024-01-27T02:41:58.236396+0000","last_fullsized":"2024-01-27T02:41:58.236396+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:10:51.840479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.227064+0000","last_change":"2024-01-27T02:41:58.226654+0000","last_active":"2024-01-27T02:41:58.227064+0000","last_peered":"2024-01-27T02:41:58.227064+0000","last_clean":"2024-01-27T02:41:58.227064+0000","last_became_active":"2024-01-27T02:41:58.226406+0000","last_became_peered":"2024-01-27T02:41:58.226406+0000","last_unstale":"2024-01-27T02:41:58.227064+0000","last_undegraded":"2024-01-27T02:41:58.227064+0000","last_fullsized":"2024-01-27T02:41:58.227064+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:46:14.298161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"163'1","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.238982+0000","last_change":"2024-01-27T02:41:57.231266+0000","last_active":"2024-01-27T02:41:58.238982+0000","last_peered":"2024-01-27T02:41:58.238982+0000","last_clean":"2024-01-27T02:41:58.238982+0000","last_became_active":"2024-01-27T02:41:57.230744+0000","last_became_peered":"2024-01-27T02:41:57.230744+0000","last_unstale":"2024-01-27T02:41:58.238982+0000","last_undegraded":"2024-01-27T02:41:58.238982+0000","last_fullsized":"2024-01-27T02:41:58.238982+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:11:25.491467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.232727+0000","last_change":"2024-01-27T02:41:58.232556+0000","last_active":"2024-01-27T02:41:58.232727+0000","last_peered":"2024-01-27T02:41:58.232727+0000","last_clean":"2024-01-27T02:41:58.232727+0000","last_became_active":"2024-01-27T02:41:58.232319+0000","last_became_peered":"2024-01-27T02:41:58.232319+0000","last_unstale":"2024-01-27T02:41:58.232727+0000","last_undegraded":"2024-01-27T02:41:58.232727+0000","last_fullsized":"2024-01-27T02:41:58.232727+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:43:12.603658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"163'1","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.238383+0000","last_change":"2024-01-27T02:41:57.229425+0000","last_active":"2024-01-27T02:41:58.238383+0000","last_peered":"2024-01-27T02:41:58.238383+0000","last_clean":"2024-01-27T02:41:58.238383+0000","last_became_active":"2024-01-27T02:41:57.228786+0000","last_became_peered":"2024-01-27T02:41:57.228786+0000","last_unstale":"2024-01-27T02:41:58.238383+0000","last_undegraded":"2024-01-27T02:41:58.238383+0000","last_fullsized":"2024-01-27T02:41:58.238383+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:19:41.016029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.223536+0000","last_change":"2024-01-27T02:41:58.223426+0000","last_active":"2024-01-27T02:41:58.223536+0000","last_peered":"2024-01-27T02:41:58.223536+0000","last_clean":"2024-01-27T02:41:58.223536+0000","last_became_active":"2024-01-27T02:41:58.222684+0000","last_became_peered":"2024-01-27T02:41:58.222684+0000","last_unstale":"2024-01-27T02:41:58.223536+0000","last_undegraded":"2024-01-27T02:41:58.223536+0000","last_fullsized":"2024-01-27T02:41:58.223536+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:38:12.599262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.221284+0000","last_change":"2024-01-27T02:41:57.228906+0000","last_active":"2024-01-27T02:41:58.221284+0000","last_peered":"2024-01-27T02:41:58.221284+0000","last_clean":"2024-01-27T02:41:58.221284+0000","last_became_active":"2024-01-27T02:41:57.225731+0000","last_became_peered":"2024-01-27T02:41:57.225731+0000","last_unstale":"2024-01-27T02:41:58.221284+0000","last_undegraded":"2024-01-27T02:41:58.221284+0000","last_fullsized":"2024-01-27T02:41:58.221284+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:56:47.517031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.228858+0000","last_change":"2024-01-27T02:41:58.228795+0000","last_active":"2024-01-27T02:41:58.228858+0000","last_peered":"2024-01-27T02:41:58.228858+0000","last_clean":"2024-01-27T02:41:58.228858+0000","last_became_active":"2024-01-27T02:41:58.228570+0000","last_became_peered":"2024-01-27T02:41:58.228570+0000","last_unstale":"2024-01-27T02:41:58.228858+0000","last_undegraded":"2024-01-27T02:41:58.228858+0000","last_fullsized":"2024-01-27T02:41:58.228858+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:36:00.504815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.227225+0000","last_change":"2024-01-27T02:41:57.228643+0000","last_active":"2024-01-27T02:41:58.227225+0000","last_peered":"2024-01-27T02:41:58.227225+0000","last_clean":"2024-01-27T02:41:58.227225+0000","last_became_active":"2024-01-27T02:41:57.228341+0000","last_became_peered":"2024-01-27T02:41:57.228341+0000","last_unstale":"2024-01-27T02:41:58.227225+0000","last_undegraded":"2024-01-27T02:41:58.227225+0000","last_fullsized":"2024-01-27T02:41:58.227225+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:10:18.418960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.232734+0000","last_change":"2024-01-27T02:41:58.231409+0000","last_active":"2024-01-27T02:41:58.232734+0000","last_peered":"2024-01-27T02:41:58.232734+0000","last_clean":"2024-01-27T02:41:58.232734+0000","last_became_active":"2024-01-27T02:41:58.231158+0000","last_became_peered":"2024-01-27T02:41:58.231158+0000","last_unstale":"2024-01-27T02:41:58.232734+0000","last_undegraded":"2024-01-27T02:41:58.232734+0000","last_fullsized":"2024-01-27T02:41:58.232734+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:10:52.845400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.219196+0000","last_change":"2024-01-27T02:41:57.230321+0000","last_active":"2024-01-27T02:41:58.219196+0000","last_peered":"2024-01-27T02:41:58.219196+0000","last_clean":"2024-01-27T02:41:58.219196+0000","last_became_active":"2024-01-27T02:41:57.230006+0000","last_became_peered":"2024-01-27T02:41:57.230006+0000","last_unstale":"2024-01-27T02:41:58.219196+0000","last_undegraded":"2024-01-27T02:41:58.219196+0000","last_fullsized":"2024-01-27T02:41:58.219196+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:21:15.480614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.231225+0000","last_change":"2024-01-27T02:41:58.230653+0000","last_active":"2024-01-27T02:41:58.231225+0000","last_peered":"2024-01-27T02:41:58.231225+0000","last_clean":"2024-01-27T02:41:58.231225+0000","last_became_active":"2024-01-27T02:41:58.230378+0000","last_became_peered":"2024-01-27T02:41:58.230378+0000","last_unstale":"2024-01-27T02:41:58.231225+0000","last_undegraded":"2024-01-27T02:41:58.231225+0000","last_fullsized":"2024-01-27T02:41:58.231225+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:45:13.570151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.235761+0000","last_change":"2024-01-27T02:41:57.233559+0000","last_active":"2024-01-27T02:41:58.235761+0000","last_peered":"2024-01-27T02:41:58.235761+0000","last_clean":"2024-01-27T02:41:58.235761+0000","last_became_active":"2024-01-27T02:41:57.232866+0000","last_became_peered":"2024-01-27T02:41:57.232866+0000","last_unstale":"2024-01-27T02:41:58.235761+0000","last_undegraded":"2024-01-27T02:41:58.235761+0000","last_fullsized":"2024-01-27T02:41:58.235761+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:02:27.107496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.234618+0000","last_change":"2024-01-27T02:41:58.234566+0000","last_active":"2024-01-27T02:41:58.234618+0000","last_peered":"2024-01-27T02:41:58.234618+0000","last_clean":"2024-01-27T02:41:58.234618+0000","last_became_active":"2024-01-27T02:41:58.234317+0000","last_became_peered":"2024-01-27T02:41:58.234317+0000","last_unstale":"2024-01-27T02:41:58.234618+0000","last_undegraded":"2024-01-27T02:41:58.234618+0000","last_fullsized":"2024-01-27T02:41:58.234618+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:37:54.293485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"163'6","reported_seq":16,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.240271+0000","last_change":"2024-01-27T02:41:57.225500+0000","last_active":"2024-01-27T02:41:58.240271+0000","last_peered":"2024-01-27T02:41:58.240271+0000","last_clean":"2024-01-27T02:41:58.240271+0000","last_became_active":"2024-01-27T02:41:57.225243+0000","last_became_peered":"2024-01-27T02:41:57.225243+0000","last_unstale":"2024-01-27T02:41:58.240271+0000","last_undegraded":"2024-01-27T02:41:58.240271+0000","last_fullsized":"2024-01-27T02:41:58.240271+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:55:23.803469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.234975+0000","last_change":"2024-01-27T02:41:58.233414+0000","last_active":"2024-01-27T02:41:58.234975+0000","last_peered":"2024-01-27T02:41:58.234975+0000","last_clean":"2024-01-27T02:41:58.234975+0000","last_became_active":"2024-01-27T02:41:58.233182+0000","last_became_peered":"2024-01-27T02:41:58.233182+0000","last_unstale":"2024-01-27T02:41:58.234975+0000","last_undegraded":"2024-01-27T02:41:58.234975+0000","last_fullsized":"2024-01-27T02:41:58.234975+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:20:57.415345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"163'2","reported_seq":12,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.233497+0000","last_change":"2024-01-27T02:41:57.222999+0000","last_active":"2024-01-27T02:41:58.233497+0000","last_peered":"2024-01-27T02:41:58.233497+0000","last_clean":"2024-01-27T02:41:58.233497+0000","last_became_active":"2024-01-27T02:41:57.222807+0000","last_became_peered":"2024-01-27T02:41:57.222807+0000","last_unstale":"2024-01-27T02:41:58.233497+0000","last_undegraded":"2024-01-27T02:41:58.233497+0000","last_fullsized":"2024-01-27T02:41:58.233497+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:18:27.914551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.228808+0000","last_change":"2024-01-27T02:41:58.228731+0000","last_active":"2024-01-27T02:41:58.228808+0000","last_peered":"2024-01-27T02:41:58.228808+0000","last_clean":"2024-01-27T02:41:58.228808+0000","last_became_active":"2024-01-27T02:41:58.228475+0000","last_became_peered":"2024-01-27T02:41:58.228475+0000","last_unstale":"2024-01-27T02:41:58.228808+0000","last_undegraded":"2024-01-27T02:41:58.228808+0000","last_fullsized":"2024-01-27T02:41:58.228808+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:35:12.009262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.223674+0000","last_change":"2024-01-27T02:41:57.225422+0000","last_active":"2024-01-27T02:41:58.223674+0000","last_peered":"2024-01-27T02:41:58.223674+0000","last_clean":"2024-01-27T02:41:58.223674+0000","last_became_active":"2024-01-27T02:41:57.224949+0000","last_became_peered":"2024-01-27T02:41:57.224949+0000","last_unstale":"2024-01-27T02:41:58.223674+0000","last_undegraded":"2024-01-27T02:41:58.223674+0000","last_fullsized":"2024-01-27T02:41:58.223674+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:13:50.404221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.237308+0000","last_change":"2024-01-27T02:41:58.237050+0000","last_active":"2024-01-27T02:41:58.237308+0000","last_peered":"2024-01-27T02:41:58.237308+0000","last_clean":"2024-01-27T02:41:58.237308+0000","last_became_active":"2024-01-27T02:41:58.236388+0000","last_became_peered":"2024-01-27T02:41:58.236388+0000","last_unstale":"2024-01-27T02:41:58.237308+0000","last_undegraded":"2024-01-27T02:41:58.237308+0000","last_fullsized":"2024-01-27T02:41:58.237308+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:10:19.423880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.222725+0000","last_change":"2024-01-27T02:41:57.223933+0000","last_active":"2024-01-27T02:41:58.222725+0000","last_peered":"2024-01-27T02:41:58.222725+0000","last_clean":"2024-01-27T02:41:58.222725+0000","last_became_active":"2024-01-27T02:41:57.223693+0000","last_became_peered":"2024-01-27T02:41:57.223693+0000","last_unstale":"2024-01-27T02:41:58.222725+0000","last_undegraded":"2024-01-27T02:41:58.222725+0000","last_fullsized":"2024-01-27T02:41:58.222725+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:34:18.049031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.232161+0000","last_change":"2024-01-27T02:41:58.231557+0000","last_active":"2024-01-27T02:41:58.232161+0000","last_peered":"2024-01-27T02:41:58.232161+0000","last_clean":"2024-01-27T02:41:58.232161+0000","last_became_active":"2024-01-27T02:41:58.231312+0000","last_became_peered":"2024-01-27T02:41:58.231312+0000","last_unstale":"2024-01-27T02:41:58.232161+0000","last_undegraded":"2024-01-27T02:41:58.232161+0000","last_fullsized":"2024-01-27T02:41:58.232161+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:06:04.234073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"163'8","reported_seq":18,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.239917+0000","last_change":"2024-01-27T02:41:57.227028+0000","last_active":"2024-01-27T02:41:58.239917+0000","last_peered":"2024-01-27T02:41:58.239917+0000","last_clean":"2024-01-27T02:41:58.239917+0000","last_became_active":"2024-01-27T02:41:57.226710+0000","last_became_peered":"2024-01-27T02:41:57.226710+0000","last_unstale":"2024-01-27T02:41:58.239917+0000","last_undegraded":"2024-01-27T02:41:58.239917+0000","last_fullsized":"2024-01-27T02:41:58.239917+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:25:17.160949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.231834+0000","last_change":"2024-01-27T02:41:58.231707+0000","last_active":"2024-01-27T02:41:58.231834+0000","last_peered":"2024-01-27T02:41:58.231834+0000","last_clean":"2024-01-27T02:41:58.231834+0000","last_became_active":"2024-01-27T02:41:58.231476+0000","last_became_peered":"2024-01-27T02:41:58.231476+0000","last_unstale":"2024-01-27T02:41:58.231834+0000","last_undegraded":"2024-01-27T02:41:58.231834+0000","last_fullsized":"2024-01-27T02:41:58.231834+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:56:27.990324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"163'1","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.235061+0000","last_change":"2024-01-27T02:41:57.225058+0000","last_active":"2024-01-27T02:41:58.235061+0000","last_peered":"2024-01-27T02:41:58.235061+0000","last_clean":"2024-01-27T02:41:58.235061+0000","last_became_active":"2024-01-27T02:41:57.224513+0000","last_became_peered":"2024-01-27T02:41:57.224513+0000","last_unstale":"2024-01-27T02:41:58.235061+0000","last_undegraded":"2024-01-27T02:41:58.235061+0000","last_fullsized":"2024-01-27T02:41:58.235061+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:41:08.867101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.231668+0000","last_change":"2024-01-27T02:41:58.231587+0000","last_active":"2024-01-27T02:41:58.231668+0000","last_peered":"2024-01-27T02:41:58.231668+0000","last_clean":"2024-01-27T02:41:58.231668+0000","last_became_active":"2024-01-27T02:41:58.231283+0000","last_became_peered":"2024-01-27T02:41:58.231283+0000","last_unstale":"2024-01-27T02:41:58.231668+0000","last_undegraded":"2024-01-27T02:41:58.231668+0000","last_fullsized":"2024-01-27T02:41:58.231668+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:28:07.248554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"163'2","reported_seq":12,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.234920+0000","last_change":"2024-01-27T02:41:57.222690+0000","last_active":"2024-01-27T02:41:58.234920+0000","last_peered":"2024-01-27T02:41:58.234920+0000","last_clean":"2024-01-27T02:41:58.234920+0000","last_became_active":"2024-01-27T02:41:57.222033+0000","last_became_peered":"2024-01-27T02:41:57.222033+0000","last_unstale":"2024-01-27T02:41:58.234920+0000","last_undegraded":"2024-01-27T02:41:58.234920+0000","last_fullsized":"2024-01-27T02:41:58.234920+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:16:05.334560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.234867+0000","last_change":"2024-01-27T02:41:58.232682+0000","last_active":"2024-01-27T02:41:58.234867+0000","last_peered":"2024-01-27T02:41:58.234867+0000","last_clean":"2024-01-27T02:41:58.234867+0000","last_became_active":"2024-01-27T02:41:58.232126+0000","last_became_peered":"2024-01-27T02:41:58.232126+0000","last_unstale":"2024-01-27T02:41:58.234867+0000","last_undegraded":"2024-01-27T02:41:58.234867+0000","last_fullsized":"2024-01-27T02:41:58.234867+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:02:28.112417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"163'1","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.237621+0000","last_change":"2024-01-27T02:41:57.230038+0000","last_active":"2024-01-27T02:41:58.237621+0000","last_peered":"2024-01-27T02:41:58.237621+0000","last_clean":"2024-01-27T02:41:58.237621+0000","last_became_active":"2024-01-27T02:41:57.229763+0000","last_became_peered":"2024-01-27T02:41:57.229763+0000","last_unstale":"2024-01-27T02:41:58.237621+0000","last_undegraded":"2024-01-27T02:41:58.237621+0000","last_fullsized":"2024-01-27T02:41:58.237621+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:40:29.355463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.228056+0000","last_change":"2024-01-27T02:41:58.227519+0000","last_active":"2024-01-27T02:41:58.228056+0000","last_peered":"2024-01-27T02:41:58.228056+0000","last_clean":"2024-01-27T02:41:58.228056+0000","last_became_active":"2024-01-27T02:41:58.227286+0000","last_became_peered":"2024-01-27T02:41:58.227286+0000","last_unstale":"2024-01-27T02:41:58.228056+0000","last_undegraded":"2024-01-27T02:41:58.228056+0000","last_fullsized":"2024-01-27T02:41:58.228056+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:16:40.241138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.218431+0000","last_change":"2024-01-27T02:41:57.231163+0000","last_active":"2024-01-27T02:41:58.218431+0000","last_peered":"2024-01-27T02:41:58.218431+0000","last_clean":"2024-01-27T02:41:58.218431+0000","last_became_active":"2024-01-27T02:41:57.230980+0000","last_became_peered":"2024-01-27T02:41:57.230980+0000","last_unstale":"2024-01-27T02:41:58.218431+0000","last_undegraded":"2024-01-27T02:41:58.218431+0000","last_fullsized":"2024-01-27T02:41:58.218431+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:40:27.220792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.232404+0000","last_change":"2024-01-27T02:41:58.232049+0000","last_active":"2024-01-27T02:41:58.232404+0000","last_peered":"2024-01-27T02:41:58.232404+0000","last_clean":"2024-01-27T02:41:58.232404+0000","last_became_active":"2024-01-27T02:41:58.231836+0000","last_became_peered":"2024-01-27T02:41:58.231836+0000","last_unstale":"2024-01-27T02:41:58.232404+0000","last_undegraded":"2024-01-27T02:41:58.232404+0000","last_fullsized":"2024-01-27T02:41:58.232404+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:20:57.415345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"163'2","reported_seq":12,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.238377+0000","last_change":"2024-01-27T02:41:57.224112+0000","last_active":"2024-01-27T02:41:58.238377+0000","last_peered":"2024-01-27T02:41:58.238377+0000","last_clean":"2024-01-27T02:41:58.238377+0000","last_became_active":"2024-01-27T02:41:57.223817+0000","last_became_peered":"2024-01-27T02:41:57.223817+0000","last_unstale":"2024-01-27T02:41:58.238377+0000","last_undegraded":"2024-01-27T02:41:58.238377+0000","last_fullsized":"2024-01-27T02:41:58.238377+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:41:51.998569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.238841+0000","last_change":"2024-01-27T02:41:58.238780+0000","last_active":"2024-01-27T02:41:58.238841+0000","last_peered":"2024-01-27T02:41:58.238841+0000","last_clean":"2024-01-27T02:41:58.238841+0000","last_became_active":"2024-01-27T02:41:58.238075+0000","last_became_peered":"2024-01-27T02:41:58.238075+0000","last_unstale":"2024-01-27T02:41:58.238841+0000","last_undegraded":"2024-01-27T02:41:58.238841+0000","last_fullsized":"2024-01-27T02:41:58.238841+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:03:29.567386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.225949+0000","last_change":"2024-01-27T02:41:57.230721+0000","last_active":"2024-01-27T02:41:58.225949+0000","last_peered":"2024-01-27T02:41:58.225949+0000","last_clean":"2024-01-27T02:41:58.225949+0000","last_became_active":"2024-01-27T02:41:57.230457+0000","last_became_peered":"2024-01-27T02:41:57.230457+0000","last_unstale":"2024-01-27T02:41:58.225949+0000","last_undegraded":"2024-01-27T02:41:58.225949+0000","last_fullsized":"2024-01-27T02:41:58.225949+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:22:02.038987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.231762+0000","last_change":"2024-01-27T02:41:58.230958+0000","last_active":"2024-01-27T02:41:58.231762+0000","last_peered":"2024-01-27T02:41:58.231762+0000","last_clean":"2024-01-27T02:41:58.231762+0000","last_became_active":"2024-01-27T02:41:58.230634+0000","last_became_peered":"2024-01-27T02:41:58.230634+0000","last_unstale":"2024-01-27T02:41:58.231762+0000","last_undegraded":"2024-01-27T02:41:58.231762+0000","last_fullsized":"2024-01-27T02:41:58.231762+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:24:56.326639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.225646+0000","last_change":"2024-01-27T02:41:57.231323+0000","last_active":"2024-01-27T02:41:58.225646+0000","last_peered":"2024-01-27T02:41:58.225646+0000","last_clean":"2024-01-27T02:41:58.225646+0000","last_became_active":"2024-01-27T02:41:57.231103+0000","last_became_peered":"2024-01-27T02:41:57.231103+0000","last_unstale":"2024-01-27T02:41:58.225646+0000","last_undegraded":"2024-01-27T02:41:58.225646+0000","last_fullsized":"2024-01-27T02:41:58.225646+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:06:03.229153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.233362+0000","last_change":"2024-01-27T02:41:58.233254+0000","last_active":"2024-01-27T02:41:58.233362+0000","last_peered":"2024-01-27T02:41:58.233362+0000","last_clean":"2024-01-27T02:41:58.233362+0000","last_became_active":"2024-01-27T02:41:58.232847+0000","last_became_peered":"2024-01-27T02:41:58.232847+0000","last_unstale":"2024-01-27T02:41:58.233362+0000","last_undegraded":"2024-01-27T02:41:58.233362+0000","last_fullsized":"2024-01-27T02:41:58.233362+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:15:53.724042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.223776+0000","last_change":"2024-01-27T02:41:57.225212+0000","last_active":"2024-01-27T02:41:58.223776+0000","last_peered":"2024-01-27T02:41:58.223776+0000","last_clean":"2024-01-27T02:41:58.223776+0000","last_became_active":"2024-01-27T02:41:57.224598+0000","last_became_peered":"2024-01-27T02:41:57.224598+0000","last_unstale":"2024-01-27T02:41:58.223776+0000","last_undegraded":"2024-01-27T02:41:58.223776+0000","last_fullsized":"2024-01-27T02:41:58.223776+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:16:54.556346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.237291+0000","last_change":"2024-01-27T02:41:58.236931+0000","last_active":"2024-01-27T02:41:58.237291+0000","last_peered":"2024-01-27T02:41:58.237291+0000","last_clean":"2024-01-27T02:41:58.237291+0000","last_became_active":"2024-01-27T02:41:58.236065+0000","last_became_peered":"2024-01-27T02:41:58.236065+0000","last_unstale":"2024-01-27T02:41:58.237291+0000","last_undegraded":"2024-01-27T02:41:58.237291+0000","last_fullsized":"2024-01-27T02:41:58.237291+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:49:12.407976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"163'2","reported_seq":12,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.242124+0000","last_change":"2024-01-27T02:41:57.233141+0000","last_active":"2024-01-27T02:41:58.242124+0000","last_peered":"2024-01-27T02:41:58.242124+0000","last_clean":"2024-01-27T02:41:58.242124+0000","last_became_active":"2024-01-27T02:41:57.231882+0000","last_became_peered":"2024-01-27T02:41:57.231882+0000","last_unstale":"2024-01-27T02:41:58.242124+0000","last_undegraded":"2024-01-27T02:41:58.242124+0000","last_fullsized":"2024-01-27T02:41:58.242124+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:24:55.321719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.234933+0000","last_change":"2024-01-27T02:41:58.232825+0000","last_active":"2024-01-27T02:41:58.234933+0000","last_peered":"2024-01-27T02:41:58.234933+0000","last_clean":"2024-01-27T02:41:58.234933+0000","last_became_active":"2024-01-27T02:41:58.232450+0000","last_became_peered":"2024-01-27T02:41:58.232450+0000","last_unstale":"2024-01-27T02:41:58.234933+0000","last_undegraded":"2024-01-27T02:41:58.234933+0000","last_fullsized":"2024-01-27T02:41:58.234933+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:11:30.446141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.224456+0000","last_change":"2024-01-27T02:41:57.228792+0000","last_active":"2024-01-27T02:41:58.224456+0000","last_peered":"2024-01-27T02:41:58.224456+0000","last_clean":"2024-01-27T02:41:58.224456+0000","last_became_active":"2024-01-27T02:41:57.228498+0000","last_became_peered":"2024-01-27T02:41:57.228498+0000","last_unstale":"2024-01-27T02:41:58.224456+0000","last_undegraded":"2024-01-27T02:41:58.224456+0000","last_fullsized":"2024-01-27T02:41:58.224456+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:45:32.325959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.232568+0000","last_change":"2024-01-27T02:41:58.232400+0000","last_active":"2024-01-27T02:41:58.232568+0000","last_peered":"2024-01-27T02:41:58.232568+0000","last_clean":"2024-01-27T02:41:58.232568+0000","last_became_active":"2024-01-27T02:41:58.232159+0000","last_became_peered":"2024-01-27T02:41:58.232159+0000","last_unstale":"2024-01-27T02:41:58.232568+0000","last_undegraded":"2024-01-27T02:41:58.232568+0000","last_fullsized":"2024-01-27T02:41:58.232568+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:03:29.567386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.227175+0000","last_change":"2024-01-27T02:41:57.226223+0000","last_active":"2024-01-27T02:41:58.227175+0000","last_peered":"2024-01-27T02:41:58.227175+0000","last_clean":"2024-01-27T02:41:58.227175+0000","last_became_active":"2024-01-27T02:41:57.225947+0000","last_became_peered":"2024-01-27T02:41:57.225947+0000","last_unstale":"2024-01-27T02:41:58.227175+0000","last_undegraded":"2024-01-27T02:41:58.227175+0000","last_fullsized":"2024-01-27T02:41:58.227175+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:38:16.363217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.234700+0000","last_change":"2024-01-27T02:41:58.234599+0000","last_active":"2024-01-27T02:41:58.234700+0000","last_peered":"2024-01-27T02:41:58.234700+0000","last_clean":"2024-01-27T02:41:58.234700+0000","last_became_active":"2024-01-27T02:41:58.233379+0000","last_became_peered":"2024-01-27T02:41:58.233379+0000","last_unstale":"2024-01-27T02:41:58.234700+0000","last_undegraded":"2024-01-27T02:41:58.234700+0000","last_fullsized":"2024-01-27T02:41:58.234700+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:18:28.919472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.225062+0000","last_change":"2024-01-27T02:41:57.227151+0000","last_active":"2024-01-27T02:41:58.225062+0000","last_peered":"2024-01-27T02:41:58.225062+0000","last_clean":"2024-01-27T02:41:58.225062+0000","last_became_active":"2024-01-27T02:41:57.226892+0000","last_became_peered":"2024-01-27T02:41:57.226892+0000","last_unstale":"2024-01-27T02:41:58.225062+0000","last_undegraded":"2024-01-27T02:41:58.225062+0000","last_fullsized":"2024-01-27T02:41:58.225062+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:16:20.195961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.232338+0000","last_change":"2024-01-27T02:41:58.232150+0000","last_active":"2024-01-27T02:41:58.232338+0000","last_peered":"2024-01-27T02:41:58.232338+0000","last_clean":"2024-01-27T02:41:58.232338+0000","last_became_active":"2024-01-27T02:41:58.231574+0000","last_became_peered":"2024-01-27T02:41:58.231574+0000","last_unstale":"2024-01-27T02:41:58.232338+0000","last_undegraded":"2024-01-27T02:41:58.232338+0000","last_fullsized":"2024-01-27T02:41:58.232338+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:15:13.889275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.227183+0000","last_change":"2024-01-27T02:41:57.230166+0000","last_active":"2024-01-27T02:41:58.227183+0000","last_peered":"2024-01-27T02:41:58.227183+0000","last_clean":"2024-01-27T02:41:58.227183+0000","last_became_active":"2024-01-27T02:41:57.229940+0000","last_became_peered":"2024-01-27T02:41:57.229940+0000","last_unstale":"2024-01-27T02:41:58.227183+0000","last_undegraded":"2024-01-27T02:41:58.227183+0000","last_fullsized":"2024-01-27T02:41:58.227183+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:47:14.775401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.225312+0000","last_change":"2024-01-27T02:41:58.225237+0000","last_active":"2024-01-27T02:41:58.225312+0000","last_peered":"2024-01-27T02:41:58.225312+0000","last_clean":"2024-01-27T02:41:58.225312+0000","last_became_active":"2024-01-27T02:41:58.224897+0000","last_became_peered":"2024-01-27T02:41:58.224897+0000","last_unstale":"2024-01-27T02:41:58.225312+0000","last_undegraded":"2024-01-27T02:41:58.225312+0000","last_fullsized":"2024-01-27T02:41:58.225312+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:05:51.451369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.235982+0000","last_change":"2024-01-27T02:41:57.233354+0000","last_active":"2024-01-27T02:41:58.235982+0000","last_peered":"2024-01-27T02:41:58.235982+0000","last_clean":"2024-01-27T02:41:58.235982+0000","last_became_active":"2024-01-27T02:41:57.232311+0000","last_became_peered":"2024-01-27T02:41:57.232311+0000","last_unstale":"2024-01-27T02:41:58.235982+0000","last_undegraded":"2024-01-27T02:41:58.235982+0000","last_fullsized":"2024-01-27T02:41:58.235982+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:11:29.441221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.235803+0000","last_change":"2024-01-27T02:41:58.235364+0000","last_active":"2024-01-27T02:41:58.235803+0000","last_peered":"2024-01-27T02:41:58.235803+0000","last_clean":"2024-01-27T02:41:58.235803+0000","last_became_active":"2024-01-27T02:41:58.234487+0000","last_became_peered":"2024-01-27T02:41:58.234487+0000","last_unstale":"2024-01-27T02:41:58.235803+0000","last_undegraded":"2024-01-27T02:41:58.235803+0000","last_fullsized":"2024-01-27T02:41:58.235803+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:52:43.126709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.225047+0000","last_change":"2024-01-27T02:41:57.224148+0000","last_active":"2024-01-27T02:41:58.225047+0000","last_peered":"2024-01-27T02:41:58.225047+0000","last_clean":"2024-01-27T02:41:58.225047+0000","last_became_active":"2024-01-27T02:41:57.223447+0000","last_became_peered":"2024-01-27T02:41:57.223447+0000","last_unstale":"2024-01-27T02:41:58.225047+0000","last_undegraded":"2024-01-27T02:41:58.225047+0000","last_fullsized":"2024-01-27T02:41:58.225047+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:36:45.232232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.232752+0000","last_change":"2024-01-27T02:41:58.231765+0000","last_active":"2024-01-27T02:41:58.232752+0000","last_peered":"2024-01-27T02:41:58.232752+0000","last_clean":"2024-01-27T02:41:58.232752+0000","last_became_active":"2024-01-27T02:41:58.231548+0000","last_became_peered":"2024-01-27T02:41:58.231548+0000","last_unstale":"2024-01-27T02:41:58.232752+0000","last_undegraded":"2024-01-27T02:41:58.232752+0000","last_fullsized":"2024-01-27T02:41:58.232752+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:55:28.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.219588+0000","last_change":"2024-01-27T02:41:57.229029+0000","last_active":"2024-01-27T02:41:58.219588+0000","last_peered":"2024-01-27T02:41:58.219588+0000","last_clean":"2024-01-27T02:41:58.219588+0000","last_became_active":"2024-01-27T02:41:57.228428+0000","last_became_peered":"2024-01-27T02:41:57.228428+0000","last_unstale":"2024-01-27T02:41:58.219588+0000","last_undegraded":"2024-01-27T02:41:58.219588+0000","last_fullsized":"2024-01-27T02:41:58.219588+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:28:21.096641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.238864+0000","last_change":"2024-01-27T02:41:58.238781+0000","last_active":"2024-01-27T02:41:58.238864+0000","last_peered":"2024-01-27T02:41:58.238864+0000","last_clean":"2024-01-27T02:41:58.238864+0000","last_became_active":"2024-01-27T02:41:58.237662+0000","last_became_peered":"2024-01-27T02:41:58.237662+0000","last_unstale":"2024-01-27T02:41:58.238864+0000","last_undegraded":"2024-01-27T02:41:58.238864+0000","last_fullsized":"2024-01-27T02:41:58.238864+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:57.209242+0000","last_clean_scrub_stamp":"2024-01-27T02:41:57.209242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:08:12.315061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"163'3","reported_seq":13,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.236554+0000","last_change":"2024-01-27T02:41:57.229465+0000","last_active":"2024-01-27T02:41:58.236554+0000","last_peered":"2024-01-27T02:41:58.236554+0000","last_clean":"2024-01-27T02:41:58.236554+0000","last_became_active":"2024-01-27T02:41:57.229195+0000","last_became_peered":"2024-01-27T02:41:57.229195+0000","last_unstale":"2024-01-27T02:41:58.236554+0000","last_undegraded":"2024-01-27T02:41:58.236554+0000","last_fullsized":"2024-01-27T02:41:58.236554+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:41:56.204322+0000","last_clean_scrub_stamp":"2024-01-27T02:41:56.204322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:53:40.529059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":288,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-01-27T02:41:58.226039+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:41:58.226039+0000","last_peered":"2024-01-27T02:41:58.226039+0000","last_clean":"2024-01-27T02:41:58.226039+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:41:58.226039+0000","last_undegraded":"2024-01-27T02:41:58.226039+0000","last_fullsized":"2024-01-27T02:41:58.226039+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574971,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1468,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193844,"statfs":{"total":200000000,"available":198496370,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:41:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.44500000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.85499999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.48499999999999999,"5min":0.376,"15min":0.30099999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.66700000000000004}]},{"osd":1,"last update":"Sat Jan 27 02:41:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.52000000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76500000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.59599999999999997,"15min":0.56999999999999995},"min":{"1min":0.432,"5min":0.41799999999999998,"15min":0.34999999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.59799999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:41:14 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.441,"5min":0.376,"15min":0.376},"max":{"1min":0.77600000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.60899999999999999,"15min":0.59399999999999997},"min":{"1min":0.51300000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.83899999999999997},"last":0.65700000000000003}]},{"osd":3,"last update":"Sat Jan 27 02:41:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.54800000000000004,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78900000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63,"15min":0.60699999999999998},"min":{"1min":0.50600000000000001,"5min":0.41799999999999998,"15min":0.40999999999999998},"max":{"1min":0.81599999999999995,"5min":0.85099999999999998,"15min":0.88600000000000001},"last":0.69299999999999995}]},{"osd":4,"last update":"Sat Jan 27 02:41:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.48099999999999998,"15min":0.48599999999999999},"min":{"1min":0.29799999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68400000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.50700000000000001,"15min":0.51100000000000001},"min":{"1min":0.37,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70399999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.52300000000000002}]},{"osd":5,"last update":"Sat Jan 27 02:41:14 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.53800000000000003,"15min":0.53500000000000003},"min":{"1min":0.39400000000000002,"5min":0.32400000000000001,"15min":0.307},"max":{"1min":0.75900000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.36599999999999999,"5min":0.33900000000000002,"15min":0.33500000000000002},"max":{"1min":0.72699999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.63100000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:41:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.41399999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.79000000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.42699999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81799999999999995,"5min":0.83499999999999996,"15min":0.90800000000000003},"last":0.68000000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574971,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":891,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194421,"statfs":{"total":200000000,"available":199087216,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63800000000000001,"15min":0.56200000000000006},"min":{"1min":0.46300000000000002,"5min":0.375,"15min":0.29999999999999999},"max":{"1min":0.81499999999999995,"5min":4.125,"15min":4.125},"last":0.623},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60599999999999998,"15min":0.55500000000000005},"min":{"1min":0.439,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73499999999999999,"5min":1.216,"15min":1.216},"last":0.64700000000000002}]},{"osd":1,"last update":"Sat Jan 27 02:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.67400000000000004,"15min":0.57299999999999995},"min":{"1min":0.46400000000000002,"5min":0.39900000000000002,"15min":0.34100000000000003},"max":{"1min":0.81200000000000006,"5min":8.2699999999999996,"15min":8.2699999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.88900000000000001,"15min":0.66800000000000004},"min":{"1min":0.46100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79100000000000004,"5min":26.411999999999999,"15min":26.411999999999999},"last":0.66100000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.76000000000000001,"15min":0.61699999999999999},"min":{"1min":0.39500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78300000000000003,"5min":17.143999999999998,"15min":17.143999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63500000000000001,"15min":0.60199999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83699999999999997,"5min":1.76,"15min":1.76},"last":0.69999999999999996}]},{"osd":3,"last update":"Sat Jan 27 02:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.622,"15min":0.57499999999999996},"min":{"1min":0.47999999999999998,"5min":0.40699999999999997,"15min":0.376},"max":{"1min":0.79800000000000004,"5min":4.3689999999999998,"15min":4.3689999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.625,"15min":0.59199999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41199999999999998},"max":{"1min":0.81000000000000005,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.67200000000000004}]},{"osd":4,"last update":"Sat Jan 27 02:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.47199999999999998,"15min":0.48299999999999998},"min":{"1min":0.34699999999999998,"5min":0.311,"15min":0.30199999999999999},"max":{"1min":0.67500000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.50700000000000001,"15min":0.53600000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76500000000000001,"5min":0.84899999999999998,"15min":0.89600000000000002},"last":0.56599999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52300000000000002,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.65800000000000003,"5min":0.80800000000000005,"15min":0.81299999999999994},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.55300000000000005,"15min":0.58699999999999997},"min":{"1min":0.36799999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77000000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.42099999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:41:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.57999999999999996,"15min":0.60299999999999998},"min":{"1min":0.40400000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.70499999999999996,"5min":0.88700000000000001,"15min":1.0009999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54800000000000004,"15min":0.57499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78000000000000003,"5min":0.82899999999999996,"15min":1.038},"last":0.51700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":891,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194420,"statfs":{"total":200000000,"available":199086808,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.40799999999999997,"15min":0.38700000000000001},"min":{"1min":0.29299999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.64900000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.42999999999999999,"15min":0.40899999999999997},"min":{"1min":0.34799999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56399999999999995,"5min":1.121,"15min":1.121},"last":0.47099999999999997}]},{"osd":2,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.41999999999999998,"15min":0.38400000000000001},"min":{"1min":0.318,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":0.70199999999999996,"5min":1.25,"15min":1.25},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.45400000000000001,"15min":0.439},"min":{"1min":0.317,"5min":0.29999999999999999,"15min":0.29899999999999999},"max":{"1min":0.66700000000000004,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.40200000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.49399999999999999,"15min":0.47799999999999998},"min":{"1min":0.38400000000000001,"5min":0.33100000000000002,"15min":0.32100000000000001},"max":{"1min":0.64700000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.47599999999999998,"15min":0.46300000000000002},"min":{"1min":0.35099999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.63200000000000001,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.38100000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.68799999999999994,"15min":0.64700000000000002},"min":{"1min":0.54100000000000004,"5min":0.442,"15min":0.373},"max":{"1min":1.075,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.623},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.67800000000000005,"15min":0.65000000000000002},"min":{"1min":0.59999999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97199999999999998,"5min":1.026,"15min":1.026},"last":0.72399999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71399999999999997,"15min":0.65500000000000003},"min":{"1min":0.58299999999999996,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":0.94599999999999995,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71499999999999997,"15min":0.68200000000000005},"min":{"1min":0.52800000000000002,"5min":0.44,"15min":0.42399999999999999},"max":{"1min":1.004,"5min":3.181,"15min":3.181},"last":0.68500000000000005}]},{"osd":6,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.72199999999999998,"15min":0.70499999999999996},"min":{"1min":0.68200000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.75600000000000001,"15min":0.70099999999999996},"min":{"1min":0.65000000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.109,"5min":1.869,"15min":1.869},"last":0.78300000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.754,"15min":0.70599999999999996},"min":{"1min":0.64500000000000002,"5min":0.51800000000000002,"15min":0.47299999999999998},"max":{"1min":1.056,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.73999999999999999,"15min":0.70599999999999996},"min":{"1min":0.66600000000000004,"5min":0.51900000000000002,"15min":0.45300000000000001},"max":{"1min":1.0429999999999999,"5min":1.708,"15min":1.708},"last":0.64800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574971,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1469,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193843,"statfs":{"total":200000000,"available":198495424,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.40000000000000002,"15min":0.39300000000000002},"min":{"1min":0.25700000000000001,"5min":0.252,"15min":0.23799999999999999},"max":{"1min":0.58999999999999997,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.442,"5min":0.433,"15min":0.41999999999999998},"min":{"1min":0.34000000000000002,"5min":0.26000000000000001,"15min":0.223},"max":{"1min":0.65300000000000002,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.41499999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42599999999999999,"15min":0.42299999999999999},"min":{"1min":0.30499999999999999,"5min":0.29899999999999999,"15min":0.28199999999999997},"max":{"1min":0.625,"5min":0.625,"15min":0.64000000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44900000000000001,"15min":0.439},"min":{"1min":0.32300000000000001,"5min":0.308,"15min":0.28799999999999998},"max":{"1min":0.59099999999999997,"5min":0.745,"15min":0.745},"last":0.47599999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.443,"15min":0.435},"min":{"1min":0.315,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.61499999999999999,"5min":0.66100000000000003,"15min":0.67300000000000004},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.495,"15min":0.47999999999999998},"min":{"1min":0.35999999999999999,"5min":0.34300000000000003,"15min":0.29099999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71699999999999997},"last":0.436}]},{"osd":4,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66900000000000004,"15min":0.64900000000000002},"min":{"1min":0.502,"5min":0.432,"15min":0.42099999999999999},"max":{"1min":0.89500000000000002,"5min":0.997,"15min":0.997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65200000000000002,"15min":0.65600000000000003},"min":{"1min":0.48899999999999999,"5min":0.42199999999999999,"15min":0.41999999999999998},"max":{"1min":0.81699999999999995,"5min":0.92100000000000004,"15min":0.96699999999999997},"last":0.57199999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67200000000000004,"15min":0.66100000000000003},"min":{"1min":0.505,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":2.4860000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.68799999999999994,"15min":0.68300000000000005},"min":{"1min":0.51800000000000002,"5min":0.5,"15min":0.44},"max":{"1min":0.86199999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.54300000000000004}]},{"osd":6,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.46899999999999997,"5min":0.45600000000000002,"15min":0.42499999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70199999999999996,"15min":0.68600000000000005},"min":{"1min":0.50700000000000001,"5min":0.46500000000000002,"15min":0.45000000000000001},"max":{"1min":0.96499999999999997,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.58399999999999996}]},{"osd":7,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70399999999999996,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.45900000000000002,"15min":0.42599999999999999},"max":{"1min":0.94999999999999996,"5min":1.014,"15min":1.014},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69499999999999995,"15min":0.69199999999999995},"min":{"1min":0.499,"5min":0.498,"15min":0.498},"max":{"1min":0.89300000000000002,"5min":0.95799999999999996,"15min":0.97299999999999998},"last":0.499}]}]},{"osd":2,"up_from":13,"seq":55834574971,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":892,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194419,"statfs":{"total":200000000,"available":199085756,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.247,"5min":0.23599999999999999,"15min":0.23000000000000001},"max":{"1min":0.59299999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.57199999999999995},"min":{"1min":0.378,"5min":0.32100000000000001,"15min":0.30499999999999999},"max":{"1min":0.67700000000000005,"5min":0.72799999999999998,"15min":6.2729999999999997},"last":0.42499999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.41999999999999998,"15min":0.42099999999999999},"min":{"1min":0.312,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.64100000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.439,"15min":0.44400000000000001},"min":{"1min":0.32200000000000001,"5min":0.29399999999999998,"15min":0.28899999999999998},"max":{"1min":0.58699999999999997,"5min":0.65300000000000002,"15min":0.70099999999999996},"last":0.47899999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.41899999999999998,"15min":0.41699999999999998},"min":{"1min":0.30399999999999999,"5min":0.28499999999999998,"15min":0.27800000000000002},"max":{"1min":0.60699999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.48299999999999998,"15min":0.48499999999999999},"min":{"1min":0.379,"5min":0.314,"15min":0.314},"max":{"1min":0.70699999999999996,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.45200000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67800000000000005,"15min":0.66300000000000003},"min":{"1min":0.501,"5min":0.46800000000000003,"15min":0.39300000000000002},"max":{"1min":0.89200000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.70199999999999996,"15min":0.76400000000000001},"min":{"1min":0.55600000000000005,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92600000000000005,"5min":1.006,"15min":3.581},"last":0.80400000000000005}]},{"osd":5,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.65700000000000003,"15min":0.71099999999999997},"min":{"1min":0.51100000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85399999999999998,"5min":0.95899999999999996,"15min":3.5369999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65700000000000003,"15min":0.66700000000000004},"min":{"1min":0.44800000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89000000000000001,"5min":0.92200000000000004,"15min":1.0529999999999999},"last":0.77700000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.67900000000000005,"15min":0.67000000000000004},"min":{"1min":0.48699999999999999,"5min":0.45000000000000001,"15min":0.42299999999999999},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69599999999999995,"15min":0.68500000000000005},"min":{"1min":0.54800000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.72299999999999998}]},{"osd":7,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69899999999999995,"15min":0.68300000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.71899999999999997,"15min":0.69899999999999995},"min":{"1min":0.53200000000000003,"5min":0.441,"15min":0.42699999999999999},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.68400000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574971,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":891,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194421,"statfs":{"total":200000000,"available":199087272,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.46100000000000002,"15min":0.39400000000000002},"min":{"1min":0.27500000000000002,"5min":0.255,"15min":0.253},"max":{"1min":0.54400000000000004,"5min":4.7380000000000004,"15min":4.7380000000000004},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.42799999999999999,"15min":0.40600000000000003},"min":{"1min":0.29899999999999999,"5min":0.27600000000000002,"15min":0.26500000000000001},"max":{"1min":0.58499999999999996,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.57999999999999996}]},{"osd":1,"last update":"Sat Jan 27 02:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.46400000000000002,"15min":0.41799999999999998},"min":{"1min":0.28299999999999997,"5min":0.24399999999999999,"15min":0.22900000000000001},"max":{"1min":0.52200000000000002,"5min":4.5739999999999998,"15min":4.5739999999999998},"last":0.436},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.53300000000000003,"15min":0.45900000000000002},"min":{"1min":0.314,"5min":0.254,"15min":0.254},"max":{"1min":0.65100000000000002,"5min":5.0529999999999999,"15min":5.0529999999999999},"last":0.46100000000000002}]},{"osd":2,"last update":"Sat Jan 27 02:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.49199999999999999,"15min":0.45100000000000001},"min":{"1min":0.40899999999999997,"5min":0.33700000000000002,"15min":0.28100000000000003},"max":{"1min":0.61399999999999999,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.505},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55600000000000005,"15min":0.48599999999999999},"min":{"1min":0.33000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73499999999999999,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":0.52500000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70499999999999996,"15min":0.64400000000000002},"min":{"1min":0.42999999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91100000000000003,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.624},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.629},"min":{"1min":0.49299999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94799999999999995,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.64700000000000002}]},{"osd":5,"last update":"Sat Jan 27 02:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67400000000000004,"15min":0.63500000000000001},"min":{"1min":0.52600000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.89000000000000001,"5min":1.409,"15min":1.409},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.79300000000000004,"15min":0.70699999999999996},"min":{"1min":0.48799999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.96599999999999997,"5min":4.7960000000000003,"15min":4.7960000000000003},"last":0.67300000000000004}]},{"osd":6,"last update":"Sat Jan 27 02:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.74299999999999999,"15min":0.68400000000000005},"min":{"1min":0.56599999999999995,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93500000000000005,"5min":5.0069999999999997,"15min":5.0069999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71099999999999997,"15min":0.66600000000000004},"min":{"1min":0.54300000000000004,"5min":0.47099999999999997,"15min":0.435},"max":{"1min":0.85599999999999998,"5min":1.103,"15min":1.103},"last":0.69599999999999995}]},{"osd":7,"last update":"Sat Jan 27 02:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.754,"15min":0.67800000000000005},"min":{"1min":0.495,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92100000000000004,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72699999999999998,"15min":0.68600000000000005},"min":{"1min":0.58699999999999997,"5min":0.435,"15min":0.435},"max":{"1min":0.85999999999999999,"5min":1.361,"15min":1.361},"last":0.59499999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574971,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":891,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194421,"statfs":{"total":200000000,"available":199087362,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.47799999999999998,"5min":0.34599999999999997,"15min":0.314},"max":{"1min":0.77800000000000002,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66300000000000003,"15min":0.61599999999999999},"min":{"1min":0.47399999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.78500000000000003,"5min":7.0670000000000002,"15min":7.0670000000000002},"last":0.51900000000000002}]},{"osd":1,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63800000000000001,"15min":0.61199999999999999},"min":{"1min":0.54900000000000004,"5min":0.39500000000000002,"15min":0.35399999999999998},"max":{"1min":0.74399999999999999,"5min":5.1529999999999996,"15min":5.1529999999999996},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64600000000000002,"15min":0.61199999999999999},"min":{"1min":0.51800000000000002,"5min":0.40799999999999997,"15min":0.39000000000000001},"max":{"1min":0.76900000000000002,"5min":3.0419999999999998,"15min":3.0419999999999998},"last":0.45200000000000001}]},{"osd":2,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63600000000000001,"15min":0.61199999999999999},"min":{"1min":0.44400000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73999999999999999,"5min":5.0019999999999998,"15min":5.0019999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.628,"15min":0.61799999999999999},"min":{"1min":0.48099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.75900000000000001,"5min":2.073,"15min":2.073},"last":0.60199999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.66400000000000003,"15min":0.60899999999999999},"min":{"1min":0.53000000000000003,"5min":0.42099999999999999,"15min":0.36199999999999999},"max":{"1min":0.76400000000000001,"5min":6.7220000000000004,"15min":6.7220000000000004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.68500000000000005,"15min":0.66000000000000003},"min":{"1min":0.53100000000000003,"5min":0.45400000000000001,"15min":0.41399999999999998},"max":{"1min":0.82399999999999995,"5min":6.5510000000000002,"15min":6.5510000000000002},"last":0.57399999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.496,"15min":0.48799999999999999},"min":{"1min":0.375,"5min":0.312,"15min":0.312},"max":{"1min":0.71299999999999997,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.56200000000000006,"15min":0.51400000000000001},"min":{"1min":0.35799999999999998,"5min":0.32500000000000001,"15min":0.29999999999999999},"max":{"1min":0.69199999999999995,"5min":6.9269999999999996,"15min":6.9269999999999996},"last":0.42399999999999999}]},{"osd":6,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.55700000000000005,"15min":0.51900000000000002},"min":{"1min":0.373,"5min":0.35899999999999999,"15min":0.32900000000000001},"max":{"1min":0.58299999999999996,"5min":5.2229999999999999,"15min":5.2229999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56000000000000005,"15min":0.54700000000000004},"min":{"1min":0.41599999999999998,"5min":0.35099999999999998,"15min":0.33200000000000002},"max":{"1min":0.70299999999999996,"5min":1.46,"15min":1.46},"last":0.65900000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.61099999999999999,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.34799999999999998,"15min":0.33600000000000002},"max":{"1min":0.72199999999999998,"5min":7.5339999999999998,"15min":7.5339999999999998},"last":0.623},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.622,"15min":0.59099999999999997},"min":{"1min":0.442,"5min":0.34699999999999998,"15min":0.34599999999999997},"max":{"1min":0.73999999999999999,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":0.72699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574971,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":891,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194421,"statfs":{"total":200000000,"available":199087340,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.33900000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78300000000000003,"5min":1.071,"15min":1.071},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.58199999999999996},"min":{"1min":0.38500000000000001,"5min":0.34499999999999997,"15min":0.27600000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":3.242},"last":0.59199999999999997}]},{"osd":1,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63100000000000001,"15min":0.62},"min":{"1min":0.496,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78500000000000003,"5min":1.018,"15min":1.3380000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.61499999999999999,"15min":0.64000000000000001},"min":{"1min":0.47999999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.95499999999999996},"last":0.54600000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.59899999999999998},"min":{"1min":0.48999999999999999,"5min":0.39100000000000001,"15min":0.372},"max":{"1min":0.81999999999999995,"5min":0.96699999999999997,"15min":3.04},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.47899999999999998,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":0.79700000000000004,"5min":0.81000000000000005,"15min":0.89400000000000002},"last":0.64000000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60899999999999999,"15min":0.66100000000000003},"min":{"1min":0.48999999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79000000000000004,"5min":1.0409999999999999,"15min":4.032},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65800000000000003,"15min":0.67800000000000005},"min":{"1min":0.48799999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":4.282},"last":0.65400000000000003}]},{"osd":4,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.501,"15min":0.51700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.26500000000000001},"max":{"1min":0.73099999999999998,"5min":0.81399999999999995,"15min":1.2869999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.498,"15min":0.54100000000000004},"min":{"1min":0.376,"5min":0.32800000000000001,"15min":0.30399999999999999},"max":{"1min":0.748,"5min":0.76300000000000001,"15min":1.101},"last":0.70499999999999996}]},{"osd":6,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.496,"15min":0.53600000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.29099999999999998},"max":{"1min":0.75800000000000001,"5min":0.76100000000000001,"15min":1.7969999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55400000000000005,"15min":0.57399999999999995},"min":{"1min":0.40400000000000003,"5min":0.39100000000000001,"15min":0.33600000000000002},"max":{"1min":0.77500000000000002,"5min":0.83399999999999996,"15min":1.6739999999999999},"last":0.627}]},{"osd":7,"last update":"Sat Jan 27 02:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.56000000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.32700000000000001},"max":{"1min":0.69799999999999995,"5min":0.745,"15min":1.008},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59399999999999997,"15min":0.63800000000000001},"min":{"1min":0.40600000000000003,"5min":0.38400000000000001,"15min":0.33700000000000002},"max":{"1min":0.76800000000000002,"5min":0.91300000000000003,"15min":2},"last":0.69199999999999995}]}]}],"pool_statfs":[]}} 2024-01-27T02:42:02.842 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-27T02:42:02.842 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-27T02:42:03.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.018+0000 7fe56584b700 1 Processor -- start 2024-01-27T02:42:03.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.018+0000 7fe56584b700 1 -- start start 2024-01-27T02:42:03.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.019+0000 7fe56584b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe56015a310 0x7fe5600cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:03.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.019+0000 7fe56584b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5600cfb40 con 0x7fe5600cc110 2024-01-27T02:42:03.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.019+0000 7fe55effd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe5600cc110 0x7fe5600cf2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34940/0 (socket says 172.21.15.7:34940) 2024-01-27T02:42:03.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.019+0000 7fe55effd700 1 -- 172.21.15.7:0/3648000716 learned_addr learned my addr 172.21.15.7:0/3648000716 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:42:03.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.019+0000 7fe56584b700 1 -- 172.21.15.7:0/3648000716 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5600cfe20 con 0x7fe56015ea60 2024-01-27T02:42:03.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.019+0000 7fe56584b700 1 -- 172.21.15.7:0/3648000716 --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5600d0100 con 0x7fe56015a310 2024-01-27T02:42:03.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.019+0000 7fe55f7fe700 1 --2- 172.21.15.7:0/3648000716 >> v2:172.21.15.7:3300/0 conn(0x7fe56015a310 0x7fe5600cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:03.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.020+0000 7fe55dffb700 1 -- 172.21.15.7:0/3648000716 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1866939940 0 0) 0x7fe5600cfb40 con 0x7fe5600cc110 2024-01-27T02:42:03.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.020+0000 7fe55dffb700 1 -- 172.21.15.7:0/3648000716 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe54c003570 con 0x7fe5600cc110 2024-01-27T02:42:03.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.020+0000 7fe55dffb700 1 -- 172.21.15.7:0/3648000716 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3432928019 0 0) 0x7fe5600cfe20 con 0x7fe56015ea60 2024-01-27T02:42:03.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.020+0000 7fe55dffb700 1 -- 172.21.15.7:0/3648000716 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5600cfb40 con 0x7fe56015ea60 2024-01-27T02:42:03.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.020+0000 7fe55f7fe700 1 -- 172.21.15.7:0/3648000716 >> v1:172.21.15.188:6789/0 conn(0x7fe56015ea60 legacy=0x7fe560159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:03.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.020+0000 7fe55f7fe700 1 -- 172.21.15.7:0/3648000716 >> v1:172.21.15.7:6789/0 conn(0x7fe5600cc110 legacy=0x7fe5600cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:03.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.020+0000 7fe55f7fe700 1 -- 172.21.15.7:0/3648000716 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5600d03e0 con 0x7fe56015a310 2024-01-27T02:42:03.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.020+0000 7fe55f7fe700 1 --2- 172.21.15.7:0/3648000716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe56015a310 0x7fe5600cc4e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fe558009e60 tx=0x7fe558009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7cbef4ddc9f203 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:03.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.020+0000 7fe55dffb700 1 -- 172.21.15.7:0/3648000716 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55800e070 con 0x7fe56015a310 2024-01-27T02:42:03.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.020+0000 7fe55dffb700 1 -- 172.21.15.7:0/3648000716 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55800bab0 con 0x7fe56015a310 2024-01-27T02:42:03.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.020+0000 7fe55dffb700 1 -- 172.21.15.7:0/3648000716 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55800be70 con 0x7fe56015a310 2024-01-27T02:42:03.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.021+0000 7fe56584b700 1 -- 172.21.15.7:0/3648000716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe56015a310 msgr2=0x7fe5600cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:03.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.021+0000 7fe56584b700 1 --2- 172.21.15.7:0/3648000716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe56015a310 0x7fe5600cc4e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fe558009e60 tx=0x7fe558009780 comp rx=0 tx=0).stop 2024-01-27T02:42:03.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.021+0000 7fe56584b700 1 -- 172.21.15.7:0/3648000716 shutdown_connections 2024-01-27T02:42:03.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.021+0000 7fe56584b700 1 --2- 172.21.15.7:0/3648000716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe56015a310 0x7fe5600cc4e0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:03.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.021+0000 7fe56584b700 1 -- 172.21.15.7:0/3648000716 >> 172.21.15.7:0/3648000716 conn(0x7fe560063580 msgr2=0x7fe560063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:03.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.021+0000 7fe56584b700 1 -- 172.21.15.7:0/3648000716 shutdown_connections 2024-01-27T02:42:03.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.021+0000 7fe56584b700 1 -- 172.21.15.7:0/3648000716 wait complete. 2024-01-27T02:42:03.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.022+0000 7fe56584b700 1 Processor -- start 2024-01-27T02:42:03.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.022+0000 7fe56584b700 1 -- start start 2024-01-27T02:42:03.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.023+0000 7fe56584b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5600cc110 0x7fe5600c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:03.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.023+0000 7fe56584b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe56015a310 0x7fe560151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:03.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.023+0000 7fe56584b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe56015ea60 0x7fe560151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:03.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.023+0000 7fe56584b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5601510e0 con 0x7fe56015ea60 2024-01-27T02:42:03.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.023+0000 7fe56584b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5601504d0 con 0x7fe5600cc110 2024-01-27T02:42:03.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.023+0000 7fe56584b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5601507b0 con 0x7fe56015a310 2024-01-27T02:42:03.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.023+0000 7fe55f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe56015ea60 0x7fe560151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.023+0000 7fe55f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe56015ea60 0x7fe560151ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50444/0 (socket says 172.21.15.7:50444) 2024-01-27T02:42:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.023+0000 7fe55f7fe700 1 -- 172.21.15.7:0/196826870 learned_addr learned my addr 172.21.15.7:0/196826870 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:42:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.023+0000 7fe55effd700 1 --2- 172.21.15.7:0/196826870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5600cc110 0x7fe5600c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.023+0000 7fe55e7fc700 1 --2- 172.21.15.7:0/196826870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe56015a310 0x7fe560151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.024+0000 7fe55f7fe700 1 -- 172.21.15.7:0/196826870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe56015a310 msgr2=0x7fe560151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.024+0000 7fe55f7fe700 1 --2- 172.21.15.7:0/196826870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe56015a310 0x7fe560151570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.024+0000 7fe55f7fe700 1 -- 172.21.15.7:0/196826870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5600cc110 msgr2=0x7fe5600c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.024+0000 7fe55f7fe700 1 --2- 172.21.15.7:0/196826870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5600cc110 0x7fe5600c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.024+0000 7fe55f7fe700 1 -- 172.21.15.7:0/196826870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe558009a20 con 0x7fe56015ea60 2024-01-27T02:42:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.024+0000 7fe55f7fe700 1 --2- 172.21.15.7:0/196826870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe56015ea60 0x7fe560151ab0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fe56008a040 tx=0x7fe558019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.024+0000 7fe556ffd700 1 -- 172.21.15.7:0/196826870 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe558012db0 con 0x7fe56015ea60 2024-01-27T02:42:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.024+0000 7fe556ffd700 1 -- 172.21.15.7:0/196826870 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe558026480 con 0x7fe56015ea60 2024-01-27T02:42:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.024+0000 7fe56584b700 1 -- 172.21.15.7:0/196826870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe560150a90 con 0x7fe56015ea60 2024-01-27T02:42:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.024+0000 7fe556ffd700 1 -- 172.21.15.7:0/196826870 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe558026840 con 0x7fe56015ea60 2024-01-27T02:42:03.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.024+0000 7fe56584b700 1 -- 172.21.15.7:0/196826870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe56014ffc0 con 0x7fe56015ea60 2024-01-27T02:42:03.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.025+0000 7fe56584b700 1 -- 172.21.15.7:0/196826870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe560151290 con 0x7fe56015ea60 2024-01-27T02:42:03.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.026+0000 7fe556ffd700 1 -- 172.21.15.7:0/196826870 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe55801c400 con 0x7fe56015ea60 2024-01-27T02:42:03.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.026+0000 7fe556ffd700 1 --2- 172.21.15.7:0/196826870 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe540066f30 0x7fe5400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:03.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.026+0000 7fe556ffd700 1 -- 172.21.15.7:0/196826870 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe558086410 con 0x7fe56015ea60 2024-01-27T02:42:03.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.026+0000 7fe55effd700 1 --2- 172.21.15.7:0/196826870 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe540066f30 0x7fe5400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:03.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.027+0000 7fe55effd700 1 --2- 172.21.15.7:0/196826870 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe540066f30 0x7fe5400693e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe550007fd0 tx=0x7fe550009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:03.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.030+0000 7fe556ffd700 1 -- 172.21.15.7:0/196826870 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe560151290 con 0x7fe56015ea60 2024-01-27T02:42:03.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.318+0000 7fe56584b700 1 -- 172.21.15.7:0/196826870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe5600cc4e0 con 0x7fe56015ea60 2024-01-27T02:42:03.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.322+0000 7fe556ffd700 1 -- 172.21.15.7:0/196826870 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+0 (secure 0 0 0) 0x7fe558053680 con 0x7fe56015ea60 2024-01-27T02:42:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.324+0000 7fe56584b700 1 -- 172.21.15.7:0/196826870 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe540066f30 msgr2=0x7fe5400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.324+0000 7fe56584b700 1 --2- 172.21.15.7:0/196826870 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe540066f30 0x7fe5400693e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe550007fd0 tx=0x7fe550009040 comp rx=0 tx=0).stop 2024-01-27T02:42:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.324+0000 7fe56584b700 1 -- 172.21.15.7:0/196826870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe56015ea60 msgr2=0x7fe560151ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.324+0000 7fe56584b700 1 --2- 172.21.15.7:0/196826870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe56015ea60 0x7fe560151ab0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fe56008a040 tx=0x7fe558019000 comp rx=0 tx=0).stop 2024-01-27T02:42:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.324+0000 7fe56584b700 1 -- 172.21.15.7:0/196826870 shutdown_connections 2024-01-27T02:42:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.324+0000 7fe56584b700 1 --2- 172.21.15.7:0/196826870 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe540066f30 0x7fe5400693e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.324+0000 7fe56584b700 1 --2- 172.21.15.7:0/196826870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe56015ea60 0x7fe560151ab0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.324+0000 7fe56584b700 1 --2- 172.21.15.7:0/196826870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5600cc110 0x7fe5600c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.324+0000 7fe56584b700 1 --2- 172.21.15.7:0/196826870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe56015a310 0x7fe560151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.324+0000 7fe56584b700 1 -- 172.21.15.7:0/196826870 >> 172.21.15.7:0/196826870 conn(0x7fe560063580 msgr2=0x7fe560159be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.324+0000 7fe56584b700 1 -- 172.21.15.7:0/196826870 shutdown_connections 2024-01-27T02:42:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.324+0000 7fe56584b700 1 -- 172.21.15.7:0/196826870 wait complete. 2024-01-27T02:42:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-01-27T02:42:03.339 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:42:03.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.470+0000 7f49e2d6c700 1 Processor -- start 2024-01-27T02:42:03.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.470+0000 7f49e2d6c700 1 -- start start 2024-01-27T02:42:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.470+0000 7f49e2d6c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49dc0b2f80 0x7f49dc06a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.470+0000 7f49e2d6c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f49dc0b3350 0x7f49dc0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.470+0000 7f49e2d6c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f49dc0ce810 0x7f49dc1553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.470+0000 7f49e2d6c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49dc06a900 con 0x7f49dc0ce810 2024-01-27T02:42:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.470+0000 7f49e2d6c700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49dc159390 con 0x7f49dc0b3350 2024-01-27T02:42:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.470+0000 7f49e2d6c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc158780 con 0x7f49dc0b2f80 2024-01-27T02:42:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.471+0000 7f49e1309700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f49dc0ce810 0x7f49dc1553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.471+0000 7f49e1309700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f49dc0ce810 0x7f49dc1553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50448/0 (socket says 172.21.15.7:50448) 2024-01-27T02:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.471+0000 7f49e1309700 1 -- 172.21.15.7:0/4069683216 learned_addr learned my addr 172.21.15.7:0/4069683216 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.471+0000 7f49e0b08700 1 --2- 172.21.15.7:0/4069683216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49dc0b2f80 0x7f49dc06a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.471+0000 7f49dbfff700 1 --2- 172.21.15.7:0/4069683216 >> v2:172.21.15.188:3300/0 conn(0x7f49dc0b3350 0x7f49dc0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.471+0000 7f49e1309700 1 -- 172.21.15.7:0/4069683216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49dc0b2f80 msgr2=0x7f49dc06a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.471+0000 7f49e1309700 1 --2- 172.21.15.7:0/4069683216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49dc0b2f80 0x7f49dc06a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.471+0000 7f49e1309700 1 -- 172.21.15.7:0/4069683216 >> v2:172.21.15.188:3300/0 conn(0x7f49dc0b3350 msgr2=0x7f49dc0ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.471+0000 7f49e1309700 1 --2- 172.21.15.7:0/4069683216 >> v2:172.21.15.188:3300/0 conn(0x7f49dc0b3350 0x7f49dc0ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.471+0000 7f49e1309700 1 -- 172.21.15.7:0/4069683216 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49dc158a60 con 0x7f49dc0ce810 2024-01-27T02:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.471+0000 7f49e1309700 1 --2- 172.21.15.7:0/4069683216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49dc0ce810 0x7f49dc1553e0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f49d4009f20 tx=0x7f49d4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a54ce01c1f0584e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.471+0000 7f49db7fe700 1 -- 172.21.15.7:0/4069683216 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d400e070 con 0x7f49dc0ce810 2024-01-27T02:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.472+0000 7f49db7fe700 1 -- 172.21.15.7:0/4069683216 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d400b930 con 0x7f49dc0ce810 2024-01-27T02:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.472+0000 7f49db7fe700 1 -- 172.21.15.7:0/4069683216 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d40092e0 con 0x7f49dc0ce810 2024-01-27T02:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.472+0000 7f49e2d6c700 1 -- 172.21.15.7:0/4069683216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49dc0ce810 msgr2=0x7f49dc1553e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.472+0000 7f49e2d6c700 1 --2- 172.21.15.7:0/4069683216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49dc0ce810 0x7f49dc1553e0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f49d4009f20 tx=0x7f49d4009840 comp rx=0 tx=0).stop 2024-01-27T02:42:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.472+0000 7f49e2d6c700 1 -- 172.21.15.7:0/4069683216 shutdown_connections 2024-01-27T02:42:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.472+0000 7f49e2d6c700 1 --2- 172.21.15.7:0/4069683216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49dc0ce810 0x7f49dc1553e0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.472+0000 7f49e2d6c700 1 --2- 172.21.15.7:0/4069683216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49dc0b2f80 0x7f49dc06a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.472+0000 7f49e2d6c700 1 --2- 172.21.15.7:0/4069683216 >> v2:172.21.15.188:3300/0 conn(0x7f49dc0b3350 0x7f49dc0ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.472+0000 7f49e2d6c700 1 -- 172.21.15.7:0/4069683216 >> 172.21.15.7:0/4069683216 conn(0x7f49dc068430 msgr2=0x7f49dc06be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.472+0000 7f49e2d6c700 1 -- 172.21.15.7:0/4069683216 shutdown_connections 2024-01-27T02:42:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.472+0000 7f49e2d6c700 1 -- 172.21.15.7:0/4069683216 wait complete. 2024-01-27T02:42:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.473+0000 7f49e2d6c700 1 Processor -- start 2024-01-27T02:42:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.474+0000 7f49e2d6c700 1 -- start start 2024-01-27T02:42:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.474+0000 7f49e2d6c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49dc0b2f80 0x7f49dc16aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.474+0000 7f49e2d6c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49dc0b3350 0x7f49dc15dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.474+0000 7f49e2d6c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49dc0ce810 0x7f49dc15e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.474+0000 7f49e2d6c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc16af80 con 0x7f49dc0b3350 2024-01-27T02:42:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.474+0000 7f49e2d6c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc16b260 con 0x7f49dc0b2f80 2024-01-27T02:42:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.474+0000 7f49e2d6c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc15e960 con 0x7f49dc0ce810 2024-01-27T02:42:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.475+0000 7f49dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49dc0b3350 0x7f49dc15dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.475+0000 7f49dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49dc0b3350 0x7f49dc15dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50464/0 (socket says 172.21.15.7:50464) 2024-01-27T02:42:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.475+0000 7f49dbfff700 1 -- 172.21.15.7:0/3291428090 learned_addr learned my addr 172.21.15.7:0/3291428090 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:42:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.475+0000 7f49e0b08700 1 --2- 172.21.15.7:0/3291428090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49dc0b2f80 0x7f49dc16aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.475+0000 7f49e1309700 1 --2- 172.21.15.7:0/3291428090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49dc0ce810 0x7f49dc15e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.475+0000 7f49dbfff700 1 -- 172.21.15.7:0/3291428090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49dc0ce810 msgr2=0x7f49dc15e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.475+0000 7f49dbfff700 1 --2- 172.21.15.7:0/3291428090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49dc0ce810 0x7f49dc15e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.475+0000 7f49dbfff700 1 -- 172.21.15.7:0/3291428090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49dc0b2f80 msgr2=0x7f49dc16aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.475+0000 7f49dbfff700 1 --2- 172.21.15.7:0/3291428090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49dc0b2f80 0x7f49dc16aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:03.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.475+0000 7f49dbfff700 1 -- 172.21.15.7:0/3291428090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d4009ae0 con 0x7f49dc0b3350 2024-01-27T02:42:03.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.475+0000 7f49dbfff700 1 --2- 172.21.15.7:0/3291428090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49dc0b3350 0x7f49dc15dee0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f49c800ecb0 tx=0x7f49c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:03.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.475+0000 7f49d8ff9700 1 -- 172.21.15.7:0/3291428090 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c800c7f0 con 0x7f49dc0b3350 2024-01-27T02:42:03.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.475+0000 7f49d8ff9700 1 -- 172.21.15.7:0/3291428090 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c800c950 con 0x7f49dc0b3350 2024-01-27T02:42:03.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.475+0000 7f49d8ff9700 1 -- 172.21.15.7:0/3291428090 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c800cd10 con 0x7f49dc0b3350 2024-01-27T02:42:03.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.475+0000 7f49e2d6c700 1 -- 172.21.15.7:0/3291428090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49dc0b1d40 con 0x7f49dc0b3350 2024-01-27T02:42:03.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.475+0000 7f49e2d6c700 1 -- 172.21.15.7:0/3291428090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49dc0b2300 con 0x7f49dc0b3350 2024-01-27T02:42:03.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.476+0000 7f49e2d6c700 1 -- 172.21.15.7:0/3291428090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49dc04c710 con 0x7f49dc0b3350 2024-01-27T02:42:03.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.477+0000 7f49d8ff9700 1 -- 172.21.15.7:0/3291428090 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f49c8013680 con 0x7f49dc0b3350 2024-01-27T02:42:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.477+0000 7f49d8ff9700 1 --2- 172.21.15.7:0/3291428090 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f49c0066f60 0x7f49c0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.478+0000 7f49d8ff9700 1 -- 172.21.15.7:0/3291428090 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f49c80846f0 con 0x7f49dc0b3350 2024-01-27T02:42:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.478+0000 7f49e0b08700 1 --2- 172.21.15.7:0/3291428090 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f49c0066f60 0x7f49c0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.478+0000 7f49e0b08700 1 --2- 172.21.15.7:0/3291428090 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f49c0066f60 0x7f49c0069410 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009db0 tx=0x7f49d00091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:03.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.482+0000 7f49d8ff9700 1 -- 172.21.15.7:0/3291428090 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f49c80518c0 con 0x7f49dc0b3350 2024-01-27T02:42:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.730+0000 7f49e2d6c700 1 -- 172.21.15.7:0/3291428090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49dc16b3a0 con 0x7f49dc0b3350 2024-01-27T02:42:03.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.730+0000 7f49d8ff9700 1 -- 172.21.15.7:0/3291428090 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f49c8054ee0 con 0x7f49dc0b3350 2024-01-27T02:42:03.732 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:42:03.732 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6473,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/3449593528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3449593528},{"type":"v1","addr":"172.21.15.188:6836","nonce":3449593528}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3473642033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3473642033},{"type":"v1","addr":"172.21.15.188:6837","nonce":3473642033}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6479,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2550009140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2550009140},{"type":"v1","addr":"172.21.15.7:6833","nonce":2550009140}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:41:57.646012+0000","modified":"2024-01-27T02:42:01.251419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6485":{"gid":6485,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3213683076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3213683076},{"type":"v1","addr":"172.21.15.7:6835","nonce":3213683076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2024-01-27T02:42:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.732+0000 7f49e2d6c700 1 -- 172.21.15.7:0/3291428090 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f49c0066f60 msgr2=0x7f49c0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.732+0000 7f49e2d6c700 1 --2- 172.21.15.7:0/3291428090 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f49c0066f60 0x7f49c0069410 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009db0 tx=0x7f49d00091a0 comp rx=0 tx=0).stop 2024-01-27T02:42:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.732+0000 7f49e2d6c700 1 -- 172.21.15.7:0/3291428090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49dc0b3350 msgr2=0x7f49dc15dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.732+0000 7f49e2d6c700 1 --2- 172.21.15.7:0/3291428090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49dc0b3350 0x7f49dc15dee0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f49c800ecb0 tx=0x7f49c80091a0 comp rx=0 tx=0).stop 2024-01-27T02:42:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.732+0000 7f49e2d6c700 1 -- 172.21.15.7:0/3291428090 shutdown_connections 2024-01-27T02:42:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.732+0000 7f49e2d6c700 1 --2- 172.21.15.7:0/3291428090 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f49c0066f60 0x7f49c0069410 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.732+0000 7f49e2d6c700 1 --2- 172.21.15.7:0/3291428090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49dc0ce810 0x7f49dc15e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.732+0000 7f49e2d6c700 1 --2- 172.21.15.7:0/3291428090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49dc0b2f80 0x7f49dc16aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.732+0000 7f49e2d6c700 1 --2- 172.21.15.7:0/3291428090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49dc0b3350 0x7f49dc15dee0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.732+0000 7f49e2d6c700 1 -- 172.21.15.7:0/3291428090 >> 172.21.15.7:0/3291428090 conn(0x7f49dc068430 msgr2=0x7f49dc06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.732+0000 7f49e2d6c700 1 -- 172.21.15.7:0/3291428090 shutdown_connections 2024-01-27T02:42:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:03.733+0000 7f49e2d6c700 1 -- 172.21.15.7:0/3291428090 wait complete. 2024-01-27T02:42:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 86 2024-01-27T02:42:04.150 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-27T02:41:57.646012+0000', 'modified': '2024-01-27T02:42:01.251419+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6485}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6485': {'gid': 6485, 'name': 'c', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6835/3213683076', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 3213683076}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 3213683076}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-27T02:42:04.150 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-27T02:42:04.150 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-27T02:42:04.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.292+0000 7f5b7be76700 1 Processor -- start 2024-01-27T02:42:04.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.292+0000 7f5b7be76700 1 -- start start 2024-01-27T02:42:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.293+0000 7f5b7be76700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b740cd770 0x7f5b740c9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.293+0000 7f5b7be76700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5b74068af0 0x7f5b74068f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.293+0000 7f5b7be76700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5b740cdb40 0x7f5b740cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.293+0000 7f5b7be76700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b740c9890 con 0x7f5b74068af0 2024-01-27T02:42:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.293+0000 7f5b7be76700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b74067a50 con 0x7f5b740cdb40 2024-01-27T02:42:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.293+0000 7f5b7be76700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b74067d30 con 0x7f5b740cd770 2024-01-27T02:42:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.293+0000 7f5b79411700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5b74068af0 0x7f5b74068f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.293+0000 7f5b79411700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5b74068af0 0x7f5b74068f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50468/0 (socket says 172.21.15.7:50468) 2024-01-27T02:42:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.293+0000 7f5b79411700 1 -- 172.21.15.7:0/90722771 learned_addr learned my addr 172.21.15.7:0/90722771 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:42:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.293+0000 7f5b7a413700 1 --2- 172.21.15.7:0/90722771 >> v2:172.21.15.188:3300/0 conn(0x7f5b740cdb40 0x7f5b740cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.293+0000 7f5b79c12700 1 --2- 172.21.15.7:0/90722771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b740cd770 0x7f5b740c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.294+0000 7f5b79411700 1 -- 172.21.15.7:0/90722771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b740cd770 msgr2=0x7f5b740c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.294+0000 7f5b79411700 1 --2- 172.21.15.7:0/90722771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b740cd770 0x7f5b740c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.294+0000 7f5b79411700 1 -- 172.21.15.7:0/90722771 >> v2:172.21.15.188:3300/0 conn(0x7f5b740cdb40 msgr2=0x7f5b740cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.294+0000 7f5b79411700 1 --2- 172.21.15.7:0/90722771 >> v2:172.21.15.188:3300/0 conn(0x7f5b740cdb40 0x7f5b740cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.294+0000 7f5b79411700 1 -- 172.21.15.7:0/90722771 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b74068010 con 0x7f5b74068af0 2024-01-27T02:42:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.294+0000 7f5b79c12700 1 --2- 172.21.15.7:0/90722771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b740cd770 0x7f5b740c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:42:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.294+0000 7f5b79411700 1 --2- 172.21.15.7:0/90722771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b74068af0 0x7f5b74068f60 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f5b7406df80 tx=0x7f5b68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6847bec8dde52a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.294+0000 7f5b78c10700 1 -- 172.21.15.7:0/90722771 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6800b860 con 0x7f5b74068af0 2024-01-27T02:42:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.294+0000 7f5b78c10700 1 -- 172.21.15.7:0/90722771 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6800b9c0 con 0x7f5b74068af0 2024-01-27T02:42:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.294+0000 7f5b78c10700 1 -- 172.21.15.7:0/90722771 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6800bd60 con 0x7f5b74068af0 2024-01-27T02:42:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.294+0000 7f5b7be76700 1 -- 172.21.15.7:0/90722771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b74068af0 msgr2=0x7f5b74068f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.294+0000 7f5b7be76700 1 --2- 172.21.15.7:0/90722771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b74068af0 0x7f5b74068f60 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f5b7406df80 tx=0x7f5b68009250 comp rx=0 tx=0).stop 2024-01-27T02:42:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.295+0000 7f5b7be76700 1 -- 172.21.15.7:0/90722771 shutdown_connections 2024-01-27T02:42:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.295+0000 7f5b7be76700 1 --2- 172.21.15.7:0/90722771 >> v2:172.21.15.188:3300/0 conn(0x7f5b740cdb40 0x7f5b740cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.295+0000 7f5b7be76700 1 --2- 172.21.15.7:0/90722771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b740cd770 0x7f5b740c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.295+0000 7f5b7be76700 1 --2- 172.21.15.7:0/90722771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b74068af0 0x7f5b74068f60 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.295+0000 7f5b7be76700 1 -- 172.21.15.7:0/90722771 >> 172.21.15.7:0/90722771 conn(0x7f5b74063d60 msgr2=0x7f5b740ced10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:04.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.295+0000 7f5b7be76700 1 -- 172.21.15.7:0/90722771 shutdown_connections 2024-01-27T02:42:04.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.295+0000 7f5b7be76700 1 -- 172.21.15.7:0/90722771 wait complete. 2024-01-27T02:42:04.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.296+0000 7f5b7be76700 1 Processor -- start 2024-01-27T02:42:04.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.296+0000 7f5b7be76700 1 -- start start 2024-01-27T02:42:04.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.297+0000 7f5b7be76700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b74068af0 0x7f5b74099b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.297+0000 7f5b7be76700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b740cd770 0x7f5b7409ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.297+0000 7f5b7be76700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b740cdb40 0x7f5b7415e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.297+0000 7f5b7be76700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7409b0a0 con 0x7f5b740cdb40 2024-01-27T02:42:04.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.297+0000 7f5b7be76700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7415e780 con 0x7f5b740cd770 2024-01-27T02:42:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.297+0000 7f5b7be76700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b741294a0 con 0x7f5b74068af0 2024-01-27T02:42:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.297+0000 7f5b7a413700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b740cdb40 0x7f5b7415e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.297+0000 7f5b7a413700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b740cdb40 0x7f5b7415e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50476/0 (socket says 172.21.15.7:50476) 2024-01-27T02:42:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.297+0000 7f5b7a413700 1 -- 172.21.15.7:0/596573931 learned_addr learned my addr 172.21.15.7:0/596573931 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:42:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.297+0000 7f5b79c12700 1 --2- 172.21.15.7:0/596573931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b74068af0 0x7f5b74099b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.297+0000 7f5b79411700 1 --2- 172.21.15.7:0/596573931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b740cd770 0x7f5b7409ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.297+0000 7f5b7a413700 1 -- 172.21.15.7:0/596573931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b74068af0 msgr2=0x7f5b74099b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.297+0000 7f5b7a413700 1 --2- 172.21.15.7:0/596573931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b74068af0 0x7f5b74099b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.297+0000 7f5b7a413700 1 -- 172.21.15.7:0/596573931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b740cd770 msgr2=0x7f5b7409ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.297+0000 7f5b7a413700 1 --2- 172.21.15.7:0/596573931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b740cd770 0x7f5b7409ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.297+0000 7f5b7a413700 1 -- 172.21.15.7:0/596573931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b680094f0 con 0x7f5b740cdb40 2024-01-27T02:42:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.297+0000 7f5b79c12700 1 --2- 172.21.15.7:0/596573931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b74068af0 0x7f5b74099b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:42:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.297+0000 7f5b7a413700 1 --2- 172.21.15.7:0/596573931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b740cdb40 0x7f5b7415e240 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c007f50 tx=0x7f5b6c009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.298+0000 7f5b65ffb700 1 -- 172.21.15.7:0/596573931 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c00ecd0 con 0x7f5b740cdb40 2024-01-27T02:42:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.298+0000 7f5b65ffb700 1 -- 172.21.15.7:0/596573931 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6c00ee50 con 0x7f5b740cdb40 2024-01-27T02:42:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.298+0000 7f5b7be76700 1 -- 172.21.15.7:0/596573931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b74129720 con 0x7f5b740cdb40 2024-01-27T02:42:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.298+0000 7f5b65ffb700 1 -- 172.21.15.7:0/596573931 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c014670 con 0x7f5b740cdb40 2024-01-27T02:42:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.298+0000 7f5b7be76700 1 -- 172.21.15.7:0/596573931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b7409c680 con 0x7f5b740cdb40 2024-01-27T02:42:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.299+0000 7f5b65ffb700 1 -- 172.21.15.7:0/596573931 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5b6c0149e0 con 0x7f5b740cdb40 2024-01-27T02:42:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.300+0000 7f5b65ffb700 1 --2- 172.21.15.7:0/596573931 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5b58066f30 0x7f5b580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.300+0000 7f5b65ffb700 1 -- 172.21.15.7:0/596573931 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5b6c084d00 con 0x7f5b740cdb40 2024-01-27T02:42:04.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.300+0000 7f5b79c12700 1 --2- 172.21.15.7:0/596573931 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5b58066f30 0x7f5b580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.301+0000 7f5b79c12700 1 --2- 172.21.15.7:0/596573931 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5b58066f30 0x7f5b580693e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000ac10 tx=0x7f5b70006f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:04.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.302+0000 7f5b85179b80 1 Processor -- start 2024-01-27T02:42:04.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.302+0000 7f5b85179b80 1 -- start start 2024-01-27T02:42:04.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.303+0000 7f5b85179b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e5010 0x55e0938e53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.303+0000 7f5b85179b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e09392d360 0x55e09390f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.303+0000 7f5b85179b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0938e45d0 0x55e09390fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.303+0000 7f5b85179b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0937e89f0 con 0x55e0938e5010 2024-01-27T02:42:04.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.303+0000 7f5b85179b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09383a130 con 0x55e0938e45d0 2024-01-27T02:42:04.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.303+0000 7f5b85179b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e093837930 con 0x55e09392d360 2024-01-27T02:42:04.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.303+0000 7f5b79c12700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e5010 0x55e0938e53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.303+0000 7f5b79c12700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e5010 0x55e0938e53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50484/0 (socket says 172.21.15.7:50484) 2024-01-27T02:42:04.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.303+0000 7f5b79c12700 1 -- 172.21.15.7:0/3525237324 learned_addr learned my addr 172.21.15.7:0/3525237324 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:42:04.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.303+0000 7f5b79411700 1 --2- 172.21.15.7:0/3525237324 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e09392d360 0x55e09390f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.303+0000 7f5b7a413700 1 --2- 172.21.15.7:0/3525237324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0938e45d0 0x55e09390fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.303+0000 7f5b79c12700 1 -- 172.21.15.7:0/3525237324 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e09392d360 msgr2=0x55e09390f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:04.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.303+0000 7f5b79c12700 1 --2- 172.21.15.7:0/3525237324 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e09392d360 0x55e09390f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.303+0000 7f5b79c12700 1 -- 172.21.15.7:0/3525237324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0938e45d0 msgr2=0x55e09390fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:04.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.303+0000 7f5b79c12700 1 --2- 172.21.15.7:0/3525237324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0938e45d0 0x55e09390fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.303+0000 7f5b79c12700 1 -- 172.21.15.7:0/3525237324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e093672b10 con 0x55e0938e5010 2024-01-27T02:42:04.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.303+0000 7f5b79411700 1 --2- 172.21.15.7:0/3525237324 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e09392d360 0x55e09390f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:42:04.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.303+0000 7f5b79c12700 1 --2- 172.21.15.7:0/3525237324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e5010 0x55e0938e53e0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000fc10 tx=0x7f5b70016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d49793810509c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:04.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.304+0000 7f5b5effd700 1 -- 172.21.15.7:0/3525237324 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b700148f0 con 0x55e0938e5010 2024-01-27T02:42:04.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.304+0000 7f5b5effd700 1 -- 172.21.15.7:0/3525237324 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b70019070 con 0x55e0938e5010 2024-01-27T02:42:04.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.304+0000 7f5b5effd700 1 -- 172.21.15.7:0/3525237324 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b70014c60 con 0x55e0938e5010 2024-01-27T02:42:04.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.304+0000 7f5b85179b80 1 -- 172.21.15.7:0/3525237324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e5010 msgr2=0x55e0938e53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:04.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.304+0000 7f5b85179b80 1 --2- 172.21.15.7:0/3525237324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e5010 0x55e0938e53e0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000fc10 tx=0x7f5b70016000 comp rx=0 tx=0).stop 2024-01-27T02:42:04.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.304+0000 7f5b85179b80 1 -- 172.21.15.7:0/3525237324 shutdown_connections 2024-01-27T02:42:04.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.304+0000 7f5b85179b80 1 --2- 172.21.15.7:0/3525237324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0938e45d0 0x55e09390fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.304+0000 7f5b85179b80 1 --2- 172.21.15.7:0/3525237324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e5010 0x55e0938e53e0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.304+0000 7f5b85179b80 1 --2- 172.21.15.7:0/3525237324 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e09392d360 0x55e09390f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.304+0000 7f5b85179b80 1 -- 172.21.15.7:0/3525237324 >> 172.21.15.7:0/3525237324 conn(0x55e0938e4a70 msgr2=0x55e09392d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:04.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.304+0000 7f5b85179b80 1 -- 172.21.15.7:0/3525237324 shutdown_connections 2024-01-27T02:42:04.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.304+0000 7f5b85179b80 1 -- 172.21.15.7:0/3525237324 wait complete. 2024-01-27T02:42:04.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.305+0000 7f5b85179b80 1 -- start start 2024-01-27T02:42:04.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.305+0000 7f5b85179b80 1 Processor -- start 2024-01-27T02:42:04.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.306+0000 7f5b85179b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e45d0 0x55e09392c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.306+0000 7f5b85179b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0938e5010 0x55e0939259f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.306+0000 7f5b85179b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e09392d360 0x55e093925020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.306+0000 7f5b85179b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e093833720 con 0x55e0938e45d0 2024-01-27T02:42:04.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.306+0000 7f5b85179b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0937df1b0 con 0x55e0938e5010 2024-01-27T02:42:04.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.306+0000 7f5b85179b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0937dee50 con 0x55e09392d360 2024-01-27T02:42:04.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.306+0000 7f5b85179b80 10 client.0 _ll_register_callbacks cb 0x55e0936417b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-27T02:42:04.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.306+0000 7f5b79c12700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e45d0 0x55e09392c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.306+0000 7f5b79c12700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e45d0 0x55e09392c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50492/0 (socket says 172.21.15.7:50492) 2024-01-27T02:42:04.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.306+0000 7f5b79c12700 1 -- 172.21.15.7:0/937381414 learned_addr learned my addr 172.21.15.7:0/937381414 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:42:04.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.306+0000 7f5b7a413700 1 --2- 172.21.15.7:0/937381414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e09392d360 0x55e093925020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.306+0000 7f5b79411700 1 --2- 172.21.15.7:0/937381414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0938e5010 0x55e0939259f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.307+0000 7f5b79c12700 1 -- 172.21.15.7:0/937381414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e09392d360 msgr2=0x55e093925020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:04.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.307+0000 7f5b79c12700 1 --2- 172.21.15.7:0/937381414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e09392d360 0x55e093925020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.307+0000 7f5b79c12700 1 -- 172.21.15.7:0/937381414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0938e5010 msgr2=0x55e0939259f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:04.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.307+0000 7f5b79c12700 1 --2- 172.21.15.7:0/937381414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0938e5010 0x55e0939259f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.307+0000 7f5b79c12700 1 -- 172.21.15.7:0/937381414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b7000e040 con 0x55e0938e45d0 2024-01-27T02:42:04.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.307+0000 7f5b79c12700 1 --2- 172.21.15.7:0/937381414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e45d0 0x55e09392c500 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000b010 tx=0x7f5b70019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4f7af598e8909a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:04.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.307+0000 7f5b53fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-27T02:42:04.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.307+0000 7f5b53fff700 1 -- 172.21.15.7:0/937381414 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b70020930 con 0x55e0938e45d0 2024-01-27T02:42:04.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.307+0000 7f5b85179b80 1 -- 172.21.15.7:0/937381414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0938d8190 con 0x55e0938e45d0 2024-01-27T02:42:04.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.307+0000 7f5b53fff700 1 -- 172.21.15.7:0/937381414 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5b70020a90 con 0x55e0938e45d0 2024-01-27T02:42:04.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.307+0000 7f5b53fff700 1 -- 172.21.15.7:0/937381414 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b70020e60 con 0x55e0938e45d0 2024-01-27T02:42:04.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.307+0000 7f5b85179b80 10 client.7920 fetch_fsmap learned FSMap version 86 2024-01-27T02:42:04.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.307+0000 7f5b85179b80 1 -- 172.21.15.7:0/937381414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x55e093801290 con 0x55e0938e45d0 2024-01-27T02:42:04.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.307+0000 7f5b53fff700 1 -- 172.21.15.7:0/937381414 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e0938d8190 con 0x55e0938e45d0 2024-01-27T02:42:04.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.307+0000 7f5b53fff700 1 -- 172.21.15.7:0/937381414 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 86) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5b70032430 con 0x55e0938e45d0 2024-01-27T02:42:04.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.308+0000 7f5b85179b80 10 client.7920 fetch_fsmap finished waiting for FSMap version 86 2024-01-27T02:42:04.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.308+0000 7f5b85179b80 10 client.7920 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-27T02:42:04.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.308+0000 7f5b85179b80 20 client.7920 populate_metadata read hostname 'smithi007' 2024-01-27T02:42:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.308+0000 7f5b85179b80 1 --2- 172.21.15.7:0/937381414 >> [v2:172.21.15.7:6834/3213683076,v1:172.21.15.7:6835/3213683076] conn(0x55e093926170 0x55e093911860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.308+0000 7f5b85179b80 4 client.7920 mds_command: new command op to 6485 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-27T02:42:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.308+0000 7f5b85179b80 1 -- 172.21.15.7:0/937381414 --> [v2:172.21.15.7:6834/3213683076,v1:172.21.15.7:6835/3213683076] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0937d5ee0 con 0x55e093926170 2024-01-27T02:42:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.308+0000 7f5b79411700 1 --2- 172.21.15.7:0/937381414 >> [v2:172.21.15.7:6834/3213683076,v1:172.21.15.7:6835/3213683076] conn(0x55e093926170 0x55e093911860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.308+0000 7f5b53fff700 10 client.7920 ms_handle_connect on v2:172.21.15.7:6834/3213683076 2024-01-27T02:42:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.308+0000 7f5b79411700 1 --2- 172.21.15.7:0/937381414 >> [v2:172.21.15.7:6834/3213683076,v1:172.21.15.7:6835/3213683076] conn(0x55e093926170 0x55e093911860 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd29d3243525d757 server_cookie=6468648df1a36a49 in_seq=0 out_seq=0 2024-01-27T02:42:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.309+0000 7f5b53fff700 1 -- 172.21.15.7:0/937381414 <== mds.0 v2:172.21.15.7:6834/3213683076 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17146 (crc 0 0 0) 0x55e0937d5ee0 con 0x55e093926170 2024-01-27T02:42:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.309+0000 7f5b53fff700 10 client.7920 handle_command_reply: tid=0 2024-01-27T02:42:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.309+0000 7f5b85179b80 1 client.7920 shutdown 2024-01-27T02:42:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.309+0000 7f5b85179b80 1 -- 172.21.15.7:0/937381414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e45d0 msgr2=0x55e09392c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.309+0000 7f5b85179b80 1 --2- 172.21.15.7:0/937381414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e45d0 0x55e09392c500 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000b010 tx=0x7f5b70019000 comp rx=0 tx=0).stop 2024-01-27T02:42:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.309+0000 7f5b85179b80 1 -- 172.21.15.7:0/937381414 shutdown_connections 2024-01-27T02:42:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.309+0000 7f5b85179b80 1 --2- 172.21.15.7:0/937381414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e09392d360 0x55e093925020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.309+0000 7f5b85179b80 1 --2- 172.21.15.7:0/937381414 >> [v2:172.21.15.7:6834/3213683076,v1:172.21.15.7:6835/3213683076] conn(0x55e093926170 0x55e093911860 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.309+0000 7f5b85179b80 1 --2- 172.21.15.7:0/937381414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e45d0 0x55e09392c500 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.309+0000 7f5b85179b80 1 --2- 172.21.15.7:0/937381414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0938e5010 0x55e0939259f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.309+0000 7f5b53fff700 0 client.7920 ms_handle_reset on v2:172.21.15.7:6834/3213683076 2024-01-27T02:42:04.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.309+0000 7f5b85179b80 1 -- 172.21.15.7:0/937381414 >> 172.21.15.7:0/937381414 conn(0x55e0938e4a70 msgr2=0x55e0938dfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:04.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.309+0000 7f5b85179b80 1 -- 172.21.15.7:0/937381414 shutdown_connections 2024-01-27T02:42:04.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.309+0000 7f5b85179b80 1 -- 172.21.15.7:0/937381414 wait complete. 2024-01-27T02:42:04.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.309+0000 7f5b85179b80 20 client.7920 trim_cache size 0 max 16384 2024-01-27T02:42:04.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.327+0000 7f5b85179b80 1 Processor -- start 2024-01-27T02:42:04.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.328+0000 7f5b85179b80 1 -- start start 2024-01-27T02:42:04.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.328+0000 7f5b85179b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e45d0 0x55e0938defa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.328+0000 7f5b85179b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0938e5010 0x55e09390a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.328+0000 7f5b85179b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e09392d360 0x55e0939290a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.328+0000 7f5b85179b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09372be40 con 0x55e0938e45d0 2024-01-27T02:42:04.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.328+0000 7f5b85179b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09392d170 con 0x55e0938e5010 2024-01-27T02:42:04.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.328+0000 7f5b85179b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e09390c5b0 con 0x55e09392d360 2024-01-27T02:42:04.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.328+0000 7f5b79c12700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e45d0 0x55e0938defa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.328+0000 7f5b79c12700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e45d0 0x55e0938defa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50494/0 (socket says 172.21.15.7:50494) 2024-01-27T02:42:04.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.328+0000 7f5b79c12700 1 -- 172.21.15.7:0/23678150 learned_addr learned my addr 172.21.15.7:0/23678150 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:42:04.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.329+0000 7f5b7a413700 1 --2- 172.21.15.7:0/23678150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e09392d360 0x55e0939290a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.329+0000 7f5b79411700 1 --2- 172.21.15.7:0/23678150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0938e5010 0x55e09390a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.329+0000 7f5b79c12700 1 -- 172.21.15.7:0/23678150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e09392d360 msgr2=0x55e0939290a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:04.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.329+0000 7f5b79c12700 1 --2- 172.21.15.7:0/23678150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e09392d360 0x55e0939290a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.329+0000 7f5b79c12700 1 -- 172.21.15.7:0/23678150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0938e5010 msgr2=0x55e09390a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:04.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.329+0000 7f5b79c12700 1 --2- 172.21.15.7:0/23678150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0938e5010 0x55e09390a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.329+0000 7f5b79c12700 1 -- 172.21.15.7:0/23678150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0936cac80 con 0x55e0938e45d0 2024-01-27T02:42:04.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.329+0000 7f5b7a413700 1 --2- 172.21.15.7:0/23678150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e09392d360 0x55e0939290a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:42:04.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.329+0000 7f5b79c12700 1 --2- 172.21.15.7:0/23678150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e45d0 0x55e0938defa0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000fa50 tx=0x7f5b7000a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ffae09c9f92a12 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:04.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.329+0000 7f5b5f7fe700 1 -- 172.21.15.7:0/23678150 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7000a8d0 con 0x55e0938e45d0 2024-01-27T02:42:04.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.329+0000 7f5b5f7fe700 1 -- 172.21.15.7:0/23678150 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b700328f0 con 0x55e0938e45d0 2024-01-27T02:42:04.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.329+0000 7f5b5f7fe700 1 -- 172.21.15.7:0/23678150 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b70020930 con 0x55e0938e45d0 2024-01-27T02:42:04.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.330+0000 7f5b85179b80 1 -- 172.21.15.7:0/23678150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e45d0 msgr2=0x55e0938defa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:04.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.330+0000 7f5b85179b80 1 --2- 172.21.15.7:0/23678150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e45d0 0x55e0938defa0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000fa50 tx=0x7f5b7000a050 comp rx=0 tx=0).stop 2024-01-27T02:42:04.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.330+0000 7f5b85179b80 1 -- 172.21.15.7:0/23678150 shutdown_connections 2024-01-27T02:42:04.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.330+0000 7f5b85179b80 1 --2- 172.21.15.7:0/23678150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e09392d360 0x55e0939290a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.330+0000 7f5b85179b80 1 --2- 172.21.15.7:0/23678150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e45d0 0x55e0938defa0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.330+0000 7f5b85179b80 1 --2- 172.21.15.7:0/23678150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e0938e5010 0x55e09390a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.330+0000 7f5b85179b80 1 -- 172.21.15.7:0/23678150 >> 172.21.15.7:0/23678150 conn(0x55e0938e4a70 msgr2=0x55e0938de240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:04.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.330+0000 7f5b85179b80 1 -- 172.21.15.7:0/23678150 shutdown_connections 2024-01-27T02:42:04.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.330+0000 7f5b85179b80 1 -- 172.21.15.7:0/23678150 wait complete. 2024-01-27T02:42:04.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.330+0000 7f5b85179b80 1 -- start start 2024-01-27T02:42:04.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.331+0000 7f5b85179b80 1 Processor -- start 2024-01-27T02:42:04.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.331+0000 7f5b85179b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e45d0 0x55e093923df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.332+0000 7f5b85179b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e0938e5010 0x55e09398e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.332+0000 7f5b85179b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e09392d360 0x55e093984730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.332+0000 7f5b85179b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e093924770 con 0x55e0938e45d0 2024-01-27T02:42:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.332+0000 7f5b85179b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09398e160 con 0x55e09392d360 2024-01-27T02:42:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.332+0000 7f5b85179b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e09398e440 con 0x55e0938e5010 2024-01-27T02:42:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.332+0000 7f5b85179b80 10 client.0 _ll_register_callbacks cb 0x55e0936417b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-27T02:42:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.332+0000 7f5b79c12700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e45d0 0x55e093923df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.332+0000 7f5b79c12700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e45d0 0x55e093923df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50510/0 (socket says 172.21.15.7:50510) 2024-01-27T02:42:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.332+0000 7f5b79c12700 1 -- 172.21.15.7:0/3076000651 learned_addr learned my addr 172.21.15.7:0/3076000651 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:42:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.332+0000 7f5b7a413700 1 --2- 172.21.15.7:0/3076000651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e09392d360 0x55e093984730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.332+0000 7f5b79411700 1 --2- 172.21.15.7:0/3076000651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e0938e5010 0x55e09398e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.332+0000 7f5b79c12700 1 -- 172.21.15.7:0/3076000651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e0938e5010 msgr2=0x55e09398e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.332+0000 7f5b79c12700 1 --2- 172.21.15.7:0/3076000651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e0938e5010 0x55e09398e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.332+0000 7f5b79c12700 1 -- 172.21.15.7:0/3076000651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e09392d360 msgr2=0x55e093984730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.332+0000 7f5b79c12700 1 --2- 172.21.15.7:0/3076000651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e09392d360 0x55e093984730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.332+0000 7f5b79c12700 1 -- 172.21.15.7:0/3076000651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b70021e70 con 0x55e0938e45d0 2024-01-27T02:42:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.332+0000 7f5b79411700 1 --2- 172.21.15.7:0/3076000651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e0938e5010 0x55e09398e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:42:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.332+0000 7f5b79c12700 1 --2- 172.21.15.7:0/3076000651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e45d0 0x55e093923df0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f5b70032870 tx=0x7f5b70017070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1537a7ac1ce3d322 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.332+0000 7f5b53fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-27T02:42:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.332+0000 7f5b53fff700 1 -- 172.21.15.7:0/3076000651 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b70020af0 con 0x55e0938e45d0 2024-01-27T02:42:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.332+0000 7f5b85179b80 1 -- 172.21.15.7:0/3076000651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b70019070 con 0x55e0938e45d0 2024-01-27T02:42:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.332+0000 7f5b53fff700 1 -- 172.21.15.7:0/3076000651 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5b70046430 con 0x55e0938e45d0 2024-01-27T02:42:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.333+0000 7f5b53fff700 1 -- 172.21.15.7:0/3076000651 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b70046820 con 0x55e0938e45d0 2024-01-27T02:42:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.333+0000 7f5b85179b80 10 client.7926 fetch_fsmap learned FSMap version 86 2024-01-27T02:42:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.333+0000 7f5b85179b80 1 -- 172.21.15.7:0/3076000651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x55e093670fe0 con 0x55e0938e45d0 2024-01-27T02:42:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.333+0000 7f5b53fff700 1 -- 172.21.15.7:0/3076000651 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b70019070 con 0x55e0938e45d0 2024-01-27T02:42:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.333+0000 7f5b53fff700 1 -- 172.21.15.7:0/3076000651 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 86) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5b70046980 con 0x55e0938e45d0 2024-01-27T02:42:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.333+0000 7f5b85179b80 10 client.7926 fetch_fsmap finished waiting for FSMap version 86 2024-01-27T02:42:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.333+0000 7f5b85179b80 10 client.7926 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-27T02:42:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.333+0000 7f5b85179b80 20 client.7926 populate_metadata read hostname 'smithi007' 2024-01-27T02:42:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.333+0000 7f5b85179b80 1 --2- 172.21.15.7:0/3076000651 >> [v2:172.21.15.7:6834/3213683076,v1:172.21.15.7:6835/3213683076] conn(0x55e093926590 0x55e093926960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.333+0000 7f5b85179b80 4 client.7926 mds_command: new command op to 6485 tid=0[{"prefix": "status"}] 2024-01-27T02:42:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.333+0000 7f5b85179b80 1 -- 172.21.15.7:0/3076000651 --> [v2:172.21.15.7:6834/3213683076,v1:172.21.15.7:6835/3213683076] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e0937d5c00 con 0x55e093926590 2024-01-27T02:42:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.333+0000 7f5b79411700 1 --2- 172.21.15.7:0/3076000651 >> [v2:172.21.15.7:6834/3213683076,v1:172.21.15.7:6835/3213683076] conn(0x55e093926590 0x55e093926960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.333+0000 7f5b53fff700 10 client.7926 ms_handle_connect on v2:172.21.15.7:6834/3213683076 2024-01-27T02:42:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.333+0000 7f5b79411700 1 --2- 172.21.15.7:0/3076000651 >> [v2:172.21.15.7:6834/3213683076,v1:172.21.15.7:6835/3213683076] conn(0x55e093926590 0x55e093926960 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fa84ef567102e3e server_cookie=5f3834bcc93d5de7 in_seq=0 out_seq=0 2024-01-27T02:42:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.334+0000 7f5b53fff700 1 -- 172.21.15.7:0/3076000651 <== mds.0 v2:172.21.15.7:6834/3213683076 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55e0937d5c00 con 0x55e093926590 2024-01-27T02:42:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.334+0000 7f5b53fff700 10 client.7926 handle_command_reply: tid=0 2024-01-27T02:42:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.334+0000 7f5b85179b80 1 client.7926 shutdown 2024-01-27T02:42:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.334+0000 7f5b85179b80 1 -- 172.21.15.7:0/3076000651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e45d0 msgr2=0x55e093923df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.334+0000 7f5b85179b80 1 --2- 172.21.15.7:0/3076000651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e45d0 0x55e093923df0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f5b70032870 tx=0x7f5b70017070 comp rx=0 tx=0).stop 2024-01-27T02:42:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.334+0000 7f5b85179b80 1 -- 172.21.15.7:0/3076000651 shutdown_connections 2024-01-27T02:42:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.334+0000 7f5b85179b80 1 --2- 172.21.15.7:0/3076000651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e09392d360 0x55e093984730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.334+0000 7f5b85179b80 1 --2- 172.21.15.7:0/3076000651 >> [v2:172.21.15.7:6834/3213683076,v1:172.21.15.7:6835/3213683076] conn(0x55e093926590 0x55e093926960 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.334+0000 7f5b85179b80 1 --2- 172.21.15.7:0/3076000651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e0938e45d0 0x55e093923df0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.334+0000 7f5b85179b80 1 --2- 172.21.15.7:0/3076000651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e0938e5010 0x55e09398e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.334+0000 7f5b53fff700 0 client.7926 ms_handle_reset on v2:172.21.15.7:6834/3213683076 2024-01-27T02:42:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.334+0000 7f5b85179b80 1 -- 172.21.15.7:0/3076000651 >> 172.21.15.7:0/3076000651 conn(0x55e0938e4a70 msgr2=0x55e0939290a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.334+0000 7f5b85179b80 1 -- 172.21.15.7:0/3076000651 shutdown_connections 2024-01-27T02:42:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.334+0000 7f5b85179b80 1 -- 172.21.15.7:0/3076000651 wait complete. 2024-01-27T02:42:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.334+0000 7f5b85179b80 20 client.7926 trim_cache size 0 max 16384 2024-01-27T02:42:04.344 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:42:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:42:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-27T02:42:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 6485, 2024-01-27T02:42:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-27T02:42:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-27T02:42:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-27T02:42:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.1177142440000001, 2024-01-27T02:42:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 86, 2024-01-27T02:42:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 164, 2024-01-27T02:42:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 164, 2024-01-27T02:42:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 11.093591197 2024-01-27T02:42:04.346 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:42:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.335+0000 7f5b7be76700 1 -- 172.21.15.7:0/596573931 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5b58066f30 msgr2=0x7f5b580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.335+0000 7f5b7be76700 1 --2- 172.21.15.7:0/596573931 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5b58066f30 0x7f5b580693e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000ac10 tx=0x7f5b70006f90 comp rx=0 tx=0).stop 2024-01-27T02:42:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.335+0000 7f5b7be76700 1 -- 172.21.15.7:0/596573931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b740cdb40 msgr2=0x7f5b7415e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.335+0000 7f5b7be76700 1 --2- 172.21.15.7:0/596573931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b740cdb40 0x7f5b7415e240 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c007f50 tx=0x7f5b6c009860 comp rx=0 tx=0).stop 2024-01-27T02:42:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.335+0000 7f5b7be76700 1 -- 172.21.15.7:0/596573931 shutdown_connections 2024-01-27T02:42:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.335+0000 7f5b7be76700 1 --2- 172.21.15.7:0/596573931 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5b58066f30 0x7f5b580693e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.335+0000 7f5b7be76700 1 --2- 172.21.15.7:0/596573931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b740cdb40 0x7f5b7415e240 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.335+0000 7f5b7be76700 1 --2- 172.21.15.7:0/596573931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b74068af0 0x7f5b74099b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.335+0000 7f5b7be76700 1 --2- 172.21.15.7:0/596573931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b740cd770 0x7f5b7409ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.335+0000 7f5b7be76700 1 -- 172.21.15.7:0/596573931 >> 172.21.15.7:0/596573931 conn(0x7f5b74063d60 msgr2=0x7f5b7406c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.335+0000 7f5b7be76700 1 -- 172.21.15.7:0/596573931 shutdown_connections 2024-01-27T02:42:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.335+0000 7f5b7be76700 1 -- 172.21.15.7:0/596573931 wait complete. 2024-01-27T02:42:04.350 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:42:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.479+0000 7f966508b700 1 Processor -- start 2024-01-27T02:42:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.479+0000 7f966508b700 1 -- start start 2024-01-27T02:42:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.480+0000 7f966508b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96600b1c00 0x7f966006e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.480+0000 7f966508b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f966015b5f0 0x7f96601566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.480+0000 7f966508b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9660157610 con 0x7f96600b1fd0 2024-01-27T02:42:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.480+0000 7f966508b700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9660157310 con 0x7f966015b5f0 2024-01-27T02:42:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.480+0000 7f966508b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f966006ee00 con 0x7f96600b1c00 2024-01-27T02:42:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.480+0000 7f965f59e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f96600b1fd0 0x7f9660156c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34954/0 (socket says 172.21.15.7:34954) 2024-01-27T02:42:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.480+0000 7f965f59e700 1 -- 172.21.15.7:0/2436432535 learned_addr learned my addr 172.21.15.7:0/2436432535 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:42:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.480+0000 7f965e59c700 1 --2- 172.21.15.7:0/2436432535 >> v2:172.21.15.188:3300/0 conn(0x7f966015b5f0 0x7f96601566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.480+0000 7f965ed9d700 1 --2- 172.21.15.7:0/2436432535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96600b1c00 0x7f966006e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.480+0000 7f965dd9b700 1 -- 172.21.15.7:0/2436432535 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4007008218 0 0) 0x7f9660157610 con 0x7f96600b1fd0 2024-01-27T02:42:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.480+0000 7f965dd9b700 1 -- 172.21.15.7:0/2436432535 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9648003570 con 0x7f96600b1fd0 2024-01-27T02:42:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.480+0000 7f965ed9d700 1 -- 172.21.15.7:0/2436432535 >> v2:172.21.15.188:3300/0 conn(0x7f966015b5f0 msgr2=0x7f96601566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.480+0000 7f965ed9d700 1 --2- 172.21.15.7:0/2436432535 >> v2:172.21.15.188:3300/0 conn(0x7f966015b5f0 0x7f96601566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.480+0000 7f965ed9d700 1 -- 172.21.15.7:0/2436432535 >> v1:172.21.15.7:6789/0 conn(0x7f96600b1fd0 legacy=0x7f9660156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.480+0000 7f965ed9d700 1 -- 172.21.15.7:0/2436432535 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f966006f0e0 con 0x7f96600b1c00 2024-01-27T02:42:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.480+0000 7f965e59c700 1 --2- 172.21.15.7:0/2436432535 >> v2:172.21.15.188:3300/0 conn(0x7f966015b5f0 0x7f96601566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:42:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.481+0000 7f965ed9d700 1 --2- 172.21.15.7:0/2436432535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96600b1c00 0x7f966006e540 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f96500098e0 tx=0x7f96500091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f02aa1e5bd2f0553 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.481+0000 7f965dd9b700 1 -- 172.21.15.7:0/2436432535 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9650011630 con 0x7f96600b1c00 2024-01-27T02:42:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.481+0000 7f965dd9b700 1 -- 172.21.15.7:0/2436432535 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9650011790 con 0x7f96600b1c00 2024-01-27T02:42:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.481+0000 7f965dd9b700 1 -- 172.21.15.7:0/2436432535 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9650011b30 con 0x7f96600b1c00 2024-01-27T02:42:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.481+0000 7f966508b700 1 -- 172.21.15.7:0/2436432535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96600b1c00 msgr2=0x7f966006e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.481+0000 7f966508b700 1 --2- 172.21.15.7:0/2436432535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96600b1c00 0x7f966006e540 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f96500098e0 tx=0x7f96500091a0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.481+0000 7f966508b700 1 -- 172.21.15.7:0/2436432535 shutdown_connections 2024-01-27T02:42:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.481+0000 7f966508b700 1 --2- 172.21.15.7:0/2436432535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96600b1c00 0x7f966006e540 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.481+0000 7f966508b700 1 --2- 172.21.15.7:0/2436432535 >> v2:172.21.15.188:3300/0 conn(0x7f966015b5f0 0x7f96601566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.481+0000 7f966508b700 1 -- 172.21.15.7:0/2436432535 >> 172.21.15.7:0/2436432535 conn(0x7f9660068430 msgr2=0x7f966006be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.481+0000 7f966508b700 1 -- 172.21.15.7:0/2436432535 shutdown_connections 2024-01-27T02:42:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.481+0000 7f966508b700 1 -- 172.21.15.7:0/2436432535 wait complete. 2024-01-27T02:42:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.483+0000 7f966508b700 1 Processor -- start 2024-01-27T02:42:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.483+0000 7f966508b700 1 -- start start 2024-01-27T02:42:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.484+0000 7f966508b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96600b1c00 0x7f966016dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.484+0000 7f966508b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96600b1fd0 0x7f966016e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.484+0000 7f966508b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f966015b5f0 0x7f96601516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.484+0000 7f966508b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9660153c40 con 0x7f966015b5f0 2024-01-27T02:42:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.484+0000 7f966508b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9660152f00 con 0x7f96600b1fd0 2024-01-27T02:42:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.484+0000 7f966508b700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96601522f0 con 0x7f96600b1c00 2024-01-27T02:42:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.484+0000 7f965f59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f966015b5f0 0x7f96601516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.484+0000 7f965f59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f966015b5f0 0x7f96601516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50526/0 (socket says 172.21.15.7:50526) 2024-01-27T02:42:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.484+0000 7f965f59e700 1 -- 172.21.15.7:0/651714112 learned_addr learned my addr 172.21.15.7:0/651714112 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:42:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.484+0000 7f965ed9d700 1 --2- 172.21.15.7:0/651714112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96600b1c00 0x7f966016dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.484+0000 7f965e59c700 1 --2- 172.21.15.7:0/651714112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96600b1fd0 0x7f966016e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.484+0000 7f965f59e700 1 -- 172.21.15.7:0/651714112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96600b1c00 msgr2=0x7f966016dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.484+0000 7f965f59e700 1 --2- 172.21.15.7:0/651714112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96600b1c00 0x7f966016dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.484+0000 7f965f59e700 1 -- 172.21.15.7:0/651714112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96600b1fd0 msgr2=0x7f966016e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.484+0000 7f965f59e700 1 --2- 172.21.15.7:0/651714112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96600b1fd0 0x7f966016e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.484+0000 7f965f59e700 1 -- 172.21.15.7:0/651714112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9654009420 con 0x7f966015b5f0 2024-01-27T02:42:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.484+0000 7f965e59c700 1 --2- 172.21.15.7:0/651714112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96600b1fd0 0x7f966016e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:42:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.484+0000 7f965f59e700 1 --2- 172.21.15.7:0/651714112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f966015b5f0 0x7f96601516e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f9658001460 tx=0x7f9658001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.485+0000 7f9646ffd700 1 -- 172.21.15.7:0/651714112 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965800a940 con 0x7f966015b5f0 2024-01-27T02:42:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.485+0000 7f9646ffd700 1 -- 172.21.15.7:0/651714112 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965800aaa0 con 0x7f966015b5f0 2024-01-27T02:42:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.485+0000 7f9646ffd700 1 -- 172.21.15.7:0/651714112 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965800ae20 con 0x7f966015b5f0 2024-01-27T02:42:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.485+0000 7f966508b700 1 -- 172.21.15.7:0/651714112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9650009420 con 0x7f966015b5f0 2024-01-27T02:42:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.485+0000 7f966508b700 1 -- 172.21.15.7:0/651714112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96601528b0 con 0x7f966015b5f0 2024-01-27T02:42:04.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.486+0000 7f966508b700 1 -- 172.21.15.7:0/651714112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9660153070 con 0x7f966015b5f0 2024-01-27T02:42:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.486+0000 7f9646ffd700 1 -- 172.21.15.7:0/651714112 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9658007300 con 0x7f966015b5f0 2024-01-27T02:42:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.487+0000 7f9646ffd700 1 --2- 172.21.15.7:0/651714112 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f964c066f30 0x7f964c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.487+0000 7f9646ffd700 1 -- 172.21.15.7:0/651714112 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f96580854a0 con 0x7f966015b5f0 2024-01-27T02:42:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.487+0000 7f965ed9d700 1 --2- 172.21.15.7:0/651714112 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f964c066f30 0x7f964c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.488+0000 7f965ed9d700 1 --2- 172.21.15.7:0/651714112 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f964c066f30 0x7f964c0693e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f96500091e0 tx=0x7f9650017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.491+0000 7f9646ffd700 1 -- 172.21.15.7:0/651714112 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96580527f0 con 0x7f966015b5f0 2024-01-27T02:42:04.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.742+0000 7f966508b700 1 -- 172.21.15.7:0/651714112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f966004c710 con 0x7f966015b5f0 2024-01-27T02:42:04.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.742+0000 7f9646ffd700 1 -- 172.21.15.7:0/651714112 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9658055e10 con 0x7f966015b5f0 2024-01-27T02:42:04.744 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:42:04.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.744+0000 7f966508b700 1 -- 172.21.15.7:0/651714112 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f964c066f30 msgr2=0x7f964c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:04.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.744+0000 7f966508b700 1 --2- 172.21.15.7:0/651714112 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f964c066f30 0x7f964c0693e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f96500091e0 tx=0x7f9650017000 comp rx=0 tx=0).stop 2024-01-27T02:42:04.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.744+0000 7f966508b700 1 -- 172.21.15.7:0/651714112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f966015b5f0 msgr2=0x7f96601516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:04.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.744+0000 7f966508b700 1 --2- 172.21.15.7:0/651714112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f966015b5f0 0x7f96601516e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f9658001460 tx=0x7f9658001ae0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.744+0000 7f966508b700 1 -- 172.21.15.7:0/651714112 shutdown_connections 2024-01-27T02:42:04.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.744+0000 7f966508b700 1 --2- 172.21.15.7:0/651714112 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f964c066f30 0x7f964c0693e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.744+0000 7f966508b700 1 --2- 172.21.15.7:0/651714112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f966015b5f0 0x7f96601516e0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.744+0000 7f966508b700 1 --2- 172.21.15.7:0/651714112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96600b1c00 0x7f966016dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.744+0000 7f966508b700 1 --2- 172.21.15.7:0/651714112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96600b1fd0 0x7f966016e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:04.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.744+0000 7f966508b700 1 -- 172.21.15.7:0/651714112 >> 172.21.15.7:0/651714112 conn(0x7f9660068430 msgr2=0x7f9660155240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:04.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.744+0000 7f966508b700 1 -- 172.21.15.7:0/651714112 shutdown_connections 2024-01-27T02:42:04.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:04.744+0000 7f966508b700 1 -- 172.21.15.7:0/651714112 wait complete. 2024-01-27T02:42:04.759 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-27T02:42:04.759 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:42:04.759 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-27T02:42:04.759 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-27T02:42:04.760 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-27T02:42:04.760 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-27T02:42:04.760 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-27T02:42:04.760 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:42:04.760 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:42:04.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:42:04.760 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-27T02:42:04.780 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:42:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:42:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:42:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:42:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-27T02:42:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:42:04.780 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:42:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-27T02:42:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-27T02:42:04.780 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:42:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-27T02:42:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-27T02:42:04.781 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:42:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-27T02:42:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-27T02:42:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-27T02:42:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 41786sec preferred_lft 41786sec 2024-01-27T02:42:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-27T02:42:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:42:04.781 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:42:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-27T02:42:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-27T02:42:04.781 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:42:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 4a:57:d9:3c:c5:a0 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:42:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-27T02:42:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:42:04.782 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::2477:9eff:fe39:433c/64 scope link 2024-01-27T02:42:04.782 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:42:04.782 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:42:04.782 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 4a:57:d9:3c:c5:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-27T02:42:04.782 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::4857:d9ff:fe3c:c5a0/64 scope link 2024-01-27T02:42:04.782 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:42:04.782 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:42:04.782 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ca:6e:b8:df:53:bf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-27T02:42:04.782 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c86e:b8ff:fedf:53bf/64 scope link 2024-01-27T02:42:04.782 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:42:04.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:42:04.783 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:42:04.851 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-27T02:42:04.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:42:04.853 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-27T02:42:04.918 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-27T02:42:04.918 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:42:04.918 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-27T02:42:04.918 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-27T02:42:04.918 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:42:04.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:42:04.919 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-27T02:42:04.919 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-27T02:42:04.919 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-27T02:42:04.919 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-27T02:42:04.919 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-27T02:42:04.919 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-27T02:42:05.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:42:05.041 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-27T02:42:05.061 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-27T02:42:05.061 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-27T02:42:05.062 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:42:05.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:42:05.062 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-27T02:42:05.138 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:42:05.138 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-27T02:42:05.202 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:42:05.202 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:42:05.202 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-27T02:42:05.264 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:42:05.264 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:42:05.264 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:42:05.358 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:42:05.359 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:42:05.359 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:42:05.369 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:42:05.369 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:42:05.369 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:42:05.369 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:42:05.369 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:42:05.388 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:42:05.389 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:42:05.389 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:42:05.389 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:42:05.389 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:42:05.389 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:42:05.389 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:42:05.389 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:42:05.390 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:42:05.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:42:05.390 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-27T02:42:05.423 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:42:05.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.601+0000 7effb7c11700 1 Processor -- start 2024-01-27T02:42:05.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.602+0000 7effb7c11700 1 -- start start 2024-01-27T02:42:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.603+0000 7effb7c11700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7effb00cad20 0x7effb00cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.603+0000 7effb7c11700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7effb00cc8e0 0x7effb00ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.603+0000 7effb7c11700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb00cd280 con 0x7effb015b740 2024-01-27T02:42:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.603+0000 7effb7c11700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb00c64d0 con 0x7effb00cad20 2024-01-27T02:42:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.603+0000 7effb7c11700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb00c4a30 con 0x7effb00cc8e0 2024-01-27T02:42:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.603+0000 7effb51ac700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7effb00cad20 0x7effb00cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.603+0000 7effb59ad700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7effb015b740 0x7effb00c5dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34958/0 (socket says 172.21.15.7:34958) 2024-01-27T02:42:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.603+0000 7effb59ad700 1 -- 172.21.15.7:0/3374391101 learned_addr learned my addr 172.21.15.7:0/3374391101 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:42:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.603+0000 7effb61ae700 1 --2- 172.21.15.7:0/3374391101 >> v2:172.21.15.188:3300/0 conn(0x7effb00cc8e0 0x7effb00ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.603+0000 7effb51ac700 1 -- 172.21.15.7:0/3374391101 >> v2:172.21.15.188:3300/0 conn(0x7effb00cc8e0 msgr2=0x7effb00ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.603+0000 7effb51ac700 1 --2- 172.21.15.7:0/3374391101 >> v2:172.21.15.188:3300/0 conn(0x7effb00cc8e0 0x7effb00ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.603+0000 7effb51ac700 1 -- 172.21.15.7:0/3374391101 >> v1:172.21.15.7:6789/0 conn(0x7effb015b740 legacy=0x7effb00c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.603+0000 7effb51ac700 1 -- 172.21.15.7:0/3374391101 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb015bb10 con 0x7effb00cad20 2024-01-27T02:42:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.603+0000 7effb49ab700 1 -- 172.21.15.7:0/3374391101 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2840548568 0 0) 0x7effac001110 con 0x7effb015b740 2024-01-27T02:42:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.603+0000 7effb61ae700 1 --2- 172.21.15.7:0/3374391101 >> v2:172.21.15.188:3300/0 conn(0x7effb00cc8e0 0x7effb00ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:42:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.603+0000 7effb51ac700 1 --2- 172.21.15.7:0/3374391101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effb00cad20 0x7effb00cb1c0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7effa00098e0 tx=0x7effa00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45eed870efcdb1ff server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.604+0000 7effb49ab700 1 -- 172.21.15.7:0/3374391101 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa0009e80 con 0x7effb00cad20 2024-01-27T02:42:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.604+0000 7effb49ab700 1 -- 172.21.15.7:0/3374391101 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa000b820 con 0x7effb00cad20 2024-01-27T02:42:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.604+0000 7effb49ab700 1 -- 172.21.15.7:0/3374391101 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa000bba0 con 0x7effb00cad20 2024-01-27T02:42:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.604+0000 7effb7c11700 1 -- 172.21.15.7:0/3374391101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effb00cad20 msgr2=0x7effb00cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.604+0000 7effb7c11700 1 --2- 172.21.15.7:0/3374391101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effb00cad20 0x7effb00cb1c0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7effa00098e0 tx=0x7effa00091a0 comp rx=0 tx=0).stop 2024-01-27T02:42:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.604+0000 7effb7c11700 1 -- 172.21.15.7:0/3374391101 shutdown_connections 2024-01-27T02:42:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.604+0000 7effb7c11700 1 --2- 172.21.15.7:0/3374391101 >> v2:172.21.15.188:3300/0 conn(0x7effb00cc8e0 0x7effb00ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.604+0000 7effb7c11700 1 --2- 172.21.15.7:0/3374391101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effb00cad20 0x7effb00cb1c0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.604+0000 7effb7c11700 1 -- 172.21.15.7:0/3374391101 >> 172.21.15.7:0/3374391101 conn(0x7effb0063d60 msgr2=0x7effb00ced10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.604+0000 7effb7c11700 1 -- 172.21.15.7:0/3374391101 shutdown_connections 2024-01-27T02:42:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.604+0000 7effb7c11700 1 -- 172.21.15.7:0/3374391101 wait complete. 2024-01-27T02:42:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.605+0000 7effb7c11700 1 Processor -- start 2024-01-27T02:42:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.606+0000 7effb7c11700 1 -- start start 2024-01-27T02:42:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.606+0000 7effb7c11700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effb00cad20 0x7effb01579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.606+0000 7effb7c11700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effb00cc8e0 0x7effb00b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.606+0000 7effb7c11700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effb015b740 0x7effb00682e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.606+0000 7effb7c11700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb00b49e0 con 0x7effb00cad20 2024-01-27T02:42:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.606+0000 7effb7c11700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb00b1450 con 0x7effb00cc8e0 2024-01-27T02:42:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.606+0000 7effb7c11700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb0157ee0 con 0x7effb015b740 2024-01-27T02:42:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.606+0000 7effb59ad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effb00cad20 0x7effb01579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.607+0000 7effb59ad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effb00cad20 0x7effb01579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50540/0 (socket says 172.21.15.7:50540) 2024-01-27T02:42:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.607+0000 7effb59ad700 1 -- 172.21.15.7:0/4225256709 learned_addr learned my addr 172.21.15.7:0/4225256709 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:42:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.607+0000 7effb51ac700 1 --2- 172.21.15.7:0/4225256709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effb00cc8e0 0x7effb00b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.607+0000 7effb61ae700 1 --2- 172.21.15.7:0/4225256709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effb015b740 0x7effb00682e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.607+0000 7effb59ad700 1 -- 172.21.15.7:0/4225256709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effb015b740 msgr2=0x7effb00682e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.607+0000 7effb59ad700 1 --2- 172.21.15.7:0/4225256709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effb015b740 0x7effb00682e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.607+0000 7effb59ad700 1 -- 172.21.15.7:0/4225256709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effb00cc8e0 msgr2=0x7effb00b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.607+0000 7effb59ad700 1 --2- 172.21.15.7:0/4225256709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effb00cc8e0 0x7effb00b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.607+0000 7effb59ad700 1 -- 172.21.15.7:0/4225256709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa0009420 con 0x7effb00cad20 2024-01-27T02:42:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.607+0000 7effb51ac700 1 --2- 172.21.15.7:0/4225256709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effb00cc8e0 0x7effb00b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:42:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.607+0000 7effb59ad700 1 --2- 172.21.15.7:0/4225256709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effb00cad20 0x7effb01579a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7effac00c5b0 tx=0x7effac001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.607+0000 7effa5ffb700 1 -- 172.21.15.7:0/4225256709 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac005b30 con 0x7effb00cad20 2024-01-27T02:42:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.607+0000 7effa5ffb700 1 -- 172.21.15.7:0/4225256709 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effac005c90 con 0x7effb00cad20 2024-01-27T02:42:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.607+0000 7effb7c11700 1 -- 172.21.15.7:0/4225256709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb00b0840 con 0x7effb00cad20 2024-01-27T02:42:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.607+0000 7effa5ffb700 1 -- 172.21.15.7:0/4225256709 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac007300 con 0x7effb00cad20 2024-01-27T02:42:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.607+0000 7effb7c11700 1 -- 172.21.15.7:0/4225256709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb00b0e00 con 0x7effb00cad20 2024-01-27T02:42:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.608+0000 7effb7c11700 1 -- 172.21.15.7:0/4225256709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb004c710 con 0x7effb00cad20 2024-01-27T02:42:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.609+0000 7effa5ffb700 1 -- 172.21.15.7:0/4225256709 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7effac01b3b0 con 0x7effb00cad20 2024-01-27T02:42:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.609+0000 7effa5ffb700 1 --2- 172.21.15.7:0/4225256709 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7eff94066f30 0x7eff940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.609+0000 7effa5ffb700 1 -- 172.21.15.7:0/4225256709 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7effac0839b0 con 0x7effb00cad20 2024-01-27T02:42:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.610+0000 7effb51ac700 1 --2- 172.21.15.7:0/4225256709 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7eff94066f30 0x7eff940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:05.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.610+0000 7effb51ac700 1 --2- 172.21.15.7:0/4225256709 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7eff94066f30 0x7eff940693e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7effa00093f0 tx=0x7effa0017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:05.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.614+0000 7effa5ffb700 1 -- 172.21.15.7:0/4225256709 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7effac050b80 con 0x7effb00cad20 2024-01-27T02:42:05.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.865+0000 7effb7c11700 1 -- 172.21.15.7:0/4225256709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7effb00cd280 con 0x7effb00cad20 2024-01-27T02:42:05.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.866+0000 7effa5ffb700 1 -- 172.21.15.7:0/4225256709 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) v1 ==== 53+0+73 (secure 0 0 0) 0x7effb00cd280 con 0x7effb00cad20 2024-01-27T02:42:05.867 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:42:05.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.868+0000 7effb7c11700 1 -- 172.21.15.7:0/4225256709 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7eff94066f30 msgr2=0x7eff940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:05.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.868+0000 7effb7c11700 1 --2- 172.21.15.7:0/4225256709 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7eff94066f30 0x7eff940693e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7effa00093f0 tx=0x7effa0017000 comp rx=0 tx=0).stop 2024-01-27T02:42:05.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.868+0000 7effb7c11700 1 -- 172.21.15.7:0/4225256709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effb00cad20 msgr2=0x7effb01579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:05.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.868+0000 7effb7c11700 1 --2- 172.21.15.7:0/4225256709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effb00cad20 0x7effb01579a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7effac00c5b0 tx=0x7effac001070 comp rx=0 tx=0).stop 2024-01-27T02:42:05.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.868+0000 7effb7c11700 1 -- 172.21.15.7:0/4225256709 shutdown_connections 2024-01-27T02:42:05.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.868+0000 7effb7c11700 1 --2- 172.21.15.7:0/4225256709 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7eff94066f30 0x7eff940693e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:05.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.868+0000 7effb7c11700 1 --2- 172.21.15.7:0/4225256709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effb015b740 0x7effb00682e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:05.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.868+0000 7effb7c11700 1 --2- 172.21.15.7:0/4225256709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effb00cad20 0x7effb01579a0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:05.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.868+0000 7effb7c11700 1 --2- 172.21.15.7:0/4225256709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effb00cc8e0 0x7effb00b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:05.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.868+0000 7effb7c11700 1 -- 172.21.15.7:0/4225256709 >> 172.21.15.7:0/4225256709 conn(0x7effb0063d60 msgr2=0x7effb01587f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:05.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.868+0000 7effb7c11700 1 -- 172.21.15.7:0/4225256709 shutdown_connections 2024-01-27T02:42:05.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:05.868+0000 7effb7c11700 1 -- 172.21.15.7:0/4225256709 wait complete. 2024-01-27T02:42:05.883 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-27T02:42:05.884 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:42:05.884 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-01-27T02:42:05.884 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-27T02:42:05.884 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-27T02:42:05.884 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-27T02:42:05.884 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-27T02:42:05.884 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:42:05.884 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:42:05.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:42:05.885 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-01-27T02:42:05.907 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:42:05.908 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:42:05.908 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:42:05.908 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:42:05.908 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-01-27T02:42:05.908 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:42:05.908 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:42:05.909 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:42:05.909 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-01-27T02:42:05.909 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:42:05.909 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:42:05.909 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-01-27T02:42:05.909 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:42:05.909 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:42:05.909 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:42:05.909 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:42:05.909 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-27T02:42:05.909 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 41793sec preferred_lft 41793sec 2024-01-27T02:42:05.909 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b1:d6f7:baf0:bc17/64 scope link noprefixroute 2024-01-27T02:42:05.909 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:42:05.910 INFO:teuthology.orchestra.run.smithi188.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:42:05.910 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b0:92:71:90:9d brd ff:ff:ff:ff:ff:ff 2024-01-27T02:42:05.910 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-27T02:42:05.910 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:42:05.910 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::10f8:32ff:feed:a752/64 scope link 2024-01-27T02:42:05.910 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:42:05.910 INFO:teuthology.orchestra.run.smithi188.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:42:05.910 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b0:92:71:90:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-27T02:42:05.910 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b8b0:92ff:fe71:909d/64 scope link 2024-01-27T02:42:05.910 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:42:05.910 INFO:teuthology.orchestra.run.smithi188.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:42:05.910 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ee:78:03:20:3a:a9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-27T02:42:05.910 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec78:3ff:fe20:3aa9/64 scope link 2024-01-27T02:42:05.911 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:42:05.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:42:05.911 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:42:05.979 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-01-27T02:42:05.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:42:05.981 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-01-27T02:42:06.042 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-01-27T02:42:06.042 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:42:06.042 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-27T02:42:06.042 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-27T02:42:06.043 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:42:06.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:42:06.043 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-27T02:42:06.043 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-27T02:42:06.043 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-27T02:42:06.043 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-27T02:42:06.043 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-27T02:42:06.043 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-27T02:42:06.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:42:06.153 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-01-27T02:42:06.171 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-27T02:42:06.171 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-27T02:42:06.171 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:42:06.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:42:06.172 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2024-01-27T02:42:06.240 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:42:06.241 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-27T02:42:06.298 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:42:06.298 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:42:06.298 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-27T02:42:06.354 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:42:06.355 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:42:06.355 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:42:06.438 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:42:06.438 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:42:06.438 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:42:06.447 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:42:06.447 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:42:06.447 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:42:06.447 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:42:06.447 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:42:06.464 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:42:06.465 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:42:06.465 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:42:06.465 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:42:06.465 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:42:06.465 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:42:06.465 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:42:06.465 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:42:06.466 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:42:06.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:42:06.467 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-27T02:42:06.534 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:42:06.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.673+0000 7f2425718700 1 Processor -- start 2024-01-27T02:42:06.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.673+0000 7f2425718700 1 -- start start 2024-01-27T02:42:06.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.673+0000 7f2425718700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f24200d10a0 0x7f24200b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:06.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.673+0000 7f2425718700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f24200d1470 0x7f24200b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:06.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.673+0000 7f2425718700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2420159b40 0x7f2420159f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:06.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.673+0000 7f2425718700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24200b4650 con 0x7f24200d1470 2024-01-27T02:42:06.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.673+0000 7f2425718700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24200b3d60 con 0x7f24200d10a0 2024-01-27T02:42:06.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.673+0000 7f2425718700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24200b4000 con 0x7f2420159b40 2024-01-27T02:42:06.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.674+0000 7f241e7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f24200d1470 0x7f24200b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:06.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.674+0000 7f241e7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f24200d1470 0x7f24200b2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50550/0 (socket says 172.21.15.7:50550) 2024-01-27T02:42:06.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.674+0000 7f241e7fc700 1 -- 172.21.15.7:0/3770548039 learned_addr learned my addr 172.21.15.7:0/3770548039 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:42:06.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.674+0000 7f241effd700 1 --2- 172.21.15.7:0/3770548039 >> v2:172.21.15.188:3300/0 conn(0x7f24200d10a0 0x7f24200b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:06.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.674+0000 7f241f7fe700 1 --2- 172.21.15.7:0/3770548039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2420159b40 0x7f2420159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:06.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.674+0000 7f241e7fc700 1 -- 172.21.15.7:0/3770548039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2420159b40 msgr2=0x7f2420159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:06.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.674+0000 7f241e7fc700 1 --2- 172.21.15.7:0/3770548039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2420159b40 0x7f2420159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:06.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.674+0000 7f241e7fc700 1 -- 172.21.15.7:0/3770548039 >> v2:172.21.15.188:3300/0 conn(0x7f24200d10a0 msgr2=0x7f24200b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:06.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.674+0000 7f241e7fc700 1 --2- 172.21.15.7:0/3770548039 >> v2:172.21.15.188:3300/0 conn(0x7f24200d10a0 0x7f24200b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:06.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.674+0000 7f241e7fc700 1 -- 172.21.15.7:0/3770548039 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24200b42e0 con 0x7f24200d1470 2024-01-27T02:42:06.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.674+0000 7f241e7fc700 1 --2- 172.21.15.7:0/3770548039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24200d1470 0x7f24200b2dc0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f2420069ab0 tx=0x7f2414009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b501bef5373b9a53 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:06.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.674+0000 7f241dffb700 1 -- 172.21.15.7:0/3770548039 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241400b860 con 0x7f24200d1470 2024-01-27T02:42:06.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.674+0000 7f241dffb700 1 -- 172.21.15.7:0/3770548039 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241400b9c0 con 0x7f24200d1470 2024-01-27T02:42:06.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.674+0000 7f241dffb700 1 -- 172.21.15.7:0/3770548039 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241400bd60 con 0x7f24200d1470 2024-01-27T02:42:06.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.675+0000 7f2425718700 1 -- 172.21.15.7:0/3770548039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24200d1470 msgr2=0x7f24200b2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:06.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.675+0000 7f2425718700 1 --2- 172.21.15.7:0/3770548039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24200d1470 0x7f24200b2dc0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f2420069ab0 tx=0x7f2414009250 comp rx=0 tx=0).stop 2024-01-27T02:42:06.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.675+0000 7f2425718700 1 -- 172.21.15.7:0/3770548039 shutdown_connections 2024-01-27T02:42:06.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.675+0000 7f2425718700 1 --2- 172.21.15.7:0/3770548039 >> v2:172.21.15.188:3300/0 conn(0x7f24200d10a0 0x7f24200b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:06.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.675+0000 7f2425718700 1 --2- 172.21.15.7:0/3770548039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2420159b40 0x7f2420159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:06.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.675+0000 7f2425718700 1 --2- 172.21.15.7:0/3770548039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24200d1470 0x7f24200b2dc0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:06.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.675+0000 7f2425718700 1 -- 172.21.15.7:0/3770548039 >> 172.21.15.7:0/3770548039 conn(0x7f24200679b0 msgr2=0x7f2420067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:06.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.675+0000 7f2425718700 1 -- 172.21.15.7:0/3770548039 shutdown_connections 2024-01-27T02:42:06.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.675+0000 7f2425718700 1 -- 172.21.15.7:0/3770548039 wait complete. 2024-01-27T02:42:06.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.677+0000 7f2425718700 1 Processor -- start 2024-01-27T02:42:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.677+0000 7f2425718700 1 -- start start 2024-01-27T02:42:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.677+0000 7f2425718700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24200d10a0 0x7f24201494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.677+0000 7f2425718700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24200d1470 0x7f242016dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.677+0000 7f2425718700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2420159b40 0x7f24200c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.677+0000 7f2425718700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24200c5c00 con 0x7f2420159b40 2024-01-27T02:42:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.677+0000 7f2425718700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242016e400 con 0x7f24200d1470 2024-01-27T02:42:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.677+0000 7f2425718700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242016e6e0 con 0x7f24200d10a0 2024-01-27T02:42:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.678+0000 7f241f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2420159b40 0x7f24200c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.678+0000 7f241e7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24200d1470 0x7f242016dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.678+0000 7f241f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2420159b40 0x7f24200c7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50566/0 (socket says 172.21.15.7:50566) 2024-01-27T02:42:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.678+0000 7f241f7fe700 1 -- 172.21.15.7:0/3277845768 learned_addr learned my addr 172.21.15.7:0/3277845768 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:42:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.678+0000 7f241effd700 1 --2- 172.21.15.7:0/3277845768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24200d10a0 0x7f24201494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.678+0000 7f241f7fe700 1 -- 172.21.15.7:0/3277845768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24200d10a0 msgr2=0x7f24201494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.678+0000 7f241f7fe700 1 --2- 172.21.15.7:0/3277845768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24200d10a0 0x7f24201494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.678+0000 7f241f7fe700 1 -- 172.21.15.7:0/3277845768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24200d1470 msgr2=0x7f242016dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.678+0000 7f241f7fe700 1 --2- 172.21.15.7:0/3277845768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24200d1470 0x7f242016dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.678+0000 7f241f7fe700 1 -- 172.21.15.7:0/3277845768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24140094f0 con 0x7f2420159b40 2024-01-27T02:42:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.678+0000 7f241f7fe700 1 --2- 172.21.15.7:0/3277845768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2420159b40 0x7f24200c7f70 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f2418007f20 tx=0x7f2418009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.678+0000 7f2406ffd700 1 -- 172.21.15.7:0/3277845768 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241800eb90 con 0x7f2420159b40 2024-01-27T02:42:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.678+0000 7f2406ffd700 1 -- 172.21.15.7:0/3277845768 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241800ed10 con 0x7f2420159b40 2024-01-27T02:42:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.678+0000 7f2425718700 1 -- 172.21.15.7:0/3277845768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f242016e9c0 con 0x7f2420159b40 2024-01-27T02:42:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.678+0000 7f2406ffd700 1 -- 172.21.15.7:0/3277845768 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418014670 con 0x7f2420159b40 2024-01-27T02:42:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.678+0000 7f2425718700 1 -- 172.21.15.7:0/3277845768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24200cbe70 con 0x7f2420159b40 2024-01-27T02:42:06.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.679+0000 7f2425718700 1 -- 172.21.15.7:0/3277845768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f242016ec00 con 0x7f2420159b40 2024-01-27T02:42:06.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.680+0000 7f2406ffd700 1 -- 172.21.15.7:0/3277845768 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f24180149e0 con 0x7f2420159b40 2024-01-27T02:42:06.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.680+0000 7f2406ffd700 1 --2- 172.21.15.7:0/3277845768 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f240c066f30 0x7f240c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:06.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.681+0000 7f2406ffd700 1 -- 172.21.15.7:0/3277845768 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2418084ba0 con 0x7f2420159b40 2024-01-27T02:42:06.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.681+0000 7f241effd700 1 --2- 172.21.15.7:0/3277845768 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f240c066f30 0x7f240c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:06.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.681+0000 7f241effd700 1 --2- 172.21.15.7:0/3277845768 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f240c066f30 0x7f240c0693e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2410006fd0 tx=0x7f2410008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:06.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.685+0000 7f2406ffd700 1 -- 172.21.15.7:0/3277845768 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f242016ec00 con 0x7f2420159b40 2024-01-27T02:42:06.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.938+0000 7f2425718700 1 -- 172.21.15.7:0/3277845768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f242016ec00 con 0x7f2420159b40 2024-01-27T02:42:06.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.939+0000 7f2406ffd700 1 -- 172.21.15.7:0/3277845768 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f242016ec00 con 0x7f2420159b40 2024-01-27T02:42:06.940 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:42:06.940 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6473,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/3449593528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3449593528},{"type":"v1","addr":"172.21.15.188:6836","nonce":3449593528}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3473642033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3473642033},{"type":"v1","addr":"172.21.15.188:6837","nonce":3473642033}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6479,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2550009140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2550009140},{"type":"v1","addr":"172.21.15.7:6833","nonce":2550009140}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:41:57.646012+0000","modified":"2024-01-27T02:42:01.251419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6485":{"gid":6485,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3213683076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3213683076},{"type":"v1","addr":"172.21.15.7:6835","nonce":3213683076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2024-01-27T02:42:06.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.941+0000 7f2425718700 1 -- 172.21.15.7:0/3277845768 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f240c066f30 msgr2=0x7f240c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:06.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.941+0000 7f2425718700 1 --2- 172.21.15.7:0/3277845768 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f240c066f30 0x7f240c0693e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2410006fd0 tx=0x7f2410008040 comp rx=0 tx=0).stop 2024-01-27T02:42:06.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.941+0000 7f2425718700 1 -- 172.21.15.7:0/3277845768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2420159b40 msgr2=0x7f24200c7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:06.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.941+0000 7f2425718700 1 --2- 172.21.15.7:0/3277845768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2420159b40 0x7f24200c7f70 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f2418007f20 tx=0x7f2418009860 comp rx=0 tx=0).stop 2024-01-27T02:42:06.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.941+0000 7f2425718700 1 -- 172.21.15.7:0/3277845768 shutdown_connections 2024-01-27T02:42:06.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.941+0000 7f2425718700 1 --2- 172.21.15.7:0/3277845768 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f240c066f30 0x7f240c0693e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:06.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.941+0000 7f2425718700 1 --2- 172.21.15.7:0/3277845768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2420159b40 0x7f24200c7f70 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:06.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.941+0000 7f2425718700 1 --2- 172.21.15.7:0/3277845768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24200d10a0 0x7f24201494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:06.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.941+0000 7f2425718700 1 --2- 172.21.15.7:0/3277845768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24200d1470 0x7f242016dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:06.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.941+0000 7f2425718700 1 -- 172.21.15.7:0/3277845768 >> 172.21.15.7:0/3277845768 conn(0x7f24200679b0 msgr2=0x7f2420067d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:06.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.941+0000 7f2425718700 1 -- 172.21.15.7:0/3277845768 shutdown_connections 2024-01-27T02:42:06.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:06.941+0000 7f2425718700 1 -- 172.21.15.7:0/3277845768 wait complete. 2024-01-27T02:42:06.944 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 86 2024-01-27T02:42:06.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-01-27T02:42:07.109 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","whoami":0,"id":6485,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.8801825119999993,"mdsmap_epoch":86,"osdmap_epoch":164,"osdmap_epoch_barrier":164,"uptime":13.856061594} 2024-01-27T02:42:07.109 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", "fs_name": "cephfs", "id": 6485, "mdsmap_epoch": 86, "osdmap_epoch": 164, "osdmap_epoch_barrier": 164, "rank_uptime": 8.880182512, "state": "up:active", "uptime": 13.856061594, "want_state": "up:active", "whoami": 0 } 2024-01-27T02:42:07.110 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-01-27T02:42:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.244+0000 7fbfd8290700 1 Processor -- start 2024-01-27T02:42:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.244+0000 7fbfd8290700 1 -- start start 2024-01-27T02:42:07.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.245+0000 7fbfd8290700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfd00a7fe0 0x7fbfd006f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.245+0000 7fbfd8290700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbfd015dfd0 0x7fbfd006f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.245+0000 7fbfd8290700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfd00afcf0 con 0x7fbfd00a7c10 2024-01-27T02:42:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.245+0000 7fbfd8290700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd00aea50 con 0x7fbfd015dfd0 2024-01-27T02:42:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.245+0000 7fbfd8290700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd015a880 con 0x7fbfd00a7fe0 2024-01-27T02:42:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.245+0000 7fbfd682d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbfd015dfd0 0x7fbfd006f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.245+0000 7fbfd582b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfd00a7fe0 0x7fbfd006f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.245+0000 7fbfd582b700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfd00a7fe0 0x7fbfd006f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:56742/0 (socket says 172.21.15.7:56742) 2024-01-27T02:42:07.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.245+0000 7fbfd682d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbfd015dfd0 0x7fbfd006f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:40994/0 (socket says 172.21.15.7:40994) 2024-01-27T02:42:07.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.245+0000 7fbfd602c700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fbfd00a7c10 0x7fbfd00ae340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:42524/0 (socket says 172.21.15.7:42524) 2024-01-27T02:42:07.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.245+0000 7fbfd682d700 1 -- 172.21.15.7:0/249295432 learned_addr learned my addr 172.21.15.7:0/249295432 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:42:07.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.246+0000 7fbfd502a700 1 -- 172.21.15.7:0/249295432 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2935785000 0 0) 0x7fbfd00afcf0 con 0x7fbfd00a7c10 2024-01-27T02:42:07.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.246+0000 7fbfd582b700 1 -- 172.21.15.7:0/249295432 >> v2:172.21.15.188:3300/0 conn(0x7fbfd015dfd0 msgr2=0x7fbfd006f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:42:07.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.246+0000 7fbfd582b700 1 --2- 172.21.15.7:0/249295432 >> v2:172.21.15.188:3300/0 conn(0x7fbfd015dfd0 0x7fbfd006f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:07.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.246+0000 7fbfd582b700 1 -- 172.21.15.7:0/249295432 >> v1:172.21.15.188:6789/0 conn(0x7fbfd00a7c10 legacy=0x7fbfd00ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:07.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.246+0000 7fbfd582b700 1 -- 172.21.15.7:0/249295432 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd00aec50 con 0x7fbfd00a7fe0 2024-01-27T02:42:07.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.246+0000 7fbfd582b700 1 --2- 172.21.15.7:0/249295432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfd00a7fe0 0x7fbfd006f390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc0098e0 tx=0x7fbfcc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8df0dac8c7963498 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:07.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.246+0000 7fbfd502a700 1 -- 172.21.15.7:0/249295432 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc00b640 con 0x7fbfd00a7fe0 2024-01-27T02:42:07.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.246+0000 7fbfd502a700 1 -- 172.21.15.7:0/249295432 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfcc00b7a0 con 0x7fbfd00a7fe0 2024-01-27T02:42:07.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.246+0000 7fbfd502a700 1 -- 172.21.15.7:0/249295432 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc00bb40 con 0x7fbfd00a7fe0 2024-01-27T02:42:07.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.247+0000 7fbfd8290700 1 -- 172.21.15.7:0/249295432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfd00a7fe0 msgr2=0x7fbfd006f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:07.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.247+0000 7fbfd8290700 1 --2- 172.21.15.7:0/249295432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfd00a7fe0 0x7fbfd006f390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc0098e0 tx=0x7fbfcc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:42:07.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.247+0000 7fbfd8290700 1 -- 172.21.15.7:0/249295432 shutdown_connections 2024-01-27T02:42:07.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.247+0000 7fbfd8290700 1 --2- 172.21.15.7:0/249295432 >> v2:172.21.15.188:3300/0 conn(0x7fbfd015dfd0 0x7fbfd006f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:07.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.247+0000 7fbfd8290700 1 --2- 172.21.15.7:0/249295432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfd00a7fe0 0x7fbfd006f390 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:07.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.247+0000 7fbfd8290700 1 -- 172.21.15.7:0/249295432 >> 172.21.15.7:0/249295432 conn(0x7fbfd00679b0 msgr2=0x7fbfd0067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:07.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.247+0000 7fbfd8290700 1 -- 172.21.15.7:0/249295432 shutdown_connections 2024-01-27T02:42:07.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.247+0000 7fbfd8290700 1 -- 172.21.15.7:0/249295432 wait complete. 2024-01-27T02:42:07.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.248+0000 7fbfd8290700 1 Processor -- start 2024-01-27T02:42:07.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.248+0000 7fbfd8290700 1 -- start start 2024-01-27T02:42:07.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.249+0000 7fbfd8290700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfd00a7c10 0x7fbfd014f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:07.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.249+0000 7fbfd8290700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfd00a7fe0 0x7fbfd013d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:07.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.249+0000 7fbfd8290700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfd015dfd0 0x7fbfd00b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:07.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.249+0000 7fbfd8290700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd013ee30 con 0x7fbfd015dfd0 2024-01-27T02:42:07.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.249+0000 7fbfd8290700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd013e220 con 0x7fbfd00a7c10 2024-01-27T02:42:07.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.249+0000 7fbfd8290700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd013e500 con 0x7fbfd00a7fe0 2024-01-27T02:42:07.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.249+0000 7fbfd682d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfd015dfd0 0x7fbfd00b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:07.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.249+0000 7fbfd682d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfd015dfd0 0x7fbfd00b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50568/0 (socket says 172.21.15.7:50568) 2024-01-27T02:42:07.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.249+0000 7fbfd682d700 1 -- 172.21.15.7:0/1373606755 learned_addr learned my addr 172.21.15.7:0/1373606755 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:42:07.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.249+0000 7fbfd582b700 1 --2- 172.21.15.7:0/1373606755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfd00a7fe0 0x7fbfd013d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:07.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.249+0000 7fbfd602c700 1 --2- 172.21.15.7:0/1373606755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfd00a7c10 0x7fbfd014f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:07.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.249+0000 7fbfd682d700 1 -- 172.21.15.7:0/1373606755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfd00a7fe0 msgr2=0x7fbfd013d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:07.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.249+0000 7fbfd682d700 1 --2- 172.21.15.7:0/1373606755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfd00a7fe0 0x7fbfd013d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:07.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.249+0000 7fbfd682d700 1 -- 172.21.15.7:0/1373606755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfd00a7c10 msgr2=0x7fbfd014f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:07.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.249+0000 7fbfd682d700 1 --2- 172.21.15.7:0/1373606755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfd00a7c10 0x7fbfd014f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:07.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.249+0000 7fbfd682d700 1 -- 172.21.15.7:0/1373606755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfcc009420 con 0x7fbfd015dfd0 2024-01-27T02:42:07.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.249+0000 7fbfd602c700 1 --2- 172.21.15.7:0/1373606755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfd00a7c10 0x7fbfd014f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:42:07.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.249+0000 7fbfd682d700 1 --2- 172.21.15.7:0/1373606755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfd015dfd0 0x7fbfd00b0160 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009db0 tx=0x7fbfc800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:07.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.250+0000 7fbfc67fc700 1 -- 172.21.15.7:0/1373606755 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc800c940 con 0x7fbfd015dfd0 2024-01-27T02:42:07.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.250+0000 7fbfc67fc700 1 -- 172.21.15.7:0/1373606755 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc800f040 con 0x7fbfd015dfd0 2024-01-27T02:42:07.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.250+0000 7fbfc67fc700 1 -- 172.21.15.7:0/1373606755 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc800ccf0 con 0x7fbfd015dfd0 2024-01-27T02:42:07.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.250+0000 7fbfd8290700 1 -- 172.21.15.7:0/1373606755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd013e7e0 con 0x7fbfd015dfd0 2024-01-27T02:42:07.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.250+0000 7fbfd8290700 1 -- 172.21.15.7:0/1373606755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd015abd0 con 0x7fbfd015dfd0 2024-01-27T02:42:07.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.251+0000 7fbfd8290700 1 -- 172.21.15.7:0/1373606755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd013efa0 con 0x7fbfd015dfd0 2024-01-27T02:42:07.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.251+0000 7fbfc67fc700 1 -- 172.21.15.7:0/1373606755 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbfc8014650 con 0x7fbfd015dfd0 2024-01-27T02:42:07.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.252+0000 7fbfc67fc700 1 --2- 172.21.15.7:0/1373606755 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbfb4066f30 0x7fbfb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:42:07.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.252+0000 7fbfc67fc700 1 -- 172.21.15.7:0/1373606755 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbfc8084b20 con 0x7fbfd015dfd0 2024-01-27T02:42:07.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.252+0000 7fbfd602c700 1 --2- 172.21.15.7:0/1373606755 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbfb4066f30 0x7fbfb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:42:07.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.252+0000 7fbfd602c700 1 --2- 172.21.15.7:0/1373606755 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbfb4066f30 0x7fbfb40693e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0068160 tx=0x7fbfc0007c80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:42:07.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.256+0000 7fbfc67fc700 1 -- 172.21.15.7:0/1373606755 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfc8051d90 con 0x7fbfd015dfd0 2024-01-27T02:42:07.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:07.466+0000 7fbfd8290700 1 -- 172.21.15.7:0/1373606755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7fbfd004c710 con 0x7fbfd015dfd0 2024-01-27T02:42:08.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:08.335+0000 7fbfc67fc700 1 -- 172.21.15.7:0/1373606755 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v165) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbfc80553b0 con 0x7fbfd015dfd0 2024-01-27T02:42:08.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:08.338+0000 7fbfd8290700 1 -- 172.21.15.7:0/1373606755 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbfb4066f30 msgr2=0x7fbfb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:08.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:08.338+0000 7fbfd8290700 1 --2- 172.21.15.7:0/1373606755 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbfb4066f30 0x7fbfb40693e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0068160 tx=0x7fbfc0007c80 comp rx=0 tx=0).stop 2024-01-27T02:42:08.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:08.338+0000 7fbfd8290700 1 -- 172.21.15.7:0/1373606755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfd015dfd0 msgr2=0x7fbfd00b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:42:08.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:08.338+0000 7fbfd8290700 1 --2- 172.21.15.7:0/1373606755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfd015dfd0 0x7fbfd00b0160 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009db0 tx=0x7fbfc800b040 comp rx=0 tx=0).stop 2024-01-27T02:42:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:08.338+0000 7fbfd8290700 1 -- 172.21.15.7:0/1373606755 shutdown_connections 2024-01-27T02:42:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:08.338+0000 7fbfd8290700 1 --2- 172.21.15.7:0/1373606755 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbfb4066f30 0x7fbfb40693e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:08.338+0000 7fbfd8290700 1 --2- 172.21.15.7:0/1373606755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfd015dfd0 0x7fbfd00b0160 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:08.338+0000 7fbfd8290700 1 --2- 172.21.15.7:0/1373606755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfd00a7c10 0x7fbfd014f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:08.338+0000 7fbfd8290700 1 --2- 172.21.15.7:0/1373606755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfd00a7fe0 0x7fbfd013d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:42:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:08.339+0000 7fbfd8290700 1 -- 172.21.15.7:0/1373606755 >> 172.21.15.7:0/1373606755 conn(0x7fbfd00679b0 msgr2=0x7fbfd0151a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:42:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:08.339+0000 7fbfd8290700 1 -- 172.21.15.7:0/1373606755 shutdown_connections 2024-01-27T02:42:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:42:08.339+0000 7fbfd8290700 1 -- 172.21.15.7:0/1373606755 wait complete. 2024-01-27T02:42:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-01-27T02:42:08.356 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-01-27T02:42:08.508 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_mon_report_interval":"5"} 2024-01-27T02:42:08.508 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-01-27T02:42:08.509 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-01-27T02:42:08.650 INFO:teuthology.orchestra.run.smithi007.stdout:{"mon_tick_interval":"5"} 2024-01-27T02:42:08.650 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-01-27T02:42:08.651 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2024-01-27T02:42:08.651 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-01-27T02:42:08.651 DEBUG:teuthology.orchestra.run.smithi007:> import time 2024-01-27T02:42:08.651 DEBUG:teuthology.orchestra.run.smithi007:> import datetime 2024-01-27T02:42:08.652 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2024-01-27T02:42:08.652 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-01-27T02:42:08.652 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:42:08.652 DEBUG:teuthology.orchestra.run.smithi007:> # Write some buffered data through before going full, all should be well 2024-01-27T02:42:08.652 DEBUG:teuthology.orchestra.run.smithi007:> print("writing some data through which we expect to succeed") 2024-01-27T02:42:08.652 DEBUG:teuthology.orchestra.run.smithi007:> bytes = 0 2024-01-27T02:42:08.652 DEBUG:teuthology.orchestra.run.smithi007:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-01-27T02:42:08.653 DEBUG:teuthology.orchestra.run.smithi007:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-01-27T02:42:08.653 DEBUG:teuthology.orchestra.run.smithi007:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2024-01-27T02:42:08.653 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f) 2024-01-27T02:42:08.653 DEBUG:teuthology.orchestra.run.smithi007:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-01-27T02:42:08.653 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:42:08.653 DEBUG:teuthology.orchestra.run.smithi007:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-01-27T02:42:08.654 DEBUG:teuthology.orchestra.run.smithi007:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-01-27T02:42:08.654 DEBUG:teuthology.orchestra.run.smithi007:> # buffered IO, the error should always only appear from fsync and not 2024-01-27T02:42:08.654 DEBUG:teuthology.orchestra.run.smithi007:> # from write 2024-01-27T02:42:08.654 DEBUG:teuthology.orchestra.run.smithi007:> full = False 2024-01-27T02:42:08.654 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:42:08.654 DEBUG:teuthology.orchestra.run.smithi007:> for n in range(0, int(32 * 1.1)): 2024-01-27T02:42:08.655 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-01-27T02:42:08.655 DEBUG:teuthology.orchestra.run.smithi007:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-01-27T02:42:08.655 DEBUG:teuthology.orchestra.run.smithi007:> print("wrote bytes via buffered write, moving on to fsync") 2024-01-27T02:42:08.655 DEBUG:teuthology.orchestra.run.smithi007:> except OSError as e: 2024-01-27T02:42:08.655 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-01-27T02:42:08.655 DEBUG:teuthology.orchestra.run.smithi007:> print("Unexpected error %s from write() instead of fsync()" % e) 2024-01-27T02:42:08.656 DEBUG:teuthology.orchestra.run.smithi007:> raise 2024-01-27T02:42:08.656 DEBUG:teuthology.orchestra.run.smithi007:> else: 2024-01-27T02:42:08.656 DEBUG:teuthology.orchestra.run.smithi007:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-01-27T02:42:08.656 DEBUG:teuthology.orchestra.run.smithi007:> full = True 2024-01-27T02:42:08.656 DEBUG:teuthology.orchestra.run.smithi007:> break 2024-01-27T02:42:08.656 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:42:08.656 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-01-27T02:42:08.657 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f) 2024-01-27T02:42:08.657 DEBUG:teuthology.orchestra.run.smithi007:> print("fsync'"'"'ed successfully") 2024-01-27T02:42:08.657 DEBUG:teuthology.orchestra.run.smithi007:> except OSError as e: 2024-01-27T02:42:08.657 DEBUG:teuthology.orchestra.run.smithi007:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-01-27T02:42:08.657 DEBUG:teuthology.orchestra.run.smithi007:> full = True 2024-01-27T02:42:08.657 DEBUG:teuthology.orchestra.run.smithi007:> break 2024-01-27T02:42:08.657 DEBUG:teuthology.orchestra.run.smithi007:> else: 2024-01-27T02:42:08.657 DEBUG:teuthology.orchestra.run.smithi007:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-01-27T02:42:08.657 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:42:08.657 DEBUG:teuthology.orchestra.run.smithi007:> if n > 32 * 0.9: 2024-01-27T02:42:08.657 DEBUG:teuthology.orchestra.run.smithi007:> # Be cautious in the last region where we expect to hit 2024-01-27T02:42:08.657 DEBUG:teuthology.orchestra.run.smithi007:> # the full condition, so that we don'"'"'t overshoot too dramatically 2024-01-27T02:42:08.658 DEBUG:teuthology.orchestra.run.smithi007:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-01-27T02:42:08.658 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(15.0) 2024-01-27T02:42:08.658 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:42:08.658 DEBUG:teuthology.orchestra.run.smithi007:> if not full: 2024-01-27T02:42:08.658 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2024-01-27T02:42:08.658 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:42:08.658 DEBUG:teuthology.orchestra.run.smithi007:> # close() should not raise an error because we already caught it in 2024-01-27T02:42:08.658 DEBUG:teuthology.orchestra.run.smithi007:> # fsync. There shouldn'"'"'t have been any more writeback errors 2024-01-27T02:42:08.658 DEBUG:teuthology.orchestra.run.smithi007:> # since then because all IOs got cancelled on the full flag. 2024-01-27T02:42:08.658 DEBUG:teuthology.orchestra.run.smithi007:> print("calling close") 2024-01-27T02:42:08.658 DEBUG:teuthology.orchestra.run.smithi007:> os.close(f) 2024-01-27T02:42:08.658 DEBUG:teuthology.orchestra.run.smithi007:> print("close() did not raise error") 2024-01-27T02:42:08.659 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-27T02:42:08.659 DEBUG:teuthology.orchestra.run.smithi007:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-01-27T02:42:08.659 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-01-27T02:42:09.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:42:09.348 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:42:39.416 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:42:39.419 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:43:09.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:43:09.483 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:43:13.420 INFO:teuthology.orchestra.run.smithi007.stdout:writing some data through which we expect to succeed 2024-01-27T02:43:13.420 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-01-27T02:43:13.420 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.420 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.420 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 1.00 MB 2024-01-27T02:43:13.420 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.420 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.420 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 2.00 MB 2024-01-27T02:43:13.421 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.421 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.421 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 3.00 MB 2024-01-27T02:43:13.421 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.421 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.421 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 4.00 MB 2024-01-27T02:43:13.421 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.421 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.421 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 5.00 MB 2024-01-27T02:43:13.421 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.421 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.421 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 6.00 MB 2024-01-27T02:43:13.421 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.421 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.422 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 7.00 MB 2024-01-27T02:43:13.422 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.422 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.422 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 8.00 MB 2024-01-27T02:43:13.422 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.422 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.422 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 9.00 MB 2024-01-27T02:43:13.422 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.422 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.422 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 10.00 MB 2024-01-27T02:43:13.422 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.422 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.422 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 11.00 MB 2024-01-27T02:43:13.422 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.423 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.423 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 12.00 MB 2024-01-27T02:43:13.423 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.423 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.423 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 13.00 MB 2024-01-27T02:43:13.423 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.423 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.423 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 14.00 MB 2024-01-27T02:43:13.423 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.423 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.423 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 15.00 MB 2024-01-27T02:43:13.423 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.423 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.423 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 16.00 MB 2024-01-27T02:43:13.424 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.424 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.424 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 17.00 MB 2024-01-27T02:43:13.424 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.424 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.424 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 18.00 MB 2024-01-27T02:43:13.424 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.424 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.424 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 19.00 MB 2024-01-27T02:43:13.424 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.424 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.424 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 20.00 MB 2024-01-27T02:43:13.424 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.424 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.425 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 21.00 MB 2024-01-27T02:43:13.425 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.425 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.425 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 22.00 MB 2024-01-27T02:43:13.425 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.425 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.425 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 23.00 MB 2024-01-27T02:43:13.425 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.425 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.425 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 24.00 MB 2024-01-27T02:43:13.425 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.425 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.425 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 25.00 MB 2024-01-27T02:43:13.425 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.426 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.426 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 26.00 MB 2024-01-27T02:43:13.426 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.426 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.426 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 27.00 MB 2024-01-27T02:43:13.426 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.426 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.426 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 28.00 MB 2024-01-27T02:43:13.426 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.426 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.426 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 29.00 MB 2024-01-27T02:43:13.426 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.426 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.426 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 30.00 MB 2024-01-27T02:43:13.427 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-27T02:43:13.427 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.427 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.427 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 31.00 MB 2024-01-27T02:43:13.427 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-27T02:43:13.427 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.427 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.427 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 32.00 MB 2024-01-27T02:43:13.427 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-27T02:43:13.427 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-27T02:43:13.427 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-27T02:43:13.427 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 33.00 MB 2024-01-27T02:43:13.427 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-27T02:43:13.427 INFO:teuthology.orchestra.run.smithi007.stdout:Reached fullness after 33.00 MB 2024-01-27T02:43:13.428 INFO:teuthology.orchestra.run.smithi007.stdout:calling close 2024-01-27T02:43:13.428 INFO:teuthology.orchestra.run.smithi007.stdout:close() did not raise error 2024-01-27T02:43:13.615 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-27T02:43:13.615 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:43:13.671 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:43:13.725 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-27T02:43:13.725 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-27T02:43:13.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:43:13.725 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-27T02:43:13.781 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:43:13.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:43:13.781 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-27T02:43:13.803 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-27T02:43:13.803 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:43:13.803 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-01-27T02:43:13.850 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:43:13.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:43:13.851 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-27T02:43:13.870 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:43:14.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.009+0000 7f3d23c92700 1 Processor -- start 2024-01-27T02:43:14.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.009+0000 7f3d23c92700 1 -- start start 2024-01-27T02:43:14.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.010+0000 7f3d23c92700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3d1c0c8e80 0x7f3d1c15dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:14.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.010+0000 7f3d23c92700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d1c15a9b0 con 0x7f3d1c0cbbc0 2024-01-27T02:43:14.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.010+0000 7f3d23c92700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d1c15ecb0 con 0x7f3d1c0c8ab0 2024-01-27T02:43:14.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.010+0000 7f3d23c92700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d1c15e410 con 0x7f3d1c0c8e80 2024-01-27T02:43:14.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.010+0000 7f3d2122d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3d1c0c8e80 0x7f3d1c15dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:14.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.010+0000 7f3d2222f700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3d1c0cbbc0 0x7f3d1c15ae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46694/0 (socket says 172.21.15.7:46694) 2024-01-27T02:43:14.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.010+0000 7f3d2222f700 1 -- 172.21.15.7:0/3599259217 learned_addr learned my addr 172.21.15.7:0/3599259217 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:14.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.010+0000 7f3d2122d700 1 -- 172.21.15.7:0/3599259217 >> v1:172.21.15.188:6789/0 conn(0x7f3d1c0c8ab0 legacy=0x7f3d1c0cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.011+0000 7f3d2122d700 1 -- 172.21.15.7:0/3599259217 >> v1:172.21.15.7:6789/0 conn(0x7f3d1c0cbbc0 legacy=0x7f3d1c15ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.011+0000 7f3d2122d700 1 -- 172.21.15.7:0/3599259217 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d1c15e6f0 con 0x7f3d1c0c8e80 2024-01-27T02:43:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.011+0000 7f3d20a2c700 1 -- 172.21.15.7:0/3599259217 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1877446688 0 0) 0x7f3d1c15a9b0 con 0x7f3d1c0cbbc0 2024-01-27T02:43:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.011+0000 7f3d2122d700 1 --2- 172.21.15.7:0/3599259217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d1c0c8e80 0x7f3d1c15dd80 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0098e0 tx=0x7f3d0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fccd45f81e3ec9e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.011+0000 7f3d20a2c700 1 -- 172.21.15.7:0/3599259217 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c009e80 con 0x7f3d1c0c8e80 2024-01-27T02:43:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.011+0000 7f3d20a2c700 1 -- 172.21.15.7:0/3599259217 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0c00b860 con 0x7f3d1c0c8e80 2024-01-27T02:43:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.011+0000 7f3d20a2c700 1 -- 172.21.15.7:0/3599259217 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c00bbe0 con 0x7f3d1c0c8e80 2024-01-27T02:43:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.011+0000 7f3d23c92700 1 -- 172.21.15.7:0/3599259217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d1c0c8e80 msgr2=0x7f3d1c15dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.011+0000 7f3d23c92700 1 --2- 172.21.15.7:0/3599259217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d1c0c8e80 0x7f3d1c15dd80 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0098e0 tx=0x7f3d0c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.011+0000 7f3d23c92700 1 -- 172.21.15.7:0/3599259217 shutdown_connections 2024-01-27T02:43:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.011+0000 7f3d23c92700 1 --2- 172.21.15.7:0/3599259217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d1c0c8e80 0x7f3d1c15dd80 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.011+0000 7f3d23c92700 1 -- 172.21.15.7:0/3599259217 >> 172.21.15.7:0/3599259217 conn(0x7f3d1c0cdc30 msgr2=0x7f3d1c0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.012+0000 7f3d23c92700 1 -- 172.21.15.7:0/3599259217 shutdown_connections 2024-01-27T02:43:14.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.012+0000 7f3d23c92700 1 -- 172.21.15.7:0/3599259217 wait complete. 2024-01-27T02:43:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.013+0000 7f3d23c92700 1 Processor -- start 2024-01-27T02:43:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.013+0000 7f3d23c92700 1 -- start start 2024-01-27T02:43:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.014+0000 7f3d23c92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d1c0c8ab0 0x7f3d1c157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.014+0000 7f3d23c92700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d1c0c8e80 0x7f3d1c0b1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.014+0000 7f3d23c92700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d1c0cbbc0 0x7f3d1c0c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.014+0000 7f3d23c92700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c0b29d0 con 0x7f3d1c0c8ab0 2024-01-27T02:43:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.014+0000 7f3d23c92700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c0b1dc0 con 0x7f3d1c0c8e80 2024-01-27T02:43:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.014+0000 7f3d23c92700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c0b20a0 con 0x7f3d1c0cbbc0 2024-01-27T02:43:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.014+0000 7f3d21a2e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d1c0c8ab0 0x7f3d1c157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.014+0000 7f3d21a2e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d1c0c8ab0 0x7f3d1c157790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43602/0 (socket says 172.21.15.7:43602) 2024-01-27T02:43:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.014+0000 7f3d21a2e700 1 -- 172.21.15.7:0/2860530627 learned_addr learned my addr 172.21.15.7:0/2860530627 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.014+0000 7f3d2122d700 1 --2- 172.21.15.7:0/2860530627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d1c0c8e80 0x7f3d1c0b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.014+0000 7f3d2222f700 1 --2- 172.21.15.7:0/2860530627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d1c0cbbc0 0x7f3d1c0c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.014+0000 7f3d21a2e700 1 -- 172.21.15.7:0/2860530627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d1c0cbbc0 msgr2=0x7f3d1c0c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.014+0000 7f3d21a2e700 1 --2- 172.21.15.7:0/2860530627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d1c0cbbc0 0x7f3d1c0c3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.014+0000 7f3d21a2e700 1 -- 172.21.15.7:0/2860530627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d1c0c8e80 msgr2=0x7f3d1c0b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.014+0000 7f3d21a2e700 1 --2- 172.21.15.7:0/2860530627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d1c0c8e80 0x7f3d1c0b1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.014+0000 7f3d21a2e700 1 -- 172.21.15.7:0/2860530627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0c009420 con 0x7f3d1c0c8ab0 2024-01-27T02:43:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.014+0000 7f3d2122d700 1 --2- 172.21.15.7:0/2860530627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d1c0c8e80 0x7f3d1c0b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:43:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.014+0000 7f3d21a2e700 1 --2- 172.21.15.7:0/2860530627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d1c0c8ab0 0x7f3d1c157790 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800c590 tx=0x7f3d1800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.014+0000 7f3d11ffb700 1 -- 172.21.15.7:0/2860530627 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18011590 con 0x7f3d1c0c8ab0 2024-01-27T02:43:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.014+0000 7f3d11ffb700 1 -- 172.21.15.7:0/2860530627 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d180116f0 con 0x7f3d1c0c8ab0 2024-01-27T02:43:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.014+0000 7f3d23c92700 1 -- 172.21.15.7:0/2860530627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1c0b2380 con 0x7f3d1c0c8ab0 2024-01-27T02:43:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.015+0000 7f3d11ffb700 1 -- 172.21.15.7:0/2860530627 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18011ab0 con 0x7f3d1c0c8ab0 2024-01-27T02:43:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.015+0000 7f3d23c92700 1 -- 172.21.15.7:0/2860530627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1c157cd0 con 0x7f3d1c0c8ab0 2024-01-27T02:43:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.016+0000 7f3d23c92700 1 -- 172.21.15.7:0/2860530627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d1c0ad900 con 0x7f3d1c0c8ab0 2024-01-27T02:43:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.016+0000 7f3d11ffb700 1 -- 172.21.15.7:0/2860530627 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3d1801b3b0 con 0x7f3d1c0c8ab0 2024-01-27T02:43:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.017+0000 7f3d11ffb700 1 --2- 172.21.15.7:0/2860530627 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3d00066f30 0x7f3d000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.017+0000 7f3d11ffb700 1 -- 172.21.15.7:0/2860530627 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f3d18083ba0 con 0x7f3d1c0c8ab0 2024-01-27T02:43:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.017+0000 7f3d11ffb700 1 -- 172.21.15.7:0/2860530627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=169}) v3 -- 0x7f3d0006eab0 con 0x7f3d1c0c8ab0 2024-01-27T02:43:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.017+0000 7f3d2122d700 1 --2- 172.21.15.7:0/2860530627 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3d00066f30 0x7f3d000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.018+0000 7f3d2122d700 1 --2- 172.21.15.7:0/2860530627 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3d00066f30 0x7f3d000693e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c06c440 tx=0x7f3d0c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.021+0000 7f3d11ffb700 1 -- 172.21.15.7:0/2860530627 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d18017070 con 0x7f3d1c0c8ab0 2024-01-27T02:43:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.285+0000 7f3d23c92700 1 -- 172.21.15.7:0/2860530627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d1c04c710 con 0x7f3d1c0c8ab0 2024-01-27T02:43:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.285+0000 7f3d11ffb700 1 -- 172.21.15.7:0/2860530627 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f3d1c0ad900 con 0x7f3d1c0c8ab0 2024-01-27T02:43:14.287 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:14.287 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6473,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/3449593528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3449593528},{"type":"v1","addr":"172.21.15.188:6836","nonce":3449593528}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3473642033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3473642033},{"type":"v1","addr":"172.21.15.188:6837","nonce":3473642033}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6479,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2550009140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2550009140},{"type":"v1","addr":"172.21.15.7:6833","nonce":2550009140}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:41:57.646012+0000","modified":"2024-01-27T02:42:01.251419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6485":{"gid":6485,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3213683076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3213683076},{"type":"v1","addr":"172.21.15.7:6835","nonce":3213683076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2024-01-27T02:43:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.288+0000 7f3d23c92700 1 -- 172.21.15.7:0/2860530627 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3d00066f30 msgr2=0x7f3d000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.288+0000 7f3d23c92700 1 --2- 172.21.15.7:0/2860530627 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3d00066f30 0x7f3d000693e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c06c440 tx=0x7f3d0c017000 comp rx=0 tx=0).stop 2024-01-27T02:43:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.288+0000 7f3d23c92700 1 -- 172.21.15.7:0/2860530627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d1c0c8ab0 msgr2=0x7f3d1c157790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.288+0000 7f3d23c92700 1 --2- 172.21.15.7:0/2860530627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d1c0c8ab0 0x7f3d1c157790 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800c590 tx=0x7f3d1800c950 comp rx=0 tx=0).stop 2024-01-27T02:43:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.288+0000 7f3d23c92700 1 -- 172.21.15.7:0/2860530627 shutdown_connections 2024-01-27T02:43:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.288+0000 7f3d23c92700 1 --2- 172.21.15.7:0/2860530627 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3d00066f30 0x7f3d000693e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.288+0000 7f3d23c92700 1 --2- 172.21.15.7:0/2860530627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d1c0cbbc0 0x7f3d1c0c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.288+0000 7f3d23c92700 1 --2- 172.21.15.7:0/2860530627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d1c0c8ab0 0x7f3d1c157790 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.288+0000 7f3d23c92700 1 --2- 172.21.15.7:0/2860530627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d1c0c8e80 0x7f3d1c0b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.288+0000 7f3d23c92700 1 -- 172.21.15.7:0/2860530627 >> 172.21.15.7:0/2860530627 conn(0x7f3d1c0cdc30 msgr2=0x7f3d1c06ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.288+0000 7f3d23c92700 1 -- 172.21.15.7:0/2860530627 shutdown_connections 2024-01-27T02:43:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.288+0000 7f3d23c92700 1 -- 172.21.15.7:0/2860530627 wait complete. 2024-01-27T02:43:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 86 2024-01-27T02:43:14.303 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:43:14.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.434+0000 7f1eab581700 1 Processor -- start 2024-01-27T02:43:14.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.434+0000 7f1eab581700 1 -- start start 2024-01-27T02:43:14.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.435+0000 7f1eab581700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1ea40679b0 0x7f1ea40cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:14.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.435+0000 7f1eab581700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea40c9e50 con 0x7f1ea415d590 2024-01-27T02:43:14.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.435+0000 7f1eab581700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea40c9520 con 0x7f1ea40ce2c0 2024-01-27T02:43:14.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.435+0000 7f1eab581700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea40ce780 con 0x7f1ea40679b0 2024-01-27T02:43:14.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.435+0000 7f1ea931d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1ea415d590 0x7f1ea40c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46706/0 (socket says 172.21.15.7:46706) 2024-01-27T02:43:14.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.435+0000 7f1ea931d700 1 -- 172.21.15.7:0/3471514139 learned_addr learned my addr 172.21.15.7:0/3471514139 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:14.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.435+0000 7f1ea8b1c700 1 --2- 172.21.15.7:0/3471514139 >> v2:172.21.15.188:3300/0 conn(0x7f1ea40679b0 0x7f1ea40cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:14.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.435+0000 7f1e97fff700 1 -- 172.21.15.7:0/3471514139 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3091204072 0 0) 0x7f1ea40c9e50 con 0x7f1ea415d590 2024-01-27T02:43:14.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.435+0000 7f1e97fff700 1 -- 172.21.15.7:0/3471514139 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e90003570 con 0x7f1ea415d590 2024-01-27T02:43:14.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.435+0000 7f1e97fff700 1 -- 172.21.15.7:0/3471514139 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2464304272 0 0) 0x7f1e90003570 con 0x7f1ea415d590 2024-01-27T02:43:14.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1e97fff700 1 -- 172.21.15.7:0/3471514139 >> v2:172.21.15.188:3300/0 conn(0x7f1ea40679b0 msgr2=0x7f1ea40cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:14.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1e97fff700 1 --2- 172.21.15.7:0/3471514139 >> v2:172.21.15.188:3300/0 conn(0x7f1ea40679b0 0x7f1ea40cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1e97fff700 1 -- 172.21.15.7:0/3471514139 >> v1:172.21.15.188:6789/0 conn(0x7f1ea40ce2c0 legacy=0x7f1ea40ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:14.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1e97fff700 1 -- 172.21.15.7:0/3471514139 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea40c9800 con 0x7f1ea415d590 2024-01-27T02:43:14.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1e97fff700 1 -- 172.21.15.7:0/3471514139 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f1e98001470 con 0x7f1ea415d590 2024-01-27T02:43:14.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1e97fff700 1 -- 172.21.15.7:0/3471514139 >> v1:172.21.15.7:6789/0 conn(0x7f1ea415d590 legacy=0x7f1ea40c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:14.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1e97fff700 1 --2- 172.21.15.7:0/3471514139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e90003a60 0x7f1e90003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:14.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1e97fff700 1 --2- 172.21.15.7:0/3471514139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e90004810 0x7f1e90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:14.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1e97fff700 1 --2- 172.21.15.7:0/3471514139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e900051c0 0x7f1e9000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:14.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1e97fff700 1 -- 172.21.15.7:0/3471514139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea40ce780 con 0x7f1e90004810 2024-01-27T02:43:14.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1e97fff700 1 -- 172.21.15.7:0/3471514139 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9000c050 con 0x7f1e90003a60 2024-01-27T02:43:14.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1e97fff700 1 -- 172.21.15.7:0/3471514139 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9000c210 con 0x7f1e900051c0 2024-01-27T02:43:14.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1ea8b1c700 1 --2- 172.21.15.7:0/3471514139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e90004810 0x7f1e90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:14.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1ea931d700 1 --2- 172.21.15.7:0/3471514139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e90003a60 0x7f1e90003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:14.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1ea9b1e700 1 --2- 172.21.15.7:0/3471514139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e900051c0 0x7f1e9000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:14.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1ea8b1c700 1 -- 172.21.15.7:0/3471514139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e900051c0 msgr2=0x7f1e9000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:14.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1ea8b1c700 1 --2- 172.21.15.7:0/3471514139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e900051c0 0x7f1e9000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1ea8b1c700 1 -- 172.21.15.7:0/3471514139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e90003a60 msgr2=0x7f1e90003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:14.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1ea8b1c700 1 --2- 172.21.15.7:0/3471514139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e90003a60 0x7f1e90003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1ea8b1c700 1 -- 172.21.15.7:0/3471514139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e90003570 con 0x7f1e90004810 2024-01-27T02:43:14.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1ea9b1e700 1 -- 172.21.15.7:0/3471514139 reap_dead start 2024-01-27T02:43:14.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1ea8b1c700 1 --2- 172.21.15.7:0/3471514139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e90004810 0x7f1e90004c80 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f1ea406f700 tx=0x7f1ea00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab2d0c9b888156e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:14.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.436+0000 7f1e97fff700 1 -- 172.21.15.7:0/3471514139 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea000e6e0 con 0x7f1e90004810 2024-01-27T02:43:14.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.437+0000 7f1e97fff700 1 -- 172.21.15.7:0/3471514139 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea000e840 con 0x7f1e90004810 2024-01-27T02:43:14.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.437+0000 7f1e97fff700 1 -- 172.21.15.7:0/3471514139 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea000ec00 con 0x7f1e90004810 2024-01-27T02:43:14.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.437+0000 7f1eab581700 1 -- 172.21.15.7:0/3471514139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e90004810 msgr2=0x7f1e90004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:14.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.437+0000 7f1eab581700 1 --2- 172.21.15.7:0/3471514139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e90004810 0x7f1e90004c80 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f1ea406f700 tx=0x7f1ea00091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.437+0000 7f1eab581700 1 -- 172.21.15.7:0/3471514139 shutdown_connections 2024-01-27T02:43:14.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.437+0000 7f1eab581700 1 --2- 172.21.15.7:0/3471514139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e90004810 0x7f1e90004c80 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.437+0000 7f1eab581700 1 -- 172.21.15.7:0/3471514139 >> 172.21.15.7:0/3471514139 conn(0x7f1ea4098020 msgr2=0x7f1ea40711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:14.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.437+0000 7f1eab581700 1 -- 172.21.15.7:0/3471514139 shutdown_connections 2024-01-27T02:43:14.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.437+0000 7f1eab581700 1 -- 172.21.15.7:0/3471514139 wait complete. 2024-01-27T02:43:14.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.438+0000 7f1eab581700 1 Processor -- start 2024-01-27T02:43:14.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.438+0000 7f1eab581700 1 -- start start 2024-01-27T02:43:14.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.439+0000 7f1eab581700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e90004810 0x7f1ea414fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:14.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.439+0000 7f1eab581700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ea416a7e0 0x7f1ea416ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:14.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.439+0000 7f1eab581700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ea406b4c0 0x7f1ea406b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:14.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.439+0000 7f1eab581700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea406be70 con 0x7f1ea416a7e0 2024-01-27T02:43:14.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.439+0000 7f1eab581700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea40c6ca0 con 0x7f1e90004810 2024-01-27T02:43:14.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.439+0000 7f1eab581700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea40c7260 con 0x7f1ea406b4c0 2024-01-27T02:43:14.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.439+0000 7f1ea8b1c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ea416a7e0 0x7f1ea416ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:14.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.440+0000 7f1ea8b1c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ea416a7e0 0x7f1ea416ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43612/0 (socket says 172.21.15.7:43612) 2024-01-27T02:43:14.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.440+0000 7f1ea8b1c700 1 -- 172.21.15.7:0/508722554 learned_addr learned my addr 172.21.15.7:0/508722554 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:14.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.440+0000 7f1ea931d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e90004810 0x7f1ea414fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:14.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.440+0000 7f1ea9b1e700 1 --2- 172.21.15.7:0/508722554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ea406b4c0 0x7f1ea406b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:14.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.440+0000 7f1ea8b1c700 1 -- 172.21.15.7:0/508722554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ea406b4c0 msgr2=0x7f1ea406b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:14.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.440+0000 7f1ea8b1c700 1 --2- 172.21.15.7:0/508722554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ea406b4c0 0x7f1ea406b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.440+0000 7f1ea8b1c700 1 -- 172.21.15.7:0/508722554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e90004810 msgr2=0x7f1ea414fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:14.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.440+0000 7f1ea8b1c700 1 --2- 172.21.15.7:0/508722554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e90004810 0x7f1ea414fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.440+0000 7f1ea8b1c700 1 -- 172.21.15.7:0/508722554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea41502b0 con 0x7f1ea416a7e0 2024-01-27T02:43:14.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.440+0000 7f1ea931d700 1 --2- 172.21.15.7:0/508722554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e90004810 0x7f1ea414fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:43:14.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.440+0000 7f1ea8b1c700 1 --2- 172.21.15.7:0/508722554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ea416a7e0 0x7f1ea416ac50 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000f010 tx=0x7f1ea001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:14.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.440+0000 7f1e957fa700 1 -- 172.21.15.7:0/508722554 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0022070 con 0x7f1ea416a7e0 2024-01-27T02:43:14.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.440+0000 7f1e957fa700 1 -- 172.21.15.7:0/508722554 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea0013af0 con 0x7f1ea416a7e0 2024-01-27T02:43:14.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.440+0000 7f1e957fa700 1 -- 172.21.15.7:0/508722554 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea001c450 con 0x7f1ea416a7e0 2024-01-27T02:43:14.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.440+0000 7f1eab581700 1 -- 172.21.15.7:0/508722554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea416b350 con 0x7f1ea416a7e0 2024-01-27T02:43:14.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.440+0000 7f1eab581700 1 -- 172.21.15.7:0/508722554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea40c6ef0 con 0x7f1ea416a7e0 2024-01-27T02:43:14.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.441+0000 7f1eab581700 1 -- 172.21.15.7:0/508722554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea406be70 con 0x7f1ea416a7e0 2024-01-27T02:43:14.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.442+0000 7f1e957fa700 1 -- 172.21.15.7:0/508722554 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1ea001c840 con 0x7f1ea416a7e0 2024-01-27T02:43:14.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.442+0000 7f1e957fa700 1 --2- 172.21.15.7:0/508722554 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1e88066f30 0x7f1e880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:14.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.442+0000 7f1e957fa700 1 -- 172.21.15.7:0/508722554 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f1ea0085c40 con 0x7f1ea416a7e0 2024-01-27T02:43:14.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.442+0000 7f1ea931d700 1 --2- 172.21.15.7:0/508722554 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1e88066f30 0x7f1e880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:14.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.443+0000 7f1e957fa700 1 -- 172.21.15.7:0/508722554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=169}) v3 -- 0x7f1e8806eab0 con 0x7f1ea416a7e0 2024-01-27T02:43:14.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.443+0000 7f1ea931d700 1 --2- 172.21.15.7:0/508722554 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1e88066f30 0x7f1e880693e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1e98001d00 tx=0x7f1e98001e70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:14.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.447+0000 7f1e957fa700 1 -- 172.21.15.7:0/508722554 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ea002b020 con 0x7f1ea416a7e0 2024-01-27T02:43:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.650+0000 7f1eab581700 1 -- 172.21.15.7:0/508722554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ea416b7f0 con 0x7f1ea416a7e0 2024-01-27T02:43:14.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.651+0000 7f1e957fa700 1 -- 172.21.15.7:0/508722554 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) v1 ==== 74+0+15392 (secure 0 0 0) 0x7f1ea0071c30 con 0x7f1ea416a7e0 2024-01-27T02:43:14.653 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:14.653 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":168,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:42:58.993229+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:41:56.176076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-01-27T02:41:56.620523+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.a","mappings":[{"from":4,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:43:14.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.653+0000 7f1eab581700 1 -- 172.21.15.7:0/508722554 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1e88066f30 msgr2=0x7f1e880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:14.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.653+0000 7f1eab581700 1 --2- 172.21.15.7:0/508722554 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1e88066f30 0x7f1e880693e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1e98001d00 tx=0x7f1e98001e70 comp rx=0 tx=0).stop 2024-01-27T02:43:14.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.653+0000 7f1eab581700 1 -- 172.21.15.7:0/508722554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ea416a7e0 msgr2=0x7f1ea416ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:14.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.653+0000 7f1eab581700 1 --2- 172.21.15.7:0/508722554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ea416a7e0 0x7f1ea416ac50 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000f010 tx=0x7f1ea001a000 comp rx=0 tx=0).stop 2024-01-27T02:43:14.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.653+0000 7f1eab581700 1 -- 172.21.15.7:0/508722554 shutdown_connections 2024-01-27T02:43:14.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.653+0000 7f1eab581700 1 --2- 172.21.15.7:0/508722554 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1e88066f30 0x7f1e880693e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.653+0000 7f1eab581700 1 --2- 172.21.15.7:0/508722554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ea406b4c0 0x7f1ea406b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.653+0000 7f1eab581700 1 --2- 172.21.15.7:0/508722554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e90004810 0x7f1ea414fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.653+0000 7f1eab581700 1 --2- 172.21.15.7:0/508722554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ea416a7e0 0x7f1ea416ac50 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.653+0000 7f1eab581700 1 -- 172.21.15.7:0/508722554 >> 172.21.15.7:0/508722554 conn(0x7f1ea4098020 msgr2=0x7f1ea415bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:14.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.653+0000 7f1eab581700 1 -- 172.21.15.7:0/508722554 shutdown_connections 2024-01-27T02:43:14.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.653+0000 7f1eab581700 1 -- 172.21.15.7:0/508722554 wait complete. 2024-01-27T02:43:14.669 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-27T02:41:56.176076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '163', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-01-27T02:41:56.620523+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '166', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-27T02:43:14.669 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:43:14.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.803+0000 7f8b76b15700 1 Processor -- start 2024-01-27T02:43:14.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.803+0000 7f8b76b15700 1 -- start start 2024-01-27T02:43:14.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.804+0000 7f8b76b15700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8b70143670 0x7f8b70063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:14.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.804+0000 7f8b76b15700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b700cc290 con 0x7f8b70143a40 2024-01-27T02:43:14.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.804+0000 7f8b76b15700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b700cb960 con 0x7f8b700aa8b0 2024-01-27T02:43:14.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.804+0000 7f8b76b15700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b700cbc40 con 0x7f8b70143670 2024-01-27T02:43:14.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.804+0000 7f8b750b2700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8b70143a40 0x7f8b700cc720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46710/0 (socket says 172.21.15.7:46710) 2024-01-27T02:43:14.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.804+0000 7f8b750b2700 1 -- 172.21.15.7:0/1329559308 learned_addr learned my addr 172.21.15.7:0/1329559308 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:14.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.804+0000 7f8b748b1700 1 --2- 172.21.15.7:0/1329559308 >> v2:172.21.15.188:3300/0 conn(0x7f8b70143670 0x7f8b70063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:14.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.804+0000 7f8b6f7fe700 1 -- 172.21.15.7:0/1329559308 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1545671828 0 0) 0x7f8b700cc290 con 0x7f8b70143a40 2024-01-27T02:43:14.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.804+0000 7f8b6f7fe700 1 -- 172.21.15.7:0/1329559308 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b60003570 con 0x7f8b70143a40 2024-01-27T02:43:14.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.804+0000 7f8b6f7fe700 1 -- 172.21.15.7:0/1329559308 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1974609677 0 0) 0x7f8b700cb960 con 0x7f8b700aa8b0 2024-01-27T02:43:14.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.804+0000 7f8b6f7fe700 1 -- 172.21.15.7:0/1329559308 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b700cc290 con 0x7f8b700aa8b0 2024-01-27T02:43:14.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.804+0000 7f8b6f7fe700 1 -- 172.21.15.7:0/1329559308 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3882437777 0 0) 0x7f8b60003570 con 0x7f8b70143a40 2024-01-27T02:43:14.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.804+0000 7f8b6f7fe700 1 -- 172.21.15.7:0/1329559308 >> v2:172.21.15.188:3300/0 conn(0x7f8b70143670 msgr2=0x7f8b70063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.804+0000 7f8b6f7fe700 1 --2- 172.21.15.7:0/1329559308 >> v2:172.21.15.188:3300/0 conn(0x7f8b70143670 0x7f8b70063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.804+0000 7f8b6f7fe700 1 -- 172.21.15.7:0/1329559308 >> v1:172.21.15.188:6789/0 conn(0x7f8b700aa8b0 legacy=0x7f8b700c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.804+0000 7f8b6f7fe700 1 -- 172.21.15.7:0/1329559308 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b700cbf20 con 0x7f8b70143a40 2024-01-27T02:43:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.804+0000 7f8b6f7fe700 1 -- 172.21.15.7:0/1329559308 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f8b680017f0 con 0x7f8b70143a40 2024-01-27T02:43:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.804+0000 7f8b748b1700 1 --2- 172.21.15.7:0/1329559308 >> v2:172.21.15.188:3300/0 conn(0x7f8b70143670 0x7f8b70063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:43:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.804+0000 7f8b6f7fe700 1 -- 172.21.15.7:0/1329559308 >> v1:172.21.15.7:6789/0 conn(0x7f8b70143a40 legacy=0x7f8b700cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.804+0000 7f8b6f7fe700 1 --2- 172.21.15.7:0/1329559308 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b60003bf0 0x7f8b60003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.804+0000 7f8b6f7fe700 1 --2- 172.21.15.7:0/1329559308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b60004810 0x7f8b60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.805+0000 7f8b6f7fe700 1 --2- 172.21.15.7:0/1329559308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b600051c0 0x7f8b6000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.805+0000 7f8b6f7fe700 1 -- 172.21.15.7:0/1329559308 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700cbc40 con 0x7f8b600051c0 2024-01-27T02:43:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.805+0000 7f8b6f7fe700 1 -- 172.21.15.7:0/1329559308 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6000c0b0 con 0x7f8b60004810 2024-01-27T02:43:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.805+0000 7f8b6f7fe700 1 -- 172.21.15.7:0/1329559308 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6000c270 con 0x7f8b60003bf0 2024-01-27T02:43:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.805+0000 7f8b750b2700 1 --2- 172.21.15.7:0/1329559308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b600051c0 0x7f8b6000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.805+0000 7f8b748b1700 1 --2- 172.21.15.7:0/1329559308 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b60003bf0 0x7f8b60003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.805+0000 7f8b6ffff700 1 --2- 172.21.15.7:0/1329559308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b60004810 0x7f8b60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.805+0000 7f8b750b2700 1 -- 172.21.15.7:0/1329559308 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b60003bf0 msgr2=0x7f8b60003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.805+0000 7f8b750b2700 1 --2- 172.21.15.7:0/1329559308 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b60003bf0 0x7f8b60003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.805+0000 7f8b750b2700 1 -- 172.21.15.7:0/1329559308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b60004810 msgr2=0x7f8b60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.805+0000 7f8b750b2700 1 --2- 172.21.15.7:0/1329559308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b60004810 0x7f8b60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.805+0000 7f8b750b2700 1 -- 172.21.15.7:0/1329559308 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b60003570 con 0x7f8b600051c0 2024-01-27T02:43:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.805+0000 7f8b748b1700 1 --2- 172.21.15.7:0/1329559308 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b60003bf0 0x7f8b60003fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:43:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.805+0000 7f8b750b2700 1 -- 172.21.15.7:0/1329559308 reap_dead start 2024-01-27T02:43:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.805+0000 7f8b750b2700 1 --2- 172.21.15.7:0/1329559308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b600051c0 0x7f8b6000ba60 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800a8d0 tx=0x7f8b68002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c5fb5322d877db server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.805+0000 7f8b6f7fe700 1 -- 172.21.15.7:0/1329559308 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b680070d0 con 0x7f8b600051c0 2024-01-27T02:43:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.805+0000 7f8b6f7fe700 1 -- 172.21.15.7:0/1329559308 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b68007270 con 0x7f8b600051c0 2024-01-27T02:43:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.805+0000 7f8b6f7fe700 1 -- 172.21.15.7:0/1329559308 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b68007630 con 0x7f8b600051c0 2024-01-27T02:43:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.806+0000 7f8b76b15700 1 -- 172.21.15.7:0/1329559308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b600051c0 msgr2=0x7f8b6000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:14.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.806+0000 7f8b76b15700 1 --2- 172.21.15.7:0/1329559308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b600051c0 0x7f8b6000ba60 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800a8d0 tx=0x7f8b68002190 comp rx=0 tx=0).stop 2024-01-27T02:43:14.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.806+0000 7f8b76b15700 1 -- 172.21.15.7:0/1329559308 shutdown_connections 2024-01-27T02:43:14.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.806+0000 7f8b76b15700 1 --2- 172.21.15.7:0/1329559308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b600051c0 0x7f8b6000ba60 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.806+0000 7f8b76b15700 1 -- 172.21.15.7:0/1329559308 >> 172.21.15.7:0/1329559308 conn(0x7f8b700cdbf0 msgr2=0x7f8b700cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:14.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.806+0000 7f8b76b15700 1 -- 172.21.15.7:0/1329559308 shutdown_connections 2024-01-27T02:43:14.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.806+0000 7f8b76b15700 1 -- 172.21.15.7:0/1329559308 wait complete. 2024-01-27T02:43:14.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.808+0000 7f8b76b15700 1 Processor -- start 2024-01-27T02:43:14.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.808+0000 7f8b76b15700 1 -- start start 2024-01-27T02:43:14.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.809+0000 7f8b76b15700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b600051c0 0x7f8b7006b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:14.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.809+0000 7f8b76b15700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b700c74d0 0x7f8b7015de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:14.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.809+0000 7f8b76b15700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b7015e3b0 0x7f8b7015e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:14.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.809+0000 7f8b76b15700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700c50b0 con 0x7f8b7015e3b0 2024-01-27T02:43:14.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.809+0000 7f8b76b15700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7015ee20 con 0x7f8b700c74d0 2024-01-27T02:43:14.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.809+0000 7f8b76b15700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7015f100 con 0x7f8b600051c0 2024-01-27T02:43:14.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.809+0000 7f8b750b2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b7015e3b0 0x7f8b7015e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:14.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.809+0000 7f8b750b2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b7015e3b0 0x7f8b7015e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43630/0 (socket says 172.21.15.7:43630) 2024-01-27T02:43:14.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.809+0000 7f8b750b2700 1 -- 172.21.15.7:0/1546770010 learned_addr learned my addr 172.21.15.7:0/1546770010 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:14.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.809+0000 7f8b6ffff700 1 --2- 172.21.15.7:0/1546770010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b700c74d0 0x7f8b7015de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:14.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.809+0000 7f8b748b1700 1 --2- 172.21.15.7:0/1546770010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b600051c0 0x7f8b7006b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:14.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.809+0000 7f8b750b2700 1 -- 172.21.15.7:0/1546770010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b600051c0 msgr2=0x7f8b7006b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:14.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.809+0000 7f8b750b2700 1 --2- 172.21.15.7:0/1546770010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b600051c0 0x7f8b7006b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.809+0000 7f8b750b2700 1 -- 172.21.15.7:0/1546770010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b700c74d0 msgr2=0x7f8b7015de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:14.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.809+0000 7f8b750b2700 1 --2- 172.21.15.7:0/1546770010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b700c74d0 0x7f8b7015de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:14.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.809+0000 7f8b750b2700 1 -- 172.21.15.7:0/1546770010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7015f3e0 con 0x7f8b7015e3b0 2024-01-27T02:43:14.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.809+0000 7f8b6ffff700 1 --2- 172.21.15.7:0/1546770010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b700c74d0 0x7f8b7015de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:14.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.810+0000 7f8b750b2700 1 --2- 172.21.15.7:0/1546770010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b7015e3b0 0x7f8b7015e820 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f8b700c9020 tx=0x7f8b68016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:14.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.810+0000 7f8b6cff9700 1 -- 172.21.15.7:0/1546770010 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b68004e90 con 0x7f8b7015e3b0 2024-01-27T02:43:14.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.810+0000 7f8b6cff9700 1 -- 172.21.15.7:0/1546770010 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6801d420 con 0x7f8b7015e3b0 2024-01-27T02:43:14.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.810+0000 7f8b76b15700 1 -- 172.21.15.7:0/1546770010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b700c4d40 con 0x7f8b7015e3b0 2024-01-27T02:43:14.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.810+0000 7f8b6cff9700 1 -- 172.21.15.7:0/1546770010 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6801d800 con 0x7f8b7015e3b0 2024-01-27T02:43:14.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.810+0000 7f8b76b15700 1 -- 172.21.15.7:0/1546770010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b700c7940 con 0x7f8b7015e3b0 2024-01-27T02:43:14.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.811+0000 7f8b76b15700 1 -- 172.21.15.7:0/1546770010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b70072060 con 0x7f8b7015e3b0 2024-01-27T02:43:14.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.811+0000 7f8b6cff9700 1 -- 172.21.15.7:0/1546770010 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8b6801dbb0 con 0x7f8b7015e3b0 2024-01-27T02:43:14.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.812+0000 7f8b6cff9700 1 --2- 172.21.15.7:0/1546770010 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b54066f30 0x7f8b540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:14.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.812+0000 7f8b6cff9700 1 -- 172.21.15.7:0/1546770010 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f8b68086070 con 0x7f8b7015e3b0 2024-01-27T02:43:14.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.812+0000 7f8b748b1700 1 --2- 172.21.15.7:0/1546770010 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b54066f30 0x7f8b540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:14.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.812+0000 7f8b6cff9700 1 -- 172.21.15.7:0/1546770010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=169}) v3 -- 0x7f8b5406eab0 con 0x7f8b7015e3b0 2024-01-27T02:43:14.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.812+0000 7f8b748b1700 1 --2- 172.21.15.7:0/1546770010 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b54066f30 0x7f8b540693e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8b64009170 tx=0x7f8b64008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:14.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:14.816+0000 7f8b6cff9700 1 -- 172.21.15.7:0/1546770010 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b68053220 con 0x7f8b7015e3b0 2024-01-27T02:43:15.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.018+0000 7f8b76b15700 1 -- 172.21.15.7:0/1546770010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b70063580 con 0x7f8b7015e3b0 2024-01-27T02:43:15.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.018+0000 7f8b6cff9700 1 -- 172.21.15.7:0/1546770010 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v168) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b68023030 con 0x7f8b7015e3b0 2024-01-27T02:43:15.021 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:43:15.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.021+0000 7f8b76b15700 1 -- 172.21.15.7:0/1546770010 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b54066f30 msgr2=0x7f8b540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:15.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.021+0000 7f8b76b15700 1 --2- 172.21.15.7:0/1546770010 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b54066f30 0x7f8b540693e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8b64009170 tx=0x7f8b64008040 comp rx=0 tx=0).stop 2024-01-27T02:43:15.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.021+0000 7f8b76b15700 1 -- 172.21.15.7:0/1546770010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b7015e3b0 msgr2=0x7f8b7015e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:15.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.021+0000 7f8b76b15700 1 --2- 172.21.15.7:0/1546770010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b7015e3b0 0x7f8b7015e820 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f8b700c9020 tx=0x7f8b68016000 comp rx=0 tx=0).stop 2024-01-27T02:43:15.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.021+0000 7f8b76b15700 1 -- 172.21.15.7:0/1546770010 shutdown_connections 2024-01-27T02:43:15.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.021+0000 7f8b76b15700 1 --2- 172.21.15.7:0/1546770010 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b54066f30 0x7f8b540693e0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.021+0000 7f8b76b15700 1 --2- 172.21.15.7:0/1546770010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b7015e3b0 0x7f8b7015e820 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.021+0000 7f8b76b15700 1 --2- 172.21.15.7:0/1546770010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b600051c0 0x7f8b7006b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.021+0000 7f8b76b15700 1 --2- 172.21.15.7:0/1546770010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b700c74d0 0x7f8b7015de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.021+0000 7f8b76b15700 1 -- 172.21.15.7:0/1546770010 >> 172.21.15.7:0/1546770010 conn(0x7f8b700cdbf0 msgr2=0x7f8b70142420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:15.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.021+0000 7f8b76b15700 1 -- 172.21.15.7:0/1546770010 shutdown_connections 2024-01-27T02:43:15.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.021+0000 7f8b76b15700 1 -- 172.21.15.7:0/1546770010 wait complete. 2024-01-27T02:43:15.036 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-27T02:43:15.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.169+0000 7f07f9775700 1 Processor -- start 2024-01-27T02:43:15.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.169+0000 7f07f9775700 1 -- start start 2024-01-27T02:43:15.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.169+0000 7f07f9775700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f07f406bbf0 0x7f07f40ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:15.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.169+0000 7f07f9775700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07f40679b0 con 0x7f07f40cba60 2024-01-27T02:43:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.169+0000 7f07f9775700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07f4067c90 con 0x7f07f40cbe30 2024-01-27T02:43:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.169+0000 7f07f9775700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07f40cca50 con 0x7f07f406bbf0 2024-01-27T02:43:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.170+0000 7f07f37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f07f406bbf0 0x7f07f40ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.170+0000 7f07f2ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f07f40cba60 0x7f07f415b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46714/0 (socket says 172.21.15.7:46714) 2024-01-27T02:43:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.170+0000 7f07f2ffd700 1 -- 172.21.15.7:0/2667346256 learned_addr learned my addr 172.21.15.7:0/2667346256 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.170+0000 7f07f1ffb700 1 -- 172.21.15.7:0/2667346256 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507670375 0 0) 0x7f07f40679b0 con 0x7f07f40cba60 2024-01-27T02:43:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.170+0000 7f07f1ffb700 1 -- 172.21.15.7:0/2667346256 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07dc003570 con 0x7f07f40cba60 2024-01-27T02:43:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.170+0000 7f07f37fe700 1 -- 172.21.15.7:0/2667346256 >> v1:172.21.15.188:6789/0 conn(0x7f07f40cbe30 legacy=0x7f07f406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.170+0000 7f07f37fe700 1 -- 172.21.15.7:0/2667346256 >> v1:172.21.15.7:6789/0 conn(0x7f07f40cba60 legacy=0x7f07f415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.170+0000 7f07f37fe700 1 -- 172.21.15.7:0/2667346256 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f40ccd30 con 0x7f07f406bbf0 2024-01-27T02:43:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.170+0000 7f07f37fe700 1 --2- 172.21.15.7:0/2667346256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07f406bbf0 0x7f07f40ce410 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009e60 tx=0x7f07ec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e345ac765c1fb0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.170+0000 7f07f1ffb700 1 -- 172.21.15.7:0/2667346256 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec00e070 con 0x7f07f406bbf0 2024-01-27T02:43:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.171+0000 7f07f1ffb700 1 -- 172.21.15.7:0/2667346256 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07ec00bb60 con 0x7f07f406bbf0 2024-01-27T02:43:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.171+0000 7f07f1ffb700 1 -- 172.21.15.7:0/2667346256 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec012430 con 0x7f07f406bbf0 2024-01-27T02:43:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.171+0000 7f07f9775700 1 -- 172.21.15.7:0/2667346256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07f406bbf0 msgr2=0x7f07f40ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.171+0000 7f07f9775700 1 --2- 172.21.15.7:0/2667346256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07f406bbf0 0x7f07f40ce410 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009e60 tx=0x7f07ec009780 comp rx=0 tx=0).stop 2024-01-27T02:43:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.171+0000 7f07f9775700 1 -- 172.21.15.7:0/2667346256 shutdown_connections 2024-01-27T02:43:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.171+0000 7f07f9775700 1 --2- 172.21.15.7:0/2667346256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07f406bbf0 0x7f07f40ce410 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.171+0000 7f07f9775700 1 -- 172.21.15.7:0/2667346256 >> 172.21.15.7:0/2667346256 conn(0x7f07f4063580 msgr2=0x7f07f4063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.171+0000 7f07f9775700 1 -- 172.21.15.7:0/2667346256 shutdown_connections 2024-01-27T02:43:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.171+0000 7f07f9775700 1 -- 172.21.15.7:0/2667346256 wait complete. 2024-01-27T02:43:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.173+0000 7f07f9775700 1 Processor -- start 2024-01-27T02:43:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.173+0000 7f07f9775700 1 -- start start 2024-01-27T02:43:15.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.174+0000 7f07f9775700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07f406bbf0 0x7f07f40ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:15.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.174+0000 7f07f9775700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07f40cba60 0x7f07f416dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:15.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.174+0000 7f07f9775700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07f40cbe30 0x7f07f40b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:15.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.174+0000 7f07f9775700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f40ac210 con 0x7f07f40cba60 2024-01-27T02:43:15.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.174+0000 7f07f9775700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f40ac4f0 con 0x7f07f406bbf0 2024-01-27T02:43:15.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.174+0000 7f07f9775700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f416e1a0 con 0x7f07f40cbe30 2024-01-27T02:43:15.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.174+0000 7f07f27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07f40cba60 0x7f07f416dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:15.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.174+0000 7f07f27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07f40cba60 0x7f07f416dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43658/0 (socket says 172.21.15.7:43658) 2024-01-27T02:43:15.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.174+0000 7f07f27fc700 1 -- 172.21.15.7:0/1319719675 learned_addr learned my addr 172.21.15.7:0/1319719675 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:15.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.174+0000 7f07f2ffd700 1 --2- 172.21.15.7:0/1319719675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07f406bbf0 0x7f07f40ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:15.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.174+0000 7f07f37fe700 1 --2- 172.21.15.7:0/1319719675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07f40cbe30 0x7f07f40b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:15.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.175+0000 7f07f27fc700 1 -- 172.21.15.7:0/1319719675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07f40cbe30 msgr2=0x7f07f40b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:15.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.175+0000 7f07f27fc700 1 --2- 172.21.15.7:0/1319719675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07f40cbe30 0x7f07f40b0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.175+0000 7f07f27fc700 1 -- 172.21.15.7:0/1319719675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07f406bbf0 msgr2=0x7f07f40ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:15.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.175+0000 7f07f27fc700 1 --2- 172.21.15.7:0/1319719675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07f406bbf0 0x7f07f40ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.175+0000 7f07f27fc700 1 -- 172.21.15.7:0/1319719675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ec009a20 con 0x7f07f40cba60 2024-01-27T02:43:15.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.175+0000 7f07f27fc700 1 --2- 172.21.15.7:0/1319719675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07f40cba60 0x7f07f416dc60 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f07e800c5c0 tx=0x7f07e8007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:15.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.175+0000 7f07daffd700 1 -- 172.21.15.7:0/1319719675 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e8011590 con 0x7f07f40cba60 2024-01-27T02:43:15.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.175+0000 7f07daffd700 1 -- 172.21.15.7:0/1319719675 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e8011710 con 0x7f07f40cba60 2024-01-27T02:43:15.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.175+0000 7f07daffd700 1 -- 172.21.15.7:0/1319719675 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e8011ad0 con 0x7f07f40cba60 2024-01-27T02:43:15.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.175+0000 7f07f9775700 1 -- 172.21.15.7:0/1319719675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f416e440 con 0x7f07f40cba60 2024-01-27T02:43:15.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.175+0000 7f07f9775700 1 -- 172.21.15.7:0/1319719675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f416ea00 con 0x7f07f40cba60 2024-01-27T02:43:15.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.176+0000 7f07f9775700 1 -- 172.21.15.7:0/1319719675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f4072060 con 0x7f07f40cba60 2024-01-27T02:43:15.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.176+0000 7f07daffd700 1 -- 172.21.15.7:0/1319719675 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f07e801a360 con 0x7f07f40cba60 2024-01-27T02:43:15.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.177+0000 7f07daffd700 1 --2- 172.21.15.7:0/1319719675 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f07e0066f30 0x7f07e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:15.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.177+0000 7f07daffd700 1 -- 172.21.15.7:0/1319719675 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f07e8083510 con 0x7f07f40cba60 2024-01-27T02:43:15.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.177+0000 7f07daffd700 1 -- 172.21.15.7:0/1319719675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=169}) v3 -- 0x7f07e006eab0 con 0x7f07f40cba60 2024-01-27T02:43:15.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.177+0000 7f07f2ffd700 1 --2- 172.21.15.7:0/1319719675 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f07e0066f30 0x7f07e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:15.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.178+0000 7f07f2ffd700 1 --2- 172.21.15.7:0/1319719675 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f07e0066f30 0x7f07e00693e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f07e4001360 tx=0x7f07e4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:15.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.181+0000 7f07daffd700 1 -- 172.21.15.7:0/1319719675 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f07e8050760 con 0x7f07f40cba60 2024-01-27T02:43:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.383+0000 7f07f9775700 1 -- 172.21.15.7:0/1319719675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f07f40ac630 con 0x7f07f40cba60 2024-01-27T02:43:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.384+0000 7f07daffd700 1 -- 172.21.15.7:0/1319719675 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v168) v1 ==== 104+0+11 (secure 0 0 0) 0x7f07e8017070 con 0x7f07f40cba60 2024-01-27T02:43:15.386 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-27T02:43:15.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.386+0000 7f07f9775700 1 -- 172.21.15.7:0/1319719675 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f07e0066f30 msgr2=0x7f07e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:15.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.386+0000 7f07f9775700 1 --2- 172.21.15.7:0/1319719675 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f07e0066f30 0x7f07e00693e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f07e4001360 tx=0x7f07e4009040 comp rx=0 tx=0).stop 2024-01-27T02:43:15.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.386+0000 7f07f9775700 1 -- 172.21.15.7:0/1319719675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07f40cba60 msgr2=0x7f07f416dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:15.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.386+0000 7f07f9775700 1 --2- 172.21.15.7:0/1319719675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07f40cba60 0x7f07f416dc60 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f07e800c5c0 tx=0x7f07e8007f50 comp rx=0 tx=0).stop 2024-01-27T02:43:15.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.386+0000 7f07f9775700 1 -- 172.21.15.7:0/1319719675 shutdown_connections 2024-01-27T02:43:15.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.386+0000 7f07f9775700 1 --2- 172.21.15.7:0/1319719675 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f07e0066f30 0x7f07e00693e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.386+0000 7f07f9775700 1 --2- 172.21.15.7:0/1319719675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07f40cbe30 0x7f07f40b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.386+0000 7f07f9775700 1 --2- 172.21.15.7:0/1319719675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07f406bbf0 0x7f07f40ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.386+0000 7f07f9775700 1 --2- 172.21.15.7:0/1319719675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07f40cba60 0x7f07f416dc60 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.386+0000 7f07f9775700 1 -- 172.21.15.7:0/1319719675 >> 172.21.15.7:0/1319719675 conn(0x7f07f4063580 msgr2=0x7f07f4063950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:15.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.386+0000 7f07f9775700 1 -- 172.21.15.7:0/1319719675 shutdown_connections 2024-01-27T02:43:15.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.386+0000 7f07f9775700 1 -- 172.21.15.7:0/1319719675 wait complete. 2024-01-27T02:43:15.401 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-27T02:43:15.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.535+0000 7f5b757d7700 1 Processor -- start 2024-01-27T02:43:15.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.535+0000 7f5b757d7700 1 -- start start 2024-01-27T02:43:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.536+0000 7f5b757d7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b70154600 0x7f5b700c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.536+0000 7f5b757d7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b7015b5f0 con 0x7f5b701549d0 2024-01-27T02:43:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.536+0000 7f5b757d7700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b7015aa20 con 0x7f5b700c9a10 2024-01-27T02:43:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.536+0000 7f5b757d7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7006dc80 con 0x7f5b70154600 2024-01-27T02:43:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.536+0000 7f5b6e7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5b701549d0 0x7f5b700c7f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46718/0 (socket says 172.21.15.7:46718) 2024-01-27T02:43:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.536+0000 7f5b6e7fc700 1 -- 172.21.15.7:0/940542073 learned_addr learned my addr 172.21.15.7:0/940542073 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.536+0000 7f5b6effd700 1 --2- 172.21.15.7:0/940542073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b70154600 0x7f5b700c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.536+0000 7f5b6dffb700 1 -- 172.21.15.7:0/940542073 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839843461 0 0) 0x7f5b7015b5f0 con 0x7f5b701549d0 2024-01-27T02:43:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.536+0000 7f5b6dffb700 1 -- 172.21.15.7:0/940542073 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b58003570 con 0x7f5b701549d0 2024-01-27T02:43:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.536+0000 7f5b6dffb700 1 -- 172.21.15.7:0/940542073 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2857198825 0 0) 0x7f5b7015aa20 con 0x7f5b700c9a10 2024-01-27T02:43:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.536+0000 7f5b6dffb700 1 -- 172.21.15.7:0/940542073 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b7015b5f0 con 0x7f5b700c9a10 2024-01-27T02:43:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.536+0000 7f5b6effd700 1 -- 172.21.15.7:0/940542073 >> v1:172.21.15.188:6789/0 conn(0x7f5b700c9a10 legacy=0x7f5b700cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.536+0000 7f5b6effd700 1 -- 172.21.15.7:0/940542073 >> v1:172.21.15.7:6789/0 conn(0x7f5b701549d0 legacy=0x7f5b700c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.536+0000 7f5b6effd700 1 -- 172.21.15.7:0/940542073 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7015acc0 con 0x7f5b70154600 2024-01-27T02:43:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.537+0000 7f5b6effd700 1 --2- 172.21.15.7:0/940542073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b70154600 0x7f5b700c94d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009910 tx=0x7f5b600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4acadbe26d29ca4a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.537+0000 7f5b6dffb700 1 -- 172.21.15.7:0/940542073 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b60011630 con 0x7f5b70154600 2024-01-27T02:43:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.537+0000 7f5b6dffb700 1 -- 172.21.15.7:0/940542073 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b60011790 con 0x7f5b70154600 2024-01-27T02:43:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.537+0000 7f5b6dffb700 1 -- 172.21.15.7:0/940542073 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b60011b30 con 0x7f5b70154600 2024-01-27T02:43:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.537+0000 7f5b757d7700 1 -- 172.21.15.7:0/940542073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b70154600 msgr2=0x7f5b700c94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.537+0000 7f5b757d7700 1 --2- 172.21.15.7:0/940542073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b70154600 0x7f5b700c94d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009910 tx=0x7f5b600091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.537+0000 7f5b757d7700 1 -- 172.21.15.7:0/940542073 shutdown_connections 2024-01-27T02:43:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.537+0000 7f5b757d7700 1 --2- 172.21.15.7:0/940542073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b70154600 0x7f5b700c94d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.537+0000 7f5b757d7700 1 -- 172.21.15.7:0/940542073 >> 172.21.15.7:0/940542073 conn(0x7f5b70063580 msgr2=0x7f5b70063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.537+0000 7f5b757d7700 1 -- 172.21.15.7:0/940542073 shutdown_connections 2024-01-27T02:43:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.537+0000 7f5b757d7700 1 -- 172.21.15.7:0/940542073 wait complete. 2024-01-27T02:43:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.539+0000 7f5b757d7700 1 Processor -- start 2024-01-27T02:43:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.539+0000 7f5b757d7700 1 -- start start 2024-01-27T02:43:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.540+0000 7f5b757d7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b700c9a10 0x7f5b700aead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.540+0000 7f5b757d7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b70154600 0x7f5b70159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.540+0000 7f5b757d7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b701549d0 0x7f5b700ac990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.540+0000 7f5b757d7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b70168250 con 0x7f5b70154600 2024-01-27T02:43:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.540+0000 7f5b757d7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b701599c0 con 0x7f5b701549d0 2024-01-27T02:43:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.540+0000 7f5b757d7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b70158920 con 0x7f5b700c9a10 2024-01-27T02:43:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.540+0000 7f5b6e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b70154600 0x7f5b70159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.540+0000 7f5b6e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b70154600 0x7f5b70159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43670/0 (socket says 172.21.15.7:43670) 2024-01-27T02:43:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.540+0000 7f5b6e7fc700 1 -- 172.21.15.7:0/1660154395 learned_addr learned my addr 172.21.15.7:0/1660154395 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.540+0000 7f5b6effd700 1 --2- 172.21.15.7:0/1660154395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b700c9a10 0x7f5b700aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.540+0000 7f5b6f7fe700 1 --2- 172.21.15.7:0/1660154395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b701549d0 0x7f5b700ac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.540+0000 7f5b6e7fc700 1 -- 172.21.15.7:0/1660154395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b700c9a10 msgr2=0x7f5b700aead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.540+0000 7f5b6e7fc700 1 --2- 172.21.15.7:0/1660154395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b700c9a10 0x7f5b700aead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.540+0000 7f5b6e7fc700 1 -- 172.21.15.7:0/1660154395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b701549d0 msgr2=0x7f5b700ac990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.540+0000 7f5b6e7fc700 1 --2- 172.21.15.7:0/1660154395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b701549d0 0x7f5b700ac990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.540+0000 7f5b6e7fc700 1 -- 172.21.15.7:0/1660154395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b60009420 con 0x7f5b70154600 2024-01-27T02:43:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.540+0000 7f5b6e7fc700 1 --2- 172.21.15.7:0/1660154395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b70154600 0x7f5b70159ed0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400c790 tx=0x7f5b64007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.541+0000 7f5b56ffd700 1 -- 172.21.15.7:0/1660154395 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b64005b30 con 0x7f5b70154600 2024-01-27T02:43:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.541+0000 7f5b56ffd700 1 -- 172.21.15.7:0/1660154395 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b64005c90 con 0x7f5b70154600 2024-01-27T02:43:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.541+0000 7f5b757d7700 1 -- 172.21.15.7:0/1660154395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b70158c00 con 0x7f5b70154600 2024-01-27T02:43:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.541+0000 7f5b56ffd700 1 -- 172.21.15.7:0/1660154395 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b640072e0 con 0x7f5b70154600 2024-01-27T02:43:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.541+0000 7f5b757d7700 1 -- 172.21.15.7:0/1660154395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b701591c0 con 0x7f5b70154600 2024-01-27T02:43:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.542+0000 7f5b757d7700 1 -- 172.21.15.7:0/1660154395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b70072060 con 0x7f5b70154600 2024-01-27T02:43:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.542+0000 7f5b56ffd700 1 -- 172.21.15.7:0/1660154395 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5b64007690 con 0x7f5b70154600 2024-01-27T02:43:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.542+0000 7f5b56ffd700 1 --2- 172.21.15.7:0/1660154395 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5b5c066f30 0x7f5b5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:15.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.543+0000 7f5b56ffd700 1 -- 172.21.15.7:0/1660154395 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f5b64083980 con 0x7f5b70154600 2024-01-27T02:43:15.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.543+0000 7f5b56ffd700 1 -- 172.21.15.7:0/1660154395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=169}) v3 -- 0x7f5b5c06eab0 con 0x7f5b70154600 2024-01-27T02:43:15.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.543+0000 7f5b6effd700 1 --2- 172.21.15.7:0/1660154395 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5b5c066f30 0x7f5b5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:15.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.543+0000 7f5b6effd700 1 --2- 172.21.15.7:0/1660154395 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5b5c066f30 0x7f5b5c0693e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009860 tx=0x7f5b60017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:15.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.547+0000 7f5b56ffd700 1 -- 172.21.15.7:0/1660154395 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b64017020 con 0x7f5b70154600 2024-01-27T02:43:15.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.750+0000 7f5b757d7700 1 -- 172.21.15.7:0/1660154395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5b700cb0d0 con 0x7f5b70154600 2024-01-27T02:43:15.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.751+0000 7f5b56ffd700 1 -- 172.21.15.7:0/1660154395 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v168) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5b64050c80 con 0x7f5b70154600 2024-01-27T02:43:15.752 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-27T02:43:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.753+0000 7f5b757d7700 1 -- 172.21.15.7:0/1660154395 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5b5c066f30 msgr2=0x7f5b5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.753+0000 7f5b757d7700 1 --2- 172.21.15.7:0/1660154395 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5b5c066f30 0x7f5b5c0693e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009860 tx=0x7f5b60017000 comp rx=0 tx=0).stop 2024-01-27T02:43:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.753+0000 7f5b757d7700 1 -- 172.21.15.7:0/1660154395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b70154600 msgr2=0x7f5b70159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.753+0000 7f5b757d7700 1 --2- 172.21.15.7:0/1660154395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b70154600 0x7f5b70159ed0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400c790 tx=0x7f5b64007e40 comp rx=0 tx=0).stop 2024-01-27T02:43:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.753+0000 7f5b757d7700 1 -- 172.21.15.7:0/1660154395 shutdown_connections 2024-01-27T02:43:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.753+0000 7f5b757d7700 1 --2- 172.21.15.7:0/1660154395 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5b5c066f30 0x7f5b5c0693e0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.753+0000 7f5b757d7700 1 --2- 172.21.15.7:0/1660154395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b701549d0 0x7f5b700ac990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.753+0000 7f5b757d7700 1 --2- 172.21.15.7:0/1660154395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b700c9a10 0x7f5b700aead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.753+0000 7f5b757d7700 1 --2- 172.21.15.7:0/1660154395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b70154600 0x7f5b70159ed0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.753+0000 7f5b757d7700 1 -- 172.21.15.7:0/1660154395 >> 172.21.15.7:0/1660154395 conn(0x7f5b70063580 msgr2=0x7f5b70152b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.753+0000 7f5b757d7700 1 -- 172.21.15.7:0/1660154395 shutdown_connections 2024-01-27T02:43:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.753+0000 7f5b757d7700 1 -- 172.21.15.7:0/1660154395 wait complete. 2024-01-27T02:43:15.768 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:43:15.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.899+0000 7fa0814d6700 1 Processor -- start 2024-01-27T02:43:15.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.899+0000 7fa0814d6700 1 -- start start 2024-01-27T02:43:15.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.900+0000 7fa0814d6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa07c157ca0 0x7fa07c0c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:15.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.900+0000 7fa0814d6700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa07c0cdbf0 0x7fa07c0ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:15.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.900+0000 7fa0814d6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa07c0ca020 con 0x7fa07c0c3ae0 2024-01-27T02:43:15.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.900+0000 7fa0814d6700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa07c0c4f40 con 0x7fa07c0cdbf0 2024-01-27T02:43:15.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.900+0000 7fa0814d6700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c0c5210 con 0x7fa07c157ca0 2024-01-27T02:43:15.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.900+0000 7fa07a7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa07c0c3ae0 0x7fa07c0cd4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46720/0 (socket says 172.21.15.7:46720) 2024-01-27T02:43:15.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.900+0000 7fa07a7fc700 1 -- 172.21.15.7:0/2002794886 learned_addr learned my addr 172.21.15.7:0/2002794886 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:15.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.901+0000 7fa07affd700 1 --2- 172.21.15.7:0/2002794886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa07c157ca0 0x7fa07c0c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:15.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.901+0000 7fa07b7fe700 1 --2- 172.21.15.7:0/2002794886 >> v2:172.21.15.188:3300/0 conn(0x7fa07c0cdbf0 0x7fa07c0ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:15.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.901+0000 7fa079ffb700 1 -- 172.21.15.7:0/2002794886 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3031642964 0 0) 0x7fa07c0ca020 con 0x7fa07c0c3ae0 2024-01-27T02:43:15.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.901+0000 7fa079ffb700 1 -- 172.21.15.7:0/2002794886 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa064003570 con 0x7fa07c0c3ae0 2024-01-27T02:43:15.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.901+0000 7fa07affd700 1 -- 172.21.15.7:0/2002794886 >> v2:172.21.15.188:3300/0 conn(0x7fa07c0cdbf0 msgr2=0x7fa07c0ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:15.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.901+0000 7fa07affd700 1 --2- 172.21.15.7:0/2002794886 >> v2:172.21.15.188:3300/0 conn(0x7fa07c0cdbf0 0x7fa07c0ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.901+0000 7fa07affd700 1 -- 172.21.15.7:0/2002794886 >> v1:172.21.15.7:6789/0 conn(0x7fa07c0c3ae0 legacy=0x7fa07c0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:15.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.901+0000 7fa07affd700 1 -- 172.21.15.7:0/2002794886 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07c0c54f0 con 0x7fa07c157ca0 2024-01-27T02:43:15.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.901+0000 7fa07b7fe700 1 --2- 172.21.15.7:0/2002794886 >> v2:172.21.15.188:3300/0 conn(0x7fa07c0cdbf0 0x7fa07c0ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:43:15.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.901+0000 7fa07affd700 1 --2- 172.21.15.7:0/2002794886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa07c157ca0 0x7fa07c0c4a00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0098e0 tx=0x7fa06c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=108329aae43cdf05 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:15.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.902+0000 7fa079ffb700 1 -- 172.21.15.7:0/2002794886 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c011630 con 0x7fa07c157ca0 2024-01-27T02:43:15.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.902+0000 7fa079ffb700 1 -- 172.21.15.7:0/2002794886 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06c011790 con 0x7fa07c157ca0 2024-01-27T02:43:15.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.902+0000 7fa079ffb700 1 -- 172.21.15.7:0/2002794886 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c011b30 con 0x7fa07c157ca0 2024-01-27T02:43:15.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.902+0000 7fa0814d6700 1 -- 172.21.15.7:0/2002794886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa07c157ca0 msgr2=0x7fa07c0c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:15.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.902+0000 7fa0814d6700 1 --2- 172.21.15.7:0/2002794886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa07c157ca0 0x7fa07c0c4a00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0098e0 tx=0x7fa06c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.902+0000 7fa0814d6700 1 -- 172.21.15.7:0/2002794886 shutdown_connections 2024-01-27T02:43:15.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.902+0000 7fa0814d6700 1 --2- 172.21.15.7:0/2002794886 >> v2:172.21.15.188:3300/0 conn(0x7fa07c0cdbf0 0x7fa07c0ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.902+0000 7fa0814d6700 1 --2- 172.21.15.7:0/2002794886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa07c157ca0 0x7fa07c0c4a00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.902+0000 7fa0814d6700 1 -- 172.21.15.7:0/2002794886 >> 172.21.15.7:0/2002794886 conn(0x7fa07c098020 msgr2=0x7fa07c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:15.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.902+0000 7fa0814d6700 1 -- 172.21.15.7:0/2002794886 shutdown_connections 2024-01-27T02:43:15.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.902+0000 7fa0814d6700 1 -- 172.21.15.7:0/2002794886 wait complete. 2024-01-27T02:43:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.904+0000 7fa0814d6700 1 Processor -- start 2024-01-27T02:43:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.904+0000 7fa0814d6700 1 -- start start 2024-01-27T02:43:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.904+0000 7fa0814d6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa07c0c3ae0 0x7fa07c1545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.904+0000 7fa0814d6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa07c0cdbf0 0x7fa07c0cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.904+0000 7fa0814d6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa07c157ca0 0x7fa07c0cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.904+0000 7fa0814d6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c0c6f00 con 0x7fa07c0cdbf0 2024-01-27T02:43:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.904+0000 7fa0814d6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c157090 con 0x7fa07c157ca0 2024-01-27T02:43:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.904+0000 7fa0814d6700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c157370 con 0x7fa07c0c3ae0 2024-01-27T02:43:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.905+0000 7fa07a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa07c0cdbf0 0x7fa07c0cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.905+0000 7fa07a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa07c0cdbf0 0x7fa07c0cb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43672/0 (socket says 172.21.15.7:43672) 2024-01-27T02:43:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.905+0000 7fa07a7fc700 1 -- 172.21.15.7:0/1701913024 learned_addr learned my addr 172.21.15.7:0/1701913024 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.905+0000 7fa07b7fe700 1 --2- 172.21.15.7:0/1701913024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa07c157ca0 0x7fa07c0cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.905+0000 7fa07affd700 1 --2- 172.21.15.7:0/1701913024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa07c0c3ae0 0x7fa07c1545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.905+0000 7fa07a7fc700 1 -- 172.21.15.7:0/1701913024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa07c0c3ae0 msgr2=0x7fa07c1545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.905+0000 7fa07a7fc700 1 --2- 172.21.15.7:0/1701913024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa07c0c3ae0 0x7fa07c1545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.905+0000 7fa07a7fc700 1 -- 172.21.15.7:0/1701913024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa07c157ca0 msgr2=0x7fa07c0cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.905+0000 7fa07a7fc700 1 --2- 172.21.15.7:0/1701913024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa07c157ca0 0x7fa07c0cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.905+0000 7fa07a7fc700 1 -- 172.21.15.7:0/1701913024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa074009a20 con 0x7fa07c0cdbf0 2024-01-27T02:43:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.905+0000 7fa07a7fc700 1 --2- 172.21.15.7:0/1701913024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa07c0cdbf0 0x7fa07c0cb200 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fa07000c7c0 tx=0x7fa070007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.905+0000 7fa062ffd700 1 -- 172.21.15.7:0/1701913024 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070005b30 con 0x7fa07c0cdbf0 2024-01-27T02:43:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.905+0000 7fa062ffd700 1 -- 172.21.15.7:0/1701913024 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa070005cb0 con 0x7fa07c0cdbf0 2024-01-27T02:43:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.905+0000 7fa062ffd700 1 -- 172.21.15.7:0/1701913024 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070007320 con 0x7fa07c0cdbf0 2024-01-27T02:43:15.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.905+0000 7fa0814d6700 1 -- 172.21.15.7:0/1701913024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06c009420 con 0x7fa07c0cdbf0 2024-01-27T02:43:15.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.905+0000 7fa0814d6700 1 -- 172.21.15.7:0/1701913024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa07c0c6b90 con 0x7fa07c0cdbf0 2024-01-27T02:43:15.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.906+0000 7fa0814d6700 1 -- 172.21.15.7:0/1701913024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa07c0c70b0 con 0x7fa07c0cdbf0 2024-01-27T02:43:15.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.907+0000 7fa062ffd700 1 -- 172.21.15.7:0/1701913024 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa07001b3b0 con 0x7fa07c0cdbf0 2024-01-27T02:43:15.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.907+0000 7fa062ffd700 1 --2- 172.21.15.7:0/1701913024 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa068066f30 0x7fa0680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:15.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.907+0000 7fa062ffd700 1 -- 172.21.15.7:0/1701913024 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5734+0+0 (secure 0 0 0) 0x7fa070083990 con 0x7fa07c0cdbf0 2024-01-27T02:43:15.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.908+0000 7fa062ffd700 1 -- 172.21.15.7:0/1701913024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=169}) v3 -- 0x7fa06806eab0 con 0x7fa07c0cdbf0 2024-01-27T02:43:15.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.908+0000 7fa07affd700 1 --2- 172.21.15.7:0/1701913024 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa068066f30 0x7fa0680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:15.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.908+0000 7fa07affd700 1 --2- 172.21.15.7:0/1701913024 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa068066f30 0x7fa0680693e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0091e0 tx=0x7fa06c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:15.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:15.912+0000 7fa062ffd700 1 -- 172.21.15.7:0/1701913024 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa070017020 con 0x7fa07c0cdbf0 2024-01-27T02:43:16.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.161+0000 7fa0814d6700 1 -- 172.21.15.7:0/1701913024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa07c04c710 con 0x7fa07c0cdbf0 2024-01-27T02:43:16.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.162+0000 7fa062ffd700 1 -- 172.21.15.7:0/1701913024 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fa07c0c70b0 con 0x7fa07c0cdbf0 2024-01-27T02:43:16.163 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:16.164 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6473,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/3449593528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3449593528},{"type":"v1","addr":"172.21.15.188:6836","nonce":3449593528}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3473642033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3473642033},{"type":"v1","addr":"172.21.15.188:6837","nonce":3473642033}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6479,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2550009140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2550009140},{"type":"v1","addr":"172.21.15.7:6833","nonce":2550009140}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:41:57.646012+0000","modified":"2024-01-27T02:42:01.251419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6485":{"gid":6485,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3213683076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3213683076},{"type":"v1","addr":"172.21.15.7:6835","nonce":3213683076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2024-01-27T02:43:16.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.164+0000 7fa0814d6700 1 -- 172.21.15.7:0/1701913024 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa068066f30 msgr2=0x7fa0680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:16.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.164+0000 7fa0814d6700 1 --2- 172.21.15.7:0/1701913024 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa068066f30 0x7fa0680693e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0091e0 tx=0x7fa06c017000 comp rx=0 tx=0).stop 2024-01-27T02:43:16.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.164+0000 7fa0814d6700 1 -- 172.21.15.7:0/1701913024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa07c0cdbf0 msgr2=0x7fa07c0cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:16.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.164+0000 7fa0814d6700 1 --2- 172.21.15.7:0/1701913024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa07c0cdbf0 0x7fa07c0cb200 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fa07000c7c0 tx=0x7fa070007e00 comp rx=0 tx=0).stop 2024-01-27T02:43:16.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.164+0000 7fa0814d6700 1 -- 172.21.15.7:0/1701913024 shutdown_connections 2024-01-27T02:43:16.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.164+0000 7fa0814d6700 1 --2- 172.21.15.7:0/1701913024 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa068066f30 0x7fa0680693e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.164+0000 7fa0814d6700 1 --2- 172.21.15.7:0/1701913024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa07c157ca0 0x7fa07c0cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.164+0000 7fa0814d6700 1 --2- 172.21.15.7:0/1701913024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa07c0c3ae0 0x7fa07c1545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.164+0000 7fa0814d6700 1 --2- 172.21.15.7:0/1701913024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa07c0cdbf0 0x7fa07c0cb200 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.164+0000 7fa0814d6700 1 -- 172.21.15.7:0/1701913024 >> 172.21.15.7:0/1701913024 conn(0x7fa07c098020 msgr2=0x7fa07c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:16.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.164+0000 7fa0814d6700 1 -- 172.21.15.7:0/1701913024 shutdown_connections 2024-01-27T02:43:16.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.164+0000 7fa0814d6700 1 -- 172.21.15.7:0/1701913024 wait complete. 2024-01-27T02:43:16.167 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 86 2024-01-27T02:43:16.179 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:43:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.309+0000 7fe026318700 1 Processor -- start 2024-01-27T02:43:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.309+0000 7fe026318700 1 -- start start 2024-01-27T02:43:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.310+0000 7fe026318700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe02015a310 0x7fe0200aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.310+0000 7fe026318700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe0200aadf0 0x7fe020063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.310+0000 7fe026318700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0200c30c0 con 0x7fe020159f40 2024-01-27T02:43:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.310+0000 7fe026318700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0200c4550 con 0x7fe02015a310 2024-01-27T02:43:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.310+0000 7fe026318700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0200ab270 con 0x7fe0200aadf0 2024-01-27T02:43:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.310+0000 7fe01f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe02015a310 0x7fe0200aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.310+0000 7fe01f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe02015a310 0x7fe0200aa8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43686/0 (socket says 172.21.15.7:43686) 2024-01-27T02:43:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.310+0000 7fe01f7fe700 1 -- 172.21.15.7:0/514713688 learned_addr learned my addr 172.21.15.7:0/514713688 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.311+0000 7fe0248b5700 1 --2- 172.21.15.7:0/514713688 >> v2:172.21.15.188:3300/0 conn(0x7fe0200aadf0 0x7fe020063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.311+0000 7fe01f7fe700 1 -- 172.21.15.7:0/514713688 >> v2:172.21.15.188:3300/0 conn(0x7fe0200aadf0 msgr2=0x7fe020063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.311+0000 7fe01f7fe700 1 --2- 172.21.15.7:0/514713688 >> v2:172.21.15.188:3300/0 conn(0x7fe0200aadf0 0x7fe020063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.311+0000 7fe01f7fe700 1 -- 172.21.15.7:0/514713688 >> v1:172.21.15.188:6789/0 conn(0x7fe020159f40 legacy=0x7fe0200c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.311+0000 7fe01f7fe700 1 -- 172.21.15.7:0/514713688 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02015e960 con 0x7fe02015a310 2024-01-27T02:43:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.311+0000 7fe0248b5700 1 --2- 172.21.15.7:0/514713688 >> v2:172.21.15.188:3300/0 conn(0x7fe0200aadf0 0x7fe020063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.311+0000 7fe01f7fe700 1 --2- 172.21.15.7:0/514713688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02015a310 0x7fe0200aa8b0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fe0200ca640 tx=0x7fe014009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70569c74cb1b160 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.311+0000 7fe01effd700 1 -- 172.21.15.7:0/514713688 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01400b860 con 0x7fe02015a310 2024-01-27T02:43:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.311+0000 7fe01effd700 1 -- 172.21.15.7:0/514713688 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01400b9c0 con 0x7fe02015a310 2024-01-27T02:43:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.311+0000 7fe01effd700 1 -- 172.21.15.7:0/514713688 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01400bd60 con 0x7fe02015a310 2024-01-27T02:43:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.312+0000 7fe026318700 1 -- 172.21.15.7:0/514713688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02015a310 msgr2=0x7fe0200aa8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.312+0000 7fe026318700 1 --2- 172.21.15.7:0/514713688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02015a310 0x7fe0200aa8b0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fe0200ca640 tx=0x7fe014009250 comp rx=0 tx=0).stop 2024-01-27T02:43:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.312+0000 7fe026318700 1 -- 172.21.15.7:0/514713688 shutdown_connections 2024-01-27T02:43:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.312+0000 7fe026318700 1 --2- 172.21.15.7:0/514713688 >> v2:172.21.15.188:3300/0 conn(0x7fe0200aadf0 0x7fe020063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.312+0000 7fe026318700 1 --2- 172.21.15.7:0/514713688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02015a310 0x7fe0200aa8b0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.312+0000 7fe026318700 1 -- 172.21.15.7:0/514713688 >> 172.21.15.7:0/514713688 conn(0x7fe0200cdc30 msgr2=0x7fe0200ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.312+0000 7fe026318700 1 -- 172.21.15.7:0/514713688 shutdown_connections 2024-01-27T02:43:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.312+0000 7fe026318700 1 -- 172.21.15.7:0/514713688 wait complete. 2024-01-27T02:43:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.313+0000 7fe026318700 1 Processor -- start 2024-01-27T02:43:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.313+0000 7fe026318700 1 -- start start 2024-01-27T02:43:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.314+0000 7fe026318700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0200aadf0 0x7fe020158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.314+0000 7fe026318700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe020159f40 0x7fe0200c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.314+0000 7fe026318700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02015a310 0x7fe0200c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.314+0000 7fe026318700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0200c5df0 con 0x7fe02015a310 2024-01-27T02:43:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.314+0000 7fe026318700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02015c180 con 0x7fe020159f40 2024-01-27T02:43:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.314+0000 7fe026318700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe020158c50 con 0x7fe0200aadf0 2024-01-27T02:43:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.314+0000 7fe0248b5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02015a310 0x7fe0200c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.314+0000 7fe0248b5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02015a310 0x7fe0200c71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43702/0 (socket says 172.21.15.7:43702) 2024-01-27T02:43:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.314+0000 7fe0248b5700 1 -- 172.21.15.7:0/508018658 learned_addr learned my addr 172.21.15.7:0/508018658 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.314+0000 7fe01f7fe700 1 --2- 172.21.15.7:0/508018658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe020159f40 0x7fe0200c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.314+0000 7fe01ffff700 1 --2- 172.21.15.7:0/508018658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0200aadf0 0x7fe020158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.314+0000 7fe0248b5700 1 -- 172.21.15.7:0/508018658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0200aadf0 msgr2=0x7fe020158340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.314+0000 7fe0248b5700 1 --2- 172.21.15.7:0/508018658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0200aadf0 0x7fe020158340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.314+0000 7fe0248b5700 1 -- 172.21.15.7:0/508018658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe020159f40 msgr2=0x7fe0200c56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.314+0000 7fe0248b5700 1 --2- 172.21.15.7:0/508018658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe020159f40 0x7fe0200c56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.315+0000 7fe0248b5700 1 -- 172.21.15.7:0/508018658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0140094f0 con 0x7fe02015a310 2024-01-27T02:43:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.315+0000 7fe01f7fe700 1 --2- 172.21.15.7:0/508018658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe020159f40 0x7fe0200c56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.315+0000 7fe0248b5700 1 --2- 172.21.15.7:0/508018658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02015a310 0x7fe0200c71e0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fe018009fc0 tx=0x7fe0180099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.315+0000 7fe007fff700 1 -- 172.21.15.7:0/508018658 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01800dd40 con 0x7fe02015a310 2024-01-27T02:43:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.315+0000 7fe007fff700 1 -- 172.21.15.7:0/508018658 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01800f040 con 0x7fe02015a310 2024-01-27T02:43:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.315+0000 7fe026318700 1 -- 172.21.15.7:0/508018658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02015ccc0 con 0x7fe02015a310 2024-01-27T02:43:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.315+0000 7fe007fff700 1 -- 172.21.15.7:0/508018658 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe018015690 con 0x7fe02015a310 2024-01-27T02:43:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.315+0000 7fe026318700 1 -- 172.21.15.7:0/508018658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02015c670 con 0x7fe02015a310 2024-01-27T02:43:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.316+0000 7fe026318700 1 -- 172.21.15.7:0/508018658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe02015ce50 con 0x7fe02015a310 2024-01-27T02:43:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.317+0000 7fe007fff700 1 -- 172.21.15.7:0/508018658 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe01801f440 con 0x7fe02015a310 2024-01-27T02:43:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.318+0000 7fe007fff700 1 --2- 172.21.15.7:0/508018658 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe00c066f30 0x7fe00c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.318+0000 7fe007fff700 1 -- 172.21.15.7:0/508018658 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5734+0+0 (secure 0 0 0) 0x7fe018084f50 con 0x7fe02015a310 2024-01-27T02:43:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.318+0000 7fe007fff700 1 -- 172.21.15.7:0/508018658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=169}) v3 -- 0x7fe00c06eab0 con 0x7fe02015a310 2024-01-27T02:43:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.318+0000 7fe01ffff700 1 --2- 172.21.15.7:0/508018658 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe00c066f30 0x7fe00c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.319+0000 7fe01ffff700 1 --2- 172.21.15.7:0/508018658 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe00c066f30 0x7fe00c0693e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe010003860 tx=0x7fe010009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.322+0000 7fe007fff700 1 -- 172.21.15.7:0/508018658 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe01801a020 con 0x7fe02015a310 2024-01-27T02:43:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.525+0000 7fe026318700 1 -- 172.21.15.7:0/508018658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe02004c710 con 0x7fe02015a310 2024-01-27T02:43:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.526+0000 7fe007fff700 1 -- 172.21.15.7:0/508018658 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) v1 ==== 74+0+15392 (secure 0 0 0) 0x7fe018052220 con 0x7fe02015a310 2024-01-27T02:43:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:16.528 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":168,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:42:58.993229+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:41:56.176076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-01-27T02:41:56.620523+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.a","mappings":[{"from":4,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:43:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.528+0000 7fe026318700 1 -- 172.21.15.7:0/508018658 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe00c066f30 msgr2=0x7fe00c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.528+0000 7fe026318700 1 --2- 172.21.15.7:0/508018658 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe00c066f30 0x7fe00c0693e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe010003860 tx=0x7fe010009040 comp rx=0 tx=0).stop 2024-01-27T02:43:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.528+0000 7fe026318700 1 -- 172.21.15.7:0/508018658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02015a310 msgr2=0x7fe0200c71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.528+0000 7fe026318700 1 --2- 172.21.15.7:0/508018658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02015a310 0x7fe0200c71e0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fe018009fc0 tx=0x7fe0180099b0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.528+0000 7fe026318700 1 -- 172.21.15.7:0/508018658 shutdown_connections 2024-01-27T02:43:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.528+0000 7fe026318700 1 --2- 172.21.15.7:0/508018658 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fe00c066f30 0x7fe00c0693e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.528+0000 7fe026318700 1 --2- 172.21.15.7:0/508018658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02015a310 0x7fe0200c71e0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.528+0000 7fe026318700 1 --2- 172.21.15.7:0/508018658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0200aadf0 0x7fe020158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.528+0000 7fe026318700 1 --2- 172.21.15.7:0/508018658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe020159f40 0x7fe0200c56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.528+0000 7fe026318700 1 -- 172.21.15.7:0/508018658 >> 172.21.15.7:0/508018658 conn(0x7fe0200cdc30 msgr2=0x7fe0200c3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.528+0000 7fe026318700 1 -- 172.21.15.7:0/508018658 shutdown_connections 2024-01-27T02:43:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.528+0000 7fe026318700 1 -- 172.21.15.7:0/508018658 wait complete. 2024-01-27T02:43:16.544 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-27T02:43:16.544 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:43:16.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.674+0000 7f361e752700 1 Processor -- start 2024-01-27T02:43:16.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.674+0000 7f361e752700 1 -- start start 2024-01-27T02:43:16.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.675+0000 7f361e752700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f36181404e0 0x7f36180cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:16.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.675+0000 7f361e752700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f36180b09f0 0x7f36180cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:16.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.675+0000 7f361e752700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36180cce30 con 0x7f36181408b0 2024-01-27T02:43:16.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.675+0000 7f361e752700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f361806b3b0 con 0x7f36181404e0 2024-01-27T02:43:16.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.675+0000 7f361e752700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36180cc3e0 con 0x7f36180b09f0 2024-01-27T02:43:16.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.675+0000 7f3617fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f36181404e0 0x7f36180cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:16.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.675+0000 7f3617fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f36181404e0 0x7f36180cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43704/0 (socket says 172.21.15.7:43704) 2024-01-27T02:43:16.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.676+0000 7f3617fff700 1 -- 172.21.15.7:0/741591494 learned_addr learned my addr 172.21.15.7:0/741591494 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:16.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.676+0000 7f36177fe700 1 --2- 172.21.15.7:0/741591494 >> v2:172.21.15.188:3300/0 conn(0x7f36180b09f0 0x7f36180cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:16.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.676+0000 7f3617fff700 1 -- 172.21.15.7:0/741591494 >> v2:172.21.15.188:3300/0 conn(0x7f36180b09f0 msgr2=0x7f36180cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:16.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.676+0000 7f3617fff700 1 --2- 172.21.15.7:0/741591494 >> v2:172.21.15.188:3300/0 conn(0x7f36180b09f0 0x7f36180cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.676+0000 7f3617fff700 1 -- 172.21.15.7:0/741591494 >> v1:172.21.15.188:6789/0 conn(0x7f36181408b0 legacy=0x7f361806e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:16.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.676+0000 7f3617fff700 1 -- 172.21.15.7:0/741591494 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36180cc6c0 con 0x7f36181404e0 2024-01-27T02:43:16.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.676+0000 7f3617fff700 1 --2- 172.21.15.7:0/741591494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36181404e0 0x7f36180cb800 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f361806e0b0 tx=0x7f3608009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03310af104f2183 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:16.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.676+0000 7f3616ffd700 1 -- 172.21.15.7:0/741591494 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360800b860 con 0x7f36181404e0 2024-01-27T02:43:16.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.676+0000 7f3616ffd700 1 -- 172.21.15.7:0/741591494 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360800b9c0 con 0x7f36181404e0 2024-01-27T02:43:16.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.676+0000 7f3616ffd700 1 -- 172.21.15.7:0/741591494 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360800bd60 con 0x7f36181404e0 2024-01-27T02:43:16.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.677+0000 7f361e752700 1 -- 172.21.15.7:0/741591494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36181404e0 msgr2=0x7f36180cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:16.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.677+0000 7f361e752700 1 --2- 172.21.15.7:0/741591494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36181404e0 0x7f36180cb800 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f361806e0b0 tx=0x7f3608009250 comp rx=0 tx=0).stop 2024-01-27T02:43:16.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.677+0000 7f361e752700 1 -- 172.21.15.7:0/741591494 shutdown_connections 2024-01-27T02:43:16.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.677+0000 7f361e752700 1 --2- 172.21.15.7:0/741591494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36181404e0 0x7f36180cb800 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.677+0000 7f361e752700 1 --2- 172.21.15.7:0/741591494 >> v2:172.21.15.188:3300/0 conn(0x7f36180b09f0 0x7f36180cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.677+0000 7f361e752700 1 -- 172.21.15.7:0/741591494 >> 172.21.15.7:0/741591494 conn(0x7f36180cdc30 msgr2=0x7f36180ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:16.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.677+0000 7f361e752700 1 -- 172.21.15.7:0/741591494 shutdown_connections 2024-01-27T02:43:16.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.677+0000 7f361e752700 1 -- 172.21.15.7:0/741591494 wait complete. 2024-01-27T02:43:16.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.678+0000 7f361e752700 1 Processor -- start 2024-01-27T02:43:16.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.678+0000 7f361e752700 1 -- start start 2024-01-27T02:43:16.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.679+0000 7f361e752700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36180b09f0 0x7f3618154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:16.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.679+0000 7f361e752700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36181404e0 0x7f36180c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.679+0000 7f361e752700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36181408b0 0x7f36180b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.679+0000 7f361e752700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3618155130 con 0x7f36180b09f0 2024-01-27T02:43:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.679+0000 7f361e752700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36180c8cc0 con 0x7f36181408b0 2024-01-27T02:43:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.679+0000 7f361e752700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3618139dd0 con 0x7f36181404e0 2024-01-27T02:43:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.679+0000 7f3617fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36180b09f0 0x7f3618154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.679+0000 7f3617fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36180b09f0 0x7f3618154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43712/0 (socket says 172.21.15.7:43712) 2024-01-27T02:43:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.679+0000 7f3617fff700 1 -- 172.21.15.7:0/2635679948 learned_addr learned my addr 172.21.15.7:0/2635679948 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.679+0000 7f36177fe700 1 --2- 172.21.15.7:0/2635679948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36181404e0 0x7f36180c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.679+0000 7f361ccef700 1 --2- 172.21.15.7:0/2635679948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36181408b0 0x7f36180b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.680+0000 7f3617fff700 1 -- 172.21.15.7:0/2635679948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36181404e0 msgr2=0x7f36180c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.680+0000 7f3617fff700 1 --2- 172.21.15.7:0/2635679948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36181404e0 0x7f36180c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.680+0000 7f3617fff700 1 -- 172.21.15.7:0/2635679948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36181408b0 msgr2=0x7f36180b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.680+0000 7f3617fff700 1 --2- 172.21.15.7:0/2635679948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36181408b0 0x7f36180b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.680+0000 7f3617fff700 1 -- 172.21.15.7:0/2635679948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36080094f0 con 0x7f36180b09f0 2024-01-27T02:43:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.680+0000 7f361ccef700 1 --2- 172.21.15.7:0/2635679948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36181408b0 0x7f36180b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.680+0000 7f3617fff700 1 --2- 172.21.15.7:0/2635679948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36180b09f0 0x7f3618154bf0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f361806dfa0 tx=0x7f3608017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.680+0000 7f35fffff700 1 -- 172.21.15.7:0/2635679948 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608024440 con 0x7f36180b09f0 2024-01-27T02:43:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.680+0000 7f35fffff700 1 -- 172.21.15.7:0/2635679948 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36080245a0 con 0x7f36180b09f0 2024-01-27T02:43:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.680+0000 7f35fffff700 1 -- 172.21.15.7:0/2635679948 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608024960 con 0x7f36180b09f0 2024-01-27T02:43:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.680+0000 7f361e752700 1 -- 172.21.15.7:0/2635679948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f361815c8a0 con 0x7f36180b09f0 2024-01-27T02:43:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.680+0000 7f361e752700 1 -- 172.21.15.7:0/2635679948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f361815ce60 con 0x7f36180b09f0 2024-01-27T02:43:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.681+0000 7f361e752700 1 -- 172.21.15.7:0/2635679948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f361804c710 con 0x7f36180b09f0 2024-01-27T02:43:16.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.682+0000 7f35fffff700 1 -- 172.21.15.7:0/2635679948 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3608024d50 con 0x7f36180b09f0 2024-01-27T02:43:16.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.682+0000 7f35fffff700 1 --2- 172.21.15.7:0/2635679948 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3604066f30 0x7f36040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:16.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.682+0000 7f35fffff700 1 -- 172.21.15.7:0/2635679948 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f36080861b0 con 0x7f36180b09f0 2024-01-27T02:43:16.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.682+0000 7f35fffff700 1 -- 172.21.15.7:0/2635679948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=169}) v3 -- 0x7f360406eab0 con 0x7f36180b09f0 2024-01-27T02:43:16.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.682+0000 7f36177fe700 1 --2- 172.21.15.7:0/2635679948 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3604066f30 0x7f36040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:16.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.683+0000 7f36177fe700 1 --2- 172.21.15.7:0/2635679948 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3604066f30 0x7f36040693e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f360c006fd0 tx=0x7f360c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:16.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.687+0000 7f35fffff700 1 -- 172.21.15.7:0/2635679948 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3608053400 con 0x7f36180b09f0 2024-01-27T02:43:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.935+0000 7f361e752700 1 -- 172.21.15.7:0/2635679948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f361806f4b0 con 0x7f36180b09f0 2024-01-27T02:43:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.935+0000 7f35fffff700 1 -- 172.21.15.7:0/2635679948 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f3608015070 con 0x7f36180b09f0 2024-01-27T02:43:16.937 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:16.937 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6473,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/3449593528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3449593528},{"type":"v1","addr":"172.21.15.188:6836","nonce":3449593528}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3473642033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3473642033},{"type":"v1","addr":"172.21.15.188:6837","nonce":3473642033}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6479,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2550009140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2550009140},{"type":"v1","addr":"172.21.15.7:6833","nonce":2550009140}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:41:57.646012+0000","modified":"2024-01-27T02:42:01.251419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6485":{"gid":6485,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3213683076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3213683076},{"type":"v1","addr":"172.21.15.7:6835","nonce":3213683076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2024-01-27T02:43:16.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.937+0000 7f361e752700 1 -- 172.21.15.7:0/2635679948 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3604066f30 msgr2=0x7f36040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:16.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.937+0000 7f361e752700 1 --2- 172.21.15.7:0/2635679948 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3604066f30 0x7f36040693e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f360c006fd0 tx=0x7f360c008040 comp rx=0 tx=0).stop 2024-01-27T02:43:16.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.937+0000 7f361e752700 1 -- 172.21.15.7:0/2635679948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36180b09f0 msgr2=0x7f3618154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:16.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.937+0000 7f361e752700 1 --2- 172.21.15.7:0/2635679948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36180b09f0 0x7f3618154bf0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f361806dfa0 tx=0x7f3608017000 comp rx=0 tx=0).stop 2024-01-27T02:43:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.937+0000 7f361e752700 1 -- 172.21.15.7:0/2635679948 shutdown_connections 2024-01-27T02:43:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.937+0000 7f361e752700 1 --2- 172.21.15.7:0/2635679948 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3604066f30 0x7f36040693e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.937+0000 7f361e752700 1 --2- 172.21.15.7:0/2635679948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36181408b0 0x7f36180b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.937+0000 7f361e752700 1 --2- 172.21.15.7:0/2635679948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36180b09f0 0x7f3618154bf0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.937+0000 7f361e752700 1 --2- 172.21.15.7:0/2635679948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36181404e0 0x7f36180c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.937+0000 7f361e752700 1 -- 172.21.15.7:0/2635679948 >> 172.21.15.7:0/2635679948 conn(0x7f36180cdc30 msgr2=0x7f361806e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.938+0000 7f361e752700 1 -- 172.21.15.7:0/2635679948 shutdown_connections 2024-01-27T02:43:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:16.938+0000 7f361e752700 1 -- 172.21.15.7:0/2635679948 wait complete. 2024-01-27T02:43:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 86 2024-01-27T02:43:16.953 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:43:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.084+0000 7f11125ca700 1 Processor -- start 2024-01-27T02:43:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.084+0000 7f11125ca700 1 -- start start 2024-01-27T02:43:17.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.085+0000 7f11125ca700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f110c157ca0 0x7f110c0c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:17.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.085+0000 7f11125ca700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f110c0cdbf0 0x7f110c0ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:17.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.085+0000 7f11125ca700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f110c0ca020 con 0x7f110c0c3ae0 2024-01-27T02:43:17.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.085+0000 7f11125ca700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f110c0c4f40 con 0x7f110c157ca0 2024-01-27T02:43:17.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.085+0000 7f11125ca700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110c0c5210 con 0x7f110c0cdbf0 2024-01-27T02:43:17.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.085+0000 7f110bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f110c157ca0 0x7f110c0c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.085+0000 7f110b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f110c0c3ae0 0x7f110c0cd4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46724/0 (socket says 172.21.15.7:46724) 2024-01-27T02:43:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.085+0000 7f110b7fe700 1 -- 172.21.15.7:0/646734413 learned_addr learned my addr 172.21.15.7:0/646734413 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.085+0000 7f1110b67700 1 --2- 172.21.15.7:0/646734413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f110c0cdbf0 0x7f110c0ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.086+0000 7f110bfff700 1 -- 172.21.15.7:0/646734413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f110c0cdbf0 msgr2=0x7f110c0ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.086+0000 7f110bfff700 1 --2- 172.21.15.7:0/646734413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f110c0cdbf0 0x7f110c0ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.086+0000 7f110bfff700 1 -- 172.21.15.7:0/646734413 >> v1:172.21.15.7:6789/0 conn(0x7f110c0c3ae0 legacy=0x7f110c0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.086+0000 7f110bfff700 1 -- 172.21.15.7:0/646734413 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110c0c54f0 con 0x7f110c157ca0 2024-01-27T02:43:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.086+0000 7f110affd700 1 -- 172.21.15.7:0/646734413 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1523291091 0 0) 0x7f110c0ca020 con 0x7f110c0c3ae0 2024-01-27T02:43:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.086+0000 7f1110b67700 1 --2- 172.21.15.7:0/646734413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f110c0cdbf0 0x7f110c0ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.086+0000 7f110bfff700 1 --2- 172.21.15.7:0/646734413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f110c157ca0 0x7f110c0c4a00 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0098e0 tx=0x7f10fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab6bcb4db63da6f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.086+0000 7f110affd700 1 -- 172.21.15.7:0/646734413 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc009e80 con 0x7f110c157ca0 2024-01-27T02:43:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.086+0000 7f110affd700 1 -- 172.21.15.7:0/646734413 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10fc00b860 con 0x7f110c157ca0 2024-01-27T02:43:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.086+0000 7f110affd700 1 -- 172.21.15.7:0/646734413 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc00bbe0 con 0x7f110c157ca0 2024-01-27T02:43:17.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.086+0000 7f11125ca700 1 -- 172.21.15.7:0/646734413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f110c157ca0 msgr2=0x7f110c0c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:17.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.086+0000 7f11125ca700 1 --2- 172.21.15.7:0/646734413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f110c157ca0 0x7f110c0c4a00 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0098e0 tx=0x7f10fc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.087+0000 7f11125ca700 1 -- 172.21.15.7:0/646734413 shutdown_connections 2024-01-27T02:43:17.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.087+0000 7f11125ca700 1 --2- 172.21.15.7:0/646734413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f110c0cdbf0 0x7f110c0ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.087+0000 7f11125ca700 1 --2- 172.21.15.7:0/646734413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f110c157ca0 0x7f110c0c4a00 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.087+0000 7f11125ca700 1 -- 172.21.15.7:0/646734413 >> 172.21.15.7:0/646734413 conn(0x7f110c098020 msgr2=0x7f110c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:17.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.087+0000 7f11125ca700 1 -- 172.21.15.7:0/646734413 shutdown_connections 2024-01-27T02:43:17.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.087+0000 7f11125ca700 1 -- 172.21.15.7:0/646734413 wait complete. 2024-01-27T02:43:17.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.088+0000 7f11125ca700 1 Processor -- start 2024-01-27T02:43:17.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.089+0000 7f11125ca700 1 -- start start 2024-01-27T02:43:17.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.089+0000 7f11125ca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f110c0c3ae0 0x7f110c1507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:17.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.089+0000 7f11125ca700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f110c0cdbf0 0x7f110c156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:17.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.090+0000 7f11125ca700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f110c157ca0 0x7f110c156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:17.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.090+0000 7f11125ca700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c154d10 con 0x7f110c0c3ae0 2024-01-27T02:43:17.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.090+0000 7f11125ca700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c154fa0 con 0x7f110c157ca0 2024-01-27T02:43:17.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.090+0000 7f11125ca700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110c155280 con 0x7f110c0cdbf0 2024-01-27T02:43:17.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.090+0000 7f110bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f110c0c3ae0 0x7f110c1507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:17.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.090+0000 7f110bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f110c0c3ae0 0x7f110c1507e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43742/0 (socket says 172.21.15.7:43742) 2024-01-27T02:43:17.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.090+0000 7f110bfff700 1 -- 172.21.15.7:0/4146804199 learned_addr learned my addr 172.21.15.7:0/4146804199 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:17.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.090+0000 7f110b7fe700 1 --2- 172.21.15.7:0/4146804199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f110c0cdbf0 0x7f110c156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:17.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.090+0000 7f1110b67700 1 --2- 172.21.15.7:0/4146804199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f110c157ca0 0x7f110c156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:17.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.090+0000 7f110bfff700 1 -- 172.21.15.7:0/4146804199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f110c0cdbf0 msgr2=0x7f110c156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:17.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.090+0000 7f110bfff700 1 --2- 172.21.15.7:0/4146804199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f110c0cdbf0 0x7f110c156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.090+0000 7f110bfff700 1 -- 172.21.15.7:0/4146804199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f110c157ca0 msgr2=0x7f110c156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:17.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.090+0000 7f110bfff700 1 --2- 172.21.15.7:0/4146804199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f110c157ca0 0x7f110c156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.090+0000 7f110bfff700 1 -- 172.21.15.7:0/4146804199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10fc009420 con 0x7f110c0c3ae0 2024-01-27T02:43:17.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.090+0000 7f110b7fe700 1 --2- 172.21.15.7:0/4146804199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f110c0cdbf0 0x7f110c156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:43:17.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.090+0000 7f1110b67700 1 --2- 172.21.15.7:0/4146804199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f110c157ca0 0x7f110c156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:43:17.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.090+0000 7f110bfff700 1 --2- 172.21.15.7:0/4146804199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f110c0c3ae0 0x7f110c1507e0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f110c06ea80 tx=0x7f10fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:17.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.090+0000 7f10f3fff700 1 -- 172.21.15.7:0/4146804199 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc00bbe0 con 0x7f110c0c3ae0 2024-01-27T02:43:17.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.090+0000 7f10f3fff700 1 -- 172.21.15.7:0/4146804199 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10fc011420 con 0x7f110c0c3ae0 2024-01-27T02:43:17.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.090+0000 7f11125ca700 1 -- 172.21.15.7:0/4146804199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f110c155560 con 0x7f110c0c3ae0 2024-01-27T02:43:17.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.090+0000 7f10f3fff700 1 -- 172.21.15.7:0/4146804199 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc0117a0 con 0x7f110c0c3ae0 2024-01-27T02:43:17.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.091+0000 7f11125ca700 1 -- 172.21.15.7:0/4146804199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f110c155b20 con 0x7f110c0c3ae0 2024-01-27T02:43:17.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.092+0000 7f11125ca700 1 -- 172.21.15.7:0/4146804199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f110c155d10 con 0x7f110c0c3ae0 2024-01-27T02:43:17.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.092+0000 7f10f3fff700 1 -- 172.21.15.7:0/4146804199 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f10fc011bd0 con 0x7f110c0c3ae0 2024-01-27T02:43:17.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.092+0000 7f10f3fff700 1 --2- 172.21.15.7:0/4146804199 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f10f8066f30 0x7f10f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:17.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.092+0000 7f10f3fff700 1 -- 172.21.15.7:0/4146804199 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f10fc085f40 con 0x7f110c0c3ae0 2024-01-27T02:43:17.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.093+0000 7f10f3fff700 1 -- 172.21.15.7:0/4146804199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=169}) v3 -- 0x7f10f806eab0 con 0x7f110c0c3ae0 2024-01-27T02:43:17.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.093+0000 7f110b7fe700 1 --2- 172.21.15.7:0/4146804199 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f10f8066f30 0x7f10f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:17.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.093+0000 7f110b7fe700 1 --2- 172.21.15.7:0/4146804199 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f10f8066f30 0x7f10f80693e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f110c09c600 tx=0x7f1100009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:17.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.097+0000 7f10f3fff700 1 -- 172.21.15.7:0/4146804199 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f10fc053190 con 0x7f110c0c3ae0 2024-01-27T02:43:17.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.352+0000 7f11125ca700 1 -- 172.21.15.7:0/4146804199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f110c04c710 con 0x7f110c0c3ae0 2024-01-27T02:43:17.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.353+0000 7f10f3fff700 1 -- 172.21.15.7:0/4146804199 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f10fc015030 con 0x7f110c0c3ae0 2024-01-27T02:43:17.354 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:17.354 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6473,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/3449593528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3449593528},{"type":"v1","addr":"172.21.15.188:6836","nonce":3449593528}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3473642033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3473642033},{"type":"v1","addr":"172.21.15.188:6837","nonce":3473642033}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6479,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2550009140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2550009140},{"type":"v1","addr":"172.21.15.7:6833","nonce":2550009140}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:41:57.646012+0000","modified":"2024-01-27T02:42:01.251419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6485":{"gid":6485,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3213683076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3213683076},{"type":"v1","addr":"172.21.15.7:6835","nonce":3213683076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2024-01-27T02:43:17.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.355+0000 7f11125ca700 1 -- 172.21.15.7:0/4146804199 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f10f8066f30 msgr2=0x7f10f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:17.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.355+0000 7f11125ca700 1 --2- 172.21.15.7:0/4146804199 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f10f8066f30 0x7f10f80693e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f110c09c600 tx=0x7f1100009040 comp rx=0 tx=0).stop 2024-01-27T02:43:17.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.355+0000 7f11125ca700 1 -- 172.21.15.7:0/4146804199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f110c0c3ae0 msgr2=0x7f110c1507e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:17.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.355+0000 7f11125ca700 1 --2- 172.21.15.7:0/4146804199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f110c0c3ae0 0x7f110c1507e0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f110c06ea80 tx=0x7f10fc017000 comp rx=0 tx=0).stop 2024-01-27T02:43:17.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.355+0000 7f11125ca700 1 -- 172.21.15.7:0/4146804199 shutdown_connections 2024-01-27T02:43:17.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.355+0000 7f11125ca700 1 --2- 172.21.15.7:0/4146804199 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f10f8066f30 0x7f10f80693e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.355+0000 7f11125ca700 1 --2- 172.21.15.7:0/4146804199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f110c157ca0 0x7f110c156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.355+0000 7f11125ca700 1 --2- 172.21.15.7:0/4146804199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f110c0c3ae0 0x7f110c1507e0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.355+0000 7f11125ca700 1 --2- 172.21.15.7:0/4146804199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f110c0cdbf0 0x7f110c156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.355+0000 7f11125ca700 1 -- 172.21.15.7:0/4146804199 >> 172.21.15.7:0/4146804199 conn(0x7f110c098020 msgr2=0x7f110c06bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:17.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.355+0000 7f11125ca700 1 -- 172.21.15.7:0/4146804199 shutdown_connections 2024-01-27T02:43:17.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.355+0000 7f11125ca700 1 -- 172.21.15.7:0/4146804199 wait complete. 2024-01-27T02:43:17.358 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 86 2024-01-27T02:43:17.370 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:43:17.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.500+0000 7f4519cf6700 1 Processor -- start 2024-01-27T02:43:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.500+0000 7f4519cf6700 1 -- start start 2024-01-27T02:43:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.501+0000 7f4519cf6700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f451415be10 0x7f45140b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.501+0000 7f4519cf6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45140b0180 0x7f45140aee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.501+0000 7f4519cf6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f451415c1e0 0x7f45140c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.501+0000 7f4519cf6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45140b1470 con 0x7f451415c1e0 2024-01-27T02:43:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.501+0000 7f4519cf6700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45140b05d0 con 0x7f451415be10 2024-01-27T02:43:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.501+0000 7f4519cf6700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45140af370 con 0x7f45140b0180 2024-01-27T02:43:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.501+0000 7f4513fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f451415c1e0 0x7f45140c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.501+0000 7f4513fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f451415c1e0 0x7f45140c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43754/0 (socket says 172.21.15.7:43754) 2024-01-27T02:43:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.501+0000 7f4513fff700 1 -- 172.21.15.7:0/1630729145 learned_addr learned my addr 172.21.15.7:0/1630729145 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.501+0000 7f45137fe700 1 --2- 172.21.15.7:0/1630729145 >> v2:172.21.15.188:3300/0 conn(0x7f451415be10 0x7f45140b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.501+0000 7f4512ffd700 1 --2- 172.21.15.7:0/1630729145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45140b0180 0x7f45140aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.501+0000 7f4513fff700 1 -- 172.21.15.7:0/1630729145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45140b0180 msgr2=0x7f45140aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.501+0000 7f4513fff700 1 --2- 172.21.15.7:0/1630729145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45140b0180 0x7f45140aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.501+0000 7f4513fff700 1 -- 172.21.15.7:0/1630729145 >> v2:172.21.15.188:3300/0 conn(0x7f451415be10 msgr2=0x7f45140b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.501+0000 7f4513fff700 1 --2- 172.21.15.7:0/1630729145 >> v2:172.21.15.188:3300/0 conn(0x7f451415be10 0x7f45140b1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.501+0000 7f4513fff700 1 -- 172.21.15.7:0/1630729145 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45140b0860 con 0x7f451415c1e0 2024-01-27T02:43:17.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.501+0000 7f45137fe700 1 --2- 172.21.15.7:0/1630729145 >> v2:172.21.15.188:3300/0 conn(0x7f451415be10 0x7f45140b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:43:17.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.501+0000 7f4512ffd700 1 --2- 172.21.15.7:0/1630729145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45140b0180 0x7f45140aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:43:17.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.502+0000 7f4513fff700 1 --2- 172.21.15.7:0/1630729145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f451415c1e0 0x7f45140c9500 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f45140693e0 tx=0x7f450c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d14a411b806c86 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:17.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.502+0000 7f45127fc700 1 -- 172.21.15.7:0/1630729145 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c00e070 con 0x7f451415c1e0 2024-01-27T02:43:17.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.502+0000 7f45127fc700 1 -- 172.21.15.7:0/1630729145 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450c00ba50 con 0x7f451415c1e0 2024-01-27T02:43:17.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.502+0000 7f45127fc700 1 -- 172.21.15.7:0/1630729145 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c0092e0 con 0x7f451415c1e0 2024-01-27T02:43:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.502+0000 7f4519cf6700 1 -- 172.21.15.7:0/1630729145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f451415c1e0 msgr2=0x7f45140c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.502+0000 7f4519cf6700 1 --2- 172.21.15.7:0/1630729145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f451415c1e0 0x7f45140c9500 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f45140693e0 tx=0x7f450c009840 comp rx=0 tx=0).stop 2024-01-27T02:43:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.502+0000 7f4519cf6700 1 -- 172.21.15.7:0/1630729145 shutdown_connections 2024-01-27T02:43:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.502+0000 7f4519cf6700 1 --2- 172.21.15.7:0/1630729145 >> v2:172.21.15.188:3300/0 conn(0x7f451415be10 0x7f45140b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.502+0000 7f4519cf6700 1 --2- 172.21.15.7:0/1630729145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f451415c1e0 0x7f45140c9500 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.502+0000 7f4519cf6700 1 --2- 172.21.15.7:0/1630729145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45140b0180 0x7f45140aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.502+0000 7f4519cf6700 1 -- 172.21.15.7:0/1630729145 >> 172.21.15.7:0/1630729145 conn(0x7f45140679b0 msgr2=0x7f4514067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.503+0000 7f4519cf6700 1 -- 172.21.15.7:0/1630729145 shutdown_connections 2024-01-27T02:43:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.503+0000 7f4519cf6700 1 -- 172.21.15.7:0/1630729145 wait complete. 2024-01-27T02:43:17.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.504+0000 7f4519cf6700 1 Processor -- start 2024-01-27T02:43:17.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.504+0000 7f4519cf6700 1 -- start start 2024-01-27T02:43:17.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.505+0000 7f4519cf6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45140b0180 0x7f45140c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:17.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.505+0000 7f4519cf6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f451415be10 0x7f4514099bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:17.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.505+0000 7f4519cf6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f451415c1e0 0x7f451416bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:17.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.505+0000 7f4519cf6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45141555e0 con 0x7f451415c1e0 2024-01-27T02:43:17.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.505+0000 7f4519cf6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4514099760 con 0x7f451415be10 2024-01-27T02:43:17.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.505+0000 7f4519cf6700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4514098b50 con 0x7f45140b0180 2024-01-27T02:43:17.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.505+0000 7f4513fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f451415c1e0 0x7f451416bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:17.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.505+0000 7f4513fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f451415c1e0 0x7f451416bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43758/0 (socket says 172.21.15.7:43758) 2024-01-27T02:43:17.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.505+0000 7f4513fff700 1 -- 172.21.15.7:0/3669829468 learned_addr learned my addr 172.21.15.7:0/3669829468 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:17.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.505+0000 7f4512ffd700 1 --2- 172.21.15.7:0/3669829468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f451415be10 0x7f4514099bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:17.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.505+0000 7f45137fe700 1 --2- 172.21.15.7:0/3669829468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45140b0180 0x7f45140c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:17.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.505+0000 7f4513fff700 1 -- 172.21.15.7:0/3669829468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45140b0180 msgr2=0x7f45140c2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:17.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.505+0000 7f4513fff700 1 --2- 172.21.15.7:0/3669829468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45140b0180 0x7f45140c2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.505+0000 7f4513fff700 1 -- 172.21.15.7:0/3669829468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f451415be10 msgr2=0x7f4514099bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:17.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.505+0000 7f4513fff700 1 --2- 172.21.15.7:0/3669829468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f451415be10 0x7f4514099bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.505+0000 7f4513fff700 1 -- 172.21.15.7:0/3669829468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450c009ae0 con 0x7f451415c1e0 2024-01-27T02:43:17.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.505+0000 7f4513fff700 1 --2- 172.21.15.7:0/3669829468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f451415c1e0 0x7f451416bab0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f4514068eb0 tx=0x7f450c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:17.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.506+0000 7f450b7fe700 1 -- 172.21.15.7:0/3669829468 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c008e80 con 0x7f451415c1e0 2024-01-27T02:43:17.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.506+0000 7f450b7fe700 1 -- 172.21.15.7:0/3669829468 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450c01c410 con 0x7f451415c1e0 2024-01-27T02:43:17.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.506+0000 7f4519cf6700 1 -- 172.21.15.7:0/3669829468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4514098e30 con 0x7f451415c1e0 2024-01-27T02:43:17.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.506+0000 7f450b7fe700 1 -- 172.21.15.7:0/3669829468 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c01c7d0 con 0x7f451415c1e0 2024-01-27T02:43:17.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.506+0000 7f4519cf6700 1 -- 172.21.15.7:0/3669829468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45140993f0 con 0x7f451415c1e0 2024-01-27T02:43:17.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.507+0000 7f4519cf6700 1 -- 172.21.15.7:0/3669829468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4514099910 con 0x7f451415c1e0 2024-01-27T02:43:17.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.507+0000 7f450b7fe700 1 -- 172.21.15.7:0/3669829468 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f450c003040 con 0x7f451415c1e0 2024-01-27T02:43:17.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.508+0000 7f450b7fe700 1 --2- 172.21.15.7:0/3669829468 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f44f4066f30 0x7f44f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:17.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.508+0000 7f450b7fe700 1 -- 172.21.15.7:0/3669829468 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f450c085f50 con 0x7f451415c1e0 2024-01-27T02:43:17.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.508+0000 7f450b7fe700 1 -- 172.21.15.7:0/3669829468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=169}) v3 -- 0x7f44f406eab0 con 0x7f451415c1e0 2024-01-27T02:43:17.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.508+0000 7f45137fe700 1 --2- 172.21.15.7:0/3669829468 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f44f4066f30 0x7f44f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:17.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.508+0000 7f45137fe700 1 --2- 172.21.15.7:0/3669829468 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f44f4066f30 0x7f44f40693e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f450400ac00 tx=0x7f4504006f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:17.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.512+0000 7f450b7fe700 1 -- 172.21.15.7:0/3669829468 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4514099910 con 0x7f451415c1e0 2024-01-27T02:43:17.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.715+0000 7f4519cf6700 1 -- 172.21.15.7:0/3669829468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45140998d0 con 0x7f451415c1e0 2024-01-27T02:43:17.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.716+0000 7f450b7fe700 1 -- 172.21.15.7:0/3669829468 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) v1 ==== 74+0+15392 (secure 0 0 0) 0x7f450c02b020 con 0x7f451415c1e0 2024-01-27T02:43:17.717 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:17.718 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":168,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:42:58.993229+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:41:56.176076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-01-27T02:41:56.620523+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.a","mappings":[{"from":4,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:43:17.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.718+0000 7f4519cf6700 1 -- 172.21.15.7:0/3669829468 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f44f4066f30 msgr2=0x7f44f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:17.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.718+0000 7f4519cf6700 1 --2- 172.21.15.7:0/3669829468 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f44f4066f30 0x7f44f40693e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f450400ac00 tx=0x7f4504006f90 comp rx=0 tx=0).stop 2024-01-27T02:43:17.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.718+0000 7f4519cf6700 1 -- 172.21.15.7:0/3669829468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f451415c1e0 msgr2=0x7f451416bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:17.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.718+0000 7f4519cf6700 1 --2- 172.21.15.7:0/3669829468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f451415c1e0 0x7f451416bab0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f4514068eb0 tx=0x7f450c018000 comp rx=0 tx=0).stop 2024-01-27T02:43:17.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.718+0000 7f4519cf6700 1 -- 172.21.15.7:0/3669829468 shutdown_connections 2024-01-27T02:43:17.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.718+0000 7f4519cf6700 1 --2- 172.21.15.7:0/3669829468 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f44f4066f30 0x7f44f40693e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.718+0000 7f4519cf6700 1 --2- 172.21.15.7:0/3669829468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f451415c1e0 0x7f451416bab0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.718+0000 7f4519cf6700 1 --2- 172.21.15.7:0/3669829468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45140b0180 0x7f45140c2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.718+0000 7f4519cf6700 1 --2- 172.21.15.7:0/3669829468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f451415be10 0x7f4514099bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.718+0000 7f4519cf6700 1 -- 172.21.15.7:0/3669829468 >> 172.21.15.7:0/3669829468 conn(0x7f45140679b0 msgr2=0x7f45140c9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:17.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.718+0000 7f4519cf6700 1 -- 172.21.15.7:0/3669829468 shutdown_connections 2024-01-27T02:43:17.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.718+0000 7f4519cf6700 1 -- 172.21.15.7:0/3669829468 wait complete. 2024-01-27T02:43:17.733 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-27T02:43:17.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.864+0000 7fa02f6da700 1 Processor -- start 2024-01-27T02:43:17.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.864+0000 7fa02f6da700 1 -- start start 2024-01-27T02:43:17.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.865+0000 7fa02f6da700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa028159f40 0x7fa0280c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:17.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.865+0000 7fa02f6da700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0280aafc0 0x7fa028063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:17.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.865+0000 7fa02f6da700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0280c30c0 con 0x7fa0280c3a90 2024-01-27T02:43:17.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.865+0000 7fa02f6da700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0280c4550 con 0x7fa028159f40 2024-01-27T02:43:17.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.865+0000 7fa02f6da700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa02815e960 con 0x7fa0280aafc0 2024-01-27T02:43:17.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.865+0000 7fa02d476700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa028159f40 0x7fa0280c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:17.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.865+0000 7fa02cc75700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa0280c3a90 0x7fa0280aa8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46730/0 (socket says 172.21.15.7:46730) 2024-01-27T02:43:17.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.865+0000 7fa02cc75700 1 -- 172.21.15.7:0/447016541 learned_addr learned my addr 172.21.15.7:0/447016541 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:17.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.866+0000 7fa02dc77700 1 --2- 172.21.15.7:0/447016541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0280aafc0 0x7fa028063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:17.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.866+0000 7fa02d476700 1 -- 172.21.15.7:0/447016541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0280aafc0 msgr2=0x7fa028063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:17.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.866+0000 7fa02d476700 1 --2- 172.21.15.7:0/447016541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0280aafc0 0x7fa028063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.866+0000 7fa02d476700 1 -- 172.21.15.7:0/447016541 >> v1:172.21.15.7:6789/0 conn(0x7fa0280c3a90 legacy=0x7fa0280aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.866+0000 7fa02d476700 1 -- 172.21.15.7:0/447016541 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02815ec40 con 0x7fa028159f40 2024-01-27T02:43:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.866+0000 7fa01bfff700 1 -- 172.21.15.7:0/447016541 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2178304396 0 0) 0x7fa0280c30c0 con 0x7fa0280c3a90 2024-01-27T02:43:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.866+0000 7fa02d476700 1 --2- 172.21.15.7:0/447016541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa028159f40 0x7fa0280c3550 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fa01c0098e0 tx=0x7fa01c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0593811e17f54c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.866+0000 7fa01bfff700 1 -- 172.21.15.7:0/447016541 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c009e80 con 0x7fa028159f40 2024-01-27T02:43:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.866+0000 7fa01bfff700 1 -- 172.21.15.7:0/447016541 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01c00b860 con 0x7fa028159f40 2024-01-27T02:43:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.866+0000 7fa01bfff700 1 -- 172.21.15.7:0/447016541 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c00bbe0 con 0x7fa028159f40 2024-01-27T02:43:17.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.867+0000 7fa02f6da700 1 -- 172.21.15.7:0/447016541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa028159f40 msgr2=0x7fa0280c3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:17.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.867+0000 7fa02f6da700 1 --2- 172.21.15.7:0/447016541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa028159f40 0x7fa0280c3550 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fa01c0098e0 tx=0x7fa01c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.867+0000 7fa02f6da700 1 -- 172.21.15.7:0/447016541 shutdown_connections 2024-01-27T02:43:17.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.867+0000 7fa02f6da700 1 --2- 172.21.15.7:0/447016541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0280aafc0 0x7fa028063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.867+0000 7fa02f6da700 1 --2- 172.21.15.7:0/447016541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa028159f40 0x7fa0280c3550 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.867+0000 7fa02f6da700 1 -- 172.21.15.7:0/447016541 >> 172.21.15.7:0/447016541 conn(0x7fa0280cdc30 msgr2=0x7fa0280ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:17.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.867+0000 7fa02f6da700 1 -- 172.21.15.7:0/447016541 shutdown_connections 2024-01-27T02:43:17.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.867+0000 7fa02f6da700 1 -- 172.21.15.7:0/447016541 wait complete. 2024-01-27T02:43:17.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.868+0000 7fa02f6da700 1 Processor -- start 2024-01-27T02:43:17.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.868+0000 7fa02f6da700 1 -- start start 2024-01-27T02:43:17.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.869+0000 7fa02f6da700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0280aafc0 0x7fa02816e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.869+0000 7fa02f6da700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0280c3a90 0x7fa0280c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.869+0000 7fa02f6da700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa028159f40 0x7fa028156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.869+0000 7fa02f6da700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0280c6ef0 con 0x7fa028159f40 2024-01-27T02:43:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.869+0000 7fa02f6da700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0280c7100 con 0x7fa0280aafc0 2024-01-27T02:43:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.869+0000 7fa02f6da700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0280c73e0 con 0x7fa0280c3a90 2024-01-27T02:43:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.869+0000 7fa02dc77700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa028159f40 0x7fa028156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.869+0000 7fa02dc77700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa028159f40 0x7fa028156640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43772/0 (socket says 172.21.15.7:43772) 2024-01-27T02:43:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.869+0000 7fa02dc77700 1 -- 172.21.15.7:0/2788381470 learned_addr learned my addr 172.21.15.7:0/2788381470 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.870+0000 7fa02d476700 1 --2- 172.21.15.7:0/2788381470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0280aafc0 0x7fa02816e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.870+0000 7fa02cc75700 1 --2- 172.21.15.7:0/2788381470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0280c3a90 0x7fa0280c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.870+0000 7fa02dc77700 1 -- 172.21.15.7:0/2788381470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0280c3a90 msgr2=0x7fa0280c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.870+0000 7fa02dc77700 1 --2- 172.21.15.7:0/2788381470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0280c3a90 0x7fa0280c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.870+0000 7fa02dc77700 1 -- 172.21.15.7:0/2788381470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0280aafc0 msgr2=0x7fa02816e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:17.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.870+0000 7fa02dc77700 1 --2- 172.21.15.7:0/2788381470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0280aafc0 0x7fa02816e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:17.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.870+0000 7fa02dc77700 1 -- 172.21.15.7:0/2788381470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01c009420 con 0x7fa028159f40 2024-01-27T02:43:17.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.870+0000 7fa02dc77700 1 --2- 172.21.15.7:0/2788381470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa028159f40 0x7fa028156640 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fa020007f20 tx=0x7fa020009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:17.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.870+0000 7fa0197fa700 1 -- 172.21.15.7:0/2788381470 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02000eb90 con 0x7fa028159f40 2024-01-27T02:43:17.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.870+0000 7fa0197fa700 1 -- 172.21.15.7:0/2788381470 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02000ed10 con 0x7fa028159f40 2024-01-27T02:43:17.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.870+0000 7fa0197fa700 1 -- 172.21.15.7:0/2788381470 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa020014670 con 0x7fa028159f40 2024-01-27T02:43:17.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.870+0000 7fa02f6da700 1 -- 172.21.15.7:0/2788381470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0280c76c0 con 0x7fa028159f40 2024-01-27T02:43:17.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.870+0000 7fa02f6da700 1 -- 172.21.15.7:0/2788381470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa028156b80 con 0x7fa028159f40 2024-01-27T02:43:17.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.871+0000 7fa02f6da700 1 -- 172.21.15.7:0/2788381470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa02804c710 con 0x7fa028159f40 2024-01-27T02:43:17.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.872+0000 7fa0197fa700 1 -- 172.21.15.7:0/2788381470 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa020014a20 con 0x7fa028159f40 2024-01-27T02:43:17.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.872+0000 7fa0197fa700 1 --2- 172.21.15.7:0/2788381470 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa00c066f30 0x7fa00c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:17.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.872+0000 7fa0197fa700 1 -- 172.21.15.7:0/2788381470 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5734+0+0 (secure 0 0 0) 0x7fa020084b50 con 0x7fa028159f40 2024-01-27T02:43:17.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.873+0000 7fa02d476700 1 --2- 172.21.15.7:0/2788381470 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa00c066f30 0x7fa00c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:17.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.873+0000 7fa0197fa700 1 -- 172.21.15.7:0/2788381470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=169}) v3 -- 0x7fa00c06eab0 con 0x7fa028159f40 2024-01-27T02:43:17.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.873+0000 7fa02d476700 1 --2- 172.21.15.7:0/2788381470 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa00c066f30 0x7fa00c0693e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa01c0091e0 tx=0x7fa01c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:17.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:17.877+0000 7fa0197fa700 1 -- 172.21.15.7:0/2788381470 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa020051d00 con 0x7fa028159f40 2024-01-27T02:43:18.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.131+0000 7fa02f6da700 1 -- 172.21.15.7:0/2788381470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa0280cce40 con 0x7fa028159f40 2024-01-27T02:43:18.133 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-27T02:43:18.131+0000 7fb5c503a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-27T02:43:18.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.133+0000 7fa0197fa700 1 -- 172.21.15.7:0/2788381470 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(169..169 src has 1..169) v4 ==== 366+0+0 (secure 0 0 0) 0x7fa0200559c0 con 0x7fa028159f40 2024-01-27T02:43:18.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.133+0000 7fa0197fa700 1 -- 172.21.15.7:0/2788381470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=170}) v3 -- 0x7fa00c06f3a0 con 0x7fa028159f40 2024-01-27T02:43:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.136+0000 7fa0197fa700 1 -- 172.21.15.7:0/2788381470 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v87) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa020055320 con 0x7fa028159f40 2024-01-27T02:43:18.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.139+0000 7fa02f6da700 1 -- 172.21.15.7:0/2788381470 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa00c066f30 msgr2=0x7fa00c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.139+0000 7fa02f6da700 1 --2- 172.21.15.7:0/2788381470 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa00c066f30 0x7fa00c0693e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa01c0091e0 tx=0x7fa01c017000 comp rx=0 tx=0).stop 2024-01-27T02:43:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.139+0000 7fa02f6da700 1 -- 172.21.15.7:0/2788381470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa028159f40 msgr2=0x7fa028156640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.139+0000 7fa02f6da700 1 --2- 172.21.15.7:0/2788381470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa028159f40 0x7fa028156640 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fa020007f20 tx=0x7fa020009800 comp rx=0 tx=0).stop 2024-01-27T02:43:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.139+0000 7fa02f6da700 1 -- 172.21.15.7:0/2788381470 shutdown_connections 2024-01-27T02:43:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.139+0000 7fa02f6da700 1 --2- 172.21.15.7:0/2788381470 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa00c066f30 0x7fa00c0693e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.139+0000 7fa02f6da700 1 --2- 172.21.15.7:0/2788381470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa028159f40 0x7fa028156640 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.139+0000 7fa02f6da700 1 --2- 172.21.15.7:0/2788381470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0280aafc0 0x7fa02816e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.139+0000 7fa02f6da700 1 --2- 172.21.15.7:0/2788381470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0280c3a90 0x7fa0280c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.139+0000 7fa02f6da700 1 -- 172.21.15.7:0/2788381470 >> 172.21.15.7:0/2788381470 conn(0x7fa0280cdc30 msgr2=0x7fa02806ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.139+0000 7fa02f6da700 1 -- 172.21.15.7:0/2788381470 shutdown_connections 2024-01-27T02:43:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.139+0000 7fa02f6da700 1 -- 172.21.15.7:0/2788381470 wait complete. 2024-01-27T02:43:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-27T02:43:18.146 INFO:tasks.ceph.mds.c.smithi007.stderr: -8950> 2024-01-27T02:42:00.210+0000 7fb83d9a1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-27T02:43:18.159 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-27T02:43:18.183 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-27T02:43:18.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.296+0000 7f6f05092700 1 Processor -- start 2024-01-27T02:43:18.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.296+0000 7f6f05092700 1 -- start start 2024-01-27T02:43:18.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.297+0000 7f6f05092700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f001416c0 0x7f6f000cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:18.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.297+0000 7f6f05092700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6f000cb210 0x7f6f000cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:18.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.297+0000 7f6f05092700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f0015c7d0 con 0x7f6f00141a90 2024-01-27T02:43:18.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.297+0000 7f6f05092700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f000cd9c0 con 0x7f6f000cb210 2024-01-27T02:43:18.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.297+0000 7f6f05092700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f000c9640 con 0x7f6f001416c0 2024-01-27T02:43:18.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.297+0000 7f6eff59e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6f000cb210 0x7f6f000cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:18.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.297+0000 7f6efed9d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f001416c0 0x7f6f000cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:18.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.297+0000 7f6eff59e700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6f000cb210 0x7f6f000cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:59832/0 (socket says 172.21.15.7:59832) 2024-01-27T02:43:18.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.297+0000 7f6eff59e700 1 -- 172.21.15.7:0/647918762 learned_addr learned my addr 172.21.15.7:0/647918762 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:18.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.297+0000 7f6efed9d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f001416c0 0x7f6f000cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:46412/0 (socket says 172.21.15.7:46412) 2024-01-27T02:43:18.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.297+0000 7f6efe59c700 1 --1- 172.21.15.7:0/647918762 >> v1:172.21.15.188:6789/0 conn(0x7f6f00141a90 0x7f6f000cab00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:43556/0 (socket says 172.21.15.7:43556) 2024-01-27T02:43:18.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.297+0000 7f6eff59e700 1 -- 172.21.15.7:0/647918762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f001416c0 msgr2=0x7f6f000cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:18.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.297+0000 7f6eff59e700 1 --2- 172.21.15.7:0/647918762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f001416c0 0x7f6f000cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:18.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.297+0000 7f6eff59e700 1 -- 172.21.15.7:0/647918762 >> v1:172.21.15.188:6789/0 conn(0x7f6f00141a90 legacy=0x7f6f000cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:18.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.298+0000 7f6eff59e700 1 -- 172.21.15.7:0/647918762 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0015bbc0 con 0x7f6f000cb210 2024-01-27T02:43:18.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.298+0000 7f6eff59e700 1 --2- 172.21.15.7:0/647918762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f000cb210 0x7f6f000cbc20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009f20 tx=0x7f6ef8009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2132600e5fe9534 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:18.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.298+0000 7f6efdd9b700 1 -- 172.21.15.7:0/647918762 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef800e070 con 0x7f6f000cb210 2024-01-27T02:43:18.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.298+0000 7f6efdd9b700 1 -- 172.21.15.7:0/647918762 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef800b9d0 con 0x7f6f000cb210 2024-01-27T02:43:18.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.298+0000 7f6efdd9b700 1 -- 172.21.15.7:0/647918762 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef80092e0 con 0x7f6f000cb210 2024-01-27T02:43:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.298+0000 7f6f05092700 1 -- 172.21.15.7:0/647918762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f000cb210 msgr2=0x7f6f000cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.298+0000 7f6f05092700 1 --2- 172.21.15.7:0/647918762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f000cb210 0x7f6f000cbc20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009f20 tx=0x7f6ef8009840 comp rx=0 tx=0).stop 2024-01-27T02:43:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.298+0000 7f6f05092700 1 -- 172.21.15.7:0/647918762 shutdown_connections 2024-01-27T02:43:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.299+0000 7f6f05092700 1 --2- 172.21.15.7:0/647918762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f000cb210 0x7f6f000cbc20 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.299+0000 7f6f05092700 1 --2- 172.21.15.7:0/647918762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f001416c0 0x7f6f000cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.299+0000 7f6f05092700 1 -- 172.21.15.7:0/647918762 >> 172.21.15.7:0/647918762 conn(0x7f6f00063580 msgr2=0x7f6f00063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.299+0000 7f6f05092700 1 -- 172.21.15.7:0/647918762 shutdown_connections 2024-01-27T02:43:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.299+0000 7f6f05092700 1 -- 172.21.15.7:0/647918762 wait complete. 2024-01-27T02:43:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.300+0000 7f6f05092700 1 Processor -- start 2024-01-27T02:43:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.300+0000 7f6f05092700 1 -- start start 2024-01-27T02:43:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.301+0000 7f6f05092700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f000cb210 0x7f6f000c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.301+0000 7f6f05092700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f001416c0 0x7f6f00155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.301+0000 7f6f05092700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f00141a90 0x7f6f00156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.301+0000 7f6f05092700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0013e700 con 0x7f6f00141a90 2024-01-27T02:43:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.301+0000 7f6f05092700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f000681a0 con 0x7f6f000cb210 2024-01-27T02:43:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.301+0000 7f6f05092700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f000c6b70 con 0x7f6f001416c0 2024-01-27T02:43:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.301+0000 7f6eff59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f00141a90 0x7f6f00156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.301+0000 7f6eff59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f00141a90 0x7f6f00156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43802/0 (socket says 172.21.15.7:43802) 2024-01-27T02:43:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.301+0000 7f6eff59e700 1 -- 172.21.15.7:0/1396725700 learned_addr learned my addr 172.21.15.7:0/1396725700 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.301+0000 7f6efed9d700 1 --2- 172.21.15.7:0/1396725700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f000cb210 0x7f6f000c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.301+0000 7f6efe59c700 1 --2- 172.21.15.7:0/1396725700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f001416c0 0x7f6f00155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.301+0000 7f6eff59e700 1 -- 172.21.15.7:0/1396725700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f001416c0 msgr2=0x7f6f00155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.301+0000 7f6eff59e700 1 --2- 172.21.15.7:0/1396725700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f001416c0 0x7f6f00155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.301+0000 7f6eff59e700 1 -- 172.21.15.7:0/1396725700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f000cb210 msgr2=0x7f6f000c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.301+0000 7f6eff59e700 1 --2- 172.21.15.7:0/1396725700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f000cb210 0x7f6f000c6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.301+0000 7f6eff59e700 1 -- 172.21.15.7:0/1396725700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef8009ae0 con 0x7f6f00141a90 2024-01-27T02:43:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.301+0000 7f6eff59e700 1 --2- 172.21.15.7:0/1396725700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f00141a90 0x7f6f00156310 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800b610 tx=0x7f6ef8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.302+0000 7f6ee6ffd700 1 -- 172.21.15.7:0/1396725700 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8008c30 con 0x7f6f00141a90 2024-01-27T02:43:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.302+0000 7f6ee6ffd700 1 -- 172.21.15.7:0/1396725700 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef8008db0 con 0x7f6f00141a90 2024-01-27T02:43:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.302+0000 7f6f05092700 1 -- 172.21.15.7:0/1396725700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f0013e420 con 0x7f6f00141a90 2024-01-27T02:43:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.302+0000 7f6ee6ffd700 1 -- 172.21.15.7:0/1396725700 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef80256b0 con 0x7f6f00141a90 2024-01-27T02:43:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.302+0000 7f6f05092700 1 -- 172.21.15.7:0/1396725700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f00067e30 con 0x7f6f00141a90 2024-01-27T02:43:18.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.303+0000 7f6f05092700 1 -- 172.21.15.7:0/1396725700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f000679b0 con 0x7f6f00141a90 2024-01-27T02:43:18.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.303+0000 7f6ee6ffd700 1 -- 172.21.15.7:0/1396725700 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6ef8003040 con 0x7f6f00141a90 2024-01-27T02:43:18.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.303+0000 7f6ee6ffd700 1 --2- 172.21.15.7:0/1396725700 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6eec066f30 0x7f6eec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:18.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.304+0000 7f6ee6ffd700 1 -- 172.21.15.7:0/1396725700 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5820+0+0 (secure 0 0 0) 0x7f6ef8086360 con 0x7f6f00141a90 2024-01-27T02:43:18.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.304+0000 7f6ee6ffd700 1 -- 172.21.15.7:0/1396725700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=170}) v3 -- 0x7f6eec06eb50 con 0x7f6f00141a90 2024-01-27T02:43:18.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.304+0000 7f6efed9d700 1 --2- 172.21.15.7:0/1396725700 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6eec066f30 0x7f6eec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:18.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.304+0000 7f6efed9d700 1 --2- 172.21.15.7:0/1396725700 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6eec066f30 0x7f6eec0693e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0009d30 tx=0x7f6ef0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:18.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.308+0000 7f6ee6ffd700 1 -- 172.21.15.7:0/1396725700 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f000679b0 con 0x7f6f00141a90 2024-01-27T02:43:18.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:18.560+0000 7f6f05092700 1 -- 172.21.15.7:0/1396725700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6f00068350 con 0x7f6f00141a90 2024-01-27T02:43:19.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.136+0000 7f6ee6ffd700 1 -- 172.21.15.7:0/1396725700 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v88) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6f00068350 con 0x7f6f00141a90 2024-01-27T02:43:19.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.139+0000 7f6f05092700 1 -- 172.21.15.7:0/1396725700 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6eec066f30 msgr2=0x7f6eec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:19.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.139+0000 7f6f05092700 1 --2- 172.21.15.7:0/1396725700 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6eec066f30 0x7f6eec0693e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0009d30 tx=0x7f6ef0008040 comp rx=0 tx=0).stop 2024-01-27T02:43:19.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.139+0000 7f6f05092700 1 -- 172.21.15.7:0/1396725700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f00141a90 msgr2=0x7f6f00156310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:19.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.139+0000 7f6f05092700 1 --2- 172.21.15.7:0/1396725700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f00141a90 0x7f6f00156310 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800b610 tx=0x7f6ef8018000 comp rx=0 tx=0).stop 2024-01-27T02:43:19.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.139+0000 7f6f05092700 1 -- 172.21.15.7:0/1396725700 shutdown_connections 2024-01-27T02:43:19.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.139+0000 7f6f05092700 1 --2- 172.21.15.7:0/1396725700 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6eec066f30 0x7f6eec0693e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:19.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.139+0000 7f6f05092700 1 --2- 172.21.15.7:0/1396725700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f00141a90 0x7f6f00156310 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:19.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.139+0000 7f6f05092700 1 --2- 172.21.15.7:0/1396725700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f000cb210 0x7f6f000c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:19.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.139+0000 7f6f05092700 1 --2- 172.21.15.7:0/1396725700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f001416c0 0x7f6f00155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:19.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.139+0000 7f6f05092700 1 -- 172.21.15.7:0/1396725700 >> 172.21.15.7:0/1396725700 conn(0x7f6f00063580 msgr2=0x7f6f00063950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:19.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.139+0000 7f6f05092700 1 -- 172.21.15.7:0/1396725700 shutdown_connections 2024-01-27T02:43:19.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.139+0000 7f6f05092700 1 -- 172.21.15.7:0/1396725700 wait complete. 2024-01-27T02:43:19.156 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-27T02:43:19.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.338+0000 7fd0454c2700 1 Processor -- start 2024-01-27T02:43:19.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.338+0000 7fd0454c2700 1 -- start start 2024-01-27T02:43:19.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.339+0000 7fd0454c2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04015a450 0x7fd0400b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:19.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.339+0000 7fd0454c2700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd0400cdff0 0x7fd04015b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:19.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.339+0000 7fd0454c2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0400cb6e0 con 0x7fd0400cbf40 2024-01-27T02:43:19.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.339+0000 7fd0454c2700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd04015b980 con 0x7fd0400cdff0 2024-01-27T02:43:19.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.339+0000 7fd0454c2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0400cc390 con 0x7fd04015a450 2024-01-27T02:43:19.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.339+0000 7fd03e7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd0400cbf40 0x7fd0400cd8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46746/0 (socket says 172.21.15.7:46746) 2024-01-27T02:43:19.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.339+0000 7fd03e7fc700 1 -- 172.21.15.7:0/3935977706 learned_addr learned my addr 172.21.15.7:0/3935977706 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:19.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.339+0000 7fd03f7fe700 1 --2- 172.21.15.7:0/3935977706 >> v2:172.21.15.188:3300/0 conn(0x7fd0400cdff0 0x7fd04015b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.339+0000 7fd03dffb700 1 -- 172.21.15.7:0/3935977706 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432263326 0 0) 0x7fd030001110 con 0x7fd0400cbf40 2024-01-27T02:43:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.339+0000 7fd03effd700 1 --2- 172.21.15.7:0/3935977706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04015a450 0x7fd0400b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03dffb700 1 -- 172.21.15.7:0/3935977706 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd028003570 con 0x7fd0400cbf40 2024-01-27T02:43:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03dffb700 1 -- 172.21.15.7:0/3935977706 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4006701934 0 0) 0x7fd028003570 con 0x7fd0400cbf40 2024-01-27T02:43:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03dffb700 1 -- 172.21.15.7:0/3935977706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04015a450 msgr2=0x7fd0400b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03dffb700 1 --2- 172.21.15.7:0/3935977706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04015a450 0x7fd0400b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03dffb700 1 -- 172.21.15.7:0/3935977706 >> v2:172.21.15.188:3300/0 conn(0x7fd0400cdff0 msgr2=0x7fd04015b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03dffb700 1 --2- 172.21.15.7:0/3935977706 >> v2:172.21.15.188:3300/0 conn(0x7fd0400cdff0 0x7fd04015b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03dffb700 1 -- 172.21.15.7:0/3935977706 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04015afb0 con 0x7fd0400cbf40 2024-01-27T02:43:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03dffb700 1 -- 172.21.15.7:0/3935977706 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7fd030001600 con 0x7fd0400cbf40 2024-01-27T02:43:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03dffb700 1 -- 172.21.15.7:0/3935977706 >> v1:172.21.15.7:6789/0 conn(0x7fd0400cbf40 legacy=0x7fd0400cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03dffb700 1 --2- 172.21.15.7:0/3935977706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04015a450 0x7fd028003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:19.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03dffb700 1 --2- 172.21.15.7:0/3935977706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd028003fe0 0x7fd028004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:19.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03dffb700 1 --2- 172.21.15.7:0/3935977706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd028004d50 0x7fd0280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:19.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03dffb700 1 -- 172.21.15.7:0/3935977706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04015b980 con 0x7fd028003fe0 2024-01-27T02:43:19.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03dffb700 1 -- 172.21.15.7:0/3935977706 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0400cc390 con 0x7fd028004d50 2024-01-27T02:43:19.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03dffb700 1 -- 172.21.15.7:0/3935977706 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02800ba90 con 0x7fd04015a450 2024-01-27T02:43:19.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03e7fc700 1 --2- 172.21.15.7:0/3935977706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd028003fe0 0x7fd028004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:19.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03f7fe700 1 --2- 172.21.15.7:0/3935977706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd028004d50 0x7fd0280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:19.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03effd700 1 --2- 172.21.15.7:0/3935977706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04015a450 0x7fd028003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:19.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03e7fc700 1 -- 172.21.15.7:0/3935977706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04015a450 msgr2=0x7fd028003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:19.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03e7fc700 1 --2- 172.21.15.7:0/3935977706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04015a450 0x7fd028003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:19.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03e7fc700 1 -- 172.21.15.7:0/3935977706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd028004d50 msgr2=0x7fd0280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:19.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03e7fc700 1 --2- 172.21.15.7:0/3935977706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd028004d50 0x7fd0280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03e7fc700 1 -- 172.21.15.7:0/3935977706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd028003570 con 0x7fd028003fe0 2024-01-27T02:43:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.340+0000 7fd03effd700 1 --2- 172.21.15.7:0/3935977706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04015a450 0x7fd028003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.341+0000 7fd03e7fc700 1 --2- 172.21.15.7:0/3935977706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd028003fe0 0x7fd028004810 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fd03000c5f0 tx=0x7fd030007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8574fd2ff1631450 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.341+0000 7fd03dffb700 1 -- 172.21.15.7:0/3935977706 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03000cd90 con 0x7fd028003fe0 2024-01-27T02:43:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.341+0000 7fd03dffb700 1 -- 172.21.15.7:0/3935977706 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd030005920 con 0x7fd028003fe0 2024-01-27T02:43:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.341+0000 7fd03dffb700 1 -- 172.21.15.7:0/3935977706 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd030005ce0 con 0x7fd028003fe0 2024-01-27T02:43:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.341+0000 7fd0454c2700 1 -- 172.21.15.7:0/3935977706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd028003fe0 msgr2=0x7fd028004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.341+0000 7fd0454c2700 1 --2- 172.21.15.7:0/3935977706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd028003fe0 0x7fd028004810 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fd03000c5f0 tx=0x7fd030007a80 comp rx=0 tx=0).stop 2024-01-27T02:43:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.341+0000 7fd03f7fe700 1 -- 172.21.15.7:0/3935977706 reap_dead start 2024-01-27T02:43:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.341+0000 7fd0454c2700 1 -- 172.21.15.7:0/3935977706 shutdown_connections 2024-01-27T02:43:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.341+0000 7fd0454c2700 1 -- 172.21.15.7:0/3935977706 >> 172.21.15.7:0/3935977706 conn(0x7fd040068440 msgr2=0x7fd040063580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.341+0000 7fd0454c2700 1 -- 172.21.15.7:0/3935977706 shutdown_connections 2024-01-27T02:43:19.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.341+0000 7fd0454c2700 1 -- 172.21.15.7:0/3935977706 wait complete. 2024-01-27T02:43:19.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.343+0000 7fd0454c2700 1 Processor -- start 2024-01-27T02:43:19.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.343+0000 7fd0454c2700 1 -- start start 2024-01-27T02:43:19.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.344+0000 7fd0454c2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd028003fe0 0x7fd0400b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:19.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.344+0000 7fd0454c2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd040155380 0x7fd0400c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:19.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.344+0000 7fd0454c2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0401557f0 0x7fd0400c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:19.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.344+0000 7fd0454c2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0400c89d0 con 0x7fd0401557f0 2024-01-27T02:43:19.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.344+0000 7fd0454c2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0400b3cb0 con 0x7fd040155380 2024-01-27T02:43:19.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.344+0000 7fd0454c2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0400c8be0 con 0x7fd028003fe0 2024-01-27T02:43:19.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.344+0000 7fd03f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0401557f0 0x7fd0400c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:19.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.344+0000 7fd03f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0401557f0 0x7fd0400c9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43806/0 (socket says 172.21.15.7:43806) 2024-01-27T02:43:19.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.344+0000 7fd03effd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd028003fe0 0x7fd0400b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:19.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.344+0000 7fd03f7fe700 1 -- 172.21.15.7:0/870505071 learned_addr learned my addr 172.21.15.7:0/870505071 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:19.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.345+0000 7fd03e7fc700 1 --2- 172.21.15.7:0/870505071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd040155380 0x7fd0400c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:19.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.345+0000 7fd03f7fe700 1 -- 172.21.15.7:0/870505071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd028003fe0 msgr2=0x7fd0400b5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:19.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.345+0000 7fd03f7fe700 1 --2- 172.21.15.7:0/870505071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd028003fe0 0x7fd0400b5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:19.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.345+0000 7fd03f7fe700 1 -- 172.21.15.7:0/870505071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd040155380 msgr2=0x7fd0400c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:19.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.345+0000 7fd03f7fe700 1 --2- 172.21.15.7:0/870505071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd040155380 0x7fd0400c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:19.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.345+0000 7fd03f7fe700 1 -- 172.21.15.7:0/870505071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0400c8ec0 con 0x7fd0401557f0 2024-01-27T02:43:19.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.345+0000 7fd03f7fe700 1 --2- 172.21.15.7:0/870505071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0401557f0 0x7fd0400c9f40 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fd0400d2020 tx=0x7fd03800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:19.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.345+0000 7fd026ffd700 1 -- 172.21.15.7:0/870505071 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03800e040 con 0x7fd0401557f0 2024-01-27T02:43:19.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.345+0000 7fd026ffd700 1 -- 172.21.15.7:0/870505071 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03800f040 con 0x7fd0401557f0 2024-01-27T02:43:19.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.345+0000 7fd026ffd700 1 -- 172.21.15.7:0/870505071 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03800fd70 con 0x7fd0401557f0 2024-01-27T02:43:19.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.345+0000 7fd0454c2700 1 -- 172.21.15.7:0/870505071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0400c91a0 con 0x7fd0401557f0 2024-01-27T02:43:19.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.345+0000 7fd0454c2700 1 -- 172.21.15.7:0/870505071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0400b6400 con 0x7fd0401557f0 2024-01-27T02:43:19.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.346+0000 7fd0454c2700 1 -- 172.21.15.7:0/870505071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0400b3ed0 con 0x7fd0401557f0 2024-01-27T02:43:19.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.347+0000 7fd026ffd700 1 -- 172.21.15.7:0/870505071 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd038015640 con 0x7fd0401557f0 2024-01-27T02:43:19.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.347+0000 7fd026ffd700 1 --2- 172.21.15.7:0/870505071 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd02c066fb0 0x7fd02c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:19.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.347+0000 7fd026ffd700 1 -- 172.21.15.7:0/870505071 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5820+0+0 (secure 0 0 0) 0x7fd038084dd0 con 0x7fd0401557f0 2024-01-27T02:43:19.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.347+0000 7fd026ffd700 1 -- 172.21.15.7:0/870505071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=170}) v3 -- 0x7fd02c06ebd0 con 0x7fd0401557f0 2024-01-27T02:43:19.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.347+0000 7fd03effd700 1 --2- 172.21.15.7:0/870505071 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd02c066fb0 0x7fd02c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:19.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.348+0000 7fd03effd700 1 --2- 172.21.15.7:0/870505071 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd02c066fb0 0x7fd02c069460 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd034009e00 tx=0x7fd034008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:19.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.352+0000 7fd026ffd700 1 -- 172.21.15.7:0/870505071 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0400b3ed0 con 0x7fd0401557f0 2024-01-27T02:43:19.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.553+0000 7fd0454c2700 1 -- 172.21.15.7:0/870505071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd04006dfd0 con 0x7fd0401557f0 2024-01-27T02:43:19.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.928+0000 7fd026ffd700 1 -- 172.21.15.7:0/870505071 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(170..170 src has 1..170) v4 ==== 679+0+0 (secure 0 0 0) 0x7fd0380562f0 con 0x7fd0401557f0 2024-01-27T02:43:19.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.928+0000 7fd026ffd700 1 -- 172.21.15.7:0/870505071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=171}) v3 -- 0x7fd02c06d000 con 0x7fd0401557f0 2024-01-27T02:43:19.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.932+0000 7fd026ffd700 1 -- 172.21.15.7:0/870505071 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v170) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd038089070 con 0x7fd0401557f0 2024-01-27T02:43:19.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.938+0000 7fd0454c2700 1 -- 172.21.15.7:0/870505071 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd02c066fb0 msgr2=0x7fd02c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:19.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.938+0000 7fd0454c2700 1 --2- 172.21.15.7:0/870505071 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd02c066fb0 0x7fd02c069460 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd034009e00 tx=0x7fd034008040 comp rx=0 tx=0).stop 2024-01-27T02:43:19.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.938+0000 7fd0454c2700 1 -- 172.21.15.7:0/870505071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0401557f0 msgr2=0x7fd0400c9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:19.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.938+0000 7fd0454c2700 1 --2- 172.21.15.7:0/870505071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0401557f0 0x7fd0400c9f40 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fd0400d2020 tx=0x7fd03800b040 comp rx=0 tx=0).stop 2024-01-27T02:43:19.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.938+0000 7fd0454c2700 1 -- 172.21.15.7:0/870505071 shutdown_connections 2024-01-27T02:43:19.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.938+0000 7fd0454c2700 1 --2- 172.21.15.7:0/870505071 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd02c066fb0 0x7fd02c069460 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:19.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.938+0000 7fd0454c2700 1 --2- 172.21.15.7:0/870505071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0401557f0 0x7fd0400c9f40 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:19.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.938+0000 7fd0454c2700 1 --2- 172.21.15.7:0/870505071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd028003fe0 0x7fd0400b5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:19.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.938+0000 7fd0454c2700 1 --2- 172.21.15.7:0/870505071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd040155380 0x7fd0400c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:19.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.939+0000 7fd0454c2700 1 -- 172.21.15.7:0/870505071 >> 172.21.15.7:0/870505071 conn(0x7fd040068440 msgr2=0x7fd0401599f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:19.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.939+0000 7fd0454c2700 1 -- 172.21.15.7:0/870505071 shutdown_connections 2024-01-27T02:43:19.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:19.939+0000 7fd0454c2700 1 -- 172.21.15.7:0/870505071 wait complete. 2024-01-27T02:43:19.942 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-27T02:43:19.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-27T02:43:20.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.096+0000 7f0e88b67700 1 Processor -- start 2024-01-27T02:43:20.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.096+0000 7f0e88b67700 1 -- start start 2024-01-27T02:43:20.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.097+0000 7f0e88b67700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e8415d7c0 0x7f0e84063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:20.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.097+0000 7f0e88b67700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0e84063ac0 0x7f0e840aacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:20.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.097+0000 7f0e88b67700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e840cbf10 con 0x7f0e8415d3f0 2024-01-27T02:43:20.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.097+0000 7f0e88b67700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e84063f40 con 0x7f0e84063ac0 2024-01-27T02:43:20.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.097+0000 7f0e88b67700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e840cc1e0 con 0x7f0e8415d7c0 2024-01-27T02:43:20.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.098+0000 7f0e82d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0e84063ac0 0x7f0e840aacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:20.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.098+0000 7f0e8259c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0e8415d3f0 0x7f0e840cb800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46762/0 (socket says 172.21.15.7:46762) 2024-01-27T02:43:20.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.098+0000 7f0e82d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0e84063ac0 0x7f0e840aacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43820/0 (socket says 172.21.15.7:43820) 2024-01-27T02:43:20.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.098+0000 7f0e8259c700 1 -- 172.21.15.7:0/3368937072 learned_addr learned my addr 172.21.15.7:0/3368937072 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:20.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.098+0000 7f0e81d9b700 1 --2- 172.21.15.7:0/3368937072 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e8415d7c0 0x7f0e84063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:20.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.098+0000 7f0e82d9d700 1 -- 172.21.15.7:0/3368937072 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e8415d7c0 msgr2=0x7f0e84063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.098+0000 7f0e82d9d700 1 --2- 172.21.15.7:0/3368937072 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e8415d7c0 0x7f0e84063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.098+0000 7f0e82d9d700 1 -- 172.21.15.7:0/3368937072 >> v1:172.21.15.7:6789/0 conn(0x7f0e8415d3f0 legacy=0x7f0e840cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.098+0000 7f0e82d9d700 1 -- 172.21.15.7:0/3368937072 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e840cc4c0 con 0x7f0e84063ac0 2024-01-27T02:43:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.098+0000 7f0e8159a700 1 -- 172.21.15.7:0/3368937072 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3103586601 0 0) 0x7f0e840cbf10 con 0x7f0e8415d3f0 2024-01-27T02:43:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.098+0000 7f0e82d9d700 1 --2- 172.21.15.7:0/3368937072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e84063ac0 0x7f0e840aacb0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009e60 tx=0x7f0e7c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2942ac3f0bfd64 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.098+0000 7f0e8159a700 1 -- 172.21.15.7:0/3368937072 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c00e070 con 0x7f0e84063ac0 2024-01-27T02:43:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.098+0000 7f0e8159a700 1 -- 172.21.15.7:0/3368937072 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7c00bc30 con 0x7f0e84063ac0 2024-01-27T02:43:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.098+0000 7f0e8159a700 1 -- 172.21.15.7:0/3368937072 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c012430 con 0x7f0e84063ac0 2024-01-27T02:43:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.099+0000 7f0e88b67700 1 -- 172.21.15.7:0/3368937072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e84063ac0 msgr2=0x7f0e840aacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.099+0000 7f0e88b67700 1 --2- 172.21.15.7:0/3368937072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e84063ac0 0x7f0e840aacb0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009e60 tx=0x7f0e7c009780 comp rx=0 tx=0).stop 2024-01-27T02:43:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.099+0000 7f0e88b67700 1 -- 172.21.15.7:0/3368937072 shutdown_connections 2024-01-27T02:43:20.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.099+0000 7f0e88b67700 1 --2- 172.21.15.7:0/3368937072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e84063ac0 0x7f0e840aacb0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:20.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.099+0000 7f0e88b67700 1 --2- 172.21.15.7:0/3368937072 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e8415d7c0 0x7f0e84063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:20.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.099+0000 7f0e88b67700 1 -- 172.21.15.7:0/3368937072 >> 172.21.15.7:0/3368937072 conn(0x7f0e840cdc30 msgr2=0x7f0e840ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:20.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.099+0000 7f0e88b67700 1 -- 172.21.15.7:0/3368937072 shutdown_connections 2024-01-27T02:43:20.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.099+0000 7f0e88b67700 1 -- 172.21.15.7:0/3368937072 wait complete. 2024-01-27T02:43:20.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.100+0000 7f0e88b67700 1 Processor -- start 2024-01-27T02:43:20.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.100+0000 7f0e88b67700 1 -- start start 2024-01-27T02:43:20.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.101+0000 7f0e88b67700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e84063ac0 0x7f0e840b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:20.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.101+0000 7f0e88b67700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8415d3f0 0x7f0e8406c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:20.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.101+0000 7f0e88b67700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e8415d7c0 0x7f0e840b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:20.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.101+0000 7f0e88b67700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e84155c60 con 0x7f0e8415d3f0 2024-01-27T02:43:20.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.101+0000 7f0e88b67700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e840b2e30 con 0x7f0e84063ac0 2024-01-27T02:43:20.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.101+0000 7f0e88b67700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8406cc60 con 0x7f0e8415d7c0 2024-01-27T02:43:20.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.101+0000 7f0e81d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8415d3f0 0x7f0e8406c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:20.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.101+0000 7f0e81d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8415d3f0 0x7f0e8406c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43826/0 (socket says 172.21.15.7:43826) 2024-01-27T02:43:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.101+0000 7f0e81d9b700 1 -- 172.21.15.7:0/3074537980 learned_addr learned my addr 172.21.15.7:0/3074537980 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.101+0000 7f0e8259c700 1 --2- 172.21.15.7:0/3074537980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e84063ac0 0x7f0e840b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.101+0000 7f0e82d9d700 1 --2- 172.21.15.7:0/3074537980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e8415d7c0 0x7f0e840b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.102+0000 7f0e81d9b700 1 -- 172.21.15.7:0/3074537980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e8415d7c0 msgr2=0x7f0e840b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.102+0000 7f0e81d9b700 1 --2- 172.21.15.7:0/3074537980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e8415d7c0 0x7f0e840b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.102+0000 7f0e81d9b700 1 -- 172.21.15.7:0/3074537980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e84063ac0 msgr2=0x7f0e840b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.102+0000 7f0e81d9b700 1 --2- 172.21.15.7:0/3074537980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e84063ac0 0x7f0e840b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:20.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.102+0000 7f0e81d9b700 1 -- 172.21.15.7:0/3074537980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7c009a20 con 0x7f0e8415d3f0 2024-01-27T02:43:20.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.102+0000 7f0e81d9b700 1 --2- 172.21.15.7:0/3074537980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8415d3f0 0x7f0e8406c720 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00d890 tx=0x7f0e6c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:20.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.102+0000 7f0e7a7fc700 1 -- 172.21.15.7:0/3074537980 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0125f0 con 0x7f0e8415d3f0 2024-01-27T02:43:20.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.102+0000 7f0e7a7fc700 1 -- 172.21.15.7:0/3074537980 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c012770 con 0x7f0e8415d3f0 2024-01-27T02:43:20.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.102+0000 7f0e88b67700 1 -- 172.21.15.7:0/3074537980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e840b3800 con 0x7f0e8415d3f0 2024-01-27T02:43:20.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.102+0000 7f0e7a7fc700 1 -- 172.21.15.7:0/3074537980 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c012b30 con 0x7f0e8415d3f0 2024-01-27T02:43:20.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.102+0000 7f0e88b67700 1 -- 172.21.15.7:0/3074537980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e840b2220 con 0x7f0e8415d3f0 2024-01-27T02:43:20.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.103+0000 7f0e88b67700 1 -- 172.21.15.7:0/3074537980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e840b2fc0 con 0x7f0e8415d3f0 2024-01-27T02:43:20.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.103+0000 7f0e7a7fc700 1 -- 172.21.15.7:0/3074537980 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0e6c01e460 con 0x7f0e8415d3f0 2024-01-27T02:43:20.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.104+0000 7f0e7a7fc700 1 --2- 172.21.15.7:0/3074537980 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0e64066f30 0x7f0e640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:20.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.104+0000 7f0e7a7fc700 1 -- 172.21.15.7:0/3074537980 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f0e6c084a70 con 0x7f0e8415d3f0 2024-01-27T02:43:20.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.104+0000 7f0e8259c700 1 --2- 172.21.15.7:0/3074537980 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0e64066f30 0x7f0e640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:20.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.105+0000 7f0e8259c700 1 --2- 172.21.15.7:0/3074537980 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0e64066f30 0x7f0e640693e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0e740011e0 tx=0x7f0e74009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:20.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.108+0000 7f0e7a7fc700 1 -- 172.21.15.7:0/3074537980 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e6c051e20 con 0x7f0e8415d3f0 2024-01-27T02:43:20.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:20.314+0000 7f0e88b67700 1 -- 172.21.15.7:0/3074537980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0e8404c710 con 0x7f0e8415d3f0 2024-01-27T02:43:21.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.147+0000 7f0e7a7fc700 1 -- 172.21.15.7:0/3074537980 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v171) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0e6c055440 con 0x7f0e8415d3f0 2024-01-27T02:43:21.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.156+0000 7f0e88b67700 1 -- 172.21.15.7:0/3074537980 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0e64066f30 msgr2=0x7f0e640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:21.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.156+0000 7f0e88b67700 1 --2- 172.21.15.7:0/3074537980 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0e64066f30 0x7f0e640693e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0e740011e0 tx=0x7f0e74009040 comp rx=0 tx=0).stop 2024-01-27T02:43:21.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.156+0000 7f0e88b67700 1 -- 172.21.15.7:0/3074537980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8415d3f0 msgr2=0x7f0e8406c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:21.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.156+0000 7f0e88b67700 1 --2- 172.21.15.7:0/3074537980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8415d3f0 0x7f0e8406c720 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00d890 tx=0x7f0e6c008040 comp rx=0 tx=0).stop 2024-01-27T02:43:21.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.156+0000 7f0e88b67700 1 -- 172.21.15.7:0/3074537980 shutdown_connections 2024-01-27T02:43:21.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.156+0000 7f0e88b67700 1 --2- 172.21.15.7:0/3074537980 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0e64066f30 0x7f0e640693e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:21.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.156+0000 7f0e88b67700 1 --2- 172.21.15.7:0/3074537980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e8415d7c0 0x7f0e840b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:21.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.156+0000 7f0e88b67700 1 --2- 172.21.15.7:0/3074537980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e84063ac0 0x7f0e840b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:21.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.156+0000 7f0e88b67700 1 --2- 172.21.15.7:0/3074537980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8415d3f0 0x7f0e8406c720 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:21.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.156+0000 7f0e88b67700 1 -- 172.21.15.7:0/3074537980 >> 172.21.15.7:0/3074537980 conn(0x7f0e840cdc30 msgr2=0x7f0e8415c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:21.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.156+0000 7f0e88b67700 1 -- 172.21.15.7:0/3074537980 shutdown_connections 2024-01-27T02:43:21.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.156+0000 7f0e88b67700 1 -- 172.21.15.7:0/3074537980 wait complete. 2024-01-27T02:43:21.161 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-27T02:43:21.173 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-27T02:43:21.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.310+0000 7f0c71d03700 1 Processor -- start 2024-01-27T02:43:21.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.310+0000 7f0c71d03700 1 -- start start 2024-01-27T02:43:21.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.311+0000 7f0c71d03700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0c6c141830 0x7f0c6c0ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:21.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.311+0000 7f0c71d03700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c6c06bc00 0x7f0c6c0b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:21.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.311+0000 7f0c71d03700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c6c0cd8a0 con 0x7f0c6c0ce8d0 2024-01-27T02:43:21.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.311+0000 7f0c71d03700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c6c0ccc90 con 0x7f0c6c141830 2024-01-27T02:43:21.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.311+0000 7f0c71d03700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c0ccf70 con 0x7f0c6c06bc00 2024-01-27T02:43:21.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.311+0000 7f0c6b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0c6c141830 0x7f0c6c0ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:21.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.311+0000 7f0c6affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0c6c0ce8d0 0x7f0c6c06b4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46764/0 (socket says 172.21.15.7:46764) 2024-01-27T02:43:21.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.311+0000 7f0c6affd700 1 -- 172.21.15.7:0/1432007408 learned_addr learned my addr 172.21.15.7:0/1432007408 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:21.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.311+0000 7f0c6bfff700 1 --2- 172.21.15.7:0/1432007408 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c6c06bc00 0x7f0c6c0b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:21.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.311+0000 7f0c6b7fe700 1 -- 172.21.15.7:0/1432007408 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c6c06bc00 msgr2=0x7f0c6c0b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:21.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.311+0000 7f0c6b7fe700 1 --2- 172.21.15.7:0/1432007408 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c6c06bc00 0x7f0c6c0b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:21.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.311+0000 7f0c6b7fe700 1 -- 172.21.15.7:0/1432007408 >> v1:172.21.15.7:6789/0 conn(0x7f0c6c0ce8d0 legacy=0x7f0c6c06b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:21.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.311+0000 7f0c6a7fc700 1 -- 172.21.15.7:0/1432007408 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2123203861 0 0) 0x7f0c6c0cd8a0 con 0x7f0c6c0ce8d0 2024-01-27T02:43:21.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.311+0000 7f0c6b7fe700 1 -- 172.21.15.7:0/1432007408 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6c0cd250 con 0x7f0c6c141830 2024-01-27T02:43:21.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.311+0000 7f0c6b7fe700 1 --2- 172.21.15.7:0/1432007408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c6c141830 0x7f0c6c0ce390 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0098e0 tx=0x7f0c5c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e33cbda5c58486 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:21.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.312+0000 7f0c6a7fc700 1 -- 172.21.15.7:0/1432007408 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c00b640 con 0x7f0c6c141830 2024-01-27T02:43:21.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.312+0000 7f0c6a7fc700 1 -- 172.21.15.7:0/1432007408 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5c00b7a0 con 0x7f0c6c141830 2024-01-27T02:43:21.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.312+0000 7f0c6a7fc700 1 -- 172.21.15.7:0/1432007408 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c00bb40 con 0x7f0c6c141830 2024-01-27T02:43:21.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.312+0000 7f0c71d03700 1 -- 172.21.15.7:0/1432007408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c6c141830 msgr2=0x7f0c6c0ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:21.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.312+0000 7f0c71d03700 1 --2- 172.21.15.7:0/1432007408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c6c141830 0x7f0c6c0ce390 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0098e0 tx=0x7f0c5c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:21.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.312+0000 7f0c71d03700 1 -- 172.21.15.7:0/1432007408 shutdown_connections 2024-01-27T02:43:21.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.312+0000 7f0c71d03700 1 --2- 172.21.15.7:0/1432007408 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c6c06bc00 0x7f0c6c0b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:21.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.312+0000 7f0c71d03700 1 --2- 172.21.15.7:0/1432007408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c6c141830 0x7f0c6c0ce390 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:21.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.312+0000 7f0c71d03700 1 -- 172.21.15.7:0/1432007408 >> 172.21.15.7:0/1432007408 conn(0x7f0c6c064050 msgr2=0x7f0c6c0ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:21.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.312+0000 7f0c71d03700 1 -- 172.21.15.7:0/1432007408 shutdown_connections 2024-01-27T02:43:21.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.312+0000 7f0c71d03700 1 -- 172.21.15.7:0/1432007408 wait complete. 2024-01-27T02:43:21.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.314+0000 7f0c71d03700 1 Processor -- start 2024-01-27T02:43:21.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.314+0000 7f0c71d03700 1 -- start start 2024-01-27T02:43:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.314+0000 7f0c71d03700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c6c06bc00 0x7f0c6c156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.314+0000 7f0c71d03700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c6c0ce8d0 0x7f0c6c15e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.315+0000 7f0c71d03700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c6c141830 0x7f0c6c1589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.315+0000 7f0c71d03700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c0685a0 con 0x7f0c6c0ce8d0 2024-01-27T02:43:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.315+0000 7f0c71d03700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c0679b0 con 0x7f0c6c141830 2024-01-27T02:43:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.315+0000 7f0c71d03700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c157050 con 0x7f0c6c06bc00 2024-01-27T02:43:21.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.315+0000 7f0c6affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c6c0ce8d0 0x7f0c6c15e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:21.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.315+0000 7f0c6affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c6c0ce8d0 0x7f0c6c15e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43846/0 (socket says 172.21.15.7:43846) 2024-01-27T02:43:21.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.315+0000 7f0c6affd700 1 -- 172.21.15.7:0/2976946427 learned_addr learned my addr 172.21.15.7:0/2976946427 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:21.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.315+0000 7f0c6b7fe700 1 --2- 172.21.15.7:0/2976946427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c6c06bc00 0x7f0c6c156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:21.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.315+0000 7f0c6bfff700 1 --2- 172.21.15.7:0/2976946427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c6c141830 0x7f0c6c1589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:21.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.315+0000 7f0c6affd700 1 -- 172.21.15.7:0/2976946427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c6c06bc00 msgr2=0x7f0c6c156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:21.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.315+0000 7f0c6affd700 1 --2- 172.21.15.7:0/2976946427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c6c06bc00 0x7f0c6c156b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:21.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.315+0000 7f0c6affd700 1 -- 172.21.15.7:0/2976946427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c6c141830 msgr2=0x7f0c6c1589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:21.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.315+0000 7f0c6affd700 1 --2- 172.21.15.7:0/2976946427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c6c141830 0x7f0c6c1589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:21.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.315+0000 7f0c6affd700 1 -- 172.21.15.7:0/2976946427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c009420 con 0x7f0c6c0ce8d0 2024-01-27T02:43:21.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.315+0000 7f0c6affd700 1 --2- 172.21.15.7:0/2976946427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c6c0ce8d0 0x7f0c6c15e3a0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000c7e0 tx=0x7f0c60007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:21.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.315+0000 7f0c537fe700 1 -- 172.21.15.7:0/2976946427 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c60005c00 con 0x7f0c6c0ce8d0 2024-01-27T02:43:21.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.315+0000 7f0c537fe700 1 -- 172.21.15.7:0/2976946427 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c60005d60 con 0x7f0c6c0ce8d0 2024-01-27T02:43:21.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.315+0000 7f0c71d03700 1 -- 172.21.15.7:0/2976946427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c6c067c70 con 0x7f0c6c0ce8d0 2024-01-27T02:43:21.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.315+0000 7f0c537fe700 1 -- 172.21.15.7:0/2976946427 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c600072e0 con 0x7f0c6c0ce8d0 2024-01-27T02:43:21.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.315+0000 7f0c71d03700 1 -- 172.21.15.7:0/2976946427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c6c068230 con 0x7f0c6c0ce8d0 2024-01-27T02:43:21.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.316+0000 7f0c71d03700 1 -- 172.21.15.7:0/2976946427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c6c072060 con 0x7f0c6c0ce8d0 2024-01-27T02:43:21.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.317+0000 7f0c537fe700 1 -- 172.21.15.7:0/2976946427 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0c6001b3b0 con 0x7f0c6c0ce8d0 2024-01-27T02:43:21.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.317+0000 7f0c537fe700 1 --2- 172.21.15.7:0/2976946427 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0c58066f30 0x7f0c580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:21.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.317+0000 7f0c537fe700 1 -- 172.21.15.7:0/2976946427 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0c600836a0 con 0x7f0c6c0ce8d0 2024-01-27T02:43:21.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.318+0000 7f0c6b7fe700 1 --2- 172.21.15.7:0/2976946427 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0c58066f30 0x7f0c580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:21.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.318+0000 7f0c6b7fe700 1 --2- 172.21.15.7:0/2976946427 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0c58066f30 0x7f0c580693e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0093f0 tx=0x7f0c5c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:21.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.322+0000 7f0c537fe700 1 -- 172.21.15.7:0/2976946427 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c6004b5a0 con 0x7f0c6c0ce8d0 2024-01-27T02:43:21.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.528+0000 7f0c71d03700 1 -- 172.21.15.7:0/2976946427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0c6c0cd530 con 0x7f0c6c0ce8d0 2024-01-27T02:43:21.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.532+0000 7f0c537fe700 1 -- 172.21.15.7:0/2976946427 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0c60050cc0 con 0x7f0c6c0ce8d0 2024-01-27T02:43:21.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.535+0000 7f0c71d03700 1 -- 172.21.15.7:0/2976946427 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0c58066f30 msgr2=0x7f0c580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:21.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.535+0000 7f0c71d03700 1 --2- 172.21.15.7:0/2976946427 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0c58066f30 0x7f0c580693e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0093f0 tx=0x7f0c5c017000 comp rx=0 tx=0).stop 2024-01-27T02:43:21.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.535+0000 7f0c71d03700 1 -- 172.21.15.7:0/2976946427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c6c0ce8d0 msgr2=0x7f0c6c15e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:21.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.535+0000 7f0c71d03700 1 --2- 172.21.15.7:0/2976946427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c6c0ce8d0 0x7f0c6c15e3a0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000c7e0 tx=0x7f0c60007d90 comp rx=0 tx=0).stop 2024-01-27T02:43:21.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.535+0000 7f0c71d03700 1 -- 172.21.15.7:0/2976946427 shutdown_connections 2024-01-27T02:43:21.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.535+0000 7f0c71d03700 1 --2- 172.21.15.7:0/2976946427 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0c58066f30 0x7f0c580693e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:21.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.535+0000 7f0c71d03700 1 --2- 172.21.15.7:0/2976946427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c6c141830 0x7f0c6c1589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:21.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.535+0000 7f0c71d03700 1 --2- 172.21.15.7:0/2976946427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c6c06bc00 0x7f0c6c156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:21.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.535+0000 7f0c71d03700 1 --2- 172.21.15.7:0/2976946427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c6c0ce8d0 0x7f0c6c15e3a0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:21.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.535+0000 7f0c71d03700 1 -- 172.21.15.7:0/2976946427 >> 172.21.15.7:0/2976946427 conn(0x7f0c6c064050 msgr2=0x7f0c6c140320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:21.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.535+0000 7f0c71d03700 1 -- 172.21.15.7:0/2976946427 shutdown_connections 2024-01-27T02:43:21.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.535+0000 7f0c71d03700 1 -- 172.21.15.7:0/2976946427 wait complete. 2024-01-27T02:43:21.550 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2024-01-27T02:43:21.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.682+0000 7ff3fe4ad700 1 Processor -- start 2024-01-27T02:43:21.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.682+0000 7ff3fe4ad700 1 -- start start 2024-01-27T02:43:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.683+0000 7ff3fe4ad700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3f815be10 0x7ff3f80b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.683+0000 7ff3fe4ad700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff3f815c1e0 0x7ff3f80c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.683+0000 7ff3fe4ad700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3f80b1470 con 0x7ff3f80b0180 2024-01-27T02:43:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.683+0000 7ff3fe4ad700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3f80b05d0 con 0x7ff3f815c1e0 2024-01-27T02:43:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.683+0000 7ff3fe4ad700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f80b0860 con 0x7ff3f815be10 2024-01-27T02:43:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.683+0000 7ff3f7fff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3f815be10 0x7ff3f80b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.683+0000 7ff3fca4a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff3f815c1e0 0x7ff3f80c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.683+0000 7ff3fca4a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff3f815c1e0 0x7ff3f80c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:44976/0 (socket says 172.21.15.7:44976) 2024-01-27T02:43:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.683+0000 7ff3f7fff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3f815be10 0x7ff3f80b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:56138/0 (socket says 172.21.15.7:56138) 2024-01-27T02:43:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.683+0000 7ff3fca4a700 1 -- 172.21.15.7:0/3169149802 learned_addr learned my addr 172.21.15.7:0/3169149802 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.683+0000 7ff3f77fe700 1 --1- 172.21.15.7:0/3169149802 >> v1:172.21.15.188:6789/0 conn(0x7ff3f80b0180 0x7ff3f80aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:47718/0 (socket says 172.21.15.7:47718) 2024-01-27T02:43:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.684+0000 7ff3f7fff700 1 -- 172.21.15.7:0/3169149802 >> v2:172.21.15.188:3300/0 conn(0x7ff3f815c1e0 msgr2=0x7ff3f80c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.684+0000 7ff3f7fff700 1 --2- 172.21.15.7:0/3169149802 >> v2:172.21.15.188:3300/0 conn(0x7ff3f815c1e0 0x7ff3f80c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.684+0000 7ff3f7fff700 1 -- 172.21.15.7:0/3169149802 >> v1:172.21.15.188:6789/0 conn(0x7ff3f80b0180 legacy=0x7ff3f80aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.684+0000 7ff3f7fff700 1 -- 172.21.15.7:0/3169149802 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f80b0b40 con 0x7ff3f815be10 2024-01-27T02:43:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.684+0000 7ff3f6ffd700 1 -- 172.21.15.7:0/3169149802 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 564740099 0 0) 0x7ff3f80b1470 con 0x7ff3f80b0180 2024-01-27T02:43:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.684+0000 7ff3fca4a700 1 --2- 172.21.15.7:0/3169149802 >> v2:172.21.15.188:3300/0 conn(0x7ff3f815c1e0 0x7ff3f80c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:43:21.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.684+0000 7ff3f7fff700 1 --2- 172.21.15.7:0/3169149802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3f815be10 0x7ff3f80b1900 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80098e0 tx=0x7ff3e80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=301abcb11649d2ce server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:21.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.684+0000 7ff3f6ffd700 1 -- 172.21.15.7:0/3169149802 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e800b640 con 0x7ff3f815be10 2024-01-27T02:43:21.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.684+0000 7ff3f6ffd700 1 -- 172.21.15.7:0/3169149802 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e800b7a0 con 0x7ff3f815be10 2024-01-27T02:43:21.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.684+0000 7ff3f6ffd700 1 -- 172.21.15.7:0/3169149802 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e800bb40 con 0x7ff3f815be10 2024-01-27T02:43:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.685+0000 7ff3fe4ad700 1 -- 172.21.15.7:0/3169149802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3f815be10 msgr2=0x7ff3f80b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.685+0000 7ff3fe4ad700 1 --2- 172.21.15.7:0/3169149802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3f815be10 0x7ff3f80b1900 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80098e0 tx=0x7ff3e80091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.685+0000 7ff3fe4ad700 1 -- 172.21.15.7:0/3169149802 shutdown_connections 2024-01-27T02:43:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.685+0000 7ff3fe4ad700 1 --2- 172.21.15.7:0/3169149802 >> v2:172.21.15.188:3300/0 conn(0x7ff3f815c1e0 0x7ff3f80c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.685+0000 7ff3fe4ad700 1 --2- 172.21.15.7:0/3169149802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3f815be10 0x7ff3f80b1900 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.685+0000 7ff3fe4ad700 1 -- 172.21.15.7:0/3169149802 >> 172.21.15.7:0/3169149802 conn(0x7ff3f80679b0 msgr2=0x7ff3f8067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.685+0000 7ff3fe4ad700 1 -- 172.21.15.7:0/3169149802 shutdown_connections 2024-01-27T02:43:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.685+0000 7ff3fe4ad700 1 -- 172.21.15.7:0/3169149802 wait complete. 2024-01-27T02:43:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.686+0000 7ff3fe4ad700 1 Processor -- start 2024-01-27T02:43:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.686+0000 7ff3fe4ad700 1 -- start start 2024-01-27T02:43:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.687+0000 7ff3fe4ad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3f80b0180 0x7ff3f80c6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.687+0000 7ff3fe4ad700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3f815be10 0x7ff3f80abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.687+0000 7ff3fe4ad700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3f815c1e0 0x7ff3f8097a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.687+0000 7ff3fe4ad700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f80ccf10 con 0x7ff3f80b0180 2024-01-27T02:43:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.687+0000 7ff3fe4ad700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f806e620 con 0x7ff3f815be10 2024-01-27T02:43:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.687+0000 7ff3fe4ad700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f80cc950 con 0x7ff3f815c1e0 2024-01-27T02:43:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.687+0000 7ff3f7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3f80b0180 0x7ff3f80c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.687+0000 7ff3f7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3f80b0180 0x7ff3f80c6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42648/0 (socket says 172.21.15.7:42648) 2024-01-27T02:43:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.687+0000 7ff3f7fff700 1 -- 172.21.15.7:0/4059723191 learned_addr learned my addr 172.21.15.7:0/4059723191 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.687+0000 7ff3f77fe700 1 --2- 172.21.15.7:0/4059723191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3f815be10 0x7ff3f80abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:21.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.687+0000 7ff3fca4a700 1 --2- 172.21.15.7:0/4059723191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3f815c1e0 0x7ff3f8097a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:21.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.688+0000 7ff3f7fff700 1 -- 172.21.15.7:0/4059723191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3f815c1e0 msgr2=0x7ff3f8097a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:21.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.688+0000 7ff3f7fff700 1 --2- 172.21.15.7:0/4059723191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3f815c1e0 0x7ff3f8097a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:21.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.688+0000 7ff3f7fff700 1 -- 172.21.15.7:0/4059723191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3f815be10 msgr2=0x7ff3f80abeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:21.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.688+0000 7ff3f7fff700 1 --2- 172.21.15.7:0/4059723191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3f815be10 0x7ff3f80abeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:21.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.688+0000 7ff3f7fff700 1 -- 172.21.15.7:0/4059723191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f0009ae0 con 0x7ff3f80b0180 2024-01-27T02:43:21.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.688+0000 7ff3f7fff700 1 --2- 172.21.15.7:0/4059723191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3f80b0180 0x7ff3f80c6570 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8009f80 tx=0x7ff3e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:21.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.688+0000 7ff3dffff700 1 -- 172.21.15.7:0/4059723191 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8000bc0 con 0x7ff3f80b0180 2024-01-27T02:43:21.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.688+0000 7ff3dffff700 1 -- 172.21.15.7:0/4059723191 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e800b910 con 0x7ff3f80b0180 2024-01-27T02:43:21.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.688+0000 7ff3fe4ad700 1 -- 172.21.15.7:0/4059723191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e8009420 con 0x7ff3f80b0180 2024-01-27T02:43:21.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.688+0000 7ff3dffff700 1 -- 172.21.15.7:0/4059723191 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8000d20 con 0x7ff3f80b0180 2024-01-27T02:43:21.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.688+0000 7ff3fe4ad700 1 -- 172.21.15.7:0/4059723191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f80ac3f0 con 0x7ff3f80b0180 2024-01-27T02:43:21.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.689+0000 7ff3fe4ad700 1 -- 172.21.15.7:0/4059723191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f804c710 con 0x7ff3f80b0180 2024-01-27T02:43:21.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.690+0000 7ff3dffff700 1 -- 172.21.15.7:0/4059723191 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff3e8023070 con 0x7ff3f80b0180 2024-01-27T02:43:21.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.690+0000 7ff3dffff700 1 --2- 172.21.15.7:0/4059723191 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff3e4066f30 0x7ff3e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:21.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.690+0000 7ff3dffff700 1 -- 172.21.15.7:0/4059723191 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff3e80856e0 con 0x7ff3f80b0180 2024-01-27T02:43:21.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.690+0000 7ff3f77fe700 1 --2- 172.21.15.7:0/4059723191 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff3e4066f30 0x7ff3e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:21.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.691+0000 7ff3f77fe700 1 --2- 172.21.15.7:0/4059723191 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff3e4066f30 0x7ff3e40693e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff3f806b540 tx=0x7ff3ec007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.695+0000 7ff3dffff700 1 -- 172.21.15.7:0/4059723191 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3e8052c50 con 0x7ff3f80b0180 2024-01-27T02:43:21.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:21.980+0000 7ff3fe4ad700 1 -- 172.21.15.7:0/4059723191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) v1 -- 0x7ff3f80b0e20 con 0x7ff3f80b0180 2024-01-27T02:43:22.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.145+0000 7ff3dffff700 1 -- 172.21.15.7:0/4059723191 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v527) v1 ==== 131+0+0 (secure 0 0 0) 0x7ff3e8056270 con 0x7ff3f80b0180 2024-01-27T02:43:22.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.147+0000 7ff3fe4ad700 1 -- 172.21.15.7:0/4059723191 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff3e4066f30 msgr2=0x7ff3e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:22.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.147+0000 7ff3fe4ad700 1 --2- 172.21.15.7:0/4059723191 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff3e4066f30 0x7ff3e40693e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff3f806b540 tx=0x7ff3ec007d50 comp rx=0 tx=0).stop 2024-01-27T02:43:22.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.147+0000 7ff3fe4ad700 1 -- 172.21.15.7:0/4059723191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3f80b0180 msgr2=0x7ff3f80c6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:22.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.147+0000 7ff3fe4ad700 1 --2- 172.21.15.7:0/4059723191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3f80b0180 0x7ff3f80c6570 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8009f80 tx=0x7ff3e8017000 comp rx=0 tx=0).stop 2024-01-27T02:43:22.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.147+0000 7ff3fe4ad700 1 -- 172.21.15.7:0/4059723191 shutdown_connections 2024-01-27T02:43:22.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.147+0000 7ff3fe4ad700 1 --2- 172.21.15.7:0/4059723191 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff3e4066f30 0x7ff3e40693e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:22.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.147+0000 7ff3fe4ad700 1 --2- 172.21.15.7:0/4059723191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3f815c1e0 0x7ff3f8097a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:22.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.147+0000 7ff3fe4ad700 1 --2- 172.21.15.7:0/4059723191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3f80b0180 0x7ff3f80c6570 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:22.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.147+0000 7ff3fe4ad700 1 --2- 172.21.15.7:0/4059723191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3f815be10 0x7ff3f80abeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:22.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.147+0000 7ff3fe4ad700 1 -- 172.21.15.7:0/4059723191 >> 172.21.15.7:0/4059723191 conn(0x7ff3f80679b0 msgr2=0x7ff3f815c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:22.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.147+0000 7ff3fe4ad700 1 -- 172.21.15.7:0/4059723191 shutdown_connections 2024-01-27T02:43:22.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.147+0000 7ff3fe4ad700 1 -- 172.21.15.7:0/4059723191 wait complete. 2024-01-27T02:43:22.163 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2024-01-27T02:43:22.164 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) 2024-01-27T02:43:22.164 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2024-01-27T02:43:22.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.297+0000 7fa89b615700 1 Processor -- start 2024-01-27T02:43:22.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.297+0000 7fa89b615700 1 -- start start 2024-01-27T02:43:22.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.298+0000 7fa89b615700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa894143520 0x7fa894063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:22.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.298+0000 7fa89b615700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8940cb7a0 con 0x7fa8940aa8b0 2024-01-27T02:43:22.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.298+0000 7fa89b615700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8940cb9f0 con 0x7fa8941438f0 2024-01-27T02:43:22.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.298+0000 7fa89b615700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8940cbcd0 con 0x7fa894143520 2024-01-27T02:43:22.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.298+0000 7fa899bb2700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa8940aa8b0 0x7fa89406e2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40306/0 (socket says 172.21.15.7:40306) 2024-01-27T02:43:22.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.298+0000 7fa8993b1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa894143520 0x7fa894063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:22.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.298+0000 7fa899bb2700 1 -- 172.21.15.7:0/2676099791 learned_addr learned my addr 172.21.15.7:0/2676099791 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:22.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.298+0000 7fa8993b1700 1 -- 172.21.15.7:0/2676099791 >> v1:172.21.15.188:6789/0 conn(0x7fa8941438f0 legacy=0x7fa8940c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:22.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.298+0000 7fa8993b1700 1 -- 172.21.15.7:0/2676099791 >> v1:172.21.15.7:6789/0 conn(0x7fa8940aa8b0 legacy=0x7fa89406e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:22.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.298+0000 7fa8993b1700 1 -- 172.21.15.7:0/2676099791 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8940cbfb0 con 0x7fa894143520 2024-01-27T02:43:22.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.298+0000 7fa887fff700 1 -- 172.21.15.7:0/2676099791 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029032769 0 0) 0x7fa8940cb7a0 con 0x7fa8940aa8b0 2024-01-27T02:43:22.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.298+0000 7fa8993b1700 1 --2- 172.21.15.7:0/2676099791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa894143520 0x7fa894063580 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fa8880098d0 tx=0x7fa888009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4bd6ce4cd7dc1d8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:22.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.299+0000 7fa887fff700 1 -- 172.21.15.7:0/2676099791 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888009e70 con 0x7fa894143520 2024-01-27T02:43:22.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.299+0000 7fa887fff700 1 -- 172.21.15.7:0/2676099791 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa888005c10 con 0x7fa894143520 2024-01-27T02:43:22.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.299+0000 7fa887fff700 1 -- 172.21.15.7:0/2676099791 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88800b650 con 0x7fa894143520 2024-01-27T02:43:22.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.299+0000 7fa89b615700 1 -- 172.21.15.7:0/2676099791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa894143520 msgr2=0x7fa894063580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:22.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.299+0000 7fa89b615700 1 --2- 172.21.15.7:0/2676099791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa894143520 0x7fa894063580 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fa8880098d0 tx=0x7fa888009120 comp rx=0 tx=0).stop 2024-01-27T02:43:22.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.299+0000 7fa89b615700 1 -- 172.21.15.7:0/2676099791 shutdown_connections 2024-01-27T02:43:22.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.299+0000 7fa89b615700 1 --2- 172.21.15.7:0/2676099791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa894143520 0x7fa894063580 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:22.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.299+0000 7fa89b615700 1 -- 172.21.15.7:0/2676099791 >> 172.21.15.7:0/2676099791 conn(0x7fa8940cdbd0 msgr2=0x7fa8940cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:22.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.299+0000 7fa89b615700 1 -- 172.21.15.7:0/2676099791 shutdown_connections 2024-01-27T02:43:22.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.299+0000 7fa89b615700 1 -- 172.21.15.7:0/2676099791 wait complete. 2024-01-27T02:43:22.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.301+0000 7fa89b615700 1 Processor -- start 2024-01-27T02:43:22.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.301+0000 7fa89b615700 1 -- start start 2024-01-27T02:43:22.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.302+0000 7fa89b615700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8940aa8b0 0x7fa8940b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:22.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.302+0000 7fa89b615700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa894143520 0x7fa8940af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:22.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.302+0000 7fa89b615700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8941438f0 0x7fa894159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:22.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.302+0000 7fa89b615700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8941595d0 con 0x7fa8941438f0 2024-01-27T02:43:22.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.302+0000 7fa89b615700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8940aec60 con 0x7fa894143520 2024-01-27T02:43:22.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.302+0000 7fa89b615700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8940ae330 con 0x7fa8940aa8b0 2024-01-27T02:43:22.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.302+0000 7fa899bb2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8941438f0 0x7fa894159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:22.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.302+0000 7fa899bb2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8941438f0 0x7fa894159090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42664/0 (socket says 172.21.15.7:42664) 2024-01-27T02:43:22.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.302+0000 7fa899bb2700 1 -- 172.21.15.7:0/533784675 learned_addr learned my addr 172.21.15.7:0/533784675 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:22.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.302+0000 7fa8993b1700 1 --2- 172.21.15.7:0/533784675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8940aa8b0 0x7fa8940b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:22.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.302+0000 7fa898bb0700 1 --2- 172.21.15.7:0/533784675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa894143520 0x7fa8940af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:22.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.302+0000 7fa899bb2700 1 -- 172.21.15.7:0/533784675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8940aa8b0 msgr2=0x7fa8940b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:22.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.302+0000 7fa899bb2700 1 --2- 172.21.15.7:0/533784675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8940aa8b0 0x7fa8940b13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:22.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.302+0000 7fa899bb2700 1 -- 172.21.15.7:0/533784675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa894143520 msgr2=0x7fa8940af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:22.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.302+0000 7fa899bb2700 1 --2- 172.21.15.7:0/533784675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa894143520 0x7fa8940af0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:22.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.302+0000 7fa899bb2700 1 -- 172.21.15.7:0/533784675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8880093c0 con 0x7fa8941438f0 2024-01-27T02:43:22.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.302+0000 7fa899bb2700 1 --2- 172.21.15.7:0/533784675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8941438f0 0x7fa894159090 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00cc90 tx=0x7fa88c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:22.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.302+0000 7fa8857fa700 1 -- 172.21.15.7:0/533784675 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c00a8b0 con 0x7fa8941438f0 2024-01-27T02:43:22.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.303+0000 7fa8857fa700 1 -- 172.21.15.7:0/533784675 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88c00aa10 con 0x7fa8941438f0 2024-01-27T02:43:22.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.303+0000 7fa89b615700 1 -- 172.21.15.7:0/533784675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8940ae610 con 0x7fa8941438f0 2024-01-27T02:43:22.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.303+0000 7fa8857fa700 1 -- 172.21.15.7:0/533784675 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c00add0 con 0x7fa8941438f0 2024-01-27T02:43:22.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.303+0000 7fa89b615700 1 -- 172.21.15.7:0/533784675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8940b2aa0 con 0x7fa8941438f0 2024-01-27T02:43:22.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.304+0000 7fa89b615700 1 -- 172.21.15.7:0/533784675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8940aedf0 con 0x7fa8941438f0 2024-01-27T02:43:22.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.304+0000 7fa8857fa700 1 -- 172.21.15.7:0/533784675 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa88c007300 con 0x7fa8941438f0 2024-01-27T02:43:22.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.305+0000 7fa8857fa700 1 --2- 172.21.15.7:0/533784675 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa878066f30 0x7fa8780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:22.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.305+0000 7fa8857fa700 1 -- 172.21.15.7:0/533784675 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa88c084520 con 0x7fa8941438f0 2024-01-27T02:43:22.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.305+0000 7fa8993b1700 1 --2- 172.21.15.7:0/533784675 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa878066f30 0x7fa8780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:22.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.305+0000 7fa8993b1700 1 --2- 172.21.15.7:0/533784675 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa878066f30 0x7fa8780693e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa888009390 tx=0x7fa888016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:22.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.309+0000 7fa8857fa700 1 -- 172.21.15.7:0/533784675 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa88c051b10 con 0x7fa8941438f0 2024-01-27T02:43:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:22.601+0000 7fa89b615700 1 -- 172.21.15.7:0/533784675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) v1 -- 0x7fa89404c710 con 0x7fa8941438f0 2024-01-27T02:43:23.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.147+0000 7fa8857fa700 1 -- 172.21.15.7:0/533784675 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v528) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa88c055130 con 0x7fa8941438f0 2024-01-27T02:43:23.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.150+0000 7fa89b615700 1 -- 172.21.15.7:0/533784675 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa878066f30 msgr2=0x7fa8780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:23.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.150+0000 7fa89b615700 1 --2- 172.21.15.7:0/533784675 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa878066f30 0x7fa8780693e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa888009390 tx=0x7fa888016000 comp rx=0 tx=0).stop 2024-01-27T02:43:23.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.150+0000 7fa89b615700 1 -- 172.21.15.7:0/533784675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8941438f0 msgr2=0x7fa894159090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:23.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.150+0000 7fa89b615700 1 --2- 172.21.15.7:0/533784675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8941438f0 0x7fa894159090 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00cc90 tx=0x7fa88c001a80 comp rx=0 tx=0).stop 2024-01-27T02:43:23.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.150+0000 7fa89b615700 1 -- 172.21.15.7:0/533784675 shutdown_connections 2024-01-27T02:43:23.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.150+0000 7fa89b615700 1 --2- 172.21.15.7:0/533784675 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa878066f30 0x7fa8780693e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:23.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.150+0000 7fa89b615700 1 --2- 172.21.15.7:0/533784675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8941438f0 0x7fa894159090 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:23.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.150+0000 7fa89b615700 1 --2- 172.21.15.7:0/533784675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8940aa8b0 0x7fa8940b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:23.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.150+0000 7fa89b615700 1 --2- 172.21.15.7:0/533784675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa894143520 0x7fa8940af0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:23.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.150+0000 7fa89b615700 1 -- 172.21.15.7:0/533784675 >> 172.21.15.7:0/533784675 conn(0x7fa8940cdbd0 msgr2=0x7fa8940c33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:23.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.150+0000 7fa89b615700 1 -- 172.21.15.7:0/533784675 shutdown_connections 2024-01-27T02:43:23.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.150+0000 7fa89b615700 1 -- 172.21.15.7:0/533784675 wait complete. 2024-01-27T02:43:23.167 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-27T02:43:23.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.305+0000 7fde44002700 1 Processor -- start 2024-01-27T02:43:23.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.305+0000 7fde44002700 1 -- start start 2024-01-27T02:43:23.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.306+0000 7fde44002700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde3c15e2d0 0x7fde3c0693d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:23.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.306+0000 7fde44002700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde3c0680c0 0x7fde3c0c9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:23.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.306+0000 7fde44002700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde3c1621e0 con 0x7fde3c0cc9b0 2024-01-27T02:43:23.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.306+0000 7fde44002700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde3c162480 con 0x7fde3c0680c0 2024-01-27T02:43:23.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.306+0000 7fde44002700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c162760 con 0x7fde3c15e2d0 2024-01-27T02:43:23.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.306+0000 7fde4259f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde3c0680c0 0x7fde3c0c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:23.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.306+0000 7fde4259f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde3c0680c0 0x7fde3c0c9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42680/0 (socket says 172.21.15.7:42680) 2024-01-27T02:43:23.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.306+0000 7fde4259f700 1 -- 172.21.15.7:0/3628993743 learned_addr learned my addr 172.21.15.7:0/3628993743 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:23.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.306+0000 7fde41d9e700 1 --2- 172.21.15.7:0/3628993743 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde3c15e2d0 0x7fde3c0693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:23.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.306+0000 7fde4259f700 1 -- 172.21.15.7:0/3628993743 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde3c15e2d0 msgr2=0x7fde3c0693d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:23.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.306+0000 7fde4259f700 1 --2- 172.21.15.7:0/3628993743 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde3c15e2d0 0x7fde3c0693d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:23.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.306+0000 7fde4259f700 1 -- 172.21.15.7:0/3628993743 >> v1:172.21.15.188:6789/0 conn(0x7fde3c0cc9b0 legacy=0x7fde3c0679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:23.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.306+0000 7fde4259f700 1 -- 172.21.15.7:0/3628993743 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c162a40 con 0x7fde3c0680c0 2024-01-27T02:43:23.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.307+0000 7fde4259f700 1 --2- 172.21.15.7:0/3628993743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde3c0680c0 0x7fde3c0c9440 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fde34009f20 tx=0x7fde34009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b7adea391a60fd server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:23.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.307+0000 7fde40d9c700 1 -- 172.21.15.7:0/3628993743 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3400e070 con 0x7fde3c0680c0 2024-01-27T02:43:23.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.307+0000 7fde40d9c700 1 -- 172.21.15.7:0/3628993743 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3400ba80 con 0x7fde3c0680c0 2024-01-27T02:43:23.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.307+0000 7fde40d9c700 1 -- 172.21.15.7:0/3628993743 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde340092e0 con 0x7fde3c0680c0 2024-01-27T02:43:23.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.307+0000 7fde44002700 1 -- 172.21.15.7:0/3628993743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde3c0680c0 msgr2=0x7fde3c0c9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:23.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.307+0000 7fde44002700 1 --2- 172.21.15.7:0/3628993743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde3c0680c0 0x7fde3c0c9440 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fde34009f20 tx=0x7fde34009840 comp rx=0 tx=0).stop 2024-01-27T02:43:23.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.307+0000 7fde44002700 1 -- 172.21.15.7:0/3628993743 shutdown_connections 2024-01-27T02:43:23.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.307+0000 7fde44002700 1 --2- 172.21.15.7:0/3628993743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde3c0680c0 0x7fde3c0c9440 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:23.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.307+0000 7fde44002700 1 --2- 172.21.15.7:0/3628993743 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde3c15e2d0 0x7fde3c0693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:23.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.307+0000 7fde44002700 1 -- 172.21.15.7:0/3628993743 >> 172.21.15.7:0/3628993743 conn(0x7fde3c063580 msgr2=0x7fde3c063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:23.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.308+0000 7fde44002700 1 -- 172.21.15.7:0/3628993743 shutdown_connections 2024-01-27T02:43:23.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.308+0000 7fde44002700 1 -- 172.21.15.7:0/3628993743 wait complete. 2024-01-27T02:43:23.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.309+0000 7fde44002700 1 Processor -- start 2024-01-27T02:43:23.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.309+0000 7fde44002700 1 -- start start 2024-01-27T02:43:23.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.309+0000 7fde44002700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde3c0680c0 0x7fde3c0c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:23.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.309+0000 7fde44002700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde3c0cc9b0 0x7fde3c0b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:23.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.309+0000 7fde44002700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde3c15e2d0 0x7fde3c15a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:23.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.309+0000 7fde44002700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c0b1fe0 con 0x7fde3c0cc9b0 2024-01-27T02:43:23.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.309+0000 7fde44002700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c0b2e30 con 0x7fde3c0680c0 2024-01-27T02:43:23.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.309+0000 7fde44002700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c0b2220 con 0x7fde3c15e2d0 2024-01-27T02:43:23.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.310+0000 7fde4159d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde3c0cc9b0 0x7fde3c0b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:23.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.310+0000 7fde4159d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde3c0cc9b0 0x7fde3c0b1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42694/0 (socket says 172.21.15.7:42694) 2024-01-27T02:43:23.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.310+0000 7fde4159d700 1 -- 172.21.15.7:0/191544328 learned_addr learned my addr 172.21.15.7:0/191544328 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:23.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.310+0000 7fde4259f700 1 --2- 172.21.15.7:0/191544328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde3c15e2d0 0x7fde3c15a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:23.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.310+0000 7fde41d9e700 1 --2- 172.21.15.7:0/191544328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde3c0680c0 0x7fde3c0c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:23.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.310+0000 7fde4159d700 1 -- 172.21.15.7:0/191544328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde3c15e2d0 msgr2=0x7fde3c15a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.310+0000 7fde4159d700 1 --2- 172.21.15.7:0/191544328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde3c15e2d0 0x7fde3c15a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.310+0000 7fde4159d700 1 -- 172.21.15.7:0/191544328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde3c0680c0 msgr2=0x7fde3c0c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.310+0000 7fde4159d700 1 --2- 172.21.15.7:0/191544328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde3c0680c0 0x7fde3c0c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.310+0000 7fde4159d700 1 -- 172.21.15.7:0/191544328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde34009ae0 con 0x7fde3c0cc9b0 2024-01-27T02:43:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.310+0000 7fde4159d700 1 --2- 172.21.15.7:0/191544328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde3c0cc9b0 0x7fde3c0b1aa0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00c790 tx=0x7fde2c007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.310+0000 7fde31ffb700 1 -- 172.21.15.7:0/191544328 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c011590 con 0x7fde3c0cc9b0 2024-01-27T02:43:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.310+0000 7fde31ffb700 1 -- 172.21.15.7:0/191544328 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2c011710 con 0x7fde3c0cc9b0 2024-01-27T02:43:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.310+0000 7fde44002700 1 -- 172.21.15.7:0/191544328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde3c0b2500 con 0x7fde3c0cc9b0 2024-01-27T02:43:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.310+0000 7fde31ffb700 1 -- 172.21.15.7:0/191544328 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c011ad0 con 0x7fde3c0cc9b0 2024-01-27T02:43:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.311+0000 7fde44002700 1 -- 172.21.15.7:0/191544328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde3c0b2ac0 con 0x7fde3c0cc9b0 2024-01-27T02:43:23.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.312+0000 7fde44002700 1 -- 172.21.15.7:0/191544328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde3c072060 con 0x7fde3c0cc9b0 2024-01-27T02:43:23.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.312+0000 7fde31ffb700 1 -- 172.21.15.7:0/191544328 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fde2c01b3b0 con 0x7fde3c0cc9b0 2024-01-27T02:43:23.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.312+0000 7fde31ffb700 1 --2- 172.21.15.7:0/191544328 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fde20066f30 0x7fde200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:23.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.312+0000 7fde31ffb700 1 -- 172.21.15.7:0/191544328 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fde2c0835e0 con 0x7fde3c0cc9b0 2024-01-27T02:43:23.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.313+0000 7fde41d9e700 1 --2- 172.21.15.7:0/191544328 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fde20066f30 0x7fde200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:23.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.313+0000 7fde41d9e700 1 --2- 172.21.15.7:0/191544328 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fde20066f30 0x7fde200693e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fde38006fd0 tx=0x7fde38008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:23.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.317+0000 7fde31ffb700 1 -- 172.21.15.7:0/191544328 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde2c050b50 con 0x7fde3c0cc9b0 2024-01-27T02:43:23.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.519+0000 7fde44002700 1 -- 172.21.15.7:0/191544328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fde3c0cd210 con 0x7fde3c0cc9b0 2024-01-27T02:43:23.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.523+0000 7fde31ffb700 1 -- 172.21.15.7:0/191544328 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fde2c017020 con 0x7fde3c0cc9b0 2024-01-27T02:43:23.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.526+0000 7fde44002700 1 -- 172.21.15.7:0/191544328 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fde20066f30 msgr2=0x7fde200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:23.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.526+0000 7fde44002700 1 --2- 172.21.15.7:0/191544328 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fde20066f30 0x7fde200693e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fde38006fd0 tx=0x7fde38008040 comp rx=0 tx=0).stop 2024-01-27T02:43:23.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.526+0000 7fde44002700 1 -- 172.21.15.7:0/191544328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde3c0cc9b0 msgr2=0x7fde3c0b1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:23.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.526+0000 7fde44002700 1 --2- 172.21.15.7:0/191544328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde3c0cc9b0 0x7fde3c0b1aa0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00c790 tx=0x7fde2c007f20 comp rx=0 tx=0).stop 2024-01-27T02:43:23.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.526+0000 7fde44002700 1 -- 172.21.15.7:0/191544328 shutdown_connections 2024-01-27T02:43:23.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.526+0000 7fde44002700 1 --2- 172.21.15.7:0/191544328 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fde20066f30 0x7fde200693e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:23.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.526+0000 7fde44002700 1 --2- 172.21.15.7:0/191544328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde3c15e2d0 0x7fde3c15a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:23.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.526+0000 7fde44002700 1 --2- 172.21.15.7:0/191544328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde3c0680c0 0x7fde3c0c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:23.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.526+0000 7fde44002700 1 --2- 172.21.15.7:0/191544328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde3c0cc9b0 0x7fde3c0b1aa0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:23.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.526+0000 7fde44002700 1 -- 172.21.15.7:0/191544328 >> 172.21.15.7:0/191544328 conn(0x7fde3c063580 msgr2=0x7fde3c0c3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:23.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.526+0000 7fde44002700 1 -- 172.21.15.7:0/191544328 shutdown_connections 2024-01-27T02:43:23.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.526+0000 7fde44002700 1 -- 172.21.15.7:0/191544328 wait complete. 2024-01-27T02:43:23.541 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-27T02:43:23.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:43:23.586 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:43:23.648 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-27T02:43:23.648 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-27T02:43:23.648 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:43:23.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.785+0000 7f26d5425700 1 Processor -- start 2024-01-27T02:43:23.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.785+0000 7f26d5425700 1 -- start start 2024-01-27T02:43:23.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.786+0000 7f26d5425700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f26d0140610 0x7f26d00b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:23.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.786+0000 7f26d5425700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26d01409e0 0x7f26d006bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:23.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.786+0000 7f26d5425700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26d00c30c0 con 0x7f26d006c430 2024-01-27T02:43:23.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.786+0000 7f26d5425700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26d006ccf0 con 0x7f26d0140610 2024-01-27T02:43:23.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.786+0000 7f26d5425700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d006ca10 con 0x7f26d01409e0 2024-01-27T02:43:23.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.786+0000 7f26ceffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f26d0140610 0x7f26d00b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:23.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.786+0000 7f26ceffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f26d0140610 0x7f26d00b09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42696/0 (socket says 172.21.15.7:42696) 2024-01-27T02:43:23.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.786+0000 7f26ceffd700 1 -- 172.21.15.7:0/1092602957 learned_addr learned my addr 172.21.15.7:0/1092602957 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:23.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.787+0000 7f26c7fff700 1 --2- 172.21.15.7:0/1092602957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26d01409e0 0x7f26d006bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:23.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.787+0000 7f26ceffd700 1 -- 172.21.15.7:0/1092602957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26d01409e0 msgr2=0x7f26d006bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:23.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.787+0000 7f26ceffd700 1 --2- 172.21.15.7:0/1092602957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26d01409e0 0x7f26d006bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:23.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.787+0000 7f26ceffd700 1 -- 172.21.15.7:0/1092602957 >> v1:172.21.15.188:6789/0 conn(0x7f26d006c430 legacy=0x7f26d006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:23.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.787+0000 7f26ceffd700 1 -- 172.21.15.7:0/1092602957 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d00c2a70 con 0x7f26d0140610 2024-01-27T02:43:23.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.787+0000 7f26c7fff700 1 --2- 172.21.15.7:0/1092602957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26d01409e0 0x7f26d006bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:43:23.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.787+0000 7f26ceffd700 1 --2- 172.21.15.7:0/1092602957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26d0140610 0x7f26d00b09f0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f26c00099b0 tx=0x7f26c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b692bc1bdd9e7d server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:23.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.787+0000 7f26ce7fc700 1 -- 172.21.15.7:0/1092602957 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c000b820 con 0x7f26d0140610 2024-01-27T02:43:23.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.787+0000 7f26ce7fc700 1 -- 172.21.15.7:0/1092602957 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c000b980 con 0x7f26d0140610 2024-01-27T02:43:23.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.787+0000 7f26ce7fc700 1 -- 172.21.15.7:0/1092602957 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c000bd20 con 0x7f26d0140610 2024-01-27T02:43:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.788+0000 7f26d5425700 1 -- 172.21.15.7:0/1092602957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26d0140610 msgr2=0x7f26d00b09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.788+0000 7f26d5425700 1 --2- 172.21.15.7:0/1092602957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26d0140610 0x7f26d00b09f0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f26c00099b0 tx=0x7f26c0009250 comp rx=0 tx=0).stop 2024-01-27T02:43:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.788+0000 7f26d5425700 1 -- 172.21.15.7:0/1092602957 shutdown_connections 2024-01-27T02:43:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.788+0000 7f26d5425700 1 --2- 172.21.15.7:0/1092602957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26d0140610 0x7f26d00b09f0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.788+0000 7f26d5425700 1 --2- 172.21.15.7:0/1092602957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26d01409e0 0x7f26d006bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.788+0000 7f26d5425700 1 -- 172.21.15.7:0/1092602957 >> 172.21.15.7:0/1092602957 conn(0x7f26d00cdc30 msgr2=0x7f26d00ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.788+0000 7f26d5425700 1 -- 172.21.15.7:0/1092602957 shutdown_connections 2024-01-27T02:43:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.788+0000 7f26d5425700 1 -- 172.21.15.7:0/1092602957 wait complete. 2024-01-27T02:43:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.789+0000 7f26d5425700 1 Processor -- start 2024-01-27T02:43:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.789+0000 7f26d5425700 1 -- start start 2024-01-27T02:43:23.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.790+0000 7f26d5425700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26d006c430 0x7f26d00c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:23.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.790+0000 7f26d5425700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26d0140610 0x7f26d00b2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:23.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.790+0000 7f26d5425700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26d01409e0 0x7f26d01563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:23.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.790+0000 7f26d5425700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d00c64d0 con 0x7f26d0140610 2024-01-27T02:43:23.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.790+0000 7f26d5425700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d00c61f0 con 0x7f26d006c430 2024-01-27T02:43:23.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.790+0000 7f26d5425700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d0158980 con 0x7f26d01409e0 2024-01-27T02:43:23.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.790+0000 7f26c7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26d0140610 0x7f26d00b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:23.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.790+0000 7f26c7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26d0140610 0x7f26d00b2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42706/0 (socket says 172.21.15.7:42706) 2024-01-27T02:43:23.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.790+0000 7f26c7fff700 1 -- 172.21.15.7:0/1716221286 learned_addr learned my addr 172.21.15.7:0/1716221286 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:23.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.790+0000 7f26ceffd700 1 --2- 172.21.15.7:0/1716221286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26d006c430 0x7f26d00c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:23.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.790+0000 7f26cf7fe700 1 --2- 172.21.15.7:0/1716221286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26d01409e0 0x7f26d01563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:23.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.790+0000 7f26c7fff700 1 -- 172.21.15.7:0/1716221286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26d01409e0 msgr2=0x7f26d01563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:23.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.790+0000 7f26c7fff700 1 --2- 172.21.15.7:0/1716221286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26d01409e0 0x7f26d01563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:23.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.790+0000 7f26c7fff700 1 -- 172.21.15.7:0/1716221286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26d006c430 msgr2=0x7f26d00c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:23.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.790+0000 7f26c7fff700 1 --2- 172.21.15.7:0/1716221286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26d006c430 0x7f26d00c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:23.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.790+0000 7f26c7fff700 1 -- 172.21.15.7:0/1716221286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c00094f0 con 0x7f26d0140610 2024-01-27T02:43:23.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.791+0000 7f26c7fff700 1 --2- 172.21.15.7:0/1716221286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26d0140610 0x7f26d00b2190 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f26b8007ba0 tx=0x7f26b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:23.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.791+0000 7f26c6ffd700 1 -- 172.21.15.7:0/1716221286 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b800cea0 con 0x7f26d0140610 2024-01-27T02:43:23.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.791+0000 7f26c6ffd700 1 -- 172.21.15.7:0/1716221286 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b80087b0 con 0x7f26d0140610 2024-01-27T02:43:23.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.791+0000 7f26c6ffd700 1 -- 172.21.15.7:0/1716221286 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b8008b30 con 0x7f26d0140610 2024-01-27T02:43:23.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.791+0000 7f26d5425700 1 -- 172.21.15.7:0/1716221286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d0158c60 con 0x7f26d0140610 2024-01-27T02:43:23.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.791+0000 7f26d5425700 1 -- 172.21.15.7:0/1716221286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d00c51a0 con 0x7f26d0140610 2024-01-27T02:43:23.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.792+0000 7f26d5425700 1 -- 172.21.15.7:0/1716221286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d01587e0 con 0x7f26d0140610 2024-01-27T02:43:23.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.793+0000 7f26c6ffd700 1 -- 172.21.15.7:0/1716221286 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f26b800f400 con 0x7f26d0140610 2024-01-27T02:43:23.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.793+0000 7f26c6ffd700 1 --2- 172.21.15.7:0/1716221286 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f26b0066f30 0x7f26b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:23.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.793+0000 7f26c6ffd700 1 -- 172.21.15.7:0/1716221286 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f26b80849d0 con 0x7f26d0140610 2024-01-27T02:43:23.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.793+0000 7f26ceffd700 1 --2- 172.21.15.7:0/1716221286 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f26b0066f30 0x7f26b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:23.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.794+0000 7f26ceffd700 1 --2- 172.21.15.7:0/1716221286 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f26b0066f30 0x7f26b00693e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f26c00099b0 tx=0x7f26c0016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:23.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:23.797+0000 7f26c6ffd700 1 -- 172.21.15.7:0/1716221286 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26b804f5a0 con 0x7f26d0140610 2024-01-27T02:43:24.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.049+0000 7f26d5425700 1 -- 172.21.15.7:0/1716221286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26d01587e0 con 0x7f26d0140610 2024-01-27T02:43:24.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.049+0000 7f26c6ffd700 1 -- 172.21.15.7:0/1716221286 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f26b801cde0 con 0x7f26d0140610 2024-01-27T02:43:24.051 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:24.051 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":88,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6473,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/3449593528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3449593528},{"type":"v1","addr":"172.21.15.188:6836","nonce":3449593528}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":6476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3473642033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3473642033},{"type":"v1","addr":"172.21.15.188:6837","nonce":3473642033}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":6479,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2550009140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2550009140},{"type":"v1","addr":"172.21.15.7:6833","nonce":2550009140}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":8034,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/4125957118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":4125957118},{"type":"v1","addr":"172.21.15.7:6835","nonce":4125957118}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[]} 2024-01-27T02:43:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.052+0000 7f26d5425700 1 -- 172.21.15.7:0/1716221286 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f26b0066f30 msgr2=0x7f26b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.052+0000 7f26d5425700 1 --2- 172.21.15.7:0/1716221286 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f26b0066f30 0x7f26b00693e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f26c00099b0 tx=0x7f26c0016000 comp rx=0 tx=0).stop 2024-01-27T02:43:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.052+0000 7f26d5425700 1 -- 172.21.15.7:0/1716221286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26d0140610 msgr2=0x7f26d00b2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.052+0000 7f26d5425700 1 --2- 172.21.15.7:0/1716221286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26d0140610 0x7f26d00b2190 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f26b8007ba0 tx=0x7f26b8008040 comp rx=0 tx=0).stop 2024-01-27T02:43:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.052+0000 7f26d5425700 1 -- 172.21.15.7:0/1716221286 shutdown_connections 2024-01-27T02:43:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.052+0000 7f26d5425700 1 --2- 172.21.15.7:0/1716221286 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f26b0066f30 0x7f26b00693e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.052+0000 7f26d5425700 1 --2- 172.21.15.7:0/1716221286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26d01409e0 0x7f26d01563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.052+0000 7f26d5425700 1 --2- 172.21.15.7:0/1716221286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26d006c430 0x7f26d00c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.052+0000 7f26d5425700 1 --2- 172.21.15.7:0/1716221286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26d0140610 0x7f26d00b2190 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.052+0000 7f26d5425700 1 -- 172.21.15.7:0/1716221286 >> 172.21.15.7:0/1716221286 conn(0x7f26d00cdc30 msgr2=0x7f26d006e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:24.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.052+0000 7f26d5425700 1 -- 172.21.15.7:0/1716221286 shutdown_connections 2024-01-27T02:43:24.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.052+0000 7f26d5425700 1 -- 172.21.15.7:0/1716221286 wait complete. 2024-01-27T02:43:24.055 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 88 2024-01-27T02:43:24.067 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-27T02:43:24.067 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-27T02:43:24.068 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-27T02:43:24.068 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:43:24.068 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-27T02:43:24.068 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-27T02:43:24.068 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-27T02:43:24.068 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:43:24.068 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-27T02:43:24.069 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-27T02:43:24.069 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-27T02:43:24.069 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:43:24.069 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-27T02:43:24.069 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-27T02:43:24.069 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-27T02:43:24.069 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:43:24.077 INFO:tasks.ceph.mds.a:Stopped 2024-01-27T02:43:24.077 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-27T02:43:24.078 INFO:tasks.ceph.mds.c:Stopped 2024-01-27T02:43:24.078 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-27T02:43:24.080 INFO:tasks.ceph.mds.d:Stopped 2024-01-27T02:43:24.080 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-27T02:43:24.081 INFO:tasks.ceph.mds.b:Stopped 2024-01-27T02:43:24.081 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-27T02:43:24.120 INFO:tasks.ceph.mds.a:Started 2024-01-27T02:43:24.120 DEBUG:teuthology.parallel:result is None 2024-01-27T02:43:24.121 INFO:tasks.ceph.mds.c:Started 2024-01-27T02:43:24.121 DEBUG:teuthology.parallel:result is None 2024-01-27T02:43:24.123 INFO:tasks.ceph.mds.d:Started 2024-01-27T02:43:24.123 DEBUG:teuthology.parallel:result is None 2024-01-27T02:43:24.124 INFO:tasks.ceph.mds.b:Started 2024-01-27T02:43:24.124 DEBUG:teuthology.parallel:result is None 2024-01-27T02:43:24.124 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-27T02:43:24.207 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-01-27T02:43:24.207 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-01-27T02:43:24.208 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-27T02:43:24.208 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-27T02:43:24.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.263+0000 7f7cb49e7700 1 Processor -- start 2024-01-27T02:43:24.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.263+0000 7f7cb49e7700 1 -- start start 2024-01-27T02:43:24.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.264+0000 7f7cb49e7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cb0143860 0x7f7cb00d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:24.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.264+0000 7f7cb49e7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7cb0143c30 0x7f7cb00d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:24.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.264+0000 7f7cb49e7700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb00b3d90 con 0x7f7cb015de00 2024-01-27T02:43:24.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.264+0000 7f7cb49e7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb00d0ea0 con 0x7f7cb0143c30 2024-01-27T02:43:24.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.264+0000 7f7cb49e7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00d1180 con 0x7f7cb0143860 2024-01-27T02:43:24.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.264+0000 7f7cadd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7cb0143c30 0x7f7cb00d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:24.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.264+0000 7f7cadd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7cb0143c30 0x7f7cb00d0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42756/0 (socket says 172.21.15.7:42756) 2024-01-27T02:43:24.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.264+0000 7f7cadd9b700 1 -- 172.21.15.7:0/1476307609 learned_addr learned my addr 172.21.15.7:0/1476307609 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:24.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.264+0000 7f7cae59c700 1 --2- 172.21.15.7:0/1476307609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cb0143860 0x7f7cb00d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:24.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.265+0000 7f7cadd9b700 1 -- 172.21.15.7:0/1476307609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cb0143860 msgr2=0x7f7cb00d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:24.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.265+0000 7f7cadd9b700 1 --2- 172.21.15.7:0/1476307609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cb0143860 0x7f7cb00d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:24.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.265+0000 7f7cadd9b700 1 -- 172.21.15.7:0/1476307609 >> v1:172.21.15.188:6789/0 conn(0x7f7cb015de00 legacy=0x7f7cb00cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:24.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.265+0000 7f7cadd9b700 1 -- 172.21.15.7:0/1476307609 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb00d1460 con 0x7f7cb0143c30 2024-01-27T02:43:24.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.265+0000 7f7cadd9b700 1 --2- 172.21.15.7:0/1476307609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb0143c30 0x7f7cb00d0820 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f7c980099b0 tx=0x7f7c98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27b0346ea3fc053 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:24.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.265+0000 7f7cad59a700 1 -- 172.21.15.7:0/1476307609 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c98009ea0 con 0x7f7cb0143c30 2024-01-27T02:43:24.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.265+0000 7f7cad59a700 1 -- 172.21.15.7:0/1476307609 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9800b860 con 0x7f7cb0143c30 2024-01-27T02:43:24.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.265+0000 7f7cad59a700 1 -- 172.21.15.7:0/1476307609 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9800bc00 con 0x7f7cb0143c30 2024-01-27T02:43:24.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.265+0000 7f7cb49e7700 1 -- 172.21.15.7:0/1476307609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb0143c30 msgr2=0x7f7cb00d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:24.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.265+0000 7f7cb49e7700 1 --2- 172.21.15.7:0/1476307609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb0143c30 0x7f7cb00d0820 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f7c980099b0 tx=0x7f7c98009250 comp rx=0 tx=0).stop 2024-01-27T02:43:24.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.266+0000 7f7cb49e7700 1 -- 172.21.15.7:0/1476307609 shutdown_connections 2024-01-27T02:43:24.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.266+0000 7f7cb49e7700 1 --2- 172.21.15.7:0/1476307609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cb0143860 0x7f7cb00d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:24.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.266+0000 7f7cb49e7700 1 --2- 172.21.15.7:0/1476307609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb0143c30 0x7f7cb00d0820 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:24.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.266+0000 7f7cb49e7700 1 -- 172.21.15.7:0/1476307609 >> 172.21.15.7:0/1476307609 conn(0x7f7cb0068440 msgr2=0x7f7cb00709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:24.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.266+0000 7f7cb49e7700 1 -- 172.21.15.7:0/1476307609 shutdown_connections 2024-01-27T02:43:24.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.266+0000 7f7cb49e7700 1 -- 172.21.15.7:0/1476307609 wait complete. 2024-01-27T02:43:24.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.267+0000 7f7cb49e7700 1 Processor -- start 2024-01-27T02:43:24.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.267+0000 7f7cb49e7700 1 -- start start 2024-01-27T02:43:24.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.268+0000 7f7cb49e7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cb0143860 0x7f7cb00ab490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:24.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.268+0000 7f7cb49e7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cb0143c30 0x7f7cb00a9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:24.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.268+0000 7f7cb49e7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb015de00 0x7f7cb00ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:24.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.268+0000 7f7cb49e7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb016dde0 con 0x7f7cb015de00 2024-01-27T02:43:24.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.268+0000 7f7cb49e7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb016ec40 con 0x7f7cb0143c30 2024-01-27T02:43:24.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.268+0000 7f7cb49e7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00a98d0 con 0x7f7cb0143860 2024-01-27T02:43:24.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.268+0000 7f7caed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb015de00 0x7f7cb00ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:24.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.268+0000 7f7caed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb015de00 0x7f7cb00ac160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42760/0 (socket says 172.21.15.7:42760) 2024-01-27T02:43:24.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.268+0000 7f7caed9d700 1 -- 172.21.15.7:0/2198577018 learned_addr learned my addr 172.21.15.7:0/2198577018 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:24.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.268+0000 7f7cadd9b700 1 --2- 172.21.15.7:0/2198577018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cb0143c30 0x7f7cb00a9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:24.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.268+0000 7f7cae59c700 1 --2- 172.21.15.7:0/2198577018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cb0143860 0x7f7cb00ab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:24.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.268+0000 7f7caed9d700 1 -- 172.21.15.7:0/2198577018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cb0143860 msgr2=0x7f7cb00ab490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:24.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.268+0000 7f7caed9d700 1 --2- 172.21.15.7:0/2198577018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cb0143860 0x7f7cb00ab490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:24.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.268+0000 7f7caed9d700 1 -- 172.21.15.7:0/2198577018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cb0143c30 msgr2=0x7f7cb00a9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:24.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.268+0000 7f7caed9d700 1 --2- 172.21.15.7:0/2198577018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cb0143c30 0x7f7cb00a9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:24.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.268+0000 7f7caed9d700 1 -- 172.21.15.7:0/2198577018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c980094f0 con 0x7f7cb015de00 2024-01-27T02:43:24.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.268+0000 7f7caed9d700 1 --2- 172.21.15.7:0/2198577018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb015de00 0x7f7cb00ac160 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000c840 tx=0x7f7ca0001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:24.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.269+0000 7f7ca67fc700 1 -- 172.21.15.7:0/2198577018 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0012630 con 0x7f7cb015de00 2024-01-27T02:43:24.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.269+0000 7f7ca67fc700 1 -- 172.21.15.7:0/2198577018 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca00127b0 con 0x7f7cb015de00 2024-01-27T02:43:24.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.269+0000 7f7ca67fc700 1 -- 172.21.15.7:0/2198577018 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0012b70 con 0x7f7cb015de00 2024-01-27T02:43:24.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.269+0000 7f7cb49e7700 1 -- 172.21.15.7:0/2198577018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb016e030 con 0x7f7cb015de00 2024-01-27T02:43:24.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.269+0000 7f7cb49e7700 1 -- 172.21.15.7:0/2198577018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb016e5f0 con 0x7f7cb015de00 2024-01-27T02:43:24.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.270+0000 7f7cb49e7700 1 -- 172.21.15.7:0/2198577018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb004c710 con 0x7f7cb015de00 2024-01-27T02:43:24.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.270+0000 7f7ca67fc700 1 -- 172.21.15.7:0/2198577018 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7ca001a3a0 con 0x7f7cb015de00 2024-01-27T02:43:24.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.271+0000 7f7ca67fc700 1 --2- 172.21.15.7:0/2198577018 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7c90066f30 0x7f7c900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:24.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.271+0000 7f7ca67fc700 1 -- 172.21.15.7:0/2198577018 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7ca0084390 con 0x7f7cb015de00 2024-01-27T02:43:24.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.271+0000 7f7cae59c700 1 --2- 172.21.15.7:0/2198577018 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7c90066f30 0x7f7c900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:24.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.271+0000 7f7cae59c700 1 --2- 172.21.15.7:0/2198577018 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7c90066f30 0x7f7c900693e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009db0 tx=0x7f7ca80091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:24.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.275+0000 7f7ca67fc700 1 -- 172.21.15.7:0/2198577018 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ca0051900 con 0x7f7cb015de00 2024-01-27T02:43:24.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:24.500+0000 7f7cb49e7700 1 -- 172.21.15.7:0/2198577018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7cb00d1740 con 0x7f7cb015de00 2024-01-27T02:43:25.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.538+0000 7f7ca67fc700 1 -- 172.21.15.7:0/2198577018 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v176) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7ca0054f20 con 0x7f7cb015de00 2024-01-27T02:43:25.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.541+0000 7f7cb49e7700 1 -- 172.21.15.7:0/2198577018 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7c90066f30 msgr2=0x7f7c900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:25.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.541+0000 7f7cb49e7700 1 --2- 172.21.15.7:0/2198577018 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7c90066f30 0x7f7c900693e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009db0 tx=0x7f7ca80091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:25.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.541+0000 7f7cb49e7700 1 -- 172.21.15.7:0/2198577018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb015de00 msgr2=0x7f7cb00ac160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:25.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.541+0000 7f7cb49e7700 1 --2- 172.21.15.7:0/2198577018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb015de00 0x7f7cb00ac160 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000c840 tx=0x7f7ca0001670 comp rx=0 tx=0).stop 2024-01-27T02:43:25.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.541+0000 7f7cb49e7700 1 -- 172.21.15.7:0/2198577018 shutdown_connections 2024-01-27T02:43:25.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.541+0000 7f7cb49e7700 1 --2- 172.21.15.7:0/2198577018 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7c90066f30 0x7f7c900693e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:25.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.541+0000 7f7cb49e7700 1 --2- 172.21.15.7:0/2198577018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb015de00 0x7f7cb00ac160 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:25.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.541+0000 7f7cb49e7700 1 --2- 172.21.15.7:0/2198577018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cb0143860 0x7f7cb00ab490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:25.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.541+0000 7f7cb49e7700 1 --2- 172.21.15.7:0/2198577018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cb0143c30 0x7f7cb00a9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:25.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.541+0000 7f7cb49e7700 1 -- 172.21.15.7:0/2198577018 >> 172.21.15.7:0/2198577018 conn(0x7f7cb0068440 msgr2=0x7f7cb00b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:25.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.542+0000 7f7cb49e7700 1 -- 172.21.15.7:0/2198577018 shutdown_connections 2024-01-27T02:43:25.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.542+0000 7f7cb49e7700 1 -- 172.21.15.7:0/2198577018 wait complete. 2024-01-27T02:43:25.544 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-01-27T02:43:25.559 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-27T02:43:25.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.693+0000 7f2e77365700 1 Processor -- start 2024-01-27T02:43:25.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.693+0000 7f2e77365700 1 -- start start 2024-01-27T02:43:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.694+0000 7f2e77365700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2e7015a100 0x7f2e700cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.694+0000 7f2e77365700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e700cb5b0 0x7f2e700c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.694+0000 7f2e77365700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e700ce5e0 con 0x7f2e700c4af0 2024-01-27T02:43:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.694+0000 7f2e77365700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e700c4f70 con 0x7f2e7015a100 2024-01-27T02:43:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.694+0000 7f2e77365700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e700cd9d0 con 0x7f2e700cb5b0 2024-01-27T02:43:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.695+0000 7f2e74900700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e700cb5b0 0x7f2e700c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.695+0000 7f2e75101700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2e7015a100 0x7f2e700cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.695+0000 7f2e75902700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f2e700c4af0 0x7f2e700c9530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:47768/0 (socket says 172.21.15.7:47768) 2024-01-27T02:43:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.695+0000 7f2e75902700 1 -- 172.21.15.7:0/2712493388 learned_addr learned my addr 172.21.15.7:0/2712493388 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.695+0000 7f2e74900700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e700cb5b0 0x7f2e700c45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:56250/0 (socket says 172.21.15.7:56250) 2024-01-27T02:43:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.695+0000 7f2e75101700 1 --2- 172.21.15.7:0/2712493388 >> v2:172.21.15.188:3300/0 conn(0x7f2e7015a100 0x7f2e700cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:45076/0 (socket says 172.21.15.7:45076) 2024-01-27T02:43:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.695+0000 7f2e74900700 1 -- 172.21.15.7:0/2712493388 >> v2:172.21.15.188:3300/0 conn(0x7f2e7015a100 msgr2=0x7f2e700cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.695+0000 7f2e74900700 1 --2- 172.21.15.7:0/2712493388 >> v2:172.21.15.188:3300/0 conn(0x7f2e7015a100 0x7f2e700cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.695+0000 7f2e74900700 1 -- 172.21.15.7:0/2712493388 >> v1:172.21.15.188:6789/0 conn(0x7f2e700c4af0 legacy=0x7f2e700c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.695+0000 7f2e74900700 1 -- 172.21.15.7:0/2712493388 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e700cdcb0 con 0x7f2e700cb5b0 2024-01-27T02:43:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.695+0000 7f2e75101700 1 --2- 172.21.15.7:0/2712493388 >> v2:172.21.15.188:3300/0 conn(0x7f2e7015a100 0x7f2e700cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:43:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.695+0000 7f2e74900700 1 --2- 172.21.15.7:0/2712493388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e700cb5b0 0x7f2e700c45b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2e600098e0 tx=0x7f2e600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40b93501fbaa5feb server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.696+0000 7f2e67fff700 1 -- 172.21.15.7:0/2712493388 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6000b640 con 0x7f2e700cb5b0 2024-01-27T02:43:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.696+0000 7f2e67fff700 1 -- 172.21.15.7:0/2712493388 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6000b7a0 con 0x7f2e700cb5b0 2024-01-27T02:43:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.696+0000 7f2e67fff700 1 -- 172.21.15.7:0/2712493388 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6000bb40 con 0x7f2e700cb5b0 2024-01-27T02:43:25.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.696+0000 7f2e77365700 1 -- 172.21.15.7:0/2712493388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e700cb5b0 msgr2=0x7f2e700c45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:25.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.696+0000 7f2e77365700 1 --2- 172.21.15.7:0/2712493388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e700cb5b0 0x7f2e700c45b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2e600098e0 tx=0x7f2e600091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:25.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.696+0000 7f2e77365700 1 -- 172.21.15.7:0/2712493388 shutdown_connections 2024-01-27T02:43:25.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.696+0000 7f2e77365700 1 --2- 172.21.15.7:0/2712493388 >> v2:172.21.15.188:3300/0 conn(0x7f2e7015a100 0x7f2e700cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:25.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.696+0000 7f2e77365700 1 --2- 172.21.15.7:0/2712493388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e700cb5b0 0x7f2e700c45b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:25.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.696+0000 7f2e77365700 1 -- 172.21.15.7:0/2712493388 >> 172.21.15.7:0/2712493388 conn(0x7f2e70098020 msgr2=0x7f2e700711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:25.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.696+0000 7f2e77365700 1 -- 172.21.15.7:0/2712493388 shutdown_connections 2024-01-27T02:43:25.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.696+0000 7f2e77365700 1 -- 172.21.15.7:0/2712493388 wait complete. 2024-01-27T02:43:25.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.698+0000 7f2e77365700 1 Processor -- start 2024-01-27T02:43:25.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.698+0000 7f2e77365700 1 -- start start 2024-01-27T02:43:25.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.699+0000 7f2e77365700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e700c4af0 0x7f2e701559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:25.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.699+0000 7f2e77365700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e700cb5b0 0x7f2e700c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:25.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.699+0000 7f2e77365700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e7015a100 0x7f2e700c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:25.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.699+0000 7f2e77365700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e700c91e0 con 0x7f2e700cb5b0 2024-01-27T02:43:25.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.699+0000 7f2e77365700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e701416d0 con 0x7f2e700c4af0 2024-01-27T02:43:25.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.699+0000 7f2e77365700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e700c85d0 con 0x7f2e7015a100 2024-01-27T02:43:25.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.699+0000 7f2e74900700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e700cb5b0 0x7f2e700c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:25.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.699+0000 7f2e74900700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e700cb5b0 0x7f2e700c7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42776/0 (socket says 172.21.15.7:42776) 2024-01-27T02:43:25.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.699+0000 7f2e74900700 1 -- 172.21.15.7:0/2367386026 learned_addr learned my addr 172.21.15.7:0/2367386026 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:25.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.699+0000 7f2e75101700 1 --2- 172.21.15.7:0/2367386026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e700c4af0 0x7f2e701559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:25.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.699+0000 7f2e75902700 1 --2- 172.21.15.7:0/2367386026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e7015a100 0x7f2e700c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:25.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.699+0000 7f2e74900700 1 -- 172.21.15.7:0/2367386026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e7015a100 msgr2=0x7f2e700c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:25.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.699+0000 7f2e74900700 1 --2- 172.21.15.7:0/2367386026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e7015a100 0x7f2e700c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:25.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.699+0000 7f2e74900700 1 -- 172.21.15.7:0/2367386026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e700c4af0 msgr2=0x7f2e701559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:25.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.699+0000 7f2e74900700 1 --2- 172.21.15.7:0/2367386026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e700c4af0 0x7f2e701559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:25.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.699+0000 7f2e74900700 1 -- 172.21.15.7:0/2367386026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6c009420 con 0x7f2e700cb5b0 2024-01-27T02:43:25.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.699+0000 7f2e74900700 1 --2- 172.21.15.7:0/2367386026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e700cb5b0 0x7f2e700c7820 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000a010 tx=0x7f2e60017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:25.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.700+0000 7f2e657fa700 1 -- 172.21.15.7:0/2367386026 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6001bc80 con 0x7f2e700cb5b0 2024-01-27T02:43:25.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.700+0000 7f2e657fa700 1 -- 172.21.15.7:0/2367386026 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6001be20 con 0x7f2e700cb5b0 2024-01-27T02:43:25.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.700+0000 7f2e657fa700 1 -- 172.21.15.7:0/2367386026 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e60011670 con 0x7f2e700cb5b0 2024-01-27T02:43:25.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.700+0000 7f2e77365700 1 -- 172.21.15.7:0/2367386026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e60009420 con 0x7f2e700cb5b0 2024-01-27T02:43:25.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.700+0000 7f2e77365700 1 -- 172.21.15.7:0/2367386026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e700c8b90 con 0x7f2e700cb5b0 2024-01-27T02:43:25.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.701+0000 7f2e77365700 1 -- 172.21.15.7:0/2367386026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e7004c710 con 0x7f2e700cb5b0 2024-01-27T02:43:25.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.701+0000 7f2e657fa700 1 -- 172.21.15.7:0/2367386026 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2e60011aa0 con 0x7f2e700cb5b0 2024-01-27T02:43:25.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.702+0000 7f2e657fa700 1 --2- 172.21.15.7:0/2367386026 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2e54066f30 0x7f2e540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:25.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.702+0000 7f2e657fa700 1 -- 172.21.15.7:0/2367386026 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2e60085a20 con 0x7f2e700cb5b0 2024-01-27T02:43:25.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.702+0000 7f2e75101700 1 --2- 172.21.15.7:0/2367386026 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2e54066f30 0x7f2e540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:25.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.702+0000 7f2e75101700 1 --2- 172.21.15.7:0/2367386026 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2e54066f30 0x7f2e540693e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2e700b18a0 tx=0x7f2e6c0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:25.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.706+0000 7f2e657fa700 1 -- 172.21.15.7:0/2367386026 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e60052f40 con 0x7f2e700cb5b0 2024-01-27T02:43:25.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.990+0000 7f2e77365700 1 -- 172.21.15.7:0/2367386026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2e700cdf90 con 0x7f2e700cb5b0 2024-01-27T02:43:25.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.991+0000 7f2e657fa700 1 -- 172.21.15.7:0/2367386026 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f2e60056560 con 0x7f2e700cb5b0 2024-01-27T02:43:25.993 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:25.993 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:43:25.993 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-01-27T02:43:25.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:25.993 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-01-27T02:43:25.993 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlHdcPKBAAfhnh1I3ZSQEXWJXlXCs98Q==", 2024-01-27T02:43:25.993 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:43:25.993 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:43:25.993 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:43:25.993 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:43:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:43:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-01-27T02:43:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rl8eVYNRAAfP8rOqO50RpoNZ1+1j2DXQ==", 2024-01-27T02:43:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:43:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:43:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:43:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:43:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:43:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-01-27T02:43:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlJxRKMBAAS0j7/zZeqGglOxCK1pg0WA==", 2024-01-27T02:43:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:43:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:43:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:43:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:43:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:43:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-01-27T02:43:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlNwfUABAALRQAoHoGhniqCvTQOqbbug==", 2024-01-27T02:43:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:43:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:43:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:43:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:43:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:43:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-01-27T02:43:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7Rlh6VSJhAAttuCte1WhNEvUkv5x/IKQQ==", 2024-01-27T02:43:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:43:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:43:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:43:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:43:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-01-27T02:43:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7Rlu/RGLhAAKzaCNhzYKhsNhxkMvY++gw==", 2024-01-27T02:43:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:43:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:43:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:43:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:43:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-01-27T02:43:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAOa7RliRMyNhAAr0tG3evmVV7ZlGxbxfWPMw==", 2024-01-27T02:43:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:43:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:43:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:43:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:43:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-01-27T02:43:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAPa7RlRuSNAhAA3e9atlb+ffvlnLhRYbWZPw==", 2024-01-27T02:43:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:43:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:43:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:43:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:43:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-01-27T02:43:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7RlfeA6DBAAXpklgbNHRO+5G+fdIERjRw==", 2024-01-27T02:43:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:43:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:43:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:43:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:43:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-01-27T02:43:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7Rl77ItExAABB7uABm63ubE/YUs2B5pGw==", 2024-01-27T02:43:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:43:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:43:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:43:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:43:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-01-27T02:43:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7RljDuwHBAAPPAlhoqS03qPpq58jmevLA==", 2024-01-27T02:43:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:43:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:43:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:43:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:43:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-01-27T02:43:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQASa7Rl1YElJhAAqHP/QEdl1SEalGcbQwPwAg==", 2024-01-27T02:43:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:43:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-27T02:43:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-27T02:43:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:43:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-01-27T02:43:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlaEqaBRAATFiIeW8lT9BdNuN0m3x9dQ==", 2024-01-27T02:43:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:43:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:43:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-01-27T02:43:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-27T02:43:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-01-27T02:43:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7RlNADmEBAAJXwdJ35ylP5coy1dtHoD3g==", 2024-01-27T02:43:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:43:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:43:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:43:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:43:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:43:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-01-27T02:43:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7Rl5fRqCRAA/5swTGknnpP64TAh+2X41A==", 2024-01-27T02:43:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:43:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:43:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:43:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:43:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:43:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-01-27T02:43:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQANa7Rl0SdwFhAAhCmw9ViiJbHVfS5a2r/D+w==", 2024-01-27T02:43:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:43:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-27T02:43:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-27T02:43:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-27T02:43:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-27T02:43:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-01-27T02:43:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7RlsNgPARAA0Qm0oJgA8GO1oUT3ndT8Tg==", 2024-01-27T02:43:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:43:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:43:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-27T02:43:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-27T02:43:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:43:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-01-27T02:43:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rlfb7qIxAAlsJYiSolJ7D8EykGWP62bA==", 2024-01-27T02:43:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:43:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:43:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-27T02:43:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:43:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-01-27T02:43:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAMa7Rls7TaHxAALS3ghxKtfmgQfIELiD9+Bg==", 2024-01-27T02:43:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-27T02:43:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-27T02:43:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-27T02:43:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-27T02:43:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:26.006 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:43:26.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.993+0000 7f2e77365700 1 -- 172.21.15.7:0/2367386026 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2e54066f30 msgr2=0x7f2e540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:26.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.993+0000 7f2e77365700 1 --2- 172.21.15.7:0/2367386026 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2e54066f30 0x7f2e540693e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2e700b18a0 tx=0x7f2e6c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.993+0000 7f2e77365700 1 -- 172.21.15.7:0/2367386026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e700cb5b0 msgr2=0x7f2e700c7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:26.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.993+0000 7f2e77365700 1 --2- 172.21.15.7:0/2367386026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e700cb5b0 0x7f2e700c7820 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000a010 tx=0x7f2e60017000 comp rx=0 tx=0).stop 2024-01-27T02:43:26.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.993+0000 7f2e77365700 1 -- 172.21.15.7:0/2367386026 shutdown_connections 2024-01-27T02:43:26.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.993+0000 7f2e77365700 1 --2- 172.21.15.7:0/2367386026 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2e54066f30 0x7f2e540693e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.993+0000 7f2e77365700 1 --2- 172.21.15.7:0/2367386026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e7015a100 0x7f2e700c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.993+0000 7f2e77365700 1 --2- 172.21.15.7:0/2367386026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e700c4af0 0x7f2e701559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.993+0000 7f2e77365700 1 --2- 172.21.15.7:0/2367386026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e700cb5b0 0x7f2e700c7820 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.993+0000 7f2e77365700 1 -- 172.21.15.7:0/2367386026 >> 172.21.15.7:0/2367386026 conn(0x7f2e70098020 msgr2=0x7f2e700caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:26.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.993+0000 7f2e77365700 1 -- 172.21.15.7:0/2367386026 shutdown_connections 2024-01-27T02:43:26.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:25.993+0000 7f2e77365700 1 -- 172.21.15.7:0/2367386026 wait complete. 2024-01-27T02:43:26.009 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:43:26.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.140+0000 7f4907c08700 1 Processor -- start 2024-01-27T02:43:26.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.140+0000 7f4907c08700 1 -- start start 2024-01-27T02:43:26.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.141+0000 7f4907c08700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4900157ca0 0x7f49000c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:26.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.141+0000 7f4907c08700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f49000cdbf0 0x7f49000ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:26.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.141+0000 7f4907c08700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49000ca020 con 0x7f49000c3ae0 2024-01-27T02:43:26.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.141+0000 7f4907c08700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49000c4f40 con 0x7f49000cdbf0 2024-01-27T02:43:26.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.141+0000 7f4907c08700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49000c5210 con 0x7f4900157ca0 2024-01-27T02:43:26.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.141+0000 7f49061a5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f49000cdbf0 0x7f49000ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:26.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.141+0000 7f49061a5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f49000cdbf0 0x7f49000ca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42782/0 (socket says 172.21.15.7:42782) 2024-01-27T02:43:26.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.141+0000 7f49061a5700 1 -- 172.21.15.7:0/2487671054 learned_addr learned my addr 172.21.15.7:0/2487671054 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:26.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.141+0000 7f49059a4700 1 --2- 172.21.15.7:0/2487671054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4900157ca0 0x7f49000c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:26.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.141+0000 7f49061a5700 1 -- 172.21.15.7:0/2487671054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4900157ca0 msgr2=0x7f49000c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:26.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.141+0000 7f49061a5700 1 --2- 172.21.15.7:0/2487671054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4900157ca0 0x7f49000c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.141+0000 7f49061a5700 1 -- 172.21.15.7:0/2487671054 >> v1:172.21.15.188:6789/0 conn(0x7f49000c3ae0 legacy=0x7f49000cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:26.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.141+0000 7f49061a5700 1 -- 172.21.15.7:0/2487671054 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49000c54f0 con 0x7f49000cdbf0 2024-01-27T02:43:26.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.141+0000 7f49059a4700 1 --2- 172.21.15.7:0/2487671054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4900157ca0 0x7f49000c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:26.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.141+0000 7f49061a5700 1 --2- 172.21.15.7:0/2487671054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49000cdbf0 0x7f49000ca4b0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009f20 tx=0x7f48f8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5244c6af798e9837 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:26.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.142+0000 7f49049a2700 1 -- 172.21.15.7:0/2487671054 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f800e070 con 0x7f49000cdbf0 2024-01-27T02:43:26.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.142+0000 7f49049a2700 1 -- 172.21.15.7:0/2487671054 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f800ba80 con 0x7f49000cdbf0 2024-01-27T02:43:26.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.142+0000 7f49049a2700 1 -- 172.21.15.7:0/2487671054 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f80092e0 con 0x7f49000cdbf0 2024-01-27T02:43:26.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.142+0000 7f4907c08700 1 -- 172.21.15.7:0/2487671054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49000cdbf0 msgr2=0x7f49000ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:26.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.142+0000 7f4907c08700 1 --2- 172.21.15.7:0/2487671054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49000cdbf0 0x7f49000ca4b0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009f20 tx=0x7f48f8009840 comp rx=0 tx=0).stop 2024-01-27T02:43:26.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.142+0000 7f4907c08700 1 -- 172.21.15.7:0/2487671054 shutdown_connections 2024-01-27T02:43:26.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.142+0000 7f4907c08700 1 --2- 172.21.15.7:0/2487671054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49000cdbf0 0x7f49000ca4b0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.142+0000 7f4907c08700 1 --2- 172.21.15.7:0/2487671054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4900157ca0 0x7f49000c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.142+0000 7f4907c08700 1 -- 172.21.15.7:0/2487671054 >> 172.21.15.7:0/2487671054 conn(0x7f4900098020 msgr2=0x7f49000711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:26.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.142+0000 7f4907c08700 1 -- 172.21.15.7:0/2487671054 shutdown_connections 2024-01-27T02:43:26.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.142+0000 7f4907c08700 1 -- 172.21.15.7:0/2487671054 wait complete. 2024-01-27T02:43:26.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.144+0000 7f4907c08700 1 Processor -- start 2024-01-27T02:43:26.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.144+0000 7f4907c08700 1 -- start start 2024-01-27T02:43:26.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.144+0000 7f4907c08700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49000c3ae0 0x7f490014dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:26.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.144+0000 7f4907c08700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49000cdbf0 0x7f490016d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:26.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.144+0000 7f4907c08700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4900157ca0 0x7f490006c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:26.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.144+0000 7f4907c08700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49000b2de0 con 0x7f49000cdbf0 2024-01-27T02:43:26.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.144+0000 7f4907c08700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49000b3360 con 0x7f49000c3ae0 2024-01-27T02:43:26.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.144+0000 7f4907c08700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490016e130 con 0x7f4900157ca0 2024-01-27T02:43:26.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.144+0000 7f49051a3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49000cdbf0 0x7f490016d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:26.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.145+0000 7f49051a3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49000cdbf0 0x7f490016d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42786/0 (socket says 172.21.15.7:42786) 2024-01-27T02:43:26.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.145+0000 7f49051a3700 1 -- 172.21.15.7:0/1491327303 learned_addr learned my addr 172.21.15.7:0/1491327303 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.145+0000 7f49059a4700 1 --2- 172.21.15.7:0/1491327303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49000c3ae0 0x7f490014dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.145+0000 7f49061a5700 1 --2- 172.21.15.7:0/1491327303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4900157ca0 0x7f490006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.145+0000 7f49051a3700 1 -- 172.21.15.7:0/1491327303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4900157ca0 msgr2=0x7f490006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.145+0000 7f49051a3700 1 --2- 172.21.15.7:0/1491327303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4900157ca0 0x7f490006c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.145+0000 7f49051a3700 1 -- 172.21.15.7:0/1491327303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49000c3ae0 msgr2=0x7f490014dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.145+0000 7f49051a3700 1 --2- 172.21.15.7:0/1491327303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49000c3ae0 0x7f490014dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.145+0000 7f49051a3700 1 -- 172.21.15.7:0/1491327303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f8009ae0 con 0x7f49000cdbf0 2024-01-27T02:43:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.145+0000 7f49061a5700 1 --2- 172.21.15.7:0/1491327303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4900157ca0 0x7f490006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:43:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.145+0000 7f49059a4700 1 --2- 172.21.15.7:0/1491327303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49000c3ae0 0x7f490014dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.145+0000 7f49051a3700 1 --2- 172.21.15.7:0/1491327303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49000cdbf0 0x7f490016d700 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00c590 tx=0x7f48fc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.145+0000 7f48f1ffb700 1 -- 172.21.15.7:0/1491327303 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc011590 con 0x7f49000cdbf0 2024-01-27T02:43:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.145+0000 7f48f1ffb700 1 -- 172.21.15.7:0/1491327303 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48fc0116f0 con 0x7f49000cdbf0 2024-01-27T02:43:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.145+0000 7f4907c08700 1 -- 172.21.15.7:0/1491327303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490016e410 con 0x7f49000cdbf0 2024-01-27T02:43:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.145+0000 7f48f1ffb700 1 -- 172.21.15.7:0/1491327303 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc011ab0 con 0x7f49000cdbf0 2024-01-27T02:43:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.145+0000 7f4907c08700 1 -- 172.21.15.7:0/1491327303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490014e3e0 con 0x7f49000cdbf0 2024-01-27T02:43:26.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.146+0000 7f4907c08700 1 -- 172.21.15.7:0/1491327303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490004c710 con 0x7f49000cdbf0 2024-01-27T02:43:26.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.147+0000 7f48f1ffb700 1 -- 172.21.15.7:0/1491327303 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f48fc01b3b0 con 0x7f49000cdbf0 2024-01-27T02:43:26.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.147+0000 7f48f1ffb700 1 --2- 172.21.15.7:0/1491327303 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f48e4066f30 0x7f48e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:26.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.147+0000 7f48f1ffb700 1 -- 172.21.15.7:0/1491327303 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f48fc0837e0 con 0x7f49000cdbf0 2024-01-27T02:43:26.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.148+0000 7f49059a4700 1 --2- 172.21.15.7:0/1491327303 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f48e4066f30 0x7f48e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:26.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.148+0000 7f49059a4700 1 --2- 172.21.15.7:0/1491327303 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f48e4066f30 0x7f48e40693e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f48f4006f00 tx=0x7f48f4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:26.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.152+0000 7f48f1ffb700 1 -- 172.21.15.7:0/1491327303 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48fc050d00 con 0x7f49000cdbf0 2024-01-27T02:43:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.358+0000 7f4907c08700 1 -- 172.21.15.7:0/1491327303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49000c57d0 con 0x7f49000cdbf0 2024-01-27T02:43:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.359+0000 7f48f1ffb700 1 -- 172.21.15.7:0/1491327303 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+12097 (secure 0 0 0) 0x7f48fc017090 con 0x7f49000cdbf0 2024-01-27T02:43:26.361 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:26.361 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":176,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:43:25.536188+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:43:26.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.362+0000 7f4907c08700 1 -- 172.21.15.7:0/1491327303 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f48e4066f30 msgr2=0x7f48e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:26.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.362+0000 7f4907c08700 1 --2- 172.21.15.7:0/1491327303 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f48e4066f30 0x7f48e40693e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f48f4006f00 tx=0x7f48f4008040 comp rx=0 tx=0).stop 2024-01-27T02:43:26.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.362+0000 7f4907c08700 1 -- 172.21.15.7:0/1491327303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49000cdbf0 msgr2=0x7f490016d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:26.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.362+0000 7f4907c08700 1 --2- 172.21.15.7:0/1491327303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49000cdbf0 0x7f490016d700 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00c590 tx=0x7f48fc00c950 comp rx=0 tx=0).stop 2024-01-27T02:43:26.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.362+0000 7f4907c08700 1 -- 172.21.15.7:0/1491327303 shutdown_connections 2024-01-27T02:43:26.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.362+0000 7f4907c08700 1 --2- 172.21.15.7:0/1491327303 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f48e4066f30 0x7f48e40693e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.362+0000 7f4907c08700 1 --2- 172.21.15.7:0/1491327303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4900157ca0 0x7f490006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.362+0000 7f4907c08700 1 --2- 172.21.15.7:0/1491327303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49000c3ae0 0x7f490014dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.362+0000 7f4907c08700 1 --2- 172.21.15.7:0/1491327303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49000cdbf0 0x7f490016d700 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.362+0000 7f4907c08700 1 -- 172.21.15.7:0/1491327303 >> 172.21.15.7:0/1491327303 conn(0x7f4900098020 msgr2=0x7f4900153150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:26.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.362+0000 7f4907c08700 1 -- 172.21.15.7:0/1491327303 shutdown_connections 2024-01-27T02:43:26.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.362+0000 7f4907c08700 1 -- 172.21.15.7:0/1491327303 wait complete. 2024-01-27T02:43:26.377 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-27T02:43:26.377 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:43:26.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.511+0000 7efc5513c700 1 Processor -- start 2024-01-27T02:43:26.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.511+0000 7efc5513c700 1 -- start start 2024-01-27T02:43:26.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.512+0000 7efc5513c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc500cbd20 0x7efc500b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:26.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.512+0000 7efc5513c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc500c9660 con 0x7efc501412d0 2024-01-27T02:43:26.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.512+0000 7efc5513c700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc5015be00 con 0x7efc500cb950 2024-01-27T02:43:26.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.512+0000 7efc5513c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5015c980 con 0x7efc500cbd20 2024-01-27T02:43:26.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.512+0000 7efc4f59e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efc501412d0 0x7efc50141730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40330/0 (socket says 172.21.15.7:40330) 2024-01-27T02:43:26.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.512+0000 7efc4f59e700 1 -- 172.21.15.7:0/3805463780 learned_addr learned my addr 172.21.15.7:0/3805463780 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:26.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.512+0000 7efc4e59c700 1 --2- 172.21.15.7:0/3805463780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc500cbd20 0x7efc500b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:26.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.512+0000 7efc4dd9b700 1 -- 172.21.15.7:0/3805463780 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799909465 0 0) 0x7efc500c9660 con 0x7efc501412d0 2024-01-27T02:43:26.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.513+0000 7efc4dd9b700 1 -- 172.21.15.7:0/3805463780 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc3c003570 con 0x7efc501412d0 2024-01-27T02:43:26.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.513+0000 7efc4dd9b700 1 -- 172.21.15.7:0/3805463780 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 465348607 0 0) 0x7efc5015be00 con 0x7efc500cb950 2024-01-27T02:43:26.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.513+0000 7efc4dd9b700 1 -- 172.21.15.7:0/3805463780 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc500c9660 con 0x7efc500cb950 2024-01-27T02:43:26.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.513+0000 7efc4e59c700 1 -- 172.21.15.7:0/3805463780 >> v1:172.21.15.188:6789/0 conn(0x7efc500cb950 legacy=0x7efc500ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:26.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.513+0000 7efc4e59c700 1 -- 172.21.15.7:0/3805463780 >> v1:172.21.15.7:6789/0 conn(0x7efc501412d0 legacy=0x7efc50141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:26.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.513+0000 7efc4e59c700 1 -- 172.21.15.7:0/3805463780 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5015c0e0 con 0x7efc500cbd20 2024-01-27T02:43:26.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.513+0000 7efc4e59c700 1 --2- 172.21.15.7:0/3805463780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc500cbd20 0x7efc500b1ac0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efc38009910 tx=0x7efc380091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4df37d3ca1797f8b server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:26.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.513+0000 7efc4dd9b700 1 -- 172.21.15.7:0/3805463780 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc38011630 con 0x7efc500cbd20 2024-01-27T02:43:26.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.513+0000 7efc4dd9b700 1 -- 172.21.15.7:0/3805463780 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc38011790 con 0x7efc500cbd20 2024-01-27T02:43:26.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.513+0000 7efc4dd9b700 1 -- 172.21.15.7:0/3805463780 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc38011b30 con 0x7efc500cbd20 2024-01-27T02:43:26.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.514+0000 7efc5513c700 1 -- 172.21.15.7:0/3805463780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc500cbd20 msgr2=0x7efc500b1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:26.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.514+0000 7efc5513c700 1 --2- 172.21.15.7:0/3805463780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc500cbd20 0x7efc500b1ac0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efc38009910 tx=0x7efc380091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.514+0000 7efc5513c700 1 -- 172.21.15.7:0/3805463780 shutdown_connections 2024-01-27T02:43:26.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.514+0000 7efc5513c700 1 --2- 172.21.15.7:0/3805463780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc500cbd20 0x7efc500b1ac0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.514+0000 7efc5513c700 1 -- 172.21.15.7:0/3805463780 >> 172.21.15.7:0/3805463780 conn(0x7efc50063580 msgr2=0x7efc50063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:26.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.514+0000 7efc5513c700 1 -- 172.21.15.7:0/3805463780 shutdown_connections 2024-01-27T02:43:26.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.514+0000 7efc5513c700 1 -- 172.21.15.7:0/3805463780 wait complete. 2024-01-27T02:43:26.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.515+0000 7efc5513c700 1 Processor -- start 2024-01-27T02:43:26.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.515+0000 7efc5513c700 1 -- start start 2024-01-27T02:43:26.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.516+0000 7efc5513c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc500cb950 0x7efc501683c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:26.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.516+0000 7efc5513c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc500cbd20 0x7efc500b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:26.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.516+0000 7efc5513c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc501412d0 0x7efc5015a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:26.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.516+0000 7efc5513c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc500b2120 con 0x7efc500cbd20 2024-01-27T02:43:26.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.516+0000 7efc5513c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc500b1470 con 0x7efc500cb950 2024-01-27T02:43:26.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.516+0000 7efc5513c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc50159b40 con 0x7efc501412d0 2024-01-27T02:43:26.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.516+0000 7efc4e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc500cbd20 0x7efc500b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:26.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.516+0000 7efc4e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc500cbd20 0x7efc500b0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42790/0 (socket says 172.21.15.7:42790) 2024-01-27T02:43:26.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.516+0000 7efc4e59c700 1 -- 172.21.15.7:0/1890930227 learned_addr learned my addr 172.21.15.7:0/1890930227 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:26.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.516+0000 7efc4f59e700 1 --2- 172.21.15.7:0/1890930227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc501412d0 0x7efc5015a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:26.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.516+0000 7efc4ed9d700 1 --2- 172.21.15.7:0/1890930227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc500cb950 0x7efc501683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:26.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.517+0000 7efc4e59c700 1 -- 172.21.15.7:0/1890930227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc501412d0 msgr2=0x7efc5015a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:26.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.517+0000 7efc4e59c700 1 --2- 172.21.15.7:0/1890930227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc501412d0 0x7efc5015a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.517+0000 7efc4e59c700 1 -- 172.21.15.7:0/1890930227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc500cb950 msgr2=0x7efc501683c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:26.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.517+0000 7efc4e59c700 1 --2- 172.21.15.7:0/1890930227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc500cb950 0x7efc501683c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.517+0000 7efc4e59c700 1 -- 172.21.15.7:0/1890930227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc38009420 con 0x7efc500cbd20 2024-01-27T02:43:26.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.517+0000 7efc4e59c700 1 --2- 172.21.15.7:0/1890930227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc500cbd20 0x7efc500b0f30 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7efc38009f00 tx=0x7efc38017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:26.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.517+0000 7efc46ffd700 1 -- 172.21.15.7:0/1890930227 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3801f070 con 0x7efc500cbd20 2024-01-27T02:43:26.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.517+0000 7efc46ffd700 1 -- 172.21.15.7:0/1890930227 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc38002f70 con 0x7efc500cbd20 2024-01-27T02:43:26.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.517+0000 7efc5513c700 1 -- 172.21.15.7:0/1890930227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc500b2330 con 0x7efc500cbd20 2024-01-27T02:43:26.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.517+0000 7efc46ffd700 1 -- 172.21.15.7:0/1890930227 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc38011900 con 0x7efc500cbd20 2024-01-27T02:43:26.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.517+0000 7efc5513c700 1 -- 172.21.15.7:0/1890930227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc500b28f0 con 0x7efc500cbd20 2024-01-27T02:43:26.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.518+0000 7efc5513c700 1 -- 172.21.15.7:0/1890930227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc500c4be0 con 0x7efc500cbd20 2024-01-27T02:43:26.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.518+0000 7efc46ffd700 1 -- 172.21.15.7:0/1890930227 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7efc38023070 con 0x7efc500cbd20 2024-01-27T02:43:26.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.519+0000 7efc46ffd700 1 --2- 172.21.15.7:0/1890930227 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7efc30066f30 0x7efc300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:26.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.519+0000 7efc46ffd700 1 -- 172.21.15.7:0/1890930227 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 4853+0+0 (secure 0 0 0) 0x7efc38085460 con 0x7efc500cbd20 2024-01-27T02:43:26.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.519+0000 7efc4ed9d700 1 --2- 172.21.15.7:0/1890930227 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7efc30066f30 0x7efc300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:26.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.520+0000 7efc4ed9d700 1 --2- 172.21.15.7:0/1890930227 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7efc30066f30 0x7efc300693e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7efc40007fd0 tx=0x7efc40007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:26.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.523+0000 7efc46ffd700 1 -- 172.21.15.7:0/1890930227 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc38052a20 con 0x7efc500cbd20 2024-01-27T02:43:26.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.725+0000 7efc5513c700 1 -- 172.21.15.7:0/1890930227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc500ca1b0 con 0x7efc500cbd20 2024-01-27T02:43:26.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.726+0000 7efc46ffd700 1 -- 172.21.15.7:0/1890930227 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v176) v1 ==== 93+0+10 (secure 0 0 0) 0x7efc3802a020 con 0x7efc500cbd20 2024-01-27T02:43:26.728 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:43:26.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.728+0000 7efc5513c700 1 -- 172.21.15.7:0/1890930227 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7efc30066f30 msgr2=0x7efc300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:26.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.728+0000 7efc5513c700 1 --2- 172.21.15.7:0/1890930227 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7efc30066f30 0x7efc300693e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7efc40007fd0 tx=0x7efc40007d50 comp rx=0 tx=0).stop 2024-01-27T02:43:26.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.728+0000 7efc5513c700 1 -- 172.21.15.7:0/1890930227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc500cbd20 msgr2=0x7efc500b0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:26.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.728+0000 7efc5513c700 1 --2- 172.21.15.7:0/1890930227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc500cbd20 0x7efc500b0f30 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7efc38009f00 tx=0x7efc38017000 comp rx=0 tx=0).stop 2024-01-27T02:43:26.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.728+0000 7efc5513c700 1 -- 172.21.15.7:0/1890930227 shutdown_connections 2024-01-27T02:43:26.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.728+0000 7efc5513c700 1 --2- 172.21.15.7:0/1890930227 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7efc30066f30 0x7efc300693e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.728+0000 7efc5513c700 1 --2- 172.21.15.7:0/1890930227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc501412d0 0x7efc5015a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.728+0000 7efc5513c700 1 --2- 172.21.15.7:0/1890930227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc500cb950 0x7efc501683c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.728+0000 7efc5513c700 1 --2- 172.21.15.7:0/1890930227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc500cbd20 0x7efc500b0f30 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.728+0000 7efc5513c700 1 -- 172.21.15.7:0/1890930227 >> 172.21.15.7:0/1890930227 conn(0x7efc50063580 msgr2=0x7efc500cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:26.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.728+0000 7efc5513c700 1 -- 172.21.15.7:0/1890930227 shutdown_connections 2024-01-27T02:43:26.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.728+0000 7efc5513c700 1 -- 172.21.15.7:0/1890930227 wait complete. 2024-01-27T02:43:26.744 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-27T02:43:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.874+0000 7ff234731700 1 Processor -- start 2024-01-27T02:43:26.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.874+0000 7ff234731700 1 -- start start 2024-01-27T02:43:26.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.875+0000 7ff234731700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff22c0b1c00 0x7ff22c06e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:26.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.875+0000 7ff234731700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff22c15b5f0 0x7ff22c1566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:26.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.875+0000 7ff234731700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff22c157610 con 0x7ff22c0b1fd0 2024-01-27T02:43:26.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.875+0000 7ff234731700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff22c157310 con 0x7ff22c0b1c00 2024-01-27T02:43:26.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.875+0000 7ff234731700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff22c06ee00 con 0x7ff22c15b5f0 2024-01-27T02:43:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.875+0000 7ff2324cd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff22c0b1c00 0x7ff22c06e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.875+0000 7ff2324cd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff22c0b1c00 0x7ff22c06e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42796/0 (socket says 172.21.15.7:42796) 2024-01-27T02:43:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.875+0000 7ff2324cd700 1 -- 172.21.15.7:0/3018862179 learned_addr learned my addr 172.21.15.7:0/3018862179 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.875+0000 7ff231ccc700 1 --2- 172.21.15.7:0/3018862179 >> v2:172.21.15.188:3300/0 conn(0x7ff22c15b5f0 0x7ff22c1566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.875+0000 7ff2324cd700 1 -- 172.21.15.7:0/3018862179 >> v2:172.21.15.188:3300/0 conn(0x7ff22c15b5f0 msgr2=0x7ff22c1566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.875+0000 7ff2324cd700 1 --2- 172.21.15.7:0/3018862179 >> v2:172.21.15.188:3300/0 conn(0x7ff22c15b5f0 0x7ff22c1566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.875+0000 7ff2324cd700 1 -- 172.21.15.7:0/3018862179 >> v1:172.21.15.188:6789/0 conn(0x7ff22c0b1fd0 legacy=0x7ff22c156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.876+0000 7ff2324cd700 1 -- 172.21.15.7:0/3018862179 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff22c06f0e0 con 0x7ff22c0b1c00 2024-01-27T02:43:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.876+0000 7ff2324cd700 1 --2- 172.21.15.7:0/3018862179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff22c0b1c00 0x7ff22c06e540 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7ff22c157580 tx=0x7ff228009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4802967823f58e1 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.876+0000 7ff2314cb700 1 -- 172.21.15.7:0/3018862179 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22800b860 con 0x7ff22c0b1c00 2024-01-27T02:43:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.876+0000 7ff2314cb700 1 -- 172.21.15.7:0/3018862179 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22800b9c0 con 0x7ff22c0b1c00 2024-01-27T02:43:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.876+0000 7ff2314cb700 1 -- 172.21.15.7:0/3018862179 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22800bd60 con 0x7ff22c0b1c00 2024-01-27T02:43:26.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.876+0000 7ff234731700 1 -- 172.21.15.7:0/3018862179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff22c0b1c00 msgr2=0x7ff22c06e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:26.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.876+0000 7ff234731700 1 --2- 172.21.15.7:0/3018862179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff22c0b1c00 0x7ff22c06e540 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7ff22c157580 tx=0x7ff228009250 comp rx=0 tx=0).stop 2024-01-27T02:43:26.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.877+0000 7ff234731700 1 -- 172.21.15.7:0/3018862179 shutdown_connections 2024-01-27T02:43:26.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.877+0000 7ff234731700 1 --2- 172.21.15.7:0/3018862179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff22c0b1c00 0x7ff22c06e540 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.877+0000 7ff234731700 1 --2- 172.21.15.7:0/3018862179 >> v2:172.21.15.188:3300/0 conn(0x7ff22c15b5f0 0x7ff22c1566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.877+0000 7ff234731700 1 -- 172.21.15.7:0/3018862179 >> 172.21.15.7:0/3018862179 conn(0x7ff22c068430 msgr2=0x7ff22c06be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:26.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.877+0000 7ff234731700 1 -- 172.21.15.7:0/3018862179 shutdown_connections 2024-01-27T02:43:26.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.877+0000 7ff234731700 1 -- 172.21.15.7:0/3018862179 wait complete. 2024-01-27T02:43:26.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.878+0000 7ff234731700 1 Processor -- start 2024-01-27T02:43:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.878+0000 7ff234731700 1 -- start start 2024-01-27T02:43:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.879+0000 7ff234731700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff22c0b1c00 0x7ff22c0af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.879+0000 7ff234731700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff22c0b1fd0 0x7ff22c1492c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.879+0000 7ff234731700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff22c15b5f0 0x7ff22c153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.879+0000 7ff234731700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c0b0060 con 0x7ff22c15b5f0 2024-01-27T02:43:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.879+0000 7ff234731700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c168510 con 0x7ff22c0b1fd0 2024-01-27T02:43:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.879+0000 7ff234731700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c168080 con 0x7ff22c0b1c00 2024-01-27T02:43:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.879+0000 7ff232cce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff22c15b5f0 0x7ff22c153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.879+0000 7ff232cce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff22c15b5f0 0x7ff22c153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42798/0 (socket says 172.21.15.7:42798) 2024-01-27T02:43:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.879+0000 7ff232cce700 1 -- 172.21.15.7:0/2869059461 learned_addr learned my addr 172.21.15.7:0/2869059461 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.879+0000 7ff231ccc700 1 --2- 172.21.15.7:0/2869059461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff22c0b1fd0 0x7ff22c1492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.879+0000 7ff2324cd700 1 --2- 172.21.15.7:0/2869059461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff22c0b1c00 0x7ff22c0af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.879+0000 7ff232cce700 1 -- 172.21.15.7:0/2869059461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff22c0b1c00 msgr2=0x7ff22c0af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.879+0000 7ff232cce700 1 --2- 172.21.15.7:0/2869059461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff22c0b1c00 0x7ff22c0af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.879+0000 7ff232cce700 1 -- 172.21.15.7:0/2869059461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff22c0b1fd0 msgr2=0x7ff22c1492c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.879+0000 7ff232cce700 1 --2- 172.21.15.7:0/2869059461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff22c0b1fd0 0x7ff22c1492c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.879+0000 7ff232cce700 1 -- 172.21.15.7:0/2869059461 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2280094f0 con 0x7ff22c15b5f0 2024-01-27T02:43:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.879+0000 7ff232cce700 1 --2- 172.21.15.7:0/2869059461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff22c15b5f0 0x7ff22c153f90 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7ff22400ed50 tx=0x7ff22400c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.879+0000 7ff2227fc700 1 -- 172.21.15.7:0/2869059461 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff224004120 con 0x7ff22c15b5f0 2024-01-27T02:43:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.879+0000 7ff2227fc700 1 -- 172.21.15.7:0/2869059461 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff224011470 con 0x7ff22c15b5f0 2024-01-27T02:43:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.879+0000 7ff2227fc700 1 -- 172.21.15.7:0/2869059461 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff224011830 con 0x7ff22c15b5f0 2024-01-27T02:43:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.880+0000 7ff234731700 1 -- 172.21.15.7:0/2869059461 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff22c1674f0 con 0x7ff22c15b5f0 2024-01-27T02:43:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.880+0000 7ff234731700 1 -- 172.21.15.7:0/2869059461 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff22c167a30 con 0x7ff22c15b5f0 2024-01-27T02:43:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.881+0000 7ff234731700 1 -- 172.21.15.7:0/2869059461 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff22c168230 con 0x7ff22c15b5f0 2024-01-27T02:43:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.881+0000 7ff2227fc700 1 -- 172.21.15.7:0/2869059461 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff224011ba0 con 0x7ff22c15b5f0 2024-01-27T02:43:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.882+0000 7ff2227fc700 1 --2- 172.21.15.7:0/2869059461 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff210066f30 0x7ff2100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:26.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.882+0000 7ff2227fc700 1 -- 172.21.15.7:0/2869059461 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff224015070 con 0x7ff22c15b5f0 2024-01-27T02:43:26.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.882+0000 7ff2324cd700 1 --2- 172.21.15.7:0/2869059461 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff210066f30 0x7ff2100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:26.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.882+0000 7ff2324cd700 1 --2- 172.21.15.7:0/2869059461 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff210066f30 0x7ff2100693e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff22c157580 tx=0x7ff228017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:26.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:26.886+0000 7ff2227fc700 1 -- 172.21.15.7:0/2869059461 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff22c168230 con 0x7ff22c15b5f0 2024-01-27T02:43:27.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.093+0000 7ff234731700 1 -- 172.21.15.7:0/2869059461 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff22c168230 con 0x7ff22c15b5f0 2024-01-27T02:43:27.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.093+0000 7ff2227fc700 1 -- 172.21.15.7:0/2869059461 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v176) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff22401e060 con 0x7ff22c15b5f0 2024-01-27T02:43:27.095 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:27.095 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-27T02:43:27.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:27.095 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-27T02:43:27.095 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-27T02:43:27.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:27.095 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-27T02:43:27.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.095+0000 7ff234731700 1 -- 172.21.15.7:0/2869059461 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff210066f30 msgr2=0x7ff2100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:27.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.095+0000 7ff234731700 1 --2- 172.21.15.7:0/2869059461 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff210066f30 0x7ff2100693e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff22c157580 tx=0x7ff228017000 comp rx=0 tx=0).stop 2024-01-27T02:43:27.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.095+0000 7ff234731700 1 -- 172.21.15.7:0/2869059461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff22c15b5f0 msgr2=0x7ff22c153f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:27.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.095+0000 7ff234731700 1 --2- 172.21.15.7:0/2869059461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff22c15b5f0 0x7ff22c153f90 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7ff22400ed50 tx=0x7ff22400c570 comp rx=0 tx=0).stop 2024-01-27T02:43:27.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.095+0000 7ff234731700 1 -- 172.21.15.7:0/2869059461 shutdown_connections 2024-01-27T02:43:27.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.095+0000 7ff234731700 1 --2- 172.21.15.7:0/2869059461 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ff210066f30 0x7ff2100693e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:27.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.095+0000 7ff234731700 1 --2- 172.21.15.7:0/2869059461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff22c15b5f0 0x7ff22c153f90 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:27.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.095+0000 7ff234731700 1 --2- 172.21.15.7:0/2869059461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff22c0b1c00 0x7ff22c0af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:27.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.095+0000 7ff234731700 1 --2- 172.21.15.7:0/2869059461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff22c0b1fd0 0x7ff22c1492c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:27.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.095+0000 7ff234731700 1 -- 172.21.15.7:0/2869059461 >> 172.21.15.7:0/2869059461 conn(0x7ff22c068430 msgr2=0x7ff22c14a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:27.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.096+0000 7ff234731700 1 -- 172.21.15.7:0/2869059461 shutdown_connections 2024-01-27T02:43:27.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.096+0000 7ff234731700 1 -- 172.21.15.7:0/2869059461 wait complete. 2024-01-27T02:43:27.110 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-27T02:43:27.111 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-27T02:43:27.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.245+0000 7f8be4ba4700 1 Processor -- start 2024-01-27T02:43:27.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.245+0000 7f8be4ba4700 1 -- start start 2024-01-27T02:43:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.246+0000 7f8be4ba4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8be00b1b70 0x7f8be006e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.246+0000 7f8be4ba4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8be0156ec0 0x7f8be0157990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.246+0000 7f8be4ba4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8be006ea80 con 0x7f8be015a960 2024-01-27T02:43:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.246+0000 7f8be4ba4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8be006ed20 con 0x7f8be0156ec0 2024-01-27T02:43:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.246+0000 7f8be4ba4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be006f000 con 0x7f8be00b1b70 2024-01-27T02:43:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.246+0000 7f8bded9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8be0156ec0 0x7f8be0157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.246+0000 7f8bddd9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8be015a960 0x7f8be01567b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40336/0 (socket says 172.21.15.7:40336) 2024-01-27T02:43:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.246+0000 7f8bddd9b700 1 -- 172.21.15.7:0/24871868 learned_addr learned my addr 172.21.15.7:0/24871868 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.246+0000 7f8bde59c700 1 --2- 172.21.15.7:0/24871868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8be00b1b70 0x7f8be006e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.246+0000 7f8bded9d700 1 -- 172.21.15.7:0/24871868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8be00b1b70 msgr2=0x7f8be006e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.246+0000 7f8bded9d700 1 --2- 172.21.15.7:0/24871868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8be00b1b70 0x7f8be006e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.247+0000 7f8bded9d700 1 -- 172.21.15.7:0/24871868 >> v1:172.21.15.7:6789/0 conn(0x7f8be015a960 legacy=0x7f8be01567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.247+0000 7f8bded9d700 1 -- 172.21.15.7:0/24871868 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be006f2e0 con 0x7f8be0156ec0 2024-01-27T02:43:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.247+0000 7f8bdd59a700 1 -- 172.21.15.7:0/24871868 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2057605462 0 0) 0x7f8be006ea80 con 0x7f8be015a960 2024-01-27T02:43:27.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.247+0000 7f8bded9d700 1 --2- 172.21.15.7:0/24871868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be0156ec0 0x7f8be0157990 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009e60 tx=0x7f8bd8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af67c5b0cf5ec server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:27.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.247+0000 7f8bdd59a700 1 -- 172.21.15.7:0/24871868 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd800e070 con 0x7f8be0156ec0 2024-01-27T02:43:27.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.247+0000 7f8bdd59a700 1 -- 172.21.15.7:0/24871868 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd800bb60 con 0x7f8be0156ec0 2024-01-27T02:43:27.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.247+0000 7f8bdd59a700 1 -- 172.21.15.7:0/24871868 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8012430 con 0x7f8be0156ec0 2024-01-27T02:43:27.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.247+0000 7f8be4ba4700 1 -- 172.21.15.7:0/24871868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be0156ec0 msgr2=0x7f8be0157990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:27.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.247+0000 7f8be4ba4700 1 --2- 172.21.15.7:0/24871868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be0156ec0 0x7f8be0157990 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009e60 tx=0x7f8bd8009780 comp rx=0 tx=0).stop 2024-01-27T02:43:27.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.248+0000 7f8be4ba4700 1 -- 172.21.15.7:0/24871868 shutdown_connections 2024-01-27T02:43:27.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.248+0000 7f8be4ba4700 1 --2- 172.21.15.7:0/24871868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be0156ec0 0x7f8be0157990 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:27.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.248+0000 7f8be4ba4700 1 --2- 172.21.15.7:0/24871868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8be00b1b70 0x7f8be006e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:27.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.248+0000 7f8be4ba4700 1 -- 172.21.15.7:0/24871868 >> 172.21.15.7:0/24871868 conn(0x7f8be0068480 msgr2=0x7f8be006be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:27.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.248+0000 7f8be4ba4700 1 -- 172.21.15.7:0/24871868 shutdown_connections 2024-01-27T02:43:27.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.248+0000 7f8be4ba4700 1 -- 172.21.15.7:0/24871868 wait complete. 2024-01-27T02:43:27.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.249+0000 7f8be4ba4700 1 Processor -- start 2024-01-27T02:43:27.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.249+0000 7f8be4ba4700 1 -- start start 2024-01-27T02:43:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.250+0000 7f8be4ba4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8be00b1b70 0x7f8be0167c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.250+0000 7f8be4ba4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be0156ec0 0x7f8be0164670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.250+0000 7f8be4ba4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8be015a960 0x7f8be01694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.250+0000 7f8be4ba4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be0169060 con 0x7f8be0156ec0 2024-01-27T02:43:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.250+0000 7f8be4ba4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be013cfd0 con 0x7f8be00b1b70 2024-01-27T02:43:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.250+0000 7f8be4ba4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be0168450 con 0x7f8be015a960 2024-01-27T02:43:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.250+0000 7f8bddd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be0156ec0 0x7f8be0164670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.250+0000 7f8bddd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be0156ec0 0x7f8be0164670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42818/0 (socket says 172.21.15.7:42818) 2024-01-27T02:43:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.250+0000 7f8bddd9b700 1 -- 172.21.15.7:0/596311365 learned_addr learned my addr 172.21.15.7:0/596311365 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.250+0000 7f8bde59c700 1 --2- 172.21.15.7:0/596311365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8be00b1b70 0x7f8be0167c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.250+0000 7f8bded9d700 1 --2- 172.21.15.7:0/596311365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8be015a960 0x7f8be01694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.250+0000 7f8bddd9b700 1 -- 172.21.15.7:0/596311365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8be015a960 msgr2=0x7f8be01694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.250+0000 7f8bddd9b700 1 --2- 172.21.15.7:0/596311365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8be015a960 0x7f8be01694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.250+0000 7f8bddd9b700 1 -- 172.21.15.7:0/596311365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8be00b1b70 msgr2=0x7f8be0167c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.250+0000 7f8bddd9b700 1 --2- 172.21.15.7:0/596311365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8be00b1b70 0x7f8be0167c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.250+0000 7f8bddd9b700 1 -- 172.21.15.7:0/596311365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd8009a20 con 0x7f8be0156ec0 2024-01-27T02:43:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.250+0000 7f8bde59c700 1 --2- 172.21.15.7:0/596311365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8be00b1b70 0x7f8be0167c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.250+0000 7f8bded9d700 1 --2- 172.21.15.7:0/596311365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8be015a960 0x7f8be01694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.250+0000 7f8bddd9b700 1 --2- 172.21.15.7:0/596311365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be0156ec0 0x7f8be0164670 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400c7e0 tx=0x7f8bd4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.251+0000 7f8bce7fc700 1 -- 172.21.15.7:0/596311365 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4005b30 con 0x7f8be0156ec0 2024-01-27T02:43:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.251+0000 7f8bce7fc700 1 -- 172.21.15.7:0/596311365 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd4005c90 con 0x7f8be0156ec0 2024-01-27T02:43:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.251+0000 7f8bce7fc700 1 -- 172.21.15.7:0/596311365 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4007300 con 0x7f8be0156ec0 2024-01-27T02:43:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.251+0000 7f8be4ba4700 1 -- 172.21.15.7:0/596311365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be0168730 con 0x7f8be0156ec0 2024-01-27T02:43:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.251+0000 7f8be4ba4700 1 -- 172.21.15.7:0/596311365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be0168cf0 con 0x7f8be0156ec0 2024-01-27T02:43:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.252+0000 7f8be4ba4700 1 -- 172.21.15.7:0/596311365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8be004c710 con 0x7f8be0156ec0 2024-01-27T02:43:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.252+0000 7f8bce7fc700 1 -- 172.21.15.7:0/596311365 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8bd401b400 con 0x7f8be0156ec0 2024-01-27T02:43:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.252+0000 7f8bce7fc700 1 --2- 172.21.15.7:0/596311365 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8bc0066f30 0x7f8bc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.253+0000 7f8bce7fc700 1 -- 172.21.15.7:0/596311365 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8bd4083df0 con 0x7f8be0156ec0 2024-01-27T02:43:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.253+0000 7f8bde59c700 1 --2- 172.21.15.7:0/596311365 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8bc0066f30 0x7f8bc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.253+0000 7f8bde59c700 1 --2- 172.21.15.7:0/596311365 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8bc0066f30 0x7f8bc00693e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009ce0 tx=0x7f8bd0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:27.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.257+0000 7f8bce7fc700 1 -- 172.21.15.7:0/596311365 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8bd4051390 con 0x7f8be0156ec0 2024-01-27T02:43:27.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:27.459+0000 7f8be4ba4700 1 -- 172.21.15.7:0/596311365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8be01691a0 con 0x7f8be0156ec0 2024-01-27T02:43:28.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.162+0000 7f8bce7fc700 1 -- 172.21.15.7:0/596311365 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v177) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8bd40549b0 con 0x7f8be0156ec0 2024-01-27T02:43:28.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.167+0000 7f8be4ba4700 1 -- 172.21.15.7:0/596311365 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8bc0066f30 msgr2=0x7f8bc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:28.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.167+0000 7f8be4ba4700 1 --2- 172.21.15.7:0/596311365 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8bc0066f30 0x7f8bc00693e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009ce0 tx=0x7f8bd0008040 comp rx=0 tx=0).stop 2024-01-27T02:43:28.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.167+0000 7f8be4ba4700 1 -- 172.21.15.7:0/596311365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be0156ec0 msgr2=0x7f8be0164670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:28.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.167+0000 7f8be4ba4700 1 --2- 172.21.15.7:0/596311365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be0156ec0 0x7f8be0164670 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400c7e0 tx=0x7f8bd4007df0 comp rx=0 tx=0).stop 2024-01-27T02:43:28.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.167+0000 7f8be4ba4700 1 -- 172.21.15.7:0/596311365 shutdown_connections 2024-01-27T02:43:28.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.167+0000 7f8be4ba4700 1 --2- 172.21.15.7:0/596311365 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8bc0066f30 0x7f8bc00693e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:28.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.167+0000 7f8be4ba4700 1 --2- 172.21.15.7:0/596311365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8be015a960 0x7f8be01694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:28.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.167+0000 7f8be4ba4700 1 --2- 172.21.15.7:0/596311365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8be00b1b70 0x7f8be0167c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:28.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.167+0000 7f8be4ba4700 1 --2- 172.21.15.7:0/596311365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be0156ec0 0x7f8be0164670 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:28.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.168+0000 7f8be4ba4700 1 -- 172.21.15.7:0/596311365 >> 172.21.15.7:0/596311365 conn(0x7f8be0068480 msgr2=0x7f8be0158360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:28.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.168+0000 7f8be4ba4700 1 -- 172.21.15.7:0/596311365 shutdown_connections 2024-01-27T02:43:28.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.168+0000 7f8be4ba4700 1 -- 172.21.15.7:0/596311365 wait complete. 2024-01-27T02:43:28.170 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-27T02:43:28.185 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-27T02:43:28.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.328+0000 7f9c2617e700 1 Processor -- start 2024-01-27T02:43:28.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.328+0000 7f9c2617e700 1 -- start start 2024-01-27T02:43:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.328+0000 7f9c2617e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c20157c80 0x7f9c20158050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.328+0000 7f9c2617e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c20063580 0x7f9c20063a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.328+0000 7f9c2617e700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c200b3510 con 0x7f9c200ce510 2024-01-27T02:43:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.328+0000 7f9c2617e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c20063f40 con 0x7f9c20157c80 2024-01-27T02:43:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.328+0000 7f9c2617e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2006f8b0 con 0x7f9c20063580 2024-01-27T02:43:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.329+0000 7f9c1f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c20157c80 0x7f9c20158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.329+0000 7f9c1f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c20157c80 0x7f9c20158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42822/0 (socket says 172.21.15.7:42822) 2024-01-27T02:43:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.329+0000 7f9c1f7fe700 1 -- 172.21.15.7:0/1984176739 learned_addr learned my addr 172.21.15.7:0/1984176739 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.329+0000 7f9c1ffff700 1 --2- 172.21.15.7:0/1984176739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c20063580 0x7f9c20063a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.329+0000 7f9c1f7fe700 1 -- 172.21.15.7:0/1984176739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c20063580 msgr2=0x7f9c20063a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.329+0000 7f9c1f7fe700 1 --2- 172.21.15.7:0/1984176739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c20063580 0x7f9c20063a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.329+0000 7f9c1f7fe700 1 -- 172.21.15.7:0/1984176739 >> v1:172.21.15.188:6789/0 conn(0x7f9c200ce510 legacy=0x7f9c200b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.329+0000 7f9c1f7fe700 1 -- 172.21.15.7:0/1984176739 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c200b2900 con 0x7f9c20157c80 2024-01-27T02:43:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.329+0000 7f9c1f7fe700 1 --2- 172.21.15.7:0/1984176739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c20157c80 0x7f9c20158050 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f9c100099b0 tx=0x7f9c10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bee934a48572858 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.329+0000 7f9c1e7fc700 1 -- 172.21.15.7:0/1984176739 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10009ea0 con 0x7f9c20157c80 2024-01-27T02:43:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.330+0000 7f9c1e7fc700 1 -- 172.21.15.7:0/1984176739 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1000b860 con 0x7f9c20157c80 2024-01-27T02:43:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.330+0000 7f9c1e7fc700 1 -- 172.21.15.7:0/1984176739 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1000bc00 con 0x7f9c20157c80 2024-01-27T02:43:28.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.330+0000 7f9c2617e700 1 -- 172.21.15.7:0/1984176739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c20157c80 msgr2=0x7f9c20158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:28.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.330+0000 7f9c2617e700 1 --2- 172.21.15.7:0/1984176739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c20157c80 0x7f9c20158050 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f9c100099b0 tx=0x7f9c10009250 comp rx=0 tx=0).stop 2024-01-27T02:43:28.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.330+0000 7f9c2617e700 1 -- 172.21.15.7:0/1984176739 shutdown_connections 2024-01-27T02:43:28.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.330+0000 7f9c2617e700 1 --2- 172.21.15.7:0/1984176739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c20063580 0x7f9c20063a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:28.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.330+0000 7f9c2617e700 1 --2- 172.21.15.7:0/1984176739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c20157c80 0x7f9c20158050 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:28.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.330+0000 7f9c2617e700 1 -- 172.21.15.7:0/1984176739 >> 172.21.15.7:0/1984176739 conn(0x7f9c20068480 msgr2=0x7f9c200cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:28.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.330+0000 7f9c2617e700 1 -- 172.21.15.7:0/1984176739 shutdown_connections 2024-01-27T02:43:28.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.330+0000 7f9c2617e700 1 -- 172.21.15.7:0/1984176739 wait complete. 2024-01-27T02:43:28.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.332+0000 7f9c2617e700 1 Processor -- start 2024-01-27T02:43:28.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.332+0000 7f9c2617e700 1 -- start start 2024-01-27T02:43:28.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.332+0000 7f9c2617e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c20063580 0x7f9c200a8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.332+0000 7f9c2617e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c200ce510 0x7f9c2014ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.332+0000 7f9c2617e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c20157c80 0x7f9c2014ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.332+0000 7f9c2617e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c200a9280 con 0x7f9c20063580 2024-01-27T02:43:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.332+0000 7f9c2617e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c201595b0 con 0x7f9c200ce510 2024-01-27T02:43:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.332+0000 7f9c2617e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c20159890 con 0x7f9c20157c80 2024-01-27T02:43:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.333+0000 7f9c1f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c20063580 0x7f9c200a8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.333+0000 7f9c1f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c20063580 0x7f9c200a8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42824/0 (socket says 172.21.15.7:42824) 2024-01-27T02:43:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.333+0000 7f9c1f7fe700 1 -- 172.21.15.7:0/4025990888 learned_addr learned my addr 172.21.15.7:0/4025990888 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.333+0000 7f9c1ffff700 1 --2- 172.21.15.7:0/4025990888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c20157c80 0x7f9c2014ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.333+0000 7f9c1effd700 1 --2- 172.21.15.7:0/4025990888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c200ce510 0x7f9c2014ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.333+0000 7f9c1f7fe700 1 -- 172.21.15.7:0/4025990888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c20157c80 msgr2=0x7f9c2014ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.333+0000 7f9c1f7fe700 1 --2- 172.21.15.7:0/4025990888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c20157c80 0x7f9c2014ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.333+0000 7f9c1f7fe700 1 -- 172.21.15.7:0/4025990888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c200ce510 msgr2=0x7f9c2014ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.333+0000 7f9c1f7fe700 1 --2- 172.21.15.7:0/4025990888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c200ce510 0x7f9c2014ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.333+0000 7f9c1f7fe700 1 -- 172.21.15.7:0/4025990888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c100094f0 con 0x7f9c20063580 2024-01-27T02:43:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.333+0000 7f9c1ffff700 1 --2- 172.21.15.7:0/4025990888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c20157c80 0x7f9c2014ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.333+0000 7f9c1f7fe700 1 --2- 172.21.15.7:0/4025990888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c20063580 0x7f9c200a8c30 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f9c200700c0 tx=0x7f9c10017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.333+0000 7f9c077fe700 1 -- 172.21.15.7:0/4025990888 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1000bbe0 con 0x7f9c20063580 2024-01-27T02:43:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.333+0000 7f9c077fe700 1 -- 172.21.15.7:0/4025990888 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c10011420 con 0x7f9c20063580 2024-01-27T02:43:28.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.333+0000 7f9c2617e700 1 -- 172.21.15.7:0/4025990888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c20159b70 con 0x7f9c20063580 2024-01-27T02:43:28.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.333+0000 7f9c077fe700 1 -- 172.21.15.7:0/4025990888 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c100117c0 con 0x7f9c20063580 2024-01-27T02:43:28.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.333+0000 7f9c2617e700 1 -- 172.21.15.7:0/4025990888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c2014f480 con 0x7f9c20063580 2024-01-27T02:43:28.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.334+0000 7f9c2617e700 1 -- 172.21.15.7:0/4025990888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c20159db0 con 0x7f9c20063580 2024-01-27T02:43:28.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.335+0000 7f9c077fe700 1 -- 172.21.15.7:0/4025990888 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9c10011bf0 con 0x7f9c20063580 2024-01-27T02:43:28.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.335+0000 7f9c077fe700 1 --2- 172.21.15.7:0/4025990888 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9c0c066f30 0x7f9c0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:28.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.335+0000 7f9c077fe700 1 -- 172.21.15.7:0/4025990888 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f9c10085c80 con 0x7f9c20063580 2024-01-27T02:43:28.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.335+0000 7f9c1effd700 1 --2- 172.21.15.7:0/4025990888 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9c0c066f30 0x7f9c0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:28.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.336+0000 7f9c1effd700 1 --2- 172.21.15.7:0/4025990888 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9c0c066f30 0x7f9c0c0693e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9c200acc40 tx=0x7f9c14009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:28.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.340+0000 7f9c077fe700 1 -- 172.21.15.7:0/4025990888 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c100530d0 con 0x7f9c20063580 2024-01-27T02:43:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:28.546+0000 7f9c2617e700 1 -- 172.21.15.7:0/4025990888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9c20072060 con 0x7f9c20063580 2024-01-27T02:43:29.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.170+0000 7f9c077fe700 1 -- 172.21.15.7:0/4025990888 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v178) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9c100566f0 con 0x7f9c20063580 2024-01-27T02:43:29.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.176+0000 7f9c2617e700 1 -- 172.21.15.7:0/4025990888 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9c0c066f30 msgr2=0x7f9c0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:29.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.176+0000 7f9c2617e700 1 --2- 172.21.15.7:0/4025990888 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9c0c066f30 0x7f9c0c0693e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9c200acc40 tx=0x7f9c14009040 comp rx=0 tx=0).stop 2024-01-27T02:43:29.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.176+0000 7f9c2617e700 1 -- 172.21.15.7:0/4025990888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c20063580 msgr2=0x7f9c200a8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:29.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.176+0000 7f9c2617e700 1 --2- 172.21.15.7:0/4025990888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c20063580 0x7f9c200a8c30 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f9c200700c0 tx=0x7f9c10017000 comp rx=0 tx=0).stop 2024-01-27T02:43:29.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.176+0000 7f9c2617e700 1 -- 172.21.15.7:0/4025990888 shutdown_connections 2024-01-27T02:43:29.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.176+0000 7f9c2617e700 1 --2- 172.21.15.7:0/4025990888 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9c0c066f30 0x7f9c0c0693e0 secure :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9c200acc40 tx=0x7f9c14009040 comp rx=0 tx=0).stop 2024-01-27T02:43:29.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.176+0000 7f9c2617e700 1 --2- 172.21.15.7:0/4025990888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c20157c80 0x7f9c2014ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:29.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.176+0000 7f9c2617e700 1 --2- 172.21.15.7:0/4025990888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c20063580 0x7f9c200a8c30 secure :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f9c200700c0 tx=0x7f9c10017000 comp rx=0 tx=0).stop 2024-01-27T02:43:29.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.176+0000 7f9c2617e700 1 --2- 172.21.15.7:0/4025990888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c200ce510 0x7f9c2014ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:29.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.176+0000 7f9c2617e700 1 -- 172.21.15.7:0/4025990888 >> 172.21.15.7:0/4025990888 conn(0x7f9c20068480 msgr2=0x7f9c200af960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.178+0000 7f9c2617e700 1 -- 172.21.15.7:0/4025990888 shutdown_connections 2024-01-27T02:43:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.178+0000 7f9c2617e700 1 -- 172.21.15.7:0/4025990888 wait complete. 2024-01-27T02:43:29.182 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-27T02:43:29.197 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-27T02:43:29.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.333+0000 7f9830663700 1 Processor -- start 2024-01-27T02:43:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.334+0000 7f9830663700 1 -- start start 2024-01-27T02:43:29.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.335+0000 7f9830663700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f98280cba60 0x7f98280ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:29.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.335+0000 7f9830663700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f982806b4e0 0x7f982806b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:29.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.335+0000 7f9830663700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98280679b0 con 0x7f98280cbe30 2024-01-27T02:43:29.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.335+0000 7f9830663700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9828067c90 con 0x7f982806b4e0 2024-01-27T02:43:29.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.335+0000 7f9830663700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f982806be20 con 0x7f98280cba60 2024-01-27T02:43:29.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.335+0000 7f982ec00700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f982806b4e0 0x7f982806b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:29.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.335+0000 7f982dbfe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f98280cbe30 0x7f982815b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40344/0 (socket says 172.21.15.7:40344) 2024-01-27T02:43:29.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.335+0000 7f982dbfe700 1 -- 172.21.15.7:0/869356753 learned_addr learned my addr 172.21.15.7:0/869356753 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:29.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.335+0000 7f982e3ff700 1 --2- 172.21.15.7:0/869356753 >> v2:172.21.15.188:3300/0 conn(0x7f98280cba60 0x7f98280ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:29.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.335+0000 7f982ec00700 1 -- 172.21.15.7:0/869356753 >> v2:172.21.15.188:3300/0 conn(0x7f98280cba60 msgr2=0x7f98280ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:29.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.335+0000 7f982ec00700 1 --2- 172.21.15.7:0/869356753 >> v2:172.21.15.188:3300/0 conn(0x7f98280cba60 0x7f98280ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:29.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.335+0000 7f982ec00700 1 -- 172.21.15.7:0/869356753 >> v1:172.21.15.7:6789/0 conn(0x7f98280cbe30 legacy=0x7f982815b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:29.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.335+0000 7f982ec00700 1 -- 172.21.15.7:0/869356753 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98280cca50 con 0x7f982806b4e0 2024-01-27T02:43:29.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.335+0000 7f982d3fd700 1 -- 172.21.15.7:0/869356753 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 752562229 0 0) 0x7f98280679b0 con 0x7f98280cbe30 2024-01-27T02:43:29.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.335+0000 7f982e3ff700 1 --2- 172.21.15.7:0/869356753 >> v2:172.21.15.188:3300/0 conn(0x7f98280cba60 0x7f98280ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:29.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.336+0000 7f982ec00700 1 --2- 172.21.15.7:0/869356753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f982806b4e0 0x7f982806b8e0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f98280cfd50 tx=0x7f9820009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5336bcdc9078c931 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:29.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.336+0000 7f982d3fd700 1 -- 172.21.15.7:0/869356753 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982000e070 con 0x7f982806b4e0 2024-01-27T02:43:29.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.336+0000 7f982d3fd700 1 -- 172.21.15.7:0/869356753 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982000bc30 con 0x7f982806b4e0 2024-01-27T02:43:29.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.336+0000 7f982d3fd700 1 -- 172.21.15.7:0/869356753 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9820012430 con 0x7f982806b4e0 2024-01-27T02:43:29.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.337+0000 7f9830663700 1 -- 172.21.15.7:0/869356753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f982806b4e0 msgr2=0x7f982806b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.337+0000 7f9830663700 1 --2- 172.21.15.7:0/869356753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f982806b4e0 0x7f982806b8e0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f98280cfd50 tx=0x7f9820009780 comp rx=0 tx=0).stop 2024-01-27T02:43:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.337+0000 7f9830663700 1 -- 172.21.15.7:0/869356753 shutdown_connections 2024-01-27T02:43:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.337+0000 7f9830663700 1 --2- 172.21.15.7:0/869356753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f982806b4e0 0x7f982806b8e0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.337+0000 7f9830663700 1 --2- 172.21.15.7:0/869356753 >> v2:172.21.15.188:3300/0 conn(0x7f98280cba60 0x7f98280ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.337+0000 7f9830663700 1 -- 172.21.15.7:0/869356753 >> 172.21.15.7:0/869356753 conn(0x7f9828063580 msgr2=0x7f9828063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.337+0000 7f9830663700 1 -- 172.21.15.7:0/869356753 shutdown_connections 2024-01-27T02:43:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.337+0000 7f9830663700 1 -- 172.21.15.7:0/869356753 wait complete. 2024-01-27T02:43:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.338+0000 7f9830663700 1 Processor -- start 2024-01-27T02:43:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.338+0000 7f9830663700 1 -- start start 2024-01-27T02:43:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.339+0000 7f9830663700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f982806b4e0 0x7f9828156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.339+0000 7f9830663700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98280cba60 0x7f98280ad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.339+0000 7f9830663700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98280cbe30 0x7f982816b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:29.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.339+0000 7f9830663700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98280b1f00 con 0x7f982806b4e0 2024-01-27T02:43:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.339+0000 7f9830663700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98280ad6f0 con 0x7f98280cba60 2024-01-27T02:43:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.339+0000 7f9830663700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98280b14e0 con 0x7f98280cbe30 2024-01-27T02:43:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.339+0000 7f982e3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f982806b4e0 0x7f9828156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.339+0000 7f982e3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f982806b4e0 0x7f9828156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42840/0 (socket says 172.21.15.7:42840) 2024-01-27T02:43:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.339+0000 7f982e3ff700 1 -- 172.21.15.7:0/1353180421 learned_addr learned my addr 172.21.15.7:0/1353180421 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.339+0000 7f982e3ff700 1 -- 172.21.15.7:0/1353180421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98280cbe30 msgr2=0x7f982816b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.339+0000 7f982e3ff700 1 --2- 172.21.15.7:0/1353180421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98280cbe30 0x7f982816b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.339+0000 7f982e3ff700 1 -- 172.21.15.7:0/1353180421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98280cba60 msgr2=0x7f98280ad1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.339+0000 7f982e3ff700 1 --2- 172.21.15.7:0/1353180421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98280cba60 0x7f98280ad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.339+0000 7f982e3ff700 1 -- 172.21.15.7:0/1353180421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9820009a20 con 0x7f982806b4e0 2024-01-27T02:43:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.339+0000 7f982e3ff700 1 --2- 172.21.15.7:0/1353180421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f982806b4e0 0x7f9828156770 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f982400e920 tx=0x7f9824008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.340+0000 7f981a7fc700 1 -- 172.21.15.7:0/1353180421 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9824005a00 con 0x7f982806b4e0 2024-01-27T02:43:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.340+0000 7f981a7fc700 1 -- 172.21.15.7:0/1353180421 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9824005b60 con 0x7f982806b4e0 2024-01-27T02:43:29.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.340+0000 7f9830663700 1 -- 172.21.15.7:0/1353180421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98280b1c20 con 0x7f982806b4e0 2024-01-27T02:43:29.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.340+0000 7f981a7fc700 1 -- 172.21.15.7:0/1353180421 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9824006020 con 0x7f982806b4e0 2024-01-27T02:43:29.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.340+0000 7f9830663700 1 -- 172.21.15.7:0/1353180421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98280b2910 con 0x7f982806b4e0 2024-01-27T02:43:29.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.341+0000 7f9830663700 1 -- 172.21.15.7:0/1353180421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f982816a640 con 0x7f982806b4e0 2024-01-27T02:43:29.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.341+0000 7f981a7fc700 1 -- 172.21.15.7:0/1353180421 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9824006410 con 0x7f982806b4e0 2024-01-27T02:43:29.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.342+0000 7f981a7fc700 1 --2- 172.21.15.7:0/1353180421 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f980c066f30 0x7f980c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:29.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.342+0000 7f981a7fc700 1 -- 172.21.15.7:0/1353180421 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f9824084710 con 0x7f982806b4e0 2024-01-27T02:43:29.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.342+0000 7f982dbfe700 1 --2- 172.21.15.7:0/1353180421 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f980c066f30 0x7f980c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:29.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.342+0000 7f982dbfe700 1 --2- 172.21.15.7:0/1353180421 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f980c066f30 0x7f980c0693e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9828158340 tx=0x7f981c007bd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.346+0000 7f981a7fc700 1 -- 172.21.15.7:0/1353180421 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98240519e0 con 0x7f982806b4e0 2024-01-27T02:43:29.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:29.598+0000 7f9830663700 1 -- 172.21.15.7:0/1353180421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f98280b2490 con 0x7f982806b4e0 2024-01-27T02:43:30.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.184+0000 7f981a7fc700 1 -- 172.21.15.7:0/1353180421 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v90) v1 ==== 174+0+0 (secure 0 0 0) 0x7f98280b2490 con 0x7f982806b4e0 2024-01-27T02:43:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.187+0000 7f9830663700 1 -- 172.21.15.7:0/1353180421 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f980c066f30 msgr2=0x7f980c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.187+0000 7f9830663700 1 --2- 172.21.15.7:0/1353180421 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f980c066f30 0x7f980c0693e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9828158340 tx=0x7f981c007bd0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.187+0000 7f9830663700 1 -- 172.21.15.7:0/1353180421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f982806b4e0 msgr2=0x7f9828156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.187+0000 7f9830663700 1 --2- 172.21.15.7:0/1353180421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f982806b4e0 0x7f9828156770 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f982400e920 tx=0x7f9824008040 comp rx=0 tx=0).stop 2024-01-27T02:43:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.187+0000 7f9830663700 1 -- 172.21.15.7:0/1353180421 shutdown_connections 2024-01-27T02:43:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.187+0000 7f9830663700 1 --2- 172.21.15.7:0/1353180421 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f980c066f30 0x7f980c0693e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.187+0000 7f9830663700 1 --2- 172.21.15.7:0/1353180421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98280cbe30 0x7f982816b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.187+0000 7f9830663700 1 --2- 172.21.15.7:0/1353180421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f982806b4e0 0x7f9828156770 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.187+0000 7f9830663700 1 --2- 172.21.15.7:0/1353180421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98280cba60 0x7f98280ad1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.187+0000 7f9830663700 1 -- 172.21.15.7:0/1353180421 >> 172.21.15.7:0/1353180421 conn(0x7f9828063580 msgr2=0x7f98281584a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.187+0000 7f9830663700 1 -- 172.21.15.7:0/1353180421 shutdown_connections 2024-01-27T02:43:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.187+0000 7f9830663700 1 -- 172.21.15.7:0/1353180421 wait complete. 2024-01-27T02:43:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 22 and data pool 23 2024-01-27T02:43:30.204 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:43:30.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.340+0000 7fd11a717700 1 Processor -- start 2024-01-27T02:43:30.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.340+0000 7fd11a717700 1 -- start start 2024-01-27T02:43:30.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.341+0000 7fd11a717700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd1140b2f80 0x7fd11406a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:30.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.341+0000 7fd11a717700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd1140b3350 0x7fd1140ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:30.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.341+0000 7fd11a717700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd11406a900 con 0x7fd1140ce810 2024-01-27T02:43:30.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.341+0000 7fd11a717700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd114159390 con 0x7fd1140b2f80 2024-01-27T02:43:30.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.341+0000 7fd11a717700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd114158780 con 0x7fd1140b3350 2024-01-27T02:43:30.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.341+0000 7fd113fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd1140b2f80 0x7fd11406a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:30.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.341+0000 7fd118cb4700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd1140ce810 0x7fd1141553e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40350/0 (socket says 172.21.15.7:40350) 2024-01-27T02:43:30.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.341+0000 7fd118cb4700 1 -- 172.21.15.7:0/2884904000 learned_addr learned my addr 172.21.15.7:0/2884904000 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:30.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.341+0000 7fd113fff700 1 --2- 172.21.15.7:0/2884904000 >> v2:172.21.15.7:3300/0 conn(0x7fd1140b2f80 0x7fd11406a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42846/0 (socket says 172.21.15.7:42846) 2024-01-27T02:43:30.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.341+0000 7fd1137fe700 1 --2- 172.21.15.7:0/2884904000 >> v2:172.21.15.188:3300/0 conn(0x7fd1140b3350 0x7fd1140ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:30.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.342+0000 7fd113fff700 1 -- 172.21.15.7:0/2884904000 >> v2:172.21.15.188:3300/0 conn(0x7fd1140b3350 msgr2=0x7fd1140ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:30.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.342+0000 7fd113fff700 1 --2- 172.21.15.7:0/2884904000 >> v2:172.21.15.188:3300/0 conn(0x7fd1140b3350 0x7fd1140ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.342+0000 7fd113fff700 1 -- 172.21.15.7:0/2884904000 >> v1:172.21.15.7:6789/0 conn(0x7fd1140ce810 legacy=0x7fd1141553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:30.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.342+0000 7fd113fff700 1 -- 172.21.15.7:0/2884904000 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd114158a60 con 0x7fd1140b2f80 2024-01-27T02:43:30.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.342+0000 7fd112ffd700 1 -- 172.21.15.7:0/2884904000 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 942450929 0 0) 0x7fd11406a900 con 0x7fd1140ce810 2024-01-27T02:43:30.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.342+0000 7fd113fff700 1 --2- 172.21.15.7:0/2884904000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1140b2f80 0x7fd11406a3c0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fd1140c9ed0 tx=0x7fd1040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582201b8e0b78b7a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:30.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.342+0000 7fd112ffd700 1 -- 172.21.15.7:0/2884904000 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10400b6b0 con 0x7fd1140b2f80 2024-01-27T02:43:30.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.342+0000 7fd112ffd700 1 -- 172.21.15.7:0/2884904000 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd10400b810 con 0x7fd1140b2f80 2024-01-27T02:43:30.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.342+0000 7fd112ffd700 1 -- 172.21.15.7:0/2884904000 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10400bbb0 con 0x7fd1140b2f80 2024-01-27T02:43:30.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.342+0000 7fd11a717700 1 -- 172.21.15.7:0/2884904000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1140b2f80 msgr2=0x7fd11406a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:30.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.343+0000 7fd11a717700 1 --2- 172.21.15.7:0/2884904000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1140b2f80 0x7fd11406a3c0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fd1140c9ed0 tx=0x7fd1040091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.343+0000 7fd11a717700 1 -- 172.21.15.7:0/2884904000 shutdown_connections 2024-01-27T02:43:30.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.343+0000 7fd11a717700 1 --2- 172.21.15.7:0/2884904000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1140b2f80 0x7fd11406a3c0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.343+0000 7fd11a717700 1 --2- 172.21.15.7:0/2884904000 >> v2:172.21.15.188:3300/0 conn(0x7fd1140b3350 0x7fd1140ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.343+0000 7fd11a717700 1 -- 172.21.15.7:0/2884904000 >> 172.21.15.7:0/2884904000 conn(0x7fd114068430 msgr2=0x7fd11406be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:30.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.343+0000 7fd11a717700 1 -- 172.21.15.7:0/2884904000 shutdown_connections 2024-01-27T02:43:30.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.343+0000 7fd11a717700 1 -- 172.21.15.7:0/2884904000 wait complete. 2024-01-27T02:43:30.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.344+0000 7fd11a717700 1 Processor -- start 2024-01-27T02:43:30.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.344+0000 7fd11a717700 1 -- start start 2024-01-27T02:43:30.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.345+0000 7fd11a717700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1140b2f80 0x7fd114141570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:30.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.345+0000 7fd11a717700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1140b3350 0x7fd11415d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:30.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.345+0000 7fd11a717700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1140ce810 0x7fd11415d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:30.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.345+0000 7fd11a717700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd114168c30 con 0x7fd1140ce810 2024-01-27T02:43:30.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.345+0000 7fd11a717700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1141691c0 con 0x7fd1140b2f80 2024-01-27T02:43:30.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.345+0000 7fd11a717700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1141694a0 con 0x7fd1140b3350 2024-01-27T02:43:30.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.345+0000 7fd118cb4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1140ce810 0x7fd11415d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:30.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.345+0000 7fd118cb4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1140ce810 0x7fd11415d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42852/0 (socket says 172.21.15.7:42852) 2024-01-27T02:43:30.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.345+0000 7fd118cb4700 1 -- 172.21.15.7:0/3958382806 learned_addr learned my addr 172.21.15.7:0/3958382806 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:30.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.345+0000 7fd113fff700 1 --2- 172.21.15.7:0/3958382806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1140b2f80 0x7fd114141570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:30.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.345+0000 7fd1137fe700 1 --2- 172.21.15.7:0/3958382806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1140b3350 0x7fd11415d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:30.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.345+0000 7fd118cb4700 1 -- 172.21.15.7:0/3958382806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1140b3350 msgr2=0x7fd11415d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:30.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.345+0000 7fd118cb4700 1 --2- 172.21.15.7:0/3958382806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1140b3350 0x7fd11415d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.345+0000 7fd118cb4700 1 -- 172.21.15.7:0/3958382806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1140b2f80 msgr2=0x7fd114141570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:30.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.345+0000 7fd118cb4700 1 --2- 172.21.15.7:0/3958382806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1140b2f80 0x7fd114141570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.345+0000 7fd118cb4700 1 -- 172.21.15.7:0/3958382806 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd104009420 con 0x7fd1140ce810 2024-01-27T02:43:30.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.345+0000 7fd1137fe700 1 --2- 172.21.15.7:0/3958382806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1140b3350 0x7fd11415d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:43:30.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.345+0000 7fd118cb4700 1 --2- 172.21.15.7:0/3958382806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1140ce810 0x7fd11415d5f0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00cbf0 tx=0x7fd10c0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:30.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.346+0000 7fd0fbfff700 1 -- 172.21.15.7:0/3958382806 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10c00a9c0 con 0x7fd1140ce810 2024-01-27T02:43:30.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.346+0000 7fd0fbfff700 1 -- 172.21.15.7:0/3958382806 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd10c00ab20 con 0x7fd1140ce810 2024-01-27T02:43:30.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.346+0000 7fd11a717700 1 -- 172.21.15.7:0/3958382806 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd114169780 con 0x7fd1140ce810 2024-01-27T02:43:30.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.346+0000 7fd0fbfff700 1 -- 172.21.15.7:0/3958382806 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10c0070f0 con 0x7fd1140ce810 2024-01-27T02:43:30.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.346+0000 7fd11a717700 1 -- 172.21.15.7:0/3958382806 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd114167eb0 con 0x7fd1140ce810 2024-01-27T02:43:30.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.347+0000 7fd11a717700 1 -- 172.21.15.7:0/3958382806 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd11404c710 con 0x7fd1140ce810 2024-01-27T02:43:30.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.347+0000 7fd0fbfff700 1 -- 172.21.15.7:0/3958382806 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd10c007520 con 0x7fd1140ce810 2024-01-27T02:43:30.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.348+0000 7fd0fbfff700 1 --2- 172.21.15.7:0/3958382806 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd100066f30 0x7fd1000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:30.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.348+0000 7fd0fbfff700 1 -- 172.21.15.7:0/3958382806 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd10c0849b0 con 0x7fd1140ce810 2024-01-27T02:43:30.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.348+0000 7fd113fff700 1 --2- 172.21.15.7:0/3958382806 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd100066f30 0x7fd1000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:30.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.349+0000 7fd113fff700 1 --2- 172.21.15.7:0/3958382806 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd100066f30 0x7fd1000693e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd1040093f0 tx=0x7fd104017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:30.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.353+0000 7fd0fbfff700 1 -- 172.21.15.7:0/3958382806 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd10c051c00 con 0x7fd1140ce810 2024-01-27T02:43:30.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.558+0000 7fd11a717700 1 -- 172.21.15.7:0/3958382806 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd114169970 con 0x7fd1140ce810 2024-01-27T02:43:30.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.559+0000 7fd0fbfff700 1 -- 172.21.15.7:0/3958382806 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v179) v1 ==== 74+0+15330 (secure 0 0 0) 0x7fd10c055220 con 0x7fd1140ce810 2024-01-27T02:43:30.559 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:30.560 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":179,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:43:29.599410+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:43:27.492968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"179","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-01-27T02:43:28.579977+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"179","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:43:30.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.561+0000 7fd11a717700 1 -- 172.21.15.7:0/3958382806 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd100066f30 msgr2=0x7fd1000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:30.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.561+0000 7fd11a717700 1 --2- 172.21.15.7:0/3958382806 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd100066f30 0x7fd1000693e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd1040093f0 tx=0x7fd104017000 comp rx=0 tx=0).stop 2024-01-27T02:43:30.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.561+0000 7fd11a717700 1 -- 172.21.15.7:0/3958382806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1140ce810 msgr2=0x7fd11415d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:30.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.561+0000 7fd11a717700 1 --2- 172.21.15.7:0/3958382806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1140ce810 0x7fd11415d5f0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00cbf0 tx=0x7fd10c0019b0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.561+0000 7fd11a717700 1 -- 172.21.15.7:0/3958382806 shutdown_connections 2024-01-27T02:43:30.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.561+0000 7fd11a717700 1 --2- 172.21.15.7:0/3958382806 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd100066f30 0x7fd1000693e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.561+0000 7fd11a717700 1 --2- 172.21.15.7:0/3958382806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1140ce810 0x7fd11415d5f0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.561+0000 7fd11a717700 1 --2- 172.21.15.7:0/3958382806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1140b2f80 0x7fd114141570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.561+0000 7fd11a717700 1 --2- 172.21.15.7:0/3958382806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1140b3350 0x7fd11415d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.561+0000 7fd11a717700 1 -- 172.21.15.7:0/3958382806 >> 172.21.15.7:0/3958382806 conn(0x7fd114068430 msgr2=0x7fd114153c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.561+0000 7fd11a717700 1 -- 172.21.15.7:0/3958382806 shutdown_connections 2024-01-27T02:43:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.561+0000 7fd11a717700 1 -- 172.21.15.7:0/3958382806 wait complete. 2024-01-27T02:43:30.576 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:43:30.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.707+0000 7f1c365b5700 1 Processor -- start 2024-01-27T02:43:30.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.707+0000 7f1c365b5700 1 -- start start 2024-01-27T02:43:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.709+0000 7f1c365b5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1c300c3ae0 0x7f1c300cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.709+0000 7f1c365b5700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1c30158070 0x7f1c300cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.709+0000 7f1c365b5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c300ca020 con 0x7f1c30157ca0 2024-01-27T02:43:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.709+0000 7f1c365b5700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c300c5210 con 0x7f1c300c3ae0 2024-01-27T02:43:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.709+0000 7f1c365b5700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c300c54f0 con 0x7f1c30158070 2024-01-27T02:43:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.709+0000 7f1c27fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1c300c3ae0 0x7f1c300cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.709+0000 7f1c2ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1c30157ca0 0x7f1c300c4a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40352/0 (socket says 172.21.15.7:40352) 2024-01-27T02:43:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.709+0000 7f1c2ffff700 1 -- 172.21.15.7:0/2895250756 learned_addr learned my addr 172.21.15.7:0/2895250756 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.709+0000 7f1c27fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1c300c3ae0 0x7f1c300cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42860/0 (socket says 172.21.15.7:42860) 2024-01-27T02:43:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.709+0000 7f1c34b52700 1 --2- 172.21.15.7:0/2895250756 >> v2:172.21.15.188:3300/0 conn(0x7f1c30158070 0x7f1c300cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.709+0000 7f1c27fff700 1 -- 172.21.15.7:0/2895250756 >> v2:172.21.15.188:3300/0 conn(0x7f1c30158070 msgr2=0x7f1c300cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.709+0000 7f1c27fff700 1 --2- 172.21.15.7:0/2895250756 >> v2:172.21.15.188:3300/0 conn(0x7f1c30158070 0x7f1c300cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.709+0000 7f1c27fff700 1 -- 172.21.15.7:0/2895250756 >> v1:172.21.15.7:6789/0 conn(0x7f1c30157ca0 legacy=0x7f1c300c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.709+0000 7f1c27fff700 1 -- 172.21.15.7:0/2895250756 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c300c57d0 con 0x7f1c300c3ae0 2024-01-27T02:43:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.709+0000 7f1c34b52700 1 --2- 172.21.15.7:0/2895250756 >> v2:172.21.15.188:3300/0 conn(0x7f1c30158070 0x7f1c300cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.709+0000 7f1c2f7fe700 1 -- 172.21.15.7:0/2895250756 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 571421257 0 0) 0x7f1c300ca020 con 0x7f1c30157ca0 2024-01-27T02:43:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.709+0000 7f1c27fff700 1 --2- 172.21.15.7:0/2895250756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c300c3ae0 0x7f1c300cd4e0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f1c3006f890 tx=0x7f1c180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f113dc18d96e412 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.709+0000 7f1c2f7fe700 1 -- 172.21.15.7:0/2895250756 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1800b6b0 con 0x7f1c300c3ae0 2024-01-27T02:43:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.709+0000 7f1c2f7fe700 1 -- 172.21.15.7:0/2895250756 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1800b810 con 0x7f1c300c3ae0 2024-01-27T02:43:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.710+0000 7f1c2f7fe700 1 -- 172.21.15.7:0/2895250756 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1800bbb0 con 0x7f1c300c3ae0 2024-01-27T02:43:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.710+0000 7f1c365b5700 1 -- 172.21.15.7:0/2895250756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c300c3ae0 msgr2=0x7f1c300cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.710+0000 7f1c365b5700 1 --2- 172.21.15.7:0/2895250756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c300c3ae0 0x7f1c300cd4e0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f1c3006f890 tx=0x7f1c180091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.710+0000 7f1c365b5700 1 -- 172.21.15.7:0/2895250756 shutdown_connections 2024-01-27T02:43:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.710+0000 7f1c365b5700 1 --2- 172.21.15.7:0/2895250756 >> v2:172.21.15.188:3300/0 conn(0x7f1c30158070 0x7f1c300cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.710+0000 7f1c365b5700 1 --2- 172.21.15.7:0/2895250756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c300c3ae0 0x7f1c300cd4e0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.710+0000 7f1c365b5700 1 -- 172.21.15.7:0/2895250756 >> 172.21.15.7:0/2895250756 conn(0x7f1c30098020 msgr2=0x7f1c300711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.710+0000 7f1c365b5700 1 -- 172.21.15.7:0/2895250756 shutdown_connections 2024-01-27T02:43:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.710+0000 7f1c365b5700 1 -- 172.21.15.7:0/2895250756 wait complete. 2024-01-27T02:43:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.711+0000 7f1c365b5700 1 Processor -- start 2024-01-27T02:43:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.711+0000 7f1c365b5700 1 -- start start 2024-01-27T02:43:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.712+0000 7f1c365b5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c300c3ae0 0x7f1c300679b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.712+0000 7f1c365b5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c30157ca0 0x7f1c301523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.712+0000 7f1c365b5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c30158070 0x7f1c300c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.712+0000 7f1c365b5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c30155860 con 0x7f1c30157ca0 2024-01-27T02:43:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.712+0000 7f1c365b5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c301555a0 con 0x7f1c30158070 2024-01-27T02:43:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.712+0000 7f1c365b5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c30068b50 con 0x7f1c300c3ae0 2024-01-27T02:43:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.712+0000 7f1c27fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c30157ca0 0x7f1c301523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.712+0000 7f1c27fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c30157ca0 0x7f1c301523d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42868/0 (socket says 172.21.15.7:42868) 2024-01-27T02:43:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.712+0000 7f1c27fff700 1 -- 172.21.15.7:0/2696229436 learned_addr learned my addr 172.21.15.7:0/2696229436 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.713+0000 7f1c34b52700 1 --2- 172.21.15.7:0/2696229436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c30158070 0x7f1c300c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.713+0000 7f1c2ffff700 1 --2- 172.21.15.7:0/2696229436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c300c3ae0 0x7f1c300679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.713+0000 7f1c27fff700 1 -- 172.21.15.7:0/2696229436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c300c3ae0 msgr2=0x7f1c300679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.713+0000 7f1c27fff700 1 --2- 172.21.15.7:0/2696229436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c300c3ae0 0x7f1c300679b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.713+0000 7f1c27fff700 1 -- 172.21.15.7:0/2696229436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c30158070 msgr2=0x7f1c300c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.713+0000 7f1c27fff700 1 --2- 172.21.15.7:0/2696229436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c30158070 0x7f1c300c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.713+0000 7f1c27fff700 1 -- 172.21.15.7:0/2696229436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c18009420 con 0x7f1c30157ca0 2024-01-27T02:43:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.713+0000 7f1c34b52700 1 --2- 172.21.15.7:0/2696229436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c30158070 0x7f1c300c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:43:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.713+0000 7f1c2ffff700 1 --2- 172.21.15.7:0/2696229436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c300c3ae0 0x7f1c300679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:43:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.713+0000 7f1c27fff700 1 --2- 172.21.15.7:0/2696229436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c30157ca0 0x7f1c301523d0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f1c3006ea80 tx=0x7f1c18017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.713+0000 7f1c2cff9700 1 -- 172.21.15.7:0/2696229436 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1800bb90 con 0x7f1c30157ca0 2024-01-27T02:43:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.713+0000 7f1c2cff9700 1 -- 172.21.15.7:0/2696229436 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c18011420 con 0x7f1c30157ca0 2024-01-27T02:43:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.713+0000 7f1c365b5700 1 -- 172.21.15.7:0/2696229436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c300682b0 con 0x7f1c30157ca0 2024-01-27T02:43:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.713+0000 7f1c2cff9700 1 -- 172.21.15.7:0/2696229436 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c180117a0 con 0x7f1c30157ca0 2024-01-27T02:43:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.713+0000 7f1c365b5700 1 -- 172.21.15.7:0/2696229436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c30068870 con 0x7f1c30157ca0 2024-01-27T02:43:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.714+0000 7f1c365b5700 1 -- 172.21.15.7:0/2696229436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c301559d0 con 0x7f1c30157ca0 2024-01-27T02:43:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.715+0000 7f1c2cff9700 1 -- 172.21.15.7:0/2696229436 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1c18011b90 con 0x7f1c30157ca0 2024-01-27T02:43:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.715+0000 7f1c2cff9700 1 --2- 172.21.15.7:0/2696229436 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1c10066f30 0x7f1c100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.715+0000 7f1c2cff9700 1 -- 172.21.15.7:0/2696229436 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1c18085f00 con 0x7f1c30157ca0 2024-01-27T02:43:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.715+0000 7f1c2ffff700 1 --2- 172.21.15.7:0/2696229436 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1c10066f30 0x7f1c100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.716+0000 7f1c2ffff700 1 --2- 172.21.15.7:0/2696229436 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1c10066f30 0x7f1c100693e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1c200018d0 tx=0x7f1c20009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:30.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.720+0000 7f1c2cff9700 1 -- 172.21.15.7:0/2696229436 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c301559d0 con 0x7f1c30157ca0 2024-01-27T02:43:30.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.922+0000 7f1c365b5700 1 -- 172.21.15.7:0/2696229436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c301559a0 con 0x7f1c30157ca0 2024-01-27T02:43:30.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.923+0000 7f1c2cff9700 1 -- 172.21.15.7:0/2696229436 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v179) v1 ==== 74+0+15330 (secure 0 0 0) 0x7f1c18053170 con 0x7f1c30157ca0 2024-01-27T02:43:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:30.924 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":179,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:43:29.599410+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:43:27.492968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"179","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-01-27T02:43:28.579977+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"179","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:43:30.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.925+0000 7f1c365b5700 1 -- 172.21.15.7:0/2696229436 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1c10066f30 msgr2=0x7f1c100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:30.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.926+0000 7f1c365b5700 1 --2- 172.21.15.7:0/2696229436 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1c10066f30 0x7f1c100693e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1c200018d0 tx=0x7f1c20009040 comp rx=0 tx=0).stop 2024-01-27T02:43:30.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.926+0000 7f1c365b5700 1 -- 172.21.15.7:0/2696229436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c30157ca0 msgr2=0x7f1c301523d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:30.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.926+0000 7f1c365b5700 1 --2- 172.21.15.7:0/2696229436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c30157ca0 0x7f1c301523d0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f1c3006ea80 tx=0x7f1c18017000 comp rx=0 tx=0).stop 2024-01-27T02:43:30.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.926+0000 7f1c365b5700 1 -- 172.21.15.7:0/2696229436 shutdown_connections 2024-01-27T02:43:30.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.926+0000 7f1c365b5700 1 --2- 172.21.15.7:0/2696229436 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1c10066f30 0x7f1c100693e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.926+0000 7f1c365b5700 1 --2- 172.21.15.7:0/2696229436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c30158070 0x7f1c300c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.926+0000 7f1c365b5700 1 --2- 172.21.15.7:0/2696229436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c300c3ae0 0x7f1c300679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.926+0000 7f1c365b5700 1 --2- 172.21.15.7:0/2696229436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c30157ca0 0x7f1c301523d0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:30.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.926+0000 7f1c365b5700 1 -- 172.21.15.7:0/2696229436 >> 172.21.15.7:0/2696229436 conn(0x7f1c30098020 msgr2=0x7f1c3006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:30.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.926+0000 7f1c365b5700 1 -- 172.21.15.7:0/2696229436 shutdown_connections 2024-01-27T02:43:30.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:30.926+0000 7f1c365b5700 1 -- 172.21.15.7:0/2696229436 wait complete. 2024-01-27T02:43:30.941 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-27T02:43:31.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.075+0000 7f82e2c1a700 1 Processor -- start 2024-01-27T02:43:31.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.076+0000 7f82e2c1a700 1 -- start start 2024-01-27T02:43:31.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.076+0000 7f82e2c1a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82dc1419e0 0x7f82dc0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:31.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.076+0000 7f82e2c1a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82dc0c9660 con 0x7f82dc0cbd20 2024-01-27T02:43:31.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.076+0000 7f82e2c1a700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82dc15be00 con 0x7f82dc0cb950 2024-01-27T02:43:31.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.076+0000 7f82e2c1a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc15c980 con 0x7f82dc1419e0 2024-01-27T02:43:31.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.077+0000 7f82dbfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f82dc0cbd20 0x7f82dc1412d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40364/0 (socket says 172.21.15.7:40364) 2024-01-27T02:43:31.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.077+0000 7f82dbfff700 1 -- 172.21.15.7:0/991162422 learned_addr learned my addr 172.21.15.7:0/991162422 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:31.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.077+0000 7f82e11b7700 1 --2- 172.21.15.7:0/991162422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82dc1419e0 0x7f82dc0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:31.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.077+0000 7f82db7fe700 1 -- 172.21.15.7:0/991162422 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1612877847 0 0) 0x7f82dc0c9660 con 0x7f82dc0cbd20 2024-01-27T02:43:31.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.077+0000 7f82db7fe700 1 -- 172.21.15.7:0/991162422 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82cc003570 con 0x7f82dc0cbd20 2024-01-27T02:43:31.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.077+0000 7f82db7fe700 1 -- 172.21.15.7:0/991162422 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 472178327 0 0) 0x7f82dc15be00 con 0x7f82dc0cb950 2024-01-27T02:43:31.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.077+0000 7f82db7fe700 1 -- 172.21.15.7:0/991162422 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82dc0c9660 con 0x7f82dc0cb950 2024-01-27T02:43:31.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.077+0000 7f82db7fe700 1 -- 172.21.15.7:0/991162422 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 186484640 0 0) 0x7f82cc003570 con 0x7f82dc0cbd20 2024-01-27T02:43:31.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.077+0000 7f82db7fe700 1 -- 172.21.15.7:0/991162422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82dc1419e0 msgr2=0x7f82dc0b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:31.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.077+0000 7f82db7fe700 1 --2- 172.21.15.7:0/991162422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82dc1419e0 0x7f82dc0b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:31.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.077+0000 7f82db7fe700 1 -- 172.21.15.7:0/991162422 >> v1:172.21.15.188:6789/0 conn(0x7f82dc0cb950 legacy=0x7f82dc0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:31.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.077+0000 7f82db7fe700 1 -- 172.21.15.7:0/991162422 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc15c0e0 con 0x7f82dc0cbd20 2024-01-27T02:43:31.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.077+0000 7f82db7fe700 1 -- 172.21.15.7:0/991162422 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f82c8001470 con 0x7f82dc0cbd20 2024-01-27T02:43:31.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.077+0000 7f82db7fe700 1 -- 172.21.15.7:0/991162422 >> v1:172.21.15.7:6789/0 conn(0x7f82dc0cbd20 legacy=0x7f82dc1412d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:31.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.077+0000 7f82db7fe700 1 --2- 172.21.15.7:0/991162422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82cc003c20 0x7f82cc004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:31.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.077+0000 7f82db7fe700 1 --2- 172.21.15.7:0/991162422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82dc1419e0 0x7f82cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:31.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.077+0000 7f82db7fe700 1 --2- 172.21.15.7:0/991162422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82cc004d50 0x7f82cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:31.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.077+0000 7f82db7fe700 1 -- 172.21.15.7:0/991162422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc15c980 con 0x7f82cc003c20 2024-01-27T02:43:31.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.077+0000 7f82db7fe700 1 -- 172.21.15.7:0/991162422 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc00bbf0 con 0x7f82cc004d50 2024-01-27T02:43:31.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.077+0000 7f82db7fe700 1 -- 172.21.15.7:0/991162422 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc00bd60 con 0x7f82dc1419e0 2024-01-27T02:43:31.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.077+0000 7f82e09b6700 1 --2- 172.21.15.7:0/991162422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82cc003c20 0x7f82cc004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:31.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.078+0000 7f82e11b7700 1 --2- 172.21.15.7:0/991162422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82cc004d50 0x7f82cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:31.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.078+0000 7f82dbfff700 1 --2- 172.21.15.7:0/991162422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82dc1419e0 0x7f82cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:31.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.078+0000 7f82e09b6700 1 -- 172.21.15.7:0/991162422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82dc1419e0 msgr2=0x7f82cc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:31.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.078+0000 7f82e09b6700 1 --2- 172.21.15.7:0/991162422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82dc1419e0 0x7f82cc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:31.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.078+0000 7f82e09b6700 1 -- 172.21.15.7:0/991162422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82cc004d50 msgr2=0x7f82cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:31.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.078+0000 7f82e09b6700 1 --2- 172.21.15.7:0/991162422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82cc004d50 0x7f82cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:31.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.078+0000 7f82e09b6700 1 -- 172.21.15.7:0/991162422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82cc003570 con 0x7f82cc003c20 2024-01-27T02:43:31.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.078+0000 7f82e09b6700 1 --2- 172.21.15.7:0/991162422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82cc003c20 0x7f82cc004010 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f82d00019e0 tx=0x7f82d0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9788dbfed6598a6 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:31.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.078+0000 7f82db7fe700 1 -- 172.21.15.7:0/991162422 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d000c5d0 con 0x7f82cc003c20 2024-01-27T02:43:31.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.078+0000 7f82db7fe700 1 -- 172.21.15.7:0/991162422 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d000c730 con 0x7f82cc003c20 2024-01-27T02:43:31.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.078+0000 7f82db7fe700 1 -- 172.21.15.7:0/991162422 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d000cad0 con 0x7f82cc003c20 2024-01-27T02:43:31.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.079+0000 7f82e2c1a700 1 -- 172.21.15.7:0/991162422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82cc003c20 msgr2=0x7f82cc004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:31.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.079+0000 7f82e2c1a700 1 --2- 172.21.15.7:0/991162422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82cc003c20 0x7f82cc004010 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f82d00019e0 tx=0x7f82d0007d30 comp rx=0 tx=0).stop 2024-01-27T02:43:31.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.079+0000 7f82e11b7700 1 -- 172.21.15.7:0/991162422 reap_dead start 2024-01-27T02:43:31.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.079+0000 7f82e2c1a700 1 -- 172.21.15.7:0/991162422 shutdown_connections 2024-01-27T02:43:31.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.079+0000 7f82e2c1a700 1 -- 172.21.15.7:0/991162422 >> 172.21.15.7:0/991162422 conn(0x7f82dc063580 msgr2=0x7f82dc063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:31.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.079+0000 7f82e2c1a700 1 -- 172.21.15.7:0/991162422 shutdown_connections 2024-01-27T02:43:31.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.079+0000 7f82e2c1a700 1 -- 172.21.15.7:0/991162422 wait complete. 2024-01-27T02:43:31.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.080+0000 7f82e2c1a700 1 Processor -- start 2024-01-27T02:43:31.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.080+0000 7f82e2c1a700 1 -- start start 2024-01-27T02:43:31.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.081+0000 7f82e2c1a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82cc003c20 0x7f82dc13b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:31.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.081+0000 7f82e2c1a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc158810 0x7f82dc158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:31.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.081+0000 7f82e2c1a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82dc157350 0x7f82dc1577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:31.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.081+0000 7f82e2c1a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc15b510 con 0x7f82dc158810 2024-01-27T02:43:31.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.081+0000 7f82e2c1a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc15a120 con 0x7f82dc157350 2024-01-27T02:43:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.081+0000 7f82e2c1a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc157d30 con 0x7f82cc003c20 2024-01-27T02:43:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.081+0000 7f82dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc158810 0x7f82dc158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.081+0000 7f82dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc158810 0x7f82dc158c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42884/0 (socket says 172.21.15.7:42884) 2024-01-27T02:43:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.081+0000 7f82dbfff700 1 -- 172.21.15.7:0/3675866765 learned_addr learned my addr 172.21.15.7:0/3675866765 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.081+0000 7f82e11b7700 1 --2- 172.21.15.7:0/3675866765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82dc157350 0x7f82dc1577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.081+0000 7f82e09b6700 1 --2- 172.21.15.7:0/3675866765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82cc003c20 0x7f82dc13b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.082+0000 7f82dbfff700 1 -- 172.21.15.7:0/3675866765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82cc003c20 msgr2=0x7f82dc13b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.082+0000 7f82dbfff700 1 --2- 172.21.15.7:0/3675866765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82cc003c20 0x7f82dc13b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.082+0000 7f82dbfff700 1 -- 172.21.15.7:0/3675866765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82dc157350 msgr2=0x7f82dc1577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.082+0000 7f82dbfff700 1 --2- 172.21.15.7:0/3675866765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82dc157350 0x7f82dc1577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.082+0000 7f82dbfff700 1 -- 172.21.15.7:0/3675866765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc0b2990 con 0x7f82dc158810 2024-01-27T02:43:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.082+0000 7f82dbfff700 1 --2- 172.21.15.7:0/3675866765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc158810 0x7f82dc158c80 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f82c80073c0 tx=0x7f82c8001d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.082+0000 7f82d8ff9700 1 -- 172.21.15.7:0/3675866765 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c8000c40 con 0x7f82dc158810 2024-01-27T02:43:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.082+0000 7f82d8ff9700 1 -- 172.21.15.7:0/3675866765 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c800f640 con 0x7f82dc158810 2024-01-27T02:43:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.082+0000 7f82d8ff9700 1 -- 172.21.15.7:0/3675866765 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c800f9e0 con 0x7f82dc158810 2024-01-27T02:43:31.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.082+0000 7f82e2c1a700 1 -- 172.21.15.7:0/3675866765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82dc13b7d0 con 0x7f82dc158810 2024-01-27T02:43:31.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.082+0000 7f82e2c1a700 1 -- 172.21.15.7:0/3675866765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82dc0b1d80 con 0x7f82dc158810 2024-01-27T02:43:31.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.083+0000 7f82e2c1a700 1 -- 172.21.15.7:0/3675866765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82dc0b2b20 con 0x7f82dc158810 2024-01-27T02:43:31.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.084+0000 7f82d8ff9700 1 -- 172.21.15.7:0/3675866765 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f82c800fd90 con 0x7f82dc158810 2024-01-27T02:43:31.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.084+0000 7f82d8ff9700 1 --2- 172.21.15.7:0/3675866765 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f82c0066f30 0x7f82c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:31.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.084+0000 7f82d8ff9700 1 -- 172.21.15.7:0/3675866765 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f82c80844a0 con 0x7f82dc158810 2024-01-27T02:43:31.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.084+0000 7f82e09b6700 1 --2- 172.21.15.7:0/3675866765 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f82c0066f30 0x7f82c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:31.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.085+0000 7f82e09b6700 1 --2- 172.21.15.7:0/3675866765 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f82c0066f30 0x7f82c00693e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f82d0008fd0 tx=0x7f82d0015000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:31.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.088+0000 7f82d8ff9700 1 -- 172.21.15.7:0/3675866765 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82c8051710 con 0x7f82dc158810 2024-01-27T02:43:31.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:31.344+0000 7f82e2c1a700 1 -- 172.21.15.7:0/3675866765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f82dc072060 con 0x7f82dc158810 2024-01-27T02:43:32.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.197+0000 7f82d8ff9700 1 -- 172.21.15.7:0/3675866765 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v93) v1 ==== 118+0+0 (secure 0 0 0) 0x7f82c8054d30 con 0x7f82dc158810 2024-01-27T02:43:32.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.200+0000 7f82e2c1a700 1 -- 172.21.15.7:0/3675866765 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f82c0066f30 msgr2=0x7f82c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:32.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.200+0000 7f82e2c1a700 1 --2- 172.21.15.7:0/3675866765 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f82c0066f30 0x7f82c00693e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f82d0008fd0 tx=0x7f82d0015000 comp rx=0 tx=0).stop 2024-01-27T02:43:32.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.200+0000 7f82e2c1a700 1 -- 172.21.15.7:0/3675866765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc158810 msgr2=0x7f82dc158c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:32.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.200+0000 7f82e2c1a700 1 --2- 172.21.15.7:0/3675866765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc158810 0x7f82dc158c80 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f82c80073c0 tx=0x7f82c8001d20 comp rx=0 tx=0).stop 2024-01-27T02:43:32.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.200+0000 7f82e2c1a700 1 -- 172.21.15.7:0/3675866765 shutdown_connections 2024-01-27T02:43:32.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.200+0000 7f82e2c1a700 1 --2- 172.21.15.7:0/3675866765 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f82c0066f30 0x7f82c00693e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:32.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.200+0000 7f82e2c1a700 1 --2- 172.21.15.7:0/3675866765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82dc157350 0x7f82dc1577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:32.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.200+0000 7f82e2c1a700 1 --2- 172.21.15.7:0/3675866765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82cc003c20 0x7f82dc13b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:32.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.200+0000 7f82e2c1a700 1 --2- 172.21.15.7:0/3675866765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc158810 0x7f82dc158c80 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:32.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.200+0000 7f82e2c1a700 1 -- 172.21.15.7:0/3675866765 >> 172.21.15.7:0/3675866765 conn(0x7f82dc063580 msgr2=0x7f82dc063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:32.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.200+0000 7f82e2c1a700 1 -- 172.21.15.7:0/3675866765 shutdown_connections 2024-01-27T02:43:32.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.200+0000 7f82e2c1a700 1 -- 172.21.15.7:0/3675866765 wait complete. 2024-01-27T02:43:32.216 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-27T02:43:32.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.354+0000 7f5330b9d700 1 Processor -- start 2024-01-27T02:43:32.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.354+0000 7f5330b9d700 1 -- start start 2024-01-27T02:43:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.356+0000 7f5330b9d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f532c0cdb50 0x7f532c15b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.356+0000 7f5330b9d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532c0cf280 con 0x7f532c0cdf20 2024-01-27T02:43:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.356+0000 7f5330b9d700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532c15e500 con 0x7f532c159b90 2024-01-27T02:43:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.356+0000 7f5330b9d700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532c15e7e0 con 0x7f532c0cdb50 2024-01-27T02:43:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.356+0000 7f5329d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f532c0cdf20 0x7f532c15dd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38482/0 (socket says 172.21.15.7:38482) 2024-01-27T02:43:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.356+0000 7f5329d9b700 1 -- 172.21.15.7:0/2028087353 learned_addr learned my addr 172.21.15.7:0/2028087353 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.356+0000 7f532a59c700 1 --2- 172.21.15.7:0/2028087353 >> v2:172.21.15.188:3300/0 conn(0x7f532c0cdb50 0x7f532c15b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.356+0000 7f532959a700 1 -- 172.21.15.7:0/2028087353 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2991241532 0 0) 0x7f532c0cf280 con 0x7f532c0cdf20 2024-01-27T02:43:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.356+0000 7f532959a700 1 -- 172.21.15.7:0/2028087353 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5318003570 con 0x7f532c0cdf20 2024-01-27T02:43:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.356+0000 7f532959a700 1 -- 172.21.15.7:0/2028087353 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 488853806 0 0) 0x7f532c15e500 con 0x7f532c159b90 2024-01-27T02:43:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.356+0000 7f532959a700 1 -- 172.21.15.7:0/2028087353 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f532c0cf280 con 0x7f532c159b90 2024-01-27T02:43:32.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.356+0000 7f532959a700 1 -- 172.21.15.7:0/2028087353 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2743702149 0 0) 0x7f5318003570 con 0x7f532c0cdf20 2024-01-27T02:43:32.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.356+0000 7f532959a700 1 -- 172.21.15.7:0/2028087353 >> v2:172.21.15.188:3300/0 conn(0x7f532c0cdb50 msgr2=0x7f532c15b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:32.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.356+0000 7f532959a700 1 --2- 172.21.15.7:0/2028087353 >> v2:172.21.15.188:3300/0 conn(0x7f532c0cdb50 0x7f532c15b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:32.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.356+0000 7f532959a700 1 -- 172.21.15.7:0/2028087353 >> v1:172.21.15.188:6789/0 conn(0x7f532c159b90 legacy=0x7f532c15a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:32.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.356+0000 7f532959a700 1 -- 172.21.15.7:0/2028087353 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c15eac0 con 0x7f532c0cdf20 2024-01-27T02:43:32.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.356+0000 7f532959a700 1 -- 172.21.15.7:0/2028087353 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f5314001470 con 0x7f532c0cdf20 2024-01-27T02:43:32.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.356+0000 7f532959a700 1 -- 172.21.15.7:0/2028087353 >> v1:172.21.15.7:6789/0 conn(0x7f532c0cdf20 legacy=0x7f532c15dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:32.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.356+0000 7f532959a700 1 --2- 172.21.15.7:0/2028087353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5318003bf0 0x7f5318003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:32.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.356+0000 7f532959a700 1 --2- 172.21.15.7:0/2028087353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5318004810 0x7f5318004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:32.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.357+0000 7f532959a700 1 --2- 172.21.15.7:0/2028087353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53180051c0 0x7f531800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.357+0000 7f532959a700 1 -- 172.21.15.7:0/2028087353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c15e7e0 con 0x7f5318003bf0 2024-01-27T02:43:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.357+0000 7f532959a700 1 -- 172.21.15.7:0/2028087353 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531800c160 con 0x7f5318004810 2024-01-27T02:43:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.357+0000 7f532959a700 1 -- 172.21.15.7:0/2028087353 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531800c320 con 0x7f53180051c0 2024-01-27T02:43:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.357+0000 7f532a59c700 1 --2- 172.21.15.7:0/2028087353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5318003bf0 0x7f5318003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.357+0000 7f5329d9b700 1 --2- 172.21.15.7:0/2028087353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5318004810 0x7f5318004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.357+0000 7f532ad9d700 1 --2- 172.21.15.7:0/2028087353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53180051c0 0x7f531800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.357+0000 7f532a59c700 1 -- 172.21.15.7:0/2028087353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53180051c0 msgr2=0x7f531800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.357+0000 7f532a59c700 1 --2- 172.21.15.7:0/2028087353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53180051c0 0x7f531800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.357+0000 7f532a59c700 1 -- 172.21.15.7:0/2028087353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5318004810 msgr2=0x7f5318004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.357+0000 7f532a59c700 1 --2- 172.21.15.7:0/2028087353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5318004810 0x7f5318004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.357+0000 7f532a59c700 1 -- 172.21.15.7:0/2028087353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5318003570 con 0x7f5318003bf0 2024-01-27T02:43:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.357+0000 7f532ad9d700 1 -- 172.21.15.7:0/2028087353 reap_dead start 2024-01-27T02:43:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.357+0000 7f532a59c700 1 --2- 172.21.15.7:0/2028087353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5318003bf0 0x7f5318003fe0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f531c006f00 tx=0x7f531c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69e9614f0a7f02d server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.357+0000 7f532959a700 1 -- 172.21.15.7:0/2028087353 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c00e710 con 0x7f5318003bf0 2024-01-27T02:43:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.357+0000 7f532959a700 1 -- 172.21.15.7:0/2028087353 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531c00e870 con 0x7f5318003bf0 2024-01-27T02:43:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.357+0000 7f532959a700 1 -- 172.21.15.7:0/2028087353 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c00ec30 con 0x7f5318003bf0 2024-01-27T02:43:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.358+0000 7f5330b9d700 1 -- 172.21.15.7:0/2028087353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5318003bf0 msgr2=0x7f5318003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.358+0000 7f5330b9d700 1 --2- 172.21.15.7:0/2028087353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5318003bf0 0x7f5318003fe0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f531c006f00 tx=0x7f531c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.358+0000 7f5330b9d700 1 -- 172.21.15.7:0/2028087353 shutdown_connections 2024-01-27T02:43:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.358+0000 7f5330b9d700 1 --2- 172.21.15.7:0/2028087353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5318003bf0 0x7f5318003fe0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.358+0000 7f5330b9d700 1 -- 172.21.15.7:0/2028087353 >> 172.21.15.7:0/2028087353 conn(0x7f532c063580 msgr2=0x7f532c063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.358+0000 7f5330b9d700 1 -- 172.21.15.7:0/2028087353 shutdown_connections 2024-01-27T02:43:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.358+0000 7f5330b9d700 1 -- 172.21.15.7:0/2028087353 wait complete. 2024-01-27T02:43:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.359+0000 7f5330b9d700 1 Processor -- start 2024-01-27T02:43:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.359+0000 7f5330b9d700 1 -- start start 2024-01-27T02:43:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.360+0000 7f5330b9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5318003bf0 0x7f532c16a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.360+0000 7f5330b9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f532c14da80 0x7f532c14d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.360+0000 7f5330b9d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f532c0b6280 0x7f532c0b6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.360+0000 7f5330b9d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c0b6c60 con 0x7f532c14da80 2024-01-27T02:43:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.360+0000 7f5330b9d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c16ac90 con 0x7f5318003bf0 2024-01-27T02:43:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.360+0000 7f5330b9d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532c155700 con 0x7f532c0b6280 2024-01-27T02:43:32.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.360+0000 7f5329d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f532c14da80 0x7f532c14d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:32.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.360+0000 7f5329d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f532c14da80 0x7f532c14d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34758/0 (socket says 172.21.15.7:34758) 2024-01-27T02:43:32.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.360+0000 7f5329d9b700 1 -- 172.21.15.7:0/3251289897 learned_addr learned my addr 172.21.15.7:0/3251289897 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:32.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.360+0000 7f532a59c700 1 --2- 172.21.15.7:0/3251289897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5318003bf0 0x7f532c16a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:32.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.360+0000 7f532ad9d700 1 --2- 172.21.15.7:0/3251289897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f532c0b6280 0x7f532c0b6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:32.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.360+0000 7f5329d9b700 1 -- 172.21.15.7:0/3251289897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f532c0b6280 msgr2=0x7f532c0b6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:32.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.360+0000 7f5329d9b700 1 --2- 172.21.15.7:0/3251289897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f532c0b6280 0x7f532c0b6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:32.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.360+0000 7f5329d9b700 1 -- 172.21.15.7:0/3251289897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5318003bf0 msgr2=0x7f532c16a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:32.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.360+0000 7f5329d9b700 1 --2- 172.21.15.7:0/3251289897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5318003bf0 0x7f532c16a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:32.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.360+0000 7f5329d9b700 1 -- 172.21.15.7:0/3251289897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c14d8a0 con 0x7f532c14da80 2024-01-27T02:43:32.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.360+0000 7f532a59c700 1 --2- 172.21.15.7:0/3251289897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5318003bf0 0x7f532c16a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:32.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.361+0000 7f5329d9b700 1 --2- 172.21.15.7:0/3251289897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f532c14da80 0x7f532c14d360 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f53140073f0 tx=0x7f5314001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:32.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.361+0000 7f53227fc700 1 -- 172.21.15.7:0/3251289897 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314018030 con 0x7f532c14da80 2024-01-27T02:43:32.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.361+0000 7f53227fc700 1 -- 172.21.15.7:0/3251289897 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5314000c70 con 0x7f532c14da80 2024-01-27T02:43:32.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.361+0000 7f53227fc700 1 -- 172.21.15.7:0/3251289897 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531400fb20 con 0x7f532c14da80 2024-01-27T02:43:32.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.361+0000 7f5330b9d700 1 -- 172.21.15.7:0/3251289897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532c16aec0 con 0x7f532c14da80 2024-01-27T02:43:32.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.361+0000 7f5330b9d700 1 -- 172.21.15.7:0/3251289897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532c16b480 con 0x7f532c14da80 2024-01-27T02:43:32.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.362+0000 7f5330b9d700 1 -- 172.21.15.7:0/3251289897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f532c1558b0 con 0x7f532c14da80 2024-01-27T02:43:32.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.362+0000 7f53227fc700 1 -- 172.21.15.7:0/3251289897 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f531401b070 con 0x7f532c14da80 2024-01-27T02:43:32.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.363+0000 7f53227fc700 1 --2- 172.21.15.7:0/3251289897 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f530c066f30 0x7f530c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:32.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.363+0000 7f53227fc700 1 -- 172.21.15.7:0/3251289897 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5314019070 con 0x7f532c14da80 2024-01-27T02:43:32.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.363+0000 7f532a59c700 1 --2- 172.21.15.7:0/3251289897 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f530c066f30 0x7f530c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:32.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.363+0000 7f532a59c700 1 --2- 172.21.15.7:0/3251289897 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f530c066f30 0x7f530c0693e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f531c0091e0 tx=0x7f531c01a000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:32.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.367+0000 7f53227fc700 1 -- 172.21.15.7:0/3251289897 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f532c1558b0 con 0x7f532c14da80 2024-01-27T02:43:32.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:32.619+0000 7f5330b9d700 1 -- 172.21.15.7:0/3251289897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f532c1558b0 con 0x7f532c14da80 2024-01-27T02:43:33.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.198+0000 7f53227fc700 1 -- 172.21.15.7:0/3251289897 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v94) v1 ==== 122+0+0 (secure 0 0 0) 0x7f532c1558b0 con 0x7f532c14da80 2024-01-27T02:43:33.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.201+0000 7f5330b9d700 1 -- 172.21.15.7:0/3251289897 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f530c066f30 msgr2=0x7f530c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:33.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.201+0000 7f5330b9d700 1 --2- 172.21.15.7:0/3251289897 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f530c066f30 0x7f530c0693e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f531c0091e0 tx=0x7f531c01a000 comp rx=0 tx=0).stop 2024-01-27T02:43:33.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.201+0000 7f5330b9d700 1 -- 172.21.15.7:0/3251289897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f532c14da80 msgr2=0x7f532c14d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:33.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.201+0000 7f5330b9d700 1 --2- 172.21.15.7:0/3251289897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f532c14da80 0x7f532c14d360 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f53140073f0 tx=0x7f5314001e10 comp rx=0 tx=0).stop 2024-01-27T02:43:33.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.201+0000 7f5330b9d700 1 -- 172.21.15.7:0/3251289897 shutdown_connections 2024-01-27T02:43:33.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.201+0000 7f5330b9d700 1 --2- 172.21.15.7:0/3251289897 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f530c066f30 0x7f530c0693e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.201+0000 7f5330b9d700 1 --2- 172.21.15.7:0/3251289897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f532c0b6280 0x7f532c0b6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.201+0000 7f5330b9d700 1 --2- 172.21.15.7:0/3251289897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5318003bf0 0x7f532c16a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.201+0000 7f5330b9d700 1 --2- 172.21.15.7:0/3251289897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f532c14da80 0x7f532c14d360 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.201+0000 7f5330b9d700 1 -- 172.21.15.7:0/3251289897 >> 172.21.15.7:0/3251289897 conn(0x7f532c063580 msgr2=0x7f532c158ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.201+0000 7f5330b9d700 1 -- 172.21.15.7:0/3251289897 shutdown_connections 2024-01-27T02:43:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.201+0000 7f5330b9d700 1 -- 172.21.15.7:0/3251289897 wait complete. 2024-01-27T02:43:33.217 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:43:33.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.352+0000 7fda357e1700 1 Processor -- start 2024-01-27T02:43:33.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.352+0000 7fda357e1700 1 -- start start 2024-01-27T02:43:33.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.353+0000 7fda357e1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda300b3350 0x7fda300ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:33.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.353+0000 7fda357e1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fda300ce810 0x7fda301553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:33.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.353+0000 7fda357e1700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda30159390 con 0x7fda300b2f80 2024-01-27T02:43:33.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.353+0000 7fda357e1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda30158780 con 0x7fda300ce810 2024-01-27T02:43:33.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.353+0000 7fda357e1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda30158a60 con 0x7fda300b3350 2024-01-27T02:43:33.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.353+0000 7fda2f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fda300ce810 0x7fda301553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:33.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.353+0000 7fda2f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fda300ce810 0x7fda301553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34770/0 (socket says 172.21.15.7:34770) 2024-01-27T02:43:33.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.353+0000 7fda2f7fe700 1 -- 172.21.15.7:0/3558909502 learned_addr learned my addr 172.21.15.7:0/3558909502 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:33.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.354+0000 7fda2e7fc700 1 --2- 172.21.15.7:0/3558909502 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda300b3350 0x7fda300ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:33.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.354+0000 7fda2f7fe700 1 -- 172.21.15.7:0/3558909502 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda300b3350 msgr2=0x7fda300ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:33.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.354+0000 7fda2f7fe700 1 --2- 172.21.15.7:0/3558909502 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda300b3350 0x7fda300ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.354+0000 7fda2f7fe700 1 -- 172.21.15.7:0/3558909502 >> v1:172.21.15.188:6789/0 conn(0x7fda300b2f80 legacy=0x7fda3006a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:33.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.354+0000 7fda2f7fe700 1 -- 172.21.15.7:0/3558909502 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda30158d40 con 0x7fda300ce810 2024-01-27T02:43:33.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.354+0000 7fda2f7fe700 1 --2- 172.21.15.7:0/3558909502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda300ce810 0x7fda301553e0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fda28009f50 tx=0x7fda28009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ef72c1fef222e2 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:33.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.354+0000 7fda2dffb700 1 -- 172.21.15.7:0/3558909502 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2800e070 con 0x7fda300ce810 2024-01-27T02:43:33.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.354+0000 7fda2dffb700 1 -- 172.21.15.7:0/3558909502 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2800b9c0 con 0x7fda300ce810 2024-01-27T02:43:33.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.354+0000 7fda2dffb700 1 -- 172.21.15.7:0/3558909502 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda280092e0 con 0x7fda300ce810 2024-01-27T02:43:33.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.355+0000 7fda357e1700 1 -- 172.21.15.7:0/3558909502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda300ce810 msgr2=0x7fda301553e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:33.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.355+0000 7fda357e1700 1 --2- 172.21.15.7:0/3558909502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda300ce810 0x7fda301553e0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fda28009f50 tx=0x7fda28009840 comp rx=0 tx=0).stop 2024-01-27T02:43:33.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.355+0000 7fda357e1700 1 -- 172.21.15.7:0/3558909502 shutdown_connections 2024-01-27T02:43:33.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.355+0000 7fda357e1700 1 --2- 172.21.15.7:0/3558909502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda300ce810 0x7fda301553e0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.355+0000 7fda357e1700 1 --2- 172.21.15.7:0/3558909502 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda300b3350 0x7fda300ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.355+0000 7fda357e1700 1 -- 172.21.15.7:0/3558909502 >> 172.21.15.7:0/3558909502 conn(0x7fda30068430 msgr2=0x7fda3006be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:33.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.355+0000 7fda357e1700 1 -- 172.21.15.7:0/3558909502 shutdown_connections 2024-01-27T02:43:33.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.355+0000 7fda357e1700 1 -- 172.21.15.7:0/3558909502 wait complete. 2024-01-27T02:43:33.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.356+0000 7fda357e1700 1 Processor -- start 2024-01-27T02:43:33.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.356+0000 7fda357e1700 1 -- start start 2024-01-27T02:43:33.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.357+0000 7fda357e1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda300b2f80 0x7fda3014d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:33.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.357+0000 7fda357e1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda300b3350 0x7fda30150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:33.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.357+0000 7fda357e1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda300ce810 0x7fda30148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:33.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.357+0000 7fda357e1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda301684a0 con 0x7fda300b3350 2024-01-27T02:43:33.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.357+0000 7fda357e1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda30168720 con 0x7fda300b2f80 2024-01-27T02:43:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.357+0000 7fda357e1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda30168a00 con 0x7fda300ce810 2024-01-27T02:43:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.357+0000 7fda2e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda300b3350 0x7fda30150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.357+0000 7fda2e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda300b3350 0x7fda30150390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34780/0 (socket says 172.21.15.7:34780) 2024-01-27T02:43:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.357+0000 7fda2e7fc700 1 -- 172.21.15.7:0/3982318127 learned_addr learned my addr 172.21.15.7:0/3982318127 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.357+0000 7fda2effd700 1 --2- 172.21.15.7:0/3982318127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda300b2f80 0x7fda3014d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.357+0000 7fda2f7fe700 1 --2- 172.21.15.7:0/3982318127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda300ce810 0x7fda30148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.358+0000 7fda2e7fc700 1 -- 172.21.15.7:0/3982318127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda300ce810 msgr2=0x7fda30148e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.358+0000 7fda2e7fc700 1 --2- 172.21.15.7:0/3982318127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda300ce810 0x7fda30148e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.358+0000 7fda2e7fc700 1 -- 172.21.15.7:0/3982318127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda300b2f80 msgr2=0x7fda3014d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.358+0000 7fda2e7fc700 1 --2- 172.21.15.7:0/3982318127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda300b2f80 0x7fda3014d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.358+0000 7fda2e7fc700 1 -- 172.21.15.7:0/3982318127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda28009ae0 con 0x7fda300b3350 2024-01-27T02:43:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.358+0000 7fda2e7fc700 1 --2- 172.21.15.7:0/3982318127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda300b3350 0x7fda30150390 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fda2000ec80 tx=0x7fda200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:33.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.358+0000 7fda16ffd700 1 -- 172.21.15.7:0/3982318127 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2000c9d0 con 0x7fda300b3350 2024-01-27T02:43:33.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.358+0000 7fda16ffd700 1 -- 172.21.15.7:0/3982318127 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2000cb30 con 0x7fda300b3350 2024-01-27T02:43:33.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.358+0000 7fda357e1700 1 -- 172.21.15.7:0/3982318127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda30168ce0 con 0x7fda300b3350 2024-01-27T02:43:33.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.358+0000 7fda16ffd700 1 -- 172.21.15.7:0/3982318127 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20013470 con 0x7fda300b3350 2024-01-27T02:43:33.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.358+0000 7fda357e1700 1 -- 172.21.15.7:0/3982318127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda3014d550 con 0x7fda300b3350 2024-01-27T02:43:33.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.359+0000 7fda357e1700 1 -- 172.21.15.7:0/3982318127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda30169200 con 0x7fda300b3350 2024-01-27T02:43:33.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.360+0000 7fda16ffd700 1 -- 172.21.15.7:0/3982318127 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fda20013820 con 0x7fda300b3350 2024-01-27T02:43:33.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.360+0000 7fda16ffd700 1 --2- 172.21.15.7:0/3982318127 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fda1c066f30 0x7fda1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:33.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.360+0000 7fda16ffd700 1 -- 172.21.15.7:0/3982318127 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fda20084920 con 0x7fda300b3350 2024-01-27T02:43:33.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.360+0000 7fda2effd700 1 --2- 172.21.15.7:0/3982318127 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fda1c066f30 0x7fda1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:33.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.361+0000 7fda2effd700 1 --2- 172.21.15.7:0/3982318127 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fda1c066f30 0x7fda1c0693e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fda24003860 tx=0x7fda24009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:33.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.364+0000 7fda16ffd700 1 -- 172.21.15.7:0/3982318127 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda30169200 con 0x7fda300b3350 2024-01-27T02:43:33.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.613+0000 7fda357e1700 1 -- 172.21.15.7:0/3982318127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda30169200 con 0x7fda300b3350 2024-01-27T02:43:33.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.613+0000 7fda16ffd700 1 -- 172.21.15.7:0/3982318127 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4595 (secure 0 0 0) 0x7fda30169200 con 0x7fda300b3350 2024-01-27T02:43:33.614 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:33.614 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6704,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/1199331492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1199331492},{"type":"v1","addr":"172.21.15.7:6834","nonce":1199331492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":6707,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/721335086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":721335086},{"type":"v1","addr":"172.21.15.7:6835","nonce":721335086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":7699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/450555206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":450555206},{"type":"v1","addr":"172.21.15.188:6836","nonce":450555206}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:43:29.599748+0000","modified":"2024-01-27T02:43:33.196288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7702":{"gid":7702,"name":"b","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/3853795510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3853795510},{"type":"v1","addr":"172.21.15.188:6837","nonce":3853795510}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-27T02:43:33.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.615+0000 7fda357e1700 1 -- 172.21.15.7:0/3982318127 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fda1c066f30 msgr2=0x7fda1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:33.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.615+0000 7fda357e1700 1 --2- 172.21.15.7:0/3982318127 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fda1c066f30 0x7fda1c0693e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fda24003860 tx=0x7fda24009040 comp rx=0 tx=0).stop 2024-01-27T02:43:33.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.615+0000 7fda357e1700 1 -- 172.21.15.7:0/3982318127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda300b3350 msgr2=0x7fda30150390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:33.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.615+0000 7fda357e1700 1 --2- 172.21.15.7:0/3982318127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda300b3350 0x7fda30150390 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fda2000ec80 tx=0x7fda200091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.615+0000 7fda357e1700 1 -- 172.21.15.7:0/3982318127 shutdown_connections 2024-01-27T02:43:33.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.615+0000 7fda357e1700 1 --2- 172.21.15.7:0/3982318127 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fda1c066f30 0x7fda1c0693e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.615+0000 7fda357e1700 1 --2- 172.21.15.7:0/3982318127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda300ce810 0x7fda30148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.615+0000 7fda357e1700 1 --2- 172.21.15.7:0/3982318127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda300b2f80 0x7fda3014d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.615+0000 7fda357e1700 1 --2- 172.21.15.7:0/3982318127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda300b3350 0x7fda30150390 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.616+0000 7fda357e1700 1 -- 172.21.15.7:0/3982318127 >> 172.21.15.7:0/3982318127 conn(0x7fda30068430 msgr2=0x7fda3006be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:33.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.616+0000 7fda357e1700 1 -- 172.21.15.7:0/3982318127 shutdown_connections 2024-01-27T02:43:33.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.616+0000 7fda357e1700 1 -- 172.21.15.7:0/3982318127 wait complete. 2024-01-27T02:43:33.617 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 94 2024-01-27T02:43:33.630 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:43:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.762+0000 7f9cda4fe700 1 Processor -- start 2024-01-27T02:43:33.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.762+0000 7f9cda4fe700 1 -- start start 2024-01-27T02:43:33.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.763+0000 7f9cda4fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40c41e0 0x7f9cd415a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:33.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.763+0000 7f9cda4fe700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd415b6c0 con 0x7f9cd40637d0 2024-01-27T02:43:33.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.763+0000 7f9cda4fe700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd406f2e0 con 0x7f9cd415ac80 2024-01-27T02:43:33.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.763+0000 7f9cda4fe700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd415b100 con 0x7f9cd40c41e0 2024-01-27T02:43:33.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.763+0000 7f9cd3fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9cd40637d0 0x7f9cd406ebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38496/0 (socket says 172.21.15.7:38496) 2024-01-27T02:43:33.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.763+0000 7f9cd3fff700 1 -- 172.21.15.7:0/1852723641 learned_addr learned my addr 172.21.15.7:0/1852723641 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:33.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.763+0000 7f9cd37fe700 1 --2- 172.21.15.7:0/1852723641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40c41e0 0x7f9cd415a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:33.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.763+0000 7f9cd2ffd700 1 -- 172.21.15.7:0/1852723641 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2028736114 0 0) 0x7f9cd415b6c0 con 0x7f9cd40637d0 2024-01-27T02:43:33.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.763+0000 7f9cd2ffd700 1 -- 172.21.15.7:0/1852723641 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cbc003570 con 0x7f9cd40637d0 2024-01-27T02:43:33.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.763+0000 7f9cd2ffd700 1 -- 172.21.15.7:0/1852723641 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3972011473 0 0) 0x7f9cbc003570 con 0x7f9cd40637d0 2024-01-27T02:43:33.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.763+0000 7f9cd2ffd700 1 -- 172.21.15.7:0/1852723641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40c41e0 msgr2=0x7f9cd415a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:33.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.763+0000 7f9cd2ffd700 1 --2- 172.21.15.7:0/1852723641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40c41e0 0x7f9cd415a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.763+0000 7f9cd2ffd700 1 -- 172.21.15.7:0/1852723641 >> v1:172.21.15.188:6789/0 conn(0x7f9cd415ac80 legacy=0x7f9cd40c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:33.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.763+0000 7f9cd2ffd700 1 -- 172.21.15.7:0/1852723641 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd415b3e0 con 0x7f9cd40637d0 2024-01-27T02:43:33.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.763+0000 7f9cd2ffd700 1 -- 172.21.15.7:0/1852723641 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f9cc4001470 con 0x7f9cd40637d0 2024-01-27T02:43:33.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.763+0000 7f9cd2ffd700 1 -- 172.21.15.7:0/1852723641 >> v1:172.21.15.7:6789/0 conn(0x7f9cd40637d0 legacy=0x7f9cd406ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:33.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.763+0000 7f9cd2ffd700 1 --2- 172.21.15.7:0/1852723641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cbc003a90 0x7f9cbc003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:33.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.763+0000 7f9cd2ffd700 1 --2- 172.21.15.7:0/1852723641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40c41e0 0x7f9cbc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:33.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.763+0000 7f9cd2ffd700 1 --2- 172.21.15.7:0/1852723641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cbc004d50 0x7f9cbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:33.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.764+0000 7f9cd2ffd700 1 -- 172.21.15.7:0/1852723641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd415b100 con 0x7f9cbc004d50 2024-01-27T02:43:33.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.764+0000 7f9cd2ffd700 1 -- 172.21.15.7:0/1852723641 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc00bb10 con 0x7f9cbc003a90 2024-01-27T02:43:33.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.764+0000 7f9cd2ffd700 1 -- 172.21.15.7:0/1852723641 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc00bcd0 con 0x7f9cd40c41e0 2024-01-27T02:43:33.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.764+0000 7f9cd3fff700 1 --2- 172.21.15.7:0/1852723641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cbc003a90 0x7f9cbc003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:33.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.764+0000 7f9cd37fe700 1 --2- 172.21.15.7:0/1852723641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40c41e0 0x7f9cbc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.764+0000 7f9cd8a9b700 1 --2- 172.21.15.7:0/1852723641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cbc004d50 0x7f9cbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.764+0000 7f9cd3fff700 1 -- 172.21.15.7:0/1852723641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40c41e0 msgr2=0x7f9cbc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.764+0000 7f9cd3fff700 1 --2- 172.21.15.7:0/1852723641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40c41e0 0x7f9cbc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.764+0000 7f9cd3fff700 1 -- 172.21.15.7:0/1852723641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cbc004d50 msgr2=0x7f9cbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.764+0000 7f9cd3fff700 1 --2- 172.21.15.7:0/1852723641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cbc004d50 0x7f9cbc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.764+0000 7f9cd3fff700 1 -- 172.21.15.7:0/1852723641 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cbc003570 con 0x7f9cbc003a90 2024-01-27T02:43:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.764+0000 7f9cd8a9b700 1 --2- 172.21.15.7:0/1852723641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cbc004d50 0x7f9cbc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:43:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.764+0000 7f9cd37fe700 1 --2- 172.21.15.7:0/1852723641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40c41e0 0x7f9cbc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:43:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.764+0000 7f9cd3fff700 1 --2- 172.21.15.7:0/1852723641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cbc003a90 0x7f9cbc003e60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400c2d0 tx=0x7f9cc4001df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcf43cecd9ca56a4 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.765+0000 7f9cd2ffd700 1 -- 172.21.15.7:0/1852723641 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc400cbc0 con 0x7f9cbc003a90 2024-01-27T02:43:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.765+0000 7f9cd2ffd700 1 -- 172.21.15.7:0/1852723641 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc400cd20 con 0x7f9cbc003a90 2024-01-27T02:43:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.765+0000 7f9cd2ffd700 1 -- 172.21.15.7:0/1852723641 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4005b70 con 0x7f9cbc003a90 2024-01-27T02:43:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.765+0000 7f9cda4fe700 1 -- 172.21.15.7:0/1852723641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cbc003a90 msgr2=0x7f9cbc003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:33.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.765+0000 7f9cda4fe700 1 --2- 172.21.15.7:0/1852723641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cbc003a90 0x7f9cbc003e60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400c2d0 tx=0x7f9cc4001df0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.765+0000 7f9cd8a9b700 1 -- 172.21.15.7:0/1852723641 reap_dead start 2024-01-27T02:43:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.765+0000 7f9cda4fe700 1 -- 172.21.15.7:0/1852723641 shutdown_connections 2024-01-27T02:43:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.765+0000 7f9cda4fe700 1 -- 172.21.15.7:0/1852723641 >> 172.21.15.7:0/1852723641 conn(0x7f9cd40cdc30 msgr2=0x7f9cd40ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.765+0000 7f9cda4fe700 1 -- 172.21.15.7:0/1852723641 shutdown_connections 2024-01-27T02:43:33.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.765+0000 7f9cda4fe700 1 -- 172.21.15.7:0/1852723641 wait complete. 2024-01-27T02:43:33.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.767+0000 7f9cda4fe700 1 Processor -- start 2024-01-27T02:43:33.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.767+0000 7f9cda4fe700 1 -- start start 2024-01-27T02:43:33.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.768+0000 7f9cda4fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cbc003a90 0x7f9cd41582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:33.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.768+0000 7f9cda4fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cd40b28f0 0x7f9cd40b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:33.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.768+0000 7f9cda4fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40b1060 0x7f9cd415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:33.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.768+0000 7f9cda4fe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd40b2460 con 0x7f9cbc003a90 2024-01-27T02:43:33.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.768+0000 7f9cda4fe700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd40b1850 con 0x7f9cd40b28f0 2024-01-27T02:43:33.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.768+0000 7f9cda4fe700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd40b1b30 con 0x7f9cd40b1060 2024-01-27T02:43:33.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.768+0000 7f9cd3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cbc003a90 0x7f9cd41582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:33.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.768+0000 7f9cd3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cbc003a90 0x7f9cd41582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34786/0 (socket says 172.21.15.7:34786) 2024-01-27T02:43:33.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.768+0000 7f9cd3fff700 1 -- 172.21.15.7:0/252727834 learned_addr learned my addr 172.21.15.7:0/252727834 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:33.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.768+0000 7f9cd8a9b700 1 --2- 172.21.15.7:0/252727834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40b1060 0x7f9cd415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:33.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.768+0000 7f9cd37fe700 1 --2- 172.21.15.7:0/252727834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cd40b28f0 0x7f9cd40b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:33.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.769+0000 7f9cd3fff700 1 -- 172.21.15.7:0/252727834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40b1060 msgr2=0x7f9cd415cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:33.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.769+0000 7f9cd3fff700 1 --2- 172.21.15.7:0/252727834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40b1060 0x7f9cd415cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.769+0000 7f9cd3fff700 1 -- 172.21.15.7:0/252727834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cd40b28f0 msgr2=0x7f9cd40b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:33.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.769+0000 7f9cd3fff700 1 --2- 172.21.15.7:0/252727834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cd40b28f0 0x7f9cd40b0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.769+0000 7f9cd3fff700 1 -- 172.21.15.7:0/252727834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd40b1e10 con 0x7f9cbc003a90 2024-01-27T02:43:33.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.769+0000 7f9cd8a9b700 1 --2- 172.21.15.7:0/252727834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40b1060 0x7f9cd415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:33.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.769+0000 7f9cd3fff700 1 --2- 172.21.15.7:0/252727834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cbc003a90 0x7f9cd41582b0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4011000 tx=0x7f9cc4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:33.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.769+0000 7f9cbbfff700 1 -- 172.21.15.7:0/252727834 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc400ce90 con 0x7f9cbc003a90 2024-01-27T02:43:33.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.769+0000 7f9cbbfff700 1 -- 172.21.15.7:0/252727834 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc40072e0 con 0x7f9cbc003a90 2024-01-27T02:43:33.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.769+0000 7f9cbbfff700 1 -- 172.21.15.7:0/252727834 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40076a0 con 0x7f9cbc003a90 2024-01-27T02:43:33.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.769+0000 7f9cda4fe700 1 -- 172.21.15.7:0/252727834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd40b2460 con 0x7f9cbc003a90 2024-01-27T02:43:33.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.769+0000 7f9cda4fe700 1 -- 172.21.15.7:0/252727834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd41587f0 con 0x7f9cbc003a90 2024-01-27T02:43:33.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.770+0000 7f9cda4fe700 1 -- 172.21.15.7:0/252727834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd404c710 con 0x7f9cbc003a90 2024-01-27T02:43:33.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.770+0000 7f9cbbfff700 1 -- 172.21.15.7:0/252727834 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9cc4019030 con 0x7f9cbc003a90 2024-01-27T02:43:33.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.771+0000 7f9cbbfff700 1 --2- 172.21.15.7:0/252727834 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9cc0066f30 0x7f9cc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:33.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.771+0000 7f9cbbfff700 1 -- 172.21.15.7:0/252727834 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9cc4085c10 con 0x7f9cbc003a90 2024-01-27T02:43:33.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.771+0000 7f9cd37fe700 1 --2- 172.21.15.7:0/252727834 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9cc0066f30 0x7f9cc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:33.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.772+0000 7f9cd37fe700 1 --2- 172.21.15.7:0/252727834 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9cc0066f30 0x7f9cc00693e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9cd416d710 tx=0x7f9cc80091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:33.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.775+0000 7f9cbbfff700 1 -- 172.21.15.7:0/252727834 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9cc4052e80 con 0x7f9cbc003a90 2024-01-27T02:43:33.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.980+0000 7f9cda4fe700 1 -- 172.21.15.7:0/252727834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9cd40cd1b0 con 0x7f9cbc003a90 2024-01-27T02:43:33.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.981+0000 7f9cbbfff700 1 -- 172.21.15.7:0/252727834 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v180) v1 ==== 74+0+15317 (secure 0 0 0) 0x7f9cc40564a0 con 0x7f9cbc003a90 2024-01-27T02:43:33.982 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:33.982 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":180,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:43:30.932396+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:43:27.492968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"179","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-01-27T02:43:28.579977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:43:33.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.984+0000 7f9cda4fe700 1 -- 172.21.15.7:0/252727834 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9cc0066f30 msgr2=0x7f9cc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:33.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.984+0000 7f9cda4fe700 1 --2- 172.21.15.7:0/252727834 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9cc0066f30 0x7f9cc00693e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9cd416d710 tx=0x7f9cc80091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.984+0000 7f9cda4fe700 1 -- 172.21.15.7:0/252727834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cbc003a90 msgr2=0x7f9cd41582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:33.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.984+0000 7f9cda4fe700 1 --2- 172.21.15.7:0/252727834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cbc003a90 0x7f9cd41582b0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4011000 tx=0x7f9cc4015000 comp rx=0 tx=0).stop 2024-01-27T02:43:33.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.984+0000 7f9cda4fe700 1 -- 172.21.15.7:0/252727834 shutdown_connections 2024-01-27T02:43:33.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.984+0000 7f9cda4fe700 1 --2- 172.21.15.7:0/252727834 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9cc0066f30 0x7f9cc00693e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.984+0000 7f9cda4fe700 1 --2- 172.21.15.7:0/252727834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40b1060 0x7f9cd415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.984+0000 7f9cda4fe700 1 --2- 172.21.15.7:0/252727834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cbc003a90 0x7f9cd41582b0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.984+0000 7f9cda4fe700 1 --2- 172.21.15.7:0/252727834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cd40b28f0 0x7f9cd40b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:33.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.984+0000 7f9cda4fe700 1 -- 172.21.15.7:0/252727834 >> 172.21.15.7:0/252727834 conn(0x7f9cd40cdc30 msgr2=0x7f9cd40ab4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:33.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.984+0000 7f9cda4fe700 1 -- 172.21.15.7:0/252727834 shutdown_connections 2024-01-27T02:43:33.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:33.984+0000 7f9cda4fe700 1 -- 172.21.15.7:0/252727834 wait complete. 2024-01-27T02:43:34.000 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-27T02:43:34.000 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:43:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.132+0000 7fa7d8965700 1 Processor -- start 2024-01-27T02:43:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.132+0000 7fa7d8965700 1 -- start start 2024-01-27T02:43:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.133+0000 7fa7d8965700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7d4063c90 0x7fa7d40c8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.133+0000 7fa7d8965700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa7d41438f0 0x7fa7d40c8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.133+0000 7fa7d8965700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d40cb7a0 con 0x7fa7d4143520 2024-01-27T02:43:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.133+0000 7fa7d8965700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d40c8c90 con 0x7fa7d41438f0 2024-01-27T02:43:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.133+0000 7fa7d8965700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d40cb9f0 con 0x7fa7d4063c90 2024-01-27T02:43:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.133+0000 7fa7d2d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa7d41438f0 0x7fa7d40c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.133+0000 7fa7d259c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa7d4143520 0x7fa7d4063580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38502/0 (socket says 172.21.15.7:38502) 2024-01-27T02:43:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.133+0000 7fa7d259c700 1 -- 172.21.15.7:0/1207272708 learned_addr learned my addr 172.21.15.7:0/1207272708 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.133+0000 7fa7d1d9b700 1 --2- 172.21.15.7:0/1207272708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7d4063c90 0x7fa7d40c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.133+0000 7fa7d2d9d700 1 -- 172.21.15.7:0/1207272708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7d4063c90 msgr2=0x7fa7d40c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.133+0000 7fa7d2d9d700 1 --2- 172.21.15.7:0/1207272708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7d4063c90 0x7fa7d40c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.134+0000 7fa7d2d9d700 1 -- 172.21.15.7:0/1207272708 >> v1:172.21.15.7:6789/0 conn(0x7fa7d4143520 legacy=0x7fa7d4063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.134+0000 7fa7d2d9d700 1 -- 172.21.15.7:0/1207272708 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d40cbcd0 con 0x7fa7d41438f0 2024-01-27T02:43:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.134+0000 7fa7d159a700 1 -- 172.21.15.7:0/1207272708 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826864629 0 0) 0x7fa7d40cb7a0 con 0x7fa7d4143520 2024-01-27T02:43:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.134+0000 7fa7d2d9d700 1 --2- 172.21.15.7:0/1207272708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7d41438f0 0x7fa7d40c8750 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009e60 tx=0x7fa7cc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca81cce6f58bd7c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.134+0000 7fa7d159a700 1 -- 172.21.15.7:0/1207272708 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc00e070 con 0x7fa7d41438f0 2024-01-27T02:43:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.134+0000 7fa7d159a700 1 -- 172.21.15.7:0/1207272708 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7cc00bb60 con 0x7fa7d41438f0 2024-01-27T02:43:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.134+0000 7fa7d159a700 1 -- 172.21.15.7:0/1207272708 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc012430 con 0x7fa7d41438f0 2024-01-27T02:43:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.134+0000 7fa7d8965700 1 -- 172.21.15.7:0/1207272708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7d41438f0 msgr2=0x7fa7d40c8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.134+0000 7fa7d8965700 1 --2- 172.21.15.7:0/1207272708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7d41438f0 0x7fa7d40c8750 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009e60 tx=0x7fa7cc009780 comp rx=0 tx=0).stop 2024-01-27T02:43:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.135+0000 7fa7d8965700 1 -- 172.21.15.7:0/1207272708 shutdown_connections 2024-01-27T02:43:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.135+0000 7fa7d8965700 1 --2- 172.21.15.7:0/1207272708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7d41438f0 0x7fa7d40c8750 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.135+0000 7fa7d8965700 1 --2- 172.21.15.7:0/1207272708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7d4063c90 0x7fa7d40c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.135+0000 7fa7d8965700 1 -- 172.21.15.7:0/1207272708 >> 172.21.15.7:0/1207272708 conn(0x7fa7d40cdbd0 msgr2=0x7fa7d40cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.135+0000 7fa7d8965700 1 -- 172.21.15.7:0/1207272708 shutdown_connections 2024-01-27T02:43:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.135+0000 7fa7d8965700 1 -- 172.21.15.7:0/1207272708 wait complete. 2024-01-27T02:43:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.136+0000 7fa7d8965700 1 Processor -- start 2024-01-27T02:43:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.136+0000 7fa7d8965700 1 -- start start 2024-01-27T02:43:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.137+0000 7fa7d8965700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7d4063c90 0x7fa7d415a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.137+0000 7fa7d8965700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7d4143520 0x7fa7d41419f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.137+0000 7fa7d8965700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7d41438f0 0x7fa7d415d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.137+0000 7fa7d8965700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d40c5fc0 con 0x7fa7d4143520 2024-01-27T02:43:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.137+0000 7fa7d8965700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d40aeaf0 con 0x7fa7d41438f0 2024-01-27T02:43:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.137+0000 7fa7d8965700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d4141f30 con 0x7fa7d4063c90 2024-01-27T02:43:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.137+0000 7fa7d1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7d4143520 0x7fa7d41419f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.137+0000 7fa7d1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7d4143520 0x7fa7d41419f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34804/0 (socket says 172.21.15.7:34804) 2024-01-27T02:43:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.137+0000 7fa7d1d9b700 1 -- 172.21.15.7:0/416508170 learned_addr learned my addr 172.21.15.7:0/416508170 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.137+0000 7fa7d259c700 1 --2- 172.21.15.7:0/416508170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7d4063c90 0x7fa7d415a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.137+0000 7fa7d2d9d700 1 --2- 172.21.15.7:0/416508170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7d41438f0 0x7fa7d415d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.137+0000 7fa7d1d9b700 1 -- 172.21.15.7:0/416508170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7d4063c90 msgr2=0x7fa7d415a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.137+0000 7fa7d1d9b700 1 --2- 172.21.15.7:0/416508170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7d4063c90 0x7fa7d415a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.137+0000 7fa7d1d9b700 1 -- 172.21.15.7:0/416508170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7d41438f0 msgr2=0x7fa7d415d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.137+0000 7fa7d1d9b700 1 --2- 172.21.15.7:0/416508170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7d41438f0 0x7fa7d415d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.137+0000 7fa7d1d9b700 1 -- 172.21.15.7:0/416508170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7cc009a20 con 0x7fa7d4143520 2024-01-27T02:43:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.137+0000 7fa7d259c700 1 --2- 172.21.15.7:0/416508170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7d4063c90 0x7fa7d415a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.137+0000 7fa7d1d9b700 1 --2- 172.21.15.7:0/416508170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7d4143520 0x7fa7d41419f0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800ec80 tx=0x7fa7c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.137+0000 7fa7c27fc700 1 -- 172.21.15.7:0/416508170 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c800c9d0 con 0x7fa7d4143520 2024-01-27T02:43:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.137+0000 7fa7c27fc700 1 -- 172.21.15.7:0/416508170 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c800cb30 con 0x7fa7d4143520 2024-01-27T02:43:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.137+0000 7fa7c27fc700 1 -- 172.21.15.7:0/416508170 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8013470 con 0x7fa7d4143520 2024-01-27T02:43:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.137+0000 7fa7d8965700 1 -- 172.21.15.7:0/416508170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d40c5a00 con 0x7fa7d4143520 2024-01-27T02:43:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.138+0000 7fa7d8965700 1 -- 172.21.15.7:0/416508170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d40ae780 con 0x7fa7d4143520 2024-01-27T02:43:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.138+0000 7fa7d8965700 1 -- 172.21.15.7:0/416508170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d40aeca0 con 0x7fa7d4143520 2024-01-27T02:43:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.139+0000 7fa7c27fc700 1 -- 172.21.15.7:0/416508170 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa7c8013860 con 0x7fa7d4143520 2024-01-27T02:43:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.139+0000 7fa7c27fc700 1 --2- 172.21.15.7:0/416508170 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa7b4066f30 0x7fa7b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.140+0000 7fa7c27fc700 1 -- 172.21.15.7:0/416508170 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa7c80848b0 con 0x7fa7d4143520 2024-01-27T02:43:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.140+0000 7fa7d259c700 1 --2- 172.21.15.7:0/416508170 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa7b4066f30 0x7fa7b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.140+0000 7fa7d259c700 1 --2- 172.21.15.7:0/416508170 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa7b4066f30 0x7fa7b40693e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4001360 tx=0x7fa7c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.144+0000 7fa7c27fc700 1 -- 172.21.15.7:0/416508170 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7d40aeca0 con 0x7fa7d4143520 2024-01-27T02:43:34.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.343+0000 7fa7d8965700 1 -- 172.21.15.7:0/416508170 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa7d40aeca0 con 0x7fa7b4066f30 2024-01-27T02:43:34.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.352+0000 7fa7c27fc700 1 -- 172.21.15.7:0/416508170 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285590 (secure 0 0 0) 0x7fa7d40aeca0 con 0x7fa7b4066f30 2024-01-27T02:43:34.356 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:34.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.355+0000 7fa7d8965700 1 -- 172.21.15.7:0/416508170 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa7b4066f30 msgr2=0x7fa7b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:34.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.355+0000 7fa7d8965700 1 --2- 172.21.15.7:0/416508170 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa7b4066f30 0x7fa7b40693e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4001360 tx=0x7fa7c4009040 comp rx=0 tx=0).stop 2024-01-27T02:43:34.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.355+0000 7fa7d8965700 1 -- 172.21.15.7:0/416508170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7d4143520 msgr2=0x7fa7d41419f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:34.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.355+0000 7fa7d8965700 1 --2- 172.21.15.7:0/416508170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7d4143520 0x7fa7d41419f0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800ec80 tx=0x7fa7c80091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.355+0000 7fa7d8965700 1 -- 172.21.15.7:0/416508170 shutdown_connections 2024-01-27T02:43:34.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.355+0000 7fa7d8965700 1 --2- 172.21.15.7:0/416508170 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa7b4066f30 0x7fa7b40693e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.355+0000 7fa7d8965700 1 --2- 172.21.15.7:0/416508170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7d41438f0 0x7fa7d415d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.355+0000 7fa7d8965700 1 --2- 172.21.15.7:0/416508170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7d4063c90 0x7fa7d415a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.355+0000 7fa7d8965700 1 --2- 172.21.15.7:0/416508170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7d4143520 0x7fa7d41419f0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.355+0000 7fa7d8965700 1 -- 172.21.15.7:0/416508170 >> 172.21.15.7:0/416508170 conn(0x7fa7d40cdbd0 msgr2=0x7fa7d40ab660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:34.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.355+0000 7fa7d8965700 1 -- 172.21.15.7:0/416508170 shutdown_connections 2024-01-27T02:43:34.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.355+0000 7fa7d8965700 1 -- 172.21.15.7:0/416508170 wait complete. 2024-01-27T02:43:34.359 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:43:34.369 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":519,"stamp":"2024-01-27T02:43:33.742806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8979,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553520,"statfs":{"total":1600000000,"available":1590804860,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.579182"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.946170+0000","last_change":"2024-01-27T02:43:29.946073+0000","last_active":"2024-01-27T02:43:29.946170+0000","last_peered":"2024-01-27T02:43:29.946170+0000","last_clean":"2024-01-27T02:43:29.946170+0000","last_became_active":"2024-01-27T02:43:29.944886+0000","last_became_peered":"2024-01-27T02:43:29.944886+0000","last_unstale":"2024-01-27T02:43:29.946170+0000","last_undegraded":"2024-01-27T02:43:29.946170+0000","last_fullsized":"2024-01-27T02:43:29.946170+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:06:22.906823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.944105+0000","last_change":"2024-01-27T02:43:29.944039+0000","last_active":"2024-01-27T02:43:29.944105+0000","last_peered":"2024-01-27T02:43:29.944105+0000","last_clean":"2024-01-27T02:43:29.944105+0000","last_became_active":"2024-01-27T02:43:29.941027+0000","last_became_peered":"2024-01-27T02:43:29.941027+0000","last_unstale":"2024-01-27T02:43:29.944105+0000","last_undegraded":"2024-01-27T02:43:29.944105+0000","last_fullsized":"2024-01-27T02:43:29.944105+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:14:25.190169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.944952+0000","last_change":"2024-01-27T02:43:29.944069+0000","last_active":"2024-01-27T02:43:29.944952+0000","last_peered":"2024-01-27T02:43:29.944952+0000","last_clean":"2024-01-27T02:43:29.944952+0000","last_became_active":"2024-01-27T02:43:29.943641+0000","last_became_peered":"2024-01-27T02:43:29.943641+0000","last_unstale":"2024-01-27T02:43:29.944952+0000","last_undegraded":"2024-01-27T02:43:29.944952+0000","last_fullsized":"2024-01-27T02:43:29.944952+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:53:52.641917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.948729+0000","last_change":"2024-01-27T02:43:29.947964+0000","last_active":"2024-01-27T02:43:29.948729+0000","last_peered":"2024-01-27T02:43:29.948729+0000","last_clean":"2024-01-27T02:43:29.948729+0000","last_became_active":"2024-01-27T02:43:29.947592+0000","last_became_peered":"2024-01-27T02:43:29.947592+0000","last_unstale":"2024-01-27T02:43:29.948729+0000","last_undegraded":"2024-01-27T02:43:29.948729+0000","last_fullsized":"2024-01-27T02:43:29.948729+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:59:55.931719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.939097+0000","last_change":"2024-01-27T02:43:29.938413+0000","last_active":"2024-01-27T02:43:29.939097+0000","last_peered":"2024-01-27T02:43:29.939097+0000","last_clean":"2024-01-27T02:43:29.939097+0000","last_became_active":"2024-01-27T02:43:29.938165+0000","last_became_peered":"2024-01-27T02:43:29.938165+0000","last_unstale":"2024-01-27T02:43:29.939097+0000","last_undegraded":"2024-01-27T02:43:29.939097+0000","last_fullsized":"2024-01-27T02:43:29.939097+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:32:52.100147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.952297+0000","last_change":"2024-01-27T02:43:29.952223+0000","last_active":"2024-01-27T02:43:29.952297+0000","last_peered":"2024-01-27T02:43:29.952297+0000","last_clean":"2024-01-27T02:43:29.952297+0000","last_became_active":"2024-01-27T02:43:29.951943+0000","last_became_peered":"2024-01-27T02:43:29.951943+0000","last_unstale":"2024-01-27T02:43:29.952297+0000","last_undegraded":"2024-01-27T02:43:29.952297+0000","last_fullsized":"2024-01-27T02:43:29.952297+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:37:09.277025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.954144+0000","last_change":"2024-01-27T02:43:29.954098+0000","last_active":"2024-01-27T02:43:29.954144+0000","last_peered":"2024-01-27T02:43:29.954144+0000","last_clean":"2024-01-27T02:43:29.954144+0000","last_became_active":"2024-01-27T02:43:29.952882+0000","last_became_peered":"2024-01-27T02:43:29.952882+0000","last_unstale":"2024-01-27T02:43:29.954144+0000","last_undegraded":"2024-01-27T02:43:29.954144+0000","last_fullsized":"2024-01-27T02:43:29.954144+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:50:52.055538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.939116+0000","last_change":"2024-01-27T02:43:29.938435+0000","last_active":"2024-01-27T02:43:29.939116+0000","last_peered":"2024-01-27T02:43:29.939116+0000","last_clean":"2024-01-27T02:43:29.939116+0000","last_became_active":"2024-01-27T02:43:29.938195+0000","last_became_peered":"2024-01-27T02:43:29.938195+0000","last_unstale":"2024-01-27T02:43:29.939116+0000","last_undegraded":"2024-01-27T02:43:29.939116+0000","last_fullsized":"2024-01-27T02:43:29.939116+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:06.053165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.945800+0000","last_change":"2024-01-27T02:43:29.945741+0000","last_active":"2024-01-27T02:43:29.945800+0000","last_peered":"2024-01-27T02:43:29.945800+0000","last_clean":"2024-01-27T02:43:29.945800+0000","last_became_active":"2024-01-27T02:43:29.945356+0000","last_became_peered":"2024-01-27T02:43:29.945356+0000","last_unstale":"2024-01-27T02:43:29.945800+0000","last_undegraded":"2024-01-27T02:43:29.945800+0000","last_fullsized":"2024-01-27T02:43:29.945800+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:05:46.859143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.942953+0000","last_change":"2024-01-27T02:43:29.942807+0000","last_active":"2024-01-27T02:43:29.942953+0000","last_peered":"2024-01-27T02:43:29.942953+0000","last_clean":"2024-01-27T02:43:29.942953+0000","last_became_active":"2024-01-27T02:43:29.942529+0000","last_became_peered":"2024-01-27T02:43:29.942529+0000","last_unstale":"2024-01-27T02:43:29.942953+0000","last_undegraded":"2024-01-27T02:43:29.942953+0000","last_fullsized":"2024-01-27T02:43:29.942953+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:55:12.514206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.939598+0000","last_change":"2024-01-27T02:43:29.939520+0000","last_active":"2024-01-27T02:43:29.939598+0000","last_peered":"2024-01-27T02:43:29.939598+0000","last_clean":"2024-01-27T02:43:29.939598+0000","last_became_active":"2024-01-27T02:43:29.939290+0000","last_became_peered":"2024-01-27T02:43:29.939290+0000","last_unstale":"2024-01-27T02:43:29.939598+0000","last_undegraded":"2024-01-27T02:43:29.939598+0000","last_fullsized":"2024-01-27T02:43:29.939598+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:59:55.931719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.944869+0000","last_change":"2024-01-27T02:43:29.944691+0000","last_active":"2024-01-27T02:43:29.944869+0000","last_peered":"2024-01-27T02:43:29.944869+0000","last_clean":"2024-01-27T02:43:29.944869+0000","last_became_active":"2024-01-27T02:43:29.941386+0000","last_became_peered":"2024-01-27T02:43:29.941386+0000","last_unstale":"2024-01-27T02:43:29.944869+0000","last_undegraded":"2024-01-27T02:43:29.944869+0000","last_fullsized":"2024-01-27T02:43:29.944869+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:59:20.853498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.943197+0000","last_change":"2024-01-27T02:43:29.943130+0000","last_active":"2024-01-27T02:43:29.943197+0000","last_peered":"2024-01-27T02:43:29.943197+0000","last_clean":"2024-01-27T02:43:29.943197+0000","last_became_active":"2024-01-27T02:43:29.942896+0000","last_became_peered":"2024-01-27T02:43:29.942896+0000","last_unstale":"2024-01-27T02:43:29.943197+0000","last_undegraded":"2024-01-27T02:43:29.943197+0000","last_fullsized":"2024-01-27T02:43:29.943197+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:20:23.793438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.943143+0000","last_change":"2024-01-27T02:43:29.942426+0000","last_active":"2024-01-27T02:43:29.943143+0000","last_peered":"2024-01-27T02:43:29.943143+0000","last_clean":"2024-01-27T02:43:29.943143+0000","last_became_active":"2024-01-27T02:43:29.941598+0000","last_became_peered":"2024-01-27T02:43:29.941598+0000","last_unstale":"2024-01-27T02:43:29.943143+0000","last_undegraded":"2024-01-27T02:43:29.943143+0000","last_fullsized":"2024-01-27T02:43:29.943143+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:57:16.971601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.943081+0000","last_change":"2024-01-27T02:43:29.943008+0000","last_active":"2024-01-27T02:43:29.943081+0000","last_peered":"2024-01-27T02:43:29.943081+0000","last_clean":"2024-01-27T02:43:29.943081+0000","last_became_active":"2024-01-27T02:43:29.942776+0000","last_became_peered":"2024-01-27T02:43:29.942776+0000","last_unstale":"2024-01-27T02:43:29.943081+0000","last_undegraded":"2024-01-27T02:43:29.943081+0000","last_fullsized":"2024-01-27T02:43:29.943081+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:50:52.055538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.952834+0000","last_change":"2024-01-27T02:43:29.952299+0000","last_active":"2024-01-27T02:43:29.952834+0000","last_peered":"2024-01-27T02:43:29.952834+0000","last_clean":"2024-01-27T02:43:29.952834+0000","last_became_active":"2024-01-27T02:43:29.951352+0000","last_became_peered":"2024-01-27T02:43:29.951352+0000","last_unstale":"2024-01-27T02:43:29.952834+0000","last_undegraded":"2024-01-27T02:43:29.952834+0000","last_fullsized":"2024-01-27T02:43:29.952834+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:35:53.347541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.946553+0000","last_change":"2024-01-27T02:43:29.946151+0000","last_active":"2024-01-27T02:43:29.946553+0000","last_peered":"2024-01-27T02:43:29.946553+0000","last_clean":"2024-01-27T02:43:29.946553+0000","last_became_active":"2024-01-27T02:43:29.944510+0000","last_became_peered":"2024-01-27T02:43:29.944510+0000","last_unstale":"2024-01-27T02:43:29.946553+0000","last_undegraded":"2024-01-27T02:43:29.946553+0000","last_fullsized":"2024-01-27T02:43:29.946553+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:51:08.617993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.949322+0000","last_change":"2024-01-27T02:43:29.949245+0000","last_active":"2024-01-27T02:43:29.949322+0000","last_peered":"2024-01-27T02:43:29.949322+0000","last_clean":"2024-01-27T02:43:29.949322+0000","last_became_active":"2024-01-27T02:43:29.948964+0000","last_became_peered":"2024-01-27T02:43:29.948964+0000","last_unstale":"2024-01-27T02:43:29.949322+0000","last_undegraded":"2024-01-27T02:43:29.949322+0000","last_fullsized":"2024-01-27T02:43:29.949322+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:13:09.260726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.943932+0000","last_change":"2024-01-27T02:43:29.941796+0000","last_active":"2024-01-27T02:43:29.943932+0000","last_peered":"2024-01-27T02:43:29.943932+0000","last_clean":"2024-01-27T02:43:29.943932+0000","last_became_active":"2024-01-27T02:43:29.941526+0000","last_became_peered":"2024-01-27T02:43:29.941526+0000","last_unstale":"2024-01-27T02:43:29.943932+0000","last_undegraded":"2024-01-27T02:43:29.943932+0000","last_fullsized":"2024-01-27T02:43:29.943932+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:51:14.634758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.948614+0000","last_change":"2024-01-27T02:43:29.948543+0000","last_active":"2024-01-27T02:43:29.948614+0000","last_peered":"2024-01-27T02:43:29.948614+0000","last_clean":"2024-01-27T02:43:29.948614+0000","last_became_active":"2024-01-27T02:43:29.948330+0000","last_became_peered":"2024-01-27T02:43:29.948330+0000","last_unstale":"2024-01-27T02:43:29.948614+0000","last_undegraded":"2024-01-27T02:43:29.948614+0000","last_fullsized":"2024-01-27T02:43:29.948614+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:51:01.041063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.946535+0000","last_change":"2024-01-27T02:43:29.946064+0000","last_active":"2024-01-27T02:43:29.946535+0000","last_peered":"2024-01-27T02:43:29.946535+0000","last_clean":"2024-01-27T02:43:29.946535+0000","last_became_active":"2024-01-27T02:43:29.945781+0000","last_became_peered":"2024-01-27T02:43:29.945781+0000","last_unstale":"2024-01-27T02:43:29.946535+0000","last_undegraded":"2024-01-27T02:43:29.946535+0000","last_fullsized":"2024-01-27T02:43:29.946535+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:06.346984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.953333+0000","last_change":"2024-01-27T02:43:29.953282+0000","last_active":"2024-01-27T02:43:29.953333+0000","last_peered":"2024-01-27T02:43:29.953333+0000","last_clean":"2024-01-27T02:43:29.953333+0000","last_became_active":"2024-01-27T02:43:29.953013+0000","last_became_peered":"2024-01-27T02:43:29.953013+0000","last_unstale":"2024-01-27T02:43:29.953333+0000","last_undegraded":"2024-01-27T02:43:29.953333+0000","last_fullsized":"2024-01-27T02:43:29.953333+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:56:40.433794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.947795+0000","last_change":"2024-01-27T02:43:29.947691+0000","last_active":"2024-01-27T02:43:29.947795+0000","last_peered":"2024-01-27T02:43:29.947795+0000","last_clean":"2024-01-27T02:43:29.947795+0000","last_became_active":"2024-01-27T02:43:29.944481+0000","last_became_peered":"2024-01-27T02:43:29.944481+0000","last_unstale":"2024-01-27T02:43:29.947795+0000","last_undegraded":"2024-01-27T02:43:29.947795+0000","last_fullsized":"2024-01-27T02:43:29.947795+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:02:36.123442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.945657+0000","last_change":"2024-01-27T02:43:29.945592+0000","last_active":"2024-01-27T02:43:29.945657+0000","last_peered":"2024-01-27T02:43:29.945657+0000","last_clean":"2024-01-27T02:43:29.945657+0000","last_became_active":"2024-01-27T02:43:29.945047+0000","last_became_peered":"2024-01-27T02:43:29.945047+0000","last_unstale":"2024-01-27T02:43:29.945657+0000","last_undegraded":"2024-01-27T02:43:29.945657+0000","last_fullsized":"2024-01-27T02:43:29.945657+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:05:31.630090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.948187+0000","last_change":"2024-01-27T02:43:29.948120+0000","last_active":"2024-01-27T02:43:29.948187+0000","last_peered":"2024-01-27T02:43:29.948187+0000","last_clean":"2024-01-27T02:43:29.948187+0000","last_became_active":"2024-01-27T02:43:29.947476+0000","last_became_peered":"2024-01-27T02:43:29.947476+0000","last_unstale":"2024-01-27T02:43:29.948187+0000","last_undegraded":"2024-01-27T02:43:29.948187+0000","last_fullsized":"2024-01-27T02:43:29.948187+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:54:50.767945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.952778+0000","last_change":"2024-01-27T02:43:29.951674+0000","last_active":"2024-01-27T02:43:29.952778+0000","last_peered":"2024-01-27T02:43:29.952778+0000","last_clean":"2024-01-27T02:43:29.952778+0000","last_became_active":"2024-01-27T02:43:29.950652+0000","last_became_peered":"2024-01-27T02:43:29.950652+0000","last_unstale":"2024-01-27T02:43:29.952778+0000","last_undegraded":"2024-01-27T02:43:29.952778+0000","last_fullsized":"2024-01-27T02:43:29.952778+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:33:01.072368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.942218+0000","last_change":"2024-01-27T02:43:29.941960+0000","last_active":"2024-01-27T02:43:29.942218+0000","last_peered":"2024-01-27T02:43:29.942218+0000","last_clean":"2024-01-27T02:43:29.942218+0000","last_became_active":"2024-01-27T02:43:29.941698+0000","last_became_peered":"2024-01-27T02:43:29.941698+0000","last_unstale":"2024-01-27T02:43:29.942218+0000","last_undegraded":"2024-01-27T02:43:29.942218+0000","last_fullsized":"2024-01-27T02:43:29.942218+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:01:20.145502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.947820+0000","last_change":"2024-01-27T02:43:29.947751+0000","last_active":"2024-01-27T02:43:29.947820+0000","last_peered":"2024-01-27T02:43:29.947820+0000","last_clean":"2024-01-27T02:43:29.947820+0000","last_became_active":"2024-01-27T02:43:29.947488+0000","last_became_peered":"2024-01-27T02:43:29.947488+0000","last_unstale":"2024-01-27T02:43:29.947820+0000","last_undegraded":"2024-01-27T02:43:29.947820+0000","last_fullsized":"2024-01-27T02:43:29.947820+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:47:39.445834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.943164+0000","last_change":"2024-01-27T02:43:29.942600+0000","last_active":"2024-01-27T02:43:29.943164+0000","last_peered":"2024-01-27T02:43:29.943164+0000","last_clean":"2024-01-27T02:43:29.943164+0000","last_became_active":"2024-01-27T02:43:29.941968+0000","last_became_peered":"2024-01-27T02:43:29.941968+0000","last_unstale":"2024-01-27T02:43:29.943164+0000","last_undegraded":"2024-01-27T02:43:29.943164+0000","last_fullsized":"2024-01-27T02:43:29.943164+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:24:44.341787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.940875+0000","last_change":"2024-01-27T02:43:29.940668+0000","last_active":"2024-01-27T02:43:29.940875+0000","last_peered":"2024-01-27T02:43:29.940875+0000","last_clean":"2024-01-27T02:43:29.940875+0000","last_became_active":"2024-01-27T02:43:29.940434+0000","last_became_peered":"2024-01-27T02:43:29.940434+0000","last_unstale":"2024-01-27T02:43:29.940875+0000","last_undegraded":"2024-01-27T02:43:29.940875+0000","last_fullsized":"2024-01-27T02:43:29.940875+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:59:20.853498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.951047+0000","last_change":"2024-01-27T02:43:29.950981+0000","last_active":"2024-01-27T02:43:29.951047+0000","last_peered":"2024-01-27T02:43:29.951047+0000","last_clean":"2024-01-27T02:43:29.951047+0000","last_became_active":"2024-01-27T02:43:29.950782+0000","last_became_peered":"2024-01-27T02:43:29.950782+0000","last_unstale":"2024-01-27T02:43:29.951047+0000","last_undegraded":"2024-01-27T02:43:29.951047+0000","last_fullsized":"2024-01-27T02:43:29.951047+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:56:40.433794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.951117+0000","last_change":"2024-01-27T02:43:29.951001+0000","last_active":"2024-01-27T02:43:29.951117+0000","last_peered":"2024-01-27T02:43:29.951117+0000","last_clean":"2024-01-27T02:43:29.951117+0000","last_became_active":"2024-01-27T02:43:29.950475+0000","last_became_peered":"2024-01-27T02:43:29.950475+0000","last_unstale":"2024-01-27T02:43:29.951117+0000","last_undegraded":"2024-01-27T02:43:29.951117+0000","last_fullsized":"2024-01-27T02:43:29.951117+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:45:07.481390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.949935+0000","last_change":"2024-01-27T02:43:29.949661+0000","last_active":"2024-01-27T02:43:29.949935+0000","last_peered":"2024-01-27T02:43:29.949935+0000","last_clean":"2024-01-27T02:43:29.949935+0000","last_became_active":"2024-01-27T02:43:29.945212+0000","last_became_peered":"2024-01-27T02:43:29.945212+0000","last_unstale":"2024-01-27T02:43:29.949935+0000","last_undegraded":"2024-01-27T02:43:29.949935+0000","last_fullsized":"2024-01-27T02:43:29.949935+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:26:47.425780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"179'4","reported_seq":14,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.193267+0000","last_change":"2024-01-27T02:43:29.180404+0000","last_active":"2024-01-27T02:43:30.193267+0000","last_peered":"2024-01-27T02:43:30.193267+0000","last_clean":"2024-01-27T02:43:30.193267+0000","last_became_active":"2024-01-27T02:43:29.180149+0000","last_became_peered":"2024-01-27T02:43:29.180149+0000","last_unstale":"2024-01-27T02:43:30.193267+0000","last_undegraded":"2024-01-27T02:43:30.193267+0000","last_fullsized":"2024-01-27T02:43:30.193267+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:12:22.590221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.937923+0000","last_change":"2024-01-27T02:43:29.937637+0000","last_active":"2024-01-27T02:43:29.937923+0000","last_peered":"2024-01-27T02:43:29.937923+0000","last_clean":"2024-01-27T02:43:29.937923+0000","last_became_active":"2024-01-27T02:43:29.936865+0000","last_became_peered":"2024-01-27T02:43:29.936865+0000","last_unstale":"2024-01-27T02:43:29.937923+0000","last_undegraded":"2024-01-27T02:43:29.937923+0000","last_fullsized":"2024-01-27T02:43:29.937923+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:55:12.514206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"179'1","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.192400+0000","last_change":"2024-01-27T02:43:29.184179+0000","last_active":"2024-01-27T02:43:30.192400+0000","last_peered":"2024-01-27T02:43:30.192400+0000","last_clean":"2024-01-27T02:43:30.192400+0000","last_became_active":"2024-01-27T02:43:29.183417+0000","last_became_peered":"2024-01-27T02:43:29.183417+0000","last_unstale":"2024-01-27T02:43:30.192400+0000","last_undegraded":"2024-01-27T02:43:30.192400+0000","last_fullsized":"2024-01-27T02:43:30.192400+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:45:48.808912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.948937+0000","last_change":"2024-01-27T02:43:29.948866+0000","last_active":"2024-01-27T02:43:29.948937+0000","last_peered":"2024-01-27T02:43:29.948937+0000","last_clean":"2024-01-27T02:43:29.948937+0000","last_became_active":"2024-01-27T02:43:29.948624+0000","last_became_peered":"2024-01-27T02:43:29.948624+0000","last_unstale":"2024-01-27T02:43:29.948937+0000","last_undegraded":"2024-01-27T02:43:29.948937+0000","last_fullsized":"2024-01-27T02:43:29.948937+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:46:09.582015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"179'2","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.193688+0000","last_change":"2024-01-27T02:43:29.179956+0000","last_active":"2024-01-27T02:43:30.193688+0000","last_peered":"2024-01-27T02:43:30.193688+0000","last_clean":"2024-01-27T02:43:30.193688+0000","last_became_active":"2024-01-27T02:43:29.179706+0000","last_became_peered":"2024-01-27T02:43:29.179706+0000","last_unstale":"2024-01-27T02:43:30.193688+0000","last_undegraded":"2024-01-27T02:43:30.193688+0000","last_fullsized":"2024-01-27T02:43:30.193688+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:20:22.787969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.947792+0000","last_change":"2024-01-27T02:43:29.947721+0000","last_active":"2024-01-27T02:43:29.947792+0000","last_peered":"2024-01-27T02:43:29.947792+0000","last_clean":"2024-01-27T02:43:29.947792+0000","last_became_active":"2024-01-27T02:43:29.946465+0000","last_became_peered":"2024-01-27T02:43:29.946465+0000","last_unstale":"2024-01-27T02:43:29.947792+0000","last_undegraded":"2024-01-27T02:43:29.947792+0000","last_fullsized":"2024-01-27T02:43:29.947792+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:45:49.814381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"179'2","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.192921+0000","last_change":"2024-01-27T02:43:29.177284+0000","last_active":"2024-01-27T02:43:30.192921+0000","last_peered":"2024-01-27T02:43:30.192921+0000","last_clean":"2024-01-27T02:43:30.192921+0000","last_became_active":"2024-01-27T02:43:29.177030+0000","last_became_peered":"2024-01-27T02:43:29.177030+0000","last_unstale":"2024-01-27T02:43:30.192921+0000","last_undegraded":"2024-01-27T02:43:30.192921+0000","last_fullsized":"2024-01-27T02:43:30.192921+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:01:19.140032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.949888+0000","last_change":"2024-01-27T02:43:29.947077+0000","last_active":"2024-01-27T02:43:29.949888+0000","last_peered":"2024-01-27T02:43:29.949888+0000","last_clean":"2024-01-27T02:43:29.949888+0000","last_became_active":"2024-01-27T02:43:29.946810+0000","last_became_peered":"2024-01-27T02:43:29.946810+0000","last_unstale":"2024-01-27T02:43:29.949888+0000","last_undegraded":"2024-01-27T02:43:29.949888+0000","last_fullsized":"2024-01-27T02:43:29.949888+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:55:59.427217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"179'1","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.191060+0000","last_change":"2024-01-27T02:43:29.174746+0000","last_active":"2024-01-27T02:43:30.191060+0000","last_peered":"2024-01-27T02:43:30.191060+0000","last_clean":"2024-01-27T02:43:30.191060+0000","last_became_active":"2024-01-27T02:43:29.174151+0000","last_became_peered":"2024-01-27T02:43:29.174151+0000","last_unstale":"2024-01-27T02:43:30.191060+0000","last_undegraded":"2024-01-27T02:43:30.191060+0000","last_fullsized":"2024-01-27T02:43:30.191060+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:37:08.271555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.953857+0000","last_change":"2024-01-27T02:43:29.953801+0000","last_active":"2024-01-27T02:43:29.953857+0000","last_peered":"2024-01-27T02:43:29.953857+0000","last_clean":"2024-01-27T02:43:29.953857+0000","last_became_active":"2024-01-27T02:43:29.953484+0000","last_became_peered":"2024-01-27T02:43:29.953484+0000","last_unstale":"2024-01-27T02:43:29.953857+0000","last_undegraded":"2024-01-27T02:43:29.953857+0000","last_fullsized":"2024-01-27T02:43:29.953857+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:06.053165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"179'1","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.195848+0000","last_change":"2024-01-27T02:43:29.171966+0000","last_active":"2024-01-27T02:43:30.195848+0000","last_peered":"2024-01-27T02:43:30.195848+0000","last_clean":"2024-01-27T02:43:30.195848+0000","last_became_active":"2024-01-27T02:43:29.171694+0000","last_became_peered":"2024-01-27T02:43:29.171694+0000","last_unstale":"2024-01-27T02:43:30.195848+0000","last_undegraded":"2024-01-27T02:43:30.195848+0000","last_fullsized":"2024-01-27T02:43:30.195848+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:53:51.636448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.943815+0000","last_change":"2024-01-27T02:43:29.943742+0000","last_active":"2024-01-27T02:43:29.943815+0000","last_peered":"2024-01-27T02:43:29.943815+0000","last_clean":"2024-01-27T02:43:29.943815+0000","last_became_active":"2024-01-27T02:43:29.943243+0000","last_became_peered":"2024-01-27T02:43:29.943243+0000","last_unstale":"2024-01-27T02:43:29.943815+0000","last_undegraded":"2024-01-27T02:43:29.943815+0000","last_fullsized":"2024-01-27T02:43:29.943815+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:43:49.450067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.935189+0000","last_change":"2024-01-27T02:43:29.179484+0000","last_active":"2024-01-27T02:43:29.935189+0000","last_peered":"2024-01-27T02:43:29.935189+0000","last_clean":"2024-01-27T02:43:29.935189+0000","last_became_active":"2024-01-27T02:43:29.179282+0000","last_became_peered":"2024-01-27T02:43:29.179282+0000","last_unstale":"2024-01-27T02:43:29.935189+0000","last_undegraded":"2024-01-27T02:43:29.935189+0000","last_fullsized":"2024-01-27T02:43:29.935189+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:57:15.966132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.952537+0000","last_change":"2024-01-27T02:43:29.952480+0000","last_active":"2024-01-27T02:43:29.952537+0000","last_peered":"2024-01-27T02:43:29.952537+0000","last_clean":"2024-01-27T02:43:29.952537+0000","last_became_active":"2024-01-27T02:43:29.952241+0000","last_became_peered":"2024-01-27T02:43:29.952241+0000","last_unstale":"2024-01-27T02:43:29.952537+0000","last_undegraded":"2024-01-27T02:43:29.952537+0000","last_fullsized":"2024-01-27T02:43:29.952537+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:04:34.634885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.938798+0000","last_change":"2024-01-27T02:43:29.175577+0000","last_active":"2024-01-27T02:43:29.938798+0000","last_peered":"2024-01-27T02:43:29.938798+0000","last_clean":"2024-01-27T02:43:29.938798+0000","last_became_active":"2024-01-27T02:43:29.175317+0000","last_became_peered":"2024-01-27T02:43:29.175317+0000","last_unstale":"2024-01-27T02:43:29.938798+0000","last_undegraded":"2024-01-27T02:43:29.938798+0000","last_fullsized":"2024-01-27T02:43:29.938798+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:05.047696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.947040+0000","last_change":"2024-01-27T02:43:29.946965+0000","last_active":"2024-01-27T02:43:29.947040+0000","last_peered":"2024-01-27T02:43:29.947040+0000","last_clean":"2024-01-27T02:43:29.947040+0000","last_became_active":"2024-01-27T02:43:29.946653+0000","last_became_peered":"2024-01-27T02:43:29.946653+0000","last_unstale":"2024-01-27T02:43:29.947040+0000","last_undegraded":"2024-01-27T02:43:29.947040+0000","last_fullsized":"2024-01-27T02:43:29.947040+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:44:05.288565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.944258+0000","last_change":"2024-01-27T02:43:29.176200+0000","last_active":"2024-01-27T02:43:29.944258+0000","last_peered":"2024-01-27T02:43:29.944258+0000","last_clean":"2024-01-27T02:43:29.944258+0000","last_became_active":"2024-01-27T02:43:29.175549+0000","last_became_peered":"2024-01-27T02:43:29.175549+0000","last_unstale":"2024-01-27T02:43:29.944258+0000","last_undegraded":"2024-01-27T02:43:29.944258+0000","last_fullsized":"2024-01-27T02:43:29.944258+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:47:25.883035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.946812+0000","last_change":"2024-01-27T02:43:29.946694+0000","last_active":"2024-01-27T02:43:29.946812+0000","last_peered":"2024-01-27T02:43:29.946812+0000","last_clean":"2024-01-27T02:43:29.946812+0000","last_became_active":"2024-01-27T02:43:29.946386+0000","last_became_peered":"2024-01-27T02:43:29.946386+0000","last_unstale":"2024-01-27T02:43:29.946812+0000","last_undegraded":"2024-01-27T02:43:29.946812+0000","last_fullsized":"2024-01-27T02:43:29.946812+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:12:49.020814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"179'3","reported_seq":13,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.193520+0000","last_change":"2024-01-27T02:43:29.175735+0000","last_active":"2024-01-27T02:43:30.193520+0000","last_peered":"2024-01-27T02:43:30.193520+0000","last_clean":"2024-01-27T02:43:30.193520+0000","last_became_active":"2024-01-27T02:43:29.175458+0000","last_became_peered":"2024-01-27T02:43:29.175458+0000","last_unstale":"2024-01-27T02:43:30.193520+0000","last_undegraded":"2024-01-27T02:43:30.193520+0000","last_fullsized":"2024-01-27T02:43:30.193520+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:45:09.279278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.943382+0000","last_change":"2024-01-27T02:43:29.943316+0000","last_active":"2024-01-27T02:43:29.943382+0000","last_peered":"2024-01-27T02:43:29.943382+0000","last_clean":"2024-01-27T02:43:29.943382+0000","last_became_active":"2024-01-27T02:43:29.943046+0000","last_became_peered":"2024-01-27T02:43:29.943046+0000","last_unstale":"2024-01-27T02:43:29.943382+0000","last_undegraded":"2024-01-27T02:43:29.943382+0000","last_fullsized":"2024-01-27T02:43:29.943382+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:14:25.190169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"179'6","reported_seq":16,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.194303+0000","last_change":"2024-01-27T02:43:29.184266+0000","last_active":"2024-01-27T02:43:30.194303+0000","last_peered":"2024-01-27T02:43:30.194303+0000","last_clean":"2024-01-27T02:43:30.194303+0000","last_became_active":"2024-01-27T02:43:29.183979+0000","last_became_peered":"2024-01-27T02:43:29.183979+0000","last_unstale":"2024-01-27T02:43:30.194303+0000","last_undegraded":"2024-01-27T02:43:30.194303+0000","last_fullsized":"2024-01-27T02:43:30.194303+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:37:50.439488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.935916+0000","last_change":"2024-01-27T02:43:29.935843+0000","last_active":"2024-01-27T02:43:29.935916+0000","last_peered":"2024-01-27T02:43:29.935916+0000","last_clean":"2024-01-27T02:43:29.935916+0000","last_became_active":"2024-01-27T02:43:29.935571+0000","last_became_peered":"2024-01-27T02:43:29.935571+0000","last_unstale":"2024-01-27T02:43:29.935916+0000","last_undegraded":"2024-01-27T02:43:29.935916+0000","last_fullsized":"2024-01-27T02:43:29.935916+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:45:46.304037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"179'2","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.193865+0000","last_change":"2024-01-27T02:43:29.169880+0000","last_active":"2024-01-27T02:43:30.193865+0000","last_peered":"2024-01-27T02:43:30.193865+0000","last_clean":"2024-01-27T02:43:30.193865+0000","last_became_active":"2024-01-27T02:43:29.169617+0000","last_became_peered":"2024-01-27T02:43:29.169617+0000","last_unstale":"2024-01-27T02:43:30.193865+0000","last_undegraded":"2024-01-27T02:43:30.193865+0000","last_fullsized":"2024-01-27T02:43:30.193865+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:51:00.035593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.950762+0000","last_change":"2024-01-27T02:43:29.950684+0000","last_active":"2024-01-27T02:43:29.950762+0000","last_peered":"2024-01-27T02:43:29.950762+0000","last_clean":"2024-01-27T02:43:29.950762+0000","last_became_active":"2024-01-27T02:43:29.950409+0000","last_became_peered":"2024-01-27T02:43:29.950409+0000","last_unstale":"2024-01-27T02:43:29.950762+0000","last_undegraded":"2024-01-27T02:43:29.950762+0000","last_fullsized":"2024-01-27T02:43:29.950762+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:38:09.031394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.934755+0000","last_change":"2024-01-27T02:43:29.181779+0000","last_active":"2024-01-27T02:43:29.934755+0000","last_peered":"2024-01-27T02:43:29.934755+0000","last_clean":"2024-01-27T02:43:29.934755+0000","last_became_active":"2024-01-27T02:43:29.181019+0000","last_became_peered":"2024-01-27T02:43:29.181019+0000","last_unstale":"2024-01-27T02:43:29.934755+0000","last_undegraded":"2024-01-27T02:43:29.934755+0000","last_fullsized":"2024-01-27T02:43:29.934755+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:32:24.137664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.945997+0000","last_change":"2024-01-27T02:43:29.945881+0000","last_active":"2024-01-27T02:43:29.945997+0000","last_peered":"2024-01-27T02:43:29.945997+0000","last_clean":"2024-01-27T02:43:29.945997+0000","last_became_active":"2024-01-27T02:43:29.945624+0000","last_became_peered":"2024-01-27T02:43:29.945624+0000","last_unstale":"2024-01-27T02:43:29.945997+0000","last_undegraded":"2024-01-27T02:43:29.945997+0000","last_fullsized":"2024-01-27T02:43:29.945997+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:23:41.383567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.939631+0000","last_change":"2024-01-27T02:43:29.178721+0000","last_active":"2024-01-27T02:43:29.939631+0000","last_peered":"2024-01-27T02:43:29.939631+0000","last_clean":"2024-01-27T02:43:29.939631+0000","last_became_active":"2024-01-27T02:43:29.178413+0000","last_became_peered":"2024-01-27T02:43:29.178413+0000","last_unstale":"2024-01-27T02:43:29.939631+0000","last_undegraded":"2024-01-27T02:43:29.939631+0000","last_fullsized":"2024-01-27T02:43:29.939631+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:45:48.808912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.954619+0000","last_change":"2024-01-27T02:43:29.954533+0000","last_active":"2024-01-27T02:43:29.954619+0000","last_peered":"2024-01-27T02:43:29.954619+0000","last_clean":"2024-01-27T02:43:29.954619+0000","last_became_active":"2024-01-27T02:43:29.954357+0000","last_became_peered":"2024-01-27T02:43:29.954357+0000","last_unstale":"2024-01-27T02:43:29.954619+0000","last_undegraded":"2024-01-27T02:43:29.954619+0000","last_fullsized":"2024-01-27T02:43:29.954619+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:56:17.907654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"179'8","reported_seq":18,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.193228+0000","last_change":"2024-01-27T02:43:29.183819+0000","last_active":"2024-01-27T02:43:30.193228+0000","last_peered":"2024-01-27T02:43:30.193228+0000","last_clean":"2024-01-27T02:43:30.193228+0000","last_became_active":"2024-01-27T02:43:29.183572+0000","last_became_peered":"2024-01-27T02:43:29.183572+0000","last_unstale":"2024-01-27T02:43:30.193228+0000","last_undegraded":"2024-01-27T02:43:30.193228+0000","last_fullsized":"2024-01-27T02:43:30.193228+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:57:15.966132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.948765+0000","last_change":"2024-01-27T02:43:29.948243+0000","last_active":"2024-01-27T02:43:29.948765+0000","last_peered":"2024-01-27T02:43:29.948765+0000","last_clean":"2024-01-27T02:43:29.948765+0000","last_became_active":"2024-01-27T02:43:29.947903+0000","last_became_peered":"2024-01-27T02:43:29.947903+0000","last_unstale":"2024-01-27T02:43:29.948765+0000","last_undegraded":"2024-01-27T02:43:29.948765+0000","last_fullsized":"2024-01-27T02:43:29.948765+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:32:52.100147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"179'1","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.189644+0000","last_change":"2024-01-27T02:43:29.176120+0000","last_active":"2024-01-27T02:43:30.189644+0000","last_peered":"2024-01-27T02:43:30.189644+0000","last_clean":"2024-01-27T02:43:30.189644+0000","last_became_active":"2024-01-27T02:43:29.175827+0000","last_became_peered":"2024-01-27T02:43:29.175827+0000","last_unstale":"2024-01-27T02:43:30.189644+0000","last_undegraded":"2024-01-27T02:43:30.189644+0000","last_fullsized":"2024-01-27T02:43:30.189644+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:05.341515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.937123+0000","last_change":"2024-01-27T02:43:29.937069+0000","last_active":"2024-01-27T02:43:29.937123+0000","last_peered":"2024-01-27T02:43:29.937123+0000","last_clean":"2024-01-27T02:43:29.937123+0000","last_became_active":"2024-01-27T02:43:29.936846+0000","last_became_peered":"2024-01-27T02:43:29.936846+0000","last_unstale":"2024-01-27T02:43:29.937123+0000","last_undegraded":"2024-01-27T02:43:29.937123+0000","last_fullsized":"2024-01-27T02:43:29.937123+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:41:06.546746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"179'2","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.189671+0000","last_change":"2024-01-27T02:43:29.181342+0000","last_active":"2024-01-27T02:43:30.189671+0000","last_peered":"2024-01-27T02:43:30.189671+0000","last_clean":"2024-01-27T02:43:30.189671+0000","last_became_active":"2024-01-27T02:43:29.180819+0000","last_became_peered":"2024-01-27T02:43:29.180819+0000","last_unstale":"2024-01-27T02:43:30.189671+0000","last_undegraded":"2024-01-27T02:43:30.189671+0000","last_fullsized":"2024-01-27T02:43:30.189671+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:40:10.722633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.953725+0000","last_change":"2024-01-27T02:43:29.953664+0000","last_active":"2024-01-27T02:43:29.953725+0000","last_peered":"2024-01-27T02:43:29.953725+0000","last_clean":"2024-01-27T02:43:29.953725+0000","last_became_active":"2024-01-27T02:43:29.953132+0000","last_became_peered":"2024-01-27T02:43:29.953132+0000","last_unstale":"2024-01-27T02:43:29.953725+0000","last_undegraded":"2024-01-27T02:43:29.953725+0000","last_fullsized":"2024-01-27T02:43:29.953725+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:05:46.859143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"179'1","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.191405+0000","last_change":"2024-01-27T02:43:29.185493+0000","last_active":"2024-01-27T02:43:30.191405+0000","last_peered":"2024-01-27T02:43:30.191405+0000","last_clean":"2024-01-27T02:43:30.191405+0000","last_became_active":"2024-01-27T02:43:29.184932+0000","last_became_peered":"2024-01-27T02:43:29.184932+0000","last_unstale":"2024-01-27T02:43:30.191405+0000","last_undegraded":"2024-01-27T02:43:30.191405+0000","last_fullsized":"2024-01-27T02:43:30.191405+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:19:12.257608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.952799+0000","last_change":"2024-01-27T02:43:29.952105+0000","last_active":"2024-01-27T02:43:29.952799+0000","last_peered":"2024-01-27T02:43:29.952799+0000","last_clean":"2024-01-27T02:43:29.952799+0000","last_became_active":"2024-01-27T02:43:29.951012+0000","last_became_peered":"2024-01-27T02:43:29.951012+0000","last_unstale":"2024-01-27T02:43:29.952799+0000","last_undegraded":"2024-01-27T02:43:29.952799+0000","last_fullsized":"2024-01-27T02:43:29.952799+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:59:37.749331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.939242+0000","last_change":"2024-01-27T02:43:29.183016+0000","last_active":"2024-01-27T02:43:29.939242+0000","last_peered":"2024-01-27T02:43:29.939242+0000","last_clean":"2024-01-27T02:43:29.939242+0000","last_became_active":"2024-01-27T02:43:29.182744+0000","last_became_peered":"2024-01-27T02:43:29.182744+0000","last_unstale":"2024-01-27T02:43:29.939242+0000","last_undegraded":"2024-01-27T02:43:29.939242+0000","last_fullsized":"2024-01-27T02:43:29.939242+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:21:56.830916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.943180+0000","last_change":"2024-01-27T02:43:29.943074+0000","last_active":"2024-01-27T02:43:29.943180+0000","last_peered":"2024-01-27T02:43:29.943180+0000","last_clean":"2024-01-27T02:43:29.943180+0000","last_became_active":"2024-01-27T02:43:29.942846+0000","last_became_peered":"2024-01-27T02:43:29.942846+0000","last_unstale":"2024-01-27T02:43:29.943180+0000","last_undegraded":"2024-01-27T02:43:29.943180+0000","last_fullsized":"2024-01-27T02:43:29.943180+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:37:51.444958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"179'2","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.193825+0000","last_change":"2024-01-27T02:43:29.179334+0000","last_active":"2024-01-27T02:43:30.193825+0000","last_peered":"2024-01-27T02:43:30.193825+0000","last_clean":"2024-01-27T02:43:30.193825+0000","last_became_active":"2024-01-27T02:43:29.178831+0000","last_became_peered":"2024-01-27T02:43:29.178831+0000","last_unstale":"2024-01-27T02:43:30.193825+0000","last_undegraded":"2024-01-27T02:43:30.193825+0000","last_fullsized":"2024-01-27T02:43:30.193825+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:37:50.439488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.949858+0000","last_change":"2024-01-27T02:43:29.946529+0000","last_active":"2024-01-27T02:43:29.949858+0000","last_peered":"2024-01-27T02:43:29.949858+0000","last_clean":"2024-01-27T02:43:29.949858+0000","last_became_active":"2024-01-27T02:43:29.945124+0000","last_became_peered":"2024-01-27T02:43:29.945124+0000","last_unstale":"2024-01-27T02:43:29.949858+0000","last_undegraded":"2024-01-27T02:43:29.949858+0000","last_fullsized":"2024-01-27T02:43:29.949858+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:06:22.906823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.939587+0000","last_change":"2024-01-27T02:43:29.182867+0000","last_active":"2024-01-27T02:43:29.939587+0000","last_peered":"2024-01-27T02:43:29.939587+0000","last_clean":"2024-01-27T02:43:29.939587+0000","last_became_active":"2024-01-27T02:43:29.182604+0000","last_became_peered":"2024-01-27T02:43:29.182604+0000","last_unstale":"2024-01-27T02:43:29.939587+0000","last_undegraded":"2024-01-27T02:43:29.939587+0000","last_fullsized":"2024-01-27T02:43:29.939587+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:06:23.512245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.946876+0000","last_change":"2024-01-27T02:43:29.946674+0000","last_active":"2024-01-27T02:43:29.946876+0000","last_peered":"2024-01-27T02:43:29.946876+0000","last_clean":"2024-01-27T02:43:29.946876+0000","last_became_active":"2024-01-27T02:43:29.945198+0000","last_became_peered":"2024-01-27T02:43:29.945198+0000","last_unstale":"2024-01-27T02:43:29.946876+0000","last_undegraded":"2024-01-27T02:43:29.946876+0000","last_fullsized":"2024-01-27T02:43:29.946876+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:38:09.031394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.941364+0000","last_change":"2024-01-27T02:43:29.176997+0000","last_active":"2024-01-27T02:43:29.941364+0000","last_peered":"2024-01-27T02:43:29.941364+0000","last_clean":"2024-01-27T02:43:29.941364+0000","last_became_active":"2024-01-27T02:43:29.176143+0000","last_became_peered":"2024-01-27T02:43:29.176143+0000","last_unstale":"2024-01-27T02:43:29.941364+0000","last_undegraded":"2024-01-27T02:43:29.941364+0000","last_fullsized":"2024-01-27T02:43:29.941364+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:54:49.762475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.943957+0000","last_change":"2024-01-27T02:43:29.943410+0000","last_active":"2024-01-27T02:43:29.943957+0000","last_peered":"2024-01-27T02:43:29.943957+0000","last_clean":"2024-01-27T02:43:29.943957+0000","last_became_active":"2024-01-27T02:43:29.943179+0000","last_became_peered":"2024-01-27T02:43:29.943179+0000","last_unstale":"2024-01-27T02:43:29.943957+0000","last_undegraded":"2024-01-27T02:43:29.943957+0000","last_fullsized":"2024-01-27T02:43:29.943957+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:01:17.185547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.940325+0000","last_change":"2024-01-27T02:43:29.185642+0000","last_active":"2024-01-27T02:43:29.940325+0000","last_peered":"2024-01-27T02:43:29.940325+0000","last_clean":"2024-01-27T02:43:29.940325+0000","last_became_active":"2024-01-27T02:43:29.185247+0000","last_became_peered":"2024-01-27T02:43:29.185247+0000","last_unstale":"2024-01-27T02:43:29.940325+0000","last_undegraded":"2024-01-27T02:43:29.940325+0000","last_fullsized":"2024-01-27T02:43:29.940325+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:32:24.137664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.949053+0000","last_change":"2024-01-27T02:43:29.948584+0000","last_active":"2024-01-27T02:43:29.949053+0000","last_peered":"2024-01-27T02:43:29.949053+0000","last_clean":"2024-01-27T02:43:29.949053+0000","last_became_active":"2024-01-27T02:43:29.945982+0000","last_became_peered":"2024-01-27T02:43:29.945982+0000","last_unstale":"2024-01-27T02:43:29.949053+0000","last_undegraded":"2024-01-27T02:43:29.949053+0000","last_fullsized":"2024-01-27T02:43:29.949053+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:23:43.836258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"179'2","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.190556+0000","last_change":"2024-01-27T02:43:29.182305+0000","last_active":"2024-01-27T02:43:30.190556+0000","last_peered":"2024-01-27T02:43:30.190556+0000","last_clean":"2024-01-27T02:43:30.190556+0000","last_became_active":"2024-01-27T02:43:29.181295+0000","last_became_peered":"2024-01-27T02:43:29.181295+0000","last_unstale":"2024-01-27T02:43:30.190556+0000","last_undegraded":"2024-01-27T02:43:30.190556+0000","last_fullsized":"2024-01-27T02:43:30.190556+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:12:22.590221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.953982+0000","last_change":"2024-01-27T02:43:29.953927+0000","last_active":"2024-01-27T02:43:29.953982+0000","last_peered":"2024-01-27T02:43:29.953982+0000","last_clean":"2024-01-27T02:43:29.953982+0000","last_became_active":"2024-01-27T02:43:29.952556+0000","last_became_peered":"2024-01-27T02:43:29.952556+0000","last_unstale":"2024-01-27T02:43:29.953982+0000","last_undegraded":"2024-01-27T02:43:29.953982+0000","last_fullsized":"2024-01-27T02:43:29.953982+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:15:03.662522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.941116+0000","last_change":"2024-01-27T02:43:29.180535+0000","last_active":"2024-01-27T02:43:29.941116+0000","last_peered":"2024-01-27T02:43:29.941116+0000","last_clean":"2024-01-27T02:43:29.941116+0000","last_became_active":"2024-01-27T02:43:29.180044+0000","last_became_peered":"2024-01-27T02:43:29.180044+0000","last_unstale":"2024-01-27T02:43:29.941116+0000","last_undegraded":"2024-01-27T02:43:29.941116+0000","last_fullsized":"2024-01-27T02:43:29.941116+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:46:08.576546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.949035+0000","last_change":"2024-01-27T02:43:29.948966+0000","last_active":"2024-01-27T02:43:29.949035+0000","last_peered":"2024-01-27T02:43:29.949035+0000","last_clean":"2024-01-27T02:43:29.949035+0000","last_became_active":"2024-01-27T02:43:29.948633+0000","last_became_peered":"2024-01-27T02:43:29.948633+0000","last_unstale":"2024-01-27T02:43:29.949035+0000","last_undegraded":"2024-01-27T02:43:29.949035+0000","last_fullsized":"2024-01-27T02:43:29.949035+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:40:11.728102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.935294+0000","last_change":"2024-01-27T02:43:29.182415+0000","last_active":"2024-01-27T02:43:29.935294+0000","last_peered":"2024-01-27T02:43:29.935294+0000","last_clean":"2024-01-27T02:43:29.935294+0000","last_became_active":"2024-01-27T02:43:29.182106+0000","last_became_peered":"2024-01-27T02:43:29.182106+0000","last_unstale":"2024-01-27T02:43:29.935294+0000","last_undegraded":"2024-01-27T02:43:29.935294+0000","last_fullsized":"2024-01-27T02:43:29.935294+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:37:50.439488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.942793+0000","last_change":"2024-01-27T02:43:29.942716+0000","last_active":"2024-01-27T02:43:29.942793+0000","last_peered":"2024-01-27T02:43:29.942793+0000","last_clean":"2024-01-27T02:43:29.942793+0000","last_became_active":"2024-01-27T02:43:29.942440+0000","last_became_peered":"2024-01-27T02:43:29.942440+0000","last_unstale":"2024-01-27T02:43:29.942793+0000","last_undegraded":"2024-01-27T02:43:29.942793+0000","last_fullsized":"2024-01-27T02:43:29.942793+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:06:22.985377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.933841+0000","last_change":"2024-01-27T02:43:29.180412+0000","last_active":"2024-01-27T02:43:29.933841+0000","last_peered":"2024-01-27T02:43:29.933841+0000","last_clean":"2024-01-27T02:43:29.933841+0000","last_became_active":"2024-01-27T02:43:29.180108+0000","last_became_peered":"2024-01-27T02:43:29.180108+0000","last_unstale":"2024-01-27T02:43:29.933841+0000","last_undegraded":"2024-01-27T02:43:29.933841+0000","last_fullsized":"2024-01-27T02:43:29.933841+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:19:12.257608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.951099+0000","last_change":"2024-01-27T02:43:29.950832+0000","last_active":"2024-01-27T02:43:29.951099+0000","last_peered":"2024-01-27T02:43:29.951099+0000","last_clean":"2024-01-27T02:43:29.951099+0000","last_became_active":"2024-01-27T02:43:29.950130+0000","last_became_peered":"2024-01-27T02:43:29.950130+0000","last_unstale":"2024-01-27T02:43:29.951099+0000","last_undegraded":"2024-01-27T02:43:29.951099+0000","last_fullsized":"2024-01-27T02:43:29.951099+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:28:40.712001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.937243+0000","last_change":"2024-01-27T02:43:29.183590+0000","last_active":"2024-01-27T02:43:29.937243+0000","last_peered":"2024-01-27T02:43:29.937243+0000","last_clean":"2024-01-27T02:43:29.937243+0000","last_became_active":"2024-01-27T02:43:29.183285+0000","last_became_peered":"2024-01-27T02:43:29.183285+0000","last_unstale":"2024-01-27T02:43:29.937243+0000","last_undegraded":"2024-01-27T02:43:29.937243+0000","last_fullsized":"2024-01-27T02:43:29.937243+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:53:51.636448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.952035+0000","last_change":"2024-01-27T02:43:29.951984+0000","last_active":"2024-01-27T02:43:29.952035+0000","last_peered":"2024-01-27T02:43:29.952035+0000","last_clean":"2024-01-27T02:43:29.952035+0000","last_became_active":"2024-01-27T02:43:29.951822+0000","last_became_peered":"2024-01-27T02:43:29.951822+0000","last_unstale":"2024-01-27T02:43:29.952035+0000","last_undegraded":"2024-01-27T02:43:29.952035+0000","last_fullsized":"2024-01-27T02:43:29.952035+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:05:31.630090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.939758+0000","last_change":"2024-01-27T02:43:29.173228+0000","last_active":"2024-01-27T02:43:29.939758+0000","last_peered":"2024-01-27T02:43:29.939758+0000","last_clean":"2024-01-27T02:43:29.939758+0000","last_became_active":"2024-01-27T02:43:29.172707+0000","last_became_peered":"2024-01-27T02:43:29.172707+0000","last_unstale":"2024-01-27T02:43:29.939758+0000","last_undegraded":"2024-01-27T02:43:29.939758+0000","last_fullsized":"2024-01-27T02:43:29.939758+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:57:15.966132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.946388+0000","last_change":"2024-01-27T02:43:29.944901+0000","last_active":"2024-01-27T02:43:29.946388+0000","last_peered":"2024-01-27T02:43:29.946388+0000","last_clean":"2024-01-27T02:43:29.946388+0000","last_became_active":"2024-01-27T02:43:29.944603+0000","last_became_peered":"2024-01-27T02:43:29.944603+0000","last_unstale":"2024-01-27T02:43:29.946388+0000","last_undegraded":"2024-01-27T02:43:29.946388+0000","last_fullsized":"2024-01-27T02:43:29.946388+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:58:23.934270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.936985+0000","last_change":"2024-01-27T02:43:29.182216+0000","last_active":"2024-01-27T02:43:29.936985+0000","last_peered":"2024-01-27T02:43:29.936985+0000","last_clean":"2024-01-27T02:43:29.936985+0000","last_became_active":"2024-01-27T02:43:29.181943+0000","last_became_peered":"2024-01-27T02:43:29.181943+0000","last_unstale":"2024-01-27T02:43:29.936985+0000","last_undegraded":"2024-01-27T02:43:29.936985+0000","last_fullsized":"2024-01-27T02:43:29.936985+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:24:43.336318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.949145+0000","last_change":"2024-01-27T02:43:29.949087+0000","last_active":"2024-01-27T02:43:29.949145+0000","last_peered":"2024-01-27T02:43:29.949145+0000","last_clean":"2024-01-27T02:43:29.949145+0000","last_became_active":"2024-01-27T02:43:29.948861+0000","last_became_peered":"2024-01-27T02:43:29.948861+0000","last_unstale":"2024-01-27T02:43:29.949145+0000","last_undegraded":"2024-01-27T02:43:29.949145+0000","last_fullsized":"2024-01-27T02:43:29.949145+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:53:56.469059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.935934+0000","last_change":"2024-01-27T02:43:29.179446+0000","last_active":"2024-01-27T02:43:29.935934+0000","last_peered":"2024-01-27T02:43:29.935934+0000","last_clean":"2024-01-27T02:43:29.935934+0000","last_became_active":"2024-01-27T02:43:29.179149+0000","last_became_peered":"2024-01-27T02:43:29.179149+0000","last_unstale":"2024-01-27T02:43:29.935934+0000","last_undegraded":"2024-01-27T02:43:29.935934+0000","last_fullsized":"2024-01-27T02:43:29.935934+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:54:49.762475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.951079+0000","last_change":"2024-01-27T02:43:29.949855+0000","last_active":"2024-01-27T02:43:29.951079+0000","last_peered":"2024-01-27T02:43:29.951079+0000","last_clean":"2024-01-27T02:43:29.951079+0000","last_became_active":"2024-01-27T02:43:29.946034+0000","last_became_peered":"2024-01-27T02:43:29.946034+0000","last_unstale":"2024-01-27T02:43:29.951079+0000","last_undegraded":"2024-01-27T02:43:29.951079+0000","last_fullsized":"2024-01-27T02:43:29.951079+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:57:29.655912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.937372+0000","last_change":"2024-01-27T02:43:29.179715+0000","last_active":"2024-01-27T02:43:29.937372+0000","last_peered":"2024-01-27T02:43:29.937372+0000","last_clean":"2024-01-27T02:43:29.937372+0000","last_became_active":"2024-01-27T02:43:29.178851+0000","last_became_peered":"2024-01-27T02:43:29.178851+0000","last_unstale":"2024-01-27T02:43:29.937372+0000","last_undegraded":"2024-01-27T02:43:29.937372+0000","last_fullsized":"2024-01-27T02:43:29.937372+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:01:19.140032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":302,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.939677+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:43:29.939677+0000","last_peered":"2024-01-27T02:43:29.939677+0000","last_clean":"2024-01-27T02:43:29.939677+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:43:29.939677+0000","last_undegraded":"2024-01-27T02:43:29.939677+0000","last_fullsized":"2024-01-27T02:43:29.939677+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574989,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1554,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193757,"statfs":{"total":200000000,"available":198407730,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.32800000000000001},"max":{"1min":0.80600000000000005,"5min":0.83499999999999996,"15min":0.85499999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59899999999999998,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.30099999999999999},"max":{"1min":0.84199999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.57199999999999995}]},{"osd":1,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.47699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74399999999999999,"5min":0.78800000000000003,"15min":0.82399999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.57399999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.34999999999999998},"max":{"1min":0.81599999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.54000000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.38400000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.86399999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.75,"5min":0.80300000000000005,"15min":0.83899999999999997},"last":0.443}]},{"osd":3,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76500000000000001,"5min":0.78900000000000003,"15min":0.90300000000000002},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.63800000000000001,"15min":0.61199999999999999},"min":{"1min":0.47599999999999998,"5min":0.41799999999999998,"15min":0.40999999999999998},"max":{"1min":0.77400000000000002,"5min":0.81599999999999995,"15min":0.88600000000000001},"last":0.55100000000000005}]},{"osd":4,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48199999999999998,"15min":0.48399999999999999},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67300000000000004,"5min":0.73899999999999999,"15min":0.78600000000000003},"last":0.5},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52000000000000002,"15min":0.51000000000000001},"min":{"1min":0.36699999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.80000000000000004},"last":0.42899999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.39600000000000002,"5min":0.32400000000000001,"15min":0.307},"max":{"1min":0.746,"5min":0.79500000000000004,"15min":0.85399999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.42499999999999999,"5min":0.33900000000000002,"15min":0.33500000000000002},"max":{"1min":0.76600000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45400000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60199999999999998,"15min":0.59299999999999997},"min":{"1min":0.38900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.78900000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71799999999999997,"5min":0.83499999999999996,"15min":0.90800000000000003},"last":0.34499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574989,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":976,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194336,"statfs":{"total":200000000,"available":199000326,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60099999999999998,"15min":0.57899999999999996},"min":{"1min":0.38800000000000001,"5min":0.375,"15min":0.29999999999999999},"max":{"1min":0.78700000000000003,"5min":1.0409999999999999,"15min":4.125},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58799999999999997,"15min":0.56499999999999995},"min":{"1min":0.435,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.753,"5min":0.96599999999999997,"15min":1.216},"last":0.50600000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42599999999999999,"5min":0.376,"15min":0.34100000000000003},"max":{"1min":0.67800000000000005,"5min":0.95499999999999996,"15min":8.2699999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59699999999999998,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78000000000000003,"5min":1.0129999999999999,"15min":26.411999999999999},"last":0.56499999999999995}]},{"osd":2,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.627},"min":{"1min":0.51900000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73699999999999999,"5min":0.78300000000000003,"15min":17.143999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61499999999999999,"15min":0.60899999999999999},"min":{"1min":0.503,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73199999999999998,"5min":0.85799999999999998,"15min":1.76},"last":0.65300000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.47699999999999998,"5min":0.39400000000000002,"15min":0.376},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":4.3689999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.41199999999999998},"max":{"1min":0.77400000000000002,"5min":0.81699999999999995,"15min":2.1970000000000001},"last":0.79100000000000004}]},{"osd":4,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47099999999999997,"15min":0.48099999999999998},"min":{"1min":0.35299999999999998,"5min":0.311,"15min":0.30199999999999999},"max":{"1min":0.61499999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78900000000000003,"5min":0.84899999999999998,"15min":0.89600000000000002},"last":0.754}]},{"osd":5,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.39400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.65400000000000003,"5min":0.80800000000000005,"15min":0.81299999999999994},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55200000000000005,"15min":0.57699999999999996},"min":{"1min":0.38300000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76400000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.86699999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57799999999999996,"15min":0.59599999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.35999999999999999},"max":{"1min":0.73599999999999999,"5min":0.88700000000000001,"15min":1.0009999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54900000000000004,"15min":0.56799999999999995},"min":{"1min":0.40699999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.749,"5min":0.82899999999999996,"15min":1.038},"last":0.90200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574989,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":976,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194336,"statfs":{"total":200000000,"available":199000314,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40300000000000002,"15min":0.38900000000000001},"min":{"1min":0.26900000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.51800000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.42999999999999999,"15min":0.41299999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.27700000000000002},"max":{"1min":0.51400000000000001,"5min":0.66100000000000003,"15min":1.121},"last":0.55700000000000005}]},{"osd":2,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42199999999999999,"15min":0.39300000000000002},"min":{"1min":0.30599999999999999,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":0.56000000000000005,"5min":1.0940000000000001,"15min":1.25},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.45200000000000001,"15min":0.442},"min":{"1min":0.30099999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.65800000000000003,"5min":0.66700000000000004,"15min":0.82999999999999996},"last":0.52000000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.495,"15min":0.48199999999999998},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.32100000000000001},"max":{"1min":0.61299999999999999,"5min":0.79900000000000004,"15min":0.90900000000000003},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47399999999999998,"15min":0.46400000000000002},"min":{"1min":0.35999999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.59199999999999997,"5min":0.72599999999999998,"15min":0.98999999999999999},"last":0.62}]},{"osd":4,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66000000000000003,"15min":0.64800000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.373},"max":{"1min":0.83999999999999997,"5min":1.075,"15min":3.2890000000000001},"last":0.75},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67800000000000005,"15min":0.65500000000000003},"min":{"1min":0.48999999999999999,"5min":0.45000000000000001,"15min":0.44500000000000001},"max":{"1min":0.95499999999999996,"5min":0.98099999999999998,"15min":1.026},"last":0.90900000000000003}]},{"osd":5,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68500000000000005,"15min":0.66200000000000003},"min":{"1min":0.46700000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89900000000000002,"5min":0.94599999999999995,"15min":3.4500000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68600000000000005,"15min":0.68200000000000005},"min":{"1min":0.45600000000000002,"5min":0.44,"15min":0.42399999999999999},"max":{"1min":0.93000000000000005,"5min":1.004,"15min":3.181},"last":0.77200000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.71599999999999997,"15min":0.70599999999999996},"min":{"1min":0.52500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91400000000000003,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.74099999999999999,"15min":0.70599999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.50800000000000001},"max":{"1min":0.93899999999999995,"5min":1.143,"15min":1.869},"last":0.73899999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73799999999999999,"15min":0.71099999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85399999999999998,"5min":1.181,"15min":1.6870000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72799999999999998,"15min":0.71099999999999997},"min":{"1min":0.56299999999999994,"5min":0.501,"15min":0.45300000000000001},"max":{"1min":0.92400000000000004,"5min":1.0429999999999999,"15min":1.708},"last":0.84099999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574989,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1552,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193759,"statfs":{"total":200000000,"available":198409980,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39500000000000002,"15min":0.39300000000000002},"min":{"1min":0.28499999999999998,"5min":0.24399999999999999,"15min":0.23799999999999999},"max":{"1min":0.55000000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42499999999999999,"15min":0.41999999999999998},"min":{"1min":0.32400000000000001,"5min":0.26000000000000001,"15min":0.223},"max":{"1min":0.55800000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.504}]},{"osd":2,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42499999999999999},"min":{"1min":0.34999999999999998,"5min":0.28599999999999998,"15min":0.28199999999999997},"max":{"1min":0.60799999999999998,"5min":0.625,"15min":0.64000000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45500000000000002,"15min":0.44400000000000001},"min":{"1min":0.34200000000000003,"5min":0.29499999999999998,"15min":0.28799999999999998},"max":{"1min":0.626,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.44700000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.44600000000000001,"15min":0.436},"min":{"1min":0.29999999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58499999999999996,"5min":0.66600000000000004,"15min":0.67300000000000004},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.39000000000000001,"5min":0.34300000000000003,"15min":0.29099999999999998},"max":{"1min":0.69799999999999995,"5min":0.71399999999999997,"15min":0.71699999999999997},"last":0.47099999999999997}]},{"osd":4,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.65400000000000003,"15min":0.64900000000000002},"min":{"1min":0.46600000000000003,"5min":0.432,"15min":0.42099999999999999},"max":{"1min":0.84899999999999998,"5min":0.97599999999999998,"15min":0.997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63700000000000001,"15min":0.65000000000000002},"min":{"1min":0.46400000000000002,"5min":0.42199999999999999,"15min":0.41999999999999998},"max":{"1min":0.85999999999999999,"5min":0.87,"15min":0.96699999999999997},"last":0.79200000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.54100000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":2.4860000000000002},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.44},"max":{"1min":0.90200000000000002,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.84499999999999997}]},{"osd":6,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.51300000000000001,"5min":0.45600000000000002,"15min":0.42499999999999999},"max":{"1min":0.92600000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68899999999999995,"15min":0.68600000000000005},"min":{"1min":0.53600000000000003,"5min":0.46500000000000002,"15min":0.45000000000000001},"max":{"1min":0.96699999999999997,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.82899999999999996}]},{"osd":7,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.51400000000000001,"5min":0.47099999999999997,"15min":0.42599999999999999},"max":{"1min":0.94799999999999995,"5min":0.94999999999999996,"15min":1.014},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68899999999999995,"15min":0.68999999999999995},"min":{"1min":0.55800000000000005,"5min":0.499,"15min":0.498},"max":{"1min":0.90900000000000003,"5min":0.92500000000000004,"15min":0.97299999999999998},"last":0.90500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574989,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":982,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194330,"statfs":{"total":200000000,"available":198993970,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41099999999999998},"min":{"1min":0.27000000000000002,"5min":0.23300000000000001,"15min":0.23000000000000001},"max":{"1min":0.63200000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.46000000000000002,"15min":0.53900000000000003},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.30499999999999999},"max":{"1min":0.57999999999999996,"5min":0.68899999999999995,"15min":6.2729999999999997},"last":0.36099999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.41199999999999998,"15min":0.42099999999999999},"min":{"1min":0.33000000000000002,"5min":0.28799999999999998,"15min":0.23799999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.81399999999999995},"last":0.375},{"interface":"front","average":{"1min":0.442,"5min":0.443,"15min":0.442},"min":{"1min":0.29999999999999999,"5min":0.28999999999999998,"15min":0.28899999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.70099999999999996},"last":0.40500000000000003}]},{"osd":3,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.31,"5min":0.28499999999999998,"15min":0.27800000000000002},"max":{"1min":0.55800000000000005,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.441},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48399999999999999,"15min":0.48499999999999999},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.70399999999999996,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.41799999999999998}]},{"osd":4,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68100000000000005,"15min":0.67000000000000004},"min":{"1min":0.54600000000000004,"5min":0.47599999999999998,"15min":0.39300000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.98299999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.746},"min":{"1min":0.56499999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94399999999999995,"5min":0.95799999999999996,"15min":3.581},"last":0.71899999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65800000000000003,"15min":0.69599999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.43099999999999999},"max":{"1min":0.90400000000000003,"5min":0.92200000000000004,"15min":3.5369999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66400000000000003,"15min":0.66800000000000004},"min":{"1min":0.5,"5min":0.44800000000000001,"15min":0.41099999999999998},"max":{"1min":0.79900000000000004,"5min":0.89100000000000001,"15min":1.0529999999999999},"last":0.70599999999999996}]},{"osd":6,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.55300000000000005,"5min":0.46999999999999997,"15min":0.42299999999999999},"max":{"1min":1.0900000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70099999999999996,"15min":0.68899999999999995},"min":{"1min":0.48299999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0469999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.73399999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69599999999999995,"15min":0.68500000000000005},"min":{"1min":0.46999999999999997,"5min":0.437,"15min":0.437},"max":{"1min":1.0600000000000001,"5min":1.55,"15min":1.55},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71999999999999997,"15min":0.70099999999999996},"min":{"1min":0.54300000000000004,"5min":0.51300000000000001,"15min":0.42699999999999999},"max":{"1min":0.96999999999999997,"5min":1.615,"15min":1.615},"last":0.77300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":976,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194336,"statfs":{"total":200000000,"available":199000314,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.253},"max":{"1min":0.56299999999999994,"5min":0.67900000000000005,"15min":4.7380000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.441,"5min":0.42099999999999999,"15min":0.40899999999999997},"min":{"1min":0.30499999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":1.0189999999999999},"last":0.34599999999999997}]},{"osd":1,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40500000000000003,"15min":0.41999999999999998},"min":{"1min":0.27600000000000002,"5min":0.24399999999999999,"15min":0.22900000000000001},"max":{"1min":0.54300000000000004,"5min":0.64400000000000002,"15min":4.5739999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46300000000000002,"15min":0.46500000000000002},"min":{"1min":0.35499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.751,"5min":0.98199999999999998,"15min":5.0529999999999999},"last":0.47899999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.46800000000000003,"15min":0.45600000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.28100000000000003},"max":{"1min":0.624,"5min":0.76700000000000002,"15min":1.5980000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.48899999999999999,"15min":0.49299999999999999},"min":{"1min":0.42599999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.65700000000000003,"5min":0.73499999999999999,"15min":5.1879999999999997},"last":0.443}]},{"osd":4,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63500000000000001,"15min":0.64500000000000002},"min":{"1min":0.44,"5min":0.42999999999999999,"15min":0.41699999999999998},"max":{"1min":0.877,"5min":0.91600000000000004,"15min":4.6660000000000004},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65400000000000003,"15min":0.63600000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84199999999999997,"5min":0.94799999999999995,"15min":1.3120000000000001},"last":0.51600000000000001}]},{"osd":5,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.49199999999999999,"5min":0.438,"15min":0.40799999999999997},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.409},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66600000000000004,"15min":0.70499999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81499999999999995,"5min":0.96599999999999997,"15min":4.7960000000000003},"last":0.66600000000000004}]},{"osd":6,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67200000000000004,"15min":0.68500000000000005},"min":{"1min":0.50900000000000001,"5min":0.42799999999999999,"15min":0.39800000000000002},"max":{"1min":0.86099999999999999,"5min":0.93500000000000005,"15min":5.0069999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69299999999999995,"15min":0.67200000000000004},"min":{"1min":0.53000000000000003,"5min":0.47499999999999998,"15min":0.435},"max":{"1min":0.83499999999999996,"5min":0.94199999999999995,"15min":1.103},"last":0.62}]},{"osd":7,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46000000000000002},"max":{"1min":0.83399999999999996,"5min":0.92100000000000004,"15min":4.7130000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.68899999999999995},"min":{"1min":0.48599999999999999,"5min":0.47299999999999998,"15min":0.435},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.361},"last":0.58399999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":983,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194328,"statfs":{"total":200000000,"available":198992454,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.43099999999999999,"5min":0.35599999999999998,"15min":0.314},"max":{"1min":0.76800000000000002,"5min":1.137,"15min":1.3839999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60899999999999999,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":1.016,"15min":7.0670000000000002},"last":0.51600000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.35399999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":5.1529999999999996},"last":0.437},{"interface":"front","average":{"1min":0.623,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.48499999999999999,"5min":0.43099999999999999,"15min":0.39000000000000001},"max":{"1min":0.80100000000000005,"5min":1.054,"15min":3.0419999999999998},"last":0.47899999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60399999999999998,"15min":0.61499999999999999},"min":{"1min":0.46400000000000002,"5min":0.41199999999999998,"15min":0.39400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":5.0019999999999998},"last":0.499},{"interface":"front","average":{"1min":0.622,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.49399999999999999,"5min":0.41899999999999998,"15min":0.40500000000000003},"max":{"1min":0.86099999999999999,"5min":1.2549999999999999,"15min":2.073},"last":0.38900000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.496,"5min":0.439,"15min":0.36199999999999999},"max":{"1min":0.78800000000000003,"5min":1.1899999999999999,"15min":6.7220000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.629,"15min":0.65700000000000003},"min":{"1min":0.442,"5min":0.42499999999999999,"15min":0.41399999999999998},"max":{"1min":0.872,"5min":0.88,"15min":6.5510000000000002},"last":0.54300000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.36199999999999999,"5min":0.314,"15min":0.312},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":1.3149999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.5,"15min":0.51800000000000002},"min":{"1min":0.41399999999999998,"5min":0.32500000000000001,"15min":0.29999999999999999},"max":{"1min":0.77500000000000002,"5min":0.79800000000000004,"15min":6.9269999999999996},"last":0.45800000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.432,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":5.2229999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55600000000000005,"15min":0.55100000000000005},"min":{"1min":0.437,"5min":0.35799999999999998,"15min":0.33200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":1.46},"last":0.53100000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55200000000000005,"15min":0.57099999999999995},"min":{"1min":0.41399999999999998,"5min":0.36599999999999999,"15min":0.33600000000000002},"max":{"1min":0.80400000000000005,"5min":0.83499999999999996,"15min":7.5339999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58099999999999996,"15min":0.59399999999999997},"min":{"1min":0.47099999999999997,"5min":0.34699999999999998,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.85299999999999998,"15min":4.9550000000000001},"last":0.627}]}]},{"osd":5,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":976,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194335,"statfs":{"total":200000000,"available":198999772,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.44700000000000001,"5min":0.33900000000000002,"15min":0.28899999999999998},"max":{"1min":0.76300000000000001,"5min":1.071,"15min":1.071},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.47999999999999998,"5min":0.38500000000000001,"15min":0.27600000000000002},"max":{"1min":0.71099999999999997,"5min":0.83599999999999997,"15min":3.242},"last":0.47399999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.627},"min":{"1min":0.53500000000000003,"5min":0.38100000000000001,"15min":0.32800000000000001},"max":{"1min":0.80800000000000005,"5min":1.018,"15min":1.3380000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.45600000000000002,"5min":0.35399999999999998,"15min":0.34200000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.95499999999999996},"last":0.499}]},{"osd":2,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":0.96699999999999997,"15min":3.04},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60799999999999998,"15min":0.59099999999999997},"min":{"1min":0.52000000000000002,"5min":0.43099999999999999,"15min":0.36899999999999999},"max":{"1min":0.71299999999999997,"5min":0.81000000000000005,"15min":0.89400000000000002},"last":0.39800000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63,"15min":0.65500000000000003},"min":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.39800000000000002},"max":{"1min":0.73799999999999999,"5min":1.0409999999999999,"15min":4.032},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66900000000000004,"15min":0.68000000000000005},"min":{"1min":0.57199999999999995,"5min":0.48799999999999999,"15min":0.45600000000000002},"max":{"1min":0.753,"5min":0.879,"15min":4.282},"last":0.34300000000000003}]},{"osd":4,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.51500000000000001},"min":{"1min":0.376,"5min":0.33300000000000002,"15min":0.26500000000000001},"max":{"1min":0.76400000000000001,"5min":0.95399999999999996,"15min":1.2869999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.496,"15min":0.52900000000000003},"min":{"1min":0.35799999999999998,"5min":0.32800000000000001,"15min":0.30399999999999999},"max":{"1min":0.67900000000000005,"5min":0.81699999999999995,"15min":1.101},"last":0.44500000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.50600000000000001,"15min":0.52900000000000003},"min":{"1min":0.35299999999999998,"5min":0.32700000000000001,"15min":0.29099999999999998},"max":{"1min":0.65700000000000003,"5min":0.86599999999999999,"15min":1.7969999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.55200000000000005,"15min":0.56799999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.33600000000000002},"max":{"1min":0.69299999999999995,"5min":0.91800000000000004,"15min":1.6739999999999999},"last":0.54800000000000004}]},{"osd":7,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.55100000000000005,"15min":0.55900000000000005},"min":{"1min":0.39500000000000002,"5min":0.36899999999999999,"15min":0.32700000000000001},"max":{"1min":0.76300000000000001,"5min":0.90600000000000003,"15min":1.008},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59199999999999997,"15min":0.627},"min":{"1min":0.46000000000000002,"5min":0.40600000000000003,"15min":0.33700000000000002},"max":{"1min":0.67500000000000004,"5min":0.91300000000000003,"15min":2},"last":0.59799999999999998}]}]}],"pool_statfs":[]}} 2024-01-27T02:43:34.376 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:43:34.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.543+0000 7f6c5860c700 1 Processor -- start 2024-01-27T02:43:34.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.543+0000 7f6c5860c700 1 -- start start 2024-01-27T02:43:34.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.544+0000 7f6c5860c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c500637d0 0x7f6c5006ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.544+0000 7f6c5860c700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6c50063ba0 0x7f6c500c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.544+0000 7f6c5860c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c5015b6c0 con 0x7f6c500c41e0 2024-01-27T02:43:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.544+0000 7f6c5860c700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c5015ae50 con 0x7f6c50063ba0 2024-01-27T02:43:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.544+0000 7f6c5860c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5015b100 con 0x7f6c500637d0 2024-01-27T02:43:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.544+0000 7f6c55ba7700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6c500c41e0 0x7f6c5015a740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38506/0 (socket says 172.21.15.7:38506) 2024-01-27T02:43:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.544+0000 7f6c55ba7700 1 -- 172.21.15.7:0/84794066 learned_addr learned my addr 172.21.15.7:0/84794066 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.544+0000 7f6c56ba9700 1 --2- 172.21.15.7:0/84794066 >> v2:172.21.15.188:3300/0 conn(0x7f6c50063ba0 0x7f6c500c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.544+0000 7f6c563a8700 1 --2- 172.21.15.7:0/84794066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c500637d0 0x7f6c5006ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.544+0000 7f6c553a6700 1 -- 172.21.15.7:0/84794066 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2159594824 0 0) 0x7f6c5015b6c0 con 0x7f6c500c41e0 2024-01-27T02:43:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.545+0000 7f6c553a6700 1 -- 172.21.15.7:0/84794066 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c3c003570 con 0x7f6c500c41e0 2024-01-27T02:43:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.545+0000 7f6c563a8700 1 -- 172.21.15.7:0/84794066 >> v2:172.21.15.188:3300/0 conn(0x7f6c50063ba0 msgr2=0x7f6c500c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.545+0000 7f6c563a8700 1 --2- 172.21.15.7:0/84794066 >> v2:172.21.15.188:3300/0 conn(0x7f6c50063ba0 0x7f6c500c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.545+0000 7f6c563a8700 1 -- 172.21.15.7:0/84794066 >> v1:172.21.15.7:6789/0 conn(0x7f6c500c41e0 legacy=0x7f6c5015a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:34.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.545+0000 7f6c563a8700 1 -- 172.21.15.7:0/84794066 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5015b3e0 con 0x7f6c500637d0 2024-01-27T02:43:34.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.545+0000 7f6c56ba9700 1 --2- 172.21.15.7:0/84794066 >> v2:172.21.15.188:3300/0 conn(0x7f6c50063ba0 0x7f6c500c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:43:34.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.545+0000 7f6c563a8700 1 --2- 172.21.15.7:0/84794066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c500637d0 0x7f6c5006ebd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0098e0 tx=0x7f6c4c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba46fc28ab90ab1e server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:34.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.545+0000 7f6c553a6700 1 -- 172.21.15.7:0/84794066 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c011630 con 0x7f6c500637d0 2024-01-27T02:43:34.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.545+0000 7f6c553a6700 1 -- 172.21.15.7:0/84794066 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4c011790 con 0x7f6c500637d0 2024-01-27T02:43:34.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.545+0000 7f6c553a6700 1 -- 172.21.15.7:0/84794066 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c011b30 con 0x7f6c500637d0 2024-01-27T02:43:34.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.546+0000 7f6c5860c700 1 -- 172.21.15.7:0/84794066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c500637d0 msgr2=0x7f6c5006ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:34.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.546+0000 7f6c5860c700 1 --2- 172.21.15.7:0/84794066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c500637d0 0x7f6c5006ebd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0098e0 tx=0x7f6c4c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.546+0000 7f6c5860c700 1 -- 172.21.15.7:0/84794066 shutdown_connections 2024-01-27T02:43:34.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.546+0000 7f6c5860c700 1 --2- 172.21.15.7:0/84794066 >> v2:172.21.15.188:3300/0 conn(0x7f6c50063ba0 0x7f6c500c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.546+0000 7f6c5860c700 1 --2- 172.21.15.7:0/84794066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c500637d0 0x7f6c5006ebd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.546+0000 7f6c5860c700 1 -- 172.21.15.7:0/84794066 >> 172.21.15.7:0/84794066 conn(0x7f6c500cdc30 msgr2=0x7f6c500ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:34.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.546+0000 7f6c5860c700 1 -- 172.21.15.7:0/84794066 shutdown_connections 2024-01-27T02:43:34.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.546+0000 7f6c5860c700 1 -- 172.21.15.7:0/84794066 wait complete. 2024-01-27T02:43:34.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.547+0000 7f6c5860c700 1 Processor -- start 2024-01-27T02:43:34.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.547+0000 7f6c5860c700 1 -- start start 2024-01-27T02:43:34.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.548+0000 7f6c5860c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c500637d0 0x7f6c5016b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:34.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.548+0000 7f6c5860c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c50063ba0 0x7f6c500c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:34.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.548+0000 7f6c5860c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c500c41e0 0x7f6c50158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:34.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.548+0000 7f6c5860c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c500c5d60 con 0x7f6c50063ba0 2024-01-27T02:43:34.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.548+0000 7f6c5860c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c500ad6a0 con 0x7f6c500637d0 2024-01-27T02:43:34.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.548+0000 7f6c5860c700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c500c57a0 con 0x7f6c500c41e0 2024-01-27T02:43:34.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.548+0000 7f6c55ba7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c50063ba0 0x7f6c500c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:34.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.548+0000 7f6c55ba7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c50063ba0 0x7f6c500c4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34810/0 (socket says 172.21.15.7:34810) 2024-01-27T02:43:34.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.548+0000 7f6c55ba7700 1 -- 172.21.15.7:0/2156296270 learned_addr learned my addr 172.21.15.7:0/2156296270 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:34.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.548+0000 7f6c563a8700 1 --2- 172.21.15.7:0/2156296270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c500637d0 0x7f6c5016b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:34.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.548+0000 7f6c56ba9700 1 --2- 172.21.15.7:0/2156296270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c500c41e0 0x7f6c50158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:34.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.548+0000 7f6c55ba7700 1 -- 172.21.15.7:0/2156296270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c500c41e0 msgr2=0x7f6c50158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:34.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.548+0000 7f6c55ba7700 1 --2- 172.21.15.7:0/2156296270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c500c41e0 0x7f6c50158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.548+0000 7f6c55ba7700 1 -- 172.21.15.7:0/2156296270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c500637d0 msgr2=0x7f6c5016b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:34.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.548+0000 7f6c55ba7700 1 --2- 172.21.15.7:0/2156296270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c500637d0 0x7f6c5016b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.548+0000 7f6c55ba7700 1 -- 172.21.15.7:0/2156296270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c48009a20 con 0x7f6c50063ba0 2024-01-27T02:43:34.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.549+0000 7f6c55ba7700 1 --2- 172.21.15.7:0/2156296270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c50063ba0 0x7f6c500c4ce0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000c560 tx=0x7f6c40007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:34.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.549+0000 7f6c467fc700 1 -- 172.21.15.7:0/2156296270 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c40005c00 con 0x7f6c50063ba0 2024-01-27T02:43:34.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.549+0000 7f6c467fc700 1 -- 172.21.15.7:0/2156296270 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c40005d60 con 0x7f6c50063ba0 2024-01-27T02:43:34.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.549+0000 7f6c5860c700 1 -- 172.21.15.7:0/2156296270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4c009420 con 0x7f6c50063ba0 2024-01-27T02:43:34.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.549+0000 7f6c467fc700 1 -- 172.21.15.7:0/2156296270 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c40007300 con 0x7f6c50063ba0 2024-01-27T02:43:34.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.549+0000 7f6c5860c700 1 -- 172.21.15.7:0/2156296270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c500ad330 con 0x7f6c50063ba0 2024-01-27T02:43:34.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.550+0000 7f6c5860c700 1 -- 172.21.15.7:0/2156296270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c5006caf0 con 0x7f6c50063ba0 2024-01-27T02:43:34.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.550+0000 7f6c467fc700 1 -- 172.21.15.7:0/2156296270 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6c4001b3b0 con 0x7f6c50063ba0 2024-01-27T02:43:34.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.551+0000 7f6c467fc700 1 --2- 172.21.15.7:0/2156296270 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6c34066f30 0x7f6c340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:34.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.551+0000 7f6c467fc700 1 -- 172.21.15.7:0/2156296270 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6c40083b20 con 0x7f6c50063ba0 2024-01-27T02:43:34.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.551+0000 7f6c563a8700 1 --2- 172.21.15.7:0/2156296270 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6c34066f30 0x7f6c340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:34.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.551+0000 7f6c563a8700 1 --2- 172.21.15.7:0/2156296270 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6c34066f30 0x7f6c340693e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0091e0 tx=0x7f6c4c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:34.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.555+0000 7f6c467fc700 1 -- 172.21.15.7:0/2156296270 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c40050cf0 con 0x7f6c50063ba0 2024-01-27T02:43:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.752+0000 7f6c5860c700 1 -- 172.21.15.7:0/2156296270 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6c5004c710 con 0x7f6c34066f30 2024-01-27T02:43:34.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.761+0000 7f6c467fc700 1 -- 172.21.15.7:0/2156296270 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285590 (secure 0 0 0) 0x7f6c5004c710 con 0x7f6c34066f30 2024-01-27T02:43:34.765 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.764+0000 7f6c5860c700 1 -- 172.21.15.7:0/2156296270 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6c34066f30 msgr2=0x7f6c340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.764+0000 7f6c5860c700 1 --2- 172.21.15.7:0/2156296270 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6c34066f30 0x7f6c340693e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0091e0 tx=0x7f6c4c017000 comp rx=0 tx=0).stop 2024-01-27T02:43:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.764+0000 7f6c5860c700 1 -- 172.21.15.7:0/2156296270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c50063ba0 msgr2=0x7f6c500c4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.764+0000 7f6c5860c700 1 --2- 172.21.15.7:0/2156296270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c50063ba0 0x7f6c500c4ce0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000c560 tx=0x7f6c40007dd0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.764+0000 7f6c5860c700 1 -- 172.21.15.7:0/2156296270 shutdown_connections 2024-01-27T02:43:34.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.764+0000 7f6c5860c700 1 --2- 172.21.15.7:0/2156296270 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6c34066f30 0x7f6c340693e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.764+0000 7f6c5860c700 1 --2- 172.21.15.7:0/2156296270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c500c41e0 0x7f6c50158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.764+0000 7f6c5860c700 1 --2- 172.21.15.7:0/2156296270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c500637d0 0x7f6c5016b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.764+0000 7f6c5860c700 1 --2- 172.21.15.7:0/2156296270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c50063ba0 0x7f6c500c4ce0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.764+0000 7f6c5860c700 1 -- 172.21.15.7:0/2156296270 >> 172.21.15.7:0/2156296270 conn(0x7f6c500cdc30 msgr2=0x7f6c500aa8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:34.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.764+0000 7f6c5860c700 1 -- 172.21.15.7:0/2156296270 shutdown_connections 2024-01-27T02:43:34.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.765+0000 7f6c5860c700 1 -- 172.21.15.7:0/2156296270 wait complete. 2024-01-27T02:43:34.768 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:43:34.779 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":519,"stamp":"2024-01-27T02:43:33.742806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8979,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553520,"statfs":{"total":1600000000,"available":1590804860,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.579182"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.946170+0000","last_change":"2024-01-27T02:43:29.946073+0000","last_active":"2024-01-27T02:43:29.946170+0000","last_peered":"2024-01-27T02:43:29.946170+0000","last_clean":"2024-01-27T02:43:29.946170+0000","last_became_active":"2024-01-27T02:43:29.944886+0000","last_became_peered":"2024-01-27T02:43:29.944886+0000","last_unstale":"2024-01-27T02:43:29.946170+0000","last_undegraded":"2024-01-27T02:43:29.946170+0000","last_fullsized":"2024-01-27T02:43:29.946170+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:06:22.906823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.944105+0000","last_change":"2024-01-27T02:43:29.944039+0000","last_active":"2024-01-27T02:43:29.944105+0000","last_peered":"2024-01-27T02:43:29.944105+0000","last_clean":"2024-01-27T02:43:29.944105+0000","last_became_active":"2024-01-27T02:43:29.941027+0000","last_became_peered":"2024-01-27T02:43:29.941027+0000","last_unstale":"2024-01-27T02:43:29.944105+0000","last_undegraded":"2024-01-27T02:43:29.944105+0000","last_fullsized":"2024-01-27T02:43:29.944105+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:14:25.190169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.944952+0000","last_change":"2024-01-27T02:43:29.944069+0000","last_active":"2024-01-27T02:43:29.944952+0000","last_peered":"2024-01-27T02:43:29.944952+0000","last_clean":"2024-01-27T02:43:29.944952+0000","last_became_active":"2024-01-27T02:43:29.943641+0000","last_became_peered":"2024-01-27T02:43:29.943641+0000","last_unstale":"2024-01-27T02:43:29.944952+0000","last_undegraded":"2024-01-27T02:43:29.944952+0000","last_fullsized":"2024-01-27T02:43:29.944952+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:53:52.641917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.948729+0000","last_change":"2024-01-27T02:43:29.947964+0000","last_active":"2024-01-27T02:43:29.948729+0000","last_peered":"2024-01-27T02:43:29.948729+0000","last_clean":"2024-01-27T02:43:29.948729+0000","last_became_active":"2024-01-27T02:43:29.947592+0000","last_became_peered":"2024-01-27T02:43:29.947592+0000","last_unstale":"2024-01-27T02:43:29.948729+0000","last_undegraded":"2024-01-27T02:43:29.948729+0000","last_fullsized":"2024-01-27T02:43:29.948729+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:59:55.931719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.939097+0000","last_change":"2024-01-27T02:43:29.938413+0000","last_active":"2024-01-27T02:43:29.939097+0000","last_peered":"2024-01-27T02:43:29.939097+0000","last_clean":"2024-01-27T02:43:29.939097+0000","last_became_active":"2024-01-27T02:43:29.938165+0000","last_became_peered":"2024-01-27T02:43:29.938165+0000","last_unstale":"2024-01-27T02:43:29.939097+0000","last_undegraded":"2024-01-27T02:43:29.939097+0000","last_fullsized":"2024-01-27T02:43:29.939097+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:32:52.100147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.952297+0000","last_change":"2024-01-27T02:43:29.952223+0000","last_active":"2024-01-27T02:43:29.952297+0000","last_peered":"2024-01-27T02:43:29.952297+0000","last_clean":"2024-01-27T02:43:29.952297+0000","last_became_active":"2024-01-27T02:43:29.951943+0000","last_became_peered":"2024-01-27T02:43:29.951943+0000","last_unstale":"2024-01-27T02:43:29.952297+0000","last_undegraded":"2024-01-27T02:43:29.952297+0000","last_fullsized":"2024-01-27T02:43:29.952297+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:37:09.277025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.954144+0000","last_change":"2024-01-27T02:43:29.954098+0000","last_active":"2024-01-27T02:43:29.954144+0000","last_peered":"2024-01-27T02:43:29.954144+0000","last_clean":"2024-01-27T02:43:29.954144+0000","last_became_active":"2024-01-27T02:43:29.952882+0000","last_became_peered":"2024-01-27T02:43:29.952882+0000","last_unstale":"2024-01-27T02:43:29.954144+0000","last_undegraded":"2024-01-27T02:43:29.954144+0000","last_fullsized":"2024-01-27T02:43:29.954144+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:50:52.055538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.939116+0000","last_change":"2024-01-27T02:43:29.938435+0000","last_active":"2024-01-27T02:43:29.939116+0000","last_peered":"2024-01-27T02:43:29.939116+0000","last_clean":"2024-01-27T02:43:29.939116+0000","last_became_active":"2024-01-27T02:43:29.938195+0000","last_became_peered":"2024-01-27T02:43:29.938195+0000","last_unstale":"2024-01-27T02:43:29.939116+0000","last_undegraded":"2024-01-27T02:43:29.939116+0000","last_fullsized":"2024-01-27T02:43:29.939116+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:06.053165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.945800+0000","last_change":"2024-01-27T02:43:29.945741+0000","last_active":"2024-01-27T02:43:29.945800+0000","last_peered":"2024-01-27T02:43:29.945800+0000","last_clean":"2024-01-27T02:43:29.945800+0000","last_became_active":"2024-01-27T02:43:29.945356+0000","last_became_peered":"2024-01-27T02:43:29.945356+0000","last_unstale":"2024-01-27T02:43:29.945800+0000","last_undegraded":"2024-01-27T02:43:29.945800+0000","last_fullsized":"2024-01-27T02:43:29.945800+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:05:46.859143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.942953+0000","last_change":"2024-01-27T02:43:29.942807+0000","last_active":"2024-01-27T02:43:29.942953+0000","last_peered":"2024-01-27T02:43:29.942953+0000","last_clean":"2024-01-27T02:43:29.942953+0000","last_became_active":"2024-01-27T02:43:29.942529+0000","last_became_peered":"2024-01-27T02:43:29.942529+0000","last_unstale":"2024-01-27T02:43:29.942953+0000","last_undegraded":"2024-01-27T02:43:29.942953+0000","last_fullsized":"2024-01-27T02:43:29.942953+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:55:12.514206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.939598+0000","last_change":"2024-01-27T02:43:29.939520+0000","last_active":"2024-01-27T02:43:29.939598+0000","last_peered":"2024-01-27T02:43:29.939598+0000","last_clean":"2024-01-27T02:43:29.939598+0000","last_became_active":"2024-01-27T02:43:29.939290+0000","last_became_peered":"2024-01-27T02:43:29.939290+0000","last_unstale":"2024-01-27T02:43:29.939598+0000","last_undegraded":"2024-01-27T02:43:29.939598+0000","last_fullsized":"2024-01-27T02:43:29.939598+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:59:55.931719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.944869+0000","last_change":"2024-01-27T02:43:29.944691+0000","last_active":"2024-01-27T02:43:29.944869+0000","last_peered":"2024-01-27T02:43:29.944869+0000","last_clean":"2024-01-27T02:43:29.944869+0000","last_became_active":"2024-01-27T02:43:29.941386+0000","last_became_peered":"2024-01-27T02:43:29.941386+0000","last_unstale":"2024-01-27T02:43:29.944869+0000","last_undegraded":"2024-01-27T02:43:29.944869+0000","last_fullsized":"2024-01-27T02:43:29.944869+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:59:20.853498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.943197+0000","last_change":"2024-01-27T02:43:29.943130+0000","last_active":"2024-01-27T02:43:29.943197+0000","last_peered":"2024-01-27T02:43:29.943197+0000","last_clean":"2024-01-27T02:43:29.943197+0000","last_became_active":"2024-01-27T02:43:29.942896+0000","last_became_peered":"2024-01-27T02:43:29.942896+0000","last_unstale":"2024-01-27T02:43:29.943197+0000","last_undegraded":"2024-01-27T02:43:29.943197+0000","last_fullsized":"2024-01-27T02:43:29.943197+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:20:23.793438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.943143+0000","last_change":"2024-01-27T02:43:29.942426+0000","last_active":"2024-01-27T02:43:29.943143+0000","last_peered":"2024-01-27T02:43:29.943143+0000","last_clean":"2024-01-27T02:43:29.943143+0000","last_became_active":"2024-01-27T02:43:29.941598+0000","last_became_peered":"2024-01-27T02:43:29.941598+0000","last_unstale":"2024-01-27T02:43:29.943143+0000","last_undegraded":"2024-01-27T02:43:29.943143+0000","last_fullsized":"2024-01-27T02:43:29.943143+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:57:16.971601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.943081+0000","last_change":"2024-01-27T02:43:29.943008+0000","last_active":"2024-01-27T02:43:29.943081+0000","last_peered":"2024-01-27T02:43:29.943081+0000","last_clean":"2024-01-27T02:43:29.943081+0000","last_became_active":"2024-01-27T02:43:29.942776+0000","last_became_peered":"2024-01-27T02:43:29.942776+0000","last_unstale":"2024-01-27T02:43:29.943081+0000","last_undegraded":"2024-01-27T02:43:29.943081+0000","last_fullsized":"2024-01-27T02:43:29.943081+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:50:52.055538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.952834+0000","last_change":"2024-01-27T02:43:29.952299+0000","last_active":"2024-01-27T02:43:29.952834+0000","last_peered":"2024-01-27T02:43:29.952834+0000","last_clean":"2024-01-27T02:43:29.952834+0000","last_became_active":"2024-01-27T02:43:29.951352+0000","last_became_peered":"2024-01-27T02:43:29.951352+0000","last_unstale":"2024-01-27T02:43:29.952834+0000","last_undegraded":"2024-01-27T02:43:29.952834+0000","last_fullsized":"2024-01-27T02:43:29.952834+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:35:53.347541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.946553+0000","last_change":"2024-01-27T02:43:29.946151+0000","last_active":"2024-01-27T02:43:29.946553+0000","last_peered":"2024-01-27T02:43:29.946553+0000","last_clean":"2024-01-27T02:43:29.946553+0000","last_became_active":"2024-01-27T02:43:29.944510+0000","last_became_peered":"2024-01-27T02:43:29.944510+0000","last_unstale":"2024-01-27T02:43:29.946553+0000","last_undegraded":"2024-01-27T02:43:29.946553+0000","last_fullsized":"2024-01-27T02:43:29.946553+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:51:08.617993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.949322+0000","last_change":"2024-01-27T02:43:29.949245+0000","last_active":"2024-01-27T02:43:29.949322+0000","last_peered":"2024-01-27T02:43:29.949322+0000","last_clean":"2024-01-27T02:43:29.949322+0000","last_became_active":"2024-01-27T02:43:29.948964+0000","last_became_peered":"2024-01-27T02:43:29.948964+0000","last_unstale":"2024-01-27T02:43:29.949322+0000","last_undegraded":"2024-01-27T02:43:29.949322+0000","last_fullsized":"2024-01-27T02:43:29.949322+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:13:09.260726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.943932+0000","last_change":"2024-01-27T02:43:29.941796+0000","last_active":"2024-01-27T02:43:29.943932+0000","last_peered":"2024-01-27T02:43:29.943932+0000","last_clean":"2024-01-27T02:43:29.943932+0000","last_became_active":"2024-01-27T02:43:29.941526+0000","last_became_peered":"2024-01-27T02:43:29.941526+0000","last_unstale":"2024-01-27T02:43:29.943932+0000","last_undegraded":"2024-01-27T02:43:29.943932+0000","last_fullsized":"2024-01-27T02:43:29.943932+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:51:14.634758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.948614+0000","last_change":"2024-01-27T02:43:29.948543+0000","last_active":"2024-01-27T02:43:29.948614+0000","last_peered":"2024-01-27T02:43:29.948614+0000","last_clean":"2024-01-27T02:43:29.948614+0000","last_became_active":"2024-01-27T02:43:29.948330+0000","last_became_peered":"2024-01-27T02:43:29.948330+0000","last_unstale":"2024-01-27T02:43:29.948614+0000","last_undegraded":"2024-01-27T02:43:29.948614+0000","last_fullsized":"2024-01-27T02:43:29.948614+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:51:01.041063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.946535+0000","last_change":"2024-01-27T02:43:29.946064+0000","last_active":"2024-01-27T02:43:29.946535+0000","last_peered":"2024-01-27T02:43:29.946535+0000","last_clean":"2024-01-27T02:43:29.946535+0000","last_became_active":"2024-01-27T02:43:29.945781+0000","last_became_peered":"2024-01-27T02:43:29.945781+0000","last_unstale":"2024-01-27T02:43:29.946535+0000","last_undegraded":"2024-01-27T02:43:29.946535+0000","last_fullsized":"2024-01-27T02:43:29.946535+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:06.346984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.953333+0000","last_change":"2024-01-27T02:43:29.953282+0000","last_active":"2024-01-27T02:43:29.953333+0000","last_peered":"2024-01-27T02:43:29.953333+0000","last_clean":"2024-01-27T02:43:29.953333+0000","last_became_active":"2024-01-27T02:43:29.953013+0000","last_became_peered":"2024-01-27T02:43:29.953013+0000","last_unstale":"2024-01-27T02:43:29.953333+0000","last_undegraded":"2024-01-27T02:43:29.953333+0000","last_fullsized":"2024-01-27T02:43:29.953333+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:56:40.433794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.947795+0000","last_change":"2024-01-27T02:43:29.947691+0000","last_active":"2024-01-27T02:43:29.947795+0000","last_peered":"2024-01-27T02:43:29.947795+0000","last_clean":"2024-01-27T02:43:29.947795+0000","last_became_active":"2024-01-27T02:43:29.944481+0000","last_became_peered":"2024-01-27T02:43:29.944481+0000","last_unstale":"2024-01-27T02:43:29.947795+0000","last_undegraded":"2024-01-27T02:43:29.947795+0000","last_fullsized":"2024-01-27T02:43:29.947795+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:02:36.123442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.945657+0000","last_change":"2024-01-27T02:43:29.945592+0000","last_active":"2024-01-27T02:43:29.945657+0000","last_peered":"2024-01-27T02:43:29.945657+0000","last_clean":"2024-01-27T02:43:29.945657+0000","last_became_active":"2024-01-27T02:43:29.945047+0000","last_became_peered":"2024-01-27T02:43:29.945047+0000","last_unstale":"2024-01-27T02:43:29.945657+0000","last_undegraded":"2024-01-27T02:43:29.945657+0000","last_fullsized":"2024-01-27T02:43:29.945657+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:05:31.630090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.948187+0000","last_change":"2024-01-27T02:43:29.948120+0000","last_active":"2024-01-27T02:43:29.948187+0000","last_peered":"2024-01-27T02:43:29.948187+0000","last_clean":"2024-01-27T02:43:29.948187+0000","last_became_active":"2024-01-27T02:43:29.947476+0000","last_became_peered":"2024-01-27T02:43:29.947476+0000","last_unstale":"2024-01-27T02:43:29.948187+0000","last_undegraded":"2024-01-27T02:43:29.948187+0000","last_fullsized":"2024-01-27T02:43:29.948187+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:54:50.767945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.952778+0000","last_change":"2024-01-27T02:43:29.951674+0000","last_active":"2024-01-27T02:43:29.952778+0000","last_peered":"2024-01-27T02:43:29.952778+0000","last_clean":"2024-01-27T02:43:29.952778+0000","last_became_active":"2024-01-27T02:43:29.950652+0000","last_became_peered":"2024-01-27T02:43:29.950652+0000","last_unstale":"2024-01-27T02:43:29.952778+0000","last_undegraded":"2024-01-27T02:43:29.952778+0000","last_fullsized":"2024-01-27T02:43:29.952778+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:33:01.072368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.942218+0000","last_change":"2024-01-27T02:43:29.941960+0000","last_active":"2024-01-27T02:43:29.942218+0000","last_peered":"2024-01-27T02:43:29.942218+0000","last_clean":"2024-01-27T02:43:29.942218+0000","last_became_active":"2024-01-27T02:43:29.941698+0000","last_became_peered":"2024-01-27T02:43:29.941698+0000","last_unstale":"2024-01-27T02:43:29.942218+0000","last_undegraded":"2024-01-27T02:43:29.942218+0000","last_fullsized":"2024-01-27T02:43:29.942218+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:01:20.145502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.947820+0000","last_change":"2024-01-27T02:43:29.947751+0000","last_active":"2024-01-27T02:43:29.947820+0000","last_peered":"2024-01-27T02:43:29.947820+0000","last_clean":"2024-01-27T02:43:29.947820+0000","last_became_active":"2024-01-27T02:43:29.947488+0000","last_became_peered":"2024-01-27T02:43:29.947488+0000","last_unstale":"2024-01-27T02:43:29.947820+0000","last_undegraded":"2024-01-27T02:43:29.947820+0000","last_fullsized":"2024-01-27T02:43:29.947820+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:47:39.445834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.943164+0000","last_change":"2024-01-27T02:43:29.942600+0000","last_active":"2024-01-27T02:43:29.943164+0000","last_peered":"2024-01-27T02:43:29.943164+0000","last_clean":"2024-01-27T02:43:29.943164+0000","last_became_active":"2024-01-27T02:43:29.941968+0000","last_became_peered":"2024-01-27T02:43:29.941968+0000","last_unstale":"2024-01-27T02:43:29.943164+0000","last_undegraded":"2024-01-27T02:43:29.943164+0000","last_fullsized":"2024-01-27T02:43:29.943164+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:24:44.341787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.940875+0000","last_change":"2024-01-27T02:43:29.940668+0000","last_active":"2024-01-27T02:43:29.940875+0000","last_peered":"2024-01-27T02:43:29.940875+0000","last_clean":"2024-01-27T02:43:29.940875+0000","last_became_active":"2024-01-27T02:43:29.940434+0000","last_became_peered":"2024-01-27T02:43:29.940434+0000","last_unstale":"2024-01-27T02:43:29.940875+0000","last_undegraded":"2024-01-27T02:43:29.940875+0000","last_fullsized":"2024-01-27T02:43:29.940875+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:59:20.853498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.951047+0000","last_change":"2024-01-27T02:43:29.950981+0000","last_active":"2024-01-27T02:43:29.951047+0000","last_peered":"2024-01-27T02:43:29.951047+0000","last_clean":"2024-01-27T02:43:29.951047+0000","last_became_active":"2024-01-27T02:43:29.950782+0000","last_became_peered":"2024-01-27T02:43:29.950782+0000","last_unstale":"2024-01-27T02:43:29.951047+0000","last_undegraded":"2024-01-27T02:43:29.951047+0000","last_fullsized":"2024-01-27T02:43:29.951047+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:56:40.433794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.951117+0000","last_change":"2024-01-27T02:43:29.951001+0000","last_active":"2024-01-27T02:43:29.951117+0000","last_peered":"2024-01-27T02:43:29.951117+0000","last_clean":"2024-01-27T02:43:29.951117+0000","last_became_active":"2024-01-27T02:43:29.950475+0000","last_became_peered":"2024-01-27T02:43:29.950475+0000","last_unstale":"2024-01-27T02:43:29.951117+0000","last_undegraded":"2024-01-27T02:43:29.951117+0000","last_fullsized":"2024-01-27T02:43:29.951117+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:45:07.481390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.949935+0000","last_change":"2024-01-27T02:43:29.949661+0000","last_active":"2024-01-27T02:43:29.949935+0000","last_peered":"2024-01-27T02:43:29.949935+0000","last_clean":"2024-01-27T02:43:29.949935+0000","last_became_active":"2024-01-27T02:43:29.945212+0000","last_became_peered":"2024-01-27T02:43:29.945212+0000","last_unstale":"2024-01-27T02:43:29.949935+0000","last_undegraded":"2024-01-27T02:43:29.949935+0000","last_fullsized":"2024-01-27T02:43:29.949935+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:26:47.425780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"179'4","reported_seq":14,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.193267+0000","last_change":"2024-01-27T02:43:29.180404+0000","last_active":"2024-01-27T02:43:30.193267+0000","last_peered":"2024-01-27T02:43:30.193267+0000","last_clean":"2024-01-27T02:43:30.193267+0000","last_became_active":"2024-01-27T02:43:29.180149+0000","last_became_peered":"2024-01-27T02:43:29.180149+0000","last_unstale":"2024-01-27T02:43:30.193267+0000","last_undegraded":"2024-01-27T02:43:30.193267+0000","last_fullsized":"2024-01-27T02:43:30.193267+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:12:22.590221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.937923+0000","last_change":"2024-01-27T02:43:29.937637+0000","last_active":"2024-01-27T02:43:29.937923+0000","last_peered":"2024-01-27T02:43:29.937923+0000","last_clean":"2024-01-27T02:43:29.937923+0000","last_became_active":"2024-01-27T02:43:29.936865+0000","last_became_peered":"2024-01-27T02:43:29.936865+0000","last_unstale":"2024-01-27T02:43:29.937923+0000","last_undegraded":"2024-01-27T02:43:29.937923+0000","last_fullsized":"2024-01-27T02:43:29.937923+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:55:12.514206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"179'1","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.192400+0000","last_change":"2024-01-27T02:43:29.184179+0000","last_active":"2024-01-27T02:43:30.192400+0000","last_peered":"2024-01-27T02:43:30.192400+0000","last_clean":"2024-01-27T02:43:30.192400+0000","last_became_active":"2024-01-27T02:43:29.183417+0000","last_became_peered":"2024-01-27T02:43:29.183417+0000","last_unstale":"2024-01-27T02:43:30.192400+0000","last_undegraded":"2024-01-27T02:43:30.192400+0000","last_fullsized":"2024-01-27T02:43:30.192400+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:45:48.808912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.948937+0000","last_change":"2024-01-27T02:43:29.948866+0000","last_active":"2024-01-27T02:43:29.948937+0000","last_peered":"2024-01-27T02:43:29.948937+0000","last_clean":"2024-01-27T02:43:29.948937+0000","last_became_active":"2024-01-27T02:43:29.948624+0000","last_became_peered":"2024-01-27T02:43:29.948624+0000","last_unstale":"2024-01-27T02:43:29.948937+0000","last_undegraded":"2024-01-27T02:43:29.948937+0000","last_fullsized":"2024-01-27T02:43:29.948937+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:46:09.582015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"179'2","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.193688+0000","last_change":"2024-01-27T02:43:29.179956+0000","last_active":"2024-01-27T02:43:30.193688+0000","last_peered":"2024-01-27T02:43:30.193688+0000","last_clean":"2024-01-27T02:43:30.193688+0000","last_became_active":"2024-01-27T02:43:29.179706+0000","last_became_peered":"2024-01-27T02:43:29.179706+0000","last_unstale":"2024-01-27T02:43:30.193688+0000","last_undegraded":"2024-01-27T02:43:30.193688+0000","last_fullsized":"2024-01-27T02:43:30.193688+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:20:22.787969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.947792+0000","last_change":"2024-01-27T02:43:29.947721+0000","last_active":"2024-01-27T02:43:29.947792+0000","last_peered":"2024-01-27T02:43:29.947792+0000","last_clean":"2024-01-27T02:43:29.947792+0000","last_became_active":"2024-01-27T02:43:29.946465+0000","last_became_peered":"2024-01-27T02:43:29.946465+0000","last_unstale":"2024-01-27T02:43:29.947792+0000","last_undegraded":"2024-01-27T02:43:29.947792+0000","last_fullsized":"2024-01-27T02:43:29.947792+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:45:49.814381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"179'2","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.192921+0000","last_change":"2024-01-27T02:43:29.177284+0000","last_active":"2024-01-27T02:43:30.192921+0000","last_peered":"2024-01-27T02:43:30.192921+0000","last_clean":"2024-01-27T02:43:30.192921+0000","last_became_active":"2024-01-27T02:43:29.177030+0000","last_became_peered":"2024-01-27T02:43:29.177030+0000","last_unstale":"2024-01-27T02:43:30.192921+0000","last_undegraded":"2024-01-27T02:43:30.192921+0000","last_fullsized":"2024-01-27T02:43:30.192921+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:01:19.140032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.949888+0000","last_change":"2024-01-27T02:43:29.947077+0000","last_active":"2024-01-27T02:43:29.949888+0000","last_peered":"2024-01-27T02:43:29.949888+0000","last_clean":"2024-01-27T02:43:29.949888+0000","last_became_active":"2024-01-27T02:43:29.946810+0000","last_became_peered":"2024-01-27T02:43:29.946810+0000","last_unstale":"2024-01-27T02:43:29.949888+0000","last_undegraded":"2024-01-27T02:43:29.949888+0000","last_fullsized":"2024-01-27T02:43:29.949888+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:55:59.427217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"179'1","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.191060+0000","last_change":"2024-01-27T02:43:29.174746+0000","last_active":"2024-01-27T02:43:30.191060+0000","last_peered":"2024-01-27T02:43:30.191060+0000","last_clean":"2024-01-27T02:43:30.191060+0000","last_became_active":"2024-01-27T02:43:29.174151+0000","last_became_peered":"2024-01-27T02:43:29.174151+0000","last_unstale":"2024-01-27T02:43:30.191060+0000","last_undegraded":"2024-01-27T02:43:30.191060+0000","last_fullsized":"2024-01-27T02:43:30.191060+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T14:37:08.271555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.953857+0000","last_change":"2024-01-27T02:43:29.953801+0000","last_active":"2024-01-27T02:43:29.953857+0000","last_peered":"2024-01-27T02:43:29.953857+0000","last_clean":"2024-01-27T02:43:29.953857+0000","last_became_active":"2024-01-27T02:43:29.953484+0000","last_became_peered":"2024-01-27T02:43:29.953484+0000","last_unstale":"2024-01-27T02:43:29.953857+0000","last_undegraded":"2024-01-27T02:43:29.953857+0000","last_fullsized":"2024-01-27T02:43:29.953857+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:06.053165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"179'1","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.195848+0000","last_change":"2024-01-27T02:43:29.171966+0000","last_active":"2024-01-27T02:43:30.195848+0000","last_peered":"2024-01-27T02:43:30.195848+0000","last_clean":"2024-01-27T02:43:30.195848+0000","last_became_active":"2024-01-27T02:43:29.171694+0000","last_became_peered":"2024-01-27T02:43:29.171694+0000","last_unstale":"2024-01-27T02:43:30.195848+0000","last_undegraded":"2024-01-27T02:43:30.195848+0000","last_fullsized":"2024-01-27T02:43:30.195848+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:53:51.636448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.943815+0000","last_change":"2024-01-27T02:43:29.943742+0000","last_active":"2024-01-27T02:43:29.943815+0000","last_peered":"2024-01-27T02:43:29.943815+0000","last_clean":"2024-01-27T02:43:29.943815+0000","last_became_active":"2024-01-27T02:43:29.943243+0000","last_became_peered":"2024-01-27T02:43:29.943243+0000","last_unstale":"2024-01-27T02:43:29.943815+0000","last_undegraded":"2024-01-27T02:43:29.943815+0000","last_fullsized":"2024-01-27T02:43:29.943815+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:43:49.450067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.935189+0000","last_change":"2024-01-27T02:43:29.179484+0000","last_active":"2024-01-27T02:43:29.935189+0000","last_peered":"2024-01-27T02:43:29.935189+0000","last_clean":"2024-01-27T02:43:29.935189+0000","last_became_active":"2024-01-27T02:43:29.179282+0000","last_became_peered":"2024-01-27T02:43:29.179282+0000","last_unstale":"2024-01-27T02:43:29.935189+0000","last_undegraded":"2024-01-27T02:43:29.935189+0000","last_fullsized":"2024-01-27T02:43:29.935189+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:57:15.966132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.952537+0000","last_change":"2024-01-27T02:43:29.952480+0000","last_active":"2024-01-27T02:43:29.952537+0000","last_peered":"2024-01-27T02:43:29.952537+0000","last_clean":"2024-01-27T02:43:29.952537+0000","last_became_active":"2024-01-27T02:43:29.952241+0000","last_became_peered":"2024-01-27T02:43:29.952241+0000","last_unstale":"2024-01-27T02:43:29.952537+0000","last_undegraded":"2024-01-27T02:43:29.952537+0000","last_fullsized":"2024-01-27T02:43:29.952537+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:04:34.634885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.938798+0000","last_change":"2024-01-27T02:43:29.175577+0000","last_active":"2024-01-27T02:43:29.938798+0000","last_peered":"2024-01-27T02:43:29.938798+0000","last_clean":"2024-01-27T02:43:29.938798+0000","last_became_active":"2024-01-27T02:43:29.175317+0000","last_became_peered":"2024-01-27T02:43:29.175317+0000","last_unstale":"2024-01-27T02:43:29.938798+0000","last_undegraded":"2024-01-27T02:43:29.938798+0000","last_fullsized":"2024-01-27T02:43:29.938798+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:27:05.047696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.947040+0000","last_change":"2024-01-27T02:43:29.946965+0000","last_active":"2024-01-27T02:43:29.947040+0000","last_peered":"2024-01-27T02:43:29.947040+0000","last_clean":"2024-01-27T02:43:29.947040+0000","last_became_active":"2024-01-27T02:43:29.946653+0000","last_became_peered":"2024-01-27T02:43:29.946653+0000","last_unstale":"2024-01-27T02:43:29.947040+0000","last_undegraded":"2024-01-27T02:43:29.947040+0000","last_fullsized":"2024-01-27T02:43:29.947040+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:44:05.288565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.944258+0000","last_change":"2024-01-27T02:43:29.176200+0000","last_active":"2024-01-27T02:43:29.944258+0000","last_peered":"2024-01-27T02:43:29.944258+0000","last_clean":"2024-01-27T02:43:29.944258+0000","last_became_active":"2024-01-27T02:43:29.175549+0000","last_became_peered":"2024-01-27T02:43:29.175549+0000","last_unstale":"2024-01-27T02:43:29.944258+0000","last_undegraded":"2024-01-27T02:43:29.944258+0000","last_fullsized":"2024-01-27T02:43:29.944258+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:47:25.883035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.946812+0000","last_change":"2024-01-27T02:43:29.946694+0000","last_active":"2024-01-27T02:43:29.946812+0000","last_peered":"2024-01-27T02:43:29.946812+0000","last_clean":"2024-01-27T02:43:29.946812+0000","last_became_active":"2024-01-27T02:43:29.946386+0000","last_became_peered":"2024-01-27T02:43:29.946386+0000","last_unstale":"2024-01-27T02:43:29.946812+0000","last_undegraded":"2024-01-27T02:43:29.946812+0000","last_fullsized":"2024-01-27T02:43:29.946812+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:12:49.020814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"179'3","reported_seq":13,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.193520+0000","last_change":"2024-01-27T02:43:29.175735+0000","last_active":"2024-01-27T02:43:30.193520+0000","last_peered":"2024-01-27T02:43:30.193520+0000","last_clean":"2024-01-27T02:43:30.193520+0000","last_became_active":"2024-01-27T02:43:29.175458+0000","last_became_peered":"2024-01-27T02:43:29.175458+0000","last_unstale":"2024-01-27T02:43:30.193520+0000","last_undegraded":"2024-01-27T02:43:30.193520+0000","last_fullsized":"2024-01-27T02:43:30.193520+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:45:09.279278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.943382+0000","last_change":"2024-01-27T02:43:29.943316+0000","last_active":"2024-01-27T02:43:29.943382+0000","last_peered":"2024-01-27T02:43:29.943382+0000","last_clean":"2024-01-27T02:43:29.943382+0000","last_became_active":"2024-01-27T02:43:29.943046+0000","last_became_peered":"2024-01-27T02:43:29.943046+0000","last_unstale":"2024-01-27T02:43:29.943382+0000","last_undegraded":"2024-01-27T02:43:29.943382+0000","last_fullsized":"2024-01-27T02:43:29.943382+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:14:25.190169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"179'6","reported_seq":16,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.194303+0000","last_change":"2024-01-27T02:43:29.184266+0000","last_active":"2024-01-27T02:43:30.194303+0000","last_peered":"2024-01-27T02:43:30.194303+0000","last_clean":"2024-01-27T02:43:30.194303+0000","last_became_active":"2024-01-27T02:43:29.183979+0000","last_became_peered":"2024-01-27T02:43:29.183979+0000","last_unstale":"2024-01-27T02:43:30.194303+0000","last_undegraded":"2024-01-27T02:43:30.194303+0000","last_fullsized":"2024-01-27T02:43:30.194303+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:37:50.439488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.935916+0000","last_change":"2024-01-27T02:43:29.935843+0000","last_active":"2024-01-27T02:43:29.935916+0000","last_peered":"2024-01-27T02:43:29.935916+0000","last_clean":"2024-01-27T02:43:29.935916+0000","last_became_active":"2024-01-27T02:43:29.935571+0000","last_became_peered":"2024-01-27T02:43:29.935571+0000","last_unstale":"2024-01-27T02:43:29.935916+0000","last_undegraded":"2024-01-27T02:43:29.935916+0000","last_fullsized":"2024-01-27T02:43:29.935916+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T12:45:46.304037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"179'2","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.193865+0000","last_change":"2024-01-27T02:43:29.169880+0000","last_active":"2024-01-27T02:43:30.193865+0000","last_peered":"2024-01-27T02:43:30.193865+0000","last_clean":"2024-01-27T02:43:30.193865+0000","last_became_active":"2024-01-27T02:43:29.169617+0000","last_became_peered":"2024-01-27T02:43:29.169617+0000","last_unstale":"2024-01-27T02:43:30.193865+0000","last_undegraded":"2024-01-27T02:43:30.193865+0000","last_fullsized":"2024-01-27T02:43:30.193865+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:51:00.035593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.950762+0000","last_change":"2024-01-27T02:43:29.950684+0000","last_active":"2024-01-27T02:43:29.950762+0000","last_peered":"2024-01-27T02:43:29.950762+0000","last_clean":"2024-01-27T02:43:29.950762+0000","last_became_active":"2024-01-27T02:43:29.950409+0000","last_became_peered":"2024-01-27T02:43:29.950409+0000","last_unstale":"2024-01-27T02:43:29.950762+0000","last_undegraded":"2024-01-27T02:43:29.950762+0000","last_fullsized":"2024-01-27T02:43:29.950762+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:38:09.031394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.934755+0000","last_change":"2024-01-27T02:43:29.181779+0000","last_active":"2024-01-27T02:43:29.934755+0000","last_peered":"2024-01-27T02:43:29.934755+0000","last_clean":"2024-01-27T02:43:29.934755+0000","last_became_active":"2024-01-27T02:43:29.181019+0000","last_became_peered":"2024-01-27T02:43:29.181019+0000","last_unstale":"2024-01-27T02:43:29.934755+0000","last_undegraded":"2024-01-27T02:43:29.934755+0000","last_fullsized":"2024-01-27T02:43:29.934755+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:32:24.137664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.945997+0000","last_change":"2024-01-27T02:43:29.945881+0000","last_active":"2024-01-27T02:43:29.945997+0000","last_peered":"2024-01-27T02:43:29.945997+0000","last_clean":"2024-01-27T02:43:29.945997+0000","last_became_active":"2024-01-27T02:43:29.945624+0000","last_became_peered":"2024-01-27T02:43:29.945624+0000","last_unstale":"2024-01-27T02:43:29.945997+0000","last_undegraded":"2024-01-27T02:43:29.945997+0000","last_fullsized":"2024-01-27T02:43:29.945997+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:23:41.383567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.939631+0000","last_change":"2024-01-27T02:43:29.178721+0000","last_active":"2024-01-27T02:43:29.939631+0000","last_peered":"2024-01-27T02:43:29.939631+0000","last_clean":"2024-01-27T02:43:29.939631+0000","last_became_active":"2024-01-27T02:43:29.178413+0000","last_became_peered":"2024-01-27T02:43:29.178413+0000","last_unstale":"2024-01-27T02:43:29.939631+0000","last_undegraded":"2024-01-27T02:43:29.939631+0000","last_fullsized":"2024-01-27T02:43:29.939631+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:45:48.808912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.954619+0000","last_change":"2024-01-27T02:43:29.954533+0000","last_active":"2024-01-27T02:43:29.954619+0000","last_peered":"2024-01-27T02:43:29.954619+0000","last_clean":"2024-01-27T02:43:29.954619+0000","last_became_active":"2024-01-27T02:43:29.954357+0000","last_became_peered":"2024-01-27T02:43:29.954357+0000","last_unstale":"2024-01-27T02:43:29.954619+0000","last_undegraded":"2024-01-27T02:43:29.954619+0000","last_fullsized":"2024-01-27T02:43:29.954619+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:56:17.907654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"179'8","reported_seq":18,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.193228+0000","last_change":"2024-01-27T02:43:29.183819+0000","last_active":"2024-01-27T02:43:30.193228+0000","last_peered":"2024-01-27T02:43:30.193228+0000","last_clean":"2024-01-27T02:43:30.193228+0000","last_became_active":"2024-01-27T02:43:29.183572+0000","last_became_peered":"2024-01-27T02:43:29.183572+0000","last_unstale":"2024-01-27T02:43:30.193228+0000","last_undegraded":"2024-01-27T02:43:30.193228+0000","last_fullsized":"2024-01-27T02:43:30.193228+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:57:15.966132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.948765+0000","last_change":"2024-01-27T02:43:29.948243+0000","last_active":"2024-01-27T02:43:29.948765+0000","last_peered":"2024-01-27T02:43:29.948765+0000","last_clean":"2024-01-27T02:43:29.948765+0000","last_became_active":"2024-01-27T02:43:29.947903+0000","last_became_peered":"2024-01-27T02:43:29.947903+0000","last_unstale":"2024-01-27T02:43:29.948765+0000","last_undegraded":"2024-01-27T02:43:29.948765+0000","last_fullsized":"2024-01-27T02:43:29.948765+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:32:52.100147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"179'1","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.189644+0000","last_change":"2024-01-27T02:43:29.176120+0000","last_active":"2024-01-27T02:43:30.189644+0000","last_peered":"2024-01-27T02:43:30.189644+0000","last_clean":"2024-01-27T02:43:30.189644+0000","last_became_active":"2024-01-27T02:43:29.175827+0000","last_became_peered":"2024-01-27T02:43:29.175827+0000","last_unstale":"2024-01-27T02:43:30.189644+0000","last_undegraded":"2024-01-27T02:43:30.189644+0000","last_fullsized":"2024-01-27T02:43:30.189644+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:12:05.341515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.937123+0000","last_change":"2024-01-27T02:43:29.937069+0000","last_active":"2024-01-27T02:43:29.937123+0000","last_peered":"2024-01-27T02:43:29.937123+0000","last_clean":"2024-01-27T02:43:29.937123+0000","last_became_active":"2024-01-27T02:43:29.936846+0000","last_became_peered":"2024-01-27T02:43:29.936846+0000","last_unstale":"2024-01-27T02:43:29.937123+0000","last_undegraded":"2024-01-27T02:43:29.937123+0000","last_fullsized":"2024-01-27T02:43:29.937123+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:41:06.546746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"179'2","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.189671+0000","last_change":"2024-01-27T02:43:29.181342+0000","last_active":"2024-01-27T02:43:30.189671+0000","last_peered":"2024-01-27T02:43:30.189671+0000","last_clean":"2024-01-27T02:43:30.189671+0000","last_became_active":"2024-01-27T02:43:29.180819+0000","last_became_peered":"2024-01-27T02:43:29.180819+0000","last_unstale":"2024-01-27T02:43:30.189671+0000","last_undegraded":"2024-01-27T02:43:30.189671+0000","last_fullsized":"2024-01-27T02:43:30.189671+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:40:10.722633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.953725+0000","last_change":"2024-01-27T02:43:29.953664+0000","last_active":"2024-01-27T02:43:29.953725+0000","last_peered":"2024-01-27T02:43:29.953725+0000","last_clean":"2024-01-27T02:43:29.953725+0000","last_became_active":"2024-01-27T02:43:29.953132+0000","last_became_peered":"2024-01-27T02:43:29.953132+0000","last_unstale":"2024-01-27T02:43:29.953725+0000","last_undegraded":"2024-01-27T02:43:29.953725+0000","last_fullsized":"2024-01-27T02:43:29.953725+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:05:46.859143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"179'1","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.191405+0000","last_change":"2024-01-27T02:43:29.185493+0000","last_active":"2024-01-27T02:43:30.191405+0000","last_peered":"2024-01-27T02:43:30.191405+0000","last_clean":"2024-01-27T02:43:30.191405+0000","last_became_active":"2024-01-27T02:43:29.184932+0000","last_became_peered":"2024-01-27T02:43:29.184932+0000","last_unstale":"2024-01-27T02:43:30.191405+0000","last_undegraded":"2024-01-27T02:43:30.191405+0000","last_fullsized":"2024-01-27T02:43:30.191405+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:19:12.257608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.952799+0000","last_change":"2024-01-27T02:43:29.952105+0000","last_active":"2024-01-27T02:43:29.952799+0000","last_peered":"2024-01-27T02:43:29.952799+0000","last_clean":"2024-01-27T02:43:29.952799+0000","last_became_active":"2024-01-27T02:43:29.951012+0000","last_became_peered":"2024-01-27T02:43:29.951012+0000","last_unstale":"2024-01-27T02:43:29.952799+0000","last_undegraded":"2024-01-27T02:43:29.952799+0000","last_fullsized":"2024-01-27T02:43:29.952799+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T09:59:37.749331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.939242+0000","last_change":"2024-01-27T02:43:29.183016+0000","last_active":"2024-01-27T02:43:29.939242+0000","last_peered":"2024-01-27T02:43:29.939242+0000","last_clean":"2024-01-27T02:43:29.939242+0000","last_became_active":"2024-01-27T02:43:29.182744+0000","last_became_peered":"2024-01-27T02:43:29.182744+0000","last_unstale":"2024-01-27T02:43:29.939242+0000","last_undegraded":"2024-01-27T02:43:29.939242+0000","last_fullsized":"2024-01-27T02:43:29.939242+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T07:21:56.830916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.943180+0000","last_change":"2024-01-27T02:43:29.943074+0000","last_active":"2024-01-27T02:43:29.943180+0000","last_peered":"2024-01-27T02:43:29.943180+0000","last_clean":"2024-01-27T02:43:29.943180+0000","last_became_active":"2024-01-27T02:43:29.942846+0000","last_became_peered":"2024-01-27T02:43:29.942846+0000","last_unstale":"2024-01-27T02:43:29.943180+0000","last_undegraded":"2024-01-27T02:43:29.943180+0000","last_fullsized":"2024-01-27T02:43:29.943180+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:37:51.444958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"179'2","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.193825+0000","last_change":"2024-01-27T02:43:29.179334+0000","last_active":"2024-01-27T02:43:30.193825+0000","last_peered":"2024-01-27T02:43:30.193825+0000","last_clean":"2024-01-27T02:43:30.193825+0000","last_became_active":"2024-01-27T02:43:29.178831+0000","last_became_peered":"2024-01-27T02:43:29.178831+0000","last_unstale":"2024-01-27T02:43:30.193825+0000","last_undegraded":"2024-01-27T02:43:30.193825+0000","last_fullsized":"2024-01-27T02:43:30.193825+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:37:50.439488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.949858+0000","last_change":"2024-01-27T02:43:29.946529+0000","last_active":"2024-01-27T02:43:29.949858+0000","last_peered":"2024-01-27T02:43:29.949858+0000","last_clean":"2024-01-27T02:43:29.949858+0000","last_became_active":"2024-01-27T02:43:29.945124+0000","last_became_peered":"2024-01-27T02:43:29.945124+0000","last_unstale":"2024-01-27T02:43:29.949858+0000","last_undegraded":"2024-01-27T02:43:29.949858+0000","last_fullsized":"2024-01-27T02:43:29.949858+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:06:22.906823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.939587+0000","last_change":"2024-01-27T02:43:29.182867+0000","last_active":"2024-01-27T02:43:29.939587+0000","last_peered":"2024-01-27T02:43:29.939587+0000","last_clean":"2024-01-27T02:43:29.939587+0000","last_became_active":"2024-01-27T02:43:29.182604+0000","last_became_peered":"2024-01-27T02:43:29.182604+0000","last_unstale":"2024-01-27T02:43:29.939587+0000","last_undegraded":"2024-01-27T02:43:29.939587+0000","last_fullsized":"2024-01-27T02:43:29.939587+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:06:23.512245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.946876+0000","last_change":"2024-01-27T02:43:29.946674+0000","last_active":"2024-01-27T02:43:29.946876+0000","last_peered":"2024-01-27T02:43:29.946876+0000","last_clean":"2024-01-27T02:43:29.946876+0000","last_became_active":"2024-01-27T02:43:29.945198+0000","last_became_peered":"2024-01-27T02:43:29.945198+0000","last_unstale":"2024-01-27T02:43:29.946876+0000","last_undegraded":"2024-01-27T02:43:29.946876+0000","last_fullsized":"2024-01-27T02:43:29.946876+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:38:09.031394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.941364+0000","last_change":"2024-01-27T02:43:29.176997+0000","last_active":"2024-01-27T02:43:29.941364+0000","last_peered":"2024-01-27T02:43:29.941364+0000","last_clean":"2024-01-27T02:43:29.941364+0000","last_became_active":"2024-01-27T02:43:29.176143+0000","last_became_peered":"2024-01-27T02:43:29.176143+0000","last_unstale":"2024-01-27T02:43:29.941364+0000","last_undegraded":"2024-01-27T02:43:29.941364+0000","last_fullsized":"2024-01-27T02:43:29.941364+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:54:49.762475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.943957+0000","last_change":"2024-01-27T02:43:29.943410+0000","last_active":"2024-01-27T02:43:29.943957+0000","last_peered":"2024-01-27T02:43:29.943957+0000","last_clean":"2024-01-27T02:43:29.943957+0000","last_became_active":"2024-01-27T02:43:29.943179+0000","last_became_peered":"2024-01-27T02:43:29.943179+0000","last_unstale":"2024-01-27T02:43:29.943957+0000","last_undegraded":"2024-01-27T02:43:29.943957+0000","last_fullsized":"2024-01-27T02:43:29.943957+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:01:17.185547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.940325+0000","last_change":"2024-01-27T02:43:29.185642+0000","last_active":"2024-01-27T02:43:29.940325+0000","last_peered":"2024-01-27T02:43:29.940325+0000","last_clean":"2024-01-27T02:43:29.940325+0000","last_became_active":"2024-01-27T02:43:29.185247+0000","last_became_peered":"2024-01-27T02:43:29.185247+0000","last_unstale":"2024-01-27T02:43:29.940325+0000","last_undegraded":"2024-01-27T02:43:29.940325+0000","last_fullsized":"2024-01-27T02:43:29.940325+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:32:24.137664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.949053+0000","last_change":"2024-01-27T02:43:29.948584+0000","last_active":"2024-01-27T02:43:29.949053+0000","last_peered":"2024-01-27T02:43:29.949053+0000","last_clean":"2024-01-27T02:43:29.949053+0000","last_became_active":"2024-01-27T02:43:29.945982+0000","last_became_peered":"2024-01-27T02:43:29.945982+0000","last_unstale":"2024-01-27T02:43:29.949053+0000","last_undegraded":"2024-01-27T02:43:29.949053+0000","last_fullsized":"2024-01-27T02:43:29.949053+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:23:43.836258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"179'2","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:30.190556+0000","last_change":"2024-01-27T02:43:29.182305+0000","last_active":"2024-01-27T02:43:30.190556+0000","last_peered":"2024-01-27T02:43:30.190556+0000","last_clean":"2024-01-27T02:43:30.190556+0000","last_became_active":"2024-01-27T02:43:29.181295+0000","last_became_peered":"2024-01-27T02:43:29.181295+0000","last_unstale":"2024-01-27T02:43:30.190556+0000","last_undegraded":"2024-01-27T02:43:30.190556+0000","last_fullsized":"2024-01-27T02:43:30.190556+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:12:22.590221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.953982+0000","last_change":"2024-01-27T02:43:29.953927+0000","last_active":"2024-01-27T02:43:29.953982+0000","last_peered":"2024-01-27T02:43:29.953982+0000","last_clean":"2024-01-27T02:43:29.953982+0000","last_became_active":"2024-01-27T02:43:29.952556+0000","last_became_peered":"2024-01-27T02:43:29.952556+0000","last_unstale":"2024-01-27T02:43:29.953982+0000","last_undegraded":"2024-01-27T02:43:29.953982+0000","last_fullsized":"2024-01-27T02:43:29.953982+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T06:15:03.662522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.941116+0000","last_change":"2024-01-27T02:43:29.180535+0000","last_active":"2024-01-27T02:43:29.941116+0000","last_peered":"2024-01-27T02:43:29.941116+0000","last_clean":"2024-01-27T02:43:29.941116+0000","last_became_active":"2024-01-27T02:43:29.180044+0000","last_became_peered":"2024-01-27T02:43:29.180044+0000","last_unstale":"2024-01-27T02:43:29.941116+0000","last_undegraded":"2024-01-27T02:43:29.941116+0000","last_fullsized":"2024-01-27T02:43:29.941116+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T11:46:08.576546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.949035+0000","last_change":"2024-01-27T02:43:29.948966+0000","last_active":"2024-01-27T02:43:29.949035+0000","last_peered":"2024-01-27T02:43:29.949035+0000","last_clean":"2024-01-27T02:43:29.949035+0000","last_became_active":"2024-01-27T02:43:29.948633+0000","last_became_peered":"2024-01-27T02:43:29.948633+0000","last_unstale":"2024-01-27T02:43:29.949035+0000","last_undegraded":"2024-01-27T02:43:29.949035+0000","last_fullsized":"2024-01-27T02:43:29.949035+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:40:11.728102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.935294+0000","last_change":"2024-01-27T02:43:29.182415+0000","last_active":"2024-01-27T02:43:29.935294+0000","last_peered":"2024-01-27T02:43:29.935294+0000","last_clean":"2024-01-27T02:43:29.935294+0000","last_became_active":"2024-01-27T02:43:29.182106+0000","last_became_peered":"2024-01-27T02:43:29.182106+0000","last_unstale":"2024-01-27T02:43:29.935294+0000","last_undegraded":"2024-01-27T02:43:29.935294+0000","last_fullsized":"2024-01-27T02:43:29.935294+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:37:50.439488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.942793+0000","last_change":"2024-01-27T02:43:29.942716+0000","last_active":"2024-01-27T02:43:29.942793+0000","last_peered":"2024-01-27T02:43:29.942793+0000","last_clean":"2024-01-27T02:43:29.942793+0000","last_became_active":"2024-01-27T02:43:29.942440+0000","last_became_peered":"2024-01-27T02:43:29.942440+0000","last_unstale":"2024-01-27T02:43:29.942793+0000","last_undegraded":"2024-01-27T02:43:29.942793+0000","last_fullsized":"2024-01-27T02:43:29.942793+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T03:06:22.985377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.933841+0000","last_change":"2024-01-27T02:43:29.180412+0000","last_active":"2024-01-27T02:43:29.933841+0000","last_peered":"2024-01-27T02:43:29.933841+0000","last_clean":"2024-01-27T02:43:29.933841+0000","last_became_active":"2024-01-27T02:43:29.180108+0000","last_became_peered":"2024-01-27T02:43:29.180108+0000","last_unstale":"2024-01-27T02:43:29.933841+0000","last_undegraded":"2024-01-27T02:43:29.933841+0000","last_fullsized":"2024-01-27T02:43:29.933841+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:19:12.257608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.951099+0000","last_change":"2024-01-27T02:43:29.950832+0000","last_active":"2024-01-27T02:43:29.951099+0000","last_peered":"2024-01-27T02:43:29.951099+0000","last_clean":"2024-01-27T02:43:29.951099+0000","last_became_active":"2024-01-27T02:43:29.950130+0000","last_became_peered":"2024-01-27T02:43:29.950130+0000","last_unstale":"2024-01-27T02:43:29.951099+0000","last_undegraded":"2024-01-27T02:43:29.951099+0000","last_fullsized":"2024-01-27T02:43:29.951099+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:28:40.712001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.937243+0000","last_change":"2024-01-27T02:43:29.183590+0000","last_active":"2024-01-27T02:43:29.937243+0000","last_peered":"2024-01-27T02:43:29.937243+0000","last_clean":"2024-01-27T02:43:29.937243+0000","last_became_active":"2024-01-27T02:43:29.183285+0000","last_became_peered":"2024-01-27T02:43:29.183285+0000","last_unstale":"2024-01-27T02:43:29.937243+0000","last_undegraded":"2024-01-27T02:43:29.937243+0000","last_fullsized":"2024-01-27T02:43:29.937243+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:53:51.636448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.952035+0000","last_change":"2024-01-27T02:43:29.951984+0000","last_active":"2024-01-27T02:43:29.952035+0000","last_peered":"2024-01-27T02:43:29.952035+0000","last_clean":"2024-01-27T02:43:29.952035+0000","last_became_active":"2024-01-27T02:43:29.951822+0000","last_became_peered":"2024-01-27T02:43:29.951822+0000","last_unstale":"2024-01-27T02:43:29.952035+0000","last_undegraded":"2024-01-27T02:43:29.952035+0000","last_fullsized":"2024-01-27T02:43:29.952035+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T05:05:31.630090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.939758+0000","last_change":"2024-01-27T02:43:29.173228+0000","last_active":"2024-01-27T02:43:29.939758+0000","last_peered":"2024-01-27T02:43:29.939758+0000","last_clean":"2024-01-27T02:43:29.939758+0000","last_became_active":"2024-01-27T02:43:29.172707+0000","last_became_peered":"2024-01-27T02:43:29.172707+0000","last_unstale":"2024-01-27T02:43:29.939758+0000","last_undegraded":"2024-01-27T02:43:29.939758+0000","last_fullsized":"2024-01-27T02:43:29.939758+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:57:15.966132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.946388+0000","last_change":"2024-01-27T02:43:29.944901+0000","last_active":"2024-01-27T02:43:29.946388+0000","last_peered":"2024-01-27T02:43:29.946388+0000","last_clean":"2024-01-27T02:43:29.946388+0000","last_became_active":"2024-01-27T02:43:29.944603+0000","last_became_peered":"2024-01-27T02:43:29.944603+0000","last_unstale":"2024-01-27T02:43:29.946388+0000","last_undegraded":"2024-01-27T02:43:29.946388+0000","last_fullsized":"2024-01-27T02:43:29.946388+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T10:58:23.934270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.936985+0000","last_change":"2024-01-27T02:43:29.182216+0000","last_active":"2024-01-27T02:43:29.936985+0000","last_peered":"2024-01-27T02:43:29.936985+0000","last_clean":"2024-01-27T02:43:29.936985+0000","last_became_active":"2024-01-27T02:43:29.181943+0000","last_became_peered":"2024-01-27T02:43:29.181943+0000","last_unstale":"2024-01-27T02:43:29.936985+0000","last_undegraded":"2024-01-27T02:43:29.936985+0000","last_fullsized":"2024-01-27T02:43:29.936985+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:24:43.336318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.949145+0000","last_change":"2024-01-27T02:43:29.949087+0000","last_active":"2024-01-27T02:43:29.949145+0000","last_peered":"2024-01-27T02:43:29.949145+0000","last_clean":"2024-01-27T02:43:29.949145+0000","last_became_active":"2024-01-27T02:43:29.948861+0000","last_became_peered":"2024-01-27T02:43:29.948861+0000","last_unstale":"2024-01-27T02:43:29.949145+0000","last_undegraded":"2024-01-27T02:43:29.949145+0000","last_fullsized":"2024-01-27T02:43:29.949145+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:53:56.469059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.935934+0000","last_change":"2024-01-27T02:43:29.179446+0000","last_active":"2024-01-27T02:43:29.935934+0000","last_peered":"2024-01-27T02:43:29.935934+0000","last_clean":"2024-01-27T02:43:29.935934+0000","last_became_active":"2024-01-27T02:43:29.179149+0000","last_became_peered":"2024-01-27T02:43:29.179149+0000","last_unstale":"2024-01-27T02:43:29.935934+0000","last_undegraded":"2024-01-27T02:43:29.935934+0000","last_fullsized":"2024-01-27T02:43:29.935934+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T04:54:49.762475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.951079+0000","last_change":"2024-01-27T02:43:29.949855+0000","last_active":"2024-01-27T02:43:29.951079+0000","last_peered":"2024-01-27T02:43:29.951079+0000","last_clean":"2024-01-27T02:43:29.951079+0000","last_became_active":"2024-01-27T02:43:29.946034+0000","last_became_peered":"2024-01-27T02:43:29.946034+0000","last_unstale":"2024-01-27T02:43:29.951079+0000","last_undegraded":"2024-01-27T02:43:29.951079+0000","last_fullsized":"2024-01-27T02:43:29.951079+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":178,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:29.162311+0000","last_clean_scrub_stamp":"2024-01-27T02:43:29.162311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T08:57:29.655912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.937372+0000","last_change":"2024-01-27T02:43:29.179715+0000","last_active":"2024-01-27T02:43:29.937372+0000","last_peered":"2024-01-27T02:43:29.937372+0000","last_clean":"2024-01-27T02:43:29.937372+0000","last_became_active":"2024-01-27T02:43:29.178851+0000","last_became_peered":"2024-01-27T02:43:29.178851+0000","last_unstale":"2024-01-27T02:43:29.937372+0000","last_undegraded":"2024-01-27T02:43:29.937372+0000","last_fullsized":"2024-01-27T02:43:29.937372+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:43:28.156841+0000","last_clean_scrub_stamp":"2024-01-27T02:43:28.156841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:01:19.140032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":302,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-01-27T02:43:29.939677+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:43:29.939677+0000","last_peered":"2024-01-27T02:43:29.939677+0000","last_clean":"2024-01-27T02:43:29.939677+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:43:29.939677+0000","last_undegraded":"2024-01-27T02:43:29.939677+0000","last_fullsized":"2024-01-27T02:43:29.939677+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574989,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1554,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193757,"statfs":{"total":200000000,"available":198407730,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.32800000000000001},"max":{"1min":0.80600000000000005,"5min":0.83499999999999996,"15min":0.85499999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59899999999999998,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.30099999999999999},"max":{"1min":0.84199999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.57199999999999995}]},{"osd":1,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.47699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74399999999999999,"5min":0.78800000000000003,"15min":0.82399999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.57399999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.34999999999999998},"max":{"1min":0.81599999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.54000000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.38400000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.86399999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.75,"5min":0.80300000000000005,"15min":0.83899999999999997},"last":0.443}]},{"osd":3,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76500000000000001,"5min":0.78900000000000003,"15min":0.90300000000000002},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.63800000000000001,"15min":0.61199999999999999},"min":{"1min":0.47599999999999998,"5min":0.41799999999999998,"15min":0.40999999999999998},"max":{"1min":0.77400000000000002,"5min":0.81599999999999995,"15min":0.88600000000000001},"last":0.55100000000000005}]},{"osd":4,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48199999999999998,"15min":0.48399999999999999},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67300000000000004,"5min":0.73899999999999999,"15min":0.78600000000000003},"last":0.5},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52000000000000002,"15min":0.51000000000000001},"min":{"1min":0.36699999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.80000000000000004},"last":0.42899999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.39600000000000002,"5min":0.32400000000000001,"15min":0.307},"max":{"1min":0.746,"5min":0.79500000000000004,"15min":0.85399999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.42499999999999999,"5min":0.33900000000000002,"15min":0.33500000000000002},"max":{"1min":0.76600000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45400000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60199999999999998,"15min":0.59299999999999997},"min":{"1min":0.38900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.78900000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71799999999999997,"5min":0.83499999999999996,"15min":0.90800000000000003},"last":0.34499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574989,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":976,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194336,"statfs":{"total":200000000,"available":199000326,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60099999999999998,"15min":0.57899999999999996},"min":{"1min":0.38800000000000001,"5min":0.375,"15min":0.29999999999999999},"max":{"1min":0.78700000000000003,"5min":1.0409999999999999,"15min":4.125},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58799999999999997,"15min":0.56499999999999995},"min":{"1min":0.435,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.753,"5min":0.96599999999999997,"15min":1.216},"last":0.50600000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42599999999999999,"5min":0.376,"15min":0.34100000000000003},"max":{"1min":0.67800000000000005,"5min":0.95499999999999996,"15min":8.2699999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59699999999999998,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78000000000000003,"5min":1.0129999999999999,"15min":26.411999999999999},"last":0.56499999999999995}]},{"osd":2,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.627},"min":{"1min":0.51900000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73699999999999999,"5min":0.78300000000000003,"15min":17.143999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61499999999999999,"15min":0.60899999999999999},"min":{"1min":0.503,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73199999999999998,"5min":0.85799999999999998,"15min":1.76},"last":0.65300000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.47699999999999998,"5min":0.39400000000000002,"15min":0.376},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":4.3689999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.41199999999999998},"max":{"1min":0.77400000000000002,"5min":0.81699999999999995,"15min":2.1970000000000001},"last":0.79100000000000004}]},{"osd":4,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47099999999999997,"15min":0.48099999999999998},"min":{"1min":0.35299999999999998,"5min":0.311,"15min":0.30199999999999999},"max":{"1min":0.61499999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78900000000000003,"5min":0.84899999999999998,"15min":0.89600000000000002},"last":0.754}]},{"osd":5,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.39400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.65400000000000003,"5min":0.80800000000000005,"15min":0.81299999999999994},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55200000000000005,"15min":0.57699999999999996},"min":{"1min":0.38300000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76400000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.86699999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57799999999999996,"15min":0.59599999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.35999999999999999},"max":{"1min":0.73599999999999999,"5min":0.88700000000000001,"15min":1.0009999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54900000000000004,"15min":0.56799999999999995},"min":{"1min":0.40699999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.749,"5min":0.82899999999999996,"15min":1.038},"last":0.90200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574989,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":976,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194336,"statfs":{"total":200000000,"available":199000314,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40300000000000002,"15min":0.38900000000000001},"min":{"1min":0.26900000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.51800000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.42999999999999999,"15min":0.41299999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.27700000000000002},"max":{"1min":0.51400000000000001,"5min":0.66100000000000003,"15min":1.121},"last":0.55700000000000005}]},{"osd":2,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42199999999999999,"15min":0.39300000000000002},"min":{"1min":0.30599999999999999,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":0.56000000000000005,"5min":1.0940000000000001,"15min":1.25},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.45200000000000001,"15min":0.442},"min":{"1min":0.30099999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.65800000000000003,"5min":0.66700000000000004,"15min":0.82999999999999996},"last":0.52000000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.495,"15min":0.48199999999999998},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.32100000000000001},"max":{"1min":0.61299999999999999,"5min":0.79900000000000004,"15min":0.90900000000000003},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47399999999999998,"15min":0.46400000000000002},"min":{"1min":0.35999999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.59199999999999997,"5min":0.72599999999999998,"15min":0.98999999999999999},"last":0.62}]},{"osd":4,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66000000000000003,"15min":0.64800000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.373},"max":{"1min":0.83999999999999997,"5min":1.075,"15min":3.2890000000000001},"last":0.75},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67800000000000005,"15min":0.65500000000000003},"min":{"1min":0.48999999999999999,"5min":0.45000000000000001,"15min":0.44500000000000001},"max":{"1min":0.95499999999999996,"5min":0.98099999999999998,"15min":1.026},"last":0.90900000000000003}]},{"osd":5,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68500000000000005,"15min":0.66200000000000003},"min":{"1min":0.46700000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89900000000000002,"5min":0.94599999999999995,"15min":3.4500000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68600000000000005,"15min":0.68200000000000005},"min":{"1min":0.45600000000000002,"5min":0.44,"15min":0.42399999999999999},"max":{"1min":0.93000000000000005,"5min":1.004,"15min":3.181},"last":0.77200000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.71599999999999997,"15min":0.70599999999999996},"min":{"1min":0.52500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91400000000000003,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.74099999999999999,"15min":0.70599999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.50800000000000001},"max":{"1min":0.93899999999999995,"5min":1.143,"15min":1.869},"last":0.73899999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73799999999999999,"15min":0.71099999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85399999999999998,"5min":1.181,"15min":1.6870000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72799999999999998,"15min":0.71099999999999997},"min":{"1min":0.56299999999999994,"5min":0.501,"15min":0.45300000000000001},"max":{"1min":0.92400000000000004,"5min":1.0429999999999999,"15min":1.708},"last":0.84099999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574989,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1552,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193759,"statfs":{"total":200000000,"available":198409980,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39500000000000002,"15min":0.39300000000000002},"min":{"1min":0.28499999999999998,"5min":0.24399999999999999,"15min":0.23799999999999999},"max":{"1min":0.55000000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42499999999999999,"15min":0.41999999999999998},"min":{"1min":0.32400000000000001,"5min":0.26000000000000001,"15min":0.223},"max":{"1min":0.55800000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.504}]},{"osd":2,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42499999999999999},"min":{"1min":0.34999999999999998,"5min":0.28599999999999998,"15min":0.28199999999999997},"max":{"1min":0.60799999999999998,"5min":0.625,"15min":0.64000000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45500000000000002,"15min":0.44400000000000001},"min":{"1min":0.34200000000000003,"5min":0.29499999999999998,"15min":0.28799999999999998},"max":{"1min":0.626,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.44700000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.44600000000000001,"15min":0.436},"min":{"1min":0.29999999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58499999999999996,"5min":0.66600000000000004,"15min":0.67300000000000004},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.39000000000000001,"5min":0.34300000000000003,"15min":0.29099999999999998},"max":{"1min":0.69799999999999995,"5min":0.71399999999999997,"15min":0.71699999999999997},"last":0.47099999999999997}]},{"osd":4,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.65400000000000003,"15min":0.64900000000000002},"min":{"1min":0.46600000000000003,"5min":0.432,"15min":0.42099999999999999},"max":{"1min":0.84899999999999998,"5min":0.97599999999999998,"15min":0.997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63700000000000001,"15min":0.65000000000000002},"min":{"1min":0.46400000000000002,"5min":0.42199999999999999,"15min":0.41999999999999998},"max":{"1min":0.85999999999999999,"5min":0.87,"15min":0.96699999999999997},"last":0.79200000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.54100000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":2.4860000000000002},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.44},"max":{"1min":0.90200000000000002,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.84499999999999997}]},{"osd":6,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.51300000000000001,"5min":0.45600000000000002,"15min":0.42499999999999999},"max":{"1min":0.92600000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68899999999999995,"15min":0.68600000000000005},"min":{"1min":0.53600000000000003,"5min":0.46500000000000002,"15min":0.45000000000000001},"max":{"1min":0.96699999999999997,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.82899999999999996}]},{"osd":7,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.51400000000000001,"5min":0.47099999999999997,"15min":0.42599999999999999},"max":{"1min":0.94799999999999995,"5min":0.94999999999999996,"15min":1.014},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68899999999999995,"15min":0.68999999999999995},"min":{"1min":0.55800000000000005,"5min":0.499,"15min":0.498},"max":{"1min":0.90900000000000003,"5min":0.92500000000000004,"15min":0.97299999999999998},"last":0.90500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574989,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":982,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194330,"statfs":{"total":200000000,"available":198993970,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41099999999999998},"min":{"1min":0.27000000000000002,"5min":0.23300000000000001,"15min":0.23000000000000001},"max":{"1min":0.63200000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.46000000000000002,"15min":0.53900000000000003},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.30499999999999999},"max":{"1min":0.57999999999999996,"5min":0.68899999999999995,"15min":6.2729999999999997},"last":0.36099999999999999}]},{"osd":1,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.41199999999999998,"15min":0.42099999999999999},"min":{"1min":0.33000000000000002,"5min":0.28799999999999998,"15min":0.23799999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.81399999999999995},"last":0.375},{"interface":"front","average":{"1min":0.442,"5min":0.443,"15min":0.442},"min":{"1min":0.29999999999999999,"5min":0.28999999999999998,"15min":0.28899999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.70099999999999996},"last":0.40500000000000003}]},{"osd":3,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.31,"5min":0.28499999999999998,"15min":0.27800000000000002},"max":{"1min":0.55800000000000005,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.441},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48399999999999999,"15min":0.48499999999999999},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.70399999999999996,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.41799999999999998}]},{"osd":4,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68100000000000005,"15min":0.67000000000000004},"min":{"1min":0.54600000000000004,"5min":0.47599999999999998,"15min":0.39300000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.98299999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.746},"min":{"1min":0.56499999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94399999999999995,"5min":0.95799999999999996,"15min":3.581},"last":0.71899999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65800000000000003,"15min":0.69599999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.43099999999999999},"max":{"1min":0.90400000000000003,"5min":0.92200000000000004,"15min":3.5369999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66400000000000003,"15min":0.66800000000000004},"min":{"1min":0.5,"5min":0.44800000000000001,"15min":0.41099999999999998},"max":{"1min":0.79900000000000004,"5min":0.89100000000000001,"15min":1.0529999999999999},"last":0.70599999999999996}]},{"osd":6,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.55300000000000005,"5min":0.46999999999999997,"15min":0.42299999999999999},"max":{"1min":1.0900000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70099999999999996,"15min":0.68899999999999995},"min":{"1min":0.48299999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0469999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.73399999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69599999999999995,"15min":0.68500000000000005},"min":{"1min":0.46999999999999997,"5min":0.437,"15min":0.437},"max":{"1min":1.0600000000000001,"5min":1.55,"15min":1.55},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71999999999999997,"15min":0.70099999999999996},"min":{"1min":0.54300000000000004,"5min":0.51300000000000001,"15min":0.42699999999999999},"max":{"1min":0.96999999999999997,"5min":1.615,"15min":1.615},"last":0.77300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":976,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194336,"statfs":{"total":200000000,"available":199000314,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.253},"max":{"1min":0.56299999999999994,"5min":0.67900000000000005,"15min":4.7380000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.441,"5min":0.42099999999999999,"15min":0.40899999999999997},"min":{"1min":0.30499999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":1.0189999999999999},"last":0.34599999999999997}]},{"osd":1,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40500000000000003,"15min":0.41999999999999998},"min":{"1min":0.27600000000000002,"5min":0.24399999999999999,"15min":0.22900000000000001},"max":{"1min":0.54300000000000004,"5min":0.64400000000000002,"15min":4.5739999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46300000000000002,"15min":0.46500000000000002},"min":{"1min":0.35499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.751,"5min":0.98199999999999998,"15min":5.0529999999999999},"last":0.47899999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.46800000000000003,"15min":0.45600000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.28100000000000003},"max":{"1min":0.624,"5min":0.76700000000000002,"15min":1.5980000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.48899999999999999,"15min":0.49299999999999999},"min":{"1min":0.42599999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.65700000000000003,"5min":0.73499999999999999,"15min":5.1879999999999997},"last":0.443}]},{"osd":4,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63500000000000001,"15min":0.64500000000000002},"min":{"1min":0.44,"5min":0.42999999999999999,"15min":0.41699999999999998},"max":{"1min":0.877,"5min":0.91600000000000004,"15min":4.6660000000000004},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65400000000000003,"15min":0.63600000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84199999999999997,"5min":0.94799999999999995,"15min":1.3120000000000001},"last":0.51600000000000001}]},{"osd":5,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.49199999999999999,"5min":0.438,"15min":0.40799999999999997},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.409},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66600000000000004,"15min":0.70499999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81499999999999995,"5min":0.96599999999999997,"15min":4.7960000000000003},"last":0.66600000000000004}]},{"osd":6,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67200000000000004,"15min":0.68500000000000005},"min":{"1min":0.50900000000000001,"5min":0.42799999999999999,"15min":0.39800000000000002},"max":{"1min":0.86099999999999999,"5min":0.93500000000000005,"15min":5.0069999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69299999999999995,"15min":0.67200000000000004},"min":{"1min":0.53000000000000003,"5min":0.47499999999999998,"15min":0.435},"max":{"1min":0.83499999999999996,"5min":0.94199999999999995,"15min":1.103},"last":0.62}]},{"osd":7,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46000000000000002},"max":{"1min":0.83399999999999996,"5min":0.92100000000000004,"15min":4.7130000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.68899999999999995},"min":{"1min":0.48599999999999999,"5min":0.47299999999999998,"15min":0.435},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.361},"last":0.58399999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":983,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194328,"statfs":{"total":200000000,"available":198992454,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.43099999999999999,"5min":0.35599999999999998,"15min":0.314},"max":{"1min":0.76800000000000002,"5min":1.137,"15min":1.3839999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60899999999999999,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":1.016,"15min":7.0670000000000002},"last":0.51600000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.35399999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":5.1529999999999996},"last":0.437},{"interface":"front","average":{"1min":0.623,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.48499999999999999,"5min":0.43099999999999999,"15min":0.39000000000000001},"max":{"1min":0.80100000000000005,"5min":1.054,"15min":3.0419999999999998},"last":0.47899999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60399999999999998,"15min":0.61499999999999999},"min":{"1min":0.46400000000000002,"5min":0.41199999999999998,"15min":0.39400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":5.0019999999999998},"last":0.499},{"interface":"front","average":{"1min":0.622,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.49399999999999999,"5min":0.41899999999999998,"15min":0.40500000000000003},"max":{"1min":0.86099999999999999,"5min":1.2549999999999999,"15min":2.073},"last":0.38900000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.496,"5min":0.439,"15min":0.36199999999999999},"max":{"1min":0.78800000000000003,"5min":1.1899999999999999,"15min":6.7220000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.629,"15min":0.65700000000000003},"min":{"1min":0.442,"5min":0.42499999999999999,"15min":0.41399999999999998},"max":{"1min":0.872,"5min":0.88,"15min":6.5510000000000002},"last":0.54300000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.36199999999999999,"5min":0.314,"15min":0.312},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":1.3149999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.5,"15min":0.51800000000000002},"min":{"1min":0.41399999999999998,"5min":0.32500000000000001,"15min":0.29999999999999999},"max":{"1min":0.77500000000000002,"5min":0.79800000000000004,"15min":6.9269999999999996},"last":0.45800000000000002}]},{"osd":6,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.432,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":5.2229999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55600000000000005,"15min":0.55100000000000005},"min":{"1min":0.437,"5min":0.35799999999999998,"15min":0.33200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":1.46},"last":0.53100000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55200000000000005,"15min":0.57099999999999995},"min":{"1min":0.41399999999999998,"5min":0.36599999999999999,"15min":0.33600000000000002},"max":{"1min":0.80400000000000005,"5min":0.83499999999999996,"15min":7.5339999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58099999999999996,"15min":0.59399999999999997},"min":{"1min":0.47099999999999997,"5min":0.34699999999999998,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.85299999999999998,"15min":4.9550000000000001},"last":0.627}]}]},{"osd":5,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":976,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194335,"statfs":{"total":200000000,"available":198999772,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.44700000000000001,"5min":0.33900000000000002,"15min":0.28899999999999998},"max":{"1min":0.76300000000000001,"5min":1.071,"15min":1.071},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.47999999999999998,"5min":0.38500000000000001,"15min":0.27600000000000002},"max":{"1min":0.71099999999999997,"5min":0.83599999999999997,"15min":3.242},"last":0.47399999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.627},"min":{"1min":0.53500000000000003,"5min":0.38100000000000001,"15min":0.32800000000000001},"max":{"1min":0.80800000000000005,"5min":1.018,"15min":1.3380000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.45600000000000002,"5min":0.35399999999999998,"15min":0.34200000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.95499999999999996},"last":0.499}]},{"osd":2,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":0.96699999999999997,"15min":3.04},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60799999999999998,"15min":0.59099999999999997},"min":{"1min":0.52000000000000002,"5min":0.43099999999999999,"15min":0.36899999999999999},"max":{"1min":0.71299999999999997,"5min":0.81000000000000005,"15min":0.89400000000000002},"last":0.39800000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63,"15min":0.65500000000000003},"min":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.39800000000000002},"max":{"1min":0.73799999999999999,"5min":1.0409999999999999,"15min":4.032},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66900000000000004,"15min":0.68000000000000005},"min":{"1min":0.57199999999999995,"5min":0.48799999999999999,"15min":0.45600000000000002},"max":{"1min":0.753,"5min":0.879,"15min":4.282},"last":0.34300000000000003}]},{"osd":4,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.51500000000000001},"min":{"1min":0.376,"5min":0.33300000000000002,"15min":0.26500000000000001},"max":{"1min":0.76400000000000001,"5min":0.95399999999999996,"15min":1.2869999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.496,"15min":0.52900000000000003},"min":{"1min":0.35799999999999998,"5min":0.32800000000000001,"15min":0.30399999999999999},"max":{"1min":0.67900000000000005,"5min":0.81699999999999995,"15min":1.101},"last":0.44500000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.50600000000000001,"15min":0.52900000000000003},"min":{"1min":0.35299999999999998,"5min":0.32700000000000001,"15min":0.29099999999999998},"max":{"1min":0.65700000000000003,"5min":0.86599999999999999,"15min":1.7969999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.55200000000000005,"15min":0.56799999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.33600000000000002},"max":{"1min":0.69299999999999995,"5min":0.91800000000000004,"15min":1.6739999999999999},"last":0.54800000000000004}]},{"osd":7,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.55100000000000005,"15min":0.55900000000000005},"min":{"1min":0.39500000000000002,"5min":0.36899999999999999,"15min":0.32700000000000001},"max":{"1min":0.76300000000000001,"5min":0.90600000000000003,"15min":1.008},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59199999999999997,"15min":0.627},"min":{"1min":0.46000000000000002,"5min":0.40600000000000003,"15min":0.33700000000000002},"max":{"1min":0.67500000000000004,"5min":0.91300000000000003,"15min":2},"last":0.59799999999999998}]}]}],"pool_statfs":[]}} 2024-01-27T02:43:34.783 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-27T02:43:34.784 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-27T02:43:34.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.961+0000 7fedfe1ff700 1 Processor -- start 2024-01-27T02:43:34.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.961+0000 7fedfe1ff700 1 -- start start 2024-01-27T02:43:34.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.962+0000 7fedfe1ff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedf80cbe30 0x7fedf80ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:34.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.962+0000 7fedfe1ff700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fedf806b4e0 0x7fedf806b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:34.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.962+0000 7fedfe1ff700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf806be80 con 0x7fedf80cba60 2024-01-27T02:43:34.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.962+0000 7fedfe1ff700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf80679b0 con 0x7fedf806b4e0 2024-01-27T02:43:34.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.962+0000 7fedfe1ff700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf8067c90 con 0x7fedf80cbe30 2024-01-27T02:43:34.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.962+0000 7fedf6ffd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedf80cbe30 0x7fedf80ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:34.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.962+0000 7fedf7fff700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fedf806b4e0 0x7fedf806b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:34.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.963+0000 7fedf6ffd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedf80cbe30 0x7fedf80ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:56346/0 (socket says 172.21.15.7:56346) 2024-01-27T02:43:34.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.963+0000 7fedf6ffd700 1 -- 172.21.15.7:0/1764352910 learned_addr learned my addr 172.21.15.7:0/1764352910 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:34.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.963+0000 7fedf7fff700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fedf806b4e0 0x7fedf806b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.7:34700/0 (socket says 172.21.15.7:34700) 2024-01-27T02:43:34.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.963+0000 7fedf77fe700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fedf80cba60 0x7fedf815b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:58298/0 (socket says 172.21.15.7:58298) 2024-01-27T02:43:34.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.963+0000 7fedf6ffd700 1 -- 172.21.15.7:0/1764352910 >> v2:172.21.15.188:3300/0 conn(0x7fedf806b4e0 msgr2=0x7fedf806b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:34.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.963+0000 7fedf6ffd700 1 --2- 172.21.15.7:0/1764352910 >> v2:172.21.15.188:3300/0 conn(0x7fedf806b4e0 0x7fedf806b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.963+0000 7fedf6ffd700 1 -- 172.21.15.7:0/1764352910 >> v1:172.21.15.188:6789/0 conn(0x7fedf80cba60 legacy=0x7fedf815b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:34.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.963+0000 7fedf6ffd700 1 -- 172.21.15.7:0/1764352910 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf80cca50 con 0x7fedf80cbe30 2024-01-27T02:43:34.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.963+0000 7fedf67fc700 1 -- 172.21.15.7:0/1764352910 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4115088419 0 0) 0x7fede8001160 con 0x7fedf80cba60 2024-01-27T02:43:34.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.963+0000 7fedf7fff700 1 --2- 172.21.15.7:0/1764352910 >> v2:172.21.15.188:3300/0 conn(0x7fedf806b4e0 0x7fedf806b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:43:34.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.963+0000 7fedf6ffd700 1 --2- 172.21.15.7:0/1764352910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedf80cbe30 0x7fedf80ce410 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fedec0099b0 tx=0x7fedec009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c345e80f381c00f server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:34.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.963+0000 7fedf67fc700 1 -- 172.21.15.7:0/1764352910 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec009ea0 con 0x7fedf80cbe30 2024-01-27T02:43:34.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.963+0000 7fedf67fc700 1 -- 172.21.15.7:0/1764352910 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedec00b860 con 0x7fedf80cbe30 2024-01-27T02:43:34.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.963+0000 7fedf67fc700 1 -- 172.21.15.7:0/1764352910 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec00bc00 con 0x7fedf80cbe30 2024-01-27T02:43:34.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.964+0000 7fedfe1ff700 1 -- 172.21.15.7:0/1764352910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedf80cbe30 msgr2=0x7fedf80ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:34.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.964+0000 7fedfe1ff700 1 --2- 172.21.15.7:0/1764352910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedf80cbe30 0x7fedf80ce410 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fedec0099b0 tx=0x7fedec009250 comp rx=0 tx=0).stop 2024-01-27T02:43:34.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.964+0000 7fedfe1ff700 1 -- 172.21.15.7:0/1764352910 shutdown_connections 2024-01-27T02:43:34.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.964+0000 7fedfe1ff700 1 --2- 172.21.15.7:0/1764352910 >> v2:172.21.15.188:3300/0 conn(0x7fedf806b4e0 0x7fedf806b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.964+0000 7fedfe1ff700 1 --2- 172.21.15.7:0/1764352910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedf80cbe30 0x7fedf80ce410 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.964+0000 7fedfe1ff700 1 -- 172.21.15.7:0/1764352910 >> 172.21.15.7:0/1764352910 conn(0x7fedf8063580 msgr2=0x7fedf8063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.964+0000 7fedfe1ff700 1 -- 172.21.15.7:0/1764352910 shutdown_connections 2024-01-27T02:43:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.964+0000 7fedfe1ff700 1 -- 172.21.15.7:0/1764352910 wait complete. 2024-01-27T02:43:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.965+0000 7fedfe1ff700 1 Processor -- start 2024-01-27T02:43:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.965+0000 7fedfe1ff700 1 -- start start 2024-01-27T02:43:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.966+0000 7fedfe1ff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fedf806b4e0 0x7fedf80af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.966+0000 7fedfe1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedf80cba60 0x7fedf80afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.966+0000 7fedfe1ff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedf80cbe30 0x7fedf80ab880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.966+0000 7fedfe1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf80b12c0 con 0x7fedf80cba60 2024-01-27T02:43:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.966+0000 7fedfe1ff700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf80b03e0 con 0x7fedf806b4e0 2024-01-27T02:43:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.966+0000 7fedfe1ff700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf80b06b0 con 0x7fedf80cbe30 2024-01-27T02:43:34.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.966+0000 7fedf6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedf80cba60 0x7fedf80afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:34.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.966+0000 7fedf6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedf80cba60 0x7fedf80afea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34818/0 (socket says 172.21.15.7:34818) 2024-01-27T02:43:34.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.966+0000 7fedf6ffd700 1 -- 172.21.15.7:0/3157319032 learned_addr learned my addr 172.21.15.7:0/3157319032 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:34.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.966+0000 7fedf77fe700 1 --2- 172.21.15.7:0/3157319032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fedf806b4e0 0x7fedf80af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:34.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.966+0000 7fedf7fff700 1 --2- 172.21.15.7:0/3157319032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedf80cbe30 0x7fedf80ab880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:34.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.967+0000 7fedf6ffd700 1 -- 172.21.15.7:0/3157319032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedf80cbe30 msgr2=0x7fedf80ab880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:34.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.967+0000 7fedf6ffd700 1 --2- 172.21.15.7:0/3157319032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedf80cbe30 0x7fedf80ab880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.967+0000 7fedf6ffd700 1 -- 172.21.15.7:0/3157319032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fedf806b4e0 msgr2=0x7fedf80af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:34.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.967+0000 7fedf6ffd700 1 --2- 172.21.15.7:0/3157319032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fedf806b4e0 0x7fedf80af960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:34.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.967+0000 7fedf6ffd700 1 -- 172.21.15.7:0/3157319032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf0009a20 con 0x7fedf80cba60 2024-01-27T02:43:34.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.967+0000 7fedf6ffd700 1 --2- 172.21.15.7:0/3157319032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedf80cba60 0x7fedf80afea0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fedf8067dd0 tx=0x7fedec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:34.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.967+0000 7feddf7fe700 1 -- 172.21.15.7:0/3157319032 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec00bbe0 con 0x7fedf80cba60 2024-01-27T02:43:34.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.967+0000 7feddf7fe700 1 -- 172.21.15.7:0/3157319032 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedec011420 con 0x7fedf80cba60 2024-01-27T02:43:34.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.967+0000 7fedfe1ff700 1 -- 172.21.15.7:0/3157319032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedec0094f0 con 0x7fedf80cba60 2024-01-27T02:43:34.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.967+0000 7feddf7fe700 1 -- 172.21.15.7:0/3157319032 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec0117e0 con 0x7fedf80cba60 2024-01-27T02:43:34.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.967+0000 7fedfe1ff700 1 -- 172.21.15.7:0/3157319032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf80b0c70 con 0x7fedf80cba60 2024-01-27T02:43:34.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.968+0000 7fedfe1ff700 1 -- 172.21.15.7:0/3157319032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf8072060 con 0x7fedf80cba60 2024-01-27T02:43:34.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.969+0000 7feddf7fe700 1 -- 172.21.15.7:0/3157319032 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fedec011bd0 con 0x7fedf80cba60 2024-01-27T02:43:34.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.969+0000 7feddf7fe700 1 --2- 172.21.15.7:0/3157319032 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fede4066f30 0x7fede40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:34.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.969+0000 7feddf7fe700 1 -- 172.21.15.7:0/3157319032 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fedec085d60 con 0x7fedf80cba60 2024-01-27T02:43:34.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.969+0000 7fedf77fe700 1 --2- 172.21.15.7:0/3157319032 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fede4066f30 0x7fede40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:34.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.970+0000 7fedf77fe700 1 --2- 172.21.15.7:0/3157319032 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fede4066f30 0x7fede40693e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fede8004f70 tx=0x7fede8007b80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:34.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:34.974+0000 7feddf7fe700 1 -- 172.21.15.7:0/3157319032 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fedec053090 con 0x7fedf80cba60 2024-01-27T02:43:35.154 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-01-27T02:43:35.153+0000 7f410eb40700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-27T02:43:35.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.259+0000 7fedfe1ff700 1 -- 172.21.15.7:0/3157319032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fedf8064370 con 0x7fedf80cba60 2024-01-27T02:43:35.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.263+0000 7feddf7fe700 1 -- 172.21.15.7:0/3157319032 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v25) v1 ==== 190+0+0 (secure 0 0 0) 0x7fedec0566b0 con 0x7fedf80cba60 2024-01-27T02:43:35.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.265+0000 7fedfe1ff700 1 -- 172.21.15.7:0/3157319032 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fede4066f30 msgr2=0x7fede40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:35.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.265+0000 7fedfe1ff700 1 --2- 172.21.15.7:0/3157319032 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fede4066f30 0x7fede40693e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fede8004f70 tx=0x7fede8007b80 comp rx=0 tx=0).stop 2024-01-27T02:43:35.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.265+0000 7fedfe1ff700 1 -- 172.21.15.7:0/3157319032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedf80cba60 msgr2=0x7fedf80afea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:35.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.265+0000 7fedfe1ff700 1 --2- 172.21.15.7:0/3157319032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedf80cba60 0x7fedf80afea0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fedf8067dd0 tx=0x7fedec017000 comp rx=0 tx=0).stop 2024-01-27T02:43:35.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.265+0000 7fedfe1ff700 1 -- 172.21.15.7:0/3157319032 shutdown_connections 2024-01-27T02:43:35.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.265+0000 7fedfe1ff700 1 --2- 172.21.15.7:0/3157319032 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fede4066f30 0x7fede40693e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:35.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.265+0000 7fedfe1ff700 1 --2- 172.21.15.7:0/3157319032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fedf80cbe30 0x7fedf80ab880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:35.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.265+0000 7fedfe1ff700 1 --2- 172.21.15.7:0/3157319032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fedf806b4e0 0x7fedf80af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:35.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.265+0000 7fedfe1ff700 1 --2- 172.21.15.7:0/3157319032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedf80cba60 0x7fedf80afea0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:35.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.265+0000 7fedfe1ff700 1 -- 172.21.15.7:0/3157319032 >> 172.21.15.7:0/3157319032 conn(0x7fedf8063580 msgr2=0x7fedf8069490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:35.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.266+0000 7fedfe1ff700 1 -- 172.21.15.7:0/3157319032 shutdown_connections 2024-01-27T02:43:35.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.266+0000 7fedfe1ff700 1 -- 172.21.15.7:0/3157319032 wait complete. 2024-01-27T02:43:35.267 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-01-27T02:43:35.280 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:43:35.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.410+0000 7f0b00c7d700 1 Processor -- start 2024-01-27T02:43:35.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.410+0000 7f0b00c7d700 1 -- start start 2024-01-27T02:43:35.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.411+0000 7f0b00c7d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0afc143520 0x7f0afc063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:35.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.411+0000 7f0b00c7d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0afc0aa8b0 0x7f0afc0c8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:35.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.411+0000 7f0b00c7d700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0afc0cb7a0 con 0x7f0afc1438f0 2024-01-27T02:43:35.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.411+0000 7f0b00c7d700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0afc0cb9f0 con 0x7f0afc0aa8b0 2024-01-27T02:43:35.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.411+0000 7f0b00c7d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc0cbcd0 con 0x7f0afc143520 2024-01-27T02:43:35.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.412+0000 7f0afad9d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0afc0aa8b0 0x7f0afc0c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:35.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.412+0000 7f0afa59c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0afc143520 0x7f0afc063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:35.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.412+0000 7f0af9d9b700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f0afc1438f0 0x7f0afc0c8210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:58308/0 (socket says 172.21.15.7:58308) 2024-01-27T02:43:35.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.412+0000 7f0af9d9b700 1 -- 172.21.15.7:0/4076807036 learned_addr learned my addr 172.21.15.7:0/4076807036 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:35.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.412+0000 7f0afa59c700 1 -- 172.21.15.7:0/4076807036 >> v2:172.21.15.188:3300/0 conn(0x7f0afc0aa8b0 msgr2=0x7f0afc0c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:35.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.412+0000 7f0afa59c700 1 --2- 172.21.15.7:0/4076807036 >> v2:172.21.15.188:3300/0 conn(0x7f0afc0aa8b0 0x7f0afc0c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:35.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.412+0000 7f0afa59c700 1 -- 172.21.15.7:0/4076807036 >> v1:172.21.15.188:6789/0 conn(0x7f0afc1438f0 legacy=0x7f0afc0c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:35.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.412+0000 7f0afa59c700 1 -- 172.21.15.7:0/4076807036 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0afc0cbfb0 con 0x7f0afc143520 2024-01-27T02:43:35.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.412+0000 7f0afad9d700 1 --2- 172.21.15.7:0/4076807036 >> v2:172.21.15.188:3300/0 conn(0x7f0afc0aa8b0 0x7f0afc0c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:43:35.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.413+0000 7f0afa59c700 1 --2- 172.21.15.7:0/4076807036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0afc143520 0x7f0afc063580 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0098d0 tx=0x7f0aec009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6be722b277931606 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:35.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.413+0000 7f0af959a700 1 -- 172.21.15.7:0/4076807036 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec005c10 con 0x7f0afc143520 2024-01-27T02:43:35.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.413+0000 7f0af959a700 1 -- 172.21.15.7:0/4076807036 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aec005d70 con 0x7f0afc143520 2024-01-27T02:43:35.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.413+0000 7f0af959a700 1 -- 172.21.15.7:0/4076807036 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec006260 con 0x7f0afc143520 2024-01-27T02:43:35.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.413+0000 7f0b00c7d700 1 -- 172.21.15.7:0/4076807036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0afc143520 msgr2=0x7f0afc063580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:35.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.413+0000 7f0b00c7d700 1 --2- 172.21.15.7:0/4076807036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0afc143520 0x7f0afc063580 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0098d0 tx=0x7f0aec009120 comp rx=0 tx=0).stop 2024-01-27T02:43:35.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.413+0000 7f0b00c7d700 1 -- 172.21.15.7:0/4076807036 shutdown_connections 2024-01-27T02:43:35.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.413+0000 7f0b00c7d700 1 --2- 172.21.15.7:0/4076807036 >> v2:172.21.15.188:3300/0 conn(0x7f0afc0aa8b0 0x7f0afc0c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:35.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.413+0000 7f0b00c7d700 1 --2- 172.21.15.7:0/4076807036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0afc143520 0x7f0afc063580 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:35.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.413+0000 7f0b00c7d700 1 -- 172.21.15.7:0/4076807036 >> 172.21.15.7:0/4076807036 conn(0x7f0afc0cdbd0 msgr2=0x7f0afc0cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:35.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.413+0000 7f0b00c7d700 1 -- 172.21.15.7:0/4076807036 shutdown_connections 2024-01-27T02:43:35.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.413+0000 7f0b00c7d700 1 -- 172.21.15.7:0/4076807036 wait complete. 2024-01-27T02:43:35.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.415+0000 7f0b00c7d700 1 Processor -- start 2024-01-27T02:43:35.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.415+0000 7f0b00c7d700 1 -- start start 2024-01-27T02:43:35.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.415+0000 7f0b00c7d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0afc0aa8b0 0x7f0afc15b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:35.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.415+0000 7f0b00c7d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0afc143520 0x7f0afc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:35.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.415+0000 7f0b00c7d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0afc1438f0 0x7f0afc137110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:35.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.415+0000 7f0b00c7d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc15c0b0 con 0x7f0afc0aa8b0 2024-01-27T02:43:35.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.415+0000 7f0b00c7d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc15c390 con 0x7f0afc1438f0 2024-01-27T02:43:35.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.416+0000 7f0b00c7d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc15c670 con 0x7f0afc143520 2024-01-27T02:43:35.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.416+0000 7f0afa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0afc0aa8b0 0x7f0afc15b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:35.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.416+0000 7f0afa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0afc0aa8b0 0x7f0afc15b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34830/0 (socket says 172.21.15.7:34830) 2024-01-27T02:43:35.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.416+0000 7f0afa59c700 1 -- 172.21.15.7:0/1973356769 learned_addr learned my addr 172.21.15.7:0/1973356769 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:35.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.416+0000 7f0af9d9b700 1 --2- 172.21.15.7:0/1973356769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0afc143520 0x7f0afc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.416+0000 7f0afad9d700 1 --2- 172.21.15.7:0/1973356769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0afc1438f0 0x7f0afc137110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.416+0000 7f0afa59c700 1 -- 172.21.15.7:0/1973356769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0afc143520 msgr2=0x7f0afc0b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.416+0000 7f0afa59c700 1 --2- 172.21.15.7:0/1973356769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0afc143520 0x7f0afc0b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.416+0000 7f0afa59c700 1 -- 172.21.15.7:0/1973356769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0afc1438f0 msgr2=0x7f0afc137110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.416+0000 7f0afa59c700 1 --2- 172.21.15.7:0/1973356769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0afc1438f0 0x7f0afc137110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.416+0000 7f0afa59c700 1 -- 172.21.15.7:0/1973356769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af4009a20 con 0x7f0afc0aa8b0 2024-01-27T02:43:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.416+0000 7f0afad9d700 1 --2- 172.21.15.7:0/1973356769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0afc1438f0 0x7f0afc137110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.416+0000 7f0afa59c700 1 --2- 172.21.15.7:0/1973356769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0afc0aa8b0 0x7f0afc15b7b0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009f70 tx=0x7f0aec016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.416+0000 7f0af27fc700 1 -- 172.21.15.7:0/1973356769 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec006a70 con 0x7f0afc0aa8b0 2024-01-27T02:43:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.416+0000 7f0af27fc700 1 -- 172.21.15.7:0/1973356769 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aec01d6b0 con 0x7f0afc0aa8b0 2024-01-27T02:43:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.417+0000 7f0b00c7d700 1 -- 172.21.15.7:0/1973356769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aec0093c0 con 0x7f0afc0aa8b0 2024-01-27T02:43:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.417+0000 7f0af27fc700 1 -- 172.21.15.7:0/1973356769 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec01da50 con 0x7f0afc0aa8b0 2024-01-27T02:43:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.417+0000 7f0b00c7d700 1 -- 172.21.15.7:0/1973356769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0afc15cc30 con 0x7f0afc0aa8b0 2024-01-27T02:43:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.418+0000 7f0b00c7d700 1 -- 172.21.15.7:0/1973356769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0afc04c710 con 0x7f0afc0aa8b0 2024-01-27T02:43:35.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.418+0000 7f0af27fc700 1 -- 172.21.15.7:0/1973356769 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0aec01b480 con 0x7f0afc0aa8b0 2024-01-27T02:43:35.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.419+0000 7f0af27fc700 1 --2- 172.21.15.7:0/1973356769 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0adc066f30 0x7f0adc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:35.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.419+0000 7f0af27fc700 1 -- 172.21.15.7:0/1973356769 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0aec086110 con 0x7f0afc0aa8b0 2024-01-27T02:43:35.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.419+0000 7f0af9d9b700 1 --2- 172.21.15.7:0/1973356769 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0adc066f30 0x7f0adc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:35.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.419+0000 7f0af9d9b700 1 --2- 172.21.15.7:0/1973356769 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0adc066f30 0x7f0adc0693e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0afc0ad270 tx=0x7f0ae4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:35.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.423+0000 7f0af27fc700 1 -- 172.21.15.7:0/1973356769 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0aec053400 con 0x7f0afc0aa8b0 2024-01-27T02:43:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.672+0000 7f0b00c7d700 1 -- 172.21.15.7:0/1973356769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0afc0cc290 con 0x7f0afc0aa8b0 2024-01-27T02:43:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.672+0000 7f0af27fc700 1 -- 172.21.15.7:0/1973356769 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4595 (secure 0 0 0) 0x7f0aec0569a0 con 0x7f0afc0aa8b0 2024-01-27T02:43:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:35.673 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6704,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/1199331492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1199331492},{"type":"v1","addr":"172.21.15.7:6834","nonce":1199331492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":6707,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/721335086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":721335086},{"type":"v1","addr":"172.21.15.7:6835","nonce":721335086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":7699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/450555206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":450555206},{"type":"v1","addr":"172.21.15.188:6836","nonce":450555206}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:43:29.599748+0000","modified":"2024-01-27T02:43:33.196288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7702":{"gid":7702,"name":"b","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/3853795510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3853795510},{"type":"v1","addr":"172.21.15.188:6837","nonce":3853795510}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-27T02:43:35.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.675+0000 7f0b00c7d700 1 -- 172.21.15.7:0/1973356769 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0adc066f30 msgr2=0x7f0adc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:35.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.675+0000 7f0b00c7d700 1 --2- 172.21.15.7:0/1973356769 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0adc066f30 0x7f0adc0693e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0afc0ad270 tx=0x7f0ae4009040 comp rx=0 tx=0).stop 2024-01-27T02:43:35.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.675+0000 7f0b00c7d700 1 -- 172.21.15.7:0/1973356769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0afc0aa8b0 msgr2=0x7f0afc15b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.675+0000 7f0b00c7d700 1 --2- 172.21.15.7:0/1973356769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0afc0aa8b0 0x7f0afc15b7b0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009f70 tx=0x7f0aec016000 comp rx=0 tx=0).stop 2024-01-27T02:43:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.675+0000 7f0b00c7d700 1 -- 172.21.15.7:0/1973356769 shutdown_connections 2024-01-27T02:43:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.675+0000 7f0b00c7d700 1 --2- 172.21.15.7:0/1973356769 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0adc066f30 0x7f0adc0693e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.675+0000 7f0b00c7d700 1 --2- 172.21.15.7:0/1973356769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0afc1438f0 0x7f0afc137110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.675+0000 7f0b00c7d700 1 --2- 172.21.15.7:0/1973356769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0afc0aa8b0 0x7f0afc15b7b0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.675+0000 7f0b00c7d700 1 --2- 172.21.15.7:0/1973356769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0afc143520 0x7f0afc0b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.675+0000 7f0b00c7d700 1 -- 172.21.15.7:0/1973356769 >> 172.21.15.7:0/1973356769 conn(0x7f0afc0cdbd0 msgr2=0x7f0afc06c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.675+0000 7f0b00c7d700 1 -- 172.21.15.7:0/1973356769 shutdown_connections 2024-01-27T02:43:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:35.675+0000 7f0b00c7d700 1 -- 172.21.15.7:0/1973356769 wait complete. 2024-01-27T02:43:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 94 2024-01-27T02:43:36.093 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 94, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-27T02:43:29.599748+0000', 'modified': '2024-01-27T02:43:33.196288+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7702}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7702': {'gid': 7702, 'name': 'b', 'rank': 0, 'incarnation': 90, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6837/3853795510', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6835', 'nonce': 3853795510}, {'type': 'v1', 'addr': '172.21.15.188:6837', 'nonce': 3853795510}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-27T02:43:36.093 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-27T02:43:36.093 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-01-27T02:43:36.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.235+0000 7fdd9fef6700 1 Processor -- start 2024-01-27T02:43:36.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.235+0000 7fdd9fef6700 1 -- start start 2024-01-27T02:43:36.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.236+0000 7fdd9fef6700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdd9815b5f0 0x7fdd981566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.236+0000 7fdd9fef6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdd980b1fd0 0x7fdd98156c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.236+0000 7fdd9fef6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd98157610 con 0x7fdd980b1c00 2024-01-27T02:43:36.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.236+0000 7fdd9fef6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd9806ee00 con 0x7fdd980b1fd0 2024-01-27T02:43:36.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.236+0000 7fdd9fef6700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd9806f0e0 con 0x7fdd9815b5f0 2024-01-27T02:43:36.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.236+0000 7fdd9e493700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdd980b1fd0 0x7fdd98156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.236+0000 7fdd9dc92700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdd980b1c00 0x7fdd9806e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38522/0 (socket says 172.21.15.7:38522) 2024-01-27T02:43:36.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.236+0000 7fdd9dc92700 1 -- 172.21.15.7:0/3765752886 learned_addr learned my addr 172.21.15.7:0/3765752886 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:36.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.236+0000 7fdd9d491700 1 --2- 172.21.15.7:0/3765752886 >> v2:172.21.15.188:3300/0 conn(0x7fdd9815b5f0 0x7fdd981566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.237+0000 7fdd9e493700 1 -- 172.21.15.7:0/3765752886 >> v2:172.21.15.188:3300/0 conn(0x7fdd9815b5f0 msgr2=0x7fdd981566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:36.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.237+0000 7fdd9e493700 1 --2- 172.21.15.7:0/3765752886 >> v2:172.21.15.188:3300/0 conn(0x7fdd9815b5f0 0x7fdd981566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.237+0000 7fdd9e493700 1 -- 172.21.15.7:0/3765752886 >> v1:172.21.15.7:6789/0 conn(0x7fdd980b1c00 legacy=0x7fdd9806e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:36.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.237+0000 7fdd9e493700 1 -- 172.21.15.7:0/3765752886 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9806f3c0 con 0x7fdd980b1fd0 2024-01-27T02:43:36.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.237+0000 7fdd9cc90700 1 -- 172.21.15.7:0/3765752886 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 760456532 0 0) 0x7fdd98157610 con 0x7fdd980b1c00 2024-01-27T02:43:36.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.237+0000 7fdd9d491700 1 --2- 172.21.15.7:0/3765752886 >> v2:172.21.15.188:3300/0 conn(0x7fdd9815b5f0 0x7fdd981566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:36.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.237+0000 7fdd9e493700 1 --2- 172.21.15.7:0/3765752886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd980b1fd0 0x7fdd98156c00 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fdd98157580 tx=0x7fdd90009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6822556baf1dbc server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:36.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.237+0000 7fdd9cc90700 1 -- 172.21.15.7:0/3765752886 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9000e070 con 0x7fdd980b1fd0 2024-01-27T02:43:36.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.237+0000 7fdd9cc90700 1 -- 172.21.15.7:0/3765752886 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9000bc30 con 0x7fdd980b1fd0 2024-01-27T02:43:36.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.237+0000 7fdd9cc90700 1 -- 172.21.15.7:0/3765752886 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd90012430 con 0x7fdd980b1fd0 2024-01-27T02:43:36.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.237+0000 7fdd9fef6700 1 -- 172.21.15.7:0/3765752886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd980b1fd0 msgr2=0x7fdd98156c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:36.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.237+0000 7fdd9fef6700 1 --2- 172.21.15.7:0/3765752886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd980b1fd0 0x7fdd98156c00 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fdd98157580 tx=0x7fdd90009780 comp rx=0 tx=0).stop 2024-01-27T02:43:36.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.238+0000 7fdd9fef6700 1 -- 172.21.15.7:0/3765752886 shutdown_connections 2024-01-27T02:43:36.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.238+0000 7fdd9fef6700 1 --2- 172.21.15.7:0/3765752886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd980b1fd0 0x7fdd98156c00 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.238+0000 7fdd9fef6700 1 --2- 172.21.15.7:0/3765752886 >> v2:172.21.15.188:3300/0 conn(0x7fdd9815b5f0 0x7fdd981566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.238+0000 7fdd9fef6700 1 -- 172.21.15.7:0/3765752886 >> 172.21.15.7:0/3765752886 conn(0x7fdd98068430 msgr2=0x7fdd9806be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:36.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.238+0000 7fdd9fef6700 1 -- 172.21.15.7:0/3765752886 shutdown_connections 2024-01-27T02:43:36.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.238+0000 7fdd9fef6700 1 -- 172.21.15.7:0/3765752886 wait complete. 2024-01-27T02:43:36.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.239+0000 7fdd9fef6700 1 Processor -- start 2024-01-27T02:43:36.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.239+0000 7fdd9fef6700 1 -- start start 2024-01-27T02:43:36.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.240+0000 7fdd9fef6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd980b1c00 0x7fdd9816e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.240+0000 7fdd9fef6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd980b1fd0 0x7fdd98157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.240+0000 7fdd9fef6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd9815b5f0 0x7fdd9814c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.240+0000 7fdd9fef6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9814e1c0 con 0x7fdd980b1c00 2024-01-27T02:43:36.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.240+0000 7fdd9fef6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd981582d0 con 0x7fdd980b1fd0 2024-01-27T02:43:36.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.240+0000 7fdd9fef6700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd981585b0 con 0x7fdd9815b5f0 2024-01-27T02:43:36.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.240+0000 7fdd9dc92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd980b1c00 0x7fdd9816e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.241+0000 7fdd9dc92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd980b1c00 0x7fdd9816e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34840/0 (socket says 172.21.15.7:34840) 2024-01-27T02:43:36.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.241+0000 7fdd9dc92700 1 -- 172.21.15.7:0/427938296 learned_addr learned my addr 172.21.15.7:0/427938296 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:36.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.241+0000 7fdd9d491700 1 --2- 172.21.15.7:0/427938296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd980b1fd0 0x7fdd98157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.241+0000 7fdd9e493700 1 --2- 172.21.15.7:0/427938296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd9815b5f0 0x7fdd9814c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.241+0000 7fdd9dc92700 1 -- 172.21.15.7:0/427938296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd9815b5f0 msgr2=0x7fdd9814c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:36.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.241+0000 7fdd9dc92700 1 --2- 172.21.15.7:0/427938296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd9815b5f0 0x7fdd9814c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.241+0000 7fdd9dc92700 1 -- 172.21.15.7:0/427938296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd980b1fd0 msgr2=0x7fdd98157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:36.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.241+0000 7fdd9dc92700 1 --2- 172.21.15.7:0/427938296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd980b1fd0 0x7fdd98157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.241+0000 7fdd9dc92700 1 -- 172.21.15.7:0/427938296 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd90009a20 con 0x7fdd980b1c00 2024-01-27T02:43:36.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.241+0000 7fdd9d491700 1 --2- 172.21.15.7:0/427938296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd980b1fd0 0x7fdd98157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:36.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.241+0000 7fdd9dc92700 1 --2- 172.21.15.7:0/427938296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd980b1c00 0x7fdd9816e530 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400c7e0 tx=0x7fdd94007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:36.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.241+0000 7fdd8dffb700 1 -- 172.21.15.7:0/427938296 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd94005b30 con 0x7fdd980b1c00 2024-01-27T02:43:36.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.241+0000 7fdd8dffb700 1 -- 172.21.15.7:0/427938296 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd94005c90 con 0x7fdd980b1c00 2024-01-27T02:43:36.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.241+0000 7fdd9fef6700 1 -- 172.21.15.7:0/427938296 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd98158890 con 0x7fdd980b1c00 2024-01-27T02:43:36.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.241+0000 7fdd8dffb700 1 -- 172.21.15.7:0/427938296 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd94007300 con 0x7fdd980b1c00 2024-01-27T02:43:36.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.242+0000 7fdd9fef6700 1 -- 172.21.15.7:0/427938296 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd98157fe0 con 0x7fdd980b1c00 2024-01-27T02:43:36.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.243+0000 7fdd8dffb700 1 -- 172.21.15.7:0/427938296 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fdd9401b400 con 0x7fdd980b1c00 2024-01-27T02:43:36.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.243+0000 7fdd8dffb700 1 --2- 172.21.15.7:0/427938296 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdd7c066f30 0x7fdd7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.243+0000 7fdd8dffb700 1 -- 172.21.15.7:0/427938296 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdd940840d0 con 0x7fdd980b1c00 2024-01-27T02:43:36.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.244+0000 7fdd9d491700 1 --2- 172.21.15.7:0/427938296 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdd7c066f30 0x7fdd7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.244+0000 7fdd9d491700 1 --2- 172.21.15.7:0/427938296 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdd7c066f30 0x7fdd7c0693e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdd88006f00 tx=0x7fdd88008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:36.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.245+0000 7fdda91f9b80 1 Processor -- start 2024-01-27T02:43:36.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.245+0000 7fdda91f9b80 1 -- start start 2024-01-27T02:43:36.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.246+0000 7fdda91f9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bef1a4010 0x556bef1a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.246+0000 7fdda91f9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1ec360 0x556bef1ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.246+0000 7fdda91f9b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bef1a35d0 0x556bef1ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.246+0000 7fdda91f9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bef0a79f0 con 0x556bef1ec360 2024-01-27T02:43:36.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.246+0000 7fdda91f9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bef0f9130 con 0x556bef1a4010 2024-01-27T02:43:36.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.246+0000 7fdda91f9b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556bef0f6930 con 0x556bef1a35d0 2024-01-27T02:43:36.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.246+0000 7fdd9d491700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1ec360 0x556bef1ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.246+0000 7fdd9d491700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1ec360 0x556bef1ce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34846/0 (socket says 172.21.15.7:34846) 2024-01-27T02:43:36.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.246+0000 7fdd9d491700 1 -- 172.21.15.7:0/4226994931 learned_addr learned my addr 172.21.15.7:0/4226994931 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:36.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.246+0000 7fdd9e493700 1 --2- 172.21.15.7:0/4226994931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bef1a35d0 0x556bef1ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.246+0000 7fdd9dc92700 1 --2- 172.21.15.7:0/4226994931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bef1a4010 0x556bef1a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.246+0000 7fdd9d491700 1 -- 172.21.15.7:0/4226994931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bef1a35d0 msgr2=0x556bef1ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:36.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.247+0000 7fdd9d491700 1 --2- 172.21.15.7:0/4226994931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bef1a35d0 0x556bef1ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.247+0000 7fdd9d491700 1 -- 172.21.15.7:0/4226994931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bef1a4010 msgr2=0x556bef1a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:36.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.247+0000 7fdd9d491700 1 --2- 172.21.15.7:0/4226994931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bef1a4010 0x556bef1a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.247+0000 7fdd9d491700 1 -- 172.21.15.7:0/4226994931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556beef31b10 con 0x556bef1ec360 2024-01-27T02:43:36.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.247+0000 7fdd9e493700 1 --2- 172.21.15.7:0/4226994931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bef1a35d0 0x556bef1ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:36.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.247+0000 7fdd9d491700 1 --2- 172.21.15.7:0/4226994931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1ec360 0x556bef1ce840 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800fb20 tx=0x7fdd88016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff6e55f34bfa58d server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:36.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.247+0000 7fdd82ffd700 1 -- 172.21.15.7:0/4226994931 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8800fee0 con 0x556bef1ec360 2024-01-27T02:43:36.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.247+0000 7fdd82ffd700 1 -- 172.21.15.7:0/4226994931 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd88019070 con 0x556bef1ec360 2024-01-27T02:43:36.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.247+0000 7fdd82ffd700 1 -- 172.21.15.7:0/4226994931 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd88014b20 con 0x556bef1ec360 2024-01-27T02:43:36.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.247+0000 7fdda91f9b80 1 -- 172.21.15.7:0/4226994931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1ec360 msgr2=0x556bef1ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:36.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.247+0000 7fdda91f9b80 1 --2- 172.21.15.7:0/4226994931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1ec360 0x556bef1ce840 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800fb20 tx=0x7fdd88016000 comp rx=0 tx=0).stop 2024-01-27T02:43:36.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.248+0000 7fdda91f9b80 1 -- 172.21.15.7:0/4226994931 shutdown_connections 2024-01-27T02:43:36.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.248+0000 7fdda91f9b80 1 --2- 172.21.15.7:0/4226994931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bef1a35d0 0x556bef1ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.248+0000 7fdda91f9b80 1 --2- 172.21.15.7:0/4226994931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bef1a4010 0x556bef1a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.248+0000 7fdda91f9b80 1 --2- 172.21.15.7:0/4226994931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1ec360 0x556bef1ce840 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.248+0000 7fdda91f9b80 1 -- 172.21.15.7:0/4226994931 >> 172.21.15.7:0/4226994931 conn(0x556bef1a3a70 msgr2=0x556bef1ec820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:36.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.248+0000 7fdda91f9b80 1 -- 172.21.15.7:0/4226994931 shutdown_connections 2024-01-27T02:43:36.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.248+0000 7fdda91f9b80 1 -- 172.21.15.7:0/4226994931 wait complete. 2024-01-27T02:43:36.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.248+0000 7fdda91f9b80 1 -- start start 2024-01-27T02:43:36.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.248+0000 7fdda91f9b80 1 Processor -- start 2024-01-27T02:43:36.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.249+0000 7fdda91f9b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bef1a35d0 0x556bef1c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.249+0000 7fdda91f9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bef1a4010 0x556bef1e45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.249+0000 7fdda91f9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1ec360 0x556bef1e5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.249+0000 7fdda91f9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bef0f2720 con 0x556bef1ec360 2024-01-27T02:43:36.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.249+0000 7fdda91f9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bef09e1b0 con 0x556bef1a4010 2024-01-27T02:43:36.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.249+0000 7fdda91f9b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556bef09de50 con 0x556bef1a35d0 2024-01-27T02:43:36.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.250+0000 7fdda91f9b80 10 client.0 _ll_register_callbacks cb 0x556beef007b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-27T02:43:36.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.250+0000 7fdd9e493700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1ec360 0x556bef1e5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.250+0000 7fdd9e493700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1ec360 0x556bef1e5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34858/0 (socket says 172.21.15.7:34858) 2024-01-27T02:43:36.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.250+0000 7fdd9e493700 1 -- 172.21.15.7:0/903977686 learned_addr learned my addr 172.21.15.7:0/903977686 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:36.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.250+0000 7fdd9d491700 1 --2- 172.21.15.7:0/903977686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bef1a4010 0x556bef1e45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.250+0000 7fdd9dc92700 1 --2- 172.21.15.7:0/903977686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bef1a35d0 0x556bef1c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.250+0000 7fdd9e493700 1 -- 172.21.15.7:0/903977686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bef1a35d0 msgr2=0x556bef1c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:36.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.250+0000 7fdd9e493700 1 --2- 172.21.15.7:0/903977686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bef1a35d0 0x556bef1c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.250+0000 7fdd9e493700 1 -- 172.21.15.7:0/903977686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bef1a4010 msgr2=0x556bef1e45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:36.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.250+0000 7fdd9e493700 1 --2- 172.21.15.7:0/903977686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bef1a4010 0x556bef1e45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.250+0000 7fdd9e493700 1 -- 172.21.15.7:0/903977686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8800e040 con 0x556bef1ec360 2024-01-27T02:43:36.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.250+0000 7fdd9d491700 1 --2- 172.21.15.7:0/903977686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bef1a4010 0x556bef1e45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:36.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.250+0000 7fdd9e493700 1 --2- 172.21.15.7:0/903977686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1ec360 0x556bef1e5ca0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fdd9806dfe0 tx=0x7fdd90019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28d6160f5b693f9 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:36.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.250+0000 7fdd77fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-27T02:43:36.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.250+0000 7fdd77fff700 1 -- 172.21.15.7:0/903977686 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd90023070 con 0x556bef1ec360 2024-01-27T02:43:36.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.250+0000 7fdd77fff700 1 -- 172.21.15.7:0/903977686 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(180..180 src has 1..180) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdd9001d620 con 0x556bef1ec360 2024-01-27T02:43:36.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.250+0000 7fdda91f9b80 1 -- 172.21.15.7:0/903977686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556bef197190 con 0x556bef1ec360 2024-01-27T02:43:36.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.250+0000 7fdd77fff700 1 -- 172.21.15.7:0/903977686 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd9000be70 con 0x556bef1ec360 2024-01-27T02:43:36.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.250+0000 7fdda91f9b80 10 client.8241 fetch_fsmap learned FSMap version 94 2024-01-27T02:43:36.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.250+0000 7fdda91f9b80 1 -- 172.21.15.7:0/903977686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x556bef0c0290 con 0x556bef1ec360 2024-01-27T02:43:36.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.251+0000 7fdd77fff700 1 -- 172.21.15.7:0/903977686 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556bef197190 con 0x556bef1ec360 2024-01-27T02:43:36.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.251+0000 7fdd77fff700 1 -- 172.21.15.7:0/903977686 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 94) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fdd9001d9f0 con 0x556bef1ec360 2024-01-27T02:43:36.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.251+0000 7fdda91f9b80 10 client.8241 fetch_fsmap finished waiting for FSMap version 94 2024-01-27T02:43:36.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.251+0000 7fdda91f9b80 10 client.8241 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-27T02:43:36.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.251+0000 7fdda91f9b80 20 client.8241 populate_metadata read hostname 'smithi007' 2024-01-27T02:43:36.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.251+0000 7fdda91f9b80 1 --2- 172.21.15.7:0/903977686 >> [v2:172.21.15.188:6835/3853795510,v1:172.21.15.188:6837/3853795510] conn(0x556bef1eb340 0x556bef1eb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.251+0000 7fdda91f9b80 4 client.8241 mds_command: new command op to 7702 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-27T02:43:36.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.251+0000 7fdda91f9b80 1 -- 172.21.15.7:0/903977686 --> [v2:172.21.15.188:6835/3853795510,v1:172.21.15.188:6837/3853795510] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556bef094ee0 con 0x556bef1eb340 2024-01-27T02:43:36.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.251+0000 7fdd9dc92700 1 --2- 172.21.15.7:0/903977686 >> [v2:172.21.15.188:6835/3853795510,v1:172.21.15.188:6837/3853795510] conn(0x556bef1eb340 0x556bef1eb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.251+0000 7fdd77fff700 10 client.8241 ms_handle_connect on v2:172.21.15.188:6835/3853795510 2024-01-27T02:43:36.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.251+0000 7fdd9dc92700 1 --2- 172.21.15.7:0/903977686 >> [v2:172.21.15.188:6835/3853795510,v1:172.21.15.188:6837/3853795510] conn(0x556bef1eb340 0x556bef1eb710 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f761aab67cbd6c6 server_cookie=e5d036547a00afcb in_seq=0 out_seq=0 2024-01-27T02:43:36.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.252+0000 7fdd77fff700 1 -- 172.21.15.7:0/903977686 <== mds.0 v2:172.21.15.188:6835/3853795510 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17146 (crc 0 0 0) 0x556bef094ee0 con 0x556bef1eb340 2024-01-27T02:43:36.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.252+0000 7fdd77fff700 10 client.8241 handle_command_reply: tid=0 2024-01-27T02:43:36.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.252+0000 7fdda91f9b80 1 client.8241 shutdown 2024-01-27T02:43:36.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.252+0000 7fdda91f9b80 1 -- 172.21.15.7:0/903977686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1ec360 msgr2=0x556bef1e5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:36.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.252+0000 7fdda91f9b80 1 --2- 172.21.15.7:0/903977686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1ec360 0x556bef1e5ca0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fdd9806dfe0 tx=0x7fdd90019000 comp rx=0 tx=0).stop 2024-01-27T02:43:36.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.252+0000 7fdda91f9b80 1 -- 172.21.15.7:0/903977686 shutdown_connections 2024-01-27T02:43:36.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.252+0000 7fdda91f9b80 1 --2- 172.21.15.7:0/903977686 >> [v2:172.21.15.188:6835/3853795510,v1:172.21.15.188:6837/3853795510] conn(0x556bef1eb340 0x556bef1eb710 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.252+0000 7fdda91f9b80 1 --2- 172.21.15.7:0/903977686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1ec360 0x556bef1e5ca0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.252+0000 7fdda91f9b80 1 --2- 172.21.15.7:0/903977686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bef1a35d0 0x556bef1c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.252+0000 7fdda91f9b80 1 --2- 172.21.15.7:0/903977686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bef1a4010 0x556bef1e45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.252+0000 7fdd77fff700 0 client.8241 ms_handle_reset on v2:172.21.15.188:6835/3853795510 2024-01-27T02:43:36.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.252+0000 7fdda91f9b80 1 -- 172.21.15.7:0/903977686 >> 172.21.15.7:0/903977686 conn(0x556bef1a3a70 msgr2=0x556bef1cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:36.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.253+0000 7fdda91f9b80 1 -- 172.21.15.7:0/903977686 shutdown_connections 2024-01-27T02:43:36.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.253+0000 7fdda91f9b80 1 -- 172.21.15.7:0/903977686 wait complete. 2024-01-27T02:43:36.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.253+0000 7fdda91f9b80 20 client.8241 trim_cache size 0 max 16384 2024-01-27T02:43:36.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.270+0000 7fdda91f9b80 1 Processor -- start 2024-01-27T02:43:36.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.271+0000 7fdda91f9b80 1 -- start start 2024-01-27T02:43:36.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.271+0000 7fdda91f9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bef1a0360 0x556bef1ed270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.271+0000 7fdda91f9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1a3a70 0x556bef1cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.271+0000 7fdda91f9b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bef1a35d0 0x556bef1cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.271+0000 7fdda91f9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bef19fae0 con 0x556bef1a3a70 2024-01-27T02:43:36.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.271+0000 7fdda91f9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556beefeae40 con 0x556bef1a0360 2024-01-27T02:43:36.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.271+0000 7fdda91f9b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556bef1ed7b0 con 0x556bef1a35d0 2024-01-27T02:43:36.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.271+0000 7fdd9d491700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1a3a70 0x556bef1cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.271+0000 7fdd9d491700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1a3a70 0x556bef1cd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34862/0 (socket says 172.21.15.7:34862) 2024-01-27T02:43:36.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.271+0000 7fdd9d491700 1 -- 172.21.15.7:0/902661709 learned_addr learned my addr 172.21.15.7:0/902661709 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:36.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.272+0000 7fdd9dc92700 1 --2- 172.21.15.7:0/902661709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bef1a0360 0x556bef1ed270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.272+0000 7fdd9e493700 1 --2- 172.21.15.7:0/902661709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bef1a35d0 0x556bef1cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.272+0000 7fdd9d491700 1 -- 172.21.15.7:0/902661709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bef1a35d0 msgr2=0x556bef1cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:36.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.272+0000 7fdd9d491700 1 --2- 172.21.15.7:0/902661709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bef1a35d0 0x556bef1cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.272+0000 7fdd9d491700 1 -- 172.21.15.7:0/902661709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bef1a0360 msgr2=0x556bef1ed270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:36.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.272+0000 7fdd9d491700 1 --2- 172.21.15.7:0/902661709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bef1a0360 0x556bef1ed270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.272+0000 7fdd9d491700 1 -- 172.21.15.7:0/902661709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556beef89c80 con 0x556bef1a3a70 2024-01-27T02:43:36.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.272+0000 7fdd9d491700 1 --2- 172.21.15.7:0/902661709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1a3a70 0x556bef1cd7a0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fdd88021000 tx=0x7fdd88020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67585a66160a492 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:36.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.272+0000 7fdd837fe700 1 -- 172.21.15.7:0/902661709 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd88014870 con 0x556bef1a3a70 2024-01-27T02:43:36.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.272+0000 7fdd837fe700 1 -- 172.21.15.7:0/902661709 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd880149d0 con 0x556bef1a3a70 2024-01-27T02:43:36.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.272+0000 7fdd837fe700 1 -- 172.21.15.7:0/902661709 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd88005c10 con 0x556bef1a3a70 2024-01-27T02:43:36.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.273+0000 7fdda91f9b80 1 -- 172.21.15.7:0/902661709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1a3a70 msgr2=0x556bef1cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:36.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.273+0000 7fdda91f9b80 1 --2- 172.21.15.7:0/902661709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1a3a70 0x556bef1cd7a0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fdd88021000 tx=0x7fdd88020000 comp rx=0 tx=0).stop 2024-01-27T02:43:36.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.273+0000 7fdda91f9b80 1 -- 172.21.15.7:0/902661709 shutdown_connections 2024-01-27T02:43:36.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.273+0000 7fdda91f9b80 1 --2- 172.21.15.7:0/902661709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bef1a35d0 0x556bef1cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.273+0000 7fdda91f9b80 1 --2- 172.21.15.7:0/902661709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bef1a0360 0x556bef1ed270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.273+0000 7fdda91f9b80 1 --2- 172.21.15.7:0/902661709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1a3a70 0x556bef1cd7a0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.273+0000 7fdda91f9b80 1 -- 172.21.15.7:0/902661709 >> 172.21.15.7:0/902661709 conn(0x556bef1ceb20 msgr2=0x556bef19d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:36.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.273+0000 7fdda91f9b80 1 -- 172.21.15.7:0/902661709 shutdown_connections 2024-01-27T02:43:36.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.273+0000 7fdda91f9b80 1 -- 172.21.15.7:0/902661709 wait complete. 2024-01-27T02:43:36.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.274+0000 7fdda91f9b80 1 -- start start 2024-01-27T02:43:36.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.274+0000 7fdda91f9b80 1 Processor -- start 2024-01-27T02:43:36.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdda91f9b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1a0360 0x556bef1c6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdda91f9b80 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bef1a35d0 0x556bef24ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdda91f9b80 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bef1a3a70 0x556bef1c5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdda91f9b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bef24e940 con 0x556bef1a0360 2024-01-27T02:43:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdda91f9b80 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bef1caaa0 con 0x556bef1a3a70 2024-01-27T02:43:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdda91f9b80 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556bef1c6ba0 con 0x556bef1a35d0 2024-01-27T02:43:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdda91f9b80 10 client.0 _ll_register_callbacks cb 0x556beef007b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-27T02:43:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdd9dc92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1a0360 0x556bef1c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdd9dc92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1a0360 0x556bef1c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34866/0 (socket says 172.21.15.7:34866) 2024-01-27T02:43:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdd9dc92700 1 -- 172.21.15.7:0/1993793384 learned_addr learned my addr 172.21.15.7:0/1993793384 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdd9e493700 1 --2- 172.21.15.7:0/1993793384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bef1a3a70 0x556bef1c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdd9d491700 1 --2- 172.21.15.7:0/1993793384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bef1a35d0 0x556bef24ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdd9dc92700 1 -- 172.21.15.7:0/1993793384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bef1a35d0 msgr2=0x556bef24ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdd9dc92700 1 --2- 172.21.15.7:0/1993793384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bef1a35d0 0x556bef24ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdd9dc92700 1 -- 172.21.15.7:0/1993793384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bef1a3a70 msgr2=0x556bef1c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdd9dc92700 1 --2- 172.21.15.7:0/1993793384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bef1a3a70 0x556bef1c5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdd9dc92700 1 -- 172.21.15.7:0/1993793384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556beef31b10 con 0x556bef1a0360 2024-01-27T02:43:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdd9d491700 1 --2- 172.21.15.7:0/1993793384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bef1a35d0 0x556bef24ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:43:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdd9dc92700 1 --2- 172.21.15.7:0/1993793384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1a0360 0x556bef1c6660 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fdd94054ce0 tx=0x7fdd9401d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6005e902c8af87 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdd77fff700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-27T02:43:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdd77fff700 1 -- 172.21.15.7:0/1993793384 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd94082730 con 0x556bef1a0360 2024-01-27T02:43:36.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdd77fff700 1 -- 172.21.15.7:0/1993793384 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(180..180 src has 1..180) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdd94080070 con 0x556bef1a0360 2024-01-27T02:43:36.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.275+0000 7fdda91f9b80 1 -- 172.21.15.7:0/1993793384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd88019070 con 0x556bef1a0360 2024-01-27T02:43:36.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.276+0000 7fdd77fff700 1 -- 172.21.15.7:0/1993793384 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd94055240 con 0x556bef1a0360 2024-01-27T02:43:36.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.276+0000 7fdda91f9b80 10 client.8247 fetch_fsmap learned FSMap version 94 2024-01-27T02:43:36.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.276+0000 7fdda91f9b80 1 -- 172.21.15.7:0/1993793384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x556beef2ffe0 con 0x556bef1a0360 2024-01-27T02:43:36.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.276+0000 7fdd77fff700 1 -- 172.21.15.7:0/1993793384 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd88019070 con 0x556bef1a0360 2024-01-27T02:43:36.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.276+0000 7fdd77fff700 1 -- 172.21.15.7:0/1993793384 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 94) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fdd940544d0 con 0x556bef1a0360 2024-01-27T02:43:36.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.276+0000 7fdda91f9b80 10 client.8247 fetch_fsmap finished waiting for FSMap version 94 2024-01-27T02:43:36.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.276+0000 7fdda91f9b80 10 client.8247 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-27T02:43:36.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.276+0000 7fdda91f9b80 20 client.8247 populate_metadata read hostname 'smithi007' 2024-01-27T02:43:36.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.276+0000 7fdda91f9b80 1 --2- 172.21.15.7:0/1993793384 >> [v2:172.21.15.188:6835/3853795510,v1:172.21.15.188:6837/3853795510] conn(0x556bef1a4010 0x556bef1ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.276+0000 7fdda91f9b80 4 client.8247 mds_command: new command op to 7702 tid=0[{"prefix": "status"}] 2024-01-27T02:43:36.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.276+0000 7fdda91f9b80 1 -- 172.21.15.7:0/1993793384 --> [v2:172.21.15.188:6835/3853795510,v1:172.21.15.188:6837/3853795510] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556bef094c00 con 0x556bef1a4010 2024-01-27T02:43:36.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.276+0000 7fdd9d491700 1 --2- 172.21.15.7:0/1993793384 >> [v2:172.21.15.188:6835/3853795510,v1:172.21.15.188:6837/3853795510] conn(0x556bef1a4010 0x556bef1ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.277+0000 7fdd77fff700 10 client.8247 ms_handle_connect on v2:172.21.15.188:6835/3853795510 2024-01-27T02:43:36.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.277+0000 7fdd9d491700 1 --2- 172.21.15.7:0/1993793384 >> [v2:172.21.15.188:6835/3853795510,v1:172.21.15.188:6837/3853795510] conn(0x556bef1a4010 0x556bef1ffc70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=279babe21bf50141 server_cookie=f9fe790a7b832f1a in_seq=0 out_seq=0 2024-01-27T02:43:36.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.277+0000 7fdd77fff700 1 -- 172.21.15.7:0/1993793384 <== mds.0 v2:172.21.15.188:6835/3853795510 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x556bef094c00 con 0x556bef1a4010 2024-01-27T02:43:36.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.277+0000 7fdd77fff700 10 client.8247 handle_command_reply: tid=0 2024-01-27T02:43:36.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.277+0000 7fdda91f9b80 1 client.8247 shutdown 2024-01-27T02:43:36.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.277+0000 7fdda91f9b80 1 -- 172.21.15.7:0/1993793384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1a0360 msgr2=0x556bef1c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:36.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.277+0000 7fdda91f9b80 1 --2- 172.21.15.7:0/1993793384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1a0360 0x556bef1c6660 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fdd94054ce0 tx=0x7fdd9401d040 comp rx=0 tx=0).stop 2024-01-27T02:43:36.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.277+0000 7fdda91f9b80 1 -- 172.21.15.7:0/1993793384 shutdown_connections 2024-01-27T02:43:36.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.277+0000 7fdda91f9b80 1 --2- 172.21.15.7:0/1993793384 >> [v2:172.21.15.188:6835/3853795510,v1:172.21.15.188:6837/3853795510] conn(0x556bef1a4010 0x556bef1ffc70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.277+0000 7fdda91f9b80 1 --2- 172.21.15.7:0/1993793384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bef1a3a70 0x556bef1c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.277+0000 7fdda91f9b80 1 --2- 172.21.15.7:0/1993793384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bef1a0360 0x556bef1c6660 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.277+0000 7fdda91f9b80 1 --2- 172.21.15.7:0/1993793384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bef1a35d0 0x556bef24ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.277+0000 7fdd77fff700 0 client.8247 ms_handle_reset on v2:172.21.15.188:6835/3853795510 2024-01-27T02:43:36.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.277+0000 7fdda91f9b80 1 -- 172.21.15.7:0/1993793384 >> 172.21.15.7:0/1993793384 conn(0x556bef1ceb20 msgr2=0x556bef1cd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:36.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.277+0000 7fdda91f9b80 1 -- 172.21.15.7:0/1993793384 shutdown_connections 2024-01-27T02:43:36.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.277+0000 7fdda91f9b80 1 -- 172.21.15.7:0/1993793384 wait complete. 2024-01-27T02:43:36.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.277+0000 7fdda91f9b80 20 client.8247 trim_cache size 0 max 16384 2024-01-27T02:43:36.287 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:43:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:43:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-27T02:43:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 7702, 2024-01-27T02:43:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-27T02:43:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-27T02:43:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-27T02:43:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.0945696759999999, 2024-01-27T02:43:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 94, 2024-01-27T02:43:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 180, 2024-01-27T02:43:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 180, 2024-01-27T02:43:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 12.069084442999999 2024-01-27T02:43:36.288 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:43:36.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.278+0000 7fdd9fef6700 1 -- 172.21.15.7:0/427938296 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdd7c066f30 msgr2=0x7fdd7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:36.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.278+0000 7fdd9fef6700 1 --2- 172.21.15.7:0/427938296 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdd7c066f30 0x7fdd7c0693e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdd88006f00 tx=0x7fdd88008040 comp rx=0 tx=0).stop 2024-01-27T02:43:36.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.278+0000 7fdd9fef6700 1 -- 172.21.15.7:0/427938296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd980b1c00 msgr2=0x7fdd9816e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:36.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.278+0000 7fdd9fef6700 1 --2- 172.21.15.7:0/427938296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd980b1c00 0x7fdd9816e530 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400c7e0 tx=0x7fdd94007df0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.278+0000 7fdd9fef6700 1 -- 172.21.15.7:0/427938296 shutdown_connections 2024-01-27T02:43:36.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.278+0000 7fdd9fef6700 1 --2- 172.21.15.7:0/427938296 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdd7c066f30 0x7fdd7c0693e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.278+0000 7fdd9fef6700 1 --2- 172.21.15.7:0/427938296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd9815b5f0 0x7fdd9814c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.278+0000 7fdd9fef6700 1 --2- 172.21.15.7:0/427938296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd980b1c00 0x7fdd9816e530 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.278+0000 7fdd9fef6700 1 --2- 172.21.15.7:0/427938296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd980b1fd0 0x7fdd98157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.278+0000 7fdd9fef6700 1 -- 172.21.15.7:0/427938296 >> 172.21.15.7:0/427938296 conn(0x7fdd98068430 msgr2=0x7fdd98156490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:36.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.278+0000 7fdd9fef6700 1 -- 172.21.15.7:0/427938296 shutdown_connections 2024-01-27T02:43:36.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.278+0000 7fdd9fef6700 1 -- 172.21.15.7:0/427938296 wait complete. 2024-01-27T02:43:36.292 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:43:36.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.423+0000 7f5234dfc700 1 Processor -- start 2024-01-27T02:43:36.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.423+0000 7f5234dfc700 1 -- start start 2024-01-27T02:43:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.424+0000 7f5234dfc700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5230063c90 0x7f52300c8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.424+0000 7f5234dfc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f52301438f0 0x7f52300c8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.424+0000 7f5234dfc700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52300cb7a0 con 0x7f5230143520 2024-01-27T02:43:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.424+0000 7f5234dfc700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52300c8c90 con 0x7f52301438f0 2024-01-27T02:43:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.424+0000 7f5234dfc700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52300cb9f0 con 0x7f5230063c90 2024-01-27T02:43:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.425+0000 7f522ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f52301438f0 0x7f52300c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.425+0000 7f522ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f52301438f0 0x7f52300c8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34880/0 (socket says 172.21.15.7:34880) 2024-01-27T02:43:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.425+0000 7f522ed9d700 1 -- 172.21.15.7:0/3624582803 learned_addr learned my addr 172.21.15.7:0/3624582803 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.425+0000 7f522dd9b700 1 --2- 172.21.15.7:0/3624582803 >> v2:172.21.15.188:3300/0 conn(0x7f5230063c90 0x7f52300c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.425+0000 7f522ed9d700 1 -- 172.21.15.7:0/3624582803 >> v2:172.21.15.188:3300/0 conn(0x7f5230063c90 msgr2=0x7f52300c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.425+0000 7f522ed9d700 1 --2- 172.21.15.7:0/3624582803 >> v2:172.21.15.188:3300/0 conn(0x7f5230063c90 0x7f52300c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.425+0000 7f522ed9d700 1 -- 172.21.15.7:0/3624582803 >> v1:172.21.15.188:6789/0 conn(0x7f5230143520 legacy=0x7f5230063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.425+0000 7f522ed9d700 1 -- 172.21.15.7:0/3624582803 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52300cbcd0 con 0x7f52301438f0 2024-01-27T02:43:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.425+0000 7f522ed9d700 1 --2- 172.21.15.7:0/3624582803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52301438f0 0x7f52300c8750 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f52300c9e60 tx=0x7f5228009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce56a1d7fa093fa server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.425+0000 7f522d59a700 1 -- 172.21.15.7:0/3624582803 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522800e070 con 0x7f52301438f0 2024-01-27T02:43:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.425+0000 7f522d59a700 1 -- 172.21.15.7:0/3624582803 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522800ba50 con 0x7f52301438f0 2024-01-27T02:43:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.425+0000 7f522d59a700 1 -- 172.21.15.7:0/3624582803 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52280092e0 con 0x7f52301438f0 2024-01-27T02:43:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.426+0000 7f5234dfc700 1 -- 172.21.15.7:0/3624582803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52301438f0 msgr2=0x7f52300c8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.426+0000 7f5234dfc700 1 --2- 172.21.15.7:0/3624582803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52301438f0 0x7f52300c8750 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f52300c9e60 tx=0x7f5228009840 comp rx=0 tx=0).stop 2024-01-27T02:43:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.426+0000 7f5234dfc700 1 -- 172.21.15.7:0/3624582803 shutdown_connections 2024-01-27T02:43:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.426+0000 7f5234dfc700 1 --2- 172.21.15.7:0/3624582803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52301438f0 0x7f52300c8750 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.426+0000 7f5234dfc700 1 --2- 172.21.15.7:0/3624582803 >> v2:172.21.15.188:3300/0 conn(0x7f5230063c90 0x7f52300c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.426+0000 7f5234dfc700 1 -- 172.21.15.7:0/3624582803 >> 172.21.15.7:0/3624582803 conn(0x7f52300cdbd0 msgr2=0x7f52300cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.426+0000 7f5234dfc700 1 -- 172.21.15.7:0/3624582803 shutdown_connections 2024-01-27T02:43:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.426+0000 7f5234dfc700 1 -- 172.21.15.7:0/3624582803 wait complete. 2024-01-27T02:43:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.428+0000 7f5234dfc700 1 Processor -- start 2024-01-27T02:43:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.428+0000 7f5234dfc700 1 -- start start 2024-01-27T02:43:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.429+0000 7f5234dfc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5230063c90 0x7f523006eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.429+0000 7f5234dfc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5230143520 0x7f52301405b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.429+0000 7f5234dfc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52301438f0 0x7f523016d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.429+0000 7f5234dfc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52300c67e0 con 0x7f52301438f0 2024-01-27T02:43:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.429+0000 7f5234dfc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52300c6ac0 con 0x7f5230143520 2024-01-27T02:43:36.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.429+0000 7f5234dfc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52300c6da0 con 0x7f5230063c90 2024-01-27T02:43:36.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.429+0000 7f522ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52301438f0 0x7f523016d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.429+0000 7f522ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52301438f0 0x7f523016d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34892/0 (socket says 172.21.15.7:34892) 2024-01-27T02:43:36.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.429+0000 7f522ed9d700 1 -- 172.21.15.7:0/2993923773 learned_addr learned my addr 172.21.15.7:0/2993923773 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:36.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.429+0000 7f522dd9b700 1 --2- 172.21.15.7:0/2993923773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5230143520 0x7f52301405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.429+0000 7f522e59c700 1 --2- 172.21.15.7:0/2993923773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5230063c90 0x7f523006eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.429+0000 7f522ed9d700 1 -- 172.21.15.7:0/2993923773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5230063c90 msgr2=0x7f523006eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:36.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.429+0000 7f522ed9d700 1 --2- 172.21.15.7:0/2993923773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5230063c90 0x7f523006eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.429+0000 7f522ed9d700 1 -- 172.21.15.7:0/2993923773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5230143520 msgr2=0x7f52301405b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:36.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.429+0000 7f522ed9d700 1 --2- 172.21.15.7:0/2993923773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5230143520 0x7f52301405b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.429+0000 7f522ed9d700 1 -- 172.21.15.7:0/2993923773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5228009ae0 con 0x7f52301438f0 2024-01-27T02:43:36.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.429+0000 7f522dd9b700 1 --2- 172.21.15.7:0/2993923773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5230143520 0x7f52301405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:36.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.430+0000 7f522ed9d700 1 --2- 172.21.15.7:0/2993923773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52301438f0 0x7f523016d7c0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f52300c49d0 tx=0x7f5228018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:36.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.430+0000 7f521e7fc700 1 -- 172.21.15.7:0/2993923773 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228007000 con 0x7f52301438f0 2024-01-27T02:43:36.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.430+0000 7f521e7fc700 1 -- 172.21.15.7:0/2993923773 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522800be80 con 0x7f52301438f0 2024-01-27T02:43:36.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.430+0000 7f5234dfc700 1 -- 172.21.15.7:0/2993923773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52300c7080 con 0x7f52301438f0 2024-01-27T02:43:36.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.430+0000 7f521e7fc700 1 -- 172.21.15.7:0/2993923773 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522801c660 con 0x7f52301438f0 2024-01-27T02:43:36.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.430+0000 7f5234dfc700 1 -- 172.21.15.7:0/2993923773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f523016e490 con 0x7f52301438f0 2024-01-27T02:43:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.431+0000 7f5234dfc700 1 -- 172.21.15.7:0/2993923773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f523004c710 con 0x7f52301438f0 2024-01-27T02:43:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.431+0000 7f521e7fc700 1 -- 172.21.15.7:0/2993923773 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5228003050 con 0x7f52301438f0 2024-01-27T02:43:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.432+0000 7f521e7fc700 1 --2- 172.21.15.7:0/2993923773 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5210066f30 0x7f52100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.432+0000 7f521e7fc700 1 -- 172.21.15.7:0/2993923773 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5228086050 con 0x7f52301438f0 2024-01-27T02:43:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.432+0000 7f522e59c700 1 --2- 172.21.15.7:0/2993923773 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5210066f30 0x7f52100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.433+0000 7f522e59c700 1 --2- 172.21.15.7:0/2993923773 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5210066f30 0x7f52100693e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f5220003890 tx=0x7f5220009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:36.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.436+0000 7f521e7fc700 1 -- 172.21.15.7:0/2993923773 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52280532c0 con 0x7f52301438f0 2024-01-27T02:43:36.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.689+0000 7f5234dfc700 1 -- 172.21.15.7:0/2993923773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f52300cbfb0 con 0x7f52301438f0 2024-01-27T02:43:36.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.690+0000 7f521e7fc700 1 -- 172.21.15.7:0/2993923773 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v94) v1 ==== 53+0+73 (secure 0 0 0) 0x7f52280568e0 con 0x7f52301438f0 2024-01-27T02:43:36.691 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:43:36.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.692+0000 7f5234dfc700 1 -- 172.21.15.7:0/2993923773 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5210066f30 msgr2=0x7f52100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:36.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.692+0000 7f5234dfc700 1 --2- 172.21.15.7:0/2993923773 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5210066f30 0x7f52100693e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f5220003890 tx=0x7f5220009040 comp rx=0 tx=0).stop 2024-01-27T02:43:36.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.692+0000 7f5234dfc700 1 -- 172.21.15.7:0/2993923773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52301438f0 msgr2=0x7f523016d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:36.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.692+0000 7f5234dfc700 1 --2- 172.21.15.7:0/2993923773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52301438f0 0x7f523016d7c0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f52300c49d0 tx=0x7f5228018000 comp rx=0 tx=0).stop 2024-01-27T02:43:36.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.692+0000 7f5234dfc700 1 -- 172.21.15.7:0/2993923773 shutdown_connections 2024-01-27T02:43:36.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.692+0000 7f5234dfc700 1 --2- 172.21.15.7:0/2993923773 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5210066f30 0x7f52100693e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.692+0000 7f5234dfc700 1 --2- 172.21.15.7:0/2993923773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52301438f0 0x7f523016d7c0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.692+0000 7f5234dfc700 1 --2- 172.21.15.7:0/2993923773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5230063c90 0x7f523006eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.692+0000 7f5234dfc700 1 --2- 172.21.15.7:0/2993923773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5230143520 0x7f52301405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:36.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.692+0000 7f5234dfc700 1 -- 172.21.15.7:0/2993923773 >> 172.21.15.7:0/2993923773 conn(0x7f52300cdbd0 msgr2=0x7f52300c2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:36.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.692+0000 7f5234dfc700 1 -- 172.21.15.7:0/2993923773 shutdown_connections 2024-01-27T02:43:36.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:36.693+0000 7f5234dfc700 1 -- 172.21.15.7:0/2993923773 wait complete. 2024-01-27T02:43:36.707 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-27T02:43:36.707 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:43:36.707 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-27T02:43:36.707 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-27T02:43:36.707 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-27T02:43:36.707 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-27T02:43:36.707 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-27T02:43:36.707 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:43:36.708 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:43:36.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:43:36.708 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-27T02:43:36.728 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:43:36.728 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:43:36.728 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:43:36.729 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:43:36.729 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-27T02:43:36.729 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:43:36.729 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:43:36.729 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-27T02:43:36.729 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-27T02:43:36.729 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:43:36.729 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-27T02:43:36.729 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-27T02:43:36.729 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:43:36.729 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-27T02:43:36.729 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-27T02:43:36.730 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-27T02:43:36.730 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 41694sec preferred_lft 41694sec 2024-01-27T02:43:36.730 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-27T02:43:36.730 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:43:36.730 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:43:36.730 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-27T02:43:36.730 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-27T02:43:36.730 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:43:36.730 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 4a:57:d9:3c:c5:a0 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:43:36.730 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-27T02:43:36.731 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:43:36.731 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::2477:9eff:fe39:433c/64 scope link 2024-01-27T02:43:36.731 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:43:36.731 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:43:36.731 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 4a:57:d9:3c:c5:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-27T02:43:36.731 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::4857:d9ff:fe3c:c5a0/64 scope link 2024-01-27T02:43:36.731 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:43:36.731 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:43:36.731 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ca:6e:b8:df:53:bf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-27T02:43:36.731 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c86e:b8ff:fedf:53bf/64 scope link 2024-01-27T02:43:36.731 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:43:36.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:43:36.732 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:43:36.799 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-27T02:43:36.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:43:36.801 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-27T02:43:36.864 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-27T02:43:36.864 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:43:36.864 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-27T02:43:36.864 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-27T02:43:36.864 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:43:36.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:43:36.865 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-27T02:43:36.866 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-27T02:43:36.866 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-27T02:43:36.866 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-27T02:43:36.866 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-27T02:43:36.866 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-27T02:43:36.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:43:36.990 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-27T02:43:37.010 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-27T02:43:37.010 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-27T02:43:37.011 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:43:37.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:43:37.011 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-27T02:43:37.085 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:43:37.085 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-27T02:43:37.147 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-27T02:43:37.148 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:43:37.148 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-27T02:43:37.210 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:43:37.210 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:43:37.210 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:43:37.305 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:43:37.305 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:43:37.305 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:43:37.315 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:43:37.315 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:43:37.315 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:43:37.315 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:43:37.315 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:43:37.333 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-27T02:43:37.333 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-27T02:43:37.333 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-27T02:43:37.333 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:43:37.333 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:43:37.333 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:43:37.333 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:43:37.333 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:43:37.336 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:43:37.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:43:37.336 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-27T02:43:37.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-27T02:43:37.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.550+0000 7f2f0eb02700 1 Processor -- start 2024-01-27T02:43:37.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.550+0000 7f2f0eb02700 1 -- start start 2024-01-27T02:43:37.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.551+0000 7f2f0eb02700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f08157ca0 0x7f2f080c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:37.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.551+0000 7f2f0eb02700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2f080c3ae0 0x7f2f080cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:37.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.551+0000 7f2f0eb02700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f080ca020 con 0x7f2f08158070 2024-01-27T02:43:37.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.551+0000 7f2f0eb02700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f080c4f40 con 0x7f2f080c3ae0 2024-01-27T02:43:37.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.551+0000 7f2f0eb02700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f080c5210 con 0x7f2f08157ca0 2024-01-27T02:43:37.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.551+0000 7f2f07fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2f080c3ae0 0x7f2f080cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:37.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.551+0000 7f2f07fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2f080c3ae0 0x7f2f080cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34902/0 (socket says 172.21.15.7:34902) 2024-01-27T02:43:37.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.551+0000 7f2f07fff700 1 -- 172.21.15.7:0/1469381069 learned_addr learned my addr 172.21.15.7:0/1469381069 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:37.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.551+0000 7f2f0c89e700 1 --2- 172.21.15.7:0/1469381069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f08157ca0 0x7f2f080c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:37.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.552+0000 7f2f07fff700 1 -- 172.21.15.7:0/1469381069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f08157ca0 msgr2=0x7f2f080c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:37.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.552+0000 7f2f07fff700 1 --2- 172.21.15.7:0/1469381069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f08157ca0 0x7f2f080c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:37.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.552+0000 7f2f07fff700 1 -- 172.21.15.7:0/1469381069 >> v1:172.21.15.188:6789/0 conn(0x7f2f08158070 legacy=0x7f2f0815bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:37.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.552+0000 7f2f07fff700 1 -- 172.21.15.7:0/1469381069 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f080c54f0 con 0x7f2f080c3ae0 2024-01-27T02:43:37.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.552+0000 7f2f07fff700 1 --2- 172.21.15.7:0/1469381069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f080c3ae0 0x7f2f080cd4e0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0099b0 tx=0x7f2efc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5467175fa6c3fd47 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:37.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.552+0000 7f2f077fe700 1 -- 172.21.15.7:0/1469381069 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc00b860 con 0x7f2f080c3ae0 2024-01-27T02:43:37.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.552+0000 7f2f077fe700 1 -- 172.21.15.7:0/1469381069 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2efc00b9c0 con 0x7f2f080c3ae0 2024-01-27T02:43:37.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.552+0000 7f2f077fe700 1 -- 172.21.15.7:0/1469381069 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc00bd60 con 0x7f2f080c3ae0 2024-01-27T02:43:37.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.553+0000 7f2f0eb02700 1 -- 172.21.15.7:0/1469381069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f080c3ae0 msgr2=0x7f2f080cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:37.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.553+0000 7f2f0eb02700 1 --2- 172.21.15.7:0/1469381069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f080c3ae0 0x7f2f080cd4e0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0099b0 tx=0x7f2efc009250 comp rx=0 tx=0).stop 2024-01-27T02:43:37.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.553+0000 7f2f0eb02700 1 -- 172.21.15.7:0/1469381069 shutdown_connections 2024-01-27T02:43:37.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.553+0000 7f2f0eb02700 1 --2- 172.21.15.7:0/1469381069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f08157ca0 0x7f2f080c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:37.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.553+0000 7f2f0eb02700 1 --2- 172.21.15.7:0/1469381069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f080c3ae0 0x7f2f080cd4e0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:37.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.553+0000 7f2f0eb02700 1 -- 172.21.15.7:0/1469381069 >> 172.21.15.7:0/1469381069 conn(0x7f2f08098020 msgr2=0x7f2f080711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:37.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.553+0000 7f2f0eb02700 1 -- 172.21.15.7:0/1469381069 shutdown_connections 2024-01-27T02:43:37.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.553+0000 7f2f0eb02700 1 -- 172.21.15.7:0/1469381069 wait complete. 2024-01-27T02:43:37.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.554+0000 7f2f0eb02700 1 Processor -- start 2024-01-27T02:43:37.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.554+0000 7f2f0eb02700 1 -- start start 2024-01-27T02:43:37.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.555+0000 7f2f0eb02700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f080c3ae0 0x7f2f0813f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:37.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.555+0000 7f2f0eb02700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f08157ca0 0x7f2f08155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:37.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.555+0000 7f2f0eb02700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f08158070 0x7f2f081518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:37.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.555+0000 7f2f0eb02700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f08156fe0 con 0x7f2f080c3ae0 2024-01-27T02:43:37.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.555+0000 7f2f0eb02700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f08140150 con 0x7f2f08158070 2024-01-27T02:43:37.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.555+0000 7f2f0eb02700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f08156740 con 0x7f2f08157ca0 2024-01-27T02:43:37.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.555+0000 7f2f0c89e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f080c3ae0 0x7f2f0813f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:37.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.555+0000 7f2f0c89e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f080c3ae0 0x7f2f0813f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34904/0 (socket says 172.21.15.7:34904) 2024-01-27T02:43:37.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.555+0000 7f2f0c89e700 1 -- 172.21.15.7:0/1093493596 learned_addr learned my addr 172.21.15.7:0/1093493596 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:37.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.555+0000 7f2f07fff700 1 --2- 172.21.15.7:0/1093493596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f08157ca0 0x7f2f08155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:37.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.555+0000 7f2f0d09f700 1 --2- 172.21.15.7:0/1093493596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f08158070 0x7f2f081518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:37.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.555+0000 7f2f0c89e700 1 -- 172.21.15.7:0/1093493596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f08157ca0 msgr2=0x7f2f08155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:37.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.555+0000 7f2f0c89e700 1 --2- 172.21.15.7:0/1093493596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f08157ca0 0x7f2f08155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:37.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.555+0000 7f2f0c89e700 1 -- 172.21.15.7:0/1093493596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f08158070 msgr2=0x7f2f081518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:37.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.555+0000 7f2f0c89e700 1 --2- 172.21.15.7:0/1093493596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f08158070 0x7f2f081518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:37.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.555+0000 7f2f0c89e700 1 -- 172.21.15.7:0/1093493596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2efc0094f0 con 0x7f2f080c3ae0 2024-01-27T02:43:37.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.555+0000 7f2f0d09f700 1 --2- 172.21.15.7:0/1093493596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f08158070 0x7f2f081518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:37.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.556+0000 7f2f0c89e700 1 --2- 172.21.15.7:0/1093493596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f080c3ae0 0x7f2f0813f6f0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800d8c0 tx=0x7f2ef8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:37.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.556+0000 7f2f04ff9700 1 -- 172.21.15.7:0/1093493596 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80125f0 con 0x7f2f080c3ae0 2024-01-27T02:43:37.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.556+0000 7f2f0eb02700 1 -- 172.21.15.7:0/1093493596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f08156a20 con 0x7f2f080c3ae0 2024-01-27T02:43:37.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.556+0000 7f2f04ff9700 1 -- 172.21.15.7:0/1093493596 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef8012750 con 0x7f2f080c3ae0 2024-01-27T02:43:37.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.556+0000 7f2f04ff9700 1 -- 172.21.15.7:0/1093493596 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8012af0 con 0x7f2f080c3ae0 2024-01-27T02:43:37.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.556+0000 7f2f0eb02700 1 -- 172.21.15.7:0/1093493596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f0813fc30 con 0x7f2f080c3ae0 2024-01-27T02:43:37.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.557+0000 7f2f0eb02700 1 -- 172.21.15.7:0/1093493596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f0804c710 con 0x7f2f080c3ae0 2024-01-27T02:43:37.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.557+0000 7f2f04ff9700 1 -- 172.21.15.7:0/1093493596 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2ef801e460 con 0x7f2f080c3ae0 2024-01-27T02:43:37.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.558+0000 7f2f04ff9700 1 --2- 172.21.15.7:0/1093493596 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2ef4066f30 0x7f2ef40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:37.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.558+0000 7f2f04ff9700 1 -- 172.21.15.7:0/1093493596 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2ef8084b50 con 0x7f2f080c3ae0 2024-01-27T02:43:37.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.558+0000 7f2f07fff700 1 --2- 172.21.15.7:0/1093493596 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2ef4066f30 0x7f2ef40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:37.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.558+0000 7f2f07fff700 1 --2- 172.21.15.7:0/1093493596 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2ef4066f30 0x7f2ef40693e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0099b0 tx=0x7f2efc017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:37.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.562+0000 7f2f04ff9700 1 -- 172.21.15.7:0/1093493596 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ef8051dc0 con 0x7f2f080c3ae0 2024-01-27T02:43:37.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.814+0000 7f2f0eb02700 1 -- 172.21.15.7:0/1093493596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2f080c57d0 con 0x7f2f080c3ae0 2024-01-27T02:43:37.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.814+0000 7f2f04ff9700 1 -- 172.21.15.7:0/1093493596 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v94) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2ef80553e0 con 0x7f2f080c3ae0 2024-01-27T02:43:37.815 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-27T02:43:37.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.816+0000 7f2f0eb02700 1 -- 172.21.15.7:0/1093493596 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2ef4066f30 msgr2=0x7f2ef40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:37.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.816+0000 7f2f0eb02700 1 --2- 172.21.15.7:0/1093493596 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2ef4066f30 0x7f2ef40693e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0099b0 tx=0x7f2efc017000 comp rx=0 tx=0).stop 2024-01-27T02:43:37.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.817+0000 7f2f0eb02700 1 -- 172.21.15.7:0/1093493596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f080c3ae0 msgr2=0x7f2f0813f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:37.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.817+0000 7f2f0eb02700 1 --2- 172.21.15.7:0/1093493596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f080c3ae0 0x7f2f0813f6f0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800d8c0 tx=0x7f2ef8008040 comp rx=0 tx=0).stop 2024-01-27T02:43:37.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.817+0000 7f2f0eb02700 1 -- 172.21.15.7:0/1093493596 shutdown_connections 2024-01-27T02:43:37.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.817+0000 7f2f0eb02700 1 --2- 172.21.15.7:0/1093493596 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2ef4066f30 0x7f2ef40693e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:37.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.817+0000 7f2f0eb02700 1 --2- 172.21.15.7:0/1093493596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f08158070 0x7f2f081518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:37.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.817+0000 7f2f0eb02700 1 --2- 172.21.15.7:0/1093493596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f080c3ae0 0x7f2f0813f6f0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:37.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.817+0000 7f2f0eb02700 1 --2- 172.21.15.7:0/1093493596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f08157ca0 0x7f2f08155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:37.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.817+0000 7f2f0eb02700 1 -- 172.21.15.7:0/1093493596 >> 172.21.15.7:0/1093493596 conn(0x7f2f08098020 msgr2=0x7f2f080cd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:37.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.817+0000 7f2f0eb02700 1 -- 172.21.15.7:0/1093493596 shutdown_connections 2024-01-27T02:43:37.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:37.817+0000 7f2f0eb02700 1 -- 172.21.15.7:0/1093493596 wait complete. 2024-01-27T02:43:37.831 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-27T02:43:37.831 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-27T02:43:37.831 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-01-27T02:43:37.831 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-27T02:43:37.831 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-27T02:43:37.831 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-27T02:43:37.831 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-27T02:43:37.831 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-27T02:43:37.831 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:43:37.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:43:37.832 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-01-27T02:43:37.853 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-27T02:43:37.853 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-27T02:43:37.853 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-01-27T02:43:37.853 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:43:37.854 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-01-27T02:43:37.854 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:43:37.854 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:43:37.854 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:43:37.854 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-01-27T02:43:37.854 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:43:37.854 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:43:37.854 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-01-27T02:43:37.854 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-27T02:43:37.854 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:43:37.854 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-27T02:43:37.854 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-01-27T02:43:37.854 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-27T02:43:37.854 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 41701sec preferred_lft 41701sec 2024-01-27T02:43:37.854 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b1:d6f7:baf0:bc17/64 scope link noprefixroute 2024-01-27T02:43:37.855 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:43:37.855 INFO:teuthology.orchestra.run.smithi188.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-27T02:43:37.855 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b0:92:71:90:9d brd ff:ff:ff:ff:ff:ff 2024-01-27T02:43:37.855 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-27T02:43:37.855 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:43:37.855 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::10f8:32ff:feed:a752/64 scope link 2024-01-27T02:43:37.855 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:43:37.855 INFO:teuthology.orchestra.run.smithi188.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:43:37.855 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ba:b0:92:71:90:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-27T02:43:37.855 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b8b0:92ff:fe71:909d/64 scope link 2024-01-27T02:43:37.856 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:43:37.856 INFO:teuthology.orchestra.run.smithi188.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-27T02:43:37.856 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether ee:78:03:20:3a:a9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-27T02:43:37.856 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec78:3ff:fe20:3aa9/64 scope link 2024-01-27T02:43:37.856 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-27T02:43:37.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:43:37.856 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-27T02:43:37.925 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-01-27T02:43:37.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:43:37.926 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-01-27T02:43:37.987 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-01-27T02:43:37.988 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-27T02:43:37.988 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-27T02:43:37.988 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-27T02:43:37.988 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-27T02:43:37.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:43:37.988 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-27T02:43:37.988 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-27T02:43:37.988 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-27T02:43:37.989 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-27T02:43:37.989 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-27T02:43:37.989 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-27T02:43:38.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:43:38.100 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-01-27T02:43:38.118 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-27T02:43:38.118 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-27T02:43:38.119 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:43:38.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:43:38.119 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2024-01-27T02:43:38.186 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:43:38.186 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-27T02:43:38.243 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-27T02:43:38.243 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-27T02:43:38.243 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-27T02:43:38.299 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-27T02:43:38.300 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-27T02:43:38.300 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:43:38.382 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:43:38.383 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:43:38.383 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:43:38.391 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:43:38.392 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:43:38.392 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:43:38.392 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:43:38.392 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:43:38.409 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-27T02:43:38.409 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-27T02:43:38.410 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-01-27T02:43:38.410 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-01-27T02:43:38.410 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-27T02:43:38.410 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-01-27T02:43:38.410 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-01-27T02:43:38.410 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=17e1bcca-e647-4a58-b0ac-7d39d8508814" will pass to kernel 2024-01-27T02:43:38.412 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-27T02:43:38.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:43:38.412 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-27T02:43:38.479 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:43:38.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.618+0000 7fd1b7dff700 1 Processor -- start 2024-01-27T02:43:38.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.619+0000 7fd1b7dff700 1 -- start start 2024-01-27T02:43:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.620+0000 7fd1b7dff700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd1b01438f0 0x7fd1b00c8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.620+0000 7fd1b7dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1b00cb7a0 con 0x7fd1b0063c90 2024-01-27T02:43:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.620+0000 7fd1b7dff700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1b00cb9f0 con 0x7fd1b0143520 2024-01-27T02:43:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.620+0000 7fd1b7dff700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1b00cbcd0 con 0x7fd1b01438f0 2024-01-27T02:43:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.620+0000 7fd1b539a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd1b0063c90 0x7fd1b00c8210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38532/0 (socket says 172.21.15.7:38532) 2024-01-27T02:43:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.620+0000 7fd1b539a700 1 -- 172.21.15.7:0/2189821940 learned_addr learned my addr 172.21.15.7:0/2189821940 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.620+0000 7fd1b639c700 1 --2- 172.21.15.7:0/2189821940 >> v2:172.21.15.188:3300/0 conn(0x7fd1b01438f0 0x7fd1b00c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.620+0000 7fd1b4b99700 1 -- 172.21.15.7:0/2189821940 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1674271359 0 0) 0x7fd1b00cb7a0 con 0x7fd1b0063c90 2024-01-27T02:43:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.620+0000 7fd1b4b99700 1 -- 172.21.15.7:0/2189821940 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd19c003570 con 0x7fd1b0063c90 2024-01-27T02:43:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.620+0000 7fd1b4b99700 1 -- 172.21.15.7:0/2189821940 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2178268928 0 0) 0x7fd1b00cb9f0 con 0x7fd1b0143520 2024-01-27T02:43:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.620+0000 7fd1b4b99700 1 -- 172.21.15.7:0/2189821940 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1b00cb7a0 con 0x7fd1b0143520 2024-01-27T02:43:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.620+0000 7fd1b639c700 1 -- 172.21.15.7:0/2189821940 >> v1:172.21.15.188:6789/0 conn(0x7fd1b0143520 legacy=0x7fd1b0063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.620+0000 7fd1b639c700 1 -- 172.21.15.7:0/2189821940 >> v1:172.21.15.7:6789/0 conn(0x7fd1b0063c90 legacy=0x7fd1b00c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.620+0000 7fd1b639c700 1 -- 172.21.15.7:0/2189821940 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b00cbfb0 con 0x7fd1b01438f0 2024-01-27T02:43:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.621+0000 7fd1b639c700 1 --2- 172.21.15.7:0/2189821940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1b01438f0 0x7fd1b00c8920 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009e60 tx=0x7fd1a8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a91b034bb486b3a server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.621+0000 7fd1b4b99700 1 -- 172.21.15.7:0/2189821940 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a800e070 con 0x7fd1b01438f0 2024-01-27T02:43:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.621+0000 7fd1b4b99700 1 -- 172.21.15.7:0/2189821940 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a800bab0 con 0x7fd1b01438f0 2024-01-27T02:43:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.621+0000 7fd1b4b99700 1 -- 172.21.15.7:0/2189821940 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a800be50 con 0x7fd1b01438f0 2024-01-27T02:43:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.621+0000 7fd1b7dff700 1 -- 172.21.15.7:0/2189821940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1b01438f0 msgr2=0x7fd1b00c8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.621+0000 7fd1b7dff700 1 --2- 172.21.15.7:0/2189821940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1b01438f0 0x7fd1b00c8920 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009e60 tx=0x7fd1a8009780 comp rx=0 tx=0).stop 2024-01-27T02:43:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.621+0000 7fd1b7dff700 1 -- 172.21.15.7:0/2189821940 shutdown_connections 2024-01-27T02:43:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.621+0000 7fd1b7dff700 1 --2- 172.21.15.7:0/2189821940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1b01438f0 0x7fd1b00c8920 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.621+0000 7fd1b7dff700 1 -- 172.21.15.7:0/2189821940 >> 172.21.15.7:0/2189821940 conn(0x7fd1b00cdbd0 msgr2=0x7fd1b00cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.621+0000 7fd1b7dff700 1 -- 172.21.15.7:0/2189821940 shutdown_connections 2024-01-27T02:43:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.622+0000 7fd1b7dff700 1 -- 172.21.15.7:0/2189821940 wait complete. 2024-01-27T02:43:38.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.623+0000 7fd1b7dff700 1 Processor -- start 2024-01-27T02:43:38.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.623+0000 7fd1b7dff700 1 -- start start 2024-01-27T02:43:38.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.623+0000 7fd1b7dff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1b0063c90 0x7fd1b015f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:38.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.623+0000 7fd1b7dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b0143520 0x7fd1b00ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:38.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.623+0000 7fd1b7dff700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1b01438f0 0x7fd1b013e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:38.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.623+0000 7fd1b7dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b00b09b0 con 0x7fd1b0143520 2024-01-27T02:43:38.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.623+0000 7fd1b7dff700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b00b1250 con 0x7fd1b0063c90 2024-01-27T02:43:38.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.623+0000 7fd1b7dff700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b00af360 con 0x7fd1b01438f0 2024-01-27T02:43:38.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.624+0000 7fd1b539a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b0143520 0x7fd1b00ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:38.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.624+0000 7fd1b5b9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1b0063c90 0x7fd1b015f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:38.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.624+0000 7fd1b539a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b0143520 0x7fd1b00ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34918/0 (socket says 172.21.15.7:34918) 2024-01-27T02:43:38.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.624+0000 7fd1b539a700 1 -- 172.21.15.7:0/2029325262 learned_addr learned my addr 172.21.15.7:0/2029325262 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:38.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.624+0000 7fd1b639c700 1 --2- 172.21.15.7:0/2029325262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1b01438f0 0x7fd1b013e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:38.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.624+0000 7fd1b5b9b700 1 -- 172.21.15.7:0/2029325262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1b01438f0 msgr2=0x7fd1b013e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:38.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.624+0000 7fd1b5b9b700 1 --2- 172.21.15.7:0/2029325262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1b01438f0 0x7fd1b013e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:38.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.624+0000 7fd1b5b9b700 1 -- 172.21.15.7:0/2029325262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b0143520 msgr2=0x7fd1b00ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:38.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.624+0000 7fd1b5b9b700 1 --2- 172.21.15.7:0/2029325262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b0143520 0x7fd1b00ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:38.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.624+0000 7fd1b5b9b700 1 -- 172.21.15.7:0/2029325262 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a8009a20 con 0x7fd1b0063c90 2024-01-27T02:43:38.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.624+0000 7fd1b539a700 1 --2- 172.21.15.7:0/2029325262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b0143520 0x7fd1b00ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:43:38.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.624+0000 7fd1b5b9b700 1 --2- 172.21.15.7:0/2029325262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1b0063c90 0x7fd1b015f570 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00c590 tx=0x7fd1ac007d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:38.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.624+0000 7fd1a5ffb700 1 -- 172.21.15.7:0/2029325262 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac011590 con 0x7fd1b0063c90 2024-01-27T02:43:38.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.624+0000 7fd1a5ffb700 1 -- 172.21.15.7:0/2029325262 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ac0116f0 con 0x7fd1b0063c90 2024-01-27T02:43:38.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.624+0000 7fd1b7dff700 1 -- 172.21.15.7:0/2029325262 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b00b0c00 con 0x7fd1b0063c90 2024-01-27T02:43:38.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.624+0000 7fd1a5ffb700 1 -- 172.21.15.7:0/2029325262 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac011ab0 con 0x7fd1b0063c90 2024-01-27T02:43:38.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.625+0000 7fd1b7dff700 1 -- 172.21.15.7:0/2029325262 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b0142000 con 0x7fd1b0063c90 2024-01-27T02:43:38.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.626+0000 7fd1b7dff700 1 -- 172.21.15.7:0/2029325262 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b004c710 con 0x7fd1b0063c90 2024-01-27T02:43:38.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.626+0000 7fd1a5ffb700 1 -- 172.21.15.7:0/2029325262 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd1ac012420 con 0x7fd1b0063c90 2024-01-27T02:43:38.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.626+0000 7fd1a5ffb700 1 --2- 172.21.15.7:0/2029325262 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd194066f30 0x7fd1940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:38.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.627+0000 7fd1a5ffb700 1 -- 172.21.15.7:0/2029325262 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd1ac083e40 con 0x7fd1b0063c90 2024-01-27T02:43:38.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.627+0000 7fd1b539a700 1 --2- 172.21.15.7:0/2029325262 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd194066f30 0x7fd1940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:38.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.627+0000 7fd1b539a700 1 --2- 172.21.15.7:0/2029325262 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd194066f30 0x7fd1940693e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0141200 tx=0x7fd1a0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:38.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.631+0000 7fd1a5ffb700 1 -- 172.21.15.7:0/2029325262 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd1ac0510b0 con 0x7fd1b0063c90 2024-01-27T02:43:38.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.879+0000 7fd1b7dff700 1 -- 172.21.15.7:0/2029325262 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1b00cc290 con 0x7fd1b0063c90 2024-01-27T02:43:38.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.879+0000 7fd1a5ffb700 1 -- 172.21.15.7:0/2029325262 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4595 (secure 0 0 0) 0x7fd1ac0546d0 con 0x7fd1b0063c90 2024-01-27T02:43:38.881 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:38.881 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6704,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/1199331492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1199331492},{"type":"v1","addr":"172.21.15.7:6834","nonce":1199331492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":6707,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/721335086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":721335086},{"type":"v1","addr":"172.21.15.7:6835","nonce":721335086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":7699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/450555206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":450555206},{"type":"v1","addr":"172.21.15.188:6836","nonce":450555206}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:43:29.599748+0000","modified":"2024-01-27T02:43:33.196288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7702":{"gid":7702,"name":"b","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/3853795510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3853795510},{"type":"v1","addr":"172.21.15.188:6837","nonce":3853795510}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-27T02:43:38.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.882+0000 7fd1b7dff700 1 -- 172.21.15.7:0/2029325262 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd194066f30 msgr2=0x7fd1940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:38.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.882+0000 7fd1b7dff700 1 --2- 172.21.15.7:0/2029325262 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd194066f30 0x7fd1940693e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0141200 tx=0x7fd1a0009040 comp rx=0 tx=0).stop 2024-01-27T02:43:38.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.882+0000 7fd1b7dff700 1 -- 172.21.15.7:0/2029325262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1b0063c90 msgr2=0x7fd1b015f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:38.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.882+0000 7fd1b7dff700 1 --2- 172.21.15.7:0/2029325262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1b0063c90 0x7fd1b015f570 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00c590 tx=0x7fd1ac007d40 comp rx=0 tx=0).stop 2024-01-27T02:43:38.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.882+0000 7fd1b7dff700 1 -- 172.21.15.7:0/2029325262 shutdown_connections 2024-01-27T02:43:38.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.882+0000 7fd1b7dff700 1 --2- 172.21.15.7:0/2029325262 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fd194066f30 0x7fd1940693e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:38.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.882+0000 7fd1b7dff700 1 --2- 172.21.15.7:0/2029325262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1b01438f0 0x7fd1b013e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:38.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.882+0000 7fd1b7dff700 1 --2- 172.21.15.7:0/2029325262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1b0063c90 0x7fd1b015f570 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:38.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.882+0000 7fd1b7dff700 1 --2- 172.21.15.7:0/2029325262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b0143520 0x7fd1b00ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:38.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.882+0000 7fd1b7dff700 1 -- 172.21.15.7:0/2029325262 >> 172.21.15.7:0/2029325262 conn(0x7fd1b00cdbd0 msgr2=0x7fd1b00aa930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:38.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.882+0000 7fd1b7dff700 1 -- 172.21.15.7:0/2029325262 shutdown_connections 2024-01-27T02:43:38.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:38.882+0000 7fd1b7dff700 1 -- 172.21.15.7:0/2029325262 wait complete. 2024-01-27T02:43:38.885 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 94 2024-01-27T02:43:38.896 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2024-01-27T02:43:39.032 INFO:teuthology.orchestra.run.smithi188.stdout:{"cluster_fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","whoami":0,"id":7702,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.8398974339999992,"mdsmap_epoch":94,"osdmap_epoch":180,"osdmap_epoch_barrier":180,"uptime":14.81441341} 2024-01-27T02:43:39.033 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", "fs_name": "cephfs", "id": 7702, "mdsmap_epoch": 94, "osdmap_epoch": 180, "osdmap_epoch_barrier": 180, "rank_uptime": 8.839897434, "state": "up:active", "uptime": 14.81441341, "want_state": "up:active", "whoami": 0 } 2024-01-27T02:43:39.033 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-01-27T02:43:39.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.173+0000 7f28cfee5700 1 Processor -- start 2024-01-27T02:43:39.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.173+0000 7f28cfee5700 1 -- start start 2024-01-27T02:43:39.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.174+0000 7f28cfee5700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f28c80b1b70 0x7f28c806e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:39.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.174+0000 7f28cfee5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28c815a960 0x7f28c81567b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:39.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.174+0000 7f28cfee5700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c806ea80 con 0x7f28c80b1f40 2024-01-27T02:43:39.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.174+0000 7f28cfee5700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c806ed20 con 0x7f28c80b1b70 2024-01-27T02:43:39.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.174+0000 7f28cfee5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c806f000 con 0x7f28c815a960 2024-01-27T02:43:39.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.175+0000 7f28cdc81700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f28c80b1b70 0x7f28c806e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:39.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.175+0000 7f28cd480700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28c815a960 0x7f28c81567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:39.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.175+0000 7f28ce482700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f28c80b1f40 0x7f28c806a3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:58360/0 (socket says 172.21.15.7:58360) 2024-01-27T02:43:39.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.175+0000 7f28ce482700 1 -- 172.21.15.7:0/3345242732 learned_addr learned my addr 172.21.15.7:0/3345242732 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:39.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.175+0000 7f28ccc7f700 1 -- 172.21.15.7:0/3345242732 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2930298691 0 0) 0x7f28c806ea80 con 0x7f28c80b1f40 2024-01-27T02:43:39.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.175+0000 7f28ccc7f700 1 -- 172.21.15.7:0/3345242732 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28b4003570 con 0x7f28c80b1f40 2024-01-27T02:43:39.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.175+0000 7f28cdc81700 1 -- 172.21.15.7:0/3345242732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28c815a960 msgr2=0x7f28c81567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:39.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.175+0000 7f28cdc81700 1 --2- 172.21.15.7:0/3345242732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28c815a960 0x7f28c81567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:39.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.175+0000 7f28cdc81700 1 -- 172.21.15.7:0/3345242732 >> v1:172.21.15.188:6789/0 conn(0x7f28c80b1f40 legacy=0x7f28c806a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:39.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.175+0000 7f28cdc81700 1 -- 172.21.15.7:0/3345242732 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c806f2e0 con 0x7f28c80b1b70 2024-01-27T02:43:39.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.175+0000 7f28cd480700 1 --2- 172.21.15.7:0/3345242732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28c815a960 0x7f28c81567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:43:39.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.175+0000 7f28cdc81700 1 --2- 172.21.15.7:0/3345242732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28c80b1b70 0x7f28c806e540 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0098e0 tx=0x7f28bc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1477fe8fd1f39c46 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:39.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.175+0000 7f28ccc7f700 1 -- 172.21.15.7:0/3345242732 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc011630 con 0x7f28c80b1b70 2024-01-27T02:43:39.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.175+0000 7f28ccc7f700 1 -- 172.21.15.7:0/3345242732 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc011790 con 0x7f28c80b1b70 2024-01-27T02:43:39.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.175+0000 7f28ccc7f700 1 -- 172.21.15.7:0/3345242732 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc011b30 con 0x7f28c80b1b70 2024-01-27T02:43:39.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.176+0000 7f28cfee5700 1 -- 172.21.15.7:0/3345242732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28c80b1b70 msgr2=0x7f28c806e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:39.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.176+0000 7f28cfee5700 1 --2- 172.21.15.7:0/3345242732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28c80b1b70 0x7f28c806e540 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0098e0 tx=0x7f28bc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:39.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.176+0000 7f28cfee5700 1 -- 172.21.15.7:0/3345242732 shutdown_connections 2024-01-27T02:43:39.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.176+0000 7f28cfee5700 1 --2- 172.21.15.7:0/3345242732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28c80b1b70 0x7f28c806e540 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:39.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.176+0000 7f28cfee5700 1 --2- 172.21.15.7:0/3345242732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28c815a960 0x7f28c81567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:39.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.176+0000 7f28cfee5700 1 -- 172.21.15.7:0/3345242732 >> 172.21.15.7:0/3345242732 conn(0x7f28c8068480 msgr2=0x7f28c806be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:39.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.176+0000 7f28cfee5700 1 -- 172.21.15.7:0/3345242732 shutdown_connections 2024-01-27T02:43:39.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.176+0000 7f28cfee5700 1 -- 172.21.15.7:0/3345242732 wait complete. 2024-01-27T02:43:39.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.177+0000 7f28cfee5700 1 Processor -- start 2024-01-27T02:43:39.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.177+0000 7f28cfee5700 1 -- start start 2024-01-27T02:43:39.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.178+0000 7f28cfee5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28c80b1b70 0x7f28c8168850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.178+0000 7f28cfee5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28c80b1f40 0x7f28c8165a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.178+0000 7f28cfee5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28c815a960 0x7f28c813b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.178+0000 7f28cfee5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c8168d90 con 0x7f28c80b1b70 2024-01-27T02:43:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.178+0000 7f28cfee5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c8169560 con 0x7f28c80b1f40 2024-01-27T02:43:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.178+0000 7f28cfee5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c8169840 con 0x7f28c815a960 2024-01-27T02:43:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.178+0000 7f28cdc81700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28c80b1b70 0x7f28c8168850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.178+0000 7f28cdc81700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28c80b1b70 0x7f28c8168850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34926/0 (socket says 172.21.15.7:34926) 2024-01-27T02:43:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.178+0000 7f28cdc81700 1 -- 172.21.15.7:0/1621304376 learned_addr learned my addr 172.21.15.7:0/1621304376 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.178+0000 7f28cd480700 1 --2- 172.21.15.7:0/1621304376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28c80b1f40 0x7f28c8165a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.178+0000 7f28ce482700 1 --2- 172.21.15.7:0/1621304376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28c815a960 0x7f28c813b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.179+0000 7f28cdc81700 1 -- 172.21.15.7:0/1621304376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28c815a960 msgr2=0x7f28c813b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.179+0000 7f28cdc81700 1 --2- 172.21.15.7:0/1621304376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28c815a960 0x7f28c813b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.179+0000 7f28cdc81700 1 -- 172.21.15.7:0/1621304376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28c80b1f40 msgr2=0x7f28c8165a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.179+0000 7f28cdc81700 1 --2- 172.21.15.7:0/1621304376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28c80b1f40 0x7f28c8165a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.179+0000 7f28cdc81700 1 -- 172.21.15.7:0/1621304376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c4009420 con 0x7f28c80b1b70 2024-01-27T02:43:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.179+0000 7f28cdc81700 1 --2- 172.21.15.7:0/1621304376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28c80b1b70 0x7f28c8168850 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f28c806dfe0 tx=0x7f28bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.179+0000 7f28b9ffb700 1 -- 172.21.15.7:0/1621304376 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc000d10 con 0x7f28c80b1b70 2024-01-27T02:43:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.179+0000 7f28b9ffb700 1 -- 172.21.15.7:0/1621304376 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc011b10 con 0x7f28c80b1b70 2024-01-27T02:43:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.179+0000 7f28b9ffb700 1 -- 172.21.15.7:0/1621304376 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc025660 con 0x7f28c80b1b70 2024-01-27T02:43:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.179+0000 7f28cfee5700 1 -- 172.21.15.7:0/1621304376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28bc009420 con 0x7f28c80b1b70 2024-01-27T02:43:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.179+0000 7f28cfee5700 1 -- 172.21.15.7:0/1621304376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c8169e00 con 0x7f28c80b1b70 2024-01-27T02:43:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.180+0000 7f28cfee5700 1 -- 172.21.15.7:0/1621304376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c804c710 con 0x7f28c80b1b70 2024-01-27T02:43:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.181+0000 7f28b9ffb700 1 -- 172.21.15.7:0/1621304376 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f28bc025a50 con 0x7f28c80b1b70 2024-01-27T02:43:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.181+0000 7f28b9ffb700 1 --2- 172.21.15.7:0/1621304376 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f28ac066f30 0x7f28ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.181+0000 7f28b9ffb700 1 -- 172.21.15.7:0/1621304376 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f28bc085d30 con 0x7f28c80b1b70 2024-01-27T02:43:39.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.181+0000 7f28cd480700 1 --2- 172.21.15.7:0/1621304376 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f28ac066f30 0x7f28ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:39.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.182+0000 7f28cd480700 1 --2- 172.21.15.7:0/1621304376 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f28ac066f30 0x7f28ac0693e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f28c815c4f0 tx=0x7f28c40091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:39.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.186+0000 7f28b9ffb700 1 -- 172.21.15.7:0/1621304376 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28bc052f00 con 0x7f28c80b1b70 2024-01-27T02:43:39.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.389+0000 7f28cfee5700 1 -- 172.21.15.7:0/1621304376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f28c806cb40 con 0x7f28c80b1b70 2024-01-27T02:43:39.555 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:43:39.558 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:43:39.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.950+0000 7f28b9ffb700 1 -- 172.21.15.7:0/1621304376 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v181) v1 ==== 181+0+0 (secure 0 0 0) 0x7f28bc056520 con 0x7f28c80b1b70 2024-01-27T02:43:39.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.953+0000 7f28cfee5700 1 -- 172.21.15.7:0/1621304376 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f28ac066f30 msgr2=0x7f28ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:39.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.953+0000 7f28cfee5700 1 --2- 172.21.15.7:0/1621304376 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f28ac066f30 0x7f28ac0693e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f28c815c4f0 tx=0x7f28c40091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:39.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.953+0000 7f28cfee5700 1 -- 172.21.15.7:0/1621304376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28c80b1b70 msgr2=0x7f28c8168850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:39.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.953+0000 7f28cfee5700 1 --2- 172.21.15.7:0/1621304376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28c80b1b70 0x7f28c8168850 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f28c806dfe0 tx=0x7f28bc017000 comp rx=0 tx=0).stop 2024-01-27T02:43:39.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.953+0000 7f28cfee5700 1 -- 172.21.15.7:0/1621304376 shutdown_connections 2024-01-27T02:43:39.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.953+0000 7f28cfee5700 1 --2- 172.21.15.7:0/1621304376 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f28ac066f30 0x7f28ac0693e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:39.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.953+0000 7f28cfee5700 1 --2- 172.21.15.7:0/1621304376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28c815a960 0x7f28c813b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:39.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.953+0000 7f28cfee5700 1 --2- 172.21.15.7:0/1621304376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28c80b1b70 0x7f28c8168850 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:39.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.953+0000 7f28cfee5700 1 --2- 172.21.15.7:0/1621304376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28c80b1f40 0x7f28c8165a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:39.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.953+0000 7f28cfee5700 1 -- 172.21.15.7:0/1621304376 >> 172.21.15.7:0/1621304376 conn(0x7f28c8068480 msgr2=0x7f28c80a7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:39.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.953+0000 7f28cfee5700 1 -- 172.21.15.7:0/1621304376 shutdown_connections 2024-01-27T02:43:39.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:39.953+0000 7f28cfee5700 1 -- 172.21.15.7:0/1621304376 wait complete. 2024-01-27T02:43:39.955 INFO:teuthology.orchestra.run.smithi007.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-01-27T02:43:39.969 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-01-27T02:43:40.122 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_mon_report_interval":"5"} 2024-01-27T02:43:40.123 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-01-27T02:43:40.123 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2024-01-27T02:43:40.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:43:40.123 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2024-01-27T02:43:40.253 INFO:teuthology.orchestra.run.smithi007.stderr:16+0 records in 2024-01-27T02:43:40.253 INFO:teuthology.orchestra.run.smithi007.stderr:16+0 records out 2024-01-27T02:43:40.253 INFO:teuthology.orchestra.run.smithi007.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.106941 s, 157 MB/s 2024-01-27T02:43:40.253 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:43:40.253 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2024-01-27T02:43:40.376 INFO:teuthology.orchestra.run.smithi007.stderr:17+0 records in 2024-01-27T02:43:40.376 INFO:teuthology.orchestra.run.smithi007.stderr:17+0 records out 2024-01-27T02:43:40.376 INFO:teuthology.orchestra.run.smithi007.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.100915 s, 177 MB/s 2024-01-27T02:43:40.376 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-01-27T02:43:40.376 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:43:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.507+0000 7f9a60f28700 1 Processor -- start 2024-01-27T02:43:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.508+0000 7f9a60f28700 1 -- start start 2024-01-27T02:43:40.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.509+0000 7f9a60f28700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a5c1560b0 0x7f9a5c159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:40.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.509+0000 7f9a60f28700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9a5c0c75c0 0x7f9a5c0c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:40.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.509+0000 7f9a60f28700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9a5c06b240 0x7f9a5c0af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:40.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.509+0000 7f9a60f28700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a5c0c9260 con 0x7f9a5c06b240 2024-01-27T02:43:40.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.509+0000 7f9a60f28700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a5c15a5b0 con 0x7f9a5c0c75c0 2024-01-27T02:43:40.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.509+0000 7f9a60f28700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c15a890 con 0x7f9a5c1560b0 2024-01-27T02:43:40.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.509+0000 7f9a5ad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9a5c06b240 0x7f9a5c0af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.509+0000 7f9a5ad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9a5c06b240 0x7f9a5c0af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34930/0 (socket says 172.21.15.7:34930) 2024-01-27T02:43:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.509+0000 7f9a5ad9d700 1 -- 172.21.15.7:0/2847577687 learned_addr learned my addr 172.21.15.7:0/2847577687 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.509+0000 7f9a59d9b700 1 --2- 172.21.15.7:0/2847577687 >> v2:172.21.15.188:3300/0 conn(0x7f9a5c0c75c0 0x7f9a5c0c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.509+0000 7f9a5a59c700 1 --2- 172.21.15.7:0/2847577687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a5c1560b0 0x7f9a5c159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.509+0000 7f9a5ad9d700 1 -- 172.21.15.7:0/2847577687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a5c1560b0 msgr2=0x7f9a5c159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.509+0000 7f9a5ad9d700 1 --2- 172.21.15.7:0/2847577687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a5c1560b0 0x7f9a5c159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.509+0000 7f9a5ad9d700 1 -- 172.21.15.7:0/2847577687 >> v2:172.21.15.188:3300/0 conn(0x7f9a5c0c75c0 msgr2=0x7f9a5c0c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.509+0000 7f9a5ad9d700 1 --2- 172.21.15.7:0/2847577687 >> v2:172.21.15.188:3300/0 conn(0x7f9a5c0c75c0 0x7f9a5c0c96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.509+0000 7f9a5ad9d700 1 -- 172.21.15.7:0/2847577687 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5c0c8ef0 con 0x7f9a5c06b240 2024-01-27T02:43:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.509+0000 7f9a5ad9d700 1 --2- 172.21.15.7:0/2847577687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a5c06b240 0x7f9a5c0af8b0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c06c210 tx=0x7f9a5400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c794c21b50574a03 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.510+0000 7f9a5959a700 1 -- 172.21.15.7:0/2847577687 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a54010070 con 0x7f9a5c06b240 2024-01-27T02:43:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.510+0000 7f9a5959a700 1 -- 172.21.15.7:0/2847577687 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5400dc80 con 0x7f9a5c06b240 2024-01-27T02:43:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.510+0000 7f9a5959a700 1 -- 172.21.15.7:0/2847577687 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a54016400 con 0x7f9a5c06b240 2024-01-27T02:43:40.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.510+0000 7f9a60f28700 1 -- 172.21.15.7:0/2847577687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a5c06b240 msgr2=0x7f9a5c0af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:40.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.510+0000 7f9a60f28700 1 --2- 172.21.15.7:0/2847577687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a5c06b240 0x7f9a5c0af8b0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c06c210 tx=0x7f9a5400ba50 comp rx=0 tx=0).stop 2024-01-27T02:43:40.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.510+0000 7f9a60f28700 1 -- 172.21.15.7:0/2847577687 shutdown_connections 2024-01-27T02:43:40.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.510+0000 7f9a60f28700 1 --2- 172.21.15.7:0/2847577687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a5c06b240 0x7f9a5c0af8b0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:40.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.510+0000 7f9a60f28700 1 --2- 172.21.15.7:0/2847577687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a5c1560b0 0x7f9a5c159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:40.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.510+0000 7f9a60f28700 1 --2- 172.21.15.7:0/2847577687 >> v2:172.21.15.188:3300/0 conn(0x7f9a5c0c75c0 0x7f9a5c0c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:40.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.510+0000 7f9a60f28700 1 -- 172.21.15.7:0/2847577687 >> 172.21.15.7:0/2847577687 conn(0x7f9a5c063580 msgr2=0x7f9a5c063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:40.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.510+0000 7f9a60f28700 1 -- 172.21.15.7:0/2847577687 shutdown_connections 2024-01-27T02:43:40.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.510+0000 7f9a60f28700 1 -- 172.21.15.7:0/2847577687 wait complete. 2024-01-27T02:43:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.512+0000 7f9a60f28700 1 Processor -- start 2024-01-27T02:43:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.512+0000 7f9a60f28700 1 -- start start 2024-01-27T02:43:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.512+0000 7f9a60f28700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a5c06b240 0x7f9a5c0c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.512+0000 7f9a60f28700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a5c0c75c0 0x7f9a5c16b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.512+0000 7f9a60f28700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a5c1560b0 0x7f9a5c1585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.512+0000 7f9a60f28700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c0c3dd0 con 0x7f9a5c1560b0 2024-01-27T02:43:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.512+0000 7f9a60f28700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c159c10 con 0x7f9a5c0c75c0 2024-01-27T02:43:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.512+0000 7f9a60f28700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c16b8a0 con 0x7f9a5c06b240 2024-01-27T02:43:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.513+0000 7f9a5ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a5c1560b0 0x7f9a5c1585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.513+0000 7f9a5ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a5c1560b0 0x7f9a5c1585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34940/0 (socket says 172.21.15.7:34940) 2024-01-27T02:43:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.513+0000 7f9a5ad9d700 1 -- 172.21.15.7:0/702103990 learned_addr learned my addr 172.21.15.7:0/702103990 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.513+0000 7f9a5a59c700 1 --2- 172.21.15.7:0/702103990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a5c06b240 0x7f9a5c0c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.513+0000 7f9a59d9b700 1 --2- 172.21.15.7:0/702103990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a5c0c75c0 0x7f9a5c16b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.513+0000 7f9a5ad9d700 1 -- 172.21.15.7:0/702103990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a5c06b240 msgr2=0x7f9a5c0c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.513+0000 7f9a5ad9d700 1 --2- 172.21.15.7:0/702103990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a5c06b240 0x7f9a5c0c2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.513+0000 7f9a5ad9d700 1 -- 172.21.15.7:0/702103990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a5c0c75c0 msgr2=0x7f9a5c16b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.513+0000 7f9a5ad9d700 1 --2- 172.21.15.7:0/702103990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a5c0c75c0 0x7f9a5c16b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.513+0000 7f9a5ad9d700 1 -- 172.21.15.7:0/702103990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5400bcf0 con 0x7f9a5c1560b0 2024-01-27T02:43:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.513+0000 7f9a5ad9d700 1 --2- 172.21.15.7:0/702103990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a5c1560b0 0x7f9a5c1585c0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c0ca1c0 tx=0x7f9a5401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.513+0000 7f9a4a7fc700 1 -- 172.21.15.7:0/702103990 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a54016400 con 0x7f9a5c1560b0 2024-01-27T02:43:40.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.513+0000 7f9a4a7fc700 1 -- 172.21.15.7:0/702103990 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a540032f0 con 0x7f9a5c1560b0 2024-01-27T02:43:40.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.513+0000 7f9a60f28700 1 -- 172.21.15.7:0/702103990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a5c158130 con 0x7f9a5c1560b0 2024-01-27T02:43:40.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.513+0000 7f9a4a7fc700 1 -- 172.21.15.7:0/702103990 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5401f660 con 0x7f9a5c1560b0 2024-01-27T02:43:40.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.513+0000 7f9a60f28700 1 -- 172.21.15.7:0/702103990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5c157b20 con 0x7f9a5c1560b0 2024-01-27T02:43:40.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.514+0000 7f9a60f28700 1 -- 172.21.15.7:0/702103990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a5c0c88c0 con 0x7f9a5c1560b0 2024-01-27T02:43:40.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.515+0000 7f9a4a7fc700 1 -- 172.21.15.7:0/702103990 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9a54025030 con 0x7f9a5c1560b0 2024-01-27T02:43:40.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.515+0000 7f9a4a7fc700 1 --2- 172.21.15.7:0/702103990 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9a3c066f30 0x7f9a3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:40.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.515+0000 7f9a4a7fc700 1 -- 172.21.15.7:0/702103990 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9a54023070 con 0x7f9a5c1560b0 2024-01-27T02:43:40.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.515+0000 7f9a5a59c700 1 --2- 172.21.15.7:0/702103990 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9a3c066f30 0x7f9a3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:40.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.516+0000 7f9a5a59c700 1 --2- 172.21.15.7:0/702103990 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9a3c066f30 0x7f9a3c0693e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c006fd0 tx=0x7f9a4c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:40.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.520+0000 7f9a4a7fc700 1 -- 172.21.15.7:0/702103990 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a54053600 con 0x7f9a5c1560b0 2024-01-27T02:43:40.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.723+0000 7f9a60f28700 1 -- 172.21.15.7:0/702103990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f9a5c0c80d0 con 0x7f9a5c1560b0 2024-01-27T02:43:40.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.724+0000 7f9a4a7fc700 1 -- 172.21.15.7:0/702103990 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v181) v1 ==== 81+0+32234 (secure 0 0 0) 0x7f9a54056c20 con 0x7f9a5c1560b0 2024-01-27T02:43:40.725 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:40.725 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:43:40.725 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 181, 2024-01-27T02:43:40.725 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:43:40.725 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:43:40.725 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:43:39.942398+0000", 2024-01-27T02:43:40.725 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:43:40.725 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:43:40.725 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:43:40.725 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:43:40.725 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:43:40.725 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:43:40.726 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:43:40.726 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:43:40.726 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:43:40.726 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:40.726 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:43:40.726 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:43:40.726 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:43:40.726 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:43:40.726 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:43:40.726 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 23, 2024-01-27T02:43:40.726 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:43:40.726 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:43:40.727 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:43:40.727 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:43:40.727 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:43:40.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.727 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:43:40.727 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:43:40.727 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:43:40.727 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:43:40.727 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:43:40.727 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:43:40.727 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:43:40.727 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:43:40.727 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:43:40.727 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:43:40.727 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:43:40.728 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:43:40.728 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:43:40.728 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:43:40.728 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:43:40.728 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:43:40.728 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:43:40.728 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:43:40.728 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:43:40.728 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:43:40.728 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:43:40.728 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:43:40.728 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:43:40.728 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:43:40.728 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:43:40.729 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:43:40.729 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:43:40.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.729 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "106", 2024-01-27T02:43:40.729 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:43:40.729 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:43:40.729 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:43:40.729 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:43:40.729 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:43:40.729 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:43:40.729 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:43:40.729 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:43:40.729 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:43:40.729 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:43:40.729 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:43:40.730 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:43:40.730 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:43:40.730 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:43:40.730 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:43:40.730 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:43:40.730 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:43:40.730 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:43:40.730 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:43:40.730 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:43:40.730 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:43:40.730 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:43:40.730 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:43:40.730 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:43:40.730 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:43:40.731 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:43:40.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.731 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:43:40.731 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:43:40.731 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:43:40.731 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:43:40.731 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:43:40.731 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:43:40.731 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:43:40.731 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:43:40.731 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:43:40.731 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:43:40.731 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:43:40.731 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:43:40.732 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:43:40.732 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:43:40.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.732 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:43:40.732 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:43:40.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.732 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 22, 2024-01-27T02:43:40.732 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:43:40.732 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:43:27.492968+0000", 2024-01-27T02:43:40.732 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:43:40.732 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:43:40.732 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:43:40.733 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:43:40.733 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:43:40.733 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:43:40.733 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:43:40.733 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:43:40.733 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:43:40.733 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:43:40.733 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:43:40.733 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:43:40.733 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:43:40.733 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:43:40.733 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:43:40.734 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:43:40.734 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:43:40.734 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:43:40.734 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:43:40.734 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:43:40.734 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:43:40.734 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:43:40.734 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:43:40.734 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:43:40.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.734 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "179", 2024-01-27T02:43:40.734 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:43:40.734 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:43:40.735 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:43:40.735 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:43:40.735 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:43:40.735 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:43:40.735 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:43:40.735 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:43:40.735 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:43:40.735 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:43:40.735 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:43:40.735 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:43:40.735 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:43:40.735 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:43:40.735 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:43:40.735 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:43:40.736 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:43:40.736 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:43:40.736 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:43:40.736 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:43:40.736 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:43:40.736 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:43:40.736 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:43:40.736 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:43:40.736 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:43:40.736 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:43:40.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.736 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:43:40.736 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:43:40.737 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:43:40.737 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:43:40.737 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:43:40.737 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:43:40.737 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:43:40.737 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:43:40.737 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:43:40.737 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:43:40.737 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:43:40.737 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:43:40.737 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:43:40.737 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:43:40.737 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:43:40.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.738 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:43:40.738 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:43:40.738 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:43:40.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.738 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 23, 2024-01-27T02:43:40.738 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:43:40.738 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:43:28.579977+0000", 2024-01-27T02:43:40.738 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:43:40.738 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:43:40.739 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:43:40.739 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:43:40.739 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:43:40.739 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:43:40.739 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:43:40.739 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:43:40.739 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:43:40.739 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:43:40.739 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:43:40.739 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:43:40.739 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:43:40.739 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:43:40.739 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:43:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:43:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:43:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:43:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:43:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:43:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:43:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:43:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:43:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:43:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "181", 2024-01-27T02:43:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:43:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:43:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:43:40.741 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:43:40.741 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:43:40.741 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:43:40.741 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:43:40.741 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:43:40.741 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:43:40.741 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-27T02:43:40.741 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:43:40.741 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:43:40.741 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:43:40.741 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:43:40.741 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:43:40.741 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:43:40.741 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:43:40.742 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:43:40.742 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:43:40.742 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:43:40.742 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:43:40.742 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:43:40.742 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:43:40.742 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:43:40.742 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:43:40.742 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:43:40.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.742 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:43:40.742 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:43:40.742 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:43:40.743 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:43:40.743 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:43:40.743 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:43:40.743 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:43:40.743 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:43:40.743 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:43:40.743 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:43:40.743 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:43:40.743 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:43:40.743 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:43:40.743 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:43:40.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.743 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:43:40.744 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:43:40.744 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:43:40.744 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.744 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.744 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.744 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:40.744 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:43:40.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.744 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:43:40.744 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:40.744 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:40.744 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:40.744 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:40.744 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:40.745 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:40.745 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:40.745 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:40.745 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:40.745 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:40.745 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:40.745 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:40.745 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.745 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.745 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:43:40.745 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:40.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.746 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.746 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:43:40.746 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:40.746 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.746 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.746 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:40.746 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.746 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.746 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:43:40.746 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:40.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.747 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.747 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:43:40.747 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:40.747 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.747 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.747 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:40.747 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.747 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.747 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:43:40.747 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:40.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.748 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.748 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:43:40.748 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:40.748 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.748 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.748 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:40.748 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.748 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.748 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:43:40.748 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:40.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.749 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:43:40.749 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:40.749 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.749 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.749 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:43:40.749 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:43:40.749 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:43:40.749 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:43:40.749 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:40.749 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:40.749 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:40.750 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.750 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:43:40.750 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:40.750 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:40.750 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:40.750 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:40.750 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:40.750 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:40.750 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:40.750 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:40.750 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:40.750 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:40.751 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:40.751 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:40.751 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.751 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.751 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:43:40.751 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:40.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.751 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.751 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:43:40.751 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:40.751 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.751 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.752 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:40.752 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.752 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.752 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:43:40.752 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:40.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.752 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.752 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:43:40.752 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:40.752 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.752 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.753 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:40.753 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.753 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.753 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:43:40.753 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:40.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.753 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.753 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:43:40.753 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:40.753 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:43:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:43:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:43:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:43:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:43:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:43:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:43:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:43:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:43:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:43:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:43:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:43:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:43:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:43:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:43:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:43:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:43:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:43:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:43:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:43:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:43:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:43:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:43:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:43:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:43:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:43:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:43:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:43:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:43:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:43:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:43:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:43:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:43:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:43:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:43:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:43:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:43:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:40.771 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.771 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.771 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:40.771 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.771 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:43:40.771 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:40.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.771 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:43:40.772 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:40.772 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.772 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.772 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:40.772 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.772 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.772 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:43:40.772 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:40.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.773 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.773 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:43:40.773 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:40.773 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.773 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.773 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:43:40.773 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:43:40.773 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:43:40.773 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:43:40.773 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:40.773 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:40.773 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:40.774 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.774 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:43:40.774 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:40.774 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:40.774 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:40.774 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:40.774 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:40.774 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:40.774 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:40.774 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:40.775 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:40.775 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:40.775 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:40.775 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:40.775 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.775 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.775 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:43:40.775 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:40.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.776 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.776 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:43:40.777 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:40.777 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.778 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.778 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:40.778 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.778 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.778 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:43:40.778 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:40.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.778 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.779 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:43:40.779 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:40.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.780 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.780 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:40.780 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.780 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.780 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:43:40.781 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:40.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.781 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.781 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:43:40.781 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:40.781 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.781 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.782 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:40.782 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.782 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.782 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:43:40.782 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:40.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.783 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.783 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:43:40.783 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:40.783 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.783 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.784 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:43:40.784 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:43:40.784 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:43:40.784 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:43:40.784 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:40.784 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:40.785 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:40.785 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.785 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:43:40.785 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:40.785 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:40.785 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:40.785 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:40.785 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:40.785 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:40.785 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:40.786 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:40.786 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:40.786 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:40.786 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:40.786 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:40.786 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.786 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.786 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:43:40.786 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:40.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.786 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.786 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:43:40.787 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:40.787 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.787 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.787 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:40.787 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.787 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:43:40.787 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:40.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.788 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:43:40.788 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:40.788 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.788 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.788 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:40.788 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.788 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.788 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:43:40.788 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:40.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.788 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.789 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:43:40.789 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:40.789 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.789 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.789 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:40.789 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.789 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.789 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:43:40.789 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:40.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.789 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.790 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:43:40.790 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:40.790 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.790 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.790 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:43:40.790 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:43:40.790 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:43:40.790 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:43:40.790 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:40.790 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:40.790 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:40.790 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.791 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:43:40.791 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:40.791 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:40.791 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:40.791 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:40.791 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:40.791 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:40.791 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:40.791 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:40.791 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:40.791 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:40.792 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:40.792 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:40.792 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.792 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.792 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:43:40.792 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:40.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.793 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.793 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:43:40.793 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:40.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.793 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.794 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:40.794 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.794 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:43:40.794 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:40.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.794 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:43:40.795 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:40.795 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.795 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.795 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:40.795 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.795 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.795 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:43:40.796 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:40.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.796 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.796 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:43:40.796 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:40.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.796 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.797 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:40.797 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:40.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:40.797 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:43:40.797 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:40.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:40.798 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:43:40.798 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:40.798 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.798 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.798 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:43:40.798 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:43:40.798 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:43:40.798 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:43:40.798 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:40.798 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:40.798 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:40.799 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:40.799 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.799 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:40.799 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:43:40.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.799 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:43:40.799 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:40.799 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:40.799 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:40.799 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:40.799 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:40.799 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:43:40.799 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:40.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.800 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:43:40.800 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:40.800 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:40.800 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:40.800 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:40.800 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:40.800 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:43:40.800 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:40.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.800 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:43:40.800 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:40.801 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:40.801 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:40.801 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:40.801 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:40.801 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:43:40.801 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:40.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.801 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:43:40.801 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:40.801 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:40.802 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:40.802 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:40.802 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:40.802 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:43:40.802 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:40.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.802 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:43:40.802 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:40.802 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:40.802 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:40.802 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:40.802 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:40.802 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:43:40.803 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:40.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.803 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:43:40.803 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:40.803 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:40.803 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:40.803 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:40.803 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:40.803 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:43:40.803 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:40.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.804 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:43:40.804 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:40.804 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:40.804 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:40.804 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:40.804 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:40.804 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:43:40.804 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:40.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:40.804 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:43:40.804 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:40.805 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:40.805 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:40.805 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:40.805 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:40.805 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:43:40.805 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:40.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.805 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:40.805 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:43:40.805 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:43:40.805 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:43:40.805 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:43:40.806 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:43:40.806 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:43:40.806 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:43:40.806 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:43:40.806 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:43:40.806 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:43:40.806 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:43:40.806 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:43:40.806 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:43:40.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:40.806 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:43:40.806 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:43:40.806 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:43:40.806 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:43:40.807 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:43:40.807 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:43:40.807 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:43:40.807 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:43:40.807 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:43:40.807 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:43:40.807 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:43:40.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:40.807 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:43:40.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.726+0000 7f9a60f28700 1 -- 172.21.15.7:0/702103990 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9a3c066f30 msgr2=0x7f9a3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:40.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.726+0000 7f9a60f28700 1 --2- 172.21.15.7:0/702103990 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9a3c066f30 0x7f9a3c0693e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c006fd0 tx=0x7f9a4c008040 comp rx=0 tx=0).stop 2024-01-27T02:43:40.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.726+0000 7f9a60f28700 1 -- 172.21.15.7:0/702103990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a5c1560b0 msgr2=0x7f9a5c1585c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:40.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.726+0000 7f9a60f28700 1 --2- 172.21.15.7:0/702103990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a5c1560b0 0x7f9a5c1585c0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c0ca1c0 tx=0x7f9a5401c000 comp rx=0 tx=0).stop 2024-01-27T02:43:40.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.726+0000 7f9a60f28700 1 -- 172.21.15.7:0/702103990 shutdown_connections 2024-01-27T02:43:40.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.726+0000 7f9a60f28700 1 --2- 172.21.15.7:0/702103990 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9a3c066f30 0x7f9a3c0693e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:40.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.726+0000 7f9a60f28700 1 --2- 172.21.15.7:0/702103990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a5c1560b0 0x7f9a5c1585c0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:40.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.726+0000 7f9a60f28700 1 --2- 172.21.15.7:0/702103990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a5c06b240 0x7f9a5c0c2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:40.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.726+0000 7f9a60f28700 1 --2- 172.21.15.7:0/702103990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a5c0c75c0 0x7f9a5c16b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:40.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.726+0000 7f9a60f28700 1 -- 172.21.15.7:0/702103990 >> 172.21.15.7:0/702103990 conn(0x7f9a5c063580 msgr2=0x7f9a5c0ca5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:40.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.726+0000 7f9a60f28700 1 -- 172.21.15.7:0/702103990 shutdown_connections 2024-01-27T02:43:40.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:40.726+0000 7f9a60f28700 1 -- 172.21.15.7:0/702103990 wait complete. 2024-01-27T02:43:40.810 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-27T02:43:45.812 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:43:45.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.953+0000 7f87e44a8700 1 Processor -- start 2024-01-27T02:43:45.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.953+0000 7f87e44a8700 1 -- start start 2024-01-27T02:43:45.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.955+0000 7f87e44a8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87dc0c3ae0 0x7f87dc0cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:45.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.955+0000 7f87e44a8700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f87dc158070 0x7f87dc0cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:45.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.955+0000 7f87e44a8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87dc0ca020 con 0x7f87dc157ca0 2024-01-27T02:43:45.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.955+0000 7f87e44a8700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87dc0c5210 con 0x7f87dc158070 2024-01-27T02:43:45.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.955+0000 7f87e44a8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0c54f0 con 0x7f87dc0c3ae0 2024-01-27T02:43:45.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.955+0000 7f87e2244700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f87dc157ca0 0x7f87dc0c4a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52720/0 (socket says 172.21.15.7:52720) 2024-01-27T02:43:45.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.955+0000 7f87e2244700 1 -- 172.21.15.7:0/2972094133 learned_addr learned my addr 172.21.15.7:0/2972094133 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:45.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.955+0000 7f87e1a43700 1 --2- 172.21.15.7:0/2972094133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87dc0c3ae0 0x7f87dc0cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:45.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.955+0000 7f87e2a45700 1 --2- 172.21.15.7:0/2972094133 >> v2:172.21.15.188:3300/0 conn(0x7f87dc158070 0x7f87dc0cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:45.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.955+0000 7f87e1242700 1 -- 172.21.15.7:0/2972094133 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321065314 0 0) 0x7f87dc0ca020 con 0x7f87dc157ca0 2024-01-27T02:43:45.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.955+0000 7f87e1242700 1 -- 172.21.15.7:0/2972094133 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87c8003570 con 0x7f87dc157ca0 2024-01-27T02:43:45.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.956+0000 7f87e2a45700 1 -- 172.21.15.7:0/2972094133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87dc0c3ae0 msgr2=0x7f87dc0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:45.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.956+0000 7f87e2a45700 1 --2- 172.21.15.7:0/2972094133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87dc0c3ae0 0x7f87dc0cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:45.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.956+0000 7f87e2a45700 1 -- 172.21.15.7:0/2972094133 >> v1:172.21.15.7:6789/0 conn(0x7f87dc157ca0 legacy=0x7f87dc0c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:45.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.956+0000 7f87e2a45700 1 -- 172.21.15.7:0/2972094133 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc0c57d0 con 0x7f87dc158070 2024-01-27T02:43:45.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.956+0000 7f87e1a43700 1 --2- 172.21.15.7:0/2972094133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87dc0c3ae0 0x7f87dc0cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:43:45.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.956+0000 7f87e2a45700 1 --2- 172.21.15.7:0/2972094133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87dc158070 0x7f87dc0cda20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009e60 tx=0x7f87d4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad67552f00786ba5 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:45.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.959+0000 7f87e1242700 1 -- 172.21.15.7:0/2972094133 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d400e070 con 0x7f87dc158070 2024-01-27T02:43:45.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.959+0000 7f87e1242700 1 -- 172.21.15.7:0/2972094133 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d400bab0 con 0x7f87dc158070 2024-01-27T02:43:45.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.959+0000 7f87e1242700 1 -- 172.21.15.7:0/2972094133 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d400be70 con 0x7f87dc158070 2024-01-27T02:43:45.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.960+0000 7f87e44a8700 1 -- 172.21.15.7:0/2972094133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87dc158070 msgr2=0x7f87dc0cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:45.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.960+0000 7f87e44a8700 1 --2- 172.21.15.7:0/2972094133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87dc158070 0x7f87dc0cda20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009e60 tx=0x7f87d4009780 comp rx=0 tx=0).stop 2024-01-27T02:43:45.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.960+0000 7f87e44a8700 1 -- 172.21.15.7:0/2972094133 shutdown_connections 2024-01-27T02:43:45.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.960+0000 7f87e44a8700 1 --2- 172.21.15.7:0/2972094133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87dc158070 0x7f87dc0cda20 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:45.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.960+0000 7f87e44a8700 1 --2- 172.21.15.7:0/2972094133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87dc0c3ae0 0x7f87dc0cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:45.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.960+0000 7f87e44a8700 1 -- 172.21.15.7:0/2972094133 >> 172.21.15.7:0/2972094133 conn(0x7f87dc098020 msgr2=0x7f87dc0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:45.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.960+0000 7f87e44a8700 1 -- 172.21.15.7:0/2972094133 shutdown_connections 2024-01-27T02:43:45.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.960+0000 7f87e44a8700 1 -- 172.21.15.7:0/2972094133 wait complete. 2024-01-27T02:43:45.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.961+0000 7f87e44a8700 1 Processor -- start 2024-01-27T02:43:45.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.961+0000 7f87e44a8700 1 -- start start 2024-01-27T02:43:45.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.962+0000 7f87e44a8700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87dc0c3ae0 0x7f87dc152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:45.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.962+0000 7f87e44a8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87dc157ca0 0x7f87dc0c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:45.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.962+0000 7f87e44a8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87dc158070 0x7f87dc0c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:45.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.962+0000 7f87e44a8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc154970 con 0x7f87dc157ca0 2024-01-27T02:43:45.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.962+0000 7f87e44a8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0b1e10 con 0x7f87dc158070 2024-01-27T02:43:45.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.962+0000 7f87e44a8700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0b20f0 con 0x7f87dc0c3ae0 2024-01-27T02:43:45.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.962+0000 7f87e1a43700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87dc157ca0 0x7f87dc0c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:45.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.962+0000 7f87e1a43700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87dc157ca0 0x7f87dc0c7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48466/0 (socket says 172.21.15.7:48466) 2024-01-27T02:43:45.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.962+0000 7f87e1a43700 1 -- 172.21.15.7:0/3538568571 learned_addr learned my addr 172.21.15.7:0/3538568571 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:45.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.962+0000 7f87e2244700 1 --2- 172.21.15.7:0/3538568571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87dc0c3ae0 0x7f87dc152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:45.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.962+0000 7f87e2a45700 1 --2- 172.21.15.7:0/3538568571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87dc158070 0x7f87dc0c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:45.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.963+0000 7f87e1a43700 1 -- 172.21.15.7:0/3538568571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87dc0c3ae0 msgr2=0x7f87dc152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:45.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.963+0000 7f87e1a43700 1 --2- 172.21.15.7:0/3538568571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87dc0c3ae0 0x7f87dc152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:45.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.963+0000 7f87e1a43700 1 -- 172.21.15.7:0/3538568571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87dc158070 msgr2=0x7f87dc0c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:45.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.963+0000 7f87e1a43700 1 --2- 172.21.15.7:0/3538568571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87dc158070 0x7f87dc0c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:45.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.963+0000 7f87e1a43700 1 -- 172.21.15.7:0/3538568571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc009420 con 0x7f87dc157ca0 2024-01-27T02:43:45.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.963+0000 7f87e1a43700 1 --2- 172.21.15.7:0/3538568571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87dc157ca0 0x7f87dc0c7b70 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00ec80 tx=0x7f87cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:45.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.963+0000 7f87d27fc700 1 -- 172.21.15.7:0/3538568571 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc00c920 con 0x7f87dc157ca0 2024-01-27T02:43:45.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.963+0000 7f87d27fc700 1 -- 172.21.15.7:0/3538568571 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc00ca80 con 0x7f87dc157ca0 2024-01-27T02:43:45.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.963+0000 7f87e44a8700 1 -- 172.21.15.7:0/3538568571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d4009a20 con 0x7f87dc157ca0 2024-01-27T02:43:45.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.963+0000 7f87d27fc700 1 -- 172.21.15.7:0/3538568571 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc00ce00 con 0x7f87dc157ca0 2024-01-27T02:43:45.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.963+0000 7f87e44a8700 1 -- 172.21.15.7:0/3538568571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87dc0b26b0 con 0x7f87dc157ca0 2024-01-27T02:43:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.964+0000 7f87e44a8700 1 -- 172.21.15.7:0/3538568571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87dc04c710 con 0x7f87dc157ca0 2024-01-27T02:43:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.965+0000 7f87d27fc700 1 -- 172.21.15.7:0/3538568571 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f87cc013680 con 0x7f87dc157ca0 2024-01-27T02:43:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.965+0000 7f87d27fc700 1 --2- 172.21.15.7:0/3538568571 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f87c0066f30 0x7f87c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.965+0000 7f87d27fc700 1 -- 172.21.15.7:0/3538568571 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f87cc084900 con 0x7f87dc157ca0 2024-01-27T02:43:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.965+0000 7f87d27fc700 1 -- 172.21.15.7:0/3538568571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7f87c006ea50 con 0x7f87dc157ca0 2024-01-27T02:43:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.965+0000 7f87e2244700 1 --2- 172.21.15.7:0/3538568571 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f87c0066f30 0x7f87c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.966+0000 7f87e2244700 1 --2- 172.21.15.7:0/3538568571 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f87c0066f30 0x7f87c00693e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f87d8007fd0 tx=0x7f87d8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:45.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:45.970+0000 7f87d27fc700 1 -- 172.21.15.7:0/3538568571 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87cc017060 con 0x7f87dc157ca0 2024-01-27T02:43:46.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.173+0000 7f87e44a8700 1 -- 172.21.15.7:0/3538568571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f87dc0c9bb0 con 0x7f87dc157ca0 2024-01-27T02:43:46.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.174+0000 7f87d27fc700 1 -- 172.21.15.7:0/3538568571 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v182) v1 ==== 81+0+32253 (secure 0 0 0) 0x7f87cc051b70 con 0x7f87dc157ca0 2024-01-27T02:43:46.204 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:46.204 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:43:46.204 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 182, 2024-01-27T02:43:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:43:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:43:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:43:44.933767+0000", 2024-01-27T02:43:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:43:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:43:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:43:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:43:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:43:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:43:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:43:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:43:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:43:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:43:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:43:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:43:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:43:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:43:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 23, 2024-01-27T02:43:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:43:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:43:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:43:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:43:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:43:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:43:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:43:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:43:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:43:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:43:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:43:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:43:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:43:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:43:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:43:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:43:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:43:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:43:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:43:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:43:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:43:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:43:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:43:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:43:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:43:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:43:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:43:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:43:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:43:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:43:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:43:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:43:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "106", 2024-01-27T02:43:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:43:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:43:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:43:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:43:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:43:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:43:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:43:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:43:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:43:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:43:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:43:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:43:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:43:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:43:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:43:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:43:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:43:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:43:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:43:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:43:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:43:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:43:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:43:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:43:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:43:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:43:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:43:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:43:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:43:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:43:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:43:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:43:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:43:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:43:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:43:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:43:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:43:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:43:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:43:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:43:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:43:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:43:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 22, 2024-01-27T02:43:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:43:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:43:27.492968+0000", 2024-01-27T02:43:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:43:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:43:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:43:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:43:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:43:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:43:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:43:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:43:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:43:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:43:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:43:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:43:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:43:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:43:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:43:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:43:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:43:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:43:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:43:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:43:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:43:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:43:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:43:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:43:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "179", 2024-01-27T02:43:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:43:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:43:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:43:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:43:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:43:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:43:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:43:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:43:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:43:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:43:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:43:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:43:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:43:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:43:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:43:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:43:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:43:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:43:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:43:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:43:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:43:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:43:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:43:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:43:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:43:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:43:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:43:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:43:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:43:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:43:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:43:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:43:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:43:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:43:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:43:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:43:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:43:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:43:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:43:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:43:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:43:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:43:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:43:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:43:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 23, 2024-01-27T02:43:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:43:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:43:28.579977+0000", 2024-01-27T02:43:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1027, 2024-01-27T02:43:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full,full_quota", 2024-01-27T02:43:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:43:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:43:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:43:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:43:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:43:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:43:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:43:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:43:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:43:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:43:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:43:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:43:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:43:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:43:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:43:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:43:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:43:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:43:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:43:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:43:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:43:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:43:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "182", 2024-01-27T02:43:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:43:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:43:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:43:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:43:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:43:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:43:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:43:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:43:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:43:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-27T02:43:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:43:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:43:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:43:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:43:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:43:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:43:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:43:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:43:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:43:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:43:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:43:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:43:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:43:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:43:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:43:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:43:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:43:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:43:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:43:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:43:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:43:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:43:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:43:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:43:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:43:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:43:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:43:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:43:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:43:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:43:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:43:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:43:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:43:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:43:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:43:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:43:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:43:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:43:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:43:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:43:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:43:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:43:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:43:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:43:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:43:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:43:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:43:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:43:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:43:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:43:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:43:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:43:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:43:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:43:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:43:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:43:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:43:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:43:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:43:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:43:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:43:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:43:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:43:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:43:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:43:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:43:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:43:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:43:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:43:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:43:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:43:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:43:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:43:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:43:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:43:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:43:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:43:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:43:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:43:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:43:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:43:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:43:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:43:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:43:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:43:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:43:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:43:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:43:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:43:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:43:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:43:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:43:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:43:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:43:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:43:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:43:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:43:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:43:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:43:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:43:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:43:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:43:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:43:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:43:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:43:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:43:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:43:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:43:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:43:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:43:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:43:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:43:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:43:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:43:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:43:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:43:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:43:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:43:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:43:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:43:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:43:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:43:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:43:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:43:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:43:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:46.261 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:46.261 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.261 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:43:46.261 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:46.261 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:46.261 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:46.261 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:46.261 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:46.261 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:46.261 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:46.261 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:46.261 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:46.261 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:46.262 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:46.262 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:46.262 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.262 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:43:46.262 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:46.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.262 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:43:46.262 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:46.262 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.262 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.263 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:46.263 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.263 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:43:46.263 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:46.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.263 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:43:46.263 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:46.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.263 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.263 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:43:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:43:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:43:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:43:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:43:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:43:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:43:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:43:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:43:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:43:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:43:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:43:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:43:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:43:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:43:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:43:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:43:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:43:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:43:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:43:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:43:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:43:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:43:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:43:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:43:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:43:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:43:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:43:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:43:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:43:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:43:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:43:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:43:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:43:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:43:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:43:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:43:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:43:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:43:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:43:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:43:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:43:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:43:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:43:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:43:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:43:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:43:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:43:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:43:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.273 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:43:46.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.177+0000 7f87e44a8700 1 -- 172.21.15.7:0/3538568571 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f87c0066f30 msgr2=0x7f87c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:46.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.177+0000 7f87e44a8700 1 --2- 172.21.15.7:0/3538568571 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f87c0066f30 0x7f87c00693e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f87d8007fd0 tx=0x7f87d8009040 comp rx=0 tx=0).stop 2024-01-27T02:43:46.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.177+0000 7f87e44a8700 1 -- 172.21.15.7:0/3538568571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87dc157ca0 msgr2=0x7f87dc0c7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:46.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.177+0000 7f87e44a8700 1 --2- 172.21.15.7:0/3538568571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87dc157ca0 0x7f87dc0c7b70 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00ec80 tx=0x7f87cc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:46.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.177+0000 7f87e44a8700 1 -- 172.21.15.7:0/3538568571 shutdown_connections 2024-01-27T02:43:46.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.177+0000 7f87e44a8700 1 --2- 172.21.15.7:0/3538568571 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f87c0066f30 0x7f87c00693e0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:46.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.177+0000 7f87e44a8700 1 --2- 172.21.15.7:0/3538568571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87dc158070 0x7f87dc0c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:46.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.177+0000 7f87e44a8700 1 --2- 172.21.15.7:0/3538568571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87dc0c3ae0 0x7f87dc152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:46.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.177+0000 7f87e44a8700 1 --2- 172.21.15.7:0/3538568571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87dc157ca0 0x7f87dc0c7b70 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:46.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.177+0000 7f87e44a8700 1 -- 172.21.15.7:0/3538568571 >> 172.21.15.7:0/3538568571 conn(0x7f87dc098020 msgr2=0x7f87dc06c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:46.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.177+0000 7f87e44a8700 1 -- 172.21.15.7:0/3538568571 shutdown_connections 2024-01-27T02:43:46.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.177+0000 7f87e44a8700 1 -- 172.21.15.7:0/3538568571 wait complete. 2024-01-27T02:43:46.276 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-01-27T02:43:46.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:43:46.277 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2024-01-27T02:43:46.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-27T02:43:46.556 INFO:teuthology.orchestra.run.smithi007.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2024-01-27T02:43:46.556 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records in 2024-01-27T02:43:46.557 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records out 2024-01-27T02:43:46.557 INFO:teuthology.orchestra.run.smithi007.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.255965 s, 205 MB/s 2024-01-27T02:43:46.557 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:43:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.690+0000 7fece0ee2700 1 Processor -- start 2024-01-27T02:43:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.690+0000 7fece0ee2700 1 -- start start 2024-01-27T02:43:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.691+0000 7fece0ee2700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fecdc15a100 0x7fecdc0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.691+0000 7fece0ee2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fecdc0cb5b0 0x7fecdc0c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.691+0000 7fece0ee2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecdc0c4af0 0x7fecdc0c9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.691+0000 7fece0ee2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecdc0c4f70 con 0x7fecdc0cb5b0 2024-01-27T02:43:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.692+0000 7fece0ee2700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecdc0ce5e0 con 0x7fecdc15a100 2024-01-27T02:43:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.692+0000 7fece0ee2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc0cd9d0 con 0x7fecdc0c4af0 2024-01-27T02:43:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.692+0000 7fecd9d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fecdc0cb5b0 0x7fecdc0c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.692+0000 7fecd9d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fecdc0cb5b0 0x7fecdc0c45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48472/0 (socket says 172.21.15.7:48472) 2024-01-27T02:43:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.692+0000 7fecd9d9b700 1 -- 172.21.15.7:0/2072350675 learned_addr learned my addr 172.21.15.7:0/2072350675 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.692+0000 7fecda59c700 1 --2- 172.21.15.7:0/2072350675 >> v2:172.21.15.188:3300/0 conn(0x7fecdc15a100 0x7fecdc0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.692+0000 7fecdad9d700 1 --2- 172.21.15.7:0/2072350675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecdc0c4af0 0x7fecdc0c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.692+0000 7fecd9d9b700 1 -- 172.21.15.7:0/2072350675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecdc0c4af0 msgr2=0x7fecdc0c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.692+0000 7fecd9d9b700 1 --2- 172.21.15.7:0/2072350675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecdc0c4af0 0x7fecdc0c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.692+0000 7fecd9d9b700 1 -- 172.21.15.7:0/2072350675 >> v2:172.21.15.188:3300/0 conn(0x7fecdc15a100 msgr2=0x7fecdc0cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.692+0000 7fecd9d9b700 1 --2- 172.21.15.7:0/2072350675 >> v2:172.21.15.188:3300/0 conn(0x7fecdc15a100 0x7fecdc0cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.692+0000 7fecd9d9b700 1 -- 172.21.15.7:0/2072350675 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc0cdcb0 con 0x7fecdc0cb5b0 2024-01-27T02:43:46.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.692+0000 7fecd9d9b700 1 --2- 172.21.15.7:0/2072350675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecdc0cb5b0 0x7fecdc0c45b0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fecdc06d710 tx=0x7fecd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca334138ad3c9a2 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:46.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.693+0000 7fecd959a700 1 -- 172.21.15.7:0/2072350675 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd000b640 con 0x7fecdc0cb5b0 2024-01-27T02:43:46.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.693+0000 7fecd959a700 1 -- 172.21.15.7:0/2072350675 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd000b7a0 con 0x7fecdc0cb5b0 2024-01-27T02:43:46.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.693+0000 7fecd959a700 1 -- 172.21.15.7:0/2072350675 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd000bb60 con 0x7fecdc0cb5b0 2024-01-27T02:43:46.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.693+0000 7fece0ee2700 1 -- 172.21.15.7:0/2072350675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecdc0cb5b0 msgr2=0x7fecdc0c45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:46.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.693+0000 7fece0ee2700 1 --2- 172.21.15.7:0/2072350675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecdc0cb5b0 0x7fecdc0c45b0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fecdc06d710 tx=0x7fecd0009250 comp rx=0 tx=0).stop 2024-01-27T02:43:46.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.693+0000 7fece0ee2700 1 -- 172.21.15.7:0/2072350675 shutdown_connections 2024-01-27T02:43:46.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.693+0000 7fece0ee2700 1 --2- 172.21.15.7:0/2072350675 >> v2:172.21.15.188:3300/0 conn(0x7fecdc15a100 0x7fecdc0cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:46.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.693+0000 7fece0ee2700 1 --2- 172.21.15.7:0/2072350675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecdc0c4af0 0x7fecdc0c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:46.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.693+0000 7fece0ee2700 1 --2- 172.21.15.7:0/2072350675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecdc0cb5b0 0x7fecdc0c45b0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:46.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.693+0000 7fece0ee2700 1 -- 172.21.15.7:0/2072350675 >> 172.21.15.7:0/2072350675 conn(0x7fecdc098020 msgr2=0x7fecdc0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:46.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.693+0000 7fece0ee2700 1 -- 172.21.15.7:0/2072350675 shutdown_connections 2024-01-27T02:43:46.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.693+0000 7fece0ee2700 1 -- 172.21.15.7:0/2072350675 wait complete. 2024-01-27T02:43:46.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.694+0000 7fece0ee2700 1 Processor -- start 2024-01-27T02:43:46.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.694+0000 7fece0ee2700 1 -- start start 2024-01-27T02:43:46.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.695+0000 7fece0ee2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecdc0c4af0 0x7fecdc0c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:46.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.695+0000 7fece0ee2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecdc0cb5b0 0x7fecdc0c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:46.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.695+0000 7fece0ee2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecdc15a100 0x7fecdc1542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:46.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.695+0000 7fece0ee2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc0b1ae0 con 0x7fecdc0cb5b0 2024-01-27T02:43:46.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.695+0000 7fece0ee2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc15a860 con 0x7fecdc0c4af0 2024-01-27T02:43:46.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.695+0000 7fece0ee2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc0c7e50 con 0x7fecdc15a100 2024-01-27T02:43:46.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.695+0000 7fecd9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecdc0cb5b0 0x7fecdc0c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:46.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.695+0000 7fecd9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecdc0cb5b0 0x7fecdc0c6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48474/0 (socket says 172.21.15.7:48474) 2024-01-27T02:43:46.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.695+0000 7fecd9d9b700 1 -- 172.21.15.7:0/2090057684 learned_addr learned my addr 172.21.15.7:0/2090057684 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:46.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.695+0000 7fecda59c700 1 --2- 172.21.15.7:0/2090057684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecdc0c4af0 0x7fecdc0c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:46.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.696+0000 7fecdad9d700 1 --2- 172.21.15.7:0/2090057684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecdc15a100 0x7fecdc1542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:46.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.696+0000 7fecd9d9b700 1 -- 172.21.15.7:0/2090057684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecdc15a100 msgr2=0x7fecdc1542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:46.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.696+0000 7fecd9d9b700 1 --2- 172.21.15.7:0/2090057684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecdc15a100 0x7fecdc1542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:46.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.696+0000 7fecd9d9b700 1 -- 172.21.15.7:0/2090057684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecdc0c4af0 msgr2=0x7fecdc0c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:46.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.696+0000 7fecd9d9b700 1 --2- 172.21.15.7:0/2090057684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecdc0c4af0 0x7fecdc0c7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:46.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.696+0000 7fecd9d9b700 1 -- 172.21.15.7:0/2090057684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd00094f0 con 0x7fecdc0cb5b0 2024-01-27T02:43:46.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.696+0000 7fecd9d9b700 1 --2- 172.21.15.7:0/2090057684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecdc0cb5b0 0x7fecdc0c6870 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fecdc06ea80 tx=0x7fecd0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:46.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.696+0000 7fecca7fc700 1 -- 172.21.15.7:0/2090057684 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd000b910 con 0x7fecdc0cb5b0 2024-01-27T02:43:46.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.696+0000 7fecca7fc700 1 -- 172.21.15.7:0/2090057684 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd000ba70 con 0x7fecdc0cb5b0 2024-01-27T02:43:46.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.696+0000 7fece0ee2700 1 -- 172.21.15.7:0/2090057684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecdc0b1cc0 con 0x7fecdc0cb5b0 2024-01-27T02:43:46.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.696+0000 7fecca7fc700 1 -- 172.21.15.7:0/2090057684 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd0011630 con 0x7fecdc0cb5b0 2024-01-27T02:43:46.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.696+0000 7fece0ee2700 1 -- 172.21.15.7:0/2090057684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecdc0b2280 con 0x7fecdc0cb5b0 2024-01-27T02:43:46.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.697+0000 7fece0ee2700 1 -- 172.21.15.7:0/2090057684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecdc0c6e70 con 0x7fecdc0cb5b0 2024-01-27T02:43:46.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.698+0000 7fecca7fc700 1 -- 172.21.15.7:0/2090057684 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fecd00119a0 con 0x7fecdc0cb5b0 2024-01-27T02:43:46.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.698+0000 7fecca7fc700 1 --2- 172.21.15.7:0/2090057684 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fecbc066f30 0x7fecbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:46.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.698+0000 7fecca7fc700 1 -- 172.21.15.7:0/2090057684 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fecd0085b50 con 0x7fecdc0cb5b0 2024-01-27T02:43:46.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.698+0000 7fecca7fc700 1 -- 172.21.15.7:0/2090057684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7fecbc06ea50 con 0x7fecdc0cb5b0 2024-01-27T02:43:46.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.698+0000 7fecda59c700 1 --2- 172.21.15.7:0/2090057684 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fecbc066f30 0x7fecbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:46.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.699+0000 7fecda59c700 1 --2- 172.21.15.7:0/2090057684 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fecbc066f30 0x7fecbc0693e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7feccc009d10 tx=0x7feccc0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:46.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.702+0000 7fecca7fc700 1 -- 172.21.15.7:0/2090057684 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fecd0052dc0 con 0x7fecdc0cb5b0 2024-01-27T02:43:46.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.904+0000 7fece0ee2700 1 -- 172.21.15.7:0/2090057684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fecdc04c710 con 0x7fecdc0cb5b0 2024-01-27T02:43:46.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.906+0000 7fecca7fc700 1 -- 172.21.15.7:0/2090057684 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v182) v1 ==== 81+0+32253 (secure 0 0 0) 0x7fecd0015070 con 0x7fecdc0cb5b0 2024-01-27T02:43:46.906 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:46.906 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:43:46.907 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 182, 2024-01-27T02:43:46.907 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:43:46.907 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:43:46.907 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:43:44.933767+0000", 2024-01-27T02:43:46.907 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:43:46.907 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:43:46.907 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:43:46.907 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:43:46.907 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:43:46.907 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:43:46.907 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:43:46.907 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:43:46.907 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:43:46.908 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:46.908 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:43:46.908 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:43:46.908 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:43:46.908 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:43:46.908 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:43:46.908 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 23, 2024-01-27T02:43:46.908 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:43:46.908 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:43:46.908 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:43:46.908 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:43:46.908 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:43:46.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.909 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:43:46.909 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:43:46.909 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:43:46.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:43:46.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:43:46.909 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:43:46.909 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:43:46.909 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:43:46.909 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:43:46.909 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:43:46.909 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:43:46.909 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:43:46.909 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:43:46.909 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:43:46.910 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:43:46.910 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:43:46.910 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:43:46.910 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:43:46.910 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:43:46.910 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:43:46.910 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:43:46.910 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:43:46.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:43:46.910 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:43:46.910 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:43:46.910 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:43:46.910 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:43:46.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.911 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "106", 2024-01-27T02:43:46.911 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:43:46.911 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:43:46.911 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:43:46.911 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:43:46.911 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:43:46.911 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:43:46.911 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:43:46.911 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:43:46.911 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:43:46.911 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:43:46.911 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:43:46.911 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:43:46.911 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:43:46.912 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:43:46.912 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:43:46.912 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:43:46.912 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:43:46.912 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:43:46.912 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:43:46.912 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:43:46.912 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:43:46.912 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:43:46.912 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:43:46.912 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:43:46.912 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:43:46.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:43:46.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.913 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:43:46.913 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:43:46.913 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:43:46.913 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:43:46.913 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:43:46.913 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:43:46.913 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:43:46.913 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:43:46.913 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:43:46.913 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:43:46.913 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:43:46.913 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:43:46.913 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:43:46.913 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:43:46.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.914 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:43:46.914 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:43:46.914 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.914 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 22, 2024-01-27T02:43:46.914 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:43:46.914 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:43:27.492968+0000", 2024-01-27T02:43:46.914 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:43:46.914 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:43:46.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:43:46.914 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:43:46.914 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:43:46.914 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:43:46.914 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:43:46.915 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:43:46.915 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:43:46.915 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:43:46.915 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:43:46.915 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:43:46.915 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:43:46.915 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:43:46.915 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:43:46.915 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:43:46.915 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:43:46.915 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:43:46.915 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:43:46.915 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:43:46.915 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:43:46.916 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:43:46.916 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:43:46.916 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:43:46.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.916 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "179", 2024-01-27T02:43:46.916 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:43:46.916 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:43:46.916 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:43:46.916 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:43:46.916 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:43:46.916 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:43:46.916 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:43:46.916 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:43:46.917 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:43:46.917 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:43:46.917 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:43:46.917 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:43:46.917 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:43:46.917 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:43:46.917 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:43:46.917 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:43:46.917 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:43:46.917 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:43:46.917 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:43:46.917 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:43:46.917 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:43:46.917 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:43:46.918 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:43:46.918 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:43:46.919 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:43:46.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:43:46.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.919 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:43:46.919 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:43:46.919 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:43:46.919 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:43:46.919 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:43:46.919 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:43:46.919 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:43:46.919 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:43:46.919 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:43:46.920 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:43:46.920 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:43:46.920 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:43:46.920 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:43:46.920 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:43:46.920 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:43:46.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.920 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:43:46.920 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:43:46.920 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:43:46.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.921 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 23, 2024-01-27T02:43:46.921 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:43:46.921 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:43:28.579977+0000", 2024-01-27T02:43:46.921 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1027, 2024-01-27T02:43:46.921 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full,full_quota", 2024-01-27T02:43:46.921 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:43:46.921 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:43:46.921 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:43:46.921 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:43:46.921 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:43:46.921 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:43:46.921 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:43:46.922 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:43:46.922 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:43:46.922 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:43:46.922 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:43:46.922 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:43:46.922 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:43:46.922 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:43:46.922 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:43:46.922 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:43:46.922 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:43:46.922 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:43:46.922 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:43:46.922 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:43:46.922 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:43:46.923 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:43:46.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.923 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "182", 2024-01-27T02:43:46.923 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:43:46.923 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:43:46.923 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:43:46.923 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:43:46.923 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:43:46.923 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:43:46.923 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:43:46.923 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:43:46.923 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:43:46.923 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-27T02:43:46.923 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:43:46.923 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:43:46.924 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:43:46.924 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:43:46.924 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:43:46.924 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:43:46.924 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:43:46.924 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:43:46.924 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:43:46.924 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:43:46.924 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:43:46.924 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:43:46.924 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:43:46.924 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:43:46.924 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:43:46.925 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:43:46.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.925 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:43:46.925 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:43:46.925 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:43:46.925 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:43:46.925 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:43:46.925 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:43:46.925 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:43:46.925 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:43:46.925 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:43:46.925 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:43:46.925 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:43:46.926 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:43:46.926 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:43:46.926 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:43:46.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.926 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:43:46.926 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:43:46.926 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:43:46.926 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.926 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.926 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.926 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:46.926 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:43:46.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.926 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:43:46.926 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:46.927 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:46.927 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:46.927 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:46.927 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:46.927 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:46.927 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:46.927 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:46.927 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:46.927 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:46.927 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:46.927 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:46.927 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.928 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.928 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:43:46.928 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:46.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.928 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.928 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:43:46.928 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:46.928 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.928 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.928 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:46.928 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.929 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:43:46.929 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:46.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.929 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:43:46.929 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:46.929 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.929 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.929 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:46.929 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.930 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.930 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:43:46.930 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:46.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.930 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.930 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:43:46.930 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:46.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.930 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.930 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:46.930 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.931 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:43:46.931 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:46.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.931 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:43:46.931 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:46.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.931 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.931 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:43:46.931 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:43:46.931 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:43:46.932 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:43:46.932 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:46.932 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:46.932 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:46.932 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.932 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:43:46.932 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:46.932 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:46.932 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:46.932 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:46.932 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:46.932 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:46.933 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:46.933 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:46.933 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:46.933 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:46.933 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:46.933 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:46.933 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.933 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:43:46.933 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:46.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.933 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:43:46.934 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:46.934 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.934 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.934 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:46.934 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.934 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.934 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:43:46.934 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:46.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.934 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.935 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:43:46.935 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:46.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.935 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.935 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:46.935 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.935 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:43:46.935 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:46.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.935 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:43:46.936 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:46.936 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.936 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.936 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:46.936 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.936 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:43:46.936 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:46.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.937 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:43:46.937 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:46.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.937 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.937 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:43:46.937 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:43:46.937 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:43:46.937 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:43:46.937 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:46.937 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:46.937 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:46.937 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.938 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:43:46.938 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:46.938 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:46.938 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:46.938 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:46.938 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:46.938 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:46.938 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:46.938 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:46.938 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:46.938 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:46.938 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:46.938 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:46.939 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.939 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:43:46.939 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:46.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.939 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:43:46.939 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:46.939 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.939 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.940 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:46.940 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.940 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.940 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:43:46.940 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:46.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.940 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.940 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:43:46.940 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:46.940 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.940 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.941 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:46.941 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.941 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:43:46.941 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:46.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.941 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:43:46.941 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:46.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.941 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.941 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:46.942 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.942 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:43:46.942 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:46.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.942 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:43:46.942 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:46.942 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.942 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.943 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:43:46.943 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:43:46.943 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:43:46.943 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:43:46.943 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:46.943 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:46.943 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:46.943 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.943 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:43:46.943 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:46.943 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:46.943 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:46.944 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:46.944 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:46.944 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:46.944 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:46.944 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:46.944 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:46.944 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:46.944 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:46.944 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:46.944 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.944 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:43:46.945 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:46.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.945 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:43:46.945 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:46.945 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.945 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.945 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:46.945 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.945 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:43:46.946 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:46.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.946 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:43:46.946 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:46.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.946 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.946 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:46.946 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.947 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:43:46.947 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:46.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.947 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:43:46.947 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:46.947 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.947 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.947 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:46.947 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.947 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:43:46.948 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:46.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.948 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:43:46.948 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:46.948 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.948 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.948 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:43:46.948 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:43:46.948 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:43:46.948 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:43:46.949 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:46.949 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:46.949 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:46.949 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.949 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:43:46.949 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:46.949 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:46.949 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:46.949 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:46.949 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:46.949 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:46.949 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:46.950 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:46.950 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:46.950 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:46.950 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:46.950 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:46.950 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.950 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:43:46.950 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:46.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.950 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:43:46.951 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:46.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.951 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.951 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:46.951 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.951 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:43:46.951 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:46.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.951 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:43:46.952 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:46.952 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.952 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.952 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:46.952 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.952 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.952 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:43:46.952 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:46.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.952 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.952 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:43:46.953 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:46.953 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.953 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.953 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:46.953 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.953 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:43:46.953 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:46.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.953 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:43:46.954 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:46.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.954 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.954 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:43:46.954 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:43:46.954 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:43:46.954 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:43:46.954 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:46.954 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:46.954 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:46.954 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.955 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:43:46.955 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:46.955 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:46.955 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:46.955 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:46.955 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:46.955 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:46.955 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:46.955 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:46.955 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:46.955 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:46.955 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:46.955 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:46.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:43:46.956 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:46.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:43:46.956 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:46.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.956 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.956 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:46.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.957 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:43:46.957 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:46.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.957 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:43:46.957 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:46.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.957 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.957 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:46.957 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.958 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.958 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:43:46.958 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:46.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.958 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.958 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:43:46.958 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:46.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.958 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.958 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:46.958 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.959 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:43:46.959 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:46.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.959 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:43:46.959 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:46.959 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.959 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.959 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:43:46.959 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:43:46.959 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:43:46.960 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:43:46.960 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:46.960 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:46.960 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:46.960 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.960 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:43:46.960 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:46.960 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:46.960 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:46.960 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:46.960 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:46.960 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:46.961 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:46.961 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:46.961 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:46.961 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:46.961 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:46.961 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:46.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:43:46.961 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:46.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:43:46.962 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:46.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.962 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.962 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:46.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:43:46.962 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:46.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.963 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:43:46.963 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:46.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.963 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.963 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:46.963 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.963 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:43:46.963 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:46.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.964 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:43:46.964 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:46.964 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.964 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.964 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:46.964 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.964 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:43:46.964 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:46.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.965 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:43:46.965 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:46.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.965 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.965 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:43:46.965 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:43:46.965 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:43:46.965 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:43:46.965 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:46.965 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:46.965 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:46.965 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.966 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:43:46.966 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:46.966 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:46.966 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:46.966 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:46.966 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:46.966 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:46.966 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:46.966 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:46.966 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:46.966 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:46.966 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:46.967 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:46.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:43:46.967 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:46.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:43:46.967 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:46.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.967 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.968 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:46.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:43:46.968 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:46.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:43:46.968 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:46.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.968 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.969 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:46.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:43:46.969 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:46.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:43:46.969 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:46.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.969 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.970 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:46.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:46.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:46.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:43:46.970 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:46.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:46.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:43:46.970 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:46.970 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.970 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.971 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:43:46.971 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:43:46.971 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:43:46.971 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:43:46.971 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:46.971 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:46.971 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:46.971 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:46.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.971 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:46.971 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:43:46.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.971 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:43:46.972 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:46.972 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:46.972 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:46.972 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:46.972 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:46.972 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:43:46.972 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:46.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.972 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:43:46.972 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:46.972 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:46.972 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:46.972 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:46.973 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:46.973 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:43:46.973 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:46.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.973 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:43:46.973 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:46.973 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:46.973 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:46.973 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:46.973 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:46.973 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:43:46.973 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:46.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.974 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:43:46.974 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:46.974 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:46.974 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:46.974 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:46.974 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:46.974 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:43:46.974 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:46.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.974 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:43:46.974 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:46.974 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:46.975 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:46.975 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:46.975 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:46.975 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:43:46.975 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:46.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.975 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:43:46.975 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:46.975 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:46.975 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:46.975 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:46.975 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:46.975 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:43:46.976 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:46.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.976 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:43:46.976 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:46.976 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:46.976 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:46.976 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:46.976 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:46.976 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:43:46.976 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:46.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:46.976 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:43:46.977 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:46.977 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:46.977 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:46.977 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:46.977 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:46.977 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:43:46.977 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:46.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.977 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:46.977 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:43:46.977 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:43:46.977 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:43:46.977 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:43:46.977 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:43:46.977 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:43:46.978 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:43:46.978 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:43:46.978 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:43:46.978 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:43:46.978 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:43:46.978 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:43:46.978 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:43:46.978 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:46.978 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:43:46.978 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:43:46.978 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:43:46.978 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:43:46.978 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:43:46.978 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:43:46.979 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:43:46.979 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:43:46.979 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:43:46.979 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:43:46.979 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:43:46.979 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:46.979 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:43:46.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.908+0000 7fece0ee2700 1 -- 172.21.15.7:0/2090057684 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fecbc066f30 msgr2=0x7fecbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:46.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.908+0000 7fece0ee2700 1 --2- 172.21.15.7:0/2090057684 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fecbc066f30 0x7fecbc0693e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7feccc009d10 tx=0x7feccc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:46.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.908+0000 7fece0ee2700 1 -- 172.21.15.7:0/2090057684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecdc0cb5b0 msgr2=0x7fecdc0c6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:46.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.908+0000 7fece0ee2700 1 --2- 172.21.15.7:0/2090057684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecdc0cb5b0 0x7fecdc0c6870 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fecdc06ea80 tx=0x7fecd0017000 comp rx=0 tx=0).stop 2024-01-27T02:43:46.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.908+0000 7fece0ee2700 1 -- 172.21.15.7:0/2090057684 shutdown_connections 2024-01-27T02:43:46.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.908+0000 7fece0ee2700 1 --2- 172.21.15.7:0/2090057684 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fecbc066f30 0x7fecbc0693e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:46.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.908+0000 7fece0ee2700 1 --2- 172.21.15.7:0/2090057684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecdc15a100 0x7fecdc1542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:46.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.908+0000 7fece0ee2700 1 --2- 172.21.15.7:0/2090057684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecdc0c4af0 0x7fecdc0c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:46.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.908+0000 7fece0ee2700 1 --2- 172.21.15.7:0/2090057684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecdc0cb5b0 0x7fecdc0c6870 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:46.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.908+0000 7fece0ee2700 1 -- 172.21.15.7:0/2090057684 >> 172.21.15.7:0/2090057684 conn(0x7fecdc098020 msgr2=0x7fecdc06c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:46.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.908+0000 7fece0ee2700 1 -- 172.21.15.7:0/2090057684 shutdown_connections 2024-01-27T02:43:46.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:46.908+0000 7fece0ee2700 1 -- 172.21.15.7:0/2090057684 wait complete. 2024-01-27T02:43:46.982 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:43:47.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.117+0000 7f9b03fc6700 1 Processor -- start 2024-01-27T02:43:47.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.117+0000 7f9b03fc6700 1 -- start start 2024-01-27T02:43:47.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.118+0000 7f9b03fc6700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9afc151570 0x7f9afc157f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:47.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.118+0000 7f9b03fc6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9afc06b400 0x7f9afc06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.118+0000 7f9b03fc6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9afc151940 con 0x7f9afc06e870 2024-01-27T02:43:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.118+0000 7f9b03fc6700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9afc13a1a0 con 0x7f9afc151570 2024-01-27T02:43:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.118+0000 7f9b03fc6700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc158490 con 0x7f9afc06b400 2024-01-27T02:43:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.118+0000 7f9b02563700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9afc06e870 0x7f9afc0c8270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52734/0 (socket says 172.21.15.7:52734) 2024-01-27T02:43:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.118+0000 7f9b02563700 1 -- 172.21.15.7:0/3763564342 learned_addr learned my addr 172.21.15.7:0/3763564342 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.118+0000 7f9b01561700 1 --2- 172.21.15.7:0/3763564342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9afc06b400 0x7f9afc06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.118+0000 7f9b01d62700 1 --2- 172.21.15.7:0/3763564342 >> v2:172.21.15.188:3300/0 conn(0x7f9afc151570 0x7f9afc157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.118+0000 7f9b00d60700 1 -- 172.21.15.7:0/3763564342 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1102437562 0 0) 0x7f9afc151940 con 0x7f9afc06e870 2024-01-27T02:43:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.118+0000 7f9b00d60700 1 -- 172.21.15.7:0/3763564342 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ae8003570 con 0x7f9afc06e870 2024-01-27T02:43:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.119+0000 7f9b01561700 1 -- 172.21.15.7:0/3763564342 >> v2:172.21.15.188:3300/0 conn(0x7f9afc151570 msgr2=0x7f9afc157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.119+0000 7f9b01561700 1 --2- 172.21.15.7:0/3763564342 >> v2:172.21.15.188:3300/0 conn(0x7f9afc151570 0x7f9afc157f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.119+0000 7f9b01561700 1 -- 172.21.15.7:0/3763564342 >> v1:172.21.15.7:6789/0 conn(0x7f9afc06e870 legacy=0x7f9afc0c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.119+0000 7f9b00d60700 1 -- 172.21.15.7:0/3763564342 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3591992839 0 0) 0x7f9ae8003570 con 0x7f9afc06e870 2024-01-27T02:43:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.119+0000 7f9b01561700 1 -- 172.21.15.7:0/3763564342 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc06c7f0 con 0x7f9afc06b400 2024-01-27T02:43:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.119+0000 7f9b01561700 1 --2- 172.21.15.7:0/3763564342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9afc06b400 0x7f9afc06e330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0098e0 tx=0x7f9aec0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a7436689d71df54 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.119+0000 7f9b00d60700 1 -- 172.21.15.7:0/3763564342 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af40017f0 con 0x7f9afc06b400 2024-01-27T02:43:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.119+0000 7f9b00d60700 1 -- 172.21.15.7:0/3763564342 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aec011420 con 0x7f9afc06b400 2024-01-27T02:43:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.119+0000 7f9b00d60700 1 -- 172.21.15.7:0/3763564342 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec0117a0 con 0x7f9afc06b400 2024-01-27T02:43:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.120+0000 7f9b03fc6700 1 -- 172.21.15.7:0/3763564342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9afc06b400 msgr2=0x7f9afc06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.120+0000 7f9b03fc6700 1 --2- 172.21.15.7:0/3763564342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9afc06b400 0x7f9afc06e330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0098e0 tx=0x7f9aec0091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.120+0000 7f9b03fc6700 1 -- 172.21.15.7:0/3763564342 shutdown_connections 2024-01-27T02:43:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.120+0000 7f9b03fc6700 1 --2- 172.21.15.7:0/3763564342 >> v2:172.21.15.188:3300/0 conn(0x7f9afc151570 0x7f9afc157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.120+0000 7f9b03fc6700 1 --2- 172.21.15.7:0/3763564342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9afc06b400 0x7f9afc06e330 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.120+0000 7f9b03fc6700 1 -- 172.21.15.7:0/3763564342 >> 172.21.15.7:0/3763564342 conn(0x7f9afc0cdc30 msgr2=0x7f9afc0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.120+0000 7f9b03fc6700 1 -- 172.21.15.7:0/3763564342 shutdown_connections 2024-01-27T02:43:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.120+0000 7f9b03fc6700 1 -- 172.21.15.7:0/3763564342 wait complete. 2024-01-27T02:43:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.121+0000 7f9b03fc6700 1 Processor -- start 2024-01-27T02:43:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.121+0000 7f9b03fc6700 1 -- start start 2024-01-27T02:43:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.122+0000 7f9b03fc6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9afc06b400 0x7f9afc147780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.122+0000 7f9b03fc6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9afc06e870 0x7f9afc139680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.122+0000 7f9b03fc6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9afc151570 0x7f9afc0c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.122+0000 7f9b03fc6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc0c8010 con 0x7f9afc06b400 2024-01-27T02:43:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.122+0000 7f9b03fc6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc14a450 con 0x7f9afc151570 2024-01-27T02:43:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.122+0000 7f9b03fc6700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc147cc0 con 0x7f9afc06e870 2024-01-27T02:43:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.122+0000 7f9b01d62700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9afc06b400 0x7f9afc147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.122+0000 7f9b01d62700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9afc06b400 0x7f9afc147780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48480/0 (socket says 172.21.15.7:48480) 2024-01-27T02:43:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.122+0000 7f9b01d62700 1 -- 172.21.15.7:0/2532354577 learned_addr learned my addr 172.21.15.7:0/2532354577 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.122+0000 7f9b02563700 1 --2- 172.21.15.7:0/2532354577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9afc151570 0x7f9afc0c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.122+0000 7f9b01561700 1 --2- 172.21.15.7:0/2532354577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9afc06e870 0x7f9afc139680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.122+0000 7f9b01d62700 1 -- 172.21.15.7:0/2532354577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9afc06e870 msgr2=0x7f9afc139680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.122+0000 7f9b01d62700 1 --2- 172.21.15.7:0/2532354577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9afc06e870 0x7f9afc139680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.122+0000 7f9b01d62700 1 -- 172.21.15.7:0/2532354577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9afc151570 msgr2=0x7f9afc0c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.122+0000 7f9b01d62700 1 --2- 172.21.15.7:0/2532354577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9afc151570 0x7f9afc0c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.122+0000 7f9b01d62700 1 -- 172.21.15.7:0/2532354577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aec009420 con 0x7f9afc06b400 2024-01-27T02:43:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.122+0000 7f9b01561700 1 --2- 172.21.15.7:0/2532354577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9afc06e870 0x7f9afc139680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:43:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.122+0000 7f9b02563700 1 --2- 172.21.15.7:0/2532354577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9afc151570 0x7f9afc0c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:43:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.122+0000 7f9b01d62700 1 --2- 172.21.15.7:0/2532354577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9afc06b400 0x7f9afc147780 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f9af800e930 tx=0x7f9af80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.123+0000 7f9af1ffb700 1 -- 172.21.15.7:0/2532354577 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af800eed0 con 0x7f9afc06b400 2024-01-27T02:43:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.123+0000 7f9af1ffb700 1 -- 172.21.15.7:0/2532354577 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af800c850 con 0x7f9afc06b400 2024-01-27T02:43:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.123+0000 7f9b03fc6700 1 -- 172.21.15.7:0/2532354577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9afc0c7d30 con 0x7f9afc06b400 2024-01-27T02:43:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.123+0000 7f9af1ffb700 1 -- 172.21.15.7:0/2532354577 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af800cbf0 con 0x7f9afc06b400 2024-01-27T02:43:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.123+0000 7f9b03fc6700 1 -- 172.21.15.7:0/2532354577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc14a0e0 con 0x7f9afc06b400 2024-01-27T02:43:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.124+0000 7f9b03fc6700 1 -- 172.21.15.7:0/2532354577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9afc149c30 con 0x7f9afc06b400 2024-01-27T02:43:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.124+0000 7f9af1ffb700 1 -- 172.21.15.7:0/2532354577 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9af8013470 con 0x7f9afc06b400 2024-01-27T02:43:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.125+0000 7f9af1ffb700 1 --2- 172.21.15.7:0/2532354577 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9ae0066f30 0x7f9ae00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.125+0000 7f9af1ffb700 1 -- 172.21.15.7:0/2532354577 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9af8084760 con 0x7f9afc06b400 2024-01-27T02:43:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.125+0000 7f9af1ffb700 1 -- 172.21.15.7:0/2532354577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7f9ae006ea50 con 0x7f9afc06b400 2024-01-27T02:43:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.125+0000 7f9b01561700 1 --2- 172.21.15.7:0/2532354577 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9ae0066f30 0x7f9ae00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.126+0000 7f9b01561700 1 --2- 172.21.15.7:0/2532354577 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9ae0066f30 0x7f9ae00693e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0c5120 tx=0x7f9aec016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.129+0000 7f9af1ffb700 1 -- 172.21.15.7:0/2532354577 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9af8021060 con 0x7f9afc06b400 2024-01-27T02:43:47.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.381+0000 7f9b03fc6700 1 -- 172.21.15.7:0/2532354577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9afc04c710 con 0x7f9afc06b400 2024-01-27T02:43:47.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.382+0000 7f9af1ffb700 1 -- 172.21.15.7:0/2532354577 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4595 (secure 0 0 0) 0x7f9af80519d0 con 0x7f9afc06b400 2024-01-27T02:43:47.383 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:47.383 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6704,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/1199331492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1199331492},{"type":"v1","addr":"172.21.15.7:6834","nonce":1199331492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":6707,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/721335086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":721335086},{"type":"v1","addr":"172.21.15.7:6835","nonce":721335086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":7699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/450555206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":450555206},{"type":"v1","addr":"172.21.15.188:6836","nonce":450555206}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:43:29.599748+0000","modified":"2024-01-27T02:43:33.196288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7702":{"gid":7702,"name":"b","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/3853795510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3853795510},{"type":"v1","addr":"172.21.15.188:6837","nonce":3853795510}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-27T02:43:47.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.384+0000 7f9b03fc6700 1 -- 172.21.15.7:0/2532354577 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9ae0066f30 msgr2=0x7f9ae00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:47.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.384+0000 7f9b03fc6700 1 --2- 172.21.15.7:0/2532354577 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9ae0066f30 0x7f9ae00693e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0c5120 tx=0x7f9aec016000 comp rx=0 tx=0).stop 2024-01-27T02:43:47.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.384+0000 7f9b03fc6700 1 -- 172.21.15.7:0/2532354577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9afc06b400 msgr2=0x7f9afc147780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:47.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.384+0000 7f9b03fc6700 1 --2- 172.21.15.7:0/2532354577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9afc06b400 0x7f9afc147780 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f9af800e930 tx=0x7f9af80091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:47.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.384+0000 7f9b03fc6700 1 -- 172.21.15.7:0/2532354577 shutdown_connections 2024-01-27T02:43:47.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.384+0000 7f9b03fc6700 1 --2- 172.21.15.7:0/2532354577 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f9ae0066f30 0x7f9ae00693e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:47.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.384+0000 7f9b03fc6700 1 --2- 172.21.15.7:0/2532354577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9afc151570 0x7f9afc0c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:47.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.384+0000 7f9b03fc6700 1 --2- 172.21.15.7:0/2532354577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9afc06b400 0x7f9afc147780 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:47.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.384+0000 7f9b03fc6700 1 --2- 172.21.15.7:0/2532354577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9afc06e870 0x7f9afc139680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:47.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.384+0000 7f9b03fc6700 1 -- 172.21.15.7:0/2532354577 >> 172.21.15.7:0/2532354577 conn(0x7f9afc0cdc30 msgr2=0x7f9afc0c9490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:47.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.384+0000 7f9b03fc6700 1 -- 172.21.15.7:0/2532354577 shutdown_connections 2024-01-27T02:43:47.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.384+0000 7f9b03fc6700 1 -- 172.21.15.7:0/2532354577 wait complete. 2024-01-27T02:43:47.386 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 94 2024-01-27T02:43:47.398 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2024-01-27T02:43:47.546 INFO:teuthology.orchestra.run.smithi188.stdout:{"cluster_fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","whoami":0,"id":7702,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":17.353081799000002,"mdsmap_epoch":94,"osdmap_epoch":182,"osdmap_epoch_barrier":182,"uptime":23.327599476} 2024-01-27T02:43:47.546 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", "fs_name": "cephfs", "id": 7702, "mdsmap_epoch": 94, "osdmap_epoch": 182, "osdmap_epoch_barrier": 182, "rank_uptime": 17.353081799, "state": "up:active", "uptime": 23.327599476, "want_state": "up:active", "whoami": 0 } 2024-01-27T02:43:47.546 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-01-27T02:43:47.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:43:47.547 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-01-27T02:43:47.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:43:47.569 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-01-27T02:43:47.638 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:43:47.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.820+0000 7f8f4d900700 1 Processor -- start 2024-01-27T02:43:47.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.820+0000 7f8f4d900700 1 -- start start 2024-01-27T02:43:47.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.821+0000 7f8f4d900700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8f480c3ae0 0x7f8f480cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:47.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.821+0000 7f8f4d900700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f48158070 0x7f8f480cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:47.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.821+0000 7f8f4d900700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f480ca020 con 0x7f8f48157ca0 2024-01-27T02:43:47.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.821+0000 7f8f4d900700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f480c5210 con 0x7f8f480c3ae0 2024-01-27T02:43:47.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.821+0000 7f8f4d900700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f480c54f0 con 0x7f8f48158070 2024-01-27T02:43:47.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.821+0000 7f8f46ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8f48157ca0 0x7f8f480c4a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52750/0 (socket says 172.21.15.7:52750) 2024-01-27T02:43:47.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.821+0000 7f8f46ffd700 1 -- 172.21.15.7:0/2177900729 learned_addr learned my addr 172.21.15.7:0/2177900729 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:47.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.822+0000 7f8f477fe700 1 --2- 172.21.15.7:0/2177900729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f48158070 0x7f8f480cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:47.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.822+0000 7f8f467fc700 1 --2- 172.21.15.7:0/2177900729 >> v2:172.21.15.188:3300/0 conn(0x7f8f480c3ae0 0x7f8f480cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.822+0000 7f8f45ffb700 1 -- 172.21.15.7:0/2177900729 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3978918849 0 0) 0x7f8f480ca020 con 0x7f8f48157ca0 2024-01-27T02:43:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.822+0000 7f8f45ffb700 1 -- 172.21.15.7:0/2177900729 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f30003570 con 0x7f8f48157ca0 2024-01-27T02:43:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.822+0000 7f8f45ffb700 1 -- 172.21.15.7:0/2177900729 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1951570082 0 0) 0x7f8f30003570 con 0x7f8f48157ca0 2024-01-27T02:43:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.822+0000 7f8f477fe700 1 -- 172.21.15.7:0/2177900729 >> v2:172.21.15.188:3300/0 conn(0x7f8f480c3ae0 msgr2=0x7f8f480cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.822+0000 7f8f477fe700 1 --2- 172.21.15.7:0/2177900729 >> v2:172.21.15.188:3300/0 conn(0x7f8f480c3ae0 0x7f8f480cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.822+0000 7f8f477fe700 1 -- 172.21.15.7:0/2177900729 >> v1:172.21.15.7:6789/0 conn(0x7f8f48157ca0 legacy=0x7f8f480c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.822+0000 7f8f477fe700 1 -- 172.21.15.7:0/2177900729 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f480c57d0 con 0x7f8f48158070 2024-01-27T02:43:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.822+0000 7f8f467fc700 1 --2- 172.21.15.7:0/2177900729 >> v2:172.21.15.188:3300/0 conn(0x7f8f480c3ae0 0x7f8f480cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:43:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.822+0000 7f8f477fe700 1 --2- 172.21.15.7:0/2177900729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f48158070 0x7f8f480cda20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009e60 tx=0x7f8f40009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c61758d13535fb18 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.822+0000 7f8f45ffb700 1 -- 172.21.15.7:0/2177900729 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4000e070 con 0x7f8f48158070 2024-01-27T02:43:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.823+0000 7f8f45ffb700 1 -- 172.21.15.7:0/2177900729 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4000b7a0 con 0x7f8f48158070 2024-01-27T02:43:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.823+0000 7f8f45ffb700 1 -- 172.21.15.7:0/2177900729 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f38001470 con 0x7f8f48158070 2024-01-27T02:43:47.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.823+0000 7f8f4d900700 1 -- 172.21.15.7:0/2177900729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f48158070 msgr2=0x7f8f480cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:47.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.823+0000 7f8f4d900700 1 --2- 172.21.15.7:0/2177900729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f48158070 0x7f8f480cda20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009e60 tx=0x7f8f40009780 comp rx=0 tx=0).stop 2024-01-27T02:43:47.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.823+0000 7f8f4d900700 1 -- 172.21.15.7:0/2177900729 shutdown_connections 2024-01-27T02:43:47.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.823+0000 7f8f4d900700 1 --2- 172.21.15.7:0/2177900729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f48158070 0x7f8f480cda20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:47.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.823+0000 7f8f4d900700 1 --2- 172.21.15.7:0/2177900729 >> v2:172.21.15.188:3300/0 conn(0x7f8f480c3ae0 0x7f8f480cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:47.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.823+0000 7f8f4d900700 1 -- 172.21.15.7:0/2177900729 >> 172.21.15.7:0/2177900729 conn(0x7f8f48098020 msgr2=0x7f8f480711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:47.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.823+0000 7f8f4d900700 1 -- 172.21.15.7:0/2177900729 shutdown_connections 2024-01-27T02:43:47.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.823+0000 7f8f4d900700 1 -- 172.21.15.7:0/2177900729 wait complete. 2024-01-27T02:43:47.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.824+0000 7f8f4d900700 1 Processor -- start 2024-01-27T02:43:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.824+0000 7f8f4d900700 1 -- start start 2024-01-27T02:43:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.825+0000 7f8f4d900700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f480c3ae0 0x7f8f4813a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.825+0000 7f8f4d900700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f48157ca0 0x7f8f48167e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.825+0000 7f8f4d900700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f48158070 0x7f8f481547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.825+0000 7f8f4d900700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4813ac50 con 0x7f8f48157ca0 2024-01-27T02:43:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.825+0000 7f8f4d900700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f480c5e90 con 0x7f8f48158070 2024-01-27T02:43:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.825+0000 7f8f4d900700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f480c6170 con 0x7f8f480c3ae0 2024-01-27T02:43:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.825+0000 7f8f467fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f48157ca0 0x7f8f48167e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.825+0000 7f8f467fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f48157ca0 0x7f8f48167e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48490/0 (socket says 172.21.15.7:48490) 2024-01-27T02:43:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.825+0000 7f8f467fc700 1 -- 172.21.15.7:0/3149001817 learned_addr learned my addr 172.21.15.7:0/3149001817 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.825+0000 7f8f477fe700 1 --2- 172.21.15.7:0/3149001817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f48158070 0x7f8f481547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.825+0000 7f8f46ffd700 1 --2- 172.21.15.7:0/3149001817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f480c3ae0 0x7f8f4813a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.826+0000 7f8f467fc700 1 -- 172.21.15.7:0/3149001817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f480c3ae0 msgr2=0x7f8f4813a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.826+0000 7f8f467fc700 1 --2- 172.21.15.7:0/3149001817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f480c3ae0 0x7f8f4813a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:47.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.826+0000 7f8f467fc700 1 -- 172.21.15.7:0/3149001817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f48158070 msgr2=0x7f8f481547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:47.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.826+0000 7f8f467fc700 1 --2- 172.21.15.7:0/3149001817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f48158070 0x7f8f481547a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:47.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.826+0000 7f8f467fc700 1 -- 172.21.15.7:0/3149001817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c009420 con 0x7f8f48157ca0 2024-01-27T02:43:47.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.826+0000 7f8f467fc700 1 --2- 172.21.15.7:0/3149001817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f48157ca0 0x7f8f48167e20 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00ec80 tx=0x7f8f3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:47.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.826+0000 7f8f2effd700 1 -- 172.21.15.7:0/3149001817 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c00c850 con 0x7f8f48157ca0 2024-01-27T02:43:47.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.826+0000 7f8f2effd700 1 -- 172.21.15.7:0/3149001817 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c00c9b0 con 0x7f8f48157ca0 2024-01-27T02:43:47.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.826+0000 7f8f2effd700 1 -- 172.21.15.7:0/3149001817 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c00cd30 con 0x7f8f48157ca0 2024-01-27T02:43:47.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.826+0000 7f8f4d900700 1 -- 172.21.15.7:0/3149001817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f40009a20 con 0x7f8f48157ca0 2024-01-27T02:43:47.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.826+0000 7f8f4d900700 1 -- 172.21.15.7:0/3149001817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f480c6730 con 0x7f8f48157ca0 2024-01-27T02:43:47.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.827+0000 7f8f4d900700 1 -- 172.21.15.7:0/3149001817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4804c710 con 0x7f8f48157ca0 2024-01-27T02:43:47.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.828+0000 7f8f2effd700 1 -- 172.21.15.7:0/3149001817 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8f3c013680 con 0x7f8f48157ca0 2024-01-27T02:43:47.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.828+0000 7f8f2effd700 1 --2- 172.21.15.7:0/3149001817 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f34066f30 0x7f8f340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:47.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.828+0000 7f8f2effd700 1 -- 172.21.15.7:0/3149001817 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8f3c084770 con 0x7f8f48157ca0 2024-01-27T02:43:47.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.828+0000 7f8f2effd700 1 -- 172.21.15.7:0/3149001817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7f8f3406ea50 con 0x7f8f48157ca0 2024-01-27T02:43:47.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.828+0000 7f8f46ffd700 1 --2- 172.21.15.7:0/3149001817 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f34066f30 0x7f8f340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:47.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.829+0000 7f8f46ffd700 1 --2- 172.21.15.7:0/3149001817 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f34066f30 0x7f8f340693e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8f380011e0 tx=0x7f8f38001aa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:47.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:47.833+0000 7f8f2effd700 1 -- 172.21.15.7:0/3149001817 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f3c055470 con 0x7f8f48157ca0 2024-01-27T02:43:48.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:48.035+0000 7f8f4d900700 1 -- 172.21.15.7:0/3149001817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f8f480c9bb0 con 0x7f8f48157ca0 2024-01-27T02:43:48.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:48.036+0000 7f8f2effd700 1 -- 172.21.15.7:0/3149001817 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v182) v1 ==== 81+0+32253 (secure 0 0 0) 0x7f8f3c055000 con 0x7f8f48157ca0 2024-01-27T02:43:48.037 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:48.037 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:43:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 182, 2024-01-27T02:43:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:43:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:43:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:43:44.933767+0000", 2024-01-27T02:43:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:43:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:43:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:43:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:43:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:43:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:43:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:43:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:43:48.038 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:43:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:43:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:43:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:43:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:43:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:43:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 23, 2024-01-27T02:43:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:43:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:43:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:43:48.039 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:43:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:43:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:43:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:43:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:43:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:43:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:43:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:43:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:43:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:43:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:43:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:43:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:43:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:43:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:43:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:43:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:43:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:43:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:43:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:43:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:43:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:43:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:43:48.041 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:43:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:43:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:43:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:43:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:43:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:43:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "106", 2024-01-27T02:43:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:43:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:43:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:43:48.042 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:43:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:43:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:43:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:43:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:43:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:43:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:43:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:43:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:43:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:43:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:43:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:43:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:43:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:43:48.043 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:43:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:43:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:43:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:43:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:43:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:43:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:43:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:43:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:43:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:43:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:43:48.044 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:43:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:43:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:43:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:43:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:43:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:43:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:43:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:43:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:43:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:43:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:43:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:43:48.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:43:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:43:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 22, 2024-01-27T02:43:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:43:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:43:27.492968+0000", 2024-01-27T02:43:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:43:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:43:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:43:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:43:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:43:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:43:48.046 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:43:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:43:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:43:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:43:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:43:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:43:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:43:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:43:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:43:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:43:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:43:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:43:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:43:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:43:48.047 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:43:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:43:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:43:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:43:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "179", 2024-01-27T02:43:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:43:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:43:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:43:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:43:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:43:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:43:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:43:48.048 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:43:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:43:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:43:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:43:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:43:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:43:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:43:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:43:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:43:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:43:48.049 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:43:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:43:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:43:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:43:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:43:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:43:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:43:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:43:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:43:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:43:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:43:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:43:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:43:48.050 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:43:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:43:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:43:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:43:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:43:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:43:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:43:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:43:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:43:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:43:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:43:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:43:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:43:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:43:48.051 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 23, 2024-01-27T02:43:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:43:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:43:28.579977+0000", 2024-01-27T02:43:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1027, 2024-01-27T02:43:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full,full_quota", 2024-01-27T02:43:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:43:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:43:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:43:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:43:48.052 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:43:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:43:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:43:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:43:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:43:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:43:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:43:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:43:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:43:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:43:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:43:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:43:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:43:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:43:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:43:48.053 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:43:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:43:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:43:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "182", 2024-01-27T02:43:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:43:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:43:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:43:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:43:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:43:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:43:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:43:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:43:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:43:48.054 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-27T02:43:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:43:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:43:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:43:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:43:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:43:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:43:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:43:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:43:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:43:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:43:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:43:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:43:48.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:43:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:43:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:43:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:43:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:43:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:43:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:43:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:43:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:43:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:43:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:43:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:43:48.056 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:43:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:43:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:43:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:43:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:43:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:43:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:43:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:43:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:43:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:43:48.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:43:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:48.058 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:43:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:43:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.059 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:43:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.060 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:43:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:43:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:43:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:43:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.062 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:43:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:43:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:43:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:43:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:43:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:43:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:48.064 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:43:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:43:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.065 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:43:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:43:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:43:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:43:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:43:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:43:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:43:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:43:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:43:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:43:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:43:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:48.069 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:43:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:43:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:43:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:43:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:48.072 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.073 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:43:48.073 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:48.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.074 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:43:48.074 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:48.074 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.074 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.074 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:43:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:43:48.075 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:43:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:43:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:43:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:43:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.076 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:43:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:48.077 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:43:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:43:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:48.078 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:43:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:43:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:48.079 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:43:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:43:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:43:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:43:48.081 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:43:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:43:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:43:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:43:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.082 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:43:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:48.083 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:43:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:43:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:43:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:43:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.085 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:43:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:43:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:43:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:43:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.087 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:43:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:43:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:43:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:43:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:43:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:48.088 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:48.089 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:43:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:43:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:48.090 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:43:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:43:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.091 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:43:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:43:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.092 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:43:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:43:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:48.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:43:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:43:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:43:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:43:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:43:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:48.095 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:43:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:43:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:43:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:43:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:48.097 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:43:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:43:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:48.098 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:43:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:43:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:43:48.099 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:43:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:43:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:43:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:43:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:48.100 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:43:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:43:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:43:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:43:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:43:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:43:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:48.104 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:43:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:43:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:43:48.105 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:43:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:43:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:43:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:43:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:43:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:48.106 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:43:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:43:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:48.107 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:43:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:43:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:43:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.108 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:43:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:43:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:43:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:48.109 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:43:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:43:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:43:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:48.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:43:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:43:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:43:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:48.111 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:43:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:43:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:43:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:43:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:43:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:43:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:43:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:43:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:43:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:43:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:43:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:43:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:43:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:43:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:43:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:43:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:43:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:43:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:43:48.113 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:43:48.114 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:43:48.114 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:43:48.114 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:43:48.114 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:43:48.114 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:43:48.114 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:48.114 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:43:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:48.038+0000 7f8f4d900700 1 -- 172.21.15.7:0/3149001817 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f34066f30 msgr2=0x7f8f340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:48.038+0000 7f8f4d900700 1 --2- 172.21.15.7:0/3149001817 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f34066f30 0x7f8f340693e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8f380011e0 tx=0x7f8f38001aa0 comp rx=0 tx=0).stop 2024-01-27T02:43:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:48.038+0000 7f8f4d900700 1 -- 172.21.15.7:0/3149001817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f48157ca0 msgr2=0x7f8f48167e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:48.038+0000 7f8f4d900700 1 --2- 172.21.15.7:0/3149001817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f48157ca0 0x7f8f48167e20 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00ec80 tx=0x7f8f3c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:48.038+0000 7f8f4d900700 1 -- 172.21.15.7:0/3149001817 shutdown_connections 2024-01-27T02:43:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:48.038+0000 7f8f4d900700 1 --2- 172.21.15.7:0/3149001817 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8f34066f30 0x7f8f340693e0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:48.038+0000 7f8f4d900700 1 --2- 172.21.15.7:0/3149001817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f48158070 0x7f8f481547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:48.038+0000 7f8f4d900700 1 --2- 172.21.15.7:0/3149001817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f480c3ae0 0x7f8f4813a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:48.038+0000 7f8f4d900700 1 --2- 172.21.15.7:0/3149001817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f48157ca0 0x7f8f48167e20 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:48.038+0000 7f8f4d900700 1 -- 172.21.15.7:0/3149001817 >> 172.21.15.7:0/3149001817 conn(0x7f8f48098020 msgr2=0x7f8f480b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:48.039+0000 7f8f4d900700 1 -- 172.21.15.7:0/3149001817 shutdown_connections 2024-01-27T02:43:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:48.039+0000 7f8f4d900700 1 -- 172.21.15.7:0/3149001817 wait complete. 2024-01-27T02:43:48.117 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-27T02:43:53.118 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:43:53.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.259+0000 7fa3daf14700 1 Processor -- start 2024-01-27T02:43:53.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.259+0000 7fa3daf14700 1 -- start start 2024-01-27T02:43:53.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.260+0000 7fa3daf14700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa3d40c4cc0 0x7fa3d40cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:53.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.260+0000 7fa3daf14700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3d40ca540 0x7fa3d4163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:53.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.260+0000 7fa3daf14700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3d40ce5e0 con 0x7fa3d415a100 2024-01-27T02:43:53.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.260+0000 7fa3daf14700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d40cd9d0 con 0x7fa3d40c4cc0 2024-01-27T02:43:53.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.260+0000 7fa3daf14700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d40cdcb0 con 0x7fa3d40ca540 2024-01-27T02:43:53.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.260+0000 7fa3d3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa3d40c4cc0 0x7fa3d40cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:53.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.260+0000 7fa3d3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa3d40c4cc0 0x7fa3d40cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36526/0 (socket says 172.21.15.7:36526) 2024-01-27T02:43:53.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.260+0000 7fa3d3fff700 1 -- 172.21.15.7:0/4019438635 learned_addr learned my addr 172.21.15.7:0/4019438635 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:53.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.260+0000 7fa3d94b1700 1 --2- 172.21.15.7:0/4019438635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3d40ca540 0x7fa3d4163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:53.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.260+0000 7fa3d3fff700 1 -- 172.21.15.7:0/4019438635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3d40ca540 msgr2=0x7fa3d4163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:53.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.260+0000 7fa3d3fff700 1 --2- 172.21.15.7:0/4019438635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3d40ca540 0x7fa3d4163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:53.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.260+0000 7fa3d3fff700 1 -- 172.21.15.7:0/4019438635 >> v1:172.21.15.188:6789/0 conn(0x7fa3d415a100 legacy=0x7fa3d40c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:53.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.260+0000 7fa3d3fff700 1 -- 172.21.15.7:0/4019438635 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d40cdf90 con 0x7fa3d40c4cc0 2024-01-27T02:43:53.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.260+0000 7fa3d3fff700 1 --2- 172.21.15.7:0/4019438635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d40c4cc0 0x7fa3d40cd2f0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80099b0 tx=0x7fa3c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be555868b10434c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:53.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.260+0000 7fa3d37fe700 1 -- 172.21.15.7:0/4019438635 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8009ea0 con 0x7fa3d40c4cc0 2024-01-27T02:43:53.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.261+0000 7fa3d37fe700 1 -- 172.21.15.7:0/4019438635 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c800b860 con 0x7fa3d40c4cc0 2024-01-27T02:43:53.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.261+0000 7fa3d37fe700 1 -- 172.21.15.7:0/4019438635 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c800bc00 con 0x7fa3d40c4cc0 2024-01-27T02:43:53.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.261+0000 7fa3daf14700 1 -- 172.21.15.7:0/4019438635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d40c4cc0 msgr2=0x7fa3d40cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:53.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.261+0000 7fa3daf14700 1 --2- 172.21.15.7:0/4019438635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d40c4cc0 0x7fa3d40cd2f0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fa3c80099b0 tx=0x7fa3c8009250 comp rx=0 tx=0).stop 2024-01-27T02:43:53.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.261+0000 7fa3daf14700 1 -- 172.21.15.7:0/4019438635 shutdown_connections 2024-01-27T02:43:53.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.261+0000 7fa3daf14700 1 --2- 172.21.15.7:0/4019438635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3d40ca540 0x7fa3d4163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:53.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.261+0000 7fa3daf14700 1 --2- 172.21.15.7:0/4019438635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d40c4cc0 0x7fa3d40cd2f0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:53.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.261+0000 7fa3daf14700 1 -- 172.21.15.7:0/4019438635 >> 172.21.15.7:0/4019438635 conn(0x7fa3d4098020 msgr2=0x7fa3d40711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:53.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.261+0000 7fa3daf14700 1 -- 172.21.15.7:0/4019438635 shutdown_connections 2024-01-27T02:43:53.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.261+0000 7fa3daf14700 1 -- 172.21.15.7:0/4019438635 wait complete. 2024-01-27T02:43:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.262+0000 7fa3daf14700 1 Processor -- start 2024-01-27T02:43:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.263+0000 7fa3daf14700 1 -- start start 2024-01-27T02:43:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.263+0000 7fa3daf14700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3d40c4cc0 0x7fa3d4152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.263+0000 7fa3daf14700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d40ca540 0x7fa3d40b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.263+0000 7fa3daf14700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3d415a100 0x7fa3d40b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.263+0000 7fa3daf14700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d40c6900 con 0x7fa3d40ca540 2024-01-27T02:43:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.263+0000 7fa3daf14700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d40b2a30 con 0x7fa3d40c4cc0 2024-01-27T02:43:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.263+0000 7fa3daf14700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d40b2d10 con 0x7fa3d415a100 2024-01-27T02:43:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.263+0000 7fa3d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d40ca540 0x7fa3d40b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.263+0000 7fa3d8cb0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3d40c4cc0 0x7fa3d4152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.263+0000 7fa3d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d40ca540 0x7fa3d40b3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36542/0 (socket says 172.21.15.7:36542) 2024-01-27T02:43:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.264+0000 7fa3d3fff700 1 -- 172.21.15.7:0/1091479455 learned_addr learned my addr 172.21.15.7:0/1091479455 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.264+0000 7fa3d94b1700 1 --2- 172.21.15.7:0/1091479455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3d415a100 0x7fa3d40b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.264+0000 7fa3d3fff700 1 -- 172.21.15.7:0/1091479455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3d415a100 msgr2=0x7fa3d40b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.264+0000 7fa3d3fff700 1 --2- 172.21.15.7:0/1091479455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3d415a100 0x7fa3d40b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.264+0000 7fa3d3fff700 1 -- 172.21.15.7:0/1091479455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3d40c4cc0 msgr2=0x7fa3d4152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.264+0000 7fa3d3fff700 1 --2- 172.21.15.7:0/1091479455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3d40c4cc0 0x7fa3d4152710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.264+0000 7fa3d3fff700 1 -- 172.21.15.7:0/1091479455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c80094f0 con 0x7fa3d40ca540 2024-01-27T02:43:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.264+0000 7fa3d3fff700 1 --2- 172.21.15.7:0/1091479455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d40ca540 0x7fa3d40b3c40 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fa3d406ea80 tx=0x7fa3c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.264+0000 7fa3d0ff9700 1 -- 172.21.15.7:0/1091479455 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c800b9d0 con 0x7fa3d40ca540 2024-01-27T02:43:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.264+0000 7fa3d0ff9700 1 -- 172.21.15.7:0/1091479455 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c800bb50 con 0x7fa3d40ca540 2024-01-27T02:43:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.264+0000 7fa3daf14700 1 -- 172.21.15.7:0/1091479455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d40b2ff0 con 0x7fa3d40ca540 2024-01-27T02:43:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.264+0000 7fa3d0ff9700 1 -- 172.21.15.7:0/1091479455 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8025670 con 0x7fa3d40ca540 2024-01-27T02:43:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.264+0000 7fa3daf14700 1 -- 172.21.15.7:0/1091479455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d4152c50 con 0x7fa3d40ca540 2024-01-27T02:43:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.265+0000 7fa3daf14700 1 -- 172.21.15.7:0/1091479455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d40b3510 con 0x7fa3d40ca540 2024-01-27T02:43:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.266+0000 7fa3d0ff9700 1 -- 172.21.15.7:0/1091479455 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa3c8023070 con 0x7fa3d40ca540 2024-01-27T02:43:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.266+0000 7fa3d0ff9700 1 --2- 172.21.15.7:0/1091479455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa3c0066f30 0x7fa3c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.266+0000 7fa3d0ff9700 1 -- 172.21.15.7:0/1091479455 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa3c8085a30 con 0x7fa3d40ca540 2024-01-27T02:43:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.266+0000 7fa3d0ff9700 1 -- 172.21.15.7:0/1091479455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=183}) v3 -- 0x7fa3c006ea50 con 0x7fa3d40ca540 2024-01-27T02:43:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.266+0000 7fa3d8cb0700 1 --2- 172.21.15.7:0/1091479455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa3c0066f30 0x7fa3c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:53.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.267+0000 7fa3d8cb0700 1 --2- 172.21.15.7:0/1091479455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa3c0066f30 0x7fa3c00693e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4003a80 tx=0x7fa3c4007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:53.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.271+0000 7fa3d0ff9700 1 -- 172.21.15.7:0/1091479455 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3d40b3510 con 0x7fa3d40ca540 2024-01-27T02:43:53.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.475+0000 7fa3daf14700 1 -- 172.21.15.7:0/1091479455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fa3d40b3510 con 0x7fa3d40ca540 2024-01-27T02:43:53.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.476+0000 7fa3d0ff9700 1 -- 172.21.15.7:0/1091479455 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v182) v1 ==== 81+0+32253 (secure 0 0 0) 0x7fa3d40b3510 con 0x7fa3d40ca540 2024-01-27T02:43:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:53.477 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:43:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 182, 2024-01-27T02:43:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:43:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:43:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:43:44.933767+0000", 2024-01-27T02:43:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:43:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:43:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:43:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:43:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:43:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:43:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:43:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:43:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:43:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:43:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:43:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:43:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:43:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:43:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 23, 2024-01-27T02:43:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:43:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:43:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:43:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:43:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:43:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:43:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:43:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:43:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:43:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:43:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:43:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:43:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:43:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:43:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:43:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:43:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:43:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:43:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:43:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:43:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:43:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:43:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:43:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:43:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:43:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:43:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:43:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:43:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:43:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:43:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:43:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:43:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "106", 2024-01-27T02:43:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:43:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:43:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:43:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:43:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:43:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:43:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:43:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:43:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:43:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:43:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:43:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:43:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:43:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:43:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:43:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:43:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:43:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:43:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:43:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:43:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:43:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:43:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:43:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:43:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:43:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:43:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:43:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:43:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:43:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:43:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:43:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:43:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:43:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:43:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:43:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:43:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:43:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:43:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:43:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:43:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:43:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:43:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 22, 2024-01-27T02:43:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:43:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:43:27.492968+0000", 2024-01-27T02:43:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:43:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:43:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:43:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:43:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:43:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:43:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:43:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:43:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:43:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:43:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:43:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:43:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:43:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:43:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:43:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:43:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:43:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:43:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:43:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:43:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:43:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:43:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:43:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:43:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "179", 2024-01-27T02:43:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:43:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:43:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:43:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:43:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:43:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:43:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:43:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:43:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:43:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:43:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:43:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:43:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:43:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:43:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:43:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:43:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:43:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:43:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:43:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:43:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:43:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:43:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:43:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:43:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:43:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:43:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:43:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:43:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:43:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:43:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:43:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:43:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:43:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:43:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:43:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:43:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:43:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:43:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:43:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:43:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:43:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:43:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:43:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:43:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 23, 2024-01-27T02:43:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:43:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:43:28.579977+0000", 2024-01-27T02:43:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1027, 2024-01-27T02:43:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full,full_quota", 2024-01-27T02:43:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:43:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:43:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:43:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:43:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:43:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:43:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:43:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:43:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:43:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:43:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:43:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:43:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:43:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:43:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:43:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:43:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:43:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:43:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:43:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:43:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:43:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:43:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "182", 2024-01-27T02:43:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:43:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:43:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:43:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:43:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:43:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:43:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:43:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:43:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:43:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-27T02:43:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:43:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:43:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:43:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:43:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:43:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:43:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:43:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:43:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:43:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:43:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:43:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:43:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:43:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:43:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:43:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:43:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:43:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:43:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:43:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:43:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:43:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:43:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:43:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:43:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:43:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:43:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:43:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:43:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:43:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:43:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:43:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:43:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:43:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:43:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:43:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:43:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:43:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:43:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:43:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:43:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:43:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:43:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:43:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:43:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:43:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:43:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:43:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:43:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:43:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:43:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:43:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:43:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:43:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:43:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:43:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:43:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:43:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:43:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:43:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:43:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:43:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:43:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:43:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:43:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:43:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:43:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:43:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:43:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:43:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:43:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:43:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:43:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:43:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:43:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:43:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:43:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:43:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:43:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:43:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:43:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:43:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:43:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:43:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:43:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:43:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:43:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:43:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:43:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:43:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:43:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:43:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:43:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:43:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:43:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:43:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:43:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:43:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:43:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:43:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:43:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:43:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:43:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:43:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:43:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:43:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:43:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:43:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:43:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:43:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:43:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:43:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:43:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:43:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:43:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:43:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:43:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:43:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:43:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:43:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:43:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:43:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:43:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:43:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:43:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:43:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:43:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:43:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:43:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:43:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:43:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:43:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:43:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:43:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:43:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:43:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:43:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:43:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:43:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:43:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:43:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:43:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:43:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:43:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:43:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:43:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:43:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:43:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:43:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:43:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:43:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:43:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:43:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:43:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:43:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:43:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:43:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:43:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:43:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:43:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:43:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:43:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:43:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:43:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:43:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:43:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:43:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:43:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:43:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:43:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:43:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:43:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:43:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:43:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:43:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:43:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:43:53.550 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:43:53.550 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:43:53.550 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:43:53.550 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:53.550 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:43:53.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.478+0000 7fa3daf14700 1 -- 172.21.15.7:0/1091479455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa3c0066f30 msgr2=0x7fa3c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:53.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.478+0000 7fa3daf14700 1 --2- 172.21.15.7:0/1091479455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa3c0066f30 0x7fa3c00693e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4003a80 tx=0x7fa3c4007ec0 comp rx=0 tx=0).stop 2024-01-27T02:43:53.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.478+0000 7fa3daf14700 1 -- 172.21.15.7:0/1091479455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d40ca540 msgr2=0x7fa3d40b3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:53.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.478+0000 7fa3daf14700 1 --2- 172.21.15.7:0/1091479455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d40ca540 0x7fa3d40b3c40 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fa3d406ea80 tx=0x7fa3c8017000 comp rx=0 tx=0).stop 2024-01-27T02:43:53.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.478+0000 7fa3daf14700 1 -- 172.21.15.7:0/1091479455 shutdown_connections 2024-01-27T02:43:53.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.478+0000 7fa3daf14700 1 --2- 172.21.15.7:0/1091479455 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa3c0066f30 0x7fa3c00693e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:53.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.478+0000 7fa3daf14700 1 --2- 172.21.15.7:0/1091479455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3d415a100 0x7fa3d40b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:53.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.478+0000 7fa3daf14700 1 --2- 172.21.15.7:0/1091479455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3d40c4cc0 0x7fa3d4152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:53.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.478+0000 7fa3daf14700 1 --2- 172.21.15.7:0/1091479455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d40ca540 0x7fa3d40b3c40 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:53.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.478+0000 7fa3daf14700 1 -- 172.21.15.7:0/1091479455 >> 172.21.15.7:0/1091479455 conn(0x7fa3d4098020 msgr2=0x7fa3d4154650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:53.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.478+0000 7fa3daf14700 1 -- 172.21.15.7:0/1091479455 shutdown_connections 2024-01-27T02:43:53.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:53.478+0000 7fa3daf14700 1 -- 172.21.15.7:0/1091479455 wait complete. 2024-01-27T02:43:53.553 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-27T02:43:58.554 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:43:58.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.694+0000 7f7977039700 1 Processor -- start 2024-01-27T02:43:58.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.694+0000 7f7977039700 1 -- start start 2024-01-27T02:43:58.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.694+0000 7f7977039700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f79701404e0 0x7f79700cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:58.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.694+0000 7f7977039700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79700b09f0 0x7f79700cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:58.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.695+0000 7f7977039700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79700cce30 con 0x7f79701408b0 2024-01-27T02:43:58.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.695+0000 7f7977039700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f797006b3b0 con 0x7f79701404e0 2024-01-27T02:43:58.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.695+0000 7f7977039700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79700cc3e0 con 0x7f79700b09f0 2024-01-27T02:43:58.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.695+0000 7f7974dd5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f79701404e0 0x7f79700cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:58.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.695+0000 7f7974dd5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f79701404e0 0x7f79700cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36556/0 (socket says 172.21.15.7:36556) 2024-01-27T02:43:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.695+0000 7f7974dd5700 1 -- 172.21.15.7:0/347463271 learned_addr learned my addr 172.21.15.7:0/347463271 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.695+0000 7f796ffff700 1 --2- 172.21.15.7:0/347463271 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79700b09f0 0x7f79700cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.695+0000 7f7974dd5700 1 -- 172.21.15.7:0/347463271 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79700b09f0 msgr2=0x7f79700cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.695+0000 7f7974dd5700 1 --2- 172.21.15.7:0/347463271 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79700b09f0 0x7f79700cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.695+0000 7f7974dd5700 1 -- 172.21.15.7:0/347463271 >> v1:172.21.15.188:6789/0 conn(0x7f79701408b0 legacy=0x7f797006e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.695+0000 7f7974dd5700 1 -- 172.21.15.7:0/347463271 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79700cc6c0 con 0x7f79701404e0 2024-01-27T02:43:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.695+0000 7f7974dd5700 1 --2- 172.21.15.7:0/347463271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79701404e0 0x7f79700cb800 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f79640099b0 tx=0x7f7964009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0aabfa8d17258cb server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.695+0000 7f796f7fe700 1 -- 172.21.15.7:0/347463271 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7964009ea0 con 0x7f79701404e0 2024-01-27T02:43:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.695+0000 7f796f7fe700 1 -- 172.21.15.7:0/347463271 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796400b860 con 0x7f79701404e0 2024-01-27T02:43:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.696+0000 7f796f7fe700 1 -- 172.21.15.7:0/347463271 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796400bc00 con 0x7f79701404e0 2024-01-27T02:43:58.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.696+0000 7f7977039700 1 -- 172.21.15.7:0/347463271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79701404e0 msgr2=0x7f79700cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:58.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.696+0000 7f7977039700 1 --2- 172.21.15.7:0/347463271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79701404e0 0x7f79700cb800 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f79640099b0 tx=0x7f7964009250 comp rx=0 tx=0).stop 2024-01-27T02:43:58.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.696+0000 7f7977039700 1 -- 172.21.15.7:0/347463271 shutdown_connections 2024-01-27T02:43:58.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.696+0000 7f7977039700 1 --2- 172.21.15.7:0/347463271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79701404e0 0x7f79700cb800 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:58.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.696+0000 7f7977039700 1 --2- 172.21.15.7:0/347463271 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79700b09f0 0x7f79700cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:58.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.696+0000 7f7977039700 1 -- 172.21.15.7:0/347463271 >> 172.21.15.7:0/347463271 conn(0x7f79700cdc30 msgr2=0x7f79700ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:58.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.696+0000 7f7977039700 1 -- 172.21.15.7:0/347463271 shutdown_connections 2024-01-27T02:43:58.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.696+0000 7f7977039700 1 -- 172.21.15.7:0/347463271 wait complete. 2024-01-27T02:43:58.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.697+0000 7f7977039700 1 Processor -- start 2024-01-27T02:43:58.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.698+0000 7f7977039700 1 -- start start 2024-01-27T02:43:58.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.698+0000 7f7977039700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79700b09f0 0x7f79701315a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:58.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.698+0000 7f7977039700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79701404e0 0x7f79701596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:58.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.698+0000 7f7977039700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79701408b0 0x7f79700c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:58.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.698+0000 7f7977039700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7970159260 con 0x7f79701404e0 2024-01-27T02:43:58.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.698+0000 7f7977039700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7970131ae0 con 0x7f79700b09f0 2024-01-27T02:43:58.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.698+0000 7f7977039700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7970158650 con 0x7f79701408b0 2024-01-27T02:43:58.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.698+0000 7f796ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79701404e0 0x7f79701596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.698+0000 7f796ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79701404e0 0x7f79701596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36558/0 (socket says 172.21.15.7:36558) 2024-01-27T02:43:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.699+0000 7f796ffff700 1 -- 172.21.15.7:0/2317757251 learned_addr learned my addr 172.21.15.7:0/2317757251 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.699+0000 7f79755d6700 1 --2- 172.21.15.7:0/2317757251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79701408b0 0x7f79700c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.699+0000 7f7974dd5700 1 --2- 172.21.15.7:0/2317757251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79700b09f0 0x7f79701315a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.699+0000 7f796ffff700 1 -- 172.21.15.7:0/2317757251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79701408b0 msgr2=0x7f79700c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.699+0000 7f796ffff700 1 --2- 172.21.15.7:0/2317757251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79701408b0 0x7f79700c6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.699+0000 7f796ffff700 1 -- 172.21.15.7:0/2317757251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79700b09f0 msgr2=0x7f79701315a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.699+0000 7f796ffff700 1 --2- 172.21.15.7:0/2317757251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79700b09f0 0x7f79701315a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.699+0000 7f796ffff700 1 -- 172.21.15.7:0/2317757251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79640094f0 con 0x7f79701404e0 2024-01-27T02:43:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.699+0000 7f796ffff700 1 --2- 172.21.15.7:0/2317757251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79701404e0 0x7f79701596f0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f795c00ec80 tx=0x7f795c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.699+0000 7f796cff9700 1 -- 172.21.15.7:0/2317757251 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c00c850 con 0x7f79701404e0 2024-01-27T02:43:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.699+0000 7f796cff9700 1 -- 172.21.15.7:0/2317757251 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795c00c9b0 con 0x7f79701404e0 2024-01-27T02:43:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.699+0000 7f7977039700 1 -- 172.21.15.7:0/2317757251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7970158930 con 0x7f79701404e0 2024-01-27T02:43:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.699+0000 7f796cff9700 1 -- 172.21.15.7:0/2317757251 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c00cd70 con 0x7f79701404e0 2024-01-27T02:43:58.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.699+0000 7f7977039700 1 -- 172.21.15.7:0/2317757251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7970158ef0 con 0x7f79701404e0 2024-01-27T02:43:58.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.700+0000 7f7977039700 1 -- 172.21.15.7:0/2317757251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7970159410 con 0x7f79701404e0 2024-01-27T02:43:58.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.701+0000 7f796cff9700 1 -- 172.21.15.7:0/2317757251 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f795c013680 con 0x7f79701404e0 2024-01-27T02:43:58.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.701+0000 7f796cff9700 1 --2- 172.21.15.7:0/2317757251 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7954066f30 0x7f79540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:58.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.701+0000 7f796cff9700 1 -- 172.21.15.7:0/2317757251 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f795c084700 con 0x7f79701404e0 2024-01-27T02:43:58.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.702+0000 7f7974dd5700 1 --2- 172.21.15.7:0/2317757251 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7954066f30 0x7f79540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:58.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.702+0000 7f7974dd5700 1 --2- 172.21.15.7:0/2317757251 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7954066f30 0x7f79540693e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f79640099b0 tx=0x7f7964017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:58.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.706+0000 7f796cff9700 1 -- 172.21.15.7:0/2317757251 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7970159410 con 0x7f79701404e0 2024-01-27T02:43:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.909+0000 7f7977039700 1 -- 172.21.15.7:0/2317757251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f7970159410 con 0x7f79701404e0 2024-01-27T02:43:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.910+0000 7f796cff9700 1 -- 172.21.15.7:0/2317757251 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v183) v1 ==== 81+0+32234 (secure 0 0 0) 0x7f7970159410 con 0x7f79701404e0 2024-01-27T02:43:58.911 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:58.911 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:43:58.911 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 183, 2024-01-27T02:43:58.912 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:43:58.912 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:43:58.912 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:43:54.936156+0000", 2024-01-27T02:43:58.912 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:43:58.912 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:43:58.912 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:43:58.912 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:43:58.912 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:43:58.912 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:43:58.912 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:43:58.912 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:43:58.912 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:43:58.912 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:58.913 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:43:58.913 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:43:58.913 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:43:58.913 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:43:58.913 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:43:58.913 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 23, 2024-01-27T02:43:58.913 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:43:58.913 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:43:58.913 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:43:58.913 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:43:58.913 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:43:58.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.913 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:43:58.913 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:43:58.914 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:43:58.914 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:43:58.914 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:43:58.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:43:58.914 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:43:58.914 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:43:58.914 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:43:58.914 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:43:58.914 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:43:58.914 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:43:58.914 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:43:58.914 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:43:58.914 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:43:58.914 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:43:58.915 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:43:58.915 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:43:58.915 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:43:58.915 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:43:58.915 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:43:58.915 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:43:58.915 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:43:58.915 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:43:58.915 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:43:58.915 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:43:58.915 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:43:58.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.915 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "106", 2024-01-27T02:43:58.915 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:43:58.916 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:43:58.916 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:43:58.916 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:43:58.916 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:43:58.916 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:43:58.916 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:43:58.916 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:43:58.916 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:43:58.916 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:43:58.916 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:43:58.916 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:43:58.916 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:43:58.916 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:43:58.916 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:43:58.917 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:43:58.917 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:43:58.917 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:43:58.917 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:43:58.917 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:43:58.917 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:43:58.917 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:43:58.917 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:43:58.917 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:43:58.917 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:43:58.917 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:43:58.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.917 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:43:58.917 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:43:58.917 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:43:58.918 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:43:58.918 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:43:58.918 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:43:58.918 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:43:58.918 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:43:58.918 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:43:58.918 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:43:58.918 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:43:58.918 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:43:58.918 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:43:58.918 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:43:58.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.918 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:43:58.918 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:43:58.919 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.919 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 22, 2024-01-27T02:43:58.919 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:43:58.919 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:43:27.492968+0000", 2024-01-27T02:43:58.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:43:58.920 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:43:58.920 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:43:58.920 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:43:58.920 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:43:58.920 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:43:58.920 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:43:58.920 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:43:58.920 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:43:58.920 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:43:58.920 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:43:58.920 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:43:58.920 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:43:58.920 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:43:58.920 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:43:58.921 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:43:58.921 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:43:58.921 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:43:58.921 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:43:58.921 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:43:58.921 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:43:58.921 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:43:58.921 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:43:58.921 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:43:58.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.921 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "179", 2024-01-27T02:43:58.921 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:43:58.921 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:43:58.921 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:43:58.922 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:43:58.922 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:43:58.922 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:43:58.922 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:43:58.922 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:43:58.922 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:43:58.922 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:43:58.922 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:43:58.922 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:43:58.922 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:43:58.922 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:43:58.922 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:43:58.922 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:43:58.922 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:43:58.923 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:43:58.923 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:43:58.923 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:43:58.923 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:43:58.923 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:43:58.923 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:43:58.923 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:43:58.923 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:43:58.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:43:58.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.923 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:43:58.923 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:43:58.923 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:43:58.923 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:43:58.923 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:43:58.924 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:43:58.924 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:43:58.924 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:43:58.924 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:43:58.924 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:43:58.924 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:43:58.924 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:43:58.924 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:43:58.924 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:43:58.924 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:43:58.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.924 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:43:58.924 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:43:58.925 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:43:58.925 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.925 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.925 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 23, 2024-01-27T02:43:58.925 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:43:58.925 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:43:28.579977+0000", 2024-01-27T02:43:58.925 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:43:58.925 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:43:58.925 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:43:58.925 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:43:58.925 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:43:58.925 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:43:58.925 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:43:58.926 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:43:58.926 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:43:58.926 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:43:58.926 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:43:58.926 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:43:58.926 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:43:58.926 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:43:58.926 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:43:58.926 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:43:58.926 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:43:58.926 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:43:58.926 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:43:58.926 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:43:58.926 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:43:58.926 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:43:58.927 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:43:58.927 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:43:58.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.927 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "183", 2024-01-27T02:43:58.927 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:43:58.927 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:43:58.927 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:43:58.927 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:43:58.927 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:43:58.927 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:43:58.927 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:43:58.927 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:43:58.927 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:43:58.927 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-27T02:43:58.927 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:43:58.928 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:43:58.928 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:43:58.928 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:43:58.928 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:43:58.928 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:43:58.928 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:43:58.928 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:43:58.928 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:43:58.928 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:43:58.928 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:43:58.928 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:43:58.928 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:43:58.928 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:43:58.928 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:43:58.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:43:58.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.929 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:43:58.929 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:43:58.929 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:43:58.929 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:43:58.929 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:43:58.929 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:43:58.929 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:43:58.929 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:43:58.929 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:43:58.930 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:43:58.930 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:43:58.930 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:43:58.930 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:43:58.930 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:43:58.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.930 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:43:58.930 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:43:58.930 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:43:58.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.931 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:58.931 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:43:58.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.931 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:43:58.931 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:58.931 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:58.931 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:58.931 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:58.931 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:58.931 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:58.931 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:58.931 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:58.932 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:58.932 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:58.932 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:58.932 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:58.932 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.932 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:43:58.932 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:58.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.932 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:43:58.932 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:58.933 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.933 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.933 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:58.933 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.933 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:43:58.933 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:58.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.933 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:43:58.933 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:58.934 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.934 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.934 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:58.934 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.934 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.934 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:43:58.934 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:58.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.934 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.934 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:43:58.934 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:58.934 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.935 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.935 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:58.935 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.935 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:43:58.935 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:58.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.935 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:43:58.935 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:58.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.935 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.936 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:43:58.936 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:43:58.936 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:43:58.936 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:43:58.936 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:58.936 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:58.936 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:58.936 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.936 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:43:58.936 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:58.936 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:58.937 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:58.937 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:58.937 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:58.937 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:58.937 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:58.937 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:58.937 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:58.937 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:58.937 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:58.937 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:58.937 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.937 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:43:58.937 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:58.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.938 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:43:58.938 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:58.938 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.938 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.938 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:58.938 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.938 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:43:58.938 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:58.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.939 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:43:58.939 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:58.939 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.939 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.939 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:58.939 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.939 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:43:58.939 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:58.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.940 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.940 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:43:58.940 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:58.940 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.940 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.940 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:58.940 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.940 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.940 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:43:58.940 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:58.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.941 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:43:58.941 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:58.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.941 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.941 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:43:58.941 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:43:58.941 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:43:58.941 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:43:58.941 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:58.941 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:58.941 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:58.942 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.942 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:43:58.942 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:58.942 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:58.942 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:58.942 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:58.942 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:58.942 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:58.942 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:58.942 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:58.942 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:58.942 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:58.942 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:58.943 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:58.943 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.943 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:43:58.943 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:58.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.943 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:43:58.943 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:58.943 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.943 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.944 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:58.944 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.944 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:43:58.944 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:58.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.944 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:43:58.944 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:58.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.944 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.945 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:58.945 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.945 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:43:58.945 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:58.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.945 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:43:58.945 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:58.945 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.945 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.945 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:43:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:43:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:43:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:43:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:43:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:43:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:43:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:43:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:43:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:43:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:43:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:43:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:43:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:43:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:43:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:43:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:43:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:43:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:43:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:43:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:43:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:43:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:43:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:43:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:43:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:43:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:43:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:43:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:43:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:43:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:43:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:43:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:43:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:43:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:43:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:43:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:43:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:43:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:43:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:43:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:43:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:43:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:43:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:43:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:43:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:43:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:43:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:43:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:43:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:43:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:43:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:43:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:43:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:43:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:43:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:43:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:43:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:43:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:43:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:43:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:43:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:43:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:43:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:43:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:43:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:43:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:43:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:43:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:43:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:43:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:43:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:43:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:43:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:43:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:43:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:43:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:43:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:43:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:43:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:43:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:43:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:43:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:43:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:43:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:43:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:43:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:43:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:43:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:43:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:43:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:43:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:43:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:43:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:43:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:43:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:43:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:43:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:43:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:43:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:43:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:43:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:43:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:43:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:43:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:43:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:43:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:43:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:43:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:43:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:43:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:43:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:43:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:58.982 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:43:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.913+0000 7f7977039700 1 -- 172.21.15.7:0/2317757251 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7954066f30 msgr2=0x7f79540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.913+0000 7f7977039700 1 --2- 172.21.15.7:0/2317757251 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7954066f30 0x7f79540693e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f79640099b0 tx=0x7f7964017000 comp rx=0 tx=0).stop 2024-01-27T02:43:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.913+0000 7f7977039700 1 -- 172.21.15.7:0/2317757251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79701404e0 msgr2=0x7f79701596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.913+0000 7f7977039700 1 --2- 172.21.15.7:0/2317757251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79701404e0 0x7f79701596f0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f795c00ec80 tx=0x7f795c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:43:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.913+0000 7f7977039700 1 -- 172.21.15.7:0/2317757251 shutdown_connections 2024-01-27T02:43:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.913+0000 7f7977039700 1 --2- 172.21.15.7:0/2317757251 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7954066f30 0x7f79540693e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.913+0000 7f7977039700 1 --2- 172.21.15.7:0/2317757251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79701408b0 0x7f79700c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.913+0000 7f7977039700 1 --2- 172.21.15.7:0/2317757251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79700b09f0 0x7f79701315a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.913+0000 7f7977039700 1 --2- 172.21.15.7:0/2317757251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79701404e0 0x7f79701596f0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:58.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.913+0000 7f7977039700 1 -- 172.21.15.7:0/2317757251 >> 172.21.15.7:0/2317757251 conn(0x7f79700cdc30 msgr2=0x7f79700c29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:58.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.913+0000 7f7977039700 1 -- 172.21.15.7:0/2317757251 shutdown_connections 2024-01-27T02:43:58.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:58.913+0000 7f7977039700 1 -- 172.21.15.7:0/2317757251 wait complete. 2024-01-27T02:43:58.985 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-01-27T02:43:58.985 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-27T02:43:59.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.120+0000 7f4ce01a5700 1 Processor -- start 2024-01-27T02:43:59.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.121+0000 7f4ce01a5700 1 -- start start 2024-01-27T02:43:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.122+0000 7f4ce01a5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4cd80c8ab0 0x7f4cd815dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.122+0000 7f4ce01a5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4cd80cbbc0 0x7f4cd80cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.122+0000 7f4ce01a5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cd815a9b0 con 0x7f4cd80c8e80 2024-01-27T02:43:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.122+0000 7f4ce01a5700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd815ecb0 con 0x7f4cd80cbbc0 2024-01-27T02:43:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.122+0000 7f4ce01a5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd815e410 con 0x7f4cd80c8ab0 2024-01-27T02:43:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.122+0000 7f4cde742700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4cd80cbbc0 0x7f4cd80cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.122+0000 7f4cdd740700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4cd80c8e80 0x7f4cd80cd4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44850/0 (socket says 172.21.15.7:44850) 2024-01-27T02:43:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.122+0000 7f4cdd740700 1 -- 172.21.15.7:0/3575968194 learned_addr learned my addr 172.21.15.7:0/3575968194 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:43:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.122+0000 7f4cddf41700 1 --2- 172.21.15.7:0/3575968194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4cd80c8ab0 0x7f4cd815dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.122+0000 7f4cde742700 1 -- 172.21.15.7:0/3575968194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4cd80c8ab0 msgr2=0x7f4cd815dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.122+0000 7f4cde742700 1 --2- 172.21.15.7:0/3575968194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4cd80c8ab0 0x7f4cd815dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.122+0000 7f4cde742700 1 -- 172.21.15.7:0/3575968194 >> v1:172.21.15.7:6789/0 conn(0x7f4cd80c8e80 legacy=0x7f4cd80cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.122+0000 7f4cde742700 1 -- 172.21.15.7:0/3575968194 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd815e6f0 con 0x7f4cd80cbbc0 2024-01-27T02:43:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.122+0000 7f4cdcf3f700 1 -- 172.21.15.7:0/3575968194 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3869983263 0 0) 0x7f4cd815a9b0 con 0x7f4cd80c8e80 2024-01-27T02:43:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.122+0000 7f4cde742700 1 --2- 172.21.15.7:0/3575968194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd80cbbc0 0x7f4cd80cc040 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009e60 tx=0x7f4cd0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8faee7e0093b549 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.122+0000 7f4cdcf3f700 1 -- 172.21.15.7:0/3575968194 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd000e070 con 0x7f4cd80cbbc0 2024-01-27T02:43:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.123+0000 7f4cdcf3f700 1 -- 172.21.15.7:0/3575968194 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd000bc30 con 0x7f4cd80cbbc0 2024-01-27T02:43:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.123+0000 7f4cdcf3f700 1 -- 172.21.15.7:0/3575968194 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0012430 con 0x7f4cd80cbbc0 2024-01-27T02:43:59.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.123+0000 7f4ce01a5700 1 -- 172.21.15.7:0/3575968194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd80cbbc0 msgr2=0x7f4cd80cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:59.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.123+0000 7f4ce01a5700 1 --2- 172.21.15.7:0/3575968194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd80cbbc0 0x7f4cd80cc040 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009e60 tx=0x7f4cd0009780 comp rx=0 tx=0).stop 2024-01-27T02:43:59.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.123+0000 7f4ce01a5700 1 -- 172.21.15.7:0/3575968194 shutdown_connections 2024-01-27T02:43:59.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.123+0000 7f4ce01a5700 1 --2- 172.21.15.7:0/3575968194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd80cbbc0 0x7f4cd80cc040 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:59.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.123+0000 7f4ce01a5700 1 --2- 172.21.15.7:0/3575968194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4cd80c8ab0 0x7f4cd815dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:59.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.123+0000 7f4ce01a5700 1 -- 172.21.15.7:0/3575968194 >> 172.21.15.7:0/3575968194 conn(0x7f4cd80cdc30 msgr2=0x7f4cd80ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:59.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.123+0000 7f4ce01a5700 1 -- 172.21.15.7:0/3575968194 shutdown_connections 2024-01-27T02:43:59.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.123+0000 7f4ce01a5700 1 -- 172.21.15.7:0/3575968194 wait complete. 2024-01-27T02:43:59.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.125+0000 7f4ce01a5700 1 Processor -- start 2024-01-27T02:43:59.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.125+0000 7f4ce01a5700 1 -- start start 2024-01-27T02:43:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.126+0000 7f4ce01a5700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4cd80c8ab0 0x7f4cd8169640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.126+0000 7f4ce01a5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4cd80c8e80 0x7f4cd80ab680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.126+0000 7f4ce01a5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd80cbbc0 0x7f4cd80abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.126+0000 7f4ce01a5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd80b34e0 con 0x7f4cd80cbbc0 2024-01-27T02:43:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.126+0000 7f4ce01a5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd80b28d0 con 0x7f4cd80c8e80 2024-01-27T02:43:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.126+0000 7f4ce01a5700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd80b2bb0 con 0x7f4cd80c8ab0 2024-01-27T02:43:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.126+0000 7f4cde742700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd80cbbc0 0x7f4cd80abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.126+0000 7f4cde742700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd80cbbc0 0x7f4cd80abbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36572/0 (socket says 172.21.15.7:36572) 2024-01-27T02:43:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.126+0000 7f4cde742700 1 -- 172.21.15.7:0/2833169206 learned_addr learned my addr 172.21.15.7:0/2833169206 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.126+0000 7f4cddf41700 1 --2- 172.21.15.7:0/2833169206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4cd80c8ab0 0x7f4cd8169640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.126+0000 7f4cdd740700 1 --2- 172.21.15.7:0/2833169206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4cd80c8e80 0x7f4cd80ab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.126+0000 7f4cde742700 1 -- 172.21.15.7:0/2833169206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4cd80c8ab0 msgr2=0x7f4cd8169640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.126+0000 7f4cde742700 1 --2- 172.21.15.7:0/2833169206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4cd80c8ab0 0x7f4cd8169640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.126+0000 7f4cde742700 1 -- 172.21.15.7:0/2833169206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4cd80c8e80 msgr2=0x7f4cd80ab680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.126+0000 7f4cde742700 1 --2- 172.21.15.7:0/2833169206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4cd80c8e80 0x7f4cd80ab680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.126+0000 7f4cde742700 1 -- 172.21.15.7:0/2833169206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd0009a20 con 0x7f4cd80cbbc0 2024-01-27T02:43:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.127+0000 7f4cde742700 1 --2- 172.21.15.7:0/2833169206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd80cbbc0 0x7f4cd80abbc0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0013000 tx=0x7f4cd0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.127+0000 7f4ccdffb700 1 -- 172.21.15.7:0/2833169206 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0012e50 con 0x7f4cd80cbbc0 2024-01-27T02:43:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.127+0000 7f4ccdffb700 1 -- 172.21.15.7:0/2833169206 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd001d630 con 0x7f4cd80cbbc0 2024-01-27T02:43:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.127+0000 7f4ce01a5700 1 -- 172.21.15.7:0/2833169206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd80b2e90 con 0x7f4cd80cbbc0 2024-01-27T02:43:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.127+0000 7f4ccdffb700 1 -- 172.21.15.7:0/2833169206 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd001d9f0 con 0x7f4cd80cbbc0 2024-01-27T02:43:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.127+0000 7f4ce01a5700 1 -- 172.21.15.7:0/2833169206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd8169b80 con 0x7f4cd80cbbc0 2024-01-27T02:43:59.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.128+0000 7f4ce01a5700 1 -- 172.21.15.7:0/2833169206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd80b3690 con 0x7f4cd80cbbc0 2024-01-27T02:43:59.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.128+0000 7f4ccdffb700 1 -- 172.21.15.7:0/2833169206 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4cd0024070 con 0x7f4cd80cbbc0 2024-01-27T02:43:59.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.129+0000 7f4ccdffb700 1 --2- 172.21.15.7:0/2833169206 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4cbc066f60 0x7f4cbc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:59.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.129+0000 7f4ccdffb700 1 -- 172.21.15.7:0/2833169206 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4cd0086de0 con 0x7f4cd80cbbc0 2024-01-27T02:43:59.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.129+0000 7f4cddf41700 1 --2- 172.21.15.7:0/2833169206 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4cbc066f60 0x7f4cbc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:59.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.130+0000 7f4cddf41700 1 --2- 172.21.15.7:0/2833169206 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4cbc066f60 0x7f4cbc069410 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4006fd0 tx=0x7f4cd4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:59.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.133+0000 7f4ccdffb700 1 -- 172.21.15.7:0/2833169206 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4cd80b3690 con 0x7f4cd80cbbc0 2024-01-27T02:43:59.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.339+0000 7f4ce01a5700 1 -- 172.21.15.7:0/2833169206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f4cd804c710 con 0x7f4cd80cbbc0 2024-01-27T02:43:59.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.340+0000 7f4ccdffb700 1 -- 172.21.15.7:0/2833169206 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v183) v1 ==== 81+0+32234 (secure 0 0 0) 0x7f4cd002c090 con 0x7f4cd80cbbc0 2024-01-27T02:43:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:59.341 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:43:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 183, 2024-01-27T02:43:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", 2024-01-27T02:43:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-27T02:31:50.016376+0000", 2024-01-27T02:43:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-27T02:43:54.936156+0000", 2024-01-27T02:43:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-27T02:31:55.531473+0000", 2024-01-27T02:43:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-27T02:31:52.519857+0000", 2024-01-27T02:43:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-27T02:43:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-27T02:43:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-27T02:43:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-27T02:43:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-27T02:43:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-27T02:43:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-27T02:43:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-27T02:43:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-27T02:43:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-27T02:43:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-27T02:43:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-27T02:43:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 23, 2024-01-27T02:43:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-27T02:43:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-27T02:43:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-27T02:43:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-27T02:43:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-27T02:43:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-27T02:43:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-27T02:43:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:31:57.628744+0000", 2024-01-27T02:43:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:43:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:43:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:43:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:43:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:43:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:43:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:43:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:43:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:43:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:43:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:43:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:43:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-27T02:43:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-27T02:43:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-27T02:43:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-27T02:43:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-27T02:43:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:43:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:43:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:43:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:43:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:43:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:43:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:43:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "106", 2024-01-27T02:43:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:43:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:43:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:43:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:43:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:43:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:43:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:43:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:43:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:43:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:43:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:43:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:43:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:43:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:43:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:43:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:43:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:43:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:43:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:43:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:43:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:43:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:43:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:43:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:43:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:43:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:43:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:43:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:43:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:43:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:43:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:43:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:43:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:43:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:43:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:43:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:43:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:43:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:43:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-27T02:43:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-27T02:43:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:43:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-27T02:43:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 22, 2024-01-27T02:43:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-27T02:43:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:43:27.492968+0000", 2024-01-27T02:43:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:43:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:43:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:43:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:43:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:43:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:43:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:43:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:43:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:43:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:43:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:43:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:43:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-27T02:43:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-27T02:43:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-27T02:43:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-27T02:43:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-27T02:43:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:43:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:43:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:43:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:43:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:43:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:43:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:43:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "179", 2024-01-27T02:43:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:43:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:43:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:43:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:43:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:43:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:43:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:43:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:43:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:43:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-27T02:43:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:43:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:43:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:43:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:43:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:43:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:43:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:43:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:43:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:43:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:43:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:43:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:43:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:43:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:43:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:43:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:43:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:43:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:43:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:43:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:43:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:43:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:43:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:43:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:43:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:43:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:43:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:43:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:43:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-27T02:43:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-27T02:43:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-27T02:43:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:43:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:43:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-27T02:43:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 23, 2024-01-27T02:43:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-27T02:43:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-27T02:43:28.579977+0000", 2024-01-27T02:43:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-27T02:43:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-27T02:43:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-27T02:43:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-27T02:43:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-27T02:43:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-27T02:43:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-27T02:43:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-27T02:43:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-27T02:43:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-27T02:43:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-27T02:43:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-27T02:43:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-27T02:43:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-27T02:43:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-27T02:43:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-27T02:43:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-27T02:43:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-27T02:43:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-27T02:43:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-27T02:43:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-27T02:43:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-27T02:43:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-27T02:43:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-27T02:43:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "183", 2024-01-27T02:43:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-27T02:43:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-27T02:43:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-27T02:43:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-27T02:43:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-27T02:43:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-27T02:43:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-27T02:43:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-27T02:43:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-27T02:43:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-27T02:43:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-27T02:43:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-27T02:43:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-27T02:43:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-27T02:43:59.359 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-27T02:43:59.359 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-27T02:43:59.359 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-27T02:43:59.359 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-27T02:43:59.359 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-27T02:43:59.359 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-27T02:43:59.359 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-27T02:43:59.359 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-27T02:43:59.360 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-27T02:43:59.360 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-27T02:43:59.360 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-27T02:43:59.360 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-27T02:43:59.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-27T02:43:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-27T02:43:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-27T02:43:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-27T02:43:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-27T02:43:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-27T02:43:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-27T02:43:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-27T02:43:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-27T02:43:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-27T02:43:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-27T02:43:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-27T02:43:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-27T02:43:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-27T02:43:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-27T02:43:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-27T02:43:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-27T02:43:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-27T02:43:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:43:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-27T02:43:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-27T02:43:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-27T02:43:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-27T02:43:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-27T02:43:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-27T02:43:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-27T02:43:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-27T02:43:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39319 2024-01-27T02:43:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/39319", 2024-01-27T02:43:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/39319", 2024-01-27T02:43:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/39319", 2024-01-27T02:43:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/39319", 2024-01-27T02:43:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:43:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-27T02:43:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-27T02:43:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-27T02:43:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-27T02:43:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-27T02:43:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-27T02:43:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-27T02:43:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-27T02:43:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39309 2024-01-27T02:43:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/39309", 2024-01-27T02:43:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/39309", 2024-01-27T02:43:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/39309", 2024-01-27T02:43:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/39309", 2024-01-27T02:43:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:43:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-27T02:43:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-27T02:43:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-27T02:43:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-27T02:43:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-27T02:43:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-27T02:43:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-27T02:43:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-27T02:43:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39325 2024-01-27T02:43:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/39325", 2024-01-27T02:43:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/39325", 2024-01-27T02:43:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/39325", 2024-01-27T02:43:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/39325", 2024-01-27T02:43:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:43:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-27T02:43:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-27T02:43:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-27T02:43:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-27T02:43:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-27T02:43:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-27T02:43:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-27T02:43:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-27T02:43:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39310 2024-01-27T02:43:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/39310", 2024-01-27T02:43:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/39310", 2024-01-27T02:43:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/39310", 2024-01-27T02:43:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/39310", 2024-01-27T02:43:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:43:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6801", 2024-01-27T02:43:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6803", 2024-01-27T02:43:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6805", 2024-01-27T02:43:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6807", 2024-01-27T02:43:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6814", 2024-01-27T02:43:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6815", 2024-01-27T02:43:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6809", 2024-01-27T02:43:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6811", 2024-01-27T02:43:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39374 2024-01-27T02:43:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6803/39374", 2024-01-27T02:43:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6807/39374", 2024-01-27T02:43:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6815/39374", 2024-01-27T02:43:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6811/39374", 2024-01-27T02:43:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:43:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6800", 2024-01-27T02:43:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6802", 2024-01-27T02:43:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6804", 2024-01-27T02:43:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6806", 2024-01-27T02:43:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6812", 2024-01-27T02:43:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6813", 2024-01-27T02:43:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6808", 2024-01-27T02:43:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6810", 2024-01-27T02:43:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39373 2024-01-27T02:43:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6802/39373", 2024-01-27T02:43:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6806/39373", 2024-01-27T02:43:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6813/39373", 2024-01-27T02:43:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6810/39373", 2024-01-27T02:43:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:43:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6824", 2024-01-27T02:43:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6825", 2024-01-27T02:43:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6826", 2024-01-27T02:43:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6827", 2024-01-27T02:43:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6830", 2024-01-27T02:43:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6831", 2024-01-27T02:43:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6828", 2024-01-27T02:43:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6829", 2024-01-27T02:43:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39412 2024-01-27T02:43:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6825/39412", 2024-01-27T02:43:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6827/39412", 2024-01-27T02:43:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6831/39412", 2024-01-27T02:43:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6829/39412", 2024-01-27T02:43:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:43:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-27T02:43:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-27T02:43:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-27T02:43:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-27T02:43:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-27T02:43:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-27T02:43:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-27T02:43:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-27T02:43:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 178, 2024-01-27T02:43:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-27T02:43:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-27T02:43:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-27T02:43:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6816", 2024-01-27T02:43:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6817", 2024-01-27T02:43:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-27T02:43:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6818", 2024-01-27T02:43:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6819", 2024-01-27T02:43:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-27T02:43:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6822", 2024-01-27T02:43:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6823", 2024-01-27T02:43:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-27T02:43:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-27T02:43:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-27T02:43:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6820", 2024-01-27T02:43:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-27T02:43:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.188:6821", 2024-01-27T02:43:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 39375 2024-01-27T02:43:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.188:6817/39375", 2024-01-27T02:43:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.188:6819/39375", 2024-01-27T02:43:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.188:6823/39375", 2024-01-27T02:43:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.188:6821/39375", 2024-01-27T02:43:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-27T02:43:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-27T02:43:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-27T02:43:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-27T02:43:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-27T02:43:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-27T02:43:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.678769+0000", 2024-01-27T02:43:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-27T02:43:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.691345+0000", 2024-01-27T02:43:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-27T02:43:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.644306+0000", 2024-01-27T02:43:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-27T02:43:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.707270+0000", 2024-01-27T02:43:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-27T02:43:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.670376+0000", 2024-01-27T02:43:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-27T02:43:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.615210+0000", 2024-01-27T02:43:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-27T02:43:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.694216+0000", 2024-01-27T02:43:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-27T02:43:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-27T02:43:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-27T02:43:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-27T02:43:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-27T02:43:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-27T02:43:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-27T02:43:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-27T02:31:53.669154+0000", 2024-01-27T02:43:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-27T02:43:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-27T02:43:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-27T02:43:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-27T02:43:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-27T02:43:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-27T02:43:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-27T02:43:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-27T02:43:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-27T02:43:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-27T02:43:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-27T02:43:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-27T02:43:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-27T02:43:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-27T02:43:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-27T02:43:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-27T02:43:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-27T02:43:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-27T02:43:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-27T02:43:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-27T02:43:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-27T02:43:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-27T02:43:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-27T02:43:59.416 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-27T02:43:59.416 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-27T02:43:59.416 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-27T02:43:59.416 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-27T02:43:59.416 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-27T02:43:59.416 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:43:59.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.342+0000 7f4ce01a5700 1 -- 172.21.15.7:0/2833169206 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4cbc066f60 msgr2=0x7f4cbc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:59.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.342+0000 7f4ce01a5700 1 --2- 172.21.15.7:0/2833169206 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4cbc066f60 0x7f4cbc069410 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4006fd0 tx=0x7f4cd4008040 comp rx=0 tx=0).stop 2024-01-27T02:43:59.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.342+0000 7f4ce01a5700 1 -- 172.21.15.7:0/2833169206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd80cbbc0 msgr2=0x7f4cd80abbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:59.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.342+0000 7f4ce01a5700 1 --2- 172.21.15.7:0/2833169206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd80cbbc0 0x7f4cd80abbc0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0013000 tx=0x7f4cd0019000 comp rx=0 tx=0).stop 2024-01-27T02:43:59.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.342+0000 7f4ce01a5700 1 -- 172.21.15.7:0/2833169206 shutdown_connections 2024-01-27T02:43:59.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.342+0000 7f4ce01a5700 1 --2- 172.21.15.7:0/2833169206 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f4cbc066f60 0x7f4cbc069410 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:59.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.342+0000 7f4ce01a5700 1 --2- 172.21.15.7:0/2833169206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd80cbbc0 0x7f4cd80abbc0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:59.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.342+0000 7f4ce01a5700 1 --2- 172.21.15.7:0/2833169206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4cd80c8ab0 0x7f4cd8169640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:59.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.342+0000 7f4ce01a5700 1 --2- 172.21.15.7:0/2833169206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4cd80c8e80 0x7f4cd80ab680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:59.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.342+0000 7f4ce01a5700 1 -- 172.21.15.7:0/2833169206 >> 172.21.15.7:0/2833169206 conn(0x7f4cd80cdc30 msgr2=0x7f4cd80cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:59.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.342+0000 7f4ce01a5700 1 -- 172.21.15.7:0/2833169206 shutdown_connections 2024-01-27T02:43:59.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.342+0000 7f4ce01a5700 1 -- 172.21.15.7:0/2833169206 wait complete. 2024-01-27T02:43:59.419 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:43:59.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.553+0000 7fbe2b7e9700 1 Processor -- start 2024-01-27T02:43:59.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.554+0000 7fbe2b7e9700 1 -- start start 2024-01-27T02:43:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.554+0000 7fbe2b7e9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbe24159f40 0x7fbe240c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.554+0000 7fbe2b7e9700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbe240c3a90 0x7fbe240aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.554+0000 7fbe2b7e9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe2415a310 0x7fbe240aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.554+0000 7fbe2b7e9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe240c4550 con 0x7fbe24159f40 2024-01-27T02:43:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.554+0000 7fbe2b7e9700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe240c30c0 con 0x7fbe240c3a90 2024-01-27T02:43:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.554+0000 7fbe2b7e9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2415e960 con 0x7fbe2415a310 2024-01-27T02:43:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.555+0000 7fbe29585700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbe24159f40 0x7fbe240c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.555+0000 7fbe29585700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbe24159f40 0x7fbe240c3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36580/0 (socket says 172.21.15.7:36580) 2024-01-27T02:43:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.555+0000 7fbe29585700 1 -- 172.21.15.7:0/3917866829 learned_addr learned my addr 172.21.15.7:0/3917866829 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.555+0000 7fbe28d84700 1 --2- 172.21.15.7:0/3917866829 >> v2:172.21.15.188:3300/0 conn(0x7fbe240c3a90 0x7fbe240aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.555+0000 7fbe29d86700 1 --2- 172.21.15.7:0/3917866829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe2415a310 0x7fbe240aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.555+0000 7fbe29585700 1 -- 172.21.15.7:0/3917866829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe2415a310 msgr2=0x7fbe240aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.555+0000 7fbe29585700 1 --2- 172.21.15.7:0/3917866829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe2415a310 0x7fbe240aadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.555+0000 7fbe29585700 1 -- 172.21.15.7:0/3917866829 >> v2:172.21.15.188:3300/0 conn(0x7fbe240c3a90 msgr2=0x7fbe240aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:43:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.555+0000 7fbe29585700 1 --2- 172.21.15.7:0/3917866829 >> v2:172.21.15.188:3300/0 conn(0x7fbe240c3a90 0x7fbe240aa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.555+0000 7fbe29585700 1 -- 172.21.15.7:0/3917866829 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2415ec40 con 0x7fbe24159f40 2024-01-27T02:43:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.555+0000 7fbe28d84700 1 --2- 172.21.15.7:0/3917866829 >> v2:172.21.15.188:3300/0 conn(0x7fbe240c3a90 0x7fbe240aa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:43:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.555+0000 7fbe29585700 1 --2- 172.21.15.7:0/3917866829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe24159f40 0x7fbe240c3550 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fbe24064660 tx=0x7fbe20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960a1c7a8b82570 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.555+0000 7fbe1bfff700 1 -- 172.21.15.7:0/3917866829 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2000b860 con 0x7fbe24159f40 2024-01-27T02:43:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.555+0000 7fbe1bfff700 1 -- 172.21.15.7:0/3917866829 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2000b9c0 con 0x7fbe24159f40 2024-01-27T02:43:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.555+0000 7fbe1bfff700 1 -- 172.21.15.7:0/3917866829 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2000bd60 con 0x7fbe24159f40 2024-01-27T02:43:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.556+0000 7fbe2b7e9700 1 -- 172.21.15.7:0/3917866829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe24159f40 msgr2=0x7fbe240c3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.556+0000 7fbe2b7e9700 1 --2- 172.21.15.7:0/3917866829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe24159f40 0x7fbe240c3550 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fbe24064660 tx=0x7fbe20009250 comp rx=0 tx=0).stop 2024-01-27T02:43:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.556+0000 7fbe2b7e9700 1 -- 172.21.15.7:0/3917866829 shutdown_connections 2024-01-27T02:43:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.556+0000 7fbe2b7e9700 1 --2- 172.21.15.7:0/3917866829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe2415a310 0x7fbe240aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:59.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.556+0000 7fbe2b7e9700 1 --2- 172.21.15.7:0/3917866829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe24159f40 0x7fbe240c3550 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:59.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.556+0000 7fbe2b7e9700 1 --2- 172.21.15.7:0/3917866829 >> v2:172.21.15.188:3300/0 conn(0x7fbe240c3a90 0x7fbe240aa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:59.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.556+0000 7fbe2b7e9700 1 -- 172.21.15.7:0/3917866829 >> 172.21.15.7:0/3917866829 conn(0x7fbe240cdc30 msgr2=0x7fbe240ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:59.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.556+0000 7fbe2b7e9700 1 -- 172.21.15.7:0/3917866829 shutdown_connections 2024-01-27T02:43:59.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.556+0000 7fbe2b7e9700 1 -- 172.21.15.7:0/3917866829 wait complete. 2024-01-27T02:43:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.557+0000 7fbe2b7e9700 1 Processor -- start 2024-01-27T02:43:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.557+0000 7fbe2b7e9700 1 -- start start 2024-01-27T02:43:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.558+0000 7fbe2b7e9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe240c3a90 0x7fbe24157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.558+0000 7fbe2b7e9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe24159f40 0x7fbe240c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.558+0000 7fbe2b7e9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe2415a310 0x7fbe24157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.558+0000 7fbe2b7e9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe24158470 con 0x7fbe24159f40 2024-01-27T02:43:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.558+0000 7fbe2b7e9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe24158750 con 0x7fbe2415a310 2024-01-27T02:43:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.558+0000 7fbe2b7e9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe24158a30 con 0x7fbe240c3a90 2024-01-27T02:43:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.559+0000 7fbe28d84700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe24159f40 0x7fbe240c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.559+0000 7fbe28d84700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe24159f40 0x7fbe240c7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36588/0 (socket says 172.21.15.7:36588) 2024-01-27T02:43:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.559+0000 7fbe28d84700 1 -- 172.21.15.7:0/3433687996 learned_addr learned my addr 172.21.15.7:0/3433687996 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:43:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.559+0000 7fbe29585700 1 --2- 172.21.15.7:0/3433687996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe240c3a90 0x7fbe24157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.559+0000 7fbe29d86700 1 --2- 172.21.15.7:0/3433687996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe2415a310 0x7fbe24157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.559+0000 7fbe28d84700 1 -- 172.21.15.7:0/3433687996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe240c3a90 msgr2=0x7fbe24157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.559+0000 7fbe28d84700 1 --2- 172.21.15.7:0/3433687996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe240c3a90 0x7fbe24157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.559+0000 7fbe28d84700 1 -- 172.21.15.7:0/3433687996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe2415a310 msgr2=0x7fbe24157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.559+0000 7fbe28d84700 1 --2- 172.21.15.7:0/3433687996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe2415a310 0x7fbe24157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.559+0000 7fbe28d84700 1 -- 172.21.15.7:0/3433687996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe200094f0 con 0x7fbe24159f40 2024-01-27T02:43:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.559+0000 7fbe28d84700 1 --2- 172.21.15.7:0/3433687996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe24159f40 0x7fbe240c7b80 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400d890 tx=0x7fbe14008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.559+0000 7fbe197fa700 1 -- 172.21.15.7:0/3433687996 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe140125f0 con 0x7fbe24159f40 2024-01-27T02:43:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.559+0000 7fbe197fa700 1 -- 172.21.15.7:0/3433687996 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe14012750 con 0x7fbe24159f40 2024-01-27T02:43:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.559+0000 7fbe2b7e9700 1 -- 172.21.15.7:0/3433687996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe24158d10 con 0x7fbe24159f40 2024-01-27T02:43:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.559+0000 7fbe197fa700 1 -- 172.21.15.7:0/3433687996 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe14012af0 con 0x7fbe24159f40 2024-01-27T02:43:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.559+0000 7fbe2b7e9700 1 -- 172.21.15.7:0/3433687996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe2416a0a0 con 0x7fbe24159f40 2024-01-27T02:43:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.560+0000 7fbe2b7e9700 1 -- 172.21.15.7:0/3433687996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe2404c710 con 0x7fbe24159f40 2024-01-27T02:43:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.561+0000 7fbe197fa700 1 -- 172.21.15.7:0/3433687996 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbe14014430 con 0x7fbe24159f40 2024-01-27T02:43:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.561+0000 7fbe197fa700 1 --2- 172.21.15.7:0/3433687996 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbe08066f30 0x7fbe080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:43:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.561+0000 7fbe197fa700 1 -- 172.21.15.7:0/3433687996 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbe140848b0 con 0x7fbe24159f40 2024-01-27T02:43:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.562+0000 7fbe29585700 1 --2- 172.21.15.7:0/3433687996 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbe08066f30 0x7fbe080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:43:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.562+0000 7fbe29585700 1 --2- 172.21.15.7:0/3433687996 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbe08066f30 0x7fbe080693e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fbe24064660 tx=0x7fbe20017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:43:59.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.566+0000 7fbe197fa700 1 -- 172.21.15.7:0/3433687996 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe14051b20 con 0x7fbe24159f40 2024-01-27T02:43:59.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.815+0000 7fbe2b7e9700 1 -- 172.21.15.7:0/3433687996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe24157bc0 con 0x7fbe24159f40 2024-01-27T02:43:59.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.816+0000 7fbe197fa700 1 -- 172.21.15.7:0/3433687996 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4595 (secure 0 0 0) 0x7fbe14055140 con 0x7fbe24159f40 2024-01-27T02:43:59.816 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:43:59.817 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6704,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/1199331492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1199331492},{"type":"v1","addr":"172.21.15.7:6834","nonce":1199331492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":6707,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/721335086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":721335086},{"type":"v1","addr":"172.21.15.7:6835","nonce":721335086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":7699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/450555206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":450555206},{"type":"v1","addr":"172.21.15.188:6836","nonce":450555206}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:43:29.599748+0000","modified":"2024-01-27T02:43:33.196288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7702":{"gid":7702,"name":"b","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/3853795510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3853795510},{"type":"v1","addr":"172.21.15.188:6837","nonce":3853795510}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-27T02:43:59.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.818+0000 7fbe2b7e9700 1 -- 172.21.15.7:0/3433687996 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbe08066f30 msgr2=0x7fbe080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:59.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.818+0000 7fbe2b7e9700 1 --2- 172.21.15.7:0/3433687996 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbe08066f30 0x7fbe080693e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fbe24064660 tx=0x7fbe20017000 comp rx=0 tx=0).stop 2024-01-27T02:43:59.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.818+0000 7fbe2b7e9700 1 -- 172.21.15.7:0/3433687996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe24159f40 msgr2=0x7fbe240c7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:43:59.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.818+0000 7fbe2b7e9700 1 --2- 172.21.15.7:0/3433687996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe24159f40 0x7fbe240c7b80 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400d890 tx=0x7fbe14008040 comp rx=0 tx=0).stop 2024-01-27T02:43:59.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.818+0000 7fbe2b7e9700 1 -- 172.21.15.7:0/3433687996 shutdown_connections 2024-01-27T02:43:59.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.818+0000 7fbe2b7e9700 1 --2- 172.21.15.7:0/3433687996 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fbe08066f30 0x7fbe080693e0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:59.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.818+0000 7fbe2b7e9700 1 --2- 172.21.15.7:0/3433687996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe2415a310 0x7fbe24157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:59.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.818+0000 7fbe2b7e9700 1 --2- 172.21.15.7:0/3433687996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe240c3a90 0x7fbe24157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:59.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.818+0000 7fbe2b7e9700 1 --2- 172.21.15.7:0/3433687996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe24159f40 0x7fbe240c7b80 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:43:59.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.818+0000 7fbe2b7e9700 1 -- 172.21.15.7:0/3433687996 >> 172.21.15.7:0/3433687996 conn(0x7fbe240cdc30 msgr2=0x7fbe241597c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:43:59.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.818+0000 7fbe2b7e9700 1 -- 172.21.15.7:0/3433687996 shutdown_connections 2024-01-27T02:43:59.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:43:59.818+0000 7fbe2b7e9700 1 -- 172.21.15.7:0/3433687996 wait complete. 2024-01-27T02:43:59.820 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 94 2024-01-27T02:43:59.832 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2024-01-27T02:43:59.981 INFO:teuthology.orchestra.run.smithi188.stdout:{"cluster_fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","whoami":0,"id":7702,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":29.788146201,"mdsmap_epoch":94,"osdmap_epoch":183,"osdmap_epoch_barrier":182,"uptime":35.762664022000003} 2024-01-27T02:43:59.982 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "17e1bcca-e647-4a58-b0ac-7d39d8508814", "fs_name": "cephfs", "id": 7702, "mdsmap_epoch": 94, "osdmap_epoch": 183, "osdmap_epoch_barrier": 182, "rank_uptime": 29.788146201, "state": "up:active", "uptime": 35.762664022, "want_state": "up:active", "whoami": 0 } 2024-01-27T02:43:59.982 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-01-27T02:43:59.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:43:59.982 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-01-27T02:44:00.303 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records in 2024-01-27T02:44:00.303 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records out 2024-01-27T02:44:00.303 INFO:teuthology.orchestra.run.smithi007.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.298781 s, 175 MB/s 2024-01-27T02:44:00.304 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-27T02:44:00.304 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:44:00.348 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-27T02:44:00.389 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-27T02:44:00.389 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-27T02:44:00.389 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:44:00.389 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-27T02:44:00.471 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:44:00.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:44:00.471 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-27T02:44:00.495 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-27T02:44:00.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-27T02:44:00.495 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-01-27T02:44:00.536 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:44:00.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:44:00.536 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-27T02:44:00.593 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:44:00.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.733+0000 7fcb1bf13700 1 Processor -- start 2024-01-27T02:44:00.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.734+0000 7fcb1bf13700 1 -- start start 2024-01-27T02:44:00.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.735+0000 7fcb1bf13700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcb140c9f80 0x7fcb14157f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:00.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.735+0000 7fcb1bf13700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb1406c880 con 0x7fcb1406e330 2024-01-27T02:44:00.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.735+0000 7fcb1bf13700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb1406bc70 con 0x7fcb140c9bb0 2024-01-27T02:44:00.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.735+0000 7fcb1bf13700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb1406cd10 con 0x7fcb140c9f80 2024-01-27T02:44:00.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.735+0000 7fcb194ae700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcb140c9f80 0x7fcb14157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:00.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.735+0000 7fcb1a4b0700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcb1406e330 0x7fcb1406e750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44866/0 (socket says 172.21.15.7:44866) 2024-01-27T02:44:00.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.735+0000 7fcb1a4b0700 1 -- 172.21.15.7:0/536162925 learned_addr learned my addr 172.21.15.7:0/536162925 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:00.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.735+0000 7fcb194ae700 1 -- 172.21.15.7:0/536162925 >> v1:172.21.15.188:6789/0 conn(0x7fcb140c9bb0 legacy=0x7fcb1406b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:00.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.735+0000 7fcb194ae700 1 -- 172.21.15.7:0/536162925 >> v1:172.21.15.7:6789/0 conn(0x7fcb1406e330 legacy=0x7fcb1406e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:00.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.735+0000 7fcb194ae700 1 -- 172.21.15.7:0/536162925 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1406bf50 con 0x7fcb140c9f80 2024-01-27T02:44:00.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.735+0000 7fcb194ae700 1 --2- 172.21.15.7:0/536162925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb140c9f80 0x7fcb14157f70 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fcb040098e0 tx=0x7fcb040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e7bc5c40623cef server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:00.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.735+0000 7fcb18cad700 1 -- 172.21.15.7:0/536162925 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04009e80 con 0x7fcb140c9f80 2024-01-27T02:44:00.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.736+0000 7fcb18cad700 1 -- 172.21.15.7:0/536162925 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0400b860 con 0x7fcb140c9f80 2024-01-27T02:44:00.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.736+0000 7fcb18cad700 1 -- 172.21.15.7:0/536162925 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0400bbe0 con 0x7fcb140c9f80 2024-01-27T02:44:00.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.736+0000 7fcb1bf13700 1 -- 172.21.15.7:0/536162925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb140c9f80 msgr2=0x7fcb14157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:00.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.736+0000 7fcb1bf13700 1 --2- 172.21.15.7:0/536162925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb140c9f80 0x7fcb14157f70 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fcb040098e0 tx=0x7fcb040091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:00.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.736+0000 7fcb1bf13700 1 -- 172.21.15.7:0/536162925 shutdown_connections 2024-01-27T02:44:00.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.736+0000 7fcb1bf13700 1 --2- 172.21.15.7:0/536162925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb140c9f80 0x7fcb14157f70 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:00.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.736+0000 7fcb1bf13700 1 -- 172.21.15.7:0/536162925 >> 172.21.15.7:0/536162925 conn(0x7fcb140cdc30 msgr2=0x7fcb140ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:00.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.736+0000 7fcb1bf13700 1 -- 172.21.15.7:0/536162925 shutdown_connections 2024-01-27T02:44:00.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.736+0000 7fcb1bf13700 1 -- 172.21.15.7:0/536162925 wait complete. 2024-01-27T02:44:00.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.737+0000 7fcb1bf13700 1 Processor -- start 2024-01-27T02:44:00.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.738+0000 7fcb1bf13700 1 -- start start 2024-01-27T02:44:00.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.738+0000 7fcb1bf13700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb1406e330 0x7fcb140c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:00.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.738+0000 7fcb1bf13700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb140c9bb0 0x7fcb140b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:00.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.738+0000 7fcb1bf13700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb140c9f80 0x7fcb140c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:00.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.738+0000 7fcb1bf13700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb140c6540 con 0x7fcb1406e330 2024-01-27T02:44:00.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.738+0000 7fcb1bf13700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb140c4560 con 0x7fcb140c9bb0 2024-01-27T02:44:00.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.738+0000 7fcb1bf13700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb140b3a80 con 0x7fcb140c9f80 2024-01-27T02:44:00.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.738+0000 7fcb19caf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb1406e330 0x7fcb140c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:00.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.738+0000 7fcb19caf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb1406e330 0x7fcb140c7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36606/0 (socket says 172.21.15.7:36606) 2024-01-27T02:44:00.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.738+0000 7fcb19caf700 1 -- 172.21.15.7:0/1918150730 learned_addr learned my addr 172.21.15.7:0/1918150730 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:00.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.739+0000 7fcb194ae700 1 --2- 172.21.15.7:0/1918150730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb140c9bb0 0x7fcb140b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:00.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.739+0000 7fcb1a4b0700 1 --2- 172.21.15.7:0/1918150730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb140c9f80 0x7fcb140c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:00.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.739+0000 7fcb19caf700 1 -- 172.21.15.7:0/1918150730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb140c9f80 msgr2=0x7fcb140c2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:00.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.739+0000 7fcb19caf700 1 --2- 172.21.15.7:0/1918150730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb140c9f80 0x7fcb140c2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:00.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.739+0000 7fcb19caf700 1 -- 172.21.15.7:0/1918150730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb140c9bb0 msgr2=0x7fcb140b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:00.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.739+0000 7fcb19caf700 1 --2- 172.21.15.7:0/1918150730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb140c9bb0 0x7fcb140b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:00.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.739+0000 7fcb19caf700 1 -- 172.21.15.7:0/1918150730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb04009420 con 0x7fcb1406e330 2024-01-27T02:44:00.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.739+0000 7fcb19caf700 1 --2- 172.21.15.7:0/1918150730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb1406e330 0x7fcb140c7630 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000c7c0 tx=0x7fcb10007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:00.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.739+0000 7fcb09ffb700 1 -- 172.21.15.7:0/1918150730 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10011590 con 0x7fcb1406e330 2024-01-27T02:44:00.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.739+0000 7fcb09ffb700 1 -- 172.21.15.7:0/1918150730 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb10011710 con 0x7fcb1406e330 2024-01-27T02:44:00.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.739+0000 7fcb1bf13700 1 -- 172.21.15.7:0/1918150730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb140c40d0 con 0x7fcb1406e330 2024-01-27T02:44:00.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.739+0000 7fcb09ffb700 1 -- 172.21.15.7:0/1918150730 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10011ad0 con 0x7fcb1406e330 2024-01-27T02:44:00.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.740+0000 7fcb1bf13700 1 -- 172.21.15.7:0/1918150730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb140c3220 con 0x7fcb1406e330 2024-01-27T02:44:00.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.740+0000 7fcb1bf13700 1 -- 172.21.15.7:0/1918150730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb1404c710 con 0x7fcb1406e330 2024-01-27T02:44:00.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.741+0000 7fcb09ffb700 1 -- 172.21.15.7:0/1918150730 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcb1001b3b0 con 0x7fcb1406e330 2024-01-27T02:44:00.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.741+0000 7fcb09ffb700 1 --2- 172.21.15.7:0/1918150730 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcaf8066f30 0x7fcaf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:00.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.741+0000 7fcb09ffb700 1 -- 172.21.15.7:0/1918150730 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcb100839c0 con 0x7fcb1406e330 2024-01-27T02:44:00.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.742+0000 7fcb194ae700 1 --2- 172.21.15.7:0/1918150730 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcaf8066f30 0x7fcaf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:00.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.742+0000 7fcb194ae700 1 --2- 172.21.15.7:0/1918150730 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcaf8066f30 0x7fcaf80693e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fcb140ca380 tx=0x7fcb04017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:00.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.746+0000 7fcb09ffb700 1 -- 172.21.15.7:0/1918150730 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb10050c30 con 0x7fcb1406e330 2024-01-27T02:44:00.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.997+0000 7fcb1bf13700 1 -- 172.21.15.7:0/1918150730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb140cc5e0 con 0x7fcb1406e330 2024-01-27T02:44:00.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:00.997+0000 7fcb09ffb700 1 -- 172.21.15.7:0/1918150730 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4595 (secure 0 0 0) 0x7fcb10017070 con 0x7fcb1406e330 2024-01-27T02:44:00.998 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:44:00.998 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6704,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/1199331492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1199331492},{"type":"v1","addr":"172.21.15.7:6834","nonce":1199331492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":6707,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/721335086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":721335086},{"type":"v1","addr":"172.21.15.7:6835","nonce":721335086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":7699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/450555206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":450555206},{"type":"v1","addr":"172.21.15.188:6836","nonce":450555206}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:43:29.599748+0000","modified":"2024-01-27T02:43:33.196288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7702":{"gid":7702,"name":"b","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/3853795510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3853795510},{"type":"v1","addr":"172.21.15.188:6837","nonce":3853795510}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-27T02:44:01.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.000+0000 7fcb1bf13700 1 -- 172.21.15.7:0/1918150730 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcaf8066f30 msgr2=0x7fcaf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:01.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.000+0000 7fcb1bf13700 1 --2- 172.21.15.7:0/1918150730 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcaf8066f30 0x7fcaf80693e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fcb140ca380 tx=0x7fcb04017000 comp rx=0 tx=0).stop 2024-01-27T02:44:01.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.000+0000 7fcb1bf13700 1 -- 172.21.15.7:0/1918150730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb1406e330 msgr2=0x7fcb140c7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.000+0000 7fcb1bf13700 1 --2- 172.21.15.7:0/1918150730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb1406e330 0x7fcb140c7630 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000c7c0 tx=0x7fcb10007f60 comp rx=0 tx=0).stop 2024-01-27T02:44:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.000+0000 7fcb1bf13700 1 -- 172.21.15.7:0/1918150730 shutdown_connections 2024-01-27T02:44:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.000+0000 7fcb1bf13700 1 --2- 172.21.15.7:0/1918150730 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fcaf8066f30 0x7fcaf80693e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.000+0000 7fcb1bf13700 1 --2- 172.21.15.7:0/1918150730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb140c9f80 0x7fcb140c2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.000+0000 7fcb1bf13700 1 --2- 172.21.15.7:0/1918150730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb1406e330 0x7fcb140c7630 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.000+0000 7fcb1bf13700 1 --2- 172.21.15.7:0/1918150730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb140c9bb0 0x7fcb140b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.000+0000 7fcb1bf13700 1 -- 172.21.15.7:0/1918150730 >> 172.21.15.7:0/1918150730 conn(0x7fcb140cdc30 msgr2=0x7fcb14150d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.000+0000 7fcb1bf13700 1 -- 172.21.15.7:0/1918150730 shutdown_connections 2024-01-27T02:44:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.000+0000 7fcb1bf13700 1 -- 172.21.15.7:0/1918150730 wait complete. 2024-01-27T02:44:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 94 2024-01-27T02:44:01.014 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:44:01.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.145+0000 7f74a3d09700 1 Processor -- start 2024-01-27T02:44:01.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.145+0000 7f74a3d09700 1 -- start start 2024-01-27T02:44:01.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.146+0000 7f74a3d09700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f749c0c3ae0 0x7f749c0cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:01.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.146+0000 7f74a3d09700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f749c158070 0x7f749c0cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:01.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.146+0000 7f74a3d09700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749c0ca020 con 0x7f749c157ca0 2024-01-27T02:44:01.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.146+0000 7f74a3d09700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f749c0c5210 con 0x7f749c158070 2024-01-27T02:44:01.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.146+0000 7f74a3d09700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f749c0c54f0 con 0x7f749c0c3ae0 2024-01-27T02:44:01.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.146+0000 7f74a1aa5700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f749c157ca0 0x7f749c0c4a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44876/0 (socket says 172.21.15.7:44876) 2024-01-27T02:44:01.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.146+0000 7f74a22a6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f749c158070 0x7f749c0cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:01.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.146+0000 7f74a1aa5700 1 -- 172.21.15.7:0/4011518772 learned_addr learned my addr 172.21.15.7:0/4011518772 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:01.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.146+0000 7f74a12a4700 1 --2- 172.21.15.7:0/4011518772 >> v2:172.21.15.188:3300/0 conn(0x7f749c0c3ae0 0x7f749c0cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:01.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.147+0000 7f74a22a6700 1 -- 172.21.15.7:0/4011518772 >> v2:172.21.15.188:3300/0 conn(0x7f749c0c3ae0 msgr2=0x7f749c0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:01.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.147+0000 7f74a22a6700 1 --2- 172.21.15.7:0/4011518772 >> v2:172.21.15.188:3300/0 conn(0x7f749c0c3ae0 0x7f749c0cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.147+0000 7f74a22a6700 1 -- 172.21.15.7:0/4011518772 >> v1:172.21.15.7:6789/0 conn(0x7f749c157ca0 legacy=0x7f749c0c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:01.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.147+0000 7f74a22a6700 1 -- 172.21.15.7:0/4011518772 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c0c57d0 con 0x7f749c158070 2024-01-27T02:44:01.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.147+0000 7f74a0aa3700 1 -- 172.21.15.7:0/4011518772 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2161588222 0 0) 0x7f749c0ca020 con 0x7f749c157ca0 2024-01-27T02:44:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.147+0000 7f74a22a6700 1 --2- 172.21.15.7:0/4011518772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f749c158070 0x7f749c0cda20 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f749c06e170 tx=0x7f7494009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00282a6a998bff4 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.147+0000 7f74a0aa3700 1 -- 172.21.15.7:0/4011518772 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749400e070 con 0x7f749c158070 2024-01-27T02:44:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.147+0000 7f74a0aa3700 1 -- 172.21.15.7:0/4011518772 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749400bc30 con 0x7f749c158070 2024-01-27T02:44:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.147+0000 7f74a0aa3700 1 -- 172.21.15.7:0/4011518772 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7494012430 con 0x7f749c158070 2024-01-27T02:44:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.147+0000 7f74a3d09700 1 -- 172.21.15.7:0/4011518772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f749c158070 msgr2=0x7f749c0cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.147+0000 7f74a3d09700 1 --2- 172.21.15.7:0/4011518772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f749c158070 0x7f749c0cda20 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f749c06e170 tx=0x7f7494009780 comp rx=0 tx=0).stop 2024-01-27T02:44:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.148+0000 7f74a3d09700 1 -- 172.21.15.7:0/4011518772 shutdown_connections 2024-01-27T02:44:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.148+0000 7f74a3d09700 1 --2- 172.21.15.7:0/4011518772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f749c158070 0x7f749c0cda20 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.148+0000 7f74a3d09700 1 --2- 172.21.15.7:0/4011518772 >> v2:172.21.15.188:3300/0 conn(0x7f749c0c3ae0 0x7f749c0cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.148+0000 7f74a3d09700 1 -- 172.21.15.7:0/4011518772 >> 172.21.15.7:0/4011518772 conn(0x7f749c098020 msgr2=0x7f749c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.148+0000 7f74a3d09700 1 -- 172.21.15.7:0/4011518772 shutdown_connections 2024-01-27T02:44:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.148+0000 7f74a3d09700 1 -- 172.21.15.7:0/4011518772 wait complete. 2024-01-27T02:44:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.149+0000 7f74a3d09700 1 Processor -- start 2024-01-27T02:44:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.149+0000 7f74a3d09700 1 -- start start 2024-01-27T02:44:01.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.150+0000 7f74a3d09700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f749c0c3ae0 0x7f749c0c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:01.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.150+0000 7f74a3d09700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f749c157ca0 0x7f749c0b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:01.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.150+0000 7f74a3d09700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f749c158070 0x7f749c0679b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:01.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.150+0000 7f74a3d09700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c16a6f0 con 0x7f749c158070 2024-01-27T02:44:01.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.150+0000 7f74a3d09700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c169de0 con 0x7f749c0c3ae0 2024-01-27T02:44:01.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.150+0000 7f74a3d09700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749c0b12b0 con 0x7f749c157ca0 2024-01-27T02:44:01.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.150+0000 7f74a22a6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f749c158070 0x7f749c0679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:01.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.150+0000 7f74a22a6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f749c158070 0x7f749c0679b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36618/0 (socket says 172.21.15.7:36618) 2024-01-27T02:44:01.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.150+0000 7f74a22a6700 1 -- 172.21.15.7:0/2402602303 learned_addr learned my addr 172.21.15.7:0/2402602303 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:01.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.150+0000 7f74a12a4700 1 --2- 172.21.15.7:0/2402602303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f749c157ca0 0x7f749c0b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.150+0000 7f74a1aa5700 1 --2- 172.21.15.7:0/2402602303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f749c0c3ae0 0x7f749c0c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.150+0000 7f74a22a6700 1 -- 172.21.15.7:0/2402602303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f749c157ca0 msgr2=0x7f749c0b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.150+0000 7f74a22a6700 1 --2- 172.21.15.7:0/2402602303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f749c157ca0 0x7f749c0b40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.150+0000 7f74a22a6700 1 -- 172.21.15.7:0/2402602303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f749c0c3ae0 msgr2=0x7f749c0c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.150+0000 7f74a22a6700 1 --2- 172.21.15.7:0/2402602303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f749c0c3ae0 0x7f749c0c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.150+0000 7f74a22a6700 1 -- 172.21.15.7:0/2402602303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7494009a20 con 0x7f749c158070 2024-01-27T02:44:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.150+0000 7f74a22a6700 1 --2- 172.21.15.7:0/2402602303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f749c158070 0x7f749c0679b0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f749c06ea80 tx=0x7f7494019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.151+0000 7f7491ffb700 1 -- 172.21.15.7:0/2402602303 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7494012ea0 con 0x7f749c158070 2024-01-27T02:44:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.151+0000 7f7491ffb700 1 -- 172.21.15.7:0/2402602303 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749401d630 con 0x7f749c158070 2024-01-27T02:44:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.151+0000 7f74a3d09700 1 -- 172.21.15.7:0/2402602303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749c16a0a0 con 0x7f749c158070 2024-01-27T02:44:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.151+0000 7f7491ffb700 1 -- 172.21.15.7:0/2402602303 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749401d9f0 con 0x7f749c158070 2024-01-27T02:44:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.151+0000 7f74a3d09700 1 -- 172.21.15.7:0/2402602303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749c0c6480 con 0x7f749c158070 2024-01-27T02:44:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.152+0000 7f74a3d09700 1 -- 172.21.15.7:0/2402602303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f749c04c710 con 0x7f749c158070 2024-01-27T02:44:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.152+0000 7f7491ffb700 1 -- 172.21.15.7:0/2402602303 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7494024070 con 0x7f749c158070 2024-01-27T02:44:01.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.153+0000 7f7491ffb700 1 --2- 172.21.15.7:0/2402602303 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7480066f30 0x7f74800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:01.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.153+0000 7f7491ffb700 1 -- 172.21.15.7:0/2402602303 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7494086dd0 con 0x7f749c158070 2024-01-27T02:44:01.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.153+0000 7f74a1aa5700 1 --2- 172.21.15.7:0/2402602303 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7480066f30 0x7f74800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:01.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.153+0000 7f74a1aa5700 1 --2- 172.21.15.7:0/2402602303 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7480066f30 0x7f74800693e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f74980011e0 tx=0x7f7498009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:01.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.157+0000 7f7491ffb700 1 -- 172.21.15.7:0/2402602303 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7494054040 con 0x7f749c158070 2024-01-27T02:44:01.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.366+0000 7f74a3d09700 1 -- 172.21.15.7:0/2402602303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f749c16a830 con 0x7f749c158070 2024-01-27T02:44:01.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.367+0000 7f7491ffb700 1 -- 172.21.15.7:0/2402602303 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+15324 (secure 0 0 0) 0x7f7494057660 con 0x7f749c158070 2024-01-27T02:44:01.368 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:44:01.368 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":183,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:43:54.936156+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:43:27.492968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"179","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-01-27T02:43:28.579977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:44:01.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.369+0000 7f74a3d09700 1 -- 172.21.15.7:0/2402602303 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7480066f30 msgr2=0x7f74800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:01.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.369+0000 7f74a3d09700 1 --2- 172.21.15.7:0/2402602303 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7480066f30 0x7f74800693e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f74980011e0 tx=0x7f7498009040 comp rx=0 tx=0).stop 2024-01-27T02:44:01.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.369+0000 7f74a3d09700 1 -- 172.21.15.7:0/2402602303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f749c158070 msgr2=0x7f749c0679b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:01.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.369+0000 7f74a3d09700 1 --2- 172.21.15.7:0/2402602303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f749c158070 0x7f749c0679b0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f749c06ea80 tx=0x7f7494019000 comp rx=0 tx=0).stop 2024-01-27T02:44:01.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.369+0000 7f74a3d09700 1 -- 172.21.15.7:0/2402602303 shutdown_connections 2024-01-27T02:44:01.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.369+0000 7f74a3d09700 1 --2- 172.21.15.7:0/2402602303 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7480066f30 0x7f74800693e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.369+0000 7f74a3d09700 1 --2- 172.21.15.7:0/2402602303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f749c158070 0x7f749c0679b0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.369+0000 7f74a3d09700 1 --2- 172.21.15.7:0/2402602303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f749c0c3ae0 0x7f749c0c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.369+0000 7f74a3d09700 1 --2- 172.21.15.7:0/2402602303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f749c157ca0 0x7f749c0b40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.369+0000 7f74a3d09700 1 -- 172.21.15.7:0/2402602303 >> 172.21.15.7:0/2402602303 conn(0x7f749c098020 msgr2=0x7f749c06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:01.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.369+0000 7f74a3d09700 1 -- 172.21.15.7:0/2402602303 shutdown_connections 2024-01-27T02:44:01.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.369+0000 7f74a3d09700 1 -- 172.21.15.7:0/2402602303 wait complete. 2024-01-27T02:44:01.384 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-27T02:31:57.628744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-27T02:43:27.492968+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '179', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-01-27T02:43:28.579977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '183', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-27T02:44:01.384 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-27T02:44:01.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.518+0000 7f92515ab700 1 Processor -- start 2024-01-27T02:44:01.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.518+0000 7f92515ab700 1 -- start start 2024-01-27T02:44:01.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.519+0000 7f92515ab700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f924c0cc8d0 0x7f924c0693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:01.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.519+0000 7f92515ab700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f924c069930 con 0x7f924c0b0f70 2024-01-27T02:44:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.519+0000 7f92515ab700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f924c15e820 con 0x7f924c0680c0 2024-01-27T02:44:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.519+0000 7f92515ab700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f924c068840 con 0x7f924c0cc8d0 2024-01-27T02:44:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.519+0000 7f924b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f924c0cc8d0 0x7f924c0693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.519+0000 7f924affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f924c0b0f70 0x7f924c0679b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56764/0 (socket says 172.21.15.7:56764) 2024-01-27T02:44:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.519+0000 7f924affd700 1 -- 172.21.15.7:0/2294063167 learned_addr learned my addr 172.21.15.7:0/2294063167 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.520+0000 7f924b7fe700 1 -- 172.21.15.7:0/2294063167 >> v1:172.21.15.188:6789/0 conn(0x7f924c0680c0 legacy=0x7f924c0c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.520+0000 7f924b7fe700 1 -- 172.21.15.7:0/2294063167 >> v1:172.21.15.7:6789/0 conn(0x7f924c0b0f70 legacy=0x7f924c0679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.520+0000 7f924b7fe700 1 -- 172.21.15.7:0/2294063167 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c15dc10 con 0x7f924c0cc8d0 2024-01-27T02:44:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.520+0000 7f924b7fe700 1 --2- 172.21.15.7:0/2294063167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924c0cc8d0 0x7f924c0693f0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f9244009e60 tx=0x7f9244009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96dcbb268f9532eb server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.520+0000 7f9249ffb700 1 -- 172.21.15.7:0/2294063167 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924400e070 con 0x7f924c0cc8d0 2024-01-27T02:44:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.520+0000 7f9249ffb700 1 -- 172.21.15.7:0/2294063167 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924400bc30 con 0x7f924c0cc8d0 2024-01-27T02:44:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.520+0000 7f9249ffb700 1 -- 172.21.15.7:0/2294063167 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244012430 con 0x7f924c0cc8d0 2024-01-27T02:44:01.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.521+0000 7f92515ab700 1 -- 172.21.15.7:0/2294063167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924c0cc8d0 msgr2=0x7f924c0693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:01.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.521+0000 7f92515ab700 1 --2- 172.21.15.7:0/2294063167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924c0cc8d0 0x7f924c0693f0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f9244009e60 tx=0x7f9244009780 comp rx=0 tx=0).stop 2024-01-27T02:44:01.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.521+0000 7f92515ab700 1 -- 172.21.15.7:0/2294063167 shutdown_connections 2024-01-27T02:44:01.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.521+0000 7f92515ab700 1 --2- 172.21.15.7:0/2294063167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924c0cc8d0 0x7f924c0693f0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.521+0000 7f92515ab700 1 -- 172.21.15.7:0/2294063167 >> 172.21.15.7:0/2294063167 conn(0x7f924c063580 msgr2=0x7f924c063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:01.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.521+0000 7f92515ab700 1 -- 172.21.15.7:0/2294063167 shutdown_connections 2024-01-27T02:44:01.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.521+0000 7f92515ab700 1 -- 172.21.15.7:0/2294063167 wait complete. 2024-01-27T02:44:01.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.522+0000 7f92515ab700 1 Processor -- start 2024-01-27T02:44:01.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.522+0000 7f92515ab700 1 -- start start 2024-01-27T02:44:01.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.523+0000 7f92515ab700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f924c0680c0 0x7f924c0ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:01.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.523+0000 7f92515ab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924c0b0f70 0x7f924c0c7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:01.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.523+0000 7f92515ab700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f924c0cc8d0 0x7f924c0c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:01.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.523+0000 7f92515ab700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c0c7cb0 con 0x7f924c0b0f70 2024-01-27T02:44:01.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.523+0000 7f92515ab700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c0c7f00 con 0x7f924c0680c0 2024-01-27T02:44:01.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.523+0000 7f92515ab700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924c0c81e0 con 0x7f924c0cc8d0 2024-01-27T02:44:01.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.523+0000 7f924a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924c0b0f70 0x7f924c0c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:01.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.523+0000 7f924a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924c0b0f70 0x7f924c0c7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48514/0 (socket says 172.21.15.7:48514) 2024-01-27T02:44:01.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.523+0000 7f924a7fc700 1 -- 172.21.15.7:0/1891374966 learned_addr learned my addr 172.21.15.7:0/1891374966 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:01.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.523+0000 7f924affd700 1 --2- 172.21.15.7:0/1891374966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f924c0680c0 0x7f924c0ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:01.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.524+0000 7f924b7fe700 1 --2- 172.21.15.7:0/1891374966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f924c0cc8d0 0x7f924c0c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:01.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.524+0000 7f924a7fc700 1 -- 172.21.15.7:0/1891374966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f924c0cc8d0 msgr2=0x7f924c0c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:01.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.524+0000 7f924a7fc700 1 --2- 172.21.15.7:0/1891374966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f924c0cc8d0 0x7f924c0c7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.524+0000 7f924a7fc700 1 -- 172.21.15.7:0/1891374966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f924c0680c0 msgr2=0x7f924c0ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:01.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.524+0000 7f924a7fc700 1 --2- 172.21.15.7:0/1891374966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f924c0680c0 0x7f924c0ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.524+0000 7f924a7fc700 1 -- 172.21.15.7:0/1891374966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9244009a20 con 0x7f924c0b0f70 2024-01-27T02:44:01.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.524+0000 7f924a7fc700 1 --2- 172.21.15.7:0/1891374966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924c0b0f70 0x7f924c0c7100 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f923400c4e0 tx=0x7f9234007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:01.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.524+0000 7f9242ffd700 1 -- 172.21.15.7:0/1891374966 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9234011590 con 0x7f924c0b0f70 2024-01-27T02:44:01.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.524+0000 7f9242ffd700 1 -- 172.21.15.7:0/1891374966 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92340116f0 con 0x7f924c0b0f70 2024-01-27T02:44:01.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.524+0000 7f92515ab700 1 -- 172.21.15.7:0/1891374966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f924c0c84c0 con 0x7f924c0b0f70 2024-01-27T02:44:01.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.524+0000 7f9242ffd700 1 -- 172.21.15.7:0/1891374966 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9234011ab0 con 0x7f924c0b0f70 2024-01-27T02:44:01.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.524+0000 7f92515ab700 1 -- 172.21.15.7:0/1891374966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f924c0c8a80 con 0x7f924c0b0f70 2024-01-27T02:44:01.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.525+0000 7f92515ab700 1 -- 172.21.15.7:0/1891374966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f924c0c8c70 con 0x7f924c0b0f70 2024-01-27T02:44:01.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.525+0000 7f9242ffd700 1 -- 172.21.15.7:0/1891374966 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f923401b3b0 con 0x7f924c0b0f70 2024-01-27T02:44:01.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.526+0000 7f9242ffd700 1 --2- 172.21.15.7:0/1891374966 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f922c066f30 0x7f922c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:01.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.526+0000 7f9242ffd700 1 -- 172.21.15.7:0/1891374966 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f92340839d0 con 0x7f924c0b0f70 2024-01-27T02:44:01.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.526+0000 7f924affd700 1 --2- 172.21.15.7:0/1891374966 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f922c066f30 0x7f922c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:01.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.527+0000 7f924affd700 1 --2- 172.21.15.7:0/1891374966 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f922c066f30 0x7f922c0693e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f923c001820 tx=0x7f923c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:01.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.530+0000 7f9242ffd700 1 -- 172.21.15.7:0/1891374966 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9234050c40 con 0x7f924c0b0f70 2024-01-27T02:44:01.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.731+0000 7f92515ab700 1 -- 172.21.15.7:0/1891374966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f924c072060 con 0x7f924c0b0f70 2024-01-27T02:44:01.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.731+0000 7f9242ffd700 1 -- 172.21.15.7:0/1891374966 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v183) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9234017070 con 0x7f924c0b0f70 2024-01-27T02:44:01.732 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-27T02:44:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.733+0000 7f92515ab700 1 -- 172.21.15.7:0/1891374966 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f922c066f30 msgr2=0x7f922c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.733+0000 7f92515ab700 1 --2- 172.21.15.7:0/1891374966 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f922c066f30 0x7f922c0693e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f923c001820 tx=0x7f923c009040 comp rx=0 tx=0).stop 2024-01-27T02:44:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.733+0000 7f92515ab700 1 -- 172.21.15.7:0/1891374966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924c0b0f70 msgr2=0x7f924c0c7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:01.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.733+0000 7f92515ab700 1 --2- 172.21.15.7:0/1891374966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924c0b0f70 0x7f924c0c7100 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f923400c4e0 tx=0x7f9234007f00 comp rx=0 tx=0).stop 2024-01-27T02:44:01.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.733+0000 7f92515ab700 1 -- 172.21.15.7:0/1891374966 shutdown_connections 2024-01-27T02:44:01.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.733+0000 7f92515ab700 1 --2- 172.21.15.7:0/1891374966 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f922c066f30 0x7f922c0693e0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.733+0000 7f92515ab700 1 --2- 172.21.15.7:0/1891374966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f924c0cc8d0 0x7f924c0c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.734+0000 7f92515ab700 1 --2- 172.21.15.7:0/1891374966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f924c0680c0 0x7f924c0ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.734+0000 7f92515ab700 1 --2- 172.21.15.7:0/1891374966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924c0b0f70 0x7f924c0c7100 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.734+0000 7f92515ab700 1 -- 172.21.15.7:0/1891374966 >> 172.21.15.7:0/1891374966 conn(0x7f924c063580 msgr2=0x7f924c159890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:01.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.734+0000 7f92515ab700 1 -- 172.21.15.7:0/1891374966 shutdown_connections 2024-01-27T02:44:01.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.734+0000 7f92515ab700 1 -- 172.21.15.7:0/1891374966 wait complete. 2024-01-27T02:44:01.747 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-27T02:44:01.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.881+0000 7faeeea7f700 1 Processor -- start 2024-01-27T02:44:01.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.881+0000 7faeeea7f700 1 -- start start 2024-01-27T02:44:01.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.882+0000 7faeeea7f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faee8156b60 0x7faee80ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:01.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.882+0000 7faeeea7f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faee80ad9f0 0x7faee80aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:01.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.882+0000 7faeeea7f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee80aeaa0 con 0x7faee80af470 2024-01-27T02:44:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.882+0000 7faeeea7f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faee80ade90 con 0x7faee8156b60 2024-01-27T02:44:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.882+0000 7faeeea7f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee80ae170 con 0x7faee80ad9f0 2024-01-27T02:44:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.882+0000 7faeed01c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7faee80af470 0x7faee815a9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56772/0 (socket says 172.21.15.7:56772) 2024-01-27T02:44:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.882+0000 7faeed01c700 1 -- 172.21.15.7:0/3681485496 learned_addr learned my addr 172.21.15.7:0/3681485496 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.882+0000 7faeec81b700 1 --2- 172.21.15.7:0/3681485496 >> v2:172.21.15.188:3300/0 conn(0x7faee8156b60 0x7faee80ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.882+0000 7faee7fff700 1 --2- 172.21.15.7:0/3681485496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faee80ad9f0 0x7faee80aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.883+0000 7faee77fe700 1 -- 172.21.15.7:0/3681485496 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3111902277 0 0) 0x7faee80aeaa0 con 0x7faee80af470 2024-01-27T02:44:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.883+0000 7faee77fe700 1 -- 172.21.15.7:0/3681485496 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faed8003570 con 0x7faee80af470 2024-01-27T02:44:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.883+0000 7faee7fff700 1 -- 172.21.15.7:0/3681485496 >> v2:172.21.15.188:3300/0 conn(0x7faee8156b60 msgr2=0x7faee80ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.883+0000 7faee7fff700 1 --2- 172.21.15.7:0/3681485496 >> v2:172.21.15.188:3300/0 conn(0x7faee8156b60 0x7faee80ad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.883+0000 7faee7fff700 1 -- 172.21.15.7:0/3681485496 >> v1:172.21.15.7:6789/0 conn(0x7faee80af470 legacy=0x7faee815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:01.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.883+0000 7faee7fff700 1 -- 172.21.15.7:0/3681485496 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee80ae450 con 0x7faee80ad9f0 2024-01-27T02:44:01.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.883+0000 7faeec81b700 1 --2- 172.21.15.7:0/3681485496 >> v2:172.21.15.188:3300/0 conn(0x7faee8156b60 0x7faee80ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:44:01.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.883+0000 7faee7fff700 1 --2- 172.21.15.7:0/3681485496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faee80ad9f0 0x7faee80aef30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faed40098e0 tx=0x7faed40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ecd46e5529f4645 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:01.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.883+0000 7faee77fe700 1 -- 172.21.15.7:0/3681485496 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed4011630 con 0x7faee80ad9f0 2024-01-27T02:44:01.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.883+0000 7faee77fe700 1 -- 172.21.15.7:0/3681485496 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed4011790 con 0x7faee80ad9f0 2024-01-27T02:44:01.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.883+0000 7faee77fe700 1 -- 172.21.15.7:0/3681485496 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed4011b30 con 0x7faee80ad9f0 2024-01-27T02:44:01.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.884+0000 7faeeea7f700 1 -- 172.21.15.7:0/3681485496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faee80ad9f0 msgr2=0x7faee80aef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:01.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.884+0000 7faeeea7f700 1 --2- 172.21.15.7:0/3681485496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faee80ad9f0 0x7faee80aef30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faed40098e0 tx=0x7faed40091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.884+0000 7faeeea7f700 1 -- 172.21.15.7:0/3681485496 shutdown_connections 2024-01-27T02:44:01.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.884+0000 7faeeea7f700 1 --2- 172.21.15.7:0/3681485496 >> v2:172.21.15.188:3300/0 conn(0x7faee8156b60 0x7faee80ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.884+0000 7faeeea7f700 1 --2- 172.21.15.7:0/3681485496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faee80ad9f0 0x7faee80aef30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.884+0000 7faeeea7f700 1 -- 172.21.15.7:0/3681485496 >> 172.21.15.7:0/3681485496 conn(0x7faee8068480 msgr2=0x7faee80cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:01.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.884+0000 7faeeea7f700 1 -- 172.21.15.7:0/3681485496 shutdown_connections 2024-01-27T02:44:01.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.884+0000 7faeeea7f700 1 -- 172.21.15.7:0/3681485496 wait complete. 2024-01-27T02:44:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.885+0000 7faeeea7f700 1 Processor -- start 2024-01-27T02:44:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.885+0000 7faeeea7f700 1 -- start start 2024-01-27T02:44:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.886+0000 7faeeea7f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faee80ad9f0 0x7faee8169fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.886+0000 7faeeea7f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faee80af470 0x7faee80b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.886+0000 7faeeea7f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faee8156b60 0x7faee80ac690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.886+0000 7faeeea7f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee80b2cf0 con 0x7faee8156b60 2024-01-27T02:44:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.886+0000 7faeeea7f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee80b20e0 con 0x7faee80ad9f0 2024-01-27T02:44:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.886+0000 7faeeea7f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee80b23c0 con 0x7faee80af470 2024-01-27T02:44:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.886+0000 7faeed01c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faee8156b60 0x7faee80ac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:01.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.886+0000 7faeed01c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faee8156b60 0x7faee80ac690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48528/0 (socket says 172.21.15.7:48528) 2024-01-27T02:44:01.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.886+0000 7faeed01c700 1 -- 172.21.15.7:0/1175787889 learned_addr learned my addr 172.21.15.7:0/1175787889 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:01.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.886+0000 7faeec81b700 1 --2- 172.21.15.7:0/1175787889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faee80ad9f0 0x7faee8169fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:01.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.886+0000 7faee7fff700 1 --2- 172.21.15.7:0/1175787889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faee80af470 0x7faee80b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:01.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.887+0000 7faeed01c700 1 -- 172.21.15.7:0/1175787889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faee80af470 msgr2=0x7faee80b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:01.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.887+0000 7faeed01c700 1 --2- 172.21.15.7:0/1175787889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faee80af470 0x7faee80b1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.887+0000 7faeed01c700 1 -- 172.21.15.7:0/1175787889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faee80ad9f0 msgr2=0x7faee8169fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:01.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.887+0000 7faeed01c700 1 --2- 172.21.15.7:0/1175787889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faee80ad9f0 0x7faee8169fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:01.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.887+0000 7faeed01c700 1 -- 172.21.15.7:0/1175787889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faedc009420 con 0x7faee8156b60 2024-01-27T02:44:01.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.887+0000 7faeed01c700 1 --2- 172.21.15.7:0/1175787889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faee8156b60 0x7faee80ac690 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7faee0001460 tx=0x7faee0001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:01.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.887+0000 7faee4ff9700 1 -- 172.21.15.7:0/1175787889 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee000a890 con 0x7faee8156b60 2024-01-27T02:44:01.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.887+0000 7faee4ff9700 1 -- 172.21.15.7:0/1175787889 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee000a9f0 con 0x7faee8156b60 2024-01-27T02:44:01.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.887+0000 7faeeea7f700 1 -- 172.21.15.7:0/1175787889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed4009420 con 0x7faee8156b60 2024-01-27T02:44:01.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.887+0000 7faee4ff9700 1 -- 172.21.15.7:0/1175787889 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee000ad70 con 0x7faee8156b60 2024-01-27T02:44:01.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.887+0000 7faeeea7f700 1 -- 172.21.15.7:0/1175787889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee80b2980 con 0x7faee8156b60 2024-01-27T02:44:01.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.888+0000 7faeeea7f700 1 -- 172.21.15.7:0/1175787889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee80b2e60 con 0x7faee8156b60 2024-01-27T02:44:01.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.889+0000 7faee4ff9700 1 -- 172.21.15.7:0/1175787889 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7faee0007300 con 0x7faee8156b60 2024-01-27T02:44:01.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.889+0000 7faee4ff9700 1 --2- 172.21.15.7:0/1175787889 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faecc066f30 0x7faecc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:01.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.889+0000 7faee4ff9700 1 -- 172.21.15.7:0/1175787889 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faee0084900 con 0x7faee8156b60 2024-01-27T02:44:01.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.889+0000 7faeec81b700 1 --2- 172.21.15.7:0/1175787889 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faecc066f30 0x7faecc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.890+0000 7faeec81b700 1 --2- 172.21.15.7:0/1175787889 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faecc066f30 0x7faecc0693e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7faedc009ea0 tx=0x7faedc0091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:01.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:01.893+0000 7faee4ff9700 1 -- 172.21.15.7:0/1175787889 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faee0051b60 con 0x7faee8156b60 2024-01-27T02:44:02.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.102+0000 7faeeea7f700 1 -- 172.21.15.7:0/1175787889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faee8072060 con 0x7faee8156b60 2024-01-27T02:44:02.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.103+0000 7faee4ff9700 1 -- 172.21.15.7:0/1175787889 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v183) v1 ==== 104+0+11 (secure 0 0 0) 0x7faee0055180 con 0x7faee8156b60 2024-01-27T02:44:02.103 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-27T02:44:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.105+0000 7faeeea7f700 1 -- 172.21.15.7:0/1175787889 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faecc066f30 msgr2=0x7faecc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.105+0000 7faeeea7f700 1 --2- 172.21.15.7:0/1175787889 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faecc066f30 0x7faecc0693e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7faedc009ea0 tx=0x7faedc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.105+0000 7faeeea7f700 1 -- 172.21.15.7:0/1175787889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faee8156b60 msgr2=0x7faee80ac690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.105+0000 7faeeea7f700 1 --2- 172.21.15.7:0/1175787889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faee8156b60 0x7faee80ac690 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7faee0001460 tx=0x7faee0001ae0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.105+0000 7faeeea7f700 1 -- 172.21.15.7:0/1175787889 shutdown_connections 2024-01-27T02:44:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.105+0000 7faeeea7f700 1 --2- 172.21.15.7:0/1175787889 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7faecc066f30 0x7faecc0693e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.105+0000 7faeeea7f700 1 --2- 172.21.15.7:0/1175787889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faee8156b60 0x7faee80ac690 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.105+0000 7faeeea7f700 1 --2- 172.21.15.7:0/1175787889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faee80ad9f0 0x7faee8169fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.105+0000 7faeeea7f700 1 --2- 172.21.15.7:0/1175787889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faee80af470 0x7faee80b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.105+0000 7faeeea7f700 1 -- 172.21.15.7:0/1175787889 >> 172.21.15.7:0/1175787889 conn(0x7faee8068480 msgr2=0x7faee806aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.105+0000 7faeeea7f700 1 -- 172.21.15.7:0/1175787889 shutdown_connections 2024-01-27T02:44:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.105+0000 7faeeea7f700 1 -- 172.21.15.7:0/1175787889 wait complete. 2024-01-27T02:44:02.119 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-27T02:44:02.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.253+0000 7fac11d17700 1 Processor -- start 2024-01-27T02:44:02.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.253+0000 7fac11d17700 1 -- start start 2024-01-27T02:44:02.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.254+0000 7fac11d17700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac0c140510 0x7fac0c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.254+0000 7fac11d17700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac0c0cb7c0 con 0x7fac0c0cc4f0 2024-01-27T02:44:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.254+0000 7fac11d17700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac0c0cc060 con 0x7fac0c1408e0 2024-01-27T02:44:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.254+0000 7fac11d17700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c15b6c0 con 0x7fac0c140510 2024-01-27T02:44:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.254+0000 7fac0bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fac0c0cc4f0 0x7fac0c06b250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56788/0 (socket says 172.21.15.7:56788) 2024-01-27T02:44:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.254+0000 7fac0bfff700 1 -- 172.21.15.7:0/3020538778 learned_addr learned my addr 172.21.15.7:0/3020538778 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.255+0000 7fac0b7fe700 1 --2- 172.21.15.7:0/3020538778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac0c140510 0x7fac0c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.255+0000 7fac0a7fc700 1 -- 172.21.15.7:0/3020538778 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123265010 0 0) 0x7fac0c0cb7c0 con 0x7fac0c0cc4f0 2024-01-27T02:44:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.255+0000 7fac0a7fc700 1 -- 172.21.15.7:0/3020538778 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabf4003570 con 0x7fac0c0cc4f0 2024-01-27T02:44:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.255+0000 7fac0a7fc700 1 -- 172.21.15.7:0/3020538778 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3598205758 0 0) 0x7fac0c0cc060 con 0x7fac0c1408e0 2024-01-27T02:44:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.255+0000 7fac0a7fc700 1 -- 172.21.15.7:0/3020538778 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac0c0cb7c0 con 0x7fac0c1408e0 2024-01-27T02:44:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.255+0000 7fac0b7fe700 1 -- 172.21.15.7:0/3020538778 >> v1:172.21.15.188:6789/0 conn(0x7fac0c1408e0 legacy=0x7fac0c15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.255+0000 7fac0b7fe700 1 -- 172.21.15.7:0/3020538778 >> v1:172.21.15.7:6789/0 conn(0x7fac0c0cc4f0 legacy=0x7fac0c06b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.255+0000 7fac0b7fe700 1 -- 172.21.15.7:0/3020538778 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0c0cba10 con 0x7fac0c140510 2024-01-27T02:44:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.255+0000 7fac0b7fe700 1 --2- 172.21.15.7:0/3020538778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac0c140510 0x7fac0c06e320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fabfc009910 tx=0x7fabfc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13c61e9e98f04b40 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.255+0000 7fac0a7fc700 1 -- 172.21.15.7:0/3020538778 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc00f6b0 con 0x7fac0c140510 2024-01-27T02:44:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.255+0000 7fac0a7fc700 1 -- 172.21.15.7:0/3020538778 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabfc00f810 con 0x7fac0c140510 2024-01-27T02:44:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.255+0000 7fac0a7fc700 1 -- 172.21.15.7:0/3020538778 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc00fbb0 con 0x7fac0c140510 2024-01-27T02:44:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.256+0000 7fac11d17700 1 -- 172.21.15.7:0/3020538778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac0c140510 msgr2=0x7fac0c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.256+0000 7fac11d17700 1 --2- 172.21.15.7:0/3020538778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac0c140510 0x7fac0c06e320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fabfc009910 tx=0x7fabfc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.256+0000 7fac11d17700 1 -- 172.21.15.7:0/3020538778 shutdown_connections 2024-01-27T02:44:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.256+0000 7fac11d17700 1 --2- 172.21.15.7:0/3020538778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac0c140510 0x7fac0c06e320 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.256+0000 7fac11d17700 1 -- 172.21.15.7:0/3020538778 >> 172.21.15.7:0/3020538778 conn(0x7fac0c0cdbf0 msgr2=0x7fac0c0cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.256+0000 7fac11d17700 1 -- 172.21.15.7:0/3020538778 shutdown_connections 2024-01-27T02:44:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.256+0000 7fac11d17700 1 -- 172.21.15.7:0/3020538778 wait complete. 2024-01-27T02:44:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.257+0000 7fac11d17700 1 Processor -- start 2024-01-27T02:44:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.257+0000 7fac11d17700 1 -- start start 2024-01-27T02:44:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.258+0000 7fac11d17700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac0c0cc4f0 0x7fac0c154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.258+0000 7fac11d17700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac0c140510 0x7fac0c0c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.258+0000 7fac11d17700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac0c1408e0 0x7fac0c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.258+0000 7fac11d17700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c0c5fd0 con 0x7fac0c0cc4f0 2024-01-27T02:44:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.258+0000 7fac11d17700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c155aa0 con 0x7fac0c140510 2024-01-27T02:44:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.258+0000 7fac11d17700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c1578a0 con 0x7fac0c1408e0 2024-01-27T02:44:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.258+0000 7fac0b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac0c0cc4f0 0x7fac0c154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.258+0000 7fac0b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac0c0cc4f0 0x7fac0c154610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48536/0 (socket says 172.21.15.7:48536) 2024-01-27T02:44:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.258+0000 7fac0b7fe700 1 -- 172.21.15.7:0/1402495354 learned_addr learned my addr 172.21.15.7:0/1402495354 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.259+0000 7fac0b7fe700 1 -- 172.21.15.7:0/1402495354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac0c1408e0 msgr2=0x7fac0c0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.259+0000 7fac0b7fe700 1 --2- 172.21.15.7:0/1402495354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac0c1408e0 0x7fac0c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.259+0000 7fac0b7fe700 1 -- 172.21.15.7:0/1402495354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac0c140510 msgr2=0x7fac0c0c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.259+0000 7fac0affd700 1 --2- 172.21.15.7:0/1402495354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac0c140510 0x7fac0c0c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.259+0000 7fac0b7fe700 1 --2- 172.21.15.7:0/1402495354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac0c140510 0x7fac0c0c61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.259+0000 7fac0b7fe700 1 -- 172.21.15.7:0/1402495354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabfc009420 con 0x7fac0c0cc4f0 2024-01-27T02:44:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.259+0000 7fac0b7fe700 1 --2- 172.21.15.7:0/1402495354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac0c0cc4f0 0x7fac0c154610 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fabfc0093f0 tx=0x7fabfc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.259+0000 7fabf37fe700 1 -- 172.21.15.7:0/1402495354 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc000c90 con 0x7fac0c0cc4f0 2024-01-27T02:44:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.259+0000 7fabf37fe700 1 -- 172.21.15.7:0/1402495354 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabfc00f980 con 0x7fac0c0cc4f0 2024-01-27T02:44:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.259+0000 7fac11d17700 1 -- 172.21.15.7:0/1402495354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac0c155cf0 con 0x7fac0c0cc4f0 2024-01-27T02:44:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.259+0000 7fabf37fe700 1 -- 172.21.15.7:0/1402495354 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc010470 con 0x7fac0c0cc4f0 2024-01-27T02:44:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.259+0000 7fac11d17700 1 -- 172.21.15.7:0/1402495354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac0c157530 con 0x7fac0c0cc4f0 2024-01-27T02:44:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.260+0000 7fac11d17700 1 -- 172.21.15.7:0/1402495354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac0c157350 con 0x7fac0c0cc4f0 2024-01-27T02:44:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.261+0000 7fabf37fe700 1 -- 172.21.15.7:0/1402495354 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fabfc022070 con 0x7fac0c0cc4f0 2024-01-27T02:44:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.261+0000 7fabf37fe700 1 --2- 172.21.15.7:0/1402495354 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fabf8066f30 0x7fabf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.261+0000 7fabf37fe700 1 -- 172.21.15.7:0/1402495354 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fabfc014070 con 0x7fac0c0cc4f0 2024-01-27T02:44:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.262+0000 7fac0affd700 1 --2- 172.21.15.7:0/1402495354 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fabf8066f30 0x7fabf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.262+0000 7fac0affd700 1 --2- 172.21.15.7:0/1402495354 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fabf8066f30 0x7fabf80693e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fac0c159c20 tx=0x7fac00009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.266+0000 7fabf37fe700 1 -- 172.21.15.7:0/1402495354 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fabfc052970 con 0x7fac0c0cc4f0 2024-01-27T02:44:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.468+0000 7fac11d17700 1 -- 172.21.15.7:0/1402495354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fac0c072060 con 0x7fac0c0cc4f0 2024-01-27T02:44:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.468+0000 7fabf37fe700 1 -- 172.21.15.7:0/1402495354 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v183) v1 ==== 100+0+11 (secure 0 0 0) 0x7fabfc024420 con 0x7fac0c0cc4f0 2024-01-27T02:44:02.469 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-27T02:44:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.470+0000 7fac11d17700 1 -- 172.21.15.7:0/1402495354 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fabf8066f30 msgr2=0x7fabf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.470+0000 7fac11d17700 1 --2- 172.21.15.7:0/1402495354 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fabf8066f30 0x7fabf80693e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fac0c159c20 tx=0x7fac00009040 comp rx=0 tx=0).stop 2024-01-27T02:44:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.470+0000 7fac11d17700 1 -- 172.21.15.7:0/1402495354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac0c0cc4f0 msgr2=0x7fac0c154610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.470+0000 7fac11d17700 1 --2- 172.21.15.7:0/1402495354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac0c0cc4f0 0x7fac0c154610 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fabfc0093f0 tx=0x7fabfc019000 comp rx=0 tx=0).stop 2024-01-27T02:44:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.470+0000 7fac11d17700 1 -- 172.21.15.7:0/1402495354 shutdown_connections 2024-01-27T02:44:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.470+0000 7fac11d17700 1 --2- 172.21.15.7:0/1402495354 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fabf8066f30 0x7fabf80693e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.470+0000 7fac11d17700 1 --2- 172.21.15.7:0/1402495354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac0c1408e0 0x7fac0c0c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.470+0000 7fac11d17700 1 --2- 172.21.15.7:0/1402495354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac0c0cc4f0 0x7fac0c154610 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.470+0000 7fac11d17700 1 --2- 172.21.15.7:0/1402495354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac0c140510 0x7fac0c0c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.470+0000 7fac11d17700 1 -- 172.21.15.7:0/1402495354 >> 172.21.15.7:0/1402495354 conn(0x7fac0c0cdbf0 msgr2=0x7fac0c06e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.471+0000 7fac11d17700 1 -- 172.21.15.7:0/1402495354 shutdown_connections 2024-01-27T02:44:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.471+0000 7fac11d17700 1 -- 172.21.15.7:0/1402495354 wait complete. 2024-01-27T02:44:02.485 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:44:02.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.618+0000 7f1bf5cb2700 1 Processor -- start 2024-01-27T02:44:02.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.619+0000 7f1bf5cb2700 1 -- start start 2024-01-27T02:44:02.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.619+0000 7f1bf5cb2700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1bf0157ca0 0x7f1bf00c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:02.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.619+0000 7f1bf5cb2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bf00cdbf0 0x7f1bf00ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:02.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.619+0000 7f1bf5cb2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bf00ca020 con 0x7f1bf00c3ae0 2024-01-27T02:44:02.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.619+0000 7f1bf5cb2700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bf00c4f40 con 0x7f1bf0157ca0 2024-01-27T02:44:02.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.619+0000 7f1bf5cb2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf00c5210 con 0x7f1bf00cdbf0 2024-01-27T02:44:02.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.620+0000 7f1beeffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1bf00c3ae0 0x7f1bf00cd4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56802/0 (socket says 172.21.15.7:56802) 2024-01-27T02:44:02.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.620+0000 7f1beeffd700 1 -- 172.21.15.7:0/1209808376 learned_addr learned my addr 172.21.15.7:0/1209808376 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:02.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.620+0000 7f1bef7fe700 1 --2- 172.21.15.7:0/1209808376 >> v2:172.21.15.188:3300/0 conn(0x7f1bf0157ca0 0x7f1bf00c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:02.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.620+0000 7f1beffff700 1 --2- 172.21.15.7:0/1209808376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bf00cdbf0 0x7f1bf00ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:02.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.620+0000 7f1bee7fc700 1 -- 172.21.15.7:0/1209808376 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2023276806 0 0) 0x7f1bf00ca020 con 0x7f1bf00c3ae0 2024-01-27T02:44:02.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.620+0000 7f1bee7fc700 1 -- 172.21.15.7:0/1209808376 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bd8003570 con 0x7f1bf00c3ae0 2024-01-27T02:44:02.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.620+0000 7f1beffff700 1 -- 172.21.15.7:0/1209808376 >> v2:172.21.15.188:3300/0 conn(0x7f1bf0157ca0 msgr2=0x7f1bf00c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:02.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.620+0000 7f1beffff700 1 --2- 172.21.15.7:0/1209808376 >> v2:172.21.15.188:3300/0 conn(0x7f1bf0157ca0 0x7f1bf00c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.620+0000 7f1beffff700 1 -- 172.21.15.7:0/1209808376 >> v1:172.21.15.7:6789/0 conn(0x7f1bf00c3ae0 legacy=0x7f1bf00cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:02.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.620+0000 7f1beffff700 1 -- 172.21.15.7:0/1209808376 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf00c54f0 con 0x7f1bf00cdbf0 2024-01-27T02:44:02.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.620+0000 7f1beffff700 1 --2- 172.21.15.7:0/1209808376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bf00cdbf0 0x7f1bf00ca4b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009e60 tx=0x7f1be8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=524a82296a263781 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:02.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.621+0000 7f1bee7fc700 1 -- 172.21.15.7:0/1209808376 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be800e070 con 0x7f1bf00cdbf0 2024-01-27T02:44:02.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.621+0000 7f1bee7fc700 1 -- 172.21.15.7:0/1209808376 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be800bab0 con 0x7f1bf00cdbf0 2024-01-27T02:44:02.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.621+0000 7f1bee7fc700 1 -- 172.21.15.7:0/1209808376 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be800be70 con 0x7f1bf00cdbf0 2024-01-27T02:44:02.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.621+0000 7f1bf5cb2700 1 -- 172.21.15.7:0/1209808376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bf00cdbf0 msgr2=0x7f1bf00ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:02.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.621+0000 7f1bf5cb2700 1 --2- 172.21.15.7:0/1209808376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bf00cdbf0 0x7f1bf00ca4b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009e60 tx=0x7f1be8009780 comp rx=0 tx=0).stop 2024-01-27T02:44:02.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.621+0000 7f1bf5cb2700 1 -- 172.21.15.7:0/1209808376 shutdown_connections 2024-01-27T02:44:02.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.621+0000 7f1bf5cb2700 1 --2- 172.21.15.7:0/1209808376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bf00cdbf0 0x7f1bf00ca4b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.621+0000 7f1bf5cb2700 1 --2- 172.21.15.7:0/1209808376 >> v2:172.21.15.188:3300/0 conn(0x7f1bf0157ca0 0x7f1bf00c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.621+0000 7f1bf5cb2700 1 -- 172.21.15.7:0/1209808376 >> 172.21.15.7:0/1209808376 conn(0x7f1bf0098020 msgr2=0x7f1bf00711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:02.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.621+0000 7f1bf5cb2700 1 -- 172.21.15.7:0/1209808376 shutdown_connections 2024-01-27T02:44:02.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.621+0000 7f1bf5cb2700 1 -- 172.21.15.7:0/1209808376 wait complete. 2024-01-27T02:44:02.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.623+0000 7f1bf5cb2700 1 Processor -- start 2024-01-27T02:44:02.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.623+0000 7f1bf5cb2700 1 -- start start 2024-01-27T02:44:02.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.623+0000 7f1bf5cb2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bf00c3ae0 0x7f1bf014d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:02.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.623+0000 7f1bf5cb2700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bf00cdbf0 0x7f1bf00cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:02.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.623+0000 7f1bf5cb2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bf0157ca0 0x7f1bf00cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:02.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.623+0000 7f1bf5cb2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf00b4980 con 0x7f1bf0157ca0 2024-01-27T02:44:02.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.623+0000 7f1bf5cb2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf00b38e0 con 0x7f1bf00c3ae0 2024-01-27T02:44:02.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.623+0000 7f1bf5cb2700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf00b3bc0 con 0x7f1bf00cdbf0 2024-01-27T02:44:02.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.624+0000 7f1beffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bf0157ca0 0x7f1bf00cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:02.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.624+0000 7f1beffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bf0157ca0 0x7f1bf00cb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48540/0 (socket says 172.21.15.7:48540) 2024-01-27T02:44:02.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.624+0000 7f1beffff700 1 -- 172.21.15.7:0/350588300 learned_addr learned my addr 172.21.15.7:0/350588300 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:02.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.624+0000 7f1bef7fe700 1 --2- 172.21.15.7:0/350588300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bf00c3ae0 0x7f1bf014d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:02.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.624+0000 7f1beeffd700 1 --2- 172.21.15.7:0/350588300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bf00cdbf0 0x7f1bf00cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:02.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.624+0000 7f1beffff700 1 -- 172.21.15.7:0/350588300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bf00cdbf0 msgr2=0x7f1bf00cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:02.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.624+0000 7f1beffff700 1 --2- 172.21.15.7:0/350588300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bf00cdbf0 0x7f1bf00cacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.624+0000 7f1beffff700 1 -- 172.21.15.7:0/350588300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bf00c3ae0 msgr2=0x7f1bf014d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:02.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.624+0000 7f1beffff700 1 --2- 172.21.15.7:0/350588300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bf00c3ae0 0x7f1bf014d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.624+0000 7f1beffff700 1 -- 172.21.15.7:0/350588300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be8009a20 con 0x7f1bf0157ca0 2024-01-27T02:44:02.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.624+0000 7f1beffff700 1 --2- 172.21.15.7:0/350588300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bf0157ca0 0x7f1bf00cb1f0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f1be8013000 tx=0x7f1be8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:02.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.624+0000 7f1bd77fe700 1 -- 172.21.15.7:0/350588300 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be8012e10 con 0x7f1bf0157ca0 2024-01-27T02:44:02.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.624+0000 7f1bd77fe700 1 -- 172.21.15.7:0/350588300 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be8027470 con 0x7f1bf0157ca0 2024-01-27T02:44:02.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.624+0000 7f1bf5cb2700 1 -- 172.21.15.7:0/350588300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bf00b3ea0 con 0x7f1bf0157ca0 2024-01-27T02:44:02.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.624+0000 7f1bd77fe700 1 -- 172.21.15.7:0/350588300 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be8027830 con 0x7f1bf0157ca0 2024-01-27T02:44:02.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.624+0000 7f1bf5cb2700 1 -- 172.21.15.7:0/350588300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bf00b4460 con 0x7f1bf0157ca0 2024-01-27T02:44:02.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.625+0000 7f1bf5cb2700 1 -- 172.21.15.7:0/350588300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bf00b4680 con 0x7f1bf0157ca0 2024-01-27T02:44:02.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.626+0000 7f1bd77fe700 1 -- 172.21.15.7:0/350588300 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1be8024070 con 0x7f1bf0157ca0 2024-01-27T02:44:02.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.626+0000 7f1bd77fe700 1 --2- 172.21.15.7:0/350588300 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1bdc066f30 0x7f1bdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:02.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.627+0000 7f1bd77fe700 1 -- 172.21.15.7:0/350588300 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1be8086f30 con 0x7f1bf0157ca0 2024-01-27T02:44:02.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.627+0000 7f1bef7fe700 1 --2- 172.21.15.7:0/350588300 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1bdc066f30 0x7f1bdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:02.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.627+0000 7f1bef7fe700 1 --2- 172.21.15.7:0/350588300 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1bdc066f30 0x7f1bdc0693e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009d10 tx=0x7f1be00091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.631+0000 7f1bd77fe700 1 -- 172.21.15.7:0/350588300 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1be80541a0 con 0x7f1bf0157ca0 2024-01-27T02:44:02.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.882+0000 7f1bf5cb2700 1 -- 172.21.15.7:0/350588300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1bf00b2f40 con 0x7f1bf0157ca0 2024-01-27T02:44:02.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.882+0000 7f1bd77fe700 1 -- 172.21.15.7:0/350588300 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4595 (secure 0 0 0) 0x7f1be80577c0 con 0x7f1bf0157ca0 2024-01-27T02:44:02.883 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:44:02.883 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6704,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/1199331492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1199331492},{"type":"v1","addr":"172.21.15.7:6834","nonce":1199331492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":6707,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/721335086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":721335086},{"type":"v1","addr":"172.21.15.7:6835","nonce":721335086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":7699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/450555206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":450555206},{"type":"v1","addr":"172.21.15.188:6836","nonce":450555206}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:43:29.599748+0000","modified":"2024-01-27T02:43:33.196288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7702":{"gid":7702,"name":"b","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/3853795510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3853795510},{"type":"v1","addr":"172.21.15.188:6837","nonce":3853795510}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-27T02:44:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.884+0000 7f1bf5cb2700 1 -- 172.21.15.7:0/350588300 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1bdc066f30 msgr2=0x7f1bdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.884+0000 7f1bf5cb2700 1 --2- 172.21.15.7:0/350588300 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1bdc066f30 0x7f1bdc0693e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009d10 tx=0x7f1be00091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.884+0000 7f1bf5cb2700 1 -- 172.21.15.7:0/350588300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bf0157ca0 msgr2=0x7f1bf00cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.884+0000 7f1bf5cb2700 1 --2- 172.21.15.7:0/350588300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bf0157ca0 0x7f1bf00cb1f0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f1be8013000 tx=0x7f1be8019000 comp rx=0 tx=0).stop 2024-01-27T02:44:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.884+0000 7f1bf5cb2700 1 -- 172.21.15.7:0/350588300 shutdown_connections 2024-01-27T02:44:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.884+0000 7f1bf5cb2700 1 --2- 172.21.15.7:0/350588300 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1bdc066f30 0x7f1bdc0693e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.884+0000 7f1bf5cb2700 1 --2- 172.21.15.7:0/350588300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bf0157ca0 0x7f1bf00cb1f0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.884+0000 7f1bf5cb2700 1 --2- 172.21.15.7:0/350588300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1bf00c3ae0 0x7f1bf014d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.884+0000 7f1bf5cb2700 1 --2- 172.21.15.7:0/350588300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1bf00cdbf0 0x7f1bf00cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:02.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.884+0000 7f1bf5cb2700 1 -- 172.21.15.7:0/350588300 >> 172.21.15.7:0/350588300 conn(0x7f1bf0098020 msgr2=0x7f1bf00ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:02.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.884+0000 7f1bf5cb2700 1 -- 172.21.15.7:0/350588300 shutdown_connections 2024-01-27T02:44:02.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:02.884+0000 7f1bf5cb2700 1 -- 172.21.15.7:0/350588300 wait complete. 2024-01-27T02:44:02.886 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 94 2024-01-27T02:44:02.899 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:44:03.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.030+0000 7f86c3062700 1 Processor -- start 2024-01-27T02:44:03.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.030+0000 7f86c3062700 1 -- start start 2024-01-27T02:44:03.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.031+0000 7f86c3062700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f86bc0d10a0 0x7f86bc0b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:03.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.031+0000 7f86c3062700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86bc159b40 0x7f86bc159f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:03.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.031+0000 7f86c3062700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86bc0b4650 con 0x7f86bc0d1470 2024-01-27T02:44:03.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.032+0000 7f86c3062700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86bc0b34d0 con 0x7f86bc0d10a0 2024-01-27T02:44:03.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.032+0000 7f86c3062700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc0b3d60 con 0x7f86bc159b40 2024-01-27T02:44:03.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.032+0000 7f86c0dfe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f86bc0d10a0 0x7f86bc0b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:03.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.032+0000 7f86c0dfe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f86bc0d10a0 0x7f86bc0b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48546/0 (socket says 172.21.15.7:48546) 2024-01-27T02:44:03.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.032+0000 7f86c0dfe700 1 -- 172.21.15.7:0/2373737741 learned_addr learned my addr 172.21.15.7:0/2373737741 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:03.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.032+0000 7f86c15ff700 1 --2- 172.21.15.7:0/2373737741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86bc159b40 0x7f86bc159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:03.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.032+0000 7f86c0dfe700 1 -- 172.21.15.7:0/2373737741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86bc159b40 msgr2=0x7f86bc159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:03.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.032+0000 7f86c0dfe700 1 --2- 172.21.15.7:0/2373737741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86bc159b40 0x7f86bc159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:03.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.032+0000 7f86c0dfe700 1 -- 172.21.15.7:0/2373737741 >> v1:172.21.15.188:6789/0 conn(0x7f86bc0d1470 legacy=0x7f86bc0b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:03.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.032+0000 7f86c0dfe700 1 -- 172.21.15.7:0/2373737741 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc0b4000 con 0x7f86bc0d10a0 2024-01-27T02:44:03.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.032+0000 7f86c0dfe700 1 --2- 172.21.15.7:0/2373737741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86bc0d10a0 0x7f86bc0b1590 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0099b0 tx=0x7f86ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2f15585b8a3fc9 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:03.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.032+0000 7f86bb7fe700 1 -- 172.21.15.7:0/2373737741 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac009ea0 con 0x7f86bc0d10a0 2024-01-27T02:44:03.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.032+0000 7f86bb7fe700 1 -- 172.21.15.7:0/2373737741 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ac00b860 con 0x7f86bc0d10a0 2024-01-27T02:44:03.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.033+0000 7f86bb7fe700 1 -- 172.21.15.7:0/2373737741 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac00bc00 con 0x7f86bc0d10a0 2024-01-27T02:44:03.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.033+0000 7f86c3062700 1 -- 172.21.15.7:0/2373737741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86bc0d10a0 msgr2=0x7f86bc0b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:03.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.033+0000 7f86c3062700 1 --2- 172.21.15.7:0/2373737741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86bc0d10a0 0x7f86bc0b1590 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0099b0 tx=0x7f86ac009250 comp rx=0 tx=0).stop 2024-01-27T02:44:03.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.033+0000 7f86c3062700 1 -- 172.21.15.7:0/2373737741 shutdown_connections 2024-01-27T02:44:03.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.033+0000 7f86c3062700 1 --2- 172.21.15.7:0/2373737741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86bc159b40 0x7f86bc159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:03.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.033+0000 7f86c3062700 1 --2- 172.21.15.7:0/2373737741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86bc0d10a0 0x7f86bc0b1590 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:03.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.033+0000 7f86c3062700 1 -- 172.21.15.7:0/2373737741 >> 172.21.15.7:0/2373737741 conn(0x7f86bc0679b0 msgr2=0x7f86bc067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:03.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.033+0000 7f86c3062700 1 -- 172.21.15.7:0/2373737741 shutdown_connections 2024-01-27T02:44:03.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.033+0000 7f86c3062700 1 -- 172.21.15.7:0/2373737741 wait complete. 2024-01-27T02:44:03.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.034+0000 7f86c3062700 1 Processor -- start 2024-01-27T02:44:03.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.035+0000 7f86c3062700 1 -- start start 2024-01-27T02:44:03.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.036+0000 7f86c3062700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86bc0d10a0 0x7f86bc0c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:03.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.036+0000 7f86c3062700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86bc0d1470 0x7f86bc16ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:03.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.036+0000 7f86c3062700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86bc159b40 0x7f86bc0c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:03.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.036+0000 7f86c3062700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc0c7f00 con 0x7f86bc159b40 2024-01-27T02:44:03.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.036+0000 7f86c3062700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc0c8d60 con 0x7f86bc0d1470 2024-01-27T02:44:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.036+0000 7f86c3062700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc0c62a0 con 0x7f86bc0d10a0 2024-01-27T02:44:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.036+0000 7f86c15ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86bc159b40 0x7f86bc0c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.036+0000 7f86c15ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86bc159b40 0x7f86bc0c91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48548/0 (socket says 172.21.15.7:48548) 2024-01-27T02:44:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.036+0000 7f86c15ff700 1 -- 172.21.15.7:0/2327023365 learned_addr learned my addr 172.21.15.7:0/2327023365 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.036+0000 7f86c0dfe700 1 --2- 172.21.15.7:0/2327023365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86bc0d10a0 0x7f86bc0c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.036+0000 7f86bbfff700 1 --2- 172.21.15.7:0/2327023365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86bc0d1470 0x7f86bc16ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.036+0000 7f86c15ff700 1 -- 172.21.15.7:0/2327023365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86bc0d10a0 msgr2=0x7f86bc0c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.036+0000 7f86c15ff700 1 --2- 172.21.15.7:0/2327023365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86bc0d10a0 0x7f86bc0c5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.036+0000 7f86c15ff700 1 -- 172.21.15.7:0/2327023365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86bc0d1470 msgr2=0x7f86bc16ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.036+0000 7f86c15ff700 1 --2- 172.21.15.7:0/2327023365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86bc0d1470 0x7f86bc16ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.036+0000 7f86c15ff700 1 -- 172.21.15.7:0/2327023365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ac0094f0 con 0x7f86bc159b40 2024-01-27T02:44:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.036+0000 7f86c15ff700 1 --2- 172.21.15.7:0/2327023365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86bc159b40 0x7f86bc0c91f0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009df0 tx=0x7f86b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.036+0000 7f86b8ff9700 1 -- 172.21.15.7:0/2327023365 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b400caa0 con 0x7f86bc159b40 2024-01-27T02:44:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.036+0000 7f86b8ff9700 1 -- 172.21.15.7:0/2327023365 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b400f040 con 0x7f86bc159b40 2024-01-27T02:44:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.037+0000 7f86c3062700 1 -- 172.21.15.7:0/2327023365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86bc0c8150 con 0x7f86bc159b40 2024-01-27T02:44:03.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.037+0000 7f86b8ff9700 1 -- 172.21.15.7:0/2327023365 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b400ce70 con 0x7f86bc159b40 2024-01-27T02:44:03.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.037+0000 7f86c3062700 1 -- 172.21.15.7:0/2327023365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86bc0c8710 con 0x7f86bc159b40 2024-01-27T02:44:03.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.037+0000 7f86c3062700 1 -- 172.21.15.7:0/2327023365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86bc04c710 con 0x7f86bc159b40 2024-01-27T02:44:03.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.038+0000 7f86b8ff9700 1 -- 172.21.15.7:0/2327023365 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f86b4014690 con 0x7f86bc159b40 2024-01-27T02:44:03.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.039+0000 7f86b8ff9700 1 --2- 172.21.15.7:0/2327023365 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f869c066f30 0x7f869c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:03.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.039+0000 7f86b8ff9700 1 -- 172.21.15.7:0/2327023365 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f86b4085640 con 0x7f86bc159b40 2024-01-27T02:44:03.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.039+0000 7f86c0dfe700 1 --2- 172.21.15.7:0/2327023365 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f869c066f30 0x7f869c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:03.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.040+0000 7f86c0dfe700 1 --2- 172.21.15.7:0/2327023365 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f869c066f30 0x7f869c0693e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0099b0 tx=0x7f86ac017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.043+0000 7f86b8ff9700 1 -- 172.21.15.7:0/2327023365 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f86b4056320 con 0x7f86bc159b40 2024-01-27T02:44:03.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.245+0000 7f86c3062700 1 -- 172.21.15.7:0/2327023365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86bc0b42e0 con 0x7f86bc159b40 2024-01-27T02:44:03.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.246+0000 7f86b8ff9700 1 -- 172.21.15.7:0/2327023365 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+15324 (secure 0 0 0) 0x7f86b4055eb0 con 0x7f86bc159b40 2024-01-27T02:44:03.246 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:44:03.247 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":183,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:43:54.936156+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:43:27.492968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"179","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-01-27T02:43:28.579977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:44:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.248+0000 7f86c3062700 1 -- 172.21.15.7:0/2327023365 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f869c066f30 msgr2=0x7f869c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.248+0000 7f86c3062700 1 --2- 172.21.15.7:0/2327023365 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f869c066f30 0x7f869c0693e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0099b0 tx=0x7f86ac017000 comp rx=0 tx=0).stop 2024-01-27T02:44:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.248+0000 7f86c3062700 1 -- 172.21.15.7:0/2327023365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86bc159b40 msgr2=0x7f86bc0c91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.248+0000 7f86c3062700 1 --2- 172.21.15.7:0/2327023365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86bc159b40 0x7f86bc0c91f0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009df0 tx=0x7f86b400b040 comp rx=0 tx=0).stop 2024-01-27T02:44:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.248+0000 7f86c3062700 1 -- 172.21.15.7:0/2327023365 shutdown_connections 2024-01-27T02:44:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.248+0000 7f86c3062700 1 --2- 172.21.15.7:0/2327023365 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f869c066f30 0x7f869c0693e0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.248+0000 7f86c3062700 1 --2- 172.21.15.7:0/2327023365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86bc159b40 0x7f86bc0c91f0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.248+0000 7f86c3062700 1 --2- 172.21.15.7:0/2327023365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86bc0d10a0 0x7f86bc0c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.248+0000 7f86c3062700 1 --2- 172.21.15.7:0/2327023365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86bc0d1470 0x7f86bc16ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.248+0000 7f86c3062700 1 -- 172.21.15.7:0/2327023365 >> 172.21.15.7:0/2327023365 conn(0x7f86bc0679b0 msgr2=0x7f86bc0cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.248+0000 7f86c3062700 1 -- 172.21.15.7:0/2327023365 shutdown_connections 2024-01-27T02:44:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.248+0000 7f86c3062700 1 -- 172.21.15.7:0/2327023365 wait complete. 2024-01-27T02:44:03.262 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-27T02:44:03.263 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:44:03.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.394+0000 7f5676995700 1 Processor -- start 2024-01-27T02:44:03.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.394+0000 7f5676995700 1 -- start start 2024-01-27T02:44:03.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.395+0000 7f5676995700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f567015b5f0 0x7f56701566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:03.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.395+0000 7f5676995700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5670157610 con 0x7f56700b1c00 2024-01-27T02:44:03.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.395+0000 7f5676995700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5670157310 con 0x7f56700b1fd0 2024-01-27T02:44:03.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.395+0000 7f5676995700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f567006ee00 con 0x7f567015b5f0 2024-01-27T02:44:03.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.395+0000 7f566f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f567015b5f0 0x7f56701566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:03.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.395+0000 7f566ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f56700b1c00 0x7f567006e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56810/0 (socket says 172.21.15.7:56810) 2024-01-27T02:44:03.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.395+0000 7f566ffff700 1 -- 172.21.15.7:0/3376000148 learned_addr learned my addr 172.21.15.7:0/3376000148 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:03.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.395+0000 7f566effd700 1 -- 172.21.15.7:0/3376000148 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1173679575 0 0) 0x7f5670157610 con 0x7f56700b1c00 2024-01-27T02:44:03.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.395+0000 7f566effd700 1 -- 172.21.15.7:0/3376000148 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f565c003570 con 0x7f56700b1c00 2024-01-27T02:44:03.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.395+0000 7f566f7fe700 1 -- 172.21.15.7:0/3376000148 >> v1:172.21.15.188:6789/0 conn(0x7f56700b1fd0 legacy=0x7f5670156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:03.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.395+0000 7f566f7fe700 1 -- 172.21.15.7:0/3376000148 >> v1:172.21.15.7:6789/0 conn(0x7f56700b1c00 legacy=0x7f567006e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:03.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.395+0000 7f566f7fe700 1 -- 172.21.15.7:0/3376000148 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567006f0e0 con 0x7f567015b5f0 2024-01-27T02:44:03.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.395+0000 7f566f7fe700 1 --2- 172.21.15.7:0/3376000148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f567015b5f0 0x7f56701566c0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f56580098e0 tx=0x7f56580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265975a95d12287c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:03.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.396+0000 7f566effd700 1 -- 172.21.15.7:0/3376000148 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565800b640 con 0x7f567015b5f0 2024-01-27T02:44:03.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.396+0000 7f566effd700 1 -- 172.21.15.7:0/3376000148 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565800b7a0 con 0x7f567015b5f0 2024-01-27T02:44:03.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.396+0000 7f566effd700 1 -- 172.21.15.7:0/3376000148 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565800bb20 con 0x7f567015b5f0 2024-01-27T02:44:03.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.396+0000 7f5676995700 1 -- 172.21.15.7:0/3376000148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f567015b5f0 msgr2=0x7f56701566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:03.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.396+0000 7f5676995700 1 --2- 172.21.15.7:0/3376000148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f567015b5f0 0x7f56701566c0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f56580098e0 tx=0x7f56580091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:03.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.396+0000 7f5676995700 1 -- 172.21.15.7:0/3376000148 shutdown_connections 2024-01-27T02:44:03.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.396+0000 7f5676995700 1 --2- 172.21.15.7:0/3376000148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f567015b5f0 0x7f56701566c0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:03.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.396+0000 7f5676995700 1 -- 172.21.15.7:0/3376000148 >> 172.21.15.7:0/3376000148 conn(0x7f5670068430 msgr2=0x7f567006be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:03.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.396+0000 7f5676995700 1 -- 172.21.15.7:0/3376000148 shutdown_connections 2024-01-27T02:44:03.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.396+0000 7f5676995700 1 -- 172.21.15.7:0/3376000148 wait complete. 2024-01-27T02:44:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.397+0000 7f5676995700 1 Processor -- start 2024-01-27T02:44:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.398+0000 7f5676995700 1 -- start start 2024-01-27T02:44:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.398+0000 7f5676995700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56700b1c00 0x7f567014d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.398+0000 7f5676995700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56700b1fd0 0x7f56701683f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.398+0000 7f5676995700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f567015b5f0 0x7f567014cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.398+0000 7f5676995700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567014d080 con 0x7f56700b1c00 2024-01-27T02:44:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.398+0000 7f5676995700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5670168ab0 con 0x7f567015b5f0 2024-01-27T02:44:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.398+0000 7f5676995700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5670168d90 con 0x7f56700b1fd0 2024-01-27T02:44:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.399+0000 7f566ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56700b1c00 0x7f567014d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.399+0000 7f566ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56700b1c00 0x7f567014d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48572/0 (socket says 172.21.15.7:48572) 2024-01-27T02:44:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.399+0000 7f566ffff700 1 -- 172.21.15.7:0/1154484143 learned_addr learned my addr 172.21.15.7:0/1154484143 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.399+0000 7f5674f32700 1 --2- 172.21.15.7:0/1154484143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f567015b5f0 0x7f567014cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:03.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.399+0000 7f566f7fe700 1 --2- 172.21.15.7:0/1154484143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56700b1fd0 0x7f56701683f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:03.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.399+0000 7f566ffff700 1 -- 172.21.15.7:0/1154484143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56700b1fd0 msgr2=0x7f56701683f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:03.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.399+0000 7f566ffff700 1 --2- 172.21.15.7:0/1154484143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56700b1fd0 0x7f56701683f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:03.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.399+0000 7f566ffff700 1 -- 172.21.15.7:0/1154484143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f567015b5f0 msgr2=0x7f567014cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:03.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.399+0000 7f566ffff700 1 --2- 172.21.15.7:0/1154484143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f567015b5f0 0x7f567014cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:03.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.399+0000 7f566ffff700 1 -- 172.21.15.7:0/1154484143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5658009420 con 0x7f56700b1c00 2024-01-27T02:44:03.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.399+0000 7f566ffff700 1 --2- 172.21.15.7:0/1154484143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56700b1c00 0x7f567014d2e0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f566000c7c0 tx=0x7f5660007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:03.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.399+0000 7f5667fff700 1 -- 172.21.15.7:0/1154484143 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5660005cd0 con 0x7f56700b1c00 2024-01-27T02:44:03.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.399+0000 7f5667fff700 1 -- 172.21.15.7:0/1154484143 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5660005e30 con 0x7f56700b1c00 2024-01-27T02:44:03.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.399+0000 7f5667fff700 1 -- 172.21.15.7:0/1154484143 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5660007320 con 0x7f56700b1c00 2024-01-27T02:44:03.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.399+0000 7f5676995700 1 -- 172.21.15.7:0/1154484143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5670169070 con 0x7f56700b1c00 2024-01-27T02:44:03.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.399+0000 7f5676995700 1 -- 172.21.15.7:0/1154484143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5670169350 con 0x7f56700b1c00 2024-01-27T02:44:03.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.401+0000 7f5676995700 1 -- 172.21.15.7:0/1154484143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5670151350 con 0x7f56700b1c00 2024-01-27T02:44:03.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.401+0000 7f5667fff700 1 -- 172.21.15.7:0/1154484143 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f566001b3b0 con 0x7f56700b1c00 2024-01-27T02:44:03.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.401+0000 7f5667fff700 1 --2- 172.21.15.7:0/1154484143 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5650066f30 0x7f56500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.402+0000 7f5667fff700 1 -- 172.21.15.7:0/1154484143 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5660083ba0 con 0x7f56700b1c00 2024-01-27T02:44:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.402+0000 7f566f7fe700 1 --2- 172.21.15.7:0/1154484143 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5650066f30 0x7f56500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.402+0000 7f566f7fe700 1 --2- 172.21.15.7:0/1154484143 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5650066f30 0x7f56500693e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f56580091e0 tx=0x7f5658017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.406+0000 7f5667fff700 1 -- 172.21.15.7:0/1154484143 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5670151350 con 0x7f56700b1c00 2024-01-27T02:44:03.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.653+0000 7f5676995700 1 -- 172.21.15.7:0/1154484143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5670151350 con 0x7f56700b1c00 2024-01-27T02:44:03.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.653+0000 7f5667fff700 1 -- 172.21.15.7:0/1154484143 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4595 (secure 0 0 0) 0x7f5670151350 con 0x7f56700b1c00 2024-01-27T02:44:03.654 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:44:03.654 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6704,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/1199331492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1199331492},{"type":"v1","addr":"172.21.15.7:6834","nonce":1199331492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":6707,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/721335086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":721335086},{"type":"v1","addr":"172.21.15.7:6835","nonce":721335086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":7699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/450555206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":450555206},{"type":"v1","addr":"172.21.15.188:6836","nonce":450555206}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:43:29.599748+0000","modified":"2024-01-27T02:43:33.196288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7702":{"gid":7702,"name":"b","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/3853795510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3853795510},{"type":"v1","addr":"172.21.15.188:6837","nonce":3853795510}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-27T02:44:03.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.655+0000 7f5676995700 1 -- 172.21.15.7:0/1154484143 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5650066f30 msgr2=0x7f56500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:03.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.655+0000 7f5676995700 1 --2- 172.21.15.7:0/1154484143 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5650066f30 0x7f56500693e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f56580091e0 tx=0x7f5658017000 comp rx=0 tx=0).stop 2024-01-27T02:44:03.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.655+0000 7f5676995700 1 -- 172.21.15.7:0/1154484143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56700b1c00 msgr2=0x7f567014d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:03.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.655+0000 7f5676995700 1 --2- 172.21.15.7:0/1154484143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56700b1c00 0x7f567014d2e0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f566000c7c0 tx=0x7f5660007e00 comp rx=0 tx=0).stop 2024-01-27T02:44:03.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.655+0000 7f5676995700 1 -- 172.21.15.7:0/1154484143 shutdown_connections 2024-01-27T02:44:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.655+0000 7f5676995700 1 --2- 172.21.15.7:0/1154484143 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5650066f30 0x7f56500693e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.655+0000 7f5676995700 1 --2- 172.21.15.7:0/1154484143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f567015b5f0 0x7f567014cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.655+0000 7f5676995700 1 --2- 172.21.15.7:0/1154484143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56700b1c00 0x7f567014d2e0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.655+0000 7f5676995700 1 --2- 172.21.15.7:0/1154484143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56700b1fd0 0x7f56701683f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.655+0000 7f5676995700 1 -- 172.21.15.7:0/1154484143 >> 172.21.15.7:0/1154484143 conn(0x7f5670068430 msgr2=0x7f567006e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.655+0000 7f5676995700 1 -- 172.21.15.7:0/1154484143 shutdown_connections 2024-01-27T02:44:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.655+0000 7f5676995700 1 -- 172.21.15.7:0/1154484143 wait complete. 2024-01-27T02:44:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 94 2024-01-27T02:44:03.670 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:44:03.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.801+0000 7f217a4ae700 1 Processor -- start 2024-01-27T02:44:03.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.802+0000 7f217a4ae700 1 -- start start 2024-01-27T02:44:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.802+0000 7f217a4ae700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2174159f40 0x7f21740c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.802+0000 7f217a4ae700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21740c30c0 con 0x7f21740c3a90 2024-01-27T02:44:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.802+0000 7f217a4ae700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f217415e960 con 0x7f21740aafc0 2024-01-27T02:44:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.802+0000 7f217a4ae700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21740c4550 con 0x7f2174159f40 2024-01-27T02:44:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.803+0000 7f2173fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2174159f40 0x7f21740c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.803+0000 7f21737fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f21740c3a90 0x7f21740aa8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56816/0 (socket says 172.21.15.7:56816) 2024-01-27T02:44:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.803+0000 7f21737fe700 1 -- 172.21.15.7:0/298688873 learned_addr learned my addr 172.21.15.7:0/298688873 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.803+0000 7f2173fff700 1 -- 172.21.15.7:0/298688873 >> v1:172.21.15.188:6789/0 conn(0x7f21740aafc0 legacy=0x7f2174063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:03.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.803+0000 7f2173fff700 1 -- 172.21.15.7:0/298688873 >> v1:172.21.15.7:6789/0 conn(0x7f21740c3a90 legacy=0x7f21740aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:03.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.803+0000 7f2172ffd700 1 -- 172.21.15.7:0/298688873 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4151759038 0 0) 0x7f21740c30c0 con 0x7f21740c3a90 2024-01-27T02:44:03.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.803+0000 7f2173fff700 1 -- 172.21.15.7:0/298688873 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217415ec40 con 0x7f2174159f40 2024-01-27T02:44:03.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.803+0000 7f2173fff700 1 --2- 172.21.15.7:0/298688873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2174159f40 0x7f21740c3550 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f21640098e0 tx=0x7f21640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93b7dab7c83e144 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:03.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.803+0000 7f2172ffd700 1 -- 172.21.15.7:0/298688873 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2164009e80 con 0x7f2174159f40 2024-01-27T02:44:03.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.803+0000 7f2172ffd700 1 -- 172.21.15.7:0/298688873 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216400b860 con 0x7f2174159f40 2024-01-27T02:44:03.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.803+0000 7f2172ffd700 1 -- 172.21.15.7:0/298688873 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216400bbe0 con 0x7f2174159f40 2024-01-27T02:44:03.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.804+0000 7f217a4ae700 1 -- 172.21.15.7:0/298688873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2174159f40 msgr2=0x7f21740c3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:03.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.804+0000 7f217a4ae700 1 --2- 172.21.15.7:0/298688873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2174159f40 0x7f21740c3550 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f21640098e0 tx=0x7f21640091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:03.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.804+0000 7f217a4ae700 1 -- 172.21.15.7:0/298688873 shutdown_connections 2024-01-27T02:44:03.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.804+0000 7f217a4ae700 1 --2- 172.21.15.7:0/298688873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2174159f40 0x7f21740c3550 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:03.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.804+0000 7f217a4ae700 1 -- 172.21.15.7:0/298688873 >> 172.21.15.7:0/298688873 conn(0x7f21740cdc30 msgr2=0x7f21740ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:03.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.804+0000 7f217a4ae700 1 -- 172.21.15.7:0/298688873 shutdown_connections 2024-01-27T02:44:03.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.804+0000 7f217a4ae700 1 -- 172.21.15.7:0/298688873 wait complete. 2024-01-27T02:44:03.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.805+0000 7f217a4ae700 1 Processor -- start 2024-01-27T02:44:03.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.806+0000 7f217a4ae700 1 -- start start 2024-01-27T02:44:03.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.806+0000 7f217a4ae700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21740aafc0 0x7f2174150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.806+0000 7f217a4ae700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21740c3a90 0x7f217416afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.806+0000 7f217a4ae700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2174159f40 0x7f217415a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.806+0000 7f217a4ae700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217416a5c0 con 0x7f21740aafc0 2024-01-27T02:44:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.806+0000 7f217a4ae700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2174150ca0 con 0x7f2174159f40 2024-01-27T02:44:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.806+0000 7f217a4ae700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217415ab60 con 0x7f21740c3a90 2024-01-27T02:44:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.806+0000 7f2173fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21740aafc0 0x7f2174150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.806+0000 7f2173fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21740aafc0 0x7f2174150760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48600/0 (socket says 172.21.15.7:48600) 2024-01-27T02:44:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.806+0000 7f2173fff700 1 -- 172.21.15.7:0/2783927305 learned_addr learned my addr 172.21.15.7:0/2783927305 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.807+0000 7f21737fe700 1 --2- 172.21.15.7:0/2783927305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21740c3a90 0x7f217416afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.807+0000 7f2178a4b700 1 --2- 172.21.15.7:0/2783927305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2174159f40 0x7f217415a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.807+0000 7f2173fff700 1 -- 172.21.15.7:0/2783927305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21740c3a90 msgr2=0x7f217416afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.807+0000 7f2173fff700 1 --2- 172.21.15.7:0/2783927305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21740c3a90 0x7f217416afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:03.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.807+0000 7f2173fff700 1 -- 172.21.15.7:0/2783927305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2174159f40 msgr2=0x7f217415a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:03.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.807+0000 7f2173fff700 1 --2- 172.21.15.7:0/2783927305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2174159f40 0x7f217415a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:03.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.807+0000 7f2173fff700 1 -- 172.21.15.7:0/2783927305 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2164009420 con 0x7f21740aafc0 2024-01-27T02:44:03.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.807+0000 7f21737fe700 1 --2- 172.21.15.7:0/2783927305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21740c3a90 0x7f217416afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:44:03.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.807+0000 7f2178a4b700 1 --2- 172.21.15.7:0/2783927305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2174159f40 0x7f217415a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:44:03.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.807+0000 7f2173fff700 1 --2- 172.21.15.7:0/2783927305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21740aafc0 0x7f2174150760 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f217406dcc0 tx=0x7f2164017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:03.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.807+0000 7f215bfff700 1 -- 172.21.15.7:0/2783927305 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216401f070 con 0x7f21740aafc0 2024-01-27T02:44:03.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.807+0000 7f215bfff700 1 -- 172.21.15.7:0/2783927305 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216400b9d0 con 0x7f21740aafc0 2024-01-27T02:44:03.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.807+0000 7f217a4ae700 1 -- 172.21.15.7:0/2783927305 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217415ae40 con 0x7f21740aafc0 2024-01-27T02:44:03.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.807+0000 7f215bfff700 1 -- 172.21.15.7:0/2783927305 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2164025670 con 0x7f21740aafc0 2024-01-27T02:44:03.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.807+0000 7f217a4ae700 1 -- 172.21.15.7:0/2783927305 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217416aab0 con 0x7f21740aafc0 2024-01-27T02:44:03.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.808+0000 7f217a4ae700 1 -- 172.21.15.7:0/2783927305 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217415b050 con 0x7f21740aafc0 2024-01-27T02:44:03.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.809+0000 7f215bfff700 1 -- 172.21.15.7:0/2783927305 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2164023070 con 0x7f21740aafc0 2024-01-27T02:44:03.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.809+0000 7f215bfff700 1 --2- 172.21.15.7:0/2783927305 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2160066f30 0x7f21600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:03.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.809+0000 7f215bfff700 1 -- 172.21.15.7:0/2783927305 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2164085c10 con 0x7f21740aafc0 2024-01-27T02:44:03.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.809+0000 7f21737fe700 1 --2- 172.21.15.7:0/2783927305 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2160066f30 0x7f21600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:03.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.810+0000 7f21737fe700 1 --2- 172.21.15.7:0/2783927305 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2160066f30 0x7f21600693e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f21740c8140 tx=0x7f2168009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:03.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:03.814+0000 7f215bfff700 1 -- 172.21.15.7:0/2783927305 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2164052e80 con 0x7f21740aafc0 2024-01-27T02:44:04.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.067+0000 7f217a4ae700 1 -- 172.21.15.7:0/2783927305 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f217404c710 con 0x7f21740aafc0 2024-01-27T02:44:04.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.068+0000 7f215bfff700 1 -- 172.21.15.7:0/2783927305 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4595 (secure 0 0 0) 0x7f216402a090 con 0x7f21740aafc0 2024-01-27T02:44:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:44:04.068 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6704,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/1199331492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1199331492},{"type":"v1","addr":"172.21.15.7:6834","nonce":1199331492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":6707,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/721335086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":721335086},{"type":"v1","addr":"172.21.15.7:6835","nonce":721335086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":7699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/450555206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":450555206},{"type":"v1","addr":"172.21.15.188:6836","nonce":450555206}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-27T02:43:29.599748+0000","modified":"2024-01-27T02:43:33.196288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7702},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7702":{"gid":7702,"name":"b","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6837/3853795510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3853795510},{"type":"v1","addr":"172.21.15.188:6837","nonce":3853795510}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-27T02:44:04.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.070+0000 7f217a4ae700 1 -- 172.21.15.7:0/2783927305 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2160066f30 msgr2=0x7f21600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:04.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.070+0000 7f217a4ae700 1 --2- 172.21.15.7:0/2783927305 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2160066f30 0x7f21600693e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f21740c8140 tx=0x7f2168009040 comp rx=0 tx=0).stop 2024-01-27T02:44:04.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.070+0000 7f217a4ae700 1 -- 172.21.15.7:0/2783927305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21740aafc0 msgr2=0x7f2174150760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:04.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.070+0000 7f217a4ae700 1 --2- 172.21.15.7:0/2783927305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21740aafc0 0x7f2174150760 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f217406dcc0 tx=0x7f2164017000 comp rx=0 tx=0).stop 2024-01-27T02:44:04.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.070+0000 7f217a4ae700 1 -- 172.21.15.7:0/2783927305 shutdown_connections 2024-01-27T02:44:04.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.070+0000 7f217a4ae700 1 --2- 172.21.15.7:0/2783927305 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2160066f30 0x7f21600693e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.070+0000 7f217a4ae700 1 --2- 172.21.15.7:0/2783927305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2174159f40 0x7f217415a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.070+0000 7f217a4ae700 1 --2- 172.21.15.7:0/2783927305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21740aafc0 0x7f2174150760 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.070+0000 7f217a4ae700 1 --2- 172.21.15.7:0/2783927305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21740c3a90 0x7f217416afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.070+0000 7f217a4ae700 1 -- 172.21.15.7:0/2783927305 >> 172.21.15.7:0/2783927305 conn(0x7f21740cdc30 msgr2=0x7f21741583c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:04.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.070+0000 7f217a4ae700 1 -- 172.21.15.7:0/2783927305 shutdown_connections 2024-01-27T02:44:04.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.070+0000 7f217a4ae700 1 -- 172.21.15.7:0/2783927305 wait complete. 2024-01-27T02:44:04.072 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 94 2024-01-27T02:44:04.084 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:44:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.215+0000 7f1b792fa700 1 Processor -- start 2024-01-27T02:44:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.215+0000 7f1b792fa700 1 -- start start 2024-01-27T02:44:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.216+0000 7f1b792fa700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1b74157ca0 0x7f1b740c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.216+0000 7f1b792fa700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1b740cdbf0 0x7f1b740ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.216+0000 7f1b792fa700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b740ca020 con 0x7f1b740c3ae0 2024-01-27T02:44:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.216+0000 7f1b792fa700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b740c4f40 con 0x7f1b74157ca0 2024-01-27T02:44:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.216+0000 7f1b792fa700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b740c5210 con 0x7f1b740cdbf0 2024-01-27T02:44:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.216+0000 7f1b72ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1b74157ca0 0x7f1b740c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.217+0000 7f1b72ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1b74157ca0 0x7f1b740c4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48602/0 (socket says 172.21.15.7:48602) 2024-01-27T02:44:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.217+0000 7f1b72ffd700 1 -- 172.21.15.7:0/1015243127 learned_addr learned my addr 172.21.15.7:0/1015243127 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.217+0000 7f1b737fe700 1 --2- 172.21.15.7:0/1015243127 >> v2:172.21.15.188:3300/0 conn(0x7f1b740cdbf0 0x7f1b740ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.217+0000 7f1b72ffd700 1 -- 172.21.15.7:0/1015243127 >> v2:172.21.15.188:3300/0 conn(0x7f1b740cdbf0 msgr2=0x7f1b740ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.217+0000 7f1b72ffd700 1 --2- 172.21.15.7:0/1015243127 >> v2:172.21.15.188:3300/0 conn(0x7f1b740cdbf0 0x7f1b740ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.217+0000 7f1b72ffd700 1 -- 172.21.15.7:0/1015243127 >> v1:172.21.15.188:6789/0 conn(0x7f1b740c3ae0 legacy=0x7f1b740cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.217+0000 7f1b72ffd700 1 -- 172.21.15.7:0/1015243127 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b740c54f0 con 0x7f1b74157ca0 2024-01-27T02:44:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.217+0000 7f1b737fe700 1 --2- 172.21.15.7:0/1015243127 >> v2:172.21.15.188:3300/0 conn(0x7f1b740cdbf0 0x7f1b740ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:44:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.217+0000 7f1b72ffd700 1 --2- 172.21.15.7:0/1015243127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b74157ca0 0x7f1b740c4a00 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f1b7406f890 tx=0x7f1b64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639a7e2359360d87 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.217+0000 7f1b71ffb700 1 -- 172.21.15.7:0/1015243127 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6400b860 con 0x7f1b74157ca0 2024-01-27T02:44:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.217+0000 7f1b71ffb700 1 -- 172.21.15.7:0/1015243127 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6400b9c0 con 0x7f1b74157ca0 2024-01-27T02:44:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.217+0000 7f1b71ffb700 1 -- 172.21.15.7:0/1015243127 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6400bd60 con 0x7f1b74157ca0 2024-01-27T02:44:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.218+0000 7f1b792fa700 1 -- 172.21.15.7:0/1015243127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b74157ca0 msgr2=0x7f1b740c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.218+0000 7f1b792fa700 1 --2- 172.21.15.7:0/1015243127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b74157ca0 0x7f1b740c4a00 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f1b7406f890 tx=0x7f1b64009250 comp rx=0 tx=0).stop 2024-01-27T02:44:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.218+0000 7f1b792fa700 1 -- 172.21.15.7:0/1015243127 shutdown_connections 2024-01-27T02:44:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.218+0000 7f1b792fa700 1 --2- 172.21.15.7:0/1015243127 >> v2:172.21.15.188:3300/0 conn(0x7f1b740cdbf0 0x7f1b740ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.218+0000 7f1b792fa700 1 --2- 172.21.15.7:0/1015243127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b74157ca0 0x7f1b740c4a00 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.218+0000 7f1b792fa700 1 -- 172.21.15.7:0/1015243127 >> 172.21.15.7:0/1015243127 conn(0x7f1b74098020 msgr2=0x7f1b740711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:04.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.218+0000 7f1b792fa700 1 -- 172.21.15.7:0/1015243127 shutdown_connections 2024-01-27T02:44:04.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.218+0000 7f1b792fa700 1 -- 172.21.15.7:0/1015243127 wait complete. 2024-01-27T02:44:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.219+0000 7f1b792fa700 1 Processor -- start 2024-01-27T02:44:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.219+0000 7f1b792fa700 1 -- start start 2024-01-27T02:44:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.220+0000 7f1b792fa700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b740c3ae0 0x7f1b741513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.220+0000 7f1b792fa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b740cdbf0 0x7f1b74154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.220+0000 7f1b792fa700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b74157ca0 0x7f1b74156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.220+0000 7f1b792fa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b74068620 con 0x7f1b740cdbf0 2024-01-27T02:44:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.220+0000 7f1b792fa700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b74067a50 con 0x7f1b74157ca0 2024-01-27T02:44:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.220+0000 7f1b792fa700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b74067cf0 con 0x7f1b740c3ae0 2024-01-27T02:44:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.220+0000 7f1b727fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b740cdbf0 0x7f1b74154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.220+0000 7f1b727fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b740cdbf0 0x7f1b74154a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48612/0 (socket says 172.21.15.7:48612) 2024-01-27T02:44:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.220+0000 7f1b727fc700 1 -- 172.21.15.7:0/3756052420 learned_addr learned my addr 172.21.15.7:0/3756052420 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.221+0000 7f1b737fe700 1 --2- 172.21.15.7:0/3756052420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b74157ca0 0x7f1b74156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.221+0000 7f1b72ffd700 1 --2- 172.21.15.7:0/3756052420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b740c3ae0 0x7f1b741513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.221+0000 7f1b727fc700 1 -- 172.21.15.7:0/3756052420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b740c3ae0 msgr2=0x7f1b741513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.221+0000 7f1b727fc700 1 --2- 172.21.15.7:0/3756052420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b740c3ae0 0x7f1b741513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.221+0000 7f1b727fc700 1 -- 172.21.15.7:0/3756052420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b74157ca0 msgr2=0x7f1b74156700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.221+0000 7f1b727fc700 1 --2- 172.21.15.7:0/3756052420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b74157ca0 0x7f1b74156700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.221+0000 7f1b727fc700 1 -- 172.21.15.7:0/3756052420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b640094f0 con 0x7f1b740cdbf0 2024-01-27T02:44:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.221+0000 7f1b727fc700 1 --2- 172.21.15.7:0/3756052420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b740cdbf0 0x7f1b74154a10 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00c590 tx=0x7f1b5c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.221+0000 7f1b6affd700 1 -- 172.21.15.7:0/3756052420 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c011590 con 0x7f1b740cdbf0 2024-01-27T02:44:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.221+0000 7f1b6affd700 1 -- 172.21.15.7:0/3756052420 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5c011710 con 0x7f1b740cdbf0 2024-01-27T02:44:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.221+0000 7f1b792fa700 1 -- 172.21.15.7:0/3756052420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b74067fd0 con 0x7f1b740cdbf0 2024-01-27T02:44:04.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.221+0000 7f1b6affd700 1 -- 172.21.15.7:0/3756052420 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c011ad0 con 0x7f1b740cdbf0 2024-01-27T02:44:04.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.221+0000 7f1b792fa700 1 -- 172.21.15.7:0/3756052420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b74154240 con 0x7f1b740cdbf0 2024-01-27T02:44:04.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.222+0000 7f1b792fa700 1 -- 172.21.15.7:0/3756052420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b7404c710 con 0x7f1b740cdbf0 2024-01-27T02:44:04.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.223+0000 7f1b6affd700 1 -- 172.21.15.7:0/3756052420 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1b5c01b3b0 con 0x7f1b740cdbf0 2024-01-27T02:44:04.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.223+0000 7f1b6affd700 1 --2- 172.21.15.7:0/3756052420 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1b54066f30 0x7f1b540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.223+0000 7f1b6affd700 1 -- 172.21.15.7:0/3756052420 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1b5c0839a0 con 0x7f1b740cdbf0 2024-01-27T02:44:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.223+0000 7f1b72ffd700 1 --2- 172.21.15.7:0/3756052420 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1b54066f30 0x7f1b540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.224+0000 7f1b72ffd700 1 --2- 172.21.15.7:0/3756052420 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1b54066f30 0x7f1b540693e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1b7406f890 tx=0x7f1b64017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:04.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.227+0000 7f1b6affd700 1 -- 172.21.15.7:0/3756052420 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b5c050c10 con 0x7f1b740cdbf0 2024-01-27T02:44:04.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.430+0000 7f1b792fa700 1 -- 172.21.15.7:0/3756052420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b74068760 con 0x7f1b740cdbf0 2024-01-27T02:44:04.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.431+0000 7f1b6affd700 1 -- 172.21.15.7:0/3756052420 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+15324 (secure 0 0 0) 0x7f1b5c017070 con 0x7f1b740cdbf0 2024-01-27T02:44:04.432 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:44:04.432 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":183,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:43:54.936156+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-01-27T02:43:27.492968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"179","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-01-27T02:43:28.579977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:44:04.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.433+0000 7f1b792fa700 1 -- 172.21.15.7:0/3756052420 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1b54066f30 msgr2=0x7f1b540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:04.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.433+0000 7f1b792fa700 1 --2- 172.21.15.7:0/3756052420 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1b54066f30 0x7f1b540693e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1b7406f890 tx=0x7f1b64017000 comp rx=0 tx=0).stop 2024-01-27T02:44:04.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.433+0000 7f1b792fa700 1 -- 172.21.15.7:0/3756052420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b740cdbf0 msgr2=0x7f1b74154a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:04.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.433+0000 7f1b792fa700 1 --2- 172.21.15.7:0/3756052420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b740cdbf0 0x7f1b74154a10 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00c590 tx=0x7f1b5c00c8a0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.434+0000 7f1b792fa700 1 -- 172.21.15.7:0/3756052420 shutdown_connections 2024-01-27T02:44:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.434+0000 7f1b792fa700 1 --2- 172.21.15.7:0/3756052420 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1b54066f30 0x7f1b540693e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.434+0000 7f1b792fa700 1 --2- 172.21.15.7:0/3756052420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b74157ca0 0x7f1b74156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.434+0000 7f1b792fa700 1 --2- 172.21.15.7:0/3756052420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b740c3ae0 0x7f1b741513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.434+0000 7f1b792fa700 1 --2- 172.21.15.7:0/3756052420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b740cdbf0 0x7f1b74154a10 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.434+0000 7f1b792fa700 1 -- 172.21.15.7:0/3756052420 >> 172.21.15.7:0/3756052420 conn(0x7f1b74098020 msgr2=0x7f1b740ca4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.434+0000 7f1b792fa700 1 -- 172.21.15.7:0/3756052420 shutdown_connections 2024-01-27T02:44:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.434+0000 7f1b792fa700 1 -- 172.21.15.7:0/3756052420 wait complete. 2024-01-27T02:44:04.448 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-27T02:44:04.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.580+0000 7f5dd9157700 1 Processor -- start 2024-01-27T02:44:04.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.580+0000 7f5dd9157700 1 -- start start 2024-01-27T02:44:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.580+0000 7f5dd9157700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5dd406ea40 0x7f5dd4157f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.580+0000 7f5dd9157700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd406c880 con 0x7f5dd40c9f80 2024-01-27T02:44:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.581+0000 7f5dd9157700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd406bc70 con 0x7f5dd40c9bb0 2024-01-27T02:44:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.581+0000 7f5dd9157700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd406cd10 con 0x7f5dd406ea40 2024-01-27T02:44:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.581+0000 7f5dd259c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5dd40c9f80 0x7f5dd406e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56820/0 (socket says 172.21.15.7:56820) 2024-01-27T02:44:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.581+0000 7f5dd259c700 1 -- 172.21.15.7:0/4288519681 learned_addr learned my addr 172.21.15.7:0/4288519681 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.581+0000 7f5dd1d9b700 1 -- 172.21.15.7:0/4288519681 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 789739318 0 0) 0x7f5dd406c880 con 0x7f5dd40c9f80 2024-01-27T02:44:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.581+0000 7f5dd359e700 1 --2- 172.21.15.7:0/4288519681 >> v2:172.21.15.188:3300/0 conn(0x7f5dd406ea40 0x7f5dd4157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.581+0000 7f5dd1d9b700 1 -- 172.21.15.7:0/4288519681 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dbc003570 con 0x7f5dd40c9f80 2024-01-27T02:44:04.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.581+0000 7f5dd1d9b700 1 -- 172.21.15.7:0/4288519681 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1258765631 0 0) 0x7f5dbc003570 con 0x7f5dd40c9f80 2024-01-27T02:44:04.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.581+0000 7f5dd1d9b700 1 -- 172.21.15.7:0/4288519681 >> v2:172.21.15.188:3300/0 conn(0x7f5dd406ea40 msgr2=0x7f5dd4157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:04.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.581+0000 7f5dd1d9b700 1 --2- 172.21.15.7:0/4288519681 >> v2:172.21.15.188:3300/0 conn(0x7f5dd406ea40 0x7f5dd4157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.581+0000 7f5dd1d9b700 1 -- 172.21.15.7:0/4288519681 >> v1:172.21.15.188:6789/0 conn(0x7f5dd40c9bb0 legacy=0x7f5dd406b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:04.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.581+0000 7f5dd1d9b700 1 -- 172.21.15.7:0/4288519681 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd406bf50 con 0x7f5dd40c9f80 2024-01-27T02:44:04.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.581+0000 7f5dd1d9b700 1 -- 172.21.15.7:0/4288519681 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f5dc8001470 con 0x7f5dd40c9f80 2024-01-27T02:44:04.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.581+0000 7f5dd1d9b700 1 -- 172.21.15.7:0/4288519681 >> v1:172.21.15.7:6789/0 conn(0x7f5dd40c9f80 legacy=0x7f5dd406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:04.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.581+0000 7f5dd1d9b700 1 --2- 172.21.15.7:0/4288519681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dbc003a60 0x7f5dbc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:04.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.581+0000 7f5dd1d9b700 1 --2- 172.21.15.7:0/4288519681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dbc004810 0x7f5dbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:04.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.581+0000 7f5dd1d9b700 1 --2- 172.21.15.7:0/4288519681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dbc0051c0 0x7f5dbc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:04.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.581+0000 7f5dd1d9b700 1 -- 172.21.15.7:0/4288519681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd406cd10 con 0x7f5dbc0051c0 2024-01-27T02:44:04.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.581+0000 7f5dd1d9b700 1 -- 172.21.15.7:0/4288519681 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc00c0d0 con 0x7f5dbc003a60 2024-01-27T02:44:04.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.581+0000 7f5dd1d9b700 1 -- 172.21.15.7:0/4288519681 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc00c290 con 0x7f5dbc004810 2024-01-27T02:44:04.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.582+0000 7f5dd359e700 1 --2- 172.21.15.7:0/4288519681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dbc0051c0 0x7f5dbc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:04.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.582+0000 7f5dd2d9d700 1 --2- 172.21.15.7:0/4288519681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dbc003a60 0x7f5dbc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.582+0000 7f5dd259c700 1 --2- 172.21.15.7:0/4288519681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dbc004810 0x7f5dbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.582+0000 7f5dd359e700 1 -- 172.21.15.7:0/4288519681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dbc004810 msgr2=0x7f5dbc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.582+0000 7f5dd359e700 1 --2- 172.21.15.7:0/4288519681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dbc004810 0x7f5dbc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.582+0000 7f5dd359e700 1 -- 172.21.15.7:0/4288519681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dbc003a60 msgr2=0x7f5dbc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.582+0000 7f5dd359e700 1 --2- 172.21.15.7:0/4288519681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dbc003a60 0x7f5dbc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.582+0000 7f5dd359e700 1 -- 172.21.15.7:0/4288519681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc003570 con 0x7f5dbc0051c0 2024-01-27T02:44:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.582+0000 7f5dd359e700 1 -- 172.21.15.7:0/4288519681 reap_dead start 2024-01-27T02:44:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.582+0000 7f5dd359e700 1 --2- 172.21.15.7:0/4288519681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dbc0051c0 0x7f5dbc00ba60 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f5dd406ee40 tx=0x7f5dcc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbdc6112136354bd server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.582+0000 7f5dd1d9b700 1 -- 172.21.15.7:0/4288519681 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc00ea90 con 0x7f5dbc0051c0 2024-01-27T02:44:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.582+0000 7f5dd1d9b700 1 -- 172.21.15.7:0/4288519681 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dcc00f040 con 0x7f5dbc0051c0 2024-01-27T02:44:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.582+0000 7f5dd1d9b700 1 -- 172.21.15.7:0/4288519681 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc00ee40 con 0x7f5dbc0051c0 2024-01-27T02:44:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.583+0000 7f5dd9157700 1 -- 172.21.15.7:0/4288519681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dbc0051c0 msgr2=0x7f5dbc00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.583+0000 7f5dd9157700 1 --2- 172.21.15.7:0/4288519681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dbc0051c0 0x7f5dbc00ba60 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f5dd406ee40 tx=0x7f5dcc00b040 comp rx=0 tx=0).stop 2024-01-27T02:44:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.583+0000 7f5dd9157700 1 -- 172.21.15.7:0/4288519681 shutdown_connections 2024-01-27T02:44:04.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.583+0000 7f5dd9157700 1 --2- 172.21.15.7:0/4288519681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dbc0051c0 0x7f5dbc00ba60 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.583+0000 7f5dd9157700 1 -- 172.21.15.7:0/4288519681 >> 172.21.15.7:0/4288519681 conn(0x7f5dd40cdc30 msgr2=0x7f5dd40ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:04.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.583+0000 7f5dd9157700 1 -- 172.21.15.7:0/4288519681 shutdown_connections 2024-01-27T02:44:04.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.583+0000 7f5dd9157700 1 -- 172.21.15.7:0/4288519681 wait complete. 2024-01-27T02:44:04.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.584+0000 7f5dd9157700 1 Processor -- start 2024-01-27T02:44:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.584+0000 7f5dd9157700 1 -- start start 2024-01-27T02:44:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.585+0000 7f5dd9157700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dbc0051c0 0x7f5dd4139610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.585+0000 7f5dd9157700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dd40c6770 0x7f5dd40c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.585+0000 7f5dd9157700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dd41507f0 0x7f5dd4150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.585+0000 7f5dd9157700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd4147e90 con 0x7f5dbc0051c0 2024-01-27T02:44:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.585+0000 7f5dd9157700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd4147bd0 con 0x7f5dd41507f0 2024-01-27T02:44:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.585+0000 7f5dd9157700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd41511a0 con 0x7f5dd40c6770 2024-01-27T02:44:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.585+0000 7f5dd2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dbc0051c0 0x7f5dd4139610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.585+0000 7f5dd2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dbc0051c0 0x7f5dd4139610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48618/0 (socket says 172.21.15.7:48618) 2024-01-27T02:44:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.585+0000 7f5dd2d9d700 1 -- 172.21.15.7:0/4273680530 learned_addr learned my addr 172.21.15.7:0/4273680530 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.585+0000 7f5dd359e700 1 --2- 172.21.15.7:0/4273680530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dd41507f0 0x7f5dd4150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.585+0000 7f5dd2d9d700 1 -- 172.21.15.7:0/4273680530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dd40c6770 msgr2=0x7f5dd40c6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.585+0000 7f5dd2d9d700 1 --2- 172.21.15.7:0/4273680530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dd40c6770 0x7f5dd40c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.585+0000 7f5dd2d9d700 1 -- 172.21.15.7:0/4273680530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dd41507f0 msgr2=0x7f5dd4150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.585+0000 7f5dd2d9d700 1 --2- 172.21.15.7:0/4273680530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dd41507f0 0x7f5dd4150c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.585+0000 7f5dd2d9d700 1 -- 172.21.15.7:0/4273680530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd40b20c0 con 0x7f5dbc0051c0 2024-01-27T02:44:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.585+0000 7f5dd359e700 1 --2- 172.21.15.7:0/4273680530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dd41507f0 0x7f5dd4150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:44:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.586+0000 7f5dd2d9d700 1 --2- 172.21.15.7:0/4273680530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dbc0051c0 0x7f5dd4139610 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40029e0 tx=0x7f5dc4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.586+0000 7f5dbaffd700 1 -- 172.21.15.7:0/4273680530 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc400f960 con 0x7f5dbc0051c0 2024-01-27T02:44:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.586+0000 7f5dbaffd700 1 -- 172.21.15.7:0/4273680530 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc400fac0 con 0x7f5dbc0051c0 2024-01-27T02:44:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.586+0000 7f5dd9157700 1 -- 172.21.15.7:0/4273680530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd40b23a0 con 0x7f5dbc0051c0 2024-01-27T02:44:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.586+0000 7f5dbaffd700 1 -- 172.21.15.7:0/4273680530 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc400fe60 con 0x7f5dbc0051c0 2024-01-27T02:44:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.586+0000 7f5dd9157700 1 -- 172.21.15.7:0/4273680530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd40b2960 con 0x7f5dbc0051c0 2024-01-27T02:44:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.587+0000 7f5dd9157700 1 -- 172.21.15.7:0/4273680530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd404c710 con 0x7f5dbc0051c0 2024-01-27T02:44:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.587+0000 7f5dbaffd700 1 -- 172.21.15.7:0/4273680530 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5dc4013600 con 0x7f5dbc0051c0 2024-01-27T02:44:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.588+0000 7f5dbaffd700 1 --2- 172.21.15.7:0/4273680530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5dc0066f30 0x7f5dc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.588+0000 7f5dbaffd700 1 -- 172.21.15.7:0/4273680530 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5dc4084a40 con 0x7f5dbc0051c0 2024-01-27T02:44:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.588+0000 7f5dd259c700 1 --2- 172.21.15.7:0/4273680530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5dc0066f30 0x7f5dc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.589+0000 7f5dd259c700 1 --2- 172.21.15.7:0/4273680530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5dc0066f30 0x7f5dc00693e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5dd415a6c0 tx=0x7f5dc8001e50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.592+0000 7f5dbaffd700 1 -- 172.21.15.7:0/4273680530 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5dc4086e60 con 0x7f5dbc0051c0 2024-01-27T02:44:04.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.842+0000 7f5dd9157700 1 -- 172.21.15.7:0/4273680530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5dd40cc720 con 0x7f5dbc0051c0 2024-01-27T02:44:04.843 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-27T02:44:04.842+0000 7fb5c503a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-27T02:44:04.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.846+0000 7f5dbaffd700 1 -- 172.21.15.7:0/4273680530 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v95) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5dc4051cb0 con 0x7f5dbc0051c0 2024-01-27T02:44:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.849+0000 7f5dd9157700 1 -- 172.21.15.7:0/4273680530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5dc0066f30 msgr2=0x7f5dc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.849+0000 7f5dd9157700 1 --2- 172.21.15.7:0/4273680530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5dc0066f30 0x7f5dc00693e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5dd415a6c0 tx=0x7f5dc8001e50 comp rx=0 tx=0).stop 2024-01-27T02:44:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.849+0000 7f5dd9157700 1 -- 172.21.15.7:0/4273680530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dbc0051c0 msgr2=0x7f5dd4139610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.849+0000 7f5dd9157700 1 --2- 172.21.15.7:0/4273680530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dbc0051c0 0x7f5dd4139610 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40029e0 tx=0x7f5dc4007d30 comp rx=0 tx=0).stop 2024-01-27T02:44:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.849+0000 7f5dd9157700 1 -- 172.21.15.7:0/4273680530 shutdown_connections 2024-01-27T02:44:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.849+0000 7f5dd9157700 1 --2- 172.21.15.7:0/4273680530 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5dc0066f30 0x7f5dc00693e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.849+0000 7f5dd9157700 1 --2- 172.21.15.7:0/4273680530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dd41507f0 0x7f5dd4150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.849+0000 7f5dd9157700 1 --2- 172.21.15.7:0/4273680530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dbc0051c0 0x7f5dd4139610 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.849+0000 7f5dd9157700 1 --2- 172.21.15.7:0/4273680530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dd40c6770 0x7f5dd40c6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.849+0000 7f5dd9157700 1 -- 172.21.15.7:0/4273680530 >> 172.21.15.7:0/4273680530 conn(0x7f5dd40cdc30 msgr2=0x7f5dd414fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.849+0000 7f5dd9157700 1 -- 172.21.15.7:0/4273680530 shutdown_connections 2024-01-27T02:44:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:04.849+0000 7f5dd9157700 1 -- 172.21.15.7:0/4273680530 wait complete. 2024-01-27T02:44:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-27T02:44:04.854 INFO:tasks.ceph.mds.b.smithi188.stderr: -7612> 2024-01-27T02:43:35.153+0000 7f410eb40700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-27T02:44:04.864 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-27T02:44:04.884 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-01-27T02:44:05.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.001+0000 7f1694edd700 1 Processor -- start 2024-01-27T02:44:05.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.002+0000 7f1694edd700 1 -- start start 2024-01-27T02:44:05.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.003+0000 7f1694edd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f16900cba60 0x7f16900ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:05.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.003+0000 7f1694edd700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f16900cbe30 0x7f169015b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:05.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.003+0000 7f1694edd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f169006b4e0 0x7f169006b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:05.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.003+0000 7f1694edd700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16900679b0 con 0x7f16900cba60 2024-01-27T02:44:05.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.003+0000 7f1694edd700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1690067c90 con 0x7f16900cbe30 2024-01-27T02:44:05.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.003+0000 7f1694edd700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169006be60 con 0x7f169006b4e0 2024-01-27T02:44:05.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.003+0000 7f168e59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f16900cba60 0x7f16900ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:05.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.003+0000 7f168e59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f16900cba60 0x7f16900ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48628/0 (socket says 172.21.15.7:48628) 2024-01-27T02:44:05.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.003+0000 7f168e59c700 1 -- 172.21.15.7:0/2599928281 learned_addr learned my addr 172.21.15.7:0/2599928281 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.003+0000 7f168dd9b700 1 --2- 172.21.15.7:0/2599928281 >> v2:172.21.15.188:3300/0 conn(0x7f16900cbe30 0x7f169015b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.003+0000 7f168ed9d700 1 --2- 172.21.15.7:0/2599928281 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f169006b4e0 0x7f169006b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.003+0000 7f168e59c700 1 -- 172.21.15.7:0/2599928281 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f169006b4e0 msgr2=0x7f169006b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.003+0000 7f168e59c700 1 --2- 172.21.15.7:0/2599928281 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f169006b4e0 0x7f169006b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.003+0000 7f168e59c700 1 -- 172.21.15.7:0/2599928281 >> v2:172.21.15.188:3300/0 conn(0x7f16900cbe30 msgr2=0x7f169015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.003+0000 7f168e59c700 1 --2- 172.21.15.7:0/2599928281 >> v2:172.21.15.188:3300/0 conn(0x7f16900cbe30 0x7f169015b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.003+0000 7f168e59c700 1 -- 172.21.15.7:0/2599928281 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169015b7b0 con 0x7f16900cba60 2024-01-27T02:44:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.003+0000 7f168dd9b700 1 --2- 172.21.15.7:0/2599928281 >> v2:172.21.15.188:3300/0 conn(0x7f16900cbe30 0x7f169015b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:44:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.003+0000 7f168e59c700 1 --2- 172.21.15.7:0/2599928281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16900cba60 0x7f16900ce410 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f169006d590 tx=0x7f1680009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa0d0211cfdbb90 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.004+0000 7f168d59a700 1 -- 172.21.15.7:0/2599928281 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1680005c10 con 0x7f16900cba60 2024-01-27T02:44:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.004+0000 7f168d59a700 1 -- 172.21.15.7:0/2599928281 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1680005d70 con 0x7f16900cba60 2024-01-27T02:44:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.004+0000 7f168d59a700 1 -- 172.21.15.7:0/2599928281 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1680006260 con 0x7f16900cba60 2024-01-27T02:44:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.004+0000 7f1694edd700 1 -- 172.21.15.7:0/2599928281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16900cba60 msgr2=0x7f16900ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.004+0000 7f1694edd700 1 --2- 172.21.15.7:0/2599928281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16900cba60 0x7f16900ce410 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f169006d590 tx=0x7f1680009220 comp rx=0 tx=0).stop 2024-01-27T02:44:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.004+0000 7f1694edd700 1 -- 172.21.15.7:0/2599928281 shutdown_connections 2024-01-27T02:44:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.004+0000 7f1694edd700 1 --2- 172.21.15.7:0/2599928281 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f169006b4e0 0x7f169006b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.004+0000 7f1694edd700 1 --2- 172.21.15.7:0/2599928281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16900cba60 0x7f16900ce410 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.004+0000 7f1694edd700 1 --2- 172.21.15.7:0/2599928281 >> v2:172.21.15.188:3300/0 conn(0x7f16900cbe30 0x7f169015b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.004+0000 7f1694edd700 1 -- 172.21.15.7:0/2599928281 >> 172.21.15.7:0/2599928281 conn(0x7f1690063580 msgr2=0x7f1690063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.004+0000 7f1694edd700 1 -- 172.21.15.7:0/2599928281 shutdown_connections 2024-01-27T02:44:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.004+0000 7f1694edd700 1 -- 172.21.15.7:0/2599928281 wait complete. 2024-01-27T02:44:05.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.006+0000 7f1694edd700 1 Processor -- start 2024-01-27T02:44:05.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.006+0000 7f1694edd700 1 -- start start 2024-01-27T02:44:05.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.006+0000 7f1694edd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f169006b4e0 0x7f169015cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:05.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.006+0000 7f1694edd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16900cba60 0x7f16900b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:05.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.006+0000 7f1694edd700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16900cbe30 0x7f169016abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:05.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.006+0000 7f1694edd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16900ac920 con 0x7f16900cba60 2024-01-27T02:44:05.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.006+0000 7f1694edd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16900ab880 con 0x7f169006b4e0 2024-01-27T02:44:05.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.006+0000 7f1694edd700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16900abb60 con 0x7f16900cbe30 2024-01-27T02:44:05.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.006+0000 7f168dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16900cba60 0x7f16900b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:05.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.007+0000 7f168dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16900cba60 0x7f16900b4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48630/0 (socket says 172.21.15.7:48630) 2024-01-27T02:44:05.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.007+0000 7f168dd9b700 1 -- 172.21.15.7:0/2538686280 learned_addr learned my addr 172.21.15.7:0/2538686280 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:05.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.007+0000 7f168e59c700 1 --2- 172.21.15.7:0/2538686280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f169006b4e0 0x7f169015cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:05.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.007+0000 7f168ed9d700 1 --2- 172.21.15.7:0/2538686280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16900cbe30 0x7f169016abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:05.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.007+0000 7f168e59c700 1 -- 172.21.15.7:0/2538686280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16900cbe30 msgr2=0x7f169016abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:05.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.007+0000 7f168e59c700 1 --2- 172.21.15.7:0/2538686280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16900cbe30 0x7f169016abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:05.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.007+0000 7f168e59c700 1 -- 172.21.15.7:0/2538686280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16900cba60 msgr2=0x7f16900b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:05.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.007+0000 7f168e59c700 1 --2- 172.21.15.7:0/2538686280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16900cba60 0x7f16900b4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:05.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.007+0000 7f168e59c700 1 -- 172.21.15.7:0/2538686280 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1684009420 con 0x7f169006b4e0 2024-01-27T02:44:05.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.007+0000 7f168ed9d700 1 --2- 172.21.15.7:0/2538686280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16900cbe30 0x7f169016abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:44:05.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.007+0000 7f168dd9b700 1 --2- 172.21.15.7:0/2538686280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16900cba60 0x7f16900b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:44:05.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.007+0000 7f168e59c700 1 --2- 172.21.15.7:0/2538686280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f169006b4e0 0x7f169015cda0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1680015000 tx=0x7f1680016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:05.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.007+0000 7f167e7fc700 1 -- 172.21.15.7:0/2538686280 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16800117d0 con 0x7f169006b4e0 2024-01-27T02:44:05.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.007+0000 7f167e7fc700 1 -- 172.21.15.7:0/2538686280 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1680011930 con 0x7f169006b4e0 2024-01-27T02:44:05.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.007+0000 7f1694edd700 1 -- 172.21.15.7:0/2538686280 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1680009480 con 0x7f169006b4e0 2024-01-27T02:44:05.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.007+0000 7f167e7fc700 1 -- 172.21.15.7:0/2538686280 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1680011cf0 con 0x7f169006b4e0 2024-01-27T02:44:05.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.007+0000 7f1694edd700 1 -- 172.21.15.7:0/2538686280 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16900ac120 con 0x7f169006b4e0 2024-01-27T02:44:05.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.009+0000 7f1694edd700 1 -- 172.21.15.7:0/2538686280 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1690072060 con 0x7f169006b4e0 2024-01-27T02:44:05.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.009+0000 7f167e7fc700 1 -- 172.21.15.7:0/2538686280 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f168001b640 con 0x7f169006b4e0 2024-01-27T02:44:05.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.009+0000 7f167e7fc700 1 --2- 172.21.15.7:0/2538686280 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1670066f30 0x7f16700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:05.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.009+0000 7f167e7fc700 1 -- 172.21.15.7:0/2538686280 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f16800858a0 con 0x7f169006b4e0 2024-01-27T02:44:05.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.009+0000 7f168dd9b700 1 --2- 172.21.15.7:0/2538686280 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1670066f30 0x7f16700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:05.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.010+0000 7f167e7fc700 1 -- 172.21.15.7:0/2538686280 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=186}) v3 -- 0x7f167006eaf0 con 0x7f169006b4e0 2024-01-27T02:44:05.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.010+0000 7f168dd9b700 1 --2- 172.21.15.7:0/2538686280 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1670066f30 0x7f16700693e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f16900af590 tx=0x7f16840091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:05.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.014+0000 7f167e7fc700 1 -- 172.21.15.7:0/2538686280 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1680052ac0 con 0x7f169006b4e0 2024-01-27T02:44:05.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.263+0000 7f1694edd700 1 -- 172.21.15.7:0/2538686280 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f169006b920 con 0x7f169006b4e0 2024-01-27T02:44:05.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.848+0000 7f167e7fc700 1 -- 172.21.15.7:0/2538686280 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v96) v1 ==== 104+0+0 (secure 0 0 0) 0x7f16800560e0 con 0x7f169006b4e0 2024-01-27T02:44:05.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.851+0000 7f1694edd700 1 -- 172.21.15.7:0/2538686280 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1670066f30 msgr2=0x7f16700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:05.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.851+0000 7f1694edd700 1 --2- 172.21.15.7:0/2538686280 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1670066f30 0x7f16700693e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f16900af590 tx=0x7f16840091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:05.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.851+0000 7f1694edd700 1 -- 172.21.15.7:0/2538686280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f169006b4e0 msgr2=0x7f169015cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:05.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.851+0000 7f1694edd700 1 --2- 172.21.15.7:0/2538686280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f169006b4e0 0x7f169015cda0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1680015000 tx=0x7f1680016000 comp rx=0 tx=0).stop 2024-01-27T02:44:05.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.851+0000 7f1694edd700 1 -- 172.21.15.7:0/2538686280 shutdown_connections 2024-01-27T02:44:05.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.851+0000 7f1694edd700 1 --2- 172.21.15.7:0/2538686280 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f1670066f30 0x7f16700693e0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:05.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.851+0000 7f1694edd700 1 --2- 172.21.15.7:0/2538686280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16900cbe30 0x7f169016abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:05.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.851+0000 7f1694edd700 1 --2- 172.21.15.7:0/2538686280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f169006b4e0 0x7f169015cda0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:05.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.851+0000 7f1694edd700 1 --2- 172.21.15.7:0/2538686280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16900cba60 0x7f16900b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:05.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.851+0000 7f1694edd700 1 -- 172.21.15.7:0/2538686280 >> 172.21.15.7:0/2538686280 conn(0x7f1690063580 msgr2=0x7f1690156c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:05.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.851+0000 7f1694edd700 1 -- 172.21.15.7:0/2538686280 shutdown_connections 2024-01-27T02:44:05.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:05.851+0000 7f1694edd700 1 -- 172.21.15.7:0/2538686280 wait complete. 2024-01-27T02:44:05.867 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-27T02:44:06.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.007+0000 7f85302b6700 1 Processor -- start 2024-01-27T02:44:06.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.007+0000 7f85302b6700 1 -- start start 2024-01-27T02:44:06.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.008+0000 7f85302b6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f85280a7fe0 0x7f852806f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:06.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.008+0000 7f85302b6700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f852815dfd0 0x7f852806f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:06.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.008+0000 7f85302b6700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85280afcf0 con 0x7f85280a7c10 2024-01-27T02:44:06.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.008+0000 7f85302b6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85280aea50 con 0x7f85280a7fe0 2024-01-27T02:44:06.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.008+0000 7f85302b6700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f852815a880 con 0x7f852815dfd0 2024-01-27T02:44:06.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.009+0000 7f852d851700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f85280a7fe0 0x7f852806f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:06.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.009+0000 7f852d851700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f85280a7fe0 0x7f852806f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48632/0 (socket says 172.21.15.7:48632) 2024-01-27T02:44:06.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.009+0000 7f852d851700 1 -- 172.21.15.7:0/3262940672 learned_addr learned my addr 172.21.15.7:0/3262940672 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.009+0000 7f852e853700 1 --2- 172.21.15.7:0/3262940672 >> v2:172.21.15.188:3300/0 conn(0x7f852815dfd0 0x7f852806f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.009+0000 7f852d851700 1 -- 172.21.15.7:0/3262940672 >> v2:172.21.15.188:3300/0 conn(0x7f852815dfd0 msgr2=0x7f852806f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.009+0000 7f852d851700 1 --2- 172.21.15.7:0/3262940672 >> v2:172.21.15.188:3300/0 conn(0x7f852815dfd0 0x7f852806f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.009+0000 7f852d851700 1 -- 172.21.15.7:0/3262940672 >> v1:172.21.15.188:6789/0 conn(0x7f85280a7c10 legacy=0x7f85280ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.009+0000 7f852d851700 1 -- 172.21.15.7:0/3262940672 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85280aec50 con 0x7f85280a7fe0 2024-01-27T02:44:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.009+0000 7f852d851700 1 --2- 172.21.15.7:0/3262940672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85280a7fe0 0x7f852806f390 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f8528070060 tx=0x7f851c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44821cce6282050 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.009+0000 7f852d050700 1 -- 172.21.15.7:0/3262940672 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c009e70 con 0x7f85280a7fe0 2024-01-27T02:44:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.009+0000 7f852d050700 1 -- 172.21.15.7:0/3262940672 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c00b860 con 0x7f85280a7fe0 2024-01-27T02:44:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.009+0000 7f852d050700 1 -- 172.21.15.7:0/3262940672 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c00bc00 con 0x7f85280a7fe0 2024-01-27T02:44:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.010+0000 7f85302b6700 1 -- 172.21.15.7:0/3262940672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85280a7fe0 msgr2=0x7f852806f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.010+0000 7f85302b6700 1 --2- 172.21.15.7:0/3262940672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85280a7fe0 0x7f852806f390 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f8528070060 tx=0x7f851c009250 comp rx=0 tx=0).stop 2024-01-27T02:44:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.010+0000 7f85302b6700 1 -- 172.21.15.7:0/3262940672 shutdown_connections 2024-01-27T02:44:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.010+0000 7f85302b6700 1 --2- 172.21.15.7:0/3262940672 >> v2:172.21.15.188:3300/0 conn(0x7f852815dfd0 0x7f852806f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.010+0000 7f85302b6700 1 --2- 172.21.15.7:0/3262940672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85280a7fe0 0x7f852806f390 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.010+0000 7f85302b6700 1 -- 172.21.15.7:0/3262940672 >> 172.21.15.7:0/3262940672 conn(0x7f85280679b0 msgr2=0x7f8528067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.010+0000 7f85302b6700 1 -- 172.21.15.7:0/3262940672 shutdown_connections 2024-01-27T02:44:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.010+0000 7f85302b6700 1 -- 172.21.15.7:0/3262940672 wait complete. 2024-01-27T02:44:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.011+0000 7f85302b6700 1 Processor -- start 2024-01-27T02:44:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.011+0000 7f85302b6700 1 -- start start 2024-01-27T02:44:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.012+0000 7f85302b6700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85280a7c10 0x7f85280cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.012+0000 7f85302b6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85280a7fe0 0x7f852815c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.012+0000 7f85302b6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f852815dfd0 0x7f85281543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.012+0000 7f85302b6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852815d3e0 con 0x7f852815dfd0 2024-01-27T02:44:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.012+0000 7f85302b6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8528152720 con 0x7f85280a7fe0 2024-01-27T02:44:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.012+0000 7f85302b6700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85280ce2d0 con 0x7f85280a7c10 2024-01-27T02:44:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.012+0000 7f852e853700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f852815dfd0 0x7f85281543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.012+0000 7f852e853700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f852815dfd0 0x7f85281543e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48640/0 (socket says 172.21.15.7:48640) 2024-01-27T02:44:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.012+0000 7f852e853700 1 -- 172.21.15.7:0/3179566395 learned_addr learned my addr 172.21.15.7:0/3179566395 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.012+0000 7f852e052700 1 --2- 172.21.15.7:0/3179566395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85280a7c10 0x7f85280cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.012+0000 7f852d851700 1 --2- 172.21.15.7:0/3179566395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85280a7fe0 0x7f852815c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.012+0000 7f852e853700 1 -- 172.21.15.7:0/3179566395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85280a7c10 msgr2=0x7f85280cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.012+0000 7f852e853700 1 --2- 172.21.15.7:0/3179566395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85280a7c10 0x7f85280cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.012+0000 7f852e853700 1 -- 172.21.15.7:0/3179566395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85280a7fe0 msgr2=0x7f852815c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.013+0000 7f852e853700 1 --2- 172.21.15.7:0/3179566395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85280a7fe0 0x7f852815c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.013+0000 7f852e853700 1 -- 172.21.15.7:0/3179566395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c0094f0 con 0x7f852815dfd0 2024-01-27T02:44:06.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.013+0000 7f852d851700 1 --2- 172.21.15.7:0/3179566395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85280a7fe0 0x7f852815c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:44:06.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.013+0000 7f852e853700 1 --2- 172.21.15.7:0/3179566395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f852815dfd0 0x7f85281543e0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f852000e8d0 tx=0x7f85200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:06.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.013+0000 7f85267fc700 1 -- 172.21.15.7:0/3179566395 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852000ee70 con 0x7f852815dfd0 2024-01-27T02:44:06.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.013+0000 7f85267fc700 1 -- 172.21.15.7:0/3179566395 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852000c850 con 0x7f852815dfd0 2024-01-27T02:44:06.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.013+0000 7f85302b6700 1 -- 172.21.15.7:0/3179566395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852815ce20 con 0x7f852815dfd0 2024-01-27T02:44:06.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.013+0000 7f85267fc700 1 -- 172.21.15.7:0/3179566395 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852000cc50 con 0x7f852815dfd0 2024-01-27T02:44:06.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.013+0000 7f85302b6700 1 -- 172.21.15.7:0/3179566395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85281523b0 con 0x7f852815dfd0 2024-01-27T02:44:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.014+0000 7f85302b6700 1 -- 172.21.15.7:0/3179566395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8528152890 con 0x7f852815dfd0 2024-01-27T02:44:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.014+0000 7f85267fc700 1 -- 172.21.15.7:0/3179566395 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8520013680 con 0x7f852815dfd0 2024-01-27T02:44:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.015+0000 7f85267fc700 1 --2- 172.21.15.7:0/3179566395 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8514066f30 0x7f85140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.015+0000 7f85267fc700 1 -- 172.21.15.7:0/3179566395 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f8520084740 con 0x7f852815dfd0 2024-01-27T02:44:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.015+0000 7f85267fc700 1 -- 172.21.15.7:0/3179566395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=186}) v3 -- 0x7f851406eaf0 con 0x7f852815dfd0 2024-01-27T02:44:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.015+0000 7f852e052700 1 --2- 172.21.15.7:0/3179566395 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8514066f30 0x7f85140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:06.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.016+0000 7f852e052700 1 --2- 172.21.15.7:0/3179566395 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8514066f30 0x7f85140693e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f851800e230 tx=0x7f8518009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:06.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.019+0000 7f85267fc700 1 -- 172.21.15.7:0/3179566395 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8520017060 con 0x7f852815dfd0 2024-01-27T02:44:06.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:06.226+0000 7f85302b6700 1 -- 172.21.15.7:0/3179566395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f852804c710 con 0x7f852815dfd0 2024-01-27T02:44:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.006+0000 7f85267fc700 1 -- 172.21.15.7:0/3179566395 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(186..186 src has 1..186) v4 ==== 288+0+0 (secure 0 0 0) 0x7f8520055d20 con 0x7f852815dfd0 2024-01-27T02:44:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.006+0000 7f85267fc700 1 -- 172.21.15.7:0/3179566395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f851406cf20 con 0x7f852815dfd0 2024-01-27T02:44:07.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.013+0000 7f85267fc700 1 -- 172.21.15.7:0/3179566395 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v186) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8520021070 con 0x7f852815dfd0 2024-01-27T02:44:07.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.017+0000 7f85302b6700 1 -- 172.21.15.7:0/3179566395 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8514066f30 msgr2=0x7f85140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:07.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.017+0000 7f85302b6700 1 --2- 172.21.15.7:0/3179566395 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8514066f30 0x7f85140693e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f851800e230 tx=0x7f8518009040 comp rx=0 tx=0).stop 2024-01-27T02:44:07.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.017+0000 7f85302b6700 1 -- 172.21.15.7:0/3179566395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f852815dfd0 msgr2=0x7f85281543e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:07.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.017+0000 7f85302b6700 1 --2- 172.21.15.7:0/3179566395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f852815dfd0 0x7f85281543e0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f852000e8d0 tx=0x7f85200091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:07.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.017+0000 7f85302b6700 1 -- 172.21.15.7:0/3179566395 shutdown_connections 2024-01-27T02:44:07.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.017+0000 7f85302b6700 1 --2- 172.21.15.7:0/3179566395 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8514066f30 0x7f85140693e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:07.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.017+0000 7f85302b6700 1 --2- 172.21.15.7:0/3179566395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f852815dfd0 0x7f85281543e0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:07.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.017+0000 7f85302b6700 1 --2- 172.21.15.7:0/3179566395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85280a7c10 0x7f85280cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:07.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.017+0000 7f85302b6700 1 --2- 172.21.15.7:0/3179566395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85280a7fe0 0x7f852815c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:07.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.017+0000 7f85302b6700 1 -- 172.21.15.7:0/3179566395 >> 172.21.15.7:0/3179566395 conn(0x7f85280679b0 msgr2=0x7f85281519d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:07.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.017+0000 7f85302b6700 1 -- 172.21.15.7:0/3179566395 shutdown_connections 2024-01-27T02:44:07.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.018+0000 7f85302b6700 1 -- 172.21.15.7:0/3179566395 wait complete. 2024-01-27T02:44:07.019 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-27T02:44:07.033 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-27T02:44:07.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.172+0000 7f38d0b54700 1 Processor -- start 2024-01-27T02:44:07.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.172+0000 7f38d0b54700 1 -- start start 2024-01-27T02:44:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.173+0000 7f38d0b54700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38cc06ea40 0x7f38cc0c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.173+0000 7f38d0b54700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38cc0c30c0 con 0x7f38cc0b09f0 2024-01-27T02:44:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.173+0000 7f38d0b54700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38cc06ccf0 con 0x7f38cc1406f0 2024-01-27T02:44:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.173+0000 7f38d0b54700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc06ca10 con 0x7f38cc06ea40 2024-01-27T02:44:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.174+0000 7f38c9d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f38cc0b09f0 0x7f38cc06e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56826/0 (socket says 172.21.15.7:56826) 2024-01-27T02:44:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.174+0000 7f38c9d9b700 1 -- 172.21.15.7:0/3496776291 learned_addr learned my addr 172.21.15.7:0/3496776291 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.174+0000 7f38cad9d700 1 --2- 172.21.15.7:0/3496776291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38cc06ea40 0x7f38cc0c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.174+0000 7f38c959a700 1 -- 172.21.15.7:0/3496776291 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 758466893 0 0) 0x7f38cc0c30c0 con 0x7f38cc0b09f0 2024-01-27T02:44:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.174+0000 7f38c959a700 1 -- 172.21.15.7:0/3496776291 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38b8003570 con 0x7f38cc0b09f0 2024-01-27T02:44:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.174+0000 7f38c959a700 1 -- 172.21.15.7:0/3496776291 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2321248672 0 0) 0x7f38cc06ccf0 con 0x7f38cc1406f0 2024-01-27T02:44:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.174+0000 7f38c959a700 1 -- 172.21.15.7:0/3496776291 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38cc0c30c0 con 0x7f38cc1406f0 2024-01-27T02:44:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.174+0000 7f38cad9d700 1 -- 172.21.15.7:0/3496776291 >> v1:172.21.15.188:6789/0 conn(0x7f38cc1406f0 legacy=0x7f38cc06bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.174+0000 7f38cad9d700 1 -- 172.21.15.7:0/3496776291 >> v1:172.21.15.7:6789/0 conn(0x7f38cc0b09f0 legacy=0x7f38cc06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.174+0000 7f38cad9d700 1 -- 172.21.15.7:0/3496776291 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc0c2a70 con 0x7f38cc06ea40 2024-01-27T02:44:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.174+0000 7f38cad9d700 1 --2- 172.21.15.7:0/3496776291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38cc06ea40 0x7f38cc0c3550 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0c3070 tx=0x7f38c4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abce877259057c05 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.175+0000 7f38c959a700 1 -- 172.21.15.7:0/3496776291 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c4016070 con 0x7f38cc06ea40 2024-01-27T02:44:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.175+0000 7f38c959a700 1 -- 172.21.15.7:0/3496776291 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c400bab0 con 0x7f38cc06ea40 2024-01-27T02:44:07.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.175+0000 7f38c959a700 1 -- 172.21.15.7:0/3496776291 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c400e070 con 0x7f38cc06ea40 2024-01-27T02:44:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.175+0000 7f38d0b54700 1 -- 172.21.15.7:0/3496776291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38cc06ea40 msgr2=0x7f38cc0c3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.175+0000 7f38d0b54700 1 --2- 172.21.15.7:0/3496776291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38cc06ea40 0x7f38cc0c3550 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0c3070 tx=0x7f38c4009780 comp rx=0 tx=0).stop 2024-01-27T02:44:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.175+0000 7f38d0b54700 1 -- 172.21.15.7:0/3496776291 shutdown_connections 2024-01-27T02:44:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.175+0000 7f38d0b54700 1 --2- 172.21.15.7:0/3496776291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38cc06ea40 0x7f38cc0c3550 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:07.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.175+0000 7f38d0b54700 1 -- 172.21.15.7:0/3496776291 >> 172.21.15.7:0/3496776291 conn(0x7f38cc0cdc30 msgr2=0x7f38cc0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.175+0000 7f38d0b54700 1 -- 172.21.15.7:0/3496776291 shutdown_connections 2024-01-27T02:44:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.175+0000 7f38d0b54700 1 -- 172.21.15.7:0/3496776291 wait complete. 2024-01-27T02:44:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.177+0000 7f38d0b54700 1 Processor -- start 2024-01-27T02:44:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.177+0000 7f38d0b54700 1 -- start start 2024-01-27T02:44:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.177+0000 7f38d0b54700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38cc06ea40 0x7f38cc136960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.177+0000 7f38d0b54700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38cc0b09f0 0x7f38cc0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.177+0000 7f38d0b54700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38cc1406f0 0x7f38cc139ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.177+0000 7f38d0b54700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc137c00 con 0x7f38cc0b09f0 2024-01-27T02:44:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.177+0000 7f38d0b54700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc0b2440 con 0x7f38cc1406f0 2024-01-27T02:44:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.177+0000 7f38d0b54700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc136ff0 con 0x7f38cc06ea40 2024-01-27T02:44:07.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.178+0000 7f38c9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38cc0b09f0 0x7f38cc0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.178+0000 7f38c9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38cc0b09f0 0x7f38cc0b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48654/0 (socket says 172.21.15.7:48654) 2024-01-27T02:44:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.178+0000 7f38c9d9b700 1 -- 172.21.15.7:0/2734546793 learned_addr learned my addr 172.21.15.7:0/2734546793 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.178+0000 7f38ca59c700 1 --2- 172.21.15.7:0/2734546793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38cc06ea40 0x7f38cc136960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.178+0000 7f38cad9d700 1 --2- 172.21.15.7:0/2734546793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38cc1406f0 0x7f38cc139ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.178+0000 7f38c9d9b700 1 -- 172.21.15.7:0/2734546793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38cc06ea40 msgr2=0x7f38cc136960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.178+0000 7f38c9d9b700 1 --2- 172.21.15.7:0/2734546793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38cc06ea40 0x7f38cc136960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.178+0000 7f38c9d9b700 1 -- 172.21.15.7:0/2734546793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38cc1406f0 msgr2=0x7f38cc139ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.178+0000 7f38c9d9b700 1 --2- 172.21.15.7:0/2734546793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38cc1406f0 0x7f38cc139ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.178+0000 7f38c9d9b700 1 -- 172.21.15.7:0/2734546793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c4009a20 con 0x7f38cc0b09f0 2024-01-27T02:44:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.178+0000 7f38c9d9b700 1 --2- 172.21.15.7:0/2734546793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38cc0b09f0 0x7f38cc0b1f00 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f38b400c790 tx=0x7f38b4007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.178+0000 7f38c27fc700 1 -- 172.21.15.7:0/2734546793 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b4005b30 con 0x7f38cc0b09f0 2024-01-27T02:44:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.178+0000 7f38c27fc700 1 -- 172.21.15.7:0/2734546793 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b4005c90 con 0x7f38cc0b09f0 2024-01-27T02:44:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.178+0000 7f38c27fc700 1 -- 172.21.15.7:0/2734546793 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b4007300 con 0x7f38cc0b09f0 2024-01-27T02:44:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.178+0000 7f38d0b54700 1 -- 172.21.15.7:0/2734546793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38cc1372d0 con 0x7f38cc0b09f0 2024-01-27T02:44:07.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.178+0000 7f38d0b54700 1 -- 172.21.15.7:0/2734546793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38cc137890 con 0x7f38cc0b09f0 2024-01-27T02:44:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.180+0000 7f38d0b54700 1 -- 172.21.15.7:0/2734546793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38cc04c710 con 0x7f38cc0b09f0 2024-01-27T02:44:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.180+0000 7f38c27fc700 1 -- 172.21.15.7:0/2734546793 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f38b401b3b0 con 0x7f38cc0b09f0 2024-01-27T02:44:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.180+0000 7f38c27fc700 1 --2- 172.21.15.7:0/2734546793 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f38ac066f30 0x7f38ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.180+0000 7f38c27fc700 1 -- 172.21.15.7:0/2734546793 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f38b40838f0 con 0x7f38cc0b09f0 2024-01-27T02:44:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.181+0000 7f38c27fc700 1 -- 172.21.15.7:0/2734546793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f38ac06dcf0 con 0x7f38cc0b09f0 2024-01-27T02:44:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.181+0000 7f38ca59c700 1 --2- 172.21.15.7:0/2734546793 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f38ac066f30 0x7f38ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:07.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.181+0000 7f38ca59c700 1 --2- 172.21.15.7:0/2734546793 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f38ac066f30 0x7f38ac0693e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f38bc007fd0 tx=0x7f38bc007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:07.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.185+0000 7f38c27fc700 1 -- 172.21.15.7:0/2734546793 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38b4017020 con 0x7f38cc0b09f0 2024-01-27T02:44:07.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:07.387+0000 7f38d0b54700 1 -- 172.21.15.7:0/2734546793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f38cc1570f0 con 0x7f38cc0b09f0 2024-01-27T02:44:08.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.021+0000 7f38c27fc700 1 -- 172.21.15.7:0/2734546793 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(187..187 src has 1..187) v4 ==== 288+0+0 (secure 0 0 0) 0x7f38b4055080 con 0x7f38cc0b09f0 2024-01-27T02:44:08.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.021+0000 7f38c27fc700 1 -- 172.21.15.7:0/2734546793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=188}) v3 -- 0x7f38ac06ca80 con 0x7f38cc0b09f0 2024-01-27T02:44:08.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.024+0000 7f38c27fc700 1 -- 172.21.15.7:0/2734546793 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v187) v1 ==== 169+0+0 (secure 0 0 0) 0x7f38b4054750 con 0x7f38cc0b09f0 2024-01-27T02:44:08.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.035+0000 7f38d0b54700 1 -- 172.21.15.7:0/2734546793 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f38ac066f30 msgr2=0x7f38ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:08.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.035+0000 7f38d0b54700 1 --2- 172.21.15.7:0/2734546793 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f38ac066f30 0x7f38ac0693e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f38bc007fd0 tx=0x7f38bc007d30 comp rx=0 tx=0).stop 2024-01-27T02:44:08.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.035+0000 7f38d0b54700 1 -- 172.21.15.7:0/2734546793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38cc0b09f0 msgr2=0x7f38cc0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:08.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.035+0000 7f38d0b54700 1 --2- 172.21.15.7:0/2734546793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38cc0b09f0 0x7f38cc0b1f00 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f38b400c790 tx=0x7f38b4007ea0 comp rx=0 tx=0).stop 2024-01-27T02:44:08.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.036+0000 7f38d0b54700 1 -- 172.21.15.7:0/2734546793 shutdown_connections 2024-01-27T02:44:08.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.036+0000 7f38d0b54700 1 --2- 172.21.15.7:0/2734546793 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f38ac066f30 0x7f38ac0693e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:08.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.036+0000 7f38d0b54700 1 --2- 172.21.15.7:0/2734546793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38cc1406f0 0x7f38cc139ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:08.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.036+0000 7f38d0b54700 1 --2- 172.21.15.7:0/2734546793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38cc06ea40 0x7f38cc136960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:08.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.036+0000 7f38d0b54700 1 --2- 172.21.15.7:0/2734546793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38cc0b09f0 0x7f38cc0b1f00 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:08.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.036+0000 7f38d0b54700 1 -- 172.21.15.7:0/2734546793 >> 172.21.15.7:0/2734546793 conn(0x7f38cc0cdc30 msgr2=0x7f38cc0ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:08.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.036+0000 7f38d0b54700 1 -- 172.21.15.7:0/2734546793 shutdown_connections 2024-01-27T02:44:08.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.037+0000 7f38d0b54700 1 -- 172.21.15.7:0/2734546793 wait complete. 2024-01-27T02:44:08.039 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-27T02:44:08.053 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-27T02:44:08.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.190+0000 7fdf310dc700 1 Processor -- start 2024-01-27T02:44:08.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.190+0000 7fdf310dc700 1 -- start start 2024-01-27T02:44:08.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.191+0000 7fdf310dc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdf2c06e630 0x7fdf2c0c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:08.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.191+0000 7fdf310dc700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdf2c0ca2f0 0x7fdf2c0cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:08.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.191+0000 7fdf310dc700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf2c06ead0 con 0x7fdf2c0af7e0 2024-01-27T02:44:08.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.191+0000 7fdf310dc700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2c0b1ae0 con 0x7fdf2c06e630 2024-01-27T02:44:08.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.191+0000 7fdf310dc700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2c0ca770 con 0x7fdf2c0ca2f0 2024-01-27T02:44:08.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.191+0000 7fdf2a59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdf2c06e630 0x7fdf2c0c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:08.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.191+0000 7fdf2ad9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdf2c0af7e0 0x7fdf2c0b13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56838/0 (socket says 172.21.15.7:56838) 2024-01-27T02:44:08.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.191+0000 7fdf2ad9d700 1 -- 172.21.15.7:0/4110282863 learned_addr learned my addr 172.21.15.7:0/4110282863 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:08.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.192+0000 7fdf2b59e700 1 --2- 172.21.15.7:0/4110282863 >> v2:172.21.15.188:3300/0 conn(0x7fdf2c0ca2f0 0x7fdf2c0cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:08.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.192+0000 7fdf2a59c700 1 -- 172.21.15.7:0/4110282863 >> v2:172.21.15.188:3300/0 conn(0x7fdf2c0ca2f0 msgr2=0x7fdf2c0cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:08.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.192+0000 7fdf2a59c700 1 --2- 172.21.15.7:0/4110282863 >> v2:172.21.15.188:3300/0 conn(0x7fdf2c0ca2f0 0x7fdf2c0cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:08.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.192+0000 7fdf2a59c700 1 -- 172.21.15.7:0/4110282863 >> v1:172.21.15.7:6789/0 conn(0x7fdf2c0af7e0 legacy=0x7fdf2c0b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:08.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.192+0000 7fdf2a59c700 1 -- 172.21.15.7:0/4110282863 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c06ed10 con 0x7fdf2c06e630 2024-01-27T02:44:08.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.192+0000 7fdf29d9b700 1 -- 172.21.15.7:0/4110282863 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585076342 0 0) 0x7fdf1c001110 con 0x7fdf2c0af7e0 2024-01-27T02:44:08.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.192+0000 7fdf2a59c700 1 --2- 172.21.15.7:0/4110282863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf2c06e630 0x7fdf2c0c9db0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0caba0 tx=0x7fdf200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82823e2bdf788c7c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:08.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.192+0000 7fdf29d9b700 1 -- 172.21.15.7:0/4110282863 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf20009e80 con 0x7fdf2c06e630 2024-01-27T02:44:08.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.192+0000 7fdf29d9b700 1 -- 172.21.15.7:0/4110282863 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2000b820 con 0x7fdf2c06e630 2024-01-27T02:44:08.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.192+0000 7fdf29d9b700 1 -- 172.21.15.7:0/4110282863 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2000bba0 con 0x7fdf2c06e630 2024-01-27T02:44:08.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.193+0000 7fdf310dc700 1 -- 172.21.15.7:0/4110282863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf2c06e630 msgr2=0x7fdf2c0c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:08.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.193+0000 7fdf310dc700 1 --2- 172.21.15.7:0/4110282863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf2c06e630 0x7fdf2c0c9db0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0caba0 tx=0x7fdf200091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:08.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.193+0000 7fdf310dc700 1 -- 172.21.15.7:0/4110282863 shutdown_connections 2024-01-27T02:44:08.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.193+0000 7fdf310dc700 1 --2- 172.21.15.7:0/4110282863 >> v2:172.21.15.188:3300/0 conn(0x7fdf2c0ca2f0 0x7fdf2c0cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:08.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.193+0000 7fdf310dc700 1 --2- 172.21.15.7:0/4110282863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf2c06e630 0x7fdf2c0c9db0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:08.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.193+0000 7fdf310dc700 1 -- 172.21.15.7:0/4110282863 >> 172.21.15.7:0/4110282863 conn(0x7fdf2c0cd3b0 msgr2=0x7fdf2c0cd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:08.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.193+0000 7fdf310dc700 1 -- 172.21.15.7:0/4110282863 shutdown_connections 2024-01-27T02:44:08.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.193+0000 7fdf310dc700 1 -- 172.21.15.7:0/4110282863 wait complete. 2024-01-27T02:44:08.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.195+0000 7fdf310dc700 1 Processor -- start 2024-01-27T02:44:08.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.195+0000 7fdf310dc700 1 -- start start 2024-01-27T02:44:08.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.196+0000 7fdf310dc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf2c06e630 0x7fdf2c0c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:08.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.196+0000 7fdf310dc700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf2c0af7e0 0x7fdf2c097a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:08.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.196+0000 7fdf310dc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf2c0ca2f0 0x7fdf2c158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:08.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.196+0000 7fdf310dc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c098b70 con 0x7fdf2c0ca2f0 2024-01-27T02:44:08.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.196+0000 7fdf310dc700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c0c77f0 con 0x7fdf2c06e630 2024-01-27T02:44:08.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.196+0000 7fdf310dc700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c097f80 con 0x7fdf2c0af7e0 2024-01-27T02:44:08.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.196+0000 7fdf2b59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf2c0ca2f0 0x7fdf2c158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:08.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.196+0000 7fdf2b59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf2c0ca2f0 0x7fdf2c158d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48664/0 (socket says 172.21.15.7:48664) 2024-01-27T02:44:08.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.196+0000 7fdf2b59e700 1 -- 172.21.15.7:0/2520546568 learned_addr learned my addr 172.21.15.7:0/2520546568 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:08.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.196+0000 7fdf2ad9d700 1 --2- 172.21.15.7:0/2520546568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf2c06e630 0x7fdf2c0c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:08.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.196+0000 7fdf2a59c700 1 --2- 172.21.15.7:0/2520546568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf2c0af7e0 0x7fdf2c097a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:08.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.196+0000 7fdf2b59e700 1 -- 172.21.15.7:0/2520546568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf2c0af7e0 msgr2=0x7fdf2c097a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:08.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.196+0000 7fdf2b59e700 1 --2- 172.21.15.7:0/2520546568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf2c0af7e0 0x7fdf2c097a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:08.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.196+0000 7fdf2b59e700 1 -- 172.21.15.7:0/2520546568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf2c06e630 msgr2=0x7fdf2c0c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:08.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.196+0000 7fdf2b59e700 1 --2- 172.21.15.7:0/2520546568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf2c06e630 0x7fdf2c0c79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:08.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.196+0000 7fdf2b59e700 1 -- 172.21.15.7:0/2520546568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf20009420 con 0x7fdf2c0ca2f0 2024-01-27T02:44:08.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.196+0000 7fdf2b59e700 1 --2- 172.21.15.7:0/2520546568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf2c0ca2f0 0x7fdf2c158d40 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fdf24007f20 tx=0x7fdf24009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:08.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.197+0000 7fdf12ffd700 1 -- 172.21.15.7:0/2520546568 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2400eb90 con 0x7fdf2c0ca2f0 2024-01-27T02:44:08.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.197+0000 7fdf12ffd700 1 -- 172.21.15.7:0/2520546568 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2400ed10 con 0x7fdf2c0ca2f0 2024-01-27T02:44:08.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.197+0000 7fdf310dc700 1 -- 172.21.15.7:0/2520546568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2c0c7230 con 0x7fdf2c0ca2f0 2024-01-27T02:44:08.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.197+0000 7fdf12ffd700 1 -- 172.21.15.7:0/2520546568 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24014670 con 0x7fdf2c0ca2f0 2024-01-27T02:44:08.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.197+0000 7fdf310dc700 1 -- 172.21.15.7:0/2520546568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2c098520 con 0x7fdf2c0ca2f0 2024-01-27T02:44:08.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.198+0000 7fdf310dc700 1 -- 172.21.15.7:0/2520546568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2c15c3d0 con 0x7fdf2c0ca2f0 2024-01-27T02:44:08.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.198+0000 7fdf12ffd700 1 -- 172.21.15.7:0/2520546568 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fdf240149e0 con 0x7fdf2c0ca2f0 2024-01-27T02:44:08.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.199+0000 7fdf12ffd700 1 --2- 172.21.15.7:0/2520546568 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdf18066f30 0x7fdf180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:08.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.199+0000 7fdf12ffd700 1 -- 172.21.15.7:0/2520546568 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fdf240847d0 con 0x7fdf2c0ca2f0 2024-01-27T02:44:08.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.199+0000 7fdf2ad9d700 1 --2- 172.21.15.7:0/2520546568 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdf18066f30 0x7fdf180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:08.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.200+0000 7fdf2ad9d700 1 --2- 172.21.15.7:0/2520546568 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdf18066f30 0x7fdf180693e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c006fc0 tx=0x7fdf1c007b90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:08.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.203+0000 7fdf12ffd700 1 -- 172.21.15.7:0/2520546568 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf2c15c3d0 con 0x7fdf2c0ca2f0 2024-01-27T02:44:08.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.406+0000 7fdf310dc700 1 -- 172.21.15.7:0/2520546568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fdf2c15c3d0 con 0x7fdf2c0ca2f0 2024-01-27T02:44:08.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.410+0000 7fdf12ffd700 1 -- 172.21.15.7:0/2520546568 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdf2c15c3d0 con 0x7fdf2c0ca2f0 2024-01-27T02:44:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.413+0000 7fdf310dc700 1 -- 172.21.15.7:0/2520546568 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdf18066f30 msgr2=0x7fdf180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.413+0000 7fdf310dc700 1 --2- 172.21.15.7:0/2520546568 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdf18066f30 0x7fdf180693e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c006fc0 tx=0x7fdf1c007b90 comp rx=0 tx=0).stop 2024-01-27T02:44:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.413+0000 7fdf310dc700 1 -- 172.21.15.7:0/2520546568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf2c0ca2f0 msgr2=0x7fdf2c158d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.413+0000 7fdf310dc700 1 --2- 172.21.15.7:0/2520546568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf2c0ca2f0 0x7fdf2c158d40 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fdf24007f20 tx=0x7fdf24009860 comp rx=0 tx=0).stop 2024-01-27T02:44:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.413+0000 7fdf310dc700 1 -- 172.21.15.7:0/2520546568 shutdown_connections 2024-01-27T02:44:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.413+0000 7fdf310dc700 1 --2- 172.21.15.7:0/2520546568 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fdf18066f30 0x7fdf180693e0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.413+0000 7fdf310dc700 1 --2- 172.21.15.7:0/2520546568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf2c0ca2f0 0x7fdf2c158d40 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.413+0000 7fdf310dc700 1 --2- 172.21.15.7:0/2520546568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf2c06e630 0x7fdf2c0c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.413+0000 7fdf310dc700 1 --2- 172.21.15.7:0/2520546568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf2c0af7e0 0x7fdf2c097a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.413+0000 7fdf310dc700 1 -- 172.21.15.7:0/2520546568 >> 172.21.15.7:0/2520546568 conn(0x7fdf2c0cd3b0 msgr2=0x7fdf2c069620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:08.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.413+0000 7fdf310dc700 1 -- 172.21.15.7:0/2520546568 shutdown_connections 2024-01-27T02:44:08.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.413+0000 7fdf310dc700 1 -- 172.21.15.7:0/2520546568 wait complete. 2024-01-27T02:44:08.427 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2024-01-27T02:44:08.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.559+0000 7f2409be1700 1 Processor -- start 2024-01-27T02:44:08.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.559+0000 7f2409be1700 1 -- start start 2024-01-27T02:44:08.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.560+0000 7f2409be1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24040c9f80 0x7f2404157f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:08.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.560+0000 7f2409be1700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f240406e330 0x7f240406e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:08.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.560+0000 7f2409be1700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f240406c880 con 0x7f24040c9bb0 2024-01-27T02:44:08.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.560+0000 7f2409be1700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f240406cd10 con 0x7f240406e330 2024-01-27T02:44:08.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.560+0000 7f2409be1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f240406ecf0 con 0x7f24040c9f80 2024-01-27T02:44:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.560+0000 7f24037fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f24040c9bb0 0x7f240406b400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56848/0 (socket says 172.21.15.7:56848) 2024-01-27T02:44:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.560+0000 7f24037fe700 1 -- 172.21.15.7:0/893125453 learned_addr learned my addr 172.21.15.7:0/893125453 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.560+0000 7f2402ffd700 1 --2- 172.21.15.7:0/893125453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24040c9f80 0x7f2404157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.560+0000 7f2403fff700 1 --2- 172.21.15.7:0/893125453 >> v2:172.21.15.188:3300/0 conn(0x7f240406e330 0x7f240406e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.560+0000 7f24027fc700 1 -- 172.21.15.7:0/893125453 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1674359145 0 0) 0x7f240406c880 con 0x7f24040c9bb0 2024-01-27T02:44:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.560+0000 7f24027fc700 1 -- 172.21.15.7:0/893125453 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23ec003570 con 0x7f24040c9bb0 2024-01-27T02:44:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.560+0000 7f24027fc700 1 -- 172.21.15.7:0/893125453 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2841169016 0 0) 0x7f23ec003570 con 0x7f24040c9bb0 2024-01-27T02:44:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.560+0000 7f2402ffd700 1 -- 172.21.15.7:0/893125453 >> v2:172.21.15.188:3300/0 conn(0x7f240406e330 msgr2=0x7f240406e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.560+0000 7f2402ffd700 1 --2- 172.21.15.7:0/893125453 >> v2:172.21.15.188:3300/0 conn(0x7f240406e330 0x7f240406e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.560+0000 7f2402ffd700 1 -- 172.21.15.7:0/893125453 >> v1:172.21.15.7:6789/0 conn(0x7f24040c9bb0 legacy=0x7f240406b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.560+0000 7f2402ffd700 1 -- 172.21.15.7:0/893125453 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240406bc70 con 0x7f24040c9f80 2024-01-27T02:44:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.560+0000 7f2403fff700 1 --2- 172.21.15.7:0/893125453 >> v2:172.21.15.188:3300/0 conn(0x7f240406e330 0x7f240406e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:44:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.561+0000 7f2402ffd700 1 --2- 172.21.15.7:0/893125453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24040c9f80 0x7f2404157f70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f23f80098e0 tx=0x7f23f80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a0a4a874b1f4b38 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:08.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.561+0000 7f24027fc700 1 -- 172.21.15.7:0/893125453 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4001470 con 0x7f24040c9f80 2024-01-27T02:44:08.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.561+0000 7f24027fc700 1 -- 172.21.15.7:0/893125453 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f8011420 con 0x7f24040c9f80 2024-01-27T02:44:08.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.561+0000 7f24027fc700 1 -- 172.21.15.7:0/893125453 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f80117a0 con 0x7f24040c9f80 2024-01-27T02:44:08.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.561+0000 7f2409be1700 1 -- 172.21.15.7:0/893125453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24040c9f80 msgr2=0x7f2404157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:08.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.561+0000 7f2409be1700 1 --2- 172.21.15.7:0/893125453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24040c9f80 0x7f2404157f70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f23f80098e0 tx=0x7f23f80091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:08.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.562+0000 7f2409be1700 1 -- 172.21.15.7:0/893125453 shutdown_connections 2024-01-27T02:44:08.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.562+0000 7f2409be1700 1 --2- 172.21.15.7:0/893125453 >> v2:172.21.15.188:3300/0 conn(0x7f240406e330 0x7f240406e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:08.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.562+0000 7f2409be1700 1 --2- 172.21.15.7:0/893125453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24040c9f80 0x7f2404157f70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:08.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.562+0000 7f2409be1700 1 -- 172.21.15.7:0/893125453 >> 172.21.15.7:0/893125453 conn(0x7f24040cdc30 msgr2=0x7f24040ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:08.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.562+0000 7f2409be1700 1 -- 172.21.15.7:0/893125453 shutdown_connections 2024-01-27T02:44:08.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.562+0000 7f2409be1700 1 -- 172.21.15.7:0/893125453 wait complete. 2024-01-27T02:44:08.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.563+0000 7f2409be1700 1 Processor -- start 2024-01-27T02:44:08.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.563+0000 7f2409be1700 1 -- start start 2024-01-27T02:44:08.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.564+0000 7f2409be1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240406e330 0x7f24041396a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:08.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.564+0000 7f2409be1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24040c9bb0 0x7f24040c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.564+0000 7f2409be1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24040c9f80 0x7f24040cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.564+0000 7f2409be1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24041464b0 con 0x7f240406e330 2024-01-27T02:44:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.564+0000 7f2409be1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24040b3880 con 0x7f24040c9bb0 2024-01-27T02:44:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.564+0000 7f2409be1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2404145e60 con 0x7f24040c9f80 2024-01-27T02:44:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.564+0000 7f24037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240406e330 0x7f24041396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.564+0000 7f24037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240406e330 0x7f24041396a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48678/0 (socket says 172.21.15.7:48678) 2024-01-27T02:44:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.564+0000 7f24037fe700 1 -- 172.21.15.7:0/875816935 learned_addr learned my addr 172.21.15.7:0/875816935 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.564+0000 7f2402ffd700 1 --2- 172.21.15.7:0/875816935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24040c9bb0 0x7f24040c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.564+0000 7f2403fff700 1 --2- 172.21.15.7:0/875816935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24040c9f80 0x7f24040cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.564+0000 7f24037fe700 1 -- 172.21.15.7:0/875816935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24040c9f80 msgr2=0x7f24040cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.564+0000 7f24037fe700 1 --2- 172.21.15.7:0/875816935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24040c9f80 0x7f24040cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.564+0000 7f24037fe700 1 -- 172.21.15.7:0/875816935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24040c9bb0 msgr2=0x7f24040c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.564+0000 7f24037fe700 1 --2- 172.21.15.7:0/875816935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24040c9bb0 0x7f24040c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.564+0000 7f24037fe700 1 -- 172.21.15.7:0/875816935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23fc009a20 con 0x7f240406e330 2024-01-27T02:44:08.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.564+0000 7f2403fff700 1 --2- 172.21.15.7:0/875816935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24040c9f80 0x7f24040cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:44:08.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.564+0000 7f2402ffd700 1 --2- 172.21.15.7:0/875816935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24040c9bb0 0x7f24040c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:44:08.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.564+0000 7f24037fe700 1 --2- 172.21.15.7:0/875816935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240406e330 0x7f24041396a0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f23f4001660 tx=0x7f23f4001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:08.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.565+0000 7f23eb7fe700 1 -- 172.21.15.7:0/875816935 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4001d10 con 0x7f240406e330 2024-01-27T02:44:08.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.565+0000 7f23eb7fe700 1 -- 172.21.15.7:0/875816935 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f4007360 con 0x7f240406e330 2024-01-27T02:44:08.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.565+0000 7f2409be1700 1 -- 172.21.15.7:0/875816935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f8009420 con 0x7f240406e330 2024-01-27T02:44:08.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.565+0000 7f23eb7fe700 1 -- 172.21.15.7:0/875816935 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4004830 con 0x7f240406e330 2024-01-27T02:44:08.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.565+0000 7f2409be1700 1 -- 172.21.15.7:0/875816935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2404139be0 con 0x7f240406e330 2024-01-27T02:44:08.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.566+0000 7f2409be1700 1 -- 172.21.15.7:0/875816935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f240404c710 con 0x7f240406e330 2024-01-27T02:44:08.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.566+0000 7f23eb7fe700 1 -- 172.21.15.7:0/875816935 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f23f4004c20 con 0x7f240406e330 2024-01-27T02:44:08.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.567+0000 7f23eb7fe700 1 --2- 172.21.15.7:0/875816935 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f23f0066f30 0x7f23f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:08.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.567+0000 7f23eb7fe700 1 -- 172.21.15.7:0/875816935 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f23f4083a50 con 0x7f240406e330 2024-01-27T02:44:08.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.567+0000 7f2402ffd700 1 --2- 172.21.15.7:0/875816935 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f23f0066f30 0x7f23f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:08.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.567+0000 7f2402ffd700 1 --2- 172.21.15.7:0/875816935 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f23f0066f30 0x7f23f00693e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2404145c50 tx=0x7f23f8016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:08.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.571+0000 7f23eb7fe700 1 -- 172.21.15.7:0/875816935 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f23f4051040 con 0x7f240406e330 2024-01-27T02:44:08.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:08.860+0000 7f2409be1700 1 -- 172.21.15.7:0/875816935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) v1 -- 0x7f2404148340 con 0x7f240406e330 2024-01-27T02:44:09.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.024+0000 7f23eb7fe700 1 -- 172.21.15.7:0/875816935 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v566) v1 ==== 135+0+0 (secure 0 0 0) 0x7f23f4018020 con 0x7f240406e330 2024-01-27T02:44:09.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.027+0000 7f2409be1700 1 -- 172.21.15.7:0/875816935 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f23f0066f30 msgr2=0x7f23f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:09.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.027+0000 7f2409be1700 1 --2- 172.21.15.7:0/875816935 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f23f0066f30 0x7f23f00693e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2404145c50 tx=0x7f23f8016000 comp rx=0 tx=0).stop 2024-01-27T02:44:09.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.027+0000 7f2409be1700 1 -- 172.21.15.7:0/875816935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240406e330 msgr2=0x7f24041396a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:09.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.027+0000 7f2409be1700 1 --2- 172.21.15.7:0/875816935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240406e330 0x7f24041396a0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f23f4001660 tx=0x7f23f4001a80 comp rx=0 tx=0).stop 2024-01-27T02:44:09.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.027+0000 7f2409be1700 1 -- 172.21.15.7:0/875816935 shutdown_connections 2024-01-27T02:44:09.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.027+0000 7f2409be1700 1 --2- 172.21.15.7:0/875816935 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f23f0066f30 0x7f23f00693e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:09.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.027+0000 7f2409be1700 1 --2- 172.21.15.7:0/875816935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24040c9f80 0x7f24040cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:09.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.027+0000 7f2409be1700 1 --2- 172.21.15.7:0/875816935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240406e330 0x7f24041396a0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:09.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.027+0000 7f2409be1700 1 --2- 172.21.15.7:0/875816935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24040c9bb0 0x7f24040c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:09.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.027+0000 7f2409be1700 1 -- 172.21.15.7:0/875816935 >> 172.21.15.7:0/875816935 conn(0x7f24040cdc30 msgr2=0x7f240414bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:09.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.027+0000 7f2409be1700 1 -- 172.21.15.7:0/875816935 shutdown_connections 2024-01-27T02:44:09.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.027+0000 7f2409be1700 1 -- 172.21.15.7:0/875816935 wait complete. 2024-01-27T02:44:09.043 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2024-01-27T02:44:09.043 INFO:tasks.cephfs_test_runner: 2024-01-27T02:44:09.043 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-01-27T02:44:09.043 INFO:tasks.cephfs_test_runner:Ran 10 tests in 690.975s 2024-01-27T02:44:09.043 INFO:tasks.cephfs_test_runner: 2024-01-27T02:44:09.043 INFO:tasks.cephfs_test_runner:OK 2024-01-27T02:44:09.043 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-01-27T02:44:09.052 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-01-27T02:44:09.060 INFO:tasks.kclient:Unmounting kernel clients... 2024-01-27T02:44:09.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:44:09.060 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-27T02:44:09.081 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-27T02:44:09.081 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-27T02:44:09.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:44:09.081 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-27T02:44:09.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:44:09.171 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-27T02:44:09.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:44:09.241 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-01-27T02:44:09.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:44:09.328 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-27T02:44:09.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:44:09.351 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-01-27T02:44:09.431 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2024-01-27T02:44:09.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-27T02:44:09.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:44:09.434 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-01-27T02:44:09.455 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-27T02:44:09.455 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-27T02:44:09.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:44:09.456 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-27T02:44:09.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:44:09.543 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-27T02:44:09.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:44:09.601 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2024-01-27T02:44:09.642 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:44:09.644 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:44:09.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:44:09.692 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-01-27T02:44:09.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-27T02:44:09.709 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2024-01-27T02:44:09.780 INFO:teuthology.orchestra.run.smithi188.stderr:Cannot find device "ceph-brx" 2024-01-27T02:44:09.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-27T02:44:09.782 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-01-27T02:44:09.790 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-01-27T02:44:09.790 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:44:09.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.931+0000 7ffb88e53700 1 Processor -- start 2024-01-27T02:44:09.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.931+0000 7ffb88e53700 1 -- start start 2024-01-27T02:44:09.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.932+0000 7ffb88e53700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffb84157ca0 0x7ffb840c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:09.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.932+0000 7ffb88e53700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb840cdbf0 0x7ffb840ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:09.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.932+0000 7ffb88e53700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb840ca020 con 0x7ffb840c3ae0 2024-01-27T02:44:09.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.932+0000 7ffb88e53700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb840c4f40 con 0x7ffb84157ca0 2024-01-27T02:44:09.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.932+0000 7ffb88e53700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb840c5210 con 0x7ffb840cdbf0 2024-01-27T02:44:09.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.932+0000 7ffb81d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ffb840c3ae0 0x7ffb840cd4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56860/0 (socket says 172.21.15.7:56860) 2024-01-27T02:44:09.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.932+0000 7ffb81d9b700 1 -- 172.21.15.7:0/3572761079 learned_addr learned my addr 172.21.15.7:0/3572761079 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:09.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.932+0000 7ffb8259c700 1 --2- 172.21.15.7:0/3572761079 >> v2:172.21.15.188:3300/0 conn(0x7ffb84157ca0 0x7ffb840c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:09.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.932+0000 7ffb82d9d700 1 --2- 172.21.15.7:0/3572761079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb840cdbf0 0x7ffb840ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:09.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.933+0000 7ffb8159a700 1 -- 172.21.15.7:0/3572761079 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 300670892 0 0) 0x7ffb840ca020 con 0x7ffb840c3ae0 2024-01-27T02:44:09.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.933+0000 7ffb8159a700 1 -- 172.21.15.7:0/3572761079 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb70003570 con 0x7ffb840c3ae0 2024-01-27T02:44:09.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.933+0000 7ffb82d9d700 1 -- 172.21.15.7:0/3572761079 >> v2:172.21.15.188:3300/0 conn(0x7ffb84157ca0 msgr2=0x7ffb840c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:09.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.933+0000 7ffb82d9d700 1 --2- 172.21.15.7:0/3572761079 >> v2:172.21.15.188:3300/0 conn(0x7ffb84157ca0 0x7ffb840c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:09.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.933+0000 7ffb82d9d700 1 -- 172.21.15.7:0/3572761079 >> v1:172.21.15.7:6789/0 conn(0x7ffb840c3ae0 legacy=0x7ffb840cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:09.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.933+0000 7ffb82d9d700 1 -- 172.21.15.7:0/3572761079 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb840c54f0 con 0x7ffb840cdbf0 2024-01-27T02:44:09.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.933+0000 7ffb8259c700 1 --2- 172.21.15.7:0/3572761079 >> v2:172.21.15.188:3300/0 conn(0x7ffb84157ca0 0x7ffb840c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:44:09.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.933+0000 7ffb82d9d700 1 --2- 172.21.15.7:0/3572761079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb840cdbf0 0x7ffb840ca4b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009e60 tx=0x7ffb7c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba489832fee0e403 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:09.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.933+0000 7ffb8159a700 1 -- 172.21.15.7:0/3572761079 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c00e070 con 0x7ffb840cdbf0 2024-01-27T02:44:09.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.933+0000 7ffb8159a700 1 -- 172.21.15.7:0/3572761079 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c00bab0 con 0x7ffb840cdbf0 2024-01-27T02:44:09.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.933+0000 7ffb8159a700 1 -- 172.21.15.7:0/3572761079 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c00be70 con 0x7ffb840cdbf0 2024-01-27T02:44:09.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.934+0000 7ffb88e53700 1 -- 172.21.15.7:0/3572761079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb840cdbf0 msgr2=0x7ffb840ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:09.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.934+0000 7ffb88e53700 1 --2- 172.21.15.7:0/3572761079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb840cdbf0 0x7ffb840ca4b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009e60 tx=0x7ffb7c009780 comp rx=0 tx=0).stop 2024-01-27T02:44:09.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.934+0000 7ffb88e53700 1 -- 172.21.15.7:0/3572761079 shutdown_connections 2024-01-27T02:44:09.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.934+0000 7ffb88e53700 1 --2- 172.21.15.7:0/3572761079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb840cdbf0 0x7ffb840ca4b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:09.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.934+0000 7ffb88e53700 1 --2- 172.21.15.7:0/3572761079 >> v2:172.21.15.188:3300/0 conn(0x7ffb84157ca0 0x7ffb840c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:09.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.934+0000 7ffb88e53700 1 -- 172.21.15.7:0/3572761079 >> 172.21.15.7:0/3572761079 conn(0x7ffb84098020 msgr2=0x7ffb840711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:09.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.934+0000 7ffb88e53700 1 -- 172.21.15.7:0/3572761079 shutdown_connections 2024-01-27T02:44:09.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.934+0000 7ffb88e53700 1 -- 172.21.15.7:0/3572761079 wait complete. 2024-01-27T02:44:09.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.935+0000 7ffb88e53700 1 Processor -- start 2024-01-27T02:44:09.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.935+0000 7ffb88e53700 1 -- start start 2024-01-27T02:44:09.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.936+0000 7ffb88e53700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb840c3ae0 0x7ffb84149c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:09.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.936+0000 7ffb88e53700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb840cdbf0 0x7ffb840c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:09.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.936+0000 7ffb88e53700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb84157ca0 0x7ffb840c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:09.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.936+0000 7ffb88e53700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb840b24a0 con 0x7ffb84157ca0 2024-01-27T02:44:09.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.936+0000 7ffb88e53700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb840b3500 con 0x7ffb840c3ae0 2024-01-27T02:44:09.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.936+0000 7ffb88e53700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb840b2740 con 0x7ffb840cdbf0 2024-01-27T02:44:09.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.936+0000 7ffb82d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb84157ca0 0x7ffb840c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:09.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.936+0000 7ffb82d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb84157ca0 0x7ffb840c6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48682/0 (socket says 172.21.15.7:48682) 2024-01-27T02:44:09.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.936+0000 7ffb82d9d700 1 -- 172.21.15.7:0/4106012396 learned_addr learned my addr 172.21.15.7:0/4106012396 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:09.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.936+0000 7ffb81d9b700 1 --2- 172.21.15.7:0/4106012396 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb840cdbf0 0x7ffb840c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:09.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.936+0000 7ffb8259c700 1 --2- 172.21.15.7:0/4106012396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb840c3ae0 0x7ffb84149c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:09.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.937+0000 7ffb82d9d700 1 -- 172.21.15.7:0/4106012396 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb840cdbf0 msgr2=0x7ffb840c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:09.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.937+0000 7ffb82d9d700 1 --2- 172.21.15.7:0/4106012396 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb840cdbf0 0x7ffb840c5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:09.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.937+0000 7ffb82d9d700 1 -- 172.21.15.7:0/4106012396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb840c3ae0 msgr2=0x7ffb84149c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:09.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.937+0000 7ffb82d9d700 1 --2- 172.21.15.7:0/4106012396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb840c3ae0 0x7ffb84149c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:09.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.937+0000 7ffb82d9d700 1 -- 172.21.15.7:0/4106012396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb74009420 con 0x7ffb84157ca0 2024-01-27T02:44:09.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.937+0000 7ffb82d9d700 1 --2- 172.21.15.7:0/4106012396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb84157ca0 0x7ffb840c6330 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c013000 tx=0x7ffb7c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:09.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.937+0000 7ffb7a7fc700 1 -- 172.21.15.7:0/4106012396 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c012e10 con 0x7ffb84157ca0 2024-01-27T02:44:09.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.937+0000 7ffb7a7fc700 1 -- 172.21.15.7:0/4106012396 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c027470 con 0x7ffb84157ca0 2024-01-27T02:44:09.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.937+0000 7ffb88e53700 1 -- 172.21.15.7:0/4106012396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb7c009a20 con 0x7ffb84157ca0 2024-01-27T02:44:09.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.937+0000 7ffb7a7fc700 1 -- 172.21.15.7:0/4106012396 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c027830 con 0x7ffb84157ca0 2024-01-27T02:44:09.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.937+0000 7ffb88e53700 1 -- 172.21.15.7:0/4106012396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb840b2d00 con 0x7ffb84157ca0 2024-01-27T02:44:09.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.938+0000 7ffb88e53700 1 -- 172.21.15.7:0/4106012396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8404c710 con 0x7ffb84157ca0 2024-01-27T02:44:09.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.939+0000 7ffb7a7fc700 1 -- 172.21.15.7:0/4106012396 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ffb7c024070 con 0x7ffb84157ca0 2024-01-27T02:44:09.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.939+0000 7ffb7a7fc700 1 --2- 172.21.15.7:0/4106012396 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb64066f30 0x7ffb640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:09.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.939+0000 7ffb7a7fc700 1 -- 172.21.15.7:0/4106012396 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ffb7c086b60 con 0x7ffb84157ca0 2024-01-27T02:44:09.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.939+0000 7ffb8259c700 1 --2- 172.21.15.7:0/4106012396 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb64066f30 0x7ffb640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:09.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.940+0000 7ffb8259c700 1 --2- 172.21.15.7:0/4106012396 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb64066f30 0x7ffb640693e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009ea0 tx=0x7ffb74012040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:09.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:09.944+0000 7ffb7a7fc700 1 -- 172.21.15.7:0/4106012396 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb7c0540d0 con 0x7ffb84157ca0 2024-01-27T02:44:10.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.148+0000 7ffb88e53700 1 -- 172.21.15.7:0/4106012396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb840c57d0 con 0x7ffb84157ca0 2024-01-27T02:44:10.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.149+0000 7ffb7a7fc700 1 -- 172.21.15.7:0/4106012396 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v187) v1 ==== 74+0+12228 (secure 0 0 0) 0x7ffb7c0576f0 con 0x7ffb84157ca0 2024-01-27T02:44:10.150 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:44:10.150 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":187,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:44:08.017381+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.188:6837/3853795510":"2024-01-28T02:44:04.843025+0000","172.21.15.188:6835/3853795510":"2024-01-28T02:44:04.843025+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:44:10.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.152+0000 7ffb88e53700 1 -- 172.21.15.7:0/4106012396 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb64066f30 msgr2=0x7ffb640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:10.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.152+0000 7ffb88e53700 1 --2- 172.21.15.7:0/4106012396 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb64066f30 0x7ffb640693e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009ea0 tx=0x7ffb74012040 comp rx=0 tx=0).stop 2024-01-27T02:44:10.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.152+0000 7ffb88e53700 1 -- 172.21.15.7:0/4106012396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb84157ca0 msgr2=0x7ffb840c6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:10.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.152+0000 7ffb88e53700 1 --2- 172.21.15.7:0/4106012396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb84157ca0 0x7ffb840c6330 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c013000 tx=0x7ffb7c019000 comp rx=0 tx=0).stop 2024-01-27T02:44:10.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.152+0000 7ffb88e53700 1 -- 172.21.15.7:0/4106012396 shutdown_connections 2024-01-27T02:44:10.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.152+0000 7ffb88e53700 1 --2- 172.21.15.7:0/4106012396 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb64066f30 0x7ffb640693e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.152+0000 7ffb88e53700 1 --2- 172.21.15.7:0/4106012396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb84157ca0 0x7ffb840c6330 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.152+0000 7ffb88e53700 1 --2- 172.21.15.7:0/4106012396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb840c3ae0 0x7ffb84149c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.152+0000 7ffb88e53700 1 --2- 172.21.15.7:0/4106012396 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb840cdbf0 0x7ffb840c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.152+0000 7ffb88e53700 1 -- 172.21.15.7:0/4106012396 >> 172.21.15.7:0/4106012396 conn(0x7ffb84098020 msgr2=0x7ffb8406c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:10.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.152+0000 7ffb88e53700 1 -- 172.21.15.7:0/4106012396 shutdown_connections 2024-01-27T02:44:10.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.152+0000 7ffb88e53700 1 -- 172.21.15.7:0/4106012396 wait complete. 2024-01-27T02:44:10.166 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-27T02:44:10.166 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:44:10.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.297+0000 7ffb98aab700 1 Processor -- start 2024-01-27T02:44:10.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.297+0000 7ffb98aab700 1 -- start start 2024-01-27T02:44:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.298+0000 7ffb98aab700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb940a7bd0 0x7ffb940af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.298+0000 7ffb98aab700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffb940b1aa0 0x7ffb9406e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.298+0000 7ffb98aab700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb9406ea30 con 0x7ffb940a7fa0 2024-01-27T02:44:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.298+0000 7ffb98aab700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb9406fe40 con 0x7ffb940a7bd0 2024-01-27T02:44:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.298+0000 7ffb98aab700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb9406f870 con 0x7ffb940b1aa0 2024-01-27T02:44:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.298+0000 7ffb9259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb940a7bd0 0x7ffb940af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.298+0000 7ffb9259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb940a7bd0 0x7ffb940af6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48696/0 (socket says 172.21.15.7:48696) 2024-01-27T02:44:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.298+0000 7ffb9259c700 1 -- 172.21.15.7:0/2727538443 learned_addr learned my addr 172.21.15.7:0/2727538443 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.298+0000 7ffb92d9d700 1 --2- 172.21.15.7:0/2727538443 >> v2:172.21.15.188:3300/0 conn(0x7ffb940b1aa0 0x7ffb9406e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.299+0000 7ffb9259c700 1 -- 172.21.15.7:0/2727538443 >> v2:172.21.15.188:3300/0 conn(0x7ffb940b1aa0 msgr2=0x7ffb9406e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.299+0000 7ffb9259c700 1 --2- 172.21.15.7:0/2727538443 >> v2:172.21.15.188:3300/0 conn(0x7ffb940b1aa0 0x7ffb9406e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.299+0000 7ffb9259c700 1 -- 172.21.15.7:0/2727538443 >> v1:172.21.15.188:6789/0 conn(0x7ffb940a7fa0 legacy=0x7ffb940b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.299+0000 7ffb9259c700 1 -- 172.21.15.7:0/2727538443 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9406ec60 con 0x7ffb940a7bd0 2024-01-27T02:44:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.299+0000 7ffb9259c700 1 --2- 172.21.15.7:0/2727538443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb940a7bd0 0x7ffb940af6d0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7ffb94071240 tx=0x7ffb8400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93a8db1c1727ac4 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.299+0000 7ffb9159a700 1 -- 172.21.15.7:0/2727538443 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8400dba0 con 0x7ffb940a7bd0 2024-01-27T02:44:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.299+0000 7ffb9159a700 1 -- 172.21.15.7:0/2727538443 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8400dd00 con 0x7ffb940a7bd0 2024-01-27T02:44:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.299+0000 7ffb9159a700 1 -- 172.21.15.7:0/2727538443 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb840196d0 con 0x7ffb940a7bd0 2024-01-27T02:44:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.300+0000 7ffb98aab700 1 -- 172.21.15.7:0/2727538443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb940a7bd0 msgr2=0x7ffb940af6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.300+0000 7ffb98aab700 1 --2- 172.21.15.7:0/2727538443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb940a7bd0 0x7ffb940af6d0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7ffb94071240 tx=0x7ffb8400b4f0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.300+0000 7ffb98aab700 1 -- 172.21.15.7:0/2727538443 shutdown_connections 2024-01-27T02:44:10.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.300+0000 7ffb98aab700 1 --2- 172.21.15.7:0/2727538443 >> v2:172.21.15.188:3300/0 conn(0x7ffb940b1aa0 0x7ffb9406e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.300+0000 7ffb98aab700 1 --2- 172.21.15.7:0/2727538443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb940a7bd0 0x7ffb940af6d0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.300+0000 7ffb98aab700 1 -- 172.21.15.7:0/2727538443 >> 172.21.15.7:0/2727538443 conn(0x7ffb940679b0 msgr2=0x7ffb94067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:10.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.300+0000 7ffb98aab700 1 -- 172.21.15.7:0/2727538443 shutdown_connections 2024-01-27T02:44:10.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.300+0000 7ffb98aab700 1 -- 172.21.15.7:0/2727538443 wait complete. 2024-01-27T02:44:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.301+0000 7ffb98aab700 1 Processor -- start 2024-01-27T02:44:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.301+0000 7ffb98aab700 1 -- start start 2024-01-27T02:44:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.302+0000 7ffb98aab700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb940a7bd0 0x7ffb941533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.302+0000 7ffb98aab700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb940a7fa0 0x7ffb94159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.302+0000 7ffb98aab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb940b1aa0 0x7ffb940995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.302+0000 7ffb98aab700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb940a98c0 con 0x7ffb940b1aa0 2024-01-27T02:44:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.302+0000 7ffb98aab700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb940a9b10 con 0x7ffb940a7bd0 2024-01-27T02:44:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.302+0000 7ffb98aab700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb940a9df0 con 0x7ffb940a7fa0 2024-01-27T02:44:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.302+0000 7ffb92d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb940b1aa0 0x7ffb940995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.302+0000 7ffb92d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb940b1aa0 0x7ffb940995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48710/0 (socket says 172.21.15.7:48710) 2024-01-27T02:44:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.302+0000 7ffb92d9d700 1 -- 172.21.15.7:0/953224537 learned_addr learned my addr 172.21.15.7:0/953224537 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.302+0000 7ffb91d9b700 1 --2- 172.21.15.7:0/953224537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb940a7fa0 0x7ffb94159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.302+0000 7ffb9259c700 1 --2- 172.21.15.7:0/953224537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb940a7bd0 0x7ffb941533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.302+0000 7ffb92d9d700 1 -- 172.21.15.7:0/953224537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb940a7fa0 msgr2=0x7ffb94159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.302+0000 7ffb92d9d700 1 --2- 172.21.15.7:0/953224537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb940a7fa0 0x7ffb94159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.302+0000 7ffb92d9d700 1 -- 172.21.15.7:0/953224537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb940a7bd0 msgr2=0x7ffb941533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.302+0000 7ffb92d9d700 1 --2- 172.21.15.7:0/953224537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb940a7bd0 0x7ffb941533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.302+0000 7ffb92d9d700 1 -- 172.21.15.7:0/953224537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8400b790 con 0x7ffb940b1aa0 2024-01-27T02:44:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.302+0000 7ffb91d9b700 1 --2- 172.21.15.7:0/953224537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb940a7fa0 0x7ffb94159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:44:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.302+0000 7ffb92d9d700 1 --2- 172.21.15.7:0/953224537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb940b1aa0 0x7ffb940995a0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009d70 tx=0x7ffb8c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.303+0000 7ffb8a7fc700 1 -- 172.21.15.7:0/953224537 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c00cb50 con 0x7ffb940b1aa0 2024-01-27T02:44:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.303+0000 7ffb8a7fc700 1 -- 172.21.15.7:0/953224537 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8c00f040 con 0x7ffb940b1aa0 2024-01-27T02:44:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.303+0000 7ffb8a7fc700 1 -- 172.21.15.7:0/953224537 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c014480 con 0x7ffb940b1aa0 2024-01-27T02:44:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.303+0000 7ffb98aab700 1 -- 172.21.15.7:0/953224537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb940aa0d0 con 0x7ffb940b1aa0 2024-01-27T02:44:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.303+0000 7ffb98aab700 1 -- 172.21.15.7:0/953224537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb940aa690 con 0x7ffb940b1aa0 2024-01-27T02:44:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.304+0000 7ffb98aab700 1 -- 172.21.15.7:0/953224537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb9404c710 con 0x7ffb940b1aa0 2024-01-27T02:44:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.304+0000 7ffb8a7fc700 1 -- 172.21.15.7:0/953224537 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ffb8c014830 con 0x7ffb940b1aa0 2024-01-27T02:44:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.304+0000 7ffb8a7fc700 1 --2- 172.21.15.7:0/953224537 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb74066f30 0x7ffb740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.305+0000 7ffb8a7fc700 1 -- 172.21.15.7:0/953224537 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ffb8c084f20 con 0x7ffb940b1aa0 2024-01-27T02:44:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.305+0000 7ffb9259c700 1 --2- 172.21.15.7:0/953224537 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb74066f30 0x7ffb740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.306+0000 7ffb9259c700 1 --2- 172.21.15.7:0/953224537 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb74066f30 0x7ffb740693e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ffb94071240 tx=0x7ffb8401b000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.309+0000 7ffb8a7fc700 1 -- 172.21.15.7:0/953224537 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb8c052490 con 0x7ffb940b1aa0 2024-01-27T02:44:10.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.507+0000 7ffb98aab700 1 -- 172.21.15.7:0/953224537 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffb9406ef40 con 0x7ffb74066f30 2024-01-27T02:44:10.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.509+0000 7ffb8a7fc700 1 -- 172.21.15.7:0/953224537 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50301 (secure 0 0 0) 0x7ffb9406ef40 con 0x7ffb74066f30 2024-01-27T02:44:10.510 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:44:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.511+0000 7ffb98aab700 1 -- 172.21.15.7:0/953224537 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb74066f30 msgr2=0x7ffb740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.511+0000 7ffb98aab700 1 --2- 172.21.15.7:0/953224537 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb74066f30 0x7ffb740693e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ffb94071240 tx=0x7ffb8401b000 comp rx=0 tx=0).stop 2024-01-27T02:44:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.511+0000 7ffb98aab700 1 -- 172.21.15.7:0/953224537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb940b1aa0 msgr2=0x7ffb940995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.511+0000 7ffb98aab700 1 --2- 172.21.15.7:0/953224537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb940b1aa0 0x7ffb940995a0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009d70 tx=0x7ffb8c00b040 comp rx=0 tx=0).stop 2024-01-27T02:44:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.511+0000 7ffb98aab700 1 -- 172.21.15.7:0/953224537 shutdown_connections 2024-01-27T02:44:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.511+0000 7ffb98aab700 1 --2- 172.21.15.7:0/953224537 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7ffb74066f30 0x7ffb740693e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.511+0000 7ffb98aab700 1 --2- 172.21.15.7:0/953224537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb940b1aa0 0x7ffb940995a0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.511+0000 7ffb98aab700 1 --2- 172.21.15.7:0/953224537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb940a7bd0 0x7ffb941533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.511+0000 7ffb98aab700 1 --2- 172.21.15.7:0/953224537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb940a7fa0 0x7ffb94159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.511+0000 7ffb98aab700 1 -- 172.21.15.7:0/953224537 >> 172.21.15.7:0/953224537 conn(0x7ffb940679b0 msgr2=0x7ffb941540a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:10.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.511+0000 7ffb98aab700 1 -- 172.21.15.7:0/953224537 shutdown_connections 2024-01-27T02:44:10.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.511+0000 7ffb98aab700 1 -- 172.21.15.7:0/953224537 wait complete. 2024-01-27T02:44:10.513 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:44:10.525 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":544,"stamp":"2024-01-27T02:44:09.751800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":111774,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1450725,"statfs":{"total":1600000000,"available":1485542802,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52489243,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-120,"num_write_kb":-85081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.905843"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":307,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-01-27T02:44:04.852416+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:44:04.852416+0000","last_peered":"2024-01-27T02:44:04.852416+0000","last_clean":"2024-01-27T02:44:04.852416+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:44:04.852416+0000","last_undegraded":"2024-01-27T02:44:04.852416+0000","last_fullsized":"2024-01-27T02:44:04.852416+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574996,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9840,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185472,"statfs":{"total":200000000,"available":189923592,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.32800000000000001},"max":{"1min":0.80600000000000005,"5min":0.83499999999999996,"15min":0.85499999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59899999999999998,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.30099999999999999},"max":{"1min":0.84199999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.59699999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.47699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74399999999999999,"5min":0.78800000000000003,"15min":0.82399999999999995},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.57399999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.34999999999999998},"max":{"1min":0.81599999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.49099999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.38400000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.86399999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.75,"5min":0.80300000000000005,"15min":0.83899999999999997},"last":0.55500000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76500000000000001,"5min":0.78900000000000003,"15min":0.90300000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.63800000000000001,"15min":0.61199999999999999},"min":{"1min":0.47599999999999998,"5min":0.41799999999999998,"15min":0.40999999999999998},"max":{"1min":0.77400000000000002,"5min":0.81599999999999995,"15min":0.88600000000000001},"last":0.63900000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48199999999999998,"15min":0.48399999999999999},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67300000000000004,"5min":0.73899999999999999,"15min":0.78600000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52000000000000002,"15min":0.51000000000000001},"min":{"1min":0.36699999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.80000000000000004},"last":0.625}]},{"osd":5,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.39600000000000002,"5min":0.32400000000000001,"15min":0.307},"max":{"1min":0.746,"5min":0.79500000000000004,"15min":0.85399999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.42499999999999999,"5min":0.33900000000000002,"15min":0.33500000000000002},"max":{"1min":0.76600000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.53800000000000003}]},{"osd":6,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60199999999999998,"15min":0.59299999999999997},"min":{"1min":0.38900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.78900000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71799999999999997,"5min":0.83499999999999996,"15min":0.90800000000000003},"last":0.57899999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574996,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":19445,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":175867,"statfs":{"total":200000000,"available":180088161,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60099999999999998,"15min":0.57899999999999996},"min":{"1min":0.38800000000000001,"5min":0.375,"15min":0.29999999999999999},"max":{"1min":0.78700000000000003,"5min":1.0409999999999999,"15min":4.125},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58799999999999997,"15min":0.56499999999999995},"min":{"1min":0.435,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.753,"5min":0.96599999999999997,"15min":1.216},"last":0.45000000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42599999999999999,"5min":0.376,"15min":0.34100000000000003},"max":{"1min":0.67800000000000005,"5min":0.95499999999999996,"15min":8.2699999999999996},"last":0.623},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59699999999999998,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78000000000000003,"5min":1.0129999999999999,"15min":26.411999999999999},"last":0.42099999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.627},"min":{"1min":0.51900000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73699999999999999,"5min":0.78300000000000003,"15min":17.143999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61499999999999999,"15min":0.60899999999999999},"min":{"1min":0.503,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73199999999999998,"5min":0.85799999999999998,"15min":1.76},"last":0.55800000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.47699999999999998,"5min":0.39400000000000002,"15min":0.376},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":4.3689999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.41199999999999998},"max":{"1min":0.77400000000000002,"5min":0.81699999999999995,"15min":2.1970000000000001},"last":0.54000000000000004}]},{"osd":4,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47099999999999997,"15min":0.48099999999999998},"min":{"1min":0.35299999999999998,"5min":0.311,"15min":0.30199999999999999},"max":{"1min":0.61499999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78900000000000003,"5min":0.84899999999999998,"15min":0.89600000000000002},"last":0.46800000000000003}]},{"osd":5,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.39400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.65400000000000003,"5min":0.80800000000000005,"15min":0.81299999999999994},"last":0.502},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55200000000000005,"15min":0.57699999999999996},"min":{"1min":0.38300000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76400000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.58299999999999996}]},{"osd":7,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57799999999999996,"15min":0.59599999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.35999999999999999},"max":{"1min":0.73599999999999999,"5min":0.88700000000000001,"15min":1.0009999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54900000000000004,"15min":0.56799999999999995},"min":{"1min":0.40699999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.749,"5min":0.82899999999999996,"15min":1.038},"last":0.59399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574996,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":13300,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":182011,"statfs":{"total":200000000,"available":186379807,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40300000000000002,"15min":0.38900000000000001},"min":{"1min":0.26900000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.51800000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.42999999999999999,"15min":0.41299999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.27700000000000002},"max":{"1min":0.51400000000000001,"5min":0.66100000000000003,"15min":1.121},"last":0.34200000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42199999999999999,"15min":0.39300000000000002},"min":{"1min":0.30599999999999999,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":0.56000000000000005,"5min":1.0940000000000001,"15min":1.25},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.45200000000000001,"15min":0.442},"min":{"1min":0.30099999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.65800000000000003,"5min":0.66700000000000004,"15min":0.82999999999999996},"last":0.46400000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.495,"15min":0.48199999999999998},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.32100000000000001},"max":{"1min":0.61299999999999999,"5min":0.79900000000000004,"15min":0.90900000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47399999999999998,"15min":0.46400000000000002},"min":{"1min":0.35999999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.59199999999999997,"5min":0.72599999999999998,"15min":0.98999999999999999},"last":0.38600000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66000000000000003,"15min":0.64800000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.373},"max":{"1min":0.83999999999999997,"5min":1.075,"15min":3.2890000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67800000000000005,"15min":0.65500000000000003},"min":{"1min":0.48999999999999999,"5min":0.45000000000000001,"15min":0.44500000000000001},"max":{"1min":0.95499999999999996,"5min":0.98099999999999998,"15min":1.026},"last":0.53300000000000003}]},{"osd":5,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68500000000000005,"15min":0.66200000000000003},"min":{"1min":0.46700000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89900000000000002,"5min":0.94599999999999995,"15min":3.4500000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68600000000000005,"15min":0.68200000000000005},"min":{"1min":0.45600000000000002,"5min":0.44,"15min":0.42399999999999999},"max":{"1min":0.93000000000000005,"5min":1.004,"15min":3.181},"last":0.58299999999999996}]},{"osd":6,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.71599999999999997,"15min":0.70599999999999996},"min":{"1min":0.52500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91400000000000003,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.74099999999999999,"15min":0.70599999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.50800000000000001},"max":{"1min":0.93899999999999995,"5min":1.143,"15min":1.869},"last":0.61699999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73799999999999999,"15min":0.71099999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85399999999999998,"5min":1.181,"15min":1.6870000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72799999999999998,"15min":0.71099999999999997},"min":{"1min":0.56299999999999994,"5min":0.501,"15min":0.45300000000000001},"max":{"1min":0.92400000000000004,"5min":1.0429999999999999,"15min":1.708},"last":0.63600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574996,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9781,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185531,"statfs":{"total":200000000,"available":189983777,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39500000000000002,"15min":0.39300000000000002},"min":{"1min":0.28499999999999998,"5min":0.24399999999999999,"15min":0.23799999999999999},"max":{"1min":0.55000000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.378},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42499999999999999,"15min":0.41999999999999998},"min":{"1min":0.32400000000000001,"5min":0.26000000000000001,"15min":0.223},"max":{"1min":0.55800000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.34200000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42499999999999999},"min":{"1min":0.34999999999999998,"5min":0.28599999999999998,"15min":0.28199999999999997},"max":{"1min":0.60799999999999998,"5min":0.625,"15min":0.64000000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45500000000000002,"15min":0.44400000000000001},"min":{"1min":0.34200000000000003,"5min":0.29499999999999998,"15min":0.28799999999999998},"max":{"1min":0.626,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.41399999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.44600000000000001,"15min":0.436},"min":{"1min":0.29999999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58499999999999996,"5min":0.66600000000000004,"15min":0.67300000000000004},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.39000000000000001,"5min":0.34300000000000003,"15min":0.29099999999999998},"max":{"1min":0.69799999999999995,"5min":0.71399999999999997,"15min":0.71699999999999997},"last":0.42599999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.65400000000000003,"15min":0.64900000000000002},"min":{"1min":0.46600000000000003,"5min":0.432,"15min":0.42099999999999999},"max":{"1min":0.84899999999999998,"5min":0.97599999999999998,"15min":0.997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63700000000000001,"15min":0.65000000000000002},"min":{"1min":0.46400000000000002,"5min":0.42199999999999999,"15min":0.41999999999999998},"max":{"1min":0.85999999999999999,"5min":0.87,"15min":0.96699999999999997},"last":0.61699999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.54100000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":2.4860000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.44},"max":{"1min":0.90200000000000002,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.78100000000000003}]},{"osd":6,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.51300000000000001,"5min":0.45600000000000002,"15min":0.42499999999999999},"max":{"1min":0.92600000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68899999999999995,"15min":0.68600000000000005},"min":{"1min":0.53600000000000003,"5min":0.46500000000000002,"15min":0.45000000000000001},"max":{"1min":0.96699999999999997,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.66400000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.51400000000000001,"5min":0.47099999999999997,"15min":0.42599999999999999},"max":{"1min":0.94799999999999995,"5min":0.94999999999999996,"15min":1.014},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68899999999999995,"15min":0.68999999999999995},"min":{"1min":0.55800000000000005,"5min":0.499,"15min":0.498},"max":{"1min":0.90900000000000003,"5min":0.92500000000000004,"15min":0.97299999999999998},"last":0.67800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574996,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21493,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173819,"statfs":{"total":200000000,"available":177991143,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41099999999999998},"min":{"1min":0.27000000000000002,"5min":0.23300000000000001,"15min":0.23000000000000001},"max":{"1min":0.63200000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.46000000000000002,"15min":0.53900000000000003},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.30499999999999999},"max":{"1min":0.57999999999999996,"5min":0.68899999999999995,"15min":6.2729999999999997},"last":0.33200000000000002}]},{"osd":1,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.41199999999999998,"15min":0.42099999999999999},"min":{"1min":0.33000000000000002,"5min":0.28799999999999998,"15min":0.23799999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.81399999999999995},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.442,"5min":0.443,"15min":0.442},"min":{"1min":0.29999999999999999,"5min":0.28999999999999998,"15min":0.28899999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.70099999999999996},"last":0.39300000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.31,"5min":0.28499999999999998,"15min":0.27800000000000002},"max":{"1min":0.55800000000000005,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48399999999999999,"15min":0.48499999999999999},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.70399999999999996,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.438}]},{"osd":4,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68100000000000005,"15min":0.67000000000000004},"min":{"1min":0.54600000000000004,"5min":0.47599999999999998,"15min":0.39300000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.98299999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.746},"min":{"1min":0.56499999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94399999999999995,"5min":0.95799999999999996,"15min":3.581},"last":0.61499999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65800000000000003,"15min":0.69599999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.43099999999999999},"max":{"1min":0.90400000000000003,"5min":0.92200000000000004,"15min":3.5369999999999999},"last":0.499},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66400000000000003,"15min":0.66800000000000004},"min":{"1min":0.5,"5min":0.44800000000000001,"15min":0.41099999999999998},"max":{"1min":0.79900000000000004,"5min":0.89100000000000001,"15min":1.0529999999999999},"last":0.48299999999999998}]},{"osd":6,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.55300000000000005,"5min":0.46999999999999997,"15min":0.42299999999999999},"max":{"1min":1.0900000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70099999999999996,"15min":0.68899999999999995},"min":{"1min":0.48299999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0469999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.68100000000000005}]},{"osd":7,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69599999999999995,"15min":0.68500000000000005},"min":{"1min":0.46999999999999997,"5min":0.437,"15min":0.437},"max":{"1min":1.0600000000000001,"5min":1.55,"15min":1.55},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71999999999999997,"15min":0.70099999999999996},"min":{"1min":0.54300000000000004,"5min":0.51300000000000001,"15min":0.42699999999999999},"max":{"1min":0.96999999999999997,"5min":1.615,"15min":1.615},"last":0.58699999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574996,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9205,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186107,"statfs":{"total":200000000,"available":190573909,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.253},"max":{"1min":0.56299999999999994,"5min":0.67900000000000005,"15min":4.7380000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.42099999999999999,"15min":0.40899999999999997},"min":{"1min":0.30499999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":1.0189999999999999},"last":0.40000000000000002}]},{"osd":1,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40500000000000003,"15min":0.41999999999999998},"min":{"1min":0.27600000000000002,"5min":0.24399999999999999,"15min":0.22900000000000001},"max":{"1min":0.54300000000000004,"5min":0.64400000000000002,"15min":4.5739999999999998},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46300000000000002,"15min":0.46500000000000002},"min":{"1min":0.35499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.751,"5min":0.98199999999999998,"15min":5.0529999999999999},"last":0.54100000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.46800000000000003,"15min":0.45600000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.28100000000000003},"max":{"1min":0.624,"5min":0.76700000000000002,"15min":1.5980000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.48899999999999999,"15min":0.49299999999999999},"min":{"1min":0.42599999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.65700000000000003,"5min":0.73499999999999999,"15min":5.1879999999999997},"last":0.46600000000000003}]},{"osd":4,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63500000000000001,"15min":0.64500000000000002},"min":{"1min":0.44,"5min":0.42999999999999999,"15min":0.41699999999999998},"max":{"1min":0.877,"5min":0.91600000000000004,"15min":4.6660000000000004},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65400000000000003,"15min":0.63600000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84199999999999997,"5min":0.94799999999999995,"15min":1.3120000000000001},"last":0.69899999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.49199999999999999,"5min":0.438,"15min":0.40799999999999997},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.409},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66600000000000004,"15min":0.70499999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81499999999999995,"5min":0.96599999999999997,"15min":4.7960000000000003},"last":0.86599999999999999}]},{"osd":6,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67200000000000004,"15min":0.68500000000000005},"min":{"1min":0.50900000000000001,"5min":0.42799999999999999,"15min":0.39800000000000002},"max":{"1min":0.86099999999999999,"5min":0.93500000000000005,"15min":5.0069999999999997},"last":0.751},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69299999999999995,"15min":0.67200000000000004},"min":{"1min":0.53000000000000003,"5min":0.47499999999999998,"15min":0.435},"max":{"1min":0.83499999999999996,"5min":0.94199999999999995,"15min":1.103},"last":0.66200000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46000000000000002},"max":{"1min":0.83399999999999996,"5min":0.92100000000000004,"15min":4.7130000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.68899999999999995},"min":{"1min":0.48599999999999999,"5min":0.47299999999999998,"15min":0.435},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.361},"last":0.77200000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574996,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":7215,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":188097,"statfs":{"total":200000000,"available":192611756,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.43099999999999999,"5min":0.35599999999999998,"15min":0.314},"max":{"1min":0.76800000000000002,"5min":1.137,"15min":1.3839999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60899999999999999,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":1.016,"15min":7.0670000000000002},"last":0.622}]},{"osd":1,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.35399999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":5.1529999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.48499999999999999,"5min":0.43099999999999999,"15min":0.39000000000000001},"max":{"1min":0.80100000000000005,"5min":1.054,"15min":3.0419999999999998},"last":0.60999999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60399999999999998,"15min":0.61499999999999999},"min":{"1min":0.46400000000000002,"5min":0.41199999999999998,"15min":0.39400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":5.0019999999999998},"last":0.749},{"interface":"front","average":{"1min":0.622,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.49399999999999999,"5min":0.41899999999999998,"15min":0.40500000000000003},"max":{"1min":0.86099999999999999,"5min":1.2549999999999999,"15min":2.073},"last":0.52300000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.496,"5min":0.439,"15min":0.36199999999999999},"max":{"1min":0.78800000000000003,"5min":1.1899999999999999,"15min":6.7220000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.629,"15min":0.65700000000000003},"min":{"1min":0.442,"5min":0.42499999999999999,"15min":0.41399999999999998},"max":{"1min":0.872,"5min":0.88,"15min":6.5510000000000002},"last":0.71399999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.36199999999999999,"5min":0.314,"15min":0.312},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":1.3149999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.5,"15min":0.51800000000000002},"min":{"1min":0.41399999999999998,"5min":0.32500000000000001,"15min":0.29999999999999999},"max":{"1min":0.77500000000000002,"5min":0.79800000000000004,"15min":6.9269999999999996},"last":0.505}]},{"osd":6,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.432,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":5.2229999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55600000000000005,"15min":0.55100000000000005},"min":{"1min":0.437,"5min":0.35799999999999998,"15min":0.33200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":1.46},"last":0.69199999999999995}]},{"osd":7,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55200000000000005,"15min":0.57099999999999995},"min":{"1min":0.41399999999999998,"5min":0.36599999999999999,"15min":0.33600000000000002},"max":{"1min":0.80400000000000005,"5min":0.83499999999999996,"15min":7.5339999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58099999999999996,"15min":0.59399999999999997},"min":{"1min":0.47099999999999997,"5min":0.34699999999999998,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.85299999999999998,"15min":4.9550000000000001},"last":0.67300000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574996,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21493,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173819,"statfs":{"total":200000000,"available":177990657,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.44700000000000001,"5min":0.33900000000000002,"15min":0.28899999999999998},"max":{"1min":0.76300000000000001,"5min":1.071,"15min":1.071},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.47999999999999998,"5min":0.38500000000000001,"15min":0.27600000000000002},"max":{"1min":0.71099999999999997,"5min":0.83599999999999997,"15min":3.242},"last":0.502}]},{"osd":1,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.627},"min":{"1min":0.53500000000000003,"5min":0.38100000000000001,"15min":0.32800000000000001},"max":{"1min":0.80800000000000005,"5min":1.018,"15min":1.3380000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.45600000000000002,"5min":0.35399999999999998,"15min":0.34200000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.95499999999999996},"last":0.54200000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":0.96699999999999997,"15min":3.04},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60799999999999998,"15min":0.59099999999999997},"min":{"1min":0.52000000000000002,"5min":0.43099999999999999,"15min":0.36899999999999999},"max":{"1min":0.71299999999999997,"5min":0.81000000000000005,"15min":0.89400000000000002},"last":0.45400000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63,"15min":0.65500000000000003},"min":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.39800000000000002},"max":{"1min":0.73799999999999999,"5min":1.0409999999999999,"15min":4.032},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66900000000000004,"15min":0.68000000000000005},"min":{"1min":0.57199999999999995,"5min":0.48799999999999999,"15min":0.45600000000000002},"max":{"1min":0.753,"5min":0.879,"15min":4.282},"last":0.55200000000000005}]},{"osd":4,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.51500000000000001},"min":{"1min":0.376,"5min":0.33300000000000002,"15min":0.26500000000000001},"max":{"1min":0.76400000000000001,"5min":0.95399999999999996,"15min":1.2869999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.496,"15min":0.52900000000000003},"min":{"1min":0.35799999999999998,"5min":0.32800000000000001,"15min":0.30399999999999999},"max":{"1min":0.67900000000000005,"5min":0.81699999999999995,"15min":1.101},"last":0.433}]},{"osd":6,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.50600000000000001,"15min":0.52900000000000003},"min":{"1min":0.35299999999999998,"5min":0.32700000000000001,"15min":0.29099999999999998},"max":{"1min":0.65700000000000003,"5min":0.86599999999999999,"15min":1.7969999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.55200000000000005,"15min":0.56799999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.33600000000000002},"max":{"1min":0.69299999999999995,"5min":0.91800000000000004,"15min":1.6739999999999999},"last":0.38400000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.55100000000000005,"15min":0.55900000000000005},"min":{"1min":0.39500000000000002,"5min":0.36899999999999999,"15min":0.32700000000000001},"max":{"1min":0.76300000000000001,"5min":0.90600000000000003,"15min":1.008},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59199999999999997,"15min":0.627},"min":{"1min":0.46000000000000002,"5min":0.40600000000000003,"15min":0.33700000000000002},"max":{"1min":0.67500000000000004,"5min":0.91300000000000003,"15min":2},"last":0.46400000000000002}]}]}],"pool_statfs":[]}} 2024-01-27T02:44:10.526 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:44:10.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.658+0000 7f3c69977700 1 Processor -- start 2024-01-27T02:44:10.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.658+0000 7f3c69977700 1 -- start start 2024-01-27T02:44:10.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.659+0000 7f3c69977700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3c640d09f0 0x7f3c6415de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:10.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.659+0000 7f3c69977700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c64143c30 0x7f3c6415e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:10.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.659+0000 7f3c69977700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c640b3d90 con 0x7f3c64143860 2024-01-27T02:44:10.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.659+0000 7f3c69977700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c640d0ea0 con 0x7f3c640d09f0 2024-01-27T02:44:10.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.659+0000 7f3c69977700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c640d1180 con 0x7f3c64143c30 2024-01-27T02:44:10.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.659+0000 7f3c637fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c64143c30 0x7f3c6415e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:10.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.659+0000 7f3c627fc700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3c640d09f0 0x7f3c6415de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:10.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.659+0000 7f3c637fe700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c64143c30 0x7f3c6415e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.7:38090/0 (socket says 172.21.15.7:38090) 2024-01-27T02:44:10.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.659+0000 7f3c637fe700 1 -- 172.21.15.7:0/3111696089 learned_addr learned my addr 172.21.15.7:0/3111696089 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:10.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.659+0000 7f3c62ffd700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f3c64143860 0x7f3c640d02e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:58300/0 (socket says 172.21.15.7:58300) 2024-01-27T02:44:10.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.660+0000 7f3c637fe700 1 -- 172.21.15.7:0/3111696089 >> v2:172.21.15.188:3300/0 conn(0x7f3c640d09f0 msgr2=0x7f3c6415de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:10.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.660+0000 7f3c637fe700 1 --2- 172.21.15.7:0/3111696089 >> v2:172.21.15.188:3300/0 conn(0x7f3c640d09f0 0x7f3c6415de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.660+0000 7f3c637fe700 1 -- 172.21.15.7:0/3111696089 >> v1:172.21.15.188:6789/0 conn(0x7f3c64143860 legacy=0x7f3c640d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:10.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.660+0000 7f3c637fe700 1 -- 172.21.15.7:0/3111696089 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c640d1460 con 0x7f3c64143c30 2024-01-27T02:44:10.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.660+0000 7f3c637fe700 1 --2- 172.21.15.7:0/3111696089 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c64143c30 0x7f3c6415e340 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009f20 tx=0x7f3c5c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3d2565ffd03d393 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:10.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.660+0000 7f3c61ffb700 1 -- 172.21.15.7:0/3111696089 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c00e070 con 0x7f3c64143c30 2024-01-27T02:44:10.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.660+0000 7f3c61ffb700 1 -- 172.21.15.7:0/3111696089 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5c00b9d0 con 0x7f3c64143c30 2024-01-27T02:44:10.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.660+0000 7f3c61ffb700 1 -- 172.21.15.7:0/3111696089 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c0092e0 con 0x7f3c64143c30 2024-01-27T02:44:10.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.661+0000 7f3c69977700 1 -- 172.21.15.7:0/3111696089 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c64143c30 msgr2=0x7f3c6415e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:10.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.661+0000 7f3c69977700 1 --2- 172.21.15.7:0/3111696089 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c64143c30 0x7f3c6415e340 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009f20 tx=0x7f3c5c009840 comp rx=0 tx=0).stop 2024-01-27T02:44:10.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.661+0000 7f3c69977700 1 -- 172.21.15.7:0/3111696089 shutdown_connections 2024-01-27T02:44:10.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.661+0000 7f3c69977700 1 --2- 172.21.15.7:0/3111696089 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c64143c30 0x7f3c6415e340 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.661+0000 7f3c69977700 1 --2- 172.21.15.7:0/3111696089 >> v2:172.21.15.188:3300/0 conn(0x7f3c640d09f0 0x7f3c6415de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.661+0000 7f3c69977700 1 -- 172.21.15.7:0/3111696089 >> 172.21.15.7:0/3111696089 conn(0x7f3c64068440 msgr2=0x7f3c640709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:10.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.661+0000 7f3c69977700 1 -- 172.21.15.7:0/3111696089 shutdown_connections 2024-01-27T02:44:10.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.661+0000 7f3c69977700 1 -- 172.21.15.7:0/3111696089 wait complete. 2024-01-27T02:44:10.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.663+0000 7f3c69977700 1 Processor -- start 2024-01-27T02:44:10.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.663+0000 7f3c69977700 1 -- start start 2024-01-27T02:44:10.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.664+0000 7f3c69977700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c640d09f0 0x7f3c641412e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:10.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.664+0000 7f3c69977700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c64143860 0x7f3c640caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:10.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.664+0000 7f3c69977700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c64143c30 0x7f3c640c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:10.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.664+0000 7f3c69977700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c640b0550 con 0x7f3c64143c30 2024-01-27T02:44:10.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.664+0000 7f3c69977700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c640c7960 con 0x7f3c640d09f0 2024-01-27T02:44:10.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.664+0000 7f3c69977700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c640c6d50 con 0x7f3c64143860 2024-01-27T02:44:10.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.664+0000 7f3c637fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c64143c30 0x7f3c640c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:10.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.664+0000 7f3c637fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c64143c30 0x7f3c640c7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48716/0 (socket says 172.21.15.7:48716) 2024-01-27T02:44:10.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.664+0000 7f3c637fe700 1 -- 172.21.15.7:0/1871988330 learned_addr learned my addr 172.21.15.7:0/1871988330 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:10.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.664+0000 7f3c627fc700 1 --2- 172.21.15.7:0/1871988330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c64143860 0x7f3c640caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:10.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.664+0000 7f3c62ffd700 1 --2- 172.21.15.7:0/1871988330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c640d09f0 0x7f3c641412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:10.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.664+0000 7f3c637fe700 1 -- 172.21.15.7:0/1871988330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c64143860 msgr2=0x7f3c640caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:10.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.664+0000 7f3c637fe700 1 --2- 172.21.15.7:0/1871988330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c64143860 0x7f3c640caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.664+0000 7f3c637fe700 1 -- 172.21.15.7:0/1871988330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c640d09f0 msgr2=0x7f3c641412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:10.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.664+0000 7f3c637fe700 1 --2- 172.21.15.7:0/1871988330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c640d09f0 0x7f3c641412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.664+0000 7f3c637fe700 1 -- 172.21.15.7:0/1871988330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c009ae0 con 0x7f3c64143c30 2024-01-27T02:44:10.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.664+0000 7f3c637fe700 1 --2- 172.21.15.7:0/1871988330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c64143c30 0x7f3c640c7df0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c012000 tx=0x7f3c5c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:10.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.665+0000 7f3c5affd700 1 -- 172.21.15.7:0/1871988330 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c008b10 con 0x7f3c64143c30 2024-01-27T02:44:10.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.665+0000 7f3c5affd700 1 -- 172.21.15.7:0/1871988330 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5c008cb0 con 0x7f3c64143c30 2024-01-27T02:44:10.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.665+0000 7f3c69977700 1 -- 172.21.15.7:0/1871988330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c640c7030 con 0x7f3c64143c30 2024-01-27T02:44:10.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.665+0000 7f3c5affd700 1 -- 172.21.15.7:0/1871988330 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c01c660 con 0x7f3c64143c30 2024-01-27T02:44:10.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.665+0000 7f3c69977700 1 -- 172.21.15.7:0/1871988330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c640c75f0 con 0x7f3c64143c30 2024-01-27T02:44:10.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.666+0000 7f3c69977700 1 -- 172.21.15.7:0/1871988330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c6404c710 con 0x7f3c64143c30 2024-01-27T02:44:10.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.666+0000 7f3c5affd700 1 -- 172.21.15.7:0/1871988330 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3c5c003040 con 0x7f3c64143c30 2024-01-27T02:44:10.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.666+0000 7f3c5affd700 1 --2- 172.21.15.7:0/1871988330 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3c44066f30 0x7f3c440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:10.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.667+0000 7f3c5affd700 1 -- 172.21.15.7:0/1871988330 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f3c5c085990 con 0x7f3c64143c30 2024-01-27T02:44:10.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.667+0000 7f3c62ffd700 1 --2- 172.21.15.7:0/1871988330 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3c44066f30 0x7f3c440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:10.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.667+0000 7f3c62ffd700 1 --2- 172.21.15.7:0/1871988330 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3c44066f30 0x7f3c440693e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3c54004fd0 tx=0x7f3c54007eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:10.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.671+0000 7f3c5affd700 1 -- 172.21.15.7:0/1871988330 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c5c052f00 con 0x7f3c64143c30 2024-01-27T02:44:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.871+0000 7f3c69977700 1 -- 172.21.15.7:0/1871988330 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3c640d1740 con 0x7f3c44066f30 2024-01-27T02:44:10.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.874+0000 7f3c5affd700 1 -- 172.21.15.7:0/1871988330 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50301 (secure 0 0 0) 0x7f3c640d1740 con 0x7f3c44066f30 2024-01-27T02:44:10.875 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:44:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.876+0000 7f3c69977700 1 -- 172.21.15.7:0/1871988330 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3c44066f30 msgr2=0x7f3c440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.876+0000 7f3c69977700 1 --2- 172.21.15.7:0/1871988330 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3c44066f30 0x7f3c440693e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3c54004fd0 tx=0x7f3c54007eb0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.876+0000 7f3c69977700 1 -- 172.21.15.7:0/1871988330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c64143c30 msgr2=0x7f3c640c7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.876+0000 7f3c69977700 1 --2- 172.21.15.7:0/1871988330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c64143c30 0x7f3c640c7df0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c012000 tx=0x7f3c5c018000 comp rx=0 tx=0).stop 2024-01-27T02:44:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.876+0000 7f3c69977700 1 -- 172.21.15.7:0/1871988330 shutdown_connections 2024-01-27T02:44:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.876+0000 7f3c69977700 1 --2- 172.21.15.7:0/1871988330 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3c44066f30 0x7f3c440693e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.876+0000 7f3c69977700 1 --2- 172.21.15.7:0/1871988330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c64143c30 0x7f3c640c7df0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.876+0000 7f3c69977700 1 --2- 172.21.15.7:0/1871988330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c640d09f0 0x7f3c641412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.876+0000 7f3c69977700 1 --2- 172.21.15.7:0/1871988330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c64143860 0x7f3c640caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.876+0000 7f3c69977700 1 -- 172.21.15.7:0/1871988330 >> 172.21.15.7:0/1871988330 conn(0x7f3c64068440 msgr2=0x7f3c640b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.876+0000 7f3c69977700 1 -- 172.21.15.7:0/1871988330 shutdown_connections 2024-01-27T02:44:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:10.876+0000 7f3c69977700 1 -- 172.21.15.7:0/1871988330 wait complete. 2024-01-27T02:44:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:44:10.890 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":544,"stamp":"2024-01-27T02:44:09.751800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":111774,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1450725,"statfs":{"total":1600000000,"available":1485542802,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52489243,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-120,"num_write_kb":-85081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.905843"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":307,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-01-27T02:44:04.852416+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:44:04.852416+0000","last_peered":"2024-01-27T02:44:04.852416+0000","last_clean":"2024-01-27T02:44:04.852416+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:44:04.852416+0000","last_undegraded":"2024-01-27T02:44:04.852416+0000","last_fullsized":"2024-01-27T02:44:04.852416+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574996,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9840,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185472,"statfs":{"total":200000000,"available":189923592,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.32800000000000001},"max":{"1min":0.80600000000000005,"5min":0.83499999999999996,"15min":0.85499999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59899999999999998,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.30099999999999999},"max":{"1min":0.84199999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.59699999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.47699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74399999999999999,"5min":0.78800000000000003,"15min":0.82399999999999995},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.57399999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.34999999999999998},"max":{"1min":0.81599999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.49099999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.38400000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.86399999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.75,"5min":0.80300000000000005,"15min":0.83899999999999997},"last":0.55500000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76500000000000001,"5min":0.78900000000000003,"15min":0.90300000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.63800000000000001,"15min":0.61199999999999999},"min":{"1min":0.47599999999999998,"5min":0.41799999999999998,"15min":0.40999999999999998},"max":{"1min":0.77400000000000002,"5min":0.81599999999999995,"15min":0.88600000000000001},"last":0.63900000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48199999999999998,"15min":0.48399999999999999},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67300000000000004,"5min":0.73899999999999999,"15min":0.78600000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52000000000000002,"15min":0.51000000000000001},"min":{"1min":0.36699999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.80000000000000004},"last":0.625}]},{"osd":5,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.39600000000000002,"5min":0.32400000000000001,"15min":0.307},"max":{"1min":0.746,"5min":0.79500000000000004,"15min":0.85399999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.42499999999999999,"5min":0.33900000000000002,"15min":0.33500000000000002},"max":{"1min":0.76600000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.53800000000000003}]},{"osd":6,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60199999999999998,"15min":0.59299999999999997},"min":{"1min":0.38900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.78900000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71799999999999997,"5min":0.83499999999999996,"15min":0.90800000000000003},"last":0.57899999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574996,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":19445,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":175867,"statfs":{"total":200000000,"available":180088161,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60099999999999998,"15min":0.57899999999999996},"min":{"1min":0.38800000000000001,"5min":0.375,"15min":0.29999999999999999},"max":{"1min":0.78700000000000003,"5min":1.0409999999999999,"15min":4.125},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58799999999999997,"15min":0.56499999999999995},"min":{"1min":0.435,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.753,"5min":0.96599999999999997,"15min":1.216},"last":0.45000000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42599999999999999,"5min":0.376,"15min":0.34100000000000003},"max":{"1min":0.67800000000000005,"5min":0.95499999999999996,"15min":8.2699999999999996},"last":0.623},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59699999999999998,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78000000000000003,"5min":1.0129999999999999,"15min":26.411999999999999},"last":0.42099999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.627},"min":{"1min":0.51900000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73699999999999999,"5min":0.78300000000000003,"15min":17.143999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61499999999999999,"15min":0.60899999999999999},"min":{"1min":0.503,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73199999999999998,"5min":0.85799999999999998,"15min":1.76},"last":0.55800000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.47699999999999998,"5min":0.39400000000000002,"15min":0.376},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":4.3689999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.41199999999999998},"max":{"1min":0.77400000000000002,"5min":0.81699999999999995,"15min":2.1970000000000001},"last":0.54000000000000004}]},{"osd":4,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47099999999999997,"15min":0.48099999999999998},"min":{"1min":0.35299999999999998,"5min":0.311,"15min":0.30199999999999999},"max":{"1min":0.61499999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78900000000000003,"5min":0.84899999999999998,"15min":0.89600000000000002},"last":0.46800000000000003}]},{"osd":5,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.39400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.65400000000000003,"5min":0.80800000000000005,"15min":0.81299999999999994},"last":0.502},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55200000000000005,"15min":0.57699999999999996},"min":{"1min":0.38300000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76400000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.58299999999999996}]},{"osd":7,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57799999999999996,"15min":0.59599999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.35999999999999999},"max":{"1min":0.73599999999999999,"5min":0.88700000000000001,"15min":1.0009999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54900000000000004,"15min":0.56799999999999995},"min":{"1min":0.40699999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.749,"5min":0.82899999999999996,"15min":1.038},"last":0.59399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574996,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":13300,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":182011,"statfs":{"total":200000000,"available":186379807,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40300000000000002,"15min":0.38900000000000001},"min":{"1min":0.26900000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.51800000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.42999999999999999,"15min":0.41299999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.27700000000000002},"max":{"1min":0.51400000000000001,"5min":0.66100000000000003,"15min":1.121},"last":0.34200000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42199999999999999,"15min":0.39300000000000002},"min":{"1min":0.30599999999999999,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":0.56000000000000005,"5min":1.0940000000000001,"15min":1.25},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.45200000000000001,"15min":0.442},"min":{"1min":0.30099999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.65800000000000003,"5min":0.66700000000000004,"15min":0.82999999999999996},"last":0.46400000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.495,"15min":0.48199999999999998},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.32100000000000001},"max":{"1min":0.61299999999999999,"5min":0.79900000000000004,"15min":0.90900000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47399999999999998,"15min":0.46400000000000002},"min":{"1min":0.35999999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.59199999999999997,"5min":0.72599999999999998,"15min":0.98999999999999999},"last":0.38600000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66000000000000003,"15min":0.64800000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.373},"max":{"1min":0.83999999999999997,"5min":1.075,"15min":3.2890000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67800000000000005,"15min":0.65500000000000003},"min":{"1min":0.48999999999999999,"5min":0.45000000000000001,"15min":0.44500000000000001},"max":{"1min":0.95499999999999996,"5min":0.98099999999999998,"15min":1.026},"last":0.53300000000000003}]},{"osd":5,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68500000000000005,"15min":0.66200000000000003},"min":{"1min":0.46700000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89900000000000002,"5min":0.94599999999999995,"15min":3.4500000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68600000000000005,"15min":0.68200000000000005},"min":{"1min":0.45600000000000002,"5min":0.44,"15min":0.42399999999999999},"max":{"1min":0.93000000000000005,"5min":1.004,"15min":3.181},"last":0.58299999999999996}]},{"osd":6,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.71599999999999997,"15min":0.70599999999999996},"min":{"1min":0.52500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91400000000000003,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.74099999999999999,"15min":0.70599999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.50800000000000001},"max":{"1min":0.93899999999999995,"5min":1.143,"15min":1.869},"last":0.61699999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73799999999999999,"15min":0.71099999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85399999999999998,"5min":1.181,"15min":1.6870000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72799999999999998,"15min":0.71099999999999997},"min":{"1min":0.56299999999999994,"5min":0.501,"15min":0.45300000000000001},"max":{"1min":0.92400000000000004,"5min":1.0429999999999999,"15min":1.708},"last":0.63600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574996,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9781,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185531,"statfs":{"total":200000000,"available":189983777,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39500000000000002,"15min":0.39300000000000002},"min":{"1min":0.28499999999999998,"5min":0.24399999999999999,"15min":0.23799999999999999},"max":{"1min":0.55000000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.378},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42499999999999999,"15min":0.41999999999999998},"min":{"1min":0.32400000000000001,"5min":0.26000000000000001,"15min":0.223},"max":{"1min":0.55800000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.34200000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42499999999999999},"min":{"1min":0.34999999999999998,"5min":0.28599999999999998,"15min":0.28199999999999997},"max":{"1min":0.60799999999999998,"5min":0.625,"15min":0.64000000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45500000000000002,"15min":0.44400000000000001},"min":{"1min":0.34200000000000003,"5min":0.29499999999999998,"15min":0.28799999999999998},"max":{"1min":0.626,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.41399999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.44600000000000001,"15min":0.436},"min":{"1min":0.29999999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58499999999999996,"5min":0.66600000000000004,"15min":0.67300000000000004},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.39000000000000001,"5min":0.34300000000000003,"15min":0.29099999999999998},"max":{"1min":0.69799999999999995,"5min":0.71399999999999997,"15min":0.71699999999999997},"last":0.42599999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.65400000000000003,"15min":0.64900000000000002},"min":{"1min":0.46600000000000003,"5min":0.432,"15min":0.42099999999999999},"max":{"1min":0.84899999999999998,"5min":0.97599999999999998,"15min":0.997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63700000000000001,"15min":0.65000000000000002},"min":{"1min":0.46400000000000002,"5min":0.42199999999999999,"15min":0.41999999999999998},"max":{"1min":0.85999999999999999,"5min":0.87,"15min":0.96699999999999997},"last":0.61699999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.54100000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":2.4860000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.44},"max":{"1min":0.90200000000000002,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.78100000000000003}]},{"osd":6,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.51300000000000001,"5min":0.45600000000000002,"15min":0.42499999999999999},"max":{"1min":0.92600000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68899999999999995,"15min":0.68600000000000005},"min":{"1min":0.53600000000000003,"5min":0.46500000000000002,"15min":0.45000000000000001},"max":{"1min":0.96699999999999997,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.66400000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.51400000000000001,"5min":0.47099999999999997,"15min":0.42599999999999999},"max":{"1min":0.94799999999999995,"5min":0.94999999999999996,"15min":1.014},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68899999999999995,"15min":0.68999999999999995},"min":{"1min":0.55800000000000005,"5min":0.499,"15min":0.498},"max":{"1min":0.90900000000000003,"5min":0.92500000000000004,"15min":0.97299999999999998},"last":0.67800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574996,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21493,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173819,"statfs":{"total":200000000,"available":177991143,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41099999999999998},"min":{"1min":0.27000000000000002,"5min":0.23300000000000001,"15min":0.23000000000000001},"max":{"1min":0.63200000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.46000000000000002,"15min":0.53900000000000003},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.30499999999999999},"max":{"1min":0.57999999999999996,"5min":0.68899999999999995,"15min":6.2729999999999997},"last":0.33200000000000002}]},{"osd":1,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.41199999999999998,"15min":0.42099999999999999},"min":{"1min":0.33000000000000002,"5min":0.28799999999999998,"15min":0.23799999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.81399999999999995},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.442,"5min":0.443,"15min":0.442},"min":{"1min":0.29999999999999999,"5min":0.28999999999999998,"15min":0.28899999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.70099999999999996},"last":0.39300000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.31,"5min":0.28499999999999998,"15min":0.27800000000000002},"max":{"1min":0.55800000000000005,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48399999999999999,"15min":0.48499999999999999},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.70399999999999996,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.438}]},{"osd":4,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68100000000000005,"15min":0.67000000000000004},"min":{"1min":0.54600000000000004,"5min":0.47599999999999998,"15min":0.39300000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.98299999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.746},"min":{"1min":0.56499999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94399999999999995,"5min":0.95799999999999996,"15min":3.581},"last":0.61499999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65800000000000003,"15min":0.69599999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.43099999999999999},"max":{"1min":0.90400000000000003,"5min":0.92200000000000004,"15min":3.5369999999999999},"last":0.499},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66400000000000003,"15min":0.66800000000000004},"min":{"1min":0.5,"5min":0.44800000000000001,"15min":0.41099999999999998},"max":{"1min":0.79900000000000004,"5min":0.89100000000000001,"15min":1.0529999999999999},"last":0.48299999999999998}]},{"osd":6,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.55300000000000005,"5min":0.46999999999999997,"15min":0.42299999999999999},"max":{"1min":1.0900000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70099999999999996,"15min":0.68899999999999995},"min":{"1min":0.48299999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0469999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.68100000000000005}]},{"osd":7,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69599999999999995,"15min":0.68500000000000005},"min":{"1min":0.46999999999999997,"5min":0.437,"15min":0.437},"max":{"1min":1.0600000000000001,"5min":1.55,"15min":1.55},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71999999999999997,"15min":0.70099999999999996},"min":{"1min":0.54300000000000004,"5min":0.51300000000000001,"15min":0.42699999999999999},"max":{"1min":0.96999999999999997,"5min":1.615,"15min":1.615},"last":0.58699999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574996,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9205,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186107,"statfs":{"total":200000000,"available":190573909,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.253},"max":{"1min":0.56299999999999994,"5min":0.67900000000000005,"15min":4.7380000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.42099999999999999,"15min":0.40899999999999997},"min":{"1min":0.30499999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":1.0189999999999999},"last":0.40000000000000002}]},{"osd":1,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40500000000000003,"15min":0.41999999999999998},"min":{"1min":0.27600000000000002,"5min":0.24399999999999999,"15min":0.22900000000000001},"max":{"1min":0.54300000000000004,"5min":0.64400000000000002,"15min":4.5739999999999998},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46300000000000002,"15min":0.46500000000000002},"min":{"1min":0.35499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.751,"5min":0.98199999999999998,"15min":5.0529999999999999},"last":0.54100000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.46800000000000003,"15min":0.45600000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.28100000000000003},"max":{"1min":0.624,"5min":0.76700000000000002,"15min":1.5980000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.48899999999999999,"15min":0.49299999999999999},"min":{"1min":0.42599999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.65700000000000003,"5min":0.73499999999999999,"15min":5.1879999999999997},"last":0.46600000000000003}]},{"osd":4,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63500000000000001,"15min":0.64500000000000002},"min":{"1min":0.44,"5min":0.42999999999999999,"15min":0.41699999999999998},"max":{"1min":0.877,"5min":0.91600000000000004,"15min":4.6660000000000004},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65400000000000003,"15min":0.63600000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84199999999999997,"5min":0.94799999999999995,"15min":1.3120000000000001},"last":0.69899999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.49199999999999999,"5min":0.438,"15min":0.40799999999999997},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.409},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66600000000000004,"15min":0.70499999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81499999999999995,"5min":0.96599999999999997,"15min":4.7960000000000003},"last":0.86599999999999999}]},{"osd":6,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67200000000000004,"15min":0.68500000000000005},"min":{"1min":0.50900000000000001,"5min":0.42799999999999999,"15min":0.39800000000000002},"max":{"1min":0.86099999999999999,"5min":0.93500000000000005,"15min":5.0069999999999997},"last":0.751},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69299999999999995,"15min":0.67200000000000004},"min":{"1min":0.53000000000000003,"5min":0.47499999999999998,"15min":0.435},"max":{"1min":0.83499999999999996,"5min":0.94199999999999995,"15min":1.103},"last":0.66200000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46000000000000002},"max":{"1min":0.83399999999999996,"5min":0.92100000000000004,"15min":4.7130000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.68899999999999995},"min":{"1min":0.48599999999999999,"5min":0.47299999999999998,"15min":0.435},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.361},"last":0.77200000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574996,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":7215,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":188097,"statfs":{"total":200000000,"available":192611756,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.43099999999999999,"5min":0.35599999999999998,"15min":0.314},"max":{"1min":0.76800000000000002,"5min":1.137,"15min":1.3839999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60899999999999999,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":1.016,"15min":7.0670000000000002},"last":0.622}]},{"osd":1,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.35399999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":5.1529999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.48499999999999999,"5min":0.43099999999999999,"15min":0.39000000000000001},"max":{"1min":0.80100000000000005,"5min":1.054,"15min":3.0419999999999998},"last":0.60999999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60399999999999998,"15min":0.61499999999999999},"min":{"1min":0.46400000000000002,"5min":0.41199999999999998,"15min":0.39400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":5.0019999999999998},"last":0.749},{"interface":"front","average":{"1min":0.622,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.49399999999999999,"5min":0.41899999999999998,"15min":0.40500000000000003},"max":{"1min":0.86099999999999999,"5min":1.2549999999999999,"15min":2.073},"last":0.52300000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.496,"5min":0.439,"15min":0.36199999999999999},"max":{"1min":0.78800000000000003,"5min":1.1899999999999999,"15min":6.7220000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.629,"15min":0.65700000000000003},"min":{"1min":0.442,"5min":0.42499999999999999,"15min":0.41399999999999998},"max":{"1min":0.872,"5min":0.88,"15min":6.5510000000000002},"last":0.71399999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.36199999999999999,"5min":0.314,"15min":0.312},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":1.3149999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.5,"15min":0.51800000000000002},"min":{"1min":0.41399999999999998,"5min":0.32500000000000001,"15min":0.29999999999999999},"max":{"1min":0.77500000000000002,"5min":0.79800000000000004,"15min":6.9269999999999996},"last":0.505}]},{"osd":6,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.432,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":5.2229999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55600000000000005,"15min":0.55100000000000005},"min":{"1min":0.437,"5min":0.35799999999999998,"15min":0.33200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":1.46},"last":0.69199999999999995}]},{"osd":7,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55200000000000005,"15min":0.57099999999999995},"min":{"1min":0.41399999999999998,"5min":0.36599999999999999,"15min":0.33600000000000002},"max":{"1min":0.80400000000000005,"5min":0.83499999999999996,"15min":7.5339999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58099999999999996,"15min":0.59399999999999997},"min":{"1min":0.47099999999999997,"5min":0.34699999999999998,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.85299999999999998,"15min":4.9550000000000001},"last":0.67300000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574996,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21493,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173819,"statfs":{"total":200000000,"available":177990657,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.44700000000000001,"5min":0.33900000000000002,"15min":0.28899999999999998},"max":{"1min":0.76300000000000001,"5min":1.071,"15min":1.071},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.47999999999999998,"5min":0.38500000000000001,"15min":0.27600000000000002},"max":{"1min":0.71099999999999997,"5min":0.83599999999999997,"15min":3.242},"last":0.502}]},{"osd":1,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.627},"min":{"1min":0.53500000000000003,"5min":0.38100000000000001,"15min":0.32800000000000001},"max":{"1min":0.80800000000000005,"5min":1.018,"15min":1.3380000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.45600000000000002,"5min":0.35399999999999998,"15min":0.34200000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.95499999999999996},"last":0.54200000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":0.96699999999999997,"15min":3.04},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60799999999999998,"15min":0.59099999999999997},"min":{"1min":0.52000000000000002,"5min":0.43099999999999999,"15min":0.36899999999999999},"max":{"1min":0.71299999999999997,"5min":0.81000000000000005,"15min":0.89400000000000002},"last":0.45400000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63,"15min":0.65500000000000003},"min":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.39800000000000002},"max":{"1min":0.73799999999999999,"5min":1.0409999999999999,"15min":4.032},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66900000000000004,"15min":0.68000000000000005},"min":{"1min":0.57199999999999995,"5min":0.48799999999999999,"15min":0.45600000000000002},"max":{"1min":0.753,"5min":0.879,"15min":4.282},"last":0.55200000000000005}]},{"osd":4,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.51500000000000001},"min":{"1min":0.376,"5min":0.33300000000000002,"15min":0.26500000000000001},"max":{"1min":0.76400000000000001,"5min":0.95399999999999996,"15min":1.2869999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.496,"15min":0.52900000000000003},"min":{"1min":0.35799999999999998,"5min":0.32800000000000001,"15min":0.30399999999999999},"max":{"1min":0.67900000000000005,"5min":0.81699999999999995,"15min":1.101},"last":0.433}]},{"osd":6,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.50600000000000001,"15min":0.52900000000000003},"min":{"1min":0.35299999999999998,"5min":0.32700000000000001,"15min":0.29099999999999998},"max":{"1min":0.65700000000000003,"5min":0.86599999999999999,"15min":1.7969999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.55200000000000005,"15min":0.56799999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.33600000000000002},"max":{"1min":0.69299999999999995,"5min":0.91800000000000004,"15min":1.6739999999999999},"last":0.38400000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.55100000000000005,"15min":0.55900000000000005},"min":{"1min":0.39500000000000002,"5min":0.36899999999999999,"15min":0.32700000000000001},"max":{"1min":0.76300000000000001,"5min":0.90600000000000003,"15min":1.008},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59199999999999997,"15min":0.627},"min":{"1min":0.46000000000000002,"5min":0.40600000000000003,"15min":0.33700000000000002},"max":{"1min":0.67500000000000004,"5min":0.91300000000000003,"15min":2},"last":0.46400000000000002}]}]}],"pool_statfs":[]}} 2024-01-27T02:44:10.892 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-01-27T02:44:10.892 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:44:11.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.023+0000 7f5c37650700 1 Processor -- start 2024-01-27T02:44:11.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.024+0000 7f5c37650700 1 -- start start 2024-01-27T02:44:11.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.024+0000 7f5c37650700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5c300c89a0 0x7f5c3006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:11.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.024+0000 7f5c37650700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c30063c90 0x7f5c3015a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:11.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.024+0000 7f5c37650700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c300c4450 con 0x7f5c300c8d70 2024-01-27T02:44:11.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.024+0000 7f5c37650700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c3006e9e0 con 0x7f5c300c89a0 2024-01-27T02:44:11.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.025+0000 7f5c37650700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3006e3d0 con 0x7f5c30063c90 2024-01-27T02:44:11.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.025+0000 7f5c353ec700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5c300c89a0 0x7f5c3006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:11.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.025+0000 7f5c35bed700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c30063c90 0x7f5c3015a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:11.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.025+0000 7f5c34beb700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f5c300c8d70 0x7f5c30063580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:58310/0 (socket says 172.21.15.7:58310) 2024-01-27T02:44:11.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.025+0000 7f5c34beb700 1 -- 172.21.15.7:0/2751652874 learned_addr learned my addr 172.21.15.7:0/2751652874 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:11.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.025+0000 7f5c27fff700 1 -- 172.21.15.7:0/2751652874 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3209007770 0 0) 0x7f5c300c4450 con 0x7f5c300c8d70 2024-01-27T02:44:11.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.025+0000 7f5c35bed700 1 -- 172.21.15.7:0/2751652874 >> v2:172.21.15.188:3300/0 conn(0x7f5c300c89a0 msgr2=0x7f5c3006ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:11.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.025+0000 7f5c35bed700 1 --2- 172.21.15.7:0/2751652874 >> v2:172.21.15.188:3300/0 conn(0x7f5c300c89a0 0x7f5c3006ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:11.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.025+0000 7f5c35bed700 1 -- 172.21.15.7:0/2751652874 >> v1:172.21.15.188:6789/0 conn(0x7f5c300c8d70 legacy=0x7f5c30063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:11.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.025+0000 7f5c35bed700 1 -- 172.21.15.7:0/2751652874 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3015b360 con 0x7f5c30063c90 2024-01-27T02:44:11.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.025+0000 7f5c353ec700 1 --2- 172.21.15.7:0/2751652874 >> v2:172.21.15.188:3300/0 conn(0x7f5c300c89a0 0x7f5c3006ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:44:11.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.025+0000 7f5c35bed700 1 --2- 172.21.15.7:0/2751652874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c30063c90 0x7f5c3015a710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009e60 tx=0x7f5c28009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45fbdafb9bdd4d95 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:11.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.026+0000 7f5c27fff700 1 -- 172.21.15.7:0/2751652874 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2800e070 con 0x7f5c30063c90 2024-01-27T02:44:11.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.026+0000 7f5c27fff700 1 -- 172.21.15.7:0/2751652874 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2800bb60 con 0x7f5c30063c90 2024-01-27T02:44:11.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.026+0000 7f5c27fff700 1 -- 172.21.15.7:0/2751652874 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c28012430 con 0x7f5c30063c90 2024-01-27T02:44:11.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.026+0000 7f5c37650700 1 -- 172.21.15.7:0/2751652874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c30063c90 msgr2=0x7f5c3015a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:11.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.026+0000 7f5c37650700 1 --2- 172.21.15.7:0/2751652874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c30063c90 0x7f5c3015a710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009e60 tx=0x7f5c28009780 comp rx=0 tx=0).stop 2024-01-27T02:44:11.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.026+0000 7f5c37650700 1 -- 172.21.15.7:0/2751652874 shutdown_connections 2024-01-27T02:44:11.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.026+0000 7f5c37650700 1 --2- 172.21.15.7:0/2751652874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c30063c90 0x7f5c3015a710 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:11.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.026+0000 7f5c37650700 1 --2- 172.21.15.7:0/2751652874 >> v2:172.21.15.188:3300/0 conn(0x7f5c300c89a0 0x7f5c3006ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:11.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.026+0000 7f5c37650700 1 -- 172.21.15.7:0/2751652874 >> 172.21.15.7:0/2751652874 conn(0x7f5c300cdc30 msgr2=0x7f5c300ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:11.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.026+0000 7f5c37650700 1 -- 172.21.15.7:0/2751652874 shutdown_connections 2024-01-27T02:44:11.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.026+0000 7f5c37650700 1 -- 172.21.15.7:0/2751652874 wait complete. 2024-01-27T02:44:11.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.028+0000 7f5c37650700 1 Processor -- start 2024-01-27T02:44:11.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.028+0000 7f5c37650700 1 -- start start 2024-01-27T02:44:11.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.029+0000 7f5c37650700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c30063c90 0x7f5c300abf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:11.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.029+0000 7f5c37650700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c300c89a0 0x7f5c3006bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:11.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.029+0000 7f5c37650700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c300c8d70 0x7f5c3014e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:11.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.029+0000 7f5c37650700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3006a530 con 0x7f5c30063c90 2024-01-27T02:44:11.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.029+0000 7f5c37650700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c300c50c0 con 0x7f5c300c8d70 2024-01-27T02:44:11.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.029+0000 7f5c37650700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c300c53a0 con 0x7f5c300c89a0 2024-01-27T02:44:11.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.029+0000 7f5c353ec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c30063c90 0x7f5c300abf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:11.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.030+0000 7f5c353ec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c30063c90 0x7f5c300abf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48732/0 (socket says 172.21.15.7:48732) 2024-01-27T02:44:11.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.030+0000 7f5c353ec700 1 -- 172.21.15.7:0/2444742452 learned_addr learned my addr 172.21.15.7:0/2444742452 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:11.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.030+0000 7f5c34beb700 1 --2- 172.21.15.7:0/2444742452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c300c89a0 0x7f5c3006bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:11.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.030+0000 7f5c35bed700 1 --2- 172.21.15.7:0/2444742452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c300c8d70 0x7f5c3014e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:11.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.030+0000 7f5c353ec700 1 -- 172.21.15.7:0/2444742452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c300c89a0 msgr2=0x7f5c3006bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:11.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.030+0000 7f5c353ec700 1 --2- 172.21.15.7:0/2444742452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c300c89a0 0x7f5c3006bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:11.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.030+0000 7f5c353ec700 1 -- 172.21.15.7:0/2444742452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c300c8d70 msgr2=0x7f5c3014e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:11.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.030+0000 7f5c353ec700 1 --2- 172.21.15.7:0/2444742452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c300c8d70 0x7f5c3014e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:11.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.030+0000 7f5c353ec700 1 -- 172.21.15.7:0/2444742452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2c009420 con 0x7f5c30063c90 2024-01-27T02:44:11.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.030+0000 7f5c34beb700 1 --2- 172.21.15.7:0/2444742452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c300c89a0 0x7f5c3006bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:44:11.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.030+0000 7f5c35bed700 1 --2- 172.21.15.7:0/2444742452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c300c8d70 0x7f5c3014e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:44:11.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.030+0000 7f5c353ec700 1 --2- 172.21.15.7:0/2444742452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c30063c90 0x7f5c300abf00 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00ec80 tx=0x7f5c2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:11.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.030+0000 7f5c257fa700 1 -- 172.21.15.7:0/2444742452 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c00c9d0 con 0x7f5c30063c90 2024-01-27T02:44:11.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.030+0000 7f5c257fa700 1 -- 172.21.15.7:0/2444742452 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2c00cb30 con 0x7f5c30063c90 2024-01-27T02:44:11.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.030+0000 7f5c37650700 1 -- 172.21.15.7:0/2444742452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c28009a20 con 0x7f5c30063c90 2024-01-27T02:44:11.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.030+0000 7f5c257fa700 1 -- 172.21.15.7:0/2444742452 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c00ceb0 con 0x7f5c30063c90 2024-01-27T02:44:11.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.030+0000 7f5c37650700 1 -- 172.21.15.7:0/2444742452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c300c5960 con 0x7f5c30063c90 2024-01-27T02:44:11.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.032+0000 7f5c37650700 1 -- 172.21.15.7:0/2444742452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c3006a300 con 0x7f5c30063c90 2024-01-27T02:44:11.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.032+0000 7f5c257fa700 1 -- 172.21.15.7:0/2444742452 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5c2c013680 con 0x7f5c30063c90 2024-01-27T02:44:11.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.032+0000 7f5c257fa700 1 --2- 172.21.15.7:0/2444742452 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5c14066f30 0x7f5c140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:11.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.032+0000 7f5c257fa700 1 -- 172.21.15.7:0/2444742452 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f5c2c0846e0 con 0x7f5c30063c90 2024-01-27T02:44:11.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.033+0000 7f5c34beb700 1 --2- 172.21.15.7:0/2444742452 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5c14066f30 0x7f5c140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:11.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.033+0000 7f5c34beb700 1 --2- 172.21.15.7:0/2444742452 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5c14066f30 0x7f5c140693e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5c3016d9a0 tx=0x7f5c20009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:11.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.038+0000 7f5c257fa700 1 -- 172.21.15.7:0/2444742452 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c3006a300 con 0x7f5c30063c90 2024-01-27T02:44:11.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.239+0000 7f5c37650700 1 -- 172.21.15.7:0/2444742452 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5c3006a670 con 0x7f5c14066f30 2024-01-27T02:44:11.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.242+0000 7f5c257fa700 1 -- 172.21.15.7:0/2444742452 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50301 (secure 0 0 0) 0x7f5c200058a0 con 0x7f5c14066f30 2024-01-27T02:44:11.243 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:44:11.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.244+0000 7f5c37650700 1 -- 172.21.15.7:0/2444742452 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5c14066f30 msgr2=0x7f5c140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:11.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.244+0000 7f5c37650700 1 --2- 172.21.15.7:0/2444742452 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5c14066f30 0x7f5c140693e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5c3016d9a0 tx=0x7f5c20009040 comp rx=0 tx=0).stop 2024-01-27T02:44:11.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.244+0000 7f5c37650700 1 -- 172.21.15.7:0/2444742452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c30063c90 msgr2=0x7f5c300abf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:11.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.244+0000 7f5c37650700 1 --2- 172.21.15.7:0/2444742452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c30063c90 0x7f5c300abf00 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00ec80 tx=0x7f5c2c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:11.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.244+0000 7f5c37650700 1 -- 172.21.15.7:0/2444742452 shutdown_connections 2024-01-27T02:44:11.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.244+0000 7f5c37650700 1 --2- 172.21.15.7:0/2444742452 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5c14066f30 0x7f5c140693e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:11.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.244+0000 7f5c37650700 1 --2- 172.21.15.7:0/2444742452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c300c8d70 0x7f5c3014e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:11.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.244+0000 7f5c37650700 1 --2- 172.21.15.7:0/2444742452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c30063c90 0x7f5c300abf00 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:11.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.244+0000 7f5c37650700 1 --2- 172.21.15.7:0/2444742452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c300c89a0 0x7f5c3006bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:11.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.244+0000 7f5c37650700 1 -- 172.21.15.7:0/2444742452 >> 172.21.15.7:0/2444742452 conn(0x7f5c300cdc30 msgr2=0x7f5c30065700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:11.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.244+0000 7f5c37650700 1 -- 172.21.15.7:0/2444742452 shutdown_connections 2024-01-27T02:44:11.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.244+0000 7f5c37650700 1 -- 172.21.15.7:0/2444742452 wait complete. 2024-01-27T02:44:11.246 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:44:11.258 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":544,"stamp":"2024-01-27T02:44:09.751800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":111774,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1450725,"statfs":{"total":1600000000,"available":1485542802,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52489243,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-120,"num_write_kb":-85081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.905843"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":307,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-01-27T02:44:04.852416+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:44:04.852416+0000","last_peered":"2024-01-27T02:44:04.852416+0000","last_clean":"2024-01-27T02:44:04.852416+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:44:04.852416+0000","last_undegraded":"2024-01-27T02:44:04.852416+0000","last_fullsized":"2024-01-27T02:44:04.852416+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574996,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9840,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185472,"statfs":{"total":200000000,"available":189923592,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.32800000000000001},"max":{"1min":0.80600000000000005,"5min":0.83499999999999996,"15min":0.85499999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59899999999999998,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.30099999999999999},"max":{"1min":0.84199999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.59699999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.47699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74399999999999999,"5min":0.78800000000000003,"15min":0.82399999999999995},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.57399999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.34999999999999998},"max":{"1min":0.81599999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.49099999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.38400000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.86399999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.75,"5min":0.80300000000000005,"15min":0.83899999999999997},"last":0.55500000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76500000000000001,"5min":0.78900000000000003,"15min":0.90300000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.63800000000000001,"15min":0.61199999999999999},"min":{"1min":0.47599999999999998,"5min":0.41799999999999998,"15min":0.40999999999999998},"max":{"1min":0.77400000000000002,"5min":0.81599999999999995,"15min":0.88600000000000001},"last":0.63900000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48199999999999998,"15min":0.48399999999999999},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67300000000000004,"5min":0.73899999999999999,"15min":0.78600000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52000000000000002,"15min":0.51000000000000001},"min":{"1min":0.36699999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.80000000000000004},"last":0.625}]},{"osd":5,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.39600000000000002,"5min":0.32400000000000001,"15min":0.307},"max":{"1min":0.746,"5min":0.79500000000000004,"15min":0.85399999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.42499999999999999,"5min":0.33900000000000002,"15min":0.33500000000000002},"max":{"1min":0.76600000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.53800000000000003}]},{"osd":6,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60199999999999998,"15min":0.59299999999999997},"min":{"1min":0.38900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.78900000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71799999999999997,"5min":0.83499999999999996,"15min":0.90800000000000003},"last":0.57899999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574996,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":19445,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":175867,"statfs":{"total":200000000,"available":180088161,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60099999999999998,"15min":0.57899999999999996},"min":{"1min":0.38800000000000001,"5min":0.375,"15min":0.29999999999999999},"max":{"1min":0.78700000000000003,"5min":1.0409999999999999,"15min":4.125},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58799999999999997,"15min":0.56499999999999995},"min":{"1min":0.435,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.753,"5min":0.96599999999999997,"15min":1.216},"last":0.45000000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42599999999999999,"5min":0.376,"15min":0.34100000000000003},"max":{"1min":0.67800000000000005,"5min":0.95499999999999996,"15min":8.2699999999999996},"last":0.623},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59699999999999998,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78000000000000003,"5min":1.0129999999999999,"15min":26.411999999999999},"last":0.42099999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.627},"min":{"1min":0.51900000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73699999999999999,"5min":0.78300000000000003,"15min":17.143999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61499999999999999,"15min":0.60899999999999999},"min":{"1min":0.503,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73199999999999998,"5min":0.85799999999999998,"15min":1.76},"last":0.55800000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.47699999999999998,"5min":0.39400000000000002,"15min":0.376},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":4.3689999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.41199999999999998},"max":{"1min":0.77400000000000002,"5min":0.81699999999999995,"15min":2.1970000000000001},"last":0.54000000000000004}]},{"osd":4,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47099999999999997,"15min":0.48099999999999998},"min":{"1min":0.35299999999999998,"5min":0.311,"15min":0.30199999999999999},"max":{"1min":0.61499999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78900000000000003,"5min":0.84899999999999998,"15min":0.89600000000000002},"last":0.46800000000000003}]},{"osd":5,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.39400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.65400000000000003,"5min":0.80800000000000005,"15min":0.81299999999999994},"last":0.502},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55200000000000005,"15min":0.57699999999999996},"min":{"1min":0.38300000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76400000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.58299999999999996}]},{"osd":7,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57799999999999996,"15min":0.59599999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.35999999999999999},"max":{"1min":0.73599999999999999,"5min":0.88700000000000001,"15min":1.0009999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54900000000000004,"15min":0.56799999999999995},"min":{"1min":0.40699999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.749,"5min":0.82899999999999996,"15min":1.038},"last":0.59399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574996,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":13300,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":182011,"statfs":{"total":200000000,"available":186379807,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40300000000000002,"15min":0.38900000000000001},"min":{"1min":0.26900000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.51800000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.42999999999999999,"15min":0.41299999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.27700000000000002},"max":{"1min":0.51400000000000001,"5min":0.66100000000000003,"15min":1.121},"last":0.34200000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42199999999999999,"15min":0.39300000000000002},"min":{"1min":0.30599999999999999,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":0.56000000000000005,"5min":1.0940000000000001,"15min":1.25},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.45200000000000001,"15min":0.442},"min":{"1min":0.30099999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.65800000000000003,"5min":0.66700000000000004,"15min":0.82999999999999996},"last":0.46400000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.495,"15min":0.48199999999999998},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.32100000000000001},"max":{"1min":0.61299999999999999,"5min":0.79900000000000004,"15min":0.90900000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47399999999999998,"15min":0.46400000000000002},"min":{"1min":0.35999999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.59199999999999997,"5min":0.72599999999999998,"15min":0.98999999999999999},"last":0.38600000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66000000000000003,"15min":0.64800000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.373},"max":{"1min":0.83999999999999997,"5min":1.075,"15min":3.2890000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67800000000000005,"15min":0.65500000000000003},"min":{"1min":0.48999999999999999,"5min":0.45000000000000001,"15min":0.44500000000000001},"max":{"1min":0.95499999999999996,"5min":0.98099999999999998,"15min":1.026},"last":0.53300000000000003}]},{"osd":5,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68500000000000005,"15min":0.66200000000000003},"min":{"1min":0.46700000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89900000000000002,"5min":0.94599999999999995,"15min":3.4500000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68600000000000005,"15min":0.68200000000000005},"min":{"1min":0.45600000000000002,"5min":0.44,"15min":0.42399999999999999},"max":{"1min":0.93000000000000005,"5min":1.004,"15min":3.181},"last":0.58299999999999996}]},{"osd":6,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.71599999999999997,"15min":0.70599999999999996},"min":{"1min":0.52500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91400000000000003,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.74099999999999999,"15min":0.70599999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.50800000000000001},"max":{"1min":0.93899999999999995,"5min":1.143,"15min":1.869},"last":0.61699999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73799999999999999,"15min":0.71099999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85399999999999998,"5min":1.181,"15min":1.6870000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72799999999999998,"15min":0.71099999999999997},"min":{"1min":0.56299999999999994,"5min":0.501,"15min":0.45300000000000001},"max":{"1min":0.92400000000000004,"5min":1.0429999999999999,"15min":1.708},"last":0.63600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574996,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9781,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185531,"statfs":{"total":200000000,"available":189983777,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39500000000000002,"15min":0.39300000000000002},"min":{"1min":0.28499999999999998,"5min":0.24399999999999999,"15min":0.23799999999999999},"max":{"1min":0.55000000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.378},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42499999999999999,"15min":0.41999999999999998},"min":{"1min":0.32400000000000001,"5min":0.26000000000000001,"15min":0.223},"max":{"1min":0.55800000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.34200000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42499999999999999},"min":{"1min":0.34999999999999998,"5min":0.28599999999999998,"15min":0.28199999999999997},"max":{"1min":0.60799999999999998,"5min":0.625,"15min":0.64000000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45500000000000002,"15min":0.44400000000000001},"min":{"1min":0.34200000000000003,"5min":0.29499999999999998,"15min":0.28799999999999998},"max":{"1min":0.626,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.41399999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.44600000000000001,"15min":0.436},"min":{"1min":0.29999999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58499999999999996,"5min":0.66600000000000004,"15min":0.67300000000000004},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.39000000000000001,"5min":0.34300000000000003,"15min":0.29099999999999998},"max":{"1min":0.69799999999999995,"5min":0.71399999999999997,"15min":0.71699999999999997},"last":0.42599999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.65400000000000003,"15min":0.64900000000000002},"min":{"1min":0.46600000000000003,"5min":0.432,"15min":0.42099999999999999},"max":{"1min":0.84899999999999998,"5min":0.97599999999999998,"15min":0.997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63700000000000001,"15min":0.65000000000000002},"min":{"1min":0.46400000000000002,"5min":0.42199999999999999,"15min":0.41999999999999998},"max":{"1min":0.85999999999999999,"5min":0.87,"15min":0.96699999999999997},"last":0.61699999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.54100000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":2.4860000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.44},"max":{"1min":0.90200000000000002,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.78100000000000003}]},{"osd":6,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.51300000000000001,"5min":0.45600000000000002,"15min":0.42499999999999999},"max":{"1min":0.92600000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68899999999999995,"15min":0.68600000000000005},"min":{"1min":0.53600000000000003,"5min":0.46500000000000002,"15min":0.45000000000000001},"max":{"1min":0.96699999999999997,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.66400000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.51400000000000001,"5min":0.47099999999999997,"15min":0.42599999999999999},"max":{"1min":0.94799999999999995,"5min":0.94999999999999996,"15min":1.014},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68899999999999995,"15min":0.68999999999999995},"min":{"1min":0.55800000000000005,"5min":0.499,"15min":0.498},"max":{"1min":0.90900000000000003,"5min":0.92500000000000004,"15min":0.97299999999999998},"last":0.67800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574996,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21493,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173819,"statfs":{"total":200000000,"available":177991143,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41099999999999998},"min":{"1min":0.27000000000000002,"5min":0.23300000000000001,"15min":0.23000000000000001},"max":{"1min":0.63200000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.46000000000000002,"15min":0.53900000000000003},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.30499999999999999},"max":{"1min":0.57999999999999996,"5min":0.68899999999999995,"15min":6.2729999999999997},"last":0.33200000000000002}]},{"osd":1,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.41199999999999998,"15min":0.42099999999999999},"min":{"1min":0.33000000000000002,"5min":0.28799999999999998,"15min":0.23799999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.81399999999999995},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.442,"5min":0.443,"15min":0.442},"min":{"1min":0.29999999999999999,"5min":0.28999999999999998,"15min":0.28899999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.70099999999999996},"last":0.39300000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.31,"5min":0.28499999999999998,"15min":0.27800000000000002},"max":{"1min":0.55800000000000005,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48399999999999999,"15min":0.48499999999999999},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.70399999999999996,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.438}]},{"osd":4,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68100000000000005,"15min":0.67000000000000004},"min":{"1min":0.54600000000000004,"5min":0.47599999999999998,"15min":0.39300000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.98299999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.746},"min":{"1min":0.56499999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94399999999999995,"5min":0.95799999999999996,"15min":3.581},"last":0.61499999999999999}]},{"osd":5,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65800000000000003,"15min":0.69599999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.43099999999999999},"max":{"1min":0.90400000000000003,"5min":0.92200000000000004,"15min":3.5369999999999999},"last":0.499},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66400000000000003,"15min":0.66800000000000004},"min":{"1min":0.5,"5min":0.44800000000000001,"15min":0.41099999999999998},"max":{"1min":0.79900000000000004,"5min":0.89100000000000001,"15min":1.0529999999999999},"last":0.48299999999999998}]},{"osd":6,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.55300000000000005,"5min":0.46999999999999997,"15min":0.42299999999999999},"max":{"1min":1.0900000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70099999999999996,"15min":0.68899999999999995},"min":{"1min":0.48299999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0469999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.68100000000000005}]},{"osd":7,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69599999999999995,"15min":0.68500000000000005},"min":{"1min":0.46999999999999997,"5min":0.437,"15min":0.437},"max":{"1min":1.0600000000000001,"5min":1.55,"15min":1.55},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71999999999999997,"15min":0.70099999999999996},"min":{"1min":0.54300000000000004,"5min":0.51300000000000001,"15min":0.42699999999999999},"max":{"1min":0.96999999999999997,"5min":1.615,"15min":1.615},"last":0.58699999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574996,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9205,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186107,"statfs":{"total":200000000,"available":190573909,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.253},"max":{"1min":0.56299999999999994,"5min":0.67900000000000005,"15min":4.7380000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.42099999999999999,"15min":0.40899999999999997},"min":{"1min":0.30499999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":1.0189999999999999},"last":0.40000000000000002}]},{"osd":1,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40500000000000003,"15min":0.41999999999999998},"min":{"1min":0.27600000000000002,"5min":0.24399999999999999,"15min":0.22900000000000001},"max":{"1min":0.54300000000000004,"5min":0.64400000000000002,"15min":4.5739999999999998},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46300000000000002,"15min":0.46500000000000002},"min":{"1min":0.35499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.751,"5min":0.98199999999999998,"15min":5.0529999999999999},"last":0.54100000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.46800000000000003,"15min":0.45600000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.28100000000000003},"max":{"1min":0.624,"5min":0.76700000000000002,"15min":1.5980000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.48899999999999999,"15min":0.49299999999999999},"min":{"1min":0.42599999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.65700000000000003,"5min":0.73499999999999999,"15min":5.1879999999999997},"last":0.46600000000000003}]},{"osd":4,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63500000000000001,"15min":0.64500000000000002},"min":{"1min":0.44,"5min":0.42999999999999999,"15min":0.41699999999999998},"max":{"1min":0.877,"5min":0.91600000000000004,"15min":4.6660000000000004},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65400000000000003,"15min":0.63600000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84199999999999997,"5min":0.94799999999999995,"15min":1.3120000000000001},"last":0.69899999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.49199999999999999,"5min":0.438,"15min":0.40799999999999997},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.409},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66600000000000004,"15min":0.70499999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81499999999999995,"5min":0.96599999999999997,"15min":4.7960000000000003},"last":0.86599999999999999}]},{"osd":6,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67200000000000004,"15min":0.68500000000000005},"min":{"1min":0.50900000000000001,"5min":0.42799999999999999,"15min":0.39800000000000002},"max":{"1min":0.86099999999999999,"5min":0.93500000000000005,"15min":5.0069999999999997},"last":0.751},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69299999999999995,"15min":0.67200000000000004},"min":{"1min":0.53000000000000003,"5min":0.47499999999999998,"15min":0.435},"max":{"1min":0.83499999999999996,"5min":0.94199999999999995,"15min":1.103},"last":0.66200000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46000000000000002},"max":{"1min":0.83399999999999996,"5min":0.92100000000000004,"15min":4.7130000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.68899999999999995},"min":{"1min":0.48599999999999999,"5min":0.47299999999999998,"15min":0.435},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.361},"last":0.77200000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574996,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":7215,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":188097,"statfs":{"total":200000000,"available":192611756,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.43099999999999999,"5min":0.35599999999999998,"15min":0.314},"max":{"1min":0.76800000000000002,"5min":1.137,"15min":1.3839999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60899999999999999,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":1.016,"15min":7.0670000000000002},"last":0.622}]},{"osd":1,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.35399999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":5.1529999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.48499999999999999,"5min":0.43099999999999999,"15min":0.39000000000000001},"max":{"1min":0.80100000000000005,"5min":1.054,"15min":3.0419999999999998},"last":0.60999999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60399999999999998,"15min":0.61499999999999999},"min":{"1min":0.46400000000000002,"5min":0.41199999999999998,"15min":0.39400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":5.0019999999999998},"last":0.749},{"interface":"front","average":{"1min":0.622,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.49399999999999999,"5min":0.41899999999999998,"15min":0.40500000000000003},"max":{"1min":0.86099999999999999,"5min":1.2549999999999999,"15min":2.073},"last":0.52300000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.496,"5min":0.439,"15min":0.36199999999999999},"max":{"1min":0.78800000000000003,"5min":1.1899999999999999,"15min":6.7220000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.629,"15min":0.65700000000000003},"min":{"1min":0.442,"5min":0.42499999999999999,"15min":0.41399999999999998},"max":{"1min":0.872,"5min":0.88,"15min":6.5510000000000002},"last":0.71399999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.36199999999999999,"5min":0.314,"15min":0.312},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":1.3149999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.5,"15min":0.51800000000000002},"min":{"1min":0.41399999999999998,"5min":0.32500000000000001,"15min":0.29999999999999999},"max":{"1min":0.77500000000000002,"5min":0.79800000000000004,"15min":6.9269999999999996},"last":0.505}]},{"osd":6,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.432,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":5.2229999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55600000000000005,"15min":0.55100000000000005},"min":{"1min":0.437,"5min":0.35799999999999998,"15min":0.33200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":1.46},"last":0.69199999999999995}]},{"osd":7,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55200000000000005,"15min":0.57099999999999995},"min":{"1min":0.41399999999999998,"5min":0.36599999999999999,"15min":0.33600000000000002},"max":{"1min":0.80400000000000005,"5min":0.83499999999999996,"15min":7.5339999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58099999999999996,"15min":0.59399999999999997},"min":{"1min":0.47099999999999997,"5min":0.34699999999999998,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.85299999999999998,"15min":4.9550000000000001},"last":0.67300000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574996,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21493,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173819,"statfs":{"total":200000000,"available":177990657,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.44700000000000001,"5min":0.33900000000000002,"15min":0.28899999999999998},"max":{"1min":0.76300000000000001,"5min":1.071,"15min":1.071},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.47999999999999998,"5min":0.38500000000000001,"15min":0.27600000000000002},"max":{"1min":0.71099999999999997,"5min":0.83599999999999997,"15min":3.242},"last":0.502}]},{"osd":1,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.627},"min":{"1min":0.53500000000000003,"5min":0.38100000000000001,"15min":0.32800000000000001},"max":{"1min":0.80800000000000005,"5min":1.018,"15min":1.3380000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.45600000000000002,"5min":0.35399999999999998,"15min":0.34200000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.95499999999999996},"last":0.54200000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":0.96699999999999997,"15min":3.04},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60799999999999998,"15min":0.59099999999999997},"min":{"1min":0.52000000000000002,"5min":0.43099999999999999,"15min":0.36899999999999999},"max":{"1min":0.71299999999999997,"5min":0.81000000000000005,"15min":0.89400000000000002},"last":0.45400000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63,"15min":0.65500000000000003},"min":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.39800000000000002},"max":{"1min":0.73799999999999999,"5min":1.0409999999999999,"15min":4.032},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66900000000000004,"15min":0.68000000000000005},"min":{"1min":0.57199999999999995,"5min":0.48799999999999999,"15min":0.45600000000000002},"max":{"1min":0.753,"5min":0.879,"15min":4.282},"last":0.55200000000000005}]},{"osd":4,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.51500000000000001},"min":{"1min":0.376,"5min":0.33300000000000002,"15min":0.26500000000000001},"max":{"1min":0.76400000000000001,"5min":0.95399999999999996,"15min":1.2869999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.496,"15min":0.52900000000000003},"min":{"1min":0.35799999999999998,"5min":0.32800000000000001,"15min":0.30399999999999999},"max":{"1min":0.67900000000000005,"5min":0.81699999999999995,"15min":1.101},"last":0.433}]},{"osd":6,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.50600000000000001,"15min":0.52900000000000003},"min":{"1min":0.35299999999999998,"5min":0.32700000000000001,"15min":0.29099999999999998},"max":{"1min":0.65700000000000003,"5min":0.86599999999999999,"15min":1.7969999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.55200000000000005,"15min":0.56799999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.33600000000000002},"max":{"1min":0.69299999999999995,"5min":0.91800000000000004,"15min":1.6739999999999999},"last":0.38400000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.55100000000000005,"15min":0.55900000000000005},"min":{"1min":0.39500000000000002,"5min":0.36899999999999999,"15min":0.32700000000000001},"max":{"1min":0.76300000000000001,"5min":0.90600000000000003,"15min":1.008},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59199999999999997,"15min":0.627},"min":{"1min":0.46000000000000002,"5min":0.40600000000000003,"15min":0.33700000000000002},"max":{"1min":0.67500000000000004,"5min":0.91300000000000003,"15min":2},"last":0.46400000000000002}]}]}],"pool_statfs":[]}} 2024-01-27T02:44:11.259 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-27T02:44:11.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.391+0000 7f2a81f76700 1 Processor -- start 2024-01-27T02:44:11.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.391+0000 7f2a81f76700 1 -- start start 2024-01-27T02:44:11.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.392+0000 7f2a81f76700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a7c0ccff0 0x7f2a7c0cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:11.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.392+0000 7f2a81f76700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a7c0c4a30 con 0x7f2a7c0cad20 2024-01-27T02:44:11.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.392+0000 7f2a81f76700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a7c15bb10 con 0x7f2a7c15b740 2024-01-27T02:44:11.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.392+0000 7f2a81f76700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c0c64d0 con 0x7f2a7c0ccff0 2024-01-27T02:44:11.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.392+0000 7f2a7affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2a7c0cad20 0x7f2a7c0cc8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46898/0 (socket says 172.21.15.7:46898) 2024-01-27T02:44:11.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.392+0000 7f2a7affd700 1 -- 172.21.15.7:0/1917847609 learned_addr learned my addr 172.21.15.7:0/1917847609 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:11.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.392+0000 7f2a7bfff700 1 --2- 172.21.15.7:0/1917847609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a7c0ccff0 0x7f2a7c0cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:11.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.392+0000 7f2a7a7fc700 1 -- 172.21.15.7:0/1917847609 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808807765 0 0) 0x7f2a7c0c4a30 con 0x7f2a7c0cad20 2024-01-27T02:44:11.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.392+0000 7f2a7a7fc700 1 -- 172.21.15.7:0/1917847609 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a64003570 con 0x7f2a7c0cad20 2024-01-27T02:44:11.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.392+0000 7f2a7a7fc700 1 -- 172.21.15.7:0/1917847609 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045774537 0 0) 0x7f2a7c15bb10 con 0x7f2a7c15b740 2024-01-27T02:44:11.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.392+0000 7f2a7a7fc700 1 -- 172.21.15.7:0/1917847609 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a7c0c4a30 con 0x7f2a7c15b740 2024-01-27T02:44:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.392+0000 7f2a7bfff700 1 -- 172.21.15.7:0/1917847609 >> v1:172.21.15.188:6789/0 conn(0x7f2a7c15b740 legacy=0x7f2a7c0c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.392+0000 7f2a7bfff700 1 -- 172.21.15.7:0/1917847609 >> v1:172.21.15.7:6789/0 conn(0x7f2a7c0cad20 legacy=0x7f2a7c0cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.392+0000 7f2a7bfff700 1 -- 172.21.15.7:0/1917847609 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7c0c3e20 con 0x7f2a7c0ccff0 2024-01-27T02:44:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.393+0000 7f2a7bfff700 1 --2- 172.21.15.7:0/1917847609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a7c0ccff0 0x7f2a7c0cb1c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2a74009ec0 tx=0x7f2a74009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bce8e8164a1968f8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.393+0000 7f2a7a7fc700 1 -- 172.21.15.7:0/1917847609 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7400e070 con 0x7f2a7c0ccff0 2024-01-27T02:44:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.393+0000 7f2a7a7fc700 1 -- 172.21.15.7:0/1917847609 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7400bb40 con 0x7f2a7c0ccff0 2024-01-27T02:44:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.393+0000 7f2a7a7fc700 1 -- 172.21.15.7:0/1917847609 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a74012430 con 0x7f2a7c0ccff0 2024-01-27T02:44:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.393+0000 7f2a81f76700 1 -- 172.21.15.7:0/1917847609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a7c0ccff0 msgr2=0x7f2a7c0cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.393+0000 7f2a81f76700 1 --2- 172.21.15.7:0/1917847609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a7c0ccff0 0x7f2a7c0cb1c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2a74009ec0 tx=0x7f2a74009780 comp rx=0 tx=0).stop 2024-01-27T02:44:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.394+0000 7f2a81f76700 1 -- 172.21.15.7:0/1917847609 shutdown_connections 2024-01-27T02:44:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.394+0000 7f2a81f76700 1 --2- 172.21.15.7:0/1917847609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a7c0ccff0 0x7f2a7c0cb1c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.394+0000 7f2a81f76700 1 -- 172.21.15.7:0/1917847609 >> 172.21.15.7:0/1917847609 conn(0x7f2a7c063d60 msgr2=0x7f2a7c0ced10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.394+0000 7f2a81f76700 1 -- 172.21.15.7:0/1917847609 shutdown_connections 2024-01-27T02:44:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.394+0000 7f2a81f76700 1 -- 172.21.15.7:0/1917847609 wait complete. 2024-01-27T02:44:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.395+0000 7f2a81f76700 1 Processor -- start 2024-01-27T02:44:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.395+0000 7f2a81f76700 1 -- start start 2024-01-27T02:44:11.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.396+0000 7f2a81f76700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c0cad20 0x7f2a7c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:11.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.396+0000 7f2a81f76700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a7c0ccff0 0x7f2a7c144880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:11.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.396+0000 7f2a81f76700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a7c15b740 0x7f2a7c0c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:11.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.396+0000 7f2a81f76700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c144dc0 con 0x7f2a7c0cad20 2024-01-27T02:44:11.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.396+0000 7f2a81f76700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c0c8980 con 0x7f2a7c15b740 2024-01-27T02:44:11.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.396+0000 7f2a81f76700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c15dcb0 con 0x7f2a7c0ccff0 2024-01-27T02:44:11.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.396+0000 7f2a7b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c0cad20 0x7f2a7c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:11.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.396+0000 7f2a7b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c0cad20 0x7f2a7c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52066/0 (socket says 172.21.15.7:52066) 2024-01-27T02:44:11.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.396+0000 7f2a7b7fe700 1 -- 172.21.15.7:0/2116961845 learned_addr learned my addr 172.21.15.7:0/2116961845 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:11.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.396+0000 7f2a7bfff700 1 --2- 172.21.15.7:0/2116961845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a7c15b740 0x7f2a7c0c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:11.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.396+0000 7f2a7affd700 1 --2- 172.21.15.7:0/2116961845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a7c0ccff0 0x7f2a7c144880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:11.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.396+0000 7f2a7b7fe700 1 -- 172.21.15.7:0/2116961845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a7c0ccff0 msgr2=0x7f2a7c144880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:11.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.396+0000 7f2a7b7fe700 1 --2- 172.21.15.7:0/2116961845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a7c0ccff0 0x7f2a7c144880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:11.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.396+0000 7f2a7b7fe700 1 -- 172.21.15.7:0/2116961845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a7c15b740 msgr2=0x7f2a7c0c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:11.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.396+0000 7f2a7b7fe700 1 --2- 172.21.15.7:0/2116961845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a7c15b740 0x7f2a7c0c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:11.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.396+0000 7f2a7b7fe700 1 -- 172.21.15.7:0/2116961845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a74009a20 con 0x7f2a7c0cad20 2024-01-27T02:44:11.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.396+0000 7f2a7affd700 1 --2- 172.21.15.7:0/2116961845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a7c0ccff0 0x7f2a7c144880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:44:11.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.397+0000 7f2a7b7fe700 1 --2- 172.21.15.7:0/2116961845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c0cad20 0x7f2a7c15d5d0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00c580 tx=0x7f2a6c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:11.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.397+0000 7f2a637fe700 1 -- 172.21.15.7:0/2116961845 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c010510 con 0x7f2a7c0cad20 2024-01-27T02:44:11.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.397+0000 7f2a637fe700 1 -- 172.21.15.7:0/2116961845 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6c010670 con 0x7f2a7c0cad20 2024-01-27T02:44:11.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.397+0000 7f2a637fe700 1 -- 172.21.15.7:0/2116961845 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c010a30 con 0x7f2a7c0cad20 2024-01-27T02:44:11.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.397+0000 7f2a81f76700 1 -- 172.21.15.7:0/2116961845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7c15df90 con 0x7f2a7c0cad20 2024-01-27T02:44:11.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.397+0000 7f2a81f76700 1 -- 172.21.15.7:0/2116961845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7c15e550 con 0x7f2a7c0cad20 2024-01-27T02:44:11.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.398+0000 7f2a81f76700 1 -- 172.21.15.7:0/2116961845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a7c0c5da0 con 0x7f2a7c0cad20 2024-01-27T02:44:11.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.398+0000 7f2a637fe700 1 -- 172.21.15.7:0/2116961845 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2a6c01b400 con 0x7f2a7c0cad20 2024-01-27T02:44:11.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.399+0000 7f2a637fe700 1 --2- 172.21.15.7:0/2116961845 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2a68066f30 0x7f2a680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:11.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.399+0000 7f2a637fe700 1 -- 172.21.15.7:0/2116961845 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2a6c083de0 con 0x7f2a7c0cad20 2024-01-27T02:44:11.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.399+0000 7f2a7affd700 1 --2- 172.21.15.7:0/2116961845 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2a68066f30 0x7f2a680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:11.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.400+0000 7f2a7affd700 1 --2- 172.21.15.7:0/2116961845 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2a68066f30 0x7f2a680693e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0cdf20 tx=0x7f2a70009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:11.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.403+0000 7f2a637fe700 1 -- 172.21.15.7:0/2116961845 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a6c051350 con 0x7f2a7c0cad20 2024-01-27T02:44:11.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.609+0000 7f2a81f76700 1 -- 172.21.15.7:0/2116961845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a7c04c710 con 0x7f2a7c0cad20 2024-01-27T02:44:11.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.610+0000 7f2a637fe700 1 -- 172.21.15.7:0/2116961845 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v187) v1 ==== 74+0+12228 (secure 0 0 0) 0x7f2a6c054970 con 0x7f2a7c0cad20 2024-01-27T02:44:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:44:11.610 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":187,"fsid":"17e1bcca-e647-4a58-b0ac-7d39d8508814","created":"2024-01-27T02:31:50.016376+0000","modified":"2024-01-27T02:44:08.017381+0000","last_up_change":"2024-01-27T02:31:55.531473+0000","last_in_change":"2024-01-27T02:31:52.519857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-27T02:31:57.628744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":39319},{"type":"v1","addr":"172.21.15.7:6817","nonce":39319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":39319},{"type":"v1","addr":"172.21.15.7:6819","nonce":39319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":39319},{"type":"v1","addr":"172.21.15.7:6823","nonce":39319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":39319},{"type":"v1","addr":"172.21.15.7:6821","nonce":39319}]},"public_addr":"172.21.15.7:6817/39319","cluster_addr":"172.21.15.7:6819/39319","heartbeat_back_addr":"172.21.15.7:6823/39319","heartbeat_front_addr":"172.21.15.7:6821/39319","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":39309},{"type":"v1","addr":"172.21.15.7:6801","nonce":39309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":39309},{"type":"v1","addr":"172.21.15.7:6803","nonce":39309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":39309},{"type":"v1","addr":"172.21.15.7:6807","nonce":39309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":39309},{"type":"v1","addr":"172.21.15.7:6805","nonce":39309}]},"public_addr":"172.21.15.7:6801/39309","cluster_addr":"172.21.15.7:6803/39309","heartbeat_back_addr":"172.21.15.7:6807/39309","heartbeat_front_addr":"172.21.15.7:6805/39309","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":39325},{"type":"v1","addr":"172.21.15.7:6825","nonce":39325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":39325},{"type":"v1","addr":"172.21.15.7:6827","nonce":39325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":39325},{"type":"v1","addr":"172.21.15.7:6831","nonce":39325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":39325},{"type":"v1","addr":"172.21.15.7:6829","nonce":39325}]},"public_addr":"172.21.15.7:6825/39325","cluster_addr":"172.21.15.7:6827/39325","heartbeat_back_addr":"172.21.15.7:6831/39325","heartbeat_front_addr":"172.21.15.7:6829/39325","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":39310},{"type":"v1","addr":"172.21.15.7:6809","nonce":39310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":39310},{"type":"v1","addr":"172.21.15.7:6811","nonce":39310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":39310},{"type":"v1","addr":"172.21.15.7:6815","nonce":39310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":39310},{"type":"v1","addr":"172.21.15.7:6813","nonce":39310}]},"public_addr":"172.21.15.7:6809/39310","cluster_addr":"172.21.15.7:6811/39310","heartbeat_back_addr":"172.21.15.7:6815/39310","heartbeat_front_addr":"172.21.15.7:6813/39310","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6801","nonce":39374},{"type":"v1","addr":"172.21.15.188:6803","nonce":39374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6805","nonce":39374},{"type":"v1","addr":"172.21.15.188:6807","nonce":39374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":39374},{"type":"v1","addr":"172.21.15.188:6815","nonce":39374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":39374},{"type":"v1","addr":"172.21.15.188:6811","nonce":39374}]},"public_addr":"172.21.15.188:6803/39374","cluster_addr":"172.21.15.188:6807/39374","heartbeat_back_addr":"172.21.15.188:6815/39374","heartbeat_front_addr":"172.21.15.188:6811/39374","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":39373},{"type":"v1","addr":"172.21.15.188:6802","nonce":39373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":39373},{"type":"v1","addr":"172.21.15.188:6806","nonce":39373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":39373},{"type":"v1","addr":"172.21.15.188:6813","nonce":39373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":39373},{"type":"v1","addr":"172.21.15.188:6810","nonce":39373}]},"public_addr":"172.21.15.188:6802/39373","cluster_addr":"172.21.15.188:6806/39373","heartbeat_back_addr":"172.21.15.188:6813/39373","heartbeat_front_addr":"172.21.15.188:6810/39373","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":39412},{"type":"v1","addr":"172.21.15.188:6825","nonce":39412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":39412},{"type":"v1","addr":"172.21.15.188:6827","nonce":39412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":39412},{"type":"v1","addr":"172.21.15.188:6831","nonce":39412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":39412},{"type":"v1","addr":"172.21.15.188:6829","nonce":39412}]},"public_addr":"172.21.15.188:6825/39412","cluster_addr":"172.21.15.188:6827/39412","heartbeat_back_addr":"172.21.15.188:6831/39412","heartbeat_front_addr":"172.21.15.188:6829/39412","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":39375},{"type":"v1","addr":"172.21.15.188:6817","nonce":39375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":39375},{"type":"v1","addr":"172.21.15.188:6819","nonce":39375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":39375},{"type":"v1","addr":"172.21.15.188:6823","nonce":39375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":39375},{"type":"v1","addr":"172.21.15.188:6821","nonce":39375}]},"public_addr":"172.21.15.188:6817/39375","cluster_addr":"172.21.15.188:6819/39375","heartbeat_back_addr":"172.21.15.188:6823/39375","heartbeat_front_addr":"172.21.15.188:6821/39375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.678769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.691345+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.644306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.707270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.670376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.615210+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.694216+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-27T02:31:53.669154+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.188:6837/3853795510":"2024-01-28T02:44:04.843025+0000","172.21.15.188:6835/3853795510":"2024-01-28T02:44:04.843025+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-27T02:44:11.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.612+0000 7f2a81f76700 1 -- 172.21.15.7:0/2116961845 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2a68066f30 msgr2=0x7f2a680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:11.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.612+0000 7f2a81f76700 1 --2- 172.21.15.7:0/2116961845 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2a68066f30 0x7f2a680693e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0cdf20 tx=0x7f2a70009040 comp rx=0 tx=0).stop 2024-01-27T02:44:11.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.612+0000 7f2a81f76700 1 -- 172.21.15.7:0/2116961845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c0cad20 msgr2=0x7f2a7c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:11.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.612+0000 7f2a81f76700 1 --2- 172.21.15.7:0/2116961845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c0cad20 0x7f2a7c15d5d0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00c580 tx=0x7f2a6c007e10 comp rx=0 tx=0).stop 2024-01-27T02:44:11.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.612+0000 7f2a81f76700 1 -- 172.21.15.7:0/2116961845 shutdown_connections 2024-01-27T02:44:11.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.612+0000 7f2a81f76700 1 --2- 172.21.15.7:0/2116961845 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2a68066f30 0x7f2a680693e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:11.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.612+0000 7f2a81f76700 1 --2- 172.21.15.7:0/2116961845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a7c15b740 0x7f2a7c0c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:11.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.612+0000 7f2a81f76700 1 --2- 172.21.15.7:0/2116961845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c0cad20 0x7f2a7c15d5d0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:11.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.612+0000 7f2a81f76700 1 --2- 172.21.15.7:0/2116961845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a7c0ccff0 0x7f2a7c144880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:11.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.612+0000 7f2a81f76700 1 -- 172.21.15.7:0/2116961845 >> 172.21.15.7:0/2116961845 conn(0x7f2a7c063d60 msgr2=0x7f2a7c1548f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:11.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.612+0000 7f2a81f76700 1 -- 172.21.15.7:0/2116961845 shutdown_connections 2024-01-27T02:44:11.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:11.612+0000 7f2a81f76700 1 -- 172.21.15.7:0/2116961845 wait complete. 2024-01-27T02:44:12.627 INFO:tasks.ceph:Scrubbing osd.0 2024-01-27T02:44:12.627 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-01-27T02:44:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.770+0000 7fa0e5de4700 1 Processor -- start 2024-01-27T02:44:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.770+0000 7fa0e5de4700 1 -- start start 2024-01-27T02:44:12.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.771+0000 7fa0e5de4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e013ad50 0x7fa0e0159070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:12.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.771+0000 7fa0e5de4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa0e015bb40 0x7fa0e015bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:12.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.771+0000 7fa0e5de4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0e0158be0 con 0x7fa0e0063580 2024-01-27T02:44:12.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.771+0000 7fa0e5de4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e00c8f70 con 0x7fa0e015bb40 2024-01-27T02:44:12.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.771+0000 7fa0e5de4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e015c600 con 0x7fa0e013ad50 2024-01-27T02:44:12.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.771+0000 7fa0deffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa0e015bb40 0x7fa0e015bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:12.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.771+0000 7fa0deffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa0e015bb40 0x7fa0e015bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52076/0 (socket says 172.21.15.7:52076) 2024-01-27T02:44:12.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.771+0000 7fa0dffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa0e0063580 0x7fa0e00c8270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46908/0 (socket says 172.21.15.7:46908) 2024-01-27T02:44:12.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.771+0000 7fa0deffd700 1 -- 172.21.15.7:0/947709217 learned_addr learned my addr 172.21.15.7:0/947709217 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:12.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.772+0000 7fa0df7fe700 1 --2- 172.21.15.7:0/947709217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e013ad50 0x7fa0e0159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:12.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.772+0000 7fa0deffd700 1 -- 172.21.15.7:0/947709217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e013ad50 msgr2=0x7fa0e0159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:12.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.772+0000 7fa0deffd700 1 --2- 172.21.15.7:0/947709217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e013ad50 0x7fa0e0159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:12.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.772+0000 7fa0deffd700 1 -- 172.21.15.7:0/947709217 >> v1:172.21.15.7:6789/0 conn(0x7fa0e0063580 legacy=0x7fa0e00c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:12.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.772+0000 7fa0deffd700 1 -- 172.21.15.7:0/947709217 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e015c8e0 con 0x7fa0e015bb40 2024-01-27T02:44:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.772+0000 7fa0de7fc700 1 -- 172.21.15.7:0/947709217 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691177450 0 0) 0x7fa0e0158be0 con 0x7fa0e0063580 2024-01-27T02:44:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.772+0000 7fa0deffd700 1 --2- 172.21.15.7:0/947709217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0e015bb40 0x7fa0e015bf70 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fa0c80099b0 tx=0x7fa0c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05301df51f75fb2 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.772+0000 7fa0de7fc700 1 -- 172.21.15.7:0/947709217 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c800b860 con 0x7fa0e015bb40 2024-01-27T02:44:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.772+0000 7fa0de7fc700 1 -- 172.21.15.7:0/947709217 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c800b9c0 con 0x7fa0e015bb40 2024-01-27T02:44:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.772+0000 7fa0de7fc700 1 -- 172.21.15.7:0/947709217 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c800bd60 con 0x7fa0e015bb40 2024-01-27T02:44:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.773+0000 7fa0e5de4700 1 -- 172.21.15.7:0/947709217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0e015bb40 msgr2=0x7fa0e015bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.773+0000 7fa0e5de4700 1 --2- 172.21.15.7:0/947709217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0e015bb40 0x7fa0e015bf70 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fa0c80099b0 tx=0x7fa0c8009250 comp rx=0 tx=0).stop 2024-01-27T02:44:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.773+0000 7fa0e5de4700 1 -- 172.21.15.7:0/947709217 shutdown_connections 2024-01-27T02:44:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.773+0000 7fa0e5de4700 1 --2- 172.21.15.7:0/947709217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e013ad50 0x7fa0e0159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.773+0000 7fa0e5de4700 1 --2- 172.21.15.7:0/947709217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0e015bb40 0x7fa0e015bf70 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.773+0000 7fa0e5de4700 1 -- 172.21.15.7:0/947709217 >> 172.21.15.7:0/947709217 conn(0x7fa0e00cdc30 msgr2=0x7fa0e00ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.773+0000 7fa0e5de4700 1 -- 172.21.15.7:0/947709217 shutdown_connections 2024-01-27T02:44:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.773+0000 7fa0e5de4700 1 -- 172.21.15.7:0/947709217 wait complete. 2024-01-27T02:44:12.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.774+0000 7fa0e5de4700 1 Processor -- start 2024-01-27T02:44:12.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.774+0000 7fa0e5de4700 1 -- start start 2024-01-27T02:44:12.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.775+0000 7fa0e5de4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0e0063580 0x7fa0e00c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:12.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.775+0000 7fa0e5de4700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e013ad50 0x7fa0e00b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:12.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.775+0000 7fa0e5de4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e015bb40 0x7fa0e0157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:12.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.775+0000 7fa0e5de4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e0156320 con 0x7fa0e0063580 2024-01-27T02:44:12.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.775+0000 7fa0e5de4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e0156570 con 0x7fa0e015bb40 2024-01-27T02:44:12.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.775+0000 7fa0e5de4700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e0156850 con 0x7fa0e013ad50 2024-01-27T02:44:12.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.775+0000 7fa0df7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0e0063580 0x7fa0e00c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:12.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.775+0000 7fa0df7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0e0063580 0x7fa0e00c7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52086/0 (socket says 172.21.15.7:52086) 2024-01-27T02:44:12.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.775+0000 7fa0df7fe700 1 -- 172.21.15.7:0/245733238 learned_addr learned my addr 172.21.15.7:0/245733238 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:12.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.775+0000 7fa0deffd700 1 --2- 172.21.15.7:0/245733238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e013ad50 0x7fa0e00b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:12.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.775+0000 7fa0dffff700 1 --2- 172.21.15.7:0/245733238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e015bb40 0x7fa0e0157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.776+0000 7fa0df7fe700 1 -- 172.21.15.7:0/245733238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e013ad50 msgr2=0x7fa0e00b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.776+0000 7fa0df7fe700 1 --2- 172.21.15.7:0/245733238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e013ad50 0x7fa0e00b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.776+0000 7fa0df7fe700 1 -- 172.21.15.7:0/245733238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e015bb40 msgr2=0x7fa0e0157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.776+0000 7fa0df7fe700 1 --2- 172.21.15.7:0/245733238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e015bb40 0x7fa0e0157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.776+0000 7fa0df7fe700 1 -- 172.21.15.7:0/245733238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c80094f0 con 0x7fa0e0063580 2024-01-27T02:44:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.776+0000 7fa0df7fe700 1 --2- 172.21.15.7:0/245733238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0e0063580 0x7fa0e00c7500 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fa0d000d890 tx=0x7fa0d0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.776+0000 7fa0d77fe700 1 -- 172.21.15.7:0/245733238 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d000b600 con 0x7fa0e0063580 2024-01-27T02:44:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.776+0000 7fa0d77fe700 1 -- 172.21.15.7:0/245733238 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d000b760 con 0x7fa0e0063580 2024-01-27T02:44:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.776+0000 7fa0e5de4700 1 -- 172.21.15.7:0/245733238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e0156b30 con 0x7fa0e0063580 2024-01-27T02:44:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.776+0000 7fa0d77fe700 1 -- 172.21.15.7:0/245733238 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d000bae0 con 0x7fa0e0063580 2024-01-27T02:44:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.776+0000 7fa0e5de4700 1 -- 172.21.15.7:0/245733238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e01570f0 con 0x7fa0e0063580 2024-01-27T02:44:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.777+0000 7fa0e5de4700 1 -- 172.21.15.7:0/245733238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa0e004c710 con 0x7fa0e0063580 2024-01-27T02:44:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.778+0000 7fa0d77fe700 1 -- 172.21.15.7:0/245733238 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa0d001e460 con 0x7fa0e0063580 2024-01-27T02:44:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.778+0000 7fa0d77fe700 1 --2- 172.21.15.7:0/245733238 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa0c0066f30 0x7fa0c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.778+0000 7fa0d77fe700 1 -- 172.21.15.7:0/245733238 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa0d0084910 con 0x7fa0e0063580 2024-01-27T02:44:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.778+0000 7fa0d77fe700 1 --2- 172.21.15.7:0/245733238 >> [v2:172.21.15.7:6816/39319,v1:172.21.15.7:6817/39319] conn(0x7fa0c006cd80 0x7fa0c006f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.778+0000 7fa0d77fe700 1 -- 172.21.15.7:0/245733238 --> [v2:172.21.15.7:6816/39319,v1:172.21.15.7:6817/39319] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0c006f860 con 0x7fa0c006cd80 2024-01-27T02:44:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.778+0000 7fa0d77fe700 1 -- 172.21.15.7:0/245733238 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=187) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0d0089240 con 0x7fa0e0063580 2024-01-27T02:44:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.778+0000 7fa0dffff700 1 --2- 172.21.15.7:0/245733238 >> [v2:172.21.15.7:6816/39319,v1:172.21.15.7:6817/39319] conn(0x7fa0c006cd80 0x7fa0c006f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.778+0000 7fa0deffd700 1 --2- 172.21.15.7:0/245733238 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa0c0066f30 0x7fa0c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.779+0000 7fa0dffff700 1 --2- 172.21.15.7:0/245733238 >> [v2:172.21.15.7:6816/39319,v1:172.21.15.7:6817/39319] conn(0x7fa0c006cd80 0x7fa0c006f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:12.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.779+0000 7fa0deffd700 1 --2- 172.21.15.7:0/245733238 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa0c0066f30 0x7fa0c00693e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa0c80099b0 tx=0x7fa0c8017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:12.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.779+0000 7fa0d77fe700 1 -- 172.21.15.7:0/245733238 <== osd.0 v2:172.21.15.7:6816/39319 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7fa0c006f860 con 0x7fa0c006cd80 2024-01-27T02:44:12.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.793+0000 7fa0e5de4700 1 -- 172.21.15.7:0/245733238 --> [v2:172.21.15.7:6816/39319,v1:172.21.15.7:6817/39319] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa0e006f650 con 0x7fa0c006cd80 2024-01-27T02:44:12.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.793+0000 7fa0d77fe700 1 -- 172.21.15.7:0/245733238 <== osd.0 v2:172.21.15.7:6816/39319 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7fa0e006f650 con 0x7fa0c006cd80 2024-01-27T02:44:12.795 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:44:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-01-27T02:44:12.796 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:44:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.793+0000 7fa0e5de4700 1 -- 172.21.15.7:0/245733238 >> [v2:172.21.15.7:6816/39319,v1:172.21.15.7:6817/39319] conn(0x7fa0c006cd80 msgr2=0x7fa0c006f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.793+0000 7fa0e5de4700 1 --2- 172.21.15.7:0/245733238 >> [v2:172.21.15.7:6816/39319,v1:172.21.15.7:6817/39319] conn(0x7fa0c006cd80 0x7fa0c006f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.793+0000 7fa0e5de4700 1 -- 172.21.15.7:0/245733238 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa0c0066f30 msgr2=0x7fa0c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.793+0000 7fa0e5de4700 1 --2- 172.21.15.7:0/245733238 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fa0c0066f30 0x7fa0c00693e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa0c80099b0 tx=0x7fa0c8017000 comp rx=0 tx=0).stop 2024-01-27T02:44:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.793+0000 7fa0e5de4700 1 -- 172.21.15.7:0/245733238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0e0063580 msgr2=0x7fa0e00c7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:12.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.793+0000 7fa0e5de4700 1 --2- 172.21.15.7:0/245733238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0e0063580 0x7fa0e00c7500 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fa0d000d890 tx=0x7fa0d0008040 comp rx=0 tx=0).stop 2024-01-27T02:44:12.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.793+0000 7fa0dffff700 1 -- 172.21.15.7:0/245733238 reap_dead start 2024-01-27T02:44:12.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.793+0000 7fa0e5de4700 1 -- 172.21.15.7:0/245733238 shutdown_connections 2024-01-27T02:44:12.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.793+0000 7fa0e5de4700 1 -- 172.21.15.7:0/245733238 >> 172.21.15.7:0/245733238 conn(0x7fa0e00cdc30 msgr2=0x7fa0e00cacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:12.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.794+0000 7fa0e5de4700 1 -- 172.21.15.7:0/245733238 shutdown_connections 2024-01-27T02:44:12.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.794+0000 7fa0e5de4700 1 -- 172.21.15.7:0/245733238 wait complete. 2024-01-27T02:44:12.810 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-01-27T02:44:12.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.943+0000 7f42cd68d700 1 Processor -- start 2024-01-27T02:44:12.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.944+0000 7f42cd68d700 1 -- start start 2024-01-27T02:44:12.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.944+0000 7f42cd68d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f42c80b00c0 0x7f42c80b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:12.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.944+0000 7f42cd68d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42c80c83c0 0x7f42c806aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:12.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.944+0000 7f42cd68d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42c806e1d0 con 0x7f42c8063580 2024-01-27T02:44:12.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.944+0000 7f42cd68d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c80ca620 con 0x7f42c80c83c0 2024-01-27T02:44:12.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.944+0000 7f42cd68d700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c815da70 con 0x7f42c80b00c0 2024-01-27T02:44:12.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.945+0000 7f42c67fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42c80c83c0 0x7f42c806aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:12.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.945+0000 7f42c77fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f42c8063580 0x7f42c806e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46910/0 (socket says 172.21.15.7:46910) 2024-01-27T02:44:12.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.945+0000 7f42c77fe700 1 -- 172.21.15.7:0/2637692053 learned_addr learned my addr 172.21.15.7:0/2637692053 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:12.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.945+0000 7f42c6ffd700 1 --2- 172.21.15.7:0/2637692053 >> v2:172.21.15.188:3300/0 conn(0x7f42c80b00c0 0x7f42c80b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:12.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.945+0000 7f42c67fc700 1 -- 172.21.15.7:0/2637692053 >> v2:172.21.15.188:3300/0 conn(0x7f42c80b00c0 msgr2=0x7f42c80b09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:12.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.945+0000 7f42c67fc700 1 --2- 172.21.15.7:0/2637692053 >> v2:172.21.15.188:3300/0 conn(0x7f42c80b00c0 0x7f42c80b09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:12.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.945+0000 7f42c67fc700 1 -- 172.21.15.7:0/2637692053 >> v1:172.21.15.7:6789/0 conn(0x7f42c8063580 legacy=0x7f42c806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:12.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.945+0000 7f42c5ffb700 1 -- 172.21.15.7:0/2637692053 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3726350087 0 0) 0x7f42c806e1d0 con 0x7f42c8063580 2024-01-27T02:44:12.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.945+0000 7f42c67fc700 1 -- 172.21.15.7:0/2637692053 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c806e4b0 con 0x7f42c80c83c0 2024-01-27T02:44:12.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.945+0000 7f42c67fc700 1 --2- 172.21.15.7:0/2637692053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42c80c83c0 0x7f42c806aec0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f42c806dc90 tx=0x7f42b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628272cc3a754042 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:12.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.945+0000 7f42c5ffb700 1 -- 172.21.15.7:0/2637692053 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b0009e80 con 0x7f42c80c83c0 2024-01-27T02:44:12.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.945+0000 7f42c5ffb700 1 -- 172.21.15.7:0/2637692053 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b000b820 con 0x7f42c80c83c0 2024-01-27T02:44:12.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.945+0000 7f42c5ffb700 1 -- 172.21.15.7:0/2637692053 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b000bba0 con 0x7f42c80c83c0 2024-01-27T02:44:12.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.946+0000 7f42cd68d700 1 -- 172.21.15.7:0/2637692053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42c80c83c0 msgr2=0x7f42c806aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:12.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.946+0000 7f42cd68d700 1 --2- 172.21.15.7:0/2637692053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42c80c83c0 0x7f42c806aec0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f42c806dc90 tx=0x7f42b00091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:12.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.946+0000 7f42cd68d700 1 -- 172.21.15.7:0/2637692053 shutdown_connections 2024-01-27T02:44:12.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.946+0000 7f42cd68d700 1 --2- 172.21.15.7:0/2637692053 >> v2:172.21.15.188:3300/0 conn(0x7f42c80b00c0 0x7f42c80b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:12.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.946+0000 7f42cd68d700 1 --2- 172.21.15.7:0/2637692053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42c80c83c0 0x7f42c806aec0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:12.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.946+0000 7f42cd68d700 1 -- 172.21.15.7:0/2637692053 >> 172.21.15.7:0/2637692053 conn(0x7f42c80679b0 msgr2=0x7f42c8067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:12.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.946+0000 7f42cd68d700 1 -- 172.21.15.7:0/2637692053 shutdown_connections 2024-01-27T02:44:12.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.946+0000 7f42cd68d700 1 -- 172.21.15.7:0/2637692053 wait complete. 2024-01-27T02:44:12.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.947+0000 7f42cd68d700 1 Processor -- start 2024-01-27T02:44:12.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.948+0000 7f42cd68d700 1 -- start start 2024-01-27T02:44:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.948+0000 7f42cd68d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42c8063580 0x7f42c814bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.948+0000 7f42cd68d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42c80b00c0 0x7f42c80c3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.948+0000 7f42cd68d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42c80c83c0 0x7f42c80c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.948+0000 7f42cd68d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c80c1490 con 0x7f42c80c83c0 2024-01-27T02:44:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.948+0000 7f42cd68d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c80c6160 con 0x7f42c8063580 2024-01-27T02:44:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.948+0000 7f42cd68d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c80c58c0 con 0x7f42c80b00c0 2024-01-27T02:44:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.948+0000 7f42c77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42c80c83c0 0x7f42c80c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:12.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.949+0000 7f42c77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42c80c83c0 0x7f42c80c3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52110/0 (socket says 172.21.15.7:52110) 2024-01-27T02:44:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.949+0000 7f42c77fe700 1 -- 172.21.15.7:0/53068033 learned_addr learned my addr 172.21.15.7:0/53068033 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.949+0000 7f42c6ffd700 1 --2- 172.21.15.7:0/53068033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42c8063580 0x7f42c814bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.949+0000 7f42c67fc700 1 --2- 172.21.15.7:0/53068033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42c80b00c0 0x7f42c80c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.949+0000 7f42c77fe700 1 -- 172.21.15.7:0/53068033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42c80b00c0 msgr2=0x7f42c80c3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.949+0000 7f42c77fe700 1 --2- 172.21.15.7:0/53068033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42c80b00c0 0x7f42c80c3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.949+0000 7f42c77fe700 1 -- 172.21.15.7:0/53068033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42c8063580 msgr2=0x7f42c814bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.949+0000 7f42c77fe700 1 --2- 172.21.15.7:0/53068033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42c8063580 0x7f42c814bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.949+0000 7f42c77fe700 1 -- 172.21.15.7:0/53068033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b0009420 con 0x7f42c80c83c0 2024-01-27T02:44:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.949+0000 7f42c6ffd700 1 --2- 172.21.15.7:0/53068033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42c8063580 0x7f42c814bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:44:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.949+0000 7f42c77fe700 1 --2- 172.21.15.7:0/53068033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42c80c83c0 0x7f42c80c3bc0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f42c000cbf0 tx=0x7f42c00019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.949+0000 7f42beffd700 1 -- 172.21.15.7:0/53068033 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c000a9c0 con 0x7f42c80c83c0 2024-01-27T02:44:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.949+0000 7f42beffd700 1 -- 172.21.15.7:0/53068033 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c000ab20 con 0x7f42c80c83c0 2024-01-27T02:44:12.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.949+0000 7f42cd68d700 1 -- 172.21.15.7:0/53068033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c80c5ba0 con 0x7f42c80c83c0 2024-01-27T02:44:12.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.949+0000 7f42beffd700 1 -- 172.21.15.7:0/53068033 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c00070f0 con 0x7f42c80c83c0 2024-01-27T02:44:12.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.949+0000 7f42cd68d700 1 -- 172.21.15.7:0/53068033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c814c480 con 0x7f42c80c83c0 2024-01-27T02:44:12.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.950+0000 7f42cd68d700 1 -- 172.21.15.7:0/53068033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42c80c1600 con 0x7f42c80c83c0 2024-01-27T02:44:12.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.951+0000 7f42beffd700 1 -- 172.21.15.7:0/53068033 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f42c0007520 con 0x7f42c80c83c0 2024-01-27T02:44:12.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.951+0000 7f42beffd700 1 --2- 172.21.15.7:0/53068033 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f42a8066f30 0x7f42a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:12.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.951+0000 7f42beffd700 1 -- 172.21.15.7:0/53068033 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f42c0084610 con 0x7f42c80c83c0 2024-01-27T02:44:12.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.951+0000 7f42c6ffd700 1 --2- 172.21.15.7:0/53068033 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f42a8066f30 0x7f42a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:12.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.952+0000 7f42c6ffd700 1 --2- 172.21.15.7:0/53068033 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f42a8066f30 0x7f42a80693e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f42b8006fd0 tx=0x7f42b8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:12.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:12.955+0000 7f42beffd700 1 -- 172.21.15.7:0/53068033 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f42c80c1600 con 0x7f42c80c83c0 2024-01-27T02:44:13.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.160+0000 7f42cd68d700 1 -- 172.21.15.7:0/53068033 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f42c80c15d0 con 0x7f42a8066f30 2024-01-27T02:44:13.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.161+0000 7f42beffd700 1 -- 172.21.15.7:0/53068033 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f42c80c15d0 con 0x7f42a8066f30 2024-01-27T02:44:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.163+0000 7f42cd68d700 1 -- 172.21.15.7:0/53068033 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f42a8066f30 msgr2=0x7f42a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.163+0000 7f42cd68d700 1 --2- 172.21.15.7:0/53068033 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f42a8066f30 0x7f42a80693e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f42b8006fd0 tx=0x7f42b8008040 comp rx=0 tx=0).stop 2024-01-27T02:44:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.163+0000 7f42cd68d700 1 -- 172.21.15.7:0/53068033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42c80c83c0 msgr2=0x7f42c80c3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.163+0000 7f42cd68d700 1 --2- 172.21.15.7:0/53068033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42c80c83c0 0x7f42c80c3bc0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f42c000cbf0 tx=0x7f42c00019b0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.163+0000 7f42cd68d700 1 -- 172.21.15.7:0/53068033 shutdown_connections 2024-01-27T02:44:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.163+0000 7f42cd68d700 1 --2- 172.21.15.7:0/53068033 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f42a8066f30 0x7f42a80693e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.163+0000 7f42cd68d700 1 --2- 172.21.15.7:0/53068033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42c80c83c0 0x7f42c80c3bc0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.163+0000 7f42cd68d700 1 --2- 172.21.15.7:0/53068033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42c8063580 0x7f42c814bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.163+0000 7f42cd68d700 1 --2- 172.21.15.7:0/53068033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42c80b00c0 0x7f42c80c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.163+0000 7f42cd68d700 1 -- 172.21.15.7:0/53068033 >> 172.21.15.7:0/53068033 conn(0x7f42c80679b0 msgr2=0x7f42c815d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:13.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.163+0000 7f42cd68d700 1 -- 172.21.15.7:0/53068033 shutdown_connections 2024-01-27T02:44:13.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.163+0000 7f42cd68d700 1 -- 172.21.15.7:0/53068033 wait complete. 2024-01-27T02:44:13.166 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 0 to deep-scrub 2024-01-27T02:44:13.178 INFO:tasks.ceph:Scrubbing osd.1 2024-01-27T02:44:13.179 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-01-27T02:44:13.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.314+0000 7f2687aa3700 1 Processor -- start 2024-01-27T02:44:13.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.314+0000 7f2687aa3700 1 -- start start 2024-01-27T02:44:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.315+0000 7f2687aa3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f268006f190 0x7f2680063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.315+0000 7f2687aa3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2680063ac0 0x7f268015eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.315+0000 7f2687aa3700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26800b14e0 con 0x7f2680147f30 2024-01-27T02:44:13.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.315+0000 7f2687aa3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26800b2260 con 0x7f2680063ac0 2024-01-27T02:44:13.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.315+0000 7f2687aa3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2680063f40 con 0x7f268006f190 2024-01-27T02:44:13.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.315+0000 7f2686040700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2680063ac0 0x7f268015eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:13.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.315+0000 7f2686040700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2680063ac0 0x7f268015eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52124/0 (socket says 172.21.15.7:52124) 2024-01-27T02:44:13.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.315+0000 7f2686040700 1 -- 172.21.15.7:0/2765664851 learned_addr learned my addr 172.21.15.7:0/2765664851 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:13.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.315+0000 7f268503e700 1 --2- 172.21.15.7:0/2765664851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f268006f190 0x7f2680063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:13.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.315+0000 7f2686040700 1 -- 172.21.15.7:0/2765664851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f268006f190 msgr2=0x7f2680063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:13.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.315+0000 7f2686040700 1 --2- 172.21.15.7:0/2765664851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f268006f190 0x7f2680063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.315+0000 7f2686040700 1 -- 172.21.15.7:0/2765664851 >> v1:172.21.15.188:6789/0 conn(0x7f2680147f30 legacy=0x7f26800b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:13.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.315+0000 7f2686040700 1 -- 172.21.15.7:0/2765664851 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268006ed00 con 0x7f2680063ac0 2024-01-27T02:44:13.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.315+0000 7f2686040700 1 --2- 172.21.15.7:0/2765664851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2680063ac0 0x7f268015eae0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f2678009f20 tx=0x7f2678009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c2ad3036e968f8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:13.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.316+0000 7f268483d700 1 -- 172.21.15.7:0/2765664851 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267800e070 con 0x7f2680063ac0 2024-01-27T02:44:13.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.316+0000 7f268483d700 1 -- 172.21.15.7:0/2765664851 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267800ba80 con 0x7f2680063ac0 2024-01-27T02:44:13.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.316+0000 7f268483d700 1 -- 172.21.15.7:0/2765664851 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26780092e0 con 0x7f2680063ac0 2024-01-27T02:44:13.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.316+0000 7f2687aa3700 1 -- 172.21.15.7:0/2765664851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2680063ac0 msgr2=0x7f268015eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:13.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.316+0000 7f2687aa3700 1 --2- 172.21.15.7:0/2765664851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2680063ac0 0x7f268015eae0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f2678009f20 tx=0x7f2678009840 comp rx=0 tx=0).stop 2024-01-27T02:44:13.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.316+0000 7f2687aa3700 1 -- 172.21.15.7:0/2765664851 shutdown_connections 2024-01-27T02:44:13.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.316+0000 7f2687aa3700 1 --2- 172.21.15.7:0/2765664851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2680063ac0 0x7f268015eae0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.316+0000 7f2687aa3700 1 --2- 172.21.15.7:0/2765664851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f268006f190 0x7f2680063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.316+0000 7f2687aa3700 1 -- 172.21.15.7:0/2765664851 >> 172.21.15.7:0/2765664851 conn(0x7f2680068480 msgr2=0x7f268006be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:13.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.316+0000 7f2687aa3700 1 -- 172.21.15.7:0/2765664851 shutdown_connections 2024-01-27T02:44:13.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.316+0000 7f2687aa3700 1 -- 172.21.15.7:0/2765664851 wait complete. 2024-01-27T02:44:13.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.318+0000 7f2687aa3700 1 Processor -- start 2024-01-27T02:44:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.318+0000 7f2687aa3700 1 -- start start 2024-01-27T02:44:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.318+0000 7f2687aa3700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2680063ac0 0x7f26801633b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.318+0000 7f2687aa3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268006f190 0x7f268015f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.318+0000 7f2687aa3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2680147f30 0x7f26800acb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.318+0000 7f2687aa3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26800ad0b0 con 0x7f268006f190 2024-01-27T02:44:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.318+0000 7f2687aa3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26800ad350 con 0x7f2680147f30 2024-01-27T02:44:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.318+0000 7f2687aa3700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268015e930 con 0x7f2680063ac0 2024-01-27T02:44:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.319+0000 7f268503e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268006f190 0x7f268015f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.319+0000 7f268503e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268006f190 0x7f268015f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52128/0 (socket says 172.21.15.7:52128) 2024-01-27T02:44:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.319+0000 7f268503e700 1 -- 172.21.15.7:0/525830813 learned_addr learned my addr 172.21.15.7:0/525830813 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.319+0000 7f268583f700 1 --2- 172.21.15.7:0/525830813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2680063ac0 0x7f26801633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.319+0000 7f2686040700 1 --2- 172.21.15.7:0/525830813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2680147f30 0x7f26800acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.319+0000 7f268503e700 1 -- 172.21.15.7:0/525830813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2680063ac0 msgr2=0x7f26801633b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.319+0000 7f268503e700 1 --2- 172.21.15.7:0/525830813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2680063ac0 0x7f26801633b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.319+0000 7f268503e700 1 -- 172.21.15.7:0/525830813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2680147f30 msgr2=0x7f26800acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.319+0000 7f268503e700 1 --2- 172.21.15.7:0/525830813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2680147f30 0x7f26800acb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.319+0000 7f268503e700 1 -- 172.21.15.7:0/525830813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2678009ae0 con 0x7f268006f190 2024-01-27T02:44:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.319+0000 7f268503e700 1 --2- 172.21.15.7:0/525830813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268006f190 0x7f268015f6f0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f267c00d890 tx=0x7f267c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.319+0000 7f2675ffb700 1 -- 172.21.15.7:0/525830813 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c0125f0 con 0x7f268006f190 2024-01-27T02:44:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.319+0000 7f2675ffb700 1 -- 172.21.15.7:0/525830813 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c012750 con 0x7f268006f190 2024-01-27T02:44:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.319+0000 7f2675ffb700 1 -- 172.21.15.7:0/525830813 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c012af0 con 0x7f268006f190 2024-01-27T02:44:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.319+0000 7f2687aa3700 1 -- 172.21.15.7:0/525830813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268015ec10 con 0x7f268006f190 2024-01-27T02:44:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.320+0000 7f2687aa3700 1 -- 172.21.15.7:0/525830813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268015f1d0 con 0x7f268006f190 2024-01-27T02:44:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.320+0000 7f2687aa3700 1 -- 172.21.15.7:0/525830813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f26800638d0 con 0x7f268006f190 2024-01-27T02:44:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.321+0000 7f2675ffb700 1 -- 172.21.15.7:0/525830813 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f267c01e460 con 0x7f268006f190 2024-01-27T02:44:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.321+0000 7f2675ffb700 1 --2- 172.21.15.7:0/525830813 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2664066f30 0x7f26640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.322+0000 7f2675ffb700 1 -- 172.21.15.7:0/525830813 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f267c0847a0 con 0x7f268006f190 2024-01-27T02:44:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.322+0000 7f2675ffb700 1 --2- 172.21.15.7:0/525830813 >> [v2:172.21.15.7:6800/39309,v1:172.21.15.7:6801/39309] conn(0x7f266406cd80 0x7f266406f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.322+0000 7f2675ffb700 1 -- 172.21.15.7:0/525830813 --> [v2:172.21.15.7:6800/39309,v1:172.21.15.7:6801/39309] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f266406f860 con 0x7f266406cd80 2024-01-27T02:44:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.322+0000 7f2675ffb700 1 -- 172.21.15.7:0/525830813 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=187) v2 ==== 24+0+0 (secure 0 0 0) 0x7f267c086de0 con 0x7f268006f190 2024-01-27T02:44:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.322+0000 7f268583f700 1 --2- 172.21.15.7:0/525830813 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2664066f30 0x7f26640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.322+0000 7f2686040700 1 --2- 172.21.15.7:0/525830813 >> [v2:172.21.15.7:6800/39309,v1:172.21.15.7:6801/39309] conn(0x7f266406cd80 0x7f266406f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.322+0000 7f2686040700 1 --2- 172.21.15.7:0/525830813 >> [v2:172.21.15.7:6800/39309,v1:172.21.15.7:6801/39309] conn(0x7f266406cd80 0x7f266406f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.322+0000 7f268583f700 1 --2- 172.21.15.7:0/525830813 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2664066f30 0x7f26640693e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f26700039e0 tx=0x7f2670007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.323+0000 7f2675ffb700 1 -- 172.21.15.7:0/525830813 <== osd.1 v2:172.21.15.7:6800/39309 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f266406f860 con 0x7f266406cd80 2024-01-27T02:44:13.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.336+0000 7f2687aa3700 1 -- 172.21.15.7:0/525830813 --> [v2:172.21.15.7:6800/39309,v1:172.21.15.7:6801/39309] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f268004c710 con 0x7f266406cd80 2024-01-27T02:44:13.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.337+0000 7f2675ffb700 1 -- 172.21.15.7:0/525830813 <== osd.1 v2:172.21.15.7:6800/39309 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7f268004c710 con 0x7f266406cd80 2024-01-27T02:44:13.339 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:44:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-01-27T02:44:13.339 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:44:13.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.337+0000 7f2687aa3700 1 -- 172.21.15.7:0/525830813 >> [v2:172.21.15.7:6800/39309,v1:172.21.15.7:6801/39309] conn(0x7f266406cd80 msgr2=0x7f266406f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:13.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.337+0000 7f2687aa3700 1 --2- 172.21.15.7:0/525830813 >> [v2:172.21.15.7:6800/39309,v1:172.21.15.7:6801/39309] conn(0x7f266406cd80 0x7f266406f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.337+0000 7f2687aa3700 1 -- 172.21.15.7:0/525830813 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2664066f30 msgr2=0x7f26640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:13.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.337+0000 7f2687aa3700 1 --2- 172.21.15.7:0/525830813 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f2664066f30 0x7f26640693e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f26700039e0 tx=0x7f2670007ec0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.337+0000 7f2687aa3700 1 -- 172.21.15.7:0/525830813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268006f190 msgr2=0x7f268015f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:13.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.337+0000 7f2687aa3700 1 --2- 172.21.15.7:0/525830813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268006f190 0x7f268015f6f0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f267c00d890 tx=0x7f267c008040 comp rx=0 tx=0).stop 2024-01-27T02:44:13.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.337+0000 7f2686040700 1 -- 172.21.15.7:0/525830813 reap_dead start 2024-01-27T02:44:13.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.337+0000 7f2687aa3700 1 -- 172.21.15.7:0/525830813 shutdown_connections 2024-01-27T02:44:13.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.337+0000 7f2687aa3700 1 -- 172.21.15.7:0/525830813 >> 172.21.15.7:0/525830813 conn(0x7f2680068480 msgr2=0x7f2680144b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:13.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.337+0000 7f2687aa3700 1 -- 172.21.15.7:0/525830813 shutdown_connections 2024-01-27T02:44:13.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.337+0000 7f2687aa3700 1 -- 172.21.15.7:0/525830813 wait complete. 2024-01-27T02:44:13.352 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-01-27T02:44:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.483+0000 7f0bc983a700 1 Processor -- start 2024-01-27T02:44:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.483+0000 7f0bc983a700 1 -- start start 2024-01-27T02:44:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.484+0000 7f0bc983a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0bc415e4e0 0x7f0bc40b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.484+0000 7f0bc983a700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0bc415b8e0 0x7f0bc40ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.484+0000 7f0bc983a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc40b3900 con 0x7f0bc40b2ff0 2024-01-27T02:44:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.484+0000 7f0bc983a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc40ce7e0 con 0x7f0bc415e4e0 2024-01-27T02:44:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.484+0000 7f0bc983a700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc415ebe0 con 0x7f0bc415b8e0 2024-01-27T02:44:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.484+0000 7f0bc27fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0bc415e4e0 0x7f0bc40b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.484+0000 7f0bc2ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0bc40b2ff0 0x7f0bc415ddd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46918/0 (socket says 172.21.15.7:46918) 2024-01-27T02:44:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.484+0000 7f0bc2ffd700 1 -- 172.21.15.7:0/2554540671 learned_addr learned my addr 172.21.15.7:0/2554540671 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.485+0000 7f0bc37fe700 1 --2- 172.21.15.7:0/2554540671 >> v2:172.21.15.188:3300/0 conn(0x7f0bc415b8e0 0x7f0bc40ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.485+0000 7f0bc27fc700 1 -- 172.21.15.7:0/2554540671 >> v2:172.21.15.188:3300/0 conn(0x7f0bc415b8e0 msgr2=0x7f0bc40ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.485+0000 7f0bc27fc700 1 --2- 172.21.15.7:0/2554540671 >> v2:172.21.15.188:3300/0 conn(0x7f0bc415b8e0 0x7f0bc40ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.485+0000 7f0bc27fc700 1 -- 172.21.15.7:0/2554540671 >> v1:172.21.15.7:6789/0 conn(0x7f0bc40b2ff0 legacy=0x7f0bc415ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.485+0000 7f0bc27fc700 1 -- 172.21.15.7:0/2554540671 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc41587b0 con 0x7f0bc415e4e0 2024-01-27T02:44:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.485+0000 7f0bc37fe700 1 --2- 172.21.15.7:0/2554540671 >> v2:172.21.15.188:3300/0 conn(0x7f0bc415b8e0 0x7f0bc40ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:44:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.485+0000 7f0bc27fc700 1 --2- 172.21.15.7:0/2554540671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc415e4e0 0x7f0bc40b33c0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4151790 tx=0x7f0bac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66c5ddc6f5e63ac server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.485+0000 7f0bc1ffb700 1 -- 172.21.15.7:0/2554540671 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac00b6b0 con 0x7f0bc415e4e0 2024-01-27T02:44:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.485+0000 7f0bc1ffb700 1 -- 172.21.15.7:0/2554540671 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bac00b810 con 0x7f0bc415e4e0 2024-01-27T02:44:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.485+0000 7f0bc1ffb700 1 -- 172.21.15.7:0/2554540671 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac00bbb0 con 0x7f0bc415e4e0 2024-01-27T02:44:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.486+0000 7f0bc983a700 1 -- 172.21.15.7:0/2554540671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc415e4e0 msgr2=0x7f0bc40b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.486+0000 7f0bc983a700 1 --2- 172.21.15.7:0/2554540671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc415e4e0 0x7f0bc40b33c0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4151790 tx=0x7f0bac0091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.486+0000 7f0bc983a700 1 -- 172.21.15.7:0/2554540671 shutdown_connections 2024-01-27T02:44:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.486+0000 7f0bc983a700 1 --2- 172.21.15.7:0/2554540671 >> v2:172.21.15.188:3300/0 conn(0x7f0bc415b8e0 0x7f0bc40ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.486+0000 7f0bc983a700 1 --2- 172.21.15.7:0/2554540671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc415e4e0 0x7f0bc40b33c0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.486+0000 7f0bc983a700 1 -- 172.21.15.7:0/2554540671 >> 172.21.15.7:0/2554540671 conn(0x7f0bc40679b0 msgr2=0x7f0bc4067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.486+0000 7f0bc983a700 1 -- 172.21.15.7:0/2554540671 shutdown_connections 2024-01-27T02:44:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.486+0000 7f0bc983a700 1 -- 172.21.15.7:0/2554540671 wait complete. 2024-01-27T02:44:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.487+0000 7f0bc983a700 1 Processor -- start 2024-01-27T02:44:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.487+0000 7f0bc983a700 1 -- start start 2024-01-27T02:44:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.488+0000 7f0bc983a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc40b2ff0 0x7f0bc414e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.488+0000 7f0bc983a700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0bc415b8e0 0x7f0bc415a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.488+0000 7f0bc983a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0bc415e4e0 0x7f0bc40b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.488+0000 7f0bc983a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc414ec40 con 0x7f0bc40b2ff0 2024-01-27T02:44:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.488+0000 7f0bc983a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc41517c0 con 0x7f0bc415e4e0 2024-01-27T02:44:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.488+0000 7f0bc983a700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc416a570 con 0x7f0bc415b8e0 2024-01-27T02:44:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.488+0000 7f0bc2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc40b2ff0 0x7f0bc414e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.488+0000 7f0bc2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc40b2ff0 0x7f0bc414e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52150/0 (socket says 172.21.15.7:52150) 2024-01-27T02:44:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.488+0000 7f0bc2ffd700 1 -- 172.21.15.7:0/2561511767 learned_addr learned my addr 172.21.15.7:0/2561511767 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.488+0000 7f0bc27fc700 1 --2- 172.21.15.7:0/2561511767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0bc415b8e0 0x7f0bc415a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.488+0000 7f0bc37fe700 1 --2- 172.21.15.7:0/2561511767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0bc415e4e0 0x7f0bc40b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.488+0000 7f0bc2ffd700 1 -- 172.21.15.7:0/2561511767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0bc415b8e0 msgr2=0x7f0bc415a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:13.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.488+0000 7f0bc2ffd700 1 --2- 172.21.15.7:0/2561511767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0bc415b8e0 0x7f0bc415a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.488+0000 7f0bc2ffd700 1 -- 172.21.15.7:0/2561511767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0bc415e4e0 msgr2=0x7f0bc40b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:13.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.488+0000 7f0bc2ffd700 1 --2- 172.21.15.7:0/2561511767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0bc415e4e0 0x7f0bc40b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.488+0000 7f0bc2ffd700 1 -- 172.21.15.7:0/2561511767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bac009420 con 0x7f0bc40b2ff0 2024-01-27T02:44:13.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.488+0000 7f0bc27fc700 1 --2- 172.21.15.7:0/2561511767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0bc415b8e0 0x7f0bc415a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:44:13.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.488+0000 7f0bc37fe700 1 --2- 172.21.15.7:0/2561511767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0bc415e4e0 0x7f0bc40b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:44:13.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.489+0000 7f0bc2ffd700 1 --2- 172.21.15.7:0/2561511767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc40b2ff0 0x7f0bc414e700 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400c590 tx=0x7f0bb4007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:13.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.489+0000 7f0bbaffd700 1 -- 172.21.15.7:0/2561511767 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb40072e0 con 0x7f0bc40b2ff0 2024-01-27T02:44:13.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.489+0000 7f0bbaffd700 1 -- 172.21.15.7:0/2561511767 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb4007460 con 0x7f0bc40b2ff0 2024-01-27T02:44:13.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.489+0000 7f0bc983a700 1 -- 172.21.15.7:0/2561511767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc416a850 con 0x7f0bc40b2ff0 2024-01-27T02:44:13.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.489+0000 7f0bbaffd700 1 -- 172.21.15.7:0/2561511767 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb40057f0 con 0x7f0bc40b2ff0 2024-01-27T02:44:13.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.489+0000 7f0bc983a700 1 -- 172.21.15.7:0/2561511767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc416ae10 con 0x7f0bc40b2ff0 2024-01-27T02:44:13.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.490+0000 7f0bc983a700 1 -- 172.21.15.7:0/2561511767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc404c710 con 0x7f0bc40b2ff0 2024-01-27T02:44:13.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.490+0000 7f0bbaffd700 1 -- 172.21.15.7:0/2561511767 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0bb4005be0 con 0x7f0bc40b2ff0 2024-01-27T02:44:13.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.491+0000 7f0bbaffd700 1 --2- 172.21.15.7:0/2561511767 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0ba4066f30 0x7f0ba40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:13.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.491+0000 7f0bbaffd700 1 -- 172.21.15.7:0/2561511767 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0bb40835a0 con 0x7f0bc40b2ff0 2024-01-27T02:44:13.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.491+0000 7f0bc27fc700 1 --2- 172.21.15.7:0/2561511767 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0ba4066f30 0x7f0ba40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:13.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.492+0000 7f0bc27fc700 1 --2- 172.21.15.7:0/2561511767 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0ba4066f30 0x7f0ba40693e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4151790 tx=0x7f0bac017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:13.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.495+0000 7f0bbaffd700 1 -- 172.21.15.7:0/2561511767 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0bb404b5a0 con 0x7f0bc40b2ff0 2024-01-27T02:44:13.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.699+0000 7f0bc983a700 1 -- 172.21.15.7:0/2561511767 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0bc415be30 con 0x7f0ba4066f30 2024-01-27T02:44:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.700+0000 7f0bbaffd700 1 -- 172.21.15.7:0/2561511767 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0bc415be30 con 0x7f0ba4066f30 2024-01-27T02:44:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.702+0000 7f0bc983a700 1 -- 172.21.15.7:0/2561511767 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0ba4066f30 msgr2=0x7f0ba40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.702+0000 7f0bc983a700 1 --2- 172.21.15.7:0/2561511767 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0ba4066f30 0x7f0ba40693e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4151790 tx=0x7f0bac017000 comp rx=0 tx=0).stop 2024-01-27T02:44:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.702+0000 7f0bc983a700 1 -- 172.21.15.7:0/2561511767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc40b2ff0 msgr2=0x7f0bc414e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.702+0000 7f0bc983a700 1 --2- 172.21.15.7:0/2561511767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc40b2ff0 0x7f0bc414e700 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400c590 tx=0x7f0bb4007e40 comp rx=0 tx=0).stop 2024-01-27T02:44:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.702+0000 7f0bc983a700 1 -- 172.21.15.7:0/2561511767 shutdown_connections 2024-01-27T02:44:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.702+0000 7f0bc983a700 1 --2- 172.21.15.7:0/2561511767 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0ba4066f30 0x7f0ba40693e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.702+0000 7f0bc983a700 1 --2- 172.21.15.7:0/2561511767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0bc415e4e0 0x7f0bc40b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.702+0000 7f0bc983a700 1 --2- 172.21.15.7:0/2561511767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc40b2ff0 0x7f0bc414e700 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.702+0000 7f0bc983a700 1 --2- 172.21.15.7:0/2561511767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0bc415b8e0 0x7f0bc415a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.702+0000 7f0bc983a700 1 -- 172.21.15.7:0/2561511767 >> 172.21.15.7:0/2561511767 conn(0x7f0bc40679b0 msgr2=0x7f0bc414fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.702+0000 7f0bc983a700 1 -- 172.21.15.7:0/2561511767 shutdown_connections 2024-01-27T02:44:13.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.702+0000 7f0bc983a700 1 -- 172.21.15.7:0/2561511767 wait complete. 2024-01-27T02:44:13.705 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 1 to deep-scrub 2024-01-27T02:44:13.717 INFO:tasks.ceph:Scrubbing osd.2 2024-01-27T02:44:13.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-01-27T02:44:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.852+0000 7f65a871f700 1 Processor -- start 2024-01-27T02:44:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.852+0000 7f65a871f700 1 -- start start 2024-01-27T02:44:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.854+0000 7f65a871f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f65a00ce330 0x7f65a006d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.854+0000 7f65a871f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65a00caf20 0x7f65a00cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.854+0000 7f65a871f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a00c9720 con 0x7f65a0141700 2024-01-27T02:44:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.854+0000 7f65a871f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a00cd950 con 0x7f65a00ce330 2024-01-27T02:44:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.854+0000 7f65a871f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a006c2b0 con 0x7f65a00caf20 2024-01-27T02:44:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.854+0000 7f65a5cba700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f65a00ce330 0x7f65a006d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.854+0000 7f65a64bb700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f65a0141700 0x7f65a006ca60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46930/0 (socket says 172.21.15.7:46930) 2024-01-27T02:44:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.854+0000 7f65a5cba700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f65a00ce330 0x7f65a006d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52164/0 (socket says 172.21.15.7:52164) 2024-01-27T02:44:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.854+0000 7f65a64bb700 1 -- 172.21.15.7:0/957266183 learned_addr learned my addr 172.21.15.7:0/957266183 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.854+0000 7f65a6cbc700 1 --2- 172.21.15.7:0/957266183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65a00caf20 0x7f65a00cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.854+0000 7f65a5cba700 1 -- 172.21.15.7:0/957266183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65a00caf20 msgr2=0x7f65a00cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.854+0000 7f65a5cba700 1 --2- 172.21.15.7:0/957266183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65a00caf20 0x7f65a00cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.854+0000 7f65a5cba700 1 -- 172.21.15.7:0/957266183 >> v1:172.21.15.7:6789/0 conn(0x7f65a0141700 legacy=0x7f65a006ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.854+0000 7f65a5cba700 1 -- 172.21.15.7:0/957266183 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a00cd670 con 0x7f65a00ce330 2024-01-27T02:44:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.855+0000 7f65a5cba700 1 --2- 172.21.15.7:0/957266183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a00ce330 0x7f65a006d170 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f659c0098e0 tx=0x7f659c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d3d4c633209431 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.855+0000 7f65a54b9700 1 -- 172.21.15.7:0/957266183 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c009e80 con 0x7f65a00ce330 2024-01-27T02:44:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.855+0000 7f65a54b9700 1 -- 172.21.15.7:0/957266183 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c00b860 con 0x7f65a00ce330 2024-01-27T02:44:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.855+0000 7f65a54b9700 1 -- 172.21.15.7:0/957266183 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c00bbe0 con 0x7f65a00ce330 2024-01-27T02:44:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.855+0000 7f65a871f700 1 -- 172.21.15.7:0/957266183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a00ce330 msgr2=0x7f65a006d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.855+0000 7f65a871f700 1 --2- 172.21.15.7:0/957266183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a00ce330 0x7f65a006d170 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f659c0098e0 tx=0x7f659c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.855+0000 7f65a871f700 1 -- 172.21.15.7:0/957266183 shutdown_connections 2024-01-27T02:44:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.855+0000 7f65a871f700 1 --2- 172.21.15.7:0/957266183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65a00caf20 0x7f65a00cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.855+0000 7f65a871f700 1 --2- 172.21.15.7:0/957266183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a00ce330 0x7f65a006d170 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.855+0000 7f65a871f700 1 -- 172.21.15.7:0/957266183 >> 172.21.15.7:0/957266183 conn(0x7f65a00714b0 msgr2=0x7f65a006b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:13.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.856+0000 7f65a871f700 1 -- 172.21.15.7:0/957266183 shutdown_connections 2024-01-27T02:44:13.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.856+0000 7f65a871f700 1 -- 172.21.15.7:0/957266183 wait complete. 2024-01-27T02:44:13.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.857+0000 7f65a871f700 1 Processor -- start 2024-01-27T02:44:13.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.857+0000 7f65a871f700 1 -- start start 2024-01-27T02:44:13.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.857+0000 7f65a871f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a00caf20 0x7f65a00b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:13.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.858+0000 7f65a871f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65a00ce330 0x7f65a015ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:13.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.858+0000 7f65a871f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65a0141700 0x7f65a015b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:13.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.858+0000 7f65a871f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a0169120 con 0x7f65a00caf20 2024-01-27T02:44:13.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.858+0000 7f65a871f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a0168510 con 0x7f65a0141700 2024-01-27T02:44:13.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.858+0000 7f65a871f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a01687f0 con 0x7f65a00ce330 2024-01-27T02:44:13.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.858+0000 7f65a64bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a00caf20 0x7f65a00b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.858+0000 7f65a64bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a00caf20 0x7f65a00b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52176/0 (socket says 172.21.15.7:52176) 2024-01-27T02:44:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.858+0000 7f65a64bb700 1 -- 172.21.15.7:0/211511477 learned_addr learned my addr 172.21.15.7:0/211511477 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.858+0000 7f65a6cbc700 1 --2- 172.21.15.7:0/211511477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65a0141700 0x7f65a015b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.858+0000 7f65a5cba700 1 --2- 172.21.15.7:0/211511477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65a00ce330 0x7f65a015ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.858+0000 7f65a64bb700 1 -- 172.21.15.7:0/211511477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65a00ce330 msgr2=0x7f65a015ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.858+0000 7f65a64bb700 1 --2- 172.21.15.7:0/211511477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65a00ce330 0x7f65a015ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.858+0000 7f65a64bb700 1 -- 172.21.15.7:0/211511477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65a0141700 msgr2=0x7f65a015b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.858+0000 7f65a64bb700 1 --2- 172.21.15.7:0/211511477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65a0141700 0x7f65a015b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.858+0000 7f65a64bb700 1 -- 172.21.15.7:0/211511477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c009420 con 0x7f65a00caf20 2024-01-27T02:44:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.858+0000 7f65a6cbc700 1 --2- 172.21.15.7:0/211511477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65a0141700 0x7f65a015b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:44:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.858+0000 7f65a64bb700 1 --2- 172.21.15.7:0/211511477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a00caf20 0x7f65a00b0cf0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f659400c560 tx=0x7f6594001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.858+0000 7f65927fc700 1 -- 172.21.15.7:0/211511477 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6594005b30 con 0x7f65a00caf20 2024-01-27T02:44:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.858+0000 7f65927fc700 1 -- 172.21.15.7:0/211511477 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6594005cb0 con 0x7f65a00caf20 2024-01-27T02:44:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.858+0000 7f65a871f700 1 -- 172.21.15.7:0/211511477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a0168ad0 con 0x7f65a00caf20 2024-01-27T02:44:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.858+0000 7f65927fc700 1 -- 172.21.15.7:0/211511477 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6594007320 con 0x7f65a00caf20 2024-01-27T02:44:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.859+0000 7f65a871f700 1 -- 172.21.15.7:0/211511477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a01681f0 con 0x7f65a00caf20 2024-01-27T02:44:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.859+0000 7f65a871f700 1 -- 172.21.15.7:0/211511477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f65a004c710 con 0x7f65a00caf20 2024-01-27T02:44:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.860+0000 7f65927fc700 1 -- 172.21.15.7:0/211511477 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f659401b3b0 con 0x7f65a00caf20 2024-01-27T02:44:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.860+0000 7f65927fc700 1 --2- 172.21.15.7:0/211511477 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6584066f30 0x7f65840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.860+0000 7f65927fc700 1 -- 172.21.15.7:0/211511477 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6594083d60 con 0x7f65a00caf20 2024-01-27T02:44:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.861+0000 7f65927fc700 1 --2- 172.21.15.7:0/211511477 >> [v2:172.21.15.7:6824/39325,v1:172.21.15.7:6825/39325] conn(0x7f658406cd80 0x7f658406f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.861+0000 7f65927fc700 1 -- 172.21.15.7:0/211511477 --> [v2:172.21.15.7:6824/39325,v1:172.21.15.7:6825/39325] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f658406f860 con 0x7f658406cd80 2024-01-27T02:44:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.861+0000 7f65927fc700 1 -- 172.21.15.7:0/211511477 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=187) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6594084050 con 0x7f65a00caf20 2024-01-27T02:44:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.861+0000 7f65a5cba700 1 --2- 172.21.15.7:0/211511477 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6584066f30 0x7f65840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.861+0000 7f65a6cbc700 1 --2- 172.21.15.7:0/211511477 >> [v2:172.21.15.7:6824/39325,v1:172.21.15.7:6825/39325] conn(0x7f658406cd80 0x7f658406f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.861+0000 7f65a5cba700 1 --2- 172.21.15.7:0/211511477 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6584066f30 0x7f65840693e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f659c0098e0 tx=0x7f659c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.861+0000 7f65a6cbc700 1 --2- 172.21.15.7:0/211511477 >> [v2:172.21.15.7:6824/39325,v1:172.21.15.7:6825/39325] conn(0x7f658406cd80 0x7f658406f190 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.862+0000 7f65927fc700 1 -- 172.21.15.7:0/211511477 <== osd.2 v2:172.21.15.7:6824/39325 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f658406f860 con 0x7f658406cd80 2024-01-27T02:44:13.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.875+0000 7f65a871f700 1 -- 172.21.15.7:0/211511477 --> [v2:172.21.15.7:6824/39325,v1:172.21.15.7:6825/39325] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f65a00c9720 con 0x7f658406cd80 2024-01-27T02:44:13.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.876+0000 7f65927fc700 1 -- 172.21.15.7:0/211511477 <== osd.2 v2:172.21.15.7:6824/39325 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7f65a00c9720 con 0x7f658406cd80 2024-01-27T02:44:13.877 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:44:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-01-27T02:44:13.878 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:44:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.876+0000 7f65a871f700 1 -- 172.21.15.7:0/211511477 >> [v2:172.21.15.7:6824/39325,v1:172.21.15.7:6825/39325] conn(0x7f658406cd80 msgr2=0x7f658406f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.876+0000 7f65a871f700 1 --2- 172.21.15.7:0/211511477 >> [v2:172.21.15.7:6824/39325,v1:172.21.15.7:6825/39325] conn(0x7f658406cd80 0x7f658406f190 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.876+0000 7f65a871f700 1 -- 172.21.15.7:0/211511477 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6584066f30 msgr2=0x7f65840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.876+0000 7f65a871f700 1 --2- 172.21.15.7:0/211511477 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6584066f30 0x7f65840693e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f659c0098e0 tx=0x7f659c017000 comp rx=0 tx=0).stop 2024-01-27T02:44:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.876+0000 7f65a871f700 1 -- 172.21.15.7:0/211511477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a00caf20 msgr2=0x7f65a00b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.876+0000 7f65a871f700 1 --2- 172.21.15.7:0/211511477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a00caf20 0x7f65a00b0cf0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f659400c560 tx=0x7f6594001020 comp rx=0 tx=0).stop 2024-01-27T02:44:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.876+0000 7f65a6cbc700 1 -- 172.21.15.7:0/211511477 reap_dead start 2024-01-27T02:44:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.876+0000 7f65a871f700 1 -- 172.21.15.7:0/211511477 shutdown_connections 2024-01-27T02:44:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.876+0000 7f65a871f700 1 -- 172.21.15.7:0/211511477 >> 172.21.15.7:0/211511477 conn(0x7f65a00714b0 msgr2=0x7f65a00c3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.876+0000 7f65a871f700 1 -- 172.21.15.7:0/211511477 shutdown_connections 2024-01-27T02:44:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:13.876+0000 7f65a871f700 1 -- 172.21.15.7:0/211511477 wait complete. 2024-01-27T02:44:13.891 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-01-27T02:44:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.022+0000 7f602b2e9700 1 Processor -- start 2024-01-27T02:44:14.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.022+0000 7f602b2e9700 1 -- start start 2024-01-27T02:44:14.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.023+0000 7f602b2e9700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6024069780 0x7f60240c4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.023+0000 7f602b2e9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f60240cdc50 0x7f60240c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.023+0000 7f602b2e9700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60240c44a0 con 0x7f60240cd880 2024-01-27T02:44:14.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.023+0000 7f602b2e9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60240692d0 con 0x7f60240cdc50 2024-01-27T02:44:14.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.023+0000 7f602b2e9700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60240c3b70 con 0x7f6024069780 2024-01-27T02:44:14.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.023+0000 7f6029886700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f60240cdc50 0x7f60240c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.023+0000 7f6029886700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f60240cdc50 0x7f60240c4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52182/0 (socket says 172.21.15.7:52182) 2024-01-27T02:44:14.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.023+0000 7f6029886700 1 -- 172.21.15.7:0/669922535 learned_addr learned my addr 172.21.15.7:0/669922535 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:14.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.023+0000 7f6028884700 1 --2- 172.21.15.7:0/669922535 >> v2:172.21.15.188:3300/0 conn(0x7f6024069780 0x7f60240c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.024+0000 7f6029886700 1 -- 172.21.15.7:0/669922535 >> v2:172.21.15.188:3300/0 conn(0x7f6024069780 msgr2=0x7f60240c4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:14.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.024+0000 7f6017fff700 1 -- 172.21.15.7:0/669922535 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 389705516 0 0) 0x7f60240c44a0 con 0x7f60240cd880 2024-01-27T02:44:14.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.024+0000 7f6029886700 1 --2- 172.21.15.7:0/669922535 >> v2:172.21.15.188:3300/0 conn(0x7f6024069780 0x7f60240c4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.024+0000 7f6029886700 1 -- 172.21.15.7:0/669922535 >> v1:172.21.15.188:6789/0 conn(0x7f60240cd880 legacy=0x7f60240c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.024+0000 7f6029886700 1 -- 172.21.15.7:0/669922535 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60240c3e50 con 0x7f60240cdc50 2024-01-27T02:44:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.024+0000 7f6029886700 1 --2- 172.21.15.7:0/669922535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60240cdc50 0x7f60240c4e70 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f6024070160 tx=0x7f601c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b573fcadcdd6db9d server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.024+0000 7f6017fff700 1 -- 172.21.15.7:0/669922535 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c00e070 con 0x7f60240cdc50 2024-01-27T02:44:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.024+0000 7f6017fff700 1 -- 172.21.15.7:0/669922535 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c00ba20 con 0x7f60240cdc50 2024-01-27T02:44:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.024+0000 7f6017fff700 1 -- 172.21.15.7:0/669922535 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c0092e0 con 0x7f60240cdc50 2024-01-27T02:44:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.025+0000 7f602b2e9700 1 -- 172.21.15.7:0/669922535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60240cdc50 msgr2=0x7f60240c4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.025+0000 7f602b2e9700 1 --2- 172.21.15.7:0/669922535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60240cdc50 0x7f60240c4e70 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f6024070160 tx=0x7f601c009840 comp rx=0 tx=0).stop 2024-01-27T02:44:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.025+0000 7f602b2e9700 1 -- 172.21.15.7:0/669922535 shutdown_connections 2024-01-27T02:44:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.025+0000 7f602b2e9700 1 --2- 172.21.15.7:0/669922535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60240cdc50 0x7f60240c4e70 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.025+0000 7f602b2e9700 1 --2- 172.21.15.7:0/669922535 >> v2:172.21.15.188:3300/0 conn(0x7f6024069780 0x7f60240c4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.025+0000 7f602b2e9700 1 -- 172.21.15.7:0/669922535 >> 172.21.15.7:0/669922535 conn(0x7f60240711c0 msgr2=0x7f60240ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.025+0000 7f602b2e9700 1 -- 172.21.15.7:0/669922535 shutdown_connections 2024-01-27T02:44:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.025+0000 7f602b2e9700 1 -- 172.21.15.7:0/669922535 wait complete. 2024-01-27T02:44:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.026+0000 7f602b2e9700 1 Processor -- start 2024-01-27T02:44:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.026+0000 7f602b2e9700 1 -- start start 2024-01-27T02:44:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.027+0000 7f602b2e9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6024069780 0x7f60240b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.027+0000 7f602b2e9700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60240cd880 0x7f602415c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.027+0000 7f602b2e9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60240cdc50 0x7f6024169ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.027+0000 7f602b2e9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602415c080 con 0x7f6024069780 2024-01-27T02:44:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.027+0000 7f602b2e9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60240b45e0 con 0x7f60240cdc50 2024-01-27T02:44:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.027+0000 7f602b2e9700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602416a9d0 con 0x7f60240cd880 2024-01-27T02:44:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.027+0000 7f6029085700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6024069780 0x7f60240b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.027+0000 7f6029085700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6024069780 0x7f60240b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52196/0 (socket says 172.21.15.7:52196) 2024-01-27T02:44:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.027+0000 7f6029085700 1 -- 172.21.15.7:0/2943677162 learned_addr learned my addr 172.21.15.7:0/2943677162 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.027+0000 7f6028884700 1 --2- 172.21.15.7:0/2943677162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60240cd880 0x7f602415c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.027+0000 7f6029886700 1 --2- 172.21.15.7:0/2943677162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60240cdc50 0x7f6024169ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.028+0000 7f6029085700 1 -- 172.21.15.7:0/2943677162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60240cd880 msgr2=0x7f602415c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.028+0000 7f6029085700 1 --2- 172.21.15.7:0/2943677162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60240cd880 0x7f602415c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.028+0000 7f6029085700 1 -- 172.21.15.7:0/2943677162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60240cdc50 msgr2=0x7f6024169ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.028+0000 7f6029085700 1 --2- 172.21.15.7:0/2943677162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60240cdc50 0x7f6024169ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.028+0000 7f6029085700 1 -- 172.21.15.7:0/2943677162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c009ae0 con 0x7f6024069780 2024-01-27T02:44:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.028+0000 7f6029886700 1 --2- 172.21.15.7:0/2943677162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60240cdc50 0x7f6024169ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:44:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.028+0000 7f6028884700 1 --2- 172.21.15.7:0/2943677162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60240cd880 0x7f602415c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:44:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.028+0000 7f6029085700 1 --2- 172.21.15.7:0/2943677162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6024069780 0x7f60240b40a0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f601800c810 tx=0x7f6018007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.028+0000 7f60157fa700 1 -- 172.21.15.7:0/2943677162 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6018012700 con 0x7f6024069780 2024-01-27T02:44:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.028+0000 7f60157fa700 1 -- 172.21.15.7:0/2943677162 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60180128a0 con 0x7f6024069780 2024-01-27T02:44:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.028+0000 7f602b2e9700 1 -- 172.21.15.7:0/2943677162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602416acb0 con 0x7f6024069780 2024-01-27T02:44:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.028+0000 7f60157fa700 1 -- 172.21.15.7:0/2943677162 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6018012c60 con 0x7f6024069780 2024-01-27T02:44:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.028+0000 7f602b2e9700 1 -- 172.21.15.7:0/2943677162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602415d3e0 con 0x7f6024069780 2024-01-27T02:44:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.029+0000 7f602b2e9700 1 -- 172.21.15.7:0/2943677162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f602415bb70 con 0x7f6024069780 2024-01-27T02:44:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.030+0000 7f60157fa700 1 -- 172.21.15.7:0/2943677162 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f601801b400 con 0x7f6024069780 2024-01-27T02:44:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.030+0000 7f60157fa700 1 --2- 172.21.15.7:0/2943677162 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6008066f30 0x7f60080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.030+0000 7f60157fa700 1 -- 172.21.15.7:0/2943677162 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6018083d80 con 0x7f6024069780 2024-01-27T02:44:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.030+0000 7f6028884700 1 --2- 172.21.15.7:0/2943677162 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6008066f30 0x7f60080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.031+0000 7f6028884700 1 --2- 172.21.15.7:0/2943677162 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6008066f30 0x7f60080693e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f60240b2d00 tx=0x7f6020008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.035+0000 7f60157fa700 1 -- 172.21.15.7:0/2943677162 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f602415bb70 con 0x7f6024069780 2024-01-27T02:44:14.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.240+0000 7f602b2e9700 1 -- 172.21.15.7:0/2943677162 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f602415bb70 con 0x7f6008066f30 2024-01-27T02:44:14.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.241+0000 7f60157fa700 1 -- 172.21.15.7:0/2943677162 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f602415bb70 con 0x7f6008066f30 2024-01-27T02:44:14.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.243+0000 7f602b2e9700 1 -- 172.21.15.7:0/2943677162 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6008066f30 msgr2=0x7f60080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.243+0000 7f602b2e9700 1 --2- 172.21.15.7:0/2943677162 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6008066f30 0x7f60080693e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f60240b2d00 tx=0x7f6020008040 comp rx=0 tx=0).stop 2024-01-27T02:44:14.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.243+0000 7f602b2e9700 1 -- 172.21.15.7:0/2943677162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6024069780 msgr2=0x7f60240b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.243+0000 7f602b2e9700 1 --2- 172.21.15.7:0/2943677162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6024069780 0x7f60240b40a0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f601800c810 tx=0x7f6018007dd0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.243+0000 7f602b2e9700 1 -- 172.21.15.7:0/2943677162 shutdown_connections 2024-01-27T02:44:14.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.243+0000 7f602b2e9700 1 --2- 172.21.15.7:0/2943677162 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6008066f30 0x7f60080693e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.243+0000 7f602b2e9700 1 --2- 172.21.15.7:0/2943677162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60240cdc50 0x7f6024169ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.243+0000 7f602b2e9700 1 --2- 172.21.15.7:0/2943677162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6024069780 0x7f60240b40a0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.243+0000 7f602b2e9700 1 --2- 172.21.15.7:0/2943677162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60240cd880 0x7f602415c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.243+0000 7f602b2e9700 1 -- 172.21.15.7:0/2943677162 >> 172.21.15.7:0/2943677162 conn(0x7f60240711c0 msgr2=0x7f602406c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:14.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.243+0000 7f602b2e9700 1 -- 172.21.15.7:0/2943677162 shutdown_connections 2024-01-27T02:44:14.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.243+0000 7f602b2e9700 1 -- 172.21.15.7:0/2943677162 wait complete. 2024-01-27T02:44:14.246 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 2 to deep-scrub 2024-01-27T02:44:14.258 INFO:tasks.ceph:Scrubbing osd.3 2024-01-27T02:44:14.258 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-01-27T02:44:14.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.391+0000 7fea804cf700 1 Processor -- start 2024-01-27T02:44:14.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.391+0000 7fea804cf700 1 -- start start 2024-01-27T02:44:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.392+0000 7fea804cf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fea781406f0 0x7fea780b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.392+0000 7fea804cf700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea7806bef0 0x7fea7806c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.392+0000 7fea804cf700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fea7806e330 0x7fea7806e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.392+0000 7fea804cf700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea7806ecf0 con 0x7fea781406f0 2024-01-27T02:44:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.392+0000 7fea804cf700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea780c30c0 con 0x7fea7806e330 2024-01-27T02:44:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.392+0000 7fea804cf700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea7806ccf0 con 0x7fea7806bef0 2024-01-27T02:44:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.392+0000 7fea7e26b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fea781406f0 0x7fea780b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.392+0000 7fea7e26b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fea781406f0 0x7fea780b09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52206/0 (socket says 172.21.15.7:52206) 2024-01-27T02:44:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.392+0000 7fea7e26b700 1 -- 172.21.15.7:0/3493797169 learned_addr learned my addr 172.21.15.7:0/3493797169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.392+0000 7fea7ea6c700 1 --2- 172.21.15.7:0/3493797169 >> v2:172.21.15.188:3300/0 conn(0x7fea7806e330 0x7fea7806e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.393+0000 7fea7da6a700 1 --2- 172.21.15.7:0/3493797169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea7806bef0 0x7fea7806c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.393+0000 7fea7e26b700 1 -- 172.21.15.7:0/3493797169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea7806bef0 msgr2=0x7fea7806c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.393+0000 7fea7e26b700 1 --2- 172.21.15.7:0/3493797169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea7806bef0 0x7fea7806c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.393+0000 7fea7e26b700 1 -- 172.21.15.7:0/3493797169 >> v2:172.21.15.188:3300/0 conn(0x7fea7806e330 msgr2=0x7fea7806e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.393+0000 7fea7e26b700 1 --2- 172.21.15.7:0/3493797169 >> v2:172.21.15.188:3300/0 conn(0x7fea7806e330 0x7fea7806e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.393+0000 7fea7e26b700 1 -- 172.21.15.7:0/3493797169 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7806ca10 con 0x7fea781406f0 2024-01-27T02:44:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.393+0000 7fea7ea6c700 1 --2- 172.21.15.7:0/3493797169 >> v2:172.21.15.188:3300/0 conn(0x7fea7806e330 0x7fea7806e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-27T02:44:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.393+0000 7fea7e26b700 1 --2- 172.21.15.7:0/3493797169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea781406f0 0x7fea780b09f0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fea7806e100 tx=0x7fea74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f442498c871ce4 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.393+0000 7fea7d269700 1 -- 172.21.15.7:0/3493797169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7400b860 con 0x7fea781406f0 2024-01-27T02:44:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.393+0000 7fea7d269700 1 -- 172.21.15.7:0/3493797169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7400b9c0 con 0x7fea781406f0 2024-01-27T02:44:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.393+0000 7fea7d269700 1 -- 172.21.15.7:0/3493797169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7400bd60 con 0x7fea781406f0 2024-01-27T02:44:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.393+0000 7fea804cf700 1 -- 172.21.15.7:0/3493797169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea781406f0 msgr2=0x7fea780b09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.393+0000 7fea804cf700 1 --2- 172.21.15.7:0/3493797169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea781406f0 0x7fea780b09f0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fea7806e100 tx=0x7fea74009250 comp rx=0 tx=0).stop 2024-01-27T02:44:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.394+0000 7fea804cf700 1 -- 172.21.15.7:0/3493797169 shutdown_connections 2024-01-27T02:44:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.394+0000 7fea804cf700 1 --2- 172.21.15.7:0/3493797169 >> v2:172.21.15.188:3300/0 conn(0x7fea7806e330 0x7fea7806e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.394+0000 7fea804cf700 1 --2- 172.21.15.7:0/3493797169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea781406f0 0x7fea780b09f0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.394+0000 7fea804cf700 1 --2- 172.21.15.7:0/3493797169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea7806bef0 0x7fea7806c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.394+0000 7fea804cf700 1 -- 172.21.15.7:0/3493797169 >> 172.21.15.7:0/3493797169 conn(0x7fea780cdc30 msgr2=0x7fea780ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.394+0000 7fea804cf700 1 -- 172.21.15.7:0/3493797169 shutdown_connections 2024-01-27T02:44:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.394+0000 7fea804cf700 1 -- 172.21.15.7:0/3493797169 wait complete. 2024-01-27T02:44:14.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.395+0000 7fea804cf700 1 Processor -- start 2024-01-27T02:44:14.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.395+0000 7fea804cf700 1 -- start start 2024-01-27T02:44:14.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.396+0000 7fea804cf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea7806bef0 0x7fea780c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.396+0000 7fea804cf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea7806e330 0x7fea78167550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.396+0000 7fea804cf700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea781406f0 0x7fea7813d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.396+0000 7fea804cf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea78159810 con 0x7fea7806bef0 2024-01-27T02:44:14.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.396+0000 7fea804cf700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea78167a90 con 0x7fea7806e330 2024-01-27T02:44:14.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.396+0000 7fea804cf700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea78157ca0 con 0x7fea781406f0 2024-01-27T02:44:14.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.396+0000 7fea7e26b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea7806bef0 0x7fea780c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.396+0000 7fea7e26b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea7806bef0 0x7fea780c4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52222/0 (socket says 172.21.15.7:52222) 2024-01-27T02:44:14.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.396+0000 7fea7e26b700 1 -- 172.21.15.7:0/2790706517 learned_addr learned my addr 172.21.15.7:0/2790706517 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:14.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.396+0000 7fea7ea6c700 1 --2- 172.21.15.7:0/2790706517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea781406f0 0x7fea7813d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.396+0000 7fea7e26b700 1 -- 172.21.15.7:0/2790706517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea781406f0 msgr2=0x7fea7813d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.396+0000 7fea7e26b700 1 --2- 172.21.15.7:0/2790706517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea781406f0 0x7fea7813d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.396+0000 7fea7e26b700 1 -- 172.21.15.7:0/2790706517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea7806e330 msgr2=0x7fea78167550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.396+0000 7fea7e26b700 1 --2- 172.21.15.7:0/2790706517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea7806e330 0x7fea78167550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.396+0000 7fea7e26b700 1 -- 172.21.15.7:0/2790706517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea740094f0 con 0x7fea7806bef0 2024-01-27T02:44:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.396+0000 7fea7ea6c700 1 --2- 172.21.15.7:0/2790706517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea781406f0 0x7fea7813d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:44:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.397+0000 7fea7e26b700 1 --2- 172.21.15.7:0/2790706517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea7806bef0 0x7fea780c4ce0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fea7806dcc0 tx=0x7fea74017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.397+0000 7fea6a7fc700 1 -- 172.21.15.7:0/2790706517 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea74024440 con 0x7fea7806bef0 2024-01-27T02:44:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.397+0000 7fea6a7fc700 1 -- 172.21.15.7:0/2790706517 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea740245a0 con 0x7fea7806bef0 2024-01-27T02:44:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.397+0000 7fea804cf700 1 -- 172.21.15.7:0/2790706517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea78157690 con 0x7fea7806bef0 2024-01-27T02:44:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.397+0000 7fea6a7fc700 1 -- 172.21.15.7:0/2790706517 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea74024960 con 0x7fea7806bef0 2024-01-27T02:44:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.397+0000 7fea804cf700 1 -- 172.21.15.7:0/2790706517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea78159530 con 0x7fea7806bef0 2024-01-27T02:44:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.398+0000 7fea804cf700 1 -- 172.21.15.7:0/2790706517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fea781574f0 con 0x7fea7806bef0 2024-01-27T02:44:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.399+0000 7fea6a7fc700 1 -- 172.21.15.7:0/2790706517 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fea7401a420 con 0x7fea7806bef0 2024-01-27T02:44:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.399+0000 7fea6a7fc700 1 --2- 172.21.15.7:0/2790706517 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fea5c066f30 0x7fea5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.399+0000 7fea6a7fc700 1 -- 172.21.15.7:0/2790706517 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fea740865c0 con 0x7fea7806bef0 2024-01-27T02:44:14.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.399+0000 7fea6a7fc700 1 --2- 172.21.15.7:0/2790706517 >> [v2:172.21.15.7:6808/39310,v1:172.21.15.7:6809/39310] conn(0x7fea5c06cd80 0x7fea5c06f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.399+0000 7fea6a7fc700 1 -- 172.21.15.7:0/2790706517 --> [v2:172.21.15.7:6808/39310,v1:172.21.15.7:6809/39310] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fea5c06f860 con 0x7fea5c06cd80 2024-01-27T02:44:14.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.399+0000 7fea7da6a700 1 --2- 172.21.15.7:0/2790706517 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fea5c066f30 0x7fea5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.399+0000 7fea6a7fc700 1 -- 172.21.15.7:0/2790706517 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=187) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea74086990 con 0x7fea7806bef0 2024-01-27T02:44:14.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.399+0000 7fea7ea6c700 1 --2- 172.21.15.7:0/2790706517 >> [v2:172.21.15.7:6808/39310,v1:172.21.15.7:6809/39310] conn(0x7fea5c06cd80 0x7fea5c06f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.400+0000 7fea7da6a700 1 --2- 172.21.15.7:0/2790706517 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fea5c066f30 0x7fea5c0693e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fea780c9250 tx=0x7fea6c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:14.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.400+0000 7fea7ea6c700 1 --2- 172.21.15.7:0/2790706517 >> [v2:172.21.15.7:6808/39310,v1:172.21.15.7:6809/39310] conn(0x7fea5c06cd80 0x7fea5c06f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:14.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.400+0000 7fea6a7fc700 1 -- 172.21.15.7:0/2790706517 <== osd.3 v2:172.21.15.7:6808/39310 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7fea5c06f860 con 0x7fea5c06cd80 2024-01-27T02:44:14.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.414+0000 7fea804cf700 1 -- 172.21.15.7:0/2790706517 --> [v2:172.21.15.7:6808/39310,v1:172.21.15.7:6809/39310] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fea7804c710 con 0x7fea5c06cd80 2024-01-27T02:44:14.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.414+0000 7fea6a7fc700 1 -- 172.21.15.7:0/2790706517 <== osd.3 v2:172.21.15.7:6808/39310 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7fea7804c710 con 0x7fea5c06cd80 2024-01-27T02:44:14.416 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:44:14.416 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-01-27T02:44:14.416 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:44:14.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.415+0000 7fea804cf700 1 -- 172.21.15.7:0/2790706517 >> [v2:172.21.15.7:6808/39310,v1:172.21.15.7:6809/39310] conn(0x7fea5c06cd80 msgr2=0x7fea5c06f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.415+0000 7fea804cf700 1 --2- 172.21.15.7:0/2790706517 >> [v2:172.21.15.7:6808/39310,v1:172.21.15.7:6809/39310] conn(0x7fea5c06cd80 0x7fea5c06f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.415+0000 7fea804cf700 1 -- 172.21.15.7:0/2790706517 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fea5c066f30 msgr2=0x7fea5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.415+0000 7fea804cf700 1 --2- 172.21.15.7:0/2790706517 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fea5c066f30 0x7fea5c0693e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fea780c9250 tx=0x7fea6c008040 comp rx=0 tx=0).stop 2024-01-27T02:44:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.415+0000 7fea804cf700 1 -- 172.21.15.7:0/2790706517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea7806bef0 msgr2=0x7fea780c4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.415+0000 7fea804cf700 1 --2- 172.21.15.7:0/2790706517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea7806bef0 0x7fea780c4ce0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fea7806dcc0 tx=0x7fea74017000 comp rx=0 tx=0).stop 2024-01-27T02:44:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.415+0000 7fea7ea6c700 1 -- 172.21.15.7:0/2790706517 reap_dead start 2024-01-27T02:44:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.415+0000 7fea804cf700 1 -- 172.21.15.7:0/2790706517 shutdown_connections 2024-01-27T02:44:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.415+0000 7fea804cf700 1 -- 172.21.15.7:0/2790706517 >> 172.21.15.7:0/2790706517 conn(0x7fea780cdc30 msgr2=0x7fea780cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.415+0000 7fea804cf700 1 -- 172.21.15.7:0/2790706517 shutdown_connections 2024-01-27T02:44:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.415+0000 7fea804cf700 1 -- 172.21.15.7:0/2790706517 wait complete. 2024-01-27T02:44:14.429 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-01-27T02:44:14.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.560+0000 7f56a5820700 1 Processor -- start 2024-01-27T02:44:14.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.560+0000 7f56a5820700 1 -- start start 2024-01-27T02:44:14.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.561+0000 7f56a5820700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56a00d10a0 0x7f56a00b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.561+0000 7f56a5820700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f56a00d1470 0x7f56a00b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.561+0000 7f56a5820700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56a00b4650 con 0x7f56a0159b40 2024-01-27T02:44:14.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.561+0000 7f56a5820700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56a00b3d60 con 0x7f56a00d1470 2024-01-27T02:44:14.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.561+0000 7f56a5820700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a00b4000 con 0x7f56a00d10a0 2024-01-27T02:44:14.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.561+0000 7f569e7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f56a00d1470 0x7f56a00b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.561+0000 7f569f7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f56a0159b40 0x7f56a0159f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46932/0 (socket says 172.21.15.7:46932) 2024-01-27T02:44:14.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.561+0000 7f569f7fe700 1 -- 172.21.15.7:0/3621778068 learned_addr learned my addr 172.21.15.7:0/3621778068 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:14.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.561+0000 7f569effd700 1 --2- 172.21.15.7:0/3621778068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56a00d10a0 0x7f56a00b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.561+0000 7f569dffb700 1 -- 172.21.15.7:0/3621778068 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3491154426 0 0) 0x7f56a00b4650 con 0x7f56a0159b40 2024-01-27T02:44:14.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.561+0000 7f569dffb700 1 -- 172.21.15.7:0/3621778068 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f568c003570 con 0x7f56a0159b40 2024-01-27T02:44:14.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.562+0000 7f569e7fc700 1 -- 172.21.15.7:0/3621778068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56a00d10a0 msgr2=0x7f56a00b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:14.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.562+0000 7f569e7fc700 1 --2- 172.21.15.7:0/3621778068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56a00d10a0 0x7f56a00b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.562+0000 7f569e7fc700 1 -- 172.21.15.7:0/3621778068 >> v1:172.21.15.7:6789/0 conn(0x7f56a0159b40 legacy=0x7f56a0159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.562+0000 7f569e7fc700 1 -- 172.21.15.7:0/3621778068 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a00b42e0 con 0x7f56a00d1470 2024-01-27T02:44:14.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.562+0000 7f569e7fc700 1 --2- 172.21.15.7:0/3621778068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56a00d1470 0x7f56a00b2dc0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f56880098e0 tx=0x7f56880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59c1c8de1c3080 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:14.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.562+0000 7f569dffb700 1 -- 172.21.15.7:0/3621778068 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5688011630 con 0x7f56a00d1470 2024-01-27T02:44:14.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.562+0000 7f569dffb700 1 -- 172.21.15.7:0/3621778068 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5688011790 con 0x7f56a00d1470 2024-01-27T02:44:14.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.562+0000 7f569dffb700 1 -- 172.21.15.7:0/3621778068 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5688011b30 con 0x7f56a00d1470 2024-01-27T02:44:14.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.562+0000 7f56a5820700 1 -- 172.21.15.7:0/3621778068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56a00d1470 msgr2=0x7f56a00b2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.562+0000 7f56a5820700 1 --2- 172.21.15.7:0/3621778068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56a00d1470 0x7f56a00b2dc0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f56880098e0 tx=0x7f56880091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.563+0000 7f56a5820700 1 -- 172.21.15.7:0/3621778068 shutdown_connections 2024-01-27T02:44:14.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.563+0000 7f56a5820700 1 --2- 172.21.15.7:0/3621778068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56a00d10a0 0x7f56a00b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.563+0000 7f56a5820700 1 --2- 172.21.15.7:0/3621778068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56a00d1470 0x7f56a00b2dc0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.563+0000 7f56a5820700 1 -- 172.21.15.7:0/3621778068 >> 172.21.15.7:0/3621778068 conn(0x7f56a00679b0 msgr2=0x7f56a0067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:14.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.563+0000 7f56a5820700 1 -- 172.21.15.7:0/3621778068 shutdown_connections 2024-01-27T02:44:14.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.563+0000 7f56a5820700 1 -- 172.21.15.7:0/3621778068 wait complete. 2024-01-27T02:44:14.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.564+0000 7f56a5820700 1 Processor -- start 2024-01-27T02:44:14.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.564+0000 7f56a5820700 1 -- start start 2024-01-27T02:44:14.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.565+0000 7f56a5820700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56a00d10a0 0x7f56a015d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.565+0000 7f56a5820700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56a00d1470 0x7f56a00c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.565+0000 7f56a5820700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56a0159b40 0x7f56a0148290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.565+0000 7f56a5820700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a015a830 con 0x7f56a0159b40 2024-01-27T02:44:14.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.565+0000 7f56a5820700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a00ca470 con 0x7f56a00d10a0 2024-01-27T02:44:14.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.565+0000 7f56a5820700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a00ca750 con 0x7f56a00d1470 2024-01-27T02:44:14.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.565+0000 7f569f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56a0159b40 0x7f56a0148290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.565+0000 7f569f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56a0159b40 0x7f56a0148290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52254/0 (socket says 172.21.15.7:52254) 2024-01-27T02:44:14.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.565+0000 7f569f7fe700 1 -- 172.21.15.7:0/3556158038 learned_addr learned my addr 172.21.15.7:0/3556158038 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:14.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.565+0000 7f569e7fc700 1 --2- 172.21.15.7:0/3556158038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56a00d1470 0x7f56a00c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.565+0000 7f569effd700 1 --2- 172.21.15.7:0/3556158038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56a00d10a0 0x7f56a015d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.565+0000 7f569f7fe700 1 -- 172.21.15.7:0/3556158038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56a00d1470 msgr2=0x7f56a00c7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.565+0000 7f569f7fe700 1 --2- 172.21.15.7:0/3556158038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56a00d1470 0x7f56a00c7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.565+0000 7f569f7fe700 1 -- 172.21.15.7:0/3556158038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56a00d10a0 msgr2=0x7f56a015d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.565+0000 7f569f7fe700 1 --2- 172.21.15.7:0/3556158038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56a00d10a0 0x7f56a015d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.565+0000 7f569f7fe700 1 -- 172.21.15.7:0/3556158038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5688009420 con 0x7f56a0159b40 2024-01-27T02:44:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.565+0000 7f569f7fe700 1 --2- 172.21.15.7:0/3556158038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56a0159b40 0x7f56a0148290 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f5698001460 tx=0x7f5698001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.566+0000 7f5696ffd700 1 -- 172.21.15.7:0/3556158038 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569800a870 con 0x7f56a0159b40 2024-01-27T02:44:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.566+0000 7f5696ffd700 1 -- 172.21.15.7:0/3556158038 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569800a9d0 con 0x7f56a0159b40 2024-01-27T02:44:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.566+0000 7f5696ffd700 1 -- 172.21.15.7:0/3556158038 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569800ad50 con 0x7f56a0159b40 2024-01-27T02:44:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.566+0000 7f56a5820700 1 -- 172.21.15.7:0/3556158038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a00caa30 con 0x7f56a0159b40 2024-01-27T02:44:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.566+0000 7f56a5820700 1 -- 172.21.15.7:0/3556158038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a01487d0 con 0x7f56a0159b40 2024-01-27T02:44:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.567+0000 7f56a5820700 1 -- 172.21.15.7:0/3556158038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a004c710 con 0x7f56a0159b40 2024-01-27T02:44:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.567+0000 7f5696ffd700 1 -- 172.21.15.7:0/3556158038 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5698007300 con 0x7f56a0159b40 2024-01-27T02:44:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.568+0000 7f5696ffd700 1 --2- 172.21.15.7:0/3556158038 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5680066f30 0x7f56800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.568+0000 7f5696ffd700 1 -- 172.21.15.7:0/3556158038 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f5698084560 con 0x7f56a0159b40 2024-01-27T02:44:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.568+0000 7f569effd700 1 --2- 172.21.15.7:0/3556158038 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5680066f30 0x7f56800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.569+0000 7f569effd700 1 --2- 172.21.15.7:0/3556158038 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5680066f30 0x7f56800693e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f56a0159700 tx=0x7f56900091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.572+0000 7f5696ffd700 1 -- 172.21.15.7:0/3556158038 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5698051ac0 con 0x7f56a0159b40 2024-01-27T02:44:14.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.773+0000 7f56a5820700 1 -- 172.21.15.7:0/3556158038 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f56a015a660 con 0x7f5680066f30 2024-01-27T02:44:14.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.774+0000 7f5696ffd700 1 -- 172.21.15.7:0/3556158038 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f56a015a660 con 0x7f5680066f30 2024-01-27T02:44:14.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.776+0000 7f56a5820700 1 -- 172.21.15.7:0/3556158038 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5680066f30 msgr2=0x7f56800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.776+0000 7f56a5820700 1 --2- 172.21.15.7:0/3556158038 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5680066f30 0x7f56800693e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f56a0159700 tx=0x7f56900091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.776+0000 7f56a5820700 1 -- 172.21.15.7:0/3556158038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56a0159b40 msgr2=0x7f56a0148290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.776+0000 7f56a5820700 1 --2- 172.21.15.7:0/3556158038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56a0159b40 0x7f56a0148290 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f5698001460 tx=0x7f5698001a40 comp rx=0 tx=0).stop 2024-01-27T02:44:14.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.776+0000 7f56a5820700 1 -- 172.21.15.7:0/3556158038 shutdown_connections 2024-01-27T02:44:14.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.776+0000 7f56a5820700 1 --2- 172.21.15.7:0/3556158038 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f5680066f30 0x7f56800693e0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.776+0000 7f56a5820700 1 --2- 172.21.15.7:0/3556158038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56a0159b40 0x7f56a0148290 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.776+0000 7f56a5820700 1 --2- 172.21.15.7:0/3556158038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56a00d10a0 0x7f56a015d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.776+0000 7f56a5820700 1 --2- 172.21.15.7:0/3556158038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56a00d1470 0x7f56a00c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.776+0000 7f56a5820700 1 -- 172.21.15.7:0/3556158038 >> 172.21.15.7:0/3556158038 conn(0x7f56a00679b0 msgr2=0x7f56a0159f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:14.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.776+0000 7f56a5820700 1 -- 172.21.15.7:0/3556158038 shutdown_connections 2024-01-27T02:44:14.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.776+0000 7f56a5820700 1 -- 172.21.15.7:0/3556158038 wait complete. 2024-01-27T02:44:14.779 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 3 to deep-scrub 2024-01-27T02:44:14.791 INFO:tasks.ceph:Scrubbing osd.4 2024-01-27T02:44:14.791 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-01-27T02:44:14.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.926+0000 7f0cfdf5f700 1 Processor -- start 2024-01-27T02:44:14.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.926+0000 7f0cfdf5f700 1 -- start start 2024-01-27T02:44:14.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.926+0000 7f0cfdf5f700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0cf806faa0 0x7f0cf8165310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.927+0000 7f0cfdf5f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf80afcf0 con 0x7f0cf80a7fe0 2024-01-27T02:44:14.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.927+0000 7f0cfdf5f700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf815a880 con 0x7f0cf80a7c10 2024-01-27T02:44:14.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.927+0000 7f0cfdf5f700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf80aea50 con 0x7f0cf806faa0 2024-01-27T02:44:14.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.927+0000 7f0cf6ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0cf80a7fe0 0x7f0cf806f390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46942/0 (socket says 172.21.15.7:46942) 2024-01-27T02:44:14.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.927+0000 7f0cf6ffd700 1 -- 172.21.15.7:0/2691603911 learned_addr learned my addr 172.21.15.7:0/2691603911 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:14.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.927+0000 7f0cf7fff700 1 --2- 172.21.15.7:0/2691603911 >> v2:172.21.15.188:3300/0 conn(0x7f0cf806faa0 0x7f0cf8165310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.927+0000 7f0cf67fc700 1 -- 172.21.15.7:0/2691603911 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4206680697 0 0) 0x7f0cf80afcf0 con 0x7f0cf80a7fe0 2024-01-27T02:44:14.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.927+0000 7f0cf67fc700 1 -- 172.21.15.7:0/2691603911 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ce0003570 con 0x7f0cf80a7fe0 2024-01-27T02:44:14.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.927+0000 7f0cf67fc700 1 -- 172.21.15.7:0/2691603911 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 522954327 0 0) 0x7f0cf815a880 con 0x7f0cf80a7c10 2024-01-27T02:44:14.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.927+0000 7f0cf67fc700 1 -- 172.21.15.7:0/2691603911 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cf80afcf0 con 0x7f0cf80a7c10 2024-01-27T02:44:14.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.927+0000 7f0cf7fff700 1 -- 172.21.15.7:0/2691603911 >> v1:172.21.15.188:6789/0 conn(0x7f0cf80a7c10 legacy=0x7f0cf80ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.927+0000 7f0cf7fff700 1 -- 172.21.15.7:0/2691603911 >> v1:172.21.15.7:6789/0 conn(0x7f0cf80a7fe0 legacy=0x7f0cf806f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.927+0000 7f0cf7fff700 1 -- 172.21.15.7:0/2691603911 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf80aec50 con 0x7f0cf806faa0 2024-01-27T02:44:14.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.927+0000 7f0cf7fff700 1 --2- 172.21.15.7:0/2691603911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cf806faa0 0x7f0cf8165310 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009e60 tx=0x7f0cf0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db62b637edcac5b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:14.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.928+0000 7f0cf67fc700 1 -- 172.21.15.7:0/2691603911 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf000e070 con 0x7f0cf806faa0 2024-01-27T02:44:14.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.928+0000 7f0cf67fc700 1 -- 172.21.15.7:0/2691603911 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf000bab0 con 0x7f0cf806faa0 2024-01-27T02:44:14.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.928+0000 7f0cf67fc700 1 -- 172.21.15.7:0/2691603911 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf000be70 con 0x7f0cf806faa0 2024-01-27T02:44:14.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.928+0000 7f0cfdf5f700 1 -- 172.21.15.7:0/2691603911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cf806faa0 msgr2=0x7f0cf8165310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.928+0000 7f0cfdf5f700 1 --2- 172.21.15.7:0/2691603911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cf806faa0 0x7f0cf8165310 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009e60 tx=0x7f0cf0009780 comp rx=0 tx=0).stop 2024-01-27T02:44:14.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.928+0000 7f0cfdf5f700 1 -- 172.21.15.7:0/2691603911 shutdown_connections 2024-01-27T02:44:14.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.928+0000 7f0cfdf5f700 1 --2- 172.21.15.7:0/2691603911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cf806faa0 0x7f0cf8165310 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.928+0000 7f0cfdf5f700 1 -- 172.21.15.7:0/2691603911 >> 172.21.15.7:0/2691603911 conn(0x7f0cf80679b0 msgr2=0x7f0cf8067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:14.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.928+0000 7f0cfdf5f700 1 -- 172.21.15.7:0/2691603911 shutdown_connections 2024-01-27T02:44:14.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.928+0000 7f0cfdf5f700 1 -- 172.21.15.7:0/2691603911 wait complete. 2024-01-27T02:44:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.930+0000 7f0cfdf5f700 1 Processor -- start 2024-01-27T02:44:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.930+0000 7f0cfdf5f700 1 -- start start 2024-01-27T02:44:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.930+0000 7f0cfdf5f700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0cf806faa0 0x7f0cf816b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.930+0000 7f0cfdf5f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cf80a7c10 0x7f0cf816b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.931+0000 7f0cfdf5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf80a7fe0 0x7f0cf816d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.931+0000 7f0cfdf5f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf816b550 con 0x7f0cf80a7fe0 2024-01-27T02:44:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.931+0000 7f0cfdf5f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf816d980 con 0x7f0cf80a7c10 2024-01-27T02:44:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.931+0000 7f0cfdf5f700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf815d500 con 0x7f0cf806faa0 2024-01-27T02:44:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.931+0000 7f0cf7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf80a7fe0 0x7f0cf816d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.931+0000 7f0cf7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf80a7fe0 0x7f0cf816d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52266/0 (socket says 172.21.15.7:52266) 2024-01-27T02:44:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.931+0000 7f0cf7fff700 1 -- 172.21.15.7:0/794880882 learned_addr learned my addr 172.21.15.7:0/794880882 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.931+0000 7f0cf6ffd700 1 --2- 172.21.15.7:0/794880882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cf80a7c10 0x7f0cf816b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.931+0000 7f0cf77fe700 1 --2- 172.21.15.7:0/794880882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0cf806faa0 0x7f0cf816b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.931+0000 7f0cf7fff700 1 -- 172.21.15.7:0/794880882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0cf806faa0 msgr2=0x7f0cf816b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.931+0000 7f0cf7fff700 1 --2- 172.21.15.7:0/794880882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0cf806faa0 0x7f0cf816b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.931+0000 7f0cf7fff700 1 -- 172.21.15.7:0/794880882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cf80a7c10 msgr2=0x7f0cf816b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.931+0000 7f0cf7fff700 1 --2- 172.21.15.7:0/794880882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0cf80a7c10 0x7f0cf816b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.931+0000 7f0cf7fff700 1 -- 172.21.15.7:0/794880882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf0009a20 con 0x7f0cf80a7fe0 2024-01-27T02:44:14.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.931+0000 7f0cf7fff700 1 --2- 172.21.15.7:0/794880882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf80a7fe0 0x7f0cf816d440 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8068eb0 tx=0x7f0cf0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:14.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.931+0000 7f0cdf7fe700 1 -- 172.21.15.7:0/794880882 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0012db0 con 0x7f0cf80a7fe0 2024-01-27T02:44:14.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.931+0000 7f0cfdf5f700 1 -- 172.21.15.7:0/794880882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf815d7e0 con 0x7f0cf80a7fe0 2024-01-27T02:44:14.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.931+0000 7f0cdf7fe700 1 -- 172.21.15.7:0/794880882 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf0026480 con 0x7f0cf80a7fe0 2024-01-27T02:44:14.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.931+0000 7f0cdf7fe700 1 -- 172.21.15.7:0/794880882 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0026840 con 0x7f0cf80a7fe0 2024-01-27T02:44:14.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.932+0000 7f0cfdf5f700 1 -- 172.21.15.7:0/794880882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf815dda0 con 0x7f0cf80a7fe0 2024-01-27T02:44:14.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.932+0000 7f0cfdf5f700 1 -- 172.21.15.7:0/794880882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0cf804c710 con 0x7f0cf80a7fe0 2024-01-27T02:44:14.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.933+0000 7f0cdf7fe700 1 -- 172.21.15.7:0/794880882 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0cf001c400 con 0x7f0cf80a7fe0 2024-01-27T02:44:14.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.933+0000 7f0cdf7fe700 1 --2- 172.21.15.7:0/794880882 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0ce4066f30 0x7f0ce40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.933+0000 7f0cdf7fe700 1 -- 172.21.15.7:0/794880882 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0cf00865f0 con 0x7f0cf80a7fe0 2024-01-27T02:44:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.934+0000 7f0cdf7fe700 1 --2- 172.21.15.7:0/794880882 >> [v2:172.21.15.188:6801/39374,v1:172.21.15.188:6803/39374] conn(0x7f0ce406cd80 0x7f0ce406f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.934+0000 7f0cf77fe700 1 --2- 172.21.15.7:0/794880882 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0ce4066f30 0x7f0ce40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.934+0000 7f0cdf7fe700 1 -- 172.21.15.7:0/794880882 --> [v2:172.21.15.188:6801/39374,v1:172.21.15.188:6803/39374] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0ce406f860 con 0x7f0ce406cd80 2024-01-27T02:44:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.934+0000 7f0cdf7fe700 1 -- 172.21.15.7:0/794880882 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=187) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cf00869c0 con 0x7f0cf80a7fe0 2024-01-27T02:44:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.934+0000 7f0cf6ffd700 1 --2- 172.21.15.7:0/794880882 >> [v2:172.21.15.188:6801/39374,v1:172.21.15.188:6803/39374] conn(0x7f0ce406cd80 0x7f0ce406f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.934+0000 7f0cf77fe700 1 --2- 172.21.15.7:0/794880882 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0ce4066f30 0x7f0ce40693e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8003ae0 tx=0x7f0ce8007d40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.934+0000 7f0cf6ffd700 1 --2- 172.21.15.7:0/794880882 >> [v2:172.21.15.188:6801/39374,v1:172.21.15.188:6803/39374] conn(0x7f0ce406cd80 0x7f0ce406f190 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.935+0000 7f0cdf7fe700 1 -- 172.21.15.7:0/794880882 <== osd.4 v2:172.21.15.188:6801/39374 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f0ce406f860 con 0x7f0ce406cd80 2024-01-27T02:44:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.949+0000 7f0cfdf5f700 1 -- 172.21.15.7:0/794880882 --> [v2:172.21.15.188:6801/39374,v1:172.21.15.188:6803/39374] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0cf80aef30 con 0x7f0ce406cd80 2024-01-27T02:44:14.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.949+0000 7f0cdf7fe700 1 -- 172.21.15.7:0/794880882 <== osd.4 v2:172.21.15.188:6801/39374 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7f0cf80aef30 con 0x7f0ce406cd80 2024-01-27T02:44:14.951 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:44:14.951 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-01-27T02:44:14.951 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:44:14.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.950+0000 7f0cfdf5f700 1 -- 172.21.15.7:0/794880882 >> [v2:172.21.15.188:6801/39374,v1:172.21.15.188:6803/39374] conn(0x7f0ce406cd80 msgr2=0x7f0ce406f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.950+0000 7f0cfdf5f700 1 --2- 172.21.15.7:0/794880882 >> [v2:172.21.15.188:6801/39374,v1:172.21.15.188:6803/39374] conn(0x7f0ce406cd80 0x7f0ce406f190 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:14.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.950+0000 7f0cfdf5f700 1 -- 172.21.15.7:0/794880882 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0ce4066f30 msgr2=0x7f0ce40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.950+0000 7f0cfdf5f700 1 --2- 172.21.15.7:0/794880882 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f0ce4066f30 0x7f0ce40693e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8003ae0 tx=0x7f0ce8007d40 comp rx=0 tx=0).stop 2024-01-27T02:44:14.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.950+0000 7f0cfdf5f700 1 -- 172.21.15.7:0/794880882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf80a7fe0 msgr2=0x7f0cf816d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:14.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.950+0000 7f0cfdf5f700 1 --2- 172.21.15.7:0/794880882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf80a7fe0 0x7f0cf816d440 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8068eb0 tx=0x7f0cf0019000 comp rx=0 tx=0).stop 2024-01-27T02:44:14.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.950+0000 7f0cf7fff700 1 -- 172.21.15.7:0/794880882 reap_dead start 2024-01-27T02:44:14.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.950+0000 7f0cfdf5f700 1 -- 172.21.15.7:0/794880882 shutdown_connections 2024-01-27T02:44:14.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.950+0000 7f0cfdf5f700 1 -- 172.21.15.7:0/794880882 >> 172.21.15.7:0/794880882 conn(0x7f0cf80679b0 msgr2=0x7f0cf806ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:14.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.950+0000 7f0cfdf5f700 1 -- 172.21.15.7:0/794880882 shutdown_connections 2024-01-27T02:44:14.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:14.950+0000 7f0cfdf5f700 1 -- 172.21.15.7:0/794880882 wait complete. 2024-01-27T02:44:14.965 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-01-27T02:44:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.096+0000 7f881d2d1700 1 Processor -- start 2024-01-27T02:44:15.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.096+0000 7f881d2d1700 1 -- start start 2024-01-27T02:44:15.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.096+0000 7f881d2d1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f881806da50 0x7f88180c0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:15.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.097+0000 7f881d2d1700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88180c7a00 con 0x7f8818156580 2024-01-27T02:44:15.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.097+0000 7f881d2d1700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88180c8850 con 0x7f88180cbe20 2024-01-27T02:44:15.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.097+0000 7f881d2d1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88180c7c40 con 0x7f881806da50 2024-01-27T02:44:15.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.097+0000 7f88177fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f881806da50 0x7f88180c0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:15.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.097+0000 7f88177fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f881806da50 0x7f88180c0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52268/0 (socket says 172.21.15.7:52268) 2024-01-27T02:44:15.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.097+0000 7f8816ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8818156580 0x7f88180cb710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46946/0 (socket says 172.21.15.7:46946) 2024-01-27T02:44:15.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.097+0000 7f88177fe700 1 -- 172.21.15.7:0/3304019370 learned_addr learned my addr 172.21.15.7:0/3304019370 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:15.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.097+0000 7f88177fe700 1 -- 172.21.15.7:0/3304019370 >> v1:172.21.15.188:6789/0 conn(0x7f88180cbe20 legacy=0x7f881806d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:15.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.097+0000 7f88177fe700 1 -- 172.21.15.7:0/3304019370 >> v1:172.21.15.7:6789/0 conn(0x7f8818156580 legacy=0x7f88180cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:15.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.097+0000 7f88177fe700 1 -- 172.21.15.7:0/3304019370 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88180c7f20 con 0x7f881806da50 2024-01-27T02:44:15.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.097+0000 7f88177fe700 1 --2- 172.21.15.7:0/3304019370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f881806da50 0x7f88180c0500 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f8810009f20 tx=0x7f8810009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1512ce44b77cba server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:15.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.097+0000 7f8815ffb700 1 -- 172.21.15.7:0/3304019370 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88100092e0 con 0x7f881806da50 2024-01-27T02:44:15.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.097+0000 7f8815ffb700 1 -- 172.21.15.7:0/3304019370 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881000ba80 con 0x7f881806da50 2024-01-27T02:44:15.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.097+0000 7f8815ffb700 1 -- 172.21.15.7:0/3304019370 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881000be20 con 0x7f881806da50 2024-01-27T02:44:15.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.098+0000 7f881d2d1700 1 -- 172.21.15.7:0/3304019370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f881806da50 msgr2=0x7f88180c0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:15.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.098+0000 7f881d2d1700 1 --2- 172.21.15.7:0/3304019370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f881806da50 0x7f88180c0500 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f8810009f20 tx=0x7f8810009840 comp rx=0 tx=0).stop 2024-01-27T02:44:15.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.098+0000 7f881d2d1700 1 -- 172.21.15.7:0/3304019370 shutdown_connections 2024-01-27T02:44:15.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.098+0000 7f881d2d1700 1 --2- 172.21.15.7:0/3304019370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f881806da50 0x7f88180c0500 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.098+0000 7f881d2d1700 1 -- 172.21.15.7:0/3304019370 >> 172.21.15.7:0/3304019370 conn(0x7f8818063580 msgr2=0x7f8818063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:15.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.098+0000 7f881d2d1700 1 -- 172.21.15.7:0/3304019370 shutdown_connections 2024-01-27T02:44:15.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.098+0000 7f881d2d1700 1 -- 172.21.15.7:0/3304019370 wait complete. 2024-01-27T02:44:15.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.099+0000 7f881d2d1700 1 Processor -- start 2024-01-27T02:44:15.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.099+0000 7f881d2d1700 1 -- start start 2024-01-27T02:44:15.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.100+0000 7f881d2d1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f881806da50 0x7f8818156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:15.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.100+0000 7f881d2d1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88180cbe20 0x7f8818158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:15.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.100+0000 7f881d2d1700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8818156580 0x7f8818159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:15.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.100+0000 7f881d2d1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8818157030 con 0x7f88180cbe20 2024-01-27T02:44:15.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.100+0000 7f881d2d1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8818157310 con 0x7f881806da50 2024-01-27T02:44:15.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.100+0000 7f881d2d1700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88181575f0 con 0x7f8818156580 2024-01-27T02:44:15.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.100+0000 7f88167fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88180cbe20 0x7f8818158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:15.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.100+0000 7f88167fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88180cbe20 0x7f8818158c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52274/0 (socket says 172.21.15.7:52274) 2024-01-27T02:44:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.100+0000 7f88167fc700 1 -- 172.21.15.7:0/2734607553 learned_addr learned my addr 172.21.15.7:0/2734607553 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.100+0000 7f8816ffd700 1 --2- 172.21.15.7:0/2734607553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f881806da50 0x7f8818156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.100+0000 7f88177fe700 1 --2- 172.21.15.7:0/2734607553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8818156580 0x7f8818159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.101+0000 7f88167fc700 1 -- 172.21.15.7:0/2734607553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8818156580 msgr2=0x7f8818159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.101+0000 7f88167fc700 1 --2- 172.21.15.7:0/2734607553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8818156580 0x7f8818159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.101+0000 7f88167fc700 1 -- 172.21.15.7:0/2734607553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f881806da50 msgr2=0x7f8818156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.101+0000 7f88167fc700 1 --2- 172.21.15.7:0/2734607553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f881806da50 0x7f8818156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.101+0000 7f88167fc700 1 -- 172.21.15.7:0/2734607553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8810009ae0 con 0x7f88180cbe20 2024-01-27T02:44:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.101+0000 7f88167fc700 1 --2- 172.21.15.7:0/2734607553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88180cbe20 0x7f8818158c40 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f880c00c590 tx=0x7f880c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.101+0000 7f87feffd700 1 -- 172.21.15.7:0/2734607553 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880c011590 con 0x7f88180cbe20 2024-01-27T02:44:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.101+0000 7f87feffd700 1 -- 172.21.15.7:0/2734607553 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f880c011710 con 0x7f88180cbe20 2024-01-27T02:44:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.101+0000 7f881d2d1700 1 -- 172.21.15.7:0/2734607553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88181578d0 con 0x7f88180cbe20 2024-01-27T02:44:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.101+0000 7f87feffd700 1 -- 172.21.15.7:0/2734607553 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880c011ad0 con 0x7f88180cbe20 2024-01-27T02:44:15.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.101+0000 7f881d2d1700 1 -- 172.21.15.7:0/2734607553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88181580d0 con 0x7f88180cbe20 2024-01-27T02:44:15.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.102+0000 7f881d2d1700 1 -- 172.21.15.7:0/2734607553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8818157df0 con 0x7f88180cbe20 2024-01-27T02:44:15.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.103+0000 7f87feffd700 1 -- 172.21.15.7:0/2734607553 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f880c01b3b0 con 0x7f88180cbe20 2024-01-27T02:44:15.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.103+0000 7f87feffd700 1 --2- 172.21.15.7:0/2734607553 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8804066f30 0x7f88040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:15.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.103+0000 7f87feffd700 1 -- 172.21.15.7:0/2734607553 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f880c0836a0 con 0x7f88180cbe20 2024-01-27T02:44:15.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.103+0000 7f8816ffd700 1 --2- 172.21.15.7:0/2734607553 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8804066f30 0x7f88040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:15.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.104+0000 7f8816ffd700 1 --2- 172.21.15.7:0/2734607553 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8804066f30 0x7f88040693e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8808004fd0 tx=0x7f8808007eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:15.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.107+0000 7f87feffd700 1 -- 172.21.15.7:0/2734607553 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8818157df0 con 0x7f88180cbe20 2024-01-27T02:44:15.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.312+0000 7f881d2d1700 1 -- 172.21.15.7:0/2734607553 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f8818157df0 con 0x7f8804066f30 2024-01-27T02:44:15.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.312+0000 7f87feffd700 1 -- 172.21.15.7:0/2734607553 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8818157df0 con 0x7f8804066f30 2024-01-27T02:44:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.315+0000 7f881d2d1700 1 -- 172.21.15.7:0/2734607553 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8804066f30 msgr2=0x7f88040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.315+0000 7f881d2d1700 1 --2- 172.21.15.7:0/2734607553 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8804066f30 0x7f88040693e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8808004fd0 tx=0x7f8808007eb0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.315+0000 7f881d2d1700 1 -- 172.21.15.7:0/2734607553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88180cbe20 msgr2=0x7f8818158c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.315+0000 7f881d2d1700 1 --2- 172.21.15.7:0/2734607553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88180cbe20 0x7f8818158c40 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f880c00c590 tx=0x7f880c00c8a0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.315+0000 7f881d2d1700 1 -- 172.21.15.7:0/2734607553 shutdown_connections 2024-01-27T02:44:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.315+0000 7f881d2d1700 1 --2- 172.21.15.7:0/2734607553 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8804066f30 0x7f88040693e0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.315+0000 7f881d2d1700 1 --2- 172.21.15.7:0/2734607553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8818156580 0x7f8818159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.315+0000 7f881d2d1700 1 --2- 172.21.15.7:0/2734607553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f881806da50 0x7f8818156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.315+0000 7f881d2d1700 1 --2- 172.21.15.7:0/2734607553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88180cbe20 0x7f8818158c40 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.315+0000 7f881d2d1700 1 -- 172.21.15.7:0/2734607553 >> 172.21.15.7:0/2734607553 conn(0x7f8818063580 msgr2=0x7f8818154180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.315+0000 7f881d2d1700 1 -- 172.21.15.7:0/2734607553 shutdown_connections 2024-01-27T02:44:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.315+0000 7f881d2d1700 1 -- 172.21.15.7:0/2734607553 wait complete. 2024-01-27T02:44:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 4 to deep-scrub 2024-01-27T02:44:15.330 INFO:tasks.ceph:Scrubbing osd.5 2024-01-27T02:44:15.330 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-01-27T02:44:15.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.465+0000 7f6f3d1ad700 1 Processor -- start 2024-01-27T02:44:15.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.465+0000 7f6f3d1ad700 1 -- start start 2024-01-27T02:44:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.466+0000 7f6f3d1ad700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6f380a7b80 0x7f6f380aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.466+0000 7f6f3d1ad700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f380a7f50 0x7f6f380b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.466+0000 7f6f3d1ad700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6f380b1900 0x7f6f3815be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.466+0000 7f6f3d1ad700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f380b0f30 con 0x7f6f380b1900 2024-01-27T02:44:15.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.466+0000 7f6f3d1ad700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f380b0320 con 0x7f6f380a7b80 2024-01-27T02:44:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.466+0000 7f6f3d1ad700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f380b0600 con 0x7f6f380a7f50 2024-01-27T02:44:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.466+0000 7f6f3759e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6f380b1900 0x7f6f3815be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.466+0000 7f6f3759e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6f380b1900 0x7f6f3815be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52276/0 (socket says 172.21.15.7:52276) 2024-01-27T02:44:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.466+0000 7f6f3759e700 1 -- 172.21.15.7:0/1966904311 learned_addr learned my addr 172.21.15.7:0/1966904311 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.466+0000 7f6f3659c700 1 --2- 172.21.15.7:0/1966904311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f380a7f50 0x7f6f380b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.466+0000 7f6f36d9d700 1 --2- 172.21.15.7:0/1966904311 >> v2:172.21.15.188:3300/0 conn(0x7f6f380a7b80 0x7f6f380aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.467+0000 7f6f3759e700 1 -- 172.21.15.7:0/1966904311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f380a7f50 msgr2=0x7f6f380b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.467+0000 7f6f3759e700 1 --2- 172.21.15.7:0/1966904311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f380a7f50 0x7f6f380b13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.467+0000 7f6f3759e700 1 -- 172.21.15.7:0/1966904311 >> v2:172.21.15.188:3300/0 conn(0x7f6f380a7b80 msgr2=0x7f6f380aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:15.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.467+0000 7f6f3759e700 1 --2- 172.21.15.7:0/1966904311 >> v2:172.21.15.188:3300/0 conn(0x7f6f380a7b80 0x7f6f380aecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.467+0000 7f6f3759e700 1 -- 172.21.15.7:0/1966904311 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f380b08e0 con 0x7f6f380b1900 2024-01-27T02:44:15.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.467+0000 7f6f3759e700 1 --2- 172.21.15.7:0/1966904311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f380b1900 0x7f6f3815be10 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f6f380b33e0 tx=0x7f6f30009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1352f06b09c9be server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:15.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.467+0000 7f6f35d9b700 1 -- 172.21.15.7:0/1966904311 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3000e070 con 0x7f6f380b1900 2024-01-27T02:44:15.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.467+0000 7f6f35d9b700 1 -- 172.21.15.7:0/1966904311 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3000ba50 con 0x7f6f380b1900 2024-01-27T02:44:15.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.467+0000 7f6f35d9b700 1 -- 172.21.15.7:0/1966904311 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f300092e0 con 0x7f6f380b1900 2024-01-27T02:44:15.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.468+0000 7f6f3d1ad700 1 -- 172.21.15.7:0/1966904311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f380b1900 msgr2=0x7f6f3815be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:15.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.468+0000 7f6f3d1ad700 1 --2- 172.21.15.7:0/1966904311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f380b1900 0x7f6f3815be10 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f6f380b33e0 tx=0x7f6f30009840 comp rx=0 tx=0).stop 2024-01-27T02:44:15.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.468+0000 7f6f3d1ad700 1 -- 172.21.15.7:0/1966904311 shutdown_connections 2024-01-27T02:44:15.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.468+0000 7f6f3d1ad700 1 --2- 172.21.15.7:0/1966904311 >> v2:172.21.15.188:3300/0 conn(0x7f6f380a7b80 0x7f6f380aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.468+0000 7f6f3d1ad700 1 --2- 172.21.15.7:0/1966904311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f380b1900 0x7f6f3815be10 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.468+0000 7f6f3d1ad700 1 --2- 172.21.15.7:0/1966904311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f380a7f50 0x7f6f380b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.468+0000 7f6f3d1ad700 1 -- 172.21.15.7:0/1966904311 >> 172.21.15.7:0/1966904311 conn(0x7f6f380679b0 msgr2=0x7f6f38067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.468+0000 7f6f3d1ad700 1 -- 172.21.15.7:0/1966904311 shutdown_connections 2024-01-27T02:44:15.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.468+0000 7f6f3d1ad700 1 -- 172.21.15.7:0/1966904311 wait complete. 2024-01-27T02:44:15.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.469+0000 7f6f3d1ad700 1 Processor -- start 2024-01-27T02:44:15.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.469+0000 7f6f3d1ad700 1 -- start start 2024-01-27T02:44:15.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.470+0000 7f6f3d1ad700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f380a7b80 0x7f6f3816bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:15.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.470+0000 7f6f3d1ad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f380a7f50 0x7f6f38167c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:15.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.470+0000 7f6f3d1ad700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f380b1900 0x7f6f3815e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:15.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.470+0000 7f6f3d1ad700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f38168ba0 con 0x7f6f380a7f50 2024-01-27T02:44:15.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.470+0000 7f6f3d1ad700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f381685e0 con 0x7f6f380b1900 2024-01-27T02:44:15.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.470+0000 7f6f3d1ad700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f381688c0 con 0x7f6f380a7b80 2024-01-27T02:44:15.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.470+0000 7f6f3659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f380a7f50 0x7f6f38167c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:15.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.470+0000 7f6f3659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f380a7f50 0x7f6f38167c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52286/0 (socket says 172.21.15.7:52286) 2024-01-27T02:44:15.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.470+0000 7f6f3659c700 1 -- 172.21.15.7:0/753160275 learned_addr learned my addr 172.21.15.7:0/753160275 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:15.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.470+0000 7f6f36d9d700 1 --2- 172.21.15.7:0/753160275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f380a7b80 0x7f6f3816bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:15.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.470+0000 7f6f3759e700 1 --2- 172.21.15.7:0/753160275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f380b1900 0x7f6f3815e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:15.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.470+0000 7f6f3659c700 1 -- 172.21.15.7:0/753160275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f380a7b80 msgr2=0x7f6f3816bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:15.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.470+0000 7f6f3659c700 1 --2- 172.21.15.7:0/753160275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f380a7b80 0x7f6f3816bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.470+0000 7f6f3659c700 1 -- 172.21.15.7:0/753160275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f380b1900 msgr2=0x7f6f3815e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:15.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.470+0000 7f6f3659c700 1 --2- 172.21.15.7:0/753160275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f380b1900 0x7f6f3815e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.471+0000 7f6f3659c700 1 -- 172.21.15.7:0/753160275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f30009ae0 con 0x7f6f380a7f50 2024-01-27T02:44:15.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.471+0000 7f6f3759e700 1 --2- 172.21.15.7:0/753160275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f380b1900 0x7f6f3815e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:44:15.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.471+0000 7f6f3659c700 1 --2- 172.21.15.7:0/753160275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f380a7f50 0x7f6f38167c00 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f6f2000d890 tx=0x7f6f20008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:15.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.471+0000 7f6f2effd700 1 -- 172.21.15.7:0/753160275 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2000b600 con 0x7f6f380a7f50 2024-01-27T02:44:15.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.471+0000 7f6f2effd700 1 -- 172.21.15.7:0/753160275 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2000b760 con 0x7f6f380a7f50 2024-01-27T02:44:15.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.471+0000 7f6f2effd700 1 -- 172.21.15.7:0/753160275 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2000bae0 con 0x7f6f380a7f50 2024-01-27T02:44:15.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.471+0000 7f6f3d1ad700 1 -- 172.21.15.7:0/753160275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f38168ed0 con 0x7f6f380a7f50 2024-01-27T02:44:15.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.471+0000 7f6f3d1ad700 1 -- 172.21.15.7:0/753160275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f38168140 con 0x7f6f380a7f50 2024-01-27T02:44:15.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.472+0000 7f6f3d1ad700 1 -- 172.21.15.7:0/753160275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6f38168d10 con 0x7f6f380a7f50 2024-01-27T02:44:15.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.473+0000 7f6f2effd700 1 -- 172.21.15.7:0/753160275 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6f2001e460 con 0x7f6f380a7f50 2024-01-27T02:44:15.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.473+0000 7f6f2effd700 1 --2- 172.21.15.7:0/753160275 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6f18066f30 0x7f6f180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:15.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.473+0000 7f6f2effd700 1 -- 172.21.15.7:0/753160275 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6f20085330 con 0x7f6f380a7f50 2024-01-27T02:44:15.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.473+0000 7f6f2effd700 1 --2- 172.21.15.7:0/753160275 >> [v2:172.21.15.188:6800/39373,v1:172.21.15.188:6802/39373] conn(0x7f6f1806cd80 0x7f6f1806f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:15.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.473+0000 7f6f2effd700 1 -- 172.21.15.7:0/753160275 --> [v2:172.21.15.188:6800/39373,v1:172.21.15.188:6802/39373] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6f1806f860 con 0x7f6f1806cd80 2024-01-27T02:44:15.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.473+0000 7f6f2effd700 1 -- 172.21.15.7:0/753160275 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=187) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f20085700 con 0x7f6f380a7f50 2024-01-27T02:44:15.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.473+0000 7f6f36d9d700 1 --2- 172.21.15.7:0/753160275 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6f18066f30 0x7f6f180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:15.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.473+0000 7f6f3759e700 1 --2- 172.21.15.7:0/753160275 >> [v2:172.21.15.188:6800/39373,v1:172.21.15.188:6802/39373] conn(0x7f6f1806cd80 0x7f6f1806f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:15.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.474+0000 7f6f36d9d700 1 --2- 172.21.15.7:0/753160275 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6f18066f30 0x7f6f180693e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6f28006aa0 tx=0x7f6f28008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:15.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.474+0000 7f6f3759e700 1 --2- 172.21.15.7:0/753160275 >> [v2:172.21.15.188:6800/39373,v1:172.21.15.188:6802/39373] conn(0x7f6f1806cd80 0x7f6f1806f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:15.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.475+0000 7f6f2effd700 1 -- 172.21.15.7:0/753160275 <== osd.5 v2:172.21.15.188:6800/39373 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f6f1806f860 con 0x7f6f1806cd80 2024-01-27T02:44:15.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.488+0000 7f6f3d1ad700 1 -- 172.21.15.7:0/753160275 --> [v2:172.21.15.188:6800/39373,v1:172.21.15.188:6802/39373] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6f38168360 con 0x7f6f1806cd80 2024-01-27T02:44:15.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.489+0000 7f6f2effd700 1 -- 172.21.15.7:0/753160275 <== osd.5 v2:172.21.15.188:6800/39373 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7f6f38168360 con 0x7f6f1806cd80 2024-01-27T02:44:15.490 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:44:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-01-27T02:44:15.490 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:44:15.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.489+0000 7f6f3d1ad700 1 -- 172.21.15.7:0/753160275 >> [v2:172.21.15.188:6800/39373,v1:172.21.15.188:6802/39373] conn(0x7f6f1806cd80 msgr2=0x7f6f1806f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:15.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.489+0000 7f6f3d1ad700 1 --2- 172.21.15.7:0/753160275 >> [v2:172.21.15.188:6800/39373,v1:172.21.15.188:6802/39373] conn(0x7f6f1806cd80 0x7f6f1806f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.489+0000 7f6f3d1ad700 1 -- 172.21.15.7:0/753160275 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6f18066f30 msgr2=0x7f6f180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:15.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.489+0000 7f6f3d1ad700 1 --2- 172.21.15.7:0/753160275 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f6f18066f30 0x7f6f180693e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6f28006aa0 tx=0x7f6f28008040 comp rx=0 tx=0).stop 2024-01-27T02:44:15.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.489+0000 7f6f3d1ad700 1 -- 172.21.15.7:0/753160275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f380a7f50 msgr2=0x7f6f38167c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:15.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.489+0000 7f6f3d1ad700 1 --2- 172.21.15.7:0/753160275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f380a7f50 0x7f6f38167c00 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f6f2000d890 tx=0x7f6f20008040 comp rx=0 tx=0).stop 2024-01-27T02:44:15.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.489+0000 7f6f3759e700 1 -- 172.21.15.7:0/753160275 reap_dead start 2024-01-27T02:44:15.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.489+0000 7f6f3d1ad700 1 -- 172.21.15.7:0/753160275 shutdown_connections 2024-01-27T02:44:15.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.489+0000 7f6f3d1ad700 1 -- 172.21.15.7:0/753160275 >> 172.21.15.7:0/753160275 conn(0x7f6f380679b0 msgr2=0x7f6f38158ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:15.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.489+0000 7f6f3d1ad700 1 -- 172.21.15.7:0/753160275 shutdown_connections 2024-01-27T02:44:15.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.489+0000 7f6f3d1ad700 1 -- 172.21.15.7:0/753160275 wait complete. 2024-01-27T02:44:15.504 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-01-27T02:44:15.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.635+0000 7f8b33e85700 1 Processor -- start 2024-01-27T02:44:15.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.635+0000 7f8b33e85700 1 -- start start 2024-01-27T02:44:15.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.636+0000 7f8b33e85700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8b2c157ca0 0x7f8b2c0c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:15.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.636+0000 7f8b33e85700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8b2c0c3ae0 0x7f8b2c0cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:15.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.636+0000 7f8b33e85700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b2c0ca020 con 0x7f8b2c158070 2024-01-27T02:44:15.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.636+0000 7f8b33e85700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b2c0c4f40 con 0x7f8b2c157ca0 2024-01-27T02:44:15.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.636+0000 7f8b33e85700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b2c0c5210 con 0x7f8b2c0c3ae0 2024-01-27T02:44:15.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.636+0000 7f8b31c21700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8b2c157ca0 0x7f8b2c0c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:15.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.636+0000 7f8b31c21700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8b2c157ca0 0x7f8b2c0c4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52290/0 (socket says 172.21.15.7:52290) 2024-01-27T02:44:15.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.636+0000 7f8b31c21700 1 -- 172.21.15.7:0/1481336535 learned_addr learned my addr 172.21.15.7:0/1481336535 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:15.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.636+0000 7f8b31420700 1 --2- 172.21.15.7:0/1481336535 >> v2:172.21.15.188:3300/0 conn(0x7f8b2c0c3ae0 0x7f8b2c0cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:15.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.637+0000 7f8b31c21700 1 -- 172.21.15.7:0/1481336535 >> v2:172.21.15.188:3300/0 conn(0x7f8b2c0c3ae0 msgr2=0x7f8b2c0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:15.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.637+0000 7f8b31c21700 1 --2- 172.21.15.7:0/1481336535 >> v2:172.21.15.188:3300/0 conn(0x7f8b2c0c3ae0 0x7f8b2c0cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.637+0000 7f8b31c21700 1 -- 172.21.15.7:0/1481336535 >> v1:172.21.15.188:6789/0 conn(0x7f8b2c158070 legacy=0x7f8b2c15bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:15.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.637+0000 7f8b31c21700 1 -- 172.21.15.7:0/1481336535 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2c0c54f0 con 0x7f8b2c157ca0 2024-01-27T02:44:15.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.637+0000 7f8b31c21700 1 --2- 172.21.15.7:0/1481336535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b2c157ca0 0x7f8b2c0c4a00 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c06e170 tx=0x7f8b28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc2b21ddb709cb1 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:15.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.637+0000 7f8b30c1f700 1 -- 172.21.15.7:0/1481336535 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b28009e70 con 0x7f8b2c157ca0 2024-01-27T02:44:15.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.637+0000 7f8b30c1f700 1 -- 172.21.15.7:0/1481336535 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2800b860 con 0x7f8b2c157ca0 2024-01-27T02:44:15.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.637+0000 7f8b30c1f700 1 -- 172.21.15.7:0/1481336535 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2800bc00 con 0x7f8b2c157ca0 2024-01-27T02:44:15.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.638+0000 7f8b33e85700 1 -- 172.21.15.7:0/1481336535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b2c157ca0 msgr2=0x7f8b2c0c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:15.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.638+0000 7f8b33e85700 1 --2- 172.21.15.7:0/1481336535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b2c157ca0 0x7f8b2c0c4a00 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c06e170 tx=0x7f8b28009250 comp rx=0 tx=0).stop 2024-01-27T02:44:15.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.638+0000 7f8b33e85700 1 -- 172.21.15.7:0/1481336535 shutdown_connections 2024-01-27T02:44:15.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.638+0000 7f8b33e85700 1 --2- 172.21.15.7:0/1481336535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b2c157ca0 0x7f8b2c0c4a00 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.638+0000 7f8b33e85700 1 --2- 172.21.15.7:0/1481336535 >> v2:172.21.15.188:3300/0 conn(0x7f8b2c0c3ae0 0x7f8b2c0cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.638+0000 7f8b33e85700 1 -- 172.21.15.7:0/1481336535 >> 172.21.15.7:0/1481336535 conn(0x7f8b2c098020 msgr2=0x7f8b2c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:15.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.638+0000 7f8b33e85700 1 -- 172.21.15.7:0/1481336535 shutdown_connections 2024-01-27T02:44:15.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.638+0000 7f8b33e85700 1 -- 172.21.15.7:0/1481336535 wait complete. 2024-01-27T02:44:15.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.639+0000 7f8b33e85700 1 Processor -- start 2024-01-27T02:44:15.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.639+0000 7f8b33e85700 1 -- start start 2024-01-27T02:44:15.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.640+0000 7f8b33e85700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b2c0c3ae0 0x7f8b2c0c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:15.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.640+0000 7f8b33e85700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b2c157ca0 0x7f8b2c155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:15.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.640+0000 7f8b33e85700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b2c158070 0x7f8b2c14fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:15.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.640+0000 7f8b33e85700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c0b21f0 con 0x7f8b2c158070 2024-01-27T02:44:15.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.640+0000 7f8b33e85700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c157180 con 0x7f8b2c0c3ae0 2024-01-27T02:44:15.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.640+0000 7f8b33e85700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c157460 con 0x7f8b2c157ca0 2024-01-27T02:44:15.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.640+0000 7f8b32422700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b2c158070 0x7f8b2c14fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:15.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.640+0000 7f8b32422700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b2c158070 0x7f8b2c14fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52298/0 (socket says 172.21.15.7:52298) 2024-01-27T02:44:15.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.640+0000 7f8b32422700 1 -- 172.21.15.7:0/1017626541 learned_addr learned my addr 172.21.15.7:0/1017626541 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:15.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.640+0000 7f8b31c21700 1 --2- 172.21.15.7:0/1017626541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b2c0c3ae0 0x7f8b2c0c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:15.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.640+0000 7f8b31420700 1 --2- 172.21.15.7:0/1017626541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b2c157ca0 0x7f8b2c155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:15.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.640+0000 7f8b32422700 1 -- 172.21.15.7:0/1017626541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b2c157ca0 msgr2=0x7f8b2c155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:15.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.640+0000 7f8b32422700 1 --2- 172.21.15.7:0/1017626541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b2c157ca0 0x7f8b2c155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.640+0000 7f8b32422700 1 -- 172.21.15.7:0/1017626541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b2c0c3ae0 msgr2=0x7f8b2c0c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:15.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.640+0000 7f8b32422700 1 --2- 172.21.15.7:0/1017626541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b2c0c3ae0 0x7f8b2c0c5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.640+0000 7f8b32422700 1 -- 172.21.15.7:0/1017626541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b280094f0 con 0x7f8b2c158070 2024-01-27T02:44:15.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.640+0000 7f8b31420700 1 --2- 172.21.15.7:0/1017626541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b2c157ca0 0x7f8b2c155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:44:15.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.640+0000 7f8b31c21700 1 --2- 172.21.15.7:0/1017626541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b2c0c3ae0 0x7f8b2c0c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:44:15.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.641+0000 7f8b32422700 1 --2- 172.21.15.7:0/1017626541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b2c158070 0x7f8b2c14fed0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400c840 tx=0x7f8b24001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:15.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.641+0000 7f8b21ffb700 1 -- 172.21.15.7:0/1017626541 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24012630 con 0x7f8b2c158070 2024-01-27T02:44:15.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.641+0000 7f8b21ffb700 1 -- 172.21.15.7:0/1017626541 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b240127b0 con 0x7f8b2c158070 2024-01-27T02:44:15.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.641+0000 7f8b21ffb700 1 -- 172.21.15.7:0/1017626541 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24012b70 con 0x7f8b2c158070 2024-01-27T02:44:15.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.641+0000 7f8b33e85700 1 -- 172.21.15.7:0/1017626541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b2c0b1ba0 con 0x7f8b2c158070 2024-01-27T02:44:15.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.641+0000 7f8b33e85700 1 -- 172.21.15.7:0/1017626541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b2c156420 con 0x7f8b2c158070 2024-01-27T02:44:15.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.642+0000 7f8b33e85700 1 -- 172.21.15.7:0/1017626541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b2c0b23a0 con 0x7f8b2c158070 2024-01-27T02:44:15.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.642+0000 7f8b21ffb700 1 -- 172.21.15.7:0/1017626541 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8b24003030 con 0x7f8b2c158070 2024-01-27T02:44:15.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.643+0000 7f8b21ffb700 1 --2- 172.21.15.7:0/1017626541 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b10066f30 0x7f8b100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:15.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.643+0000 7f8b21ffb700 1 -- 172.21.15.7:0/1017626541 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8b240841a0 con 0x7f8b2c158070 2024-01-27T02:44:15.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.643+0000 7f8b31c21700 1 --2- 172.21.15.7:0/1017626541 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b10066f30 0x7f8b100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:15.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.644+0000 7f8b31c21700 1 --2- 172.21.15.7:0/1017626541 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b10066f30 0x7f8b100693e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c06e170 tx=0x7f8b28017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.647+0000 7f8b21ffb700 1 -- 172.21.15.7:0/1017626541 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b2c0b23a0 con 0x7f8b2c158070 2024-01-27T02:44:15.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.850+0000 7f8b33e85700 1 -- 172.21.15.7:0/1017626541 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b2c06ec20 con 0x7f8b10066f30 2024-01-27T02:44:15.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.851+0000 7f8b21ffb700 1 -- 172.21.15.7:0/1017626541 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8b2c06ec20 con 0x7f8b10066f30 2024-01-27T02:44:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.853+0000 7f8b33e85700 1 -- 172.21.15.7:0/1017626541 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b10066f30 msgr2=0x7f8b100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.853+0000 7f8b33e85700 1 --2- 172.21.15.7:0/1017626541 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b10066f30 0x7f8b100693e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c06e170 tx=0x7f8b28017000 comp rx=0 tx=0).stop 2024-01-27T02:44:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.853+0000 7f8b33e85700 1 -- 172.21.15.7:0/1017626541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b2c158070 msgr2=0x7f8b2c14fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.853+0000 7f8b33e85700 1 --2- 172.21.15.7:0/1017626541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b2c158070 0x7f8b2c14fed0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400c840 tx=0x7f8b24001670 comp rx=0 tx=0).stop 2024-01-27T02:44:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.853+0000 7f8b33e85700 1 -- 172.21.15.7:0/1017626541 shutdown_connections 2024-01-27T02:44:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.853+0000 7f8b33e85700 1 --2- 172.21.15.7:0/1017626541 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f8b10066f30 0x7f8b100693e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.854+0000 7f8b33e85700 1 --2- 172.21.15.7:0/1017626541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b2c158070 0x7f8b2c14fed0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.854+0000 7f8b33e85700 1 --2- 172.21.15.7:0/1017626541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b2c0c3ae0 0x7f8b2c0c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.854+0000 7f8b33e85700 1 --2- 172.21.15.7:0/1017626541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b2c157ca0 0x7f8b2c155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.854+0000 7f8b33e85700 1 -- 172.21.15.7:0/1017626541 >> 172.21.15.7:0/1017626541 conn(0x7f8b2c098020 msgr2=0x7f8b2c06bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.854+0000 7f8b33e85700 1 -- 172.21.15.7:0/1017626541 shutdown_connections 2024-01-27T02:44:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:15.854+0000 7f8b33e85700 1 -- 172.21.15.7:0/1017626541 wait complete. 2024-01-27T02:44:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 5 to deep-scrub 2024-01-27T02:44:15.869 INFO:tasks.ceph:Scrubbing osd.6 2024-01-27T02:44:15.869 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-01-27T02:44:16.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.004+0000 7fee19809700 1 Processor -- start 2024-01-27T02:44:16.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.004+0000 7fee19809700 1 -- start start 2024-01-27T02:44:16.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.005+0000 7fee19809700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fee1415a230 0x7fee140ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.005+0000 7fee19809700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee140c4cc0 0x7fee140679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.005+0000 7fee19809700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee14067ef0 con 0x7fee140cd2f0 2024-01-27T02:44:16.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.005+0000 7fee19809700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee140cd770 con 0x7fee1415a230 2024-01-27T02:44:16.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.005+0000 7fee19809700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee140c5320 con 0x7fee140c4cc0 2024-01-27T02:44:16.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.005+0000 7fee127fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fee140cd2f0 0x7fee140c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46958/0 (socket says 172.21.15.7:46958) 2024-01-27T02:44:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.006+0000 7fee127fc700 1 -- 172.21.15.7:0/634993492 learned_addr learned my addr 172.21.15.7:0/634993492 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.005+0000 7fee12ffd700 1 --2- 172.21.15.7:0/634993492 >> v2:172.21.15.7:3300/0 conn(0x7fee1415a230 0x7fee140ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.006+0000 7fee137fe700 1 --2- 172.21.15.7:0/634993492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee140c4cc0 0x7fee140679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.006+0000 7fee11ffb700 1 -- 172.21.15.7:0/634993492 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1162382741 0 0) 0x7fee14067ef0 con 0x7fee140cd2f0 2024-01-27T02:44:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.006+0000 7fee11ffb700 1 -- 172.21.15.7:0/634993492 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedfc003570 con 0x7fee140cd2f0 2024-01-27T02:44:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.006+0000 7fee12ffd700 1 -- 172.21.15.7:0/634993492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee140c4cc0 msgr2=0x7fee140679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.006+0000 7fee12ffd700 1 --2- 172.21.15.7:0/634993492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee140c4cc0 0x7fee140679b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.006+0000 7fee12ffd700 1 -- 172.21.15.7:0/634993492 >> v1:172.21.15.7:6789/0 conn(0x7fee140cd2f0 legacy=0x7fee140c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.006+0000 7fee12ffd700 1 -- 172.21.15.7:0/634993492 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee140c9520 con 0x7fee1415a230 2024-01-27T02:44:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.006+0000 7fee12ffd700 1 --2- 172.21.15.7:0/634993492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1415a230 0x7fee140ce310 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fee040098d0 tx=0x7fee04009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e6c8e7f7868d45 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.006+0000 7fee11ffb700 1 -- 172.21.15.7:0/634993492 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee04005c10 con 0x7fee1415a230 2024-01-27T02:44:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.006+0000 7fee11ffb700 1 -- 172.21.15.7:0/634993492 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee04005d70 con 0x7fee1415a230 2024-01-27T02:44:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.006+0000 7fee11ffb700 1 -- 172.21.15.7:0/634993492 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee04006280 con 0x7fee1415a230 2024-01-27T02:44:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.007+0000 7fee19809700 1 -- 172.21.15.7:0/634993492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1415a230 msgr2=0x7fee140ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.007+0000 7fee19809700 1 --2- 172.21.15.7:0/634993492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1415a230 0x7fee140ce310 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fee040098d0 tx=0x7fee04009120 comp rx=0 tx=0).stop 2024-01-27T02:44:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.007+0000 7fee19809700 1 -- 172.21.15.7:0/634993492 shutdown_connections 2024-01-27T02:44:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.007+0000 7fee19809700 1 --2- 172.21.15.7:0/634993492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee140c4cc0 0x7fee140679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.007+0000 7fee19809700 1 --2- 172.21.15.7:0/634993492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1415a230 0x7fee140ce310 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.007+0000 7fee19809700 1 -- 172.21.15.7:0/634993492 >> 172.21.15.7:0/634993492 conn(0x7fee14098020 msgr2=0x7fee140711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.007+0000 7fee19809700 1 -- 172.21.15.7:0/634993492 shutdown_connections 2024-01-27T02:44:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.007+0000 7fee19809700 1 -- 172.21.15.7:0/634993492 wait complete. 2024-01-27T02:44:16.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.008+0000 7fee19809700 1 Processor -- start 2024-01-27T02:44:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.009+0000 7fee19809700 1 -- start start 2024-01-27T02:44:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.009+0000 7fee19809700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee140c4cc0 0x7fee1416af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.009+0000 7fee19809700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee140cd2f0 0x7fee14156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.009+0000 7fee19809700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1415a230 0x7fee140c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.009+0000 7fee19809700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1416aab0 con 0x7fee1415a230 2024-01-27T02:44:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.009+0000 7fee19809700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee140c7ab0 con 0x7fee140c4cc0 2024-01-27T02:44:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.009+0000 7fee19809700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee140c7d90 con 0x7fee140cd2f0 2024-01-27T02:44:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.009+0000 7fee137fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1415a230 0x7fee140c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.009+0000 7fee137fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1415a230 0x7fee140c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52302/0 (socket says 172.21.15.7:52302) 2024-01-27T02:44:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.009+0000 7fee137fe700 1 -- 172.21.15.7:0/104808055 learned_addr learned my addr 172.21.15.7:0/104808055 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.010+0000 7fee12ffd700 1 --2- 172.21.15.7:0/104808055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee140c4cc0 0x7fee1416af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.010+0000 7fee127fc700 1 --2- 172.21.15.7:0/104808055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee140cd2f0 0x7fee14156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.010+0000 7fee137fe700 1 -- 172.21.15.7:0/104808055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee140cd2f0 msgr2=0x7fee14156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.010+0000 7fee137fe700 1 --2- 172.21.15.7:0/104808055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee140cd2f0 0x7fee14156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.010+0000 7fee137fe700 1 -- 172.21.15.7:0/104808055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee140c4cc0 msgr2=0x7fee1416af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.010+0000 7fee137fe700 1 --2- 172.21.15.7:0/104808055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee140c4cc0 0x7fee1416af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.010+0000 7fee137fe700 1 -- 172.21.15.7:0/104808055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee040093c0 con 0x7fee1415a230 2024-01-27T02:44:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.010+0000 7fee137fe700 1 --2- 172.21.15.7:0/104808055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1415a230 0x7fee140c8810 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009ec0 tx=0x7fee0c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.010+0000 7fedfaffd700 1 -- 172.21.15.7:0/104808055 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c00cc20 con 0x7fee1415a230 2024-01-27T02:44:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.010+0000 7fedfaffd700 1 -- 172.21.15.7:0/104808055 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0c00f040 con 0x7fee1415a230 2024-01-27T02:44:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.010+0000 7fee19809700 1 -- 172.21.15.7:0/104808055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee1416a460 con 0x7fee1415a230 2024-01-27T02:44:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.010+0000 7fedfaffd700 1 -- 172.21.15.7:0/104808055 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c014480 con 0x7fee1415a230 2024-01-27T02:44:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.010+0000 7fee19809700 1 -- 172.21.15.7:0/104808055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee140c8d50 con 0x7fee1415a230 2024-01-27T02:44:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.011+0000 7fee19809700 1 -- 172.21.15.7:0/104808055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fee1416a2f0 con 0x7fee1415a230 2024-01-27T02:44:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.012+0000 7fedfaffd700 1 -- 172.21.15.7:0/104808055 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fee0c014810 con 0x7fee1415a230 2024-01-27T02:44:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.012+0000 7fedfaffd700 1 --2- 172.21.15.7:0/104808055 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fee00066f30 0x7fee000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.012+0000 7fedfaffd700 1 -- 172.21.15.7:0/104808055 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fee0c084ff0 con 0x7fee1415a230 2024-01-27T02:44:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.012+0000 7fedfaffd700 1 --2- 172.21.15.7:0/104808055 >> [v2:172.21.15.188:6824/39412,v1:172.21.15.188:6825/39412] conn(0x7fee0006cd80 0x7fee0006f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.012+0000 7fedfaffd700 1 -- 172.21.15.7:0/104808055 --> [v2:172.21.15.188:6824/39412,v1:172.21.15.188:6825/39412] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee0006f860 con 0x7fee0006cd80 2024-01-27T02:44:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.012+0000 7fedfaffd700 1 -- 172.21.15.7:0/104808055 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=187) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee0c087630 con 0x7fee1415a230 2024-01-27T02:44:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.012+0000 7fee12ffd700 1 --2- 172.21.15.7:0/104808055 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fee00066f30 0x7fee000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.013+0000 7fee127fc700 1 --2- 172.21.15.7:0/104808055 >> [v2:172.21.15.188:6824/39412,v1:172.21.15.188:6825/39412] conn(0x7fee0006cd80 0x7fee0006f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.013+0000 7fee12ffd700 1 --2- 172.21.15.7:0/104808055 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fee00066f30 0x7fee000693e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fee040098d0 tx=0x7fee04016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.013+0000 7fee127fc700 1 --2- 172.21.15.7:0/104808055 >> [v2:172.21.15.188:6824/39412,v1:172.21.15.188:6825/39412] conn(0x7fee0006cd80 0x7fee0006f190 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:16.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.014+0000 7fedfaffd700 1 -- 172.21.15.7:0/104808055 <== osd.6 v2:172.21.15.188:6824/39412 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7fee0006f860 con 0x7fee0006cd80 2024-01-27T02:44:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.027+0000 7fee19809700 1 -- 172.21.15.7:0/104808055 --> [v2:172.21.15.188:6824/39412,v1:172.21.15.188:6825/39412] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fee1404c710 con 0x7fee0006cd80 2024-01-27T02:44:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.028+0000 7fedfaffd700 1 -- 172.21.15.7:0/104808055 <== osd.6 v2:172.21.15.188:6824/39412 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7fee1404c710 con 0x7fee0006cd80 2024-01-27T02:44:16.029 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:44:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-01-27T02:44:16.030 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:44:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.028+0000 7fee19809700 1 -- 172.21.15.7:0/104808055 >> [v2:172.21.15.188:6824/39412,v1:172.21.15.188:6825/39412] conn(0x7fee0006cd80 msgr2=0x7fee0006f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.028+0000 7fee19809700 1 --2- 172.21.15.7:0/104808055 >> [v2:172.21.15.188:6824/39412,v1:172.21.15.188:6825/39412] conn(0x7fee0006cd80 0x7fee0006f190 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.028+0000 7fee19809700 1 -- 172.21.15.7:0/104808055 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fee00066f30 msgr2=0x7fee000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.028+0000 7fee19809700 1 --2- 172.21.15.7:0/104808055 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fee00066f30 0x7fee000693e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fee040098d0 tx=0x7fee04016000 comp rx=0 tx=0).stop 2024-01-27T02:44:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.028+0000 7fee19809700 1 -- 172.21.15.7:0/104808055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1415a230 msgr2=0x7fee140c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.028+0000 7fee19809700 1 --2- 172.21.15.7:0/104808055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1415a230 0x7fee140c8810 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009ec0 tx=0x7fee0c00b040 comp rx=0 tx=0).stop 2024-01-27T02:44:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.028+0000 7fee137fe700 1 -- 172.21.15.7:0/104808055 reap_dead start 2024-01-27T02:44:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.028+0000 7fee19809700 1 -- 172.21.15.7:0/104808055 shutdown_connections 2024-01-27T02:44:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.028+0000 7fee19809700 1 -- 172.21.15.7:0/104808055 >> 172.21.15.7:0/104808055 conn(0x7fee14098020 msgr2=0x7fee1406b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.028+0000 7fee19809700 1 -- 172.21.15.7:0/104808055 shutdown_connections 2024-01-27T02:44:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.028+0000 7fee19809700 1 -- 172.21.15.7:0/104808055 wait complete. 2024-01-27T02:44:16.043 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-01-27T02:44:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.174+0000 7f3c1180e700 1 Processor -- start 2024-01-27T02:44:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.174+0000 7f3c1180e700 1 -- start start 2024-01-27T02:44:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.175+0000 7f3c1180e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c0c143860 0x7f3c0c0d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.175+0000 7f3c1180e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3c0c143c30 0x7f3c0c0d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.175+0000 7f3c1180e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c0c0b3d90 con 0x7f3c0c15de00 2024-01-27T02:44:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.175+0000 7f3c1180e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c0c0d0ea0 con 0x7f3c0c143c30 2024-01-27T02:44:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.175+0000 7f3c1180e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c0d1180 con 0x7f3c0c143860 2024-01-27T02:44:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.175+0000 7f3c027fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3c0c143c30 0x7f3c0c0d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.175+0000 7f3c027fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3c0c143c30 0x7f3c0c0d0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52314/0 (socket says 172.21.15.7:52314) 2024-01-27T02:44:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.175+0000 7f3c0b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3c0c15de00 0x7f3c0c0cd190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46968/0 (socket says 172.21.15.7:46968) 2024-01-27T02:44:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.175+0000 7f3c027fc700 1 -- 172.21.15.7:0/688474488 learned_addr learned my addr 172.21.15.7:0/688474488 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.175+0000 7f3c0affd700 1 --2- 172.21.15.7:0/688474488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c0c143860 0x7f3c0c0d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.175+0000 7f3c027fc700 1 -- 172.21.15.7:0/688474488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c0c143860 msgr2=0x7f3c0c0d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.175+0000 7f3c027fc700 1 --2- 172.21.15.7:0/688474488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c0c143860 0x7f3c0c0d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.175+0000 7f3c027fc700 1 -- 172.21.15.7:0/688474488 >> v1:172.21.15.7:6789/0 conn(0x7f3c0c15de00 legacy=0x7f3c0c0cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.175+0000 7f3c027fc700 1 -- 172.21.15.7:0/688474488 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0c0d1460 con 0x7f3c0c143c30 2024-01-27T02:44:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.175+0000 7f3c0a7fc700 1 -- 172.21.15.7:0/688474488 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2754959623 0 0) 0x7f3c0c0b3d90 con 0x7f3c0c15de00 2024-01-27T02:44:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.176+0000 7f3c027fc700 1 --2- 172.21.15.7:0/688474488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c143c30 0x7f3c0c0d0820 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40099b0 tx=0x7f3bf4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb1e6a6d4768d8c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.176+0000 7f3c0a7fc700 1 -- 172.21.15.7:0/688474488 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4009ea0 con 0x7f3c0c143c30 2024-01-27T02:44:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.176+0000 7f3c0a7fc700 1 -- 172.21.15.7:0/688474488 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf400b860 con 0x7f3c0c143c30 2024-01-27T02:44:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.176+0000 7f3c0a7fc700 1 -- 172.21.15.7:0/688474488 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf400bc00 con 0x7f3c0c143c30 2024-01-27T02:44:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.176+0000 7f3c1180e700 1 -- 172.21.15.7:0/688474488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c143c30 msgr2=0x7f3c0c0d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.176+0000 7f3c1180e700 1 --2- 172.21.15.7:0/688474488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c143c30 0x7f3c0c0d0820 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40099b0 tx=0x7f3bf4009250 comp rx=0 tx=0).stop 2024-01-27T02:44:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.176+0000 7f3c1180e700 1 -- 172.21.15.7:0/688474488 shutdown_connections 2024-01-27T02:44:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.176+0000 7f3c1180e700 1 --2- 172.21.15.7:0/688474488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c0c143860 0x7f3c0c0d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.176+0000 7f3c1180e700 1 --2- 172.21.15.7:0/688474488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c143c30 0x7f3c0c0d0820 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.176+0000 7f3c1180e700 1 -- 172.21.15.7:0/688474488 >> 172.21.15.7:0/688474488 conn(0x7f3c0c068440 msgr2=0x7f3c0c0709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.176+0000 7f3c1180e700 1 -- 172.21.15.7:0/688474488 shutdown_connections 2024-01-27T02:44:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.176+0000 7f3c1180e700 1 -- 172.21.15.7:0/688474488 wait complete. 2024-01-27T02:44:16.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.178+0000 7f3c1180e700 1 Processor -- start 2024-01-27T02:44:16.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.178+0000 7f3c1180e700 1 -- start start 2024-01-27T02:44:16.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.179+0000 7f3c1180e700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c0c143860 0x7f3c0c0a7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.179+0000 7f3c1180e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c0c143c30 0x7f3c0c15cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.179+0000 7f3c1180e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c15de00 0x7f3c0c14bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.179+0000 7f3c1180e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c15d1e0 con 0x7f3c0c15de00 2024-01-27T02:44:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.179+0000 7f3c1180e700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c13e200 con 0x7f3c0c143c30 2024-01-27T02:44:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.179+0000 7f3c1180e700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c1494e0 con 0x7f3c0c143860 2024-01-27T02:44:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.179+0000 7f3c0b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c15de00 0x7f3c0c14bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.179+0000 7f3c0b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c15de00 0x7f3c0c14bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52316/0 (socket says 172.21.15.7:52316) 2024-01-27T02:44:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.179+0000 7f3c0b7fe700 1 -- 172.21.15.7:0/83887773 learned_addr learned my addr 172.21.15.7:0/83887773 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.179+0000 7f3c027fc700 1 --2- 172.21.15.7:0/83887773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c0c143c30 0x7f3c0c15cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.179+0000 7f3c0affd700 1 --2- 172.21.15.7:0/83887773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c0c143860 0x7f3c0c0a7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.179+0000 7f3c0b7fe700 1 -- 172.21.15.7:0/83887773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c0c143860 msgr2=0x7f3c0c0a7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.179+0000 7f3c0b7fe700 1 --2- 172.21.15.7:0/83887773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c0c143860 0x7f3c0c0a7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.179+0000 7f3c0b7fe700 1 -- 172.21.15.7:0/83887773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c0c143c30 msgr2=0x7f3c0c15cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.179+0000 7f3c0b7fe700 1 --2- 172.21.15.7:0/83887773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c0c143c30 0x7f3c0c15cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.179+0000 7f3c0b7fe700 1 -- 172.21.15.7:0/83887773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf40094f0 con 0x7f3c0c15de00 2024-01-27T02:44:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.179+0000 7f3c0affd700 1 --2- 172.21.15.7:0/83887773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c0c143860 0x7f3c0c0a7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:44:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.179+0000 7f3c0b7fe700 1 --2- 172.21.15.7:0/83887773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c15de00 0x7f3c0c14bdd0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400ccc0 tx=0x7f3c04001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.179+0000 7f3c037fe700 1 -- 172.21.15.7:0/83887773 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c04005df0 con 0x7f3c0c15de00 2024-01-27T02:44:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.179+0000 7f3c037fe700 1 -- 172.21.15.7:0/83887773 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0400a720 con 0x7f3c0c15de00 2024-01-27T02:44:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.179+0000 7f3c1180e700 1 -- 172.21.15.7:0/83887773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c0c0a8d70 con 0x7f3c0c15de00 2024-01-27T02:44:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.180+0000 7f3c037fe700 1 -- 172.21.15.7:0/83887773 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0400aae0 con 0x7f3c0c15de00 2024-01-27T02:44:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.180+0000 7f3c1180e700 1 -- 172.21.15.7:0/83887773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c0c0a8440 con 0x7f3c0c15de00 2024-01-27T02:44:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.181+0000 7f3c1180e700 1 -- 172.21.15.7:0/83887773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c0c0b3f20 con 0x7f3c0c15de00 2024-01-27T02:44:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.181+0000 7f3c037fe700 1 -- 172.21.15.7:0/83887773 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3c040070f0 con 0x7f3c0c15de00 2024-01-27T02:44:16.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.181+0000 7f3c037fe700 1 --2- 172.21.15.7:0/83887773 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3bec066f30 0x7f3bec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.181+0000 7f3c037fe700 1 -- 172.21.15.7:0/83887773 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f3c04084270 con 0x7f3c0c15de00 2024-01-27T02:44:16.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.182+0000 7f3c0affd700 1 --2- 172.21.15.7:0/83887773 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3bec066f30 0x7f3bec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.182+0000 7f3c0affd700 1 --2- 172.21.15.7:0/83887773 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3bec066f30 0x7f3bec0693e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009d80 tx=0x7f3bfc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:16.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.186+0000 7f3c037fe700 1 -- 172.21.15.7:0/83887773 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c04051860 con 0x7f3c0c15de00 2024-01-27T02:44:16.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.392+0000 7f3c1180e700 1 -- 172.21.15.7:0/83887773 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c0c04c710 con 0x7f3bec066f30 2024-01-27T02:44:16.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.393+0000 7f3c037fe700 1 -- 172.21.15.7:0/83887773 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3c0c04c710 con 0x7f3bec066f30 2024-01-27T02:44:16.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.395+0000 7f3c1180e700 1 -- 172.21.15.7:0/83887773 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3bec066f30 msgr2=0x7f3bec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.395+0000 7f3c1180e700 1 --2- 172.21.15.7:0/83887773 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3bec066f30 0x7f3bec0693e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009d80 tx=0x7f3bfc008040 comp rx=0 tx=0).stop 2024-01-27T02:44:16.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.395+0000 7f3c1180e700 1 -- 172.21.15.7:0/83887773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c15de00 msgr2=0x7f3c0c14bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.395+0000 7f3c1180e700 1 --2- 172.21.15.7:0/83887773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c15de00 0x7f3c0c14bdd0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400ccc0 tx=0x7f3c04001a20 comp rx=0 tx=0).stop 2024-01-27T02:44:16.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.395+0000 7f3c1180e700 1 -- 172.21.15.7:0/83887773 shutdown_connections 2024-01-27T02:44:16.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.395+0000 7f3c1180e700 1 --2- 172.21.15.7:0/83887773 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f3bec066f30 0x7f3bec0693e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.395+0000 7f3c1180e700 1 --2- 172.21.15.7:0/83887773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c15de00 0x7f3c0c14bdd0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.395+0000 7f3c1180e700 1 --2- 172.21.15.7:0/83887773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c0c143860 0x7f3c0c0a7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.395+0000 7f3c1180e700 1 --2- 172.21.15.7:0/83887773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c0c143c30 0x7f3c0c15cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.395+0000 7f3c1180e700 1 -- 172.21.15.7:0/83887773 >> 172.21.15.7:0/83887773 conn(0x7f3c0c068440 msgr2=0x7f3c0c0b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.395+0000 7f3c1180e700 1 -- 172.21.15.7:0/83887773 shutdown_connections 2024-01-27T02:44:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.395+0000 7f3c1180e700 1 -- 172.21.15.7:0/83887773 wait complete. 2024-01-27T02:44:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 6 to deep-scrub 2024-01-27T02:44:16.411 INFO:tasks.ceph:Scrubbing osd.7 2024-01-27T02:44:16.411 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-01-27T02:44:16.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.546+0000 7f47ef62d700 1 Processor -- start 2024-01-27T02:44:16.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.547+0000 7f47ef62d700 1 -- start start 2024-01-27T02:44:16.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.548+0000 7f47ef62d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f47e8157dd0 0x7f47e80adf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.548+0000 7f47ef62d700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f47e80cd9c0 0x7f47e80ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.548+0000 7f47ef62d700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e80c51a0 con 0x7f47e81581a0 2024-01-27T02:44:16.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.548+0000 7f47ef62d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47e80c4590 con 0x7f47e8157dd0 2024-01-27T02:44:16.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.548+0000 7f47ef62d700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47e80c4870 con 0x7f47e80cd9c0 2024-01-27T02:44:16.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.548+0000 7f47ed3c9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f47e8157dd0 0x7f47e80adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.548+0000 7f47ed3c9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f47e8157dd0 0x7f47e80adf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52320/0 (socket says 172.21.15.7:52320) 2024-01-27T02:44:16.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.548+0000 7f47ed3c9700 1 -- 172.21.15.7:0/4132858016 learned_addr learned my addr 172.21.15.7:0/4132858016 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:16.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.548+0000 7f47edbca700 1 --2- 172.21.15.7:0/4132858016 >> v2:172.21.15.188:3300/0 conn(0x7f47e80cd9c0 0x7f47e80ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.548+0000 7f47ed3c9700 1 -- 172.21.15.7:0/4132858016 >> v2:172.21.15.188:3300/0 conn(0x7f47e80cd9c0 msgr2=0x7f47e80ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:16.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.548+0000 7f47ed3c9700 1 --2- 172.21.15.7:0/4132858016 >> v2:172.21.15.188:3300/0 conn(0x7f47e80cd9c0 0x7f47e80ab7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.548+0000 7f47ed3c9700 1 -- 172.21.15.7:0/4132858016 >> v1:172.21.15.188:6789/0 conn(0x7f47e81581a0 legacy=0x7f47e80cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.548+0000 7f47ed3c9700 1 -- 172.21.15.7:0/4132858016 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e80c4b50 con 0x7f47e8157dd0 2024-01-27T02:44:16.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.548+0000 7f47ed3c9700 1 --2- 172.21.15.7:0/4132858016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47e8157dd0 0x7f47e80adf20 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f47e806e9d0 tx=0x7f47e4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a42bbd19d84cd server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:16.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.548+0000 7f47dffff700 1 -- 172.21.15.7:0/4132858016 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e4005c10 con 0x7f47e8157dd0 2024-01-27T02:44:16.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.549+0000 7f47dffff700 1 -- 172.21.15.7:0/4132858016 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e4005d70 con 0x7f47e8157dd0 2024-01-27T02:44:16.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.549+0000 7f47dffff700 1 -- 172.21.15.7:0/4132858016 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e4006260 con 0x7f47e8157dd0 2024-01-27T02:44:16.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.549+0000 7f47ef62d700 1 -- 172.21.15.7:0/4132858016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47e8157dd0 msgr2=0x7f47e80adf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.549+0000 7f47ef62d700 1 --2- 172.21.15.7:0/4132858016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47e8157dd0 0x7f47e80adf20 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f47e806e9d0 tx=0x7f47e4009220 comp rx=0 tx=0).stop 2024-01-27T02:44:16.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.549+0000 7f47ef62d700 1 -- 172.21.15.7:0/4132858016 shutdown_connections 2024-01-27T02:44:16.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.549+0000 7f47ef62d700 1 --2- 172.21.15.7:0/4132858016 >> v2:172.21.15.188:3300/0 conn(0x7f47e80cd9c0 0x7f47e80ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.549+0000 7f47ef62d700 1 --2- 172.21.15.7:0/4132858016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47e8157dd0 0x7f47e80adf20 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.549+0000 7f47ef62d700 1 -- 172.21.15.7:0/4132858016 >> 172.21.15.7:0/4132858016 conn(0x7f47e8098020 msgr2=0x7f47e80711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:16.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.549+0000 7f47ef62d700 1 -- 172.21.15.7:0/4132858016 shutdown_connections 2024-01-27T02:44:16.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.549+0000 7f47ef62d700 1 -- 172.21.15.7:0/4132858016 wait complete. 2024-01-27T02:44:16.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.550+0000 7f47ef62d700 1 Processor -- start 2024-01-27T02:44:16.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.550+0000 7f47ef62d700 1 -- start start 2024-01-27T02:44:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.551+0000 7f47ef62d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47e80cd9c0 0x7f47e8147870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.551+0000 7f47ef62d700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47e8157dd0 0x7f47e80679b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.551+0000 7f47ef62d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47e81581a0 0x7f47e80ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.551+0000 7f47ef62d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e8147db0 con 0x7f47e81581a0 2024-01-27T02:44:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.551+0000 7f47ef62d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e8068da0 con 0x7f47e80cd9c0 2024-01-27T02:44:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.551+0000 7f47ef62d700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e8067ef0 con 0x7f47e8157dd0 2024-01-27T02:44:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.552+0000 7f47edbca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47e81581a0 0x7f47e80ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.552+0000 7f47edbca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47e81581a0 0x7f47e80ae480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52336/0 (socket says 172.21.15.7:52336) 2024-01-27T02:44:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.552+0000 7f47edbca700 1 -- 172.21.15.7:0/1157323524 learned_addr learned my addr 172.21.15.7:0/1157323524 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.552+0000 7f47ed3c9700 1 --2- 172.21.15.7:0/1157323524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47e80cd9c0 0x7f47e8147870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.552+0000 7f47ecbc8700 1 --2- 172.21.15.7:0/1157323524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47e8157dd0 0x7f47e80679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.552+0000 7f47edbca700 1 -- 172.21.15.7:0/1157323524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47e8157dd0 msgr2=0x7f47e80679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.552+0000 7f47edbca700 1 --2- 172.21.15.7:0/1157323524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47e8157dd0 0x7f47e80679b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.552+0000 7f47edbca700 1 -- 172.21.15.7:0/1157323524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47e80cd9c0 msgr2=0x7f47e8147870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.552+0000 7f47edbca700 1 --2- 172.21.15.7:0/1157323524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47e80cd9c0 0x7f47e8147870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.552+0000 7f47edbca700 1 -- 172.21.15.7:0/1157323524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e4009480 con 0x7f47e81581a0 2024-01-27T02:44:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.552+0000 7f47edbca700 1 --2- 172.21.15.7:0/1157323524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47e81581a0 0x7f47e80ae480 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009e90 tx=0x7f47e000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.552+0000 7f47dd7fa700 1 -- 172.21.15.7:0/1157323524 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e000cc20 con 0x7f47e81581a0 2024-01-27T02:44:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.552+0000 7f47dd7fa700 1 -- 172.21.15.7:0/1157323524 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e000f040 con 0x7f47e81581a0 2024-01-27T02:44:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.552+0000 7f47dd7fa700 1 -- 172.21.15.7:0/1157323524 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0014480 con 0x7f47e81581a0 2024-01-27T02:44:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.552+0000 7f47ef62d700 1 -- 172.21.15.7:0/1157323524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e8068190 con 0x7f47e81581a0 2024-01-27T02:44:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.552+0000 7f47ef62d700 1 -- 172.21.15.7:0/1157323524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e8068750 con 0x7f47e81581a0 2024-01-27T02:44:16.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.553+0000 7f47ef62d700 1 -- 172.21.15.7:0/1157323524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f47e804c710 con 0x7f47e81581a0 2024-01-27T02:44:16.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.554+0000 7f47dd7fa700 1 -- 172.21.15.7:0/1157323524 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f47e00147f0 con 0x7f47e81581a0 2024-01-27T02:44:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.554+0000 7f47dd7fa700 1 --2- 172.21.15.7:0/1157323524 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f47cc066f30 0x7f47cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.554+0000 7f47dd7fa700 1 -- 172.21.15.7:0/1157323524 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f47e0085030 con 0x7f47e81581a0 2024-01-27T02:44:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.554+0000 7f47ed3c9700 1 --2- 172.21.15.7:0/1157323524 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f47cc066f30 0x7f47cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.554+0000 7f47dd7fa700 1 --2- 172.21.15.7:0/1157323524 >> [v2:172.21.15.188:6816/39375,v1:172.21.15.188:6817/39375] conn(0x7f47cc06cd80 0x7f47cc06f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.554+0000 7f47dd7fa700 1 -- 172.21.15.7:0/1157323524 --> [v2:172.21.15.188:6816/39375,v1:172.21.15.188:6817/39375] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47cc06f860 con 0x7f47cc06cd80 2024-01-27T02:44:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.554+0000 7f47dd7fa700 1 -- 172.21.15.7:0/1157323524 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=187) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47e0087590 con 0x7f47e81581a0 2024-01-27T02:44:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.555+0000 7f47ecbc8700 1 --2- 172.21.15.7:0/1157323524 >> [v2:172.21.15.188:6816/39375,v1:172.21.15.188:6817/39375] conn(0x7f47cc06cd80 0x7f47cc06f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.555+0000 7f47ed3c9700 1 --2- 172.21.15.7:0/1157323524 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f47cc066f30 0x7f47cc0693e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009260 tx=0x7f47e4016000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.555+0000 7f47ecbc8700 1 --2- 172.21.15.7:0/1157323524 >> [v2:172.21.15.188:6816/39375,v1:172.21.15.188:6817/39375] conn(0x7f47cc06cd80 0x7f47cc06f190 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:16.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.556+0000 7f47dd7fa700 1 -- 172.21.15.7:0/1157323524 <== osd.7 v2:172.21.15.188:6816/39375 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f47cc06f860 con 0x7f47cc06cd80 2024-01-27T02:44:16.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.569+0000 7f47ef62d700 1 -- 172.21.15.7:0/1157323524 --> [v2:172.21.15.188:6816/39375,v1:172.21.15.188:6817/39375] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f47e80c4e30 con 0x7f47cc06cd80 2024-01-27T02:44:16.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.570+0000 7f47dd7fa700 1 -- 172.21.15.7:0/1157323524 <== osd.7 v2:172.21.15.188:6816/39375 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7f47e80c4e30 con 0x7f47cc06cd80 2024-01-27T02:44:16.572 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-27T02:44:16.572 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-01-27T02:44:16.572 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-27T02:44:16.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.570+0000 7f47ef62d700 1 -- 172.21.15.7:0/1157323524 >> [v2:172.21.15.188:6816/39375,v1:172.21.15.188:6817/39375] conn(0x7f47cc06cd80 msgr2=0x7f47cc06f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.570+0000 7f47ef62d700 1 --2- 172.21.15.7:0/1157323524 >> [v2:172.21.15.188:6816/39375,v1:172.21.15.188:6817/39375] conn(0x7f47cc06cd80 0x7f47cc06f190 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.571+0000 7f47ef62d700 1 -- 172.21.15.7:0/1157323524 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f47cc066f30 msgr2=0x7f47cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.571+0000 7f47ef62d700 1 --2- 172.21.15.7:0/1157323524 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f47cc066f30 0x7f47cc0693e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009260 tx=0x7f47e4016000 comp rx=0 tx=0).stop 2024-01-27T02:44:16.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.571+0000 7f47ef62d700 1 -- 172.21.15.7:0/1157323524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47e81581a0 msgr2=0x7f47e80ae480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.571+0000 7f47ef62d700 1 --2- 172.21.15.7:0/1157323524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47e81581a0 0x7f47e80ae480 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009e90 tx=0x7f47e000b040 comp rx=0 tx=0).stop 2024-01-27T02:44:16.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.571+0000 7f47edbca700 1 -- 172.21.15.7:0/1157323524 reap_dead start 2024-01-27T02:44:16.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.571+0000 7f47ef62d700 1 -- 172.21.15.7:0/1157323524 shutdown_connections 2024-01-27T02:44:16.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.571+0000 7f47ef62d700 1 -- 172.21.15.7:0/1157323524 >> 172.21.15.7:0/1157323524 conn(0x7f47e8098020 msgr2=0x7f47e8156e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:16.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.571+0000 7f47ef62d700 1 -- 172.21.15.7:0/1157323524 shutdown_connections 2024-01-27T02:44:16.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.571+0000 7f47ef62d700 1 -- 172.21.15.7:0/1157323524 wait complete. 2024-01-27T02:44:16.585 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-01-27T02:44:16.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.717+0000 7fc697d80700 1 Processor -- start 2024-01-27T02:44:16.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.717+0000 7fc697d80700 1 -- start start 2024-01-27T02:44:16.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.718+0000 7fc697d80700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc69015a100 0x7fc6900cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.718+0000 7fc697d80700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6900cb5b0 0x7fc6900c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.718+0000 7fc697d80700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc6900c4af0 0x7fc6900c9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.718+0000 7fc697d80700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6900c4f70 con 0x7fc69015a100 2024-01-27T02:44:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.718+0000 7fc697d80700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6900ce5e0 con 0x7fc6900c4af0 2024-01-27T02:44:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.718+0000 7fc697d80700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6900cd9d0 con 0x7fc6900cb5b0 2024-01-27T02:44:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.718+0000 7fc695b1c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc69015a100 0x7fc6900cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.718+0000 7fc695b1c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc69015a100 0x7fc6900cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52348/0 (socket says 172.21.15.7:52348) 2024-01-27T02:44:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.718+0000 7fc695b1c700 1 -- 172.21.15.7:0/2182289688 learned_addr learned my addr 172.21.15.7:0/2182289688 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.718+0000 7fc69631d700 1 --2- 172.21.15.7:0/2182289688 >> v2:172.21.15.188:3300/0 conn(0x7fc6900c4af0 0x7fc6900c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.718+0000 7fc69531b700 1 --2- 172.21.15.7:0/2182289688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6900cb5b0 0x7fc6900c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.719+0000 7fc695b1c700 1 -- 172.21.15.7:0/2182289688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6900cb5b0 msgr2=0x7fc6900c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.719+0000 7fc695b1c700 1 --2- 172.21.15.7:0/2182289688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6900cb5b0 0x7fc6900c45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.719+0000 7fc695b1c700 1 -- 172.21.15.7:0/2182289688 >> v2:172.21.15.188:3300/0 conn(0x7fc6900c4af0 msgr2=0x7fc6900c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.719+0000 7fc695b1c700 1 --2- 172.21.15.7:0/2182289688 >> v2:172.21.15.188:3300/0 conn(0x7fc6900c4af0 0x7fc6900c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.719+0000 7fc695b1c700 1 -- 172.21.15.7:0/2182289688 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6900cdcb0 con 0x7fc69015a100 2024-01-27T02:44:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.719+0000 7fc695b1c700 1 --2- 172.21.15.7:0/2182289688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc69015a100 0x7fc6900cd2f0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fc69006e7a0 tx=0x7fc68c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5997ea55a2d5215c server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.719+0000 7fc694b1a700 1 -- 172.21.15.7:0/2182289688 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c00b640 con 0x7fc69015a100 2024-01-27T02:44:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.719+0000 7fc694b1a700 1 -- 172.21.15.7:0/2182289688 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c00b7a0 con 0x7fc69015a100 2024-01-27T02:44:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.719+0000 7fc694b1a700 1 -- 172.21.15.7:0/2182289688 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c00bb60 con 0x7fc69015a100 2024-01-27T02:44:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.720+0000 7fc697d80700 1 -- 172.21.15.7:0/2182289688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc69015a100 msgr2=0x7fc6900cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.720+0000 7fc697d80700 1 --2- 172.21.15.7:0/2182289688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc69015a100 0x7fc6900cd2f0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fc69006e7a0 tx=0x7fc68c009250 comp rx=0 tx=0).stop 2024-01-27T02:44:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.720+0000 7fc697d80700 1 -- 172.21.15.7:0/2182289688 shutdown_connections 2024-01-27T02:44:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.720+0000 7fc697d80700 1 --2- 172.21.15.7:0/2182289688 >> v2:172.21.15.188:3300/0 conn(0x7fc6900c4af0 0x7fc6900c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.720+0000 7fc697d80700 1 --2- 172.21.15.7:0/2182289688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc69015a100 0x7fc6900cd2f0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.720+0000 7fc697d80700 1 --2- 172.21.15.7:0/2182289688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6900cb5b0 0x7fc6900c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.720+0000 7fc697d80700 1 -- 172.21.15.7:0/2182289688 >> 172.21.15.7:0/2182289688 conn(0x7fc690098020 msgr2=0x7fc6900711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.720+0000 7fc697d80700 1 -- 172.21.15.7:0/2182289688 shutdown_connections 2024-01-27T02:44:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.720+0000 7fc697d80700 1 -- 172.21.15.7:0/2182289688 wait complete. 2024-01-27T02:44:16.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.721+0000 7fc697d80700 1 Processor -- start 2024-01-27T02:44:16.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.721+0000 7fc697d80700 1 -- start start 2024-01-27T02:44:16.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.722+0000 7fc697d80700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6900c4af0 0x7fc6901523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.722+0000 7fc697d80700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6900cb5b0 0x7fc6900c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.722+0000 7fc697d80700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6900c9a70 0x7fc6900b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.722+0000 7fc697d80700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6900c9f10 con 0x7fc6900cb5b0 2024-01-27T02:44:16.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.722+0000 7fc697d80700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6900b29d0 con 0x7fc6900c4af0 2024-01-27T02:44:16.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.722+0000 7fc697d80700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6900b2cb0 con 0x7fc6900c9a70 2024-01-27T02:44:16.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.722+0000 7fc69531b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6900cb5b0 0x7fc6900c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.722+0000 7fc69531b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6900cb5b0 0x7fc6900c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52356/0 (socket says 172.21.15.7:52356) 2024-01-27T02:44:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.722+0000 7fc69531b700 1 -- 172.21.15.7:0/527559492 learned_addr learned my addr 172.21.15.7:0/527559492 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.722+0000 7fc695b1c700 1 --2- 172.21.15.7:0/527559492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6900c4af0 0x7fc6901523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.722+0000 7fc69631d700 1 --2- 172.21.15.7:0/527559492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6900c9a70 0x7fc6900b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.722+0000 7fc69531b700 1 -- 172.21.15.7:0/527559492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6900c9a70 msgr2=0x7fc6900b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.722+0000 7fc69531b700 1 --2- 172.21.15.7:0/527559492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6900c9a70 0x7fc6900b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.722+0000 7fc69531b700 1 -- 172.21.15.7:0/527559492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6900c4af0 msgr2=0x7fc6901523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.722+0000 7fc69531b700 1 --2- 172.21.15.7:0/527559492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6900c4af0 0x7fc6901523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.722+0000 7fc69531b700 1 -- 172.21.15.7:0/527559492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68c0094f0 con 0x7fc6900cb5b0 2024-01-27T02:44:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.722+0000 7fc69631d700 1 --2- 172.21.15.7:0/527559492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6900c9a70 0x7fc6900b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:44:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.722+0000 7fc695b1c700 1 --2- 172.21.15.7:0/527559492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6900c4af0 0x7fc6901523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:44:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.722+0000 7fc69531b700 1 --2- 172.21.15.7:0/527559492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6900cb5b0 0x7fc6900c9530 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fc68000ec80 tx=0x7fc6800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.723+0000 7fc685ffb700 1 -- 172.21.15.7:0/527559492 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68000c9d0 con 0x7fc6900cb5b0 2024-01-27T02:44:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.723+0000 7fc685ffb700 1 -- 172.21.15.7:0/527559492 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68000cb30 con 0x7fc6900cb5b0 2024-01-27T02:44:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.723+0000 7fc697d80700 1 -- 172.21.15.7:0/527559492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6900b2f90 con 0x7fc6900cb5b0 2024-01-27T02:44:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.723+0000 7fc685ffb700 1 -- 172.21.15.7:0/527559492 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc680013470 con 0x7fc6900cb5b0 2024-01-27T02:44:16.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.723+0000 7fc697d80700 1 -- 172.21.15.7:0/527559492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6900b3550 con 0x7fc6900cb5b0 2024-01-27T02:44:16.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.724+0000 7fc697d80700 1 -- 172.21.15.7:0/527559492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc69004c710 con 0x7fc6900cb5b0 2024-01-27T02:44:16.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.724+0000 7fc685ffb700 1 -- 172.21.15.7:0/527559492 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc680013820 con 0x7fc6900cb5b0 2024-01-27T02:44:16.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.725+0000 7fc685ffb700 1 --2- 172.21.15.7:0/527559492 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc674066f30 0x7fc6740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:16.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.725+0000 7fc685ffb700 1 -- 172.21.15.7:0/527559492 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fc680084620 con 0x7fc6900cb5b0 2024-01-27T02:44:16.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.725+0000 7fc695b1c700 1 --2- 172.21.15.7:0/527559492 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc674066f30 0x7fc6740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:16.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.726+0000 7fc695b1c700 1 --2- 172.21.15.7:0/527559492 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc674066f30 0x7fc6740693e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc69006e7a0 tx=0x7fc68c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:16.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.729+0000 7fc685ffb700 1 -- 172.21.15.7:0/527559492 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc680051b90 con 0x7fc6900cb5b0 2024-01-27T02:44:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.935+0000 7fc697d80700 1 -- 172.21.15.7:0/527559492 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6900cdf90 con 0x7fc674066f30 2024-01-27T02:44:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.936+0000 7fc685ffb700 1 -- 172.21.15.7:0/527559492 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc6900cdf90 con 0x7fc674066f30 2024-01-27T02:44:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.938+0000 7fc697d80700 1 -- 172.21.15.7:0/527559492 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc674066f30 msgr2=0x7fc6740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.938+0000 7fc697d80700 1 --2- 172.21.15.7:0/527559492 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc674066f30 0x7fc6740693e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc69006e7a0 tx=0x7fc68c017000 comp rx=0 tx=0).stop 2024-01-27T02:44:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.938+0000 7fc697d80700 1 -- 172.21.15.7:0/527559492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6900cb5b0 msgr2=0x7fc6900c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.938+0000 7fc697d80700 1 --2- 172.21.15.7:0/527559492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6900cb5b0 0x7fc6900c9530 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fc68000ec80 tx=0x7fc6800091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.938+0000 7fc697d80700 1 -- 172.21.15.7:0/527559492 shutdown_connections 2024-01-27T02:44:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.938+0000 7fc697d80700 1 --2- 172.21.15.7:0/527559492 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7fc674066f30 0x7fc6740693e0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.938+0000 7fc697d80700 1 --2- 172.21.15.7:0/527559492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6900c9a70 0x7fc6900b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.938+0000 7fc697d80700 1 --2- 172.21.15.7:0/527559492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6900c4af0 0x7fc6901523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.938+0000 7fc697d80700 1 --2- 172.21.15.7:0/527559492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6900cb5b0 0x7fc6900c9530 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.938+0000 7fc697d80700 1 -- 172.21.15.7:0/527559492 >> 172.21.15.7:0/527559492 conn(0x7fc690098020 msgr2=0x7fc690154320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.938+0000 7fc697d80700 1 -- 172.21.15.7:0/527559492 shutdown_connections 2024-01-27T02:44:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:16.938+0000 7fc697d80700 1 -- 172.21.15.7:0/527559492 wait complete. 2024-01-27T02:44:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 7 to deep-scrub 2024-01-27T02:44:16.953 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:44:17.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.083+0000 7f69f5706700 1 Processor -- start 2024-01-27T02:44:17.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.083+0000 7f69f5706700 1 -- start start 2024-01-27T02:44:17.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.084+0000 7f69f5706700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f69f00c83c0 0x7f69f00b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:17.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.084+0000 7f69f5706700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69f0063c90 0x7f69f006aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.084+0000 7f69f5706700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f006e1d0 con 0x7f69f00b00c0 2024-01-27T02:44:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.084+0000 7f69f5706700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f00ca620 con 0x7f69f00c83c0 2024-01-27T02:44:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.084+0000 7f69f5706700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f015da70 con 0x7f69f0063c90 2024-01-27T02:44:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.084+0000 7f69eeffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f69f00b00c0 0x7f69f0063580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46970/0 (socket says 172.21.15.7:46970) 2024-01-27T02:44:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.084+0000 7f69eeffd700 1 -- 172.21.15.7:0/583360074 learned_addr learned my addr 172.21.15.7:0/583360074 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.084+0000 7f69ee7fc700 1 --2- 172.21.15.7:0/583360074 >> v2:172.21.15.188:3300/0 conn(0x7f69f00c83c0 0x7f69f00b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.084+0000 7f69ef7fe700 1 --2- 172.21.15.7:0/583360074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69f0063c90 0x7f69f006aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.084+0000 7f69edffb700 1 -- 172.21.15.7:0/583360074 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 400643617 0 0) 0x7f69f006e1d0 con 0x7f69f00b00c0 2024-01-27T02:44:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.084+0000 7f69edffb700 1 -- 172.21.15.7:0/583360074 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69d8003570 con 0x7f69f00b00c0 2024-01-27T02:44:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69edffb700 1 -- 172.21.15.7:0/583360074 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2424898184 0 0) 0x7f69d8003570 con 0x7f69f00b00c0 2024-01-27T02:44:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69edffb700 1 -- 172.21.15.7:0/583360074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69f0063c90 msgr2=0x7f69f006aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69edffb700 1 --2- 172.21.15.7:0/583360074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69f0063c90 0x7f69f006aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69edffb700 1 -- 172.21.15.7:0/583360074 >> v2:172.21.15.188:3300/0 conn(0x7f69f00c83c0 msgr2=0x7f69f00b09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69edffb700 1 --2- 172.21.15.7:0/583360074 >> v2:172.21.15.188:3300/0 conn(0x7f69f00c83c0 0x7f69f00b09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69edffb700 1 -- 172.21.15.7:0/583360074 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f006e4b0 con 0x7f69f00b00c0 2024-01-27T02:44:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69edffb700 1 -- 172.21.15.7:0/583360074 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3508847201 0 0) 0x7f69e0001470 con 0x7f69f00b00c0 2024-01-27T02:44:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69edffb700 1 -- 172.21.15.7:0/583360074 >> v1:172.21.15.7:6789/0 conn(0x7f69f00b00c0 legacy=0x7f69f0063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69edffb700 1 --2- 172.21.15.7:0/583360074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69d8003a90 0x7f69d8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69edffb700 1 --2- 172.21.15.7:0/583360074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69d8004810 0x7f69d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69edffb700 1 --2- 172.21.15.7:0/583360074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69f0063c90 0x7f69d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69edffb700 1 -- 172.21.15.7:0/583360074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f015da70 con 0x7f69d8004810 2024-01-27T02:44:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69edffb700 1 -- 172.21.15.7:0/583360074 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d800ba60 con 0x7f69d8003a90 2024-01-27T02:44:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69edffb700 1 -- 172.21.15.7:0/583360074 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d800bba0 con 0x7f69f0063c90 2024-01-27T02:44:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69ee7fc700 1 --2- 172.21.15.7:0/583360074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69d8004810 0x7f69d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69eeffd700 1 --2- 172.21.15.7:0/583360074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69d8003a90 0x7f69d8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69ef7fe700 1 --2- 172.21.15.7:0/583360074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69f0063c90 0x7f69d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69ee7fc700 1 -- 172.21.15.7:0/583360074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69f0063c90 msgr2=0x7f69d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69ee7fc700 1 --2- 172.21.15.7:0/583360074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69f0063c90 0x7f69d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69ee7fc700 1 -- 172.21.15.7:0/583360074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69d8003a90 msgr2=0x7f69d8003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69ee7fc700 1 --2- 172.21.15.7:0/583360074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69d8003a90 0x7f69d8003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69ee7fc700 1 -- 172.21.15.7:0/583360074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d8003570 con 0x7f69d8004810 2024-01-27T02:44:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.085+0000 7f69ee7fc700 1 --2- 172.21.15.7:0/583360074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69d8004810 0x7f69d8004c80 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f69f006c5a0 tx=0x7f69e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45feb823941a807 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.086+0000 7f69edffb700 1 -- 172.21.15.7:0/583360074 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e400e710 con 0x7f69d8004810 2024-01-27T02:44:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.086+0000 7f69edffb700 1 -- 172.21.15.7:0/583360074 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e400e870 con 0x7f69d8004810 2024-01-27T02:44:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.086+0000 7f69edffb700 1 -- 172.21.15.7:0/583360074 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e400ec30 con 0x7f69d8004810 2024-01-27T02:44:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.086+0000 7f69f5706700 1 -- 172.21.15.7:0/583360074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69d8004810 msgr2=0x7f69d8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.086+0000 7f69f5706700 1 --2- 172.21.15.7:0/583360074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69d8004810 0x7f69d8004c80 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f69f006c5a0 tx=0x7f69e40091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.086+0000 7f69ef7fe700 1 -- 172.21.15.7:0/583360074 reap_dead start 2024-01-27T02:44:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.086+0000 7f69f5706700 1 -- 172.21.15.7:0/583360074 shutdown_connections 2024-01-27T02:44:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.086+0000 7f69f5706700 1 -- 172.21.15.7:0/583360074 >> 172.21.15.7:0/583360074 conn(0x7f69f00679b0 msgr2=0x7f69f0067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.086+0000 7f69f5706700 1 -- 172.21.15.7:0/583360074 shutdown_connections 2024-01-27T02:44:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.086+0000 7f69f5706700 1 -- 172.21.15.7:0/583360074 wait complete. 2024-01-27T02:44:17.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.088+0000 7f69f5706700 1 Processor -- start 2024-01-27T02:44:17.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.088+0000 7f69f5706700 1 -- start start 2024-01-27T02:44:17.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.089+0000 7f69f5706700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69d8004810 0x7f69f00c17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:17.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.089+0000 7f69f5706700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69f00c79f0 0x7f69f015c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:17.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.089+0000 7f69f5706700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69f015cd70 0x7f69f015d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:17.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.089+0000 7f69f5706700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f00c2be0 con 0x7f69f015cd70 2024-01-27T02:44:17.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.089+0000 7f69f5706700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f00c2900 con 0x7f69d8004810 2024-01-27T02:44:17.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.089+0000 7f69f5706700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f015d6e0 con 0x7f69f00c79f0 2024-01-27T02:44:17.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.089+0000 7f69ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69f015cd70 0x7f69f015d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:17.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.089+0000 7f69ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69f015cd70 0x7f69f015d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52368/0 (socket says 172.21.15.7:52368) 2024-01-27T02:44:17.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.089+0000 7f69ef7fe700 1 -- 172.21.15.7:0/4064030808 learned_addr learned my addr 172.21.15.7:0/4064030808 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:17.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.090+0000 7f69ee7fc700 1 --2- 172.21.15.7:0/4064030808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69f00c79f0 0x7f69f015c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:17.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.090+0000 7f69eeffd700 1 --2- 172.21.15.7:0/4064030808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69d8004810 0x7f69f00c17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:17.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.090+0000 7f69ef7fe700 1 -- 172.21.15.7:0/4064030808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69f00c79f0 msgr2=0x7f69f015c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:17.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.090+0000 7f69ef7fe700 1 --2- 172.21.15.7:0/4064030808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69f00c79f0 0x7f69f015c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.090+0000 7f69ef7fe700 1 -- 172.21.15.7:0/4064030808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69d8004810 msgr2=0x7f69f00c17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.090+0000 7f69ef7fe700 1 --2- 172.21.15.7:0/4064030808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69d8004810 0x7f69f00c17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.090+0000 7f69ef7fe700 1 -- 172.21.15.7:0/4064030808 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f00c1fd0 con 0x7f69f015cd70 2024-01-27T02:44:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.090+0000 7f69eeffd700 1 --2- 172.21.15.7:0/4064030808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69d8004810 0x7f69f00c17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:44:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.090+0000 7f69ef7fe700 1 --2- 172.21.15.7:0/4064030808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69f015cd70 0x7f69f015d1a0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f69f0068990 tx=0x7f69e800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.090+0000 7f69d6ffd700 1 -- 172.21.15.7:0/4064030808 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e800e040 con 0x7f69f015cd70 2024-01-27T02:44:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.090+0000 7f69d6ffd700 1 -- 172.21.15.7:0/4064030808 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e800f040 con 0x7f69f015cd70 2024-01-27T02:44:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.090+0000 7f69d6ffd700 1 -- 172.21.15.7:0/4064030808 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e800fe20 con 0x7f69f015cd70 2024-01-27T02:44:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.090+0000 7f69f5706700 1 -- 172.21.15.7:0/4064030808 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f00c22b0 con 0x7f69f015cd70 2024-01-27T02:44:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.090+0000 7f69f5706700 1 -- 172.21.15.7:0/4064030808 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f00c1ce0 con 0x7f69f015cd70 2024-01-27T02:44:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.091+0000 7f69f5706700 1 -- 172.21.15.7:0/4064030808 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f004c710 con 0x7f69f015cd70 2024-01-27T02:44:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.091+0000 7f69d6ffd700 1 -- 172.21.15.7:0/4064030808 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f69e8015640 con 0x7f69f015cd70 2024-01-27T02:44:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.092+0000 7f69d6ffd700 1 --2- 172.21.15.7:0/4064030808 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f69dc066f30 0x7f69dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.092+0000 7f69d6ffd700 1 -- 172.21.15.7:0/4064030808 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f69e8084b20 con 0x7f69f015cd70 2024-01-27T02:44:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.092+0000 7f69eeffd700 1 --2- 172.21.15.7:0/4064030808 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f69dc066f30 0x7f69dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.093+0000 7f69eeffd700 1 --2- 172.21.15.7:0/4064030808 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f69dc066f30 0x7f69dc0693e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f69e00016e0 tx=0x7f69e0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:17.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.096+0000 7f69d6ffd700 1 -- 172.21.15.7:0/4064030808 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69e8052090 con 0x7f69f015cd70 2024-01-27T02:44:17.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.292+0000 7f69f5706700 1 -- 172.21.15.7:0/4064030808 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69f00ca810 con 0x7f69dc066f30 2024-01-27T02:44:17.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.294+0000 7f69d6ffd700 1 -- 172.21.15.7:0/4064030808 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50240 (secure 0 0 0) 0x7f69f00ca810 con 0x7f69dc066f30 2024-01-27T02:44:17.296 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:44:17.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.296+0000 7f69f5706700 1 -- 172.21.15.7:0/4064030808 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f69dc066f30 msgr2=0x7f69dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:17.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.296+0000 7f69f5706700 1 --2- 172.21.15.7:0/4064030808 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f69dc066f30 0x7f69dc0693e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f69e00016e0 tx=0x7f69e0009040 comp rx=0 tx=0).stop 2024-01-27T02:44:17.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.296+0000 7f69f5706700 1 -- 172.21.15.7:0/4064030808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69f015cd70 msgr2=0x7f69f015d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:17.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.296+0000 7f69f5706700 1 --2- 172.21.15.7:0/4064030808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69f015cd70 0x7f69f015d1a0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f69f0068990 tx=0x7f69e800b040 comp rx=0 tx=0).stop 2024-01-27T02:44:17.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.296+0000 7f69f5706700 1 -- 172.21.15.7:0/4064030808 shutdown_connections 2024-01-27T02:44:17.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.296+0000 7f69f5706700 1 --2- 172.21.15.7:0/4064030808 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f69dc066f30 0x7f69dc0693e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:17.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.296+0000 7f69f5706700 1 --2- 172.21.15.7:0/4064030808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69f015cd70 0x7f69f015d1a0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:17.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.296+0000 7f69f5706700 1 --2- 172.21.15.7:0/4064030808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69d8004810 0x7f69f00c17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:17.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.296+0000 7f69f5706700 1 --2- 172.21.15.7:0/4064030808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69f00c79f0 0x7f69f015c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:17.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.296+0000 7f69f5706700 1 -- 172.21.15.7:0/4064030808 >> 172.21.15.7:0/4064030808 conn(0x7f69f00679b0 msgr2=0x7f69f00ccdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:17.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.296+0000 7f69f5706700 1 -- 172.21.15.7:0/4064030808 shutdown_connections 2024-01-27T02:44:17.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:17.296+0000 7f69f5706700 1 -- 172.21.15.7:0/4064030808 wait complete. 2024-01-27T02:44:17.299 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:44:17.311 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":547,"stamp":"2024-01-27T02:44:15.753032+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9346,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553153,"statfs":{"total":1600000000,"available":1590428936,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52428800,"num_objects":-13,"num_object_clones":0,"num_object_copies":-26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-32,"num_write_kb":-84992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.744805"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":309,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-01-27T02:44:07.014472+0000","last_change":"2024-01-27T02:31:59.595091+0000","last_active":"2024-01-27T02:44:07.014472+0000","last_peered":"2024-01-27T02:44:07.014472+0000","last_clean":"2024-01-27T02:44:07.014472+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:44:07.014472+0000","last_undegraded":"2024-01-27T02:44:07.014472+0000","last_fullsized":"2024-01-27T02:44:07.014472+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-27T02:31:58.568854+0000","last_clean_scrub_stamp":"2024-01-27T02:31:58.568854+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T02:38:52.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574997,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1600,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193711,"statfs":{"total":200000000,"available":198360841,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.32800000000000001},"max":{"1min":0.80600000000000005,"5min":0.83499999999999996,"15min":0.85499999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59899999999999998,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.30099999999999999},"max":{"1min":0.84199999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.436}]},{"osd":1,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.47699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74399999999999999,"5min":0.78800000000000003,"15min":0.82399999999999995},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59599999999999997,"15min":0.57399999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.34999999999999998},"max":{"1min":0.81599999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53300000000000003}]},{"osd":2,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.38400000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.86399999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.75,"5min":0.80300000000000005,"15min":0.83899999999999997},"last":0.49099999999999999}]},{"osd":3,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76500000000000001,"5min":0.78900000000000003,"15min":0.90300000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.63800000000000001,"15min":0.61199999999999999},"min":{"1min":0.47599999999999998,"5min":0.41799999999999998,"15min":0.40999999999999998},"max":{"1min":0.77400000000000002,"5min":0.81599999999999995,"15min":0.88600000000000001},"last":0.52100000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48199999999999998,"15min":0.48399999999999999},"min":{"1min":0.316,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67300000000000004,"5min":0.73899999999999999,"15min":0.78600000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52000000000000002,"15min":0.51000000000000001},"min":{"1min":0.36699999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.80000000000000004},"last":0.33800000000000002}]},{"osd":5,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.39600000000000002,"5min":0.32400000000000001,"15min":0.307},"max":{"1min":0.746,"5min":0.79500000000000004,"15min":0.85399999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.42499999999999999,"5min":0.33900000000000002,"15min":0.33500000000000002},"max":{"1min":0.76600000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.46899999999999997}]},{"osd":6,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60199999999999998,"15min":0.59299999999999997},"min":{"1min":0.38900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.78900000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.499},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71799999999999997,"5min":0.83499999999999996,"15min":0.90800000000000003},"last":0.44600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1024,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194288,"statfs":{"total":200000000,"available":198951209,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60099999999999998,"15min":0.57899999999999996},"min":{"1min":0.38800000000000001,"5min":0.375,"15min":0.29999999999999999},"max":{"1min":0.78700000000000003,"5min":1.0409999999999999,"15min":4.125},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58799999999999997,"15min":0.56499999999999995},"min":{"1min":0.435,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.753,"5min":0.96599999999999997,"15min":1.216},"last":0.58599999999999997}]},{"osd":1,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42599999999999999,"5min":0.376,"15min":0.34100000000000003},"max":{"1min":0.67800000000000005,"5min":0.95499999999999996,"15min":8.2699999999999996},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59699999999999998,"15min":0.67800000000000005},"min":{"1min":0.52200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78000000000000003,"5min":1.0129999999999999,"15min":26.411999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.627},"min":{"1min":0.51900000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73699999999999999,"5min":0.78300000000000003,"15min":17.143999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61499999999999999,"15min":0.60899999999999999},"min":{"1min":0.503,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73199999999999998,"5min":0.85799999999999998,"15min":1.76},"last":0.67700000000000005}]},{"osd":3,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.47699999999999998,"5min":0.39400000000000002,"15min":0.376},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":4.3689999999999998},"last":0.624},{"interface":"front","average":{"1min":0.62,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.41199999999999998},"max":{"1min":0.77400000000000002,"5min":0.81699999999999995,"15min":2.1970000000000001},"last":0.56100000000000005}]},{"osd":4,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47099999999999997,"15min":0.48099999999999998},"min":{"1min":0.35299999999999998,"5min":0.311,"15min":0.30199999999999999},"max":{"1min":0.61499999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78900000000000003,"5min":0.84899999999999998,"15min":0.89600000000000002},"last":0.54400000000000004}]},{"osd":5,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.39400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.65400000000000003,"5min":0.80800000000000005,"15min":0.81299999999999994},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55200000000000005,"15min":0.57699999999999996},"min":{"1min":0.38300000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76400000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.65000000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:43:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57799999999999996,"15min":0.59599999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.35999999999999999},"max":{"1min":0.73599999999999999,"5min":0.88700000000000001,"15min":1.0009999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54900000000000004,"15min":0.56799999999999995},"min":{"1min":0.40699999999999997,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.749,"5min":0.82899999999999996,"15min":1.038},"last":0.60199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1024,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194288,"statfs":{"total":200000000,"available":198951209,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40300000000000002,"15min":0.38900000000000001},"min":{"1min":0.26900000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.51800000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.42999999999999999,"15min":0.41299999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.27700000000000002},"max":{"1min":0.51400000000000001,"5min":0.66100000000000003,"15min":1.121},"last":0.28499999999999998}]},{"osd":2,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42199999999999999,"15min":0.39300000000000002},"min":{"1min":0.30599999999999999,"5min":0.249,"15min":0.23999999999999999},"max":{"1min":0.56000000000000005,"5min":1.0940000000000001,"15min":1.25},"last":0.313},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.45200000000000001,"15min":0.442},"min":{"1min":0.30099999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.65800000000000003,"5min":0.66700000000000004,"15min":0.82999999999999996},"last":0.371}]},{"osd":3,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.495,"15min":0.48199999999999998},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.32100000000000001},"max":{"1min":0.61299999999999999,"5min":0.79900000000000004,"15min":0.90900000000000003},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47399999999999998,"15min":0.46400000000000002},"min":{"1min":0.35999999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.59199999999999997,"5min":0.72599999999999998,"15min":0.98999999999999999},"last":0.35399999999999998}]},{"osd":4,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.66000000000000003,"15min":0.64800000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.373},"max":{"1min":0.83999999999999997,"5min":1.075,"15min":3.2890000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67800000000000005,"15min":0.65500000000000003},"min":{"1min":0.48999999999999999,"5min":0.45000000000000001,"15min":0.44500000000000001},"max":{"1min":0.95499999999999996,"5min":0.98099999999999998,"15min":1.026},"last":0.70099999999999996}]},{"osd":5,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68500000000000005,"15min":0.66200000000000003},"min":{"1min":0.46700000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89900000000000002,"5min":0.94599999999999995,"15min":3.4500000000000002},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68600000000000005,"15min":0.68200000000000005},"min":{"1min":0.45600000000000002,"5min":0.44,"15min":0.42399999999999999},"max":{"1min":0.93000000000000005,"5min":1.004,"15min":3.181},"last":0.59299999999999997}]},{"osd":6,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.71599999999999997,"15min":0.70599999999999996},"min":{"1min":0.52500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91400000000000003,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.74099999999999999,"15min":0.70599999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.50800000000000001},"max":{"1min":0.93899999999999995,"5min":1.143,"15min":1.869},"last":0.71499999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73799999999999999,"15min":0.71099999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85399999999999998,"5min":1.181,"15min":1.6870000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72799999999999998,"15min":0.71099999999999997},"min":{"1min":0.56299999999999994,"5min":0.501,"15min":0.45300000000000001},"max":{"1min":0.92400000000000004,"5min":1.0429999999999999,"15min":1.708},"last":0.54700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574997,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1600,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193711,"statfs":{"total":200000000,"available":198360841,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39500000000000002,"15min":0.39300000000000002},"min":{"1min":0.28499999999999998,"5min":0.24399999999999999,"15min":0.23799999999999999},"max":{"1min":0.55000000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42499999999999999,"15min":0.41999999999999998},"min":{"1min":0.32400000000000001,"5min":0.26000000000000001,"15min":0.223},"max":{"1min":0.55800000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.372}]},{"osd":2,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42499999999999999},"min":{"1min":0.34999999999999998,"5min":0.28599999999999998,"15min":0.28199999999999997},"max":{"1min":0.60799999999999998,"5min":0.625,"15min":0.64000000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45500000000000002,"15min":0.44400000000000001},"min":{"1min":0.34200000000000003,"5min":0.29499999999999998,"15min":0.28799999999999998},"max":{"1min":0.626,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.42099999999999999}]},{"osd":3,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.44600000000000001,"15min":0.436},"min":{"1min":0.29999999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58499999999999996,"5min":0.66600000000000004,"15min":0.67300000000000004},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.39000000000000001,"5min":0.34300000000000003,"15min":0.29099999999999998},"max":{"1min":0.69799999999999995,"5min":0.71399999999999997,"15min":0.71699999999999997},"last":0.436}]},{"osd":4,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.65400000000000003,"15min":0.64900000000000002},"min":{"1min":0.46600000000000003,"5min":0.432,"15min":0.42099999999999999},"max":{"1min":0.84899999999999998,"5min":0.97599999999999998,"15min":0.997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63700000000000001,"15min":0.65000000000000002},"min":{"1min":0.46400000000000002,"5min":0.42199999999999999,"15min":0.41999999999999998},"max":{"1min":0.85999999999999999,"5min":0.87,"15min":0.96699999999999997},"last":0.68799999999999994}]},{"osd":5,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.54100000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":2.4860000000000002},"last":0.746},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.44},"max":{"1min":0.90200000000000002,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.61799999999999999}]},{"osd":6,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.51300000000000001,"5min":0.45600000000000002,"15min":0.42499999999999999},"max":{"1min":0.92600000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68899999999999995,"15min":0.68600000000000005},"min":{"1min":0.53600000000000003,"5min":0.46500000000000002,"15min":0.45000000000000001},"max":{"1min":0.96699999999999997,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.65800000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:43:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.51400000000000001,"5min":0.47099999999999997,"15min":0.42599999999999999},"max":{"1min":0.94799999999999995,"5min":0.94999999999999996,"15min":1.014},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68899999999999995,"15min":0.68999999999999995},"min":{"1min":0.55800000000000005,"5min":0.499,"15min":0.498},"max":{"1min":0.90900000000000003,"5min":0.92500000000000004,"15min":0.97299999999999998},"last":0.76200000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1024,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194288,"statfs":{"total":200000000,"available":198951209,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41099999999999998},"min":{"1min":0.27000000000000002,"5min":0.23300000000000001,"15min":0.23000000000000001},"max":{"1min":0.63200000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.46000000000000002,"15min":0.53900000000000003},"min":{"1min":0.36899999999999999,"5min":0.34300000000000003,"15min":0.30499999999999999},"max":{"1min":0.57999999999999996,"5min":0.68899999999999995,"15min":6.2729999999999997},"last":0.44700000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.41199999999999998,"15min":0.42099999999999999},"min":{"1min":0.33000000000000002,"5min":0.28799999999999998,"15min":0.23799999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.81399999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.442,"5min":0.443,"15min":0.442},"min":{"1min":0.29999999999999999,"5min":0.28999999999999998,"15min":0.28899999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.70099999999999996},"last":0.374}]},{"osd":3,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.31,"5min":0.28499999999999998,"15min":0.27800000000000002},"max":{"1min":0.55800000000000005,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48399999999999999,"15min":0.48499999999999999},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.70399999999999996,"5min":0.77300000000000002,"15min":0.80800000000000005},"last":0.41299999999999998}]},{"osd":4,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68100000000000005,"15min":0.67000000000000004},"min":{"1min":0.54600000000000004,"5min":0.47599999999999998,"15min":0.39300000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.98299999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.746},"min":{"1min":0.56499999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94399999999999995,"5min":0.95799999999999996,"15min":3.581},"last":0.64700000000000002}]},{"osd":5,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65800000000000003,"15min":0.69599999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.43099999999999999},"max":{"1min":0.90400000000000003,"5min":0.92200000000000004,"15min":3.5369999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66400000000000003,"15min":0.66800000000000004},"min":{"1min":0.5,"5min":0.44800000000000001,"15min":0.41099999999999998},"max":{"1min":0.79900000000000004,"5min":0.89100000000000001,"15min":1.0529999999999999},"last":0.53100000000000003}]},{"osd":6,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68200000000000005,"15min":0.67100000000000004},"min":{"1min":0.55300000000000005,"5min":0.46999999999999997,"15min":0.42299999999999999},"max":{"1min":1.0900000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70099999999999996,"15min":0.68899999999999995},"min":{"1min":0.48299999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0469999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.627}]},{"osd":7,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69599999999999995,"15min":0.68500000000000005},"min":{"1min":0.46999999999999997,"5min":0.437,"15min":0.437},"max":{"1min":1.0600000000000001,"5min":1.55,"15min":1.55},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71999999999999997,"15min":0.70099999999999996},"min":{"1min":0.54300000000000004,"5min":0.51300000000000001,"15min":0.42699999999999999},"max":{"1min":0.96999999999999997,"5min":1.615,"15min":1.615},"last":0.65900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1024,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194288,"statfs":{"total":200000000,"available":198951209,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.253},"max":{"1min":0.56299999999999994,"5min":0.67900000000000005,"15min":4.7380000000000004},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.42099999999999999,"15min":0.40899999999999997},"min":{"1min":0.30499999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":1.0189999999999999},"last":0.33800000000000002}]},{"osd":1,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40500000000000003,"15min":0.41999999999999998},"min":{"1min":0.27600000000000002,"5min":0.24399999999999999,"15min":0.22900000000000001},"max":{"1min":0.54300000000000004,"5min":0.64400000000000002,"15min":4.5739999999999998},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.46300000000000002,"15min":0.46500000000000002},"min":{"1min":0.35499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.751,"5min":0.98199999999999998,"15min":5.0529999999999999},"last":0.35999999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.46800000000000003,"15min":0.45600000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.28100000000000003},"max":{"1min":0.624,"5min":0.76700000000000002,"15min":1.5980000000000001},"last":0.378},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.48899999999999999,"15min":0.49299999999999999},"min":{"1min":0.42599999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.65700000000000003,"5min":0.73499999999999999,"15min":5.1879999999999997},"last":0.439}]},{"osd":4,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63500000000000001,"15min":0.64500000000000002},"min":{"1min":0.44,"5min":0.42999999999999999,"15min":0.41699999999999998},"max":{"1min":0.877,"5min":0.91600000000000004,"15min":4.6660000000000004},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65400000000000003,"15min":0.63600000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84199999999999997,"5min":0.94799999999999995,"15min":1.3120000000000001},"last":0.56299999999999994}]},{"osd":5,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.49199999999999999,"5min":0.438,"15min":0.40799999999999997},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.409},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66600000000000004,"15min":0.70499999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81499999999999995,"5min":0.96599999999999997,"15min":4.7960000000000003},"last":0.50600000000000001}]},{"osd":6,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67200000000000004,"15min":0.68500000000000005},"min":{"1min":0.50900000000000001,"5min":0.42799999999999999,"15min":0.39800000000000002},"max":{"1min":0.86099999999999999,"5min":0.93500000000000005,"15min":5.0069999999999997},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69299999999999995,"15min":0.67200000000000004},"min":{"1min":0.53000000000000003,"5min":0.47499999999999998,"15min":0.435},"max":{"1min":0.83499999999999996,"5min":0.94199999999999995,"15min":1.103},"last":0.42399999999999999}]},{"osd":7,"last update":"Sat Jan 27 02:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46000000000000002},"max":{"1min":0.83399999999999996,"5min":0.92100000000000004,"15min":4.7130000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.68899999999999995},"min":{"1min":0.48599999999999999,"5min":0.47299999999999998,"15min":0.435},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.361},"last":0.45600000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574997,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1024,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194288,"statfs":{"total":200000000,"available":198951209,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.43099999999999999,"5min":0.35599999999999998,"15min":0.314},"max":{"1min":0.76800000000000002,"5min":1.137,"15min":1.3839999999999999},"last":0.752},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60899999999999999,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.372},"max":{"1min":0.84499999999999997,"5min":1.016,"15min":7.0670000000000002},"last":0.52500000000000002}]},{"osd":1,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.35399999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":5.1529999999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.48499999999999999,"5min":0.43099999999999999,"15min":0.39000000000000001},"max":{"1min":0.80100000000000005,"5min":1.054,"15min":3.0419999999999998},"last":0.57499999999999996}]},{"osd":2,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60399999999999998,"15min":0.61499999999999999},"min":{"1min":0.46400000000000002,"5min":0.41199999999999998,"15min":0.39400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":5.0019999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.49399999999999999,"5min":0.41899999999999998,"15min":0.40500000000000003},"max":{"1min":0.86099999999999999,"5min":1.2549999999999999,"15min":2.073},"last":0.63500000000000001}]},{"osd":3,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.496,"5min":0.439,"15min":0.36199999999999999},"max":{"1min":0.78800000000000003,"5min":1.1899999999999999,"15min":6.7220000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.629,"15min":0.65700000000000003},"min":{"1min":0.442,"5min":0.42499999999999999,"15min":0.41399999999999998},"max":{"1min":0.872,"5min":0.88,"15min":6.5510000000000002},"last":0.65500000000000003}]},{"osd":5,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.36199999999999999,"5min":0.314,"15min":0.312},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":1.3149999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.5,"15min":0.51800000000000002},"min":{"1min":0.41399999999999998,"5min":0.32500000000000001,"15min":0.29999999999999999},"max":{"1min":0.77500000000000002,"5min":0.79800000000000004,"15min":6.9269999999999996},"last":0.66900000000000004}]},{"osd":6,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.432,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":5.2229999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55600000000000005,"15min":0.55100000000000005},"min":{"1min":0.437,"5min":0.35799999999999998,"15min":0.33200000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":1.46},"last":0.71599999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:43:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55200000000000005,"15min":0.57099999999999995},"min":{"1min":0.41399999999999998,"5min":0.36599999999999999,"15min":0.33600000000000002},"max":{"1min":0.80400000000000005,"5min":0.83499999999999996,"15min":7.5339999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58099999999999996,"15min":0.59399999999999997},"min":{"1min":0.47099999999999997,"5min":0.34699999999999998,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.85299999999999998,"15min":4.9550000000000001},"last":0.76200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1024,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194288,"statfs":{"total":200000000,"available":198951209,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.44700000000000001,"5min":0.33900000000000002,"15min":0.28899999999999998},"max":{"1min":0.76300000000000001,"5min":1.071,"15min":1.071},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.47999999999999998,"5min":0.38500000000000001,"15min":0.27600000000000002},"max":{"1min":0.71099999999999997,"5min":0.83599999999999997,"15min":3.242},"last":0.59899999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65200000000000002,"15min":0.627},"min":{"1min":0.53500000000000003,"5min":0.38100000000000001,"15min":0.32800000000000001},"max":{"1min":0.80800000000000005,"5min":1.018,"15min":1.3380000000000001},"last":0.432},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.45600000000000002,"5min":0.35399999999999998,"15min":0.34200000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.95499999999999996},"last":0.54400000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":0.96699999999999997,"15min":3.04},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60799999999999998,"15min":0.59099999999999997},"min":{"1min":0.52000000000000002,"5min":0.43099999999999999,"15min":0.36899999999999999},"max":{"1min":0.71299999999999997,"5min":0.81000000000000005,"15min":0.89400000000000002},"last":0.47999999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63,"15min":0.65500000000000003},"min":{"1min":0.40699999999999997,"5min":0.40300000000000002,"15min":0.39800000000000002},"max":{"1min":0.73799999999999999,"5min":1.0409999999999999,"15min":4.032},"last":0.501},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66900000000000004,"15min":0.68000000000000005},"min":{"1min":0.57199999999999995,"5min":0.48799999999999999,"15min":0.45600000000000002},"max":{"1min":0.753,"5min":0.879,"15min":4.282},"last":0.61199999999999999}]},{"osd":4,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.51500000000000001},"min":{"1min":0.376,"5min":0.33300000000000002,"15min":0.26500000000000001},"max":{"1min":0.76400000000000001,"5min":0.95399999999999996,"15min":1.2869999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.496,"15min":0.52900000000000003},"min":{"1min":0.35799999999999998,"5min":0.32800000000000001,"15min":0.30399999999999999},"max":{"1min":0.67900000000000005,"5min":0.81699999999999995,"15min":1.101},"last":0.41299999999999998}]},{"osd":6,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.50600000000000001,"15min":0.52900000000000003},"min":{"1min":0.35299999999999998,"5min":0.32700000000000001,"15min":0.29099999999999998},"max":{"1min":0.65700000000000003,"5min":0.86599999999999999,"15min":1.7969999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.55200000000000005,"15min":0.56799999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.33600000000000002},"max":{"1min":0.69299999999999995,"5min":0.91800000000000004,"15min":1.6739999999999999},"last":0.58899999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:43:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.55100000000000005,"15min":0.55900000000000005},"min":{"1min":0.39500000000000002,"5min":0.36899999999999999,"15min":0.32700000000000001},"max":{"1min":0.76300000000000001,"5min":0.90600000000000003,"15min":1.008},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59199999999999997,"15min":0.627},"min":{"1min":0.46000000000000002,"5min":0.40600000000000003,"15min":0.33700000000000002},"max":{"1min":0.67500000000000004,"5min":0.91300000000000003,"15min":2},"last":0.63700000000000001}]}]}],"pool_statfs":[]}} 2024-01-27T02:44:17.315 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-01-27T02:31:58.568854+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=27, tm_hour=2, tm_min=31, tm_sec=58, tm_wday=5, tm_yday=27, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=27, tm_hour=2, tm_min=44, tm_sec=11, tm_wday=5, tm_yday=27, tm_isdst=0) 2024-01-27T02:44:17.315 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-01-27T02:44:37.317 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-27T02:44:37.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.460+0000 7f7f90a22700 1 Processor -- start 2024-01-27T02:44:37.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.460+0000 7f7f90a22700 1 -- start start 2024-01-27T02:44:37.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.461+0000 7f7f90a22700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f8c15be10 0x7f7f8c0b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:37.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.461+0000 7f7f90a22700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7f8c0b0180 0x7f7f8c0aee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:37.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.461+0000 7f7f90a22700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f8c0b1470 con 0x7f7f8c15c1e0 2024-01-27T02:44:37.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.461+0000 7f7f90a22700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f8c0b05d0 con 0x7f7f8c0b0180 2024-01-27T02:44:37.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.461+0000 7f7f90a22700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c0af370 con 0x7f7f8c15be10 2024-01-27T02:44:37.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.461+0000 7f7f8a59c700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f8c15be10 0x7f7f8c0b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:37.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.461+0000 7f7f89d9b700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7f8c0b0180 0x7f7f8c0aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:37.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.461+0000 7f7f8ad9d700 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f7f8c15c1e0 0x7f7f8c0c9500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.7:41158/0 (socket says 172.21.15.7:41158) 2024-01-27T02:44:37.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.461+0000 7f7f8ad9d700 1 -- 172.21.15.7:0/1388135887 learned_addr learned my addr 172.21.15.7:0/1388135887 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.462+0000 7f7f8a59c700 1 -- 172.21.15.7:0/1388135887 >> v2:172.21.15.188:3300/0 conn(0x7f7f8c0b0180 msgr2=0x7f7f8c0aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.462+0000 7f7f8a59c700 1 --2- 172.21.15.7:0/1388135887 >> v2:172.21.15.188:3300/0 conn(0x7f7f8c0b0180 0x7f7f8c0aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.462+0000 7f7f8a59c700 1 -- 172.21.15.7:0/1388135887 >> v1:172.21.15.188:6789/0 conn(0x7f7f8c15c1e0 legacy=0x7f7f8c0c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.462+0000 7f7f8a59c700 1 -- 172.21.15.7:0/1388135887 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c0b0860 con 0x7f7f8c15be10 2024-01-27T02:44:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.462+0000 7f7f8959a700 1 -- 172.21.15.7:0/1388135887 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3070468119 0 0) 0x7f7f8c0b1470 con 0x7f7f8c15c1e0 2024-01-27T02:44:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.462+0000 7f7f89d9b700 1 --2- 172.21.15.7:0/1388135887 >> v2:172.21.15.188:3300/0 conn(0x7f7f8c0b0180 0x7f7f8c0aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-27T02:44:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.462+0000 7f7f8a59c700 1 --2- 172.21.15.7:0/1388135887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f8c15be10 0x7f7f8c0b1900 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0098e0 tx=0x7f7f7c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1a2b585fbeebdb9 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.462+0000 7f7f8959a700 1 -- 172.21.15.7:0/1388135887 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c00b640 con 0x7f7f8c15be10 2024-01-27T02:44:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.462+0000 7f7f8959a700 1 -- 172.21.15.7:0/1388135887 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7c00b7a0 con 0x7f7f8c15be10 2024-01-27T02:44:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.462+0000 7f7f8959a700 1 -- 172.21.15.7:0/1388135887 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c00bb40 con 0x7f7f8c15be10 2024-01-27T02:44:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.463+0000 7f7f90a22700 1 -- 172.21.15.7:0/1388135887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f8c15be10 msgr2=0x7f7f8c0b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.463+0000 7f7f90a22700 1 --2- 172.21.15.7:0/1388135887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f8c15be10 0x7f7f8c0b1900 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0098e0 tx=0x7f7f7c0091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.463+0000 7f7f90a22700 1 -- 172.21.15.7:0/1388135887 shutdown_connections 2024-01-27T02:44:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.463+0000 7f7f90a22700 1 --2- 172.21.15.7:0/1388135887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f8c15be10 0x7f7f8c0b1900 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.463+0000 7f7f90a22700 1 --2- 172.21.15.7:0/1388135887 >> v2:172.21.15.188:3300/0 conn(0x7f7f8c0b0180 0x7f7f8c0aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.463+0000 7f7f90a22700 1 -- 172.21.15.7:0/1388135887 >> 172.21.15.7:0/1388135887 conn(0x7f7f8c0679b0 msgr2=0x7f7f8c067db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.463+0000 7f7f90a22700 1 -- 172.21.15.7:0/1388135887 shutdown_connections 2024-01-27T02:44:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.463+0000 7f7f90a22700 1 -- 172.21.15.7:0/1388135887 wait complete. 2024-01-27T02:44:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.464+0000 7f7f90a22700 1 Processor -- start 2024-01-27T02:44:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.465+0000 7f7f90a22700 1 -- start start 2024-01-27T02:44:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.466+0000 7f7f90a22700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f8c0b0180 0x7f7f8c0c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.466+0000 7f7f90a22700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f8c15be10 0x7f7f8c0caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.466+0000 7f7f90a22700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f8c15c1e0 0x7f7f8c0cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.466+0000 7f7f90a22700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c098a10 con 0x7f7f8c15be10 2024-01-27T02:44:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.466+0000 7f7f90a22700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c0992b0 con 0x7f7f8c15c1e0 2024-01-27T02:44:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.466+0000 7f7f90a22700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c0c76d0 con 0x7f7f8c0b0180 2024-01-27T02:44:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.466+0000 7f7f89d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f8c15be10 0x7f7f8c0caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.466+0000 7f7f89d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f8c15be10 0x7f7f8c0caa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44376/0 (socket says 172.21.15.7:44376) 2024-01-27T02:44:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.466+0000 7f7f89d9b700 1 -- 172.21.15.7:0/744479626 learned_addr learned my addr 172.21.15.7:0/744479626 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.466+0000 7f7f8ad9d700 1 --2- 172.21.15.7:0/744479626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f8c15c1e0 0x7f7f8c0cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.466+0000 7f7f8a59c700 1 --2- 172.21.15.7:0/744479626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f8c0b0180 0x7f7f8c0c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.466+0000 7f7f89d9b700 1 -- 172.21.15.7:0/744479626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f8c0b0180 msgr2=0x7f7f8c0c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.466+0000 7f7f89d9b700 1 --2- 172.21.15.7:0/744479626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f8c0b0180 0x7f7f8c0c7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.466+0000 7f7f89d9b700 1 -- 172.21.15.7:0/744479626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f8c15c1e0 msgr2=0x7f7f8c0cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.466+0000 7f7f89d9b700 1 --2- 172.21.15.7:0/744479626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f8c15c1e0 0x7f7f8c0cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.466+0000 7f7f89d9b700 1 -- 172.21.15.7:0/744479626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f80009420 con 0x7f7f8c15be10 2024-01-27T02:44:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.466+0000 7f7f8ad9d700 1 --2- 172.21.15.7:0/744479626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f8c15c1e0 0x7f7f8c0cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:44:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.466+0000 7f7f8a59c700 1 --2- 172.21.15.7:0/744479626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f8c0b0180 0x7f7f8c0c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:44:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.466+0000 7f7f89d9b700 1 --2- 172.21.15.7:0/744479626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f8c15be10 0x7f7f8c0caa20 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000ecb0 tx=0x7f7f800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.467+0000 7f7f7a7fc700 1 -- 172.21.15.7:0/744479626 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8000ee60 con 0x7f7f8c15be10 2024-01-27T02:44:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.467+0000 7f7f7a7fc700 1 -- 172.21.15.7:0/744479626 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8000cbe0 con 0x7f7f8c15be10 2024-01-27T02:44:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.467+0000 7f7f90a22700 1 -- 172.21.15.7:0/744479626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7c009420 con 0x7f7f8c15be10 2024-01-27T02:44:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.467+0000 7f7f7a7fc700 1 -- 172.21.15.7:0/744479626 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f80013470 con 0x7f7f8c15be10 2024-01-27T02:44:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.467+0000 7f7f90a22700 1 -- 172.21.15.7:0/744479626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8c0c7930 con 0x7f7f8c15be10 2024-01-27T02:44:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.468+0000 7f7f90a22700 1 -- 172.21.15.7:0/744479626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f8c065de0 con 0x7f7f8c15be10 2024-01-27T02:44:37.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.468+0000 7f7f7a7fc700 1 -- 172.21.15.7:0/744479626 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7f80013820 con 0x7f7f8c15be10 2024-01-27T02:44:37.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.468+0000 7f7f7a7fc700 1 --2- 172.21.15.7:0/744479626 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7f6c066f30 0x7f7f6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:37.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.469+0000 7f7f7a7fc700 1 -- 172.21.15.7:0/744479626 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7f80084620 con 0x7f7f8c15be10 2024-01-27T02:44:37.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.469+0000 7f7f8a59c700 1 --2- 172.21.15.7:0/744479626 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7f6c066f30 0x7f7f6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:37.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.469+0000 7f7f8a59c700 1 --2- 172.21.15.7:0/744479626 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7f6c066f30 0x7f7f6c0693e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0091e0 tx=0x7f7f7c017000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:37.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.473+0000 7f7f7a7fc700 1 -- 172.21.15.7:0/744479626 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f80051b90 con 0x7f7f8c15be10 2024-01-27T02:44:37.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.671+0000 7f7f90a22700 1 -- 172.21.15.7:0/744479626 --> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7f8c04c710 con 0x7f7f6c066f30 2024-01-27T02:44:37.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.674+0000 7f7f7a7fc700 1 -- 172.21.15.7:0/744479626 <== mgr.4108 v2:172.21.15.188:6832/38712 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50386 (secure 0 0 0) 0x7f7f8c04c710 con 0x7f7f6c066f30 2024-01-27T02:44:37.675 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:44:37.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.676+0000 7f7f90a22700 1 -- 172.21.15.7:0/744479626 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7f6c066f30 msgr2=0x7f7f6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:37.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.676+0000 7f7f90a22700 1 --2- 172.21.15.7:0/744479626 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7f6c066f30 0x7f7f6c0693e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0091e0 tx=0x7f7f7c017000 comp rx=0 tx=0).stop 2024-01-27T02:44:37.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.676+0000 7f7f90a22700 1 -- 172.21.15.7:0/744479626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f8c15be10 msgr2=0x7f7f8c0caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:37.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.676+0000 7f7f90a22700 1 --2- 172.21.15.7:0/744479626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f8c15be10 0x7f7f8c0caa20 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000ecb0 tx=0x7f7f800091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:37.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.676+0000 7f7f90a22700 1 -- 172.21.15.7:0/744479626 shutdown_connections 2024-01-27T02:44:37.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.676+0000 7f7f90a22700 1 --2- 172.21.15.7:0/744479626 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f7f6c066f30 0x7f7f6c0693e0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:37.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.676+0000 7f7f90a22700 1 --2- 172.21.15.7:0/744479626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f8c15c1e0 0x7f7f8c0cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:37.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.676+0000 7f7f90a22700 1 --2- 172.21.15.7:0/744479626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f8c0b0180 0x7f7f8c0c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:37.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.676+0000 7f7f90a22700 1 --2- 172.21.15.7:0/744479626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f8c15be10 0x7f7f8c0caa20 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:37.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.676+0000 7f7f90a22700 1 -- 172.21.15.7:0/744479626 >> 172.21.15.7:0/744479626 conn(0x7f7f8c0679b0 msgr2=0x7f7f8c0c9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:37.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.676+0000 7f7f90a22700 1 -- 172.21.15.7:0/744479626 shutdown_connections 2024-01-27T02:44:37.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.676+0000 7f7f90a22700 1 -- 172.21.15.7:0/744479626 wait complete. 2024-01-27T02:44:37.678 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-27T02:44:37.690 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":557,"stamp":"2024-01-27T02:44:35.757068+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9346,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553153,"statfs":{"total":1600000000,"available":1590428936,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002412"},"pg_stats":[{"pgid":"1.0","version":"187'119","reported_seq":322,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-01-27T02:44:18.378579+0000","last_change":"2024-01-27T02:44:18.378541+0000","last_active":"2024-01-27T02:44:18.378579+0000","last_peered":"2024-01-27T02:44:18.378579+0000","last_clean":"2024-01-27T02:44:18.378579+0000","last_became_active":"2024-01-27T02:31:59.594460+0000","last_became_peered":"2024-01-27T02:31:59.594460+0000","last_unstale":"2024-01-27T02:44:18.378579+0000","last_undegraded":"2024-01-27T02:44:18.378579+0000","last_fullsized":"2024-01-27T02:44:18.378579+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'119","last_scrub_stamp":"2024-01-27T02:44:18.378452+0000","last_deep_scrub":"187'119","last_deep_scrub_stamp":"2024-01-27T02:44:18.378452+0000","last_clean_scrub_stamp":"2024-01-27T02:44:18.378452+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-28T13:37:16.282463+0000","scrub_duration":0.0032594579999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575001,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1600,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193711,"statfs":{"total":200000000,"available":198360841,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70799999999999996,"5min":0.83499999999999996,"15min":0.85499999999999998},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.30099999999999999},"max":{"1min":0.72199999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.67400000000000004}]},{"osd":1,"last update":"Sat Jan 27 02:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.59499999999999997,"15min":0.59399999999999997},"min":{"1min":0.40799999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69099999999999995,"5min":0.78800000000000003,"15min":0.82399999999999995},"last":0.626},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59599999999999997,"15min":0.57599999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77700000000000002,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.73999999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.39000000000000001,"5min":0.38400000000000001,"15min":0.376},"max":{"1min":0.79300000000000004,"5min":0.79400000000000004,"15min":0.86399999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.60699999999999998,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.75900000000000001,"5min":0.80300000000000005,"15min":0.83899999999999997},"last":0.79900000000000004}]},{"osd":3,"last update":"Sat Jan 27 02:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.59599999999999997,"15min":0.59199999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.74299999999999999,"5min":0.78900000000000003,"15min":0.90300000000000002},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.626,"15min":0.60999999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.40999999999999998},"max":{"1min":0.754,"5min":0.81599999999999995,"15min":0.88600000000000001},"last":0.76100000000000001}]},{"osd":4,"last update":"Sat Jan 27 02:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.47099999999999997,"15min":0.48099999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.27200000000000002},"max":{"1min":0.69499999999999995,"5min":0.73899999999999999,"15min":0.78600000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.29899999999999999},"max":{"1min":0.71399999999999997,"5min":0.747,"15min":0.80000000000000004},"last":0.64700000000000002}]},{"osd":5,"last update":"Sat Jan 27 02:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52700000000000002,"15min":0.53100000000000003},"min":{"1min":0.32900000000000001,"5min":0.32500000000000001,"15min":0.307},"max":{"1min":0.68500000000000005,"5min":0.79500000000000004,"15min":0.85399999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55500000000000005,"15min":0.56000000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72299999999999998,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.56799999999999995}]},{"osd":6,"last update":"Sat Jan 27 02:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.746,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68500000000000005,"5min":0.81799999999999995,"15min":0.90800000000000003},"last":0.59399999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1024,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194288,"statfs":{"total":200000000,"available":198951209,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.60999999999999999,"15min":0.58499999999999996},"min":{"1min":0.42999999999999999,"5min":0.38800000000000001,"15min":0.29999999999999999},"max":{"1min":0.76000000000000001,"5min":1.0409999999999999,"15min":4.125},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.58599999999999997,"15min":0.56799999999999995},"min":{"1min":0.438,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68799999999999994,"5min":0.96599999999999997,"15min":1.216},"last":0.59999999999999998}]},{"osd":1,"last update":"Sat Jan 27 02:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.437,"5min":0.376,"15min":0.34100000000000003},"max":{"1min":0.76900000000000002,"5min":0.95499999999999996,"15min":8.2699999999999996},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60999999999999999,"15min":0.68200000000000005},"min":{"1min":0.41099999999999998,"5min":0.33900000000000002,"15min":0.31900000000000001},"max":{"1min":0.83399999999999996,"5min":1.0129999999999999,"15min":26.411999999999999},"last":0.57699999999999996}]},{"osd":2,"last update":"Sat Jan 27 02:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59199999999999997,"15min":0.63},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.35799999999999998},"max":{"1min":0.76800000000000002,"5min":0.78300000000000003,"15min":17.143999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61299999999999999,"15min":0.60999999999999999},"min":{"1min":0.46400000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73499999999999999,"5min":0.83699999999999997,"15min":1.76},"last":0.48999999999999999}]},{"osd":3,"last update":"Sat Jan 27 02:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.39700000000000002,"5min":0.39400000000000002,"15min":0.376},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":4.3689999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60799999999999998,"15min":0.60099999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79100000000000004,"5min":0.81699999999999995,"15min":2.1970000000000001},"last":0.58899999999999997}]},{"osd":4,"last update":"Sat Jan 27 02:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46800000000000003,"15min":0.47899999999999998},"min":{"1min":0.34300000000000003,"5min":0.313,"15min":0.30199999999999999},"max":{"1min":0.69799999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.50600000000000001,"15min":0.52300000000000002},"min":{"1min":0.35299999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.754,"5min":0.83699999999999997,"15min":0.89600000000000002},"last":0.53600000000000003}]},{"osd":5,"last update":"Sat Jan 27 02:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50600000000000001,"15min":0.52100000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.72699999999999998,"5min":0.73799999999999999,"15min":0.81299999999999994},"last":0.501},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55000000000000004,"15min":0.57299999999999995},"min":{"1min":0.41499999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.41699999999999998}]},{"osd":7,"last update":"Sat Jan 27 02:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57799999999999996,"15min":0.59499999999999997},"min":{"1min":0.46200000000000002,"5min":0.38400000000000001,"15min":0.35999999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":1.0009999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.56499999999999995},"min":{"1min":0.36499999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":1.038},"last":0.47599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1024,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194288,"statfs":{"total":200000000,"available":198951209,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40200000000000002,"15min":0.39000000000000001},"min":{"1min":0.30499999999999999,"5min":0.248,"15min":0.23599999999999999},"max":{"1min":0.59899999999999998,"5min":0.64900000000000002,"15min":0.97199999999999998},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.435,"15min":0.41499999999999998},"min":{"1min":0.28499999999999998,"5min":0.28199999999999997,"15min":0.27700000000000002},"max":{"1min":0.56599999999999995,"5min":0.66100000000000003,"15min":1.121},"last":0.54000000000000004}]},{"osd":2,"last update":"Sat Jan 27 02:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.41899999999999998,"15min":0.39600000000000002},"min":{"1min":0.30199999999999999,"5min":0.27500000000000002,"15min":0.23999999999999999},"max":{"1min":0.57899999999999996,"5min":0.70199999999999996,"15min":1.25},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.46000000000000002,"15min":0.44400000000000001},"min":{"1min":0.317,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.82999999999999996},"last":0.44}]},{"osd":3,"last update":"Sat Jan 27 02:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.50600000000000001,"15min":0.48499999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32100000000000001},"max":{"1min":0.68799999999999994,"5min":0.79900000000000004,"15min":0.90900000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.47199999999999998,"15min":0.46200000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.62,"5min":0.72599999999999998,"15min":0.98999999999999999},"last":0.495}]},{"osd":4,"last update":"Sat Jan 27 02:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66000000000000003,"15min":0.64900000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.373},"max":{"1min":0.93999999999999995,"5min":1.075,"15min":3.2890000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68300000000000005,"15min":0.65900000000000003},"min":{"1min":0.51400000000000001,"5min":0.46400000000000002,"15min":0.44500000000000001},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.68999999999999995}]},{"osd":5,"last update":"Sat Jan 27 02:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.68600000000000005,"15min":0.66600000000000004},"min":{"1min":0.51000000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89100000000000001,"5min":0.94599999999999995,"15min":3.4500000000000002},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.58299999999999996,"5min":0.45600000000000002,"15min":0.42399999999999999},"max":{"1min":0.96499999999999997,"5min":1.004,"15min":3.181},"last":0.67100000000000004}]},{"osd":6,"last update":"Sat Jan 27 02:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72699999999999998,"15min":0.70799999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.42699999999999999},"max":{"1min":1.016,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.73399999999999999,"15min":0.70699999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.50800000000000001},"max":{"1min":0.82299999999999995,"5min":1.143,"15min":1.869},"last":0.90400000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:44:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.73799999999999999,"15min":0.71399999999999997},"min":{"1min":0.54200000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.081,"5min":1.181,"15min":1.6870000000000001},"last":0.873},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73399999999999999,"15min":0.71499999999999997},"min":{"1min":0.54700000000000004,"5min":0.501,"15min":0.45300000000000001},"max":{"1min":0.86799999999999999,"5min":1.0429999999999999,"15min":1.708},"last":0.84499999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575001,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1600,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193711,"statfs":{"total":200000000,"available":198360841,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Jan 27 02:44:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38500000000000001,"15min":0.39300000000000002},"min":{"1min":0.28399999999999997,"5min":0.24399999999999999,"15min":0.23799999999999999},"max":{"1min":0.52100000000000002,"5min":0.58999999999999997,"15min":0.81599999999999995},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41799999999999998,"15min":0.41899999999999998},"min":{"1min":0.29099999999999998,"5min":0.26000000000000001,"15min":0.223},"max":{"1min":0.58099999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.42599999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:44:16 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.56399999999999995,"5min":0.625,"15min":0.64000000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.44500000000000001,"15min":0.44400000000000001},"min":{"1min":0.32200000000000001,"5min":0.29499999999999998,"15min":0.28799999999999998},"max":{"1min":0.48399999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.30599999999999999}]},{"osd":3,"last update":"Sat Jan 27 02:44:16 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.436,"15min":0.434},"min":{"1min":0.317,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.505,"5min":0.66600000000000004,"15min":0.67300000000000004},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48799999999999999,"15min":0.48199999999999998},"min":{"1min":0.35199999999999998,"5min":0.34300000000000003,"15min":0.29099999999999998},"max":{"1min":0.61499999999999999,"5min":0.71399999999999997,"15min":0.71699999999999997},"last":0.40300000000000002}]},{"osd":4,"last update":"Sat Jan 27 02:44:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.47999999999999998,"5min":0.432,"15min":0.42099999999999999},"max":{"1min":0.79100000000000004,"5min":0.97599999999999998,"15min":0.997},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63400000000000001,"15min":0.65000000000000002},"min":{"1min":0.47299999999999998,"5min":0.42199999999999999,"15min":0.41999999999999998},"max":{"1min":0.79200000000000004,"5min":0.87,"15min":0.96699999999999997},"last":0.58899999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:44:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66200000000000003,"15min":0.66400000000000003},"min":{"1min":0.48499999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.84699999999999998,"5min":0.91900000000000004,"15min":2.4860000000000002},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68200000000000005,"15min":0.67900000000000005},"min":{"1min":0.53400000000000003,"5min":0.504,"15min":0.44},"max":{"1min":0.85499999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.56499999999999995}]},{"osd":6,"last update":"Sat Jan 27 02:44:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66500000000000004,"15min":0.66800000000000004},"min":{"1min":0.56000000000000005,"5min":0.46400000000000002,"15min":0.42499999999999999},"max":{"1min":0.86599999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69399999999999995,"15min":0.68700000000000006},"min":{"1min":0.46600000000000003,"5min":0.46500000000000002,"15min":0.45000000000000001},"max":{"1min":0.82899999999999996,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.57599999999999996}]},{"osd":7,"last update":"Sat Jan 27 02:44:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68500000000000005,"15min":0.69099999999999995},"min":{"1min":0.54200000000000004,"5min":0.47099999999999997,"15min":0.42599999999999999},"max":{"1min":0.83499999999999996,"5min":0.94999999999999996,"15min":1.014},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.69599999999999995,"15min":0.69199999999999995},"min":{"1min":0.53100000000000003,"5min":0.499,"15min":0.498},"max":{"1min":0.90500000000000003,"5min":0.92500000000000004,"15min":0.97299999999999998},"last":0.72699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1024,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194288,"statfs":{"total":200000000,"available":198951209,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.40200000000000002,"15min":0.40999999999999998},"min":{"1min":0.29699999999999999,"5min":0.23300000000000001,"15min":0.23000000000000001},"max":{"1min":0.63,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.44800000000000001,"15min":0.52100000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.30499999999999999},"max":{"1min":0.51100000000000001,"5min":0.67700000000000005,"15min":6.2729999999999997},"last":0.57799999999999996}]},{"osd":1,"last update":"Sat Jan 27 02:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41199999999999998,"15min":0.41999999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.23799999999999999},"max":{"1min":0.57399999999999995,"5min":0.71999999999999997,"15min":0.81399999999999995},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.432,"15min":0.438},"min":{"1min":0.31,"5min":0.28999999999999998,"15min":0.28899999999999998},"max":{"1min":0.52900000000000003,"5min":0.65800000000000003,"15min":0.70099999999999996},"last":0.52200000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40600000000000003,"15min":0.41599999999999998},"min":{"1min":0.28899999999999998,"5min":0.28499999999999998,"15min":0.27800000000000002},"max":{"1min":0.51100000000000001,"5min":0.60699999999999998,"15min":0.74299999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.47299999999999998,"15min":0.48199999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.314},"max":{"1min":0.56399999999999995,"5min":0.70699999999999996,"15min":0.80800000000000005},"last":0.55700000000000005}]},{"osd":4,"last update":"Sat Jan 27 02:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68100000000000005,"15min":0.67200000000000004},"min":{"1min":0.53200000000000003,"5min":0.47599999999999998,"15min":0.39300000000000002},"max":{"1min":0.80300000000000005,"5min":0.95199999999999996,"15min":0.98299999999999998},"last":0.871},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.68999999999999995,"15min":0.73299999999999998},"min":{"1min":0.47399999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.78000000000000003,"5min":0.94399999999999995,"15min":3.581},"last":0.77300000000000002}]},{"osd":5,"last update":"Sat Jan 27 02:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.65000000000000002,"15min":0.68500000000000005},"min":{"1min":0.437,"5min":0.432,"15min":0.43099999999999999},"max":{"1min":0.80400000000000005,"5min":0.90400000000000003,"15min":3.5369999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65400000000000003,"15min":0.66400000000000003},"min":{"1min":0.45400000000000001,"5min":0.44800000000000001,"15min":0.41099999999999998},"max":{"1min":0.81999999999999995,"5min":0.89000000000000001,"15min":1.0529999999999999},"last":0.84299999999999997}]},{"osd":6,"last update":"Sat Jan 27 02:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.66900000000000004,"15min":0.66800000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42299999999999999},"max":{"1min":0.745,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69799999999999995,"15min":0.68799999999999994},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.81799999999999995,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.90900000000000003}]},{"osd":7,"last update":"Sat Jan 27 02:44:14 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.68300000000000005,"15min":0.68200000000000005},"min":{"1min":0.51200000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.83599999999999997,"5min":1.55,"15min":1.55},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.70499999999999996,"15min":0.69899999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.42699999999999999},"max":{"1min":0.77300000000000002,"5min":1.615,"15min":1.615},"last":0.72499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1024,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194288,"statfs":{"total":200000000,"available":198951209,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.40600000000000003,"15min":0.40699999999999997},"min":{"1min":0.27000000000000002,"5min":0.25800000000000001,"15min":0.253},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":4.7380000000000004},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41799999999999998,"15min":0.40899999999999997},"min":{"1min":0.28299999999999997,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.54400000000000004,"5min":0.63700000000000001,"15min":1.0189999999999999},"last":0.38900000000000001}]},{"osd":1,"last update":"Sat Jan 27 02:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.40899999999999997,"15min":0.42299999999999999},"min":{"1min":0.26200000000000001,"5min":0.24399999999999999,"15min":0.22900000000000001},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":4.5739999999999998},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.45900000000000002,"15min":0.46899999999999997},"min":{"1min":0.32000000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.67000000000000004,"5min":0.751,"15min":5.0529999999999999},"last":0.43099999999999999}]},{"osd":2,"last update":"Sat Jan 27 02:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.47499999999999998,"15min":0.46300000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.28100000000000003},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.379,"5min":0.33000000000000002,"15min":0.32800000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":5.1879999999999997},"last":0.378}]},{"osd":4,"last update":"Sat Jan 27 02:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63300000000000001,"15min":0.64700000000000002},"min":{"1min":0.46899999999999997,"5min":0.42999999999999999,"15min":0.41699999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":4.6660000000000004},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.64900000000000002,"15min":0.64000000000000001},"min":{"1min":0.45300000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.59699999999999998}]},{"osd":5,"last update":"Sat Jan 27 02:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.65800000000000003,"15min":0.64700000000000002},"min":{"1min":0.47499999999999998,"5min":0.46899999999999997,"15min":0.40799999999999997},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.409},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66600000000000004,"15min":0.70499999999999996},"min":{"1min":0.47799999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.90700000000000003,"5min":0.96599999999999997,"15min":4.7960000000000003},"last":0.69499999999999995}]},{"osd":6,"last update":"Sat Jan 27 02:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67500000000000004,"15min":0.68799999999999994},"min":{"1min":0.436,"5min":0.436,"15min":0.39800000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":5.0069999999999997},"last":0.624},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.68000000000000005,"15min":0.67300000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":1.103},"last":0.71599999999999997}]},{"osd":7,"last update":"Sat Jan 27 02:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67900000000000005,"15min":0.68799999999999994},"min":{"1min":0.46899999999999997,"5min":0.46500000000000002,"15min":0.46000000000000002},"max":{"1min":0.995,"5min":0.995,"15min":4.7130000000000001},"last":0.75},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69599999999999995,"15min":0.69299999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.435},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.68300000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1024,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194288,"statfs":{"total":200000000,"available":198951209,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.58799999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.314},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":1.3839999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.58699999999999997,"15min":0.61199999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.66300000000000003,"5min":0.84499999999999997,"15min":7.0670000000000002},"last":0.57399999999999995}]},{"osd":1,"last update":"Sat Jan 27 02:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59099999999999997,"15min":0.60999999999999999},"min":{"1min":0.437,"5min":0.39400000000000002,"15min":0.35399999999999998},"max":{"1min":0.70599999999999996,"5min":0.81799999999999995,"15min":5.1529999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.60099999999999998,"15min":0.61099999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.39000000000000001},"max":{"1min":0.83999999999999997,"5min":0.879,"15min":3.0419999999999998},"last":0.56200000000000006}]},{"osd":2,"last update":"Sat Jan 27 02:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.61399999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.749,"5min":0.83099999999999996,"15min":5.0019999999999998},"last":0.504},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59699999999999998,"15min":0.61499999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78100000000000003,"5min":0.86099999999999999,"15min":2.073},"last":0.60099999999999998}]},{"osd":3,"last update":"Sat Jan 27 02:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59999999999999998,"15min":0.61199999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.36199999999999999},"max":{"1min":0.76100000000000001,"5min":0.78800000000000003,"15min":6.7220000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61099999999999999,"15min":0.65200000000000002},"min":{"1min":0.46000000000000002,"5min":0.42499999999999999,"15min":0.41399999999999998},"max":{"1min":0.71399999999999997,"5min":0.872,"15min":6.5510000000000002},"last":0.47099999999999997}]},{"osd":5,"last update":"Sat Jan 27 02:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.48999999999999999,"15min":0.49099999999999999},"min":{"1min":0.36599999999999999,"5min":0.314,"15min":0.312},"max":{"1min":0.68100000000000005,"5min":0.78900000000000003,"15min":1.3149999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.495,"15min":0.51900000000000002},"min":{"1min":0.35799999999999998,"5min":0.32500000000000001,"15min":0.29999999999999999},"max":{"1min":0.66900000000000004,"5min":0.79800000000000004,"15min":6.9269999999999996},"last":0.41199999999999998}]},{"osd":6,"last update":"Sat Jan 27 02:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.51200000000000001,"15min":0.52700000000000002},"min":{"1min":0.39100000000000001,"5min":0.34899999999999998,"15min":0.32900000000000001},"max":{"1min":0.73699999999999999,"5min":0.85399999999999998,"15min":5.2229999999999999},"last":0.371},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54800000000000004,"15min":0.55100000000000005},"min":{"1min":0.373,"5min":0.35799999999999998,"15min":0.33200000000000002},"max":{"1min":0.71599999999999997,"5min":0.83299999999999996,"15min":1.46},"last":0.51400000000000001}]},{"osd":7,"last update":"Sat Jan 27 02:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53900000000000003,"15min":0.56999999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33600000000000002},"max":{"1min":0.69599999999999995,"5min":0.83499999999999996,"15min":7.5339999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57399999999999995,"15min":0.59599999999999997},"min":{"1min":0.39300000000000002,"5min":0.34699999999999998,"15min":0.34599999999999997},"max":{"1min":0.76200000000000001,"5min":0.80900000000000005,"15min":4.9550000000000001},"last":0.55000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1024,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194288,"statfs":{"total":200000000,"available":198951209,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Jan 27 02:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56699999999999995,"15min":0.57699999999999996},"min":{"1min":0.35699999999999998,"5min":0.33900000000000002,"15min":0.28899999999999998},"max":{"1min":0.69099999999999995,"5min":0.81299999999999994,"15min":1.071},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.42499999999999999,"5min":0.38500000000000001,"15min":0.27600000000000002},"max":{"1min":0.74099999999999999,"5min":0.83599999999999997,"15min":3.242},"last":0.58199999999999996}]},{"osd":1,"last update":"Sat Jan 27 02:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.63600000000000001,"15min":0.625},"min":{"1min":0.41899999999999998,"5min":0.38100000000000001,"15min":0.32800000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":1.3380000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.62,"15min":0.629},"min":{"1min":0.48499999999999999,"5min":0.375,"15min":0.34200000000000003},"max":{"1min":0.77600000000000002,"5min":0.81299999999999994,"15min":0.95499999999999996},"last":0.75}]},{"osd":2,"last update":"Sat Jan 27 02:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.378,"15min":0.372},"max":{"1min":0.75800000000000001,"5min":0.81999999999999995,"15min":3.04},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.59299999999999997,"15min":0.58799999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.68799999999999994,"5min":0.81000000000000005,"15min":0.89400000000000002},"last":0.64300000000000002}]},{"osd":3,"last update":"Sat Jan 27 02:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63,"15min":0.65000000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.39800000000000002},"max":{"1min":1.212,"5min":1.212,"15min":4.032},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67200000000000004,"15min":0.68000000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.224,"5min":1.224,"15min":4.282},"last":0.70699999999999996}]},{"osd":4,"last update":"Sat Jan 27 02:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.499,"15min":0.51300000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.26500000000000001},"max":{"1min":0.79400000000000004,"5min":0.95399999999999996,"15min":1.2869999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.50600000000000001,"15min":0.52400000000000002},"min":{"1min":0.371,"5min":0.35799999999999998,"15min":0.30399999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":1.101},"last":0.47899999999999998}]},{"osd":6,"last update":"Sat Jan 27 02:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.5,"15min":0.52100000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.29099999999999998},"max":{"1min":0.72199999999999998,"5min":0.86599999999999999,"15min":1.7969999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55200000000000005,"15min":0.56699999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.33600000000000002},"max":{"1min":0.88700000000000001,"5min":0.91800000000000004,"15min":1.6739999999999999},"last":0.45800000000000002}]},{"osd":7,"last update":"Sat Jan 27 02:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55000000000000004,"15min":0.55600000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.32700000000000001},"max":{"1min":0.76500000000000001,"5min":0.90600000000000003,"15min":1.008},"last":0.495},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.59399999999999997,"15min":0.622},"min":{"1min":0.45000000000000001,"5min":0.40600000000000003,"15min":0.33700000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":2},"last":0.56000000000000005}]}]}],"pool_statfs":[]}} 2024-01-27T02:44:37.692 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-01-27T02:44:37.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.823+0000 7f57eedb7700 1 Processor -- start 2024-01-27T02:44:37.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.823+0000 7f57eedb7700 1 -- start start 2024-01-27T02:44:37.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.824+0000 7f57eedb7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f57e80b0f70 0x7f57e80693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:37.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.824+0000 7f57eedb7700 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f57e80cc8d0 0x7f57e80679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:37.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.824+0000 7f57eedb7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e8069930 con 0x7f57e80b1340 2024-01-27T02:44:37.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.824+0000 7f57eedb7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e8068840 con 0x7f57e80b0f70 2024-01-27T02:44:37.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.824+0000 7f57eedb7700 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e815e820 con 0x7f57e80cc8d0 2024-01-27T02:44:37.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.824+0000 7f57ecb53700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f57e80b0f70 0x7f57e80693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:37.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.824+0000 7f57ed354700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f57e80b1340 0x7f57e806b4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46574/0 (socket says 172.21.15.7:46574) 2024-01-27T02:44:37.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.824+0000 7f57ed354700 1 -- 172.21.15.7:0/4125061951 learned_addr learned my addr 172.21.15.7:0/4125061951 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:37.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.824+0000 7f57e7fff700 1 --2- 172.21.15.7:0/4125061951 >> v2:172.21.15.188:3300/0 conn(0x7f57e80cc8d0 0x7f57e80679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:37.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.824+0000 7f57ecb53700 1 -- 172.21.15.7:0/4125061951 >> v2:172.21.15.188:3300/0 conn(0x7f57e80cc8d0 msgr2=0x7f57e80679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-27T02:44:37.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.824+0000 7f57ecb53700 1 --2- 172.21.15.7:0/4125061951 >> v2:172.21.15.188:3300/0 conn(0x7f57e80cc8d0 0x7f57e80679b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:37.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.824+0000 7f57ecb53700 1 -- 172.21.15.7:0/4125061951 >> v1:172.21.15.7:6789/0 conn(0x7f57e80b1340 legacy=0x7f57e806b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:37.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.824+0000 7f57ecb53700 1 -- 172.21.15.7:0/4125061951 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e815dc10 con 0x7f57e80b0f70 2024-01-27T02:44:37.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.824+0000 7f57e77fe700 1 -- 172.21.15.7:0/4125061951 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4195277725 0 0) 0x7f57e8069930 con 0x7f57e80b1340 2024-01-27T02:44:37.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.824+0000 7f57e7fff700 1 --2- 172.21.15.7:0/4125061951 >> v2:172.21.15.188:3300/0 conn(0x7f57e80cc8d0 0x7f57e80679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-27T02:44:37.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.824+0000 7f57ecb53700 1 --2- 172.21.15.7:0/4125061951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e80b0f70 0x7f57e80693f0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f57e806d3c0 tx=0x7f57dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fdfd3d2004c429 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:37.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.824+0000 7f57e77fe700 1 -- 172.21.15.7:0/4125061951 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00b6b0 con 0x7f57e80b0f70 2024-01-27T02:44:37.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.825+0000 7f57e77fe700 1 -- 172.21.15.7:0/4125061951 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc00b810 con 0x7f57e80b0f70 2024-01-27T02:44:37.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.825+0000 7f57e77fe700 1 -- 172.21.15.7:0/4125061951 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00bbb0 con 0x7f57e80b0f70 2024-01-27T02:44:37.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.825+0000 7f57eedb7700 1 -- 172.21.15.7:0/4125061951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e80b0f70 msgr2=0x7f57e80693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:37.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.825+0000 7f57eedb7700 1 --2- 172.21.15.7:0/4125061951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e80b0f70 0x7f57e80693f0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f57e806d3c0 tx=0x7f57dc0091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:37.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.825+0000 7f57eedb7700 1 -- 172.21.15.7:0/4125061951 shutdown_connections 2024-01-27T02:44:37.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.825+0000 7f57eedb7700 1 --2- 172.21.15.7:0/4125061951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e80b0f70 0x7f57e80693f0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:37.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.825+0000 7f57eedb7700 1 --2- 172.21.15.7:0/4125061951 >> v2:172.21.15.188:3300/0 conn(0x7f57e80cc8d0 0x7f57e80679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:37.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.825+0000 7f57eedb7700 1 -- 172.21.15.7:0/4125061951 >> 172.21.15.7:0/4125061951 conn(0x7f57e8063580 msgr2=0x7f57e8063980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:37.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.825+0000 7f57eedb7700 1 -- 172.21.15.7:0/4125061951 shutdown_connections 2024-01-27T02:44:37.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.825+0000 7f57eedb7700 1 -- 172.21.15.7:0/4125061951 wait complete. 2024-01-27T02:44:37.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.827+0000 7f57eedb7700 1 Processor -- start 2024-01-27T02:44:37.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.827+0000 7f57eedb7700 1 -- start start 2024-01-27T02:44:37.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.828+0000 7f57eedb7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e80b0f70 0x7f57e8154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:37.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.828+0000 7f57eedb7700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57e80b1340 0x7f57e815baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:37.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.828+0000 7f57eedb7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57e80cc8d0 0x7f57e8146730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:37.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.828+0000 7f57eedb7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e8152450 con 0x7f57e80b0f70 2024-01-27T02:44:37.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.828+0000 7f57eedb7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e815a620 con 0x7f57e80cc8d0 2024-01-27T02:44:37.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.828+0000 7f57eedb7700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e815b5f0 con 0x7f57e80b1340 2024-01-27T02:44:37.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.828+0000 7f57ecb53700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e80b0f70 0x7f57e8154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:37.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.828+0000 7f57ecb53700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e80b0f70 0x7f57e8154f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44392/0 (socket says 172.21.15.7:44392) 2024-01-27T02:44:37.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.828+0000 7f57ecb53700 1 -- 172.21.15.7:0/3926209137 learned_addr learned my addr 172.21.15.7:0/3926209137 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:37.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.828+0000 7f57ed354700 1 --2- 172.21.15.7:0/3926209137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57e80cc8d0 0x7f57e8146730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:37.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.828+0000 7f57e7fff700 1 --2- 172.21.15.7:0/3926209137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57e80b1340 0x7f57e815baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:37.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.829+0000 7f57ecb53700 1 -- 172.21.15.7:0/3926209137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57e80b1340 msgr2=0x7f57e815baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:37.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.829+0000 7f57ecb53700 1 --2- 172.21.15.7:0/3926209137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57e80b1340 0x7f57e815baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:37.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.829+0000 7f57ecb53700 1 -- 172.21.15.7:0/3926209137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57e80cc8d0 msgr2=0x7f57e8146730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:37.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.829+0000 7f57ecb53700 1 --2- 172.21.15.7:0/3926209137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57e80cc8d0 0x7f57e8146730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:37.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.829+0000 7f57ecb53700 1 -- 172.21.15.7:0/3926209137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc009420 con 0x7f57e80b0f70 2024-01-27T02:44:37.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.829+0000 7f57ecb53700 1 --2- 172.21.15.7:0/3926209137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e80b0f70 0x7f57e8154f70 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f57dc0093f0 tx=0x7f57dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:37.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.829+0000 7f57e4ff9700 1 -- 172.21.15.7:0/3926209137 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00bb90 con 0x7f57e80b0f70 2024-01-27T02:44:37.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.829+0000 7f57e4ff9700 1 -- 172.21.15.7:0/3926209137 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc011420 con 0x7f57e80b0f70 2024-01-27T02:44:37.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.829+0000 7f57eedb7700 1 -- 172.21.15.7:0/3926209137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e815c030 con 0x7f57e80b0f70 2024-01-27T02:44:37.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.829+0000 7f57e4ff9700 1 -- 172.21.15.7:0/3926209137 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc0117c0 con 0x7f57e80b0f70 2024-01-27T02:44:37.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.829+0000 7f57eedb7700 1 -- 172.21.15.7:0/3926209137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e815b160 con 0x7f57e80b0f70 2024-01-27T02:44:37.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.830+0000 7f57eedb7700 1 -- 172.21.15.7:0/3926209137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e8072060 con 0x7f57e80b0f70 2024-01-27T02:44:37.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.831+0000 7f57e4ff9700 1 -- 172.21.15.7:0/3926209137 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f57dc011b30 con 0x7f57e80b0f70 2024-01-27T02:44:37.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.831+0000 7f57e4ff9700 1 --2- 172.21.15.7:0/3926209137 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f57cc066f00 0x7f57cc0693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:37.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.831+0000 7f57e4ff9700 1 -- 172.21.15.7:0/3926209137 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f57dc085780 con 0x7f57e80b0f70 2024-01-27T02:44:37.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.831+0000 7f57e7fff700 1 --2- 172.21.15.7:0/3926209137 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f57cc066f00 0x7f57cc0693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:37.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.832+0000 7f57e7fff700 1 --2- 172.21.15.7:0/3926209137 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f57cc066f00 0x7f57cc0693b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f57d4006f00 tx=0x7f57d4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:37.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:37.836+0000 7f57e4ff9700 1 -- 172.21.15.7:0/3926209137 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f57dc052cf0 con 0x7f57e80b0f70 2024-01-27T02:44:38.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.038+0000 7f57eedb7700 1 -- 172.21.15.7:0/3926209137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f57e815def0 con 0x7f57e80b0f70 2024-01-27T02:44:38.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.042+0000 7f57e4ff9700 1 -- 172.21.15.7:0/3926209137 <== mon.0 v2:172.21.15.7:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f57dc052cf0 con 0x7f57e80b0f70 2024-01-27T02:44:38.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.042+0000 7f57e4ff9700 1 -- 172.21.15.7:0/3926209137 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v22)=0 v22) v1 ==== 123+0+0 (secure 0 0 0) 0x7f57dc006cb0 con 0x7f57e80b0f70 2024-01-27T02:44:38.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.044+0000 7f57eedb7700 1 -- 172.21.15.7:0/3926209137 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f57cc066f00 msgr2=0x7f57cc0693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:38.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.044+0000 7f57eedb7700 1 --2- 172.21.15.7:0/3926209137 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f57cc066f00 0x7f57cc0693b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f57d4006f00 tx=0x7f57d4008040 comp rx=0 tx=0).stop 2024-01-27T02:44:38.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.044+0000 7f57eedb7700 1 -- 172.21.15.7:0/3926209137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e80b0f70 msgr2=0x7f57e8154f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:38.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.044+0000 7f57eedb7700 1 --2- 172.21.15.7:0/3926209137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e80b0f70 0x7f57e8154f70 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f57dc0093f0 tx=0x7f57dc017000 comp rx=0 tx=0).stop 2024-01-27T02:44:38.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.044+0000 7f57eedb7700 1 -- 172.21.15.7:0/3926209137 shutdown_connections 2024-01-27T02:44:38.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.044+0000 7f57eedb7700 1 --2- 172.21.15.7:0/3926209137 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f57cc066f00 0x7f57cc0693b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:38.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.044+0000 7f57eedb7700 1 --2- 172.21.15.7:0/3926209137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57e80cc8d0 0x7f57e8146730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:38.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.044+0000 7f57eedb7700 1 --2- 172.21.15.7:0/3926209137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e80b0f70 0x7f57e8154f70 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:38.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.044+0000 7f57eedb7700 1 --2- 172.21.15.7:0/3926209137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57e80b1340 0x7f57e815baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:38.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.044+0000 7f57eedb7700 1 -- 172.21.15.7:0/3926209137 >> 172.21.15.7:0/3926209137 conn(0x7f57e8063580 msgr2=0x7f57e80b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:38.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.044+0000 7f57eedb7700 1 -- 172.21.15.7:0/3926209137 shutdown_connections 2024-01-27T02:44:38.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.045+0000 7f57eedb7700 1 -- 172.21.15.7:0/3926209137 wait complete. 2024-01-27T02:44:38.059 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-27T02:44:38.060 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-27T02:44:38.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.192+0000 7f580fd91700 1 Processor -- start 2024-01-27T02:44:38.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.192+0000 7f580fd91700 1 -- start start 2024-01-27T02:44:38.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.193+0000 7f580fd91700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5808159f40 0x7f58080c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:38.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.193+0000 7f580fd91700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58080c30c0 con 0x7f58080aafc0 2024-01-27T02:44:38.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.193+0000 7f580fd91700 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f580815e960 con 0x7f58080c3a90 2024-01-27T02:44:38.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.193+0000 7f580fd91700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58080c4550 con 0x7f5808159f40 2024-01-27T02:44:38.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.193+0000 7f580db2d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5808159f40 0x7f58080c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:38.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.193+0000 7f580e32e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f58080aafc0 0x7f5808063580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46576/0 (socket says 172.21.15.7:46576) 2024-01-27T02:44:38.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.193+0000 7f580db2d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5808159f40 0x7f58080c3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44398/0 (socket says 172.21.15.7:44398) 2024-01-27T02:44:38.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.193+0000 7f580e32e700 1 -- 172.21.15.7:0/1179861993 learned_addr learned my addr 172.21.15.7:0/1179861993 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-27T02:44:38.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.193+0000 7f580db2d700 1 -- 172.21.15.7:0/1179861993 >> v1:172.21.15.188:6789/0 conn(0x7f58080c3a90 legacy=0x7f58080aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:38.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.193+0000 7f580db2d700 1 -- 172.21.15.7:0/1179861993 >> v1:172.21.15.7:6789/0 conn(0x7f58080aafc0 legacy=0x7f5808063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:38.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.193+0000 7f580db2d700 1 -- 172.21.15.7:0/1179861993 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580815ec40 con 0x7f5808159f40 2024-01-27T02:44:38.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.194+0000 7f580db2d700 1 --2- 172.21.15.7:0/1179861993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5808159f40 0x7f58080c3550 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f58040098e0 tx=0x7f58040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e201df8a4a182b8 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:38.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.194+0000 7f580cb2b700 1 -- 172.21.15.7:0/1179861993 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804009e80 con 0x7f5808159f40 2024-01-27T02:44:38.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.194+0000 7f580cb2b700 1 -- 172.21.15.7:0/1179861993 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f580400b860 con 0x7f5808159f40 2024-01-27T02:44:38.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.194+0000 7f580cb2b700 1 -- 172.21.15.7:0/1179861993 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580400bc00 con 0x7f5808159f40 2024-01-27T02:44:38.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.194+0000 7f580fd91700 1 -- 172.21.15.7:0/1179861993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5808159f40 msgr2=0x7f58080c3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:38.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.194+0000 7f580fd91700 1 --2- 172.21.15.7:0/1179861993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5808159f40 0x7f58080c3550 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f58040098e0 tx=0x7f58040091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:38.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.194+0000 7f580fd91700 1 -- 172.21.15.7:0/1179861993 shutdown_connections 2024-01-27T02:44:38.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.194+0000 7f580fd91700 1 --2- 172.21.15.7:0/1179861993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5808159f40 0x7f58080c3550 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:38.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.194+0000 7f580fd91700 1 -- 172.21.15.7:0/1179861993 >> 172.21.15.7:0/1179861993 conn(0x7f58080cdc30 msgr2=0x7f58080ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:38.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.195+0000 7f580fd91700 1 -- 172.21.15.7:0/1179861993 shutdown_connections 2024-01-27T02:44:38.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.195+0000 7f580fd91700 1 -- 172.21.15.7:0/1179861993 wait complete. 2024-01-27T02:44:38.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.196+0000 7f580fd91700 1 Processor -- start 2024-01-27T02:44:38.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.196+0000 7f580fd91700 1 -- start start 2024-01-27T02:44:38.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.196+0000 7f580fd91700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58080aafc0 0x7f58080c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:38.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.196+0000 7f580fd91700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58080c3a90 0x7f58080c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:38.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.196+0000 7f580fd91700 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5808159f40 0x7f5808145040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:38.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.196+0000 7f580fd91700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58080c8050 con 0x7f58080c3a90 2024-01-27T02:44:38.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.196+0000 7f580fd91700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58080c53e0 con 0x7f58080aafc0 2024-01-27T02:44:38.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.197+0000 7f580fd91700 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58080b2e10 con 0x7f5808159f40 2024-01-27T02:44:38.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.197+0000 7f580d32c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58080c3a90 0x7f58080c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:38.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.197+0000 7f580d32c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58080c3a90 0x7f58080c7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44410/0 (socket says 172.21.15.7:44410) 2024-01-27T02:44:38.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.197+0000 7f580d32c700 1 -- 172.21.15.7:0/3935923293 learned_addr learned my addr 172.21.15.7:0/3935923293 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-27T02:44:38.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.197+0000 7f580db2d700 1 --2- 172.21.15.7:0/3935923293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58080aafc0 0x7f58080c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:38.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.197+0000 7f580e32e700 1 --2- 172.21.15.7:0/3935923293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5808159f40 0x7f5808145040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:38.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.197+0000 7f580d32c700 1 -- 172.21.15.7:0/3935923293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5808159f40 msgr2=0x7f5808145040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:38.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.197+0000 7f580d32c700 1 --2- 172.21.15.7:0/3935923293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5808159f40 0x7f5808145040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:38.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.197+0000 7f580d32c700 1 -- 172.21.15.7:0/3935923293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58080aafc0 msgr2=0x7f58080c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:38.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.197+0000 7f580d32c700 1 --2- 172.21.15.7:0/3935923293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58080aafc0 0x7f58080c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:38.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.197+0000 7f580d32c700 1 -- 172.21.15.7:0/3935923293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5804009420 con 0x7f58080c3a90 2024-01-27T02:44:38.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.197+0000 7f580d32c700 1 --2- 172.21.15.7:0/3935923293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58080c3a90 0x7f58080c7b10 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f57f800c590 tx=0x7f57f8007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:38.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.197+0000 7f57fdffb700 1 -- 172.21.15.7:0/3935923293 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8011590 con 0x7f58080c3a90 2024-01-27T02:44:38.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.197+0000 7f57fdffb700 1 -- 172.21.15.7:0/3935923293 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f57f8011710 con 0x7f58080c3a90 2024-01-27T02:44:38.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.197+0000 7f580fd91700 1 -- 172.21.15.7:0/3935923293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58080b24e0 con 0x7f58080c3a90 2024-01-27T02:44:38.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.197+0000 7f57fdffb700 1 -- 172.21.15.7:0/3935923293 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8011af0 con 0x7f58080c3a90 2024-01-27T02:44:38.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.198+0000 7f580fd91700 1 -- 172.21.15.7:0/3935923293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58080b2aa0 con 0x7f58080c3a90 2024-01-27T02:44:38.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.199+0000 7f580fd91700 1 -- 172.21.15.7:0/3935923293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58080b2fc0 con 0x7f58080c3a90 2024-01-27T02:44:38.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.199+0000 7f57fdffb700 1 -- 172.21.15.7:0/3935923293 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f57f801b3b0 con 0x7f58080c3a90 2024-01-27T02:44:38.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.199+0000 7f57fdffb700 1 --2- 172.21.15.7:0/3935923293 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f57ec066f30 0x7f57ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-27T02:44:38.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.199+0000 7f57fdffb700 1 -- 172.21.15.7:0/3935923293 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f57f8083540 con 0x7f58080c3a90 2024-01-27T02:44:38.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.200+0000 7f580db2d700 1 --2- 172.21.15.7:0/3935923293 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f57ec066f30 0x7f57ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-27T02:44:38.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.200+0000 7f580db2d700 1 --2- 172.21.15.7:0/3935923293 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f57ec066f30 0x7f57ec0693e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f58040093f0 tx=0x7f58040091a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-27T02:44:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.204+0000 7f57fdffb700 1 -- 172.21.15.7:0/3935923293 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f58080b2fc0 con 0x7f58080c3a90 2024-01-27T02:44:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.451+0000 7f580fd91700 1 -- 172.21.15.7:0/3935923293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58080b2f80 con 0x7f58080c3a90 2024-01-27T02:44:38.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.452+0000 7f57fdffb700 1 -- 172.21.15.7:0/3935923293 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+3523 (secure 0 0 0) 0x7f57f8017020 con 0x7f58080c3a90 2024-01-27T02:44:38.522 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-27T02:44:38.522 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":96,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6704,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/1199331492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1199331492},{"type":"v1","addr":"172.21.15.7:6834","nonce":1199331492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":6707,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/721335086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":721335086},{"type":"v1","addr":"172.21.15.7:6835","nonce":721335086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":6911,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/2500745005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":2500745005},{"type":"v1","addr":"172.21.15.188:6837","nonce":2500745005}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/450555206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":450555206},{"type":"v1","addr":"172.21.15.188:6836","nonce":450555206}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[]} 2024-01-27T02:44:38.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.454+0000 7f580fd91700 1 -- 172.21.15.7:0/3935923293 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f57ec066f30 msgr2=0x7f57ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:38.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.454+0000 7f580fd91700 1 --2- 172.21.15.7:0/3935923293 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f57ec066f30 0x7f57ec0693e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f58040093f0 tx=0x7f58040091a0 comp rx=0 tx=0).stop 2024-01-27T02:44:38.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.454+0000 7f580fd91700 1 -- 172.21.15.7:0/3935923293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58080c3a90 msgr2=0x7f58080c7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-27T02:44:38.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.454+0000 7f580fd91700 1 --2- 172.21.15.7:0/3935923293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58080c3a90 0x7f58080c7b10 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f57f800c590 tx=0x7f57f8007f50 comp rx=0 tx=0).stop 2024-01-27T02:44:38.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.454+0000 7f580fd91700 1 -- 172.21.15.7:0/3935923293 shutdown_connections 2024-01-27T02:44:38.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.454+0000 7f580fd91700 1 --2- 172.21.15.7:0/3935923293 >> [v2:172.21.15.188:6832/38712,v1:172.21.15.188:6833/38712] conn(0x7f57ec066f30 0x7f57ec0693e0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:38.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.454+0000 7f580fd91700 1 --2- 172.21.15.7:0/3935923293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5808159f40 0x7f5808145040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:38.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.454+0000 7f580fd91700 1 --2- 172.21.15.7:0/3935923293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58080aafc0 0x7f58080c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:38.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.454+0000 7f580fd91700 1 --2- 172.21.15.7:0/3935923293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58080c3a90 0x7f58080c7b10 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-27T02:44:38.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.454+0000 7f580fd91700 1 -- 172.21.15.7:0/3935923293 >> 172.21.15.7:0/3935923293 conn(0x7f58080cdc30 msgr2=0x7f5808159340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-27T02:44:38.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.454+0000 7f580fd91700 1 -- 172.21.15.7:0/3935923293 shutdown_connections 2024-01-27T02:44:38.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-27T02:44:38.454+0000 7f580fd91700 1 -- 172.21.15.7:0/3935923293 wait complete. 2024-01-27T02:44:38.527 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 96 2024-01-27T02:44:38.527 DEBUG:tasks.cephfs.filesystem:already dead... 2024-01-27T02:44:38.528 INFO:teuthology.misc:Shutting down mds daemons... 2024-01-27T02:44:38.528 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-27T02:44:38.528 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:44:38.538 INFO:tasks.ceph.mds.a:Stopped 2024-01-27T02:44:38.538 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-27T02:44:38.539 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:44:38.593 INFO:tasks.ceph.mds.c:Stopped 2024-01-27T02:44:38.594 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-27T02:44:38.594 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:44:38.606 INFO:tasks.ceph.mds.b:Stopped 2024-01-27T02:44:38.606 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-27T02:44:38.607 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:44:38.657 INFO:tasks.ceph.mds.d:Stopped 2024-01-27T02:44:38.657 INFO:teuthology.misc:Shutting down osd daemons... 2024-01-27T02:44:38.657 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-01-27T02:44:38.657 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:44:38.682 INFO:tasks.ceph.osd.0:Stopped 2024-01-27T02:44:38.682 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-01-27T02:44:38.682 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:44:38.752 INFO:tasks.ceph.osd.1:Stopped 2024-01-27T02:44:38.752 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-01-27T02:44:38.752 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:44:38.825 INFO:tasks.ceph.osd.2:Stopped 2024-01-27T02:44:38.825 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-01-27T02:44:38.825 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:44:38.894 INFO:tasks.ceph.osd.3:Stopped 2024-01-27T02:44:38.894 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-01-27T02:44:38.894 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:44:38.917 INFO:tasks.ceph.osd.4:Stopped 2024-01-27T02:44:38.917 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-01-27T02:44:38.918 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:44:38.980 INFO:tasks.ceph.osd.5:Stopped 2024-01-27T02:44:38.980 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-01-27T02:44:38.981 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:44:39.054 INFO:tasks.ceph.osd.6:Stopped 2024-01-27T02:44:39.054 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-01-27T02:44:39.055 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:44:39.118 INFO:tasks.ceph.osd.7:Stopped 2024-01-27T02:44:39.118 INFO:teuthology.misc:Shutting down mgr daemons... 2024-01-27T02:44:39.118 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-01-27T02:44:39.118 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:44:39.147 INFO:tasks.ceph.mgr.y:Stopped 2024-01-27T02:44:39.147 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-01-27T02:44:39.147 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:44:39.185 INFO:tasks.ceph.mgr.x:Stopped 2024-01-27T02:44:39.185 INFO:teuthology.misc:Shutting down mon daemons... 2024-01-27T02:44:39.185 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-01-27T02:44:39.185 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:44:39.202 INFO:tasks.ceph.mon.a:Stopped 2024-01-27T02:44:39.202 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-01-27T02:44:39.202 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:44:39.237 INFO:tasks.ceph.mon.b:Stopped 2024-01-27T02:44:39.237 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-01-27T02:44:39.237 INFO:teuthology.orchestra.run:waiting for 300 2024-01-27T02:44:39.290 INFO:tasks.ceph.mon.c:Stopped 2024-01-27T02:44:39.291 INFO:tasks.ceph:Checking cluster log for badness... 2024-01-27T02:44:39.291 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2024-01-27T02:44:39.336 INFO:teuthology.orchestra.run.smithi007.stdout:1706322714.5416682 osd.7 (osd.7) 3 : cluster [WRN] OSD bench result of 221318.978486 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.7. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]. 2024-01-27T02:44:39.337 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-01-27T02:44:39.337 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2024-01-27T02:44:39.381 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2024-01-27T02:44:39.476 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2024-01-27T02:44:39.519 INFO:teuthology.orchestra.run.smithi007.stdout:1706322714.5416682 osd.7 (osd.7) 3 : cluster [WRN] OSD bench result of 221318.978486 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.7. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]. 2024-01-27T02:44:39.519 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:44:39.520 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-01-27T02:44:39.745 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:44:39.745 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-01-27T02:44:39.792 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:44:39.795 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:44:39.895 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:44:39.895 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-01-27T02:44:40.016 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:44:40.016 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-01-27T02:44:40.114 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:44:40.114 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-01-27T02:44:40.287 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:44:40.287 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-01-27T02:44:40.404 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:44:40.404 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-01-27T02:44:40.482 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:44:40.482 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-01-27T02:44:40.553 INFO:tasks.ceph:Archiving mon data... 2024-01-27T02:44:40.555 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533593/data/mon.a.tgz 2024-01-27T02:44:40.556 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-01-27T02:44:40.578 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.NOrjR2vPGQ 2024-01-27T02:44:40.578 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.NOrjR2vPGQ 2024-01-27T02:44:40.780 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.NOrjR2vPGQ 2024-01-27T02:44:40.881 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.NOrjR2vPGQ is 408KB 2024-01-27T02:44:40.966 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.NOrjR2vPGQ 2024-01-27T02:44:40.987 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533593/data/mon.b.tgz 2024-01-27T02:44:40.987 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2024-01-27T02:44:41.007 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.k9I53vjzZr 2024-01-27T02:44:41.008 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.k9I53vjzZr 2024-01-27T02:44:41.187 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.k9I53vjzZr 2024-01-27T02:44:41.274 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.k9I53vjzZr is 420KB 2024-01-27T02:44:41.370 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.k9I53vjzZr 2024-01-27T02:44:41.386 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533593/data/mon.c.tgz 2024-01-27T02:44:41.386 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2024-01-27T02:44:41.443 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.RLLgBKf244 2024-01-27T02:44:41.443 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.RLLgBKf244 2024-01-27T02:44:41.585 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.RLLgBKf244 2024-01-27T02:44:41.671 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.RLLgBKf244 is 420KB 2024-01-27T02:44:41.769 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.RLLgBKf244 2024-01-27T02:44:41.785 INFO:tasks.ceph:Cleaning ceph cluster... 2024-01-27T02:44:41.786 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-01-27T02:44:41.788 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-01-27T02:44:41.852 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-01-27T02:44:41.852 DEBUG:teuthology.orchestra.run.smithi007:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-01-27T02:44:41.864 DEBUG:teuthology.orchestra.run.smithi188:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-01-27T02:44:41.897 INFO:teuthology.orchestra.run.smithi007.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-01-27T02:44:41.919 INFO:teuthology.orchestra.run.smithi188.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-01-27T02:44:41.925 INFO:tasks.ceph:Archiving crash dumps... 2024-01-27T02:44:41.927 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533593/remote/smithi007/crash 2024-01-27T02:44:41.927 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-27T02:44:41.980 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533593/remote/smithi188/crash 2024-01-27T02:44:41.981 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-27T02:44:42.007 INFO:tasks.ceph:Shutting down logrotate 2024-01-27T02:44:42.007 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:44:42.025 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-27T02:44:42.108 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-01-27T02:44:42.143 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-01-27T02:44:42.169 INFO:tasks.ceph:Compressing logs... 2024-01-27T02:44:42.170 DEBUG:teuthology.orchestra.run.smithi007:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-27T02:44:42.186 DEBUG:teuthology.orchestra.run.smithi188:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-27T02:44:42.218 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57824.log 2024-01-27T02:44:42.219 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60399.log 2024-01-27T02:44:42.219 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62557.log 2024-01-27T02:44:42.220 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57824.log: /var/log/ceph/ceph-client.admin.60399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57387.log 2024-01-27T02:44:42.221 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60399.log.gz 2024-01-27T02:44:42.221 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74332.log 2024-01-27T02:44:42.221 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.57824.log.gz 2024-01-27T02:44:42.221 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62557.log.gz 2024-01-27T02:44:42.221 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47733.log 2024-01-27T02:44:42.221 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57387.log.gz 2024-01-27T02:44:42.221 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65034.log 2024-01-27T02:44:42.226 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68320.log 2024-01-27T02:44:42.226 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74332.log.gz 2024-01-27T02:44:42.226 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65034.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47733.log.gz 2024-01-27T02:44:42.226 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56507.log 2024-01-27T02:44:42.226 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.65034.log.gz 2024-01-27T02:44:42.226 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57732.log 2024-01-27T02:44:42.226 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68320.log.gz 2024-01-27T02:44:42.226 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39873.log 2024-01-27T02:44:42.226 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57732.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56507.log.gz 2024-01-27T02:44:42.226 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70471.log 2024-01-27T02:44:42.226 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57732.log.gz 2024-01-27T02:44:42.226 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72684.log 2024-01-27T02:44:42.227 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39873.log.gz 2024-01-27T02:44:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.70471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41070.log 2024-01-27T02:44:42.227 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70471.log.gz 2024-01-27T02:44:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53369.log 2024-01-27T02:44:42.227 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72684.log.gz 2024-01-27T02:44:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54997.log 2024-01-27T02:44:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41070.log: /var/log/ceph/ceph-client.admin.53369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46127.log 2024-01-27T02:44:42.227 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41070.log.gz 2024-01-27T02:44:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54997.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53369.log.gz 2024-01-27T02:44:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66260.log 2024-01-27T02:44:42.227 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54997.log.gz 2024-01-27T02:44:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56645.log 2024-01-27T02:44:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66260.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46127.log.gz 2024-01-27T02:44:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59773.log 2024-01-27T02:44:42.228 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66260.log.gz 2024-01-27T02:44:42.228 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63329.log 2024-01-27T02:44:42.228 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56645.log.gz 2024-01-27T02:44:42.228 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69192.log 2024-01-27T02:44:42.235 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63329.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59773.log.gz 2024-01-27T02:44:42.235 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68504.log 2024-01-27T02:44:42.236 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63329.log.gz 2024-01-27T02:44:42.236 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58997.log 2024-01-27T02:44:42.236 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69192.log.gz 2024-01-27T02:44:42.236 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73180.log 2024-01-27T02:44:42.236 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68504.log.gz 2024-01-27T02:44:42.236 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49516.log 2024-01-27T02:44:42.236 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58997.log.gz 2024-01-27T02:44:42.236 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41061.log 2024-01-27T02:44:42.236 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73180.log.gz 2024-01-27T02:44:42.236 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42199.log 2024-01-27T02:44:42.236 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38994.log 2024-01-27T02:44:42.236 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41061.log.gz 2024-01-27T02:44:42.236 INFO:teuthology.orchestra.run.smithi007.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.49516.log.gz 2024-01-27T02:44:42.236 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42199.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-01-27T02:44:42.237 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42199.log.gz 2024-01-27T02:44:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:gzip /var/log/ceph/ceph-client.admin.38994.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.59386.log 2024-01-27T02:44:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49424.log 2024-01-27T02:44:42.237 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38994.log.gz 2024-01-27T02:44:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61115.log 2024-01-27T02:44:42.237 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59386.log.gz 2024-01-27T02:44:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74056.log 2024-01-27T02:44:42.237 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49424.log.gz 2024-01-27T02:44:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40819.log 2024-01-27T02:44:42.237 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61115.log.gz 2024-01-27T02:44:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53848.log 2024-01-27T02:44:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74056.log: /var/log/ceph/ceph-client.admin.40819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45291.log 2024-01-27T02:44:42.237 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74056.log.gz 2024-01-27T02:44:42.237 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40819.log.gz 2024-01-27T02:44:42.238 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48396.log 2024-01-27T02:44:42.238 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53848.log.gz 2024-01-27T02:44:42.238 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64154.log 2024-01-27T02:44:42.238 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45291.log.gz 2024-01-27T02:44:42.238 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41391.log 2024-01-27T02:44:42.238 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64154.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48396.log.gz 2024-01-27T02:44:42.238 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60677.log 2024-01-27T02:44:42.238 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64154.log.gz 2024-01-27T02:44:42.238 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42975.log 2024-01-27T02:44:42.238 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41391.log.gz 2024-01-27T02:44:42.238 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40589.log 2024-01-27T02:44:42.239 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.42975.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60677.log.gz 2024-01-27T02:44:42.239 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42975.log.gz 2024-01-27T02:44:42.239 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50932.log 2024-01-27T02:44:42.239 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49819.log 2024-01-27T02:44:42.239 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40589.log.gz 2024-01-27T02:44:42.239 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67541.log 2024-01-27T02:44:42.239 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50932.log.gz 2024-01-27T02:44:42.239 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.49819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56001.log 2024-01-27T02:44:42.239 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49819.log.gz 2024-01-27T02:44:42.239 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67119.log 2024-01-27T02:44:42.239 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66979.log 2024-01-27T02:44:42.257 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-01-27T02:44:42.258 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43708.log 2024-01-27T02:44:42.258 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41320.log 2024-01-27T02:44:42.258 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.43708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42229.log 2024-01-27T02:44:42.258 INFO:teuthology.orchestra.run.smithi188.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43708.log.gz 2024-01-27T02:44:42.258 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.41320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45162.log 2024-01-27T02:44:42.258 INFO:teuthology.orchestra.run.smithi188.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41320.log.gz 2024-01-27T02:44:42.258 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.42229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44165.log 2024-01-27T02:44:42.258 INFO:teuthology.orchestra.run.smithi188.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42229.log.gz 2024-01-27T02:44:42.258 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.45162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39019.log 2024-01-27T02:44:42.258 INFO:teuthology.orchestra.run.smithi188.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45162.log.gz 2024-01-27T02:44:42.258 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.44165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43260.log 2024-01-27T02:44:42.258 INFO:teuthology.orchestra.run.smithi188.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44165.log.gz 2024-01-27T02:44:42.258 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.39019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43704.log 2024-01-27T02:44:42.258 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.43260.log: gzip -5 88.6%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.39019.log.gz/var/log/ceph/ceph-client.3.40840.log 2024-01-27T02:44:42.259 INFO:teuthology.orchestra.run.smithi188.stderr: 2024-01-27T02:44:42.259 INFO:teuthology.orchestra.run.smithi188.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43260.log.gz 2024-01-27T02:44:42.259 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.43704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44614.log 2024-01-27T02:44:42.259 INFO:teuthology.orchestra.run.smithi188.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43704.log.gz 2024-01-27T02:44:42.259 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.3.40840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42754.log 2024-01-27T02:44:42.259 INFO:teuthology.orchestra.run.smithi188.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.40840.log.gz 2024-01-27T02:44:42.259 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.44614.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-01-27T02:44:42.259 INFO:teuthology.orchestra.run.smithi188.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44614.log.gz 2024-01-27T02:44:42.259 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.42754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42225.log 2024-01-27T02:44:42.259 INFO:teuthology.orchestra.run.smithi188.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42754.log.gz 2024-01-27T02:44:42.259 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38923.log 2024-01-27T02:44:42.259 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.42225.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-01-27T02:44:42.259 INFO:teuthology.orchestra.run.smithi188.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42225.log.gz 2024-01-27T02:44:42.259 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.38923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.40836.log 2024-01-27T02:44:42.259 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38923.log.gz 2024-01-27T02:44:42.260 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38971.log 2024-01-27T02:44:42.260 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41760.log 2024-01-27T02:44:42.260 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.3.40836.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.40836.log.gz 2024-01-27T02:44:42.260 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.38971.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-01-27T02:44:42.260 INFO:teuthology.orchestra.run.smithi188.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38971.log.gz 2024-01-27T02:44:42.260 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.41760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39072.log 2024-01-27T02:44:42.260 INFO:teuthology.orchestra.run.smithi188.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41760.log.gz 2024-01-27T02:44:42.260 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39257.log 2024-01-27T02:44:42.260 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-client.admin.39072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39211.log 2024-01-27T02:44:42.260 INFO:teuthology.orchestra.run.smithi188.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39072.log.gz 2024-01-27T02:44:42.260 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.39257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40401.log 2024-01-27T02:44:42.260 INFO:teuthology.orchestra.run.smithi188.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39257.log.gz 2024-01-27T02:44:42.260 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.39211.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-01-27T02:44:42.260 INFO:teuthology.orchestra.run.smithi188.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39211.log.gz 2024-01-27T02:44:42.261 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.40401.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40401.log.gz 2024-01-27T02:44:42.261 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph.audit.log: 91.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-01-27T02:44:42.261 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-01-27T02:44:42.261 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-01-27T02:44:42.261 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39163.log 2024-01-27T02:44:42.261 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.39163.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39163.log.gz 2024-01-27T02:44:42.261 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40393.log 2024-01-27T02:44:42.261 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.40393.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40393.log.gz 2024-01-27T02:44:42.261 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-01-27T02:44:42.261 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mon.b.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-01-27T02:44:42.261 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42750.log 2024-01-27T02:44:42.261 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.42750.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42750.log.gz 2024-01-27T02:44:42.261 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39117.log 2024-01-27T02:44:42.261 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.39117.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39117.log.gz 2024-01-27T02:44:42.261 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-01-27T02:44:42.262 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45158.log 2024-01-27T02:44:42.262 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.45158.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45158.log.gz 2024-01-27T02:44:42.262 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44169.log 2024-01-27T02:44:42.262 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.44169.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44169.log.gz 2024-01-27T02:44:42.262 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43264.log 2024-01-27T02:44:42.263 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.43264.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43264.log.gz 2024-01-27T02:44:42.263 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45658.log 2024-01-27T02:44:42.264 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.45658.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45658.log.gz 2024-01-27T02:44:42.264 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41764.log 2024-01-27T02:44:42.264 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.41764.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41764.log.gz 2024-01-27T02:44:42.264 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-01-27T02:44:42.277 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-01-27T02:44:42.297 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44610.log 2024-01-27T02:44:42.298 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56001.log.gz 2024-01-27T02:44:42.298 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67119.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.67541.log.gz 2024-01-27T02:44:42.298 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40451.log 2024-01-27T02:44:42.298 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67119.log.gz 2024-01-27T02:44:42.298 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72270.log 2024-01-27T02:44:42.298 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66979.log.gz 2024-01-27T02:44:42.298 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46403.log 2024-01-27T02:44:42.298 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46311.log 2024-01-27T02:44:42.298 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40451.log.gz 2024-01-27T02:44:42.299 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72270.log.gz 2024-01-27T02:44:42.299 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49470.log 2024-01-27T02:44:42.299 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46403.log.gz 2024-01-27T02:44:42.299 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.43472.log 2024-01-27T02:44:42.299 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46311.log: /var/log/ceph/ceph-client.admin.49470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40497.log 2024-01-27T02:44:42.299 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46311.log.gz 2024-01-27T02:44:42.299 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49470.log.gz 2024-01-27T02:44:42.299 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.2.43472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66122.log 2024-01-27T02:44:42.299 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.43472.log.gz 2024-01-27T02:44:42.299 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.43476.log 2024-01-27T02:44:42.299 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40497.log.gz 2024-01-27T02:44:42.299 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63811.log 2024-01-27T02:44:42.299 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66122.log.gz 2024-01-27T02:44:42.299 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.2.43476.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.43476.log.gz 2024-01-27T02:44:42.300 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75457.log 2024-01-27T02:44:42.300 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70567.log 2024-01-27T02:44:42.300 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75457.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63811.log.gz 2024-01-27T02:44:42.300 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52498.log 2024-01-27T02:44:42.300 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.75457.log.gz 2024-01-27T02:44:42.300 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56185.log 2024-01-27T02:44:42.300 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52498.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.70567.log.gz 2024-01-27T02:44:42.300 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75135.log 2024-01-27T02:44:42.300 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52498.log.gz 2024-01-27T02:44:42.300 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72362.log 2024-01-27T02:44:42.300 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75135.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56185.log.gz 2024-01-27T02:44:42.300 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56139.log 2024-01-27T02:44:42.300 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75135.log.gz 2024-01-27T02:44:42.300 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45109.log 2024-01-27T02:44:42.300 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56139.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72362.log.gz 2024-01-27T02:44:42.301 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40359.log 2024-01-27T02:44:42.301 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56139.log.gz 2024-01-27T02:44:42.301 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44220.log 2024-01-27T02:44:42.301 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45109.log.gz 2024-01-27T02:44:42.301 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61663.log 2024-01-27T02:44:42.301 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44220.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40359.log.gz 2024-01-27T02:44:42.301 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74378.log 2024-01-27T02:44:42.301 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44220.log.gz 2024-01-27T02:44:42.301 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59432.log 2024-01-27T02:44:42.301 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61663.log.gz 2024-01-27T02:44:42.301 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41397.log 2024-01-27T02:44:42.301 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59432.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74378.log.gz 2024-01-27T02:44:42.301 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52406.log 2024-01-27T02:44:42.301 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59432.log.gz 2024-01-27T02:44:42.302 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44971.log 2024-01-27T02:44:42.302 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52406.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41397.log.gz 2024-01-27T02:44:42.302 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62991.log 2024-01-27T02:44:42.302 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52406.log.gz 2024-01-27T02:44:42.302 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57894.log 2024-01-27T02:44:42.302 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62991.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44971.log.gz 2024-01-27T02:44:42.302 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69366.log 2024-01-27T02:44:42.302 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62991.log.gz 2024-01-27T02:44:42.302 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61709.log 2024-01-27T02:44:42.302 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69366.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57894.log.gz 2024-01-27T02:44:42.302 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49092.log 2024-01-27T02:44:42.302 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.69366.log.gz 2024-01-27T02:44:42.302 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75549.log 2024-01-27T02:44:42.302 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61709.log.gz 2024-01-27T02:44:42.303 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51978.log 2024-01-27T02:44:42.303 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75549.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49092.log.gz 2024-01-27T02:44:42.303 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62465.log 2024-01-27T02:44:42.303 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75549.log.gz 2024-01-27T02:44:42.303 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58127.log 2024-01-27T02:44:42.303 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62465.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51978.log.gz 2024-01-27T02:44:42.303 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45653.log 2024-01-27T02:44:42.303 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62465.log.gz 2024-01-27T02:44:42.303 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.58127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64599.log 2024-01-27T02:44:42.303 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58127.log.gz 2024-01-27T02:44:42.303 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61518.log 2024-01-27T02:44:42.303 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.64599.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45653.log.gz 2024-01-27T02:44:42.303 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67944.log 2024-01-27T02:44:42.303 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64599.log.gz 2024-01-27T02:44:42.303 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59478.log 2024-01-27T02:44:42.304 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61518.log.gz 2024-01-27T02:44:42.304 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46219.log 2024-01-27T02:44:42.304 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67944.log.gz 2024-01-27T02:44:42.304 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69600.log 2024-01-27T02:44:42.304 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46219.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59478.log.gz 2024-01-27T02:44:42.304 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75181.log 2024-01-27T02:44:42.304 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46219.log.gz 2024-01-27T02:44:42.304 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66029.log 2024-01-27T02:44:42.304 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75181.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69600.log.gz 2024-01-27T02:44:42.304 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68550.log 2024-01-27T02:44:42.304 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0%/var/log/ceph/ceph-client.admin.66029.log: -- replaced with /var/log/ceph/ceph-client.admin.75181.log.gz 2024-01-27T02:44:42.304 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75687.log 2024-01-27T02:44:42.304 INFO:teuthology.orchestra.run.smithi007.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.66029.log.gz 2024-01-27T02:44:42.305 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39920.log 2024-01-27T02:44:42.305 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68550.log.gz 2024-01-27T02:44:42.305 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50500.log 2024-01-27T02:44:42.305 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.75687.log.gz 2024-01-27T02:44:42.305 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62649.log 2024-01-27T02:44:42.305 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39920.log.gz 2024-01-27T02:44:42.305 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60769.log 2024-01-27T02:44:42.305 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62649.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50500.log.gz 2024-01-27T02:44:42.305 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73088.log 2024-01-27T02:44:42.305 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62649.log.gz 2024-01-27T02:44:42.305 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59043.log 2024-01-27T02:44:42.305 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73088.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60769.log.gz 2024-01-27T02:44:42.305 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75595.log 2024-01-27T02:44:42.305 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73088.log.gz 2024-01-27T02:44:42.305 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53461.log 2024-01-27T02:44:42.306 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59043.log.gz 2024-01-27T02:44:42.306 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75878.log 2024-01-27T02:44:42.306 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53461.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.75595.log.gz 2024-01-27T02:44:42.306 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39966.log 2024-01-27T02:44:42.306 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53461.log.gz 2024-01-27T02:44:42.306 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73423.log 2024-01-27T02:44:42.306 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75878.log.gz 2024-01-27T02:44:42.306 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59979.log 2024-01-27T02:44:42.306 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73423.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39966.log.gz 2024-01-27T02:44:42.306 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58181.log 2024-01-27T02:44:42.306 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.73423.log.gz 2024-01-27T02:44:42.306 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66644.log 2024-01-27T02:44:42.306 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59979.log.gz 2024-01-27T02:44:42.306 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58181.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.37252.log 2024-01-27T02:44:42.307 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66644.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58181.log.gz 2024-01-27T02:44:42.307 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57145.log 2024-01-27T02:44:42.307 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66644.log.gz 2024-01-27T02:44:42.307 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.tmp-client.admin.37252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75089.log 2024-01-27T02:44:42.307 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.37252.log.gz 2024-01-27T02:44:42.307 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48055.log 2024-01-27T02:44:42.307 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75089.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57145.log.gz 2024-01-27T02:44:42.307 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53691.log 2024-01-27T02:44:42.307 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75089.log.gz 2024-01-27T02:44:42.307 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71460.log 2024-01-27T02:44:42.307 INFO:teuthology.orchestra.run.smithi007.stderr: 89.5%/var/log/ceph/ceph-client.admin.53691.log: -- replaced with /var/log/ceph/ceph-client.admin.48055.log.gz 2024-01-27T02:44:42.307 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60445.log 2024-01-27T02:44:42.307 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71460.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53691.log.gz 2024-01-27T02:44:42.307 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68412.log 2024-01-27T02:44:42.308 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71460.log.gz 2024-01-27T02:44:42.308 INFO:teuthology.orchestra.run.smithi007.stderr:gzip /var/log/ceph/ceph-client.admin.60445.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.68734.log 2024-01-27T02:44:42.308 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68412.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60445.log.gz 2024-01-27T02:44:42.308 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41439.log 2024-01-27T02:44:42.308 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68734.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.68412.log.gz 2024-01-27T02:44:42.308 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-01-27T02:44:42.308 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68734.log.gz 2024-01-27T02:44:42.308 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53507.log 2024-01-27T02:44:42.308 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41439.log.gz 2024-01-27T02:44:42.308 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65328.log 2024-01-27T02:44:42.308 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51457.log 2024-01-27T02:44:42.308 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53507.log.gz 2024-01-27T02:44:42.308 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41739.log 2024-01-27T02:44:42.308 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65328.log.gz 2024-01-27T02:44:42.309 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55340.log 2024-01-27T02:44:42.309 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51457.log.gz 2024-01-27T02:44:42.309 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59635.log 2024-01-27T02:44:42.309 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41739.log.gz 2024-01-27T02:44:42.309 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41877.log 2024-01-27T02:44:42.309 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55340.log: /var/log/ceph/ceph-client.admin.59635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53553.log 2024-01-27T02:44:42.309 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59635.log.gz 2024-01-27T02:44:42.309 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54581.log 2024-01-27T02:44:42.309 INFO:teuthology.orchestra.run.smithi007.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.55340.log.gz 2024-01-27T02:44:42.309 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41877.log.gz 2024-01-27T02:44:42.309 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46752.log 2024-01-27T02:44:42.309 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51116.log 2024-01-27T02:44:42.309 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53553.log.gz 2024-01-27T02:44:42.309 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.46752.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.54581.log.gz 2024-01-27T02:44:42.310 INFO:teuthology.orchestra.run.smithi007.stderr:-5 --verbose -- 0.0%/var/log/ceph/ceph-client.admin.66306.log 2024-01-27T02:44:42.310 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.0.46752.log.gz 2024-01-27T02:44:42.310 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48101.log 2024-01-27T02:44:42.310 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51116.log.gz 2024-01-27T02:44:42.310 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44266.log 2024-01-27T02:44:42.310 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66306.log.gz 2024-01-27T02:44:42.310 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62511.log 2024-01-27T02:44:42.310 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48101.log: /var/log/ceph/ceph-client.admin.44266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71073.log 2024-01-27T02:44:42.310 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48101.log.gz 2024-01-27T02:44:42.310 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62511.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44266.log.gz 2024-01-27T02:44:42.310 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58519.log 2024-01-27T02:44:42.310 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62511.log.gz 2024-01-27T02:44:42.310 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60063.log 2024-01-27T02:44:42.310 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71073.log.gz 2024-01-27T02:44:42.310 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64246.log 2024-01-27T02:44:42.311 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60063.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58519.log.gz 2024-01-27T02:44:42.311 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3%/var/log/ceph/ceph-client.admin.64246.log: -- replaced with /var/log/ceph/ceph-client.admin.60063.log.gz 2024-01-27T02:44:42.311 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64246.log.gz 2024-01-27T02:44:42.311 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40154.log 2024-01-27T02:44:42.311 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72800.log 2024-01-27T02:44:42.311 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73601.log 2024-01-27T02:44:42.311 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3%/var/log/ceph/ceph-client.admin.72800.log: -- replaced with /var/log/ceph/ceph-client.admin.40154.log.gz 2024-01-27T02:44:42.311 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73033.log 2024-01-27T02:44:42.311 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72800.log.gz 2024-01-27T02:44:42.311 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69029.log 2024-01-27T02:44:42.311 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73601.log: /var/log/ceph/ceph-client.0.73033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64595.log 2024-01-27T02:44:42.311 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.73033.log.gz 2024-01-27T02:44:42.311 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73601.log.gz 2024-01-27T02:44:42.312 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65622.log 2024-01-27T02:44:42.312 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.64595.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69029.log.gz 2024-01-27T02:44:42.312 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64595.log.gz 2024-01-27T02:44:42.312 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65622.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65622.log.gz 2024-01-27T02:44:42.312 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73693.log 2024-01-27T02:44:42.312 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40012.log 2024-01-27T02:44:42.312 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55089.log 2024-01-27T02:44:42.312 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40012.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73693.log.gz 2024-01-27T02:44:42.312 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45885.log 2024-01-27T02:44:42.312 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40012.log.gz 2024-01-27T02:44:42.312 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51273.log 2024-01-27T02:44:42.312 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68891.log 2024-01-27T02:44:42.312 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45885.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55089.log.gz 2024-01-27T02:44:42.313 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51273.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45885.log.gz 2024-01-27T02:44:42.313 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44741.log 2024-01-27T02:44:42.313 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51273.log.gz 2024-01-27T02:44:42.313 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65781.log 2024-01-27T02:44:42.313 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68891.log: /var/log/ceph/ceph-client.admin.44741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69794.log 2024-01-27T02:44:42.313 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68891.log.gz 2024-01-27T02:44:42.313 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65781.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44741.log.gz 2024-01-27T02:44:42.313 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40200.log 2024-01-27T02:44:42.313 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65781.log.gz 2024-01-27T02:44:42.313 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62760.log 2024-01-27T02:44:42.313 INFO:teuthology.orchestra.run.smithi007.stderr:gzip 87.5%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.69794.log.gz-- 2024-01-27T02:44:42.313 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44695.log 2024-01-27T02:44:42.313 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40200.log: /var/log/ceph/ceph-client.admin.62760.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40200.log.gz 2024-01-27T02:44:42.313 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63523.log 2024-01-27T02:44:42.314 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62760.log.gz 2024-01-27T02:44:42.314 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63625.log 2024-01-27T02:44:42.314 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44695.log: /var/log/ceph/ceph-client.admin.63523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71165.log 2024-01-27T02:44:42.314 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44695.log.gz 2024-01-27T02:44:42.314 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63625.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.63523.log.gz-5 2024-01-27T02:44:42.314 INFO:teuthology.orchestra.run.smithi007.stderr:--verbose -- /var/log/ceph/ceph-client.admin.53801.log 2024-01-27T02:44:42.314 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63625.log.gz 2024-01-27T02:44:42.314 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48536.log 2024-01-27T02:44:42.314 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71165.log.gz 2024-01-27T02:44:42.314 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55743.log 2024-01-27T02:44:42.314 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53801.log: /var/log/ceph/ceph-client.admin.48536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44312.log 2024-01-27T02:44:42.314 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53801.log.gz 2024-01-27T02:44:42.314 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55743.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48536.log.gz 2024-01-27T02:44:42.314 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71413.log 2024-01-27T02:44:42.315 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55743.log.gz 2024-01-27T02:44:42.315 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62853.log 2024-01-27T02:44:42.315 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44312.log.gz 2024-01-27T02:44:42.315 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62696.log 2024-01-27T02:44:42.315 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62853.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71413.log.gz 2024-01-27T02:44:42.315 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70026.log 2024-01-27T02:44:42.315 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62853.log.gz 2024-01-27T02:44:42.315 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48992.log 2024-01-27T02:44:42.315 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70026.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62696.log.gz 2024-01-27T02:44:42.315 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65282.log 2024-01-27T02:44:42.315 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70026.log.gz/var/log/ceph/ceph-client.admin.48992.log: 2024-01-27T02:44:42.315 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59589.log 2024-01-27T02:44:42.315 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48992.log.gz 2024-01-27T02:44:42.315 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65282.log: /var/log/ceph/ceph-client.admin.59589.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65282.log.gz 2024-01-27T02:44:42.316 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59589.log.gz 2024-01-27T02:44:42.316 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74240.log 2024-01-27T02:44:42.316 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59294.log 2024-01-27T02:44:42.316 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72546.log 2024-01-27T02:44:42.316 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59294.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74240.log.gz 2024-01-27T02:44:42.316 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57007.log 2024-01-27T02:44:42.316 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59294.log.gz 2024-01-27T02:44:42.316 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56093.log 2024-01-27T02:44:42.316 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57481.log 2024-01-27T02:44:42.316 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57007.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72546.log.gz 2024-01-27T02:44:42.316 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56093.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57007.log.gz 2024-01-27T02:44:42.316 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57481.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56093.log.gz 2024-01-27T02:44:42.316 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57481.log.gz 2024-01-27T02:44:42.316 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52893.log 2024-01-27T02:44:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49286.log 2024-01-27T02:44:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.52893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54767.log 2024-01-27T02:44:42.317 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52893.log.gz 2024-01-27T02:44:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49286.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-01-27T02:44:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54767.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49286.log.gz 2024-01-27T02:44:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60723.log 2024-01-27T02:44:42.317 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54767.log.gz 2024-01-27T02:44:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63949.log 2024-01-27T02:44:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.60723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41693.log 2024-01-27T02:44:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63949.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60723.log.gz 2024-01-27T02:44:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39827.log 2024-01-27T02:44:42.317 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63949.log.gz 2024-01-27T02:44:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66214.log 2024-01-27T02:44:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39827.log: 89.1%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.41693.log.gz--verbose 2024-01-27T02:44:42.318 INFO:teuthology.orchestra.run.smithi007.stderr:-- /var/log/ceph/ceph-client.admin.43711.log 2024-01-27T02:44:42.318 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39827.log.gz 2024-01-27T02:44:42.318 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74470.log 2024-01-27T02:44:42.318 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.66214.log.gz 2024-01-27T02:44:42.318 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43803.log 2024-01-27T02:44:42.318 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43711.log: /var/log/ceph/ceph-client.admin.74470.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43711.log.gz 2024-01-27T02:44:42.318 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74470.log.gz 2024-01-27T02:44:42.318 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43803.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43803.log.gz 2024-01-27T02:44:42.318 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51411.log 2024-01-27T02:44:42.318 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75319.log 2024-01-27T02:44:42.318 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69980.log 2024-01-27T02:44:42.318 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51411.log.gz 2024-01-27T02:44:42.318 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75319.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75319.log.gz 2024-01-27T02:44:42.318 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69980.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69980.log.gz 2024-01-27T02:44:42.319 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59227.log 2024-01-27T02:44:42.319 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45201.log 2024-01-27T02:44:42.319 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48850.log 2024-01-27T02:44:42.319 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59227.log.gz 2024-01-27T02:44:42.319 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47687.log 2024-01-27T02:44:42.319 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48850.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45201.log.gz 2024-01-27T02:44:42.319 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43849.log 2024-01-27T02:44:42.319 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48850.log.gz 2024-01-27T02:44:42.319 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65144.log 2024-01-27T02:44:42.319 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63571.log 2024-01-27T02:44:42.319 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43849.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47687.log.gz 2024-01-27T02:44:42.319 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65144.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43849.log.gz 2024-01-27T02:44:42.319 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65144.log.gz 2024-01-27T02:44:42.320 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45421.log 2024-01-27T02:44:42.320 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72316.log 2024-01-27T02:44:42.320 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63571.log.gz 2024-01-27T02:44:42.320 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52074.log 2024-01-27T02:44:42.320 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57433.log 2024-01-27T02:44:42.320 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45421.log.gz 2024-01-27T02:44:42.320 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72316.log: /var/log/ceph/ceph-client.admin.52074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74424.log 2024-01-27T02:44:42.320 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72316.log.gz 2024-01-27T02:44:42.320 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52074.log.gz 2024-01-27T02:44:42.320 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57433.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57433.log.gz 2024-01-27T02:44:42.320 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74424.log: 88.4%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.74424.log.gz--verbose 2024-01-27T02:44:42.320 INFO:teuthology.orchestra.run.smithi007.stderr:-- /var/log/ceph/ceph-client.admin.41785.log 2024-01-27T02:44:42.320 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54675.log 2024-01-27T02:44:42.320 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62603.log 2024-01-27T02:44:42.321 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41785.log.gz 2024-01-27T02:44:42.321 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56553.log 2024-01-27T02:44:42.321 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55294.log 2024-01-27T02:44:42.321 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54675.log.gz 2024-01-27T02:44:42.321 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56553.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62603.log.gz 2024-01-27T02:44:42.321 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56553.log.gz 2024-01-27T02:44:42.321 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49138.log 2024-01-27T02:44:42.321 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71211.log 2024-01-27T02:44:42.321 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55294.log.gz 2024-01-27T02:44:42.321 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44379.log 2024-01-27T02:44:42.321 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55043.log 2024-01-27T02:44:42.321 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71211.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49138.log.gz 2024-01-27T02:44:42.321 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71211.log.gz 2024-01-27T02:44:42.321 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70935.log 2024-01-27T02:44:42.322 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44379.log.gz 2024-01-27T02:44:42.322 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62235.log 2024-01-27T02:44:42.322 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55043.log.gz 2024-01-27T02:44:42.323 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.44610.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44610.log.gz 2024-01-27T02:44:42.323 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41324.log 2024-01-27T02:44:42.328 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.41324.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41324.log.gz 2024-01-27T02:44:42.328 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45654.log 2024-01-27T02:44:42.332 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75273.log 2024-01-27T02:44:42.332 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70935.log.gz 2024-01-27T02:44:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63671.log 2024-01-27T02:44:42.333 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0%/var/log/ceph/ceph-client.admin.75273.log: -- replaced with /var/log/ceph/ceph-client.admin.62235.log.gz 2024-01-27T02:44:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-01-27T02:44:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63671.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75273.log.gz 2024-01-27T02:44:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mds.a.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63671.log.gz 2024-01-27T02:44:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72500.log 2024-01-27T02:44:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54032.log 2024-01-27T02:44:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75227.log 2024-01-27T02:44:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54032.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72500.log.gz 2024-01-27T02:44:42.333 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54032.log.gz 2024-01-27T02:44:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74010.log 2024-01-27T02:44:42.333 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75227.log.gz 2024-01-27T02:44:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45987.log 2024-01-27T02:44:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49378.log 2024-01-27T02:44:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45987.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74010.log.gz 2024-01-27T02:44:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49378.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45987.log.gz 2024-01-27T02:44:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67349.log 2024-01-27T02:44:42.334 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49378.log.gz 2024-01-27T02:44:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63283.log 2024-01-27T02:44:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67349.log: /var/log/ceph/ceph-client.admin.63283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44449.log 2024-01-27T02:44:42.334 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67349.log.gz 2024-01-27T02:44:42.334 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63283.log.gz 2024-01-27T02:44:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44449.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44449.log.gz 2024-01-27T02:44:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60815.log 2024-01-27T02:44:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47641.log 2024-01-27T02:44:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60815.log: /var/log/ceph/ceph-client.admin.47641.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60815.log.gz 2024-01-27T02:44:42.334 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47641.log.gz 2024-01-27T02:44:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73134.log 2024-01-27T02:44:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71760.log 2024-01-27T02:44:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73134.log: /var/log/ceph/ceph-client.admin.71760.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73134.log.gz 2024-01-27T02:44:42.335 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71760.log.gz 2024-01-27T02:44:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65827.log 2024-01-27T02:44:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-01-27T02:44:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66691.log 2024-01-27T02:44:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65827.log.gz 2024-01-27T02:44:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66691.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66691.log.gz 2024-01-27T02:44:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73918.log 2024-01-27T02:44:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49046.log 2024-01-27T02:44:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73918.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.73918.log.gz 2024-01-27T02:44:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49046.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49046.log.gz 2024-01-27T02:44:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40246.log 2024-01-27T02:44:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50454.log 2024-01-27T02:44:42.336 INFO:teuthology.orchestra.run.smithi007.stderr: 85.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-01-27T02:44:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42245.log 2024-01-27T02:44:42.336 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40246.log.gz 2024-01-27T02:44:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50454.log: /var/log/ceph/ceph-client.admin.42245.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50454.log.gz 2024-01-27T02:44:42.336 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42245.log.gz 2024-01-27T02:44:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49586.log 2024-01-27T02:44:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48350.log 2024-01-27T02:44:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49586.log: /var/log/ceph/ceph-client.admin.48350.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49586.log.gz 2024-01-27T02:44:42.336 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48350.log.gz 2024-01-27T02:44:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39559.log 2024-01-27T02:44:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51548.log 2024-01-27T02:44:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39559.log: /var/log/ceph/ceph-client.admin.51548.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39559.log.gz 2024-01-27T02:44:42.336 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51548.log.gz 2024-01-27T02:44:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55248.log 2024-01-27T02:44:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43527.log 2024-01-27T02:44:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55248.log: /var/log/ceph/ceph-client.admin.43527.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55248.log.gz 2024-01-27T02:44:42.337 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43527.log.gz 2024-01-27T02:44:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45155.log 2024-01-27T02:44:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54721.log 2024-01-27T02:44:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45155.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-01-27T02:44:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54721.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45155.log.gz 2024-01-27T02:44:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.3.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54721.log.gz 2024-01-27T02:44:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70637.log 2024-01-27T02:44:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57099.log 2024-01-27T02:44:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70637.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70637.log.gz 2024-01-27T02:44:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65714.log 2024-01-27T02:44:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57099.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57099.log.gz 2024-01-27T02:44:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51319.log 2024-01-27T02:44:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65714.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65714.log.gz 2024-01-27T02:44:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68014.log 2024-01-27T02:44:42.338 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51319.log.gz 2024-01-27T02:44:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68014.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68014.log.gz 2024-01-27T02:44:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46357.log 2024-01-27T02:44:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57619.log 2024-01-27T02:44:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46357.log: /var/log/ceph/ceph-client.admin.57619.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46357.log.gz 2024-01-27T02:44:42.338 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57619.log.gz 2024-01-27T02:44:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68366.log 2024-01-27T02:44:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70521.log 2024-01-27T02:44:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68366.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68366.log.gz 2024-01-27T02:44:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67844.log 2024-01-27T02:44:42.339 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70521.log.gz 2024-01-27T02:44:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-01-27T02:44:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.67844.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67844.log.gz 2024-01-27T02:44:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64041.log 2024-01-27T02:44:42.340 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71119.log 2024-01-27T02:44:42.340 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64041.log.gz 2024-01-27T02:44:42.341 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68458.log 2024-01-27T02:44:42.341 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71119.log.gz 2024-01-27T02:44:42.342 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68458.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68458.log.gz 2024-01-27T02:44:42.342 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51070.log 2024-01-27T02:44:42.343 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51070.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51070.log.gz 2024-01-27T02:44:42.343 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73555.log 2024-01-27T02:44:42.344 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73555.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73555.log.gz 2024-01-27T02:44:42.344 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60203.log 2024-01-27T02:44:42.345 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60203.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60203.log.gz 2024-01-27T02:44:42.345 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46079.log 2024-01-27T02:44:42.346 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46079.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46079.log.gz 2024-01-27T02:44:42.346 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70981.log 2024-01-27T02:44:42.347 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70981.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70981.log.gz 2024-01-27T02:44:42.347 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67211.log 2024-01-27T02:44:42.348 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67211.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67211.log.gz 2024-01-27T02:44:42.348 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51226.log 2024-01-27T02:44:42.349 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51226.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51226.log.gz 2024-01-27T02:44:42.350 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69934.log 2024-01-27T02:44:42.350 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.45654.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45654.log.gz 2024-01-27T02:44:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69934.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69934.log.gz 2024-01-27T02:44:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64876.log 2024-01-27T02:44:42.352 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64876.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64876.log.gz 2024-01-27T02:44:42.352 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49919.log 2024-01-27T02:44:42.353 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49919.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49919.log.gz 2024-01-27T02:44:42.353 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71506.log 2024-01-27T02:44:42.354 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71506.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71506.log.gz 2024-01-27T02:44:42.354 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44787.log 2024-01-27T02:44:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44787.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44787.log.gz 2024-01-27T02:44:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56921.log 2024-01-27T02:44:42.356 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54395.log 2024-01-27T02:44:42.356 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56921.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56921.log.gz 2024-01-27T02:44:42.357 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54395.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54395.log.gz 2024-01-27T02:44:42.357 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52128.log 2024-01-27T02:44:42.358 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52128.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52128.log.gz 2024-01-27T02:44:42.358 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41395.log 2024-01-27T02:44:42.365 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61448.log 2024-01-27T02:44:42.366 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41395.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41395.log.gz 2024-01-27T02:44:42.366 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43619.log 2024-01-27T02:44:42.367 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61448.log.gz 2024-01-27T02:44:42.367 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50362.log 2024-01-27T02:44:42.367 INFO:teuthology.orchestra.run.smithi188.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-01-27T02:44:42.368 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43619.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43619.log.gz 2024-01-27T02:44:42.368 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50362.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50362.log.gz 2024-01-27T02:44:42.368 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61352.log 2024-01-27T02:44:42.372 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71598.log 2024-01-27T02:44:42.372 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.61352.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61352.log.gz 2024-01-27T02:44:42.372 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44603.log 2024-01-27T02:44:42.372 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71598.log.gz 2024-01-27T02:44:42.372 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69842.log 2024-01-27T02:44:42.372 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44603.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44603.log.gz 2024-01-27T02:44:42.372 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59089.log 2024-01-27T02:44:42.372 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69842.log.gz 2024-01-27T02:44:42.372 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44079.log 2024-01-27T02:44:42.373 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59089.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59089.log.gz 2024-01-27T02:44:42.373 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44079.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44079.log.gz 2024-01-27T02:44:42.373 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54533.log 2024-01-27T02:44:42.374 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54533.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54533.log.gz 2024-01-27T02:44:42.374 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45837.log 2024-01-27T02:44:42.375 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45837.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45837.log.gz 2024-01-27T02:44:42.375 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71850.log 2024-01-27T02:44:42.376 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71850.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71850.log.gz 2024-01-27T02:44:42.376 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51162.log 2024-01-27T02:44:42.377 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51162.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51162.log.gz 2024-01-27T02:44:42.377 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75043.log 2024-01-27T02:44:42.378 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75043.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75043.log.gz 2024-01-27T02:44:42.378 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56831.log 2024-01-27T02:44:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56831.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56831.log.gz 2024-01-27T02:44:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42107.log 2024-01-27T02:44:42.380 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56415.log 2024-01-27T02:44:42.381 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42107.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42107.log.gz/var/log/ceph/ceph-client.admin.56415.log: 2024-01-27T02:44:42.381 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56415.log.gz 2024-01-27T02:44:42.381 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40865.log 2024-01-27T02:44:42.382 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40865.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40865.log.gz 2024-01-27T02:44:42.382 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74997.log 2024-01-27T02:44:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74997.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74997.log.gz 2024-01-27T02:44:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48212.log 2024-01-27T02:44:42.384 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48212.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48212.log.gz 2024-01-27T02:44:42.384 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71982.log 2024-01-27T02:44:42.385 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71982.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71982.log.gz 2024-01-27T02:44:42.385 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53277.log 2024-01-27T02:44:42.387 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53277.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53277.log.gz 2024-01-27T02:44:42.387 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74951.log 2024-01-27T02:44:42.388 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74951.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74951.log.gz 2024-01-27T02:44:42.388 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48711.log 2024-01-27T02:44:42.389 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48711.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48711.log.gz 2024-01-27T02:44:42.389 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63383.log 2024-01-27T02:44:42.389 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45017.log 2024-01-27T02:44:42.391 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63383.log: /var/log/ceph/ceph-client.admin.45017.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45017.log.gz 2024-01-27T02:44:42.391 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63383.log.gz 2024-01-27T02:44:42.391 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57573.log 2024-01-27T02:44:42.392 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57573.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57573.log.gz 2024-01-27T02:44:42.392 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54487.log 2024-01-27T02:44:42.393 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54487.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54487.log.gz 2024-01-27T02:44:42.393 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53986.log 2024-01-27T02:44:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53986.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53986.log.gz 2024-01-27T02:44:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65919.log 2024-01-27T02:44:42.395 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65919.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65919.log.gz 2024-01-27T02:44:42.395 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43941.log 2024-01-27T02:44:42.396 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58795.log 2024-01-27T02:44:42.397 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43941.log: /var/log/ceph/ceph-client.admin.58795.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43941.log.gz 2024-01-27T02:44:42.397 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58795.log.gz 2024-01-27T02:44:42.397 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49186.log 2024-01-27T02:44:42.398 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49186.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49186.log.gz 2024-01-27T02:44:42.398 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51618.log 2024-01-27T02:44:42.400 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51618.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51618.log.gz 2024-01-27T02:44:42.400 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75641.log 2024-01-27T02:44:42.401 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75641.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75641.log.gz 2024-01-27T02:44:42.401 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-01-27T02:44:42.412 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59819.log 2024-01-27T02:44:42.413 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59819.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59819.log.gz 2024-01-27T02:44:42.422 INFO:teuthology.orchestra.run.smithi188.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-01-27T02:44:42.423 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40106.log 2024-01-27T02:44:42.425 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40106.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40106.log.gz 2024-01-27T02:44:42.425 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60631.log 2024-01-27T02:44:42.426 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60631.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60631.log.gz 2024-01-27T02:44:42.426 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69702.log 2024-01-27T02:44:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69702.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69702.log.gz 2024-01-27T02:44:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48466.log 2024-01-27T02:44:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48466.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48466.log.gz 2024-01-27T02:44:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53737.log 2024-01-27T02:44:42.430 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53737.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53737.log.gz 2024-01-27T02:44:42.430 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66785.log 2024-01-27T02:44:42.432 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66785.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.66785.log.gz 2024-01-27T02:44:42.432 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62899.log 2024-01-27T02:44:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62899.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62899.log.gz 2024-01-27T02:44:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73311.log 2024-01-27T02:44:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73311.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73311.log.gz 2024-01-27T02:44:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58703.log 2024-01-27T02:44:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58703.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58703.log.gz 2024-01-27T02:44:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40727.log 2024-01-27T02:44:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40727.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40727.log.gz 2024-01-27T02:44:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69888.log 2024-01-27T02:44:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69888.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69888.log.gz 2024-01-27T02:44:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46756.log 2024-01-27T02:44:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.46756.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46756.log.gz 2024-01-27T02:44:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58649.log 2024-01-27T02:44:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58649.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58649.log.gz 2024-01-27T02:44:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75832.log 2024-01-27T02:44:42.443 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75832.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75832.log.gz 2024-01-27T02:44:42.443 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63107.log 2024-01-27T02:44:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63107.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63107.log.gz 2024-01-27T02:44:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40773.log 2024-01-27T02:44:42.446 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40773.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40773.log.gz 2024-01-27T02:44:42.446 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52268.log 2024-01-27T02:44:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52268.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52268.log.gz 2024-01-27T02:44:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45467.log 2024-01-27T02:44:42.449 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45467.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45467.log.gz 2024-01-27T02:44:42.449 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62419.log 2024-01-27T02:44:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62419.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62419.log.gz 2024-01-27T02:44:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40635.log 2024-01-27T02:44:42.452 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40635.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40635.log.gz 2024-01-27T02:44:42.452 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48626.log 2024-01-27T02:44:42.453 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48626.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48626.log.gz 2024-01-27T02:44:42.453 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73037.log 2024-01-27T02:44:42.454 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.73037.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.73037.log.gz 2024-01-27T02:44:42.454 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41923.log 2024-01-27T02:44:42.455 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41923.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41923.log.gz 2024-01-27T02:44:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60249.log 2024-01-27T02:44:42.457 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60249.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60249.log.gz 2024-01-27T02:44:42.457 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68688.log 2024-01-27T02:44:42.458 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68688.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68688.log.gz 2024-01-27T02:44:42.458 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50408.log 2024-01-27T02:44:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50408.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50408.log.gz 2024-01-27T02:44:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46033.log 2024-01-27T02:44:42.461 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46033.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46033.log.gz 2024-01-27T02:44:42.461 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75733.log 2024-01-27T02:44:42.462 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75733.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75733.log.gz 2024-01-27T02:44:42.462 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60953.log 2024-01-27T02:44:42.463 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60953.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60953.log.gz 2024-01-27T02:44:42.464 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50702.log 2024-01-27T02:44:42.465 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50702.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50702.log.gz 2024-01-27T02:44:42.465 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55955.log 2024-01-27T02:44:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55955.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55955.log.gz 2024-01-27T02:44:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73739.log 2024-01-27T02:44:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73739.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73739.log.gz 2024-01-27T02:44:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61921.log 2024-01-27T02:44:42.469 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61921.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61921.log.gz 2024-01-27T02:44:42.469 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47113.log 2024-01-27T02:44:42.470 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47113.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47113.log.gz 2024-01-27T02:44:42.470 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40058.log 2024-01-27T02:44:42.471 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40058.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40058.log.gz 2024-01-27T02:44:42.471 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71257.log 2024-01-27T02:44:42.473 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71257.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71257.log.gz 2024-01-27T02:44:42.474 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59340.log 2024-01-27T02:44:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59340.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59340.log.gz 2024-01-27T02:44:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41384.log 2024-01-27T02:44:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41384.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41384.log.gz 2024-01-27T02:44:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52314.log 2024-01-27T02:44:42.485 INFO:teuthology.orchestra.run.smithi188.stderr: 94.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-01-27T02:44:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52314.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52314.log.gz 2024-01-27T02:44:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53323.log 2024-01-27T02:44:42.487 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53323.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53323.log.gz 2024-01-27T02:44:42.487 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72730.log 2024-01-27T02:44:42.489 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72730.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.72730.log.gz 2024-01-27T02:44:42.489 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41389.log 2024-01-27T02:44:42.491 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41389.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41389.log.gz 2024-01-27T02:44:42.491 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51794.log 2024-01-27T02:44:42.492 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51794.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51794.log.gz 2024-01-27T02:44:42.492 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54102.log 2024-01-27T02:44:42.493 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54102.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54102.log.gz 2024-01-27T02:44:42.493 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61755.log 2024-01-27T02:44:42.494 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61755.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61755.log.gz 2024-01-27T02:44:42.494 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72074.log 2024-01-27T02:44:42.496 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72074.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72074.log.gz 2024-01-27T02:44:42.496 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48304.log 2024-01-27T02:44:42.497 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48304.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48304.log.gz 2024-01-27T02:44:42.497 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55410.log 2024-01-27T02:44:42.498 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55410.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55410.log.gz 2024-01-27T02:44:42.498 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74813.log 2024-01-27T02:44:42.499 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74813.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74813.log.gz 2024-01-27T02:44:42.500 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72224.log 2024-01-27T02:44:42.501 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72224.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72224.log.gz 2024-01-27T02:44:42.501 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-01-27T02:44:42.505 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.audit.log: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-01-27T02:44:42.505 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63717.log 2024-01-27T02:44:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63717.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63717.log.gz 2024-01-27T02:44:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68596.log 2024-01-27T02:44:42.508 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68596.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68596.log.gz 2024-01-27T02:44:42.508 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42337.log 2024-01-27T02:44:42.509 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59681.log 2024-01-27T02:44:42.510 INFO:teuthology.orchestra.run.smithi007.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.42337.log.gz 2024-01-27T02:44:42.511 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59681.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59681.log.gz 2024-01-27T02:44:42.511 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54629.log 2024-01-27T02:44:42.512 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54629.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54629.log.gz 2024-01-27T02:44:42.512 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62807.log 2024-01-27T02:44:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62807.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62807.log.gz 2024-01-27T02:44:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40681.log 2024-01-27T02:44:42.515 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40681.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40681.log.gz 2024-01-27T02:44:42.515 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68844.log 2024-01-27T02:44:42.515 INFO:teuthology.orchestra.run.smithi007.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-01-27T02:44:42.516 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68844.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68844.log.gz 2024-01-27T02:44:42.516 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52544.log 2024-01-27T02:44:42.517 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52544.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52544.log.gz 2024-01-27T02:44:42.517 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50546.log 2024-01-27T02:44:42.518 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50546.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50546.log.gz 2024-01-27T02:44:42.518 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60861.log 2024-01-27T02:44:42.519 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60861.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60861.log.gz 2024-01-27T02:44:42.519 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72408.log 2024-01-27T02:44:42.520 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72408.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72408.log.gz 2024-01-27T02:44:42.521 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58131.log 2024-01-27T02:44:42.521 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.58131.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58131.log.gz 2024-01-27T02:44:42.521 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60999.log 2024-01-27T02:44:42.523 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60999.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60999.log.gz 2024-01-27T02:44:42.523 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72454.log 2024-01-27T02:44:42.524 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72454.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72454.log.gz 2024-01-27T02:44:42.524 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48896.log 2024-01-27T02:44:42.525 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48896.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48896.log.gz 2024-01-27T02:44:42.525 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53599.log 2024-01-27T02:44:42.525 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70467.log 2024-01-27T02:44:42.527 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53599.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53599.log.gz 2024-01-27T02:44:42.527 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.70467.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70467.log.gz 2024-01-27T02:44:42.527 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75503.log 2024-01-27T02:44:42.528 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75503.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75503.log.gz 2024-01-27T02:44:42.528 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58841.log 2024-01-27T02:44:42.529 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58841.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58841.log.gz 2024-01-27T02:44:42.529 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44174.log 2024-01-27T02:44:42.531 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44174.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44174.log.gz 2024-01-27T02:44:42.531 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41068.log 2024-01-27T02:44:42.532 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41068.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41068.log.gz 2024-01-27T02:44:42.532 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72638.log 2024-01-27T02:44:42.534 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72638.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72638.log.gz 2024-01-27T02:44:42.534 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60491.log 2024-01-27T02:44:42.534 INFO:teuthology.orchestra.run.smithi007.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-01-27T02:44:42.535 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60491.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60491.log.gz 2024-01-27T02:44:42.535 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58473.log 2024-01-27T02:44:42.535 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48165.log 2024-01-27T02:44:42.536 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58473.log: /var/log/ceph/ceph-client.admin.48165.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58473.log.gz 2024-01-27T02:44:42.536 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48165.log.gz 2024-01-27T02:44:42.536 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67257.log 2024-01-27T02:44:42.537 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71303.log 2024-01-27T02:44:42.537 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67257.log: /var/log/ceph/ceph-client.admin.71303.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67257.log.gz 2024-01-27T02:44:42.538 INFO:teuthology.orchestra.run.smithi007.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.71303.log.gz 2024-01-27T02:44:42.538 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50748.log 2024-01-27T02:44:42.538 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74859.log 2024-01-27T02:44:42.539 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50748.log: /var/log/ceph/ceph-client.admin.74859.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50748.log.gz 2024-01-27T02:44:42.539 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74859.log.gz 2024-01-27T02:44:42.539 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42153.log 2024-01-27T02:44:42.540 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67303.log 2024-01-27T02:44:42.540 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42153.log: /var/log/ceph/ceph-client.admin.67303.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42153.log.gz 2024-01-27T02:44:42.541 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67303.log.gz 2024-01-27T02:44:42.541 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52993.log 2024-01-27T02:44:42.541 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72168.log 2024-01-27T02:44:42.542 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52993.log: /var/log/ceph/ceph-client.admin.72168.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52993.log.gz 2024-01-27T02:44:42.542 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72168.log.gz 2024-01-27T02:44:42.542 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69506.log 2024-01-27T02:44:42.543 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69506.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69506.log.gz 2024-01-27T02:44:42.543 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74905.log 2024-01-27T02:44:42.544 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71027.log 2024-01-27T02:44:42.544 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74905.log: /var/log/ceph/ceph-client.admin.71027.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74905.log.gz 2024-01-27T02:44:42.545 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71027.log.gz 2024-01-27T02:44:42.545 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42983.log 2024-01-27T02:44:42.545 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41058.log 2024-01-27T02:44:42.546 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.42983.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42983.log.gz 2024-01-27T02:44:42.546 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41058.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41058.log.gz 2024-01-27T02:44:42.546 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49823.log 2024-01-27T02:44:42.547 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44649.log 2024-01-27T02:44:42.547 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.49823.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49823.log.gz 2024-01-27T02:44:42.548 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44649.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44649.log.gz 2024-01-27T02:44:42.548 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61402.log 2024-01-27T02:44:42.548 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61348.log 2024-01-27T02:44:42.549 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61402.log: /var/log/ceph/ceph-client.0.61348.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61402.log.gz 2024-01-27T02:44:42.549 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61348.log.gz 2024-01-27T02:44:42.549 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52660.log 2024-01-27T02:44:42.549 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58227.log 2024-01-27T02:44:42.550 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52660.log: /var/log/ceph/ceph-client.admin.58227.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52660.log.gz 2024-01-27T02:44:42.550 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58227.log.gz 2024-01-27T02:44:42.551 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56461.log 2024-01-27T02:44:42.551 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64766.log 2024-01-27T02:44:42.551 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56461.log: /var/log/ceph/ceph-client.admin.64766.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56461.log.gz 2024-01-27T02:44:42.552 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.64766.log.gz 2024-01-27T02:44:42.552 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41071.log 2024-01-27T02:44:42.552 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70234.log 2024-01-27T02:44:42.553 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41071.log: /var/log/ceph/ceph-client.admin.70234.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41071.log.gz 2024-01-27T02:44:42.553 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70234.log.gz 2024-01-27T02:44:42.554 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53894.log 2024-01-27T02:44:42.554 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64200.log 2024-01-27T02:44:42.554 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53894.log: /var/log/ceph/ceph-client.admin.64200.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53894.log.gz 2024-01-27T02:44:42.555 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64200.log.gz 2024-01-27T02:44:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54859.log 2024-01-27T02:44:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60539.log 2024-01-27T02:44:42.556 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54859.log: /var/log/ceph/ceph-client.admin.60539.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54859.log.gz 2024-01-27T02:44:42.556 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60539.log.gz 2024-01-27T02:44:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66737.log 2024-01-27T02:44:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60155.log 2024-01-27T02:44:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66737.log: /var/log/ceph/ceph-client.admin.60155.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66737.log.gz 2024-01-27T02:44:42.558 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60155.log.gz 2024-01-27T02:44:42.558 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63475.log 2024-01-27T02:44:42.558 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71936.log 2024-01-27T02:44:42.559 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63475.log: /var/log/ceph/ceph-client.admin.71936.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63475.log.gz 2024-01-27T02:44:42.559 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71936.log.gz 2024-01-27T02:44:42.559 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48258.log 2024-01-27T02:44:42.560 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55647.log 2024-01-27T02:44:42.561 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48258.log: /var/log/ceph/ceph-client.0.55647.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48258.log.gz 2024-01-27T02:44:42.561 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55647.log.gz 2024-01-27T02:44:42.561 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46265.log 2024-01-27T02:44:42.561 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72120.log 2024-01-27T02:44:42.562 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46265.log: /var/log/ceph/ceph-client.admin.72120.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46265.log.gz 2024-01-27T02:44:42.562 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72120.log.gz 2024-01-27T02:44:42.562 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69075.log 2024-01-27T02:44:42.563 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67449.log 2024-01-27T02:44:42.563 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69075.log: /var/log/ceph/ceph-client.admin.67449.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69075.log.gz 2024-01-27T02:44:42.564 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67449.log.gz 2024-01-27T02:44:42.564 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73469.log 2024-01-27T02:44:42.564 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46806.log 2024-01-27T02:44:42.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73469.log: /var/log/ceph/ceph-client.admin.46806.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73469.log.gz 2024-01-27T02:44:42.565 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46806.log.gz 2024-01-27T02:44:42.565 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56761.log 2024-01-27T02:44:42.566 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66887.log 2024-01-27T02:44:42.566 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56761.log: 87.5%/var/log/ceph/ceph-client.admin.66887.log: -- replaced with /var/log/ceph/ceph-client.admin.56761.log.gz 2024-01-27T02:44:42.567 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66887.log.gz 2024-01-27T02:44:42.567 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45375.log 2024-01-27T02:44:42.567 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63037.log 2024-01-27T02:44:42.568 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45375.log: /var/log/ceph/ceph-client.admin.63037.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45375.log.gz 2024-01-27T02:44:42.568 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63037.log.gz 2024-01-27T02:44:42.568 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74767.log 2024-01-27T02:44:42.568 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52897.log 2024-01-27T02:44:42.569 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74767.log: /var/log/ceph/ceph-client.0.52897.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74767.log.gz 2024-01-27T02:44:42.569 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52897.log.gz 2024-01-27T02:44:42.569 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47871.log 2024-01-27T02:44:42.570 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74516.log 2024-01-27T02:44:42.570 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47871.log: /var/log/ceph/ceph-client.admin.74516.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47871.log.gz 2024-01-27T02:44:42.571 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74516.log.gz 2024-01-27T02:44:42.571 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46852.log 2024-01-27T02:44:42.571 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52590.log 2024-01-27T02:44:42.572 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46852.log: /var/log/ceph/ceph-client.admin.52590.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46852.log.gz 2024-01-27T02:44:42.573 INFO:teuthology.orchestra.run.smithi007.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52590.log.gz 2024-01-27T02:44:42.573 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64362.log 2024-01-27T02:44:42.573 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71552.log 2024-01-27T02:44:42.574 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64362.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.64362.log.gz 2024-01-27T02:44:42.574 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71552.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71552.log.gz 2024-01-27T02:44:42.574 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54349.log 2024-01-27T02:44:42.575 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52220.log 2024-01-27T02:44:42.575 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54349.log: /var/log/ceph/ceph-client.admin.52220.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54349.log.gz 2024-01-27T02:44:42.576 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52220.log.gz 2024-01-27T02:44:42.576 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70072.log 2024-01-27T02:44:42.576 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65530.log 2024-01-27T02:44:42.577 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70072.log: 87.4%/var/log/ceph/ceph-client.admin.65530.log: -- replaced with /var/log/ceph/ceph-client.admin.70072.log.gz 2024-01-27T02:44:42.577 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65530.log.gz 2024-01-27T02:44:42.577 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44128.log 2024-01-27T02:44:42.578 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53415.log 2024-01-27T02:44:42.579 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44128.log: /var/log/ceph/ceph-client.admin.53415.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44128.log.gz 2024-01-27T02:44:42.579 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53415.log.gz 2024-01-27T02:44:42.579 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47825.log 2024-01-27T02:44:42.579 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45933.log 2024-01-27T02:44:42.580 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47825.log: /var/log/ceph/ceph-client.admin.45933.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47825.log.gz 2024-01-27T02:44:42.580 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45933.log.gz 2024-01-27T02:44:42.580 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45745.log 2024-01-27T02:44:42.581 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75788.log 2024-01-27T02:44:42.581 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45745.log: /var/log/ceph/ceph-client.admin.75788.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45745.log.gz 2024-01-27T02:44:42.582 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.75788.log.gz 2024-01-27T02:44:42.582 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63765.log 2024-01-27T02:44:42.582 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62945.log 2024-01-27T02:44:42.583 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63765.log: /var/log/ceph/ceph-client.admin.62945.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63765.log.gz 2024-01-27T02:44:42.583 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62945.log.gz 2024-01-27T02:44:42.583 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54173.log 2024-01-27T02:44:42.584 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63903.log 2024-01-27T02:44:42.584 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54173.log: /var/log/ceph/ceph-client.admin.63903.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54173.log.gz 2024-01-27T02:44:42.585 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63903.log.gz 2024-01-27T02:44:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56047.log 2024-01-27T02:44:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69282.log 2024-01-27T02:44:42.586 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56047.log: /var/log/ceph/ceph-client.admin.69282.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56047.log.gz 2024-01-27T02:44:42.586 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69282.log.gz 2024-01-27T02:44:42.586 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57053.log 2024-01-27T02:44:42.587 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52026.log 2024-01-27T02:44:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43987.log 2024-01-27T02:44:42.588 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57053.log.gz 2024-01-27T02:44:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52026.log: /var/log/ceph/ceph-client.admin.43987.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52026.log.gz 2024-01-27T02:44:42.588 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43987.log.gz 2024-01-27T02:44:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43757.log 2024-01-27T02:44:42.589 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51024.log 2024-01-27T02:44:42.589 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43757.log: /var/log/ceph/ceph-client.admin.51024.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43757.log.gz 2024-01-27T02:44:42.590 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51024.log.gz 2024-01-27T02:44:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66377.log 2024-01-27T02:44:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65873.log 2024-01-27T02:44:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66377.log: /var/log/ceph/ceph-client.admin.65873.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.66377.log.gz 2024-01-27T02:44:42.591 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65873.log.gz 2024-01-27T02:44:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42491.log 2024-01-27T02:44:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56323.log 2024-01-27T02:44:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42491.log: /var/log/ceph/ceph-client.admin.56323.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42491.log.gz 2024-01-27T02:44:42.593 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56323.log.gz 2024-01-27T02:44:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42061.log 2024-01-27T02:44:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48944.log 2024-01-27T02:44:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42061.log: /var/log/ceph/ceph-client.admin.48944.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42061.log.gz 2024-01-27T02:44:42.594 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48944.log.gz 2024-01-27T02:44:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52174.log 2024-01-27T02:44:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44833.log 2024-01-27T02:44:42.595 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52174.log: /var/log/ceph/ceph-client.admin.44833.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52174.log.gz 2024-01-27T02:44:42.595 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44833.log.gz 2024-01-27T02:44:42.596 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61045.log 2024-01-27T02:44:42.596 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49240.log 2024-01-27T02:44:42.597 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61045.log: /var/log/ceph/ceph-client.admin.49240.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49240.log.gz 2024-01-27T02:44:42.597 INFO:teuthology.orchestra.run.smithi007.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.61045.log.gz 2024-01-27T02:44:42.597 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49873.log 2024-01-27T02:44:42.597 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67848.log 2024-01-27T02:44:42.598 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49873.log: /var/log/ceph/ceph-client.0.67848.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49873.log.gz 2024-01-27T02:44:42.598 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67848.log.gz 2024-01-27T02:44:42.598 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55643.log 2024-01-27T02:44:42.599 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50794.log 2024-01-27T02:44:42.599 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.55643.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55643.log.gz 2024-01-27T02:44:42.600 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50794.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50794.log.gz 2024-01-27T02:44:42.600 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69121.log 2024-01-27T02:44:42.600 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54905.log 2024-01-27T02:44:42.601 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69121.log: /var/log/ceph/ceph-client.admin.54905.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69121.log.gz 2024-01-27T02:44:42.601 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54905.log.gz 2024-01-27T02:44:42.601 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68937.log 2024-01-27T02:44:42.602 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57287.log 2024-01-27T02:44:42.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68937.log: /var/log/ceph/ceph-client.admin.57287.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68937.log.gz 2024-01-27T02:44:42.603 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57287.log.gz 2024-01-27T02:44:42.603 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62281.log 2024-01-27T02:44:42.603 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74562.log 2024-01-27T02:44:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62281.log: /var/log/ceph/ceph-client.admin.74562.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62281.log.gz 2024-01-27T02:44:42.604 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74562.log.gz 2024-01-27T02:44:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45607.log 2024-01-27T02:44:42.605 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45561.log 2024-01-27T02:44:42.605 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45607.log: /var/log/ceph/ceph-client.admin.45561.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45607.log.gz 2024-01-27T02:44:42.606 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45561.log.gz 2024-01-27T02:44:42.606 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64988.log 2024-01-27T02:44:42.606 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50978.log 2024-01-27T02:44:42.607 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64988.log: /var/log/ceph/ceph-client.admin.50978.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64988.log.gz 2024-01-27T02:44:42.607 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51886.log 2024-01-27T02:44:42.607 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50978.log.gz 2024-01-27T02:44:42.607 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60345.log 2024-01-27T02:44:42.608 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51886.log: /var/log/ceph/ceph-client.admin.60345.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51886.log.gz 2024-01-27T02:44:42.608 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41067.log 2024-01-27T02:44:42.608 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60345.log.gz 2024-01-27T02:44:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75411.log 2024-01-27T02:44:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41067.log: 88.2%/var/log/ceph/ceph-client.admin.75411.log: -- replaced with /var/log/ceph/ceph-client.admin.41067.log.gz 2024-01-27T02:44:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51932.log 2024-01-27T02:44:42.609 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75411.log.gz 2024-01-27T02:44:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51932.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51932.log.gz 2024-01-27T02:44:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73647.log 2024-01-27T02:44:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69748.log 2024-01-27T02:44:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73647.log: /var/log/ceph/ceph-client.admin.69748.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73647.log.gz 2024-01-27T02:44:42.612 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.69748.log.gz 2024-01-27T02:44:42.612 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57191.log 2024-01-27T02:44:42.612 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64292.log 2024-01-27T02:44:42.613 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57191.log: /var/log/ceph/ceph-client.admin.64292.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57191.log.gz 2024-01-27T02:44:42.614 INFO:teuthology.orchestra.run.smithi007.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.64292.log.gz 2024-01-27T02:44:42.614 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68780.log 2024-01-27T02:44:42.614 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53940.log 2024-01-27T02:44:42.615 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68780.log: /var/log/ceph/ceph-client.admin.53940.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68780.log.gz 2024-01-27T02:44:42.615 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53940.log.gz 2024-01-27T02:44:42.615 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43895.log 2024-01-27T02:44:42.616 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40405.log 2024-01-27T02:44:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54951.log 2024-01-27T02:44:42.617 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43895.log.gz 2024-01-27T02:44:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40405.log: /var/log/ceph/ceph-client.admin.54951.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40405.log.gz 2024-01-27T02:44:42.617 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54951.log.gz 2024-01-27T02:44:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47779.log 2024-01-27T02:44:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44879.log 2024-01-27T02:44:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47779.log: /var/log/ceph/ceph-client.admin.44879.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47779.log.gz 2024-01-27T02:44:42.619 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44879.log.gz 2024-01-27T02:44:42.619 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56691.log 2024-01-27T02:44:42.619 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65484.log 2024-01-27T02:44:42.620 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56691.log: /var/log/ceph/ceph-client.admin.65484.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56691.log.gz 2024-01-27T02:44:42.620 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65484.log.gz 2024-01-27T02:44:42.620 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42015.log 2024-01-27T02:44:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40543.log 2024-01-27T02:44:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42015.log: /var/log/ceph/ceph-client.admin.40543.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42015.log.gz 2024-01-27T02:44:42.622 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40543.log.gz 2024-01-27T02:44:42.622 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44033.log 2024-01-27T02:44:42.622 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52947.log 2024-01-27T02:44:42.623 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44033.log: /var/log/ceph/ceph-client.admin.52947.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44033.log.gz 2024-01-27T02:44:42.623 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52947.log.gz 2024-01-27T02:44:42.623 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69552.log 2024-01-27T02:44:42.624 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47327.log 2024-01-27T02:44:42.624 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69552.log: /var/log/ceph/ceph-client.admin.47327.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69552.log.gz 2024-01-27T02:44:42.625 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47327.log.gz 2024-01-27T02:44:42.625 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48803.log 2024-01-27T02:44:42.625 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57665.log 2024-01-27T02:44:42.626 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48803.log: /var/log/ceph/ceph-client.admin.57665.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48803.log.gz 2024-01-27T02:44:42.626 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57665.log.gz 2024-01-27T02:44:42.626 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74148.log 2024-01-27T02:44:42.627 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73964.log 2024-01-27T02:44:42.627 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74148.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74148.log.gz 2024-01-27T02:44:42.628 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73964.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73964.log.gz 2024-01-27T02:44:42.628 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63429.log 2024-01-27T02:44:42.628 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66552.log 2024-01-27T02:44:42.629 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63429.log: /var/log/ceph/ceph-client.admin.66552.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63429.log.gz 2024-01-27T02:44:42.629 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66552.log.gz 2024-01-27T02:44:42.629 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57527.log 2024-01-27T02:44:42.630 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50232.log 2024-01-27T02:44:42.630 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57527.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57527.log.gz 2024-01-27T02:44:42.631 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50232.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50232.log.gz 2024-01-27T02:44:42.631 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55135.log 2024-01-27T02:44:42.631 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45063.log 2024-01-27T02:44:42.632 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55135.log: /var/log/ceph/ceph-client.admin.45063.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55135.log.gz 2024-01-27T02:44:42.632 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45063.log.gz 2024-01-27T02:44:42.632 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57778.log 2024-01-27T02:44:42.632 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56277.log 2024-01-27T02:44:42.633 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57778.log: /var/log/ceph/ceph-client.admin.56277.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57778.log.gz 2024-01-27T02:44:42.634 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56277.log.gz 2024-01-27T02:44:42.634 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65576.log 2024-01-27T02:44:42.635 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65576.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.65576.log.gz 2024-01-27T02:44:42.635 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66598.log 2024-01-27T02:44:42.635 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39742.log 2024-01-27T02:44:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66598.log: /var/log/ceph/ceph-client.admin.39742.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.66598.log.gz 2024-01-27T02:44:42.636 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39742.log.gz 2024-01-27T02:44:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57239.log 2024-01-27T02:44:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58407.log 2024-01-27T02:44:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57239.log: /var/log/ceph/ceph-client.admin.58407.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57239.log.gz 2024-01-27T02:44:42.638 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58407.log.gz 2024-01-27T02:44:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58749.log 2024-01-27T02:44:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48757.log 2024-01-27T02:44:42.639 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58749.log: /var/log/ceph/ceph-client.admin.48757.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58749.log.gz 2024-01-27T02:44:42.639 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48757.log.gz 2024-01-27T02:44:42.639 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66076.log 2024-01-27T02:44:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59889.log 2024-01-27T02:44:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66076.log: /var/log/ceph/ceph-client.admin.59889.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66076.log.gz 2024-01-27T02:44:42.641 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59889.log.gz 2024-01-27T02:44:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50886.log 2024-01-27T02:44:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42291.log 2024-01-27T02:44:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50886.log: /var/log/ceph/ceph-client.admin.42291.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50886.log.gz 2024-01-27T02:44:42.642 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42291.log.gz 2024-01-27T02:44:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71349.log 2024-01-27T02:44:42.643 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41072.log 2024-01-27T02:44:42.643 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71349.log: /var/log/ceph/ceph-client.admin.41072.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71349.log.gz 2024-01-27T02:44:42.644 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41072.log.gz 2024-01-27T02:44:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64922.log 2024-01-27T02:44:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66168.log 2024-01-27T02:44:42.645 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64922.log: /var/log/ceph/ceph-client.admin.66168.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64922.log.gz 2024-01-27T02:44:42.645 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66168.log.gz 2024-01-27T02:44:42.645 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56369.log 2024-01-27T02:44:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56599.log 2024-01-27T02:44:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56369.log: /var/log/ceph/ceph-client.admin.56599.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56369.log.gz 2024-01-27T02:44:42.647 INFO:teuthology.orchestra.run.smithi188.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-01-27T02:44:42.647 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56599.log.gz 2024-01-27T02:44:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45791.log 2024-01-27T02:44:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70164.log 2024-01-27T02:44:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45791.log: /var/log/ceph/ceph-client.admin.70164.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45791.log.gz 2024-01-27T02:44:42.648 INFO:teuthology.orchestra.run.smithi007.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.70164.log.gz 2024-01-27T02:44:42.649 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56231.log 2024-01-27T02:44:42.649 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53645.log 2024-01-27T02:44:42.650 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56231.log: /var/log/ceph/ceph-client.admin.53645.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56231.log.gz 2024-01-27T02:44:42.650 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53645.log.gz 2024-01-27T02:44:42.650 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47067.log 2024-01-27T02:44:42.650 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43573.log 2024-01-27T02:44:42.651 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47067.log: /var/log/ceph/ceph-client.admin.43573.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47067.log.gz 2024-01-27T02:44:42.652 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43573.log.gz 2024-01-27T02:44:42.652 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58361.log 2024-01-27T02:44:42.652 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73357.log 2024-01-27T02:44:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58361.log: /var/log/ceph/ceph-client.admin.73357.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58361.log.gz 2024-01-27T02:44:42.653 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73357.log.gz 2024-01-27T02:44:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51708.log 2024-01-27T02:44:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44925.log 2024-01-27T02:44:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51708.log: /var/log/ceph/ceph-client.admin.44925.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51708.log.gz 2024-01-27T02:44:42.655 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44925.log.gz 2024-01-27T02:44:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70889.log 2024-01-27T02:44:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65965.log 2024-01-27T02:44:42.656 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70889.log: /var/log/ceph/ceph-client.admin.65965.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70889.log.gz 2024-01-27T02:44:42.656 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65965.log.gz 2024-01-27T02:44:42.656 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39308.log 2024-01-27T02:44:42.657 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60109.log 2024-01-27T02:44:42.657 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39308.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39308.log.gz 2024-01-27T02:44:42.658 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60109.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60109.log.gz 2024-01-27T02:44:42.658 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-01-27T02:44:42.658 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59181.log 2024-01-27T02:44:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.59181.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59181.log.gz 2024-01-27T02:44:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50186.log 2024-01-27T02:44:42.665 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50186.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50186.log.gz 2024-01-27T02:44:42.665 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47963.log 2024-01-27T02:44:42.666 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47963.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47963.log.gz 2024-01-27T02:44:42.666 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41831.log 2024-01-27T02:44:42.667 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41831.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41831.log.gz 2024-01-27T02:44:42.667 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41069.log 2024-01-27T02:44:42.669 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65668.log 2024-01-27T02:44:42.669 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41069.log: /var/log/ceph/ceph-client.admin.65668.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41069.log.gz 2024-01-27T02:44:42.669 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65668.log.gz 2024-01-27T02:44:42.669 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74102.log 2024-01-27T02:44:42.670 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74102.log: 94.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-01-27T02:44:42.670 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74102.log.gz 2024-01-27T02:44:42.670 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74194.log 2024-01-27T02:44:42.670 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41440.log 2024-01-27T02:44:42.671 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74194.log: /var/log/ceph/ceph-client.admin.41440.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74194.log.gz 2024-01-27T02:44:42.671 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41440.log.gz 2024-01-27T02:44:42.671 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57341.log 2024-01-27T02:44:42.672 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67165.log 2024-01-27T02:44:42.672 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57341.log: /var/log/ceph/ceph-client.admin.67165.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57341.log.gz 2024-01-27T02:44:42.673 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67165.log.gz 2024-01-27T02:44:42.673 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47180.log 2024-01-27T02:44:42.673 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51365.log 2024-01-27T02:44:42.674 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64696.log 2024-01-27T02:44:42.674 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47180.log.gz 2024-01-27T02:44:42.674 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51365.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51365.log.gz 2024-01-27T02:44:42.675 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64696.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64696.log.gz 2024-01-27T02:44:42.675 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43665.log 2024-01-27T02:44:42.676 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72028.log 2024-01-27T02:44:42.677 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69458.log 2024-01-27T02:44:42.677 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43665.log.gz 2024-01-27T02:44:42.677 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72028.log: /var/log/ceph/ceph-client.admin.69458.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72028.log.gz 2024-01-27T02:44:42.677 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69458.log.gz 2024-01-27T02:44:42.677 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60907.log 2024-01-27T02:44:42.678 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46173.log 2024-01-27T02:44:42.678 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60907.log: /var/log/ceph/ceph-client.admin.46173.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60907.log.gz 2024-01-27T02:44:42.679 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46173.log.gz 2024-01-27T02:44:42.679 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63857.log 2024-01-27T02:44:42.680 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63857.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63857.log.gz 2024-01-27T02:44:42.680 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45515.log 2024-01-27T02:44:42.680 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69412.log 2024-01-27T02:44:42.681 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45515.log: /var/log/ceph/ceph-client.admin.69412.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45515.log.gz 2024-01-27T02:44:42.681 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.69412.log.gz 2024-01-27T02:44:42.681 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69648.log 2024-01-27T02:44:42.682 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52360.log 2024-01-27T02:44:42.682 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69648.log: /var/log/ceph/ceph-client.admin.52360.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69648.log.gz 2024-01-27T02:44:42.683 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52360.log.gz 2024-01-27T02:44:42.683 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39633.log 2024-01-27T02:44:42.683 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66933.log 2024-01-27T02:44:42.684 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39633.log: /var/log/ceph/ceph-client.admin.66933.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39633.log.gz 2024-01-27T02:44:42.684 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66933.log.gz 2024-01-27T02:44:42.684 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50840.log 2024-01-27T02:44:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60297.log 2024-01-27T02:44:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50840.log: /var/log/ceph/ceph-client.admin.60297.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50840.log.gz 2024-01-27T02:44:42.686 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60297.log.gz 2024-01-27T02:44:42.686 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59727.log 2024-01-27T02:44:42.686 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67073.log 2024-01-27T02:44:42.687 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59727.log: /var/log/ceph/ceph-client.admin.67073.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59727.log.gz 2024-01-27T02:44:42.687 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67073.log.gz 2024-01-27T02:44:42.687 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40292.log 2024-01-27T02:44:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65236.log 2024-01-27T02:44:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40292.log: /var/log/ceph/ceph-client.admin.65236.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40292.log.gz 2024-01-27T02:44:42.689 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65236.log.gz 2024-01-27T02:44:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48009.log 2024-01-27T02:44:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67495.log 2024-01-27T02:44:42.690 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48009.log: /var/log/ceph/ceph-client.admin.67495.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48009.log.gz 2024-01-27T02:44:42.690 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67495.log.gz 2024-01-27T02:44:42.690 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50120.log 2024-01-27T02:44:42.691 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66833.log 2024-01-27T02:44:42.691 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50120.log: /var/log/ceph/ceph-client.admin.66833.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50120.log.gz 2024-01-27T02:44:42.692 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66833.log.gz 2024-01-27T02:44:42.692 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63217.log 2024-01-27T02:44:42.692 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62327.log 2024-01-27T02:44:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63217.log: /var/log/ceph/ceph-client.admin.62327.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63217.log.gz 2024-01-27T02:44:42.693 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62327.log.gz 2024-01-27T02:44:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75365.log 2024-01-27T02:44:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65190.log 2024-01-27T02:44:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75365.log: /var/log/ceph/ceph-client.admin.65190.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.75365.log.gz 2024-01-27T02:44:42.695 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65190.log.gz 2024-01-27T02:44:42.695 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68983.log 2024-01-27T02:44:42.695 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46519.log 2024-01-27T02:44:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68983.log: /var/log/ceph/ceph-client.admin.46519.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68983.log.gz 2024-01-27T02:44:42.696 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46519.log.gz 2024-01-27T02:44:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67898.log 2024-01-27T02:44:42.697 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45699.log 2024-01-27T02:44:42.697 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67898.log: /var/log/ceph/ceph-client.admin.45699.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67898.log.gz 2024-01-27T02:44:42.698 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45699.log.gz 2024-01-27T02:44:42.698 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47917.log 2024-01-27T02:44:42.698 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62373.log 2024-01-27T02:44:42.699 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47917.log: /var/log/ceph/ceph-client.admin.62373.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47917.log.gz 2024-01-27T02:44:42.699 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62373.log.gz 2024-01-27T02:44:42.699 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71644.log 2024-01-27T02:44:42.700 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67611.log 2024-01-27T02:44:42.700 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71644.log: /var/log/ceph/ceph-client.admin.67611.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71644.log.gz 2024-01-27T02:44:42.701 INFO:teuthology.orchestra.run.smithi007.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.67611.log.gz 2024-01-27T02:44:42.701 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54441.log 2024-01-27T02:44:42.701 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43033.log 2024-01-27T02:44:42.702 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54441.log: /var/log/ceph/ceph-client.admin.43033.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54441.log.gz 2024-01-27T02:44:42.702 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43033.log.gz 2024-01-27T02:44:42.702 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54263.log 2024-01-27T02:44:42.702 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68642.log 2024-01-27T02:44:42.703 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54263.log: /var/log/ceph/ceph-client.admin.68642.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54263.log.gz 2024-01-27T02:44:42.704 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68642.log.gz 2024-01-27T02:44:42.704 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74286.log 2024-01-27T02:44:42.704 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41969.log 2024-01-27T02:44:42.705 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74286.log: /var/log/ceph/ceph-client.admin.41969.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74286.log.gz 2024-01-27T02:44:42.705 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41969.log.gz 2024-01-27T02:44:42.705 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66467.log 2024-01-27T02:44:42.705 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55202.log 2024-01-27T02:44:42.706 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66467.log: /var/log/ceph/ceph-client.admin.55202.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.66467.log.gz 2024-01-27T02:44:42.706 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55202.log.gz 2024-01-27T02:44:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50074.log 2024-01-27T02:44:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46449.log 2024-01-27T02:44:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50074.log: /var/log/ceph/ceph-client.admin.46449.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50074.log.gz 2024-01-27T02:44:42.708 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.46449.log.gz 2024-01-27T02:44:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49332.log 2024-01-27T02:44:42.709 INFO:teuthology.orchestra.run.smithi188.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-01-27T02:44:42.710 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72592.log 2024-01-27T02:44:42.710 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49332.log: /var/log/ceph/ceph-client.admin.72592.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49332.log.gz 2024-01-27T02:44:42.710 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72592.log.gz 2024-01-27T02:44:42.710 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64650.log 2024-01-27T02:44:42.710 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64087.log 2024-01-27T02:44:42.711 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64650.log: /var/log/ceph/ceph-client.admin.64087.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64650.log.gz 2024-01-27T02:44:42.711 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64087.log.gz 2024-01-27T02:44:42.711 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41388.log 2024-01-27T02:44:42.712 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52452.log 2024-01-27T02:44:42.712 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41388.log: /var/log/ceph/ceph-client.admin.52452.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41388.log.gz 2024-01-27T02:44:42.713 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52452.log.gz 2024-01-27T02:44:42.713 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67403.log 2024-01-27T02:44:42.713 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46922.log 2024-01-27T02:44:42.714 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67403.log: /var/log/ceph/ceph-client.admin.46922.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67403.log.gz 2024-01-27T02:44:42.714 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46922.log.gz 2024-01-27T02:44:42.715 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71690.log 2024-01-27T02:44:42.715 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70118.log 2024-01-27T02:44:42.716 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71690.log: /var/log/ceph/ceph-client.admin.70118.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71690.log.gz 2024-01-27T02:44:42.716 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.70118.log.gz 2024-01-27T02:44:42.716 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59542.log 2024-01-27T02:44:42.716 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60585.log 2024-01-27T02:44:42.717 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59542.log: /var/log/ceph/ceph-client.admin.60585.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59542.log.gz 2024-01-27T02:44:42.717 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60585.log.gz 2024-01-27T02:44:42.718 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67027.log 2024-01-27T02:44:42.718 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73872.log 2024-01-27T02:44:42.718 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67027.log: /var/log/ceph/ceph-client.admin.73872.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67027.log.gz 2024-01-27T02:44:42.719 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73872.log.gz 2024-01-27T02:44:42.719 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51840.log 2024-01-27T02:44:42.719 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55697.log 2024-01-27T02:44:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51840.log: /var/log/ceph/ceph-client.admin.55697.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51840.log.gz 2024-01-27T02:44:42.720 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55697.log.gz 2024-01-27T02:44:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41647.log 2024-01-27T02:44:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63995.log 2024-01-27T02:44:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41647.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41647.log.gz 2024-01-27T02:44:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63995.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63995.log.gz 2024-01-27T02:44:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54813.log 2024-01-27T02:44:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59135.log 2024-01-27T02:44:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54813.log: /var/log/ceph/ceph-client.admin.59135.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54813.log.gz 2024-01-27T02:44:42.723 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59135.log.gz 2024-01-27T02:44:42.939 INFO:teuthology.orchestra.run.smithi188.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-01-27T02:44:43.060 INFO:teuthology.orchestra.run.smithi188.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-01-27T02:44:43.065 INFO:teuthology.orchestra.run.smithi188.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-01-27T02:44:43.079 INFO:teuthology.orchestra.run.smithi188.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-01-27T02:44:43.080 INFO:teuthology.orchestra.run.smithi188.stderr: 2024-01-27T02:44:43.080 INFO:teuthology.orchestra.run.smithi188.stderr:real 0m0.854s 2024-01-27T02:44:43.080 INFO:teuthology.orchestra.run.smithi188.stderr:user 0m4.288s 2024-01-27T02:44:43.081 INFO:teuthology.orchestra.run.smithi188.stderr:sys 0m0.190s 2024-01-27T02:44:43.166 INFO:teuthology.orchestra.run.smithi007.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-01-27T02:44:43.179 INFO:teuthology.orchestra.run.smithi007.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-01-27T02:44:43.206 INFO:teuthology.orchestra.run.smithi007.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-01-27T02:44:43.236 INFO:teuthology.orchestra.run.smithi007.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-01-27T02:44:43.458 INFO:teuthology.orchestra.run.smithi007.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-01-27T02:44:43.460 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-01-27T02:44:43.460 INFO:teuthology.orchestra.run.smithi007.stderr:real 0m1.256s 2024-01-27T02:44:43.460 INFO:teuthology.orchestra.run.smithi007.stderr:user 0m5.264s 2024-01-27T02:44:43.461 INFO:teuthology.orchestra.run.smithi007.stderr:sys 0m0.618s 2024-01-27T02:44:43.461 INFO:tasks.ceph:Archiving logs... 2024-01-27T02:44:43.461 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533593/remote/smithi007/log 2024-01-27T02:44:43.462 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-27T02:44:44.668 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533593/remote/smithi188/log 2024-01-27T02:44:44.669 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-27T02:44:45.315 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-01-27T02:44:45.324 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-01-27T02:44:45.324 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-27T02:44:45.327 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-27T02:44:45.415 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-01-27T02:44:45.424 INFO:teuthology.task.clock:Checking final clock skew... 2024-01-27T02:44:45.424 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-27T02:44:45.427 DEBUG:teuthology.orchestra.run.smithi188:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-27T02:44:45.445 INFO:teuthology.orchestra.run.smithi188.stderr:bash: ntpq: command not found 2024-01-27T02:44:45.445 INFO:teuthology.orchestra.run.smithi007.stderr:bash: ntpq: command not found 2024-01-27T02:44:45.450 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-27T02:44:45.450 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-01-27T02:44:45.450 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 41 +271us[ +271us] +/- 89ms 2024-01-27T02:44:45.450 INFO:teuthology.orchestra.run.smithi188.stdout:^* hv02.front.sepia.ceph.com 3 6 377 41 +989us[ +993us] +/- 62ms 2024-01-27T02:44:45.450 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 -1231us[-1228us] +/- 66ms 2024-01-27T02:44:45.450 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-01-27T02:44:45.452 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-27T02:44:45.452 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-01-27T02:44:45.452 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 42 +228us[ +228us] +/- 89ms 2024-01-27T02:44:45.452 INFO:teuthology.orchestra.run.smithi007.stdout:^* hv02.front.sepia.ceph.com 3 6 377 44 +929us[ +940us] +/- 62ms 2024-01-27T02:44:45.452 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 -1278us[-1278us] +/- 66ms 2024-01-27T02:44:45.453 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-01-27T02:44:45.453 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-01-27T02:44:45.462 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-01-27T02:44:45.462 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-01-27T02:44:45.472 DEBUG:teuthology.orchestra.run.smithi007:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-27T02:44:45.592 DEBUG:teuthology.orchestra.run.smithi188:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-27T02:44:45.666 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-27T02:44:45.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-27T02:44:45.705 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-27T02:44:45.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-27T02:44:45.737 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-01-27T02:44:45.748 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-01-27T02:44:45.757 INFO:teuthology.task.internal:Duration was 1519.200075 seconds 2024-01-27T02:44:45.757 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-01-27T02:44:45.771 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-01-27T02:44:45.771 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-27T02:44:45.774 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-27T02:44:45.825 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-27T02:44:45.831 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-27T02:44:46.049 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-01-27T02:44:46.049 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2024-01-27T02:44:46.049 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-27T02:44:46.078 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi188.front.sepia.ceph.com 2024-01-27T02:44:46.079 DEBUG:teuthology.orchestra.run.smithi188:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-27T02:44:46.101 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-01-27T02:44:46.101 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-27T02:44:46.121 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-27T02:44:46.185 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-01-27T02:44:46.185 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-27T02:44:46.188 DEBUG:teuthology.orchestra.run.smithi188:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-27T02:44:46.294 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-01-27T02:44:46.304 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-01-27T02:44:46.305 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-27T02:44:46.335 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-27T02:44:46.364 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-01-27T02:44:46.377 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-27T02:44:46.380 DEBUG:teuthology.orchestra.run.smithi188:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-27T02:44:46.410 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2024-01-27T02:44:46.430 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = core 2024-01-27T02:44:46.444 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-27T02:44:46.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-27T02:44:46.491 DEBUG:teuthology.orchestra.run.smithi188:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-27T02:44:46.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-27T02:44:46.508 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-01-27T02:44:46.518 INFO:teuthology.task.internal:Transferring archived files... 2024-01-27T02:44:46.519 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533593/remote/smithi007 2024-01-27T02:44:46.519 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-27T02:44:46.585 DEBUG:teuthology.misc:Transferring archived files from smithi188:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533593/remote/smithi188 2024-01-27T02:44:46.586 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-27T02:44:46.624 INFO:teuthology.task.internal:Removing archive directory... 2024-01-27T02:44:46.625 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-27T02:44:46.627 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-27T02:44:46.677 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-01-27T02:44:46.689 INFO:teuthology.task.internal:Not uploading archives. 2024-01-27T02:44:46.689 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-01-27T02:44:46.704 INFO:teuthology.task.internal:Tidying up after the test... 2024-01-27T02:44:46.704 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-27T02:44:46.706 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-27T02:44:46.725 INFO:teuthology.orchestra.run.smithi007.stdout: 262175 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 27 02:44 /home/ubuntu/cephtest 2024-01-27T02:44:46.733 INFO:teuthology.orchestra.run.smithi188.stdout: 262179 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 27 02:44 /home/ubuntu/cephtest 2024-01-27T02:44:46.735 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-01-27T02:44:46.748 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-01-27T02:44:46.903 INFO:teuthology.nuke:Checking targets against current locks 2024-01-27T02:44:46.929 DEBUG:teuthology.nuke:shortname: smithi007 2024-01-27T02:44:46.930 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-27T02:44:46.952 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533593', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-27 02:14:06.922275', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKinMYP627ZsZXwAlG5DwvLGSjUmKPgeWgJdoetEIMrsmJ9/ttaaTGA8euU9fQXMA1AiUIRCyYETVGpzaMcdBW4='} 2024-01-27T02:44:46.957 DEBUG:teuthology.nuke:shortname: smithi188 2024-01-27T02:44:46.957 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-27T02:44:46.981 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533593', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-27 02:14:06.920906', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDaXqOH8orBjhRAqKw+EP2xxh7LgX535tX97+GEXhxwOtfxlNkcktqRwp6fCNryFWaHXhZcGtAIVYedLyqQvsOM='} 2024-01-27T02:44:47.010 INFO:teuthology.orchestra.console.smithi007:Power off 2024-01-27T02:44:47.010 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-27T02:44:47.035 DEBUG:teuthology.orchestra.console.smithi007:power off output: Chassis Power Control: Down/Off 2024-01-27T02:44:47.036 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T02:44:47.051 INFO:teuthology.orchestra.console.smithi188:Power off 2024-01-27T02:44:47.051 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-27T02:44:47.061 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2024-01-27T02:44:47.074 DEBUG:teuthology.orchestra.console.smithi188:power off output: Chassis Power Control: Down/Off 2024-01-27T02:44:47.074 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T02:44:47.098 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2024-01-27T02:44:51.062 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T02:44:51.099 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T02:44:51.172 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2024-01-27T02:44:51.208 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2024-01-27T02:44:55.176 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T02:44:55.210 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T02:44:55.286 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off 2024-01-27T02:44:55.320 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2024-01-27T02:44:55.388 INFO:teuthology.orchestra.console.smithi007:Power off completed 2024-01-27T02:44:59.324 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T02:44:59.436 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off 2024-01-27T02:44:59.536 INFO:teuthology.orchestra.console.smithi188:Power off completed 2024-01-27T02:44:59.642 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} duration: 1519.200074672699 failure_reason: '"1706322714.5416682 osd.7 (osd.7) 3 : cluster [WRN] OSD bench result of 221318.978486 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.7. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd''s IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]." in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2024-01-27T02:44:59.642 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-27T02:44:59.732 INFO:teuthology.run:FAIL